If your vision is good enough to see the guy across from you during the fight, you had better take your contacts out in case that your contacts will be knocked out and you are likely to hurt your eyes when the fight is fierce. I think fighting is not a good way to solve problems, it can only make the situation worse. Dealing with problems in a peaceful way is best.
Do you mean you gonna to fight with others? Oh, it is really bad. Why not fight? Perhaps you can try to solve this problem by communicate with them. If you really need fight, you'd better not wear contact lenses. In fact, you can wear neither eyeglasses nor contact lenses if your eyesight is not very bad. If s/he hit in your eyes when you wear contact/ eyeglasses, it will cause badly damages to eyes. Anyway, you shall not fight if you don't want to get hurt.
I think it is bad for your eyes while fighting. Because it is to fall off or trapped in your eyes when you hitted by others. If you really want to fight, here I kindly suggest you to have a Lasik surgery.