You can not wear the contacts lenses when you want to go swimming. We all know that contact lenses is the ideal glasses for sports and other outdoor activities, because they can provide better peripheral vision, and will not collect moisture such as snow, rain, condensation, or sweat and so on. But some chemical elements in the swimming pool can make chemical reaction with the contact lenses.
I think no one can always keep his head above water surface all the time. Though you can do that, you can not avoid wetting your contacts lenses. I think you can swim without wearing the glasses if you are not the extremely short-sightedness. Or you can borrow a pair of swimming goggles.
Wearing swim goggles is a safer way for you to see clearly both above the surface and underwater. If water gets in your eyes when swimming, you should remove, clean and disinfect your contact lenses as soon as possible to reduce your risk of eye irritation and infection