It is possible but risky.Water, even chlorinated pool water, contains a number of agents that can cause very serious corneal infection. Swimming around pools and ocean waters on the other hand, decreases your susceptibility to microbial infection, but other problems may evolve. In the pool--eye irritation is possible when chlorine sticks on the surface of your lens; in the ocean--your contact lenses can be dislodge when you encounter large waves. You can still wear your lenses, but do
so but with proper precautions.
I will not recommend you to go swimming with contacts in because there is a lot of bactera in the water which will make your eyes infected. If you do need to wear contacts in while swimming, you can wear swimming goggles over the contacts, which can prevent the water getting into the contat lenses and eyes.