There are four steps for you to help your contact lenses get rid of smudges. First, pour solution which is just used to clean contact on your contacts. You can use a good brand of solution and make a trial to make sure that the solution is suitable for your contacts. Second, your case of lens may are full of smudges, then you should dump the solution of your lens case and rinse it under water. Third, after washing, you should put your contacts right in the correct side of the case. Four, you should wash your contact regularly to make sure that your contact lenses are clean.
So, I don't know what it is precisely, but here my advice for you is to soak the contact lenses into your solution case then try to flush your contacts with clean water. If the smudges do not go away, then you probably should throw them away because you cannot expect to take them off by hand right, that would lead to scratched contact lenses.