Alright, forget all this "your video card sucks" mumbo jumbo. It's not correct. I have a 256mb 6600GT and have the exact same problem you do - on some servers. If you have 1.03 (which I suspect might be one of the problems), there's really no way to fix it other than joining another server; for example, yesterday there was a server that I had about a 100 ping on, that played fine at the start, then started lagging like hell. I'm a level designer, so I know exactly the difference between FPS lag and latency. I disconnected, found a server with one slot open, joined it, and played for a straight 4 hours with no lag whatsoever. While there isn't very many servers open for 1.03, you'll find one eventually. If you want to play a game, add me at [email]email@example.com[/email] and we'll join an empty server or something.
Hope this helps.