Ok so I finally figured out what was causing this.
If you have zone alarm running you cannot kill the hl2.exe process. When you quit tf2 freezes. Then when you try and kill the process in task explorer nothing happens on the first attempt and subsequent attempts have Vista tell you "access denied" forcing you to reboot the machine to get rid of hte process (+ graphics left over from it). Killing the process without attempting to quit tf2 normally causes the same thing.
Just thought I'd tell people as I've been getting annoyed at having to reboot a bunch :-/
I'm pretty sure I did but I'll check again. However, the issue wasn't that TF2 wouldn't connect or play, the problem was that when zone alarm is running I cannot kill the hl2.exe process.