I had a weird issue where I could connect just fine to any VM on any of my many HyperV enabled servers… Except one. I just installed a new Windows Server 2008 R2 SP1 in a HyperV VM and the RDP connection was horrible slow. No network traffic, no CPU… Eventually I found this article and in specific, NicMic's comment worked for me.

You have to disable IPv4 Checksum Offload in the Virtual Machine's NIC adapter.