A frustrating lack of apps is the main issue. On WP7, the game of the week was the same game for over a month once, because no new Xbox Live games had been released.
Alongside near total lack of ability to customize your phone, and no updates whatsoever. If you buy a Windows Phone, it will almost definitely die on the same OS version it had when you received it.
It also suffers from admittedly rare, but annoying random reboots, and a number of aesthetic bugs. My Zune live tile displayed the face of an artist I had never heard of for months, and the "People Hub" displayed peoples profile pictures from years ago.
Also, if you change your Windows Live Email, you will have to wipe your device and repurchase all your apps.
Their battery life is often poor, too, which is a problem in phones without removable batteries, which is most of them.
I have never been happier to smash an old phone to pieces than I was with my Windows Phone, and it is certainly not an OS I will ever be using again. Ever.
I am not a fan of iPhones, but I would take one in a heartbeat over even the most attractively priced Windows Phone. Honestly, after having one, I wouldn't take one for free.