The hardest thing I found, when starting out, was finding a good, qualified teacher.
There are a lot of what I call backyard Sifus around and, of course, it doesn't take much to fool the uninitiated. I think I must have moved from one to another, over a series of up to a dozen of them before I found somebody that was capable of offering anything of substance.
They waste a lot learner's time and money, in teaching a lot of bad movement habits (if the material has any application at all), and when the learner finally stumbles on the reality of the situation, that alone can hit them hard enough to give up.
I'm watching a couple of them on-line, on a daily basis now, and playing with the idea of writing something up that will expose them to their audiences for what they are. Cheap, scamming, self-appointed alternative life-style earners, leaching off others naivety.