we'll just have to make Anti-Drone drones to intercept unauthorized drones. The sky above the city of the future will be like a tiny robotic battle of Brittan. Pedestrians will either invest in Kevlar umbrellas, or buy personal defense drones to catch the falling debris. Eventualy drones will get so tiny that we won't even notice them, and when they fight, it will be like the toner wars in The Diamond Age.
On a more serious note, people already own guns and cars, both of which are probably more dangerous than the drone they were flying. Seeing as how they recovered it by flying it into somebody and catching it. I could only imagine it hurting somebody if they were standing on a ledge and it made them lose balance. If it hit a larger aircraft that might be a problem, but it never went high enough to conflict with anything else.
Like guns and cars, the government will probably regulate them by making you have a license and registration. And like motor-scooters and bb-guns, small enough drones will be unregulated below specific altitudes.