I won’t spend too much time on the first part of the sentence, because presumably you’re familiar with the ways in which free markets can work well. Basically, if two people freely engage in a trade, by definition it makes both parties happier; if it didn’t, they wouldn’t have engaged in that trade in the first place. The more trade is possible, the more people can accumulate things that make them happy – e.g., more free time to spend with their families while getting the same amount of food. There are all sorts of mechanisms which amplify that; probably the most important is “comparative advantage,” which shows how if there are enough people around so that everyone can specialize in what they’re best at, everyone comes out ahead. (See http://en.wikipedia.org/wiki/Comparative_advantage if you want to learn more about it; Ricardo’s example is a good place to start)
However, if you’ve paid any attention at all to the world in the past few centuries, you’ve probably already got an objection to what I just said: “You think they wouldn’t have engaged in that trade if it didn’t make them happier? What if they didn’t have a choice?” And you would be perfectly right, and that part is the heart of “except when they don’t.” There are a few ways in which free markets can fail, but most of them are obscure and relatively easy to fix (e.g., limited money supply). The big one, which represents the overwhelming majority of all the problems, is coercion: the argument that a free trade makes both parties happier doesn't apply to trades which are not free.
(And note, by the way, that coercion harms free markets both locally – in that the individuals being coerced lose out, often by a lot – and globally, in that coercion is essentially a “tax” levied by one person on another. Those of you familiar with economics will recognize that whenever the price of something is pushed away from its natural equilibrium, overall value goes down. So when I talk about failures of capitalism, I mean failures that are simultaneously moral and financial failures.)
As far as I can tell, the large majority of coercion comes in three forms, which I’ll call externalization, capture, and inelasticity. These three overlap and most real situations combine them, but what they have in common is that they’re ways in which one person can force someone else to accept a trade that they would normally never take.
Externalization, also known as “negative externalities,” is when I can make everyone else pay my costs. The most common forms which don't overlap heavily with capture (the next item) are when I can take my costs from a poorly-policed commons (e.g., dump waste into an air supply that nobody is monitoring), or when I can spread the costs out so widely that every individual's cost of recovery would exceed the damages to them. (Playing games with individuals less able to individually recover is part of capture and inelasticity)
Capture is the classic positive-feedback problem of any system of human power: you can use your power to get more of it. Pure capture (as opposed to inelasticity) happens when there are central methods of regulation or other power-amplifying systems which you bring under your control via bribery, coercion, etc., and suddenly rather than regulating you they amplify you. That can be anything from regulation of pollution of commons to having the police keep Those People "under control" for you.
Inelasticity is the result of the fact that the value of money isn't linear. If you have $100,000, then $1 means a lot less to you than if you have $10,000. In particular, people's needs up to a certain wealth level are dominated by constant overheads such as food and housing; so long as your total resource access is of the same scale as those overheads, your financial life is dominated by the severe consequences of falling below any one threshold. Once you're far from those overheads, the value of resources becomes far more linear. Often, a good way to measure this is in terms of "being one <event> away from disaster;" if the event is as small as a flat tire, versus if it's as big as inoperable cancer, it's a huge difference in life.
If two people are about to engage in a deal, and one of them is near-threshold while the other isn't, the one who isn't has huge negotiating leverage. For example, you can give someone a shitty job like mining coal that they would never accept if they had some option other than starvation.
(Economists will note that in “inelasticity,” I’ve abused a term which actually represents something very natural, which is that the price of some goods isn’t very sensitive to supply or demand. The problem I’m describing here is really the problem of extreme inelasticity: specifically, when a person’s net resources are low enough that the costs of white-pill [“you need this in order to live,” nearly infinitely inelastic] goods become a dominant part of their financial calculus. Better terms for any of these would be welcome.)
There are methods to deal with all of these problems, and they work to varying extents.
The way you solve the first kind of externality is to not have poorly-policed commons. That doesn't necessarily mean enclosure, i.e. giving ownership of those commons to private individuals: in fact, enclosure is extremely susceptible to capture effects, as we saw with the original enclosures in 18th-century England. Methods like cap-and-trade, or simply either (a) requiring people to pay their own damned cleanup costs or (b) being up-front about the fact that we’ve decided to pay those costs as a community in exchange for the net benefits of whatever it is those people do, can be quite effective.
The second kind of externality, where people spread costs around widely – e.g., stealing $5 from every household in a city – can be solved by mechanisms like class-action suits, central regulators and law enforcement. In each case, the idea is to create some kind of entity (be it an ad-hoc collective of individuals or a dedicated set of professionals) which is strong enough, and incented properly, to pursue recovery whenever someone tries to steal from the community.
There’s a hybrid kind of externality which also uses inelasticity: if you steal from the poor, they’re less likely to have the resources to go after you. This is based on the fact that recovery of any sort tends to have fixed overhead costs, e.g. the time, money, and knowledge required to sue someone. When people’s total spare budget of any of these is tiny because of their basic cost of survival – e.g., when someone is working hourly and can’t afford the time to engage in a lawsuit – it becomes a lot easier to steal with impunity. This is just a nastier variation of the second kind of externality, and is often best solved with a swift kick to a tender area of the anatomy.
You can solve inelasticity by reducing the overhead costs to zero. There are many different overhead costs -- food, shelter, transport, child care, etc., etc. -- and each can be analyzed separately and each creates value in its reduction. One way to reduce those is to literally reduce the cost of the items; e.g., the real cost of clothing has plummeted over the past few centuries, and now lacking clothing is only a major factor for people near the absolute bottom of the resource curve, where clothing is really being used as a substitute for shelter. Another way is to reduce need for the items, e.g. by having housing close enough to work and so on that people don't need lots of transport. A third way is to socialize the costs of these items, e.g. by having functional public transit or health care. That doesn't reduce their intrinsic costs, but it eliminates the inelasticity effect on individuals by averaging their cost over the entire population.
Capture is harder to solve. The best solution proposed so far has been democracy, but as we're well aware, that's more like "the worst solution except for all the other ones that have been proposed." There are some fields in which it can be solved by things like direct competition among regulators, but that has strange failure modes: consider, e.g., the competition among bond rating agencies which was supposed to ensure that bond ratings were meaningful. Unfortunately, all three of them were funded by bond sellers, not buyers, and so had the same incentives; a lot of the mortgage crisis was a consequence of that. (Specifically, that the various CDO's being traded had been AAA-rated based on completely nonsensical models, which any rater that had an incentive to actually think through would have known)
Importantly, though, I think that the first part of that original sentence about free trade is at least as important as the second: capitalism works well when coercion is brought under control. Communism was a recipe for making everybody miserable – and ultimately proved even more vulnerable to capture than capitalism. Feudalism and so on are even worse. Even more significantly, capitalism works better by its own metrics when coercion is eliminated: in fact, economists tend to model these things as a kind of tax on the system.
However, benefit to the system as a whole doesn't translate to benefit as individuals. In particular, if you're making a lot of money off (say) inelasticity, by having a large pool of desperate and subservient workers, then even though the aggregate wealth of society would go up a lot if they weren't so desperate, your own wealth (especially your effective wealth, divided by the cost of converting it to goods) would go down. That creates one hell of an incentive for capture, in forms ranging from direct regulatory capture by bribes, to funding astroturf organizations, to creating entire media and social panics, all the way out to creating social institutions – e.g., feudalism or slavery.
So while solving externalization and inelasticity are extremely important, capture is often the key to the whole deal.
Fortunately, it isn't always the key to the deal: sometimes, for example, someone figures out a way to knock the price of some key good down through the floor so quickly that existing groups can't figure out how to capture it in time. The expansion of Europeans through North America was an interesting example of that case (where the price of land became cheap because anyone could just kill people and take it -- or, once the cost of killing people got socialized into an army, just take it); their expansion through South America much less so, because there expansion was effectively captured as an industry by entrenched nobles. The rise of new technologies is a somewhat less bloodthirsty example: e.g., when companies like FedEx cratered the price of small-scale many-to-many shipping logistics, thousands of new businesses suddenly emerged.
Technology, wisely applied, can knock the pillars of inelasticity out from under people. That creates a sudden chance to rapidly equalize wealth distributions, which in turn has a strong negative effect on capture. The question of whether you can maintain that negative effect after the initial pulse, or whether wealth will inevitably end up re-aggregating through a combination of capture and externalization even in a zero-inelasticity world, is one of the major hard questions we're likely to face in the next few decades.
But it’s one of my major interests, and it’s why I spend my time working on technologies which I think can alter the basic cost calculus of the world, by making something previously necessary and expensive (be it knowledge, or communication, or data storage) nearly free.
Illustrative cartoon by Ape Lad (http://goo.gl/QI13CH), CC-NC-ND. Thanks as well to for the initial conversation which sparked this post, and for many other such arguments about the nature of capitalism.