One such unintuitive fact (basically a special case of Simpson's paradox) is the following: the number of failures that a person experiences in a field can be positively correlated with how competent that person is in that field: the more failures a person generates, the more likely that person is to be competent!
Let's see how. For sake of discussion, let's say that the field is book writing, and that there are only two types of book one can produce: good books (successes) and bad books (failures). Let's also say for simplicity that authors fall into two classes: competent writers, who produce good books 3/4 of the time and bad books 1/4 of the time, and incompetent writers, who produce good books only 1/4 of the time and bad books 3/4 of the time.
Intuitively, one would then expect that an author who has written a lot of bad books is more likely to be an incompetent author than an author who has written very few bad books. But this is not necessarily the case! The problem is that there is a confounding variable, namely the number of books each author writes. This variable is partly determined by internal author traits, such as the author's persistence, but is also influenced by the external success of the author's works, and this can be enough to flip the correlation.
To make a concrete numerical illustration of this, let us consider four types of authors, who are either competent or incompetent, or persistent or not persistent.
1. A competent, not persistent author might write just 4 books (say) during his or her career, of which 3 are good and 1 is bad.
2. An incompetent, not persistent author might also write just 4 books in his or her career, of which 1 is good and 3 are bad.
3. A competent, persistent author might write as many as 40 books over his or her career, of which 30 are good and 10 are bad.
4. An incompetent, persistent author might aspire to write 40 books in his or her career, however faced with frequent negative reviews and poor sales, he or she may eventually quit after writing only 8 books, of which 2 are good and 6 are bad.
With this simple example, one sees that it is the competent author who ends up writing the most bad books; also, out of all the bad books published by the above four authors, the majority were written by competent authors rather than incompetent ones. Because of the biasing effect of discouragement of persistent, incompetent authors, the fact that an author has a large number of bad books to his or her name becomes evidence in favor of competent writing, rather than incompetent writing!
EDIT: It is important though to note that correlation does not imply causation: the above analysis does not indicate that one can become a better writer by intentionally producing bad books. (See also "Goodhart's law".)
not to be mean but please do not imply mathematicians are to be "bested". we are not interested in your 100 TeV pissing match.
- Steadfast NetworksDirector of Research, 2007 - present
This Is Why You Shouldn't Interrupt a Programmer | The Slightly Disgrunt...
Yep. One time someone came in and quietly stood by my desk until I noticed him. Literally just stood there. I ignored him for a little bit h
Question about frame metadata in assembly/linker - Google Groups
Google Groups. Question about frame metadata in assembly/linker. Keith Randall, Jun 12, 2013 10:27 AM. Posted in group: golang-dev. As part
D. J. Bernstein Mathematics and computer science Computer hardware ...
D. J. Bernstein. Mathematics and computer science. Computer hardware: chip speed, buying computers, etc. Data structures and program structu
Occupy Wall Street Crowd Greets Fox Host With Chants Of ‘Fox News Lies’
Conservative Writer Admits 'Infiltrating' 99% Movement To 'Undermine' It. NEWS FLASH. Occupy Wall Street Crowd Greets Fox Ho