Who introduced the term “deep learning” to the field of Machine Learning (ML) and Neural Networks (NNs)? Just a few days ago we had an interesting discussion about this on the connectionists mailing list http://www.cnbc.cmu.edu/connectionists

Although Ivakhnenko had working, deep learning nets in the 1960s (still used in the new millennium), and Fukushima had them in the 1970s, and backpropagation also was invented back then, nobody called this “deep learning.”

In other contexts, the term has been around for centuries, but apparently it was first introduced to the field of Machine Learning in a paper by Rina Dechter at AAAI 1986 (thanks to Brian Mingus for pointing this out). She wrote not only about “deep learning,” but also “deep first-order learning” and “second-order deep learning.” Her paper was not about NNs though: http://www.aaai.org/Papers/AAAI/1986/AAAI86-029.pdf

To my knowledge, the term was introduced to the NN field by Aizenberg & Aizenberg & Vandewalle's book (2000): "Multi-Valued and Universal Binary Neurons: Theory, Learning and Applications.” They wrote about “deep learning of the features of threshold Boolean functions, one of the most important objects considered in the theory of perceptrons …”  http://www.researchbooks.org/0792378245/MULTI-VALUED-UNIVERSAL-BINARY-NEURONS/

The Google-generated graph seems to indicate that the term’s popularity went up right after Aizenberg et al.’s book came out in 2000. However, this graph is not limited to NN-specific usage. (Thanks to Antoine Bordes and Yoshua Bengio for pointing this out.)

Although my own team has published on deep learning for a quarter-century, we adopted the terminology only in the new millennium. Our first paper with the word combination “learn deep” in the title appeared at GECCO 2005

Of course, all of this is just syntax, not semantics. The real deep learning pioneers did their work in the 1960s and 70s!

I also mentioned this somewhere deep down in the AMA at reddit: http://www.reddit.com/r/MachineLearning/comments/2xcyrl/i_am_j%C3%BCrgen_schmidhuber_ama/cpfrrnr
Shared publiclyView activity