Thoughts on Geoff Hinton's Coursera class on Neural Nets
I finally had a chance to give the course a brief look! There are only 2 lectures available so far but there is already quite a bit of discussion going on in the forums about Neural Nets and related.

One concerning thing I've noticed is that there seem to be many people who don't know much about Machine Learning, but they are somehow extremely eager about Neural Nets (I'm guessing Multilayer Perceptrons specifically), they completely buy that this is how the brain works, and reading through some of the posts they express intentions of using them as a black-box hammer in all kinds of problems that, to me, look like a screw.

Taking binary classification as example task to make this more concrete, Layered perceptrons are a tricky, specific architecture that shines when you have a lot of data in tighter spaces (relatively speaking), and also if your data is not simply linearly separable but contains richer, folded structure. At least for myself, I always think of NNs as just one of the tools in my toolbox (especially when it is convenient to have a test-time efficient, space efficient, parametric model that can also be trained online), but in all honesty and at least in my experience SVMs almost always perform better and are faster and less painful to train, as there is much less sprinkle dust and expertise required. And yet, I don't suppose we'll be seeing an SVMs or Random Forests courses anytime soon, and even if there was one there would be next to 0 interest because that's not how "the brain works". Somehow NN's get away with this even though we know perfectly well just how strong of an approximation a perceptron is to a biological neuron.

I'm wondering how much attention the course would get if it was renamed to "Optimization hacks for Stacked Logistic Regressions training and related family of learning algorithms".

So... I love Neural Nets, but I worry that they are oversold and overhyped to laymen, without context of all the other models and without intuitions as to when they are appropriate to apply. Now let's see what happens.
Shared publiclyView activity