Profile cover photo
Profile photo
Vincent Boucher
Guinness World Record™ | General Artificial Intelligence (Strong AI)
Guinness World Record™ | General Artificial Intelligence (Strong AI)
About
Posts

Post has attachment

Post has shared content
The federal government has almost $1 billion to give to high-tech clusters, and Montreal's booming artificial intelligence community said it was left out.
(picture: +Yoshua Bengio)

Post has attachment

Post has attachment

Post has attachment

Post has attachment

Post has attachment

Post has shared content
Don't mean to be a science astrologer but I want to make certain predictions about what we will see in 2017, particularly, in Machine Learning. Having seen the rapid and immense progress the field is making, I am quite tempted to make some predictions.

- GANs will continue to dazzle and I think it makes a lot more sense to consider training models where we don't have to explicitly hand-craft cost functions. I do think that we need to consider cost functions directly in higher dimensional space than low-level pixel colour values. I'm quite confident that significant progress will be made - we only saw cosmetic touches in 2016 (e.g. see some in https://github.com/soumith/ganhacks) - on improving the optimisation and many other areas where GANs still lack some grounded theory. We have come quite far from Ian Goodfellow's first paper on GANs in 2014.

- RL will become even more popular as more datasets and testbeds become available and I think we will the rise of these kind of papers https://fsadeghi.github.io/CAD2RL/ (from Fereshteh Sadeghi) where no fine-tuning is needed to do any testing on real world. Robotics is going to benefit so much from this.

- We will see more work on transfer learning (and my hope is that we see more in real world than just ATARI games) and ideas on "train-a-network-that-does-multi-tasking" will become more popular. UberNet http://cvn.ecp.fr/ubernet/ was one of its kind but we will start seeing networks that are able to do well on more concrete problems.

- New hardware (and I'm betting on Intel particularly after their acquisition of Movidius and Nervana) will accelerate our progress in deploying deep learning for commercial applications. Specific chips tailored towards deep learning will become more and more popular. I hope we also see differentiable hardware in the loop.

- MetaLearning will become more fashionable as I think it makes sense now to get the human out of the loop when designing architectures and optimizers (see learning to learn https://arxiv.org/abs/1606.04474 and in part HyperLSTM https://arxiv.org/abs/1609.09106 and Neural Architecture Search with Reinforcement Learning https://arxiv.org/abs/1611.01578).

- I think we will see the rise of gated convolutions (e.g. Highway networks, WaveNet, and recently https://arxiv.org/pdf/1612.08083.pdf) than just blindly applying LSTMs and consequently fewer people will get schmidhubered.

- 3D environments will reconcile the so-far-still-very-isolated communities in SLAM and deep learning to help our understanding of scenes in general. I also make this bold prediction that we will see NPIs for 3D (https://arxiv.org/abs/1511.06279) solving Tower of Hanoi kind of problems (with physics engine).

- Backpropagation will continue to be challenged and I think we may find a way to train networks that do resonably welly on CIFAR etc. datasets without backpropagation.

- Evolutionary algorithms will also see some (if not a lot) revival.

- Semi-supervised learning will be even more popular (I don't believe pure unsupervised learning is possible yet and we will still need supervised data anyway!).

- and of course, Pieter and Sergey will get a joint Order of the Reinforcement Learning Empire from Rich Sutton and Barto.

Hope most of you go out of your comfort zone and learn something new to contribute to the advancement of the community. Being creative is the most important and satisfying thing (no best paper award or 1000+ citations can match that!) and I hope more and more people come up with out-of-the-box ideas and think seriously about its value beyond the community and see if they can bring about a change (even if tiny) to the world through their work.

Post has shared content

Post has shared content
Wait while more posts are being loaded