Profile cover photo
Profile photo
Kai Arulkumaran
Researcher, programmer, DJ, transhumanist.
Researcher, programmer, DJ, transhumanist.

Kai's posts

Post has attachment
Kaixhin/dockerfiles is moving to NVIDIA Docker. If you're using one of my Docker images for deep learning (mentioned in the docs of several frameworks), be aware that from now on you'll need to use `nvidia-docker` for CUDA support. This means that you no longer need a precise driver version to use these images.

Post has attachment
It seems like +Nando de Freitas has taken off a day to do an AMA over at r/MachineLearning, and still appears to be answering questions!

I would say that it's one of the more informative AMAs on ML, as it seems to have attracted a smaller crowd asking more questions on more academic specifics rather than "hype" topics (although there are of course several of those). On top of that, he (IMO) has a balanced set of views. Of course the gist of it is pro-deep learning, but he's worked on many (many) areas over the years and does give a reasoned set of advantages and disadvantages to various fields within his knowledge (e.g. Bayesian inference).

And to top it off, not only does he make some deep remarks (no pun intended) on AI, he also champions underrepresented groups in the field. I concur - as researchers it's not enough to not be discriminatory (that should be a given), but we should actively participate in engaging those who would not otherwise receive the same level of encouragement as those who typically enter our fields in academia.

Post has attachment
Started work on implementing some of the latest results in deep reinforcement learning with DQNs. Still a work in progress, but potentially able to beat state-of-the-art published results (as it combines two ICLR 2016 submissions)!

Post has attachment
Just pushed out `nninit` - a weight initialisation toolbox for +Torch. For a lot of neural networks being used right now, initialisation isn't a priority. However, for very deep networks - and therefore recurrent neural networks - proper initialisation can be key.

This currently supports linear and convolutional layers, with a few simple and more complex weight initialisation schemes. Ideally this would be more closely integrated with `nn`, but I think it's worth getting out there first to settle on the API. Please give me your feedback, or even better, contribute!

Post has attachment
I am chairing a BMVA technical meeting on "Deep Learning for Computer Vision" in London, UK, on Friday 8th July, 2016. Confirmed keynote speakers include +Andrea Vedaldi, Ben Graham, Nicolas Koumchatzky, +samim dot IO, +Sander Dieleman and +Soumith Chintala.

We are currently looking for poster submissions; the full programme will become available in May. More details about the scope of the meeting, registration and submitting entries is available on the Eventbrite page.

Post has attachment
So Google finally decided to release TensorFlow - it's been a poorly kept secret for years, so researchers outside of Google have been anticipating an eventual release.

The spec for TensorFlow was to make something that would move Geoff Hinton over from MATLAB, so naturally it would have a nice API. Given that it was developed by the Google Brain team, it also needed to scale across multiple machines of different types. Not an easy task to combine the two!

The first line of the description - "data flow graphs" - establishes what I held to be the fundamental paradigm to implementing this successfully. Dataflow programming is sadly not a common programming paradigm, but it is a natural way to think about neural networks (both artificial and biological).

We get the experience of practical ML researchers and engineers in this library, as well as the lessons learnt from previous NN frameworks. C++ and Python will keep most people happy. Although I've grown fond of Lua thanks to Torch7, I'm tempted by the promise of JS in the future... And finally there's first class support for RNNs baked in from the start.

The major disadvantage is the lack of an established community (naturally). How will it compare with the vast amount of packages available for Torch, or the many libraries built on top of Theano? Only time will tell.

Post has shared content
Nice AMA going on at the moment with deep learning researchers from Google. I met +Greg Corrado briefly at the Deep Learning Summit in San Francisco after his talk on DistBelief, so perhaps it's fitting that I asked about how some of their papers rely on lots of resources. The response was that it sure helps, but at least Google try to get outside researchers involved, which is probably the best response that you could get.

For anyone else who keeps up to date with deep learning, so far there's nothing new, but the answers are still worth going over as a form of consolidation :)
Google Researchers +Greg Corrado and +Christopher Olah  are doing an AMA on Reddit  about #machinelearning and #deepneuralnets. Ask your questions at 
Animated Photo

Post has attachment
There's not much new in this video about DeepMind's research and goals (apart from a brief mention of 3D games, simulators and robotics), but this video is worth watching if you haven't already heard Demis' direction for solving AGI.

Post has attachment
Very nice summary and analysis by +Ferenc Huszár on an upcoming NIPS paper from the NYU lab on unsupervised learning with video data. This work further explores temporal coherence/the slowness principle - thought to be a promising prior to utilise in deep learning. Ferenc brings in a comparison with Slow Feature Analysis, which would be interesting to explore further.

Post has attachment
Another great blog post from +Christopher Olah, although more speculative than usual. As the term "computational graph" has become more widely used in deep learning, I think the field is in a prime position for new ideas on how to interpret neural networks. Functional programming is certainly an elegant interpretation. In the back of my mind I had some sort of intuition that an analogy could be made, but putting actual functional names to operations was great.

If I were to speculate on the nature of neural networks, I would like to see them explored via flow-based programming; this programming paradigm also fits nicely with biological neural networks. One thing missing from a lot of deep learning models is adaptability - a set structure and number of parameters are trained, and then the model is used for inference. Even with pretraining, computational graphs tend to be mainly static - not the case within a message-passing system.
Wait while more posts are being loaded