Profile cover photo
Profile photo
Russ Salakhutdinov
1,094 followers
1,094 followers
About
Posts

Post has attachment
Very excited about taking on a new role as a director of AI research at Apple in addition to my work at CMU as an associate professor in the Machine Learning Department. Lots of truly amazing AI and Machine Learning projects are taking place all over Apple and I am thrilled to be part of it.

We will be building a team of top-notch Machine Learning and Deep Learning researchers, working on challenging R&D projects as well as conducting fundamental research to advance the state-of-the-art in AI! We are looking for both full-time research scientists as well as postdoctoral fellows to join my team. Apply or ping me if you are interested.

https://jobs.apple.com/us/search?pr=1#mixes&ss=52662972&t=0&so=&lo=0*USA&pN=0&openJobId=52662972

Post has attachment
Heading to the Deep Learning Summer School in Montreal.
Will give a focused tutorial on Learning Deep Generative Models,
covering mathematical basics of RBMs, Deep Boltzmann Machines,
Helmholtz Machines, Variational and Importance Weighted Autoencoders (i-ways).
Slides should be up online very soon.

https://sites.google.com/site/deeplearningsummerschool2016/home

Post has attachment
Gated-Attention Readers for Text Comprehension
http://arxiv.org/abs/1606.01549
with Bhuwan Dhingra, Hanxiao Liu, William W. Cohen

The paper looks at the problem of answering cloze-style questions over short documents. Key point: use multiplicative interactions between the query embedding and intermediate states of a recurrent neural network reader.

Post has attachment
Machine Learning Faculty Retreat at CMU -- about 25 machine learning faculty and we are growing!

If you want to do machine learning, this is the place -- with over 70 ML PhD students, about 10 postdocs, and over 20 MSc in our own Machine Learning Department.
Photo

Post has attachment
New paper on Multiplicative Integration with Recurrent Neural Networks with Tony Wu, Saizheng Zhang, Ying Zhang, Yoshua Bengio.
http://arxiv.org/abs/1606.06630

The paper introduces a simple structural design called Multiplicative Integration (MI) to improve recurrent neural networks (RNNs). MI changes the way in which information from difference sources flows and is integrated in the computational building block of an RNN, while introducing almost no extra parameters. It can be easily embedded into many popular RNN models, including LSTMs and GRUs.

Post has attachment
Encode, Review, and Decode: Reviewer Module for Caption Generation
Zhilin Yang, Ye Yuan, Yuexin Wu, Ruslan Salakhutdinov, William Cohen
http://arxiv.org/abs/1605.07912

This work develops a generic reviewer module that can be plugged into an existing encoder-decoder model. The reviewer module performs a number of review steps with attention mechanism on the encoder hidden states, and outputs a fact vector after each review step; the fact vectors are then used as the input of the attention mechanism in the decoder.

Post has attachment
New paper on Path-Normalized Optimization of Recurrent Neural Networks with ReLU Activations.

http://arxiv.org/abs/1605.07154
with Behnam Neyshabur, Tony Wu, and Nathan Srebro

Here, we investigate the parameter-space geometry of recurrent neural networks, and develop an adaptation of path-SGD optimization method, attuned to this geometry, that can learn plain RNNs with ReLU activations.

Congratulations to my two amazing postdocs +Roger Grosse  and +Yura Burda  -- Yura is joining OpenAI as a research scientist and Roger is joining Department of Computer Science at the University of Toronto as an assistant professor!

Post has attachment
If you are going to ICLR, check out these posters/papers, and more importantly, check out (some of the) associated code -- I am trying to encourage all my students to release their code!


Importance Weighted Autoencoders, Yuri Burda, Roger Grosse, Ruslan Salakhutdinov, ICLR 2016
http://arxiv.org/abs/1509.00519
Code: https://github.com/yburda/iwae

Generating Images from Captions with Attention, Elman Mansimov, Emilio Parisotto, Jimmy Lei Ba, Ruslan Salakhutdinov, ICLR 2016
http://arxiv.org/abs/1511.02793
Code: https://github.com/emansim/text2image

Action Recognition using Visual Attention, Shikhar Sharma, Ryan Kiros, Ruslan Salakhutdinov, ICLR workshop 2016
http://arxiv.org/abs/1511.04119
Code: https://github.com/kracwarlock/action-recognition-visual-attention

Actor-Mimic: Deep Multitask and Transfer Reinforcement Learning, Emilio Parisotto, Jimmy Lei Ba, Ruslan Salakhutdinov, ICLR 2016
http://arxiv.org/abs/1511.06342
Code: coming soon.

Data-Dependent Path Normalization in Neural Networks, Behnam Neyshabur, Ryota Tomioka, Ruslan Salakhutdinov, Nathan Srebro, ICLR 2016
http://arxiv.org/abs/1511.06747

Post has attachment
Check out code by Elman Mansimov for our paper on Generating Images from Captions with Attention. If you want to learn a model that knows how to "draw" images of "elephants flying in blue skies", here you go:)

https://github.com/emansim/text2image
http://arxiv.org/abs/1511.02793
Wait while more posts are being loaded