Profile cover photo
Profile photo
Lukas Zilka
132 followers -
anyplace, anywhere, anytime...
anyplace, anywhere, anytime...

132 followers
About
Posts

Post has attachment
Is there a good LSTM implementation in Theano or Torch7?

I'm aware of [1] but it is very application specific. I'm looking for a library-like implementation.

[1] https://github.com/wojciechz/learning_to_execute/

Post has attachment
Andrew Ng sees neuroscience as a great source of inspiration for our algorithms.

Post has attachment
GRAMMAR AS A FOREIGN LANGUAGE
aka LSTM for parsing (using sequence-to-sequence approach).
http://arxiv.org/pdf/1412.7449.pdf

Post has attachment
Learning Longer Memory in RNN
http://arxiv.org/pdf/1412.7753.pdf

It seems they are getting close to LSTM performance with much simpler structure.

Post has attachment

Post has shared content
Yet another one on err-ing deep nets.
+Jon Shlens+Christian Szegedy  and I have submitted a paper to ICLR 2015 ( https://drive.google.com/file/d/0B64011x02sIkaV96OVdxQjNnTms/view?usp=sharing ) explaining why neural networks are vulnerable to "adversarial examples". The simple explanation is that neural networks are too linear, so when their inputs are high dimensional they can add up several minuscule pieces of evidence and produce a highly confident output. We show that we can train neural networks to resist this kind of perturbation and reduce their error rate on permutation invariant MNIST.

Post has shared content

Post has attachment

Unit Tests for Stochastic Optimization

From people at DeepMind. They develop a set of isolated tests on which they test optimization algorithms.

http://arxiv.org/pdf/1312.6055v3.pdf

Post has attachment
Very nice and intuitive tutorial how to implement any kind of gradient-based learning. Instructive examples in JavaScript. And I'm now hesitating whether Theano even makes sense =)

http://karpathy.github.io/2014/09/02/what-i-learned-from-competing-against-a-convnet-on-imagenet/
Wait while more posts are being loaded