Profile cover photo
Profile photo
Nickolay Shmyrev
Nickolay's posts

Post has shared content
The LSH part is very interesting
Yesterday, we announced the launch of Android Wear 2.0, along with brand new wearable devices, that will run Google's first entirely “on-device” ML technology for powering smart messaging.

This on-device ML system enables technologies like Smart Reply to be used for any application, including third-party messaging apps, without ever having to connect with the cloud…so now you can respond to incoming chat messages directly from your watch, with a tap. Learn more, below.

Post has attachment

Post has attachment
CMUSphinx-powered app for League of Legends

Post has attachment

Post has attachment
This is a big technical problem to solve, pretty interesting one

and, i-vectors do not really work for short utterances.

Post has attachment

Learning with huge memory

Recently a set of papers were published about "memorization" in neural networks. For example:

Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer


Understanding deep learning requires rethinking generalization

It seems that large memory system has a point, you don’t need millions of computing cores in CPU and, it is too power-expensive, you could just go ahead with very large memory and reasonable amount of cores to access memory with hashing (think of Shazam or randlm, or G2P by analogy). You probably do not need heavy tying either.

Advantages are: you can quickly incorporate new knowledge, just put new values in memory, you can model corner cases since they are all still accessible, and, again, you are much more energy-efficient.

Maybe we will see mobile phones with 1Tb of memory sometimes.

Not quite a scientific paper, but "memorization" is a very promising concept.

Chiyuan Zhang et al

Post has attachment

Post has attachment
Wait while more posts are being loaded