Profile

Cover photo
Roman Shapovalov
Works at MSU Graphics & Media Lab
Attends Lomonosov Moscow State University
Lives in Moscow, Russia
181 followers|72,958 views
AboutPostsPhotosVideos+1'sReviews

Stream

Roman Shapovalov

Shared publicly  - 
 
Повторник считал, что стать посредником почетно, но потом пошел на попятную.

 ·  Translate
1
Add a comment...

Roman Shapovalov

Shared publicly  - 
 
Если тебя поцелуют в левую щёку, подставь правую. А потом — снова левую.

 ·  Translate
1
Add a comment...

Roman Shapovalov

Shared publicly  - 
1
Roman Shapovalov's profile photoArsen Kostenko's profile photo
3 comments
 
That's so cool! Hope you liked it :)
Add a comment...

Roman Shapovalov

Shared publicly  - 
 
Я деньги не печатаю! Ну, не считая биткойнов.

 ·  Translate
1
Add a comment...
Have him in circles
181 people
Alexey Popow's profile photo
Дмитрий Масленников's profile photo
Artem Frolov's profile photo
Sergey Loshkaryov's profile photo
Дмитрий Ковалев's profile photo
Anton Osokin's profile photo
Jennifer Nguyen's profile photo
Satrio Damardjati's profile photo
Алексей Штукатуров's profile photo

Roman Shapovalov

Shared publicly  - 
 
 
Deep Learning and Graphical Models

I sometimes get questions like "how does deep learning compare with graphical models?". There is no answer to this question because deep learning and graphical models are orthogonal concepts that can be (and have been) combined.

Let me state this very clearly: there is no opposition between the two paradigms. They can be advantageously combined.

Of course, deep Boltzmann Machines are a form of probabilistic factor graph themselves. But there are other ways in which the concepts can be combined.

For example, you could imagine a factor graph in which the factors themselves contain a deep neural net. A good example would be a dynamical factor graph in which the state vector at time t, Z(t) is predicted from the states and inputs at previous times through a deep neural net (perhaps a temporal convolutional net). A simple instance is when the log factor is equal to ||Z(t) - G(Z(t-1), X(t))||^2, where G is a deep neural net.
This simply says that the conditional distribution of Z(t) given Z(t-1) and X(t) is a Gaussian of mean G(Z(t-1), X(t)) and covariance unity.

This type of dynamic factor graph can be used to model multi-dimensional time series. When a sequence X(t) is observed, one can infer the most likely sequence of hidden states Z(t) by minimizing the sum of the log factors (which we can call an energy function).

Once the optimal Z(t) is found, one can update the parameters of the network G() to make the energy smaller.

A more sophisticated version of this could be used to learn the covariance of the Gaussians, or to marginalize over the Z(t) sequence instead of just doing MAP inference (only taking into account the sequence with the lowest energy).

An example of such "factor graph with deep factors" was described in 2009 ECML paper with my former student +Piotr Mirowski (who is now at Bell Labs) "Factor Graphs for Time Series Modeling"
(Piotr Mirowski & Yann LeCun, ECML 2009): http://yann.lecun.com/exdb/publis/pdf/mirowski-ecml-09.pdf

A similar model used auto-encoder-type unsupervised pre-training to do language modeling "Dynamic Auto-Encoders for Semantic Indexing" (Piotr Mirowski & Yann LeCun, NIPS Workshop on Deep Learning, 2010): 
http://yann.lecun.com/exdb/publis/pdf/mirowski-nipsdl-10.pdf

Another way to combine deep learning with graphical models is through structured prediction. To some, this may sound like a new idea, but the history of this goes back to the early 90's. +Leon Bottou  and Xavier Driancourt used a sequence alignment on top of a temporal convolutional net to do spoken work recognition. They trained the convnet and the elastic word models simultaneously, at the word level, by back-propagating gradients through the time alignment module (which you can see as a kind of factor graph in which the time warping function is a latent variable).

In the early 90's Leon, +Yoshua Bengio and +Patrick Haffner built "hybrid" speech recognition systems in which a temporal convolutional net and an HMM were trained simultaneously using a discriminative criterion at the word (or sentence) level.

A few years later, Leon, Yoshua, Patrick and I used similar ideas to train our handwriting recognition system. Instead of a normalized HMM, we used a kind of energy-based factor graph without normalization. The normalization is superfluous (even hurtful) when the training is discriminative. We called this "Graph Transformer Networks". This was first published at CVPR 1997 and ICASSP 1997, but the best explanation of it is in our 1998 Proc, IEEE paper: http://yann.lecun.com/exdb/publis/pdf/lecun-98.pdf

Some of the history of this with detailed bibliography is available in the paper "A Tutorial on Energy-Based Learning": http://yann.lecun.com/exdb/publis/pdf/lecun-06.pdf (starting around Section 6).
1
Add a comment...

Roman Shapovalov

Shared publicly  - 
 
Долой кровавый режим приготовления стейков!

 ·  Translate
1
Add a comment...

Roman Shapovalov

Shared publicly  - 
 
День народного единства… и народного пьянства
Всенародно едим и пьём!
 ·  Translate
1
Add a comment...
People
Have him in circles
181 people
Alexey Popow's profile photo
Дмитрий Масленников's profile photo
Artem Frolov's profile photo
Sergey Loshkaryov's profile photo
Дмитрий Ковалев's profile photo
Anton Osokin's profile photo
Jennifer Nguyen's profile photo
Satrio Damardjati's profile photo
Алексей Штукатуров's profile photo
Education
  • Lomonosov Moscow State University
    2005 - present
Basic Information
Gender
Male
Other names
Роман Шаповалов
Roman Shapovalov's +1's are the things they like, agree with, or want to recommend.
SIGGRAPH 2014 Talk 'Reflections in Thief'
petersikachev.blogspot.com

Hi guys! I've decided to start a new dev blog where I'll write about stuff I'm working on as well as book reviews and my random thoughts on

Расписание автобусов
market.android.com

Расписание автобусов и транспорт Москвы, Санкт-Петербурга, Перми, Минска. Также троллейбусы и трамваи Москвы, Питера, Перми. Расписание авто

記帳 CWMoney 理財筆記【標準版】
market.android.com

※超過2,000,000使用者的記帳,橫跨Android/iOS金融類冠軍的 CWMoney 理財筆記-標準版是一套免費好用,專業版最高登上iOS付費總排行榜#1 ,GooglePlay中華區總排行榜第7名,盤踞財經類第一名15個月的中文免費記帳軟體,萬物齊漲的年代,養成記帳好習

gReader | Feedly | News | RSS
market.android.com

gReader is a simple, fast and intuitive feed/rss reader for Android, featuring beautiful themes, podcast support and full offline support. R

СМС не надо
market.android.com

Клиентская программа для социального сервиса «СМСненадо» http://www.smsnenado.ru/, позволяющего отписаться от рекламных SMS-рассылок. Теперь

Light Flow - LED&Notifications
market.android.com

Light Flow allows you to take control of your notification LED colors and makes them successively flash one color after another. It also all

КАНДИДАТ В МЭРЫ МОСКВЫ 2013 Навальный МЕДИЦИНА ЖКХ МИГРАЦИЯ ...
five.navalny.ru

Ужесточение правил. Я прослежу за тем, чтобы чиновники и подрядчики не нанимали нелегальных мигрантов за копейки, а «сэкономленную» часть вы

µTorrent® Remote
market.android.com

Light. Limitless. Access µTorrent® (uTorrent) on your home computer from anywhere.What is µTorrent® (uTorrent) Remote? µTorrent (uTorrent) R

VuDroid
market.android.com

Djvu and pdf viewer/reader.See http://code.google.com/p/vudroid/ for usage.

ZEDGE™
market.android.com

For years ZEDGE™ has been the most trusted and popular source of free ringtones and wallpapers in the world. 50 million people get more than

Trello - Organize Anything
market.android.com

Whether you're planning a surprise birthday party for your best friend, writing an epic screenplay, tracking million-dollar sales leads, or

Last.fm free music player
chrome.google.com

!! Yes it plays music! Browser as a music player? Why not? Free music player for Google Chrome with Last.fm integration.

Conference Listing for Future Image Analysis and Related Topics with Arc...
iris.usc.edu

Welcome to the complete listing of Computer Image Analysis Meetings, Workshops, Conferences and Special Journal Issue Announcements. Inc

Handcent SMS
market.android.com

Handcent SMS is a powerful free sms/mms tools for your android phone The most popular messaging app on the Android Market, Handcent SMS is a

Financisto
market.android.com

Financisto is an open-source personal finance manager. Open-source personal finance manager. - Multiple accounts, multiple currencies - Home

Мой чай — моя крепость
tratatahedron.blogspot.com

Posted 23rd April by Alexander Voronov. Labels: коротко очень коротко нечаянно · Трататаэдральные заметки. Subscribe. Subscribe. RSS Feed. A

PowerShell’s Security Guiding Principles - Windows PowerShell Blog - Sit...
blogs.msdn.com

One of most common issues we face with PowerShell comes from users or ISVs misunderstanding PowerShell's security guiding principles. At

std::copy_n -
en.cppreference.com

Language · Concepts · Utilities library · Strings library · Containers library · Algorithms library · Iterators library · Numerics library ·

ADmented Reality - Google Glasses Remixed with Google Ads - YouTube
www.youtube.com

When I saw Google had somehow forgotten to include any ads in their Project Glass promotional video I just couldn't resist fixing that overs

Probably not the most authentic place, but margaritas are good.
Public - a month ago
reviewed a month ago
I liked Bilyar when I was in the city for the first time, but disappointed later. Last time, they lacked several tartar cuisine dishes and beers other than lager, and served red wine cold. Food is okay, prices are low.
Public - a year ago
reviewed a year ago
The staff is friendly, though not English-speaking, so some Spanish is required to understand the menu. Two tapas (like small burgers) and a local beer for 3.50 euro -- quite a bargain!
Public - a year ago
reviewed a year ago
4 reviews
Map
Map
Map
Great choice of Armenian and other Caucasian dishes. Many types of shashlyk (smoked meat). Food and service is good and fairly priced, but wine is expensive, a bottle would cost at least 2k rubles.
Public - a year ago
reviewed a year ago