Profile

Cover photo
KyungHyun Cho
395 followers|166,959 views
AboutPostsPhotos+1's

Stream

KyungHyun Cho

Shared publicly  - 
 
 
 Announcing Theano 0.7

This is a release for a major version, with lots of new features, bug fixes, and some interface changes (deprecated or potentially misleading features were removed).
Upgrading to Theano 0.7 is recommended for everyone, but you should first make sure that your code does not raise deprecation warnings with the version you are currently using.
For those using the bleeding edge version in the git repository, we encourage you to update to the `rel-0.7` tag.

What's New:
 * Integration of CuDNN for 2D convolutions and pooling on supported GPUs
 * Too many optimizations and new features to count
 * Various fixes and improvements to scan
 * Better support for GPU on Windows
 * On Mac OS X, clang is used by default
 * Many crash fixes
 * Some bug fixes as well

Description:
Theano is a Python library that allows you to define, optimize, and
efficiently evaluate mathematical expressions involving multi-dimensional arrays. It is built on top of NumPy. Theano features:
 * tight integration with NumPy: a similar interface to NumPy's. numpy.ndarrays are also used internally in Theano-compiled functions.
 * transparent use of a GPU: perform data-intensive computations up to 140x faster than on a CPU (support for float32 only).
 * efficient symbolic differentiation: Theano can compute derivatives for functions of one or many inputs.
 * speed and stability optimizations: avoid nasty bugs when computing expressions such as log(1+ exp(x)) for large values of x.
 * dynamic C code generation: evaluate expressions faster.
 * extensive unit-testing and self-verification: includes tools for detecting and diagnosing bugs and/or potential problems.

Theano has been powering large-scale computationally intensive scientific research since 2007, but it is also approachable enough to be used in the classroom (IFT6266 at the University of Montreal).

About Theano: http://deeplearning.net/software/theano/
Related projects: http://github.com/Theano/Theano/wiki/Related-projects
Machine Learning Tutorial with Theano on Deep Architectures:
http://deeplearning.net/tutorial/

Acknowledgments:
I would like to thank all contributors of Theano. For this particular release, many people have helped, and to list them all would be impractical.
I would also like to thank users who submitted bug reports.
Also, thank you to all NumPy and Scipy developers as Theano builds on
their strengths.
All questions/comments are always welcome on the Theano mailing-lists ( http://deeplearning.net/software/theano/#community )
View original post
6
1
노현우's profile photo
Add a comment...

KyungHyun Cho

Shared publicly  - 
 
CFP for the Deep Learning Workshop @ICML 2015 is out! Prepare your manuscripts!

Call for Paper

The Deep Learning Workshop will be held in conjunction with the International Conference on Machine Learning (ICML) 2015 on the 10th (Friday) and 11th (Saturday) of July, 2015 in Lille, France.

Submission Deadline: May 1, 2015 
Submission via https://sites.google.com/site/deeplearning2015/home/submission

Overview

Deep learning is a fast-growing field of Machine Learning concerned with the study and design of computer algorithms for learning good representations of data, at multiple levels of abstraction. There has been rapid progress in this area in recent years, both in terms of methods and in terms of applications, which are attracting the major IT companies. Many challenges remain, however, in aspects like large-scale (hyper-) parameter optimization, modeling of temporal data with long-term dependencies, generative modeling, efficient Bayesian inference for deep learning, multi-modal data and models, and learning representations for reinforcement learning. The workshop aims at bringing together researchers in the field of deep learning to discuss recent advances, ongoing developments and the road that lies ahead.

The workshop invites extended abstracts that will be presented either orally or in poster format. We encourage submissions on the following (non-exhaustive) list of topics:

 Optimization
  - Large-scale optimization
  - Hyper-parameter optimization
  - Model structure optimization
 Regularization
  - Observation-dependent Regularization
  - Generative models as regularization: Semi-supervised learning
 Structured Learning
  - Temporal models with long-term dependencies
  - Deep learning with multiple modalities, including vision, speech and languages
 Unsupervised/generative Modeling
  - Efficient (Bayesian) inference for deep learning
  - Large-scale generative modelling
  - Reinforcement learning
  - Learning representations for reinforcement learning
  - Deep model-based and data-efficient reinforcement learning

Through invited talks, a panel discussion and presentations by the participants, this workshop will showcase the latest advances in deep learning and address questions that are at the center of current deep learning research. 

Invited Speakers

The Workshop on Deep Learning will have ten invited speakers.

First day
 - Tara Sainath, Google
 - Yann Ollivier, Paris-Sud University
 - Oriol Vinyals, Google
 - Jason Weston, Facebook
 - Jorge Nocedal, Northwestern University

Second day
 - Neil Lawrence , Sheffield University
 - Roland Memisevic, University of Montreal
 - Rajesh Ranganath, Princeton University
 - Ian Goodfellow, Google
 - Karol Gregor, Google DeepMind

Important Dates

 - Call-for-Paper Available: 25 March, 2015
 - Submission Deadline: 1 May, 2015 (tentative)
 - Author Notification:  10 May, 2015 (tentative)
 - Early Registration Deadline (ICML '15): 15 May, 2015
 - Workshop: 10 - 11 July, 2015

Submission

Authors should submit an extended abstract whose length we recommend, but do not enforce, to be between 2 and 8 pages with up to 2 additional pages for references. The submitted abstract may be a shortened version of a longer paper or technical report, potentially with a different title, in which case the longer paper should be referred from the submission but reviewers will be asked to judge the submission solely based on the submitted extended abstract. This is to ensure that the original, longer manuscript may be submitted to another publication venue without violating dual submission policy that considers peer-reviewed workshop papers as ordinary publications.

All submissions must be in PDF format, and we encourage authors to follow the style guidelines of ICML 2015 (see http://icml.cc/2015/?page_id=151). Submissions must be made through: https://cmt.research.microsoft.com/DL2015/.

Workshop Organizers

 -  +Geoffrey Hinton 
 -  +Yoshua Bengio 
 -  +Yann LeCun 
 -  +Max Welling 
 - Kyunghyun Cho​
 -  +Durk Kingma 
9
1
Rasool Fakoor's profile photo
Add a comment...

KyungHyun Cho

Shared publicly  - 
19
11
Deniz Yuret's profile photoMikhail Bakhterev's profile photoolgert denas's profile photoSai Rajeshwar's profile photo
2 comments
 
I should note I messed up the initialization bias counteracting in the implementation used for these slides. >.>
Add a comment...

KyungHyun Cho

Shared publicly  - 
 
https://github.com/mesnilgr/nbsvm NB-SVM for Sentiment Analysis +Gregoire Mesnil 
Contribute to nbsvm development by creating an account on GitHub.
1
Add a comment...
 
Neural machine translation applied to English->German this time with a nice result: http://arxiv.org/abs/1412.2007 +Sébastien Jean +Roland Memisevic +Yoshua Bengio 
Abstract: Neural machine translation, a recently proposed approach to machine translation based purely on neural networks, has shown promising results compared to the existing approaches such as phrase-based statistical machine translation. Despite its recent success, neural machine translation ...
9
1
Macduff Hughes's profile photoAlbert Zeyer's profile photo
Add a comment...

KyungHyun Cho

Shared publicly  - 
 
<End-to-end Continuous Speech Recognition using Attention-based Recurrent NN: First Results> by +Jan Chorowski+Dzmitry Bahdanau, me and +Yoshua Bengiohttp://arxiv.org/abs/1412.1602
Abstract: We replace the Hidden Markov Model (HMM) which is traditionally used in in continuous speech recognition with a bi-directional recurrent neural network encoder coupled to a recurrent neural network decoder that directly emits a stream of phonemes. The alignment between the input and ...
12
3
Albert Zeyer's profile photoraia hadsell's profile photo
Add a comment...
In their circles
468 people
Have them in circles
395 people
Jongchan Park's profile photo
Yoan Miche's profile photo
Alexander Ilin's profile photo
Nishtha Tirpathi's profile photo
Ross Goroshin's profile photo
Heechul Jung's profile photo
Liza Stone's profile photo
Frédéric Bastien's profile photo
Arneau van der Merwe's profile photo

KyungHyun Cho

Discussion  - 
 
CFP for the Deep Learning Workshop @ICML 2015 is out! Prepare your manuscripts!


CFP for the Deep Learning Workshop @ICML 2015 is out! Prepare your manuscripts!

Call for Paper

The Deep Learning Workshop will be held in conjunction with the International Conference on Machine Learning (ICML) 2015 on the 10th (Friday) and 11th (Saturday) of July, 2015 in Lille, France.

Submission Deadline: May 1, 2015 
Submission via https://sites.google.com/site/deeplearning2015/home/submission

Overview

Deep learning is a fast-growing field of Machine Learning concerned with the study and design of computer algorithms for learning good representations of data, at multiple levels of abstraction. There has been rapid progress in this area in recent years, both in terms of methods and in terms of applications, which are attracting the major IT companies. Many challenges remain, however, in aspects like large-scale (hyper-) parameter optimization, modeling of temporal data with long-term dependencies, generative modeling, efficient Bayesian inference for deep learning, multi-modal data and models, and learning representations for reinforcement learning. The workshop aims at bringing together researchers in the field of deep learning to discuss recent advances, ongoing developments and the road that lies ahead.

The workshop invites extended abstracts that will be presented either orally or in poster format. We encourage submissions on the following (non-exhaustive) list of topics:

 Optimization
  - Large-scale optimization
  - Hyper-parameter optimization
  - Model structure optimization
 Regularization
  - Observation-dependent Regularization
  - Generative models as regularization: Semi-supervised learning
 Structured Learning
  - Temporal models with long-term dependencies
  - Deep learning with multiple modalities, including vision, speech and languages
 Unsupervised/generative Modeling
  - Efficient (Bayesian) inference for deep learning
  - Large-scale generative modelling
  - Reinforcement learning
  - Learning representations for reinforcement learning
  - Deep model-based and data-efficient reinforcement learning

Through invited talks, a panel discussion and presentations by the participants, this workshop will showcase the latest advances in deep learning and address questions that are at the center of current deep learning research. 

Invited Speakers

The Workshop on Deep Learning will have ten invited speakers.

First day
 - Tara Sainath, Google
 - Yann Ollivier, Paris-Sud University
 - Oriol Vinyals, Google
 - Jason Weston, Facebook
 - Jorge Nocedal, Northwestern University

Second day
 - Neil Lawrence , Sheffield University
 - Roland Memisevic, University of Montreal
 - Rajesh Ranganath, Princeton University
 - Ian Goodfellow, Google
 - Karol Gregor, Google DeepMind

Important Dates

 - Call-for-Paper Available: 25 March, 2015
 - Submission Deadline: 1 May, 2015 (tentative)
 - Author Notification:  10 May, 2015 (tentative)
 - Early Registration Deadline (ICML '15): 15 May, 2015
 - Workshop: 10 - 11 July, 2015

Submission

Authors should submit an extended abstract whose length we recommend, but do not enforce, to be between 2 and 8 pages with up to 2 additional pages for references. The submitted abstract may be a shortened version of a longer paper or technical report, potentially with a different title, in which case the longer paper should be referred from the submission but reviewers will be asked to judge the submission solely based on the submitted extended abstract. This is to ensure that the original, longer manuscript may be submitted to another publication venue without violating dual submission policy that considers peer-reviewed workshop papers as ordinary publications.

All submissions must be in PDF format, and we encourage authors to follow the style guidelines of ICML 2015 (see http://icml.cc/2015/?page_id=151). Submissions must be made through: https://cmt.research.microsoft.com/DL2015/.

Workshop Organizers

 - +Yoshua Bengio 
 - +Geoffrey Hinton 
 - +Yann LeCun 
 - +Max Welling 
 - Kyunghyun Cho​
 - +Durk Kingma 
2
1
Wesley De Neve's profile photo
Add a comment...

KyungHyun Cho

Shared publicly  - 
 
Very nice paper! This may be a way toward building a generative model for large, realistic images: http://arxiv.org/abs/1412.6604
Abstract: We propose a strong baseline model for unsupervised feature learning using video data. By learning to predict missing frames or extrapolate future frames from an input video sequence, the model discovers both spatial and temporal correlations which are useful to represent complex ...
6
2
Vincent Vanhoucke's profile photo노현우's profile photo
Add a comment...

KyungHyun Cho

Discussion  - 
Abstract: While depth tends to improve network performances, it also makes gradient-based training more difficult since deeper networks tend to be more non-linear. The recently proposed knowledge distillation approach is aimed at obtaining small and fast-to-execute models, and it has shown that ...
37
8
Dai Wei's profile photoHao Wooi Lim's profile photo
Add a comment...

KyungHyun Cho

Shared publicly  - 
4
Add a comment...

KyungHyun Cho

Shared publicly  - 
Summary¶. This tutorial aims to provide an example of how a Recurrent Neural Network (RNN) using the Long Short Term Memory (LSTM) architecture can be implemented using Theano. In this tutorial, this model is used to perform sentiment analysis on movie reviews from the Large Movie Review Dataset ...
16
3
Junyoung Chung's profile photoKyle Kastner's profile photoAlec Radford's profile photoPo-Sen Huang's profile photo
12 comments
 
My impression on GRU is that it does something really good on discrete sequence modeling (tasks such as language modeling, polyphonic music modeling, etc.).
Add a comment...

KyungHyun Cho

Shared publicly  - 
 
http://lisa.iro.umontreal.ca/mt-demo/embs/
The web demo of <Not All Neural Embeddings are Born Equal>: +Felix Hill, +Sébastien Jean Coline Devin +Yoshua Bengio 
4
1
Kyle Kastner's profile photo
Add a comment...
People
In their circles
468 people
Have them in circles
395 people
Jongchan Park's profile photo
Yoan Miche's profile photo
Alexander Ilin's profile photo
Nishtha Tirpathi's profile photo
Ross Goroshin's profile photo
Heechul Jung's profile photo
Liza Stone's profile photo
Frédéric Bastien's profile photo
Arneau van der Merwe's profile photo
Links
KyungHyun Cho's +1's are the things they like, agree with, or want to recommend.
BMW 316 E46 2003 - Used vehicle - Nettiauto
www.nettiauto.com

Nyt myynnissä BMW 316 E46 109 000 km 2003 - Espoo. Klikkaa tästä kuvat ja lisätiedot vaihtoautosta.

InoReader: News and RSS Reader
chrome.google.com

RSS reader built with simplicity and speed in mind. Inspired by Google Reader. Made for Google Reader fans.

ICML 2013 Reading List
peekaboo-vision.blogspot.com

The ICML is now already over for two weeks, but I still wanted to write about my reading list, as there have been some quite interesting pap

The Benefits of Double-Blind Review « Machine Learning (Theory)
hunch.net

This post is a (near) transcript of a talk that I gave at the ICML 2013 Workshop on Peer Review and Publishing Models. Although there's a PD

What it took 20 years on Wall Street to learn | Guest Blog, Scientific A...
blogs.scientificamerican.com

I was always attracted to numbers. In middle school my baseball card collection spilled into a second drawer, driven by my love of the stati

Is It Quantum Computing or Not? | Guest Blog, Scientific American Blog N...
rss.sciam.com

This week I had a fascinating discussion on BBC Radio 4 with Dr Geordie Rose, the CTO of DWave, triggered by the news that NASA and ...

End Piracy, Not Liberty – Google
www.google.com

Millions of Americans oppose SOPA and PIPA because these bills would censor the Internet and slow economic growth in the U.S.. Two bills bef

DNA - Puhelimen tarkemmat tiedot
www.dna.fi

Etusivulle. Sivukartta; Yhteystiedot; In English. Hakusana. Yksityisille; Yrityksille; Nettikauppa; DNA Oy. Puhe; Liikkuva Laajakaista; Laaj

Daum - 생활이 바뀐다! Life On Daum
www.daum.net

전체서비스. 가계부; 검색쇼; 검색트렌드; 공연; 게임; 금융; 날씨; 디렉토리검색; 루리웹; 모바일; 모터쇼; 문자. 문학속세상; 문화유산; 미즈넷; 미즈맘; 미즈쿡; 보안넷. 소설; 소셜쇼핑; 쇼핑원; 쇼핑하우; 스타일; 스포츠. 아이찾기; 여행;

[단독] 박주영, 군대 10년 연기…‘은퇴 후 공익근무요원 복무’ - Daum 스포츠
sports.media.daum.net

[일간스포츠 박린]박주영(28·아스널)이 군대를 미뤘다.박주영 측 대리인인 법무법인 DLS의 이성희 변호사는 16일 "박주영이 국외여행기간 연장 허가 되었음으로 국외 입영연기가 허가됐다. 외국에 거주하면 군대를 미룰 수있다는 허가서를 발급 받았다"고

Docs PDF/PowerPoint Viewer (by Google) - Chrome Web Store
chrome.google.com

Automatically previews pdfs, powerpoint presentations, and other documents in Google Docs Viewer.

Chrome Multitask Mode
www.google.com

Chrome Multitask Mode lets you browse the web with multiple mice at the same time, so you can get more done, clickety-split.

Winter tyre change due soon
www.yle.fi

Approaching colder weather means changing over to winter tyres should be done as soon as possible. According to a survey carried out by Y

News: Two Tenure Track or Tenured Positions in Information and Computer ...
ics.aalto.fi

The positions are open to outstanding individuals who hold a doctorate and have excellent potential for a successful scientific career. Cand

Surplus - Chrome Web Store
chrome.google.com

Google Plus Integration for Chrome

Introducing the ZenRobotics R&amp;D Blog
techblog.zenrobotics.com

Short and sweet: we are launching a blog, by the technical staff, for other technical people, to: Share what it is like to work as an engine

이우진의 이글루 : 줄것처럼 하다가 안주는게 더 싫어
stonepc.egloos.com

줄것처럼 하다가 안주는게 더 싫어. 보통 보면... 얼마 되지도 않는데 오라가라... 이래라 저래라... 그러면서 결제일은 또 늦고.... 그나마 조금이나마 되면 다행인데 뭐 줄것처럼 잔뜩 생색내놓고, 결국엔 &quot;미안합니다 다음 기회에...&amp;q

You're Lucky if You're in Tallinn
online.wsj.com

This charming medieval city is one of Europe's most attractive capitals, with its people still reveling over reestablishing their national i

News: 10 Postdoc / Research Associate positions in the new Finnish Cente...
ics.tkk.fi

Department of Information and Computer Science. Aalto Yliopisto logo. Suomeksi. Contact information; Text version. Department of Information