Profile cover photo
Profile photo
Arthur Chan
142 followers -
Startup Guy, Speech Scientist/Engineer, Machine/Deep Learning Enthusiast, Blogger
Startup Guy, Speech Scientist/Engineer, Machine/Deep Learning Enthusiast, Blogger

142 followers
About
Posts

Post has shared content
Originally shared by ****
I have seen years of lost time on CV, LeNet5 on mnist, CNNs on Lush. All these moments, lost in time, like tears in the rain...
Add a comment...

Post has shared content
Pretty neat idea -- I highly recommend this video. I hadn't thought of this from a state superposition standpoint before. I was also not aware of the graphical model interpretation either (I will probably try to look this up later if I don't forget).
Add a comment...

Post has shared content
Intriguing.
TIL: p(A∩B)≥p(A)p(B) for p Gaussian and A,B centered sets.
Also: a cautionary tale about picking the right publishing venue.
Add a comment...

Post has shared content
The Bandwagon
(using in the words of Claude Shannon, 1956)

Machine Learning has, in the last few years, become something of a scientific bandwagon. Starting as a technical tool for the computer scientist, it has received an extraordinary amount of publicity in the popular as well as the scientific press. In part, this has been due to connections with such fashionable fields computing machines, cybernetics, and automation; and in part, to the novelty of the subject matter. As a consequence, it has perhaps been ballooned to an importance beyond its actual accomplishments. Our fellow scientists in many different fields, attracted by the fanfare and by the new avenues opened to scientific analysis, are using these ideas in their own problems. Applications are being made to biology, psychology, linguistics, fundamental physics, economics, the theory of organisation, and many others. In short, machine learning is currently partaking of a somewhat heady draught of general popularity.

Although this wave of popularity is certainly pleasant and exciting for those of us working in the field, it carries at the same time an element of danger. While we feel that machine learning is indeed a valuable tool in providing fundamental insights into the nature of computing problems and will continue to grow in importance, it is certainly no panacea for the computer scientist or, a fortiori, for anyone else. Seldom do more than a few of natures' secrets give way at one time. It will be all too easy for our somewhat artificial prosperity to collapse overnight when it is realised that the use of a few exciting words like deep learning, artificial intelligence, data science, do not solve all our problems.

What can be done to inject a note of moderation in this situation? In the first place, workers in other fields should realise that the basic results of the subject are aimed in a very specific direction, a direction that is not necessarily relevant to such fields as psychology, economics, and other social sciences. Indeed, the hard core of machine learning is, essentially, a branch of mathematics and statistics, a strictly deductive system. A thorough understanding of the mathematical foundation and its computing application is surely a prerequisite to other applications. I personally believe that many of the concepts of machine learning will prove useful in these other fields -- and, indeed, some results are already quite promising -- but the establishing of such applications is not a trivial matter of translating words to a new domain, but rather the slow tedious process of hypothesis and experimental verification. If, for example, the human being acts in some situations like an ideal predictor, this is an experimental and not a mathematical fact, and as such must be tested under a wide variety of experimental situations.

Secondly, we must keep our own house in first class order. The subject of machine learning has certainly been sold, if not oversold. We should now turn our attention to the business of research and development at the highest scientific plane we can maintain. Research rather than exposition is the keynote, and our critical thresholds should be raised. Authors should submit only their best efforts, and these only after careful criticism by themselves and their colleagues. A few first rate research papers are preferable to a large number that are poorly conceived or half-finished. The latter are no credit to their writers and a waste of time to their readers. Only by maintaining a thoroughly scientific attitude can we achieve real progress in machine learning and consolidate our present position.


http://www.eoht.info/page/Shannon+bandwagon
Add a comment...

Post has shared content
It's funny because it's real.
"[…] bug caused by a massive brain slip of that Gleixner dude […]" Another example about the rude tone in Linux kernel development. That tglx guy who prepared this merge comment really shouldn't talk about Thomas like that ! ;-)
Photo
Add a comment...

Post has shared content
I often hear researchers complaining how Google tends to publish a lot about large-scale, comparatively dumb approaches to solving problems. Guilty as charged: think about ProdLM and 'stupid backoff', or the 'billion neuron' cat paper, AlphaGo, the more recent work on obscenely large mixture of experts or the large-scale learning-to-learn papers.
The charges levied against this line of work is that they're inefficiently using large amounts of resources, not being 'clever', and that nobody else can reproduce them as a result. But that's exactly the point!! The marginal benefit of us exploring the computational regimes that every other academic lab can do just as well is inherently limited. Better explore the frontier that few others have the resources to explore: what happens when we go all out; try the simple stuff first, and then if it looks promising we can work backwards and make it more efficient. ProdLM gave us the focus on data for machine translation that made production-grade neural MT possible. The 'cat paper' gave us DistBelief and eventually TensorFlow. That's not waste, that's progress.
Add a comment...

Post has shared content
Add a comment...

Post has shared content
Probably the first time we see CTC-based system better the standard context-dependent system. It requires 125k hour of data, perhaps it also explains why we see mixed results of CTC from academic institutions, which usually trained with less than 10k hours.
There goes the Language models and Decoding in Speech Recognition. An LSTM based Neural Network directly outputs words from simple speech features. Caveat is, it requires more than a hundred thousand hours of data to train. This is probably required to capture word level information within the network. Impressive anyway.
Add a comment...

Post has shared content
Deep Mind continues tweaking the Neural Turing Machine for better performance: "Hybrid computing using a neural network with dynamic external memory", Graves et al 2016 https://www.dropbox.com/s/0a40xi702grx3dq/2016-graves.pdf https://deepmind.com/blog/differentiable-neural-computers/

"Artificial neural networks are remarkably adept at sensory processing, sequence learning and reinforcement learning, but are limited in their ability to represent variables and data structures and to store data over long timescales, owing to the lack of an external memory. Here we introduce a machine learning model called a differentiable neural computer (DNC), which consists of a neural network that can read from and write to an external memory matrix, analogous to the random-access memory in a conventional computer. Like a conventional computer, it can use its memory to represent and manipulate complex data structures, but, like a neural network, it can learn to do so from data. When trained with supervised learning, we demonstrate that a DNC can successfully answer synthetic questions designed to emulate reasoning and inference problems in natural language. We show that it can learn tasks such as finding the shortest path between specified points and inferring the missing links in randomly generated graphs, and then generalize these tasks to specific graphs such as transport networks and family trees. When trained with reinforcement learning, a DNC can complete a moving blocks puzzle in which changing goals are specified by sequences of symbols. Taken together, our results demonstrate that DNCs have the capacity to solve complex, structured tasks that are inaccessible to neural networks without external read–write memory."
Add a comment...

Post has shared content
Imagenet 2016 results summary: Chinese teams are going strong. But many big names seems to lose interest on the game.
* Winning team on classification is Trimps-Sousen. According to the description:
"Object classification/localization (CLS-LOC)
Based on image classification models like Inception, Inception-Resnet, ResNet and Wide Residual Network (WRN), we predict the class labels of the image. Then we refer to the framework of "Faster R-CNN" to predict bounding boxes based on the labels. Results from multiple models are fused in different ways, using the model accuracy as weights. "
* Only two teams beat the previous baseline 3.08%, and the best result from Trimps-Sousen only beat it slightly (2.99%).
* CUIMage from Chinese University got the highest MAP score on in object detection and object detection from video, on the other hand, NUIST top video identification task. I would really want to read both papers soon.
* Hikvision, a Shanghai Team, won scene classification.
One comment echoing Vincent Vanhoucke, imagenet classification probably finish its purpose. The Coco task (image captioning) and VID task are probably more in focus from now on. From what I see from the industry, perhaps scene classification and object detection, both from image and video, still have room for research and growth.
ImageNet results are out! A ton of new entrants, yet on image tasks, only two teams beating the best published results, and not by much (e.g. classification went from 3.08% error down to 2.99%). This really confirms to me that this contest has run its course. None of the usual suspects entered, likely focussing instead on COCO (except Facebook, who apparently didn't get the memo :D ).
Interesting showing from NUIST on the video tasks. Looking forward to reading all the papers. Congratulations to the winners!
Add a comment...
Wait while more posts are being loaded