Profile cover photo
Profile photo
Gunnar Aastrand Grimnes
347 followers
347 followers
About
Gunnar's posts

Post has shared content
AI startup Vicarious claims to have a system that can solve CAPTCHAs with "succes rate up to 90%".  

Beware: It's a textbook example of AI hype of the worst kind

Hype is dangerous to AI. Hype killed AI four times in the last five decades. AI Hype must be stopped.

Perhaps Vicarious can get "up to 90%" accuracy on some CAPTCHA dataset they cooked up, but
- (1) breaking CAPTCHAs is hardly an interesting task, unless you are a spammer;
- (2) it's easy to claim success on a dataset you cooked up yourself. There is no risk someone else will beat you. 
- (3) recognizing object in images is much, much harder than breaking CAPTCHAs. Some deep learning systems can already do this with decent accuracy. Some such systems have been deployed by Google and Baidu.
- (4) doing simultaneous segmentation and recognition of character strings is hardly a breakthrough. See demos of a 20 year-old system here: http://yann.lecun.com/exdb/lenet/index.html

The sad thing is that this announcement is being picked up by a number publications, including MIT Tech Review, Forbes, etc.

Here is an advice to scientific/tech journalists: please, please do not believe vague claims by AI startups unless they produce state of the art results on widely accepted benchmarks. 

This is particularly true for claims in image and speech recognition for which good benchmarks exists. For image recognition, a good example of such benchmark would be the ImageNet Large Scale Visual Recognition Challenge.

Whenever a startup claims "90% accuracy" on some random task, do not consider this newsworthy.  If the company also makes claims like "we are developing machine learning software based on the computational principles of the human brain" or uses impressive-sounding names like "Recursive Cortical Network", be even more suspicious. 

There are extremely impressive applications of deep learning out there (e.g. deployed by Google, Baidu, Microsoft, IBM, and a few startups), but this is not one of them.

Google's automatic photo tagger and Baidu's image retrieval system are much, much more impressive than the system in this announcement. Even if we just talk about challenging character recognition tasks, Google's system for picking out house numbers in StreetView images is way more impressive than this. 

AI "died" about four times in five decades because of hype: people made wild claims (often to impress potential investors or funding agencies) and could not deliver. Backlash ensued. It happened twice with neural nets already: once in the late 60's and again in the mid-90's.

Don't let it happen again. Beware of hype.

And by the way, no one is interested in breaking CAPTCHAs except spammers and computer security researchers. That's why you won't find many computer vision papers on the topic. That's also why it would be easy to break records, even if a standard dataset existed. 

Post has attachment
Christmas can come to +Bakken & Bæck !
Photo

Post has attachment

Post has shared content

Post has attachment
Wait while more posts are being loaded