captcha

How Good are Humans at Solving CAPTCHAs A Large Scale Evaluation

Elie Bursztein, Steven Bethard, Celine Fabry, Dan Jurafsky, John C. Mitchell   @S&P 2010
1 reaction(s) | 2779 downloads
Captchas are designed to be easy for humans but hard for machines. However, most recent research has focused only on making them hard for machines. In this paper, we present what is to the best of our knowledge the first large scale evaluation of captchas from the human perspective, with the goal of assessing how much friction captchas present to the average user. For the purpose of this study we have asked workers from Amazon's Mechanical Turk and an underground captcha-breaking service to solve more than 318 000 captchas issued from the 21 most popular captcha schemes (13 images schemes and 8 audio scheme). Analysis of the resulting data reveals that captchas are often difficult for humans, with audio captchas being particularly problematic. We also find some demographic trends indicating, for example, that non-native speakers of English are slower in general and less accurate on English-centric captcha schemes. Evidence from a week's worth of eBay captchas (14,000,000 samples) suggests that the solving accuracies found in our study are close to real-world values, and that improving audio captchas should become a priority, as nearly 1% of all captchas are delivered as audio rather than images. Finally our study also reveals that it is more effective for an attacker to use Mechanical Turk to solve captchas than an underground service.
Downloads
paper
slides
You might also like reading

security 2011

Five Surprising CAPTCHA Schemes

captcha 2011

The Failure of Noise-Based Non-Continuous Audio Captchas

captcha 2009

Decaptcha Breaking 75% of eBay Audio CAPTCHAs

captcha 2011

Text-based CAPTCHA Strengths and Weaknesses

reports 2011

Report from the security front-lines July 11th to July 17th

Comments
About me
Lead Google's anti-abuse research. Develop new ways to protect users and disrupt bad guys. Make Chrome safer and faster. Help keeping G+ and Gmail clean. Wear berets. Do magic tricks.
Performance
headers: 0.000620126724243
files: 0.00117993354797
related: 0.0194199085236
sidebar: 0.0925900936127
get publication: 0.126369953156
total: 0.240200042725