Another fun paper on A/B testing out of Ronny Kohavi's group. One interesting tidbit is that Bing found that every 100ms faster they deliver search result pages to searchers yields 0.6% more in revenue (which is a similar result to an old Amazon test). Another great piece is the advocacy for not just A/B testing experimental features, but just go ahead and build your entire deployment system so that most software rollouts are done as A/B tests to check for costly bugs (like bad data or big increases in page latency).
4 plus ones
Shared publicly•View activity
- Ed Chi+2Similar to findings at Google many many years ago.May 21, 2013
- Jun 24, 2013
Add a comment...