A recurring theme in the last meeting about monetization was the importance of measuring what’s working and what isn’t. So this week we are going to focus on bringing a data driven product development approach to your startup through experimentation and A/B testing (aka split testing).
We are delighted to have two amazing groups volunteer to speak:
Both of these teams are at the forefront of bringing a more scientific approach to product development. If you’ve ever been interested in learning the why, how, and what about A/B testing, you’re not going to want to miss this meeting.
Over the last three years, we built an experimentation platform system (ExP) at Microsoft, capable of running and analyzing controlled experiments on web sites and services. The goal is to accelerate innovation through trustworthy experimentation and to enable a more scientific approach to planning and prioritization of features and designs (Foley, 2008). Along the way, we ran many experiments on over a dozen Microsoft properties and had to tackle both technical and cultural challenges. The talk will focus on problems not commonly addressed in technical papers: cultural challenges, lessons, and the ROI of running controlled experiments.
The talk is based on the paper at http://exp-platform.com/expMicrosoft.aspx , which won 3rd place at the Third Workshop on Data Mining Case Studies and Practice Prize.
Ronny Kohavi is the General Manager for Microsoft’s Experimentation Platform. Prior to joining Microsoft in June 2005, Ronny was the director of data mining and personalization at Amazon.com, where he was responsible for personalization, automation, search engine marketing (SEM), consumer behavior / data mining, site experimentation, and automated e-mail. Prior to Amazon, Ronny was the Vice President of Business Intelligence at Blue Martini Software, where he led the engineering group responsible for the data collection, analysis, visualization, reporting, and campaign management modules in Blue Martini’s applications. Prior to joining Blue Martini, Kohavi managed the MineSet product, Silicon Graphics’ award-winning product for data mining and visualization.
Ronny received a Ph.D. in Machine Learning from Stanford University and a BA from the Technion, Israel. He was the General Chair for KDD 2004, he co-chaired KDD 99’s industrial track with Jim Gray, and he co-chaired the KDD Cup 2000. More information about Ronny is available at http://www.kohavi.com
Your website’s structure, design, and content are the keys to getting visitors to take the action you want. Website testing, using tools like Google Website Optimizer, helps you experiment with your website to find what drives the most conversions. In this session you’ll learn how to start experimenting with your website and how successful startups like DailyBurn and 37 Signals have used Website Optimizer to increase conversion rates and get more customers.
Sandra Cheng is a Product Manager at Google where she is responsible for the overall product strategy and roadmap for Google Website Optimizer. She also manages ads optimization related products for AdWords. Previously, Sandra worked at Amazon.com, where she led product management for Endless.com.
Victor Chudnovsky has been a software engineer on the Website Optimizer team since he joined Google in 2007. Previously, he worked on the Physical Modeling team at The MathWorks. He was educated at MIT, where he earned bachelor degrees in Physics and Mathematics and a doctorate in Physics.