Last night I attended a Google Tech Talk in Seattle – they were doing an overview of Google Website Optimizer. For those who haven’t heard of this, it’s a feature buried farther than most, under the AdWords application functionality. But what it does is pretty damn cool – and available for use for free…
The whole idea is enabling A/B (or multi-variate) testing of websites to see which is “better”. I’m pretty sure it’s bolted into the AdWords because it was originally devised to help folks who were spending $$ with AdWords to get folks to their site. One of the key components of this kind of testing is determining “which is better” – which means you need a goal or target action to occur to determine it. In this case, Google’s taken advantage of their own Analytics engine and uses the conversion as that goal measure.
The cool thing that I learned last night (or one of them, the speakers were excellent – part of the engineering team that is currently developing out this tool) is that you don’t need to spend $$ to make use of the tool. You’ve got to “sign up” for AdWords, but that doesn’t mean putting in any billing information or even setting up an adwords campaign. The result is that pretty much anyone can now take advantage of the testing mechanisms – something that Amazon and other “big guys” have been doing for ages. Normally its a hell of a lot of infrastructure and thought into enabling that kind of setup, and even more effort on the statistical side of the analysis.
Of course this kind of testing requires traffic. Google has their reports set up so that it’s clear what’s happening with the results, so depending on how many variations and such you’re providing, you’ll get details on what’s doing better or worse, and with what confidence interval. They suggested running any test for at least a week to take out weekly variations, but you can certainly run the tests longer. If you have a low traffic site, you can reduce the number of variations to get meaningful results a little quicker.