Last night I attended a Google Tech Talk in Seattle – they were doing an overview of Google Website Optimizer. For those who haven’t heard of this, it’s a feature buried farther than most, under the AdWords application functionality. But what it does is pretty damn cool – and available for use for free…
The whole idea is enabling A/B (or multi-variate) testing of websites to see which is “better”. I’m pretty sure it’s bolted into the AdWords because it was originally devised to help folks who were spending $$ with AdWords to get folks to their site. One of the key components of this kind of testing is determining “which is better” – which means you need a goal or target action to occur to determine it. In this case, Google’s taken advantage of their own Analytics engine and uses the conversion as that goal measure.
A super-brief overview of what this thing does – you set up two or more variations of a page that you want to test. The “goal” of the test (how you know it worked) is determined by folks viewing your site getting to a goal page – maybe a thank you for signing up page or the like. Then you start plugging in javascript tidbits that do the tracking and control what the user will see by some nifty javascript document.location.replace() method calls. There is a lot of failsafe setup – if javascript isn’t available or the like – and attention paid to make sure that the testing doesn’t break your site, and that the “choose A vs B (vs any others)” is kept as light and quick as possible. Since it was a Google tech talk, I’d expect that you could find the video of this talk in another few days or so, and if you’re interested in this topic, watch it.
The cool thing that I learned last night (or one of them, the speakers were excellent – part of the engineering team that is currently developing out this tool) is that you don’t need to spend $$ to make use of the tool. You’ve got to “sign up” for AdWords, but that doesn’t mean putting in any billing information or even setting up an adwords campaign. The result is that pretty much anyone can now take advantage of the testing mechanisms – something that Amazon and other “big guys” have been doing for ages. Normally its a hell of a lot of infrastructure and thought into enabling that kind of setup, and even more effort on the statistical side of the analysis.
Of course this kind of testing requires traffic. Google has their reports set up so that it’s clear what’s happening with the results, so depending on how many variations and such you’re providing, you’ll get details on what’s doing better or worse, and with what confidence interval. They suggested running any test for at least a week to take out weekly variations, but you can certainly run the tests longer. If you have a low traffic site, you can reduce the number of variations to get meaningful results a little quicker.
Rhonabwy 