Unequal visits in Google Experiments

Having run my new Google Experiment for a couple of weeks I have noticed that the distribution of traffic between the default and test page has become hugely unequal.
After a bit of trawling around the web I discovered that this was most likely caused by Google applying a Multi-armed bandit test approach (here)

Google states "a multi-armed bandit is a type of experiment where:
the goal is to find the best, or most profitable action, and
you learn about payoff probabilities as the experiment progresses."

"Once per day, we take a fresh look at your experiment to see how each of the variations has performed, and we adjust the fraction of traffic that each variation will receive going forward. A variation that appears to be doing well gets more traffic, and a variation that is clearly under performing get less."

So essentially Google are no longer retaining an equal traffic share for each variation in the test. If a variation is perceived to be doing worse than another or the default it gets less number of visitors. Equally if a test variant is doing better than another variant or the default it gets a greater share of the traffic. This is fine and will be rewarding for people who want a quick win during experiementation time but it's harder to swallow for those that have been doing conventional A/B or MVT testing up until now. In the past Google have let you adjust test weighting manually and it would be good to offer an overide option whereby the Multi Arm Bandit approach can be disabled by those who want clarity of their tests.

Google Content Experiments

Google has announced its moving away from Google Website Optimizer to 'Content Experiments'. This effectively means Optimizer has been fully integrated into the new version of Google Analytics. There are as ever positives & negatives involved. First off it means less coding to be uploaded to your test pages. Great. On the downside tests are now wholley driven around Google Goals. In the past I've found setting up goals to be hard work and they dont always work so coupling testing to goals fills me with concern. Finally I think Google have missed a trick. I think they had an opportunity to make their testing tool a 'tagless solution' under this approach. If they worked in a bit of dynamic javascript into their normal analytics tags you would never have to upload test code for each new experiment. This would greatly increase peoples abilities to conduct testing without further IT input, essentially the holy grail of professional testers. I'm just lining up my 1st test under this regime, I'll you know how it goes.

1st iPhone MVT test. So proud...

Had to share this because, well frankly it was a ball ache & also there is little if not naff all documented on the internet about this (at time of writing). Basically today I managed to get Google Optimizer (now branded as Google Experiments)  to work for MVT testing on an iPhone landing page and also circumvent all the issues that go with coding GWO tests on a HTML & jQuery page. The test page in question is here: 
Essentially it's very hard to do conventional MVT testing on mobile pages as the use of jQuery etc can really complicate straightforward test building & coding. My tip is to knock out all jQuery references such as 'data-role' and make inline requests to restore your CSS. I may add further detail in a later post but this mobile testing is in its infancy as far as my experience is concerned.

Google Analytics Real-time

Well I have to say I've been blown away today by a very slick new function in GA. Real-time (beta) gives you live visitor data for your website. Essentially It gives you live unique visitor volumes, page views, popular pages, top referrers, keywords and geolocation, oh and device breakdown. ITS BRILLIANT! It gives you the last 30 minutes of activity and leaves you wanting the data gap filled between Real-time and your standard reports with their 24 hr lag! It's also very addictive. I've just been watching activity on our main site for the last couple of hours. It's held firm with circa 4.5k visitors at any one moment only for Eastenders to come on telly and instantly see the figures drop to around 500, mainly in the South East : )
Real-time has actually been around since autumn 2011, but available only to the chosen few. I've overlooked it's existence purely because like many I've hated the new version of GA, or the interface at least and have continually flicked back to the old version where this feature is not available. Have to say I will be revising my view of the new version given this little gem of a feature. To get to the report click on the 'home' tab in GA (new) and it's the top link on the left hand nav. Works super fast in Chrome but slower in other browsers. Enjoy!

First mobile landing page test

I launched a Google Otimizer Ab test at the start of the week to see if I could drive an uplift in Mobile Banking registrations. Essentially the newly launched landing page for mobile banking had a CTA for the registration form but it was available on only the last tab of a 4 tab page design. Using GWO I added the same CTA to the footer of the page so that it was available across all tabs. Within just a few days the new design was beating the old design hands down with a +200% uplift in people clicking through to the registration form with a 99% confidence index. I'll add an image of the design soon. Update: the design has sustained it's uplift 3 weeks on. We now have a 10.5% conversion versus 1.5% for the old design.