Are your competitors A/B testing?

by David Mannheim

posted on: December 29th 2015

More than half (53%) of companies plan to increase their budget for optimization in the next 12 months [Source] and 70% of marketers are not aware of their competitors’ conversion rates, nor the average of their industry according to a survey by Millward Brown.

I've always wondered if companies are A/B testing. If they're not, they soon will be! Once you’re ahead as a company that undertakes a robust and methodical practice of conversion rate optimisation, you gain money; if you're behind you’re always playing catch-up and therefore, guess what? You’re losing money. Doing conversion rate optimisation gives you the first mover advantage.

Did you know you can tell whether your competitors are A/B testing? In some cases you can even tell what they are A/B testing! But sorry, still no word on whether you can see their results.

There are a few ways of doing it and whilst they're not fool proof they are quite nifty.

See if your Competitors are A/B testing

Ghostery is a Chrome extension where you can see exactly what software the website has on your site by determining what the site is tracking.

It comes in use when you see some familiar names such as "Optimizely" or "Adobe Test and Target". When you see these names you know that they are A/B testing, or, at least, have dipped their toe in the water - even if the script is within Google Tag Manager! Although be wary, there are some pitfalls:

  • They might have the script on the site but not be actively testing; unfortunately there's no real way to find out.
  • It won't take into account Google content experiments as it's in built within Analytics (but who uses that anyway?)
  • It also doesn't take into account those site that have built their own A/B testing platforms for Cloudera’s Gertrude, Etsy’s Feature, or Facebook’s PlanOut. These are, however, more enterprise level approaches to A/B experiments.

The names that you want to look out for are:

What your competitors are A/B testing

Optimizely and VWO are two testing platforms where, when you create experiments, you should be QA'ing properly. And lucky for us, there are a couple of extensions that allow us to QA, not just our own experiments, but those experiments for our competitors.

These extensions are great when spying but does involve some manual labour as they are identifiable on a page by page basis. By clicking through a typical customer journey, if the company is indeed testing, you should be able to stumble across a page that has an active experiment running on it. Here's how:

  • Optimizely

If a website is using Optimizely and we have Conversion.com's Optimizely Chrome extension (PS. Great job, Conversion.com!) installed we can see, not just if Optimizely is running on the page, not just what active experiments are on the page, but the ability to select and view a different variation of an experiment. Take Adidas.com. By using the extension, I was able to see that they are testing a variation on the homepage, within the navigation called "EUA-69 Design Your Own Header vs Customise" (29.12.15) and it looked like this:

adidas-ab-test

  • VWO

A chap called Andre Scholten created something similar with VWO where you can view live experiments running on a page using VWO. Unlike the above extension this is actually a piece of js that you bookmark and call when needed. With this, as an example, I was able to identify that Slack.com are testing the messaging on their homepage:

slack-ab-testing

 

Get updated when your competitors are A/B testing

Ah this is the section everyone wants to read. Well unfortunately it comes at a cost and lacking accuracy; because it's not solely around who is and isn't A/B testing. There are a few tools available that allow you to see if your competitors have made a change on a page by using various screenshot captures on a daily or weekly basis. So it's not necessarily A/B testing but the foundations of "changes to a page" are similar and equally valuable.

Those that I have read about and been recommended are:

  • CRO Monitor is a small example by the guys at Rocketmill that's both extremely limited, free and extremely limited.
  • Diffbot tracks content changes by using image processing to identify changes in a page
  • RivalFox does something similar but in a more holistic sense tracking everything a competitor does - from changes to traffic levels and social mentions.
  • Ispionage tracks, mainly PPC keywords, but also landing page changes denoting those who are and aren't testing their landing pages.
avatar for author

David Mannheim

David is an experienced conversion optimiser and has worked across a series of core optimisation disciplines including web analytics, user experience and AB & MVT testing.

Are your competitors A/B testing?

by David Mannheim Time to read: 3 min