A better way to A/B Testing Development

by Andy Seaton

posted on: January 27th 2017

Like any good developer, there was always a nagging at the back of my head when I was developing AB Tests using any of the main platforms, saying ‘There MUST be a better way to do this!’ This was usually at 1am or so when I was staring at yet another ‘Uploading to CDN’ message and waiting to see whether my test was now working.

So finally, when I managed to have time to think, I wanted to see if I could improve the development process to remove some of the roadblocks that using the inbuilt editors that the main platforms (VWO, Optimizely, etc) provide.

First, I began by gathering some info on the main issues that my fellow devs faced on a day-to-day basis. The main ones were:

The dreaded ‘Uploading to CDN’ in Optimizely - sometimes takes half a second, sometimes takes five seconds. Given that it occurs any time after you save your experiment changes, this is one of the main bugbears we had.
Not being able to use any of our favourite editors (Sublime, Brackets, etc) and pre-processors (like SASS and associated technologies) to assist in experiment development
Images are quite tricky to deal with - if you use ‘code-only’ mode in VWO, for example, the interface doesn’t allow you to upload images at all.
There was no version-control or central storage of experiments - and also no real way of knowing who worked on what.

So armed with this information, I set out to build a small proof of concept system using Node.JS, the Gulp automation service, and Github to manage the files.

The result was ABGulpFlow - a system to allow developers to work on experiments locally and see the results on the actual target webpage, without having to resort to any browser extensions, etc.

It works by taking the experiment JS and SCSS files, processing them using Gulp, and then automatically deploying them to a secure server that we control. Once the processed files are on the server, we link to them from the AB Platform (Optimizely, VWO, etc) editor and use the AB testing preview link to view our altered code.

So rather than having to use the in-built editors, we develop locally (and with the assistance of any extensions like SASS, Autoprefixer, etc) and any time that we make a change to these files, they are automatically deployed to the secure server. This means the changes are deployed immediately - no waiting around for the code to upload to the Optimizely CDN!

From ABGulpFlow, we now come to UCGulpFlow - the v2 if you like, after the whole team had a great and constructive discussion about the pros and cons of such a system. We added support for uploading images (which you can link to in your experiment code), discussed how best to add version control (which means all of our experiments are now backed up to our Github account) and discussed how to implement a ‘Pattern Library’ of common code that we could use within our experiments so that we’re not re-inventing the wheel each time.

This system is still in active development, but it has provided an environment where we can develop experiments with our favourite development tools and extensions, and which is fast and consistent. It also means that we can control conventions a bit more rigorously as well - to ensure that everyone is using the same basic methods to implement tests.

For the future, we are working on a method of using REST API’s to pull experiment code from the relevant Platform (and upload it back again once we’re done) to hopefully integrate closely with the AB Platforms and make our development pipeline even more streamlined. Stay tuned!

avatar for author

Andy Seaton

Andy is a front-end developer who specialises in creating fast, optimized experiences for clients using responsive design and the latest web development techniques.

A better way to A/B Testing Development

by Andy Seaton Time to read: 3 min