What are experiments?

Also known as A/B testing or multi-variate testing, experiments present different versions of your website to a subset of your customers. For example, 50% will see the original website and the other 50% will see an updated version of your website. The change might be as simple as changing the colour of the call-to-action buttons or it might be as drastic as a complete re-design. Experiments will have a goal such as increased purchases or more time spent on your website. The data collected throughout the experiment is then used to statistically determined the “winner”, which is the variant that best reaches the defined goal of the experiment.

What are the benefits to my business?

Experiments can be set up on any website, regardless of industry. Your website will have some action(s) that you want your customers to perform, such as make a purchase or contact you for more information. Experiments can be used to inform changes that would increase the likelihood of your customer taking the actions you want them to. The main takeaways from why you should be running experiments are:

  • Use unbiased data to back up experience/opinion/gut feel
  • Only make changes that will bring real business value
  • Increased confidence that changes will bring you closer to your business goals

How long do experiments run for?

To get the most reliable outcome from an experiment, it’s best to run it for at least two weeks. However, the longer they run, the more accurate the result will be. The longest experiment would be three months. What happens at the end of an experiment?

All visitors to your website will see the original version again until you make permanent changes to your website. Ideally, this will be based on the data and result from the experiment. This may mean doing nothing at all if the original outperformed the experimental version.

How much does an experiment cost?

As a guide, here are a couple of examples:

First experiment – simple changes

  • One-off setup of experiment framework: £499
  • Setup of specific experiment – hide slideshow on homepage: £199
  • Results analysis and recommendations of post-experiment actions: £399

Future experiments – simple changes

  • Setup of specific experiment – change button colour site-wide: £299
  • Results analysis and recommendations of post-experiment actions: £399

Future experiments – detailed changes

  • Setup of specific experiment – add new content block to every page, change button colour site-wide and hide slideshow: £899
  • Results analysis and recommendations of post-experiment actions: £399

These examples are just a guide and actual cost will depend on your website, the access available to change it and how detailed and wide-spread the changes you want to include in the experiment are.

The length of the experiment doesn’t affect the cost. Like you, we want the most accurate data and if it take three months to get that, that’s what will happen.