Back to blog

Three Things To Remember During an A/B Test

Published

Three Things To Remember During an A/B Test

A/B testing can provide you with invaluable insights into how your customers behave and interact with the content on your website. By testing out two variations of one landing page, you can see which one best converts those clicks into customers. 

Need some help on how to get started with A/B testing? Read our tips on what to do before your next A/B test here. 

In this short guide, we walk you through the things you need to remember during your A/B test. 

Run your tests at the same time. 

Remember, during an A/B test, you need to focus on one variable at a time. There’s no point in creating two very different landing pages for your test as you won’t be able to pinpoint which variable made your customer convert. 

The same goes for the timing of your campaign – you need to make sure that you run the two different versions simultaneously to rule out timing as a factor in your results.

If you run version A one month and version B the next, you won’t be able to collect trustworthy data as the high performance of version A could be down to sheer timing, rather than the design or copy. 

The only time you may find changing up times to be useful is for email or social media campaigns. However, if you are going down this route, you must make sure time is the only variable. 

Give it a chance.

Your A/B test needs to run for an adequate amount of time in order for you to collect useful, actionable data. 

Setting the length of your campaign depends on a few factors, most notably how much traffic your website gets on a regular basis. 

As a general rule, the lower the traffic, the longer you should run the test. 

On the other hand, if you’re getting consistently high traffic, you may only need to run the A/B test for a few days, or even hours. 

Reflect back on what you decided to measure your success on and take a look at your previous website analytics to determine how long your test should run for you to gather proper results. 

Discover the why behind the what.

A straightforward A/B test gives you a lot of statistical data – how many people are clicking through and taking action and how many people are not converting into customers/enquiries. 

But what about finding out the real why behind their actions?

Sure, we can assume this from the data, but it may be a good idea to actually find out from the horse’s mouth, so to speak. 

For example, you could include a survey poll as people exit the site to ask them why they didn’t click through. Or, link a feedback form to those customers who did actually take action on your website. 

This additional insight into people’s actions not only supports the data but can give you even more ideas on how to improve your landing page.


Want to discuss how A/B testing could improve your website? Talk to a member of our team today