You’ve promised to increase that new landing page’s conversions and now everyone’s watching your hands.
And the idea seemed so simple at the time. You wanted to replicate split tests that worked for others and voila…. lift in conversions guaranteed.
But it didn’t work.
Implemented one by one, other people’s tests brought no result whatsoever.
What has worked for these people most likely wouldn’t work for you anyway.
Your landing page is different. So is the audience you target. Your business model, product and offer, among millions of other things.
And most importantly, these people have been solving a completely different problem than you.
So if you’re now stuck at having to come up with an A/B test to improve conversions and scratch your head wondering how, here’s a real treat for you. A quick guide to designing a landing page A/B test.
Few Basic Guidelines for Effective A/B Tests
Plenty of factors affect the success of an A/B test. But the most important, in my opinion are:
First, you should never run more than one test at a time. Anything above that will just muddle the results. These test might actually work and identify elements that could increase conversions. But since you ran a number of them simultaneously, you’d never know what has actually worked.
Also, test only one thing at a time. Even if you run only a single test, don’t test more than one variable by it. You’ll never know which one has worked either.
Test big things but don’t skip evaluating minor changes too. It’s easy to test a button color or another big element. But even a word change in the headline might bring a lift in conversions. So when planning your test don’t ignore smaller variables too.
Have a clear success metric. Define only one success metric that you’re going to use to measure the results.
Now, with that off the way, let’s look at the process of designing a proper landing page A/B test.
The Testing Process
Many approaches exist for conducting experiments. Personally I’m in favor of The Scientific Method.
The Oxford Dictionary defines it as:
“A method of procedure that has characterized natural science since the 17th century, consisting in systematic observation, measurement, and experiment, and the formulation, testing, and modification of hypotheses.” (source)
The Scientific Method begins with an identification of a potential problem, followed by posing a question that becomes the foundation for further research.
Then, based on the preliminary research findings, the researcher constructs a hypothesis she then tests via a properly designed test.
Once done, she analyzes test results and draws conclusions regarding the hypothesis. These conclusions form a foundation for new observation and the process starts anew.
Sounds very … ummm …. Scientific, doesn’t it? It’s a simple process though.
Here’s a little graph that explains it more clearly:
And here’s how to apply the Scientific Method to designing an A/B test:
Start by asking a question or posing a problem.
For instance: “Why this landing page’s conversion is less than 1%?”
Then proceed to analyze the page’s performance.
This is your initial research stage.
Compare your Google Analytics data with the key KPIs you identified for the page.
In case if you’re unfamiliar with the term, KPIs (Key Performance Indicators) are metrics that help you establish how a page performs against objectives you identified for it.
In our example of low conversion page, these KPIs could include:
- Total traffic,
- Average time on page,
- Bounce rate,
- Traffic per source,
- Specific goal completions and many others.
The point of this exercise is to analyze the overall performance of the page and get a full overview of the situation.
Next, review the visitor’s behavior.
It will allow you to reveal potential problems to test.
Heatmaps, clickmaps, video recordings, funnels and other conversion tools could help you identify:
- How visitors engage with the page,
- How much of the content they actually view. If the majority of visitors don’t scroll till the very bottom of the page, and that’s where your c2A is located, it might suggest a possible test.
- Where they drop off.
- What areas they click (or what they attempt to click) and much more.
Use software like Hotjar to collect data you need to identify the visitor’s behavior.
Based on the data you’ve gathered; you can now construct a hypothesis to test.
This hypothesis will define what in your opinion might be causing the problem. In case of a poorly converting landing page, based on the data you’ve gathered your hypothesis might be that “visitors do not see the call to action. Displaying additional C2A beside the headline will increase conversions”.
Based on this hypothesis you can create a split test variations of the page to test.
Calculate how long you need to run the test to receive significant results.
There are generally two approaches to running A/B tests:
Some people launch them and then, stop when they finally remember about them again. I don’t advocate that approach.
Instead, you should calculate how long you should run the test for to get significant results.
The quickest way to do it is use one of the tools available on the market, for instance, this Test Duration Calculator from Visual Website Optimizer.
Simply input all required data and the tool will output the number of days you should run the test for.
Now it’s time for you to test your hypothesis.
This is your A/B test in action.
To run it, you need to create a second version of the page, called the challenger, that will include the change you suggested in the hypothesis. The original version of the page, called Control remains unchanged though.
In the test, half of the traffic will see the Control page while the other half, the Challenger.
In our example, the Challenger will include additional Call to Action beside the headline. And you will test its performance against the original and measure the difference in signups.
Once your test concludes, analyze the data and draw conclusions.
If adding the second Call to Action resulted in higher conversions, then you can safely conclude that it is one of the factors that could increase conversions on the page.
If, however there was no lift in conversions, go back and construct another hypothesis.
Remember, not every hypothesis is correct. That’s the idea of A/B testing, to identify what could work and what is irrelevant and do it via scientific means.
Lastly, report your findings. Let everyone involved in this particular marketing campaign know of any elements they should incorporate on the landing page to improve its performance.
Bonus: A List of Potential Elements to Test
Here are some of the most common elements that influence the conversions and performance of a landing page. By far, you shouldn’t simply jump on and start testing each of them in turn.
But once you have a hypothesis, look at these for inspiration of potential tests that could prove it right or wrong.
- Call to Actions, their placement, copy, color, size.
- Wording on a Page, from the message to amount of content on the page.
- Page’s Layout, position of various elements, their size and color / contrast.
- Images, type of image, relevancy with the page’s topic, what they feature, size, position.
- Pricing, USP and Offer.