Learn what A / B testing is and how it can help you get better results – WAU
A / B tests are experiments performed with the objective of comparing variables in Marketing strategies. Through them it is possible to define which variable generates the best results, further optimizing the conversion of the pages, as well as other success metrics.
One of the great advantages of Digital Marketing is the possibility of measuring the result of each action in your strategy. Thus, we can understand what works, to repeat in the future, and what does not, to find points for improvement.
The most efficient way to refine your marketing strategy and focus on your goal is to perform A / B testing. With them, you can do two tests changing only one variable, and in the end evaluate what generated the best results.
This way, you don’t waste time guessing what should or should not work best with your personas.
A / B testing can reveal that some adjustments to your website pages or email marketing can generate many more visits and leads.
Read on to learn how to do A / B tests that yield results!
What are A / B tests
A / B tests are tests that perform comparisons between marketing variables, in order to define which variable generates the best responses. For example, you can test two landing page formats to determine which generates the highest conversion rate.
It is worth mentioning that A / B tests are different from multivariate tests. Multivariate tests test more than one variable at once, while A / B tests test only one variable (two variations) at a time in order to choose the best one between the two.
To perform an A / B test of the color of a Call-to-Action button, for example, you must choose two variations, such as red vs. blue, and determine which one is best according to your goals.
Tests eliminate the need to make assumptions and allow you to make decisions based on numbers, which decreases the chance that your decision will result in future failures.
With that in mind, we made a video in order to explain even more clearly what A / B testing is and how it works. Check out:
What to test
As already mentioned, A / B tests can be performed on Landing Pages, Calls-to-Action and emails. Have you ever wondered, for example, if your emails would get more clicks if you added an image?
Although we sometimes feel like testing all possible variables, some tests are more relevant than others. So be careful when choosing what to test.
Some variables will not contribute to increase your click or conversion rate, so recognize the most important ones for optimizing your Digital Marketing strategy.
Below, we present some variables that are most critical in the lead generation process.
If you’re conducting A / B testing on your landing pages for the first time, start with offer types. What formats or subjects convert more visitors into leads, and leads into customers?
If you already have some experience with testing on Landing Pages, you can test other factors such as:
- Organization of elements;
- Form fields;
- Information about a product on a landing page;
- Visual elements such as colors, arrows, asterisks, etc.
In the case of A / B email tests, the best way to increase your engagement is to test different email formats. For example, you can test whether your contacts engage more with emails which contains only one topic, such as promotions, or with newsletters, which aggregate various information in an email.
Other tests in the email that can generate results are images, different CTAs, time of submission and the subject.
A / B tests on CTA are very common to optimize conversion rates. Therefore, the best test of a Call-To-Action on a page is its location, as it can reveal interesting data and increase its conversion. In addition, it is interesting to test the size, color, text and graphic elements of the CTA.
How to do A / B tests
Now that you know what they are and what A / B Tests can look at, let’s show you how they are conducted.
Bet on specialized tools
Generally, A / B testing is done with specialized tools or through marketing automation software that has this functionality. Some examples of tools that perform these types of tests are Google Analytics, Optimizely, Unbounce and
The only type of A / B testing that can be performed without specialized tools is emails. For this type, an email tool can handle the job. However, the process will not be as automated and can lead to errors.
Ensure statistical significance
The first step in conducting an A / B test is to determine the sample of people that will give your test statistical significance.
That is, how many people do you need to send an email to, for example, so that your A / B test has statistical significance? Or how many people need to visit your landing page?
To find out what this number is, you can use a significance calculator, like this Kissmetrics calculator
Create variations that will show results
Once you have determined what your A / B test will be sampling, you need to create the two variations that you will test. Remember that they will necessarily have only one element of difference.
For example, if you are going to test the effectiveness of an image on your landing page, you need to create a landing page with an image and the other without an image. The other elements of the page must remain exactly the same.
Understand how A / B tests are implemented
The implementation of the A / B test varies depending on the tool you decide to use. Some tools separate sampling in half.
That is, in the case of emails, 50% of the sample will receive one variation and 50% will receive the other. With landing pages and CTAs, half of your website visitors will see one version of the page or CTA, and half will see the other.
This way, at the end of the test period you will be able to identify which variation gave the most results, and repeat in the future.
Nowadays, A / B testing tools are already very developed. Some have a dynamic component, in which the tool itself determines which variation is most successful according to a chosen success metric. Thus, the tool automatically implements the variation that generated the most results.
This type of A / B test is done more often on landing pages, because as soon as the tool determines that the pages participating in the A / B test have already received enough traffic for the test to have statistical significance, it determines the winner and from there 100% of the traffic will see this page optimized.
This way, you optimize your landing pages as quickly as possible, without risking visits to a page that you know is not efficient.
How to measure your results
Now let’s talk about the last step, but no less important: the measurement of results.
Give time to time
When you start doing A / B tests, you may be tempted to check the results every minute. This will not do you any harm, but you will not measure your actual results, just by monitoring your test.
It is as if a soccer player measures his success in the first goal of his life. It doesn’t mean he’s going to become a professional, right?
The actual result of an A / B test can take hours, or even days, to reveal. In the case of email tests, the time period can be short, ranging from 1 to 3 hours depending on how many people received it.
This is because not everyone will open or click on your email in the first 30 minutes. Therefore, you need to wait until enough people have opened or clicked to reach statistical significance.
In the case of landing pages and CTAs, the trial period should be much longer to complete significant results, especially if your pages have a low volume of traffic.
The recommended period is 15 to 30 days, as it will guarantee that the pages of your website or blog will receive enough visits for your test to have statistical significance. The results of the landing page and CTA tests will take longer to appear because we cannot control the volume of visits that a page receives.
If after 30 days your test still does not show significant results, it means that it does not have as much impact on your strategy.
That is? Time to abandon this test and move on to the next one!
Use only one metric to choose the winner
After you’ve waited the time necessary to reap the results of your test, analyzing whether it was successful is easy since by the time you have chosen what you were going to test, you have already established a success metric.
For example, if you chose to test the color of a CTA, the metric of success will be the number of clicks on that CTA. So when you complete the test, identify which of the two variations generated the most results according to your metric!
This will be declared the winner.
If you have an A / B test tool that identifies alone which variation generated the best result, you don’t need to do anything else. Otherwise, remember to implement this variation. Update your landing pages and CTAs and, in the case of emails, use the same “formula” again.
How to optimize your strategy to generate even more results
Contrary to what many people think, the true potential of an A / B test does not end when you decide the winning variable. To generate even more results, think about what you learned from the test, what actions you can take in the future, and especially how you can improve it.
Repeating the same type of test can be extremely effective to discover more and more in-depth information, in addition to saving time, since you will have the basic test structure in hand.
For example, perform subject tests on emails several times to find out which combination of factors works best with your contacts. Do the same with landing pages and CTAs.
Keep your tests organized in a spreadsheet. Add information such as: the type of test, which metric of success you will use, start and end dates, hypothesis and result obtained at the end of the test.
So, you avoid repeating tests that have already been carried out, you can improve those that worked and discard things that have been proven ineffective for good. With that your marketing strategy will always be improving!
Don’t forget to communicate the test results to others in your company! They can reveal relevant findings for several departments, not just the Marketing team.
As you can see, A / B testing is a great way to optimize the conversion rates for your landing pages, emails and CTAs. In addition, through testing you and your company will learn a lot about your personas, and how they behave on the Internet.
So, have you ever done or done A / B tests? Have any questions? Tell me in the comments!
Want to understand more about essential elements in digital marketing? Read our full ebook and get to know them!