Skip to main content
Blog Banner

The ABCs of A/B Testing

abtesting-smallAs a marketer, you run numerous campaigns every year. But, have you stopped to examine if you are doing it right and driving the maximum possible benefit? Has your CTA been positioned at the best place on the page? Is the CTA as a link more beneficial for you than a button? Such questions are effectively answered by A/B tests.

A/B testing involves taking up two separate content pieces, running them parallel for equally sized audiences and determining the better performing page. This practice is a boon for marketers. A/B testing is trusted so much in the marketing field because it provides solid evidence on why something works better than the other. However, you should never test more than one variable at a time.

1. Goals for A/B testing:

Usually, A/B tests fail if they would have unclear goals (you are not totally sure of what you are testing for). Use it to test things like, does adding a video/audio narrative help increase conversions? Will people click more on a link or a button? Such questions are ones that can be easily quantified. A/B tests do not succeed when they are measuring two entirely different page designs with totally different variables. Though you can test two completely different pages, if the results don’t show a clear winner, you will be confused about the cause of increase in conversions.

2. Testing variations:

Even if you have many great ideas to be tested, you need to rein in your enthusiasm to do all of it at once. The more you complicate the A/B test, the more unclear the results will be. Different variables from each design will feature in the results and end up confusing you. A/B tests are done to get dependable results. By testing many designs at a time, you will surely not get such a result. So, it’s best to test two variants at a time.

3. Statistical significance of results:

Every A/B test should reach statistical significance for you to have sufficient confidence in the results. To calculate if your test has done so, you can use the built-in statistical significance in many A/B testing tools. If your tool does not have this feature, you can use the many free calculators available.

4. Testing points:

You can choose the page points you want to test. However, there are a few major aspects which you need to test:

  • Page headline: Usually, a headline is the first thing that comes to a page visitor’s notice when they visit your page
  • Pictures: Test different types of images in your A/B tests. For example, you can test if a picture of someone using your product is more effective than a picture of just the product.
  • Text length: You need to decide if short, crisp text works for you or if you need more detailed content to explain your offer. Test different versions of text to know the extent of explanation a customer needs before conversion. For this to work, you need to use similar text with a difference only in the text volume.
5. Non webpage A/B tests:

Other than web pages, you can run A/B tests on:

AB-Testing (1)

  • Online ads: You can run A/B tests for your online ads by testing two ads with minor differences like different headlines. After comparing the results, you can test more ads against the best performing advertisement. This will get you the best performing ad of all your options. Also, use an image wherever possible with your online advertisement. A picture with your body text gives your ad more power. If you are using an image, you need to test different images also and pick the best performing image.
  • Emails: You can test the subject line, personalization features, sender name, body copy, layout and design, CTAs etc.
  • Pay Per Click (PPC): You can test the headline, body text, link text or keywords used in the campaign.
6. A/B testing frequency:

The best person to decide on how often you should run A/B tests is yourself. But, it is good to keep testing and making necessary changes to pages/content. You just need to make sure that all the tests you run have clear goals and lead to changes for the better.

7. A/B testing and SEO:

A/B testing negatively affects search engine rankings because it could be classified as duplicate content by search engines - this is just a myth. On the contrary, you can improve your site’s functionality with A/B tests. If you are still not able to shake off the concern, you can add a “no index tag” to your variation page.

While A/B testing, if you focus too much on microscopic details, the bigger picture goes unnoticed. You need to be careful that you don’t change the whole picture while changing the smaller details.

What other tactics have you employed in your A/B testing? Let us know in your comments.

Image credit: Smashing Magazine, getbusymedia

Team Position²

Position2 is an innovative digital marketing agency that accelerates demand through integrated Content Marketing, Paid Acquisition, and Marketing Technology solutions. Our demand acceleration solutions map the buyer's journey across multiple touch points such as search, social, mobile, media, and email. These integrated campaigns are powered by cutting-edge content creation, digital advertising, web design/development, marketing automation, and analytics. We serve hundreds of clients ranging from VC funded startups to Fortune 500 multinational firms. Our 200-member team is based in Silicon Valley and Bangalore and consists of marketing gurus, engineers, data scientists, writers, and designers.