What is Split Testing?

In this article, we’ll go over what split testing is, why you need to do it, how, and more. 

So you’ve got your website set up, your copy written, and your design looks fantastic. Your site is converting at a decent rate and you’re happy with the overall results. You’re done now, right? Ready to retire and let the site make you money for the rest of your days?

Nah, there’s more to do because you’re leaving money on the table if you’re not split testing. 

Split testing is a common synonym for “a/b testing”, which is the practice of testing your website’s traffic on two or more different versions of the same page. The idea of this is to see which version of the page performs better so you can increase your conversions long term, whether that means sales, leads, clicks, etc. 

Why should you do split testing?

You should perform split testing on your website on your high-priority pages to increase your conversion rate, increase time on page, minimize the risk of design changes, and reduce bounce rates. 

Let’s break these down below.

Increase conversion rate

Your conversion rate is the most important thing to you at the end of the day. When you look at your statistics, whether it’s in Google Analytics or another conversion tracking software, it’s nice to see an overall site conversion rate of 3% or better. The average conversion rate for ecommerce websites right now is just under that at 2.86%.  

But if your current conversion rate on the site as a whole is 3%, what does it do to your revenues when you increase it by only half a percent? Let’s do the math. 

If your site is making $1,000 per day with a conversion rate of only 3% and you increase your conversion rate to 4%, you’ve added 30% more to your site’s revenues. Double that to 6% and you’ve doubled your site’s income to $2,000 per day. 

But even if you only increase it by 0.5%, you’ve added 15% more income ($150) to your site’s daily income. It may not sound like a lot of money, but it is. Small changes to your pages can create big gains in your conversion rate. 

Increase time on page

This sounds like it’s a little less important than conversion rate, but the longer your visitor spends on the page, the more likely they are to convert. This is also very indicative of whether their experience on your page is an engaging one. If they aren’t interested in your content or if it’s not laid out in a way that makes sense, they’re going to leave. This brings us to bounce rate. 

Reduce bounce rate

Bounce rate is the percentage of people who land on your page from any source (such as Google organic search results) and leave without doing anything. If they came from Google and hit their browser’s “back” button, Google will consider that a negative interaction with your site and your organic ranking will drop if you get enough of them. 

The average bounce rate on ecommerce sites is between 20 and 45% according to HotJar. If your bounce rate isn’t within this range, your content isn’t laid out effectively enough to get your visitors to stay on the page. If you want to know out what’s wrong, split testing is a great way to find out. 

Minimize design change risk

Are you considering changing your website’s design? Consider split testing a small portion of your site’s traffic and see how well it converts before making the change sitewide. We can look at this the same way we did increasing conversion rate above, just in reverse. If you make a significant design change on your website and it decreases your conversion rate by 1%, how much money have you lost?

It’s much better to take a tiny bit of traffic and test how they react to the new design proposed before you run it sitewide. Your bank account will thank you later. 

Who should do split testing?

Everyone with a business should do split testing!

Even if you don’t have a website and you’re doing traditional marketing with flyers and such, you can still split test your marketing materials to see what gets the best reaction. Putting up flyers in town? Use different phone numbers for each area so you know where your flyers are working. If you’re sending out emails, use different tracking links based on what segment of your list your emails are going to. 

There are many different ways to do split testing besides just on your website. Be creative!

What can you do split testing on?

As we just mentioned above, you can do split testing on almost any aspect of your marketing to determine what is getting you the best results. 

But when we’re talking about websites specifically, there are hundreds of things that you can test at any given time. Here are a few ideas.

• Headlines
• Add to cart color
• Add to cart location on page
• Product image
• Landing page image
• Call to action
• Page speed

This is a short list of example items that you can split test on your website that can increase your conversion rate. What works great on one page may not work great on another, so always be testing!

What are the types of split tests?

If you’ve recently learned about split testing, then you may not know that there are at least 3 different types of split tests that you can do: A/B testing, which is the most common type, A/B/n testing, and finally, multivariate testing. 

We’ll explore each of these below. 

A/B testing

A/B testing is what most people are familiar with when they think of “split testing”. It involves testing one page against another with only one thing on it that has been changed. This could be something as simple as headline text or changing the main product image. It’s important that only one element on a page be tested with one variation for it to be considered A/B testing. 

A/B/n testing

A/B/n testing is an extension of A/B testing where only one element is changed, but that element may be changed in multiple different ways. 

For example, if you want to test the headline element, creating 4 different variations and splitting the traffic between 4 different versions of that page would be A/B/n testing. You can test as many variations of an element as you want this way as long as you divide the traffic up evenly between them all. 

Multivariate testing

Multivariate testing is testing multiple elements of a page at a time and allowing software like Visual Website Optimizer or Optimizely to create the different page variations for you. For example, if you want to test 4 different elements on a page and have 4 different variations for each one, that’s a total of 64 different pages that need to be generated for your test. 

When you want to quickly test a lot of variations of different elements, multivariate testing is the way to go. 

How to perform a split test

Performing a split test isn’t as difficult as it may sound, even if you decide to go with a multivariate test. But, you’ll need to use software to do it effectively. 

Decide what split testing software to use

There are various split testing softwares available and while the basic principles may be the same for each one, the setup, user interface, and the features for each can be very different. 

Here is a short list of quality split testing software below.

• Optimizely
• Visual Website Optimizer 
• Google Optimize (Free, but Google Optimize 360 is their paid version)
• Adobe Target

Pick your page

Choosing what page to do your split test on is something that you should consider carefully. Most of the time, you want to test pages that are crucial to the conversion process, such as product pages, landing pages, appointment booking pages, etc. 

Any page you choose should get a decent amount of traffic. If it doesn’t, then your split test is going to take longer to determine a true winner. 

Decide on what element to test

There are many different elements on a page that you can test. For example, if you have a page where potential customers can book an appointment, maybe you want to try a new booking form to see if it’s easier to fill out. With split testing, it’s easy to change out the old booking form for a new one to see if your appointments increase. 

You can test nearly anything on a page to see if it makes a difference. 

Decide which type of split test to use

Choose whether you want your split test to be an A/B, an A/B/n, or a multivariate test. For something like an appointment booking form, you will likely want to choose either an A/B or even an A/B/n if you have multiple different booking forms that you want to try out. 

Decide how much traffic to give each variation

If you only have 2 variations of a page, then it would make sense to have the traffic split 50/50 between them, right? Well, maybe, but if it’s a high-converting page already, you could be potentially throwing away half of your revenue if your alternate version is a near-total failure. Who wants to give away half their income for any period of time?

If your page is already getting traffic and you don’t want to risk a severe income drop, you can always split the traffic differently. Maybe split it 90% to your original page and only 10% to your new version. This helps minimize your risk, but the test will take longer to decide if your challenger is the better page. 

Implement the test

After you’ve done all this, it’s time to turn your test on. You will want to let your test run for at least a week or two, depending on how much traffic your test pages get. If it’s split 90/10 as we stated earlier, the test will take longer. 

Wait

Now that you’ve turned your test on, the hard part is waiting.

You’ll want to check the test results every single day, maybe even more than once a day to see what page is winning. Resist checking it more than once daily, though, because you will be tempted to end the test early if it looks like one is a clear winner. 

Measure your results

After a couple of weeks, you should have enough data to judge which page is the winner. End it too soon and your data may not be correct, which will mean the test was wasted.

Once you’ve decided which version of the page will remain, make the change (if applicable) and wait before split testing again to see if you were correct.  

Split testing mistakes

Even though split testing is valuable, it can be detrimental to your business if you’re doing it too much or simply doing it wrong. Here are a few mistakes you may make when you split testing your pages. Try to avoid these!

Split testing too many things

If you’ve decided to test a particular page on your site, it’s always best to do one item at a time to see if that makes a difference. Maybe you should move the add to cart button closer to the product image or maybe that button is below the page's fold and should be moved above it. Whatever you decide to test, test just one thing. If you try to test more than one thing at a time, you won’t really know what change made the difference. 

Split testing more than one thing at a time is called “multivariate testing” and this is a more difficult thing to do effectively. You’ll need software such as Optimizely or Visual Website Optimizer. 

Multivariate testing can be more efficient than normal A/B testing, because the software will take the changes you’ve made and generate pages to test based off of them. For example, if you make 4 changes to a page, the software will create 16 different versions of that page using all possible combinations of those changes. 

Split testing during the holidays

The holidays aren’t when you should be split testing, because you may get skewed results. 

Does split testing affect SEO?

If you do split testing incorrectly, it can affect your SEO negatively. Done correctly, however, it won’t affect it at all.

Google encourages you to perform split testing on your website to the point that they have created their own software specifically for this purpose called Google Optimize. 

What Google doesn’t like is a practice called “cloaking”, which is where you deliberately serve Googlebot a different version of the page than you would show your visitors to trick them into ranking your page higher in search results. When you’re using your website optimization software correctly, it doesn’t fall under Google’s definition of cloaking. 

How to do split testing for SEO

Split testing with your content's SEO in mind isn’t as difficult as it might sound. You’re not likely to be split testing anything on your pages that are ranking in Google, anyway. Most split testing is done on pages where conversions are likely to happen, whether that’s sales, email sign-ups, etc. 

But on the chance that you are split testing a page that’s ranking in Google, there are a few things you don’t want to touch without thinking critically and those are the following:

• SEO title
• Page title/H1 tag
• Subheadings (H2, H3, etc)
• The text of anything that appears in a Google featured snippet or in “people also ask”
• Images ranking in Google Images

Changing these items can negatively affect your SEO, so you want to avoid doing that unless you’re willing to risk rankings. If you're careful, you can create more than one version of your SEO title that is friendly to Google’s algorithm but be aware that your position in search may flux if you test it. 

Conclusion

We hope we’ve explained how split tests work and why you should be concerned with performing them on your website. Why focus on getting more traffic when you haven’t maximized the potential of the traffic you already have? More traffic is great, but it’s not the end all to increasing your revenues.

Copy.ai has a variety of free tools that you can use to help make split testing your content easier, including our Product Description Generator, Sentence Rewriter, and Call To Action Generator. Or you can sign up for our 7 day free trial today (no credit card needed) and check out all of our other tools.

‍