What is A/B testing for websites?

If you are building a website, you have probably heard about A/B Testing.  What is it??? It may sound fancy, or like something happening in elementary school classrooms.  You may have even guessed that it has to do with the medical field. None of that is the case.  A/B testing is a type of research that helps you to develop the very best website that you can.

What is A/B Testing?

In short, it is a way to compare two web pages to see which is best. You put out two webpages and have them randomly viewed by visitors.  Whichever site creates the best conversion ratings, wins!  Pretty simple, really.

Reasons to Run A/B Tests

  1. All websites exist to be viewed and, therefore, it makes sense that you would want people to view, click and keep coming back. A/B testing can help you to do that.
  2. It’s simple and inexpensive
  3. You might be surprised how just the tiniest tweaks to a page can create a huge difference in conversions.

Basic Process

The basic process may remind you of a 5th grade science project.  In general, it is done the same way as one would do any experiment.

  1. Question – Why aren’t my visitors becoming customers? Why aren’t those sales converting?
  2. Research – Take some time to think about it and look through your data.
  3. Hypothesis – Make an educated guess about the element you think may need improvement. For example, is it the placement of links or buttons?  Is there a specific bit of content that may need to be tweaked?
  4. Calculate the # of visitors required to run the test. This can be done with an online calculator.
  5. Test your hypothesis by creating two versions of the page. Keep one the same and make the hypothesized changes to the other.  See what happens.
  6. Analyze your data.
  7. Make conclusions.
  8. Share your findings.

Common Elements Tested

While you will want to specifically analyze your website, and testing elements will be different than others depending on your design, there are some commonly tested elements within a page. These include the following:

  1. Headlines
  2. Calls to action
  3. Graphics
  4. Content, copy or product descriptions

A/B Testing Tips

When beginning testing, keep these following tips in mind.

No room for assumptions

Now that A/B testing is available, there is no reason to make changes to a website based on assumptions. If you have ideas about elements that can be improved upon on your website, use them as your hypotheses for A/B tests.

Quality of Tests

In order to make sure your test data is reliable, there are a few metrics to check. Make sure that the sample size is large enough, the conversion range is low and statistical significance is high.  The sample size needs to be at least 100 conversions, as the more data points provided, the higher the accuracy of results. The conversion range should be less than 1% and the statistical significance to look for is 99%.  In addition, the time of testing can skew results. For example, testing done at Christmas time may have very different results than tests during summer.  So, keep these things in mind.

Test multiple variables

Test as many possible variables as you can.  For example, you may find a small correlation between the sales copy and conversions and think “I found the issue!”   But, if you tested color scheme, graphics or a call to action button, you might see that their impact is far greater than the effect of the sales copy. Be sure you keep a wide view of all elements and gradually run tests using different variables. Testing should not be a once a year event, or even a once a month one. It should be an ongoing practice to continuously optimize your website.

In short, A/B Testing is the process of pinpointing issues with a webpage and then comparing different versions of the page until you see better results.  It is not difficult or expensive, but does require time and effort. However, the results can bring you more traffic and higher conversion rates, which means more money.  Therefore, It is beneficial to take your time, perform adequate research, and implement your findings.

Share on facebook
Share on email
Share on twitter
Share on linkedin
Share on pocket
Colby Albarado

Colby Albarado

Colby is a fullstack developer

Back to Home Page