A/B testing for small business websites

Short answer

A/B testing on a small business website only works when a page gets enough conversions to detect a real difference, typically at least a few hundred conversions per version. Most SME pages do not. Below that level, make bold changes based on evidence, compare before and after over several weeks, and use user testing to find problems.

• Rule of thumb: A/B tests need hundreds of conversions per version to be reliable.
• Most SME pages lack that volume, so small tweaks cannot be measured.
• Test big changes first: headline, offer, proof, form length.
• Low traffic? Use before-and-after comparisons and five-person user tests instead.

What is A/B testing?

A/B testing is a method of comparing two versions of a web page by showing each to a random share of visitors at the same time, then measuring which version produces more of a defined action, such as enquiries or sales.

Running both versions at once is what makes it a test. It removes the effect of seasons, campaigns and other changes that would confuse a simple before-and-after comparison.

Why do A/B tests mislead small businesses?

Small tests mislead because random variation looks like a result. If each version receives 15 enquiries, a difference of three either way is well within chance. Stopping the test the moment one version pulls ahead makes the problem worse.

The outcome is decisions based on noise: a button colour changed because of a result that would have reversed next month, and a belief that testing works when nothing was actually learned.

• Too few conversions per version.
• Stopping as soon as one version leads.
• Testing tiny changes that could never show a measurable effect.
• Measuring clicks rather than enquiries or sales.

How much traffic do you need for A/B testing?

As a rough guide, reliable A/B tests need at least a few hundred conversions per version, and more when the expected difference is small. A page converting at 3% needs roughly 10,000 visits per version to reach 300 conversions each.

Use a sample size calculator before starting, decide the duration in advance, run for at least two full weeks to cover weekly patterns, and do not stop early. If the numbers show you would need six months, do not run the test.

• Website conversion rate benchmarks for UK businesses

What should you test first?

Test the changes most likely to produce a large difference, because large differences need less traffic to detect.

• The headline and value proposition.
• The offer: free survey versus fixed-price quote, for example.
• Showing a price range versus no price.
• Form length: four fields versus eight.
• Proof placement: reviews under the headline versus lower down.

• Write a website value proposition visitors understand in five seconds

What should you do if you do not have enough traffic?

Use sequential changes with discipline. Make one meaningful change, record the date, and compare enquiry rate over the four to six weeks before and after, checking that traffic mix has not changed. It is less rigorous than a test but far better than guessing.

Combine it with qualitative evidence. Five-person user tests, session recordings and the questions customers ask on calls reveal problems that no A/B test at SME scale could detect.

• User testing for small business websites

Does A/B testing affect SEO?

Not if done properly. Google's guidance on website testing says not to cloak, to use rel=canonical on variant URLs pointing to the original, to use temporary 302 redirects rather than permanent ones when redirecting to variants, and to run tests only as long as necessary.

Once a test ends, implement the winning version on the original URL and remove the testing code, which also avoids leaving extra scripts slowing the page.

When is testing a waste of time?

Testing is a waste when the fundamentals are broken. If the form fails on some phones, the headline is vague and there is no proof on the page, fix those first. They are known problems, not hypotheses.

The trade-off is speed against certainty. For most small businesses, making evidence-based fixes quickly produces more enquiries than slowly testing small variations.

What does this look like in practice?

A pattern in small e-commerce and service sites: a testing tool installed, a button colour test running for weeks on a page with a few enquiries a day, and a winner declared from a handful of conversions.

Removing the testing script, fixing the known issues and then testing only headline and offer changes on the highest-traffic page gives testing a fair chance to produce something real.

• What makes a business website effective
• See the three problems we fix for UK businesses

Testing checklist

Answer these before starting any test.

• Fix known problems first.
• Choose one page with the most conversions.
• Calculate the sample size needed.
• Test a big change, not a small tweak.
• Fix the duration in advance, at least two weeks.
• Measure enquiries or sales, not clicks.
• Follow Google's testing guidance for SEO.

Next step

If you want to improve conversion without guessing, we will review your traffic with you in 30 minutes and tell you whether to test or to fix.