Incrementality testing: is social media really driving sales?

Key takeaways

An incrementality test measures the sales social media causes, not just the sales it gets credit for, by comparing a group that saw your activity with a similar group that did not. Small businesses can run simple versions: pause adverts in some postcodes, or switch a campaign off for a few weeks, and compare enquiries.

• Attribution shows credit; incrementality shows cause.
• Postcode holdouts suit local businesses with steady demand.
• Run tests for 4 to 8 weeks to smooth weekly noise.
• Meta's lift studies exist but usually need larger budgets.

What is incrementality?

Incrementality is the extra sales or enquiries that happen because of a marketing activity, compared with what would have happened without it. An advert can be credited with sales from people who would have bought anyway. An incrementality test removes the activity for a comparable group to see the true difference.

• How UK businesses can turn social media followers into customers
• Marketing attribution for small businesses, kept simple

Why does attribution overstate some results?

Attribution overstates results when adverts reach people already about to buy, such as retargeting existing customers who would have reordered. Platform reports credit the advert because it was seen before the purchase. Incrementality testing asks a harder question: would the sale have happened anyway?

• Social media retargeting: win back followers who nearly bought

How can a small business run a holdout test?

A local business can run a postcode holdout: split its area into two groups of postcodes with similar past sales, run adverts in one and not the other for four to eight weeks, then compare enquiries per household or per previous sales. The difference, beyond normal variation, is the incremental effect.

• 1. Choose matched postcode groups using last year's sales.
• 2. Run activity in one group only.
• 3. Keep everything else the same.
• 4. Compare enquiries after 4 to 8 weeks.

• How to test an advertising channel in six weeks

What about on-off tests?

An on-off test pauses a campaign for a few weeks and compares enquiries with the weeks before and after. It is simple but less reliable, because seasons, weather and news change demand too. Use it for activity with a clear expected effect, and avoid it at peak times.

What are Meta's lift studies?

Meta offers conversion lift studies, which randomly hold back a share of the audience from seeing adverts and compare their conversions. They are more rigorous than home-made tests, but typically need enough spend and conversions to be meaningful, and availability depends on the account. Ask Meta or your agency whether one is possible.

When is incrementality testing not worth it?

It is not worth it when spend is small, when conversions are too few to compare, or when the business cannot afford to pause activity. A firm with ten enquiries a month cannot detect a modest effect. Asking every customer how they found you is a better first step.

• 'How did you hear about us?' Asking the question properly

What does this look like in practice?

A pattern we see: retargeting adverts to existing customers report high returns, and the budget grows. A short holdout often shows many of those customers would have reordered anyway, and the budget does more reaching new buyers.

• See client work
• Social Media Marketing for UK businesses
• The follower conversion scorecard: find where buyers drop out

Incrementality checklist

• Decide the question: does this activity add sales?
• Choose a test: postcode holdout or on-off.
• Match the groups on past sales.
• Run for four to eight weeks.
• Change nothing else during the test.
• Compare enquiries and sales, not clicks.

Next step

Pick your largest social media spend and ask whether its sales would have happened anyway. If you are unsure, it is the one to test.