A geo experiment splits your market into two groups of regions. One group sees your ads as normal, the other does not. You compare outcomes to find out how much of your revenue your advertising actually caused.
Platform dashboards tell you what they want to take credit for. That number is almost always too high. A geo experiment cuts through the noise by using real geography as your control.
What is a geo experiment?
A geo experiment, sometimes called a geo holdout test, divides your advertising markets into two buckets. The treatment group runs your campaign as planned. The holdout group has that campaign paused or suppressed. After the test period, you compare sales or conversions between the two groups.
Because geography is physical, people in the holdout regions genuinely cannot see the ads you turned off. This makes the comparison clean in a way that audience-level splits often are not.
Pick the right regions
The most important step happens before you launch the experiment. You need to choose holdout regions that behave like your treatment regions when there is no advertising difference between them. Analysts call this "parallel trends": the two groups move together in normal times, so any gap you see during the test is genuinely caused by the ads.
Match regions on historical sales patterns, population size, and seasonal cycles. Avoid regions where a local competitor is about to open a new store or where a major event will distort normal buying behaviour.
Decide what to turn off
You do not have to pause every channel at once. A geo experiment works best when it isolates one variable. If you want to know whether your paid social is incremental, suppress paid social in the holdout regions while leaving search, TV, and other channels running normally in both groups.
Mixing too many changes into one test makes the results impossible to interpret. Keep it simple and test one thing at a time.
Set the duration
Most geo experiments need at least three weeks to produce reliable results. Shorter tests are noisy because week-to-week sales swings can mask the true effect. If your product has a long purchase cycle, you may need six to eight weeks.
Avoid running a test over a major promotional period like Black Friday unless your specific question is about that period. Seasonal spikes in the treatment group will muddy the comparison.
Measure the lift
Once the test ends, calculate the revenue difference between the treatment and holdout groups, adjusted for their baseline share of your business. The percentage difference is your incremental lift. Multiply that by revenue to get the incremental sales your advertising caused.
- Incremental lift (%) = (Treatment revenue per capita - Holdout revenue per capita) / Holdout revenue per capita
- Incremental revenue = Lift % x total revenue in treatment regions during test
- Incremental ROAS = Incremental revenue / Spend in treatment regions
A common mistake is comparing raw revenue totals without accounting for the size difference between treatment and holdout regions. Always normalise by population or baseline sales share before drawing conclusions.
What to do with the results
If your incremental ROAS is above your target, the channel is working and you can consider scaling spend. If it is below, you have a real decision to make about whether to cut, reallocate, or restructure the creative and targeting before retesting.
Either outcome is valuable. A negative result is not a failure. It is information that your budget could work harder somewhere else.
How many regions do I need for a geo experiment?
As a general rule, aim for at least ten regions in each group. Fewer regions make it hard to detect a real signal above the natural noise of sales variation. If you operate in only a handful of markets, an audience-level holdout may be more practical.
Can I run a geo experiment on a single channel?
Yes, and that is usually the best approach. Isolating one channel gives you a clear answer about that channel's contribution. Running multi-channel tests simultaneously makes it very difficult to attribute lift to any single activity.
Do I need a data scientist to run a geo experiment?
Basic geo experiments can be set up and analysed in a spreadsheet if your data is clean. More rigorous analysis using synthetic control methods or Bayesian time-series models benefits from statistical support, but is not always necessary for a first test.