How Accurate is Marketing Mix Modelling?

A well-built MMM is typically accurate within 10 to 20 percent for each channel's contribution estimate. That level of accuracy is enough to make better budget decisions. No model is perfect, and understanding where the uncertainty sits helps you use the results correctly.

Every CMO who sees an MMM readout for the first time asks the same question: how much should I trust these numbers? It is a fair question. Knowing the realistic limits of the method stops you from over-relying on a single number and helps you frame the results correctly for your CFO.

What accuracy means in an MMM context

Modellers measure accuracy in two ways. The first is model fit, usually expressed as R-squared, which tells you how much of the variation in your sales the model explains. A good MMM typically has an R-squared above 0.8, meaning it accounts for at least 80 percent of the week-to-week movement in your revenue.

The second measure is how well the model predicts sales in a period it was not trained on. Modellers hold back a few months of data, build the model on the rest, and then check whether it would have predicted the held-out period accurately. This out-of-sample test is a more honest measure of real-world accuracy.

Factors that improve accuracy

The length and quality of your data is the most important driver. Models built on three or more years of weekly data, with complete spend information across all channels, tend to produce tighter estimates than models built on patchy or short datasets.

• Longer data history: more variation in spend gives the model more to learn from
• Complete channel coverage: missing a channel causes its effect to be absorbed by others
• Consistent spend patterns: large sudden changes in media strategy make it harder to isolate effects
• External calibration: using results from geo tests or holdout experiments to anchor the model
• Accurate prior beliefs in Bayesian models: well-informed priors reduce uncertainty in the estimates

Where models tend to be less reliable

Channels with low and stable spend are hard to model accurately. If you spent roughly the same amount on radio every week for two years, the model cannot easily separate the radio effect from the general trend in your sales. You need variation in spend for the model to detect an effect.

Very new channels are also problematic. If you started running connected TV ads six months ago, there is not enough data for the model to produce a reliable coefficient. Most modellers will flag this and either exclude the channel or treat its estimate with a wide confidence interval.

A confidence interval tells you the range within which the true value is likely to fall. If your TV ROI estimate has a 90 percent confidence interval of 1.5 to 3.5, that means you can be 90 percent confident the real ROI sits somewhere in that range. Wide intervals are not a failure; they are honest uncertainty.

Validation: how to check if your model is telling the truth

The best validation method is a geo test (also called a geo holdout test). You take a group of regions, pause all advertising in them for a period, and compare sales against control regions where advertising continued. The observed lift matches closely with what the model predicts if the model is well-calibrated.

Not every brand can run geo tests. The alternative is sense-checking: reviewing whether the channel coefficients match your intuition and experience, whether the baseline (the sales you would make with no advertising) looks reasonable, and whether the model reacts correctly to known events such as a major promotion.

Accuracy vs. usefulness

MMM does not need to be perfectly accurate to be useful. If it tells you that you are overspending on a channel that has a return on investment below 1.0, meaning it is losing money, that insight is actionable even if the exact figure is off by 15 percent. The model is a tool for better decisions, not a precise accounting of every pound.