Incrementality: Are You Measuring What Marketing Causes?

Incrementality measures the additional outcomes that would not have occurred without a specific marketing activity. It answers one question that most attribution models never actually ask: would this conversion have happened anyway? If the answer is yes, the credit you’re assigning to that channel or campaign is largely fictional.

That distinction, between outcomes your marketing caused and outcomes that were already going to happen, is where most measurement frameworks quietly fall apart. And the gap between the two is almost always larger than marketers expect.

I spent years running agencies and sitting in client meetings where attribution dashboards were treated as ground truth. Tidy charts. Channel contributions neatly stacked. Everyone confident the numbers told the full story. The problem was that those dashboards were measuring correlation and calling it causation. The question of what marketing actually caused was rarely asked, because the answer was harder to find and occasionally inconvenient to hear.

Why Attribution Models Miss the Point

Attribution models, whether last-click, first-click, linear, or data-driven, share a common flaw: they divide credit among touchpoints that were present in the conversion path. They do not ask whether any of those touchpoints changed the outcome. That is a fundamentally different question, and it is the one that actually matters when you are deciding where to spend money.

Think about a customer who searches for your brand by name, clicks a paid search ad, and converts. The paid search channel claims the conversion. But that customer already knew your brand, typed your name into a search engine, and was almost certainly going to find you regardless of whether you had a paid ad sitting above the organic result. The incremental value of that click is close to zero. The attribution credit is 100%.

This is not a niche edge case. Branded search and retargeting are probably the two most systematically over-credited channels in digital marketing, because both tend to intercept intent that already existed rather than create new intent. If you want to understand how attribution theory actually works and where its limits sit, it is worth being clear on this before you build any measurement framework around it.

I judged the Effie Awards for several years. The entries that impressed me most were not the ones with the biggest reach numbers or the most impressive ROAS figures. They were the ones where the team could articulate what the marketing actually changed. Sales uplift versus a control region. Conversion rate shift versus a holdout group. Evidence of causation, not just correlation. Those entries were a minority, which tells you something about how the industry measures itself by default.

If you are working with marketing analytics rather than web analytics, the distinction matters even more. Web analytics tells you what happened on your site. Marketing analytics should tell you what your marketing caused. Those are related but not the same thing.

What Incrementality Actually Measures

At its core, incrementality is about counterfactual thinking. What would have happened if this marketing activity had not occurred? The gap between what did happen and what would have happened is the incremental effect.

Measuring that gap requires a comparison group: people who were not exposed to the campaign, or a region where the campaign did not run. The difference in outcomes between the exposed group and the unexposed group is your incremental lift.

There are three main approaches in practice:

Holdout Testing

A holdout test withholds a campaign from a randomly selected segment of your audience. You run the campaign to 80% of your target audience, hold back 20%, and compare conversion rates. The difference is your incremental lift. This is the cleanest method when you can execute it properly, but it requires a large enough audience to produce statistically meaningful results, and it means deliberately not advertising to a portion of your potential customers, which creates internal friction.

Geo-Lift Experiments

Geo-lift tests run a campaign in selected geographic markets while keeping comparable markets dark. The control markets act as a proxy for what would have happened in the test markets without the campaign. This approach works well for TV, out-of-home, and digital campaigns with strong geographic signals. The challenge is finding genuinely comparable markets, which is harder than it sounds when you factor in local seasonality, competitive activity, and distribution differences.

Media Mix Modelling

Media mix modelling (MMM) uses historical data to estimate the contribution of each channel to overall sales, controlling for external factors like seasonality, pricing, and competitor activity. It does not require a holdout group, which makes it useful for channels where withholding exposure is impractical, such as TV or out-of-home. The trade-off is that MMM relies on statistical inference rather than direct observation, so the outputs are estimates with confidence intervals, not precise measurements. They are useful, but they are not the same as a controlled experiment.

Each method has a different cost structure, a different level of precision, and a different set of assumptions. Understanding what you are measuring, and what assumptions you are accepting, is part of the work. The data-driven marketing approach only holds up when you are honest about the limitations of your data sources.

If you are running affiliate programmes, the same logic applies with particular force. Affiliate channels are especially prone to claiming credit for conversions that were already in progress. The mechanics of measuring affiliate marketing incrementality are worth understanding separately, because the incentive structures in affiliate marketing create systematic over-reporting that holdout testing tends to expose quickly.

The Channels Where Incrementality Testing Matters Most

Not every channel needs a rigorous incrementality test. Some channels are clearly driving new demand. Others are more ambiguous. The ones worth testing first are those where you suspect you might be paying for outcomes that were already going to happen.

Branded paid search is the obvious starting point. If your organic listing is already ranking first and your brand awareness is high, the incremental value of a paid ad above it is genuinely unclear. I have seen clients pause branded search for six weeks and watch conversions hold almost flat. Not always, and not for every brand, but often enough that the assumption of high incrementality deserves scrutiny rather than acceptance.

Retargeting sits in similar territory. Retargeting campaigns by definition target people who have already visited your site and demonstrated intent. That intent existed before the ad appeared. The retargeting ad may accelerate conversion, or it may simply be present when conversion happens anyway. Holdout testing on retargeting audiences regularly produces lower incremental lift than the raw ROAS figures suggest.

Social prospecting and display advertising tend to show more genuine incrementality because they reach audiences who were not already in-market. Upper-funnel activity that creates awareness and intent, rather than capturing it, is where the incremental story is usually stronger.

Email marketing is a more complicated case. Sending an email to an existing customer who was already planning to repurchase shows low incrementality. Sending a reactivation email to a lapsed customer who had not engaged in twelve months shows higher incrementality. The channel is the same. The audience segment determines the incremental value. Email marketing reporting that does not segment by audience behaviour is missing the most important part of the story.

Newer formats create new measurement challenges. As AI-generated content and synthetic media become more common in marketing, the question of what those formats actually cause, rather than what they correlate with, becomes more pressing. The same incrementality logic applies when you are trying to measure the effectiveness of AI avatars in marketing: reach and engagement metrics are not the same as evidence of incremental impact.

The Practical Barriers to Running Incrementality Tests

Incrementality testing is not technically difficult. The concepts are straightforward. The barriers are mostly organisational and commercial.

The first barrier is the holdout problem. Deliberately not advertising to a segment of your audience feels counterintuitive, especially when you are under pressure to hit short-term targets. The argument that you need to withhold budget from potential customers to understand whether your advertising works is a hard sell in most organisations. I have had this conversation more times than I can count, and the answer is usually to run smaller holdout tests in lower-stakes periods rather than trying to get sign-off on a full-scale experiment during peak season.

The second barrier is statistical significance. Incrementality tests require enough volume to produce reliable results. If your monthly conversion volume is in the hundreds rather than the thousands, a holdout test may not give you statistically meaningful output. Geo-lift tests can help here, because you are comparing markets rather than individual users, but they introduce their own comparability challenges.

The third barrier is that incrementality findings are sometimes unwelcome. I ran a geo-lift test for a client several years ago that showed their display retargeting was generating almost no incremental conversions. The channel looked excellent on a last-click basis. The holdout data told a different story. The right response was to reallocate that budget toward channels with demonstrated incrementality. The actual response was a lengthy internal debate about methodology. When measurement produces inconvenient results, the methodology tends to come under scrutiny in ways it does not when the results are favourable.

The tools available for this kind of analysis have improved significantly. GA4 has expanded its experimentation capabilities, though it has its own constraints worth understanding. A working knowledge of GA4 custom reports helps when you are trying to isolate audience segments for holdout analysis. It is also worth being clear on what data Google Analytics goals are unable to track, because there are meaningful gaps in the default setup that affect how you interpret any conversion data you are using as the basis for an incrementality test.

Incrementality and the Broader Measurement Picture

Incrementality testing does not replace your other measurement frameworks. It sits alongside them and adds a layer of causal validation that attribution models cannot provide on their own.

Think of it this way. Attribution tells you where conversions were present in the funnel. Incrementality tells you where conversions were caused. You need both perspectives to make good budget decisions. Attribution without incrementality testing leads to over-investment in channels that are good at being present at the moment of conversion but are not actually driving it. Incrementality without attribution gives you causal evidence but no granularity about which specific touchpoints within a channel are performing.

For inbound programmes specifically, the question of what content and which channels are genuinely driving new pipeline, rather than simply being visited by people who were already going to convert, is central to understanding inbound marketing ROI honestly. Content that ranks well and attracts high-intent organic traffic may show strong conversion rates, but some of that conversion would have happened through direct or branded search anyway. Incrementality thinking applied to content performance changes how you evaluate what to produce and where to invest in distribution.

The same principle applies to emerging channels. As generative search changes how people find information and brands, understanding what is genuinely incremental versus what is channel-shifting from existing traffic sources becomes more important. The measurement approaches for generative engine optimisation campaigns are still developing, but the incrementality question is the right one to be asking from the start, before you build reporting frameworks that assume causation.

Early in my career, I was told that if a metric looked good, it probably was good. Twenty years of managing P&Ls and watching agencies lose clients because their reported numbers did not match business outcomes has given me a different view. A metric is useful in context. On its own, it is just a number. Marketing metrics only earn their place in a dashboard when you can connect them to decisions that affect commercial outcomes. Incrementality is the test that separates metrics that matter from metrics that merely look good.

Building Incrementality Into Your Measurement Routine

The practical starting point is not a full measurement overhaul. It is picking one channel where you suspect you might be over-crediting performance and designing a simple holdout test.

Choose a channel with enough volume to produce meaningful results. Define a clean holdout group, ideally randomised by user ID or device ID rather than by geography if you want to avoid geo-confounds. Run the test for long enough to capture a complete purchase cycle, not just a few days. Compare conversion rates between exposed and holdout groups, and calculate the incremental lift as a percentage of total conversions attributed to that channel.

If the incremental lift is significantly lower than the attribution credit, you have found budget that can be redeployed more effectively. If the incremental lift is roughly in line with attribution, you have validated the channel. Either outcome is useful. The worst outcome is not running the test and continuing to make budget decisions based on attribution data that may be systematically misleading.

For teams using A/B testing infrastructure, the methodology overlaps enough that you can often run incrementality tests within existing frameworks. The A/B testing capabilities in GA4 are a reasonable starting point for audience-level holdout experiments, though they have limitations for more complex geo-lift designs.

Once you have run one incrementality test and found something actionable, the methodology tends to spread. Teams that have seen the gap between attributed performance and incremental performance start asking the question more routinely. That is the shift worth making: from a culture that accepts attribution data as truth to one that treats it as a starting point for a harder question.

I built my first website by teaching myself to code after my MD refused the budget to hire someone. That experience taught me something that has been more useful than any marketing tool I have used since: if you want to understand whether something works, you have to get close enough to the mechanics to test it yourself. Incrementality is the same principle applied to measurement. You cannot outsource the question of whether your marketing is actually causing outcomes. You have to build the tests, look at the data, and be willing to act on what you find.

If you are building or refining a measurement framework, the wider marketing analytics resources on this site cover the full stack, from attribution and GA4 configuration to channel-specific measurement approaches. Incrementality sits at the top of that stack, as the validation layer that tells you whether the rest of your measurement is pointing in the right direction.

The goal is not perfect measurement. It is honest approximation. Knowing that your branded search is delivering 20% incremental lift rather than 100% does not require a controlled experiment run to academic standards. It requires a reasonable test, a willingness to look at the results, and the commercial discipline to act on them. Most organisations are capable of that. Fewer actually do it.

About the Author

Keith Lacy is a marketing strategist and former agency CEO with 20+ years of experience across agency leadership, performance marketing, and commercial strategy. He writes The Marketing Juice to cut through the noise and share what works.

Frequently Asked Questions