Incrementality Measurement: What’s Changed and Why It Matters

The Methods That Are Actually Being Used

The Methods That Are Actually Being Used

There are three methods that dominate practical incrementality measurement right now. Each has a different trade-off between precision, cost, and operational complexity.

Geo-based holdout tests split markets geographically, run your campaign in some and not others, and compare outcomes. This is the most widely used method for mid-to-large advertisers because it does not require platform cooperation and it produces results that are relatively easy to explain to a CFO. The limitation is that geographic markets are not perfectly comparable, and the minimum detectable effect size requires either a long test window or significant spend. Running a two-week geo test on a £50,000 budget will not give you statistically meaningful results. That is a practical constraint most teams underestimate.

Ghost ad experiments (sometimes called conversion lift studies) show a control group a placeholder ad instead of your real ad and measure the difference in conversion rates. Meta, Google, and Pinterest all offer versions of this. The problem is that you are trusting the platform to administer the test, which creates an obvious conflict of interest. Platforms have a commercial incentive to show positive lift. That does not mean the results are wrong, but it means they warrant scrutiny. I always recommend running a platform-administered lift study alongside an independent geo test when the budget allows, and treating the comparison itself as data.

Marketing mix modelling (MMM) has had a significant revival, partly because it does not rely on user-level tracking data and therefore survives privacy changes better than most attribution approaches. Modern MMM is faster and cheaper than the traditional econometric models that required months of consultancy time. Lightweight MMM tools have made the methodology accessible to teams that would never have considered it five years ago. The trade-off is that MMM is inherently backward-looking and operates at an aggregate level, which makes it less useful for tactical optimisation and better suited to strategic budget allocation.

What the Recent Platform Changes Mean in Practice

The deprecation of third-party cookies in Chrome, the ongoing impact of Apple’s App Tracking Transparency, and the continued tightening of browser-level privacy controls have all reduced the quality of user-level attribution data. This is not news. But the downstream consequence for incrementality measurement is worth spelling out clearly.

When user-level signals degrade, platforms fill the gap with modelled data. Google’s Consent Mode fills in conversion gaps using modelling. Meta’s Advantage+ campaigns use algorithmic optimisation that makes it harder to isolate what is actually driving results. The reported numbers look clean. The underlying data is increasingly estimated. Understanding what data Google Analytics goals cannot track is a useful starting point for understanding where your measurement has blind spots before you run any incrementality test.

The practical implication is that modelled measurement is not a fallback for when proper measurement fails. It is now the primary measurement reality for most advertisers. The question is not whether you are using modelled data. You are. The question is whether you are being honest about the confidence intervals attached to that modelling, or whether you are presenting estimates as facts.

I have always believed that an honest approximation, presented as an approximation, is more useful than false precision. A CFO who understands that your incrementality estimate has a confidence range of plus or minus 20% can make a better decision than one who is handed a single number that implies certainty the underlying methodology cannot support.

Incrementality Testing Across Specific Channels

The incrementality conversation is not uniform across channels. Different channels have different baseline conversion rates, different test design requirements, and different levels of platform cooperation. A few channels worth addressing specifically:

Paid search branded terms are where incrementality testing tends to produce the most uncomfortable results. Branded search consistently shows lower incremental lift than almost any other channel, because a significant proportion of branded searchers would have converted through direct or organic anyway. I have seen branded search campaigns that, on incrementality testing, showed true incremental ROAS of less than one. The campaign was losing money on a net basis. It had been running for three years. Nobody had questioned it because the attributed ROAS looked strong.

Affiliate marketing is a channel where incrementality testing is particularly important and particularly underused. The affiliate model creates strong incentives for last-click attribution gaming, and the gap between reported and incremental performance can be significant. There is a detailed breakdown of how to measure affiliate marketing incrementality that covers the specific methodological challenges, including how to handle coupon sites and loyalty publishers that tend to over-claim credit.

Connected TV and streaming audio are channels where incrementality testing is genuinely difficult because the conversion pathway is long and the targeting data is imprecise. Geo-based holdout tests are usually the most practical approach, but the minimum spend required to detect meaningful lift is high. Many mid-market advertisers are running CTV budgets that are too small to test incrementally with any statistical confidence.

AI-driven channels and formats add another layer of complexity. When you are running campaigns through AI-optimised delivery systems, the black-box nature of the targeting makes it harder to design clean experiments. The question of how to measure the effectiveness of AI avatars in marketing illustrates how new format types create measurement gaps that standard incrementality frameworks were not designed to handle.

Attribution Theory and Incrementality: Where They Intersect

Incrementality measurement and attribution modelling are related but distinct. Attribution attempts to assign credit for conversions that happened. Incrementality attempts to identify which marketing activity caused conversions that would not otherwise have happened. These are different questions, and conflating them leads to bad decisions.

The most common mistake I see is teams running data-driven attribution in GA4, seeing a more “fair” distribution of credit across channels, and concluding that they have solved their measurement problem. Data-driven attribution is better than last-click. It is not the same as incrementality measurement. A channel can receive a large share of attributed credit in a data-driven model and still have low or negative incremental value. Understanding attribution theory in marketing provides the conceptual foundation for understanding why these two approaches answer different questions and why you need both.

Forrester’s analysis of whether marketing measurement is undermining the buyer’s experience makes a related point: measurement frameworks that are optimised for channel efficiency can inadvertently penalise the channels that do the hardest work in creating demand, because that work is difficult to attribute directly to conversions. Incrementality testing, done properly, is one of the few methods that can surface this problem.

The Organisational Resistance Problem

The Organisational Resistance Problem

There is a reason incrementality testing has been slow to become standard practice, and it is not technical. It is political. Incrementality testing tends to reduce the apparent value of marketing activity. That is not because marketing is ineffective. It is because most attribution systems overstate effectiveness, and incrementality testing corrects for that overstatement.

When I ran agencies, the conversations around incrementality were rarely about methodology. They were about who owned the narrative. A channel manager whose budget depended on reported ROAS had no incentive to run a test that might show lower incremental ROAS. A platform account manager had even less incentive. The resistance was not irrational from their perspective. It was rational self-interest operating against the client’s commercial interest.

The way through this is to position incrementality testing as a tool for protecting budget, not cutting it. When a channel tests well on incrementality, that result is defensible in a way that attributed ROAS is not. A channel with strong incremental lift is genuinely hard to cut. A channel with strong attributed ROAS but untested incrementality is always vulnerable to the question: “But would those customers have bought anyway?”

Measuring inbound activity presents similar challenges. Inbound marketing ROI is notoriously difficult to isolate because the conversion pathways are long and multi-touch, and the same organisational resistance applies: teams that have built their case around attributed metrics are reluctant to run tests that might complicate the story.

Practical Steps for Running Your First Incrementality Test

If you have not run an incrementality test before, the most important thing is to start with a question that has commercial stakes. Do not test a channel because it is easy to test. Test the channel where the answer would change a budget decision.

For a geo-based holdout test, the basic design is: identify comparable geographic markets, assign them randomly to test and control groups, run your campaign in test markets only, and compare conversion rates (or revenue, or whatever outcome metric matters) between the two groups over the test period. The key variables are test duration (longer is more reliable), market comparability (more similar is better), and the size of the effect you are trying to detect (smaller effects require larger tests).

A/B testing infrastructure in GA4 can support some elements of experiment design, and Semrush’s guide to A/B testing in GA4 covers the technical setup in detail. Incrementality tests are not pure A/B tests, but the principles of clean experimental design, adequate sample sizes, and pre-defined success metrics apply to both.

UTM tracking discipline is also essential. If your campaign tracking is inconsistent, you cannot reliably compare test and control group performance. Semrush’s UTM tracking guide is worth reviewing before you design any test, because attribution gaps in your tracking will corrupt your results regardless of how clean your experimental design is.

Report the results honestly, including the confidence intervals. If the test was underpowered, say so. An inconclusive result from a well-designed test is more useful than a confident result from a poorly designed one, because at least you know what you do not know.

Where Incrementality Measurement Is Heading

The direction of travel is toward triangulation rather than any single measurement method. The most sophisticated measurement approaches combine MMM for strategic allocation, geo-based holdout tests for channel validation, and platform lift studies (with appropriate scepticism) for tactical optimisation. No single method answers all the questions. The goal is to triangulate toward an honest approximation of truth.

Generative AI is beginning to affect measurement in ways that are not yet fully understood. When AI-driven content and GEO (generative engine optimisation) campaigns generate conversions through pathways that do not show up in standard analytics, the incrementality question becomes harder to answer. Understanding how to measure the success of generative engine optimisation campaigns is part of the broader challenge of keeping measurement frameworks current as the channels themselves change.

The underlying principle, though, does not change. The question is always: would this have happened without the marketing? Everything else is detail.

If you want to go deeper on the analytics frameworks that sit behind incrementality measurement, the Marketing Analytics hub covers attribution, GA4, measurement strategy, and the emerging tools that are reshaping how performance is tracked and reported.

About the Author

Keith Lacy is a marketing strategist and former agency CEO with 20+ years of experience across agency leadership, performance marketing, and commercial strategy. He writes The Marketing Juice to cut through the noise and share what works.

Frequently Asked Questions

Incrementality measurement is the practice of isolating how much of your marketing-driven revenue would not have happened without that marketing. It answers the question most attribution models quietly avoid: would this customer have converted anyway? The field has moved significantly over the past few years, driven by the collapse of third-party cookies, the rise of privacy-first platforms, and a broader reckoning with how badly last-click attribution has misled marketing budgets for the better part of a decade.

The short version of where things stand: incrementality testing is no longer a luxury reserved for enterprise teams with data science departments. The tooling has matured, the methodologies have become more accessible, and the commercial pressure to justify spend more honestly has made it a practical priority for mid-market teams as well.

Why Incrementality Has Become Urgent

I spent years watching agencies present attribution reports that made every channel look essential. Paid search took credit for the conversion. Paid social took credit for the assist. Display took credit for the awareness. The customer, who had been googling the brand name for six months before clicking a branded search ad, was attributed to paid search. The media budget grew. The agency looked effective. Nobody asked the harder question.

That harder question is incrementality. And the reason it has become urgent is not philosophical. It is commercial. As signal quality has degraded across platforms, the gap between what platforms report and what is actually happening has widened. Meta’s reported ROAS and your actual incremental ROAS can be separated by a factor of two or three. That is not a rounding error. That is a budget allocation problem.

Forrester has written clearly about how marketing measurement and sales measurement need to be aligned but are not identical, and the incrementality debate sits right at that intersection. When sales teams look at revenue and marketing teams look at attributed conversions, they are often looking at different numbers for the same period. Incrementality testing is one of the few methods that can bridge that gap with something resembling evidence.

If you want broader context on how measurement frameworks fit together across the analytics stack, the Marketing Analytics hub covers the full landscape, from attribution theory to GA4 implementation to emerging measurement approaches.

The Methods That Are Actually Being Used

The Methods That Are Actually Being Used

There are three methods that dominate practical incrementality measurement right now. Each has a different trade-off between precision, cost, and operational complexity.

Geo-based holdout tests split markets geographically, run your campaign in some and not others, and compare outcomes. This is the most widely used method for mid-to-large advertisers because it does not require platform cooperation and it produces results that are relatively easy to explain to a CFO. The limitation is that geographic markets are not perfectly comparable, and the minimum detectable effect size requires either a long test window or significant spend. Running a two-week geo test on a £50,000 budget will not give you statistically meaningful results. That is a practical constraint most teams underestimate.

Ghost ad experiments (sometimes called conversion lift studies) show a control group a placeholder ad instead of your real ad and measure the difference in conversion rates. Meta, Google, and Pinterest all offer versions of this. The problem is that you are trusting the platform to administer the test, which creates an obvious conflict of interest. Platforms have a commercial incentive to show positive lift. That does not mean the results are wrong, but it means they warrant scrutiny. I always recommend running a platform-administered lift study alongside an independent geo test when the budget allows, and treating the comparison itself as data.

Marketing mix modelling (MMM) has had a significant revival, partly because it does not rely on user-level tracking data and therefore survives privacy changes better than most attribution approaches. Modern MMM is faster and cheaper than the traditional econometric models that required months of consultancy time. Lightweight MMM tools have made the methodology accessible to teams that would never have considered it five years ago. The trade-off is that MMM is inherently backward-looking and operates at an aggregate level, which makes it less useful for tactical optimisation and better suited to strategic budget allocation.

What the Recent Platform Changes Mean in Practice

The deprecation of third-party cookies in Chrome, the ongoing impact of Apple’s App Tracking Transparency, and the continued tightening of browser-level privacy controls have all reduced the quality of user-level attribution data. This is not news. But the downstream consequence for incrementality measurement is worth spelling out clearly.

When user-level signals degrade, platforms fill the gap with modelled data. Google’s Consent Mode fills in conversion gaps using modelling. Meta’s Advantage+ campaigns use algorithmic optimisation that makes it harder to isolate what is actually driving results. The reported numbers look clean. The underlying data is increasingly estimated. Understanding what data Google Analytics goals cannot track is a useful starting point for understanding where your measurement has blind spots before you run any incrementality test.

The practical implication is that modelled measurement is not a fallback for when proper measurement fails. It is now the primary measurement reality for most advertisers. The question is not whether you are using modelled data. You are. The question is whether you are being honest about the confidence intervals attached to that modelling, or whether you are presenting estimates as facts.

I have always believed that an honest approximation, presented as an approximation, is more useful than false precision. A CFO who understands that your incrementality estimate has a confidence range of plus or minus 20% can make a better decision than one who is handed a single number that implies certainty the underlying methodology cannot support.

Incrementality Testing Across Specific Channels

The incrementality conversation is not uniform across channels. Different channels have different baseline conversion rates, different test design requirements, and different levels of platform cooperation. A few channels worth addressing specifically:

Paid search branded terms are where incrementality testing tends to produce the most uncomfortable results. Branded search consistently shows lower incremental lift than almost any other channel, because a significant proportion of branded searchers would have converted through direct or organic anyway. I have seen branded search campaigns that, on incrementality testing, showed true incremental ROAS of less than one. The campaign was losing money on a net basis. It had been running for three years. Nobody had questioned it because the attributed ROAS looked strong.

Affiliate marketing is a channel where incrementality testing is particularly important and particularly underused. The affiliate model creates strong incentives for last-click attribution gaming, and the gap between reported and incremental performance can be significant. There is a detailed breakdown of how to measure affiliate marketing incrementality that covers the specific methodological challenges, including how to handle coupon sites and loyalty publishers that tend to over-claim credit.

Connected TV and streaming audio are channels where incrementality testing is genuinely difficult because the conversion pathway is long and the targeting data is imprecise. Geo-based holdout tests are usually the most practical approach, but the minimum spend required to detect meaningful lift is high. Many mid-market advertisers are running CTV budgets that are too small to test incrementally with any statistical confidence.

AI-driven channels and formats add another layer of complexity. When you are running campaigns through AI-optimised delivery systems, the black-box nature of the targeting makes it harder to design clean experiments. The question of how to measure the effectiveness of AI avatars in marketing illustrates how new format types create measurement gaps that standard incrementality frameworks were not designed to handle.

Attribution Theory and Incrementality: Where They Intersect

Incrementality measurement and attribution modelling are related but distinct. Attribution attempts to assign credit for conversions that happened. Incrementality attempts to identify which marketing activity caused conversions that would not otherwise have happened. These are different questions, and conflating them leads to bad decisions.

The most common mistake I see is teams running data-driven attribution in GA4, seeing a more “fair” distribution of credit across channels, and concluding that they have solved their measurement problem. Data-driven attribution is better than last-click. It is not the same as incrementality measurement. A channel can receive a large share of attributed credit in a data-driven model and still have low or negative incremental value. Understanding attribution theory in marketing provides the conceptual foundation for understanding why these two approaches answer different questions and why you need both.

Forrester’s analysis of whether marketing measurement is undermining the buyer’s experience makes a related point: measurement frameworks that are optimised for channel efficiency can inadvertently penalise the channels that do the hardest work in creating demand, because that work is difficult to attribute directly to conversions. Incrementality testing, done properly, is one of the few methods that can surface this problem.

The Organisational Resistance Problem

The Organisational Resistance Problem

There is a reason incrementality testing has been slow to become standard practice, and it is not technical. It is political. Incrementality testing tends to reduce the apparent value of marketing activity. That is not because marketing is ineffective. It is because most attribution systems overstate effectiveness, and incrementality testing corrects for that overstatement.

When I ran agencies, the conversations around incrementality were rarely about methodology. They were about who owned the narrative. A channel manager whose budget depended on reported ROAS had no incentive to run a test that might show lower incremental ROAS. A platform account manager had even less incentive. The resistance was not irrational from their perspective. It was rational self-interest operating against the client’s commercial interest.

The way through this is to position incrementality testing as a tool for protecting budget, not cutting it. When a channel tests well on incrementality, that result is defensible in a way that attributed ROAS is not. A channel with strong incremental lift is genuinely hard to cut. A channel with strong attributed ROAS but untested incrementality is always vulnerable to the question: “But would those customers have bought anyway?”

Measuring inbound activity presents similar challenges. Inbound marketing ROI is notoriously difficult to isolate because the conversion pathways are long and multi-touch, and the same organisational resistance applies: teams that have built their case around attributed metrics are reluctant to run tests that might complicate the story.

Practical Steps for Running Your First Incrementality Test

If you have not run an incrementality test before, the most important thing is to start with a question that has commercial stakes. Do not test a channel because it is easy to test. Test the channel where the answer would change a budget decision.

For a geo-based holdout test, the basic design is: identify comparable geographic markets, assign them randomly to test and control groups, run your campaign in test markets only, and compare conversion rates (or revenue, or whatever outcome metric matters) between the two groups over the test period. The key variables are test duration (longer is more reliable), market comparability (more similar is better), and the size of the effect you are trying to detect (smaller effects require larger tests).

A/B testing infrastructure in GA4 can support some elements of experiment design, and Semrush’s guide to A/B testing in GA4 covers the technical setup in detail. Incrementality tests are not pure A/B tests, but the principles of clean experimental design, adequate sample sizes, and pre-defined success metrics apply to both.

UTM tracking discipline is also essential. If your campaign tracking is inconsistent, you cannot reliably compare test and control group performance. Semrush’s UTM tracking guide is worth reviewing before you design any test, because attribution gaps in your tracking will corrupt your results regardless of how clean your experimental design is.

Report the results honestly, including the confidence intervals. If the test was underpowered, say so. An inconclusive result from a well-designed test is more useful than a confident result from a poorly designed one, because at least you know what you do not know.

Where Incrementality Measurement Is Heading

The direction of travel is toward triangulation rather than any single measurement method. The most sophisticated measurement approaches combine MMM for strategic allocation, geo-based holdout tests for channel validation, and platform lift studies (with appropriate scepticism) for tactical optimisation. No single method answers all the questions. The goal is to triangulate toward an honest approximation of truth.

Generative AI is beginning to affect measurement in ways that are not yet fully understood. When AI-driven content and GEO (generative engine optimisation) campaigns generate conversions through pathways that do not show up in standard analytics, the incrementality question becomes harder to answer. Understanding how to measure the success of generative engine optimisation campaigns is part of the broader challenge of keeping measurement frameworks current as the channels themselves change.

The underlying principle, though, does not change. The question is always: would this have happened without the marketing? Everything else is detail.

If you want to go deeper on the analytics frameworks that sit behind incrementality measurement, the Marketing Analytics hub covers attribution, GA4, measurement strategy, and the emerging tools that are reshaping how performance is tracked and reported.

About the Author

Keith Lacy is a marketing strategist and former agency CEO with 20+ years of experience across agency leadership, performance marketing, and commercial strategy. He writes The Marketing Juice to cut through the noise and share what works.

Frequently Asked Questions