#incrementality

[digital-measurement-hub]

Attribution In 2025 Is A Stack, Not A Single Source Of Truth

Everyone wants the one attribution model that will finally settle every budget argument. In practice, anything that actually works in 2025 looks less like a magic formula and more like a small stack of imperfect signals that each answer a different question.

Here is how I’ve started structuring attribution work for media and analytics teams, in a way that stays realistic about privacy limits, GA4 weirdness, and messy channel mixes.

Layer 1: Fast, directional attribution

This is the “what is probably working right now” layer.

Tools are usually:

GA4 or similar analytics platform

Channel level cost data

A few basic models, like last click, data driven, time decay or position based

The point is not perfection. The point is to have a quick view that can answer questions like:

Did that new branded search campaign tank non brand performance

Are we spending on channels that never show up in any path

Is direct somehow “winning” everything because tracking is broken

Things that actually help here:

Trends over time, not one report Look at how channels move together or in opposite directions week to week. If CTV flights line up with spikes in branded search, that is already a useful hint.

Path patterns instead of precise credit shares Seeing search show up mainly in the middle of the path, or email frequently closing journeys, changes how you talk about those channels, even if the exact percentage credit is fuzzy.

Cleaning up obvious nonsense If “direct” is half your conversions, or display somehow has a better CPA than branded search, that is usually a tracking issue, not a miracle.

Layer 1 is good for weekly and monthly discussions, on off decisions, and quick “move a little budget” actions. It is not strong enough to defend large shifts by itself.

Layer 2: Incrementality and geo testing

This is where things start getting closer to causality.

Questions at this layer sound like:

What actually happens if we pause this channel in a few regions

How much lift does retargeting really provide when you hold out a group

Does that “upper funnel” campaign move branded search or direct in a measurable way

Simple approaches that work:

Geo split tests, where some cities or regions get media and others do not

Holdout groups for retargeting or loyalty campaigns

Treated vs control store sets when you mix offline and online

Two rules keep these tests from turning into endless science projects:

Test one thing at a time Pick one channel, tactic, or audience and isolate it as much as you reasonably can. You are looking for “directionally clear” results, not something that will go in a journal.

Pick one primary KPI before you start Decide if the test is about revenue per user, new customer rate, lead quality, or something else. If you measure ten KPIs and chase the one that looks nice, the result will never feel trustworthy.

The goal is to end up with multipliers that you can remember and use in conversation. For example, “when we run this prospecting campaign at X level, we usually see about a 10 to 15 percent lift in branded search conversions in treated regions.”

Those small rules of thumb are gold later when people fight about budgets.

Layer 3: Model based planning, like MMM

The slowest layer, but also the one leadership tends to respect once it is framed correctly.

This is where you might use:

Marketing mix modeling

Advanced data driven models across channels

Long term historical data with seasonality and promo variables

You are mostly looking for three things:

Elasticities How outcomes change when spend moves up or down, instead of just average ROAS at one spend level. That is what helps answer “what happens if we add 20 percent here and remove 15 percent there.”

Cross channel effects Maybe TV or CTV lifts search and direct. Maybe display does not close many sales but helps feeds retargeting pools. Good models quantify those relationships instead of leaving them as vague hand waving.

Diminishing returns curves Every channel breaks at some point. Being able to show “this is where paid search goes from strong to very flat” is a huge help when people want to dump extra budget into the same comfortable channel forever.

I treat these models as decision calculators during planning seasons, not as permanent truth. They serve best when they guide discussions like “how should we shift the next 10 to 20 percent of spend” rather than “who gets 100 percent of all credit forever.”

How the three layers work together

The most useful pattern has been:

Layer 1 to keep a live narrative of what is working and what looks broken

Layer 2 to validate or challenge that narrative with real tests

Layer 3 a few times per year to guide bigger mix decisions and answer “what if” questions for leadership

The result is not a single model, it is a small set of views and rules that you can actually explain in plain language.

Helpful resources if you live in this world

A few places that post solid material in this space:

Adswerve has a lot of practical content on GA4, Google Marketing Platform, and BigQuery, especially around measurement setups and reporting patterns for real media teams: https://www.adswerve.com

Bounteous publishes good pieces on analytics architecture, experimentation, and how to connect product analytics with marketing data without turning everything into a dashboard beauty contest: https://www.bounteous.com

I treat both as idea banks, not rulebooks, but they are handy when you need to sanity check your own approach or grab a new angle for explaining attribution tradeoffs to non technical folks.

If you are building your own attribution setup, I am curious how close or far this is from what you are doing.

Do you run a similar three layer approach, or lean heavily into one model Where have tests surprised you and made you rethink a favorite channel

[rtbhouseblog]

What Should You Know about Incrementality Testing?

Have you ever wondered how much impact your retargeting campaign truly has on your conversions? The answer lies within the realm of incrementality testing, a powerful tool that allows you to unravel the effectiveness of your retargeting providers. By delving into the depths of testing, you can unlock valuable insights that will help you optimize your marketing efforts and maximize your ROI. 

The Essence of Incrementality Testing

The purpose of this testing is to shed light on the portion of conversions that can be directly attributed to your retargeting campaign. By comparing the conversion rates of a control group (those not exposed to retargeting ads) and a test group (those exposed to retargeting ads), you can discern the incremental lift generated by your retargeting efforts. This insight enables you to refine your retargeting strategies, ensuring that you invest your resources where they will yield the greatest returns.

The Role of a Dedicated Analytics Team

To truly harness the power of this testing, you need a dedicated analytics team by your side. These analytical wizards possess the expertise to design and execute precise experiments, interpret the results with finesse, and uncover the hidden gems buried within the data. However, incrementality testing is expensive. So it is worth knowing and remembering that there are other methods by which we will check the effectiveness of our retargeting campaign. 

[pathtopurchase]

High Bounce Rates considered Not Harmful

People say high bounce rates are bad. At first glance, this makes sense. After all, who’d want customers to visit a (web or app) page and leave without seeing any more pages? We might decide, therefore, that digital advertising may be made more efficient by eliminating channels with high bounce rates.

This is a mistake. Let me explain why.

Consider the following:

1. Salesperson A stands just outside a retail store handing out pamphlets. 90% of the people who get pamphlets visit the store. Of the people who visit the store, all of them end up shopping at the store. 10% Bounce rate, 100% Conversion rate (=customers/visitors).

2. Salesperson B expends shoe-leather, visits customers’s homes and hands out pamphlets, even speaks to a few customers. 10% of the people who’ve seen the pamphlets end up visiting the store; of these, 50% end up shopping. 90% of people “bounce” i.e. they see the pamphlet but don’t visit the store. 90% Bounce rate, 50% Conversion rate.

3. Salesperson A and B have the same fixed salary.

Would you, on the basis of the above data, conclude that Salesperson A is the better salesperson? Or, that Salesperson A is attracting “higher-quality traffic” than Salesperson B?

Consider the following question instead:

Suppose Salespersons A and B hadn’t handed out pamphlets. What percentage of people would have become customers anyway?

The conclusion is the very opposite of what we’d get if we went by bounce rates or conversion rates. 

Granted, it’s a lot easier to measure things based on last-click based attribution, and a lot harder to measure Incrementality. This would be alright if a sole emphasis on last-click attribution and bounce rates resulted merely in a suboptimal solution. The trouble is, it can lead you in exactly the wrong direction.

Notes

1. The title of this essay is inspired by the computer scientist Edsger Dijkstra’s essay “Go to statement considered Harmful”, see http://www.u.arizona.edu/~rubinson/copyright_violations/Go_To_Considered_Harmful.html

2. I’ve taken some liberties in extending the concept of bounce rates to the offline world. Specifically, bounce rate is defined as the % of total sessions that begin and end at the landing page. In the Salesperson analogy, I’m considering a “view” of the pamphlet to be a visit to the landing page, and a visit to the retail store as the negation of bounce. Imperfect though the analogy is, I think it captures the essence of the problem well enough.

3. For an example of how an emphasis on different metrics, each seemingly sensible, leads to entirely opposite conclusions, see Jeff Bezos’s 2004 letter to shareholders, where he compares the effect of optimizing for free cash flows versus net profits. https://www.sec.gov/Archives/edgar/data/1018724/000119312505070440/dex991.htm. 

In this letter, Bezos says: “Though some may find it counterintuitive, a company can actually impair shareholder value in certain circumstances by growing earnings.” 

Analogously, marketing teams can actually impair their effectiveness in certain circumstances by trying to reduce bounce rates. 

There’s another great quote from the same letter: “Cash flow statements often don’t receive as much attention as they deserve. Discerning investors don’t stop with the income statement.” 

Of marketing, I’d say the following: Incrementality doesn’t receive as much attention as it deserves. Discerning marketers don’t stop at bounce rates and conversion rates, they also consider incrementality.

4. If we compare marketing channels with the same Incrementality, lower bounce rates are certainly preferable. The trouble is not considering Incrementality at all.

[varshasingh208us]

Incremental Lift: Measuring Marketing Efforts Beyond Attribution

Incremental lift gives an insight into the contribution of different channels in revenue/campaign performance that help in planning better spending avenues.

[nschenck]

Measuring Incremental Conversions On Paid Acquisition Channels — 3rd & Lamar Media

Determine how many incremental conversions your ad campaigns are generating by following the steps in this model.

[retailtouchpoints]

How To Determine Whether Your Marketing Is Working

All retail marketers try to measure the impact of their strategies and tactics. They have been asked to prove return on investment for too long to have survived without an answer. Yet many marketing teams use the wrong approach. In fact, the most common approaches to knowing what marketing is working don’t actually measure ROI at all.

Forrester found that marketers spent approximately $1.09 billion in 2017 to measure their impact. In many large companies, that investment goes to multitouch attribution. Smaller companies rely on attribution rules, such as last-click attribution or weighted attribution. Attribution models have different reasons for their allure, but they have one thing in common: they don’t actually measure the incremental sales driven by marketing.

What’s Wrong With Attribution 

Every attribution model seeks to help marketers determine which tactics drive revenue. Unfortunately, multitouch attribution and all rules-based attribution models have several flaws.

For starters, these models cannot measure incrementality. In other words, they can’t measure the difference between running your advertising or doing nothing at all. This is one of the reasons multitouch attribution is considered to be in the “trough of disillusionment.” It can’t actually deliver the most foundational thing that all marketers are being pressed to prove. Without measuring incrementality, you cannot measure the ROI of your marketing investment.

Furthermore, multitouch attribution needs to capture every customer interaction and use that data to judge touchpoint influence on purchase decisions. But capturing every touch point is impossible. Too many customers engage with brands in untrackable ways, like offline conversations. Customers also have existing brand affinity that could drive purchases.

Lastly, the complex methodologies underpinning multitouch attribution are various forms of forecasting. They use the past to predict the future. But past results do not guarantee that trends will continue. Different creative, competitive activity and changing media habits are just three of the reasons multitouch attribution is a poor fit for evaluating the performance of your current campaigns.

To drive more sales, retail marketers must use a different method.

The Science-Based Approach To Measuring Incrementality

The only way to measure incrementality is to compare a test group to a control group. That’s why pharmaceutical companies are required by the FDA to conduct double-blind studies. By comparing a group using a new drug (aka exposed group) to a group taking a placebo (aka control group), you know whether the drug is actually working.

Historically, this test vs. control approach was part of most marketers' tool kit via test markets. For example, a retailer would choose one or more representative metro areas in which to launch a new product line extension. They would then compare their overall brand sales in the test markets to other markets where they had not launched the new product. Test markets are seldom used these days because they are unacceptably slow and give the competition too much insight into future plans.

Today, retail marketers can accurately measure incremental sales without having to launch a test market. By tracking ad exposures at an individual level and connecting those exposures to an ID graph, they can find out who has seen an ad and who has not. This tracking happens all day, every day in the digital marketing ecosystem. What’s relatively new is the ability to measure sales among the exposed group to sales among the control/unexposed group. The result is an accurate and fast measurement of incremental sales.

Advice For Making The Switch

Retail marketers seeking to improve the accuracy of their incrementality testing should follow three scientific rules:

1. Test for statistical significance.

Not all results are relevant. Test for statistical significance to determine whether the results of marketing suggest real differences between exposed and control groups or whether standard deviation (aka noise) accounts for the differences. Just because a coin lands on heads in six of the first 10 flips does not mean the odds favor heads in future flips.

My company, Commerce Signals, worked with a large retailer to measure the sales lift driven by 10 different digital marketing tactics. Three of the tactics showed statistical significance, so the retailer reinvested in those approaches to more than double the incremental in-store sales that were generated by this campaign.

2. Use large sample sizes.

Large samples have big benefits beyond making statistically significant results more likely. The bigger the sample size, the more you can reliably drill into the results to see the incremental sales impact of varied creative, promotions, audience targets, demand-side platforms and more.

Let's use the coin flip example again. The more times you flip a coin, the less likely it is that you'll end up with a majority of heads. In your context, this means that with larger sample sizes, you are less likely to get results that reflect randomness.

3. Compare test and control groups before new campaigns.

Just like the test market approach, you should compare the sales of your test and control groups in a time period before your marketing starts. Essentially, this shows you whether you have a “good” control group or your interpretation needs to be adjusted.

For example, if your exposed group bought 5% more than the control before your marketing started, you’d want to know that it bought at least 6% more during and after your ads aired to declare a positive sales lift.

Measure sales. Shift spend. Grow faster. And look good while doing it.

Nick Mangiapane is Chief Marketing Officer of Commerce Signals, a data insights platform that helps marketers make better decisions in near-real time. Mangiapane is a pragmatic consumer marketer with leadership experience from Procter & Gamble, Newell Rubbermaid, ​and Ingersoll Rand in addition to Commerce Signals. Mangiapane is an alum of Boston College and Cornell's business school.

[netmininginsights-blog]

Digital has long been viewed as the one advertising channel where everything is measurable. Sometimes that’s easier said than done (as anyone in the industry will tell you). But measuring incremental lift of ads is one of those things that is within reach, and will go a long way in helping marketers make smart decisions about their media plans.

[netmininginsights-blog]

The concept of incrementality is still maturing and different tests with the same name actually cover very different realities, ranging from accurate depiction of the truth to pure science fiction. But while testing incrementality is littered with pitfalls, such as misallocating users or premature decision-making, there are two methodologies that can help avoid them:  Taking Ddifferent approaches and the right set-up for your goals.