Menu
Incrementality is the most precise and accurate method to measure the real value generated by individual marketing activities.
In marketing, we always strive to optimize your marketing to drive real growth and achieve tangible business results.
Most of us spend considerable time tracking attribution and various metrics and KPIs and continually adjusting and refining our campaigns to achieve the best results.
In marketing optimization, particularly in growth marketing, incrementality is the North Star of marketing measurement!
It can show you which marketing techniques and channels generate real value and which are just taking credit for organic results.
It can also provide data-driven insights into every aspect of your marketing strategy, from the best marketing mix for your business to the most effective (and cost-effective) marketing activities at the individual and ad set levels.
So, what, exactly, is incrementality? Why is it so important? How do you test and measure incrementality? And, most importantly, how can you use it to optimize your marketing?
Today, we’re covering everything you need to know about incrementality – what it is, how it differs from attribution, why it is important, how to test and measure incrementality, and two incrementality case studies to show you exactly how you can leverage the power incrementality in your marketing!
In marketing, incrementality is a way to measure causation and identify events/outcomes, such as conversions, that would not have occurred without a specific interaction, such as an ad view.
Incrementality measures the effectiveness of marketing activities and demonstrates the real value they generated, which would not have occurred without them.
No marketing activity exists in isolation, which makes it difficult to measure the true impact of each component of your marketing mix.
The overlap between organic outcomes and outcomes generated by your marketing, for example, an ad run, unique campaign, or use of a particular channel, can be hard to separate and accurately measure. This often leads to imprecise decision-making and budgeting, which leads to wasted ad spend.
Incrementality measuring is the most accurate method for identifying which specific elements of your marketing are driving real value and which are merely cannibalizing credit for organic outcomes.
It is important to note the distinction between incrementality and attribution, so let’s take a closer look at how incrementality and attribution differ:
Attribution refers to how marketers track and credit the various touchpoints that led to conversion along the customer lifetime value. Several models, including single-touch and multi-touch attribution (MTA), are used to measure attribution.
MTA is the most detailed or holistic attribution model. It tracks interactions at various points along the customer journey, as opposed to just the first or last touchpoint. While this provides some important insights, they are quite limited compared to the insights provided by incrementality.
Take a quick look at this video by Google Ads for a nice refresher or overview of attribution vs incrementality:
MTA is limited to data from actions taken in an online setting, such as clicks or web visits. It does not take into account offline data from, say, print or TV ads or online data with no immediate action, such as an ad or social media impressions.
This is especially limiting for channels where impressions are difficult to track and quantify, such as offline campaigns and ‘walled garden’ social media channels like Facebook or Pinterest.
Incrementality, on the other hand, is measured using a different approach (more on that below), and data from a variety of sources. It determines the desired outcomes that would not have occurred without that marketing activity.
It takes into account impressions and clicks for each platform or campaign being measured. Additionally, it provides insights into the impact of that activity compared to the impact of other campaigns/channels and the organic outcomes that would have occurred anyway.
As such, incrementality provides your attribution data and a whole lot more. With this nuanced and detailed picture of your marketing activities, you can make more precise and effective decisions to maximize the value of your marketing spend across your entire portfolio.
TLDR: Attribution vs. Incrementality
Attribution credits the touchpoints that lead to the desired outcome, such as a conversion, in the presence of various marketing activities. Incrementality measures the desired outcomes achieved with and without a specific marketing activity, such as an ad run, giving you a true reflection of the return on investment that specific activity generated, which would not have occurred in its absence.
Incrementality is frequently referred to as the ‘North Star’ of marketing. While this might seem like quite a grand claim to make, measuring incrementality will profoundly affect the results you achieve and the direction of your overall marketing strategy.
Measuring incrementality allows you to identify and address the impact of otherwise difficult variables to isolate and assess. It allows you to answer questions like:
Incrementality testing provides the data and actionable insights you need to make strategic data-driven decisions. It will inform and justify budget allocation, prevent wasted ad spend, and allow you to continually fine-tune your marketing to drive tangible results.
Measuring incrementality determines the real value generated by a marketing activity. When you measure incrementality, you can see what happened with that activity and what happened without it. Knowing the return on marketing investment provided by individual activities, such as ads, campaigns, channels, and even individual variations of each, allows you to know precisely where to focus your efforts and where you’re wasting resources on activities that do not generate measurable results.
Measuring incrementality can be done in various ways, depending on the use case and the insights required. Both Marketing Mix Modeling (MMM) and Multi-touch Attribution (MTA) can be considered ways to measure incrementality, in the right scenario.
However, the Design of Experiments (DoE) test is the most commonly used method to test incrementality. This is similar to A/B testing but leans closer to statistical analysis and uses test and control groups.
In its simplest form, incrementality testing involves segmenting your target audience into two groups: your test and control groups.
The test group is then exposed to the marketing activity that you’re measuring (e.g., a video ad or a specific campaign on a specific channel or platform), and the control group is either exposed to no ads or random, unrelated ads/public service announcements, for a set period.
The difference between the two groups regarding the number of desired outcomes (e.g., conversions, sign-ups, downloads, etc.) demonstrates the incremental lift contributed by that activity, which would not have occurred without it.
The control group results represent a baseline of what would naturally occur in the absence of that particular marketing activity.
The formula for measuring incremental lift is: test group result minus control group result, divided by control group result, equals incremental lift.
For example, your test group performed 1000 desired outcomes (e.g., sign-ups or conversions), and your control group performed 800 desired outcomes.
The formula would then be 1000 minus 800 divided by 800, which equals 0.25. To convert that to a percentage, multiply the final result by 100 (e.g., 0.25 x 100 = 25% incremental lift).
If you prefer, you can use the same formula using your results when converted to percentages.
For example, if the test group achieved a 2% conversion rate, and your control group achieved a 1.5% conversion rate, the formula would be 2% minus 1.5%, divided by 1.5%, equals 0.33% incremental lift.
Incrementality testing can be as simple as the example above, measuring just one or two variables. Or it can be much more complex, involving multiple variables and requiring specialized expertise to design, deploy, and interpret.
There is no limit to how many data sources you can use for incrementality testing. As long as your experiments are well-designed, controlled, and properly executed, you can measure the incremental contribution of any variable against any measurable outcome.
Summary
Measuring incrementality identifies causation between the specific activity being measured and the positive, negative, or neutral impact of that activity on the desired outcome when compared to results from the control group. This accurately shows you the incremental contribution of that activity towards your objectives and overall marketing goals, which shows you exactly how valuable that activity is (or isn’t).
Now that we have covered incrementality, how it differs from attribution, why it is important to measure it, and how to measure it, let’s examine how incrementality testing works and some tips for designing an effective incrementality test.
So, how, exactly, does incrementality testing work? Let’s look at the process step by step:
Incrementality is all about testing and experimenting and then adjusting your approach based on the results you get.
Like all scientific experiments, you need to plan carefully and determine exactly what you’re testing, what outcomes you want to achieve, and how you will determine your results.
You also need to identify your parameters and variables and plan how you will control them to limit interference from factors outside of your control.
Take a look at this video by AppsFlyer for an overview of incrementality testing and how it works in practice:
Broadly, there are five stages of a Design of Experiments incrementality test:
Let’s take a quick look at each stage individually:
Incrementality tests must be carefully planned and thought through from end to end before you begin.
Initially, you need to define the question you’re asking, what you want to find out by asking it, and what KPIs you can use to determine your answer. To do this, you need to define:
Once you have defined the above, your question (or hypothesis, if you will) might look something like this:
“Does running x ad on y channel, for z period, have any statistically significant impact on our conversion rate?”
Ideally, results should be statistically significant, which is a measurement of the likelihood that the incremental lift you observe is not merely a coincidence. In the simplest terms, a statistically significant result is very unlikely to have occurred by chance or due to a sampling error.
Now that you have defined the parameters of your incrementality test, you need to select the target audience and segment it into test and control groups.
These groups should be randomly segregated, have similar characteristics, and, as far as possible, not overlap.
Using the test and control groups accounts for some of the factors you can’t control, such as the impact of organic behavior or exposure to your other marketing activities, because both groups are impacted by the same factors.
The larger your two groups are in terms of size, the better. Larger sample sizes are less prone to sampling errors, where coincidental factors skew the test results. The control group should comprise at least 20% of the total audience.
Attribution platforms can be a useful place to start when defining and segmenting your audience.
However, where the target audience does not already have any unique identifiers, you will need to use other factors to define your audience.
These can be demographic, geographic, time frames, products, etc. The parameters you use will depend on the test you’re conducting and the nature of your business.
Deploy the marketing activity you’re testing to the test group of your target audience.
The test duration will depend on your parameters, requirements, group/sample sizes, the volume of data you’re working with, and the average business cycle for your type of business.
However, in general, the test should run for at least 7 days and be deployed at a time when you are not deploying any other campaigns.
This will give you a more accurate result and reduce the potential for other marketing activities impacting results.
When your incrementality test is complete, collect the data and measure your results to see the incremental lift in the KPIs you determined to gauge your results.
Looking at the relationship between the test and control group results will give an idea of how they behaved differently and how and why you got the results you see from the test.
Remember that your incremental results may be positive (incremental lift), negative (the test group performed poorly compared to the control group), or neutral (no significant difference between the two groups).
If the gap between the two groups’ results is much wider than expected, you may need to reconfigure your test and run it again.
Incrementality testing can be a complex and challenging process, and depending on the type of test you need to run, you may need to get help from a professional. There are also platforms you can use that simplify the process and give a bird’s eye view of your data and what you can learn from your results.
When you know which marketing activities drive real value and how much channels cost compared to their profits, you can focus your budget effectively and reduce wasted spending.
One of the most useful things about incrementality is that it gives you a better indication of your ROAS vs organic revenue. Sometimes, you can determine that you can spend less on advertising altogether because your marketing activities have just been cannibalizing organic results.
To calculate incremental ROAS (iROAS), you can subtract the revenue generated by your test group from the revenue generated by the control group and divide it by your total marketing spend. This will remove organic conversions from the equation and show you your marketing activity’s real, quantifiable impact.
In marketing, the more you know about your results and how you achieved them, the more effective your optimization efforts will be. When you’re making data-driven decisions and continually testing and optimizing your campaigns, you can achieve the most while spending the least.
Incrementality can be used to measure all marketing-related activities and help determine which ones are the most valuable. It can also identify which ones are taking credit for organic results and which ones cost more than they’re worth.
To see how incrementality can be used in real-world applications, let’s take a look at some case studies from the incrementality experts at Measured:
Measured was approached by Shinola, a luxury lifestyle brand based in Detroit, for help after the introduction of Apple’s App Tracking Transparency (ATT) came into effect. ATT and the associated data restrictions made accessing accurate measurements and reporting on customer acquisition through Facebook campaigns challenging for the brand.
Shinola observed a drop in the number of conversions Facebook reported but knew that their campaigns were still generating sales.
It was suspected that the platform was underreporting conversions. However, Shinola had no way of accurately measuring the incremental contribution of those campaigns. Shinola then hired Measured to solve the problem, using incrementality testing to show the true value their Facebook campaigns were delivering.
To determine the number of incremental conversions generated by Shinola’s Awareness and DABA (Dynamic Ads for a Broad Audience) campaigns on Facebook, they ran a geo-matched market test deployed at the zip-code level.
Geographic experimentation was chosen because it could be run without relying on the platform’s reporting and did not require any data restricted by Apple’s ATT. They chose the zip-code level for increased precision and to reduce risks associated with a media holdout (holdouts are frequently used in incrementality testing and involve ceasing all other marketing activity for the duration of the test).
When conducting the test, a random selection of zip codes was excluded (control group). By comparing the test group results (zip codes included and shown in the campaigns), they could determine the number of conversions generated by the campaign and the number that would have occurred anyway without the campaign.
The incremental test results showed an incremental lift of 14.3% for both the awareness and DABA campaigns. This meant that Facebook was underreporting Shinola’s campaigns’ overall performance by 413%!
Knowing this allowed Shinola to identify that the channel was more valuable, in real terms, than the platform’s reporting indicated. This meant that they could appropriately allocate funds to campaigns that were driving real results instead of reallocating that budget to other channels, which might be less effective.
Read the full case study here: Measured – Shinola Case Study
The Bottom Line
Incrementality testing allowed the client to:
Faherty approached Measured, a family-run clothing and lifestyle brand with a rapidly expanding e-commerce store on Shopify.
Faherty’s growth was driven primarily by advertising on Facebook and Instagram. They needed to know if they had exhausted these channels and saturated their audience or if there was still room to scale.
They also wanted to know how to continue driving aggressive growth by diversifying their marketing channels. Faherty was operating on a lean marketing budget and needed to ensure they were spending appropriately to drive true incremental value while wasting as little as possible.
They also wanted insight and guidance on applying the insights gained from incrementality testing to their existing business.
Measured implemented a four-week cross-channel incrementality measuring and reporting framework. This assisted Faherty’s choice of media channels and decisions to scale different channels.
They looked at Granular incrementality insights for:
They implemented a marketing data warehouse (MDW) for robust measurement and reporting. This unified data asset consolidated online and offline transactions, media platform reporting, CRM, LTV, web traffic data, third-party ID mapping, and various incrementality measurement outputs.
Using a cross-channel incrementality measurement and data warehouse approach enabled Measured and Faherty to:
The above led to positive, double-digit percentage improvements YoY for all core marketing and business KPIs. This allowed for a 53% increase in sales revenue.
Read the full case study here: Measured – Faherty Case Study
The Bottom Line
Incrementality measurement provided an accurate method for analyzing large data across multiple channels. It also allows them to accurately measure which channels were the most efficient at providing real results. They could assess how much they were able to scale each channel without compromising profitability.
Incrementality is the best method for accurately testing and measuring the real impact of any marketing activity.
It can provide nuanced and detailed insights into every aspect of your marketing strategy for your whole marketing portfolio.
Using incrementality testing can help you refine and optimize your marketing. This will minimize risks through data-driven decision-making.
Today, we have covered everything you need to know to understand how incrementality testing works. We have also explained how to measure incrementality and what insights you can gain from measuring it in your marketing. We hope you have found all the information you need to use these methods to succeed in your marketing efforts.
Incrementality in marketing refers to the incremental benefit or incremental lift generated by each action taken as a result of a particular marketing activity. Incremental lift refers to the increased benefits or number of desired outcomes (such as conversions or web visits) experienced as a direct result of a marketing event or activity (such as an ad campaign or proportional offer). Read the full guide to learn more about incrementality in marketing.
Attribution refers to the process of matching two data points, such as clicks to conversions, to assign credit to the interaction or event that led to the conversion. Incrementality, on the other hand, refers to a method of measuring the true impact of any marketing activity. Read the full guide to learn more about incrementality, and how it differs from attribution in marketing.
An incrementality test is an experiment that is designed to measure the true effectiveness of a marketing activity. The target audience is segmented into two groups, a test group, and a control group. The test group is then exposed to a marketing activity and the control group is not exposed or exposed to a neutral or unrelated equivalent. The difference between the number of desired outcomes achieved by the test group and the control group demonstrates the true incremental lift/benefit that resulted from the marketing activity in question, and would NOT have occurred in its absence. Check out the full guide to learn more about incrementality testing.
In marketing, incrementality is measured in terms of the incremental lift or benefit generated by a marketing activity, which would not otherwise have occurred organically. Incrementality is measured by running an incrementality test, using a test group and a control group. The difference between the test and control group results demonstrates the true, incremental benefit generated by the marketing activity. The incrementality formula is: test group result, minus control group result, divided by the control group result, equals incremental lift. Incremental lift may be positive, negative, or neutral. Read the full guide to learn more about how to measure incrementality in marketing.
Calculating incrementally: Test group result minus control group result, divided by the control group result, equals incrementality. Multiply the final result by 100 to convert it into a percentage. For example, if the test group achieved 1000 conversions and the control group achieved 800 conversions, the calculation would be (1000 – 800) ÷ 800 = 0.25. You can then convert the final result to a percentage by multiplying it by 100 and conclude that the marketing activity you tested resulted in a positive incremental lift of 25%. That is the true incremental value of that activity, which would not have occurred without it/organically. Take a look at the full guide to learn more about how to calculate incrementality.
Measured: What’s the Difference between Attribution vs Incrementality?
Meta for Business: The Benefits of Using Incrementality Measurements for Your Business
Nielsen: The Importance of Incremental Lift
Skai: Difference Between Incrementality and Multi-Touch Attribution
Smart Insights: Incremental Measurement – Advantage and Opportunities
Social Media Today: How to Fix Ad Measurement with Incrementality Testing and Experiments
Think with Google: Your Measurement Resolution for 2021: Get a Grip on Incrementality
Towards Data Science: A/B Testing: A Complete Guide to Statistical Testing
So, you’ve invested loads cash into marketing, but you’re not driving the revenue you need to ju...
Evaluating and monitoring your marketing campaigns, and overall performance, is tricky without using...
Pay-per-click (PPC) advertising offers businesses a direct path to drive traffic and conversions. ...
Enroll in our free 5-lesson email course today!
Free for limited time only.
We created P2P to provide free resources to brands that believe in the power of peers to promote their service or products.
© 2019 P2P Marketing All Rights Reserved.