Incrementality School, E2: What Can You Incrementality Test?

“In any brand at any moment, there’s money being wasted,” says Maddie Dault, Haus’ Customer Marketing Lead and former Director of Growth Marketing at vitamin brand Ritual. “Incrementality is about finding that money and preventing the waste, but then also reallocating it to things that are actually working.”

Things that are actually working – that’s the key part of the equation, isn’t it? In our first episode of Incrementality School, we defined incrementality in the context of causality (no, not ‘casual’ – causal) and likened it to randomized control trials in healthcare. Our Head of Science, Joe Wyer, reminds us:

Attribution and incrementality are two frameworks for understanding marketing impacts. Attribution frames impact as "crediting" each customer event to some strategy or tactic. Incrementality frames impact as the change in customer events caused by a change in the application of strategy or tactic.

That’s key, because understanding incrementality is step number one before designing tests to identify the things that are actually working. 

Which brings us here: In this second of installment of Incrementality School, we’ll run through concrete examples of what you can test with incrementality experiments – and why there’s no better time than now to give it a shot.

So, what can you test with incrementality experiments?

“Most of the major media buying and media planning questions that you’ve been circling for a while can be answered with incrementality tests,” says Maddie. “For example, you can test the ideal amount of spend to pour into a channel or tactic, and also find the point of diminishing returns.”

Depending on your business, you might start off big-picture. For example: What is the incrementality of Google brand search? Understanding whether it’s worth putting spend behind branded search terms is a common conundrum – and incrementality tests can help reveal if that money is well-spent.

For omnichannel brands, incrementality tests can provide a fact-based view into how marketing efforts affect sales across a variety of storefronts and channels. For example, a brand can test whether an Instagram ad that leads to their Shopify store has a halo effect on Amazon and retail sales (or not).

Alternatively, brands can get hyper-specific with what to test and learn. For example, rather than testing the impacts of channels at large, you could design an incrementality test to optimize your specific buying strategy – like whether you should optimize Meta spend towards 7-day click or 7-day click 1-day view. 

Incrementality tests routinely help brands answer questions like:

• What’s the optimal investment for every campaign?
• Are customers purchasing from the places marketing promotes, or are they buying elswewhere? For example, if we point folks to our Shopify website from a Meta campaign, are customers incrementally converting there, on Amazon, or in retail? 
• How can we determine if a new channel is incrementally viable?
• Is my sale incremental or just pulling forward existing demand?
• Should I include or exclude branded search terms in my PMax campaigns?
• What’s the right media mix optimization for incremental growth? 
• Are UCG or polished brand ads more incremental for my brand?
• Which paid marketing efforts are incrementally contributing during peak seasonal or holiday periods? How does this differ during low season? 
• What’s the right ratio of upper and lower funnel media?
  ‍

Reflecting on her own experiences leading growth marketing, Maddie recalls life before and after incrementality tests:

“At Ritual, we were wasting $10,000 a day – and about to waste more – on an untested TikTok campaign. It seemed like were crushing it from our attribution numbers, and we were considering majorly ramping up our spend, but I wanted to test before we put any more money towards it. Turns out it wasn’t incremental at all – truly, zero lift to our business. Incrementality testing instantly saved us a ton of money. Ultimately, we iterated on our TikTok strategy and found a way to make it incrementally viable, but there’s such an opportunity cost to not testing.”

No better time than now to embrace incrementality

There will always be a laundry list of to-dos in business – competing priorities, quarterly planning, big partnerships, new product launches – but when it comes to marketing spend, “marketing without knowing your incremental returns is just lighting money on fire,” says Nick Doren, Haus’ Customer Success Lead. With a deep background in growth marketing and analytics, Nick’s seen a lot on the frontlines and working closely with some of Haus’ biggest customers.

“If you don’t have a sense of the incrementality of your channels, you’re likely spending a lot of money that is not driving any actual value for the company – you’re paying for things that are already likely to convert anyway, so you’re hurting the bottom line of the company,” he says.

“Take this example: Let’s say you’re advertising in a channel that’s zero percent incremental, but you don’t know that yet, and you’re continuing to spend on it. That investment is theoretically driving $0 in return – so marketing goes from a revenue driver to purely a cost center, when that money could instead be reallocated towards something that’s driving incremental revenue. You might be spending a million dollars a month on something that’s zero percent incremental.”

If it sounds far-fetched, you’d be surprised. Incrementality tests are just as valuable in identifying what is working as what’s not working – so you can finally answer with objective confidence that media spend question you’ve been mulling over, and reallocate to what's effective.

That’s all for our second episode – check out case study library anytime for more in-depth tests and experiment inspiration. See you next week for a deeper look at the various ways brands measure incrementality.