Tread carefully when trying to split ROI into long- and short-term

Opinion: Strategy Leaders

The big digital players getting into econometrics can only be good for the industry… right? December19’s head of performance isn’t so sure.

It’s no secret that historically Google and Meta (formerly Facebook) heavily relied on last click attribution or digital attribution models to measure the effectiveness of their advertising campaigns.

It’s also no secret that these models only gave credit to the last click or touchpoint in a customer’s journey, which led to either blind belief in whatever the platforms were telling you — or an awareness that you’re not getting the full picture (depending on who you speak to). Both also operate primarily as walled gardens, meaning your options were to trust their data or bring in your own third-party analysis (often econometrics) to better understand what your marketing is delivering.

In recent years it’s becoming more apparent that both players are getting a little more vocal and involved in the world of media mix modelling (MMM) and econometrics. There have been a few interesting and well researched (and well meaning) thought leadership pieces on measurement and effectiveness published by both, some of which are sourced at the bottom of this piece. Meanwhile Meta is also currently offering a free, open-source “experimental” package for running your own econometrics. All of this has also been met warmly by the Godfather of effectiveness Les Binet from whom I’ve noticed a few Tweets and articles praising the steps forward. From what we can see, the end of the measurement world isn’t exactly nigh.

Now, I don’t have any agenda for or against the big digital players moving into econometrics. It is for the best. And if ITV told me they’d help me run and interpret my econometrics, I’d be equally sceptical.

Furthermore, I accept that there are people in the research and measurement teams at both companies who will just be wanting to ensure more robust measurement for all, that’s cheaper to access and easier to understand. Great.

What the literature tells us

However, call me cynical, but I can’t help but wonder whether these billion-dollar (Meta) and trillion-dollar (Google/Alphabet) companies got to where they are by purely altruistic means, just doing things for the good of the industry. After all, advertising is their biggest revenue stream, it’s not unreasonable to question their motives, is it?

In WARC’s 2022 guide to “How Does Effectiveness Vary By Media Channel” the top five media channels from an Ebiquity study were TV, Radio, Newspapers, Magazines and Out of Home. There’s another source showing TV and Generic PPC are the biggest contributors to short term sales by quite some margin (with about 50% of all short-terms between them) — this was, however, from a piece of work published by Thinkbox (although the research was run by GroupM and Gain Theory).

In either study TV has over double the contribution to paid social. TV and traditional channels outperforming expectations when looked at through an MMM lens just feels like one of those things that’s been drummed into me over the years.

Now, don’t get me wrong. I don’t really care where a client spends their money, whether it’s on TV or social media. I think arguments about channel effectiveness can be quite tedious and pointless, as you almost always need a blend of media channels to achieve optimal results.

I’m only referencing other sources relating to TV as, when reading Meta’s 2022 “The Short- and Long-Term Impact Of Advertising”, it seemed so at odds with other studies I’ve seen over the years. Meta instead presented the key finding that Instagram and Facebook have the strongest contribution to ROI specifically when compared to TV. My interest was piqued.

Why do we need another ‘versus’?

In Meta’s 2022 “The Short- and Long-Term Impact Of Advertising,” I noticed something I hadn’t seen before: an attempt to split out long and short-term ROI. As a non-econometrician writing a thought piece, I don’t have the time or inclination to fully delve into their methodology.

However, my question is: what was wrong with “long-term” channel analysis capturing everything from the first attributable response to the last? Why did they need to split it out? My hunch is that the short-term effects will be disproportionately attributed to channels that drive an immediate action within the model. Maybe they’re right and I’m wrong, and this is the best way to look at channel effectiveness now. But I can’t help feeling a little suspicious.

Why do we need to introduce another “vs.”?

Looking at Google’s recent publications around econometrics I recently came across “The MMM Handbook” which they’ve aimed at CMOs. For 99% of it I was just thinking how this is a helpful resource, great tips to think about when setting up and interpreting an MMM model, what data to include and to ignore. Then I got to the below point for what data you should include when looking at branded search:

“It is up to the modeller’s discretion to use clicks or impressions as inputs in the model, but it is recommended to avoid using costs as input variables, as this can lead to misleading results. Costs can vary based on targeting, advertiser’s, and competitors’ bidding strategies, and can cause misrepresentation of ad effectiveness.”

Be sceptical

Fair enough. That sounds like good advice, right? We don’t want to have a misrepresentation of effectiveness, right? Would Google have any bias here to make you want to do that?

Surely you need to know if competitors being more aggressive in the search space pushing up your costs is making search a less effective channel? Or if being too targeted makes your CPM too expensive to be workable? Why would Google recommend you put your head in the sand and base it on cost-agnostic clicks or impressions? (Maybe because it protects them from elements that make the channel look less effective?)

If that’s an intrinsic part of the channel then that is, by default, a factor in channel effectiveness that needs to be measured. With that logic perhaps also just measure TV based on impressions in September to November when it’s at its most expensive, as competitors spending more in the channel is inflating your cost-per-thousands — or ignore the premium attached to your drivetime-only radio campaign as the targeting has made it more expensive.

If 99% of what comes out of this is good, that’s a great step forward for the industry. All I would say is continue to critique what you hear and the advice that you receive.

What’s the likelihood of Thinkbox or ITV, Meta or Google telling you something for altruistic means? Always question and challenge the results you’re seeing.

David Lucy is head of performance at independent media agency December19

Strategy Leaders: The Media Leader‘s weekly bulletin with thought leadership, news and analysis dedicated to excellence in commercial media strategy.
Sign up for free to ensure you stay up to date every Thursday.