MMM is getting smarter, but it still can't do the whole job alone

Boathouse chief strategy officer Sonia Chung argues that the rise of off-the-shelf automated marketing mix modelling has created a new risk: mistaking a polished answer for a definitive one. In a head-to-head test, an off-the-shelf platform estimated that 10 paid-media channels drove roughly 75% of new-customer volume, while an independent custom build on the same data put the combined incremental contribution at under 1%. Chung writes that she has seen models identify 1-18% incrementality from paid media, and that when client data is sparse, a Bayesian model's priors from other brands can shape the answer more than marketers realize. The gap means "either conclusion is impossible to treat as decision-ready without further investigation".

The crux

Automated MMM has made measurement faster and more accessible, but it has not made it more trustworthy, and a model that is not challenged on its assumptions and specification can overattribute performance to the channels it can see.

Sell-side read

Publishers selling on outcomes now face buyers whose MMM may misattribute their channel, so knowing the caveat, that statistical fit is not causation and that the model only sees the variables it is given, is the ammunition to push back on unfair attribution.

Watch

Whether MMM platforms bolt on experimentation, geo holdouts and conversion-lift studies to close the gap between modelled and tested conclusions.