The Control-Group Methodology Every Brand Needs to Demand of Mobile Marketing

The time has come to revise the way the online ad industry measures media impact and incremental sales lift. Future credibility depends on it.

When it comes to measuring incremental lift, there is a key question every brand needs to ask its mobile-marketing provider: does my unexposed control group precisely match the behaviors and context in which my brand’s mobile ads were delivered?

If the answer is anything other than yes, then brands are almost certainly not seeing an accurate picture of incremental lift. Advertisers know this — and mobile marketers know it — and the time has come to revise the way we measure media impact. The future of our credibility depends on it.

In the sections that follow, we’ll consider where the data to measure truer incremental lift can be found, and how improving our measurement methodology will drive a stronger and more sophisticated approach to campaign tactics as well.

Incremental Lift: Measurement Requires Context … and a Window onto Tactics
Measuring truer lift means developing an ongoing series of unique control groups, ones that precisely match all the details of the exposed users — at the moment they receive a given ad — to the control group that remains unexposed to the impression in question.

Beyond demographics and all the details we typically include, we have to look even deeper at the exposed-group and the control-group dynamics. If Consumer A is served an ad because they are within a mile of a given store, then the unexposed control-group users must also be within a mile of the retail location. Similarly, if Consumer A visited the relevant store on a Saturday prior, the control group needs to have visited at that time. If it’s snowing for Consumer A, control-group consumers are in the snow as well.

And it’s not just the demographics, user behaviors, and granular information around time, place, and circumstance — measuring a truer incremental lift means knowing the precise tactic marketing used. We need to know exactly how the ad was served.

What we find, when we measure in this way, is that control-group construction becomes an ongoing process, built from scratch every time, accounting for every instance of exposed-group variables. And how do we achieve this kind of specificity? We turn to rich data. In the next section, we examine what rich data means and where we can find it.

Rich Data: Deeper Insights and First-Party Relationships
Rich data represents information that marketers gather from more than one location feed, and it requires access to the kind of first-party permissions that consumers can grant us via SDKs and other direct-app integrations. At this level of detail, marketers now have the opportunity to create the kind of control groups that precisely match the ad-exposed users we want to measure. We can see all the demographic details, visit history, information around factors such as time and weather and location — and rich data gives us the tactic used as well.

Conversely, we know we almost never get to this deep level of insight via ad exchanges — there’s very little rich data there. Meanwhile, third-party data sources might give us details such as demographics and user-visit history, but they won’t provide us the tactic-level information.

Getting rich data takes more work. And it takes more resources. But when we have it in hand, we’re able to access the insights that fuel better measurements, and out of this process we also open a new opportunity to address campaign tactics. Meaning, we can narrow down our tactical options to the ones that work best in a given location, with a certain type of consumer, at a certain time of year — accounting for all these factors and more. If geofencing isn’t working well, for example, our rich-data methodology for truer lift can show us that targeting qualified audiences instead is the approach that generates the best results.

Reality-based reporting. Strategic insights. Brands deserve the antiseptic of good measurement.

It’s not always going to be pleasant, initially, the truth about incremental lift this better methodology reveals. We may well have to revise strategies to account for what we discover about our approaches, but the alternative — pressing on with tactics bound to underperform — is unacceptable.

When the industry adopts this new and clear-eyed methodology, we can get down to the real work at hand — driving results that reflect verified visits and capture incremental lift with accuracy, driving sophisticated tactics for the long term.

Control groups built on deep insights around context — and tactics — are how we’ll achieve these goals. Is it time to take a look at your control-group process?

This article originally appeared on

More Stories