Geo AB Test

AB test on different geo zones

AB Tests are powerful tools to analyze the effectiveness of a media plan. But it must follow defined steps to reach its goal.

What are geo AB tests

We’ll define a control group (= 1 or several geographical zones) and an exposed group. The control group won’t see the ad. The exposed group will.

We’ll analyze the results of both zones and see if the advertising has an impact on sales (or revenue).

Several steps

If the process seems quite easy, we must take precautions

Step 1
Design control & exposed groups
Design control & exposed groups

We’ll design groups that look like each other.

The temptation to take two different markets is high. But it can bias the analysis: if external events occur in one market and not in the other one, you can start the process over again.

Therefore we make sure both groups are comparable and homogeneous, to ensure we won’t have biases in the results.

Step 2
Analysis
Analysis

We design a test plan, which contains:

Length of the test

A few days, weeks, months?

Strength of desired effects

If you are looking for a +100% effect on sales, the test will be pretty short, you’ll know quickly if you can double your sales with a marketing action. But if the effect is less obvious, you’ll need a long period and a long exposition to be sure it’s due to your marketing strategy. - budget needed: based on impressions and CPM, we’ll deduct the minimal investment needed.

Step 3
Modelisation of both groups
Modelisation of both groups

We’ll forecast data of both groups, first check the model is correct on the control, and secondly if real sales are increasing on the exposed.

Step 4
Compute the increment
Compute the increment

When the test is over, we check the results and deduct the incremental value of your marketing actions. It’s time now to enlarge the actions on a bigger scale.

Bonus : calibration of the Marketing Mix Model

The insights found during a geo AB test can be used to calibrate a Marketing Mix Model. It is a process that increases the accuracy of the MMM, making both analyses very complementary.

Do You Have Any Questions?