Testing challenges (and opportunity) on the front lines of the marketplace

Testing has always been a valuable tool in the retail marketplace.  Retailers test new circulars, in-store displays, promotions and offers.  Consumer packaged goods firms test new products, packaging, prices and supermarket displays.  Advertisers test media mix, frequency and reach (GRPs), content and offers, and new channels.

In-market testing is valuable, but not without challenges:
     ●  High advertising and testing costs
     ●  Large amount of market noise (high experimental error)
     ●  Market instability (constant change)
     ●  Irrational, ever-changing consumer behavior

Even with these challenges, when you're spending $10s to $100s of millions on advertising each year, the potential ROI on in-market testing can be impressive.  The key is to use the most efficient scientific techniques available to maximize your insights and minimize testing time and cost.

Scientific multivariable testing is more widely used in direct marketing channels, but retail markets hold immense potential.  With the right techniques, you change many variables at once and still separate out the impact of each.  You can test dozens of variables, all at once, with greater accuracy, deeper insights, yet using the same small sample size as a basic champion-challenger test.

Recent tests have shown impressive results:

●  One consumer packaged goods test of 10 in-store elements was run in 48 stores over 8 weeks.  Eliminating unprofitable tactics and implementing the five most important changes, sales jumped 21.8%.  Standard methods would have required 316 stores or 1 year to test the same elements with equal confidence.

●  A 4-week circular test of 16 elements uncovered 5 significant effects (including layout, promotional, and creative elements). Results were worth $10s of millions in the first year implemented.

●  Testing radio, newspaper, direct mail, and other media mix variables, one retailer increased sales 8% while reducing advertising spend by 12%.  This test would have taken a year-and-a-half longer with common techniques.
 

Good sources for more information

To learn more about the techniques and see examples from other leading marketers, you can look over:

1. A recent article, Navigating the Depths of Multivariable Testing, with a good overview of the scientific techniques and benefits.

2. A retail marketing-mix case study showing details of a 10-element consumer packaged goods test.

3. A newsstand price test case study explaining the in-depth multivariable analysis of main effects, interactions, and curvature in price effects.

4. To see how these advanced tests are created (with examples of advertising tests and a 19-element retail test) look over the paper, Innovations in Retail Testing and Measurement, from the 2003 Advertising Research Federation Annual Convention.
 

Benefits of scientific testing in retail and advertising

When the time, effort, and cost of testing is high, scientific techniques offer distinct advantages.  You can test many marketing-mix elements at once, with the same sample size (run length and number of regions/stores) as a simple champion-challenger test, deeper insights into the impact of each element on its own (main effect) and in combination with others (interactions), and with a vast array of techniques available to customize your approach.

What John Caples wrote in the 1930s still holds true today:
"Test everything on a small scale before you spend money on a large scale. Testing enables you to keep your finger on the public pulse. It enables you to sense trends in advance. It enables you to separate the wheat from the chaff, the sheep from the goats, the winning ideas from the duds. It enables you to multiply the result you get from the dollars you spend in advertising."

Shifting from the test-control approach to scientific multivariable testing is like trading in Caples' Model T Ford for a new Ferrari.  The core benefits remain, but with a quantum leap forward in power, responsiveness, and value.  Even within complex, dynamic programs, you now have the freedom to learn 10-times faster than ever before. 

If "test" is an important word in your advertising programs, now's the time to leverage the power of scientific multivariable testing.

If you would like more information, feel free to contact us with any questions and to discuss your unique marketing challenges.

Back to top