Scientific testing - managing a scientific test

Managing a scientific test

"Before we acquire great power, we must acquire wisdom to use it well."
– Ralph Waldo Emerson

Scientific testing

Introduction

How to test many variables at once

Creating large test designs

Variety of test designs and techniques

Managing a scientific test

Sample size

Scientific testing offers you the power to see your marketplace more clearly. But with greater power comes greater responsibility. Like replacing your kid’s bicycle with a Ferrari, moving from split-run to scientific testing requires much more skill and experience to use it correctly.

Managing the Test = guiding the process from start-to-finish

A experienced “guide” will let your marketing team do what they do best, streamline the process so the technical complexity does not get in their way, and guide the team through the detailed execution (managing every detail so they don’t have to). An expert must have a solid understanding of the marketing program, experience with sophisticated statistics, and skill in leading a team from start-to-finish. In particular, a guide should:

1. Leverage your team’s skills and ideas
Building a house, you might say “we want that style, these rooms, and this view.” The architect/builder is responsible for making it happen. He might suggest some changes based on cost or physical constraints, but you probably don’t care about the spacing of the roof trusses or clearance for the plumbing, as long as it’s done right.

The same is true for testing. You should have the creative freedom to test what you want, with minimal constraints. An experienced guide can give you the freedom to leverage your expertise, help focus it on the best test elements, and work diligently to remove any physical barriers to your success.

2. Simplify the statistics
A true expert can simplify complex concepts and make the statistics accessible to every marketer on your team. Clear charts and pictures are always better than lists of numbers and pages of analyses. Often a few alternatives should be discussed, but a good guide will make the statistics look easy and answer any questions along the way (testing should not be a “black box”).

3. Speed the process
As is true in marketing, effective testing involves hundreds of details that need to come together in just the right way to achieve the optimal outcome. Your test expert must understand all those details while speeding the process and giving your team clear guidance. Some steps need careful attention, others you can speed through.

4. Increase your ROI
An expert knows how to squeeze greater potential from every test. In the selection of test elements and levels, the choice of test design, test execution, and the analysis of results, an experienced guide can bring out the best in your team.

Now’s a good time to look over case studies & articles to see some real-world examples, or continue to the final topic and learn the truth about sample size.