Testing Your Marketing: When Is As Important As What

I was talking with a client about a current mailing they were running. They told me that letter B was converting at a better rates than letter C. When I asked how they ran the test I was baffled at the response. I thought this was common knowledge. Ultimately, it became a lesson in market testing factors.

When you test your marketing piece is just as important as what you test. Here’s what they did in a nutshell.

They took a list of prospects and cut it in half. They sent sales letters to a group A and measured the results. They sent group B a letter with a different headline and tracked the results. This is not bad. It’s a good way to start testing your marketing however, the second group was sent letters one month after the first group received their letters.

The easiest way to test your marketing is to test one variable at a time and measure the results. If headline X pulls better than headline Y then use headline X as your control and continue testing to see if you can beat those results. This process is called split testing and is nothing new to anyone with direct marketing knowledge.

One factor commonly overlooked when performing split testing is time. The split test has to be run at the same time to be considered an effective test. If you send one mailing this week and another mailing four weeks later then you’ve essentially thrown in another testing variable called "time."

“So, what’s the big deal it’s only a couple of weeks difference?” he asked. Imagine mailing a mortgage marketing piece right before the recent financial meltdown and a month later after all the headlines hit the news. If you’re selling earthquake insurance you’ll see a difference in response from a mailing before and a mailing after the recent Haiti and Chile earthquakes. As you can imagine, awareness in society changes over time and is another piece of the marketing puzzle you need to put in place.

What About Online Marketing?

The same rules apply online. If you’re marketing through email, most email responders allow you to perform split tests. Run tests changing the subject to test open rates. If your email system allows you to run more than two then test two different offers with each headline to test click-through rates too. Make sure it’s the same mailing and you’re not sending one email blast today and one email blast tomorrow. (that’s time and sending a second mailing to the same list, another variable)

What if you’re testing landing pages or opt-in forms? You want to run split tests as people visit the site. Testing one page for 3 weeks and another page for 3 weeks will not give you accurate results. Time can skew your statistics plus your source of visitors could change.

Most email providers offer a form of split testing your emails but, do you use it? If you have a sales page, opt-in form, squeeze page, etc. are you split testing to see what works or are you going by what the gurus tell you works? (I recommend ActiveCampaign.)

Marketing Is Not A Science

Marketing is not a science but it does require one to step back, analyze the variables and, look at the results. The only scientific rule I know that works every time is test and measure.

What's your experience?