How large do my test mailings need to be?
Answered by Brendan Talty
Question:
How large do my test mailings need to be?
Answer:
Good question, but just a bit off target. It's not the mail quantity that matters when you want to ensure that results are valid for rollout consideration, it's the number of RESPONSES.
Consider this: A catalog client wanted to make sure that the results of a price test would be significant enough to roll out to the entire base. Their response averaged 12% over the years (true, believe it or not). So how many packages to mail? While there are computer models that will compute the required test quantity for you, I use a rule of thumb that 300 responses per test panel is enough to make a test significant at the 95% confidence level (the industry standard in direct marketing).
So...300 divided by 12% yields a mail quantity of 2,500. Now go ahead and plug in your expected response rate to determine what your test quantity should be next time you're in the mail.