• http://www.facebook.com/karen.marchetti1 Karen J Marchetti

    Also important to run the test of statistical significance before drawing conclusions about which version performed better.  In the Offers test above, for example, the results aren’t statistically significant, which means if the test was replicated again, you may not get the same results.

    The range of possible response rates for the first offer is 2.08% – 3.06%, while the range of response rates for the second offer is 1.64% – 2.52%.  There is overlap between these ranges, indicating the results aren’t statistically significant, and so can’t be relied upon.

  • http://twitter.com/psiphon B. Hopkins

    I think to take the most advantage of increasing your email conversion rates, you need to know your overall online marketing strategy and know where email fits into it.  So many just do bits and pieces here and there and don’t allow themselves to see the full effect of what their efforts will bring them.  For instance, how and where your list member signs up will influence how effective your emails are on them.

  • http://www.seoskeptic.com/ Aaron Bradley

    B. Hopkins – Absolutely.  The post here had a narrow focus on a small set of metrics, but an effective overall email conversion strategy has more pieces to it than this (list segmentation and email personalization, for example, come to mind).

    Karen J. Marchetti. – Thanks for pointing that out (and providing such exact ranges!).  I should have made it clearer that the numbers used in the post were only broad examples.  The results of any A/B test should always be interpreted in the light of the test’s statistical significance.