Tip:
Highlight text to annotate it
X
There’s a theory in website testing that when two or more tests run concurrently that
the results of each test are invalid. But it’s not actually true. In a world where
each person is exactly like everyone else in every way, we’d only need a sample size
of two to conclusively measure the impact of a test. But visitors to your website are
all different.
Two people may live in Chicago and have come to your website via AdWords, but they still
earn different salaries, drive different cars, and will take different vacations this year.
In a clinical trial, we’d be able to ask hundreds of questions to weed out these differences.
But online markers can’t actually do that. For all the control that markers want to have
in website testing, there’s still many factors that we neither know about nor can control.
And many of these factors can wield significantly more influence on our results than what we’re
actually testing on the page. And that’s exactly why concurrent independent tests are
still equally valid. The net effect is exactly the same as there being just one more difference
between the visitors and your test.
Read more: http://monetate.com/video/monetate-monday-minute-running-overlapping-tests/#ixzz28GYJRF11