I see a very common mistake made by webmasters, site owners and even conversion optimization firms when it comes to conversion rate optimization.
A Large Number of Variations Is Bad
You don’t want to test hundreds of page variations if a site has lower traffic volume. Each variation needs at least several hundred visitors and at least 10 days of testing to get correct results. If you have 300 page variations then you will need around 100,000 visitors to that page and at least 10 days of running the test to really know what version is best.
We were testing a page for a client recently with only a few variations and one variation had a 300% improvement over the original in the first couple days of testing with over 500 visitors to each variation. The testing software said enough data was used to determine which variation was the winner.
When it comes to split testing, you don’t want to miss the best path early on.I could have ended the test at that point and shouted that I got a 300% increase, but that is not entirely true. I left the test running for at least 10 days and sent 4000 visitors to each variation. In the end, the variation with a 300% improvement turned out to be a 25% improvement.
Imagine if we had hundreds of variations and just threw a few visitors to each variation and let the software tell us the winner after a couple days. The results would be very inaccurate. This is why you will need to extend your tests to 10 days and send at least several hundred visitors to each variation. You will see a much more clear picture and see who the real winner is. The more variations and the fewer the visitors, the more inaccurate your results will be. You could easily see loosing variations as winning variation in the first few days of testing. I have seen this many, many times.
When it comes to split testing, you don’t want to miss the best path early on. You could easily head down the wrong path and miss the path to the highest revenue. So, take your time with initial tests.
If you don’t have traffic volume to test hundreds of variations, then you need to research other sites to see what they are doing. What works on other sites might not work on your site, but if you think a specific change will fit on your site and makes sense, you can add it to your testing. Always start testing with your own eyes. Look at your site from a customer’s point of view and ask your customers their opinion.
You can’t start adding several variations just for the sake of testing or your first test will never get a true result if you don’t have the traffic volume. Use A/B or A/B/C testing initially, then do small multivariate testing to tweak the page. The number of variation used for multivariate testing should depend on traffic volume. You need to have enough traffic for each variation or the test is worthless.
I see many companies getting carried away with conversion testing and they wonder why their revenue does not increase when all of their testing is showing massive improvements in conversions. Test results can be way off if they are not done right.
Contact Us Form
