CRO Case Study #4: (Revenue Optimization) 100 Day Trial Mattress Outperforms Control

Revenue Optimization Increase of 22% - What We Tested

In this split test, we hypothesized that by increasing the time a customer can test out the mattress, up to 100 days versus just the standard return policy, the 100 day return policy should outperform conversion-wise vs. the 30 day control.  In CRO we never guess, we test our hypothesis with actual data and keep the winning variation.  

As you can see on the graphic below, the original version of the product page on the left side of the graphic (Control) has no risk-free trial expressly stated.  On the right hand side of the graphic, the variation, shows the new verbiage with a graphic and description of the 100 day trial policy below the green "Add to Cart" button.  

Mattress Variation vs. Control CRO

100 Day Trial vs. Control

Test Results with AB Tasty 

Using our favorite conversion rate optimization software, AB Tasty, we tested the variation of the page against the original control page.  The test results are impressive.  The variation page outperformed the original design by over 21% and resulted in a bump of $5,200 in increased revenue.    

CRO AB Tasty Variation vs. Control

CRO - AB Tasty test results (Variation wins!)

Test Results with Google Analytics

Here's a screenshot of the same test in Google Analytics:  

CRO Google Analytics

Google Analytics shows increased conversion by 26%

Summary

Here's a summary of the A/B Testing:

  • The A/B Test ran in two phases Jan 19th, 2017 to Jan 24th, 2017; and then from Feb 8th, 2017 to Mar 9th, 2017.
  • Only one variation was created and tested against the control page.
  • The test was ran on all devices (mobile, desktop, etc.)
  • The test showed increase of around 22% with a $5,000 uplift in revenue.
  • After learning about the risk-free trial scheme, visitors were more confident in moving forward and completing the purchase.
  • As a result Number of transactions and Average order value increased.
  • The test needs to be implemented on the site for 100% of visitors