75% More E-commerce Orders: A/B Test Case Study

Significant Sales Increase Through Conversion Optimization

A leading conversion rate optimization (CRO) consultancy, The Conversion Wizards, was commissioned to enhance the conversion performance of a large, multibillion-dollar organization.

Our approach centered on data-driven research to refine the webpage, followed by the implementation and analysis of an A/B test.

The variant ultimately deemed successful, and designated as “radical,” yielded a substantial 75% uplift in sales figures.

Interestingly, the initial and double-control versions of the page were, in fact, structurally identical.

Establishing a Reliable Baseline

To guarantee the validity of our findings, a double-control was incorporated into the testing process, as is our standard practice.

The average performance of these two identical control pages was used as the baseline for calculating the improvement.

This analysis revealed the 75% increase in sales with a high degree of confidence – 99% statistical significance.

Google Optimize Results

Below are screenshots captured from Google Optimize, illustrating the A/B test results.

A complete image of the original page can be accessed through this link.

The full image of the winning page variant is available via this link.

Understanding the Initial Research

Prior to detailing the modifications that resulted in increased conversions, a review of the underlying research is essential. This is a vital component of the optimization process, yet frequently overlooked by conversion rate optimization (CRO) specialists seeking to understand why potential customers aren't completing purchases.

Both visitors who left the site immediately (bounced) and those who enrolled in the Subscribe & Save program were included in our research. A key question posed to the bouncing visitors was: “What prevented you from making a purchase today?”

The survey findings are illustrated below:

Analysis of the survey responses revealed the primary reasons customers did not proceed with a purchase. The data, highlighted in red and violet, indicated that we could potentially address two significant concerns:

• Perceived high cost or unaffordability of the products.
• Uncertainty among customers regarding the appropriate product formula for their needs.

The following word cloud visually demonstrates the prominence of price as a barrier to purchase:

Designing Pages for Increased Conversion Rates

Achieving a significant increase in sales through your webpage design hinges on effectively addressing two fundamental inquiries.

Understanding Purchase Motivations

Determining the precise reasons customers choose your product, articulated in their own language, is crucial. Frequently, the motivations behind customer purchases differ substantially from initial assumptions.

Consider the example of Thomas Edison, who initially envisioned his phonograph being utilized for recording legal testaments!

Analyzing Non-Conversion Factors

A typical conversion rate for thriving online retailers is approximately 3% across all traffic channels. Data from Capital & Growth indicates that a staggering 95% of newly established Shopify stores fail to generate any sales.

Gaining insight into the reasons visitors choose not to purchase can be profoundly impactful.

After establishing clear answers to these two questions, the next step is to construct a page that prominently highlights the factors driving customer purchases while directly confronting and resolving any stated objections.

It’s remarkable how often conversion rate optimization (CRO) specialists commence with complex artificial intelligence and machine learning techniques, despite the simplicity of this foundational approach.

Addressing Price Concerns and Reducing Customer Uncertainty

Our investigations revealed that a successful strategy needed to focus on two key objectives: resolving price-related objections by demonstrating enhanced value, and minimizing any potential confusion for the user.

To achieve these goals, the following strategies were implemented:

A Headline Focused on a Distinct Benefit and Alleviating Worries

Customer surveys within the Subscribe & Save program consistently highlighted the desire to “never run out of formula.” Consequently, this phrase was incorporated into the headline – a crucial component of any landing page.

Customer feedback included statements like, “I never want to experience running out of formula again,” and “I was forced to visit Walmart late at night because my formula supply was depleted.”

Quantifying Potential Cost Savings

While stating a “10% discount” is acceptable, presenting this as “save up to $424 per year” proved to be more impactful.

Presenting Benefits in a More Persuasive Manner

The image below illustrates a more compelling presentation of the benefits associated with the company’s Subscribe & Save program.

Image Credits: Conversion WizardsIn contrast, the original page offered only standard features, such as schedule adjustments, product additions/removals, and billing/shipping updates – functionalities customers already anticipate.

Employing a Call to Action with Reduced Commitment

Given the prevalence of subscription boxes and challenging cancellation processes, customers are often wary of new subscriptions. Therefore, utilizing a call to action like “Try Subscribe & Save Now” is beneficial. Reassuring customers with “Easily Cancel Anytime” can significantly increase engagement.

Streamlining the Product Selection Process

By simplifying product categorization, as shown in the screenshot below, we successfully reduced customer confusion and mitigated choice overload.

Previously, clicking “Shop Now” presented customers with 39 products arranged in a 3×13 grid, many of which appeared remarkably similar. This presented a cognitive burden.

Collapsing the FAQ Section

To prevent information overload and reduce unnecessary page length, we implemented a collapsed Frequently Asked Questions section.

Incorporating Social Proof

We integrated various forms of social proof, including the brand’s substantial following on Facebook (500,000+ likes), a subscriber base exceeding 3,000, and positive customer testimonials.

Offering a Low Price and Satisfaction Guarantee

Survey data indicated that customers often anticipated lower prices from competitors like Walmart and Amazon. Therefore, we included a low price and a satisfaction guarantee to address this perception.

Resolving Confusion and Eliminating “Click Rage”

Heat maps and video recordings revealed that visitors frequently clicked on the original screenshots, expecting to initiate a transaction. To rectify this, we displayed the screenshot within a device frame (as shown on the right below), effectively eliminating these misdirected clicks.

Comprehensive Redesigns Versus Isolated Element Testing

While the precise impact of each individual component on sales growth remains unclear, and the potential negative effects of some alterations are unknown, the revised design demonstrably increased orders by 75% overall.

Substantial improvements are more frequently achieved through comprehensive redesigns that integrate numerous variables, as opposed to testing single elements in isolation.

A further benefit of this approach is a reduction in the time and website traffic needed to attain statistically significant test results.

It is widely understood within the conversion rate optimization (CRO) field that a test failing to produce actionable insights represents a significant expenditure of time and resources.

Such inconclusive outcomes can frequently lead teams to reassess the perceived value of CRO initiatives.

The Efficiency of Holistic Changes

Focusing on multiple changes simultaneously accelerates the testing process. This is because the combined effect is often larger and easier to detect.

Instead of painstakingly isolating each variable, a broader approach allows for quicker identification of impactful improvements.

Statistical significance is reached faster, enabling more rapid iteration and optimization.

Avoiding Wasted Effort

Tests that don't yield clear results can be demoralizing. They consume valuable time and budget without providing useful data.

By embracing more substantial redesigns, CRO teams can minimize the risk of inconclusive tests and maximize their return on investment.

This strategy helps to reinforce the importance and effectiveness of conversion optimization efforts.