Critical Testing Methods for Conversion Rate Optimization

In part I, we introduced the fundamental concepts of conversion rate optimization. In part II, we started talking about the critical methods to kick-start your optimization efforts. Here in part III, we will round out this guide by discussing testing, copy optimization, triggered emails, and the role they all play in your conversion rate optimization efforts.


Testing is the most important method of the CRO process – and the most resource intensive part as well. This is because testing requires a significant investment of time, money, and energy, in order to receive realistic results. These results can then be applied to your on and off-site campaigns to maximize your conversion rate optimization goals.

There are three types of tests that are worth performing:

  • User Testing
  • A/B Testing
  • Multivariate Testing

You do not necessarily need to perform each one, but it certainly does not hurt. So, what exactly are the key differences between each of these testing methods?

User Testing:

User testing exists to help you take the guesswork out of your CRO efforts by learning exactly how real people (who are acting as your potential shoppers) react when interacting with your site.

The reason this form of testing is so important is that everyone in your business is likely to have an opinion about the site and what they think should change or remain the same. User testing slices through those opinions to deliver an exceptional level of practical insight direct from the shoppers themselves.

The basic principle behind user testing is that users complete a series of tasks set up by the lab, and their efforts are recorded. The tasks themselves most often involve users going to a site to complete a purchase, and thus determine how easy it is to actually do so.

Below are a few examples of the questions user testing can help answer about your site:

  • What questions do shoppers want to be answered before they continue to check out?
  • What features do shoppers expect to see on a modern eCommerce site?
  • Are shoppers able to easily find key features such as site search?
  • Is the copy used for products descriptive enough for most shoppers?
  • What issues are shoppers having when filling out forms on the site?

Because user responses are recorded, the answers to these (and other questions) from user testing can help uncover areas in need of improvement across the site.

These days, thanks to the marvels of the communication age, user testing can be carried out remotely, with participants testing the site from the comfort of their home, as opposed to behind a two-way mirror.

User testing in the show "Mad Men"

User testing in the show “Mad Men”

It is important that results must be properly analyzed after testing to understand why shoppers interacted with your site in a specific manner.

A/B Testing:

At its core, A/B Testing simply involves running two different versions of the same offer, webpage, or call to action, against one another to learn which variant yields better results. Results are measured in terms of shopper engagement which is typically measured by the number of clicks, opens, etc. The wonderful thing about A/B testing is that it can be easily done “in-house” – potentially saving your business a lot of money (compared to hiring a private company for the same tasks).

A/B testing typically comes down to measuring the differences between attributes such as:

  • Font size (the size of the print).
  • Font type (Italic, Times New Roman, Comic Sans, Gothic).
  • Design elements (color, imagery, sizing).
  • Copy (the words used to convey a message).

A typical example would go as follows: You created two different versions of what is otherwise ostensibly the same webpage. Next, you would redirect half of the incoming traffic to one page and half to the other. Finally, you record the engagement to see which of the two pages received higher engagement based on a predetermined indicator such as a specific CTA or object you were expecting visitors to interact with.

Another example would involve running an email campaign with two different subject lines sent out to shoppers. This show below as “A” and “B.”

B) Open for incredible savings on the latest spring collections!

B) Want to save money on our spring collection? Check inside!

When A/B testing is scaled up across 1000 email recipients, it becomes very easy to determine which of the messages is more appealing on a broad scale.

Featured below are the steps for a successful A/B testing process.

  • Collect Data: Use your analytics to find areas that need improvements, such as pages with a lower than average CVR, or high exit rates.
  • Choose Goals: Which metrics are most important to improve? Page engagement, the number of items added to cart, increased email sign ups etc.
  • Hypothesize: How can you improve any one specific feature? Generate ideas and prioritize based on the odds of success.
  • Use Variation: This can be anything from different images, page layout, page length, etc.
  • Test: …And then do some more testing as time goes on.
  • Analyze Your Results: Did the results match your hypothesis? Furthermore, is the test statistically significant?

The last part is critical to the process because testing will succeed or fail based on how the results are interpreted. It is important to learn why a variation succeeded or failed, as this guides all your subsequent testing. Furthermore, there needs to be statistical significance to weigh upon the test results. Did the test run for enough time? Were there enough clicks or conversions?

Because of the ease of setting up A/B tests, it is important to keep in mind that there are some drawbacks. A slight change in your test (with regards to anything from the font to the imagery) can make a significant difference in the outcome. Reliable test results take time to collect; so, to be sure, A/B testing should be one part of the overall testing process.

Copy Optimization:

Copy plays a very important role in A/B testing and conversion rate optimization. Creating effective copy is about more than just good writing. This section delves into the elements of style. Effective copy is an essential part of connecting with your audience and reaching your desired optimization goals.

All websites rely on a plethora of different copy types such as taglines, headlines, email subject lines, product descriptions, sales copy, and micro-copy used for a CTA. Simple changes to web copy can produce improvements in overall site engagement.

Here are a few copy types that are important to test.

  • Headlines: Test different wording and font size.
  • Copy Length: The length of the copy is important. As a rule, shorter is better – it’s not the size of the sentence, but the impact of the words used. Concise and to the point is the way to go.
  • Copy Style and Format: Can visitors read and easily digest the copy used? For example, small sentences can make it easier for viewers to digest a lot of information about a product.
  • CTA Text: Variations of the micro-copy used in your CTA’s can be tested to decide the most effective variant for regular usage in both on and off-site campaigns.

Multivariate Testing:

While A/B testing measures two variations against each other, multivariate testing tests changes to a range of elements on a page. For example, different images and headlines on a landing page could be changed, which will then lead to six or more variations of the page being served to visitors. Just like with A/B testing, a structured approach is the best way to go about this. You need to have a clear goal and success criteria. This means you need to figure out the sample sizes for a reliable test. Finally, it comes down to interpreting the results.

The frequency of Testing:

An effective CRO strategy involves continuous improvement to deliver the best results. Think of CRO as a muscle which needs a regular mix of exercises to keep the best form and function. Because shopper behavior is constantly evolving, what works today may be irrelevant in half a year from now.

Furthermore, quality over quantity is key when it comes to testing. This is to ensure that the quality of the data used for analysis is reliable and delivers the correct results. So, for best results, A/B testing should be done at least twice a month for a duration of 7 days. Alternatively, you can take the number of conversion per month and divide it by 2000 and use that number as an indicator of how many tests you should be running to achieve statistical significance.

Common Problems Discovered Through Testing:

Featured below are some of the most common problems that are discovered following a spate of tests.

  • Form Usability: Form usability is a huge part of the online experience, and one of the easiest spots to improve for CRO. Poor form design just pisses people off and makes them abandon without a second thought.
  • Choice Paralysis: Choice paralysis is a well-documented issue in eCommerce. Some websites use filters, sorting tools, and other search tools to deal with the issue. But these are not always super effective. The challenge is to introduce filtering as part of the natural selection process without making it overly obtrusive
  • Moving Between Channels: Multichannel retail has the disadvantage of not being smooth because when a shopper starts on their phone the transition to a laptop isn’t always great. Using a “send me my cart” popup can alleviate this issue by allowing shoppers to send themselves a link to their cart and they can continue shopping up from any device they use
  • Lack of Key Information: Key information includes things like delivery options, returns, purchase (and taxes). A lack of visibility when it comes to simple information is shown to result in a 23% cart-abandonment rate overall.
  • Too Many Distractions on a Page:

Sites need to be clear about what they want to achieve from a single page, and many distractions can confuse visitors and reduce the page effectiveness. Common problems that result from having too much on a page include:

  • Difficult Navigation: If visitors cannot quickly find what they are looking for, the likelihood of abandonment increases significantly.
  • Annoying Pop-Ups: Popups that take up the whole page (and do not provide a compelling reason to engage with) that are difficult to close out simply frustrate visitors to no end.
  • Slow Page Loads: A one-second delay in page response can result in a 7% reduction in conversions.
  • Poor Mobile Experience: Around half of eCommerce visitors use mobile. Make sure the site is optimized for mobile browsing.
  • Mandatory Sign-Up: Sign-up (or site registration) is an unnecessary barrier to completing a purchase that just increases the chances of abandonment. Allow for simple guest checkout to reduce cart abandonment and increase conversions.

Event-Triggered Emails:

Emails play a critical role in most conversion rate optimization efforts. Event triggered emails are in use by at least 90% of companies because they can be easily implemented. Furthermore, they help keep brands in the shoppers’ mind and provide plenty of opportunities to drive repeat purchases.

Here are a few examples of Event-triggered emails and their objectives:

  • Welcome emails: For new shoppers following their registration.
  • Confirmation emails: Sent out following an order to confirm the details.
  • Tracking emails: Sent to update the shopper on the delivery status of their order.
  • Reactivation emails: Sent to shoppers who have not made a purchase in some time.
  • Date Specific emails: Sent on a shopper’s birthday, the anniversary of their site registration etc.
  • Reorder Reminders: Useful for perishable items such as food, office, and medical supplies that are likely to be depleted within a few weeks of purchase.
  • Browse Abandonment emails: Sent after a shopper views an item and leaves the site. These emails have an 80.9% higher open rate and 50.5% higher CTR than traditional emails.
  • Cart Abandonment emails: sent after a shopper adds an item to their cart and then leaves. The cart-abandonment rate is around 76% on average for most eCommerce merchants.

The emails detailed above can be easily automated and sent in response to a pre-determined event or action. The effectiveness depends on the quality of data, and the content included in the email. It is important to point out that shopper data is a necessity to achieve the best results. They key to success is the use of relevant data to recommend the right product based on what the shopper was looking at.

In Summary:

This guide started by talking about the importance of CRO, and how it works. Next, it moved on to discuss critical methods used in the CRO process. Finally, it concluded with testing, copy optimization, and triggered emails. So, what are the key takeaways from all of this?

  • A structured approach is an absolute necessity. Make sure all the key players in your business are on the same page from day 1 regarding CRO efforts.
  • Shopper Journey Analysis will help you better understand where and how shoppers interact with your brand.
  • Segmentation will help you split shoppers into unique groups for retargeting.
  • Personalization will ensure those shoppers get the right message every time.
  • Surveys and Shopper Feedback through Social Media can help deliver critical information directly to you.
  • Testing needs time, energy, and patience to display the correct results. Do not rush it, and always pay attention to the numbers.
  • Triggered Emails are incredibly useful when personalized.

Individually, everything covered in this guide can help boost your sites CVR. But for best results – you should apply all the methods and techniques discussed to experience significant growth in the long run.

Datacrushers 14-day free trial


Founded in 2015, Datacrushers uses Machine Learning and A.I. along with NLP to identify and recover revenue loss, cart abandonment and discover new revenue sources across any site.  The revenue discovery platform completes the deep ongoing analysis of eCommerce websites by monitoring the three main focal points of any site: The User, Site, and Product.

Unlike traditional “cart abandonment platforms,” Datacrushers does not require shoppers and customers to be logged-in to conduct both on and offsite campaigns. We use a wide range of data-driven and analytics based conversion tools to target the shopper at the right time with the most accurate and effective campaign to drive the sale.

Datacrushers is platform, language, and currency agnostic and requires only a few lines of code to get started therefore delivering an ultra-fast go-to-market with minimal set-up time and tech intervention.

Based out of Jerusalem, Israel, Datacrushers has clients worldwide including,  The US, China, Russia, UK, Germany, and more.

By | 2018-03-27T17:26:35+00:00 February 20th, 2018|Conversion Rate Optimization, Guide|Comments Off on Critical Testing Methods for Conversion Rate Optimization