04 Sep 2018  |  Product

Moving On to our New Checkout A/B Testing Campaign

2 minute read

We're stopping a long-running A/B test of our checkout to start our biggest A/B test yet, focusing on gradually introducing the new checkout we've building from the ground up.

At Paddle, we use A/B testing and experimentation to help us make the right decisions on our checkout. We’ve had a long-running A/B test which randomly assigned buyers to different variants of our checkout. These variants changed the UI and merged steps around the payment methods and card details step in the payment funnel.

It is now time for this test to come to an end. Using the data from the months this test has been running, we’ve determined that none of these variants converted any better than the original “multipage” variant. Because of this, as of September 12th 2018, all checkouts will now use the multipage variant (even if you’re forcing another checkout variant).

Why are we stopping the test now?

We need to make way for our biggest A/B test yet. One of our biggest projects right now is to rebuild our checkout from the ground up. Aside from a few UX improvements along the way, the main objective is to use more modern technologies, making it quicker and easier to add new features in the future.

Over the next few months, we’ll be starting a gradual rollout of an experiment which tests our new checkout against the existing one. This will be rolled out based on the features used. We’ll start off with checkouts that use the most basic features (like card payments and PayPal), moving onto checkouts that use our more complex feature sets (like price overrides and custom pay links).

There is nothing you need to do in order to get the new checkout. It’s built to be completely compatible with your existing implementation. You may see it on your site over the next few months. Although the UI is intentionally similar to the existing checkout, it’s fairly easy to spot by looking at the language selector, which has been moved to the bottom-left corner of the checkout.

Old Checkout

Our old Checkout

New Checkout

Our new Checkout

What’s happening next?

Once our A/B test is complete and the new checkout is live to 100% of traffic, we’ll begin focusing on improving it. Primarily, we’ll be looking at improving conversion rate and average order value.

Immediately after this test ends, we’ll be running an A/B test targeted at our new way of applying a coupon or tax identifier to the checkout. Many of you requested for these buttons to be integrated into the checkout and we agreed, so they’ve been moved underneath the price. Initially these will be displayed as links, but this test will see if a button gets more interaction from buyers. Here’s a sneak-peek:

Another big focus for us for the remainder of the year (and beyond) will be payment methods. We are already building a roadmap of the ones we should be adding first, especially local payment methods for specific countries.

We’ll keep you updated on progress. If you have any questions, please contact us at sellers@paddle.com.