WordPress A/B testing: plugins, theme split tests and WooCommerce revenue

WordPress A/B testing plugins compared

Prices below were read from each vendor's site on October 11, 2026. Nelio A/B Testing has a free version on WordPress.org with unlimited tests and a capped number of monthly page views. Its Basic plan is US$44.85/month billed annually for 5,000 tested page views a month on one site, and Professional is US$171.35/month billed annually for 50,000 views across 5 sites. Nelio's pricing page says VAT is included where applicable, so the pre-tax price may be lower. It tests pages, headlines, WooCommerce products, themes, templates, CSS and JavaScript, and includes heatmaps.

Thrive Optimize is an add-on to Thrive Architect, sold together at $199 for the first year for one site and renewing at the $399 list price. It only tests pages built with Thrive Architect, not blog posts or custom post types, and its goals are page visits, opt-in form submissions and revenue. Split Hero is a WordPress-only service from $29/month for one active campaign, $49 for three and $99 for ten, and it splits traffic with 302 redirects. Optibase, best known on Webflow, also ships a WordPress plugin as a zip download, with a free plan for 10,000 tested users and one active test, then Basic at $79/month or $66/month billed annually.

Two pricing models hide in that list. Nelio and Optibase meter tested page views or tested users, so the bill rises with traffic. Split Hero meters active campaigns instead. Otter meters neither: Starter is $39/month for one site with unlimited visitors and tests. We make Otter, so weigh that accordingly, and see it next to more tools in our A/B testing tools comparison.

How the Otter WordPress plugin works

Otter's plugin, Otter A/B Testing for WordPress, is a zip you download from the install page in your Otter project and upload under Plugins > Add New > Upload Plugin. It isn't listed on WordPress.org. It needs WordPress 6.0 or later and PHP 7.4 or later. Once you paste your project API key, it prints the Otter snippet early in every page's head, with the same anti-flicker style and 3-second failsafe as the hand-pasted version, so you never edit theme files.

With WooCommerce active, the plugin also tracks the store. It sends a purchase with the order total, currency and items from the order-received page, using the order data WooCommerce renders rather than prices scraped from the page. It also sends product views, searches, add-to-cart events from both AJAX buttons and product forms, and checkout started. Otter ignores repeat purchases with the same order ID, so a refreshed thank-you page never double-counts. The plugin declares compatibility with WooCommerce's High-Performance Order Storage. By default it also sends the customer's billing email with each purchase; turn that setting off if you don't want emails stored in Otter.

Those events become goals in any test. Each variant then shows conversion rate, total revenue, revenue per visitor and average order value, with frequentist or Bayesian statistics at an 80, 90, 95 or 99% threshold.

Whole-theme split tests on WordPress

Testing a redesign is the hardest job in WordPress A/B testing, and it's the one Otter's plugin was built for. You install the new theme without activating it, approve it in the plugin settings, and click Create theme test in Otter. That opens a prefilled test with your current theme as the control. For each assigned visitor, the plugin swaps the theme WordPress renders for that request, so the site's active theme never changes for anyone else.

The rules are strict on purpose. A theme test covers the whole site, so it uses a site-wide URL with no extra URL rules. Each project can run one active theme test at a time. A visitor's first assigned page view reloads once to apply the theme. Theme tests aren't available in the Preview Bar, and the custom CSS and JavaScript fields are hidden for them. Don't switch the active theme or edit either theme while the test runs; pause or complete the test before deploying theme changes, so each group keeps one stable design.

Caching: why WordPress split testing breaks

Full-page caching is the most common reason a WordPress split test fails quietly. A cache plugin, your host or a CDN stores one copy of the HTML and serves it to everyone, so a tool that picks the variant on the server ends up serving whichever variant was cached first. WooCommerce has the same problem with carts: its documentation says Cart, My Account and Checkout must be excluded from page caching, and it lists cookies such as woocommerce_cart_hash and wp_woocommerce_session_ that caches should respect.

Otter's visual and split URL tests don't need different HTML. Every visitor gets the same cached page, and the browser applies the assigned variant after asking Otter for the visitor's test settings. Theme tests are different, because the server renders a different theme. The plugin sends private no-cache headers for assigned visitors, but a cache that answers before WordPress runs never sees them. So every full-page cache and CDN in front of the site must bypass requests that carry the otter_theme_assignment cookie, and the plugin keeps theme testing switched off until an administrator confirms that.

Performance plugins can cause a second problem. If a plugin delays, defers or combines JavaScript, exclude the Otter snippet so it loads in the head as written. WooCommerce's own caching guide also recommends against JavaScript minification in WP Rocket.

What to test on WordPress and WooCommerce

On content and lead-generation sites, start with the pages that bring in money: service pages, pricing pages and landing pages. Test the headline, the call to action, form placement and social proof. Otter's visual editor works on the rendered page in the browser, so it doesn't depend on whether Elementor, Divi or the block editor built it, though complex builder widgets deserve a careful preview before launch. Otter has no form-submit goal type, so track leads with a pageview goal on the thank-you page, a click goal on the submit button, or a custom event. The landing page A/B testing guide covers lead goals in more depth.

On WooCommerce, start with product pages and the cart: image order, delivery and returns information near Add to cart, trust badges and cross-sells. Use revenue per visitor as the primary goal, since a change that lifts orders can still lower order value. If you're testing a redesign, use a theme test rather than rebuilding the same pages twice. Size every test first with the sample size calculator: a theme test gets all your traffic, but a small store may still need several weeks.

WordPress A/B testing and SEO

Google's guidance on website testing comes down to four rules. Don't cloak. Use 302 (temporary) or JavaScript redirects when a test sends visitors to another URL. Put rel="canonical" on alternate URLs pointing at the original. End the test promptly. WordPress tools follow these in different ways: Thrive Optimize noindexes variations and points their canonical to the control, while Split Hero uses 302 redirects and tells you to set canonical URLs in your SEO plugin.

With Otter, visual tests keep the same URL and change the page with JavaScript, which Google lists as a valid approach. Split URL tests redirect in the browser, so set the variant page's canonical to the original in Yoast, Rank Math or whichever SEO plugin you use. Theme tests keep every URL the same, and Otter doesn't assign known bots to a variant, so crawlers see your active theme. When a test ends, ship the winner and delete the losing variant pages.