What A/B testing software does
A/B testing software splits visitors between two or more versions of a page or feature, keeps each visitor in the same version on return visits, counts the goals you care about, and runs a statistical test to tell you whether a difference is real or noise. Split testing software, A/B testing platforms and experimentation platforms are mostly names for the same job. What separates the tools is where the split happens and where the numbers are calculated.
Client-side tools run as a script in the visitor's browser. The script decides which version a visitor gets, then changes the page or redirects to another URL. Otter, VWO, AB Tasty, Convert, Crazy Egg, Mida, Zoho PageSense and Optibase work this way, and it's why a marketer can build a test in a visual editor without a code release. The trade-off is that the browser has to load the script before it can show the right version, so these tools manage flicker by briefly hiding the page or by asking you to load the script early in the page head.
Server-side tools make the decision in your own code through an SDK, usually behind a feature flag. The server sends each visitor only their version, so nothing flashes, but every test needs a developer and a deploy. GrowthBook, PostHog, Statsig, Amplitude Experiment, Optimizely's feature experimentation and Firebase Remote Config work this way, and most also offer a client-side editor for simple web tests. Convert's Pro plan, Kameleoon and Adobe Target sell both kinds.
Warehouse-native describes the analysis rather than the split. GrowthBook, and Statsig on its Enterprise tier, read exposure and conversion data from your own Snowflake, BigQuery, Redshift or Databricks warehouse and compute results there, so experiment metrics match the numbers your data team already reports. That matters to companies with a data team and a warehouse. A marketing team with a website and GA4 rarely needs it, and it adds a dependency on SQL and data engineering.
How A/B testing tools are priced, and what they cost as traffic grows
A/B testing tools meter five different things, and the meter matters more than the entry price. Monthly tested users count each visitor who enters at least one experiment in a month, once; Convert, Mida and Optibase work this way. Monthly active or tracked users count visitors whether or not they're tested: Wingify prices VWO and AB Tasty on monthly active users, and Zoho PageSense and Kameleoon's Starter plan count monthly tracked users. Shoplift counts every store visitor, and Crazy Egg and Webflow Optimize meter page views. Developer platforms meter usage: Statsig bills metered events and PostHog bills feature flag requests. GrowthBook charges per seat, and Otter charges a flat price per project with unlimited visitors.
Here's what a site with 100,000 monthly visitors would pay if every visitor entered a test, using prices checked on October 11, 2026 and billed annually unless noted. Otter Starter costs $31/mo ($39 billed monthly) for one site. Convert Growth and Mida Growth each cost $299/mo at 100,000 tested users. Optibase Professional costs $266/mo for up to 120,000 tested users, and Shoplift Advanced costs $299/mo for up to 100,000 store visitors. Webflow Optimize costs $549/mo for its 100,000 page view tier plus a paid Site plan, and 100,000 visitors generate at least that many page views. Crazy Egg Plus costs $99/mo for 150,000 tracked pageviews, which is enough only if visitors average 1.5 tracked pageviews or fewer. Kameleoon's Starter plan, from $495/mo, stops at 50,000 tracked users, so this site would need a custom Enterprise quote, as it would with Wingify, Optimizely and Adobe Target.
Developer platforms depend on how much you log. Statsig Pro's $150/mo includes 5 million metered events, which is 50 events per visitor at this traffic, then $0.05 per 1,000. PostHog's free 1 million flag requests a month is 10 requests per visitor, then from $0.0001 per request. GrowthBook Pro costs $40 per seat whatever the traffic, so a team of three pays $120/mo.
What changes as you grow is the slope. At 250,000 tested users, Convert Growth costs $560/mo billed annually; Mida charges $599/mo at 200,000 and $899/mo at 300,000; and Webflow Optimize's 250,000 page view tier costs $799/mo billed yearly. Otter still costs $31/mo for one site, or $63/mo on Growth if you test up to five sites. Metered tools also behave differently at the cap: Crazy Egg, Zoho PageSense, Shoplift and Kameleoon pause tests until you upgrade or the period resets, while Convert bills an overage you can switch off. Compare Otter's plans with the meter that fits your traffic before you sign anything.
How to choose an A/B testing tool for your team
Start with who will build the tests and where the site lives. A marketing team on WordPress, Webflow, Framer or a landing page builder needs a client-side tool with a visual editor, split URL tests and goals that match how it already measures success, often GA4 events. Otter, Convert, VWO, Mida and Crazy Egg fit here, and our guides cover the platform details for WordPress, Webflow, Framer and landing pages. Some site builders now test natively: Webflow Optimize is a paid add-on from $299/mo billed yearly, and Framer's own Convert add-on (no relation to Convert.com) includes up to five A/B tests, though its cookieless identifier resets daily, so it only counts conversions that happen the same day.
An online store should judge tests on revenue per visitor, not just conversion rate, because a variant can win more orders at a lower order value. On Shopify, Shopify's own Rollouts can A/B test themes and checkout configurations on the Grow plan and up, but its experiment reports show conversion, add-to-cart and reached-checkout rates rather than revenue or average order value. Shoplift tests themes and templates inside the Theme Editor. Intelligems tests prices, discounts, shipping and checkout offers and reports profit; it starts at $69/mo, and price tests need its Smart Pricing module or the $349/mo Unlimited plan, each with a three-month minimum. Convert includes Shopify price, shipping and theme tests on Growth. Otter runs on Shopify through a theme snippet and reports revenue per visitor from a Custom Pixel, but it has no Shopify app and can't test checkout or the price charged. Our guides to Shopify A/B testing and ecommerce A/B testing go further.
A product or engineering team testing features, onboarding flows or logic inside an app needs feature flags, server-side SDKs and analysis built on product events. GrowthBook, PostHog, Statsig and Amplitude Experiment are built for this, and Optimizely sells it to enterprises. On Next.js, Vercel's Flags marketplace has native integrations for Statsig, PostHog and GrowthBook, so a flag can be evaluated before the page renders. A client-side tool like Otter still works on a Next.js site for marketing pages, but it can't test logic that runs on your server.
A mobile app needs an SDK inside the app. Firebase A/B Testing is free for Firebase apps, and Statsig, GrowthBook, PostHog, Amplitude and Kameleoon all ship iOS and Android SDKs. Web-only tools, Otter included, can't test native app screens.
Large organizations with procurement, SSO and data residency requirements usually shortlist Optimizely, Adobe Target and Wingify's Enterprise plans, all priced through sales. Adobe Target makes most sense if you already run Adobe Analytics or Adobe Experience Platform. For head-to-head detail on the tools you're weighing, our comparison pages set Otter side by side with the main alternatives, row by row.
Free and low-cost A/B testing tools: what's actually free
Free plans exist, but most come with a catch that matters to a website team. Firebase A/B Testing is free for apps, and since September 1, 2026 only Remote Config fetches above 100,000 a day per project cost money. GrowthBook's open-source edition is free to self-host, and its free Cloud Starter plan allows 3 users and 1 project with unlimited traffic, but it leaves out the visual editor and split URL tests. PostHog gives every account 1 million feature flag requests a month, which cover experiments, though its no-code editor is still in beta.
Statsig's free Developer tier includes 2 million metered events and 50,000 session replays a month, and Amplitude's free plan includes 2 million events and one active web experiment, though Amplitude reserves its Experiment Results chart for the paid Growth and Enterprise plans. Statsig's pricing was unchanged when we checked, but now that Amplitude owns it, expect changes as the two products are integrated.
For websites, Mida's free Sandbox covers up to 100,000 tested users a month but allows 2 live tests, 2 goals and one challenger per test, with 60-day data retention. Optibase's free plan covers 10,000 tested users with one active test. Crazy Egg and Zoho PageSense have permanent free plans, but neither includes A/B testing. Optimizely's free Feature Flagging account allows one concurrent experiment on flags and no web testing.
Otter doesn't have a free plan. It has a 14-day free trial with no credit card, after which Starter costs $39/mo, or $31/mo billed annually. The cheapest paid plans with A/B testing on this list are Zoho PageSense Enterprise at $29.42/mo billed annually for 10,000 tracked users, Otter Starter at $31/mo with no visitor cap, Optibase Basic at $66/mo for 20,000 tested users and Shoplift Core at $74/mo for up to 50,000 store visitors. Otter's free A/B testing calculators cost nothing, but the testing product itself is paid.
What happened to Google Optimize, and why VWO and AB Tasty are now Wingify
Google shut down Optimize and Optimize 360 on September 30, 2023, and any experiments still running that day ended. GA4 has no built-in A/B testing. Google said it would invest in third-party testing integrations with Google Analytics, naming AB Tasty, Optimizely and VWO, and made its APIs public so any testing tool could connect. Our Google Optimize alternatives page maps what Optimize did to what's available now.
Two of those three partners are now one company. VWO and AB Tasty announced they were joining forces on January 20, 2026 and launched the combined Wingify brand in September 2026, reporting more than $105 million in revenue, 4,000 customers, 700 employees and 11 offices. Everstone Capital took a majority stake in Wingify in 2025. Both vwo.com and abtasty.com now redirect to wingify.com.
For buyers, the products haven't merged yet. Wingify says VWO and AB Tasty each still run as their own suite while engineering converges over the coming quarters, and that contracts and pricing are unchanged for existing customers. New buyers see one Wingify pricing page with no published prices, custom proposals and pricing based on monthly active users. If you're evaluating either, ask which platform you'll be on in two years and how your experiment history will move. Our VWO comparison and AB Tasty comparison go row by row.
Consolidation is happening among developer tools too: Statsig went from independent to OpenAI (announced September 2, 2025) to Amplitude (May 5, 2026) in eight months. An owner change can bring a new roadmap, pricing model or data policy, so check contract terms and how you'd export your data before you commit to any tool on this list.
A/B testing and SEO: what Google recommends
Google's guidance on website testing says small changes such as button size, color, placement or call-to-action text often have little or no effect on a page's ranking or search snippet. Most A/B tests don't put rankings at risk. Problems come from how a test is set up, and the guidance boils down to four rules.
Don't cloak: never show Googlebot a different set of URLs from the ones people see, whether through server logic, robots.txt or anything else. For split URL tests, put a rel=canonical tag on each variant URL pointing at the original, and prefer it to a noindex tag, which Google says can have unexpected bad effects. When you redirect visitors from the original to a variant, use a temporary 302 redirect rather than a permanent 301; Google says JavaScript-based redirects are also fine. And run each experiment only as long as you need to: once it ends, ship the winner and remove the alternate URLs and testing scripts, because Google says an experiment left running for an unnecessarily long time may be interpreted as an attempt to deceive search engines.
Two practical notes. Googlebot generally doesn't support cookies, so in a cookie-based test it sees the version served to browsers that don't accept cookies, usually the original. And browser-based tools, Otter included, redirect with JavaScript, which fits Google's guidance. Whatever tool you use, check that each variant page carries a canonical tag pointing at the original before the test starts; on most sites that's a setting in your CMS.
Do A/B testing tools slow down your site?
Any client-side tool adds a script that has to load before visitors see the right version, so compare tools on script weight, how they handle flicker, and where you install them. Vendors publish different figures: Otter's SDK is about 32 KB gzipped, Mida says its script is 15 KB compressed, and Convert's FAQ gives 93 KB gzipped at baseline, rising with active experiments. Crazy Egg doesn't publish a size.
Flicker is the flash of the original page before a variant appears. Otter's built-in anti-flicker hides the page until test settings load, with a 3-second failsafe so a slow connection never leaves the page blank for long. Convert loads synchronously by default with a body-hide step, Crazy Egg supplies a test-specific flicker snippet through support, and Optibase asks you to keep only the main variant visible by default. Unbounce says testing built into its own page builder avoids the delay because there's no separate testing script.
Install the snippet directly in the page head rather than through Google Tag Manager, which loads late and makes variants flash; Otter's documentation, Convert's and Crazy Egg's all say so. If any visible delay is unacceptable on a page, test it with a server-side tool such as GrowthBook, Statsig or PostHog, where the server sends only the assigned version.
How much traffic you need before A/B testing pays off
The sample a test needs depends on your baseline conversion rate and the smallest lift worth detecting. At a 3% conversion rate, detecting a 20% relative lift (from 3.0% to 3.6%) at 95% significance and 80% power takes about 13,900 visitors per variant, or roughly 27,800 for a simple A/B test. At a 2% baseline the same lift needs about 21,100 per variant. Run your own numbers in the sample size calculator.
Time is the other constraint. A site with 1,000 visitors a day needs about four weeks for that 3% example, and a site with 100,000 visitors a month needs under two weeks. Run tests for at least one or two full weeks anyway, so weekday and weekend behavior both count: Shoplift's guideline is at least 10,000 visitors per variant and 14 days, Unbounce suggests about two weeks, and Firebase recommends at least two weeks for a typical experiment. The test duration calculator turns your daily traffic into days.
If the numbers come out at months, change the test rather than the tool. Test bigger changes that could plausibly move conversion by 20% or more, test on your highest-traffic pages, or use a goal earlier in the funnel, such as add-to-cart instead of purchase. The minimum detectable effect calculator shows the smallest lift your traffic can detect, and Otter's launch check warns when a test's estimated duration runs past eight weeks. Convert itself says its platform may not be worth it for low-traffic sites, and that applies to every paid tool on this list.