How it works

Paste one line. Run a real test. See what it earned.

That is how A/B testing works in Otter: install a snippet, build variants without touching code, then watch conversion rate and revenue per visitor land on a live dashboard. Most teams have their first split test running inside five minutes.

Start free
01

Add the snippet

Copy one line of code into your website's <head>. That's it. Works with any HTML page, WordPress, Shopify, Next.js, Nuxt, or any other framework.

<!-- Otter -->
<style id="optimo-hide">body{opacity:0 !important}</style>
<script data-optimo-reveal>setTimeout(function(){var s=document.getElementById("optimo-hide");if(s&&s.parentNode){s.parentNode.removeChild(s)}},3000)</script>
<script src="https://www.otterab.com/sdk/optimo.js"
  key="YOUR_API_KEY" async></script>
index.html123<script src="optimo.js" key="your_api_key" async></script>CopyHTMLLn 2, Col 18
02

Create your test

Name your test, set the target URL, and define your variants. Add DOM changes — swap headlines, images, buttons, or entire sections. Set your conversion goals: page views, clicks, custom events, revenue, DataLayer events, or GA4 events.

  • Define multiple variants with different weights
  • 10 DOM change types: text, HTML, attributes, styles, classes, and more
  • 6 goal types: pageview, click, custom event, revenue, DataLayer event, GA4 event
OtterVariantVisitorsConv. RateStatusControl12,4833.2%BaselineVariant A12,2914.7%WinnerScore97.3%Uplift+47%Total Visitors24.7kConversion trend
03

Get results

Visitors are assigned and conversions are tracked in real time — including revenue, average order value, and revenue per visitor. Manual-split experiments can use frequentist or Bayesian reporting. Multi-armed bandits use Bayesian reporting automatically while adapting new-visitor traffic.

0%5%10%15%thresholdControlVariant A+47%8.8%12.9%

Under the hood

What happens between a visit and a decision

A reliable A/B test depends on four things working together: a fast SDK, a stable variant assignment, a goal that maps to revenue or behaviour, and a statistical decision that actually means something. Here is how each piece fits.

Step 1

The SDK loads early and hides the page briefly

The snippet you paste into your <head> does two things before anything else renders. First, it applies an anti-flicker style that hides the body for up to 300 milliseconds while variants are resolved. Second, it loads optimo.js asynchronously so it never blocks the rest of the page. A 3-second failsafe removes the anti-flicker style automatically if the SDK fails to load — your visitors are never stuck on a blank screen.

The whole SDK is under 9KB gzipped. After the first page, it lives in the browser cache, so subsequent navigations add essentially no latency. There is no virtual DOM, no React tree, and no framework — it is plain JavaScript that touches the DOM directly so changes apply on the first paint.

Step 2

Visitors are assigned deterministically

Each visitor gets a stable assignment ID stored in a first-party cookie and replayed on every request. Variant assignment is a deterministic hash of (visitor_id, test_id) — the same visitor sees the same variant across sessions, devices that share the cookie, and any subdomains under the project domain.

Manual splits use the weights you configure for every new assignment. Multi-armed bandits start evenly and refresh the new-visitor split about every five minutes from primary-goal results. Existing visitors always keep their first version.

Step 3

Goals connect to revenue, not vanity

Otter supports six goal types: pageview, click, custom event, revenue, DataLayer event, and GA4 event. Pageview, click, custom, DataLayer, and GA4 goals measure conversion actions. The revenue goal answers the harder question — “did this variant make us more money?” — by attaching a monetary value to a conversion and computing average order value, revenue per visitor, and incremental revenue alongside the conversion lift.

For ecommerce teams, this matters because conversion-rate winners can lose on revenue if the variant moves customers toward cheaper purchases. Revenue tracking surfaces that conflict instead of hiding it.

Step 4

The decision uses the right math for the test

Manual-split experiments can use frequentist or Bayesian reporting. Frequentist mode reports p-values and confidence intervals with a configurable threshold. For multivariate tests, the threshold adjusts automatically so you do not “win” by running enough variants.

Bayesian mode reports the probability that each variant beats control plus the expected lift distribution. Multi-armed bandits use Bayesian reporting automatically because the traffic strategy learns from the same primary-goal results it reports.

Manual frequentist splits have fixed-horizon sample-size guidance. Bayesian manual splits do not show that estimate, though either manual method can auto-stop on a winner. Bandits adapt new-visitor traffic continuously and do not use winner auto-stop; visitor and conversion caps remain available. Every method exposes the same resolved decision contract in the results page, API, and CSV export.

Setting expectations

When A/B testing pays off — and when it doesn't

Testing is a decision tool, not a religion. Knowing when to skip it is what keeps the results you do get worth trusting.

Good fit

A/B testing is the right tool when

  • You have enough traffic to detect the lift you care about — typically a few thousand conversions per variant.
  • The change you are testing is meaningful enough that a 5–20% lift is plausible.
  • The decision is reversible and the cost of being wrong is low.
  • You care about a measurable outcome — conversion, revenue, retention — more than a qualitative feeling.

Poor fit

A/B testing is the wrong tool when

  • Your traffic is too small for the effect you can detect in a reasonable timeframe.
  • You are testing brand or strategic decisions where the right answer is “what should we mean,” not “what converts higher.”
  • The change is irreversible and being wrong is expensive.
  • You are looking for explanations, not decisions — qualitative research and session replay are usually better.

The fastest way to find out whether you have enough traffic is the sample size calculator. Run it before launching anything.

Integrations

Works with any stack

Drop in the snippet and you're live.

Custom JavaScriptShopifyWordPressWebflowWixWooCommerceClickFunnelsSquarespaceFramerNext.jsGoogle Tag Manager

Five minutes from now

Ready to run your first test?

14-day free trial, no credit card, and a revenue number on your first experiment.

Start free trialRead the docs
  • 14-day free trial
  • No credit card required
  • Cancel anytime