# Concepts Variant assignment, statistical significance, and the anti-flicker snippet. The handful of ideas you need in your head to use Otter confidently — how variants are assigned, how the math actually works, and how the SDK avoids flicker. You don't need a maths degree to use Otter — but three simple ideas explain why your results look the way they do and how much to trust them. Here they are in plain language. (If you love the detail, the box at the bottom names the exact methods.) ## Variant Assignment Each visitor is sorted into one version and stays there. The choice is random across all your visitors, but it never changes for any one person — we remember it, so the same visitor sees the same version every time they come back, even days later or on a new page. ## Statistical Significance This is how we tell a real difference from random luck. You choose how the math is done: the standard way asks how unlikely your result would be if the versions were really equal, while the "Bayesian" way gives the chance a version truly beats the original. Either way, Otter picks the right calculation for your data automatically — including a special one for money, which behaves differently from simple yes/no conversions. ## Anti-Flicker Stops visitors from glimpsing the original page for a split second before your version loads — that flash is called "flicker." The snippet briefly keeps the page hidden until the right version is ready, then reveals it (usually in a few hundred milliseconds). A 3-second safety timer makes sure the page always appears, even if something goes wrong. > **Why this matters in practice** > > **Deterministic assignment means you can A/B everything safely.** A visitor never bounces between variants between page loads. If you see someone "flip" in session replay, it's almost always because they cleared cookies or switched browsers. > > **Test selection adapts to your data.** Otter picks Fisher's exact test for sparse counts and a z-test for typical comparisons automatically — you don't need to think about which to use. Revenue gets Welch's t-test because revenue distributions are heavy-tailed and a z-test would lie to you. > > **Anti-flicker is non-negotiable.** If visitors see the original page flash before the variant, they've already responded to the original — your test is measuring noise. The 3-second failsafe keeps the page from disappearing entirely if the SDK ever fails to load, but the normal case is a few hundred milliseconds. ## Frequently asked questions ### How does Otter decide which variant a visitor sees? When a new visitor lands on a test, the SDK generates a UUID and stores it in a first-party cookie. The visitor UUID and the test ID are hashed together to produce a number in [0, 1). That number is mapped to a variant via the configured traffic split. Because the inputs are deterministic, the same visitor on the same test always gets the same variant — even after cache clears, because the assignment is also persisted server-side. ### What if a visitor switches devices or browsers? By default, they get reassigned because the visitor UUID lives in a cookie. If you call optimo.userID('some_id') with a stable user identifier (after login, say), the assignment becomes user-scoped: the same user gets the same variant across devices, browsers, and incognito sessions. Without userID, anonymous device-level assignment is the model. ### Are assignments shared across tests? No. Each test gets an independent assignment driven by a different hash seed. A visitor who lands on variant B for test #1 has independent odds of landing on any variant for test #2. This avoids any "groove" where a visitor is consistently assigned to (e.g.) the second variant across multiple tests. ### Which statistical test does Otter actually run? It depends on the data: Fisher's exact test when conversion counts are sparse (< 5 in any cell), giving you valid p-values even at small sample sizes; a two-proportion z-test for typical conversion-rate comparisons when counts are larger; and Welch's t-test with Satterthwaite degrees of freedom for revenue metrics. Bayesian mode replaces all of these with a Bayesian bootstrap that estimates the posterior distribution of each variant's metric and computes P(variant > control). ### Why use Fisher's exact test instead of just a z-test? The z-test relies on a normal approximation that breaks down when conversion counts are very small. With only a handful of conversions per variant, the approximation can report 'significant' results that aren't real. Fisher's exact test enumerates the actual probability and gives you a correct p-value even at small sample sizes. Otter switches automatically when it detects sparse counts. ### What is the anti-flicker snippet doing? It's a single inline