# Stop Conditions Configure a test to auto-complete when it hits a visitor count, conversion count, or when a variant wins. Hands-off lifecycle management. Configure a test to auto-complete when it hits a visitor cap, a conversion cap, or when a variant reaches the decision threshold. Lets you walk away from a test and trust the platform to call it. Stop conditions are the hands-off way to manage a running test. Set the rules upfront, launch, and Otter auto-completes the test when any rule trips. The test transitions to the same `completed` state you'd get from clicking Complete manually — including the same ended-at timestamp and the same frozen-results behavior. You don't have to use stop conditions. A test without any configured stop conditions runs until you complete it manually or hit a scheduled end date. Most teams set at least one — usually a visitor cap as a sample-size floor — to keep tests from running longer than they need to. ## The three conditions ### Visitor limit (`visitor_limit`) Stop when total unique human visitors assigned to the test reaches this number. Counts unique humans (bots, impersonation sessions, and excluded-IP traffic are excluded). The count is total across all variants, not per-variant. ### Conversion limit (`conversion_limit`) Stop when total unique converters on the primary goal reaches this number. Counts visitors who converted at least once on the primary goal. Multiple conversions from the same visitor count as one. Secondary goals don't affect this counter. ### Auto-stop on winner (`auto_stop_on_winner`) Stop when the primary goal has a variant in 'winner' status, against the test's effective confidence threshold. Uses the same decision logic as the results page — Bayesian or frequentist with the configured confidence level, Bonferroni-adjusted for multivariate frequentist tests. ## Evaluation rules - **Checked every 5 minutes** via a background job. Expect a small lag — up to one batch interval of traffic — between hitting a limit and the test transitioning to completed. - **First-to-trip wins.** Internally the checker evaluates in this order: visitor limit, then conversion limit, then decision threshold. As soon as one is satisfied, the test stops; it doesn't wait for the others. - **Zero or blank disables a condition.** Only positive integers count. Leave a field blank to skip the condition entirely. - **The reason is recorded.** The test's activity log captures whether it stopped on visitor limit, conversion limit, or decision threshold so future-you knows why it ended when it did. - **Conditions are wizard-locked after launch.** Like other test settings, stop conditions can't be edited mid-flight. Complete and duplicate to change them, or contact support if you need a manual override. > **How to pick good stop conditions:** > > - **Set a visitor floor alongside auto-stop-on-winner.** Auto-stop the moment confidence is reached can inflate false positives in frequentist mode. Combine it with a visitor cap (the sample size the wizard estimated, say) so the test can't end too early on a thin sample. > - **Don't make limits too tight on low-traffic tests.** A 1000-visitor cap on a page that gets 30 visitors per day means a long-running test. Use the wizard's sample-size advisor to ground your limits in something concrete. > - **Re-think conditions before resuming.** If a test auto-stopped and you resume it, the same conditions will be re-evaluated against the same traffic that tripped them. Usually you want to raise the cap or remove the condition before resuming, otherwise you'll just re-trigger it. > - **Bayesian tests handle peeking better.** If you're uncomfortable with the auto-stop-on-winner risk for frequentist, run the test in Bayesian mode — the probability interpretation is mathematically safer to stop on the threshold. ## Frequently asked questions ### What can stop a test automatically? Three conditions, set per test: a visitor cap (stop when N humans have been assigned), a conversion cap (stop when N unique visitors have converted on the primary goal), and a decision-threshold trigger (stop when the primary goal has a variant in 'winner' status). You can configure any combination; the first one to trip ends the test. ### How does the visitor limit count traffic? It counts unique human visitors assigned to the test — the same number you see in the visitor column on the results page. Bots, impersonation sessions, and excluded-IP traffic are not counted. The check is total visitors across all variants, not per-variant. ### How does the conversion limit count conversions? It counts unique converters on the primary goal — visitors who converted at least once. Multiple conversions from the same visitor count as one. Conversions on secondary goals don't affect this counter. ### What does 'decision threshold reached' actually mean? Otter's results engine assigns each variant a status — winner, loser, inconclusive, etc. — based on the primary goal's score against the effective confidence threshold. Auto-stop on winner ends the test when any variant's status flips to winner. Multivariate tests get the Bonferroni-adjusted threshold applied automatically, so the bar is appropriately stricter when you're testing several challengers. ### When does Otter actually evaluate these conditions? Every five minutes via a background job. So in the worst case there's about a five-minute window between hitting your stop condition and the test actually transitioning to completed. For high-traffic tests with tight conditions, expect to see the visitor or conversion count slightly over your configured cap — the difference is one batch interval of traffic. ### Which condition wins if more than one is set? The first one that trips. Internally the checker evaluates in this order: visitor limit, then conversion limit, then decision threshold. The test completes once any single condition is satisfied — it doesn't wait for all of them. The stop reason recorded on the test tells you which one triggered. ### What if a limit is set to 0 or left blank? It's ignored. The checker only enforces a limit when the configured value is greater than zero. Leave a field blank to skip that condition entirely; set it to a positive integer to enforce it. ### Can I add or change stop conditions while the test is running? You can't edit the test directly once it leaves draft — the wizard is locked. Stop conditions are part of that lock. If you need to change them mid-flight, your options are: complete the test (manually), duplicate it as a new draft with updated conditions, and relaunch; or contact support if there's a specific reason to override the lock for safety. ### Does auto-stop on winner risk false positives? Yes, slightly — and that's the same risk as stopping early manually. The decision threshold is the same one shown on the results page; if you configured 95% confidence, the test stops at 95%. Stopping the moment confidence is reached is a recognized cause of false positives in frequentist mode. Set a sample-size floor (via the visitor limit) alongside auto-stop-on-winner to mitigate this, or use Bayesian analysis where stopping at the threshold is mathematically safer. ### What happens when auto-stop fires? The test transitions to completed (the same state as if you'd clicked Complete manually). ended_at is recorded with the moment of transition. The reason is logged ('Visitor limit reached', 'Conversion limit reached', or 'Decision threshold reached') and is visible in the test's activity log. Results are frozen at that point — no new assignments, no new conversions. ### Can I resume a test that auto-stopped? Yes — the same rules as a manually completed test. Resume from the dashboard, and as long as no scheduled end date has passed, the test re-enters running. Stop conditions stay configured, so the next batch of traffic will be re-evaluated and could auto-stop again. Reconsider the conditions before resuming so you don't immediately re-trigger them. --- Canonical page: https://www.otterab.com/docs/building-tests/stop-conditions