# How to State a Hypothesis That You Can Actually Test

_2026-08-29_

Your growth team has noticed checkout drop-off climbing. Someone says, “Let's test a new checkout flow,” and the room moves on to mock-ups.

That sentence describes an **idea**, not a hypothesis. It doesn't identify which users are affected, what outcome should change, how much change would matter, or what result would justify shipping the variation. If the test wins, you won't know why. If it loses, you won't know what you disproved.

Learning **how to state a hypothesis** means translating an observation into a decision-ready statement. The practical standard is simple: another person should be able to read it, build the experiment, identify the primary metric, and judge the result without asking what you meant.

## The Real Job Behind Stating a Hypothesis

A working hypothesis connects three things:

1. **The change**, such as redesigning step two of checkout.
2. **The affected segment**, such as mobile users who reach that step.
3. **The expected movement in a named metric**, such as a reduction in step-two abandonment.

The vague version, “A better checkout will improve conversions,” fails all three tests. “For mobile users reaching step two, replacing the current form with a shorter grouped layout will reduce step-two abandonment” is much stronger. It names the intervention, the audience, and the expected outcome, even before you add a threshold.

### From aspiration to falsifiable statement

Work through the checkout example in layers:

- **Wish:** Improve checkout.
- **Test idea:** Try a new checkout flow.
- **Early hypothesis:** A shorter step-two form will reduce abandonment for mobile users.
- **Testable hypothesis:** For mobile users who reach step two, the shorter form will reduce abandonment from the current baseline to a lower rate during the defined measurement window.
- **Decision-ready hypothesis:** For mobile users who reach step two, the shorter form will reduce step-two abandonment by at least the minimum effect the team has agreed is commercially meaningful.

The final version gives your team something to defend in review. It also makes the experiment less vulnerable to post-launch storytelling, where people reinterpret the original idea after seeing the dashboard.

> **Practical rule:** If you can't name the user segment and the primary metric, you haven't stated the hypothesis yet.

Before launch, document the control, variation, audience, primary outcome, baseline, expected direction, and decision rule. That discipline sits alongside broader [A/B test best practices](https://www.oviond.com/how-to-conduct-a-b-testing/) such as keeping the comparison controlled and interpreting results against a pre-agreed plan.

A hypothesis isn't a prediction designed to sound confident. It's a claim that could be supported or contradicted by the evidence your experiment collects.

## Null, Alternative, and the Directional Choice

The **null hypothesis**, written as H0, represents the status quo. In a CRO test, it usually says that the variation produces no meaningful difference in the target metric compared with the control.

The **alternative hypothesis**, written as H1, is the competing claim. It says the variation changes the metric in a way the test is designed to detect.

For the checkout example:

- **H0:** The redesigned step-two checkout has the same abandonment rate as the control.
- **H1, non-directional:** The redesigned step-two checkout changes the abandonment rate.
- **H1, directional:** The redesigned step-two checkout reduces the abandonment rate.

The choice between the two H1 versions is a judgement call, not a formatting preference. A **non-directional hypothesis** allows the result to move either way. It's appropriate when the evidence doesn't justify ruling out an increase as well as a decrease. A **directional hypothesis** commits to one side, which can make the test more sensitive to that specified effect, but it creates a serious risk if the variation moves in the opposite direction.

Use a directional framing only when prior behavioural data, a strong heuristic review, or a clear UX rationale makes the opposite outcome implausible. A colour preference, personal conviction, or stakeholder enthusiasm isn't enough.

For a pricing-page CTA colour test, the two framings could look like this:

| Test | H0 (Null) | H1 (Alternative) | Directional? | Practical Trade-Off |
|---|---|---|---|---|
| CTA colour change | The new colour has the same conversion rate as the control | The new colour changes conversion rate | No | Detects movement in either direction, but generally requires more evidence for a result |
| CTA colour change | The new colour has the same conversion rate as the control | The new colour increases conversion rate | Yes | Focuses the test on an expected lift, but a negative movement can invalidate the planned interpretation |

A non-directional test is usually the safer choice when the underlying mechanism is uncertain. A directional test can be efficient when the decision is one-sided, but the cost of being wrong is higher.

For a focused explanation of the statistical distinction and its practical implications, see this guide to [one-tailed and two-tailed testing](https://www.otterab.com/blog/one-tailed-vs-two-tailed). The important operational point is to choose the framing **before** looking at the result. Changing from non-directional to directional after launch turns a planned analysis into a convenient reinterpretation.

## Operationalising the Variables You Will Measure

Most experiments fail before traffic reaches the variation. The team starts with a concept such as “better checkout” or “more engagement”, then discovers that nobody agreed what those words meant.

Operationalisation turns the concept into observable variables. University guidance emphasises that a hypothesis needs to be specific and answerable, while UK psychology guidance highlights measurable terms such as scores, times, and ratings. The same principle applies in CRO: a vague construct has to become a defined user action or business outcome. University of Southampton guidance on hypotheses and University of Hull's research guidance both reinforce the need to define the phenomenon precisely. The [UK psychology explanation of measurable variables](https://mrkpsychology.co.uk/how-to-write-hypotheses-in-aqa-a-level-psychology/) makes the translation especially clear.

![A diagram illustrating the four-step process for operationalizing business variables from vague goals to measurable metrics.](https://cdnimg.co/3716ee4f-bd1a-44a8-ac85-c2df5af21725/2397811e-580e-4fc0-a04b-9396fe44ac29/how-to-state-a-hypothesis-operationalising-variables.jpg)

Use four moves.

### Name the primary metric

“Checkout performance” isn't a metric. Choose one outcome, such as **step-two abandonment rate**, **payment success rate**, or **revenue per session**. Make the primary metric the measure that decides the main test question.

### Capture the baseline

Record the current value, the date range, and the relevant segment. Suppose mobile users reaching step two currently abandon at **38%**. That figure is only useful if the team also records how it was calculated and which users it includes.

### Set the minimum detectable effect

The **minimum detectable effect**, or MDE, is the smallest change worth acting on. If the team agrees that a three-percentage-point reduction matters, the target moves from **38% abandonment to 35% abandonment**. In plain language, the experiment needs to distinguish a change that reaches the agreed threshold from ordinary variation that isn't worth shipping.

### Lock the measurement window

Choose the period before launch. Don't let a promotion, holiday, pricing change, or major acquisition campaign redefine the audience. The window should be long enough to collect the planned evidence and stable enough that the comparison answers the original question.

Your operationalised hypothesis might now read: “For mobile users who reach step two, the shorter form will reduce abandonment from the recorded baseline of 38% to 35% or lower during the planned measurement window.” The decision criteria give that sentence its final force.

## Setting the Decision Criteria Before You Launch

A test can produce a dashboard full of movement without producing a defensible decision. Before splitting traffic, agree on the evidence required to call the result.

The usual planning inputs are **confidence level**, **statistical power**, **minimum detectable effect**, and **sample size per variant**. The brief's checkout example uses a **95% confidence level**, **80% statistical power**, and a target of a **10% relative lift** against a **5% baseline conversion rate**. Those values define the sensitivity you're asking the experiment to achieve. They don't guarantee that the variation will reach the target.

### Write the stopping rule in advance

A useful decision rule might say:

- Ship the variation if the primary metric beats the control by at least the agreed MDE and the result reaches the planned confidence threshold.
- Keep the control if the result fails to meet the threshold.
- Treat an inconclusive result as inconclusive, rather than converting a small observed movement into a winner.
- Stop for a guardrail breach if an important secondary outcome deteriorates beyond the agreed tolerance.

The exact sample requirement depends on the baseline, effect size, confidence level, power, allocation, and test design. For a practical explanation of how to [calculate sample size for tests](https://hello.quikly.com/blog/sample-size-calculation), use a calculator that exposes those assumptions rather than hiding them behind a single output. You can also review the mechanics in this guide to [calculating sample size](https://www.otterab.com/blog/how-to-calculate-sample-size).

| Confidence Level | Statistical Power | Min. Detectable Effect (relative) | Sample Size per Variant |
|---|---|---|---|
| Agreed before launch | Agreed before launch | Based on the smallest commercially meaningful change | Calculated from the inputs and reachable test window |
| 95% in the example plan | 80% in the example plan | 10% relative lift against a 5% baseline conversion rate | Must be calculated before traffic allocation |

Don't choose a sample size because the dashboard has become interesting. Choose it because the planned rule says the experiment has collected enough evidence to answer the question.

Tools can help enforce the sequence. Otter A/B lets a team define the goal and monitor statistical significance against its stated threshold, while tracking outcomes such as purchases, average order value, and revenue by variant. The useful workflow is **hypothesis first, decision criteria second, traffic allocation third**. That order limits the temptation to rationalise a result after the fact.

## Common Pitfalls and How to Dodge Them

The most damaging hypothesis errors aren't grammatical. They're operational. They leave the team with too many interpretations, too many outcomes, or no credible stopping point.

![An infographic titled Common Pitfalls and How to Dodge Them, listing three A/B testing mistakes and their solutions.](https://cdnimg.co/3716ee4f-bd1a-44a8-ac85-c2df5af21725/1d199006-e027-4cd6-a709-50b9a4bbf44c/how-to-state-a-hypothesis-ab-testing.jpg)

### Vague metrics

**Junior version:** “The new product page will increase engagement.”

Engagement could mean clicks, time on page, scrolling, video starts, add-to-cart actions, or purchases. A senior version names the event and threshold: “For paid-search visitors, the new product page will increase the share reaching 75% scroll depth.”

That metric still needs a business rationale. A scroll event can diagnose attention, but it shouldn't automatically replace a purchase or revenue outcome when the test affects buying behaviour.

### Peeking and early stopping

**Junior version:** “We'll check the results every day and stop when the variation looks ahead.”

Repeated checking changes the practical meaning of the statistical result. It also rewards random early movement, especially when the team is eager to ship. The corrected version states the planned sample, measurement window, confidence requirement, and stopping rule before launch.

> A dashboard is for monitoring implementation and safety. It isn't permission to rewrite the hypothesis each morning.

### Multiple hypotheses in one variation

**Junior version:** “The new checkout will reduce friction, increase trust, improve mobile usability, and raise revenue.”

That's a bundle of claims. If revenue rises but abandonment doesn't, which hypothesis won? Separate the primary hypothesis from supporting diagnostics, and list guardrails such as page-load time, payment errors, or revenue per session where they matter.

A defensible post-mortem can explain exactly what was tested, what happened to the primary metric, and whether the pre-launch rule was met. An informal statement usually gets buried under a more convenient narrative.

## Templates and a Pre-Launch Checklist

A reusable template should be specific enough to paste into an experimentation brief, but flexible enough to work across landing pages, onboarding, and ecommerce flows.

### Non-directional CRO template

> For **[defined user segment]**, changing **[specific element or experience]** will change **[primary metric]** compared with **[control]** during **[measurement window]**. The null hypothesis is that there's no meaningful difference. The alternative hypothesis is that the metric changes in either direction.

Use this when your evidence supports testing movement but doesn't justify predicting the side.

### Directional CRO template

> For **[defined user segment]**, changing **[specific element or experience]** will **increase or decrease [primary metric]** by at least **[agreed MDE]** compared with **[control]** during **[measurement window]**.

Only choose the direction when your prior evidence gives you a reason to exclude the opposite interpretation.

### Guardrail-augmented template

> For **[defined user segment]**, changing **[specific element or experience]** will improve **[primary metric]** by at least **[agreed MDE]**, while keeping **[guardrail metric]** within **[agreed tolerance]**, compared with **[control]** during **[measurement window]**.

This version suits tests that could improve conversion while creating downstream harm, such as a faster checkout that increases payment failures or lowers revenue per session.

![A diagram illustrating three CRO templates and a pre-launch checklist for website testing and optimization strategy.](https://cdnimg.co/3716ee4f-bd1a-44a8-ac85-c2df5af21725/8cc8bc29-d226-443c-a71c-7eb09cbff3bf/how-to-state-a-hypothesis-cro-checklist.jpg)

Run this checklist before traffic is allocated:

1. **Capture the baseline:** Record the current primary metric, segment, source, and measurement window.
2. **Write H0 and H1:** State the null and alternative claims, including whether H1 is directional.
3. **Define the primary metric:** Name one outcome that determines the main decision.
4. **Agree the MDE and confidence:** Make the smallest worthwhile change and evidence threshold explicit.
5. **Check reachability:** Confirm the required sample can arrive within a stable, meaningful window.
6. **List guardrails:** Include metrics that could reveal performance, quality, or revenue damage.
7. **Set the stopping rule:** Decide what counts as ship, keep, extend, or stop before launch.
8. **Assign analysis ownership:** Name the person responsible for reading the result against the rule.
9. **Schedule the debrief:** Reserve time to record the decision and the next test.

If the team has an experiment idea but the statement is still loose, the [Otter A/B hypothesis generator](https://www.otterab.com/free-tools/hypothesis-generator) can help turn it into a structured test statement and success criteria for review. Use the output as a starting point, then validate the segment, metric, baseline, and decision rule with the people who own the result.

Stop polishing “we should test this” in a document. Write the directional choice, operational metric, MDE, and stopping rule, then put the next experiment into your testing workflow.

---

Otter A/B helps teams create variants, define goals, split traffic, and evaluate conversion and revenue outcomes against a pre-planned significance threshold. Visit [Otter A/B](https://www.otterab.com) to turn your next hypothesis into a launchable experiment without waiting for another round of hand-wavy debate.

---

Canonical page: https://www.otterab.com/blog/how-to-state-a-hypothesis