# Executive Summary

A plain-English card at the top of every results page that tells you, in one line, whether to ship the change, keep testing, or stop — and why.

A card at the top of every results page that reads the test for you and says, in one line, what to do next — Ship, Continue running, Inconclusive, or Stop test — with the reason and a suggested next step.

Numbers are great, but most people just want to know one thing: *should I make this change or not?* The **Executive summary** answers exactly that. It sits at the very top of the results page, reads your main goal for you, and gives a plain-English recommendation with a short reason — no stats knowledge needed.

It also shows up on shared report links, so when you send results to a manager or client they see the same clear takeaway you do. It only gives advice: it never starts, stops, or edits your test — you always make the final call.

## The four recommendations

| Recommendation | What it means |
| --- | --- |
| **Ship** | A version clearly won. Roll it out, then keep an eye on your other numbers. |
| **Continue running** | Something is leading, but it hasn't proven itself yet. Keep collecting data. |
| **Inconclusive** | The test ended without a clear winner. Usually a sign to try a bolder idea next. |
| **Stop test** | The version you're testing is doing worse than your current one. Stop and keep what you have. |

## What else the card shows

- **Estimated uplift** — how much better (or worse) the leading version is doing than your original, as a percentage.
- **Score** — the same headline confidence score as the rest of the page ("Significance" or "Chance to Beat Original"), and whether it's above or below the line it needs to clear.
- **Incremental revenue / yr** — for tests with a revenue goal, a rough projection of the extra money per year if you ship the winner. (Tests without a revenue goal show the visitor count here instead.)
- **A next-step tip** — one concrete suggestion, like "keep the test running until it reaches its decision threshold" or "promote the winning experience."

> **How to use it well**
>
> **Read it with the Experiment health card.** The Executive summary tells you *what to do*; the [Experiment health](https://www.otterab.com/docs/analyzing-results/experiment-health) card tells you *whether you can trust it yet*. A confident recommendation plus a clean health card is the strongest green light to act.
>
> **It speaks for your primary goal.** The recommendation is based on the single most important outcome you picked. Before you ship, still glance at your secondary goals in the detailed results — a winner on one goal can sometimes cost you on another.
>
> **Treat revenue projections as a guide.** The yearly revenue figure is an estimate based on what's happened so far; real results depend on your traffic staying similar. Use it to weigh a decision, not as a guarantee.

## Frequently asked questions

### What is the Executive summary?

It's a card at the very top of your results page that reads the test for you and says, in plain English, what to do next. Instead of staring at numbers wondering whether they're good enough, you get a one-line recommendation — like "Ship variant B" — plus a short reason and a suggested next step. It's the fastest way to understand where a test stands.

### Where do I find it?

At the top of any test's results page, above the detailed numbers. It also appears on shared report links, so anyone you send a report to sees the same clear recommendation — no stats knowledge needed.

### What do the four recommendations mean?

Ship — a version clearly won, so you can roll it out. Continue running — something is leading but it hasn't proven itself yet, so keep collecting data. Inconclusive — the test finished without a clear answer either way. Stop test — the version you're testing is doing worse than your current one, so stop and keep what you already have. Each comes with a headline and a short explanation of why.

### What does 'estimated uplift' mean?

How much better (or worse) the leading version is doing than your original, shown as a percentage. A +12% uplift on a sign-up goal means the leading version is getting about 12% more sign-ups per visitor than the original. A negative number means it's doing worse.

### What's the 'incremental revenue / yr' number?

For tests with a revenue goal, this estimates how much extra money per year you'd expect if you shipped the winning version, based on the difference measured so far. It's a projection to help you weigh the decision, not a promise — real results depend on traffic staying similar. If your test isn't measuring revenue, the card shows your visitor count here instead.

### What's the score shown in the summary?

It's the same headline score as the rest of the results page — how confident we are in the result — shown next to the decision threshold it needs to beat. Depending on your test's analysis method it's labelled "Significance" or "Chance to Beat Original." The summary tells you whether that score is above or below the line.

### Does the Executive summary decide for me, or change my test?

Neither. It's advice, not an action. It reads your main (primary) goal and suggests what to do, but you make the call, and it never starts, stops, or edits anything on its own. Treat it as a knowledgeable second opinion that saves you from misreading the numbers.

### Why does it say 'Continue running'?

Because a version is in the lead but hasn't yet crossed the line that proves the result is real. Stopping now would risk acting on a fluke. The card's next-step tip will point you to keep the test going until it reaches its decision threshold or one of your stop conditions trips.

### Why does it say 'Inconclusive'?

Because the test has ended (completed or archived) without any version clearly beating the others. That's a normal and useful outcome — it means the change you tried didn't make a measurable difference. The suggested next step is usually to keep your current experience and try a bolder idea next time.

### Is the Executive summary the same as the Experiment health card?

No — they answer different questions, and it's worth reading both. The Executive summary says what to do ("Ship variant B"). The Experiment health card says whether you can trust the result yet (enough visitors? fair traffic split? no mid-test changes?). A confident recommendation paired with a clean health card is the strongest signal to act.

### Does it look at all my goals or just one?

The headline recommendation is based on your primary goal — the single most important outcome you chose for the test. Your other (secondary) goals are summarised in the detailed results below, and the Experiment health card will warn you if a secondary goal is dropping. So check the full results before shipping, especially if a secondary goal matters to you.

---

Canonical page: https://www.otterab.com/docs/analyzing-results/executive-summary