# Notifications & Webhooks

Send test events to Slack or your own webhook, and schedule a daily digest. Five event types, idempotent delivery, signed payloads.

Send Otter test events to Slack, your own webhook handler, or as a scheduled daily digest. Five event types, idempotent delivery, signed payloads.

Notifications turn test lifecycle moments into messages somewhere your team already looks. A "test crossed significance" ping in #growth-experiments is more actionable than checking the dashboard every morning. A "test started" ping to your own webhook lets you log experiments into the same system you use for feature flags.

The system has two halves: **destinations** (where messages go — a Slack webhook URL or your own webhook endpoint) and **subscriptions** (which events on which tests get sent to which destination). One destination can have many subscriptions; one subscription targets one destination.

## Destinations

### Slack (`slack_webhook`)

Paste an Incoming Webhook URL from Slack. URL must start with `https://hooks.slack.com/`. Slack treats the URL itself as the secret — payloads are not separately signed.

### Generic webhook (`generic_webhook`)

POST to any HTTPS endpoint you control. A signing_secret is generated automatically, and every request carries an `X-OtterAB-Signature` header (HMAC-SHA256 of the body) so you can verify authenticity on your side.

> **Note: URL rules enforced at save time**
>
> - **HTTPS only.** Plain HTTP is rejected.
> - **No private or local hosts.** URLs that resolve to localhost, 127.0.0.1, 0.0.0.0, ::1, or any RFC 1918 / link-local address are rejected — this prevents webhooks being aimed at the Otter server's internal network.
> - **Slack destinations must use a Slack URL.** The hostname check above is in addition to the Slack-specific URL pattern.

## Event triggers

Five test events. Each subscription targets exactly one event key (or the daily digest).

| Event | What it means |
| --- | --- |
| `test.started` | A test moved from draft to running. |
| `test.paused` | A running test was paused — new assignments stopped. |
| `test.resumed` | A paused or completed test was resumed. |
| `test.completed` | A test was marked complete — results are frozen. |
| `test.decision_threshold_reached` | A variant's score crossed the effective confidence threshold. (Older name: test.significance_reached — still accepted.) |

## Daily digest

A daily_digest subscription rolls up activity across your subscribed tests and fires once a day at the hour and timezone you set. Schedule fields are required:

- `hour` — integer 0–23.
- `timezone` — any IANA timezone (e.g. `America/New_York`, `Europe/London`).

## Scoping a subscription to specific tests

By default a subscription fires for *every* test in the account. Set the scope to a list of test IDs and it only fires for those tests. Useful when one Slack channel cares about a single campaign but you don't want every test pinging it.

## Delivery lifecycle & idempotency

Each delivery is keyed by a `subject_key` that's unique per (subscription, event). If a delivery already exists for a given subject, the system reuses it — the same event never fires twice. Failed deliveries are retried under the same subject_key.

| State | Detail |
| --- | --- |
| `pending` | Created but not yet attempted. A worker will claim it shortly. |
| `delivering` | A worker has claimed this delivery and is sending it now. Stale claims (older than 5 min) are auto-reclaimed for retry. |
| `delivered` | Delivered successfully. Records the delivered_at timestamp. |
| `failed` | Last attempt errored. The attempts counter and last_error message are tracked for debugging; the delivery is retried with the same subject_key. |

> **Note: Webhook handler tips**
>
> - **Verify the signature on every request.** Every webhook carries an `X-OtterAB-Signature` header — an HMAC-SHA256 hex digest of the raw request body keyed with your destination's signing_secret. Recompute on your side and compare in constant time; reject any mismatch. Anyone who guesses your URL can send you a payload otherwise.
> - **Respond with 2xx fast.** Treat the webhook as a notification, not a job — enqueue real work to a background processor and return 200. Slow responses risk timeouts and unnecessary retries.
> - **Make your handler idempotent.** Otter avoids duplicate sends via subject_key, but transient network errors can still cause the same subject to retry. Use the subject_key in your own dedupe logic.
> - **Pause a subscription before launching big tests.** If you're about to spin up dozens of tests via the API, toggle the subscription off, finish the rollout, then toggle it back on — otherwise you'll flood Slack with start events.

## Frequently asked questions

### What can I get notified about?

Five test events plus a daily digest. The events are test.started, test.paused, test.resumed, test.completed, and test.decision_threshold_reached (formerly test.significance_reached — the older name is still accepted for back-compat). The daily digest summarizes activity across your subscribed tests and arrives at the hour and timezone you configure.

### Where do notifications go?

Two destination kinds today: Slack via an Incoming Webhook URL (any URL starting with https://hooks.slack.com/) and a generic webhook for your own backend. Email notifications are not part of the destinations system — they're handled separately via in-app email preferences.

### Can I subscribe a destination to only specific tests?

Yes. Every subscription has an optional scope. Leave it empty and the subscription fires for every test in the account. Set test_ids on the scope and it fires only for those tests. This is handy when a particular Slack channel cares about one campaign but you don't want it pinged about everything.

### How do I prove a webhook payload actually came from Otter?

Each generic webhook destination has a signing_secret. Every outbound request includes an X-OtterAB-Signature header containing an HMAC-SHA256 hex digest of the raw request body, keyed with your signing_secret. To verify: read the header, recompute HMAC-SHA256(secret, body), and compare in constant time. Mismatches mean the payload didn't come from us — reject. Slack webhooks are unsigned because Slack treats the URL itself as the secret.

### What HTTPS / network restrictions apply to webhook URLs?

Three rules enforced at save time: (1) the URL must use HTTPS — plain HTTP is rejected; (2) the URL cannot point at localhost, 127.0.0.1, 0.0.0.0, or ::1; (3) the URL cannot resolve to a private or link-local IP address. These prevent webhooks from being weaponized against the Otter server's internal network. Slack-kind destinations additionally must start with https://hooks.slack.com/.

### Will the same event fire twice if I retry?

No. Every delivery is keyed by a subject_key that's unique per subscription per event. If a delivery already exists for a (subscription, subject) pair, the system reuses it instead of inserting a duplicate. Failed deliveries are retried — the same subject_key gets retried, never re-fired.

### What does the delivery lifecycle look like?

Four states: pending (created, not yet attempted), delivering (a worker has claimed it and is sending now), delivered (success), and failed (last attempt errored). Deliveries stuck in delivering for over 5 minutes are auto-reclaimed for retry. Failed deliveries track an attempts counter and the last_error message so you can debug.

### How do I configure the daily digest?

Add a daily_digest subscription on a destination and set its schedule with two fields: hour (0-23) and timezone (any IANA timezone, e.g. America/New_York). The digest fires once per day at that local hour. The schedule is required for daily_digest subscriptions and rejected as invalid otherwise.

### Why might I want a generic webhook over Slack?

Three common reasons: (1) you want to drive a tool that isn't Slack — Discord, Mattermost, PagerDuty, your own dashboard; (2) you want to update a record in your own database when a test completes; (3) you want richer routing logic than what Slack lets you express. Generic webhooks send a JSON body you can parse and act on however you want.

### What's in the webhook payload?

Every payload includes the event name, a timestamp, the account and project context, the test being notified about (with its current state, traffic, primary goal, and statistical contract fields), and an activity block with the action, the actor name (or "System" for automated transitions), and any event-specific metadata. Inspect a real delivery in the dashboard to see the exact shape.

### Do scheduled or auto-stopped transitions fire notifications too?

Yes. A scheduled start fires test.started just like a manual Start; a scheduled end fires test.completed just like a manual Complete; and an auto-stop on visitor or conversion limit fires test.completed. The activity block in the payload lets you tell which path triggered it — the actor is "System" for automated transitions and a teammate name for manual ones. The decision-threshold-reached auto-stop is the one exception: it fires test.decision_threshold_reached instead of a duplicate test.completed.

### Can I turn a destination off without deleting it?

Yes — every destination has an active flag (and so does every subscription). Toggle it off to pause delivery without losing the configuration. Turn it back on and deliveries resume. The Notifications tab in account settings exposes the toggle.

---

Canonical page: https://www.otterab.com/docs/developer-reference/notifications