# Website Testing Checklist: 10 Steps for Better Experiments _2026-10-06_ A promising variant appears to be winning. The dashboard shows more conversions, but nobody has checked whether the purchase event fires consistently, whether traffic is allocated correctly, or whether the result comes from desktop users while mobile visitors struggle with the new layout. The team hasn't reviewed Core Web Vitals, consent states, checkout errors, or whether the test has enough evidence to support a decision. That isn't an experiment you can safely roll out. A reliable website testing checklist is an operational control system that follows a test from pre-launch QA through audience allocation, measurement, statistical validation, performance monitoring, rollout, rollback and reporting. It should protect both the user experience and the business metric. The ten checks below cover conversion goals, copy, CTAs, layouts, forms, pricing, segmentation, statistical validation, multivariate testing and personalisation. Otter A/B can support this workflow with traffic splitting, goal tracking, significance calculations, Slack notifications and password-protected, brandable reports. The platform's preview and QA tools can also help teams inspect variants before exposing them to the full audience. ## 1. Conversion Rate Testing A test needs one clearly defined business action before anyone edits a page. That action might be a completed purchase, a qualified lead, a signup or a download. Define the primary conversion, secondary actions and guardrail metrics in the experiment brief, then confirm that each event fires once, carries the correct value and remains attributable to the right variant. For an online shop, the critical path runs beyond the product page. Test product discovery, cart updates, delivery selection, payment failure, confirmation, refunds and revenue reporting. UK internet retail sales reached **£2,223.8 million in August 2024** and **£2,329.7 million in September 2024**, according to the [Office for National Statistics retail internet sales series](https://www.ons.gov.uk/businessindustryandtrade/retailindustry/timeseries/je2j). A broken checkout event or duplicate order can therefore corrupt both customer experience and experiment conclusions. ### Measure business value, not just clicks Conversion rate gives you the first comparison, but it doesn't always identify the better commercial outcome. Track **revenue per variant**, average order value and refunds where the business model supports them. A variant that creates more low-value actions may lose to one that produces fewer, higher-value transactions. Before launch, complete a controlled pass that covers: - **Goal integrity:** Trigger each conversion action in a test environment and verify the recorded value. - **Journey coverage:** Run representative paths as a new visitor, returning visitor, logged-in user and guest. - **Mobile behaviour:** Test current iOS and Android devices, small and large screens, major browsers and slower connections. - **Analytics continuity:** Confirm product views, add-to-cart events, checkout steps, orders, refunds and revenue reach the intended analytics systems. Use [landing page optimisation advice from Assertive Agency](https://assertiveagency.com/8-common-landing-page-optimization-mistakes-and-how-to-avoid-them/) as a qualitative review prompt, not as a substitute for event and transaction validation. ## 2. Headline and Copy Testing Copy tests are easy to launch and easy to contaminate. Changing the headline, subheading, CTA and supporting proof at the same time may produce a lift, but it won't tell the team which message caused it. Start with a specific hypothesis, such as, “A clearer outcome-led headline will help first-time visitors understand the offer,” then change the smallest useful element. A SaaS team might compare a product-led headline with an outcome-led alternative. An ecommerce team could test whether delivery reassurance belongs beside the product title or nearer the purchase action. An agency might test different value propositions across several client sites, provided each client has separate goals, audiences and reporting. ### Keep the message test interpretable Use the page's traffic source and intent as part of the brief. Paid visitors may arrive with a narrow offer expectation, while organic visitors may need more explanation. If the treatment changes the promise made in the ad, search result or email, record that relationship rather than treating the landing page as an isolated asset. The practical copy review should include: - **Message clarity:** Can a new visitor identify the audience, offer and next step without relying on surrounding context? - **Claim support:** Does every material promise have visible evidence, terms or qualifying information? - **Microcopy behaviour:** Do error messages, consent text and delivery details remain accurate in every variant? - **Accessibility:** Does the new wording preserve meaningful link purpose, heading structure and accessible names? For a focused editorial pass, use this guide on [how to write better headlines](https://www.otterab.com/blog/how-to-write-better-headlines), then validate the resulting hypothesis with behavioural data. A polished sentence isn't a winning experiment until the measurement, audience and user journey hold up. ## 3. Call-to-Action Testing A CTA test should answer more than “Which button gets more clicks?” A click can indicate interest, confusion or an accidental interaction. Connect the CTA to the next meaningful action, such as a completed form, successful checkout or activated account, and retain the click as a diagnostic metric. Test one dimension at a time where possible. Button wording, placement, contrast, size, surrounding explanation and repetition each affect how users interpret the next step. A first-person phrase may feel more direct for a trial signup, while a descriptive phrase may reduce uncertainty for a complex service. The right choice depends on intent and context, not on a universal colour or wording rule. ![A sketched illustration of a website interface highlighting design elements like button size and page whitespace.](https://cdnimg.co/3716ee4f-bd1a-44a8-ac85-c2df5af21725/d6b02544-cba0-4d7e-95e3-69b408255a48/website-testing-checklist-ui-design.jpg) ### Check the action after the click A visually prominent CTA can still fail keyboard users, mobile users or visitors who need more information before committing. Check focus visibility, tab order, tap-target usability, loading states, disabled states and error recovery. If the CTA opens a modal, the modal must receive focus correctly and return focus to a sensible location when it closes. > **Practical rule:** Treat the CTA as a complete interaction, not a coloured rectangle. Test the state before the click, the response during the click and the result after the action. Consent state also belongs in the test plan. The [Information Commissioner's Office guidance on cookie compliance action](https://ico.org.uk/about-the-ico/media-centre/news-and-blogs/2025/01/ico-takes-action-to-tackle-cookie-compliance-across-the-uk-s-top-1-000-websites/) shows why teams need to test meaningful choice, not just whether a banner appears. Check reject-all, granular choices, withdrawal, consent logging and whether experiment identifiers or analytics tags activate before permission. ## 4. Layout and Design Testing Layout tests often change several user decisions at once. Moving a form, changing the hero structure or removing a sidebar can alter comprehension, trust, scanning behaviour and interaction cost. That makes the hypothesis more important than the visual difference. State what the new structure should improve, for whom and on which journey. Test the control and treatment at realistic viewport sizes, not only in a design file. Check text wrapping, image cropping, sticky elements, navigation, modals, error states and content that loads asynchronously. A layout that looks balanced on a large monitor may push the primary action below the useful content on a narrow screen. ![A hand-drawn comparison sketch illustrating responsive website layout differences between mobile and desktop screen interfaces.](https://cdnimg.co/3716ee4f-bd1a-44a8-ac85-c2df5af21725/1875827c-438d-42c0-b14b-36f52d1f0a2e/website-testing-checklist-responsive-design.jpg) ### Use observation to explain the result Quantitative results tell you which variant performed better against the selected goal. Session recordings, search behaviour, support queries and moderated task reviews can help explain why. Use those observations as diagnostic evidence, while keeping the primary experiment decision tied to pre-defined metrics. Accessibility must be part of layout release QA. The UK public-sector accessibility regulations came into force on **23 September 2018** and require public-sector digital services to meet accessibility obligations and publish accessibility statements, as described in the [GOV.UK accessibility requirements](https://www.gov.uk/guidance/accessibility-requirements-for-public-sector-websites-and-apps). Test keyboard navigation, visible focus, colour contrast, headings, form labels, image alternatives, zoom, reflow and accessible names and roles. A visual winner that introduces focus loss, low contrast or unreadable reflow isn't ready for rollout. Fix the experience or reject the variant, even if the aggregate conversion metric looks favourable. ## 5. Form Optimisation Testing Forms expose friction in a way that page-level metrics can hide. A visitor may click the CTA, start typing and abandon after an unclear error, an unexpected required field or an input that doesn't work with autofill. Record starts, field errors, completions and downstream quality, not only final submissions. Begin with the smallest change that can answer the question. Remove a non-essential field, improve the label or alter the order before redesigning the entire form. For higher-friction flows, compare a single-page form with a multi-step experience, but make sure each step preserves entered data and clearly communicates progress. ### Test successful recovery A form isn't working properly because valid data submits once. Test invalid email formats, missing required fields, pasted values, browser autofill, keyboard navigation, interrupted sessions, duplicate submissions and server-side errors. Error messages should identify the field, explain the correction and remain available to assistive technology. The release pass should include: - **Field semantics:** Use appropriate input types, labels, autocomplete values and accessible descriptions. - **Validation timing:** Avoid showing errors before a user has had a fair opportunity to complete the field. - **Recovery:** Preserve entered information when a submission fails and make the next action obvious. - **Transaction integrity:** Confirm that one user action creates one intended lead, account or order. Automated accessibility scanning helps identify common failures, but it doesn't prove that a person can complete the journey. The [2020 to 2021 GOV.UK monitoring cycle](https://www.gov.uk/guidance/accessibility-requirements-for-public-sector-websites-and-apps) found sampled issues involving focus visibility, accessible names and roles, colour contrast, and information and relationships. Pair automated checks with manual keyboard, screen-reader, responsive-layout, form and content testing before a major release. ## 6. Pricing Page and Product Display Testing Pricing tests carry more commercial and ethical risk than a headline test. Decide whether the experiment changes price, packaging, presentation or perceived value, then define how existing customers, returning visitors and users with saved carts will be treated. A visitor shouldn't see inconsistent terms because a test assignment changed halfway through a purchase journey. For SaaS, useful comparisons include monthly versus annual framing, feature grouping, plan order and the prominence of usage limits. For ecommerce, test image order, product information, delivery details, discount framing and trust signals. Don't judge these experiments by conversion rate alone. Monitor revenue per variant, average order value, refunds, cancellations and support contacts. ### Separate presentation from price A pricing display test can often run safely across a shared price if the question concerns clarity or hierarchy. A true price test needs stronger controls. Consider sequential or time-based approaches, document eligibility, keep assignment stable and involve legal, finance and customer-support owners before launch. The experiment brief should state whether tax, delivery charges, currency and promotional terms remain identical. Useful checks include: - **Price accuracy:** Confirm every displayed and submitted amount matches the intended catalogue or billing value. - **Eligibility:** Exclude users whose existing contract, voucher or account status makes the treatment invalid. - **Persistence:** Ensure the selected price remains consistent through cart, checkout, confirmation and receipt. - **Commercial guardrails:** Review margin, refunds, cancellation and order value alongside the primary conversion. A pricing winner isn't the variant with the highest immediate purchase rate. It is the treatment whose commercial result remains acceptable after the full customer journey is verified. ## 7. Traffic Segmentation and Audience Testing Traffic allocation affects experiment validity. Before launch, decide whether the test covers all eligible visitors or a defined segment. Record device, location, referrer, customer status and behavioural rules before assigning anyone. If the audience mix changes during the test, the result becomes difficult to reproduce or interpret. Keep the audience broad enough to answer the primary question. Create segments only when user intent or product context justifies them. Mobile and desktop visitors may face different constraints, so compare them when the experience differs. A segment report should replace the overall result only when the experiment was designed to make that decision. ### Protect allocation quality Verify that users keep the same variant between sessions, excluded traffic stays excluded and redirects, caches, consent choices or login states cannot overwrite assignments. Test every rule in a QA environment. After launch, inspect assignment logs and analytics for unexpected splits. Privacy checks belong in the same pre-launch workflow: - **Consent states:** Compare measurement for accepted, rejected and withdrawn consent where lawful and technically possible. - **Regional defaults:** Confirm location rules apply the correct consent and content experience. - **Vendor activation:** Check that analytics and testing identifiers wait for the required permission. - **Reporting bias:** Label results by consent state, so missing measurement is not mistaken for poor treatment performance. Use the ICO's cookie compliance guidance when reviewing consent rules and tracking activation. Privacy QA should be completed before assignment begins, not added as a banner check after results appear. This protects both the experiment's evidence and the experience delivered to each audience. ## 8. Statistical Significance and Sample Size Testing A dashboard can show a leading variant long before the evidence supports a decision. Set the stopping rule before launch. Define the primary metric, minimum meaningful effect, intended audience, test duration assumptions, required sample and conditions for stopping because of harm or technical failure. Sample-size planning isn't a formality. If the site receives limited eligible traffic, the team may need to run the test longer, simplify the hypothesis or choose a larger effect to detect. Don't promise a reliable answer from a test that cannot collect enough relevant observations. ### Validate the decision, not just the confidence label Otter A/B's frequentist z-test engine is designed to calculate significance continuously at a **95% confidence threshold**, but the team still needs to interpret the output responsibly. Statistical significance doesn't establish practical importance, data quality or long-term performance. Review the absolute result, the business value, segment consistency and guardrail metrics. The government monitoring programme assessed **1,203 public-sector websites and 21 mobile apps between January 2022 and September 2024** against WCAG Level A and AA success criteria, according to the [GOV.UK accessibility monitoring report](https://www.gov.uk/government/publications/accessibility-monitoring-of-public-sector-websites-and-mobile-apps-from-2022-to-2024/accessibility-monitoring-of-public-sector-websites-and-mobile-apps-from-2022-to-2024). The operational lesson is to define representative coverage and document failures, owners and retests. Experiment analysis deserves the same discipline. Use this resource on [how to calculate sample size](https://www.otterab.com/blog/how-to-calculate-sample-size) when preparing the brief. Don't stop because the graph looks exciting, and don't extend a test indefinitely because the result is inconvenient. Follow the agreed rule, investigate anomalies and record any deviation. ## 9. Multivariate Testing and Sequential Testing Begin with a clean A/B test. Two variants make allocation, QA, analysis and communication easier. Multivariate testing becomes useful when you have a reason to expect interaction effects, such as a price presentation that only works with a particular trust message or a headline that depends on a supporting image. The trade-off is interpretability and traffic. Each additional combination creates another experience to validate and another result to explain. Before launching, list every combination, confirm that each one has valid content and calculate whether the available audience can support the decision. Don't use multivariate testing to bundle unrelated ideas just because the platform allows multiple variants. ### Sequence learning when traffic is constrained Sequential testing lets a team use one answer to inform the next question. First test the message, then carry the strongest message into a CTA test, then assess the resulting page structure. This approach reduces simultaneous combinations and creates a clearer learning record, although it takes longer to explore interactions and can carry an early mistake into later phases. A practical sequence looks like this: - **Phase one:** Establish which core proposition or page direction deserves further investment. - **Phase two:** Test the action language, placement or form treatment within that direction. - **Phase three:** Validate the combined experience against revenue, quality and performance guardrails. - **Decision point:** Roll back or redesign if the later change weakens the original business outcome. For teams considering adaptive approaches, the [adaptive testing guide](https://www.otterab.com/blog/adaptive-testing) provides a useful planning reference. Keep the experiment log explicit about which phase produced each conclusion. ## 10. Personalisation and Dynamic Content Testing Personalisation changes the unit of analysis. Instead of asking whether one page beats another for everyone, the team asks whether a defined experience helps a defined audience under defined conditions. That can be useful for new versus returning visitors, referrer-specific expectations, product interest or onboarding context, but it introduces more rules to test and more ways for assignment to fail. Start with a simple distinction that the analytics system can identify reliably. For example, a first-time visitor might receive orientation content while a returning visitor sees a deeper product route. Test the personalisation approach against a consistent control, rather than assuming that a more customized experience is automatically better. ### Make the rules observable Write each condition in plain language, then map it to the implementation. Check what happens when a user changes device, clears storage, signs in, rejects consent or arrives through an unexpected referrer. Personalised content should fail safely, with a valid default page and no broken layout when data is missing. Before rollout, verify: - **Rule precedence:** Decide which experience wins when a visitor matches multiple audiences. - **Fallback behaviour:** Serve a complete control experience when the required attribute isn't available. - **Measurement separation:** Report performance by audience and treatment, not only as one combined total. - **Privacy controls:** Collect and use first-party data according to the applicable consent and governance requirements. - **Content consistency:** Keep prices, availability, legal terms and accessibility information accurate in every dynamic state. Personalisation needs a debugging pass before a statistical pass. If the team can't explain who saw which experience and why, it can't trust the resulting comparison. Use [ecommerce personalisation at scale](https://heycarti.com/blog/personalization-at-scale) as a strategic reference, then document the actual rules, fallback and measurement plan in the experiment record. ## 10-Point Website Testing Checklist Comparison | Approach | Implementation Complexity 🔄 | Resource Requirements ⚡ | Expected Outcomes ⭐📊 | Ideal Use Cases 💡 | Key Advantages | |---|---:|---|---|---|---| | Conversion Rate Testing (CRT) | Medium 🔄, experiment setup & revenue tracking | Moderate ⚡, analytics, tracking integration, sufficient traffic | High ⭐, measurable revenue and AOV lifts 📊 | E‑commerce checkout, SaaS signups, revenue-focused pages | Direct revenue attribution; stakeholder-friendly | | Headline and Copy Testing | Low 🔄, content-only variants | Low ⚡, marketing time, minimal dev | Medium–High ⭐, fast conversion uplifts 📊 | Landing pages, CTAs, email subject lines | Highest ROI; quick to iterate | | Call-to-Action (CTA) Testing | Low–Medium 🔄, visual and placement tweaks | Low ⚡, design time; visual builder often sufficient | High ⭐, immediate click/conversion improvements 📊 | Buttons, checkout CTAs, signup flows | High-impact, easy to test; generalizes well | | Layout and Design Testing | High 🔄, front-end and design changes | High ⚡, designers, devs, more traffic | High ⭐, significant UX-driven conversion gains 📊 | Homepage, checkout, complex landing pages | Improves UX; uncovers usability issues | | Form Optimization Testing | Medium 🔄, form logic & validation changes | Moderate ⚡, backend integration, tracking | High ⭐, clear increases in form completions 📊 | Lead gen forms, signups, multi-step checkout forms | Reduces friction quickly; strong ROI | | Pricing Page & Product Display Testing | Medium–High 🔄, pricing logic & messaging changes | High ⚡, revenue tracking, marketing/legal input, traffic | Very High ⭐, direct AOV and revenue impact 📊 | Pricing pages, product listings, bundles | Direct revenue impact; informs pricing strategy | | Traffic Segmentation & Audience Testing | Medium–High 🔄, segmentation rules and targeting | Moderate ⚡, audience data, analytics, increased sample needs | High ⭐, segment-specific lifts and insights 📊 | Paid vs organic, geo/device targeting, onboarding | Prevents false winners; reveals audience differences | | Statistical Significance & Sample Size Testing | Low–Medium 🔄, setup of rules and monitoring | Low ⚡, platform automates calculations; needs traffic | Critical ⭐, ensures results are valid and reliable 📊 | All experiments requiring confident decisions | Prevents false positives; provides mathematical rigor | | Multivariate & Sequential Testing | High 🔄, complex design and interaction analysis | Very High ⚡, many variants, large traffic, advanced analysis | Variable ⭐, detects interactions but needs scale 📊 | High-traffic sites testing interacting variables | Reveals interaction effects; sequential testing saves traffic | | Personalization & Dynamic Content Testing | High 🔄, data logic, targeting rules, integration | Very High ⚡, CDP/data infra, engineering, privacy compliance | High ⭐, greater relevance, engagement, and conversions 📊 | E‑commerce recommendations, role-based onboarding, localization | Scales tailored experiences; leverages first‑party data ## Turn Test Results Into Safer Decisions A result becomes useful only after the team can explain how it was produced. Start the post-run review by confirming the winning metric, eligible audience and segment. Check that the primary event fired correctly, that allocation was stable and that the result wasn't driven by duplicate orders, missing consent, bot traffic, a broken integration or a device-specific defect. Then review significance alongside guardrails. A treatment may lead on conversion while creating worse loading, layout movement, accessibility or revenue quality. The UK ecommerce performance study from [Ryte on Core Web Vitals readiness](https://en.ryte.com/company/newsroom/study-top-ecommerce-websites-uk-reveals-wide-ranging-lack-readiness-googles-core-web-vitals/) found that **93.75% of mobile sites** in its sample received a poor Core Web Vitals classification, compared with **33.93% of desktop sites**. The same source reported poor mobile Largest Contentful Paint for **91.96% of analysed domains**. Those findings reinforce a practical rule: measure each variant on realistic mobile conditions and don't approve a conversion lift that damages the experience. Review homepage, category, product, cart and checkout templates separately when the treatment affects more than one page. Third-party scripts, consent tools, image payloads and dynamic content can change performance from one template to another. Check cold and repeat loads, interaction responsiveness, layout stability and field data after release. ### Record the learning in a form others can use Your final report should answer six questions: - **What was the hypothesis:** Which user problem or business opportunity did the test address? - **Who was included:** Which audience, devices, locations, consent states and exclusions applied? - **What won:** Which metric and segment produced the decision, and was the result statistically supported? - **What changed:** Did the treatment affect revenue, average order value, quality, accessibility, performance or support demand? - **What happens next:** Will the team roll out, iterate, continue monitoring or reject the treatment? - **How do we recover:** What specific condition triggers a rollback? Share a password-protected report with stakeholders and link the result to the implementation ticket, analytics definition and QA evidence. Otter A/B can support this handoff with significance alerts, revenue-aware reporting and stakeholder sharing, but the report still needs human interpretation. A notification can identify a milestone. It can't explain why a segment behaved differently or whether a trade-off is acceptable. Roll out gradually when the risk warrants it, monitor the primary metric and guardrails, and keep the control available until the new experience has passed its agreed observation period. A winner is valuable when the team can reproduce the setup, defend the trade-off and apply the learning to the next experiment. A dashboard number without that operating discipline is only a promising lead. --- Use Otter A/B to define goals, split traffic, preview variants and monitor experiment significance as your website testing checklist moves from QA to rollout. Visit [Otter A/B](https://www.otterab.com) to set up a controlled experiment and share the result with your team or clients. --- Canonical page: https://www.otterab.com/blog/website-testing-checklist