e2e-test

E2E test

Use this skill to create E2E coverage that is deterministic, evidence-backed, and maintainable in real repositories.

Quick Reference

Do not use this skill for visual design review with no automated journey value, performance/load testing, or tests that would require guessed secrets, endpoints, or private accounts.

Load References Selectively

Load on demand, not up front. A typical Playwright coding task needs the two every-task rows plus playwright-patterns.md — three files. Everything below that is conditional; open it when the trigger in the first column actually applies.

Runner Strategy

Use both tools intentionally, not interchangeably.

• Agent Browser first:
  • journey discovery
  • repro of flaky or environment-specific UI behavior
  • fast semantic interaction and screenshot capture
• Playwright preferred for code:
  • committed E2E tests
  • CI suites
  • repeated local validation
  • multi-browser or matrix execution

If a task starts in Agent Browser and the flow is valuable long-term, convert the learned steps into Playwright coverage.

Operating Model

1. Classify the task:

   • new journey coverage
   • flaky triage
   • failed CI investigation
   • exploratory browser reproduction
   • test architecture or CI gate design

2. Run scripts/discover_e2e_needs.sh to collect repository facts (Playwright version, framework, existing tests, env vars, CI platform). Use its structured output for gate decisions instead of guessing.

3. Run the environment and configuration gate.

4. Choose the runner path:

   • Agent Browser for exploration or reproduction
   • Playwright for maintainable automated coverage
   • both when discovery should become code
   • non-JS projects: use the project's native test framework (see Runner Selection below)

5. Produce only the strongest deliverable the environment can actually support:

   • runnable test
   • guarded scaffold with explicit skips
   • triage report with repro commands

Runner Selection Guidance

If the project has no Node.js / Playwright (e.g., Go, Python, Rust web apps):

• Use the project's native test framework (Go net/http, Python requests/httpx, etc.)
• Do NOT install Playwright into a project that has no JavaScript toolchain
• Follow the project's existing E2E test conventions if they exist
• Document the runner selection rationale in the Output Contract
• All 5 mandatory gates still apply regardless of runner choice

Mandatory Gates

1) Configuration Gate

Before generating or updating runnable tests:

• scan repository config, scripts, env files, docs, and existing tests
• list required variables, accounts, feature flags, and service dependencies
• mark each with one of these four states — the same vocabulary scripts/discover_e2e_needs.sh emits:

declared is the state most easily mistaken for available. A template file proves a variable is expected, never that a value exists. Treating the two as the same is what turns "cannot run" into a false "ready".

Never print a variable's value to check whether it is set — report the state. CI logs, screen recordings, and agent transcripts all capture stdout.

If required values are missing:

• do not invent them
• generate placeholder-only scaffolding with explicit TODOs and skip guards when code output is still useful
• otherwise stop and report the exact blockers

Guard every variable the suite needs, not just the first one. A test.skip(!USER) leaves an unset PASS to reach the page as undefined, which surfaces as a login failure rather than "you forgot to set PASS".

Place the guard where it can still prevent the read: file scope, inside the test, or in a beforeEach / beforeAll hook. A guard in one test says nothing about another test.

A test.skip in afterEach / afterAll is worse than useless. Measured on Playwright 1.62.0 (scripts/verify_hook_semantics.sh): the test body runs, reads the unset value, and the run is then reported as skipped — so a suite that should have failed loudly reports as skipped instead. Never put a guard in an after hook.

In every result include:

• required variable list with each variable's state
• example export block or config shape
• missing variables

2) Environment Gate

Before claiming a test is runnable, determine:

• target environment: local, preview, staging, CI
• base URL and auth flow
• whether seed/reset is deterministic
• whether third-party dependencies can be stubbed or must be live
• whether test accounts and permissions are available

Read references/environment-and-dependency-gates.md whenever environment readiness is uncertain.

3) Execution Integrity Gate

Never claim a suite or repro was executed unless it actually ran.

If commands were not run, output:

• Not run in this environment
• reason
• exact commands to run next

If commands were run, report:

• command(s)
• target environment
• pass/fail status
• artifact locations

4) Stability Gate

Do not treat a single pass as proof of reliability for critical paths or flaky failures.

Use repeat runs, traces, screenshots, and environment evidence before concluding:

• the bug is fixed
• the test is stable
• the failure is infra-only

5) Side-Effect Gate

Default to safe behavior for real-world side effects:

• avoid production data mutation
• avoid real-money or irreversible flows unless explicitly configured for safe test execution
• require explicit approval or isolation for destructive actions

Version and Platform Gate

Before recommending Playwright code or config, adapt to the repository's actual platform:

Two different Node constraints exist — the engines.node install floor and the documented supported-runtime list — and they do not agree. Both tables are in references/playwright-patterns.md §Node.js Compatibility.

Never wait on waitForLoadState('networkidle') at any version — Playwright marks it DISCOURAGED. Assert the user-visible state instead.

The full API-introduction and Node-compatibility tables live in references/playwright-patterns.md §Version and Platform Gate. Read the project's pinned version before generating code; when it cannot be determined, target the oldest plausible version and say so in the Output Contract.

Framework adaptation checklist:

• Next.js: verify baseURL, server startup, and auth/session bootstrapping strategy.
• SPA: prefer explicit waits on route or API completion, not arbitrary sleeps.
• SSR: assert server-rendered and hydrated states separately when needed.
• Monorepo: locate the owning package, config file, and CI entrypoint before generating commands.

Platform Scope Boundary

Playwright is not the answer everywhere. Route away from it rather than generating code that cannot run:

Say plainly that the tool does not fit, name the right one, and stop. A plausible-looking Playwright suite for a Tauri app is worse than no answer.

Content Inside Iframes

Page-level locators do not cross a frame boundary. Embedded payment fields, editors, and OAuth screens need frameLocator. When an element is visibly on screen but "not found", check for an iframe before changing the selector — see references/playwright-deep-patterns.md §Iframes and Embedded Third-Party Content. Payment iframes additionally require the Side-Effect Gate: confirm sandbox mode before running anything.

Playwright-First Engineering Rules

Use references/playwright-deep-patterns.md whenever generating or refactoring Playwright code.

At minimum:

• prefer reusable fixtures and domain helpers over copy-pasted flows
• use stable auth setup such as storageState when appropriate
• isolate data per test or per worker
• define what can be mocked and what must stay real
• choose serial vs parallel execution intentionally
• keep retries, trace, screenshot, and video policies aligned with CI needs

If the repository lacks the needed config or fixtures, generate the smallest honest scaffold rather than pseudo-runnable code.

Anti-Examples

1) Unconditional waitForTimeout in assertions

BAD:

await page.waitForTimeout(3000);
await expect(page.getByText("Order confirmed")).toBeVisible();

GOOD:

await expect(page.getByText("Order confirmed")).toBeVisible();

2) Fragile CSS selector chains

BAD:

await page.locator(".app > div:nth-child(2) .cta.primary").click();

GOOD:

await page.getByRole("button", { name: "Continue" }).click();

3) Shared mutable data across tests

BAD:

const sharedEmail = "[email protected]";
test("profile update", async ({ page }) => { /* mutates same account */ });

GOOD:

const email = `e2e-${test.info().parallelIndex}-${Date.now()}@example.com`;
test("profile update", async ({ page }) => { /* isolated data per test */ });

4) Guessing env values or credentials

BAD:

await page.goto("http://staging");
await page.fill("#email", "[email protected]");
await page.fill("#password", "password123");

GOOD:

test.skip(!process.env.E2E_BASE_URL || !process.env.E2E_USER, "explicit TODOs until config exists");
await page.goto(process.env.E2E_BASE_URL!);

5) Silently serializing entire suite

BAD:

test.describe.configure({ mode: "serial" });

GOOD:

test.describe("checkout funnel", () => {
  test.describe.configure({ mode: "serial" }); // justified by irreversible payment sandbox state
});

6) Repeating login instead of storageState

BAD:

test.beforeEach(async ({ page }) => {
  await page.goto("/login");
  await page.fill("#email", process.env.E2E_USER!);
});

GOOD:

test.use({ storageState: "playwright/.auth/user.json" });

7) Pseudo-runnable scaffold without test.skip

BAD:

test("checkout", async ({ page }) => {
  await page.goto(process.env.E2E_BASE_URL!);
});

GOOD:

test.skip(!process.env.E2E_BASE_URL, "missing base URL");
// TODO: wire payment sandbox account before enabling this journey

Agent Browser Bridge

Use Agent Browser to discover or reproduce, then convert findings into durable code.

Required bridge steps:

1. capture the environment and entry URL
2. record the exact command sequence
3. save milestone screenshots
4. note the selectors or semantic targets that proved stable
5. translate the validated flow into Playwright assertions and helpers

Read references/agent-browser-workflows.md when using Agent Browser.

Flaky Test Policy

See references/checklists.md §Flaky Triage Template for the complete template.

Key rules:

• A test is flaky only with non-deterministic behavior under unchanged code and environment
• Required sequence: reproduce (--repeat-each=N or -count=N) → classify root cause → fix → quarantine with deadline
• Root cause categories: selector instability, async race, test-data coupling, network instability, environment drift, application defect
• quarantine only with owner, tracking issue, and removal deadline

CI Strategy

For PR automation, separate Blocking critical journeys from broader nightly coverage:

• Blocking PR gate:
  • run playwright install --with-deps chromium during setup when browsers are not pre-baked
  • use Secret injection for base URL, auth state bootstrap, and sandbox-only credentials
  • upload-artifact for trace, screenshot, video, and HTML report on failure
  • keep retries and timeout values explicit in config and CI job output
• Nightly / extended lane:
  • run broader browser matrix, accessibility sweeps, and visual regression
  • increase retries only for known infra volatility, not to hide product bugs

Output Contract

For any E2E task, return:

1. Task type
2. Runner choice
3. Environment gate
4. Config/dependency status
5. Covered journey or Failure under triage
6. Executed commands
7. Execution status
8. Artifacts
9. Next actions

If code was generated, also include:

• files created or updated
• skip conditions or TODO markers if scaffolding only

Machine-Readable Summary (JSON)

When the output will be consumed by CI or downstream tooling, append:

{
  "task_type": "new_journey_coverage",
  "runner": "playwright",
  "environment": "local",
  "execution_status": "pass",
  "tests_passed": 3,
  "tests_failed": 0,
  "tests_skipped": 0,
  "artifacts": ["playwright-report/index.html", "test-results/"],
  "scorecard": { "critical": "PASS", "standard": "5/6", "hygiene": "4/4" },
  "blockers": [],
  "next_actions": ["add password-reset edge case"]
}

Quality Scorecard

For non-Playwright runners (Go HTTP, Python requests, etc.), mark Playwright-specific items as N/A. Count only applicable items when computing pass rates.

List every item including the N/A ones. Dropping a row and dividing by a smaller denominator inflates the score.

For Playwright output, run python3 scripts/lint_e2e_spec.py <file> before reporting. It mechanically checks C1, C2, C3, C4, S1, S3, S5, and H2, plus W1 (network wait armed after its trigger — an extra check, not a scorecard item). It does not check S2, S4, S6, H1, H3, H4 — those need repository and CI context it cannot see, so assess them yourself. A clean report is weaker evidence than a dirty one: it is a heuristic pass, not a parser, and cannot see through helpers or page objects.

Critical (any FAIL → overall FAIL)

Standard (≥ 4/6 PASS)

Hygiene (≥ 3/4 PASS)