QA and QoE Testing with Playwright MCP

When to use it: an agent needs to check a running app in a real browser —
"QA this", "does it actually work", "why is it slow", "measure Web Vitals /
LCP / CLS", "does it work on mobile", "screenshot it" — or before shipping any
UI change. Language- and framework-agnostic: it drives the browser, not your
stack. The why is QA and QoE; this is the how.

Two jobs, one browser, and they must be reported separately:

• QA — does it work? Binary, per-run. Exception, 500, dead control, form that
  won't submit.
• QoE — what is it like? Continuous, distributional. Time to first paint,
  layout shift, interaction latency, page weight.

A build passes every QA check and still takes six seconds to render. That is not
a pass, and a report format that can't express it will keep recording it as one.

Setup

claude mcp add playwright -- npx -y @playwright/mcp@latest

Or project-scoped and committed, in .mcp.json:

{ "mcpServers": { "playwright": { "type": "stdio",
  "command": "npx", "args": ["-y", "@playwright/mcp@latest"] } } }

The tool set changes between releases. Enumerate it rather than trusting a doc
(this one was written against 24 tools, server version 1.63.0-alpha):

printf '%s\n' \
 '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2024-11-05","capabilities":{},"clientInfo":{"name":"p","version":"1"}}}' \
 '{"jsonrpc":"2.0","method":"notifications/initialized"}' \
 '{"jsonrpc":"2.0","id":2,"method":"tools/list","params":{}}' \
 | npx -y @playwright/mcp@latest | jq -r '.result.tools[]?.name'

The tools

Five gotchas that produce false passes

Each of these returns a clean-looking result while hiding the defect.

1. Screenshots cannot drive actions. browser_take_screenshot is for humans;
browser_snapshot is for the agent. The snapshot is an accessibility tree whose
refs are what browser_click and browser_type target — the tool's own
description says you can't act on a screenshot. Useful corollary: an element
missing from the snapshot is itself a finding, because not in the a11y tree
means not reachable by a screen reader.

2. browser_network_requests hides static assets by default. static is a
required parameter that defaults to false, omitting images, fonts and scripts
— precisely what dominates page weight. Any QoE pass over asset size that forgets
it will report a light, fast page that isn't:

{ "static": true, "filter": "\\.(png|jpg|webp|woff2?|js|css)$" }

3. Console scope resets on navigation. browser_console_messages returns
messages since the last navigation unless you pass all: true. Errors thrown
during initial load disappear the moment you navigate again — the classic clean
console that isn't. For an end-of-run report always use
{ "level": "error", "all": true }.

4. scale is required on screenshots. The call is rejected without it. Use
"css" for device-independent output you can diff across runs; "device" only
when DPR rendering is the thing under test.

5. Long output belongs in a file. browser_snapshot,
browser_console_messages, browser_network_requests and browser_evaluate all
accept a filename — the result goes to disk instead of into the context window.
A snapshot of a real page runs to thousands of lines. Use browser_find when you
need one element; use filename when you genuinely need all of it.

Measuring QoE

browser_evaluate awaits a returned promise, so observers have time to settle.

Web Vitals

() => new Promise((resolve) => {
  const out = { ttfb: null, lcp: null, cls: 0, longTasks: 0 };
  const nav = performance.getEntriesByType('navigation')[0];
  if (nav) { out.ttfb = nav.responseStart; out.domContentLoaded = nav.domContentLoadedEventEnd; }

  new PerformanceObserver((l) => {
    const e = l.getEntries(); out.lcp = e[e.length - 1].startTime;
  }).observe({ type: 'largest-contentful-paint', buffered: true });

  new PerformanceObserver((l) => {
    for (const entry of l.getEntries()) if (!entry.hadRecentInput) out.cls += entry.value;
  }).observe({ type: 'layout-shift', buffered: true });

  new PerformanceObserver((l) => { out.longTasks += l.getEntries().length; })
    .observe({ type: 'longtask', buffered: true });

  setTimeout(() => resolve(out), 3000);
})

Core Web Vitals "good" thresholds: LCP ≤ 2.5s, CLS ≤ 0.1, INP ≤ 200ms — all
specified at the 75th percentile, not the mean, for the reason argued in
QA and QoE.

INP cannot be measured passively. It needs real interactions: drive the page
first, then read interaction latencies. What you get is the worst interaction
in one scripted run, which is an approximation of INP, not INP — the real metric
is a high percentile over field sessions.

() => new Promise((resolve) => {
  const events = [];
  new PerformanceObserver((l) => {
    for (const e of l.getEntries()) events.push({ name: e.name, dur: e.duration });
  }).observe({ type: 'event', buffered: true, durationThreshold: 16 });
  setTimeout(() => resolve({ worst: events.sort((a, b) => b.dur - a.dur).slice(0, 5) }), 1000);
})

Page weight

() => {
  const r = performance.getEntriesByType('resource');
  const by = {};
  for (const e of r) {
    const k = e.initiatorType || 'other';
    by[k] = (by[k] || 0) + (e.transferSize || 0);
  }
  return {
    totalKB: Math.round(r.reduce((s, e) => s + (e.transferSize || 0), 0) / 1024),
    byType: by, requests: r.length,
  };
}

Responsive breakpoints

browser_resize, then re-check. Horizontal overflow is the standard bug and the
DOM can prove it — no eyeballing a screenshot:

() => ({
  scrollW: document.documentElement.scrollWidth,
  clientW: document.documentElement.clientWidth,
  overflows: document.documentElement.scrollWidth > document.documentElement.clientWidth,
  culprits: [...document.querySelectorAll('*')]
    .filter((el) => el.getBoundingClientRect().right > document.documentElement.clientWidth + 1)
    .slice(0, 10)
    .map((el) => el.tagName + (el.className ? '.' + String(el.className).split(' ')[0] : '')),
})

Check 375 (mobile), 768 (tablet), 1440 (desktop) at minimum.

A run that doesn't lie

1. browser_resize to a fixed viewport — otherwise runs aren't comparable.
2. browser_navigate to the target.
3. browser_evaluate the Web Vitals snippet. Record the numbers.
4. browser_snapshot, then walk the critical path with browser_click /
   browser_fill_form.
5. browser_console_messages with { level: "error", all: true }.
6. browser_network_requests with { static: true }; pull any non-2xx through
   browser_network_request for headers and body.
7. browser_take_screenshot (scale: "css") only for findings a human must judge.
8. Report QA findings and QoE numbers in separate sections.

Rules

• Never report a pass from a screenshot alone. A page that renders can still
  throw on every click. Check the console.
• A slow page is a finding, even with every assertion green. That is the
  entire reason to measure QoE alongside QA.
• Measure cold when the number is the deliverable. A warm second run flatters
  LCP and page weight badly.
• One run is one sample. QoE is a distribution; a single LCP reading is an
  anecdote. Repeat before declaring a regression — and prefer a percentile over
  a mean whenever you have enough samples to have one.
• Avoid browser_run_code_unsafe. Its own description calls it
  RCE-equivalent; everything above works with browser_evaluate, which is scoped
  to the page.

See also

• QA and QoE — why the split matters, what Netflix measures, and the evidence
  that experience drives retention
• Trust but Verify — this is the browser-level arm of independent verification
• React: Rules & Project Structure (2026) — where E2E sits in a frontend stack