Five habits for turning a flaky test suite into a trusted one

October 2026 · ~5 min read

An end-to-end suite that fails at random is worse than no suite at all. Nobody can tell a real regression from noise, so people stop looking, and the suite ends up running out of habit rather than being trusted.

Over the years I've worked on several suites like that. Fixing them has rarely come down to one big change. These are the habits that have mattered most, with examples in Playwright.

1. "Flaky" is a diagnosis, not an explanation

The easiest thing to do with an intermittent failure is to add a retry and move on. I prefer a simple rule: every failure gets classified before anything changes.

• Test bug: the test is wrong, racy, or depends on something it doesn't control.
• Environment issue: the network, a third-party service, or a shared environment changing mid-run.
• Application bug: the product really is broken, sometimes only under timing the test happens to hit.

The third category is why retries are risky. An intermittent failure can be a real bug that only shows up under load or unusual timing, and that's exactly what a regression suite is meant to catch.

2. Look for "act, then wait" race conditions

One of the most common Playwright patterns I see causing flakiness is an action followed by a wait for the network response it triggers:

// Racy: the response can arrive before we start listening
await page.getByRole('button', { name: 'Save' }).click();
await page.waitForResponse(r => r.url().includes('/api/items'));

On a fast run, the response lands between those two lines and the wait hangs until it times out. The fix is to start listening before acting:

// Safe: the listener exists before the request is sent
const saved = page.waitForResponse(r => r.url().includes('/api/items'));
await page.getByRole('button', { name: 'Save' }).click();
await saved;

The same applies to typing into search fields, dialogs that load data on open, and anything else where the UI starts a request. Once you start looking for this pattern, you tend to find it in many places.

3. Parallel workers expose shared test data

Running tests in parallel is the obvious way to speed up a suite, and also the fastest way to find out which tests share data. Two tests editing the same record, or sharing one logged-in session, will pass alone and fail together.

• Each test creates the data it needs, ideally through an API rather than the UI, and cleans up after itself.
• Each worker gets its own authenticated session.
• Anything that genuinely has to be shared is reset to a known state before and after use.

Making the worker count configurable also helps diagnosis: if a test still fails with a single worker, concurrency isn't the cause.

4. Report results people actually read

A green suite isn't much use if nobody sees the result.

• Reports with history (Allure works well) make trends visible, so a test that fails once in twenty runs stands out.
• Short chat notifications after each run, listing failures and the reason for each, tell the team quickly whether a build is healthy.
• A one-line description per spec, checked in CI, keeps documentation from drifting away from what the tests actually do.

5. Keep CI fast and cheap to run

Large repositories, slow checkouts, and heavyweight setup all make people run CI less often. Frequent runs are how intermittent problems get found, so time spent making the pipeline lean pays for itself.

The short version

If you're working on something similar, I'm happy to compare notes. Reach me on LinkedIn or at syeddanishdev@gmail.com.