Skip to main content

Common Mistakes in Test Automation

Prerequisites: You should already understand every module in Sections 1–5 of this path. Leads to: After this, you'll be ready for the Automation Testing Capstone.

Every module in this path named mistakes specific to its own concern. This module steps back and asks which mistakes recur across concerns, visible only once you've seen the whole path. These aren't new rules — they're the handful of underlying habits that, left unchecked, undermine almost any concern this path taught.

Pattern 1: Automating by Convenience Instead of Risk and Value​

A team chooses what to automate based on what's easiest to script, rather than what actually protects the business.

How this leads to an escaped defect: Selecting the Right Test Cases for Automation's own example — a logistics company automating 150 easy, low-value checks while a genuinely complex, high-risk inventory-reconciliation feature stayed manual, where a real defect eventually shipped undetected.

Prevention: rank automation candidates by frequency and business risk together, explicitly, before considering implementation difficulty.

Pattern 2: Guessing a Wait Duration Instead of Waiting for the Real Condition​

A test pauses for a fixed, hopeful duration rather than waiting for the specific condition it actually needs to be true.

How this leads to an escaped defect: Synchronization and Wait Strategies's AtlasBank compliance-check example — a hardcoded pause that "usually worked" started failing the moment real processing time changed, and can just as easily mask a genuine timing-dependent defect as expose one.

Prevention: default to explicit waits targeting the actual condition, with a generous maximum timeout — never a guessed, fixed duration.

Pattern 3: Retrying an Intermittent Failure Instead of Diagnosing It​

A team treats a flaky test's automatic retry as a fix, rather than a symptom worth investigating.

How this leads to an escaped defect: Test Stability and Flaky Tests's central warning — an undiagnosed retry can mask a real regression exactly as readily as it masks a genuine test-authoring flaw, since the two look identical from the outside.

Prevention: diagnose an intermittent failure's root cause before relying on a retry as anything more than a temporary, documented mitigation.

Pattern 4: Asserting on Presence Instead of Correctness​

A test checks that something exists or appears, rather than checking that it has the specific, correct value or content.

How this leads to an escaped defect: Assertions and Verification Strategies's account-statement example — a download-triggered assertion missed a real truncation defect entirely, because presence was never the same claim as correctness.

Prevention: assert on the specific data or state a user would actually care about — exact text, exact values — not a weaker, easier-to-write proxy for it.

Pattern 5: Treating "Runs Automatically" as Equivalent to "Actually Enforced"​

A team builds and runs automation in CI without configuring it as a required, merge-blocking check.

How this leads to an escaped defect: CI/CD Integration's fintech startup example — a comprehensive, well-built suite that only ran via a manually-clicked button gradually stopped being run under routine time pressure, and a significant regression shipped undetected for three weeks.

Prevention: verify the specific "required check" configuration is actually in place — a suite that merely can run in CI provides little protection if nothing stops a failure from merging anyway.

Pattern 6: Letting Automation Debt Accumulate Without Active Maintenance​

A team only ever adds tests, never reviewing or removing ones that no longer earn their keep.

How this leads to an escaped defect: Maintaining Automation at Scale's four-year, 1,200-test example — roughly a third of the suite was silently testing removed or unrecognizably redesigned features, and nobody currently on the team could confidently say which third, eroding real confidence in what remained.

Prevention: treat a significant feature redesign as a deliberate trigger to review its existing tests, and schedule periodic suite health review as a recurring practice, not a reaction to an unavoidable problem.

Every pattern above has the same underlying shape: a shortcut that feels reasonable in the specific moment it's taken — guessing a wait duration because the real condition is harder to identify, retrying a failure because diagnosing it takes longer, keeping a test because deleting it feels riskier than it actually is. Recognizing the temptation itself as a signal worth pausing on, not just knowing the list of patterns, is what actually prevents them from recurring.

Key Takeaways​

  • Six recurring patterns cut across this path's individual modules: automating by convenience, guessing wait durations, retrying instead of diagnosing, asserting on presence instead of correctness, treating automatic execution as equivalent to enforcement, and letting automation debt accumulate unchecked.
  • Each pattern is a shortcut that feels reasonable in the specific moment it's taken — recognizing the moment of temptation is the actual skill, more than memorizing the list.
  • Every pattern traces back to a real, worked defect example from earlier in this path — these are patterns that have already produced real, specific consequences throughout this path's own teaching examples.
  • Prevention in every case is a specific, learnable habit, not a vague call for more thoroughness.

What You Just Learned​

  • Six mistake patterns that recur across this path's individual concerns, not within any single one
  • How each pattern, left unchecked, produced a real, specific defect in this path's own worked examples
  • The specific prevention habit for each pattern
  • Why recognizing the moment of temptation to take a shortcut matters more than memorizing a list of mistakes

Next: Automation Testing Capstone

Interview Questions​

Q1: What's a common mistake you've seen (or made) in test automation, and how would you prevent it?

What to look for: A specific, real pattern (not a generic "not testing enough") with a concrete prevention habit attached — ideally one of this module's six, explained in the candidate's own words, showing genuine understanding rather than a memorized list.

Q2: Why might "our tests run automatically in CI" not be enough to actually protect a codebase?

What to look for: A candidate who recognizes that running and reporting is different from actually blocking a merge, directly citing Pattern 5's distinction — a suite that runs but isn't a required check can still let a failing change merge.

Glossary​

Escaped Defect: A defect that reached production despite testing having occurred, typically traceable to a specific gap in test design or execution rather than an absence of testing effort.

Automation Anti-Pattern: A recurring, specific way test automation goes wrong, distinct from a one-off mistake — recognizable and preventable once named.

Quick Revision​

Remember these five points:

✓ Automate by risk and value (frequency + business risk), not by what's easiest to script.

✓ Default to explicit waits targeting the real condition — never guess a fixed pause duration.

✓ Diagnose an intermittent failure's root cause before relying on a retry as anything more than a temporary mitigation.

✓ Assert on specific, correct content — not just that something exists or appears.

✓ Verify automation is configured as a required, merge-blocking check — running automatically isn't the same as being enforced.