How do you decide what NOT to test, and how do you get the team to agree with it?
Strategy and riskSenior
In shortI make the out-of-scope list explicit, explain the reasoning in terms of risk and cost, and get the product owner to confirm it in writing. Skipping something openly is a decision; skipping it silently is a gamble.
Model answer
Every test plan leaves things out. The skill is doing it on purpose and out loud.
What I usually consider leaving out
- Areas that didn’t change and have no shared code with what did change, and have a clean history.
- Things already covered well at another level, such as validation rules covered by API or unit tests.
- Low-impact, low-usage features: if analytics show 0.5% of users open a screen and a failure would be cosmetic, it gets a smoke check at most.
- Platform combinations outside the agreed device and browser matrix.
- Third-party systems we don’t own, beyond checking our integration with them.
How I get agreement
- Write it down in the test plan under “Out of scope”, with one line of reasoning per item.
- Frame it as a trade-off, not a request: “If we test the old export screen, we lose half a day on the new checkout. I recommend skipping export because it hasn’t changed since March.”
- Ask the product owner to confirm. A quick “agreed” on the ticket or in the planning meeting is enough. It becomes the team’s decision, not mine.
- Name the trigger to revisit: “If the export code is touched, it moves back into scope.”
Example: for a redesign of an e-commerce home page, I proposed not testing the legacy gift card flow on old Android versions, which had a tiny share of users and no code changes. The PM agreed, and we spent the time on the new product carousel, where we found a crash on small screens.
If something I chose not to test breaks later, the written decision makes the retro about improving our risk judgment, not about blame.
Learn more: Writing a Test Plan
What they're looking for
Comfort with deliberately leaving things untested, clear criteria for it, and the communication skill to make it a shared, documented decision that can be revisited.
Common mistakes
- Claiming you test everything.
- Quietly skipping areas without telling anyone.
- Asking for approval with no reasoning, so stakeholders say "test everything".
- Never revisiting the list when the situation changes.
More strategy and risk questions
- You join a company building a new product from scratch. How do you create the test strategy?Senior
- Walk me through how you actually run risk-based testing when you have a fraction of the time you need.Senior
- You have a limited budget and need to choose a device and OS matrix for a mobile app. How do you decide?Senior
- How would you approach testing a big migration or redesign, for example moving to a new backend or rebuilding the app?Senior