Testing Without Documentation

Where to find the “truth”

Think of yourself as a detective arriving at a scene with no witness statement. There are still clues everywhere; you just have to know where to look. Spend your first 30 to 60 minutes collecting them before you test in depth. It pays back all week.

The product itself

The app is the most honest document you have: it shows exactly what was built. Use it to learn the vocabulary, the screens and the flows, and to find internal contradictions (two screens that disagree, one button that behaves differently in two places). Check every platform you have: the iOS app, the Android app and the website often disagree with each other, and each disagreement is a clue.

Outside the company

• Competitors and comparable products. How do other coffee, airline or supermarket loyalty programs handle refunds, expiry and redemption? Users bring those expectations with them.
• Platform conventions. Apple’s Human Interface Guidelines, Google’s Material Design and web standards like WCAGWCAGThe Web Content Accessibility Guidelines, the main international standard for accessibility.Open in glossary → describe what “normal” looks like: back navigation, touch targetsTouch targetThe area of the screen that responds to a tap, which should be large enough for a finger.Open in glossary →, dialogs, permissions. See Accessibility Basics for the most common checks.
• App store listing and marketing pages. These are public promises. If the listing says “earn 1 point per $1”, that’s practically a requirement.
• Reviews. One-star reviews often describe real bugs in the users’ own words (“my points disappeared after I canceled”).

Inside the company

• Support tickets. Ask the support team for the top complaints about the area you’re testing. They know where users struggle.
• Analytics. Which features are used most? Which screens do people abandon? Heavy use means high impact.
• Designs. Figma files show what the designer intended, often including empty statesEmpty stateThe screen a user sees when there is no content yet, like an empty cart or no search results.Open in glossary →, errors and loading screens that never made it into a ticket.
• API documentation. A Swagger/OpenAPIOpenAPIA standard file format that describes an API's endpoints, parameters and responses (formerly called Swagger).Open in glossary → page lists fields, required values, limits and error codes. A field called expiresAt tells you points expire, even if nobody mentioned it. API Testing with Postman shows how to read and call these endpoints.
• Code history. You don’t need to read code. Commit messages and pull request (PR) descriptions are written in plain language: “Award points when order is paid, not on pickup” is a requirement in disguise.
• Previous bug reports. Search the tracker for the feature name. Old bugs show weak spots, and closed-as-”works as designed” tickets record decisions.
• Release notesRelease notesA short list of what changed, what was fixed and the known issues in a build.Open in glossary → and old wiki pages, even outdated ones, show how the feature evolved.

People

Developers, the product owner (PO), designers, support agents and sales people each hold part of the picture. Sales and support know what customers were promised; developers know what was built; the PO decides what should happen.

Oracles: deciding what’s a bug

An oracle is anything that helps you decide whether a result is right or wrong. A spec is one oracle, and a very convenient one. But there are many others, and experienced testers use several at once.

The consistency heuristics (HICCUPPS)

Michael Bolton and James Bach, two well-known testing teachers, collected these oracles into a memorable heuristic called HICCUPPS. Each letter is a kind of consistency. A behavior that’s inconsistent with one of them is a candidate problem.

Consistent with…	Ask yourself	Brewly example
History	Did it work differently before?	Last version showed the expiry date; now it doesn’t
Image	Does it fit the brand the company wants?	A “premium” brand shows “ERR_NULL” in a popup
Comparable products	How do similar apps do it?	Other loyalty apps reverse points on refunds
Claims	Does it match what was promised?	Listing: “free drink of any size”, app: “small only”
User expectations	What would a reasonable user expect?	Canceling an order shouldn’t add points
Product	Is it consistent with itself?	Home shows 112 points, Rewards shows 120
Purpose	Does it serve what the feature is for?	Redeem is hidden two screens deep in Settings
Statutes and standards	Does it follow laws and guidelines?	Redeem link is 20 pt tall (iOS asks for 44×44 pt)

You don’t have to memorize the acronym. What matters is the habit: for every odd behavior, ask “inconsistent with what?” If you can answer, you have a reason. If you can’t, you probably have a question, not a bug.

Build a feature map while you explore

A feature map (or mind map) is a tree of everything the product does: areas, features, and the details inside them. Start with the app name in the middle, add the main areas (Rewards, Ordering, Payments, Account, Notifications) and keep adding branches as you discover things.

Why it’s worth the effort:

• It shows what exists, so you can see what you haven’t tested yet. That’s your coverageTest coverageHow much of the product, requirements or risks your tests actually check.Open in glossary → without a spec.
• It’s a perfect place to note risk (high, medium, low) and open questions next to each feature.
• It becomes documentation that the team didn’t have before.

Use a whiteboard, a mind map tool (XMind, Miro, FigJam) or a simple indented list. You’ll build one for Brewly in the practice section below.

Explore in focused sessions of 60 to 90 minutes, each with a short goal written down first, a test charterTest charterThe goal and scope of an exploratory testing session, written before it starts.Open in glossary →: “Explore earning and refunding points with card and gift card payments, to find out how the balance changes.” Exploratory Testing covers charters and sessions in depth.

Write down assumptions and get them confirmed

Without a spec, you’ll constantly make assumptions: “points are probably awarded when you pay”, “a refund probably removes them”. That’s fine, as long as you write them down and get them confirmed by someone who can decide (usually the product owner). Unwritten assumptions turn into silent gaps in your testing.

A simple assumption log:

#	Assumption	Based on	Status
A1	Points are awarded when the order is paid	PR description	Confirmed by dev
A2	$4.50 earns 4 points (round down)	Listing: “1 point per $1”	Asked PO
A3	A refund removes the earned points	User expectation	Confirmed by PO
A4	Tips don’t earn points	Guess	Open

Create lightweight documentation as you go

You’re not expected to write the missing 40-page spec. But leaving small, useful artifacts behind is one of the most valuable things a tester can do on an undocumented product:

• A checklistChecklistA simple list of things to check, lighter and faster to use than full test cases.Open in glossary → of what to check for each feature. Reusable for every release and for regression testingRegression testingRe-running tests to check that new changes didn't break things that used to work.Open in glossary →.
• The feature map, with risk and open questions.
• A glossary of the app. Every product invents words: “beans”, “tier”, “streak”, “pickup slot”. Writing them down (with the exact spelling the app uses) helps new teammates and reveals inconsistencies, like “Beans” on iOS and “Points” on Android.
• Confirmed decisions, copied from chats into the ticket or wiki.

A template you can copy into your team’s wiki:

Feature: Loyalty points (Brewly)        Last updated: 14 Oct, by Sam (QA)

What it does (in one sentence):
  Customers earn points on paid orders and swap 100 points for a free drink.

Sources:  App Store listing, Figma "Rewards v3", Swagger /loyalty, PR #812

Glossary: bean = 1 loyalty point · reward = free drink · tier = Silver/Gold

Confirmed rules:
  - Points awarded when the order is paid (dev, PR #812)
  - Refund removes the points earned by that order (PO, 12 Oct)

Open questions:
  - Do tips earn points?

Checklist: earn · balance on all screens · redeem · refund · expiry · offline

Prioritize by risk

You’ll never test everything, especially when you’re still discovering what “everything” is. So test the riskiest things first. Risk is roughly impact × likelihood (the same idea as the risk scores in Writing a Test Plan):

• Impact: what happens if this breaks? Money, personal data, legal problems and core flows (sign in, pay, order) are high impact.
• Likelihood: how likely is it to break? New code, complex rules (rounding, time zones), many integrations, and areas with past bugs are more likely to fail.

Risk	Typical examples	When to test
High	Earning and redeeming points, refunds, payments, balance shown to users	First, deeply
Medium	Expiry, tiers, store selection, notifications	Next
Low	Profile settings, activity history layout, promo banners	If time allows

Support tickets and analytics are excellent inputs here: a feature that gets many complaints or a lot of traffic deserves more attention. When you report what you found, you’ll also suggest severitySeverityHow much damage a bug does to the product or the user.Open in glossary →; see Severity vs Priority for how that differs from the team’s fixing priorityPriorityHow soon a bug should be fixed, based on business needs.Open in glossary →.

Reporting bugs when there’s no spec

Everything you learned in How to Write a Bug Report still applies. The one difference is the Expected result: you can’t write “as per requirement REQ-12”. Instead, name the oracle in brackets and, if possible, link or quote it.

A few more tips:

• Separate facts from judgment. The actual result is a fact. The expected result is your judgment, backed by an oracle. Readers can then disagree with the oracle without doubting your observation.
• Use “inconsistent with” language when two sources disagree: “The app allows small drinks only, which is inconsistent with the App Store listing (‘any size’). One of them needs to change.”
• Don’t file opinions as bugs. If no oracle supports it, raise it as a question or an improvement suggestion.
• Expect “works as designed”. Sometimes the team decides the odd behavior is intended. That’s a good outcome too: you now have a confirmed rule to add to your documentation.

The method, step by step

Worked example: Brewly’s undocumented Loyalty points

Back to Tuesday morning. Here’s how an experienced tester tackles the Friday release.

Tuesday, 9:00: collecting sources

Source	What it revealed
App Store listing	“Earn 1 bean for every $1. 100 beans = a free drink of any size.” (Beans are Brewly’s points.)
Figma “Rewards v3”	Rewards tab, a points card on Home, an expiry banner, Silver/Gold tiers
Swagger	GET /loyalty/balance, POST /orders/{id}/refund, an expiresAt field on each points entry
PR #812	“Award points when the order is paid (not at pickup)”
Support tickets	Two complaints from the beta: “points vanished after canceling”
Developer chat	iOS and Android teams built the screens separately

Already a lot: a public promise, intended screens, an expiry rule nobody mentioned, the moment points are awarded, a known weak spot (cancellations) and a hint that the two platforms may differ.

Tuesday, 10:30: first tour and feature map

The tester orders a flat white on iOS, checks the Rewards tab, refunds the order from the order history, then repeats everything on Android. The feature map grows: Rewards → Earning, Balance, Redeeming, Refunds, Expiry, History, plus Ordering, Payments and Account around it.

Tuesday, 11:30: assumptions and questions

The tester posts four questions in the team channel and the ticket:

1. “Should $4.50 earn 4 points (round down, as ‘1 per $1’ suggests) or 5?” → PO: round down.
2. “Should a refund remove the points the order earned?” → PO: yes.
3. “Do gift card payments earn points?” → PO: yes, any payment.
4. “Do tips earn points?” → PO: not decided. Becomes a new user story, out of scope for Friday.

Tuesday afternoon to Wednesday: exploring by risk

High-risk features first: earning, balance, refunds, redeeming. What the tester found:

#	Finding	Oracle	Severity
1	Refunding an order adds points instead of removing them	Confirmed by PO (question 2), user expectation	Critical
2	$4.50 earns 5 points	Confirmed by PO: round down	Major
3	Gift card payments earn 0 points	Claim (“every $1”), confirmed by PO	Major
4	Android calls them “Points”, iOS “Beans”	Consistency within the product	Minor

Finding 1 is critical because customers could order, refund and repeat to collect free drinks: real money for the business. Finding 4 surfaced only because the tester was writing the app’s glossary.

Wednesday: the bug report

Here’s how finding 3 looks as a report. Note how the expected result names its oracles.

Title:        Rewards: Paying with a gift card earns 0 points (card payments earn points)
Environment:  Brewly 3.2.0 (build 410), iPhone 15, iOS 18.0.1, Staging
Preconditions: Signed in as test user sam.qa; gift card balance $20; 40 points
Steps:
  1. Add a Flat white ($4.20) to the cart
  2. Tap Checkout and choose "Gift card"
  3. Tap Pay, then open the Rewards tab
Expected (App Store claim "Earn 1 bean for every $1", confirmed by PO in SHOP-318):
  Balance increases by 4 points, to 44.
Actual:
  Balance stays at 40. No activity entry is added. Same order paid by card earns 4 points.
Reproducibility: 3/3 attempts
Attachments:  gift-card-no-points.mp4

Thursday: leaving documentation behind

Confirmed rules from the PO turn into proper test cases (see How to Write Test Cases) for every future release:

The feature map, assumption log, glossary and these test cases go onto one wiki page. On Friday, the next tester to touch loyalty points starts with documentation that didn’t exist on Tuesday.

Common mistakes

Mistake	Why it hurts	Instead
Refusing to test until there’s a spec	The release goes out anyway, untested	Collect sources and start exploring
Filing opinions as bugs	Developers stop trusting your reports	Name an oracle, or ask a question
Keeping assumptions in your head	Gaps nobody knows about	Log them and get them confirmed
Treating the app as always right	You only confirm what was built	Compare it with claims, users, rivals
Treating one oracle as the truth	Outdated pages mislead you	Report inconsistencies, let the PO decide
Testing in random order	Critical bugs found on Friday evening	Map features, rate risk, start high
Leaving nothing behind	The next tester starts from zero	Feature map, checklist, glossary