Usability Testing Basics

Nielsen’s 10 usability heuristics, with examples

Jakob Nielsen developed his heuristics with Rolf Molich in 1990 and refined them into the list of 10 in 1994. Nielsen Norman Group still publishes them, with lightly updated wording, and they remain the most widely used checklist for interface reviews. A heuristicHeuristicA rule of thumb that helps you find problems or make decisions quickly, but can be wrong.Open in glossary → is a rule of thumb: breaking one doesn’t automatically mean a bug, but it’s a strong hint.

#	Heuristic	A typical violation in an app
1	Visibility of system status	Tapping Pay shows nothing for 4 seconds
2	Match between the system and the real world	A toggle labeled “Enable telemetry payload”
3	User control and freedom	A full-screen promo with no close button
4	Consistency and standards	“Save” on one screen, “Apply” and “Update” on others for the same action
5	Error prevention	“Delete all workouts” runs with one tap
6	Recognition rather than recall	“Enter your plan code from the welcome email” instead of a list of plans
7	Flexibility and efficiency of use	No way to reorder a favorite meal; frequent users rebuild it every time
8	Aesthetic and minimalist design	Checkout buries the total under six banners
9	Help users recognize, diagnose, and recover from errors	“Error 422” instead of “Weight must be more than 0”
10	Help and documentation	Help center only reachable from the website footer

Two of them deserve extra attention from testers, because they overlap with things you already check:

• Heuristic 1 is behind every loading state, progress bar and toast you test.
• Heuristic 9 is about error messages: what went wrong, in plain words, and how to fix it.

Heuristic evaluation

In a heuristic evaluationHeuristic evaluationAn expert review where evaluators check an interface against usability heuristics.Open in glossary →, evaluators walk through the product and note every place that breaks a heuristic. Nielsen recommends three to five evaluators working separately first, because each person spots different problems, then merging their lists and rating severity together.

It’s cheap and fast, and testers do a light version of it every day without calling it that. Its weakness: it predicts problems. Real users may stumble on things no evaluator expected, and shrug at things evaluators worried about.

Usability testing with real users

Moderated and unmoderated

	Moderated	Unmoderated
Setup	A facilitator runs the session, in person or on a video call	Participants do tasks alone, guided by a tool that records the screen and voice
Strength	You can ask follow-up questions and see hesitation	Cheaper, faster, more participants, natural environment
Weakness	Takes time; the facilitator can influence people	No follow-up questions; unclear tasks can’t be fixed mid-session

Think-aloud

The think-aloud protocolThink-aloud protocolA usability test method where participants say what they are thinking while they try tasks.Open in glossary → asks participants to say what they’re thinking: “I’m looking for the address… I’d expect it under Profile… no, maybe Settings?” That running commentary explains why people struggle, not just where.

Good facilitation is mostly about not helping:

• Give realistic tasks, not instructions: “You’re moving next week. Make sure your next order goes to the new address.” Not: “Tap Profile, then Addresses.”
• Stay neutral: “What are you thinking?” rather than “Did you see the button at the top?”
• Watch what people do, not only what they say. People often succeed while saying it was hard, or fail while saying it was fine.

How many users? The five-user guideline

Jakob Nielsen’s well-known guideline (from his 2000 article “Why You Only Need to Test with 5 Users”, based on earlier research with Tom Landauer) says that about 5 participants usually reveal most of the usability problems in a small qualitative test. His model estimated around 85%, under the assumption that a typical problem affects roughly a third of users.

Read it as a guideline, not a law:

• It’s meant for iterative testing: test with 5, fix what you found, test again with 5. Several small rounds beat one big one.
• It applies per distinct user group. Testing both shoppers and restaurant owners means about 5 of each.
• It’s for qualitative tests (finding problems). Measuring things like success rates or timings precisely needs many more participants.
• Rare problems that affect few users can easily be missed by 5 people.

The System Usability Scale (SUS)

The System Usability ScaleSystem Usability Scale (SUS)A 10-question questionnaire that gives a usability score from 0 to 100.Open in glossary → is a short questionnaire created by John Brooke in 1986. Participants rate 10 statements (like “I found the system unnecessarily complex”) from “strongly disagree” to “strongly agree”. The answers convert to a score from 0 to 100. It’s not a percentage: a score of around 68 is often quoted as average. SUS is useful for comparing versions or products over time, not for finding specific problems.

Testing usability without real users

Most testers can’t run formal studies every sprint. You can still find a lot:

• Heuristic walkthroughs of new screens, like the one in the practice below.
• Platform guidelines: Apple’s Human Interface Guidelines and Google’s Material Design describe expected patterns, like where Back lives and how destructive actions are confirmed. Breaking them violates heuristic 4.
• Hallway tests: ask a colleague who has never seen the feature to try one task while you watch quietly. Five minutes, often eye-opening.
• Real-world signals: support tickets, app store reviews and analytics (where do people drop out of a flow?).
• Fresh eyes: your own first impression of a feature is valuable data. Write it down before you get used to it.

Reporting usability problems so they get fixed

This is where many testers struggle. “This is confusing” sounds like an opinion, and opinions are easy to dismiss. A good usability report is built from evidence and impact.

What to include

• What you observed, as facts: the screen, the steps, what happened. Screenshots or a short screen recording.
• Who and how many: “3 of 5 participants”, “new users”, “on small phones”. Quotes from think-aloud are powerful.
• User impact: time lost, errors made, tasks abandoned, money or data at risk.
• The heuristic or guideline it breaks, as a shared reference, not as a weapon.
• A severity rating.
• A suggestion is optional. Describe the problem first; the designer owns the solution.

If the screen matches the spec but still causes problems, say so clearly (“works as specified, but users can’t find it”): that’s a validation finding, and it’s just as valuable.

Severity for usability problems

Nielsen suggests rating usability problems on a 0 to 4 scale, judged by three factors: frequency (how many users hit it), impact (how hard it is to overcome) and persistence (once, or every time).

Rating	Meaning	Example
0	Not a usability problem	A reviewer’s personal taste
1	Cosmetic, fix if there’s time	Slightly uneven spacing in a list
2	Minor, low priority	Jargon in a rarely used setting
3	Major, important to fix	Save gives no feedback, users repeat it
4	Catastrophe, fix before release	One tap deletes the account with no confirmation

Your team’s bug tracker probably uses its own severity levels (critical, major, minor…). Map honestly: a usability catastrophe can be just as severe as a crash. See severity vs priority and writing bug reports.