Usability testing: watching people try to use your app
Usability testing answers a question that QA cannot: can a person who has never seen this app work out how to use it? Nothing is necessarily broken. The button works, the form saves, the data round-trips. And testers still cannot find the feature, still hesitate on the screen you thought was obvious, and still abandon halfway through.
We run task-based sessions with first-time testers on their own Android devices and report what happened, task by task, with the screens where it went wrong. The findings are written for a designer or product owner, not for a bug tracker.
What do we measure?
Usability findings are only useful when they point at something specific. Each of these four produces a number or a moment you can take to a design discussion.
Task completion
Can a first-time user finish the thing they came to do? Each session runs against a written task list, and the outcome is recorded per task: completed unaided, completed with a struggle, or abandoned.
A task completed by 4 of 10 testers is not a design detail. It is the flow you will lose most new users in.
Effort to complete
How much wandering happened on the way. Not a stopwatch to the millisecond, but whether a task took two steps or eleven, and whether the tester had to go back and read something twice.
Effort is what separates a task that works from a task people will repeat. Users tolerate a slow path once and avoid it afterwards.
Hesitation points
The screens where testers pause, tap the wrong control, or say they are not sure what happens next. These are the highest-value findings in the whole report, because nothing is technically broken.
Hesitation does not appear in crash logs, analytics funnels, or your automated tests. It only appears when someone is watching a person use the app.
Abandonment triggers
The specific moment a tester stopped. Sometimes it is a permission prompt with no explanation, sometimes a mandatory field the app never asked about, sometimes a wait with no feedback.
Recording the exact screen where people leave tells you which screen to redesign rather than which flow to extend.
How does a session run?
The method is uncomplicated. What makes it work is leaving the tester alone with a task list instead of walking them through the app, which is the mistake most internal usability testing makes.
01
Write the task list from your goals
You tell us what the app is for and what success looks like. We turn that into five to eight tasks phrased the way a user would think about them, not the way the app is structured. "Find and buy the cheapest option for next Tuesday" beats "test the search module".
02
Test with people who have never seen the app
Testers are allocated from the pool without prior exposure to your build. Nobody is briefed on the intended flow, because a tester who knows the design cannot tell you where it fails.
03
Run the tasks and note what happens
Testers work through the list on their own physical devices and record the path they took, where they paused, what they tried first, and what they expected to happen at each step.
04
Capture the evidence
Each finding arrives with a screenshot of the screen in question and the steps that led there, so a designer can see the state rather than read a description of it.
05
Group findings into themes
Ten individual complaints are hard to prioritise. Three themes with the sessions behind each one are straightforward: a navigation problem, a labelling problem, or a missing-feedback problem. The grouping is the deliverable.
The heuristics behind the review
Alongside the task results, testers run the app against the established usability heuristics published by Nielsen Norman Group. It gives the review a consistent structure, so a finding about your app is described the same way as a finding about any other.
| Heuristic | What testers check | A typical finding |
|---|---|---|
| Visibility of system status | Spinners, progress, confirmation messages, and whether the app ever leaves the user wondering if anything happened. | The report shows an upload with no progress indicator and no completion state. |
| Match with the real world | Whether labels use the words your users use, or internal terms that only make sense inside your team. | A settings screen labelled with a backend concept no tester recognised. |
| User control and freedom | Undo, back, cancel, and whether a mistaken tap can be reversed without restarting a flow. | A multi-step form where Back discarded every field rather than the last step. |
| Consistency and standards | Whether similar things behave similarly, and whether platform conventions are followed. | A pull-to-refresh gesture that only worked on two of the five screens that needed one. |
| Error prevention and recovery | Whether destructive actions are confirmed, and whether error messages say what to do next. | A delete action with no confirmation and an error state reading only "Something went wrong". |
| Recognition over recall | Whether users have to remember information from an earlier screen to complete a later one. | A verification code entry screen that did not repeat the email address the code was sent to. |
Findings are usually a mixture: a navigation problem shows up as both a task failure and a consistency violation, and the report cross-references the two rather than listing them separately.
Is usability testing the same as QA testing?
The two get bundled under "testing" and they produce completely different work. Booking usability sessions when you need defect hunting, or the reverse, wastes a cycle.
| Aspect | Usability testing | QA testing |
|---|---|---|
| Question answered | Can a person use this successfully? | Does this feature work as specified? |
| Typical finding | Testers did not find the export option; eight of ten looked in Settings first. | Export produces a corrupt file when the filename contains a slash. |
| Severity basis | How much of the audience is blocked or slowed, and how badly | Impact on function and data integrity |
| Fixed by | Restructuring, renaming, or reordering screens — design work | A code change |
| Useful when | Before a redesign, after an unexplained drop in activation, or before a launch | Every release |
Worth running when
- Activation drops and the funnel numbers do not explain why.
- You are planning a redesign and want to know what to keep.
- A feature exists and nobody uses it — before you build more of it.
- You are close to launch and want a first-impression read.
Skip it when
- The build crashes on launch. Fix that first; testers cannot evaluate a broken app.
- You need statistically representative data for a business case.
- You already know the problem and need implementation work, not more evidence.
- Your only goal is a Google Play policy window. That is a different deliverable.
Usability work pairs naturally with functional testing on the same build. Both can run inside a single 14-day cycle, which is usually cheaper than booking two. The Android testing scope shows what a combined cycle covers.
Usability testing questions
Method, sample size, and what a report actually contains.
How is usability testing different from QA testing?
Usability testing asks whether a person can use the app successfully; QA asks whether each feature works as specified. A usability finding might be that eight of ten first-time testers looked for the export option in Settings first. A QA finding might be that export produces a corrupt file when the filename contains a slash. The first is fixed by design work, the second by a code change.
How many testers do you need for usability testing?
Ten is enough to surface the problems most of your users will hit. It is not a statistically representative sample, and we will not present it as one. If your decision needs quantitative confidence, you want a research panel rather than a tester pool.
What does a usability report contain?
Task-by-task outcomes with completion status, the paths testers actually took, hesitation points with screenshots, abandonment triggers, and findings grouped into themes. Heuristic findings are cross-referenced to the task failures they relate to rather than listed separately.
Do testers know how the app is supposed to work?
No. Testers are allocated without prior exposure to your build and are not briefed on the intended flow. A tester who knows the design cannot tell you where it fails, which is the mistake most internal usability testing makes.
What accessibility checks are included?
The usability pass covers practical barriers — unlabelled controls, touch targets, contrast, and layouts that break at increased system font scale. It is a baseline review, not a conformance audit, and we do not certify accessibility against any legal standard.
When is usability testing not worth doing?
When the build crashes on launch, because testers cannot evaluate a broken app. When you need statistically representative data for a business case. When you already know the problem and need implementation work rather than more evidence. And when your only goal is a Google Play policy window, which is a different deliverable.
Written and maintained by
Afrin Asha
Founder, OnTesters
Android developer and QA specialist. Built OnTesters after working through Google Play’s 12-tester closed testing requirement on real devices.
Platform figures on this page come from campaigns run through OnTesters. Read how the platform works or see the guides library.
Sources
- 10 Usability Heuristics for User Interface Design— the heuristic set this review is built on
- Set up an open, closed, or internal test— the Play tracks a usability cycle can run inside
- App testing requirements for new personal developer accounts— the policy window, when both needs overlap
Finding examples on this page are illustrative composites drawn from usability sessions run on our platform, with app details removed.
Policy and pricing reviewed September 2026
See your app through a stranger's eyes
Send us the app and what it is for. We write the task list, run the sessions with first-time testers, and hand back findings grouped into themes.