Beta testing that produces findings, not just installs
Beta testing is the practice of putting a near-final Android build in front of people who did not write it, and learning what happens. The output should be a stability picture, a set of grouped feedback themes, a usage pattern across the period, and a written record of what changed. If a beta produces none of those, it was a distribution exercise.
This page is for teams who want a beta run properly. If your account has an active Google Play closed testing obligation, the testing we run here can satisfy both purposes — the 14-day cycle page covers that side.
What should a beta hand you?
Judge any beta — ours or your own — by whether these four things exist at the end of it.
A stability picture
What breaks, how often, and on which hardware. Two weeks of use across a real device spread tells you whether a crash is universal or confined to one manufacturer — a distinction you cannot make from a single phone.
Instead of: one crash log from your own device and a guess about how widespread it is.
Feedback you can group
Individual complaints are hard to act on. Themes are easy: five testers independently confused by the same settings screen is a design problem, and one tester who dislikes your colour scheme is not.
Instead of: a folder of unrelated messages with no sense of which ones matter.
A usage pattern over time
Who came back on day two, and who came back on day ten. Beta retention is the closest cheap proxy you have for how real users will behave after launch.
Instead of: install counts, which tell you nothing about whether anyone opened the app twice.
Something to write down
Release notes, a changelog, a list of fixes with versions. If you ever need to explain what your testing period produced — to Google, to a stakeholder, or to yourself in three months — this is the record.
Instead of: reconstructing the last two weeks from memory.
Managed beta versus running it yourself
Neither column is wrong. A small app with an engaged audience can absolutely run its own beta, and we would say so even though we sell the other column.
| Aspect | Managed cycle | Self-recruited beta |
|---|---|---|
| Group size | 12 to 25 testers, sized to what you need to learn | Whatever you can find, usually 5 to 12 people who already know you |
| Device spread | Samsung, Pixel, Xiaomi, OnePlus, Motorola across Android 11 to 15 | Whoever owns what — typically a narrow band of newer Android phones |
| Session behaviour | Daily activity tracked for the whole cycle, with replacements for dropouts | Strong in week one, thinning by week two |
| Feedback format | Structured findings with steps, device, and severity | Freeform messages in a group chat |
| Cost | From $14.99 per app for a full cycle | Nothing, plus the time spent recruiting and chasing people |
| Failure mode | You pay for structure you may not need on a tiny app | The group goes quiet and you learn nothing from the two weeks |
If you are running it yourself, do these five things
Most failed DIY betas fail for the same five reasons. None of them require money to fix, which is why this list is free.
Write down what you want to learn before you recruit
Three specific questions beat "get feedback". Something like: can a new user finish onboarding without help, does the export feature work on non-Samsung phones, and where do people give up.
Recruit for behaviour, not enthusiasm
People who volunteer in a comment thread are volunteering for the idea, not the work. Ask directly whether they will open the app on ten separate days, and treat a hesitant answer as a no.
Give testers a task list
A short list of flows to attempt, with permission to ignore it when something else breaks. Unstructured testers open the app, look at the home screen, and close it.
Check activity mid-cycle, not at the end
Day seven is the decision point. If a third of the group has gone quiet, you have a week left to fix it, and no way to do that if you only look at the data on day fourteen.
Act on something visible
Ship a fix that a tester asked for and tell them. Testers who see their report turn into a change are the ones still engaged on day eleven.
Where DIY recruitment tends to break down is the fourth point. There is a long-standing pattern in the communities where developers swap testers: everyone opts in enthusiastically, and by day nine a meaningful share have stopped opening anything. Our write-up on Android beta tester communities goes through what each option actually delivers.
Beta testing, or closed testing?
The two terms get used interchangeably and they describe different things. Beta testing is a practice — a way of evaluating a build. Closed testing is a Google Play release track, and on personal developer accounts created from 13 November 2023 it carries a hard requirement of 12 opted-in testers for 14 consecutive days.
You can run a beta on the closed track, on the open track, or through a direct APK distribution to people who never touch Google Play. Only the closed track satisfies the policy. If your account is exempt — an organization account, or one created before the policy date — you can run a beta purely for the engineering value and ignore the compliance side entirely.
For teams that want both, we run one cycle that covers the requirement and the testing at once. The Android app testing scope is what runs inside it.
When teams book a cycle
- Before a public launch, to find the crashes that only appear when strangers use the app.
- After a rewrite, when the device coverage you relied on no longer matches the code.
- When a policy window and a testing need land at the same time, and doing two cycles is pointless.
Pricing shape
One-time fee per app, from $14.99 for 12 testers, with higher tiers for 15, 20 and 25 when you want a wider device spread. No subscription, and no charge for replacing a tester who drops out.
See the plansBeta testing questions
Scope, deliverables, and the difference between a beta and a policy requirement.
What is the difference between beta testing and closed testing?
Beta testing is a practice — a way of evaluating a near-final build with people outside your team. Closed testing is a specific Google Play release track. You can run a beta on the closed track, on the open track, or through direct APK distribution. Only the closed testing track satisfies Google's requirement for personal developer accounts created on or after 13 November 2023.
Do you test iOS apps or TestFlight builds?
No. We are an Android-only service. There is no iOS, iPadOS, or TestFlight coverage, and no cross-platform test management. If you need both platforms handled in one cycle, we are not the right service.
How many testers do you put on a beta cycle?
Between 12 and 25 testers depending on what you want to learn. Twelve is the policy minimum if your account carries a closed testing obligation; the larger tiers are for broader device coverage, more feedback volume, or apps with a narrow feature set where more sessions are needed to surface issues.
What feedback format do testers use?
Structured findings rather than freeform messages: a summary, steps to reproduce where relevant, the device and Android version, and a severity. Grouped themes are produced at the end of the cycle so the output is a set of prioritised problems rather than a folder of unrelated comments.
Can I run a beta without a service?
Yes, and for a small app with an engaged audience it is often the right call. The five things that make DIY betas work are writing down what you want to learn before recruiting, recruiting for behaviour rather than enthusiasm, giving testers a task list, checking activity at the mid-point, and shipping a fix that testers can see.
What happens if testers go quiet during a beta?
We track activity daily and replace testers who stop engaging, at no extra cost. Dropout is the normal pattern for a two-week commitment rather than a failure of any individual tester, which is why cycles are sized above the minimum rather than at it.
Written and maintained by
Afrin Asha
Founder, OnTesters
Android developer and QA specialist. Built OnTesters after working through Google Play’s 12-tester closed testing requirement on real devices.
Platform figures on this page come from campaigns run through OnTesters. Read how the platform works or see the guides library.
Sources
- Set up an open, closed, or internal test— the tracks that host a beta on Google Play
- App testing requirements for new personal developer accounts— when a beta also carries a policy obligation
- r/AndroidClosedTesting— where most DIY Android beta recruitment happens, and where it usually goes wrong
Engagement and dropout patterns referenced here come from 1,500+ campaigns analyzed on our platform. Google's own documentation covers the release tracks, not beta methodology.
Policy and pricing reviewed September 2026
Run a beta worth reading
Tell us the flows you care about and we will put 12 to 25 testers on them, with findings arriving while the cycle is still running.