Health Measures
Distinguishing a vocal minority from a wider customer need
Assess whether a community complaint reflects a wider customer need by checking distinct cases, missing groups and evidence beyond the thread.
A busy community thread shows some members care about an issue; it does not show how many customers share it. To judge the wider need, define the affected customer group, count distinct cases, check other channels and speak with people who did not volunteer for the thread.
Start with the decision question
Ask whether a particular customer group encounters the problem, how it affects their task and what happens when it does. Decide whether to estimate how common it is or to understand a serious failure mode. A rare but costly barrier may warrant action even if it is not widespread.
Write down who could have posted. Community access, language, timing and willingness to speak publicly shape who appears in a thread. People who experienced the problem may be absent, and so may those who completed the task without trouble. A poll of thread participants answers a narrower question than research designed around the wider customer group.
Separate voices from volume
Review the discussion by distinct person and circumstance. One member may post several clarifications; several may repeat the same account after reading it. Neither pattern is a count of independent cases. Preserve disagreement too: someone who uses the workflow successfully may reveal a condition that changes the diagnosis.
For each theme, record:
- the task and customer group involved;
- the number of distinct reports observed, without treating it as a customer prevalence rate;
- the conditions under which the problem occurs;
- related evidence from support, observed use or other research; and
- groups whose experience remains unknown.
These are investigation notes, not a score. Dividing comments by registered members does not produce a customer rate: members may be inactive, and customers outside the community had no equivalent opportunity to answer.
Key indicators of customer need beyond community threads
- Distinct reports observed
- Number of unique cases identified (not members)
- Customer groups with unknown experience
- E.g., inactive members, non-posters, new users
- Support contact mentions
- Related incidents found in ATO or service support data
Check beyond the thread
Search support contacts for the same task, including different words customers may use. Ask frontline staff what they have observed, then check examples rather than treating recollections as counts.
If the decision depends on reach, ask a defined customer group and disclose how participants were invited. Opt-in responses can explain experiences but need care before being generalised.
Recruit for missing perspectives, such as recent customers, people who stopped using the feature or customers who rarely post. Ask what they tried and what happened. Do not recruit only people who endorsed the thread's proposed fix.
Make uncertainty visible
A conclusion such as “confirmed difficulty for customers using this workflow; reach unknown” is more useful than saying either that everyone wants a change or that the reports came from “only a vocal minority”. State whether the decision rests on severity, breadth or both. If the evidence is thin, specify the next check and the decision it could change.
Tell the original contributors what their reports uncovered, even if wider research points to another solution. Their experience is real; a claim about all customers needs different evidence.
