A false positive classifies legitimate use as risky. Its impact extends beyond support-ticket volume: it can interrupt payments, access and account recovery.
Evaluation approach
Group events by device family, operating system and application version. Retain enough context to identify an affected cohort without collecting personal data unnecessarily.
Application example
If a new policy increases events only on one manufacturer's devices, investigate that cohort before applying a block to the wider user base.
Limits and considerations
A lower block rate alone does not establish improvement. Demonstrate that attack scenarios are still detected.
A rate needs the right denominator
A thousand events among a million operations has a different meaning from a thousand among a thousand. Record both affected legitimate operations and all eligible operations when assessing false positives. Repeated copies of one event must not be counted as separate users.
Distinguish cases confirmed through appeals from suspicious cases that remain unreviewed. Few support requests do not necessarily mean little impact; some users leave without contacting support. Measurements by release, device group and rule help locate the decision that went wrong.
Checks and decisions
- Measure by cohort
- Define an appeal route
- Make policies reversible
Sources
The primary references above provide the technical basis. Example workflows and evaluation suggestions are this publication’s explanations, not independent test results for a particular product.