Scoring Likelihood and Severity Without False Precision
Understanding the Problem with False Precision
Many practitioners make the mistake of creating risk scores with excessive decimal places or overly specific numerical values. This false precision creates a false sense of accuracy that can mislead decision makers. When assessing AI risks, scores should reflect genuine uncertainty rather than artificial precision. The goal is to communicate risk levels clearly without implying more certainty than exists.
Consider a scenario where a data protection officer evaluates an AI system’s privacy risk. They might calculate a score of 4.73 out of 10, but this level of precision suggests certainty that rarely exists in risk assessment. The difference between 4.7 and 4.8 represents a minimal change that may not meaningfully impact risk treatment decisions. Practitioners should instead focus on meaningful categories that guide practical actions.
- Unnecessary decimal places create confusion about actual risk levels
- Overly specific scores can mislead stakeholders about risk certainty
- False precision makes risk scores appear more authoritative than they are
Applying Meaningful Scoring Ranges
Effective risk scoring uses ranges that reflect practical decision-making needs rather than mathematical precision. The scoring system should group similar risk levels together so that scores within the same range trigger identical treatment approaches. This approach makes risk assessment easier to apply consistently across different projects or systems.
A practical example involves assessing AI bias risk in recruitment systems. Rather than calculating scores to two decimal places, practitioners might use categories such as low (0-2), moderate (3-5), high (6-7), and very high (8-10). These ranges provide enough distinction to guide treatment decisions while avoiding false precision. The difference between scores of 4.7 and 4.9 rarely matters for practical risk management purposes.
ISO 31000 clause 4.10 recommends considering risk levels as categories rather than precise measurements. This approach aligns with practical risk management where decisions often depend on broad risk classifications rather than exact numerical values. The focus should be on whether risk falls into acceptable, tolerable, or unacceptable categories rather than precise numerical distinctions.
- Use broad categories that guide practical treatment decisions
- Ensure similar scores trigger identical risk responses
- Apply ranges that reflect real-world risk management needs
Documenting Risk Assessment Without False Precision
Documentation of risk assessments should clearly explain the reasoning behind scores without implying false accuracy. Practitioners must record their assessment process and evidence rather than simply presenting final scores with excessive precision. The documentation should focus on the factors that influenced risk levels rather than the precise numerical outcomes.
When documenting AI risk assessments, practitioners should describe the evidence that led to their risk categorization. For example, instead of stating “risk score of 6.3”, they should explain that the system showed moderate bias in demographic data, required high data protection controls, and had limited audit trails. This approach provides meaningful context for decision makers while avoiding false precision.
Effective documentation includes noting the sources of uncertainty in risk assessment. Practitioners should identify which factors contributed most to risk levels and acknowledge areas where data or evidence is limited. This transparency helps stakeholders understand the basis for risk classifications and makes the assessment process more defensible during reviews.
- Document underlying evidence and reasoning rather than final scores
- Explain factors that contributed to risk classifications
- Identify sources of uncertainty in risk assessment
The key principle is that risk scores should support practical decision-making rather than demonstrate mathematical precision. Practitioners who focus on meaningful categories and clear documentation create risk assessments that serve their intended purpose. This approach makes risk management more practical and easier to maintain through organizational reviews. The goal remains to communicate risk levels clearly while avoiding the false impression of accuracy that excessive precision creates.