Choosing Metrics That Match the Decision Being Automated

The metric you optimise for shapes what the model learns. If you optimise for accuracy, the model learns to match the training data labels. If the training data has class imbalance (for example, 95 percent non-default and 5 percent default loans), accuracy can be 95 percent by simply predicting non-default for every applicant.

That is useless because the model never approves anyone.

Which metric is right depends entirely on what decision is being automated and what kinds of error matter most. In fraud detection, missing a fraud (false negative) costs real money. In screening for a disease with low prevalence, a false positive sends someone for expensive follow-up testing.

These have different costs and you should optimise for the metric that matches those costs. You cannot use the same metric for every problem.

Accuracy is rarely the right metric. Precision (of all positive predictions, how many are actually positive) and recall (of all true positives, how many did you find) are more useful for imbalanced problems. For a model that approves loans, you might care about precision: of all loans you approved, what fraction actually repaid?

This tells you how reliable your approvals are. For a model that screens for disease, you might care about recall: of all truly ill patients, what fraction did you catch? This tells you whether the screening tool catches enough cases to be useful.

For problems where the decision depends on a threshold (approve if score above 0.7, decline if below), you must think carefully about the false positive rate and false negative rate and choose a threshold that balances them appropriately for your use case. A higher threshold approves fewer loans but makes more errors on borderline cases.

A lower threshold approves more loans but has higher default rate. There is always a trade-off.

Area under the ROC curve (AUC) tells you how well the model separates positive and negative cases across all possible thresholds. It is useful for comparing models but does not tell you whether a particular model should be deployed for your specific decision.

You must document your choice of metric. Why does this metric matter for the decision being made? What are the consequences of false positives and false negatives in your context?

Have you considered metrics that reveal performance on underrepresented subgroups, not just overall performance?

Practical Example: Fraud Detection in Banking

Consider a UK bank using a machine learning model to detect fraudulent transactions. The model must decide whether a transaction is likely to be fraudulent. Here, a false negative (missing a fraud) is extremely costly.

If a £5000 transaction is flagged as legitimate but turns out to be fraudulent, the bank bears the loss. A false positive (flagging a legitimate transaction as fraud) is less costly but still problematic. It causes customer inconvenience and may lead to chargebacks.

Therefore, the bank might optimise for recall to ensure that nearly all fraudulent transactions are caught. This means accepting a higher false positive rate to reduce the risk of missing frauds. A practical step here is to set a lower decision threshold to increase recall, even if it means more legitimate transactions are flagged for review.

Real-World Example: Medical Screening

In medical diagnostics, such as screening for rare diseases like lung cancer in smokers, the decision process is critical. A model that predicts whether a patient has lung cancer must be evaluated using recall. If the model misses a case (false negative), it can delay treatment and potentially lead to a worse outcome.

A false positive (incorrectly diagnosing a healthy person) may lead to unnecessary anxiety and further tests. Therefore, a healthcare provider may choose to optimise for recall to ensure that all true cases are identified. A practical step is to adjust the model’s decision threshold to prioritise identifying all positive cases, even at the cost of increased false positives.

This is especially important in low-prevalence scenarios where missing even one case can have severe consequences.

Loan Approval: Balancing Precision and Recall

When automating loan approvals, a lending institution must balance precision and recall. Precision is important here because approving a loan that defaults is costly. If a model has high precision, it means that most of the loans it approves are likely to be repaid.

However, if the model is too precise, it may reject many borderline cases, reducing the number of approved loans. Recall is also important; a model that misses many default cases (low recall) will have a higher default rate. A practical step is to use a confusion matrix to evaluate performance and adjust thresholds.

For example, a bank may decide to accept a slightly lower precision to gain higher recall, especially if the cost of default is significantly higher than the cost of missed approvals. This decision must be based on the specific financial and regulatory environment.

Threshold Adjustment: A Step-by-Step Approach

When deploying a model, you must consider how to set the decision threshold. Start by evaluating performance at multiple thresholds. For example, a model may have a recall of 0.9 at a threshold of 0.6, but a precision of 0.7.

If the cost of a false negative is higher than a false positive, you may choose to lower the threshold to increase recall. A step-by-step process includes: 1) identifying the cost of each type of error, 2) running performance metrics at multiple thresholds, 3) selecting a threshold that aligns with business objectives, and 4) validating the chosen threshold on a hold-out dataset.

This process ensures that the model is not only accurate but also aligned with the decision-making process.

Model Evaluation Beyond AUC

While AUC is a valuable metric for comparing models, it is not sufficient for decision-making. A model with a high AUC may still perform poorly in a specific use case. For example, a model with 0.9 AUC may have a very low recall at a critical threshold.

A practical example is a model used to predict customer churn. Although it may have a high AUC, if it fails to predict churn for a specific customer segment, it is not useful. Therefore, it is essential to evaluate performance at the specific thresholds used in production.

This includes looking at precision, recall, and F1-score at those thresholds. A step to take is to create performance reports that show metrics at multiple thresholds and highlight the trade-offs involved.

Equity and Fairness in Model Metrics

When deploying models in real-world environments, it is essential to consider how performance varies across different subgroups. For example, a model used for hiring may have different performance rates for male and female candidates. A model that performs well overall but poorly for underrepresented groups can lead to unfair outcomes.

A practical step is to evaluate metrics such as precision and recall for each subgroup. This ensures that the model is not only accurate but also fair. For instance, a UK employment agency using a model to screen CVs must ensure that the model does not disproportionately reject candidates from certain demographics.

This involves monitoring performance metrics across gender, age, and ethnic groups and adjusting the model or thresholds accordingly.

Ultimately, choosing the right metric is not just about numbers; it is about aligning the model with the real-world decision process. You must understand the consequences of each type of error and choose metrics that reflect those consequences. Documenting this decision is critical for transparency and future audits.

This ensures that the model is not only performing well but also serving its intended purpose in a responsible and ethical manner.