Working With Legacy Data and Historic Bias in Records

Many government algorithms are trained on historical data. Benefit records from the past ten years. Housing allocation data from past decisions. Enforcement records showing which premises have been inspected. This historical data is often biased. Past decisions may have been discriminatory. Past records may reflect systemic unfairness that the organisation is now trying to overcome.

When an algorithm is trained on biased historical data, it learns to reproduce the bias. If historical housing data shows that caseworkers allocated fewer properties to disabled people, the algorithm trained on this data will be less likely to allocate properties to disabled people. If historical employment records show that women were less often promoted, an algorithm trained on this data will be less likely to recommend women for promotion. The algorithm does not know it is reproducing discrimination; it is merely learning patterns.

The first step is recognising that historical data may be biased. This requires examining the data and its origins. Who collected it? What was the decision-making process? Have there been complaints of discrimination? Have audit or investigation reports identified problems? If the data comes from a period when the organisation was less attentive to equality, the data likely contains bias.

The second step is assessing what bias the data contains. If the organisation was historically less likely to allocate housing to disabled people, the data will show this pattern. Testing the algorithm on historical data will reveal bias. Testing should include comparison of outcomes for different protected groups. If the algorithm is significantly more likely to deny housing to disabled people than to non-disabled people, this bias has been detected.

The third step is deciding how to address the bias. One option is not to use certain variables in the algorithm. If postcode is a proxy for protected characteristics or if postcode carries historical bias, remove it. If employment history carries gender bias, reconsider whether it is necessary. Removing variables is crude but can work.

A second option is to reweigh or rebalance the data. If historical data has systematic bias in one direction, the algorithm can be trained to partially correct for this. This is technically complex and requires expertise.

A third option is to use data from a more recent and less biased period. If the organisation has changed its practices and recent decisions are fairer, training on recent data only might produce a less biased algorithm. This requires having enough recent data.

A fourth option is to accept that historical data carries bias and to put additional safeguards in place. Before deployment, test the algorithm extensively for bias across protected groups. After deployment, monitor outcomes continuously and adjust the algorithm if bias emerges.

Some organisations take a principled stance: they do not train algorithms on data from a period when historical discrimination was systemic. They start from scratch with current practices. This is labour-intensive but sends a clear message about commitment to fairness.

Working with legacy data also means being honest about its limitations. If historical data is incomplete or was recorded in a way that reflected bias, this should be disclosed. The algorithmic transparency record should note that the system was trained on historical data that may contain bias and should describe what testing has been done and what safeguards are in place.