Biometric Categorisation That Infers Protected Characteristics

The seventh prohibited practice covers AI systems that use biometric data to infer protected characteristics and then use those inferences to make decisions affecting a person’s rights or access to services. Protected characteristics under anti-discrimination law include race, ethnicity, sex or gender, disability, sexual orientation, gender identity, age, religion or belief, and in some jurisdictions, political affiliation or social status.

The core concern is that biometric data, when analysed by machine learning models, can be used to infer these sensitive characteristics even if the raw biometric data does not obviously reveal them. A person’s face contains information that AI models can extract beyond what they intentionally show. Facial analysis AI can estimate a person’s race or ethnicity from their facial features. Gait analysis algorithms can infer aspects of physical disability or mobility status from the way a person walks. Voice analysis can infer perceived gender or other characteristics. Heart rate patterns might indirectly infer stress or anxiety related to a protected characteristic.

Using such inferences to make decisions that affect a person’s life is prohibited. An employer cannot use facial biometric analysis to infer race or ethnicity and then use that inference in hiring decisions. A financial services provider cannot use biometric data to infer disability status and then use that inference to modify terms of credit, set insurance premiums or restrict access to financial products. A government agency cannot use biometric analysis to infer sexual orientation or gender identity.

The prohibition is specifically about using biometric inferences of protected characteristics in a way that causes disadvantage. The act of inferring the characteristic itself is not automatically prohibited if the inference stays in the system and never reaches any decision-making process. However, using the inferred characteristic to restrict access, deny services or make unfavourable decisions is prohibited.

There is a narrow exception for law enforcement and border control using biometric categorisation for specific security or criminal investigation purposes. Border security officials can use biometric analysis to verify identity as part of travel document control. Police can use biometric identification in criminal investigations targeting specific individuals if they have legal authority and appropriate judicial oversight.

The challenge for organisations is this: if you use any biometric data, you must have procedures to prevent misuse for protected characteristic inference. You should conduct impact assessments on any biometric system to identify the risk that inferences about protected characteristics could be drawn and subsequently used in decisions. You should build in technical controls to prevent those inferences being used in decision-making systems. You should regularly audit that protected characteristic inferences are not leaking into automated decision systems.

For many organisations, the safest and most practical path is to avoid collecting biometric data unless you have a specific, lawful, transparent purpose that does not rely on characteristic inference or that explicitly excludes protected characteristics.

Examples of prohibited biometric characteristic inference include: an insurance company using facial analysis to infer age and genetic predisposition to disease, then using that in underwriting decisions; a retailer using gait analysis to infer disability status, then using that information to target marketing or adjust pricing; a hiring company using voice analysis to infer gender or age, then using that in candidate screening; a financial services company using video interview analysis to infer race or ethnicity, then using that in credit or lending decisions; an employment agency using facial recognition to infer sexual orientation from facial characteristics, then using that to screen candidates; a credit card company using video call analysis to infer disability from movement patterns or speech patterns, then adjusting credit limits.

In contrast, a company can use biometric data for identity verification at authentication points (like fingerprint to unlock a phone), or for explicitly demographic-based advertising if done transparently and consensually (like running ads targeted at a specific age group that opted in), as long as protected characteristics are never inferred from biometric data or used in decisions that affect access to services.

The principle is: you can collect biometric data if you have a specific purpose and can prove informed consent. You cannot use that data to infer protected characteristics. If you need protected characteristic information, ask the person directly rather than inferring it.