Auditing Data: Sourcing, Quality, Consent and Retention

Data quality failures are among the most common causes of AI system failures and regulatory breaches. A model built on poor quality data will make poor decisions, no matter how sophisticated the algorithm. This lesson walks through the audit procedures for data: where it came from, how clean it is, whether consent was obtained, and whether it is being retained appropriately. These audits uncover a surprising number of issues in most organisations.

Data Sourcing and Legal Basis

Start by understanding where the data came from. Ask the model owner or data team to produce a data sourcing document that lists each source of data, when the organisation acquired it, what legal basis exists for using it (contract, consent, legitimate interest), and whether it was used for the original purpose or a new one. Many organisations reuse data in ways that were never contemplated when it was originally collected. A customer contact list collected for direct marketing might be reused for training a fraud detection model. That shift in purpose could breach data protection obligations if the original consent did not cover the new use.

Audit steps include checking that data sharing agreements exist and are signed, that data governance approved the new use if data sourcing has changed, and that if personal data is involved, a data protection impact assessment (DPIA) was completed. The UK GDPR requires DPIAs for automated decision-making involving personal data. Many organisations have never completed one, or completed them as tick-box exercises without real substance.

Data Quality and Completeness

Data quality audit involves two parallel tracks: documentation review and sampling. First, request data quality documentation. Has the team documented known data quality issues, such as missing values, outliers, or data entry errors? Have they documented how they handled these issues? If the answer is no or vague, that is a problem. Decisions made by AI should rest on a clear foundation of documented data decisions.

Second, sample the actual data. You do not need to be a data scientist to audit data quality. Ask to see a random sample of one hundred records. Check whether required fields are populated, whether dates make sense, whether numeric ranges are reasonable. If you sample randomly and find obvious errors, then you have discovered gaps in data governance. A simple random sample often surfaces issues faster than asking how quality is monitored.

Consent and Opt-Out Rights

If the data includes personal information or behavioural data about individuals, check that consent was obtained for the use of that data in AI systems. GDPR and many other privacy regimes require consent that is freely given, specific, informed, and unambiguous. Consent obtained for one purpose does not automatically cover use in AI. A customer who consented to marketing emails might reasonably refuse to have their data used to train an AI model that predicts their behaviour.

Ask for proof of consent. If the system relies on the legal basis of “legitimate interest,” audit whether a legitimate interest assessment was completed and reviewed independently. If the organisation claims individuals have opt-out rights, verify that the opt-out is actually working and that data for people who opted out is not in the training data.

Data Retention

Finally, check that data is not being kept indefinitely. If a model was trained on customer data from five years ago, ask whether that data is still in the system and whether retention is justified. Many organisations have poor data retention practices and hold data out of habit rather than business need. This creates both data security risk and regulatory exposure. A data subject can request deletion of their data (right to erasure), and if your organisation still has it because no one deleted it, you have a compliance problem.