Dataset Documentation, Provenance and Licence Checks

The data you train a model on is the foundation for everything that follows. You must know where each dataset came from, who collected it, for what original purpose, and whether you have legal rights to use it for model training. This knowledge is called dataset provenance.

You must document the licence under which the data is available and verify that the licence permits AI training or commercial deployment. You must record the size, scope, content and known limitations of each dataset you used.

Dataset documentation serves multiple purposes. First, it allows someone else to rebuild your model from scratch using the same data. For example, a data scientist at a financial services firm may need to recreate a fraud detection model after a team member leaves.

Without proper documentation, this process becomes extremely difficult and time-consuming. Second, it allows auditors to assess whether the data is appropriate for your intended purpose. A compliance officer reviewing an AI model for healthcare diagnostics will scrutinise the data sources to ensure they meet regulatory standards.

Third, it reveals data quality issues that might affect model fairness or accuracy. A model trained on a dataset with gender bias in historical hiring data may perpetuate those biases in future predictions. Fourth, it protects your organisation from inadvertent misuse of data that you do not have legal rights to.

If you train a model on data you are not permitted to use, the model itself becomes legally problematic and cannot be deployed. For instance, a UK-based e-commerce company using a dataset with unclear commercial usage rights could face legal action if the model is deployed in a commercial product.

Provenance means you know the origin of each record in your dataset. If you are using customer data collected by your own organisation, do you have a legal basis to use it for model training? Is there a clause in the customer privacy notice that permits this?

For example, a customer service chatbot trained on support conversations must ensure that users have consented to their data being used for AI training. If you are using open datasets from the internet, which version did you use and is that version stable? A data scientist working on a natural language processing model may use a version of the Common Crawl dataset from 2020.

If that version is updated or deprecated, it could affect model performance. If you licensed data from a third-party supplier, what exactly are the licence terms? A company using a dataset from a commercial provider must ensure that the terms allow for model training and deployment.

If you collected data yourself, what informed consent did you obtain from participants? A research team conducting a survey for a predictive model must ensure that participants understand how their responses will be used and have provided explicit consent.

Licence rights matter more than many data scientists realise. Some datasets are released under licence terms that prohibit commercial use. Others permit use only for research or non-profit purposes.

Some prohibit creating derivatives or require attribution. If you train a model on data you do not have rights to use, the model itself may inherit those restrictions. You cannot legally deploy a model trained on research-only data into a commercial service without violating the licence.

Your organisation could face legal liability. For example, a UK startup using a dataset under a Creative Commons Attribution-NonCommercial licence for commercial model training could face legal challenges. A model trained on a dataset with a “no derivatives” clause may not be allowed to be fine-tuned or adapted for specific use cases.

A company using a dataset that requires attribution must ensure that all relevant credits are included in model documentation and public-facing materials. This is especially important when deploying models in open-source environments or publishing research findings.

You must also document known errors, limitations and biases in the data. Did the original collectors mention any known errors in the dataset? Are there dates when data collection changed procedures or quality?

Are there subgroups underrepresented in the dataset? Are there fields with high rates of missing values or obvious outliers? Is the dataset representative of the population your model will serve in production?

For instance, a model designed to predict loan default risk may be biased if the training data is predominantly from urban areas and excludes rural demographics. A dataset used for a customer churn prediction model may have missing values in customer satisfaction scores, which could lead to inaccurate predictions.

A healthcare model trained on data from a specific hospital may not generalise well to other populations. A data scientist working on a facial recognition system must consider that datasets may underrepresent certain ethnicities or age groups, which can lead to performance disparities. Documenting these limitations helps ensure that model developers and users understand the boundaries of their models and take appropriate steps to mitigate risks.