Evidence Issues: Deepfakes, Authenticity and Provenance
AI-generated content raises serious questions about evidence. Courts must be certain that evidence is genuine. As AI systems become capable of generating convincing text, images and video, the risk of fabricated evidence grows.
Deepfakes are synthetic images or videos created by AI to depict events that did not occur or to put words in someone’s mouth. A deepfake video might show a person saying something they never said or committing an act they never committed. Deepfakes raise several evidence issues. First, is the video genuine or fabricated? Courts currently have limited tools to determine this. A video that appears authentic might be a deepfake. Second, if a deepfake is used as evidence, it misleads the fact finder. The adversary may not spot the fabrication. Third, even if both parties know the evidence is a deepfake, its use in proceedings may be prejudicial because fact finders may be unconsciously influenced by it.
Current evidence law treats deepfakes skeptically but is still developing. In criminal proceedings, deepfakes would almost certainly be excluded under Evidence Act provisions relating to fabricated or unreliable evidence. In civil proceedings, deepfakes would be excluded as misleading and unhelpful. But the law is not yet settled.
Your advice to clients should be that deepfakes should not be used as evidence in legal proceedings. If a client has a deepfake and wants to use it, explain the risks. The evidence will likely be excluded. The court may draw an adverse inference about the client’s credibility. The opposing party will argue that the deepfake suggests the client has no genuine evidence.
The second evidence issue is authenticity of AI-generated text. If a client has a document that was generated by an AI system, is it admissible as evidence? The answer depends on what it is being used to prove. If it is being used to prove what the AI generated, it is fine. The document authenticates itself as AI output. If it is being used to prove facts about events (such as “the AI says X happened”), that is hearsay unless the AI system itself testifies.
In practice, documents generated by AI are problematic as evidence of fact because they are hearsay. If a client asks an AI system “did this company violate the law?” and the AI responds “yes, they violated section X”, the response is hearsay. The AI system is not a witness. It has no personal knowledge. Its output is based on patterns in its training data, not on investigation of this specific case.
The third evidence issue is provenance and chain of custody. Where did this evidence come from? If a document was generated by an AI, the chain of custody should include information about the AI system used, the input provided, the date and time of generation, and the person who obtained the output. This provenance information is important for authentication.
Similarly, if AI was used to analyse or process evidence, the provenance of that analysis should be documented. If an AI was used to transcribe audio recordings, the transcription report should document the AI system used, the accuracy rate, how the system was configured and any known limitations. If AI was used to modify an image, the modification process should be documented so the other party can evaluate whether the modification changed the material aspects.
Courts now expect documentation of AI use in evidence. If a client uses AI to modify an image and plans to use it as evidence, they should provide a detailed report of the modification process. If they use AI to transcribe audio, they should provide transcription metadata. This transparency allows the other party to challenge the evidence and allows the court to evaluate reliability.
A practical approach for clients is to treat AI-generated content as tentative. Use it as a starting point for investigation, not as conclusive evidence. If an AI analyses documents and suggests a pattern, investigate further to confirm the pattern using traditional methods. If an AI transcribes audio, have a human verify the transcription. If an AI modifies an image, document the process and be prepared to explain it to the court.