Confidentiality and Price Sensitive Information in Prompts
When you put financial data into an AI system, you are making a calculation: the efficiency gain from using AI versus the risk of data exposure. This risk calculation must be explicit. Many finance teams send invoices, contracts, personal data and confidential business information into language models without understanding where that data goes and whether vendors might use it.
The default practice for most commercial language model APIs is to log prompts and responses. This logging serves multiple purposes for the vendor: it allows them to monitor system performance, to investigate misuse, and (in some cases) to improve their models using customer data. When you send an invoice to an API to extract the line items, that invoice is logged by the vendor. That log might be retained for weeks, months or years. It might be accessed by vendor employees. It might be deleted after a period, but you do not control when. If the invoice contains confidential pricing, supplier payment terms or customer identities, this information is now in the vendor’s log.
Some vendors offer privacy-preserving options. They might allow you to opt out of logging, though this often costs more or requires a higher-tier contract. Some vendors delete logs after a shorter period (e.g., 30 days instead of 90 days). Some vendors offer on-premise deployment where models run on your own infrastructure and logs never leave your systems. If your data is highly confidential, these options are worth investigating.
Data protection law also constrains what you can send to a language model. The UK General Data Protection Regulation and GDPR require that you only process personal data where you have a lawful basis and an appropriate contract. If you send customer invoices (which contain customer names and addresses) to a public language model API, you are processing personal data. You need a Data Processing Agreement with the vendor that specifies how personal data will be handled. Many vendor terms make this difficult; they might process data globally or might not commit to European data residency. Check the contract carefully.
Confidentiality agreements with customers and suppliers also constrain what you can disclose. If your customer contract states “customer pricing is confidential and must not be shared with third parties,” you cannot send customer invoices to a language model vendor without breaching that agreement. Some vendors are comfortable with confidentiality obligations; others are not. Agreeing to a vendor’s standard terms might require amending your customer agreements, which is expensive and time-consuming.
Price sensitive information is an additional category. If your company has announced a forecast or stated a plan, and that becomes publicly material information, you cannot send it to a language model. Language models are Internet-connected systems. Data sent to them might be stored or might be observed by vendor employees. This creates risk of inadvertent disclosure.
The safest approach for highly sensitive data is to avoid language models entirely. Instead, use rule-based extraction systems or traditional OCR software that runs on your own infrastructure. These systems do not learn from your data and they do not send data elsewhere. They are less flexible than language models but they preserve confidentiality completely. For less sensitive data (for example, non-confidential expense categorisation), commercial language models might be acceptable if you have negotiated appropriate vendor protections.
If you decide to use a language model on sensitive data, build compensating controls. First, do not send the entire document. Extract or redact sensitive fields before sending. If you need a language model to classify transaction type, send only the transaction description; do not send vendor name, amount or date. Second, do not use your company name or customer names. If you send a prompt “classify this transaction from XYZ Pharma for £500,000 of equipment,” you are unnecessarily exposing your customer relationship. Send instead “classify this transaction for equipment of £500,000.” Third, do not send documents to a language model API; send them to a language model you run locally. Open-source language models like Llama or Mistral can be downloaded and run on your own infrastructure. They do not send data anywhere. They are smaller and somewhat less capable than commercial models but they preserve confidentiality.
Documentation matters here too. Keep records of what data was sent to which vendors, what protections you negotiated and what compensating controls you put in place. If a confidentiality breach later occurs (for example, a vendor employee discloses information), you want to be able to show that you took reasonable precautions.