Stop renewing AI contracts until they say who can read your prompts

A contractor working on Microsoft's Copilot opens a task and sees a stranger's prompt, the photo that stranger uploaded and two edits the AI made of it. The job is to judge which edit did what the user asked. According to internal documents obtained by 404 Media's Joseph Cox and published on 28 September, those reviewers are constantly bombarded with sexual editing requests, and they are grading whether the machine did them well. Prolific, identified as one of the hiring firms, did not respond. Microsoft pointed to its terms of use.

My first reaction was that this was a consumer scandal and that nobody running a B2B company needed to lose sleep over it. Microsoft's consumer privacy FAQ openly says some Copilot conversations get automated and human review. Its commercial documentation, updated on 18 August 2026, says Microsoft Copilot services have opted out of the abuse monitoring, human review included, that exists in Azure OpenAI. It also says prompts and data pulled through Microsoft Graph are not used to train foundation models. On paper, the enterprise buyer is fine. I went to confirm that and came back less comfortable.

Look at where the protection lives. The human-review opt-out is a sentence on a Microsoft Learn page, and that page carries an update date because Microsoft edits it. The same page says your Copilot data is handled in line with the contractual commitments covering the rest of your Microsoft 365 content, which is the part a lawyer can lean on. The specific promise that no person at Microsoft or a subcontractor will read your staff's prompts is, for most customers I suspect, sitting in documentation and nowhere in the order form or data processing agreement. If your procurement team is relying on a URL, it is relying on something the vendor can rewrite between renewals.

Then look at the product your engineers actually build on. On Azure OpenAI, human review is on by default. When automated checks flag content or a pattern of use, authorised Microsoft employees may assess it, and the data is kept for 30 days for that purpose. Switching it off means applying through your Microsoft account team for something called Modified Abuse Monitoring and waiting for approval. Even then, Microsoft's own answer to a customer says "automated review may still be conducted." A company that bought Microsoft 365 Copilot for its sales team and let its developers build a support bot on Azure is running two opposite privacy defaults from the same supplier, and I would bet most finance directors signing those invoices could not say which is which.

And then there are employees who never touch the enterprise tenant. Coverage of the 404 Media story points out that consumer Copilot lacks the stronger protections given to enterprise users. Microsoft's training opt-out setting is not even shown to people signed in with organisational Entra ID accounts, because their chats are excluded from training anyway. So the dividing line is which account someone happened to be logged into when they pasted a customer's contract into a chat box. Uploaded images on consumer Copilot are deleted within 30 days, which is 30 days longer than you want a stranger looking at your pricing sheet.

The obvious pushback is that Microsoft does not negotiate bespoke terms with a mid-sized customer, and that asking for audit rights over human review is a fantasy for anyone without a giant enterprise agreement. I disagree, because of who needs whom. Copilot is an add-on Microsoft badly wants attached to every seat it already sells, and renewal is the one moment a buyer holds that leverage. You do not need a custom contract. You need three things written into the DPA or order form: that no human review of prompts, outputs or uploads occurs without notice to you, a hard retention cap, and a right to receive a log of any human access. If Microsoft refuses to put its own published position into a contract, I would plan as if the documentation page could change before your next renewal.

Audit rights matter more here than usual because nobody outside Microsoft knows the numbers. The reporting does not establish how many contractors had access to uploads or what share of image tasks got human eyes. Without a clause, you will never find out either, for your own data. The same three clauses belong in the contract with every other AI supplier you use; Microsoft is the one whose internal documents leaked this week.

Before your next Microsoft renewal, send your account team one question in writing: which Microsoft staff or contractors could view our prompts in the past year, and under what documented process. If the reply is a link to a Microsoft Learn page, send it back and ask for those same words in the DPA, signed, before the purchase order goes out.

Prompted by Humans Are Reading Copilot Prompts — And They're Horrified, 404media.co.