PII Redaction for LLMs: How to Keep Customer Data and Secrets Out of AI Tools

To keep personal data and secrets out of AI tools, do three things. Remove them before you paste: strip the columns the task does not need and swap names and e-mail addresses for placeholders. Turn off training in every AI tool that offers the setting. Then put a redaction layer in the traffic for what slips through. That layer is what PII redaction for LLMs means: software that finds personal data (names, e-mail addresses, phone numbers, IDs) and credentials such as API keys, and replaces them before they reach the model, come back in a response or land in a log. Below: where the leaks happen, the settings that matter, and the redaction options, free to paid.

Written for marketers, so the data is familiar: CRM exports, survey responses, support tickets, client briefs, and an API key in a .env file or a screenshot.

You pasted the customer list

It happens on an ordinary Tuesday. You export 400 CRM rows to ask which segments churned, or drop 200 survey responses into a chat with names and e-mail addresses still attached.

Nobody attacked anything. That is exactly what OWASP files under LLM02:2025 Sensitive Information Disclosure: private information ending up where it should not, through what the model says or what you put in (our plain-English OWASP LLM Top 10 walks the list). The best-known case has the same shape: Samsung engineers pasted proprietary code and a meeting transcript into ChatGPT, and in May 2023 Samsung restricted staff use of generative AI, as Bloomberg reported.

The fix is not "never use AI with customer data". It is knowing where the data goes after you press Enter.

Where personal data and secrets actually leak

Six places, and people only think about the first.

Where it leaks	Marketer's example	What stops it
What you paste in (the prompt)	A CRM export with names, e-mails and deal notes	Strip and pseudonymise before pasting; prompt-side redaction
What comes back (the response)	A ticket summary quoting customers' e-mail addresses, copied into Slack	Response-side redaction before return or logging
What agents read	An assistant with your inbox, Drive or a folder holding a .env file	Narrow access; deny rules; credential scanning
Provider retention and training	A consumer chat kept in history and used for training	Training off; Temporary or Incognito chats; business plans or API
Shared chats and links	A share link anyone holding it can open	Share only what you would publish; delete old links
Logs of every tool in the chain	The automation platform, the observability tool and the gateway each keep a copy	Know each tool's retention; prefer fingerprints over text

Responses travel further than prompts. You paste once, into one tool. The answer gets copied into a doc, a Slack thread, a Zapier step and an e-mail to the client. If the summary repeats twelve customer e-mail addresses, they now live in four more systems.

Agents read what you never pasted. An assistant connected to your inbox or files sends whatever it opens to the model in the next request. Coding agents sit in folders where .env files live; our Claude Code security checklist has the deny rules that keep them out of reach, and the MCP security checklist covers connectors (if you followed our first MCP tutorial, that means you).

Shared links are public by design. In summer 2025 Fast Company found thousands of shared ChatGPT conversations in Google results, shared through an opt-in "Make this chat discoverable" box. OpenAI removed the option (The Register, 1 August 2025).

Provider bugs are the one leak habits cannot prevent. On 20 March 2023 a bug let some ChatGPT users see the titles of other users' chats, and OpenAI said it may have exposed limited payment details (not full card numbers) for 1.2% of Plus subscribers active in a nine-hour window. Whatever you paste is only as private as the provider's next bug allows.

The provider settings that matter

Checked on the providers' own pages on 23 September 2026. Settings move; check yours.

Tool	Training on your chats	Retention to know about	The setting
ChatGPT (consumer)	Yes, unless you turn it off	Temporary Chats: kept 30 days for safety, then deleted	Settings → Data Controls → "Improve the model for everyone" off; Temporary Chat for one-offs
ChatGPT business plans and the OpenAI API	Not by default	API abuse-monitoring logs kept up to 30 days; Zero Data Retention only on approval	Nothing to switch; ask sales about Zero Data Retention
Claude (Free, Pro, Max)	Only if "Help improve Claude" is on	On: de-identified data kept up to 5 years for training. Deleted chats: gone from back-end systems within 30 days	Settings → Privacy → "Help improve Claude" off; Incognito for one-offs
Claude for Work and the Anthropic API	Not by default	API inputs and outputs deleted within 30 days; zero data retention by agreement	Nothing to switch; ask about zero data retention

Turning training off is not deleting. Your chats stay in your history, and Anthropic notes that data already in a training run that has started cannot be taken back. Switch it off before the customer list, not after. For one-offs, use ChatGPT's Temporary Chat (no history, no memories, no training) or Claude's Incognito mode, which is never used for training even with the setting on.

The simplest upgrade for a small team is structural: use a business plan or the API for client data. Neither trains on your content by default, and both come with contract terms you can show a client.

Prompt leaking: your system prompt is not a secret

A second leak runs the other way. Prompt leaking is getting a chatbot or agent to recite its own hidden instructions. OWASP lists it as LLM07:2025 System Prompt Leakage, and in the 2026 edition it becomes hidden context exposure (LLM08:2026), widened to tool schemas and retrieved policy text. The classic case: in February 2023 Kevin Liu got the new Bing Chat to print its rules and its internal codename, Sydney, by asking it to ignore previous instructions.

OWASP's key sentence: the system prompt "should not be considered a secret, nor should it be used as a security control." Its first attack scenario is someone finding credentials in a system prompt and using them. For marketers, the system prompt is a custom GPT's instructions, a Claude project's instructions or the prompt behind a client's website chatbot.

System prompt leak prevention, then, is mostly about what you do not put there:

• No keys, passwords or connection strings. A tool that needs an API key should get it from a secret store, not from the prompt.
• No customer data "for context". A customer list in a public GPT's knowledge is a breach waiting for the right question.
• No rules that only work if nobody reads them. "Never give discounts over 10%" belongs in the checkout; a prompt instruction can be talked around, which is what prompt injection is.

Write every system prompt as if it will be screenshotted and posted.

Four ways to redact PII, from free to paid

1. By hand, before you paste (free)

Most marketing analysis does not need identities. Churn patterns need plan, signup month, last activity and reason, not names.

• Delete columns the question does not need: name, e-mail, phone, address, and free-text notes that contain them.
• Pseudonymise what stays: replace each customer with Customer_001 and keep the key file on your machine. If the answer names Customer_047, you can look them up; the model cannot.
• Aggregate where you can: counts per segment usually answer the question as well as raw rows.
• Scrub screenshots: tabs, terminal and address bar. A key in the corner of a screenshot is a classic.

Pseudonymised data is still personal data under GDPR if you can link it back (more below). It cuts the risk; it is not an exemption.

2. An open-source detector (free, needs a developer)

Presidio, the open-source PII detection and anonymisation framework Microsoft started (its repository now lives at github.com/data-privacy-stack/presidio), finds personal data in text, images and structured data using named-entity recognition, regular expressions and rule-based checks such as checksums, then replaces, redacts, masks, hashes or encrypts it. It is MIT-licensed, and its README carries the caveat every detector should: "there is no guarantee that Presidio will find all sensitive information." Useful in your own pipeline or app; overkill if you only use ChatGPT in a browser.

3. Cloud DLP features (free to paid, if you already use the cloud)

• Cloudflare AI Gateway DLP scans prompts and responses, including tool call arguments and results, and can flag or block a request; the docs list no redact action. DLP scanning in AI Gateway is free on all plans, but without a Zero Trust subscription you get two predefined profiles: Financial Information, and Social, Insurance and National Identifier Numbers. E-mail addresses are not in either.
• Google Cloud Model Armor screens prompts and responses and, with Sensitive Data Protection templates, can de-identify what it finds, from hundreds of built-in data types such as EMAIL_ADDRESS. Aimed at teams building on Google Cloud.

4. A gateway that redacts in the traffic (paid)

A gateway sits between your AI tools and the model and redacts as traffic passes, including the response on its way back. It covers the "what comes back" and "what agents read" rows above without code. More below, with its limits.

Before you paste: a checklist

Pin it next to your screen, or put it in your team's one-page AI note.

• Would I post this? If a row, screenshot or instruction would be a problem on social media, it does not go in unchanged.
• Does the task need identities? If not, delete name, e-mail, phone and address columns first.
• Are there secrets in it? API keys, passwords, tokens, .env contents, connection strings, and the corner of that screenshot.
• Is training off in this tool, or am I in Temporary Chat, Incognito, a business plan or the API?
• Is this a client's data? Then check the contract: some clients restrict which processors may see their customers' data.
• Where will the answer go next? A shared doc or an automation copies whatever personal data the answer carries.
• Am I about to share this chat? Share a summary instead.
• Did a key slip through? Rotate it. Deleting the chat does not un-send it.

What GDPR means for a marketer with a CRM export

Plain terms, from the text of the regulation, not legal advice. If a client contract or a large dataset is involved, ask your privacy adviser.

• It is personal data. Article 4(1): any information relating to an identified or identifiable natural person. A CRM export with names or e-mail addresses qualifies.
• Pasting it is processing. Article 4(2) lists operations including use, storage and disclosure by transmission. Sending rows to an AI provider is one, so the usual questions apply: purpose, legal basis, minimisation.
• The provider is usually a processor. Article 4(8): a body that processes personal data on the controller's behalf. Article 28 says a controller may use only processors providing sufficient guarantees, under a binding contract, the data processing agreement (DPA). Business plans and APIs are where AI providers offer those terms.
• Pseudonymised is still personal. Recital 26: data that could be attributed to a person with additional information is information on an identifiable person. Only truly anonymous data falls outside the regulation.
• A gateway is another processor. Every tool in the chain that sees the content, a redaction layer included, is one more party to record.

Where a gateway fits, and where it does not

Two rows of the leak table no habit fully covers: what comes back in responses, and what agents read on their own. That is the gap we use Constellation Gate AI for (our review; setup and pricing on our Gate page). Affiliate link: we earn a commission on Pro seats that start from our link, which funds the testing and has no say in the verdict.

Gate calls itself "the accountability layer for AI" and sits "between your agent and the model". Its Policies screen has three scans: prompt-injection detection, PII/PHI redaction, and credential and secret leak detection. Each can be pointed at prompts, responses or both. Redaction strips e-mail addresses, credentials and other personal data before a response is returned or logged. Gate's own example is a support-ticket summary with three e-mail addresses redacted: "the summary still arrives; the addresses don't."

The audit ledger stores fingerprints, not content, so the permanent record holds no customer data (how that works). Admins see usage and cost, never anyone's prompts. Gate says it does not train on prompts.

Now the plain part.

• It is a third party. Your prompts and responses pass through Gate's infrastructure. Check client contracts and Gate's DPA before routing client data through it; the Enterprise plan offers private cloud or VPC hosting, data isolation and a DPA.
• It only sees routed traffic. Claude Code, Codex, Cursor and OpenCode connect with one toggle, Claude Desktop and ChatGPT through a local proxy. A browser tab that is not routed through it, with a list pasted in, is invisible to it.
• Response redaction does not un-send the prompt. If the list went into the prompt, the model received it; only prompt-side redaction prevents that. Our team rollout plan starts with redaction on responses and decides on prompts after a week, because prompt redaction sometimes gets in the way of legitimate work. If you paste exports daily, turn on both sooner.

Free records, Pro blocks. Free is $0 for 20,000 recorded requests a month with the audit trail, so you see what goes through your AI tools. Redaction and credential scanning are Pro, $20 per user per month. Weighing other kinds of screening? Our AI firewall explainer sets out the categories. Start free →

Your first fifteen minutes

This follows the method from how to actually use the tools you pay for: one task, one sitting, something done at the end.

1. Minutes 0 to 3: the switches. ChatGPT: Settings, Data Controls, "Improve the model for everyone" off. Claude: Settings, Privacy, "Help improve Claude" off. Same in every other AI tool you use.
2. Minutes 3 to 6: the history. Search your chats for "@", "key", "sk-" and "password". Delete chats holding customer lists or keys, and shared links you no longer need.
3. Minutes 6 to 9: the keys. Rotate any key you find in a chat, screenshot or system prompt. Then strip your custom GPT and Claude project instructions of anything you would not publish.
4. Minutes 9 to 12: the template. Save a copy of your usual CRM export with names, e-mails and phones removed and a Customer_001 column added. That is the version AI tools get.
5. Minutes 12 to 15: the screen. Put Gate on the free plan in front of the AI tool you use most, and set a reminder to look at what went through it in a week.

Output: training off everywhere, no keys in old chats, a safe export template, and a week of evidence to decide whether redaction is worth $20. Add the checklist to your team note. That is the whole job.

Sources: OpenAI Help Center, "Data controls in ChatGPT" and "Temporary chat in ChatGPT"; OpenAI, "How your data is used to improve model performance"; OpenAI platform documentation, "Data controls in the OpenAI platform"; OpenAI, "March 20 ChatGPT outage: Here's what happened"; Anthropic Privacy Center, "How long do you store my data?", "How long do you store my organization's data?" and "How do I change my model improvement privacy settings?"; Anthropic, "Updates to Consumer Terms and Privacy Policy" (August 2025); OWASP GenAI Security Project, LLM02:2025 Sensitive Information Disclosure and LLM07:2025 System Prompt Leakage (genai.owasp.org); Presidio repository and documentation (github.com/data-privacy-stack/presidio, presidio.dataprivacystack.org); Cloudflare AI Gateway documentation, "Data Loss Prevention" and "Pricing"; Google Cloud documentation, Model Armor; Regulation (EU) 2016/679 (GDPR), Articles 4 and 28 and Recital 26 (gdpr-info.eu); Bloomberg on Samsung (2 May 2023); The Register, "OpenAI removes ChatGPT self-doxing option" (1 August 2025); Constellation Gate AI documentation. All checked 23 September 2026.