What Happens to Client Information When an AI Model Trains on It
The consumer versions of AI tools like ChatGPT, Claude, and Gemini can use your conversations to train future AI models. Training means the provider may use a conversation to improve later models, and the conversation can have effects that persist after deletion. This article explains what that means for lawyers who wish to use AI with confidential information.
Three risks when consumer AI trains on client information
The developer keeps it. Allowing training means you allow the model developer to keep your conversations for a longer period of time. Anthropic’s retention policy (opens in new tab) for consumer Claude shows the difference. If the user allows training, Anthropic “may retain your data in a de-identified format for up to 5 years in model training pipelines.” If the user does not allow training, a conversation the user deletes is removed from Anthropic’s back-end systems within 30 days.
People may read it. Allowing training on your data can mean that you are granting permission for the developer to send your AI conversations to human reviewers for selection, labeling, or quality assessment. OpenAI’s consumer data FAQ (opens in new tab) says that “a limited number of authorized OpenAI personnel, as well as trusted service providers that are subject to confidentiality and security obligations, may access user content” for four listed reasons, the last of which is “to improve model performance (unless you have opted out).” The same page tells users: “Please do not enter sensitive information that you would not want reviewed or used.” Google’s Gemini Apps Privacy Hub (opens in new tab) says that “a subset of chats are reviewed by human reviewers (including Google’s trained service providers) to help improve Google services.” Chats that have been reviewed “are not deleted when you delete your activity. Instead, they are retained for up to three years.” Google gives its users this instruction: “Please don’t enter confidential information that you wouldn’t want a reviewer to see or Google to use to improve our services, including machine-learning technologies.”
It can be disclosed to others. In some instances, AI models have disclosed the exact text of confidential information that was used for their training. If a provider uses a conversation in training, it may influence the behavior of a model that is then available to other users. Deleting the conversation afterward does not undo that influence.
Retention, human review, and training use are separate risks, each governed by the provider’s terms. Any one of them can expose client information to people outside the lawyer’s control. Training, retention, and human-review rights vary by product, account type, and account settings. A free consumer-facing plan may train by default. A paid consumer plan may not. A business or enterprise API plan typically prohibits training by contract. The answers for any given account depend on the terms that account accepted.
What the bar rules say about AI training on client information
ABA Model Rule 1.6(c) (opens in new tab) requires a lawyer to make reasonable efforts to prevent unauthorized access to information relating to the representation. A developer’s reviewers, and the contractors who work alongside them, are people with access to that information. The lawyer did not choose them and does not know who they are.
ABA Formal Opinion 512 (opens in new tab) addresses tools that train on input, which it calls “self-learning.” It concludes that “a client’s informed consent is required prior to inputting information relating to the representation into such a GAI tool.” The opinion also says that boilerplate language in an engagement letter does not supply that consent.
The D.C. Bar’s Ethics Opinion 388 (opens in new tab) tells a lawyer to ask two questions before using a generative AI tool: “Will information I provide to the GAI be visible to the GAI provider or other strangers to the attorney-client relationship?” and “Will my interactions with the GAI affect answers that later users of the GAI will get in a way that could reveal information I provided to the GAI?” For a tool that trains on input, the answer to the first question is yes, and the answer to the second is that the lawyer cannot rule it out.
The New York City Bar’s Formal Opinion 2024-5 (opens in new tab), quoting guidance from the State Bar of California, says a lawyer “should ensure that the provider does not share inputted information with third parties or use the information for its own use in any manner, including to train or improve its product.” A developer that trains on a lawyer’s input is using client information for the developer’s own purposes. Unless the client has given informed consent, the lawyer has no authority to permit that use, and the lawyer cannot retrieve the information afterward.
California’s SB 574, signed September 30, 2026 and effective January 1, 2027, puts a similar test into statute. A lawyer may enter nonpublic information into a generative AI system only if access is restricted to the lawyer and “persons authorized by the attorney under obligations to protect the confidentiality of the information.” Our article on SB 574 walks through that five-question test.
How Bells Up AI handles training
Bells Up AI does not use your conversations or documents to train AI models. Every company that runs a model for us is prohibited by contract from training on your prompts and responses. These no-training terms apply to every plan, including our free plan, and to every user at a firm.
For the models we run through Amazon Bedrock (Claude, Llama, DeepSeek, Mistral, Qwen, and older Kimi and GLM models), the company that developed the model never receives your prompts and responses. For newer Kimi and GLM models, Fireworks runs the model and the developer never receives them.
OpenAI and Google keep prompts and responses for a limited period to monitor for abuse (up to 30 days for OpenAI and 55 days for Google) but both are prohibited by contract from training on them.
Our privacy reference page has the provider-by-provider detail, with links to the governing terms.