You generally cannot use the consumer ChatGPT product with protected health information, and you can use certain API and enterprise arrangements if a business associate agreement is in place and the deployment is configured correctly. The distinction matters more than the brand: consumer chat interfaces are typically not BAA-eligible, retain conversations by default, and may use inputs to improve the service, all of which are disqualifying. API and enterprise tiers can be covered by a BAA and configured for zero retention, which makes compliant use possible rather than automatic. The second question is the one teams skip: whether the prompt needs PHI at all. HIPAA’s minimum necessary standard applies to AI requests exactly as it does anywhere else, and a great many clinical use cases work on de-identified or structurally abstracted input, which removes the compliance question rather than managing it.
The Short Answer
Not with the consumer ChatGPT product; yes under specific API or enterprise arrangements with a signed BAA and correct configuration — and even then, only if the prompt genuinely needs PHI in it at all. The conditions attached to that “yes” matter as much as the yes itself, and the rest of this post covers exactly what they are.
Consumer ChatGPT vs. API vs. Enterprise
The consumer ChatGPT product is generally not BAA-eligible, retains conversations by default, and may use input to improve future models depending on account settings — all disqualifying for PHI. OpenAI does offer a BAA for the API, which also requires its Modified Retention setting on your organization, and for ChatGPT Enterprise in its Regulated Workspace configuration. Zero data retention applies only to specified API endpoints, and, as I go through in the full BAA breakdown, not every API feature is covered even within an otherwise eligible tier.
If we build a custom integration for you, you do not need your own BAA with OpenAI or another model provider. We reach these models through our existing AWS agreement, which is covered by a BAA, so the model layer sits under agreements we already hold with our vendors. Your agreement is with us.
What a BAA Changes
Briefly here, since the full picture — including exactly what the BAA does and doesn’t cover, and the application-side logging exposure it doesn’t reach — lives in that dedicated post. The short version: a BAA makes compliant use possible on the covered tier. It does not make your specific implementation compliant automatically.
Does the Prompt Need PHI at All?
This is a genuinely different question than “which tier is compliant,” and teams often skip it. Minimum necessary applies to AI prompts the same as any other PHI use — ask whether the actual task needs the patient’s name, full chart, and every identifying detail, or whether a de-identified or structurally abstracted version of the same clinical question would work just as well. A large share of real clinical AI use cases — summarization, drafting, classification — work fine on abstracted input, and doing that removes the entire BAA-and-configuration question rather than managing around it.
Where Teams Actually Get Caught
Not the model vendor — application logs, debugging traces, and screenshots pasted into support tickets are the real, boring places PHI leaks in AI-assisted tools. A perfectly configured, BAA-covered API call is still a problem if the request and response get logged in plaintext somewhere your own team can see them without proper access controls.
Shadow AI Is the Real Risk
The risk leaders actually have and rarely discuss openly: staff pasting PHI into consumer AI tools on their own, unsanctioned, because it’s faster than the approved workflow. No amount of API-level BAA architecture solves this — it requires both a clear policy and technical controls (network-level blocking of consumer AI endpoints, for instance) working together, because policy alone doesn’t stop a well-intentioned staff member trying to save time.
A Defensible Position
Document which tier you’re using and why it’s BAA-covered, what data minimization you’ve applied before any PHI reaches a prompt, and what technical controls address the shadow-AI risk above. That documentation is what actually demonstrates a defensible compliance posture if it’s ever questioned — not a vendor’s marketing claim that their product “is HIPAA compliant.”
Next Steps
The short answer is genuinely conditional, and the conditions are the substance — verify your specific tier, minimize what reaches the prompt, and control the shadow-AI risk explicitly. For the full BAA picture, see our dedicated post, and for the ambient-scribe use case specifically, see our post on ambient AI scribes. Talk to our team if you’re architecting an AI-assisted healthcare product.