FDA didn't warn against artificial intelligence in pharmaceutical manufacturing. It warned against a specific architectural failure — and most of the industry is drawing exactly the wrong lesson.
The sentence that will define FDA’s first AI warning letter is buried in the third paragraph of the findings. The company under inspection had distributed drug product without conducting process validation — a requirement under 21 CFR 211.100 that has been in effect for decades. When FDA investigators raised this during the inspection, the company’s response was recorded verbatim: they were not aware of the legal requirement, because the AI agent they relied on had never told them it was required.
Read that sentence carefully. Not because it reveals that AI is dangerous in pharmaceutical manufacturing. Because it reveals exactly what kind of failure this letter is about — and why most of the industry’s response to it will be precisely wrong.
FDA’s warning letter cited a company that used AI agents to create drug product specifications, procedures, and master production and control records, then distributed drug product without qualified review of those documents and without conducting process validation. The agency cited 21 CFR 211.22(c), which requires the quality control unit to review and approve written procedures. It cited 21 CFR 211.100, governing procedures and process validation. None of these regulations are new. All of them have existed for decades. FDA applied existing law to a new context — and the company had no argument in response, because it had no quality unit capable of knowing the law applied.
The industry will read this as a warning against AI in pharma. That reading will produce a predictable response: additional caution around AI adoption, new sign-off requirements, more checkboxes before distribution. These responses are directionally correct and substantively insufficient. They address the surface of this warning letter and miss the architectural failure it actually describes.
The company did not trust AI too much. They used AI without the architecture to understand what it had done or why. Adding a human sign-off to that architecture does not fix it.
The regulatory citations are precise and entirely predictable. 21 CFR 211.22(c) has always required the quality unit to review and approve written procedures. It did not require review of human-drafted procedures any differently from AI-generated ones — because the obligation rests on the quality unit, not on the document’s author. The quality unit is responsible for procedures. That responsibility has never been delegable to software, contractors, or contract labour. AI is not a new case. It is a new context for a principle that has governed pharmaceutical manufacturing since 1978.
What is new is the speed at which AI can produce documentation that looks like a compliant quality system. The company appears to have had no functioning quality unit. They had AI — and AI produced professional-looking SOPs, specifications, and production records at a pace no manual process could match. What AI could not produce was the GMP expertise required to know that the documentation was inadequate, that process validation had not been done, and that distribution should not have begun.
First
FDA's warning letter is the first enforcement action to cite artificial intelligence as a specific factor in pharmaceutical manufacturing compliance failures — establishing how existing CGMP regulations apply to AI-generated documents and AI-assisted quality decisions.
211.22(c)
The primary citation: the quality control unit must review and approve written procedures. Written in 1978. Applied to AI-generated documents for the first time in 2026 — not as new law, but as the inevitable application of existing accountability requirements to a new context. QU responsibility is non-delegable.
211.100
The company had not conducted process validation prior to drug distribution and was unaware the requirement existed — because the AI had not flagged it. An AI system without a reasoning audit trail does not just produce incomplete outputs. It produces unknown gaps that the quality team has no mechanism to find.
The instinctive response to this warning letter — require human sign-off on all AI-generated documents — is correct as far as it goes. It does not go far enough. The harder question, which most responses will avoid, is what “human review” means under 21 CFR 211.22(c), and whether a sign-off on an AI output the reviewer could not have meaningfully evaluated satisfies that standard.
211.22(c) requires the quality unit to “review and approve” written procedures. That is a substantive accountability requirement — not a presence requirement. The quality unit is responsible for the accuracy and completeness of procedures. A representative who signs AI-generated output without access to the reasoning behind it has produced a signature. FDA’s interpretation of 211.22(c) requires a review. An inspector who asks “can you walk me through the basis for this procedure” will find the difference quickly.
The company that received this letter did not know that process validation was a legal requirement. That is not an AI failure. That is an organisation with no functioning quality unit that used AI to generate documentation it lacked the competence to review. AI made incompetent look compliant at speed and scale. The warning letter is about the absence of GMP expertise — AI made that absence invisible until inspection. No AI system can compensate for the absence of qualified personnel. What it can do is make that absence harder to detect from the outside.
Every company that adds a sign-off step to its AI workflow will believe it has addressed this warning letter. Most will be wrong. A QU representative who approves AI-generated output without seeing what the AI consulted, what it concluded, and what it dismissed has conducted a checkbox review. They have read a document and signed it. FDA's 211.22(c) requires them to have reviewed and taken accountability for the procedures — which requires understanding the reasoning well enough to have caught an error if one existed. Checkpoint without reasoning access is not review. It is liability transfer.
An AI that generates a SOP and shows only its conclusion puts the human reviewer in an impossible position. They can read the document. They cannot verify that the AI consulted the right regulatory references, evaluated the right process controls, or flagged the gaps it should have flagged. When the AI is wrong in ways that don't look wrong — a missed validation requirement, a misapplied standard, a plausible but non-compliant procedure — the reviewer without reasoning access cannot catch it. Not because they failed. Because the architecture made it structurally impossible.
The AI in this warning letter most likely generated coherent SOPs and specifications within the scope of what it was asked to produce. Process validation was never discussed because nobody asked about it. AI operates within the conceptual frame of the questions it receives — if the person asking does not know that process validation is a regulatory requirement, they will not ask about it, and the AI will not raise it. This is not AI failure. It is the epistemological reality of any tool: the output is complete within the question asked, not within the full scope of what should have been asked. The quality unit exists precisely to hold that larger frame — to know what must be asked regardless of what the user thinks to raise. When that expertise is absent, AI does not compensate for it. It accelerates the gap.
The comparison below is not about how quickly AI outputs are reviewed or how many approvers are in the chain. It is about whether the quality unit representative has the information required to conduct the review that 21 CFR 211.22(c) requires — and whether an FDA inspector examining the record would find evidence of that review, or evidence of a signature.
The QU representative receives the AI-generated document. They read it. They assess whether it appears correct. They have no access to what regulatory sources the AI consulted, what process controls it assumed, what requirements it may have missed, or where its reasoning was uncertain. Their review is bounded by the visible quality of the output — and errors that don't look like errors remain invisible.
Output review without reasoning context
The QU representative receives the AI-generated document alongside its reasoning record: sources consulted, regulatory references evaluated, gaps identified, decisions made in drafting, and areas of low confidence. They review both what the AI produced and how it arrived there. Their sign-off attests to their assessment of the reasoning, not only the output.
Output review with full reasoning context
The audit trail records that a human approved an AI-generated document on a given date. It does not record what the AI considered, what the human reviewed, or the basis on which the human concluded the output was compliant. FDA can verify that a signature exists. They cannot verify that a substantive review occurred — and under 211.22(c), that distinction is the substance of the citation.
Signature without reviewable basis
The audit trail captures two distinct, independently retrievable events: the AI's work — what it was asked, what it accessed, what it produced, what it flagged as uncertain — and the human's decision — what they reviewed, what they assessed, and the basis for their approval or required revision. The AI's contribution is auditable. The human's decision is auditable. They are separate records.
Dual audit trail, independently verifiable
An inspector can verify that a procedure exists, that it carries a QU approval signature, and that the signature is dated. They cannot verify whether the reviewer had access to the AI's reasoning, whether they understood the basis for the procedure's claims, or whether their review was substantive under 211.22(c). The question 'walk me through the basis for this procedure' has no good answer.
Compliance appearance under routine inspection
An inspector can retrieve the complete decision chain: what the AI considered, what the QU representative reviewed, and the explicit basis for their approval. The review is not merely signed — it is demonstrable. The QU's accountability is not asserted; it is evidenced in the record. The question 'walk me through the basis for this procedure' has a documented answer.
Compliance substance, demonstrable under any inspection
If the AI produces a non-compliant procedure in plausible language, a reviewer without reasoning access has limited means to detect it. They can catch what looks wrong. A missed validation requirement that isn't flagged, a misapplied standard written confidently, a process control assumption that doesn't hold — none of these look wrong in the output. They surface at the next inspection.
Errors surface at inspection
If the AI produces a non-compliant output, the reasoning record exposes where the error originated — a misread regulatory reference, a gap the AI dismissed, a confidence assertion that was not warranted. The QU representative reviewing the reasoning can identify the failure point before it reaches the floor. The error surface is the reasoning, which is visible and reviewable, rather than the output alone.
Errors caught at review
FDA’s remedy in the warning letter is precise: AI output “must be reviewed and cleared by an authorized human representative of your firm’s QU in accordance with section 501(a)(2)(B) of the FD&C Act.” Building an architecture that satisfies that standard — not in appearance but in substance — requires four capabilities that most AI systems deployed in pharmaceutical settings today were not built to provide.
AI systems used in GMP activities must expose what they considered, not only what they concluded. Every document generated by an AI agent must be accompanied by a retrievable record of the regulatory sources consulted, the process controls evaluated, the gaps identified, and the decisions made in drafting. Without this record, the QU representative cannot conduct a substantive review — regardless of their qualifications.
The 'review and approval' required by 21 CFR 211.22(c) is substantive. The QU representative must be able to evaluate whether the AI's reasoning was sound — whether the right regulatory references were consulted, whether the right controls were applied, whether the right gaps were flagged. That review requires both the qualifications to assess GMP compliance and access to the AI's reasoning that makes assessment possible. Neither condition alone is sufficient.
Compliant AI deployments require two independently auditable records: the AI's work — what it was asked, what it accessed, what it produced, and what it identified as uncertain — and the human's decision — what they reviewed, what they assessed, and the basis for their approval or required revision. These records must be distinct and separately retrievable. An audit trail that captures only the approval timestamp cannot demonstrate that a substantive review occurred.
The test of a compliant AI governance model is whether the QU representative can answer the question FDA will ask: 'Show me what the AI considered, and explain why you concluded its output was compliant.' That question requires an architecture — not a policy. The AI must expose its reasoning. The human must have reviewed it. The audit trail must capture both. A sign-off on output the reviewer could not have meaningfully evaluated does not survive that question.
The question “did a human review this?” is answerable with a signature. The question FDA is now asking — “was the human in a position to have meaningfully reviewed this?” — requires an architecture to answer. Building that architecture is not optional. It is what 211.22(c) has always required.
The companies that will navigate the next phase of AI adoption in pharmaceutical manufacturing are not the ones that slow down in response to this warning letter, or the ones that add checkbox sign-off steps to AI workflows that were not designed to support substantive review. They are the ones that recognise what this letter actually established: FDA has defined, for the first time, what responsible AI in pharmaceutical manufacturing looks like. It looks like an architecture where the reasoning is visible, the human review is genuine, the audit trail captures both, and the quality unit’s accountability is intact at every step.
That architecture is not complicated to describe. It is genuinely difficult to build, and most AI systems deployed in pharmaceutical settings today were not built with it in mind. They were built to generate outputs faster — to produce SOPs, specifications, and procedures at a pace manual processes cannot match. The outputs are often good. The reasoning is often invisible. The audit trail often captures only the result.
The warning letter company lacked GMP expertise and used AI to paper over its absence. That failure is categorical and will be easy for the industry to dismiss as the behaviour of an outlier. The harder conversation — the one that will mostly not happen — is about the organisations with real GMP expertise that are currently deploying AI systems which prevent that expertise from being applied to the AI’s reasoning, because the AI never exposes it. Those organisations are compliant in appearance. They are exposed in substance. The difference between the two becomes visible when the inspector asks the right question.
FDA has now published, for the first time, exactly what that question is.