Can an AI agent take over CDMO paperwork? Start with the check

A planner raises a purchase order in SAP and emails the production request to the contract development and manufacturing organisation anyway, because the CDMO portal connects to nothing. Weeks later a certificate of analysis arrives as a PDF attachment. A coordinator opens it beside the approved specification, reads down twenty or thirty tested attributes, assay, water content, residual solvents, microbial limits, checks each against its acceptance criteria, works out retest and expiry dates from the manufacture date, and types the numbers into an SAP inspection lot. Somewhere in that chain sits a spreadsheet one person built and nobody validated. Then a second person signs to say the first got it right.

Every step of that is something a current computer-use model can attempt. None of it is something it is allowed to finish. That gap is where the argument lives.

Point an agent at it now, when all three are true

The output is a comparison rather than an entry: result against specification, date against rule, PDF against what is already in SAP, and nothing it produces writes to a record. A false positive costs a human ten seconds of looking, and a false negative leaves you exactly where you already were. And you can count the volume and the time, because an automation you cannot size is an automation you cannot justify.

Do not let it sign, when any one is true

The step ends in a release decision or a goods receipt, which US regulation requires a named person to perform and a second named person to review. Or you cannot state, as a number, how well the humans currently do the job, in which case you have nothing to hold the model to. The generative question is more permissive than it is usually reported: draft Annex 22 puts generative models and large language models outside its own scope and says they should not be used in critical GMP applications, then expressly allows them in non-critical ones provided a qualified human is responsible for the output. That is the sentence the shadow reader lives in. Annex 22 is a consultation draft rather than law, and the wider European timetable has already moved once, which we covered in what still applies after the EU AI Act delay. Waiting for the text to settle is not a plan, because the measurement it will ask for takes a quarter to produce whenever you start.

Change the organisation, not only the tool

The people who would build these workflows already build them, in spreadsheets, unsupervised. Hand them a better tool without a review discipline and you industrialise the same problem. Train the operators and then monitor the operators, because a model feeding a human decision makes that human part of the system. And say what happens to the jobs before the pilot rather than after it.

These are usually drawn as a maturity curve, as though a company graduates from one to the next. They are better understood as three different regulatory objects, each with its own evidence requirement, and there is no rule that says you must pass through all three.

Expand

Regulatory status is our reading of the Annex 22 consultation draft, which is not law. The draft keeps generative models out of critical GMP use rather than banning them outright, so watch and check remain open to a language model provided a qualified human owns the output. The three positions are a framing, not a standard.

Almost every pilot described in public starts at do, because that is where the headcount is. Watch is where the evidence is. Check is where the money is.

Why the ordering matters

Errors cascade. For a task of n steps at error rate e, the chance of at least one error is one minus (1 minus e) to the power n, a formula Panko adapts from Lorge and Solomon, and humans make undetected errors in about 0.5 percent of simple mechanical actions such as typing. Across the twenty-five or so fields of a typical certificate that is about a twelve percent chance any given transcription carries an uncaught error. That figure is our own arithmetic on a published error rate, not a measurement, and it is an estimate precisely because the measurement does not exist. An agent in watch mode replaces our arithmetic with your data inside a quarter.

The obvious alternative is to point the best available model at the keystrokes and let it drive SAP. Published benchmarks argue against it. On OSWorld, which is generic desktop work, the current frontier model scores 72.6 percent against 70.2 percent for the nearest competitor. On AutomationBench, which measures multi-step professional workflows, the same model scores 41.4 percent, with the next model at 31.4 percent and the previous flagship at 18.1 percent. The year-on-year improvement is steep. The absolute number is nowhere near a release decision. These are vendor-run evaluations on vendor-chosen tasks, and the vendor post itself blocks automated retrieval, so we take them from the published system card and read them generously.

Read the vendor evaluations generously and the conclusion still holds: autonomous execution of a long professional workflow is roughly a coin flip. Detection is a different job with a different tolerance. An agent that says this assay value does not match the specification you hold can be wrong two times in three and still be worth having, because being wrong costs a glance. We have written separately about where agentic systems fail in practice and why the failure modes cluster in retrieval and handoff rather than in reasoning.

O2 in the United Kingdom is the most carefully documented back-office automation in the academic literature, and what made it work was not the software. By April 2015 it had automated 15 core processes, deployed more than 160 software robots and was handling 400,000 to 500,000 transactions a month, with a twelve-month payback and a three-year return on investment between 650 and 800 percent. Business-operations staff, not developers, built the automations.

Here is the part that transfers to pharma. O2 used a heuristic of roughly three full-time equivalents saved before it bothered automating a process, and it could apply that rule because its work-allocation system already told it what a human took, per process. The manager who ran it put it plainly: "I know to the second how long that process has taken to complete over a number of years." The screening heuristic fell out of that measurement rather than preceding it. O2 had, in commercial form, exactly the thing Annex 22 asks for and pharma does not have.

Two details carry over. O2 IT first dismissed the whole programme as screen scraping built by self-taught staff and expected to need constant babysitting, which is a fair description of what already runs in most supply-chain departments and the reason the stigma exists. And when O2 removed humans from processes it thought were simple, it had to write common-sense rules that had never been needed, because a customer pre-ordering the same handset twice looked to the software like two orders. Judgment you did not know you were buying disappears the moment you stop buying it.

Between 25 September and 2 October 2020, 15,841 positive COVID-19 test results did not reach England contact tracing on time. An automated pipeline loaded laboratory results into a pre-2007 spreadsheet format capped at 65,536 rows. Rows past the ceiling were dropped with no error raised. The official statement said only that files had exceeded a maximum size; the format and the row ceiling come from BBC reporting, as does the estimate that each template held only about 1,400 cases, which has been questioned since because it implies more rows per result than anyone has explained. The cases were transferred once the fault was found on 3 October, so this was a delay rather than a permanent loss. No government figure was ever published for contacts missed. The widely repeated 48,000 came from the shadow health secretary extrapolating in the Commons on 5 October, and should be read as his estimate rather than as a count.

There was no model and no agent here, only an automation nobody owned, validated or monitored, which is the artefact already sitting inside most supply-chain departments. Note what actually failed. Not the arithmetic: the pipeline did what it was built to do. What was missing was any check that the records going in matched the records coming out. It failed silently because nothing was watching. An agent whose only job is to reconcile two representations of the same data is precisely the control that was absent.

Pharma is not careless. It is exhaustive about the things it has decided to be exhaustive about, and blind everywhere else. Most new drugs now depend on this interface. Of the novel drugs FDA approved in 2025, 65 percent used outsourced finished-dose manufacturing, an eleven-year high, and 73 percent used outsourced active-ingredient manufacturing, just short of the 74 percent recorded in 2024. Every one of those relationships runs on documents crossing a company boundary.

The regulator already knows the documents are the weak point. Of the 303 drug and biologics warning letters FDA issued in fiscal 2025, 48 cited 21 CFR 211.84(d)(1) or (d)(2), the third most-cited current good manufacturing practice regulation that year. Those clauses are stricter than they are usually remembered: a manufacturer may lean on a supplier report of analysis only if it still runs at least one specific identity test itself and periodically validates that the supplier results are reliable. And the tooling in the gap is what you would fear. In a 2022 letter FDA recorded that a firm calculated active-ingredient assay results for validation lots on a non-validated spreadsheet whose formulas were printed at the time of calculation and never saved, so no electronic copy could be produced during the inspection.

Set that beside what the spreadsheet literature has known for two decades. Across the seven studies since 1995 that reported how many spreadsheets they audited, 88 percent of the 113 examined contained errors. Individual reviewers find roughly half the errors seeded into a document. And in one experiment the people who had just built a spreadsheet put the chance it contained an error at 18 percent on average, against an actual 86 percent. They were business students rather than professional developers, which makes the gap easier to dismiss and no less uncomfortable.

So the honest comparison is not a validated agent against a reliable human. It is a measured agent against an unmeasured spreadsheet, run by someone confident and wrong about how often it fails. Annex 22 does not obstruct that comparison. It demands somebody finally run it. The same logic drives how to tier AI governance by consequence, where the low tier exists precisely so that scarce oversight lands on the work that can hurt someone.

Build the shadow reader first. It needs a mailbox, a PDF parser, the specification already sitting in your ERP, and an agent that says these two things do not agree. It writes nothing to a validated system, so it needs no validation package. Run it beside the coordinator for a quarter and you end up with three numbers you have never had: how many certificates carry a discrepancy, how many the coordinator caught, and how many the agent caught that the coordinator did not.

Only then do the return-on-investment arithmetic, and do it the way O2 did: transactions per week, minutes per transaction, error rate, and a threshold below which you do not bother. We have deliberately not given a headline hours-saved figure, because every number we could find for certificate processing time traces back to a vendor selling certificate processing software. That the industry cannot say what this costs is not a gap in the argument. It is the argument.

The uncomfortable part, and our reading rather than anyone official position, is that pharma caution about AI in manufacturing has never really been caution about error rates. It has been caution about attributable error rates. An unvalidated spreadsheet fails invisibly and nobody is named. A model fails visibly, with a log, a confidence score and a version number. Annex 22 asks for exactly that. It reads as a burden because it is a standard the incumbent process has never once had to meet. The companion question, how you qualify a system that does not give the same answer twice, is the subject of validating a non-deterministic agent under GxP.

Shadow mode is cheap to describe and awkward to staff. Somebody has to sit the agent beside a real coordinator, agree what counts as a disagreement before the first one arrives, keep the log in a form a quality function will later accept as test data, and resist the pull to let it start writing the moment it looks good. That is a measurement exercise wearing the clothes of a software project, and it fails for organisational reasons far more often than technical ones.

That is the work we do with companies. Our data and AI strategy engagements start with where the leverage is and what to build first, which in a regulated supply chain means deciding which handoffs are worth measuring before anything is automated. From there it is custom AI workflows on proprietary data and bespoke development from proof-of-concept to production. Our in vivo augmentation and portfolio monitoring case studies are both work that had to survive scrutiny from an industrial or academic partner rather than simply produce an answer, which is the same test a quality function will apply here.

The pattern is not specific to incoming goods. Any regulated process built on documents crossing a boundary has the same shape: a high-volume clerical layer nobody has measured, a signature that has to stay with a person, and a case for automation that cannot be made until the baseline exists. We applied the same reasoning to adverse event intake in what AI-native pharmacovigilance would actually look like, and the constraint there was identical.