In life sciences, the AI you can scale is the AI you can defend. The organizations that build governance into the platform, rather than into the approval step at the end, are the ones concretely running agents across regulated functions — discovery, clinical, manufacturing, regulatory, medical affairs, pharmacovigilance — while everyone else is still assembling evidence. That speed gap determines who captures market share and who stays stuck in pilot paralysis.

Executive takeaways

• Scaling AI in life sciences is a throughput problem. Bespoke review works for one or two use cases and breaks at thirty. Governance that lives in the project rather than the platform cannot scale a portfolio of solutions.

• The gate that stalls your release is internal. Use cases stall because the evidence a quality reviewer needs was never captured while the solution was being built.

• Three moves change the dynamics. Assign a tier at the ideation stage, name an accountable owner beside the tier, and write the change control plan before the first drift alert.

Why AI fails differently in Life Sciences

AI in life sciences does not fail the way it fails elsewhere. A recommendation engine that gets a suggestion wrong loses a click. A pharmacovigilance triage that misses a signal, a model influencing batch release, or a clinical algorithm shaping site selection each carries consequences for patients, product quality, and submissions that no accuracy metric captures. The scrutiny is earned.

What it costs is a self-inflicted hit to velocity. The model is rarely the hard part. A team builds something that works in a sandbox in weeks, then spends two or three quarters reconstructing — after the demo is built — the evidence that it works as intended without risk of drift: where every input came from, why the design suited the decision, who reviewed it, and where the intended use stops. None of that is optional here, and almost none of it was captured while the solution was being built.

Most organizations are moving portfolios of dozens of models into production, increasingly including agents, owned by teams who each believe their case is the exception. Reconstruction is survivable once or twice. At the thirtieth time it becomes the bottleneck. Hiring reviewers does not clear it, because the cost sits in the rebuilding rather than in the judgment.

None of this is going to slow down. FDA and EMA expectations, and the ICH guidance harmonizing them, have asked for data integrity, validated computerized systems, and a documented rationale for intended use for two decades. AI-specific rules are arriving on top, the EU AI Act among them, but none of them sets your pace. What has changed is the volume: you are no longer documenting a handful of static tools a year, but dozens of continuously evolving models.

Governance should be part of the road, not a toll booth at the end of it.

What good machine learning practice actually means

In 2021 the FDA, Health Canada, and the MHRA jointly published ten guiding principles for Good Machine Learning Practice (GMLP), originally written for the development of AI-enabled medical devices — one part of a much larger Life Sciences landscape where drugs and biologicals dominate. The agencies have since added companion guidance on transparency and on predetermined change control. The principles are short, non-binding, and applied in practice across GxP AI use cases, from R&D to manufacturing, medical affairs, and pharmacovigilance.

GMLP remains the most useful thing regulators are elaborating on governing AI in life sciences because it describes a discipline rather than a deliverable. It asks you to know your intended use, know where your data stops being representative, test against the populations the model will affect, judge the human-AI team rather than the model alone, and manage the model across its whole life. A triage model and an AI-enabled diagnostic sit under different regimes, but a reviewer asks nearly identical questions of both.

Evidence should be a by-product of the work. The moment it becomes a project of its own, the timeline is already lost.

Data you can defend

The data exists in more than one place, and it was never designed to be reused. A trial touches a CTMS and an EDC. Results live in a LIMS, regulatory information in a RIM, quality events in a QMS, and manufacturing signals in an MES. Add protocols, case narratives, and inspection reports, and most of a project goes into establishing where data came from and whether it can be trusted as the single source of truth.

The shape of the problem differs by company. Large pharma contends with twenty years of undocumented decisions buried inside overlapping systems. Biotech defends data it did not generate. A CRO proves lineage to a sponsor who will be inspected on it. Medtech has the hardest version, collecting from devices in the field as the population drifts from the one the product was validated on. Health systems hold the richest and most complex data of all, with the least freedom to move it.

Transforming clinical and lab data into standards like CDISC or OMOP CDM taught us that the value sits less in the mapping than in making it legible to whoever signs off. A reviewer or an inspector should see what was filtered, masked, and changed without reading code. Do that, and traceability is never reconstructed, because it already exists by design.

That work should pay off more than once. A team that builds a defensible data asset does not start from zero on the next project. A catalog of what each asset is, and what it may be reused for, prevents that. Anonymization for secondary use extends the asset into new questions, as does external data held to your own standard -PubMed, Open Targets, or a federated arrangement with a health system. 

Trust compounds: every governed use case makes the next one faster to approve, not just faster to build.

Govern early, not after the fact

Most organizations already have a risk framework. It usually lives in a document nobody opens until someone asks for evidence. A leader deploying an agent in medical affairs or manufacturing should know its tier of scrutiny, and what that means for the timeline, before writing the use case brief.

Tiering tells you how much oversight each use case needs, although not whether to apply it. Take four candidates: an agent drafting a first-pass answer to a medical information request; a forecast behind a launch supply commitment; an agent flagging a possible safety signal in pharmacovigilance; a model informing batch release. All four can affect patients, and all need monitoring so the risk is understood and minimized, with systematic human review of any doubtful outcome. 

For most of them, a human in the loop is required by regulation. What differs is the weight of that oversight. For instance, batch release demands rigorous, high-assurance control, while the medical information draft needs a reviewer who rewrites before anything goes out. Yet in most organizations all four queue behind the same review, which is how a low-risk, high-value use case arrives nine months late without any gain in safety.

The same brief should settle who owns the model a year from now, when its builder has changed teams, and where a vendor's responsibility stops. A vendor can qualify its software. Your intended use, acceptance criteria, and sign-off stay yours.

These answers need somewhere to live. A use case registry, or a control tower across the portfolio, turns tier, owner, gate, and status into something a leader sees at a glance rather than chases across inboxes. Without it, the RACI exists on paper and nowhere else. Build the trail as you go, and sign-off becomes a formality rather than a reconstruction.

Trust the model, the agent, and the version in production

Explainability is the settled part. Standard techniques like Shapley values and feature importance let a reviewer see why a model reached a prediction, and that belongs in the platform rather than bolted on afterward. It has a limit worth naming.

Likewise, a feature importance chart explains a prediction without justifying a decision.

Agents change the question. With a model you can ask why it made certain decisions. With an agent you must also constrain what it may do, and those boundaries belong in the platform versus the prompt. These boundaries extend to: ceilings on cost and spend, filtering on toxic output, mitigation for prompt injection when an agent reads documents it did not author, and an evaluation store scoring quality continuously so degradation surfaces as a number rather than a complaint.

Guardrails constrain the agent, but they are not sufficient on their own. In a regulated environment, the software is only half the equation. This is what GMLP is getting at when it asks you to evaluate the performance of the human-AI team rather than the model on its own. In a human-in-the-loop system, the true safeguard is the pair. A reviewer who can't realistically say no isn't a safeguard, and someone told to approve a hundred agent outputs a day is an oversight in name only.

This is why structured agents, constrained to a defined set of steps and tools, make an agentic use case defensible. A leader proposing an agent for pharmacovigilance triage or prior authorization has to say exactly what it can do, then show it did only that.

Then the problem underneath all of it is computerized system validation which assumes software changes when somebody decides to change it. Machine learning changes because the world does. Left alone, that tension resolves itself badly: a model frozen at the version that passed review two years ago, quietly degrading while every change-control record stays clean.The fix is a change plan agreed in advance. What change is in scope, what evidence it requires, whether revalidation, retraining, or suspension follows are settled in the SOPs rather than debated once the alert fires.

Written up front, a change plan turns drift into an expected event with a known procedure. Written afterward, it is a post-mortem.

For the leaders absorbing the friction of scaling AI

If you are the leader bringing an agentic AI use case into your organization, you may absorb the friction between ambition and governance personally.

Tier at ideation, name an owner, write the change plan early. Making those three stick is a procedural problem. New tiers, gates, and monitoring habits ask people to work differently than they did six months ago, and any mechanism that is not written into the SOP teams actually follow decays into a tool nobody opens.

None of those three moves requires a platform. All of them get considerably cheaper on one.

The use case you are championing should arrive at governance already carrying the answers: where the data came from, what tier it falls under, why the model or agent behaves as it does, and what happens when something changes after launch. That is what moves a leader from proposing AI to deploying it.

Dataiku supports your end-to-end governance journey. It provides a data catalog to govern access and track quality, built-in model evaluation, agent guardrails, and continuous operational monitoring — distinct capabilities connected through central process governance. Whether you integrate your own AI lifecycle or adopt our built-in GMLP framework, Dataiku keeps the evidence documented, the right people in the loop, and the system trustworthy.