Why AI Cannot Be Recalled Like a Drug—and How to Govern It

A defective medicine can be removed from pharmacies and distribution channels, although no recall can undo doses already taken. AI has a different control problem: a model, checkpoint, fine-tune, or derivative can be copied, modified, and redeployed anywhere software can run. A provider may disable a cloud endpoint, but it cannot reliably retrieve every downloaded copy or erase capabilities learned by another model.

That does not make AI governance impossible. It means the familiar “test, launch, monitor, and recall” model is only one layer. Effective governance must also control who receives a model, what tools a deployed system can use, how downstream copies are tracked, when humans must approve an action, and how an organization contains harm when rollback is incomplete.

The drug-recall analogy is useful—but limited

Medicines and AI systems both require evidence, staged evaluation, post-deployment monitoring, incident reporting, and accountable organizations. Those similarities make pharmaceutical regulation a useful reference. The analogy breaks down when it assumes that digital products behave like physical inventory.

Control question	Physical medicine	Cloud AI service	Downloadable model
Where copies come from	Manufacturing and distribution facilities	Provider-controlled infrastructure	Original publisher and unlimited downstream copies
Can new use be stopped centrally?	Distribution can be halted, with remaining stock recalled	Provider can restrict accounts, tools, regions, or the endpoint	Not for copies already downloaded
Can the product change after release?	Normally through a controlled manufacturing and approval process	Models, prompts, tools, policies, and routing can change quickly	Users can fine-tune, quantize, remove safeguards, or combine components
Main checkpoints	Trials, authorization, factories, pharmacies, clinicians	Training, deployment, API access, identity, monitoring	Pre-release evaluation, hosting sites, hardware, licenses, downstream governance
What a recall cannot undo	Products already consumed and harm already caused	Prior outputs, copied data, and actions already taken	Existing replicas, derivatives, learned capabilities, and prior actions

The correct conclusion is not that drugs are fully recoverable while software is permanent. It is that the available levers differ. Governance should be designed around the actual release and deployment model.

Why AI control becomes difficult after release

1. Digital copies are cheap and exact

A model checkpoint is data. Once legitimately or illegitimately copied, it can be duplicated without the original publisher's infrastructure. A smaller open model may run on consumer hardware; a large model may require expensive accelerators, but the copying problem remains distinct from the cost of operating it.

Distribution also creates derivatives. A recipient can fine-tune a model, compress it, merge it with another checkpoint, wrap it in a new application, or train a separate model on its outputs. Hashes can identify an unchanged file, but they do not find every modified descendant.

2. Cloud control and open-weight control are different

A cloud-only provider retains meaningful levers: it can patch the service, revoke credentials, change rate limits, filter requests, block a tool, or retire a model. Those actions resemble a product withdrawal, but they still cannot reverse an email already sent, code already deployed, private data already exposed, or outputs already used for training.

With downloadable weights, the publisher can remove its own link and ask hosting services to cooperate, but independent copies may remain. For the highest-risk releases, access decisions and safeguards therefore matter before distribution.

3. Capability can move without model theft

In February 2026, Anthropic said it had identified campaigns involving about 24,000 fraudulent accounts and more than 16 million exchanges with Claude. The company alleged that DeepSeek, Moonshot, and MiniMax used the outputs to improve their own models through distillation, in violation of its terms and access restrictions. This is Anthropic's account of the incidents, not an independent finding presented here as settled fact.

The case illustrates a broader issue: access to a hosted model's outputs can transfer useful behavior even when the underlying weights never leave the provider. Defenses therefore need identity checks, abuse monitoring, rate and pattern analysis, and coordinated response—not only protection of model files.

4. AI systems are changing stacks, not static products

Risk depends on the whole system: model, system prompt, retrieval data, tools, permissions, user interface, human workflow, and connected services. Changing any layer can introduce a new failure mode even when the model name stays the same. A one-time approval cannot cover every later prompt, tool, connector, or model update.

Pre-release testing is necessary, not sufficient

Evaluation before deployment can reveal unsafe behavior, bias, poor reliability, security weaknesses, or misuse pathways. Staged rollouts reduce exposure and produce evidence from real workflows. Monitoring can detect incidents and performance drift. None of these controls should be abandoned.

The limitation is that tests sample possible behavior rather than exhaust every input and context. A system connected to email, source code, payments, or production infrastructure can also fail through the surrounding tools and permissions. Once an action leaves the system boundary, rolling back the model may not reverse the consequence.

Organizations should classify actions by reversibility:

• Easily reversible: drafting text in a private workspace, creating a disposable test file, or producing a recommendation that no one has acted on.
• Costly to reverse: editing a shared database, changing a customer record, or deploying code that can be rolled back but may interrupt service.
• Effectively irreversible: disclosing confidential data, sending a public message, approving a payment, rejecting a candidate, changing medical care, or deleting the only copy of information.

The less reversible the action, the stronger the identity, authorization, human review, logging, and fallback requirements should be. TipsMake's guide to AI data controls and privacy explains why sending information to a service can itself be an irreversible step.

A practical governance stack

No single framework covers every jurisdiction and use case. The NIST AI Risk Management Framework organizes voluntary risk work around Govern, Map, Measure, and Manage. Its generative AI profile adds considerations specific to generative systems. Organizations should also identify the laws and sector rules that apply where the system is developed and used.

Govern: assign decisions and accountability

• Name an accountable executive and operational owner for each deployed use case.
• Maintain an inventory of approved and unapproved “shadow AI” tools.
• Document the purpose, users, data, model, vendor, tools, and jurisdictions for each system.
• Define prohibited uses and who can accept residual risk.
• Require procurement, privacy, security, legal, and domain review in proportion to impact.

Map: understand the complete system

• Trace data into the model and outputs into downstream systems.
• Identify affected people, especially those who cannot easily challenge a decision.
• Map external dependencies, model updates, plugins, retrieval sources, and human handoffs.
• Mark every point where an output becomes public, triggers an action, or cannot be fully recalled.
• Document plausible misuse, prompt injection, credential theft, and model-extraction paths.

Measure: test the risks that matter

• Evaluate performance on representative and difficult cases, not only a polished demo set.
• Test security boundaries, unsafe requests, tool misuse, data leakage, and recovery behavior.
• Measure false positives, false negatives, calibration, subgroup performance, and human reliance where relevant.
• Record test limitations and risks that cannot yet be measured reliably.
• Repeat evaluation when a model, prompt, data source, tool, or workflow changes materially.

Manage: reduce exposure and prepare for failure

• Use least-privilege identities, tool allowlists, rate limits, and separate environments.
• Keep a human approval gate for high-impact or irreversible actions.
• Monitor logs and outcomes, with thresholds that pause the system automatically.
• Create notification, remediation, rollback, and compensation procedures for affected users.
• Preserve a tested manual fallback when the AI service is unavailable or suspended.

Readers planning a deployment can use TipsMake's security guides for general account and system hygiene and its AI guides for related implementation concepts.

Use CARE as a pre-mortem and exit-planning lens

The original CARE formulation—Catastrophize, Assess, Regulate, Exit—is one proposed management lens, not a universal standard or replacement for legal requirements. Its value is that it forces teams to plan for severe failure before launch.

1. Catastrophize: describe the worst plausible failures in concrete terms. Who could be harmed, what could escape the organization, and which consequences would be irreversible?
2. Assess: estimate likelihood, exposure, severity, detectability, and uncertainty. Separate evidence from assumptions and record disagreements.
3. Regulate: apply controls such as narrower scope, data minimization, access restrictions, evaluation thresholds, human approval, monitoring, and contractual obligations.
4. Exit: define stop triggers, authority, communication, credential revocation, data handling, rollback, replacement workflows, and support for affected users.

An exit plan should not say only “turn off the model.” It should answer what happens to queued actions, cached data, derivative models, connected tools, users who depend on the system, and decisions already made.

Global coordination still needs local controls

AI models and services cross borders, while laws, enforcement powers, infrastructure, and social priorities remain distributed. International standards can improve shared terminology, evaluation methods, incident reporting, and cooperation on severe cross-border risks. They cannot replace the responsibilities of the organizations deploying AI into specific workplaces and services.

The European Union's AI Act overview, for example, distinguishes between obligations for AI systems and rules for general-purpose models, with additional duties for general-purpose models considered to pose systemic risk. Other countries and sectors use different regulatory structures. A company must map the rules that actually apply rather than borrowing one headline analogy.

The real lesson of the recall problem

AI is not uniquely uncontrollable, and medicine is not perfectly recallable. The important difference is where control can still be exercised. For cloud systems, access and deployment controls remain powerful. For downloadable models, pre-release decisions and downstream stewardship carry more weight. For agents, tool permissions and human approval can matter more than the model alone.

Good governance combines evaluation, controlled release, least privilege, continuous monitoring, incident reporting, and a realistic exit plan. The goal is not to promise that every copy or consequence can be recovered. It is to prevent avoidable harm, detect problems early, limit their reach, and assign responsibility for the parts that cannot be undone.