Clinical-AI trust rests on three pillars, not two. Governance is the intent layer: policy, accountability, and oversight for how AI is used. Security is the protection layer: defending the model and data from attack and leakage at runtime. Provenance is the proof layer: verifiable, tamper-evident evidence of what a specific AI produced, on what input, and who signed it. Most trust advice covers the first two and stops, which leaves the question that actually settles a dispute, can you prove what the AI did, unanswered. The gap matters now that AI is everywhere in care: 81 percent of physicians use AI professionally,[2] and roughly two thirds of US hospitals on major EHRs run ambient AI.[3] Governance and security assume an evidentiary record they do not produce.
This guide defines the two-pillar model, shows where it breaks, defines the third pillar, compares all three, and lays out how a small organization sequences them. RankShieldMD provides the provenance pillar and attests what a model did without rendering the decision. It complements, and does not replace, your governance and security. See how it connects to clinical AI provenance and how it differs from choosing verifiable AI or AI governance.
The two-pillar model everyone repeats
The standard framing splits clinical-AI trust into governance and security, and both are genuinely necessary.
Open almost any current guide and you will find the same two categories. Governance is the intent-and-accountability layer: the policies that decide which AI is allowed, the roles that own the risk, the human-oversight requirements, and the risk assessments that keep it aligned. The NIST AI Risk Management Framework is the canonical structure here, organizing trustworthy AI around govern, map, measure, and manage rather than a list of prohibitions.[1] Security is the runtime-protection layer: defending the model and its data from adversarial attacks, prompt manipulation, model theft, and leakage while the system runs. Both are real, both are well served by vendors and frameworks, and a serious program needs each. The problem is not that the two-pillar model is wrong. It is that it is incomplete, and the missing piece happens to be the one that matters most when something goes wrong and a decision has to be defended.
Where the two-pillar model leaves a gap
Governance decides what should happen and security protects it, but neither proves what actually happened in a specific case.
Picture a real dispute. Months after an AI-assisted note, read, or order, a payer or a plaintiff asks the concrete question: did this output come from the model you claim, on the data it claims, and did a clinician review it. Governance can show you had a policy. Security can show the system was protected. Neither can show what the AI actually did in that specific instance, because neither produces a per-decision, verifiable record. That is the gap. It is invisible on a normal day and decisive on a bad one. And it is exactly the gap adversaries and auditors probe, because an unprovable decision is a deniable one. The two-pillar model implicitly assumes an evidentiary layer, that somewhere there is a trustworthy record of what happened, without specifying who produces it or how anyone verifies it. In practice, that assumed layer usually does not exist, or exists as a mutable log that cannot survive scrutiny.
Want the proof pillar your governance and security assume?
Request early access →The third pillar: verifiable provenance defined
Provenance is verifiable, tamper-evident evidence of what a specific AI produced, on what input, and who signed it, captured as the decision happens.
Provenance is the proof layer. For each consequential AI decision it seals a record that binds the model version, a one-way digest of the input, the output, a timestamp, and the verified identity of the reviewing clinician, made tamper-evident and ideally anchored to an external transparency log so a third party can confirm it was not rewritten. It is distinct from governance, which is about intent, and from security, which is about protection. Provenance is about what actually happened, expressed as evidence a reviewer can independently check rather than a claim they must accept. Crucially it does this without holding patient data: the record contains digests and identities, never the chart. That is what lets a governed, protected AI decision also be a provable one. RankShieldMD is a provenance system in exactly this sense. It attests decisions and never renders them, which keeps it non-device, and it never substitutes for the governance and security work around it.
Governance, security, and provenance compared
The three pillars answer three different questions, and a complete program needs all three.
The table makes the division of labor explicit. Read it as complementary layers, not competing options. A program with strong governance and security but no provenance is well run and unprovable. A program that adds provenance can finally demonstrate, not just assert, that its sanctioned and protected AI behaved as claimed.
| Pillar | Question it answers | Primary artifact | Who provides it |
|---|---|---|---|
| Governance | What should happen, and who is accountable | Policy, roles, risk assessment, oversight | Your organization, guided by NIST AI RMF and AMA toolkits |
| Security | How is the model and data protected | Runtime controls, access control, monitoring | Your vendors and platform, verified by you |
| Provenance | What did this specific AI actually do | Per-decision, tamper-evident, PHI-free record | A provenance layer such as RankShieldMD |
How the three pillars work together in a small organization
Govern to set intent, verify the security you inherit, then add provenance so the whole thing is provable, in that order.
For a small practice or vendor, the pillars sequence naturally. Governance comes first because it is inexpensive to begin and directs everything else: decide which AI is sanctioned, name who owns the risk, and set oversight, using the NIST framework and the AMA governance toolkit as scaffolding.[1] Security you largely inherit from your EHR, your AI vendors, and your cloud platform, so your task is to verify it and document the shared responsibility rather than build it from scratch. Provenance is where a small organization earns the most, because it is the pillar almost no one has and the one that defends you when a decision is questioned. Added last, it makes the sanctioned, protected AI produce evidence you can stand behind. The result is a posture that does not depend on your size: not the biggest compliance department, but the ability to prove, on demand, what your AI did. That is what turns clinical-AI trust from a claim into something checkable.
