Two weeks ago, I wrote about the independence problem in APRA’s April letter.
Much of the AI capability used by regulated organisations arrives inside a provider’s platform. Think about where your foundation model, training data, infrastructure and fourth-party dependencies sit right now.
That makes independent assessment difficult. You are expected to assure a system containing components you cannot adequately inspect.
Nearly every response you’ll get back when you ask your provders will not give you what you need. Things like contractual rights, vendor questionnaires, audit reports and testing performed against the service from the outside. Each of these has some value. Each also leaves you dependent on what the provider or its assessor tells you.
…But hardware attestation offers something different.
What confidential computing protects
A trusted execution environment, or TEE, creates a hardware-isolated area in which data and code can be processed.
Memory is protected from software outside the trusted environment, including privileged infrastructure components such as the host operating system and hypervisor. In a properly configured confidential-computing deployment, this reduces the ability of cloud administrators or a compromised host to inspect or alter the workload.
For AI, the assets inside that boundary can include training data, prompts, model weights and intermediate computations.
NVIDIA introduced GPU confidential computing with the H100. Its Blackwell architecture extends the design with TEE-I/O and protection across high-speed GPU interconnects. Older A100 GPUs do not have the same confidential-computing capability.
Isolation like this is a handy thing to have from a general standpoint. However, what is really useful from an assurance standpoint is the attestation aspect it gives you.
Why is this?
Attestation can be independently verified
When a confidential-computing environment starts, it can produce a cryptographically signed report containing measurements of the hardware, firmware and security configuration within the attested boundary.
A relying party can check that report before releasing data, model weights or encryption keys to the environment.
This changes the nature of your evidence.
A vendor questionnaire records what the vendor says it does.
An audit report records an assessor’s opinion about defined controls over a defined period.
A contract gives you rights that may or may not be exercised.
But attestation allows you to verify selected facts about the environment that is running now.
That does not make the entire AI system trustworthy…
Attestation does not tell you whether a model is accurate, fair or safe. It does not establish the provenance of the training data. Nor does it prove that the application’s logic is correct.
It proves something more specific: that the measurements presented by the environment match approved reference values and the attested components have started in the expected security state.
That is the key.
Attestation produces strong evidence about a small set of claims. Most other AI assurance mechanisms produce broader evidence with more human judgement involved.
The two should complement each other.
The performance cost depends on the workload
Confidential computing is often dismissed as too expensive or too slow. But evidence to date does not support such a simple conclusion.
A benchmark study of H100 and H200 GPUs found that throughput overhead varied by GPU, model size and sequence length. On an H100, the measured reduction was about 7 per cent for Llama 3.1 8B, roughly 4.5–5 per cent for Phi-3 14B, and close to zero for Llama 3.1 70B. Some latency measures were higher, particularly for smaller models.
This makes sense. Computation within the GPU is largely unaffected. Much of the additional cost comes from protecting data moving between the confidential CPU environment and the GPU. As the amount of computation grows relative to that transfer, the proportional overhead falls.
Blackwell is designed to reduce this cost further through TEE-I/O and protected NVLink communication. NVIDIA describes its throughput as close to unencrypted operation, although buyers should validate that claim against their own workload and deployment configuration.
There is no honest universal overhead figure. “Low single digits” may be reasonable for some large-model workloads. It should not be treated as a guaranteed property of confidential computing.
Where the additional control is justified
I would not recommend confidential computing for every enterprise AI workload. In many cases, the additional infrastructure and operational complexity would add little to the organisation’s actual risk position.
There are, however, several cases where it deserves serious consideration:
1. Sensitive data that must be protected from the infrastructure operator
Health records, financial information and other restricted data may require protection from privileged access within the hosting environment, not merely from other customers or external attackers.
Confidential computing provides a technical control against that threat. Contractual restrictions alone do not.
2. Collaboration between organisations that cannot disclose their data to each other
Two institutions may want to perform joint analysis or model training without either party handing its raw data to the other.
A TEE can provide a controlled environment for that computation. It does not solve every aspect of multi-party trust, but it can make arrangements possible that would otherwise be blocked by confidentiality requirements.
3. Models whose weights are valuable intellectual property
If an organisation has invested heavily in a proprietary model, unencrypted weights in accessible infrastructure create a concentrated exposure.
Protecting those weights while they are loaded and used may justify the added cost, particularly in shared or externally operated environments.
4. Assurance that an operator cannot inspect the workload
This is likely to become the most likely case for regulated institutions.
Where a provider claims that its administrators cannot access a workload, attestation can form part of the evidence supporting that claim. The assurance team still needs to examine what falls within the attested boundary, how reference measurements are approved, how keys are released and what remains outside the TEE.
Attestation is useful only when somebody verifies it and acts on the result.
What now?
I am not suggesting that every Australian organisation procure confidential-computing infrastructure this quarter. For most, it will not be the first constraint they need to solve.
What you need to ask, simply put, is:
Could the infrastructure supporting your material AI workloads use confidential computing if the risk required it?
If the workload runs on A100 GPUs, the answer is no. If it runs on H100, H200 or Blackwell infrastructure, the answer may be yes, subject to the surrounding CPU, firmware, drivers, cloud service and deployment configuration.
That is worth establishing before the next hardware refresh or cloud commitment.
Then look at the assurance file for one of your material AI systems. Separate the evidence that can be independently verified from the statements you have accepted from other parties.
Attestation will not replace testing, audit or supplier assurance. It can, however, give you direct, cryptographically verifiable evidence about parts of a system you do not operate.
For AI assurance, that is still unusually rare.
Sources