The AI model you approved vs the AI model that’s in production

So you know you approved an AI model going to production. And you assumed what’s running in production right now is the same thing you approved.

Question: how do you know?

Depending on how it was built, the thing you actually approved thinking it was just one thing… may have actually been five to eleven separate artifacts that need to be considered together.

Each of those artifacts could be modified by different teams. Yet you only got an approval request for a model name and model version thinking that represents everything.

Let’s explore this further.

Four ways your production can drift

1. Your vendor updates the model: Assuming that as part of your approval process, you approved a model name and version. What happens when your vendor ships a new version with your unknowing.
2. You team tweaked a prompt: Your product team may tweak the wording of a prompt through a config file. This results in behaviour change you didn’t know about.
3. An adapter gets swapped: The governance process for this sits outside your approval process.
4. A parameter was moved: A team member tweaks the temperature to increase creativity of answers, and now you have drift again.

Note that none of these involved an attacker, yet all of them resulted in drift.

Which leads me to a fifth way your production can drift: it was deliberately tampered with.

The solution

You need to have three things in place:

1. A full component list in your approval record, each with a hash identifier
2. Signatures produced at training that are verified at deployment, so you can prove it’s the same thing
3. Change detection for every component change, including the model itself

The implementation of these is relatively trivial: a hash in your approval record, a verification step in your deployment pipeline, monitoring the config locations you already control.

The organisational challenge will come with the fact that each of these belongs to different teams.

What should boards ask

If you’re on the board, here’s a good question to ask your teams:

“Who would notice, and how long until they did notice, if an AI system making decisions about our customers started behaving differently from the one we approved?”

In a board setting, this question essentially boils down to two things: detection and time. Par for the course for directors to ask about.

All my posts from this week led up to this moment

My previous posts this week covered a number of different areas:

• The things that make up a deployed model
• Three key attack paths
• Signing your models as an integrity control
• AIBOMs

Combining these gives you a solid claim for your assurance: the system running in production is the system that was assessed, approved and evidenced.

---

That’s the end of this week. For week 5, we’ll focus on deployment and inference: the serving layer, AI gateways and inference-time attacks.