The Vendor API Was Your Security Boundary. With Open Weights, You Inherit It.

When you call a hosted model behind an API, somebody else owns the hard problems. Content filtering, abuse detection, the physical security of the weights, the liability when it says something it shouldn’t — all of that sits on the vendor’s side of a key you rotate once a quarter. Pull a LLaMA, Mistral, or Gemma checkpoint onto your own GPUs and that boundary evaporates. The weights are now a multi-gigabyte blob in your storage account, served by your inference endpoint, governed by whatever RBAC you got around to configuring. The vendor’s security model was the product. Now it’s your backlog.

That’s the real content of the taxonomy Microsoft and NVIDIA published on October 16. The three-bucket framing — security (prompt injection, model extraction, weight exfiltration), safety (jailbreaks, unfiltered output), compliance (provenance, data residency, audit) — is fine, and it’s genuinely useful to have a cross-vendor vocabulary both companies will point customers at. But a taxonomy doesn’t stop anything. The part worth your afternoon is the translation into Azure control-plane primitives, because that’s where “model risk” stops being a slide and becomes a deny-assignment someone has to approve.

Start with the asset nobody treats as an asset: the weights. A fine-tuned 70B checkpoint encodes your training data and represents real money in GPU-hours. Treat it like a secret, not like a build artifact. That means a dedicated storage account with a customer-managed key in Key Vault, and — critically — no public endpoint. The blast radius of a misconfigured container here isn’t “a leaked file,” it’s your model, your fine-tuning data, and the inference pipeline that trusts it.

1
2
3
4
5
6
7
8
9
10

# Lock the weights store to private networking + CMK, dry-run first
az storage account update \
  --name stmodelweights \
  --resource-group rg-ai-platform \
  --public-network-access Disabled \
  --allow-blob-public-access false
​
az storage account private-endpoint-connection list \
  --account-name stmodelweights \
  --resource-group rg-ai-platform

Then stop trusting humans to do the above. Enforce it with Azure Policy at the management-group scope that holds your AI workloads, so a deny effect kicks in before a storage account with public access ever exists. A policy that denies publicNetworkAccess set to anything but Disabled on the resource group holding model artifacts is ten lines of Bicep and worth more than any whitepaper.

1
2
3
4
5
6
7
8

resource denyPublicWeights 'Microsoft.Authorization/policyAssignments@2022-06-01' = {
  name: 'deny-public-model-storage'
  properties: {
    policyDefinitionId: subscriptionResourceId('Microsoft.Authorization/policyDefinitions', 'denyStoragePublicAccess')
    scope: managementGroup().id
    enforcementMode: 'Default'
  }
}

RBAC is where most teams quietly fail. The identity that pulls weights at pod startup should not be the identity a data scientist uses to experiment, and neither should be a standing credential. Use a user-assigned managed identity scoped to Storage Blob Data Reader on exactly one container, nothing broader. If you’re running inference on AKS, Workload Identity Federation means no secret in the cluster at all — the pod federates to an Entra Workload ID and reads the blob. Deployment rights (who can push a new model version into production) belong to a separate custom role gated behind PIM. Reading a model and replacing a model are different verbs; give them different roles.

Network isolation completes the control plane. Private Link on the storage account, the inference endpoint on a private AKS cluster or a locked NSG, and egress controlled so the model host can’t phone home with whatever a prompt-injection payload told it to exfiltrate. Then Defender for Cloud and the resource logs give you the audit trail — not of what the model said, but of who touched the weights, which identity invoked the endpoint, and when an access pattern stops looking like your traffic.

Here’s the limitation the framework is honest enough to carry, and you should be too: none of this makes the model safe. Azure controls secure the deployment — the weights, the endpoint, the audit log. They do nothing about what the model generates once a legitimate caller sends a legitimate request. Prompt injection that stays inside your network, a jailbreak that produces toxic output, a model that confidently hallucinates into a downstream system — platform RBAC has no opinion on any of it. You still need content filtering and red-teaming at the application layer, and that’s your code, not a policy assignment.

The taxonomy didn’t give you anything you couldn’t build in Terraform last year. What it gives you is a shared noun to put in the risk register, so the next time someone wants to docker pull a 70B model onto an unmanaged VM, you have a document with two vendor logos on it that says: the key was never the boundary. You are.