The model used to live where you could see it. It sat in your subscription, behind your Private Link endpoints, guarded by conditional access and a Key Vault you actually controlled. Now someone in manufacturing wants the same model running on a box bolted to a factory floor in a jurisdiction you can’t fly to, and the retail team wants it on edge devices in 300 stores, and the sovereign-cloud people want it in a data center you are contractually forbidden from touching.
That’s what “customer-owned environment” means in Microsoft’s September 4 guidance: on-prem servers, edge and IoT devices, and third-party or sovereign clouds. Hardware you don’t physically control, running software you didn’t necessarily build, asking for your weights and your credentials. The guidance — and it is guidance, a reference pattern, not a GA product with an SLA — is built around one blunt idea. Don’t release anything sensitive until the other end proves it’s what it claims to be. That proof is attestation. Everything below is the order I’d actually build it in.
1. Write down what’s leaving the boundary
Before any of the clever cryptography, inventory the assets. Model weights are the obvious one, but the list that bites you at 2 a.m. is longer: training and fine-tuning datasets, the credentials the workload uses to call back into Azure, API keys for downstream services, and the prompts and system instructions that encode your business logic. Each of these is a separate release decision with a separate blast radius. A leaked weights file is a one-time loss of IP. A leaked long-lived credential is a persistent foothold into your tenant. Treat them differently or you’ll protect the wrong one.
2. Put the edge hardware under the Azure control plane with Arc
You cannot govern what you can’t see, and the edge is invisible by default. Azure Arc is how you drag on-prem and third-party machines into the control plane so that Policy, Defender for Cloud, and managed identity actually reach them. This is the unglamorous prerequisite everyone skips.
Once it’s Arc-connected, the machine gets a system-assigned managed identity and shows up as a first-class resource. That identity is the thing you’ll scope access to later — not a secret baked into a container.
3. Establish a hardware root of trust before you trust a single byte
Attestation is only as good as what’s rooted in silicon. That means TPM 2.0 doing measured boot: the firmware measures the bootloader, the bootloader measures the kernel, the kernel measures what it loads, and each measurement is extended into the TPM’s platform configuration registers. The TPM then signs a quote over those measurements with a key that never leaves the chip. If someone swapped the kernel or slipped a rootkit under your inference runtime, the measurements don’t match and the quote gives it away.
No TPM, no root of trust, no attestation. An edge box without a functioning TPM 2.0 is a box you treat as hostile. That’s not paranoia, that’s the whole point of the exercise.
4. Verify the chain with Azure Attestation before release
Here’s where people get the architecture backwards. The edge node does not get to decide it’s trustworthy. It submits its TPM quote and measurements to Microsoft Azure Attestation, which evaluates them against a policy you author and returns a signed attestation token only if they pass. Your release logic checks that token. The device never marks its own homework.
The gotcha that costs you a weekend: attestation policy is code, and stale policy is worse than no policy. The day your golden image’s boot measurements change — a legitimate kernel patch — every node fails attestation and your fleet goes dark. Version your expected measurements alongside your images and roll them together.
5. Decide if you actually need confidential compute
The guidance treats encrypted-memory confidential compute — AMD SEV-SNP is the named technology — as recommended, not universally mandatory. That’s the honest answer and you should hold vendors to it. SEV-SNP encrypts VM memory and extends attestation to the runtime, so the host operator can’t read your weights or your data out of RAM. If your threat model includes the people running the edge hardware — a third-party cloud, an untrusted colo, a partner’s data center — then “recommended” is functionally “required” and you should stop pretending otherwise. If the edge box sits in your own locked room and the threat is a stolen disk, measured boot plus disk encryption may be enough. Match the control to who you’re actually defending against.
6. Kill the long-lived secrets with workload identity federation
Shipping a client secret or certificate to an edge device is shipping a credential you will never be able to rotate in time. Workload identity federation fixes this: the edge workload presents a token it already has — from its Arc identity or an external OIDC issuer — and trades it for a short-lived Entra token. No stored secret to leak.
Narrow the subject. A wildcard here means any workload at that issuer can assume the identity, which defeats the purpose entirely.
7. Gate the release on the attestation token, scoped and time-boxed
“Releasing credentials” does not mean handing over a Key Vault reader role for a year. It means: the node attests, your release service validates the signed attestation token, and only then does it hand back a narrowly scoped, short-lived token — or a wrapped key to decrypt the weights that stops working in an hour. No attestation, no token. Attestation expired, token expired. The release is conditional, time-limited, and re-evaluated, not a one-off grant you forget about.
8. Enforce it continuously with Defender for Cloud and Policy
Attestation at boot tells you the node was clean at boot. It says nothing about hour nine. Defender for Cloud on the Arc-enabled fleet watches runtime integrity and flags drift; Azure Policy keeps the guardrails — required extensions, disallowed configurations — from quietly decaying. Wire the alerts to something a human reads. Build the whole chain and route its warnings to an unmonitored inbox and you’ve built expensive theatre.
If you only do one thing, do number seven: never let a credential or a weights file leave your boundary except in exchange for a valid, signed attestation token. Everything else is how you make that gate trustworthy. Skip the gate and the rest is decoration on a door you left open.
