Your Azure OpenAI private endpoint may not be carrying your traffic

A regional healthcare group’s architecture said its Azure OpenAI traffic used private endpoints, and its audit evidence relied on that. In practice every call from its application networks resolved to a public IP and went through the public endpoint. A private endpoint only carries traffic if DNS sends clients to it, and two separate DNS problems meant it did not.

Why “the private endpoint exists” proves very little

A private endpoint is a network interface with a private IP in your subnet. Clients don’t connect to that IP directly. They use the service’s normal name, such as <account>.openai.azure.com, and rely on DNS to return the private address.

Public DNS answers that name with a CNAME to <account>.privatelink.openai.azure.com. A private DNS zone called privatelink.openai.azure.com, linked to your virtual network, is meant to answer the second name with the endpoint’s private IP. Only Azure’s own resolver can see that zone. If your clients’ resolver never asks Azure, it follows the public CNAME chain and returns a public IP. (If Azure is asked but the zone lacks a matching record, it returns NXDOMAIN by default, which at least fails visibly.)

In this estate the public path still worked, so nothing failed. Public network access was still enabled on the OpenAI resources, with a virtual network service-endpoint rule allowing the application subnets. Service-endpoint traffic stays on Microsoft’s backbone, but it reaches the public endpoint and depends on the public-access firewall, which is not the isolation the design described. The only symptom was a DNS answer nobody was checking.

Problem 1: records named after the endpoint, not the account

The private DNS zone had an A record for each environment, but each record was named after the private endpoint resource instead of the OpenAI account. Clients query the account name, never the endpoint’s resource name. The records pointed at the right IPs, but no client lookup could ever match them.

Neither endpoint had a working privateDnsZoneGroup. Development had none at all. Production had a zone group with an empty zone list, which looks fine at a glance and does nothing. Without a zone group, Azure doesn’t create or maintain the A record, so the records had been added by hand under the wrong name. Any hand-managed record carries this risk: it must match the resource’s name and track the endpoint’s IP.

Problem 2: the resolver never asked Azure

Correcting the record was not enough. The virtual networks used custom DNS servers (domain controllers reached over peering) that, as far as we could tell, had no conditional forwarder for the OpenAI zones. They resolved the name from public root hints and returned a public IP.

The Azure platform resolver at 168.63.129.16 is the only route into the private zone, and it is only reachable from inside an Azure virtual network. A DNS server running in Azure can forward to it directly. A server on-premises can’t reach that address at all. It needs to forward to an Azure DNS Private Resolver inbound endpoint, or to a forwarder running in Azure.

Linux VMs had one more catch. systemd-resolved uses one upstream server at a time and only moves to the next one if the current server fails. The domain controllers returned a valid public answer, so it never tried anything else. Our first per-VM routing rule matched ~privatelink.openai.azure.com, and it didn’t work. systemd-resolved picks a route based on the name in the original query, which ends in .openai.azure.com. It doesn’t look at the CNAME targets, because the upstream recursive resolver follows those on its own. Widening the match to ~openai.azure.com fixed it.

How to check yours

Run the lookup from inside the virtual network, on a VM or container that actually calls the service. A lookup from your laptop shows nothing about what workloads see.

# Linux
dig +short <account>.openai.azure.com
getent hosts <account>.openai.azure.com

# Windows
Resolve-DnsName <account>.openai.azure.com

A correct answer is short: a CNAME to <account>.privatelink.openai.azure.com, then an address in your private range. A broken answer usually continues past the privatelink name through regional API and Traffic Manager names and ends at a public IP. If you see that, check three things in order:

• Connection state. Make sure the private endpoint connection is approved.
• Zone records. Confirm each A record in the privatelink zone is named after the account, not the endpoint. Check that each endpoint has a zone group and that the group actually lists a zone.
• Resolver chain. Find out which DNS servers the virtual network hands out. Check whether they forward openai.azure.com (Microsoft’s recommended forwarder zone) to Azure DNS, and confirm the private zone is linked to the client’s virtual network.

Repeat the check for every other private endpoint. Key Vault, Storage and SQL endpoints use the same DNS chain and fail the same way.

The fix

We made the changes in development, verified them, then repeated them in production:

1. Attach a privateDnsZoneGroup to each private endpoint, referencing the privatelink.openai.azure.com zone. Azure then creates the correctly named record and keeps it in step with the endpoint’s IP. In production we had to delete the empty zone group first.
2. Delete the misnamed manual records.
3. Append 168.63.129.16 to the virtual network’s DNS server list. The client’s policy was append-only, so internal DNS stayed first.
4. Route the OpenAI names to Azure DNS on each VM with a narrow systemd-resolved drop-in:

# /etc/systemd/resolved.conf.d/10-azure-privatelink.conf
[Resolve]
DNS=168.63.129.16
Domains=~openai.azure.com

Queries under openai.azure.com now go only to Azure DNS. One side effect is worth knowing about: systemd-resolved also treats a global DNS server as a default route, so other names are sent to it in parallel with the per-link servers, and the first successful answer wins. Names that only the domain controllers know still resolve correctly, because Azure DNS returns no answer for them. Public names may be answered by either. If you need strict ordering, put the forwarding on the DNS servers instead.

After the change, the account name resolved to the endpoint’s private IP in both environments, and TCP 443 to that IP succeeded.

The per-VM drop-in is a stopgap, and the change record says so. The durable fix is conditional forwarders on the internal DNS servers or an Azure DNS Private Resolver, plus an Azure Policy guardrail requiring a zone group on every new private endpoint. Once every legitimate client is confirmed on the private path, disable public network access on the resource so the public endpoint is closed for good.

No downtime, still under change control

These changes only alter which IP a name resolves to, and the service was reachable on both paths, so nothing was interrupted. We still ran it as a production change: a formal change notice, development first, a before-and-after record of each step, and a written rollback for each. That record lets an auditor confirm the fix rather than take our word for it.

How we can help

If your architecture says your AI or data traffic is private, the DNS answer from inside the network is the evidence that settles it. We run this check across all of a subscription’s private endpoints and fix what we find under your change process. The healthcare SIEM case study shows how this fit into a wider audit-readiness engagement.