Case 01 Shipped
Private DNS for a PrivateLink‑only data platform
The platform rule said no public control-plane endpoints. I built the private DNS that makes the rule work in every region and account, traced a missing return route, and cut the account allowlist to entries with owners.
Stack: Route 53 private hosted zones · AWS PrivateLink · AWS Cloud WAN · Databricks · OpenTofu · Terragrunt · Atlantis
- 3.19M audit-log events checked against the account IP allowlist Measured
- 37 → 28 allowlist entries after the audit, each with a named owner Shipped
On this page
Context
The AI platform of a global mobility company runs on two clouds. Since May 2026 I have owned infrastructure workstreams on it, inside a small platform team, with everything in OpenTofu and Terragrunt and every change applied by Atlantis after review.
The platform has a rule: Databricks clusters reach the Databricks control plane over AWS PrivateLink, never over public endpoints.
Problem
PrivateLink puts the Databricks control plane behind VPC endpoints, but endpoints alone do nothing. A cluster reaches them only if the workspace hostnames resolve to the endpoint addresses inside its own VPC. Every VPC that runs or reaches a workspace needs that answer, including VPCs in other AWS accounts. Where the answer is missing, traffic fails or heads for a public address.
Constraints
- Several AWS accounts, and every region and environment of the platform.
- The corporate network reaches platform VPCs through AWS Cloud WAN, which a separate networking team owns.
- Every change is code: reviewed in a pull request, then planned and applied by Atlantis.
Options
For name resolution across accounts there were three usual answers: a private zone per VPC, which duplicates records and drifts; Route 53 Resolver rules that forward to a central resolver, which adds a hop and a component to run; or one set of private hosted zones, associated with every VPC across accounts. I chose the shared zones.
Engineering decisions
- Route 53 private hosted zones answer for the workspace endpoint names.
- Each zone is associated with every VPC that needs it, across accounts, in code.
- Allowlist entries on the account console need a named owner, or they go.
- Corporate
- Corporate network engineers and internal services
- Core network
- AWS Cloud WAN routing between corporate and platform networks
- Connected through VPC attachmentsPlatform VPCs
- Workspace VPC, region A cluster compute
- Workspace VPC, region B cluster compute
- More regions every region and environment
- Connected through cluster traffic stays in the VPCVPC endpoints
- Workspace REST API interface endpoint
- Secure cluster connectivity relay interface endpoint
- Connected through AWS PrivateLinkControl plane
- Databricks control plane operated by Databricks
- DNS (my part) Route 53 private hosted zones answer for the workspace names. Each zone is associated with the VPCs that need it, across accounts, so a name resolves to the private endpoint everywhere.
- Routing Cloud WAN carries corporate traffic to the platform VPCs. A networking team owns it; I traced a missing return route and raised the fix.
- Access The account console keeps an IP access list, and every entry has a named owner.
Implementation
A private hosted zone answers only inside the VPCs associated with it, and an association across accounts takes two steps: the zone’s account authorises it, then the VPC’s account accepts it. Both steps live in code, so a new VPC gets name resolution in the same change that creates it.
When the corporate network could not reach a workspace, I traced the failure to a missing return route in Cloud WAN and raised the fix with the networking team that owns it.
Validation
In each region and environment I checked that workspace hostnames resolve to private endpoint addresses from every associated VPC, and that clusters start through the private path.
Outcome
- ShippedPrivate DNS for PrivateLink-only Databricks access, in every region and environment, kept in code.
- ShippedA diagnosed Cloud WAN return-route gap, fixed by the team that owns the network.
- ShippedAn audit of the account IP allowlist against 3.19 million audit-log events. The list went from 37 entries to 28, each with a named owner, after nine stale entries came out.