Python ops: Interface VPC Endpoint census — boto3 PrivateLink inventory

List Interface endpoints. Group by service. Flag Private DNS and single-AZ gaps.

<!-- hal:authoritative:yaml -->

A private subnet needs Secrets Manager without a NAT gateway. An Interface VPC Endpoint puts an ENI in the subnet. The census asks which endpoints exist, which services they cover, and whether Private DNS is on.

§I — Frame

Friday's Azure storage redundancy walk counted LRS and GRS copies. The EventBridge lesson before that counted buses, rules, and targets. Neither shirt is today's job.

Today the sprint track is Python against the AWS service family. The cert seat is AWS Solutions Architect Professional. The networking facet is Interface VPC Endpoints powered by PrivateLink. Transit Gateway (08-14) stitches VPCs together. Route53 failover (08-26) steers DNS health. PrivateLink keeps API traffic on the AWS backbone with private IPs inside your VPC.

Bootcamp privatelink.md draws the line cleanly: Gateway endpoints cover S3 and DynamoDB with prefix lists on route tables. Interface endpoints place ENIs in subnets, take security groups, and publish regional plus zonal DNS names. Private DNS can override the public service hostname so SDKs need no endpoint_url rewrite.

The ops job is a repeatable boto3 census: list Interface endpoints, group by service name, flag missing Private DNS, and report subnet and security-group coverage per AZ.

§II — Foundations: five facts for the census

Fact one. Interface endpoints are ENIs, not route-table prefixes.

describe_vpc_endpoints returns VpcEndpointType of Interface, Gateway, or GatewayLoadBalancer. Filter to Interface for PrivateLink consumer endpoints. Each Interface endpoint lists SubnetIds and Groups (security groups). Gateway endpoints list RouteTableIds instead. Mixing the types in one report blurs the blast radius.

Fact two. High availability is one ENI per AZ you care about.

Bootcamp guidance: deploy the endpoint into every AZ subnet that must consume the service. A single-AZ endpoint fails closed for callers in other AZs when that AZ is impaired. The census should emit len(SubnetIds) and the AZ set resolved from those subnets.

Fact three. Private DNS changes what the SDK dials.

When PrivateDnsEnabled is true, the regional service DNS (for example secretsmanager.us-east-1.amazonaws.com) resolves to the endpoint ENIs inside the VPC. When it is false, callers must use the endpoint-specific DNS names from DnsEntries. Most production private-only VPCs want Private DNS on for AWS managed services.

Fact four. Endpoint policies do not grant IAM.

An endpoint policy only restricts what crosses that endpoint. Identities still need IAM permission on the API. A census that prints PolicyDocument size or a parsed Effect summary helps reviewers spot an accidental Deny on secretsmanager:*.

Fact five. Service names are regional strings.

com.amazonaws.us-east-1.secretsmanager is not interchangeable with com.amazonaws.us-west-2.secretsmanager. Cross-region Interface endpoints for AWS managed services do not replace a multi-region design. The census keys on ServiceName plus VpcId.

§III — Worked census

import boto3
from collections import defaultdict

ec2 = boto3.client("ec2")
paginator = ec2.get_paginator("describe_vpc_endpoints")

rows = []
for page in paginator.paginate(
    Filters=[{"Name": "vpc-endpoint-type", "Values": ["Interface"]}]
):
    for ep in page.get("VpcEndpoints", []):
        rows.append({
            "id": ep["VpcEndpointId"],
            "vpc": ep["VpcId"],
            "service": ep["ServiceName"],
            "state": ep["State"],
            "private_dns": ep.get("PrivateDnsEnabled"),
            "subnets": list(ep.get("SubnetIds") or []),
            "sgs": [g["GroupId"] for g in ep.get("Groups") or []],
            "dns_count": len(ep.get("DnsEntries") or []),
        })

by_service = defaultdict(list)
for r in rows:
    by_service[r["service"]].append(r)

for service, items in sorted(by_service.items()):
    print(f"## {service} ({len(items)})")
    for r in items:
        az_hint = f"{len(r['subnets'])} subnet(s)"
        dns = "PrivateDNS=on" if r["private_dns"] else "PrivateDNS=off"
        print(
            f"- {r['id']} vpc={r['vpc']} state={r['state']} "
            f"{dns} {az_hint} sgs={len(r['sgs'])}"
        )

Run this under a read-only role that can ec2:DescribeVpcEndpoints and optionally ec2:DescribeSubnets if you resolve subnet IDs to AZ names in a second pass.

§III.B — Second pass: AZ coverage and gaps

Resolve subnet IDs once:

subnets = {}
ids = sorted({s for r in rows for s in r["subnets"]})
for i in range(0, len(ids), 100):
    chunk = ids[i : i + 100]
    resp = ec2.describe_subnets(SubnetIds=chunk)
    for sn in resp["Subnets"]:
        subnets[sn["SubnetId"]] = sn["AvailabilityZone"]

for r in rows:
    azs = sorted({subnets[s] for s in r["subnets"] if s in subnets})
    r["azs"] = azs
    if len(azs) < 2 and r["state"] == "available":
        print(
            f"WARN single-AZ endpoint {r['id']} "
            f"service={r['service']} azs={azs}"
        )

Single-AZ Interface endpoints are the usual silent DR gap in private VPCs. SAP scenarios love that trap: the diagram shows "PrivateLink," the exam answer wants multi-AZ ENIs.

§III.C — Security-group and DNS attribute checks

Pull the endpoint security groups and confirm ingress:

sg_ids = sorted({g for r in rows for g in r["sgs"]})
if sg_ids:
    sg_resp = ec2.describe_security_groups(GroupIds=sg_ids)
    for sg in sg_resp["SecurityGroups"]:
        for perm in sg.get("IpPermissions") or []:
            from_port = perm.get("FromPort")
            cidrs = [r.get("CidrIp") for r in perm.get("IpRanges") or []]
            if "0.0.0.0/0" in cidrs:
                print(f"WARN open SG {sg['GroupId']} port={from_port}")

Also verify the VPC can use Private DNS:

vpc_ids = sorted({r["vpc"] for r in rows})
attr = ec2.describe_vpc_attribute
for vpc in vpc_ids:
    hostnames = attr(VpcId=vpc, Attribute="enableDnsHostnames")
    support = attr(VpcId=vpc, Attribute="enableDnsSupport")
    h = hostnames["EnableDnsHostnames"]["Value"]
    s = support["EnableDnsSupport"]["Value"]
    if not (h and s):
        print(f"WARN VPC {vpc} DNS hostnames={h} support={s}")

Private DNS on an endpoint is useless when the VPC itself cannot resolve DNS hostnames.

§III.D — What the census refuses to do

Do not invent Gateway endpoint rows in the Interface report. Do not treat available as "Private DNS works" without checking PrivateDnsEnabled and VPC DNS attributes. Do not open security groups to 0.0.0.0/0 on the endpoint ENI "to make the demo pass." Restrict the endpoint SG to the worker SG on TCP 443.

Do not confuse this with Transit Gateway attachments. TGW moves packets between VPCs. Interface endpoints move packets to a service ENI or to an NLB-backed endpoint service in another account.

§IV — Tie to Cert, Dev, and Go

Cert lesson covers Gateway versus Interface, NLB-backed endpoint services, and policy semantics for SAP design questions. This Ops lesson is the inventory you run after Terraform or ClickOps claims the endpoints exist.

Dev lesson uses TypedDict with total=False so partial describe_vpc_endpoints shapes stay typed when optional keys (PolicyDocument, DnsEntries) are absent. The census dicts above are the motivating shape.

Go lesson walks the same API with AWS SDK for Go v2 paginators for fleet-side inventory binaries.

§V — Operator checklist

1. Filter vpc-endpoint-type=Interface before grouping.
2. Key rows by ServiceName + VpcId.
3. Flag PrivateDnsEnabled=false on AWS managed service endpoints.
4. Require ≥2 AZs (or every AZ the workload uses) for production.
5. Record security-group IDs; verify ingress is worker-SG scoped.
6. Confirm VPC DNS hostnames and DNS support are enabled when Private DNS is on.
7. Re-run after every networking PR that touches aws_vpc_endpoint.

§VI — Close

PrivateLink Interface endpoints are ENIs with a service name and a DNS story. The census makes the story visible: which services, which AZs, whether Private DNS is on, which SGs gate the ENI. Run it before you trust the architecture diagram.

Examine the WARN lines for single-AZ endpoints before you call the VPC private-only ready.

Related

• AWS SAP: Interface VPC Endpoints and PrivateLink
• Python TypedDict and total=False
• Go DescribeVpcEndpoints inventory
• Transit Gateway and RAM shares
• Cross-References/Archmagus-Stack — Earth-DevOps hub
• Grounding: Bootcamp privatelink.md Interface Endpoints; SAP notes VPC Endpoints Core Topic