Hedronite · Go Lesson · Polyglot-Dev / Go · Sun 2026-09-06

client-go IRSA inventory — informers over annotated ServiceAccounts

SharedInformerFactory watches ServiceAccounts. The cache keeps eks.amazonaws.com/role-arn records typed and current.

Lesson Class: Go (client-go informers + EKS IRSA inventory)

Cloud Referent: Typed corev1.ServiceAccount annotation index on EKS

Paired Ops: EKS IRSA wire-up

Paired Dev: Python one-shot census

Paired Cert: CKS projected tokens

Grounding: KUR Ch.14 · Bootcamp EKS.md · client-go informers

Informer

Watch ServiceAccounts; WaitForCacheSync.

Index

Keep annotated non-default SAs only.

Join

Second informer on Pods for usage counts.

List once in Python. Watch always in Go.

*Python listed once. Go keeps a typed cache. An informer watches ServiceAccounts and maintains the set that carries eks.amazonaws.com/role-arn.*

§I — Frame

Ops defined IRSA. Dev wrote a one-shot Python census. Cert hardened tokens on the same ServiceAccount object. Go's job is the long-running inventory pattern: SharedInformerFactory, typed corev1.ServiceAccount, and event handlers that update an in-memory index.

Yesterday's Go lesson listed S3 backend objects with AWS SDK v2. Today stays on Kubernetes types. This is not a quiz and not terratest. It is client-go as the DevOps control-plane client.

§II — Module and client

module github.com/hedronite/irsa-inventory

go 1.22

require k8s.io/client-go v0.31.0

Bootstrap:

package main

import (
  "context"
  "fmt"
  "os"
  "time"

  corev1 "k8s.io/api/core/v1"
  "k8s.io/apimachinery/pkg/labels"
  "k8s.io/client-go/informers"
  "k8s.io/client-go/kubernetes"
  "k8s.io/client-go/rest"
  "k8s.io/client-go/tools/cache"
  "k8s.io/client-go/tools/clientcmd"
)

const irsaKey = "eks.amazonaws.com/role-arn"

func kubeConfig() (*rest.Config, error) {
  if cfg, err := rest.InClusterConfig(); err == nil {
    return cfg, nil
  }
  return clientcmd.BuildConfigFromFlags("", os.Getenv("KUBECONFIG"))
}

In-cluster for a Deployment; kubeconfig for local. Same split as the Python tool.

§III — Informer factory scoped to namespaces

For two namespaces:

func main() {
  cfg, err := kubeConfig()
  if err != nil {
    panic(err)
  }
  cs, err := kubernetes.NewForConfig(cfg)
  if err != nil {
    panic(err)
  }

  ctx := context.Background()
  factory := informers.NewSharedInformerFactoryWithOptions(
    cs,
    5*time.Minute,
    informers.WithNamespace("billing"),
  )
  // For multiple namespaces, prefer one factory per namespace
  // or a cluster-scoped factory with a filter predicate.

  saInformer := factory.Core().V1().ServiceAccounts().Informer()
  indexer := newIRSAIndex()

  saInformer.AddEventHandler(cache.ResourceEventHandlerFuncs{
    AddFunc: func(obj interface{}) {
      sa := obj.(*corev1.ServiceAccount)
      indexer.upsert(sa)
    },
    UpdateFunc: func(_, newObj interface{}) {
      sa := newObj.(*corev1.ServiceAccount)
      indexer.upsert(sa)
    },
    DeleteFunc: func(obj interface{}) {
      sa, ok := obj.(*corev1.ServiceAccount)
      if !ok {
        tomb, ok := obj.(cache.DeletedFinalStateUnknown)
        if !ok {
          return
        }
        sa, ok = tomb.Obj.(*corev1.ServiceAccount)
        if !ok {
          return
        }
      }
      indexer.remove(sa)
    },
  })

  factory.Start(ctx.Done())
  if !cache.WaitForCacheSync(ctx.Done(), saInformer.HasSynced) {
    panic("sync failed")
  }

  // periodic report
  t := time.NewTicker(30 * time.Second)
  defer t.Stop()
  for {
    select {
    case <-ctx.Done():
      return
    case <-t.C:
      indexer.report()
    }
  }
}

Resync period 5*time.Minute is a safety net. Event handlers keep the index fresh between resyncs.

§IV — Typed index

type record struct {
  Namespace string
  Name      string
  RoleARN   string
}

type irsaIndex struct {
  byKey map[string]record // namespace/name -> record
}

func newIRSAIndex() *irsaIndex {
  return &irsaIndex{byKey: map[string]record{}}
}

func keyOf(sa *corev1.ServiceAccount) string {
  return sa.Namespace + "/" + sa.Name
}

func (x *irsaIndex) upsert(sa *corev1.ServiceAccount) {
  if sa.Name == "default" {
    x.remove(sa)
    return
  }
  arn := ""
  if sa.Annotations != nil {
    arn = sa.Annotations[irsaKey]
  }
  if arn == "" {
    x.remove(sa)
    return
  }
  x.byKey[keyOf(sa)] = record{
    Namespace: sa.Namespace,
    Name:      sa.Name,
    RoleARN:   arn,
  }
}

func (x *irsaIndex) remove(sa *corev1.ServiceAccount) {
  delete(x.byKey, keyOf(sa))
}

func (x *irsaIndex) report() {
  fmt.Printf("irsa_annotated_count=%d\n", len(x.byKey))
  for _, r := range x.byKey {
    fmt.Printf("%s/%s -> %s\n", r.Namespace, r.Name, r.RoleARN)
  }
}

Only annotated non-default ServiceAccounts stay in the map. Deletion and annotation removal both drop the key.

§V — Join Pods with a second informer

Mirror the Python join without re-listing every thirty seconds:

podInformer := factory.Core().V1().Pods().Informer()
podInformer.AddEventHandler(cache.ResourceEventHandlerFuncs{
  AddFunc:    func(obj interface{}) { /* bump usage */ },
  UpdateFunc: func(_, n interface{}) { /* bump usage */ },
  DeleteFunc: func(obj interface{}) { /* decrement */ },
})

Keep a map[string]int keyed by namespace/saName. On Pod add/update, read pod.Spec.ServiceAccountName (default to "default" if empty). Skip terminal phases. On delete, decrement carefully (floor at zero).

A lister after sync also works for a one-shot dump:

saLister := factory.Core().V1().ServiceAccounts().Lister()
list, err := saLister.ServiceAccounts("billing").List(labels.Everything())

Informers shine when the process stays up: GitOps may annotate a ServiceAccount minutes after Pod create; the handler fires without a full list storm.

§V.B — Full usage counters

type usageIndex struct {
  counts map[string]int // namespace/sa -> running pods
}

func saKey(ns, name string) string { return ns + "/" + name }

func (u *usageIndex) applyPod(pod *corev1.Pod, delta int) {
  if pod.Status.Phase == corev1.PodSucceeded || pod.Status.Phase == corev1.PodFailed {
    return
  }
  name := pod.Spec.ServiceAccountName
  if name == "" {
    name = "default"
  }
  k := saKey(pod.Namespace, name)
  u.counts[k] += delta
  if u.counts[k] <= 0 {
    delete(u.counts, k)
  }
}

func (u *usageIndex) onAdd(obj interface{}) {
  pod, ok := obj.(*corev1.Pod)
  if !ok {
    return
  }
  u.applyPod(pod, +1)
}

func (u *usageIndex) onDelete(obj interface{}) {
  pod, ok := obj.(*corev1.Pod)
  if !ok {
    if tomb, ok := obj.(cache.DeletedFinalStateUnknown); ok {
      pod, _ = tomb.Obj.(*corev1.Pod)
    }
  }
  if pod == nil {
    return
  }
  u.applyPod(pod, -1)
}

On Update, the blunt approach is onDelete(old); onAdd(new) when ServiceAccountName or phase changes. That avoids double-counting across resyncs if you also rebuild from the lister periodically.

Unused IRSA report:

func reportUnused(irsa *irsaIndex, usage *usageIndex) {
  for k, rec := range irsa.byKey {
    if usage.counts[k] == 0 {
      fmt.Printf("unused_irsa %s/%s arn=%s\n", rec.Namespace, rec.Name, rec.RoleARN)
    }
  }
}

§V.C — RBAC manifest for the inventory controller

apiVersion: v1
kind: ServiceAccount
metadata:
  name: irsa-inventory
  namespace: platform-observe
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: irsa-inventory
rules:
- apiGroups: [""]
  resources: ["serviceaccounts", "pods"]
  verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: irsa-inventory
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: irsa-inventory
subjects:
- kind: ServiceAccount
  name: irsa-inventory
  namespace: platform-observe

Cluster scope is justified only if the platform team owns cross-namespace inventory. Otherwise use Role + RoleBinding per namespace and pass those namespaces into the factory options. Cert's least-privilege habit applies to the inventory tool itself.

§VI.B — Contrast with yesterday's AWS SDK lesson

09-05 Go listed S3 objects with config.LoadDefaultConfig and s3.ListObjectsV2. That binary needed AWS credentials (often IRSA when run in-cluster). Today's binary needs Kubernetes credentials and should usually avoid IRSA so a bug cannot also read the invoice bucket. Split the tools: inventory watches the API; a separate canary Pod with IRSA proves STS.

If you deliberately combine them, the canary should be a Job with the annotated workload SA, not the inventory controller SA.

§VI.C — Local test with envtest or fake client

For unit tests without a cluster, k8s.io/client-go/kubernetes/fake plus a simple reactor is enough to assert upsert/remove logic. Feed a ServiceAccount with the annotation, call indexer.upsert, assert map length 1, clear the annotation via update path, assert length 0. Keep informer integration tests behind a build tag that CI runs only when a kind cluster is present.

§VI — Operator packaging

Build a small static binary, drop it in Distroless or scratch variants your platform allows, run as a Deployment with RBAC list,watch on ServiceAccounts and Pods. Give it a ServiceAccount without IRSA unless it must call AWS. Reading Kubernetes objects does not require eks.amazonaws.com/role-arn.

Expose metrics if you already run Prometheus in-cluster:

irsa_serviceaccounts gauge (count)
irsa_pods_on_default gauge
irsa_unused_serviceaccounts gauge

That turns today's inventory into a continuous control, which is what Maghrib labs can point at later without inventing a fourth quiz for Go.

§VII — Failure modes specific to client-go

Forgot WaitForCacheSync. First report is empty; you page yourself. Always wait.

Wrong namespace option. Factory scoped to billing will never see payments. Multi-namespace controllers either use cluster scope + filter or one factory each.

Asserting types without tombstone handling. Deletes often arrive as DeletedFinalStateUnknown. The handler above unwraps that case.

Treating informers as IAM truth. The cache shows Kubernetes intent. STS and IAM trust remain AWS-side. Pair with Dev's optional GetRole hop when you need both planes.

§VIII — Close

client-go gives you typed ServiceAccounts and a cache that tracks IRSA annotations as they change. Python remains the sharp one-shot census. Go remains the always-on inventory. Same annotation key, same least-privilege RBAC story as Cert, same Ops wire-up underneath.

Examine the informer sync before you trust the first gauge reading.

EKS IRSA Ops
Python IRSA annotation census
Prior Go AWS SDK inventory
Grounding: KUR Ch.14 Service Account Management; Bootcamp EKS.md IRSA; client-go informer patterns