Skip to content

Infrastructure SOPs

Standard operating procedures for infrastructure tasks.

Running things on the cluster

  • Executing jobs / images / runs on the cluster — pathways, build systems, run mechanisms, gotchas, and the Argo creator-label/Kyverno identity trap.
  • vxData production DB restore + cluster storage gotchas — Postgres restore-from-S3 runbook, the vxData production deployment contract, and the cluster-wide traps: topolvm-no-replication, ArgoCD self-heal, the Argo 128KB ARGO_TEMPLATE limit, nginx worker_processes, imagePullSecrets, registry mirrors vs pull-through caches, and bulk rclone migrations.
  • Coder workspace template — where the template lives, the gpu_enabled MIG toggle, and the ArgoCD exclusion that makes template changes a manual Coder update.
  • Keel auto-redeploy — how Keel is deployed (polling, no webhooks) and the annotations that wire a Deployment to a mutable tag.

vxData

  • vxData SDK + schema model — resource/payload model, the current SDK surface, the child_identifiers tree-navigation gotcha, and the apps/vxdata-api local dev loop.
  • vxData jobs ingestion patterns — writing an ingestion job in apps/vxdata-jobs: discovery, mapping, stable identifiers, linking, plus the polars and vxAnnotate data traps.
  • vxAnnotate ↔ vxData sync — ID mapping, why mask decoding belongs in vxa-sdk, and which job template to copy.
  • PHI-safe data handling — what deliberately holds PHI, the missing guardrails, and the PHI-safe profiling recipe. Read before profiling or exporting anything.

Apps and packaging

Navigation

Type to search…

↑↓ navigate↵ selectEsc close