---
title: "Deployment"
image: "https://docs.virdx.dev/img/virdx-social-card.png"
---

> Documentation Index
> Fetch the complete documentation index at: https://docs.virdx.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Deployment

There are two deployments of vxData on our Kubernetes cluster:

- **production** — available at [https://data.fra.virdx.dev/](https://data.fra.virdx.dev/).
- **staging** — available at [https://staging.fra.virdx.dev/vxdata](https://staging.fra.virdx.dev/vxdata).

Additionally, you can run a **dev** deployment locally on your machine via `process-compose`. See below for more.

At its core a deployment is a frontend/edge, an API service, and a Postgres database. 
File (blob) data is not managed by this repo; it lives in our central Ceph and the API hands clients a target bucket to read/write against.

## Production (Kubernetes)

The production stack is defined by the flat manifests in
`infra_k8s/current/vxdata-production/` (moved out of this repo) and synced by
ArgoCD. Two `Application` manifests in `infra_k8s/current/argo-cd/` drive the
namespace:

- `vxdata-production.yaml` — the manifests in `infra_k8s/current/vxdata-production/`.
- `vxdata-production-db.yaml` — the Bitnami `postgresql` chart (Postgres
  StatefulSet/Service/PVC). Its Service `vxdata-production-db-postgresql` is the
  `POSTGRES_HOST` in `configmap.yaml`.

Cluster contract:

- namespace: `vxdata-production`
- public URL: `https://data.fra.virdx.dev/`
- public IP: `10.10.0.104` (MetalLB LoadBalancer on the edge Service)

Components (`infra_k8s/current/vxdata-production/`):

- `api.yaml` — API Deployment + ClusterIP Service on port 8000.
- `edge.yaml` — nginx TLS terminator + MetalLB LoadBalancer for the public URL.
- `backup-cronjob.yaml` — nightly `pg_dump` to S3.
- `configmap.yaml` — non-secret env config, including the S3 `S3_ENDPOINT`/bucket
  for blob data.
- `ipa-ca-cert.yaml` — FreeIPA CA mounted for TLS trust against the internal S3
  endpoint. Secrets are created out-of-band in the cluster (not tracked in git).

**Production contains sensitive patient data and therefore must never be accessed by internet-connected LLMs or other public-facing processes.**

### Access control and the token migration

The API resolves every request to a principal via `API_PRINCIPALS_FILE`, which
points at `principals.json` from the `vxdata-production-principals` secret
(`vxdata-staging-principals` in `services-staging`), created out-of-band like
every other secret here. It is a single table of token to principal:

```json
{
  "tokens": {
"<secret-token>": {"subject": "read-bot", "grants": [...]},
"": {
  "subject": "anonymous",
  "grants": [
    {"action": "read",  "payload_types": ["*"], "licenses": ["*"], "access_levels": ["*"]},
    {"action": "write", "payload_types": ["*"], "licenses": ["*"], "access_levels": ["*"]}
  ]
}
  }
}
```

The empty-string token is the **transitional** entry: it is the principal a
request with no `Authorization` header resolves to, so callers on pre-token SDK
builds keep working unchanged. It is an ordinary entry in every other respect —
no special case in the code. A request carrying a token still gets that token's
policy, and a token that is present but unknown is still a 401; only the bare
request falls back. `""` can never collide with a real token, because
`HTTPBearer` reports no credentials at all for an empty header value.

Staging and production currently run with an anonymous master grant, which
means **access control is advisory there**: any caller can drop its token and
get full access. The API logs `anonymous_access_enabled` at startup as a
standing reminder.

#### Turning enforcement on

Every request logs `principal.subject`, and pre-token callers show up as
`anonymous`. Use that to find who still needs migrating:

```bash
kubectl -n vxdata-production logs deployment/api \
  | jq -r 'select(.["principal.subject"] == "token:anonymous")
       | [.["http.client_ip"], .["http.user_agent"], .["http.route"]] | @tsv' \
  | sort -u
```

Once that comes back empty, drop the empty-string token from the secret and
restart. `load_principals` runs once in the lifespan, so editing the secret
alone changes nothing until the pod restarts:

```bash
kubectl -n vxdata-production create secret generic vxdata-production-principals \
  --from-file=principals.json=./principals.json --dry-run=client -o yaml \
  | kubectl apply -f -
kubectl -n vxdata-production rollout restart deployment/api
```

The flip is instant and all-or-nothing: every remaining bare caller starts
getting 401s at once. Re-adding the key and restarting reverses it.

### Releasing a new image

The script at `apps/vxdata-api/deployments/build.sh` submits a Tekton buildkit pipeline to the Cluster.
The current version of the API, as defined in the `pyproject.toml` *in your current checkout*, will be included the build request:
The image will be built to `zot.fra.virdx.dev:5000/vxdata-api:{VERSION}` and will use the current state of `main` on the remote!

Ergo: to release a new API version, merge all your changes into `main`, make sure to have bumped the version number in `pyproject.toml` by hand, and then run

```bash
bash deployments/build.sh                              # build the latest tag from main
```

Builds tagged with the same version number will be cached.

Once this image is uploaded to the registry, open a PR in `infra_k8s` that bumps the requested image version number for the deployed API image:

- In `current/vxdata-production/api.yaml`, specify the desired version for the production deployment
- In `current/staging-services/vxdata.yaml`, specify the desired version for the staging deployment.

Once that PR is merged, ArgoCD will deploy the new image within 5 minutes.

### Backups

The nightly CronJob (`0 3 * * *`) writes a `pg_dump` to
`s3://vxdata-production-backups/` and prunes the bucket itself: dumps older than
30 days are deleted on every run, except Sunday dumps (suffix `_dow0`) which are
kept indefinitely.

### Seeding from a backup

To seed a Postgres target from the latest S3 dump, `kubectl port-forward`
against `service/vxdata-production-db-postgresql` and pipe a dump through
`pg_restore --clean --if-exists`.

## Dev Deployment

The local dev deployment runs natively on the developer's machine via
`process-compose` — no Docker. It is defined by
`apps/vxdata-api/deployments/dev/process-compose.yaml` and reads
`apps/vxdata-api/deployments/dev/.env` and `.env.secrets` for ports and
credentials. The stack supervises Postgres, a local MinIO, and the API; all
runtime state (Postgres data, MinIO data, sockets, logs) lives under
`apps/vxdata-api/deployments/dev/.runtime/`. The API runs with uvicorn
`--reload`, so source edits hot-reload automatically.

Run from `apps/vxdata-api/`:

```bash
just deploy                                   # start the native Postgres + MinIO + API stack
just inject-mock                              # seed the running stack with mock data
just import-backup-file --path=./my.dump      # restore a local pg_dump into dev Postgres
just cleanup                                  # stop the stack and wipe .runtime
```

`just` is intentionally limited to local development. Production operations are
handled via the Kubernetes manifests and ArgoCD.

Source: https://docs.virdx.dev/documentation/vxdata-api/explainers/deployment/index.mdx