---
title: "PHI-safe data handling in vxData"
description: "What engineers must know before profiling or exporting vxData content—which parts store PHI, architectural gaps, and the PHI-safe profiling recipe."
image: "https://docs.virdx.dev/img/virdx-social-card.png"
---

> Documentation Index
> Fetch the complete documentation index at: https://docs.virdx.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# PHI-safe data handling in vxData

## What deliberately holds PHI

**`DICOMFile` payloads store direct DICOM identifiers and PHI-adjacent fields** in `mono/packages/vxdata-schemas/src/vxdata/schemas/payloads.py`: `PatientID`, `PatientName`, `PatientBirthDate`, `AccessionNumber`, `ReferringPhysicianName`, `InstitutionName`, plus raw `file_metadata` dict. These fields exist by design so downstream processing jobs can correlate studies, but they leak PHI when queried or profiled naively.

Dataset anonymization status is encoded in the `license` field of each datasource definition in `mono/apps/vxdata-jobs/src/vxdata/jobs/source_2026_01_28_ingest_files/dataset_definition.py`. Example values: `"VIRDX-ANONYMIZED"`, `"VIRDX-PSEUDONYMIZED"`, `"CC-BY-4.0"`. **This field documents the source dataset's anonymization state but provides no runtime enforcement.**

## Architectural gaps (as of 2026-08)

- **No central PHI-safe projection/redaction layer** — there is no allowlist-based view or API wrapper that strips PHI fields from query results.
- **No k-anonymity or small-cell suppression guardrails** — aggregate queries can return counts ≤ 5 or unique-identifier distributions with no warning or blocking.
- **No bounded generic dataset profiler in the repo** — no utility that profiles schema/nulls/cardinality while refusing to output values or hashed identifiers.
- **No active anonymization pipeline** — `dicom-anonymizer` is not present in the current mono dependency tree; no local pipeline uses it (verified 2026-08).

These gaps mean **any ad-hoc profiling, export, or aggregate query can leak PHI unless the operator manually restricts output to PHI-safe summaries.**

## PHI-safe profiling recipe

Safe for sharing with external agents or in public tickets:

1. **Schema inspection**: column names, dtypes, nullability → use `/schemas` endpoint (`mono/apps/vxdata-api/src/vxdata/api/routes/meta.py`) or parquet preview (`/parquet` at `mono/apps/vxdata-api/src/vxdata/api/routes/storage.py`), both return schema + row counts.
2. **Null rates**: `COUNT(*)` vs `COUNT(column)` → value-free.
3. **Distinct counts**: `COUNT(DISTINCT column)` or Polars `.n_unique()` → numeric summary, no literal values.
4. **Candidate keys**: columns with `n_unique == n_rows` or high uniqueness ratios → structural, no values.
5. **String-shape summaries**: regex-based patterns like `A-9` (letter-digit) or `YYYY-MM-DD`, counts-of-counts only, never literal strings.
6. **Relationship overlap**: FK-like in-memory joins to compute overlap counts/ratios between columns → never output the actual keys.

**Never export:**

- Sample rows, literal values, or hashed values.
- Frequency distributions that include the literal value (ok: "342 values appear once", not ok: "PatientID=12345 appears 3 times").
- Small-cell counts (e.g., "2 patients in this subgroup") without manual review.

## Reference profiler script (local dev artifact)

A single-file PHI-safe dataset profiler exists at `~/work/tries/2026-04-01-data-integrator/vxdata_dataset_profiler.py` (verified 2026-08). It uses:

- uv script metadata (PEP 723) with Polars + fastexcel.
- `--input <dir/file> --output <dir>` CLI.
- Outputs: `manifest.json`, `tables/*.json` (per-table schema/nulls/distinct counts/candidate keys), `relationships.json` (FK overlap metrics, no values), `vxdata_payload_catalog.json`, `vxdata_mapping_suggestions.json`, `integration_handoff.json`, `SUMMARY.md`.
- **Safety invariant**: never writes sample rows, literal values, hashed values, or frequency distributions with identifiers — only counts-of-counts and structural signatures.

**This script lives in a personal `~/work/tries` directory and is not in any virdx repo.** Reference the technique (schema + nulls + distinct counts + string shapes + candidate keys + relationship overlap), not the path, when describing PHI-safe profiling to others.

## API utilities that already exist (mono, 2026-08)

- `mono/apps/vxdata-api/src/vxdata/api/routes/meta.py`:
  - `/datasources/detailed-stats` — datasource-level aggregate stats: counts, distributions, date ranges, scanner metadata.
  - `/schemas` — field definitions and active resource count for every payload type.
- `mono/apps/vxdata-api/src/vxdata/api/routes/storage.py`:
  - `/parquet` — parquet preview returning column schema, row counts, and a limited sample (default 100 rows). **This endpoint returns literal data rows; restrict to PHI-free tables or redact columns manually.**
- `mono/apps/vxdata-jobs/src/vxdata/jobs/f_20251215_dicoms/utils.py` + `organize_bamberg.py`: DICOM metadata extraction + lightweight cardinality/uniqueness diagnostics.
- `mono/apps/vxdata-api/src/vxdata/api/mock/mock_data.py`: tiny seed dataset for UI/dev smoke tests.

---

**History**: data-platform repo (archived 2026-05-27) migrated to mono with systematic renaming: `apps/api` → `apps/vxdata-api`, `apps/worker` → `apps/vxdata-jobs`, `packages/client` → `packages/vxdata-sdk`. The architectural PHI-safety gaps identified in 2026-04 remain unfixed as of 2026-08.

Source: https://docs.virdx.dev/knowledge/wiki/workstreams/infrastructure/sops/phi-safe-data-handling/index.mdx