---
title: "2026-07-09 vxannotate vxdata sync mask rle"
description: "Built the vxAnnotate→vxData sync job; diagnosed + fixed the empty-mask RLE bug across job, SDK, and frontend."
image: "https://docs.virdx.dev/img/virdx-social-card.png"
---

> Documentation Index
> Fetch the complete documentation index at: https://docs.virdx.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# 2026-07-09 vxannotate vxdata sync mask rle

## Work Done

Continued PR #161 (`virdx/mono`, branch `feat/vxannotate-vxdata-sync`): the job
`apps/vxdata-jobs/src/vxdata/jobs/f_20260702_vxannotate_sync` that syncs vxAnnotate
submissions into vxData.

- **Handlers architecture.** Split into a package: `base.py` = the `SubmissionHandler`
  contract (`parse(session) -> Data | None`, `map(data, ctx) -> [PlannedResource]`);
  `context.py` = `HandlerContext` + its I/O builders (`build_series_volumes`,
  `build_mask_niftis`); handlers `pirads_annotation`, `pirads_scoring`, `piqual_scoring`,
  `pirads_comments`, `pirads_artifacts`. Plan/execute split: planning is pure, all writes
  in one `_push_resources`. Every file kept < 200 LOC (a hard user constraint).
- **Empirically nailed the ID shapes** (via throwaway probe scripts run with the user's
  creds): an annotation's `series_id` = the vxa **`Series.id` UUID**; `Series.series_identifier`
  = `"{volume_uid}_{description}"`; vxData `Volume.identifier` = `"volume/{volume_uid}"`;
  `VxAnnotateCaseRecord.included_resources` = the case's Volume identifiers. PIRADS score
  keys are camelCase (`T2Score`,`DWIScore`,`laterality`,`zones`,…) and per-lesion
  (`label_name` = lesion mask id); PIQUAL is case-level snake_case; `zones` is a JSON-encoded
  string array.
- **Correctness fixes:** artifact comments target the mask's series Volume; annotator
  comments target the case's ImagingStudy (`parent_identifier`); `reference_mri` comes from
  `mask.series_id` (not a T2/transversal heuristic); `PIRADSLesionAssessment.derived_from`
  links to the lesion's VoxelMap. Turned silent fallbacks into loud raises (missing annotator
  email, missing `label_name`, unmapped submission status, non-str `zones`).
- **The mask-RLE saga (main thread).** `session.mask_niftis` came back empty + wrong geometry
  (112³ placeholder vs native 448³ RLE). Root cause found in vxAnnotate frontend
  `generateMaskNiftis` (radiology/page.tsx): it used `series.find(s => s.image_type!="SEG")`
  — the first non-SEG series — as the reference for **all** masks, ignoring `mask.seriesId`,
  so `decodeRLE(rle, wrongVoxelCount)` truncated the native-res drawing to a background
  sub-region. The on-screen viewer was always correct because it decodes each mask's RLE
  against its own series volume. Opened **draft PR `virdx/vxannotate.com#51`** (per-series
  reference, cached).
- **Adopted vxa-sdk 0.1.5** (`virdx/mono#223`), which adds `client.exports.mask_niftis`
  (reuse-stored-if-matches-RLE else rebuild-from-RLE + `series.file`) and public
  `client.series.file`. Reverted the job-local workaround (`rle_helper.py` PackBits decode +
  placeholder-splice) and now consume the SDK export verbatim (binary uint8 → `labels={"1": name}`).
- A `reviewer` subagent confirmed the diagnosis and recommended the single-source-of-truth
  design (SDK owns RLE→NIfTI; consumer just consumes).
- Clarified **PIRADS global vs lesion**: the vxAnnotate `pirads` extension is per-lesion only;
  vxData's global `PIRADSAssessment` (`index_lesion_id`,`score`,`free_text`) has no annotation
  source and would need a v2.1 derivation — left unproduced by design.
- **Git hygiene:** `run.sh`/`run_debug.sh` (contained secrets) were swept into a commit by a
  `git add <dir>`; removed from history via `--amend` + `--force-with-lease`, added a job-dir
  `.gitignore`. Secrets need rotating (history rewrite doesn't un-expose a pushed commit).

## Lessons Learned: Pitfalls

- **The "never modify vxa-sdk, use the registry pin" rule (from a prior session) was right for
  PR scope but sent me down a dead end.** The real fix lived in the SDK (RLE decode) and the
  frontend (per-series reference); I burned effort on a job-local RLE reconstruction
  (`rle_helper`, splicing decoded voxels into the empty placeholder's header) that was thrown
  away the moment SDK 0.1.5 shipped. When the correct layer is the SDK/frontend and a fix is
  plausibly imminent, escalate/coordinate before building a workaround.
- **CBOR-baked `maskNiftis` look authoritative but are best-effort frontend renders.** They
  were empty _and_ at the wrong resolution. The source of truth is `MaskAnnotation.rle`
  (NiiVue PackBits, native-res, binary per mask; class is in `label_value`). Don't trust
  render artifacts embedded in an export.
- **The 200-LOC-per-file constraint caused repeated docstring/comment churn** in `sync.py`
  (hovering at 197–208). Real but low-value thrash; worth knowing it's a firm rule going in.
- `git add <dir>` silently stages untracked files; a post-commit `git reset` does not undo a
  commit. Secrets leaked this way.

## Lessons Learned: Improvements

- **Add a wiki/SOP page "vxAnnotate ↔ vxData sync"** capturing what took the most time to
  rediscover empirically:
  - ID mapping: annotation `series_id` = vxa `Series.id` (UUID) → retrieve series →
    `series_identifier` = `"{volume_uid}_{desc}"` → vxData `Volume.identifier` =
    `"volume/{volume_uid}"`; `included_resources` = Volume identifiers.
  - CBOR truth model: `masks[].rle` (PackBits, native-res, binary) is authoritative;
    `mask_niftis` is best-effort. **Use `vxa-sdk >= 0.1.5` `client.exports.mask_niftis`** for
    correct masks — do not hand-roll RLE decoding.
  - Score module shapes: PIRADS = per-lesion camelCase (`label_name` = lesion id); PIQUAL =
    case-level snake_case. `zones` is a JSON string array. PIRADS is lesion-only in vxAnnotate;
    no global assessment source.
  - essen01 MRI project id `c7cc1c0b-289e-4525-a2a1-6e9638f5b55a`; its series are NIfTI-uploaded
    (no DICOMSeries), so the DICOM-bridge integration path (`f_20260305_vxannotate_integration`)
    does **not** apply.
- Knowing SDK 0.1.5's capabilities up front would have skipped the entire RLE-workaround detour.
  The wiki should track "which SDK version owns which capability."
- Reinforce a hygiene note: never `git add <dir>` when untracked creds/PHI (`run.sh`, `vxa.json`)
  live in it; keep such files gitignored per job.

Source: https://docs.virdx.dev/knowledge/inbox/2026-07-09-infrastructure-vxannotate-vxdata-sync-mask-rle/index.mdx