---
title: "2026-09-01 inverse ve2e PI CAI data architecture investigation"
description: "Traced the PI-CAI inference data path and identified an immutable benchmark-artifact route that preserves canonical preprocessing."
image: "https://docs.virdx.dev/img/virdx-social-card.png"
---

> Documentation Index
> Fetch the complete documentation index at: https://docs.virdx.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# 2026-09-01 inverse ve2e PI CAI data architecture investigation

## Work Done

- Performed a read-only architecture investigation of `virdx/ve2e` branch `feat/isup-inference`.
- Traced `src/ve2e/inference_api/picai/data.py` through its 24-step vxData pipeline, materialization, Lightning data module, clinical dataset, and canonical resample/crop/pad/z-score preprocessing path.
- Searched source, scripts, tests, and configs for inference, artifact loading, dataset dumping, materialization, and preprocessing seams.
- Found that model input preparation, benchmark cohort selection, pathology target construction, expert-mask selection, split routing, and training-table construction are mixed in one 254-line module.
- Recommended keeping `PICAIInference` array-only, moving benchmark cohort and target creation out of the general inference API, freezing native-data benchmark manifests as immutable vxData artifacts, and loading them through the existing materialization and clinical dataset preprocessing path.
- Identified the current shared canonical preprocessing seam in `clinical_dataset._preprocess_vols`, which is already used by both eager and on-demand clinical dataset loading.
- Made no changes to the `ve2e` checkout.

## Lessons Learned: Pitfalls

- The initial workstream inference was too broad. The repository README calls ve2e "the inverse model", so the inverse workstream index should be read before product or infrastructure context.
- Repository-wide searches can be noisy because this branch includes a large restructuring. Narrow symbol and AST-based traces were more useful than branch-wide diffs.
- The current inference-data module is not covered by tests and the sole benchmark caller passes the sample-loader arguments in the opposite positional order. Static API tracing is therefore important before assuming the branch path has run successfully.

## Lessons Learned: Improvements

- The inverse workstream would benefit from a short SOP that records the canonical native-MRI preprocessing boundary and distinguishes immutable training/benchmark artifacts from online inference inputs.
- A documented benchmark-manifest contract should define cohort immutability, expert-mask provenance, target provenance, and allowed split semantics without overloading the training table builder.
- The public inference API documentation should state that `PICAIInference` consumes model-ready arrays only and that data acquisition and benchmark targets belong to separate adapters.

Source: https://docs.virdx.dev/knowledge/inbox/2026-09-01-inverse-ve2e-picai-data-architecture-investigation/index.mdx