---
title: "2026-08-28 ve2e ISUP cleanup compatible inference and temporary legacy adapter"
description: "Replayed ISUP inference onto cleanup-plus-unify and validated a removable old-checkpoint schema adapter on CUDA."
image: "https://docs.virdx.dev/img/virdx-social-card.png"
---

> Documentation Index
> Fetch the complete documentation index at: https://docs.virdx.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# 2026-08-28 ve2e ISUP cleanup compatible inference and temporary legacy adapter

## Work Done

- Rebuilt `feat/isup-inference` from `origin/merge-unify-vits-into-cleanup-vits` as a focused implementation using `ve2e.vxdata_pipeline`, typed cleanup configs, unified DinoFlex forward processing, PHI-safe logging, canonical split preservation, separate biopsy/RPE QWK, and mode-`0600` JSON output.
- Dropped the obsolete data-pipeline consolidation and Pixi/Docker compatibility commits because cleanup already supplies those facilities.
- Pushed focused commit `a9636cef` and built image `feat-isup-inference-a9636ce` with Tekton build `build-ve2e-k6stx`.
- Confirmed ClearML model `56d3c7efa29847eba0b250983c2c7abb` initially failed under cleanup because its embedded config references removed `ve2e.base_configs` classes.
- Added the explicitly temporary, volume-DinoFlex-only adapter `src/ve2e/legacy_checkpoint.py`. Native checkpoints remain on the normal load path; fallback occurs only for the removed module. Pushed commit `5205d25c` and built image `feat-isup-inference-5205d25` with Tekton build `build-ve2e-vqqc2`.
- Validated full deserialization, typed `TrainConfig` conversion, current DinoFlex reconstruction, EMA state-dict loading, and CUDA placement in Argo workflow `run-ve2e-legacy-model-check-f5gz8` on standard `rtx-6000`. The workflow succeeded.
- The first full-cohort attempt exposed a cohort-semantics bug: the exact-one-study guard rejected the whole cohort when any patient was ineligible. Replaced it with unbiased eligibility filtering in commit `66f29d58` and built image `feat-isup-inference-66f29d5` with Tekton build `build-ve2e-577h8`.
- Full non-training evaluation succeeded in Argo workflow `run-ve2e-isup-full-eligible-bdf7p`: 640 eligible patients, biopsy QWK `0.1523730126351761`, and RPE QWK `0.08081882876638757`.
- Verified the result artifact was valid JSON with mode `0600`, retrieved it privately to `~/Downloads/ve2e-isup-full-results.json`, and deleted the temporary reader pod. No PR was created.

## Lessons Learned: Pitfalls

- A checkpoint being self-contained does not imply compatibility after config module deletion: Python-pickled config class paths remain runtime dependencies.
- The first cleanup image and model implementation were sound, but validation could not pass without either a cleanup-native model or a narrow translation boundary.
- Zot image pulls can incur transient TLS handshake timeouts; a later pull may recover without rebuilding.
- Raw cleanup did not contain unified DinoFlex. Work must target the combined cleanup-plus-unify tree until the merge lands.
- A strict exact-one-study assertion is appropriate for a hand-picked request but not a broad split query. A cohort workflow must filter to patients with exactly one eligible study without using outcome or mask information for ranking.

## Lessons Learned: Improvements

- Prefer storing checkpoint configs as plain versioned dictionaries rather than pickled config class instances. Include an explicit schema version and migration function.
- Keep checkpoint compatibility adapters in one named module with a narrowly guarded fallback. This makes later deletion mechanical.
- Produce at least one cleanup-native six-class GG EMA model before removing the adapter. Then delete `src/ve2e/legacy_checkpoint.py` and the fallback in `src/ve2e/isup.py`.
- Document the cleanup schema mapping for flat legacy DinoFlex fields to `encoder`, `tasks`, `trainer`, `clearml`, and `checkpointing` groups.
- Add PHI-safe aggregate forward-pass progress. The 640-patient run spent about 81 minutes in inference with no batch counter, which made monitoring less precise than pipeline preprocessing.

Source: https://docs.virdx.dev/knowledge/inbox/2026-08-28-clinical-ve2e-isup-legacy-adapter/index.mdx