Skip to content

2026-09-18 NanoVirDx histology preprocessing

Added a path-backed Histo preprocessing transformer to NanoVirDx and validated its one-to-many WSI output path.

Work Done

  • Implemented HistoPreprocessingTransformer in packages/nanovirdx as a thin local WSIBatch -> WSIBatch adapter. It lazily imports Histo, builds the Factory preprocessing parameters, calls preprocess_sample once per WSI, and translates the result into PreprocessingOutputBatch. The output preserves original-coordinate tissue masks on input_wsi and groups each source WSI’s component images with their aligned tissue masks. Histo owns model resolution, validation, errors, and artifact creation.
  • Kept Factory-owned concerns out of the transformer: cohort selection, vxData identifiers and metadata, uploads, registration, and workflow retry behavior remain Factory responsibilities.
  • Implemented WSIBatch.from_vxdata(...) to resolve and download HistoScans while preserving requested order.
  • Added the internal Artifactory dependency histo 0.0.127 for the CUDA-enabled Linux platform, updated virdx-ome to the API-compatible 0.0.24 series, and regenerated packages/nanovirdx/pixi.lock.
  • Validated an Otsu synthetic OME-TIFF end to end, including the component TIFF, foreground masks, and transform JSON. Revalidated the existing DiffSim pathway after the dependency update.
  • Ran Ruff, Pyright, compileall, Pixi lock checking, diff checking, Nx affected checks, and module-boundary checks. Nx had no registered NanoVirDx tasks; the direct package checks covered the implementation.
  • Committed the initial implementation as 2c2d4613, then removed the redundant wrapper logic and moved the class to preprocessing.py in d88a023b. Commit d97e88dd added the dedicated preprocessing output contract and preserved both original and component tissue masks. All commits were pushed to origin/fk/nanovirdx.
  • Referenced Factory’s feat/factory-histo preprocessing service and the Histo preprocess_sample, InputSample, TissueDetectionParams, and GrandQC implementations.

Lessons Learned: Pitfalls

  • The initial adapter duplicated responsibilities already encapsulated by Histo or owned by Factory: cache manifests, artifact completion validation, per-slide error aggregation, parameter checks, and direct GrandQC model loading. This grew a straightforward wrapper to 264 lines. The corrected implementation is 81 lines and delegates those concerns.
  • dev.felixknispel.coder.dev did not resolve, so the packaged Linux/CUDA runtime could not be executed there. The macOS smoke test used the local Histo source environment (0.0.120) and the model-free Otsu detector; packaged Histo 0.0.127 was still resolved and lock-checked from Artifactory.
  • Released Histo 0.0.127 remains effectively Linux/CUDA-only because its runtime unconditionally depends on GPU packages. This prevents direct packaged preprocessing validation on macOS.
  • Histo tag 0.0.127 still resolves default GrandQC weights from the standard mounted path; the later Hugging Face download work is not an ancestor of that release tag. NanoVirDx therefore documents an explicit grandqc_model_path unless the standard mount exists.
  • NanoVirDx is not registered as an Nx project, so workspace affected checks report no tasks even though the Python package changed.

Lessons Learned: Improvements

  • Release a slim CPU-capable Histo preprocessing package or feature split. That would allow macOS development and avoid pulling the training and CUDA dependency closure into CPU Factory workers.
  • Publish the GrandQC runtime-download change in a new Histo release, then update NanoVirDx’s pin and documentation so default setup does not depend on an external mount.
  • Add a NanoVirDx project.json with lint, typecheck, test, and build targets. Add small behavior-level tests once pre-provided test files are available, especially for grouped one-to-many component outputs, aligned tissue masks, and ordered vxData downloads.
  • Keep a small approved WSI fixture or a documented low-volume vxData scan for packaged Linux integration checks. This would avoid downloading gigabyte-scale production slides for routine validation.
Navigation

Type to search…

↑↓ navigate↵ selectEsc close