Replaced the ISUP benchmark with a PI-CAI benchmark and moved vxData cohort preparation, expert lesion-target selection, materialization, and canonical model preprocessing into generate_picai_sample_from_vxdata.
Split the reusable PI-CAI package into schemas, data generation, and model inference modules. Added batched prediction for full-cohort inputs.
Replaced the EMA-only ClearML loader with load_clearml_model, which normalizes embedded configs and loads prediction weights from both EMA and main checkpoints.
Validated recent real ClearML EMA model b745c96bbb9f4b50a45bd96d26f201af and main model a5677ed0e22a4d669f2d9c2c72ac5ab4.
Pushed ve2e commits 9bcf0c91 and 91c5a234 to feat/isup-inference.
Completed real workflow run-ve2e-picai-source-fix-ptpb2 across 46 eligible cases, including vxData preparation, preprocessing, target construction, inference, shape alignment, finite maps, and probability bounds.
Lessons Learned: Pitfalls
Lightning load_from_checkpoint restored training-only state and rejected a missing loss_fn.weight. Inference should load only module.model prediction weights.
Some otherwise eligible vxData cases have no biopsy/RPE provenance. They cannot populate a target schema restricted to those two values and must be excluded explicitly.
Taking the first patient IDs from the global non-training query can select no patients from pipeline-configured datasources. A distributed sample across the query ordering produced a representative smoke cohort.
Lessons Learned: Improvements
Document that PI-CAI target generation requires non-null biopsy/RPE provenance in addition to expert lesion masks.
Keep a small cluster smoke workflow for reusable inference that selects a privacy-safe distributed cohort and validates real main and EMA checkpoints.
Treat ClearML main and EMA payloads as packaging differences only; normalize both to model-only state before inference.
