Skip to content

2026-08-12 Histo calldiff architecture audit

Reviewed histo layering and call-flow architecture with calldiff on the S3 I/O branch.

Work Done

  • Audited histo on feat/fk/s3-io at f216f8da against main/merge base 655332dc.
  • Cached and built tanishqkancharla/calldiff, then inspected focused call trees/diffs for process_single_wsi, preprocess_sample, create_panoptic_segmentation_tiled, run_stage, SemanticSegPredictor.predict_wsi, and gland/tissue workflows.
  • Confirmed the branch substantially improves storage layering by routing major workflows through histo.io/hio, completion-aware writers, physical geometry, and tiled reads.
  • Identified remaining architecture smells: deferred/package-level dependency cycles, eager heavy panoptic_segmentation initialization, large mixed-responsibility modules, inference-to-training coupling, duplicated Python/Argo stage topology, misplaced shared utilities, and a partially migrated preprocessing path that still mixes hio with legacy Zarr/Vizarr/TIFF operations.
  • No files in the histo repository were changed. Existing user changes to packages/hio/pixi.lock and packages/hio/pyproject.toml were left untouched.

Lessons Learned: Pitfalls

  • Unscoped calldiff output was too large (6,832 lines) to support architectural review; focused distinctive entrypoints and bounded depth were required.
  • Calldiff is syntactic: it cannot resolve io.* or receiver method calls through the hio implementation, cannot follow dynamic run_func() callbacks, and indexes duplicate function names ambiguously. It should not be treated as a whole-package dependency graph or correctness proof.
  • The mandatory knowledge-base context was useful for branch scope, but its warning that the S3 I/O branch is unmerged should be surfaced earlier alongside the exact checked-out commit.

Lessons Learned: Improvements

  • Document an intended package dependency direction for histo and enforce it with an import-boundary test/tool. This would make deferred cycles and boundary regressions explicit.
  • Add a reusable architecture-audit recipe: use calldiff with --entry, path filters, --max-depth, and merge-base comparison; complement it with AST import analysis and import-cost measurements.
  • Document stable public modules for pipeline taxonomy, artifact contracts, visualization, shared color operations, and checkpoint loading so scripts do not need to import implementation modules.
Navigation

Type to search…

↑↓ navigate↵ selectEsc close