2026-09-04 Histo domain representation v2 pipeline call graph

Work Done

• Traced jobs/run-domain-representation-v2.yaml from its two-node Argo DAG through the CPU worklist discovery entrypoint and the fanned-out GPU worker entrypoint.

• Used pyan3 2.8.1 to generate a static symbol graph, then checked and extended it manually for Hydra, executor callbacks, PyTorch model dispatch, and NumPy/CuPy dispatch.

• Narrowed the result to one platform-backed WSI rooted at generate_domain_representation.py.

• Replaced all prose, diagrams, tables, and methodology in the project document with one tree-like call list, as requested.

• The tree includes configuration, model loading, materialization, per-window embedding and heads, watershed, representation merge, writes, publication, and cleanup.

• Added docs/domain-representation-v2-callgraph.md to the histo working tree. It remains uncommitted as requested.

• Asked two independent fresh-context reviewers to critique only the focused call tree. Both identified the same main levers: compile geometry once, replace per-stage tiling/streaming with one bounded runner, linearize lifecycle orchestration, centralize model/runtime policy, isolate the scientific representation kernel, and use one artifact manifest/sink.

• Reconciled that a 10x reduction is credible for application-owned orchestration and inference glue, but not for the scientific algorithms unless required behavior is removed. The more aggressive deletion of row streaming, host reservation, backend alternatives, and concurrency requires memory and throughput evidence first.

• Resumed both reviewers with permission to inspect source, tests, and config. Both independently refuted most of the call-tree-only 10x hypothesis: all three heads already share sweep_model; the representation is already a pure kernel; WsiOutputs is already a sink; and geometry, row streaming, buffers, cache controls, and concurrency encode tested or documented correctness/performance constraints.

• Source-backed consolidation candidates are a per-scan resolved plan for values currently rebuilt per super-window, one canonical output/artifact specification, migration of the older parallel Gleason embedding_decode.py path to shared WSI inference, and deletion of confirmed-unused compatibility flags. The reconciled safe reduction is roughly 5–15% of relevant code, not 10x, unless production capabilities are deliberately removed.

• One reviewer ran focused CPU suites covering generation, geometry, reservation, representation, sweep, tiles, and reduction: 126 passed and 4 GPU-only tests skipped.

Lessons Learned: Pitfalls

• pyan3 --function ... --direction down did not follow imported callees across the repository’s src/ and scripts/ source roots, despite those edges appearing in the full graph. Generating the full graph and traversing its DOT edges from each entrypoint worked.
• A static uses graph is not an exact runtime call graph. It includes class and attribute uses, while missing executor callbacks, model __call__, monkey-patched or compiled methods, and array-backend dispatch unless these are added manually.
• The broad knowledge-base startup material was not needed for this narrow repository architecture trace.

Lessons Learned: Improvements

• A histo-specific call-graph SOP could standardize full-graph generation, entrypoint traversal, source-root normalization, and explicit annotations for Hydra, executors, PyTorch, and NumPy/CuPy boundaries.
• Keeping a short pipeline architecture document near the production Argo job would make the orchestration hidden inside the GPU pod easier to see without reconstructing it from source.