---
title: "2026-08-07 Hio hard design review"
description: "Consolidated adversarial review of the unmerged S3-first hio branch, including demonstrated transaction, resume, spatial, migration, and benchmark defects."
image: "https://docs.virdx.dev/img/virdx-social-card.png"
---

> Documentation Index
> Fetch the complete documentation index at: https://docs.virdx.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# 2026-08-07 Hio hard design review

## Work Done

- Reviewed `proposal.md`, the relevant quantitative and standards sections of `references.md`, all of `packages/hio/src/hio`, its README/tests/benchmark, the `histo.io` seam, and the application migration from `main`.
- Ran the complete hio suite against the local MinIO fixture (35 passed) and the seam suite (12 passed).
- Built read-only `/tmp` demonstrations showing that in-place overwrite remains publicly readable while partial, concurrent writers can tear data from one run with metadata from another, incompatible geometries reuse stale shards, reopened writers cannot complete all-fill shards, non-integer `Level.mpp` reports geometry inconsistent with returned arrays, NGFF 0.5 `dimension_names` are absent, and the seam drops `fill_value`.
- Compared migration semantics and found deleted panoptic input fingerprints, model windows re-anchored at storage shard boundaries, and TIFF OME-XML calibration/independent-axis scale regressions.
- Wrote the full review to `/tmp/hio_review.md`. The recommendation is not to merge or deploy until writes use immutable generations plus conditional dataset-scope publication, resume validates a complete creation/job manifest, and level resampling is made truthful.

## Lessons Learned: Pitfalls

- The histo knowledge index had no project-specific material for this new hio branch, so the mandatory knowledge read did not reduce repository investigation time.
- Passing happy-path local/S3 tests gave false confidence because the tests omit overwrite visibility, duplicate writers, incompatible resume, resumed all-fill output, and read-shape/geometry agreement.
- The long research bibliography made strong numerical claims, but several decisive choices were implemented before the proposal's own required production benchmark and some benchmark numbers were not traceable to a reproducible artifact.
- Delegated reviewers each created narrow recap entries before the consolidated review finished, creating some inbox duplication.

## Lessons Learned: Improvements

- Add an hio project page only after the design is accepted, recording explicit invariants: immutable published keys, generation identity, dataset-scope commit, conditional publication, writer fencing, resume fingerprint contents, and `read().shape == geometry.shape`.
- Add a review checklist/test matrix for object-store writers covering first publication, overwrite, crash/retry, all-fill shards, stale configuration, duplicate workers, finalize races, post-finalize mutation, multi-array commits, cache invalidation, and Ceph-specific consistency/conditional-PUT behavior.
- Store benchmark commands, raw results, request/byte instrumentation, fixture identity, and hardware with each policy decision; distinguish decoded-array throughput from network throughput.
- Coordinate subagent recap ownership so only the parent writes a consolidated knowledge entry unless a child discovers a separately durable topic.

Source: https://docs.virdx.dev/knowledge/inbox/2026-08-07-histo-hio-hard-design-review/index.mdx