Skip to content

2026-09-01 histo vxData SDK rootless directory uploads

Added an opt-in flat directory upload mode and used it for Factory histo preprocessing.

Work Done

Implemented StorageNamespace.upload(..., include_root=False) in virdx/mono while preserving the existing True default. The option uploads one local directory’s contents directly beneath its single server-allocated timestamp/token prefix. Factory histo preprocessing now opts in, removing the redundant local histo_preprocessing_<timestamp> layer without changing its one-upload-per-job behavior. Bumped vxdata-sdk from 3.2.0 to 3.3.0, added focused coverage, and opened draft PR https://github.com/virdx/mono/pull/365 at signed commit c003299f. The full 49-test SDK suite, SDK and Factory format/lint/type checks, compile validation, lock checks, commit hooks, and independent review passed.

Lessons Learned: Pitfalls

A list of directories passed to storage.upload does not share one allocation. The SDK loops over each directory and the API creates a new timestamp/token prefix per presign/upload-dir request. This behavior is easy to misread because transfers are later executed together in one engine pass. An instrumented backend confirmed the allocation count before choosing the API shape.

Lessons Learned: Improvements

Document directory-upload shape explicitly: one directory input means one allocated prefix; each directory leaf in structured list/dict input gets its own allocation; include_root=False changes only whether that directory’s basename is retained. Include both API-backed and direct-S3 control planes, return trailing slashes, multipart behavior, and structured-input compatibility in future SDK upload reviews.

Navigation

Type to search…

↑↓ navigate↵ selectEsc close