Skip to content

2026-08-28 vxData multipart SDK architecture review

Reviewed PR 356 SDK multipart upload boundaries and identified focused reuse and encapsulation improvements.

Work Done

  • Reviewed PR 356 from merge base d2ab806c through HEAD, focusing on the vxData SDK storage namespace, transfer engine, API transport, and Direct S3 adapter.
  • Reported prioritized architecture suggestions to the parent reviewer without editing the monorepo.
  • Recommended keeping wire policy inside the transfer engine, moving multipart presign batching into the transport layer, typing the engine against narrow control-plane interfaces, and reusing batched uploads for DataFrame convenience paths.
  • Marked generic transfer-task hierarchies, upload/download state inheritance, and a public multipart planning API as over-abstraction.

Lessons Learned: Pitfalls

  • The general vxData schema guide gave useful SDK context but did not cover the new multipart branch. Direct inspection of the merge-base diff and focused upload tests was required.
  • A private test seam (StorageNamespace._wire) currently shapes production ownership, which makes the documented claim that request shapes are not caller-tunable weaker than it appears.

Lessons Learned: Improvements

  • Future SDK architecture reviews should compare where analogous download and upload policies live. Here that comparison quickly exposed multipart batching in the wrong layer.
  • Review convenience methods as well as the main primitive. The main upload path batches work, while DataFrame and referenced-file helpers still reduce it to sequential one-file calls.
Navigation

Type to search…

↑↓ navigate↵ selectEsc close