Benchmarked anonymous vxData-presigned downloads from s3://vxdata/source-data/leopard/training/ on main.dev.felixknispel.coder using temporary directories on the XFS home volume.
Used the SDK 3 wire policy without persisting code: deterministic 64 MiB HTTP ranges with a global cap of 16 concurrent streams, positional writes, exact Content-Range and byte-count checks, and final fdatasync.
One 3.677 GiB HistoScan downloaded in 4.685 seconds at 803.6 MiB/s (6.741 Gbit/s), with 59 chunks and no retries.
A three-HistoScan batch totaling 8.924 GiB downloaded in 9.530 seconds at 958.9 MiB/s (8.044 Gbit/s), with 144 chunks and no retries.
A 4 GiB local sequential write plus fdatasync measured 452.2 MiB/s. All downloaded data and scratch files were removed after measurement.
Swept the only successful-path tuning knob, TransferConfig.max_streams. For one 3.677 GiB slide, three-run medians were 888.5 MiB/s at 4 streams, 1,114.0 at 8, 1,007.5 at 12, 1,006.3 at 16, 918.2 at 24, and 962.9 at 32.
For the representative three-slide batch, 16 streams averaged 1,523.6 MiB/s versus 1,464.0 at 8; one-off checks at 12, 24, and 32 were slower. The SDK default of 16 is therefore retained for batched downloads, while 8 is the measured single-large-file optimum on this Coder.
Lessons Learned: Pitfalls
The Coder lacked curl, mc, and authenticated access to the internal Conda channel, so the published SDK could not be installed there for the check.
A single transfer can be noisy due to gateway and filesystem caching; the three-object batch supplied a more useful sustained result.
Lessons Learned: Improvements
Provide Coder instances with internal Artifactory authentication or a preinstalled vxData SDK environment for direct production transfer benchmarks.
Keep a documented, non-persistent benchmark command for the fixed 64 MiB/16-stream policy so future storage changes can compare like-for-like results.
