Retrieved ~/work/playground/nfs_to_ceph_mapping.csv: 222,494 rows, 217,353 unique source paths, no conflicting mappings or distinct-source target collisions.
Confirmed September 14 Prime session 01a08fc6-7d73-76f1-9f3e-37f4454acdcb, messages at 13:49–13:52 UTC, explicitly uses s3://vxdata/nfs-copy/, not nfs-sync.
Prefixes /mnt/storage/data/, /mnt/artifacts/data/, and /mnt/artifacts/ map to that destination, preserving the remaining suffix.
Original extraction approach survives in ~/work/playground/fix_essen02_crop_status.py: histo_scans.url, histo_maps.url, diffsim_scans.url, generic_files.s3_url; export mapping, transfer bytes separately, then replace metadata references.
Proposed exact CSV lookups and typed SDK partial updates, dry-run by default, with unknown local paths rejected before writes. No production reads or writes were performed.
Pitfalls
Retrieved a later confirmation conversation, not the original mapping-generation conversation.
Original extraction explicitly leaves nested DiffsimScan.other_data references unresolved. Four fields do not prove deployment-wide coverage.
Mapping existence is not proof that copies completed or that consumers support S3. Verify both before applying.
SDK update batches are separate transactions; the complete migration is not atomic.
Improvements
Reuse the exact mapping CSV rather than regenerating destinations from a broad regex.
Preserve an identifier/field/old/new manifest before applying; pause concurrent writers during the switch.
Unmapped paths intentionally skipped
User clarified that new /mnt data may have arrived since the NFS copy. The flip script now leaves unmapped paths untouched and counts these separately from missing-copy exclusions.
Added a dry-run/apply regression test for copied, missing, and unmapped records. Formatting, lint, and scoped type checking pass. Direct pytest initially failed on a missing fsspec import, but the normal Nx/Pixi test command passed all three tests, including both new cases. No production operations were run.
Script saved
Saved the proposed job in mono/apps/vxdata-jobs/src/vxdata/jobs/f_20260917_nfs_to_ceph/main.py, dry-run by default with explicit --apply.
Added exact exclusions from ~/Downloads/vxdata-missing.txt; all 611 entries resolve after normalizing /mnt/data/storage/artifacts/ to /mnt/artifacts/.
Scoped formatting, lint, typecheck, CLI help, and offline loading of the real mapping/exclusions passed. No production queries or writes were run.
