Configured a local AWS profile to retrieve S3 interoperability credentials from 1Password through credential_process and verified read access to the HistAI GCS bucket without exposing object names or credentials.
Prepared ~/work/playground/setup-secrets.sh. It accepts four credential inputs as either 1Password secret references or literal values, then streams a temporary Kubernetes Secret into argo-workflows without putting secrets in command arguments, temporary files, or Workflow YAML.
Prepared ~/work/playground/submit-rclone.sh. It exposes four source/target path constants and submits an explicitly confirmed, destructive rclone sync Workflow from GCS S3 to Ceph S3.
The Workflow uses a versioned rclone image, secret-backed remote configuration, the FreeIPA CA, trailing-slash paths, bounded multipart buffering, retries, and compact progress output. The full dataset is not materialized in the pod filesystem; cross-provider bytes stream through pod memory and network.
No Kubernetes Secret or Workflow was created. The scripts were handed to the operator for manual review and execution.
Lessons Learned: Pitfalls
The Ceph credentials came through a shared 1Password item, so they did not have usable op:// references. A secret setup script must support literal values as a fallback while clearly warning that they remain on local disk.
rclone sync is destructive at the target. Treat generic requests to “sync” as ambiguous until deletion semantics are explicit, and require an exact interactive confirmation before submission.
The current infrastructure SOP references both ipa-ca and ipa-ca-cert for the Ceph CA in different contexts. Namespace and live resource names must be checked rather than assumed.
Lessons Learned: Improvements
Add a maintained one-off cross-provider rclone WorkflowTemplate with standard secret key names, CA wiring, resource limits, retries, and non-verbose progress logging. Operators would then only create a temporary source credential Secret and submit four path parameters.
Document the distinction between a 1Password vault item, which supports op:// references, and an externally shared item, which may require a literal copy or migration into an accessible vault.
Add an approved automatic cleanup mechanism for temporary transfer credentials. The current safe fallback is manual deletion after Workflow completion or failure because the workflow service account may not be allowed to delete Secrets.
