Inspected infra_k8s/current/dashboard/ and the live dashboard namespace using metadata-only commands.
Confirmed that the nginx frontend mounts only TLS and optional vxData proxy Secrets, while dashboard-api maps individual keys from the out-of-band dashboard-api-secrets Secret into environment variables.
Confirmed the source argo-workflows/clearml-credentials-production Secret contains the expected key names without reading or decoding their values.
Pulled the latest infra_k8smain, created feat/dashboard-clearml-credentials, and added mappings for DASHBOARD_CLEARML_API_URL, DASHBOARD_CLEARML_ACCESS_KEY, and DASHBOARD_CLEARML_SECRET_KEY.
Validated YAML parsing, exact Secret/key references, and whitespace.
Copied the three encoded fields directly from argo-workflows/clearml-credentials-production into dashboard/dashboard-api-secrets through a silent pipeline after internal DNS/VPN access recovered. Verified only the destination key names; no Secret value was decoded, displayed, or persisted outside Kubernetes.
Lessons Learned: Pitfalls
“Dashboard frontend” is ambiguous in this deployment. Runtime application credentials belong on dashboard-api; the browser-facing nginx frontend must not receive them.
Argo CD self-heal means a manual Deployment patch would be temporary. The environment mapping belongs in Git, while Secret values remain provisioned out of band.
Required Secret references can block a rollout when keys are absent. Provision the destination Secret keys before merging PR #187.
Cluster access depends on internal DNS/VPN because kubectl obtains OIDC credentials from Keycloak.
Lessons Learned: Improvements
Add declarative secret synchronization from the secret manager so credentials do not need cross-namespace kubectl copies.
Document the ownership and renewal process for dashboard-api-secrets and other out-of-band application Secrets.
Keep credential-copy validation metadata-only: verify key names and workload references, never values.
