Skip to content

2026-08-31 OpenVPN stale route diagnosis

Diagnosed VPN and bundle-download failures as a stale macOS bypass route, not an SSH key mismatch.

Work Done

  • Investigated infra_onboarding/virdx_openvpn, its endpoint variables, the user’s OpenVPN Connect logs, current macOS routes, and live TCP behavior.
  • Confirmed the primary endpoint 88.205.102.106 had a stale /32 route via the old local gateway 172.18.0.1, while the active interface was 192.168.4.50 with gateway 192.168.4.1.
  • Linked that stale route to both OpenVPN’s repeated NETWORK_SEND_ERROR / server poll timeouts and SFTP’s Can't assign requested address before authentication.
  • Confirmed id_ed25519_virdx.pub matches one of the public keys at https://github.com/felixknispelVBC.keys.
  • Confirmed the fallback 88.205.102.98:2222 returned Connection refused; current infra_ansible inventory uses router virdx-router-002 on 88.205.102.106 and forwards UDP 1194 and TCP 2222. The .98 address belongs to the old hkn_a network and does not provide a working fallback in the current inventory.
  • Ran the recovery sequence. Quitting OpenVPN Connect removed the stale route, SFTP bundle download succeeded, and onboarding completed successfully.
  • Worked around a second script issue: inherited NODE_OPTIONS and color variables added Electron warnings before the CLI JSON, causing the script to report a successful profile removal as failure. Rerunning with those variables unset completed profile import.
  • Verified OpenVPN connected to 88.205.102.106:1194 over UDP as 172.16.2.4, gateway 172.16.2.1 was reachable, the public endpoint route used 192.168.4.1, and the watchdog was active.

Lessons Learned: Pitfalls

  • The onboarding script reports every bundle retrieval failure as an SSH/GitHub key problem. This is misleading when TCP connection setup fails before authentication.
  • ovpn_cli assumes stdout contains only JSON. Electron or Node environment warnings make jq parsing fail even when OpenVPN Connect reports "status": "success" and performed the requested change.
  • The recently added watchdog restarts OpenVPN Connect, but a restart can preserve or recreate a stale public-endpoint bypass route after the Mac changes networks.
  • The documented .98 fallback is not backed by the active router inventory, so it creates false confidence and adds repeated failed key attempts.

Lessons Learned: Improvements

  • Add an infrastructure SOP for macOS OpenVPN failures: inspect route -n get <remote>, compare it with route -n get default, and clear stale /32 bypass routes after network changes.
  • Make virdx_openvpn classify routing, refusal/timeout, host-key, and authentication failures separately.
  • Remove .98 from OVPN_DROPBOX_HOSTS or restore and monitor the corresponding router/NAT/SFTP service.
  • Consider teaching the watchdog to validate and repair the endpoint bypass route, not only relaunch the client.
Navigation

Type to search…

↑↓ navigate↵ selectEsc close