No technical need for separate vars; one holding both known_hosts
lines (pasted straight from an existing trusted known_hosts file) is
simpler to configure.
Replaces the ssh-keyscan-based known_hosts population (and the
diagnostic step added to debug its flakiness) with two vars,
GITEA_HOST_KEY and DEPLOY_HOST_KEY, each a full known_hosts-format
line. No extra network calls before the real connections, no TOFU, and
no more risk of tripping aquaria's connection-rate limiting — the
actual root cause of the earlier flakiness was ssh-keyscan opening
multiple near-simultaneous connections (one per probed key type).
Latest diagnostic run showed aquaria's OpenSSH responding fine on the
first of several near-simultaneous connections ssh-keyscan opens (one
per default-probed key type), then hanging on the rest with no
response — a rate-limiting signature, not an outright ban. Both hosts
confirmed to offer an RSA host key, so scan just that type: one
connection per host instead of ~4 at once. Left the diagnostic step in
to confirm before removing it. Dropped -v now that the cause is known.
ssh-keyscan's -v only traces SSH-protocol activity; a pre-handshake
failure (bad DNS, TCP connect hang/drop) prints nothing regardless.
Firewall allow-listing is ruled out (same external IP as a working
manual connection), so check DNS resolution and a raw nc TCP connect
from inside the runner container directly. Remove once root-caused.
Plain ssh-keyscan prints nothing on a per-host failure (by design, so
scanning long host lists doesn't spam errors for hosts that are down),
which is why removing 2>/dev/null earlier revealed nothing. -v forces
it to actually report what's failing.
Missed two spots: the git.haemka.in keyscan and the submodule fetch's
GIT_SSH_COMMAND. If git.haemka.in's DNS answer includes an AAAA record
(depends on which resolver the runner hits), these hit the same no
IPv6 route issue intermittently, explaining the flaky failures.
aquaria.haemka.net has both A and AAAA records; the runner container has
no IPv6 route, so ssh picked the AAAA address and failed with 'Network
is unreachable'. Force -4 on both the keyscan and the actual rsync
connection.
The step failed silently (exit 1, no message) because stderr was
redirected to /dev/null while sh -e aborts on ssh-keyscan's non-zero
exit. Let the real error (DNS/firewall/etc.) show up in the log.
None of these are sensitive, so use Actions variables (vars) instead of
secrets. DEPLOY_PORT is now its own variable (default 22) rather than
being packed into DEPLOY_HOST. Only DEPLOY_SSH_KEY remains a secret.
ssh-keyscan and rsync's -e ssh both need the port passed via -p rather
than embedded in the hostname; parse DEPLOY_HOST accordingly, defaulting
to 22 when no port is given.
Avoids tying CI to a personal/bot Gitea account: the same DEPLOY_SSH_KEY
secret used for the webserver rsync step is reused as a read-only Gitea
deploy key on hmk/pelican-latex. The submodule's tracked HTTPS URL is
overridden to SSH just for this checkout so it authenticates with that
key instead of needing a token with access to both repos.
The default per-job Actions token only covers the triggering repo; the
theme submodule lives in a separate private repo (hmk/pelican-latex), so
cloning it needs a token with read access to both. Add CHECKOUT_TOKEN
as a repo secret (a Gitea access token with repo read scope) for this
to work.
haemka
haemka