digitalocean: delete SSH keys when deleting a cluster - #18697
Merged
kubernetes-prow[bot] merged 1 commit intoAug 16, 2026
Merged
Conversation
kops uploads an SSH key to the DigitalOcean account for every cluster (dotasks/sshkey.go createKeypair) and names it after the cluster and the key fingerprint, but the resource deleter only knows about droplets, volumes, DNS records, load balancers and VPCs. The keys are never removed. This has gone unnoticed because the CI jobs all supply the same SSH key, so the fingerprint -- and therefore the key name -- is identical on every run and the existing key is found and reused. A cluster created with any other key leaks one key per cluster into the account, and DigitalOcean scopes SSH keys to the account rather than the cluster, so they accumulate with nothing to attribute them to. AWS already deletes its ephemeral EC2 key pairs (pkg/resources/aws/aws.go DeleteKeypair) and so does OpenStack; this brings DigitalOcean in line. Keys are matched on the "kubernetes.<cluster name>-" prefix that pkg/model/names.go gives them. The trailing hyphen is required so that "foo.k8s.local" does not also match keys belonging to "foo.k8s.local.example.com", and a cluster using spec.sshKeyName keeps a name kops did not choose, so its pre-existing key is left alone.
Contributor
|
Skipping CI for Draft Pull Request. |
Member
Author
|
/test pull-kops-do-dns-none |
rifelpet
force-pushed
the
do-delete-sshkeys
branch
from
August 15, 2026 23:11
1fe6a19 to
19a8af3
Compare
Member
Author
|
This job exercised the DO ssh key creation and deletion successfully, so the PR is ready for review: https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/kops/18697/pull-kops-do-dns-none/2088748301577883648 |
rifelpet
marked this pull request as ready for review
August 15, 2026 23:12
hakman
approved these changes
Aug 15, 2026
Contributor
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: hakman The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
kops uploads an SSH key to the DigitalOcean account for every cluster (
dotasks/sshkey.gocreateKeypair→KeysService().Create), naming itkubernetes.<cluster name>-<fingerprint>perpkg/model/names.go. Butpkg/resources/digitalocean/resources.goonly deletesdroplet,volume,dns-record,loadbalancerandvpc— the SSH key is never removed.AWS already cleans up its key pairs (
pkg/resources/aws/aws.goDeleteKeypair, wired in atListKeypairs), and OpenStack haspkg/resources/openstack/sshkey.go. DigitalOcean was the gap.Why it hasn't bitten yet
Every DO CI job supplies the same SSH key through the
preset-do-sshprow preset, so the fingerprint — and therefore the key name — is identical on every run.SSHKey.Findmatches the existing key and reuses it, and nothing accumulates.That stops being true as soon as a cluster is created with a per-cluster key. It leaks one key per cluster, and because DigitalOcean scopes SSH keys to the account rather than to a cluster, they pile up with nothing identifying what they belonged to.
This is a prerequisite for kubernetes/test-infra#37690, which switches the DO e2e jobs to the ephemeral keys
kubetest2-kopsnow generates (#18686). That PR is on hold until this ships.Matching
Keys are selected on the
kubernetes.<cluster name>-prefix. Two properties matter, and both are covered by tests:foo.k8s.localwould also match keys belonging tofoo.k8s.local.example.com. The concrete case in CI ise2e-kops-do-dns-noneande2e-kops-do-dns-none-ha, which run against the same account.spec.sshKeyNameis left alone. That key keeps a name kops did not choose, so it does not match the prefix — consistent withFindindotasks/sshkey.go, which adopts such a key rather than creating one. Deleting a user's pre-existing key would be considerably worse than leaking one.Deletion tolerates a 404 so a retried teardown is idempotent.
Testing
TestFilterClusterSSHKeyscovers seven cases: the normal match, several stale keys from repeated runs of one cluster, a longer cluster name that must not match, the-hasibling, an unrelated cluster, a user-named key, and the bare cluster name with no fingerprint.go build ./...,go vet, andgo test ./pkg/resources/... ./upup/pkg/fi/cloudup/do/... ./upup/pkg/fi/cloudup/dotasks/...all pass.The listing follows the existing
GetAllVPCspagination pattern, and theDOCloudmock gains the matching no-op.🤖 Generated with Claude Code