Narrow instance cloud permissions to actual consumers - #18654
Conversation
The DIGITALOCEAN_ACCESS_TOKEN, HCLOUD_TOKEN and SCW_* exports in the bootstrap script existed only for protokube's gossip seed discovery. Workers stopped receiving them in kOps 1.36 and protokube itself was removed in kOps 1.37, so the control-plane exports and their persistence in /etc/sysconfig/kops-configuration have no consumer left. All remaining consumers (etcd-manager, kops-controller, dns-controller, CCM, CSI) receive credentials via manifests built at kops update time. Also write /etc/sysconfig/kops-configuration with mode 0600; it can still contain state store and OpenStack credentials and was previously world-readable.
Skip the dns-controller Route 53 grant on the control-plane instance role when useServiceAccountExternalPermissions is set; dns-controller and external-dns get dedicated IAM roles in that mode, matching how the other addon permissions are handled. Scope the kube-router and kindnet ec2:ModifyInstanceAttribute grants to cluster-tagged instances, matching the CCM and IRSA variants of the same permission. Add golden coverage for the kube-router node policy.
|
/test ? |
|
/test pull-kops-aws-kindnet-debian13 |
|
/test pull-kops-e2e-cni-kuberouter |
|
/test pull-kops-e2e-cni-calico-ipv6 |
|
/retest |
| # Other changes of note | ||
|
|
||
| * TODO | ||
| * On DigitalOcean, Hetzner, and Scaleway, control-plane user data no longer includes cloud API credentials. Components that still require them receive them through manifests generated by `kops update cluster`. |
There was a problem hiding this comment.
Someone could be relying on these in their additionalUserData scripts. Might be worth suggesting how to add them back if desired.
There was a problem hiding this comment.
I guess they would just have to add them to their scripts. Is there a better way?
There was a problem hiding this comment.
i think thats the only option, should we call that out in this release note?
There was a problem hiding this comment.
Added a small clarification:
If an
additionalUserDatascript orspec.hooksunit relied on these variables, set them in the script or unit itself.
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: rifelpet The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
/override pull-kops-e2e-cni-cilium-eni |
|
@hakman: Overrode contexts on behalf of hakman: pull-kops-e2e-cni-cilium-eni DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
The removal of gossip DNS and protokube in kOps 1.37 lets us narrow instance permissions and credentials to their actual consumers.
useServiceAccountExternalPermissionsis enabled; dns-controller and external-dns use dedicated service account roles in that mode.ec2:ModifyInstanceAttributeto cluster-tagged instances./etc/sysconfig/kops-configurationwith mode0600./cc @rifelpet @ameukam