kops: add COS 129 to the GCE grid and age out COS 121 - #37691
Conversation
|
Skipping CI for Draft Pull Request. |
|
/test all |
Adds cos129 and cos129arm64, pinned to cos-129-19506-299-116, tested against kops latest and 1.36. This follows the convention the other recently added distros use: a new image is only exercised on the newer kops branches. At the same time, stop testing cos121 on kops latest and 1.36. kops master now requires containerd 2.1.0 or newer (kubernetes/kops#18683), while cos121 is pinned to containerd 2.0.7 because COS 121 cannot run 2.1+, so every cos121 job on kops latest fails at cluster creation: Error: spec.containerd.version: Invalid value: "2.0.7": unsupported legacy version; containerd 2.1.0 or newer is required The 1.36 jobs still pass, since the release branch does not carry that floor; dropping them as well is an age-out decision. Note that dropping the newer branches requires passing None to drop_unsupported_versions, because None is the entry that maps to "latest kops" -- filtering only on version strings would have left cos121 on latest, the broken combination, while removing the 1.36 jobs that still work. Also register the kops-distro-cos129 testgrid dashboard, in both the dashboard group and the dashboard list, so the new jobs' annotations resolve. No kops changes are needed. ContainerOS is a single version-agnostic distribution there: util/pkg/distributions/identify.go matches any "cos-" prefix and DistributionContainerOS carries no version, so every COS behavior in nodeup keys off that one constant. Net effect is 78 job additions and 72 removals. The remaining line churn in the generated files is cron reassignment for jobs whose position shifted, not a change to those jobs.
79675af to
134344b
Compare
|
/test all |
|
/cc @hakman |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: hakman, rifelpet The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
@rifelpet: Updated the
DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
Adds
cos129/cos129arm64to the GCE distro matrix, pinned tocos-129-19506-299-116, and stops testingcos121on the newer kops branches.cos129's gating follows the convention the other recently added distros use (
ubuntu2510,ubuntu2604,rhel10,rocky10): a new image is exercised only on the newer kops branches.Why cos121 is being dropped from kops latest
kubernetes/kops#18683 (
validation: require containerd 2.1.0 or newer, merged 2026-08-13) raised the containerd floor in kops master.helpers.pypins cos121 to containerd 2.0.7 because COS 121 cannot run 2.1+, so every cos121 job on kops latest now fails atkops create cluster:Confirmed on:
The failure happens at cluster creation, so it produces no JUnit failures — the jobs just go red with an empty test report. Every cos121 latest job that has run since 08-14 has failed this way; the few still showing green last ran between 08-09 and 08-11, before #18683 merged, and will fail on their next run.
The kops 1.36 jobs are not affected — the release branch does not carry the raised floor and
-ko36jobs pass today. Dropping cos121 from 1.36 as well is an age-out decision, not something these failures force.Dropping "latest" requires passing
Nonekops_versionsis[None, "1.34", "1.35", "1.36"], whereNoneis the entry that maps to latest kops.drop_unsupported_versionsfilters by value, so a list of version strings alone cannot remove it.Every existing use of the helper drops old versions to gate a new distro, where keeping latest is exactly right. Ageing a distro out is the opposite direction and is new here: filtering on
['1.36']alone would have left cos121 running on latest — the broken combination — while removing the 1.36 jobs that still pass. Hence[None, '1.36'], with a comment at the call site.No kops changes required
ContainerOS is a single version-agnostic distribution in kops:
util/pkg/distributions/identify.go:78matches anycos-prefix and returnsDistributionContainerOS, which is declared withversion: 0. Every COS behavior in nodeup keys off that one constant — skipping containerd install and applying the systemd override, skipping ntp and logrotate, the kubelet flag differences, the/etc/systemd/systemunit path, and the/etc/resolv.confbind-mount.gce.SSHUsernameForImagealso needs nothing; COS 129 resolves toadminlike the other COS images.The one COS-version-specific thing in the kops tree does not apply either:
tests/e2e/pkg/tester/skip_regex.goskipsImageVolumeforcos-121because that test needs containerd ≥ 2.1 and COS 121 ships older — the same root cause as the failures above. COS 125 already runs without that skip, so 129 will too. That is an inference from version ordering rather than something verified against COS 129 directly; ifImageVolumefails on the new jobs, this is the first thing to check.Diff shape
The generated diff is large but the job-set change is small. Comparing job names as sets against master:
The remaining churn is cron reassignment for jobs whose position shifted when cos121 lost an entry and cos129 gained one — the grid loop is networking-major, so everything after the change point in each group gets a new slot. No unrelated job was added, removed, or otherwise altered.
Testing
make generate-jobs, which preservespinned.listrather than re-resolving every image.go test ./config/tests/jobs/...passes.ko34/ko35jobs only and no latest-kops jobs, and cos129 has latest plusko36./sig testing
/area jobs
🤖 Generated with Claude Code