Objects with Changed Resource Versions

[alimaazamat]

Which jobs are failing?

sig-cluster-lifecycle-cluster-api

• capi-e2e-main
  • periodic-cluster-api-e2e-main https://prow.k8s.io/view/gs/kubernetes-ci-logs/logs/periodic-cluster-api-e2e-main/1931619241354596352
• capi-e2e-minik8s-main
  • periodic-cluster-api-e2e-mink8s-main https://prow.k8s.io/view/gs/kubernetes-ci-logs/logs/periodic-cluster-api-e2e-mink8s-main/1930077039872708608

sig-cluster-lifecycle-cluster-api-1.8

• capi-e2e-release-1-8
  • periodic-cluster-api-e2e-release-1-8 https://prow.k8s.io/view/gs/kubernetes-ci-logs/logs/periodic-cluster-api-e2e-release-1-8/1927571757439389696

sig-cluster-lifecycle-cluster-api-1.9

• capi-e2e-release-1-9
  • periodic-cluster-api-e2e-release-1-9 https://prow.k8s.io/view/gs/kubernetes-ci-logs/logs/periodic-cluster-api-e2e-release-1-9/1932017368377069568

sig-cluster-lifecycle-cluster-api-1.10

• capi-e2e-mink8s-release-1-10
  • periodic-cluster-api-e2e-mink8s-release-1-10 https://prow.k8s.io/view/gs/kubernetes-ci-logs/logs/periodic-cluster-api-e2e-mink8s-release-1-10/1931732236751605760

Which tests are failing?

[FAIL] When following the Cluster API quick-start [It] Should create a workload cluster
/home/prow/go/src/sigs.k8s.io/cluster-api/test/framework/resourceversion_helpers.go:60

capi-e2e: [It] When following the Cluster API quick-start with ClusterClass [PR-Blocking] [ClusterClass] Should create a workload cluster [PR-Blocking, ClusterClass]

Since when has it been failing?

6/4/2025 in sig-cluster-lifecycle-cluster-api
6/5/2025 in sig-cluster-lifecycle-cluster-api-1.8
6/8/2025 in sig-cluster-lifecycle-cluster-api-1.9
6/3/2025 in sig-cluster-lifecycle-cluster-api-1.10

Testgrid link

https://testgrid.k8s.io/sig-cluster-lifecycle-cluster-api#capi-e2e-main

Reason for failure (if possible)

capi-e2e: [It] When following the Cluster API quick-start Should create a workload cluster

{Failed after 81.750s.
Resource versions didn't stay stable
The function passed to Consistently failed at /home/prow/go/src/sigs.k8s.io/cluster-api/test/framework/resourceversion_helpers.go:59 with:

Detected objects with changed resourceVersion

Object with changed resourceVersion Cluster/quick-start-z8fdyo/quick-start-3biz68:
  strings.Join({
  	... // 6117 identical bytes
  	"{}\n          f:upToDateReplicas: {}\n    manager: manager\n    ope",
  	"ration: Update\n    subresource: status\n    time: \"2025-06-08T08:",
- 	"28:25",
+ 	"30:04",
  	"Z\"\n  name: quick-start-3biz68\n  namespace: quick-start-z8fdyo\n  ",
  	`resourceVersion: "1`,
- 	"5172",
+ 	"6747",
  	"\"\n  uid: e4cc243a-1d13-46b0-a40d-02459cb1b903\n",
  }, "")


Object with changed resourceVersion KubeadmControlPlane/quick-start-z8fdyo/quick-start-3biz68-control-plane:
  strings.Join({
  	... // 5191 identical bytes
  	"eReplicas: {}\n        f:version: {}\n    manager: manager\n    ope",
  	"ration: Update\n    subresource: status\n    time: \"2025-06-08T08:",
- 	"28:25",
+ 	"30:04",
  	"Z\"\n  name: quick-start-3biz68-control-plane\n  namespace: quick-s",
  	"tart-z8fdyo\n  ownerReferences:\n  - apiVersion: cluster.x-k8s.io/",
  	"v1beta2\n    blockOwnerDeletion: true\n    controller: true\n    ki",
  	"nd: Cluster\n    name: quick-start-3biz68\n    uid: e4cc243a-1d13-",
  	"46b0-a40d-02459cb1b903\n  resourceVersion: \"1",
- 	"5162",
+ 	"6745",
  	"\"\n  uid: c022a22b-499f-45af-8a71-1feb8630c867\n",
  }, "")


Object with changed resourceVersion Machine/quick-start-z8fdyo/quick-start-3biz68-control-plane-kntnl:
  strings.Join({
  	... // 6458 identical bytes
  	"dGeneration: {}\n        f:phase: {}\n    manager: manager\n    ope",
  	"ration: Update\n    subresource: status\n    time: \"2025-06-08T08:",
- 	"28:25",
+ 	"30:04",
  	"Z\"\n  name: quick-start-3biz68-control-plane-kntnl\n  namespace: q",
  	"uick-start-z8fdyo\n  ownerReferences:\n  - apiVersion: controlplan",
  	... // 89 identical bytes
  	"beadmControlPlane\n    name: quick-start-3biz68-control-plane\n   ",
  	" uid: c022a22b-499f-45af-8a71-1feb8630c867\n  resourceVersion: \"1",
- 	"5164",
+ 	"6736",
  	"\"\n  uid: 5a08802e-852a-459a-a007-ee24195c6279\n",
  }, "")


Expected object to be comparable, diff:   map[string]string{
- 	"Cluster/quick-start-z8fdyo/quick-start-3biz68":                   "16747",
+ 	"Cluster/quick-start-z8fdyo/quick-start-3biz68":                   "15172",
  	"ClusterResourceSet/quick-start-z8fdyo/quick-start-3biz68-crs-0":  "15121",
  	"ClusterResourceSetBinding/quick-start-z8fdyo/quick-start-3biz68": "14989",
  	... // 7 identical entries
  	"KubeadmConfig/quick-start-z8fdyo/quick-start-3biz68-md-0-xp6gp-c9v8n":    "15155",
  	"KubeadmConfigTemplate/quick-start-z8fdyo/quick-start-3biz68-md-0":        "15006",
- 	"KubeadmControlPlane/quick-start-z8fdyo/quick-start-3biz68-control-plane": "16745",
+ 	"KubeadmControlPlane/quick-start-z8fdyo/quick-start-3biz68-control-plane": "15162",
- 	"Machine/quick-start-z8fdyo/quick-start-3biz68-control-plane-kntnl":       "16736",
+ 	"Machine/quick-start-z8fdyo/quick-start-3biz68-control-plane-kntnl":       "15164",
  	"Machine/quick-start-z8fdyo/quick-start-3biz68-md-0-xp6gp-c9v8n":          "15168",
  	"MachineDeployment/quick-start-z8fdyo/quick-start-3biz68-md-0":            "15163",
  	... // 8 identical entries
  }
 failed [FAILED] Failed after 81.750s.
Resource versions didn't stay stable
The function passed to Consistently failed at /home/prow/go/src/sigs.k8s.io/cluster-api/test/framework/resourceversion_helpers.go:59 with:

Detected objects with changed resourceVersion

Object with changed resourceVersion Cluster/quick-start-z8fdyo/quick-start-3biz68:
  strings.Join({
  	... // 6117 identical bytes
  	"{}\n          f:upToDateReplicas: {}\n    manager: manager\n    ope",
  	"ration: Update\n    subresource: status\n    time: \"2025-06-08T08:",
- 	"28:25",
+ 	"30:04",
  	"Z\"\n  name: quick-start-3biz68\n  namespace: quick-start-z8fdyo\n  ",
  	`resourceVersion: "1`,
- 	"5172",
+ 	"6747",
  	"\"\n  uid: e4cc243a-1d13-46b0-a40d-02459cb1b903\n",
  }, "")


Object with changed resourceVersion KubeadmControlPlane/quick-start-z8fdyo/quick-start-3biz68-control-plane:
  strings.Join({
  	... // 5191 identical bytes
  	"eReplicas: {}\n        f:version: {}\n    manager: manager\n    ope",
  	"ration: Update\n    subresource: status\n    time: \"2025-06-08T08:",
- 	"28:25",
+ 	"30:04",
  	"Z\"\n  name: quick-start-3biz68-control-plane\n  namespace: quick-s",
  	"tart-z8fdyo\n  ownerReferences:\n  - apiVersion: cluster.x-k8s.io/",
  	"v1beta2\n    blockOwnerDeletion: true\n    controller: true\n    ki",
  	"nd: Cluster\n    name: quick-start-3biz68\n    uid: e4cc243a-1d13-",
  	"46b0-a40d-02459cb1b903\n  resourceVersion: \"1",
- 	"5162",
+ 	"6745",
  	"\"\n  uid: c022a22b-499f-45af-8a71-1feb8630c867\n",
  }, "")


Object with changed resourceVersion Machine/quick-start-z8fdyo/quick-start-3biz68-control-plane-kntnl:
  strings.Join({
  	... // 6458 identical bytes
  	"dGeneration: {}\n        f:phase: {}\n    manager: manager\n    ope",
  	"ration: Update\n    subresource: status\n    time: \"2025-06-08T08:",
- 	"28:25",
+ 	"30:04",
  	"Z\"\n  name: quick-start-3biz68-control-plane-kntnl\n  namespace: q",
  	"uick-start-z8fdyo\n  ownerReferences:\n  - apiVersion: controlplan",
  	... // 89 identical bytes
  	"beadmControlPlane\n    name: quick-start-3biz68-control-plane\n   ",
  	" uid: c022a22b-499f-45af-8a71-1feb8630c867\n  resourceVersion: \"1",
- 	"5164",
+ 	"6736",
  	"\"\n  uid: 5a08802e-852a-459a-a007-ee24195c6279\n",
  }, "")


Expected object to be comparable, diff:   map[string]string{
- 	"Cluster/quick-start-z8fdyo/quick-start-3biz68":                   "16747",
+ 	"Cluster/quick-start-z8fdyo/quick-start-3biz68":                   "15172",
  	"ClusterResourceSet/quick-start-z8fdyo/quick-start-3biz68-crs-0":  "15121",
  	"ClusterResourceSetBinding/quick-start-z8fdyo/quick-start-3biz68": "14989",
  	... // 7 identical entries
  	"KubeadmConfig/quick-start-z8fdyo/quick-start-3biz68-md-0-xp6gp-c9v8n":    "15155",
  	"KubeadmConfigTemplate/quick-start-z8fdyo/quick-start-3biz68-md-0":        "15006",
- 	"KubeadmControlPlane/quick-start-z8fdyo/quick-start-3biz68-control-plane": "16745",
+ 	"KubeadmControlPlane/quick-start-z8fdyo/quick-start-3biz68-control-plane": "15162",
- 	"Machine/quick-start-z8fdyo/quick-start-3biz68-control-plane-kntnl":       "16736",
+ 	"Machine/quick-start-z8fdyo/quick-start-3biz68-control-plane-kntnl":       "15164",
  	"Machine/quick-start-z8fdyo/quick-start-3biz68-md-0-xp6gp-c9v8n":          "15168",
  	"MachineDeployment/quick-start-z8fdyo/quick-start-3biz68-md-0":            "15163",
  	... // 8 identical entries
  }
In [It] at: /home/prow/go/src/sigs.k8s.io/cluster-api/test/framework/resourceversion_helpers.go:60 @ 06/08/25 08:30:05.297
}

Anything else we need to know?

No response

Label(s) to be applied

/kind failing-test
One or more /area label. See https://github.com/kubernetes-sigs/cluster-api/labels?q=area for the list of labels.

[sbueringer]

I assume this is happening more often on main?

(I'll try to look into the main failures soon)

[sbueringer]

This should fix a few of those cases on main: #12341

Please let me know if it still occurs on main after #12341 is merged

[chrischdi]

/help

[k8s-ci-robot]

@chrischdi:
This request has been marked as needing help from a contributor.

Guidelines

Please ensure that the issue body includes answers to the following questions:

• Why are we solving this issue?
• To address this issue, are there any code changes? If there are code changes, what needs to be done in the code and what places can the assignee treat as reference points?
• How can the assignee reach out to you for help?

For more details on the requirements of such an issue, please see here and ensure that they are met.

If this request no longer meets these requirements, the label can be removed
by commenting with the /remove-help command.

In response to this:

/help

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.