Skip to content

Conversation

hakman
Copy link
Member

@hakman hakman commented Aug 18, 2025

Iterating on updated Karpenter setup.
Includes:

  • storing a copy of the startup script that can be used later by kops-controller to generate the EC2NodeClass
  • disabling LaunchTemplate generation, which can no longer be used by Karpenter.

/cc @rifelpet @justinsb @ameukam

@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. area/addons labels Aug 18, 2025
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign johngmyers for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added area/api area/provider/aws Issues or PRs related to aws provider labels Aug 18, 2025
@hakman
Copy link
Member Author

hakman commented Aug 18, 2025

/override pull-kops-e2e-aws-upgrade-k129-ko129-to-k130-kolatest-karpenter

@k8s-ci-robot
Copy link
Contributor

@hakman: Overrode contexts on behalf of hakman: pull-kops-e2e-aws-upgrade-k129-ko129-to-k130-kolatest-karpenter

In response to this:

/override pull-kops-e2e-aws-upgrade-k129-ko129-to-k130-kolatest-karpenter

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@rifelpet
Copy link
Member

FYI I started working on a model that translates an InstanceGroup to EC2NodeClass and NodePool resources:

master...rifelpet:kops:karpenter-model

I was imagining that we'd add a new karpenter controller to kops-controller that periodically fetches the instance groups (and this new userdata object) from VFS, generates the model, and reconciles the resulting resources. It should be straight forward to add models for other cloud providers too.

@hakman
Copy link
Member Author

hakman commented Aug 19, 2025

FYI I started working on a model that translates an InstanceGroup to EC2NodeClass and NodePool resources:

master...rifelpet:kops:karpenter-model

I was imagining that we'd add a new karpenter controller to kops-controller that periodically fetches the instance groups (and this new userdata object) from VFS, generates the model, and reconciles the resulting resources. It should be straight forward to add models for other cloud providers too.

Very nice @rifelpet, I think that is exactly the missing piece. This PR adds generates a file with the user data, which is not that easy to generate separately. This way, everything needed for generating the EC2NodeClass is in the state store.

@hakman
Copy link
Member Author

hakman commented Aug 22, 2025

@rifelpet @justinsb could you take a look at this one, I think it should be ready.

@hakman hakman force-pushed the karpenter-1.6.2 branch 3 times, most recently from c8e16df to 5132f34 Compare August 22, 2025 17:13
@hakman
Copy link
Member Author

hakman commented Aug 22, 2025

/test pull-kops-verify-terraform

@kubernetes kubernetes deleted a comment from k8s-ci-robot Aug 23, 2025
@hakman
Copy link
Member Author

hakman commented Aug 23, 2025

/override pull-kops-e2e-aws-karpenter

@k8s-ci-robot
Copy link
Contributor

@hakman: Overrode contexts on behalf of hakman: pull-kops-e2e-aws-karpenter

In response to this:

/override pull-kops-e2e-aws-karpenter

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@hakman
Copy link
Member Author

hakman commented Aug 23, 2025

/test pull-kops-scenario-aws-karpenter

@hakman
Copy link
Member Author

hakman commented Aug 23, 2025

/test pull-kops-scenario-aws-karpenter

@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 23, 2025
@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 23, 2025
@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 29, 2025
@hakman hakman force-pushed the karpenter-1.6.2 branch 2 times, most recently from 40206be to a12909e Compare August 29, 2025 07:39
@hakman
Copy link
Member Author

hakman commented Aug 29, 2025

/test pull-kops-scenario-aws-karpenter

@hakman hakman requested a review from justinsb August 29, 2025 07:44
@hakman hakman marked this pull request as ready for review August 29, 2025 09:43
@k8s-ci-robot k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Aug 29, 2025
@hakman
Copy link
Member Author

hakman commented Aug 29, 2025

/test pull-kops-scenario-aws-karpenter

@hakman
Copy link
Member Author

hakman commented Aug 29, 2025

/test pull-kops-kubernetes-e2e-ubuntu-gce-build

@hakman
Copy link
Member Author

hakman commented Aug 29, 2025

/override pull-kops-e2e-aws-upgrade-k133-ko133-to-klatest-kolatest-many-addons

@k8s-ci-robot
Copy link
Contributor

@hakman: Overrode contexts on behalf of hakman: pull-kops-e2e-aws-upgrade-k133-ko133-to-klatest-kolatest-many-addons

In response to this:

/override pull-kops-e2e-aws-upgrade-k133-ko133-to-klatest-kolatest-many-addons

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@hakman
Copy link
Member Author

hakman commented Aug 29, 2025

@justinsb @rifelpet @ameukam please take another look, I think it's ready to merge.
I will add more updates to the docs and release notes separately.

@ameukam
Copy link
Member

ameukam commented Aug 29, 2025

/lgtm
/hold

If others have opinions.
I would have expected that peter's get merged in this PR before we merge it.

@k8s-ci-robot k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Aug 29, 2025
@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Aug 29, 2025
@ameukam
Copy link
Member

ameukam commented Aug 29, 2025

/retest

@k8s-ci-robot
Copy link
Contributor

@hakman: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
pull-kops-kubernetes-e2e-ubuntu-gce-build 64a8e79 link false /test pull-kops-kubernetes-e2e-ubuntu-gce-build

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@hakman
Copy link
Member Author

hakman commented Aug 30, 2025

I would have expected that peter's get merged in this PR before we merge it.

@ameukam I think we decided to merge and wait for Peter's PR another 3 weeks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/addons area/api area/documentation area/provider/aws Issues or PRs related to aws provider cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. lgtm "Looks good to me", indicates that a PR is ready to be merged. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants