Skip to content

Conversation

TomerNewman
Copy link
Contributor

Because Kaniko is archived and thus not supported, it would be wise to change the build and sign feature to use Buildah container instead of Kaniko.
That way we can also match the build and sign d/s to behave the same.


/cc @ybettan @yevgeny-shnaidman

@k8s-ci-robot k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Aug 28, 2025
Copy link

netlify bot commented Aug 28, 2025

Deploy Preview for kubernetes-sigs-kmm ready!

Name Link
🔨 Latest commit 8372fbc
🔍 Latest deploy log https://app.netlify.com/projects/kubernetes-sigs-kmm/deploys/68b7f5593721880008dc06fc
😎 Deploy Preview https://deploy-preview-1177--kubernetes-sigs-kmm.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@k8s-ci-robot k8s-ci-robot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Aug 28, 2025
@k8s-ci-robot
Copy link
Contributor

Hi @TomerNewman. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-ci-robot k8s-ci-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Aug 28, 2025
@ybettan
Copy link
Contributor

ybettan commented Aug 28, 2025

/ok-to-test

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Aug 28, 2025
@codecov-commenter
Copy link

codecov-commenter commented Aug 28, 2025

Codecov Report

❌ Patch coverage is 68.00000% with 24 lines in your changes missing coverage. Please review.
✅ Project coverage is 73.39%. Comparing base (fa23a9b) to head (8372fbc).
⚠️ Report is 324 commits behind head on main.

Files with missing lines Patch % Lines
internal/buildsign/resource/common.go 72.72% 11 Missing and 7 partials ⚠️
internal/module/buildargoverrider.go 0.00% 6 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1177      +/-   ##
==========================================
- Coverage   79.09%   73.39%   -5.70%     
==========================================
  Files          51       66      +15     
  Lines        5109     5894     +785     
==========================================
+ Hits         4041     4326     +285     
- Misses        882     1400     +518     
+ Partials      186      168      -18     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@TomerNewman TomerNewman force-pushed the buildah branch 2 times, most recently from b9ad744 to ab20c33 Compare August 31, 2025 12:37
@yevgeny-shnaidman
Copy link
Contributor

/approve

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: TomerNewman, yevgeny-shnaidman

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 31, 2025
@k8s-ci-robot k8s-ci-robot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Sep 1, 2025
@TomerNewman TomerNewman force-pushed the buildah branch 2 times, most recently from 6febb63 to 6a384a0 Compare September 1, 2025 12:14
@k8s-ci-robot k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Sep 1, 2025
@TomerNewman TomerNewman changed the title [WIP] Changing build and sign to use Buildah instead of Kaniko container Changing build and sign to use Buildah instead of Kaniko container Sep 2, 2025
@k8s-ci-robot k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Sep 2, 2025
fieldPath: metadata.namespace
- name: RELATED_IMAGE_BUILD
value: gcr.io/kaniko-project/executor:latest
value: quay.io/buildah/stable:latest
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this the official image that is being used by the u/s Shipwright project?
(I don't remember if when we tested it it was with the u/s version of Shipwright)

Copy link
Contributor Author

@TomerNewman TomerNewman Sep 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

they use in their samples quay.io/containers/buildah:v1.40.1
and bump it each time.
shipwright-io/build@10936ee

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in their official docs (buildah)
container images will be available at https://quay.io/repository/buildah/stable and https://quay.io/repository/containers/buildah.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ack. Then let's use the official one as you did. Is there a way to use a specific version instead of latest? I think we should use v1.41.3 instead of latest - we don't want things to start breaking under our legs.

@ybettan
Copy link
Contributor

ybettan commented Sep 2, 2025

What will all the rest of the kaniko refrences?

api/v1beta1/module_types.go:type KanikoParams struct {
api/v1beta1/module_types.go:    // Kaniko image tag to use when creating the build Pod
api/v1beta1/module_types.go:    // KanikoParams is used to customize the building process of the image.
api/v1beta1/module_types.go:    KanikoParams *KanikoParams `json:"kanikoParams,omitempty"`
api/v1beta1/zz_generated.deepcopy.go:   if in.KanikoParams != nil {
api/v1beta1/zz_generated.deepcopy.go:           in, out := &in.KanikoParams, &out.KanikoParams
api/v1beta1/zz_generated.deepcopy.go:           *out = new(KanikoParams)
api/v1beta1/zz_generated.deepcopy.go:func (in *KanikoParams) DeepCopyInto(out *KanikoParams) {
api/v1beta1/zz_generated.deepcopy.go:// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new KanikoParams.
api/v1beta1/zz_generated.deepcopy.go:func (in *KanikoParams) DeepCopy() *KanikoParams {
api/v1beta1/zz_generated.deepcopy.go:   out := new(KanikoParams)
ci/module-kmm-ci-build-sign.yaml:            # Optional. If kanikoParams.tag is empty, the default value will be: 'latest'
ci/module-kmm-ci-build-sign.yaml:            kanikoParams:
config/crd-hub/bases/hub.kmm.sigs.x-k8s.io_managedclustermodules.yaml:                              kanikoParams:
config/crd-hub/bases/hub.kmm.sigs.x-k8s.io_managedclustermodules.yaml:                                description: KanikoParams is used to customize the
config/crd-hub/bases/hub.kmm.sigs.x-k8s.io_managedclustermodules.yaml:                                    description: Kaniko image tag to use when creating
config/crd-hub/bases/hub.kmm.sigs.x-k8s.io_managedclustermodules.yaml:                                    kanikoParams:
config/crd-hub/bases/hub.kmm.sigs.x-k8s.io_managedclustermodules.yaml:                                      description: KanikoParams is used to customize
config/crd-hub/bases/hub.kmm.sigs.x-k8s.io_managedclustermodules.yaml:                                          description: Kaniko image tag to use when
config/crd-hub/bases/kmm.sigs.x-k8s.io_modulebuildsignconfigs.yaml:                        kanikoParams:
config/crd-hub/bases/kmm.sigs.x-k8s.io_modulebuildsignconfigs.yaml:                          description: KanikoParams is used to customize the building
config/crd-hub/bases/kmm.sigs.x-k8s.io_modulebuildsignconfigs.yaml:                              description: Kaniko image tag to use when creating the
config/crd-hub/bases/kmm.sigs.x-k8s.io_moduleimagesconfigs.yaml:                        kanikoParams:
config/crd-hub/bases/kmm.sigs.x-k8s.io_moduleimagesconfigs.yaml:                          description: KanikoParams is used to customize the building
config/crd-hub/bases/kmm.sigs.x-k8s.io_moduleimagesconfigs.yaml:                              description: Kaniko image tag to use when creating the
config/crd-hub/bases/kmm.sigs.x-k8s.io_modules.yaml:                          kanikoParams:
config/crd-hub/bases/kmm.sigs.x-k8s.io_modules.yaml:                            description: KanikoParams is used to customize the building
config/crd-hub/bases/kmm.sigs.x-k8s.io_modules.yaml:                                description: Kaniko image tag to use when creating
config/crd-hub/bases/kmm.sigs.x-k8s.io_modules.yaml:                                kanikoParams:
config/crd-hub/bases/kmm.sigs.x-k8s.io_modules.yaml:                                  description: KanikoParams is used to customize the
config/crd-hub/bases/kmm.sigs.x-k8s.io_modules.yaml:                                      description: Kaniko image tag to use when creating
config/crd/bases/kmm.sigs.x-k8s.io_modulebuildsignconfigs.yaml:                        kanikoParams:
config/crd/bases/kmm.sigs.x-k8s.io_modulebuildsignconfigs.yaml:                          description: KanikoParams is used to customize the building
config/crd/bases/kmm.sigs.x-k8s.io_modulebuildsignconfigs.yaml:                              description: Kaniko image tag to use when creating the
config/crd/bases/kmm.sigs.x-k8s.io_moduleimagesconfigs.yaml:                        kanikoParams:
config/crd/bases/kmm.sigs.x-k8s.io_moduleimagesconfigs.yaml:                          description: KanikoParams is used to customize the building
config/crd/bases/kmm.sigs.x-k8s.io_moduleimagesconfigs.yaml:                              description: Kaniko image tag to use when creating the
config/crd/bases/kmm.sigs.x-k8s.io_modules.yaml:                          kanikoParams:
config/crd/bases/kmm.sigs.x-k8s.io_modules.yaml:                            description: KanikoParams is used to customize the building
config/crd/bases/kmm.sigs.x-k8s.io_modules.yaml:                                description: Kaniko image tag to use when creating
config/crd/bases/kmm.sigs.x-k8s.io_modules.yaml:                                kanikoParams:
config/crd/bases/kmm.sigs.x-k8s.io_modules.yaml:                                  description: KanikoParams is used to customize the
config/crd/bases/kmm.sigs.x-k8s.io_modules.yaml:                                      description: Kaniko image tag to use when creating
docs/mkdocs/documentation/kmod_image.md:Otherwise, KMM will create a Pod to build your image using [kaniko](https://github.com/GoogleContainerTools/kaniko).
internal/buildsign/resource/common_test.go:             kanikoImage             = "some-kaniko-image:some-tag"
internal/buildsign/resource/common_test.go:             GinkgoT().Setenv(relatedImageEnvVar, kanikoImage)
internal/buildsign/resource/common_test.go:                                             Image:   kanikoImage,
internal/buildsign/resource/common_test.go:             buildImage    = "some-kaniko-image:some-tag"

template.ParseFS(templateFS, "templates/Dockerfile.gotmpl"),
)

func formatBuildArgs(buildArgs []kmmv1beta1.BuildArg) string {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we put this method in the BuildArgOverrider interface?

Comment on lines 119 to 122
const (
Build BuildOperation = iota
Sign
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use the API's BuildOrSignAction instead of defining a new "type"?

pushDescription = "signed image"
}

script := fmt.Sprintf(`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any reason we are not using go templates? Like we did until now for signing?

@TomerNewman
Copy link
Contributor Author

What will all the rest of the kaniko refrences?

api/v1beta1/module_types.go:type KanikoParams struct {
api/v1beta1/module_types.go:    // Kaniko image tag to use when creating the build Pod
api/v1beta1/module_types.go:    // KanikoParams is used to customize the building process of the image.
api/v1beta1/module_types.go:    KanikoParams *KanikoParams `json:"kanikoParams,omitempty"`
api/v1beta1/zz_generated.deepcopy.go:   if in.KanikoParams != nil {
api/v1beta1/zz_generated.deepcopy.go:           in, out := &in.KanikoParams, &out.KanikoParams
api/v1beta1/zz_generated.deepcopy.go:           *out = new(KanikoParams)
api/v1beta1/zz_generated.deepcopy.go:func (in *KanikoParams) DeepCopyInto(out *KanikoParams) {
api/v1beta1/zz_generated.deepcopy.go:// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new KanikoParams.
api/v1beta1/zz_generated.deepcopy.go:func (in *KanikoParams) DeepCopy() *KanikoParams {
api/v1beta1/zz_generated.deepcopy.go:   out := new(KanikoParams)
ci/module-kmm-ci-build-sign.yaml:            # Optional. If kanikoParams.tag is empty, the default value will be: 'latest'
ci/module-kmm-ci-build-sign.yaml:            kanikoParams:
config/crd-hub/bases/hub.kmm.sigs.x-k8s.io_managedclustermodules.yaml:                              kanikoParams:
config/crd-hub/bases/hub.kmm.sigs.x-k8s.io_managedclustermodules.yaml:                                description: KanikoParams is used to customize the
config/crd-hub/bases/hub.kmm.sigs.x-k8s.io_managedclustermodules.yaml:                                    description: Kaniko image tag to use when creating
config/crd-hub/bases/hub.kmm.sigs.x-k8s.io_managedclustermodules.yaml:                                    kanikoParams:
config/crd-hub/bases/hub.kmm.sigs.x-k8s.io_managedclustermodules.yaml:                                      description: KanikoParams is used to customize
config/crd-hub/bases/hub.kmm.sigs.x-k8s.io_managedclustermodules.yaml:                                          description: Kaniko image tag to use when
config/crd-hub/bases/kmm.sigs.x-k8s.io_modulebuildsignconfigs.yaml:                        kanikoParams:
config/crd-hub/bases/kmm.sigs.x-k8s.io_modulebuildsignconfigs.yaml:                          description: KanikoParams is used to customize the building
config/crd-hub/bases/kmm.sigs.x-k8s.io_modulebuildsignconfigs.yaml:                              description: Kaniko image tag to use when creating the
config/crd-hub/bases/kmm.sigs.x-k8s.io_moduleimagesconfigs.yaml:                        kanikoParams:
config/crd-hub/bases/kmm.sigs.x-k8s.io_moduleimagesconfigs.yaml:                          description: KanikoParams is used to customize the building
config/crd-hub/bases/kmm.sigs.x-k8s.io_moduleimagesconfigs.yaml:                              description: Kaniko image tag to use when creating the
config/crd-hub/bases/kmm.sigs.x-k8s.io_modules.yaml:                          kanikoParams:
config/crd-hub/bases/kmm.sigs.x-k8s.io_modules.yaml:                            description: KanikoParams is used to customize the building
config/crd-hub/bases/kmm.sigs.x-k8s.io_modules.yaml:                                description: Kaniko image tag to use when creating
config/crd-hub/bases/kmm.sigs.x-k8s.io_modules.yaml:                                kanikoParams:
config/crd-hub/bases/kmm.sigs.x-k8s.io_modules.yaml:                                  description: KanikoParams is used to customize the
config/crd-hub/bases/kmm.sigs.x-k8s.io_modules.yaml:                                      description: Kaniko image tag to use when creating
config/crd/bases/kmm.sigs.x-k8s.io_modulebuildsignconfigs.yaml:                        kanikoParams:
config/crd/bases/kmm.sigs.x-k8s.io_modulebuildsignconfigs.yaml:                          description: KanikoParams is used to customize the building
config/crd/bases/kmm.sigs.x-k8s.io_modulebuildsignconfigs.yaml:                              description: Kaniko image tag to use when creating the
config/crd/bases/kmm.sigs.x-k8s.io_moduleimagesconfigs.yaml:                        kanikoParams:
config/crd/bases/kmm.sigs.x-k8s.io_moduleimagesconfigs.yaml:                          description: KanikoParams is used to customize the building
config/crd/bases/kmm.sigs.x-k8s.io_moduleimagesconfigs.yaml:                              description: Kaniko image tag to use when creating the
config/crd/bases/kmm.sigs.x-k8s.io_modules.yaml:                          kanikoParams:
config/crd/bases/kmm.sigs.x-k8s.io_modules.yaml:                            description: KanikoParams is used to customize the building
config/crd/bases/kmm.sigs.x-k8s.io_modules.yaml:                                description: Kaniko image tag to use when creating
config/crd/bases/kmm.sigs.x-k8s.io_modules.yaml:                                kanikoParams:
config/crd/bases/kmm.sigs.x-k8s.io_modules.yaml:                                  description: KanikoParams is used to customize the
config/crd/bases/kmm.sigs.x-k8s.io_modules.yaml:                                      description: Kaniko image tag to use when creating
docs/mkdocs/documentation/kmod_image.md:Otherwise, KMM will create a Pod to build your image using [kaniko](https://github.com/GoogleContainerTools/kaniko).
internal/buildsign/resource/common_test.go:             kanikoImage             = "some-kaniko-image:some-tag"
internal/buildsign/resource/common_test.go:             GinkgoT().Setenv(relatedImageEnvVar, kanikoImage)
internal/buildsign/resource/common_test.go:                                             Image:   kanikoImage,
internal/buildsign/resource/common_test.go:             buildImage    = "some-kaniko-image:some-tag"

I will look about the rest but regarding the CRD I thought that if we decide to change it, we can do it in a different PR.
Notice that we will need to change Module CRD for that, meaning breaking API

Because Kaniko is archived and thus not supported, it would
be wise to change the build and sign feature to use
buildah container instead of kaniko.
@TomerNewman TomerNewman changed the title Changing build and sign to use Buildah instead of Kaniko container [WIP] Changing build and sign to use Buildah instead of Kaniko container Sep 4, 2025
@k8s-ci-robot k8s-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Sep 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants