Skip to content

[Bug] stale resource not cleaned up #2351

Open
@nandiheath

Description

@nandiheath

chainsaw version Version

v0.2.4

Description

We're running some repetitive tests that reuse the same resource name. Normally, the resource is cleaned up once the test completes.However if the test exits unexpectedly, the cleanup may not occur, leaving the resource behind. When this happens, subsequent test runs don't attempt to clean up the existing resource, resulting in it being left stale indefinitely.

I’m wondering if this is because the current cleanup logic only considers resources that were explicitly created during the test run.

Steps to reproduce

  1. Start a test that creates a PVC and a Job which mounts it.
  2. Interrupt the test after the PVC is created.
  3. Run the test again — the previously created PVC will now be left stale after the test completes.

Expected behavior

All resources should be cleaned up after the test.

Screenshots

No response

Logs

The log for the second test showing the PVC is `PATCH` instead of `CREATE`

    | 09:00:28 | ebs-csi-driver | ephemeral storageClass     | TRY       | BEGIN |
    | 09:00:28 | ebs-csi-driver | ephemeral storageClass     | APPLY     | RUN   | storage.k8s.io/v1/StorageClass @ chainsaw-ebs-csi-driver-kuberhealthy
    | 09:00:29 | ebs-csi-driver | ephemeral storageClass     | PATCH     | OK    | storage.k8s.io/v1/StorageClass @ chainsaw-ebs-csi-driver-kuberhealthy
    | 09:00:29 | ebs-csi-driver | ephemeral storageClass     | APPLY     | DONE  | storage.k8s.io/v1/StorageClass @ chainsaw-ebs-csi-driver-kuberhealthy
    | 09:00:29 | ebs-csi-driver | ephemeral storageClass     | TRY       | END   |
    | 09:00:29 | ebs-csi-driver | persistent volume claim    | TRY       | BEGIN |
    | 09:00:29 | ebs-csi-driver | persistent volume claim    | APPLY     | RUN   | v1/PersistentVolumeClaim @ kube-system/chainsaw-ebs-csi-driver-kuberhealthy
    | 09:00:29 | ebs-csi-driver | persistent volume claim    | PATCH     | OK    | v1/PersistentVolumeClaim @ kube-system/chainsaw-ebs-csi-driver-kuberhealthy
    | 09:00:29 | ebs-csi-driver | persistent volume claim    | APPLY     | DONE  | v1/PersistentVolumeClaim @ kube-system/chainsaw-ebs-csi-driver-kuberhealthy
    | 09:00:29 | ebs-csi-driver | persistent volume claim    | CMD       | RUN   |

Slack discussion

No response

Troubleshooting

  • I have searched other issues in this repository and mine is not recorded.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions