What would you like to be added (User Story)?

As an operator, I expect ClusterCache to support uncached clients to reduce unnecessary memory usage.

Detailed Description

Currently, ClusterCache's GetClient method returns the cached client. While we can specify which objects are not cached using the DisableFor, this is cumbersome if we want to cache some objects but not most. It's easy to forget to configure the DisableFor, causing some objects to be cached and consuming unnecessary memory.

Therefore, I think ClusterCache can support uncached clients.

Anything else you would like to add?

No response

Label(s) to be applied

/kind feature
One or more /area label. See https://github.com/kubernetes-sigs/cluster-api/labels?q=area for the list of labels.