In #12726 we first noticed that we weren't running all tests in the test/infrastructure folder.
Then when running all the tests we noticed that there are some race conditions in inmemory tests.

Because of this we removed the -race flag for test-infrastructure* targets for now: https://github.com/kubernetes-sigs/cluster-api/pull/12726/files#diff-76ed074a9305c04054cdebb9e9aad2d818052b07091de1f20cad0bbac34ffb52

The goal of this issue is to add -race to the test-infrastructure and test-infrastructure-junit targets and fix the race conditions.

Here is an example test run with the failures: https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/kubernetes-sigs_cluster-api/12726/pull-cluster-api-test-main/1966457623695855616 (but this can be easily reproduced by adding the -race flags again)