Provide configurable `useCaches` option for `URLConnection` usage in `UrlResource` (avoiding jar file leak)

[reckart]

A variation of #34528 seems to be back again with Spring 6.2.9 / 6.2.8.

Spring 6.2.7 was good and Resource.exists() worked without a file leak.

Here is the stack with the new version created again with the https://github.com/jenkinsci/lib-file-leak-detector

... testfiles.zip by thread:ForkJoinPool-1-worker-1 
at java.base/java.util.zip.ZipFile.<init>(ZipFile.java:173)
at java.base/java.util.jar.JarFile.<init>(JarFile.java:347)
at java.base/sun.net.www.protocol.jar.URLJarFile.<init>(URLJarFile.java:103)
at java.base/sun.net.www.protocol.jar.URLJarFile.getJarFile(URLJarFile.java:72)
at java.base/sun.net.www.protocol.jar.JarFileFactory.getOrCreate(JarFileFactory.java:106)
at java.base/sun.net.www.protocol.jar.JarURLConnection.connect(JarURLConnection.java:132)
at java.base/sun.net.www.protocol.jar.JarURLConnection.getJarFile(JarURLConnection.java:92)
at org.springframework.core.io.AbstractFileResolvingResource.exists(AbstractFileResolvingResource.java:92)

The relevant change seems to be this:

a266e1b

I don't see how as I user I could influence the resource to tell it to avoid using caches.

[jhoeller]

That change was necessary to fix the regression reported in #34955, in order to not disagree with the JVM's own jar caching assumptions. You'll have to turn off jar caching for the entire JVM through URLConnection.setDefaultUseCaches("jar", false);.

[reckart]

@jhoeller Hm, I believe what you are saying is that if I obtain multiple JarURLConnection to the same JAR, it wouldn't be possible to set different caching states on them?

[jhoeller]

With the JVM's rather convoluted URLConnection architecture, I would not recommend to mix and match the caching state there. A resource which tries to close the JarFile can pull the file from underneath a cached jar accessor otherwise, unfortunately. We had two subsequent regressions reported against that original change, and the only outcome that consistently worked for all scenarios was a JVM-level setting that we explicitly align with now.

[reckart]

So adding a Resource.setUseCache(boolean) to control whether connections should be set to disable caching before the connection is used wouldn't be reasonable?

If/when I use UrlConnections outside spring, that's typically what I'd be doing. Get the connection, disable caching, use it, close it. Because caching was turned off before I first use the connection, it would open its own private JarFile (or whatever) so closing it afterwards shouldn't affect any shared state.

[jhoeller]

Indeed, the problem just shows when trying to manage the JarFile state on top of JVM-level caching. Only with useCaches consistently disabled for all common access paths to the same resource, such explicit jar file management on Spring's side is actually possible. Trying to be defensive for exists() calls only does not seem to be feasible (see that regression report where an exists() call was interleaved with an open InputStream).

We could allow for turning off useCaches at the individual resource level but I don't see how common resource access code would manually set such a resource-level flag... in a consistent enough fashion for that to reliably work in practice. And hard-turning it off for all Spring access paths is not an option either since we do want to benefit from jar caching when enabled at the JVM level.

We have a setUseCaches setting at PathMatchingResourcePatternResolver level but that's for a discovery run across the entire classpath, so a little different in purpose and scope.

Theoretically, we could introduce a Spring-level property (as a system property and for spring.properties) that suppresses caching for all Spring jar access code. However, I have a hard time seeing why you wouldn't turn off default jar caching for the entire JVM at that point, otherwise there's always a risk that some non-Spring access code creates a jar leak on the side.

[reckart]

I'm already setting setUseCaches() to false on the PathMatchingResourcePatternResolver. Would it make sense if any Resources obtained through a resolver with disabled caching would inherit that state?

However, I have a hard time seeing why you wouldn't turn off default jar caching for the entire JVM at that point, otherwise there's always a risk that some non-Spring access code creates a jar leak on the side.

Basically, in the environment I am working, we have ZIPs and JARs. JARs are typically static. ZIPs may be replaced. It is quite convenient to be able to access both through the jar: protocol. I wouldn't want to disable JAR caching across the board because I fear that would negatively affect performance. However, when I use PathMatchingResourcePatternResolver with the jar: protocol to search stuff in ZIP files, I do not want caching because thos ZIPs have a different life cycle. Also, when I create archives (ZIP/JAR) during unit tests in temporary folders, I can't delete those temp folders on Windows after the test because the files are still open due to the caching mechanism.

[jhoeller]

Good point, we could extend the semantics of PathMatchingResourcePatternResolver.setUseCaches that way, applying to later access from any Resource handles returned by it as well. That might be a sweet spot indeed, I'll see what we can do there.