java.io.IOException: java.lang.NullPointerException: Cannot invoke "java.lang.Short.intValue()" because "oid" is null

[runnerzz]

When attempting to read a file in zarr format, an error occurred. The error message is similar to that of the.zarry file in the zarr format as shown in the figure. Could you please help reply and explain the reason for this error?

[lesserwhirls]

Greetings @runnerzz!

The underlying issue is that the netCDF-Java Zarr implementation does not handle the compressor id gzip (but it does handle zlib). We'd need to add support for gzip specifically.

[runnerzz]

`
import xarray as xr
from numcodecs import Zlib
import os

def nc_to_zarr_zlib(nc_file, zarr_out, level=1):

ds = xr.open_dataset(nc_file, decode_cf=True, engine="netcdf4")

if os.path.exists(zarr_out):
    import shutil
    shutil.rmtree(zarr_out)

compressor = Zlib(level=level)

encoding = {}
for var in ds.data_vars:
    encoding[var] = {"compressor": compressor}

print(f"write Zarr (zlib, level={level}) to: {zarr_out}")
ds.to_zarr(zarr_out, mode="w", encoding=encoding)
print("✅ finish！")

`

The code is as above.
Do you have any conversion scripts available? Could you please provide them?

[lesserwhirls]

I do not have any conversion scripts available. Is there any chance you could upload a copy of both zarr datasets? I suspect this has to do with a difference in what zlib "compression" means in python/Java/C (likely comes down to extra bytes header/footer bytes).

[runnerzz]

Excuse me again, apart from supporting the zlib compressor, which other compressors can be used?

[lesserwhirls]

zlib is the only compressor supported (more specifically, the ZLIB Compressed Data Format). I'm looking to add support for blosc in the near future, but that requires building / distributing native code for ease-of-use.

Can you tell me which versions of zarr, xarray, and numcodecs you are using?