Skip to content

Add .to_icechunk() method to ManifestGroup #591

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 2 commits into
base: develop
Choose a base branch
from

Conversation

TomNicholas
Copy link
Member

@TomNicholas TomNicholas commented May 11, 2025

This (WIP) PR adds a to_icechunk() method directly on ManifestGroup, and refactors the .to_icechunk() virtual dataset accessor method to go via the new ManifestGroup method.

The point is

  1. To further modularize the code by providing the dual of going from ManifestStore to virtual dataset, by adding the ability to go back from a virtual dataset to a ManifestStore,
  2. To therefore allow users to create and persist virtual references for a single file without ever using xarray, i.e. enable Make xarray an optional dependency? #521.

After this PR we should do the same for .to_kerchunk() - at that point we could actually make xarray an optional dependency if we want to.

  • Closes first step of Make xarray an optional dependency? #521
  • Tests added
  • Tests passing
  • Full type hint coverage
  • Changes are documented in docs/releases.rst
  • New functions/methods are listed in api.rst
  • New functionality has documentation


return cls(arrays=manifestarrays, attributes=attributes)

def to_icechunk(
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The point of adding this on ManifestGroup instead of ManifestStore initially is to make it easier to compose to deal with virtual datatrees later.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Icechunk 🧊 Relates to Icechunk library / spec internals
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant