Skip to content

Conversation

@saikrishnanc-nv
Copy link
Collaborator

PhysicsNeMo Pull Request

Description

This PR adds a Zarr reader for Crash. A corresponding Zarr preprocessor/writer has been merged to PhysicsNeMo-Curator already.

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.
  • The CHANGELOG.md is up to date with these changes.
  • An issue is linked to this pull request.

Dependencies

Review Process

All PRs are reviewed by the PhysicsNeMo team before merging.

Depending on which files are changed, GitHub may automatically assign a maintainer for review.

We are also testing AI-based code review tools (e.g., Greptile), which may add automated comments with a confidence score.
This score reflects the AI’s assessment of merge readiness and is not a qualitative judgment of your work, nor is
it an indication that the PR will be accepted / rejected.

AI-generated feedback should be reviewed critically for usefulness.
You are not required to respond to every AI comment, but they are intended to help both authors and reviewers.
Please react to Greptile comments with 👍 or 👎 to provide feedback on their accuracy.

@saikrishnanc-nv saikrishnanc-nv self-assigned this Nov 13, 2025
@greptile-apps
Copy link
Contributor

greptile-apps bot commented Nov 13, 2025

Greptile Overview

Greptile Summary

Adds a Zarr reader for crash simulation data to complement existing d3plot and VTP readers. The reader loads pre-processed Zarr stores created by PhysicsNeMo-Curator, where computational work (node filtering, edge building) has already been done during ETL.

Key changes:

  • New zarr_reader.py with Reader class that loads mesh_pos, edges, and point data features from Zarr stores
  • Comprehensive validation for 1D [N] and 2D [N,K] feature arrays (addresses previous review feedback)
  • Natural sorting of Zarr stores for consistent ordering
  • Full test coverage including edge cases, validation logic, and 2D feature support
  • Updated documentation with Zarr format examples and PhysicsNeMo-Curator workflow
  • New Hydra configuration file conf/reader/zarr.yaml

All previously flagged issues with the validation logic have been resolved. The code now correctly handles both 1D and 2D feature arrays as documented.

Confidence Score: 5/5

  • This PR is safe to merge with no issues found
  • The implementation is clean, well-tested, and follows existing patterns in the codebase. All previous validation issues have been resolved, comprehensive tests cover edge cases and 2D features, documentation is thorough, and the code integrates seamlessly with the existing reader infrastructure
  • No files require special attention

Important Files Changed

File Analysis

Filename Score Overview
examples/structural_mechanics/crash/zarr_reader.py 5/5 New Zarr reader implementation with proper validation for 1D and 2D feature arrays, comprehensive error handling, and natural sorting
examples/structural_mechanics/crash/tests/test_zarr_reader.py 5/5 Comprehensive test suite covering edge cases, validation logic, 2D features, natural sorting, and error conditions
examples/structural_mechanics/crash/README.md 5/5 Updated documentation to describe Zarr format, reader usage, and PhysicsNeMo-Curator ETL workflow

Sequence Diagram

sequenceDiagram
    participant User
    participant Reader
    participant process_zarr_data
    participant find_zarr_stores
    participant load_zarr_store
    participant zarr as Zarr Store

    User->>Reader: __call__(data_dir, num_samples)
    Reader->>process_zarr_data: process_zarr_data(data_dir, num_samples, logger)
    process_zarr_data->>find_zarr_stores: find_zarr_stores(data_dir)
    find_zarr_stores->>find_zarr_stores: List *.zarr directories
    find_zarr_stores->>find_zarr_stores: Natural sort by name
    find_zarr_stores-->>process_zarr_data: List of zarr_paths
    
    loop For each zarr_path (up to num_samples)
        process_zarr_data->>load_zarr_store: load_zarr_store(zarr_path)
        load_zarr_store->>zarr: Open Zarr store
        load_zarr_store->>zarr: Read mesh_pos [T,N,3]
        load_zarr_store->>zarr: Read edges [E,2]
        load_zarr_store->>zarr: Read point data fields (thickness, etc.)
        zarr-->>load_zarr_store: mesh_pos, edges, point_data_dict
        load_zarr_store-->>process_zarr_data: mesh_pos, edges, point_data_dict
        
        process_zarr_data->>process_zarr_data: Validate mesh_pos shape [T,N,3]
        process_zarr_data->>process_zarr_data: Validate edges shape [E,2]
        process_zarr_data->>process_zarr_data: Validate point data (1D [N] or 2D [N,K])
        process_zarr_data->>process_zarr_data: Validate edge indices in bounds
        process_zarr_data->>process_zarr_data: Extract src, dst from edges
        process_zarr_data->>process_zarr_data: Build record with coords + features
    end
    
    process_zarr_data-->>Reader: srcs, dsts, point_data_all
    Reader-->>User: srcs, dsts, point_data
Loading

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

5 files reviewed, 1 comment

Edit Code Review Agent Settings | Greptile

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

5 files reviewed, 1 comment

Edit Code Review Agent Settings | Greptile

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

5 files reviewed, no comments

Edit Code Review Agent Settings | Greptile

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

5 files reviewed, 1 comment

Edit Code Review Agent Settings | Greptile

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

5 files reviewed, no comments

Edit Code Review Agent Settings | Greptile

@saikrishnanc-nv
Copy link
Collaborator Author

/blossom-ci

Copy link
Collaborator

@mnabian mnabian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

@saikrishnanc-nv
Copy link
Collaborator Author

/blossom-ci

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

5 files reviewed, no comments

Edit Code Review Agent Settings | Greptile

@saikrishnanc-nv saikrishnanc-nv merged commit 7277097 into NVIDIA:main Nov 14, 2025
1 check passed
@saikrishnanc-nv saikrishnanc-nv deleted the saikrishnanc/update_crash_zarr branch November 14, 2025 22:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants