[#714] feat(Java): Chunk Info Reader #714
Conversation
sapienza88
changed the title
| return false; | ||
| } | ||
|
|
||
| public String getPropertyGroupChunkPath(PropertyGroup propertyGroup, long chunkIndex) { |
There was a problem hiding this comment.
Why move from VertexInfo.java to here
There was a problem hiding this comment.
it seems to be more useful in this class
There was a problem hiding this comment.
Putting related functions in the same file helps organize the code better, increases cohesion, reduces coupling, and makes future maintenance easier.
There was a problem hiding this comment.
Yes it is already refactored
| // TODO check equality test for type | ||
| String type = propertyGroup.getFileType().toString(); | ||
| numberOfParts = vertexInfo.getChunkSize() | ||
| chunkBasePath = vertexInfo.getPropertyGroupPrefix() + "/part"; |
There was a problem hiding this comment.
Is "part" or "chunk"? Please confirm.
There was a problem hiding this comment.
I don't see "chunk" for vertex logical table here https://graphar.apache.org/docs/specification/format/#physical-table-of-vertices
There was a problem hiding this comment.
Apologies — the docs are a bit outdated. I’ll update them as soon as possible.
In the meantime, you can check the latest data format in this repo: https://github.com/apache/incubator-graphar-testing
maven-projects/info/src/main/java/org/apache/graphar/info/FileReader.java
Outdated
Show resolved
Hide resolved
How to know about "m"? the count of the data file associated with the vertex? |
It can be read from vertex_count. |
|
@yangxk1 pls provide review and merge |
|
@yangxk1 pls allow CI to re-trigger automatically when a new code is commited by me so that I don't have to wait for you to do it manually. Thanks. |
To ensure security, workflow execution for first-time contributors requires approval from a project committer. You can change the |
|
@yangxk1 thanks for merging the PR on the version parsing, pls also do the same for this PR to let us merge this. PS: Allow edits by maintainers is enabled. |
This PR does not use protobuf, so it is not that urgent. I will still help you as soon as possible. |
|
@yangxk1 pls approve and merge this or let me know the changes required to merge before the next release so it can be included in the next release. |
|
Hi @unical1988 ,I think you should open an issue to discuss the necessity of this pr. |
|
@yangxk1 i already describe here whats the pr intended for, couldnt be clearer |
|
You need to think about these:
Open an Issue focus on discussion rather than description. This is also the specification of the CONTRIBUTING document. |
|
@yangxk1 i can and will add tests anything else has been discussed here, pls note that this is implemented in C++ |
|
2 participants
This PR introduces a set of utility methods designed to efficiently read information pertaining to graph vertex chunks. These utilities will streamline the process of accessing and interpreting how vertices are grouped and stored, which is crucial for optimizing graph processing and analysis tasks.
Specifically, this PR provides:
Methods to query and retrieve metadata about vertex chunks.
Functions to read the boundaries and contents of individual vertex chunks.
Utilities to facilitate the navigation and processing of vertex data in a chunked manner.
This enhancement will benefit features requiring granular access to graph vertex data, such as distributed graph algorithms, incremental graph updates, and optimized data loading.