Repository navigation
[FEATURE][Java] Add the GraphAr write facade #973
Description
Describe the enhancement requested
Add the GraphAr write facade that turns a stream of edges into a published GraphAr dataset: vertex and edge chunk files, ordered adjacency, offset chunks, and the metadata that describes them. This is the layer directly above graphar-core (#967) and the physical write contract in graphar-io-api (#959), and it is what makes a Java-produced dataset readable by the C++ and Spark implementations.
Component(s): Java
Scope
- Add
graphar-writerwith aGraphWriterfacade that accepts edge records and emits a complete GraphAr dataset through the existingPhysicalWritercontract. - Cover edge write options (chunk size, ordering, adjacency list type) and reported write statistics.
- Keep the writer format-neutral: it must talk to
graphar-io-apionly, never to a concrete format module. - Add unit tests for chunk boundary emission, offset chunk correctness, and rejected option combinations.
Non-goals
No new format backend, no Arrow dependency, no reader facade, no query semantics, and no changes to the physical write contract itself.
Compatibility and acceptance
Output must be byte-compatible with the canonical GraphAr layout: a dataset written by this module loads in the existing implementations without conversion. Java 11 compatible.
Dependency order
This slice depends on graphar-core (#967, PR #968) and on a merged physical format backend (#964) to have anything to write through. It should land after both.
Activity
keksmd commentedon Sep 9, 2026
Dependency check before I cut this slice, so the ordering is agreed rather than discovered in review.
The writer's own sources need only modules that are already on
main:graphar-core,graphar-io-api,graphar-storage-api. Nothing insrc/mainreaches for a reader or a format backend.The block is entirely in test scope. Of the eight test classes, six import
org.apache.graphar.reader, and every one of them verifies a write by reading the bytes back through a real Parquet backend. So a writer PR today would either arrive with no meaningful verification, or drag in bothgraphar-readerandgraphar-io-parquet— which is the dependent chain we have been avoiding.There is also a size problem worth naming:
GraphWriteris ~1600 lines in one class. That is too much for a single reviewable slice regardless of dependencies, so I plan to split it — chunk placement and offset maintenance separately from the property/topology write paths.Proposed order, which I would like to agree on before opening anything:
io-parquet([FEATURE][Java] Adapt Parquet physical IO to columnar batches #964) — the shared gate; every downstream module verifies through it. Blocked only on feat(java): let a read request name what it wants pushed down #971 and feat(java): make a declined pushdown visible instead of silent #972.reader— needed by the writer's round-trip tests and by the validator.writer(this issue), split as above.validator([FEATURE][Java] Add a GraphAr dataset validator #974).@SemyonSinchenko does that order work for you, and would you rather see the writer split the way I described or along a different seam? Happy to reshape it before writing the PR rather than after.