From be2212cd60331ea9e1dff27e68b4e38250cf047c Mon Sep 17 00:00:00 2001 From: DavIvek Date: Fri, 31 Jul 2026 14:40:18 +0200 Subject: [PATCH 1/5] docs(memgql): cache attribute requirement + element ids as properties Two user-visible changes from the cache and ClickHouse fixes in memgraph/zero#71, plus the changelog entry for that work. Caching now needs every cached label to declare `attributes`. The cache copies only the columns the mapping names, so a label declaring none would be cached id-only and every property read back would be NULL; MemGQL declines to cache rather than serve those rows. This is the one thing that tripped an external reporter whose mapping was otherwise correct, so it is documented both where caching is set up and next to the existing routing note in the schema-file reference, which already covers the parallel "declared attributes required" rule. `metaFields.id` is now exposed as a property on vertices and edges, so `RETURN n` includes it and `n.` reads it by name. Also replaces the note claiming the cache is validated only against Iceberg: it now runs in CI against six sources, and graph-native sources cannot be cached at all. --- pages/memgraph-zero/memgql/changelog.mdx | 34 +++++++++++++++++++ .../memgraph-zero/memgql/multiple-graphs.mdx | 18 ++++++++-- pages/memgraph-zero/memgql/schema-file.mdx | 11 ++++-- 3 files changed, 59 insertions(+), 4 deletions(-) diff --git a/pages/memgraph-zero/memgql/changelog.mdx b/pages/memgraph-zero/memgql/changelog.mdx index 046d9f859..708b9a37a 100644 --- a/pages/memgraph-zero/memgql/changelog.mdx +++ b/pages/memgraph-zero/memgql/changelog.mdx @@ -5,6 +5,40 @@ description: MemGQL release notes # MemGQL Changelog +## Unreleased + +### ⚠️ Behavior changes + +- **A vertex's or edge's `metaFields.id` is now exposed as a property.** + `RETURN n` / `RETURN r` include it, and `n.` reads it by name — + including after a `WITH n` boundary, which previously resolved only when the + id column happened to be called `id`. This aligns relational sources with native-graph ones and + with a graph's Memgraph cache, which stores the id as its merge key. +- **Caching now requires every cached label to declare `attributes`.** A label + that declares none would be cached id-only, so MemGQL refuses to cache that + graph and names the label instead of serving property-stripped rows. Queries + continue to run against the source. See + [Caching a graph in Memgraph](/memgraph-zero/memgql/multiple-graphs#caching-a-graph-in-memgraph). + +### Fixed + +- **ClickHouse returned result columns in alphabetical order** rather than the + order the query projected them. Column names travelled with their values, so + most clients were unaffected, but anything reading positionally was not: a + `RETURN n` whose mapping had a property sorting before `id` came back with that + property's value as the node's element id. +- **The Memgraph cache dropped most edges and every undeclared property.** Cached + edges pointed at the wrong endpoints or were missing entirely, and cached nodes + carried only their id, so a query served from the cache silently returned + different rows than the same query against the source. Edge properties are now + copied into the cache as well. +- **`count(DISTINCT )` and `count(DISTINCT )` produced invalid + SQL** on relational backends. Counting distinct properties was unaffected. +- **Grouping by a whole element produced invalid SQL** — `WITH n, count(…)` and + `WITH r, count(…)` on relational backends. +- **The cache's warm-up log counted rows read from the source**, so it reported + success even when no rows reached the cache. + ## MemGQL v0.8.0 - July 19th, 2026 ### ⚠️ Breaking changes diff --git a/pages/memgraph-zero/memgql/multiple-graphs.mdx b/pages/memgraph-zero/memgql/multiple-graphs.mdx index e3e151394..760629d05 100644 --- a/pages/memgraph-zero/memgql/multiple-graphs.mdx +++ b/pages/memgraph-zero/memgql/multiple-graphs.mdx @@ -443,6 +443,17 @@ stale result. `TTL` is in seconds; `MAX_BYTES` accepts a plain byte count or a `K`/`M`/`G`/`T` binary suffix (`8G` = 8 × 1024³ bytes). +**Every cached label must declare its `attributes`.** The cache copies only the +columns the mapping names, so a label that declares none would land in Memgraph +carrying nothing but its id, and every property read from the cache would come +back `NULL`. Rather than serve those rows as if they were the source, MemGQL +declines to cache the graph and logs an actionable message naming the label to +fix. The query still runs against the source, exactly as it would on a cache +miss, so this costs you caching and not correctness. Add the properties you want +cached to that vertex's [`attributes`](/memgraph-zero/memgql/schema-file#attributes) +and re-run. This is stricter than querying, where an undeclared property still +resolves to a same-named column. + The cache engages per routed part, whether the part is pinned with `USE` or routed by its labels: @@ -472,8 +483,11 @@ graph, its cache connector, `TTL`, `MAX_BYTES`, and the number of cached fragments). Cache activity is logged at `DEBUG` as `[CACHE] MISS … teeing …` and `[CACHE] HIT … serving from cache connector …`. -> **Note:** The query-driven cache is currently validated with Iceberg sources; -> the mechanism generalizes to the other warehouse connectors. +> **Note:** The query-driven cache is exercised in CI against PostgreSQL, MySQL, +> DuckDB, SQL Server, ClickHouse and the native Iceberg connector, each +> replaying the same query corpus from its cache that it runs against the +> source, and asserting identical rows. Graph-native sources (Memgraph, Neo4j) cannot be cached — +> there is no relational mapping to copy from. ## Cross-Graph Query Rules diff --git a/pages/memgraph-zero/memgql/schema-file.mdx b/pages/memgraph-zero/memgql/schema-file.mdx index 8f7239bca..12e186fbb 100644 --- a/pages/memgraph-zero/memgql/schema-file.mdx +++ b/pages/memgraph-zero/memgql/schema-file.mdx @@ -224,7 +224,7 @@ A vertex maps a **label** to a backend source. Exactly one source kind is set: | `mappedTableSource.connector` | relational | Connector name (alias `catalog`). | | `mappedTableSource.schema` | relational | Optional middle namespace (PostgreSQL schema, etc.). | | `mappedTableSource.table` | relational | The backing table. | -| `mappedTableSource.metaFields.id` | relational | Primary-key column; the node's identity and the join key for edges. | +| `mappedTableSource.metaFields.id` | relational | Primary-key column; the node's identity and the join key for edges. Also exposed as a property, so `RETURN n` includes it and `n.` reads it by name. | | `mappedGraphSource.connector` | native | Connector name (Memgraph / Neo4j); the query passes through as Cypher. | ### Edges @@ -271,7 +271,7 @@ A relational edge adds two more `metaFields` on top of `id`: | `metaFields` key | Description | |------------------|-------------| -| `id` | The edge row's own key. | +| `id` | The edge row's own key. Exposed as a property, like a vertex's. | | `from` | Foreign-key column pointing at the **source** vertex's id column. | | `to` | Foreign-key column pointing at the **target** vertex's id column. | @@ -303,6 +303,13 @@ queries by. > type, or add an explicit `USE`, when a property isn't declared. See > [routing](/memgraph-zero/memgql/multiple-graphs#routing). +> **Note:** [caching a graph](/memgraph-zero/memgql/multiple-graphs#caching-a-graph-in-memgraph) +> needs them too, and unlike routing it is all-or-nothing: the cache copies only +> the columns the mapping names, so a label with no `attributes` would be cached +> with nothing but its id. Rather than serve property-stripped rows, MemGQL +> refuses to cache that graph and logs which label to declare. Undeclared +> properties stay queryable against the source either way. + ## Complete example One engine over **Memgraph** (a social graph) and **PostgreSQL** (a store), From bf3bc60d8b6dff3587cc89bd648bdd181b566833 Mon Sep 17 00:00:00 2001 From: DavIvek Date: Fri, 31 Jul 2026 15:10:57 +0200 Subject: [PATCH 2/5] docs(memgql): cache holds declared properties, not all-or-nothing MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Rewrites the note added earlier in this branch. MemGQL no longer refuses to cache a label that declares no attributes — it caches its ids and topology, which is what traversals and graph algorithms run on, and treats a read of an undeclared property as a cache miss served from the source. Same rule routing applies, so the mapping format gains no obligation. Also documents the new `cached_properties` column on SHOW GRAPH CACHES, and corrects the claim that every `USE`-split part engages the cache independently: UNION composites dispatch as a whole and never consult it (memgraph/zero#75). --- pages/memgraph-zero/memgql/changelog.mdx | 10 ++++--- .../memgraph-zero/memgql/multiple-graphs.mdx | 29 ++++++++++++------- pages/memgraph-zero/memgql/schema-file.mdx | 10 +++---- 3 files changed, 29 insertions(+), 20 deletions(-) diff --git a/pages/memgraph-zero/memgql/changelog.mdx b/pages/memgraph-zero/memgql/changelog.mdx index 708b9a37a..ba8a0e1c2 100644 --- a/pages/memgraph-zero/memgql/changelog.mdx +++ b/pages/memgraph-zero/memgql/changelog.mdx @@ -14,10 +14,12 @@ description: MemGQL release notes including after a `WITH n` boundary, which previously resolved only when the id column happened to be called `id`. This aligns relational sources with native-graph ones and with a graph's Memgraph cache, which stores the id as its merge key. -- **Caching now requires every cached label to declare `attributes`.** A label - that declares none would be cached id-only, so MemGQL refuses to cache that - graph and names the label instead of serving property-stripped rows. Queries - continue to run against the source. See +- **The cache now answers only for properties it holds.** A fragment holds a + label's id and its declared `attributes`; a query reading anything else is a + cache miss and reads the source, so results never differ from the uncached + run. A label declaring no `attributes` still caches its ids and topology, so + traversals and graph algorithms work against it. `SHOW GRAPH CACHES` gained a + `cached_properties` column reporting what each fragment holds. See [Caching a graph in Memgraph](/memgraph-zero/memgql/multiple-graphs#caching-a-graph-in-memgraph). ### Fixed diff --git a/pages/memgraph-zero/memgql/multiple-graphs.mdx b/pages/memgraph-zero/memgql/multiple-graphs.mdx index 760629d05..549d572f0 100644 --- a/pages/memgraph-zero/memgql/multiple-graphs.mdx +++ b/pages/memgraph-zero/memgql/multiple-graphs.mdx @@ -443,16 +443,21 @@ stale result. `TTL` is in seconds; `MAX_BYTES` accepts a plain byte count or a `K`/`M`/`G`/`T` binary suffix (`8G` = 8 × 1024³ bytes). -**Every cached label must declare its `attributes`.** The cache copies only the -columns the mapping names, so a label that declares none would land in Memgraph -carrying nothing but its id, and every property read from the cache would come -back `NULL`. Rather than serve those rows as if they were the source, MemGQL -declines to cache the graph and logs an actionable message naming the label to -fix. The query still runs against the source, exactly as it would on a cache -miss, so this costs you caching and not correctness. Add the properties you want -cached to that vertex's [`attributes`](/memgraph-zero/memgql/schema-file#attributes) -and re-run. This is stricter than querying, where an undeclared property still -resolves to a same-named column. +**The cache holds a label's id and its declared +[`attributes`](/memgraph-zero/memgql/schema-file#attributes)** — the columns the +mapping names. A query reading any other property is a cache miss and reads the +source, so results never differ from the uncached run; you lose the speed-up for +that query, not correctness. This is the same rule +[routing](#routing) applies: a property has to be declared to carry a signal. + +A label declaring no `attributes` at all still caches its **ids and topology**, +which is what traversals, path finding and graph algorithms run on. That is +usually the reason to cache a warehouse graph in Memgraph in the first place — +`CALL pagerank.get()` and friends need no properties. Declare the attributes you +also want answered from the cache. + +`SHOW GRAPH CACHES` reports a `cached_properties` column listing what each +fragment holds, which is how you tell a property-driven miss from a cold one. The cache engages per routed part, whether the part is pinned with `USE` or routed by its labels: @@ -476,7 +481,9 @@ routed by its labels: In a cross-backend query (split by explicit `USE`, or routed by label), each cache-enabled part is served from its cache independently while the other parts -run on their own sources. +run on their own sources. `UNION` / `UNION ALL` queries are the exception: they +dispatch as a whole and never consult the cache, even when every scope they +touch is resident. Inspect what is cached with `SHOW GRAPH CACHES` (returns each cache-enabled graph, its cache connector, `TTL`, `MAX_BYTES`, and the number of cached diff --git a/pages/memgraph-zero/memgql/schema-file.mdx b/pages/memgraph-zero/memgql/schema-file.mdx index 12e186fbb..387c651eb 100644 --- a/pages/memgraph-zero/memgql/schema-file.mdx +++ b/pages/memgraph-zero/memgql/schema-file.mdx @@ -304,11 +304,11 @@ queries by. > [routing](/memgraph-zero/memgql/multiple-graphs#routing). > **Note:** [caching a graph](/memgraph-zero/memgql/multiple-graphs#caching-a-graph-in-memgraph) -> needs them too, and unlike routing it is all-or-nothing: the cache copies only -> the columns the mapping names, so a label with no `attributes` would be cached -> with nothing but its id. Rather than serve property-stripped rows, MemGQL -> refuses to cache that graph and logs which label to declare. Undeclared -> properties stay queryable against the source either way. +> follows the same rule. The cache holds a label's id and its declared +> attributes, so a query reading an undeclared property is served from the +> source rather than from the cache. A label declaring no `attributes` still +> caches its ids and topology, which is what traversals and graph algorithms +> run on. ## Complete example From 12445b2db5e80ac1dff06ae932ed7896ab8b0c01 Mon Sep 17 00:00:00 2001 From: DavIvek Date: Fri, 31 Jul 2026 15:26:58 +0200 Subject: [PATCH 3/5] docs(memgql): SHOW GRAPH CACHES reports hits/misses --- pages/memgraph-zero/memgql/multiple-graphs.mdx | 6 ++++-- pages/memgraph-zero/memgql/reference.mdx | 2 +- 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/pages/memgraph-zero/memgql/multiple-graphs.mdx b/pages/memgraph-zero/memgql/multiple-graphs.mdx index 549d572f0..30c303555 100644 --- a/pages/memgraph-zero/memgql/multiple-graphs.mdx +++ b/pages/memgraph-zero/memgql/multiple-graphs.mdx @@ -456,8 +456,10 @@ usually the reason to cache a warehouse graph in Memgraph in the first place — `CALL pagerank.get()` and friends need no properties. Declare the attributes you also want answered from the cache. -`SHOW GRAPH CACHES` reports a `cached_properties` column listing what each -fragment holds, which is how you tell a property-driven miss from a cold one. +`SHOW GRAPH CACHES` reports `hits` / `misses` per graph and a +`cached_properties` column listing what each fragment holds — between them you +can tell a cache that is serving from one that never engages, and a +property-driven miss from a cold one. The cache engages per routed part, whether the part is pinned with `USE` or routed by its labels: diff --git a/pages/memgraph-zero/memgql/reference.mdx b/pages/memgraph-zero/memgql/reference.mdx index 467f904bc..dab19e998 100644 --- a/pages/memgraph-zero/memgql/reference.mdx +++ b/pages/memgraph-zero/memgql/reference.mdx @@ -88,7 +88,7 @@ ALTER GRAPH SET READ WRITE; -- Query-driven cache ALTER GRAPH SET CACHE CONNECTOR [TTL ] [MAX_BYTES [K|M|G|T]]; ALTER GRAPH REMOVE CACHE; -SHOW GRAPH CACHES; -- graph, cache_connector, ttl_secs, max_bytes, fragment count +SHOW GRAPH CACHES; -- graph, cache_connector, ttl_secs, max_bytes, fragments, hits, misses, cached_properties ``` A connector is a **connection only**; it carries no graph shape. `GRAPH ` From cfef4664b6d5c5049281a86a7ce8e83bc0e91db5 Mon Sep 17 00:00:00 2001 From: DavIvek Date: Fri, 31 Jul 2026 15:42:50 +0200 Subject: [PATCH 4/5] docs(memgql): cache hit/miss counters, and the tee no longer re-scans MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Catches the changelog up with two changes it was missing: SHOW GRAPH CACHES now reports `hits` / `misses` as well as `cached_properties`, and a warm table is no longer re-scanned from the source when a newly-touched edge type pulls it in as an endpoint. Also rewrites the cache-fidelity entry. It described the defect as "cached nodes carried only their id", which now reads as contradicting the behaviour change above it — caching a label id-only is supported. The defect was answering property reads out of a fragment that never held them. --- pages/memgraph-zero/memgql/changelog.mdx | 19 ++++++++++++------- .../memgraph-zero/memgql/multiple-graphs.mdx | 4 +++- 2 files changed, 15 insertions(+), 8 deletions(-) diff --git a/pages/memgraph-zero/memgql/changelog.mdx b/pages/memgraph-zero/memgql/changelog.mdx index ba8a0e1c2..079ea0433 100644 --- a/pages/memgraph-zero/memgql/changelog.mdx +++ b/pages/memgraph-zero/memgql/changelog.mdx @@ -18,8 +18,10 @@ description: MemGQL release notes label's id and its declared `attributes`; a query reading anything else is a cache miss and reads the source, so results never differ from the uncached run. A label declaring no `attributes` still caches its ids and topology, so - traversals and graph algorithms work against it. `SHOW GRAPH CACHES` gained a - `cached_properties` column reporting what each fragment holds. See + traversals and graph algorithms work against it. `SHOW GRAPH CACHES` gained + `hits`, `misses` and `cached_properties` columns — enough to tell a cache that + is serving from one that never engages, and a property-driven miss from a cold + one. See [Caching a graph in Memgraph](/memgraph-zero/memgql/multiple-graphs#caching-a-graph-in-memgraph). ### Fixed @@ -29,15 +31,18 @@ description: MemGQL release notes most clients were unaffected, but anything reading positionally was not: a `RETURN n` whose mapping had a property sorting before `id` came back with that property's value as the node's element id. -- **The Memgraph cache dropped most edges and every undeclared property.** Cached - edges pointed at the wrong endpoints or were missing entirely, and cached nodes - carried only their id, so a query served from the cache silently returned - different rows than the same query against the source. Edge properties are now - copied into the cache as well. +- **The Memgraph cache returned different rows than its source.** Cached edges + pointed at the wrong endpoints or were missing entirely, edge properties were + never copied at all, and a cache holding only a label's id answered property + reads with `NULL` instead of declining them — so a query silently changed its + answer once the cache was warm. - **`count(DISTINCT )` and `count(DISTINCT )` produced invalid SQL** on relational backends. Counting distinct properties was unaffected. - **Grouping by a whole element produced invalid SQL** — `WITH n, count(…)` and `WITH r, count(…)` on relational backends. +- **A warm table was re-scanned from the source** every time a newly-touched + edge type pulled it in as an endpoint, repeating the full scan the cache + exists to avoid. - **The cache's warm-up log counted rows read from the source**, so it reported success even when no rows reached the cache. diff --git a/pages/memgraph-zero/memgql/multiple-graphs.mdx b/pages/memgraph-zero/memgql/multiple-graphs.mdx index 30c303555..590b7ffd8 100644 --- a/pages/memgraph-zero/memgql/multiple-graphs.mdx +++ b/pages/memgraph-zero/memgql/multiple-graphs.mdx @@ -466,7 +466,9 @@ routed by its labels: - **First read (cache MISS)**: the query runs on the source and the engine *tees* the scanned labels and edge types into the Memgraph cache, recording - which scopes it cached. + which scopes it cached. Scopes already resident are left alone — teeing an + edge type pulls in its endpoint labels, and those are not re-scanned if the + cache already holds them. - **Later covered read (cache HIT)**: a query whose scans are covered by existing cached scopes is served entirely from Memgraph, with no source I/O. - **Capability upgrade**: because cached data is a real graph in Memgraph, From 2b4c9598b8513ac182bc5a4d81b6b6a7d253493e Mon Sep 17 00:00:00 2001 From: DavIvek Date: Fri, 31 Jul 2026 15:45:46 +0200 Subject: [PATCH 5/5] docs(memgql): drop the id-only caching paragraph --- pages/memgraph-zero/memgql/multiple-graphs.mdx | 6 ------ 1 file changed, 6 deletions(-) diff --git a/pages/memgraph-zero/memgql/multiple-graphs.mdx b/pages/memgraph-zero/memgql/multiple-graphs.mdx index 590b7ffd8..0c7669798 100644 --- a/pages/memgraph-zero/memgql/multiple-graphs.mdx +++ b/pages/memgraph-zero/memgql/multiple-graphs.mdx @@ -450,12 +450,6 @@ source, so results never differ from the uncached run; you lose the speed-up for that query, not correctness. This is the same rule [routing](#routing) applies: a property has to be declared to carry a signal. -A label declaring no `attributes` at all still caches its **ids and topology**, -which is what traversals, path finding and graph algorithms run on. That is -usually the reason to cache a warehouse graph in Memgraph in the first place — -`CALL pagerank.get()` and friends need no properties. Declare the attributes you -also want answered from the cache. - `SHOW GRAPH CACHES` reports `hits` / `misses` per graph and a `cached_properties` column listing what each fragment holds — between them you can tell a cache that is serving from one that never engages, and a