Computed Shapes

[ItsJustMeChris]

Computed Shapes

I’ve been experimenting with “Computed Shapes”: named, server-defined reactive queries whose results are delivered through the existing Shape protocol.

They’re conceptually similar to Convex reactive functions or incrementally maintained materialized views. Instead of syncing every underlying row to clients, Electric maintains a keyed derived result and streams only changes to that result.

For example, rather than syncing every app open/close event, a client could subscribe to:

Daily app usage for this user, grouped by app and day.

The prototype supports:

• A general recompute-and-diff path for bounded SQL reports.
• Shared incremental COUNT/SUM aggregations.
• DISTINCT, grouped MIN/MAX, top-N results, inner joins, and session rollups.
• Parameterized results with shared computation across subscribers.
• Existing Shape handles, offsets, long polling, txids, and insert/update/delete messages.
• Durable checkpoints and journal-based crash recovery.

Performance looks promising for high-reduction queries. In an end-to-end Postgres → WAL → HTTP test, 100 source events became one derived update:

• Raw Shape output: 28,241 bytes
• Computed Shape output: 441 bytes
• 98.44% less data sent to the client

At 100k changes, incremental COUNT/SUM took about 1.48× the CPU of raw Shape conversion while retaining 0.86% of the state and producing 0.97% of the output. Top-N and joins cost more CPU—about 2.5× and 3.1× respectively—but can still dramatically reduce client-side data.

The important finding is that generic recompute-and-diff does not scale well for large, sparsely changing results. Incremental maintenance is what makes this practical.

There are real tradeoffs beyond implementation complexity: Electric takes on more CPU, memory, durable state, recovery semantics, type/collation correctness, and potential join amplification. The source events also still pass through Postgres WAL and Electric; they simply no longer need to reach every client.

The experimental implementation and detailed benchmarks are available here:

• [Computed Shapes branch](https://github.com/ItsJustMeChris/electric/tree/computed-shapes-experiment)
• [Design and performance report](https://github.com/ItsJustMeChris/electric/blob/computed-shapes-experiment/docs/computed-shapes-experiment.md)

I’d be interested in feedback on whether this fits Electric’s model and which query patterns would be most valuable to support first.