MeridianDB

MeridianDB is an experimental embeddable Java bitemporal database. It preserves both when a fact applies in the domain and when the database recorded that knowledge. Built-in local persistence is the foundation; standalone use requires no external database, streaming service or application server.

Development priority: Linux local storage, production qualification and a workload-specific temporal file layout. Windows has a basic compatibility role. The optional S3 integration is retained as experimental, with further work deferred. Replication, Kafka-topic storage and other persistence integrations remain future work. No production readiness or comparative performance result is claimed.

Status: project foundation and architectural research. The documentation site is live at meridiandb.org and can be built locally. The initial temporal contract and exact workload fixtures are defined. The Java 25/Maven build now includes a memory-only embedded engine, immutable temporal types, an independent test reference and a runnable embedding example. InMemoryEngine implements the initial numeric temporal contract and remains nonpersistent. A separate local journal supports append, force, exclusive writer access and bounded replay. Owned prefix verification also accepts a per-call encoded-byte budget. An immutable history-segment codec preserves exact transaction ranges in bounded, checksummed byte images. The local segment adapter adds create-only file writes and verified heap readers. A separate manifest codec and verifier check a complete segment set against its reconstructed journal-prefix binding. Local checkpoint publication now verifies the actual owned journal prefix and selects immutable generations through ordered forces and atomic rename on supported filesystems. An opt-in bounded checkpoint history loader retains verified original records. Explicit read-only/writable recovery factories reuse those records once and release their container before complete journal replay; ordinary opens remain unchanged, with measured adoption pending. A separate test-only comparison core supplies fresh equivalent recovery cohorts, with explicit native-capture-disabled controls and instrumented work accounting. A separate bounded Java worker exports raw comparison envelopes; supervised admission, memory diagnostics and evaluation remain pending. Local database recovery restores historical queries and request lookup from checkpoint history plus the journal tail, retaining the stored lineage. Its explicit writable mode now forces each new journal record before atomic query publication and acknowledgement. The local protocol is experimental. Durability conformance checks process exits, lost acknowledgements, retries and seeded writable reopen; it does not simulate device power loss. There is no published library or stable API. The test-only workload corpus now supplies bounded seeded histories, independent expected answers and memory/local-reopen checks. A test-only local measurement core now measures checked operation lifecycles with scoped JVM telemetry. The Linux supervisor adds bounded execution, strict result admission, independent workload dimensions and scoped process/allocated-file accounting. Saved workload reports check those recorded results and render per-iteration metrics after the original files are gone. Bounded Linux observations now cover independent dimensions and four temporal profiles. An internal I/O counter foundation records completed adapter bytes and force calls in developer tests. The test-only attributed lifecycle core adds ordered operation/auxiliary scopes and logical-history denominators. Attributed evidence now adds separate supervised runs, original-history decoding and checked saved ratios. Evaluated operation file work now separates original append bytes from repeated recovery and maintenance reads. Controlled comparisons and production qualification remain planned. The broader storage capabilities below remain the intended direction.

Why temporal storage?

A correction received today can change what we believe about yesterday without erasing what we believed yesterday. MeridianDB aims to make that distinction explicit and queryable.

• Valid time: the interval in which an asserted fact applies in the domain.
• System time: the database-managed history of committed knowledge.
• Event history: the observations or messages received, which may produce changes to temporal state but are not themselves the same thing as that state.

For example, a salary recorded in January might be corrected in March with effect from February. Compare the salary effective in February as known in February with the salary effective in February as known after the March correction. The effective date is the same; the recorded knowledge differs. Audit records, configuration history, inventory and financial events are possible workloads; finance and FIX are examples rather than boundaries on the engine's purpose.

Language direction

Java is the default for the initial engine and reference implementation. It matches the initial JVM embedding target and provides static checking, standard diagnostic tooling and one build environment for production and reference code.

Clojure and Scala remain candidates for idiomatic JVM integrations. Rust may be useful for a measured native bottleneck or a non-JVM embedding requirement; Go may serve a Go integration or separate service. None is a current dependency or promised module. These choices depend on concrete deployment and workload needs.

See the language decision for alternatives, tradeoffs, evidence and conditions for revisiting this default. Maven is the chosen Java build tool for the initial artifact. The initial build foundation uses Java 25 and a checksum-pinned Maven Wrapper. Maven artifact versions are researched from current primary sources before selection and then pinned; see the dated evidence in the build guide.

Research direction

The initial investigation will focus on:

• Precise semantics for insertion, partial retroactive correction, retraction and queries across valid time and system time.
• A simple reference model against which optimized implementations can be tested.
• Immutable persistent segments, explicit commit visibility and reliable recovery.
• Linux filesystem qualification and measured local ingestion, query, recovery and memory behavior before further persistence integrations.
• Optional replication with explicit ordering, follower progress and durability guarantees, developed separately from storage adapters.
• Purpose-built temporal blocks and indexes compared with Parquet and simpler layouts against representative temporal workloads, followed by measured Linux I/O experiments where justified.

The persistence boundaries separate commit authority, ordered journal history, immutable objects, checkpoint selection and derived search indexes. Local persistence and an optional S3 byte adapter are implemented. Managed S3 read recovery is available separately. Managed S3 writable recovery composes exact admission with forced local commits. Replication and near-real-time indexing remain planned and unmeasured. The immutable-object interface now includes a JDK-only local adapter, complete identity/length validation, explicit identical reuse and a segment consumer. The separate meridiandb-s3 artifact runs the same object contract through a real SDK/HTTP fixture. A pure checkpoint-head codec defines bounded remote metadata and adjacent-transition checks. A separate conditional S3 head API now pairs GET bytes/tokens and issues exact conditional writes. JDK-only object graph verification now checks complete dependencies and actual owned-journal prefixes under shared budgets. Its pinned history loader now returns original immutable checkpoint records from the same verified reads; read-only core recovery now reconstructs the engine from that prefix and the existing complete local journal under factory ownership. An optional publication-state codec preserves exact expected/pending metadata with pure checked transitions. Durable local state now installs those snapshots under exclusive local ownership. Verified preparation now composes actual client scope, graph/journal checks and exact durable pending intent without selecting a remote head. Verified publication now adds exact conditional submission and durable candidate reconciliation. Managed S3 read recovery now admits exact durable expected selection before core startup; managed S3 writable recovery is implemented separately. Existing local checkpoint and write protocols are unchanged.

Parquet is a candidate file format; an object store is a persistence service; MergeTree-like organization is a candidate storage technique. These choices are related but separate. Snapshot history also remains distinct from bitemporal history. The transaction journal format now has a tested codec and local file adapter; a history-segment codec now reuses those exact transaction frames. Query-optimized layouts remain open. The storage research compares XTDB, Kafka, RocksDB, Lucene/Elasticsearch, Quickwit and ClickHouse, keeping durable append, index publication and query visibility distinct.

The temporal geometry guide explains bitemporal state as two-dimensional regions with explicit temporal rules, and evaluates reusable Lucene/Solr indexing ideas. Specialized indexes remain candidates for measured experiments.

SQL, document APIs, network protocols and streaming integrations remain open. Replication is a separate planned optional extension. Public interface implementation can wait while query semantics and workloads guide the storage experiments. Current-state lookups, historical entity lookups, interval scans and correction-sensitive aggregations have specified expected results, without committing to a public API.

Development roadmap

The technical roadmap describes the current memory engine and local journal, immutable segment files, checkpoint publication and read recovery, plus forced local commit publication, process crash/retry conformance and typed temporal query operators, bounded pure history repacking and verified local compaction publication. Lazy checkpoint readers now hold shared cross-process protection independently of store lifetime. A bounded reclamation inventory now verifies retained roots and reports obsolete physical objects without deletion. Protected physical removal now retires verified obsolete metadata before unused dependencies, retaining full logical history. Integrated conformance now checks canonical/seeded histories, richer process stops and exact named-file space accounting through actual publication and removal. The optional S3 checkpoint integration is experimental, with further work deferred. The next milestones qualify Linux storage profiles, establish a local workload baseline, compare file layouts and close evidenced operational/resource gaps before an experimental candidate. Local work does not depend on completing S3. The Linux qualification harness now runs the complete local-core inventory on one observed filesystem with fresh reports and explicit native capabilities. It supplies no production or power-loss qualification. Java and Maven remain the foundation.

Build the Java foundation

Use JDK 25 and run from the repository root; Maven is downloaded by the wrapper:

./mvnw -B verify

On Windows PowerShell:

.\mvnw.cmd -B verify

This runs 1,602 JUnit test invocations: nineteen recovery worker checks, sixteen uninstrumented-control/mode checks, ten recovery comparison core checks, 97 additional explicit-history recovery, canonical/seeded and process-exit checks, plus the existing 38 bounded local checkpoint-history checks, 34 attributed lifecycle/bridge/fault checks, 28 internal I/O counter checks, 25 supervised worker checks, 26 local measurement checks, 36 workload corpus checks, one filesystem qualification probe, 66 pinned checkpoint write checks, 69 pinned read recovery checks, 22 pinned object-history loading checks, 23 bounded compaction/reclamation preflight checks, 34 object graph/actual-prefix checks, 71 temporal/engine, 29 journal codec, 84 local journal I/O/process/prefix, 43 immutable segment codec, 36 local segment I/O/lifetime, 45 manifest codec/consistency, 39 local checkpoint publication, 37 local database recovery and 49 commit preparation/initialization/publication checks, plus 50 durable process/retry/temporal conformance checks, 55 typed query checks, 57 pure compaction checks, 55 local compaction publication/integration/process checks, 70 reader protection/identity/cleanup checks, 69 verified inventory/root/budget/process checks and 93 protected physical removal/fault/process checks, plus 46 integrated compaction/reclamation cases and 84 immutable-object/type/local/ fault/segment-consumer cases and 46 pure object-checkpoint-head format/binding cases. Coverage includes all 84 contract expectations against both engine and reference, 16 seeded histories with 512 new transactions applied to each, lifecycle/concurrency, golden journal bytes, short/failed I/O, repair evidence and separate-process writer exclusion/recovery. It creates the local experimental target/meridiandb-core-0.1.0-SNAPSHOT.jar. The reference model and fixture parser remain test-only. See the Java/Maven guide for setup, version research, wrapper verification and Linux/Windows CI.

The optional S3 adapter guide gives the separate core-install and adapter-verification commands. Its SDK is not required for the root core build or local embedding.

Run the memory-only embedding example after verification:

java --class-path target/classes examples/EmbeddedExample.java

It prints February salary values of 100 before a correction and 120 after it, then confirms an identical retry returns commit 2. The engine guide explains the experimental API, read-view lifetimes and memory limits. Closing or crashing loses the database. The separate local journal example exercises append/force and exact reopen. The local checkpoint example adds verified generation publication and reopen. The recovery example reopens temporal queries through LocalDatabase. The writable example adds forced commits and an identical retry after reopen. InMemoryEngine remains nonpersistent. The typed query guide explains signed-value predicates, deterministic entity selection and filtered interval scans at one pinned cutoff. Run its checked example after verification:

java --class-path target/classes examples/QueryExample.java

History compaction validates a complete checkpoint and repacks exact original transactions under byte/record budgets. It preserves all historical answers and retry identities. The current pure API creates, publishes and deletes no files. Run its checked example after verification:

java --class-path target/classes examples/CompactionExample.java

The separate local publisher rechecks source selection and the actual owned journal, verifies/reuses identical existing content, installs missing objects and publishes at the same cutoff. All old files and the full journal remain retained. Run its installation/reuse/reopen example:

java --class-path target/classes examples/LocalCompactionExample.java

Reclamation inventory verifies every published graph and explicit retained root under exclusive reader protection, then reports obsolete metadata/dependencies without deleting files. A live reader defers inspection; unknown and failed evidence stays preserved. Run its example:

java --class-path target/classes examples/ReclamationInventoryExample.java

Protected physical reclamation recomputes that inventory under both owners, synchronizes retained roots, then removes obsolete metadata before unused dependencies. Explicit retained/backup roots and the full journal remain intact. Run its checked removal/reopen example:

java --class-path target/classes examples/ReclamationExample.java

The immutable byte adapter has a checked history/reuse/range example:

java --class-path target/classes examples/ObjectStoreExample.java

Working in this repository

See Contributing for focused changes and validation. Architectural rationale lives in decision records. Use the Java command above and the separate documentation checks for public docs changes.

Documentation website

The website uses Zensical with MkDocs configuration in mkdocs.yml and the Material-style classic theme. Content lives in docs/; the published site is meridiandb.org. Use Python 3.13 (recorded in .python-version) and run these commands from the repository root.

Create the isolated documentation environment:

python3 -m venv .venv

Install the locked dependencies:

.venv/bin/python -m pip install --require-hashes -r requirements-docs.txt

Start the local preview:

.venv/bin/zensical serve --dev-addr 127.0.0.1:8000

Build the static site with strict checks:

.venv/bin/python scripts/build_docs.py

The generated site is site/. Python documentation tooling is independent of the Maven Java build. Content uses ordinary Markdown and source-relative links to help preserve portability to other generators, including Hugo.

GitHub Pages is the initial hosting target. Affected pull requests and pushes to main run documentation checks. Documentation changes pushed to main also run Publish documentation, which builds and deploys automatically. Run workflow remains available on main for a manual rebuild. Cloudflare can also host the static output if hosting needs change. See the documentation and publishing guide and toolchain decision for setup, compatibility, metadata and future versioning.

Support MeridianDB

Support MeridianDB's open-source development, testing and documentation through MeridianDB on GitHub Sponsors. MeridianDB is experimental; sponsorship is voluntary support for ongoing work.

License

Apache License 2.0.