fix: lazily load database clients via dynamic import()

[elrrrrrrr]

Summary

• Defer database client loading to first getConnection() call across all drivers
• Replace synchronous require(client) with await import(client) (ESM-compatible)
• Handle both CJS and ESM modules via module.default ?? module
• Keep optional database clients out of static bundler dependency graphs

Why

Driver constructors previously called require(options.client) synchronously, which:

1. Breaks under ESM-only bundlers (Turbopack, esbuild ESM output)
2. Forces bundlers to statically resolve optional peer dependencies
3. Blocks future Node ESM entry support (step 1 of the plan in docs/design/node-esm-entry.md)

Using await import() at first connection keeps resolution at runtime and works in both CJS and ESM contexts.

Validation

• SQLite, MySQL, and PostgreSQL lazy-loading unit tests pass
• Existing test suite passes (realm.test.js updated for async behavior)
• esbuild bundle succeeds without resolving unused database clients

[cyjake]

@elrrrrrrr could you give a bit more information? which bundler couldn't handle static require(var)?

[elrrrrrrr]

Thanks for asking. To be precise, the concrete case is @utoo/pack (Turbopack-based), used by the server-side Egg 4 bundle flow through @eggjs/egg-bundler; this is not a browser bundle.

With the dynamic require(options.client) call, @utoo/pack completes the build but replaces the expression in the generated worker with a throwing stub. App startup then fails with:

Error: Cannot find module as expression is too dynamic

This happens even when options.client is a valid runtime module name. Using createRequire(__filename)(options.client) keeps it as a Node runtime load, and the same bundled fixture starts successfully.

So the change is specifically for this server-bundling case, not a claim about every bundler. Leoric's existing SQLCipher integration test already covers the normal configured-client behavior; I will tighten the PR description accordingly.

[elrrrrrrr]

One further clarification: esbuild has the same problem when it emits an ESM server bundle. It preserves require() for CommonJS output, but converts a dynamic require in ESM output into a __require shim that throws at runtime.

Minimal reproduction:

import { EventEmitter } from 'node:events';

class Pool extends EventEmitter {
  constructor(options: { client: string }) {
    super();
    this.client = require(options.client);
  }
}

new Pool({ client: 'node:events' });

esbuild pool.ts --bundle --platform=node --format=esm --outfile=pool.mjs
node pool.mjs
# Error: Dynamic require of "node:events" is not supported

The longer-term direction should therefore be native ESM loading rather than any require shim:

async loadClient() {
  const mod = await import(this.options.client);
  this.client = mod.default ?? mod;
}

The same fixture bundles and runs successfully with esbuild ESM output. That requires an async initialization boundary for the SQLite client; the createRequire change is the narrower compatibility fix for the current synchronous CommonJS API.

[elrrrrrrr]

BTW, is there any plan to provide a native ESM entry for Leoric (for example, through conditional exports) alongside the CommonJS build? It would let ESM server bundles load Leoric without a CommonJS require bridge, especially for runtime-selected optional clients. No expectation for this PR; I am just curious whether this is already on the roadmap.

[yicai-dev]

Thanks for the detailed investigation and the reproduction case — very helpful.

We're planning to add a first-class Node ESM entry via conditional exports shortly (dual CJS + ESM). Given that direction, we think the better fix for the dynamic require(options.client) calls is await import() rather than createRequire, so we avoid introducing a shim that gets immediately replaced.

The concrete drivers already load clients inside an async connect() chain, so await import(options.client) should be a straightforward change for mysql and postgres. The sqlite pool constructor is synchronous today, but we can defer the client load to an async init step.

This would also cover the mysql driver (src/drivers/mysql/index.ts) which has the same dynamic-require pattern.

Would you be interested in updating this PR to use await import() instead? If so, the scope would be:

1. src/drivers/sqlite/pool.ts — lazy-load client via await import() (requires moving the load out of the constructor)
2. src/drivers/mysql/index.ts — same treatment for require(client).createPool(...)
3. src/drivers/postgres/index.ts — require('pg') is a string literal so lower priority, but nice to align

We'll handle the package.json conditional exports on our side. Let us know what you think!

[elrrrrrrr]

Thanks — I updated the PR in commit 7785871.

• SQLite keeps its synchronous constructor, but lazily loads the configured optional client from getConnection() with a memoized await import().
• MySQL and PostgreSQL now lazily create their pools through the same async boundary; CommonJS driver modules are normalized with mod.default ?? mod.
• I also changed the MySQL sqlstring usage to a default import, since Node ESM cannot reliably consume that package through named imports.
• Added a fake-client regression test: two concurrent first connections initialize each SQLite/MySQL/PostgreSQL client exactly once.

Local validation: the three paths passed both CJS and ESM esbuild bundles at runtime. The ESM bundles externalize package dependencies, matching the intended conditional-exports setup. I left package.json conditional exports and the dual-build layout to the upstream work as discussed.

One intentional behavior change to call out: for MySQL/PostgreSQL, driver.pool is populated on the first connection rather than in the constructor. That is the async initialization boundary needed to avoid a CommonJS loader shim.

[elrrrrrrr]

Follow-up: the first upstream CI run exposed two SQLite lazy-initialization regressions. Fixed in 0b3ac8f:

• getConnection() now awaits the memoized SQLite client load before checking pool capacity, so concurrent first callers cannot both pass connectionLimit: 1.
• The Realm client-switch test now asserts a missing optional client at the asynchronous getConnection() boundary, then checks driver.pool after the first successful connection.

Validated on the fork with the same Node 20 + MySQL 5.7 + PostgreSQL full coverage workflow: https://github.com/elrrrrrrr/leoric/actions/runs/29738803369

[Copilot]

Pull request overview

This PR aims to keep optional database client packages (SQLite/MySQL/Postgres) out of static bundler dependency graphs by deferring client/pool creation to runtime, while preserving support for custom client modules/paths.

Changes:

• Updated SQLite pool to lazily load the configured client once and reuse it across concurrent connection requests.
• Refactored MySQL and Postgres drivers to lazily create pools and dynamically load the configured client.
• Updated/added unit tests and fixtures to cover dynamic client loading, concurrency, and deferred “module not found” failures.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
test/unit/realm.test.js	Adjusts expectations to reflect deferred client resolution (error now occurs on connection acquisition).
test/unit/drivers/dynamic-client.test.js	Adds coverage for lazy/dynamic client loading across SQLite/MySQL/Postgres and concurrent calls.
test/fixtures/dynamic-client.js	Provides a lightweight fake client module used by the new dynamic-loading tests.
src/drivers/sqlite/pool.ts	Implements lazy client loading with a cached promise and retries on failure.
src/drivers/postgres/index.ts	Makes pool creation lazy and loads pg (or a custom client) at runtime.
src/drivers/mysql/index.ts	Makes pool creation lazy and loads the configured MySQL client at runtime.
src/drivers/mysql/attribute.ts	Switches sqlstring usage to a default import and destructuring for escape helpers.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[yicai-dev]

copilot seems taking pr desc as the source of truth instead of the adjusted implementation. @elrrrrrrr plz update pr title/desc accordingly