Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/benchmark-comparison.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ jobs:

strategy:
matrix:
node-version: [20, 22, 24]
node-version: [27]

steps:
- name: Checkout code
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ jobs:

strategy:
matrix:
node-version: [18, 20, 22, 24, 25]
node-version: [27]

env:
CI: true
Expand Down
60 changes: 48 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,14 +22,16 @@
[![Downloads][downloads-img]][downloads-url]
[![Issues][issues-img]][issues-url]

The `bench-node` module allows you to measure operations per second of Node.js code blocks.
The `bench-node` module allows you to measure operations per second of Node.js code blocks. Its benchmark execution and sample lifecycle are powered by the built-in `node:bench` module.

## Install

```bash
$ npm install bench-node
```

`bench-node` requires Node.js 27 or a development build that provides `node:bench`.

## Usage

```cjs
Expand All @@ -49,12 +51,30 @@ suite.add('Using delete property', () => {
suite.run()
```

Run the file directly to use the suite's configured reporter:

```bash
node --allow-natives-syntax my-benchmark.js
```

The same file can be run by Node's native benchmark CLI. In this mode, Node's
reporter options control output and the `Suite` reporter is not invoked:

```bash
node --allow-natives-syntax --bench --bench-reporter=spec my-benchmark.js
```

When using `--bench`, call `suite.run()` while the benchmark module is being
evaluated; do not block module evaluation with top-level `await`. If
`useWorkers` is enabled, the native CLI's process isolation takes precedence
and the benchmarks execute in its managed child process.

```bash
$ node --allow-natives-syntax my-benchmark.js
Using delete property x 3,326,913 ops/sec (11 runs sampled) v8-never-optimize=true min..max=(0ns ... 0ns) p75=0ns p99=0ns
```

This module uses V8 deoptimization to help ensure that the code block is not optimized away, producing accurate benchmarks -- but not realistic.
This module uses `node:bench` for measurement and V8 deoptimization to help ensure that the code block is not optimized away, producing accurate benchmarks -- but not realistic.
See the [Writing JavaScript Microbenchmark Mistakes](#writing-javascript-mistakes) section for more details.

The [`bench-node-cli`](https://github.com/RafaelGSS/bench-node-cli) tool allows you to execute a `bench-node` benchmark
Expand Down Expand Up @@ -121,9 +141,10 @@ A `Suite` manages and executes benchmark functions. It provides two methods: `ad
* `reporter` {Function} Callback function for reporting results. Receives two arguments:
* `results` {Object[]} Array of benchmark results:
* `name` {string} Benchmark name.
* `opsSec` {string} Operations per second.
* `opsSec` {number} Operations per second.
* `iterations` {Number} Number of iterations.
* `histogram` {Histogram} Histogram instance.
* `histogram` {Object} Normalized histogram data with `samples`, `min`,
`max`, and `sampleData` properties.
* `ttest` {boolean} Enable Welch's t-test for statistical significance testing. Automatically sets `repeatSuite=30`. **Default:** `false`.
* `reporterOptions` {Object} Reporter-specific options.
* `printHeader` {boolean} Whether to print system information header. **Default:** `true`.
Expand All @@ -133,10 +154,10 @@ A `Suite` manages and executes benchmark functions. It provides two methods: `ad
* `'ops'` - Measures operations per second (traditional benchmarking).
* `'time'` - Measures actual execution time for a single run.
* `useWorkers` {boolean} Whether to run benchmarks in worker threads. **Default:** `false`.
* `plugins` {Array} Array of plugin instances to use.
* `repeatSuite` {number} Number of times to repeat each benchmark. Automatically set to `30` when `ttest: true`. **Default:** `1`.
* `plugins` {Array} Array of plugin instances to use. **Default:** `[V8NeverOptimizePlugin]`.
* `minSamples` {number} Minimum number of samples per round for all benchmarks in the suite. Can be overridden per benchmark. **Default:** `10` samples.
* `pretty` {boolean} Use the pretty reporter when no explicit reporter is provided. **Default:** `false`.
* `detectDeadCodeElimination` {boolean} Enable dead code elimination detection. When enabled, default plugins are disabled to allow V8 optimizations. **Default:** `false`.
* `dceThreshold` {number} Threshold multiplier for DCE detection. Benchmarks faster than baseline × threshold will trigger warnings. **Default:** `10`.

Expand Down Expand Up @@ -180,9 +201,9 @@ Using delete property x 5,853,505 ops/sec (10 runs sampled) min..max=(169ns ...
* `opsSecPerRun` {Array} Array of operations per second (useful when repeatSuite > 1).
* `totalTime` {number} Mean execution time in seconds per sample (only in `'time'` mode).
* `iterations` {number} Number of executions of `fn`.
* `histogram` {Histogram} Histogram of benchmark iterations.
* `histogram` {Object} Normalized histogram of nanoseconds per operation.
* `name` {string} Benchmark name.
* `plugins` {Object} Object with plugin results if any plugins are active.
* `plugins` {Object[]} Plugin results if any plugins are active.

Runs all added benchmarks and returns the results.

Expand Down Expand Up @@ -254,7 +275,7 @@ suite.add('computation', () => {
});
```

**Note:** DCE detection only works in `'ops'` benchmark mode and when not using worker threads. It is automatically disabled for `'time'` mode and worker-based benchmarks.
**Note:** DCE detection only works in direct `'ops'` mode. It is automatically disabled for `'time'` mode, worker-based benchmarks, and the native `--bench` CLI.

See [examples/dce-detection/](./examples/dce-detection/) for more examples.

Expand Down Expand Up @@ -284,18 +305,26 @@ See [Plugins](./doc/Plugins.md) for details.
class V8OptimizeOnNextCallPlugin {
isSupported() {
try {
new Function(`%OptimizeFunctionOnNextCall(() => {})`)();
new Function(`
const fn = () => {};
%PrepareFunctionForOptimization(fn);
fn();
fn();
%OptimizeFunctionOnNextCall(fn);
fn();
`)();
return true;
} catch (e) {
return false;
}
}

beforeClockTemplate({ awaitOrEmpty, bench }) {
beforeClockTemplate({ awaitOrEmpty, bench, timer }) {
let code = '';
code += `%PrepareFunctionForOptimization(${bench}.fn);\n`;
code += `${awaitOrEmpty}${bench}.fn(${timer});\n`;
code += `${awaitOrEmpty}${bench}.fn(${timer});\n`;
code += `%OptimizeFunctionOnNextCall(${bench}.fn);\n`;
code += `${awaitOrEmpty}${bench}.fn();\n`;
code += `${awaitOrEmpty}${bench}.fn();\n`;
return [code];
}

Expand Down Expand Up @@ -662,6 +691,13 @@ const suite = new Suite({
});
```

When the file is launched with `node --bench`, Node's benchmark-file isolation
takes precedence and `useWorkers` does not create a nested worker. Use the
native CLI's `--bench-concurrency` option to run benchmark files concurrently.
For direct runs, worker benchmark functions are serialized and therefore cannot
close over variables from the declaring module. Plugin sample context must be
structured-cloneable so it can be returned to the parent thread.

## Benchmark Modes

`bench-node` supports multiple benchmarking modes to measure code performance in different ways.
Expand Down
50 changes: 34 additions & 16 deletions doc/Plugins.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,11 @@ plugins within the benchmarking framework.

[V8NeverOptimizePlugin](#class-v8neveroptimizeplugin) is enabled by default.

Plugin templates are compiled around the callback executed by `node:bench`.
For unmanaged benchmarks, setup runs before `BenchContext.start()` and teardown
runs after `BenchContext.end()`. Managed benchmarks report their explicit timer
through `BenchContext.record()`.

To observe how a plugin is used, see the `plugin-api-doc.js` file in tests and explore its results.

## Structure
Expand Down Expand Up @@ -42,10 +47,11 @@ this method ensures the environment supports them.
* `context` {string} - Name for the context variable.
* `timer` {string} - Name for the timer variable.
* `awaitOrEmpty` {string} - A string with `await` or empty string (`''`).
* `managed` {boolean} - Whether the benchmark uses the explicit timer API.

Some plugins need to modify or prepare the code before the benchmark starts.
The `beforeClockTemplate()` method allows you to inject code before the timing
process begins.
Some plugins need to modify or prepare the code before a benchmark sample
starts. The `beforeClockTemplate()` method allows you to inject code before the
timed region of each native warmup and measurement callback.

This method must return an array where:

Expand Down Expand Up @@ -90,9 +96,10 @@ These two protections address different parts of the generated code:
* `context` {string} - Name for the context variable.
* `timer` {string} - Name for the timer variable.
* `awaitOrEmpty` {string} - A string with `await` or empty string (`''`).
* `managed` {boolean} - Whether the benchmark uses the explicit timer API.

After the benchmark runs, this method can inject code to gather performance data
or reset configurations. It must return an array where:
After each benchmark sample runs, this method can inject code to gather
performance data or reset configurations. It must return an array where:

* The first element is a string containing the JavaScript code to be executed
after the benchmark finishes.
Expand All @@ -101,15 +108,18 @@ Unlike `beforeClockTemplate`, `afterClockTemplate` does not support a second
element in the returned array, as it only runs cleanup or data collection code
after the benchmark is executed.

### `onCompleteBenchmark(result)`
### `onCompleteBenchmark(result, benchmark)`

* `result` {Object}
* `duration` {number} - Benchmark duration
* `count` {number} - Number of iterations
* `context` {Object} - A object used to store results after the benchmark clock
* `result` {Array}
* `result[0]` {number} - Sample duration in nanoseconds.
* `result[1]` {number} - Number of operations in the sample.
* `result[2]` {Object} - Context populated by plugin templates.
* `benchmark` {Object} Benchmark metadata.

This method is called when the benchmark completes. Plugins can collect and
process data from the benchmark results in this step.
This method is called after each native warmup or measurement sample. Plugins
can collect and process data from the sample in this step. In worker mode, the
context must be structured-cloneable because the hook is replayed in the parent
thread.

### `toString()` (required)

Expand All @@ -124,19 +134,27 @@ Here are examples of plugins that follow the required structure and functionalit
class V8OptimizeOnNextCallPlugin {
isSupported() {
try {
new Function(`%OptimizeFunctionOnNextCall(() => {})`)();
new Function(`
const fn = () => {};
%PrepareFunctionForOptimization(fn);
fn();
fn();
%OptimizeFunctionOnNextCall(fn);
fn();
`)();
return true;
} catch (e) {
return false;
}
}

beforeClockTemplate({ awaitOrEmpty, bench }) {
beforeClockTemplate({ awaitOrEmpty, bench, timer }) {
let code = '';

code += `%PrepareFunctionForOptimization(${ bench }.fn);\n`;
code += `${ awaitOrEmpty }${ bench }.fn(${ timer });\n`;
code += `${ awaitOrEmpty }${ bench }.fn(${ timer });\n`;
code += `%OptimizeFunctionOnNextCall(${ bench }.fn);\n`;
code += `${ awaitOrEmpty }${ bench }.fn();\n`;
code += `${ awaitOrEmpty }${ bench }.fn();\n`;

return [code];
}
Expand Down
Loading
Loading