Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions docs/gitbook/guide/workers/sandboxed-processors.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,12 @@ worker = new Worker(queueName, processorUrl);
Recommended for Windows OS.
{% endhint %}

### Running under `node --watch`

BullMQ spawns sandboxed processors as separate Node.js processes (or worker threads) using the parent process's `execArgv`. The `--watch` family of flags (`--watch`, `--watch-path`, `--watch-preserve-output`, etc.) is stripped before being forwarded to the child so that the sandboxed runtime does not also enter watch mode, which can interfere with IPC and leave jobs stuck in the `active` state.

If you pass additional flags through `workerForkOptions.execArgv` or `workerThreadsOptions.execArgv`, make sure none of them enable watch mode on the child.

### Worker Threads

The default mechanism for launching sandboxed workers is using Node's spawn process library. From BullMQ version v3.13.0, it is also possible to launch the workers using Node's new Worker Threads library. These threads are supposed to be less resource-demanding than the previous approach, however, they are still not as lightweight as we could expect since Node's runtime needs to be duplicated by every thread.
Expand Down
21 changes: 21 additions & 0 deletions src/classes/child.ts
Original file line number Diff line number Diff line change
Expand Up @@ -235,12 +235,33 @@ const getFreePort = async () => {
});
};

/**
* Returns true when the given execArgv entry is a Node.js `--watch*` family
* flag (e.g. `--watch`, `--watch=path`, `--watch-path=foo`,
* `--watch-preserve-output`, `--watch-kill-signal=SIGTERM`).
*
* These flags must not be forwarded to sandboxed child processes or worker
* threads. Forwarding them causes the child Node.js runtime to also enter
* watch mode, which interferes with IPC between the parent worker and the
* sandboxed processor and can leave jobs stuck in the `active` state.
*
* See https://github.com/taskforcesh/bullmq/issues/1833
*/
const isWatchFlag = (arg: string): boolean => {
return arg === '--watch' || arg.startsWith('--watch=') || arg.startsWith('--watch-');
};

const convertExecArgv = async (execArgv: string[]): Promise<string[]> => {
const standard: string[] = [];
const convertedArgs: string[] = [];

for (let i = 0; i < execArgv.length; i++) {
const arg = execArgv[i];
if (isWatchFlag(arg)) {
// Skip `--watch*` flags so sandboxed children don't inherit watch mode
// from the parent process (see GH #1833).
continue;
}
if (arg.indexOf('--inspect') === -1) {
standard.push(arg);
} else {
Expand Down
39 changes: 39 additions & 0 deletions tests/child-pool.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -115,5 +115,44 @@ function sandboxProcessTests(
expect(child.childProcess.spawnargs).toContain('--no-warnings');
}
});

// Regression: https://github.com/taskforcesh/bullmq/issues/1833
// When the parent Node.js process is launched with `--watch`, the flag
// must not be forwarded to sandboxed children. Inheriting it causes the
// child runtime to also enter watch mode, which interferes with IPC and
// leaves jobs stuck in the `active` state.
it('should strip node --watch flags from execArgv before spawning the child', async () => {
const processor = __dirname + '/fixtures/fixture_processor_bar.js';
const watchFlags = [
'--watch',
'--watch-path=./src',
'--watch-preserve-output',
'--watch-kill-signal=SIGTERM',
];
process.execArgv.push(...watchFlags);

try {
const child = await pool.retain(processor, NoopProc);
expect(child).toBeTruthy();
if (!useWorkerThreads) {
const args = child.childProcess.spawnargs;
for (const flag of watchFlags) {
expect(args).not.toContain(flag);
}
}
} finally {
// Remove only the flags we added so we don't disturb sibling tests
// that also mutate process.execArgv. We push() each flag onto the
// end, so lastIndexOf() targets our own entry — using indexOf()
// would remove a pre-existing --watch* if the runner itself was
// started with `node --watch`.
for (const flag of watchFlags) {
const idx = process.execArgv.lastIndexOf(flag);
if (idx !== -1) {
process.execArgv.splice(idx, 1);
}
}
Comment thread
mohanrajvenkatesan23-04 marked this conversation as resolved.
}
});
});
}
Loading