Skip to content

Perf/prefetch lists - #703

Merged
SanderDeclerck merged 3 commits into
AikidoSec:mainfrom
JoviDeCroock:perf/prefetch-lists
Nov 6, 2026
Merged

SanderDeclerck merged 3 commits into
AikidoSec:mainfrom
JoviDeCroock:perf/prefetch-lists

Conversation

@SanderDeclerck

Copy link
Copy Markdown
Collaborator

The malware, new-packages and safe-patches lists start loading before the proxy starts, so they overlap with the package manager booting.

// install commands main.js prefetches for (pip, uv, npx, ... don't declare
// commandInstallsPackages). Loads are cached, so this is free when
// prefetchLists already started it.
void openSafePatchesDatabase();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Low - Proxy startup fetches safe-patches for commands that never download packages

Invoking a proxied command such as npx --version, pip --version, or npm publish starts the safe-patches feed load even though these commands do not need package-age exemptions. The detached load still performs the feed request and can emit safe-patches failure warnings/retries, adding an unnecessary network dependency and user-visible noise to otherwise local or publish-only commands.

Show fix

Gate this warmup on a command that can perform a package download or on the minimum-package-age setting, or add an explicit commandInstallsPackages/download predicate for the package managers that currently always request the proxy. Leave request-time loading as the fallback for managers without that predicate.

More info - Reply on this comment to give feedback or ignore the issue.

@JoviDeCroock
JoviDeCroock force-pushed the perf/prefetch-lists branch 2 times, most recently from 876ea4e to 59c4298 Compare November 6, 2026 09:28
JoviDeCroock and others added 3 commits November 6, 2026 13:08
The malware and new packages lists were only loaded when the first
tarball request arrived, so a lockfile install waited on their download
and parse. Start all three lists, alongside the existing safe patches
prefetch, before the proxy starts. Requests still await the same cached
promises and report a failed load themselves.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
commandNeedsProxy is true for many commands that never install, such as
npm view, npm config and every python invocation behind the shim. Those
now load the lists on first use as before, instead of downloading the
malware list and the new-packages feed up front.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Prefetching only runs for install commands of package managers that
declare commandInstallsPackages, so removing the warm-up from
startServer meant pip, uv, npx and friends loaded the safe patches list
on their first request instead. Loads are cached, so keeping both is
free.
@SanderDeclerck
SanderDeclerck merged commit b7ec603 into AikidoSec:main Nov 6, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants