Improve performance of status on windows

[Special Bread (special-bread)]

This creates a cache of file metadata that is then prefilled by windows API calls to allow per-directory walking instead of per file. As a result performance is much faster.

The cache method is made to minimise the surface area of the change, and is also windows-only where other targets should be unaffected.

Testing status on the linux repo improves speed from ~1000ms to ~300ms - putting this to be roughly on par with libgit2. Faster speeds are possible but would require larger changes, so this is an initial pass while avoiding doing too much.

---

Additional things to consider and discuss perhaps:

1. This does have a little drift I feel, the cache works but perhaps it should not be considered a cache since its thrown away after every git status, and often invalidating these is equivalent to rebuilding these. So using a cache like an actual cache over multiple git statuses is up to the caller, and its pretty complex so the caller would need to know a lot to be able to use this, also for dubious benefit.
2. I did leave some room open for linux based speedups later, but I believe that a different implementation would be needed as lstat on linux is fast and a cache wouldnt really speed things up, the only option here is to instead include a directory keyed cache which would be able to check for untracked files, and meaning that you can do fewer lstats overall, but that would be a perhaps 10-20% speedup, not a 300% (with 1000% possible) speedup like on windows.
3. for reference check out this custom implementation of git status I have here: https://github.com/special-bread/tests-git-status - this can do a git status of linux (the above test case) in ~70ms, but is redone entirely, and also has some slightly different behaviour which is fine for my purposes but not identical to git - i.e. how it considers case sensitivity, how it treats some states as clean if git index entries cancel out, and some other details. I think that its possible to reach and beat the time in there, but that would require more invasive changes which I thought would be fairly rough for a PR that touches a piece of core functionality.
4. see related issue here: "gix status" is slow on Windows #2296

Given that this is a common piece of functionality I would love for someone else to test this too, I myself have been embarrassingly busy recently so this PR cooked for a while, and I may have missed some stuff while working on it on and off.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c27c3aebe0

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "Codex (@codex) review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "Codex (@codex) address that feedback".

[Sebastian Thiel (Byron)]

My apologies for the late response. I guess that I thought I would wait until CI is green. But maybe it makes sense to align and see what your plans are with the PR.

Meantime, let me put it back to draft and you can bring it out of draft once you think it's ready for review.
Merging this is very valuable to me, so I can certainly also do last-mile work if you say so.

[Special Bread (special-bread)]

No worries at all, I too am very busy!

This is me trying to squeeze it out since there were simply too many things and it was a shame to let the work sit without making it upstream. If anything I should be apologizing for taking so long!

(Edit - I fixed a doc link didnt resolve on non windows machines and reran cargo fmt, but otherwise it seemed like previous tests were mostly passing, but a few failed to load and so failed overall, and now rerunning them without changing any of the code seems to make them all pass)

---

I think that some general guidance would be helpful on whether we want to land a minimal scope first (i.e. this) which matches to libgit performance, or look into more invasive changes which will get more speed. Also on the minimal implementation it is also worth asking the question if its a bit much even in the current state as perhaps the idea of the cache should be cleaned up into a windows only preprocessing step for the status, mostly the same thing, but the idea of maybe-later caching seems perhaps more work than its work since invalidating the cache is as expensive as recreating one in many cases.

[Sebastian Thiel (Byron)]

I'd definitely think that as a first pass, this PR doesn't need to get larger, so if you are happy with the Windows speedups, we should work on cashing those in.

1. This does have a little drift I feel, the cache works but perhaps it should not be considered a cache since its thrown away after every git status, and often invalidating these is equivalent to rebuilding these. So using a cache like an actual cache over multiple git statuses is up to the caller, and its pretty complex so the caller would need to know a lot to be able to use this, also for dubious benefit.

Then let's not expose it, and if you want, even keep it Windows only. I am running some performance tests right now, so let's see.

2. I did leave some room open for linux based speedups later, but I believe that a different implementation would be needed as lstat on linux is fast and a cache wouldnt really speed things up, the only option here is to instead include a directory keyed cache which would be able to check for untracked files, and meaning that you can do fewer lstats overall, but that would be a perhaps 10-20% speedup, not a 300% (with 1000% possible) speedup like on windows.

There is a untracked-cache in the Git index which I believe is at least related, and #2503 might be worth a look. If you think it's related, I could prioritize its review - it's a biggie and of unknown quality (i.e. it might be faster to re-prompt it), but maybe it's good to get additional work off the ground more quickly or prevents reinventing something.

Codex says it's related, but I don't know how easy it is for you to evaluate beyond trying the other PR in isolation.

Codex

How is the cache used? Does it affect Windows? Also, is there any overlap with https://github.com/
/pull/2503 ?

The cache is used only in the tracked-file status path.

Call path:

1. On Windows, Repository::status() defaults metadata_cache to Auto.
2. Platform::into_iter() sees Auto and calls build_metadata_cache(repo, None).
3. build_metadata_cache() does a gitignore-aware worktree walk and calls gix_status::metadata_cache::prepare().
4. prepare() uses Windows GetFileInformationByHandleEx(FileIdBothDirectoryInfo) to batch-enumerate metadata into a HashMap<relative_path, CachedMetadata>.
5. That cache is passed into index_worktree_status(), then into gix-status’s index_as_worktree context.
6. In compute_status(), each tracked index entry first checks metadata_cache.get(rela_path). On hit, it uses cached metadata; on miss, it falls back to live Metadata::from_path_no_follow().

So yes, it affects Windows directly, and only Windows. The module, fields, API methods, and cache plumbing are behind #[cfg(windows)]. On macOS/Linux it is not compiled and cannot affect behavior. On Windows it is default-on for normal Repository::status(), but best-effort: if cache construction fails, it silently falls back to live stat calls.

PR #2503 overlaps conceptually but not at the same layer:

• This branch: caches metadata for tracked index entries to avoid per-file lstat during index-vs-worktree status.
• PR use the IOUC (UNTR extension) to speed up directory walks (EXPLORATION) #2503: uses Git’s UNTR / IOUC untracked-cache extension in gix-dir to avoid read_dir calls while discovering untracked files.

They are complementary in status: PR #2503 speeds the dirwalk/untracked side; this branch speeds tracked-file metadata checks, especially on Windows.

There is likely merge overlap, though. Both touch status plumbing and outcomes/tests, and both are sensitive to stat semantics. PR #2503 also fixes index ctime/mtime decoding, which is adjacent to this branch’s Windows cached-stat comparison. I would treat them as interacting PRs: separate goals, compatible design, but expect conflicts or review coordination around status outcomes, stat comparison correctness, and cache invalidation semantics.

3. for reference check out this custom implementation of git status I have here: https://github.com/special-bread/tests-git-status - this can do a git status of linux (the above test case) in ~70ms, but is redone entirely, and also has some slightly different behaviour which is fine for my purposes but not identical to git - i.e. how it considers case sensitivity, how it treats some states as clean if git index entries cancel out, and some other details. I think that its possible to reach and beat the time in there, but that would require more invasive changes which I thought would be fairly rough for a PR that touches a piece of core functionality.

Yes, let's not rush this. git status shows how it's supposed to be done and gix must be compatible (but not bug for bug), with the same results. Ideally there is a complete baseline with Git so it won't be too easy to break. But then… everything is allowed I suppose, even though I'd already be happy with Git-equivalent performance which right now it also didn't always have.

4. see related issue here: "gix status" is slow on Windows #2296

Given that this is a common piece of functionality I would love for someone else to test this too, I myself have been embarrassingly busy recently so this PR cooked for a while, and I may have missed some stuff while working on it on and off.

It can't hurt the let people try the PR!

I hope my answers help somewhat, and my plan is to wait until it comes out of draft for a proper review. Meantime, here is some generated info at your discretion (I just skimmed it).

Codex

Review outcome: changes requested. I confirmed the P2 correctness issue: Windows directory reparse points can be cached with both is_dir=true and is_symlink=true (gix-status/src/metadata_cache.rs:167-168), and compute_status() handles metadata.is_dir() before symlink/type handling (gix-status/src/index_as_worktree/function.rs:478-488). For a tracked symlink to a directory on Windows, this can report Removed from the cached path while the live from_path_no_follow() path would preserve lstat semantics. Reparse points should either carry more precise metadata or be skipped from the cache so the live stat fallback handles them.

Performance note: these benchmarks ran on macOS, so they do not exercise the Windows-only metadata cache path gated by #[cfg(windows)]. They are useful as a non-Windows regression check only.

Environment:

• macOS 26.4.1, Apple M4 Max, 16 logical CPUs, 64 GiB RAM
• rustc 1.95.0 (59807616e 2026-04-14)
• hyperfine 1.20.0
• git 2.50.1 (Apple Git-155)
• ein v0.51.0-23-g6183fd092d

Compared binaries:

• main: origin/main at 8af2691270, gix v0.53.0-89-g8af2691270
• head: PR branch at 64cdbf47e1, gix v0.53.0-91-g64cdbf47e1

Repos selected from ein t find ~/dev:

Repo	Path	Repo HEAD	Tracked files	Worktree size	Dirty entries (git status --porcelain -uno)
small	/Users/byron/dev/github.com/Byron/small	363849d	8	20M	1
gitoxide	/Users/byron/dev/github.com/GitoxideLabs/gitoxide.windows-status-performance	64cdbf47e1	2,857	2.6G	0
linux	/Users/byron/dev/git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux	707df3375124	88,834	7.3G	13
rust	/Users/byron/dev/github.com/rust-lang/rust	ea346509168	54,192	9.2G	10
webkit	/Users/byron/dev/github.com/WebKit/WebKit	6322a8384e74	417,002	16G	9

Command shape:

hyperfine --style basic --warmup 2 --min-runs 10 \
  --command-name "main <repo>" "/tmp/gitoxide-bench-bins/gix-main-8af2691270 --no-verbose -r '<repo-path>' status --no-write > /dev/null" \
  --command-name "head <repo>" "/tmp/gitoxide-bench-bins/gix-head-64cdbf47e1 --no-verbose -r '<repo-path>' status --no-write > /dev/null"

Results:

Repo	main mean	head mean	Outcome
small	7.3 ms ± 0.5 ms	6.9 ms ± 0.4 ms	head 1.05x faster, high relative noise
gitoxide	21.0 ms ± 0.8 ms	20.4 ms ± 0.4 ms	head 1.03x faster
linux	183.8 ms ± 1.4 ms	183.1 ms ± 1.6 ms	effectively neutral
rust	502.7 ms ± 3.9 ms	499.2 ms ± 5.5 ms	effectively neutral
webkit	1.523 s ± 0.022 s	1.537 s ± 0.011 s	main 1.01x faster, effectively neutral

Conclusion: no meaningful non-Windows performance regression showed up across the selected repo sizes. A Windows benchmark is still needed to validate the intended metadata-cache speedup and to verify the corrected symlink/reparse-point semantics once fixed.

[Copilot]

Pull request overview

This PR introduces a Windows-only metadata cache for status to avoid per-file lstat calls (slow on Windows) by pre-enumerating directories and reusing the collected metadata during tracked-file checks, aiming to bring gix status performance closer to libgit2/Git for Windows.

Changes:

• Add a Windows-only gix_status::metadata_cache module that walks the worktree via Windows APIs and builds a path→metadata map.
• Thread the optional metadata cache through gix status configuration (MetadataCacheConfig) and into the gix-status index-as-worktree pipeline.
• Extend mode-change logic to accept pre-extracted file-type/permission bits (to support cached metadata).

Reviewed changes

Copilot reviewed 14 out of 15 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
gix/src/status/platform.rs	Adds Windows-only builder methods to provide/disable/prepare the metadata cache from the status() platform.
gix/src/status/mod.rs	Introduces MetadataCacheConfig and a Windows-only helper to build the cache (gitignore-aware).
gix/src/status/iter/mod.rs	Wires MetadataCacheConfig into iterator construction and passes cache refs into index-worktree status.
gix/src/status/index_worktree.rs	Extends index_worktree_status() signature/context to accept an optional Windows metadata cache.
gix-status/src/lib.rs	Exposes the Windows-only metadata_cache module and re-exports its types.
gix-status/src/metadata_cache.rs	Implements the cache builder and Windows directory enumeration walker.
gix-status/src/index_as_worktree/types.rs	Adds an optional Windows-only metadata_cache field to the tracked-modifications context.
gix-status/src/index_as_worktree/function.rs	Consults the cache on Windows before falling back to live syscalls; adapts mode/stat extraction to cached metadata.
gix-status/src/index_as_worktree_with_renames/types.rs	Threads the optional cache through the “with renames” context on Windows.
gix-status/src/index_as_worktree_with_renames/mod.rs	Passes the cache into the underlying tracked-modifications status call on Windows.
gix-status/tests/status/index_as_worktree.rs	Updates test contexts to include the new Windows-only metadata_cache field.
gix-status/tests/status/index_as_worktree_with_renames.rs	Updates test contexts to include the new Windows-only metadata_cache field.
gix-status/Cargo.toml	Adds hashbrown and Windows-only windows-sys dependency for the cache implementation.
gix-index/src/entry/mode.rs	Adds change_to_match_fs_with_values() to reuse mode-change logic with cached metadata bits.
Cargo.lock	Records new dependency resolutions for added crates/features.

[Special Bread (special-bread)]

I cleaned up the code to be more minimal and instead of styling it as a cache its now styled as a windows only preprocessing step. There is still a parameter to run status without the cache but that is intended mostly for regression tests and such.

In theory there is a tiny amount of overhead due to starting the walk first, then doing git status afterwards. But in practice in every repo with more than a few dozen files in it should be faster since your directory wide walk will capture those in one go as opposed to multiple single file queries

---

Re untracked caches: I admit i didnt comb through the other pr, so this is just my thoughts on those:

In my experience on windows the main slow down is the lstat approach to querying files, and untracked cache doesnt fix that, so it can only provide a modest speedup, and in my experience never helped that much. I believe that you can get a really fast status without it on windows

The linux story for that is nicer - where in linux you dont get any speedup from directory wide queries, but I believe that you can use the untracked cache to skip some comparisons if you have a bunch of files that are untracked, and directories too. Also you can swap some queries for cheaper queries if you rely on it, so thats nice. But im not on a linux machine so this is mostly theory not practice.

An untracked cache is additive on top of a good status implementation, and with windows in particular there is a bit of tension since the fastest way to walk is by directory anyway so it does little good to have an untracked cache, which mostly only works for where entire directories are untracked so you can return early. For that reason in my custom implementation I didnt add support for that since it was more complexity and didnt really help.

---

I havent looked super deeply into why the current implementation is 350ms and not 100ms for example, but I imagine that those speedups would carry over to linux too, since that would be touching the core status implementation, not just adding a windows preprocessing step.

So to speed things up more i would focus on that next, i.e. making sure that the status is fast without an untracked cache first, then once that is fast, looking at what the cache can do - i.e. -> make the current code strong first, then add complexity after

Also to manage expectations a bit - I am sadly amazingly busy so wont be able to devote my time to a full rewrite of status (which is also terrifying since this is public code and must be completely correct and there are lots of edge cases to status!)

Hope this helps :>

[Special Bread (special-bread)]

Ive had another pass at this and added some minor improvements, I think this is ready for review perhaps :> I think that it would ideally be good to have someone else test this or check the edge cases.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5f7a9ca646

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "Codex (@codex) review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "Codex (@codex) address that feedback".

[Sebastian Thiel (Byron)]

Awe and behold: I will review and merge this PR, hopefully in a few ours, but tomorrow latest definitely tomorrow.
Thanks again for working on this, it's more appreciated than I can put into words. Maybe emojies? 🙏🙏🙏🙇🙇🙇😁.

In any case, I am not at all married to how this implementation looks like right now, and maybe we can have both "correct and fast", in that order. You are definitely the expert here, having spent a lot of time both here and in your own from-scratch implementation, so that really should be leveraged.

The UNC extension PR I did close by the way, based on the idea that it can probably be regenerated when needed. It did report great speedups on Linux in the presences of many untracked files in big repositories, but just like you, thus far I didn't really experience these with Git myself. Or I don't remember. Definitely needs work when the time comes :).
From what you write it looks like this would only be useful on the Linux side, and maybe that's how it can be done, eventually, to keep the Windows side as simple as possible.

So to speed things up more i would focus on that next, i.e. making sure that the status is fast without an untracked cache first, then once that is fast, looking at what the cache can do - i.e. -> make the current code strong first, then add complexity after

That sounds very sane and I agree 👍.

Also to manage expectations a bit - I am sadly amazingly busy so wont be able to devote my time to a full rewrite of status (which is also terrifying since this is public code and must be completely correct and there are lots of edge cases to status!)

That's understandable, and is also in my interest as I at least think that the current version is made for correctness first. And while often it's faster than Git, that's not always the case for some reason, so there definitely is work to do.
In any case, I think it's totally fair to pick the low-hanging fruit first, and this PR is definitely ripe :D.

[Sebastian Thiel (Byron)]

Tasks

• restructure commits and rebase
• refackiew (first rough pass, knowing a lot will change)
• can cache computation be lazy instead?
  • Maybe that would affect the multi-threaded operation negatively due to duplicate dir traversals… so maybe keep it an implementation detail and it's just an option with a cache not reused between calls.
  • But if the per-thread fscache and its inefficiencies would be acceptable, then it wouldn't need access to excludes anymore and could also cache tracked files within ignored directories (probably more rare, but Git does it, too)
  • currently there is a lot of complexity due to the explicit multi-threaded dirwalk, which is much like gix-dirwalk but without the extra classification (complications) there.
  • verdict: yes, let's make this lazy, thread-local, while assuming extra accesses at thread-boundaries are fine
• make impl lazy, but only after it can be test-run on Windows VM
  • unfortunately the VM proxy setup doesn't work anymore, so not possible to get it right now. Flying blind.
• can parallel implementation use gix-features better to get rid of the explicit feature-toggle? - lazy takes care of that
• rename to fscache-like and make it an implementation detail, controlled with just a boolean
• have unix implementation with all the tests (and count cache hits and cache-related syscall costs) - can't be done as unix FS syscalls don't provide stat info with the directory traversal.

• what happens if pathspecs are given? It respects pathspecs for precomputation, but has to evaluate them twice - sorted by lazy impl and could be fixed with choosing better boundaries (along directory lines)
• integrate with core.fscache
• flyby: use attributes_only() here via fix: initialise attributes only for possibly better performance #2688
• get it validated, particularly in terms of performance

[Sebastian Thiel (Byron)]

Special Bread (@special-bread) I think I am done with the review and for the last mile validation I will kindly have to ask for your help, in the hopes that you will get to it before the 8th of July which is when my WinVM should be functional again.

First of all, to test it in the real, you can now run cargo run --release --bin gix -- --trace status -s --no-write --untracked no. As baseline, I recommend the commit with title review, which is basically what you submitted but with visibility in the traces for timing information.

From there, you could move right to the last commit of this branch which is the lazy version, much simpler, but it will do a little extra work in the treaded version due to duplicate work along the thread boundaries. I truly hope that the performance is not much worse due to that as the impl for it is so much simpler. This version won't do traces, but one should see the effect in the statistics and basically no actual stat calls.
In that version, you can turn the feature off with cargo run --release --bin gix -- -c core.fscache=false --trace status -s --no-write -u no.

Thanks again for your tremendous help! Fingers crossed I didn't break anything.

PS: Before I forget: i squashed commits, used conventional commit titles were needed, but accidentally added method docs in the feat: add entry::Mode::change_to_match_fs_with_values() commit. The latter I hope isn't a problem.

[Special Bread (special-bread)]

Thanks for cleaning that up! I admit I wasn't sure of the naming conventions so I just picked stuff that seemed clean.

So testing this looks like its fine, its about the same speed as before, and also if you read

Overall the speed of the status tested against the linux repo with untracked is about 500ms on this machine today, compared to 400ms for the git cli, so this mostly is consistent. Without untracked its a bit faster - about 250ms, so thats a neat improvement in speed for that case.

The various feature combos compile on windows - i.e. gix with status, but without parallel works

As a quick note some intermediate commits in here dont compile but I assume thats not a big deal since the tip compiles.

The smaller amount of code is nice, I was starting to think it wasn't as small as it could be, and I guess you did it there!

Also, there is an interesting thing about the fscache - on windows by default its set to on typically, but gix default sets it to off, which is maybe nice. but i guess if you have git installed you get it on by default, and if you use gix with no install on that machine its off by default.

cache on and off gives the same results when i tested, so thats nice.

lastly theres a typo in the test command, it needs = in the untracked flag like so:
cargo run --release --bin gix -- --trace status -s --no-write --untracked=no

so: who knows, but looks good to me :>

[Sebastian Thiel (Byron)]

Thanks so much for testing and for doing that so promptly! This will help to get this merged today 🎉.

Overall the speed of the status tested against the linux repo with untracked is about 500ms on this machine today, compared to 400ms for the git cli, so this mostly is consistent. Without untracked its a bit faster - about 250ms, so thats a neat improvement in speed for that case.

Great news! I'm also glad that we can now see the focus timings without the deer work interfering with the index stats. The reason Git is faster here is that it doesn't interleave both. It runs 1 after the other, and then it can reuse the fscache. So that would definitely be an avenue, but it would be windows specific, add some complexity to gix-dirwalk and to gix, so it's probably something to do only after the official UNC cache is implemented to see what's left on the table. A simpler experiment could be to run the deer walk before or after the index stats so that it doesn't interleave reducing concurrency in the process. But yeah, I will leave all of that to you for now 😅.

As a quick note some intermediate commits in here dont compile but I assume thats not a big deal since the tip compiles.

Indeed, that could be, most commonly because I have to split changes to crates into individual commits, even if they're breaking, for better changelogs.

The smaller amount of code is nice, I was starting to think it wasn't as small as it could be, and I guess you did it there!

To me the amount of code here was less important, but what was important to me is the complexity that now is greatly reduced. So in this case, 2 good things came together :).

Also, there is an interesting thing about the fscache - on windows by default its set to on typically, but gix default sets it to off, which is maybe nice. but i guess if you have git installed you get it on by default, and if you use gix with no install on that machine its off by default.

This is critical and I will look into it. Because Git is supposed to pick up installation-wide settings, and the fsache should be turned on there. So if it doesn't pick it up, it either didn't see the installation configuration or something else happened that I can't yet explain. The assumption was that this will be automatically on, on Windows.

Ah, and now I get it: when Git is not there, and the value isn't set in the configuration that should default to enabled. Yes, let's do that, great catch!

cache on and off gives the same results when i tested, so thats nice.

I don't think I follow this one here because I would expect the cache to have an effect, right? What am I missing?

[Special Bread (special-bread)]

Thanks for merging! I am glad this got in all the way through.

Just to clarify, the fscache is faster, but gives same results for correctness, which is what we want, so that was just an "all good" comment :>