Skip to content

feat: barebones initial server-side upload-pack implementation#2465

Open
James Gill (JamesPatrickGill) wants to merge 3 commits into
GitoxideLabs:mainfrom
JamesPatrickGill:main
Open

feat: barebones initial server-side upload-pack implementation#2465
James Gill (JamesPatrickGill) wants to merge 3 commits into
GitoxideLabs:mainfrom
JamesPatrickGill:main

Conversation

@JamesPatrickGill

@JamesPatrickGill James Gill (JamesPatrickGill) commented Mar 10, 2026

Copy link
Copy Markdown

tl;dr

I am building a git forge and needed server-side git functionality. I found gitoxide, really liked it and decided to build on it. This is me attempting to upstream it.

I'm happy to rework it and take any feedback!


Hey!

Sorry if me opening this PR out of the blue and is not the process but I have recently started building a small toy git forge in Rust for some fun and when it came to the "git" part I came across gitoxide and really liked it. However, I noticed that the server-side support was not ready for me to build what I wanted, this is my assumption from reading the code anyway.

So I was interested enough to just try to implement it myself and this is what this PR contains. I attempted to go as shallow as possible as to have something to maybe discuss if this is something gitoxide is interested in before I got carried away into all the nitty gritty. That being said it does do some (very little) real stuff:

What is here:

  • gix-transport: Added server-side connection handling: connect message parser and Connection struct
  • gix-protocol: Added server-side upload-pack: ref advertisement writers, wants/haves parser,
    ACK/NAK writers, and upload-pack orchestrators for both protocol versions
  • gix-serve: New crate wiring it together: ref collection from a ref store, pack generation from wanted
    objects, and a top-level serve_upload_pack() entrypoint

There are a lot of caveats though...

Caveats:

  • Ignoring "haves" so servers always send all objects reachable from wants
  • Ignored capabilities for now in the actual serve implementation - they do get parsed though
  • No add-on features supported i.e. shallow-clone etc
  • The pack generation is as naive as it gets, just an unthreaded implementation

However despite being pretty far from done I have an ssh server in my git forge able to serve git clone requests - this can be seen here: https://github.com/JamesPatrickGill/steid


I am very open to guidance and getting this ship shape if you see value. I attempted to match conventions from other crates in the repo as closely as possible.

@JamesPatrickGill James Gill (JamesPatrickGill) changed the title A barebones initial server-side upload-pack implementation feat: barebones initial server-side upload-pack implementation Mar 10, 2026

@Byron Sebastian Thiel (Byron) left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, I had a first brief look and hoped you could squash this work into one commit per crate, in order of dependency?

Thanks a lot!

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As I ramp up for review, could you remove the .tar file so I can add them myself (or chose to not add them at all)?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for getting back!

I've just done those suggested changes and pushed them.

Understand this might be far from the desired state but very happy to put the time in to get the feature (and also figure out what the contribution looks like to gitoxide haha, the first is always hardest)

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi Sebastian Thiel (@Byron), just checking in to see if you've had any chance to review? There's no rush on this obviously, I just have some time coming up I can sink into getting this feature compatible with the codebase.

Any feedback is greatly appreciated 🙏

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sebastian Thiel (@Byron) Just a quick bump on this as I see I have a conflict - before I resolve is this a feature you see merging in the near future?

Matthew Dowell (Mbd06b) added a commit to ethosengine/brit that referenced this pull request Apr 25, 2026
## Summary

Closes 9 compat-mode rows in tests/journey/parity/branch.sh, lands three
reusable foundations consumed by future sprints (commit, push, tag, merge),
and folds in 6 upstream syncs that arrived during the sprint.

## Closed parity rows (all bytes-mode, steward-verified, on-disk state asserted)

- `-v` / `--verbose` — column-aligned name + SHA + subject
- `-vv` — adds `[upstream: ahead N, behind N]` tracking annotation
- `--abbrev=N` / `--no-abbrev` — SHA width override
- `--column=always` — column-major packing
- `-u` / `--set-upstream-to` — writes branch.<n>.{remote,merge}
- `--unset-upstream` — clears them; emits `fatal: branch '<n>' has no upstream information` + exit 128 when absent
- `--track` (create-side) — writes upstream config after ref creation
- `--edit-description` — spawns editor, writes branch.<n>.description

Plus three on-disk-state guard rows (chained gix → gix invocations) so
bytes-mode parity tests can no longer mask missing-side-effect regressions
in branch-config writers.

## Foundations introduced

- gix::Repository::{set_branch_upstream, unset_branch_upstream, set_branch_description}
  with $GIT_DIR/config persistence via gix_config::File::write_to_filter
  on Source::Local sections (matches gix/src/clone/fetch/util.rs::setup_branch_config).
- Branch::DESCRIPTION key constant + SnapshotMut::clear_subsection_value helper.
- gitoxide-core::shared::columns::write_columns — column-major packer (reusable
  for status --column).
- gitoxide-core::shared::editor::edit_file — $GIT_EDITOR/core.editor/$VISUAL/$EDITOR/vi
  resolver + shell-spawn (reusable for commit -e, tag --edit, merge --edit).

## Upstream syncs folded in

- merge: pull upstream gix-main into gix-brit (security hardening + fuzz expansion)
- merge: pull GitoxideLabs#2508 — allow checkouts into non-empty directories
- merge: pull GitoxideLabs#2375 — filters and partial cloning: initial support
- merge: pull GitoxideLabs#2503 — IOUC (UNTR extension) dirwalk speedup
- merge: pull GitoxideLabs#2457 — gix-blame: Incremental
- merge: pull GitoxideLabs#2465 — barebones server-side upload-pack

## Adjustments

- steward agent: surface idiomatic Rust as an explicit design priority,
  reframing vendor/git as the *what* (flag surface, exit codes, output bytes)
  while gitoxide idioms anchor *how*.
- Post-merge cleanup: dropped duplicate clone --filter stub (superseded by GitoxideLabs#2375)
  and pr-2457's orphaned log helpers (our log_all already handles paths).

Plan: etc/plan/foundations-sprint-branch-render-and-config.md
Pre-squash backup tag: pre-squash-foundations

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@Byron

Copy link
Copy Markdown
Member

Thanks James Gill (@JamesPatrickGill) for following up and sorry for the late response! In future I should respond much more swiftly.

Without having looked at the code, I'd be happy to work with you on making this mergeable.
First off, the contribution guidelines changed and now AI disclosure is required, particularly in the commits via message trailers at least. Maybe these can be added next time the PR is rebased.
Generally, I'd love to be able to merge early code by marking it as experimental, which reduces the time it takes to review, while pushing the can down the road as that review will eventually have to happen.

The pack generation is as naive as it gets, just an unthreaded implementation

Packs can already be created via gix-pack and I think there is a gix free pack create command to try it out. From there I'd hope any custom implementation can be removed (if there is one).

Further, to make the implementation useful, I'd love it to deal with capabilities as far as correctness is concerned. Unsupported ones should be rejected, for instance, while SHA-256 can probably be supported without much hassle (but it's fine to leave it as long it rejects what it doesn't do right).

The last mile to make it useful

From what I know, this implementation should already be useful for cloning file:// URLs, which currently spawn git-upload-pack by hand. Ideally, there is an experimental feature-gated version of this which adds a runtime option (in some way) that allows using the built-in upload pack implementation.

From there, gix (CLI) can be built with experimental features (by default, it's a dev-tool), and a flag that allows to turn that on. This can then be used in a basic journey test (gix.sh) to clone with the built-in option.

Finally, you could adjust the test structure to match the other crates.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants