Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 59 additions & 0 deletions .github/workflows/copr-deps.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
name: COPR dependencies

# Dependency RPMs are pinned (see copr-deps/README.md) and published to
# quadsdev/qiip-deps once, not per qiip release. Run this on demand only
# when a dependency version is bumped; the qiip/qiip-dev COPR builds reuse
# the published RPMs as-is.
on:
workflow_dispatch:

permissions:
contents: read

concurrency:
group: copr-deps
cancel-in-progress: false

jobs:
build_deps:
name: Build and submit dependency RPMs
runs-on: ubuntu-latest
container: fedora:43
if: github.repository == 'quadsproject/qiip'
steps:
- name: Install tooling
run: dnf -y install git make rpm-build python3-devel pyproject-rpm-macros python3-pip
Comment thread
sadsfae marked this conversation as resolved.
Outdated

- name: Check out repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1

- name: Work around GHA permission issue
run: git config --global --add safe.directory "$GITHUB_WORKSPACE"

- name: Setup COPR config
env:
API_TOKEN_CONTENT: ${{ secrets.COPR_API_TOKEN }}
run: |
umask 077
mkdir -p "$HOME/.config"
echo "$API_TOKEN_CONTENT" > "$HOME/.config/copr"

- name: Fetch sdist sources
run: |
cd copr-deps
for spec in *.spec; do
version="$(awk '/^Version: /{print $2; exit}' "$spec")"
pkg="${spec%.spec}"
pip download --quiet --no-deps --no-binary=:all: "${pkg}==${version}" -d .
done

- name: Build SRPMs and submit to COPR
run: |
cd copr-deps
for spec in *.spec; do
rpmbuild -bs --define "_sourcedir $PWD" --define "_srcrpmdir $PWD" "$spec"
done
for srpm in *.src.rpm; do
echo "Submitting $srpm to quadsdev/qiip-deps"
copr build quadsdev/qiip-deps "$srpm"
done
127 changes: 127 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
name: Release

on:
push:
branches:
- main
- development

permissions:
contents: read

concurrency:
# Serialize runs on the same branch, but never cancel: python-semantic-
# release pushes use GITHUB_TOKEN (which does not re-trigger workflows),
# and cancelling a running release run would drop its COPR submission.
group: release-${{ github.workflow }}-${{ github.ref }}

jobs:
# Only run release machinery when code (not docs/markdown/assets) changed
# since the last release. Message-level gating (fix:/feat: vs docs:/chore:)
# is python-semantic-release's job; this gate covers markdown-only commits.
gate:
name: Code-change gate
runs-on: ubuntu-latest
outputs:
code_changed: ${{ steps.gate.outputs.code_changed }}
steps:
- name: Check out repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
fetch-depth: 0

- name: Detect code changes since last release
id: gate
run: |
# On main, ignore vX.Y.Z-dev.N prerelease tags from the development
# train so the first development->main merge is treated as a release
# (its tree equals the dev-tagged tree). On development, dev tags are
# the right base and are used as-is.
case "$GITHUB_REF_NAME" in
main) extra="--exclude=*-dev.*";;
*) extra="";;
esac
last_tag="$(git describe --tags --abbrev=0 $extra 2>/dev/null || true)"
if [[ -z "$last_tag" ]]; then
# First release: treat everything as code until the first tag.
echo "code_changed=true" >> "$GITHUB_OUTPUT"
exit 0
fi
changed="$(git diff --name-only "$last_tag" HEAD -- . \
':(exclude)*.md' ':(exclude)docs/**' ':(exclude)assets/**')"
if [[ -n "$changed" ]]; then
echo "code_changed=true" >> "$GITHUB_OUTPUT"
else
echo "Only docs/markdown/assets changed since $last_tag; skipping release"
echo "code_changed=false" >> "$GITHUB_OUTPUT"
fi

semantic:
name: Semantic release
needs: gate
if: >-
needs.gate.outputs.code_changed == 'true' &&
github.repository == 'quadsproject/qiip'
runs-on: ubuntu-latest
permissions:
contents: write
outputs:
released: ${{ steps.psr.outputs.released }}
tag: ${{ steps.psr.outputs.tag }}
version: ${{ steps.psr.outputs.version }}
steps:
- name: Check out repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
fetch-depth: 0

- name: Python semantic release
id: psr
uses: python-semantic-release/python-semantic-release@9a026e9303981c866c3425723009becb2437c757 # v10.6.2
with:
github_token: ${{ secrets.GITHUB_TOKEN }}

copr_build:
name: COPR build
needs: semantic
if: >-
needs.semantic.outputs.released == 'true' &&
github.repository == 'quadsproject/qiip'
runs-on: ubuntu-latest
container: fedora:43
permissions:
contents: read
steps:
- name: Install tooling
run: dnf -y install git @development-tools @rpm-development-tools \
copr-cli make python3-devel pyproject-rpm-macros

- name: Check out release tag
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
ref: ${{ needs.semantic.outputs.tag }}

- name: Work around GHA permission issue
run: git config --global --add safe.directory "$GITHUB_WORKSPACE"

- name: Setup COPR config
env:
API_TOKEN_CONTENT: ${{ secrets.COPR_API_TOKEN }}
run: |
umask 077
mkdir -p "$HOME/.config"
echo "$API_TOKEN_CONTENT" > "$HOME/.config/copr"

- name: Build SRPM and submit to COPR
env:
PROJECT: ${{ github.ref_name == 'main' && 'qiip' || 'qiip-dev' }}
run: |
cd rpm
make srpm NAME="$PROJECT"
srpm_file="$(find . -name '*.src.rpm' -type f | head -n 1)"
if [[ -z "$srpm_file" ]]; then
echo "No .src.rpm found after 'make srpm'" >&2
exit 1
fi
echo "Submitting $srpm_file to quadsdev/$PROJECT"
copr build "quadsdev/$PROJECT" "$srpm_file"
67 changes: 63 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,10 @@
[![Coverage](https://img.shields.io/endpoint?url=https://gist.githubusercontent.com/sadsfae/188b760b19592c8913101f598f7cb382/raw/qiip-coverage.json)](https://github.com/quadsproject/qiip/actions/workflows/ci.yml)
[![vLLM](https://img.shields.io/endpoint?url=https://gist.githubusercontent.com/sadsfae/188b760b19592c8913101f598f7cb382/raw/qiip-vllm.json)](https://docs.vllm.ai/)
[![llama.cpp](https://img.shields.io/endpoint?url=https://gist.githubusercontent.com/sadsfae/188b760b19592c8913101f598f7cb382/raw/qiip-llamacpp.json)](https://github.com/ggml-org/llama.cpp)
[![Release](https://img.shields.io/github/v/release/quadsproject/qiip)](https://github.com/quadsproject/qiip/releases)
[![Dev release](https://img.shields.io/github/v/release/quadsproject/qiip?include_prereleases&sort=semver)](https://github.com/quadsproject/qiip/releases)
[![COPR qiip](https://copr.fedorainfracloud.org/coprs/quadsdev/qiip/package/qiip/status_image/last_build.png)](https://copr.fedorainfracloud.org/coprs/quadsdev/qiip/package/qiip/)
[![COPR qiip-dev](https://copr.fedorainfracloud.org/coprs/quadsdev/qiip-dev/package/qiip-dev/status_image/last_build.png)](https://copr.fedorainfracloud.org/coprs/quadsdev/qiip-dev/package/qiip-dev/)

A QUADS-native inference abstraction framework that automates installation,
drivers, setup, and presentation of disparate, free or idle NVIDIA GPU systems
Expand Down Expand Up @@ -75,6 +79,7 @@ Clients ──► NGINX ──► Inference Proxy ──► vLLM Node A
- [Send a request](#send-a-request)
- [Use with the OpenAI Python SDK](#use-with-the-openai-python-sdk)
- [Chat playground](#chat-playground)
- [RPM installation](#rpm-installation)
- [API Endpoints](#api-endpoints)
- [Claude Code and Codex](#claude-code-and-codex)
- [Administrative access](#administrative-access)
Expand Down Expand Up @@ -104,6 +109,7 @@ Clients ──► NGINX ──► Inference Proxy ──► vLLM Node A
- [Run tests](#run-tests)
- [Lint and format](#lint-and-format)
- [Type check](#type-check)
- [Releases (stable and development trains)](docs/releases.md)
- [Durable provisioning evidence](#durable-provisioning-evidence)
- [Troubleshooting](#troubleshooting)
- [Reading provisioning logs offline](#reading-provisioning-logs-offline)
Expand All @@ -112,7 +118,7 @@ Clients ──► NGINX ──► Inference Proxy ──► vLLM Node A

## Requirements

- Python 3.12 or 3.13
- Python 3.12, 3.13, or 3.14
- [uv](https://github.com/astral-sh/uv) (package manager)
- Node.js for the frontend behavioral tests (CI uses version 24; not required
at runtime)
Expand Down Expand Up @@ -230,8 +236,9 @@ print(response.choices[0].message.content)

### Deploy with systemd

A production deployment ships a systemd unit (`systemd/inference-proxy.service`)
matching the stage/dev convention: repo checkout at `/opt/inference-proxy`
For RPM installs see [RPM installation](#rpm-installation). A git-checkout
deployment (stage/dev convention) uses
`systemd/inference-proxy.service`: repo checkout at `/opt/inference-proxy`
(uv-synced), settings in `/opt/inference-proxy/.env`, the service listening on
port **5000**, and nginx terminating TLS and proxying to it
(`nginx/nginx.conf`). Install it with:
Expand All @@ -257,6 +264,54 @@ System Prompt and retry. Failed turns are not retained in the next request's
history; partial assistant text already shown after a connection failure is
retained so the visible transcript and future context stay aligned.

## RPM installation

Published on COPR for Fedora 43/44. Install the stable train:

```bash
sudo dnf copr enable quadsdev/qiip-deps # dependency RPMs, one-time
sudo dnf copr enable quadsdev/qiip
sudo dnf install qiip
```

> [!NOTE]
> RPM installs use the system Python 3.14 (the Fedora 43/44 default). The
> packages require `python3 >= 3.12` and `< 3.15`.

Two packages, one train each; they ship the same files and cannot be
installed together:

| Package | Train | COPR project |
|---------|-------|--------------|
| `qiip` | Stable (`main`) | `quadsdev/qiip` |
| `qiip-dev` | Development (`development`) | `quadsdev/qiip-dev` |

`qiip-dev` conflicts with `qiip`: enable `quadsdev/qiip-dev` and
`dnf install qiip-dev` when you want the development branch. See
[releases](docs/releases.md) for the versioning, changelog, and badge
details.

Configure and start:

```bash
sudo cp /etc/qiip/conf/qiip.yml.example /etc/qiip/conf/qiip.yml
# edit qiip.yml (admin credentials, huggingface.cache_dir), then:
sudo systemctl enable --now inference-proxy
```

The package ships nginx support (`nginx.conf` and `gen-cert.sh` under
`/usr/share/qiip/nginx`; see [nginx.md](nginx/nginx.md) Method 1), the node
engine bundles under `/usr/share/qiip`, and writes data to `/var/lib/qiip`.

> [!NOTE]
> The `Release` workflow needs three COPR projects before its first run:
> `quadsdev/qiip` (exists), `quadsdev/qiip-dev`, and `quadsdev/qiip-deps`
> (see [copr-deps/README.md](copr-deps/README.md)). Keep the
> `COPR_API_TOKEN` repository secret set, build and enable the dependency
> project, and give the qiip projects Fedora 43/44 chroots with the qiip-deps
> repository enabled. Fedora 45 is not yet covered (its default python3 is
> 3.15, above the supported `<3.15` range).

## API Endpoints

Public endpoints:
Expand Down Expand Up @@ -1416,7 +1471,11 @@ correctly.
### CI badges

After `Quality` passes on `main`, a separate non-blocking job publishes the
coverage, vLLM, and llama.cpp badges to the configured Gist. `GIST_SECRET` must
coverage, vLLM, and llama.cpp badges to the configured Gist. The release and
COPR badges at the top of this README are live: the GitHub release badges come
from the `Release` workflow (stable releases on `main`, dev prereleases on
`development`), and the COPR badges reflect the last build of
`quadsdev/qiip` and `quadsdev/qiip-dev`. `GIST_SECRET` must
be a fine-grained personal access token with only the **Gists: write** user
permission. Prefer a service identity, record the token's expiration, and
replace the repository secret before it expires. To change the publishing
Expand Down
29 changes: 29 additions & 0 deletions UPGRADING.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ The guide separates three kinds of change:
- [Catalog profiles and automatic placement](#28-catalog-profiles-and-automatic-model-placement)
- [Artifact sources and mirror policy](#artifact-sources-and-mirror-policy)
- [Client-visible compatibility changes](#client-visible-compatibility-changes)
- [RPM install migration](#rpm-install-migration)
- [Operational runbooks](#operational-runbooks)
- [Verification checklist](#verification-checklist)

Expand Down Expand Up @@ -753,6 +754,34 @@ until every gateway that may provision or display the node understands it.
| **New surface** | `POST /admin/nodes/pool` accepts `"admin_only": true` (implies `"self_setup": true`) to register an admin-only inference server from an existing OpenAI-compatible URL, plus an optional operator-facing `"name"` shown in the admin fleet view. Admin-only nodes are routable only to bearer tokens of admin-role users or the full-access trust list (HTTP Basic covers UI surfaces only; `/v1` is Bearer-only), never appear in `/fleet/nodes` or public `/v1/models`, render bold with an `admin_only` badge on the admin fleet page, and are removed with the normal pool deletion endpoint. `/admin/nodes[]` gains `admin_only` and `name`. | No action unless you adopt admin-only servers; the admin-only flag and name are additive and default to false/empty for all existing records. |
| **New surface** | `GET /fleet/nodes` returns the registered-node view for signed-in non-admins: admin-only servers removed, operational actions stripped, and nodes owned by another user excluded (endpoint/model/engine/artifact/GPU identity stays private per RFE-107, matching `/v1/models` and the endpoint picker). The dashboard JS uses it for non-admin viewers while admins keep `/admin/nodes`. | Signed-in non-admin users now see the fleet (unowned nodes plus their own); treat `/fleet/nodes` as org-internal inventory that never enumerates another user's private nodes. |

## RPM install migration

The qiip RPM uses a different layout from the git-checkout convention
(`/opt/inference-proxy` + uv venv):

- service unit: `/usr/lib/systemd/system/inference-proxy.service` (system
`python3`, `WorkingDirectory=/usr/share/qiip`, `EnvironmentFile=-/etc/qiip/qiip.env`)
- node engine bundles: `/usr/share/qiip/{auto-vllm,auto-llamacpp,common}`
- config examples: `/etc/qiip/conf/*.yml.example`
- writable data: `/var/lib/qiip` (`provisioning-logs.sqlite3`, `qiip.db`)
- nginx bundle: `/usr/share/qiip/nginx/{nginx.conf,gen-cert.sh}`

To move an existing gateway to the RPM install:

1. Remove any stale unit copy so the RPM unit wins:
`sudo rm -f /etc/systemd/system/inference-proxy.service && sudo systemctl daemon-reload`.
2. Stop the checkout service: `sudo systemctl stop inference-proxy`.
3. Copy writable data: `sudo mv /opt/inference-proxy/data/qiip.db /var/lib/qiip/`
and `.../data/provisioning-logs.sqlite3` (new location defaults).
4. Move settings: copy `INFERENCE_PROXY_*` values from
`/opt/inference-proxy/.env` into `/etc/qiip/qiip.env`.
5. `sudo dnf install qiip` (after `dnf copr enable quadsdev/qiip-deps` and
`quadsdev/qiip`), copy the config examples, then
`sudo systemctl enable --now inference-proxy`.

The old `/opt/inference-proxy` checkout is no longer needed by the service;
keep it only for development.

## Operational Runbooks

### Planned gateway maintenance
Expand Down
Loading
Loading