BenchScope

English | 简体中文

BenchScope is an open‑source LLM inference benchmarking platform built with Vibe Coding Harness Coding.

A visualization testing platform for LLM model performance & accuracy, supporting models inference with vLLM / SGLang and any OpenAI‑compatible API.

---

Features

• Easy to install — pip install and one command starts the whole web platform.
• Performance testing dual mode — Concurrency Mode (multi-level concurrency load) and Threshold Mode (auto-search the max concurrency meeting the threshold).
• Accuracy testing dual mode — Native（local model weights）/ Serving（API）double model tasks。
• Real-time data feedback — every concurrency result streams into tables, charts and progress in real time.
• Visualization curves — multi-dimensional charts for throughput / TTFT / TPOT / ITL.
• Log cache & download — run logs, mean/P99 summaries and Excel export with online preview & download.

Quick Start

# Install from PyPI
pip install benchscope

# Start
benchscope

# Options
benchscope --port 8080 --no-browser

Development

See docs/Readme.md.

Open Source

• License — Apache License 2.0
• Published on — PyPI: benchscope
• Source — https://github.com/LABELNET/benchscope
• Contributing — Feel free to open issues / pull requests on the source repository.