Skip to content

ci: dual-lane simple-math smoke (no-think + thinking) — from #67 - #73

Open
antmikinka wants to merge 4 commits into
mainfrom
merge/pr67-into-main
Open

ci: dual-lane simple-math smoke (no-think + thinking) — from #67#73
antmikinka wants to merge 4 commits into
mainfrom
merge/pr67-into-main

Conversation

@antmikinka

Copy link
Copy Markdown
Contributor

Why

Mainline vehicle for stacked #67 dual-lane CI after product stacks landed:

Stack Mainline PR
#64 ChatSession / EOS #70
#65 stop / thinking floor #71
#66 load hygiene #72

There was no prior merge/pr67-into-main. Stacked #67 still targets obsolete split/pr63-03-load-hygiene.

What

CI-only (3 files):

Area Change
build-mlx-engine.yml Dual-lane matrix: no-think + thinking × ubuntu-rocm / ubuntu-cpu / macos-arm64
budgets ROCm thinking max_tokens=4096; CPU/mac thinking 512 (prior OOM/timeout class)
other smokes enable_thinking: false in action.yml + test-mlx-engine.yml
benches Mode-scoped artifacts bench-results-${{ matrix.os }}-${{ matrix.mode }}

Explicitly NOT in this PR

  • No C++ / thinking_budget.h / server floor policy changes
  • Does not re-land stacked raise-low floor (main keeps fix(server): stop sequences, thinking floor, role:tool 400 (from #65) #71 nullopt-only)
  • No HTTP floor-probe hard gate (omit max_tokens cannot hit nullopt — ChatCompletionRequest::max_tokens defaults to 4096). Floor covered by tests/test_thinking_budget.cpp

Supersedes

Stacked #67 (split/pr63-04-ci-dual-smokesplit/pr63-03-load-hygiene). Close #67 in favor of this PR when green.

Merge policy

Mainline vehicle for stacked #67 dual-lane CI after #70/#71/#72 landed.

- Matrix: no-think + thinking on ubuntu-rocm/cpu and macos-arm64
- Floor probe matches #71 nullopt-only (omit max_tokens; expect -1 → 4096)
- CPU/mac thinking budgets 512 (ROCm stays 4096) to reduce OOM/timeout class fails
- enable_thinking:false on other smoke curls; mode-scoped bench artifacts
- Does not change thinking_budget.h / server floor policy
Quality review: ChatCompletionRequest::max_tokens is int default 4096,
so omitting the field never yields GenerateParameters nullopt and never
emits thinking_budget_floor -1→4096. #71 floor remains covered by
tests/test_thinking_budget.cpp. Dual-lane inference smoke unchanged.
ubuntu-cpu-thinking failed: 512 tokens @ ~1.2 tok/s took ~421s wall,
exceeding curl --max-time 300 (empty client response). Keep infer
max_tokens at 512 on CPU and macOS; only raise infer_curl_max_time to 600.
Keep short job/matrix notes only; drop narrative about floor policy and PR history.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant