Fix Triton failing to start: move its HTTP port off Ray Serve's 8000

[kondratyevd]

What

Triton cannot start in the pods #212 deploys:

E0907 12:02:17.927315 1 main.cc:257] "failed to start HTTP service:
Unavailable - Socket '0.0.0.0:8000' already in use "

This commit was written while #212 was in review but landed on the branch after it was merged, so it missed main. Everything else from that PR is already in.

Why it happens

The Ray and Triton containers share the pod's network namespace, and Ray Serve's HTTP proxy binds 8000 on every node, workers included. Triton's default HTTP port is the same 8000, so it loses the race and exits.

Serve's proxy is the one that has to keep 8000. KubeRay gives RayService worker pods a readiness probe that wgets the proxy's health path on the container port named serve, defaulting to 8000 (ray-operator/controllers/ray/common/pod.go). Move Serve and the pods never go ready. So Triton moves instead.

The fix

--http-port=8100. Triton's gRPC stays on 8001, which is what the forwarder dials, and its metrics stay on 8002.

port	who
8000	Ray Serve HTTP proxy
9000	Ray Serve gRPC proxy
8080	Ray metrics
8100	Triton HTTP (moved)
8001	Triton gRPC
8002	Triton metrics

So it cannot drift again, both Triton ports are chart values (triton.httpPort, triton.grpcPort) instead of numbers written into the pod template, and the chart refuses to render if either collides with a port Ray binds, if they disagree with the --http-port / --grpc-port in triton.args, or if the args leave Triton on 8000. The Ray containers are handed TRITON_GRPC explicitly rather than relying on the forwarder's fallback default.

The probes needed no change: they address the port by name.

Tests

tests/manifests/test_ray.py gains the bug class itself — no two containers in the worker pod may declare the same port — plus: Serve keeps 8000, Triton's HTTP is not 8000, the args name the same ports as the values, and the forwarder is told where to dial. Three new refusal cases cover the chart's new failures.

Also checked, no change needed

• Ray only warns about a slow-starting replica (SLOW_STARTUP_WARNING_S); nothing kills it. The forwarder waiting up to fifteen minutes for Triton to load models from a cold CVMFS cache is safe.
• KubeRay 1.7 does not tear down a cluster over a Serve application that is slow to come up.

Verified

pytest tests 970 passing, validate-manifests.sh green, chart renders, ruff (0.16.6, matching the pinned hook) and prettier clean. Not deployed from here.

One thing for whoever debugs inside a pod after this: curl localhost:8000 now reaches Ray's proxy, and Triton's HTTP is at 8100.

🤖 Generated with Claude Code

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!