Skip to content

Latest commit

 

History

History
333 lines (214 loc) · 33.5 KB

EXCHANGE_NOTES.md

File metadata and controls

333 lines (214 loc) · 33.5 KB

Exchange Notes

Empirical knowledge of the integrated venues — the things that aren't in their documentation and had to be learned by hand, through tests, sniffers, and incidents.


Hosting / infrastructure

Where the endpoints actually run (verified with dig + ipinfo.io). It matters for co-location decisions and for reading network failures correctly.

Exchange Host ASN / provider Physical location Type
Lighter mainnet.zklighter.elliot.ai AS16509 Amazon CloudFront edge (Warsaw from a Mac; Tokyo edge from a Tokyo server) CDN-fronted
Extended api.starknet.extended.exchange AS16509 Amazon Tokyo (EC2 ap-northeast-1) Direct EC2
Zero1 zo-mainnet.n1.xyz AS396356 Latitude.sh Tokyo (bare-metal) Direct bare-metal
Hyperliquid api.hyperliquid.xyz AS16509 Amazon CloudFront edge (Frankfurt from a Mac; Tokyo edge from Tokyo) CDN-fronted
Bybit api.bybit.com / stream.bybit.com AS16509 Amazon CloudFront edge (Warsaw from a Mac; Tokyo edge from Tokyo) CDN-fronted

The consequences:

  1. Lighter, Hyperliquid, and Bybit sit behind CloudFront. The edge is chosen by geo-DNS, so different clients hit different PoPs, and WS stability depends on the CloudFront edge network rather than the exchange's own gateway. Rolling edge restarts are normal for CloudFront. The Tokyo production servers land on a Tokyo edge (nrt*.r.cloudfront.net).
  2. Extended and Zero1 are direct Tokyo. Co-located with the Tokyo bot server — intra-region, ~ms latency. A network glitch on a Tokyo→Tokyo path is unlikely.
  3. A local Mac → any Tokyo venue is a trans-Pacific hop (~10,000+ km), exposed to NAT/ISP flaps. Local WS instability must not be extrapolated to production.
  4. For a new exchange, always verify hosting with dig + curl ipinfo.io/<IP> before making any latency claim.

Incidents

Lighter WS mass disconnect

A synchronous Unexpected EOF on every per-market OB WS and the trading WS at once, across three independent clients (two Tokyo servers and a local Mac). The killed connections had different ages (23 / 88 / 185 min) but died at the same second — so this was a server-side incident, not a connection-age cap. It came in two waves ~3 min apart; the second triggered a halt. There were no Text frames or error messages before the EOF — the server just closed the TCP with no WS Close, which points at the LB/edge layer, not the WS application. Lighter's status page never acknowledged it (below their alerting threshold).

Most likely a rolling CloudFront-edge restart or a Lighter gateway pod rotation. The bot's defenses held completely: reconnect + backoff, plus the 50ms per-symbol staleness gate in spread_detector.rs (max_recv_staleness_secs = 0.05 in production), so stale BestLevels physically could not reach an entry signal. Trading-halt recovery was sub-second in Tokyo (548ms / 555ms) and ~2.5s locally (the trans-Pacific handshake); the full per-symbol recovery cycle ran in 6 minutes. Nothing was changed — the backoff parameters were already optimal and the gates worked. The one follow-up was to log the Close-frame code + reason in ws.rs, which wouldn't have helped here (there was no Close frame) but is useful for routine maintenance windows.

Zero1 local read_timeout (Mac only)

read_frame: timed out (our 30s TCP read timeout) on three per-market sockets — PUMP, BTC, ARB. BTC can't genuinely go 30s silent at the market level, so the socket simply wasn't getting packets: a NAT/ISP/WiFi flap on the long hop. The Tokyo production servers were unaffected (confirmed in the logs). Nothing changed; a WS-level ping on Zero1 per-market would speed up death detection, but it's overkill for a one-off local glitch that doesn't happen in production.


WS behavior — summary

Property Lighter Extended Zero1 Hyperliquid Bybit
Server pings? No Yes, ~15s Yes, every 20s No No
Client must ping? Yes (WS OpCode 0x9, every 90s) No (a ping kills the subscription!) No Yes, JSON {"method":"ping"} (account WS, 30s) Yes, JSON {"op":"ping"} (20s; Bybit kills at 30s idle)
Keepalive Client WS Ping → server Pong Server ping + auto-pong Server ping + auto-pong Client text ping → server text pong Client text ping → server text pong
Timeout with no keepalive ~120s ~30s 60s (3 pings) 60s (account WS) 30s
Dead-connection detection Missed-pong counter (2 in a row → reconnect) Watchdog: data_age_ms > 30s → abort read read_timeout_secs = 30 → read error Watchdog on 60s data age WS_STALE_RECONNECT_MS = 30s watchdog
Max subscriptions per WS No limit No limit 10 1000 (but only 10 connections) Not measured
Recommended ws_ob_mode per_market per_market per_market shared (10-conn limit) single (default)
REST snapshot needed? Yes Yes No (AbsoluteReplace) No (l2Book = AbsoluteReplace) No (orderbook.1 = AbsoluteReplace)
Stagger at per-market 1500ms (78 markets) 50ms 100ms (25 markets) — — (single mode)
Account-WS auth Signed handshake Signed handshake Public (account_id only) Subscribe-time auth HMAC after connect

HOL blocking: all/batched vs per-market

Measured with the bot stopped, across every bot-traded market on each venue, over a 5-minute window after warmup. The match key is (exchange_event_id, asks_json, bids_json).

Metric Lighter Extended Zero1
Comparison consolidated (1 WS × 79 subs) vs per-market (79 WS) all-markets (1 WS) vs per-market (79 WS) batched (3 WS × ≤10) vs per-market (26 WS)
Markets matched 77/79 78/79 26/26
n (matched events) 39,606 42,035 31,377
diff avg +13.8ms +25.0ms +7.4ms
diff p50 +6ms +13ms +7ms
diff p95 +44ms +76ms +21ms
diff p99 +122ms +428ms +43ms
diff max +1567ms +849ms +340ms
Per-market faster 60.4% 83.4% 66.9%
Abs lag, all/batched p99 286ms 651ms — (no exchange ts)
Abs lag, per-market p99 198ms 269ms —

The takeaways: Extended has the worst head-of-line blocking — 3× Lighter's median, 5× at p99, with p99 tails 4× fatter, which is critical for arbitrage. Lighter is moderate, with rare tails past +500ms. Zero1 is mild, because its batched mode is capped at 10 markets per WS (vs 79 for the others in all-markets mode). On all three, per-market wins at every percentile without exception — so the default WsObMode::PerMarket is correct. Regression tests: ws_many_vs_single (Lighter), ws_all_vs_per_market (Extended), ws_batch_vs_per_market (Zero1).


Lighter

Parquet schema cutover: orderbook_depth 5 → 1

The Lighter tick Parquet now writes orderbook_depth = 1 and carries only level 0 in both bid/ask arrays (levels 1–9 zero-padded), after migrating from order_book/{mid} (full L2, 50ms batched) to ticker/{mid} (BBO on every nonce update). The probe measurements and decision are in STRATEGY_OPTIMIZE.md. For backtests that straddle the cutover, set TickLoadOptions.orderbook_depth = 1 on both periods for a clean comparison; the reader warns once and zero-pads when the requested depth exceeds what a file holds. The old full-L2 path (order_book/{mid} + delta reassembly + gap resubscribe) stays in the tree, re-enabled by [exchanges.lighter] orderbook_depth = "full" in local.toml.

WS per-IP limits + rate-limit error 23003

Official limits: 100 concurrent connections per IP, 100 subscriptions per connection, 1000 subscriptions total per IP, 80 new connections per minute (rolling window), 200 msg/min and 50 in-flight from the client. On overflow, Lighter sends a JSON {"error":{"code":23003,"message":"Too Many Connections!"}} and closes the connection — not an HTTP 429, but a text frame inside an already-open WS, easy to miss. In practice, re-running a 79-market test from one IP easily hits the 80/min limit and gives a false "concurrent cap 20" impression — an artifact of the rolling window, not a real cap; leave ≥2 min between full runs. The bot currently catches "Too Many Connections" through the generic error path and retries with backoff, which masks it in logs — worth detecting code 23003 explicitly.

WS payload fields are not identical across connections

In each update/order_book message the order_book object carries offset, nonce, and begin_nonce — per-connection counters that differ between sockets for the same matching-engine event — and the top-level timestamp is a per-receiver send time (ms), also different. What's identical across connections is last_updated_at (matching-engine event time, µs), asks, bids, and code. So to match events across two WS connections you must hash only (last_updated_at, asks, bids), never the full payload.

last_updated_at is strictly monotonic, per event

A 180s live probe (five thin alts — AVAX, FARTCOIN, ZORA, LIT, EDGE — the ones that once "burned") collected 5,065 frames with zero backward jumps on last_updated_at across every market, at the documented ~50ms cadence. Multi-level deltas dominated (700–1300 of ~1000–1400 frames per market), so this is exactly the "one delta touches many levels" case. The engine→publish delay (timestamp − last_updated_at) was p50 24–33ms, p95 53–58ms, max ~142ms, consistent across markets.

So delta.last_updated_at is a real per-batch matching-engine event time, strictly monotonic per market — not "per-touched-level." An earlier incident diagnosis that called it structurally unreliable and prone to 9s backward jumps was wrong (or described a one-off snapshot-replay artifact). The wall-clock max_recv_staleness filter remains the primary defense not because the exchange timestamp is bad, but because wall-clock is cheaper and covers the desync of two feeds on different exchanges — where wall-clock is the only shared reference. A cheap monotonic guard (if new > old { last_exchange_ts = new }) stays in spread_detector.rs as insurance against a resubscribe snapshot-replay edge case. For Lighter, typical Tick→Recv is ~25ms (engine→publish) plus network RTT, so the default 1500ms max_tick_age_ms never blocks in normal conditions.

trade/{mid} leads order_book/{mid} — worth recording public trades

On a batched order book (50ms), public trades are event-driven and can lead. A 240s probe across BTC/ETH/SOL/HYPE/FARTCOIN, measuring the signed wall-clock offset (trade_recv − ob_recv):

Symbol OB gap p50 trades n p5 p50 p95 trade leads OB leads
BTC 50ms 352 −28 +4 +24 40% 51%
ETH 50ms 875 −100 −2 +24 52% 38%
SOL 51ms 206 −44 −1 +20 49% 44%
HYPE 50ms 333 −52 −6 +25 60% 32%
FARTCOIN 101ms 59 −39 +51 +51 14% 86% (low vol)

On liquid markets, trades lead the order book roughly half the time, with a p5 lead of 52–100ms; on the illiquid one the effect vanishes (the OB ticks empty deltas when there are no trades). So the trades feed carries real signal value: if a trade price crosses the current best on the OB, the OB is stale and its best can be ignored for the entry/exit decision. log_public_trades = true for Lighter is the right call. (Caveat: the engine-timestamp difference is unreliable for this — Lighter replays a buffer of recent trades on subscribe with their original engine timestamps, so compare on wall-clock recv, not engine ts.)


Extended

Symbol alias: WTI ↔ CL (dual crude-oil listing)

Extended has two oil listings: WTI-USD (assetName WTI, ACTIVE) and CL-USD (assetName CL, DELISTED but active=true). The exchange accepts orders on WTI-USD but returns positions, fills, and WS updates under CL-USD — input WTI, output CL. The fallout: close_all_positions() doesn't find the CL-USD position when looking up by market_name (market_info only knows WTI-USD); the liq_prices WS cache stores under CL-USD while fetch_liquidation_price() reads WTI-USD → a cache miss; and a fill arrives as CL-USD → sym_id = 0. The fix is a reverse alias map (CL → WTI) when mapping Extended market → SymbolId; any future symbol relisting on Extended can create the same duplicate, so the pattern is worth handling generally, not as a one-off.

Don't send client-side pings

Sending {"type":"ping"} makes the subscription silently die after a reconnect. The server pings on its own and fastwebsockets auto-pong answers. (Confirmed against the Extended docs and their Python SDK, which never calls send() in the WS code.)

publicTrades/{sym} leads orderbooks/{sym} more than Lighter — the feed is critical

Same method as Lighter, over 240s:

Symbol OB gap p50 trades n p5 p50 p95 trade leads OB leads
BTC 99ms 294 −695 −7 +45 60% 38%
ETH 99ms 104 −78 −46 +37 83% 15%
SOL 100ms 67 −207 −207 +20 85% 15%
HYPE 100ms 157 −448 −26 +43 66% 32%
FARTCOIN 103ms 54 −15 +31 +31 7% 93% (low vol)

The lead is larger than Lighter's, which follows: Extended batches the OB at 100ms (half as often) while publicTrades is event-driven, so the gap is wider — a p50 lead of 46ms on ETH, 207ms on SOL. So the public-trades feed matters even more here; it's already on (log_public_trades = true).

Exchange OB batch trades p50 lead (liquid) trades p5 lead
Lighter 50ms 1–6ms 28–100ms
Extended 100ms 7–207ms 78–695ms

Zero1 (01.xyz / N1 / Nord)

  • Max 10 subscriptions per connection. 11+ → an instant "too many subscriptions" close. The bot batches by 10 or goes per-market (confirmed with an n=1..25 sweep).
  • Server pings every 20s; closes after 60s with no pong (3 pings, empty payload). fastwebsockets auto-pong handles it; no client ping needed.
  • No REST snapshots needed — deltas are AbsoluteReplace, so the book builds from deltas. (A REST snapshot once blocked the read loop and starved the server pings, closing the connection at ~7s.)
  • Per-market stagger 100ms; batches of 10+10+N are stable — 25 per-market connections at 100ms stagger stayed alive with zero rate-limiting; 3 batches (10+10+6) served all 26 markets.
  • The account stream is public — account@<id> needs no auth, just the account_id.

The first account frame after connect is a snapshot, not a delta

After every (re)connect, wss://zo-mainnet.n1.xyz/ws/account@<id> sends a snapshot of recent state as its first text frame — fills/places/cancels/balances from a window before the subscription, not a realtime delta. It's undocumented. We caught it in production: 169ms after a restart's account-WS connect, a zero1 WS fill order_id=3380422745 ... arrived — the bot couldn't have placed an order in 169ms, so it was a fill from the previous session (the only fill in the following 2-hour session).

The danger: handle_account_frame saw is_our_taker=true, found no match in pending_orders (the bot was fresh) or active_orders, and called report_external_close(ExternalManual) → on_external_leg_close. That time it was harmless only because the position tracker was still empty (a WARN). With a live delta-neutral position it would have closed the counterparty leg on Extended/Lighter and broken the pair. The guard: run_account_session holds first_text_frame = true and passes is_snapshot to handle_account_frame, which skips fill events from that frame (if is_snapshot { continue; }). Don't touch that guard — any new actionable path off an account frame must honor is_snapshot. The safety net for a real fill missed during a mid-session reconnect is the REST reconcile + periodic liq refresh in the executor.

No matching-engine timestamp (structural)

Neither deltas@<sym> nor trades@<sym> carries a time field. The raw frame:

{"delta":{"last_update_id":3380616434,"update_id":3380616472,"market_symbol":"HYPEUSD","asks":[[42.835,58.61]],"bids":[[42.469,58.61]]}}

update_id is not time — it's a global engine sequence counter for gap detection: frame N's update_id == frame N+1's last_update_id, chained (Binance's U/u), incrementing ~40 per frame on HYPE (shared across markets), used by apply_delta for SequenceGap → reconnect. (The zero1_schema.proto is unrelated to the WS — it's the REST /action envelope for signing and replay protection.)

So ws.rs uses now_ns() for both ts_ns and exchange_ts_ns — a stand-in, because Zero1 has no real event time. As a result the dashboard's Tick→Recv for Zero1 is structurally 0 (the UI shows —). This is a missing source, not lost data. Contrast Extended (msg.ts, a real ~146ms tick→sig) and Lighter (delta.last_updated_at, real per-event time, ~25ms median engine→publish). What you can't do: measure exchange→bot latency for Zero1, or filter on max_recv_staleness_ns against a Zero1 "exchange time." What you can: use last_recv_ns (wall-clock receive) — which the freshness gates already do.

deltas@ batches at exactly 100ms; trades@ ships in the same batch

On Lighter/Extended the public-trades feed can lead the batched order book. On Zero1 it can't. A 60s cadence probe showed deltas@ batched by the server at ~100ms (like Extended, half Lighter's 50ms). A 300s alignment probe then found, for 100% of trade frames, a delta frame with an identical update_id:

Symbol deltas n trades n |trade→nearest delta| p50 / max trade.uid − delta.uid (min/p50/max)
BTC 2869 45 18.7ms / 27.2ms 0 / 0 / 0
SOL 2720 24 1.4ms / 8.9ms 0 / 0 / 0
HYPE 2674 29 10.1ms / 14.1ms 0 / 0 / 0
ETH 2673 24 7.9ms / 18.5ms 0 / 0 / 0

So trades@ and deltas@ are two outputs of one engine commit, not independent channels — the wall-clock spread (up to 27ms) is client-side jitter between two parallel sockets, not a publish-time difference. trades@ gives no latency advantage over deltas@, so log_public_trades for Zero1 is off. The most available information on Zero1 is update_id (the engine sequence) and last_recv_ns (wall-clock, the only honest axis for freshness gates). Regression tests: deltas_cadence_probe, deltas_vs_trades_align.

An unknown market name in a shared URL silences every subscription

If even one market name in the path wss://.../ws/deltas@X&deltas@Y&… doesn't exist on Zero1, the server silently accepts the handshake, sends no close/error, and sends no text frame on any market, valid ones included — the connection just hangs until the server ping timeout. We caught this with a filler list containing markets Zero1 doesn't list (DOGE/AVAX/LINK): a SOLO deltas@BTCUSD socket collected 1,144 frames while a SHARED socket with the same 10 subs got 0 in 120s; swapping in REST-verified names restored 1,081 frames. So any code that builds a URL from a symbol list must validate against Zero1's traded markets (SQLite exchange_symbols or REST /info) — and a live-but-silent connection points first at a typo/nonexistent market name, not a network problem. (The same probe also showed a dedicated socket on the hot leg gives no cadence advantage — Δp50 +0.7ms, Δp99 +2.7ms, pure noise — so there's no "separate socket for the signal leg" option; the 100ms batch is entirely server-side.)


Hyperliquid

Hosting: CloudFront (like Lighter)

api.hyperliquid.xyz → 18.245.60.* → a CloudFront edge (AS16509 Amazon), geo-DNS to the nearest PoP. RTT from a Mac to an EU edge is ~460ms median (POST /info); from Tokyo, expect ~10–50ms.

BBO channel — event-driven, ~10/s on BTC, 5.6× faster than l2Book

A 60s side-by-side of bbo vs l2Book for BTC:

Channel frames rate gap p50 gap p90 gap p99 gap max
bbo 632 10.5/s 73ms 209ms 413ms 580ms
l2Book 113 1.9/s 536ms 764ms 1186ms 1492ms

BBO delivers 5.6× more updates — it's event-driven (fires on every best bid/ask change), while l2Book is a full 20-level snapshot each time. Frame format: {"channel":"bbo","data":{"coin":"BTC","time":...,"bbo":[{bid},{ask}]}}, first element bid, second ask. So bbo is the primary tick feed; l2Book isn't needed in BBO mode.

Tick→Recv floor is block-gated at ~150–200ms (architectural, not a bug)

The dashboard shows Hyperliquid at avg 178ms / p50 250ms / p99 500ms (Tokyo production). This is a fundamental floor of the Hyperliquid L1, not a processing artifact or a network problem:

  1. time in the BBO frame is the L1 block timestamp, not a send time or matcher wall-clock. The docs: BBO is "sent only if the bbo changes on a block" — the market-data channels are block-gated, not tick-by-tick.
  2. HyperBFT block cadence is ~0.1s median, p99 <0.5s — which matches what we see.
  3. Every public market-data channel (bbo, allMids, activeAssetCtx, l2Book, webData2) is block-gated; bbo is the fastest. There is nothing faster than BBO.

Local measurement (Mac → Warsaw edge, BTC, 60s) gave min 203ms / avg 525ms / p50 475ms / p99 1145ms; subtracting the ~350ms geographic RTT leaves a ~150–200ms Tokyo floor, consistent with HyperBFT p50 ~100ms + WS propagation. So now − data.time measures block_production_lag + consensus_propagation + network — which is exactly the real cost of price freshness for arbitrage. The only documented way to reduce it is running a non-validating HL node (from the "Optimizing Latency" docs) — there's no premium endpoint, CloudFront bypass, or faster WS channel.

Practical consequences: keep max_tick_age_ms at 500–750ms for HL (50–100ms, as on Lighter, would discard healthy HL ticks); use HL-involving signals only for events longer than ~200ms (a fast Lighter↔Extended arb gains nothing from HL, which is always a block staler). The dashboard difference (Lighter 2ms / Extended 6ms / HL 178ms) reflects how each venue publishes market data, not "CEX vs DEX" — all four of Lighter, Extended, Zero1, and Hyperliquid are non-custodial DEXs. HL's feeds are block-gated (a HyperBFT snapshot once per block); Lighter (zk-rollup sequencer) and Extended (off-chain CLOB + Starknet settlement) push updates off-block, so their floor is network only, not consensus. Test: hyperliquid/bbo_latency.

Other WS facts

  • l2Book = AbsoluteReplace — each message is the full 20 levels per side, no isSnapshot flag, no update_id for gap detection. Fine for a future full-book mode; not needed in BBO mode.
  • The client must ping (Camp B), 60s timeout. The server closes if it hasn't sent the client anything for 60s; the client sends {"method":"ping"} → {"channel":"pong"}. BBO/trades don't need it (data flows continuously), but the account WS does when idle — an early bug had it dropping every ~50s with "Unexpected EOF." The account WS uses WriteTaskMode::TextPingRelay at 30s (50% of the timeout).
  • Public trades carry the taker side ("B" = buyer aggressor, "A" = seller aggressor) and an engine timestamp; the first frame after subscribe is a ~30-trade snapshot replay (like the Zero1 account stream).
  • Thin-market idle gap maxes at ~3.3s (STX), so ws_read_timeout_secs = 30 is safe.
  • Connection limits: max 10 connections, 30 new/min, 1000 subscriptions, 2000 msg/min. With ~70 symbols, per-market would need ~140 connections — so BBO/trades use a shared/batched supervisor (1 WS, N subscribe messages), as on Zero1; the account WS is already a single connection.

REST: two JSON-POST endpoints

Reads go through POST /info {"type":"..."}, trades through POST /exchange with an EIP-712 signature: meta (190 markets, szDecimals 0–5, maxLeverage 3–40×), metaAndAssetCtxs (mark/mid, funding, OI), l2Book, clearinghouseState (positions, margin, equity, and liquidationPx — the engine gives the liq price directly), userFees (base tier maker 0.015% / taker 0.045%), userFills (per-fill fee, crossed flag, cloid). Auth is EIP-712: action → msgpack → nonce → keccak → Agent struct → sign with domain {name:"Exchange", chainId:1337}. Replay protection: nonce = timestamp_ms.


Bybit

The one centralized venue in the set (off-exchange, unlike the four DEXs). Its matching engine is proprietary and not block-gated, so its latency floor is network + processing, not consensus.

Hosting: CloudFront

api.bybit.com and stream.bybit.com both resolve through CloudFront geo-DNS (AS16509) — Warsaw from a Mac, a Tokyo edge (nrt*) from Tokyo, like Lighter and Hyperliquid. REST RTT from a Mac to the Warsaw edge is ~390ms median (/v5/market/time); from Tokyo, ~5–15ms.

WS

  • orderbook.1 = AbsoluteReplace, depth 1, no gap detection. Every frame after subscribe orderbook.1.BTCUSDT is type: "snapshot" with data.b/data.a holding exactly one element (best bid / best ask) — a full replacement, so no REST pre-fetch or delta merging. Fields: {topic, type, ts, data:{s, b, a, u, seq}}, where ts is the matching-engine publish timestamp (ms), used as exchange_ts_ns.
  • Keepalive (Camp B), 20s text ping. Over 60s of raw opcodes the server sent zero WS-level Pings; a client {"op":"ping"} gets a JSON {"success":true,"ret_msg":"pong","op":"ping"}. Bybit kills at 30s idle, so all three Bybit WS (OB, account, trade) use WriteTaskMode::TextPingRelay at 20s.
  • Tick latency (BTCUSDT, Mac → Warsaw edge, 120s): 16.4 frames/s; inter-arrival p50 25ms / p99 459ms; Tick→Recv p50 82ms / p99 155ms. From Tokyo, expect ~10–20ms — network only, since the matching engine is event-driven.
  • depth=1 vs depth=50: same frame rate and Tick→Recv, but depth=50 has ~2× smaller p99 inter-arrival gap (62ms vs 116ms) — it batches levels 2–50 into one frame, so fewer micro-frames with the same best level. Current phase stays on orderbook.1 (BBO-only, minimum bytes); orderbook.25 doesn't exist for linear (subscribe returns 0 frames — a Phase 1 finding).
  • Thin markets (e.g. MNTPERP, the thinnest USDT-settled perp) gap to p99 5.7s but always quote both sides (0 empty bid/ask over 60s), with the same Tick→Recv as BTC — so freshness must be measured on ts, not receive time, which the BestLevels::exchange_ts_ns + max_tick_age_ms path already does. The REST filter (status=Trading & contractType=LinearPerpetual & settleCoin=USDT) correctly drops dated futures like BTCUSDT-08MAY26.

REST: instruments-info

Where the limits live: lotSizeFilter.qtyStep/minOrderQty/maxOrderQty (all String); lotSizeFilter.minNotionalValue = "5" (5 USDT — the docs' minOrderAmt is null for linear perps); priceFilter.tickSize; leverageFilter.maxLeverage (String, e.g. "100.00"). riskParameters holds only priceLimitRatioX/Y — there is no MMF field. So the code sets mmf: None and relies on liqPrice from the WS position stream for the SL trigger; an "MMR = IMR/2" heuristic (as on Hyperliquid) is a sanity check, not a primary source, since Bybit's MMF is tiered.


Margin / leverage / MMF — per-exchange API sources

Fetched at startup via ExchangeFactory::fetch_market_meta() into SQLite exchange_symbols. Don't copy an MMF formula from another exchange — each defines it differently; verify in the docs.

Exchange Endpoint max_leverage MMF
Lighter GET /api/v1/orderBookDetails 10000 / min_initial_margin_fraction (BTC: 200 → 50×) maintenance_margin_fraction / 10000 (BTC: 120 → 1.2%)
Extended GET /api/v1/info/markets → tradingConfig maxLeverage (String, e.g. "50.00") riskFactorConfig[0].riskFactor — this is the MMR, don't halve it
Zero1 GET /info → markets[] floor(1 / imf) (BTC: 0.02 → 50×) mmf directly (BTC: 0.01)
Hyperliquid POST /info {"type":"meta"} → universe[] maxLeverage (BTC: 40) not in the API — heuristic 0.5 / maxLeverage (= IMF/2, per the margin-tiers docs)
Bybit GET /v5/market/instruments-info?category=linear leverageFilter.maxLeverage (BTC: "100.00") not published → mmf: None, use liqPrice from the WS position topic

Notes: Lighter's margin data is in /orderBookDetails, not /orderBooks. Extended, Hyperliquid, and Bybit all use tiered margin — the base tier applies to small positions; Bybit's tiered MMR lives in a separate /v5/market/risk-limit API that isn't integrated yet (the engine's liqPrice covers it).


Nado

Status: pre-integration research. Phase 1 empirical probes haven't run yet. Data is from the official docs and the Rust SDK (nadohq/nado-rust-sdk); ⚠️ marks facts that need an empirical probe to confirm.

A non-custodial CLOB DEX, a fork of Vertex Protocol, on Ink L2 (EVM-compatible, chainId 57073). Products: BTC/ETH/SOL/BNB/XRP perps (USDT0 settlement), spot wETH/USDT0, and money markets. Ex-Kraken team; 5–15ms latency (off-chain sequencer + on-chain settlement batches). The Rust SDK is the reference for signing.

Endpoints (mainnet)

Service URL
Gateway REST https://gateway.prod.nado.xyz/v1
Gateway WS (writes + queries) wss://gateway.prod.nado.xyz/v1/ws
Subscriptions WS (market data) wss://gateway.prod.nado.xyz/v1/subscribe
Archive V1 / V2 https://archive.prod.nado.xyz/v1 · /v2
Trigger API (SL/TP) https://trigger.prod.nado.xyz/v1

(Testnet: prod → test.)

Signing: EIP-712

ECDSA with an Ethereum key (k256), domain {name: "Nado", version: "0.0.1", chainId: 57073, verifyingContract: <op-dependent>}. verifyingContract is address(product_id) for Place Order and the endpoint contract for everything else (Cancel, StreamAuth, Withdraw, LinkSigner). The subaccount sender (bytes32) is a 20-byte wallet address + a 12-byte ASCII subaccount name, zero-padded.

struct Order {
    bytes32 sender;    // 20b address + 12b subaccount name
    int128  priceX18;  // price × 10^18, positive
    int128  amount;    // qty × 10^18, positive=buy, negative=sell
    uint64  expiration;
    uint64  nonce;     // (recv_time_ms << 20) | random(0..1_000_000)
    uint128 appendix;  // bit-packed flags
}

The appendix bit layout packs order_type (bits 9–10: 0=GTC, 1=IOC, 2=FOK, 3=PostOnly), reduce_only (bit 11), trigger_type (bits 12–13), isolated-margin flags, and builder fields; e.g. IOC = 1 << 9 = 512, IOC + reduce-only = 2560. Expiration is embedded in the nonce (recv_time = nonce >> 20, a ~90s window); for parallel sends, use an AtomicU64 with distinct random components. Signing stack: ethers::Wallet<k256::ecdsa::SigningKey> → get_eip712_digest() → keccak256(0x1901 ++ domain_sep ++ struct_hash) → 65-byte hex.

WS

  • Subscriptions (/subscribe), Camp B: the client must ping every 30s (⚠️ frame format TBD), permessage-deflate is mandatory, max 5 authenticated connections per wallet, and only order_update needs auth (via a StreamAuthentication EIP-712) — everything else is public.
  • Gateway (/ws), write path: place/cancel over {"type":"execute","execute":{"place_order":{...}}}; keepalive ⚠️ TBD.
  • Streams: best_bid_offer (on-change BBO — the tick-feed candidate), book_depth (~50ms incremental — ⚠️ delta mode TBD), trade, order_update (auth, fill routing), fill, position_change (absolute position, needs local state tracking), funding_rate, funding_payment, liquidation.

REST queries

all_products (margin weights, min_size), symbols, subaccount_info (positions, balances, health), fee_rates (taker_fee_rates_x18[] by product_id), subaccount_orders, market_prices (oracle/mark), contracts, nonces. All numbers are x18 (×10¹⁸) as strings, e.g. "300000000000000" = 0.03% = 3 bps. Rate limits: 1200 queries/min, 600 executes/min; a place-order with leverage is weight 1 (600/min), without leverage weight 20 (30/min).

Margin / MMF

20× for perps (BTC/ETH/SOL/BNB/XRP), 5× spot. Maintenance weights (maintenance_asset_weight, maintenance_liability_weight, x18-encoded) drive the liq-price formula: Long oracle × (1 − max((1 − maint_asset_weight)/5, 0.005)), Short oracle × (1 + max((maint_liability_weight − 1)/5, 0.005)). The liq price is a client-side compute from a Chaos Labs oracle price (on-chain), settled in USDT0.

Trigger API (SL/TP)

A separate POST https://trigger.prod.nado.xyz/v1/execute (not the Gateway WS): a base Order + a PriceRequirement (oracle / last-trade / mid-book source), with appendix bits 12–13 = trigger_type.

Linked signer

A delegated EOA key that signs on behalf of a subaccount, so the bot holds only the linked-signer key, not the main key. Bound via a LinkSigner execute (EIP-712, on-chain slow mode, 50 ops/7 days); after that, all orders and cancels are signed by the linked key.

To resolve in Phase 1

WS ping format (WS OpCode vs text JSON) for both the Subscriptions and Gateway sockets; book_depth delta mode (incremental vs AbsoluteReplace) and whether a REST snapshot is needed; thin-market idle gap; connection/subscription caps; WS-vs-REST latency; exact product_ids; per-symbol min qty/notional; the trigger-order signing scheme; whether position_change replays a snapshot on connect; and whether order_update auth uses the linked signer or the main key.