lianad: fix JSON-RPC server truncating responses larger than the kernel send buffer - #2115
Conversation
…el send buffer The Unix-domain JSON-RPC server in `lianad/src/jsonrpc/server/unix.rs` silently truncates response bodies on macOS once they exceed the default `SO_SNDBUF` size (8 KiB). Reproduces deterministically with any `createspend` whose PSBT exceeds ~6 KiB pre-base64 — typical of multipath multi-key descriptors. Two interacting issues: 1. The connection inherits the listener's non-blocking flag on macOS (`accept(2)` propagates `O_NONBLOCK`; Rust's `std` does not normalise this, unlike Linux's `accept4`). 2. The response write path uses `serde_json::to_writer(&stream, ...)`, which calls `Write::write` exactly once. Once the kernel send buffer fills, `write` returns either a short count or `WouldBlock`, the trailing bytes are dropped, and the connection is closed — leaving the client with truncated JSON. Fix: - Force the accepted connection to blocking (`stream.set_nonblocking(false)`) so the kernel back-pressures the writer rather than returning `WouldBlock`. - Serialize the response into a `Vec<u8>` first, then `write_all` it. `write_all` loops on short writes; combined with a blocking socket this guarantees all bytes are flushed before the connection closes. Either change in isolation is insufficient on macOS — the `write_all` loop alone still aborts on `WouldBlock` from a non-blocking socket, and the blocking-socket alone still relies on a single `write` call fully accepting the response (which is not guaranteed for any size). Reproduced on macOS 14, aarch64; v13.1 and v14.0 both affected. Verified pre-fix: `createspend` truncated at exactly 8192 bytes mid-base64. Post-fix: full 38628-byte response delivered on the same descriptor + same input. Reported via private email to edouard@wizardsardine.com 2026-04-29 (BufWriter hypothesis — incorrect; the actual mechanism is the one described above). Public disclosure cleared by Edouard 2026-05-05.
|
Hi, could you try if this fix your issue first: https://github.com/pythcoiner/liana/tree/unix_socket |
|
Thanks — built your branch and ran it on macOS 26.3.1 (arm64). It does fix the truncation: 40 KB response comes through intact (vs 8234 bytes on master). The accepted stream inherits blocking mode from the listener, so WouldBlock never fires mid-write. The reason I went the per-connection route instead is shutdown. With a blocking listener, accept() parks until the next client connects, so the loop never re-checks the shutdown atomic. I mirrored both variants in a small harness — setting shutdown=true and timing until the loop exits: master: ~80 ms That bites us because DaemonHandle::stop() sets the flag and then join()s the rpc thread. With the blocking listener, that join hangs on app shutdown — we hit it every time we stop our Tauri sidecar. Keeping the listener non-blocking and flipping the accepted stream to blocking gets the same fix without the regression. The to_vec + write_all swap is just defense-in-depth — POSIX lets write short-return on blocking sockets too (signal interrupts), and to_writer won't loop. Happy to take it in a different direction if you'd rather, though. |
TL;DR
`lianad`'s Unix-domain JSON-RPC server silently truncates response bodies on macOS once they exceed the default `SO_SNDBUF` size (8 KiB). Reproduces deterministically with any `createspend` whose PSBT exceeds ~6 KiB pre-base64 — typical of multipath multi-key descriptors. Two interacting issues; this PR fixes both.
(I reported this privately to Edouard 2026-04-29 with a different — incorrect — hypothesis about `BufWriter` flushing. The mechanism described below is the actual one. Edouard cleared public disclosure today.)
Mechanism
Fix
Two small changes to `connection_handler`:
Either change in isolation is insufficient on macOS — the `write_all` loop alone still aborts on `WouldBlock` from a non-blocking socket, and the blocking-socket alone still relies on a single `write` call fully accepting the response (not guaranteed for any size).
Reproducer
Tested on macOS 14, aarch64. Same Python script reproduces against any deathlock-style multipath descriptor:
```python
import socket, json
def call(method, params):
req = (json.dumps({"jsonrpc": "2.0", "id": 1, "method": method, "params": params}) + "\n").encode()
s = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
s.connect("")
s.sendall(req); s.shutdown(socket.SHUT_WR)
buf = b""
while True:
chunk = s.recv(65536)
if not chunk: break
buf += chunk
return buf
addr = json.loads(call("getnewaddress", {}))["result"]["address"]
buf = call("createspend", {"outpoints": [], "destinations": {addr: 250000}, "feerate": 2})
print(f"bytes: {len(buf)}")
try:
json.loads(buf)
print("ok")
except json.JSONDecodeError as e:
print(f"truncated: {e}")
```
Affected versions
v13.1 and v14.0 both have the buggy code unchanged at the same line numbers. The bug is macOS-specific in practice because Linux `SO_SNDBUF` defaults are higher and `accept` flag-inheritance differs, but the fix is correct on both platforms (`set_nonblocking(false)` is a no-op on Linux when the flag isn't inherited, and `write_all` is strictly an improvement over `to_writer` everywhere).
Why it matters in practice
Per Edouard's reply on the private thread: lianad's RPC API is largely unused by Wizard Sardine's services, and the GUI uses the embedded daemon mode. Out-of-band consumers — `liana-cli` for PSBT-producing operations, or third-party Tauri/Electron wallets running lianad as a sidecar — do hit it. Our use case is the latter; the bug was a hard blocker on the wallet's check-in / sweep flow.
I'm happy to follow up with a Linux `accept4` migration as a separate PR if it's of interest, but that's a larger change. This PR is the minimal fix.