Catch file:sendfile badmatch to keep replica reader alive

[lukebakken]

Note

This PR was prepared by Claude (Anthropic's Claude Code) under the direction of @lukebakken, who reviewed the change before opening it. The fix was surfaced by a 24-hour long-running high-throughput test. The code and analysis are AI-drafted and human-reviewed.

Problem

A 24-hour high-throughput test crashed osiris_replica_reader three times with {badmatch,undefined} raised from inside file:sendfile/8.

The crash is a TOCTOU race in OTP's prim_inet:sendfile/4: it guards on erlang:port_info(S, connected), then calls getprotocol/1, which does a second, unguarded {name,Drv} = erlang:port_info(S, name). When the peer closes the replication socket between those two calls, the second port_info/2 returns undefined and the match raises. (A fully-closed socket instead fails the connected guard and returns {error, einval}; only the race raises.)

osiris_log:sendfile/6 calls file:sendfile/8 inside a plain case that only handles a returned {ok,_} / {error,_}, so the thrown exception is uncaught and propagates through send_file/3 to do_sendfile0/1, terminating the reader with a noisy [error] crash report even though the condition is benign and already recovered by reconnect.

Full analysis in #230.

Solution

Wrap the file:sendfile/8 call in a try that catches error:{badmatch,_} and maps it to {error, {sendfile, Reason}}, the shape do_sendfile0/1 already handles gracefully. Every other exception still propagates so real faults crash loudly.

Add a regression test (send_file_raises_are_returned_as_errors) that mocks file:sendfile to raise the badmatch, since the real race is not reproducible on demand, and add meck as a test dependency.

The getprotocol/1 badmatch is an OTP bug (see #230) to be patched upstream; this guard is defence-in-depth for current OTP releases.

Closes #230

[mergify]

Tick the box to add this pull request to the merge queue (same as @mergifyio queue).

• Queue this pull request