Repository navigation
Can the Backend trait be unsealed? #2253
Description
Activity
The
Backendtrait is sealed on purpose, the API is still evolving (and we've currently added things likeArchiveBackendandCombinedBackend). This allows us to avoid semver breaking changes for users.that communication with the node happens through an RpcClient and this new Backend we're thinking about implementing would instead call functions in the same compiled binary, bypassing the RPC entirely.
I think the right layer to plug into here is the
RpcClientTrather thanBackend. Its a small trait withrequest_rawandsubscribe_rawwhich is already public so that people can provide their own transports. So my suggestion would be to implement the in-progress RpcClientT, which calls directly into the node.Would love to get your thoughts on this 🙏
The Backend trait is sealed on purpose, the API is still evolving (and we've currently added things like ArchiveBackend and CombinedBackend). This allows us to avoid semver breaking changes for users.
I got what you mean. I think that if the
Backendtrait is unsealed, then we would pin the subxt version we use in the eth-rpc to a specific version and we would be conscious of when subxt gets a bump to the major version and plan around that knowing that there could potentially be breaking changes introduced to this trait.I think the right layer to plug into here is the RpcClientT rather than Backend. Its a small trait with request_raw and subscribe_raw which is already public so that people can provide their own transports. So my suggestion would be to implement the in-progress RpcClientT, which calls directly into the node.
Yup, I was looking into this and I found the
RpcClientTtrait. I think that this trait was designed around the idea of async RPC communications happening over JSON, which is quite different from what we're exploring. It would definitely work in this case but there are some issues with it:- We end up paying the cost of JSON serialization and deserialization even though all of the code we're calling is in-process and doesn't require the additional serde round trip (which happens a total of 4 times: 2 serializations + 2 deserializations)
- The
Backendtrait on the other hand does not require us to have the additional serialization/deserialization cost since it's a layer above theRpcClientTabstraction. Also, all of the functions that it exposes are typed functions which is excellent for what we're trying to do.
So, the
RpcClientTtrait would definitely work but it's not quite the most ideal solution for this due to the added overhead that it introducesLet's stick with the
RcpClientTfor a little while, at least until we understand the subxt needs a bit better 🙏The serialization/deserialization impact should be negligible in the eth-rpc context, especially compared to the overhead of a full RPC connection vs in-process
@lexnv following up from polkadot-sdk #11297.
I prototyped the
RpcClientTroute (anInProcessRpcClientoversc_service::RpcHandlers::rpc_query) and it works. The reservation is Omar's above: nothing here is remote, so the JSON round trip is overhead we don't need, and it gets measurable on largedebug_trace*responses. Numbers to follow.On your semver concern, would a feature gate work? Unseal
Backendbehindunstable-backend, same convention asunstable-metadata, so you keep full freedom to change it and nobody who doesn't opt in is affected. Happy to open that PR.Reacted by Alexandru VasileYep an
unstable-backendsounds like the best of both words 🙏
This issue relates to #11297 which is one of the PRs for integrating the eth-rpc into the parachain nodes (e.g., polkadot-omni-node).
Currently, the eth-rpc communicates with the parachain node through a standard WS connection where subxt is used as the client to facilitate this connection. When the eth-rpc is integrated into the node everything will be running in the same process and therefore there will no longer be need for the eth-rpc to communicate with the node through web-sockets or for an RPC connection to the node at all.
One of the ways in which we're thinking about implementing this is through a custom implementation of the
Backendtrait. Currently, all of theBackendimplementations in the codebase assume that communication with the node happens through anRpcClientand this newBackendwe're thinking about implementing would instead call functions in the same compiled binary, bypassing the RPC entirely.In order to do this, we would need the
Backendtrait to be un-sealed to allow us to implement our own custom backend.The questions are:
Backendtrait sealed? Is it truly intended that no user of subxt ever implements their ownBackend?Backend?