Module 10 · Chapter III — The engines
The BEAM↔Rust contract.
The native engine lives in the BEAM; the Rust engine does not. So the platform reaches it the way it reaches everything else — over EchoMQ. EchoStore.GraftBackend on the BEAM drives the echo_graft_backend Rust sidecar through one byte-frozen wire, echo_graft_proto: a version handshake on connect, a command/reply lane for requests, and the eg.3 feed lane for advances. The contract is the only coupling — an engine crash is a restart, not a downed orchestrator — and a shared fixture set both runtimes assert against is what keeps them in lockstep. This is rung eg.4. Drive the handshake, a commit, and a crash below.
egraft:cmd:{vol}, the Rust runtime takes the eg.2 fence, the Ack returns on egraft:reply:{client}, and only a durable LSN advance publishes a FeedEvent on egraft:feed:{vol}. A no-op push publishes nothing; a crash resumes from the last-seen LSN.connect runs the version handshake (Hello → Welcome). commit + push sends a Commit, takes the fence, returns an Ack, then publishes a FeedEvent. no-op push shows the S-1 negative: no advance, no event. crash proves recovery from the last-seen LSN.The wire is the contract
graft.4.md · eg.4Transport is RESP3 over EchoMQ. On connect the client sends Hello{proto_min, proto_max}; the backend selects a version and replies Welcome{proto}, or refuses with Incompatible — and a refusal touches no Volume (the S-2 negative: assert the Volume set is byte-identical before and after a rejected connect). Each request carries a corr id the response echoes, on a per-client reply lane egraft:reply:{client}; commands ride per-Volume lanes egraft:cmd:{vol} so one hot Volume can never head-of-line-block another. The message set is small and closed — OpenVolume, Commit, Push, Pull, Read, Snapshot, GetCommit, the Ack/Pages/Err responses, and the opaque eg.3 FeedEvent — and the error enum is exactly {conflict, not_found, version_mismatch, unavailable}; a new kind is a version bump, never a silent addition. The backend is a thin 1:1 dispatch onto the real Rust Runtime (Commit→volume_writer, Push→volume_push, Read→volume_reader→lazy fault): no new engine logic, just session, dispatch, and publish.
Why a sidecar, and how the proof holds
graft.4.md · D-7The Rust engine does blocking object-storage and LSM I/O, so it runs as a supervised backend process, not an in-VM NIF: a NIF would couple crash domains (a segfault takes the orchestrator with it) and force dirty schedulers, while a backend turns an engine crash into a restart. The local-socket/bus hop is noise next to an object-storage commit, the latency regime the engine already lives in. On a crash the client reconnects and resubscribes from its last-seen LSN — the feed cursor is the recovery key, and replay is monotone and gap-free, so a restart is a hiccup, not a gap. The hard part is the cross-runtime contract, and the HIGH-risk mitigation is a single byte-frozen fixture set both runtimes assert against. The eg.4 proof is compositional (Operator ruling D-7 = A): the Rust dispatch is proven in-process over a real Runtime and an in-memory feed sink; the BEAM client’s bus mechanics are proven over a live Valkey on :6390 against an in-Elixir responder that speaks the same Proto; and the two meet at the shared fixtures (identical sha256). The one literally-connected Rust-to-Valkey socket is a deployment concern, deferred by ruling to eg.5/eg.6 — the contract is already proven.
The three dives
module 10The byte-frozen wire & the handshake
echo_graft_proto message by message, RESP3 framing, the Hello/Welcome/Incompatible negotiation, the closed error enum, and one fixture set both runtimes assert byte-for-byte.
Commit, ack, and the feed advance
The command path end to end: Commit → the Rust fence → Ack(lsn) → Push → FeedEvent(lsn) — and the S-1 liveness invariant: a no-op push publishes nothing.
Crash, reconnect & the compositional proof
Supervised restart, resubscribe-from-last-seen-LSN as the recovery key, and why three legs meeting at one byte-frozen wire prove the cross-runtime contract without a single live socket.
Build & check
Trace one commit across the boundary: name the lane it leaves on, the Rust call it dispatches to, where the fence runs, the lane the Ack returns on, and the single condition under which a FeedEvent is published. If you can also say what a no-op push publishes, you have the contract.
What is the one thing that keeps the BEAM and Rust sides in lockstep, and why is a crash a restart rather than an outage? “One byte-frozen fixture set both assert against; the engine is a supervised backend, not a NIF” means you have the module.
References & sources
- Echo records
- graft specs / graft.4.mdeg.4 — the backend, the proto, the handshake, the declared keys, the stories
- graft.roadmap.mdeg.4 in the eg.1–eg.6 ladder; byte-frozen wire as a gate
- graft.engine-split.design.mdthe COEXIST ruling — GraftBackend a peer, native engine untouched
- ewr.design.mdEchoWire / the RESP3 connector the client rides
- External
- RESP3 protocolthe framing echo_graft_proto encodes to
- Erlang NIFsthe crash-domain coupling a sidecar avoids
- orbitinghail/graftthe upstream engine echo_graft is seeded from