jonnify·redis /redis-patterns/production-operations/operating-echomq/the-polyglot-fleet

R8.06.3 · Operating EchoMQ — dive 3

The polyglot fleet.

Scaling a job system eventually means more than more workers in one language. A platform grows services in whatever runtime fits, and they all want one bus. That works only if every runtime agrees, byte for byte, on the identity.

Grounding: the branded-id conformance (echo/apps/echo_data/lib/echo_data/branded_id.ex, self_check!) · the boot-asserted vectors, verbatim · the manuscript home is bcs.0.md / bcs.2.md. The engine is Valkey 9.

Four properties, one that scales a fleet. Select a property of the branded id; the readout names what it gives, and conformance is the one a fleet depends on.
id canonone source of truth Elixir Node Go PostgreSQL · WASM
readout · Select a property. The id is typed, ordered, placed, and conformant; conformance — one canon across five runtimes — is what lets a polyglot worker fleet share one bus.

Static verdicts — typed → the namespace is the type · ordered → the text sorts as the mint instant · placed → hash32 locates the row · conformant → one canon across five runtimes, the property a fleet needs.

§1

The four properties, and the fourth that scales

typed · ordered · placed · conformant

A branded id is a 3-character uppercase namespace plus 11 Base62 characters carrying a 63-bit snowflake ts(41) | node(10) | seq(12). It has four properties: typed — the namespace declares the domain class on the wire and in the type system; ordered — the name sorts as its mint instant sorts, so a table keyed by it is a timeline; placed — one function, hash32, locates a row from the identity alone; and conformant — one source of truth and one vector file make encode and decode identical across five runtimes: Elixir, Node, Go, PostgreSQL, and WASM. Which language runs a worker is a deployment detail.

Conformance is the property a fleet needs. If a Go worker and the Elixir bus disagree by one bit on how an id encodes, the Go worker cannot read a JOB key the bus wrote — the fleet is broken at the identity. One canon, one vector file, removes that failure mode by construction.

Notes on ValkeyThe identity is the only value that crosses between systems on the bus, and the keyspace addresses everything by it — so a worker that agrees on the id can address the same braced keys over the same wire, whatever language it runs — valkey.io/topics/cluster-spec.

The branded id is typed, ordered, placed, and conformant. The first three make it a good key; the fourth — one canon across five runtimes — is what makes a polyglot worker fleet over one bus possible.

§2

The vectors, proven at boot

branded_id.ex · self_check!

Conformance is not a claim on a slide — it is asserted at boot. self_check! mints, encodes, decodes, and hashes a known id down both the native and pure-Elixir paths and checks each result against a committed vector. A node whose native core disagrees with the pure one never reaches its first message. Select a vector below and read what it proves.

Three boot vectors, three guarantees. Select a vector and read which property of the id it asserts — placement, round trip, and total decoding.
self_check!all pass → boot placement → 234878118 parse → {:ok, …} decode → :error
readout · Select a vector. Placement asserts hash32 agrees across paths; parse asserts a round trip; decode asserts an overflow is refused. All three must pass for a node to boot.

Static verdicts — placement → native and pure agree on hash32 · parse → the round trip recovers the namespace and snowflake · decode → an overflow is refused, not wrapped.

echo/apps/echo_data/lib/echo_data/branded_id.ex · self_check! — the conformance vectorsfrozen
# The runtime asserts these at boot, down the native and pure paths.
placement("USR0KHTOWnGLuC")  →  234878118        ← placed: native & pure agree
parse("USR0NgWEfAEJfs")      →  {:ok, "USR", 320636799581945856}
decode("USRzzzzzzzzzzz")     →  :error           ← overflow refused, not wrapped

The placement line is the placed property read at boot: the id USR0KHTOWnGLuC encodes the snowflake 274557032793636864, and hash32(274557032793636864) is 234878118 down both paths. The parse line proves a round trip recovers the namespace and the snowflake exactly. The decode line proves the codec is total in the safe direction — USRzzzzzzzzzzz would overflow 63 bits, and the codec answers :error rather than wrap to a wrong number. These are source truths, not benchmarks — no number on this page is a measurement.

The USR namespace in these vectors is the manuscript's illustrative brand for teaching the id contract; it is the figure exactly as the runtime asserts it. A real application uses its own brands — codemojex mints PLR, ROM, GAM, GES, and JOB.

Conformance is executable: self_check! asserts placement, a round trip, and a refused overflow at boot, down both codec paths. A node that cannot reproduce the committed vectors never serves a message — agreement is a precondition, not a hope.

§3

The honest frame — the conformance target

shipped Elixir · the target

What ships today is honest to state: the bus is Elixir, in echo/apps/echo_mq. A Go or Node worker fleet does not run in production now. The polyglot fleet is the conformance target — the wire protocol and the id contract are precisely what make it reachable, because a worker in any of the five runtimes can mint, parse, and place the same id and address the same braced keys over the same wire. The id is the thread that ties a worker to the bus regardless of the worker's language.

That target is the dedicated EchoMQ course's subject — the polyglot protocol, the full Lua inventory, and the conformance suite that proves a second runtime agrees with the first. This dive is the door to it.

# the conformance target — one id canon, five runtimes, one wire.
Elixir   # the shipped bus today
Node     # a conformant codec + the same vector file → a peer on the bus
Go
PostgreSQL
WASM
# a worker that passes the vectors can drain the same braced queues.
§4

The bridge — conformance to the fleet

pattern → application
the pattern

Share one identity canon across runtimes: one source of truth and one vector file make an id encode and decode identically everywhere, so any runtime can address the same data.

its EchoMQ application

the branded id is conformant across Elixir, Node, Go, PostgreSQL, and WASM; self_check! asserts placement("USR0KHTOWnGLuC") → 234878118 at boot, so a polyglot worker fleet can drain one bus over one wire.

A polyglot fleet is possible only when the identity is conformant. One id canon, proven at boot against committed vectors, lets a worker in any of five runtimes share the bus — the conformant id is the contract that makes the wire portable.

§5

The production angle

grow the fleet

When the fleet grows beyond Elixir, the wire and the id contract are already in place: a new-runtime worker implements the same codec, passes the same vector file, and drains the same braced queues. The operator's path to a second runtime is to satisfy the vectors — not to negotiate a new protocol.

the door · the protocol that makes a peer

The bridge to the EchoMQ course

For codemojex, the bot workers draining the cm queue are an Elixir fleet today, and they mint the conformant ids — PLR, ROM, GAM, GES, JOB — that a future polyglot worker would share. The protocol that makes a second runtime a peer on the bus — the polyglot codec, the full Lua inventory, the conformance suite — is the dedicated EchoMQ Proof pillar, and the scaling layer is the Queue pillar. For the architecture the conformant id expresses, see the bus; for where the bus runs, Codemojex on Fly.

The path to a polyglot fleet is the vector file, not a new protocol. A worker that reproduces the committed vectors is a peer on the wire — the conformance suite that proves it is the dedicated EchoMQ course.

§6

Recap — one canon, five runtimes

end · return to the chapter

The branded id is conformant: one source of truth and one vector file make encode and decode identical across Elixir, Node, Go, PostgreSQL, and WASM, and self_check! asserts the vectors at boot. That is what lets a polyglot worker fleet share one bus over one wire — the scale move of operating the bus, and the bridge to the dedicated EchoMQ course. Colocate, observe, scale: three operator moves, one bus.

§7

References

sources · related

Sources

Related in this course