Codemojex · chapter C8

Production deployment.

The whole engine deploys as two machines on one private network. A single OTP release runs the app and everything it supervises; a second, deliberately small machine runs the Valkey the bus and the competitive state live on. The chapter teaches production the way the code records it: a dependency-ordered supervision tree where delivery is at-least-once and every handler is idempotent; a pinned release image built from the umbrella root; and a single-thread datastore sized for a steady latency tail, private on the 6PN.

Grounding — application.ex (the supervision tree) · the release Dockerfile + fly.toml · config/runtime.exs · design §Fault tolerance / §Production deployment / §The pragmatic Valkey node / §Configuration.

Two machines, one 6PN — select a production concern

the release → mix release codemojex builds from the umbrella root in the pinned Elixir 1.18.4 / OTP 28.5.0.1 image. The multi-stage Dockerfile compiles the app over echo_mq · echo_data · echo_wire · echo_store · echo_bot, builds the native codec and the SQLite C-NIF, and ships bin/codemojex start. A rolling deploy keeps the endpoint up; GET /api/health takes a silent machine out of rotation. Only echo_graft stays out.

§1

One tree, in dependency order

application.ex · Codemojex.Supervisor

The runtime is a single supervision tree, and the order is the dependency order. The system of record and the fan-out start first; then the bus and the near-cache tier the hot path needs; then the rate limiter and the bot the notifier depends on; then the four bus consumers; then the leaderboard projection and the periodic sweep; and the Phoenix endpoint last, so the surface only opens once everything behind it is up. The whole tree is :one_for_one; the cache tier alone is :rest_for_one, so a directory restart re-registers its tables.

echo/…/codemojex/application.ex · the supervision tree source

children =
  [
    Codemojex.Repo,                                  # the system of record
    {Phoenix.PubSub, name: Codemojex.PubSub},        # the live fan-out
    {Codemojex.Bus, conn},                           # the shared RESP3 connector

    {Codemojex.Tables, conn},                        # the EchoStore near-cache tier

    Codemojex.RateLimiter,                           # precede the workers that use them
    Codemojex.EchoBot,

    Supervisor.child_spec({EchoMQ.Consumer, score_opts(conn)},   id: :cm_score),
    Supervisor.child_spec({EchoMQ.Consumer, settle_opts(conn)},  id: :cm_settle),
    Supervisor.child_spec({EchoMQ.Consumer, notify_opts(conn)},  id: :cm_notify),
    Supervisor.child_spec({EchoMQ.Consumer, command_opts(conn)}, id: :cm_commands),

    {EchoData.ChampServer, name: Codemojex.Leaderboard},  # in-memory CHAMP view
    Codemojex.Sweep,                                      # the periodic game sweep (cm.5)

    CodemojexWeb.Endpoint                             # the HTTP/WS surface, last
  ]
  |> maybe_committer(conn)                            # optional Graft, when :graft_volume set

Supervisor.start_link(children, strategy: :one_for_one, name: Codemojex.Supervisor)

Delivery is at-least-once and every handler is idempotent, so re-delivery is harmless: scoring is pure, settlement is guarded by the one-shot SET NX, the golden sealed pass re-runs identically, and the wallet is a CHECK-backed transaction. Each consumer leases the job it works, so a crash makes the work visible again rather than lost.

§2

The three dives

C8.1 · C8.2 · C8.3

C8.1

The release

mix release codemojex from the umbrella root in the pinned 1.18.4 / OTP 28.5.0.1 image — the multi-stage build, the native codec, and the rolling deploy behind /api/health.

Dockerfile · fly.toml · bin/codemojex start

builtdive · C8.1

C8.2

The pragmatic Valkey node

One command thread, so echo-valkey is sized for the latency tail: Valkey 9.1 on a shared-cpu-2x, io-threads 1, a 512-MB noeviction guardrail, AOF alone, private on the 6PN.

Valkey 9.1 · shared-cpu-2x · :6390

builtdive · C8.2

C8.3

Fault tolerance and correctness

The one_for_one tree in dependency order, at-least-once delivery, and the idempotent handlers — pure scoring, the SET NX close lock, the CHECK-backed wallet — that make a restart harmless.

application.ex · leases · rest_for_one

builtdive · C8.3

The order is the deploy’s: first the release image and the machine it becomes (C8.1), then the small Valkey node it talks to (C8.2), then the supervision tree and the correctness guarantees that make at-least-once delivery safe (C8.3).

§3

The doors

out of this chapter

→ /echd

References

sources · related

Sources

Erlang/OTP — the supervisor behaviourthe dependency-ordered tree and the rest-for-one cache tier
Valkey — Benchmarking (single-threaded execution)the single-thread sizing the node is built around
Valkey — Persistence (RDB and the AOF)the AOF-alone durability with a one-second loss bound
Fly.io — Fly Machinesthe machines the release and the Valkey node deploy as

Echd, In Depththe CAP weave over the same substrate
EchoMQ — the protocolthe bus whose leases make re-delivery harmless
C7 — The Live Surface on Phoenixthe endpoint the tree starts last
Codemojex — the course homeback to the map