F7.3 · deep dive 3 · the boot-time DB coupling
The boot reads the log
Portal.Engine.init/1 folds the event log synchronously at startup —
init(:ok), do: {:ok, load()}, and load/0 reaches
Portal.EventStore.Postgres.read_stream/1, a real SELECT … FROM events
under the connection pool’s 4‑second queue_timeout. So the boot makes one hard demand of the
database before it can return: an unreachable one times out and init/1 raises —
failed to start child Portal.Engine, exit 1, and Fly suspends the machine after
ten restarts; a reachable‑but‑empty one boots straight into
relation "events" does not exist. This dive observes that coupling and names its fix; the engine is left
untouched.
We read echo/apps/portal/lib/portal/engine.ex and
event_store/postgres.ex as built — the synchronous call chain
(init/1 → load/0 → read_stream/1), the two crash
modes (unreachable vs reachable‑but‑empty), the stacked precondition (reachable and migrated), and the carried
hardening that touches Portal.Engine below the facade — named here, not built in F7.3.
The synchronous read
Portal.Engine is a single process that holds the Portal’s folded state in memory, and it rebuilds that
state on every start by replaying the event log from the beginning. The replay is the first thing the process does, and it does it
synchronously, before init/1 returns. The path is short and exact:
init(:ok), do: {:ok, load()} calls load/0, which reads the whole stream through
Portal.EventStore.adapter().read_stream(@stream) — the adapter/0 indirection that
resolves to whichever store the release ships with. Under prod that store is
Portal.EventStore.Postgres, so the read is a real query.
Portal.EventStore.Postgres.read_stream/1 answers with a single
Repo.all(from e in Event, where: e.stream == ^stream, order_by: e.seq) — a
SELECT … FROM events ordered by :seq, run through
Portal.Repo over the connection pool. Because init/1 blocks on the result, the pool’s
queue_timeout — the 4‑second window a checkout waits for a free connection — is the clock the whole
boot runs against. If a connection cannot be had in that window, the read does not return; it raises, and a raise in
init/1 means the process never starts.
Step the chain below to read what each call does, and to confirm that every link is synchronous — the boot is waiting on the
SELECT, under the 4‑second queue_timeout, the entire time.
The synchronous boot read · step by step
The whole chain is synchronous: init/1 does not return until read_stream/1’s
SELECT answers — so the database is a hard dependency of the boot, not a thing the engine reaches for later.
Two crash modes, one read
Because the boot blocks on that one read, two different database faults end the same way — the process fails to start.
An unreachable database (the F7.2 reachability faults, left unresolved — a name that does not resolve, a port
that does not answer) means the checkout never completes inside the queue_timeout; the read times out,
init/1 raises, the supervisor reports failed to start child Portal.Engine, and the
node exits non‑zero. Fly restarts the machine, the same read fails again, and after ten such restarts the platform
suspends the machine. This is the floor under the gotcha‑2/3 crash stack: the same symptom family, reached through the same
synchronous read.
A reachable‑but‑empty database — the trap this module names — fails one step later. The connection
succeeds, so the queue_timeout never fires; but the SELECT … FROM events runs against a
schema with no events table, and Postgrex surfaces relation "events" does not exist.
The connection is fine; the table is not there. init/1 raises the same way, and the crash loop is the same.
So the precondition the boot demands is stacked, not single: the database must be reachable (the F7.2 half —
a host that resolves and answers) and migrated (this module’s half — the schema applied release‑native,
the F7.3.1 dive) before read_stream/1 can
return a result. Either half missing, and the boot does not complete.
# echo/apps/portal/lib/portal/engine.ex — the synchronous boot read def init(:ok), do: {:ok, load()} # init/1 blocks on load() before it returns defp load do {:ok, log} = Portal.EventStore.adapter().read_stream(@stream) # … fold the log back into state, synchronously … end # echo/apps/portal/lib/portal/event_store/postgres.ex — the real query def read_stream(stream) when is_binary(stream) do {:ok, Repo.all(from e in Event, where: e.stream == ^stream, order_by: e.seq)} end # unreachable → checkout exceeds the 4s queue_timeout → init/1 raises → exit 1 → suspend after 10 # reachable, empty → SELECT hits a missing table → relation "events" does not exist → init/1 raises
One synchronous read, two ways to fail: a timeout when the database is unreachable, a missing‑relation error when it is reachable but empty — so the boot demands both halves, reachable and migrated, at once.
The carried hardening — named, not built
The obvious fix is to stop coupling the boot to the database. A lazy or retried replay would let
init/1 return immediately and fold the log a moment later — or retry the read a few times before giving up —
so a momentary database blip during a deploy would no longer take the whole release down. That is a real and reasonable hardening, and it
is the right shape. It is also not built in F7.3.
The reason is a boundary, not an oversight. A lazy or retried replay changes how Portal.Engine starts and folds
its state — it edits the engine, which sits below the facade, beneath the deploy layer’s mandate. The deploy chapter’s
rule is to ship the release and document what it depends on, not to reshape the domain underneath it; the engine is left untouched
(F6.8.2-INV7). So this dive does what the deploy layer is allowed to do: it observes the coupling, names the fix, and stops there. No
engine code changes in F7.3.
Until the hardening is built, the coupling stands, and so does the operational rule it implies: bring the database up and migrate it before the release boots. The interactive below contrasts the two behaviours — the synchronous coupling as built, and the lazy/retried replay that is carried — and badges the second clearly as not built.
As built vs the carried hardening
The hardening is named, not built: a lazy or retried replay would survive a momentary blip, but it touches
Portal.Engine below the facade, so it stays outside F7.3’s mandate — the engine is unchanged, and the rule
stands: migrate before the boot.
observed in F7.3
the boot’s synchronous coupling to the database — init/1 folds the log through read_stream/1, so an
unreachable or unmigrated database crashes the release. Documented, with the engine left untouched.
carried, not built
a lazy or retried replay that lets init/1 return and re‑fold later — a change to Portal.Engine, below the
facade and beyond the deploy layer’s mandate (F6.8.2-INV7). Named here; built elsewhere.
That closes F7.3: the schema is applied release‑native, with no mix on the artifact
(F7.3.1), the empty‑database trap is sprung against the writable
primary through the fly proxy path (F7.3.2), and the boot’s
synchronous coupling is named with its hardening carried (here). The reachable half of the precondition was settled in
F7.2 · runtime config & the IPv6 network; together they are the
reachable and migrated the boot demands. Back to the
module hub for the next module of F7.
References
Why a raising init/1 stops a process from starting, the supervisor restart behaviour behind the crash loop, the
connection pool’s queue_timeout that bounds the synchronous read, and the driver that surfaces the missing‑relation and
connection errors.
Sources
- Elixir — GenServer —
init/1, and why a raising init prevents the process from starting. - Elixir — Supervisor —
failed to start childand the restart behaviour behind the crash loop. - DBConnection — DBConnection — the connection pool and
queue_timeout, the 4‑second window the synchronous read waits under. - Postgrex — Postgrex — the PostgreSQL driver that surfaces
relation "events" does not existand the connection errors.
Related in this course
- F7.3.1 · Release-native migration — how a release with no
mixapplies the schema release‑native. - F7.3.2 · The empty-DB trap & the fly proxy path — the reachable‑but‑empty database and the
fly proxypath that migrates the writable primary. - F7.3 · The database goes live — the module hub these three dives open.
- F7.2 · Runtime config & the IPv6 network — the reachability half of the boot’s stacked precondition.
- F7 · Deploy to Fly — the chapter this module lives in.