jonnifyelixir
F7 · Deploy to Fly / F7.2 · runtime config / F7.2.2 · the IPv6 trio

F7.2 · deep dive 2 · Fly’s IPv6-only private network

F7.2.2

Fly’s IPv6-only private network

Fly’s private network — the 6PN — is IPv6-only. A .internal name like portal-postgres.internal carries only an AAAA record. So before a BEAM node can resolve a private name, connect Postgrex to it, and cluster over it, three independent settings in echo/fly.toml must agree: ERL_INETRC (the resolver config), ECTO_IPV6 (the Postgrex address family), and ERL_AFLAGS (the BEAM distribution transport). Two of the three take effect from being set. The third — ECTO_IPV6 — takes effect only when runtime.exs reads it, and that gap is the trap the first bring-up walked into.

We read the echo/fly.toml [env] trio against echo/config/runtime.exs and echo/infra/inetrc as built. The headline lesson is the set-versus-consume distinction: naming an env var configures nothing the application does not also read.

Three settings, one v6-only network A vs AAAA — the decisive fix Set is not consume

Three settings, one v6-only network

The echo/fly.toml [env] block carries three values that share one purpose: making the BEAM and its database connection speak IPv6 on a network where nothing else is offered. Each governs a different layer.

ERL_INETRC = "/app/inetrc" points Erlang’s resolver at the config file F7.1 ships. Its content (echo/infra/inetrc) sets {lookup, [native, file, dns]} — the native OS resolver first, so /etc/resolv.conf and both address families are honoured, then /etc/hosts, then Erlang’s built-in resolver. It names the Fly internal nameserver {nameserver, {16#fdaa, 0, 0, 0, 0, 0, 0, 3}} (fdaa::3, which resolves .internal and .flycast within the private network), the public fallbacks {1, 1, 1, 1} and {8, 8, 8, 8}, and {search, ["internal"]} so a short name portal-postgres resolves as portal-postgres.internal, alongside the timeout, retry, and cache terms. Erlang consumes ERL_INETRC on its own — naming the file in the env var is enough, provided the file exists (that condition is gotcha 1, F7.1’s concern).

ECTO_IPV6 = "true" is the trap. Setting it in fly.toml does nothing on its own; it takes effect only where runtime.exs reads it into the Repo’s socket_options. That is the next section.

ERL_AFLAGS = "-proto_dist inet6_tcp" makes BEAM distribution run over inet6, so libcluster’s DNSPoll over echo-portal.internal (the echo/config/prod.exs fly_dns topology) forms the cluster on Fly’s IPv6 substrate. This is distinct from Postgrex’s socket_options: one governs node-to-node distribution, the other the database connection — and both must speak inet6 on a v6-only network. Like ERL_INETRC, the BEAM runtime consumes it from being set.

The fly.toml [env] trio · three layers of one v6-only network

RESOLVER — ERL_INETRC = "/app/inetrc" native→file→dns · fdaa::3 · search internal BEAM reads it DB FAMILY — ECTO_IPV6 = "true" set in fly.toml · inert until runtime.exs reads it app must read it DISTRIBUTION — ERL_AFLAGS = "-proto_dist inet6_tcp" libcluster DNSPoll over echo-portal.internal forms the cluster BEAM reads it

Two of the three settings reach the runtime from being named in the deploy config; the middle one, ECTO_IPV6, reaches nothing until application code reads it.

A vs AAAA — the decisive fix

Postgrex resolves its host with the address family of its socket_options. Left on the default :inet, it issues an A-record (IPv4) query. Against an AAAA-only .internal name, an A-query returns :nxdomain — a connection failure that crash-loops the boot, not a resolver bug. The name is sound; the question was asked in the wrong address family.

The fix is a single line in echo/config/runtime.exs: the maybe_ipv6 binding reads the env var, and socket_options on Portal.Repo carries the result. With [:inet6] in place, Postgrex issues an AAAA query, the .internal name resolves to its fdaa:… address, and the boot connects. The empty-list default (else: []) keeps a local or dev DATABASE_URL — an IPv4 loopback, or the fly proxy migration path — on its default family, unaffected.

Toggle the family below against a fixed AAAA-only host and watch the query, the resolver result, and the boot outcome move together.

# echo/config/runtime.exs — the maybe_ipv6 binding + the Repo socket_options (the decisive line)
# Fly `.internal`/`.flycast` names carry ONLY AAAA records. The default `:inet` issues an
# A-record query against an AAAA-only host and gets `:nxdomain` — a connection failure, NOT
# a resolver bug. `fly.toml` sets ECTO_IPV6=true; THIS line consumes it.
maybe_ipv6 = if System.get_env("ECTO_IPV6") in ~w(true 1), do: [:inet6], else: []

case {System.get_env("DATABASE_URL"), env} do
  {url, _} ->
    config :portal, Portal.Repo,
      url: url,
      pool_size: String.to_integer(System.get_env("POOL_SIZE", "10")),
      socket_options: maybe_ipv6     # [:inet6] when ECTO_IPV6 is set, else []
end

Resolve portal-postgres.internal (AAAA-only) · pick Postgrex’s socket family

HOST · portal-postgres.internal — AAAA fdaa:0:1:a7b:2c::2 (no A record) QUERY ISSUED A  portal-postgres.internal  (IPv4) RESOLVER RESULT :nxdomain — no A record for an AAAA-only name BOOT OUTCOME Postgrex cannot connect — the boot crash-loops

The default :inet A-queries an AAAA-only name and gets :nxdomain; [:inet6] AAAA-queries it and connects — the address family, not the network, is the difference.

Set is not consume

Here is the headline, stated plainly. ERL_INETRC and ERL_AFLAGS are consumed by the BEAM runtime the moment they are set — no application code reads them. ECTO_IPV6 is consumed only by application code: the maybe_ipv6 binding and the Portal.Repo socket_options in runtime.exs. So setting ECTO_IPV6 = "true" in fly.toml without that code is a silent no-op — it produces the same :nxdomain as an unreachable host. A Phoenix-generated runtime.exs ships the consume line by default; a hand-built one (this umbrella) can set the env var and forget the line.

The map below has three toggles, one per setting. Flip them and read the three independent outcomes: whether the resolver is visible, whether the database connects over IPv6, and whether the cluster forms over inet6. The case worth dwelling on: ECTO_IPV6 stays "true" in fly.toml the whole time — remove only the consume line in runtime.exs and the database still fails with :nxdomain, while the resolver and the cluster, which take effect from being set, are unaffected.

The trio consume map · what each setting actually wires

RESOLVER ERL_INETRC visible BEAM reads it — set is enough DB OVER IPv6 ECTO_IPV6 connects runtime.exs reads it CLUSTER ERL_AFLAGS forms BEAM reads it — set is enough All three wired — the v6-only network is fully reachable.

An env var named in fly.toml configures the application only where the application reads it — ECTO_IPV6 via the runtime.exs consume line, the other two via the BEAM itself.

set in fly.toml

The deploy config names ECTO_IPV6 = "true", the same line that wired the resolver and the cluster. Three values in one [env] block.

consumed in runtime.exs

Only one of the three needs a reader: maybe_ipv6 threads ECTO_IPV6 into Portal.Repo’s socket_options. Without that line, the value sits unread.

A wrong address family and an unreachable host both crash with the identical :nxdomain. The next dive, F7.2.3 · .internal vs .flycast, gives the getent discriminator that tells the two apart, and back at F7.2.1 is the runtime-vs-compile split that decides which values runtime.exs reads at all.

References

The IPv6-only private network, the Postgrex socket family that resolves its host, the resolver config Erlang loads, and the clustering that rides the same v6 substrate.

Sources

Related in this course