F7.2 · deep dive 3 · .internal vs .flycast & the discriminator
The two private names & the discriminator
Fly exposes two private-name schemes for reaching one app from another, and choosing the
wrong one — or the right one for a stopped instance — produces the same Postgrex
:nxdomain as the address-family fault from
dive 2.
.internal is direct 6PN: org-scoped, AAAA-only, and it returns a record
only for an app with a live machine in the requester’s 6PN. .flycast reaches an app
through Fly’s internal proxy and additionally needs a flycast IP. Two schemes, two failure modes,
one error string. This dive gives the test that tells them apart:
getent hosts <db>.internal run from another running machine in the org.
We read the first bring-up’s record as built: the prod.exs
fly_dns topology polls echo-portal.internal, the DB was
reached at portal-postgres.internal, and the cross-machine
getent test settled which of two identical-looking crashes was the real one.
Two private names, one org
An app on Fly reaches another app over the private IPv6 network (6PN) by a private name, and there are
two name schemes. .internal is direct 6PN. A name like
portal-postgres.internal is org-scoped, carries only AAAA
(IPv6) records, and resolves only for an app that has a live machine in the requester’s
6PN. It requires the two apps to sit in the same organisation. This is the convention the deploy uses:
the prod.exs fly_dns topology polls
echo-portal.internal to form the cluster, and the first bring-up reached the
database at portal-postgres.internal. When that database is auto-stopped, its
.internal name returns no record, and the boot crashes with
:nxdomain — this is gotcha 2.
.flycast is the Fly-proxy private name. It reaches an app through Fly’s
internal proxy rather than directly, and it additionally needs a flycast IP — for a Fly Managed
Postgres, that IP is allocated by fly mpg attach. The first bring-up’s
portal-postgres is a same-org postgres-flex HA cluster
reached over .internal, not a Managed Postgres over .flycast;
F7.3 names that discriminator so the right host scheme is chosen for the right kind of database. Pick the
database kind to read which scheme its host carries.
Host-scheme chooser · which private name reaches this database?
Both private names carry only AAAA records, so both demand an inet6 socket; the scheme they differ on is direct 6PN versus the Fly proxy — and the database kind, not a guess, picks which.
One symptom, two faults, the discriminator
The trap is that two different faults raise the same error. Gotcha 2 —
the name has no record because the instance is stopped or out of org — crashes Postgrex with
:nxdomain. Gotcha 3 — the name has an AAAA record
but the app A-queries it on the default :inet family (dive 2’s fault) —
crashes with the identical :nxdomain. The error string does not say which. The
test that isolates them is a single command run from another running machine in the org:
getent hosts <db>.internal. That command uses glibc
getaddrinfo, the same resolution path Erlang’s native
lookup takes. If the other machine resolves the name to an AAAA, the name and DNS are sound — so the
fault is the app’s address family (gotcha 3, fixed in runtime.exs), not the
network (gotcha 2).
The first bring-up ran exactly this test:
getent hosts portal-postgres.internal SUCCEEDED from inside the running,
same-org portal-postgres machine, returning an AAAA. The name was sound, so the
app’s IPv4 default was the fault — which is how gotcha 3 was identified as the decisive one. A
complementary check guards the other direction: if even echo-portal.internal
— the app’s OWN name — fails to resolve from inside a machine, the machine’s 6PN DNS
itself is broken, not the target. Set the database host’s machine state and the app’s Postgrex
address family below and read the diagnosis the discriminator returns for that cell.
:nxdomain discriminator · which fault is it, and what does getent say?
The error string is the same in every failing cell; the discriminator is what differs —
a successful getent from another machine clears the network and convicts the app’s
address family.
The IPv6 literal, named as a diagnostic
One more unblock is worth naming for completeness, and then setting aside. A bracketed IPv6 literal in
DATABASE_URL — a fixed address rather than a name — sidesteps
resolution entirely: with no DNS lookup, neither gotcha 2 nor gotcha 3 can fire. It confirms the network
path works. But it pins the URL to a machine IP that Fly can recreate, so it is brittle as a deploy value.
The durable fix is the one dive 2 wired: socket_options: [:inet6] on
Portal.Repo, so the .internal AAAA resolves over IPv6
through a name that survives a machine being recreated. The literal is a diagnostic, not the deploy path.
# DATABASE_URL — a bracketed IPv6 literal sidesteps DNS (diagnostic only) # fixed address, no resolution — pins the URL to a recreatable machine IP DATABASE_URL="ecto://portal:****@[fdaa:0:1:a7b:1c2:3:4:2]:5432/portal" # the durable path — resolve the AAAA-only .internal name over IPv6 # runtime.exs consumes ECTO_IPV6 → socket_options on Portal.Repo DATABASE_URL="ecto://portal:****@portal-postgres.internal:5432/portal" # + socket_options: [:inet6]
a name that has no record
gotcha 2 — the instance is stopped or out of org, so .internal returns nothing and the
boot :nxdomains.
a name with a record the app cannot read
gotcha 3 — the AAAA exists, but a default :inet A-query returns the same
:nxdomain; the discriminator separates the two.
The config now reads its secrets, binds the proxy-reachable port, speaks IPv6 to an AAAA-only host, and carries the test that tells two identical crashes apart. What it has not yet done is reach a database with a schema in it: F7.3, where the database goes live, brings up the migration and gotcha 4 over at F7 · Deploy to Fly.
References
The two private-name schemes, the same-org HA Postgres reached over the direct one, and the
glibc resolver path the cross-machine discriminator and Erlang’s native lookup share.
Sources
- Fly.io — Private networking —
.internaldirect 6PN,.flycastproxy names, and the AAAA-only records both carry. - Fly.io — Postgres — the same-org
postgres-flexcluster reached over.internal. - getaddrinfo(3) — the glibc resolver path that
getent hostsand Erlang’snativelookup both take.
Related in this course
- F7.2 · the module hub — runtime config and the IPv6 private network, the module this dive closes.
- F7.2.1 · the runtime-vs-compile split — what
runtime.exsreads at boot versus whatprod.exsbakes. - F7.2.2 · Fly’s IPv6-only private network — gotcha 3, the address-family fault this dive disambiguates.
- F7 · Deploy to Fly — the chapter; F7.3 brings the database live.