R7.01.1 · dive 1

The row is the record.

When Redis is the authoritative store, the row is the record — there is no truth elsewhere to fall back on. A cache holds a copy; a system of record holds the value itself.

EchoMQ's job is the worked case: a hash at the job key carrying state, attempts, payload, written once on admission and mutated in place for the life of the job — no shadow copy in Postgres. Select a field; the readout reports what it holds and who writes it. The row at emq:{q}:job:<id> is the job's only canonical state.

The row is the record Per datum In codemojex The Oban trade References

The job row — three fields, one canonical home. Select a field of the hash at emq:{q}:job:<id>; the readout reports it.

§1

The row is the record

echo_mq/jobs.ex · the hash at the key

A cache holds a copy of a value that lives somewhere durable; a system of record holds the value itself. The distinction is not about the data structure — both can be a hash — but about where the truth is. A primary-database datum is one whose state has no second home.

EchoMQ's job is the worked case. A job's identity is a branded id under the JOB namespace; its row is a hash at the job key, and that hash is the job's only canonical state. There is no copy in Postgres that the hash mirrors. The moduledoc states the shape directly:

echo/apps/echo_mq/lib/echo_mq/jobs.ex · moduledocfrozen

Jobs are entities. A job's identity is a branded id under the JOB
namespace; its row is a hash at the job key; the pending set is a
same-score sorted set whose members are the ids themselves …
Enqueue is one idempotent script: kind policy, duplicate refusal,
row write, and pending insertion happen on the server in one atomic step.

The hash carries three fields, written once on admission and mutated in place for the life of the job: state (pending · active · scheduled · dead), attempts (the integer fence, incremented on each delivery so a redelivery is detectable), and payload (the body, opaque to the bus). The admission write is a single inline script, run server-side so it is one atomic step, not a read-then-write race:

-- @enqueue (EchoMQ.Script.new(:enqueue, …)) — the row write, verbatim
if string.sub(ARGV[1], 1, 3) ~= 'JOB' then
  return redis.error_reply('EMQKIND job id must be JOB-namespaced')  -- refuse a non-JOB id
end
if redis.call('EXISTS', KEYS[1]) == 1 then
  return 0                                  -- refuse a duplicate; the existing key wins
end
redis.call('HSET', KEYS[1], 'state', 'pending', 'attempts', '0', 'payload', ARGV[2])
redis.call('ZADD', KEYS[2], 0, ARGV[1])           -- the id joins the pending set
return 1

After admission, claim flips state to active, complete deletes the row, and retry increments attempts and re-parks the id — every one of those mutates the same hash at the same key. The job's truth never leaves Valkey, so there is nothing to keep in sync and nothing to fall back to.

The record's address is built by EchoMQ.Keyspace, which gates the branded id before any key is touched: job_key/2 calls EchoData.BrandedId.valid?/1 and raises on a malformed id, so the address is always emq:{q}:job:<JOB-branded-id> and a bad identity never reaches the store.

Notes on ValkeyHSET key field value … writes fields on a hash and HGETALL key reads them back whole; a Hash is encoded compactly as a listpack until it grows past the configured thresholds, so a small fixed-field row like the job costs little — valkey.io/commands/hset.

The job row is the record: one hash at emq:{q}:job:<id>, written by the @enqueue HSET and mutated in place by claim, complete and retry — no shadow copy, so its truth has no second home.

§2

Per datum, not per system

system-of-record.md · the judgment

"Redis as a primary database" is not an all-or-nothing choice for an application. It is a judgment made per datum. The same service can keep one kind of data in Valkey because it is hot and tolerates a small loss bound, and another in Postgres because it needs the non-negative invariant, the row lock, and an all-or-nothing transaction. The question for each datum is two-fold: how much loss is tolerable, and does it need ACID.

Below, five real codemojex data live on one side or the other. Hover or select a datum to read where its truth lives and why — the answer follows from loss-tolerance and the need for ACID, not from a single switch.

Where does this datum live? Select a datum; the readout reports its store and the reason. Operational, loss-tolerant state on the left (Valkey); money on the right (Postgres).

The boundary runs per datum: hot, loss-tolerant operational state lives in Valkey; money — which needs ACID and an invariant — lives in Postgres. "Redis as a primary database" is a judgment made one datum at a time.

§3

In codemojex — the split, in one function

codemojex/game.ex · Guesses.submit/3

codemojex is the live consumer of this stack, and the split runs straight through its play path. The game's competitive state lives in Valkey: Codemojex.Board writes the leaderboard sorted set cm:<game>:board (“The competitive state, in Valkey”); Codemojex.Locks writes a per-player guess hash for the round, cm:<round>:lock:<player> (“Position locking, in Valkey”). These are hot, operational, and tolerant of a roughly one-second loss bound.

Money lives in Postgres. Codemojex.Wallet holds the balances and mutates them inside database transactions; its moduledoc is the contrast in one sentence:

echo/apps/codemojex/lib/codemojex/wallet.ex · moduledocfrozen

The balances, on Postgres, mutated inside database transactions. A
mutation locks the player row (SELECT … FOR UPDATE), checks the
non-negative invariant, writes the new balance, and inserts the paired
ledger row — all or nothing.

Codemojex.Ledger is the append-only record of every balance change, “written by Codemojex.Wallet inside the same database transaction as the balance update — so … no balance ever moved without a paired record.” A balance is the wrong datum for a one-second loss bound: it needs the invariant, the lock, and the all-or-nothing transaction, so its truth has its home in Postgres.

The play path crosses the line in one call. Codemojex.Guesses.submit/3 charges the guess through the wallet (Postgres, transactional) and then enqueues the work as a branded JOB on the bus (Valkey). Its moduledoc names the principle:

# Codemojex.Guesses.submit/3 — two stores, chosen per datum (real, echo/apps/codemojex)
# "The game's mutable state is read from the system of record; the cache is
#  trusted only for the immutable secret on the scoring path."
case Wallet.charge_guess(player, r, game) do     # Postgres — locked row, invariant, paired ledger
  {:ok, _balance} ->
    job = EchoData.BrandedId.generate!("JOB")     # mint the JOB id (the record's identity)
    payload = :erlang.term_to_binary({:guess, game, player, guess})
    Lanes.enqueue(Bus.conn(), @queue, player, job, payload)  # Valkey — the job hash is the record
  {:error, reason} -> {:error, reason}
end

Same function, two stores, chosen per datum on loss-tolerance and the need for ACID. The charge is a Postgres transaction because money cannot tolerate a one-second loss; the enqueue writes a job hash in Valkey because the job is hot operational state whose truth has no second home. The decision is not made once for the application — it is made for each datum the function touches.

the pattern

Use Redis as the authoritative data store for a datum whose truth has no second home — the row is the record, kept in memory for the hot path.

its codemojex application

The emq:{q}:job:<JOB> hash is the job's only canonical state; codemojex keeps the job, the board and the lock in Valkey, and the wallet and ledger in Postgres — the split decided per datum in Guesses.submit/3.

The split is the pattern in real code: submit/3 charges money through Postgres and enqueues the job in Valkey, placing each datum by its loss-tolerance and its need for ACID.

§4

The Oban trade

system-of-record.md · the trade-off

Putting the job's truth in Valkey buys an in-memory hot path and a durability dial — and it costs the coupling Oban has. Oban keeps its jobs in the same Postgres as the application's data, so a job and the business row it concerns commit in one transaction: enqueue-the-job and update-the-row either both happen or neither does.

EchoMQ separates the bus (the job hash in Valkey) from the store, so it cannot make that single-transaction guarantee — the enqueue and a Postgres write are two operations, not one atomic step. What it gets in return is the in-memory hot path (every dequeue and ack touches the store, not a database) and the durability dial: how much loss the operator tolerates is a tuning choice, not a fixed cost paid on every write. State the trade plainly: Echo gives up Oban's transactional coupling and buys the hot path and the dial. It does not have Oban's coupling; that is the deliberate price.

Oban — jobs in the same Postgres

The job and the business row commit in one transaction: enqueue and update are atomic, all or nothing. The cost is that every job write is a database write.

EchoMQ — the job hash in Valkey

An in-memory hot path and a durability dial; the enqueue and a Postgres write are two operations, so the single-transaction coupling is given up. The trade is stated, not hidden.

Notes on ValkeyDurability is set by appendonly and appendfsync: everysec bounds worst-case loss to about one second, fsynced on a background thread. That bound is the dial's first rung, and it is what makes the job hash durable enough to be a record of truth — valkey.io/topics/persistence.

EchoMQ trades Oban's one-transaction coupling for an in-memory hot path and a durability dial. The win and its cost are stated together; Echo does not claim a coupling it gave up by design.

/bcs/persistence

The dial in depth

How much loss the job hash tolerates is a dial the operator turns: hold nothing, a bounded in-heap window with a checkpoint per K, or commit-per-record with an off-box replica. The dial — and the Oban trade taken further — is the subject of the Branded Component System course, the persistence floor.

/bcs/store

The cache that is not a record

The opposite of a system of record is a near-cache: a tier that holds a copy of a value whose truth lives elsewhere, so a miss or an eviction is recoverable. EchoStore — the declared near-cache, one fill per herd, coherence by mint time — is in the Branded Component System course, the store.

§5

References

sources · related

Sources

Valkey — HSET — set fields on a hash; the job row is written and mutated as one hash.
Valkey — HGETALL — read a hash's fields; how the record is read back whole.
Valkey — Persistence — AOF and RDB; the durability a primary store depends on.
Valkey — Cluster specification — the {q} hash tag forcing a queue's keys onto one of the 16384 slots.

Related in this course

R7.01 · Redis as a primary databasethe module hub: the store as the system of record
R7.01.2 · noevictionwhy a record of truth refuses the write under pressure
R7.01.3 · Persistence (RDB / AOF)the loss bound that makes the row durable
/bcs · The persistence floorthe durability dial and the Oban trade in depth
/bcs · The storethe near-cache that is not a system of record