The four sets

The keys are built by the keyspace, not the data

# echo_mq — EchoMQ.Keyspace
# queue_key builds emq:{q}:<type>. The braces are a cluster hashtag: every key
# of one queue hashes on the {q} substring, so pending / active / schedule / dead
# all land on ONE slot — which is what lets one Lua script touch them atomically.
def queue_key(queue, type) when is_binary(queue) and is_binary(type),
  do: IO.iodata_to_binary(["emq:{", queue, "}:", type])

# …
# job_key gates the id BEFORE it ever reaches a key. A malformed id raises here,
# never reaching the wire — the row key cannot be forged from bad data.
def job_key(queue, branded) when is_binary(branded) do
  if EchoData.BrandedId.valid?(branded) do
    # the row key: emq:{q}:job:<id>
    queue_key(queue, "job:") <> branded
  else
    raise ArgumentError, "job_key requires a valid branded id"
  end
end

A job's state is not a column to read — it is a set to be a member of. The keyspace builds the four addresses; the id is what lives at them.

Mint order is byte order, so browse needs no index

# echo_mq — EchoMQ.Jobs
# browse the pending set newest-first. The members ARE the ids, ordered by their
# own bytes (mint order), so ZRANGE ... REV BYLEX walks newest-to-oldest with no
# second index. + and - are the lexical max / min bounds.
def browse(conn, queue, n) when is_integer(n) and n > 0 do
  key = Keyspace.queue_key(queue, "pending")
  Connector.command(conn, ["ZRANGE", key, "+", "-", "BYLEX", "REV", "LIMIT", "0", Integer.to_string(n)])
end

# …
# depth is one count. ZCARD over the pending set — no scan, no index.
def pending_size(conn, queue) do
  Connector.command(conn, ["ZCARD", Keyspace.queue_key(queue, "pending")])
end

A second index is a thing that can drift. By making the id the member and the bytes the order, the queue has nothing to keep in sync — the one structure is both the membership and the order.

Inspect the sets

The browse takes the newest n from the high end of the lexical order (REV from +). Nothing was sorted at read time — the set was already in mint order because the members are the ids.

Pattern & implementation

The pattern says: let the location be the state. The implementation makes the member the id too — so the one structure carries the membership and the order.

Recap — the set is the state, the id is the member

Next in the lifecycle: Claim & the lease — the @claim2 transition, the active-set score that IS the lease deadline, and attempts as the fencing token.

References