jonnifyechomq

The command grammar · dive 03

The closed error registry

Eleven wire-4 texts byte-exact, twenty-two wire-6 additions, four host refusals: every refusal an EMQ handler sends but the server's own is in one file, and a test keeps wire 4 in step with Elixir.

The first word of a text is its class. The rest is fixed, or built by one function from the handler's own verb and, in two places, from bytes the handler did not choose. A unit test holds the wire-4 texts to the Elixir scripts byte for byte. Pick a class on the right and read its texts, where they come from and which native commands send them.

The registry by class · pick a class

EMQKIND 5 EMQLOCK · EMQRATE 2 EMQSTALE 6 EMQSTATE 18 EMQWIRE 2 + 1 ERR <VERB> (shape) 12 host 4 wire 4 wire 6 builder host EMQSTALE: 5 wire 4 · 1 wire 6
EMQSTALE — 5 wire 4, 1 wire 6
EMQSTALE complete token mismatch STALE_COMPLETE · wire 4 · sent by: EMQ.COMPLETE
EMQSTALE delay token mismatch STALE_DELAY · wire 4 · sent by: EMQ.DELAY
EMQSTALE extend token mismatch STALE_EXTEND · wire 4 · sent by: EMQ.EXTEND
EMQSTALE job gone STALE_GONE · wire 4 · sent by: EMQ.DELAY, EMQ.FAIL, EMQ.REQUEUE
EMQSTALE retry token mismatch STALE_RETRY · wire 4 · sent by: EMQ.FAIL
EMQSTALE requeue token mismatch STALE_REQUEUE · wire 6 · sent by: EMQ.REQUEUE
the whole registry: 11 wire-4 texts, 22 wire-6 texts, 13 builders plus the server passthrough, 4 host refusals

One file

errors.rs opens with what it promises: every refusal an EMQ handler sends is a constant or a builder in this file, so the reply bytes are reviewable in one place.

// trafikey — emq/errors.rs, the registry's own statement
//! The closed EMQ error registry. Every refusal an EMQ handler sends is a constant or a builder
//! in this file, so the reply bytes are reviewable in one place.
//!
//! - Wire 4: the eleven distinct `redis.error_reply('EMQ…')` texts in `echo/apps/echo_mq/lib`,
//!   byte-exact. The unit test `wire4_registry_is_closed` re-reads that source tree and fails
//!   when a text is added, removed or changed there.
// … the wire-6 line names the grammar revisions that froze the additions
//! - Host refusals are `ERR`-class and are not EMQ registry entries.
//!
//! Every text is ASCII and NUL-free by construction; dynamic parts pass through [`printable`].

Three families follow: wire 4, shared with the Elixir scripts; wire 6, the native verbs' additions; and the host refusals, which are not registry entries. The first word of each text names its class, the way a Valkey error leads with ERR or WRONGTYPE: EMQKIND, EMQLOCK, EMQRATE, EMQSTALE, EMQSTATE, EMQWIRE, and ERR for the shape texts.

A client branches on the first word; the bytes after it are fixed and reviewable in one file.

Wire 4, byte for byte

The wire-4 texts are the eleven distinct redis.error_reply('EMQ…') literals of the Elixir scripts in echo_mq, copied byte for byte:

// trafikey — errors::WIRE4, eleven texts shared byte for byte with the Elixir scripts
// Wire 4, byte-exact (jobs.ex, lanes.ex, admin.ex, flows.ex, …).
pub const KIND_JOB: &str = "EMQKIND job id must be JOB-namespaced";
pub const LOCK: &str = "EMQLOCK job is locked";
pub const RATE: &str = "EMQRATE at concurrency ceiling";
pub const STALE_COMPLETE: &str = "EMQSTALE complete token mismatch";
pub const STALE_DELAY: &str = "EMQSTALE delay token mismatch";
pub const STALE_EXTEND: &str = "EMQSTALE extend token mismatch";
pub const STALE_GONE: &str = "EMQSTALE job gone";
pub const STALE_RETRY: &str = "EMQSTALE retry token mismatch";
pub const STATE_ACTIVE_PRESENT: &str = "EMQSTATE active jobs present";
pub const STATE_NOT_DEAD: &str = "EMQSTATE not dead";
pub const STATE_NOT_PAUSED: &str = "EMQSTATE not paused";

/// The wire-4 registry, closed: compared with the echo_mq source by the unit tests.
pub const WIRE4: [&str; 11] = [
    KIND_JOB,
    LOCK,
    RATE,
    STALE_COMPLETE,
    STALE_DELAY,
    STALE_EXTEND,
    STALE_GONE,
    STALE_RETRY,
    STATE_ACTIVE_PRESENT,
    STATE_NOT_DEAD,
    STATE_NOT_PAUSED,
];

The same situation gets the same bytes from a script and from a native command: a stale @complete and a stale EMQ.COMPLETE both answer EMQSTALE complete token mismatch. Seven of the eleven have a native sender: EMQKIND job id must be JOB-namespaced from EMQ.ENQUEUE, EMQ.SCHEDULE and EMQ.DELAY; the five wire-4 EMQSTALE texts from EMQ.COMPLETE, EMQ.DELAY, EMQ.EXTEND, EMQ.FAIL and EMQ.REQUEUE; EMQSTATE not dead from EMQ.REPROCESS. Four come from no native command: EMQLOCK job is locked (EchoMQ.Jobs @remove_job2, which remove_job/4 sends), EMQRATE at concurrency ceiling (EchoMQ.Metrics @is_maxed), and EMQSTATE active jobs present and EMQSTATE not paused (EchoMQ.Admin @obliterate). They stay in the registry so that it equals the Elixir source. One of them, in two beats, the handle and then the body:

# echo_mq — EchoMQ.Metrics, the @is_maxed handle and the call that maps its refusal
  @is_maxed Script.new(:is_maxed, """
            # … the Lua body, shown next
            """)
  # …
  def is_maxed(conn, queue) do
    keys = [Keyspace.queue_key(queue, "meta"), Keyspace.queue_key(queue, "active")]

    case Connector.eval(conn, @is_maxed, keys, []) do
      {:ok, 0} -> :ok
      {:error, {:server, "EMQRATE" <> _}} -> {:error, :rate}
      other -> other
    end
  end
-- EchoMQ.Metrics @is_maxed, the body: a read and a typed refusal
local cap = tonumber(redis.call('HGET', KEYS[1], 'concurrency') or '0')
if cap > 0 and redis.call('ZCARD', KEYS[2]) >= cap then
  return redis.error_reply('EMQRATE at concurrency ceiling')
end
return 0

The script reads the ceiling and refuses with a typed text; the Elixir caller maps the class, EMQRATE, to {:error, :rate}. No native command sends that text, and the registry still holds it.

Byte-exact means a match on a refusal reads the same from either executor.

The closure test

The registry is closed by a test, not by review. wire4_registry_is_closed walks echo/apps/echo_mq/lib, collects every error_reply('EMQ…') literal from every .ex file, and requires the set to equal WIRE4:

// trafikey — the closure test: the registry against every .ex file under echo_mq/lib
    /// Every `error_reply('EMQ…')` literal under echo/apps/echo_mq/lib, deduplicated.
    fn wire4_source() -> BTreeSet<String> {
        let dir = concat!(
            env!("CARGO_MANIFEST_DIR"),
            "/../../../echo/apps/echo_mq/lib"
        );
        // … a walk of the tree collects every .ex file and counts it in `files`
                let body = std::fs::read_to_string(&p).unwrap_or_default();
                for part in body.split("error_reply('").skip(1) {
                    if let Some((text, _)) = part.split_once('\'') {
                        if part.starts_with("EMQ") {
                            found.insert(text.to_string());
                        }
                    }
                }
            }
        }
        assert!(
            files > 10,
            "could not measure: only {files} .ex files under {dir}"
        );
        found
    }

    #[test]
    fn wire4_registry_is_closed() {
        let ours: BTreeSet<String> = WIRE4.iter().map(|s| s.to_string()).collect();
        assert_eq!(ours.len(), WIRE4.len(), "duplicate registry entry");
        assert_eq!(ours, wire4_source());
    }

Add a text to a script, change one byte of one, or remove the last use of one, and the test fails until errors.rs changes with it. The walk refuses to pass on too little: ten or fewer .ex files is “could not measure”, not a pass. A second test, wire6_texts_are_exact, pins every builder's output and the whole WIRE6 array as literal strings, and checks that every fixed text starts with EMQ, is ASCII and holds no NUL.

The wire-4 texts cannot drift from the Elixir source without a failing test.

Twenty-two additions

Wire 6 adds twenty-two fixed texts: the fence's two EMQWIRE refusals, one new EMQSTALE text, fifteen EMQSTATE texts (fourteen for the lanes, flows and states the native verbs do not serve, one for a stream append out of order) and four EMQKIND texts for the bus verbs' records and brands.

// trafikey — errors::WIRE6, the fixed texts the native verbs add
// the fence's refusal when the brand is absent
pub const WIRE_ABSENT: &str = "EMQWIRE expected echomq:6.0.0 but {emq}:version is absent";
// the fence's refusal on a cluster-enabled server, before any key is read
pub const WIRE_CLUSTER: &str = "EMQWIRE expected a standalone server but cluster mode is enabled";
// the completion family
pub const COMPLETE_GROUP: &str = "EMQSTATE complete lane not native";
pub const COMPLETE_FLOW: &str = "EMQSTATE complete flow not native";
pub const COMPLETE_NOT_ACTIVE: &str = "EMQSTATE complete not active";
// the scheduling verbs
pub const DELAY_GROUP: &str = "EMQSTATE delay lane not native";
pub const DELAY_NOT_ACTIVE: &str = "EMQSTATE delay not active";
pub const PROMOTE_GROUP: &str = "EMQSTATE promote lane not native";
pub const RESCHEDULE_NOT_SCHEDULED: &str = "EMQSTATE reschedule not scheduled";
// the retry verbs
pub const FAIL_GROUP: &str = "EMQSTATE fail lane not native";
pub const FAIL_FLOW: &str = "EMQSTATE fail flow not native";
pub const FAIL_NOT_ACTIVE: &str = "EMQSTATE fail not active";
pub const REPROCESS_GROUP: &str = "EMQSTATE reprocess lane not native";
// the recovery verbs; STALE_REQUEUE is a new text in the existing EMQSTALE class
pub const STALE_REQUEUE: &str = "EMQSTALE requeue token mismatch";
pub const REQUEUE_GROUP: &str = "EMQSTATE requeue lane not native";
pub const REQUEUE_NOT_ACTIVE: &str = "EMQSTATE requeue not active";
pub const STALLED_GROUP: &str = "EMQSTATE stalled lane not native";
// the stream append and the durable publish, checked after the fence
pub const KIND_RECORD: &str = "EMQKIND record id must be EVT-namespaced";
pub const APPEND_NOT_AFTER_TOP: &str = "EMQSTATE append id not after the stream top";
// the failure count: the two brands of its key
pub const KIND_SUB_BRAND: &str = "EMQKIND fail key requires a valid SUB brand";
pub const KIND_MSG_BRAND: &str = "EMQKIND fail key requires a valid MSG brand";
// the durable publish: the envelope's msg
pub const KIND_MSG: &str = "EMQKIND msg must be MSG-namespaced";

/// The fixed wire-6 additions; the builders below add the per-verb shape texts and `EMQWIRE … found`.
pub const WIRE6: [&str; 22] = [
    WIRE_ABSENT,
    // … the fifteen texts between them, in the order above
    WIRE_CLUSTER,
    KIND_RECORD,
    APPEND_NOT_AFTER_TOP,
    KIND_SUB_BRAND,
    KIND_MSG_BRAND,
    KIND_MSG,
];

The … lane not native and … flow not native texts mark the edge of the native verbs: each is sent before any write and names the verb. EMQ.COMPLETE, EMQ.DELAY, EMQ.FAIL, EMQ.REPROCESS and EMQ.REQUEUE refuse a grouped row; EMQ.PROMOTE and EMQ.STALLED.CHECK refuse the whole call for one grouped member; only EMQ.COMPLETE and EMQ.FAIL refuse a parent field or a flow layout; and EMQ.CLAIM, EMQ.EXTEND, EMQ.EXTENDS and EMQ.RESCHEDULE read no group. The … not active texts refuse a job that holds the right token but is not in active, and EMQSTATE reschedule not scheduled a job whose row exists outside the schedule. STALE_REQUEUE joins the existing EMQSTALE class, so a client that already branches on EMQSTALE handles it unchanged. The two EMQWIRE texts are the fence's: no brand in {emq}:version, and a cluster-enabled server, refused before any key is read; a wrong brand is the builder below.

The five bus texts are domain checks, sent after the fence and before any write. EMQKIND record id must be EVT-namespaced refuses an entry of EMQ.APPEND or EMQ.PUBLISH whose id is not an EVT brand, and EMQSTATE append id not after the stream top an entry id at or below the stream's top, or ids that do not increase. EMQ.FAILURES.INCR refuses a key whose <sub> or <msg> is not a brand, with EMQKIND fail key requires a valid SUB brand or … valid MSG brand, and EMQ.PUBLISH an envelope whose msg is not MSG-namespaced, with EMQKIND msg must be MSG-namespaced.

The new texts reuse the old classes, so a branch on the first word keeps working.

Builders and printable

The shape texts carry the verb, and the brand refusal carries what the dataset holds. Each is built by one function:

// trafikey — the builders: the per-verb shape texts, the found brand, the server passthrough
/// G0: `ERR <VERB> numkeys must be <n>`.
pub fn numkeys(verb: &str, n: usize) -> ValkeyError {
    ValkeyError::String(format!("ERR {verb} numkeys must be {n}"))
}
/// G0: `ERR <VERB> wrong number of arguments` (variable-arity verbs; fixed arities get Valkey's
/// own arity error). Never `ValkeyError::WrongArity` (it replies `Wrong Arity`).
pub fn arg_count(verb: &str) -> ValkeyError {
    ValkeyError::String(format!("ERR {verb} wrong number of arguments"))
}
// … layout, lease, mode, due, nonneg, positive, entries, empty_field, one_entry and not_envelope follow the same form
/// G0: `EMQWIRE expected echomq:6.0.0 found <brand>`; the brand comes from the dataset.
pub fn wire_found(brand: &[u8]) -> ValkeyError {
    ValkeyError::String(format!(
        "EMQWIRE expected echomq:6.0.0 found {}",
        printable(brand)
    ))
}
/// A server error from a host call (for example `WRONGTYPE …`), passed through without the leading `-`.
pub fn server(text: &[u8]) -> ValkeyError {
    let text = text.strip_prefix(b"-").unwrap_or(text);
    if text.is_empty() {
        return emq(HOST_CALL);
    }
    ValkeyError::String(printable(text))
}

/// Bytes `0x20`–`0x7E` except `\` stay literal; every other byte becomes `\xHH`. The reply path
/// maps characters to bytes one by one, so non-ASCII must never reach it.
pub fn printable(bytes: &[u8]) -> String {
    let mut out = String::with_capacity(bytes.len());
    for &b in bytes {
        if (0x20..=0x7E).contains(&b) && b != b'\\' {
            out.push(b as char);
        } else {
            out.push_str(&format!("\\x{b:02x}"));
        }
    }
    out
}

The verb builders format a &'static str the handler itself passes, so their bytes are as fixed as a constant's. Two builders carry bytes the handler did not choose. wire_found puts the brand it found in {emq}:version into the reply, and server passes a server's own error through without its leading -. Both go through printable: bytes from 0x20 to 0x7E other than \ stay literal and every other byte becomes \xHH, so a reply holds only printable ASCII whatever the dataset holds. An empty server error becomes ERR EMQ host call failed. arg_count's doc names a rule the module root states: an EMQ handler never returns ValkeyError::WrongArity, which the SDK sends as Valkey's generic arity error (RedisModule_WrongArity, naming the command) instead of the verb's own ERR <VERB> wrong number of arguments; Wrong Arity, which the doc names, is only the type's display string. Build each refusal the test pins:

Build a refusal · pick a test vector

the brand: bytes from the dataset a61 \x0000 b62 \x5c5c \xffff the reply EMQWIRE expected echomq:6.0.0 found a\x00b\x5c\xff the handler's verb kept literal escaped as \xHH stripped
wire_found(b"a\0b\\\xff") → EMQWIRE expected echomq:6.0.0 found a\x00b\x5c\xff
the dynamic part comes from: the dataset ({emq}:version), through printable
asserted byte for byte by: wire6_texts_are_exact

Fixed text, the handler's own verb, and the dataset's bytes only through printable.

Host refusals

Four refusals come from the host side, not from a verb's rules:

// trafikey — the host refusals, ERR class except Valkey's own NOPERM
// Host refusals (ERR class).
/// `AvoidReplicaTraffic` is set: refused before the first write, so nothing is half-applied.
pub const WRITE_PAUSED: &str = "ERR EMQ write-paused";
// Valkey's own reply text for a denied key, so a native caller and a wire-4 Lua caller get the
// same class and the same bytes for the same key
pub const NOPERM_KEY: &str = "NOPERM No permissions to access a key";
/// RM_Call returned no reply object, or a host API pointer is missing.
pub const HOST_CALL: &str = "ERR EMQ host call failed";
/// A reply type an EMQ handler never expects under RESP2 (R13: typed error, never a panic).
pub const HOST_REPLY: &str = "ERR EMQ unsupported host reply";

ERR EMQ write-paused is the guard's. ERR EMQ host call failed and ERR EMQ unsupported host reply are the adapter's, for a call that produced no reply and for a reply type no handler expects, and the second also answers a write whose reply differs from the prediction, as the claim's checks after the guard show. NOPERM No permissions to access a key is Valkey's own text for a denied key, so a native caller and a wire-4 Lua caller get the same class and the same bytes for the same key. Beside them stand the server's own texts: WRONGTYPE …, passed through, and the two HINCRBY texts, ERR hash value is not an integer and ERR increment or decrement would overflow, which live beside the claim in jobs.rs so that its prediction refuses, before the guard, with the bytes HINCRBY itself would reply. INCR's ERR value is not an integer or out of range lives beside EMQ.FAILURES.INCR in bus.rs for the same reason; its overflow text is the HINCRBY one.

An ERR EMQ reply names a host condition; an EMQ… reply names a rule of the protocol.

TRA's own grammar

The TRA.* commands share the module and not the grammar. They take no numkeys, read no {emq}:version and run no fence. Valkey enforces their arity, and TRA.SETBIT's else arm, which the host never lets run, answers with ValkeyError::WrongArity, the reply an EMQ handler never sends. Their refusals are three ERR TRA constants of their own, outside errors.rs:

// trafikey — tra/bitmap.rs, the TRA family's own texts and its arity arm
const INVALID_OFFSET: ValkeyError = ValkeyError::Str("ERR TRA invalid-offset");
const INVALID_ARGUMENT: ValkeyError = ValkeyError::Str("ERR TRA invalid-argument");
const LIMIT: ValkeyError = ValkeyError::Str("ERR TRA limit");
// …
pub(crate) fn setbit(ctx: &Context, args: Vec<ValkeyString>) -> ValkeyResult {
    // The host enforces arity 4 before the handler runs; the else arm is its arity reply.
    let [_, name, offset, bit] = args.as_slice() else {
        return Err(ValkeyError::WrongArity);
    };
    let offset = parse_offset(offset.as_slice()).ok_or(INVALID_OFFSET)?;
    let bit = parse_bit(bit.as_slice()).ok_or(INVALID_ARGUMENT)?;
    // … the bit is read, the string grown when needed (or ERR TRA limit), the bit written
    ctx.notify_keyspace_event(NotifyEvent::STRING, "setbit", name);
    ctx.replicate_verbatim();
    Ok(ValkeyValue::Integer(i64::from(previous)))
}

An offset is ASCII digits, leading zeroes allowed (the spelling numkeys refuses), up to u32::MAX; a bit is exactly 0 or 1; another key type gets WRONGTYPE. And TRA.SETBIT replicates its command as it came, where every EMQ write replicates as its effect.

the pattern (Redis Patterns Applied)

A protocol a bus declares is closed: a key grammar, a job row, and typed refusals that lead with their class. The immutable core teaches it.

the implementation (trafikey)

errors.rs closes the refusals: every text but the server's own in one file, the class first, wire 4 byte-exact with the scripts and held there by a test, and the dataset's bytes only through printable.

Two families, two grammars: the EMQ order and its registry bind the EMQ.* commands only.

References

Sources

Related in this course