jonnifyechomq

The command grammar · dive 01

Shape before the fence

Count it, spell it, rebuild every key from one name: a call malformed in shape, layout or value reads nothing.

Before an EMQ.* handler reads a single key, it has checked the call three ways. Shape: numkeys is the exact decimal spelling of the verb's count, the keys are present, the ARGV count is right. Layout: every declared key, and the base when one is passed, is rebuilt from one non-empty queue name and compared byte for byte; a bus verb's one key is checked against its form. Values: lease_ms, counts, a mode and a due time are parsed, and a bus verb's entries, field names and TTL. Only then does the fence read {emq}:version, the first read of the call. Pick a token on the right and read how three parsers in the source treat it.

Three parsers, one token · pick a token

numkeys_is(_, 3) refused parse_lease 7 parse_count 7
the token "007"
as numkeys of EMQ.CLAIM: ERR EMQ.CLAIM numkeys must be 3
as lease_ms of EMQ.CLAIM: 7
as delay_ms of EMQ.FAIL: 7
host reads made: none

numkeys, exactly

Two checks run before the handler does. Valkey compares the call with the registered arity: a fixed arity such as EMQ.CLAIM's 6 gets Valkey's own arity error, and a negative arity such as EMQ.FAIL's −12 is a minimum. Then, because every write verb registers EVAL's key-spec, Valkey reads numkeys to find the declared keys and checks them against the caller's ACL. The handler's first check is that the numkeys token is the verb's count, spelled the one way Valkey's key finder reads it.

// trafikey — host::shape, the first check in every EMQ handler
/// `args[1]` is exactly the decimal `n` (`02`, `+2` and ` 2` are not: Valkey's own numkeys
/// parser would see zero keys for them, so the handler must refuse them too).
pub fn numkeys_is<S: Bytes>(args: &[S], n: usize) -> bool {
    args.get(1)
        .is_some_and(|a| a.bytes() == n.to_string().as_bytes())
}
// …
/// G0.3 (a), first two checks: numkeys is the verb's count, and the keys are all present.
pub fn shape<'a, S: Bytes>(
    verb: &'static str,
    args: &'a [S],
    numkeys: usize,
) -> Result<Shape<'a, S>, ValkeyError> {
    // a byte compare against the decimal spelling of the verb's own count
    if !numkeys_is(args, numkeys) {
        return Err(errors::numkeys(verb, numkeys));
    }
    // the keys are args[2..2+n] and the values the rest; a short call has no key slice at all
    match (args.get(2..2 + numkeys), args.get(2 + numkeys..)) {
        (Some(keys), Some(argv)) => Ok(Shape { verb, keys, argv }),
        _ => Err(errors::arg_count(verb)),
    }
}

The compare is on bytes: 3 passes, 03, +3 and ␣3 do not. The source gives the reason: for those spellings Valkey's own numkeys parser finds zero keys, so the server checked no key against the caller's ACL, and the handler refuses the call with ERR EMQ.CLAIM numkeys must be 3. The match never indexes past the end: a call too short to hold its keys gets ERR <VERB> wrong number of arguments instead of a panic. A fixed arity such as the claim's never reaches that text, because Valkey's own arity error comes first; a variable-arity verb such as EMQ.FAIL gets it from expect_argc when its ARGV count is off. None of these replies needed a read.

The keys a handler works on are the keys the server found: numkeys has one spelling, and any other is refused before a key is touched.

A call read through counts

A handler never indexes args[n]. It reads the call through Shape: key(i) and arg(i) are 1-indexed like the Lua body they port and return an empty slice out of range, so a wrong count is a refusal and never a panic. expect_argc requires the exact ARGV count, and lease parses a lease or refuses with the verb's own text.

// trafikey — the Shape a handler reads its call through: counted, 1-indexed, never a panic
impl<'a, S: Bytes> Shape<'a, S> {
    // …
    /// Exactly `n` ARGV, or `ERR <VERB> wrong number of arguments`.
    pub fn expect_argc(&self, n: usize) -> Result<(), ValkeyError> {
        if self.argv.len() == n {
            Ok(())
        } else {
            Err(errors::arg_count(self.verb))
        }
    }
    /// `KEYS[i]`; empty when out of range (check the counts first).
    pub fn key(&self, i: usize) -> &'a [u8] {
        i.checked_sub(1)
            .and_then(|j| self.keys.get(j))
            .map_or(&[][..], |s| s.bytes())
    }
    // …
    /// `ARGV[i]` as lease_ms (G0.7), or `ERR <VERB> lease_ms must be a positive integer`.
    pub fn lease(&self, i: usize) -> Result<i64, ValkeyError> {
        parse_lease(self.arg(i)).ok_or_else(|| errors::lease(self.verb))
    }
}

/// G0.7: ASCII digits only (no sign, space, point or exponent), at least 1, within i64.
pub fn parse_lease(raw: &[u8]) -> Option<i64> {
    if raw.is_empty() {
        return None;
    }
    let mut value: i64 = 0;
    for &c in raw {
        if !c.is_ascii_digit() {
            return None;
        }
        // an overflow past i64 ends the parse with None, so it refuses like any other bad value
        value = value.checked_mul(10)?.checked_add(i64::from(c - b'0'))?;
    }
    (value >= 1).then_some(value)
}

The lease rule is not the numkeys rule. 007 is a valid lease of 7 and an invalid numkeys: numkeys must equal what Valkey's key finder reads, while a value only has to be a number in range. A sign, a space, a point, an exponent, an empty token and anything past i64 are refused, the overflow through checked_mul and checked_add. The scheduling, retry and recovery verbs, and the bus verbs, count the same way, with 0 allowed where a count may be zero:

// trafikey — plan::parse_count and parse_positive, shared by the verbs that take counts
/// ASCII digits only (no sign, space, point or exponent), at least one, within i64; `0` allowed.
pub(super) fn parse_count(raw: &[u8]) -> Option<i64> {
    if raw.is_empty() {
        return None;
    }
    let mut value: i64 = 0;
    for &c in raw {
        if !c.is_ascii_digit() {
            return None;
        }
        value = value.checked_mul(10)?.checked_add(i64::from(c - b'0'))?;
    }
    Some(value)
}

/// A count argument that must be ≥ 1 (`batch`; later `max_attempts`, `max_stalled`, `limit`).
pub(super) fn parse_positive(raw: &[u8]) -> Option<i64> {
    parse_count(raw).filter(|&n| n >= 1)
}

Each parser has its own refusal builder in the registry: ERR <VERB> lease_ms must be a positive integer, ERR <VERB> <name> must be a non-negative integer, ERR <VERB> <name> must be a positive integer, and for the scheduling verbs ERR <VERB> mode must be in or at and ERR <VERB> due must be a non-negative integer. The bus verbs' entries add four more: ERR <VERB> entries do not match n when the arguments do not split into exactly n entries, ERR <VERB> field names must be non-empty, and for the durable publish ERR <VERB> n must be 1 and ERR <VERB> entry is not an envelope.

One accessor, counted and 1-indexed; one parser per kind of value; one fixed text per refusal.

One name, every key

The layout check recovers one queue name from a declared key and requires every other key of the call to be that queue's key, byte for byte. split_queue_key accepts exactly emq:{<q>}:<type> with a name that holds no brace and no NUL; Queue::from_key also refuses the empty name.

// trafikey — keyspace: one queue name recovered from a declared key, every other key compared to it
/// Splits `emq:{<q>}:<type>` with a valid `<q>`; `None` for any other shape.
pub fn split_queue_key(key: &[u8]) -> Option<(&[u8], &[u8])> {
    let rest = key.strip_prefix(b"emq:{")?;
    // the name ends at the first closing brace, and a colon must follow it
    let close = rest.iter().position(|&b| b == b'}')?;
    let (queue, tail) = rest.split_at(close);
    let ty = tail.strip_prefix(b"}:")?;
    // a name holding a brace or a NUL is no name
    valid_queue(queue).then_some((queue, ty))
}
// …
impl<'a> Queue<'a> {
    /// The queue of `key` when `key` is exactly `emq:{<q>}:<ty>`.
    pub fn from_key(key: &'a [u8], ty: &[u8]) -> Option<Self> {
        match split_queue_key(key) {
            // the empty name is refused here: emq:{}: has no hashtag and spreads one queue over slots
            Some((name, t)) if t == ty && !name.is_empty() => Some(Self { name }),
            _ => None,
        }
    }
    /// The queue of a base `emq:{<q>}:` (the `p` argument of claim, complete and extends).
    pub fn from_base(base: &'a [u8]) -> Option<Self> {
        Self::from_key(base, b"")
    }
    // …
    /// Whether `key` is exactly this queue's `<ty>` key (a `Queue` name is never empty).
    pub fn is(&self, key: &[u8], ty: &[u8]) -> bool {
        split_queue_key(key) == Some((self.name, ty))
    }
    // …
}
// trafikey — the claim's layout check: the base names the queue, the two keys must be its own
/// G3 layout: `KEYS[3]` is the base `emq:{q}:`, `KEYS[1]` its `pending`, `KEYS[2]` its `active`.
fn claim_queue<'a, S: Bytes>(s: &Shape<'a, S>) -> Option<Queue<'a>> {
    let q = Queue::from_base(s.key(3))?;
    (q.is(s.key(1), b"pending") && q.is(s.key(2), b"active")).then_some(q)
}

For the claim the base KEYS[3] names the queue, and the pending and active keys must be that queue's own; the && stops at the first mismatch. One non-empty name means one hashtag, so every key of the call sits on one slot. A bus verb declares one key, so there is no second key to compare: keyspace::split_prefixed checks its form instead, emq:{<q>}:stream:<name> for EMQ.APPEND, the same with the name msgs for EMQ.PUBLISH, and emq:{<topic>}:fail:<sub>:<msg> for EMQ.FAILURES.INCR, each with a non-empty name in the braces and after the type, and any other form gets the same layout refusal. The scripts have no such check because their host builds the keys through EchoMQ.Keyspace; a native command takes its keys from any client, so the handler rebuilds them. Try the shapes a client could send:

EMQ.CLAIM's layout check · pick the keys

KEYS[3] base emq:{orders}: KEYS[1] pending emq:{orders}:pending KEYS[2] active emq:{orders}:active the fence reached
one queue, three keys — layout ok: lease_ms next, then the fence
call: EMQ.CLAIM 3 emq:{orders}:pending emq:{orders}:active emq:{orders}: 30000
Queue::from_base(KEYS[3]): the queue orders · KEYS[1] is its pending: yes · KEYS[2] is its active: yes
host reads made: none

A layout check is an equality: rebuild the key from the one name, compare the bytes, refuse on the first difference.

Values before the fence

EMQ.FAIL runs every zone that comes before a read, in order. Its first lines refuse the two wire-4 flow layouts straight after numkeys, before even the shape; then shape, then a layout whose base arrives as a value in ARGV[6], then the two values, and only then the fence.

// trafikey — EMQ.FAIL's first lines: everything before the fence reads only the arguments
pub(crate) fn fail(ctx: &Context, args: Vec<ValkeyString>) -> ValkeyResult {
    // Wire 4's cross-queue (5) and same-queue (8) flow layouts, straight after numkeys.
    if host::numkeys_is(&args, 5) || host::numkeys_is(&args, 8) {
        return Err(errors::emq(errors::FAIL_FLOW));
    }
    // shape: numkeys 4, the four keys present, then exactly six ARGV
    let s = host::shape("EMQ.FAIL", &args, 4)?;
    s.expect_argc(6)?;
    // layout: the base in ARGV[6] names the queue; active, schedule, dead and the row must be its own
    let q = fail_queue(&s).ok_or_else(|| s.layout_error())?;
    // values: delay_ms may be 0, max_attempts may not
    let delay_ms = parse_count(s.arg(3)).ok_or_else(|| errors::nonneg(s.verb, "delay_ms"))?;
    let max_attempts =
        parse_positive(s.arg(4)).ok_or_else(|| errors::positive(s.verb, "max_attempts"))?;
    // only now the first read: the fence
    let caller = host::fence(ctx)?;
    // …
}

The flow refusal is EMQSTATE fail flow not native: the native verb serves flat jobs, and a call in a flow layout is told so before anything else is checked. EMQ.COMPLETE opens the same way for its own flow layouts. The values come before the fence because a bad value needs no dataset to refuse it: delay_ms may be 0, max_attempts may not, and both are settled from the argument bytes alone.

Shape, layout and the form of every value are settled before the first read. Two kinds of check wait: a namespace or brand (EMQKIND: the id's JOB namespace, a bus verb's EVT, SUB and MSG brands) comes after the fence, and a due time, delay or lease deadline that would overflow once added to the server's clock is refused after the reads.

What a refusal here reads

Nothing. A call refused for its shape, its layout or a value has read no key: not the brand, not a row, not a set, not the pause flag. Its reply is a fixed text from the registry with the verb's name in it, built from the call alone. The next dive follows a call past the fence, through its reads and type checks to the one guard in front of the writes.

the pattern (Redis Patterns Applied)

Keys that share one {tag} land on one slot, so a multi-key step is legal and stays on one node. Hash-tag co-location teaches it.

the implementation (trafikey)

The layout check rebuilds every declared key from one non-empty queue name and compares the bytes: one name, one hashtag, one slot, settled before any read, beside the exact numkeys and the parsed values.

A malformed call costs the server a parse and a reply, never a read.

References

Sources

Related in this course