R7.02.2 · dive 2
Names are paid per row.
A key's name is held on every key; a Hash field's name is held on every entry; and at a million rows, the names can outweigh the values.
The technique is to keep names few and short — short enough to stay under
hash-max-listpack-value, so the row stays the compact listpack. EchoMQ's
state/attempts/payload is the applied example: three short whole words,
not abbreviated to noise. Select a naming style; the readout reports the bytes the name costs
per row.
Every name is paid per row
memory-optimization.md.txt · key & field namesA value of 25 is two bytes. The key user:profile:12345:display_name is 31 bytes
— fifteen times the value, stored once per key. For a million such keys, the names dominate. Two moves
cut it:
- Shorten the name.
user:profile:12345:display_name→u:12345:dn. The same identity, a quarter of the bytes — paid a million times. - Move the repeated prefix into a Hash.
HSET u:12345 dn "Alice" karma 42storesu:12345once and the field names once per row, instead of repeating the full prefix on every String key.
The second move is usually the larger win, because it both removes the repeated prefix and lands the fields in a single compact Hash. Below, the same field is named three ways and multiplied by a row count — the per-row difference is small, but the multiplier is the whole point:
Names are paid per row, so the saving scales with the keyspace. Shortening the field name and moving the prefix into the key are the two moves; the multiplier is what makes them worth doing.
Short, but not cryptic — the trade
memory-optimization.md.txt · readable vs byte-tightNames pull in two directions. Byte-tightness wants dn; readability wants
display_name. The honest middle is the rule EchoMQ follows: pick names that are short and
whole — small enough to keep the value under hash-max-listpack-value 64, but still a
word a reader recognizes. A field name is not where the memory battle is won or lost; the encoding is.
Abbreviating state to st saves three bytes per row and costs every future reader a
lookup — a poor trade when three short words already keep the row a listpack.
Short field names are a real technique, but the point is the encoding, not the abbreviation. Pick names short enough to keep the compact encoding and whole enough to read.
EchoMQ's deliberate three
echo_mq/jobs.ex · state / attempts / payloadThe job HASH is named for reading, not for the byte counter:
# EchoMQ.Jobs — the row, verbatim from the @enqueue script
redis.call('HSET', KEYS[1], 'state', 'pending', 'attempts', '0', 'payload', ARGV[2])
state, attempts, payload — three fields, each a recognizable
word, each short. There is no abbreviated schema behind them; the row is exactly these three.
The brevity that matters is the count (three, far under 512) and the value
size (each under 64 bytes), and those two facts are what keep Valkey storing the row as a listpack.
The moduledoc puts the design plainly:
Jobs are entities. A job's identity is a branded id under the JOB
namespace; its row is a hash at the job key; the pending set is a
same-score sorted set whose members are the ids themselves …
A small, well-named row is the memory win and the readable record at once. The key name itself is built by
EchoMQ.Keyspace — emq:{q}:job:<JOB-branded-id> — where the brace is
the cluster hash tag, not padding; every byte of the name earns its place.
Notes on ValkeyHSET key field value … stores the field
names once per row and the key prefix once on the key; a Hash stays a compact listpack while every
value is under hash-max-listpack-value (64 bytes), so short fields keep the row in the compact
encoding — valkey.io/commands/hset.
Few, short field names keep a row's values under hash-max-listpack-value and the row in the
compact encoding; the prefix lives once in the key, not once per field.
The job HASH is state/attempts/payload — short whole words, not
abbreviations — so the values stay under 64 bytes and the row stays a listpack, while remaining a record
a person can read.
Short field names are a real memory technique, but the point is the encoding, not the abbreviation. EchoMQ keeps three short, readable fields and gets the compact listpack for free — readable record, small footprint, no cryptic schema.
The same row, as the record of truth
The three fields that keep this row compact are the same three that make it the job's only canonical state. System of record reads the row from the other side — not its encoding, but the fact that its truth has no second home.
Capped structures
A small, well-named row stays compact; a List or Stream that grows must be bounded.
Capped structures takes the
LTRIM and MAXLEN ~ caps EchoMQ puts on the structures that would otherwise leak.
References
sources · relatedSources
- Valkey — Memory optimization — key and field name overhead, and the listpack value-size threshold.
- Valkey — HSET — fields on a Hash; the prefix is stored once in the key, the field names once per row.
- Valkey — HGET — read one field; the Hash approach stores the key prefix once rather than per field.
- Valkey — OBJECT ENCODING — confirm the row is still a
listpackafter the field names are chosen.
Related in this course
- R7.02 · Memory optimizationthe module hub
- R7.02.1 · listpack and intsetthe encoding the threshold keeps
- R7.02.3 · capped structuresbounding the structures that grow
- R7.01.1 · System of recordthe same 3-field job HASH as the record of truth