jonnify·redis /redis-patterns/data-modeling/memory-optimization/short-field-names

R7.02.2 · dive 2

Names are paid per row.

A key's name is held on every key; a Hash field's name is held on every entry; and at a million rows, the names can outweigh the values.

The technique is to keep names few and short — short enough to stay under hash-max-listpack-value, so the row stays the compact listpack. EchoMQ's state/attempts/payload is the applied example: three short whole words, not abbreviated to noise. Select a naming style; the readout reports the bytes the name costs per row.

Three ways to name the same field. Select one; the readout reports the bytes it costs and whether a reader can still read it.
display_name 15 B state 5 B st 2 B
§1

Every name is paid per row

memory-optimization.md.txt · key & field names

A value of 25 is two bytes. The key user:profile:12345:display_name is 31 bytes — fifteen times the value, stored once per key. For a million such keys, the names dominate. Two moves cut it:

  • Shorten the name. user:profile:12345:display_name → u:12345:dn. The same identity, a quarter of the bytes — paid a million times.
  • Move the repeated prefix into a Hash. HSET u:12345 dn "Alice" karma 42 stores u:12345 once and the field names once per row, instead of repeating the full prefix on every String key.

The second move is usually the larger win, because it both removes the repeated prefix and lands the fields in a single compact Hash. Below, the same field is named three ways and multiplied by a row count — the per-row difference is small, but the multiplier is the whole point:

Per row times N rows. Pick a row count; the bars compare a verbose schema against a short one across the whole keyspace.
— field names only, 30 B verbose vs 9 B short per row
verbose schema · 30 B/row — short schema · 9 B/row —

Names are paid per row, so the saving scales with the keyspace. Shortening the field name and moving the prefix into the key are the two moves; the multiplier is what makes them worth doing.

§2

Short, but not cryptic — the trade

memory-optimization.md.txt · readable vs byte-tight

Names pull in two directions. Byte-tightness wants dn; readability wants display_name. The honest middle is the rule EchoMQ follows: pick names that are short and whole — small enough to keep the value under hash-max-listpack-value 64, but still a word a reader recognizes. A field name is not where the memory battle is won or lost; the encoding is. Abbreviating state to st saves three bytes per row and costs every future reader a lookup — a poor trade when three short words already keep the row a listpack.

Short field names are a real technique, but the point is the encoding, not the abbreviation. Pick names short enough to keep the compact encoding and whole enough to read.

§3

EchoMQ's deliberate three

echo_mq/jobs.ex · state / attempts / payload

The job HASH is named for reading, not for the byte counter:

# EchoMQ.Jobs — the row, verbatim from the @enqueue script
redis.call('HSET', KEYS[1], 'state', 'pending', 'attempts', '0', 'payload', ARGV[2])

state, attempts, payload — three fields, each a recognizable word, each short. There is no abbreviated schema behind them; the row is exactly these three. The brevity that matters is the count (three, far under 512) and the value size (each under 64 bytes), and those two facts are what keep Valkey storing the row as a listpack. The moduledoc puts the design plainly:

echo/apps/echo_mq/lib/echo_mq/jobs.ex · moduledocfrozen
Jobs are entities. A job's identity is a branded id under the JOB
namespace; its row is a hash at the job key; the pending set is a
same-score sorted set whose members are the ids themselves …

A small, well-named row is the memory win and the readable record at once. The key name itself is built by EchoMQ.Keyspace — emq:{q}:job:<JOB-branded-id> — where the brace is the cluster hash tag, not padding; every byte of the name earns its place.

Notes on ValkeyHSET key field value … stores the field names once per row and the key prefix once on the key; a Hash stays a compact listpack while every value is under hash-max-listpack-value (64 bytes), so short fields keep the row in the compact encoding — valkey.io/commands/hset.

the pattern

Few, short field names keep a row's values under hash-max-listpack-value and the row in the compact encoding; the prefix lives once in the key, not once per field.

its EchoMQ application

The job HASH is state/attempts/payload — short whole words, not abbreviations — so the values stay under 64 bytes and the row stays a listpack, while remaining a record a person can read.

Short field names are a real memory technique, but the point is the encoding, not the abbreviation. EchoMQ keeps three short, readable fields and gets the compact listpack for free — readable record, small footprint, no cryptic schema.

R7.01.1 · in this course

The same row, as the record of truth

The three fields that keep this row compact are the same three that make it the job's only canonical state. System of record reads the row from the other side — not its encoding, but the fact that its truth has no second home.

R7.02.3 · in this course

Capped structures

A small, well-named row stays compact; a List or Stream that grows must be bounded. Capped structures takes the LTRIM and MAXLEN ~ caps EchoMQ puts on the structures that would otherwise leak.

§4

References

sources · related

Sources

Related in this course