Memory optimization.

Redis automatically uses compact encodings for small collections. A Hash with few fields is stored as a listpack — a single sequential byte array — rather than a hash table. The listpack eliminates the per-entry pointer and bucket overhead, which can consume up to 80% of memory for small objects. An all-integer Set under its threshold uses the even tighter intset — a sorted array of fixed-width integers, no hashing at all.

The engine switches from the compact encoding to the standard one when a structure crosses a configured threshold. The defaults are Valkey's, not EchoMQ's:

A Hash with fewer than 512 entries, where each value is under 64 bytes, uses the listpack encoding. EchoMQ's job HASH carries exactly three fields — state, attempts, payload (echo/apps/echo_mq/lib/echo_mq/jobs.ex) — far under 512, with short values, so Valkey stores the row as a listpack. Not because EchoMQ tuned the thresholds (the committed infra/valkey/conf/valkey.conf sets none), but because the row is deliberately minimal.

Each individual key has overhead — memory for the key name, the expiration metadata, and internal bookkeeping. For millions of small values, that per-key overhead dominates the values themselves. Instead of a String key per item, group items into Hash buckets:

With 1000 items per bucket you have 1000 Hash keys instead of 1,000,000 String keys, and each Hash stays small enough for the compact encoding. For item ID 12345 with bucket size 1000: bucket = 12345 / 1000 = 12, key = "users:12", field = "12345:name". The per-key overhead is paid 1000 times, not a million.

Every byte in a key name consumes memory, and the name is stored on every key. At scale, short keys matter: user:profile:12345:display_name becomes u:12345:dn, or — better — a Hash so the prefix is stored once rather than per field (HGET u:12345 dn). EchoMQ names its job-HASH fields state/attempts/payload: short, but not abbreviated to noise — small enough to keep the value under hash-max-listpack-value while staying a word a reader recognizes.

Redis also uses an integer encoding for numeric strings, which share a pool of pre-allocated small integers: SET counter "42" is efficient, SET counter "42 items" is not. When a value can be a pure number, store it as one — EchoMQ's attempts holds the bare integer fence advanced by HINCRBY, never "3 tries". And every key with a TTL costs extra memory for the expiration; grouping keys that expire together under one Hash TTL avoids paying it per entry. EchoMQ's job rows carry no TTL — a job lives until completed or reaped.

Measure before optimizing: INFO memory for the overall picture, MEMORY USAGE <key> for one key, MEMORY STATS for samples, MEMORY DOCTOR for a report. Then change the lowest-effort technique and measure again.

When memory is exhausted, the maxmemory-policy setting determines what happens. The cache policies (allkeys-lru, allkeys-lfu, volatile-ttl, …) delete keys to keep serving; noeviction returns errors on writes instead. For a cache, LRU or LFU makes sense; for data that must not be lost, noeviction is the only safe posture — which is exactly why EchoMQ runs it. That posture is taught in depth in R7.1 and is not re-taught here:

EchoMQ applies three of these techniques by construction. Its job HASH stays a listpack because the row is three short fields. Its growing structures — the per-group wake list and the streams — are capped where they are written, so they cannot leak memory no matter the traffic. Select a structure to read how it is bounded:

Each dive takes one slice of the memory discipline, in order: the compact encodings and the threshold flip, the field names that keep the row in them, and the caps that bound the structures that grow. Read them in order — the encoding first, the names second, the caps third.