TRA and native values · dive 03
The native value type
The default build registers no module data type. The one data type in the source,
traprobe1, compiles only behind a feature that is off by default, and it exists to prove
that a module-owned value survives RDB, AOF and a restart. A native bitmap value type is designed, not built.
A module data type is how a Valkey module owns a value: the server keeps an opaque pointer under
the key and calls the module’s callbacks to save the value to an RDB file, load it back, rewrite it into the
AOF and free it. Every value trafikey’s default build writes is one of the
server’s own types: a TRA bitmap is a string, an EMQ job
row is a hash. The probe type is the exception, compiled only on request, and its callbacks are the recorded
proof that a module-owned value persists. Pick a value to read who owns its bytes and who writes it to disk; then walk
the probe through a save, a rewrite and a restart.
No data type in the default build
The module declaration lists its data types, and the list is empty. The default build creates none anywhere
else either: the one call that creates a data type sits behind the probe-type feature.
So every value the default build writes is one of the server’s own types. A
TRA bitmap is a string: the server keeps its bytes, saves it to RDB, writes it on an AOF
rewrite and frees it, as it does for any string, and GET reads it with or without the
module. An EMQ job row is a hash the module writes through the server’s own
commands. The manifest ends on the same fact: no bitmap data type is implemented in the crate.
// trafikey — the module declaration: the list of data types is empty
valkey_module! {
name: "trafikey",
version: 100,
allocator: (ValkeyAlloc, ValkeyAlloc),
data_types: [],
preload: preload,
init: init,
commands: [],
}
Who owns a value · pick one
owner: the server: a string, written in place by TRA.SETBIT
RDB: written by the server, as a string
AOF: TRA.SETBIT as the client sent it; a rewrite writes the string as it writes any string
a restart without the module: the RDB copy loads as an ordinary string, and GET reads the bytes
In the default build the server owns every value; the module owns commands, a C table and three Lua sources.
Off by default
The crate’s feature table declares probe-type with no dependencies and
declares no default set, so a plain build leaves the probe out. The repository’s build line is plain: it builds the
package and passes no feature.
# trafikey — Cargo.toml, the feature table and the closing comment [features] # the persistence probe for a module data type, never part of a shipped artifact; # no default feature is declared, so a plain build leaves it off probe-type = [] # … # The cdylib is loaded by Valkey; no standalone RESP listener or bitmap data type is implemented here.
# deploy — build-local.sh, the build line: the package, no feature flag
cargo build --locked --release -p trafikey
The feature gates three places. The probe_type module
compiles only under it. The first step of init, also under it, creates the data type,
and a failure refuses the load. Two rows of the registration table, also under it, register
TRAPROBE.SET and TRAPROBE.GET in a family of their own,
Probe, with no capability token, so neither EMQ.VERSION nor
TRA.CAPABILITIES reports them. A default build has neither the type nor the two
commands.
// trafikey — lib.rs: the probe module and init's first step, both under the feature mod emq; #[cfg(feature = "probe-type")] mod probe_type; mod registry; mod tra; // … fn init(ctx: &Context, _: &[ValkeyString]) -> Status { #[cfg(feature = "probe-type")] if probe_type::PROBE_TYPE.create_data_type(ctx.ctx).is_err() { return Status::Err; } // … every later step of init, the shared API export last Status::Ok }
// trafikey — registry::TABLE, the two probe rows: family Probe, no capability token
#[cfg(feature = "probe-type")]
probe_set => crate::probe_type::set, Row { name: c"TRAPROBE.SET", family: Family::Probe, arity: 3,
flags: "write deny-oom", keys: &[Keys { flags: &["READ_WRITE", "UPDATE"], index: 1, find: ARGV1 }],
acl: c"write slow", caps: &[], summary: None, complexity: None, since: None };
#[cfg(feature = "probe-type")]
probe_get => crate::probe_type::get, Row { name: c"TRAPROBE.GET", family: Family::Probe, arity: 2,
flags: "readonly fast", keys: &[Keys { flags: &["READ_ONLY", "ACCESS"], index: 1, find: ARGV1 }],
acl: c"read fast", caps: &[], summary: None, complexity: None, since: None };
One feature switches the probe on in three places at once, and nothing in the repository’s build switches it on.
Four callbacks, two commands
The type is named traprobe1, nine characters as the module API requires, with an encoding
version of 0. Its value is a box of bytes in module memory. It fills four callbacks: rdb_save
writes the bytes as one string buffer; rdb_load reads them back and returns NULL for
any encoding version but 0; aof_rewrite emits one
TRAPROBE.SET key bytes command per value; free drops the box.
It reports no digest and no memory usage.
// trafikey — probe_type: the traprobe1 type and its four callbacks pub struct Probe(Vec<u8>); pub static PROBE_TYPE: ValkeyType = ValkeyType::new( "traprobe1", 0, raw::RedisModuleTypeMethods { version: raw::REDISMODULE_TYPE_METHOD_VERSION as u64, rdb_load: Some(rdb_load), rdb_save: Some(rdb_save), aof_rewrite: Some(aof_rewrite), free: Some(free), digest: None, mem_usage: None, // … aux_*, free_effort, unlink, copy, defrag and the rest: None or 0 }, ); unsafe extern "C" fn rdb_load(rdb: *mut raw::RedisModuleIO, encver: c_int) -> *mut c_void { if encver != 0 { return std::ptr::null_mut(); } match raw::load_string_buffer(rdb) { Ok(buffer) => Box::into_raw(Box::new(Probe(buffer.as_ref().to_vec()))).cast(), Err(_) => std::ptr::null_mut(), } } unsafe extern "C" fn rdb_save(rdb: *mut raw::RedisModuleIO, value: *mut c_void) { raw::save_slice(rdb, unsafe { &(*value.cast::<Probe>()).0 }); } unsafe extern "C" fn aof_rewrite( aof: *mut raw::RedisModuleIO, key: *mut raw::RedisModuleString, value: *mut c_void, ) { let bytes = unsafe { &(*value.cast::<Probe>()).0 }; #[expect( clippy::unwrap_used, reason = "the preload gate refuses the load without EmitAOF; aof_rewrite has no error channel" )] unsafe { raw::RedisModule_EmitAOF.unwrap()( aof, c"TRAPROBE.SET".as_ptr(), c"sb".as_ptr(), key, bytes.as_ptr(), bytes.len(), ); } } unsafe extern "C" fn free(value: *mut c_void) { drop(unsafe { Box::from_raw(value.cast::<Probe>()) }); }
TRAPROBE.SET key bytes stores a new box under the key
through the binding’s typed key API, replicates verbatim and replies OK.
TRAPROBE.GET key returns the bytes, or a null reply for a missing key. On a key of any other type the binding knows, its own check answers Existing key has wrong Valkey type, not the standard WRONGTYPE reply, and the record keeps that as a finding for any handler built on the typed key API. That check first converts the raw key type, so on a Valkey 9.2 path hash key it is the abort trap that Bitmaps on strings reads.
// trafikey — probe_type: TRAPROBE.SET and TRAPROBE.GET pub(crate) fn set(ctx: &Context, args: Vec<ValkeyString>) -> ValkeyResult { // The host enforces arity 3 before the handler runs; the else arm is its arity reply. let [_, name, payload] = args.as_slice() else { return Err(ValkeyError::WrongArity); }; ctx.open_key_writable(name) .set_value(&PROBE_TYPE, Probe(payload.as_slice().to_vec()))?; ctx.replicate_verbatim(); Ok(ValkeyValue::SimpleStringStatic("OK")) } pub(crate) fn get(ctx: &Context, args: Vec<ValkeyString>) -> ValkeyResult { // The host enforces arity 2 before the handler runs; the else arm is its arity reply. let [_, name] = args.as_slice() else { return Err(ValkeyError::WrongArity); }; Ok(match ctx.open_key(name).get_value::<Probe>(&PROBE_TYPE)? { Some(probe) => ValkeyValue::StringBuffer(probe.0.clone()), None => ValkeyValue::Null, }) }
The probe’s evidence is a recorded scratch run on Valkey 9.1.2, on scratch
ports apart from the stand’s two nodes. TRAPROBE.SET pk "hello\x01world", SAVE and
a restart gave back TYPE pk as traprobe1 and the bytes exactly.
With appendonly yes and aof-use-rdb-preamble no,
BGREWRITEAOF wrote a base file holding two TRAPROBE.SET
commands from the rewrite callback, and a restart restored both keys. The control started a server without the
module on the same dump.rdb, and the server refused it:
The RDB file contains module data I can't load: no matching module type 'traprobe1'.
The probe, step by step · pick a step
what runs: rdb_save writes the bytes as one string buffer; at the restart rdb_load checks that the encoding version is 0 and rebuilds the box
callbacks: rdb_save, rdb_load
reply: TYPE pk → traprobe1; TRAPROBE.GET pk → the same 11 bytes
The probe proves the mechanism, a module value through RDB, AOF and a restart; it carries no bitmap and ships in no artifact.
Pattern & implementation
Where Redis Patterns Applied keeps one bitmap string per day and names the cost of a sparse id space, this dive reads a module whose bitmaps are that same flat string.
One bitmap per day, a bit per user id, counted and combined; a bitmap allocates to its highest offset, so a sparse id space wastes memory. Daily-active patterns teaches it.
A TRA bitmap is the server’s string: TRA.SETBIT at a
high offset grows it with zero bytes up to that offset. The module’s one data type is a persistence
probe, compiled only on request.
This dive closes the module. Return to the hub for the whole family in one picture, or go on to the workshop.
References
Sources
- Valkey — Modules native types — a module data type, its nine-character name, encoding version and callbacks.
- Valkey — Modules API reference — CreateDataType, EmitAOF, the typed key API and verbatim replication.
- Valkey — Persistence — RDB snapshots, the AOF and its rewrite.
- Valkey — BGREWRITEAOF — the rewrite; without the RDB preamble it calls each module value’s AOF callback.
- Valkey — SAVE — the synchronous snapshot the probe run used.
- Cargo — Features — optional features and the default set.
Related in this course
- TRA and native values — the module this dive closes.
- Bitmaps on strings — the string a TRA bitmap is today.
- The shared ABI and bundled Lua — the previous dive.
- The registration table — the rows the feature adds, and the families they belong to.
- The workshop — the part’s closing exercise.
- redis-patterns · Daily-active patterns — the near side of the door.