Valkey and the module · dive 01
Loading the module
One gate before anything registers, then one init that runs its steps in a fixed order and stops at the first that fails.
The module's declaration names two functions. preload runs first,
before any command exists, and asks one question: can the module run on this host? It requires three
things at once: every host API the module calls, no module arguments, and a Valkey 9 server. Then the
table's commands register, and init runs its steps in order: the ACL categories of
every row, the server's run_id, the server mode, the shared C API. Set the gate's
three inputs on the right, then step through init below.
The preload gate · set the host
apis: true · args.is_empty(): true · v.major == 9: true
next: every row of the table registers its command, then init runs
The declaration: two hooks, no commands, no value types
Everything the server learns about the module starts at one macro. valkey_module!
names the module, its allocator, the two hooks the load runs, and two lists, both empty.
// trafikey — the module declaration valkey_module! { name: "trafikey", // the declaration's own number; EMQ.VERSION and TRA.CAPABILITIES report other strings version: 100, allocator: (ValkeyAlloc, ValkeyAlloc), // no module value type in the default build data_types: [], // the gate, before anything registers preload: preload, // the steps after every row has registered init: init, // no command from the macro: each one comes from a row of registry::TABLE commands: [], }
data_types: [] means the default build registers no
module value type; the persistence probe type exists only under a crate feature that no shipped artifact
enables. commands: [] means no command comes from the macro: each one comes from
a row of the registration table, which the next dive reads. And version: 100 is the
declaration's own number, one of three separate version strings: EMQ.VERSION
reports the crate's version and its major as module_version and
module_major, and TRA.CAPABILITIES reports a version of
its own. A client checks module_major.
Two hooks and two empty lists: the macro declares when the module checks, and the table declares what it serves.
The gate: three conditions, one match arm
preload gathers two facts before it answers.
registry::host_apis_present() checks every host function pointer the module calls;
ctx.get_server_version() asks the server for its version. The one arm that answers
Status::Ok needs all three conditions at once. Every other case answers
Status::Err, a version the host cannot report included, and the load is
refused before a single command exists.
// trafikey — preload, the gate the declaration names /// Load gate, before anything registers: no module arguments, a Valkey 9.x host, and every /// host API this module calls present. raw wrappers unwrap the host pointers and the release /// profile aborts on panic, so a missing API must refuse the load instead of killing the node. fn preload(ctx: &Context, args: &[ValkeyString]) -> Status { // one boolean over the whole API list (read below) let apis = registry::host_apis_present(); // the server's own version; an Err here falls to the refusing arm match ctx.get_server_version() { // the API list, the loadmodule line's arguments, the major: all three or nothing Ok(v) if apis && args.is_empty() && v.major == 9 => Status::Ok, _ => Status::Err, } }
The doc comment gives the reason the gate refuses rather than trusting the host. The release profile aborts on a panic, and that abort is no recovery: it keeps a panic from unwinding across the C boundary by ending the process, and inside a loaded module the process is the node.
# trafikist — the workspace release profile [profile.release] # … panic = "abort" # … # Release abort semantics prevent unwinding across FFI; they do not provide panic recovery.
The major-version condition lives in preload because the
binding the crate compiles against cannot carry it. That binding is pinned, and its compatibility feature
is the Valkey 8 level; the module's own requirement, a Valkey 9 host, is enforced at load:
# trafikey — the host binding the crate compiles against [dependencies] # Pinned host binding: crates.io valkey-module 0.1.14 (checksummed; its feature set predates git 9ed1bbf, # which adds a 9-0 feature). Bindgen runs over the crate's bundled header; the 9.x gate is in preload(). valkey-module = { version = "=0.1.14", default-features = false, features = ["min-valkey-compatibility-version-8-0"] }
The gate is the one place a host is judged: compiled against a binding at the Valkey 8 level, loaded only on a Valkey 9 server, with arguments refused outright.
The API half of the gate
host_apis_present is one boolean over every host function the module calls, in
groups: key and string access, replies, verbatim replication, keyspace notifications, the shared-API
export, command registration, module data types, then the raw host adapter the
EMQ.* handlers use (Call and its reply readers, the cached
clock, the replica-traffic check), the caller's ACL check, the server-info reads behind the epoch, and the
context flags behind the server mode.
// trafikey — registry::host_apis_present, the API half of the gate macro_rules! all_present { ($($api:ident),* $(,)?) => { true $(&& unsafe { (*std::ptr::addr_of!(raw::$api)).is_some() })* }; } /// The preload API gate: every host API this module calls is present. raw wrappers unwrap the /// host pointers and the release profile aborts on panic, so a missing API must refuse the load. pub fn host_apis_present() -> bool { all_present!( RedisModule_OpenKey, // … RedisModule_CreateCommand, RedisModule_GetCommand, RedisModule_SetCommandInfo, RedisModule_SetCommandACLCategories, // … // emq: the raw host adapter (emq/host.rs) and its replies. RedisModule_Call, // … RedisModule_CachedMicroseconds, RedisModule_AvoidReplicaTraffic, // … // the caller's ACL check, for keys outside a command's key-spec RedisModule_ACLCheckKeyPermissions, // … // the three server-info reads behind the epoch init loads RedisModule_GetServerInfo, RedisModule_ServerInfoGetFieldC, RedisModule_FreeServerInfo, // the context flags behind the server mode init records RedisModule_GetContextFlags, ) }
The macro expands to true && … over one
presence test per name, so a single absent pointer makes the whole gate false. The list also says what the
module never calls: it names no blocking-client API, no ready-key signal and no timer. Nothing in the module
blocks a client or runs on a clock of its own; every sweep runs when a client calls it.
The API list is the module's whole contract with its host, checked once, before the first registration.
init, in order
Once the gate has passed and the table's commands have registered, init runs its
steps in a fixed order. A step that fails refuses the load at once, and the steps after it never run.
// trafikey — init, after every row has registered fn init(ctx: &Context, _: &[ValkeyString]) -> Status { // step 0, only in a build with the probe-type feature: the probe's module data type #[cfg(feature = "probe-type")] if probe_type::PROBE_TYPE.create_data_type(ctx.ctx).is_err() { return Status::Err; } // step 1: every row's ACL categories; a table that differs from the registered set refuses if registry::set_acl_categories(ctx) == Status::Err { return Status::Err; } // step 2: the server epoch, run_id, read once and held in memory; no key is written if let Err(e) = emq::epoch::load(ctx) { ctx.log_warning(&format!( "trafikey: emq epoch not loaded: {}", emq::errors::text(&e) )); return Status::Err; } // step 3: the server mode, read once; this step cannot refuse emq::host::load_mode(ctx); // step 4: the versioned shared C API, exported under its name for other modules registry::export_shared_api( ctx, c"trafikey_get_api_v1", tra::trafikey_get_api_v1 as *const std::ffi::c_void, ); Status::Ok }
Step 1, the ACL categories. Every row's categories are set on its registered command, and the step first proves the table and the registered set equal: distinct names, one registered command per row, each found in this module. The next dive reads it with the table.
Step 2, the epoch. The server's run_id is read from
INFO server once, checked, and held for the life of the process:
// trafikey — emq::epoch::load, init's step 2 /// Reads `INFO server` `run_id` once (host.rs [`super::host::server_run_id`]) and caches it for /// the process. A second call keeps the first value (one process, one `run_id`). A missing host /// API or a malformed value refuses typed, and module init then refuses the load. pub(crate) fn load(ctx: &Context) -> Result<(), ValkeyError> { let id = host::server_run_id(ctx)?; // exactly 40 lowercase hex characters, as the server prints a run_id if !valid_run_id(&id) { return Err(errors::emq(errors::HOST_CALL)); } let _ = RUN_ID.set(id); Ok(()) }
Every lease the job core writes records that value, and only the stall sweep reads it, to tell a restart from a stall; The job core natively reads that use. Step 3 caches whether the server is cluster-enabled, a startup-only setting, so the value cannot change while the module is loaded; the third dive reads what follows from it. Step 4 exports one name and one pointer, the versioned C API another module fetches to reach the bundled TRA Lua sources; TRA and native values reads it.
// trafikey — registry::export_shared_api, init's step 4: one name, one pointer
pub fn export_shared_api(ctx: &Context, name: &CStr, api: *const c_void) {
unsafe { raw::export_shared_api(ctx.ctx, api, name.as_ptr()) };
}
init, step by step · pick a build and an outcome
step 0, probe type: compiled out (no probe-type feature)
step 1, ACL categories: set on 24 rows; the table equals the registered set
step 2, epoch: run_id read, 40 lowercase hex characters, held in memory
step 3, server mode: recorded
step 4, shared API: trafikey_get_api_v1 exported
log: nothing
A fixed order with early returns: by the time init answers
Status::Ok, the categories, the epoch and the mode are all in place.
An immutable core checks at boot and refuses to serve on a value it cannot trust, so a wrong deployment fails loudly at start instead of quietly at the first request. The immutable core teaches it.
preload refuses a non-9 host, a missing host API or any argument before a
command exists; init refuses a table mismatch, a rejected category or an
unreadable run_id before the load completes.
Recap — loading the module
The declaration names a gate and an init, registers no value type and lists no command. The gate requires
every host API, no arguments and a Valkey 9 server, all at once, before any command exists. Then every row
registers, and init sets the ACL categories, reads the epoch, records the server
mode and exports the shared API, in that order, refusing the load at the first step that fails. The next
dive reads the table every command comes from.
Next in this module: The registration table — one row per command, and everything that derives from it.
References
Where the load is grounded, and the server behaviour it relies on.
Sources
- Valkey — Modules introduction — loading a module, its load entry point and its arguments.
- Valkey — Modules API reference — the host functions the gate checks for, one by one.
- Valkey — MODULE LOAD — the runtime form of
loadmodule, with its optional arguments. - Valkey — INFO — the
serversection and itsrun_id, read once at load. - Valkey — ACL — the command categories init sets on every row.
Related in this course
- Valkey and the module — the module this dive belongs to.
- The native module — the Protocol's introduction to the second executor.
- One transition, two executors — where
lease_epochfirst appears on a claim. - The version fence — the Elixir connector's own check at start.
- redis-patterns · The immutable core — refusing at boot, the near side of the door.