jonnifyelixir
F4 · Algorithms & Data Structures / F4.08 · persistence

F4 · Persistent maps · module 8

Branded ids & persistence

Validate before you query Three deep dives Advanced: shedding enumeration & DDoS

F4.07 built the id; this module stores it. The 64-bit Snowflake is a bigint primary key in SQLite and PostgreSQL and a namespaced string key in Redis — compact, sortable, and time-ordered everywhere. But the property that earns its keep arrives before any store is touched: a branded id validates itself. A request for a malformed or time-impossible id can be answered with a 404 in constant time, with zero I/O.

Take a request like GET /user/profile/USR0NbWMtkosp8. The edge checks the id’s shape and timestamp first; only a well-formed USR id from a plausible moment reaches the database. Select a request and watch the gate decide.

Struct → row · the id is the key

%User{} users row id: USR0NbWMtkosp8 PK name: "Ada" email: "ada@x.io" role: :admin id · PK — name — role — row not yet written · the id will be its primary key INSERT INTO users (id, name, email, role) VALUES (…)

%User{} in memory · the row is empty.
persist maps each field to its column — the branded id becomes the primary key.

Validate before you query

The endpoint expects a USR id. Four checks run with no I/O: the length is exactly fourteen, the payload is base62, the decoded integer is in 64-bit range, and the namespace matches and the timestamp is not in the future. Anything that fails is a 404 the database never sees. Select a request.

Incoming request · select one

GET /user/profile/<id> USR0NbWMtkosp8 ↓ edge validator · no I/O ↓ 200 · forward to store a well-formed USR id is forwarded to the store 

      

      
verdict: valid — forward

      
cost: one decode, zero I/O

    

    
A malformed or impossible id is rejected for the price of a shift and a comparison. The store only
      ever sees ids that could exist — and for those, the id already says when they were made.

  


  

    
Three deep dives

    

      
In order: what you actually store and index; how a relational store range-queries by time with no
        created_at column; and how Redis keys, plus edge validation, shed abusive traffic.

    


    

      
        
F4.08.1

        
Branded ids as keys 

        
Store the 64-bit integer, index it, and brand it only at the edge: eight bytes that sort by time and validate themselves.

      
      
        
F4.08.2

        
SQLite & PostgreSQL 

        
A bigint column and a time-range query expressed as id BETWEEN min AND max, no timestamp column needed.

      
      
        
F4.08.3

        
Redis keys 

        
Namespaced string keys, and an edge validator that sheds malformed and impossible ids before they reach the cache.

      
    

  


  

    
Advanced: shedding enumeration & DDoS

    

      
Two cheap attacks lean on identifiers. Enumeration (an insecure-direct-object-reference
        probe) walks /user/1, /user/2, … to harvest records;
        sequential counters make that trivial. A branded Snowflake is sparse and non-incrementing — the
        timestamp, worker, and sequence bits leave no guessable next id — so there is nothing to walk.
        Flooding sends a torrent of bogus ids hoping each becomes a database round-trip. Because a
        branded id is self-describing, the edge rejects the ones that cannot exist — wrong length, non-base62,
        out of 64-bit range, wrong namespace, or a timestamp in the future — in constant time, before a single
        query. That is a whole class of traffic answered with a 404 and no I/O.

      
The honest boundary: validation proves an id is well-formed and plausibly-timed, not that it
        exists. A correctly-shaped id still needs one lookup to confirm the record. What the check removes is
        the cheap, high-volume probe — the part that costs the attacker nothing and the database the most. And
        because the timestamp is in the id, the floor is free: any decoded id is at or after the 2024 epoch by
        construction, so “before the service existed” is unrepresentable, and “after now” is
        a one-line reject.

    

    
# a Plug that runs before the controller — no Repo call yet
def call(conn, _opts) do
  case Portal.Id.validate(conn.params["id"], :USR) do
    {:ok, _snowflake} -> conn                       # well-formed, plausible — proceed
    {:error, _reason} -> conn |> send_resp(404, "") |> halt()  # shed, zero I/O
  end
end

    

      

        
F4.07 · the id

        
A namespaced, time-ordered 64-bit integer that decodes to a moment.

      

      
      

        
F4.08 · the key and the gate

        
Stored as a bigint or a Redis key, and checked at the edge before any query runs.

      

    

    
Start with branded ids as keys, then
      SQLite & PostgreSQL, then
      Redis keys. Predecessor:
      F4.07 — Identifiers, Snowflake & branded ids.
      Next module: F4.09 — Branded CHAMP maps & GenServer, which keys an
      in-memory CHAMP on the same id, partitioned by namespace.

  


  

    
References

    
Primary sources for this lesson, and where it connects in the course.

    

      
Sources

      
      
Related in this course