R7.05.1 · dive 1

The HNSW graph.

A vector set is a Redis 8 data type where each element carries a vector. VADD inserts into a live HNSW graph; VSIM queries it in O(log N).

HNSW (Hierarchical Navigable Small World) is a graph-based approximate nearest-neighbour algorithm. In Redis 8, the graph builds automatically on each VADD — no separate index creation step. Select a command; the readout names what it does and its complexity.

What is it Commands HNSW Quantization Engine note

The four primary commands. Select one; the readout names what it does, its complexity, and the key option.

readout · Select a command. The four primary operations are VADD (insert), VSIM (nearest neighbours), VREM (delete), and VEMB (retrieve a vector). The HNSW graph updates live on VADD — no batch index build.

§1

What is a vector set?

vector-sets.md.txt · introduction

A Vector Set is a Redis 8 data type similar to a Sorted Set, but each element carries a vector instead of a score. The analogy:

Sorted Set: ZADD key score element → query by score range with ZRANGE.
Vector Set: VADD key VALUES n … element → query by vector similarity with VSIM.

Where a sorted set answers “what is ranked near this score”, a vector set answers “what is semantically similar to this vector” — using approximate nearest-neighbour search over the HNSW graph the structure maintains.

Vectors are numerical representations (embeddings) produced by machine-learning models. Similar items have vectors that point in similar directions in the embedding space: two documents that mean the same thing will have embeddings close together even if they share no keywords.

A vector set is a sorted set with a vector where the score would be. VSIM returns the elements whose vectors are closest to the query vector — a semantic neighbourhood, not a score range.

§2

Core commands

vector-sets.md.txt · Core Commands

Adding and querying are the two primary operations; the remainder are maintenance and introspection:

redis.io/docs/latest/develop/data-types/vector-sets/ · figure 14frozen

VADD key VALUES 3 0.1 0.5 0.9 my-element           # add an element with an inline vector
VADD key FP32 <binary-blob> my-element             # …or a binary blob (faster for clients)
VSIM key VALUES 3 0.1 0.5 0.9 COUNT 10 WITHSCORES  # find the 10 nearest by a query vector
VSIM key ELE existing-element COUNT 10 WITHSCORES  # …or nearest to an existing element
VCARD key  VDIM key  VEMB key ele                   # count / dimension / get element's vector
VREM  key element                                   # true deletion, memory reclaimed

Key options on VADD:

Q8 (default) — 8-bit quantization; 4× memory reduction, ~96% recall.
BIN — 1-bit quantization; 32× reduction, ~80% recall, fastest.
NOQUANT — full fp32 precision; best recall, most memory.
M num — HNSW connectivity (default 16); higher = better recall, more memory per node.
EF num — build-time exploration factor (default 200); higher = better graph quality, slower insert.
SETATTR '{...}' — JSON metadata for filtered queries.

Key options on VSIM:

COUNT n — return the top N results (default 10).
WITHSCORES — include similarity scores (0 to 1, where 1 is identical).
EF num — search exploration factor; higher = better recall at the cost of latency.
FILTER 'expr' — restrict results to elements whose JSON metadata matches.
EPSILON d — only return items with similarity ≥ (1 − d).

The two commands are VADD (insert, update the graph live) and VSIM (query by vector or by element). The graph updates on every insert; no batch rebuild is needed.

§3

The HNSW graph — O(log N) approximate nearest neighbour

vector-sets.md.txt · Performance Characteristics

HNSW (Hierarchical Navigable Small World) organises items in a multi-layer graph. The top layer holds a few items with long-range connections; lower layers are progressively denser. A query starts at the top layer and does a greedy walk at each level, pruning the search space before descending to the next. The result is approximate nearest neighbours in O(log N) time — a fraction of the O(N) cost of a linear scan.

Throughput numbers from the source:

Operation	Complexity	Typical throughput
`VSIM`	O(log N)	~50K ops/sec (3M items, 300 dims)
`VADD`	O(log N)	~5K ops/sec
`VREM`	O(log N)	Fast; memory is reclaimed immediately
Load from RDB	O(N)	~3M items in 15 seconds

The key tuning levers are M (connectivity on insert, default 16) and EF (exploration factor, default 200 on insert, 10× COUNT on query). Higher M builds a denser graph with better recall at higher memory cost per element; higher EF at query time explores more candidates for better recall at higher latency. The default settings are good for most workloads — tune only when a recall measurement shows a gap.

Memory with default Q8 quantization: about 1 KB per element at 300 dimensions (1 byte/dim + HNSW graph overhead at ~M × 2.5 pointers per element).

HNSW is the reason VSIM is O(log N) rather than O(N). The graph builds automatically on VADD — no FT.CREATE, no schema, no batch index phase — the Redis 8 surface handles it at insert time.

§4

Quantization trade-offs

vector-sets.md.txt · Quantization Trade-offs

Quantization converts the fp32 components of each vector to a smaller representation at insert time. Three modes; select one to read the trade:

Three quantization modes. Select one; the readout names its memory cost per dimension, relative speed, and recall.

readout · Select a quantization mode. Q8 is the default: 1 byte per dimension, 4× memory reduction from fp32, ~96% recall. Use NOQUANT when recall loss is unacceptable; use BIN for candidate retrieval before reranking.

Recall numbers assume the default HNSW parameters. Recall degrades gracefully: Q8 at ~96% recall is indistinguishable from NOQUANT for most semantic search tasks. BIN’s ~80% recall makes it suitable for a first-pass candidate set before reranking those candidates on NOQUANT or Q8 vectors.

With Q8 and 300-dimensional vectors, one million elements costs roughly 1 GB. With BIN and 1536 dimensions (OpenAI-scale embeddings), one million elements costs about 200 MB — a 30× reduction from NOQUANT.

Start with Q8 (the default). Measure recall against a ground-truth set; if 96% recall is acceptable, leave it. If speed at scale matters more than recall, consider BIN as a first-stage filter.

Notes on ValkeyThe BCS stack runs Valkey 9, which does not have VADD or VSIM as native commands. The same HNSW algorithm is available through the valkey-search module (C++, installed separately): you define an index with FT.CREATE myIndex SCHEMA embedding VECTOR HNSW 6 TYPE FLOAT32 DIM 3 DISTANCE_METRIC COSINE over a Hash or JSON key, then query with FT.SEARCH idx "*=>[KNN 10 @embedding $vec]". The API surface is completely different even though the underlying graph algorithm is the same. Same HNSW; opposite surface. Cite: valkey.io/topics/search · valkey.io/commands/ft.create.

§5

Engine divergence — Redis 8 vs Valkey-search

valkey.io/topics/search/ · figure 15

The two engines reach the same HNSW algorithm by opposite paths:

redis.io/…/vector-sets/ vs valkey.io/topics/search/ · figure 15 — the engine divergencefrozen

                 Redis 8 Vector Sets        Valkey-search module
What             a native type              a module (valkey-search, C++)
Index            HNSW automatic, no          FT.CREATE … VECTOR HNSW …
                 FT.CREATE                  over Hash/JSON
Add              VADD key … element         HSET/JSON.SET the doc;
                                            the index follows
Query            VSIM key … COUNT n         FT.SEARCH idx … KNN n …
Engine           Redis 8                    Valkey 9 (+ module)
Cite             redis.io/…/vector-sets/    valkey.io/topics/search/

FT.CREATE myIndex SCHEMA embedding VECTOR HNSW 6 TYPE FLOAT32 DIM 3 DISTANCE_METRIC COSINE

The standalone examples in this module use the Redis 8 path (the source). The BCS stack uses Valkey 9; to reach vector search on Valkey, install and configure the valkey-search module, create an index with FT.CREATE, and query with FT.SEARCH … KNN. The underlying HNSW algorithm is the same; the commands and data model are not.

the pattern (Redis 8)

HNSW approximate-nearest-neighbour, automatic graph on VADD, O(log N) via VSIM — a vector is a native type.

on Valkey (the BCS engine)

The same HNSW algorithm via the valkey-search module: FT.CREATE … VECTOR HNSW defines the schema; FT.SEARCH … KNN queries it. A different API; the same O(log N) graph.

Redis 8 and Valkey reach the same HNSW algorithm by completely different surfaces. Neither codemojex nor EchoMQ uses vectors today; this module teaches the pattern from the Redis 8 source as a standalone example, with the Valkey path named in the engine note.

R7.05.2 · in this course

RAG and recommendations

The HNSW graph is the engine; the next dive covers the applications: RAG, semantic caching, recommendations, and classification. rag-and-recommendations.

R7.05.3 · in this course

Filtered queries

Attach JSON metadata to elements and filter by structured predicates alongside the vector query — hybrid search. filtered-queries.

§6

References

sources · related

Sources

Redis — Vector sets — the data-type overview: commands, HNSW parameters, quantization.
Redis — VADD — insert an element with its vector; quantization options and HNSW parameters.
Valkey — Search — the valkey-search module; FT.CREATE / FT.SEARCH VECTOR HNSW.
Valkey — FT.CREATE — define a VECTOR HNSW schema field on a Valkey search index.
Valkey — Introducing Valkey Search — module design rationale and setup.

Related in this course

R7.05 · Vectors hubthe module overview
R7.05.2 · RAG and recommendationsthe applications: RAG, semantic cache, recs, kNN classify
R7.05.3 · Filtered querieshybrid vector + predicate search
R7.03 · Probabilistic data structuresanother memory/accuracy trade-off structure