R7.05.3 · dive 3
Filtered queries.
Attach JSON metadata to vector elements and query by both vector similarity and structured predicates in one step — hybrid search.
A VSIM FILTER call finds the nearest neighbours that also match a
predicate. No post-processing, no two-stage pipeline. Select an operator family below; the readout
shows an example expression of that type.
VSIM key ... FILTER '.year >= 2000 and .genre == "scifi"'
uses comparisons and logic together.Attaching metadata
filtered-queries.md.txt · Attaching MetadataUse the SETATTR option on VADD to attach a JSON object to each element at
insert time. The object can hold any JSON-serialisable values — numbers, strings, booleans, nested
objects:
# Insert three movies with metadata
VADD movies VALUES 128 <embedding> "inception" SETATTR '{"year": 2010, "genre": "scifi", "rating": 8.8}'
VADD movies VALUES 128 <embedding> "arrival" SETATTR '{"year": 2016, "genre": "scifi", "rating": 7.9}'
VADD movies VALUES 128 <embedding> "parasite" SETATTR '{"year": 2019, "genre": "drama", "rating": 8.5}'
Attributes on an element already in the set can be updated at any time with VSETATTR —
the HNSW graph entry stays in place, only the metadata changes. This makes it practical to refresh values
such as in_stock or price without touching the vector.
Elements with missing fields or malformed JSON are silently excluded from any
FILTER result that references those fields. No error surfaces; the element does not
appear in the result set. This means a schema migration — adding a new field to some elements but
not yet others — does not break running queries; elements without the field are invisible to any
predicate that mentions it.
SETATTR binds a JSON metadata object to the vector at insert time; VSETATTR
refreshes it later without touching the HNSW graph. Elements missing a referenced field are silently
excluded from filtered results.
The FILTER expression
filtered-queries.md.txt · The FILTER ExpressionPass FILTER '<expr>' to VSIM to restrict results to elements whose
metadata matches the predicate. The expression is evaluated per-candidate during graph traversal —
it does not execute as a separate post-filter pass:
# Find the 10 science-fiction films from 2000 or later most similar to the query
VSIM movies VALUES 128 <embedding> FILTER '.year >= 2000 and .genre == "scifi"' COUNT 10
| Operator type | Examples |
|---|---|
| Comparisons | .year >= 1980, .rating < 9.0, .year == 2010, .genre != "horror" |
| Logic | and / or / not (also && / || / !) |
| Arithmetic | .budget / 1000000 > 100 |
| Containment | .director in ["Spielberg", "Nolan"] |
| Selectors | .field accesses any JSON attribute |
Examples combining families:
.year >= 1980 and .year < 1990 .genre == "action" and .rating > 8.0 .director in ["Spielberg", "Nolan"] (.budget / 1000000) > 100 and .rating > 7
The FILTER grammar evaluates a predicate over each candidate’s JSON attributes
during traversal. Combine all five operator families freely in one expression string.
Filter effort (FILTER-EF)
filtered-queries.md.txt · Filter EffortBy default, a filtered VSIM explores COUNT × 100 candidates before
returning results. For highly selective predicates — where only a small fraction of elements pass
— this budget may not yield enough passing candidates to fill the requested COUNT. In
that case, increase the effort:
# A rare predicate: increase exploration to 5 000 candidates
VSIM key VALUES 128 <embedding> FILTER '.rare_field == 1' FILTER-EF 5000
Setting FILTER-EF 0 removes the candidate ceiling: the search continues until
COUNT passing elements are found, potentially scanning the entire index. The trade-off is
linear: higher FILTER-EF improves recall for selective predicates at the cost of increased
latency. The default (COUNT * 100) is adequate when a reasonable fraction of elements pass
the filter.
A practical heuristic: measure the selectivity of the predicate (what fraction of elements pass) and
set FILTER-EF so the expected number of passing candidates is at least COUNT * 2.
For a predicate that passes 1% of elements with COUNT 10, a FILTER-EF of 2000
gives a reasonable margin.
FILTER-EF controls the candidate budget. Raise it when the predicate is highly
selective and results come back short. FILTER-EF 0 scans until satisfied.
Notes on ValkeyThe BCS stack runs Valkey 9, which
does not have VADD, VSIM, or FILTER as native commands. The
valkey-search module reaches the same hybrid-search idea via
FT.SEARCH with numeric and tag filter expressions — for example,
FT.SEARCH idx "@genre:{scifi} @year:[2000 +inf]=>[KNN 10 @embedding $vec]". The
filter grammar and the FILTER-EF tuning knob are specific to Redis 8 Vector Sets; the
valkey-search path uses its own syntax and index schema instead. Same hybrid-search idea;
different API surface. Cite:
valkey.io/topics/search ·
valkey.io/commands/ft.create.
Hybrid search
filtered-queries.md.txt · Hybrid SearchA filtered vector query is hybrid search: a vector neighbourhood AND a structured predicate in a single operation. No post-processing join, no two-stage pipeline:
# 10 science-fiction films from 2000 or later, most similar to the query vector
VSIM movies VALUES 128 <embedding> FILTER '.year >= 2000 and .genre == "scifi"' COUNT 10 WITHSCORES
This finds the 10 movies most similar to the query vector that are also science fiction released after 2000. Traditional keyword search handles the predicate; vector search handles the similarity; hybrid search handles both at once. The HNSW graph traversal and the metadata predicate evaluate together during graph exploration — candidates that fail the predicate are bypassed rather than retrieved and then discarded.
.genre == "scifi" and
.year >= 2000. Select a movie to see whether it passes or is excluded, and why..genre == "scifi" and .year >= 2000. Inception and Arrival pass; Parasite,
Gladiator, and Metropolis are excluded. The two passing entries appear in the result ranked by similarity
score (0.92 then 0.88).Hybrid search: VSIM FILTER combines vector similarity and a structured predicate in one
traversal. No separate post-filter pass; HNSW candidates that fail the predicate are bypassed during
graph exploration.
The BCS stack uses Valkey 9. The valkey-search module reaches hybrid
search via FT.SEARCH with numeric and tag filter clauses combined with a
KNN vector query in the same command. Define the schema with FT.CREATE;
query with the combined syntax. No FILTER keyword; different but equivalent.
Hybrid search unifies vector similarity and predicate filtering in a single
VSIM FILTER call. The predicate is applied during HNSW traversal; only passing candidates
count toward the COUNT result. For selective predicates, tune FILTER-EF so the
search explores enough candidates to satisfy the count.
The HNSW graph
The graph structure, core commands, quantization trade-offs, and the engine note (Redis 8 native
vs valkey-search module).
the-hnsw-graph.
RAG and recommendations
The five-step RAG pipeline, semantic caching for 40–60% LLM cost reduction, and
recommendations via VSIM ELE.
rag-and-recommendations.
References
sources · relatedSources
- Redis — Vector sets — the FILTER section and SETATTR reference.
- Redis — VADD — SETATTR option at insert time; VSETATTR for later updates.
- Valkey — Search — the Valkey path to hybrid search via
FT.SEARCHand filter expressions. - Valkey — FT.CREATE — defining VECTOR schema fields on Valkey-search.
Related in this course
- R7.05 · Vectors hubthe module overview
- R7.05.1 · The HNSW graphcommands, graph structure, quantization, engine note
- R7.05.2 · RAG and recommendationsRAG pipeline, semantic cache, recommendations