Filtering with Metadata
1. Using Equality Filters ($eq operator)
Example: $eq
res = index.query(vector=qv, top_k=5,
filter={"source": {"$eq": "wiki"}})
# Shorthand:
filter={"source": "wiki"}
| Use | Note |
|---|---|
| Categorical match | Exact equality only |
| Type-sensitive | Number vs string distinct |
2. Using Inequality Filters ($ne operator)
| Operator | Effect |
|---|---|
$ne | Excludes vectors where field equals value |
3. Using Greater Than Filters ($gt, $gte)
| Op | Meaning |
|---|---|
$gt | Strictly greater |
$gte | Greater or equal |
4. Using Less Than Filters ($lt, $lte)
| Op | Meaning |
|---|---|
$lt | Strictly less |
$lte | Less or equal |
5. Using In Filters ($in operator)
6. Using Not In Filters ($nin operator)
| Op | Notes |
|---|---|
$in | Field matches any value in list |
$nin | Field matches none |
7. Combining Filters with AND ($and operator)
Example: $and
filter={"$and": [
{"published": True},
{"views": {"$gt": 100}},
{"language": "en"},
]}
# Implicit AND (default for multiple keys):
filter={"published": True, "language": "en"}
8. Combining Filters with OR ($or operator)
9. Filtering Array Fields ($in with arrays)
Example: Array Field
# metadata: {tags: ["ml", "vectors", "db"]}
filter={"tags": {"$in": ["ml", "ai"]}} # matches if any overlap
| Pattern | Match Rule |
|---|---|
$in on list field | Match if array contains any listed value |
$eq on list field | Match if array contains exactly the value |
10. Understanding Indexed Metadata Performance
| Field | Performance |
|---|---|
| Indexed (default serverless) | Fast O(log n) filter |
| Non-indexed (pod selective) | Returned but not filterable |
| High-cardinality strings | Slower filter; large index |
11. Optimizing Filter Performance
| Tip | Effect |
|---|---|
| Most-selective first | Reduces candidate set early |
| Avoid wide $in lists | 10s of values OK; 1000s slow |
| Use booleans/enums | Faster than strings |
| Pre-aggregate ranges | e.g., decade bucket vs raw year |
12. Handling Complex Filter Queries
Example: Nested $and / $or
filter={"$and": [
{"published": True},
{"$or": [
{"featured": True},
{"views": {"$gte": 1000}},
]},
]}
Warning: Deeply nested filters with high cardinality may slow queries. Test with realistic data volumes.