Implementing Advanced Query Patterns
1. Implementing Multi-Vector Queries
Example: Multi-Vector Merge
# Run N queries and merge results
queries = [embed(t) for t in ["A", "B", "C"]]
all_hits = {}
for qv in queries:
for m in index.query(vector=qv, top_k=10)["matches"]:
prev = all_hits.get(m["id"], 0)
all_hits[m["id"]] = max(prev, m["score"])
top = sorted(all_hits.items(), key=lambda x: -x[1])[:10]
2. Using Query-Time Boosting
Example: Recency Boost
import math, time
now = time.time()
for m in res["matches"]:
age_days = (now - m["metadata"]["ts"]) / 86400
m["boosted"] = m["score"] + 0.1 * math.exp(-age_days / 30)
res["matches"].sort(key=lambda m: -m["boosted"])
3. Implementing Negative Filtering
| Pattern | Use |
|---|---|
| $nin on ID/category | Exclude items |
| $ne flags | Exclude states |
4. Combining Multiple Filters
Example: Nested Combinator
filter={"$and": [
{"in_stock": True},
{"$or": [{"featured": True}, {"sales": {"$gte": 100}}]},
{"category": {"$in": ["tech", "books"]}},
]}
5. Implementing Geo-Filtering
Example: Bounding Box
filter={"$and": [
{"lat": {"$gte": 37.7, "$lte": 37.9}},
{"lng": {"$gte": -122.5, "$lte": -122.3}},
]}
Note: Pinecone has no native geo type. Use bounding-box filters or geohash strings; refine distance client-side.
6. Using Time-Based Filtering
7. Implementing Pagination
Example: Offset via Score Cursor
# Pinecone has no offset; paginate with score cursor
def page(qv, max_score=None, size=20):
f = {"score": {"$lt": max_score}} if max_score else {}
res = index.query(vector=qv, top_k=size, filter=f or None,
include_metadata=True)
return res["matches"], res["matches"][-1]["score"] if res["matches"] else None
Warning: True OFFSET pagination is not supported. Use score-based cursor, or fetch larger top_k and slice.
8. Diversifying Results
Example: MMR Diversification
def mmr(query_vec, candidates, k=5, lam=0.5):
selected, remaining = [], candidates[:]
while len(selected) < k and remaining:
best, best_score = None, -1
for c in remaining:
sim_q = cos(query_vec, c["values"])
sim_sel = max((cos(c["values"], s["values"])
for s in selected), default=0)
score = lam * sim_q - (1 - lam) * sim_sel
if score > best_score: best, best_score = c, score
selected.append(best); remaining.remove(best)
return selected
9. Implementing Fallback Strategies
| Trigger | Fallback |
|---|---|
| Empty results | Widen filter or drop filter |
| Low scores | Switch to hybrid / lexical |
| Timeout | Cached / static response |
10. Optimizing Complex Queries
| Tip | Effect |
|---|---|
| Reduce filter cardinality | Bucket numbers, normalize strings |
| Run queries in parallel | Async / threads |
| Cache hot queries | Redis / in-process |