Working with Metadata

1. Defining Metadata Schema

Field	Type	Example
title	string	`"Vector DBs 101"`
tags	list[string]	`["ml", "db"]`
score	number	`0.95`
published	boolean	`true`
timestamp	number (unix)	`1716067200`

Note: Pinecone doesn't enforce a schema. Enforce one in your application layer for filter consistency.

2. Adding Metadata to Vectors

Example: With Metadata

index.upsert(vectors=[{
    "id": "doc1",
    "values": [...],
    "metadata": {"source": "wiki", "page": 12, "tags": ["ml"]},
}])

3. Updating Existing Metadata

Example: Partial Update

index.update(id="doc1", set_metadata={"views": 42})

Op	Effect
`set_metadata`	Merge (overwrite per-key)
Upsert	Full replace

4. Validating Metadata Types

Example: Validation

ALLOWED = (str, int, float, bool)
def valid(md):
    for k, v in md.items():
        if isinstance(v, list):
            if not all(isinstance(x, str) for x in v): return False
        elif not isinstance(v, ALLOWED): return False
    return True

5. Understanding Metadata Size Limits

Limit	Value
Per vector	40 KB total
List items	Counted in total size
Nested objects	Not supported

6. Organizing Data with Metadata

Pattern	Use
Categorical filters	category, language, status
Temporal filters	created_at, expires_at
Access control	owner_id, visibility
Source tracking	doc_id, chunk_index

7. Implementing Version Tracking

Example: Version Field

index.upsert(vectors=[{
    "id": "doc1",
    "values": [...],
    "metadata": {"version": 3, "embedding_model": "text-embedding-3-small"},
}])

8. Adding Timestamps

Format	Recommendation
Unix epoch (int)	Best for range filters
ISO 8601 string	Human-readable; no range queries

9. Storing Document References

Example: Doc References

{
    "id": "chunk_abc_3",
    "metadata": {
        "doc_id": "abc",
        "chunk_index": 3,
        "source_url": "https://example.com/abc",
        "text_snippet": "...",
    },
}

10. Handling Complex Metadata Structures

Warning: Nested objects are not supported. Flatten with dot notation or store JSON-as-string (no filtering).

Strategy	Example
Flatten keys	`author.name` → `author_name`
Stringify JSON	`raw_json: "{...}"` (not filterable)
External lookup	Store doc ID; fetch full record from DB