Full-text search overview

Full-text search ranks documents by keyword and phrase relevance using BM25 scoring, with optional Lucene query syntax. Because an index with a document schema can also declare dense and sparse vector fields, the same index can rank by semantic or sparse-vector similarity, so one index can cover keyword and semantic search.

Reach for full-text search when exact words, phrases, names, codes, or identifiers matter, not just semantic similarity. For semantic-only or vector-only workloads, an index with dense vectors (the Vectors API) is simpler. See Search overview to choose the right approach.

What full-text search does and doesn't match:

Pinecone's Documents API stores typed fields you declare in a schema. End to end, the flow is short: create an index with a schema that declares your ranking fields, upsert your data as JSON documents, then search by choosing one ranking signal per search request with score_by. The end-to-end example below stitches all three steps into one runnable script.

1. You upsert data as JSON documents.
2. You declare how each field should be indexed via a schema, as a string field with full_text_search enabled (BM25 scoring), a dense_vector field, or a sparse_vector field. The schema is for ranking fields only; metadata fields aren't declared.
3. Pinecone indexes each field's content according to the type of the field declared in the schema. Any other fields on the upserted documents are automatically stored and indexed for filtering, no schema declaration required.

For the field types you can declare, see Schema field types. Filterable metadata isn't part of the schema. Any field you upsert that's not declared in the schema is stored on the document, returned via include_fields, and automatically indexed for filtering, see Metadata fields.

Newly upserted documents are indexed asynchronously, so they may not be searchable immediately. To confirm that an index holds the documents you expect, see Check data freshness.

Every search ranks by one scoring type. The score_by clause selects the scoring method for the request:

• text, BM25 token matching over one or more FTS-enabled string fields.
• query_string, Lucene query syntax across one or more FTS-enabled string fields, including cross-field boolean queries.
• dense_vector, vector similarity against a dense_vector field.
• sparse_vector, sparse-vector similarity against a sparse_vector field.

The same index can support all four when the schema declares the corresponding fields, but a given request commits to one scoring type. To narrow the candidates a vector ranking sees, combine the score_by with a metadata filter, including the text-match operators $match_phrase, $match_all, and $match_any on FTS-enabled string fields, plus the standard logical and comparison operators ($and, $or, $not, $exists, etc.). The filter narrows what's eligible, and the score_by ranks what remains. This is the most common hybrid pattern. The alternative is to run separate searches and merge the results client-side, for example with reciprocal rank fusion.

For example, on an index whose schema declares both a dense_vector field (review_embedding) and an FTS-enabled string field (review_text), this single request runs semantic search across the corpus but only over documents whose review_text contains the exact phrase "beautifully written":

index.documents.search(
    namespace="reviews",
    top_k=5,
    score_by=[
        {
            "type": "dense_vector",
            "fields": ["review_embedding"],
            "values": query_embedding,
        }
    ],
    filter={"review_text": {"$match_phrase": "beautifully written"}},
)

The dense ranking still controls the order of results; the text-match filter just narrows what's eligible to be ranked.

BM25 can also rank on several fields at once. Name multiple fields in one text clause, or pass multiple text clauses, and Pinecone combines them into a single ranking. A query_string clause can target several fields too. Every field contributes equally in 2026-07, and there's no per-field weight parameter.

A complete run from index creation through search. Copy this into a single file, set PINECONE_API_KEY, and run.

import os
import time
from pinecone import Pinecone, SchemaBuilder

pc = Pinecone(api_key=os.environ["PINECONE_API_KEY"])

# 1. Create an index. Setting `full_text_search` on a `string` field enables BM25.
schema = (
    SchemaBuilder()
      .add_string_field(name="title", full_text_search={"language": "en"})
      .add_string_field(name="body", full_text_search={"language": "en"})
      .build()
)
pc.indexes.create(name="articles-quickstart", schema=schema)

# Wait for the index to be ready before upserting (no timeout — add a loop cap in production).
while not pc.indexes.describe(name="articles-quickstart").status.ready:
    time.sleep(2)

index = pc.Index(name="articles-quickstart")

# 2. Upsert documents. Fields declared in the schema (`title`, `body`) are
#    BM25-indexed; any extra fields (`category`, `year`) are stored on the
#    document and auto-indexed for filtering as metadata.
index.documents.upsert(
    namespace="example-namespace",
    documents=[
        {
            "_id": "doc1",
            "title": "Machine learning in 2024",
            "body": "Machine learning models are revolutionizing natural language processing",
            "category": "technology",
            "year": 2024,
        },
        {
            "_id": "doc2",
            "title": "Vector databases",
            "body": "Vector databases enable fast similarity search across embeddings",
            "category": "technology",
            "year": 2023,
        },
        {
            "_id": "doc3",
            "title": "Quantum computing",
            "body": "Quantum computers leverage superposition for faster computation",
            "category": "science",
            "year": 2024,
        },
    ],
)

# 3. Search with BM25 ranking on `body`, narrowed by a metadata filter and a
#    phrase-match text filter on the FTS field. The relevance score comes back
#    as `_score` (not `score`).
response = index.documents.search(
    namespace="example-namespace",
    top_k=5,
    score_by=[
        {
            "type": "text",
            "fields": ["body"],
            "query": "machine learning",
        }
    ],
    filter={
        "year": {"$gte": 2024},
        "body": {"$match_phrase": "natural language"},
    },
    include_fields=["title", "body", "category", "year"],
)

for match in response.matches:
    print(match._id, match._score, getattr(match, "title", ""))

What each piece does:

• SchemaBuilder().add_string_field(..., full_text_search={"language": "en"}) declares a BM25-indexed text field. Without full_text_search, the string field would be rejected at index creation — schemas only declare ranking fields.
• index.documents.upsert(...) writes plain JSON documents. Schema fields are validated; non-schema fields (category, year here) are stored and auto-indexed for filtering. For large datasets, use Import instead.
• score_by=[{"type": "text", ...}] picks BM25 as the scoring type. One scoring type per request; combine scoring with text matching via filter rather than mixing scoring methods.
• filter narrows candidates before ranking. Standard operators ($eq, $gte, etc.) apply to any metadata field; the text-match operators ($match_phrase, $match_all, $match_any) only apply to FTS-enabled string fields.
• _score is the system-owned relevance score. A user metadata field named score would be returned alongside, untouched.

Filters are deterministic — each document either matches or it doesn't — and they apply before scoring. Scoring methods (text/BM25, query_string/Lucene, dense_vector, sparse_vector) order whatever remains after filtering, and only the top top_k hits are returned (max 10,000).

When you're combining text matching with vector ranking, start with the hard yes/no constraints as filters (including the text-match operators $match_phrase, $match_all, $match_any on FTS-enabled string fields), then pick a score_by method to rank whatever remains. Use BM25 (score_by text or query_string) when keyword and phrase ranking order matters, not just inclusion.

The schema is required at index creation and declares the fields that drive ranking or vector search. Filterable metadata isn't declared in the schema: any field you upsert that isn't in the schema is automatically stored and indexed for filtering.

A schema can declare up to 100 string fields with full_text_search enabled, but at most one dense_vector field and at most one sparse_vector field per index.

Field names must be unique, non-empty strings, and must not start with _ or $. The _ prefix is reserved for system-managed fields (for example, _id, _score); $ is reserved for filter operators. Field names are also limited to 64 bytes. Every document has a required _id field, which carries its unique identifier. A user metadata field named score is allowed, and match scores are returned as _score to avoid collisions.

A text-only schema. The minimal {} config enables FTS with all defaults; sub-fields like language, stemming, and stop_words are optional overrides:

{
  "name": "articles",
  "deployment": {
    "deployment_type": "managed",
    "cloud": "aws",
    "region": "us-east-1"
  },
  "schema": {
    "fields": {
      "title": {
        "type": "string",
        "full_text_search": { "language": "en" }
      },
      "body": {
        "type": "string",
        "description": "The main body text of the article",
        "full_text_search": {
          "language": "en",
          "stemming": true,
          "stop_words": true
        }
      }
    }
  }
}

A multi-field schema with text, dense, and sparse vectors:

{
  "name": "articles-hybrid",
  "deployment": {
    "deployment_type": "managed",
    "cloud": "aws",
    "region": "us-east-1"
  },
  "schema": {
    "fields": {
      "title": {
        "type": "string",
        "full_text_search": { "language": "en" }
      },
      "body": {
        "type": "string",
        "full_text_search": { "language": "en" }
      },
      "embedding": {
        "type": "dense_vector",
        "dimension": 1536,
        "metric": "cosine"
      },
      "sparse_embedding": {
        "type": "sparse_vector"
      }
    }
  }
}

Documents upserted into either schema can carry additional fields, for example, category (string), tags (array of strings), year (number), or in_stock (boolean). These fields are stored on the document, returned via include_fields, and automatically indexed for filtering. They don't need to be declared in the schema.

Metadata fields are not declared in the schema. Any field you include on an upserted document that's not declared in the schema is treated as metadata: it's stored on the document, returned via include_fields, and automatically indexed for filtering with the standard operators ($eq, $ne, $gt, $gte, $lt, $lte, $in, $nin, $exists, $and, $or, $not).

Metadata field types are inferred from the values you upsert: strings, numbers (stored as floating point), booleans, and arrays of strings are all supported. You can mix metadata field types across documents in the same index.

Because metadata isn't declared anywhere, track the field names and types you use in your application.

Documents are validated against the index schema on upsert. If any document is invalid, the entire upsert fails and nothing is written. For the validation rules, see Schema validation.

Filters are applied before the search runs, so the search only considers documents that match. On document indexes, a filter can use the comparison and set operators ($eq, $ne, $gt, $gte, $lt, $lte, $in, $nin, $exists), the logical operators $and, $or, and $not, and the text-match operators ($match_phrase, $match_all, $match_any) on FTS-enabled string fields. Multiple fields at the top level of a filter object combine with implicit AND. For operator details and examples, see Filter by metadata.

These examples combine score_by scoring with filter narrowing on a document index.

curl -X POST "https://articles-abc123.svc.us-east-1.pinecone.io/namespaces/__default__/documents/search" \
  -H "Api-Key: {{YOUR_API_KEY}}" \
  -H "Content-Type: application/json" \
  -H "X-Pinecone-Api-Version: 2026-07" \
  -d '{
    "include_fields": ["title", "body", "category", "year"],
    "filter": {
      "category": { "$eq": "technology" },
      "year": { "$gte": 2024 }
    },
    "score_by": [{
      "type": "text",
      "fields": ["body"],
      "query": "machine learning"
    }],
    "top_k": 10
  }'

curl -X POST "https://articles-abc123.svc.us-east-1.pinecone.io/namespaces/__default__/documents/search" \
  -H "Api-Key: {{YOUR_API_KEY}}" \
  -H "Content-Type: application/json" \
  -H "X-Pinecone-Api-Version: 2026-07" \
  -d '{
    "include_fields": ["title", "body"],
    "score_by": [{
      "type": "query_string",
      "query": "title:(quantum) OR body:(machine learning)"
    }],
    "top_k": 10
  }'

curl -X POST "https://articles-abc123.svc.us-east-1.pinecone.io/namespaces/__default__/documents/search" \
  -H "Api-Key: {{YOUR_API_KEY}}" \
  -H "Content-Type: application/json" \
  -H "X-Pinecone-Api-Version: 2026-07" \
  -d '{
    "include_fields": ["title", "body"],
    "filter": { "body": { "$match_phrase": "machine learning" } },
    "score_by": [{
      "type": "dense_vector",
      "fields": ["embedding"],
      "values": [0.12, 0.34, 0.56]
    }],
    "top_k": 10
  }'

curl -X POST "https://articles-abc123.svc.us-east-1.pinecone.io/namespaces/__default__/documents/search" \
  -H "Api-Key: {{YOUR_API_KEY}}" \
  -H "Content-Type: application/json" \
  -H "X-Pinecone-Api-Version: 2026-07" \
  -d '{
    "include_fields": ["body", "category", "year"],
    "filter": {
      "$and": [
        { "body": { "$match_all": "federal reserve" } },
        { "category": { "$eq": "finance" } }
      ]
    },
    "score_by": [{
      "type": "text",
      "fields": ["body"],
      "query": "monetary policy impact"
    }],
    "top_k": 10
  }'

This restricts the candidate set to finance articles whose body contains both "federal" and "reserve", then ranks those candidates by BM25 score against "monetary policy impact".

curl -X POST "https://articles-abc123.svc.us-east-1.pinecone.io/namespaces/__default__/documents/search" \
  -H "Api-Key: {{YOUR_API_KEY}}" \
  -H "Content-Type: application/json" \
  -H "X-Pinecone-Api-Version: 2026-07" \
  -d '{
    "include_fields": ["body", "category"],
    "filter": {
      "$and": [
        { "body": { "$match_phrase": "large language model" } },
        { "body": { "$not": { "$match_any": "spam advertisement" } } }
      ]
    },
    "score_by": [{
      "type": "text",
      "fields": ["body"],
      "query": "recent advances in generative AI"
    }],
    "top_k": 10
  }'

This requires the exact phrase "large language model" and excludes documents containing "spam" or "advertisement".

For the full request and response schema, see Search documents.

Everything in this section applies to indexes with document schemas. For dense, sparse, and integrated-inference indexes, see Search overview.

• Entry points: Use the REST API, the Python SDK (v10 or later), or the Pinecone console. REST requests must send X-Pinecone-Api-Version: 2026-07, and the SDK sends it for you.
• Endpoint compatibility: Indexes with document schemas use the /namespaces/{namespace}/documents/* endpoints. Dense, sparse, and integrated-inference indexes continue to use /vectors/*, and /records/* for integrated inference. The two endpoint families are index-type-specific and don't cross over.
• Deployment: Only managed (serverless) deployments are supported, with read_capacity.mode set to OnDemand or Dedicated.

• Document and schema limits: See Upsert limits for fields per schema, field and token size, document size, request size, and metadata size.
• Query limits: See Query limits for the top_k and result-size caps. Full-text search is optimized for ranked retrieval rather than for aggregation or count-style queries.
• Update limits: See Update limits for how many documents one update request can patch.

• Partial updates: Upsert replaces the whole document for a given _id. POST /namespaces/{namespace}/documents/update patches individual fields, by ID or by metadata filter. See Update documents.
• Bulk import from object storage, via JSONL files. See Prepare document-schema files (JSONL).
• Namespace operations: You can create, list, describe, and delete namespaces on an index with a document schema, and describing a namespace returns its record_count and size_bytes.
• describe_index_stats: In the REST response, totalVectorCount and each namespace's vectorCount count documents, and both are accurate for every schema. dimension and metric describe the schema's dense_vector field, and vectorType is dense. When a schema declares no dense_vector field, metric falls back to dotproduct, and dimension is 0 on a text-only schema (vectorType is text) or is omitted when the schema declares a sparse_vector field (vectorType is sparse). For the full response schema, see Get index stats.

• Schema changes after index creation aren't yet supported, including backfilling a schema onto an index created before 2026-07. Create a new index with the schema you want and reindex your documents.
• Backup and restore aren't yet supported.
• Indexes can't be created in projects with customer-managed encryption keys (CMEK) enabled.
• An index can't be changed from dedicated read capacity back to on-demand. Create a new on-demand index and reingest your data.

Reads and writes on indexes with document schemas are metered using the same read units (RUs) and write units (WUs) model as vector indexes.