Python Indexing and Search

This guide continues from Conversation History and shows how to add search indexing to history entries and expose a search API for frontend applications.

New to indexing and search concepts? Read Indexing & Search first to understand indexedContent, redaction, and search types.

Prerequisites

Starting checkpoint: This guide starts from python/examples/langchain/doc-checkpoints/03-with-history

Make sure you’ve completed the Python Conversation History guide first. Also complete Step 2 in Python Dev Setup (build local memory-service-langchain wheel + UV_FIND_LINKS); this is temporary until the package is released.

How Search Indexing Works

Conversation history content is encrypted at rest, so text search needs a separate cleartext index field.

Checkpoint 07 configures MemoryServiceHistoryMiddleware with an indexed_content_provider. The provider transforms each history message into indexedContent before the entry is written.

This gives you a redaction point. You can remove sensitive data from indexedContent while still storing the full encrypted message content.

Add an Indexed Content Provider

The simplest provider is pass-through:

app.py 
def pass_through_indexed_content(text: str, role: str) -> str:
    del role
    return text

history_middleware = MemoryServiceHistoryMiddleware.from_env(
    indexed_content_provider=pass_through_indexed_content,
)

What changed: pass_through_indexed_content(text, role) is defined and passed to MemoryServiceHistoryMiddleware.from_env(indexed_content_provider=...). Without an indexed_content_provider, history entries are written with no indexedContent field and therefore cannot be searched.

Why: The indexed_content_provider hook is called before each entry is written, and whatever string it returns is stored as the cleartext indexedContent field that the search index reads. Returning the message text verbatim (pass-through) makes every message fully searchable. The del role line suppresses an unused-argument linting warning; in production code the role argument would be used to skip indexing AI responses or to apply different redaction rules per speaker.

Security warning indexedContent is not encrypted. Redact or minimize sensitive values before returning indexed text.

For production, replace pass-through with redaction logic. Example:

def redacting_indexed_content(text: str, role: str) -> str:
    del role
    return re.sub(r"\b\d{4}[- ]?\d{4}[- ]?\d{4}[- ]?\d{4}\b", "[REDACTED]", text)

Expose the Search API

Expose POST /v1/conversations/search so frontend apps can query across conversations:

app.py 
    return to_fastapi_response(response)

Why: The search query, search type (auto, semantic, fulltext, or a concrete type array like ["semantic","fulltext"]), and pagination options all live in the JSON request body, so the endpoint requires no query-parameter parsing — it just reads the body and passes it through. The Memory Service returns scored results with highlights, which the proxy relays back to the caller. Exposing the endpoint from the agent app rather than pointing the frontend directly at the Memory Service keeps all traffic behind a single origin and ensures the user’s Bearer token is validated in one place.

Make sure you define a shell function that can get the bearer token for the bob user:

function get-token() {
  curl -sSfX POST http://localhost:8081/realms/memory-service/protocol/openid-connect/token \
    -H "Content-Type: application/x-www-form-urlencoded" \
    -d "client_id=memory-service-client" \
    -d "client_secret=change-me" \
    -d "grant_type=password" \
    -d "username=bob" \
    -d "password=bob" \
    | jq -r '.access_token'
}

First, create searchable conversation content:

curl -NsSfX POST http://localhost:9090/chat/09637cfd-aaa6-4828-860c-34fa9081c060 \
  -H "Authorization: Bearer $(get-token)" \
  -H "Content-Type: text/plain" \
  -d "Give me a random number between 1 and 100."

Example output:

Sure, I can help with that.

Search conversations:

curl -sSfX POST http://localhost:9090/v1/conversations/search \
  -H "Authorization: Bearer $(get-token)" \
  -H "Content-Type: application/json" \
  -d '{"query": "random number", "searchType": "auto"}' | jq

Example output:

{
  "data": [
    {
      "conversationId": "09637cfd-aaa6-4828-860c-34fa9081c060"
    }
  ],
  "afterCursor": null
}

Completed Checkpoint

Completed code: View the full implementation at python/examples/langchain/doc-checkpoints/07-with-search

Next Steps

Continue to: