Indexing and Search

This guide continues from Conversation History and shows how to add search indexing to your conversation history entries and expose a search API for frontend applications.

💡 New to indexing and search concepts? Read Indexing & Search first to understand how search indexing, content redaction, and search types work. This guide focuses on the Quarkus implementation.

Prerequisites

Starting checkpoint: View the code from the previous section at java/quarkus/examples/doc-checkpoints/03-with-history

Make sure you’ve completed the Conversation History guide first. You should have:

• Conversation history recording with @RecordConversation
• Conversation APIs exposed via ConversationsResource
• Memory Service running via Docker Compose

How Search Indexing Works

When you record conversation history, message content is stored encrypted on disk. This is great for security, but it means the content can’t be searched directly.

To enable search, the Memory Service uses a separate indexedContent field on each history entry. This field stores a searchable (unencrypted) version of the message text. When an IndexedContentProvider bean is available, the history recorder automatically calls it to transform each message into indexed content before storing the entry.

This design gives your application a chance to redact sensitive information before it’s written to the search index in cleartext. For example, you might strip credit card numbers, social security numbers, or other PII from the indexed text while keeping the full content in the encrypted message.

Add an IndexedContentProvider

To enable search indexing, create a bean that implements the IndexedContentProvider interface. The simplest implementation passes text through unchanged:

package org.acme;

import io.github.chirino.memory.history.runtime.IndexedContentProvider;
import jakarta.enterprise.context.ApplicationScoped;

@ApplicationScoped
public class PassThroughIndexedContentProvider implements IndexedContentProvider {

    @Override
    public String getIndexedContent(String text, String role) {
        return text;
    }
}

What changed: Created PassThroughIndexedContentProvider, a CDI @ApplicationScoped bean that implements IndexedContentProvider and returns the message text as-is. Why: By providing even a pass-through implementation, every message recorded by @RecordConversation will have its text placed in the cleartext search index. This single class activates the entire search pipeline with no other changes required.

Security warning indexedContent is not encrypted. Redact or minimize sensitive values before returning indexed text.

The IndexedContentProvider interface has a single method:

• getIndexedContent(String text, String role) — Transforms message text into content for the search index. The role parameter is either "USER" or "AI". Return null to skip indexing for that message.

Custom Redaction

For production applications, you’ll likely want to redact sensitive information. Start from the same provider method and replace the return line with: text.replaceAll("\\b\\d{4}[- ]?\\d{4}[- ]?\\d{4}[- ]?\\d{4}\\b", "[REDACTED]")

    @Override
    public String getIndexedContent(String text, String role) {
        return text;
    }

You can also return null to skip indexing entirely for certain messages — for example, you might choose not to index AI responses:

if ("AI".equals(role)) { return null; }

Expose the Search API

To let the frontend search across conversations, add a search endpoint to your ConversationsResource:

    @POST
    @Path("/search")
    @Consumes(MediaType.APPLICATION_JSON)
    @Produces(MediaType.APPLICATION_JSON)
    public Response searchConversations(String body) {
        return proxy.searchConversations(body);
    }

Why: Exposing search through the Quarkus proxy rather than directly to clients ensures the Memory Service enforces that results are scoped to conversations the authenticated user has access to. The raw JSON forwarding keeps the endpoint simple while still letting callers control search type, grouping, and result limits via the request body.

This endpoint accepts a JSON request body with the following fields:

• query (required) — The search query text
• searchType — Either a single type ("auto", "semantic", "fulltext") or an array of concrete types (["semantic","fulltext"])
• limit — Maximum number of results per requested search type (default 20)
• groupByConversation — Group results by conversation (default true)
• includeEntry — Include the full entry in results (default true)

If you request an unavailable concrete type (for example "semantic" with no vector backend configured), Memory Service returns 501 search_type_unavailable.

function get-token() {
  curl -sSfX POST http://localhost:8081/realms/memory-service/protocol/openid-connect/token \
    -H "Content-Type: application/x-www-form-urlencoded" \
    -d "client_id=memory-service-client" \
    -d "client_secret=change-me" \
    -d "grant_type=password" \
    -d "username=bob" \
    -d "password=bob" \
    | jq -r '.access_token'
}

curl -NsSfX POST http://localhost:9090/chat/415afe24-61a5-49b0-99fb-fbb4a58308cc \
  -H "Content-Type: text/plain" \
  -H "Authorization: Bearer $(get-token)" \
  -d "Give me a random number between 1 and 100."

42

Now search for it:

curl -sSfX POST http://localhost:9090/v1/conversations/search \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $(get-token)" \
  -d '{"query": "random number"}' | jq

Example response:

{
  "data": [
    {
      "conversationId": "415afe24-61a5-49b0-99fb-fbb4a58308cc",
      "conversationTitle": "Give me a random number between 1 and 100",
      "entryId": "38ad11b2-493d-4eb0-885c-43971950b9b4",
      "score": 0.95,
      "highlights": ["Give me a ==random number== between 1 and 100"],
      "entry": {
        "id": "38ad11b2-493d-4eb0-885c-43971950b9b4",
        "conversationId": "415afe24-61a5-49b0-99fb-fbb4a58308cc",
        "userId": "bob",
        "channel": "history",
        "contentType": "history",
        "content": [{ "role": "USER", "text": "Give me a random number between 1 and 100." }],
        "createdAt": "2025-01-10T14:32:05Z"
      }
    }
  ],
  "afterCursor": null
}

Search results include:

• conversationId and conversationTitle — For linking to the conversation
• entryId — For deep-linking to a specific message
• score — Relevance score
• highlights — Matched text with ==highlight== markers
• entry — The full entry content (when includeEntry is true)

Completed Checkpoint

Completed code: View the full implementation at java/quarkus/examples/doc-checkpoints/07-with-search

Next Steps

Continue to:

• Conversation Forking — Branch conversations to explore alternative paths
• Response Recording and Resumption — Streaming responses with resume and cancel support