Chunk Relevancy Sort

2 min read

The service is integrated into the following spaces and modules:

• Unique AI Space
• Document Search V2
• Question Answerer
• Web Search Module

Functionality

The Chunk Relevancy Service is an optional post-processing service that improves the quality of search results by re-ranking text chunks retrieved from semantic vector search and/or combined search. Instead of relying solely on vector similarity and/or full-text search, it evaluates the actual relevance of each chunk to the user’s query using a language model.

Purpose

While vector search and fast-text search are fast and effective, they may overlook subtle context or nuanced details. The Chunk Relevancy Service enhances precision by analyzing each chunk in-depth, ensuring the most contextually relevant information is prioritized in the final response.

How It Works

1. Initial Chunk Retrieval
   A search query returns a list of top-ranked chunks from the vector search or a combined full-text and vector search.

2. Per-Chunk Relevance Evaluation
   Each chunk is then passed, alongside the search query, into a dedicated language model call. The model evaluates the chunk’s relevance in context and classifies it as:

   • High
   • Medium
   • Low

3. Final Re-ranking
   Based on the model-assigned relevance levels, the service reorders the chunks to ensure that the most important content appears first.

Why Use Chunk Relevancy Sorting?

• ✅ Increased Accuracy: Goes beyond token similarity and key-word search by evaluating actual semantic relevance.
• ✅ Detail-Aware: Captures subtle context and phrasing missed by embeddings alone.

Trade-offs and Performance Impact

• LLM Call per Chunk: Each chunk requires its own LLM call.
  For example, re-ranking the top 100 chunks results in 100 individual LLM calls.
• Latency: Additional processing time is introduced due to the sequential evaluation of chunks.
• Cost: LLM usage increases significantly with the number of chunks being evaluated.

Configuration

The ChunkRelevancySortConfig schema defines the settings for sorting data chunks based on relevancy.

Default Configuration

{
  "enabled": false,
  "relevancyLevelsToConsider": ["high","medium","low"],
  "relevancyLevelOrder": {
    "high": 0,
    "medium": 1,
    "low": 2
  },
  "languageModel": "AZURE_GPT_35_TURBO_0125",
  "fallbackLanguageModel": "AZURE_GPT_35_TURBO_0125",
  "additionalLlmOptions": {},
  "maxTasks": null
}

Fields Documentation

| Field Name | Description | Type | Default Value | | enabled | Whether to enable the chunk relevancy sort. | boolean | false | | relevancyLevelsToConsider | The relevancy levels to consider. | array | ['high', 'medium', 'low'] | | relevancyLevelOrder | The relevancy level order. | object | { 'high': 0, 'medium': 1, 'low': 2 } | | languageModel | The language model to use for the chunk relevancy sort. | string | AZURE_GPT_35_TURBO_0125 | | fallbackLanguageModel | The fallback language model to use for the chunk relevancy sort. | string | AZURE_GPT_35_TURBO_0125 | | additionalLlmOptions | Additional parameters given to the LLM | dict | {} | | maxTasks | The maximum number of parallel tasks to use for the chunk relevancy sort. | integer | null |

Dependencies

This table describes conditions where fields depend on other fields.