Web Search Module

The Web Search module enables real-time access to internet information through a chat interface. By integrating live web searches directly into conversations, users receive accurate and up-to-date answers to their questions, helping them stay informed, make better decisions, and solve problems using the latest available data.

Functionality

This module extends the language model with web search capabilities, allowing it to respond to queries that exceed its built-in knowledge, such as current events or other time-sensitive topics.

When a user submits a question, the model follows this decision flow:

• Use of Built-in Knowledge:

The model first checks whether it can answer using its internal knowledge. If the information is sufficient, it responds directly—no web search or external sources are used.

• Triggering a Web Search:

If the model determines that up-to-date or external information is needed, it automatically:

• Generate a Search Query

The module creates a search query based on the full conversation history, optimized for effective use with search engines.

• Perform a Web Search

It uses the configured search engine (e.g., Google or Bing) to retrieve relevant web pages.

• Extract Web Page Content

The HTML content of the top-ranked search results is collected and processed.

• Split the Content

Extracted text is divided into smaller chunks to make it easier to search and understand

• [Optional] Run Similarity Search

If enabled, the text chunks are embedded and compared with the generated query to identify the most relevant information.

• Answer the User

The most relevant content is included in the context, allowing the model to generate a well-informed response to the user’s question.

Supported Search Engines

Unique supports the Google Custom Search API engine. Details on configuration, as well as security and privacy considerations, can be found here: Web Search Administration

Google allows you to restrict search results to specific domains. This means users can configure a list of trusted sources, ensuring the search engine only returns results from those domains. For more details, see below in the configuration section.

FAQ

1. When I set-up the WebSearch Module in a space, should I also include the Chat with GPT?

It is not needed to include Chat with GPT as an additional Module. The WebSearch has two “modes”: either the models decides if it can answer the question based on the model itself (eg. some translations) or needs to go to the Web for more details OR it always tries to go to the Web to try to find the answers.

2. Can I also have access to Sites behind a paywall or that need a login?

No it is not possible to access sites that are behind a paywall and need a login to read the content?

3. Does the module only retrieve the static part of the pages. Content coming from dynamic scripts cannot be retrieved.

Yes

Tool Definition

The tool definition is used when you want to combine multiple modules in the same space.

{
  "type": "function",
  "function": {
    "name": "WebSearch",
    "parameters": {
      "type": "object",
      "required": [
        "query"
      ],
      "properties": {
        "query": {
          "type": "string",
          "description": "User's query that needs to be used to retrieve information from the web using a search engine."
        }
      }
    },
    "description": "Useful to answer user's query based on information from the web."
  }
}

Configuration settings

The WebSearchConfig schema is designed to configure various aspects of a web-based question answering system. This configuration includes templates for system prompts, trigger prompts, and settings related to search engine operations. The schema provides a structured approach to define how the system interacts with users and processes queries using web search capabilities. It includes settings for token limits, interaction history, and various configurations for search engine components.