# Web Search Infrastructure

## Architecture Overview

The diagram above illustrates how **Unique AI** interacts with an external **web search engine**. While this documentation uses **Google Programmable Search Engine** as the example, the same technical flow can apply to other compatible web search engines as well.

Here’s a more detailed breakdown of how the system works:

1. **User Input**  
   The user writes a message in **Unique AI** that has web search capabilities enabled.

2. **Triggering the Web Search**  
   If the language model (LLM) determines that external information is needed, or if the user explicitly requests a web search, the web search flow is initiated.
   - **Generating the Search Query:** The LLM analyzes the conversation history and surrounding context to generate an initial search query.
   - **Refining the Query:** A separate LLM call is made to refine the original query. This step improves search accuracy and uses only the generated query (not the full conversation).
   - **Executing the Search:** The refined_query is sent to the web search engine.
   - **Receiving Search Results:** The web search engine returns a list of relevant links based on the query.

3. **Processing and Responding**  
   Unique AI scrapes the content from the returned links, processes the information, and uses it to generate a response to the user’s original question.

While all communications are secure and protected, the **risk** with the web search feature is the **potential leakage of surrounding context** in queries sent for **search engine**. This risk should be carefully evaluated before enabling web search.

## Container Overview

This container diagram for **Unique AI Web Search** outlines how user queries are handled to retrieve information from the web. The end user writes a query, which is securely transmitted via HTTPS or WSS to the Unique AI Chat interface — a React-based frontend with web search capabilities. This frontend communicates with the Unique Assistant-Core, a Python-based backend service that hosts the agentic LLM logic. The Assistant-Core securely retrieves a search engine API key from Azure Key Vault, which stores configuration values leveraged for end-to-end automation.

Once authorized, the Assistant-Core sends a web query to an external Web Search Engine ( _Google Programmable Search Engine_) over HTTPS. The search engine responds with relevant web pages, which the Assistant-Core processes and streams back the LLM response to the Unique AI Chat frontend.

### Google Connectivity

The Web Search module requires access to the **Google Custom Search API**, meaning it relies on external resources to function. It does **not** communicate with any other Unique backend services. All data is retrieved **exclusively** from the Google Custom Search API hosted at [cse.google.com](https://cse.google.com/).

# Planning

## Budget

[Google search JSON API Pricing](https://developers.google.com/custom-search/v1/overview#pricing)  
The search api is priced as follows:

- 100 search queries per day for free
- 5$ per 1000 queries, up to 10k queries per day
- It is possible to increase the 10k quota through the Google Cloud API console > Quota section.

# Provisioning

## Configuration of the `assistants-core` service

The `assistants-core` service expects 3 environment variables.

- Stored in a key vault  
   - `GOOGLE_SEARCH_API_KEY`: vault (`manual-google-search-api-key`)
- Stored in code `.yaml`
   - `GOOGLE_SEARCH_API_ENDPOINT` = `https://customsearch.googleapis.com/customsearch/v1`
   - `GOOGLE_SEARCH_ENGINE_ID` = `00c5f6e9ac4644f42`

The `GOOGLE_SEARCH_API_ENDPOINT` is exposed so that customers could use any wrapper engine around Google. The `GOOGLE_SEARCH_ENGINE_ID` is responsible for identifying the default configuration that you want to use for all your spaces (e.g. safe search). It’s possible to override this value from the search engine configuration in your space/tool. The environment variable must exist and can be supplied from your compliant secret origin.

## Administering keys

To obtain a search engine (ID) as well as an API key you can follow the [official guide](https://developers.google.com/custom-search/v1/overview).  
Ensure you store the API key in a secure spot (e.g. an Azure Key Vault) which is also available to the `assistants-core` service.

# Operating

## Monitoring

Google Search provides basic monitoring via the [Cloud Platform Console's API Dashboard](https://console.cloud.google.com/apis/dashboard). For more advanced monitoring—such as alerts and custom metrics— the [Google Cloud's Operations suite](https://cloud.google.com/products/operations) can be used.

## Authentication

Unique AI Web Search uses the public Google Custom Search API, which supports API keys only—service accounts are not supported.

Two environment variables must be configured for authentication:

- `GOOGLE_SEARCH_API_KEY`: This API key is included as a parameter in the GET request sent from Unique to the Google API.
- `GOOGLE_SEARCH_ENGINE_ID`: Provided to the assistants-core service, this value is passed as the cx query parameter to specify which custom search engine to use.

```
GET https://www.googleapis.com/customsearch/v1?key=GOOGLE_SEARCH_API_KEY&cx=GOOGLE_SEARCH_ENGINE_ID
```

## Troubleshooting

### Is the search correctly instantiated?

The Google credentials are loaded during the startup of the **assistant-core** service. If the credentials are successfully loaded, the following message is logged at `INFO` level:

```
"Google Search API is properly configured"
```

If initialization fails, a `WARNING` log will be generated with the following message:

```
Google Search API missing required settings: <missing env variables>
```

### General Troubleshooting

If the credentials are correctly loaded in the `assistants-core` service, the next step is to verify that the Google API is responding properly. You can do this by running the following command:

```
bash
curl "https://customsearch.googleapis.com/customsearch/v1?key=GOOGLE_SEARCH_API_KEY&cx=GOOGLE_SEARCH_ENGINE_ID&q=uniqueai"
```

If the API is working as expected, it will return a JSON response containing search results. If that succeeds, the next recommended step is to check the `assistants-core` logs for any error messages. These logs will help identify the root cause of the issue.