# Configuring Crawlers

1 min read

# Extended Unique Crawling Capabilities to Third Party Crawlers

Unique has added support for the following third party crawlers on demand:

- [**Firecrawl**](https://www.firecrawl.dev/)
- [**Jina Reader**](https://jina.ai/reader)
- [**Tavily**](https://www.tavily.com/)

These crawlers have more capabilities than the default internal solution (`BasicCrawler`). For example, they are capable of reading content from links pointing to pdf content.

The crawlers API keys must be configured on `assistant-core` services. If you wish to test one of these crawlers, please approach your Customer Success representative to have the corresponding API key configured in your environment.

# Selecting Crawlers in Spaces

1. Go to `Spaces > Tools > Web Search`  
2. in `Web Search` settings `Crawler Config` select the crawler

# Configuring Crawlers for the Unique Platform

## Firecrawl

The configuration of the firecrawl crawler is simple, only the timeout parameter can be altered.

## Jina Reader

Please keep the default headers and do not change them if you don’t have experience with Jina Reader. You can find full documentation of the reader under this [link](https://docs.jina.ai/).

## Tavily

Tavily Extract endpoint has two different extraction depth possible, `basic` and `advanced`. The depth of the extraction process. `advanced` extraction retrieves more data, including tables and embedded content, with higher success but may increase latency. More details can be found under this [link](https://docs.tavily.com/documentation/api-reference/endpoint/extract#body-extract-depth).