# Connect custom LXM (LLM) with Unique AI

Unique provides the ability to connect a custom LXM (Large X model where X is language, video or audio, etc.) from within Unique AI to be able to use self- or cloud-hosted language models for the standard use cases of Unique AI.

---

# Technical Insight

To fulfil this use case Unique created two entities. One is called `LanguageModelGroup` and one is called `ExternalLanguageModel`. The `LanguageModelGroup` is an object containing mainly the `groupName` and is connected to the `ExternalLanguageModel` with an n:m relationship. The `groupName` basically replaces the hardcoded LLM enums of Unique. This fulfils the requirement that clients want to manage the LXMs by themselves.

The `ExternalLanguageModel` contains all the information which is needed for Unique to be able to call this custom provided LXM. This gives the ability to clients to be able to connect custom LXM with Unique.

The reason why we have the possibility to group together multiple `ExternalLanguageModel` into a `LanguageModelGroup` is that load can be randomly distributed to multiple instances. In case the token usage is 500k tokens but the limit per model instance/deployment is capped at 300k tokens there is the possibility to deploy a 2nd instance to fully cover the usage request.

# Chat API URL

For the following documentation you might require the Chat Backend API URL as location for the request examples. This URL is marked as `<chatGraphqlUrl>` and depends on your tenant. It can be one of the following:

For CH and EU: [https://gateway.unique.app/chat-gen2/graphql](https://gateway.unique.app/chat-gen2/graphql)

For US: [https://api.us1.unique.app/chat/graphql](https://api.us1.unique.app/chat/graphql)

# How to Create and Connect

Currently Unique does not offer a user interface for it. So the following curl request will create an `ExternalLanguageModel` and connect it with the `LanguageModelGroup`. If the `LanguageModelGroup` is not existing yet, Unique will create a new `LanguageModelGroup`. If there is already one with this `groupName` Unique will add it to the existing one.

## Create Graphql-Mutation

```bash
curl --location '<chatGraphqlUrl>' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <token>' \
--data '{"query":"mutation Mutation($groupName: String!, $input: ExternalLanguageModelCreateInput!) {\n  externalLanguageModelCreate(groupName: $groupName, input: $input) {\n    id\n    internalDescription\n    languageModelGroups {\n      id\n      groupName\n    }\n  }\n}","variables":{"groupName":"<groupName>","input":{"version":"<modelVersion>","model":"<modelName>","maxTokens":<maxToken>,"internalDescription":"<internalDescription>","endpoint":"<endpoint>","deploymentName":"<deploymentName>","authHeaderValue":"<apiToken>","apiVersion":"<apiVersion>"}}}'
```

| **Field** | **Description** | **Azure OpenAI example** |
| --- | --- | --- |
| **tenantName** | Unique Single Tenant Base URL | `oleole` |
| **token** | `token` A token for your tenant of a user with role `chat.admin.all`<br>- Refer to [How To Get An API Token](https://docs.unique.ai/it-operators/identity-and-access-management-iam/how-to-get-an-api-token) to learn how to get a token | `Bearer S0VLU0UhIExFQ0tFUiEK...` |
| **groupName** | The `LanguageModelGroup` that is **referenced in further Unique configurations**. | `MY_OWN_MODEL_GPT_4_RAVEN` |
| **modelVersion** | The model version deployed at your endpoint | <br>Model Version (here turbo-2024-04-09) |
| **modelName** | The model name deployed at your endpoint | <br>Model Name (here gpt-4) |
| **maxToken** | The maximum amount of tokens that Unique will send in one request to the endpoint | `3000` |
| **internalDescription** | A description of the endpoint for Unique internal reference. You can use it later to reference information of yours to recognise the LXM. | `GPT-4-Turbo deployed at experimental Azure subscription Tango-001.` |
| **endpoint** | The endpoint of your LXM | `https://your-endpoint.openai.azure.com/`<br><br>Endpoint |
| **deploymentName** | The deployment name (gets added to the endpoint by the [@azure/openai](https://www.npmjs.com/package/@azure/openai) library) that you chose when deploying the model in a specific version | <br>Deployment Name (here gpt-4-turbo-2024-04-09) |
| **apiToken**( *** even if just a artificial - otherwise it uses Azure WID) | The API token, see [#Security](https://unique-ch.atlassian.net/wiki/spaces/SDV/pages/1363254192/Connect+custom+LXM+LLM+with+Unique+AI#Security) This is getting saved encrypted in the DB and only decrypted when Unique does the API request. There will be no possibility to access or change it afterwards, it is immutable write-only. | <br>Key |
| **apiVersion** | API Version that the [@azure/openai](https://www.npmjs.com/package/@azure/openai) library will use to talk to the model. | `2024-02-01` |
| **oauth2** | If the endpoint oauth2 should be used as authentication the query above needs to be extended with the following fields: `oauth2ClientId`, `oauth2ClientSecret`, `oauth2Endpoint`, `oauth2Scope`(optional). Follow [this documentation](https://docs.unique.ai/it-operators/custom-models-and-embeddings/oauth2-for-lxm-usage) for more information about Oauth2 authentication for LLM calls. |

**\* RequiredFields**

### Azure OpenAI GPT-4o full example

```bash
curl --location '<chatGraphqlUrl>' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer {TOKEN}' \
--data '{"query":"mutation Mutation($groupName: String!, $input: ExternalLanguageModelCreateInput!) {\n  externalLanguageModelCreate(groupName: $groupName, input: $input) {\n    id\n    internalDescription\n    languageModelGroups {\n      id\n      groupName\n    }\n  }\n}","variables":{"groupName":"GPT_4O_PTU_CUSTOM","input":{"version":"2024-05-13","model":"gpt-4o","maxTokens":126000,"internalDescription":"Uniques GPT 4o QA deployment in Sweden","endpoint":"https://unique-openai-083.openai.azure.com/","deploymentName":"gpt-4o-2024-05-13","authHeaderValue":"<secret>","apiVersion":"2024-02-01"}}}' \
```

## Renaming an Existing GroupName

During this process, after changing to the new groupName, the previous groupName will no longer be valid and all LXM configurations using the old groupName will fail! Until you have manually changed all previous configurations to the new one.

This renaming does not automatically rename all configurations done with the old groupName!

Due to this limitation, Unique recommends to first create a new group (including all the same ExternalLanguageModels - basically do the “Create API” chapter) under the new name, migrate all assistants and then execute a Delete Call for not deprecated objects.

To rename an existing `groupName` you need to know the current `groupName`. Then with this curl you can rename the groupName to a different new groupName.

```bash
curl --location '<chatGraphqlUrl>' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <token>' \
--data '{"query":"mutation LanguageModelGroupUpdate($groupName: String!, $input: LanguageModelGroupUpdateInput!) {\n  languageModelGroupUpdate(groupName: $groupName, input: $input) {\n    groupName\n    id\n  }\n}","variables":{"groupName":"<currentGroupName>","input":{"groupName":"<newGroupName>"}}}'
```

| **Field** | **Description** | **Azure OpenAI example** |
| --- | --- | --- |
| **tenantName** | Unique Single Tenant Base URL | `oleole` |
| **token** | `token` A token for your tenant of a user with role `chat.admin.all`<br>- Refer to [https://unique-ch.atlassian.net/wiki/x/ogAbV](https://unique-ch.atlassian.net/wiki/x/ogAbV) to learn how to get a token | `Bearer S0VLU0UhIExFQ0tFUiEK...` |
| **currentGroupName** | The current `LanguageModelGroup.groupName` | `MY_OWN_MODEL_GPT_4_RAVEN` |
| **newGroupName** | The new `groupName` you want to give to this group. | `NEW_MODEL_GPT_4_RAVEN_2` |

Now the `LanguageModelGroup.groupName` is renamed to the new name and all models attached to it are now available under the new `groupName`. The old `groupName` is not valid anymore!

## Deleting an Existing LanguageModelGroup

Deleting a LanguageModelGroup which is still in use will disrupt the system. Unique does not reverse engineer if a group is used. If a group gets deleted while linked to an assistant, the assistant will subsequently fail.

This case is not covered by Uniques SLAs!

When you want to delete a `LanguageModelGroup` make first sure that the `groupName` related to this group is not used anymore in any LXM configuration. Otherwise this configuration will not work anymore.

After you made sure that this `groupName` is not used anymore you can use the following curl to delete the `LanguageModelGroup`.

| **Field** | **Description** | **Azure OpenAI example** |
| --- | --- | --- |
| **tenantName** | Unique Single Tenant Base URL | `oleole` |
| **token** | `token` A token for your tenant of a user with role `chat.admin.all`<br>- Refer to [https://unique-ch.atlassian.net/wiki/x/ogAbV](https://unique-ch.atlassian.net/wiki/x/ogAbV) to learn how to get a token | `Bearer S0VLU0UhIExFQ0tFUiEK...` |
| **groupNameToDelete** | The `groupName` you want to delete | `MY_NOT_USED_GPT_4_RAVEN` |

```bash
curl --location '<chatGraphqlUrl>' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <token>' \
--data '{"query":"mutation LanguageModelGroupDelete($groupName: String!) {\n  languageModelGroupDelete(groupName: $groupName) {\n    id\n    groupName\n  }\n}","variables":{"groupName":"<groupNameToDelete>"}}'
```

## Deleting an Existing ExternalLanguageModel

Deleting the last ExternalLanguageModel assigned to an LanguageModelGroup which is still in use will disrupt the system. We can then no longer find a LXM assigned to this group. Unique does not reverse engineer if a group is used. If a group gets deleted while linked to an assistant, the assistant will subsequently fail.

This case is not covered by Uniques SLAs!

First make sure you do not delete the last `ExternalLanguageModel` assigned to a `LanguageModelGroup` which is still in use. You can do this with the following curl which will fetch all `ExternalLanguageModels` and its `LanguageModelGroups`.

| **Field** | **Description** | **Azure OpenAI example** |
| --- | --- | --- |
| **tenantName** | Unique Single Tenant Base URL | `oleole` |
| **token** | `token` A token for your tenant of a user with role `chat.admin.all`<br>- Refer to [https://unique-ch.atlassian.net/wiki/x/ogAbV](https://unique-ch.atlassian.net/wiki/x/ogAbV) to learn how to get a token | `Bearer S0VLU0UhIExFQ0tFUiEK...` |

```bash
curl --location '<chatGraphqlUrl>' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <token>' \
--data '{"query":"query ExternalLanguageModelFindMany {\n  externalLanguageModelFindMany {\n    id\n    internalDescription\n    languageModelGroups {\n      groupName\n    }\n  }\n}"}'
```

Also in the response of this list of `ExternalLanguageModels` you also can find the `id` of the `ExternalLanguageModel` you would like to delete. This `id` then can be used to delete the `ExternalLanguageModel` with the following curl:

| **Field** | **Description** | **Azure OpenAI example** |
| --- | --- | --- |
| **tenantName** | Unique Single Tenant Base URL | `oleole` |
| **token** | `token` A token for your tenant of a user with role `chat.admin.all`<br>- Refer to [https://unique-ch.atlassian.net/wiki/x/ogAbV](https://unique-ch.atlassian.net/wiki/x/ogAbV) to learn how to get a token | `Bearer S0VLU0UhIExFQ0tFUiEK...` |
| **IdOfModelToDelete** | The ID of the `ExternalLanguageModel` you want to delete | `xxxxxxxxxxxxxx` |

```bash
curl --location '<chatGraphqlUrl>' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <token>' \
--data '{"query":"mutation ExternalLanguageModelDelete($externalLanguageModelDeleteId: String!) {\n  externalLanguageModelDelete(id: $externalLanguageModelDeleteId) {\n    id\n    internalDescription\n  }\n}","variables":{"externalLanguageModelDeleteId":"<IdOfModelToDelete>"}}'
```

## Other APIs

There is also the possibility for other kind of mutations via API on the `ExternalLanguageModel` and the `LanguageModelGroup`. Consult Unique support to learn more.

## Connect

With this `groupName` you passed you are now able to change the `languageModel` attribute on the modules. From now on, this module configuration will use one of the `ExternalLanguageModel`, which is bound to this `LanguageModelGroup`.

# Security

## Connectivity

Naturally, as of now, the model must be reachable over the same network. For Unique PaaS and Single Tenants that means; **the internet**! Customer Managed or On-Premise Tenants can leverage this functionality as well where their models must just be reachable from the deployments networks.

## API Token

Unique uses the `AES-256-cbc` algorithm with a 32 bytes random string to encrypt and decrypt the API Token symmetrically. Each Unique Deployment has its own unique encryption key. This encryption key is stored in an Azure KeyVault vault of the environment and will be loaded on startup into the chat application. The same key is used for all organisations/companies within the one deployment/tenant.

By enabling clients to directly store the key encrypted, Unique must never receive the key and store it. Unique also does not see or need the key as the key gets decrypted just-in-time to send the request. Technically Unique employees can equally not retrieve the key from the database except in specially triggered incidents by the client where Unique employees get temporary access to inspect the clients data.

## Company Scoping

The complete `Connect custom LXM (LLM) with Unique AI` feature is **company-scoped**. Companies cannot reuse `LanguageModelGroup`s or `ExternalLanguageModel`s from other companies. Of course, `LanguageModelGroup`s and `ExternalLanguageModel`s can be added to two or more companies though but that requires Administrators to explicitly send and define the same LXM for two or more companies/organisation.

# Limitations

- **Reserved key words:** There are reserved key words for groupNames which cannot be used. Basically are that the ones that Unique provides you as default starting with `AZURE_GPT_`

- **API only:** Configurations can only be done via API currently.

- **GPT API standards:** For the LXM network request Unique is using the [@azure/openai](https://www.npmjs.com/package/@azure/openai) library. Means the APIs of your LXM needs to be compatible with the GPT standards. This is also known as [https://learn.microsoft.com/en-us/azure/ai-studio/reference/reference-model-inference-api?tabs=python](https://learn.microsoft.com/en-us/azure/ai-studio/reference/reference-model-inference-api?tabs=python).

- **Connecting one model to two groups:** It's not possible to connect one ExternalLanguageModel to multiple LanguageModelGroup. The API call for this mutation is currently missing and is in progress.