AI Answer Assessment — Unique AI Documentation
AI Answer Assessment
Overview
The AI Answer Assessment feature includes functionalities like Hallucination Checks. This tool ensures that users receive accurate and reliable information by systematically evaluating the quality and adherence of messages to retrieved sources.
Who is it for
This feature is ideal for:
If you’re unable to access certain features or sections of this article, it’s possible that your firm doesn’t have access or hasn’t upgraded to the latest version. Please reach out to your internal support team for further assistance.
Benefits
- Accuracy: Ensures that messages are evaluated for factual correctness.
- Reliability: Reduces the risk of misinformation by categorizing hallucination levels.
- User-Friendly: Automatically generates assessments without requiring additional user actions.
- Transparency: Provides clear insights into the quality of the information provided.
Use Cases
- Content Verification: Assessing the accuracy of information in chat messages.
- Regulatory Compliance: Ensuring that messages meet predefined compliance standards (future functionality).
- Educational Purposes: Teaching users about the importance of source adherence and factual accuracy.
Step-by-Step Guide
How to Activate the Answer Assessment
The Hallucination Check feature is automatically enabled if the Hallucination Check option is activated in Internal Knowledge Search.
How It Works
The message assessment is generated instantly after the assistant's message is completed and appears in the chat interface without requiring a page reload.
Interface
Once the message assessment is available, it appears as an accordion at the bottom of the message.
Assessment Types & Sentiments
Assessments are classified by:
- Type:
String e.g HALLUCINATION - Sentiment:
GREEN,YELLOW, orRED
| Positive/Verified assessment | Unverified assessment | Negative assessment |
|---|---|---|
Assessment with low hallucination level |
Assessment with medium hallucination level |
Assessment with high hallucination level |
For more information on the meaning of the hallucination level, please visit our Hallucination Evaluation documentation.
Tips & Tricks
- Understand Sentiments:
- Green: Positive/Verified assessment.
- Yellow: Unverified assessment.
- Red: Negative assessment.
- Learn from Examples: Review the explanations provided for each sentiment to understand the reasoning behind the evaluation.
Limitations
- Compliance Checks: This functionality is not yet supported.
- Scope: The feature currently focuses on hallucination levels and does not cover all aspects of message quality.
- Dependence on Settings: The Hallucination Check must be activated in the Internal Knowledge Search settings to function.