Agentic Metadata Extraction for Infra Admins

This feature is currently in BETA.

The Agentic Metadata Extraction Service automatically extracts structured metadata from document content using Large Language Models (LLMs). Unlike traditional metadata extraction relying on document properties, this service analyzes actual text content to intelligently extract business-relevant metadata fields.

Processing Flow

1. User uploads document → Content is ingested and chunked
2. Platform sends webhook to /metadata-extraction/webhook
3. Service fetches document chunks and configuration
4. Chunks merged up to maxInputTokens limit
5. LLM extracts metadata based on configured schema
6. Extracted metadata merged with existing content metadata
7. Returns 200 OK when complete synchronous processing

The service processes already-ingested content chunks, not raw documents. Processing is synchronous - the webhook completes when extraction finishes.

Code Path (agentic-ingestion)

POST /metadata-extraction
  └─ Receive content + metadata schema
      └─ LLM completion with structured output
          └─ Return extracted metadata fields per schema

Provisioning

Prerequisites

Service-Specific Requirements

• Feature flag: FEATURE_FLAG_ENABLE_AGENTIC_METADATA_EXTRACTION_UN_15619 =true
• Unique AI API access for content retrieval and updates
• LLM endpoints (Azure OpenAI or compatible) supporting structured output
• Network access to Unique AI API and LLM endpoints

Deployment

Environment Variables and Secrets

Sizing

Performance

Resource recommendations

We recommend a deployment with 2-4 replicas for High Availability.

Cost Estimation

LLM costs (GPT-4o example):

• ~$0.05-0.15 per document
• 10,000 docs/month: ~$500-1500/month

Configuration

Webhook Setup

To trigger the Agentic Metadata Extraction app, we need to send a POST request to the/metadata-extraction/webhook endpoint.

Metadata Schema

Schemas are configured per-folder in the Unique AI platform UI (Knowledge Upload → Folder Settings → Configure file ingestion).

Field Properties

Example Metadata Schema

{
  "summary": {
    "type": "string",
    "required": true,
    "description": "A brief one-sentence summary of the document"
  },
  "documentType": {
    "type": "string",
    "required": true,
    "description": "What type of document is this? (e.g., invoice, report, email)"
  }
}

Operating & Troubleshooting

Authentication Methods

The service uses:

• API Keys: For Unique AI API access

API Endpoints

Webhook Endpoint: POST /metadata-extraction/webhook Payload:

{
  "event": "unique.content.ingestion.finished",
  "companyId": "company-123",
  "userId": "user-456",
  "payload": {
    "contentId": "content-789"
  }
}

Troubleshooting

The metadata extraction feature is not visible on the UI, Why?

• Verify FEATURE_FLAG_ENABLE_AGENTIC_METADATA_EXTRACTION_UN_15619=true is set in Frontend (knowledge-upload app)

Metadata’s are not visible after Ingestion has finished, Why?

• Verify FEATURE_FLAG_ENABLE_AGENTIC_METADATA_EXTRACTION_UN_15619=true is set in Backend (agentic-ingestion app)
• Check if module is loaded

Key Metrics to Monitor:

1. Webhook processing lag/queue length
2. Processing time per document
3. Success/failure rates
4. Resource utilization (CPU, Memory)
5. LLM API response times
6. Content API health & authentication success rate
7. Token consumption & costs
8. Extraction quality (fields extracted, empty extraction rate)