# SharePoint Connector - Flows

## Content Sync Flow

The content sync flow runs periodically (default: every 15 minutes) to synchronize flagged documents from SharePoint to Unique.

### Overview

### Sequence Diagram

The connector is **stateless** — it does not maintain local state between sync cycles. Change detection is performed by the Unique platform's file diff API.

## Subsite Discovery and Sync Flow

When `subsitesScan` is enabled for a site, the connector extends the per-site content sync with recursive subsite discovery and content fetching. All steps run sequentially.

### Overview

### Sequence Diagram

### Key Behaviors

- **Recursive discovery**: The connector walks the full subsite tree, not just direct children.

- **Deduplication**: Subsites already configured as standalone sites (via compound ID) are skipped along with their descendants.

- **Unified scope tree**: Subsite content is placed under the parent site's root scope. A subsite at path `ParentSite/SubA` creates scopes like `/RootScope/SubA/Documents/...`.

- **File diff keying**: Subsite items carry a `syncSiteId` pointing to the parent site. The file-diff mechanism uses this to scope all items (parent + subsites) under one diff key, ensuring correct deletion detection when subsites are removed.

## Permission Sync Flow

When enabled, the permission sync flow synchronizes SharePoint permissions to Unique.

### Overview

### Sequence Diagram

### Permission Types

The connector handles different permission sources:

| Source | API | Resolution |
| --- | --- | --- |
| Direct user grant | Graph API | Map email to Unique user |
| Entra ID (Azure AD) group | Graph API | Expand group members |
| SharePoint site group | SharePoint REST | Expand group members |
| Sharing link | Graph API | Extract grantees |

### Group Visibility Requirement

For SharePoint site groups, the connector must be able to read group members. If "Who can view the membership of the group?" is **not** set to "Everyone", the connector cannot read members.

**Mitigation:**

- Set group visibility to "Everyone"

- Add app principal as group member/owner

- Grant Full Control to app principal

### Public Site and Tenant-Wide Groups

For public SharePoint sites, permissions can include tenant-wide principals such as `Everyone` or `Everyone except external users`. The connector does not expand these principals for sync. As a result, content may be accessible in SharePoint while corresponding tenant-wide visibility is not mirrored in Unique permissions.

## File Diff Mechanism

The connector uses the Unique platform's server-side file diff API (`/v2/content/file-diff`) to detect changes between sync cycles. The connector does **not** compute local content hashes — instead, it sends each file's `key`, `url`, and `updatedAt` timestamp to the diff endpoint, which returns categorized results.

### State Comparison

### Change Detection Logic

### File Diff Item Attributes

Each item sent to the diff API contains:

| Attribute | Description | Used For |
| --- | --- | --- |
| `key` | Unique key identifying the file (derived from SharePoint drive/item path) | Identity and change tracking |
| `url` | SharePoint URL of the file | Location tracking |
| `updatedAt` | Last modification timestamp from SharePoint | Change detection |

### Ingestion error handling

The connector **does not** read Unique ingestion states. Each sync cycle it only runs the download and ingest pipeline for keys returned in `newFiles` or `updatedFiles` from the file diff. If a key is in neither list, **no ingest attempt runs for that item in that cycle**; the rest of the site still syncs. Which keys appear in those lists is decided by the Unique platform (file diff, stored ingestion state, and timestamps). Successful and in-progress processing is opaque to the connector until the diff includes the key again.

When Unique reports an item in one of the failure states below, the platform throttles re-ingestion so the key is re-offered at most every **24 hours**:

- `FAILED`
- `FAILED_MALWARE_SCAN_TIMEOUT`
- `FAILED_PARSING`
- `FAILED_CREATING_CHUNKS`
- `FAILED_EMBEDDING`

Any other terminal `FAILED_*` state (for example `FAILED_MALWARE_FOUND`, `FAILED_GETTING_FILE`, `FAILED_TIMEOUT`) is handled inside Unique; the connector still only ingests when the key appears in `newFiles` or `updatedFiles`, with **no** separate per-state policy. Retry cadence for those states is defined by the Unique platform and may differ from the 24-hour list above.

## ASPX Page Processing

SharePoint site pages (`.aspx`) require special handling:

### ASPX Sync Flow

### Content Extraction

ASPX pages contain structured content in special fields:

| Field | Content Type | Description |
| --- | --- | --- |
| `CanvasContent1` | JSON/HTML | Modern page web parts |
| `WikiField` | HTML | Classic wiki content |

The connector extracts text content from these fields for ingestion.

## Error Handling

### Error Handling Strategy

The connector applies scenario-specific behavior to keep sync cycles stable while avoiding incorrect permission or content updates:

| Scenario | Typical Cause | Connector Behavior |
| --- | --- | --- |
| Authentication/configuration error | Invalid certificate, wrong tenant/app configuration | Fail the current cycle early, log actionable error, require operator fix |
| Transient API/network error | 429/5xx, temporary network failures | Retry with backoff up to retry limit, then skip affected item and continue |
| Permission denied (`403`) | Missing site/library grant or group visibility restriction | Skip affected item/permission sync path and continue remaining work |
| Not found (`404`) | Item deleted/renamed or stale state | Treat as deleted where applicable and reconcile local state |
| Malformed/unsupported content | Corrupt file or parser failure | Log item-level error, skip item, continue cycle |

### Retry Logic

### Retryable Errors

| Error Code | Description | Retry |
| --- | --- | --- |
| 429 | Rate limited | Yes (with backoff) |
| 500 | Server error | Yes |
| 502 | Bad gateway | Yes |
| 503 | Service unavailable | Yes |
| 504 | Gateway timeout | Yes |
| 401 | Unauthorized | Yes (refresh token) |

### Non-Retryable Errors

| Error Code | Description | Action |
| --- | --- | --- |
| 400 | Bad request | Skip item, log error |
| 403 | Forbidden | Skip item, log error |
| 404 | Not found | Mark as deleted |

## Related Documentation

- [Architecture](https://docs.unique.ai/it-operators/integrations-connectors/sharepoint-connector/sharepoint-connector-technical-manual/sharepoint-connector-architecture) - System components and infrastructure
- [Metadata](https://docs.unique.ai/it-operators/integrations-connectors/sharepoint-connector/sharepoint-connector-technical-manual/sharepoint-connector-metadata) - Ingested metadata fields for documents and SitePages
- [Permissions](https://docs.unique.ai/it-operators/integrations-connectors/sharepoint-connector/sharepoint-connector-technical-manual/sharepoint-connector-permissions) - Required API permissions
- [Configuration](https://docs.unique.ai/it-operators/integrations-connectors/sharepoint-connector/sharepoint-connector-operator-manual/sharepoint-connector-configuration) - Scheduler and processing settings

## Standard References

- [Microsoft Graph API - DriveItem](https://learn.microsoft.com/en-us/graph/api/resources/driveitem) - DriveItem resource
- [Microsoft Graph API - Permissions](https://learn.microsoft.com/en-us/graph/api/driveitem-list-permissions) - List permissions
- [SharePoint REST API](https://learn.microsoft.com/en-us/sharepoint/dev/sp-add-ins/get-to-know-the-sharepoint-rest-service) - REST service overview