Private RAG for Microsoft 365 Without Leaving Your Tenant

Microsoft 365 Copilot is not the only way to put a model on top of a SharePoint, Exchange and Teams corpus. A private RAG pipeline can read the same Graph endpoints, respect the same Purview labels, and keep the vector index and the model itself inside the tenant.

The architecture turns on three decisions: which Graph permission model the connector uses, how Microsoft Information Protection labels travel into the index, and where the embeddings and inference actually run. Each of those choices is the difference between a Copilot replacement and a data leak.

Graph Permissions Decide What the Model Can See

Microsoft Graph exposes the Microsoft 365 corpus through two authentication modes, and the one a RAG connector chooses governs every downstream control. Delegated permissions act on behalf of a signed-in user, so Graph enforces that user's SharePoint and Exchange ACLs at every call. Application permissions run as the daemon identity, bypass user context, and read anything the app has been granted at tenant scope.

Delegated is the safe default for an interactive assistant answering on behalf of a named user. Application permissions suit background ingestion only if the pipeline reconstructs the trimming step itself, storing the SharePoint site, Exchange folder, and Teams channel permissions alongside each chunk and intersecting them with the asker's group membership at query time. Skip that step and the model will cheerfully summarize the CFO's inbox for an intern.

The stakes are concrete. Microsoft has already scored a zero-click prompt injection in Copilot itself at CVSS 9.3, exfiltrating tenant data through a single crafted email. A custom LLM deployment buys no safety unless the retrieval side carries the ACL, the label, and the trim all the way through to the reranker.

Sensitivity Labels and Purview DLP Travel With the Chunk

Microsoft Information Protection labels and Purview retention and DLP tags are not query-time hints. They are ingest-time filters. The connector should read the sensitivityLabel on each Graph item, honor its content-marking and encryption settings, and write the label GUID into the chunk's metadata next to the ACL and the retention tag.

Three rules follow. Items labeled Highly Confidential should either stay out of the vector store or land in a separate index that only matching Entra ID groups can query. Retention-locked documents should carry the retention label into Purview Audit so eDiscovery can enumerate what the model saw and when. DLP policies on the retriever endpoint should fire against the same labels, so a label change in Purview propagates to the retriever on its next crawl rather than at the next annual review.

This is why Copilot rollouts stall. Governance work consumes real time and real headcount, and most organizations finish the first wave with the scope narrowed to a trusted subset. A private pipeline has to do the same work, but it gets to pick the moment the labels become enforceable rather than inherit Copilot's defaults.

Teams and Exchange Each Have Their Own Rules

SharePoint behaves like a filesystem for the Graph connector. Teams and Exchange do not. Teams messages live in two places at once: files sit under a hidden SharePoint folder tied to the channel, while posts and chats live in the Teams messaging service. That means two ingest paths per team and two permission models to honor. Private channels get their own site collection, so a connector cannot inherit parent-team ACLs. Meeting chats and 1:1 chats are tied to participant GUIDs rather than any site, and they expire according to the team's retention policy.

Exchange is where throttling bites first. A naive full-mailbox crawl of a mid-sized tenant trips Graph's per-app limits within hours. The practical pattern is delta queries, batched extraction, and exponential backoff on every endpoint, with a separate scheduler for calendars because those change on a different cadence. For a healthcare or financial services tenant, the compliance framework usually dictates what gets indexed at all, which is a stricter gate than throttling will ever be.

The Vector Store and the Model Stay in the Tenant

Azure AI Search sizes its HNSW vector indexes per partition. Microsoft's own vector sizing guidance states the index must stay resident in memory during query execution, while the on-disk form consumes roughly three times that footprint, and a field can carry up to 4,096 dimensions. For a tenant with a few million documents this is the single largest capacity decision in the design.

A private build does not have to use Azure AI Search. pgvector on an Azure VM, Qdrant or Weaviate in AKS, or an appliance-hosted store will all accept the same chunks; the only thing that must not move is the content. The comparison of vector stores for private RAG walks through the trade-offs for each. The embedding model, generation model, and retriever should run inside the same subscription, ideally the same VNet, behind a Private Endpoint or on an on-prem appliance. Microsoft states that Copilot prompts and responses are not used to train foundation LLMs; a private deployment can commit the same for every log line, which matters to a CISO reviewing enterprise LLM security posture under HIPAA, GLBA, or CMMC.

Where Private RAG Pays Off

Microsoft sells Copilot at $30 per user per month on an annual commitment, on top of a qualifying M365 base plan. At 2,000 seats that is $720,000 per year before the tenant pays for the retention, DLP, and audit work that every rollout still has to do. A private VPC build with an open-weight model clears that line sooner than finance teams expect once the headcount covering Copilot licenses crosses the thousand-seat mark.

Sovereignty cuts the same way. Microsoft says in-country Copilot processing will reach four countries by end of 2025 and eleven more in 2026. A regulated buyer whose jurisdiction is not on that calendar can ship a private build inside the right region now.

Service	Resource	Limit	Window
Exchange	Message trace	100 requests	Per 5 minutes, per tenant
Teams Cloud Comms	Calls	50,000 requests	Per 15 seconds, per app/tenant
Teams Cloud Comms	Presence	10,000 requests	Per 30 seconds, per app/tenant
Teams Cloud Comms	Virtual events (GET)	750 requests	Per 30 seconds, per app across tenants
Teams Meetings	Meeting info reads	2,000 meetings	Per user, per month

The Retrieval Test That Separates Real Builds From Demos

Ask any vendor pitching an M365 RAG build to show how the retriever reconstructs SharePoint item-level permissions and MIP labels for a user who was moved between two Entra ID groups yesterday. A good answer names the Graph permission mode, the delta query cadence, the label-propagation path from Purview into the chunk metadata, and the regression test that would catch a stale ACL at retrieval time. It lists the audit destination, whether that is Purview Audit, a Sentinel workspace, or a syslog sink inside the tenant. Anything shorter is a demo, not a design.