01GOV
Model gateway
One API for every model, with keys, quotas, routing rules and fallbacks.
LLM.co · Private AI & Custom AI DevelopmentPrivate AI governance
Single sign-on, role- and document-level access, every prompt and answer logged to your SIEM, and data pinned to the country or building you choose. Built in, not bolted on.
FIG. — Mission control, NASA DrydenLLM.co
Your security and compliance teams already know how to govern software: identity, least privilege, logging, retention and change control. Private AI should fit those controls instead of asking for exceptions.
We put a gateway in front of every model and application. It knows who is asking, what they may see, which model answered and what it said, and it writes all of that where your auditors already look.
| Identity | Okta, Microsoft Entra ID, SAML, OIDC, SCIM |
|---|---|
| Logging | Splunk, Microsoft Sentinel, Elastic, Datadog |
| Controls | PII redaction, topic rules, rate limits, approvals |
| Residency | Data and weights pinned to a region or site |
| Frameworks | Mapped to SOC 2, HIPAA, GLBA, CMMC, GDPR |
What we build
What this looks like in practice.
02GOV
Identity & access
SSO with Okta or Entra ID, roles, and document-level permissions mirrored from sources.
03GOV
Audit logging
Prompts, retrieved passages, answers and actions, retained under your policy.
04GOV
Policy controls
PII redaction, topic restrictions and approval workflows, enforced at the gateway.
What AI governance covers in a private AI system
AI governance is the set of technical controls and records that let you show who used an AI system, what data it saw, which model answered, what it said and what happened next. In private AI, every link in that chain runs on infrastructure you control, so the evidence lands in the systems your security team already uses.
LLM.co builds governance into every system as part of custom AI development. The controls go in during the hardening phase, before the system reaches users, and they are tested along with everything else.
The model gateway is the control point
Every application and user reaches the models through one gateway. That gives your team a single place to enforce rules and collect evidence, whatever model or application sits behind it.
- Identity through Okta, Microsoft Entra ID, SAML or OIDC, with SCIM for provisioning
- Roles that decide which models and applications each person may use
- Document-level permissions mirrored from source systems and applied at retrieval time
- PII redaction, topic restrictions, rate limits and approval steps for sensitive actions
- Routing rules and fallbacks that decide which model handles each request
What an AI audit log should record
An audit record for an AI request needs more than a timestamp and a user ID. The gateway logs the user and their groups, the application, the prompt, the passages retrieved and their source documents, the model name and version, the answer, and any tool call or action that followed. For agents, each step and each approval is recorded against the case it belongs to.
Logs go to your SIEM, such as Splunk, Microsoft Sentinel, Elastic or Datadog, under your retention and legal-hold rules. Prompts and answers can contain sensitive data, so the log pipeline gets the same encryption and access controls as the source systems.
Data residency and model version control
Data residency for AI covers more than documents. Prompts, embeddings, vector indexes, logs and model weights each need a defined location. We pin them to a region, a data center or a single building, depending on your rules.
Model versions are pinned as well. A model change goes through your change control with evaluation scores attached, so you can show which model was running on any given date and why it was approved.
Mapping AI governance to your compliance framework
We line up the system's data flows with the framework you already follow, such as SOC 2, HIPAA, GLBA, CMMC or GDPR. You receive a control matrix, architecture diagrams and data-flow diagrams ready for your reviewers.
These controls support your compliance program. Certification and attestation stay with your auditors, and the policies themselves stay with your organization. Our job is to make the AI system fit those policies with no special exceptions.
How it works
Four steps, each one reviewed.
01
Map controls
Line up your existing frameworks, such as SOC 2, HIPAA, GLBA or CMMC, with the AI system's data flows.
02
Wire identity
Connect SSO and groups; define who may use which models and data.
03
Log & retain
Ship structured logs to your SIEM with retention and legal-hold rules.
04
Document
Architecture, data-flow diagrams and a control matrix ready for review.
Questions
Common questions.
What is AI governance?
AI governance is the combination of policies and technical controls that decide who may use AI systems, with which data and models, and how that use is recorded and reviewed. In a private AI deployment it covers identity, access, policy enforcement, audit logging, data residency and model change control.
Can we see exactly what a model was asked and answered?
Yes. The gateway records the user, the application, the prompt, the passages retrieved, the model and version, the answer and any resulting action. Those records go to your SIEM under your retention and legal-hold rules, where your security and audit teams can search them.
Does private AI governance make us compliant?
No system makes an organization compliant on its own. We provide the technical controls and documentation that compliance programs ask for, mapped to frameworks such as SOC 2, HIPAA, GLBA, CMMC or GDPR. Certification and attestation remain with your auditors.
How do you stop users from seeing documents they shouldn't?
Permissions are read from each source system and applied at retrieval time. A user only gets answers drawn from documents they could already open in SharePoint, iManage, Google Drive or the original repository. Role rules at the gateway also limit which models and applications each person may use.
How is sensitive data handled in prompts and logs?
The gateway can redact PII before a prompt reaches a model or a log, based on rules you define. Logs that keep full text are encrypted and access-controlled like the source data. Retention periods and legal holds follow your existing policy.
Can the same controls govern public AI use?
Yes. The gateway can route approved, non-sensitive requests to a public API while keeping regulated data on local models, under rules you set. Every request is logged the same way regardless of which model handled it.
Which identity providers and SIEMs do you work with?
For identity, Okta and Microsoft Entra ID, plus any SAML or OIDC provider, with SCIM for provisioning. For logging, Splunk, Microsoft Sentinel, Elastic and Datadog, along with other platforms that accept structured logs. We match the formats and fields your security team already uses.
Can governance be added to an AI system we already run?
Often, yes. If the application can call an OpenAI-compatible API, it can be pointed at the gateway to gain SSO, access rules, redaction and audit logging. Systems with deeper integration needs are assessed during a discovery sprint before any changes are made.
Related
More private ai.
Start here
Submit a job card.
Tell us the workflow and where the data lives. An engineer, not a salesperson, replies within one business day with a first take on architecture and cost.