Open-Weight LLM · Private & Custom AI
gpt-oss-safeguard-20b
Purpose-built safety classifier for private LLM pipelines: filter inputs/outputs, label content, and enforce custom policies without leaving your infrastructure.
gpt-oss-safeguard-20b is a 21B-parameter safety reasoning model (3.6B active) fine-tuned on OpenAI's gpt-oss base to classify text against operator-defined policies. For ops teams, it's a drop-in safety gate for custom AI systems—reasoning about policy violations, not just scoring them—that runs entirely self-hosted.
Model facts
Private deployment
Run gpt-oss-safeguard-20b in your own environment
Fits in 16GB VRAM (FP8/MXfp4 quantization available). Self-hosting means your safety policies, flagged content, and reasoning logs never leave your environment—critical for financial, healthcare, or regulated enterprise deployments. Requires harmony response format and inference framework (vLLM compatible). No external API calls for safety decisions; you own the latency and the audit trail.
Operational AI use cases
Content Moderation at Ingress
Automatically classify user-submitted forms, support tickets, or marketplace listings against your own safety policy before they hit downstream systems. The model reasons through policy rules ("harmful content" + "context") and returns a decision + chain-of-thought, letting ops teams debug false positives without guesswork.
Internal Knowledge Filtering
Pre-screen documents before indexing into a company RAG system or internal knowledge base. Prevents sensitive, policy-violating, or inappropriate content from contaminating retrieval pipelines—reduces hallucinations tied to unsafe training data and simplifies compliance audits.
Agent Output Validation
Place it as a safety checkpoint in agentic workflows (e.g., AI drafting customer responses, financial recommendations, HR decisions). Catches policy violations before human review or customer-facing deployment. Configurable reasoning effort lets you tune latency vs. accuracy per workflow tier.
Custom AI
As a base for custom AI
Ideal as a safety sidecar in custom AI applications: pair it with domain-specific LLMs or retrieval systems to add policy enforcement without retraining. Because it accepts written policies (not just training labels), you can adapt it across products—customer-facing chat, internal automation, document processing—by updating policy rules rather than redeploying models. Use case: embed it in a custom legal-document QA system to block answers that violate compliance thresholds.
In the operating system
Where it fits
Safety layer / policy enforcement checkpoint. In an LLM.co-style operating system, it sits between the knowledge layer (RAG, documents) and the agent/workflow layer (LLM decisions, human handoff). Operates *before* generation (input filtering) or *after* (output validation). Not a primary reasoning model; a specialized filter that amplifies safety of any larger LLM you're using.
Data control & security
Self-hosting eliminates data transit to third-party APIs. Your safety classifications, policy documents, and reasoning traces remain in your infrastructure—no storage in external logging systems. Does not guarantee compliance; responsibility for policy formulation, testing, and enforcement rests with the operator. Audit trails (model reasoning) are fully transparent and auditable, supporting compliance documentation (SOC2, HIPAA workflows) but only insofar as your deployment and governance enforce them.
Hardware footprint
**Estimate (varies by precision):** FP16 ~43GB VRAM | FP8 ~22GB VRAM | MXfp4 quantization ~12–16GB VRAM. Runs comfortably on a single A100 (40GB) or RTX 6000 (48GB). For throughput, batch size 4–8 on 16GB hardware; latency ~0.5–2s per input depending on reasoning effort setting. Smaller than 120B variant; larger than typical embedding models.
Integration
Requires harmony response format (custom prompt structure; see OpenAI cookbook). Supports vLLM, Ollama, and Azure deployments. Input: text + your written policy. Output: JSON with classification, confidence, and chain-of-thought reasoning. Stateless—easily parallelizable. Integrate via HTTP endpoints (if wrapped in a service) or Python library calls in batch/streaming workflows. Compatible with standard CI/CD for versioning policies and model updates. Be prepared to validate policy effectiveness on your data before production rollout.
When it's not the right fit
- —You need a primary generative LLM—this is a classifier, not a chat/coding model. Use gpt-oss-20b for generation, this for safety filtering.
- —Your safety policies are vague or evolving rapidly. Model generalizes across policies, but it assumes clear, documented rules. Expect iteration and validation.
- —You require sub-100ms latency at scale. Reasoning effort (low/medium/high) trades speed for accuracy; high reasoning can exceed 2s per item.
- —Your domain is highly specialized (medical coding, proprietary legal language) and you have no labeled safety data. Model is trained on broad content; fine-tuning may be necessary.
Alternatives to consider
Llama-2 13B + safety fine-tune
Smaller, faster, but requires you to train or curate a labeled safety dataset. No native policy-as-prompt mechanism. Good if you have in-house ML ops.
Perspective API (Google) / Hugging Face Moderation models
Lighter-weight (not LLM-based), faster, but no reasoning transparency and limited to pre-baked taxonomies. Trade configurability for simplicity.
GPT-4 Turk (OpenAI API)
More capable, reasoning via API, but requires external calls, data transmission, and per-token costs. Not private or self-hosted. Right fit if you accept those constraints.
Related open models
FAQ
Can I run this entirely on-premises with no external calls?
Yes. Download the weights, set up vLLM or compatible inference server on your hardware, load your policy docs, and run inference locally. Data and decisions stay in your environment.
Is this commercially licensable?
Yes. Apache 2.0 license permits commercial use, distribution, and modification without copyleft or patent risk. No royalties, no gating. Review the license terms to confirm compliance with your use case.
What if I need to customize it for my safety policies?
Primary mechanism is policy-as-prompt (in-context learning). Write clear policy rules and feed them to the model; no retraining required for most cases. For domain-specific safety (e.g., medical terminology), fine-tuning on labeled data is possible but not included in the base release.
Does this model guarantee compliance with GDPR, HIPAA, or other regulations?
No. The model itself has no compliance guarantees. Self-hosting is an *architectural choice* that keeps data local, but compliance responsibility is yours: validate the model's output, maintain audit logs, and ensure your deployment meets regulatory requirements.
Build a Private AI Safety System
gpt-oss-safeguard-20b is built for ops teams that need transparent, policy-driven safety in custom AI. Self-host it, control your data, and integrate it into your LLM stack with LLM.co. Start deploying with our private-AI framework.