Open-Weight LLM · Private & Custom AI
gemma-4-E4B-it-OBLITERATED
A 7.9B parameter base model with guardrails surgically removed — built for operators who need an unrestricted private LLM they fully control.
OBLITERATED is a refusal-removal fork of Google's Gemma 4 E4B, achieved through ablation techniques that strip safety layers while preserving core language capabilities. For ops teams deploying private AI infrastructure, it trades safety guardrails for total control: no refusals, no external API dependencies, data stays in your environment. It runs on commodity hardware (4.9–7.4 GB GGUF quantizations) and is compatible with standard inference stacks (Ollama, llama.cpp, LM Studio).
Model facts
Private deployment
Run gemma-4-E4B-it-OBLITERATED in your own environment
Self-hosting is the primary deployment model: GGUF quantizations (Q4_K_M: 4.9 GB, Q5_K_M: 5.3 GB, Q8_0: 7.4 GB) run locally on CPU or consumer GPUs without touching external APIs. Data never leaves your infrastructure. Compatibility requires recent inference-engine versions (Ollama 0.20+, llama.cpp b8665+); v3 fixes critical tensor corruption from the original ablation technique. Trade-off: you assume responsibility for model behavior—no upstream safety guarantees from a vendor.
Operational AI use cases
Unrestricted Internal Documentation & Knowledge Synthesis
Feed proprietary docs, logs, and sensitive operational data into the model without redaction or compliance overhead. OBLITERATED processes sensitive business context (financial reports, incident postmortems, internal discussions) without deflection or refusal—useful for compliance-checked internal knowledge extraction where your team retains control of guardrails.
Autonomous Workflow Agents & Decision Support
Build agents for finance approvals, HR policy queries, or incident triage that operate without query refusals interrupting the workflow. Since the model won't refuse edge-case queries, agents can handle unconventional scenarios (policy loopholes, corner-case scenarios, adversarial testing) without triggering safety interlocks designed for public-facing APIs.
Custom Fine-Tuning & Domain Specialization Without Upstream Constraints
Use OBLITERATED as a base for custom fine-tuning on proprietary data (legal documents, medical workflows, financial analysis) without inherited refusal layers that might interfere with legitimate domain-specific outputs. Ablation removes the refusal substrate entirely, giving your ops team cleaner weights to adapt.
Custom AI
As a base for custom AI
Strong foundation for custom ops-AI products that need unrestricted inference. Fine-tune on proprietary datasets (finance automation, legal analysis, medical workflows) without inherited safety behaviors blocking legitimate domain outputs. The base 7.9B parameter count is modestly sized—suitable for specialized tasks where you control training data. Use as embedding backbone + generation layer in internal knowledge systems, compliance tools, or process automation where your team sets the safety boundaries.
In the operating system
Where it fits
Fits as the unrestricted generation layer in an ops-AI stack: sits below knowledge retrieval (docs/logs fed as context), workflow orchestration (agents call it for decisions), and internal APIs. Skips the safety/moderation layer entirely—you own guardrails at the application level (prompt design, output filtering, approval gates). Integrates with agent frameworks (LangChain, AutoGen) for autonomous ops tasks.
Data control & security
Self-hosting eliminates vendor observation: no API logs, no telemetry, no external processing of queries or context. Data ingestion, inference, and output remain within your infrastructure boundary. Model weights are Apache 2.0—you control deployment, versioning, and modification. Clarification: self-hosting architecture provides *data isolation*, not model-level security guarantees. The model is an ablated version of Gemma 4; security posture depends on your inference environment, not the model card. You're responsible for access controls, network isolation, and output validation.
Hardware footprint
**Estimate (varies by setup):** - **Q4_K_M (4.9 GB):** ~6–8 GB system RAM (GGUF + overhead), runs on 4GB RAM devices with swap; CPU inference slow (~10–30 tok/sec). - **Q5_K_M (5.3 GB):** ~7–9 GB RAM; 4–8 tok/sec CPU, ~30–60 tok/sec on GTX 1660 or equivalent. - **Q8_0 (7.4 GB):** ~9–11 GB RAM; better quality, slower inference (~5–10 tok/sec CPU, ~40–80 tok/sec on consumer GPU). - **Full bfloat16 (17 GB):** ~20–24 GB RAM + GPU with ≥12GB VRAM for practical speed. Test quantization on your target hardware; actual performance depends on batch size, context length, and inference engine tuning.
Integration
Runs via standard inference APIs: Ollama (REST endpoint), llama.cpp (C library + CLI), or Hugging Face `transformers` library. Wire into ops stacks via: FastAPI wrapper (expose local inference as HTTP), LangChain agent tools (e.g., `Tool(func=query_model, name='analysis')`), or direct Python inference in orchestration scripts. Quantized GGUFs load in ~4–8 seconds; typical inference ~50–200 tokens/sec on consumer hardware. Chat template required—don't use raw completion mode. Requires recent inference-engine builds (check compatibility table in model card).
When it's not the right fit
- —Safety-critical ops tasks requiring auditable refusal behavior. OBLITERATED intentionally removes refusals; if compliance rules require demonstrable safety guardrails, this model introduces liability.
- —Multi-language workflows. Model card reports ~4% wrong-language outputs (Thai/Japanese) and ~28% soft deflection on complex topics—insufficient for production multilingual ops.
- —Extreme long-context reasoning (>2k tokens). Model card does not specify context length; 4B-equivalent parameter count suggests weak long-range coherence on very long documents.
- —High-uptime SLA requirements without in-house inference expertise. GGUF compatibility changed in recent llama.cpp releases; requires ops team to manage version pinning and dependency updates.
Alternatives to consider
Llama 2 (7B, Meta, Apache 2.0)
Smaller, battle-tested, broader ecosystem support. Retains safety training (won't need ablation if you want guardrails); slower custom fine-tuning because safety is baked in. Better for teams wanting standard open-source + community.
Mistral 7B (Mistral AI, Apache 2.0)
7B baseline with cleaner training. No ablation; compatible with standard inference stacks. Lighter guardrails than Gemma 4 OG, so less need for refusal removal. Broader industry adoption for ops workflows.
Hermes 3 (NousResearch, MIT)
Instruction-tuned, designed for agent tasks and unrestricted reasoning. Officially supports role-play and edge-case scenarios; no ablation needed. Better for autonomous workflow agents; slightly larger (70B variants available for high-stakes decisions).
Related open models
FAQ
What's the difference between OBLITERATED and the original Gemma 4 E4B?
Original Gemma 4 refuses ~98.8% of requests; OBLITERATED has 0% hard refusal via surgical ablation of 21 layers. Coherence quality is similar (4B parameter baseline limits both), but OBLITERATED will not block any query. Use if you need unrestricted inference; use Gemma 4 OG if you want vendor safety layers.
Can I run this privately on-prem without external APIs?
Yes—entire premise. Download GGUF, run Ollama/llama.cpp locally, no internet required after initialization. Model, weights, and inference all stay in your environment. Data never touches Hugging Face, OpenAI, or any vendor. You manage security, access, and output validation.
Is this legal to use commercially?
License is Apache 2.0 (permissive, OSI-approved)—commercial use is explicitly allowed. However: OBLITERATED is a refusal-removal derivative; you own model behavior and liability. Ensure your use case complies with local law (export controls, content policies). If your use case would fail under Gemma 4 OG's terms, check legal before deploying.
What happens if I use the wrong inference engine version?
Gemma 4 architecture requires recent llama.cpp (b8665+), Ollama 0.20+, or updated LM Studio. Older versions will error 'unsupported architecture' or load with gibberish output. Check compatibility table in model card; update your inference engine. v3 fixes tensor corruption from earlier ablation bugs, so use v3 (not v2).
Build Unrestricted Ops AI on Your Own Infrastructure
OBLITERATED works best when integrated into a full ops-AI stack. LLM.co helps you wire private LLMs into knowledge layers, agent frameworks, and workflow automation—keeping data in your environment while controlling behavior at the application level. Explore how to deploy custom AI systems that scale with your ops.