Open-Weight LLM · Private & Custom AI
Qwen3-235B-A22B
235B MoE model with switchable reasoning modes for private deployment—designed for ops teams building custom reasoning agents and automating complex workflows while keeping data in-house.
Qwen3-235B-A22B is a mixture-of-experts causal LM with 22B activated parameters, native 32K context (expandable to 131K), and a unique thinking/non-thinking toggle for reasoning-heavy and efficiency-focused tasks. Ops and AI teams use it to build internal agents, automate document workflows, run compliance-sensitive logic, and deploy reasoning workloads on private infrastructure.
Model facts
Private deployment
Run Qwen3-235B-A22B in your own environment
Self-hostable via transformers, vLLM (≥0.8.5), or SGLang (≥0.4.6.post1); Ollama, LMStudio, and llama.cpp also supported. Estimated 470–940 GB VRAM (fp8–fp32). A company deploys it on-premise to keep financial analysis, customer support reasoning, code review, and proprietary knowledge workflows in their own data centers—no third-party API calls, no model weights leaving your environment.
Operational AI use cases
Internal Support & Knowledge Triage
Route customer/employee inquiries to the right department by reasoning through ticket text, routing rules, and KB context. Thinking mode analyzes intent; non-thinking mode handles high-volume categorization. Data stays private; reasoning is transparent via `<think>` blocks for audit.
Code & Document Review Automation
Activate thinking mode to reason through pull requests, internal policies, or compliance docs—flag violations, suggest fixes, log reasoning. Use non-thinking mode for rapid, routine checks. Keeps sensitive code and policy language entirely on-premise.
Financial & Risk Workflows
Reasoning mode analyzes ledger anomalies, contract terms, and risk signals; non-thinking mode runs fast classification on routine transactions. 235B capacity handles complex multi-document reasoning; private deployment satisfies audit and data residency requirements.
Custom AI
As a base for custom AI
Strong base for building proprietary AI products: reasoning backbone for internal tools (legal doc analyzers, compliance engines, technical RFP evaluators), or package as a private service for customers. MoE activation strategy lets you tune latency vs. reasoning depth per deployment. Chat template and thinking/non-thinking toggle give you control over inference behavior without retraining.
In the operating system
Where it fits
Sits at the **reasoning and agent core** of an ops AI OS. Acts as the primary inference engine for multi-turn workflows, decision logic, and tool integration. Feed it from knowledge/document layer (retrieval, memory); chain it with workflow automation (ticketing, CRM, finance systems) and external tools (APIs, databases). Thinking mode adds explainability for audit logs and compliance layers.
Data control & security
Private self-hosting keeps all prompts, reasoning traces, and responses in your environment—no data sent to Qwen or third parties. Thinking output (inside `<think>` blocks) and final responses remain under your control for encryption, retention, and compliance policies. Model weights are Apache 2.0 licensed and can be audited. Note: self-hosting does not automatically ensure compliance; you are responsible for access control, encryption in transit/rest, and policy enforcement.
Hardware footprint
**Estimate (on-premise):** - **fp32 (full precision):** ~940 GB VRAM - **fp16 (half-precision):** ~470 GB VRAM - **int8 (quantized):** ~235 GB VRAM - **int4 (aggressive quant):** ~60–120 GB VRAM (quality trade-off) MoE routing adds overhead; expect 10–20% memory variance. Suitable for large multi-GPU clusters (8×80GB H100 + infrastructure) or quantized inference on fewer high-VRAM nodes.
Integration
Standard transformers API; chat templates handle system/user roles and thinking toggle. vLLM and SGLang expose OpenAI-compatible endpoints for drop-in integration with existing ops tools (Zapier, Make, custom APIs). Thinking parser available in both frameworks. Context window (32K native, 131K with YaRN) suits long documents and multi-turn agent loops. Requires transformers ≥4.51.0 and tokenizer support for qwen3_moe architecture.
When it's not the right fit
- —Low-latency edge inference required—235B model is too large for mobile/on-device; needs data-center hardware.
- —Constrained on thinking/reasoning transparency—if audit trails and explainability are not a priority, a smaller, faster model (Qwen2.5) is cheaper and adequate.
- —Real-time streaming to end-users—thinking mode delays response; non-thinking mode is faster but not optimized for sub-100ms latency on commodity hardware.
- —Limited infrastructure budget—deployment cost (GPUs, cooling, scaling) may exceed API-based alternatives for low-volume ops workflows.
Alternatives to consider
Mixtral 8x22B (Mistral)
Similar MoE scale, lower total params (141B), Apache 2.0 license, strong reasoning but no native thinking/non-thinking toggle. Better for teams valuing simplicity over reasoning mode flexibility.
Llama 3.3 70B (Meta)
Dense, smaller footprint (~140 GB fp16), Llama 2 license, strong instruction-following and multilingual support. Pick if you need less reasoning overhead and tighter resource control.
DeepSeek-R1-Distill-Qwen (DeepSeek)
Distilled reasoning model, lower params, faster inference, Apache 2.0. Use if reasoning is core but you want to trade off Qwen3's 235B capacity for smaller size and lower latency.
Related open models
FAQ
Can we run Qwen3-235B-A22B entirely on-premise without any cloud calls?
Yes. Deploy it via vLLM, SGLang, or transformers on your own hardware and expose an internal OpenAI-compatible API. All reasoning, thinking traces, and responses stay in your data center. No internet calls needed (except initial model weights download).
What's the difference between thinking and non-thinking mode?
`enable_thinking=True` (default): model generates `<think>...</think>` blocks for complex reasoning, math, and code—slower but more accurate on hard problems. `enable_thinking=False`: model skips thinking, responds directly like Qwen2.5—faster, suitable for high-volume classification and Q&A.
Can we use Qwen3-235B-A22B in a commercial product or internal app?
Yes. Apache 2.0 license permits commercial use, redistribution, and modification. You can embed it in proprietary applications (SaaS, internal tools, client-facing products). Include a copy of the Apache 2.0 license. Review the full license terms if you modify the weights or bundle it with other restrictive-license software.
How do we handle long documents (contracts, policies) with this model?
Native context is 32K tokens (~24K words). For longer docs, use YaRN extension to reach 131K tokens (~98K words). Split longer documents into chunks, use retrieval/summarization to feed relevant sections, or run multiple inference passes. Thinking mode helps reason across document fragments.
Build Custom AI on Your Infrastructure
Qwen3-235B-A22B gives you reasoning capability and private control. Learn how LLM.co helps ops teams deploy open-weight models, integrate them into workflows, and manage them at scale. Let's architect a private AI system for your team.