Open-Weight LLM · Private & Custom AI
Qwythos-9B-Claude-Mythos-5-1M
A 9B reasoning model with 1M context and native tool-use—built for private deployment in ops automation, agentic workflows, and custom AI systems that need to stay in your infrastructure.
Qwythos-9B is a full-parameter fine-tune of Qwen3.5-9B trained on 500M+ tokens of reasoning traces, shipping with YaRN-enabled 1M-token context and function-calling support out of the box. For ops teams, it's a compact reasoning engine that can be self-hosted, wired into private tool-use loops (Python execution, web search, internal APIs), and extended as a foundation for custom applications without leaving your environment.
Model facts
Private deployment
Run Qwythos-9B-Claude-Mythos-5-1M in your own environment
Self-hosting Qwythos keeps inference and all intermediate reasoning inside your infrastructure—no data egress to external APIs. At 9.4B parameters, it runs on a single high-end GPU (H100/H200 handles full 1M context; an A100 or RTX 6000 covers 256–512k practical context). Deployment via vLLM or SGLang is straightforward; YaRN context scaling is baked into config.json. Trade-off: serving at 1M tokens demands multi-GPU or aggressive KV-cache offload; shorter contexts (256k and below) run efficiently on single-GPU setups. For ops workflows that don't need the full 1M window, this becomes a practical 256–512k private reasoning engine that keeps proprietary data, customer records, and internal docs fully contained.
Operational AI use cases
Agentic support and incident response
Deploy Qwythos as the reasoning backbone for internal support agents. The model natively handles function calls (ticket lookup, knowledge-base search, Python execution for log parsing, Slack API integration). When a support ticket arrives, the agent reasons through the issue, calls tools to gather context, and generates a draft response—all without leaving your infrastructure. 1M context means the entire ticket history, related incidents, and runbooks stay in a single forward pass.
Codebase reasoning and documentation automation
Feed entire repositories or large codebases (100k–500k lines) into a single 1M-token context window for refactoring analysis, defect-finding, or architectural review. Qwythos produces detailed reasoning traces, proposes fixes, and can invoke a code-execution tool to validate changes. No chunking, no RAG overhead—reasoning happens over the whole codebase in one prompt, with full chain-of-thought visible.
Private document synthesis and compliance review
Ingest regulatory docs, policies, contracts, or research papers (10–30 documents per session) into a single prompt. Qwythos synthesizes across all of them, answers compliance questions, flags risks, and generates summaries—all reasoning stays private. The model's uncensored stance means it can discuss sensitive compliance edge cases without over-hedging or refusing nuanced questions.
Custom AI
As a base for custom AI
Qwythos is a strong foundation for custom AI products that need reasoning, tool-use, and long context. Fine-tune it on your domain data (biomedical, security research, technical support, financial analysis) and deploy as a private white-label service. Its native function-calling matches OpenAI's spec, so wiring it into your tool ecosystem is straightforward. The uncensored base means it won't refuse technically demanding questions—important for security research, medical decision-support, and adversarial-analysis products where aligned models tend to hedge. At 9B parameters, it's small enough to embed in customer environments, large enough to handle reasoning tasks that 7B models struggle with.
In the operating system
Where it fits
In an AI operating system, Qwythos occupies the **reasoning and agentic agent layer**. It sits above retrieval and data ingestion (it consumes documents, codebases, and tool outputs) and connects to the **workflow orchestration layer** (managing multi-turn conversations, tool scheduling, state persistence). Deploy it as the inference backbone for long-context retrieval-augmented generation (RAG), autonomous agents that manage internal processes (support, ops, compliance), and custom applications that need to reason over proprietary data without external API calls. Its 1M context bridges the gap between short-context chat models and full-document reasoning systems.
Data control & security
Self-hosting Qwythos means your inference compute, all intermediate reasoning states, and model activations never transit external networks or third-party APIs. This is an **architectural choice**—data remains in your environment. The model itself carries no inherent encryption or compliance claims; security depends on your infrastructure (network segmentation, access controls, GPU isolation). For regulated workloads (healthcare, finance, government), private deployment eliminates data-residency concerns that arise with closed APIs. Reasoning traces—which can reveal proprietary information about your processes—stay entirely internal. Note: this is a control benefit, not a compliance guarantee; you remain responsible for auditing your deployment.
Hardware footprint
**Estimated VRAM by precision (single GPU, no quantization):** - **bfloat16 (recommended)**: ~22–24 GB (H100, H200, RTX 6000 Ada, A100 80GB) - **float16**: ~18–20 GB (A100 80GB, L40S 48GB works tight) - **int8 quantized**: ~12–14 GB (A100 40GB, RTX 6000, L40S 48GB) - **int4 quantized (GPTQ/AWQ)**: ~6–8 GB (A6000, L40, RTX 4090) For full 1M-token context inference, use multi-GPU or KV-cache offload. For practical 256–512k context, a single A100 or H100 is sufficient in bfloat16. Quantized versions trade some reasoning quality for lower VRAM but remain viable for ops use (benchmarks on quantized variants unknown; verify on your workload).
Integration
Qwythos integrates via standard LLM inference APIs (vLLM OpenAI-compatible endpoint, SGLang, text-generation-webui). Function calling follows Qwen3.5 spec: pass `tools=[{"type": "function", "function": {...}}]` and parse `<tool_call>` blocks from output. Wire it into your orchestration layer to manage multi-turn sessions, tool invocations, and state (prompting frameworks like LangChain, LlamaIndex, or custom Python handle this). For ops workflows: integrate with your internal APIs (ticket systems, knowledge bases, code repos, logs) via custom tool definitions. A typical ops deployment includes Qwythos inference + a lightweight agent loop + integrations to your existing tools (Python subprocess for code execution, your company's internal APIs for business logic).
When it's not the right fit
- —Your workflows demand absolute maximum short-context fidelity and never exceed 256k tokens—YaRN rope-scaling introduces a small quality trade-off on native-length prompts; if you need that headroom, you can revert to default rope in config.json, but benchmarks on that variant are not provided.
- —You need guaranteed compliance certifications (SOC 2, ISO 27001, HIPAA) baked into the model—Qwythos is just the inference engine; compliance is a deployment and infrastructure responsibility, not a model property.
- —Your ops environment requires fully censored, maximally-hedging responses—Qwythos is intentionally uncensored and will engage with red-teaming, security research, and sensitive medical/pharmacological topics that over-aligned models refuse. For risk-averse organizations, this is a liability.
- —You need deterministic, fully reproducible output for legal/audit purposes—language models, including Qwythos, are inherently stochastic; reasoning traces vary across runs. For use cases requiring bit-for-bit reproducibility, this is unsuitable.
Alternatives to consider
Llama 3.1 8B / 70B
Broader community support, more mature quantization ecosystem, and easier multi-GPU serving. Shorter native context (8k native, 128k with scaling). Less reasoning-optimized; no native function-calling in 8B. Better for general-purpose ops if you don't need long context or tool-use out of the box.
Mixtral 8x7B
Similar size class, better MMLU (+0.40 vs Qwythos's +0.575), sparse MoE architecture reduces per-token compute. No native 1M context; requires more careful serving setup. Good if you prioritize throughput and multi-model parallelism over single-model reasoning depth.
Claude 3.5 Sonnet (closed API)
Proprietary alternative: larger context (200k), more capable reasoning, hosted by Anthropic. Data leaves your environment; no private deployment. Use this if compliance/risk aversion outweighs data-residency concerns, or if you need the absolute best reasoning for one-off tasks (custom AI use case advantage goes to Qwythos).
Related open models
FAQ
Can I run Qwythos entirely on-premises without any external API calls?
Yes. Deploy via vLLM or SGLang on your own GPU infrastructure, integrate it with your internal tools (Python execution, internal API calls), and all inference, reasoning, and tool-use stays private. No data or inference logs transit external services.
Is Qwythos licensed for commercial use in a closed product?
Qwythos is Apache 2.0 licensed, which permits commercial use, distribution, and modification without restriction. You can fine-tune it, distribute it (with attribution), and embed it in commercial products or services, provided you comply with Apache 2.0 terms (license + copyright notice in distribution). No special commercial licensing required. Verify your specific use case, but the license is permissive.
What's the difference between the full 1M context and the 256k practical limit you mention?
The 1M context is real—YaRN scaling is configured and tested. However, a single H100 can comfortably serve ~256–512k tokens per request; the full 1M window needs multi-GPU parallelism or offloading KV cache to host memory (slower). For most ops workflows (support agents, codebase reasoning, document synthesis), 256–512k is plenty. If you need the full 1M, plan for 2–4 GPU setup or aggressive optimization.
Does Qwythos have any built-in safety guardrails or refusal mechanisms?
No—Qwythos is intentionally uncensored. It will engage with questions about security research, red-teaming, pharmacology, clinical edge cases, and other sensitive topics that over-aligned models tend to refuse. For ops teams, this is a feature (better reasoning on real-world problems); for risk-averse organizations, this is a liability. If your ops environment requires refusal-style safety, consider a more aligned model.
Build private AI workflows with Qwythos.
Qwythos is built for self-hosted deployment. LLM.co helps you wire it into your ops stack—agentic support, private document reasoning, custom AI products. Let's architect your system.