Open-Weight LLM · Private & Custom AI
Ornith-1.0-35B-MTP-APEX-GGUF
A self-improving agentic coding model optimized for private deployment: reasoning-heavy automation of software engineering workflows inside your infrastructure.
Ornith-1.0-35B is a Qwen3.5 MoE variant (35B total, 3B active per token) post-trained via RL for scaffold generation and solution rollouts. Built for agentic coding tasks—Terminal-Bench, SWE-Bench, NL2Repo—it ships as GGUF (quantized, multimodal-ready) under MIT license. For ops teams, it's a foundation for private bug-triage agents, code review automation, and internal tool generation without cloud API dependency.
Model facts
Private deployment
Run Ornith-1.0-35B-MTP-APEX-GGUF in your own environment
Deploy via llama.cpp or compatible GGUF-supporting runtime. APEX-I-Compact (15.85 GB) runs on mid-tier GPUs (RTX 4070–5070 Ti range). No multimodal dependencies forced—vision projector (mmproj) is modular. Architecture: pull weights, load quantized model, wire to your internal API/agent framework. Data stays in-environment; no model telemetry or external inference calls. Trade-off: MoE architecture requires more memory headroom than dense models at equivalent capability.
Operational AI use cases
Bug Triage & Prioritization Agent
Feed issue tickets, stack traces, and code diffs to Ornith running on-premise. The model scores root-cause likelihood, routes to the right team, and drafts severity justifications. No external API keys; reasoning stays in your VPC. Reduces noise in support intake by ~40–60% (benchmark-adjacent, verify with your data).
Internal Code Review Automation
Embed Ornith in your CI/CD pipeline to pre-review PRs: security smell-checks, refactoring suggestions, test-coverage gaps. Self-hosted means code never leaves your network. Frees senior engineers from repetitive style checks and focuses human review on architecture.
Documentation & Runbook Generation
Snapshot your codebase, internal libraries, and architecture docs; feed to Ornith with a task prompt. Generate first-draft runbooks, migration guides, or API usage examples. MTP layer and multimodal support let you include diagrams or architecture charts. All processing local—no risk of training data leakage.
Custom AI
As a base for custom AI
Strong foundation for a custom ops AI product. MoE routing lets you fine-tune expert weights on domain-specific coding tasks (e.g., your company's microservices, tech stack, security patterns). MIT license permits derivative builds and commercial products. Vision projector support enables screenshot-based debugging agents. Extended 262K context window absorbs full codebases and multi-file reasoning. Requires investment in prompt engineering and RL tuning to match internal workflows; not off-the-shelf.
In the operating system
Where it fits
**Knowledge layer**: Reason over code, logs, and docs. **Agent layer**: Route tasks to experts (8 of 256); scaffold complex workflows. **Workflow automation**: native support for tool-calling; integrates with your ops/ticketing APIs via agent orchestration. Not a chat model; expect to wrap it in ReAct or similar reasoning loops. Complements a traditional knowledge base (RAG) by adding inference-time problem-solving.
Data control & security
Self-hosting eliminates API calls; code and business logic remain in your environment. No cloud logging, no third-party model inference. Architecture choice, not inherent to the model: you control compute, storage, and network access. Quantization (GGUF) reduces memory footprint, making isolation on dedicated VMs easier. Compliance (SOC 2, HIPAA, etc.) depends on your deployment topology, not the model. Audit the base architecture (Qwen3.5, MTP layer sources) for supply-chain trust; model card is transparent.
Hardware footprint
**ESTIMATE (APEX-I-Compact, 15.85 GB)** - BF16: ~70 GB VRAM - Q8_0: ~35–40 GB VRAM - Q5_K_M: ~22–25 GB VRAM - Q4_K_M: ~16–18 GB VRAM (benchmarked, fits RTX 4070 with headroom) MoE architecture: even quantized, requires higher memory than dense models (e.g., Llama 70B Q4 is ~35 GB but uses fewer experts). Batch size, context length, and vision features scale linearly. Estimate +10–20% overhead for runtime buffers and KV cache.
Integration
Expects GGUF loader (llama.cpp, Ollama, vLLM with GGUF support). Tool/function calling: compatible with standard JSON-schema approach (function definitions as context or system prompt). Agentic workflows: wire to orchestrators like LangChain, AutoGPT-style loops, or internal task queues. Vision input: requires llama.cpp + mmproj-F16.gguf + image preprocessing (standard; not specialized). Production: use batching where possible; MoE routing adds ~10–15% latency overhead vs. dense models. Monitor expert load balance for cost optimization.
When it's not the right fit
- —Real-time latency < 500ms required: MoE routing + reasoning overhead unsuitable for sub-second chat or high-frequency inference.
- —Small or no GPU available: 16 GB VRAM minimum; CPU-only inference will be slow. Not a lightweight edge model.
- —Non-English coding tasks are primary use: trained on English benchmarks (SWE-Bench, etc.). Multilingual code reasoning requires validation.
- —You need guaranteed determinism or audit logs: model behavior is stochastic; no built-in explanation chains. Reasoning opaque without external tracing.
Alternatives to consider
DeepSeek-Coder-33B (Apache 2.0)
Similar size, dense architecture, also coding-focused. Smaller memory footprint; no MoE overhead. Fewer experts, so less domain specialization potential.
Qwen3.5-32B (MIT)
Parent family, general-purpose. Smaller than Ornith; less RL for agentic tasks. Easier to fine-tune from scratch if you don't need pre-trained reasoning.
Llama-3.1-70B (Llama Community License)
Larger, denser, broader instruction-following. No MoE, higher memory cost. Stronger general knowledge; weaker on specialized coding agents.
Related open models
FAQ
Can I run Ornith entirely on-premises without any cloud services?
Yes. Load the GGUF weights onto a GPU or CPU server, use llama.cpp or compatible loader, and wire to your internal APIs. No calls to external services required. Data never leaves your network.
Is Ornith licensed for commercial products?
MIT license permits commercial use, derivative products, and distribution. No royalties or usage tiers. Review the license terms yourself; we do not provide legal counsel. Commercial use is not restricted by the license itself.
How do I fine-tune or distill Ornith for my company's codebase?
Start with the base Qwen3.5-35B weights or Ornith itself. Use LoRA or full fine-tuning on your code examples, internal APIs, and task patterns. MoE architecture complicates full retraining; LoRA on the dense transformer layers is more tractable. Requires ML infrastructure; not a one-click operation.
What is MTP, and does it matter for my ops use case?
MTP (Multi-Token Prediction, from Qwen3.5-35B-A3B) is a layer that predicts multiple tokens at once during training, improving inference speed and reasoning. For ops automation, it helps reduce time-to-first-token and improves coherence in long reasoning chains. Optional; not required to use the model.
Build Custom Agentic AI Inside Your Company
Ornith is a foundation, not a finished product. LLM.co helps middle-market companies fine-tune, integrate, and operationalize open-weight models like Ornith into proprietary ops AI—bug-triage agents, code review automation, documentation workflows. Stay private. Stay in control. Let's talk.