Open-Weight LLM · Private & Custom AI
Ornith-1.0-35B-GGUF
Purpose-built coding agent for private deployment—automate development ops, code review, and software engineering workflows without external API dependencies.
Ornith-1.0-35B is a post-trained, MIT-licensed agentic coding model optimized for terminal tasks, software engineering benchmarks (SWE-Bench, Terminal-Bench), and multi-step reasoning. Designed for single-GPU efficiency, it's a candidate for ops teams automating developer workflows, code generation, and agent-based task resolution in a self-hosted environment.
Model facts
Private deployment
Run Ornith-1.0-35B-GGUF in your own environment
At 35B parameters in GGUF format, it runs on consumer/mid-market single-GPU hardware (estimate: 70–90 GB VRAM for full precision, ~20–30 GB quantized). Self-hosting keeps code, prompts, and internal development context in your environment—critical for teams handling proprietary codebases. Trade-off: inference latency vs. data residency; requires local orchestration (vLLM, Ollama, or custom inference server) and monitoring.
Operational AI use cases
Automated Code Review & Issue Triage
Route pull requests to Ornith for syntactic/logical flagging, bug detection, and refactor suggestions. Model reads repo context and generates structured feedback, reducing manual review load. Output integrates with GitHub/GitLab webhooks; company retains all code and reasoning logs.
Internal Developer Support Bot
Deploy as a private Slack/Teams agent answering questions about internal APIs, deployment procedures, and legacy codebase navigation. Feeds on company wiki + repo structure; no data leaves your infrastructure. Reduces tier-1 ops support tickets for common dev questions.
Operational Incident Resolution & Runbook Generation
Given error logs, stack traces, and service metrics, Ornith generates diagnostic steps and runbooks. Fine-tune on your incident history and alerting schema. Speeds triage for ops teams; all reasoning and logs stay internal.
Custom AI
As a base for custom AI
Strong foundation for custom ops AI: fine-tune on internal code patterns, company-specific coding standards, or domain-specific terminal tasks (build orchestration, infrastructure-as-code). GGUF format and agentic training enable lightweight customization and agent loop integration (ReAct, function-calling chains). Requires dataset curation (code examples, expected outputs) and inference infrastructure.
In the operating system
Where it fits
**Knowledge layer**: ingests company repos, docs, logs. **Agent layer**: executes multi-step coding tasks (search → reason → generate → validate). **Workflow layer**: triggers on CI/CD events, support tickets, or scheduled ops tasks. Sits between your data warehouse and downstream systems (GitHub, Jira, monitoring tools).
Data control & security
Self-hosting is an **architectural choice**: all code, prompts, and model outputs remain in your VPC/datacenter. No inference telemetry to external APIs. Compliance benefits (HIPAA, SOC2) depend on your infrastructure security posture, not the model. Responsibility for access control, audit logs, and model updates falls entirely on your ops team.
Hardware footprint
**Estimate (unverified):** Full precision (fp32): ~140 GB VRAM. fp16/bfloat16: ~70 GB. GGUF Q4_K_M quantization: ~20–25 GB. Single A100 (40GB) requires quantization or distributed inference; dual L40S (96GB combined) handles fp16 comfortably. Throughput ~5–15 tokens/sec per GPU depending on batch size and precision.
Integration
GGUF quantization simplifies deployment via vLLM, Ollama, or Hugging Face TGI. Expose via OpenAI-compatible API endpoint for drop-in tool use. Integrate with GitHub Actions (code review on PR), Slack bots (dev support), or observability platforms (incident response). Requires orchestration layer (e.g., Prefect, Airflow) for multi-step workflows and state management.
When it's not the right fit
- —Non-coding tasks: model is specialized for agentic coding; general chat/knowledge tasks likely underperform vs. generalist models.
- —Real-time low-latency inference: 35B model adds ~1–5 sec per inference; unfit for sub-100ms response SLAs.
- —Minimal compute budget: requires 20+ GB VRAM quantized; not viable on edge devices or CPU-only infrastructure.
- —Ambiguous benchmarking: terminal-bench and SWE-bench scores are tool-specific; performance on your internal codebase unvalidated without testing.
Alternatives to consider
DeepSeek-Coder-33B
MIT-licensed, strong code generation, lighter parameter count; less agentic reasoning but faster inference. Better for pure code completion vs. multi-step task scaffolding.
Qwen3.5-35B (base)
Similar scale, broader generalist capability; benchmark comparison in Ornith's card shows Ornith ahead on coding tasks. Choose if you need broader task coverage.
Phi-4 / Phi-3.5 (14B–16B)
Lighter, MIT-licensed, runs on single 40GB GPU; less specialized for coding agents but significantly lower operational cost. Trade task complexity for deployment simplicity.
Related open models
FAQ
Can we run this entirely on-premises, with no external calls?
Yes. Deploy the GGUF on your VPC using vLLM or Ollama, expose an OpenAI-compatible endpoint, wire into your CI/CD. All inference and logging stay internal. You own model updates and parameter tuning.
Is this MIT-licensed safe for commercial use?
MIT permits commercial use, modification, and distribution—permissive and OSI-approved. No royalties or restrictions. Review your legal/compliance team for derivative liability if you fine-tune; model itself is unencumbered.
How do we fine-tune Ornith on our internal codebase?
Unknown. Card does not document fine-tuning procedures, LoRA support, or training data format. Requires contact with deepreinforce-ai or reverse-engineering from base (Gemma/Qwen) documentation. Budget for experimentation.
What's the inference cost vs. calling Claude or GPT-4?
Self-hosted: amortized GPU cost (~$0.50–$2.00 per 1M tokens for on-premise hardware). No per-token API fees. Tradeoff: upfront infrastructure, monitoring, and DevOps toil. Breakeven ~2–5M tokens/month depending on cluster utilization.
Build Your Private AI Ops System
Ornith-1.0-35B is a strong building block for custom coding agents and ops workflows. LLM.co helps you orchestrate Ornith (or similar open models) into your existing tools—GitHub, Jira, observability stacks—while keeping all data and reasoning private. Let's architect your AI ops layer.