Open-Weight LLM · Private & Custom AI
Ornith-1.0-35B-FP8
Specialized agentic coding model for private deployment—automates software engineering workflows (repo navigation, bug fixing, code generation) entirely within your environment.
Ornith-1.0-35B is a 35B-parameter Mixture-of-Experts model fine-tuned for coding agents and tool-use tasks, trained via reinforcement learning to scaffold and solve complex engineering problems. For ops teams, it's a self-improvable foundation for building internal code-automation systems, CI/CD integrations, and documentation/refactoring agents—all running on your own infrastructure with zero data leakage to third parties.
Model facts
Private deployment
Run Ornith-1.0-35B-FP8 in your own environment
Ornith-35B can run on a single GPU in FP8 quantization (estimated ~20–25 GB VRAM). Deploy via vLLM, Ollama, or Hugging Face TGI in your own data center or VPC. This architecture keeps all code repositories, internal schemas, and execution logs entirely within your control—critical for compliance-sensitive engineering teams or those with proprietary codebases. FP8 quantization trades marginal quality for practical single-GPU operation; verify token throughput against your SLA before production.
Operational AI use cases
Automated Code Review & Refactoring
Deploy Ornith as an internal agent that scans pull requests, flags antipatterns, suggests refactors, and auto-fixes style violations before human review. Wired to your Git server and CI/CD pipeline, it reduces manual code-quality overhead and maintains your internal standards without exposing code to external services.
Internal Bug-Fix Automation (SWE-Bench Workload)
Use Ornith to triage and attempt repairs on known bugs in your monorepo or internal libraries. Feed it issue descriptions + stack traces; it navigates codebases (NL2Repo capability), proposes fixes, and generates test cases. Route high-confidence patches directly to staging; hand off uncertain ones to engineers, reducing triage time.
Documentation & Knowledge-Base Generation
Ornith reads your codebase and auto-generates or updates API docs, migration guides, and architecture decision records. Trigger it on code commits or on-demand. Store artifacts in your wiki/knowledge repo. Keeps internal documentation in sync with code without manual overhead.
Custom AI
As a base for custom AI
Strong candidate for custom AI applications where you own the training loop. Ornith's self-improving framework (RL-driven scaffold + solution co-optimization) is a foundation for building domain-specific coding agents: e.g., agents that learn to navigate *your* codebase topology, or agents fine-tuned on your internal bug patterns. Use it as a base model, add LoRA or full fine-tuning with your own code+solution pairs, then deploy the specialized version privately.
In the operating system
Where it fits
Ornith sits in the **Agent layer** of an ops-AI stack. It's the reasoning engine for autonomous workflows (code review, bug repair, docs generation). Pair it with a retrieval/indexing layer (to embed your codebase) and workflow orchestration (Temporal, Prefect) to build a complete internal software-engineering autopilot. Not suitable as a general-knowledge layer; use it downstream of routing logic that sends coding tasks to it.
Data control & security
Private deployment means your code, tests, and execution traces never leave your network—a major win for regulated industries, proprietary systems, or teams with IP-sensitive work. You control inference logs, fine-tuning data, and model outputs. However, deploying an LLM in-house introduces operational responsibilities: you own security patching, inference node hardening, and audit logging. Model weights are MIT-licensed open-source; this does not imply the model is "secure by default"—assume you apply your own access controls and monitoring.
Hardware footprint
**Estimate**: ~22–26 GB VRAM (FP8 quantized on single H100 or A100-80GB). Full FP16 would require ~70+ GB (multi-GPU setup). Measured inference throughput will depend on token length, batch size, and your hardware; assume 10–50 tokens/sec per GPU for agentic loops with tool calls. Requires testing in your environment before committing to SLA.
Integration
Ornith works with HF Transformers and is compatible with vLLM endpoints (tagged `endpoints_compatible`). Wire it to your Git API (GitHub/GitLab), CI/CD system (Jenkins/GitHub Actions), and internal knowledge store (Confluence/docs) via FastAPI or gRPC. Use LangChain or LlamaIndex to add ReAct-style tool-use loops; connect it to your code search/indexing backend (e.g., Sourcegraph, Uber's CodeSearch) to ground reasoning in your actual repo. For serious ops, containerize via Docker/K8s and integrate with your observability stack (logs, traces, metrics).
When it's not the right fit
- —Your use case is general-purpose chat/Q&A (use a balanced model like Llama 3.1; Ornith is over-specialized for coding and may underperform on other domains).
- —Context length matters critically (model card does not specify max context; verify against Terminal-Bench and SWE-Bench test setups before using on very large repos).
- —You lack GPU infrastructure or want CPU-only inference (Ornith-35B is not practical on CPU; consider smaller dense variants like the 9B).
- —You need real-time latency SLA <500ms per token (coding agents are inherently slow; tool-use loops add overhead; validate throughput in your stack before production).
Alternatives to consider
DeepSeek-Coder-33B
Another specialized coding model; smaller footprint, strong on code generation and completion. Less emphasis on agentic reasoning; better for code-completion pipelines than autonomous bug-fix loops.
Qwen 3.5-35B-Instruct
Base model Ornith was fine-tuned on. More general-purpose; weaker on Terminal-Bench/SWE-Bench tasks but available in standard instruct format. Use if you need flexibility across coding + non-coding tasks.
Llama 3.1-70B
Larger, more general, strong multi-task performance. Better for mixed ops workloads; requires more GPU capacity. Not specialized for agentic coding but a safer fallback for diverse ops use cases.
Related open models
FAQ
Can I fine-tune Ornith on my internal codebase and keep it private?
Yes. MIT license permits fine-tuning and commercial use. Download weights, run training (LoRA or full) on your infrastructure with your code examples, then deploy the adapted model in-house. This is exactly the intended workflow for custom AI at LLM.co.
Is Ornith MIT-licensed for commercial use?
Yes. MIT is permissive; you can use Ornith in commercial products, proprietary systems, and internal operations without attribution (though attribution is polite). No license restrictions on self-hosting or resale. Verify the model weights host (HuggingFace) and quantization source for any secondary terms.
What's the context length, and can it handle my 50k-line repo?
Model card does not specify max context. SWE-Bench benchmarks suggest it can handle multi-file tasks, but exact context window is unknown. Test with a slice of your codebase at inference time. If context is the bottleneck, pair Ornith with a retrieval system (embed + search your repo, feed only relevant code snippets to the model).
How do I deploy this in a Kubernetes cluster for team access?
Containerize using vLLM or TGI Dockerfile, push to your registry, deploy via Helm/Kustomize on your K8s cluster. Expose via gRPC or HTTP. Add RBAC, network policies, and audit logging. Test throughput under load. LLM.co can help architect the full ops-AI stack around this.
Build Your Own Autonomous Coding Agent
Ornith is a powerful foundation—but integrating it into your ops workflows requires architecture. At LLM.co, we help teams deploy and fine-tune open-weight models like Ornith in production environments, wiring them to your codebase, CI/CD, and internal tools. Let's design a custom AI system that keeps your code and data private.