Open-Weight LLM · Private & Custom AI
North-Mini-Code-1.0
Sparse MoE code model (30B/3B active) for private agentic automation—build tool-calling workflows, code generation agents, and terminal tasks that stay in your environment.
North-Mini-Code is a 30B-parameter sparse Mixture-of-Experts transformer optimized for code generation, agentic software engineering, and terminal automation. It supports 256K context, tool use (bash, function calling), and interleaved reasoning—designed for companies building internal AI agents that need to stay off third-party infrastructure. The sparse architecture (8 of 128 experts active per token) balances capability with efficiency, making it viable for on-premise or private-cloud deployment.
Model facts
Private deployment
Run North-Mini-Code-1.0 in your own environment
Self-hosting North-Mini-Code requires ~20–40 GB VRAM (fp16) on a single GPU or multi-GPU setup; vLLM and Transformers both supported. Running it privately means your prompts, code snippets, and terminal commands never leave your network—critical for teams handling proprietary codebases, customer data, or regulated workflows. Cohere provides vLLM main-branch support and local inference examples (OpenCode integration noted), reducing dependency on hosted APIs. Setup requires transformer source + melody library for tool-call parsing; not turnkey, but standard for ops teams comfortable with containerized LLM deployment.
Operational AI use cases
Internal DevOps & Infrastructure Automation
Use North-Mini-Code as a private agent to audit infrastructure, generate deployment scripts, diagnose system issues via bash tool calls, and suggest fixes—all without exposing system logs or configs to external services. The model's terminal-bench performance and tool-use training make it suitable for auto-remediation workflows in CI/CD pipelines.
Software Engineering Ticket Triage & Code Review
Route incoming bug reports or feature requests to North-Mini-Code running locally. It analyzes code diffs, searches your codebase (via tool calls), generates patch suggestions, and auto-assigns or flags issues—keeping proprietary source code in-house while reducing triage overhead.
Knowledge Base & Runbook Generation
Feed internal documentation, incident reports, and operational procedures into a private RAG pipeline backed by North-Mini-Code. It generates runbooks, troubleshooting guides, and knowledge articles in real-time without exporting sensitive context to external LLM providers.
Custom AI
As a base for custom AI
North-Mini-Code is a strong foundation for building custom code-generation or agentic products. Its chat template, tool-use scaffolding, and SFT+RLVR training (verifiable rewards on coding tasks) allow you to fine-tune or prompt-engineer specialized agents: GitHub issue automation, internal API code generation, security-scan interpretation, or multi-step deployment orchestration. The 256K context window supports large codebases or multi-file reasoning, and Apache 2.0 licensing permits commercial use without royalty concerns.
In the operating system
Where it fits
North-Mini-Code operates in the **agent & workflow layer** of an AI OS. It's not a general-purpose chat model (use Llama 3.1 or Qwen for that); it's purpose-built for code understanding, tool use, and terminal automation. In a typical ops stack, it sits between a retrieval/knowledge layer (document indexing, codebase search) and an execution layer (CI/CD, bash, API integrations). Interleaved thinking supports multi-step agentic loops, making it suitable for orchestrating complex operational tasks.
Data control & security
Private deployment keeps all code, logs, and terminal output in your environment—no third-party LLM provider sees your data. This is an **architecture choice**, not a guarantee from the model itself. For compliance-sensitive work (HIPAA, FedRAMP, SOC2), running North-Mini-Code on-premise or in a private VPC removes data-residency and breach risk associated with external APIs. You remain responsible for securing the deployment, managing model access, and auditing inference logs. No claims of inherent privacy or compliance certification from Cohere—only the operational benefit of ownership.
Hardware footprint
**Estimate (verify per setup):** ~20 GB VRAM (fp16, single token generation); ~40 GB for batch inference or 256K full-context loads. Sparse MoE (8/128 experts) reduces compute vs. dense 30B models but still requires modern GPU (A100 40GB, H100, or multi-GPU cluster). CPU-only inference is impractical. Quantization (GGUF, int4) not yet confirmed; requires community or vendor work.
Integration
North-Mini-Code integrates via Transformers `AutoModelForCausalLM` (HuggingFace standard) or vLLM serving (recommended for production). Tool use relies on JSON schema function definitions and transformers chat templates; connect to bash, Python exec, or custom APIs by writing tool response handlers in your orchestration layer (e.g., LangChain, LlamaIndex, or custom agents). Supports OpenCode for local IDE integration. Interleaved thinking requires preserving reasoning output in chat history for subsequent calls—native support in vLLM's reasoning parser. No native Slack/Teams connectors; build bridges via webhooks or microservice adapters.
When it's not the right fit
- —You need off-the-shelf fine-tuning on standard tasks (general Q&A, summarization, translation)—North-Mini-Code is narrowly optimized for code and tool use; consider Llama 3.1 for versatility.
- —Your ops team lacks GPU infrastructure or DevOps capacity for containerized LLM management; consider API-based alternatives if self-hosting is prohibitive.
- —You require production-grade safety/content filtering out of the box; North-Mini-Code is a research release with minimal alignment guarantees beyond code-generation RLVR.
- —Extremely low latency is critical (sub-100ms inference); MoE routing overhead + 256K context window make this slower than optimized smaller models or cached APIs.
Alternatives to consider
Qwen 2.5-Coder
Dense model, broader code and multi-language support, no sparse MoE overhead; better if you want simpler deployment and don't need agentic tool-use training.
Llama 3.1-70B (Code focus variant, if available)
Larger, denser foundation; more versatile for ops tasks beyond pure code generation; larger context window in some variants; less specialized for agentic loops.
DeepSeek-Coder (open weight variant)
Competitive code benchmarks, smaller footprint options available; fewer tool-use guarantees but simpler architecture; good if you're resource-constrained.
Related open models
FAQ
Can I deploy North-Mini-Code entirely on-premise or in a private VPC, with zero external API calls?
Yes. Download weights from HuggingFace, run vLLM or Transformers locally, and your inference stays in your environment. You own the model, the hardware, and the data. No telemetry or external calls by default—configure your network/firewall to enforce isolation.
Is North-Mini-Code licensed for commercial use without royalties?
Yes. Apache 2.0 is a permissive OSI license. You can build and sell commercial products using this model, modify it, and distribute it—subject only to license attribution. No usage fees, no Cohere royalty. Review your legal/compliance team for downstream licensing obligations.
Does the model include safety filtering or content policies?
Unknown. Model card does not specify alignment techniques or content-filtering guarantees beyond RLVR on coding tasks. This is a research release; treat it as a general-purpose code/tool model without production safety certifications. Implement your own filtering if needed.
What's the learning curve for integrating tool use (bash, APIs) into my existing ops workflows?
Moderate. The model supports transformers chat templates and JSON schema function definitions; you write tool handlers (e.g., bash subprocess calls, API wrappers) outside the model. If you're familiar with LangChain or LlamaIndex, integration is standard. Interleaved thinking requires preserving reasoning in chat history—adds slight complexity but is well-documented.
Build a Private AI Operating System with North-Mini-Code
Stop sending code and infrastructure logs to third-party LLM providers. LLM.co helps you deploy North-Mini-Code—and custom agents on top of it—entirely in your environment. Automate internal DevOps, code review, and knowledge work while keeping data in-house. Let's build your custom AI stack.