Open-Weight LLM · Private & Custom AI
dolphin-2.9.1-yi-1.5-34b
Full-parameter fine-tuned 34B conversational & coding model for private deployment—unfiltered, agentic, built for ops automation and custom AI applications where data control matters.
Dolphin 2.9.1 Yi 1.5 34B is a 34-billion-parameter instruction-tuned model trained on diverse datasets (instruction, code, math, agent/tool-use) with an 8k effective context window. It emphasizes conversational quality, coding capability, and function-calling readiness. For ops teams, it's a self-hostable foundation for custom chatbots, internal knowledge agents, support automation, and code-generation workflows.
Model facts
Private deployment
Run dolphin-2.9.1-yi-1.5-34b in your own environment
Self-hostable via standard transformers library; no proprietary dependencies. Running on-prem requires ~68GB VRAM (fp16) or ~34GB (int8 quantization)—manageable on a single A100/H100 or dual A6000 systems. Companies deploying privately keep all conversation data, operational logs, and customer interactions within their own infrastructure, eliminating data egress to third-party APIs.
Operational AI use cases
Internal Support & Knowledge Automation
Deploy as a self-hosted support bot that answers employee/customer queries against internal knowledge bases, SOPs, and ticketing systems. ChatML format supports multi-turn context; function-calling abilities allow it to look up tickets, pull docs, or escalate to humans. Data stays on-prem; compliance-sensitive teams avoid cloud vendor dependencies.
Code Review & Documentation Generation
Use for automated code review comments, PR summaries, and documentation generation in CI/CD pipelines. Model is trained on CodeFeedback and Dolphin-Coder datasets; can integrate into GitHub/GitLab workflows or internal dev tools. Keeps proprietary code off external LLM services.
Ops Workflow Agents & Task Automation
Leverage agentic abilities (Agent-FLAN, ToolBench training) to build multi-step automation workflows—ticket triage, incident routing, expense categorization, report generation. Function-calling support enables binding to Slack, JIRA, Salesforce APIs. Runs as a private reasoning engine; no vendor lock-in.
Custom AI
As a base for custom AI
Solid foundation for building vertical-specific AI applications: customer service chatbots, internal domain assistants, code-generation tools, compliance Q&A systems. Full-parameter fine-tuned on diverse datasets, so already instruction-aware. Teams can QLoRA-adapt it further on proprietary corpora (internal docs, domain terminology) or use as-is with prompt engineering. ChatML templating is transparent and standard.
In the operating system
Where it fits
Sits in the **Knowledge & Agent Layer** of a private AI OS. Serves as the core reasoning engine for multi-turn conversations, agentic orchestration, and function dispatch. Upstream: prompt routing, retrieval-augmented workflows (RAG), API bindings. Downstream: human-in-the-loop review, compliance/safety filters, and output routing to Slack, email, or ticketing systems.
Data control & security
By self-hosting, all user queries, conversation histories, and function outputs remain in your environment—no transmission to OpenAI, Anthropic, or other cloud platforms. This is an **architectural choice** for data residency, not a property of the model itself. Organizations with HIPAA, PCI-DSS, or strict data-locality requirements can audit the full fine-tuning process (Axolotl config is public) and control inference infrastructure. No guarantee of cryptographic security; standard hardening practices (TLS, auth, network isolation) apply.
Hardware footprint
**Estimate (varies by precision & quantization):** fp16 / bfloat16: ~68 GB VRAM; int8 quantization: ~34 GB; int4 (QLoRA): ~8–12 GB. Inference batch size 1–4 on single A100 (80 GB) or H100 (80 GB) practical; larger batches require tensor parallelism or multi-GPU setup. Memory scales linearly with sequence length (trained on 8k; inferences up to ~4–8k context per batch).
Integration
Standard HuggingFace transformers API; compatible with text-generation-inference (TGI) for production serving. Supports ChatML prompt template out-of-the-box. Integrates via REST/gRPC endpoints into Zapier, n8n, Langchain, or custom Python/Go clients. Function-calling format compatible with OpenAI-style tool definitions; can wire to Slack bots, JIRA webhooks, or internal APIs via standard SDK bindings.
When it's not the right fit
- —Your ops workflows need sub-100ms latency—34B model inference is ~500ms–1s per token on typical hardware; if you need real-time, sub-100ms decisions, consider smaller models or cached outputs.
- —You require guaranteed alignment, safety auditing, or built-in refusal policies—model is intentionally uncensored; creator advises implementing your own alignment layer. Not plug-and-play for regulated/public-facing services without additional compliance tuning.
- —You have <30GB VRAM and no quantization infrastructure—full-weight deployment requires significant GPU memory; int8/int4 quantization adds latency and complexity.
- —Your use case demands multi-modal (vision, audio) or very long context (>32k tokens)—model is text-only, effective context is ~8k despite RoPE tuning claims.
Alternatives to consider
Mistral 7B / Mixtral 8x7B
Smaller, faster inference; Mixtral offers mixture-of-experts scaling. Trade-off: less instruction-following, weaker coding. Better fit if hardware <20GB VRAM or latency <200ms required.
LLaMA 2 70B
Larger, more general-purpose; stronger on reasoning & code. Trade-off: ~140GB VRAM fp16, slower. Better for complex reasoning; Dolphin offers better ops-specific tuning (function-calling, agent training).
Openchat 3.5 (Yi-34B-based)
Also Yi-34B backbone; lighter fine-tuning. Trade-off: less coding/agentic depth. Simpler if you want smaller memory footprint or plan heavy custom adaptation.
Related open models
FAQ
Can I fine-tune Dolphin further on my own proprietary data?
Yes. Apache 2.0 license permits full parameter fine-tuning. Use Axolotl (same tool as creators) or standard HuggingFace trainer. QLoRA or full fine-tuning both viable; full fine-tuning requires ~8x H100 or equivalent. Keep training data private; fine-tuned weights are yours.
Is Dolphin compliant for regulated industries (healthcare, finance)?
Model itself is not inherently compliant. It is uncensored and will comply with requests including unethical ones (per creators' warning). You must implement alignment/safety layers (content filters, refusal logic, audit logging) before exposing to end-users. Self-hosting gives you control to add these; you are responsible for output.
Can I use Dolphin for commercial products?
Yes. Apache 2.0 license permits commercial use without attribution requirement (though attribution appreciated). You may monetize any product built with Dolphin, including resale or SaaS offerings. Ensure your custom content/training is also licensed appropriately.
What's the difference between the advertised 77.4 MMLU and my experience?
MMLU is a multiple-choice benchmark (general knowledge); your real-world performance depends on prompt engineering, domain, and task. Dolphin excels on conversational/coding tasks due to training data diversity. Run internal evals on your use case before production rollout.
Build Your Private Ops AI Stack with Dolphin
Stop shipping operational data to third-party APIs. Deploy Dolphin 2.9.1 as a self-hosted reasoning engine for support automation, code workflows, and intelligent agents. LLM.co helps you integrate it into your existing tools, fine-tune on proprietary data, and scale across your infrastructure. Let's architect your AI OS.