Open-Weight LLM · Private & Custom AI
solar-pro-preview-instruct
A 22B instruction-tuned model engineered to run on a single 80GB GPU, delivering 70B-class reasoning for private, ops-focused AI workflows without scale penalties.
Solar Pro Preview is a 22-billion-parameter instruction-tuned LLM from Upstage, scaled from Phi-3-medium using depth up-scaling. It targets deployment on a single high-end GPU and benchmarks competitively against models 3× its size (Llama 3.1 70B on MMLU-Pro, IFEval, GSM8K). For ops teams, it trades some multi-language and context coverage for efficient, self-contained reasoning on a single machine.
Model facts
Private deployment
Run solar-pro-preview-instruct in your own environment
Deployable entirely on-premise with a single 80GB GPU (A100/H100 class). Requires PyTorch, transformers, flash_attn, and accelerate; runs via standard HuggingFace pipeline with `device_map='cuda'`. No external API calls needed—all inference and fine-tuning can happen inside your infrastructure, keeping proprietary documents, customer data, and operational records off third-party servers. Trade-off: 4K context window and English-only (preview limitation) vs. full data isolation.
Operational AI use cases
Customer Support Triage & Escalation
Route incoming tickets by intent (refund, technical, billing) and draft first-response suggestions. Solar Pro's strong IFEval (84.37) and instruction-following enable reliable parsing of unstructured support emails. Run the entire pipeline in-house; no ticket content leaves your servers.
Internal Knowledge & Compliance Q&A
Index policy docs, SOPs, and regulatory guidelines as vector embeddings; use Solar Pro as the retrieval-augmented generation (RAG) backbone to answer employee questions ("What's the approval workflow for $50K spend?") with citations. GSM8K strength (89.69) supports structured reasoning over multi-step processes.
Document Processing & Data Extraction
Parse invoices, contracts, and forms to extract structured fields (vendor name, amounts, dates). ChatML template + instruction-tuning make it effective for classification and field mapping. Deploy in a background worker; no API costs, full audit trail of what was extracted and why.
Custom AI
As a base for custom AI
Strong foundation for domain-specific fine-tuning and RAG. Its 22B size and MIT license enable companies to adapt it to vertical workflows (legal document review, manufacturing process monitoring, field-service dispatch) without licensing friction. Transformer architecture and safetensors format support LoRA and QLoRA efficient tuning; you can specialize it on internal data and redeploy on the same GPU.
In the operating system
Where it fits
Acts as the **reasoning and response-generation layer** in a private AI operating system. Sits above retrieval (vector DB, keyword search) and below orchestration (workflow agents, multi-step planning). Pairs well as the backbone for agentic loops—querying internal tools, reasoning over results, and composing structured outputs without leaving the environment.
Data control & security
Self-hosting on private infrastructure means all prompts, internal documents, and generated outputs never traverse public APIs or third-party servers. This is an **architectural advantage** for compliance (HIPAA, financial data, trade secrets) and operational transparency—you control where training data flows, how long logs persist, and who audits inference. Solar Pro itself carries no built-in encryption or compliance guarantees; security depends on your network, access controls, and operational discipline.
Hardware footprint
**Estimate:** ~44 GB VRAM at FP16 (native precision); ~88 GB at FP32. At INT8 quantization, ~22 GB. Batch size 1 (streaming inference) requires ~48–52 GB FP16. Requires GPU with 80GB (A100, H100) or dual smaller GPUs with careful sharding. CPU fallback is impractical.
Integration
Standard Python/transformers integration via `AutoModelForCausalLM`. Supports batching for throughput. Expose via FastAPI or local gRPC for internal services. Tokenizer requires `trust_remote_code=True` (review custom code before deployment). Works with vector DBs (Pinecone, Milvus, local Chroma) for RAG. No native API gateway; you manage rate limiting, auth, and request routing through your infrastructure.
When it's not the right fit
- —Multi-language or code-heavy tasks: English-only preview; no specialized code training signaled.
- —Long-context workflows: 4K token limit (preview) is tight for multi-document reasoning or long conversations; official version (Nov 2024) promised longer context.
- —Real-time, ultra-low-latency inference: Single-GPU inference + 22B parameters implies ~100–500ms per inference step; not suited for sub-50ms SLAs.
- —No system prompt support (preview only): Reduces flexibility for role-based or persona-driven response shaping.
Alternatives to consider
Llama 3.1 8B
Smaller (8B), runs on cheaper GPUs (24GB VRAM), broader community tooling. Trade-off: lower benchmark performance (37.88 MMLU-Pro vs. 52.11), less sophisticated reasoning.
Phi-3.5-MoE
41.9B (6.6B active MoE), MIT license, instruction-tuned. Better for scale flexibility; requires ~60GB. Benchmarks slightly lower on MMLU-Pro (46.99 vs. 52.11) but useful if you need variable compute.
Gemma 2 27B
27B, runs on 80GB GPU, Apache 2.0 license. Stronger on some evals (EQBench 80.32). Google ecosystem support. Slightly larger; less efficient scaling than Solar Pro's depth approach.
Related open models
FAQ
Can we fine-tune Solar Pro on our internal data and keep it private?
Yes. MIT license permits fine-tuning and private redeployment. Use LoRA/QLoRA on the same 80GB GPU; full model + adapter weights stay in-house. No licensing conflict or external approval required.
Is this model safe for handling sensitive customer or financial data?
Self-hosting eliminates transmission to third-party APIs, which is a strong architectural benefit. However, the model itself contains no encryption or compliance certifications. Security depends on your network isolation, access controls, and operational procedures. Audit and test thoroughly before handling regulated data.
What's the difference between Solar Pro Preview and the official release (Nov 2024)?
Preview is English-only, 4K context max, and no system prompts. Official release promised multi-language support, extended context, and system prompt handling. If you need those now, use alternatives or plan an upgrade path.
Can we use this commercially in a product we sell to customers?
MIT license permits commercial use and redistribution, including in products. You may need to include copyright/license attribution in your product. No commercial restrictions—verify with legal for your specific use case, but license itself is permissive.
Build Your Private AI Operating System with Solar Pro
Solar Pro Preview gives you 70B-class reasoning in a single GPU—perfect for RAG, ops automation, and domain fine-tuning. LLM.co helps you architect, deploy, and scale private LLM workflows. No vendor lock-in. Full data control. Let's talk.