Open-Weight LLM · Private & Custom AI
VertaLily-1.2-1B-GGUF
Lightweight 1B private LLM for edge deployment, local workflows, and ops automation where data must stay in-house.
VertaLily-1.2-1B is a 1 billion parameter open-weight model optimized for ARM CPUs and edge hardware, available in three GGUF quantizations (0.60–1.25 GB). An ops/AI team would use it to run conversational AI, knowledge retrieval, and workflow automation entirely on-premises—no cloud calls, no data egress, full control.
Model facts
Private deployment
Run VertaLily-1.2-1B-GGUF in your own environment
Deploy via Ollama, LLM Farm (iOS), Off-Grid APK (Android), or OpenClaw agent framework. No API keys, no external services. Model runs on your infrastructure—laptop, on-prem server, or edge device. Q3_K (~0.60 GB) runs on phones; Q4_K_M (~0.73 GB) balances quality/speed on modest CPU; Q8_0 (~1.25 GB) for GPU or high-RAM servers. Data never leaves your environment.
Operational AI use cases
Internal Support & Knowledge Agent
Embed VertaLily into a private support portal to answer employee/customer questions against internal documentation, policies, or FAQs. Use OpenClaw or Ollama + RAG to ground responses in company knowledge. No external LLM calls—compliance-friendly for regulated industries.
Workflow Automation & Routing
Run classification/routing logic (ticket triage, expense categorization, request prioritization) on-device in a private workflow orchestrator. Low latency, full audit trail, zero third-party data exposure. Suitable for finance, HR, and ops teams handling sensitive internal data.
Edge Inference for Distributed Teams
Deploy Q3_K on employee devices or local servers for offline document summarization, meeting notes synthesis, or draft generation. Sync outputs back to central systems only after review. Reduces cloud bandwidth and keeps working drafts private.
Custom AI
As a base for custom AI
Use VertaLily as the base for a custom ops AI product (e.g., internal assistant, field-service bot, edge analytics engine). Fine-tune on your domain data in a private environment, then quantize and distribute via Ollama or mobile apps. The 1B footprint allows rapid iteration and easy deployment across your org.
In the operating system
Where it fits
Sits in the **agent/reasoning layer** of a private AI OS: handles conversational logic, retrieval grounding, and simple decision-making. Upstream (knowledge): tie to internal RAG, databases, APIs. Downstream (workflows): integrate with ticketing, CRM, ERP via OpenClaw or custom orchestrators. Complements larger cloud models for cost/privacy trade-offs.
Data control & security
Self-hosting ensures all input/output stays within your network perimeter—no third-party access, no training leakage, no SaaS terms. Audit logs and data governance remain under your control. Note: the model itself carries no built-in encryption or access controls; you must architect network/OS-level security (VPN, firewalls, role-based APIs) to enforce data boundaries.
Hardware footprint
**Estimate (based on quantization):** Q3_K ~0.80 GB VRAM (Q3_K model 0.60 GB + runtime overhead), Q4_K_M ~1.0 GB VRAM, Q8_0 ~1.6 GB VRAM. CPU-only inference on modern multi-core systems; GPU (CUDA/Metal) reduces latency 3–5×. Mobile (iOS/Android): Q3_K fits iPhone 12+, iPad with 2 GB+ RAM.
Integration
Expose via REST API (Ollama native, or wrap in FastAPI), gRPC, or CLI. Integrate with existing ops tooling: Zapier/n8n workflows, Slack bots, internal dashboards. OpenClaw framework supports tool-use (web_search, file_read, calculator) for extended capabilities. Standardize on GGUF format for model distribution. Plan inference infrastructure (CPU pooling, load balancing) based on latency SLA.
When it's not the right fit
- —Complex reasoning or multi-step math—benchmark shows 74% oracle reasoning; larger models may excel here.
- —Very long context windows (>2K tokens)—context length unknown; no published sliding-window or memory mechanism documented.
- —Production voice/multimodal tasks—model is text-only; no audio or vision capabilities.
- —High-throughput batch inference on limited CPU—consider GPU or horizontal scaling if >100 concurrent requests/minute.
Alternatives to consider
Llama 2 1B
Better baseline availability and proven fine-tuning community. Less optimized for ARM/edge. Broader ecosystem but fewer quantization variants.
Microsoft Phi-3-mini (4B)
Slightly larger (4B params), claims better instruction-following. Requires 1.5–2× more VRAM. Model card shows stronger benchmark claims but context length also undisclosed.
Qwen/Qwen2.5-1B
Alibaba's 1B instruction-tuned model. More documentation on training data and performance claims. Comparable footprint; wider language coverage but less ARM optimization marketing.
Related open models
FAQ
Can I run VertaLily entirely offline on an employee device?
Yes. Download Q3_K (0.60 GB) to any device with 2 GB+ RAM (iPhone 12+, modern Android, Linux laptop). Use LLM Farm, PocketPal, Off-Grid APK, or Ollama. No internet required after download. All inference happens locally; nothing leaves the device unless you explicitly sync outputs.
Is VertaLily licensed for commercial use in a closed-source product?
Model is Apache 2.0, which permits commercial use, derivative works, and redistribution with license inclusion. However, verify your legal team's interpretation of 'derivative work' if you fine-tune or embed in proprietary software. No additional commercial license restrictions documented.
How do I fine-tune VertaLily for my company's domain (e.g., legal, medical)?
Fine-tuning code and baseline architecture not published in the model card; check the GitHub repo (https://github.com/VLTX-Lab/VertaLily-AI) for training scripts. Once fine-tuned, quantize to GGUF and distribute privately. Plan for VRAM during training (~8–16 GB on GPU recommended).
What context length does VertaLily support?
Unknown. Model card and HuggingFace repo do not specify max context window. Assume default transformer window (likely 2–4K tokens); test empirically in your environment or contact developer for architecture details.
Build a Private AI System for Your Ops Stack
VertaLily is a lightweight, controllable foundation. LLM.co helps you architect it into a full private AI OS—custom fine-tuning, integrations with your tooling, secure deployment, and compliance automation. Let's design your self-hosted AI stack.