Open-Weight LLM · Private & Custom AI
voyage-4-nano
A 346M-parameter multilingual embedding model for private semantic search, RAG, and retrieval automation—engineered for on-premise deployment without indexing friction.
voyage-4-nano is a compact text-embedding model from Voyage AI optimized for semantic search and document retrieval in private, self-hosted environments. It's part of the Voyage 4 series with a shared embedding space, meaning a company can swap between nano/lite/large models without re-indexing. For ops teams automating internal knowledge retrieval, support ticket routing, and document classification, this is a lightweight alternative to API-based embedding services.
Model facts
Private deployment
Run voyage-4-nano in your own environment
Runs locally on modest GPU hardware (estimate: ~1.4–2.8 GB VRAM depending on precision/quantization). No external API calls; data stays entirely in the customer's VPC/air-gapped environment. Supports flexible output precision (float32, int8, uint8, binary) and dimensionality (2048, 1024, 512, 256) via quantization-aware training, enabling cost/latency trade-offs. Loads via standard transformers or sentence-transformers libraries; no proprietary inference dependencies. Apache 2.0 license removes licensing friction for internal/commercial use.
Operational AI use cases
Internal Knowledge Retrieval & Support Automation
Embed company documentation, runbooks, and past support tickets. Use nano to power a private RAG layer that routes incoming support requests to the most relevant internal knowledge without sending data to third-party APIs. Reduces latency and keeps proprietary support patterns in-house.
Document Classification & Workflow Routing
Embed incoming invoices, contracts, HR forms, or service requests. Use cosine similarity to classify and auto-route them to the correct department or approval queue. Matryoshka learning allows ops teams to down-sample embeddings to 512-dim for speed without significant quality loss, optimizing for high-throughput classification pipelines.
Semantic Search for Internal Tools & Data Governance
Deploy nano as the embedding backbone for a private search layer across email archives, internal wikis, compliance logs, and code repositories. Shared embedding space with larger Voyage models means teams can index once, then scale inference up/down per use case without re-running embeddings.
Custom AI
As a base for custom AI
Strong foundation for building proprietary retrieval-augmented generation (RAG) products and internal AI assistants. Companies can fine-tune nano on domain-specific language (e.g., legal contracts, medical records, financial reports) or use it as-is for rapid prototyping. Quantization-aware and Matryoshka training enable ops teams to dynamically adjust embedding precision and dimension at inference time, supporting A/B testing and cost optimization without retraining.
In the operating system
Where it fits
Acts as the semantic search / knowledge embedding layer in an AI operating system. Sits between data ingestion (raw documents) and retrieval/agent layers. Feeds semantic matches to downstream LLMs (via RAG) or to workflow automation agents that act on retrieved context. In a private/self-hosted stack, nano replaces external embedding APIs, reducing dependency and latency.
Data control & security
Self-hosted deployment ensures all embeddings and source text remain in the customer's environment—no transmission to external embedding services. This is an architectural control, not a claim about the model's inherent security. Companies must still manage the usual infrastructure hardening (network isolation, access control, encryption at rest). Useful for regulated industries (healthcare, finance) where data residency is contractual, though no inherent compliance certifications are documented.
Hardware footprint
Estimate (float32): ~1.4 GB VRAM; (int8 quantized): ~0.7 GB VRAM; (binary): ~0.18 GB VRAM. Context window of 32,000 tokens supports long-form document embedding. Inference latency on modern GPU (RTX 4090, A100) likely <100ms for typical documents; CPU inference feasible but slower for high-throughput workloads.
Integration
Loads via transformers (direct forward pass) or sentence-transformers (higher-level encode_query/encode_document API). vLLM support enables batched inference at scale. Custom prompts for query vs. document text ('Represent the query for retrieving…') are automatically handled via sentence-transformers; raw transformers users must manually prepend. Produces standard dense vectors (float32, int8, or binary) compatible with vector databases (Pinecone, Milvus, Weaviate) and in-memory FAISS. REST/gRPC wrappers (text-embeddings-inference) are available for containerized microservice deployment.
When it's not the right fit
- —Custom generative tasks (creative writing, code generation) — nano is embedding-only, not a text-generation model. Use as retrieval for a generative LLM, not standalone.
- —Real-time, sub-10ms latency at massive scale (>1M QPS) — even quantized, nano on single GPU hits throughput ceilings; requires careful load balancing and batch optimization.
- —Highly specialized domains with zero training data — nano is pre-trained on general multilingual corpora; domain-specific fine-tuning may be needed for legal, biomedical, or proprietary jargon.
- —Non-technical ops teams without ML/DevOps support — self-hosting requires containerization, monitoring, and incident response; API-based alternatives reduce operational burden.
Alternatives to consider
nomic-embed-text (Nomic AI)
Open, fully trained on public data, 137M params. Smaller than nano but no quantization-aware training; good for cost-conscious deployments where embedding quality is secondary.
BGE-small-en-v1.5 (BAAI)
109M params, strong retrieval benchmarks on English-only tasks. Widely used in private RAG; lighter footprint but no multilingual support or shared-embedding-space architecture.
voyage-3.5-lite (Voyage AI — proprietary)
Larger, higher quality embeddings, but requires API calls. Trade-off: better retrieval accuracy vs. data residency and inference control. Relevant if nano's quality underperforms after internal testing.
Related open models
FAQ
Can we run voyage-4-nano fully on-premises without any external API calls?
Yes. Download the model weights from HuggingFace (346M params), load via transformers or sentence-transformers, and run on your own GPU/CPU. No Voyage API dependency; all inference and data stay internal.
What's the commercial-use license status?
Apache 2.0 license explicitly permits commercial use, redistribution, and modification with attribution. No restrictions on building products, services, or proprietary applications with nano; suitable for enterprise deployment.
How does the shared embedding space with Voyage 4 models help us?
All Voyage 4 models (nano, lite, large) produce embeddings in the same semantic space. Index once with nano for development, then swap to 4-lite for higher throughput or 4-large for max quality—no re-indexing. Reduces operational friction when scaling or optimizing.
Can we quantize voyage-4-nano to reduce memory and latency?
Yes. The model is trained with quantization-aware learning and supports int8, uint8, and binary precision outputs via the `precision` parameter in sentence-transformers. Binary embeddings cut storage ~99% with minimal retrieval quality loss, though trade-off varies by use case.
Build Your Private AI Retrieval Layer
voyage-4-nano powers semantic search and RAG without external APIs. Let LLM.co help you integrate it into a self-hosted AI operating system—automating internal knowledge retrieval, document routing, and custom generative workflows. Start with a self-hosted pilot today.