Open-Weight LLM · Private & Custom AI
Step-3.7-Flash-NVFP4
Production-grade sparse MoE vision-language model for high-throughput agentic automation, financial document processing, and multi-step operational workflows in private environments.
Step 3.7 Flash is a 198B-parameter sparse MoE vision-language model activating ~11B parameters per token, achieving 400 tokens/sec throughput with 256k context and native image understanding. For ops teams, it's a self-hostable foundation for automating document-heavy workflows, agent orchestration, and perception-driven tasks while keeping data in-house.
Model facts
Private deployment
Run Step-3.7-Flash-NVFP4 in your own environment
Requires 8 GPUs (estimated 120–160GB total VRAM for FP4 quantized version across tensor-parallel setup; BF16 or FP8 higher). Deploy via vLLM, SGLang, or llama.cpp on data center hardware or high-memory workstations (DGX Station, Ryzen AI Max+, Mac Studio 128GB+). Architecture advantage: all inference and data processing remain in your environment; no external API calls unless you choose hosted inference partners.
Operational AI use cases
Financial & Legal Document Triage
Parse 10-K filings, contracts, and loan agreements in a single 256k-context pass. Extract structured data (obligations, dates, financial terms) and flag anomalies without sending documents to third-party APIs. Vision capability handles scanned PDFs, embedded tables, and charts natively.
Multi-Step Support & Knowledge Workflows
Route and resolve internal tickets by retrieving docs, verifying context visually (screenshots, dashboards), and generating precise responses. Three reasoning levels (low/medium/high) let you tune latency vs. accuracy per ticket type. Tool-calling integration (ClawEval score 67.1) ensures reliable workflows without drift.
Concurrent Code & System Agent Orchestration
Run multiple code-review or infrastructure-troubleshooting agents in parallel (400 tok/sec throughput). SWE-Bench Pro score (56.3) confirms capability to trace repos, isolate bugs, and suggest patches. MoE sparsity keeps per-agent compute cost low at scale.
Custom AI
As a base for custom AI
Solid foundation for vision-language applications: embed as the reasoning backbone in a custom chatbot, document processor, or agent framework. Apache 2.0 license and open weights enable fine-tuning on proprietary operational data (tickets, contracts, internal docs). Three reasoning modes allow product builders to offer speed/accuracy tradeoffs to end users.
In the operating system
Where it fits
Core reasoning layer in an ops AI operating system. Handles perception + language understanding for agentic workflows (detection, retrieval, tool-calling). Pair with vector DBs for RAG, workflow orchestrators for multi-step automation, and API integrations for external tool access. MoE sparsity and 256k context fit knowledge-intensive and long-horizon agent loops.
Data control & security
Private deployment keeps all inference, images, documents, and intermediate reasoning in your VPC or data center. No data leaves your environment unless you explicitly call external APIs. This architecture reduces compliance friction for regulated data (healthcare, finance) and eliminates reliance on third-party model inference. Standard ops security best practices (network isolation, access controls, audit logging) apply.
Hardware footprint
Estimate: NVFP4 quantized ~120–140GB across 4 GPUs (tensor-parallel). BF16 or FP8 ~160–200GB across 8 GPUs. Per-token activation ~11B parameters (sparse). Verify with StepFun documentation and run load tests in your target environment.
Integration
Expose via vLLM OpenAI-compatible API (port 8000) or SGLang for easy drop-in to existing apps. Parse image URLs or base64-encoded visuals in chat messages. Integrate with orchestrators (LangChain, LlamaIndex, Temporal) and vector DBs (Pinecone, Weaviate) for RAG. Tool-calling support (--enable-auto-tool-choice) enables direct binding to internal APIs (Salesforce, Jira, knowledge bases). Tensor-parallel and expert-parallel scaling lets you grow to 8+ GPU clusters as throughput demand increases.
When it's not the right fit
- —You need sub-50ms latency: 400 tok/sec throughput is fast, but not suitable for real-time chat UX without aggressive batching or caching.
- —Your deployment is single-GPU or memory-constrained: requires multi-GPU tensor parallelism; minimal viable setup is 4 high-memory GPUs.
- —You need guaranteed SLA or commercial support without involvement: self-hosted models require your team to manage scaling, monitoring, and updates.
- —Vision tasks demand absolute SOTA performance: SimpleVQA (79.2) and V-Bench (95.3) are strong but not first-place absolute; evaluate against your specific visual domain.
Alternatives to consider
Llama 3.2 Vision
Open-weight multimodal, smaller footprint (~11B), easier single-GPU deployment. Trade-off: lower agentic reasoning capability and throughput; better for document Q&A than workflow automation.
Qwen2-VL (72B)
Apache 2.0, strong vision-language, fits 2–4 GPUs. Trade-off: no MoE sparsity, lower throughput; simpler inference stack, but less tool-calling stability.
Claude 3.5 Sonnet (hosted API)
Excellent multimodal and agentic performance, zero ops overhead. Trade-off: data leaves your environment, per-token pricing, vendor lock-in; no private deployment option.
Related open models
FAQ
Can we run this entirely in our private cloud or on-prem?
Yes. Download weights from HF, deploy via vLLM or SGLang on your GPU cluster, and expose via OpenAI-compatible API. All inference stays in your environment. Requires multi-GPU setup and monitoring infrastructure you operate.
Is Apache 2.0 license permissive for commercial/proprietary products?
Yes. Apache 2.0 allows commercial use, modification, and redistribution with attribution and license inclusion. You can build and sell custom AI products on top of this model without royalties or approval.
How do we handle very long documents or multi-turn workflows?
256k context window supports most multi-document scenarios in one pass. For extremely long workflows (100+ turns), use KV cache and context reuse in vLLM. For dynamic knowledge, integrate with vector DBs and retrieval to refresh context within the window.
What's the typical ops team size to run this self-hosted?
Minimal: 1 ML engineer + 1 DevOps for initial setup and monitoring. Scales to 2–3 for large-scale multi-cluster deployments. Compare to managed inference services, which reduce this to 0 but introduce data-residency and cost tradeoffs.
Ready to build a private AI system that automates your operations?
Step 3.7 Flash is a powerful open-weight foundation, but integrating it into your workflow automation, document processing, or agent platform takes expertise. LLM.co helps middle-market companies build custom AI systems on open models—keeping data private, costs predictable, and control in your hands. Let's explore how Step 3.7 Flash fits your ops stack.