Open-Weight LLM · Private & Custom AI
Step-3.7-Flash
High-throughput vision-language agent backbone for private, production agentic workflows—parse documents, orchestrate tools, verify cross-source data without cloud lock-in.
Step-3.7-Flash is a 198B-parameter sparse MoE vision-language model that activates ~11B parameters per token, delivering 400 tok/s throughput with 256k context. It excels at multimodal perception, tool orchestration, and agentic reasoning—built for companies running private agent fleets, document automation, and structured code generation at scale.
Model facts
Private deployment
Run Step-3.7-Flash in your own environment
Self-host on modern GPU clusters (8× high-end GPUs recommended for production) or smaller workstation setups (Mac Studio/DGX Station with 128GB+ unified memory for dev/testing). Deployment backends include vLLM, SGLang, Transformers, and llama.cpp—all run entirely within your VPC. No data leaves your environment; complete operational control. Sparse MoE architecture means you can tune tensor-parallel and expert-parallel settings for your hardware footprint.
Operational AI use cases
Document Intelligence & Compliance Automation
Parse dense financial reports, contracts, and regulatory filings in a single pass (256k context). Extract structured data, flag compliance gaps, and auto-populate internal systems—all without sending documents to third parties. Vision capability handles scanned PDFs, charts, and embedded visuals.
Multi-Step Agent Orchestration
Run high-frequency agentic loops (search, verify, act) with strict policy adherence. ClawEval score of 67.1 indicates strong resistance to prompt injection and reliable tool-calling. Automate cross-departmental workflows: support ticket triage + knowledge retrieval, procurement verification, or internal process optimization.
Software Engineering Agent Pipelines
Deploy concurrent coding agents for bug isolation, patch generation, and code review at scale. SWE-Bench PRO score of 56.3 reflects real-world repository understanding. Spin up fleet instances in your private infrastructure for high-throughput CI/CD integration.
Custom AI
As a base for custom AI
Strong foundation for proprietary AI products: fine-tune on domain-specific data (financial, legal, technical support) without sharing training sets. The sparse MoE structure and tool-calling support make it ideal for building custom agent frameworks—combine with your internal APIs, knowledge bases, and business logic. Reasoning-level selector (low/medium/high) lets you ship different product tiers from the same base.
In the operating system
Where it fits
Core reasoning and perception engine in a private AI operating system. Sits between your knowledge layer (RAG, internal docs) and your workflow/agent orchestration layer. Use it to power agentic decision-making, structured output generation, and multimodal understanding—feed results into your ops systems (ticketing, CRM, ERP, code repos) via standard APIs.
Data control & security
Self-hosted deployment means documents, images, and internal data never transit external APIs. Operator controls the full inference pipeline—no logging, telemetry, or model training on your data unless you choose it. No claims of inherent security or compliance; audit and harden your own infrastructure per your regulatory needs (SOC 2, HIPAA, FedRAMP, etc.). MoE sparsity and quantization options (FP8, NVFP4) let you run on commodity hardware without sacrificing throughput.
Hardware footprint
**Estimate (unverified):** - **FP8 quantized**: ~380–420 GB VRAM (8× 48GB H100 / A100 cluster) - **BF16 full precision**: ~750–850 GB VRAM (8× high-end GPUs or multi-node setup) - **NVFP4 aggressive quantization**: ~190–220 GB VRAM (4× 48GB GPUs, local testing) Requires modern GPU with compute capability ≥8.0 (A100, H100, L40S, or newer). Tensor-parallel size typically 4–8 for production.
Integration
Expose via vLLM/SGLang OpenAI-compatible API endpoints; plug into your existing orchestration (n8n, Zapier, internal Python/Go services). Tool-calling and reasoning parsers are built-in. Supports image URLs and base64-encoded visuals. Tensor-parallel and expert-parallel flags let you scale inference across your cluster. Context window (256k) fits large documents; use speculative decoding (MTP) and KV-cache optimization for cost/latency tuning.
When it's not the right fit
- —You need sub-100ms latency on every request—400 tok/s is fast for reasoning, but sparse MoE cold-start and expert synchronization add overhead vs. dense models.
- —Your data is extremely sensitive and you lack GPU infrastructure—private deployment requires significant engineering and hardware investment.
- —You need the absolute peak performance on narrow benchmarks (e.g., pure coding tasks); other models may edge ahead in specialist domains despite Step-3.7-Flash's strong generalist profile.
- —You prefer fully managed, audit-free inference—self-hosting means you own operational security, logging, and compliance hardening.
Alternatives to consider
Llama 3.1 405B
Larger pure dense model, excellent for coding and reasoning; no vision; requires more VRAM; stronger on some narrow benchmarks but no MoE cost savings.
Mixtral 8×22B
Smaller sparse MoE alternative; lower throughput, shorter context (64k), but lighter footprint; good for cost-constrained private deployments if you don't need vision.
Claude 3.5 Sonnet (via Anthropic API)
Managed, no self-hosting; best-in-class coding + vision; but data governance trade-off—requires trust in Anthropic's infrastructure and API logging policies.
Related open models
FAQ
Can I run Step-3.7-Flash entirely on-premises without cloud?
Yes. Deploy via vLLM or SGLang on your GPU cluster behind a firewall. All inference, data processing, and model parameters stay in your environment. You control updates, logging, and access.
What's the Apache 2.0 license status for commercial use?
Apache 2.0 is permissive for commercial use, including building and selling products. You may modify, distribute, and monetize applications using Step-3.7-Flash—provided you include the license notice and grant back any patent rights. No additional fees to StepFun for commercial deployment.
How does sparse MoE affect inference cost and speed in private deployment?
Only ~11B of 198B parameters activate per token—lower VRAM footprint and faster compute per token than dense 198B models. Trade-off: expert routing adds synchronization overhead; batch size and GPU memory utilization matter. Use quantization (FP8, NVFP4) to reduce VRAM further if latency allows.
Can I fine-tune Step-3.7-Flash on proprietary data?
Yes. StepFun supports fine-tuning via NVIDIA Nemo and Megatron Core (mentioned in model card). Fine-tune on your own hardware or cloud infra; data never leaves your control. Requires GPU engineering expertise.
Ready to Build a Private AI System?
Step-3.7-Flash is engineered for companies deploying agents and automating operations without cloud lock-in. LLM.co helps you self-host, fine-tune, and integrate vision-language models into your stack. Let's architect your private AI operating system.