Open-Weight LLM · Private & Custom AI
Qwen3.6-35B-A3B-DFlash
A lightweight draft model for speculative decoding that accelerates Qwen 3.6-35B inference 2–3.6x by proposing tokens in parallel, designed for high-throughput private deployments where latency and cost per inference matter.
Qwen3.6-35B-A3B-DFlash is not a standalone LLM—it's a *speculative decoding draft model* trained by Z-Lab and Modal to work alongside Qwen's 35B-A3B target model. It uses block diffusion to propose multiple tokens in parallel, which the target model verifies, dramatically improving serving throughput without changing output quality. For ops-focused companies running their own inference infrastructure, this is a throughput multiplier that lets you serve more requests per GPU.
Model facts
Private deployment
Run Qwen3.6-35B-A3B-DFlash in your own environment
Deploy as a paired system: run both the target model (Qwen/Qwen3.6-35B-A3B) and this draft model on your own hardware via SGLang (recommended) or vLLM. All data and requests remain in your environment—no calls to external inference APIs. You control the serving layer, batching policy, and hardware allocation. The architecture trades GPU memory (both models in VRAM) for dramatic inference speedup; suitable for companies with on-prem GPU clusters or cloud accounts where inference cost is a constraint.
Operational AI use cases
High-volume document processing & classification
Automation teams processing 10k+ documents per day (contracts, support tickets, internal reports) can use the speedup to classify, extract metadata, or summarize at scale without spinning up additional GPUs. 2.5–3.6x throughput gain directly reduces wall-clock time and cost per document.
Real-time conversational support agents
Customer support or internal help-desk chatbots benefit from lower per-token latency. The draft model's parallel proposal mechanism reduces time-to-first-token and inter-token delays, making responses feel snappier. At concurrency 32 (typical for production), you see 2.5–2.9x throughput gains.
Async workflow automation (compliance, ops review)
Batch reasoning tasks (policy review, operational risk assessment, automated code/document review) run on your private cluster. The speedup lets you process larger queues in the same SLA window, reducing backlog and enabling more sophisticated multi-step reasoning without additional hardware.
Custom AI
As a base for custom AI
Use this as a serving acceleration layer, not as a base for fine-tuning. If you're building a custom AI product on top of Qwen 3.6-35B (e.g., domain-specific Q&A, workflow automation, internal knowledge agents), deploy it with DFlash speculative decoding to reduce inference latency and cost. The model card does not document fine-tuning support; confirm with Z-Lab if you need a customized version.
In the operating system
Where it fits
In an LLM.co ops-AI system: this belongs in the *inference/serving layer*. It accelerates the backbone LLM that powers agents, workflow automation, and knowledge retrieval. Pair it with your workflow orchestration (agents, chains), vector retrieval (knowledge base lookups), and API/RPA connectors. Not a direct fit for RAG embedding or knowledge extraction—it's purely a serving optimization.
Data control & security
Running DFlash + target model on your own infrastructure means all prompts, context, and generated outputs stay in your environment; no data transits to external APIs. This is an *architectural* control: you own network isolation, access logs, and data retention. Note: the model itself contains no built-in encryption or differential privacy. Data security depends on your deployment environment (VPC, firewall, RBAC, audit logging). Audit and compliance review are still required by your ops team.
Hardware footprint
**Estimate for bfloat16 (typical production precision):** Target model (Qwen 3.6-35B-A3B) ~70 GB; DFlash draft model ~70 GB. Total ~140 GB VRAM for a single-GPU deployment. Scales to multi-GPU via tensor parallelism. FP16 or INT8 quantization reduces footprint but not documented in model card—requires implementation in SGLang/vLLM.
Integration
Deployment requires SGLang (recommended) or vLLM with speculative decoding support. Launch the server with both the target model (Qwen/Qwen3.6-35B-A3B) and draft model paths, tensor parallelism config, and block size (8 for concurrency, 16 for single-user throughput). Call via OpenAI-compatible API or SGLang's native client. Integrate with your ops orchestration (Airflow, Temporal, custom workflows) via standard HTTP endpoints. Requires CUDA 12.0+, modern NVIDIA GPU (H100, H200, B200 tested); confirm vLLM PR #40898 status before committing to vLLM route.
When it's not the right fit
- —You need a standalone, inference-ready model. DFlash is a *draft-only* model; it must be paired with Qwen 3.6-35B-A3B and a speculative decoding server. Cannot be used alone.
- —Your ops workload has strict, static latency SLAs (sub-100ms time-to-first-token). Speculative decoding adds latency to draft generation; test your specific setup before committing.
- —You need guardrails, content filters, or compliance tooling baked into the model. DFlash provides no custom safety or alignment layers; implement those at the application level.
- —You require vLLM integration immediately. Support is pending (PR #40898 status unclear); SGLang is the stable route today.
Alternatives to consider
Llama 3.1 70B + Lookahead decoding (vLLM native)
Larger, more capable base model with native vLLM lookahead drafting. No separate draft model needed. Trade-off: higher VRAM, potentially slower per-token latency vs. DFlash's parallel proposals.
Mistral Large + standard batching (no speculative decoding)
Strong general-purpose model, straightforward deployment. No drafting overhead. Trade-off: lower throughput at high concurrency; suitable if your workload is latency-sensitive rather than throughput-bound.
Qwen 3.6-32B (quantized)
Smaller, faster base model with fewer parameters. Fit on single GPU more easily. Trade-off: slightly lower quality; no speculative decoding gains.
FAQ
Can I run DFlash on a single GPU, and what size?
Theoretically yes with aggressive quantization (INT4/INT8), but not tested in model card. Baseline is ~140 GB bfloat16 (two 80GB H100s or equivalent). Contact Z-Lab or run POC with quantization on your target GPU first.
Does deploying DFlash privately keep my data fully isolated?
Yes—all prompts and responses stay in your network if you run the server on-prem or in a private VPC. No telemetry or data leaves your environment by default. You must still implement network isolation, access controls, and audit logging at the infrastructure level.
Can I use DFlash commercially?
Yes. License is Apache 2.0 (permissive). You may build and sell products using this model without royalties. Review Apache 2.0 terms; no warranty or liability coverage from Z-Lab. Verify with your legal team if you're bundling it with proprietary code.
Do I have to use SGLang, or can I use vLLM today?
SGLang is stable and recommended (DFlash is integrated). vLLM support is in-progress (PR #40898); check status before production commitment. If vLLM is your standard, plan for SGLang migration or wait for vLLM merge.
Build a faster, private inference system.
DFlash shows how speculative decoding multiplies throughput on your own hardware. LLM.co helps you architect private LLM deployments, pair them with your ops workflows, and integrate them into your stack—all while keeping data in your environment. Let's design your inference layer.