Open-Weight LLM · Private & Custom AI
Qwen3.6-27B-AEON-Ultimate-Uncensored-Multimodal-NVFP4-MTP-XS
27B multimodal reasoning model optimized for high-throughput private deployment on modern GPU clusters; purpose-built for ops automation, agentic workflows, and custom AI applications where inference speed and data residency matter.
Qwen3.6-27B-AEON-Ultimate-Uncensored-Multimodal-NVFP4-MTP-XS is a 27B parameter model with image-text-to-text capabilities, quantized to NVFP4 (4-bit) for memory efficiency and speed. It ships with multi-token prediction (MTP) and speculative decoding heads, and pairs well with external DFlash drafters for high-concurrency inference. For ops teams building internal AI systems, this is a self-contained model you pull, containerize, and run entirely within your environment—no external API calls, no data leaving your network.
Model facts
Private deployment
Run Qwen3.6-27B-AEON-Ultimate-Uncensored-Multimodal-NVFP4-MTP-XS in your own environment
Deploy via the canonical `ghcr.io/aeon-7/aeon-vllm-ultimate:latest` vLLM container (v0.23.0, source-built with Triton NVFP4 KV cache, DFlash patches, prefix-caching fixes). Pull model weights and optional DFlash drafter from HuggingFace, mount into container, bind to GPU. Tested recipe provided for DGX Spark (unified memory) and dedicated-VRAM Blackwell hardware (MTP-native path). Company retains complete control: no cloud, no third-party logs, data stays in your data center or on-prem GPU cluster. vLLM handles batching, prefill/decode separation, and concurrency scaling to c=64+.
Operational AI use cases
Support ticket triage & response drafting
Route incoming tickets through the model for intent classification, urgency detection, and draft-response generation. Pair with your internal ticket system via API. The multimodal capability handles screenshots and attachment analysis. Run inference on private infrastructure; no customer data touches external services. Extraction category achieves ~57 tok/s (single-stream) with 49% DFlash acceptance—fast enough for sub-second draft turnaround.
Internal documentation & knowledge base indexing
Process proprietary docs, wikis, and runbooks through the model to auto-extract key entities, generate summaries, and tag for internal knowledge retrieval. Long-context capability (DFlash draft acceptance holds ~45% at ~9k tokens) means digesting whole docs in one pass. Run monthly batch jobs on your hardware; no API usage charges, no vendor lock-in.
Ops workflow automation & agent backbone
Build internal agents that reason over operational tasks: log parsing, incident response, resource allocation, compliance checks. The 27B size and reasoning benchmarks (50% DFlash acceptance, 49.3 tok/s decode) support multi-step tool use and structured reasoning. Containerized deployment means you can spawn inference workers on-demand within your Kubernetes cluster or VM fleet; tool-call parsing and auto-choice are built into the serving config.
Custom AI
As a base for custom AI
Use as the foundation model for a custom AI product: fine-tune on proprietary operational data (tickets, logs, internal procedures) using LoRA or full-weight tuning. The NVFP4 quantization reduces fine-tuning memory overhead. The multimodal backbone supports custom vision tasks (document analysis, diagrams, screenshots). Graft your own LoRA/QLora adapters or merge weights offline, then serve the merged model in the same vLLM pipeline. No license restrictions on derivatives under Apache 2.0.
In the operating system
Where it fits
Sits at the core reasoning layer of an AI operating system: ingests structured ops data (tickets, logs, configs) and semi-structured input (PDFs, screenshots), outputs decisions, drafts, and tool calls. Feeds into workflow-automation and knowledge-retrieval layers above (route to ticketing, retrieval-augmented document lookup, agent orchestration). Below it, you own the data ingestion and prompt engineering; above, your ops platform consumes the model's output and executes actions.
Data control & security
Running this model in your own environment is an architectural choice that keeps data residency under your control: prompts, images, and completions never transit to external inference endpoints. This simplifies compliance workflows (no 'which vendor has my data?' questions) and reduces exfiltration surface. Note: the model itself is a language model; data security depends on your deployment environment (network isolation, RBAC, container security, VRAM encryption if required). Quantization to NVFP4 does not add cryptographic guarantees—it reduces memory and latency. Validate your own security posture independently.
Hardware footprint
ESTIMATE (NVFP4 quantization, with Triton KV cache optimization): ~13–16 GB VRAM for inference at batch size 1 (prefill + decode). At c=64 concurrency and `--gpu-memory-utilization 0.85`, allocate 24–28 GB per GPU. Tested and stable on DGX Spark (unified memory, 576 GB total) and Blackwell B100 (dedicated 192 GB HBM). Single A100 80GB or H100 80GB can run this reliably; RTX 6000 Ada (48 GB) is tight but feasible for smaller batches. Prefill throughput ~200–300 tok/s; decode (with DFlash drafting) ~40–60 tok/s (category-dependent).
Integration
Expose via OpenAI-compatible API (vLLM native, port 8000). Bind to internal VPC; no public endpoint unless you explicitly create one. Call from your ops tools (Slack bots, internal dashboards, ticketing systems) using standard `POST /v1/chat/completions` or `POST /v1/completions`. For multimodal: encode images as base64 in the request. Batch non-urgent work (document indexing, periodic summarization) via async jobs; reserve real-time requests (support ticket draft, incident response) for immediate inference. Tool-use routing via `--tool-call-parser qwen3_coder` (built into the serve config); your orchestration layer consumes tool calls and executes against your APIs.
When it's not the right fit
- —Constrained latency (<50ms TTFT): at 200+ tok/s prefill, TTFT for typical support queries (200–500 input tokens) is 100–300ms. Acceptable for async draft generation; inadequate for real-time interactive co-pilot.
- —Sub-4-bit inference required: NVFP4 is the floor here. If you need 2-bit or INT2, look elsewhere.
- —No multimodal capability needed: if you're purely text-to-text, a smaller 7B or 13B variant (same AEON family) saves hardware; this model's image encoder overhead is sunk cost if unused.
- —Frequent model swaps or A/B testing: containerized deployment is fast, but restarting vLLM and reloading weights takes 30–60s. For rapid iteration, consider a model-serving platform (vLLM in k8s with shared model cache) that decouples weight loading.
Alternatives to consider
Meta Llama 3.1 70B
Larger, no quantization overhead, strong reasoning. Requires 2–3× the VRAM (45–50 GB for BF16). More expensive to run privately; slower per-token on smaller clusters. Good if you have the hardware and want maximum quality; worse fit for footprint-constrained ops.
Mistral Large 123B MoE
Dense 123B, sparse activation (fewer active params per token). Higher quality reasoning and code. Requires 60+ GB VRAM even sparsely. Overkill for most ops tasks (ticket triage, doc extraction); better for research/analysis workflows. License less permissive than Apache 2.0 for some use cases.
Qwen2.5 32B
Same Qwen family, smaller 32B, no multimodal. Simpler deployment, slightly lower memory. Good if you don't need image support and want faster iteration. Trade: less reasoning depth, smaller context, fewer ops tool-use benchmarks.
Related open models
FAQ
Can I fine-tune this model and keep it private?
Yes. Use LoRA or full-weight tuning on your infrastructure. Merge the adapter into the base model offline, then serve the merged weights in vLLM. Apache 2.0 license permits derivatives without disclosure. No license phone-home or vendor approval needed.
What about commercial use—can I build a product on top?
Apache 2.0 permits commercial use, derivative works, and distribution. You may sell a product that uses this model as its backbone without paying royalties or getting permission. Caveat: review the license fully and consult legal if your jurisdiction imposes special requirements. The license is OSI-approved and widely used; low friction for commercial deployment.
How do I monitor and log inference in a private deployment?
vLLM logs to stdout/stderr; capture via container logging (docker logs, journald, or k8s logging). Inference metrics (TTFT, TPOT, throughput) are available via Prometheus endpoints (if enabled in vLLM). Build a sidecar or scraper to ingest metrics into your observability stack (Datadog, Prometheus, Grafana). Prompts and completions stay on-disk or in memory only; you control how much you log and retain.
Do I need the external DFlash drafter, or can I use the built-in MTP head?
Depends on hardware. On unified-memory systems (DGX Spark), the external DFlash drafter is faster (~42.6 tok/s single-stream, ~340 tok/s at c=64). On dedicated-VRAM Blackwell GPUs, the native MTP head is simpler (no drafter pull required) and achieves parity speed. Both are in the same container image. The model card's GitHub repo has hardware-routing guidance; start with the quickstart (DFlash) if unsure.
Build Private Ops AI with Your Own Model
Don't send operational data to external APIs. Deploy Qwen3.6-27B privately on your infrastructure—fine-tune on proprietary docs and processes, integrate with your ops stack, retain full data control. LLM.co helps you build the automation layer: containerized serving, agent orchestration, knowledge retrieval, and workflow integration. Start a deployment conversation with our team.