Open-Weight LLM · Private & Custom AI
Nex-N2-mini-AWQ-INT4
Agentic reasoning model optimized for long-horizon task automation, tool use, and code execution—built for ops teams running multi-step workflows on private infrastructure.
Nex-N2-mini is a 35B MoE model trained on Qwen3.5 with 'Agentic Thinking' capabilities: adaptive reasoning depth, coherent tool calling, and environmental feedback loops. For ops teams, this means automating complex departmental tasks (support escalation, research, system diagnostics, code deployment) without sending reasoning chains or intermediate results to external APIs.
Model facts
Private deployment
Run Nex-N2-mini-AWQ-INT4 in your own environment
Nex-N2-mini (24.43 GB quantized) deploys on 2× H100 with tensor parallelism via a customized sglang fork (provided by Nex-AGI). Company retains full model weights and inference logs; all reasoning and tool outputs stay within your infrastructure. Requires CUDA 13.0+ and Docker support (prebuilt nexagi/sglang:v0.5.12 available). Setup is non-trivial—specialized reasoning/tool-call parsers needed—but once running, zero external dependency for operational workflows.
Operational AI use cases
Support escalation & diagnosis automation
Route tickets by running on-demand reasoning through Nex-N2-mini: extract intent, check internal knowledge bases (via tool calling), decide severity/category, and auto-draft responses. Adaptive thinking skips deep reasoning on routine queries; prioritizes it on ambiguous cases. Data stays private; no customer tickets sent to third-party APIs.
DevOps & incident response workflows
Ingest logs/alerts, trigger Nex-N2-mini to analyze anomalies, call diagnostic tools (system queries, metric checks), suggest remediation steps, and execute approved actions. Agentic loop: interpret feedback from each action, refine next step, close the loop. Keeps sensitive infrastructure data and runbooks in-house.
Research & document synthesis
Multi-turn deep research: model reasons on information gaps, calls internal search/knowledge tools, synthesizes findings, identifies what's missing, iterates. Supports long-context reasoning (context length unknown—verify against use case). Useful for competitive analysis, policy research, or compliance document generation without exfiltrating source materials.
Custom AI
As a base for custom AI
Strong base for custom ops agents. Nex-N2-mini's agentic thinking framework + tool-calling APIs let you wrap proprietary domain data (CRM, ticketing, monitoring dashboards) as callable functions. Post-train on your SOP documents and past workflow traces to personalize reasoning and tool selection. Quantized 24.43 GB footprint keeps iteration cycles fast on modest private GPU clusters.
In the operating system
Where it fits
**Agent layer**: Nex-N2-mini *is* the reasoning engine for multi-step tasks. Sits between a workflow orchestrator (receives task spec) and tool/API layer (executes actions). Feed it domain context via prompt engineering or retrieval; receive structured actions and reasoning traces. Pairs with knowledge retrieval (RAG) to ground agentic decisions in company data without bloating model context.
Data control & security
Self-hosted deployment ensures intermediate reasoning, tool outputs, and long-context session data never leave your environment. No chat logs sent to external inference services. *Architecture benefit*: you own the data lifecycle. However, this shifts responsibility: model weights themselves (Nex-N2-mini base) are not formally audited for backdoors; quantization process (AWQ-INT4 by cyankiwi) adds one more trust boundary. Compliance (HIPAA, SOC2, etc.) depends on your infrastructure hardening, not the model alone.
Hardware footprint
**Estimate (quantized AWQ-INT4):** ~24.43 GB loaded weights. Running inference: 2× H100 (80 GB each) with tensor parallelism TP=2 covers small-to-medium batch inference. Batch size 1–4 recommended for real-time ops workflows. *Unquantized BF16 base (Nex-N2-mini ~35B params) would be ~70 GB—verify exact model card for unquantized variant before committing.*
Integration
Deploy sglang server on internal GPU cluster; expose OpenAI-compatible REST API to your orchestration layer (Airflow, n8n, custom Python agents). Tool calling uses Qwen3.5 coder format—map your internal APIs (Jira, Slack, Datadog, etc.) as JSON tool schemas. Inference latency unpredictable due to adaptive thinking depth; budget 5–60s per query depending on task complexity. Requires monitoring/alerting on model server health and reasoning timeouts.
When it's not the right fit
- —Context length unknown—if your ops tasks require >100k tokens of historical context or very long reasoning chains, test against your typical workload first.
- —Reasoning latency unpredictable (adaptive thinking). Real-time support chat needing <2s response times may chafe; better for async batch workflows (hourly reports, batch code review, etc.).
- —Reasoning/tool-call parsers are Nex-AGI proprietary (sglang fork required). No official support in standard vLLM, TGI, or other inference stacks—coupling to their toolchain.
- —Small quantization experiment (cyankiwi's AWQ-INT4): no third-party validation of quality vs. original. Benchmark data is for unquantized Nex-N2-mini—actual quantized throughput/accuracy trade-off unconfirmed.
Alternatives to consider
DeepSeek-V3 (32B) or DeepSeek-R1-Distill-Qwen-32B
Similar scale, open-source agentic tuning, broader inference stack support (vLLM, TGI). No private sglang fork required. Comparable coding + tool-use benchmarks. Slightly lower reasoning cost per token.
Qwen3.5-32B-Base (unquantized)
Parent model: full control, standard quantization paths (GGUF, GPTQ, bitsandbytes). Requires your own agentic instruction tuning. More flexibility, less turn-key reasoning.
Llama 3.3-70B-Instruct
Larger scale, excellent code and reasoning on standard benchmarks. Requires 2× H100 like Nex-N2-mini. Broader community tools and integrations. Less specialized for agentic workflows (no built-in adaptive-thinking framework).
FAQ
Can I run this on a single A100 or H100?
No. Nex-N2-mini (24.43 GB quantized) + inference overhead + batch requires ~2× H100 with TP=2. A single 80GB H100 will run batch=1 slowly; not recommended for production ops.
Is this commercially usable in my private deployment?
Yes. Apache 2.0 license permits commercial use, modification, and distribution. However: (1) Nex-N2-mini is built on Qwen3.5-35B-A3B-Base—verify Qwen's license terms separately; (2) cyankiwi's quantization adds a derivative layer—review any commercial terms they may add. TL;DR: likely permissible, but audit license chain.
What if I want to fine-tune this on my proprietary workflows?
Possible. Nex-N2-mini base supports LoRA or full fine-tune on your ops task traces. Requires own GPU cluster for training (not provided by Nex-AGI). Quantized (AWQ-INT4) weights are inference-only—you'll need unquantized base from nex-agi/Nex-N2-mini to fine-tune. Training cost and convergence time unknown without trying.
What about context length for long reasoning tasks?
Unknown in provided data. Nex-N2-mini's context window is not listed. Contact nex-agi ([email protected]) or check GitHub before committing to tasks >50k tokens.
Build a Private Agentic AI System
Nex-N2-mini + LLM.co's ops infrastructure stack = multi-step workflow automation that never leaves your environment. Let's architect a custom agent for your team's biggest bottleneck. Schedule a systems review.