Open-Weight LLM · Private & Custom AI
stories15M_MOE
A 36M-parameter mixture-of-experts model for testing narrative generation; suitable only for private prototyping, not production ops automation.
stories15M_MOE is a tiny MoE model (4 expert copies of TinyLLaMA-15M) designed as a test harness for router and expert architectures, not production workloads. An ops team might use it to experiment with MoE routing strategies in a private sandbox or as a lightweight baseline for narrative-heavy internal workflows (FAQs, knowledge docs, bedtime-story-like content generation).
Model facts
Private deployment
Run stories15M_MOE in your own environment
Self-hosting is straightforward: the model is ~36M params, runs on CPU or modest GPU (see hardware section). GGUF format support means inference via llama.cpp or similar. No gating. Private deployment keeps generated story content, fine-tuning data, and routing decisions inside your infrastructure—useful if you're testing MoE routing patterns or learning architectures without external API calls.
Operational AI use cases
Internal Knowledge & FAQ Generator
Use as a lightweight narrative engine to auto-generate onboarding docs, procedural guides, or internal knowledge-base entries from templates. Fine-tune via the included LoRA adapter approach to inject domain language. Output stays private; useful for teams generating large volumes of narrative-style internal content.
Customer Support Story/Context Summarization
In a support workflow, feed ticket narratives (customer stories, issue descriptions) into the model to auto-generate context summaries or suggested response templates. The MoE routing means different expert paths can handle different ticket types. Keep data in-house for compliance.
Operational Reporting & Narrative Summaries
Automate generation of narrative-style incident reports, weekly rollups, or status summaries from structured event logs. The model's lightweight size means fast generation for high-volume internal reporting. LoRA adapters can inject company tone/terminology.
Custom AI
As a base for custom AI
Limited. This model is explicitly a test harness, not a production base. If you're building a custom narrative-generation product (e.g., a children's story platform, internal docs generator), use it only as a prototype to validate the MoE architecture before upgrading to a larger, production-grade model. The LoRA adapter pattern (Shakespeare example) shows how to fine-tune, but the base model's quality is toy-level.
In the operating system
Where it fits
Experimental / prototype layer. In an LLM.co ops-AI stack, this fits as a knowledge-layer test harness or as a learning model for understanding MoE routing—not as the backbone for production workflows. Would sit behind a simple inference API but likely not in the critical path of agent or workflow automation.
Data control & security
Self-hosting keeps all prompts, outputs, and fine-tuning activity inside your network. No external API calls; no data sent to third parties. This is a data-control architecture win for compliance-sensitive orgs. The model itself offers no inherent security guarantees—output quality and safety depend on your guardrails, LoRA fine-tuning discipline, and input validation.
Hardware footprint
Estimated: ~150–200 MB (FP16/FP32 with quantization, GGUF format). CPU inference viable; GPU acceleration optional. For comparison: FP32 unquantized ~145 MB, quantized GGUF ~50–80 MB. Verify against your inference engine and precision target.
Integration
Wire via llama.cpp or text-generation-inference (TGI) REST endpoints. Lightweight enough for CPU inference, so no GPU infrastructure required for many orgs. Connect to ops platforms (Slack, Jira, internal wikis) via standard API wrappers. LoRA adapter loading is supported; consider version-controlling your adapted weights separately. Monitor output quality and router behavior; MoE routing logs may reveal expert specialization patterns.
When it's not the right fit
- —Requiring production-grade narrative quality or domain accuracy—this is a test model, not a commercial-grade language model.
- —Complex multi-hop reasoning or structured data understanding—36M params cannot reliably handle intricate logic.
- —Real-time, low-latency ops workflows where every millisecond counts; MoE routing overhead adds latency vs. dense models.
- —Non-English or specialized technical language—training data is limited; LoRA adaptation will be necessary and may be insufficient.
Alternatives to consider
TinyLLaMA-1.1B
Larger dense baseline (10x params), better general-purpose ops use (summarization, classification, simple automation). No MoE overhead; more stable for production.
Phi-2 (2.7B)
Lightweight, MIT-licensed, optimized for instruction-following and reasoning. Better fit for ops workflows (support tickets, incident response, log analysis).
Mistral-7B
Production-ready, Apache 2.0 licensed, strong narrative and reasoning capability. Larger footprint but much better quality for custom ops-AI applications.
Related open models
FAQ
Can I run this entirely in-house?
Yes. No gating, MIT license, GGUF format support, and modest size (36M) mean you can deploy on your own servers or air-gapped infrastructure. Inference via llama.cpp or TGI. All data stays private.
Is this commercially licensable?
MIT license permits commercial use, modification, and distribution. However, the model card explicitly states it is NOT intended for production. Use commercially at your own risk; quality and reliability are unvalidated. For a production ops-AI product, choose a larger, battle-tested model.
How do I adapt it to my domain (e.g., company tone or knowledge)?
The included LoRA adapter pattern (Shakespeare example) shows the way: collect domain data, fine-tune a small LoRA adapter on top of the frozen base, and load both at inference time. This keeps compute low and lets you iterate quickly without retraining the full model.
What is the context length?
Unknown from the provided model card. Check the upstream TinyLLaMA-15M repo or test empirically. Likely 512–2048 tokens; confirm before integrating into multi-turn ops workflows.
Build Custom Ops AI with Your Own Models
stories15M_MOE is a playground for MoE architectures and private narrative generation. Ready to run production ops-AI on your own infrastructure? Let LLM.co help you design a self-hosted, custom LLM stack that keeps data in your control and workflows in your hands.