Training an Open Weights Private LLM: Cost vs Benefit Analysis

The instinct to "train our own model" arrives about six months into every enterprise AI program. The public API bill has climbed past the pilot budget, legal has flagged three jurisdictions where sending prompts offshore is now a documented risk, and someone on the board has read that DeepSeek trained a 671B parameter model for roughly $5.6 million. The question lands on the CTO's desk: should we train our own?

The honest answer is almost never "train from scratch." It is usually "start from open weights, and decide which of three lifecycle stages your problem actually requires." Pretraining, continued pretraining, and fine-tuning are three different budgets with three different risk profiles, and picking the wrong one is how AI programs run out of runway before they ship.

What follows is a line-item cost model tied to the specific open weights families most enterprises are evaluating: Llama, Mistral, DeepSeek, Qwen, and Gemma. Every one of these is available for private deployment through the open-source LLM downloads catalog, and the decision of which to run, and how far to train it, is the difference between a six-figure program and an eight-figure one.

The Three Training Budgets, and Which One You Actually Need

Full pretraining is what Meta, Google, and DeepSeek do. It means starting from randomly initialized weights and running trillions of tokens through a cluster for weeks. Meta's Llama 3.1 release consumed 39.3 million H100 GPU-hours across the 8B, 70B, and 405B variants, with the 405B alone at 30.84 million hours. At a September 2026 median on-demand rate of $3.41 per H100-hour, that 405B run alone prices out above $105 million in raw compute, before staff, data curation, or failed runs.

Continued pretraining is the middle path. You take an existing open weights checkpoint and keep training it on new tokens, usually a domain corpus: internal case law, ten years of claims notes, a decade of clinical guidelines. Budgets range from tens of thousands to low millions depending on token volume, and the output is a base model that speaks your dialect before any instruction-tuning happens.

Fine-tuning is the smallest budget and the most common answer. Techniques like LoRA and QLoRA freeze the base weights and train small adapter matrices on top. Parameter-efficient methods cut memory requirements by 10-20x versus full fine-tuning while retaining 90-95% of quality, which is why a 7B model that would need roughly $50,000 in H100 hardware for full fine-tuning can be adapted on a $1,500 RTX 4090.

What the Open Weights Options Actually Cost to Train

The cost of a training run is a function of three numbers: parameter count, token volume, and GPU-hour rate. The models on the shortlist for private enterprise deployment cluster into three tiers.

Small (7B-9B): Llama 3.1 8B, Mistral 7B, Gemma 2 9B, Qwen 2.5 7B. These fit in the memory of a single 80GB accelerator and are the workhorses of regulated deployments. A 50,000-sample instruction-tuning run on Llama 3.1 8B costs roughly $2.64 to $3.96 on a rented RTX 4090 using LoRA, and $15 to $23 for a full fine-tune on an A100. Continued pretraining on 10 billion domain tokens sits in the $8,000 to $15,000 range at current rates.

Mid (32B-70B): Llama 3.1 70B, Qwen 2.5 72B, Mistral Large. QLoRA on a single A100 80GB runs a 70B fine-tune for roughly $45 to $68 per run, versus $398 to $598 for a full fine-tune on an H100 cluster. Continued pretraining on 50 billion tokens lands between $80,000 and $180,000 depending on utilization.

Large (405B and MoE frontier): Llama 3.1 405B, DeepSeek-V3, Qwen 2.5 Max. Even QLoRA requires three H100s minimum. Full continued pretraining is a seven-figure exercise, and pretraining from scratch is the eight-to-nine-figure zone that puts you back in the neighborhood of the estimated $78 million for GPT-4 and $191 million for Gemini Ultra.

The Benefit Side, Denominated in Risk Removed

Cost is easy to model. Benefit requires naming what public API dependency actually costs, and it is not just the per-token bill. A Deloitte survey found 55% of enterprises avoid at least some AI use cases due to data security concerns, with an IBM industry survey placing 57% on data privacy as the biggest inhibitor to adoption. Those are the deals that never get signed because the model is on someone else's infrastructure.

The forward-looking risk is worse. Gartner projects that by 2027, more than 40% of AI-related data breaches will stem from the improper cross-border use of generative AI. If your model runs on your hardware, in your VPC, under your audit log, that risk category collapses to something you can actually govern. The hidden risks of public AI APIs compound quietly: prompt logging, training-data ingestion clauses, geographic routing decisions made without your knowledge, and vendor policy changes that arrive as fait accompli.

Benefit also shows up in specificity. A base Llama 3.1 70B is a generalist. A Llama 3.1 70B that has been continued-pretrained on your ten-year deposition archive and fine-tuned on your firm's brief style is not a generalist, and it is not something a competitor can rent. That asset accrues value with every quarter of use, which is the argument for treating training spend as capex rather than opex.

Model-by-Model: What Each Open Weights Family Buys You

Not every open weights model belongs in every deployment. The choice narrows fast once compliance, license, and community depth get factored in.

• Llama (Meta): the deepest tooling ecosystem, the most inference-optimizer support, and a community license that most regulated buyers can accept with legal review. The 8B and 70B variants are the default choice when you want the least friction between download and production.
• Mistral: strong Apache-2.0 licensed models, small enough to run on modest hardware, and the family most often chosen when license purity matters more than benchmark leadership.
• DeepSeek: frontier reasoning at a fraction of the training compute, but with a hosted-service data policy that should concern privacy-conscious users when using the API. The open weights, run inside your perimeter, sidestep that concern entirely.
• Qwen (Alibaba): the community's most-forked base model. Hugging Face counts 151,448 derivative models with 180 to 210 new ones per day and 39.6 million GGUF downloads per month, which translates to abundant fine-tuned variants for domain starts.
• Gemma (Google): the smaller, more permissively licensed alternative for edge and on-device work. Gemma has surpassed 150 million downloads with 70,000 variants on Hugging Face, and the 2B and 9B models are frequently the right pick for laptop-class or appliance deployments.

1. 1Llama 3.1Default 8B/70B production · Community license, needs review · Deepest tooling
2. 2MistralLicense-sensitive deployments · Apache 2.0 on most models · Broad and mature
3. 3DeepSeek-V3Reasoning, cost-per-token · Open weights, avoid hosted API · Fast growing
4. 4Qwen 2.5Multilingual and starting point for derivatives · Apache 2.0 on most sizes · Most-forked base
5. 5Gemma 2Edge, appliance, on-device · Gemma license, permissive · 150M+ downloads

The Decision Framework

The path from open weights download to a deployed private model follows a predictable sequence, and skipping steps is how budgets blow up. Every stage is a gate: if the cheaper option meets the requirement, stop there.

Most regulated enterprises land at step three or four. Retrieval augmentation on top of a stock open weights model, wrapped in a governed inference layer, solves 70% of production use cases. Fine-tuning is worth the money when you have a stylistic or format requirement RAG cannot satisfy, or when latency budgets forbid the retrieval hop. Continued pretraining is worth it when your domain vocabulary is genuinely alien to the base model, which is the case in specialized law, complex financial instruments, and a handful of clinical subspecialties. Pretraining from scratch, for the vast majority of enterprises, is not worth it and never will be.

That decision also interacts with hardware strategy. The tradeoffs between an appliance and a self-built GPU cluster shift depending on whether you are running inference only, fine-tuning quarterly, or attempting continued pretraining runs. A fine-tune-only shop can live on rented capacity. A continued-pretraining program justifies dedicated silicon.

Where the Money Should Go

The honest cost-benefit conclusion is that training spend is almost never the largest line item in a successful private LLM program. Data preparation is. Evaluation infrastructure is. Serving and governance are. A team that spends $40,000 on a fine-tune and $400,000 on a rigorous evaluation harness will outperform a team that inverts those numbers, because a model that scores well on the wrong benchmark is worse than a stock model that scores honestly on the right one.

Open weights make the training question tractable. They do not make it trivial. The right sequence is to pick the smallest model that clears your quality bar, apply the cheapest training technique that closes the remaining gap, and reinvest the savings in the evaluation and governance layers that decide whether the model is safe to put in front of a regulator, a client, or a patient. That is what a private LLM program looks like when the arithmetic is done first and the enthusiasm second.