All industriesAI & operations
Is a private LLM for business worth it, or is an enterprise API enough?
By Yath Regmi, Founder, Kodec
Verified on Updated Published
Short answer
A private LLM for business is overkill for most owner-run companies. An enterprise API or a cloud-hosted model that does not train on your data is private enough for most work. Run your own model only if a client contract or a rule bars sending data to a vendor, and someone on your team can keep it running.
When this is the right call
In-house is the right call in a few cases. A client contract says data cannot leave your network. A rule makes you control every system that touches customer records. You run heavy, steady volume all day, so a machine you own costs less than paying by use. Or your data is so sensitive that you want no outside copy at all. If none of those fit, an enterprise API with the right settings gives you the best model, no hardware, and no one on your team called in the middle of the night when something breaks.
Private LLM for business: which option fits by data sensitivity and team size
| Your data | 1 to 10 people | 11 to 50 people | 51 to 200 people |
|---|---|---|---|
| Everyday business (marketing, scheduling, quotes) | Enterprise API | Enterprise API | Enterprise API |
| Customer records (names, addresses, job history) | Enterprise API, retention off | Enterprise API or VPC-hosted (your own private space in the cloud) | VPC-hosted |
| Regulated or privileged (client files, loan files) | VPC-hosted with zero retention | VPC-hosted, or on-prem if a contract demands it | On-prem or VPC-hosted, with an owner for uptime |
| Cannot leave the building (contract or rule says so) | Fully local, one machine | On-prem open-weights model | On-prem, with a named person on call |
This table is Kodec's judgment, not a rule from a regulator. It assumes you have no full-time IT or AI engineer below 51 people. Check any contract or rule that applies to you before you pick a cell. 'Enterprise API' means a business plan where the vendor says in writing that it does not train on your data. 'Retention off' means you asked for zero data retention where the vendor offers it.
Side by side
OpenAI API
| Trained on your data? | No. "Data sent to the OpenAI API is not used to train or improve OpenAI models," unless you opt in.1 |
|---|---|
| Does the vendor keep a copy? | Abuse-monitoring logs are kept up to 30 days by default. Zero Data Retention needs OpenAI's prior approval.1 |
| Who else can read it? | OpenAI says its business products are not trained on by default. We could not open that page directly, so check it yourself.2 |
| Page date | Fetched 2026-11-14. Page shows no date.1 |
Anthropic (commercial)
| Trained on your data? | No. "Anthropic may not train models on Customer Content from Services." Commercial Terms effective Jun 17, 2025.3 |
|---|---|
| Does the vendor keep a copy? | Privacy centre page dated Aug 18, 2026 covers training. Retention length: not found on the pages we fetched.4 |
| Who else can read it? | Feedback you submit (thumbs up or down) can be used for training. Team and Enterprise owners can turn feedback off.4 |
| Page date | Terms effective Jun 17, 2025. Privacy article Aug 18, 2026.3 |
AWS Bedrock
| Trained on your data? | Model providers do not have access to Bedrock logs or to customer prompts and completions.5 |
|---|---|
| Does the vendor keep a copy? | You pick a retention mode: none, default or aws_review. Some models need aws_review, which keeps inputs and outputs up to 30 days inside AWS.6 |
| Who else can read it? | AWS may review content only for models whose provider requires human review. It is not shared with the provider.6 |
| Page date | Fetched 2026-11-14. Page shows no date.6 |
Azure (Foundry / Azure OpenAI)
| Trained on your data? | Not used to train foundation models without your permission or instruction. Not available to OpenAI or other model providers.7 |
|---|---|
| Does the vendor keep a copy? | Flagged samples may be stored for human review. Approved customers can apply to turn this off.7 |
| Who else can read it? | Authorized Microsoft reviewers, only on flagged prompts and completions.7 |
| Page date | Microsoft Learn page dated May 18, 2026, updated Jun 5, 2026.7 |
| What it is | OpenAI API | Anthropic (commercial) | AWS Bedrock | Azure (Foundry / Azure OpenAI) |
|---|---|---|---|---|
| Trained on your data? | No. "Data sent to the OpenAI API is not used to train or improve OpenAI models," unless you opt in.1 | No. "Anthropic may not train models on Customer Content from Services." Commercial Terms effective Jun 17, 2025.3 | Model providers do not have access to Bedrock logs or to customer prompts and completions.5 | Not used to train foundation models without your permission or instruction. Not available to OpenAI or other model providers.7 |
| Does the vendor keep a copy? | Abuse-monitoring logs are kept up to 30 days by default. Zero Data Retention needs OpenAI's prior approval.1 | Privacy centre page dated Aug 18, 2026 covers training. Retention length: not found on the pages we fetched.4 | You pick a retention mode: none, default or aws_review. Some models need aws_review, which keeps inputs and outputs up to 30 days inside AWS.6 | Flagged samples may be stored for human review. Approved customers can apply to turn this off.7 |
| Who else can read it? | OpenAI says its business products are not trained on by default. We could not open that page directly, so check it yourself.2 | Feedback you submit (thumbs up or down) can be used for training. Team and Enterprise owners can turn feedback off.4 | AWS may review content only for models whose provider requires human review. It is not shared with the provider.6 | Authorized Microsoft reviewers, only on flagged prompts and completions.7 |
| Page date | Fetched 2026-11-14. Page shows no date.1 | Terms effective Jun 17, 2025. Privacy article Aug 18, 2026.3 | Fetched 2026-11-14. Page shows no date.6 | Microsoft Learn page dated May 18, 2026, updated Jun 5, 2026.7 |
Vendor by vendor
1. Enterprise API, no training on your data
Pay by use. Per-token prices: not checked for this page.
2. Model hosted in your own cloud account (VPC, your own private space in the cloud)
Cloud GPU rental. Lambda lists H100 PCIe at $3.79 per GPU hour and A6000 48 GB at $1.09, taking effect Nov 16, 2026.8
Good
3. Self-hosted AI for companies: open-weights model on your own servers (on-premise LLM deployment)
Hardware purchase prices: not published by NVIDIA, AWS or Azure on the pages we fetched. Layer3 Labs, a consultancy, lists NVIDIA DGX Spark at $4,699 as of Aug 27, 2026, per its own page.11
Good
- No outside copy of your data. You control updates and logs.
- NVIDIA says DGX Spark has 128 GB of memory and can run inference on models up to 200 billion parameters.9
Not so good
- You own security, patching, backups and uptime. Open models can trail the best paid ones on hard tasks. We have no benchmark for your work, so test it.
- One seller's cost table gives three different entry server prices on the same page ($8-12k, $5-10k, $10-15k). Get independent quotes.12
4. Fully local (one machine, no network)
Workstation price: not published on NVIDIA's page. Layer3 Labs mentions a Mac Studio from about $2,500, per its own page (checked 2026-11-14).11
Good
- Nothing leaves the room. Good for one person or a small team with a few documents.
Not so good
- Small models only. Only a few people can use it at once. If the machine breaks, the assistant is gone.
Kodec builds private internal assistants on your own data, at the level your data needs
Kodec is a technology and operations firm for owner-run companies in home services, real estate, mortgage and law. One thing we build is an internal assistant that answers from your own files: past quotes, loan checklists, intake notes, firm precedents. We can run it on an enterprise API with retention settings, in your own cloud account, or on a model you host. We start by asking what the data is and who may see it, then choose the lowest-effort level that meets that rule. We then track the hours it saves against what it costs to run. If you are a large regulated firm with a security team and a long procurement process, a firm such as IBM, Accenture or Deloitte, or your cloud provider's own team, may fit better. Kodec's fit is owner-run companies of about $1M to $20M.
What we found
Verified terms
"Private" API still keeps some copies. Here is where, and the switch to turn it off.
We read each vendor's own page on 2026-11-14. OpenAI keeps abuse-monitoring logs up to 30 days by default, and Zero Data Retention needs prior approval. On AWS Bedrock, retention is a setting, and the AWS page says some models require 'aws_review', which keeps inputs and outputs up to 30 days inside AWS where AWS staff may review them. It says that content is not shared with the model provider. Azure may store flagged samples for human review unless Microsoft approves modified abuse monitoring. So 'no training on your data' and 'no copy of your data' are two different promises. Before you sign, ask for the second one in writing.167
Original math
On-premise LLM hardware or rental: what an always-on GPU costs per month
A rented GPU bills for every hour, busy or idle. At Lambda's listed rates (taking effect Nov 16, 2026), an H100 PCIe at $3.79 an hour costs about $2,767 a month if left on for 730 hours. An A6000 with 48 GB at $1.09 costs about $796. So a rented model only wins when your monthly pay-by-use bill would be higher than that, and before you count anyone's time. If your team sends a few hundred questions a day, check your real API bill first. The hours and the math are ours. The hourly rates are Lambda's.8
Original math
Hidden running costs: updates, evals and uptime
A model you host is a system you run. Updates: new model versions and security patches land often, and each one can change answers. Evals: you need a set of 30 to 50 real questions with known good answers, and you re-run them after every change, or you will not know if the assistant got worse. Uptime: someone must notice when it is down and fix it. Example: if one person spends 2 hours a week on these three jobs at $75 an hour, that is about $650 a month, which is about 80% of the rental price of the A6000 above. The 2 hours and $75 are invented inputs. Replace them with yours. None of the seller pages we read priced this time.8
Who wrote this number
Who wrote the top pages for this question
We read four of the top results on 2026-11-14. All four are written by companies that sell private-AI work: an IT services firm, an AI consultancy, a law-firm AI implementer and a hosting vendor. None cites OpenAI's, Anthropic's, AWS's or Azure's data terms. One page gives three different entry-level server prices in one article ($8,000 to $12,000, $5,000 to $10,000, and $10,000 to $15,000). Another gives hardware prices but no running costs, no break-even and no refresh plan. That is why this page cites the vendors' own pages and shows its math.1211
Verified terms
What the rule text says: law and mortgage
The word in the law rule is 'reasonable', so a lawyer must weigh the vendor's terms and settings. It does not say 'host it yourself'. Your state's version of the rule is the one that binds you. An AI vendor that sees loan files is a service provider under the Safeguards Rule. This is not legal advice.151314
| Industry | Rule | What it says |
|---|---|---|
| Law | ABA Model Rule 1.6(c) | "A lawyer shall make reasonable efforts to prevent the inadvertent or unauthorized disclosure of, or unauthorized access to, information relating to the representation of a client." |
| Mortgage | FTC Safeguards Rule, 16 CFR 314.2(h)(2)(xi) | A mortgage broker is a financial institution because brokering loans is a financial activity. |
| Mortgage | 16 CFR 314.4(c)(3) | "Protect by encryption all customer information held or transmitted by you." |
| Mortgage | 16 CFR 314.4(f)(1) | Oversee service providers, including "taking reasonable steps to select and retain service providers that are capable of maintaining appropriate safeguards." |
| Home services | No sector rule cited | Some state privacy laws may apply to customer data. Ask your lawyer. |
How we checked
On 2026-11-14 we read OpenAI's API data page, Anthropic's Commercial Terms and privacy article, AWS's Bedrock data-protection and data-retention pages, and Microsoft's Foundry data-privacy page. We read Lambda's price page, NVIDIA's DGX Spark page and AWS's P5 page. We read the rule text of 16 CFR 314 through Cornell Law's copy of the rule; check the current eCFR for later changes. We could not open OpenAI's enterprise-privacy page or the ABA's Rule 1.6 page directly. We read four seller-written guides to see what they say. Prices change. Lambda's listed prices start Nov 16, 2026, so re-check them.
Questions
A private LLM for a company is a large language model that answers from your data without your data being used to train anyone else's model. That can mean an enterprise API with a no-training promise, a model in your own cloud account, a model on your own servers, or one on a single local machine. Where it runs changes who can see your data.
Not automatically. Hosting it yourself keeps data inside your network, but you then own patching, access controls, backups and logs. A vendor's enterprise API comes with written data terms and a security team. Self-hosting is safer only if you run it well. For a company with no IT staff, a careful API setup can be the safer choice.
It depends on the model size. NVIDIA says its DGX Spark has 128 GB of memory and can run inference on models up to 200 billion parameters and tune models up to 70 billion. Cloud rental starts near $1.09 per GPU hour for a 48 GB card at Lambda (from Nov 16, 2026). Buying prices were not published on the vendor pages we read.
No rule we found says so. ABA Model Rule 1.6(c) asks a lawyer to make reasonable efforts to prevent unauthorized disclosure of client information. What is reasonable depends on the vendor's terms, settings and the client's wishes. Check your state's rule and any client agreement. This is not legal advice.
The rule treats a mortgage broker as a financial institution, and it requires encryption of customer information and oversight of service providers. An AI vendor that handles loan files is a service provider you must choose with care. It does not ban cloud AI. Ask your compliance counsel how it applies to you.
It is overkill when your data is everyday business information, your team is small, no contract or rule requires it, and nobody on staff can run it. In that case, use an enterprise plan, turn off training and retention where offered, and revisit when your data or volume changes.
Sources
- 1.OpenAI: How your data is used (API) accessed Nov 14, 2026
- 2.OpenAI: Enterprise privacy (could not open; wording seen only in search results) accessed Nov 14, 2026
- 3.Anthropic: Commercial Terms, effective Jun 17, 2025 accessed Nov 14, 2026
- 4.Anthropic: Is my data used for model training? (dated Aug 18, 2026) accessed Nov 14, 2026
- 5.AWS: Amazon Bedrock data protection accessed Nov 14, 2026
- 6.AWS: Amazon Bedrock data retention accessed Nov 14, 2026
- 7.Microsoft Learn: Data, privacy and security for Foundry Models sold by Azure (updated Jun 5, 2026) accessed Nov 14, 2026
- 8.Lambda: GPU cloud pricing (effective Nov 16, 2026) accessed Nov 14, 2026
- 9.NVIDIA: DGX Spark accessed Nov 14, 2026
- 10.AWS: EC2 P5 instances accessed Nov 14, 2026
- 11.Layer3 Labs: Private AI for business (seller, updated Oct 25, 2026) accessed Nov 14, 2026
- 12.Petronella Technology Group: private AI deployment guide (seller) accessed Nov 14, 2026
- 13.16 CFR 314.2 definitions (Cornell LII copy) accessed Nov 14, 2026
- 14.16 CFR 314.4 elements of the information security program (Cornell LII copy) accessed Nov 14, 2026
- 15.ABA Model Rule 1.6 (could not open directly; text confirmed via search result) accessed Nov 14, 2026
Where to go next
Want to see which of your own calls turned into paying customers? See how Leverage works.
Related answers
- Home servicesAI & operations
Is AI for HVAC companies worth it in 2026, and where does it pay off?
Updated Nov 14, 2026
- All industriesAI & operations
What can AI agents for business operations actually do?
Updated Nov 14, 2026
- LawAI & operations
What can AI for law firm management do for intake, billing and operations?
Updated Nov 14, 2026