AI on your own computer or in the cloud? How to compare costs

The question is not where a token costs less

A business wants an assistant to read documents and prepare answers. The alternatives seem simple: pay a remote service per use, or buy and operate a local machine. The first shows a service bill; the second seems free after purchase. But maintenance staff, idle time, and peak capacity can change the decision. This article helps operational and technical managers construct a checkable comparison before discussing brands or models. The practical question is: at what volume does local operation become cheaper, and can that volume actually be served?

We develop monthly cost equations, derive break-even volume, and compare it with a capacity constraint. All prices, volumes, and numerical throughput are educational assumptions, not current price lists, quotations, or computer measurements. We ran no models or load tests. We did execute the cost and sensitivity calculations and preserve results and figures. The conclusion recommends no provider: it explains what evidence to collect and when an apparently favorable comparison is incomplete.

First make the two services comparable

A request must mean the same work on both routes: the same objective, comparable documents, minimum quality, and acceptable response time. If a local model responds quickly but needs correction while the remote one solves the task, per-call cost does not compare equivalent services. Counting only tokens, the text units processed by a model, can also mislead: work and answer lengths vary. We assume homogeneous requests below; real data needs workload classes or a weighted mean cost on the actual case mix.

The FinOps Foundation distinguishes resource-efficiency metrics from work-outcome metrics. It is a methodological reference for choosing the denominator, not proof that cloud is cheaper or that AI creates value in a particular business. Our equations and scenarios are our analysis, not Foundation results.

Separate availability costs from usage costs

Let F be monthly cost to keep a route available even with no requests, and v the mean additional cost of a homogeneous request. Local F may include the machine' s economic allocation, management, and reserved capacity; remote F may include integration and minimum commitments. Human work is not always fixed and energy is not always variable: classify costs by actual behavior. Buying a machine with available cash does not eliminate the cost of its use or its alternative use for another task.

Use a common horizon. Do not compare an upfront purchase with one monthly bill without explaining lifetime, maintenance, and residual value. Equivalent monthly cost and cash flow answer different questions. Here F is simply an assumed monthly parameter, not tax depreciation or an offer. Initial expenses must be allocated or displayed separately under transparent assumptions. Do not count both purchase and the allocated share of that same purchase in one total.

The simplest model we can verify

N is requests per month and C euros for that month. Subscripts L and C mean local and cloud; F is euros per month and v euros per request. For a fixed month the equation adds availability and usage. It assumes constant v, no capacity step, and the same request mix. Tiers, discounts, caching, and autoscaling need different functions; a universal straight line cannot magically absorb them.

Use F_L = 600 euros/month and v_L = 0.0002 euros/request; F_C = 50 and v_C = 0.005 in the same units. These invented values illustrate the mechanism. At 1000 requests local costs 600.20 euros and cloud 55: dividing by 1000 gives 0.6002 and 0.055 euros per request including fixed costs. At zero requests costs remain 600 and 50 euros, but per-request cost is undefined. Calling unit cost zero when unused would hide the idle cost we want to assess.

Break-even: a threshold conditional on assumptions

To find equal cost, set C_L = C_C and separate fixed from proportional terms. If local has higher fixed but lower variable cost, volume can offset the initial difference. Divide only when v_C differs from v_L. Equal variable costs give no such threshold: fixed costs decide. A negative result is not a positive customer target; inspect which curve dominates for nonnegative N.

Continuous break-even is about 114583.33 requests/month. For integer counts under these exact assumptions, local becomes cheaper from 114584 requests. That does not mean a business with this demand should buy a machine: feasibility is still unchecked. The calculation makes the demand hypothesis explicit. At 100000 requests costs are 620 versus 550 euros, favoring cloud; at 200000, 640 versus 1050, favoring local only within the cost model. The table retains higher volumes to show where arithmetic can exceed operational feasibility.

In the last column, 1 means only N is within nominal K; 0 means it exceeds K. It judges neither model quality nor latency compliance.

An inexpensive machine can be insufficient

Add assumed local capacity of 0.5 requests per second over 160 available hours per month. No real model or board is claimed to achieve this. Rate times duration gives only nominal capacity: 288000 requests. It is a necessary condition for processing the month in the assumed time, not a guarantee about queues, memory, or deadlines. One million requests formally costs 800 euros on the local line versus 5050 in cloud, but exceeds assumed local capacity. That is not a valid quotation for serving a million requests: recalculate an architecture capable of doing so.

Even 200000 requests below monthly capacity can arrive in an unsuitable pattern. If mornings bring two per second while service completes 0.5, the queue grows by 1.5 per second during that interval. Idle evening capacity cannot answer the morning user earlier. Options include deferring nonurgent work, adding capacity, or a hybrid route; each changes requirements and cost. We did not simulate this queue and report no latency percentiles. The counterexample shows only that monthly average cannot certify service.

Reddi and colleagues' historical MLPerf Inference Benchmark, November 2019 preprint v1, distinguishes offline processing from latency-constrained service. It is methodological context, not a current leaderboard or local/cloud comparison on our task. We reproduced none of its tests or results.

How much does a mistaken estimate change the conclusion?

Local fixed cost is often disputed: existing equipment appears free, while a supported system can need more work than expected. Holding other parameters fixed, F_L = 300 euros gives break-even about 52083.33 requests; 1200 gives 239583.33. These are not three product estimates: they isolate one parameter. Such sensitivity means fixed cost should be measured and justified before presenting a precise threshold. Extra decimal places do not make an uncertain assumption trustworthy.

The left curves are calculations, not offers. The local segment above 288000 requests is dashed because it exceeds nominal capacity of the assumed single service; reading it as an available price is incorrect. Break-even occurs before that limit but certifies neither latency nor quality. On the right, three bars show threshold shifts with fixed cost. The views answer different questions: where curves cross, and how sensitive that position is to assumptions.

From cost per call to cost per usable task

If only some answers are usable, change the denominator. Let U be acceptable tasks after independent verification: C/U is technology cost per usable task. Add review and repair costs to the numerator when accounting for that work. Do not treat model scores as correctness probabilities: observe U under the same criteria for both routes. We report no U because no model evaluation was run. That missing evidence matters to the decision and must not be filled with plausible numbers.

Service boundaries must also match. An inference price may exclude retrieval, storage, transfers, authentication, and search maintenance. Local operation may need backup, monitoring, updates, and failure recovery. Cloud may have quotas, commitments, transfers, or contractual dependencies to inspect. These costs are not attributed to every provider: verify the actual quotation. Sensitive data and access need evaluation across the path: “local” does not automatically mean protected, and “cloud” alone does not define every flow or policy.

A reproducible comparison before deciding

To turn arithmetic into evaluation, freeze authorized tasks and a result-acceptance criterion. Repeat cases across alternatives, logging model/runtime versions, context size, concurrency, and settings. Measure time to first visible output and to usable completion, separating network, queue, and compute where possible. Record median and percentiles, errors, retries, and correction time. These are future test instructions, not results obtained here. Preserve failures and out-of-scope cases, not just fast successes.

Test demand beyond the average: short and long requests, plausible peaks, and idle periods. Shared equipment must not be assessed as exclusive if other processes consume memory. If maintenance-time availability is required, state redundancy and spare capacity and recalculate F and v. A hybrid route can handle some cases locally and others remotely, but routing, dual integrations, and cases failing both routes have costs. Include a non-AI workflow if it solves the problem at comparable quality and work.

What the code proves and which decision it supports

The program uses Decimal for monetary inputs, calculates six volumes, and repeats break-even for three local fixed costs. It is deterministic, so no seed is needed. Assertions check cloud is cheaper at 100000 requests and local at 200000 within the assumed lines, plus nominal capacity 288000. Arithmetic precision does not reduce uncertainty of invented prices. JSON and figures let readers change assumptions and audit calculations, not infer performance of the cover machine. The cover is illustrative, not an EL-AI installation.

We can answer without slogans: local can offset higher fixed cost only if useful demand, variable-cost difference, and capacity permit; cloud may suit low volumes without being best for every requirement. Here break-even lies between 100000 and 200000 requests, but assumed local capacity cannot serve one million and peaks remain untested. A concrete choice needs actual costs, equivalent tasks, and alternative tests. We have shown no business savings, investment return, or product superiority: we built auditable arithmetic exposing necessary decision conditions.

Sources and reproducible materials

FinOps Foundation — Unit Economics; consulted 28 October 2026.

Reddi et al. — MLPerf Inference Benchmark, arXiv:1911.02549v1, 6 November 2019; sections 4.5–4.7 and 6.

MLPerf Inference Benchmark — version-specific bibliographic record.

from decimal import Decimal as D
from pathlib import Path
import json,sys
# All costs and capacity are invented scenario inputs, not current vendor prices.
FL,FC=D('600'),D('50');VL,VC=D('.0002'),D('.005')
volumes=[0,1000,10000,100000,200000,1000000]
rows=[]
for n in volumes:
 local=FL+VL*n;cloud=FC+VC*n
 rows.append(dict(monthlyRequests=n,localEUR=str(local),cloudEUR=str(cloud),localEURPerRequest=str(local/n) if n else None,cloudEURPerRequest=str(cloud/n) if n else None))
break_even=(FL-FC)/(VC-VL)
assert FL+VL*100000>FC+VC*100000
assert FL+VL*200000<FC+VC*200000
assert FL+VL*114583>FC+VC*114583 and FL+VL*114584<FC+VC*114584
assert D('2')-D('.5')==D('1.5')
capacity=D('.5')*D('160')*3600
assert capacity==288000
sensitivity=[dict(localFixedEUR=str(f),breakEvenRequests=str((f-FC)/(VC-VL))) for f in [D('300'),D('600'),D('1200')]]
assert sensitivity[0]['breakEvenRequests'].startswith('52083.') and sensitivity[2]['breakEvenRequests'].startswith('239583.')
result=dict(sensitivity=sensitivity,python=sys.version.split()[0],seed=None,scope='Hypothetical equal-workload monthly cost arithmetic; no vendor quotation, hardware throughput, model quality or investment return measured',assumptions=dict(localFixedEUR=str(FL),cloudFixedEUR=str(FC),localVariableEURPerRequest=str(VL),cloudVariableEURPerRequest=str(VC),localAssumedRequestsPerSecond='.5',assumedHoursPerMonth='160'),breakEvenRequests=str(break_even),nominalMonthlyCapacityRequests=str(capacity),peakIncomingRequestsPerSecond='2',peakExceedsAssumedCapacity=True,rows=rows)
Path(__file__).with_name('results.json').write_text(json.dumps(result,indent=2),encoding='utf8')
print(json.dumps(result))

Code, data, and instructions · JSON. Educational calculations executed with Python 3.14.0; figures with Matplotlib 3.11.2. AI-assisted analysis, without claiming peer review or human review. Original illustrative ImageGen cover: it does not document EL-AI people, premises, or installations. Sources accessed 28 October 2026.