How do you choose between cloud GPUs and on-prem for AI workloads in 2027?

By Kory White · Updated Nov 2026 · 12 min read

Close-up render of an NVIDIA H100 SXM GPU module on a black background, showing the central GPU package surrounded by memory stacks and power components

Direct answer

Choose by utilization first, then by the constraints that utilization cannot fix. Rent cloud GPUs when the hardware would sit idle for much of the year, when demand is bursty, when you need capacity in days, or when you cannot yet power, cool and staff a cluster. Buy and run your own when the GPUs will stay busy for most hours of their depreciation life, your data already lives on site, and you can fund space, power and people. The whole decision reduces to one inequality: the cloud rate times the hours you would bill, against the annual owned cost, which is capex divided by its useful life, plus power, plus colocation or room cost, plus staff.

The cloud side and the electricity side of that inequality can be filled with published figures. As read on November 8, 2026, CoreWeave lists an 8-GPU NVIDIA HGX H100 node at $49.24 per hour on demand, which is $431,342.40 for all 8,760 hours of a year (my multiplication). NVIDIA lists up to 700W for the H100 SXM, and the EIA's U.S. average commercial electricity price for August 2026 was 14.47 cents per kilowatt-hour, so the GPUs alone draw about $7,098.40 of electricity a year at full power. The other owned-cost terms (server price, colocation, staff, PUE, depreciation life) are left as variables: no vendor price list or primary source for them was read, so this page states no typical break-even point, and none should be quoted without your own quotes.

The break-even formula

Every formula term below is a variable you fill with your own quote, except where the next sections give a published figure with its source and date. The multiplications are mine; no source states a break-even.

Let R be the hourly cloud rate for the node you would rent (for example an 8-GPU node), u the share of the year's 8,760 hours you would actually bill, N the number of nodes, Capex the purchase and installation cost of the same node, L the years you depreciate it over, E its annual electricity (including the cooling overhead, PUE), K the annual colocation or room cost, and S the annual staff cost attributable to it.

Cloud annual cost R x 8,760 x u x N. On demand you pay only for billed hours, so this falls with u. A one- or three-year commitment is billed whether or not you use it, so for committed capacity treat u as 1.
Owned annual cost (Capex / L + E + K + S) x N. Almost all of it is incurred whether or not the GPUs are busy.
Break-even utilization u = (Capex / L + E + K + S) / (R x 8,760). If your forecast utilization sits clearly above u, owning wins on cost; clearly below, renting wins; near it, price is not the deciding factor.
Capex ceiling Capex(max) = L x (u x R x 8,760 - E - K - S). This turns the formula around: given the utilization you expect, it is the most you can pay for the node and still match the cloud.

What the formula leaves out, on purpose: the cost of money, resale value, network and storage gear, software licenses, downtime, and the cost of data movement. Add them as extra terms if they apply to you.

Cloud side: published H100 prices

All figures are as shown on each provider's own public page on November 8, 2026. They are not the same product: on-demand, a capacity reservation and a commitment are different contracts, and one provider's listed price is not yet in effect. Treat this as a map of what each page says, not a ranking.

CoreWeave, NVIDIA HGX H100, 8 GPUs$49.24 per hour on demand; spot $19.71 per hour; $6.16 per hour for a single-GPU inference listing. The page shows no region or date (source)
Lambda, NVIDIA HGX H100, 8x plan$4.59 per GPU per hour, a price the page says takes effect November 16, 2026, with current prices visible after login. Its 1-Click Clusters list H100 at $6.16, $5.85 and $5.54 per GPU per hour for 16, 64 and 256 GPUs on terms of 2 weeks to 1 year (source)
AWS, p5.48xlarge, 8 x H100, US East (Ohio and N. Virginia)$47.757 per hour ($5.970 per accelerator) as an EC2 Capacity Blocks effective rate. This is a scheduled reservation, not the on-demand rate; the AWS on-demand page read gave no P5 price in its served text. AWS says Capacity Block prices are scheduled to be updated next in January 2027 (source)
Google Cloud, a3-highgpu-8g, 8 x H100$88.49 per hour on demand; $61.38 per hour with a 1-year commitment; $38.86 per hour with a 3-year commitment. The page lists Iowa (us-central1) first and does not label the region of the table (source)
Microsoft Azure, Standard_ND96isr_H100_v5, East US$98.32 per hour retail consumption price, effective start 2023-12-01, from Azure's public Retail Prices API (source)

Multiplied by 8,760 hours (my arithmetic), one 8-GPU node billed all year comes to $431,342.40 at CoreWeave, $321,667.20 at Lambda's listed 8x rate (8 x $4.59 = $36.72 per hour), $418,351.32 on an AWS Capacity Block rate, $775,172.40 for Google on demand, $340,413.60 for Google's 3-year commitment and $861,283.20 at the Azure East US price. At half the hours on the CoreWeave on-demand rate the figure is $215,671.20; at a quarter it is $107,835.60. Each 10 percentage points of utilization on that node is $43,134.24.

Read those as ceilings for your owned cost: if your quoted annual cost of owning the same node (Capex / L + E + K + S) is below the cloud figure at your expected utilization, ownership is cheaper on paper. Providers' discounts for large, long contracts are negotiated and are not on these pages; ask for them (see the GPU cloud provider comparison for who publishes what).

Owned side: what can be sourced and what cannot

GPU power draw (sourced). NVIDIA's H100 page lists a maximum thermal design power of up to 700W (configurable) for the SXM form factor and 350-400W (configurable) for the NVL card (NVIDIA H100 page, read November 8, 2026). Eight SXM GPUs at 700W is 5,600 W, or 5.6 kW, before CPUs, memory, network and fans, which this page does not size.
Electricity price (sourced). The EIA's Electric Power Monthly, Table 5.6.A, gives the U.S. average commercial price as 14.47 cents per kilowatt-hour for August 2026 (13.87 in August 2025), released October 23, 2026; the 2026 values are preliminary (EIA). Your tariff, demand charges and region will differ.
Electricity for one node (my arithmetic). At full TDP all year, 5.6 kW x 8,760 hours is 49,056 kWh, or $7,098.40 at 14.47 cents. At a PUE of 1.09 it would be $7,737.26. That PUE is Google's own 2025 fleet-wide figure for its large-scale data centers (Google, read November 8, 2026), a hyperscale figure and not one a typical room or colo should be assumed to reach. Use your colo's stated PUE or its power bill instead.
Server and network price (variable). Not filled. No vendor public price list for an H100 server was read, and quoted prices vary by configuration, volume and time. Get written quotes for the exact node, the fabric, storage and installation.
Colocation or room cost (variable). Not filled. No operator's public rate card for GPU-density cabinets was read. Ask for price per kW and for the maximum kW per rack, and check the node's draw against your own rack and room limits.
Staff (variable). Not filled. The Bureau of Labor Statistics page returned a 403 error and was not used. Estimate the fraction of an operations engineer's loaded cost, plus on-call coverage, plus the cost of a failed node waiting for repair.
Depreciation life L (variable). Not filled. Pick the number of years your finance team will accept and test the answer at a shorter life too, because newer GPU generations change what an older node is worth.

The sourced terms show where the money is. At these published rates, GPU electricity for a node is a small figure next to a year of cloud billing, so the outcome hinges on Capex / L, the colocation cost, the staff cost and utilization, which are exactly the terms you must quote yourself.

Eight factors that decide it

Utilization. The break-even variable. Measure what your GPUs are really doing before you extrapolate; "GPU utilization" from a monitoring tool and useful work are not the same thing, as the GPU utilization guide explains. If your honest forecast is a few steady hours a day, the formula favors renting; if it is near-continuous, ownership gets a chance.
Workload type. Steady inference serving and always-on fine-tuning queues fill hours; research bursts, hyperparameter sweeps and periodic retraining do not. Commitments and spot capacity cover the middle: Google's page shows a 3-year commitment at $38.86 per hour against $88.49 on demand for the same node, and CoreWeave lists spot at $19.71 against $49.24, though spot capacity can be interrupted. Cost levers on the serving side are covered in reducing GPU costs for LLM serving.
Data gravity and egress. Training reads a lot of data, so where it lives matters. Google lists no charge for inbound data transfer and bills outbound transfer per GiB (Google VPC pricing, read November 8, 2026); the per-GiB tables were not in the served page, so price your own egress from the provider's current sheet. If the dataset is already on site, or results must flow back out to on-prem systems, add egress and migration time to the cloud column.
Compliance and data residency. If a contract, regulator or customer requires data to stay in a place, a facility you control, or a named region, that can decide the question before cost does. The price pages show the same H100 node offered in only some regions: AWS lists P5 Capacity Blocks in a set that includes Ohio, N. Virginia, Oregon, Tokyo, Mumbai, Sydney, London and Sao Paulo. Confirm the requirement in writing with your counsel; this page cites no regulation.
Time to capacity. Lambda says self-serve instances launch in minutes on a first-come basis; AWS Capacity Blocks reserve capacity for a scheduled window, with a reservation fee charged up front; owned hardware needs procurement, delivery, racking and burn-in whose lead times you must get from your vendor. If a product deadline is closer than your supplier's delivery date, rent first.
Staffing. Owning means someone patches firmware and drivers, replaces failed parts, runs the network fabric and carries the pager. Renting from a provider shifts much of that to them but not your own cluster scheduling, images and cost control. If you have no one to own it, the staff term S is larger than it looks.
Power and cooling. The 700W figure per GPU and the PUE variable decide whether your room or colo can host the node at all, and whether it needs more than air cooling. NVIDIA's DGX page describes deployment on premises, colocated, rented from managed service providers and in private clouds (NVIDIA DGX); the facility limit is a gating question before the price question.
Depreciation and obsolescence. Cloud rates let you move to a newer GPU generation at the next contract; owned hardware locks you in for L years. AWS says it will update Capacity Block prices again in January 2027, which shows cloud prices move too. Test the formula with a short life and a long one.

The hybrid option

Hybrid is not a compromise by default; it is what the formula suggests when your demand has a steady base and a bursty top. Own (or commit to) the base, rent the peaks. The base load is the capacity you would keep busy nearly every hour; every hour above it is billed on demand or on a short reservation. This is my reasoning from the formula, not a published rule.

Own the base. Size owned capacity to the level of demand that is present most of the year, so its utilization is high and Capex / L is spread over many busy hours.
Rent the burst. Use on-demand, spot or a short reservation for experiments, retraining runs and launches. Lambda lists clusters on terms of 2 weeks to 1 year, and AWS sells scheduled Capacity Blocks, so burst capacity can be booked for a defined window.
Commit in the cloud before you buy. A 1-year or 3-year cloud commitment (Google lists both for H100) lowers the rate without capex and teaches you your real utilization. It is billed whether or not it is used, so commit only to the steady base.
Colocate instead of building. NVIDIA lists colocation as one of the ways to run its DGX systems, which gives you owned hardware without owning the building, power plant and cooling.
Keep the data path in mind. Burst capacity in the cloud is only cheap if the data can reach it. Decide where the dataset is mastered and what each copy costs to move.

A step-by-step decision checklist

Write down the workloads. For each: training or inference, steady or bursty, how many GPUs, for how many hours a week, and whether it must run in a specific place.
Measure or forecast utilization honestly. Use measured busy hours from your current systems where you have them. A forecast built from a pilot is better than a hope.
Apply the gates first. Residency, compliance, contract and facility limits can remove an option before any price is calculated. Remove those options.
Collect cloud quotes for the same GPU. On-demand, reserved or committed, in your regions, from at least two providers, with the effective date of each price. Ask for any negotiated discount in writing.
Collect owned quotes. The server, fabric, storage and installation; a colo or room price per kW with its PUE; staffing; and the depreciation life your finance team will use.
Run the formula three times. Low, base and high capex, and expected utilization at three levels. Compute u for each. A decision that flips with a small change in a variable is not a price decision.
Add the data and egress term. Price the movement of data into and out of the cloud, and of results back on site, from the provider's current sheet.
Check time to capacity. Compare your deadline with the vendor's delivery date and with the cloud provider's availability in your region.
Choose. If forecast utilization sits clearly below u, rent. If clearly above and steady for the whole depreciation life, own or colocate. If demand has a steady base and a bursty top, run hybrid.
Set a review date. Prices, GPU generations and your own utilization all move. Review at the end of each commitment term and whenever a provider updates its price sheet.

Questions people ask

At what utilization does owning GPUs beat the cloud?

It depends on your quotes, and no primary source gives a single figure. The break-even utilization is u = (Capex / L + E + K + S) / (R x 8,760), where R is the cloud rate per hour, L the depreciation life and E, K and S the annual electricity, colocation and staff costs. Fill it with your own quotes.

What do cloud providers publish for H100 prices?

As read on November 8, 2026: CoreWeave $49.24 per hour for an 8-GPU HGX H100 node on demand; AWS $47.757 per hour for p5.48xlarge as an EC2 Capacity Blocks rate; Google Cloud $88.49 per hour for a3-highgpu-8g; Azure $98.32 per hour for ND96isr H100 v5 in East US; and Lambda $4.59 per GPU per hour on its 8x plan, effective November 16, 2026. They are different contract types, so compare them carefully.

How much power does an H100 draw?

NVIDIA lists a maximum thermal design power of up to 700W (configurable) for the H100 SXM and 350-400W (configurable) for the H100 NVL, on its H100 page read November 8, 2026. That is the GPU only, not the whole server.

What electricity price should I use?

Use your own tariff. For reference, the EIA reports a U.S. average commercial price of 14.47 cents per kilowatt-hour for August 2026 (Electric Power Monthly, Table 5.6.A, preliminary).

Is a hybrid setup worth it?

It is worth testing when demand has a steady base and a bursty top: own or commit to the base so it stays busy, and rent the peaks. That is a conclusion from the formula, not a published rule.

Why does this page not give a server price?

No vendor public price list for an H100 server was read, and quoted prices vary by configuration, volume and time, so capex is left as a variable for you to fill from written quotes.

Sources

AWS: Amazon EC2 Capacity Blocks for ML pricing (page states next price update January 2027; read November 8, 2026) · p5.48xlarge $47.757 per hour effective rate, regions
CoreWeave: GPU cloud pricing (no date shown; read November 8, 2026) · HGX H100 8 GPUs $49.24 on demand, $19.71 spot
Lambda: GPU cloud pricing (prices effective November 16, 2026; read November 8, 2026) · HGX H100 $4.59 per GPU per hour, 1-Click Clusters terms
Google Cloud: accelerator-optimized machine pricing (no date shown; read November 8, 2026) · a3-highgpu-8g $88.49 per hour; 1-year and 3-year commitment rates
Google Cloud: VPC network pricing (read November 8, 2026) · No charge for inbound data transfer
Microsoft Azure: Retail Prices API, Standard_ND96isr_H100_v5 (effective 2023-12-01; read November 8, 2026) · East US $98.32 per hour
NVIDIA: H100 Tensor Core GPU page (no date shown; read November 8, 2026) · Maximum TDP up to 700W for SXM
NVIDIA: DGX platform page (no date shown; read November 8, 2026) · Deployment on premises, colocated, managed service providers, private clouds
U.S. Energy Information Administration: Electric Power Monthly, Table 5.6.A (data for August 2026, released October 23, 2026) · U.S. average commercial electricity price 14.47 cents per kWh
Google data centers: efficiency (2025 fleet-wide PUE; read November 8, 2026) · PUE of 1.09 for large-scale data centers
Hero image: NVIDIA H100 product image from NVIDIA's own H100 page (official vendor image; read November 8, 2026) · H100 SXM module render

Everything was read from the linked pages on November 8, 2026, and each source carries its date or notes that none is shown. Gaps: no server or network purchase price, colocation rate, staffing cost, enterprise PUE or depreciation life was sourced, so those stay variables; the AWS on-demand page gave no P5 price in its served text, so the AWS figure is an EC2 Capacity Blocks rate; Lambda's listed prices take effect November 16, 2026; Google's region for the H100 table is not labeled; the Uptime Institute page redirected away, the Bureau of Labor Statistics page returned a 403, and neither is cited; Google's per-GiB egress tables were not in the served page. All multiplications are mine and shown as such. Prices change; confirm on each provider's page before you commit. This is general information, not financial or legal advice.