answrsnow
AI Infrastructure · Explainer

How Much Does AI Infrastructure Cost in 2027?

By Kory White · Updated Nov 2026 · 5 min read
Front of an NVIDIA DGX Spark desktop AI computer with a metal-foam face and the NVIDIA logo, on a wooden table
Direct answer

There is no single price for AI infrastructure, because you pay in different units depending on how you get compute. Published prices read on 2026-11-04 show four routes: API tokens, rented GPU hours, reserved capacity and owned hardware.

On the API route, Anthropic lists Claude Sonnet 5.5 at $2 per million input tokens and $10 per million output tokens, and OpenAI lists gpt-6.1-sol at $2.00 input and $10.00 output per 1M tokens. On the rented-GPU route, Runpod lists an H100 SXM at $3.99/hr on its Secure Cloud and Lambda lists an HGX H100 at $3.99 per GPU per hour. Owned hardware is the one route where the NVIDIA pages read here publish no price at all.

These are list prices on one day and they change, so treat every figure below as a snapshot and check the provider before you commit.

Route 1: Pay per token through an API

If you call a hosted model, you buy tokens, not hardware. Both Anthropic and OpenAI publish rates per million tokens, with separate prices for input, output and cached input.

Two things change the bill without changing the model. Anthropic says its Batch API gives "a 50 percent discount on both input and output tokens" for asynchronous work. OpenAI's page does not state a Batch percentage; its Batch tab lists separate per-model rates, for example $5.00 input and $25.00 output for gpt-6-astra. Cached input is also priced separately on both pages.

The pages read list rates only. They do not say how many tokens a given product will use, so this page does not estimate a monthly bill.

Route 2: Rent GPUs by the hour

If you run your own models, you rent accelerators. Prices below are as read on 2026-11-04; the providers quote them in different units, so read the unit before comparing.

Note the unit differences: Runpod and Lambda quote per GPU, while CoreWeave quotes per instance of 8 GPUs (4 for the GB200 NVL72). Runpod also lists a Community Cloud starting rate next to its Secure Cloud rate, and it lists a separate Serverless rate for usage-based inference, such as H100 at $5.39/hr. Lambda's page lists several instance tables with slightly different prices, for example H100 SXM at $4.09, $4.19 and $4.29.

The AWS EC2 P5 page describes instances with up to 8 H100 GPUs but publishes no dollar rates, so no AWS on-demand price is quoted here.

Route 3: Reserve capacity for a fixed term

Providers also sell commitments. The terms differ, and in at least one case the committed price is higher than the self-serve one, so read the unit and the duration.

Route 4: Buy and run your own hardware

Owning is the route with the least published pricing. NVIDIA's DGX Spark page states no price and sends buyers to the NVIDIA Marketplace. It does list up to 128 GB of unified system memory, up to 1 petaFLOP of FP4 AI performance (a theoretical figure that uses sparsity) and a 240 Watt power supply.

At the other end, NVIDIA's DGX B200 is an 8-GPU system of 10 rack units with approximately 14.3 kW max system power. This page quotes no price for it, because none was read. Hardware is only part of the cost of owning: the IEA says data centres used around 415 TWh in 2024 and are set to more than double to around 945 TWh by 2030, which is why power and cooling belong in any ownership plan.

For the facility side, see how to budget a data center or colocation buildout. For the hardware itself, see what GPU infrastructure for AI workloads looks like.

What these numbers leave out

List prices are not the whole bill. Storage, networking, idle time and staff do not appear on the pricing pages read for this article, and none is priced here. Our guide to the hidden costs of running AI infrastructure covers them, how to reduce GPU costs when serving large language models covers the savings side, and GPU cloud providers for AI training compares the providers.

Prices on this page were read from each provider's own page on 2026-11-04. They change without notice, and some providers show different prices by region or product. Confirm current rates before you budget.

Questions people ask

How much does AI infrastructure cost?

It depends on the route. Published list prices read on 2026-11-04 include $2 per million input tokens for Claude Sonnet 5.5, $3.99 per GPU-hour for an H100 on Lambda, and $49.24 per hour for an 8-GPU HGX H100 instance on CoreWeave. NVIDIA's DGX Spark page states no price.

How much does it cost to rent an H100?

Runpod lists an H100 SXM at $3.99/hr on Secure Cloud and $2.69/hr on Community Cloud. Lambda lists an HGX H100 at $3.99 per GPU per hour. CoreWeave lists an 8-GPU HGX H100 instance at $49.24 per hour on-demand. AWS Capacity Blocks list $5.970 per accelerator for p5.48xlarge in US East (N. Virginia).

Are reserved GPUs cheaper than on-demand?

Not always. CoreWeave says committed usage can be discounted by up to 60% against on-demand. Lambda's 1-Click Clusters list H100 at $6.16 per GPU per hour for 16 GPUs, which is higher than its listed $3.99 instance price, so compare the unit and term first.

Is there a discount for batch API work?

Anthropic says its Batch API gives a 50 percent discount on both input and output tokens. OpenAI's pricing page lists separate Batch rates per model rather than one percentage.

How much does it cost to own AI hardware?

The NVIDIA DGX Spark page read for this article states no price and points to the NVIDIA Marketplace. This page does not give a purchase price for any NVIDIA system.

Sources

AI infrastructure is priced in tokens, GPU-hours, reservations or hardware, and each route has its own units. Use the providers' own pages for current rates. This page is general information, not advice for your specific business.

Top photo: Daniel Lu (User:dllu) / CC BY-SA 4.0 via Wikimedia Commons

Share this page
LinkedIn: share linkLinkedIn: share post