How Do AI Inference Costs and AI Product Gross Margins Work in 2027?

By Kory White · Updated Oct 2026 · 7 min read

Direct answer

Every time an AI feature answers a request, the vendor pays for the tokens it reads and writes. Those fees are billed per million tokens, they scale with usage, and they land in cost of revenue. Classic software costs little to serve one more user, so margins stay high. AI products pay a real variable cost per request, so a heavy user can erase the margin on a flat-price plan.

Public companies say the same thing. Snowflake's CFO told investors that AI workloads "carry a lower contribution margin today" and guided full-year product gross margin to 74%. Duolingo, by contrast, credited "continued reductions in per-unit third-party AI costs" for gross margin of 72.6%. The direction depends on which way unit costs and usage move.

The levers are well documented: send easy work to smaller models, cache repeated prompt prefixes, use batch pricing for work that can wait, keep prompts and outputs short, and price the product so heavy usage pays for itself. The sections below give the list prices, the cost lines, the filings, and an illustrative calculation, with a link for each fact.

How inference is priced

The major model providers publish list prices in dollars per million tokens, with output tokens priced higher than input tokens and discounted rates for cached input and batch jobs. These are the official pages, read on October 18, 2026:

Anthropic. On the Claude pricing page, Claude Sonnet 5.5 is $2 per million input tokens and $10 output; Claude Haiku 5.5 is $0.10 input and $0.50 output for prompts up to 100,000 tokens; Claude Opus 5.5 is $4 input and $20 output; Claude Fable 5.1 is $10 input and $50 output.
OpenAI. On the OpenAI API pricing page, gpt-6-sol is $2.00 input and $10.00 output, gpt-6-luna is $0.10 input and $0.50 output, and gpt-6-astra is $10.00 input and $50.00 output.
Google. On the Gemini API pricing page, Gemini 3.1 Pro Preview is $2.00 input and $12.00 output for prompts up to 200k tokens, and Gemini 2.5 Flash is $0.30 input and $2.50 output.

The spread is the point. The same page lists models many times apart in price per token, so the model you route a request to matters as much as how many requests you serve.

Prices change, sometimes in both directions. Anthropic's page says a scheduled increase for Claude Sonnet 5 to $3/$15 on September 1, 2026 will not occur. Google's page lists Gemini 3.8 Flash at $0.75 input and $3.75 output through December 31, 2026, then double those rates from January 1, 2027. Check the current pages before you budget.

What goes into cost of revenue for an AI product

Inference is the new line, but it is not the only one. A fair gross-margin model has these parts:

Model API fees. Input, output, cache write and cache read tokens, as priced above. This is the part that grows with every request.
Hosting and compute. Duolingo's 10-Q says its cost of revenues includes hosting fees and third-party AI costs, alongside payment processing fees. A company that runs open models itself pays for the hardware instead of a per-token fee.
Retrieval and storage. Context caching can carry a storage charge. Google's pricing page lists $4.50 per million tokens per hour of storage for cached context on Gemini 3.1 Pro Preview, on top of the cached-read rate.
Support and other people costs. Support, success and monitoring staff may or may not be counted in cost of revenue. Companies define this differently, so read each filing's own definition before comparing margins.

What public companies have reported

Only statements from filings and earnings calls are used here. Neither company gives an AI-only margin.

The firm
“For FY 2027, we now expect 74% non-GAAP product gross margin. This revised outlook includes a higher revenue mix from fast growing AI workloads, which carry a lower contribution margin today.”
— Snowflake CFO Brian Robins, Q2 fiscal 2027 call (corrected transcript), September 2, 2026

Source: Snowflake's corrected transcript. On the same call the CEO said cost "has absolutely become a concern" as models have become more powerful. For how Snowflake bills and routes these models, see our Snowflake example.

The firm
“Total gross margin increased to 72.6% from 72.4% during the three months ended June 30, 2026 and 2025.” … “reflecting continued reductions in per-unit third-party AI costs.”
— Duolingo Form 10-Q, quarter ended June 30, 2026

Source: Duolingo's 10-Q. The Q2 shareholder letter guides to "approximately 71.0% in Q3 and approximately 71.6% for the full year, better than the trajectory we outlined on our Q1 call based on AI cost trends."

Read together: AI cost is a real swing factor for margin in both directions. Snowflake attributes pressure to mix shifting toward AI workloads. Duolingo reports margin gains as the per-unit cost of AI falls, while it expands AI features at a "measured pace." Neither statement says AI products are unprofitable; both say margin now depends on unit cost and usage mix.

Levers that improve AI gross margin

Smaller models and routing. Anthropic's model-selection guide suggests starting with a cheaper model such as Claude Haiku 5.5 and upgrading only if necessary, and describes pairing a lower-cost model with a frontier model so most tokens are billed at the lower rate. Snowflake's CEO described an "auto" mode that pairs each task with the most appropriate model.
Prompt caching. On the Claude pricing page, a cache hit costs 10% of the standard input price on most models, and Sonnet 5.5 cache hits are $0.10 per million tokens against $2 for base input. OpenAI lists a cached input rate of $0.20 for gpt-6-sol against $2.00 on its pricing page.
Batching. Anthropic's Batch API gives a 50% discount on input and output tokens, and Google lists "Batch API (50% cost reduction)." OpenAI's batch input for gpt-6-sol is $1.00 against $2.00 standard. Batch suits work nobody is waiting on, such as nightly summaries.
Prompt and output length. Output tokens cost several times input tokens on every page above, and Claude Haiku 5.5 is priced by prompt length, with prompts over 100,000 tokens paying higher rates. Anthropic also notes that its newer tokenizer produces approximately 30% more tokens for the same text, so re-measure after a model upgrade.
Speed tiers cost more. OpenAI renamed priority processing to Fast mode on July 30, 2026 and lists higher rates for it, for example $4.00 input on gpt-6-sol. Pay for speed only where users feel it.
Self-hosting open models. Snowflake's CEO said "we have pretty different economics when it comes to open source models, since we run the inference ourselves." That swaps a per-token fee for hardware and operations cost, which only pays off at steady, high volume. This page does not price fine-tuning or self-hosting; see each provider's documentation.

Pricing models that protect margin

Flat per-seat pricing puts the usage risk on the vendor. Three structures move it back:

Usage-based pricing charges in proportion to the tokens or tasks consumed, so cost and revenue move together.
Credits bundle usage into a unit the vendor can reprice. Snowflake bills AI in credits per million tokens, and its pricing documentation lists an AI Credit at $2.00 with global routing and $2.20 with regional routing.
Seat plus usage keeps a predictable base fee and meters heavy use above an included allowance. The filings cited here do not quantify this model, so treat it as a design choice to test, not a proven margin figure.

Whichever you choose, the list prices above change, so build the ability to reprice or reroute into the contract and the product.

A worked example: one subscription, four setups

Illustrative arithmetic, not a company's reported numbers. It uses the Claude list prices cited above: Sonnet 5.5 at $2 input and $10 output, Haiku 5.5 at $0.10 and $0.50, and Sonnet 5.5 cache hits at $0.10. It counts inference only and ignores hosting, support and payment fees.

Assume a $20 monthly subscription, and that each request has 3,000 input tokens and 500 output tokens.

Sonnet 5.5 for everything. Each request costs $0.006 for input plus $0.005 for output, or $0.011. A light user at 200 requests costs $2.20 a month, an inference-only margin of 89%. A heavy user at 2,000 requests costs $22.00, which is -10%: the plan loses money.
Haiku 5.5 for everything. Each request costs $0.00055, so the heavy user costs $1.10, a 94.5% inference-only margin. The cost is that a smaller model may not do every task well enough.
Sonnet 5.5 with caching. If 2,000 of the 3,000 input tokens are a repeated prefix read from cache, input falls to $0.0022 and each request costs $0.0072. The heavy user costs $14.40, a 28% margin.
Routing 80% to Haiku, 20% to Sonnet. The heavy user's 1,600 simple requests cost $0.88 and 400 hard ones cost $4.40, for $5.28 total and a 73.6% inference-only margin.

The lesson is the shape, not the figures: with flat pricing, margin is set by your heaviest users and your model choice, and each lever above moves it.

Questions people ask

Why are AI product gross margins lower than classic SaaS?

Because each request carries a variable cost. Providers bill per million tokens, and that fee lands in cost of revenue. Snowflake's CFO said AI workloads "carry a lower contribution margin today."

Are inference prices going up or down?

Both happen. Anthropic's pricing page says a planned increase for Claude Sonnet 5 to $3/$15 will not occur, while Google's page lists Gemini 3.8 Flash at double its current rates from January 1, 2027. Snowflake's product executive said frontier models have been "revising prices down on a regular basis."

What is the fastest lever to improve AI margin?

Usually routing easy requests to a smaller model and caching repeated prompt prefixes. In the illustrative example above, those moves take the heavy-user margin from -10% to 73.6% and 28% respectively.

Is batch pricing worth using?

For work nobody waits on, yes. Anthropic documents a 50% batch discount and Google lists a 50% cost reduction for its Batch API. Real-time features cannot use it.

Do these numbers apply to my product?

No. The worked example is illustrative arithmetic on list prices, with assumed request sizes. Measure your own tokens per request and requests per user, then recompute.

Sources

Anthropic: Claude pricing · per-token prices, cache multipliers, Batch API discount, price footnotes
Anthropic: Choosing the right model · cost-aware model selection and combining models
OpenAI: API pricing · input, cached input and output prices; batch and Fast mode
Google: Gemini API pricing · input, output, context caching, batch and the January 2027 price change
Duolingo Form 10-Q, quarter ended June 30, 2026 · cost of revenues definition and gross margin discussion
Duolingo Q2 2026 shareholder letter · gross margin and AI cost guidance
Snowflake Q2 fiscal 2027 earnings call transcript · product gross margin outlook and AI workload mix
Snowflake docs: Cortex AI pricing · AI Credit pricing as an example of credit-based billing

Prices, models and company guidance change often. Figures here were read from the linked pages on October 18, 2026; confirm against the current pages before you commit budget. The worked example is arithmetic on list prices, not a forecast.