What are the LLM fine-tuning compute requirements in 2027?

By Kory White · Updated Nov 2026 · 6 min read

Silicon wafers on display, a dark disc patterned with bands of blue, green, red and orange light

Direct answer

There is no single number: compute for fine-tuning depends on the method, the precision and the model size. The sources agree on the shape of it. Full fine-tuning needs memory for the weights, the gradients and the optimizer states, while LoRA and QLoRA train a small adapter and shrink that bill. The QLoRA paper reports fine-tuning a 65B parameter model on a single 48GB GPU, and Hugging Face documents a 70B Llama run on 2X24GB GPUs.

Three things set the hardware: memory per GPU (the usual limit), how many GPUs you can shard across, and how long you are willing to wait. This page reprints only figures the linked sources state, each with its stated conditions. Nothing here is a 2027 forecast; the sources are current docs and papers, and we label our own reading as judgment.

What uses the memory when you fine-tune

Hugging Face's GPU memory guide lists what sits on the GPU during training. In mixed precision, model weights take 6 bytes per parameter (an fp16 copy and an fp32 copy). Adam optimizer states add 8 bytes per parameter, which a quantized Adam from bitsandbytes can compress to 2 bytes per parameter. Gradients take 4 bytes per parameter.

Activations are the part that moves with your settings. The same guide says they vary with batch size, sequence length, model depth and hidden size, which is why batch size or sequence length can exhaust GPU memory even if the model itself fits. Its example: training a 4B parameter model in mixed precision on a batch size of 16 requires roughly 85GB of GPU memory.

Judgment: the per-parameter figures above explain why full fine-tuning is expensive, but we do not add them up into a rule of thumb. The guide gives no single formula for total memory, and activations depend on your batch and sequence length.

LoRA and QLoRA: why they need far less

The LoRA paper reports that, compared with GPT-3 175B fine-tuned with Adam, LoRA can reduce the number of trainable parameters by 10,000 times and the GPU memory requirement by 3 times, with no additional inference latency. Hugging Face's PEFT docs describe the same idea: fine-tune only a small number of extra parameters, which significantly decreases computational and storage costs while yielding performance comparable to a fully fine-tuned model.

The QLoRA paper goes further. It backpropagates gradients through a frozen, 4-bit quantized pretrained model into low rank adapters, and reports fine-tuning a 65B parameter model on a single 48GB GPU while preserving full 16-bit finetuning task performance. The abstract also says the Guanaco model family needed only 24 hours of finetuning on a single GPU.

LoRA vs Adam on GPT-3 175B10,000 times fewer trainable parameters, 3 times less GPU memory (LoRA paper)
QLoRA, 65B modelA single 48GB GPU, with 16-bit finetuning task performance preserved (QLoRA paper)
QLoRA, 33B modelA single 24GB GPU (Hugging Face bitsandbytes post)

The bitsandbytes post adds that QLoRA enables 33B model finetuning on a single 24GB GPU.

Documented hardware, from one GPU to eight

Hugging Face's PEFT FSDP guide is the clearest side-by-side. It fine-tunes a 70B Llama model with LoRA and FSDP on 8xH100 80GB GPUs on a single machine, with a 2048 maximum sequence length, per-device batch size 8 and gradient checkpointing on. Memory consumed per GPU is 72-80 GB. The same guide runs QLoRA with FSDP on 2X24GB GPUs (per-device batch size 2): 19.6 GB per GPU with CPU offloading, plus around 107 GB of CPU RAM, or 35.6 GB per GPU with offloading disabled.

The guide sums it up: what took 16X80GB GPUs for full finetuning, 8X80GB GPUs with FSDP+LoRA, and a couple of 80GB GPUs with DDP+QLoRA now requires 2X24GB GPUs.

Full fine-tuning, Llama 70B. 16X80GB GPUs, per the PEFT FSDP guide.
LoRA with FSDP, Llama 70B. 8X80GB GPUs; the example run used 72-80 GB per GPU.
QLoRA with FSDP, Llama 70B. 2X24GB GPUs, with CPU offloading and about 107 GB of CPU RAM.
QLoRA on one small GPU. Per the bitsandbytes post, Llama 7B (14GB in fp16) on a single 16GB T4 with 4-bit NF4 and gradient checkpointing hit no out-of-memory error at sequence lengths 512 and 1024, at batch size 1.

The same post reports that Llama 13B at sequence length 1024 fit on that 16GB T4 only with nested quantization turned on.

These are single documented runs with their own settings (sequence length, batch size, offloading). Change any of them and the memory changes, so use them as reference points, not guarantees.

The GPUs behind those numbers

Memory per GPU is the spec that decides which row above you can reach. NVIDIA's pages list the H100 at 80 GB for the SXM version, 3.35 TB/s of memory bandwidth, and 94 GB for the NVL version at 3.9 TB/s. The H200 is described as the first GPU to offer 141 gigabytes of HBM3e memory at 4.8 terabytes per second. The A100 is listed as 80GB HBM2e, with 2,039 GB/s of bandwidth on the SXM version and 1,935 GB/s on PCIe.

Answer.AI's 70b at home post shows the other end. It targets a desktop with two or more standard gaming GPUs (RTX 3090 or 4090), and says 70b weights alone need roughly 140GB at 16-bit but about 35GB at 4-bit. It also quotes around $0.60 per hour to rent a dual 3090 system on Runpod's Community Cloud, and around $40,000 for a single H100. Those are prices from the March 6 2024 post, so check current rates, and the authors call the release alpha/preview.

NVIDIA's H200 page makes no fine-tuning performance claims, so we cite it for memory only. We did not find a source that gives fine-tuning speed for these GPUs under the same conditions, so we do not rank them for fine-tuning.

Levers that cut memory in practice

Hugging Face's TRL SFT Trainer docs show the knobs in one place. SFTTrainer accepts a peft_config, and a quantization_config that you combine with it for QLoRA training. Packing groups multiple sequences into fixed-length blocks to reduce padding. The padding_free option reduces memory usage by eliminating padding overhead, and needs FlashAttention 2 or 3. Activation offloading moves activations to the CPU, and gradient checkpointing defaults to on in SFTConfig, with a default max_length of 1024.

Train an adapter, not the model. LoRA or QLoRA, per the papers above.
Quantize the base weights. 4-bit, as in QLoRA.
Shrink the optimizer. Quantized Adam is documented at 2 bytes per parameter.
Trim activations. Gradient checkpointing, shorter sequences, smaller batches, packing.
Shard across GPUs. FSDP, as in the Hugging Face example runs.

How to size your own job

Start from the documented run closest to yours: same method, similar model size, similar sequence length. Then measure. Our judgment is that a short trial run on the GPU you intend to use, watching peak memory, tells you more than any paper figure, because the sources above each fix their own batch size and sequence length.

Questions people ask

How much GPU memory does it take to fine-tune a 65B model?

The QLoRA paper reports fine-tuning a 65B parameter model on a single 48GB GPU while preserving full 16-bit finetuning task performance.

Can I fine-tune a 70B model on consumer GPUs?

Hugging Face documents QLoRA with FSDP on a 70B Llama model on 2X24GB GPUs, using 19.6 GB per GPU with CPU offloading, per its PEFT FSDP guide. The settings are listed on that page.

How much less does LoRA need than full fine-tuning?

The LoRA paper reports 10,000 times fewer trainable parameters and 3 times less GPU memory than GPT-3 175B fine-tuned with Adam.

What takes the memory in full fine-tuning?

Weights, gradients, optimizer states and activations. Hugging Face's guide gives 6 bytes per parameter for mixed precision weights, 8 for Adam states and 4 for gradients, and notes activations vary with batch size and sequence length.

Which GPU has the most memory?

Of the NVIDIA pages we read, the H200 lists 141 gigabytes of HBM3e, against 80 GB for the H100 SXM and 94 GB for the H100 NVL.

Sources

QLoRA paper (arXiv 2305.14314) · 65B on a single 48GB GPU, 4-bit base, 24 hours
LoRA paper (arXiv 2106.09685) · Trainable parameters and GPU memory reduction
Hugging Face PEFT docs · Computational and storage cost claim
Hugging Face docs: GPU memory usage · Bytes per parameter, activations, 4B example
Hugging Face PEFT docs: FSDP guide · 70B Llama LoRA and QLoRA runs, per-GPU memory
Answer.AI: You can now train a 70b language model at home · Consumer GPUs, weight sizes, rental price
Hugging Face blog: bitsandbytes, 4-bit quantization and QLoRA · 33B on 24GB, T4 benchmark table
NVIDIA H100 · Memory and bandwidth
NVIDIA H200 · Memory and bandwidth
NVIDIA A100 · Memory and bandwidth
Hugging Face TRL: SFT Trainer · PEFT, quantization, packing, padding-free, defaults
Hero photo: "Colorful patterns in the wafers" by Don DeBold, CC BY 2.0, Wikimedia Commons · Credit for the photo at the top of this page (cropped)

Facts were read from the linked pages on November 7, 2026. Documentation and vendor pages change, and each run reported here has its own settings. Sections marked as judgment are our reading, not a source claim.

Top photo: Don DeBold from San Jose, CA, USA / CC BY 2.0 via Wikimedia Commons