.ai

the best open source ai models

Guides

Rankings

FAQ

Upload

CUDA out of memory: how much VRAM each AI model needs and how to fit it

Q: Out of memory before the first step

The weights alone exceed VRAM. A smaller precision (fp8, GGUF Q5) or offload. Check file size versus VRAM minus 2 GB.

Q: Out of memory in the sampler at high resolution

Attention activations scale with pixel count. Native resolution, then upscale with an upscaler model or a second low-denoise pass.

Q: Fits once, fails the second time

Fragmentation and cached models from the previous run. Unload models (ComfyUI Manager has a button), or set PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True before start.

Q: Slow but no error after adding --lowvram

Weights stream from system RAM every step. Expected. Reduce the model (quantise) so more stays on the GPU, or run hosted for the big families.

Topic: Errors & fixes

By Captain

Published 2026-05-18

Overview

CUDA out of memory is the most common error in local AI image and video generation, and it is not a bug: the model, the text encoders, the latent image and the attention activations must all live in the graphics card's memory at once. How much they need depends on the family, the precision of the weights, the resolution, the batch size and, for video, the number of frames.

This guide gives the numbers people actually measure, not the theoretical ones. "Comfortable" means the default workflow at the family's native resolution runs without offloading; "minimum" means it runs with quantised weights and offload tricks. Both assume a single image or clip at a time.

If a model sits above your card, the hosted route is cheaper than a new GPU:

BitVector

runs these families on cloud GPUs in a web app and Discord, and

PirateDiffusion

runs them from Telegram with the trigger words shown in this site's Run box. Many people use a local card for

SD 1.5

and

SDXL

and the hosted services for

Flux

Qwen

and video.

Reference

Name	Type	What it is
SD 1.5	4 GB min / 6 GB comfortable	512x512 native. 2 GB file (fp16). Runs on almost anything; use --lowvram on 4 GB.
SDXL / Pony / Illustrious	6 GB min / 8-12 GB comfortable	1024x1024 native. 6.5 GB file (fp16). --medvram on 8 GB; hires fix and ControlNet want 12 GB.
Flux dev	8 GB min (GGUF Q5 / NF4) / 16 GB comfortable (fp8) / 24 GB full	1024x1024. 23 GB fp16 file; 11 GB fp8; 6.8 GB NF4. T5 encoder adds 5 GB fp16 or 2.5 GB fp8.
FLUX.2 dev	16 GB min (GGUF) / 24 GB comfortable (fp8) / 32 GB+ full	Up to 4 MP. Larger text encoder; fp8 is the practical choice on consumer cards.
Qwen-Image / Qwen-Image-Edit	12 GB min (GGUF Q4) / 24 GB comfortable (fp8)	20B parameters; 40 GB fp16 file, 20 GB fp8, 12 GB Q4. The lightning LoRAs cut steps to 4-8.
Z-Image Turbo	8 GB min / 16 GB comfortable	6B parameters, 8 steps, 1024 native. The friendliest modern model for 8-12 GB cards.
Wan 2.2 5B (TI2V)	8 GB min / 12 GB comfortable	720p, 5 seconds. The 5B hybrid is built for consumer cards.
Wan 2.2 14B (T2V / I2V)	12 GB min (GGUF Q4 + offload) / 24 GB comfortable (fp8)	480p-720p, 81 frames. Two expert models (high / low noise) load in turn. Lightning LoRAs make 4 steps usable.
HunyuanVideo	12 GB min (GGUF + offload) / 24 GB comfortable	13B, 544x960, 65-129 frames. Frame count is the lever.
LTX-2	12 GB min / 24 GB comfortable	Audio and video together; 4K needs 24 GB+. Fast per frame.

No install required - run AI models on the cloud

BitVector Prism is the easiest way to start: pick a model, type a prompt and generate in a clean web app, with nothing to configure. BitVector is also available on Discord and on the web (SpyGlass).

Try BitVector

Step by step

Find the family and size of the model on its page here (family at the top, file sizes under Files) and compare with the table above and your card.
Pick the precision the card can hold: fp16 only if the file plus 4 GB fits; otherwise fp8 (same quality for images), then GGUF Q8, Q6, Q5, Q4 in that order. ComfyUI-GGUF provides the Unet and CLIP loaders.
Quantise the text encoder too: the T5-XXL fp8 file is half the size of fp16 with no visible change for most prompts.
Keep resolution at the native size and upscale afterwards: 512 for SD 1.5, 1024 for SDXL, Flux, Qwen and
Z-Image
. Doubling the side quadruples the activation memory.
Batch size 1. Generate more images in sequence instead of in parallel.
Offload: --lowvram in
ComfyUI
, --medvram in A1111, the GPU Weights slider in
Forge
. These move weights between RAM and
VRAM
and need 32 GB of system RAM to be comfortable with Flux and
Wan
.
Decode in tiles: VAE Decode (Tiled) in ComfyUI, Tiled VAE in A1111/Forge. The decode step alone can fail on a sampled latent that fit.
Video: fewer frames first (33 or 49), then shorter sides (480p), then more steps. Lightning / distillation LoRAs listed on the Wan and
Hunyuan
family pages cut steps from 30 to 4 to 8.
Still short: run it hosted. The Run box on each model page opens the model on PirateDiffusion (Telegram) or BitVector (web, Discord) with nothing to install.

Examples

ComfyUI Flux dev on 8 GB

Unet Loader (GGUF): flux1-dev-Q5_K_M.gguf

DualCLIPLoader (GGUF): t5-v1_1-xxl-encoder-Q5_K_M.gguf + clip_l.safetensors

VAE: ae.safetensors | 1024x1024 | 20 steps | --lowvram

Wan 2.2 14B I2V on 12 GB

High noise: wan2.2_i2v_high_noise_14B_Q4_K_M.gguf | Low noise: wan2.2_i2v_low_noise_14B_Q4_K_M.gguf

umt5_xxl_fp8_e4m3fn_scaled.safetensors | 480x832 | 49 frames | lightning LoRA, 4 steps

Forge SDXL on 6 GB

UI: xl | GPU Weights: 4000 | Diffusion in Low Bits: Automatic | 1024x1024 | batch 1

Tips

System RAM matters almost as much as VRAM once you offload: 32 GB is the comfortable minimum for Flux and Wan, 64 GB for
FLUX.2
and Qwen fp16.
Close the browser's hardware acceleration tabs and anything using the GPU (game launchers, video calls): 1 to 2 GB of VRAM disappears into them.
Windows reserves VRAM for the desktop; a second cheap card for the display frees the whole main card for generation.
GGUF Q8 is visually identical to fp16 for images; Q5_K_M is the sweet spot for 8 GB cards; below Q4 details degrade.
fp8 on RTX 40-series and newer is fast; on RTX 30 and older it works but weights are upcast on the fly, so GGUF can be quicker there.
Keep a 512x512 or 1024x1024 test prompt handy and raise one variable at a time (resolution, then LoRAs, then ControlNet) to find the ceiling of your card.
The heat ranking on this site shows which quantised and distilled releases the community actually uses; the lightning LoRAs for Wan and Qwen are usually near the top.

Troubleshooting

Out of memory before the first step

Why it happens

The weights alone exceed VRAM.

How to fix it

A smaller precision (fp8, GGUF Q5) or offload. Check file size versus VRAM minus 2 GB.

Out of memory in the sampler at high resolution

Why it happens

Attention activations scale with pixel count.

How to fix it

Native resolution, then upscale with an

upscaler

model or a second low-denoise pass.

Out of memory only at VAE decode

Why it happens

Decoding is the memory peak for large images and for video.

How to fix it

Tiled VAE decode; fp16/bf16 VAE flag; for video decode in temporal chunks.

Fits once, fails the second time

Why it happens

Fragmentation and cached models from the previous run.

How to fix it

Unload models (ComfyUI Manager has a button), or set PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True before start.

Slow but no error after adding --lowvram

Why it happens

Weights stream from system RAM every step.

How to fix it

Expected. Reduce the model (quantise) so more stays on the GPU, or run hosted for the big families.

System freezes or reboots

Why it happens

System RAM exhausted and the page file is too small, or a power supply at its limit.

How to fix it

System-managed page file on an SSD; a power limit of 80 to 90 percent via the driver tool costs little speed.

No install required - run AI models on the cloud

PirateDiffusion is Telegram only and built for pros: thousands of models, LoRAs and workflows driven by chat commands, with unlimited generation on a fixed price plan.

Try PirateDiffusion

Questions

Is 8 GB enough in 2026?

For SD 1.5, SDXL,

Z-Image Turbo

and

Flux dev

in GGUF, yes. For FLUX.2,

Qwen-Image

and the 14B video models it is the minimum with heavy offload, and generation is slow; most people run those hosted.

Does fp8 lower quality?

For image models, not visibly. GGUF Q8 is equivalent; Q5 shows tiny differences in fine text and textures; Q4 loses some detail.

Why does a 6.5 GB SDXL file need 8 GB?

Text encoders, VAE, the latent and attention activations come on top of the weights, and Windows keeps some VRAM for the display.

What about Apple Silicon or AMD?

Unified memory on

Apple Silicon

makes 32 GB machines run Flux fp8, slowly; AMD runs through ROCm on Linux or ZLUDA / DirectML on Windows, with less headroom. Hosted services sidestep both.

Which is cheaper, a 24 GB card or a hosted plan?

A hosted plan for a year costs less than a used 3090 and covers every family at once; a card pays off if you generate for hours every day. Many people do both.