model-compatibility

ComfyUI Model Compatibility Matrix

Stable Diffusion 1.5 (SD 1.5)

Overview

The original widely-adopted Stable Diffusion model. Huge ecosystem of fine-tunes, LoRAs, ControlNets, and embeddings. Still the most compatible and lightweight model family.

Configuration

Workflow Pattern

CheckpointLoaderSimple → MODEL(0), CLIP(1), VAE(2)
  CLIP(1) → CLIPTextEncode (positive) → CONDITIONING
  CLIP(1) → CLIPTextEncode (negative) → CONDITIONING
EmptyLatentImage (width=512, height=512) → LATENT
KSampler (cfg=7.5, steps=20, sampler="euler", scheduler="normal") → LATENT
VAEDecode → IMAGE
SaveImage

VAE Notes

• Most SD 1.5 checkpoints have a built-in VAE, but it's often mediocre
• Recommended: Use external vae-ft-mse-840000-ema-pruned.safetensors for better color accuracy
• Load via VAELoader node and connect to VAEDecode
• FP16 VAE can produce NaN on some images — FP32 VAE is more stable

ControlNet Compatibility

SD 1.5 has the largest ControlNet ecosystem:

LoRA Compatibility

• SD 1.5 LoRAs ONLY work with SD 1.5 base models
• Format: .safetensors in models/loras/
• Loader: LoraLoader node — connects between checkpoint and CLIPTextEncode
• Strength range: 0.5-1.0 (higher can cause artifacts)

SDXL (Stable Diffusion XL)

Overview

Major upgrade from SD 1.5 with dual CLIP encoders, higher native resolution, and better prompt understanding. Includes Turbo and Lightning variants for fast generation.

Configuration — SDXL 1.0 (Base)

Configuration — SDXL Turbo

Configuration — SDXL Lightning

SDXL Refiner

The optional SDXL refiner model does a second pass to improve fine details:

CheckpointLoaderSimple (base) → KSampler (steps=25, start=0, end=20)
CheckpointLoaderSimple (refiner) → KSampler (steps=25, start=20, end=25)

• The refiner uses KSamplerAdvanced with start_at_step and end_at_step
• Typically run the base for 80% of steps, refiner for the last 20%
• Refiner checkpoint: sd_xl_refiner_1.0.safetensors

Workflow Pattern

CheckpointLoaderSimple → MODEL(0), CLIP(1), VAE(2)
  CLIP(1) → CLIPTextEncode (positive) → CONDITIONING
  CLIP(1) → CLIPTextEncode (negative) → CONDITIONING
EmptyLatentImage (width=1024, height=1024) → LATENT
KSampler (cfg=7.0, steps=25, sampler="dpmpp_2m", scheduler="karras") → LATENT
VAEDecode → IMAGE
SaveImage

ControlNet Compatibility

SDXL ControlNets are separate from SD 1.5 ControlNets:

LoRA Compatibility

• SDXL LoRAs ONLY work with SDXL base models — NOT with SD 1.5
• Same LoraLoader node as SD 1.5
• Lightning LoRAs are SDXL LoRAs that enable few-step generation

Flux (Flux.1)

Overview

Black Forest Labs' model with a T5-XXL text encoder. Produces high-quality images without negative prompts. Available in schnell (fast) and dev (quality) variants.

Configuration — Flux Schnell

Configuration — Flux Dev

Loading Methods

Method 1: Single Checkpoint (simplest)

CheckpointLoaderSimple (ckpt_name="flux1-schnell.safetensors")
  → MODEL(0), CLIP(1), VAE(2)

Method 2: Split Components (recommended for FP8)

UNETLoader (unet_name="flux1-schnell-fp8.safetensors") → MODEL
DualCLIPLoader (clip_name1="t5xxl_fp16.safetensors", clip_name2="clip_l.safetensors", type="flux") → CLIP
VAELoader (vae_name="ae.safetensors") → VAE

CRITICAL Rules

• CFG MUST be 1.0 — Flux uses guidance embedded in the model, not classifier-free guidance
• No negative prompt — Empty string or don't connect the negative input at all
• Separate VAE required — Flux uses its own VAE (ae.safetensors), not SD VAEs
• FP8 strongly recommended for 24GB cards — FP16 Flux barely fits in 24GB VRAM
• T5-XXL encoder can be loaded in FP8 to save additional VRAM

Workflow Pattern

UNETLoader (flux fp8) → MODEL
DualCLIPLoader (t5xxl + clip_l, type="flux") → CLIP
VAELoader (ae.safetensors) → VAE

CLIPTextEncode (positive prompt) → CONDITIONING
  (no negative CLIPTextEncode needed)

EmptyLatentImage (width=1024, height=1024) → LATENT

KSampler (cfg=1.0, steps=4, sampler="euler", scheduler="simple") → LATENT
VAEDecode (vae from VAELoader) → IMAGE
SaveImage

ControlNet Compatibility

Flux ControlNets are model-specific:

SD 1.5 and SDXL ControlNets do NOT work with Flux.

LoRA Compatibility

• Flux LoRAs ONLY work with Flux models
• Typically loaded via LoraLoader same as SD models
• Flux LoRA ecosystem is smaller than SD 1.5/SDXL but growing
• Some Flux LoRAs require specific trigger words

Stable Diffusion 3 / 3.5 (SD3)

Overview

Stability AI's next-generation model with triple CLIP architecture. Better prompt adherence and longer prompt support via T5-XXL.

Configuration

Triple CLIP Loading

CheckpointLoaderSimple → MODEL(0), CLIP(1), VAE(2)

Or for separate CLIP control:

DualCLIPLoader (clip_l + clip_g) → CLIP
CLIPLoader (t5xxl) → CLIP

Key Differences from SD 1.5/SDXL

• Much better text rendering capabilities
• Handles spatial relationships better ("cat on the left, dog on the right")
• T5-XXL enables very long, detailed prompts (no 77-token limit concern)
• Lower CFG values (4-7 vs 7-12)
• Minimal negative prompting needed
• shift parameter in sampling affects noise schedule

ControlNet Compatibility

• SD3-specific ControlNets are limited
• Check for SD3-compatible community ControlNets
• SD 1.5 and SDXL ControlNets do NOT work with SD3

LTXV (Video Models)

Overview

Latent video diffusion models for text-to-video and image-to-video generation. Very VRAM-intensive.

Configuration

VRAM Management

• Always use FP8 quantized models on 24GB cards
• Reduce frame count if OOM persists
• Lower resolution helps significantly
• Close other GPU-using applications
• Consider --lowvram flag for ComfyUI

Cross-Family Compatibility Rules

LoRA Compatibility

LoRAs are model-family specific and are NOT interchangeable:

Using a LoRA with the wrong base model will produce garbage images or errors.

ControlNet Compatibility

ControlNets are also model-family specific:

VAE Compatibility

Rule: Never mix VAEs across model families. An SD 1.5 VAE decoding Flux latents will produce garbage.

Embedding/Textual Inversion Compatibility

Sampler/Scheduler Compatibility

Most samplers work across all models, but some combinations are optimal:

Quick Decision Guide

Choosing a Model

Choosing Resolution

Going below the recommended resolution produces blurry/low-quality results. Going above the maximum risks OOM errors or quality degradation (tiling artifacts).