ideogram-ultra

Ideogram 4 (Ideogram Ultra) Workflows

Overview

This is a LOCAL open-weights pipeline — NOT the hosted Ideogram API. There is no API key, no IdeogramGenerate API node, and no network call at generation time. Comfy-Org released the Ideogram 4 weights on Hugging Face and they run entirely on your GPU via standard UNETLoader / CLIPLoader / VAELoader nodes. (Note: ComfyUI also ships separate API/"partner" nodes that call the paid hosted Ideogram service — that is a different thing and is not what this workflow uses.)

Ideogram 4 is best known for text rendering / typography, poster and graphic-design layouts, and prompt adherence. The hallmark of this workflow is a structured JSON prompt (a "compositional deconstruction" caption with bounding boxes) instead of a plain text prompt — this is what gives precise control over where text and objects land in the frame.

Source workflow this skill is derived from: IDEOGRAM_ULTRA_WORKFLOW-V2.json (UI format, 66 nodes, 4 subgraphs), by Aitrepreneur. It provides both a TEXT TO IMAGE path and an IMAGE TO IMAGE path.

Two unusual things to know up front

1. Dual models. Two UNETs are loaded: a conditional model (ideogram4_fp8_scaled) and an ..._unconditional_fp8_scaled model. A DualModelGuider node uses both to perform asymmetric classifier-free guidance — the unconditional model provides the CFG baseline. There is no negative text prompt; negative conditioning is ConditioningZeroOut.
2. Two text models, different jobs.
   • qwen3vl_8b_fp8_scaled is the actual diffusion text encoder (loaded with CLIPLoader, type ideogram4).
   • gemma4_e4b_it_fp8_scaled is used only inside an optional prompt-builder subgraph (a TextGenerate node) that auto-writes the structured JSON from a plain idea. It is not the diffusion encoder.

Installation

Custom nodes

The workflow needs these four custom node packs (clone into ComfyUI/custom_nodes/). Exact repos from the installer scripts:

git clone https://github.com/ltdrdata/ComfyUI-Manager.git
git clone https://github.com/rgthree/rgthree-comfy
git clone https://github.com/kijai/ComfyUI-KJNodes
git clone https://github.com/cubiq/ComfyUI_essentials

• ComfyUI-KJNodes (kijai) — provides Ideogram4PromptBuilderKJ, ImageSharpenKJ, TextGenerate, and the Ideogram 4 helper nodes. Required.
• rgthree-comfy — Power Lora Loader, Fast Groups Muter/Bypasser, Label, Any Switch. (Used for UI/convenience; the core pipeline still works without them.)
• ComfyUI_essentials (cubiq) — ImageResize+ (used in the img2img path).
• ComfyUI-Manager — node/model management; not required at run time.

The core nodes used in the simplified workflows below (UNETLoader, CLIPLoader, VAELoader, DualModelGuider, SamplerCustomAdvanced, ModelSamplingAuraFlow, BasicScheduler, EmptyFlux2LatentImage, CLIPTextEncode, ConditioningZeroOut, VAEDecode) are built into ComfyUI (recent versions). Only Ideogram4PromptBuilderKJ / ImageSharpenKJ require KJNodes.

Models

Five files. Folder layout and download URLs are taken verbatim from IDEOGRAM_ULTRA-MODELS-NODES_INSTALL.bat / ...RUNPOD.sh:

Notes / things to verify:

• The two diffusion models come from the official Comfy-Org/Ideogram-4 HF repo. The text encoders + VAE are mirrored from the third-party Aitrepreneur/FLX repo in these scripts; the official ones also live on Comfy-Org / Comfy-Org-adjacent repos. Both should be identical files but the FLX mirror is what the provided installer pulls.
• File sizes are uncertain. The official ComfyUI docs page lists each diffusion model at ~13.8 GB, qwen3vl at ~8 GB, gemma4 at ~2 GB, vae at ~335 MB (~38.9 GB total). A web search result claimed ideogram4_fp8_scaled is ~9.28 GB. Treat sizes as approximate — confirm against the HF file listing.
• flux2-vae.safetensors is the same VAE used by Flux 2 / Klein workflows.

Linux / RunPod note

IDEOGRAM_ULTRA-AUTO_INSTALL-RUNPOD.sh creates a venv and installs Torch 2.4.0 + cu121 by default (override via env vars CUDA_TAG, TORCH_VERSION, etc.). Same five model files, same four node repos.

JSON template pack (optional)

IDEOGRAM-TEMPLATES.zip contains 25 ready-made structured-JSON templates for the Ideogram4PromptBuilderKJ node (film poster, book cover, logo board, character sheet, magazine cover, etc.). Per its README, copy the .json files into:

ComfyUI/user/default/kjnodes/ideogram4/templates

then pick them from the template dropdown inside the prompt-builder node.

Key Nodes

CLIPLoader (text encoder)

Ideogram 4 uses Qwen3-VL as its diffusion text encoder. Load it with CLIPLoader and type ideogram4:

{
  "class_type": "CLIPLoader",
  "inputs": {
    "clip_name": "qwen3vl_8b_fp8_scaled.safetensors",
    "type": "ideogram4",
    "device": "default"
  }
}

UNETLoader x2 (conditional + unconditional)

{ "class_type": "UNETLoader", "inputs": { "unet_name": "ideogram4_fp8_scaled.safetensors", "weight_dtype": "default" }},
{ "class_type": "UNETLoader", "inputs": { "unet_name": "ideogram4_unconditional_fp8_scaled.safetensors", "weight_dtype": "default" }}

ModelSamplingAuraFlow (shift)

Applied to both models. In the source workflow shift = 5:

{ "class_type": "ModelSamplingAuraFlow", "inputs": { "model": ["<unet>", 0], "shift": 5 }}

DualModelGuider

The heart of the pipeline. Takes the (shifted) conditional model, the (shifted) unconditional model, positive conditioning, and negative conditioning, plus a CFG value (5 in the source). This replaces the usual CFGGuider.

{
  "class_type": "DualModelGuider",
  "inputs": {
    "model": ["<conditional_model_sampling>", 0],
    "model_negative": ["<unconditional_model_sampling>", 0],
    "positive": ["<clip_text_encode>", 0],
    "negative": ["<conditioning_zero_out>", 0],
    "cfg": 5
  }
}

Input names for DualModelGuider (model_negative, cfg) are inferred from the subgraph wiring and KJNodes; verify against your installed KJNodes version — the exact widget/socket names may differ slightly.

EmptyFlux2LatentImage

Ideogram 4 uses the Flux 2 latent format, so the empty latent is EmptyFlux2LatentImage (not EmptyLatentImage):

{ "class_type": "EmptyFlux2LatentImage", "inputs": { "width": 1024, "height": 1024, "batch_size": 1 }}

BasicScheduler + KSamplerSelect + RandomNoise + SamplerCustomAdvanced

Generation uses the modular sampler stack, not KSampler:

{ "class_type": "BasicScheduler", "inputs": { "model": ["<conditional_model_sampling>", 0], "scheduler": "simple", "steps": 28, "denoise": 1 }},
{ "class_type": "KSamplerSelect", "inputs": { "sampler_name": "euler" }},
{ "class_type": "RandomNoise", "inputs": { "noise_seed": 42 }},
{ "class_type": "SamplerCustomAdvanced", "inputs": {
  "noise": ["<random_noise>", 0],
  "guider": ["<dual_model_guider>", 0],
  "sampler": ["<ksampler_select>", 0],
  "sigmas": ["<basic_scheduler>", 0],
  "latent_image": ["<empty_latent>", 0]
}}

Ideogram4PromptBuilderKJ (the JSON prompt builder)

A KJNodes node that outputs the structured caption JSON string (see "Prompt Style" below). On the canvas you draw bounding boxes for objects/text, set descriptions, a color palette, and background/style. Its string output feeds CLIPTextEncode. Source widget values include width 1920, height 1080, a high-level prompt, a background description, a color_palette array, and an elements array.

⚠️ Editing this node programmatically (panel_set_widget WILL NOT STICK)

This is the single most important thing to know about this node. Its elements_data / style_palette_data widgets are NOT the source of truth — they are serialized from a live in-browser array (node._boxes) inside the KJNodes editor JS. Two mechanisms defeat any external widget edit:

• Queue-time re-serialization. In web/js/ideogram4_prompt_builder.js, elementsWidget.serializeValue() regenerates the value from node._boxes every time the graph is queued. So panel_set_widget(14, "elements_data", ...) sets the value, but ComfyUI overwrites it with the stale editor boxes the instant you run. The edit silently reverts on every run.
• You can't reach node._boxes. It lives in the browser tab; no panel/MCP tool can touch it. Editing elements_data, ideo_editor, or forcing import_mode alone does nothing durable. node._boxes is only ever re-seeded from elements_data on workflow load (onConfigure), and even then the saved o.ideo.boxes blob wins over the widget — so a stale saved workflow reloads stale.

The symptom: you edit the JSON, the panel confirms the new value, but the render (and the visible builder JSON) still shows the OLD prompt — e.g. old subject/region text that "won't go away."

The correct, node-designed fix — drive it via import_json:

1. Add a PrimitiveStringMultiline node containing the FULL caption JSON (the high_level_description + style_description + compositional_deconstruction shape from "Prompt Style" below).
2. Wire it into node 14's import_json input.
3. Set import_mode = "always".

The Python execute() then does used_import = imported is not None and (import_mode == "always" or not boxes) → the caption is built entirely from import_json; the poisoned elements_data/node._boxes are ignored. Bonus: running once in this mode pushes the caption back into the editor via ui, which re-seeds node._boxes and flushes the stale boxes for good. From then on, edit the prompt in the wired string node, not the builder's visual editor. (import_mode = "when empty" only seeds the editor when it has no regions, then the editor wins again — so for programmatic control it MUST be "always".)

ImageSharpenKJ (post-process)

The source applies a light RCAS sharpen after decode: sharpen_mode = rcas, strength = 0.55.

Parameters / Settings

Values below are exactly what the source IDEOGRAM_ULTRA_WORKFLOW-V2.json ships with.

There is also a CFGOverride node in the sampler subgraph (widgets 3, 0.7, 1); it is an optional override and is not the primary guidance path — the primary CFG is DualModelGuider's 5.

Resolutions / Aspect Ratios

The source default is 1920x1080 (set via INTConstant WIDTH/HEIGHT, fed into the prompt builder and latent). The bundled template README recommends:

If a layout looks cramped, increase resolution while keeping the aspect ratio. Use the same width/height in the prompt-builder JSON, the latent, and (img2img) the resize node.

Prompt Style — Structured JSON ("compositional deconstruction")

Ideogram 4 in this workflow expects a JSON caption, not free text. Minimum required shape:

{
  "high_level_description": "one-sentence summary of the whole image",
  "compositional_deconstruction": {
    "background": "scene, environment, color palette, lighting, overall mood",
    "elements": [
      {
        "type": "obj",
        "bbox": [top, left, bottom, right],
        "desc": "what this object is and how it's rendered"
      },
      {
        "type": "text",
        "bbox": [top, left, bottom, right],
        "text": "ACTUAL TEXT TO RENDER",
        "desc": "font style, color, alignment, vintage/print treatment, etc."
      }
    ]
  }
}

An optional style_description object (aesthetics, lighting, medium, art_style, color_palette) can sit at the top level for global style locking.

bbox rules (critical)

• Format is [top, left, bottom, right], values 0–1000 (NOT pixels, NOT x/y/w/h).
• One bbox per major subject — do not split a person into face/hair/clothes boxes; put all detail in one desc.
• Use extra boxes only for genuinely separate items (a product, a title, a second character).
• For character groups in wide images, use vertical columns of non-overlapping boxes.
• For posters: reserve a top zone for the title, middle for the subject, bottom for subtitle/CTA.
• Overlapping text boxes cause garbled text — increase spacing or remove boxes.

Text rendering tips

• Keep rendered text short and bold ("ORDER NOW", "COMING THIS FALL"). Long/tiny text still fails sometimes.
• Lock style explicitly when needed, e.g. "rendered as an actual live-action photograph, not anime, not illustration" or "high-quality Japanese anime, cel shading, not photographic".
• 3–6 strong elements beat 20 overlapping ones.

Generating the JSON automatically

Two options the source provides:

1. Local (Gemma4 subgraph): the "JSON Prompt Builder (Gemma4)" subgraph runs a TextGenerate node on gemma4_e4b_it_fp8_scaled with a system prompt that converts a plain idea into the JSON. (TextGenerate widgets in source: max_tokens 2048, temperature 0.7, top_k 64, top_p 0.95, etc.)
2. External LLM: a workflow Note ships a full system prompt ("You are an expert Ideogram 4 structured JSON prompt assistant…") to paste into ChatGPT/Claude to produce the caption JSON, which you then paste into the manual prompt field.

Either way, the resulting JSON must be valid (double quotes, no trailing commas, exact key compositional_deconstruction) or the builder reports "NOT A VALID IDEOGRAM 4 CAPTION JSON".

Complete Workflow: Text to Image (API format, simplified)

This is the core txt2img path derived from the source, written in API format. Put your structured JSON caption into node 5's text.

{
  "1":  { "class_type": "UNETLoader", "inputs": { "unet_name": "ideogram4_fp8_scaled.safetensors", "weight_dtype": "default" }},
  "2":  { "class_type": "UNETLoader", "inputs": { "unet_name": "ideogram4_unconditional_fp8_scaled.safetensors", "weight_dtype": "default" }},
  "3":  { "class_type": "CLIPLoader", "inputs": { "clip_name": "qwen3vl_8b_fp8_scaled.safetensors", "type": "ideogram4", "device": "default" }},
  "4":  { "class_type": "VAELoader", "inputs": { "vae_name": "flux2-vae.safetensors" }},

  "5":  { "class_type": "CLIPTextEncode", "inputs": { "clip": ["3", 0], "text": "<STRUCTURED JSON CAPTION HERE>" }},
  "6":  { "class_type": "ConditioningZeroOut", "inputs": { "conditioning": ["5", 0] }},

  "7":  { "class_type": "ModelSamplingAuraFlow", "inputs": { "model": ["1", 0], "shift": 5 }},
  "8":  { "class_type": "ModelSamplingAuraFlow", "inputs": { "model": ["2", 0], "shift": 5 }},

  "9":  { "class_type": "DualModelGuider", "inputs": {
            "model": ["7", 0],
            "model_negative": ["8", 0],
            "positive": ["5", 0],
            "negative": ["6", 0],
            "cfg": 5
          }},

  "10": { "class_type": "EmptyFlux2LatentImage", "inputs": { "width": 1920, "height": 1080, "batch_size": 1 }},
  "11": { "class_type": "BasicScheduler", "inputs": { "model": ["7", 0], "scheduler": "simple", "steps": 28, "denoise": 1 }},
  "12": { "class_type": "KSamplerSelect", "inputs": { "sampler_name": "euler" }},
  "13": { "class_type": "RandomNoise", "inputs": { "noise_seed": 42 }},

  "14": { "class_type": "SamplerCustomAdvanced", "inputs": {
            "noise": ["13", 0],
            "guider": ["9", 0],
            "sampler": ["12", 0],
            "sigmas": ["11", 0],
            "latent_image": ["10", 0]
          }},

  "15": { "class_type": "VAEDecode", "inputs": { "samples": ["14", 0], "vae": ["4", 0] }},
  "16": { "class_type": "ImageSharpenKJ", "inputs": { "image": ["15", 0], "sharpen_mode": "rcas", "strength": 0.55 }},
  "17": { "class_type": "SaveImage", "inputs": { "images": ["16", 0], "filename_prefix": "IDEOGRAM" }}
}

If you don't have KJNodes / want to skip sharpening, drop node 16 and feed ["15", 0] straight into SaveImage.

Image to Image (notes)

The source img2img path adds, before sampling:

1. LoadImage → ImageResize+ (ComfyUI_essentials; keep proportion, lanczos, e.g. target 1024) → VAEEncode (with flux2-vae) to produce the input latent.
2. Feed that latent into SamplerCustomAdvanced.latent_image instead of EmptyFlux2LatentImage.
3. Set denoise 0.6 on BasicScheduler (this is the PrimitiveFloat DENOISE value in the source). Lower denoise = closer to the input image.

Everything else (dual models, DualModelGuider, scheduler, sampler, decode, sharpen) is identical to txt2img.

The exact VAEEncode wiring for img2img is inferred (the source routes it through an rgthree Any Switch and subgraph I/O); confirm sockets in your build.

Pipeline at a Glance

UNETLoader (ideogram4_fp8_scaled) ─────► ModelSamplingAuraFlow(shift=5) ─┐
UNETLoader (ideogram4_unconditional) ──► ModelSamplingAuraFlow(shift=5) ─┤
                                                                          ├─► DualModelGuider(cfg=5)
CLIPLoader (qwen3vl_8b, type=ideogram4) ─► CLIPTextEncode(JSON) ──────────┤        │
                                              └► ConditioningZeroOut ──────┘        │
EmptyFlux2LatentImage (1920x1080) ─────────────────────────────────────────────────┤
BasicScheduler (simple, 28, denoise=1) ─────────────────────────────────────────────┤
KSamplerSelect (euler) ─────────────────────────────────────────────────────────────┤
RandomNoise (fixed seed) ───────────────────────────────────────────────────────────┘
                                              └► SamplerCustomAdvanced ─► VAEDecode (flux2-vae)
                                                                          └► ImageSharpenKJ (rcas 0.55) ─► SaveImage

Optional upstream: Ideogram4PromptBuilderKJ  OR  TextGenerate(gemma4) ─► CLIPTextEncode.text

VRAM Considerations

• Both diffusion models plus the Qwen3-VL encoder are large. Loading two ~13.8 GB UNETs is the main cost; expect this to be heavy on 24 GB GPUs. The official docs cite a 16 GB minimum for the FP8 models, but that assumes ComfyUI swaps models in/out rather than holding both resident.
• Always clear_vram before switching to Ideogram 4 from another model family.
• If you OOM: rely on ComfyUI's automatic model offloading, run --lowvram, or reduce resolution.
• Exact VRAM numbers for the dual-model setup are not verified in the source files — treat the above as guidance, not measured figures.

Troubleshooting

• Prompt edits to Ideogram4PromptBuilderKJ won't stick / stale prompt keeps coming back — elements_data is re-serialized from the browser editor's node._boxes at queue time, so panel_set_widget reverts on every run and you can't reach node._boxes externally. Fix: wire a PrimitiveStringMultiline (full caption JSON) into the node's import_json input and set import_mode = "always". See "Editing this node programmatically" under Key Nodes. This is the ONLY reliable way to drive the prompt from outside the browser.
• "NOT A VALID IDEOGRAM 4 CAPTION JSON" — JSON is malformed. Use double quotes, no trailing commas, exact key compositional_deconstruction, matched brackets. Validate in any JSON linter.
• Garbled / overlapping text in the image — text bboxes overlap or text is too long. Increase spacing, shorten text, remove boxes.
• Style drift (anime when you wanted photo, etc.) — add explicit style-lock language in the element/style description.
• Wrong element placement — remember bbox is [top, left, bottom, right] 0–1000, not pixels and not x/y/w/h.
• Same seed, different image across machines — expected; differs by GPU, drivers, PyTorch/CUDA/ComfyUI versions (per template README).
• CLIPLoader type missing ideogram4 — update ComfyUI; the ideogram4 CLIP type and the Flux2/Ideogram nodes require a recent build (installer pins ComfyUI portable v0.24.0).
• Ideogram4PromptBuilderKJ / DualModelGuider not found — update KJNodes (git pull in custom_nodes/ComfyUI-KJNodes); these are recent additions.