AI Image JPEG Edge Artifacts and Banding: 5 Steps to Clean Export

Blocky edges and banding at object boundaries, especially after inpainting. Cause is VAE bit depth plus bad export. Fix with fp16-fix VAE and PNG export.

You finish an SDXL render or an inpaint composite and the output has blocky 8x8 JPEG artifacts at high-contrast edges, visible banding in gradients (especially sky and skin), and color quantization in shadow areas. The image looks fine in the live preview at 50% zoom, but at 100% it falls apart. There are two distinct causes that often combine: the VAE decode introducing bit-depth issues (especially the SDXL default fp16 VAE) and the JPEG export step quantizing the already-fragile output.

The fix is to swap to a more stable VAE (SDXL fp16-fix or 0.9.fp32), export to PNG instead of JPEG for high-quality work, and if you must use JPEG, set quality to 95+ and avoid chroma subsampling.

Common causes

Ordered by hit rate, highest first.

1. SDXL default fp16 VAE NaN issue

The original SDXL VAE on fp16 GPUs produces NaN values in some pixels, which manifest as banding, color holes, and edge artifacts. This is a well-known bug, fixed by the sdxl-vae-fp16-fix VAE on Hugging Face.

How to spot it: SDXL output specifically. Same workflow on SD 1.5 is clean. Banding visible in skies, gradients, and shadow areas.

2. JPEG export at quality 80 or below

Default save settings in many tools (Photoshop “Save for Web” medium, Telegram, Twitter) recompress to quality 70-80. That introduces 8x8 block artifacts at every edge.

How to spot it: zoom to 100%, look at any sharp edge. If you see 8x8 grid patterns, that is JPEG quantization.

3. Chroma subsampling 4:2:0

Most JPEG defaults use 4:2:0 chroma subsampling, which halves the resolution of color channels. Red/blue edges (lipstick, neon signs, sunsets) get especially bad smearing.

How to spot it: red and blue edges look fuzzy and shifted while green/white edges look sharp.

4. Upscale before composite

If you upscale a low-quality JPEG before pasting it into your render (or compositing two renders), the upscaler amplifies the artifacts.

How to spot it: source had no artifacts at native resolution, artifacts appeared after upscale.

5. Inpaint mask edge mismatch

Hard-edge inpaint masks produce visible seams where the inpainted region meets the original. The seam looks like a JPEG edge but is actually a denoise mismatch.

How to spot it: artifact follows the exact shape of your inpaint mask.

Shortest path to fix

Step 1: Swap the SDXL VAE

Download sdxl_vae_fp16_fix.safetensors from Hugging Face (stabilityai/sdxl-vae or madebyollin/sdxl-vae-fp16-fix). Place it in:

Automatic1111: stable-diffusion-webui/models/VAE/
ComfyUI: ComfyUI/models/vae/
Forge: webui/models/VAE/

Then in your workflow, explicitly select that VAE instead of “automatic” or “Baked VAE.” Re-render the same prompt — banding and edge artifacts should disappear immediately.

If you cannot use the fp16-fix variant, fall back to running the VAE decode in fp32 (--no-half-vae in Automatic1111). Slower but stable.

Step 2: Export to PNG, not JPEG

For final delivery and any time quality matters:

Automatic1111: Settings > Saving images > File format: png
ComfyUI: use Save Image node, default is PNG
Photoshop: File > Export > Export As > PNG (24-bit)
Affinity: File > Export > PNG

PNG is lossless. You will never get JPEG block artifacts. File size is 3-5x larger but for a 1024x1024 image that is fine.

Step 3: When you must use JPEG, set quality 95+

If JPEG is required (web upload limits, email):

Photoshop: File > Export > Save for Web > JPEG Quality 95+, chroma "4:4:4 progressive"
Affinity: Export > JPEG > Quality 95, "No chroma subsampling"
ImageMagick CLI: convert input.png -quality 95 -sampling-factor 1x1 output.jpg
Squoosh.app: MozJPEG, quality 92, chroma subsampling off

The chroma subsampling setting matters more than the quality slider for color artifacts.

Step 4: Final upscale at higher resolution

Render the original at 1024x1024, upscale to 2048 or 4096 with a grain-preserving upscaler, then downscale to your delivery size:

Render: 1024x1024
Upscale: 4x with 4x_NMKD-Siax to 4096x4096
Downscale: bilinear to your target (1920x1920 or 1080x1080)
Export: PNG or JPEG quality 95

Downscaling averages away minor artifacts that survived upscaling.

Step 5: Soften inpaint mask edges

If artifacts follow an inpaint mask boundary:

Mask blur: 8 to 16 pixels (was 2-4)
Mask padding: 32 pixels
Inpaint denoise: 0.4 (matches the surrounding texture better than 0.6+)
Use "Inpaint area: whole picture" for blends, "Only masked" for surgical fixes

A wider mask blur transitions the denoise gradient over more pixels, hiding the seam.

Prevention

  • Always use sdxl-vae-fp16-fix as your default SDXL VAE — never trust the baked one
  • Set Automatic1111 / Forge default save format to PNG; only switch to JPEG per-export
  • Never reupload an AI image through Twitter / Facebook / WhatsApp if you care about quality — they recompress
  • For client deliveries, always send both PNG (master) and JPEG quality 95 (preview)
  • Build a habit of zooming to 100% on every final render before shipping

Tags: #ai-image #Troubleshooting #postprocess #vae