You generate a portrait and the eyes are slightly wrong: one is a few pixels higher than the other, the irises point in subtly different directions, or one pupil is noticeably bigger. At thumbnail size the face looks fine, but anyone who opens the full-res image immediately feels it’s “off.” This happens because in a standard SDXL 1024x1024 half-body portrait the eye region is only about 50x30 pixels of latent space, and the diffusion model has no resolution budget left to lock in symmetric anatomy.
Fastest fix (works for most people): run two ADetailer / FaceDetailer passes after the base render — first a face_yolov8n.pt pass at denoise 0.4 to rebuild the whole face at full resolution, then a second eye-only pass (using mediapipe_face_mesh_eyes_only or a Civitai eye-detection model) with Mask merge mode = Merge so both eyes are repainted together and stay matched. The two-pass order matters: fix the face first, then refine the eyes inside the cleaned-up face.
Which bucket are you in?
| Symptom | Most likely cause | Jump to |
|---|---|---|
| Face is small in frame, whole face looks mushy | Low latent resolution on the face | Cause 1, Step 1 |
| Eyes only drift in tall/9:16 renders | Non-square aspect ratio | Cause 2, Step 5 prevention |
| Eyes wrong on every render, no face node in workflow | No face/eye pass at all | Cause 3, Steps 1-2 |
| Eyes correct when you lower LoRA weight | Style LoRA overdriving anatomy | Cause 4 |
| Eyes improve when you raise steps | Undersampling | Cause 5 |
Common causes
Ordered by hit rate, highest first.
1. Face region is too small in latent space
In a standard SDXL 1024x1024 render of a half-body portrait, the actual eye area is roughly 50x30 pixels. The model makes anatomy decisions at that resolution — it has no chance of placing two pupils with sub-pixel precision.
How to spot it: zoom in on the face. If the face occupies less than about 30% of the frame, you are in low-resolution-face territory and the eyes will drift.
2. Tall portrait or non-square aspect
SDXL’s native resolution is 1024x1024 (about 1.05M pixels). When you render at 832x1216 or 9:16, faces in tall frames more often skew, tilt slightly, or end up with mismatched eye height because the face occupies a thin vertical band.
How to spot it: same prompt, same seed, switch to 1024x1024. If the eyes line up there but not at 832x1216, aspect ratio is the cause.
3. No face / eye pass in the pipeline
Default ComfyUI / Automatic1111 / Forge text-to-image workflows do not run a dedicated face pass. The model outputs whatever it outputs and you ship it. Without a face-detailer step, eye misalignment is the norm, not the exception.
How to spot it: check your workflow. In ComfyUI, look for a FaceDetailer node (Impact Pack). In A1111 / Forge, check whether the ADetailer panel below the prompt is enabled. If neither exists, you have no safety net.
4. Heavy style LoRA distorting anatomy
Strong style LoRAs (especially anime-photoreal hybrids or painterly LoRAs at weight 1.0 and above) often warp facial proportions. The style imprints on top of the base anatomy and shifts eye placement.
How to spot it: drop LoRA weight from 1.0 to 0.6 and regenerate with the same seed. If the eyes correct themselves, the LoRA is overdriving.
5. Low step count or wrong sampler
Samplers like Euler a and DPM++ 2M Karras need roughly 25-35 steps to fully resolve fine anatomy. If you are at 15 steps with a turbo/lightning sampler, the model hasn’t finished refining facial features.
How to spot it: bump steps from 20 to 35 with the same seed. If the eyes improve, you were undersampling.
Shortest path to fix
Step 1: Add a face-detailer pass
In A1111 / Forge, expand the ADetailer panel (the extension is by Bing-su; install from Extensions if it isn’t there) and enable it. In ComfyUI, drop in a FaceDetailer node from ComfyUI-Impact-Pack. First-tab / first-pass settings:
ADetailer model: face_yolov8n.pt
Detection confidence: 0.3
Inpaint denoising: 0.4 (0.35-0.45 range; default 0.5 can change identity)
Mask blur: 4
Inpaint width/height: 1024 (NOT the base render size)
Sampler steps: 20-25, same sampler as baseThe detailer crops the face, upscales the crop, re-renders just the face at full resolution, then composites it back. The eye region now gets a full latent budget. In ComfyUI, if the repainted face doesn’t sit cleanly back into the image, nudge bbox_crop_factor (try 1.5) and bbox_dilation so the mask better fits the detected region.
Step 2: Add a SECOND, eye-only pass
A whole-face pass fixes most cases, but stubborn pupil/iris mismatch needs the eyes treated on their own. In ADetailer, switch to the 2nd tab and add an eye detector; in ComfyUI, chain a second FaceDetailer (or DetailerForEach) using an eye-segmentation model.
2nd-pass model: mediapipe_face_mesh_eyes_only (built into ADetailer)
or a Civitai eye model (e.g. "Eye Detailer/Segmentation",
"Eyeful") for tighter masks
Confidence: 0.35
Mask merge mode: Merge <-- critical: repaints BOTH eyes as one mask
Inpaint denoising: 0.35-0.45
ADetailer prompt: matching pupils, symmetric eyes, both eyes same size, sharp iris detailThe Mask merge mode = Merge setting (Mask preprocessing in ADetailer) is what keeps the two eyes consistent — without it, each eye is inpainted independently and can drift apart again. This eye-only pass is the single highest-leverage step for this specific symptom.
Step 3: Face-restore fallback (CodeFormer / GFPGAN)
If you can’t add detailer passes (e.g. an Extras/upscale-only workflow), fall back to a face-restore model. Mind the weight direction — it trips up most people:
A1111 "CodeFormer weight (fidelity)" slider:
0 = maximum restoration effect (sharpest, but can alter identity & smooth skin)
1 = minimum effect (closest to the original)
Recommended: 0.5-0.7 (higher = preserves more texture/identity)
GFPGAN visibility: 0.5-0.8Note: this is the same w value as the standalone CodeFormer --fidelity_weight flag in the CLI — w=0 is max quality (strongest restoration, can alter identity) and w=1 is max fidelity (closest to the original). A1111 passes the slider straight through, so a higher value means a gentler, more faithful restore. These models are trained on aligned face datasets, so they correct eye symmetry and pupil size well, but they smooth skin as they go — that’s why you don’t push the effect to maximum.
Step 4: Manual inpaint when automated fails
For images that resist both passes:
- Open the image in Automatic1111 / Forge img2img → Inpaint
- Mask just the eye region; set Mask blur to 8
- Set Denoising strength to 0.45
- Set Inpaint area to “Only masked” with Only masked padding 32
- Re-prompt with
symmetric eyes, matching pupils, both eyes same size, looking at camera, sharp iris detail - Generate a batch of 4 and pick the cleanest
Step 5: Prevent at gen time with prompt hints
Add to the positive prompt:
symmetric face, both eyes aligned, matching pupils, perfect eye anatomy, sharp iris detail, eye contactAnd to the negative prompt:
asymmetric eyes, mismatched pupils, lazy eye, cross-eyed, heterochromia, different iris color, droopy eyelidThese hints won’t fix the resolution problem on their own, but they nudge the model toward better defaults before the detailer passes clean up.
How to confirm it’s fixed
- Open the image at 100% (full resolution), not the gallery thumbnail.
- Check three things specifically: eye height (draw an imaginary horizontal line — both inner corners should sit on it), pupil size (compare diameters), and iris/gaze direction (both should point the same way).
- Flip the image horizontally. Asymmetry that you stopped noticing jumps back out on a mirrored copy — if it still looks right mirrored, it’s genuinely symmetric.
Prevention
- Enable a face-detailer / ADetailer pass by default on every portrait; save it into your default workflow or preset so you never ship a render without it.
- Keep a second eye-only pass with Mask merge mode = Merge wired in for close-up portraits.
- For tall portraits, render at 1024x1024 first, then outpaint vertically, instead of going straight to 832x1216.
- Keep the CodeFormer effect moderate (A1111 weight
0.5-0.7) so you never wipe out skin texture. - Cap style-LoRA weight at
0.7-0.8to leave base anatomy intact.
FAQ
Why do the eyes only break on close-up portraits but look fine on full-body shots? Full-body shots put the whole face — including both eyes — into a tiny patch of latent space, so the model never even attempts fine eye detail and you don’t notice. Close-ups give the eyes enough pixels that errors become visible. Both benefit from a face/eye detailer pass; the close-up just makes the flaw obvious.
My ADetailer face pass runs but the eyes are still off — what now?
That’s exactly when you add the second, eye-only pass (Step 2). Use mediapipe_face_mesh_eyes_only or a Civitai eye-detection model, set Mask merge mode = Merge, and add an eye-specific ADetailer prompt. Repainting both eyes as one merged mask is what stops them drifting apart again.
What denoising strength should the eye pass use?
Stay in the 0.35-0.45 range. The FaceDetailer default of 0.5 is fine for general cleanup but can shift the character’s identity; lower denoise fixes structure (mismatched eyes, wrong pupil size) while keeping the face you generated.
Does “Restore faces” (CodeFormer/GFPGAN) alone fix this?
Often partially, but it smooths skin and can soften the whole image. In A1111 the CodeFormer weight slider is inverted from what people expect: 0 is the strongest effect and 1 is the gentlest, so use 0.5-0.7 to keep texture. A targeted ADetailer/inpaint eye pass is cleaner because it only touches the eyes.
Is this a ComfyUI-only or A1111-only problem?
Neither — it’s a resolution limitation of the base diffusion model, so it shows up in ComfyUI, Automatic1111, Forge, and most SDXL/SD pipelines. The fix (a dedicated face pass plus an eye pass) exists in all of them: FaceDetailer in ComfyUI, ADetailer in A1111/Forge.
Related
- AI Image Hands and Fingers Wrong
- AI Image Unnatural Pose
- AI Image Symmetric Pose Broken
- AI Image Skin Tone Unnatural
- AI Image Skin Too Smooth and Plastic
- AI Image Low Detail
- AI Image Character Consistency
External references: ADetailer (Bing-su) on GitHub, ComfyUI-Impact-Pack, CodeFormer.