Model Fills Missing Details With Invented Assumptions

You asked for a quarterly summary of a customer’s account activity. You provided login dates and ticket counts. The model returned a tidy summary that included the customer’s “renewal date in Q3”, their “primary contact at the company”, and an “estimated revenue impact of approximately $42k”. You never gave it any of those. They sound right. They are all invented. The model read “write a quarterly summary” as “produce a complete-looking quarterly summary”, and by training-distribution norms a complete summary has a renewal date and a contact name. So it filled them.

Fastest fix: add one rule to the prompt — If a detail is not in the input, output "UNKNOWN". Do not infer or estimate. — and switch any structured output to a schema where every field can be null. That alone stops most gap-filling. The rest of this page is the verification layer for outputs you cannot eyeball.

A note that drives everything below: structured-output mode guarantees the shape of the answer, not the truth of it. As of June 2026 OpenAI, Anthropic, and Google all compile your JSON Schema into a token grammar so the model literally cannot emit a non-conforming field — but a schema-valid "renewal_date": "2026-09-01" can still be fabricated. Shape constraints close the malformed failure; they do not close the invented failure. You need both.

Which bucket are you in

Symptom	Likely cause	Go to
Invented details are plausible (dates, names, round dollar amounts)	Common-sense gap-filling from priors	Steps 1, 6
Prompt says “complete”, “comprehensive”, “full”	The adjective demands no gaps	Step 1
A required JSON field is always populated, never null	Schema has no missing-data path	Step 2
Output is a table or bullet list with no empty cells	Format reads “complete” as “no blanks”	Steps 1, 2
Words like “approximately”, “estimated”, “typically” appear	Linguistic hedging that signals a guess	Step 5
You cannot trace a claim back to any input line	No verification pass	Step 6

Common causes

1. Prompt asks for “complete”, “comprehensive”, or “full”

These words mean “leave no gaps”. The model takes them literally and invents the gaps closed.

How to spot it: your prompt verb or adjective implies completeness.

2. Output schema has required fields with no missing-data path

If your schema says { "renewal_date": string } with no null option, the model has to put a string there. It will, true or not. This is sharper with strict structured outputs: as of June 2026, OpenAI’s strict mode requires every property to appear in required, so a field with no nullable path is guaranteed to be filled.

How to spot it: schema has no explicit null / unknown handling.

3. No rule for “when unsure”

You never told the model what to do when it does not know. The default behavior is “produce something plausible”.

How to spot it: prompt has no if unknown: rule.

4. Common-sense gap-filling

For very common patterns (employee names, dates, addresses), models fill from prior-training expectations of “what this kind of record looks like”. It is not random — it is statistical priors.

How to spot it: invented details are plausible and follow common patterns (e.g., “Q3” for a renewal, “John Smith” for a contact, a round number for revenue).

5. Output format encourages completeness

Tables especially: an empty cell looks wrong, so the model fills it. Bullet lists are similar — half a bullet list reads as incomplete.

How to spot it: format is a table, structured list, or schema-driven object.

Before you change anything

• Identify which details in the output you never supplied.
• Save the invented output for comparison.
• Decide the policy: should missing data be "UNKNOWN", null, blank, or an error?
• Check your schema for required fields that have no missing-data path.
• Decide whether the workflow needs the missing data, or whether marking it missing is sufficient.

Information to collect

• The original input data (what you actually gave).
• The output the model produced.
• Specific invented details, marked.
• Your schema (if any) and required fields.
• Model and temperature.

Shortest path to fix

Step 1: Explicit “if unknown” rule

Rules for missing data:
- If a required detail was not provided in the input, output "UNKNOWN".
- Do not infer. Do not estimate.
- Do not write "approximately", "around", or "roughly" unless
  the input contains a number.
- If more than 3 fields would be UNKNOWN, stop and ask for the missing data.

This single block solves the majority of gap-filling in plain-text outputs. Put it near the end of the prompt, after the task, so it is the last instruction the model reads.

Step 2: Schema with explicit null handling

Schema:
{
  "renewal_date": "<ISO date OR null>",
  "primary_contact": "<name OR null>",
  "revenue_impact_usd": "<number OR null>",
  "data_gaps": ["<field name that was null and why>"]
}

The null option plus the data_gaps array makes “missing” a first-class output. If you are calling an API with strict structured outputs, encode the same idea in real JSON Schema so the grammar permits null:

{
  "type": "object",
  "additionalProperties": false,
  "required": ["renewal_date", "primary_contact", "revenue_impact_usd", "data_gaps"],
  "properties": {
    "renewal_date":      { "type": ["string", "null"] },
    "primary_contact":   { "type": ["string", "null"] },
    "revenue_impact_usd":{ "type": ["number", "null"] },
    "data_gaps":         { "type": "array", "items": { "type": "string" } }
  }
}

The ["string", "null"] type array is the canonical way to make a field nullable under OpenAI strict mode and Gemini response_json_schema as of June 2026; Anthropic’s structured outputs accept the same JSON Schema. Without a nullable type, strict mode forces a non-null value into the field — which is exactly the fabrication you are trying to kill.

Step 3: Make the model list assumptions before producing

Step 1: List every assumption you would need to make to produce
        a complete answer. Number them.
Step 2: For each, mark whether the input data supports it (YES) or
        whether you would be inventing it (NO).
Step 3: Produce the answer using only YES assumptions. For NO
        assumptions, write UNKNOWN in the output.

This converts implicit fabrication into an explicit, auditable step. On reasoning-mode models (GPT-5.5 Thinking, Claude Opus 4.7, Gemini 3.1 Pro) the model already does some of this internally, but writing it out gives you a paper trail you can inspect.

Step 4: Use few-shot examples that include UNKNOWN

Show the model what acceptable “I do not know” output looks like:

Example 1:
Input: Login dates only
Output:
{
  "renewal_date": null,
  "primary_contact": null,
  "revenue_impact_usd": null,
  "data_gaps": ["renewal_date (not in input)", "primary_contact (not in input)", "revenue_impact_usd (not in input)"]
}

Now produce for: <real input>

One worked example of a half-empty record does more than a paragraph of instruction, because it shows the model that null-heavy output is expected, not a failure.

Step 5: Forbid filler vocabulary

Forbidden phrases (do not use unless the input contains evidence):
- "approximately", "around", "roughly", "likely", "estimated"
- "based on industry norms"
- "typical", "average", "standard"

These phrases are the linguistic signature of gap-filling. Banning them forces the model to either provide evidence or say UNKNOWN. They are also the easiest thing to grep for when you audit (see “How to confirm”).

Step 6: Have the model verify its output against the input

Append:

After producing, list each non-UNKNOWN claim with the exact input line
that supports it. If you cannot point to a supporting input line,
the claim must become UNKNOWN.

The verification step catches what the rules miss. For high-volume pipelines, run it as a separate call: prompt 1 produces, a fresh prompt 2 gets only the input and the draft and is told to flag any claim with no source line. A second pass with no memory of “wanting to look complete” is harder to fool.

How to confirm the fix

• Spot-check 5 outputs: every specific claim has a supporting line in the input.
• The data_gaps array is non-empty whenever the input is partial.
• "UNKNOWN" (or null) appears in outputs where it should.
• Grep the outputs for the forbidden filler words — count should be 0.
• A teammate reviewing the output cannot point to any “wait, where did that come from” detail.

If it still fails

1. Lower the temperature. Strict factuality usually wants temperature 0 to 0.2; high temperature widens the model’s willingness to guess.
2. Switch to structured-output mode (JSON Schema with strict/grammar enforcement on OpenAI, Anthropic, or Gemini). Remember it fixes shape, not truth — pair it with the nullable schema from Step 2, not instead of it.
3. Split produce and verify into two calls so the verifier has no stake in looking complete.
4. Add retrieval (RAG) for high-stakes work so the model has explicit sources, and require a source citation per claim. A claim with no retrievable source becomes UNKNOWN.

FAQ

Why does the model invent details that are so specific (a date, a dollar figure) instead of just leaving a blank? Because a blank is, statistically, the rarer completion. In its training data a “quarterly summary” almost always has a renewal date, so the most probable next tokens are a plausible date — not the word “UNKNOWN”. You have to make UNKNOWN the explicitly sanctioned, demonstrated option (Steps 1 and 4) or the priors win.

Does turning on Structured Outputs / JSON mode stop the fabrication? No, and assuming it does is the most common mistake here. As of June 2026 OpenAI, Anthropic, and Gemini all enforce the schema with a token grammar — the model cannot emit a field that violates the shape. But a schema-valid value can still be invented. Structured output stops malformed JSON; the nullable type plus the UNKNOWN rule stop invented values.

My JSON Schema is in strict mode and the model keeps filling a field I want optional. Under OpenAI strict mode every property must be listed in required; there is no “leave it out” option. The fix is not to drop the field but to make its type nullable: "type": ["string", "null"]. Then null is a legal value and the grammar will allow the model to decline.

Should missing data be null, "UNKNOWN", blank, or an error? Pick one and use it everywhere. For JSON pipelines use null plus a data_gaps array so downstream code can branch on it. For plain prose use the literal "UNKNOWN" so you can grep for it. Blank cells are the worst choice — they read as “done” to both the model and a human reviewer.

A second verification call costs an extra request. Is it worth it? For anything that feeds a decision, a join, or a stored record: yes. The verify pass typically costs a fraction of the produce pass (it only re-reads, it does not re-generate) and it is the single highest-yield check for caught fabrication. For throwaway drafts you can skip it and rely on Steps 1, 2, and 5.

Prevention

• Default rule in every prompt: “If unknown, output UNKNOWN. Do not infer.”
• Mark required vs optional fields in schemas; make optional fields nullable rather than dropping them.
• Forbid filler vocabulary in prompts where factuality matters.
• Audit production outputs for invented specifics monthly (grep the filler list, sample 5 records).
• For repeated workflows, build the verification step into the pipeline as a separate call.
• Treat “completeness” as a separate concern from “correctness” — sometimes the incomplete output is the correct output.

Related reading

• AI hallucinated facts
• Output sounds polished but is not actionable
• Prompt lacks source hierarchy
• Negative constraints vague
• No success criteria

External references:

• OpenAI Structured Outputs guide
• Anthropic structured outputs docs

Tags: #Troubleshooting #Prompt #Prompt quality #Hallucination

Related Articles

Troubleshooting

Few-Shot Examples Have Uneven Quality and Drag Output Down

You gave the model 5 examples; 2 are great, 3 are mediocre, and it averages toward the mediocre ones. Why example variance hurts and how to curate down to 3-5 consistent ones.

May 24, 2026 #Prompt engineering #Troubleshooting

Troubleshooting

Model Returns Invalid JSON Because Schema Was Described, Not Enforced

You asked for JSON matching a schema. Most calls return valid JSON, some return prose with JSON inside, some omit fields. Description vs enforcement, and how to fix at the API layer.

May 24, 2026 #Prompt engineering #Troubleshooting

Troubleshooting

Model Invented Fake Citations and URLs

The model produced a citation like Smith et al. 2019 and the paper does not exist, or a URL that 404s. Why citation hallucination happens and how to stop it for good.

May 24, 2026 #Prompt engineering #Troubleshooting

Troubleshooting

Model Replies in the Wrong Language (How to Lock It)

You prompted in English and the model answered in Chinese, or it switched mid-output. The exact cause of language drift and the system-prompt + retry pattern that locks the output language, verified June 2026.

May 24, 2026 #Prompt engineering #Troubleshooting

Troubleshooting

LLM Response Cut Off Mid-Sentence: max_tokens Too Low (2026 Fix)

The model's reply ends mid-sentence, mid-JSON, or with an unclosed code block. It is almost always the token cap. How to size it, detect truncation per SDK, and recover.

May 24, 2026 #Prompt engineering #Troubleshooting

Troubleshooting

Prompt Asks for 10 Items, Model Returns 3 and Stops

You asked for 10 ideas and got 3, or 10 slots padded with filler. Why list-length prompts under-deliver, and the prompt + schema fixes that actually get you N distinct items.

May 24, 2026 #Prompt engineering #Troubleshooting