You build a router with three specialists — a code_agent, a test_agent, and a docs_agent. You submit “Write a unit test for the authentication module.” The router sends it to the docs agent, which writes a Markdown README about authentication. Or in an AutoGen setup, a “database migration” task routes to the general-purpose assistant instead of the migration specialist, and that agent runs ALTER TABLE directly on the production connection instead of generating a migration file. Routing misfires waste tokens, produce wrong output, and — in the worst cases — fire the wrong side effects with the wrong tools.
Fastest fix: log every routing decision with its confidence score, then add a confidence threshold (start at 0.75) that escalates low-confidence tasks to a clarification step instead of guessing. That single change catches the majority of misfires while you tighten the underlying agent descriptions. Everything below explains how to find which of the six root causes you actually have.
Which bucket are you in?
| Symptom you observe | Most likely cause | Jump to |
|---|---|---|
| Same two agents keep swapping for similar tasks | Overlapping category descriptions | Cause 1 |
| Misroutes use phrasing you never gave as an example | Sparse few-shot examples | Cause 2 |
| Routing flips when you change one word in the task | Keyword matching, not semantics | Cause 3 |
| Confident-but-wrong routes on genuinely ambiguous tasks | No “unsure” / default path | Cause 4 |
| A whole category of tasks never reaches its agent | Stale or invisible agent in the registry | Cause 5 |
| Short tasks (“fix it”) route randomly | Not enough signal in the task text | Cause 6 |
If you are on a framework, the failure usually has a framework-specific shape:
- LangGraph (
add_conditional_edges): a typo in the string your router function returns, or in apath_mapkey, silently sends the task to the wrong node or toEND. Returned strings and path-map keys must match exactly. (Graph API docs) - CrewAI (
Process.hierarchical): the auto-createdmanager_llmdecides routing from each agent’srole/goaltext. As of June 2026 this manager often executes tasks sequentially or calls the wrong worker; the common fix is a custommanager_agentwith explicit step-wise instructions, or switching toProcess.sequential/ pinningTask(agent=...). (CrewAI hierarchical docs) - AutoGen (
SelectorGroupChat): yourselector_funcfalls back to the model whenever it returnsNone, so a too-narrow function quietly hands selection back to the LLM. There is also a known intermittent bug whereselector_funcis not re-invoked after some turns. (Selector Group Chat docs)
Common causes
1. Router prompt is too vague — category descriptions overlap
The most common cause. You defined agent roles in natural language (“handles code tasks,” “handles test tasks”), but the router model must choose between them for ambiguous inputs. “Write a test” involves both code and tests. “Update the migration” involves both database and code. Overlapping category descriptions produce inconsistent routing.
How to spot it: Take the last 10 misrouted tasks and check which two agents’ descriptions are most similar. Any pair with a cosine similarity above 0.85 (when embedded) or overlapping keywords will produce consistent misrouting.
2. Few-shot examples in the router prompt are unrepresentative
The router has 3-4 example tasks per agent. Those examples all use specific terminology (“write a Jest test,” “create a Sequelize migration”). Real tasks use different phrasing (“add coverage for the login flow,” “bump the schema”). The model does not generalize from sparse examples to novel phrasing.
How to spot it: Collect 20 recent misrouted tasks and check whether any of them use phrasing similar to existing examples. If the misrouted tasks all use phrasing not in the examples, the example set is too narrow.
3. Router uses keyword matching instead of semantic classification
The router checks if "sql" in task.lower() and routes to the database agent. A task like “fix the SQL injection vulnerability in the auth layer” hits the database-agent keyword but should go to the security agent. Keyword matching cannot handle context.
How to spot it: Read the routing code. If it contains in task.lower(), startswith, re.match on keywords, or a simple if/elif chain, it is keyword-based and will misfire on context-dependent tasks.
4. Missing “default” or “ambiguous” route — router picks the closest wrong match
When no good match exists, the router routes to the first agent or the one with the highest softmax probability even when that probability is 0.52 vs. 0.48. There is no “I’m not sure” path that escalates to a human or asks for clarification.
How to spot it: Add confidence logging to the router. If routed decisions with confidence below 0.7 correlate with misrouted outputs, you need a low-confidence threshold.
5. Agent capability list is stale — agent was deprecated or renamed
The orchestrator’s routing table references agent_v2_code but the active agent is agent_v3_code_and_test. The v2 agent either no longer exists (routing fails silently and falls through to a default) or still exists but lacks recent capabilities (test writing was added in v3). In LangGraph this shows up as a path_map key that no longer matches any node name; in CrewAI as an agent that is in the crew but never selected.
How to spot it: List all agent IDs in the routing table and compare against the list of currently active agent instances. Any ID in the routing table that doesn’t match an active agent is stale.
6. Task description is too short — router lacks enough signal
“Fix it” — two words — gives the router nothing to work with. It routes by guessing, and guesses wrong. Short tasks often occur when the orchestrator summarizes a larger task before routing.
How to spot it: Check the median character length of misrouted task descriptions versus correctly routed ones. If misrouted tasks are significantly shorter (under 30 words), brevity is the cause.
Shortest path to fix
Step 1: Log every routing decision with the task text and confidence score
def route_task(task: str, router_model) -> tuple[str, float]:
response = router_model.classify(
task,
labels=list(AGENT_REGISTRY.keys()),
return_scores=True
)
top_agent = response.labels[0]
confidence = response.scores[0]
logger.info(
"ROUTE: agent=%s confidence=%.3f task=%r",
top_agent, confidence, task[:120]
)
return top_agent, confidenceReview the last 50 routing decisions to find patterns in misroutes. If you are on LangGraph, log the exact string your router function returns and compare it character-for-character against your path_map keys — a single typo is the most common silent misroute.
Step 2: Add a confidence threshold with an escalation path
CONFIDENCE_THRESHOLD = 0.75
def route_with_fallback(task: str) -> str:
agent, confidence = route_task(task, router_model)
if confidence < CONFIDENCE_THRESHOLD:
logger.warning(
"Low-confidence route (%.2f) — escalating to clarification agent",
confidence
)
return "clarification_agent"
return agentThe clarification agent asks one question to disambiguate, then re-routes with more context. In AutoGen, do this inside selector_func and return the clarification agent’s name explicitly; remember that returning None hands selection back to the model instead.
Step 3: Rewrite agent descriptions to be mutually exclusive
Replace vague descriptions with explicit scope boundaries:
AGENT_DESCRIPTIONS = {
"code_agent": (
"Writes, edits, or refactors production source code in .py, .ts, .go files. "
"Does NOT write tests, migration files, or documentation."
),
"test_agent": (
"Writes or edits test files (*.test.ts, test_*.py, *_spec.rb). "
"Does NOT edit production source files or migration files."
),
"migration_agent": (
"Generates database migration files using the project's migration framework. "
"Never runs migrations directly — only creates the migration file."
),
}The “Does NOT” clauses are as important as the “Does” clauses for preventing overlap. In CrewAI these go in each agent’s role and backstory; the manager_llm reads them when delegating.
Step 4: Expand few-shot examples to cover diverse phrasing
For each agent, add at least 10 examples that cover:
- Direct phrasing (“write a test for X”)
- Indirect phrasing (“add coverage for X”)
- Jargon variants (“spec for X,” “unit test for X,” “test case for X”)
- Cross-domain tasks that should NOT route here (“fix the code that X tests” should go to code_agent, not test_agent)
TEST_AGENT_EXAMPLES = [
"Write a unit test for the authentication module",
"Add test coverage for the payment flow",
"Create a spec for the UserService class",
"The login tests are failing — update the test assertions",
# Counter-examples (what NOT to route here):
# "Fix the authentication module so the tests pass" => code_agent
# "Write docs for the test suite" => docs_agent
]Step 5: Validate routing on a labeled evaluation set before deploying
ROUTING_EVAL = [
{"task": "Add a test for the JWT decoder", "expected": "test_agent"},
{"task": "Fix the JWT decoder implementation", "expected": "code_agent"},
{"task": "Document the JWT decoder API", "expected": "docs_agent"},
# ... 50+ examples
]
def evaluate_router(router):
correct = sum(
1 for ex in ROUTING_EVAL
if route_task(ex["task"], router)[0] == ex["expected"]
)
accuracy = correct / len(ROUTING_EVAL)
print(f"Router accuracy: {accuracy:.1%}")
assert accuracy >= 0.90, "Router accuracy below 90% threshold"Run this as a CI check whenever router prompts or agent descriptions change.
How to confirm it’s fixed
- Re-run
evaluate_routeron your labeled set; accuracy should be at or above0.90and the previously failing tasks should now pass. - Replay the last 50 production tasks through the new router and diff the chosen agent against the old log. Every former misroute should change; nothing previously correct should regress.
- Watch the low-confidence counter for one full day. If the share of tasks below
0.75confidence is small and they all land in the clarification path (not a wrong specialist), the escalation route is doing its job.
Prevention
- Define agent capabilities using explicit scope boundaries with “Does NOT handle” clauses — ambiguity in descriptions directly causes misrouting.
- Build a labeled routing evaluation set of at least 50 examples before deploying any router, and enforce a 90% accuracy threshold in CI.
- Log every routing decision with confidence score; alert on decisions below
0.75confidence. - Add a “clarification agent” or human escalation path for low-confidence routing rather than guessing.
- Version your agent registry; any time an agent is added, removed, or renamed, run the routing evaluation suite before deploying. On LangGraph, keep
path_mapkeys and node names in sync; on CrewAI, re-check the manager’s view of the crew. - Keep task descriptions sent to the router at least 20 words — add a task-enrichment step if the orchestrator generates short tasks.
- Use semantic classification (embedding similarity or a classifier model) rather than keyword matching for anything beyond trivial routing.
- Review misrouted tasks weekly in production; use them to expand the evaluation set and improve examples.
FAQ
Q: Should I use a dedicated router model or build routing into the orchestrator LLM? A: For 3-5 agents, building routing into the orchestrator prompt works well. For 10+ agents, use a dedicated lightweight classifier (a fine-tuned small model or embedding similarity) — the orchestrator’s general-purpose model degrades in accuracy as the number of choices grows.
Q: My CrewAI hierarchical crew keeps misrouting or running everything sequentially. What changed?
A: As of June 2026, CrewAI’s auto-created hierarchical manager frequently fails to coordinate as documented — it may execute tasks in order, make unnecessary tool calls, and route poorly. Define a custom manager_agent with explicit, step-wise delegation instructions, or use Process.sequential with Task(agent=...) pinned per task to bypass dynamic routing entirely. Expect the hierarchical manager to add roughly 30-50% more tokens than sequential mode.
Q: My LangGraph router function returns the right label but the task still goes to the wrong node.
A: The string your conditional-edge function returns must match a path_map key (or node name) exactly. A trailing space, a casing difference, or a renamed node sends the task to the wrong place or to END with no error. Add a type hint / explicit path_map so LangGraph can validate the destinations, and log the returned string next to the available keys.
Q: How do I handle tasks that legitimately belong to two agents? A: Split the task before routing. Add a “task decomposer” step that breaks composite tasks into atomic subtasks, each of which maps cleanly to one agent. Do not try to route a composite task to a single agent.
Q: Can vector-based routing replace prompt-based routing? A: Often, yes, especially for large agent registries. Embed each task and each agent’s capability description, then route to the agent with the highest cosine similarity. It is faster, cheaper, and more consistent than asking a large model to classify every task. The catch: semantic similarity is not the same as capability match (a “write a SQL injection report” task may embed closest to the docs agent), so still verify against your labeled eval set and gate on a minimum similarity score.