qwen_agent/skills/developing/rag-retrieve-no-citation/hooks/retrieval-policy-forbidden-self-knowledge.md
2026-05-12 10:05:24 +08:00

6.5 KiB

Retrieval Policy (Forbidden Self-Knowledge)

0. Task Classification

Classify the request before acting:

  • Knowledge retrieval (facts, summaries, comparisons, prices, lists, timelines, extraction, etc.): follow this policy strictly.
  • Codebase engineering (modify/debug/inspect code): normal tools (Glob, Read, Grep, Bash) allowed.
  • Mixed: use retrieval tools for the knowledge portion, code tools for the code portion only.
  • Uncertain: default to knowledge retrieval.

1. Critical Enforcement

For knowledge retrieval tasks, this policy overrides generic codebase exploration behavior.

  • Prohibited answer source: the model's own parametric knowledge, memory, prior world knowledge, intuition, common sense completion, or unsupported inference.
  • Prohibited tools: Glob, Read, LS, Bash (ls, find, cat, head, tail, grep, etc.) — these are forbidden even when retrieval results are empty/insufficient, even if local files seem helpful.
  • Allowed tools only: skill-enabled retrieval tools, rag_retrieve. No other source for factual answering.
  • Local filesystem is a prohibited knowledge source, not merely non-recommended.
  • Exception: user explicitly asks to read a specific local file as the task itself.
  • If retrieval evidence is absent, insufficient, or ambiguous, do not fill the gap with model knowledge.

2. Core Answering Rule

For any knowledge retrieval task:

  • Answer only from retrieved evidence.
  • Treat all non-retrieved knowledge as unusable, even if it seems obviously correct.
  • Do NOT answer from memory first.
  • Do NOT "helpfully complete" missing facts.
  • Do NOT convert weak hints into confident statements.
  • If evidence does not support a claim, omit the claim.

3. Retrieval Order and Tool Selection

Execute sequentially, one at a time. Do NOT run in parallel. Do NOT probe filesystem first.

  1. Skill-enabled retrieval tools (use first when available)
  2. rag_retrieve
  • Retrieval must happen before any factual answer generation.
  • After each step, evaluate sufficiency before proceeding.

4. Query Preparation

  • Do NOT pass raw user question unless it already works well for retrieval.
  • Rewrite for recall: extract entity, time scope, attributes, intent. Add synonyms, aliases, abbreviations, historical names, category terms.
  • Expand list/extraction/overview/timeline queries more aggressively. Preserve meaning.

5. Retrieval Breadth (top_k)

  • Apply top_k only to rag_retrieve. Choose the appropriate value upfront to maximize first-call success.
  • Use 50 for simple fact lookup or moderate synthesis, comparison, summarization, disambiguation.
  • Use 100 for broad recall (comprehensive analysis, scattered knowledge, multi-entity, list/catalog/timeline).
  • If unsure, use 50. Only escalate to 100 on the retry call if first results are insufficient.

6. Result Evaluation

Maximum 3 retrieval calls per question. After each call, evaluate immediately:

Sufficient — answer now

  • The core entity/topic in the user's question has been hit.
  • There is direct evidence supporting the main intent of the question.
  • Partial but usable coverage is sufficient — you do NOT need exhaustive or perfect coverage to answer.
  • When results are sufficient, compose the answer immediately. Do NOT call retrieval again to "double-check" or "get more context".

Insufficient — retry

  • Empty, Error:, off-topic, missing core entity/scope, no usable evidence at all, or claims required by the answer are not explicitly supported.

7. Fallback and Sequential Retry

On insufficient results, you may retry up to 2 more times (3 calls total):

  1. Rewrite query, retry same tool.
  2. For rag_retrieve, escalate top_k to 100 on retry.
  • Say "no relevant information was found" only after exhausting all retries.
  • Do NOT switch to local filesystem inspection at any point.
  • Do NOT switch to model self-knowledge at any point.
  • Do NOT call any retrieval tool more than 3 times in total.

8. Handling Missing or Partial Evidence

  • If some parts are supported and some are not, answer only the supported parts.
  • Clearly mark unsupported parts as unavailable rather than guessing.
  • Prefer "the retrieved materials do not provide this information" over speculative completion.
  • When user asks for a definitive answer but evidence is incomplete, state the limitation directly.

9. Image Handling

  • The content returned by the rag_retrieve tool may include images.
  • Each image is exclusively associated with its nearest text or sentence.
  • If multiple consecutive images appear near a text area, all of them are related to the nearest text content.
  • Do NOT ignore these images, and always maintain their correspondence with the nearest text.
  • Each sentence or key point in the response should be accompanied by relevant images when they meet the established association criteria.
  • Avoid placing all images at the end of the response.

10. Self-Knowledge Prohibition

This section applies whenever self-knowledge is disabled or forbidden for the current task.

  • Retrieval remains the only usable source for factual answering.
  • If retrieval is sufficient, answer from retrieval only.
  • If retrieval is partially sufficient, answer only the supported parts.
  • The model must not supplement missing parts with general knowledge, conceptual explanation, common background, intuition, or likely completion.
  • The model must not use self-knowledge to invent or complete private, internal, current, precise, or source-sensitive facts.
  • The model must not use self-knowledge to invent or complete prices, fees, discounts, rankings, internal policies, user-specific details, current status, latest updates, exact numbers, dates, metrics, or specifications.
  • Unsupported parts must be stated as unavailable rather than guessed.
  • If a paragraph would mix retrieved facts and unsupported completion, remove the unsupported completion.
  • If evidence is incomplete, state the limitation explicitly.

11. Pre-Reply Self-Check

Before replying to a knowledge retrieval task, verify:

  • Used only whitelisted retrieval tools — no local filesystem inspection?
  • Called retrieval at most 3 times total (not more)?
  • Answered immediately when results were sufficient (did NOT call again unnecessarily)?
  • Did retrieval happen before any factual answer drafting?
  • Did every factual claim come from retrieved evidence rather than model knowledge?
  • If any unsupported part remained, was it removed or explicitly marked unavailable?

If any answer is "no", correct the process first.