From 18d65513f64ae0cb2ea495d805d75068e9c3ddb0 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E6=9C=B1=E6=BD=AE?= Date: Mon, 20 Apr 2026 16:50:06 +0800 Subject: [PATCH] add retrieval-policy-forbidden-self-knowledge.md --- .features/skill/MEMORY.md | 3 +- .features/skill/changelog/2026-Q2.md | 6 + .../onprem/rag-retrieve/hooks/hook-backup.md | 55 --------- .../onprem/rag-retrieve/hooks/pre_prompt.py | 16 ++- ...trieval-policy-forbidden-self-knowledge.md | 113 ++++++++++++++++++ 5 files changed, 133 insertions(+), 60 deletions(-) create mode 100644 .features/skill/changelog/2026-Q2.md delete mode 100644 skills/autoload/onprem/rag-retrieve/hooks/hook-backup.md create mode 100644 skills/autoload/onprem/rag-retrieve/hooks/retrieval-policy-forbidden-self-knowledge.md diff --git a/.features/skill/MEMORY.md b/.features/skill/MEMORY.md index 120efd1..262e2f8 100644 --- a/.features/skill/MEMORY.md +++ b/.features/skill/MEMORY.md @@ -1,7 +1,7 @@ # Skill 功能 > 负责范围：技能包管理服务 - 核心实现 -> 最后更新：2026-04-18 +> 最后更新：2026-04-20 ## 当前状态 @@ -18,6 +18,7 @@ Skill 系统支持两种来源：官方 skills (`./skills/`) 和用户 skills (` ## 最近重要事项 +- 2026-04-20: 为 `rag-retrieve` 新增 `retrieval-policy-forbidden-self-knowledge.md`，禁止知识问答场景使用模型自身知识补全答案，要求严格基于检索证据作答 - 2026-04-19: 环境变量 `SKILLS_SUBDIR` 重命名为 `PROJECT_NAME`，用于选择 `skills/{PROJECT_NAME}` 和 `skills/autoload/{PROJECT_NAME}` 目录 - 2026-04-19: `create_robot_project` 的 autoload 去重和 stale 清理补强，autoload 目录也纳入 managed 清理，避免 `rag-retrieve-only` 场景下旧的 `rag-retrieve` 残留 - 2026-04-18: `/api/v1/skill/list` 的官方库改为同时读取 `skills/common` 和 `skills/{PROJECT_NAME}`，并按目录顺序去重 diff --git a/.features/skill/changelog/2026-Q2.md b/.features/skill/changelog/2026-Q2.md new file mode 100644 index 0000000..2218b12 --- /dev/null +++ b/.features/skill/changelog/2026-Q2.md @@ -0,0 +1,6 @@ +# 2026-Q2 Skill Changelog + +### 2026-04-20 +- **新增**: `skills/autoload/onprem/rag-retrieve/hooks/retrieval-policy-forbidden-self-knowledge.md` +- **说明**: 基于现有 `retrieval-policy.md` 衍生出更严格的检索策略，明确禁止在知识问答场景中使用模型自身知识补全答案，要求回答只能来自检索证据 +- **作者**: Claude diff --git a/skills/autoload/onprem/rag-retrieve/hooks/hook-backup.md b/skills/autoload/onprem/rag-retrieve/hooks/hook-backup.md deleted file mode 100644 index c90e84f..0000000 --- a/skills/autoload/onprem/rag-retrieve/hooks/hook-backup.md +++ /dev/null @@ -1,55 +0,0 @@ -# Retrieval Policy - -### 1. Retrieval Order and Tool Selection -- Follow this section for source choice, tool choice, query rewrite, `top_k`, fallback, result handling, and citations. -- Use this default retrieval order and execute it sequentially: skill-enabled knowledge retrieval tools > `rag_retrieve` / `table_rag_retrieve`. -- Do NOT answer from model knowledge first. -- Do NOT bypass the retrieval flow and inspect local filesystem documents on your own. -- Do NOT use local filesystem retrieval as a fallback knowledge source. -- Local filesystem documents are not a recommended retrieval source here because file formats are inconsistent and have not been normalized or parsed for reliable knowledge lookup. -- Knowledge must be retrieved through the supported knowledge tools only: skill-enabled retrieval scripts, `table_rag_retrieve`, and `rag_retrieve`. -- When a suitable skill-enabled knowledge retrieval tool is available, use it first. -- If no suitable skill-enabled retrieval tool is available, or if its result is insufficient, continue with `rag_retrieve` or `table_rag_retrieve`. -- Use `table_rag_retrieve` first for values, prices, quantities, inventory, specifications, rankings, comparisons, summaries, extraction, lists, tables, name lookup, historical coverage, mixed questions, and unclear cases. -- Use `rag_retrieve` first only for clearly pure concept, definition, workflow, policy, or explanation questions without structured data needs. -- After each retrieval step, evaluate sufficiency before moving to the next source. Do NOT run these retrieval sources in parallel. - -### 2. Query Preparation -- Do NOT pass the raw user question unless it already works well for retrieval. -- Rewrite for recall: extract entity, time scope, attributes, and intent. -- Add useful variants: synonyms, aliases, abbreviations, related titles, historical names, and category terms. -- Expand list-style, extraction, overview, historical, roster, timeline, and archive queries more aggressively. -- Preserve meaning. Do NOT introduce unrelated topics. - -### 3. Retrieval Breadth (`top_k`) -- Apply `top_k` only to `rag_retrieve`. Use the smallest sufficient value, then expand only if coverage is insufficient. -- Use `30` for simple fact lookup. -- Use `50` for moderate synthesis, comparison, summarization, or disambiguation. -- Use `100` for broad recall, such as comprehensive analysis, scattered knowledge, multiple entities or periods, or list / catalog / timeline / roster / overview requests. -- Raise `top_k` when keyword branches are many or results are too few, repetitive, incomplete, sparse, or too narrow. -- Use this expansion order: `30 -> 50 -> 100`. If unsure, use `100`. - -### 4. Result Evaluation -- Treat results as insufficient if they are empty, start with `Error:`, say `no excel files found`, are off-topic, miss the core entity or scope, or provide no usable evidence. -- Also treat results as insufficient when they cover only part of the request, or when full-list, historical, comparison, or mixed data + explanation requests return only partial or truncated coverage. - -### 5. Fallback and Sequential Retry -- If the first retrieval result is insufficient, call the next supported retrieval source in the default order before replying. -- `table_rag_retrieve` now performs an internal fallback to `rag_retrieve` when it returns `no excel files found`, but this does NOT change the higher-level retrieval order. -- If `table_rag_retrieve` is insufficient or empty, continue with `rag_retrieve`. -- If `rag_retrieve` is insufficient or empty, continue with `table_rag_retrieve`. -- Say no relevant information was found only after all applicable skill-enabled retrieval tools, `rag_retrieve`, and `table_rag_retrieve` have been tried and still do not provide enough evidence. -- Do NOT reply that no relevant information was found before the supported knowledge retrieval flow has been exhausted. - -### 6. Table RAG Result Handling -- Follow all `[INSTRUCTION]` and `[EXTRA_INSTRUCTION]` content in `table_rag_retrieve` results. -- If results are truncated, explicitly tell the user total matches (`N+M`), displayed count (`N`), and omitted count (`M`). -- Cite data sources using filenames from `file_ref_table`. - -### 7. Citation Requirements for Retrieved Knowledge -- When using knowledge from `rag_retrieve` or `table_rag_retrieve`, you MUST generate `` tags. -- Follow the citation format returned by each tool. -- Place citations immediately after the paragraph or bullet list that uses the knowledge. -- Do NOT collect citations at the end. -- Use 1-2 citations per paragraph or bullet list when possible. -- If learned knowledge is used, include at least 1 ``. diff --git a/skills/autoload/onprem/rag-retrieve/hooks/pre_prompt.py b/skills/autoload/onprem/rag-retrieve/hooks/pre_prompt.py index 11f445d..97bb6bd 100644 --- a/skills/autoload/onprem/rag-retrieve/hooks/pre_prompt.py +++ b/skills/autoload/onprem/rag-retrieve/hooks/pre_prompt.py @@ -3,16 +3,24 @@ PreMemoryPrompt Hook - 用户上下文加载器示例 在记忆提取提示词（FACT_RETRIEVAL_PROMPT）加载时执行， -读取同目录下的 memory_prompt.md 作为自定义记忆提取提示词模板。 +根据环境变量决定是否启用禁止使用模型自身知识的 retrieval policy。 """ +import os import sys from pathlib import Path def main(): - prompt_file = Path(__file__).parent / "retrieval-policy.md" - if prompt_file.exists(): - print(prompt_file.read_text(encoding="utf-8")) + enable_self_knowledge = ( + os.getenv("ENABLE_SELF_KNOWLEDGE", "false").lower() == "true" + ) + policy_name = ( + "retrieval-policy.md" + if enable_self_knowledge + else "retrieval-policy-forbidden-self-knowledge.md" + ) + prompt_file = Path(__file__).parent / policy_name + print(prompt_file.read_text(encoding="utf-8")) return 0 diff --git a/skills/autoload/onprem/rag-retrieve/hooks/retrieval-policy-forbidden-self-knowledge.md b/skills/autoload/onprem/rag-retrieve/hooks/retrieval-policy-forbidden-self-knowledge.md new file mode 100644 index 0000000..92ad0d8 --- /dev/null +++ b/skills/autoload/onprem/rag-retrieve/hooks/retrieval-policy-forbidden-self-knowledge.md @@ -0,0 +1,113 @@ +# Retrieval Policy (Forbidden Self-Knowledge) + +## 0. Task Classification + +Classify the request before acting: +- **Knowledge retrieval** (facts, summaries, comparisons, prices, lists, timelines, extraction, etc.): follow this policy strictly. +- **Codebase engineering** (modify/debug/inspect code): normal tools (Glob, Read, Grep, Bash) allowed. +- **Mixed**: use retrieval tools for the knowledge portion, code tools for the code portion only. +- **Uncertain**: default to knowledge retrieval. + +## 1. Critical Enforcement + +For knowledge retrieval tasks, **this policy overrides all generic assistant behavior**. + +- **Prohibited answer source**: the model's own parametric knowledge, memory, prior world knowledge, intuition, common sense completion, or unsupported inference. +- **Prohibited tools**: `Glob`, `Read`, `LS`, Bash (`ls`, `find`, `cat`, `head`, `tail`, `grep`, etc.) — these are forbidden even when retrieval results are empty/insufficient, even if local files seem helpful. +- **Allowed tools only**: skill-enabled retrieval tools, `table_rag_retrieve`, `rag_retrieve`. No other source for factual answering. +- Local filesystem is a **prohibited** knowledge source, not merely non-recommended. +- Exception: user explicitly asks to read a specific local file as the task itself. +- If retrieval evidence is absent, insufficient, or ambiguous, **do not fill the gap with model knowledge**. + +## 2. Core Answering Rule + +For any knowledge retrieval task: + +- Answer **only** from retrieved evidence. +- Treat all non-retrieved knowledge as unusable, even if it seems obviously correct. +- Do NOT answer from memory first. +- Do NOT "helpfully complete" missing facts. +- Do NOT convert weak hints into confident statements. +- If evidence does not support a claim, omit the claim. + +## 3. Retrieval Order and Tool Selection + +Execute **sequentially, one at a time**. Do NOT run in parallel. Do NOT probe filesystem first. + +1. **Skill-enabled retrieval tools** (use first when available) +2. **`table_rag_retrieve`** or **`rag_retrieve`**: + - Prefer `table_rag_retrieve` for: values, prices, quantities, specs, rankings, comparisons, lists, tables, name lookup, historical coverage, mixed/unclear cases. + - Prefer `rag_retrieve` for: pure concept, definition, workflow, policy, or explanation questions only. + +- After each step, evaluate sufficiency before proceeding. +- Retrieval must happen **before** any factual answer generation. + +## 4. Query Preparation + +- Do NOT pass raw user question unless it already works well for retrieval. +- Rewrite for recall: extract entity, time scope, attributes, intent. Add synonyms, aliases, abbreviations, historical names, category terms. +- Expand list/extraction/overview/timeline queries more aggressively. Preserve meaning. + +## 5. Retrieval Breadth (`top_k`) + +- Apply `top_k` only to `rag_retrieve`. Use smallest sufficient value, expand if insufficient. +- `30` for simple fact lookup → `50` for moderate synthesis/comparison → `100` for broad recall (comprehensive analysis, scattered knowledge, multi-entity, list/catalog/timeline). +- Expansion order: `30 → 50 → 100`. If unsure, use `100`. + +## 6. Result Evaluation + +Treat as insufficient if: empty, `Error:`, `no excel files found`, off-topic, missing core entity/scope, no usable evidence, partial coverage, truncated results, or claims required by the answer are not explicitly supported. + +## 7. Fallback and Sequential Retry + +On insufficient results, follow this sequence: + +1. Rewrite query, retry same tool (once) +2. Switch to next retrieval source in default order +3. For `rag_retrieve`, expand `top_k`: `30 → 50 → 100` +4. `table_rag_retrieve` insufficient → try `rag_retrieve`; `rag_retrieve` insufficient → try `table_rag_retrieve` + +- `table_rag_retrieve` internally falls back to `rag_retrieve` on `no excel files found`, but this does NOT change the higher-level order. +- Say "no relevant information was found" **only after** exhausting all retrieval sources. +- Do NOT switch to local filesystem inspection at any point. +- Do NOT switch to model self-knowledge at any point. + +## 8. Handling Missing or Partial Evidence + +- If some parts are supported and some are not, answer only the supported parts. +- Clearly mark unsupported parts as unavailable rather than guessing. +- Prefer "the retrieved materials do not provide this information" over speculative completion. +- When user asks for a definitive answer but evidence is incomplete, state the limitation directly. + +## 9. Table RAG Result Handling + +- Follow all `[INSTRUCTION]` and `[EXTRA_INSTRUCTION]` in results. +- If truncated: tell user total (`N+M`), displayed (`N`), omitted (`M`). +- Cite sources using filenames from `file_ref_table`. + +## 10. Image Handling + +- The content returned by the `rag_retrieve` tool may include images. +- Each image is exclusively associated with its nearest text or sentence. +- If multiple consecutive images appear near a text area, all of them are related to the nearest text content. +- Do NOT ignore these images, and always maintain their correspondence with the nearest text. +- Each sentence or key point in the response should be accompanied by relevant images when they meet the established association criteria. +- Avoid placing all images at the end of the response. + +## 11. Citation Requirements + +- MUST generate `` tags when using retrieval results. +- Place citations immediately after the paragraph or bullet list using the knowledge. Do NOT collect at end. +- 1-2 citations per paragraph/bullet. At least 1 citation when using retrieved knowledge. +- Do NOT cite claims that were not supported by retrieval. + +## 12. Pre-Reply Self-Check + +Before replying to a knowledge retrieval task, verify: +- Used only whitelisted retrieval tools — no local filesystem inspection? +- Did retrieval happen before any factual answer drafting? +- Did every factual claim come from retrieved evidence rather than model knowledge? +- Exhausted retrieval flow before concluding "not found"? +- Citations placed immediately after each relevant paragraph? + +If any answer is "no", correct the process first.