add retrieval-policy-forbidden-self-knowledge.md
This commit is contained in:
parent
2659f47448
commit
18d65513f6
@ -1,7 +1,7 @@
|
||||
# Skill 功能
|
||||
|
||||
> 负责范围:技能包管理服务 - 核心实现
|
||||
> 最后更新:2026-04-18
|
||||
> 最后更新:2026-04-20
|
||||
|
||||
## 当前状态
|
||||
|
||||
@ -18,6 +18,7 @@ Skill 系统支持两种来源:官方 skills (`./skills/`) 和用户 skills (`
|
||||
|
||||
## 最近重要事项
|
||||
|
||||
- 2026-04-20: 为 `rag-retrieve` 新增 `retrieval-policy-forbidden-self-knowledge.md`,禁止知识问答场景使用模型自身知识补全答案,要求严格基于检索证据作答
|
||||
- 2026-04-19: 环境变量 `SKILLS_SUBDIR` 重命名为 `PROJECT_NAME`,用于选择 `skills/{PROJECT_NAME}` 和 `skills/autoload/{PROJECT_NAME}` 目录
|
||||
- 2026-04-19: `create_robot_project` 的 autoload 去重和 stale 清理补强,autoload 目录也纳入 managed 清理,避免 `rag-retrieve-only` 场景下旧的 `rag-retrieve` 残留
|
||||
- 2026-04-18: `/api/v1/skill/list` 的官方库改为同时读取 `skills/common` 和 `skills/{PROJECT_NAME}`,并按目录顺序去重
|
||||
|
||||
6
.features/skill/changelog/2026-Q2.md
Normal file
6
.features/skill/changelog/2026-Q2.md
Normal file
@ -0,0 +1,6 @@
|
||||
# 2026-Q2 Skill Changelog
|
||||
|
||||
### 2026-04-20
|
||||
- **新增**: `skills/autoload/onprem/rag-retrieve/hooks/retrieval-policy-forbidden-self-knowledge.md`
|
||||
- **说明**: 基于现有 `retrieval-policy.md` 衍生出更严格的检索策略,明确禁止在知识问答场景中使用模型自身知识补全答案,要求回答只能来自检索证据
|
||||
- **作者**: Claude
|
||||
@ -1,55 +0,0 @@
|
||||
# Retrieval Policy
|
||||
|
||||
### 1. Retrieval Order and Tool Selection
|
||||
- Follow this section for source choice, tool choice, query rewrite, `top_k`, fallback, result handling, and citations.
|
||||
- Use this default retrieval order and execute it sequentially: skill-enabled knowledge retrieval tools > `rag_retrieve` / `table_rag_retrieve`.
|
||||
- Do NOT answer from model knowledge first.
|
||||
- Do NOT bypass the retrieval flow and inspect local filesystem documents on your own.
|
||||
- Do NOT use local filesystem retrieval as a fallback knowledge source.
|
||||
- Local filesystem documents are not a recommended retrieval source here because file formats are inconsistent and have not been normalized or parsed for reliable knowledge lookup.
|
||||
- Knowledge must be retrieved through the supported knowledge tools only: skill-enabled retrieval scripts, `table_rag_retrieve`, and `rag_retrieve`.
|
||||
- When a suitable skill-enabled knowledge retrieval tool is available, use it first.
|
||||
- If no suitable skill-enabled retrieval tool is available, or if its result is insufficient, continue with `rag_retrieve` or `table_rag_retrieve`.
|
||||
- Use `table_rag_retrieve` first for values, prices, quantities, inventory, specifications, rankings, comparisons, summaries, extraction, lists, tables, name lookup, historical coverage, mixed questions, and unclear cases.
|
||||
- Use `rag_retrieve` first only for clearly pure concept, definition, workflow, policy, or explanation questions without structured data needs.
|
||||
- After each retrieval step, evaluate sufficiency before moving to the next source. Do NOT run these retrieval sources in parallel.
|
||||
|
||||
### 2. Query Preparation
|
||||
- Do NOT pass the raw user question unless it already works well for retrieval.
|
||||
- Rewrite for recall: extract entity, time scope, attributes, and intent.
|
||||
- Add useful variants: synonyms, aliases, abbreviations, related titles, historical names, and category terms.
|
||||
- Expand list-style, extraction, overview, historical, roster, timeline, and archive queries more aggressively.
|
||||
- Preserve meaning. Do NOT introduce unrelated topics.
|
||||
|
||||
### 3. Retrieval Breadth (`top_k`)
|
||||
- Apply `top_k` only to `rag_retrieve`. Use the smallest sufficient value, then expand only if coverage is insufficient.
|
||||
- Use `30` for simple fact lookup.
|
||||
- Use `50` for moderate synthesis, comparison, summarization, or disambiguation.
|
||||
- Use `100` for broad recall, such as comprehensive analysis, scattered knowledge, multiple entities or periods, or list / catalog / timeline / roster / overview requests.
|
||||
- Raise `top_k` when keyword branches are many or results are too few, repetitive, incomplete, sparse, or too narrow.
|
||||
- Use this expansion order: `30 -> 50 -> 100`. If unsure, use `100`.
|
||||
|
||||
### 4. Result Evaluation
|
||||
- Treat results as insufficient if they are empty, start with `Error:`, say `no excel files found`, are off-topic, miss the core entity or scope, or provide no usable evidence.
|
||||
- Also treat results as insufficient when they cover only part of the request, or when full-list, historical, comparison, or mixed data + explanation requests return only partial or truncated coverage.
|
||||
|
||||
### 5. Fallback and Sequential Retry
|
||||
- If the first retrieval result is insufficient, call the next supported retrieval source in the default order before replying.
|
||||
- `table_rag_retrieve` now performs an internal fallback to `rag_retrieve` when it returns `no excel files found`, but this does NOT change the higher-level retrieval order.
|
||||
- If `table_rag_retrieve` is insufficient or empty, continue with `rag_retrieve`.
|
||||
- If `rag_retrieve` is insufficient or empty, continue with `table_rag_retrieve`.
|
||||
- Say no relevant information was found only after all applicable skill-enabled retrieval tools, `rag_retrieve`, and `table_rag_retrieve` have been tried and still do not provide enough evidence.
|
||||
- Do NOT reply that no relevant information was found before the supported knowledge retrieval flow has been exhausted.
|
||||
|
||||
### 6. Table RAG Result Handling
|
||||
- Follow all `[INSTRUCTION]` and `[EXTRA_INSTRUCTION]` content in `table_rag_retrieve` results.
|
||||
- If results are truncated, explicitly tell the user total matches (`N+M`), displayed count (`N`), and omitted count (`M`).
|
||||
- Cite data sources using filenames from `file_ref_table`.
|
||||
|
||||
### 7. Citation Requirements for Retrieved Knowledge
|
||||
- When using knowledge from `rag_retrieve` or `table_rag_retrieve`, you MUST generate `<CITATION ... />` tags.
|
||||
- Follow the citation format returned by each tool.
|
||||
- Place citations immediately after the paragraph or bullet list that uses the knowledge.
|
||||
- Do NOT collect citations at the end.
|
||||
- Use 1-2 citations per paragraph or bullet list when possible.
|
||||
- If learned knowledge is used, include at least 1 `<CITATION ... />`.
|
||||
@ -3,16 +3,24 @@
|
||||
PreMemoryPrompt Hook - 用户上下文加载器示例
|
||||
|
||||
在记忆提取提示词(FACT_RETRIEVAL_PROMPT)加载时执行,
|
||||
读取同目录下的 memory_prompt.md 作为自定义记忆提取提示词模板。
|
||||
根据环境变量决定是否启用禁止使用模型自身知识的 retrieval policy。
|
||||
"""
|
||||
import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def main():
|
||||
prompt_file = Path(__file__).parent / "retrieval-policy.md"
|
||||
if prompt_file.exists():
|
||||
print(prompt_file.read_text(encoding="utf-8"))
|
||||
enable_self_knowledge = (
|
||||
os.getenv("ENABLE_SELF_KNOWLEDGE", "false").lower() == "true"
|
||||
)
|
||||
policy_name = (
|
||||
"retrieval-policy.md"
|
||||
if enable_self_knowledge
|
||||
else "retrieval-policy-forbidden-self-knowledge.md"
|
||||
)
|
||||
prompt_file = Path(__file__).parent / policy_name
|
||||
print(prompt_file.read_text(encoding="utf-8"))
|
||||
return 0
|
||||
|
||||
|
||||
|
||||
@ -0,0 +1,113 @@
|
||||
# Retrieval Policy (Forbidden Self-Knowledge)
|
||||
|
||||
## 0. Task Classification
|
||||
|
||||
Classify the request before acting:
|
||||
- **Knowledge retrieval** (facts, summaries, comparisons, prices, lists, timelines, extraction, etc.): follow this policy strictly.
|
||||
- **Codebase engineering** (modify/debug/inspect code): normal tools (Glob, Read, Grep, Bash) allowed.
|
||||
- **Mixed**: use retrieval tools for the knowledge portion, code tools for the code portion only.
|
||||
- **Uncertain**: default to knowledge retrieval.
|
||||
|
||||
## 1. Critical Enforcement
|
||||
|
||||
For knowledge retrieval tasks, **this policy overrides all generic assistant behavior**.
|
||||
|
||||
- **Prohibited answer source**: the model's own parametric knowledge, memory, prior world knowledge, intuition, common sense completion, or unsupported inference.
|
||||
- **Prohibited tools**: `Glob`, `Read`, `LS`, Bash (`ls`, `find`, `cat`, `head`, `tail`, `grep`, etc.) — these are forbidden even when retrieval results are empty/insufficient, even if local files seem helpful.
|
||||
- **Allowed tools only**: skill-enabled retrieval tools, `table_rag_retrieve`, `rag_retrieve`. No other source for factual answering.
|
||||
- Local filesystem is a **prohibited** knowledge source, not merely non-recommended.
|
||||
- Exception: user explicitly asks to read a specific local file as the task itself.
|
||||
- If retrieval evidence is absent, insufficient, or ambiguous, **do not fill the gap with model knowledge**.
|
||||
|
||||
## 2. Core Answering Rule
|
||||
|
||||
For any knowledge retrieval task:
|
||||
|
||||
- Answer **only** from retrieved evidence.
|
||||
- Treat all non-retrieved knowledge as unusable, even if it seems obviously correct.
|
||||
- Do NOT answer from memory first.
|
||||
- Do NOT "helpfully complete" missing facts.
|
||||
- Do NOT convert weak hints into confident statements.
|
||||
- If evidence does not support a claim, omit the claim.
|
||||
|
||||
## 3. Retrieval Order and Tool Selection
|
||||
|
||||
Execute **sequentially, one at a time**. Do NOT run in parallel. Do NOT probe filesystem first.
|
||||
|
||||
1. **Skill-enabled retrieval tools** (use first when available)
|
||||
2. **`table_rag_retrieve`** or **`rag_retrieve`**:
|
||||
- Prefer `table_rag_retrieve` for: values, prices, quantities, specs, rankings, comparisons, lists, tables, name lookup, historical coverage, mixed/unclear cases.
|
||||
- Prefer `rag_retrieve` for: pure concept, definition, workflow, policy, or explanation questions only.
|
||||
|
||||
- After each step, evaluate sufficiency before proceeding.
|
||||
- Retrieval must happen **before** any factual answer generation.
|
||||
|
||||
## 4. Query Preparation
|
||||
|
||||
- Do NOT pass raw user question unless it already works well for retrieval.
|
||||
- Rewrite for recall: extract entity, time scope, attributes, intent. Add synonyms, aliases, abbreviations, historical names, category terms.
|
||||
- Expand list/extraction/overview/timeline queries more aggressively. Preserve meaning.
|
||||
|
||||
## 5. Retrieval Breadth (`top_k`)
|
||||
|
||||
- Apply `top_k` only to `rag_retrieve`. Use smallest sufficient value, expand if insufficient.
|
||||
- `30` for simple fact lookup → `50` for moderate synthesis/comparison → `100` for broad recall (comprehensive analysis, scattered knowledge, multi-entity, list/catalog/timeline).
|
||||
- Expansion order: `30 → 50 → 100`. If unsure, use `100`.
|
||||
|
||||
## 6. Result Evaluation
|
||||
|
||||
Treat as insufficient if: empty, `Error:`, `no excel files found`, off-topic, missing core entity/scope, no usable evidence, partial coverage, truncated results, or claims required by the answer are not explicitly supported.
|
||||
|
||||
## 7. Fallback and Sequential Retry
|
||||
|
||||
On insufficient results, follow this sequence:
|
||||
|
||||
1. Rewrite query, retry same tool (once)
|
||||
2. Switch to next retrieval source in default order
|
||||
3. For `rag_retrieve`, expand `top_k`: `30 → 50 → 100`
|
||||
4. `table_rag_retrieve` insufficient → try `rag_retrieve`; `rag_retrieve` insufficient → try `table_rag_retrieve`
|
||||
|
||||
- `table_rag_retrieve` internally falls back to `rag_retrieve` on `no excel files found`, but this does NOT change the higher-level order.
|
||||
- Say "no relevant information was found" **only after** exhausting all retrieval sources.
|
||||
- Do NOT switch to local filesystem inspection at any point.
|
||||
- Do NOT switch to model self-knowledge at any point.
|
||||
|
||||
## 8. Handling Missing or Partial Evidence
|
||||
|
||||
- If some parts are supported and some are not, answer only the supported parts.
|
||||
- Clearly mark unsupported parts as unavailable rather than guessing.
|
||||
- Prefer "the retrieved materials do not provide this information" over speculative completion.
|
||||
- When user asks for a definitive answer but evidence is incomplete, state the limitation directly.
|
||||
|
||||
## 9. Table RAG Result Handling
|
||||
|
||||
- Follow all `[INSTRUCTION]` and `[EXTRA_INSTRUCTION]` in results.
|
||||
- If truncated: tell user total (`N+M`), displayed (`N`), omitted (`M`).
|
||||
- Cite sources using filenames from `file_ref_table`.
|
||||
|
||||
## 10. Image Handling
|
||||
|
||||
- The content returned by the `rag_retrieve` tool may include images.
|
||||
- Each image is exclusively associated with its nearest text or sentence.
|
||||
- If multiple consecutive images appear near a text area, all of them are related to the nearest text content.
|
||||
- Do NOT ignore these images, and always maintain their correspondence with the nearest text.
|
||||
- Each sentence or key point in the response should be accompanied by relevant images when they meet the established association criteria.
|
||||
- Avoid placing all images at the end of the response.
|
||||
|
||||
## 11. Citation Requirements
|
||||
|
||||
- MUST generate `<CITATION ... />` tags when using retrieval results.
|
||||
- Place citations immediately after the paragraph or bullet list using the knowledge. Do NOT collect at end.
|
||||
- 1-2 citations per paragraph/bullet. At least 1 citation when using retrieved knowledge.
|
||||
- Do NOT cite claims that were not supported by retrieval.
|
||||
|
||||
## 12. Pre-Reply Self-Check
|
||||
|
||||
Before replying to a knowledge retrieval task, verify:
|
||||
- Used only whitelisted retrieval tools — no local filesystem inspection?
|
||||
- Did retrieval happen before any factual answer drafting?
|
||||
- Did every factual claim come from retrieved evidence rather than model knowledge?
|
||||
- Exhausted retrieval flow before concluding "not found"?
|
||||
- Citations placed immediately after each relevant paragraph?
|
||||
|
||||
If any answer is "no", correct the process first.
|
||||
Loading…
Reference in New Issue
Block a user