diff --git a/skills/autoload/onprem/rag-retrieve/hooks/retrieval-policy-forbidden-self-knowledge.md b/skills/autoload/onprem/rag-retrieve/hooks/retrieval-policy-forbidden-self-knowledge.md index 2dfa82e..7ea5e4b 100644 --- a/skills/autoload/onprem/rag-retrieve/hooks/retrieval-policy-forbidden-self-knowledge.md +++ b/skills/autoload/onprem/rag-retrieve/hooks/retrieval-policy-forbidden-self-knowledge.md @@ -40,6 +40,13 @@ Execute **sequentially, one at a time**. Do NOT run in parallel. Do NOT probe fi - After each step, evaluate sufficiency before proceeding. - Retrieval must happen **before** any factual answer generation. +### First-Call Success Principle + +- The first retrieval call is expected to return sufficient results for most questions. +- Your default assumption should be: **one call is enough**. +- Additional calls are the exception, not the norm. Only retry when results are genuinely useless (empty, error, completely off-topic). +- **Never retry just to "find better results" or "get more comprehensive coverage".** Good enough is sufficient. + ## 4. Query Preparation - Do NOT pass raw user question unless it already works well for retrieval. @@ -57,14 +64,30 @@ Execute **sequentially, one at a time**. Do NOT run in parallel. Do NOT probe fi **Maximum 3 retrieval calls per question.** After each call, evaluate immediately: -### Sufficient — answer now -- The core entity/topic in the user's question has been hit. -- There is direct evidence supporting the main intent of the question. -- Partial but usable coverage is sufficient — you do NOT need exhaustive or perfect coverage to answer. -- **When results are sufficient, compose the answer immediately. Do NOT call retrieval again to "double-check" or "get more context".** +### Sufficient — answer immediately, no more calls -### Insufficient — retry -- Empty, `Error:`, off-topic, missing core entity/scope, no usable evidence at all, or claims required by the answer are not explicitly supported. +ANY of the following means results are sufficient — STOP and answer now: +- The core entity/topic in the user's question appears in the results. +- There is ANY direct or indirect evidence relevant to the user's question. +- Results are partially relevant, even if not perfectly comprehensive. +- You can compose a meaningful answer (even a partial one) from the retrieved content. + +**Anti-patterns — do NOT do these:** +- ❌ "The results are good, but maybe different keywords could find something better." +- ❌ "I have enough to answer, but let me try one more query to be thorough." +- ❌ "The answer is here, but I want to double-check with a different query." +- ❌ Calling retrieval again after you have already identified the answer in previous results. + +**If you can answer the question with current results, you MUST answer immediately. Period.** + +### Insufficient — the ONLY valid reasons to retry + +- Results are completely empty or contain only `Error:` messages. +- ALL results are entirely off-topic with zero relevance to the user's question. +- No usable evidence exists at all — you cannot form even a partial answer. + +**"Results are not detailed enough" is NOT a valid reason to retry.** +**"Results might be incomplete" is NOT a valid reason to retry.** ## 7. Fallback and Sequential Retry @@ -114,6 +137,7 @@ Before replying to a knowledge retrieval task, verify: - Used only whitelisted retrieval tools — no local filesystem inspection? - Called retrieval at most 3 times total (not more)? - Answered immediately when results were sufficient (did NOT call again unnecessarily)? +- Called retrieval exactly once when first results were sufficient (did NOT retry unnecessarily)? - Did retrieval happen before any factual answer drafting? - Did every factual claim come from retrieved evidence rather than model knowledge? - If any unsupported part remained, was it removed or explicitly marked unavailable? diff --git a/skills/autoload/onprem/rag-retrieve/hooks/retrieval-policy.md b/skills/autoload/onprem/rag-retrieve/hooks/retrieval-policy.md index 95ccc00..61378ff 100644 --- a/skills/autoload/onprem/rag-retrieve/hooks/retrieval-policy.md +++ b/skills/autoload/onprem/rag-retrieve/hooks/retrieval-policy.md @@ -29,6 +29,13 @@ Execute **sequentially, one at a time**. Do NOT run in parallel. Do NOT probe fi - Do NOT answer from model knowledge first. - After each step, evaluate sufficiency before proceeding. +### First-Call Success Principle + +- The first retrieval call is expected to return sufficient results for most questions. +- Your default assumption should be: **one call is enough**. +- Additional calls are the exception, not the norm. Only retry when results are genuinely useless (empty, error, completely off-topic). +- **Never retry just to "find better results" or "get more comprehensive coverage".** Good enough is sufficient. + ## 3. Query Preparation - Do NOT pass raw user question unless it already works well for retrieval. @@ -46,14 +53,30 @@ Execute **sequentially, one at a time**. Do NOT run in parallel. Do NOT probe fi **Maximum 3 retrieval calls per question.** After each call, evaluate immediately: -### Sufficient — answer now -- The core entity/topic in the user's question has been hit. -- There is direct evidence supporting the main intent of the question. -- Partial but usable coverage is sufficient — you do NOT need exhaustive or perfect coverage to answer. -- **When results are sufficient, compose the answer immediately. Do NOT call retrieval again to "double-check" or "get more context".** +### Sufficient — answer immediately, no more calls -### Insufficient — retry -- Empty, `Error:`, `no excel files found`, off-topic, missing core entity/scope, no usable evidence at all. +ANY of the following means results are sufficient — STOP and answer now: +- The core entity/topic in the user's question appears in the results. +- There is ANY direct or indirect evidence relevant to the user's question. +- Results are partially relevant, even if not perfectly comprehensive. +- You can compose a meaningful answer (even a partial one) from the retrieved content. + +**Anti-patterns — do NOT do these:** +- ❌ "The results are good, but maybe different keywords could find something better." +- ❌ "I have enough to answer, but let me try one more query to be thorough." +- ❌ "The answer is here, but I want to double-check with a different query." +- ❌ Calling retrieval again after you have already identified the answer in previous results. + +**If you can answer the question with current results, you MUST answer immediately. Period.** + +### Insufficient — the ONLY valid reasons to retry + +- Results are completely empty or contain only `Error:` / `no excel files found` messages. +- ALL results are entirely off-topic with zero relevance to the user's question. +- No usable evidence exists at all — you cannot form even a partial answer. + +**"Results are not detailed enough" is NOT a valid reason to retry.** +**"Results might be incomplete" is NOT a valid reason to retry.** ## 6. Fallback and Sequential Retry @@ -112,6 +135,7 @@ Before replying to a knowledge retrieval task, verify: - Used only whitelisted retrieval tools — no local filesystem inspection? - Called retrieval at most 3 times total (not more)? - Answered immediately when results were sufficient (did NOT call again unnecessarily)? +- Called retrieval exactly once when first results were sufficient (did NOT retry unnecessarily)? - Citations placed immediately after each relevant paragraph? - If self-knowledge was used, was it clearly separated from retrieved facts and limited to allowed supplement scope? diff --git a/skills/autoload/support/rag-retrieve/hooks/retrieval-policy-forbidden-self-knowledge.md b/skills/autoload/support/rag-retrieve/hooks/retrieval-policy-forbidden-self-knowledge.md index 0bdd335..061c855 100644 --- a/skills/autoload/support/rag-retrieve/hooks/retrieval-policy-forbidden-self-knowledge.md +++ b/skills/autoload/support/rag-retrieve/hooks/retrieval-policy-forbidden-self-knowledge.md @@ -42,6 +42,13 @@ Execute **sequentially, one at a time**. Do NOT run in parallel. Do NOT probe fi - After each step, evaluate sufficiency before proceeding. - Retrieval must happen **before** any factual answer generation. +### First-Call Success Principle + +- The first retrieval call is expected to return sufficient results for most questions. +- Your default assumption should be: **one call is enough**. +- Additional calls are the exception, not the norm. Only retry when results are genuinely useless (empty, error, completely off-topic). +- **Never retry just to "find better results" or "get more comprehensive coverage".** Good enough is sufficient. + ## 4. Query Preparation - Do NOT pass raw user question unless it already works well for retrieval. @@ -59,14 +66,30 @@ Execute **sequentially, one at a time**. Do NOT run in parallel. Do NOT probe fi **Maximum 3 retrieval calls per question.** After each call, evaluate immediately: -### Sufficient — answer now -- The core entity/topic in the user's question has been hit. -- There is direct evidence supporting the main intent of the question. -- Partial but usable coverage is sufficient — you do NOT need exhaustive or perfect coverage to answer. -- **When results are sufficient, compose the answer immediately. Do NOT call retrieval again to "double-check" or "get more context".** +### Sufficient — answer immediately, no more calls -### Insufficient — retry -- Empty, `Error:`, `no excel files found`, off-topic, missing core entity/scope, no usable evidence at all, or claims required by the answer are not explicitly supported. +ANY of the following means results are sufficient — STOP and answer now: +- The core entity/topic in the user's question appears in the results. +- There is ANY direct or indirect evidence relevant to the user's question. +- Results are partially relevant, even if not perfectly comprehensive. +- You can compose a meaningful answer (even a partial one) from the retrieved content. + +**Anti-patterns — do NOT do these:** +- ❌ "The results are good, but maybe different keywords could find something better." +- ❌ "I have enough to answer, but let me try one more query to be thorough." +- ❌ "The answer is here, but I want to double-check with a different query." +- ❌ Calling retrieval again after you have already identified the answer in previous results. + +**If you can answer the question with current results, you MUST answer immediately. Period.** + +### Insufficient — the ONLY valid reasons to retry + +- Results are completely empty or contain only `Error:` / `no excel files found` messages. +- ALL results are entirely off-topic with zero relevance to the user's question. +- No usable evidence exists at all — you cannot form even a partial answer. + +**"Results are not detailed enough" is NOT a valid reason to retry.** +**"Results might be incomplete" is NOT a valid reason to retry.** ## 7. Fallback and Sequential Retry @@ -124,6 +147,7 @@ Before replying to a knowledge retrieval task, verify: - Used only whitelisted retrieval tools — no local filesystem inspection? - Called retrieval at most 3 times total (not more)? - Answered immediately when results were sufficient (did NOT call again unnecessarily)? +- Called retrieval exactly once when first results were sufficient (did NOT retry unnecessarily)? - Did retrieval happen before any factual answer drafting? - Did every factual claim come from retrieved evidence rather than model knowledge? - If any unsupported part remained, was it removed or explicitly marked unavailable? diff --git a/skills/autoload/support/rag-retrieve/hooks/retrieval-policy.md b/skills/autoload/support/rag-retrieve/hooks/retrieval-policy.md index 95ccc00..61378ff 100644 --- a/skills/autoload/support/rag-retrieve/hooks/retrieval-policy.md +++ b/skills/autoload/support/rag-retrieve/hooks/retrieval-policy.md @@ -29,6 +29,13 @@ Execute **sequentially, one at a time**. Do NOT run in parallel. Do NOT probe fi - Do NOT answer from model knowledge first. - After each step, evaluate sufficiency before proceeding. +### First-Call Success Principle + +- The first retrieval call is expected to return sufficient results for most questions. +- Your default assumption should be: **one call is enough**. +- Additional calls are the exception, not the norm. Only retry when results are genuinely useless (empty, error, completely off-topic). +- **Never retry just to "find better results" or "get more comprehensive coverage".** Good enough is sufficient. + ## 3. Query Preparation - Do NOT pass raw user question unless it already works well for retrieval. @@ -46,14 +53,30 @@ Execute **sequentially, one at a time**. Do NOT run in parallel. Do NOT probe fi **Maximum 3 retrieval calls per question.** After each call, evaluate immediately: -### Sufficient — answer now -- The core entity/topic in the user's question has been hit. -- There is direct evidence supporting the main intent of the question. -- Partial but usable coverage is sufficient — you do NOT need exhaustive or perfect coverage to answer. -- **When results are sufficient, compose the answer immediately. Do NOT call retrieval again to "double-check" or "get more context".** +### Sufficient — answer immediately, no more calls -### Insufficient — retry -- Empty, `Error:`, `no excel files found`, off-topic, missing core entity/scope, no usable evidence at all. +ANY of the following means results are sufficient — STOP and answer now: +- The core entity/topic in the user's question appears in the results. +- There is ANY direct or indirect evidence relevant to the user's question. +- Results are partially relevant, even if not perfectly comprehensive. +- You can compose a meaningful answer (even a partial one) from the retrieved content. + +**Anti-patterns — do NOT do these:** +- ❌ "The results are good, but maybe different keywords could find something better." +- ❌ "I have enough to answer, but let me try one more query to be thorough." +- ❌ "The answer is here, but I want to double-check with a different query." +- ❌ Calling retrieval again after you have already identified the answer in previous results. + +**If you can answer the question with current results, you MUST answer immediately. Period.** + +### Insufficient — the ONLY valid reasons to retry + +- Results are completely empty or contain only `Error:` / `no excel files found` messages. +- ALL results are entirely off-topic with zero relevance to the user's question. +- No usable evidence exists at all — you cannot form even a partial answer. + +**"Results are not detailed enough" is NOT a valid reason to retry.** +**"Results might be incomplete" is NOT a valid reason to retry.** ## 6. Fallback and Sequential Retry @@ -112,6 +135,7 @@ Before replying to a knowledge retrieval task, verify: - Used only whitelisted retrieval tools — no local filesystem inspection? - Called retrieval at most 3 times total (not more)? - Answered immediately when results were sufficient (did NOT call again unnecessarily)? +- Called retrieval exactly once when first results were sufficient (did NOT retry unnecessarily)? - Citations placed immediately after each relevant paragraph? - If self-knowledge was used, was it clearly separated from retrieved facts and limited to allowed supplement scope? diff --git a/skills/developing/rag-retrieve-no-citation/hooks/retrieval-policy-forbidden-self-knowledge.md b/skills/developing/rag-retrieve-no-citation/hooks/retrieval-policy-forbidden-self-knowledge.md index eb9139b..542cdd7 100644 --- a/skills/developing/rag-retrieve-no-citation/hooks/retrieval-policy-forbidden-self-knowledge.md +++ b/skills/developing/rag-retrieve-no-citation/hooks/retrieval-policy-forbidden-self-knowledge.md @@ -40,6 +40,13 @@ Execute **sequentially, one at a time**. Do NOT run in parallel. Do NOT probe fi - Retrieval must happen **before** any factual answer generation. - After each step, evaluate sufficiency before proceeding. +### First-Call Success Principle + +- The first retrieval call is expected to return sufficient results for most questions. +- Your default assumption should be: **one call is enough**. +- Additional calls are the exception, not the norm. Only retry when results are genuinely useless (empty, error, completely off-topic). +- **Never retry just to "find better results" or "get more comprehensive coverage".** Good enough is sufficient. + ## 4. Query Preparation - Do NOT pass raw user question unless it already works well for retrieval. @@ -57,14 +64,30 @@ Execute **sequentially, one at a time**. Do NOT run in parallel. Do NOT probe fi **Maximum 3 retrieval calls per question.** After each call, evaluate immediately: -### Sufficient — answer now -- The core entity/topic in the user's question has been hit. -- There is direct evidence supporting the main intent of the question. -- Partial but usable coverage is sufficient — you do NOT need exhaustive or perfect coverage to answer. -- **When results are sufficient, compose the answer immediately. Do NOT call retrieval again to "double-check" or "get more context".** +### Sufficient — answer immediately, no more calls -### Insufficient — retry -- Empty, `Error:`, off-topic, missing core entity/scope, no usable evidence at all, or claims required by the answer are not explicitly supported. +ANY of the following means results are sufficient — STOP and answer now: +- The core entity/topic in the user's question appears in the results. +- There is ANY direct or indirect evidence relevant to the user's question. +- Results are partially relevant, even if not perfectly comprehensive. +- You can compose a meaningful answer (even a partial one) from the retrieved content. + +**Anti-patterns — do NOT do these:** +- ❌ "The results are good, but maybe different keywords could find something better." +- ❌ "I have enough to answer, but let me try one more query to be thorough." +- ❌ "The answer is here, but I want to double-check with a different query." +- ❌ Calling retrieval again after you have already identified the answer in previous results. + +**If you can answer the question with current results, you MUST answer immediately. Period.** + +### Insufficient — the ONLY valid reasons to retry + +- Results are completely empty or contain only `Error:` messages. +- ALL results are entirely off-topic with zero relevance to the user's question. +- No usable evidence exists at all — you cannot form even a partial answer. + +**"Results are not detailed enough" is NOT a valid reason to retry.** +**"Results might be incomplete" is NOT a valid reason to retry.** ## 7. Fallback and Sequential Retry @@ -114,6 +137,7 @@ Before replying to a knowledge retrieval task, verify: - Used only whitelisted retrieval tools — no local filesystem inspection? - Called retrieval at most 3 times total (not more)? - Answered immediately when results were sufficient (did NOT call again unnecessarily)? +- Called retrieval exactly once when first results were sufficient (did NOT retry unnecessarily)? - Did retrieval happen before any factual answer drafting? - Did every factual claim come from retrieved evidence rather than model knowledge? - If any unsupported part remained, was it removed or explicitly marked unavailable? diff --git a/skills/developing/rag-retrieve-no-citation/hooks/retrieval-policy.md b/skills/developing/rag-retrieve-no-citation/hooks/retrieval-policy.md index d475c60..8dd6843 100644 --- a/skills/developing/rag-retrieve-no-citation/hooks/retrieval-policy.md +++ b/skills/developing/rag-retrieve-no-citation/hooks/retrieval-policy.md @@ -27,6 +27,13 @@ Execute **sequentially, one at a time**. Do NOT run in parallel. Do NOT probe fi - Do NOT answer from model knowledge first. - After each step, evaluate sufficiency before proceeding. +### First-Call Success Principle + +- The first retrieval call is expected to return sufficient results for most questions. +- Your default assumption should be: **one call is enough**. +- Additional calls are the exception, not the norm. Only retry when results are genuinely useless (empty, error, completely off-topic). +- **Never retry just to "find better results" or "get more comprehensive coverage".** Good enough is sufficient. + ## 3. Query Preparation - Do NOT pass raw user question unless it already works well for retrieval. @@ -44,14 +51,30 @@ Execute **sequentially, one at a time**. Do NOT run in parallel. Do NOT probe fi **Maximum 3 retrieval calls per question.** After each call, evaluate immediately: -### Sufficient — answer now -- The core entity/topic in the user's question has been hit. -- There is direct evidence supporting the main intent of the question. -- Partial but usable coverage is sufficient — you do NOT need exhaustive or perfect coverage to answer. -- **When results are sufficient, compose the answer immediately. Do NOT call retrieval again to "double-check" or "get more context".** +### Sufficient — answer immediately, no more calls -### Insufficient — retry -- Empty, `Error:`, off-topic, missing core entity/scope, no usable evidence at all. +ANY of the following means results are sufficient — STOP and answer now: +- The core entity/topic in the user's question appears in the results. +- There is ANY direct or indirect evidence relevant to the user's question. +- Results are partially relevant, even if not perfectly comprehensive. +- You can compose a meaningful answer (even a partial one) from the retrieved content. + +**Anti-patterns — do NOT do these:** +- ❌ "The results are good, but maybe different keywords could find something better." +- ❌ "I have enough to answer, but let me try one more query to be thorough." +- ❌ "The answer is here, but I want to double-check with a different query." +- ❌ Calling retrieval again after you have already identified the answer in previous results. + +**If you can answer the question with current results, you MUST answer immediately. Period.** + +### Insufficient — the ONLY valid reasons to retry + +- Results are completely empty or contain only `Error:` messages. +- ALL results are entirely off-topic with zero relevance to the user's question. +- No usable evidence exists at all — you cannot form even a partial answer. + +**"Results are not detailed enough" is NOT a valid reason to retry.** +**"Results might be incomplete" is NOT a valid reason to retry.** ## 6. Fallback and Sequential Retry @@ -93,6 +116,7 @@ Before replying to a knowledge retrieval task, verify: - Used only whitelisted retrieval tools — no local filesystem inspection? - Called retrieval at most 3 times total (not more)? - Answered immediately when results were sufficient (did NOT call again unnecessarily)? +- Called retrieval exactly once when first results were sufficient (did NOT retry unnecessarily)? - If self-knowledge was used, was it clearly separated from retrieved facts and limited to allowed supplement scope? If any answer is "no", correct the process first. diff --git a/skills/onprem/rag-retrieve-only/hooks/retrieval-policy-forbidden-self-knowledge.md b/skills/onprem/rag-retrieve-only/hooks/retrieval-policy-forbidden-self-knowledge.md index 2dfa82e..7ea5e4b 100644 --- a/skills/onprem/rag-retrieve-only/hooks/retrieval-policy-forbidden-self-knowledge.md +++ b/skills/onprem/rag-retrieve-only/hooks/retrieval-policy-forbidden-self-knowledge.md @@ -40,6 +40,13 @@ Execute **sequentially, one at a time**. Do NOT run in parallel. Do NOT probe fi - After each step, evaluate sufficiency before proceeding. - Retrieval must happen **before** any factual answer generation. +### First-Call Success Principle + +- The first retrieval call is expected to return sufficient results for most questions. +- Your default assumption should be: **one call is enough**. +- Additional calls are the exception, not the norm. Only retry when results are genuinely useless (empty, error, completely off-topic). +- **Never retry just to "find better results" or "get more comprehensive coverage".** Good enough is sufficient. + ## 4. Query Preparation - Do NOT pass raw user question unless it already works well for retrieval. @@ -57,14 +64,30 @@ Execute **sequentially, one at a time**. Do NOT run in parallel. Do NOT probe fi **Maximum 3 retrieval calls per question.** After each call, evaluate immediately: -### Sufficient — answer now -- The core entity/topic in the user's question has been hit. -- There is direct evidence supporting the main intent of the question. -- Partial but usable coverage is sufficient — you do NOT need exhaustive or perfect coverage to answer. -- **When results are sufficient, compose the answer immediately. Do NOT call retrieval again to "double-check" or "get more context".** +### Sufficient — answer immediately, no more calls -### Insufficient — retry -- Empty, `Error:`, off-topic, missing core entity/scope, no usable evidence at all, or claims required by the answer are not explicitly supported. +ANY of the following means results are sufficient — STOP and answer now: +- The core entity/topic in the user's question appears in the results. +- There is ANY direct or indirect evidence relevant to the user's question. +- Results are partially relevant, even if not perfectly comprehensive. +- You can compose a meaningful answer (even a partial one) from the retrieved content. + +**Anti-patterns — do NOT do these:** +- ❌ "The results are good, but maybe different keywords could find something better." +- ❌ "I have enough to answer, but let me try one more query to be thorough." +- ❌ "The answer is here, but I want to double-check with a different query." +- ❌ Calling retrieval again after you have already identified the answer in previous results. + +**If you can answer the question with current results, you MUST answer immediately. Period.** + +### Insufficient — the ONLY valid reasons to retry + +- Results are completely empty or contain only `Error:` messages. +- ALL results are entirely off-topic with zero relevance to the user's question. +- No usable evidence exists at all — you cannot form even a partial answer. + +**"Results are not detailed enough" is NOT a valid reason to retry.** +**"Results might be incomplete" is NOT a valid reason to retry.** ## 7. Fallback and Sequential Retry @@ -114,6 +137,7 @@ Before replying to a knowledge retrieval task, verify: - Used only whitelisted retrieval tools — no local filesystem inspection? - Called retrieval at most 3 times total (not more)? - Answered immediately when results were sufficient (did NOT call again unnecessarily)? +- Called retrieval exactly once when first results were sufficient (did NOT retry unnecessarily)? - Did retrieval happen before any factual answer drafting? - Did every factual claim come from retrieved evidence rather than model knowledge? - If any unsupported part remained, was it removed or explicitly marked unavailable? diff --git a/skills/onprem/rag-retrieve-only/hooks/retrieval-policy.md b/skills/onprem/rag-retrieve-only/hooks/retrieval-policy.md index 0b810b2..7c69ded 100644 --- a/skills/onprem/rag-retrieve-only/hooks/retrieval-policy.md +++ b/skills/onprem/rag-retrieve-only/hooks/retrieval-policy.md @@ -27,6 +27,13 @@ Execute **sequentially, one at a time**. Do NOT run in parallel. Do NOT probe fi - Do NOT answer from model knowledge first. - After each step, evaluate sufficiency before proceeding. +### First-Call Success Principle + +- The first retrieval call is expected to return sufficient results for most questions. +- Your default assumption should be: **one call is enough**. +- Additional calls are the exception, not the norm. Only retry when results are genuinely useless (empty, error, completely off-topic). +- **Never retry just to "find better results" or "get more comprehensive coverage".** Good enough is sufficient. + ## 3. Query Preparation - Do NOT pass raw user question unless it already works well for retrieval. @@ -44,14 +51,30 @@ Execute **sequentially, one at a time**. Do NOT run in parallel. Do NOT probe fi **Maximum 3 retrieval calls per question.** After each call, evaluate immediately: -### Sufficient — answer now -- The core entity/topic in the user's question has been hit. -- There is direct evidence supporting the main intent of the question. -- Partial but usable coverage is sufficient — you do NOT need exhaustive or perfect coverage to answer. -- **When results are sufficient, compose the answer immediately. Do NOT call retrieval again to "double-check" or "get more context".** +### Sufficient — answer immediately, no more calls -### Insufficient — retry -- Empty, `Error:`, off-topic, missing core entity/scope, no usable evidence at all. +ANY of the following means results are sufficient — STOP and answer now: +- The core entity/topic in the user's question appears in the results. +- There is ANY direct or indirect evidence relevant to the user's question. +- Results are partially relevant, even if not perfectly comprehensive. +- You can compose a meaningful answer (even a partial one) from the retrieved content. + +**Anti-patterns — do NOT do these:** +- ❌ "The results are good, but maybe different keywords could find something better." +- ❌ "I have enough to answer, but let me try one more query to be thorough." +- ❌ "The answer is here, but I want to double-check with a different query." +- ❌ Calling retrieval again after you have already identified the answer in previous results. + +**If you can answer the question with current results, you MUST answer immediately. Period.** + +### Insufficient — the ONLY valid reasons to retry + +- Results are completely empty or contain only `Error:` messages. +- ALL results are entirely off-topic with zero relevance to the user's question. +- No usable evidence exists at all — you cannot form even a partial answer. + +**"Results are not detailed enough" is NOT a valid reason to retry.** +**"Results might be incomplete" is NOT a valid reason to retry.** ## 6. Fallback and Sequential Retry @@ -102,6 +125,7 @@ Before replying to a knowledge retrieval task, verify: - Used only whitelisted retrieval tools — no local filesystem inspection? - Called retrieval at most 3 times total (not more)? - Answered immediately when results were sufficient (did NOT call again unnecessarily)? +- Called retrieval exactly once when first results were sufficient (did NOT retry unnecessarily)? - Citations placed immediately after each relevant paragraph? - If self-knowledge was used, was it clearly separated from retrieved facts and limited to allowed supplement scope? diff --git a/skills/support/rag-retrieve-only/hooks/retrieval-policy-forbidden-self-knowledge.md b/skills/support/rag-retrieve-only/hooks/retrieval-policy-forbidden-self-knowledge.md index 0bdd335..061c855 100644 --- a/skills/support/rag-retrieve-only/hooks/retrieval-policy-forbidden-self-knowledge.md +++ b/skills/support/rag-retrieve-only/hooks/retrieval-policy-forbidden-self-knowledge.md @@ -42,6 +42,13 @@ Execute **sequentially, one at a time**. Do NOT run in parallel. Do NOT probe fi - After each step, evaluate sufficiency before proceeding. - Retrieval must happen **before** any factual answer generation. +### First-Call Success Principle + +- The first retrieval call is expected to return sufficient results for most questions. +- Your default assumption should be: **one call is enough**. +- Additional calls are the exception, not the norm. Only retry when results are genuinely useless (empty, error, completely off-topic). +- **Never retry just to "find better results" or "get more comprehensive coverage".** Good enough is sufficient. + ## 4. Query Preparation - Do NOT pass raw user question unless it already works well for retrieval. @@ -59,14 +66,30 @@ Execute **sequentially, one at a time**. Do NOT run in parallel. Do NOT probe fi **Maximum 3 retrieval calls per question.** After each call, evaluate immediately: -### Sufficient — answer now -- The core entity/topic in the user's question has been hit. -- There is direct evidence supporting the main intent of the question. -- Partial but usable coverage is sufficient — you do NOT need exhaustive or perfect coverage to answer. -- **When results are sufficient, compose the answer immediately. Do NOT call retrieval again to "double-check" or "get more context".** +### Sufficient — answer immediately, no more calls -### Insufficient — retry -- Empty, `Error:`, `no excel files found`, off-topic, missing core entity/scope, no usable evidence at all, or claims required by the answer are not explicitly supported. +ANY of the following means results are sufficient — STOP and answer now: +- The core entity/topic in the user's question appears in the results. +- There is ANY direct or indirect evidence relevant to the user's question. +- Results are partially relevant, even if not perfectly comprehensive. +- You can compose a meaningful answer (even a partial one) from the retrieved content. + +**Anti-patterns — do NOT do these:** +- ❌ "The results are good, but maybe different keywords could find something better." +- ❌ "I have enough to answer, but let me try one more query to be thorough." +- ❌ "The answer is here, but I want to double-check with a different query." +- ❌ Calling retrieval again after you have already identified the answer in previous results. + +**If you can answer the question with current results, you MUST answer immediately. Period.** + +### Insufficient — the ONLY valid reasons to retry + +- Results are completely empty or contain only `Error:` / `no excel files found` messages. +- ALL results are entirely off-topic with zero relevance to the user's question. +- No usable evidence exists at all — you cannot form even a partial answer. + +**"Results are not detailed enough" is NOT a valid reason to retry.** +**"Results might be incomplete" is NOT a valid reason to retry.** ## 7. Fallback and Sequential Retry @@ -124,6 +147,7 @@ Before replying to a knowledge retrieval task, verify: - Used only whitelisted retrieval tools — no local filesystem inspection? - Called retrieval at most 3 times total (not more)? - Answered immediately when results were sufficient (did NOT call again unnecessarily)? +- Called retrieval exactly once when first results were sufficient (did NOT retry unnecessarily)? - Did retrieval happen before any factual answer drafting? - Did every factual claim come from retrieved evidence rather than model knowledge? - If any unsupported part remained, was it removed or explicitly marked unavailable? diff --git a/skills/support/rag-retrieve-only/hooks/retrieval-policy.md b/skills/support/rag-retrieve-only/hooks/retrieval-policy.md index 0b810b2..7c69ded 100644 --- a/skills/support/rag-retrieve-only/hooks/retrieval-policy.md +++ b/skills/support/rag-retrieve-only/hooks/retrieval-policy.md @@ -27,6 +27,13 @@ Execute **sequentially, one at a time**. Do NOT run in parallel. Do NOT probe fi - Do NOT answer from model knowledge first. - After each step, evaluate sufficiency before proceeding. +### First-Call Success Principle + +- The first retrieval call is expected to return sufficient results for most questions. +- Your default assumption should be: **one call is enough**. +- Additional calls are the exception, not the norm. Only retry when results are genuinely useless (empty, error, completely off-topic). +- **Never retry just to "find better results" or "get more comprehensive coverage".** Good enough is sufficient. + ## 3. Query Preparation - Do NOT pass raw user question unless it already works well for retrieval. @@ -44,14 +51,30 @@ Execute **sequentially, one at a time**. Do NOT run in parallel. Do NOT probe fi **Maximum 3 retrieval calls per question.** After each call, evaluate immediately: -### Sufficient — answer now -- The core entity/topic in the user's question has been hit. -- There is direct evidence supporting the main intent of the question. -- Partial but usable coverage is sufficient — you do NOT need exhaustive or perfect coverage to answer. -- **When results are sufficient, compose the answer immediately. Do NOT call retrieval again to "double-check" or "get more context".** +### Sufficient — answer immediately, no more calls -### Insufficient — retry -- Empty, `Error:`, off-topic, missing core entity/scope, no usable evidence at all. +ANY of the following means results are sufficient — STOP and answer now: +- The core entity/topic in the user's question appears in the results. +- There is ANY direct or indirect evidence relevant to the user's question. +- Results are partially relevant, even if not perfectly comprehensive. +- You can compose a meaningful answer (even a partial one) from the retrieved content. + +**Anti-patterns — do NOT do these:** +- ❌ "The results are good, but maybe different keywords could find something better." +- ❌ "I have enough to answer, but let me try one more query to be thorough." +- ❌ "The answer is here, but I want to double-check with a different query." +- ❌ Calling retrieval again after you have already identified the answer in previous results. + +**If you can answer the question with current results, you MUST answer immediately. Period.** + +### Insufficient — the ONLY valid reasons to retry + +- Results are completely empty or contain only `Error:` messages. +- ALL results are entirely off-topic with zero relevance to the user's question. +- No usable evidence exists at all — you cannot form even a partial answer. + +**"Results are not detailed enough" is NOT a valid reason to retry.** +**"Results might be incomplete" is NOT a valid reason to retry.** ## 6. Fallback and Sequential Retry @@ -102,6 +125,7 @@ Before replying to a knowledge retrieval task, verify: - Used only whitelisted retrieval tools — no local filesystem inspection? - Called retrieval at most 3 times total (not more)? - Answered immediately when results were sufficient (did NOT call again unnecessarily)? +- Called retrieval exactly once when first results were sufficient (did NOT retry unnecessarily)? - Citations placed immediately after each relevant paragraph? - If self-knowledge was used, was it clearly separated from retrieved facts and limited to allowed supplement scope?