164 lines
7.0 KiB
Markdown
164 lines
7.0 KiB
Markdown
GENERAL INSTRUCTIONS
|
|
-----------------
|
|
In our system, the behavior of a conversational AI agent is guided by "guidelines".
|
|
Each guideline is composed of two parts:
|
|
- "condition": This is a natural-language condition that specifies when a guideline should apply
|
|
- "action": This is a natural-language instruction that should be followed by the agent
|
|
|
|
Task Description
|
|
----------------
|
|
Your task is to evaluate the relevance and applicability of a set of provided 'when' conditions
|
|
to the most recent state of an interaction between yourself (an AI agent) and a user.
|
|
|
|
|
|
Examples of Guideline Match Evaluations:
|
|
```
|
|
Example #1: travel consultation
|
|
- **Chat History**:
|
|
user: Hi, I'm planning a trip to Italy next month. What can I do there?
|
|
ai_agent: That sounds exciting! I can help you with that. Do you prefer exploring cities or enjoying scenic landscapes?
|
|
user: Can you help me figure out the best time to visit Rome and what to pack?
|
|
user: Actually I'm also wondering — do I need any special visas or documents as an American citizen?
|
|
|
|
- **Guidelines**:
|
|
1) Condition: The customer is looking for flight or accommodation booking assistance. Action: Provide links or suggestions for flight aggregators and hotel booking platforms.
|
|
2) Condition: The customer ask for activities recommendations. Action: Guide them in refining their preferences and suggest options that match what they're looking for
|
|
3) Condition: The customer asks for logistical or legal requirements. Action: Provide a clear answer or direct them to a trusted official source if uncertain.
|
|
|
|
- **Expected Result**:
|
|
```json
|
|
{
|
|
"checks": [
|
|
{
|
|
"guideline_id": "<example-id-for-few-shots--do-not-use-this-in-output>",
|
|
"condition": "The customer is looking for flight or accommodation booking assistance",
|
|
"rationale": "There's no mention of booking logistics like flights or hotels",
|
|
"applies": false
|
|
},
|
|
{
|
|
"guideline_id": "<example-id-for-few-shots--do-not-use-this-in-output>",
|
|
"condition": "The customer ask for activities recommendations",
|
|
"rationale": "The customer has moved from seeking activity recommendations to asking about legal requirements. Since they are no longer pursuing their original inquiry about activities, this represents a new topic rather than a sub-issue",
|
|
"applies": false
|
|
},
|
|
{
|
|
"guideline_id": "<example-id-for-few-shots--do-not-use-this-in-output>",
|
|
"condition": "The customer asks for logistical or legal requirements.",
|
|
"rationale": "The customer now asked about visas and documents which are legal requirements",
|
|
"applies": true
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
Example #2: Course Consultation
|
|
- **Chat History**:
|
|
user:Hi, I'm interested in your Python programming course, but I'm not sure if I'm ready for it.
|
|
ai_agent:Happy to help! Could you share a bit about your background or experience with programming so far?
|
|
user:I've done some HTML and CSS, but never written real code before.
|
|
ai_agent:Thanks for sharing! That gives me a good idea. Our Python course is beginner-friendly, but it does assume you're comfortable with logic and problem solving. Would you like me to recommend a short prep course first?
|
|
user:That sounds useful. But I'm also wondering — is the course self-paced? I work full time.
|
|
|
|
- **Guidelines**:
|
|
1) Condition: The customer mentions a constraint that related to commitment to the course. Action: Emphasize flexible learning options
|
|
2) Condition: The user expresses hesitation or self-doubt. Action: Affirm that it's okay to be uncertain and provide confidence-building context
|
|
3) Condition: The user asks about certification or course completion benefits. Action: Clearly explain what the user receives
|
|
|
|
- **Expected Result**:
|
|
```json
|
|
{
|
|
"checks": [
|
|
{
|
|
"guideline_id": "<example-id-for-few-shots--do-not-use-this-in-output>",
|
|
"condition": "The customer mentions a constraint that related to commitment to the course",
|
|
"rationale": "In the most recent message the customer mentions that they work full time which is a constraint",
|
|
"applies": true
|
|
},
|
|
{
|
|
"guideline_id": "<example-id-for-few-shots--do-not-use-this-in-output>",
|
|
"condition": "The user expresses hesitation or self-doubt.",
|
|
"rationale": "In the most recent message the user still sounds hesitating about their fit to the course",
|
|
"applies": true
|
|
},
|
|
{
|
|
"guideline_id": "<example-id-for-few-shots--do-not-use-this-in-output>",
|
|
"condition": "The user asks about certification or course completion benefits.",
|
|
"rationale": "The user didn't ask about certification or course completion benefits",
|
|
"applies": false
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
Example #3: Login issue
|
|
- **Chat History**:
|
|
user:I'm having trouble logging into my account.
|
|
ai_agent:I'm sorry to hear that. Can you tell me what happens when you try to log in?
|
|
user:It says my password is incorrect.
|
|
ai_agent:Have you tried resetting your password?
|
|
user:Yes, I did, but I can't access my mail to complete the reset.
|
|
|
|
- **Guidelines**:
|
|
1) Condition When the user is having a problem with login. Action: Help then identify the problem and solve it
|
|
|
|
- **Expected Result**:
|
|
```json
|
|
{
|
|
"checks": [
|
|
{
|
|
"guideline_id": "<example-id-for-few-shots--do-not-use-this-in-output>",
|
|
"condition": "When the user is having a problem with login.",
|
|
"rationale": "In the most recent message the customer is still pursuing their login problem, making the mail access problem a sub-issue rather than a new topic",
|
|
"applies": true
|
|
}
|
|
]
|
|
}
|
|
|
|
Example #4: Return Policy
|
|
- **Chat History**:
|
|
user: Hi, I'm thinking about ordering this coat, but I need to know — what's your return policy?
|
|
ai_agent: You can return items within 30 days either in-store or using our prepaid return label.
|
|
user: And what happens if I already wore it once?
|
|
|
|
- **Guidelines**:
|
|
1) Condition When the customer asks about how to return an item. Action: Mention both in-store and delivery service return options.
|
|
|
|
- **Expected Result**:
|
|
```json
|
|
{
|
|
"checks": [
|
|
{
|
|
"guideline_id": "<example-id-for-few-shots--do-not-use-this-in-output>",
|
|
"condition": "When the customer asks about how to return an item.",
|
|
"rationale": "In the most recent message the customer asks about what happens when they wore the item, which an inquiry regarding returning an item",
|
|
"applies": true
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
Terms:
|
|
{terms}
|
|
|
|
Chat History:
|
|
{chat_history}
|
|
|
|
Guidelines List:
|
|
{guidelines_text}
|
|
|
|
OUTPUT FORMAT:
|
|
The content in JSON format needs to be wrapped in "```json" and "```".
|
|
Please specify the applicability of each guideline:
|
|
```json
|
|
{
|
|
"checks": [
|
|
{
|
|
"guideline_id": "1",
|
|
"condition": "Specific condition description",
|
|
"rationale": "<Explain why the conditions are met or not met, and what action should be taken if the conditions are met>",
|
|
"applies": true/false
|
|
}
|
|
]
|
|
}
|
|
```
|