GENERAL INSTRUCTIONS ----------------- In our system, the behavior of a conversational AI agent is guided by "guidelines". Each guideline is composed of two parts: - "condition": This is a natural-language condition that specifies when a guideline should apply - "action": This is a natural-language instruction that should be followed by the agent Task Description ---------------- Your task is to evaluate the relevance and applicability of a set of provided 'when' conditions to the most recent state of an interaction between yourself (an AI agent) and a user. Examples of Guideline Match Evaluations: ``` Example #1: travel consultation - **Chat History**: user: Hi, I'm planning a trip to Italy next month. What can I do there? ai_agent: That sounds exciting! I can help you with that. Do you prefer exploring cities or enjoying scenic landscapes? user: Can you help me figure out the best time to visit Rome and what to pack? user: Actually I'm also wondering — do I need any special visas or documents as an American citizen? - **Guidelines**: 1) Condition: The customer is looking for flight or accommodation booking assistance. Action: Provide links or suggestions for flight aggregators and hotel booking platforms. 2) Condition: The customer ask for activities recommendations. Action: Guide them in refining their preferences and suggest options that match what they're looking for 3) Condition: The customer asks for logistical or legal requirements. Action: Provide a clear answer or direct them to a trusted official source if uncertain. - **Expected Result**: ```json { "checks": [ { "guideline_id": "", "condition": "The customer is looking for flight or accommodation booking assistance", "rationale": "There's no mention of booking logistics like flights or hotels", "applies": false }, { "guideline_id": "", "condition": "The customer ask for activities recommendations", "rationale": "The customer has moved from seeking activity recommendations to asking about legal requirements. Since they are no longer pursuing their original inquiry about activities, this represents a new topic rather than a sub-issue", "applies": false }, { "guideline_id": "", "condition": "The customer asks for logistical or legal requirements.", "rationale": "The customer now asked about visas and documents which are legal requirements", "applies": true } ] } ``` Example #2: Course Consultation - **Chat History**: user:Hi, I'm interested in your Python programming course, but I'm not sure if I'm ready for it. ai_agent:Happy to help! Could you share a bit about your background or experience with programming so far? user:I've done some HTML and CSS, but never written real code before. ai_agent:Thanks for sharing! That gives me a good idea. Our Python course is beginner-friendly, but it does assume you're comfortable with logic and problem solving. Would you like me to recommend a short prep course first? user:That sounds useful. But I'm also wondering — is the course self-paced? I work full time. - **Guidelines**: 1) Condition: The customer mentions a constraint that related to commitment to the course. Action: Emphasize flexible learning options 2) Condition: The user expresses hesitation or self-doubt. Action: Affirm that it's okay to be uncertain and provide confidence-building context 3) Condition: The user asks about certification or course completion benefits. Action: Clearly explain what the user receives - **Expected Result**: ```json { "checks": [ { "guideline_id": "", "condition": "The customer mentions a constraint that related to commitment to the course", "rationale": "In the most recent message the customer mentions that they work full time which is a constraint", "applies": true }, { "guideline_id": "", "condition": "The user expresses hesitation or self-doubt.", "rationale": "In the most recent message the user still sounds hesitating about their fit to the course", "applies": true }, { "guideline_id": "", "condition": "The user asks about certification or course completion benefits.", "rationale": "The user didn't ask about certification or course completion benefits", "applies": false } ] } ``` Example #3: Login issue - **Chat History**: user:I'm having trouble logging into my account. ai_agent:I'm sorry to hear that. Can you tell me what happens when you try to log in? user:It says my password is incorrect. ai_agent:Have you tried resetting your password? user:Yes, I did, but I can't access my mail to complete the reset. - **Guidelines**: 1) Condition When the user is having a problem with login. Action: Help then identify the problem and solve it - **Expected Result**: ```json { "checks": [ { "guideline_id": "", "condition": "When the user is having a problem with login.", "rationale": "In the most recent message the customer is still pursuing their login problem, making the mail access problem a sub-issue rather than a new topic", "applies": true } ] } Example #4: Return Policy - **Chat History**: user: Hi, I'm thinking about ordering this coat, but I need to know — what's your return policy? ai_agent: You can return items within 30 days either in-store or using our prepaid return label. user: And what happens if I already wore it once? - **Guidelines**: 1) Condition When the customer asks about how to return an item. Action: Mention both in-store and delivery service return options. - **Expected Result**: ```json { "checks": [ { "guideline_id": "", "condition": "When the customer asks about how to return an item.", "rationale": "In the most recent message the customer asks about what happens when they wore the item, which an inquiry regarding returning an item", "applies": true } ] } ``` Terms: {terms} Chat History: {chat_history} Guidelines List: {guidelines_text} OUTPUT FORMAT: The content in JSON format needs to be wrapped in "```json" and "```". Please specify the applicability of each guideline: ```json { "checks": [ { "guideline_id": "1", "condition": "Specific condition description", "rationale": "", "applies": true/false } ] } ```