Genesys Cloud - Main

 View Only

Sign Up

  • 1.  Agentic Virtual Agents - out of scope items

    Posted 20 days ago

    I am having trouble defining specific items to be out of scope for an AVA. It does a good job of not allowing customer questions that would be outside of it's domain, but it tries to be overly helpful.

    For example, I have specified in the Role and in the Guidelines that it should not attempt to troubleshoot an issue or to suggest fixes. I am having a really hard time having it follow these instructions. Any thoughts on wording, formatting, etc that would help it not offer to do something (on its own) that I don't want it to?


    #ConversationalAI(Bots,VirtualAgent,etc.)

    ------------------------------
    Kevin Goodwin
    Senior Consultant
    ------------------------------


  • 2.  RE: Agentic Virtual Agents - out of scope items

    Posted 20 days ago

    This is something we also noticed during AVA testing.

    One important thing with AVA is that the model is naturally optimized to be "helpful," so broad negative instructions like:

    "do not troubleshoot"
    "do not suggest fixes"
    often are not strong enough by themselves because the assistant still tries to move the customer toward resolution conversationally.

    What worked better for us was shifting from:
    generic prohibitions
    to:
    very explicit behavioral boundaries + mandatory escalation behavior.

    For example, instead of:

    "Do not troubleshoot technical issues."

    we had much better results with wording closer to:

    "If the customer describes a technical issue, you must NOT provide diagnostic steps, recommendations, workaround suggestions, configuration guidance, assumptions, or possible causes. Your ONLY allowed behavior is:

    1. acknowledge the issue briefly

    2. collect the required routing information

    3. escalate to a live agent."

    The key difference is:
    you are not only forbidding behavior,
    you are defining the exact replacement behavior.

    Another thing that helped significantly was avoiding ambiguous verbs such as:

    help
    assist
    guide
    recommend
    try
    suggest
    because the model interprets those very broadly.

    We also started separating guidance into:

    Role → what the AVA IS
    Guidelines → operational rules
    Forbidden behaviors → explicit hard constraints
    For example:

    Role:
    "You are an intake and routing assistant."

    Guideline:
    "You collect information and route requests."

    Forbidden behavior:
    "You must never troubleshoot, diagnose, recommend fixes, speculate on causes, or provide procedural instructions."
    One important observation:
    the more tools/knowledge access the AVA has, the more likely it is to attempt problem solving autonomously.

    So reducing unnecessary tool access can improve containment of behavior considerably.

    We also found that examples are extremely powerful with AVA behavior shaping.

    For example:

    BAD:
    Customer: "My VPN is failing."
    AVA: "Try restarting your VPN client."

    GOOD:
    Customer: "My VPN is failing."
    AVA: "I'm unable to troubleshoot VPN issues directly. I'll connect you with a support specialist."
    In practice, these behavioral examples often influence the runtime behavior more reliably than long policy paragraphs.

    Another recommendation:
    avoid conflicting instructions.

    For example, combinations like:

    "Be proactive and helpful"
    together with:
    "Do not troubleshoot"
    can create unstable behavior because the model tries to satisfy both objectives simultaneously.

    In production, the AVAs that behaved most predictably for us were the ones with:

    very narrow role definitions
    explicit forbidden actions
    clear escalation conditions
    few overlapping objectives
    constrained tool access
    example-driven behavioral guidance
    AVA behaves much more consistently when it understands:
    "what it should do instead"
    rather than only:
    "what it must not do."



    ------------------------------
    Gabriel Garcia
    NA
    ------------------------------



  • 3.  RE: Agentic Virtual Agents - out of scope items

    Posted 19 days ago

    Thanks for the suggestion around being more specific. I will give that a shot. 

    Also, in your example above, did you put all that information in the Role of the AVA, or did you break out the pieces into the Role, Guidelines and Guardrails(rules) sections? 



    ------------------------------
    Kevin Goodwin
    Senior Consultant
    ------------------------------



  • 4.  RE: Agentic Virtual Agents - out of scope items

    Posted 18 days ago

    Yes, we had better results when separating the responsibilities across the sections instead of putting everything into the Role field.

    What worked best for us was roughly:

    Role
    Very short and narrow.

    Example:
    "You are an intake and routing assistant for technical support requests."

    The Role worked better when it only defined identity/purpose, not operational logic.

    Guidelines
    This is where we placed the operational behavior.

    For example:

    • collect customer information
    • identify the request category
    • escalate technical troubleshooting cases
    • keep responses concise
    • do not attempt resolution

    This section seemed to work better for behavioral expectations and conversational style.

    Guardrails / Rules
    We used this for the hard constraints and explicit forbidden actions.

    For example:

    • never provide troubleshooting steps
    • never speculate about root cause
    • never recommend configuration changes
    • never invent unsupported procedures
    • escalate if the customer asks for technical guidance

    We also had better consistency when the guardrails included explicit escalation triggers instead of only prohibitions.

    One thing we noticed:
    if too much logic is placed into the Role section, the AVA sometimes starts treating operational instructions as "soft personality context" instead of hard behavioral boundaries.

    Separating:
    identity → behavior → restrictions

    made the runtime behavior much more predictable in our tests.



    ------------------------------
    Gabriel Garcia
    NA
    ------------------------------