Workforce Engagement Management

 View Only

Sign Up

  • 1.  Automated QA Evaluations Agent Scoring Rules

    Posted 4 days ago

    Hello all-

    We are about to start testing automated QA with an AI form and had questions on how the agent scoring rules work.

    We see that the Program is the starting point and any queues that are a part of that Program will have evaluations for all interactions or a percentage. Here are our initial questions:

                                           -If we do a percentage of calls for the queues are the evaluations evenly distributed across the agents that handle those queues?

                                           -Can we set rules to only evaluate interactions for a specific period each month as we coach for the last 5-7 days of each month for the same month                                             interactions? 

                                            -If a queue is part of one Program, can it be used in another Program simultaneously? 

                                            -What are the rules as to which interactions are being evaluated? Does it evaluate agents across the month or front-load evaluations at start of the                                                month?

                                            -With the Agent Auto-Evaluation Form type, can we update during the month if we see discrepancies with how AI questions are being answered?

                                            -If we do make updates will the Program start over and evaluate another percentage of interactions?

                                            -What is the view to agents that are being auto-evaluated with respect to scores? General score for all interactions?

    If there are any organizations using automated QA that can assist with these questions, it would be much appreciated.

    Thank you.

                                             -


    #AIScoring(VirtualSupervisor)

    ------------------------------
    Joseph
    ------------------------------


  • 2.  RE: Automated QA Evaluations Agent Scoring Rules

    Posted yesterday
    Edited by Cesar Padilla yesterday

    hi @Joseph Sutich,

    We've been using Automated QA and AI Scoring, and while some behaviors can vary depending on configuration and future platform updates, here's what we've observed:

    1. Percentage-based evaluations

      • The evaluation percentage is generally applied to the interaction population that matches the program criteria.
      • In our experience, this does not necessarily guarantee an equal distribution across all agents. Agents who handle more interactions may naturally receive more evaluations.

    2. Evaluating only specific periods of the month

      • We have not seen a native option that schedules automated evaluations only during specific days of a month (for example, evaluating only interactions from days 1-23 and leaving the last week for coaching).
      • This may require program design considerations, filters, or operational processes rather than a dedicated scheduling feature.

    3. Queue participation in multiple programs

      • A queue can typically be referenced by multiple programs, but it is important to understand how overlapping criteria may affect evaluation volume and reporting.
      • We'd recommend validating this against current Genesys documentation for your release.

    4. How interactions are selected

      • Our understanding is that evaluations are generated from interactions that meet the program criteria and sampling rules.
      • We have not observed a deliberate "front-loading" strategy at the beginning of the month. Selection appears to be based on interaction availability rather than evenly spread over time.

    5. Updating AI Evaluation Forms

      • Forms can generally be modified as you learn from the AI scoring results.
      • However, changes to questions, scoring logic, or AI prompts can affect consistency when comparing historical and future evaluations.

    6. Impact of form changes on existing programs

      • In our experience, updating a form does not automatically restart the entire program or re-evaluate previously scored interactions.
      • The updated logic is typically applied to evaluations generated after the change, although this is something worth confirming in a test environment.

    7. Agent visibility

      • Agents can usually see evaluation results based on the permissions and views enabled by your organization.
      • The exact experience depends on how scorecards, evaluation visibility, and performance views are configured in Workforce Engagement Management.

    One recommendation before going live is to run a pilot with a small percentage of interactions and compare AI-generated scores against human evaluator scores for a few weeks. This helps identify scoring discrepancies, calibrate questions, and improve confidence in the automated process.

    I'd also be interested in hearing from other organizations that have been running AI Scoring in production for an extended period.



    ------------------------------
    Cesar
    INDRA COLOMBIA
    ------------------------------



  • 3.  RE: Automated QA Evaluations Agent Scoring Rules

    Posted yesterday

    Hi Cesar,

    Thank you for your responses. We will review and determine if there are concerns with rollout.



    ------------------------------
    Joseph
    ------------------------------

    Original Message


  • 4.  RE: Automated QA Evaluations Agent Scoring Rules

    Posted yesterday

    Just set up our program today, will report back on our experiences, but loved finding this thread...can anyone link me to the Aha! ideas or roadmap items re: reporting updates?  



    ------------------------------
    Chad Blanton
    Director, WFM/WEM/All Things Genesys
    ------------------------------



Related Content