Workforce Engagement Management

 View Only

Sign Up

Expand all | Collapse all

Feedback Wanted: Your Experience with Virtual Supervisor (AI Scoring)

  • 1.  Feedback Wanted: Your Experience with Virtual Supervisor (AI Scoring)

    Posted 21 days ago

    Hi everyone,

    The Virtual Supervisor (AI Scoring) feature has now been available in Genesys Cloud for several months, and we'd love to hear from you!

    We're looking to understand how this capability has impacted your organization - what benefits have you seen so far, and how has it influenced your quality management or performance processes?

    At the same time, we'd like to know about any barriers or challenges that have limited your use or expansion of the feature.

    To help you get the most out of Virtual Supervisor, we've also just launched a new AI Scoring Best Practices Guide. This resource shares recommendations and practical tips for optimizing AI scoring accuracy, aligning it with your evaluation forms, and driving better agent performance outcomes. We encourage you to review it and share your feedback or additional insights from your own experience.

    Your input will help us identify opportunities to improve the experience and guide future enhancements.

    Looking forward to hearing your thoughts - both the wins and the pain points!


    #AIScoring(VirtualSupervisor)

    ------------------------------
    Jose Ruiz
    Genesys - Employees
    Product Manager
    jose.ruiz@genesys.com
    ------------------------------


  • 2.  RE: Feedback Wanted: Your Experience with Virtual Supervisor (AI Scoring)

    Posted 14 days ago

    Let's help our PM team - don't be shy, share your thoughts



    ------------------------------
    Tracy
    Genesys
    ------------------------------



  • 3.  RE: Feedback Wanted: Your Experience with Virtual Supervisor (AI Scoring)

    Posted 10 days ago

    Hi.

    One of our customers has spent quite a lot of time refining the question wording in order to get what it needs from this, but with patience and experience in adjusting the prompt, they are getting there. One issue encountered was that the AI scoring cannot differentiate between the ACD messaging and the agent's spoken words, which is impacting the AI response in some of the questions.

    Heather



    ------------------------------
    Heather Henderson
    ------------------------------



  • 4.  RE: Feedback Wanted: Your Experience with Virtual Supervisor (AI Scoring)

    Posted 9 days ago

    Hello @Heather Henderson, Thank you for your feedback. Adjusting an existing evaluation for AI Scoring does take some time with trial and error included. We published this AI Scoring best practices article a few weeks ago, hopefully you and your customers will find it useful. 

    Segment based evaluations is an enhancement we are planning to release by mid 2026. You will be able to focus on an individual agent, virtual agents (bots) or the entire interactions (Customer Experience, etc).

    Regards,



    ------------------------------
    Jose Ruiz
    Genesys - Employees
    Product Manager
    jose.ruiz@genesys.com
    ------------------------------



  • 5.  RE: Feedback Wanted: Your Experience with Virtual Supervisor (AI Scoring)

    Posted 8 days ago

    Hi @Jose Ruiz

    In our company we have rather strict policies for Digital and AI Ethics. Where one of the no-go rules is evalution of co-worker behavior with the use of AI. For us even to start the assessment of the feature, some shifts should be made in how scores are assigned to evaluations. 
    It would be great if we could use AI scoring for general customer experience assessment without assigning a score to a single agent. Let's say we could have the scores per queue, per skill, per flow, but not in agent performance. 
    Do you think something like this might come later?



    ------------------------------
    Ekaterina (Kate) Kononova
    Product Development | Data, Analytics & Quality Management
    ------------------------------



  • 6.  RE: Feedback Wanted: Your Experience with Virtual Supervisor (AI Scoring)

    Posted 8 days ago

    Hello @Ekaterina Kononova, thank you for your feedback. Yes, we do have AI driven interaction assessment in our roadmap for 2026. 

    Based on the evaluation form that you create, it will answer the questions, calculate a score, but not assign it to an agent, but rather the interaction itself. You can pursue use cases like customer churn score, customer effort score, etc. 

    Regards,



    ------------------------------
    Jose Ruiz
    Genesys - Employees
    Product Manager
    jose.ruiz@genesys.com
    ------------------------------



  • 7.  RE: Feedback Wanted: Your Experience with Virtual Supervisor (AI Scoring)

    Posted 9 days ago

    Jose - I really appreciate the expansion of documentation and support for AI scoring.  One question I had, in the best practices document, it references "Review AI scoring reports monthly to identify low-confidence questions." under continuous improvement.  Can you point me to this reporting in Genesys?   I am digging through resources, and the tool itself, and I could not find another mention of it.



    ------------------------------
    Russell Donald
    InflowCX
    ------------------------------



  • 8.  RE: Feedback Wanted: Your Experience with Virtual Supervisor (AI Scoring)

    Posted 9 days ago

    Very good catch. That portion of the document is forward looking a bit, as we will be releasing in mid January a feature where you will be able to view the agreement rate per AI scoring or Evaluation Assistance question. The rate is calculated using the number of times a user changes an answer from what AI Scoring or Evaluation Assistance selected. A low agreement rate conveys that there are issues with how the question and/or help text are worded. 



    ------------------------------
    Jose Ruiz
    Genesys - Employees
    Product Manager
    jose.ruiz@genesys.com
    ------------------------------



  • 9.  RE: Feedback Wanted: Your Experience with Virtual Supervisor (AI Scoring)

    Posted 9 days ago

    Thanks Jose!  I am very excited about the enhanced reporting for Quality.  Looking forward to seeing this in action!



    ------------------------------
    Russell Donald
    InflowCX
    ------------------------------



  • 10.  RE: Feedback Wanted: Your Experience with Virtual Supervisor (AI Scoring)

    Posted 9 days ago

    This is primarily a UI-related observation. The help text section is crucial for providing additional guidance (especially for cases where we want to ensure it understands what is factually correct), but the input field does not expand when lengthy guidance is included. This makes it difficult for users to review and confirm the full content.



    ------------------------------
    Melissa Callender
    Senior Operations Specialist
    Ontario Teachers Pension Plan
    ------------------------------



  • 11.  RE: Feedback Wanted: Your Experience with Virtual Supervisor (AI Scoring)

    Posted 8 days ago

    @Melissa Callender thank you for your feedback. We are planning to update the help text box in the forms so it is more user friendly when entering an AI Scoring prompt. 



    ------------------------------
    Jose Ruiz
    Genesys - Employees
    Product Manager
    jose.ruiz@genesys.com
    ------------------------------



  • 12.  RE: Feedback Wanted: Your Experience with Virtual Supervisor (AI Scoring)

    Posted 8 days ago
    We did a trial of both agent Copilot and supervisor Copilot.  We found the AI scoring to be the weakest aspect of Supervisor copilot.
    We found that the reliance on the transcription service to be the major issue.  We are using the Genesys native transcription service and had Genesys teams engaged to assist with tuning for the scoring but when the transcription service can't accurately capture something like the greeting which should all be the same for each agent, it makes the automated scoring not worth it.
    Ultimately at least at this point we have decided to not rely upon the AI scoring but could revisit in the future.  We are also considering using a different transcription service rather than the Genesys native transcription service


    ------------------------------
    Bob Hall
    .
    ------------------------------



  • 13.  RE: Feedback Wanted: Your Experience with Virtual Supervisor (AI Scoring)

    Posted 8 days ago

    Thank you for your feedback @Bob Hall. May I know what language/dialect is being used in your case for transcription?.

    We have made considerable updates to the Native transcription quality for core dialects and will release a net new transcription model in the coming weeks. Feel free to send over more information on the issues you faced directly to me so we can review them in detail.

    Regards,



    ------------------------------
    Jose Ruiz
    Genesys - Employees
    Product Manager
    jose.ruiz@genesys.com
    ------------------------------



  • 14.  RE: Feedback Wanted: Your Experience with Virtual Supervisor (AI Scoring)

    Posted 8 days ago

    I think its pretty good, we have tested it on a few of our calls and it seems to be pretty accurate.  we have multiple languages we support and I havent tested it on any other than English so far.  tuning the AI "agent" to get what you need makes big improvements but out of the box with some simple questions also seems to work fairly well



    ------------------------------
    Andy Jackson
    Senior Unified Communications Engineer
    ------------------------------



  • 15.  RE: Feedback Wanted: Your Experience with Virtual Supervisor (AI Scoring)

    Posted 8 days ago

    @Andy Jackson thank you for your feedback. Once you test it for more languages, please let us know how it goes!.



    ------------------------------
    Jose Ruiz
    Genesys - Employees
    Product Manager
    jose.ruiz@genesys.com
    ------------------------------