Genesys Cloud - Main

We have been reviewing the Genesys provided Agent CoPilot Performance Dashboard closely, and we believe we have found an issue with the stats for Wrap Up that make me question all the data, leading to concerns that it may be hallucinating and giving the impression usage is better than expected. 

Do others experience this? I think the Genesys dashboard and reporting is flawed. 

To explain, we have a CoPilot set-up where Wrap Up is toggled off as we only want to use summary. And yet the performance dashboard is showing that wrap up codes were both generated and submitted. How can this be when an advisor isn't seeing the wrap up code?

We have tested this in a nonprod environment, and we get exactly the same behaviour, so we know for sure this isn't different for the agent.

We've raised a ticket with Genesys, but as is often the case with Genesys first line support (sigh), it would appear I am the one hallucinating - I suspect not.

Hello David, 

The behavior your seeing is by design, though its not well documented. According to the agent copilot documentation. 

"If you do not want Agent Copilot to generate and store summaries, disable both Display AI powered summaries and Display AI-powered wrap up codes."

This suggests that wrap-up codes may be generated in the backend even when the option is toggled off, they just are just not visible to the agent. The dashboard appears to be tracking generation activity that happens behind the scenes. 

The dashboard tracks these generated (even though they were not displayed to the agent). 

Basically the wrap up codes are being generated in the backend but hidden from the agents, the "Generated" metric would still increment, but "Submitted" and "Feedback" should still be zero since agents can't interact with what they can't see.

If you're seeing both Generated AND Submitted counts with the display toggle off, that would indeed be a bug or indicate the system is treating backend generation differently than expected. 

Hope this helps!

We have been reviewing the Genesys provided Agent CoPilot Performance Dashboard closely, and we believe we have found an issue with the stats for Wrap Up that make me question all the data, leading to concerns that it may be hallucinating and giving the impression usage is better than expected. 

Do others experience this? I think the Genesys dashboard and reporting is flawed. 

To explain, we have a CoPilot set-up where Wrap Up is toggled off as we only want to use summary. And yet the performance dashboard is showing that wrap up codes were both generated and submitted. How can this be when an advisor isn't seeing the wrap up code?

We have tested this in a nonprod environment, and we get exactly the same behaviour, so we know for sure this isn't different for the agent.

We've raised a ticket with Genesys, but as is often the case with Genesys first line support (sigh), it would appear I am the one hallucinating - I suspect not.

Hi Cameron, thanks for the swift response.

Yes, we are seeing BOTH Generated and Submitted with numbers greater than zero. The Submitted value as you say is the more concerning number, so this has got to be a bug right?

On the Generated number, if we go by the description of Generated in the resource centre, I'm not clear it should be showing whatever is presented in the back end, even if not visible. That gives the impression the advisor should be seeing something when they aren't. 

• Generated: Number of successfully generated AI summaries on Copilot ACW where at least one AI-suggested wrap-up code was generated.

I don't follow the definition of successfully generated if the person hasn't seen them and we have it turned off. This one may I suspect be an Idea, but its poorly thought out from the team at Genesys.

Either way, the data for Submitted not being zero is very concerning, screengrab below for one advisor today showing 8 submitted despite it being toggled off:

We are paying rather handsomely for an add-on which is then giving wrong (in the case of submitted) data, or misleading data (in the case of generated). This information is being used to determine the effectiveness of the product in our business, which given the added cost we need to be sure pays for itself. Without trust in the data, it makes me question whether we can trust the data, and whether we should continue with the product.

Hello David, 

The behavior your seeing is by design, though its not well documented. According to the agent copilot documentation. 

"If you do not want Agent Copilot to generate and store summaries, disable both Display AI powered summaries and Display AI-powered wrap up codes."

This suggests that wrap-up codes may be generated in the backend even when the option is toggled off, they just are just not visible to the agent. The dashboard appears to be tracking generation activity that happens behind the scenes. 

The dashboard tracks these generated (even though they were not displayed to the agent). 

Basically the wrap up codes are being generated in the backend but hidden from the agents, the "Generated" metric would still increment, but "Submitted" and "Feedback" should still be zero since agents can't interact with what they can't see.

If you're seeing both Generated AND Submitted counts with the display toggle off, that would indeed be a bug or indicate the system is treating backend generation differently than expected. 

Hope this helps!

We have been reviewing the Genesys provided Agent CoPilot Performance Dashboard closely, and we believe we have found an issue with the stats for Wrap Up that make me question all the data, leading to concerns that it may be hallucinating and giving the impression usage is better than expected. 

Do others experience this? I think the Genesys dashboard and reporting is flawed. 

To explain, we have a CoPilot set-up where Wrap Up is toggled off as we only want to use summary. And yet the performance dashboard is showing that wrap up codes were both generated and submitted. How can this be when an advisor isn't seeing the wrap up code?

We have tested this in a nonprod environment, and we get exactly the same behaviour, so we know for sure this isn't different for the agent.

We've raised a ticket with Genesys, but as is often the case with Genesys first line support (sigh), it would appear I am the one hallucinating - I suspect not.