Workforce Engagement Management

Hi @Joseph Sutich ,

Based on the documentation, I would be careful with the idea that AI Scoring "learns" from each correction as if the model were being retrained.

What Genesys Cloud does is monitor the historical accuracy of AI-generated answers based on how often human evaluators modify or override an AI-provided answer. This is calculated using a rolling window of the 25 most recent submitted evaluations for each question.

Automated scoring options in evaluation forms - Genesys Cloud Resource Center

So, if the evaluation is only left in Pending status or deleted, I would not expect it to affect that historical accuracy calculation. To impact the historical accuracy, the corrected evaluation would likely need to be submitted.

For your specific scenario, I would try to make the N/A condition the first decision point in the help text. The main goal is to avoid the AI treating "no evidence" as a negative behavior.

For example:

Question:
When a referral was applicable or referenced during the call, did the agent inform the caller to bring or obtain a referral?

Help Text:
First determine whether the referral requirement is applicable to the call.

Select N/A if:

• A referral was not mentioned during the call.

• A referral was not required for the appointment.

• The referral topic was not relevant to the caller's request.

Select Yes only if a referral was applicable or referenced and the agent informed the caller to bring the referral, obtain a referral, or contact their PCP for a referral.

Select No only if a referral was applicable or referenced and the agent failed to provide the referral instruction.

From a form design perspective, another option is to use Yes / No / N/A as the answer set and make sure N/A does not negatively impact the score, since it represents a non-applicable scenario rather than agent failure.

If your form setup allows it, you could also consider setting N/A as the default answer. Then AI Scoring would only change the answer to Yes or No when there is enough evidence in the interaction. If there is not enough evidence, the answer would remain N/A instead of being interpreted as No.

Hi Mateus,

Thank you so much for your prompt and detailed response. I will take your suggestions and rework my help text. 

Hi @Joseph Sutich ,

Based on the documentation, I would be careful with the idea that AI Scoring "learns" from each correction as if the model were being retrained.

What Genesys Cloud does is monitor the historical accuracy of AI-generated answers based on how often human evaluators modify or override an AI-provided answer. This is calculated using a rolling window of the 25 most recent submitted evaluations for each question.

Automated scoring options in evaluation forms - Genesys Cloud Resource Center

So, if the evaluation is only left in Pending status or deleted, I would not expect it to affect that historical accuracy calculation. To impact the historical accuracy, the corrected evaluation would likely need to be submitted.

For your specific scenario, I would try to make the N/A condition the first decision point in the help text. The main goal is to avoid the AI treating "no evidence" as a negative behavior.

For example:

Question:
When a referral was applicable or referenced during the call, did the agent inform the caller to bring or obtain a referral?

Help Text:
First determine whether the referral requirement is applicable to the call.

Select N/A if:

• A referral was not mentioned during the call.

• A referral was not required for the appointment.

• The referral topic was not relevant to the caller's request.

Select Yes only if a referral was applicable or referenced and the agent informed the caller to bring the referral, obtain a referral, or contact their PCP for a referral.

Select No only if a referral was applicable or referenced and the agent failed to provide the referral instruction.

From a form design perspective, another option is to use Yes / No / N/A as the answer set and make sure N/A does not negatively impact the score, since it represents a non-applicable scenario rather than agent failure.

If your form setup allows it, you could also consider setting N/A as the default answer. Then AI Scoring would only change the answer to Yes or No when there is enough evidence in the interaction. If there is not enough evidence, the answer would remain N/A instead of being interpreted as No.

Hi Mateus,

Thank you so much for your prompt and detailed response. I will take your suggestions and rework my help text. 

Hi @Joseph Sutich ,

Based on the documentation, I would be careful with the idea that AI Scoring "learns" from each correction as if the model were being retrained.

What Genesys Cloud does is monitor the historical accuracy of AI-generated answers based on how often human evaluators modify or override an AI-provided answer. This is calculated using a rolling window of the 25 most recent submitted evaluations for each question.

Automated scoring options in evaluation forms - Genesys Cloud Resource Center

So, if the evaluation is only left in Pending status or deleted, I would not expect it to affect that historical accuracy calculation. To impact the historical accuracy, the corrected evaluation would likely need to be submitted.

For your specific scenario, I would try to make the N/A condition the first decision point in the help text. The main goal is to avoid the AI treating "no evidence" as a negative behavior.

For example:

Question:
When a referral was applicable or referenced during the call, did the agent inform the caller to bring or obtain a referral?

Help Text:
First determine whether the referral requirement is applicable to the call.

Select N/A if:

• A referral was not mentioned during the call.

• A referral was not required for the appointment.

• The referral topic was not relevant to the caller's request.

Select Yes only if a referral was applicable or referenced and the agent informed the caller to bring the referral, obtain a referral, or contact their PCP for a referral.

Select No only if a referral was applicable or referenced and the agent failed to provide the referral instruction.

From a form design perspective, another option is to use Yes / No / N/A as the answer set and make sure N/A does not negatively impact the score, since it represents a non-applicable scenario rather than agent failure.

If your form setup allows it, you could also consider setting N/A as the default answer. Then AI Scoring would only change the answer to Yes or No when there is enough evidence in the interaction. If there is not enough evidence, the answer would remain N/A instead of being interpreted as No.