Genesys Cloud - Main

 View Only


Discussion Thread View
  • 1.  Knowledge Confidence score

    Posted 03-08-2024 06:06
    Edited by Hafid Bourich 03-08-2024 10:29

    I'm building a knowledge base with Genesys Knowledge. I have added a set of articles and when testing one of these articles It returns a confidence score of 85%.
    However, when testing the exact same question in a chatbot (architect Digital Bot Flow test tool) the bot return an outcome "Disambiguation Required".

    It seems that the bot is not sure of the answer while the answer should come from the samen knowledge base.

    Can anyone explain why the Knowledge test returns a confidence of 85% while the chatbot a Disambiguation Required? The confidence Threshold in the bot flow is 70% for Normal High.


    #ConversationalAI(Bots,AgentAssist,etc.)

    #Knowledge #knowledgeWorkbenchV2 #DigitalBots

    ------------------------------
    Hafid Bourich
    KPN B.V.
    ------------------------------



  • 2.  RE: Knowledge Confidence score

    Posted 03-09-2024 00:03

    I would assume you are using the exact same query including capitalization.  Also, are you passing this through just a knowledge base query or is it looking at Intents as well.  If you have something similar in your utterances, you could have ambiguous results if the two conflict.  



    ------------------------------
    Robert Wakefield-Carl
    ttec Digital
    Sr. Director - Innovation Architects
    Robert.WC@ttecdigital.com
    https://www.ttecDigital.com
    https://RobertWC.Blogspot.com
    ------------------------------



  • 3.  RE: Knowledge Confidence score

    Posted 03-10-2024 13:56

    Hi Robert,

    Yes it is exact the same query and I didn't configured any intent and utterances in my digital botflow



    ------------------------------
    Hafid Bourich
    KPN B.V.
    ------------------------------



  • 4.  RE: Knowledge Confidence score

    GENESYS
    Posted 03-11-2024 10:43

    When you are typing in the Knowledge Workbench test, do you see multiple articles pop up before you hit <Enter>?



    ------------------------------
    George Ganahl GCP, GCSME, ICCE, ICHD, etc.
    Senior Principal PS Consultant
    Genesys
    ------------------------------



  • 5.  RE: Knowledge Confidence score

    Posted 03-13-2024 08:13
    Edited by Hafid Bourich 03-13-2024 08:14

    Yes I see multiple articles pop up but when I press Enter the Knowledge Workbench test shows 1 article with a confidence of 85%



    ------------------------------
    Hafid Bourich
    KPN B.V.
    ------------------------------



  • 6.  RE: Knowledge Confidence score

    GENESYS
    Posted 03-13-2024 10:46

    The fact that multiple articles pop up shows that more than one article matches the search, and it is showing the auto-complete choices for the full article titles. The Knowledge Workbench test will only show you the highest ranking article when you press <enter> if you leave your original term in the input field.

    The Test in Architect asks for Disambiguation because there are multiple articles that match based upon the Confidence criteria.



    ------------------------------
    George Ganahl GCP, GCSME, ICCE, ICHD, etc.
    Senior Principal PS Consultant
    Genesys
    ------------------------------



  • 7.  RE: Knowledge Confidence score

    Posted 03-13-2024 11:04

    How should I interpret this in relation to the set confidence threshold in the digital botflow.

    As the confidence is 85% (>70%),  I wouldn't expect Architect asks for Disambiguation.



    ------------------------------
    Hafid Bourich
    KPN B.V.
    ------------------------------



  • 8.  RE: Knowledge Confidence score

    Posted 03-13-2024 18:27

    Hafid,

    I may be wrong about this, but I believe the confidence scores in Bot Flows are only related to Intent understanding, not knowledge article confidence. I remember doing tests a while back and scratching my head because the results didn't make sense. Would love confirmation (one way or another)

    Thanks,

    Peter



    ------------------------------
    Peter Stoltenberg
    TTEC Digital
    ------------------------------



  • 9.  RE: Knowledge Confidence score

    Posted 03-14-2024 11:17

    Thanks Peter.

    Hopefully a Genesys expert can provide clarity on this



    ------------------------------
    Hafid Bourich
    KPN B.V.
    ------------------------------



  • 10.  RE: Knowledge Confidence score

    GENESYS
    Posted 15 days ago

    Hi Hafid,

    The confidence scores in the User Input settings only apply to NLU based detections so they have nothing to do with knowledge just to dispel the confusion there.

    Without looking at the knowledge base, it is likely that the bot is collecting a couple of high confidence articles based on query. What I suspect is happening is that the bot is actually matching a few 'high confidence' articles. This is why disambiguation is required (George gave a good explanation of this here).

    Just to note, you can now use custom handling in Digital Bots to perform any confidence based logic you may need since you can get the search results via the field on Digital Menu / Wait For Input (the variable you use here can be deferenced to get the confidence of the articles like "Flow.searchResults[0].confidence" for example.

    https://help.mypurecloud.com/articles/control-knowledge-behavior-in-your-bot-flow/

    Hope that helps!



    ------------------------------
    Rory Brennan
    Genesys - Employees
    ------------------------------



Need Help finding something?

Check out the Genesys Knowledge Network - your all-in-one access point for Genesys resources