Genesys Cloud - Main

 View Only

Sign Up

Back to discussions
Expand all | Collapse all

Knowledge Confidence score

  Thread closed by the administrator, not accepting new replies.

  • 1.  Knowledge Confidence score

    Posted 03-08-2024 06:06
    Edited by Hafid Bourich 03-08-2024 10:29
    No replies, thread closed.

    I'm building a knowledge base with Genesys Knowledge. I have added a set of articles and when testing one of these articles It returns a confidence score of 85%.
    However, when testing the exact same question in a chatbot (architect Digital Bot Flow test tool) the bot return an outcome "Disambiguation Required".

    It seems that the bot is not sure of the answer while the answer should come from the samen knowledge base.

    Can anyone explain why the Knowledge test returns a confidence of 85% while the chatbot a Disambiguation Required? The confidence Threshold in the bot flow is 70% for Normal High.


    #ConversationalAI(Bots,AgentAssist,etc.)

    #Knowledge #knowledgeWorkbenchV2 #DigitalBots

    ------------------------------
    Hafid Bourich
    KPN B.V.
    ------------------------------



  • 2.  RE: Knowledge Confidence score

    Posted 03-09-2024 00:03
    No replies, thread closed.

    I would assume you are using the exact same query including capitalization.  Also, are you passing this through just a knowledge base query or is it looking at Intents as well.  If you have something similar in your utterances, you could have ambiguous results if the two conflict.  



    ------------------------------
    Robert Wakefield-Carl
    ttec Digital
    Sr. Director - Innovation Architects
    Robert.WC@ttecdigital.com
    https://www.ttecDigital.com
    https://RobertWC.Blogspot.com
    ------------------------------



  • 3.  RE: Knowledge Confidence score

    Posted 03-10-2024 13:56
    No replies, thread closed.

    Hi Robert,

    Yes it is exact the same query and I didn't configured any intent and utterances in my digital botflow



    ------------------------------
    Hafid Bourich
    KPN B.V.
    ------------------------------

    Original Message


  • 4.  RE: Knowledge Confidence score

    Posted 03-11-2024 10:43
    No replies, thread closed.

    When you are typing in the Knowledge Workbench test, do you see multiple articles pop up before you hit <Enter>?



    ------------------------------
    George Ganahl GCP, GCSME, ICCE, ICHD, etc.
    Senior Principal PS Consultant
    Genesys
    ------------------------------



  • 5.  RE: Knowledge Confidence score

    Posted 03-13-2024 08:13
    Edited by Hafid Bourich 03-13-2024 08:14
    No replies, thread closed.

    Yes I see multiple articles pop up but when I press Enter the Knowledge Workbench test shows 1 article with a confidence of 85%



    ------------------------------
    Hafid Bourich
    KPN B.V.
    ------------------------------

    Original Message


  • 6.  RE: Knowledge Confidence score

    Posted 03-13-2024 10:46
    No replies, thread closed.

    The fact that multiple articles pop up shows that more than one article matches the search, and it is showing the auto-complete choices for the full article titles. The Knowledge Workbench test will only show you the highest ranking article when you press <enter> if you leave your original term in the input field.

    The Test in Architect asks for Disambiguation because there are multiple articles that match based upon the Confidence criteria.



    ------------------------------
    George Ganahl GCP, GCSME, ICCE, ICHD, etc.
    Senior Principal PS Consultant
    Genesys
    ------------------------------

    Original Message


  • 7.  RE: Knowledge Confidence score

    Posted 03-13-2024 11:04
    No replies, thread closed.

    How should I interpret this in relation to the set confidence threshold in the digital botflow.

    As the confidence is 85% (>70%),  I wouldn't expect Architect asks for Disambiguation.



    ------------------------------
    Hafid Bourich
    KPN B.V.
    ------------------------------

    Original Message


  • 8.  RE: Knowledge Confidence score

    Posted 03-13-2024 18:27
    No replies, thread closed.

    Hafid,

    I may be wrong about this, but I believe the confidence scores in Bot Flows are only related to Intent understanding, not knowledge article confidence. I remember doing tests a while back and scratching my head because the results didn't make sense. Would love confirmation (one way or another)

    Thanks,

    Peter



    ------------------------------
    Peter Stoltenberg
    TTEC Digital
    ------------------------------

    Original Message


  • 9.  RE: Knowledge Confidence score

    Posted 03-14-2024 11:17
    No replies, thread closed.

    Thanks Peter.

    Hopefully a Genesys expert can provide clarity on this



    ------------------------------
    Hafid Bourich
    KPN B.V.
    ------------------------------

    Original Message


  • 10.  RE: Knowledge Confidence score
    Best Answer

    Posted 05-30-2024 04:46
    No replies, thread closed.

    Hi Hafid,

    The confidence scores in the User Input settings only apply to NLU based detections so they have nothing to do with knowledge just to dispel the confusion there.

    Without looking at the knowledge base, it is likely that the bot is collecting a couple of high confidence articles based on query. What I suspect is happening is that the bot is actually matching a few 'high confidence' articles. This is why disambiguation is required (George gave a good explanation of this here).

    Just to note, you can now use custom handling in Digital Bots to perform any confidence based logic you may need since you can get the search results via the field on Digital Menu / Wait For Input (the variable you use here can be deferenced to get the confidence of the articles like "Flow.searchResults[0].confidence" for example.

    https://help.mypurecloud.com/articles/control-knowledge-behavior-in-your-bot-flow/

    Hope that helps!



    ------------------------------
    Rory Brennan
    Genesys - Employees
    ------------------------------



  • 11.  RE: Knowledge Confidence score

    Posted 10-02-2024 14:05
    Edited by Lee Smith 10-02-2024 14:06
    No replies, thread closed.

    Not sure if this helps in your scenario or not, Hafid, but we had a similiar question, raised it with Genesys Support and got the following answer:

    I believe the knowledge app for messenger has it's own AI and therefore uses it's own NLU and the Knowledge workbench for the digital bot flow uses it's own NLU defined within the digital bot flow. The link below describes the following about digital bot flows and how it chooses articles based on standard settings... "The participant provides an utterance or phrase. The bot checks to see if the utterance has a high confidence on one article, or on many articles. In cases of disambiguation, the digital bot flow loops through a presentation of tiles for each knowledge article, and then loops through the articles by category until the participant confirms their intent".

    Control knowledge behavior in your digital bot flow - Genesys Cloud Resource Center

    Genesys Cloud Resource Center remove preview
    Control knowledge behavior in your digital bot flow - Genesys Cloud Resource Center
    Add knowledge to your digital bot flows to create FAQ-style bots. To... [More]

    ------------------------------
    Lee Smith
    Senior Business Analyst Contract
    Genesis Energy
    lee.smith@genesisenergy.co.nz
    ------------------------------



Related Content