Genesys Cloud - Main

 View Only

Discussion Thread View
  • 1.  Testing ChatBots with ChatGPT

    Posted 10-11-2023 05:14
    I posted this on LinkedIn, but then thought it would be great to get feedback from the Genesys community.
    I'm updating my open-source tool for testing Web Messenger chatbots that we use extensively work, to be able to use ChatGPT. The reason being is that tests are quite rigid at the moment. As the example below shows, you define what you say and the expected response, then should the chatbot deviate from this then the test will be marked as a failure:
    scenarios:
      "Borrow a book by ISBN":
        - say: hi
        - waitForReplyContaining: Welcome to 📚 Lucas's Library 📚
        - waitForReplyContaining: How can I help you?
        - say: I would like to borrow a book, can I check that you have it?
        ...


    However, by using ChatGPT we can create tests that can adapt to changes in the chatbot's behaviour/functionality. A test will then be defined by a ChatGPT prompt, which my tool uses to orchestrate a conversation between the chatbot under test and ChatGPT (video of it in action):

    scenarios:
      "Borrow a book by ISBN":
        setup:
          prompt: |
            ...
            As a customer you would like to borrow a book by its ISBN. When you say goodbye to the chatbot include the word 'PASS' in your message. If there are any errors then say 'FAIL'.
            ...
          terminatingPhrases:
            pass: ["PASS"]
            fail: ["FAIL"]

     

    The example above is rather simple, but a more advanced use-case would be to cause the test to fail if ChatGPT notices (amongst other things):

    • Errors
    • Invalid handling of inputs
    • Incorrect spelling, grammar or punctuation
    • An opportunity to improve response clarity

    Can anyone think of any other use-cases or problems I've not considered? Any feedback would be most appreciated.


    #ConversationalAI(Bots,AgentAssist,etc.)

    ------------------------------
    Lucas Woodward
    Ovo Energy Ltd
    ------------------------------


  • 2.  RE: Testing ChatBots with ChatGPT

    Posted 10-12-2023 15:26

    That's pretty cool!



    ------------------------------
    Paul Simpson
    Eventus Solutions Group
    ------------------------------



Need Help finding something?

Check out the Genesys Knowledge Network - your all-in-one access point for Genesys resources