Genesys Cloud - Developer Community!

 View Only

Sign Up

Back to discussions
Expand all | Collapse all

Feedback on open-source AI IVR testing tool idea

  • 1.  Feedback on open-source AI IVR testing tool idea

    Posted 05-28-2025 08:13
    Edited by Lucas Woodward 05-28-2025 08:13

    Yesterday I proposed an idea for an open-source tool for testing IVR flows/chatbots using AI that I'd love to get community feedback on.

    The LinkedIn post had some interesting comments to the post which I'll summarise here too.

    The idea

    You have a complex IVR journey that you're developing. You want to realistically test multiple scenarios during development (or at least before paying humans to test it).

    Using my open-source tool you could define multiple test scenarios (with different customer personalities) that you can unleash on your IVR, and report back with their outcomes and suggestions for improvement.

    Challenge

    There was a question in the comments as to question you'd want AI to test your IVR flows at all, or whether this really should be left to humans? After all, IVRs can be quite contentious so the more human involvement in the design and testing would be beneficial.

    My thinking around this is that there may be a place in supplementing human testing e.g. during the development process to test the more benign aspects of creating IVR flows, such as:

    • detecting broken journeys
    • incorrect assumptions about customer input
    • gracefully handling error scenarios
    • language aligns to the brand
    • invalid inputs handling
    • agent escalation
    • etc

    How it would work

    1. Define test: You tell AI what you want it to try to do to test your IVR.
    2. Call IVR: It uses Twilio Media Streams to call the IVR's phone number.
    3. Perform Test: Audio is streamed into Gemini's Live API, which responds like a real customer and navigates the IVR.
    4. Report on outcome: LLM as a Judge is used to assess test criteria against transcript
    5. Track Coverage: Genesys Cloud's Debugging APIs can be used to track test coverage (and find failure points).

    There may be some cross-over with my tool for using AI to test Web Messenger bots.

    Snazzy photo of the parts:

    Unknowns

    • How would I handle DTMF with Gemini's Live API - can metadata be returned that I can use to trigger touch tones from the App?

    • How much would Gemini Live API + Twilio Media Streams cost per test?

    • Is there an audience for an open-source tool that does this?


    #Integrations

    ------------------------------
    Lucas Woodward
    OVO Energy Ltd

    https://www.linkedin.com/in/lucas-woodward-the-dev
    ------------------------------



Related Content