Genesys Cloud - Main

 View Only

Sign Up

  Thread closed by the administrator, not accepting new replies.
  • 1.  Audio Mutes during call

    Posted 11-04-2022 10:46
    No replies, thread closed.
    Hello, we have been having an issue where the call mutes at about the 15 minute mark.  The agent will put the call on hold, come back, and at the 15 minute mark, the call will mute for both the agent and customer.  The call cannot be unmuted.  Agent has to call customer back in order to resolve the issue.  PCAP doesnt show any errors.  Anyone else seeing this behavior?
    #SystemAdministration

    ------------------------------
    Justin Loritz
    Nicolet Bank
    ------------------------------


  • 2.  RE: Audio Mutes during call

    Posted 11-04-2022 23:10
    No replies, thread closed.
    I know of customers with hour-long calls with no issue.  Is this across the board or only certain agents?  Have you checked the local console logs when this happens for an agent?  What type of trunks do you have?  I know of no talk-time timers that would do this.

    ------------------------------
    Robert Wakefield-Carl
    Avtex Solutions, LLC
    Contact Center Innovation Architect
    https://www.Avtex.com
    https://RobertWC.Blogspot.com
    ------------------------------



  • 3.  RE: Audio Mutes during call

    Posted 11-07-2022 04:54
    No replies, thread closed.
    Hello Justin,

    Just to add as a reminder - if you haven't done so, please open a Product Support case with the information you have collected including the ones you may gotten from Robert's troubleshooting advice.

    ------------------------------
    Nico Feliciano
    Genesys - Employees
    ------------------------------



  • 4.  RE: Audio Mutes during call

    Posted 11-08-2022 19:16
    No replies, thread closed.
    We had this issue for months after moving to BYOC-C April 2022. We had a combination of issues actually (some of these were new under BYOCC, some were there before moving to byocc)

    (1) audio (usually from customer to agent but could go both ways) would cut out around 7.5 mins, 15mins and 30mins into some calls
    (2) interactions would randomly disconnect. 
    (3) put client on hold, consult a 3rd party agent, disconnect consult and return to client, take call off hold only to find no 2 way comms. Have to disconnect + call back.
    (4) Put client on hold, wait a period, take client OFF hold, audio not there.
    (5) Audio cuts in and out in and out on the same call, say for 2 secs or 5 secs, then comms clear again.

    We spent a couple of months on and off capturing webrtc phone trunk and main external Sip trunk diagnostics to compare against Browser console logs. At the same time out network provider had to capture their logs. When an issue was found Genesys tech would review the logs, then i would send findings over to the Network provider team for feedback and they would drill into their packet captures to find answers. 
    Most of the below findings are related to (1+2+5) above however once we narrowed down a few issues, many of the above issues have now ceased (or at least down to a more acceptable level of occurrence). We are looking at QoS on our internal wifi LAN between agent device (laptop) and switch.

    Findings: (by our network provider) 
    In calls where audio drops in 1 or both directions, they found "we aren't receiving any RTP audio on the new stream after the session refresh" (session refresh is typically at 10-15 minute intervals). In another case the audio stops sending the SRTP stream immediately after the second INVITE refresh, but it still receives RTP on the core side (and SRTP from your end). The Network provider did a switch upgrade to help resolve the the audio dropping at 15/30min intervals.

    Call Disconnect finding#1
    We found the Agent side disconnected with "480 WebRTC Disconnect" with "iceIdleDetection" as error detail. Edge conducts idledetection routinely to the agent webrtc, and it's synonymous to when audio of agent appears to have lost in near the end indicating an issue with the network connectivity by the Agent WebRTC towards the internet.

    Call Disconnect finding#2
    For the disconnecting side of the issue, we'll need your SBC Vendor/Provider on their side of the configuration and SIP traces on why do they send a SIP BYE with Reason: Q.850, cause=408;text="ReINVITE Fail" header to suddenly end the call. There might be a reINVITE they are making/attempting but fails

    Other Settings to check in GCloud.
    For the External Trunk - Set PCMA to priority higher than PCMU.  For the WebRTC Trunk we set it to opus only as it is the codec primarily for Webrtc usage. )This helped things a bit.)
    We also checked our trunk setting "Disconnect on Idle RTP". We tried if OFF to see if it helped. Technically you shouldn't need to do this if systems are working fine. 

    Genesys Cloud findings (2 call examples submitted for analysis):

    Call # 1  - we stopped receiving audio from the carrier. Whether this was due to an issue on the external caller's side or on the carrier's end I have no way of saying, you'd have to discuss this with whoever services this trunk. 

    Call # 2 - Either the agent made an error trying to take the call off hold, or they disconnected from their RTP session which caused the call to drop. Does the agent have any apps installed which could be taking over their microphone such as Teams?

    Audio issues are 99% of the time caused by some sort of network condition (packet loss, jitter, latency). What we see in the example here is evidence of a transport issue. Either the agent did not receive the packets with the proposes in them, or they did receive them and tried to respond and it didn't reach us. While not the exact same thing as loss of audio, it's just further evidence of the larger picture here.



    ------------------------------
    Simon Mckenzie
    Farmers Mutual Group
    ------------------------------



  • 5.  RE: Audio Mutes during call

    Posted 11-08-2022 20:29
    No replies, thread closed.
    Simon,

    Thanks for sharing all this - great trouble-shooting info.   Questions for you - are you now running smooth or do you still experience issues?  Was this all before or after Global Media Fabric Phase I?  Are most of your agents still at home?

    ------------------------------
    Robert Wakefield-Carl
    Avtex Solutions, LLC
    Contact Center Innovation Architect
    https://www.Avtex.com
    https://RobertWC.Blogspot.com
    ------------------------------



  • 6.  RE: Audio Mutes during call

    Posted 11-08-2022 21:20
    No replies, thread closed.

    We have 350 or so agents working in a mixed environment (remotely = 10-20% at any one time and 80-90% in the office on Dock / LAN). That mix changes at times due to lock downs of course.

    We have I would say greatly reduced but not fully eliminated the issues of audio drop outs on calls, call disconnects and calls coming off hold with audio missing.

    We continue to have trouble answering calls and have done for 3+years.

    A number of these issues started when we had Edge devices. Once we moved to BYOC-C and removed all on prem edges (1 April 2022) we noticed things were slightly faster however that introduced other issues.

    We took 4 months to work on the various issues until we have it at an acceptable level. We are now trying to run a fine tooth comb over it to improve it a little bit more if possible. Some audio is due to poor reception on cell devices (3/4 of our callers use cell approx.)

    We run windows 10 laptops / 64bit with latest chrome and edge browsers. Webrtc in CRM with interaction sync plus many staff have another tab open with the full version of G cloud there for dashboard or interaction reporting. We have recently trialled turning on QoS "EF" on WEBRTC UDP ports 16248 – 32768 on wifi traffic only from end user device to switch on Lan to at least give that leg of the traffic a bit of QOS. Having good headsets was a must with correct USB port settings. We also have MS TEAMS running on same devices and I have wondered if Teams actually interferes with webrtc in any way ??

    I have no idea about the Global Media Fabric Phase I or when that started.

     

    Regards

    Simon

      Simon McKenzie
    Genesys Cloud Technical Consultant
    Ph: 06 352 5231
    Mb: +64 27 575 3520
    Call: 0800 366 466
     
     

    FMG House, 284 - 292 Church Street, Palmerston North, 4410.

    This email and any attachments is private and confidential. If you receive this email in error, please notify us and delete the email from your system. Read our full disclosure here. FMG is a Financial Advice Provider (FAP). A full disclosure statement is available free of charge from www.fmg.co.nz or by calling 0800 366 466.






  • 7.  RE: Audio Mutes during call

    Posted 11-08-2022 21:35
    No replies, thread closed.
    I can definitely see where issues could have arisen.  First, the cell phone bit is always problematic, especially if Wi-Fi is turned on.  I usually suggest you go with Wi-Fi or mobile data, but don't turn both on when counting on VoIP on the phone.  Also, you probably went through a lot of transitions between 4G and 5G.  When they were swapping networks, my phone would go down to 2G - unacceptable for any communication.  

    Yes, headsets are a big problem.  We suggest USB-only - don't even try built-in audio or analog headsets.  

    If your LAN is mostly >1GB, the QoS is pretty much not going to be considered unless you have 200 people streaming Youtube and playing games.  

    Global Media Fabric was 9/30/2020, so you also went through that phase.  It essentially allowed WebRTC sessions to be anchored to the nearest AWS region with Genesys Cloud - enhancing the last mile quality.  Phase II adds in trunks to this mix for antitromboning of audio.  Cloud edges are idea for this architecture over local since they interfere with the direct media.  

    Teams usually only interferes with WebRTC when the two fight over the microphone and you can't talk.  Speaker audio can usually be shared.  Depends on your USB headset and the audio controls built into Windows.  

    Glad to hear most of your issues are in the past.  Hope you get help along the way.

    ------------------------------
    Robert Wakefield-Carl
    Avtex Solutions, LLC
    Contact Center Innovation Architect
    https://www.Avtex.com
    https://RobertWC.Blogspot.com
    ------------------------------



  • 8.  RE: Audio Mutes during call

    Posted 11-08-2022 22:09
    No replies, thread closed.

    The laptops in the office (250+  in one office alone) will auto switch between Lan or Wifi – whichever is giving the best path.

    Regarding mobile phones I was referring to our customer base in NZ. I analysed the ANI of all calls over a 3 month period recently and found about 75% of those calls originated from or terminated onto a cellphone. Many of them will be Rural calls since we look after a lot of Rural NZ clients. I can see what you mean with mobiles on the move switching between cell sites and moving from 5 to 4g or 4 to 3g. I hadn't thought of that.

    Our staff all use webrtc headsets and we are switching to a more USB connection from a mixture of wireless base stations and wired. Thanks for your thoughts Robert

     

    Simon ��

      Simon McKenzie
    Genesys Cloud Technical Consultant
    Ph: 06 352 5231
    Mb: +64 27 575 3520
    Call: 0800 366 466
     
     

    FMG House, 284 - 292 Church Street, Palmerston North, 4410.

    This email and any attachments is private and confidential. If you receive this email in error, please notify us and delete the email from your system. Read our full disclosure here. FMG is a Financial Advice Provider (FAP). A full disclosure statement is available free of charge from www.fmg.co.nz or by calling 0800 366 466.






  • 9.  RE: Audio Mutes during call

    Posted 11-08-2022 23:05
    No replies, thread closed.
    Glad to hear. If you are ever in need of an onsite consultant, I am willing.  Always wanted to experience NZ.

    ------------------------------
    Robert Wakefield-Carl
    Avtex Solutions, LLC
    Contact Center Innovation Architect
    https://www.Avtex.com
    https://RobertWC.Blogspot.com
    ------------------------------