Genesys Cloud - Main

 View Only

Sign Up

Expand all | Collapse all

Outbound routes got disabled

  Thread closed by the administrator, not accepting new replies.
  • 1.  Outbound routes got disabled

    Posted 10-30-2018 10:06
    No replies, thread closed.
    Around 14:00 CET today all our outbound routes was disabled and we were not able to transfer calls or call out. We have two sites and both of them was disabled so we are quite sure this was not a human error. 

    Anyone else who experienced this as well?
    #PlatformAdministration

    ------------------------------
    Tobias Zackrisson
    Elkjop Nordic AS
    ------------------------------


  • 2.  RE: Outbound routes got disabled

    Posted 10-30-2018 10:09
    Edited by George Ganahl 10-30-2018 10:09
    No replies, thread closed.
    @Ann Wright in Australia said they had the same thing a couple of days ago.

    https://community.genesys.com/communities/community-home/digestviewer/viewthread?GroupId=19&MessageKey=cc82be3a-caa1-427b-9a69-83740f17ae1f&CommunityKey=bab95e9c-6bbe-4a13-8ade-8ec0faf733d4&tab=digestviewer&ReturnUrl=%2fcommunities%2fcommunity-home%2fdigestviewer%3fcommunitykey%3dbab95e9c-6bbe-4a13-8ade-8ec0faf733d4%26tab%3ddigestviewer

    What type of trunks do you have o the Telco side?

    ------------------------------
    George Ganahl CCXP, GCA
    Principal Program Manager
    Genesys
    ------------------------------



  • 3.  RE: Outbound routes got disabled

    Posted 10-30-2018 10:13
    No replies, thread closed.
    Hi ... nothing showing up here PureCloud by Genesys Status
    Mypurecloud remove preview
    PureCloud by Genesys Status
    Welcome to PureCloud by Genesys's home for real-time and historical data on system performance.
    View this on Mypurecloud >
    as of yet anyhow - nothing on site here either... regards Matt

    ------------------------------
    Matt Calton
    Grove & Dean Ltd
    ------------------------------



  • 4.  RE: Outbound routes got disabled

    Posted 10-30-2018 14:57
    No replies, thread closed.
    Interesting. We had this a couple of days ago. We were making routing changes to some edges and attributed it to that.
    Found out that just enabling the Outbound Routes doesn't help, you have to change something else (like the description) for it to save the settings.

    ------------------------------
    Sven Schiller
    Kognitiv
    ------------------------------



  • 5.  RE: Outbound routes got disabled

    Posted 10-31-2018 19:30
    No replies, thread closed.
    Had this occur overnight, unable to enable outbound routes it auto defaults to disabled. While our services are still operating there are some functions failing such as ANI/DNIS information in Interactions not appearing for Outbound dialer services.

    Have raised a ticket with support, also have another issue affecting numbers plans.

    Is there any workarounds, I only discovered it from troubleshooting an update from our carrier this morning?


    ------------------------------
    Darryn Chang
    Stuff Limited
    New Zealand
    2 Years Purecloud
    ------------------------------



  • 6.  RE: Outbound routes got disabled

    Posted 10-31-2018 19:37
    No replies, thread closed.
    Hi Darryn... Sven has the workaround nailed in his text below... Go to the outbound route section change something in the description field and save boom ..... Done.

    Matt

    sent from my from Android so forgive me any brevity or typos






  • 7.  RE: Outbound routes got disabled

    Posted 11-01-2018 06:01
    No replies, thread closed.
    We've also had this issue occur overnight.  The outbound routes on all sites were disabled, we could still make calls but the caller id was wrong.  Have now enabled the outbound routes on each site.


  • 8.  RE: Outbound routes got disabled

    Posted 11-01-2018 08:10
    No replies, thread closed.
    @Darryn Chang our ANI and DNIS case is opened too, noticed that around 0545 CT yesterday. ​Our outbound routes are all disabled as well. Trying this fix listed now.

    ------------------------------
    Angelia Harper
    Avtex
    ------------------------------



  • 9.  RE: Outbound routes got disabled

    Posted 11-01-2018 10:39
    No replies, thread closed.
    This has just happened again across 11 separate PureCloud orgs.  Customer Care are saying this is a known issue.

    @Cass Sapir why is this not on the status page? ​


  • 10.  RE: Outbound routes got disabled

    Posted 11-01-2018 10:49
    No replies, thread closed.
    Very good point Braiden - and I bet you had to push hard to get that out of them ..... sad to say, but I honestly believe secrecy wins over transparency. @Cass Sapir this needs to change​. Human nature is such that we tend to remember the worst experiences more than a whole load of great ones.

    ------------------------------
    Matt Calton
    Grove & Dean Ltd
    ------------------------------



  • 11.  RE: Outbound routes got disabled

    Posted 11-01-2018 11:11
    No replies, thread closed.
    All of our orgs are also now missing the ANI / DNIS information for 1st Sept - 30 Oct.  Case raised with Care.


  • 12.  RE: Outbound routes got disabled

    Posted 11-01-2018 12:15
    No replies, thread closed.

    Hello all, I have spun up an internal chat group to investigate and triangulate similar reports / cases around this. If you have case #s related to this please add them to the thread here for efficient internal investigation.

    Thanks all for the messages and the information.

    -Cass



    ------------------------------
    Cass Sapir
    VP, Customer Success
    PureCloud
    ------------------------------



  • 13.  RE: Outbound routes got disabled

    Posted 11-01-2018 12:41
    Edited by Cass Sapir 11-01-2018 12:41
    No replies, thread closed.

    Hello all, we have identified the likely root cause of this issue. The impact seems to be scoped to a small percentage of customers. This thread will serve as the best one-to-many communication update. In addition, we are identifying similar Customer Care cases that may be affected and will push further updates via those cases.

    Development is currently aware of the issue and tracking under <g class="gr_ gr_108 gr-alert gr_spell gr_inline_cards gr_run_anim ContextualSpelling ins-del multiReplace" id="108" data-gr-id="108">SERVOPS</g>-5023. They are working on a fix at high priority. No ETA available yet. I will update this thread with the ETA when available.

    Thanks all for the feedback and details you are sharing.



    ------------------------------
    Cass Sapir
    VP, Customer Success
    PureCloud
    ------------------------------



  • 14.  RE: Outbound routes got disabled

    Posted 11-01-2018 16:49
    Edited by Darryn Chang 11-01-2018 16:50
    No replies, thread closed.
    The unfortunate experience with this was yesterday I rang the International Support number for APAC and twice it reached the end of the recorded message on your IVR then remained connected but no one answered. It wasn't until I reached out to the assigned case manager via Chat that I got a response. 

    I raised 3 issues in my case #0002396266 one being the ANI/DNIS not appearing (this has pushed our org to manually look into alternative avenues for escalations, unfortunately that drives a poor customer experience for our org)

    2. Outbound Route disabled - Although the fix in the above threads may of worked for some unfortunately it failed to work for us.
    3. Its gone on to affect some instances in the default number plan, which 1 important aspect being "Emergency" is not able to be called from any of our Purecloud phones.

    @Cass Sapir Lastly being advised that as the TEST ORG in support is based in the US Region there was nothing support could do as I reside in APAC and there were no TEST instances available to support as they had to wait for their colleagues in APAC to test the issue. I find this highly doubtful as in the past before the current support model US based support engineers were able to access not only APAC region orgs but had the environments available to them. This has added to our frustration


    ------------------------------
    Darryn Chang
    Stuff Limited
    New Zealand
    2 Years Purecloud
    ------------------------------



  • 15.  RE: Outbound routes got disabled

    Posted 11-01-2018 13:44
    No replies, thread closed.
    The ANI/DNIS problem is fixed in the code and should roll out overnight, if all goes as planned.

    ------------------------------
    George Ganahl CCXP, GCA
    Principal Program Manager
    Genesys
    ------------------------------



  • 16.  RE: Outbound routes got disabled

    Posted 11-02-2018 05:23
    No replies, thread closed.
    Thanks @George Ganahl , the ANI / DNIS information is available again.  A shame our care tickets weren't also updated to say it was fixed and we are having to rely on the forums for support. 

    @Cass Sapir I still think this should've been on the status page, I don't believe it was a small number of customers affected.   We have 11 orgs on the EMEA Germany region and they were all affected. 

    ​​​


  • 17.  RE: Outbound routes got disabled

    Posted 11-02-2018 05:32
    No replies, thread closed.
    Strongly agree with Braiden on the publication on the status page... this is the first place I went to at the beginning of this.

    ------------------------------
    Matt Calton
    Grove & Dean Ltd
    ------------------------------



  • 18.  RE: Outbound routes got disabled

    Posted 11-02-2018 06:19
    No replies, thread closed.
    Thank <g class="gr_ gr_580 gr-alert gr_gramm gr_inline_cards gr_run_anim Punctuation only-ins replaceWithoutSep" id="580" data-gr-id="580">you</g> @Braiden Woodward and @Matthew Calton for the feedback. We will bring that feedback forth to our Incident management workstream which is part of our overall Support experience initiative.

    And @Darryn Chang, thank you for the detail on your experience as well. I will inquire about the ability to have a TEST ORG in APAC region for more rapid collaboration and investigation.



    ------------------------------
    Cass Sapir
    VP, Customer Success
    PureCloud
    ------------------------------



  • 19.  RE: Outbound routes got disabled

    Posted 11-02-2018 08:06
    No replies, thread closed.
    Thank you @George Ganahl our ANI and DNIS issues were resolved yesterday, however, it looks like the ones missing from the 31st are still not showing up. I will be double checking this morning. If they are still missing, will they be able to recover this data? ​I will post to our case as well.

    ------------------------------
    Angelia Harper
    Avtex
    ------------------------------



  • 20.  RE: Outbound routes got disabled

    Posted 11-02-2018 09:16
    No replies, thread closed.
    Angie, from what I gather they did cleanup for 11/1, and will do another run to clean up data for 10/31. So, hopefully you will see that data soon.

    ------------------------------
    George Ganahl CCXP, GCA
    Principal Program Manager
    Genesys
    ------------------------------



  • 21.  RE: Outbound routes got disabled

    Posted 11-02-2018 12:28
    No replies, thread closed.
    Just noticed that our Outbound Routes go disabled again. What's triggering this? Is there an ETA for a fix yet?

    ------------------------------
    Sven Schiller
    Kognitiv
    ------------------------------



  • 22.  RE: Outbound routes got disabled

    Posted 11-03-2018 01:40
    No replies, thread closed.
    @George Ganahl, @Cass Sapir - The ANI/DNIS numbers not appearing has restored in my org based from APAC region up until 01/11 but is not present for the 02/11 for outbound dialer contacts. Will this be cleaned up as well? 
    I won't know until Monday ​​5th (4th in your timezone) if the dialer outbound contacts have fully restored in the Interactions view for ANI/DNIS

    Purecloud reports for Detailed Outbound attempt history were also affected by the ANI/DNIS issue as well.


    ------------------------------
    Darryn Chang
    Stuff Limited
    New Zealand
    2 Years Purecloud
    ------------------------------



  • 23.  RE: Outbound routes got disabled

    Posted 11-04-2018 17:32
    No replies, thread closed.
    APAC customer still affected by ANI/DNIS not appearing in real time. Outbound Dialer has blank results for reviewing Interactions and Reports. @Cass Sapir

    Guys check your orgs, while the advise has been that the issue was resolved I have noticed a minimum of 24-48 hour wait time before the ANI/DNIS information will appear in Purecloud.

    Again rang support to advise but have to wait for a response



    ------------------------------
    Darryn Chang
    Stuff Limited
    New Zealand
    2 Years Purecloud
    ------------------------------



  • 24.  RE: Outbound routes got disabled

    Posted 11-05-2018 06:14
    No replies, thread closed.
    Hi @Darryn Chang sorry to hear about this. We will investigate internally and be back directly with more info. -Cass

    ------------------------------
    Cass Sapir
    VP, Customer Success
    PureCloud
    ------------------------------



  • 25.  RE: Outbound routes got disabled

    Posted 11-05-2018 22:45
    No replies, thread closed.
    @Darryn Chang If I saw correctly earlier today, they are still working on addressing ANI/DNIS for outbound dialing, so it may be a few days yet before you see the data.

    I will be slow to respond this next couple of weeks; I have to ​concentrate on my primary job :-)

    ------------------------------
    George Ganahl CCXP, GCA
    Principal Program Manager
    Genesys
    ------------------------------



  • 26.  RE: Outbound routes got disabled

    Posted 11-02-2018 15:41
    No replies, thread closed.

    Hi all,

    On Thursday Nov 1st, a bug was introduced into how the outbound routes were being displayed. This bug improperly displays that outbound routes were disabled even though outbound calling is working. It primarily impacts any PureCloud customers who created or updated outbound routes between October 20th and October 31st. The development team is currently working on a fix to this bug and are targeting a release for no later than Wednesday, Nov. 7th.

    The following steps should be taken if you need to view or modify your outbound routes in the PureCloud Admin UI.

    1. If you login to the administration console and see outbound routes are marked as "disabled" take the following action before saving or leaving the outbound routes screen:
    a. Set all desired outbound routes in the UI from disabled to an enabled status
    b. Immediately save your outbound routes
    c. This will save the state of your routes properly

    2. Please do not save your outbound routes if you have not moved them to an enabled status as documented in step #1 or you will cause your outbound routes to be moved out of service.

    We apologize for any inconvenience this might cause you. Our development teams are working as quickly and safely as possible to place fixes into production.



    ------------------------------
    Chrissy Linzy
    Genesys - Employees
    ------------------------------