Workforce Engagement Management

 View Only
Discussion Thread View
  • 1.  Automatic Best Method Selection vs. Weighted Historical - Forecast Volume

    Posted 11-28-2022 10:57
    Edited by Dan Fontaine 11-28-2022 15:20
    When selecting a Route Path, the Search results show that the Route Path has about 620 calls offered over the last six weeks.

    When I use the Automatic Best Method to generate a forecast, the forecasted weekly volume comes out to be 21 calls for next week - which does not add up because the weekly volume over the last six weeks is 620 calls and the volume has been consistent over the last six weeks at about 100 calls per week.

    If I use the Weighted Historical method, the forecasted weekly volume for next week is 122 calls - which makes sense because that is in line with the actual data pattern of the last six weeks.

    Why is Automatic Best Method so far off? Or, am I doing something incorrectly here? FYI, for this test I only have one Planning Group with the one Route Path.
    #Forecasting

    ------------------------------
    Dan Fontaine
    ConvergeOne, Inc.
    ------------------------------


  • 2.  RE: Automatic Best Method Selection vs. Weighted Historical - Forecast Volume

    Posted 11-28-2022 20:26

    Hey Dan,

    While I can't answer your question I'll just note that I've tried the Auto Best option but never been satisfied with it's ability to forecast... 

    EG - you cannot remove error or unusual data (eg a public holiday) from the source data - hence the forecast can be very inaccurate.

    I prefer the granularity of the Weighted Historical - I can delete strange days and tweak volumes to my hearts content!



    ------------------------------
    Andrew Doller
    Aioi Nissay Dowa Insurance Australia Pty Ltd
    ------------------------------



  • 3.  RE: Automatic Best Method Selection vs. Weighted Historical - Forecast Volume

    Posted 11-29-2022 09:17
    Thank you for the feedback.

    When the Weighted Historical Method is used, there is the option to select the number of weeks of data that will be used in the Business Unit Forecasting tab/page. The default is 6 weeks. Out of curiosity, what number of weeks do you use?

    When you tweak the volumes, do you do that by modifying the weight (percentage) of each week's data or do you add Modifications to the forecast, or both?

    Thanks!

    ------------------------------
    Dan Fontaine
    ConvergeOne, Inc.
    ------------------------------



  • 4.  RE: Automatic Best Method Selection vs. Weighted Historical - Forecast Volume

    Posted 11-29-2022 13:43
    Just another 2 cents.  I'm not sure how many months of history you have to use for your forecast but if I am recalling correctly, we were told not to use Automatic Best Method until we had at least a year's worth of data.  The recommendation was to use the weighted historical option until we had the adequate amount of data (1 year).  Then we could better use ABM instead of weighted historical.  Not sure if that helps but remember this tidbit,

    ------------------------------
    Tom Magness
    Johnson & Johnson
    ------------------------------



  • 5.  RE: Automatic Best Method Selection vs. Weighted Historical - Forecast Volume

    Posted 11-29-2022 15:09

    Hey Dan,

    Happy to help!

    I use the full 8 weeks (i'd use more if it was available) then delete the days with out of the ordinary data - public holidays, hail events etc. Then with that as the basis I'll tweak overall volumes with modifications to suit the trends and compress handle times to remove outliers. I work on a 4 week schedule so I'll make one forecast and copy it to have 4 weeks of forecasts, and tweak them further as I see emerging trends.

    We also have email queues - and as the emails come in at all hours I also have to modify those  forecasts to have the volume of emails during our business hours rather than when they arrive.

    It's a bit fiddly, but I prefer the hands on approach so that I get a fairly accurate forecast that I know intimately. Well, as accurate as I can get for our lowish volumes (~5-6k interactions a week)

    Hey Tom,

    Yeah we have well over 2 years data now but I seem to have issues with those outlier days with ABM and as you can't really tweak them out of the forecast I've stayed with the WH. Ideally I'd love a little of the smoothing features that the ABM uses within the WH system - that would give me the best of both options.




    ------------------------------
    Andrew Doller
    Aioi Nissay Dowa Insurance Australia Pty Ltd
    ------------------------------



  • 6.  RE: Automatic Best Method Selection vs. Weighted Historical - Forecast Volume

    Posted 11-30-2022 10:40
    Edited by Dan Fontaine 11-30-2022 12:59
    So, would you say that it is also possible to use Auto Best Method (ABM) and accomplish the same result? For example, if you use ABM and we are looking at offered, you could increase or decrease the number offered using Modifications.

    I do agree that it seems that you would start with a more accurate forecast when you use Weighted Historical - if you didn't need to tweak it, you could just move on to the next step and generate a schedule. If I use ABM, for my use case, I would need to add a Modification to every forecast first to increase the percentage of offered over the interval until it looks "right" and then I would make additional Modifications for holidays etc.

    Anyway, what doesn't make sense to me is that ABM should be "smarter" - if we are truly using AI here, it should "know" that I only have so much data or recognize that if I am trending at 1000 interactions per week for the last 6 weeks it should say "well, I think you are going to continue to see that."

    I would appreciate some thoughts from Genesys on this for the community.

    ------------------------------
    Dan Fontaine
    ConvergeOne, Inc.
    ------------------------------



  • 7.  RE: Automatic Best Method Selection vs. Weighted Historical - Forecast Volume

    GENESYS
    Posted 11-30-2022 12:29
    Hi Dan,

    My name is Daniel, and I am the product manager of Forecasting on the Genesys Cloud platform. You've created an excellent discussion about our Automatic Best Method forecasts. Unfortunately, I cannot explain the behaviour you described in your initial post without being able to see your specific setup, historical data and such. For that, I suggest opening a care ticket, and we can take a deeper look and hopefully explain why ABM is off.

    Our Automatic Best Method uses ML/AI technology in a few different areas, but the underlying historical data is one of the most significant factors in getting accurate forecasts. Once we have the historical data, we apply machine learning to clean the data, find any outliers and identify calendar effects. Once we have "clean" data, we then detect the patterns in the data, things like seasonality and trends. In the final step, we take the data and push it through an AI to perform extensive cross-validations to select the best forecasting method for the data. This is a simplified explanation of the technologies used, but you can see how the data is the "foundation" of an ABM-generated forecast. Once a forecast has been created, the user can make any changes they see fit by applying a modification to the forecast; for example, you can increase the offered volume on a specific day by 10% for a holiday or even decrease it.

    As some of the comments on this thread have mentioned, there is no way for users to "tag" outliers to have the forecast ignore or pay special attention to specific historical data. However, the ML/AI should pick up on some of that naturally if there is enough historical data. It is a gap that we are aware of in our ABM forecasting tool and is something we plan to address in the future (special events as well).

    If you like, we can schedule a call together, and we can discuss ABM. I would love to hear your overall feedback on our forecasting tool, and I can give you a sneak peek of where we plan to go in the future!

    I hope this answers some of your questions and concerns about ABM.

    Thanks,

    ------------------------------
    Daniel Chapdelaine
    Genesys - Employees
    ------------------------------



  • 8.  RE: Automatic Best Method Selection vs. Weighted Historical - Forecast Volume

    GENESYS
    Posted 11-30-2022 12:41

    If you are seeing significantly higher forecast error and significantly lower forecast accuracy when using ABM compared to WHI (especially with no modifications applied to either), then I would recommend opening a support ticket so we can investigate further. If there is an issue, we can address it. If it is an area of improvement, then we can evaluate when to get it into the product.

    Note: you can hover over total offered and AHT in upper right of forecast view to get error and accuracy numbers for forecasts in the past.

    General comment about modifications: very common for forecast accuracy to be significantly higher when modifications are disabled. Not in all cases, but in a significant number mods over stock make worse forecasts.

    Also, you don't need to wait a year before using ABM. We have made several improvements since first release for 'cold start' type scenarios where you have very limited historical data.



    ------------------------------
    Jay Langsford
    VP, R&D
    ------------------------------



  • 9.  RE: Automatic Best Method Selection vs. Weighted Historical - Forecast Volume

    Posted 11-30-2022 13:53
    Thank you both. I agree that ABM, based on what I know about it, should be used, but it is way off. Per your suggestion, I have opened a ticket with Support.

    ------------------------------
    Dan Fontaine
    ConvergeOne, Inc.
    ------------------------------



  • 10.  RE: Automatic Best Method Selection vs. Weighted Historical - Forecast Volume

    Posted 11-30-2022 15:53

    Hey Daniel and Jay,

    Thanks for your additional info - it's great to hear a bit more about how it all works.

    In my case - I'd kill to have the ability to either select the raw data before the ABM does it's thing, or the ability to start with the WH, remove the spurious data and then apply smoothing and trends etc like the ABM.

    As a test case, I've created a test ABM forecast for early in the new year... A few issues I had was that as we do car insurance we have hail events that greatly blow out our queues - Using WH I'd obviously not include that data, but as the events were large and have occurred at a similar time of year the ABM included a huge kick in volumes for one of our planning groups. To fix the forecast I'd have to manually set that group to zero with a modification, whereas with WH I just delete that from the source data. Public holidays can also require modifications that the ABM doesn't quite pick up on.

    So I'm definitely interested in the future developments that help the ABM!



    ------------------------------
    Andrew Doller
    Aioi Nissay Dowa Insurance Australia Pty Ltd
    ------------------------------