Workforce Engagement Management

Discussion Thread View
Expand all | Collapse all

Short term forecasting -difference in metrics between Automatic best method and weighted index

  • 1.  Short term forecasting -difference in metrics between Automatic best method and weighted index

    Posted 28 days ago
    Hi @Jay Langsford,

    We are ​in the process of onboarding our client on WFM. They have 2 business units, 1 management unit per each,1 media (voice only) with approx 100 Queues, 12 Skills, and 4 Languages. This client is PC-3 client and on Genesys Cloud since November 2019. So, no issue with the historical data for the forecasts.


    The total number of possible route paths for all the planning groups is: 275.

    We started with the configuration for one BU and one MU under it with approx 60 Queues, 6 Skills, and 4 Languages. Number of agents for this MU is 24. We followed Genesys recommendation regarding route paths configuration and considering the amount of possible combination we decided to aggregate the similar route paths for one language under one planning group. Please note, we excluded the routes with <none> for Language and/or Skill and didn't add any manual route paths, we selected only the ones offered by Genesys (Search button). The planning groups are aggregated based on Queue + Skill combo for one language that best suits their business needs. One detail worth mentioning is: with the exception to one planning group we have planning groups with route paths with different volumes.


    When we run short term forecasts for the next week July 12th (BU Start Week of Day is Sunday) using the Automatic best method and Weighted historical index (6 weeks) we are getting huge difference in the metrics. Here are the details:

    Weighted index
    Total offered: 765.5
    AHT 175.01seconds

    Automatic best method 

    Total offered: 283
    AHT: 278.17seconds

    For the planning group that is not an aggregate i.e. the planning group has one Queue, Skill, and language, the difference in the forecast for it is:

    Weighted index: 
    Total offered: 112
    AHT: 241 seconds

    Automatic best method 
    Total offered: 26
    AHT: 298 seconds

    NOTE: AHT is given here as an integer.

    Can you please give us more insights/ideas on what may be the root cause of these discrepancies and recommendations on what to change/adjust in our configuration to have the forecasts from these two methods more aligned.

    Thanks,


    #ForecastingandScheduling/Decisions

    ------------------------------
    Tatjana Knezevic
    Star Telecom
    ------------------------------


  • 2.  RE: Short term forecasting -difference in metrics between Automatic best method and weighted index

    GENESYS
    Posted 28 days ago
    See this thread and my reply for more information regarding disparity in the two - especially for 'cold start' case.

    ------------------------------
    Jay Langsford
    Senior Director, Workforce Optimization Engineering
    ------------------------------



  • 3.  RE: Short term forecasting -difference in metrics between Automatic best method and weighted index

    Posted 28 days ago
    Edited by Tatjana Knezevic 28 days ago

    Hi Jay,

    This client has the Org and therefore historical data since Nov 2019. This is the time when almost all Queues and Skills were created and configured. Ever since the start they maybe added 2-3 new Queues. No other changes to the configuration objects that would affect routing paths.

    Thanks,

    Tatjana 



    ------------------------------
    Tatjana Knezevic
    Star Telecom
    ------------------------------



  • 4.  RE: Short term forecasting -difference in metrics between Automatic best method and weighted index

    GENESYS
    Posted 28 days ago
    Not something I can or technically should troubleshoot via Community discussion thread. I would recommend opening a support ticket with Care. Information on Care along with escalation management can be found here: https://help.mypurecloud.com/articles/contact-purecloud-care/

    If you look in intraday monitoring how do actuals compare in regards to offered and AHT? Specifically is it closer to weighted historical index (WHI) or ABM forecast? You can also generate/view a forecast in the past and turn on 'Show Actuals' in display options for another way to compare forecast versus actuals.

    In general for WHI method, less weeks is better (e.g., 4 instead of 8) and good to look at one without any modifications and equal weighting of past weeks (versus one with several modifications and/or potentially counterproductive weighting scheme (e.g., 1,1,1,1 instead of 1,2,5,10).

    ------------------------------
    Jay Langsford
    Senior Director, Workforce Optimization Engineering
    ------------------------------



  • 5.  RE: Short term forecasting -difference in metrics between Automatic best method and weighted index

    Posted 28 days ago
    Hi Jay,

    We opened the ticket re this on July 1st (Priority High) and yet no response and no resolution from the Team. We can't compare in Intraday monitoring as we didn't publish the schedule yet.

    As for WHI we are going back for 6 weeks. We had a couple of tries and the forecast with 6 looked good.

    Here is how actuals vs ABM forecast look like for yesterday and today:


    Thanks,



    ------------------------------
    Tatjana Knezevic
    Star Telecom
    ------------------------------



  • 6.  RE: Short term forecasting -difference in metrics between Automatic best method and weighted index

    GENESYS
    Posted 27 days ago

    To escalate a Care case please refer to the "Global case escalation management" section in the following article: https://help.mypurecloud.com/articles/contact-purecloud-care/.

    I will tell you that forecasting for such a low volume grouping will always be stochastic and not accurate. Really anything < 10 interactions per 15m is extremely low especially if there really isn't any true pattern discernible. This is the case for any forecasting method...not just the ~18 or so we employ today.

    Example you had 6 interactions for a particular planning group from 9:15am to 9:30am one out of the past 4 weeks, one week with 0 interactions, one week with 2 interactions, and one week with 4 interactions. Simple WHI with equal weighting for the four weeks would forecast volume at 3 interactions. Forecast accuracy for this one interval would be (actual interactions -> forecast accuracy):

    • 0 -> 0%
    • 1 -> 33.33%
    • 2 -> 66.67%
    • 3 -> 100.00%
    • 4 -> 66.67%
    • 5 -> 33.33%
    • 6 -> 0%
    I will contend that there is just not enough volume to accurately forecast volume nor AHT. Now in my example if 9:15am to 9:30am _always_ had 6 interactions, we'd nail it with either method.

    I would recommend engaging @Chip Funk and his Professional Services Business Consultants group. ​

    ------------------------------
    Jay Langsford
    Senior Director, Workforce Optimization Engineering
    ------------------------------



  • 7.  RE: Short term forecasting -difference in metrics between Automatic best method and weighted index

    GENESYS
    Posted 27 days ago
    Edited by Chip Funk 27 days ago
    Just quickly reading through this thread, I think there is a larger fundamental issue that should be examined and addressed
    (assuming I understand this comment correctly).

    "We started with the configuration for one BU and one MU under it with approx 60 Queues, 6 Skills, and 4 Languages. Number of agents for this MU is 24."

    This sounds to me like a routing consolidation challenge.  Sixty queues handled by just 24 agents will drive challenge in your planning process.  Why so many queues for so few agents?  What has driven the creation of so many queues?  It appears to me there is value in taking a step back and evaluating if there are options to simplify routing/agent selection to improve the planning configuration.  


    ------------------------------
    Chip Funk
    Manager, Business Consulting
    Genesys - Employees
    ------------------------------



  • 8.  RE: Short term forecasting -difference in metrics between Automatic best method and weighted index

    Posted 27 days ago
    Hi @Chip Funk,

    The reason for the number of Queues is related to the nature of their business. The number of Queues is determined by the number of different LOBs they have for multiple clients. I don't think we can make any consolidation in that regard as the current model is applicable to their business (used for their billing as well to charge the clients).
    In my first post, I pointed out that we do have routing groups with different volumes as we organized the routing groups per their business needs.
    For example,  in one routing group we have the following routing paths with the Number of Offered calls (routing paths and volumes are based on the Offered in last 6 weeks)   :
    Queue 1 + Skill 1 + English = 205
    Queue 1 + Skill 2 + English = 30

    As you can see the ratio is 7:1, and maybe it affects the forecasting error. Should we exclude low volume paths from this in order to "correct" this? Can you please share your inputs?

    However, what  absolutely puzzles me is the routing group configured as:
    Queue  + Skill  + English with 670 Offered calls in the last 6 weeks. This routing group has totally different forecast results when we ran it using two forecast methods (see below):

    Weighted index: 
    Total offered: 112
    AHT: 241 seconds

    Automatic best method 
    Total offered: 26
    AHT: 298 seconds

    Thanks




    ------------------------------
    Tatjana Knezevic
    Star Telecom
    ------------------------------



  • 9.  RE: Short term forecasting -difference in metrics between Automatic best method and weighted index

    GENESYS
    Posted 27 days ago
    670 interactions in the past 6w is sooo small of volume. Let's say you are only open 5d a week for 8 hours which would mean 960 15m buckets. That would mean if interactions were equally spaced out 0.698 interactions per 15m. Equally spaced out every minute it would be 0.0465 interactions per minute. Even hourly it would be too low to expect an accurate forecast: 2.79 per hour.

    To me the differences in the two forecasting methods are within reason given the super small numbers - again you can expect this to be stochastic. Any forecasting method will have a high rate of forecast error.

    In general the more homogeneous your load and staff is the easier it is for forecasting and scheduling (e.g., fewer planning groups, fewer unique set of agent capabilities). The more heterogenous your load and staff is, the harder it is for forecasting and scheduling.

    This is where consulting is needed to bridge the gap between the business side and WFM side. WFM supporting super fractured/fragmented super low volume load is simply untenable.

    Surely the fractured/fragmented way the planning groups have been configured don't reflect how agents are skilled, right? Agents likely show up as associated agents in more than one planning group.

    I think what you want to get at is finding sets of agents that show up in multiple planning groups and see if you could combine planning groups. Just note that an agent must be able to handle every route path in the planning group.

    ------------------------------
    Jay Langsford
    Senior Director, Workforce Optimization Engineering
    ------------------------------



  • 10.  RE: Short term forecasting -difference in metrics between Automatic best method and weighted index

    Posted 27 days ago
    Hi @Jay Langsford,

    Thank you. Yes, we are aware of the fact in order for an agent to be assigned to one planning group she/he must have assigned all Queues/Skills from that routing group (assuming here the agent has the Language configured for the planning group). It sounds like: ALL or NOTHING rule :-)

    True, we have multiple planning with almost all the agents from the MU. However, probably we'll exclude some low volume routes from the forecasting completely and we'll create more planning groups, however, have to be careful here as planning groups are used for intra-day monitoring as well.

    I see about the volume, however, it is what it is. Can't do anything about it.

    Now, there is a "bitter" part of the whole story and that is to deliver the news to the client that most likely they want to be able to produce a multi-week schedule.

    Thanks

    ------------------------------
    Tatjana Knezevic
    Star Telecom
    ------------------------------



  • 11.  RE: Short term forecasting -difference in metrics between Automatic best method and weighted index

    Posted 27 days ago
    @Jay Langsford

    Thank you for your responses. Based on the forecast data produced it looks like the only forecast method we can use is WHI.
    Unfortunately, probably due to call arrival pattern, routing paths, and volume that may fluctuate it looks like ABM can't produce a reliable forecast.

    One of the problems we have here is that with WHI forecast we can produce a schedule for 1 week only. This limitation is not something we expected to be the possible outcome of the WEM feature.

    Thanks,


    ------------------------------
    Tatjana Knezevic
    Star Telecom
    ------------------------------



  • 12.  RE: Short term forecasting -difference in metrics between Automatic best method and weighted index

    GENESYS
    Posted 26 days ago
    Edited by Chip Funk 26 days ago
    Hi Tatjana,

    Understood.  But based on the comments here it appears as though routing and agent selection have been designed to drive reporting to facilitate billing.  As a result, there are now a plethora of workgroups for a small group of agents.  Determining if there are options to simplify routing and meet the needs for reporting and billing is likely a bigger discussion and would require change management.  The value of simplification while still meeting the reporting needs via some method other than routing, will yield value in administration, center management, planning and performance.

    ------------------------------
    Chip Funk
    Manager, Business Consulting
    Genesys - Employees
    ------------------------------



  • 13.  RE: Short term forecasting -difference in metrics between Automatic best method and weighted index

    Posted 26 days ago
    Hi @Jay Langsford and @Chip Funk,

    Thank you so much for your prompt and valuable responses.
    Before I go back to the client and explaining that due to the fragmentation to meet the business needs and a limited number of the agents, automated forecasts & schedules may not be the solution and instead to use manual schedule creation, I have more questions:

    - What is the interval for Service Goal Templates? Are they values based on a 15-minute interval? Is there a possibility to have this configurable? What would be your recommendation on how to adjust Service Level goals for the template as our client reports on SL on a daily basis (Interactions Answered, Within Time Period)?

    - From the previous posts in this thread, I explained the fragmentation of the routing configuration (multiple Queues, Skills) to best match client business needs. When we are aggregating the routing paths within one planning group what would be the minimum value to go with the Offer for the routing paths? At the moment there are routing paths with the following volume in the last 6 weeks (see below, we blurred  Queues/Skills and Planning groups). Should we include only the routes with X calls in the last 6 weeks, and what X would be (of course all the agents from the MU have all the queues/skills/language required by the planning group)?





    Thank you :-)


    ------------------------------
    Tatjana Knezevic
    Star Telecom
    ------------------------------



  • 14.  RE: Short term forecasting -difference in metrics between Automatic best method and weighted index

    GENESYS
    Posted 26 days ago
    Service goals are attempted to be met each 15m interval from a staffing perspective. In general you really wouldn't want these specified at a daily level as a goal for staffing because you want a customer to have a similar experience regardless of when they called in. E.g., if your goal is 80% answered in 20s for the day, I could call in at 8am in the morning and wait 5m or call in at 1pm and wait 30s or 3 and wait 1s. It's fine if the business reports on things at the daily, but consistency of service performance should be considered. If you find the service goals are resulting in too good of a daily level you could always make them more pessimistic (e.g., 80% in 30s); although, I wouldn't recommend that.

    I don't have a rule on what a minimum volume would be, but I would say any planning group that gets less than 10 interactions per hour (with some regularity and can have some discernable pattern weekly, daily, hourly, quarter-hourly - some sparsity intraday or intray week is fine) each week is going to be challenging on the forecasting side. Again, I would defer to the Consultants since they help consult on these types of things.

    ------------------------------
    Jay Langsford
    Senior Director, Workforce Optimization Engineering
    ------------------------------



  • 15.  RE: Short term forecasting -difference in metrics between Automatic best method and weighted index

    Posted 26 days ago
    Hi @Jay Langsford,

    Most service providers/BPOs have monthly targets for SLA. Although I agree with you philosophically, the reality of commercial agreements that most of our BPO clients have is that SVL is a monthly target. 15-minute view of hitting this target, particularly in low volume scenarios, creates unrealistic scheduling forecasts​.

    Thanks,
    Tatjana

    ------------------------------
    Tatjana Knezevic
    Star Telecom
    ------------------------------



  • 16.  RE: Short term forecasting -difference in metrics between Automatic best method and weighted index

    GENESYS
    Posted 26 days ago
    Edited by Jay Langsford 26 days ago

    Feel free to create an idea for product management's review for being able to specify durations other than interval at https://purecloud.ideas.aha.io/

    I think the possibility of having terrible service levels for hours, days, and even weeks to only squeak in just under for a less granular goal like daily, weekly, or monthly is bad practice. I understand contracts are written with SLAs in less granular terms, but it doesn't mean you can't specify more consistent interval goals (and possibly adjust slightly to be more pessimistic if you find daily, weekly, monthly actuals are too good).

    Think about customer satisfaction where you could have customers comment "every time I call in, regardless of day or time, my call gets answered in less than 2m" versus "I called in one time and hung up after waiting 20m, another time I waited 15m until I got an agent, and another I waited 7m, but once I got answered in 20s. I don't think they know how to staff their contact center".

    From a staffing perspective would it be ok if we staffed only 10h of a 12h day, assuming schedule flexibility, and just let interactions essentially drop on the floor via 100% abandonment the other 2h as long as we hit daily service goals? The answer I am sure is no, but with less granular goals it brings that up as a potential issue. One that might need some configurable countermeasure (like setting at least a secondary interval level worst-case floor goal we'd have to meet).



    ------------------------------
    Jay Langsford
    Senior Director, Workforce Optimization Engineering
    ------------------------------



  • 17.  RE: Short term forecasting -difference in metrics between Automatic best method and weighted index

    GENESYS
    Posted 26 days ago
    I think your issue is the number of planning groups.  It sounds like you have over 200 planning groups for 24 agents?  If this is correct, you should easily be able to consolidate planning groups.  What is likely happening is ABM is seeing intermittent volumes in all these PGs and causing low forecasts.  So look at which agents have the same queue skill combinations and group those RPs into 1 PG.

    ------------------------------
    Daniel Rickwalder
    Genesys
    ------------------------------



  • 18.  RE: Short term forecasting -difference in metrics between Automatic best method and weighted index

    Posted 26 days ago
    Hi @Daniel Rickwalder,

    The number of routing paths on the system is 257 (returned by the Search option). However, for BU's I aggregated only high volume routes (high volume is 100 + Offered) and for example for one of BU's I have only 4 planning groups with 36 routing paths. All the agents from the MU are members of all 4 groups. Please note, route paths with <none> for Skill and/or Language are excluded from the planning groups.

    When I ran ABM forecast I'm getting quite skewed forecast. For example for the next week the metrics are:
    Offered: 2236
    AHT: 1.2 seconds

    Below is the output of ABM selected methods for all 4 routes and corresponding metrics. 


    ------------------------------
    Tatjana Knezevic
    Star Telecom
    ------------------------------



  • 19.  RE: Short term forecasting -difference in metrics between Automatic best method and weighted index

    GENESYS
    Posted 26 days ago
    I would definietly include all routing paths available into the planning groups.  Excluding them will definitely cause a problem

    ------------------------------
    Daniel Rickwalder
    Genesys
    ------------------------------



  • 20.  RE: Short term forecasting -difference in metrics between Automatic best method and weighted index

    Posted 26 days ago
    @Daniel Rickwalder,

    Even with the volume of 1 call in the last 6 weeks? ​

    Do you agree on the exclusion of all <none> route paths regardless of the volume?

    Thanks,
    Tatjana

    ------------------------------
    Tatjana Knezevic
    Star Telecom
    ------------------------------



  • 21.  RE: Short term forecasting -difference in metrics between Automatic best method and weighted index

    GENESYS
    Posted 26 days ago
    nope you definitely want to build in all valid route paths including the none.  THey can be transfers or some other things.

    ------------------------------
    Daniel Rickwalder
    Genesys
    ------------------------------



  • 22.  RE: Short term forecasting -difference in metrics between Automatic best method and weighted index

    Posted 26 days ago
    Sure, I'll re-configure the planning groups and will re-run the forecast.


    ------------------------------
    Tatjana Knezevic
    Star Telecom
    ------------------------------



  • 23.  RE: Short term forecasting -difference in metrics between Automatic best method and weighted index

    Posted 26 days ago
    Oh, this is not going to work. All the agents from this MU have all the skills, however, they don't have all the Queues assigned. When I added all the possible route paths for the particular Skill and Language, I'm getting 0 agents in the planning group.

    Nothing is easy with WFM as one might think.

    I would suggest putting more details on Genesys' resource pages in regard to the WFM limitations. For example, call metrics (minimum offered calls), minimum number of agents, routing constraints (possible routing paths), and anything else that may help with Genesys WFM implementation.

    In addition to this, the WFM module has missing configurable parameters and UI is not user friendly in all the configuration objects.
    Unfortunately, we need to face the client and explain that WFM configuration won't work for them (even though they are PC3) simply because they don't have the volume and enough agents for ABM to work as expected. Or, just to use WHI and one week of scheduling.
    It seems to us, the only solution for them will be manual schedule creation. Not an option that we thought will be the only option.




    ------------------------------
    Tatjana Knezevic
    Star Telecom
    ------------------------------



  • 24.  RE: Short term forecasting -difference in metrics between Automatic best method and weighted index

    GENESYS
    Posted 26 days ago
    it works all the time. You heist went too far. To properly configure planning groups you need to understand how agents are skilled. Think of it as a skill profile....who can do what.  For example, trainees may have queue/skill/language combinations for all the same. Build up from your largest fully shared route path. Add another one and see if any agents disappear. If so, that's not a valid planning group. It can help to build a matrix with queue/skill combinations down and agent groups across. If agent groups all have the same queue skill combo it can be in one planning group. This exercise also uncovers errors in skill ong so don't be afraid to correct queue and skill assignments if they are wrong.

    ------------------------------
    Daniel Rickwalder
    Genesys
    ------------------------------



  • 25.  RE: Short term forecasting -difference in metrics between Automatic best method and weighted index

    Posted 26 days ago
    @Daniel Rickwalder

    We were making changes to the existing profiles to support route paths in the last couple of days (small tweaks though). I need to continue to work with the client on this, as adding a new queue/skill/language combo must be approved by them.

    Yes, you are correct, and this is exactly what we were doing. Adding a route path by route path then checking if any of the agents will disappear from the list. Very tedious but the only way to do it.

    Thank you,



    ​​​

    ------------------------------
    Tatjana Knezevic
    Star Telecom
    ------------------------------