Genesys Cloud - Developer Community!

Hello,

With Provider 1.74.0, we are experiencing extreme delays when objects are being pushed into the environment.

Pushing a single object, even during a targeted 'terraform plan', takes over 5 minutes.  Our Teams deal with large number of objects, which is causing significant delays.

Detailed log files have been uploaded to Case #0003992543.

Thank you in advance.

Hello Juraj, 

Thank you for bringing this issue to our attention. We understand this delay may be disruptive to your workflow. 

Our CXasCode engineering team is based in Ireland and India; so we will prioritize this first thing tomorrow morning. To help us investigate, can you please confirm whether you only started seeing the issue after upgrading to v1.74.0? If so, which version were you on before upgrading to v1.74.0.

Thank you, 

Naeem Tai

Hello,

With Provider 1.74.0, we are experiencing extreme delays when objects are being pushed into the environment.

Pushing a single object, even during a targeted 'terraform plan', takes over 5 minutes.  Our Teams deal with large number of objects, which is causing significant delays.

Detailed log files have been uploaded to Case #0003992543.

Thank you in advance.

Hello,

I have only noticed it today, during our attempt to recover some deleted objects.  We upgraded about a month ago if that makes any difference, but didn't have any reports from our Teams until today.  As such, I doubt it would be related to the version, may be due to restoring objects which were previously deleted?

In either case, we would like to rely on Cx-As-Code to be able to recover objects as quickly as possible.  Our environments have a lot of objects, so we would welcome any performance improvements.

Thank you.

Hello Juraj, 

Thank you for bringing this issue to our attention. We understand this delay may be disruptive to your workflow. 

Our CXasCode engineering team is based in Ireland and India; so we will prioritize this first thing tomorrow morning. To help us investigate, can you please confirm whether you only started seeing the issue after upgrading to v1.74.0? If so, which version were you on before upgrading to v1.74.0.

Thank you, 

Naeem Tai

Hello,

With Provider 1.74.0, we are experiencing extreme delays when objects are being pushed into the environment.

Pushing a single object, even during a targeted 'terraform plan', takes over 5 minutes.  Our Teams deal with large number of objects, which is causing significant delays.

Detailed log files have been uploaded to Case #0003992543.

Thank you in advance.

Hello,

I have only noticed it today, during our attempt to recover some deleted objects.  We upgraded about a month ago if that makes any difference, but didn't have any reports from our Teams until today.  As such, I doubt it would be related to the version, may be due to restoring objects which were previously deleted?

In either case, we would like to rely on Cx-As-Code to be able to recover objects as quickly as possible.  Our environments have a lot of objects, so we would welcome any performance improvements.

Thank you.

Hello Juraj, 

Thank you for bringing this issue to our attention. We understand this delay may be disruptive to your workflow. 

Our CXasCode engineering team is based in Ireland and India; so we will prioritize this first thing tomorrow morning. To help us investigate, can you please confirm whether you only started seeing the issue after upgrading to v1.74.0? If so, which version were you on before upgrading to v1.74.0.

Thank you, 

Naeem Tai

Hello,

With Provider 1.74.0, we are experiencing extreme delays when objects are being pushed into the environment.

Pushing a single object, even during a targeted 'terraform plan', takes over 5 minutes.  Our Teams deal with large number of objects, which is causing significant delays.

Detailed log files have been uploaded to Case #0003992543.

Thank you in advance.

Hi @Juraj Makacek

After analyzing your logs from Case #0003992543,  identified the root cause of the delays.

The resource module.routing_queue.genesyscloud_routing_queue.xxxx_test (ID: xxxxx) exists in your Terraform state but has been deleted from Genesys Cloud-the API is returning 404 Not Found. When this occurs, the provider enters a retry loop with exponential backoff (500ms, 1s, 2s, 4s, 8s, then 10s intervals) for up to 5 minutes to handle eventual consistency-a brief period after resource creation where the API may temporarily return 404 before data is fully propagated. The logs confirm this: repeated 404 responses from 15:34:56 until the timeout at 15:39:56-exactly 5 minutes. After the timeout, the provider correctly removes the resource from state and the plan continues.

Since you're recovering deleted objects, the Terraform state still contains references to old resource IDs that no longer exist in Genesys Cloud. Terraform attempts to refresh these resources, receives 404 responses, and waits 5 minutes per deleted resource before proceeding.

To speed up recovery,  you can clean your Terraform state before running the plan:

1. Remove specific resources from state:
    terraform state rm module.routing_queue.genesyscloud_routing_queue.xxxx_test

       2. Or start fresh with an empty state (if recovering the entire environment):
             mv terraform.tfstate terraform.tfstate.backup
             terraform plan


Thanks

Hello,

I have only noticed it today, during our attempt to recover some deleted objects.  We upgraded about a month ago if that makes any difference, but didn't have any reports from our Teams until today.  As such, I doubt it would be related to the version, may be due to restoring objects which were previously deleted?

In either case, we would like to rely on Cx-As-Code to be able to recover objects as quickly as possible.  Our environments have a lot of objects, so we would welcome any performance improvements.

Thank you.

Hello Juraj, 

Thank you for bringing this issue to our attention. We understand this delay may be disruptive to your workflow. 

Our CXasCode engineering team is based in Ireland and India; so we will prioritize this first thing tomorrow morning. To help us investigate, can you please confirm whether you only started seeing the issue after upgrading to v1.74.0? If so, which version were you on before upgrading to v1.74.0.

Thank you, 

Naeem Tai

Hello,

With Provider 1.74.0, we are experiencing extreme delays when objects are being pushed into the environment.

Pushing a single object, even during a targeted 'terraform plan', takes over 5 minutes.  Our Teams deal with large number of objects, which is causing significant delays.

Detailed log files have been uploaded to Case #0003992543.

Thank you in advance.

Hello,

Thank you very much.  The "backoff" explanation makes a lot of sense for other delays we've experienced as well.

First, a compliment, as this part I was not expecting to work: The run eventually does the "right thing", re-creates the object and updates the state file with the new ID.

However, the backoff process is a challenge, as it is blocking everything else from executing.  For example, if it was "retry every minute for 2 retries total, while allowing other objects to be processed in the meantime,  it would be an easier sell.  As-is, everything is blocked, no indication of what's happening, and with having 1000s of objects like in our case, it makes it effectively unusable for the purpose we have it in the first place - being able to recover.

To address your 2 points specifically:

1. Yes, this is an option, we are aware of it and do have a process for it.  My challenge here would be why have automation at all, if we have to do things manually - review all object types, note the ones missing, get the proper TF resources names for each, and run a command for each - even if we assume 0 human error, we are still talking about a lot of manual work with hard-to-organize parallel work.  Especially in our current situation, where we had a "bad run", and we are literally talking about 1000s.
2. In our current situation, "start fresh" is also not an option unfortunately, as "not everything" was deleted, so we would have long wait times and failures the other way: "Object already exists", which combined with the long run times we are discussing, I am not sure would be any easier.

Apologies for the long response, but the short version would be that with the current delays, the resulting timeouts, and the fact backoff is a blocking operation, our partners lost faith in our ability to recover.

Thank you.

Hi @Juraj Makacek

After analyzing your logs from Case #0003992543,  identified the root cause of the delays.

The resource module.routing_queue.genesyscloud_routing_queue.xxxx_test (ID: xxxxx) exists in your Terraform state but has been deleted from Genesys Cloud-the API is returning 404 Not Found. When this occurs, the provider enters a retry loop with exponential backoff (500ms, 1s, 2s, 4s, 8s, then 10s intervals) for up to 5 minutes to handle eventual consistency-a brief period after resource creation where the API may temporarily return 404 before data is fully propagated. The logs confirm this: repeated 404 responses from 15:34:56 until the timeout at 15:39:56-exactly 5 minutes. After the timeout, the provider correctly removes the resource from state and the plan continues.

Since you're recovering deleted objects, the Terraform state still contains references to old resource IDs that no longer exist in Genesys Cloud. Terraform attempts to refresh these resources, receives 404 responses, and waits 5 minutes per deleted resource before proceeding.

To speed up recovery,  you can clean your Terraform state before running the plan:

1. Remove specific resources from state:
    terraform state rm module.routing_queue.genesyscloud_routing_queue.xxxx_test

       2. Or start fresh with an empty state (if recovering the entire environment):
             mv terraform.tfstate terraform.tfstate.backup
             terraform plan


Thanks

Hello,

I have only noticed it today, during our attempt to recover some deleted objects.  We upgraded about a month ago if that makes any difference, but didn't have any reports from our Teams until today.  As such, I doubt it would be related to the version, may be due to restoring objects which were previously deleted?

In either case, we would like to rely on Cx-As-Code to be able to recover objects as quickly as possible.  Our environments have a lot of objects, so we would welcome any performance improvements.

Thank you.

Hello Juraj, 

Thank you for bringing this issue to our attention. We understand this delay may be disruptive to your workflow. 

Our CXasCode engineering team is based in Ireland and India; so we will prioritize this first thing tomorrow morning. To help us investigate, can you please confirm whether you only started seeing the issue after upgrading to v1.74.0? If so, which version were you on before upgrading to v1.74.0.

Thank you, 

Naeem Tai

Hello,

With Provider 1.74.0, we are experiencing extreme delays when objects are being pushed into the environment.

Pushing a single object, even during a targeted 'terraform plan', takes over 5 minutes.  Our Teams deal with large number of objects, which is causing significant delays.

Detailed log files have been uploaded to Case #0003992543.

Thank you in advance.

Hi @Juraj Makacek

Hello,

Thank you very much.  The "backoff" explanation makes a lot of sense for other delays we've experienced as well.

First, a compliment, as this part I was not expecting to work: The run eventually does the "right thing", re-creates the object and updates the state file with the new ID.

However, the backoff process is a challenge, as it is blocking everything else from executing.  For example, if it was "retry every minute for 2 retries total, while allowing other objects to be processed in the meantime,  it would be an easier sell.  As-is, everything is blocked, no indication of what's happening, and with having 1000s of objects like in our case, it makes it effectively unusable for the purpose we have it in the first place - being able to recover.

To address your 2 points specifically:

1. Yes, this is an option, we are aware of it and do have a process for it.  My challenge here would be why have automation at all, if we have to do things manually - review all object types, note the ones missing, get the proper TF resources names for each, and run a command for each - even if we assume 0 human error, we are still talking about a lot of manual work with hard-to-organize parallel work.  Especially in our current situation, where we had a "bad run", and we are literally talking about 1000s.
2. In our current situation, "start fresh" is also not an option unfortunately, as "not everything" was deleted, so we would have long wait times and failures the other way: "Object already exists", which combined with the long run times we are discussing, I am not sure would be any easier.

Apologies for the long response, but the short version would be that with the current delays, the resulting timeouts, and the fact backoff is a blocking operation, our partners lost faith in our ability to recover.

Thank you.

Hi @Juraj Makacek

After analyzing your logs from Case #0003992543,  identified the root cause of the delays.

The resource module.routing_queue.genesyscloud_routing_queue.xxxx_test (ID: xxxxx) exists in your Terraform state but has been deleted from Genesys Cloud-the API is returning 404 Not Found. When this occurs, the provider enters a retry loop with exponential backoff (500ms, 1s, 2s, 4s, 8s, then 10s intervals) for up to 5 minutes to handle eventual consistency-a brief period after resource creation where the API may temporarily return 404 before data is fully propagated. The logs confirm this: repeated 404 responses from 15:34:56 until the timeout at 15:39:56-exactly 5 minutes. After the timeout, the provider correctly removes the resource from state and the plan continues.

Since you're recovering deleted objects, the Terraform state still contains references to old resource IDs that no longer exist in Genesys Cloud. Terraform attempts to refresh these resources, receives 404 responses, and waits 5 minutes per deleted resource before proceeding.

To speed up recovery,  you can clean your Terraform state before running the plan:

1. Remove specific resources from state:
    terraform state rm module.routing_queue.genesyscloud_routing_queue.xxxx_test

       2. Or start fresh with an empty state (if recovering the entire environment):
             mv terraform.tfstate terraform.tfstate.backup
             terraform plan


Thanks

Hello,

I have only noticed it today, during our attempt to recover some deleted objects.  We upgraded about a month ago if that makes any difference, but didn't have any reports from our Teams until today.  As such, I doubt it would be related to the version, may be due to restoring objects which were previously deleted?

In either case, we would like to rely on Cx-As-Code to be able to recover objects as quickly as possible.  Our environments have a lot of objects, so we would welcome any performance improvements.

Thank you.

Hello Juraj, 

Thank you for bringing this issue to our attention. We understand this delay may be disruptive to your workflow. 

Our CXasCode engineering team is based in Ireland and India; so we will prioritize this first thing tomorrow morning. To help us investigate, can you please confirm whether you only started seeing the issue after upgrading to v1.74.0? If so, which version were you on before upgrading to v1.74.0.

Thank you, 

Naeem Tai

Hello,

With Provider 1.74.0, we are experiencing extreme delays when objects are being pushed into the environment.

Pushing a single object, even during a targeted 'terraform plan', takes over 5 minutes.  Our Teams deal with large number of objects, which is causing significant delays.

Detailed log files have been uploaded to Case #0003992543.

Thank you in advance.

Hi @Juraj Makacek

After analyzing your logs from Case #0003992543,  identified the root cause of the delays.

The resource module.routing_queue.genesyscloud_routing_queue.xxxx_test (ID: xxxxx) exists in your Terraform state but has been deleted from Genesys Cloud-the API is returning 404 Not Found. When this occurs, the provider enters a retry loop with exponential backoff (500ms, 1s, 2s, 4s, 8s, then 10s intervals) for up to 5 minutes to handle eventual consistency-a brief period after resource creation where the API may temporarily return 404 before data is fully propagated. The logs confirm this: repeated 404 responses from 15:34:56 until the timeout at 15:39:56-exactly 5 minutes. After the timeout, the provider correctly removes the resource from state and the plan continues.

Since you're recovering deleted objects, the Terraform state still contains references to old resource IDs that no longer exist in Genesys Cloud. Terraform attempts to refresh these resources, receives 404 responses, and waits 5 minutes per deleted resource before proceeding.

To speed up recovery,  you can clean your Terraform state before running the plan:

1. Remove specific resources from state:
    terraform state rm module.routing_queue.genesyscloud_routing_queue.xxxx_test

       2. Or start fresh with an empty state (if recovering the entire environment):
             mv terraform.tfstate terraform.tfstate.backup
             terraform plan


Thanks

------------------------------
Hemanth Dogiparthi
Manager, Software Engineering

Original Message:
Sent: 02-19-2026 13:54
From: Juraj Makacek
Subject: Terraform Extremely Slow

Hello,

I have only noticed it today, during our attempt to recover some deleted objects.  We upgraded about a month ago if that makes any difference, but didn't have any reports from our Teams until today.  As such, I doubt it would be related to the version, may be due to restoring objects which were previously deleted?

In either case, we would like to rely on Cx-As-Code to be able to recover objects as quickly as possible.  Our environments have a lot of objects, so we would welcome any performance improvements.

Thank you.

------------------------------
Juraj Makacek

Original Message:
Sent: 02-19-2026 12:48
From: Naeem Tai
Subject: Terraform Extremely Slow

Hello Juraj, 

Thank you for bringing this issue to our attention. We understand this delay may be disruptive to your workflow. 

Our CXasCode engineering team is based in Ireland and India; so we will prioritize this first thing tomorrow morning. To help us investigate, can you please confirm whether you only started seeing the issue after upgrading to v1.74.0? If so, which version were you on before upgrading to v1.74.0.

Thank you, 

Naeem Tai

------------------------------
Naeem Tai
Team Lead, Lead Software Engineer

Original Message:
Sent: 02-19-2026 10:55
From: Juraj Makacek
Subject: Terraform Extremely Slow

Hello,

With Provider 1.74.0, we are experiencing extreme delays when objects are being pushed into the environment.

Pushing a single object, even during a targeted 'terraform plan', takes over 5 minutes.  Our Teams deal with large number of objects, which is causing significant delays.

Detailed log files have been uploaded to Case #0003992543.

Thank you in advance.

#CXasCode

------------------------------
Juraj Makacek
------------------------------