Josh_Burack | 2021-01-25 20:07:47 UTC | #1
As we continue to use the analyticsApi.PostAnalyticsConversationsDetailsQuery(body), we have the following specific questions regarding the rate limits, thanks in advance for detailed responses:
- Do rate limits for this API get triggered from usage within an org or across the platform? We are curious if our org will get a possible rate limit due to API volume generated outside our org.
- Is there a per org rate threshold you have validated the API micro services against? If so, what is the per org max API usage threshold?
- If we need to retrieve 500K conversations in 60 minutes, will this put us over the rate limit? Is there a best practice rate formula we can apply to our code? (X queries over Y time frame)
Thank you
tim.smith | 2021-01-25 22:33:13 UTC | #2
The responses below reference sections in the Rate Limits documentation. Once our conversation has concluded, we'll review the documentation internally and see what opportunities we can find to improve it for future readers.
- Do rate limits for this API get triggered from usage within an org or across the platform? We are curious if our org will get a possible rate limit due to API volume generated outside our org.
Per the second paragraph in the rate limits docs, limits may be applied per access token, per user, per application, and per organization. Requests for your org will not be impacted by activity from other orgs, except in the rare situation that safeguards kick in for the platform's health/stability. Those situations are not a standard mode of operation and will typically be accompanied by reporting an outage and root cause analysis. General information about these types of events will be communicated via https://status.mypurecloud.com/. Outage and RCA discussions are conducted via Care cases.
- Is there a per org rate threshold you have validated the API micro services against? If so, what is the per org max API usage threshold?
There's no explicit answer to this question. As described in the rate limiting docs, there are numerous limiters that are applied in a variety of ways in different services across the platform. Many of these rates aren't advertised and all are subject to change at any time for any reason (i.e platform stability). The primary reason many of them aren't documented is that there's no way for an external application to monitor its standing with the individual limiters. There are a good number of limits documented in the Limits documentation, but please take note that the content on that page is not an exhaustive list and limits exist that are not documented on that page.
- If we need to retrieve 500K conversations in 60 minutes, will this put us over the rate limit? Is there a best practice rate formula we can apply to our code? (X queries over Y time frame)
The standard platform token rate limit is 300 (documented here in the Limits docs. This is the most common limiter that constrains an app's auth token to 300 requests per minute. Given that limit and the max page size for analytics queries of 100 results per page, the maximum number of conversations that can be retrieved via analytics queries in an hour is 1.8 million (100 conversations per request * 300 requests per minute * 60 minutes = 1,800,000).
Depending on your use case, a better option may be to use the notification service to subscribe to relevant conversation notifications and allow your application to keep track of conversations in real time. This can prevent the need to query repeatedly for conversation data. There is a Data Integration Guide that provides a matrix to help identify appropriate solution options based on use case considerations.
So, with all that in mind, what should a well-behaved application do to operate within the rate limits? The primary behavior to consider for an app is to ensure that it behaves properly when it is rate limited, meaning when it receives a 429 response. The API will respond with 429 responses when any rate limit within the platform has been hit, be it the primary 300 RPM token limiter or any other documented or undocumented rate limit. A well-behaved application will inspect the retry-after header in the response to find the number of seconds it should wait before making additional requests. In the absence of the retry-after header and for other retryable status codes (502, 503, and 504 per the rate limit docs), the application should implement an exponential backoff pattern when retrying the requests. The recommended pattern is documented in the rate limit docs. This approach makes applications resilient and able to handle being rate limited for any reason, expected or not.
To address the consideration of testability for an application, the application should be tested to ensure it complies with the recommended rate limit logic defined above. For example, a class in the application that handles executing the analytics queries could be tested by mocking the API. The mocked API can return a 429 with a retry-after header specifying 30 seconds for the first request and a successful response for the second. The test can verify that the process took more than 30 seconds to complete, which will be true if the code properly makes the first request, respects the retry-after of 30 seconds, then retries the request and gets a successful response. This is more or less what we do for the SDK's tests to verify their logic for automatically handling and retrying when rate limited. You can see this implemented in ApiClientRetryTest.java for the Java SDK; this exists for each of the SDKs if you have another language preference.
Josh_Burack | 2021-01-25 22:50:03 UTC | #3
Thank you Tim, this is exactly the level of detail we have been looking for. Very much appreciate your time and the details.
Regards, Josh
system | 2021-02-25 23:04:27 UTC | #4
This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.
This post was migrated from the old Developer Forum.
ref: 9802