Adrian_Santamaria | 2020-01-13 15:23:58 UTC | #1

Hello

I am investigating about how to make concurrent API calls using the Python SDK, but I am encountering some problems.

Here is my testing python script. Its goal is to print the usernames of a list of users given their ids:

Imports

Authentication stuff

user_ids = # List of ids

usernames = queue.Queue()

Callback function

putusernamein_queue = lambda user : usernames.put(user.username)

usersApi = PureCloudPlatformClientV2.UsersApi() threads = [usersApi.getuser(id, callback=putusernameinqueue) for id in user_ids]

for thread in threads: thread.join()

while not usernames.empty(): print(usernames.get())

However, in the console this message is printed: WARNING Connection pool is full, discarding connection: api.mypurecloud.ie

Also, not all the usernames are printed (e.g. for a list of 400 ids, it prints only 383 usernames)

Could someone explain to me how to do this concurrent calls, please?

Thank you

tim.smith | 2020-01-13 18:23:32 UTC | #2

The connection pool defaults to 4. If you need to change that, you can create your own instance of RESTClientObject and provide the desired pool size in the constructor. You can set your instance of RestClientObject on ApiClient via ApiClient.rest_client. The default instance is set in ApiClient's constructor.

Adrian_Santamaria | 2020-01-14 10:11:57 UTC | #3

Hello

I have changed the RESTClientObject pool size to 50, 100, 500 and 1000, but in all cases it keeps showing that warning messages (and their frequency are not reduced)

tim.smith | 2020-01-14 16:55:59 UTC | #4

The threading is a feature of python's internal libraries, so if you've set the pool size and are using the object correctly, it should work. I've created issue API-5046 to investigate python's thread pool behavior.

Adrian_Santamaria | 2020-01-15 08:38:36 UTC | #5

I have modified my python script to limit the requests simultaneously made to 5 too, using a semaphore (while keeping the pool size at 1000), but it still shows the WARNING message. Here is the code:

Imports

import json import queue import threading

import PureCloudPlatformClientV2 from PureCloudPlatformClientV2.rest import RESTClientObject

Client config

PureCloudPlatformClientV2.configuration.host = "https://api.mypurecloud.ie" apiclient = PureCloudPlatformClientV2.apiclient.ApiClient() apiclient.restclient = RESTClientObject(1000) clientid = "myclientid" clientsecret = "myclientsecret" apiclient = apiclient.getclientcredentialstoken(clientid, clientsecret) PureCloudPlatformClientV2.configuration.apiclient = apiclient usersApi = PureCloudPlatformClientV2.UsersApi()

User ids import

with open('ids.json') as idsfile: userids = json.load(ids_file)

usernames = queue.Queue() semaphore = threading.Semaphore(5) threads = []

Callback function

def putusernamein_queue(user): usernames.put(user.username) semaphore.release()

Requests

while userids: semaphore.acquire() try: userid = userids.pop(0) threads.append(usersApi.getuser(userid, callback=putusernameinqueue)) except: print('An exception has occurred!!') semaphore.release()

Wait until complete

[t.join() for t in threads]

Printing

while not usernames.empty(): print(usernames.get())

Adrian_Santamaria | 2020-02-03 10:22:12 UTC | #6

Hello

Have you discovered something about this behaviour?

Thanks

tim.smith | 2020-02-04 19:50:34 UTC | #7

No, the issue is currently on the backlog.

system | 2020-03-06 19:50:39 UTC | #8

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.

tim.smith | 2020-03-23 17:05:52 UTC | #9

This has been resolved as of 79.2.0. You can now set max_size in the constructor of RESTClientObject, which will control the maximum number of concurrent request threads used.

This post was migrated from the old Developer Forum.

ref: 6858