Python WS client is too heavy: it utilizes entire CPU core

[vankovp]

Hello.

I continue to explore your new SDK and I find out that Python WS implementation is too slow: I subscribed on 5 random coins, and I got full CPU utilization, and eventually it crashed:

WARNING:root:Read buffer full
ERROR:root:Exception in keep_alive: Not connected

I used yappi profiler to find a root cause, and it seems that I found it: Event.is_set method was called very frequently, please see profiler's report attached. yappi_1734990303.log

The client was initialized using this function:

def subscribe_and_process_orderbook_events(symbols):
    ws_client_option = WebSocketClientOptionBuilder().build()

    # Create a client using the specified options
    client_option = (
        ClientOptionBuilder()
        .set_key(key)
        .set_secret(secret)
        .set_passphrase(passphrase)
        .set_websocket_client_option(ws_client_option)
        .set_spot_endpoint(GLOBAL_API_ENDPOINT)
        .set_futures_endpoint(GLOBAL_FUTURES_API_ENDPOINT)
        .set_broker_endpoint(GLOBAL_BROKER_API_ENDPOINT)
        .build()
    )
    client = DefaultClient(client_option)

    spot_public = client.ws_service().new_spot_public_ws()
    spot_public.start()

    sub_id = spot_public.orderbook_level1(symbols, on_orderbook_data)

yappi was configured as follows:

import yappi

class YappiRecorder:
    def __init__(self):
        self.timout = timedelta(seconds=60)
        self.updated = datetime.now() - self.timout
        self.out_fname = "/var/data/yappi_{}.log".format(int(time.time()))

    def record(self):
        if self.updated + self.timout < datetime.now():
            self.updated = datetime.now()
            with open(self.out_fname, "w") as f:
                yappi.get_func_stats().print_all(
                    out=f,
                    columns={
                        0: ("name", 150),
                        1: ("ncall", 10),
                        2: ("tsub", 10),
                        3: ("ttot", 10),
                        4: ("tavg", 10),
                    },
                )



yappi.set_clock_type("cpu")
yappi.start()

yappi_recorder = YappiRecorder()
yappi_recorder.record()

[ISAAC-XXYYZZ]

Thank you for your feedback! We are actively working on a fix and will update it as soon as possible. Stay tuned!

[ISAAC-XXYYZZ]

Our SDK has been updated. Please try this version, and we look forward to your feedback:

pip install kucoin-universal-sdk==0.1.1a1

[vankovp]

Thank you, it is much better now. Consumption is still a bit higher than other similar SDKs I'am trying to work with, but it's ok. (Each of them subscribed to 100 same coins on corresponding exchange, network traffic is comparable)

[ISAAC-XXYYZZ]

Thank you for your feedback. Feel free to reach out anytime if you have any questions.