Performance issues when requesting large response data

[ehennestad]

I am testing one of the api clients I have generated for fetching a bucket manifest (i.e listing objects of a bucket for object storage).

The bucket in question has about 120k objects, and the api client is significantly slower (~6x slower) than using matlab.net.http.RequestMessage with send method.

Profiling suggest that the difference in performance is due to handling of json responses.

Api client      : 308 sec
RequestMessage  : 55 sec

profiler_api_client_page1.pdf
profiler_http_request_page1.pdf

[brownemi]

Indeed there is a performance penalty in marshaling the results into classes etc.

v1.1.0 added a doNotDecode method to JSONMapper. With some manual modification that can be used to bypass the decoding to instead use the built in convert response handling or a even a custom JSON decoder if the response is sufficiently simple. For example you could have a postSend function that based on the operationId of the large call uses a custom decoder or just jsondecode that should give the same performance as send with convert response turn on.

This package for Azure Data explorer uses some related approaches where performance of the data plane is critical.
The option was called doNotParse in version of JSONMApper used in that case
See: https://mathworks-ref-arch.github.io/matlab-azure-services/Performance.html

Thanks for the profiles.