Re-use http connection?

[ldeffenb]

I've got a node.js script that uses bee-js's bee.downloadData extensively along with axios for /stewardship (until the pinning requirement is removed from bee-js). I noticed when running against a bee node on Windows, that there were 1,000s of sockets in a TIME_WAIT state indicating that multiple connections were being made to the bee node in a short span of time.

I researched how to get axios to reuse a single http connection and found this in their readme.md:

  // `httpAgent` and `httpsAgent` define a custom agent to be used when performing http
  // and https requests, respectively, in node.js. This allows options to be added like
  // `keepAlive` that are not enabled by default.
  httpAgent: new http.Agent({ keepAlive: true }),
  httpsAgent: new https.Agent({ keepAlive: true }),

Upon adding that to my script, I found the number of TIME_WAIT sockets was cut in half. This indicates to me that bee-js must also not be reusing an existing http connection but is establishing a new one for each request.

Is there a way to change this behavior in bee-js in the interest of performance?

[ldeffenb]

I just changed my bee.downloadData invocation with an axios call using the keepAlive httpAgent and the TIME_WAIT sockets disappeared. That seems to prove that bee-js is not reusing connections but is getting a new socket for each request.

[AuHau]

Interesting stuff!

Btw. we have released bee-js's 2.0 version that removed the pinning check for the stewardship endpoint. As part of this release, we have migrated from axios HTTP client that uses XHR in browser to ky that uses fetch instead. This might require a different approach then. The node-fetch allows it using custom HTTP agents, but I am currently not quite sure how to achieve that with the ky library.

[ldeffenb]

Btw. we have released bee-js's 2.0 version that removed the pinning check for the stewardship endpoint.

Once I get my testnet nodes restabilized on the upcoming release, I'll get back to working on the stewardship stuff. I do it all from node.js, which I think also now uses ky/fetch from when I glanced at the source when opening this issue.

[bee-worker]

Release v8.3.0 addresses this.