Chunk retrieval delay issue in Bee 1.17.3

[IgorShadurin]

Context

Bee version 1.17.3. The problem occurred in our fdp-storage project fairDataSociety/fdp-storage#259, which is built on top of Swarm.

Summary

We encountered an issue with chunk retrieval after uploading data. Occasionally, the uploaded chunk is not available immediately, causing a delay of 1 to 40 seconds before it becomes accessible.

Expected behavior

The expectation was for the chunks to be readily available right after the upload, facilitating a seamless user experience without any delay.

Actual behavior

traced-queries.har.zip
BEE.log.zip

Upon uploading a chunk, we noticed that trying to retrieve it immediately often returns a 500 error with the message "read chunk failed". However, after waiting for a brief period (1-40 seconds), the chunk becomes available. This issue affects the performance and user experience of our fdp-storage project. We need clarification on whether this is a bug or an expected behavior, and how we can ensure immediate chunk availability without overloading the server with GET requests to check the chunk's status.

The error during the following API calls:

POST

https://bee-1.fairdatasociety.org/soc/1fbd64e43e6a18947f372e89cd7f54cb242e1247/cf29db53d5c44182b4c8b222b1e0036eb48bbedd3b6087ae80426e9e275774aa?sig=0cea581cea5e44a022c6f0c26849c46e36dd5abf3b888b8c070530251ea0e015158507dc15f4d6bac6946a5032436f2d04bf3028ab226ed4264a92ec1b1e59e31c

It returns a reference of the chunk {"reference":"d888f33a567a6451edd27d63bec25c6cbdbaa58850fcfc9d430e70c51bfa2cb8"}, but attempting to get the chunk immediately at https://bee-1.fairdatasociety.org/chunks/d888f33a567a6451edd27d63bec25c6cbdbaa58850fcfc9d430e70c51bfa2cb8 returns {"code":500,"message":"read chunk failed"}. After a short wait, the chunk becomes available.

Steps to reproduce

1. Upload a chunk using the provided API endpoint and parameters.
2. Attempt to retrieve the chunk immediately using the returned reference.
3. Observe the 500 error message.
4. Wait for a period ranging from 1 to 40 seconds.
5. Try to retrieve the chunk again and note that it is now available.

Possible solution

To address this issue, we are seeking guidance on the best approach to ensure immediate chunk availability without generating numerous GET requests that could potentially slow down the user experience and overload the server. Furthermore, we are interested in understanding if this is a known bug or expected behavior, and what steps we can take to mitigate this issue in our project.