Newly created pod can't be fetched

[tomicvladan]

This issue occurres when using Blossom in Fairdrive. After creating a pod, the pod list on the left side is automatically refreshed, but the new pod doesn't appear in the list.

To confirm that the error doesn't come from Fairdrive or Blossom, this line of code is relevant:
https://github.com/fairDataSociety/blossom/blob/a4f3e3087dee02b45916b0b70c48347cd7c3e366/src/services/fdp-storage/fdp-storage-access.ts#L138

Here the service worker script invokes appropriate methods from fdp-storage. If responses are logged out, there will be an entry for creating a pod and after it finishes another entry will appear which returns list of pods. But the list pod won't include the new pod there.

[tomicvladan]

Just to add recent findings. I reproduced this issue in Fairdrive directly, without Blossom.

Steps to reproduce:

1. Compile browser version of fdp-storage from fix/bee-js-new-762 branch.
2. Replace the compiled index.browser.min.js in node_modules of Fairdrive
3. Remove the .next folder to remove cache
4. In src/context/FdpStorageContext.tsx file, disable caching by setting isUseCache: false
5. Run Fairdrive, log in and create a pod

Pod list will refresh but without new pod.

[tomicvladan]

One more note, this issue happens only on bee-1. When using fdp-play it works fine.

[IgorShadurin]

@tomicvladan @darkobas2

The error is also sometimes reproduced in Fairdrive with "@fairdatasociety/fdp-storage": "^0.11.0" and the cache is disabled (isUseCache: false).
After a manual test, it turned out that the problem is due to the fact that the bee-1 server responds with old data for the chunk (404 error instead of current data). After some time (seconds) the data becomes available.

Apparently, the problem may be in the caching of the chunk request OR in the correctness of the responses of the Bee node.

The chunk containing new data is requested before the data is written there, so this may be the reason for caching the Bee node's responses. Although the node can make chunk data available after some time if the Bee node has been updated with some new mechanisms.

It is required to test the server side:

• Generate chunk and get its address
• Request a chunk that has not yet been uploaded to the server (example url https://bee-1.fairdatasociety.org/chunks/daa4d8d97b49d22d694ff6969e3ae02b909bef7a3db76c8feeab14c8e7a079a6)
• Upload chunk to this address
• Try to download chunk
• Analyze server responses to determine what exactly is causing the delay in data availability

If this behavior is on the side of the Bee node, then it is worth planning changes in the fdp protocol that could detect the moment when the data becomes available.

[tfius]

@darkobas2 this is probably connected with serverside caching, chunk is in cache and is not being updated. Can you disable caching on gateway so we can test this and eliminate potential issue ?

[tfius]

We tested this with caching disabled. Still exists.

[IgorShadurin]

I am attaching a har with all the site requests. har can be opened on this site https://toolbox.googleapps.com/apps/har_analyzer/.

Check the POST request to https://bee-1.fairdatasociety.org/soc/c61a782953dec13ec9614469a2801b029378ee48/cf29db53d5c44182b4c8b222b1e0036eb48bbedd3b6087ae80426e9e275774aa?sig=58d35bd3d67619ca18756c3e1487a521c7d7a8275baf5a3becd442b1aaeae8d3524cf784501dc43f27c77391c76e062d3db8f1ca7b3dc93daa2de49098c708c51b


It returns {"reference":"d70c30039aafcc857d18ddaf35c6339ca04a5357b9b7bf992a5be8b5b143a055"}

Then there is a GET to https://bee-1.fairdatasociety.org/chunks/d70c30039aafcc857d18ddaf35c6339ca04a5357b9b7bf992a5be8b5b143a055 which returns {"code":500,"message":"read chunk failed"}

After a few seconds the data becomes available

soc-error.har.zip