Possible Race Condition Getting KV Info After Creation

[brenth-monad]

Observed behavior

While using the nats.go package I noticed that if I have one resource create a KV bucket and another resource races to get the bucket info I do not get back the bucket Info or an error that the bucket does not exist. Instead it times out because the nats.go client never receives a reply.

If I use the nats cli to call

nats kv info non-existing-bucket

I get back the proper error nats: error: nats: bucket not found

The same goes for getting the bucket info using a publish with reply subject

# terminal 1
nats sub "_INBOX.test.*" 
# terminal 2
nats pub --reply _INBOX.test.info '$JS.API.STREAM.INFO.KV_non-existing-bucket' ''

However if I request the bucket info very soon after creating the bucket I never get a reply on the _INBOX.test.info subject. If the bucket does exist then I get back the json info of the bucket in the reply subject as expected.

I am reproducing using a publish with reply to mimic the same I'm seeing in the nats.go client.

This does not happen every time so I have to run the reproduce script a few times or play with the sleep time until it happens.

Expected behavior

I would expect an error to be returned to the reply subject that the bucket or stream does not exist

{"type":"io.nats.jetstream.api.v1.stream_info_response","error":{"code":404,"err_code":10059,"description":"stream not found"},"total":0,"offset":0,"limit":0}

Server and client version

Server: 2.10.25
Nats CLI: 0.1.6
Kubernetes: 1.30.2

Host environment

Nats helm chart https://github.com/nats-io/k8s/tree/main/helm/charts/nats with jetstream enabled

Steps to reproduce

1. Subscribe to the reply subjects nats sub "_INBOX.test.*" -i
2. Run below script and watch for replies on both _INBOX.test.pre and _INBOX.test.post

#!/bin/bash

BUCKET="test-bucket$(date +%s)"
echo "Using bucket: $BUCKET"

echo "Getting bucket before it exists"
nats pub --reply _INBOX.test.pre "\$JS.API.STREAM.INFO.KV_${BUCKET}" ''

echo "Creating bucket"
nats kv add --replicas=3 "${BUCKET}" &> /dev/null &

# May have to play with this time to get it to reproduce and does not happen every time
sleep .00002

echo "Getting bucket after it exists"
nats pub --reply _INBOX.test.post "\$JS.API.STREAM.INFO.KV_${BUCKET}" ''

echo "Cleaning up bucket"
sleep 2
nats kv del -f "${BUCKET}"

Sometimes the post reply will have the bucket info, sometimes it will report that the stream does not exist. This is exactly as expected. However, sometimes I will never get a reply on the post subject at all.