How to change the default change_stream_churning_interval?

[jwo-highnote]

Hi, is there a way to change the default change stream interval to say 1 sec? I would like to build regression test on a CDC module but the default is 20~40 secs which is too long for CI/CD. I saw there is a change_stream_churning_interval arg but not sure how to pass it to the docker image. Can someone provide an example of the docker run command?

[skuruppu]

You're right that we have a flag for that on the C++ side of the code. But it isn't possible to set it when running the Docker image as that runs a Go binary that calls into C++ and the flag also needs to be available in the Go binary.

For example --overrideMaxDatabasesPerInstance is a flag that is defined is Go and is plumbed through to the C++ layer.

I will reach out and see if it's possible to make it configurable through Docker.

[jwo-highnote]

Thanks for getting back to me! Also, I think if the default is set to a much smaller time (such as 500ms) is likely what most use case for emulator (testing), IMO. Is this something we can change?

[skuruppu]

I will reach out to the team to see if the default can be lowered as you suggested. I'll get back to you as soon as I find out.

[skuruppu]

The team said they chose the 20-40 seconds interval because Emulator is synchronous and will abort and retry any concurrent transactions. A separate thread is used to churn change streams in the background, and this change stream churn transactions will lock the whole database every time they run. So making it too frequent is not recommended (like the 500ms proposed).

Having said that, they said they will look into lowering the default by a few seconds and making it configurable.

We'll keep you posted.

[skuruppu]

Just a quick update that we didn't lower the default but we have a change that will make the value configurable through Docker using an environment variable. The change will be in 1.5.26 release which will go out approximately towards the end of the month. Hope this helps.

[jwo-highnote]

@skuruppu is there any instruction on how to pass in the parameters? Would like to try the new version as it seems out?