Fix deleteMany for bucket_parameters to use an index

[rkistner]

The deleteMany for clearning bucket_parameters data used a filter like this:

     "key": {
        "$gte": {
          "g": 3,
          "t": MinKey(),
          "k": MinKey()
        },
        "$lt": {
          "g": 3,
          "t": MaxKey(),
          "k": MaxKey()
        }
      }

This followed the same pattern as the _id filters on other collections. However, there is no direct index on key, only this index:

await database.bucket_parameters.createIndex(
      {
        'key.g': 1,
        lookup: 1,
        _id: 1
      },
      { name: 'lookup1' }
    );

So in this case we can just use a simpler filter on key.g, to achieve the same thing but using an index.

[changeset-bot]

🦋 Changeset detected

Latest commit: 08e6e92

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 9 packages

Name	Type
@powersync/service-module-mongodb-storage	Patch
@powersync/service-core	Patch
@powersync/service-image	Patch
@powersync/service-module-mongodb	Patch
@powersync/service-module-mysql	Patch
@powersync/service-module-postgres	Patch
@powersync/service-core-tests	Patch
@powersync/service-module-postgres-storage	Patch
test-client	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[Rentacookie]

Would a simpler filter + index like this not perhaps help with the clearing of the bucket_data collection?
I had a look and even though the index there is being used it still looked to be rather slow.

[rkistner]

Adding another index on bucket_data can significantly affect the performance for all writes there, and even make deletes slower because of that. The current filter is slightly complex, but it does efficiently use the _id index. I suspect the performance bottleneck is to a large extend just an overall write throughput bottleneck (although I haven't tested that properly).