N+1 Problem?

[Jonatthu]

https://github.com/dlukez/dataloader-dotnet
Looks like this package is resolving this, but I can not make it work, it fails, do you have another option?

[Bidou44]

I had exactly the same question.... I mean, if the N+1 problem is "by design", then it's just unusable, or am I missing something ?

[Jonatthu]

Batching is the solution but on dotnet is not yet supported it @Bidou44 we depend of open source libraries like that one, or do our own ones for now

[benmccallum]

Any more thoughts on this?

[BenjaBobs]

We're solving this by having a top-level resolver that converts the GraphQL query to a Gremlin query so we can fetch the whole dataset in one query to our CosmosDB backend.

[benmccallum]

@BenjaBobs, nice, you're using CosmosDb? I'm currently trying graphql-net and GraphQL.Net to Cosmos Db, but still seeing lots of limitations on both. Do you have anything open source you can share?

Are you saying you're handling specific cases that involve join-like queries by creating top-level named queries for them? Like "userWithFriends" or something?

[BenjaBobs]

I don't have anything open source, but I can talk a bit about my strategy.

Firstly, all of my entities are stored in Cosmos, I don't know how I will handle other data sources yet.

On the dotnet side
Each entity is a simple datamodel with properties decorated with attributes. I then use these attributes to generate GraphQL Dotnet fields. For each entity I currently generate tree queries, three mutations and one subscription.

As example, imagine an entity called Car.

• Queries
  • car
  • carList
  • carCount
• Mutations
  • createCar
  • updateCar
  • deleteCar
• Subscriptions
  • carUpdated

The EntityList queries are the interesting ones. For each of these I also generate query arguments based on the entity properties and their types.
I implemented a filtering system that then takes the GraphQL query arguments and translate them to partial Gremlin queries such as for integers: $".has(\"{fieldName}\", lt({lessThanValue}))".
All of these merge together to a composite Gremlin query.
Properties that are references to other entities are translated to

g.V('id').union(
  outE('propertyName').inV().hasLabel('propertyType'),
  outE('otherPropertyName').inV().hasLabel('otherPropertyType')
).tree()

The .tree() in the end makes Cosmos return the entire data tree with properties in GraphSON.
I then deserialize this into my entity models and return them via GraphQL Dotnet.

On the Azure side
The CosmosDb is setup to use Graph API. I don't know how I'm going to handle clever partitioning yet.

I've experienced that when doing a query with many nestings performance drops, but if I keep my queries to a max depth of 2, I get really snappy responses (ofc keeping in mind that more data = slower queries).

[benmccallum]

Awesome, thanks @BenjaBobs, I'm not too familiar with Graph API / Gremlin, I'm leaning towards using the SQL API as it's more familiar to me.

I started using GraphQL.Net which is nice because it translates queries into IQueryables for running against the DB, but it doesn't support variables, doesn't seem to work with filtering scalars on properties nested in documents, and isn't .NET Standard supporting. These issues are all starting to add up and make it difficult :P

In a few minds now, try and fix GraphQL.Net and make it more CosmosDB compatible, or make my own kind of translator from graphql queries to the CosmosDB SQL-syntax to use in graphql-dotnet resolvers as it seems more established and active.

[BenjaBobs]

If you can wrap your mind around using CosmosDB SQL api, I think that might give you better performance, and it also seem to be a bit ahead of Gremlin api in terms of implemented functionality.

Good luck. :)