Add graphql provider #111

zachyking · 2022-10-29T21:39:10Z

draft, I have no idea if it works, needs some debug, but somebody might want to get to it sooner

Allows using Cardano GraphQL instance as data provider in combination with Cardano submit API for submitting txs.
(e.g. dandelion.link for free GraphQL by the community and https://www.freeloaderz.io/ for free load balanced submit api from SPOs)

I have a question though @alessandrokonrad, I am not sure about delegation endpoint, I can get list of records with poolId and reward, so I just return first record rn. Should I count those reward amounts together from all records and return sum with latest poolId?

GGAlanSmithee · 2022-10-29T21:50:24Z

This is really cool. I was thinking about this the other day, and would love to try this PR.

If you don't mind me asking here, I haven't really been able to wrap my head around how actual transaction processing works. Does the wallets handle this, much like Metamask does on ethereum with Infura, or does the providers do that? And if so, is it possible to do that via the dandelion API? Thanks! :)

zachyking · 2022-10-29T22:07:41Z

I guess I have couple more things for now @alessandrokonrad 😃, getDatum is sh*t because I can't add condition to filter records where datum bytes are null and just take one. Some time ago datum hash wasn't available in GraphQL scheme even though it was already on-chain, so I guess it just takes time to catch-up and we might get that at some point. I noted that in comments.
Even then I am not sure if current way is the best way to try getting datum so I do welcome suggestions.

I could write some test functions to call all methods, set results into some object, then I can deep compare between providers. But changes in graphql scheme etc might be breaking builds too frequently. So maybe there could be a limit, like 70-90% similar key and values pairs means pass? Or maybe test only protocol params from each? I can also totally ignore that, but I am fine getting the test together when I'll be debugging.

zachyking · 2022-10-29T22:35:35Z

This is really cool. I was thinking about this the other day, and would love to try this PR.

If you don't mind me asking here, I haven't really been able to wrap my head around how actual transaction processing works. Does the wallets handle this, much like Metamask does on ethereum with Infura, or does the providers do that? And if so, is it possible to do that via the dandelion API? Thanks! :)

Transaction is built off-chain. For that you need live blockchain data, specifically UTXOs: e.g. Blockfrost, cardano-db-sync with postgress, GraphQL, or alternatives. If it's on website client, you can get them also from wallet and deserialize using cardano-multiplatform-lib, but you might miss some information anyway to build the tx, so data from CIP-30 Dapp connector wallet are usually not enough for whole dapp.

When you build the tx, you can submit it using freeloaderz for example, that's free and community run, load balanced. I think dandelion runs submit-api endpoint so that's fine too. If you want higher reliability, paying for blockfrost subscription might be worth it as they will resubmit txs that are dropped because of chain forks (forks are usually like 1-2 blocks, totally normal thing) in higher pay tier. Or again if it's a web client, you can use wallet endpoint to submit.

You can also run all of these pieces of infrastructure, Cardano blockchain is still pretty lightweight (even thought GraphQL/db-sync are beasts, if you think about playing with that, you might wanna look into carp, scrolls, kupo or maybe even ogmios ) @GGAlanSmithee

alessandrokonrad · 2022-10-30T09:27:41Z

src/provider/graphql.ts

+    };
+  }
+
+  async getCurrentSlot(): Promise<Slot> {


This endpoint is not necessary and not a part of the provider interface anymore. The current slot can be calculated off-chain.

alessandrokonrad · 2022-10-30T09:30:48Z

src/provider/graphql.ts

+      txHash: r.txHash,
+      outputIndex: r.index,
+      assets: (() => {
+        const a: Assets = {};
+        r.tokens.forEach((token: {
+          asset: {
+            policyId: string;
+            assetName: string;
+          };
+          quantity: string;
+        }) => {
+          a[token.asset.policyId + token.asset.assetName] = BigInt(
+            token.quantity,
+          );
+        });
+        a["lovelace"] = BigInt(r.value);
+        return a;
+      })(),
+      address,
+      datumHash: r.datum?.hash,
+      datum: r.datum?.bytes,


scriptRef is missing :)

alessandrokonrad · 2022-10-30T09:35:53Z

I have a question though @alessandrokonrad, I am not sure about delegation endpoint, I can get list of records with poolId and reward, so I just return first record rn. Should I count those reward amounts together from all records and return sum with latest poolId?

In general the getDelgation functions returns to you the current delegation state.
Are you delegated to a pool? Yes => Return pool id; No => Return null
And return the current withdrawable amount, otherwise 0.

Does the GraphQL API have insights into the current state of things? Or is it only historic data?

GGAlanSmithee · 2022-10-30T09:41:35Z

@zachyking tyvm for taking your time, very valuable!

alessandrokonrad · 2022-10-30T10:01:20Z

I can't add condition to filter records where datum bytes are null and just take one

What do you mean by that?

Some time ago datum hash wasn't available in GraphQL scheme even though it was already on-chain, so I guess it just takes time to catch-up and we might get that at some point

So the datum is not directly available to query even tho it was just exposed by a tx?

zachyking · 2022-10-30T14:05:36Z

I have a question though @alessandrokonrad, I am not sure about delegation endpoint, I can get list of records with poolId and reward, so I just return first record rn. Should I count those reward amounts together from all records and return sum with latest poolId?

In general the getDelgation functions returns to you the current delegation state. Are you delegated to a pool? Yes => Return pool id; No => Return null And return the current withdrawable amount, otherwise 0.

Does the GraphQL API have insights into the current state of things? Or is it only historic data?

GraphQL API has insights to current state sometimes, querying utxos directly seem to return only unspent ones.

I tried withdrawing my staking rewards, and I see the same records in case of Delegation. I'll try again later, maybe there is just some longer delay, but if not, I can just get both delegation and withdrawals to calculate currently available rewards.

zachyking · 2022-10-30T14:14:35Z

I can't add condition to filter records where datum bytes are null and just take one

What do you mean by that?

Some time ago datum hash wasn't available in GraphQL scheme even though it was already on-chain, so I guess it just takes time to catch-up and we might get that at some point

So the datum is not directly available to query even tho it was just exposed by a tx?

There is a datum object on utxos, which has bytes and hash. I can have where condition using hash, but not bytes. This means instead of getting only one valid record if exists where bytes are not null, I have to get all, and try finding not null bytes in lucid.
Thinking about it, it for sure needs a change to go over query { transactions(where: { outputs: { datum: { hash: { _eq: so it includes spent utxo data, and that way it should work.
But querying for datum hashes that are common will be returning a lot of results that we don't need (until it's possible to add condition to the graphql query bytes !== null, then we can add limit=1)

alessandrokonrad · 2022-10-30T17:34:25Z

There is a datum object on utxos, which has bytes and hash. I can have where condition using hash, but not bytes. This means instead of getting only one valid record if exists where bytes are not null, I have to get all, and try finding not null bytes in lucid.
Thinking about it, it for sure needs a change to go over query { transactions(where: { outputs: { datum: { hash: { _eq: so it includes spent utxo data, and that way it should work.
But querying for datum hashes that are common will be returning a lot of results that we don't need (until it's possible to add condition to the graphql query bytes !== null, then we can add limit=1)

@zachyking so you think it could be a problem because you also query other utxos where datum is not set and you can't filter them out? But the query you suggested there seems sufficient enough I think. Maybe it's not really a problem with all these outputs, but I'm not a graphql user really. What happens if you do {datum: {hash: {_neq: null}}}. Would this not remove outputs with no datum?

Maybe you want to extend the query to also redeemers and collateralOutputs, just to really capture all datums:

query queryDatum {
  transactions(
    limit: 1
    where: {
      outputs: {
        datum: {
          hash: {
            _eq: "923918e403bf43c34b4ef6b48eb2ee04babed17320d8d1b9ff9ad086e86f44ec"
          }
        }
      }
    }
  ) {
    outputs {
      datum {
        bytes
      }
    }
  }
  redeemers(
    limit: 1
    where: {
      datum: {
        hash: {
          _eq: "923918e403bf43c34b4ef6b48eb2ee04babed17320d8d1b9ff9ad086e86f44ec"
        }
      }
    }
  ) {
    datum {
      bytes
    }
  }
  collateralOutputs(
    limit: 1
    where: {
      datum: {
        hash: {
          _eq: "923918e403bf43c34b4ef6b48eb2ee04babed17320d8d1b9ff9ad086e86f44ec"
        }
      }
    }
  ) {
    datum {
      bytes
    }
  }
}

And it returns this:

{
  "data": {
    "transactions": [
      {
        "outputs": [
          {
            "datum": null
          },
          {
            "datum": {
              "bytes": "d87980"
            }
          },
          {
            "datum": {
              "bytes": "d87980"
            }
          }
        ]
      }
    ],
    "redeemers": [
      {
        "datum": {
          "bytes": "d87980"
        }
      }
    ],
    "collateralOutputs": []
  }
}

And then locally in lucid you try to find in any of these a datum where bytes is defined.

EDIT:
Trying to query from hash 0298f8f6c4731e0aad27c90421920148cc16c2e800fdf1cd13e548409b25d548 doesn't even yield a result. GraphQL is weird.

zachyking · 2022-10-31T12:17:40Z

There is a datum object on utxos, which has bytes and hash. I can have where condition using hash, but not bytes. This means instead of getting only one valid record if exists where bytes are not null, I have to get all, and try finding not null bytes in lucid.
Thinking about it, it for sure needs a change to go over query { transactions(where: { outputs: { datum: { hash: { _eq: so it includes spent utxo data, and that way it should work.
But querying for datum hashes that are common will be returning a lot of results that we don't need (until it's possible to add condition to the graphql query bytes !== null, then we can add limit=1)

@zachyking so you think it could be a problem because you also query other utxos where datum is not set and you can't filter them out? But the query you suggested there seems sufficient enough I think. Maybe it's not really a problem with all these outputs, but I'm not a graphql user really. What happens if you do {datum: {hash: {_neq: null}}}. Would this not remove outputs with no datum?

Maybe you want to extend the query to also redeemers and collateralOutputs, just to really capture all datums:
query queryDatum {
  transactions(
    limit: 1
    where: {
      outputs: {
        datum: {
          hash: {
            _eq: "923918e403bf43c34b4ef6b48eb2ee04babed17320d8d1b9ff9ad086e86f44ec"
          }
        }
      }
    }
  ) {
    outputs {
      datum {
        bytes
      }
    }
  }
  redeemers(
    limit: 1
    where: {
      datum: {
        hash: {
          _eq: "923918e403bf43c34b4ef6b48eb2ee04babed17320d8d1b9ff9ad086e86f44ec"
        }
      }
    }
  ) {
    datum {
      bytes
    }
  }
  collateralOutputs(
    limit: 1
    where: {
      datum: {
        hash: {
          _eq: "923918e403bf43c34b4ef6b48eb2ee04babed17320d8d1b9ff9ad086e86f44ec"
        }
      }
    }
  ) {
    datum {
      bytes
    }
  }
}
And it returns this:
{
  "data": {
    "transactions": [
      {
        "outputs": [
          {
            "datum": null
          },
          {
            "datum": {
              "bytes": "d87980"
            }
          },
          {
            "datum": {
              "bytes": "d87980"
            }
          }
        ]
      }
    ],
    "redeemers": [
      {
        "datum": {
          "bytes": "d87980"
        }
      }
    ],
    "collateralOutputs": []
  }
}
And then locally in lucid you try to find in any of these a datum where bytes is defined.

EDIT: Trying to query from hash 0298f8f6c4731e0aad27c90421920148cc16c2e800fdf1cd13e548409b25d548 doesn't even yield a result. GraphQL is weird.

Extending to redeemers and collateral outputs is definitely good idea. I don't think limit=1 should be used in transaction query until condition bytes: { _neq: null is possible.

Hash 0298f8f6c4731e0aad27c90421920148cc16c2e800fdf1cd13e548409b25d548 is definitely interesting, returning result from Blockfrost... I'll check BF implementation when I have a chance, maybe it will help figuring it out.

zachyking added 2 commits October 29, 2022 23:14

add graphql as provider

13b2754

fix datums data for utxos

a8c9d9f

alessandrokonrad reviewed Oct 30, 2022

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add graphql provider #111

Add graphql provider #111

zachyking commented Oct 29, 2022

GGAlanSmithee commented Oct 29, 2022

zachyking commented Oct 29, 2022

zachyking commented Oct 29, 2022 •

edited

Loading

alessandrokonrad Oct 30, 2022 •

edited

Loading

alessandrokonrad Oct 30, 2022

alessandrokonrad commented Oct 30, 2022

GGAlanSmithee commented Oct 30, 2022

alessandrokonrad commented Oct 30, 2022

zachyking commented Oct 30, 2022 •

edited

Loading

zachyking commented Oct 30, 2022 •

edited

Loading

alessandrokonrad commented Oct 30, 2022 •

edited

Loading

zachyking commented Oct 31, 2022

Add graphql provider #111

Are you sure you want to change the base?

Add graphql provider #111

Conversation

zachyking commented Oct 29, 2022

draft, I have no idea if it works, needs some debug, but somebody might want to get to it sooner

GGAlanSmithee commented Oct 29, 2022

zachyking commented Oct 29, 2022

zachyking commented Oct 29, 2022 • edited Loading

alessandrokonrad Oct 30, 2022 • edited Loading

Choose a reason for hiding this comment

alessandrokonrad Oct 30, 2022

Choose a reason for hiding this comment

alessandrokonrad commented Oct 30, 2022

GGAlanSmithee commented Oct 30, 2022

alessandrokonrad commented Oct 30, 2022

zachyking commented Oct 30, 2022 • edited Loading

zachyking commented Oct 30, 2022 • edited Loading

alessandrokonrad commented Oct 30, 2022 • edited Loading

zachyking commented Oct 31, 2022

zachyking commented Oct 29, 2022 •

edited

Loading

alessandrokonrad Oct 30, 2022 •

edited

Loading

zachyking commented Oct 30, 2022 •

edited

Loading

zachyking commented Oct 30, 2022 •

edited

Loading

alessandrokonrad commented Oct 30, 2022 •

edited

Loading