Skip to content

Tutorial: Querying for Tezos data

Jump to bottom
Vishakh edited this page Mar 28, 2020 · 3 revisions

Querying for Tezos data

In this walk-through, we will use the Conseil API to query for some Tezos mainnet blockchain data.

All queries in this query can be done in a user-friendly graphical manner using Arronax.

If you don't have a running Conseil instance, you can either use Nautilus Cloud or run your using the instructions on Running Conseil.

URLs in this tutorial have these placeholders in them:

  • {{protocol}} - HTTP or HTTPS
  • {{hostname}} - Conseil hostname, e.g. localhost
  • {{port}} - Conseil port, e.g. 443 or 1337

All queries require an API key to work. You will either need an API key from either Nautilus Cloud or your own Conseil instance. In any case, an HTTP header called apiKey needs to be set with the value set to a valid API key.

Finally, all POST queries in this tutorial expect the Content-Type header to be set to application/json.

Metadata Queries

Let us first use the metadata queries to ensure we can get the data we want.

Querying for platforms

First, let's confirm Tezos data is indeed available from the current Conseil instance:

Method: GET

URL: {{protocol}}://{{hostname}}:{{port}}/v2/metadata/platforms

Returned data:

[
    {
        "name": "tezos",
        "displayName": "Tezos"
    }
]

Great, Tezos is supported!

Querying for networks

Now let's confirm which Tezos networks this Conseil instance offers data for:

Method: GET

URL: {{protocol}}://{{hostname}}:{{port}}/v2/metadata/tezos/networks

Returned data:

[
    {
        "name": "mainnet",
        "displayName": "Mainnet",
        "platform": "tezos",
        "network": "mainnet"
    }
]

mainnet is also supported by this instance! Some other Conseil instances might support multiple networks.

Querying for available entities

Now we need to understand what kinds of data we can query from our Conseil instance.

Method: GET

URL: {{protocol}}://{{hostname}}:{{port}}/v2/metadata/tezos/mainnet/entities

Returned data:

[
  {
    "name": "accounts",
    "displayName": "Account",
    "count": 461404,
    "displayNamePlural": "Accounts"
  },
  {
    "name": "accounts_history",
    "displayName": "Account History",
    "count": 6339600,
    "displayNamePlural": "Account History"
  },
  {
    "name": "baking_rights",
    "displayName": "Baking rights",
    "count": 29239908
  },
  {
    "name": "balance_updates",
    "displayName": "Balance Update",
    "count": 66904008,
    "displayNamePlural": "Balance Updates"
  },
  {
    "name": "blocks",
    "displayName": "Block",
    "count": 850472,
    "displayNamePlural": "Blocks"
  },
  {
    "name": "delegates",
    "displayName": "Baker",
    "count": 2125,
    "displayNamePlural": "Bakers"
  },
  {
    "name": "endorsing_rights",
    "displayName": "Endorsing rights",
    "count": 27023388
  },
  {
    "name": "fees",
    "displayName": "Fee",
    "count": 324872,
    "displayNamePlural": "Fees"
  },
  {
    "name": "operation_groups",
    "displayName": "Operation Group",
    "count": 19829152,
    "displayNamePlural": "Operation Groups"
  },
  {
    "name": "operations",
    "displayName": "Operation",
    "count": 21185054,
    "displayNamePlural": "Operations"
  },
  {
    "name": "rolls",
    "displayName": "Roll",
    "count": 241625440,
    "displayNamePlural": "Rolls"
  }
]

So this means we can query for information about blocks, accounts, operation groups, operations, fees and much more!

Getting attributes for entities

The entities above effectively correspond to tables in the Conseil database. Let's now investigate what which columns from these tables are available. Here is the query for operations metadata:

Method: GET

URL: {{protocol}}://{{hostname}}:{{port}}/v2/metadata/tezos/mainnet/operations/attributes

Returned data:

[
  {
    "name": "branch",
    "displayName": "Branch",
    "dataType": "Hash",
    "cardinality": 762814,
    "keyType": "NonKey",
    "entity": "operations"
  },
  {
    "name": "number_of_slots",
    "displayName": "Slots",
    "dataType": "Int",
    "keyType": "NonKey",
    "entity": "operations"
  },
  {
    "name": "cycle",
    "displayName": "Cycle",
    "dataType": "Int",
    "keyType": "NonKey",
    "entity": "operations"
  },
  {
    "name": "operation_group_hash",
    "displayName": "Op Group Hash",
    "dataType": "Hash",
    "cardinality": 18065967,
    "keyType": "NonKey",
    "entity": "operations"
  },
  {
    "name": "kind",
    "displayName": "Kind",
    "dataType": "String",
    "cardinality": 11,
    "keyType": "NonKey",
    "entity": "operations"
  },
  {
    "name": "level",
    "displayName": "Level",
    "dataType": "Int",
    "keyType": "NonKey",
    "entity": "operations",
    "reference": {
      "entity": "blocks",
      "key": "level"
    }
  },
  {
    "name": "delegate",
    "displayName": "Delegate",
    "dataType": "AccountAddress",
    "cardinality": 4153,
    "keyType": "UniqueKey",
    "entity": "operations",
    "reference": {
      "entity": "accounts",
      "key": "account_id"
    }
  },
  {
    "name": "slots",
    "displayName": "Slots",
    "dataType": "String",
    "cardinality": 218996,
    "keyType": "NonKey",
    "entity": "operations"
  },
  {
    "name": "nonce",
    "displayName": "Nonce",
    "dataType": "String",
    "cardinality": 24111,
    "keyType": "NonKey",
    "entity": "operations"
  },
  {
    "name": "pkh",
    "displayName": "Pkh",
    "dataType": "AccountAddress",
    "cardinality": 21823,
    "keyType": "NonKey",
    "entity": "operations",
    "reference": {
      "entity": "accounts",
      "key": "account_id"
    }
  },
  {
    "name": "secret",
    "displayName": "Secret",
    "dataType": "String",
    "cardinality": 21823,
    "keyType": "NonKey",
    "entity": "operations"
  },
  {
    "name": "source",
    "displayName": "Source",
    "dataType": "AccountAddress",
    "cardinality": 243648,
    "keyType": "UniqueKey",
    "entity": "operations",
    "reference": {
      "entity": "accounts",
      "key": "account_id"
    }
  },
  {
    "name": "fee",
    "displayName": "Fee",
    "dataType": "Currency",
    "keyType": "NonKey",
    "entity": "operations",
    "scale": 6,
    "currencySymbolCode": 42793
  },
  {
    "name": "counter",
    "displayName": "Counter",
    "dataType": "Decimal",
    "keyType": "NonKey",
    "entity": "operations"
  },
  {
    "name": "gas_limit",
    "displayName": "Gas Limit",
    "dataType": "Decimal",
    "keyType": "NonKey",
    "entity": "operations"
  },
  {
    "name": "storage_limit",
    "displayName": "Storage Limit",
    "dataType": "Decimal",
    "keyType": "NonKey",
    "entity": "operations"
  },
  {
    "name": "public_key",
    "displayName": "Public key",
    "dataType": "Hash",
    "cardinality": 223752,
    "keyType": "NonKey",
    "entity": "operations"
  },
  {
    "name": "amount",
    "displayName": "Amount",
    "dataType": "Currency",
    "keyType": "NonKey",
    "entity": "operations",
    "scale": 6,
    "currencySymbolCode": 42793
  },
  {
    "name": "destination",
    "displayName": "Destination",
    "dataType": "AccountAddress",
    "cardinality": 366470,
    "keyType": "UniqueKey",
    "entity": "operations",
    "reference": {
      "entity": "accounts",
      "key": "account_id"
    }
  },
  {
    "name": "parameters",
    "displayName": "Parameters",
    "dataType": "String",
    "cardinality": 17308,
    "keyType": "NonKey",
    "entity": "operations"
  },
  {
    "name": "manager_pubkey",
    "displayName": "Manager pubkey",
    "dataType": "String",
    "cardinality": 20208,
    "keyType": "NonKey",
    "entity": "operations"
  },
  {
    "name": "balance",
    "displayName": "Balance",
    "dataType": "Currency",
    "keyType": "NonKey",
    "entity": "operations",
    "scale": 6,
    "currencySymbolCode": 42793
  },
  {
    "name": "proposal",
    "displayName": "Proposal",
    "dataType": "Hash",
    "cardinality": 13,
    "keyType": "NonKey",
    "entity": "operations"
  },
  {
    "name": "spendable",
    "displayName": "Spendable",
    "dataType": "Boolean",
    "cardinality": 2,
    "keyType": "NonKey",
    "entity": "operations"
  },
  {
    "name": "delegatable",
    "displayName": "Delegatable",
    "dataType": "Boolean",
    "cardinality": 2,
    "keyType": "NonKey",
    "entity": "operations"
  },
  {
    "name": "script",
    "displayName": "Script",
    "dataType": "String",
    "cardinality": 87,
    "keyType": "NonKey",
    "entity": "operations"
  },
  {
    "name": "storage",
    "displayName": "Storage",
    "dataType": "String",
    "cardinality": 520,
    "keyType": "NonKey",
    "entity": "operations"
  },
  {
    "name": "status",
    "displayName": "Status",
    "dataType": "String",
    "cardinality": 5,
    "keyType": "NonKey",
    "entity": "operations"
  },
  {
    "name": "consumed_gas",
    "displayName": "Consumed Gas",
    "dataType": "Decimal",
    "keyType": "NonKey",
    "entity": "operations"
  },
  {
    "name": "storage_size",
    "displayName": "Storage Size",
    "dataType": "Decimal",
    "keyType": "NonKey",
    "entity": "operations"
  },
  {
    "name": "paid_storage_size_diff",
    "displayName": "Paid storage size diff",
    "dataType": "Decimal",
    "keyType": "NonKey",
    "entity": "operations"
  },
  {
    "name": "originated_contracts",
    "displayName": "Originated Account",
    "dataType": "AccountAddress",
    "cardinality": 27935,
    "keyType": "NonKey",
    "entity": "operations",
    "reference": {
      "entity": "accounts",
      "key": "account_id"
    }
  },
  {
    "name": "block_hash",
    "displayName": "Block Hash",
    "dataType": "Hash",
    "cardinality": 771371,
    "keyType": "UniqueKey",
    "entity": "operations",
    "reference": {
      "key": "hash",
      "entity": "blocks"
    }
  },
  {
    "name": "block_level",
    "displayName": "Block Level",
    "dataType": "Int",
    "keyType": "UniqueKey",
    "entity": "operations",
    "reference": {
      "entity": "blocks",
      "key": "level"
    }
  },
  {
    "name": "ballot",
    "displayName": "Vote",
    "dataType": "String",
    "cardinality": 4,
    "keyType": "NonKey",
    "entity": "operations"
  },
  {
    "name": "internal",
    "displayName": "Internal",
    "dataType": "Boolean",
    "cardinality": 2,
    "keyType": "NonKey",
    "entity": "operations"
  },
  {
    "name": "period",
    "displayName": "Period",
    "dataType": "Int",
    "keyType": "NonKey",
    "entity": "operations"
  },
  {
    "name": "timestamp",
    "displayName": "Timestamp",
    "dataType": "DateTime",
    "keyType": "UniqueKey",
    "entity": "operations",
    "dataFormat": "YYYY MMM DD, HH:mm"
  }
]

Wow, that's a lot of attributes to explore! Observe that the returned data lists for each attribute things like its cardinality, i.e. how many unique values the attribute has, and data type.

Querying for attributes for blocks and accounts is left as an exercise for the reader!

Getting unique operation kinds

Let's now figure out how many different kinds of operations Conseil has in its database.

Method: GET

URL: {{protocol}}://{{hostname}}:{{port}}/v2/metadata/tezos/mainnet/operations/kind

Returned data:

[
  "endorsement",
  "proposals",
  "seed_nonce_revelation",
  "delegation",
  "transaction",
  "activate_account",
  "ballot",
  "origination",
  "reveal",
  "double_endorsement_evidence",
  "double_baking_evidence"
]

Of course, Tezos supports more operation kinds but these are just the ones Conseil came across in mainnet at the point at which it was queried.

Now that we know how to use metadata queries to understand the available data, we can move on to querying for data.

Data Queries

Now that we are armed with all the metadata we need, we can actually start pulling the data we want.

The query bodies for data queries can be pretty complicated so make sure you read the latest spec to ensure you understand the payloads shown here.

Fetching delegations in a block range

Let's try to find all delegations between blocks 10,000 and 12,000 and report both the delegators and the bakers they are delegating to.

Method: POST

URL: {{protocol}}://{{hostname}}:{{port}}/v2/data/tezos/mainnet/operations

Body:

{
"fields": ["operation_group_hash", "block_level", "kind", "source", "delegate"],
"predicates": [
  {
    "field": "kind",
    "operation": "in",
    "set": [
      "delegation"
    ],
    "inverse": false
  },
  {
    "field": "block_level",
    "operation": "between",
    "set": [
      10000, 12000
    ],
    "inverse": false
  }
],
  "orderBy":[
    {
      "field":"balance", 
      "direction": "desc"
    }
  ],
  "limit": 1000
}

Returned data:

[
  {
    "source": "KT1SFkaSj56TocMDMXcZFon9BwNYFHSr7zVt",
    "block_level": 10026,
    "operation_group_hash": "opaEBmfjm2jnu13JwYrfCjE4o7WYqfYhV1gs2cPXpEew9FCS7H2",
    "delegate": "tz1TDSmoZXwVevLTEvKCTHWpomG76oC9S2fJ",
    "kind": "delegation"
  },
  {
    "source": "KT1NekU2Nvfb4BSuMwmvtkyfZaJT3vgDWqCx",
    "block_level": 10029,
    "operation_group_hash": "ooQP2muYQRfrf8ZfSDbPhfWrUbEtFoBrdXBPC3ir1zfRDA7g6ns",
    "delegate": "tz1XQ7SRj4QQWjaeebNd8dFwuTrCot3GGDRF",
    "kind": "delegation"
  },
  {
    "source": "KT1GUUhDSVGLpkepfCwgPRt98TpiYbTDN55L",
    "block_level": 10034,
    "operation_group_hash": "opLvKLpY4yhR11A15M3MyUW1ACFq13PFGhX74mbK27nQCKyBZ7x",
    "delegate": "tz1WCd2jm4uSt4vntk4vSuUWoZQGhLcDuR9q",
    "kind": "delegation"
  },
  {
    "source": "KT1QYDJMoXfjoeBjsASHi9Az87kpWpkrZnui",
    "block_level": 10037,
    "operation_group_hash": "ooHigv4ZdpaJEsPJneppdfhSMdRBNVkARWLKan17KKtas6rniMt",
    "delegate": "tz1TDSmoZXwVevLTEvKCTHWpomG76oC9S2fJ",
    "kind": "delegation"
  },
  {
    "source": "KT1CJVE2WArCTHjGGW8Ht1dbF34xSPY2ier7",
    "block_level": 10038,
    "operation_group_hash": "op8WFvinTrM7NFTAWZaFJYaZ9iznkyHs5vWyhLB8JhPkF6eDShr",
    "delegate": "tz1TDSmoZXwVevLTEvKCTHWpomG76oC9S2fJ",
    "kind": "delegation"
  },
  {
    "source": "KT1So8DdkT1GdH9DGTXd3RrnmUVR7X2Nofjt",
    "block_level": 10042,
    "operation_group_hash": "oobYUiGCHshpUM4dnyHnRpDEk8xbFGLc23Qa5iWdv9ETzync2gZ",
    "delegate": "tz1XQ7SRj4QQWjaeebNd8dFwuTrCot3GGDRF",
    "kind": "delegation"
  },
  {
    "source": "KT1LVWnWEVmGhiJWjLa4PCrN9N4UvcgnxSr9",
    "block_level": 10047,
    "operation_group_hash": "oorn8tZcAURj97RV28apnNtMEBB1eiEK3eUdj9nzh9DscJLyXnu",
    "delegate": "tz1TDSmoZXwVevLTEvKCTHWpomG76oC9S2fJ",
    "kind": "delegation"
  },
  {
    "source": "KT1M4A1fvPKSyjErZXusDbpo1iNsnH3zCHqT",
    "block_level": 10066,
    "operation_group_hash": "ooes19NnB5ktUrSRGca8CGp5vS6DvyHeyCU5n8iU69ZtEDgTn6R",
    "delegate": "tz1XQ7SRj4QQWjaeebNd8dFwuTrCot3GGDRF",
    "kind": "delegation"
  },
  {
    "source": "KT1ApXCC7Egm36Rnutfs4iwLvT8NZViNyumY",
    "block_level": 10093,
    "operation_group_hash": "onmZb6iZFTrLNnxPB97Q6bwq9WPvZMuTZMLYmpQujnZMYKws6BJ",
    "delegate": "tz1TDSmoZXwVevLTEvKCTHWpomG76oC9S2fJ",
    "kind": "delegation"
  },
  {
    "source": "KT1CuPcsMqufbAy2nS2PNi3JjXV6Todqfzxs",
    "block_level": 10001,
    "operation_group_hash": "ooCQyd6QzmFELmzzJWRSDzCpAwuCJAbTsCcu7LbTfSdyEaBbJBY",
    "delegate": "tz1TDSmoZXwVevLTEvKCTHWpomG76oC9S2fJ",
    "kind": "delegation"
  }
]

In the above query we

  • asked for five specific columns
  • applied two conditions
  • set a specific sort order
  • limited the returned results to 1,000 records.

As it turns out, there were ten delegation operations between blocks 10,000 and 12,000.

Fetching the top ten accounts delegating to a specific baker

Assuming, dear reader, you did your homework and had a look at the accounts metadata, we can now fetch some data about accounts. Let's find out who the top ten accounts are by balance delegating to tz1TDSmoZXwVevLTEvKCTHWpomG76oC9S2fJ.

Method: POST

URL: {{protocol}}://{{hostname}}:{{port}}/v2/data/tezos/mainnet/accounts

Body:

{
"fields": ["account_id", "balance"],
"predicates": [
  {
    "field": "delegate_value",
    "operation": "eq",
    "set": [
      "tz1TDSmoZXwVevLTEvKCTHWpomG76oC9S2fJ"
    ],
    "inverse": false
  }
],
  "orderBy":[
    {
      "field":"balance", 
      "direction": "desc"
    }
  ],
  "limit": 10
}

Returned results:

[
  {
    "account_id": "KT1SK9tx8obg57mQT8LEMbP5BX4XqfanhJ4V",
    "balance": 857945998757
  },
  {
    "account_id": "tz1TDSmoZXwVevLTEvKCTHWpomG76oC9S2fJ",
    "balance": 680321792442
  },
  {
    "account_id": "KT1BYagVW447hJp3KKjTMHkRNYwWMvHEoKd6",
    "balance": 475461680356
  },
  {
    "account_id": "KT1RyqeKzn4RBEiVc7YnRnv3Jbjtkz8bnxe4",
    "balance": 343997423408
  },
  {
    "account_id": "KT1DrvzynoDQ3fpQFED5VWvndMb4hwHMwyXn",
    "balance": 252902876994
  },
  {
    "account_id": "tz1Rj5mbpgvuoW7DuiHFvgDnJnoGV5eKMeJ7",
    "balance": 252210858550
  },
  {
    "account_id": "tz1PQ6wH8iPSwesLF2u2QShYc31YuuAcmT1t",
    "balance": 191526844394
  },
  {
    "account_id": "KT1V6AoRaNXpjrQpumRVKEV5UocXKLUirp6z",
    "balance": 161332137735
  },
  {
    "account_id": "KT1NVatrYcqR2cmaYzW8aNddJfraKbaQnDSN",
    "balance": 134432607112
  },
  {
    "account_id": "KT1CMUY3B8fx8wNDmYfuoL2JQQaAgtva6VX9",
    "balance": 116704912990
  }
]

So there we have it..there are some well-endowed accounts delegating to this rather well-endowed baker!

Conclusion

Thanks for reading this tutorial so far. If you encounter any issues, feel free to join our Riot group for developers and get help. As people read this tutorial and provide feedback, we will add more example queries so make sure to come back here to learn more. Happy querying!

Clone this wiki locally