Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(iota-data-ingestion-core): add README.md #4570

Open
wants to merge 2 commits into
base: develop
Choose a base branch
from

Conversation

sergiupopescu199
Copy link
Contributor

Description of change

Investigate the iota-data-ingestion-core crate and provide a README.md which explore how it works and its use cases

Links to any relevant issues

fixes #1450

Type of change

  • Enhancement (a non-breaking change which adds functionality)

Change checklist

  • I have followed the contribution guidelines for this project
  • I have performed a self-review of my own code

@sergiupopescu199 sergiupopescu199 added sc-platform Issues related to the Smart Contract Platform group. infrastructure Issues related to the Infrastructure Team labels Dec 19, 2024
@sergiupopescu199 sergiupopescu199 force-pushed the sc-platform/issue-1450 branch 3 times, most recently from f87059a to 87708df Compare December 19, 2024 16:01
@sergiupopescu199 sergiupopescu199 marked this pull request as ready for review December 20, 2024 10:33
@sergiupopescu199 sergiupopescu199 requested a review from a team as a code owner December 20, 2024 10:33

- **iota-indexer**: The primary function of the indexer is to provide historical data from the ledger, which is crucial for both Explorer and developers who want to utilize ledger data in their applications. By pre-processing and storing this data, the indexer significantly reduces the load on the full node, preventing performance degradation from frequent queries. The data is stored in `PostgreSQL`.

- **iota-analytics-indexer**: This is a specialized indexer designed to analyze and store key metrics from the IOTA blockchain. It tracks network statistics like transaction volume and throughput, user activity such as the number of active addresses, and the execution of Move calls, providing insights into smart contract usage. The collected data is efficiently stored in Object Storage and subsequently transferred to columnar databases like BigQuery or Snowflake, optimized for analytical queries.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually the analytics indexer is not performing any analysis of the data, It's just exporting all the data for analysis in other tools.

It tracks network statistics like transaction volume and throughput, user activity such as the number of active addresses, and the execution of Move calls

Those are metrics currently computed by iota-indexer.

They potentially may be computed from the data exported by the analytical-indexer but it's not done currently.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
infrastructure Issues related to the Infrastructure Team sc-platform Issues related to the Smart Contract Platform group.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Task (Infrastructure)]: Review the data ingestion framework (iota-data-ingestion-core)
2 participants