Dashboard does not use metrics anymore

[bchamagne]

Description

With 21 nodes, we used to transmit more than 100GB per node per day to display the dashboard.

Before: 21 nodes * 300KB of data * 720 requests per hour * 24 hours = 100GB
After: 21 nodes * 4KB of data * 60 requests per hour * 24 hours = 121MB

I removed the fetching of metrics from nodes to nodes and replaced it with 2 new processes:

• DashboardMetrics listens to local events and keep them in buckets
• DashboardMetricsAggregator request regularly other nodes for their DashboardAggregator data

The aggregation is currently set to 1 minute and we keep 60 minutes of history. This way the dashboard can display the last hour of data as soon as we hit the page.

We could do smaller buckets / longer history if the need arises.

Fixes #1182

Type of change

• New feature (non-breaking change which adds functionality)

How Has This Been Tested?

Unit tests + tested locally with 4 nodes

Checklist:

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• Any dependent changes have been merged and published in downstream modules

[Neylix]

For the naming you could rename DashboardAggregator to DashboardMetrics and DashboardAggregatorAggregator to DashboardMetricsAggregator it looks nicer I think

[Neylix]

I also have weird behavior running 2 nodes:

Also there is something wrong in the calculation of number of transaction. Currently for a bucket of a timestamp you create a list of all the durations of all nodes, but some durations of different node refer the same transaction, so if you have only one transaction processed by 3 nodes you'll say there where 3 transactions processed in the graph while in fact there were only 1 transaction.

A solution could be to have a map with transaction address => duration, so you could easily calculate the real number of transaction, and it would also be a more accurate calculation of the average transaction validation time

[bchamagne]

[bchamagne]