Code

Enable parallel key scraping#1985

Code

Merged

liamaharon merged 21 commits intoparitytech:masterparitytech/polkadot-sdk:masterfrom eagr:para-scrapingeagr/polkadot-sdk:para-scrapingCopy head branch name to clipboardNov 29, 2023

+156-37

Commits

Commits on Oct 23, 2023

• enable parallel scraping

  eagr

  committed
  b757b99

  Copy full SHA for b757b99

• glitch

  eagr

  committed
  d39f72e

  Copy full SHA for d39f72e

Commits on Oct 25, 2023

• Merge branch 'master' into para-scraping

  liamaharon

  authored
  be9a976

  Copy full SHA for be9a976

• split fn

  eagr

  committed
  16e0e0d

  Copy full SHA for 16e0e0d

• get parallel

  eagr

  committed
  48701a3

  Copy full SHA for 48701a3

Commits on Oct 27, 2023

• rewrite rpc_get_keys_parallel()

  eagr

  committed
  ea33098

  Copy full SHA for ea33098

Commits on Nov 7, 2023

• parallel with start key instead of prefix

  eagr

  committed
  4be72d8

  Copy full SHA for 4be72d8

Commits on Nov 8, 2023

• avoid unnecessary clones

  eagr

  committed
  775b037

  Copy full SHA for 775b037

Commits on Nov 9, 2023

• make start key optional

  eagr

  committed
  9881284

  Copy full SHA for 9881284

• thanks clippy

  eagr

  committed
  05ebb51

  Copy full SHA for 05ebb51

Commits on Nov 10, 2023

• divide workload based on suffix size

  eagr

  committed
  8250746

  Copy full SHA for 8250746

Commits on Nov 17, 2023

• Merge branch 'master' into para-scraping

  eagr

  committed
  981a54c

  Copy full SHA for 981a54c

Commits on Nov 18, 2023

• bench different number of chunks

  eagr

  committed
  5043d04

  Copy full SHA for 5043d04

Commits on Nov 20, 2023

• add rpc_get_keys_paged bench

  liamaharon

  committed
  64db2e5

  Copy full SHA for 64db2e5

Commits on Nov 21, 2023

• clean up

  eagr

  committed
  605cd8d

  Copy full SHA for 605cd8d

• as per advices

  eagr

  committed
  5d2392d

  Copy full SHA for 5d2392d

Commits on Nov 22, 2023

• Merge branch 'master' into para-scraping

  liamaharon

  authored
  01f0f77

  Copy full SHA for 01f0f77

Commits on Nov 24, 2023

• Merge branch 'master' into para-scraping

  ggwpez

  authored
  6b1e5f3

  Copy full SHA for 6b1e5f3

Commits on Nov 27, 2023

• link followup

  eagr

  committed
  3530766

  Copy full SHA for 3530766

Commits on Nov 28, 2023

• Merge branch 'master' into para-scraping

  liamaharon

  authored
  45e4dfb

  Copy full SHA for 45e4dfb

• Merge branch 'master' into para-scraping

  liamaharon

  authored
  8ca5469

  Copy full SHA for 8ca5469