From 095268d5bb3391cf316513ec345322bddb79969f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Tomasz=20Drwi=C4=99ga?= Date: Thu, 31 Oct 2019 00:19:31 +0100 Subject: [PATCH] Increase priority of ImOnline heartbeats (#3972) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Make sure im-online reports are high priority. * Bump runtime. base-code edit option environment block num - test metrics fn push and finalized listing best_block_num listing Add translate API for storage values (#3947) * Add translate item. * fix * doc * fix doc * A test added. * Apply suggestions from code review Co-Authored-By: Bastian Köcher * address suggestion im-online: account for block authorship (#3973) * im-online: account for block authorship * im-online: add test for block authorship onlineness * im-online: cleanup * im-online: fix test grandpa: Use storage proofs for Grandpa authorities (#3734) * grandpa: Write Grandpa authorities to well known key. Instead of requiring execution proofs for Grandpa authorities, this enables much simpler storage proofs. * grandpa: Introduce named AuthorityList type. * grandpa: Storage migration for srml-grandpa module. * Remove no-longer-used GrandpaApi runtime API. * grandpa: Write AuthorityList to storage with encoding version. We expect the AuthorityList type may change (eg. key changes). To make upgrades smoother, include a version in the stored value. * Bump node runtime spec version. * Update srml/grandpa/src/lib.rs Co-Authored-By: André Silva Add the code for compiling node-cli for WASM-browser (#3974) * Extract CLI to separate module in node/cli * Make node/cli compile for WASM * More work on node/cli browser * More work on browser node * More work * More work * Purge a bit the CI script * More clean up * Remove substrate-finality-grandpa from the CI Its tests use tokio, which fails to compile. * Address review * Add rocksdb feature to the service * Fix substrate-service WASM CI * Apply suggestions from code review Co-Authored-By: Bastian Köcher * Don't WASM-compile substrate-service altogether Remove NetworkSpecialization::on_event (#3976) and backend reference to rpc builder (#3979) Improve doc for storages in srml-support (#3982) * improve doc * Apply suggestions from code review Retire `storage_items!` (#3950) * Retire storage_items * Add storage_items! tests to srml/support/tests * Assimilate genesis config Revert "grandpa: Use storage proofs for Grandpa authorities (#3734)" (#3983) This reverts commit c3b1a9836c3cde2e1028162091a96efefb9a1d0e. Print warning again if polling network is too long (#3984) Add events for im_online (#3991) * Add AllGood event for im_online * Another event just in case. * Bump runtime authority-discovery: futures 03 Future (#3848) * authority-discovery: futures 03 Future * make ci happy * use futures timer instead of tokio timer * Update core/authority-discovery/src/lib.rs Co-Authored-By: Bastian Köcher * remove tokio 01 runtime * trigger build * kill futures01 * rename futures Integrate Wasmtime for runtime execution (#3869) * executor: Use non wasmi-specific execution in tests. * executor: Move all runtime execution tests into tests file. * executor: Use test_case macro to easily execute tests with different Wasm execution methods. * executor: Convert errors to strings with Display, not Debug. * node-executor: Rewrite benchmarks with criterion. They were not passing compilation before and criterion seems to be more widely used in Substrate. * executor: Begin implementation of Wasm runtime. The implementation demonstrates the outline of the execution, but does not link against the external host functions. * executor: Define and implement basic FunctionExecutor. The SandboxCapabilities::invoke is still left unimplemented. * executor: Implement host function trampoline generation. * executor: Instantiate and link runtime module to env module. * executor: Provide input data during wasmtime execution. * executor: Implement SandboxCapabilites::invoke for wasmtime executor. * executor: Integrate and test wasmtime execution method. * executor: Improve FunctionExecution error messages. * Scope the unsafe blocks to be smaller. * Rename TrampolineState to EnvState. * Let EnvState own its own compiler instead of unsafe lifetime cast. * Refactor out some common wasmi/wasmtime logic. * Typos and cosmetic changes. * More trampoline comments. * Cargo.lock update. * cli: CLI option for running Substrate with compiled Wasm execution. * executor: Switch dependency from fork to official wasmtime repo. * Quiet down cranelift logs. * Explicitly catch panics during host calls. We do this to ensure that panics do not cross language boundaries. * Additional checks and clarifications in make_trampoline. * Fixes after merge from master and panic safety for wasmtime instantiation. Implement Debug for some structures (#3941) * Implement Debug for some structures `NetworkConfiguration`, `TransportConfig`, `NodeKeyConfig` and `Secret`. Needs a new release of the `rust-libp2p` crate. This PR is just a reminder. * Explicitly separate `std` and `core` * Add manual implementation for Secret Possible fix to storage cache (#3989) * Comment local_cache propagation * Add test * Deny cache when modifications are unknown * Fix indentation Don't panic in Offchain test context, when we are already panicking (#3996) executor: Move runtime version caching out of WasmRuntime interface. (#3993) * executor: Move runtime caching out of WasmRuntime interface. The runtime version is now fetched and cached at a higher level, not within the WasmRuntime trait implementations. * executor: Require successful querying of runtime version. Move config path generation into the service config for reusability (#3978) * Move config path generation into the service config for reusability * Make NoCostum Default and fix tests * Apply suggestions from code review Co-Authored-By: Bastian Köcher * remove function not used anymore * Make path into an option * remove database_path function and call it directly * remove helper functions, use consts expose offchain worker storage prefix (#3977) * expose offchain worker storage prefix * add docs * move STORAGE_PREFIX to primitives Optional serde for phragmen support (#3994) * Add serde to phragmen * Update lock file * and bump a version * and bump a version again * Apply suggestions from code review Co-Authored-By: Bastian Köcher * revert impl-tarit for tuple update * revert session. * Revert "revert session." This reverts commit 98086c9db56677068db85f74320868b2c10d1c00. * Revert "revert impl-tarit for tuple update" This reverts commit 28a7fddee2e09c5785b19883f743065e0be8f331. doc (#3995) Implement color output for wasm builder (#4004) * Implement color output for wasm builder * Fix `Cargo.lock` CI test executor with wasmtime (#4005) Avoid sending heartbeat if we are already considered online. (#3981) * Don't send a heartbeat if already online. * Remove env_logger. * Update lock. * Bump runtime. * Merge master Deprecate ValidateUnsigned and prevent duplicate heartbeats (#3975) * Add pre-dispatch checks for ValidateUnsigned * Deprecate ValidateUnsigned. * Bump specversion. * Fix test. fix formula in comment (#4006) Substrate EVM (#3927) * srml-evm: init the basic structures * srml-evm: finish executor implementation * srml-evm: implement balance deposit and withdraw * srml-evm: implement the actuall call/create * srml-evm: use crates.io version of evm * srml-evm: fix no-std compile * Remove dependency patch * Update to evm 0.14 * Use double map for account storage * Add precompiles support * Add some basic docs * Use runtime_io::chain_id() * Update srml/evm/src/lib.rs Co-Authored-By: Xiliang Chen * Update srml/evm/src/lib.rs Co-Authored-By: Xiliang Chen * Fix WithdrawReason * Unique saturate balance to u128 * Unique saturate withdraw to u128 * Remove extern crate alloc * Move account code to a separate storage and use ref for convert_account_id * More match cause for error message * Fix potential interger overflow * Use decode_len for fetching code length support crypto primitives for no_std introducing `full_crypto` feature (#3778) * introduced "with_crypto" feature and applied switches like in substrate-api-client fork * introduced "with_crypto" feature and applied switches like in substraTEE-worker fork * distinguishing core::hash vs std::hash * @bkchr's review requests fulfilled * fixes * revert dependency upgrade ed25519-dalek * added full_crypto features to all crates using app_crypto! macro * fixing CI complaints. * fix again * adding CI test for with_crypto feature * added full_crypto for ecdsa. now builds wit h--no-deafault-features --features with_crypto * remove --release from CI test * @bkchr requested changes. moved full_crypto CI test to build stage * fixing no_std issue * CI fresh copy from srml-staking * gitlab CI with +nightly * solved no-feature-in-macro dilemma * cosmetics * Update core/application-crypto/src/sr25519.rs Co-Authored-By: Bastian Köcher * Update core/application-crypto/src/ed25519.rs Co-Authored-By: Bastian Köcher * even more simple * undo line delete * refactoring app_crypto macro. splitting functionalities based on full_crypto feature * whitespace cosmetics refactor code: remove unused import (#4010) * remove unused import * fix test error Update RPC deps. (#4012) * Bump rpc deps * Update core/test-runtime/src/genesismap.rs Co-Authored-By: Bastian Köcher Enable parallel block download (#4014) Traffic statistics (#4017) * Network stats * Fixed tests Limit longevity of im-online heartbeats. (#4011) * Limit longevity of im-online heartbeats. * Unused import. * Use parameter for session duration. Header-only sync for old forks (#3942) * Header-only sync for old forks * Simplified blocks-count * Update core/consensus/common/src/block_import.rs Co-Authored-By: Marcio Diaz Revert "Header-only sync for old forks (#3942)" (#4022) This reverts commit ac78c9082206d558e8b4958e6730bc3d38b4f365. Don't re-compile on every `cargo run` (#4019) - Add new crate `substrate-build-script-utils` to unify the code of `node`, `node-template` and `polkadot-node`. - The `node-cli` build script needs to search upwards for the `.git/HEAD` file to find it. fix warnings (#4024) Less verbose console output (#4029) Apply breaking changes of new libp2p versions (#3877) * Apply breaking changes of new libp2p versions * Oops, forgot to update version * Fix tests * Fix imports with WASM * Fix WASM for real * Update core/network/src/debug_info.rs Co-Authored-By: Roman Borschel * Fix compilation Update `syn`, `proc-macro2` and `quote` to `1.x` (#4028) Make `TestExternalities` implement `Send` (#4030) * Make `TestExternalities` implement `Send` + `Sync` * Fixes offchain * Make it just `Send` Treasury rewards should pay the remainder of the 10% (#4026) * Treasury rewards should pay the remainder of the 10%.. * Event for deposits coming into treasury * Fix tests * Remove OnDilution Also replace paths in [build-dependencies] (#4039) * Also replace paths in [build-dependencies] * Update scripts/node-template-release/src/main.rs Co-Authored-By: Bastian Köcher * Slightly reduce memory usage * Update scripts/node-template-release/src/main.rs Co-Authored-By: Bastian Köcher Decrease peer reputation on bad transactions (#4035) * Decrease reputation on bad transactions * Don't punish on duplicate transactions Add `ExistenceRequirement` to `Currency` trait (#4000) * Added a public transfer_some function and a private transfer_inner fn * Move transfer_some to the end of the module impl to fix failing contracts test * Change whitespace * Remove needless change to transfer logic * Fix error * Update srml/balances/src/lib.rs Co-Authored-By: joe petrowski <25483142+joepetrowski@users.noreply.github.com> * Improve documentation and add test * Update srml/balances/src/lib.rs Co-Authored-By: Kian Paimani <5588131+kianenigma@users.noreply.github.com> * Switch to changing Currency trait instead Ban incoming invalid transactions (#4037) Improve auto-docs a little. (#4032) Check polkadot compilation. (#3980) * Check polkadot compilation. * Fix git grep. * Make sure to update properly. Use commit hash instead. Refactor out MaxPossibleReward, fix staking arithmetic (#4041) * Refactor out MaxPossibleReward, fix staking arithmetic * Fix rounding error in test grandpa: Use storage proofs for Grandpa authorities (#3985) Allow import withouth state verification (#4031) * Allow import without state verification * Explicit None Co-Authored-By: Robert Habermeier Correctly serialize code in chain spec as hex (#4025) * Correctly serialize code in chain spec as hex. Due to a bug, the runtime code was previously serialized as a JSON array of numbers, pretty printed one byte per line. * Remove panic in macro and whitelist attribute types for storage genesis config lines. * Use syn::Error to enforce whitelisted attributes on genesis config. * Blacklist genesis extra config line attributes instead of whitelist. clean node/cli/Cargo.toml (#4046) * clean node/cli/Cargo.toml * minor fix * clean node/runtime/Cargo.toml fix two typos (#4048) Increase parallel downloads to 5 (#4045) * Increase parallel downloads to 5 * CLI param Remove dependencies on OpenSSL (#4036) * Remove dependency on hyper-tls This removes one of the dependencies on native-tls, and thus on OpenSSL. I will remove the other in a separate commit. * Remove the `HttpClient` enum It only had one variant. * Apply suggestions from code review Co-Authored-By: Pierre Krieger fix inmemory (#4049) grandpa: progressively increase target gossip peers (#4050) * grandpa: stricter gossip message filtering * gossip: remove filtered message on send_message * gossip: add test for tracking of broadcast attempts * grandpa: only restrict gossip if we're connected to more than 5 authorities * grandpa: add test for progressive gossip * grandpa: add test for gossip filtering on local non-authority node * grandpa: fix doc * gossip, grandpa: tabify * grandpa: relax filtering logic for global messages Introduce thread pool for transaction validation. (#4051) Batch gossip messages (#4055) Fix minor comment typo "do" -> "do not" (#4054) Fix sync downloading ancient chains (#4060) * Update best block on announcement * Added a test grandpa: remove the periodic block announcer (#4062) * grandpa: remove the periodic block announcer * grandpa: remove periodic block announcer test Fix the ui tests (#4066) Improved cache documentation (#4067) Additional logging for the transaction pool. (#4068) * Additional logging for the pool. * Long line. Pass startup_time to telemetry (#4069) Further fix for common block update (#4071) Add a --no-private-ipv4 CLI option (#4042) * Add a --no-private-ipv4 CLI option * Fix tests * Fix tests Change max width to 100. (#4072) Don't update common block on ancient block import (#4073) Don't search for authority set change block if delay is zero (#4076) srml-timestamp: define max timestamp drift in millis (#4077) * srml-timestamp: define max timestamp drift in millis * srml-timestamp: suffix MAX_TIMESTAMP_DRIFT with unit Substrate runtime interface 2.0 (#4057) * Adds first version of traits for generating the host functions * First steps of the procedural macro * Implements generation of the host extern functions * Prefix ext host function with snake case trait name * Implement host functions implementation on the host * Change `HostFunctions` interface * Implement `HostFunctions` for tuples * Make `WasmExecutor` generic over the host functions * Begin to add a test and make it compile * Make the test succeed * Add test to ensure that host functions are not found * It's alive! Make the `set_storage` test work * Add test for mutable references * Code cleanup and documentation etc * Add marker trait for types that should be passed as SCALE encoded * Inherit the visibility from the trait and more improvements * More impls and move them into their own file * Code simplification by dropping one trait * Give it a better name * Implement traits for arrays * Refactor code to support pass by codec/inner * Docs * Implement pass by inner for some crypto types and add a test * Implement exchangeable function support * Rewrite sr-io with as runtime interface * Start reworking after master merge * Adds `PassByCodec` derive * Adds `PassByInner` derive * Fix compilation errors * More implementations * Implement runtime interface traits for `str` * Make `sr-io` compile again * Fix more compilation errors * More progress on getting stuff back to compile * More compilation fixes * Fix warnings * Remove le conversions * Add support for `wasm_only` interfaces * Implement `Allocator` interface * Improve error message * Move `WasmAllocator` to `sr-io` and more clean ups * Use correct function signature for wasm functions * Store the host functions with the Wasm runtime * Docs update * Fix compilation after master merge * Remove `sr-io/without_std` * Make `srml-support` tests run again * More compilation error fixes * Use correct doc syntax * Fix test-runtime * Fix compilation * Catch native panics when executing the wasm runtime As with the native runtime, we now catch all native panics when we execute the wasm runtime. The panics inside the wasm runtime were already catched before by the wasm executor automatically, but any panic in the host functions could bring down the node. The recent switch to execute the native counterpart of the host function in `sr-io`, makes this change required. The native `sr-io` functions just `panic` when something is not provided or any other error occured. * Fix compilation * Don't panic in a panic * Move `sr-sandbox` to new runtime interface * Fixes tests after sandbox changes * Make sure we detect invalid utf8 * Fixes after master merge * Adds pass by enum strategy * Fix wasmtime integration * Some macro structure clean up * Rework and test exchangebale host functions * PassBy derive macros documentation * Docs for `runtime_interface` macro * Support wild card argument names * Adds ui tests * Make sure that we are backwards compatible to the old runtime interfaces * Documentation * Fixes after latest master merge * Make `wasmtime` happy * Make `full_crypto` work * Make the new interface versionable * Rename `Sanboxing` to `Sandbox` * Don't finalize in test while importing * Fix Performance regression * Fix test Remove Self vote from phragmen (#4081) * first draft of everything that works * Some test fixes Allow ancient fork download after ancestry search (#4080) Typed chain state queries over rpc. (#4079) * Create typed client helpers for querying chain state storage items declared by 'decl_storage!'. * Update substrate-rpc-custom functions to use async await syntax. * The implementation of substrate-rpc-custom was a bit verbose and repetitive. This commit makes the implementation simpler by intruducing a struct which represents query for a typed value in storage. The new struct is called StorageQuery. A StorageQuery wraps a raw StorageKey but is not directy constructable. To construct a StorageQuery, the user must supply an implementation of a srml_support::storage::generator trait such as StorageValue or StorageMap. A type implementing one of the generator traits can be aquired by: A) marking a storage item as pub within a call to decl_storage (recommended) or B) implementing one of the generator types manually. While option B may sometimes me necessary, it's not recommended because separate manual implementaions may lose sync with the original definition. * drop unused dependency * fmt * Remove unnecessary pub from Authorities field in test-runtime storage declaration. This field was added to support a test in an earlier commit. The test no longer relies on test-runtime so the change can be reverted. * Move it to srml as support extension. * Fix long lines. * Fix. Cleanup filtered messages (#4082) Don't request old obsolete blocks (#4084) core/consensus: fix console output for slot duration (#4085) [WIP] .gitlab-ci.yml: Fix a check for polkadot to work on forked repos v2 (#4078) * Fix a check for polkadot to work on forked repos. * .gitlab-ci.yml: Add `git diff` for CI debugging * More future proof and less redundant. * .gitlab-ci.yml: Improve comments and refactor sed command * .gitlab-ci.yml: Remove two commas in a row * .gitlab-ci.yml: Ensure to match branch statements at the end * .gitlab-ci.yml: Reference concrete commit When one does not specify the concrete commit, cargo-update tries to checkout 'master' in the Substrate repository. * .gitlab-ci.yml: Remove 'git diff' debug line Fix runtime interface docs and remove accidentally added file (#4092) EVM should store contracts' balances in an account. (#4090) update tiny-keccak (#4093) * update tiny-keccak * fix * Update Cargo.toml * update Cargo.lock * remove keccak-hasher Do nothing for zero imbalances (#4089) Fix sync stalling on moving head (best block) prior to the tip of the chain sometimes (#4091) * Work around finalization woes * Fixed check_block * Added a test Weight annotation for block hooks. (#4058) * Initial version that works with proper tests. * get rid of todos and grumbles and such. * Cleanup and fix line-width * fix test runtime test Sensible limits for known blocks and extrinsics (#4094) Randomized tests on storage cache (#4070) * randomized tests * destroy clones * remove commented * fixes * add finalization fork State machine call proof backend (#3945) * drafting a proof extraction at the hashdb level (to include everything for the 'call' proof case). * use full proof by default (replace previous proof recorder). * fix warnings. * Cache value not found in proof recorder. * Remove need or internal backend struct. * fix type. * doc Translation for linked map (#4052) * make linked_map generic functions better * implement translation of linked map * proc-macro for linked map updated * test linked map migration * account for instances * address grumbles * cut map short if migration fails Do not download blocks too far ahead (#4095) Keep tack of best imported block precisely (#4100) Additional wasm diagnostics (#4097) * Wasm diagnostics * Pass the error * Make errno optional * Cargo.lock * Log the error Implement Copy, Deref and FromStr to (ed25519|sr25519)::Public (#3998) *: Refactor authority discovery (key mngmt, runtime API) (#3955) * {core,srml}/authority-discovery: Move generic to specific session keys * {srml,core}/authority-discovery: Verify signature outside of runtime Given that the `core/authority-discovery` uses concrete authority identifiers and signatures, one can verify a signature with the authority discovery within `core`. Given the above, the `verify` runtime api is obsolete and thus removed. * *: Add authority discovery to the set of session keys * *: Sign authority discovery DHT payload with keystore instead of runtime Instead of calling a runtime function to sign a dht payload, which then invokes the keystore, pass the keystore to the authority discovery module and use it directly. * core/authority-discovery: Give libp2p Kademlia time to start up * core/authority-discovery: Move authorities priority group name to const * node/runtime/src/lib.rs: Bump runtime spec version * *: Fix lints and node/testing test failures * *: Fix formatting * core/authority-discovery: Box dht event channel in unit tests * node/cli/src/service.rs: Fix future import * node/cli/src/service.rs: Replace unwrap by expect with proof * node/cli/src/chain_spec: Remove TODO for testnet key generation * core/authority-discovery/src/lib: Remove scale encoding TODOs * srml/authority-discovery: Make comment a doc comment * core/authority-discovery: Remove unused StreamExt import * node/runtime: Bump impl version to debug CI * Test ci. * Change the line width to 100. * Revert "Change the line width to 100." This reverts commit edff1f855bc71e0418bf3a967f81a35591d882e3. * Fix a check for polkadot to work on forked repos. * Revert "node/runtime: Bump impl version to debug CI" This reverts commit 1a90903b4c929bc55a9e0a538af34b50b7f65139. * Revert "Test ci." This reverts commit a2c9df574e645158f77cd2b3d4d9355bcae33aab. * Cargo.lock: Fix wrong lock file merge * srml/authority-discovery: Keep track of new validator set not upcoming * core/authority-discovery: Document key retrieval functions Reorganising the repository - external renames and moves (#4074) * Adding first rough ouline of the repository structure * Remove old CI stuff * add title * formatting fixes * move node-exits job's script to scripts dir * Move docs into subdir * move to bin * move maintainence scripts, configs and helpers into its own dir * add .local to ignore * move core->client * start up 'test' area * move test client * move test runtime * make test move compile * Add dependencies rule enforcement. * Fix indexing. * Update docs to reflect latest changes * Moving /srml->/paint * update docs * move client/sr-* -> primitives/ * clean old readme * remove old broken code in rhd * update lock * Step 1. * starting to untangle client * Fix after merge. * start splitting out client interfaces * move children and blockchain interfaces * Move trie and state-machine to primitives. * Fix WASM builds. * fixing broken imports * more interface moves * move backend and light to interfaces * move CallExecutor * move cli off client * moving around more interfaces * re-add consensus crates into the mix * fix subkey path * relieve client from executor * starting to pull out client from grandpa * move is_decendent_of out of client * grandpa still depends on client directly * lemme tests pass * rename srml->paint * Make it compile. * rename interfaces->client-api * Move keyring to primitives. * fixup libp2p dep * fix broken use * allow dependency enforcement to fail * move fork-tree * Moving wasm-builder * make env * move build-script-utils * fixup broken crate depdencies and names * fix imports for authority discovery * fix typo * update cargo.lock * fixing imports * Fix paths and add missing crates * re-add missing crates base-code(rebased) set global/local metric + peers_num add prres_num,best_heght,finallity_heght add TARGET_SYN_NUMBER add README.md add README.md add README.md TARGET_SYN_NUMBER add README.md add README.md fix: justification and newline at EOF PEERS_NUM README.md update README.md update README.md update README.md update README.md README.md README.md translation of introduction feat: add prometheus! in substrate feat: add match for opts options SaturatedConversion remove refactor prometheus/lib.rs chore: make macros for widgets TODO : counter metrics type add->remove chore: add prometheus! to consensus chore: update README chore: refactor README chore: Refactor README chore: Refactor README chore: Refactor README --- .dockerignore | 1 + .editorconfig | 2 +- .gitignore | 1 + .gitlab-ci.yml | 137 +- Dockerfile => .maintain/Dockerfile | 0 {scripts => .maintain}/build-only-wasm.sh | 0 {ci => .maintain}/check_for_exit.sh | 0 {scripts => .maintain}/common.sh | 0 .../docker/subkey.Dockerfile | 2 +- .../docker/substrate.Dockerfile | 2 +- .maintain/ensure-deps.sh | 61 + {scripts => .maintain}/flamingfir-deploy.sh | 0 {scripts => .maintain}/getgoing.sh | 0 .../gitlab/check_line_width.sh | 0 .../gitlab/check_runtime.sh | 6 +- {node-template/scripts => .maintain}/init.sh | 0 {scripts => .maintain}/kubernetes/Chart.yaml | 0 {scripts => .maintain}/kubernetes/README.md | 0 .../templates/poddisruptionbudget.yaml | 0 .../kubernetes/templates/secrets.yaml | 0 .../kubernetes/templates/service.yaml | 0 .../kubernetes/templates/serviceaccount.yaml | 0 .../kubernetes/templates/statefulset.yaml | 0 {scripts => .maintain}/kubernetes/values.yaml | 0 .../node-template-release.sh | 2 +- .maintain/node-template-release/Cargo.lock | 581 ++ .../node-template-release/Cargo.toml | 0 .../node-template-release/src/main.rs | 56 +- {scripts => .maintain}/runtime-dep.py | 2 +- .../rustdoc-header.html | 0 .../sentry-node/docker-compose.yml | 2 +- {scripts => .maintain}/update-copyright.sh | 0 {scripts => .maintain}/update-deps.sh | 0 {scripts => .maintain}/update.sh | 0 .travis.yml | 28 - Cargo.lock | 5338 ++++++++++------- Cargo.toml | 151 +- README.adoc | 517 +- bin/node-template/Cargo.toml | 41 + {node-template => bin/node-template}/LICENSE | 0 .../node-template}/README.md | 0 bin/node-template/build.rs | 9 + bin/node-template/runtime/Cargo.toml | 67 + .../node-template}/runtime/build.rs | 0 .../node-template}/runtime/src/lib.rs | 23 +- .../node-template}/runtime/src/template.rs | 2 +- .../node-template/scripts}/init.sh | 0 .../node-template}/src/chain_spec.rs | 0 .../node-template}/src/cli.rs | 2 +- .../node-template}/src/main.rs | 0 .../node-template}/src/service.rs | 7 +- bin/node/cli/Cargo.toml | 141 + {node => bin/node}/cli/bin/main.rs | 6 +- bin/node/cli/browser-demo/.gitignore | 1 + bin/node/cli/browser-demo/README.md | 10 + bin/node/cli/browser-demo/build.sh | 3 + bin/node/cli/browser-demo/favicon.png | Bin 0 -> 10338 bytes bin/node/cli/browser-demo/index.html | 39 + bin/node/cli/browser-demo/ws.js | 148 + {node => bin/node}/cli/build.rs | 8 +- .../node}/cli/doc/shell-completion.adoc | 0 {node => bin/node}/cli/res/flaming-fir.json | 0 bin/node/cli/src/browser.rs | 160 + {node => bin/node}/cli/src/chain_spec.rs | 43 +- .../cli/src/lib.rs => bin/node/cli/src/cli.rs | 67 +- {node => bin/node}/cli/src/factory_impl.rs | 2 +- bin/node/cli/src/lib.rs | 88 + {node => bin/node}/cli/src/service.rs | 47 +- bin/node/executor/Cargo.toml | 48 + bin/node/executor/benches/bench.rs | 196 + {node => bin/node}/executor/src/lib.rs | 38 +- {node => bin/node}/primitives/Cargo.toml | 6 +- {node => bin/node}/primitives/src/lib.rs | 0 {node => bin/node}/rpc-client/Cargo.toml | 4 +- {node => bin/node}/rpc-client/src/main.rs | 0 bin/node/rpc/Cargo.toml | 16 + {node => bin/node}/rpc/src/lib.rs | 14 +- bin/node/runtime/Cargo.toml | 122 + .../node/runtime}/build.rs | 2 +- {node => bin/node}/runtime/src/constants.rs | 0 {node => bin/node}/runtime/src/impls.rs | 4 +- {node => bin/node}/runtime/src/lib.rs | 84 +- bin/node/testing/Cargo.toml | 31 + {node => bin/node}/testing/src/client.rs | 0 {node => bin/node}/testing/src/genesis.rs | 1 + {node => bin/node}/testing/src/keyring.rs | 4 +- {node => bin/node}/testing/src/lib.rs | 0 {subkey => bin/subkey}/Cargo.toml | 10 +- {subkey => bin/subkey}/README.adoc | 0 {subkey => bin/subkey}/src/cli.yml | 0 {subkey => bin/subkey}/src/main.rs | 0 {subkey => bin/subkey}/src/vanity.rs | 0 ci/script.sh | 31 - {core/client => client}/Cargo.toml | 45 +- client/api/Cargo.toml | 41 + {core/client => client/api}/src/backend.rs | 118 +- {core/client => client/api}/src/blockchain.rs | 0 client/api/src/call_executor.rs | 148 + client/api/src/client.rs | 141 + {core/client => client/api}/src/error.rs | 6 + client/api/src/lib.rs | 84 + client/api/src/light.rs | 335 ++ .../api}/src/notifications.rs | 0 client/api/src/offchain.rs | 77 + client/authority-discovery/Cargo.toml | 32 + {core => client}/authority-discovery/build.rs | 0 .../authority-discovery/src/error.rs | 16 +- .../authority-discovery/src/lib.rs | 357 +- .../authority-discovery/src/schema/dht.proto | 0 client/basic-authorship/Cargo.toml | 22 + .../basic-authorship/src/basic_authorship.rs | 24 +- {core => client}/basic-authorship/src/lib.rs | 0 client/block-builder/Cargo.toml | 14 + .../block-builder/src/lib.rs | 92 +- client/chain-spec/Cargo.toml | 15 + {core => client}/chain-spec/derive/Cargo.toml | 4 +- .../chain-spec/derive/src/impls.rs | 0 {core => client}/chain-spec/derive/src/lib.rs | 0 .../chain-spec/res/chain_spec.json | 0 .../chain-spec/res/chain_spec2.json | 0 {core => client}/chain-spec/src/chain_spec.rs | 0 {core => client}/chain-spec/src/extension.rs | 0 {core => client}/chain-spec/src/lib.rs | 0 {core => client}/cli/Cargo.toml | 26 +- {core => client}/cli/README.adoc | 0 {core => client}/cli/src/error.rs | 4 +- .../cli/src/execution_strategy.rs | 0 {core => client}/cli/src/informant.rs | 2 +- {core => client}/cli/src/informant/display.rs | 43 +- {core => client}/cli/src/lib.rs | 130 +- {core => client}/cli/src/params.rs | 62 +- {core => client}/cli/src/traits.rs | 0 {core => client}/consensus/aura/Cargo.toml | 41 +- {core => client}/consensus/aura/src/digest.rs | 0 {core => client}/consensus/aura/src/lib.rs | 427 +- {core => client}/consensus/babe/Cargo.toml | 33 +- .../consensus/babe/src/authorship.rs | 0 .../consensus/babe/src/aux_schema.rs | 6 +- .../consensus/babe/src/epoch_changes.rs | 10 +- {core => client}/consensus/babe/src/lib.rs | 54 +- {core => client}/consensus/babe/src/tests.rs | 7 +- .../consensus/babe/src/verification.rs | 0 client/consensus/pow/Cargo.toml | 21 + {core => client}/consensus/pow/src/lib.rs | 31 +- {core => client}/consensus/slots/Cargo.toml | 12 +- {core => client}/consensus/slots/build.rs | 0 .../consensus/slots/src/aux_schema.rs | 4 +- {core => client}/consensus/slots/src/lib.rs | 11 +- {core => client}/consensus/slots/src/slots.rs | 2 +- client/consensus/uncles/Cargo.toml | 15 + {core => client}/consensus/uncles/src/lib.rs | 6 +- {core/client => client}/db/Cargo.toml | 22 +- .../db/src/cache/list_cache.rs | 2 +- .../db/src/cache/list_entry.rs | 2 +- .../db/src/cache/list_storage.rs | 2 +- {core/client => client}/db/src/cache/mod.rs | 6 +- {core/client => client/db}/src/children.rs | 2 +- {core/client => client}/db/src/lib.rs | 92 +- {core/client => client}/db/src/light.rs | 2 +- {core/client => client}/db/src/offchain.rs | 4 +- client/db/src/storage_cache.rs | 1380 +++++ {core/client => client}/db/src/utils.rs | 0 client/executor/Cargo.toml | 60 + client/executor/runtime-test/Cargo.toml | 20 + .../executor/runtime-test}/build.rs | 2 +- .../executor/runtime-test/src/lib.rs | 59 +- {core => client}/executor/src/allocator.rs | 0 .../executor/src/deprecated_host_interface.rs | 1075 ++++ {core => client}/executor/src/error.rs | 27 +- .../executor/src/host_interface.rs | 139 +- client/executor/src/integration_tests/mod.rs | 459 ++ .../executor/src/integration_tests/sandbox.rs | 360 ++ {core => client}/executor/src/lib.rs | 12 +- .../executor/src/native_executor.rs | 54 +- {core => client}/executor/src/sandbox.rs | 295 +- {core => client}/executor/src/wasm_runtime.rs | 123 +- {core => client}/executor/src/wasm_utils.rs | 17 +- .../executor/src/wasmi_execution.rs | 558 +- .../src/wasmtime/function_executor.rs | 387 ++ .../executor/src/wasmtime}/mod.rs | 15 +- client/executor/src/wasmtime/runtime.rs | 372 ++ client/executor/src/wasmtime/trampoline.rs | 347 ++ client/executor/src/wasmtime/util.rs | 113 + {core => client}/finality-grandpa/Cargo.toml | 30 +- .../finality-grandpa/src/authorities.rs | 8 +- .../finality-grandpa/src/aux_schema.rs | 13 +- .../src/communication/gossip.rs | 304 +- .../finality-grandpa/src/communication/mod.rs | 15 +- .../src/communication/periodic.rs | 110 + .../src/communication/tests.rs | 79 +- .../finality-grandpa/src/consensus_changes.rs | 4 +- .../finality-grandpa/src/environment.rs | 45 +- .../finality-grandpa/src/finality_proof.rs | 126 +- .../finality-grandpa/src/import.rs | 24 +- .../finality-grandpa/src/justification.rs | 5 +- {core => client}/finality-grandpa/src/lib.rs | 76 +- .../finality-grandpa/src/light_import.rs | 69 +- .../finality-grandpa/src/observer.rs | 5 +- .../finality-grandpa/src/tests.rs | 79 +- .../finality-grandpa/src/until_imported.rs | 4 +- .../finality-grandpa/src/voting_rule.rs | 2 +- .../header-metadata/Cargo.toml | 2 +- .../header-metadata/src/lib.rs | 0 {core => client}/keystore/Cargo.toml | 4 +- {core => client}/keystore/src/lib.rs | 0 {core => client}/network/Cargo.toml | 33 +- {core => client}/network/src/behaviour.rs | 8 +- {core => client}/network/src/chain.rs | 27 +- {core => client}/network/src/config.rs | 32 +- {core => client}/network/src/debug_info.rs | 12 +- {core => client}/network/src/discovery.rs | 58 +- {core => client}/network/src/error.rs | 4 +- .../network/src/legacy_proto/behaviour.rs | 33 +- .../network/src/legacy_proto/handler.rs | 78 +- .../network/src/legacy_proto/mod.rs | 0 .../network/src/legacy_proto/tests.rs | 57 +- .../network/src/legacy_proto/upgrade.rs | 64 +- {core => client}/network/src/lib.rs | 0 .../network/src/on_demand_layer.rs | 4 +- {core => client}/network/src/protocol.rs | 329 +- .../network/src/protocol/consensus_gossip.rs | 306 +- .../network/src/protocol/event.rs | 0 .../network/src/protocol/light_dispatch.rs | 42 +- .../network/src/protocol/message.rs | 42 +- .../network/src/protocol/specialization.rs | 3 +- {core => client}/network/src/protocol/sync.rs | 278 +- .../network/src/protocol/sync/blocks.rs | 48 +- .../src/protocol/sync/extra_requests.rs | 4 +- {core => client}/network/src/protocol/util.rs | 0 {core => client}/network/src/service.rs | 30 +- .../network/src/test/block_import.rs | 7 +- {core => client}/network/src/test/mod.rs | 51 +- {core => client}/network/src/test/sync.rs | 70 +- {core => client}/network/src/transport.rs | 18 +- {core => client}/offchain/Cargo.toml | 19 +- {core => client}/offchain/src/api.rs | 4 +- {core => client}/offchain/src/api/http.rs | 37 +- .../offchain/src/api/http_dummy.rs | 0 .../offchain/src/api/timestamp.rs | 0 {core => client}/offchain/src/lib.rs | 15 +- {core => client}/offchain/src/testing.rs | 9 +- client/prometheus/Cargo.toml | 17 + client/prometheus/README.md | 661 ++ client/prometheus/src/lib.rs | 116 + client/prometheus/src/metrics.rs | 77 + client/rpc-servers/Cargo.toml | 17 + {core => client}/rpc-servers/src/lib.rs | 0 {core => client}/rpc/Cargo.toml | 24 +- {core => client}/rpc/api/Cargo.toml | 14 +- {core => client}/rpc/api/src/author/error.rs | 0 {core => client}/rpc/api/src/author/hash.rs | 0 {core => client}/rpc/api/src/author/mod.rs | 0 {core => client}/rpc/api/src/chain/error.rs | 0 {core => client}/rpc/api/src/chain/mod.rs | 0 {core => client}/rpc/api/src/errors.rs | 0 {core => client}/rpc/api/src/helpers.rs | 0 {core => client}/rpc/api/src/lib.rs | 0 {core => client}/rpc/api/src/state/error.rs | 0 {core => client}/rpc/api/src/state/mod.rs | 0 {core => client}/rpc/api/src/subscriptions.rs | 0 {core => client}/rpc/api/src/system/error.rs | 0 .../rpc/api/src/system/helpers.rs | 0 {core => client}/rpc/api/src/system/mod.rs | 0 {core => client}/rpc/src/author/mod.rs | 18 +- {core => client}/rpc/src/author/tests.rs | 0 {core => client}/rpc/src/chain/chain_full.rs | 3 +- {core => client}/rpc/src/chain/chain_light.rs | 2 +- {core => client}/rpc/src/chain/mod.rs | 10 +- {core => client}/rpc/src/chain/tests.rs | 0 {core => client}/rpc/src/lib.rs | 0 {core => client}/rpc/src/metadata.rs | 0 {core => client}/rpc/src/state/mod.rs | 24 +- {core => client}/rpc/src/state/state_full.rs | 13 +- {core => client}/rpc/src/state/state_light.rs | 3 +- {core => client}/rpc/src/state/tests.rs | 2 +- {core => client}/rpc/src/system/mod.rs | 0 {core => client}/rpc/src/system/tests.rs | 0 client/service/Cargo.toml | 63 + {core => client}/service/src/builder.rs | 1211 ++-- {core => client}/service/src/chain_ops.rs | 1 + {core => client}/service/src/config.rs | 30 +- {core => client}/service/src/error.rs | 0 {core => client}/service/src/lib.rs | 33 +- {core => client}/service/src/status_sinks.rs | 0 {core => client}/service/test/Cargo.toml | 12 +- {core => client}/service/test/src/lib.rs | 6 +- {core/client => client}/src/call_executor.rs | 132 +- {core/client => client}/src/cht.rs | 8 +- {core/client => client}/src/client.rs | 481 +- {core/client => client}/src/genesis.rs | 0 {core/client => client}/src/in_mem.rs | 66 +- {core/client => client}/src/leaves.rs | 2 +- {core/client => client}/src/lib.rs | 51 +- {core/client => client}/src/light/backend.rs | 24 +- .../client => client}/src/light/blockchain.rs | 88 +- .../src/light/call_executor.rs | 43 +- {core/client => client}/src/light/fetcher.rs | 278 +- {core/client => client}/src/light/mod.rs | 7 +- {core => client}/state-db/Cargo.toml | 2 +- {core => client}/state-db/src/lib.rs | 0 {core => client}/state-db/src/noncanonical.rs | 0 {core => client}/state-db/src/pruning.rs | 0 {core => client}/state-db/src/test.rs | 0 {core => client}/telemetry/Cargo.toml | 2 +- {core => client}/telemetry/src/lib.rs | 0 {core => client}/telemetry/src/worker.rs | 2 +- {core => client}/telemetry/src/worker/node.rs | 0 client/transaction-pool/Cargo.toml | 21 + .../transaction-pool/graph/Cargo.toml | 6 +- .../transaction-pool/graph/benches/basics.rs | 0 .../transaction-pool/graph/src/base_pool.rs | 0 .../transaction-pool/graph/src/error.rs | 0 .../transaction-pool/graph/src/future.rs | 0 .../transaction-pool/graph/src/lib.rs | 0 .../transaction-pool/graph/src/listener.rs | 10 +- .../transaction-pool/graph/src/pool.rs | 14 +- .../transaction-pool/graph/src/ready.rs | 0 .../transaction-pool/graph/src/rotator.rs | 0 .../graph/src/validated_pool.rs | 15 +- .../transaction-pool/graph/src/watcher.rs | 0 client/transaction-pool/src/api.rs | 112 + .../transaction-pool/src/error.rs | 12 +- {core => client}/transaction-pool/src/lib.rs | 3 +- .../transaction-pool/src/tests.rs | 0 .../utils/wasm-builder-runner/Cargo.toml | 0 .../utils/wasm-builder-runner/README.md | 0 .../utils/wasm-builder-runner/src/lib.rs | 4 +- core/authority-discovery/Cargo.toml | 31 - .../authority-discovery/primitives/Cargo.toml | 21 - core/basic-authorship/Cargo.toml | 20 - core/chain-spec/Cargo.toml | 17 - core/client/db/src/storage_cache.rs | 731 --- core/consensus/aura/primitives/Cargo.toml | 23 - core/consensus/pow/Cargo.toml | 19 - core/consensus/rhd/Cargo.toml | 37 - core/consensus/rhd/src/error.rs | 50 - core/consensus/rhd/src/lib.rs | 1699 ------ core/consensus/rhd/src/misbehaviour_check.rs | 191 - core/consensus/rhd/src/service.rs | 181 - core/consensus/uncles/Cargo.toml | 15 - core/executor/Cargo.toml | 37 - core/executor/runtime-test/Cargo.toml | 20 - .../src/communication/periodic.rs | 252 - core/prometheus/Cargo.toml | 16 + core/prometheus/src/lib.rs | 62 + core/prometheus/src/metrics.rs | 25 + core/rpc-servers/Cargo.toml | 17 - core/service/Cargo.toml | 50 - core/sr-api-macros/Cargo.toml | 37 - core/sr-api-macros/tests/trybuild.rs | 11 - .../tests/ui/adding_at_parameter.rs | 9 - .../tests/ui/adding_at_parameter.stderr | 5 - .../tests/ui/adding_self_parameter.stderr | 5 - .../ui/empty_impl_runtime_apis_call.stderr | 5 - .../ui/impl_incorrect_method_signature.stderr | 51 - .../ui/impl_two_traits_with_same_name.stderr | 17 - .../tests/ui/invalid_api_version.stderr | 29 - .../tests/ui/invalid_api_version_2.stderr | 29 - .../tests/ui/invalid_api_version_3.stderr | 29 - .../ui/missing_block_generic_parameter.stderr | 13 - ...reference_in_impl_runtime_apis_call.stderr | 42 - core/sr-io/build.rs | 13 - core/sr-io/src/lib.rs | 397 -- core/sr-sandbox/build.rs | 13 - core/sr-std/build.rs | 13 - core/test-client/Cargo.toml | 18 - core/test-runtime/Cargo.toml | 75 - core/test-runtime/client/Cargo.toml | 20 - core/transaction-pool/Cargo.toml | 20 - core/transaction-pool/src/api.rs | 88 - CODEOWNERS => docs/CODEOWNERS | 16 +- .../CODE_OF_CONDUCT.adoc | 0 CONTRIBUTING.adoc => docs/CONTRIBUTING.adoc | 0 .../PULL_REQUEST_TEMPLATE.md | 0 docs/README.adoc | 528 ++ SECURITY.md => docs/SECURITY.md | 0 docs/Structure.adoc | 110 + license_header.txt => docs/license_header.txt | 0 node-template/Cargo.toml | 40 - node-template/build.rs | 24 - node-template/runtime/Cargo.toml | 61 - node/cli/Cargo.toml | 78 - node/executor/Cargo.toml | 37 - node/rpc/Cargo.toml | 16 - node/runtime/Cargo.toml | 109 - node/testing/Cargo.toml | 31 - {srml => paint}/assets/Cargo.toml | 14 +- {srml => paint}/assets/src/lib.rs | 4 +- paint/aura/Cargo.toml | 41 + {srml => paint}/aura/src/lib.rs | 22 +- {srml => paint}/aura/src/mock.rs | 0 {srml => paint}/aura/src/tests.rs | 0 paint/authority-discovery/Cargo.toml | 37 + paint/authority-discovery/src/lib.rs | 266 + {srml => paint}/authorship/Cargo.toml | 18 +- {srml => paint}/authorship/src/lib.rs | 11 +- paint/babe/Cargo.toml | 44 + {srml => paint}/babe/src/lib.rs | 22 +- {srml => paint}/babe/src/mock.rs | 0 {srml => paint}/babe/src/tests.rs | 0 {srml => paint}/balances/Cargo.toml | 18 +- {srml => paint}/balances/src/lib.rs | 63 +- {srml => paint}/balances/src/mock.rs | 0 {srml => paint}/balances/src/tests.rs | 56 +- {srml => paint}/collective/Cargo.toml | 16 +- {srml => paint}/collective/src/lib.rs | 0 {srml => paint}/contracts/COMPLEXITY.md | 0 {srml => paint}/contracts/Cargo.toml | 22 +- {srml => paint}/contracts/rpc/Cargo.toml | 18 +- .../contracts/rpc/runtime-api/Cargo.toml | 10 +- .../contracts/rpc/runtime-api/src/lib.rs | 2 +- {srml => paint}/contracts/rpc/src/lib.rs | 2 +- {srml => paint}/contracts/src/account_db.rs | 2 +- {srml => paint}/contracts/src/exec.rs | 0 {srml => paint}/contracts/src/gas.rs | 0 {srml => paint}/contracts/src/lib.rs | 6 +- {srml => paint}/contracts/src/rent.rs | 6 +- {srml => paint}/contracts/src/tests.rs | 0 .../contracts/src/wasm/code_cache.rs | 0 .../contracts/src/wasm/env_def/macros.rs | 0 .../contracts/src/wasm/env_def/mod.rs | 0 {srml => paint}/contracts/src/wasm/mod.rs | 8 +- {srml => paint}/contracts/src/wasm/prepare.rs | 0 {srml => paint}/contracts/src/wasm/runtime.rs | 0 {srml => paint}/democracy/Cargo.toml | 16 +- {srml => paint}/democracy/src/lib.rs | 0 .../democracy/src/vote_threshold.rs | 0 paint/elections-phragmen/Cargo.toml | 31 + {srml => paint}/elections-phragmen/src/lib.rs | 10 +- {srml => paint}/elections/Cargo.toml | 16 +- {srml => paint}/elections/src/lib.rs | 0 {srml => paint}/elections/src/mock.rs | 0 {srml => paint}/elections/src/tests.rs | 0 paint/evm/Cargo.toml | 40 + paint/evm/src/backend.rs | 187 + paint/evm/src/lib.rs | 299 + {srml => paint}/example/Cargo.toml | 16 +- {srml => paint}/example/src/lib.rs | 15 +- paint/executive/Cargo.toml | 33 + {srml => paint}/executive/src/lib.rs | 101 +- paint/finality-tracker/Cargo.toml | 31 + {srml => paint}/finality-tracker/src/lib.rs | 24 +- {srml => paint}/generic-asset/Cargo.toml | 14 +- {srml => paint}/generic-asset/src/lib.rs | 36 +- {srml => paint}/generic-asset/src/mock.rs | 0 {srml => paint}/generic-asset/src/tests.rs | 0 paint/grandpa/Cargo.toml | 38 + {srml => paint}/grandpa/src/lib.rs | 63 +- {srml => paint}/grandpa/src/mock.rs | 4 +- {srml => paint}/grandpa/src/tests.rs | 18 + paint/im-online/Cargo.toml | 39 + {srml => paint}/im-online/src/lib.rs | 215 +- {srml => paint}/im-online/src/mock.rs | 12 + {srml => paint}/im-online/src/tests.rs | 107 +- {srml => paint}/indices/Cargo.toml | 16 +- {srml => paint}/indices/src/address.rs | 0 {srml => paint}/indices/src/lib.rs | 0 {srml => paint}/indices/src/mock.rs | 0 {srml => paint}/indices/src/tests.rs | 0 {srml => paint}/membership/Cargo.toml | 14 +- {srml => paint}/membership/src/lib.rs | 0 {srml => paint}/metadata/Cargo.toml | 6 +- {srml => paint}/metadata/src/lib.rs | 0 {srml => paint}/nicks/Cargo.toml | 16 +- {srml => paint}/nicks/src/lib.rs | 0 paint/offences/Cargo.toml | 32 + {srml => paint}/offences/src/lib.rs | 0 {srml => paint}/offences/src/mock.rs | 2 +- {srml => paint}/offences/src/tests.rs | 0 .../randomness-collective-flip/Cargo.toml | 14 +- .../randomness-collective-flip/src/lib.rs | 2 +- {srml => paint}/scored-pool/Cargo.toml | 16 +- {srml => paint}/scored-pool/src/lib.rs | 4 +- {srml => paint}/scored-pool/src/mock.rs | 0 {srml => paint}/scored-pool/src/tests.rs | 0 {srml => paint}/session/Cargo.toml | 22 +- {srml => paint}/session/src/historical.rs | 0 {srml => paint}/session/src/lib.rs | 19 +- {srml => paint}/session/src/mock.rs | 0 paint/staking/Cargo.toml | 45 + .../staking/reward-curve/Cargo.toml | 8 +- .../staking/reward-curve/src/lib.rs | 13 +- .../staking/reward-curve/src/log.rs | 0 .../staking/reward-curve/tests/test.rs | 6 +- {srml => paint}/staking/src/inflation.rs | 51 +- {srml => paint}/staking/src/lib.rs | 53 +- {srml => paint}/staking/src/mock.rs | 10 +- {srml => paint}/staking/src/tests.rs | 0 {srml => paint}/sudo/Cargo.toml | 14 +- {srml => paint}/sudo/src/lib.rs | 4 +- paint/support/Cargo.toml | 43 + paint/support/procedural/Cargo.toml | 15 + {srml => paint}/support/procedural/src/lib.rs | 26 +- .../src/storage/genesis_config/builder_def.rs | 2 +- .../genesis_config/genesis_config_def.rs | 40 +- .../src/storage/genesis_config/mod.rs | 19 +- .../support/procedural/src/storage/getters.rs | 0 .../procedural/src/storage/instance_trait.rs | 0 .../procedural/src/storage/metadata.rs | 4 +- .../support/procedural/src/storage/mod.rs | 8 +- .../support/procedural/src/storage/parse.rs | 2 +- .../procedural/src/storage/storage_struct.rs | 13 +- .../procedural/src/storage/store_trait.rs | 0 paint/support/procedural/tools/Cargo.toml | 12 + .../procedural/tools/derive/Cargo.toml | 13 + .../procedural/tools/derive/src/lib.rs | 0 .../support/procedural/tools/src/lib.rs | 6 +- .../support/procedural/tools/src/syn_ext.rs | 20 +- paint/support/rpc/Cargo.toml | 19 + paint/support/rpc/src/lib.rs | 155 + {srml => paint}/support/src/debug.rs | 16 +- {srml => paint}/support/src/dispatch.rs | 189 +- {srml => paint}/support/src/error.rs | 4 +- {srml => paint}/support/src/event.rs | 12 +- {srml => paint}/support/src/hash.rs | 4 +- {srml => paint}/support/src/inherent.rs | 0 {srml => paint}/support/src/lib.rs | 36 +- {srml => paint}/support/src/metadata.rs | 10 +- {srml => paint}/support/src/origin.rs | 0 {srml => paint}/support/src/runtime.rs | 0 {srml => paint}/support/src/storage/child.rs | 26 +- .../src/storage/generator/double_map.rs | 0 .../src/storage/generator/linked_map.rs | 216 +- .../support/src/storage/generator/map.rs | 0 paint/support/src/storage/generator/mod.rs | 132 + .../support/src/storage/generator/value.rs | 19 +- {srml => paint}/support/src/storage/hashed.rs | 0 {srml => paint}/support/src/storage/mod.rs | 59 +- .../support/src/storage/unhashed.rs | 14 +- {srml => paint}/support/src/traits.rs | 40 +- {srml => paint}/support/src/unsigned.rs | 19 +- {srml => paint}/support/test/Cargo.toml | 14 +- {srml => paint}/support/test/src/lib.rs | 2 +- .../support/test/tests/decl_storage.rs | 290 +- .../support/test/tests/decl_storage_ui.rs | 0 .../tests/decl_storage_ui/config_duplicate.rs | 0 .../decl_storage_ui/config_duplicate.stderr | 0 .../decl_storage_ui/config_get_duplicate.rs | 0 .../config_get_duplicate.stderr | 0 .../tests/decl_storage_ui/get_duplicate.rs | 0 .../decl_storage_ui/get_duplicate.stderr | 0 .../support/test/tests/final_keys.rs | 76 +- .../support/test/tests/genesisconfig.rs | 0 .../support/test/tests/instance.rs | 10 +- .../support/test/tests/issue2219.rs | 0 .../support/test/tests/reserved_keyword.rs | 0 .../tests/reserved_keyword/on_initialize.rs | 0 .../reserved_keyword/on_initialize.stderr | 0 {srml => paint}/support/test/tests/system.rs | 0 {srml => paint}/system/Cargo.toml | 16 +- {srml => paint}/system/benches/bench.rs | 2 +- {srml => paint}/system/rpc/Cargo.toml | 20 +- .../system/rpc/runtime-api/Cargo.toml | 6 +- .../system/rpc/runtime-api/src/lib.rs | 2 +- {srml => paint}/system/rpc/src/lib.rs | 2 +- {srml => paint}/system/src/lib.rs | 31 +- {srml => paint}/system/src/offchain.rs | 4 +- paint/timestamp/Cargo.toml | 31 + {srml => paint}/timestamp/src/lib.rs | 28 +- paint/transaction-payment/Cargo.toml | 29 + paint/transaction-payment/rpc/Cargo.toml | 17 + .../rpc/runtime-api/Cargo.toml | 10 +- .../rpc/runtime-api/src/lib.rs | 2 +- .../transaction-payment/rpc/src/lib.rs | 4 +- .../transaction-payment/src/lib.rs | 0 {srml => paint}/treasury/Cargo.toml | 16 +- {srml => paint}/treasury/src/lib.rs | 95 +- {srml => paint}/utility/Cargo.toml | 16 +- {srml => paint}/utility/src/lib.rs | 0 .../application-crypto/Cargo.toml | 17 +- .../application-crypto/src/ed25519.rs | 17 +- .../application-crypto/src/lib.rs | 190 +- .../application-crypto/src/sr25519.rs | 17 +- .../application-crypto/src/traits.rs | 35 +- primitives/authority-discovery/Cargo.toml | 23 + .../authority-discovery}/src/lib.rs | 37 +- .../block-builder/runtime-api/Cargo.toml | 20 + .../block-builder/runtime-api/src/lib.rs | 19 +- .../consensus/aura}/Cargo.toml | 17 +- .../consensus/aura}/src/lib.rs | 3 +- .../consensus/babe}/Cargo.toml | 20 +- .../consensus/babe}/src/digest.rs | 0 .../consensus/babe}/src/lib.rs | 3 +- .../consensus/common/Cargo.toml | 6 +- .../consensus/common/src/block_import.rs | 9 + .../consensus/common/src/block_validation.rs | 0 .../consensus/common/src/error.rs | 2 +- .../consensus/common/src/evaluation.rs | 0 .../consensus/common/src/import_queue.rs | 18 +- .../common/src/import_queue/basic_queue.rs | 0 .../common/src/import_queue/buffered_link.rs | 0 .../consensus/common/src/lib.rs | 0 .../consensus/common/src/offline_tracker.rs | 0 .../consensus/common/src/select_chain.rs | 0 .../consensus/pow}/Cargo.toml | 10 +- .../consensus/pow}/src/lib.rs | 3 +- .../primitives => primitives/core}/Cargo.toml | 54 +- .../core}/benches/bench.rs | 0 .../core}/debug-derive/Cargo.toml | 2 +- .../core}/debug-derive/src/impls.rs | 0 .../core}/debug-derive/src/lib.rs | 0 .../core}/debug-derive/tests/tests.rs | 0 .../core}/src/changes_trie.rs | 0 .../core}/src/crypto.rs | 57 +- .../core}/src/ecdsa.rs | 56 +- .../core}/src/ed25519.rs | 66 +- .../core}/src/hash.rs | 0 .../core}/src/hasher.rs | 0 .../core}/src/hashing.rs | 10 + .../core}/src/hexdisplay.rs | 0 .../primitives => primitives/core}/src/lib.rs | 14 +- .../core}/src/offchain.rs | 21 +- .../core}/src/sandbox.rs | 0 .../core}/src/sr25519.rs | 77 +- .../core}/src/testing.rs | 0 .../core}/src/tests.rs | 0 .../core}/src/traits.rs | 0 .../core}/src/u32_trait.rs | 0 .../core}/src/uint.rs | 4 +- .../core}/storage/Cargo.toml | 0 .../core}/storage/src/lib.rs | 8 + {core => primitives}/externalities/Cargo.toml | 4 +- .../externalities/src/extensions.rs | 30 +- {core => primitives}/externalities/src/lib.rs | 6 +- .../externalities/src/scope_limited.rs | 0 primitives/finality-grandpa/Cargo.toml | 24 + .../finality-grandpa}/src/lib.rs | 66 +- {core => primitives}/inherents/Cargo.toml | 8 +- {core => primitives}/inherents/src/lib.rs | 52 +- {core => primitives}/keyring/Cargo.toml | 2 +- {core => primitives}/keyring/src/ed25519.rs | 0 {core => primitives}/keyring/src/lib.rs | 0 {core => primitives}/keyring/src/sr25519.rs | 0 .../offchain}/Cargo.toml | 6 +- .../offchain}/src/lib.rs | 6 +- {core => primitives}/panic-handler/Cargo.toml | 0 {core => primitives}/panic-handler/src/lib.rs | 0 {core => primitives}/peerset/Cargo.toml | 2 +- {core => primitives}/peerset/src/lib.rs | 0 .../peerset/src/peersstate.rs | 0 {core => primitives}/peerset/tests/fuzz.rs | 0 {core => primitives}/phragmen/Cargo.toml | 8 +- .../phragmen/benches/phragmen.rs | 0 {core => primitives}/phragmen/src/lib.rs | 125 +- {core => primitives}/phragmen/src/mock.rs | 42 +- {core => primitives}/phragmen/src/tests.rs | 45 +- .../primitives => primitives/rpc}/Cargo.toml | 2 +- .../primitives => primitives/rpc}/src/lib.rs | 0 .../rpc}/src/number.rs | 0 primitives/runtime-interface/Cargo.toml | 41 + .../runtime-interface/proc-macro/Cargo.toml | 27 + .../runtime-interface/proc-macro/src/lib.rs | 265 + .../proc-macro/src/pass_by/codec.rs | 58 + .../proc-macro/src/pass_by/enum_.rs | 101 + .../proc-macro/src/pass_by/inner.rs | 110 + .../proc-macro/src/pass_by/mod.rs | 25 + .../bare_function_interface.rs | 186 + .../host_function_interface.rs | 415 ++ .../proc-macro/src/runtime_interface/mod.rs | 65 + .../src/runtime_interface/trait_decl_impl.rs | 146 + .../runtime-interface/proc-macro/src/utils.rs | 162 + .../runtime-interface/proc-macro/tests/ui.rs | 27 + .../tests/ui/no_generic_parameters.rs | 8 + .../tests/ui/no_generic_parameters.stderr | 11 + .../tests/ui/no_method_implementation.rs | 8 + .../tests/ui/no_method_implementation.stderr | 5 + .../tests/ui/pass_by_enum_with_struct.rs | 6 + .../tests/ui/pass_by_enum_with_struct.stderr | 5 + .../ui/pass_by_enum_with_value_variant.rs | 8 + .../ui/pass_by_enum_with_value_variant.stderr | 5 + .../tests/ui/pass_by_inner_with_two_fields.rs | 9 + .../ui/pass_by_inner_with_two_fields.stderr | 5 + .../proc-macro/tests/ui/take_self_by_value.rs | 8 + .../tests/ui/take_self_by_value.stderr | 5 + primitives/runtime-interface/src/host.rs | 62 + primitives/runtime-interface/src/impls.rs | 491 ++ primitives/runtime-interface/src/lib.rs | 212 + primitives/runtime-interface/src/pass_by.rs | 384 ++ primitives/runtime-interface/src/wasm.rs | 142 + .../runtime-interface/test-wasm/Cargo.toml | 19 + .../runtime-interface/test-wasm/build.rs | 30 + .../runtime-interface/test-wasm/src/lib.rs | 194 + {core => primitives}/serializer/Cargo.toml | 0 {core => primitives}/serializer/src/lib.rs | 0 {core => primitives}/session/Cargo.toml | 5 +- {core => primitives}/session/src/lib.rs | 29 +- primitives/sr-api/Cargo.toml | 33 + .../sr-api}/benches/bench.rs | 4 +- primitives/sr-api/proc-macro/Cargo.toml | 26 + .../proc-macro}/src/decl_runtime_apis.rs | 207 +- .../proc-macro}/src/impl_runtime_apis.rs | 144 +- .../sr-api/proc-macro}/src/lib.rs | 29 +- .../sr-api/proc-macro}/src/utils.rs | 78 +- .../sr-api/src/lib.rs | 80 +- primitives/sr-api/test/Cargo.toml | 21 + .../sr-api/test}/tests/decl_and_impl.rs | 43 +- .../sr-api/test}/tests/runtime_calls.rs | 0 primitives/sr-api/test/tests/trybuild.rs | 27 + .../test}/tests/ui/adding_self_parameter.rs | 4 +- .../tests/ui/adding_self_parameter.stderr | 5 + .../tests/ui/changed_in_unknown_version.rs | 3 +- .../ui/changed_in_unknown_version.stderr | 4 +- .../test}/tests/ui/declaring_old_block.rs | 3 +- .../test}/tests/ui/declaring_old_block.stderr | 8 +- ...declaring_own_block_with_different_name.rs | 3 +- ...aring_own_block_with_different_name.stderr | 4 +- .../tests/ui/empty_impl_runtime_apis_call.rs | 5 +- .../ui/empty_impl_runtime_apis_call.stderr | 5 + .../ui/impl_incorrect_method_signature.rs | 7 +- .../ui/impl_incorrect_method_signature.stderr | 70 + .../ui/impl_two_traits_with_same_name.rs | 5 +- .../ui/impl_two_traits_with_same_name.stderr | 77 + .../test}/tests/ui/invalid_api_version.rs | 4 +- .../test/tests/ui/invalid_api_version.stderr | 33 + .../test}/tests/ui/invalid_api_version_2.rs | 4 +- .../tests/ui/invalid_api_version_2.stderr | 33 + .../test}/tests/ui/invalid_api_version_3.rs | 4 +- .../tests/ui/invalid_api_version_3.stderr | 33 + .../ui/missing_block_generic_parameter.rs | 5 +- .../ui/missing_block_generic_parameter.stderr | 11 + .../test}/tests/ui/missing_path_for_trait.rs | 5 +- .../tests/ui/missing_path_for_trait.stderr | 4 +- ...ype_reference_in_impl_runtime_apis_call.rs | 7 +- ...reference_in_impl_runtime_apis_call.stderr | 73 + {core => primitives}/sr-arithmetic/Cargo.toml | 2 +- .../sr-arithmetic/benches/bench.rs | 0 .../sr-arithmetic/fuzzer/.gitignore | 0 .../sr-arithmetic/fuzzer/Cargo.lock | 110 +- .../sr-arithmetic/fuzzer/Cargo.toml | 0 .../sr-arithmetic/fuzzer/src/biguint.rs | 0 .../sr-arithmetic/fuzzer/src/rational128.rs | 0 .../sr-arithmetic/src/biguint.rs | 0 .../sr-arithmetic/src/fixed64.rs | 0 .../sr-arithmetic/src/helpers_128bit.rs | 0 {core => primitives}/sr-arithmetic/src/lib.rs | 0 .../sr-arithmetic/src/per_things.rs | 0 .../sr-arithmetic/src/rational128.rs | 0 .../sr-arithmetic/src/traits.rs | 0 {core => primitives}/sr-io/Cargo.toml | 25 +- primitives/sr-io/src/lib.rs | 840 +++ {core => primitives}/sr-io/with_std.rs | 6 +- {core => primitives}/sr-io/without_std.rs | 0 {core => primitives}/sr-primitives/Cargo.toml | 8 +- .../sr-primitives/src/curve.rs | 7 +- .../sr-primitives/src/generic/block.rs | 0 .../src/generic/checked_extrinsic.rs | 15 +- .../sr-primitives/src/generic/digest.rs | 0 .../sr-primitives/src/generic/era.rs | 0 .../sr-primitives/src/generic/header.rs | 0 .../sr-primitives/src/generic/mod.rs | 0 .../sr-primitives/src/generic/tests.rs | 0 .../src/generic/unchecked_extrinsic.rs | 7 +- {core => primitives}/sr-primitives/src/lib.rs | 36 +- .../sr-primitives/src/offchain/http.rs | 25 +- .../sr-primitives/src/offchain/mod.rs | 0 .../sr-primitives/src/testing.rs | 26 +- .../sr-primitives/src/traits.rs | 72 +- .../sr-primitives/src/transaction_validity.rs | 0 .../sr-primitives/src/weights.rs | 39 +- {core => primitives}/sr-sandbox/Cargo.toml | 11 +- {core => primitives}/sr-sandbox/src/lib.rs | 6 +- {core => primitives}/sr-sandbox/with_std.rs | 16 +- .../sr-sandbox/without_std.rs | 138 +- .../sr-staking-primitives/Cargo.toml | 0 .../sr-staking-primitives/src/lib.rs | 0 .../sr-staking-primitives/src/offence.rs | 0 {core => primitives}/sr-std/Cargo.toml | 7 - {core => primitives}/sr-std/src/lib.rs | 8 +- {core => primitives}/sr-std/with_std.rs | 2 + {core => primitives}/sr-std/without_std.rs | 30 +- {core => primitives}/sr-version/Cargo.toml | 0 {core => primitives}/sr-version/src/lib.rs | 2 +- {core => primitives}/state-machine/Cargo.toml | 2 +- .../state-machine/src/backend.rs | 40 +- .../state-machine/src/basic.rs | 40 +- .../state-machine/src/changes_trie/build.rs | 0 .../src/changes_trie/build_cache.rs | 0 .../src/changes_trie/build_iterator.rs | 0 .../src/changes_trie/changes_iterator.rs | 4 +- .../state-machine/src/changes_trie/input.rs | 0 .../state-machine/src/changes_trie/mod.rs | 0 .../state-machine/src/changes_trie/prune.rs | 25 +- .../state-machine/src/changes_trie/storage.rs | 0 .../src/changes_trie/surface_iterator.rs | 0 .../state-machine/src/error.rs | 0 {core => primitives}/state-machine/src/ext.rs | 0 {core => primitives}/state-machine/src/lib.rs | 26 +- .../state-machine/src/overlayed_changes.rs | 40 +- .../state-machine/src/proving_backend.rs | 200 +- .../state-machine/src/testing.rs | 7 +- .../state-machine/src/trie_backend.rs | 0 .../state-machine/src/trie_backend_essence.rs | 0 .../transaction-pool/runtime-api/Cargo.toml | 14 + .../transaction-pool/runtime-api/src/lib.rs | 29 + {core => primitives}/trie/Cargo.toml | 3 +- {core => primitives}/trie/benches/bench.rs | 0 {core => primitives}/trie/src/error.rs | 0 {core => primitives}/trie/src/lib.rs | 0 {core => primitives}/trie/src/node_codec.rs | 0 {core => primitives}/trie/src/node_header.rs | 0 {core => primitives}/trie/src/trie_stream.rs | 0 .../wasm-interface/Cargo.toml | 3 +- .../wasm-interface/src/lib.rs | 93 +- .../wasm-interface/src/wasmi_impl.rs | 0 srml/README.adoc | 26 - srml/aura/Cargo.toml | 41 - srml/authority-discovery/Cargo.toml | 36 - srml/authority-discovery/src/lib.rs | 362 -- srml/babe/Cargo.toml | 44 - srml/elections-phragmen/Cargo.toml | 31 - srml/executive/Cargo.toml | 33 - srml/finality-tracker/Cargo.toml | 31 - srml/grandpa/Cargo.toml | 37 - srml/im-online/Cargo.toml | 37 - srml/offences/Cargo.toml | 32 - srml/staking/Cargo.toml | 45 - srml/support/Cargo.toml | 41 - srml/support/procedural/Cargo.toml | 16 - srml/support/procedural/tools/Cargo.toml | 12 - .../procedural/tools/derive/Cargo.toml | 13 - srml/support/src/storage/generator/mod.rs | 32 - srml/timestamp/Cargo.toml | 31 - srml/transaction-payment/Cargo.toml | 29 - srml/transaction-payment/rpc/Cargo.toml | 17 - test-utils/transaction-factory/Cargo.toml | 22 - .../utils}/chain-spec-builder/Cargo.toml | 6 +- .../utils}/chain-spec-builder/build.rs | 0 .../utils}/chain-spec-builder/src/main.rs | 7 +- test/utils/client/Cargo.toml | 19 + .../utils/client}/src/client_ext.rs | 31 +- .../utils/client}/src/lib.rs | 12 +- test/utils/primitives/Cargo.toml | 71 + test/utils/primitives/src/lib.rs | 78 + test/utils/runtime/Cargo.toml | 83 + .../utils/runtime}/build.rs | 2 +- test/utils/runtime/client/Cargo.toml | 15 + .../runtime}/client/src/block_builder_ext.rs | 21 +- .../utils/runtime}/client/src/lib.rs | 2 +- .../utils/runtime}/client/src/trait_tests.rs | 10 +- .../utils/runtime}/src/genesismap.rs | 14 +- .../utils/runtime}/src/lib.rs | 60 +- .../utils/runtime}/src/system.rs | 38 +- test/utils/transaction-factory/Cargo.toml | 19 + .../transaction-factory/src/complex_mode.rs | 9 +- .../utils}/transaction-factory/src/lib.rs | 19 +- .../utils}/transaction-factory/src/modes.rs | 0 .../transaction-factory/src/simple_modes.rs | 9 +- utils/build-script-utils/Cargo.toml | 7 + utils/build-script-utils/src/lib.rs | 44 + {core/utils => utils}/fork-tree/Cargo.toml | 0 {core/utils => utils}/fork-tree/src/lib.rs | 0 {core/utils => utils}/wasm-builder/Cargo.toml | 5 +- {core/utils => utils}/wasm-builder/README.md | 23 +- {core/utils => utils}/wasm-builder/src/lib.rs | 30 +- .../wasm-builder/src/prerequisites.rs | 0 .../wasm-builder/src/wasm_project.rs | 6 +- 856 files changed, 27613 insertions(+), 14877 deletions(-) rename Dockerfile => .maintain/Dockerfile (100%) rename {scripts => .maintain}/build-only-wasm.sh (100%) rename {ci => .maintain}/check_for_exit.sh (100%) rename {scripts => .maintain}/common.sh (100%) rename {scripts => .maintain}/docker/subkey.Dockerfile (93%) rename {scripts => .maintain}/docker/substrate.Dockerfile (96%) create mode 100755 .maintain/ensure-deps.sh rename {scripts => .maintain}/flamingfir-deploy.sh (100%) rename {scripts => .maintain}/getgoing.sh (100%) rename {scripts => .maintain}/gitlab/check_line_width.sh (100%) rename {scripts => .maintain}/gitlab/check_runtime.sh (93%) rename {node-template/scripts => .maintain}/init.sh (100%) rename {scripts => .maintain}/kubernetes/Chart.yaml (100%) rename {scripts => .maintain}/kubernetes/README.md (100%) rename {scripts => .maintain}/kubernetes/templates/poddisruptionbudget.yaml (100%) rename {scripts => .maintain}/kubernetes/templates/secrets.yaml (100%) rename {scripts => .maintain}/kubernetes/templates/service.yaml (100%) rename {scripts => .maintain}/kubernetes/templates/serviceaccount.yaml (100%) rename {scripts => .maintain}/kubernetes/templates/statefulset.yaml (100%) rename {scripts => .maintain}/kubernetes/values.yaml (100%) rename {scripts => .maintain}/node-template-release.sh (84%) create mode 100644 .maintain/node-template-release/Cargo.lock rename {scripts => .maintain}/node-template-release/Cargo.toml (100%) rename {scripts => .maintain}/node-template-release/src/main.rs (86%) rename {scripts => .maintain}/runtime-dep.py (91%) rename rustdoc-header.html => .maintain/rustdoc-header.html (100%) rename {scripts => .maintain}/sentry-node/docker-compose.yml (97%) rename {scripts => .maintain}/update-copyright.sh (100%) rename {scripts => .maintain}/update-deps.sh (100%) rename {scripts => .maintain}/update.sh (100%) delete mode 100644 .travis.yml create mode 100644 bin/node-template/Cargo.toml rename {node-template => bin/node-template}/LICENSE (100%) rename {node-template => bin/node-template}/README.md (100%) create mode 100644 bin/node-template/build.rs create mode 100644 bin/node-template/runtime/Cargo.toml rename {node-template => bin/node-template}/runtime/build.rs (100%) rename {node-template => bin/node-template}/runtime/src/lib.rs (93%) rename {node-template => bin/node-template}/runtime/src/template.rs (98%) rename {scripts => bin/node-template/scripts}/init.sh (100%) rename {node-template => bin/node-template}/src/chain_spec.rs (100%) rename {node-template => bin/node-template}/src/cli.rs (97%) rename {node-template => bin/node-template}/src/main.rs (100%) rename {node-template => bin/node-template}/src/service.rs (97%) create mode 100644 bin/node/cli/Cargo.toml rename {node => bin/node}/cli/bin/main.rs (93%) create mode 100644 bin/node/cli/browser-demo/.gitignore create mode 100644 bin/node/cli/browser-demo/README.md create mode 100755 bin/node/cli/browser-demo/build.sh create mode 100644 bin/node/cli/browser-demo/favicon.png create mode 100644 bin/node/cli/browser-demo/index.html create mode 100644 bin/node/cli/browser-demo/ws.js rename {node => bin/node}/cli/build.rs (90%) rename {node => bin/node}/cli/doc/shell-completion.adoc (100%) rename {node => bin/node}/cli/res/flaming-fir.json (100%) create mode 100644 bin/node/cli/src/browser.rs rename {node => bin/node}/cli/src/chain_spec.rs (88%) rename node/cli/src/lib.rs => bin/node/cli/src/cli.rs (78%) rename {node => bin/node}/cli/src/factory_impl.rs (99%) create mode 100644 bin/node/cli/src/lib.rs rename {node => bin/node}/cli/src/service.rs (94%) create mode 100644 bin/node/executor/Cargo.toml create mode 100644 bin/node/executor/benches/bench.rs rename {node => bin/node}/executor/src/lib.rs (98%) rename {node => bin/node}/primitives/Cargo.toml (50%) rename {node => bin/node}/primitives/src/lib.rs (100%) rename {node => bin/node}/rpc-client/Cargo.toml (66%) rename {node => bin/node}/rpc-client/src/main.rs (100%) create mode 100644 bin/node/rpc/Cargo.toml rename {node => bin/node}/rpc/src/lib.rs (81%) create mode 100644 bin/node/runtime/Cargo.toml rename {core/executor/runtime-test => bin/node/runtime}/build.rs (96%) rename {node => bin/node}/runtime/src/constants.rs (100%) rename {node => bin/node}/runtime/src/impls.rs (99%) rename {node => bin/node}/runtime/src/lib.rs (91%) create mode 100644 bin/node/testing/Cargo.toml rename {node => bin/node}/testing/src/client.rs (100%) rename {node => bin/node}/testing/src/genesis.rs (98%) rename {node => bin/node}/testing/src/keyring.rs (96%) rename {node => bin/node}/testing/src/lib.rs (100%) rename {subkey => bin/subkey}/Cargo.toml (61%) rename {subkey => bin/subkey}/README.adoc (100%) rename {subkey => bin/subkey}/src/cli.yml (100%) rename {subkey => bin/subkey}/src/main.rs (100%) rename {subkey => bin/subkey}/src/vanity.rs (100%) delete mode 100755 ci/script.sh rename {core/client => client}/Cargo.toml (53%) create mode 100644 client/api/Cargo.toml rename {core/client => client/api}/src/backend.rs (82%) rename {core/client => client/api}/src/blockchain.rs (100%) create mode 100644 client/api/src/call_executor.rs create mode 100644 client/api/src/client.rs rename {core/client => client/api}/src/error.rs (96%) create mode 100644 client/api/src/lib.rs create mode 100644 client/api/src/light.rs rename {core/client => client/api}/src/notifications.rs (100%) create mode 100644 client/api/src/offchain.rs create mode 100644 client/authority-discovery/Cargo.toml rename {core => client}/authority-discovery/build.rs (100%) rename {core => client}/authority-discovery/src/error.rs (84%) rename {core => client}/authority-discovery/src/lib.rs (66%) rename {core => client}/authority-discovery/src/schema/dht.proto (100%) create mode 100644 client/basic-authorship/Cargo.toml rename {core => client}/basic-authorship/src/basic_authorship.rs (91%) rename {core => client}/basic-authorship/src/lib.rs (100%) create mode 100644 client/block-builder/Cargo.toml rename core/client/src/block_builder/block_builder.rs => client/block-builder/src/lib.rs (61%) create mode 100644 client/chain-spec/Cargo.toml rename {core => client}/chain-spec/derive/Cargo.toml (86%) rename {core => client}/chain-spec/derive/src/impls.rs (100%) rename {core => client}/chain-spec/derive/src/lib.rs (100%) rename {core => client}/chain-spec/res/chain_spec.json (100%) rename {core => client}/chain-spec/res/chain_spec2.json (100%) rename {core => client}/chain-spec/src/chain_spec.rs (100%) rename {core => client}/chain-spec/src/extension.rs (100%) rename {core => client}/chain-spec/src/lib.rs (100%) rename {core => client}/cli/Cargo.toml (55%) rename {core => client}/cli/README.adoc (100%) rename {core => client}/cli/src/error.rs (97%) rename {core => client}/cli/src/execution_strategy.rs (100%) rename {core => client}/cli/src/informant.rs (98%) rename {core => client}/cli/src/informant/display.rs (85%) rename {core => client}/cli/src/lib.rs (92%) rename {core => client}/cli/src/params.rs (91%) rename {core => client}/cli/src/traits.rs (100%) rename {core => client}/consensus/aura/Cargo.toml (53%) rename {core => client}/consensus/aura/src/digest.rs (100%) rename {core => client}/consensus/aura/src/lib.rs (73%) rename {core => client}/consensus/babe/Cargo.toml (57%) rename {core => client}/consensus/babe/src/authorship.rs (100%) rename {core => client}/consensus/babe/src/aux_schema.rs (96%) rename {core => client}/consensus/babe/src/epoch_changes.rs (99%) rename {core => client}/consensus/babe/src/lib.rs (96%) rename {core => client}/consensus/babe/src/tests.rs (99%) rename {core => client}/consensus/babe/src/verification.rs (100%) create mode 100644 client/consensus/pow/Cargo.toml rename {core => client}/consensus/pow/src/lib.rs (94%) rename {core => client}/consensus/slots/Cargo.toml (59%) rename {core => client}/consensus/slots/build.rs (100%) rename {core => client}/consensus/slots/src/aux_schema.rs (98%) rename {core => client}/consensus/slots/src/lib.rs (97%) rename {core => client}/consensus/slots/src/slots.rs (99%) create mode 100644 client/consensus/uncles/Cargo.toml rename {core => client}/consensus/uncles/src/lib.rs (91%) rename {core/client => client}/db/Cargo.toml (67%) rename {core/client => client}/db/src/cache/list_cache.rs (99%) rename {core/client => client}/db/src/cache/list_entry.rs (99%) rename {core/client => client}/db/src/cache/list_storage.rs (99%) rename {core/client => client}/db/src/cache/mod.rs (98%) rename {core/client => client/db}/src/children.rs (99%) rename {core/client => client}/db/src/lib.rs (96%) rename {core/client => client}/db/src/light.rs (99%) rename {core/client => client}/db/src/offchain.rs (97%) create mode 100644 client/db/src/storage_cache.rs rename {core/client => client}/db/src/utils.rs (100%) create mode 100644 client/executor/Cargo.toml create mode 100644 client/executor/runtime-test/Cargo.toml rename {node/runtime => client/executor/runtime-test}/build.rs (96%) rename {core => client}/executor/runtime-test/src/lib.rs (75%) rename {core => client}/executor/src/allocator.rs (100%) create mode 100644 client/executor/src/deprecated_host_interface.rs rename {core => client}/executor/src/error.rs (81%) rename {core => client}/executor/src/host_interface.rs (89%) create mode 100644 client/executor/src/integration_tests/mod.rs create mode 100644 client/executor/src/integration_tests/sandbox.rs rename {core => client}/executor/src/lib.rs (92%) rename {core => client}/executor/src/native_executor.rs (88%) rename {core => client}/executor/src/sandbox.rs (74%) rename {core => client}/executor/src/wasm_runtime.rs (63%) rename {core => client}/executor/src/wasm_utils.rs (88%) rename {core => client}/executor/src/wasmi_execution.rs (51%) create mode 100644 client/executor/src/wasmtime/function_executor.rs rename {core/client/src/block_builder => client/executor/src/wasmtime}/mod.rs (75%) create mode 100644 client/executor/src/wasmtime/runtime.rs create mode 100644 client/executor/src/wasmtime/trampoline.rs create mode 100644 client/executor/src/wasmtime/util.rs rename {core => client}/finality-grandpa/Cargo.toml (57%) rename {core => client}/finality-grandpa/src/authorities.rs (99%) rename {core => client}/finality-grandpa/src/aux_schema.rs (97%) rename {core => client}/finality-grandpa/src/communication/gossip.rs (85%) rename {core => client}/finality-grandpa/src/communication/mod.rs (98%) create mode 100644 client/finality-grandpa/src/communication/periodic.rs rename {core => client}/finality-grandpa/src/communication/tests.rs (87%) rename {core => client}/finality-grandpa/src/consensus_changes.rs (95%) rename {core => client}/finality-grandpa/src/environment.rs (97%) rename {core => client}/finality-grandpa/src/finality_proof.rs (92%) rename {core => client}/finality-grandpa/src/import.rs (97%) rename {core => client}/finality-grandpa/src/justification.rs (98%) rename {core => client}/finality-grandpa/src/lib.rs (92%) rename {core => client}/finality-grandpa/src/light_import.rs (93%) rename {core => client}/finality-grandpa/src/observer.rs (98%) rename {core => client}/finality-grandpa/src/tests.rs (96%) rename {core => client}/finality-grandpa/src/until_imported.rs (99%) rename {core => client}/finality-grandpa/src/voting_rule.rs (99%) rename {core/client => client}/header-metadata/Cargo.toml (78%) rename {core/client => client}/header-metadata/src/lib.rs (100%) rename {core => client}/keystore/Cargo.toml (74%) rename {core => client}/keystore/src/lib.rs (100%) rename {core => client}/network/Cargo.toml (61%) rename {core => client}/network/src/behaviour.rs (97%) rename {core => client}/network/src/chain.rs (90%) rename {core => client}/network/src/config.rs (94%) rename {core => client}/network/src/debug_info.rs (96%) rename {core => client}/network/src/discovery.rs (92%) rename {core => client}/network/src/error.rs (97%) rename {core => client}/network/src/legacy_proto/behaviour.rs (97%) rename {core => client}/network/src/legacy_proto/handler.rs (91%) rename {core => client}/network/src/legacy_proto/mod.rs (100%) rename {core => client}/network/src/legacy_proto/tests.rs (88%) rename {core => client}/network/src/legacy_proto/upgrade.rs (82%) rename {core => client}/network/src/lib.rs (100%) rename {core => client}/network/src/on_demand_layer.rs (98%) rename {core => client}/network/src/protocol.rs (86%) rename {core => client}/network/src/protocol/consensus_gossip.rs (74%) rename {core => client}/network/src/protocol/event.rs (100%) rename {core => client}/network/src/protocol/light_dispatch.rs (98%) rename {core => client}/network/src/protocol/message.rs (89%) rename {core => client}/network/src/protocol/specialization.rs (97%) rename {core => client}/network/src/protocol/sync.rs (88%) rename {core => client}/network/src/protocol/sync/blocks.rs (86%) rename {core => client}/network/src/protocol/sync/extra_requests.rs (99%) rename {core => client}/network/src/protocol/util.rs (100%) rename {core => client}/network/src/service.rs (97%) rename {core => client}/network/src/test/block_import.rs (96%) rename {core => client}/network/src/test/mod.rs (95%) rename {core => client}/network/src/test/sync.rs (89%) rename {core => client}/network/src/transport.rs (89%) rename {core => client}/offchain/Cargo.toml (63%) rename {core => client}/offchain/src/api.rs (99%) rename {core => client}/offchain/src/api/http.rs (96%) rename {core => client}/offchain/src/api/http_dummy.rs (100%) rename {core => client}/offchain/src/api/timestamp.rs (100%) rename {core => client}/offchain/src/lib.rs (94%) rename {core => client}/offchain/src/testing.rs (96%) create mode 100644 client/prometheus/Cargo.toml create mode 100644 client/prometheus/README.md create mode 100644 client/prometheus/src/lib.rs create mode 100644 client/prometheus/src/metrics.rs create mode 100644 client/rpc-servers/Cargo.toml rename {core => client}/rpc-servers/src/lib.rs (100%) rename {core => client}/rpc/Cargo.toml (59%) rename {core => client}/rpc/api/Cargo.toml (65%) rename {core => client}/rpc/api/src/author/error.rs (100%) rename {core => client}/rpc/api/src/author/hash.rs (100%) rename {core => client}/rpc/api/src/author/mod.rs (100%) rename {core => client}/rpc/api/src/chain/error.rs (100%) rename {core => client}/rpc/api/src/chain/mod.rs (100%) rename {core => client}/rpc/api/src/errors.rs (100%) rename {core => client}/rpc/api/src/helpers.rs (100%) rename {core => client}/rpc/api/src/lib.rs (100%) rename {core => client}/rpc/api/src/state/error.rs (100%) rename {core => client}/rpc/api/src/state/mod.rs (100%) rename {core => client}/rpc/api/src/subscriptions.rs (100%) rename {core => client}/rpc/api/src/system/error.rs (100%) rename {core => client}/rpc/api/src/system/helpers.rs (100%) rename {core => client}/rpc/api/src/system/mod.rs (100%) rename {core => client}/rpc/src/author/mod.rs (93%) rename {core => client}/rpc/src/author/tests.rs (100%) rename {core => client}/rpc/src/chain/chain_full.rs (97%) rename {core => client}/rpc/src/chain/chain_light.rs (97%) rename {core => client}/rpc/src/chain/mod.rs (95%) rename {core => client}/rpc/src/chain/tests.rs (100%) rename {core => client}/rpc/src/lib.rs (100%) rename {core => client}/rpc/src/metadata.rs (100%) rename {core => client}/rpc/src/state/mod.rs (93%) rename {core => client}/rpc/src/state/state_full.rs (97%) rename {core => client}/rpc/src/state/state_light.rs (99%) rename {core => client}/rpc/src/state/tests.rs (99%) rename {core => client}/rpc/src/system/mod.rs (100%) rename {core => client}/rpc/src/system/tests.rs (100%) create mode 100644 client/service/Cargo.toml rename {core => client}/service/src/builder.rs (64%) rename {core => client}/service/src/chain_ops.rs (99%) rename {core => client}/service/src/config.rs (87%) rename {core => client}/service/src/error.rs (100%) rename {core => client}/service/src/lib.rs (96%) rename {core => client}/service/src/status_sinks.rs (100%) rename {core => client}/service/test/Cargo.toml (58%) rename {core => client}/service/test/src/lib.rs (98%) rename {core/client => client}/src/call_executor.rs (65%) rename {core/client => client}/src/cht.rs (98%) rename {core/client => client}/src/client.rs (90%) rename {core/client => client}/src/genesis.rs (100%) rename {core/client => client}/src/in_mem.rs (94%) rename {core/client => client}/src/leaves.rs (99%) rename {core/client => client}/src/lib.rs (76%) rename {core/client => client}/src/light/backend.rs (96%) rename {core/client => client}/src/light/blockchain.rs (78%) rename {core/client => client}/src/light/call_executor.rs (94%) rename {core/client => client}/src/light/fetcher.rs (74%) rename {core/client => client}/src/light/mod.rs (95%) rename {core => client}/state-db/Cargo.toml (78%) rename {core => client}/state-db/src/lib.rs (100%) rename {core => client}/state-db/src/noncanonical.rs (100%) rename {core => client}/state-db/src/pruning.rs (100%) rename {core => client}/state-db/src/test.rs (100%) rename {core => client}/telemetry/Cargo.toml (93%) rename {core => client}/telemetry/src/lib.rs (100%) rename {core => client}/telemetry/src/worker.rs (99%) rename {core => client}/telemetry/src/worker/node.rs (100%) create mode 100644 client/transaction-pool/Cargo.toml rename {core => client}/transaction-pool/graph/Cargo.toml (84%) rename {core => client}/transaction-pool/graph/benches/basics.rs (100%) rename {core => client}/transaction-pool/graph/src/base_pool.rs (100%) rename {core => client}/transaction-pool/graph/src/error.rs (100%) rename {core => client}/transaction-pool/graph/src/future.rs (100%) rename {core => client}/transaction-pool/graph/src/lib.rs (100%) rename {core => client}/transaction-pool/graph/src/listener.rs (86%) rename {core => client}/transaction-pool/graph/src/pool.rs (98%) rename {core => client}/transaction-pool/graph/src/ready.rs (100%) rename {core => client}/transaction-pool/graph/src/rotator.rs (100%) rename {core => client}/transaction-pool/graph/src/validated_pool.rs (96%) rename {core => client}/transaction-pool/graph/src/watcher.rs (100%) create mode 100644 client/transaction-pool/src/api.rs rename {core => client}/transaction-pool/src/error.rs (86%) rename {core => client}/transaction-pool/src/lib.rs (99%) rename {core => client}/transaction-pool/src/tests.rs (100%) rename {core => client}/utils/wasm-builder-runner/Cargo.toml (100%) rename {core => client}/utils/wasm-builder-runner/README.md (100%) rename {core => client}/utils/wasm-builder-runner/src/lib.rs (98%) delete mode 100644 core/authority-discovery/Cargo.toml delete mode 100644 core/authority-discovery/primitives/Cargo.toml delete mode 100644 core/basic-authorship/Cargo.toml delete mode 100644 core/chain-spec/Cargo.toml delete mode 100644 core/client/db/src/storage_cache.rs delete mode 100644 core/consensus/aura/primitives/Cargo.toml delete mode 100644 core/consensus/pow/Cargo.toml delete mode 100644 core/consensus/rhd/Cargo.toml delete mode 100644 core/consensus/rhd/src/error.rs delete mode 100644 core/consensus/rhd/src/lib.rs delete mode 100644 core/consensus/rhd/src/misbehaviour_check.rs delete mode 100644 core/consensus/rhd/src/service.rs delete mode 100644 core/consensus/uncles/Cargo.toml delete mode 100644 core/executor/Cargo.toml delete mode 100644 core/executor/runtime-test/Cargo.toml delete mode 100644 core/finality-grandpa/src/communication/periodic.rs create mode 100644 core/prometheus/Cargo.toml create mode 100644 core/prometheus/src/lib.rs create mode 100644 core/prometheus/src/metrics.rs delete mode 100644 core/rpc-servers/Cargo.toml delete mode 100644 core/service/Cargo.toml delete mode 100644 core/sr-api-macros/Cargo.toml delete mode 100644 core/sr-api-macros/tests/trybuild.rs delete mode 100644 core/sr-api-macros/tests/ui/adding_at_parameter.rs delete mode 100644 core/sr-api-macros/tests/ui/adding_at_parameter.stderr delete mode 100644 core/sr-api-macros/tests/ui/adding_self_parameter.stderr delete mode 100644 core/sr-api-macros/tests/ui/empty_impl_runtime_apis_call.stderr delete mode 100644 core/sr-api-macros/tests/ui/impl_incorrect_method_signature.stderr delete mode 100644 core/sr-api-macros/tests/ui/impl_two_traits_with_same_name.stderr delete mode 100644 core/sr-api-macros/tests/ui/invalid_api_version.stderr delete mode 100644 core/sr-api-macros/tests/ui/invalid_api_version_2.stderr delete mode 100644 core/sr-api-macros/tests/ui/invalid_api_version_3.stderr delete mode 100644 core/sr-api-macros/tests/ui/missing_block_generic_parameter.stderr delete mode 100644 core/sr-api-macros/tests/ui/type_reference_in_impl_runtime_apis_call.stderr delete mode 100644 core/sr-io/build.rs delete mode 100644 core/sr-io/src/lib.rs delete mode 100755 core/sr-sandbox/build.rs delete mode 100644 core/sr-std/build.rs delete mode 100644 core/test-client/Cargo.toml delete mode 100644 core/test-runtime/Cargo.toml delete mode 100644 core/test-runtime/client/Cargo.toml delete mode 100644 core/transaction-pool/Cargo.toml delete mode 100644 core/transaction-pool/src/api.rs rename CODEOWNERS => docs/CODEOWNERS (85%) rename CODE_OF_CONDUCT.adoc => docs/CODE_OF_CONDUCT.adoc (100%) rename CONTRIBUTING.adoc => docs/CONTRIBUTING.adoc (100%) rename PULL_REQUEST_TEMPLATE.md => docs/PULL_REQUEST_TEMPLATE.md (100%) create mode 100644 docs/README.adoc rename SECURITY.md => docs/SECURITY.md (100%) create mode 100644 docs/Structure.adoc rename license_header.txt => docs/license_header.txt (100%) delete mode 100644 node-template/Cargo.toml delete mode 100644 node-template/build.rs delete mode 100644 node-template/runtime/Cargo.toml delete mode 100644 node/cli/Cargo.toml delete mode 100644 node/executor/Cargo.toml delete mode 100644 node/rpc/Cargo.toml delete mode 100644 node/runtime/Cargo.toml delete mode 100644 node/testing/Cargo.toml rename {srml => paint}/assets/Cargo.toml (62%) rename {srml => paint}/assets/src/lib.rs (99%) create mode 100644 paint/aura/Cargo.toml rename {srml => paint}/aura/src/lib.rs (90%) rename {srml => paint}/aura/src/mock.rs (100%) rename {srml => paint}/aura/src/tests.rs (100%) create mode 100644 paint/authority-discovery/Cargo.toml create mode 100644 paint/authority-discovery/src/lib.rs rename {srml => paint}/authorship/Cargo.toml (50%) rename {srml => paint}/authorship/src/lib.rs (98%) create mode 100644 paint/babe/Cargo.toml rename {srml => paint}/babe/src/lib.rs (96%) rename {srml => paint}/babe/src/mock.rs (100%) rename {srml => paint}/babe/src/tests.rs (100%) rename {srml => paint}/balances/Cargo.toml (51%) rename {srml => paint}/balances/src/lib.rs (95%) rename {srml => paint}/balances/src/mock.rs (100%) rename {srml => paint}/balances/src/tests.rs (92%) rename {srml => paint}/collective/Cargo.toml (55%) rename {srml => paint}/collective/src/lib.rs (100%) rename {srml => paint}/contracts/COMPLEXITY.md (100%) rename {srml => paint}/contracts/Cargo.toml (53%) rename {srml => paint}/contracts/rpc/Cargo.toml (50%) rename {srml => paint}/contracts/rpc/runtime-api/Cargo.toml (54%) rename {srml => paint}/contracts/rpc/runtime-api/src/lib.rs (99%) rename {srml => paint}/contracts/rpc/src/lib.rs (99%) rename {srml => paint}/contracts/src/account_db.rs (99%) rename {srml => paint}/contracts/src/exec.rs (100%) rename {srml => paint}/contracts/src/gas.rs (100%) rename {srml => paint}/contracts/src/lib.rs (99%) rename {srml => paint}/contracts/src/rent.rs (97%) rename {srml => paint}/contracts/src/tests.rs (100%) rename {srml => paint}/contracts/src/wasm/code_cache.rs (100%) rename {srml => paint}/contracts/src/wasm/env_def/macros.rs (100%) rename {srml => paint}/contracts/src/wasm/env_def/mod.rs (100%) rename {srml => paint}/contracts/src/wasm/mod.rs (99%) rename {srml => paint}/contracts/src/wasm/prepare.rs (100%) rename {srml => paint}/contracts/src/wasm/runtime.rs (100%) rename {srml => paint}/democracy/Cargo.toml (56%) rename {srml => paint}/democracy/src/lib.rs (100%) rename {srml => paint}/democracy/src/vote_threshold.rs (100%) create mode 100644 paint/elections-phragmen/Cargo.toml rename {srml => paint}/elections-phragmen/src/lib.rs (99%) rename {srml => paint}/elections/Cargo.toml (55%) rename {srml => paint}/elections/src/lib.rs (100%) rename {srml => paint}/elections/src/mock.rs (100%) rename {srml => paint}/elections/src/tests.rs (100%) create mode 100644 paint/evm/Cargo.toml create mode 100644 paint/evm/src/backend.rs create mode 100644 paint/evm/src/lib.rs rename {srml => paint}/example/Cargo.toml (50%) rename {srml => paint}/example/src/lib.rs (97%) create mode 100644 paint/executive/Cargo.toml rename {srml => paint}/executive/src/lib.rs (87%) create mode 100644 paint/finality-tracker/Cargo.toml rename {srml => paint}/finality-tracker/src/lib.rs (95%) rename {srml => paint}/generic-asset/Cargo.toml (57%) rename {srml => paint}/generic-asset/src/lib.rs (98%) rename {srml => paint}/generic-asset/src/mock.rs (100%) rename {srml => paint}/generic-asset/src/tests.rs (100%) create mode 100644 paint/grandpa/Cargo.toml rename {srml => paint}/grandpa/src/lib.rs (90%) rename {srml => paint}/grandpa/src/mock.rs (94%) rename {srml => paint}/grandpa/src/tests.rs (95%) create mode 100644 paint/im-online/Cargo.toml rename {srml => paint}/im-online/src/lib.rs (72%) rename {srml => paint}/im-online/src/mock.rs (95%) rename {srml => paint}/im-online/src/tests.rs (66%) rename {srml => paint}/indices/Cargo.toml (56%) rename {srml => paint}/indices/src/address.rs (100%) rename {srml => paint}/indices/src/lib.rs (100%) rename {srml => paint}/indices/src/mock.rs (100%) rename {srml => paint}/indices/src/tests.rs (100%) rename {srml => paint}/membership/Cargo.toml (54%) rename {srml => paint}/membership/src/lib.rs (100%) rename {srml => paint}/metadata/Cargo.toml (63%) rename {srml => paint}/metadata/src/lib.rs (100%) rename {srml => paint}/nicks/Cargo.toml (50%) rename {srml => paint}/nicks/src/lib.rs (100%) create mode 100644 paint/offences/Cargo.toml rename {srml => paint}/offences/src/lib.rs (100%) rename {srml => paint}/offences/src/mock.rs (97%) rename {srml => paint}/offences/src/tests.rs (100%) rename {srml => paint}/randomness-collective-flip/Cargo.toml (55%) rename {srml => paint}/randomness-collective-flip/src/lib.rs (99%) rename {srml => paint}/scored-pool/Cargo.toml (51%) rename {srml => paint}/scored-pool/src/lib.rs (99%) rename {srml => paint}/scored-pool/src/mock.rs (100%) rename {srml => paint}/scored-pool/src/tests.rs (100%) rename {srml => paint}/session/Cargo.toml (53%) rename {srml => paint}/session/src/historical.rs (100%) rename {srml => paint}/session/src/lib.rs (98%) rename {srml => paint}/session/src/mock.rs (100%) create mode 100644 paint/staking/Cargo.toml rename {srml => paint}/staking/reward-curve/Cargo.toml (52%) rename {srml => paint}/staking/reward-curve/src/lib.rs (97%) rename {srml => paint}/staking/reward-curve/src/log.rs (100%) rename {srml => paint}/staking/reward-curve/tests/test.rs (89%) rename {srml => paint}/staking/src/inflation.rs (79%) rename {srml => paint}/staking/src/lib.rs (97%) rename {srml => paint}/staking/src/mock.rs (99%) rename {srml => paint}/staking/src/tests.rs (100%) rename {srml => paint}/sudo/Cargo.toml (55%) rename {srml => paint}/sudo/src/lib.rs (98%) create mode 100644 paint/support/Cargo.toml create mode 100644 paint/support/procedural/Cargo.toml rename {srml => paint}/support/procedural/src/lib.rs (86%) rename {srml => paint}/support/procedural/src/storage/genesis_config/builder_def.rs (98%) rename {srml => paint}/support/procedural/src/storage/genesis_config/genesis_config_def.rs (83%) rename {srml => paint}/support/procedural/src/storage/genesis_config/mod.rs (90%) rename {srml => paint}/support/procedural/src/storage/getters.rs (100%) rename {srml => paint}/support/procedural/src/storage/instance_trait.rs (100%) rename {srml => paint}/support/procedural/src/storage/metadata.rs (98%) rename {srml => paint}/support/procedural/src/storage/mod.rs (98%) rename {srml => paint}/support/procedural/src/storage/parse.rs (99%) rename {srml => paint}/support/procedural/src/storage/storage_struct.rs (97%) rename {srml => paint}/support/procedural/src/storage/store_trait.rs (100%) create mode 100644 paint/support/procedural/tools/Cargo.toml create mode 100644 paint/support/procedural/tools/derive/Cargo.toml rename {srml => paint}/support/procedural/tools/derive/src/lib.rs (100%) rename {srml => paint}/support/procedural/tools/src/lib.rs (94%) rename {srml => paint}/support/procedural/tools/src/syn_ext.rs (93%) create mode 100644 paint/support/rpc/Cargo.toml create mode 100644 paint/support/rpc/src/lib.rs rename {srml => paint}/support/src/debug.rs (96%) rename {srml => paint}/support/src/dispatch.rs (90%) rename {srml => paint}/support/src/error.rs (97%) rename {srml => paint}/support/src/event.rs (98%) rename {srml => paint}/support/src/hash.rs (94%) rename {srml => paint}/support/src/inherent.rs (100%) rename {srml => paint}/support/src/lib.rs (94%) rename {srml => paint}/support/src/metadata.rs (98%) rename {srml => paint}/support/src/origin.rs (100%) rename {srml => paint}/support/src/runtime.rs (100%) rename {srml => paint}/support/src/storage/child.rs (85%) rename {srml => paint}/support/src/storage/generator/double_map.rs (100%) rename {srml => paint}/support/src/storage/generator/linked_map.rs (61%) rename {srml => paint}/support/src/storage/generator/map.rs (100%) create mode 100644 paint/support/src/storage/generator/mod.rs rename {srml => paint}/support/src/storage/generator/value.rs (88%) rename {srml => paint}/support/src/storage/hashed.rs (100%) rename {srml => paint}/support/src/storage/mod.rs (79%) rename {srml => paint}/support/src/storage/unhashed.rs (91%) rename {srml => paint}/support/src/traits.rs (96%) rename {srml => paint}/support/src/unsigned.rs (81%) rename {srml => paint}/support/test/Cargo.toml (53%) rename {srml => paint}/support/test/src/lib.rs (90%) rename srml/support/src/storage/storage_items.rs => paint/support/test/tests/decl_storage.rs (67%) rename {srml => paint}/support/test/tests/decl_storage_ui.rs (100%) rename {srml => paint}/support/test/tests/decl_storage_ui/config_duplicate.rs (100%) rename {srml => paint}/support/test/tests/decl_storage_ui/config_duplicate.stderr (100%) rename {srml => paint}/support/test/tests/decl_storage_ui/config_get_duplicate.rs (100%) rename {srml => paint}/support/test/tests/decl_storage_ui/config_get_duplicate.stderr (100%) rename {srml => paint}/support/test/tests/decl_storage_ui/get_duplicate.rs (100%) rename {srml => paint}/support/test/tests/decl_storage_ui/get_duplicate.stderr (100%) rename {srml => paint}/support/test/tests/final_keys.rs (70%) rename {srml => paint}/support/test/tests/genesisconfig.rs (100%) rename {srml => paint}/support/test/tests/instance.rs (98%) rename {srml => paint}/support/test/tests/issue2219.rs (100%) rename {srml => paint}/support/test/tests/reserved_keyword.rs (100%) rename {srml => paint}/support/test/tests/reserved_keyword/on_initialize.rs (100%) rename {srml => paint}/support/test/tests/reserved_keyword/on_initialize.stderr (100%) rename {srml => paint}/support/test/tests/system.rs (100%) rename {srml => paint}/system/Cargo.toml (61%) rename {srml => paint}/system/benches/bench.rs (99%) rename {srml => paint}/system/rpc/Cargo.toml (52%) rename {srml => paint}/system/rpc/runtime-api/Cargo.toml (62%) rename {srml => paint}/system/rpc/runtime-api/src/lib.rs (97%) rename {srml => paint}/system/rpc/src/lib.rs (98%) rename {srml => paint}/system/src/lib.rs (96%) rename {srml => paint}/system/src/offchain.rs (98%) create mode 100644 paint/timestamp/Cargo.toml rename {srml => paint}/timestamp/src/lib.rs (94%) create mode 100644 paint/transaction-payment/Cargo.toml create mode 100644 paint/transaction-payment/rpc/Cargo.toml rename {srml => paint}/transaction-payment/rpc/runtime-api/Cargo.toml (53%) rename {srml => paint}/transaction-payment/rpc/runtime-api/src/lib.rs (98%) rename {srml => paint}/transaction-payment/rpc/src/lib.rs (94%) rename {srml => paint}/transaction-payment/src/lib.rs (100%) rename {srml => paint}/treasury/Cargo.toml (52%) rename {srml => paint}/treasury/src/lib.rs (87%) rename {srml => paint}/utility/Cargo.toml (50%) rename {srml => paint}/utility/src/lib.rs (100%) rename {core => primitives}/application-crypto/Cargo.toml (51%) rename {core => primitives}/application-crypto/src/ed25519.rs (83%) rename {core => primitives}/application-crypto/src/lib.rs (64%) rename {core => primitives}/application-crypto/src/sr25519.rs (83%) rename {core => primitives}/application-crypto/src/traits.rs (81%) create mode 100644 primitives/authority-discovery/Cargo.toml rename {core/authority-discovery/primitives => primitives/authority-discovery}/src/lib.rs (53%) create mode 100644 primitives/block-builder/runtime-api/Cargo.toml rename core/client/src/block_builder/api.rs => primitives/block-builder/runtime-api/src/lib.rs (76%) rename {core/finality-grandpa/primitives => primitives/consensus/aura}/Cargo.toml (66%) rename {core/consensus/aura/primitives => primitives/consensus/aura}/src/lib.rs (97%) rename {core/consensus/babe/primitives => primitives/consensus/babe}/Cargo.toml (60%) rename {core/consensus/babe/primitives => primitives/consensus/babe}/src/digest.rs (100%) rename {core/consensus/babe/primitives => primitives/consensus/babe}/src/lib.rs (98%) rename {core => primitives}/consensus/common/Cargo.toml (81%) rename {core => primitives}/consensus/common/src/block_import.rs (95%) rename {core => primitives}/consensus/common/src/block_validation.rs (100%) rename {core => primitives}/consensus/common/src/error.rs (98%) rename {core => primitives}/consensus/common/src/evaluation.rs (100%) rename {core => primitives}/consensus/common/src/import_queue.rs (93%) rename {core => primitives}/consensus/common/src/import_queue/basic_queue.rs (100%) rename {core => primitives}/consensus/common/src/import_queue/buffered_link.rs (100%) rename {core => primitives}/consensus/common/src/lib.rs (100%) rename {core => primitives}/consensus/common/src/offline_tracker.rs (100%) rename {core => primitives}/consensus/common/src/select_chain.rs (100%) rename {core/consensus/pow/primitives => primitives/consensus/pow}/Cargo.toml (53%) rename {core/consensus/pow/primitives => primitives/consensus/pow}/src/lib.rs (97%) rename {core/primitives => primitives/core}/Cargo.toml (62%) rename {core/primitives => primitives/core}/benches/bench.rs (100%) rename {core/primitives => primitives/core}/debug-derive/Cargo.toml (94%) rename {core/primitives => primitives/core}/debug-derive/src/impls.rs (100%) rename {core/primitives => primitives/core}/debug-derive/src/lib.rs (100%) rename {core/primitives => primitives/core}/debug-derive/tests/tests.rs (100%) rename {core/primitives => primitives/core}/src/changes_trie.rs (100%) rename {core/primitives => primitives/core}/src/crypto.rs (96%) rename {core/primitives => primitives/core}/src/ecdsa.rs (94%) rename {core/primitives => primitives/core}/src/ed25519.rs (92%) rename {core/primitives => primitives/core}/src/hash.rs (100%) rename {core/primitives => primitives/core}/src/hasher.rs (100%) rename {core/primitives => primitives/core}/src/hashing.rs (93%) rename {core/primitives => primitives/core}/src/hexdisplay.rs (100%) rename {core/primitives => primitives/core}/src/lib.rs (97%) rename {core/primitives => primitives/core}/src/offchain.rs (97%) rename {core/primitives => primitives/core}/src/sandbox.rs (100%) rename {core/primitives => primitives/core}/src/sr25519.rs (93%) rename {core/primitives => primitives/core}/src/testing.rs (100%) rename {core/primitives => primitives/core}/src/tests.rs (100%) rename {core/primitives => primitives/core}/src/traits.rs (100%) rename {core/primitives => primitives/core}/src/u32_trait.rs (100%) rename {core/primitives => primitives/core}/src/uint.rs (97%) rename {core/primitives => primitives/core}/storage/Cargo.toml (100%) rename {core/primitives => primitives/core}/storage/src/lib.rs (94%) rename {core => primitives}/externalities/Cargo.toml (76%) rename {core => primitives}/externalities/src/extensions.rs (87%) rename {core => primitives}/externalities/src/lib.rs (96%) rename {core => primitives}/externalities/src/scope_limited.rs (100%) create mode 100644 primitives/finality-grandpa/Cargo.toml rename {core/finality-grandpa/primitives => primitives/finality-grandpa}/src/lib.rs (73%) rename {core => primitives}/inherents/Cargo.toml (62%) rename {core => primitives}/inherents/src/lib.rs (94%) rename {core => primitives}/keyring/Cargo.toml (76%) rename {core => primitives}/keyring/src/ed25519.rs (100%) rename {core => primitives}/keyring/src/lib.rs (100%) rename {core => primitives}/keyring/src/sr25519.rs (100%) rename {core/offchain/primitives => primitives/offchain}/Cargo.toml (61%) rename {core/offchain/primitives => primitives/offchain}/src/lib.rs (88%) rename {core => primitives}/panic-handler/Cargo.toml (100%) rename {core => primitives}/panic-handler/src/lib.rs (100%) rename {core => primitives}/peerset/Cargo.toml (87%) rename {core => primitives}/peerset/src/lib.rs (100%) rename {core => primitives}/peerset/src/peersstate.rs (100%) rename {core => primitives}/peerset/tests/fuzz.rs (100%) rename {core => primitives}/phragmen/Cargo.toml (51%) rename {core => primitives}/phragmen/benches/phragmen.rs (100%) rename {core => primitives}/phragmen/src/lib.rs (86%) rename {core => primitives}/phragmen/src/mock.rs (92%) rename {core => primitives}/phragmen/src/tests.rs (89%) rename {core/rpc/primitives => primitives/rpc}/Cargo.toml (72%) rename {core/rpc/primitives => primitives/rpc}/src/lib.rs (100%) rename {core/rpc/primitives => primitives/rpc}/src/number.rs (100%) create mode 100644 primitives/runtime-interface/Cargo.toml create mode 100644 primitives/runtime-interface/proc-macro/Cargo.toml create mode 100644 primitives/runtime-interface/proc-macro/src/lib.rs create mode 100644 primitives/runtime-interface/proc-macro/src/pass_by/codec.rs create mode 100644 primitives/runtime-interface/proc-macro/src/pass_by/enum_.rs create mode 100644 primitives/runtime-interface/proc-macro/src/pass_by/inner.rs create mode 100644 primitives/runtime-interface/proc-macro/src/pass_by/mod.rs create mode 100644 primitives/runtime-interface/proc-macro/src/runtime_interface/bare_function_interface.rs create mode 100644 primitives/runtime-interface/proc-macro/src/runtime_interface/host_function_interface.rs create mode 100644 primitives/runtime-interface/proc-macro/src/runtime_interface/mod.rs create mode 100644 primitives/runtime-interface/proc-macro/src/runtime_interface/trait_decl_impl.rs create mode 100644 primitives/runtime-interface/proc-macro/src/utils.rs create mode 100644 primitives/runtime-interface/proc-macro/tests/ui.rs create mode 100644 primitives/runtime-interface/proc-macro/tests/ui/no_generic_parameters.rs create mode 100644 primitives/runtime-interface/proc-macro/tests/ui/no_generic_parameters.stderr create mode 100644 primitives/runtime-interface/proc-macro/tests/ui/no_method_implementation.rs create mode 100644 primitives/runtime-interface/proc-macro/tests/ui/no_method_implementation.stderr create mode 100644 primitives/runtime-interface/proc-macro/tests/ui/pass_by_enum_with_struct.rs create mode 100644 primitives/runtime-interface/proc-macro/tests/ui/pass_by_enum_with_struct.stderr create mode 100644 primitives/runtime-interface/proc-macro/tests/ui/pass_by_enum_with_value_variant.rs create mode 100644 primitives/runtime-interface/proc-macro/tests/ui/pass_by_enum_with_value_variant.stderr create mode 100644 primitives/runtime-interface/proc-macro/tests/ui/pass_by_inner_with_two_fields.rs create mode 100644 primitives/runtime-interface/proc-macro/tests/ui/pass_by_inner_with_two_fields.stderr create mode 100644 primitives/runtime-interface/proc-macro/tests/ui/take_self_by_value.rs create mode 100644 primitives/runtime-interface/proc-macro/tests/ui/take_self_by_value.stderr create mode 100644 primitives/runtime-interface/src/host.rs create mode 100644 primitives/runtime-interface/src/impls.rs create mode 100644 primitives/runtime-interface/src/lib.rs create mode 100644 primitives/runtime-interface/src/pass_by.rs create mode 100644 primitives/runtime-interface/src/wasm.rs create mode 100644 primitives/runtime-interface/test-wasm/Cargo.toml create mode 100644 primitives/runtime-interface/test-wasm/build.rs create mode 100644 primitives/runtime-interface/test-wasm/src/lib.rs rename {core => primitives}/serializer/Cargo.toml (100%) rename {core => primitives}/serializer/src/lib.rs (100%) rename {core => primitives}/session/Cargo.toml (55%) rename {core => primitives}/session/src/lib.rs (63%) create mode 100644 primitives/sr-api/Cargo.toml rename {core/sr-api-macros => primitives/sr-api}/benches/bench.rs (96%) create mode 100644 primitives/sr-api/proc-macro/Cargo.toml rename {core/sr-api-macros => primitives/sr-api/proc-macro}/src/decl_runtime_apis.rs (85%) rename {core/sr-api-macros => primitives/sr-api/proc-macro}/src/impl_runtime_apis.rs (81%) rename {core/sr-api-macros => primitives/sr-api/proc-macro}/src/lib.rs (91%) rename {core/sr-api-macros => primitives/sr-api/proc-macro}/src/utils.rs (72%) rename core/client/src/runtime_api.rs => primitives/sr-api/src/lib.rs (70%) create mode 100644 primitives/sr-api/test/Cargo.toml rename {core/sr-api-macros => primitives/sr-api/test}/tests/decl_and_impl.rs (70%) rename {core/sr-api-macros => primitives/sr-api/test}/tests/runtime_calls.rs (100%) create mode 100644 primitives/sr-api/test/tests/trybuild.rs rename {core/sr-api-macros => primitives/sr-api/test}/tests/ui/adding_self_parameter.rs (50%) create mode 100644 primitives/sr-api/test/tests/ui/adding_self_parameter.stderr rename {core/sr-api-macros => primitives/sr-api/test}/tests/ui/changed_in_unknown_version.rs (89%) rename {core/sr-api-macros => primitives/sr-api/test}/tests/ui/changed_in_unknown_version.stderr (53%) rename {core/sr-api-macros => primitives/sr-api/test}/tests/ui/declaring_old_block.rs (67%) rename {core/sr-api-macros => primitives/sr-api/test}/tests/ui/declaring_old_block.stderr (80%) rename {core/sr-api-macros => primitives/sr-api/test}/tests/ui/declaring_own_block_with_different_name.rs (66%) rename {core/sr-api-macros => primitives/sr-api/test}/tests/ui/declaring_own_block_with_different_name.stderr (84%) rename {core/sr-api-macros => primitives/sr-api/test}/tests/ui/empty_impl_runtime_apis_call.rs (80%) create mode 100644 primitives/sr-api/test/tests/ui/empty_impl_runtime_apis_call.stderr rename {core/sr-api-macros => primitives/sr-api/test}/tests/ui/impl_incorrect_method_signature.rs (81%) create mode 100644 primitives/sr-api/test/tests/ui/impl_incorrect_method_signature.stderr rename {core/sr-api-macros => primitives/sr-api/test}/tests/ui/impl_two_traits_with_same_name.rs (86%) create mode 100644 primitives/sr-api/test/tests/ui/impl_two_traits_with_same_name.stderr rename {core/sr-api-macros => primitives/sr-api/test}/tests/ui/invalid_api_version.rs (58%) create mode 100644 primitives/sr-api/test/tests/ui/invalid_api_version.stderr rename {core/sr-api-macros => primitives/sr-api/test}/tests/ui/invalid_api_version_2.rs (59%) create mode 100644 primitives/sr-api/test/tests/ui/invalid_api_version_2.stderr rename {core/sr-api-macros => primitives/sr-api/test}/tests/ui/invalid_api_version_3.rs (58%) create mode 100644 primitives/sr-api/test/tests/ui/invalid_api_version_3.stderr rename {core/sr-api-macros => primitives/sr-api/test}/tests/ui/missing_block_generic_parameter.rs (83%) create mode 100644 primitives/sr-api/test/tests/ui/missing_block_generic_parameter.stderr rename {core/sr-api-macros => primitives/sr-api/test}/tests/ui/missing_path_for_trait.rs (83%) rename {core/sr-api-macros => primitives/sr-api/test}/tests/ui/missing_path_for_trait.stderr (60%) rename {core/sr-api-macros => primitives/sr-api/test}/tests/ui/type_reference_in_impl_runtime_apis_call.rs (82%) create mode 100644 primitives/sr-api/test/tests/ui/type_reference_in_impl_runtime_apis_call.stderr rename {core => primitives}/sr-arithmetic/Cargo.toml (88%) rename {core => primitives}/sr-arithmetic/benches/bench.rs (100%) rename {core => primitives}/sr-arithmetic/fuzzer/.gitignore (100%) rename {core => primitives}/sr-arithmetic/fuzzer/Cargo.lock (81%) rename {core => primitives}/sr-arithmetic/fuzzer/Cargo.toml (100%) rename {core => primitives}/sr-arithmetic/fuzzer/src/biguint.rs (100%) rename {core => primitives}/sr-arithmetic/fuzzer/src/rational128.rs (100%) rename {core => primitives}/sr-arithmetic/src/biguint.rs (100%) rename {core => primitives}/sr-arithmetic/src/fixed64.rs (100%) rename {core => primitives}/sr-arithmetic/src/helpers_128bit.rs (100%) rename {core => primitives}/sr-arithmetic/src/lib.rs (100%) rename {core => primitives}/sr-arithmetic/src/per_things.rs (100%) rename {core => primitives}/sr-arithmetic/src/rational128.rs (100%) rename {core => primitives}/sr-arithmetic/src/traits.rs (100%) rename {core => primitives}/sr-io/Cargo.toml (57%) create mode 100644 primitives/sr-io/src/lib.rs rename {core => primitives}/sr-io/with_std.rs (99%) rename {core => primitives}/sr-io/without_std.rs (100%) rename {core => primitives}/sr-primitives/Cargo.toml (77%) rename {core => primitives}/sr-primitives/src/curve.rs (97%) rename {core => primitives}/sr-primitives/src/generic/block.rs (100%) rename {core => primitives}/sr-primitives/src/generic/checked_extrinsic.rs (85%) rename {core => primitives}/sr-primitives/src/generic/digest.rs (100%) rename {core => primitives}/sr-primitives/src/generic/era.rs (100%) rename {core => primitives}/sr-primitives/src/generic/header.rs (100%) rename {core => primitives}/sr-primitives/src/generic/mod.rs (100%) rename {core => primitives}/sr-primitives/src/generic/tests.rs (100%) rename {core => primitives}/sr-primitives/src/generic/unchecked_extrinsic.rs (99%) rename {core => primitives}/sr-primitives/src/lib.rs (95%) rename {core => primitives}/sr-primitives/src/offchain/http.rs (95%) rename {core => primitives}/sr-primitives/src/offchain/mod.rs (100%) rename {core => primitives}/sr-primitives/src/testing.rs (93%) rename {core => primitives}/sr-primitives/src/traits.rs (94%) rename {core => primitives}/sr-primitives/src/transaction_validity.rs (100%) rename {core => primitives}/sr-primitives/src/weights.rs (84%) rename {core => primitives}/sr-sandbox/Cargo.toml (65%) rename {core => primitives}/sr-sandbox/src/lib.rs (98%) rename {core => primitives}/sr-sandbox/with_std.rs (97%) rename {core => primitives}/sr-sandbox/without_std.rs (74%) rename {core => primitives}/sr-staking-primitives/Cargo.toml (100%) rename {core => primitives}/sr-staking-primitives/src/lib.rs (100%) rename {core => primitives}/sr-staking-primitives/src/offence.rs (100%) rename {core => primitives}/sr-std/Cargo.toml (56%) rename {core => primitives}/sr-std/src/lib.rs (86%) rename {core => primitives}/sr-std/with_std.rs (97%) rename {core => primitives}/sr-std/without_std.rs (73%) rename {core => primitives}/sr-version/Cargo.toml (100%) rename {core => primitives}/sr-version/src/lib.rs (99%) rename {core => primitives}/state-machine/Cargo.toml (90%) rename {core => primitives}/state-machine/src/backend.rs (94%) rename {core => primitives}/state-machine/src/basic.rs (89%) rename {core => primitives}/state-machine/src/changes_trie/build.rs (100%) rename {core => primitives}/state-machine/src/changes_trie/build_cache.rs (100%) rename {core => primitives}/state-machine/src/changes_trie/build_iterator.rs (100%) rename {core => primitives}/state-machine/src/changes_trie/changes_iterator.rs (99%) rename {core => primitives}/state-machine/src/changes_trie/input.rs (100%) rename {core => primitives}/state-machine/src/changes_trie/mod.rs (100%) rename {core => primitives}/state-machine/src/changes_trie/prune.rs (94%) rename {core => primitives}/state-machine/src/changes_trie/storage.rs (100%) rename {core => primitives}/state-machine/src/changes_trie/surface_iterator.rs (100%) rename {core => primitives}/state-machine/src/error.rs (100%) rename {core => primitives}/state-machine/src/ext.rs (100%) rename {core => primitives}/state-machine/src/lib.rs (98%) rename {core => primitives}/state-machine/src/overlayed_changes.rs (92%) rename {core => primitives}/state-machine/src/proving_backend.rs (66%) rename {core => primitives}/state-machine/src/testing.rs (98%) rename {core => primitives}/state-machine/src/trie_backend.rs (100%) rename {core => primitives}/state-machine/src/trie_backend_essence.rs (100%) create mode 100644 primitives/transaction-pool/runtime-api/Cargo.toml create mode 100644 primitives/transaction-pool/runtime-api/src/lib.rs rename {core => primitives}/trie/Cargo.toml (88%) rename {core => primitives}/trie/benches/bench.rs (100%) rename {core => primitives}/trie/src/error.rs (100%) rename {core => primitives}/trie/src/lib.rs (100%) rename {core => primitives}/trie/src/node_codec.rs (100%) rename {core => primitives}/trie/src/node_header.rs (100%) rename {core => primitives}/trie/src/trie_stream.rs (100%) rename {core => primitives}/wasm-interface/Cargo.toml (75%) rename {core => primitives}/wasm-interface/src/lib.rs (78%) rename {core => primitives}/wasm-interface/src/wasmi_impl.rs (100%) delete mode 100644 srml/README.adoc delete mode 100644 srml/aura/Cargo.toml delete mode 100644 srml/authority-discovery/Cargo.toml delete mode 100644 srml/authority-discovery/src/lib.rs delete mode 100644 srml/babe/Cargo.toml delete mode 100644 srml/elections-phragmen/Cargo.toml delete mode 100644 srml/executive/Cargo.toml delete mode 100644 srml/finality-tracker/Cargo.toml delete mode 100644 srml/grandpa/Cargo.toml delete mode 100644 srml/im-online/Cargo.toml delete mode 100644 srml/offences/Cargo.toml delete mode 100644 srml/staking/Cargo.toml delete mode 100644 srml/support/Cargo.toml delete mode 100644 srml/support/procedural/Cargo.toml delete mode 100644 srml/support/procedural/tools/Cargo.toml delete mode 100644 srml/support/procedural/tools/derive/Cargo.toml delete mode 100644 srml/support/src/storage/generator/mod.rs delete mode 100644 srml/timestamp/Cargo.toml delete mode 100644 srml/transaction-payment/Cargo.toml delete mode 100644 srml/transaction-payment/rpc/Cargo.toml delete mode 100644 test-utils/transaction-factory/Cargo.toml rename {test-utils => test/utils}/chain-spec-builder/Cargo.toml (65%) rename {test-utils => test/utils}/chain-spec-builder/build.rs (100%) rename {test-utils => test/utils}/chain-spec-builder/src/main.rs (97%) create mode 100644 test/utils/client/Cargo.toml rename {core/test-client => test/utils/client}/src/client_ext.rs (82%) rename {core/test-client => test/utils/client}/src/lib.rs (93%) create mode 100644 test/utils/primitives/Cargo.toml create mode 100644 test/utils/primitives/src/lib.rs create mode 100644 test/utils/runtime/Cargo.toml rename {core/test-runtime => test/utils/runtime}/build.rs (97%) create mode 100644 test/utils/runtime/client/Cargo.toml rename {core/test-runtime => test/utils/runtime}/client/src/block_builder_ext.rs (72%) rename {core/test-runtime => test/utils/runtime}/client/src/lib.rs (99%) rename {core/test-runtime => test/utils/runtime}/client/src/trait_tests.rs (97%) rename {core/test-runtime => test/utils/runtime}/src/genesismap.rs (89%) rename {core/test-runtime => test/utils/runtime}/src/lib.rs (94%) rename {core/test-runtime => test/utils/runtime}/src/system.rs (94%) create mode 100644 test/utils/transaction-factory/Cargo.toml rename {test-utils => test/utils}/transaction-factory/src/complex_mode.rs (96%) rename {test-utils => test/utils}/transaction-factory/src/lib.rs (93%) rename {test-utils => test/utils}/transaction-factory/src/modes.rs (100%) rename {test-utils => test/utils}/transaction-factory/src/simple_modes.rs (94%) create mode 100644 utils/build-script-utils/Cargo.toml create mode 100644 utils/build-script-utils/src/lib.rs rename {core/utils => utils}/fork-tree/Cargo.toml (100%) rename {core/utils => utils}/fork-tree/src/lib.rs (100%) rename {core/utils => utils}/wasm-builder/Cargo.toml (87%) rename {core/utils => utils}/wasm-builder/README.md (73%) rename {core/utils => utils}/wasm-builder/src/lib.rs (90%) rename {core/utils => utils}/wasm-builder/src/prerequisites.rs (100%) rename {core/utils => utils}/wasm-builder/src/wasm_project.rs (98%) diff --git a/.dockerignore b/.dockerignore index 834924b9f79d8..39dbc05c97e16 100644 --- a/.dockerignore +++ b/.dockerignore @@ -3,3 +3,4 @@ doc .idea/ Dockerfile .dockerignore +.local diff --git a/.editorconfig b/.editorconfig index f511aad460790..47fde53b690b5 100644 --- a/.editorconfig +++ b/.editorconfig @@ -6,7 +6,7 @@ tab_width=4 end_of_line=lf charset=utf-8 trim_trailing_whitespace=true -max_line_length=120 +max_line_length=100 insert_final_newline=true [*.yml] diff --git a/.gitignore b/.gitignore index 28885a4830d9b..6c79bfaf62ed1 100644 --- a/.gitignore +++ b/.gitignore @@ -21,3 +21,4 @@ rls*.log *.orig *.rej **/wip/*.stderr +.local diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index baca78fd3a4ce..f1aecb91fbaf1 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -51,7 +51,7 @@ variables: - /^[0-9]+$/ # PRs retry: max: 2 - when: + when: - runner_system_failure - unknown_failure - api_failure @@ -78,7 +78,7 @@ check-runtime: GITLAB_API: "https://gitlab.parity.io/api/v4" GITHUB_API_PROJECT: "parity%2Finfrastructure%2Fgithub-api" script: - - ./scripts/gitlab/check_runtime.sh + - ./.maintain/gitlab/check_runtime.sh allow_failure: true @@ -89,7 +89,7 @@ check-line-width: only: - /^[0-9]+$/ script: - - ./scripts/gitlab/check_line_width.sh + - ./.maintain/gitlab/check_line_width.sh allow_failure: true @@ -117,7 +117,7 @@ cargo-check-subkey: except: - /^v[0-9]+\.[0-9]+.*$/ # i.e. v1.0, v2.1rc1 script: - - cd ./subkey + - cd ./bin/subkey - BUILD_DUMMY_WASM_BINARY=1 time cargo check --release - sccache -s @@ -133,7 +133,7 @@ test-linux-stable: &test-linux variables: - $DEPLOY_TAG script: - - time cargo test --all --release --verbose --locked | + - WASM_BUILD_NO_COLOR=1 time cargo test --all --release --verbose --locked | tee output.log - sccache -s after_script: @@ -145,8 +145,17 @@ test-linux-stable: &test-linux paths: - ${CI_COMMIT_SHORT_SHA}_warnings.log +test-dependency-rules: &test-linux + stage: test + <<: *docker-env + allow_failure: true + except: + variables: + - $DEPLOY_TAG + script: + - .maintain/ensure-deps.sh -test-srml-staking: &test-srml-staking +test-paint-staking: &test-paint-staking stage: test <<: *docker-env variables: @@ -158,14 +167,25 @@ test-srml-staking: &test-srml-staking variables: - $DEPLOY_TAG script: - - cd srml/staking/ - - time cargo test --release --verbose --no-default-features --features std + - cd paint/staking/ + - WASM_BUILD_NO_COLOR=1 time cargo test --release --verbose --no-default-features --features std - sccache -s - - - - +test-wasmtime: &test-wasmtime + stage: test + <<: *docker-env + variables: + # Enable debug assertions since we are running optimized builds for testing + # but still want to have debug assertions. + RUSTFLAGS: -Cdebug-assertions=y + RUST_BACKTRACE: 1 + except: + variables: + - $DEPLOY_TAG + script: + - cd client/executor + - WASM_BUILD_NO_COLOR=1 time cargo test --release --verbose --features wasmtime + - sccache -s test-linux-stable-int: <<: *test-linux @@ -177,7 +197,7 @@ test-linux-stable-int: script: - echo "___Logs will be partly shown at the end in case of failure.___" - echo "___Full log will be saved to the job artifacts only in case of failure.___" - - RUST_LOG=sync=trace,consensus=trace,client=trace,state-db=trace,db=trace,forks=trace,state_db=trace,storage_cache=trace + - WASM_BUILD_NO_COLOR=1 RUST_LOG=sync=trace,consensus=trace,client=trace,state-db=trace,db=trace,forks=trace,state_db=trace,storage_cache=trace time cargo test -p node-cli --release --verbose --locked -- --ignored --test-threads=1 &> ${CI_COMMIT_SHORT_SHA}_int_failure.log - sccache -s @@ -202,25 +222,13 @@ check-web-wasm: - time cargo web build -p sr-io - time cargo web build -p sr-primitives - time cargo web build -p sr-std - - time cargo web build -p substrate-chain-spec - time cargo web build -p substrate-client - time cargo web build -p substrate-consensus-aura - time cargo web build -p substrate-consensus-babe - time cargo web build -p substrate-consensus-common - - time cargo web build -p substrate-keyring - - time cargo web build -p substrate-keystore - - time cargo web build -p substrate-executor - - time cargo web build -p substrate-network - - time cargo web build -p substrate-offchain - - time cargo web build -p substrate-panic-handler - - time cargo web build -p substrate-peerset - - time cargo web build -p substrate-primitives - - time cargo web build -p substrate-rpc-servers - - time cargo web build -p substrate-serializer - - time cargo web build -p substrate-state-db - - time cargo web build -p substrate-state-machine - time cargo web build -p substrate-telemetry - - time cargo web build -p substrate-trie + # Note: the command below is a bit weird because several Cargo issues prevent us from compiling the node in a more straight-forward way. + - time cargo build --manifest-path=bin/node/cli/Cargo.toml --no-default-features --features "browser" --target=wasm32-unknown-unknown - sccache -s node-exits: @@ -229,7 +237,27 @@ node-exits: except: - /^v[0-9]+\.[0-9]+.*$/ # i.e. v1.0, v2.1rc1 script: - - ./ci/check_for_exit.sh + - ./.maintain/check_for_exit.sh + + +test-full-crypto-feature: &test-full-crypto-feature + stage: test + <<: *docker-env + variables: + # Enable debug assertions since we are running optimized builds for testing + # but still want to have debug assertions. + RUSTFLAGS: -Cdebug-assertions=y + RUST_BACKTRACE: 1 + except: + variables: + - $DEPLOY_TAG + script: + - cd primitives/core/ + - time cargo +nightly build --verbose --no-default-features --features full_crypto + - cd ../application-crypto + - time cargo +nightly build --verbose --no-default-features --features full_crypto + - sccache -s + #### stage: build @@ -242,7 +270,7 @@ build-linux-substrate: variables: - $DEPLOY_TAG script: - - time cargo build --release --verbose + - WASM_BUILD_NO_COLOR=1 time cargo build --release --verbose - mkdir -p ./artifacts/substrate/ - mv ./target/release/substrate ./artifacts/substrate/. - echo -n "Substrate version = " @@ -255,8 +283,8 @@ build-linux-substrate: fi - sha256sum ./artifacts/substrate/substrate | tee ./artifacts/substrate/substrate.sha256 - printf '\n# building node-template\n\n' - - ./scripts/node-template-release.sh ./artifacts/substrate/substrate-node-template.tar.gz - - cp -r scripts/docker/substrate.Dockerfile ./artifacts/substrate/ + - ./.maintain/node-template-release.sh ./artifacts/substrate/substrate-node-template.tar.gz + - cp -r .maintain/docker/substrate.Dockerfile ./artifacts/substrate/ - sccache -s build-linux-subkey: @@ -270,7 +298,7 @@ build-linux-subkey: script: - cd ./subkey - BUILD_DUMMY_WASM_BINARY=1 time cargo build --release --verbose - - cd .. + - cd - - sccache -s - mkdir -p ./artifacts/subkey - mv ./target/release/subkey ./artifacts/subkey/. @@ -279,7 +307,7 @@ build-linux-subkey: sed -n -r 's/^subkey ([0-9.]+.*)/\1/p' | tee ./artifacts/subkey/VERSION; - sha256sum ./artifacts/subkey/subkey | tee ./artifacts/subkey/subkey.sha256 - - cp -r scripts/docker/subkey.Dockerfile ./artifacts/subkey/ + - cp -r .maintain/docker/subkey.Dockerfile ./artifacts/subkey/ - sccache -s build-rust-doc-release: @@ -295,7 +323,7 @@ build-rust-doc-release: <<: *build-only script: - rm -f ./crate-docs/index.html # use it as an indicator if the job succeeds - - BUILD_DUMMY_WASM_BINARY=1 RUSTDOCFLAGS="--html-in-header $(pwd)/rustdoc-header.html" time cargo +nightly doc --release --all --verbose + - BUILD_DUMMY_WASM_BINARY=1 RUSTDOCFLAGS="--html-in-header $(pwd).maintain/rustdoc-header.html" time cargo +nightly doc --release --all --verbose - cp -R ./target/doc ./crate-docs - echo "" > ./crate-docs/index.html - sccache -s @@ -319,6 +347,35 @@ check_warnings: fi allow_failure: true +# Check whether Polkadot 'master' branch builds using this Substrate commit. +check_polkadot: + stage: build + <<: *docker-env + allow_failure: true + dependencies: + - test-linux-stable + script: + - COMMIT_HASH=$(git rev-parse HEAD) + - SUBSTRATE_PATH=$(pwd) + # Clone the current Polkadot master branch into ./polkadot. + - git clone --depth 1 https://gitlab.parity.io/parity/polkadot.git + - cd polkadot + # Within Polkadot 'master' alter each Cargo.toml that references the + # Substrate 'polkadot-master' branch: + # 1. Replace the 'branch = "polkadot-master"' statements with the rev of our + # commit. + # 2. Replace 'git = "https://.*"' with 'git = "file://.*"' (the local + # checked out Substrate repository one folder above). + # 3. Remove any trailing commas. + - git grep -l "polkadot-master" | grep toml | xargs sed -i "s/branch.*=.*\"polkadot-master\"/rev = \"$COMMIT_HASH\"/; s~https://github.com/paritytech/substrate~file://$SUBSTRATE_PATH~; s/,\s*}/ }/" + # Make sure 'Cargo.lock' matches 'Cargo.toml'. It's enough to update one + # package, others are updated along the way. + - cargo update -p sr-io + # Check whether Polkadot 'master' branch builds with this Substrate commit. + - time cargo check + - cd - + - sccache -s + #### stage: publish .publish-docker-release: &publish-docker-release @@ -485,10 +542,10 @@ publish-gh-doc: - echo "Substrate version = ${DEPLOY_TAG}" # or use helm to render the template - helm template - --values ./scripts/kubernetes/values.yaml + --values ./.maintain/kubernetes/values.yaml --set image.tag=${DEPLOY_TAG} --set validator.keys=${VALIDATOR_KEYS} - ./scripts/kubernetes | kubectl apply -f - --dry-run=false + ./.maintain/kubernetes | kubectl apply -f - --dry-run=false - echo "# substrate namespace ${KUBE_NAMESPACE}" - kubectl -n ${KUBE_NAMESPACE} get all - echo "# substrate's nodes' external ip addresses:" @@ -550,16 +607,16 @@ deploy-ue1-tag: validator 1 4: <<: *validator-deploy script: - - ./scripts/flamingfir-deploy.sh flamingfir-validator1 + - ./.maintain/flamingfir-deploy.sh flamingfir-validator1 validator 2 4: <<: *validator-deploy script: - - ./scripts/flamingfir-deploy.sh flamingfir-validator2 + - ./.maintain/flamingfir-deploy.sh flamingfir-validator2 validator 3 4: <<: *validator-deploy script: - - ./scripts/flamingfir-deploy.sh flamingfir-validator3 + - ./.maintain/flamingfir-deploy.sh flamingfir-validator3 validator 4 4: <<: *validator-deploy script: - - ./scripts/flamingfir-deploy.sh flamingfir-validator4 + - ./.maintain/flamingfir-deploy.sh flamingfir-validator4 diff --git a/Dockerfile b/.maintain/Dockerfile similarity index 100% rename from Dockerfile rename to .maintain/Dockerfile diff --git a/scripts/build-only-wasm.sh b/.maintain/build-only-wasm.sh similarity index 100% rename from scripts/build-only-wasm.sh rename to .maintain/build-only-wasm.sh diff --git a/ci/check_for_exit.sh b/.maintain/check_for_exit.sh similarity index 100% rename from ci/check_for_exit.sh rename to .maintain/check_for_exit.sh diff --git a/scripts/common.sh b/.maintain/common.sh similarity index 100% rename from scripts/common.sh rename to .maintain/common.sh diff --git a/scripts/docker/subkey.Dockerfile b/.maintain/docker/subkey.Dockerfile similarity index 93% rename from scripts/docker/subkey.Dockerfile rename to .maintain/docker/subkey.Dockerfile index 6d5c559ea03dd..9184cad5b4058 100644 --- a/scripts/docker/subkey.Dockerfile +++ b/.maintain/docker/subkey.Dockerfile @@ -8,7 +8,7 @@ LABEL io.parity.image.authors="devops-team@parity.io" \ io.parity.image.vendor="Parity Technologies" \ io.parity.image.title="parity/subkey" \ io.parity.image.description="Subkey: key generating utility for Substrate." \ - io.parity.image.source="https://github.com/paritytech/substrate/blob/${VCS_REF}/scripts/docker/subkey.Dockerfile" \ + io.parity.image.source="https://github.com/paritytech/substrate/blob/${VCS_REF}/.maintain/docker/subkey.Dockerfile" \ io.parity.image.revision="${VCS_REF}" \ io.parity.image.created="${BUILD_DATE}" \ io.parity.image.documentation="https://github.com/paritytech/substrate/tree/${VCS_REF}/subkey" diff --git a/scripts/docker/substrate.Dockerfile b/.maintain/docker/substrate.Dockerfile similarity index 96% rename from scripts/docker/substrate.Dockerfile rename to .maintain/docker/substrate.Dockerfile index 2bdf49e2fcec2..7cd4576a9e89f 100644 --- a/scripts/docker/substrate.Dockerfile +++ b/.maintain/docker/substrate.Dockerfile @@ -8,7 +8,7 @@ LABEL io.parity.image.authors="devops-team@parity.io" \ io.parity.image.vendor="Parity Technologies" \ io.parity.image.title="parity/substrate" \ io.parity.image.description="Substrate: The platform for blockchain innovators." \ - io.parity.image.source="https://github.com/paritytech/substrate/blob/${VCS_REF}/scripts/docker/Dockerfile" \ + io.parity.image.source="https://github.com/paritytech/substrate/blob/${VCS_REF}/.maintain/docker/Dockerfile" \ io.parity.image.revision="${VCS_REF}" \ io.parity.image.created="${BUILD_DATE}" \ io.parity.image.documentation="https://wiki.parity.io/Parity-Substrate" diff --git a/.maintain/ensure-deps.sh b/.maintain/ensure-deps.sh new file mode 100755 index 0000000000000..b4477dab5392c --- /dev/null +++ b/.maintain/ensure-deps.sh @@ -0,0 +1,61 @@ +#!/usr/bin/env bash + +# The script is meant to check if the rules regarding packages +# dependencies are satisfied. +# The general format is: +# [top-lvl-dir]<[crate-name-prefix] + +# For instance no crate within `./client` directory +# is allowed to import any crate with a directory path containing `paint`. +# Such rule is just: `client /dev/null +done + +# Display violations and fail +I=0 +for v in "${VIOLATIONS[@]}" +do + cat << EOF + +=========================================== +======= Violation of rule: $v +=========================================== +${PACKAGES[$I]} + + +EOF + I=$I+1 +done + +exit ${#VIOLATIONS[@]} diff --git a/scripts/flamingfir-deploy.sh b/.maintain/flamingfir-deploy.sh similarity index 100% rename from scripts/flamingfir-deploy.sh rename to .maintain/flamingfir-deploy.sh diff --git a/scripts/getgoing.sh b/.maintain/getgoing.sh similarity index 100% rename from scripts/getgoing.sh rename to .maintain/getgoing.sh diff --git a/scripts/gitlab/check_line_width.sh b/.maintain/gitlab/check_line_width.sh similarity index 100% rename from scripts/gitlab/check_line_width.sh rename to .maintain/gitlab/check_line_width.sh diff --git a/scripts/gitlab/check_runtime.sh b/.maintain/gitlab/check_runtime.sh similarity index 93% rename from scripts/gitlab/check_runtime.sh rename to .maintain/gitlab/check_runtime.sh index cd988718d05db..2b3a93e34bdbc 100755 --- a/scripts/gitlab/check_runtime.sh +++ b/.maintain/gitlab/check_runtime.sh @@ -1,7 +1,7 @@ #!/bin/sh # # -# check for any changes in the node/src/runtime, srml/ and core/sr_* trees. if +# check for any changes in the node/src/runtime, paint/ and core/sr_* trees. if # there are any changes found, it should mark the PR breaksconsensus and # "auto-fail" the PR if there isn't a change in the runtime/src/lib.rs file # that alters the version. @@ -32,7 +32,7 @@ github_label () { # check if the wasm sources changed if ! git diff --name-only origin/master...${CI_COMMIT_SHA} \ - | grep -q -e '^node/src/runtime' -e '^srml/' -e '^core/sr-' | grep -v -e '^core/sr-arithmetic/fuzzer' + | grep -q -e '^node/src/runtime' -e '^paint/' -e '^core/sr-' | grep -v -e '^core/sr-arithmetic/fuzzer' then cat <<-EOT @@ -102,7 +102,7 @@ else source file directories: - node/src/runtime - - srml + - paint - core/sr-* versions file: ${VERSIONS_FILE} diff --git a/node-template/scripts/init.sh b/.maintain/init.sh similarity index 100% rename from node-template/scripts/init.sh rename to .maintain/init.sh diff --git a/scripts/kubernetes/Chart.yaml b/.maintain/kubernetes/Chart.yaml similarity index 100% rename from scripts/kubernetes/Chart.yaml rename to .maintain/kubernetes/Chart.yaml diff --git a/scripts/kubernetes/README.md b/.maintain/kubernetes/README.md similarity index 100% rename from scripts/kubernetes/README.md rename to .maintain/kubernetes/README.md diff --git a/scripts/kubernetes/templates/poddisruptionbudget.yaml b/.maintain/kubernetes/templates/poddisruptionbudget.yaml similarity index 100% rename from scripts/kubernetes/templates/poddisruptionbudget.yaml rename to .maintain/kubernetes/templates/poddisruptionbudget.yaml diff --git a/scripts/kubernetes/templates/secrets.yaml b/.maintain/kubernetes/templates/secrets.yaml similarity index 100% rename from scripts/kubernetes/templates/secrets.yaml rename to .maintain/kubernetes/templates/secrets.yaml diff --git a/scripts/kubernetes/templates/service.yaml b/.maintain/kubernetes/templates/service.yaml similarity index 100% rename from scripts/kubernetes/templates/service.yaml rename to .maintain/kubernetes/templates/service.yaml diff --git a/scripts/kubernetes/templates/serviceaccount.yaml b/.maintain/kubernetes/templates/serviceaccount.yaml similarity index 100% rename from scripts/kubernetes/templates/serviceaccount.yaml rename to .maintain/kubernetes/templates/serviceaccount.yaml diff --git a/scripts/kubernetes/templates/statefulset.yaml b/.maintain/kubernetes/templates/statefulset.yaml similarity index 100% rename from scripts/kubernetes/templates/statefulset.yaml rename to .maintain/kubernetes/templates/statefulset.yaml diff --git a/scripts/kubernetes/values.yaml b/.maintain/kubernetes/values.yaml similarity index 100% rename from scripts/kubernetes/values.yaml rename to .maintain/kubernetes/values.yaml diff --git a/scripts/node-template-release.sh b/.maintain/node-template-release.sh similarity index 84% rename from scripts/node-template-release.sh rename to .maintain/node-template-release.sh index 3b2c0d6f0f6b8..897677edc5c39 100755 --- a/scripts/node-template-release.sh +++ b/.maintain/node-template-release.sh @@ -11,6 +11,6 @@ if [ "$#" -ne 1 ]; then fi PATH_TO_ARCHIVE=$(pwd)/$1 -cd $PROJECT_ROOT/scripts/node-template-release +cd $PROJECT_ROOT/.maintain/node-template-release cargo run $PROJECT_ROOT/node-template $PATH_TO_ARCHIVE diff --git a/.maintain/node-template-release/Cargo.lock b/.maintain/node-template-release/Cargo.lock new file mode 100644 index 0000000000000..01bbe21dfdd85 --- /dev/null +++ b/.maintain/node-template-release/Cargo.lock @@ -0,0 +1,581 @@ +# This file is automatically @generated by Cargo. +# It is not intended for manual editing. +[[package]] +name = "adler32" +version = "1.0.4" +source = "registry+https://github.com/rust-lang/crates.io-index" + +[[package]] +name = "ansi_term" +version = "0.11.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "winapi 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "atty" +version = "0.2.13" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", + "winapi 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "autocfg" +version = "0.1.7" +source = "registry+https://github.com/rust-lang/crates.io-index" + +[[package]] +name = "bitflags" +version = "1.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" + +[[package]] +name = "c2-chacha" +version = "0.2.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "ppv-lite86 0.2.6 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "cc" +version = "1.0.47" +source = "registry+https://github.com/rust-lang/crates.io-index" + +[[package]] +name = "cfg-if" +version = "0.1.10" +source = "registry+https://github.com/rust-lang/crates.io-index" + +[[package]] +name = "clap" +version = "2.33.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "ansi_term 0.11.0 (registry+https://github.com/rust-lang/crates.io-index)", + "atty 0.2.13 (registry+https://github.com/rust-lang/crates.io-index)", + "bitflags 1.2.1 (registry+https://github.com/rust-lang/crates.io-index)", + "strsim 0.8.0 (registry+https://github.com/rust-lang/crates.io-index)", + "textwrap 0.11.0 (registry+https://github.com/rust-lang/crates.io-index)", + "unicode-width 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)", + "vec_map 0.8.1 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "crc32fast" +version = "1.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "cfg-if 0.1.10 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "curl-sys" +version = "0.4.24" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "cc 1.0.47 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", + "libz-sys 1.0.25 (registry+https://github.com/rust-lang/crates.io-index)", + "openssl-sys 0.9.53 (registry+https://github.com/rust-lang/crates.io-index)", + "pkg-config 0.3.17 (registry+https://github.com/rust-lang/crates.io-index)", + "vcpkg 0.2.7 (registry+https://github.com/rust-lang/crates.io-index)", + "winapi 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "filetime" +version = "0.2.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "cfg-if 0.1.10 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", + "redox_syscall 0.1.56 (registry+https://github.com/rust-lang/crates.io-index)", + "winapi 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "flate2" +version = "1.0.13" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "cfg-if 0.1.10 (registry+https://github.com/rust-lang/crates.io-index)", + "crc32fast 1.2.0 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", + "miniz_oxide 0.3.5 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "fs_extra" +version = "1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" + +[[package]] +name = "getrandom" +version = "0.1.13" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "cfg-if 0.1.10 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", + "wasi 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "git2" +version = "0.8.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "bitflags 1.2.1 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", + "libgit2-sys 0.7.11 (registry+https://github.com/rust-lang/crates.io-index)", + "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", + "openssl-probe 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)", + "openssl-sys 0.9.53 (registry+https://github.com/rust-lang/crates.io-index)", + "url 1.7.2 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "glob" +version = "0.2.11" +source = "registry+https://github.com/rust-lang/crates.io-index" + +[[package]] +name = "heck" +version = "0.3.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "unicode-segmentation 1.6.0 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "idna" +version = "0.1.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "matches 0.1.8 (registry+https://github.com/rust-lang/crates.io-index)", + "unicode-bidi 0.3.4 (registry+https://github.com/rust-lang/crates.io-index)", + "unicode-normalization 0.1.11 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "libc" +version = "0.2.65" +source = "registry+https://github.com/rust-lang/crates.io-index" + +[[package]] +name = "libgit2-sys" +version = "0.7.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "cc 1.0.47 (registry+https://github.com/rust-lang/crates.io-index)", + "curl-sys 0.4.24 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", + "libssh2-sys 0.2.13 (registry+https://github.com/rust-lang/crates.io-index)", + "libz-sys 1.0.25 (registry+https://github.com/rust-lang/crates.io-index)", + "openssl-sys 0.9.53 (registry+https://github.com/rust-lang/crates.io-index)", + "pkg-config 0.3.17 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "libssh2-sys" +version = "0.2.13" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "cc 1.0.47 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", + "libz-sys 1.0.25 (registry+https://github.com/rust-lang/crates.io-index)", + "openssl-sys 0.9.53 (registry+https://github.com/rust-lang/crates.io-index)", + "pkg-config 0.3.17 (registry+https://github.com/rust-lang/crates.io-index)", + "vcpkg 0.2.7 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "libz-sys" +version = "1.0.25" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "cc 1.0.47 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", + "pkg-config 0.3.17 (registry+https://github.com/rust-lang/crates.io-index)", + "vcpkg 0.2.7 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "log" +version = "0.4.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "cfg-if 0.1.10 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "matches" +version = "0.1.8" +source = "registry+https://github.com/rust-lang/crates.io-index" + +[[package]] +name = "miniz_oxide" +version = "0.3.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "adler32 1.0.4 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "node-template-release" +version = "2.0.0" +dependencies = [ + "flate2 1.0.13 (registry+https://github.com/rust-lang/crates.io-index)", + "fs_extra 1.1.0 (registry+https://github.com/rust-lang/crates.io-index)", + "git2 0.8.0 (registry+https://github.com/rust-lang/crates.io-index)", + "glob 0.2.11 (registry+https://github.com/rust-lang/crates.io-index)", + "structopt 0.3.5 (registry+https://github.com/rust-lang/crates.io-index)", + "tar 0.4.26 (registry+https://github.com/rust-lang/crates.io-index)", + "tempfile 3.1.0 (registry+https://github.com/rust-lang/crates.io-index)", + "toml 0.4.10 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "openssl-probe" +version = "0.1.2" +source = "registry+https://github.com/rust-lang/crates.io-index" + +[[package]] +name = "openssl-sys" +version = "0.9.53" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "autocfg 0.1.7 (registry+https://github.com/rust-lang/crates.io-index)", + "cc 1.0.47 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", + "pkg-config 0.3.17 (registry+https://github.com/rust-lang/crates.io-index)", + "vcpkg 0.2.7 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "percent-encoding" +version = "1.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" + +[[package]] +name = "pkg-config" +version = "0.3.17" +source = "registry+https://github.com/rust-lang/crates.io-index" + +[[package]] +name = "ppv-lite86" +version = "0.2.6" +source = "registry+https://github.com/rust-lang/crates.io-index" + +[[package]] +name = "proc-macro-error" +version = "0.2.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "proc-macro2 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "quote 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)", + "syn 1.0.8 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "proc-macro2" +version = "1.0.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "unicode-xid 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "quote" +version = "1.0.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "proc-macro2 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "rand" +version = "0.7.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "getrandom 0.1.13 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", + "rand_chacha 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)", + "rand_core 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)", + "rand_hc 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "rand_chacha" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "c2-chacha 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)", + "rand_core 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "rand_core" +version = "0.5.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "getrandom 0.1.13 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "rand_hc" +version = "0.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "rand_core 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "redox_syscall" +version = "0.1.56" +source = "registry+https://github.com/rust-lang/crates.io-index" + +[[package]] +name = "remove_dir_all" +version = "0.5.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "winapi 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "serde" +version = "1.0.103" +source = "registry+https://github.com/rust-lang/crates.io-index" + +[[package]] +name = "smallvec" +version = "1.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" + +[[package]] +name = "strsim" +version = "0.8.0" +source = "registry+https://github.com/rust-lang/crates.io-index" + +[[package]] +name = "structopt" +version = "0.3.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "clap 2.33.0 (registry+https://github.com/rust-lang/crates.io-index)", + "structopt-derive 0.3.5 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "structopt-derive" +version = "0.3.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "heck 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", + "proc-macro-error 0.2.6 (registry+https://github.com/rust-lang/crates.io-index)", + "proc-macro2 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "quote 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)", + "syn 1.0.8 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "syn" +version = "1.0.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "proc-macro2 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "quote 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)", + "unicode-xid 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "tar" +version = "0.4.26" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "filetime 0.2.8 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", + "redox_syscall 0.1.56 (registry+https://github.com/rust-lang/crates.io-index)", + "xattr 0.2.2 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "tempfile" +version = "3.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "cfg-if 0.1.10 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", + "rand 0.7.2 (registry+https://github.com/rust-lang/crates.io-index)", + "redox_syscall 0.1.56 (registry+https://github.com/rust-lang/crates.io-index)", + "remove_dir_all 0.5.2 (registry+https://github.com/rust-lang/crates.io-index)", + "winapi 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "textwrap" +version = "0.11.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "unicode-width 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "toml" +version = "0.4.10" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "serde 1.0.103 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "unicode-bidi" +version = "0.3.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "matches 0.1.8 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "unicode-normalization" +version = "0.1.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "smallvec 1.0.0 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "unicode-segmentation" +version = "1.6.0" +source = "registry+https://github.com/rust-lang/crates.io-index" + +[[package]] +name = "unicode-width" +version = "0.1.6" +source = "registry+https://github.com/rust-lang/crates.io-index" + +[[package]] +name = "unicode-xid" +version = "0.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" + +[[package]] +name = "url" +version = "1.7.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "idna 0.1.5 (registry+https://github.com/rust-lang/crates.io-index)", + "matches 0.1.8 (registry+https://github.com/rust-lang/crates.io-index)", + "percent-encoding 1.0.1 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "vcpkg" +version = "0.2.7" +source = "registry+https://github.com/rust-lang/crates.io-index" + +[[package]] +name = "vec_map" +version = "0.8.1" +source = "registry+https://github.com/rust-lang/crates.io-index" + +[[package]] +name = "wasi" +version = "0.7.0" +source = "registry+https://github.com/rust-lang/crates.io-index" + +[[package]] +name = "winapi" +version = "0.3.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "winapi-i686-pc-windows-gnu 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)", + "winapi-x86_64-pc-windows-gnu 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "winapi-i686-pc-windows-gnu" +version = "0.4.0" +source = "registry+https://github.com/rust-lang/crates.io-index" + +[[package]] +name = "winapi-x86_64-pc-windows-gnu" +version = "0.4.0" +source = "registry+https://github.com/rust-lang/crates.io-index" + +[[package]] +name = "xattr" +version = "0.2.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[metadata] +"checksum adler32 1.0.4 (registry+https://github.com/rust-lang/crates.io-index)" = "5d2e7343e7fc9de883d1b0341e0b13970f764c14101234857d2ddafa1cb1cac2" +"checksum ansi_term 0.11.0 (registry+https://github.com/rust-lang/crates.io-index)" = "ee49baf6cb617b853aa8d93bf420db2383fab46d314482ca2803b40d5fde979b" +"checksum atty 0.2.13 (registry+https://github.com/rust-lang/crates.io-index)" = "1803c647a3ec87095e7ae7acfca019e98de5ec9a7d01343f611cf3152ed71a90" +"checksum autocfg 0.1.7 (registry+https://github.com/rust-lang/crates.io-index)" = "1d49d90015b3c36167a20fe2810c5cd875ad504b39cff3d4eae7977e6b7c1cb2" +"checksum bitflags 1.2.1 (registry+https://github.com/rust-lang/crates.io-index)" = "cf1de2fe8c75bc145a2f577add951f8134889b4795d47466a54a5c846d691693" +"checksum c2-chacha 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)" = "214238caa1bf3a496ec3392968969cab8549f96ff30652c9e56885329315f6bb" +"checksum cc 1.0.47 (registry+https://github.com/rust-lang/crates.io-index)" = "aa87058dce70a3ff5621797f1506cb837edd02ac4c0ae642b4542dce802908b8" +"checksum cfg-if 0.1.10 (registry+https://github.com/rust-lang/crates.io-index)" = "4785bdd1c96b2a846b2bd7cc02e86b6b3dbf14e7e53446c4f54c92a361040822" +"checksum clap 2.33.0 (registry+https://github.com/rust-lang/crates.io-index)" = "5067f5bb2d80ef5d68b4c87db81601f0b75bca627bc2ef76b141d7b846a3c6d9" +"checksum crc32fast 1.2.0 (registry+https://github.com/rust-lang/crates.io-index)" = "ba125de2af0df55319f41944744ad91c71113bf74a4646efff39afe1f6842db1" +"checksum curl-sys 0.4.24 (registry+https://github.com/rust-lang/crates.io-index)" = "f659f3ffac9582d6177bb86d1d2aa649f4eb9d0d4de9d03ccc08b402832ea340" +"checksum filetime 0.2.8 (registry+https://github.com/rust-lang/crates.io-index)" = "1ff6d4dab0aa0c8e6346d46052e93b13a16cf847b54ed357087c35011048cc7d" +"checksum flate2 1.0.13 (registry+https://github.com/rust-lang/crates.io-index)" = "6bd6d6f4752952feb71363cffc9ebac9411b75b87c6ab6058c40c8900cf43c0f" +"checksum fs_extra 1.1.0 (registry+https://github.com/rust-lang/crates.io-index)" = "5f2a4a2034423744d2cc7ca2068453168dcdb82c438419e639a26bd87839c674" +"checksum getrandom 0.1.13 (registry+https://github.com/rust-lang/crates.io-index)" = "e7db7ca94ed4cd01190ceee0d8a8052f08a247aa1b469a7f68c6a3b71afcf407" +"checksum git2 0.8.0 (registry+https://github.com/rust-lang/crates.io-index)" = "c7339329bfa14a00223244311560d11f8f489b453fb90092af97f267a6090ab0" +"checksum glob 0.2.11 (registry+https://github.com/rust-lang/crates.io-index)" = "8be18de09a56b60ed0edf84bc9df007e30040691af7acd1c41874faac5895bfb" +"checksum heck 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)" = "20564e78d53d2bb135c343b3f47714a56af2061f1c928fdb541dc7b9fdd94205" +"checksum idna 0.1.5 (registry+https://github.com/rust-lang/crates.io-index)" = "38f09e0f0b1fb55fdee1f17470ad800da77af5186a1a76c026b679358b7e844e" +"checksum libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)" = "1a31a0627fdf1f6a39ec0dd577e101440b7db22672c0901fe00a9a6fbb5c24e8" +"checksum libgit2-sys 0.7.11 (registry+https://github.com/rust-lang/crates.io-index)" = "48441cb35dc255da8ae72825689a95368bf510659ae1ad55dc4aa88cb1789bf1" +"checksum libssh2-sys 0.2.13 (registry+https://github.com/rust-lang/crates.io-index)" = "5fcd5a428a31cbbfe059812d74f4b6cd3b9b7426c2bdaec56993c5365da1c328" +"checksum libz-sys 1.0.25 (registry+https://github.com/rust-lang/crates.io-index)" = "2eb5e43362e38e2bca2fd5f5134c4d4564a23a5c28e9b95411652021a8675ebe" +"checksum log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)" = "14b6052be84e6b71ab17edffc2eeabf5c2c3ae1fdb464aae35ac50c67a44e1f7" +"checksum matches 0.1.8 (registry+https://github.com/rust-lang/crates.io-index)" = "7ffc5c5338469d4d3ea17d269fa8ea3512ad247247c30bd2df69e68309ed0a08" +"checksum miniz_oxide 0.3.5 (registry+https://github.com/rust-lang/crates.io-index)" = "6f3f74f726ae935c3f514300cc6773a0c9492abc5e972d42ba0c0ebb88757625" +"checksum openssl-probe 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)" = "77af24da69f9d9341038eba93a073b1fdaaa1b788221b00a69bce9e762cb32de" +"checksum openssl-sys 0.9.53 (registry+https://github.com/rust-lang/crates.io-index)" = "465d16ae7fc0e313318f7de5cecf57b2fbe7511fd213978b457e1c96ff46736f" +"checksum percent-encoding 1.0.1 (registry+https://github.com/rust-lang/crates.io-index)" = "31010dd2e1ac33d5b46a5b413495239882813e0369f8ed8a5e266f173602f831" +"checksum pkg-config 0.3.17 (registry+https://github.com/rust-lang/crates.io-index)" = "05da548ad6865900e60eaba7f589cc0783590a92e940c26953ff81ddbab2d677" +"checksum ppv-lite86 0.2.6 (registry+https://github.com/rust-lang/crates.io-index)" = "74490b50b9fbe561ac330df47c08f3f33073d2d00c150f719147d7c54522fa1b" +"checksum proc-macro-error 0.2.6 (registry+https://github.com/rust-lang/crates.io-index)" = "aeccfe4d5d8ea175d5f0e4a2ad0637e0f4121d63bd99d356fb1f39ab2e7c6097" +"checksum proc-macro2 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)" = "9c9e470a8dc4aeae2dee2f335e8f533e2d4b347e1434e5671afc49b054592f27" +"checksum quote 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)" = "053a8c8bcc71fcce321828dc897a98ab9760bef03a4fc36693c231e5b3216cfe" +"checksum rand 0.7.2 (registry+https://github.com/rust-lang/crates.io-index)" = "3ae1b169243eaf61759b8475a998f0a385e42042370f3a7dbaf35246eacc8412" +"checksum rand_chacha 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)" = "03a2a90da8c7523f554344f921aa97283eadf6ac484a6d2a7d0212fa7f8d6853" +"checksum rand_core 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)" = "90bde5296fc891b0cef12a6d03ddccc162ce7b2aff54160af9338f8d40df6d19" +"checksum rand_hc 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)" = "ca3129af7b92a17112d59ad498c6f81eaf463253766b90396d39ea7a39d6613c" +"checksum redox_syscall 0.1.56 (registry+https://github.com/rust-lang/crates.io-index)" = "2439c63f3f6139d1b57529d16bc3b8bb855230c8efcc5d3a896c8bea7c3b1e84" +"checksum remove_dir_all 0.5.2 (registry+https://github.com/rust-lang/crates.io-index)" = "4a83fa3702a688b9359eccba92d153ac33fd2e8462f9e0e3fdf155239ea7792e" +"checksum serde 1.0.103 (registry+https://github.com/rust-lang/crates.io-index)" = "1217f97ab8e8904b57dd22eb61cde455fa7446a9c1cf43966066da047c1f3702" +"checksum smallvec 1.0.0 (registry+https://github.com/rust-lang/crates.io-index)" = "4ecf3b85f68e8abaa7555aa5abdb1153079387e60b718283d732f03897fcfc86" +"checksum strsim 0.8.0 (registry+https://github.com/rust-lang/crates.io-index)" = "8ea5119cdb4c55b55d432abb513a0429384878c15dde60cc77b1c99de1a95a6a" +"checksum structopt 0.3.5 (registry+https://github.com/rust-lang/crates.io-index)" = "30b3a3e93f5ad553c38b3301c8a0a0cec829a36783f6a0c467fc4bf553a5f5bf" +"checksum structopt-derive 0.3.5 (registry+https://github.com/rust-lang/crates.io-index)" = "ea692d40005b3ceba90a9fe7a78fa8d4b82b0ce627eebbffc329aab850f3410e" +"checksum syn 1.0.8 (registry+https://github.com/rust-lang/crates.io-index)" = "661641ea2aa15845cddeb97dad000d22070bb5c1fb456b96c1cba883ec691e92" +"checksum tar 0.4.26 (registry+https://github.com/rust-lang/crates.io-index)" = "b3196bfbffbba3e57481b6ea32249fbaf590396a52505a2615adbb79d9d826d3" +"checksum tempfile 3.1.0 (registry+https://github.com/rust-lang/crates.io-index)" = "7a6e24d9338a0a5be79593e2fa15a648add6138caa803e2d5bc782c371732ca9" +"checksum textwrap 0.11.0 (registry+https://github.com/rust-lang/crates.io-index)" = "d326610f408c7a4eb6f51c37c330e496b08506c9457c9d34287ecc38809fb060" +"checksum toml 0.4.10 (registry+https://github.com/rust-lang/crates.io-index)" = "758664fc71a3a69038656bee8b6be6477d2a6c315a6b81f7081f591bffa4111f" +"checksum unicode-bidi 0.3.4 (registry+https://github.com/rust-lang/crates.io-index)" = "49f2bd0c6468a8230e1db229cff8029217cf623c767ea5d60bfbd42729ea54d5" +"checksum unicode-normalization 0.1.11 (registry+https://github.com/rust-lang/crates.io-index)" = "b561e267b2326bb4cebfc0ef9e68355c7abe6c6f522aeac2f5bf95d56c59bdcf" +"checksum unicode-segmentation 1.6.0 (registry+https://github.com/rust-lang/crates.io-index)" = "e83e153d1053cbb5a118eeff7fd5be06ed99153f00dbcd8ae310c5fb2b22edc0" +"checksum unicode-width 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)" = "7007dbd421b92cc6e28410fe7362e2e0a2503394908f417b68ec8d1c364c4e20" +"checksum unicode-xid 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)" = "826e7639553986605ec5979c7dd957c7895e93eabed50ab2ffa7f6128a75097c" +"checksum url 1.7.2 (registry+https://github.com/rust-lang/crates.io-index)" = "dd4e7c0d531266369519a4aa4f399d748bd37043b00bde1e4ff1f60a120b355a" +"checksum vcpkg 0.2.7 (registry+https://github.com/rust-lang/crates.io-index)" = "33dd455d0f96e90a75803cfeb7f948768c08d70a6de9a8d2362461935698bf95" +"checksum vec_map 0.8.1 (registry+https://github.com/rust-lang/crates.io-index)" = "05c78687fb1a80548ae3250346c3db86a80a7cdd77bda190189f2d0a0987c81a" +"checksum wasi 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)" = "b89c3ce4ce14bdc6fb6beaf9ec7928ca331de5df7e5ea278375642a2f478570d" +"checksum winapi 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)" = "8093091eeb260906a183e6ae1abdba2ef5ef2257a21801128899c3fc699229c6" +"checksum winapi-i686-pc-windows-gnu 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)" = "ac3b87c63620426dd9b991e5ce0329eff545bccbbb34f3be09ff6fb6ab51b7b6" +"checksum winapi-x86_64-pc-windows-gnu 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)" = "712e227841d057c1ee1cd2fb22fa7e5a5461ae8e48fa2ca79ec42cfc1931183f" +"checksum xattr 0.2.2 (registry+https://github.com/rust-lang/crates.io-index)" = "244c3741f4240ef46274860397c7c74e50eb23624996930e484c16679633a54c" diff --git a/scripts/node-template-release/Cargo.toml b/.maintain/node-template-release/Cargo.toml similarity index 100% rename from scripts/node-template-release/Cargo.toml rename to .maintain/node-template-release/Cargo.toml diff --git a/scripts/node-template-release/src/main.rs b/.maintain/node-template-release/src/main.rs similarity index 86% rename from scripts/node-template-release/src/main.rs rename to .maintain/node-template-release/src/main.rs index e1db5af649748..7fde9c8684720 100644 --- a/scripts/node-template-release/src/main.rs +++ b/.maintain/node-template-release/src/main.rs @@ -88,33 +88,35 @@ fn replace_path_dependencies_with_git(cargo_toml_path: &Path, commit_id: &str, c // remove `Cargo.toml` cargo_toml_path.pop(); - let mut dependencies: toml::value::Table = match cargo_toml - .remove("dependencies") - .and_then(|v| v.try_into().ok()) { - Some(deps) => deps, - None => return, - }; - - let deps_rewritten = dependencies - .iter() - .filter_map(|(k, v)| v.clone().try_into::().ok().map(move |v| (k, v))) - .filter(|t| t.1.contains_key("path")) - .filter(|t| { - // if the path does not exists, we need to add this as git dependency - t.1.get("path").unwrap().as_str().map(|path| !cargo_toml_path.join(path).exists()).unwrap_or(false) - }) - .map(|(k, mut v)| { - // remove `path` and add `git` and `rev` - v.remove("path"); - v.insert("git".into(), SUBSTRATE_GIT_URL.into()); - v.insert("rev".into(), commit_id.into()); - - (k.clone(), v.into()) - }).collect::>(); - - dependencies.extend(deps_rewritten.into_iter()); - - cargo_toml.insert("dependencies".into(), dependencies.into()); + for table in &["dependencies", "build-dependencies"] { + let mut dependencies: toml::value::Table = match cargo_toml + .remove(table) + .and_then(|v| v.try_into().ok()) { + Some(deps) => deps, + None => continue, + }; + + let deps_rewritten = dependencies + .iter() + .filter_map(|(k, v)| v.clone().try_into::().ok().map(move |v| (k, v))) + .filter(|t| t.1.contains_key("path")) + .filter(|t| { + // if the path does not exists, we need to add this as git dependency + t.1.get("path").unwrap().as_str().map(|path| !cargo_toml_path.join(path).exists()).unwrap_or(false) + }) + .map(|(k, mut v)| { + // remove `path` and add `git` and `rev` + v.remove("path"); + v.insert("git".into(), SUBSTRATE_GIT_URL.into()); + v.insert("rev".into(), commit_id.into()); + + (k.clone(), v.into()) + }).collect::>(); + + dependencies.extend(deps_rewritten.into_iter()); + + cargo_toml.insert(table.into(), dependencies.into()); + } } /// Update the top level (workspace) `Cargo.toml` file. diff --git a/scripts/runtime-dep.py b/.maintain/runtime-dep.py similarity index 91% rename from scripts/runtime-dep.py rename to .maintain/runtime-dep.py index 1d0c42b2f57e8..3198bb3e2669e 100755 --- a/scripts/runtime-dep.py +++ b/.maintain/runtime-dep.py @@ -4,7 +4,7 @@ # pip install toml # sudo apt-get install graphviz # the first parameter is the runtime folder -# python ./scripts/runtime-dep.py ./substrate/runtime | dot -Tpng -o output.png +# python ./.maintain/runtime-dep.py ./substrate/runtime | dot -Tpng -o output.png import sys import os import toml diff --git a/rustdoc-header.html b/.maintain/rustdoc-header.html similarity index 100% rename from rustdoc-header.html rename to .maintain/rustdoc-header.html diff --git a/scripts/sentry-node/docker-compose.yml b/.maintain/sentry-node/docker-compose.yml similarity index 97% rename from scripts/sentry-node/docker-compose.yml rename to .maintain/sentry-node/docker-compose.yml index 78f8ba36b82ee..dd2aee6995473 100644 --- a/scripts/sentry-node/docker-compose.yml +++ b/.maintain/sentry-node/docker-compose.yml @@ -12,7 +12,7 @@ # # 1. Build `target/release/substrate` binary: `cargo build --release` # -# 2. Start networks and containers: `sudo docker-compose -f scripts/sentry-node/docker-compose.yml up` +# 2. Start networks and containers: `sudo docker-compose -f .maintain/sentry-node/docker-compose.yml up` # # 3. Reach: # - polkadot/apps on localhost:3000 diff --git a/scripts/update-copyright.sh b/.maintain/update-copyright.sh similarity index 100% rename from scripts/update-copyright.sh rename to .maintain/update-copyright.sh diff --git a/scripts/update-deps.sh b/.maintain/update-deps.sh similarity index 100% rename from scripts/update-deps.sh rename to .maintain/update-deps.sh diff --git a/scripts/update.sh b/.maintain/update.sh similarity index 100% rename from scripts/update.sh rename to .maintain/update.sh diff --git a/.travis.yml b/.travis.yml deleted file mode 100644 index 4180defedfd6a..0000000000000 --- a/.travis.yml +++ /dev/null @@ -1,28 +0,0 @@ -# Request an environment that provides sudo (that goes with larger containers) -# and a minimal language environment. -sudo: true -language: minimal - -cache: cargo - -branches: - only: - - master - -env: - global: - - RUST_BACKTRACE=1 - matrix: - - RUST_TOOLCHAIN=nightly TARGET=wasm - - RUST_TOOLCHAIN=stable TARGET=native - -before_install: - # Check how much space we've got on this machine. - - df -h - -script: - - ./ci/script.sh - -after_script: - # Check how much free disk space left after the build - - df -h diff --git a/Cargo.lock b/Cargo.lock index 75a1b797b07c3..72988d6496c00 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -1,5 +1,14 @@ # This file is automatically @generated by Cargo. # It is not intended for manual editing. +[[package]] +name = "Inflector" +version = "0.11.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "lazy_static 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)", + "regex 1.3.1 (registry+https://github.com/rust-lang/crates.io-index)", +] + [[package]] name = "adler32" version = "1.0.4" @@ -38,7 +47,7 @@ dependencies = [ [[package]] name = "ahash" -version = "0.2.16" +version = "0.2.17" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "const-random 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)", @@ -52,19 +61,6 @@ dependencies = [ "memchr 2.2.1 (registry+https://github.com/rust-lang/crates.io-index)", ] -[[package]] -name = "aio-limited" -version = "0.1.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -dependencies = [ - "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", - "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", - "parking_lot 0.9.0 (registry+https://github.com/rust-lang/crates.io-index)", - "tokio-executor 0.1.8 (registry+https://github.com/rust-lang/crates.io-index)", - "tokio-io 0.1.12 (registry+https://github.com/rust-lang/crates.io-index)", - "tokio-timer 0.2.11 (registry+https://github.com/rust-lang/crates.io-index)", -] - [[package]] name = "ansi_term" version = "0.11.0" @@ -81,6 +77,11 @@ dependencies = [ "winapi 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)", ] +[[package]] +name = "anyhow" +version = "1.0.19" +source = "registry+https://github.com/rust-lang/crates.io-index" + [[package]] name = "app_dirs" version = "1.2.1" @@ -94,7 +95,7 @@ dependencies = [ [[package]] name = "arc-swap" -version = "0.3.11" +version = "0.4.3" source = "registry+https://github.com/rust-lang/crates.io-index" [[package]] @@ -110,6 +111,11 @@ dependencies = [ "nodrop 0.1.14 (registry+https://github.com/rust-lang/crates.io-index)", ] +[[package]] +name = "arrayvec" +version = "0.5.1" +source = "registry+https://github.com/rust-lang/crates.io-index" + [[package]] name = "asn1_der" version = "0.6.3" @@ -124,7 +130,7 @@ version = "0.1.2" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "quote 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)", - "syn 1.0.7 (registry+https://github.com/rust-lang/crates.io-index)", + "syn 1.0.8 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] @@ -137,33 +143,33 @@ name = "atty" version = "0.2.13" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "libc 0.2.62 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", "winapi 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] name = "autocfg" -version = "0.1.6" +version = "0.1.7" source = "registry+https://github.com/rust-lang/crates.io-index" [[package]] name = "backtrace" -version = "0.3.38" +version = "0.3.40" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "backtrace-sys 0.1.31 (registry+https://github.com/rust-lang/crates.io-index)", + "backtrace-sys 0.1.32 (registry+https://github.com/rust-lang/crates.io-index)", "cfg-if 0.1.10 (registry+https://github.com/rust-lang/crates.io-index)", - "libc 0.2.62 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", "rustc-demangle 0.1.16 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] name = "backtrace-sys" -version = "0.1.31" +version = "0.1.32" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "cc 1.0.45 (registry+https://github.com/rust-lang/crates.io-index)", - "libc 0.2.62 (registry+https://github.com/rust-lang/crates.io-index)", + "cc 1.0.47 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] @@ -177,7 +183,7 @@ version = "0.9.3" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "byteorder 1.3.2 (registry+https://github.com/rust-lang/crates.io-index)", - "safemem 0.3.2 (registry+https://github.com/rust-lang/crates.io-index)", + "safemem 0.3.3 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] @@ -188,13 +194,23 @@ dependencies = [ "byteorder 1.3.2 (registry+https://github.com/rust-lang/crates.io-index)", ] +[[package]] +name = "bincode" +version = "1.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "autocfg 0.1.7 (registry+https://github.com/rust-lang/crates.io-index)", + "byteorder 1.3.2 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", +] + [[package]] name = "bindgen" version = "0.47.3" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "bitflags 1.2.0 (registry+https://github.com/rust-lang/crates.io-index)", - "cexpr 0.3.5 (registry+https://github.com/rust-lang/crates.io-index)", + "bitflags 1.2.1 (registry+https://github.com/rust-lang/crates.io-index)", + "cexpr 0.3.6 (registry+https://github.com/rust-lang/crates.io-index)", "cfg-if 0.1.10 (registry+https://github.com/rust-lang/crates.io-index)", "clang-sys 0.26.4 (registry+https://github.com/rust-lang/crates.io-index)", "clap 2.33.0 (registry+https://github.com/rust-lang/crates.io-index)", @@ -211,7 +227,7 @@ dependencies = [ [[package]] name = "bitflags" -version = "1.2.0" +version = "1.2.1" source = "registry+https://github.com/rust-lang/crates.io-index" [[package]] @@ -245,12 +261,13 @@ dependencies = [ ] [[package]] -name = "block-buffer" -version = "0.2.0" +name = "blake2b_simd" +version = "0.5.9" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "byte-tools 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)", - "generic-array 0.8.3 (registry+https://github.com/rust-lang/crates.io-index)", + "arrayref 0.3.5 (registry+https://github.com/rust-lang/crates.io-index)", + "arrayvec 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)", + "constant_time_eq 0.1.4 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] @@ -258,7 +275,7 @@ name = "block-buffer" version = "0.7.3" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "block-padding 0.1.4 (registry+https://github.com/rust-lang/crates.io-index)", + "block-padding 0.1.5 (registry+https://github.com/rust-lang/crates.io-index)", "byte-tools 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", "byteorder 1.3.2 (registry+https://github.com/rust-lang/crates.io-index)", "generic-array 0.12.3 (registry+https://github.com/rust-lang/crates.io-index)", @@ -274,7 +291,7 @@ dependencies = [ [[package]] name = "block-padding" -version = "0.1.4" +version = "0.1.5" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "byte-tools 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", @@ -282,7 +299,7 @@ dependencies = [ [[package]] name = "bs58" -version = "0.2.5" +version = "0.3.0" source = "registry+https://github.com/rust-lang/crates.io-index" [[package]] @@ -293,7 +310,7 @@ dependencies = [ "lazy_static 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)", "memchr 2.2.1 (registry+https://github.com/rust-lang/crates.io-index)", "regex-automata 0.1.8 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] @@ -311,12 +328,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" [[package]] name = "byte-slice-cast" -version = "0.3.2" -source = "registry+https://github.com/rust-lang/crates.io-index" - -[[package]] -name = "byte-tools" -version = "0.2.0" +version = "0.3.4" source = "registry+https://github.com/rust-lang/crates.io-index" [[package]] @@ -346,11 +358,10 @@ dependencies = [ [[package]] name = "c2-chacha" -version = "0.2.2" +version = "0.2.3" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "lazy_static 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)", - "ppv-lite86 0.2.5 (registry+https://github.com/rust-lang/crates.io-index)", + "ppv-lite86 0.2.6 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] @@ -360,12 +371,12 @@ source = "registry+https://github.com/rust-lang/crates.io-index" [[package]] name = "cargo_metadata" -version = "0.8.2" +version = "0.9.0" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "semver 0.9.0 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", - "serde_derive 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "serde_derive 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", "serde_json 1.0.41 (registry+https://github.com/rust-lang/crates.io-index)", ] @@ -376,16 +387,16 @@ source = "registry+https://github.com/rust-lang/crates.io-index" [[package]] name = "cc" -version = "1.0.45" +version = "1.0.47" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "jobserver 0.1.17 (registry+https://github.com/rust-lang/crates.io-index)", - "num_cpus 1.10.1 (registry+https://github.com/rust-lang/crates.io-index)", + "num_cpus 1.11.0 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] name = "cexpr" -version = "0.3.5" +version = "0.3.6" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "nom 4.2.3 (registry+https://github.com/rust-lang/crates.io-index)", @@ -403,7 +414,7 @@ dependencies = [ "ansi_term 0.12.1 (registry+https://github.com/rust-lang/crates.io-index)", "node-cli 2.0.0", "rand 0.7.2 (registry+https://github.com/rust-lang/crates.io-index)", - "structopt 0.3.3 (registry+https://github.com/rust-lang/crates.io-index)", + "structopt 0.3.4 (registry+https://github.com/rust-lang/crates.io-index)", "substrate-keystore 2.0.0", "substrate-primitives 2.0.0", ] @@ -413,7 +424,7 @@ name = "chrono" version = "0.4.9" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "libc 0.2.62 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", "num-integer 0.1.41 (registry+https://github.com/rust-lang/crates.io-index)", "num-traits 0.2.8 (registry+https://github.com/rust-lang/crates.io-index)", "time 0.1.42 (registry+https://github.com/rust-lang/crates.io-index)", @@ -425,7 +436,7 @@ version = "0.26.4" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "glob 0.2.11 (registry+https://github.com/rust-lang/crates.io-index)", - "libc 0.2.62 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", "libloading 0.5.2 (registry+https://github.com/rust-lang/crates.io-index)", ] @@ -436,7 +447,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "ansi_term 0.11.0 (registry+https://github.com/rust-lang/crates.io-index)", "atty 0.2.13 (registry+https://github.com/rust-lang/crates.io-index)", - "bitflags 1.2.0 (registry+https://github.com/rust-lang/crates.io-index)", + "bitflags 1.2.1 (registry+https://github.com/rust-lang/crates.io-index)", "strsim 0.8.0 (registry+https://github.com/rust-lang/crates.io-index)", "textwrap 0.11.0 (registry+https://github.com/rust-lang/crates.io-index)", "unicode-width 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)", @@ -449,7 +460,7 @@ name = "clear_on_drop" version = "0.2.3" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "cc 1.0.45 (registry+https://github.com/rust-lang/crates.io-index)", + "cc 1.0.47 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] @@ -457,7 +468,7 @@ name = "cloudabi" version = "0.0.3" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "bitflags 1.2.0 (registry+https://github.com/rust-lang/crates.io-index)", + "bitflags 1.2.1 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] @@ -465,7 +476,25 @@ name = "cmake" version = "0.1.42" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "cc 1.0.45 (registry+https://github.com/rust-lang/crates.io-index)", + "cc 1.0.47 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "console_error_panic_hook" +version = "0.1.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "cfg-if 0.1.10 (registry+https://github.com/rust-lang/crates.io-index)", + "wasm-bindgen 0.2.54 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "console_log" +version = "0.1.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", + "web-sys 0.3.31 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] @@ -474,7 +503,7 @@ version = "0.1.6" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "const-random-macro 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)", - "proc-macro-hack 0.5.10 (registry+https://github.com/rust-lang/crates.io-index)", + "proc-macro-hack 0.5.11 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] @@ -482,7 +511,7 @@ name = "const-random-macro" version = "0.1.6" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "proc-macro-hack 0.5.10 (registry+https://github.com/rust-lang/crates.io-index)", + "proc-macro-hack 0.5.11 (registry+https://github.com/rust-lang/crates.io-index)", "rand 0.7.2 (registry+https://github.com/rust-lang/crates.io-index)", ] @@ -491,13 +520,44 @@ name = "constant_time_eq" version = "0.1.4" source = "registry+https://github.com/rust-lang/crates.io-index" +[[package]] +name = "cookie" +version = "0.12.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "time 0.1.42 (registry+https://github.com/rust-lang/crates.io-index)", + "url 1.7.2 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "cookie_store" +version = "0.7.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "cookie 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)", + "failure 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)", + "idna 0.1.5 (registry+https://github.com/rust-lang/crates.io-index)", + "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", +<<<<<<< HEAD + "publicsuffix 1.5.3 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", +======= + "publicsuffix 1.5.4 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", +>>>>>>> 9e0aa9e5d... base-code(rebased) + "serde_json 1.0.41 (registry+https://github.com/rust-lang/crates.io-index)", + "time 0.1.42 (registry+https://github.com/rust-lang/crates.io-index)", + "try_from 0.3.2 (registry+https://github.com/rust-lang/crates.io-index)", + "url 1.7.2 (registry+https://github.com/rust-lang/crates.io-index)", +] + [[package]] name = "core-foundation" version = "0.6.4" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "core-foundation-sys 0.6.2 (registry+https://github.com/rust-lang/crates.io-index)", - "libc 0.2.62 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] @@ -505,6 +565,89 @@ name = "core-foundation-sys" version = "0.6.2" source = "registry+https://github.com/rust-lang/crates.io-index" +[[package]] +name = "cranelift-bforest" +version = "0.46.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "cranelift-entity 0.46.1 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "cranelift-codegen" +version = "0.46.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "cranelift-bforest 0.46.1 (registry+https://github.com/rust-lang/crates.io-index)", + "cranelift-codegen-meta 0.46.1 (registry+https://github.com/rust-lang/crates.io-index)", + "cranelift-codegen-shared 0.46.1 (registry+https://github.com/rust-lang/crates.io-index)", + "cranelift-entity 0.46.1 (registry+https://github.com/rust-lang/crates.io-index)", + "failure 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)", + "failure_derive 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)", + "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "smallvec 0.6.13 (registry+https://github.com/rust-lang/crates.io-index)", + "target-lexicon 0.8.1 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "cranelift-codegen-meta" +version = "0.46.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "cranelift-codegen-shared 0.46.1 (registry+https://github.com/rust-lang/crates.io-index)", + "cranelift-entity 0.46.1 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "cranelift-codegen-shared" +version = "0.46.1" +source = "registry+https://github.com/rust-lang/crates.io-index" + +[[package]] +name = "cranelift-entity" +version = "0.46.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "cranelift-frontend" +version = "0.46.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "cranelift-codegen 0.46.1 (registry+https://github.com/rust-lang/crates.io-index)", + "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", + "smallvec 0.6.13 (registry+https://github.com/rust-lang/crates.io-index)", + "target-lexicon 0.8.1 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "cranelift-native" +version = "0.46.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "cranelift-codegen 0.46.1 (registry+https://github.com/rust-lang/crates.io-index)", + "raw-cpuid 6.1.0 (registry+https://github.com/rust-lang/crates.io-index)", + "target-lexicon 0.8.1 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "cranelift-wasm" +version = "0.46.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "cranelift-codegen 0.46.1 (registry+https://github.com/rust-lang/crates.io-index)", + "cranelift-entity 0.46.1 (registry+https://github.com/rust-lang/crates.io-index)", + "cranelift-frontend 0.46.1 (registry+https://github.com/rust-lang/crates.io-index)", + "failure 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)", + "failure_derive 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)", + "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "wasmparser 0.39.2 (registry+https://github.com/rust-lang/crates.io-index)", +] + [[package]] name = "crc32fast" version = "1.2.0" @@ -523,17 +666,17 @@ dependencies = [ "clap 2.33.0 (registry+https://github.com/rust-lang/crates.io-index)", "criterion-plot 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", "csv 1.1.1 (registry+https://github.com/rust-lang/crates.io-index)", - "itertools 0.8.0 (registry+https://github.com/rust-lang/crates.io-index)", + "itertools 0.8.1 (registry+https://github.com/rust-lang/crates.io-index)", "lazy_static 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)", - "libc 0.2.62 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", "num-traits 0.2.8 (registry+https://github.com/rust-lang/crates.io-index)", "rand_core 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", "rand_os 0.1.3 (registry+https://github.com/rust-lang/crates.io-index)", "rand_xoshiro 0.1.0 (registry+https://github.com/rust-lang/crates.io-index)", "rayon 1.2.0 (registry+https://github.com/rust-lang/crates.io-index)", "rayon-core 1.6.0 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", - "serde_derive 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "serde_derive 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", "serde_json 1.0.41 (registry+https://github.com/rust-lang/crates.io-index)", "tinytemplate 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)", "walkdir 2.2.9 (registry+https://github.com/rust-lang/crates.io-index)", @@ -549,15 +692,15 @@ dependencies = [ "clap 2.33.0 (registry+https://github.com/rust-lang/crates.io-index)", "criterion-plot 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)", "csv 1.1.1 (registry+https://github.com/rust-lang/crates.io-index)", - "itertools 0.8.0 (registry+https://github.com/rust-lang/crates.io-index)", + "itertools 0.8.1 (registry+https://github.com/rust-lang/crates.io-index)", "lazy_static 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)", "num-traits 0.2.8 (registry+https://github.com/rust-lang/crates.io-index)", "rand_core 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)", "rand_os 0.2.2 (registry+https://github.com/rust-lang/crates.io-index)", "rand_xoshiro 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", "rayon 1.2.0 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", - "serde_derive 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "serde_derive 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", "serde_json 1.0.41 (registry+https://github.com/rust-lang/crates.io-index)", "tinytemplate 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)", "walkdir 2.2.9 (registry+https://github.com/rust-lang/crates.io-index)", @@ -570,7 +713,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "byteorder 1.3.2 (registry+https://github.com/rust-lang/crates.io-index)", "cast 0.2.2 (registry+https://github.com/rust-lang/crates.io-index)", - "itertools 0.8.0 (registry+https://github.com/rust-lang/crates.io-index)", + "itertools 0.8.1 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] @@ -579,7 +722,7 @@ version = "0.4.0" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "cast 0.2.2 (registry+https://github.com/rust-lang/crates.io-index)", - "itertools 0.8.0 (registry+https://github.com/rust-lang/crates.io-index)", + "itertools 0.8.1 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] @@ -592,23 +735,23 @@ dependencies = [ [[package]] name = "crossbeam-deque" -version = "0.7.1" +version = "0.7.2" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "crossbeam-epoch 0.7.2 (registry+https://github.com/rust-lang/crates.io-index)", - "crossbeam-utils 0.6.6 (registry+https://github.com/rust-lang/crates.io-index)", + "crossbeam-epoch 0.8.0 (registry+https://github.com/rust-lang/crates.io-index)", + "crossbeam-utils 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] name = "crossbeam-epoch" -version = "0.7.2" +version = "0.8.0" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "arrayvec 0.4.12 (registry+https://github.com/rust-lang/crates.io-index)", + "autocfg 0.1.7 (registry+https://github.com/rust-lang/crates.io-index)", "cfg-if 0.1.10 (registry+https://github.com/rust-lang/crates.io-index)", - "crossbeam-utils 0.6.6 (registry+https://github.com/rust-lang/crates.io-index)", + "crossbeam-utils 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)", "lazy_static 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)", - "memoffset 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)", + "memoffset 0.5.3 (registry+https://github.com/rust-lang/crates.io-index)", "scopeguard 1.0.0 (registry+https://github.com/rust-lang/crates.io-index)", ] @@ -630,18 +773,19 @@ dependencies = [ ] [[package]] -name = "crunchy" -version = "0.2.2" +name = "crossbeam-utils" +version = "0.7.0" source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "autocfg 0.1.7 (registry+https://github.com/rust-lang/crates.io-index)", + "cfg-if 0.1.10 (registry+https://github.com/rust-lang/crates.io-index)", + "lazy_static 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)", +] [[package]] -name = "crypto-mac" -version = "0.4.0" +name = "crunchy" +version = "0.2.2" source = "registry+https://github.com/rust-lang/crates.io-index" -dependencies = [ - "constant_time_eq 0.1.4 (registry+https://github.com/rust-lang/crates.io-index)", - "generic-array 0.8.3 (registry+https://github.com/rust-lang/crates.io-index)", -] [[package]] name = "crypto-mac" @@ -660,8 +804,8 @@ dependencies = [ "bstr 0.2.8 (registry+https://github.com/rust-lang/crates.io-index)", "csv-core 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)", "itoa 0.4.4 (registry+https://github.com/rust-lang/crates.io-index)", - "ryu 1.0.1 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", + "ryu 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] @@ -672,13 +816,21 @@ dependencies = [ "memchr 2.2.1 (registry+https://github.com/rust-lang/crates.io-index)", ] +[[package]] +name = "ct-logs" +version = "0.6.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "sct 0.6.0 (registry+https://github.com/rust-lang/crates.io-index)", +] + [[package]] name = "ctor" version = "0.1.12" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "quote 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)", - "syn 1.0.7 (registry+https://github.com/rust-lang/crates.io-index)", + "syn 1.0.8 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] @@ -745,18 +897,30 @@ source = "registry+https://github.com/rust-lang/crates.io-index" [[package]] name = "digest" -version = "0.6.2" +version = "0.8.1" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "generic-array 0.8.3 (registry+https://github.com/rust-lang/crates.io-index)", + "generic-array 0.12.3 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "digest" -version = "0.8.1" +name = "directories" +version = "2.0.2" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "generic-array 0.12.3 (registry+https://github.com/rust-lang/crates.io-index)", + "cfg-if 0.1.10 (registry+https://github.com/rust-lang/crates.io-index)", + "dirs-sys 0.3.4 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "dirs-sys" +version = "0.3.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "cfg-if 0.1.10 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", + "redox_users 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", + "winapi 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] @@ -773,6 +937,11 @@ name = "doc-comment" version = "0.3.1" source = "registry+https://github.com/rust-lang/crates.io-index" +[[package]] +name = "dtoa" +version = "0.4.4" +source = "registry+https://github.com/rust-lang/crates.io-index" + [[package]] name = "ed25519-dalek" version = "0.9.1" @@ -810,6 +979,14 @@ dependencies = [ "heapsize 0.4.2 (registry+https://github.com/rust-lang/crates.io-index)", ] +[[package]] +name = "encoding_rs" +version = "0.8.20" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "cfg-if 0.1.10 (registry+https://github.com/rust-lang/crates.io-index)", +] + [[package]] name = "env_logger" version = "0.6.2" @@ -824,7 +1001,7 @@ dependencies = [ [[package]] name = "env_logger" -version = "0.7.0" +version = "0.7.1" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "atty 0.2.13 (registry+https://github.com/rust-lang/crates.io-index)", @@ -844,94 +1021,170 @@ name = "erased-serde" version = "0.3.9" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "error-chain" -version = "0.12.1" +name = "errno" +version = "0.2.4" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "backtrace 0.3.38 (registry+https://github.com/rust-lang/crates.io-index)", - "version_check 0.1.5 (registry+https://github.com/rust-lang/crates.io-index)", + "errno-dragonfly 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", + "winapi 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "exit-future" -version = "0.1.4" +name = "errno-dragonfly" +version = "0.1.1" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", - "parking_lot 0.7.1 (registry+https://github.com/rust-lang/crates.io-index)", + "gcc 0.3.55 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "failure" -version = "0.1.6" +name = "error-chain" +version = "0.12.1" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "backtrace 0.3.38 (registry+https://github.com/rust-lang/crates.io-index)", - "failure_derive 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)", + "version_check 0.1.5 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "failure_derive" -version = "0.1.6" +name = "evm" +version = "0.14.0" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "proc-macro2 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "quote 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)", - "syn 1.0.7 (registry+https://github.com/rust-lang/crates.io-index)", - "synstructure 0.12.1 (registry+https://github.com/rust-lang/crates.io-index)", + "evm-core 0.14.0 (registry+https://github.com/rust-lang/crates.io-index)", + "evm-gasometer 0.14.0 (registry+https://github.com/rust-lang/crates.io-index)", + "evm-runtime 0.14.1 (registry+https://github.com/rust-lang/crates.io-index)", + "primitive-types 0.6.1 (registry+https://github.com/rust-lang/crates.io-index)", + "rlp 0.4.3 (registry+https://github.com/rust-lang/crates.io-index)", + "sha3 0.8.2 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "fake-simd" -version = "0.1.2" +name = "evm-core" +version = "0.14.0" source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "primitive-types 0.6.1 (registry+https://github.com/rust-lang/crates.io-index)", +] [[package]] -name = "fdlimit" -version = "0.1.1" +name = "evm-gasometer" +version = "0.14.0" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "libc 0.2.62 (registry+https://github.com/rust-lang/crates.io-index)", + "evm-core 0.14.0 (registry+https://github.com/rust-lang/crates.io-index)", + "evm-runtime 0.14.1 (registry+https://github.com/rust-lang/crates.io-index)", + "primitive-types 0.6.1 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "finality-grandpa" -version = "0.9.0" +name = "evm-runtime" +version = "0.14.1" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", - "hashmap_core 0.1.11 (registry+https://github.com/rust-lang/crates.io-index)", - "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", - "num-traits 0.2.8 (registry+https://github.com/rust-lang/crates.io-index)", - "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "parking_lot 0.9.0 (registry+https://github.com/rust-lang/crates.io-index)", + "evm-core 0.14.0 (registry+https://github.com/rust-lang/crates.io-index)", + "primitive-types 0.6.1 (registry+https://github.com/rust-lang/crates.io-index)", + "sha3 0.8.2 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "fixed-hash" -version = "0.4.0" +name = "exit-future" +version = "0.1.4" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "byteorder 1.3.2 (registry+https://github.com/rust-lang/crates.io-index)", - "libc 0.2.62 (registry+https://github.com/rust-lang/crates.io-index)", - "rand 0.5.6 (registry+https://github.com/rust-lang/crates.io-index)", - "rustc-hex 2.0.1 (registry+https://github.com/rust-lang/crates.io-index)", - "static_assertions 0.2.5 (registry+https://github.com/rust-lang/crates.io-index)", + "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", + "parking_lot 0.7.1 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "faerie" +version = "0.11.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "failure 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)", + "goblin 0.0.24 (registry+https://github.com/rust-lang/crates.io-index)", + "indexmap 1.3.0 (registry+https://github.com/rust-lang/crates.io-index)", + "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", + "scroll 0.9.2 (registry+https://github.com/rust-lang/crates.io-index)", + "string-interner 0.7.1 (registry+https://github.com/rust-lang/crates.io-index)", + "target-lexicon 0.8.1 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "failure" +version = "0.1.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "backtrace 0.3.40 (registry+https://github.com/rust-lang/crates.io-index)", + "failure_derive 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "failure_derive" +version = "0.1.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "proc-macro2 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "quote 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)", + "syn 1.0.8 (registry+https://github.com/rust-lang/crates.io-index)", + "synstructure 0.12.2 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "fake-simd" +version = "0.1.2" +source = "registry+https://github.com/rust-lang/crates.io-index" + +[[package]] +name = "fallible-iterator" +version = "0.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" + +[[package]] +name = "fdlimit" +version = "0.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "file-per-thread-logger" +version = "0.1.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "env_logger 0.6.2 (registry+https://github.com/rust-lang/crates.io-index)", + "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "finality-grandpa" +version = "0.9.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", + "hashbrown 0.6.3 (registry+https://github.com/rust-lang/crates.io-index)", + "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", + "num-traits 0.2.8 (registry+https://github.com/rust-lang/crates.io-index)", + "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "parking_lot 0.9.0 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] name = "fixed-hash" -version = "0.5.0" +version = "0.5.1" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "byteorder 1.3.2 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", "rand 0.7.2 (registry+https://github.com/rust-lang/crates.io-index)", "rustc-hex 2.0.1 (registry+https://github.com/rust-lang/crates.io-index)", - "static_assertions 0.2.5 (registry+https://github.com/rust-lang/crates.io-index)", + "static_assertions 1.1.0 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] @@ -941,15 +1194,15 @@ source = "registry+https://github.com/rust-lang/crates.io-index" [[package]] name = "flate2" -version = "1.0.12" +version = "1.0.13" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "cfg-if 0.1.10 (registry+https://github.com/rust-lang/crates.io-index)", "crc32fast 1.2.0 (registry+https://github.com/rust-lang/crates.io-index)", "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", - "libc 0.2.62 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", "libz-sys 1.0.25 (registry+https://github.com/rust-lang/crates.io-index)", - "miniz_oxide 0.3.3 (registry+https://github.com/rust-lang/crates.io-index)", + "miniz_oxide 0.3.5 (registry+https://github.com/rust-lang/crates.io-index)", "tokio-io 0.1.12 (registry+https://github.com/rust-lang/crates.io-index)", ] @@ -984,7 +1237,7 @@ version = "0.2.4" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "lazy_static 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)", - "libc 0.2.62 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", "libloading 0.5.2 (registry+https://github.com/rust-lang/crates.io-index)", "winapi 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)", ] @@ -994,7 +1247,7 @@ name = "fs2" version = "0.4.3" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "libc 0.2.62 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", "winapi 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)", ] @@ -1008,7 +1261,7 @@ name = "fuchsia-zircon" version = "0.3.3" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "bitflags 1.2.0 (registry+https://github.com/rust-lang/crates.io-index)", + "bitflags 1.2.1 (registry+https://github.com/rust-lang/crates.io-index)", "fuchsia-zircon-sys 0.3.3 (registry+https://github.com/rust-lang/crates.io-index)", ] @@ -1022,6 +1275,29 @@ name = "futures" version = "0.1.29" source = "registry+https://github.com/rust-lang/crates.io-index" +[[package]] +name = "futures" +version = "0.3.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "futures-channel 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", + "futures-core 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", + "futures-executor 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", + "futures-io 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", + "futures-sink 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", + "futures-task 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", + "futures-util 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "futures-channel" +version = "0.3.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "futures-core 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", + "futures-sink 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", +] + [[package]] name = "futures-channel-preview" version = "0.3.0-alpha.19" @@ -1031,6 +1307,11 @@ dependencies = [ "futures-sink-preview 0.3.0-alpha.19 (registry+https://github.com/rust-lang/crates.io-index)", ] +[[package]] +name = "futures-core" +version = "0.3.1" +source = "registry+https://github.com/rust-lang/crates.io-index" + [[package]] name = "futures-core-preview" version = "0.3.0-alpha.19" @@ -1042,7 +1323,18 @@ version = "0.1.8" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", - "num_cpus 1.10.1 (registry+https://github.com/rust-lang/crates.io-index)", + "num_cpus 1.11.0 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "futures-executor" +version = "0.3.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "futures-core 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", + "futures-task 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", + "futures-util 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", + "num_cpus 1.11.0 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] @@ -1052,14 +1344,30 @@ source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "futures-core-preview 0.3.0-alpha.19 (registry+https://github.com/rust-lang/crates.io-index)", "futures-util-preview 0.3.0-alpha.19 (registry+https://github.com/rust-lang/crates.io-index)", - "num_cpus 1.10.1 (registry+https://github.com/rust-lang/crates.io-index)", + "num_cpus 1.11.0 (registry+https://github.com/rust-lang/crates.io-index)", ] +[[package]] +name = "futures-io" +version = "0.3.1" +source = "registry+https://github.com/rust-lang/crates.io-index" + [[package]] name = "futures-io-preview" version = "0.3.0-alpha.19" source = "registry+https://github.com/rust-lang/crates.io-index" +[[package]] +name = "futures-macro" +version = "0.3.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "proc-macro-hack 0.5.11 (registry+https://github.com/rust-lang/crates.io-index)", + "proc-macro2 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "quote 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)", + "syn 1.0.8 (registry+https://github.com/rust-lang/crates.io-index)", +] + [[package]] name = "futures-preview" version = "0.3.0-alpha.19" @@ -1073,11 +1381,21 @@ dependencies = [ "futures-util-preview 0.3.0-alpha.19 (registry+https://github.com/rust-lang/crates.io-index)", ] +[[package]] +name = "futures-sink" +version = "0.3.1" +source = "registry+https://github.com/rust-lang/crates.io-index" + [[package]] name = "futures-sink-preview" version = "0.3.0-alpha.19" source = "registry+https://github.com/rust-lang/crates.io-index" +[[package]] +name = "futures-task" +version = "0.3.1" +source = "registry+https://github.com/rust-lang/crates.io-index" + [[package]] name = "futures-timer" version = "0.4.0" @@ -1088,6 +1406,25 @@ dependencies = [ "pin-utils 0.1.0-alpha.4 (registry+https://github.com/rust-lang/crates.io-index)", ] +[[package]] +name = "futures-util" +version = "0.3.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", + "futures-channel 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", + "futures-core 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", + "futures-io 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", + "futures-macro 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", + "futures-sink 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", + "futures-task 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", + "memchr 2.2.1 (registry+https://github.com/rust-lang/crates.io-index)", + "pin-utils 0.1.0-alpha.4 (registry+https://github.com/rust-lang/crates.io-index)", + "proc-macro-hack 0.5.11 (registry+https://github.com/rust-lang/crates.io-index)", + "proc-macro-nested 0.1.3 (registry+https://github.com/rust-lang/crates.io-index)", + "slab 0.4.2 (registry+https://github.com/rust-lang/crates.io-index)", +] + [[package]] name = "futures-util-preview" version = "0.3.0-alpha.19" @@ -1108,15 +1445,6 @@ name = "gcc" version = "0.3.55" source = "registry+https://github.com/rust-lang/crates.io-index" -[[package]] -name = "generic-array" -version = "0.8.3" -source = "registry+https://github.com/rust-lang/crates.io-index" -dependencies = [ - "nodrop 0.1.14 (registry+https://github.com/rust-lang/crates.io-index)", - "typenum 1.11.2 (registry+https://github.com/rust-lang/crates.io-index)", -] - [[package]] name = "generic-array" version = "0.12.3" @@ -1132,7 +1460,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "c_linked_list 1.1.1 (registry+https://github.com/rust-lang/crates.io-index)", "get_if_addrs-sys 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)", - "libc 0.2.62 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", "winapi 0.2.8 (registry+https://github.com/rust-lang/crates.io-index)", ] @@ -1142,17 +1470,30 @@ version = "0.1.1" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "gcc 0.3.55 (registry+https://github.com/rust-lang/crates.io-index)", - "libc 0.2.62 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] name = "getrandom" -version = "0.1.12" +version = "0.1.13" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "cfg-if 0.1.10 (registry+https://github.com/rust-lang/crates.io-index)", - "libc 0.2.62 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", "wasi 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)", + "wasm-bindgen 0.2.54 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "gimli" +version = "0.19.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "arrayvec 0.4.12 (registry+https://github.com/rust-lang/crates.io-index)", + "byteorder 1.3.2 (registry+https://github.com/rust-lang/crates.io-index)", + "fallible-iterator 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)", + "indexmap 1.3.0 (registry+https://github.com/rust-lang/crates.io-index)", + "stable_deref_trait 1.1.1 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] @@ -1177,6 +1518,16 @@ dependencies = [ "regex 1.3.1 (registry+https://github.com/rust-lang/crates.io-index)", ] +[[package]] +name = "goblin" +version = "0.0.24" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", + "plain 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)", + "scroll 0.9.2 (registry+https://github.com/rust-lang/crates.io-index)", +] + [[package]] name = "h2" version = "0.1.26" @@ -1186,8 +1537,8 @@ dependencies = [ "bytes 0.4.12 (registry+https://github.com/rust-lang/crates.io-index)", "fnv 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", - "http 0.1.18 (registry+https://github.com/rust-lang/crates.io-index)", - "indexmap 1.2.0 (registry+https://github.com/rust-lang/crates.io-index)", + "http 0.1.19 (registry+https://github.com/rust-lang/crates.io-index)", + "indexmap 1.3.0 (registry+https://github.com/rust-lang/crates.io-index)", "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", "slab 0.4.2 (registry+https://github.com/rust-lang/crates.io-index)", "string 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)", @@ -1218,18 +1569,13 @@ dependencies = [ [[package]] name = "hashbrown" -version = "0.6.1" +version = "0.6.3" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "ahash 0.2.16 (registry+https://github.com/rust-lang/crates.io-index)", - "autocfg 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)", + "ahash 0.2.17 (registry+https://github.com/rust-lang/crates.io-index)", + "autocfg 0.1.7 (registry+https://github.com/rust-lang/crates.io-index)", ] -[[package]] -name = "hashmap_core" -version = "0.1.11" -source = "registry+https://github.com/rust-lang/crates.io-index" - [[package]] name = "heapsize" version = "0.4.2" @@ -1243,7 +1589,15 @@ name = "heck" version = "0.3.1" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "unicode-segmentation 1.3.0 (registry+https://github.com/rust-lang/crates.io-index)", + "unicode-segmentation 1.6.0 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "hermit-abi" +version = "0.1.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] @@ -1251,13 +1605,18 @@ name = "hex" version = "0.3.2" source = "registry+https://github.com/rust-lang/crates.io-index" +[[package]] +name = "hex" +version = "0.4.0" +source = "registry+https://github.com/rust-lang/crates.io-index" + [[package]] name = "hex-literal" version = "0.2.1" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "hex-literal-impl 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)", - "proc-macro-hack 0.5.10 (registry+https://github.com/rust-lang/crates.io-index)", + "proc-macro-hack 0.5.11 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] @@ -1265,17 +1624,7 @@ name = "hex-literal-impl" version = "0.2.1" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "proc-macro-hack 0.5.10 (registry+https://github.com/rust-lang/crates.io-index)", -] - -[[package]] -name = "hmac" -version = "0.4.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -dependencies = [ - "crypto-mac 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)", - "digest 0.6.2 (registry+https://github.com/rust-lang/crates.io-index)", - "generic-array 0.8.3 (registry+https://github.com/rust-lang/crates.io-index)", + "proc-macro-hack 0.5.11 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] @@ -1287,16 +1636,6 @@ dependencies = [ "digest 0.8.1 (registry+https://github.com/rust-lang/crates.io-index)", ] -[[package]] -name = "hmac-drbg" -version = "0.1.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -dependencies = [ - "digest 0.6.2 (registry+https://github.com/rust-lang/crates.io-index)", - "generic-array 0.8.3 (registry+https://github.com/rust-lang/crates.io-index)", - "hmac 0.4.2 (registry+https://github.com/rust-lang/crates.io-index)", -] - [[package]] name = "hmac-drbg" version = "0.2.0" @@ -1309,7 +1648,7 @@ dependencies = [ [[package]] name = "http" -version = "0.1.18" +version = "0.1.19" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "bytes 0.4.12 (registry+https://github.com/rust-lang/crates.io-index)", @@ -1324,7 +1663,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "bytes 0.4.12 (registry+https://github.com/rust-lang/crates.io-index)", "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", - "http 0.1.18 (registry+https://github.com/rust-lang/crates.io-index)", + "http 0.1.19 (registry+https://github.com/rust-lang/crates.io-index)", "tokio-buf 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)", ] @@ -1351,7 +1690,7 @@ dependencies = [ "language-tags 0.2.2 (registry+https://github.com/rust-lang/crates.io-index)", "log 0.3.9 (registry+https://github.com/rust-lang/crates.io-index)", "mime 0.2.6 (registry+https://github.com/rust-lang/crates.io-index)", - "num_cpus 1.10.1 (registry+https://github.com/rust-lang/crates.io-index)", + "num_cpus 1.11.0 (registry+https://github.com/rust-lang/crates.io-index)", "time 0.1.42 (registry+https://github.com/rust-lang/crates.io-index)", "traitobject 0.1.0 (registry+https://github.com/rust-lang/crates.io-index)", "typeable 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)", @@ -1368,7 +1707,7 @@ dependencies = [ "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", "futures-cpupool 0.1.8 (registry+https://github.com/rust-lang/crates.io-index)", "h2 0.1.26 (registry+https://github.com/rust-lang/crates.io-index)", - "http 0.1.18 (registry+https://github.com/rust-lang/crates.io-index)", + "http 0.1.19 (registry+https://github.com/rust-lang/crates.io-index)", "http-body 0.1.0 (registry+https://github.com/rust-lang/crates.io-index)", "httparse 1.3.4 (registry+https://github.com/rust-lang/crates.io-index)", "iovec 0.1.4 (registry+https://github.com/rust-lang/crates.io-index)", @@ -1388,6 +1727,22 @@ dependencies = [ "want 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)", ] +[[package]] +name = "hyper-rustls" +version = "0.17.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "bytes 0.4.12 (registry+https://github.com/rust-lang/crates.io-index)", + "ct-logs 0.6.0 (registry+https://github.com/rust-lang/crates.io-index)", + "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", + "hyper 0.12.35 (registry+https://github.com/rust-lang/crates.io-index)", + "rustls 0.16.0 (registry+https://github.com/rust-lang/crates.io-index)", + "tokio-io 0.1.12 (registry+https://github.com/rust-lang/crates.io-index)", + "tokio-rustls 0.10.2 (registry+https://github.com/rust-lang/crates.io-index)", + "webpki 0.21.0 (registry+https://github.com/rust-lang/crates.io-index)", + "webpki-roots 0.17.0 (registry+https://github.com/rust-lang/crates.io-index)", +] + [[package]] name = "hyper-tls" version = "0.3.2" @@ -1407,7 +1762,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "matches 0.1.8 (registry+https://github.com/rust-lang/crates.io-index)", "unicode-bidi 0.3.4 (registry+https://github.com/rust-lang/crates.io-index)", - "unicode-normalization 0.1.8 (registry+https://github.com/rust-lang/crates.io-index)", + "unicode-normalization 0.1.9 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] @@ -1417,23 +1772,31 @@ source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "matches 0.1.8 (registry+https://github.com/rust-lang/crates.io-index)", "unicode-bidi 0.3.4 (registry+https://github.com/rust-lang/crates.io-index)", - "unicode-normalization 0.1.8 (registry+https://github.com/rust-lang/crates.io-index)", + "unicode-normalization 0.1.9 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] name = "impl-codec" -version = "0.4.1" +version = "0.4.2" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", ] +[[package]] +name = "impl-rlp" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "rlp 0.4.3 (registry+https://github.com/rust-lang/crates.io-index)", +] + [[package]] name = "impl-serde" version = "0.2.3" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] @@ -1443,13 +1806,16 @@ source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "proc-macro2 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", "quote 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)", - "syn 1.0.7 (registry+https://github.com/rust-lang/crates.io-index)", + "syn 1.0.8 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] name = "indexmap" -version = "1.2.0" +version = "1.3.0" source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "autocfg 0.1.7 (registry+https://github.com/rust-lang/crates.io-index)", +] [[package]] name = "integer-sqrt" @@ -1466,17 +1832,17 @@ name = "iovec" version = "0.1.4" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "libc 0.2.62 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] name = "ipnet" -version = "2.0.1" +version = "2.1.0" source = "registry+https://github.com/rust-lang/crates.io-index" [[package]] name = "itertools" -version = "0.8.0" +version = "0.8.1" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "either 1.5.3 (registry+https://github.com/rust-lang/crates.io-index)", @@ -1492,31 +1858,32 @@ name = "jobserver" version = "0.1.17" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "getrandom 0.1.12 (registry+https://github.com/rust-lang/crates.io-index)", - "libc 0.2.62 (registry+https://github.com/rust-lang/crates.io-index)", + "getrandom 0.1.13 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] name = "js-sys" -version = "0.3.28" +version = "0.3.31" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "wasm-bindgen 0.2.51 (registry+https://github.com/rust-lang/crates.io-index)", + "wasm-bindgen 0.2.54 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] name = "jsonrpc-client-transports" -version = "13.2.0" +version = "14.0.3" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "failure 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)", "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", "hyper 0.12.35 (registry+https://github.com/rust-lang/crates.io-index)", - "jsonrpc-core 13.2.0 (registry+https://github.com/rust-lang/crates.io-index)", - "jsonrpc-pubsub 13.2.0 (registry+https://github.com/rust-lang/crates.io-index)", + "hyper-tls 0.3.2 (registry+https://github.com/rust-lang/crates.io-index)", + "jsonrpc-core 14.0.3 (registry+https://github.com/rust-lang/crates.io-index)", + "jsonrpc-pubsub 14.0.3 (registry+https://github.com/rust-lang/crates.io-index)", "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", "serde_json 1.0.41 (registry+https://github.com/rust-lang/crates.io-index)", "tokio 0.1.22 (registry+https://github.com/rust-lang/crates.io-index)", "url 1.7.2 (registry+https://github.com/rust-lang/crates.io-index)", @@ -1525,43 +1892,43 @@ dependencies = [ [[package]] name = "jsonrpc-core" -version = "13.2.0" +version = "14.0.3" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", - "serde_derive 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "serde_derive 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", "serde_json 1.0.41 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] name = "jsonrpc-core-client" -version = "13.2.0" +version = "14.0.3" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "jsonrpc-client-transports 13.2.0 (registry+https://github.com/rust-lang/crates.io-index)", + "jsonrpc-client-transports 14.0.3 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] name = "jsonrpc-derive" -version = "13.2.0" +version = "14.0.3" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "proc-macro-crate 0.1.4 (registry+https://github.com/rust-lang/crates.io-index)", - "proc-macro2 0.4.30 (registry+https://github.com/rust-lang/crates.io-index)", - "quote 0.6.13 (registry+https://github.com/rust-lang/crates.io-index)", - "syn 0.15.44 (registry+https://github.com/rust-lang/crates.io-index)", + "proc-macro2 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "quote 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)", + "syn 1.0.8 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] name = "jsonrpc-http-server" -version = "13.2.0" +version = "14.0.3" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "hyper 0.12.35 (registry+https://github.com/rust-lang/crates.io-index)", - "jsonrpc-core 13.2.0 (registry+https://github.com/rust-lang/crates.io-index)", - "jsonrpc-server-utils 13.2.0 (registry+https://github.com/rust-lang/crates.io-index)", + "jsonrpc-core 14.0.3 (registry+https://github.com/rust-lang/crates.io-index)", + "jsonrpc-server-utils 14.0.3 (registry+https://github.com/rust-lang/crates.io-index)", "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", "net2 0.2.33 (registry+https://github.com/rust-lang/crates.io-index)", "parking_lot 0.9.0 (registry+https://github.com/rust-lang/crates.io-index)", @@ -1570,26 +1937,25 @@ dependencies = [ [[package]] name = "jsonrpc-pubsub" -version = "13.2.0" +version = "14.0.3" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "jsonrpc-core 13.2.0 (registry+https://github.com/rust-lang/crates.io-index)", + "jsonrpc-core 14.0.3 (registry+https://github.com/rust-lang/crates.io-index)", "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", "parking_lot 0.9.0 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] name = "jsonrpc-server-utils" -version = "13.2.0" +version = "14.0.3" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "bytes 0.4.12 (registry+https://github.com/rust-lang/crates.io-index)", "globset 0.4.4 (registry+https://github.com/rust-lang/crates.io-index)", - "jsonrpc-core 13.2.0 (registry+https://github.com/rust-lang/crates.io-index)", + "jsonrpc-core 14.0.3 (registry+https://github.com/rust-lang/crates.io-index)", "lazy_static 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)", "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", - "num_cpus 1.10.1 (registry+https://github.com/rust-lang/crates.io-index)", "tokio 0.1.22 (registry+https://github.com/rust-lang/crates.io-index)", "tokio-codec 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)", "unicase 2.5.1 (registry+https://github.com/rust-lang/crates.io-index)", @@ -1597,15 +1963,15 @@ dependencies = [ [[package]] name = "jsonrpc-ws-server" -version = "13.2.0" +version = "14.0.3" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "jsonrpc-core 13.2.0 (registry+https://github.com/rust-lang/crates.io-index)", - "jsonrpc-server-utils 13.2.0 (registry+https://github.com/rust-lang/crates.io-index)", + "jsonrpc-core 14.0.3 (registry+https://github.com/rust-lang/crates.io-index)", + "jsonrpc-server-utils 14.0.3 (registry+https://github.com/rust-lang/crates.io-index)", "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", "parking_lot 0.9.0 (registry+https://github.com/rust-lang/crates.io-index)", "slab 0.4.2 (registry+https://github.com/rust-lang/crates.io-index)", - "ws 0.9.0 (registry+https://github.com/rust-lang/crates.io-index)", + "ws 0.9.1 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] @@ -1660,7 +2026,7 @@ dependencies = [ "interleaved-ordered 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)", "kvdb 0.1.0 (git+https://github.com/paritytech/parity-common?rev=b0317f649ab2c665b7987b8475878fc4d2e1f81d)", "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", - "num_cpus 1.10.1 (registry+https://github.com/rust-lang/crates.io-index)", + "num_cpus 1.11.0 (registry+https://github.com/rust-lang/crates.io-index)", "parking_lot 0.6.4 (registry+https://github.com/rust-lang/crates.io-index)", "regex 1.3.1 (registry+https://github.com/rust-lang/crates.io-index)", "rocksdb 0.11.0 (registry+https://github.com/rust-lang/crates.io-index)", @@ -1683,7 +2049,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" [[package]] name = "libc" -version = "0.2.62" +version = "0.2.65" source = "registry+https://github.com/rust-lang/crates.io-index" [[package]] @@ -1691,42 +2057,41 @@ name = "libloading" version = "0.5.2" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "cc 1.0.45 (registry+https://github.com/rust-lang/crates.io-index)", + "cc 1.0.47 (registry+https://github.com/rust-lang/crates.io-index)", "winapi 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] name = "libp2p" -version = "0.12.0" +version = "0.13.0" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "bytes 0.4.12 (registry+https://github.com/rust-lang/crates.io-index)", "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", "lazy_static 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)", - "libp2p-core 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)", - "libp2p-core-derive 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)", - "libp2p-deflate 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)", - "libp2p-dns 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)", - "libp2p-floodsub 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)", - "libp2p-identify 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)", - "libp2p-kad 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)", - "libp2p-mdns 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)", - "libp2p-mplex 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)", - "libp2p-noise 0.10.0 (registry+https://github.com/rust-lang/crates.io-index)", - "libp2p-ping 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)", - "libp2p-plaintext 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)", - "libp2p-ratelimit 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)", - "libp2p-secio 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)", - "libp2p-swarm 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)", - "libp2p-tcp 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)", - "libp2p-uds 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)", - "libp2p-wasm-ext 0.5.0 (registry+https://github.com/rust-lang/crates.io-index)", - "libp2p-websocket 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)", - "libp2p-yamux 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)", - "parity-multiaddr 0.5.0 (registry+https://github.com/rust-lang/crates.io-index)", - "parity-multihash 0.1.3 (registry+https://github.com/rust-lang/crates.io-index)", - "parking_lot 0.8.0 (registry+https://github.com/rust-lang/crates.io-index)", - "smallvec 0.6.10 (registry+https://github.com/rust-lang/crates.io-index)", + "libp2p-core 0.13.0 (registry+https://github.com/rust-lang/crates.io-index)", + "libp2p-core-derive 0.13.0 (registry+https://github.com/rust-lang/crates.io-index)", + "libp2p-deflate 0.5.0 (registry+https://github.com/rust-lang/crates.io-index)", + "libp2p-dns 0.13.0 (registry+https://github.com/rust-lang/crates.io-index)", + "libp2p-floodsub 0.13.0 (registry+https://github.com/rust-lang/crates.io-index)", + "libp2p-identify 0.13.0 (registry+https://github.com/rust-lang/crates.io-index)", + "libp2p-kad 0.13.0 (registry+https://github.com/rust-lang/crates.io-index)", + "libp2p-mdns 0.13.0 (registry+https://github.com/rust-lang/crates.io-index)", + "libp2p-mplex 0.13.0 (registry+https://github.com/rust-lang/crates.io-index)", + "libp2p-noise 0.11.0 (registry+https://github.com/rust-lang/crates.io-index)", + "libp2p-ping 0.13.0 (registry+https://github.com/rust-lang/crates.io-index)", + "libp2p-plaintext 0.13.0 (registry+https://github.com/rust-lang/crates.io-index)", + "libp2p-secio 0.13.0 (registry+https://github.com/rust-lang/crates.io-index)", + "libp2p-swarm 0.3.0 (registry+https://github.com/rust-lang/crates.io-index)", + "libp2p-tcp 0.13.0 (registry+https://github.com/rust-lang/crates.io-index)", + "libp2p-uds 0.13.0 (registry+https://github.com/rust-lang/crates.io-index)", + "libp2p-wasm-ext 0.6.0 (registry+https://github.com/rust-lang/crates.io-index)", + "libp2p-websocket 0.13.0 (registry+https://github.com/rust-lang/crates.io-index)", + "libp2p-yamux 0.13.0 (registry+https://github.com/rust-lang/crates.io-index)", + "parity-multiaddr 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)", + "parity-multihash 0.1.4 (registry+https://github.com/rust-lang/crates.io-index)", + "parking_lot 0.9.0 (registry+https://github.com/rust-lang/crates.io-index)", + "smallvec 0.6.13 (registry+https://github.com/rust-lang/crates.io-index)", "tokio-codec 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)", "tokio-executor 0.1.8 (registry+https://github.com/rust-lang/crates.io-index)", "tokio-io 0.1.12 (registry+https://github.com/rust-lang/crates.io-index)", @@ -1735,42 +2100,42 @@ dependencies = [ [[package]] name = "libp2p-core" -version = "0.12.0" +version = "0.13.0" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "asn1_der 0.6.3 (registry+https://github.com/rust-lang/crates.io-index)", - "bs58 0.2.5 (registry+https://github.com/rust-lang/crates.io-index)", + "bs58 0.3.0 (registry+https://github.com/rust-lang/crates.io-index)", "bytes 0.4.12 (registry+https://github.com/rust-lang/crates.io-index)", "ed25519-dalek 1.0.0-pre.2 (registry+https://github.com/rust-lang/crates.io-index)", "failure 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)", "fnv 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", "lazy_static 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)", - "libsecp256k1 0.2.2 (registry+https://github.com/rust-lang/crates.io-index)", + "libsecp256k1 0.3.2 (registry+https://github.com/rust-lang/crates.io-index)", "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", - "multistream-select 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)", - "parity-multiaddr 0.5.0 (registry+https://github.com/rust-lang/crates.io-index)", - "parity-multihash 0.1.3 (registry+https://github.com/rust-lang/crates.io-index)", - "parking_lot 0.8.0 (registry+https://github.com/rust-lang/crates.io-index)", + "multistream-select 0.6.0 (registry+https://github.com/rust-lang/crates.io-index)", + "parity-multiaddr 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)", + "parity-multihash 0.1.4 (registry+https://github.com/rust-lang/crates.io-index)", + "parking_lot 0.9.0 (registry+https://github.com/rust-lang/crates.io-index)", "protobuf 2.8.1 (registry+https://github.com/rust-lang/crates.io-index)", "quick-error 1.2.2 (registry+https://github.com/rust-lang/crates.io-index)", - "rand 0.6.5 (registry+https://github.com/rust-lang/crates.io-index)", - "ring 0.14.6 (registry+https://github.com/rust-lang/crates.io-index)", + "rand 0.7.2 (registry+https://github.com/rust-lang/crates.io-index)", + "ring 0.16.9 (registry+https://github.com/rust-lang/crates.io-index)", "rw-stream-sink 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)", "sha2 0.8.0 (registry+https://github.com/rust-lang/crates.io-index)", - "smallvec 0.6.10 (registry+https://github.com/rust-lang/crates.io-index)", + "smallvec 0.6.13 (registry+https://github.com/rust-lang/crates.io-index)", "tokio-executor 0.1.8 (registry+https://github.com/rust-lang/crates.io-index)", "tokio-io 0.1.12 (registry+https://github.com/rust-lang/crates.io-index)", "unsigned-varint 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)", - "untrusted 0.6.2 (registry+https://github.com/rust-lang/crates.io-index)", + "untrusted 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)", "void 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)", "wasm-timer 0.1.3 (registry+https://github.com/rust-lang/crates.io-index)", - "zeroize 0.9.3 (registry+https://github.com/rust-lang/crates.io-index)", + "zeroize 1.0.0 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] name = "libp2p-core-derive" -version = "0.12.0" +version = "0.13.0" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "quote 0.6.13 (registry+https://github.com/rust-lang/crates.io-index)", @@ -1779,86 +2144,85 @@ dependencies = [ [[package]] name = "libp2p-deflate" -version = "0.4.0" +version = "0.5.0" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "flate2 1.0.12 (registry+https://github.com/rust-lang/crates.io-index)", + "flate2 1.0.13 (registry+https://github.com/rust-lang/crates.io-index)", "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", - "libp2p-core 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)", + "libp2p-core 0.13.0 (registry+https://github.com/rust-lang/crates.io-index)", "tokio-io 0.1.12 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] name = "libp2p-dns" -version = "0.12.0" +version = "0.13.0" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", - "libp2p-core 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)", + "libp2p-core 0.13.0 (registry+https://github.com/rust-lang/crates.io-index)", "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", "tokio-dns-unofficial 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] name = "libp2p-floodsub" -version = "0.12.0" +version = "0.13.0" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "bs58 0.2.5 (registry+https://github.com/rust-lang/crates.io-index)", + "bs58 0.3.0 (registry+https://github.com/rust-lang/crates.io-index)", "bytes 0.4.12 (registry+https://github.com/rust-lang/crates.io-index)", "cuckoofilter 0.3.2 (registry+https://github.com/rust-lang/crates.io-index)", "fnv 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", - "libp2p-core 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)", - "libp2p-swarm 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)", + "libp2p-core 0.13.0 (registry+https://github.com/rust-lang/crates.io-index)", + "libp2p-swarm 0.3.0 (registry+https://github.com/rust-lang/crates.io-index)", "protobuf 2.8.1 (registry+https://github.com/rust-lang/crates.io-index)", "rand 0.6.5 (registry+https://github.com/rust-lang/crates.io-index)", - "smallvec 0.6.10 (registry+https://github.com/rust-lang/crates.io-index)", + "smallvec 0.6.13 (registry+https://github.com/rust-lang/crates.io-index)", "tokio-io 0.1.12 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] name = "libp2p-identify" -version = "0.12.0" +version = "0.13.0" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "bytes 0.4.12 (registry+https://github.com/rust-lang/crates.io-index)", "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", - "libp2p-core 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)", - "libp2p-swarm 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)", + "libp2p-core 0.13.0 (registry+https://github.com/rust-lang/crates.io-index)", + "libp2p-swarm 0.3.0 (registry+https://github.com/rust-lang/crates.io-index)", "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", - "parity-multiaddr 0.5.0 (registry+https://github.com/rust-lang/crates.io-index)", + "parity-multiaddr 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)", "protobuf 2.8.1 (registry+https://github.com/rust-lang/crates.io-index)", - "smallvec 0.6.10 (registry+https://github.com/rust-lang/crates.io-index)", + "smallvec 0.6.13 (registry+https://github.com/rust-lang/crates.io-index)", "tokio-codec 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)", "tokio-io 0.1.12 (registry+https://github.com/rust-lang/crates.io-index)", "unsigned-varint 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)", - "void 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)", "wasm-timer 0.1.3 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] name = "libp2p-kad" -version = "0.12.0" +version = "0.13.0" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "arrayvec 0.4.12 (registry+https://github.com/rust-lang/crates.io-index)", + "arrayvec 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)", "bytes 0.4.12 (registry+https://github.com/rust-lang/crates.io-index)", "either 1.5.3 (registry+https://github.com/rust-lang/crates.io-index)", "fnv 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", - "libp2p-core 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)", - "libp2p-swarm 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)", + "libp2p-core 0.13.0 (registry+https://github.com/rust-lang/crates.io-index)", + "libp2p-swarm 0.3.0 (registry+https://github.com/rust-lang/crates.io-index)", "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", - "parity-multiaddr 0.5.0 (registry+https://github.com/rust-lang/crates.io-index)", - "parity-multihash 0.1.3 (registry+https://github.com/rust-lang/crates.io-index)", + "parity-multiaddr 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)", + "parity-multihash 0.1.4 (registry+https://github.com/rust-lang/crates.io-index)", "protobuf 2.8.1 (registry+https://github.com/rust-lang/crates.io-index)", - "rand 0.6.5 (registry+https://github.com/rust-lang/crates.io-index)", + "rand 0.7.2 (registry+https://github.com/rust-lang/crates.io-index)", "sha2 0.8.0 (registry+https://github.com/rust-lang/crates.io-index)", - "smallvec 0.6.10 (registry+https://github.com/rust-lang/crates.io-index)", + "smallvec 0.6.13 (registry+https://github.com/rust-lang/crates.io-index)", "tokio-codec 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)", "tokio-io 0.1.12 (registry+https://github.com/rust-lang/crates.io-index)", - "uint 0.8.1 (registry+https://github.com/rust-lang/crates.io-index)", + "uint 0.8.2 (registry+https://github.com/rust-lang/crates.io-index)", "unsigned-varint 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)", "void 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)", "wasm-timer 0.1.3 (registry+https://github.com/rust-lang/crates.io-index)", @@ -1866,19 +2230,19 @@ dependencies = [ [[package]] name = "libp2p-mdns" -version = "0.12.0" +version = "0.13.0" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "data-encoding 2.1.2 (registry+https://github.com/rust-lang/crates.io-index)", "dns-parser 0.8.0 (registry+https://github.com/rust-lang/crates.io-index)", "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", - "libp2p-core 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)", - "libp2p-swarm 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)", + "libp2p-core 0.13.0 (registry+https://github.com/rust-lang/crates.io-index)", + "libp2p-swarm 0.3.0 (registry+https://github.com/rust-lang/crates.io-index)", "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", "net2 0.2.33 (registry+https://github.com/rust-lang/crates.io-index)", - "parity-multiaddr 0.5.0 (registry+https://github.com/rust-lang/crates.io-index)", + "parity-multiaddr 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)", "rand 0.6.5 (registry+https://github.com/rust-lang/crates.io-index)", - "smallvec 0.6.10 (registry+https://github.com/rust-lang/crates.io-index)", + "smallvec 0.6.13 (registry+https://github.com/rust-lang/crates.io-index)", "tokio-io 0.1.12 (registry+https://github.com/rust-lang/crates.io-index)", "tokio-reactor 0.1.10 (registry+https://github.com/rust-lang/crates.io-index)", "tokio-udp 0.1.5 (registry+https://github.com/rust-lang/crates.io-index)", @@ -1888,13 +2252,13 @@ dependencies = [ [[package]] name = "libp2p-mplex" -version = "0.12.0" +version = "0.13.0" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "bytes 0.4.12 (registry+https://github.com/rust-lang/crates.io-index)", "fnv 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", - "libp2p-core 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)", + "libp2p-core 0.13.0 (registry+https://github.com/rust-lang/crates.io-index)", "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", "parking_lot 0.8.0 (registry+https://github.com/rust-lang/crates.io-index)", "tokio-codec 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)", @@ -1904,36 +2268,36 @@ dependencies = [ [[package]] name = "libp2p-noise" -version = "0.10.0" +version = "0.11.0" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "bytes 0.4.12 (registry+https://github.com/rust-lang/crates.io-index)", "curve25519-dalek 1.2.3 (registry+https://github.com/rust-lang/crates.io-index)", "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", "lazy_static 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)", - "libp2p-core 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)", + "libp2p-core 0.13.0 (registry+https://github.com/rust-lang/crates.io-index)", "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", "protobuf 2.8.1 (registry+https://github.com/rust-lang/crates.io-index)", - "rand 0.6.5 (registry+https://github.com/rust-lang/crates.io-index)", - "ring 0.14.6 (registry+https://github.com/rust-lang/crates.io-index)", - "snow 0.5.2 (registry+https://github.com/rust-lang/crates.io-index)", + "rand 0.7.2 (registry+https://github.com/rust-lang/crates.io-index)", + "ring 0.16.9 (registry+https://github.com/rust-lang/crates.io-index)", + "snow 0.6.1 (registry+https://github.com/rust-lang/crates.io-index)", "tokio-io 0.1.12 (registry+https://github.com/rust-lang/crates.io-index)", "x25519-dalek 0.5.2 (registry+https://github.com/rust-lang/crates.io-index)", - "zeroize 0.9.3 (registry+https://github.com/rust-lang/crates.io-index)", + "zeroize 1.0.0 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] name = "libp2p-ping" -version = "0.12.0" +version = "0.13.0" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "bytes 0.4.12 (registry+https://github.com/rust-lang/crates.io-index)", "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", - "libp2p-core 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)", - "libp2p-swarm 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)", + "libp2p-core 0.13.0 (registry+https://github.com/rust-lang/crates.io-index)", + "libp2p-swarm 0.3.0 (registry+https://github.com/rust-lang/crates.io-index)", "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", - "parity-multiaddr 0.5.0 (registry+https://github.com/rust-lang/crates.io-index)", - "rand 0.6.5 (registry+https://github.com/rust-lang/crates.io-index)", + "parity-multiaddr 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)", + "rand 0.7.2 (registry+https://github.com/rust-lang/crates.io-index)", "tokio-io 0.1.12 (registry+https://github.com/rust-lang/crates.io-index)", "void 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)", "wasm-timer 0.1.3 (registry+https://github.com/rust-lang/crates.io-index)", @@ -1941,31 +2305,22 @@ dependencies = [ [[package]] name = "libp2p-plaintext" -version = "0.12.0" +version = "0.13.0" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ + "bytes 0.4.12 (registry+https://github.com/rust-lang/crates.io-index)", "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", - "libp2p-core 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)", + "libp2p-core 0.13.0 (registry+https://github.com/rust-lang/crates.io-index)", + "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", + "protobuf 2.8.1 (registry+https://github.com/rust-lang/crates.io-index)", + "rw-stream-sink 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)", + "tokio-io 0.1.12 (registry+https://github.com/rust-lang/crates.io-index)", "void 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)", ] -[[package]] -name = "libp2p-ratelimit" -version = "0.12.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -dependencies = [ - "aio-limited 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)", - "bytes 0.4.12 (registry+https://github.com/rust-lang/crates.io-index)", - "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", - "libp2p-core 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)", - "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", - "tokio-executor 0.1.8 (registry+https://github.com/rust-lang/crates.io-index)", - "tokio-io 0.1.12 (registry+https://github.com/rust-lang/crates.io-index)", -] - [[package]] name = "libp2p-secio" -version = "0.12.0" +version = "0.13.0" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "aes-ctr 0.3.0 (registry+https://github.com/rust-lang/crates.io-index)", @@ -1973,32 +2328,33 @@ dependencies = [ "ctr 0.3.2 (registry+https://github.com/rust-lang/crates.io-index)", "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", "hmac 0.7.1 (registry+https://github.com/rust-lang/crates.io-index)", - "js-sys 0.3.28 (registry+https://github.com/rust-lang/crates.io-index)", + "js-sys 0.3.31 (registry+https://github.com/rust-lang/crates.io-index)", "lazy_static 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)", - "libp2p-core 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)", + "libp2p-core 0.13.0 (registry+https://github.com/rust-lang/crates.io-index)", "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", "parity-send-wrapper 0.1.0 (registry+https://github.com/rust-lang/crates.io-index)", "protobuf 2.8.1 (registry+https://github.com/rust-lang/crates.io-index)", "rand 0.6.5 (registry+https://github.com/rust-lang/crates.io-index)", - "ring 0.14.6 (registry+https://github.com/rust-lang/crates.io-index)", + "ring 0.16.9 (registry+https://github.com/rust-lang/crates.io-index)", "rw-stream-sink 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)", "sha2 0.8.0 (registry+https://github.com/rust-lang/crates.io-index)", + "tokio-codec 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)", "tokio-io 0.1.12 (registry+https://github.com/rust-lang/crates.io-index)", "twofish 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)", - "untrusted 0.6.2 (registry+https://github.com/rust-lang/crates.io-index)", - "wasm-bindgen 0.2.51 (registry+https://github.com/rust-lang/crates.io-index)", + "untrusted 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)", + "wasm-bindgen 0.2.54 (registry+https://github.com/rust-lang/crates.io-index)", "wasm-bindgen-futures 0.3.27 (registry+https://github.com/rust-lang/crates.io-index)", - "web-sys 0.3.28 (registry+https://github.com/rust-lang/crates.io-index)", + "web-sys 0.3.31 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] name = "libp2p-swarm" -version = "0.2.0" +version = "0.3.0" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", - "libp2p-core 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)", - "smallvec 0.6.10 (registry+https://github.com/rust-lang/crates.io-index)", + "libp2p-core 0.13.0 (registry+https://github.com/rust-lang/crates.io-index)", + "smallvec 0.6.13 (registry+https://github.com/rust-lang/crates.io-index)", "tokio-io 0.1.12 (registry+https://github.com/rust-lang/crates.io-index)", "void 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)", "wasm-timer 0.1.3 (registry+https://github.com/rust-lang/crates.io-index)", @@ -2006,14 +2362,14 @@ dependencies = [ [[package]] name = "libp2p-tcp" -version = "0.12.0" +version = "0.13.0" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "bytes 0.4.12 (registry+https://github.com/rust-lang/crates.io-index)", "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", "get_if_addrs 0.5.3 (registry+https://github.com/rust-lang/crates.io-index)", - "ipnet 2.0.1 (registry+https://github.com/rust-lang/crates.io-index)", - "libp2p-core 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)", + "ipnet 2.1.0 (registry+https://github.com/rust-lang/crates.io-index)", + "libp2p-core 0.13.0 (registry+https://github.com/rust-lang/crates.io-index)", "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", "tokio-io 0.1.12 (registry+https://github.com/rust-lang/crates.io-index)", "tokio-tcp 0.1.3 (registry+https://github.com/rust-lang/crates.io-index)", @@ -2022,57 +2378,57 @@ dependencies = [ [[package]] name = "libp2p-uds" -version = "0.12.0" +version = "0.13.0" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", - "libp2p-core 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)", + "libp2p-core 0.13.0 (registry+https://github.com/rust-lang/crates.io-index)", "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", "tokio-uds 0.2.5 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] name = "libp2p-wasm-ext" -version = "0.5.0" +version = "0.6.0" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", - "js-sys 0.3.28 (registry+https://github.com/rust-lang/crates.io-index)", - "libp2p-core 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)", + "js-sys 0.3.31 (registry+https://github.com/rust-lang/crates.io-index)", + "libp2p-core 0.13.0 (registry+https://github.com/rust-lang/crates.io-index)", "parity-send-wrapper 0.1.0 (registry+https://github.com/rust-lang/crates.io-index)", "tokio-io 0.1.12 (registry+https://github.com/rust-lang/crates.io-index)", - "wasm-bindgen 0.2.51 (registry+https://github.com/rust-lang/crates.io-index)", + "wasm-bindgen 0.2.54 (registry+https://github.com/rust-lang/crates.io-index)", "wasm-bindgen-futures 0.3.27 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] name = "libp2p-websocket" -version = "0.12.0" +version = "0.13.0" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "bytes 0.4.12 (registry+https://github.com/rust-lang/crates.io-index)", "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", - "libp2p-core 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)", + "libp2p-core 0.13.0 (registry+https://github.com/rust-lang/crates.io-index)", "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", "rw-stream-sink 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)", "soketto 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)", "tokio-codec 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)", "tokio-io 0.1.12 (registry+https://github.com/rust-lang/crates.io-index)", - "tokio-rustls 0.10.0-alpha.4 (registry+https://github.com/rust-lang/crates.io-index)", - "url 1.7.2 (registry+https://github.com/rust-lang/crates.io-index)", - "webpki-roots 0.16.0 (registry+https://github.com/rust-lang/crates.io-index)", + "tokio-rustls 0.10.2 (registry+https://github.com/rust-lang/crates.io-index)", + "url 2.1.0 (registry+https://github.com/rust-lang/crates.io-index)", + "webpki-roots 0.18.0 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] name = "libp2p-yamux" -version = "0.12.0" +version = "0.13.0" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", - "libp2p-core 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)", + "libp2p-core 0.13.0 (registry+https://github.com/rust-lang/crates.io-index)", "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", "tokio-io 0.1.12 (registry+https://github.com/rust-lang/crates.io-index)", - "yamux 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)", + "yamux 0.2.2 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] @@ -2081,27 +2437,14 @@ version = "5.18.3" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "bindgen 0.47.3 (registry+https://github.com/rust-lang/crates.io-index)", - "cc 1.0.45 (registry+https://github.com/rust-lang/crates.io-index)", + "cc 1.0.47 (registry+https://github.com/rust-lang/crates.io-index)", "glob 0.2.11 (registry+https://github.com/rust-lang/crates.io-index)", - "libc 0.2.62 (registry+https://github.com/rust-lang/crates.io-index)", -] - -[[package]] -name = "libsecp256k1" -version = "0.2.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -dependencies = [ - "arrayref 0.3.5 (registry+https://github.com/rust-lang/crates.io-index)", - "digest 0.6.2 (registry+https://github.com/rust-lang/crates.io-index)", - "hmac-drbg 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)", - "rand 0.4.6 (registry+https://github.com/rust-lang/crates.io-index)", - "sha2 0.6.0 (registry+https://github.com/rust-lang/crates.io-index)", - "typenum 1.11.2 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] name = "libsecp256k1" -version = "0.3.1" +version = "0.3.2" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "arrayref 0.3.5 (registry+https://github.com/rust-lang/crates.io-index)", @@ -2118,9 +2461,9 @@ name = "libz-sys" version = "1.0.25" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "cc 1.0.45 (registry+https://github.com/rust-lang/crates.io-index)", - "libc 0.2.62 (registry+https://github.com/rust-lang/crates.io-index)", - "pkg-config 0.3.16 (registry+https://github.com/rust-lang/crates.io-index)", + "cc 1.0.47 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", + "pkg-config 0.3.17 (registry+https://github.com/rust-lang/crates.io-index)", "vcpkg 0.2.7 (registry+https://github.com/rust-lang/crates.io-index)", ] @@ -2186,14 +2529,22 @@ dependencies = [ "linked-hash-map 0.5.2 (registry+https://github.com/rust-lang/crates.io-index)", ] +[[package]] +name = "mach" +version = "0.2.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", +] + [[package]] name = "malloc_size_of_derive" -version = "0.1.0" +version = "0.1.1" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "proc-macro2 0.4.30 (registry+https://github.com/rust-lang/crates.io-index)", - "syn 0.15.44 (registry+https://github.com/rust-lang/crates.io-index)", - "synstructure 0.10.2 (registry+https://github.com/rust-lang/crates.io-index)", + "proc-macro2 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "syn 1.0.8 (registry+https://github.com/rust-lang/crates.io-index)", + "synstructure 0.12.2 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] @@ -2201,17 +2552,22 @@ name = "matches" version = "0.1.8" source = "registry+https://github.com/rust-lang/crates.io-index" +[[package]] +name = "maybe-uninit" +version = "2.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" + [[package]] name = "memchr" version = "2.2.1" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "libc 0.2.62 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] name = "memoffset" -version = "0.5.1" +version = "0.5.3" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "rustc_version 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)", @@ -2223,8 +2579,8 @@ version = "0.15.2" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "hash-db 0.15.2 (registry+https://github.com/rust-lang/crates.io-index)", - "hashbrown 0.6.1 (registry+https://github.com/rust-lang/crates.io-index)", - "parity-util-mem 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)", + "hashbrown 0.6.3 (registry+https://github.com/rust-lang/crates.io-index)", + "parity-util-mem 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] @@ -2251,9 +2607,27 @@ dependencies = [ "log 0.3.9 (registry+https://github.com/rust-lang/crates.io-index)", ] +[[package]] +name = "mime" +version = "0.3.14" +source = "registry+https://github.com/rust-lang/crates.io-index" + +[[package]] +name = "mime_guess" +version = "2.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "mime 0.3.14 (registry+https://github.com/rust-lang/crates.io-index)", +<<<<<<< HEAD + "unicase 2.5.1 (registry+https://github.com/rust-lang/crates.io-index)", +======= + "unicase 2.6.0 (registry+https://github.com/rust-lang/crates.io-index)", +>>>>>>> 9e0aa9e5d... base-code(rebased) +] + [[package]] name = "miniz_oxide" -version = "0.3.3" +version = "0.3.5" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "adler32 1.0.4 (registry+https://github.com/rust-lang/crates.io-index)", @@ -2268,7 +2642,7 @@ dependencies = [ "fuchsia-zircon-sys 0.3.3 (registry+https://github.com/rust-lang/crates.io-index)", "iovec 0.1.4 (registry+https://github.com/rust-lang/crates.io-index)", "kernel32-sys 0.2.2 (registry+https://github.com/rust-lang/crates.io-index)", - "libc 0.2.62 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", "miow 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)", "net2 0.2.33 (registry+https://github.com/rust-lang/crates.io-index)", @@ -2293,7 +2667,7 @@ version = "0.6.7" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "iovec 0.1.4 (registry+https://github.com/rust-lang/crates.io-index)", - "libc 0.2.62 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", "mio 0.6.19 (registry+https://github.com/rust-lang/crates.io-index)", ] @@ -2315,13 +2689,13 @@ source = "registry+https://github.com/rust-lang/crates.io-index" [[package]] name = "multistream-select" -version = "0.5.1" +version = "0.6.0" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "bytes 0.4.12 (registry+https://github.com/rust-lang/crates.io-index)", "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", - "smallvec 0.6.10 (registry+https://github.com/rust-lang/crates.io-index)", + "smallvec 0.6.13 (registry+https://github.com/rust-lang/crates.io-index)", "tokio-io 0.1.12 (registry+https://github.com/rust-lang/crates.io-index)", "unsigned-varint 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)", ] @@ -2340,14 +2714,14 @@ version = "0.2.3" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "lazy_static 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)", - "libc 0.2.62 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", "openssl 0.10.25 (registry+https://github.com/rust-lang/crates.io-index)", "openssl-probe 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)", - "openssl-sys 0.9.51 (registry+https://github.com/rust-lang/crates.io-index)", + "openssl-sys 0.9.52 (registry+https://github.com/rust-lang/crates.io-index)", "schannel 0.1.16 (registry+https://github.com/rust-lang/crates.io-index)", - "security-framework 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", - "security-framework-sys 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", + "security-framework 0.3.3 (registry+https://github.com/rust-lang/crates.io-index)", + "security-framework-sys 0.3.3 (registry+https://github.com/rust-lang/crates.io-index)", "tempfile 3.1.0 (registry+https://github.com/rust-lang/crates.io-index)", ] @@ -2357,7 +2731,7 @@ version = "0.2.33" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "cfg-if 0.1.10 (registry+https://github.com/rust-lang/crates.io-index)", - "libc 0.2.62 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", "winapi 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)", ] @@ -2366,10 +2740,10 @@ name = "nix" version = "0.14.1" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "bitflags 1.2.0 (registry+https://github.com/rust-lang/crates.io-index)", - "cc 1.0.45 (registry+https://github.com/rust-lang/crates.io-index)", + "bitflags 1.2.1 (registry+https://github.com/rust-lang/crates.io-index)", + "cc 1.0.47 (registry+https://github.com/rust-lang/crates.io-index)", "cfg-if 0.1.10 (registry+https://github.com/rust-lang/crates.io-index)", - "libc 0.2.62 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", "void 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)", ] @@ -2377,36 +2751,48 @@ dependencies = [ name = "node-cli" version = "2.0.0" dependencies = [ + "clear_on_drop 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)", + "console_error_panic_hook 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)", + "console_log 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)", "ctrlc 3.1.3 (registry+https://github.com/rust-lang/crates.io-index)", "exit-future 0.1.4 (registry+https://github.com/rust-lang/crates.io-index)", "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", "futures-preview 0.3.0-alpha.19 (registry+https://github.com/rust-lang/crates.io-index)", "hex-literal 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)", - "jsonrpc-core 13.2.0 (registry+https://github.com/rust-lang/crates.io-index)", + "js-sys 0.3.31 (registry+https://github.com/rust-lang/crates.io-index)", + "jsonrpc-core 14.0.3 (registry+https://github.com/rust-lang/crates.io-index)", + "kvdb-memorydb 0.1.0 (git+https://github.com/paritytech/parity-common?rev=b0317f649ab2c665b7987b8475878fc4d2e1f81d)", + "libp2p 0.13.0 (registry+https://github.com/rust-lang/crates.io-index)", "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", "node-executor 2.0.0", "node-primitives 2.0.0", "node-rpc 2.0.0", "node-runtime 2.0.0", + "paint-authority-discovery 0.1.0", + "paint-balances 2.0.0", + "paint-contracts 2.0.0", + "paint-finality-tracker 2.0.0", + "paint-im-online 0.1.0", + "paint-indices 2.0.0", + "paint-support 2.0.0", + "paint-system 2.0.0", + "paint-timestamp 2.0.0", + "paint-transaction-payment 2.0.0", "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "rand 0.6.5 (registry+https://github.com/rust-lang/crates.io-index)", "rand 0.7.2 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", "sr-io 2.0.0", "sr-primitives 2.0.0", - "srml-balances 2.0.0", - "srml-contracts 2.0.0", - "srml-finality-tracker 2.0.0", - "srml-im-online 0.1.0", - "srml-indices 2.0.0", - "srml-support 2.0.0", - "srml-system 2.0.0", - "srml-timestamp 2.0.0", - "srml-transaction-payment 2.0.0", - "structopt 0.3.3 (registry+https://github.com/rust-lang/crates.io-index)", + "structopt 0.3.4 (registry+https://github.com/rust-lang/crates.io-index)", + "substrate-authority-discovery 2.0.0", + "substrate-authority-discovery-primitives 2.0.0", "substrate-basic-authorship 2.0.0", + "substrate-build-script-utils 2.0.0", "substrate-chain-spec 2.0.0", "substrate-cli 2.0.0", "substrate-client 2.0.0", + "substrate-client-api 2.0.0", "substrate-client-db 2.0.0", "substrate-consensus-babe 2.0.0", "substrate-consensus-babe-primitives 2.0.0", @@ -2428,28 +2814,31 @@ dependencies = [ "tokio 0.1.22 (registry+https://github.com/rust-lang/crates.io-index)", "transaction-factory 0.0.1", "vergen 3.0.4 (registry+https://github.com/rust-lang/crates.io-index)", + "wasm-bindgen 0.2.54 (registry+https://github.com/rust-lang/crates.io-index)", + "wasm-bindgen-futures 0.3.27 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] name = "node-executor" version = "2.0.0" dependencies = [ + "criterion 0.3.0 (registry+https://github.com/rust-lang/crates.io-index)", "node-primitives 2.0.0", "node-runtime 2.0.0", "node-testing 2.0.0", + "paint-balances 2.0.0", + "paint-contracts 2.0.0", + "paint-grandpa 2.0.0", + "paint-indices 2.0.0", + "paint-session 2.0.0", + "paint-support 2.0.0", + "paint-system 2.0.0", + "paint-timestamp 2.0.0", + "paint-transaction-payment 2.0.0", + "paint-treasury 2.0.0", "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", "sr-io 2.0.0", "sr-primitives 2.0.0", - "srml-balances 2.0.0", - "srml-contracts 2.0.0", - "srml-grandpa 2.0.0", - "srml-indices 2.0.0", - "srml-session 2.0.0", - "srml-support 2.0.0", - "srml-system 2.0.0", - "srml-timestamp 2.0.0", - "srml-transaction-payment 2.0.0", - "srml-treasury 2.0.0", "substrate-executor 2.0.0", "substrate-primitives 2.0.0", "substrate-state-machine 2.0.0", @@ -2473,13 +2862,13 @@ dependencies = [ name = "node-rpc" version = "2.0.0" dependencies = [ - "jsonrpc-core 13.2.0 (registry+https://github.com/rust-lang/crates.io-index)", + "jsonrpc-core 14.0.3 (registry+https://github.com/rust-lang/crates.io-index)", "node-primitives 2.0.0", "node-runtime 2.0.0", + "paint-contracts-rpc 2.0.0", + "paint-system-rpc 2.0.0", + "paint-transaction-payment-rpc 2.0.0", "sr-primitives 2.0.0", - "srml-contracts-rpc 2.0.0", - "srml-system-rpc 2.0.0", - "srml-transaction-payment-rpc 2.0.0", "substrate-client 2.0.0", "substrate-transaction-pool 2.0.0", ] @@ -2488,10 +2877,10 @@ dependencies = [ name = "node-rpc-client" version = "2.0.0" dependencies = [ - "env_logger 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)", + "env_logger 0.7.1 (registry+https://github.com/rust-lang/crates.io-index)", "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", "hyper 0.12.35 (registry+https://github.com/rust-lang/crates.io-index)", - "jsonrpc-core-client 13.2.0 (registry+https://github.com/rust-lang/crates.io-index)", + "jsonrpc-core-client 14.0.3 (registry+https://github.com/rust-lang/crates.io-index)", "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", "node-primitives 2.0.0", "substrate-rpc 2.0.0", @@ -2503,50 +2892,55 @@ version = "2.0.0" dependencies = [ "integer-sqrt 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)", "node-primitives 2.0.0", + "paint-authority-discovery 0.1.0", + "paint-authorship 0.1.0", + "paint-babe 2.0.0", + "paint-balances 2.0.0", + "paint-collective 2.0.0", + "paint-contracts 2.0.0", + "paint-contracts-rpc-runtime-api 2.0.0", + "paint-democracy 2.0.0", + "paint-elections-phragmen 2.0.0", + "paint-executive 2.0.0", + "paint-finality-tracker 2.0.0", + "paint-grandpa 2.0.0", + "paint-im-online 0.1.0", + "paint-indices 2.0.0", + "paint-membership 2.0.0", + "paint-nicks 2.0.0", + "paint-offences 1.0.0", + "paint-randomness-collective-flip 2.0.0", + "paint-session 2.0.0", + "paint-staking 2.0.0", + "paint-staking-reward-curve 2.0.0", + "paint-sudo 2.0.0", + "paint-support 2.0.0", + "paint-system 2.0.0", + "paint-system-rpc-runtime-api 2.0.0", + "paint-timestamp 2.0.0", + "paint-transaction-payment 2.0.0", + "paint-transaction-payment-rpc-runtime-api 2.0.0", + "paint-treasury 2.0.0", + "paint-utility 2.0.0", "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", "rustc-hex 2.0.1 (registry+https://github.com/rust-lang/crates.io-index)", "safe-mix 1.0.0 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-api 2.0.0", "sr-io 2.0.0", "sr-primitives 2.0.0", "sr-staking-primitives 2.0.0", "sr-std 2.0.0", "sr-version 2.0.0", - "srml-authorship 0.1.0", - "srml-babe 2.0.0", - "srml-balances 2.0.0", - "srml-collective 2.0.0", - "srml-contracts 2.0.0", - "srml-contracts-rpc-runtime-api 2.0.0", - "srml-democracy 2.0.0", - "srml-elections-phragmen 2.0.0", - "srml-executive 2.0.0", - "srml-finality-tracker 2.0.0", - "srml-grandpa 2.0.0", - "srml-im-online 0.1.0", - "srml-indices 2.0.0", - "srml-membership 2.0.0", - "srml-nicks 2.0.0", - "srml-offences 1.0.0", - "srml-randomness-collective-flip 2.0.0", - "srml-session 2.0.0", - "srml-staking 2.0.0", - "srml-staking-reward-curve 2.0.0", - "srml-sudo 2.0.0", - "srml-support 2.0.0", - "srml-system 2.0.0", - "srml-system-rpc-runtime-api 2.0.0", - "srml-timestamp 2.0.0", - "srml-transaction-payment 2.0.0", - "srml-transaction-payment-rpc-runtime-api 2.0.0", - "srml-treasury 2.0.0", - "srml-utility 2.0.0", - "substrate-client 2.0.0", + "substrate-authority-discovery-primitives 2.0.0", + "substrate-block-builder-runtime-api 2.0.0", "substrate-consensus-babe-primitives 2.0.0", + "substrate-inherents 2.0.0", "substrate-keyring 2.0.0", "substrate-offchain-primitives 2.0.0", "substrate-primitives 2.0.0", "substrate-session 2.0.0", + "substrate-transaction-pool-runtime-api 2.0.0", "substrate-wasm-builder-runner 1.0.4", ] @@ -2565,6 +2959,7 @@ dependencies = [ "sr-io 2.0.0", "sr-primitives 2.0.0", "substrate-basic-authorship 2.0.0", + "substrate-build-script-utils 2.0.0", "substrate-cli 2.0.0", "substrate-client 2.0.0", "substrate-consensus-aura 2.0.0", @@ -2586,30 +2981,33 @@ dependencies = [ name = "node-template-runtime" version = "2.0.0" dependencies = [ + "paint-aura 2.0.0", + "paint-balances 2.0.0", + "paint-executive 2.0.0", + "paint-grandpa 2.0.0", + "paint-indices 2.0.0", + "paint-randomness-collective-flip 2.0.0", + "paint-sudo 2.0.0", + "paint-support 2.0.0", + "paint-system 2.0.0", + "paint-timestamp 2.0.0", + "paint-transaction-payment 2.0.0", "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", "safe-mix 1.0.0 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-api 2.0.0", "sr-io 2.0.0", "sr-primitives 2.0.0", "sr-std 2.0.0", "sr-version 2.0.0", - "srml-aura 2.0.0", - "srml-balances 2.0.0", - "srml-executive 2.0.0", - "srml-grandpa 2.0.0", - "srml-indices 2.0.0", - "srml-randomness-collective-flip 2.0.0", - "srml-sudo 2.0.0", - "srml-support 2.0.0", - "srml-system 2.0.0", - "srml-timestamp 2.0.0", - "srml-transaction-payment 2.0.0", - "substrate-client 2.0.0", + "substrate-block-builder-runtime-api 2.0.0", "substrate-consensus-aura-primitives 2.0.0", + "substrate-inherents 2.0.0", "substrate-offchain-primitives 2.0.0", "substrate-primitives 2.0.0", "substrate-session 2.0.0", - "substrate-wasm-builder-runner 1.0.4 (registry+https://github.com/rust-lang/crates.io-index)", + "substrate-transaction-pool-runtime-api 2.0.0", + "substrate-wasm-builder-runner 1.0.4", ] [[package]] @@ -2619,20 +3017,20 @@ dependencies = [ "node-executor 2.0.0", "node-primitives 2.0.0", "node-runtime 2.0.0", + "paint-balances 2.0.0", + "paint-contracts 2.0.0", + "paint-grandpa 2.0.0", + "paint-indices 2.0.0", + "paint-session 2.0.0", + "paint-staking 2.0.0", + "paint-support 2.0.0", + "paint-system 2.0.0", + "paint-timestamp 2.0.0", + "paint-transaction-payment 2.0.0", + "paint-treasury 2.0.0", "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", "sr-io 2.0.0", "sr-primitives 2.0.0", - "srml-balances 2.0.0", - "srml-contracts 2.0.0", - "srml-grandpa 2.0.0", - "srml-indices 2.0.0", - "srml-session 2.0.0", - "srml-staking 2.0.0", - "srml-support 2.0.0", - "srml-system 2.0.0", - "srml-timestamp 2.0.0", - "srml-transaction-payment 2.0.0", - "srml-treasury 2.0.0", "substrate-client 2.0.0", "substrate-executor 2.0.0", "substrate-keyring 2.0.0", @@ -2651,6 +3049,11 @@ name = "nohash-hasher" version = "0.1.2" source = "registry+https://github.com/rust-lang/crates.io-index" +[[package]] +name = "nom" +version = "1.2.4" +source = "registry+https://github.com/rust-lang/crates.io-index" + [[package]] name = "nom" version = "4.2.3" @@ -2665,7 +3068,7 @@ name = "num-bigint" version = "0.2.3" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "autocfg 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)", + "autocfg 0.1.7 (registry+https://github.com/rust-lang/crates.io-index)", "num-integer 0.1.41 (registry+https://github.com/rust-lang/crates.io-index)", "num-traits 0.2.8 (registry+https://github.com/rust-lang/crates.io-index)", ] @@ -2675,7 +3078,7 @@ name = "num-integer" version = "0.1.41" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "autocfg 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)", + "autocfg 0.1.7 (registry+https://github.com/rust-lang/crates.io-index)", "num-traits 0.2.8 (registry+https://github.com/rust-lang/crates.io-index)", ] @@ -2684,7 +3087,7 @@ name = "num-rational" version = "0.2.2" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "autocfg 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)", + "autocfg 0.1.7 (registry+https://github.com/rust-lang/crates.io-index)", "num-bigint 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)", "num-integer 0.1.41 (registry+https://github.com/rust-lang/crates.io-index)", "num-traits 0.2.8 (registry+https://github.com/rust-lang/crates.io-index)", @@ -2695,15 +3098,16 @@ name = "num-traits" version = "0.2.8" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "autocfg 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)", + "autocfg 0.1.7 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] name = "num_cpus" -version = "1.10.1" +version = "1.11.0" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "libc 0.2.62 (registry+https://github.com/rust-lang/crates.io-index)", + "hermit-abi 0.1.3 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] @@ -2738,12 +3142,12 @@ name = "openssl" version = "0.10.25" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "bitflags 1.2.0 (registry+https://github.com/rust-lang/crates.io-index)", + "bitflags 1.2.1 (registry+https://github.com/rust-lang/crates.io-index)", "cfg-if 0.1.10 (registry+https://github.com/rust-lang/crates.io-index)", "foreign-types 0.3.2 (registry+https://github.com/rust-lang/crates.io-index)", "lazy_static 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)", - "libc 0.2.62 (registry+https://github.com/rust-lang/crates.io-index)", - "openssl-sys 0.9.51 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", + "openssl-sys 0.9.52 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] @@ -2753,13 +3157,13 @@ source = "registry+https://github.com/rust-lang/crates.io-index" [[package]] name = "openssl-sys" -version = "0.9.51" +version = "0.9.52" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "autocfg 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)", - "cc 1.0.45 (registry+https://github.com/rust-lang/crates.io-index)", - "libc 0.2.62 (registry+https://github.com/rust-lang/crates.io-index)", - "pkg-config 0.3.16 (registry+https://github.com/rust-lang/crates.io-index)", + "autocfg 0.1.7 (registry+https://github.com/rust-lang/crates.io-index)", + "cc 1.0.47 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", + "pkg-config 0.3.17 (registry+https://github.com/rust-lang/crates.io-index)", "vcpkg 0.2.7 (registry+https://github.com/rust-lang/crates.io-index)", ] @@ -2780,1908 +3184,2248 @@ dependencies = [ ] [[package]] -name = "parity-bytes" -version = "0.1.0" -source = "git+https://github.com/paritytech/parity-common?rev=b0317f649ab2c665b7987b8475878fc4d2e1f81d#b0317f649ab2c665b7987b8475878fc4d2e1f81d" - -[[package]] -name = "parity-multiaddr" -version = "0.5.0" -source = "registry+https://github.com/rust-lang/crates.io-index" +name = "paint-assets" +version = "2.0.0" dependencies = [ - "arrayref 0.3.5 (registry+https://github.com/rust-lang/crates.io-index)", - "bs58 0.2.5 (registry+https://github.com/rust-lang/crates.io-index)", - "byteorder 1.3.2 (registry+https://github.com/rust-lang/crates.io-index)", - "bytes 0.4.12 (registry+https://github.com/rust-lang/crates.io-index)", - "data-encoding 2.1.2 (registry+https://github.com/rust-lang/crates.io-index)", - "parity-multihash 0.1.3 (registry+https://github.com/rust-lang/crates.io-index)", - "percent-encoding 1.0.1 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", - "unsigned-varint 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)", - "url 1.7.2 (registry+https://github.com/rust-lang/crates.io-index)", + "paint-support 2.0.0", + "paint-system 2.0.0", + "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-io 2.0.0", + "sr-primitives 2.0.0", + "sr-std 2.0.0", + "substrate-primitives 2.0.0", ] [[package]] -name = "parity-multihash" -version = "0.1.3" -source = "registry+https://github.com/rust-lang/crates.io-index" +name = "paint-aura" +version = "2.0.0" dependencies = [ - "blake2 0.8.1 (registry+https://github.com/rust-lang/crates.io-index)", - "bytes 0.4.12 (registry+https://github.com/rust-lang/crates.io-index)", - "rand 0.6.5 (registry+https://github.com/rust-lang/crates.io-index)", - "sha-1 0.8.1 (registry+https://github.com/rust-lang/crates.io-index)", - "sha2 0.8.0 (registry+https://github.com/rust-lang/crates.io-index)", - "sha3 0.8.2 (registry+https://github.com/rust-lang/crates.io-index)", - "unsigned-varint 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)", + "lazy_static 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)", + "paint-session 2.0.0", + "paint-support 2.0.0", + "paint-system 2.0.0", + "paint-timestamp 2.0.0", + "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "parking_lot 0.9.0 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-io 2.0.0", + "sr-primitives 2.0.0", + "sr-std 2.0.0", + "substrate-application-crypto 2.0.0", + "substrate-consensus-aura-primitives 2.0.0", + "substrate-inherents 2.0.0", + "substrate-primitives 2.0.0", ] [[package]] -name = "parity-scale-codec" -version = "1.0.6" -source = "registry+https://github.com/rust-lang/crates.io-index" +name = "paint-authority-discovery" +version = "0.1.0" dependencies = [ - "arrayvec 0.4.12 (registry+https://github.com/rust-lang/crates.io-index)", - "bitvec 0.14.0 (registry+https://github.com/rust-lang/crates.io-index)", - "byte-slice-cast 0.3.2 (registry+https://github.com/rust-lang/crates.io-index)", - "parity-scale-codec-derive 1.0.3 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", + "paint-session 2.0.0", + "paint-support 2.0.0", + "paint-system 2.0.0", + "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-io 2.0.0", + "sr-primitives 2.0.0", + "sr-staking-primitives 2.0.0", + "sr-std 2.0.0", + "substrate-application-crypto 2.0.0", + "substrate-authority-discovery-primitives 2.0.0", + "substrate-primitives 2.0.0", ] [[package]] -name = "parity-scale-codec-derive" -version = "1.0.3" -source = "registry+https://github.com/rust-lang/crates.io-index" +name = "paint-authorship" +version = "0.1.0" dependencies = [ - "proc-macro-crate 0.1.4 (registry+https://github.com/rust-lang/crates.io-index)", - "proc-macro2 0.4.30 (registry+https://github.com/rust-lang/crates.io-index)", - "quote 0.6.13 (registry+https://github.com/rust-lang/crates.io-index)", - "syn 0.15.44 (registry+https://github.com/rust-lang/crates.io-index)", + "impl-trait-for-tuples 0.1.3 (registry+https://github.com/rust-lang/crates.io-index)", + "paint-support 2.0.0", + "paint-system 2.0.0", + "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-io 2.0.0", + "sr-primitives 2.0.0", + "sr-std 2.0.0", + "substrate-inherents 2.0.0", + "substrate-primitives 2.0.0", ] [[package]] -name = "parity-send-wrapper" -version = "0.1.0" -source = "registry+https://github.com/rust-lang/crates.io-index" - -[[package]] -name = "parity-util-mem" -version = "0.2.0" -source = "registry+https://github.com/rust-lang/crates.io-index" +name = "paint-babe" +version = "2.0.0" dependencies = [ - "cfg-if 0.1.10 (registry+https://github.com/rust-lang/crates.io-index)", - "malloc_size_of_derive 0.1.0 (registry+https://github.com/rust-lang/crates.io-index)", - "winapi 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)", + "hex-literal 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)", + "lazy_static 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)", + "paint-session 2.0.0", + "paint-support 2.0.0", + "paint-system 2.0.0", + "paint-timestamp 2.0.0", + "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "parking_lot 0.9.0 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-io 2.0.0", + "sr-primitives 2.0.0", + "sr-staking-primitives 2.0.0", + "sr-std 2.0.0", + "sr-version 2.0.0", + "substrate-consensus-babe-primitives 2.0.0", + "substrate-inherents 2.0.0", + "substrate-primitives 2.0.0", + "substrate-test-runtime 2.0.0", ] [[package]] -name = "parity-wasm" -version = "0.32.0" -source = "registry+https://github.com/rust-lang/crates.io-index" +name = "paint-balances" +version = "2.0.0" dependencies = [ - "byteorder 1.3.2 (registry+https://github.com/rust-lang/crates.io-index)", + "paint-support 2.0.0", + "paint-system 2.0.0", + "paint-transaction-payment 2.0.0", + "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "safe-mix 1.0.0 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-io 2.0.0", + "sr-primitives 2.0.0", + "sr-std 2.0.0", + "substrate-keyring 2.0.0", + "substrate-primitives 2.0.0", ] [[package]] -name = "parity-wasm" -version = "0.40.3" -source = "registry+https://github.com/rust-lang/crates.io-index" - -[[package]] -name = "parking_lot" -version = "0.6.4" -source = "registry+https://github.com/rust-lang/crates.io-index" +name = "paint-collective" +version = "2.0.0" dependencies = [ - "lock_api 0.1.5 (registry+https://github.com/rust-lang/crates.io-index)", - "parking_lot_core 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", + "hex-literal 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)", + "paint-balances 2.0.0", + "paint-support 2.0.0", + "paint-system 2.0.0", + "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "safe-mix 1.0.0 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-io 2.0.0", + "sr-primitives 2.0.0", + "sr-std 2.0.0", + "substrate-primitives 2.0.0", ] [[package]] -name = "parking_lot" -version = "0.7.1" -source = "registry+https://github.com/rust-lang/crates.io-index" +name = "paint-contracts" +version = "2.0.0" dependencies = [ - "lock_api 0.1.5 (registry+https://github.com/rust-lang/crates.io-index)", - "parking_lot_core 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)", + "assert_matches 1.3.0 (registry+https://github.com/rust-lang/crates.io-index)", + "hex 0.3.2 (registry+https://github.com/rust-lang/crates.io-index)", + "hex-literal 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)", + "paint-balances 2.0.0", + "paint-randomness-collective-flip 2.0.0", + "paint-support 2.0.0", + "paint-system 2.0.0", + "paint-timestamp 2.0.0", + "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "parity-wasm 0.40.3 (registry+https://github.com/rust-lang/crates.io-index)", + "pwasm-utils 0.11.0 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-io 2.0.0", + "sr-primitives 2.0.0", + "sr-sandbox 2.0.0", + "sr-std 2.0.0", + "substrate-primitives 2.0.0", + "wabt 0.9.2 (registry+https://github.com/rust-lang/crates.io-index)", + "wasmi-validation 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "parking_lot" -version = "0.8.0" -source = "registry+https://github.com/rust-lang/crates.io-index" +name = "paint-contracts-rpc" +version = "2.0.0" dependencies = [ - "lock_api 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)", - "parking_lot_core 0.5.0 (registry+https://github.com/rust-lang/crates.io-index)", - "rustc_version 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)", + "jsonrpc-core 14.0.3 (registry+https://github.com/rust-lang/crates.io-index)", + "jsonrpc-core-client 14.0.3 (registry+https://github.com/rust-lang/crates.io-index)", + "jsonrpc-derive 14.0.3 (registry+https://github.com/rust-lang/crates.io-index)", + "paint-contracts-rpc-runtime-api 2.0.0", + "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-primitives 2.0.0", + "substrate-client 2.0.0", + "substrate-primitives 2.0.0", + "substrate-rpc-primitives 2.0.0", ] [[package]] -name = "parking_lot" -version = "0.9.0" -source = "registry+https://github.com/rust-lang/crates.io-index" +name = "paint-contracts-rpc-runtime-api" +version = "2.0.0" dependencies = [ - "lock_api 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", - "parking_lot_core 0.6.2 (registry+https://github.com/rust-lang/crates.io-index)", - "rustc_version 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)", + "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-api 2.0.0", + "sr-primitives 2.0.0", + "sr-std 2.0.0", ] [[package]] -name = "parking_lot_core" -version = "0.3.1" -source = "registry+https://github.com/rust-lang/crates.io-index" +name = "paint-democracy" +version = "2.0.0" dependencies = [ - "libc 0.2.62 (registry+https://github.com/rust-lang/crates.io-index)", - "rand 0.5.6 (registry+https://github.com/rust-lang/crates.io-index)", - "rustc_version 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)", - "smallvec 0.6.10 (registry+https://github.com/rust-lang/crates.io-index)", - "winapi 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)", + "paint-balances 2.0.0", + "paint-support 2.0.0", + "paint-system 2.0.0", + "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "safe-mix 1.0.0 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-io 2.0.0", + "sr-primitives 2.0.0", + "sr-std 2.0.0", + "substrate-primitives 2.0.0", ] [[package]] -name = "parking_lot_core" -version = "0.4.0" -source = "registry+https://github.com/rust-lang/crates.io-index" +name = "paint-elections" +version = "2.0.0" dependencies = [ - "libc 0.2.62 (registry+https://github.com/rust-lang/crates.io-index)", - "rand 0.6.5 (registry+https://github.com/rust-lang/crates.io-index)", - "rustc_version 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)", - "smallvec 0.6.10 (registry+https://github.com/rust-lang/crates.io-index)", - "winapi 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)", + "hex-literal 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)", + "paint-balances 2.0.0", + "paint-support 2.0.0", + "paint-system 2.0.0", + "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "safe-mix 1.0.0 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-io 2.0.0", + "sr-primitives 2.0.0", + "sr-std 2.0.0", + "substrate-primitives 2.0.0", ] [[package]] -name = "parking_lot_core" -version = "0.5.0" -source = "registry+https://github.com/rust-lang/crates.io-index" +name = "paint-elections-phragmen" +version = "2.0.0" dependencies = [ - "cfg-if 0.1.10 (registry+https://github.com/rust-lang/crates.io-index)", - "cloudabi 0.0.3 (registry+https://github.com/rust-lang/crates.io-index)", - "libc 0.2.62 (registry+https://github.com/rust-lang/crates.io-index)", - "rand 0.6.5 (registry+https://github.com/rust-lang/crates.io-index)", - "redox_syscall 0.1.56 (registry+https://github.com/rust-lang/crates.io-index)", - "rustc_version 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)", - "smallvec 0.6.10 (registry+https://github.com/rust-lang/crates.io-index)", - "winapi 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)", + "hex-literal 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)", + "paint-balances 2.0.0", + "paint-support 2.0.0", + "paint-system 2.0.0", + "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-io 2.0.0", + "sr-primitives 2.0.0", + "sr-std 2.0.0", + "substrate-phragmen 2.0.0", + "substrate-primitives 2.0.0", ] [[package]] -name = "parking_lot_core" -version = "0.6.2" -source = "registry+https://github.com/rust-lang/crates.io-index" +name = "paint-evm" +version = "2.0.0" dependencies = [ - "cfg-if 0.1.10 (registry+https://github.com/rust-lang/crates.io-index)", - "cloudabi 0.0.3 (registry+https://github.com/rust-lang/crates.io-index)", - "libc 0.2.62 (registry+https://github.com/rust-lang/crates.io-index)", - "redox_syscall 0.1.56 (registry+https://github.com/rust-lang/crates.io-index)", - "rustc_version 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)", - "smallvec 0.6.10 (registry+https://github.com/rust-lang/crates.io-index)", - "winapi 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)", + "evm 0.14.0 (registry+https://github.com/rust-lang/crates.io-index)", + "paint-balances 2.0.0", + "paint-support 2.0.0", + "paint-system 2.0.0", + "paint-timestamp 2.0.0", + "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "primitive-types 0.6.1 (registry+https://github.com/rust-lang/crates.io-index)", + "rlp 0.4.3 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "sha3 0.8.2 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-io 2.0.0", + "sr-primitives 2.0.0", + "sr-std 2.0.0", + "substrate-primitives 2.0.0", ] [[package]] -name = "paste" -version = "0.1.6" -source = "registry+https://github.com/rust-lang/crates.io-index" +name = "paint-example" +version = "2.0.0" dependencies = [ - "paste-impl 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)", - "proc-macro-hack 0.5.10 (registry+https://github.com/rust-lang/crates.io-index)", + "paint-balances 2.0.0", + "paint-support 2.0.0", + "paint-system 2.0.0", + "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-io 2.0.0", + "sr-primitives 2.0.0", + "sr-std 2.0.0", + "substrate-primitives 2.0.0", ] [[package]] -name = "paste-impl" -version = "0.1.6" -source = "registry+https://github.com/rust-lang/crates.io-index" +name = "paint-executive" +version = "2.0.0" dependencies = [ - "proc-macro-hack 0.5.10 (registry+https://github.com/rust-lang/crates.io-index)", - "proc-macro2 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "quote 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)", - "syn 1.0.7 (registry+https://github.com/rust-lang/crates.io-index)", + "hex-literal 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)", + "paint-balances 2.0.0", + "paint-indices 2.0.0", + "paint-support 2.0.0", + "paint-system 2.0.0", + "paint-transaction-payment 2.0.0", + "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-io 2.0.0", + "sr-primitives 2.0.0", + "sr-std 2.0.0", + "substrate-primitives 2.0.0", ] [[package]] -name = "pbkdf2" -version = "0.3.0" -source = "registry+https://github.com/rust-lang/crates.io-index" +name = "paint-finality-tracker" +version = "2.0.0" dependencies = [ - "byteorder 1.3.2 (registry+https://github.com/rust-lang/crates.io-index)", - "crypto-mac 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)", + "impl-trait-for-tuples 0.1.3 (registry+https://github.com/rust-lang/crates.io-index)", + "paint-support 2.0.0", + "paint-system 2.0.0", + "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-io 2.0.0", + "sr-primitives 2.0.0", + "sr-std 2.0.0", + "substrate-inherents 2.0.0", + "substrate-primitives 2.0.0", ] [[package]] -name = "pdqselect" -version = "0.1.0" -source = "registry+https://github.com/rust-lang/crates.io-index" - -[[package]] -name = "peeking_take_while" -version = "0.1.2" -source = "registry+https://github.com/rust-lang/crates.io-index" - -[[package]] -name = "percent-encoding" -version = "1.0.1" -source = "registry+https://github.com/rust-lang/crates.io-index" - -[[package]] -name = "percent-encoding" -version = "2.1.0" -source = "registry+https://github.com/rust-lang/crates.io-index" - -[[package]] -name = "petgraph" -version = "0.4.13" -source = "registry+https://github.com/rust-lang/crates.io-index" +name = "paint-generic-asset" +version = "2.0.0" dependencies = [ - "fixedbitset 0.1.9 (registry+https://github.com/rust-lang/crates.io-index)", + "paint-support 2.0.0", + "paint-system 2.0.0", + "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-io 2.0.0", + "sr-primitives 2.0.0", + "sr-std 2.0.0", + "substrate-primitives 2.0.0", ] [[package]] -name = "pin-utils" -version = "0.1.0-alpha.4" -source = "registry+https://github.com/rust-lang/crates.io-index" - -[[package]] -name = "pkg-config" -version = "0.3.16" -source = "registry+https://github.com/rust-lang/crates.io-index" - -[[package]] -name = "ppv-lite86" -version = "0.2.5" -source = "registry+https://github.com/rust-lang/crates.io-index" - -[[package]] -name = "pretty_assertions" -version = "0.6.1" -source = "registry+https://github.com/rust-lang/crates.io-index" +name = "paint-grandpa" +version = "2.0.0" dependencies = [ - "ansi_term 0.11.0 (registry+https://github.com/rust-lang/crates.io-index)", - "ctor 0.1.12 (registry+https://github.com/rust-lang/crates.io-index)", - "difference 2.0.0 (registry+https://github.com/rust-lang/crates.io-index)", - "output_vt100 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)", + "paint-finality-tracker 2.0.0", + "paint-session 2.0.0", + "paint-support 2.0.0", + "paint-system 2.0.0", + "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-io 2.0.0", + "sr-primitives 2.0.0", + "sr-staking-primitives 2.0.0", + "sr-std 2.0.0", + "substrate-finality-grandpa-primitives 2.0.0", + "substrate-primitives 2.0.0", ] [[package]] -name = "primitive-types" -version = "0.5.1" -source = "registry+https://github.com/rust-lang/crates.io-index" +name = "paint-im-online" +version = "0.1.0" dependencies = [ - "fixed-hash 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)", - "impl-codec 0.4.1 (registry+https://github.com/rust-lang/crates.io-index)", - "impl-serde 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)", - "uint 0.8.1 (registry+https://github.com/rust-lang/crates.io-index)", + "paint-authorship 0.1.0", + "paint-session 2.0.0", + "paint-support 2.0.0", + "paint-system 2.0.0", + "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-io 2.0.0", + "sr-primitives 2.0.0", + "sr-staking-primitives 2.0.0", + "sr-std 2.0.0", + "substrate-application-crypto 2.0.0", + "substrate-offchain 2.0.0", + "substrate-primitives 2.0.0", ] [[package]] -name = "primitive-types" -version = "0.6.0" -source = "registry+https://github.com/rust-lang/crates.io-index" +name = "paint-indices" +version = "2.0.0" dependencies = [ - "fixed-hash 0.5.0 (registry+https://github.com/rust-lang/crates.io-index)", - "impl-codec 0.4.1 (registry+https://github.com/rust-lang/crates.io-index)", - "uint 0.8.1 (registry+https://github.com/rust-lang/crates.io-index)", + "paint-support 2.0.0", + "paint-system 2.0.0", + "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "ref_thread_local 0.0.0 (registry+https://github.com/rust-lang/crates.io-index)", + "safe-mix 1.0.0 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-io 2.0.0", + "sr-primitives 2.0.0", + "sr-std 2.0.0", + "substrate-keyring 2.0.0", + "substrate-primitives 2.0.0", ] [[package]] -name = "proc-macro-crate" -version = "0.1.4" -source = "registry+https://github.com/rust-lang/crates.io-index" +name = "paint-membership" +version = "2.0.0" dependencies = [ - "toml 0.5.3 (registry+https://github.com/rust-lang/crates.io-index)", + "paint-support 2.0.0", + "paint-system 2.0.0", + "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-io 2.0.0", + "sr-primitives 2.0.0", + "sr-std 2.0.0", + "substrate-primitives 2.0.0", ] [[package]] -name = "proc-macro-error" -version = "0.2.6" -source = "registry+https://github.com/rust-lang/crates.io-index" +name = "paint-metadata" +version = "2.0.0" dependencies = [ - "proc-macro2 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "quote 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)", - "syn 1.0.7 (registry+https://github.com/rust-lang/crates.io-index)", + "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-std 2.0.0", + "substrate-primitives 2.0.0", ] [[package]] -name = "proc-macro-hack" -version = "0.5.10" -source = "registry+https://github.com/rust-lang/crates.io-index" +name = "paint-nicks" +version = "2.0.0" dependencies = [ - "proc-macro2 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "quote 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)", - "syn 1.0.7 (registry+https://github.com/rust-lang/crates.io-index)", + "paint-balances 2.0.0", + "paint-support 2.0.0", + "paint-system 2.0.0", + "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-io 2.0.0", + "sr-primitives 2.0.0", + "sr-std 2.0.0", + "substrate-primitives 2.0.0", ] [[package]] -name = "proc-macro2" -version = "0.4.30" -source = "registry+https://github.com/rust-lang/crates.io-index" +name = "paint-offences" +version = "1.0.0" dependencies = [ - "unicode-xid 0.1.0 (registry+https://github.com/rust-lang/crates.io-index)", + "paint-balances 2.0.0", + "paint-support 2.0.0", + "paint-system 2.0.0", + "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-io 2.0.0", + "sr-primitives 2.0.0", + "sr-staking-primitives 2.0.0", + "sr-std 2.0.0", + "substrate-primitives 2.0.0", ] [[package]] -name = "proc-macro2" -version = "1.0.6" -source = "registry+https://github.com/rust-lang/crates.io-index" +name = "paint-randomness-collective-flip" +version = "2.0.0" dependencies = [ - "unicode-xid 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)", + "paint-support 2.0.0", + "paint-system 2.0.0", + "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "safe-mix 1.0.0 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-io 2.0.0", + "sr-primitives 2.0.0", + "sr-std 2.0.0", + "substrate-primitives 2.0.0", ] [[package]] -name = "prost" -version = "0.5.0" -source = "registry+https://github.com/rust-lang/crates.io-index" +name = "paint-scored-pool" +version = "1.0.0" dependencies = [ - "byteorder 1.3.2 (registry+https://github.com/rust-lang/crates.io-index)", - "bytes 0.4.12 (registry+https://github.com/rust-lang/crates.io-index)", - "prost-derive 0.5.0 (registry+https://github.com/rust-lang/crates.io-index)", + "paint-balances 2.0.0", + "paint-support 2.0.0", + "paint-system 2.0.0", + "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-io 2.0.0", + "sr-primitives 2.0.0", + "sr-std 2.0.0", + "substrate-primitives 2.0.0", ] [[package]] -name = "prost-build" -version = "0.5.0" -source = "registry+https://github.com/rust-lang/crates.io-index" +name = "paint-session" +version = "2.0.0" dependencies = [ - "bytes 0.4.12 (registry+https://github.com/rust-lang/crates.io-index)", - "heck 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", - "itertools 0.8.0 (registry+https://github.com/rust-lang/crates.io-index)", - "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", - "multimap 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)", - "petgraph 0.4.13 (registry+https://github.com/rust-lang/crates.io-index)", - "prost 0.5.0 (registry+https://github.com/rust-lang/crates.io-index)", - "prost-types 0.5.0 (registry+https://github.com/rust-lang/crates.io-index)", - "tempfile 3.1.0 (registry+https://github.com/rust-lang/crates.io-index)", - "which 2.0.1 (registry+https://github.com/rust-lang/crates.io-index)", + "impl-trait-for-tuples 0.1.3 (registry+https://github.com/rust-lang/crates.io-index)", + "lazy_static 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)", + "paint-support 2.0.0", + "paint-system 2.0.0", + "paint-timestamp 2.0.0", + "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "safe-mix 1.0.0 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-io 2.0.0", + "sr-primitives 2.0.0", + "sr-staking-primitives 2.0.0", + "sr-std 2.0.0", + "substrate-application-crypto 2.0.0", + "substrate-primitives 2.0.0", + "substrate-trie 2.0.0", ] [[package]] -name = "prost-derive" -version = "0.5.0" -source = "registry+https://github.com/rust-lang/crates.io-index" +name = "paint-staking" +version = "2.0.0" dependencies = [ - "failure 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)", - "itertools 0.8.0 (registry+https://github.com/rust-lang/crates.io-index)", - "proc-macro2 0.4.30 (registry+https://github.com/rust-lang/crates.io-index)", - "quote 0.6.13 (registry+https://github.com/rust-lang/crates.io-index)", - "syn 0.15.44 (registry+https://github.com/rust-lang/crates.io-index)", + "paint-authorship 0.1.0", + "paint-balances 2.0.0", + "paint-session 2.0.0", + "paint-staking-reward-curve 2.0.0", + "paint-support 2.0.0", + "paint-system 2.0.0", + "paint-timestamp 2.0.0", + "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "safe-mix 1.0.0 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-io 2.0.0", + "sr-primitives 2.0.0", + "sr-staking-primitives 2.0.0", + "sr-std 2.0.0", + "substrate-keyring 2.0.0", + "substrate-phragmen 2.0.0", + "substrate-primitives 2.0.0", ] [[package]] -name = "prost-types" -version = "0.5.0" -source = "registry+https://github.com/rust-lang/crates.io-index" +name = "paint-staking-reward-curve" +version = "2.0.0" dependencies = [ - "bytes 0.4.12 (registry+https://github.com/rust-lang/crates.io-index)", - "prost 0.5.0 (registry+https://github.com/rust-lang/crates.io-index)", + "proc-macro-crate 0.1.4 (registry+https://github.com/rust-lang/crates.io-index)", + "proc-macro2 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "quote 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-primitives 2.0.0", + "syn 1.0.8 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "protobuf" -version = "2.8.1" -source = "registry+https://github.com/rust-lang/crates.io-index" - -[[package]] -name = "pwasm-utils" -version = "0.11.0" -source = "registry+https://github.com/rust-lang/crates.io-index" +name = "paint-sudo" +version = "2.0.0" dependencies = [ - "byteorder 1.3.2 (registry+https://github.com/rust-lang/crates.io-index)", - "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", - "parity-wasm 0.40.3 (registry+https://github.com/rust-lang/crates.io-index)", + "paint-support 2.0.0", + "paint-system 2.0.0", + "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-io 2.0.0", + "sr-primitives 2.0.0", + "sr-std 2.0.0", + "substrate-primitives 2.0.0", ] [[package]] -name = "quick-error" -version = "0.1.4" -source = "registry+https://github.com/rust-lang/crates.io-index" - -[[package]] -name = "quick-error" -version = "1.2.2" -source = "registry+https://github.com/rust-lang/crates.io-index" - -[[package]] -name = "quickcheck" -version = "0.9.0" -source = "registry+https://github.com/rust-lang/crates.io-index" +name = "paint-support" +version = "2.0.0" dependencies = [ - "env_logger 0.6.2 (registry+https://github.com/rust-lang/crates.io-index)", + "bitmask 0.5.0 (registry+https://github.com/rust-lang/crates.io-index)", + "impl-trait-for-tuples 0.1.3 (registry+https://github.com/rust-lang/crates.io-index)", "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", - "rand 0.7.2 (registry+https://github.com/rust-lang/crates.io-index)", - "rand_core 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)", + "once_cell 0.2.4 (registry+https://github.com/rust-lang/crates.io-index)", + "paint-metadata 2.0.0", + "paint-support-procedural 2.0.0", + "paint-system 2.0.0", + "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "paste 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)", + "pretty_assertions 0.6.1 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-io 2.0.0", + "sr-primitives 2.0.0", + "sr-std 2.0.0", + "substrate-inherents 2.0.0", + "substrate-primitives 2.0.0", + "substrate-state-machine 2.0.0", ] [[package]] -name = "quote" -version = "0.6.13" -source = "registry+https://github.com/rust-lang/crates.io-index" +name = "paint-support-procedural" +version = "2.0.0" dependencies = [ - "proc-macro2 0.4.30 (registry+https://github.com/rust-lang/crates.io-index)", + "paint-support-procedural-tools 2.0.0", + "proc-macro2 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "quote 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)", + "syn 1.0.8 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "quote" -version = "1.0.2" -source = "registry+https://github.com/rust-lang/crates.io-index" +name = "paint-support-procedural-tools" +version = "2.0.0" dependencies = [ + "paint-support-procedural-tools-derive 2.0.0", + "proc-macro-crate 0.1.4 (registry+https://github.com/rust-lang/crates.io-index)", "proc-macro2 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "quote 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)", + "syn 1.0.8 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "rand" -version = "0.3.23" -source = "registry+https://github.com/rust-lang/crates.io-index" +name = "paint-support-procedural-tools-derive" +version = "2.0.0" dependencies = [ - "libc 0.2.62 (registry+https://github.com/rust-lang/crates.io-index)", - "rand 0.4.6 (registry+https://github.com/rust-lang/crates.io-index)", + "proc-macro2 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "quote 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)", + "syn 1.0.8 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "rand" -version = "0.4.6" -source = "registry+https://github.com/rust-lang/crates.io-index" +name = "paint-support-rpc" +version = "2.0.0" dependencies = [ - "fuchsia-cprng 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)", - "libc 0.2.62 (registry+https://github.com/rust-lang/crates.io-index)", - "rand_core 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", - "rdrand 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)", - "winapi 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)", + "futures 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", + "jsonrpc-client-transports 14.0.3 (registry+https://github.com/rust-lang/crates.io-index)", + "jsonrpc-core 14.0.3 (registry+https://github.com/rust-lang/crates.io-index)", + "paint-support 2.0.0", + "paint-system 2.0.0", + "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "substrate-primitives-storage 2.0.0", + "substrate-rpc-api 2.0.0", + "tokio 0.1.22 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "rand" -version = "0.5.6" -source = "registry+https://github.com/rust-lang/crates.io-index" +name = "paint-support-test" +version = "2.0.0" dependencies = [ - "cloudabi 0.0.3 (registry+https://github.com/rust-lang/crates.io-index)", - "fuchsia-cprng 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)", - "libc 0.2.62 (registry+https://github.com/rust-lang/crates.io-index)", - "rand_core 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", - "winapi 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)", + "paint-support 2.0.0", + "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "pretty_assertions 0.6.1 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-io 2.0.0", + "sr-primitives 2.0.0", + "substrate-inherents 2.0.0", + "substrate-primitives 2.0.0", + "substrate-state-machine 2.0.0", + "trybuild 1.0.17 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "rand" -version = "0.6.5" +<<<<<<< HEAD +name = "procinfo" +version = "0.3.1" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "autocfg 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)", + "byteorder 0.5.3 (registry+https://github.com/rust-lang/crates.io-index)", "libc 0.2.62 (registry+https://github.com/rust-lang/crates.io-index)", - "rand_chacha 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)", - "rand_core 0.4.2 (registry+https://github.com/rust-lang/crates.io-index)", - "rand_hc 0.1.0 (registry+https://github.com/rust-lang/crates.io-index)", - "rand_isaac 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)", - "rand_jitter 0.1.4 (registry+https://github.com/rust-lang/crates.io-index)", - "rand_os 0.1.3 (registry+https://github.com/rust-lang/crates.io-index)", - "rand_pcg 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)", - "rand_xorshift 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)", - "winapi 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)", + "nom 1.2.4 (registry+https://github.com/rust-lang/crates.io-index)", + "rustc_version 0.1.7 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "rand" -version = "0.7.2" +name = "prometheus" +version = "0.7.0" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "getrandom 0.1.12 (registry+https://github.com/rust-lang/crates.io-index)", + "cfg-if 0.1.10 (registry+https://github.com/rust-lang/crates.io-index)", + "fnv 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "lazy_static 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)", "libc 0.2.62 (registry+https://github.com/rust-lang/crates.io-index)", - "rand_chacha 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)", - "rand_core 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)", - "rand_hc 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)", + "procinfo 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", + "protobuf 2.8.1 (registry+https://github.com/rust-lang/crates.io-index)", + "quick-error 1.2.2 (registry+https://github.com/rust-lang/crates.io-index)", + "spin 0.5.2 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "rand_chacha" -version = "0.1.1" +name = "prost" +version = "0.5.0" source = "registry+https://github.com/rust-lang/crates.io-index" +======= +name = "paint-system" +version = "2.0.0" +>>>>>>> f44873dce... Reorganising the repository - external renames and moves (#4074) dependencies = [ - "autocfg 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)", - "rand_core 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", + "criterion 0.2.11 (registry+https://github.com/rust-lang/crates.io-index)", + "impl-trait-for-tuples 0.1.3 (registry+https://github.com/rust-lang/crates.io-index)", + "paint-support 2.0.0", + "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "safe-mix 1.0.0 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-io 2.0.0", + "sr-primitives 2.0.0", + "sr-std 2.0.0", + "sr-version 2.0.0", + "substrate-primitives 2.0.0", ] [[package]] -name = "rand_chacha" -version = "0.2.1" -source = "registry+https://github.com/rust-lang/crates.io-index" +name = "paint-system-rpc" +version = "2.0.0" dependencies = [ - "c2-chacha 0.2.2 (registry+https://github.com/rust-lang/crates.io-index)", - "rand_core 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)", + "env_logger 0.7.1 (registry+https://github.com/rust-lang/crates.io-index)", + "futures-preview 0.3.0-alpha.19 (registry+https://github.com/rust-lang/crates.io-index)", + "jsonrpc-core 14.0.3 (registry+https://github.com/rust-lang/crates.io-index)", + "jsonrpc-core-client 14.0.3 (registry+https://github.com/rust-lang/crates.io-index)", + "jsonrpc-derive 14.0.3 (registry+https://github.com/rust-lang/crates.io-index)", + "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", + "paint-system-rpc-runtime-api 2.0.0", + "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-primitives 2.0.0", + "substrate-client 2.0.0", + "substrate-primitives 2.0.0", + "substrate-test-runtime-client 2.0.0", + "substrate-transaction-pool 2.0.0", ] [[package]] -name = "rand_core" -version = "0.3.1" -source = "registry+https://github.com/rust-lang/crates.io-index" +name = "paint-system-rpc-runtime-api" +version = "2.0.0" dependencies = [ - "rand_core 0.4.2 (registry+https://github.com/rust-lang/crates.io-index)", + "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-api 2.0.0", ] [[package]] -name = "rand_core" -version = "0.4.2" -source = "registry+https://github.com/rust-lang/crates.io-index" +name = "paint-timestamp" +version = "2.0.0" +dependencies = [ + "impl-trait-for-tuples 0.1.3 (registry+https://github.com/rust-lang/crates.io-index)", + "paint-support 2.0.0", + "paint-system 2.0.0", + "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-io 2.0.0", + "sr-primitives 2.0.0", + "sr-std 2.0.0", + "substrate-inherents 2.0.0", + "substrate-primitives 2.0.0", +] [[package]] -name = "rand_core" -version = "0.5.1" -source = "registry+https://github.com/rust-lang/crates.io-index" +name = "paint-transaction-payment" +version = "2.0.0" dependencies = [ - "getrandom 0.1.12 (registry+https://github.com/rust-lang/crates.io-index)", + "paint-balances 2.0.0", + "paint-support 2.0.0", + "paint-system 2.0.0", + "paint-transaction-payment-rpc-runtime-api 2.0.0", + "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-io 2.0.0", + "sr-primitives 2.0.0", + "sr-std 2.0.0", + "substrate-primitives 2.0.0", ] [[package]] -name = "rand_hc" -version = "0.1.0" +<<<<<<< HEAD +name = "publicsuffix" +version = "1.5.3" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "rand_core 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", + "error-chain 0.12.1 (registry+https://github.com/rust-lang/crates.io-index)", + "idna 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)", + "lazy_static 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)", + "regex 1.3.1 (registry+https://github.com/rust-lang/crates.io-index)", + "url 2.1.0 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "rand_hc" -version = "0.2.0" +name = "pwasm-utils" +version = "0.11.0" source = "registry+https://github.com/rust-lang/crates.io-index" +======= +name = "paint-transaction-payment-rpc" +version = "2.0.0" +>>>>>>> f44873dce... Reorganising the repository - external renames and moves (#4074) dependencies = [ - "rand_core 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)", + "jsonrpc-core 14.0.3 (registry+https://github.com/rust-lang/crates.io-index)", + "jsonrpc-core-client 14.0.3 (registry+https://github.com/rust-lang/crates.io-index)", + "jsonrpc-derive 14.0.3 (registry+https://github.com/rust-lang/crates.io-index)", + "paint-transaction-payment-rpc-runtime-api 2.0.0", + "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-primitives 2.0.0", + "substrate-client 2.0.0", + "substrate-primitives 2.0.0", + "substrate-rpc-primitives 2.0.0", ] [[package]] -name = "rand_isaac" -version = "0.1.1" -source = "registry+https://github.com/rust-lang/crates.io-index" +name = "paint-transaction-payment-rpc-runtime-api" +version = "2.0.0" dependencies = [ - "rand_core 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", + "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-api 2.0.0", + "sr-primitives 2.0.0", + "sr-std 2.0.0", ] [[package]] -name = "rand_jitter" -version = "0.1.4" -source = "registry+https://github.com/rust-lang/crates.io-index" +name = "paint-treasury" +version = "2.0.0" dependencies = [ - "libc 0.2.62 (registry+https://github.com/rust-lang/crates.io-index)", - "rand_core 0.4.2 (registry+https://github.com/rust-lang/crates.io-index)", - "winapi 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)", + "paint-balances 2.0.0", + "paint-support 2.0.0", + "paint-system 2.0.0", + "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-io 2.0.0", + "sr-primitives 2.0.0", + "sr-std 2.0.0", + "substrate-primitives 2.0.0", ] [[package]] -name = "rand_os" -version = "0.1.3" -source = "registry+https://github.com/rust-lang/crates.io-index" +name = "paint-utility" +version = "2.0.0" dependencies = [ - "cloudabi 0.0.3 (registry+https://github.com/rust-lang/crates.io-index)", - "fuchsia-cprng 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)", - "libc 0.2.62 (registry+https://github.com/rust-lang/crates.io-index)", - "rand_core 0.4.2 (registry+https://github.com/rust-lang/crates.io-index)", - "rdrand 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)", - "winapi 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)", + "paint-balances 2.0.0", + "paint-support 2.0.0", + "paint-system 2.0.0", + "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-io 2.0.0", + "sr-primitives 2.0.0", + "sr-std 2.0.0", + "substrate-primitives 2.0.0", ] [[package]] -name = "rand_os" -version = "0.2.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -dependencies = [ - "getrandom 0.1.12 (registry+https://github.com/rust-lang/crates.io-index)", - "rand_core 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)", -] +name = "parity-bytes" +version = "0.1.0" +source = "git+https://github.com/paritytech/parity-common?rev=b0317f649ab2c665b7987b8475878fc4d2e1f81d#b0317f649ab2c665b7987b8475878fc4d2e1f81d" [[package]] -name = "rand_pcg" -version = "0.1.2" +name = "parity-multiaddr" +version = "0.5.1" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "autocfg 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)", - "rand_core 0.4.2 (registry+https://github.com/rust-lang/crates.io-index)", + "arrayref 0.3.5 (registry+https://github.com/rust-lang/crates.io-index)", + "bs58 0.3.0 (registry+https://github.com/rust-lang/crates.io-index)", + "byteorder 1.3.2 (registry+https://github.com/rust-lang/crates.io-index)", + "bytes 0.4.12 (registry+https://github.com/rust-lang/crates.io-index)", + "data-encoding 2.1.2 (registry+https://github.com/rust-lang/crates.io-index)", + "parity-multihash 0.1.4 (registry+https://github.com/rust-lang/crates.io-index)", + "percent-encoding 2.1.0 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "unsigned-varint 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)", + "url 2.1.0 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "rand_xorshift" -version = "0.1.1" +name = "parity-multihash" +version = "0.1.4" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "rand_core 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", + "blake2 0.8.1 (registry+https://github.com/rust-lang/crates.io-index)", + "bytes 0.4.12 (registry+https://github.com/rust-lang/crates.io-index)", + "rand 0.6.5 (registry+https://github.com/rust-lang/crates.io-index)", + "sha-1 0.8.1 (registry+https://github.com/rust-lang/crates.io-index)", + "sha2 0.8.0 (registry+https://github.com/rust-lang/crates.io-index)", + "sha3 0.8.2 (registry+https://github.com/rust-lang/crates.io-index)", + "unsigned-varint 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "rand_xoshiro" -version = "0.1.0" +name = "parity-scale-codec" +version = "1.0.6" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "byteorder 1.3.2 (registry+https://github.com/rust-lang/crates.io-index)", - "rand_core 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", + "arrayvec 0.4.12 (registry+https://github.com/rust-lang/crates.io-index)", + "bitvec 0.14.0 (registry+https://github.com/rust-lang/crates.io-index)", + "byte-slice-cast 0.3.4 (registry+https://github.com/rust-lang/crates.io-index)", + "parity-scale-codec-derive 1.0.3 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "rand_xoshiro" -version = "0.3.1" +name = "parity-scale-codec-derive" +version = "1.0.3" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "rand_core 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)", + "proc-macro-crate 0.1.4 (registry+https://github.com/rust-lang/crates.io-index)", + "proc-macro2 0.4.30 (registry+https://github.com/rust-lang/crates.io-index)", + "quote 0.6.13 (registry+https://github.com/rust-lang/crates.io-index)", + "syn 0.15.44 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "rayon" -version = "1.2.0" +name = "parity-send-wrapper" +version = "0.1.0" source = "registry+https://github.com/rust-lang/crates.io-index" -dependencies = [ - "crossbeam-deque 0.7.1 (registry+https://github.com/rust-lang/crates.io-index)", - "either 1.5.3 (registry+https://github.com/rust-lang/crates.io-index)", - "rayon-core 1.6.0 (registry+https://github.com/rust-lang/crates.io-index)", -] [[package]] -name = "rayon-core" -version = "1.6.0" +name = "parity-util-mem" +version = "0.2.1" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "crossbeam-deque 0.7.1 (registry+https://github.com/rust-lang/crates.io-index)", - "crossbeam-queue 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)", - "crossbeam-utils 0.6.6 (registry+https://github.com/rust-lang/crates.io-index)", - "lazy_static 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)", - "num_cpus 1.10.1 (registry+https://github.com/rust-lang/crates.io-index)", + "cfg-if 0.1.10 (registry+https://github.com/rust-lang/crates.io-index)", + "malloc_size_of_derive 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)", + "winapi 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "rdrand" -version = "0.4.0" +name = "parity-wasm" +version = "0.32.0" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "rand_core 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", + "byteorder 1.3.2 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "redox_syscall" -version = "0.1.56" +name = "parity-wasm" +version = "0.40.3" source = "registry+https://github.com/rust-lang/crates.io-index" [[package]] -name = "ref_thread_local" -version = "0.0.0" +name = "parity-wasm" +version = "0.41.0" source = "registry+https://github.com/rust-lang/crates.io-index" [[package]] -name = "regex" -version = "1.3.1" +name = "parking_lot" +version = "0.6.4" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "aho-corasick 0.7.6 (registry+https://github.com/rust-lang/crates.io-index)", - "memchr 2.2.1 (registry+https://github.com/rust-lang/crates.io-index)", - "regex-syntax 0.6.12 (registry+https://github.com/rust-lang/crates.io-index)", - "thread_local 0.3.6 (registry+https://github.com/rust-lang/crates.io-index)", + "lock_api 0.1.5 (registry+https://github.com/rust-lang/crates.io-index)", + "parking_lot_core 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "regex-automata" -version = "0.1.8" +name = "parking_lot" +version = "0.7.1" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "byteorder 1.3.2 (registry+https://github.com/rust-lang/crates.io-index)", + "lock_api 0.1.5 (registry+https://github.com/rust-lang/crates.io-index)", + "parking_lot_core 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "regex-syntax" -version = "0.6.12" -source = "registry+https://github.com/rust-lang/crates.io-index" - -[[package]] -name = "remove_dir_all" -version = "0.5.2" +name = "parking_lot" +version = "0.8.0" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "winapi 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)", + "lock_api 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)", + "parking_lot_core 0.5.0 (registry+https://github.com/rust-lang/crates.io-index)", + "rustc_version 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "rhododendron" -version = "0.7.0" +name = "parking_lot" +version = "0.9.0" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "error-chain 0.12.1 (registry+https://github.com/rust-lang/crates.io-index)", - "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", - "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", - "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "lock_api 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", + "parking_lot_core 0.6.2 (registry+https://github.com/rust-lang/crates.io-index)", + "rustc_version 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "ring" -version = "0.14.6" +name = "parking_lot_core" +version = "0.3.1" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "cc 1.0.45 (registry+https://github.com/rust-lang/crates.io-index)", - "lazy_static 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)", - "libc 0.2.62 (registry+https://github.com/rust-lang/crates.io-index)", - "spin 0.5.2 (registry+https://github.com/rust-lang/crates.io-index)", - "untrusted 0.6.2 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", + "rand 0.5.6 (registry+https://github.com/rust-lang/crates.io-index)", + "rustc_version 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)", + "smallvec 0.6.13 (registry+https://github.com/rust-lang/crates.io-index)", "winapi 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "rocksdb" -version = "0.11.0" +name = "parking_lot_core" +version = "0.4.0" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "libc 0.2.62 (registry+https://github.com/rust-lang/crates.io-index)", - "librocksdb-sys 5.18.3 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", + "rand 0.6.5 (registry+https://github.com/rust-lang/crates.io-index)", + "rustc_version 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)", + "smallvec 0.6.13 (registry+https://github.com/rust-lang/crates.io-index)", + "winapi 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "rpassword" -version = "4.0.1" +name = "parking_lot_core" +version = "0.5.0" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "libc 0.2.62 (registry+https://github.com/rust-lang/crates.io-index)", + "cfg-if 0.1.10 (registry+https://github.com/rust-lang/crates.io-index)", + "cloudabi 0.0.3 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", + "rand 0.6.5 (registry+https://github.com/rust-lang/crates.io-index)", + "redox_syscall 0.1.56 (registry+https://github.com/rust-lang/crates.io-index)", + "rustc_version 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)", + "smallvec 0.6.13 (registry+https://github.com/rust-lang/crates.io-index)", "winapi 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "rustc-demangle" -version = "0.1.16" +name = "parking_lot_core" +version = "0.6.2" source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "cfg-if 0.1.10 (registry+https://github.com/rust-lang/crates.io-index)", + "cloudabi 0.0.3 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", + "redox_syscall 0.1.56 (registry+https://github.com/rust-lang/crates.io-index)", + "rustc_version 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)", + "smallvec 0.6.13 (registry+https://github.com/rust-lang/crates.io-index)", + "winapi 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)", +] [[package]] -name = "rustc-hex" -version = "2.0.1" +name = "paste" +version = "0.1.6" source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "paste-impl 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)", + "proc-macro-hack 0.5.11 (registry+https://github.com/rust-lang/crates.io-index)", +] [[package]] -name = "rustc_version" -version = "0.2.3" +name = "paste-impl" +version = "0.1.6" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "semver 0.9.0 (registry+https://github.com/rust-lang/crates.io-index)", + "proc-macro-hack 0.5.11 (registry+https://github.com/rust-lang/crates.io-index)", + "proc-macro2 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "quote 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)", + "syn 1.0.8 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "rustls" -version = "0.15.2" +name = "pbkdf2" +version = "0.3.0" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "base64 0.10.1 (registry+https://github.com/rust-lang/crates.io-index)", - "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", - "ring 0.14.6 (registry+https://github.com/rust-lang/crates.io-index)", - "sct 0.5.0 (registry+https://github.com/rust-lang/crates.io-index)", - "untrusted 0.6.2 (registry+https://github.com/rust-lang/crates.io-index)", - "webpki 0.19.1 (registry+https://github.com/rust-lang/crates.io-index)", + "byteorder 1.3.2 (registry+https://github.com/rust-lang/crates.io-index)", + "crypto-mac 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "rustversion" -version = "1.0.0" +name = "pdqselect" +version = "0.1.0" source = "registry+https://github.com/rust-lang/crates.io-index" -dependencies = [ - "proc-macro2 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "quote 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)", - "syn 1.0.7 (registry+https://github.com/rust-lang/crates.io-index)", -] [[package]] -name = "rw-stream-sink" +name = "peeking_take_while" version = "0.1.2" source = "registry+https://github.com/rust-lang/crates.io-index" -dependencies = [ - "bytes 0.4.12 (registry+https://github.com/rust-lang/crates.io-index)", - "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", - "tokio-io 0.1.12 (registry+https://github.com/rust-lang/crates.io-index)", -] [[package]] -name = "ryu" +name = "percent-encoding" version = "1.0.1" source = "registry+https://github.com/rust-lang/crates.io-index" [[package]] -name = "safe-mix" -version = "1.0.0" +name = "percent-encoding" +version = "2.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" + +[[package]] +name = "petgraph" +version = "0.4.13" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "rustc_version 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)", + "fixedbitset 0.1.9 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "safemem" -version = "0.3.2" +name = "pin-utils" +version = "0.1.0-alpha.4" source = "registry+https://github.com/rust-lang/crates.io-index" [[package]] -name = "same-file" -version = "1.0.5" +name = "pkg-config" +version = "0.3.17" source = "registry+https://github.com/rust-lang/crates.io-index" -dependencies = [ - "winapi-util 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)", -] [[package]] -name = "schannel" -version = "0.1.16" +name = "plain" +version = "0.2.3" source = "registry+https://github.com/rust-lang/crates.io-index" -dependencies = [ - "lazy_static 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)", - "winapi 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)", -] [[package]] -name = "schnorrkel" -version = "0.8.5" -source = "registry+https://github.com/rust-lang/crates.io-index" -dependencies = [ - "curve25519-dalek 1.2.3 (registry+https://github.com/rust-lang/crates.io-index)", - "failure 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)", - "merlin 1.2.1 (registry+https://github.com/rust-lang/crates.io-index)", - "rand 0.6.5 (registry+https://github.com/rust-lang/crates.io-index)", - "rand_core 0.4.2 (registry+https://github.com/rust-lang/crates.io-index)", - "rand_os 0.1.3 (registry+https://github.com/rust-lang/crates.io-index)", - "sha2 0.8.0 (registry+https://github.com/rust-lang/crates.io-index)", - "subtle 2.2.1 (registry+https://github.com/rust-lang/crates.io-index)", - "zeroize 0.9.3 (registry+https://github.com/rust-lang/crates.io-index)", -] - -[[package]] -name = "scopeguard" -version = "0.3.3" +name = "ppv-lite86" +version = "0.2.6" source = "registry+https://github.com/rust-lang/crates.io-index" [[package]] -name = "scopeguard" -version = "1.0.0" +name = "pretty_assertions" +version = "0.6.1" source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "ansi_term 0.11.0 (registry+https://github.com/rust-lang/crates.io-index)", + "ctor 0.1.12 (registry+https://github.com/rust-lang/crates.io-index)", + "difference 2.0.0 (registry+https://github.com/rust-lang/crates.io-index)", + "output_vt100 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)", +] [[package]] -name = "sct" -version = "0.5.0" +name = "primitive-types" +version = "0.6.1" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "ring 0.14.6 (registry+https://github.com/rust-lang/crates.io-index)", - "untrusted 0.6.2 (registry+https://github.com/rust-lang/crates.io-index)", + "fixed-hash 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)", + "impl-codec 0.4.2 (registry+https://github.com/rust-lang/crates.io-index)", + "impl-rlp 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)", + "impl-serde 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)", + "uint 0.8.2 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "security-framework" -version = "0.3.1" +name = "proc-macro-crate" +version = "0.1.4" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "core-foundation 0.6.4 (registry+https://github.com/rust-lang/crates.io-index)", - "core-foundation-sys 0.6.2 (registry+https://github.com/rust-lang/crates.io-index)", - "libc 0.2.62 (registry+https://github.com/rust-lang/crates.io-index)", - "security-framework-sys 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", + "toml 0.5.5 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "security-framework-sys" -version = "0.3.1" +name = "proc-macro-error" +version = "0.2.6" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "core-foundation-sys 0.6.2 (registry+https://github.com/rust-lang/crates.io-index)", + "proc-macro2 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "quote 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)", + "syn 1.0.8 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "semver" -version = "0.6.0" +name = "proc-macro-hack" +version = "0.5.11" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "semver-parser 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)", + "proc-macro2 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "quote 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)", + "syn 1.0.8 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "semver" -version = "0.9.0" +<<<<<<< HEAD +name = "reqwest" +version = "0.9.22" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "semver-parser 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)", + "base64 0.10.1 (registry+https://github.com/rust-lang/crates.io-index)", + "bytes 0.4.12 (registry+https://github.com/rust-lang/crates.io-index)", + "cookie 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)", + "cookie_store 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)", + "encoding_rs 0.8.20 (registry+https://github.com/rust-lang/crates.io-index)", + "flate2 1.0.12 (registry+https://github.com/rust-lang/crates.io-index)", + "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", + "http 0.1.18 (registry+https://github.com/rust-lang/crates.io-index)", + "hyper 0.12.35 (registry+https://github.com/rust-lang/crates.io-index)", + "hyper-tls 0.3.2 (registry+https://github.com/rust-lang/crates.io-index)", + "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", + "mime 0.3.14 (registry+https://github.com/rust-lang/crates.io-index)", + "mime_guess 2.0.1 (registry+https://github.com/rust-lang/crates.io-index)", + "native-tls 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)", "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", + "serde_json 1.0.41 (registry+https://github.com/rust-lang/crates.io-index)", + "serde_urlencoded 0.5.5 (registry+https://github.com/rust-lang/crates.io-index)", + "time 0.1.42 (registry+https://github.com/rust-lang/crates.io-index)", + "tokio 0.1.22 (registry+https://github.com/rust-lang/crates.io-index)", + "tokio-executor 0.1.8 (registry+https://github.com/rust-lang/crates.io-index)", + "tokio-io 0.1.12 (registry+https://github.com/rust-lang/crates.io-index)", + "tokio-threadpool 0.1.16 (registry+https://github.com/rust-lang/crates.io-index)", + "tokio-timer 0.2.11 (registry+https://github.com/rust-lang/crates.io-index)", + "url 1.7.2 (registry+https://github.com/rust-lang/crates.io-index)", + "uuid 0.7.4 (registry+https://github.com/rust-lang/crates.io-index)", + "winreg 0.6.2 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "semver-parser" +name = "rhododendron" version = "0.7.0" +======= +name = "proc-macro-nested" +version = "0.1.3" +>>>>>>> f44873dce... Reorganising the repository - external renames and moves (#4074) source = "registry+https://github.com/rust-lang/crates.io-index" [[package]] -name = "send_wrapper" -version = "0.2.0" +name = "proc-macro2" +version = "0.4.30" source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "unicode-xid 0.1.0 (registry+https://github.com/rust-lang/crates.io-index)", +] [[package]] -name = "serde" -version = "1.0.101" +name = "proc-macro2" +version = "1.0.6" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "serde_derive 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", + "unicode-xid 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "serde_derive" -version = "1.0.101" +name = "procinfo" +version = "0.3.1" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "proc-macro2 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "quote 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)", - "syn 1.0.7 (registry+https://github.com/rust-lang/crates.io-index)", + "byteorder 0.5.3 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", + "nom 1.2.4 (registry+https://github.com/rust-lang/crates.io-index)", + "rustc_version 0.1.7 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "serde_json" -version = "1.0.41" +name = "prometheus" +version = "0.7.0" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "itoa 0.4.4 (registry+https://github.com/rust-lang/crates.io-index)", - "ryu 1.0.1 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", + "cfg-if 0.1.10 (registry+https://github.com/rust-lang/crates.io-index)", + "fnv 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "lazy_static 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", + "procinfo 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", + "protobuf 2.8.1 (registry+https://github.com/rust-lang/crates.io-index)", + "quick-error 1.2.2 (registry+https://github.com/rust-lang/crates.io-index)", + "spin 0.5.2 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "sha-1" -version = "0.8.1" +name = "prost" +version = "0.5.0" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "block-buffer 0.7.3 (registry+https://github.com/rust-lang/crates.io-index)", - "digest 0.8.1 (registry+https://github.com/rust-lang/crates.io-index)", - "fake-simd 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)", - "opaque-debug 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)", + "byteorder 1.3.2 (registry+https://github.com/rust-lang/crates.io-index)", + "bytes 0.4.12 (registry+https://github.com/rust-lang/crates.io-index)", + "prost-derive 0.5.0 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "sha1" -version = "0.6.0" +name = "prost-build" +version = "0.5.0" source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "bytes 0.4.12 (registry+https://github.com/rust-lang/crates.io-index)", + "heck 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", + "itertools 0.8.1 (registry+https://github.com/rust-lang/crates.io-index)", + "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", + "multimap 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)", + "petgraph 0.4.13 (registry+https://github.com/rust-lang/crates.io-index)", + "prost 0.5.0 (registry+https://github.com/rust-lang/crates.io-index)", + "prost-types 0.5.0 (registry+https://github.com/rust-lang/crates.io-index)", + "tempfile 3.1.0 (registry+https://github.com/rust-lang/crates.io-index)", + "which 2.0.1 (registry+https://github.com/rust-lang/crates.io-index)", +] [[package]] -name = "sha2" -version = "0.6.0" +name = "prost-derive" +version = "0.5.0" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "block-buffer 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)", - "byte-tools 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)", - "digest 0.6.2 (registry+https://github.com/rust-lang/crates.io-index)", - "fake-simd 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)", - "generic-array 0.8.3 (registry+https://github.com/rust-lang/crates.io-index)", + "failure 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)", + "itertools 0.8.1 (registry+https://github.com/rust-lang/crates.io-index)", + "proc-macro2 0.4.30 (registry+https://github.com/rust-lang/crates.io-index)", + "quote 0.6.13 (registry+https://github.com/rust-lang/crates.io-index)", + "syn 0.15.44 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "sha2" -version = "0.8.0" +name = "prost-types" +version = "0.5.0" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "block-buffer 0.7.3 (registry+https://github.com/rust-lang/crates.io-index)", - "digest 0.8.1 (registry+https://github.com/rust-lang/crates.io-index)", - "fake-simd 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)", - "opaque-debug 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)", + "bytes 0.4.12 (registry+https://github.com/rust-lang/crates.io-index)", + "prost 0.5.0 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "sha3" -version = "0.8.2" +name = "protobuf" +version = "2.8.1" source = "registry+https://github.com/rust-lang/crates.io-index" -dependencies = [ - "block-buffer 0.7.3 (registry+https://github.com/rust-lang/crates.io-index)", - "byte-tools 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", - "digest 0.8.1 (registry+https://github.com/rust-lang/crates.io-index)", - "keccak 0.1.0 (registry+https://github.com/rust-lang/crates.io-index)", - "opaque-debug 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)", -] [[package]] -name = "shell32-sys" -version = "0.1.2" +<<<<<<< HEAD +<<<<<<< HEAD +name = "rustc_version" +version = "0.1.7" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "winapi 0.2.8 (registry+https://github.com/rust-lang/crates.io-index)", - "winapi-build 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)", + "semver 0.1.20 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "slab" -version = "0.4.2" +name = "rustc_version" +version = "0.2.3" +======= +======= +name = "publicsuffix" +version = "1.5.4" source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "error-chain 0.12.1 (registry+https://github.com/rust-lang/crates.io-index)", + "idna 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)", + "lazy_static 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)", + "regex 1.3.1 (registry+https://github.com/rust-lang/crates.io-index)", + "url 2.1.0 (registry+https://github.com/rust-lang/crates.io-index)", +] [[package]] -name = "slog" -version = "2.5.2" +>>>>>>> 9e0aa9e5d... base-code(rebased) +name = "pwasm-utils" +version = "0.11.0" +>>>>>>> f44873dce... Reorganising the repository - external renames and moves (#4074) source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "erased-serde 0.3.9 (registry+https://github.com/rust-lang/crates.io-index)", + "byteorder 1.3.2 (registry+https://github.com/rust-lang/crates.io-index)", + "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", + "parity-wasm 0.40.3 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "slog-async" -version = "2.3.0" -source = "git+https://github.com/paritytech/slog-async#107848e7ded5e80dc43f6296c2b96039eb92c0a5" +name = "quick-error" +version = "1.2.2" +source = "registry+https://github.com/rust-lang/crates.io-index" + +[[package]] +name = "quickcheck" +version = "0.8.5" +source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "crossbeam-channel 0.3.9 (registry+https://github.com/rust-lang/crates.io-index)", - "slog 2.5.2 (registry+https://github.com/rust-lang/crates.io-index)", - "take_mut 0.2.2 (registry+https://github.com/rust-lang/crates.io-index)", - "thread_local 0.3.6 (registry+https://github.com/rust-lang/crates.io-index)", + "env_logger 0.6.2 (registry+https://github.com/rust-lang/crates.io-index)", + "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", + "rand 0.6.5 (registry+https://github.com/rust-lang/crates.io-index)", + "rand_core 0.4.2 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "slog-json" -version = "2.3.0" +name = "quickcheck" +version = "0.9.0" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "chrono 0.4.9 (registry+https://github.com/rust-lang/crates.io-index)", - "erased-serde 0.3.9 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", - "serde_json 1.0.41 (registry+https://github.com/rust-lang/crates.io-index)", - "slog 2.5.2 (registry+https://github.com/rust-lang/crates.io-index)", + "env_logger 0.6.2 (registry+https://github.com/rust-lang/crates.io-index)", + "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", + "rand 0.7.2 (registry+https://github.com/rust-lang/crates.io-index)", + "rand_core 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "slog-scope" -version = "4.1.2" +name = "quote" +version = "0.6.13" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "arc-swap 0.3.11 (registry+https://github.com/rust-lang/crates.io-index)", - "lazy_static 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)", - "slog 2.5.2 (registry+https://github.com/rust-lang/crates.io-index)", + "proc-macro2 0.4.30 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "slog_derive" -version = "0.1.1" +name = "quote" +version = "1.0.2" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "proc-macro2 0.4.30 (registry+https://github.com/rust-lang/crates.io-index)", - "quote 0.6.13 (registry+https://github.com/rust-lang/crates.io-index)", - "syn 0.15.44 (registry+https://github.com/rust-lang/crates.io-index)", + "proc-macro2 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "smallvec" -version = "0.6.10" +name = "rand" +version = "0.3.23" source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", + "rand 0.4.6 (registry+https://github.com/rust-lang/crates.io-index)", +] [[package]] -name = "snow" -version = "0.5.2" +name = "rand" +version = "0.4.6" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "arrayref 0.3.5 (registry+https://github.com/rust-lang/crates.io-index)", - "byteorder 1.3.2 (registry+https://github.com/rust-lang/crates.io-index)", - "failure 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)", - "failure_derive 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)", - "rand_core 0.4.2 (registry+https://github.com/rust-lang/crates.io-index)", - "ring 0.14.6 (registry+https://github.com/rust-lang/crates.io-index)", - "rustc_version 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)", - "smallvec 0.6.10 (registry+https://github.com/rust-lang/crates.io-index)", - "static_slice 0.0.3 (registry+https://github.com/rust-lang/crates.io-index)", - "subtle 2.2.1 (registry+https://github.com/rust-lang/crates.io-index)", + "fuchsia-cprng 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", + "rand_core 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", + "rdrand 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)", + "winapi 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "soketto" -version = "0.2.3" +name = "rand" +version = "0.5.6" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "base64 0.10.1 (registry+https://github.com/rust-lang/crates.io-index)", - "bytes 0.4.12 (registry+https://github.com/rust-lang/crates.io-index)", - "flate2 1.0.12 (registry+https://github.com/rust-lang/crates.io-index)", - "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", - "http 0.1.18 (registry+https://github.com/rust-lang/crates.io-index)", - "httparse 1.3.4 (registry+https://github.com/rust-lang/crates.io-index)", - "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", - "rand 0.6.5 (registry+https://github.com/rust-lang/crates.io-index)", - "sha1 0.6.0 (registry+https://github.com/rust-lang/crates.io-index)", - "smallvec 0.6.10 (registry+https://github.com/rust-lang/crates.io-index)", - "tokio-codec 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)", - "tokio-io 0.1.12 (registry+https://github.com/rust-lang/crates.io-index)", + "cloudabi 0.0.3 (registry+https://github.com/rust-lang/crates.io-index)", + "fuchsia-cprng 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", + "rand_core 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", + "winapi 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "sourcefile" -version = "0.1.4" +name = "rand" +version = "0.6.5" source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "autocfg 0.1.7 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", + "rand_chacha 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)", + "rand_core 0.4.2 (registry+https://github.com/rust-lang/crates.io-index)", + "rand_hc 0.1.0 (registry+https://github.com/rust-lang/crates.io-index)", + "rand_isaac 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)", + "rand_jitter 0.1.4 (registry+https://github.com/rust-lang/crates.io-index)", + "rand_os 0.1.3 (registry+https://github.com/rust-lang/crates.io-index)", + "rand_pcg 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)", + "rand_xorshift 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)", + "winapi 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)", +] [[package]] -name = "spin" -version = "0.5.2" +name = "rand" +version = "0.7.2" source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "getrandom 0.1.13 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", + "rand_chacha 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)", + "rand_core 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)", + "rand_hc 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)", +] [[package]] -name = "sr-api-macros" -version = "2.0.0" +name = "rand_chacha" +version = "0.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "blake2-rfc 0.2.18 (registry+https://github.com/rust-lang/crates.io-index)", - "criterion 0.3.0 (registry+https://github.com/rust-lang/crates.io-index)", - "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "proc-macro-crate 0.1.4 (registry+https://github.com/rust-lang/crates.io-index)", - "proc-macro2 0.4.30 (registry+https://github.com/rust-lang/crates.io-index)", - "quote 0.6.13 (registry+https://github.com/rust-lang/crates.io-index)", - "rustversion 1.0.0 (registry+https://github.com/rust-lang/crates.io-index)", - "sr-primitives 2.0.0", - "sr-version 2.0.0", - "substrate-client 2.0.0", - "substrate-consensus-common 2.0.0", - "substrate-primitives 2.0.0", - "substrate-state-machine 2.0.0", - "substrate-test-runtime-client 2.0.0", - "syn 0.15.44 (registry+https://github.com/rust-lang/crates.io-index)", - "trybuild 1.0.17 (registry+https://github.com/rust-lang/crates.io-index)", + "autocfg 0.1.7 (registry+https://github.com/rust-lang/crates.io-index)", + "rand_core 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "sr-arithmetic" -version = "2.0.0" +name = "rand_chacha" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "criterion 0.3.0 (registry+https://github.com/rust-lang/crates.io-index)", - "integer-sqrt 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)", - "num-traits 0.2.8 (registry+https://github.com/rust-lang/crates.io-index)", - "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "primitive-types 0.6.0 (registry+https://github.com/rust-lang/crates.io-index)", - "rand 0.7.2 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", - "sr-std 2.0.0", - "substrate-debug-derive 2.0.0", + "c2-chacha 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)", + "rand_core 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "sr-io" -version = "2.0.0" +name = "rand_core" +version = "0.3.1" +source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "hash-db 0.15.2 (registry+https://github.com/rust-lang/crates.io-index)", - "libsecp256k1 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", - "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", - "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "rustc_version 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)", - "sr-std 2.0.0", - "substrate-externalities 2.0.0", - "substrate-primitives 2.0.0", - "substrate-state-machine 2.0.0", - "substrate-trie 2.0.0", - "tiny-keccak 1.5.0 (registry+https://github.com/rust-lang/crates.io-index)", + "rand_core 0.4.2 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "sr-primitives" -version = "2.0.0" +name = "rand_core" +version = "0.4.2" +source = "registry+https://github.com/rust-lang/crates.io-index" + +[[package]] +<<<<<<< HEAD +name = "semver" +version = "0.1.20" +source = "registry+https://github.com/rust-lang/crates.io-index" + +[[package]] +name = "semver" +version = "0.6.0" +======= +name = "rand_core" +version = "0.5.1" +>>>>>>> f44873dce... Reorganising the repository - external renames and moves (#4074) +source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "impl-trait-for-tuples 0.1.3 (registry+https://github.com/rust-lang/crates.io-index)", - "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", - "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "paste 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)", - "rand 0.7.2 (registry+https://github.com/rust-lang/crates.io-index)", + "getrandom 0.1.13 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "rand_hc" +version = "0.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "rand_core 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "rand_hc" +version = "0.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "rand_core 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "rand_isaac" +version = "0.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "rand_core 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "rand_jitter" +version = "0.1.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", + "rand_core 0.4.2 (registry+https://github.com/rust-lang/crates.io-index)", + "winapi 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +<<<<<<< HEAD +name = "serde_urlencoded" +version = "0.5.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "dtoa 0.4.4 (registry+https://github.com/rust-lang/crates.io-index)", + "itoa 0.4.4 (registry+https://github.com/rust-lang/crates.io-index)", "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", - "serde_json 1.0.41 (registry+https://github.com/rust-lang/crates.io-index)", - "sr-arithmetic 2.0.0", - "sr-io 2.0.0", - "sr-std 2.0.0", - "substrate-application-crypto 2.0.0", - "substrate-offchain 2.0.0", - "substrate-primitives 2.0.0", + "url 1.7.2 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "sr-sandbox" -version = "2.0.0" +name = "sha-1" +version = "0.8.1" +======= +name = "rand_os" +version = "0.1.3" +>>>>>>> f44873dce... Reorganising the repository - external renames and moves (#4074) +source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "assert_matches 1.3.0 (registry+https://github.com/rust-lang/crates.io-index)", - "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "cloudabi 0.0.3 (registry+https://github.com/rust-lang/crates.io-index)", + "fuchsia-cprng 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", + "rand_core 0.4.2 (registry+https://github.com/rust-lang/crates.io-index)", + "rdrand 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)", + "wasm-bindgen 0.2.54 (registry+https://github.com/rust-lang/crates.io-index)", + "winapi 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "rand_os" +version = "0.2.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "getrandom 0.1.13 (registry+https://github.com/rust-lang/crates.io-index)", + "rand_core 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "rand_pcg" +version = "0.1.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "autocfg 0.1.7 (registry+https://github.com/rust-lang/crates.io-index)", + "rand_core 0.4.2 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "rand_xorshift" +version = "0.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "rand_core 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "rand_xoshiro" +version = "0.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "byteorder 1.3.2 (registry+https://github.com/rust-lang/crates.io-index)", + "rand_core 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "rand_xoshiro" +version = "0.3.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "rand_core 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "raw-cpuid" +version = "6.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "bitflags 1.2.1 (registry+https://github.com/rust-lang/crates.io-index)", + "cc 1.0.47 (registry+https://github.com/rust-lang/crates.io-index)", "rustc_version 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)", - "sr-std 2.0.0", - "substrate-primitives 2.0.0", - "wabt 0.9.2 (registry+https://github.com/rust-lang/crates.io-index)", - "wasmi 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "sr-staking-primitives" -version = "2.0.0" +name = "rayon" +version = "1.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "crossbeam-deque 0.7.2 (registry+https://github.com/rust-lang/crates.io-index)", + "either 1.5.3 (registry+https://github.com/rust-lang/crates.io-index)", + "rayon-core 1.6.0 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "rayon-core" +version = "1.6.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "crossbeam-deque 0.7.2 (registry+https://github.com/rust-lang/crates.io-index)", + "crossbeam-queue 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)", + "crossbeam-utils 0.6.6 (registry+https://github.com/rust-lang/crates.io-index)", + "lazy_static 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)", + "num_cpus 1.11.0 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "rdrand" +version = "0.4.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "rand_core 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "redox_syscall" +version = "0.1.56" +source = "registry+https://github.com/rust-lang/crates.io-index" + +[[package]] +name = "redox_users" +version = "0.3.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "failure 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)", + "rand_os 0.1.3 (registry+https://github.com/rust-lang/crates.io-index)", + "redox_syscall 0.1.56 (registry+https://github.com/rust-lang/crates.io-index)", + "rust-argon2 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "ref_thread_local" +version = "0.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" + +[[package]] +name = "regex" +version = "1.3.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "aho-corasick 0.7.6 (registry+https://github.com/rust-lang/crates.io-index)", + "memchr 2.2.1 (registry+https://github.com/rust-lang/crates.io-index)", + "regex-syntax 0.6.12 (registry+https://github.com/rust-lang/crates.io-index)", + "thread_local 0.3.6 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "regex-automata" +version = "0.1.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "byteorder 1.3.2 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "regex-syntax" +version = "0.6.12" +source = "registry+https://github.com/rust-lang/crates.io-index" + +[[package]] +name = "region" +version = "2.1.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "bitflags 1.2.1 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", + "mach 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)", + "winapi 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "remove_dir_all" +version = "0.5.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "winapi 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "reqwest" +version = "0.9.22" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "base64 0.10.1 (registry+https://github.com/rust-lang/crates.io-index)", + "bytes 0.4.12 (registry+https://github.com/rust-lang/crates.io-index)", + "cookie 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)", + "cookie_store 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)", + "encoding_rs 0.8.20 (registry+https://github.com/rust-lang/crates.io-index)", + "flate2 1.0.13 (registry+https://github.com/rust-lang/crates.io-index)", + "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", + "http 0.1.19 (registry+https://github.com/rust-lang/crates.io-index)", + "hyper 0.12.35 (registry+https://github.com/rust-lang/crates.io-index)", + "hyper-tls 0.3.2 (registry+https://github.com/rust-lang/crates.io-index)", + "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", + "mime 0.3.14 (registry+https://github.com/rust-lang/crates.io-index)", + "mime_guess 2.0.1 (registry+https://github.com/rust-lang/crates.io-index)", + "native-tls 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "serde_json 1.0.41 (registry+https://github.com/rust-lang/crates.io-index)", + "serde_urlencoded 0.5.5 (registry+https://github.com/rust-lang/crates.io-index)", + "time 0.1.42 (registry+https://github.com/rust-lang/crates.io-index)", + "tokio 0.1.22 (registry+https://github.com/rust-lang/crates.io-index)", + "tokio-executor 0.1.8 (registry+https://github.com/rust-lang/crates.io-index)", + "tokio-io 0.1.12 (registry+https://github.com/rust-lang/crates.io-index)", + "tokio-threadpool 0.1.16 (registry+https://github.com/rust-lang/crates.io-index)", + "tokio-timer 0.2.11 (registry+https://github.com/rust-lang/crates.io-index)", + "url 1.7.2 (registry+https://github.com/rust-lang/crates.io-index)", + "uuid 0.7.4 (registry+https://github.com/rust-lang/crates.io-index)", + "winreg 0.6.2 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "ring" +version = "0.16.9" +source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "sr-primitives 2.0.0", - "sr-std 2.0.0", + "cc 1.0.47 (registry+https://github.com/rust-lang/crates.io-index)", + "lazy_static 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", + "spin 0.5.2 (registry+https://github.com/rust-lang/crates.io-index)", + "untrusted 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)", + "web-sys 0.3.31 (registry+https://github.com/rust-lang/crates.io-index)", + "winapi 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "sr-std" -version = "2.0.0" +name = "rlp" +version = "0.4.3" +source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "rustc_version 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)", + "rustc-hex 2.0.1 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "sr-version" -version = "2.0.0" +name = "rocksdb" +version = "0.11.0" +source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "impl-serde 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)", - "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", - "sr-primitives 2.0.0", - "sr-std 2.0.0", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", + "librocksdb-sys 5.18.3 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "srml-assets" -version = "2.0.0" +name = "rpassword" +version = "4.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", - "sr-io 2.0.0", - "sr-primitives 2.0.0", - "sr-std 2.0.0", - "srml-support 2.0.0", - "srml-system 2.0.0", - "substrate-primitives 2.0.0", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", + "winapi 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "srml-aura" -version = "2.0.0" +name = "rust-argon2" +version = "0.5.1" +source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "lazy_static 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)", - "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "parking_lot 0.9.0 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", - "sr-io 2.0.0", - "sr-primitives 2.0.0", - "sr-std 2.0.0", - "srml-session 2.0.0", - "srml-support 2.0.0", - "srml-system 2.0.0", - "srml-timestamp 2.0.0", - "substrate-application-crypto 2.0.0", - "substrate-consensus-aura-primitives 2.0.0", - "substrate-inherents 2.0.0", - "substrate-primitives 2.0.0", + "base64 0.10.1 (registry+https://github.com/rust-lang/crates.io-index)", + "blake2b_simd 0.5.9 (registry+https://github.com/rust-lang/crates.io-index)", + "crossbeam-utils 0.6.6 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "srml-authority-discovery" -version = "0.1.0" +name = "rustc-demangle" +version = "0.1.16" +source = "registry+https://github.com/rust-lang/crates.io-index" + +[[package]] +name = "rustc-hex" +version = "2.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" + +[[package]] +name = "rustc_version" +version = "0.1.7" +source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", - "sr-io 2.0.0", - "sr-primitives 2.0.0", - "sr-staking-primitives 2.0.0", - "sr-std 2.0.0", - "srml-session 2.0.0", - "srml-support 2.0.0", - "srml-system 2.0.0", - "substrate-application-crypto 2.0.0", - "substrate-consensus-babe-primitives 2.0.0", - "substrate-primitives 2.0.0", + "semver 0.1.20 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "srml-authorship" -version = "0.1.0" +name = "rustc_version" +version = "0.2.3" +source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "impl-trait-for-tuples 0.1.3 (registry+https://github.com/rust-lang/crates.io-index)", - "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "sr-io 2.0.0", - "sr-primitives 2.0.0", - "sr-std 2.0.0", - "srml-support 2.0.0", - "srml-system 2.0.0", - "substrate-inherents 2.0.0", - "substrate-primitives 2.0.0", + "semver 0.9.0 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "srml-babe" -version = "2.0.0" +name = "rustls" +version = "0.16.0" +source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "hex-literal 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)", - "lazy_static 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)", - "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "parking_lot 0.9.0 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", - "sr-io 2.0.0", - "sr-primitives 2.0.0", - "sr-staking-primitives 2.0.0", - "sr-std 2.0.0", - "sr-version 2.0.0", - "srml-session 2.0.0", - "srml-support 2.0.0", - "srml-system 2.0.0", - "srml-timestamp 2.0.0", - "substrate-consensus-babe-primitives 2.0.0", - "substrate-inherents 2.0.0", - "substrate-primitives 2.0.0", - "substrate-test-runtime 2.0.0", + "base64 0.10.1 (registry+https://github.com/rust-lang/crates.io-index)", + "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", + "ring 0.16.9 (registry+https://github.com/rust-lang/crates.io-index)", + "sct 0.6.0 (registry+https://github.com/rust-lang/crates.io-index)", + "webpki 0.21.0 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "srml-balances" -version = "2.0.0" +name = "rustversion" +version = "1.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "safe-mix 1.0.0 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", - "sr-io 2.0.0", - "sr-primitives 2.0.0", - "sr-std 2.0.0", - "srml-support 2.0.0", - "srml-system 2.0.0", - "srml-transaction-payment 2.0.0", - "substrate-keyring 2.0.0", - "substrate-primitives 2.0.0", + "proc-macro2 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "quote 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)", + "syn 1.0.8 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "srml-collective" -version = "2.0.0" +name = "rw-stream-sink" +version = "0.1.2" +source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "hex-literal 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)", - "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "safe-mix 1.0.0 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", - "sr-io 2.0.0", - "sr-primitives 2.0.0", - "sr-std 2.0.0", - "srml-balances 2.0.0", - "srml-support 2.0.0", - "srml-system 2.0.0", - "substrate-primitives 2.0.0", + "bytes 0.4.12 (registry+https://github.com/rust-lang/crates.io-index)", + "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", + "tokio-io 0.1.12 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "srml-contracts" -version = "2.0.0" +name = "ryu" +version = "1.0.2" +source = "registry+https://github.com/rust-lang/crates.io-index" + +[[package]] +name = "safe-mix" +version = "1.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "assert_matches 1.3.0 (registry+https://github.com/rust-lang/crates.io-index)", - "hex 0.3.2 (registry+https://github.com/rust-lang/crates.io-index)", - "hex-literal 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)", - "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "parity-wasm 0.40.3 (registry+https://github.com/rust-lang/crates.io-index)", - "pwasm-utils 0.11.0 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", - "sr-io 2.0.0", - "sr-primitives 2.0.0", - "sr-sandbox 2.0.0", - "sr-std 2.0.0", - "srml-balances 2.0.0", - "srml-randomness-collective-flip 2.0.0", - "srml-support 2.0.0", - "srml-system 2.0.0", - "srml-timestamp 2.0.0", - "substrate-primitives 2.0.0", - "wabt 0.9.2 (registry+https://github.com/rust-lang/crates.io-index)", - "wasmi-validation 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)", + "rustc_version 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "srml-contracts-rpc" -version = "2.0.0" +name = "safemem" +version = "0.3.3" +source = "registry+https://github.com/rust-lang/crates.io-index" + +[[package]] +name = "same-file" +version = "1.0.5" +source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "jsonrpc-core 13.2.0 (registry+https://github.com/rust-lang/crates.io-index)", - "jsonrpc-core-client 13.2.0 (registry+https://github.com/rust-lang/crates.io-index)", - "jsonrpc-derive 13.2.0 (registry+https://github.com/rust-lang/crates.io-index)", - "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", - "sr-primitives 2.0.0", - "srml-contracts-rpc-runtime-api 2.0.0", - "substrate-client 2.0.0", - "substrate-primitives 2.0.0", - "substrate-rpc-primitives 2.0.0", + "winapi-util 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "srml-contracts-rpc-runtime-api" -version = "2.0.0" +name = "schannel" +version = "0.1.16" +source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", - "sr-primitives 2.0.0", - "sr-std 2.0.0", - "substrate-client 2.0.0", + "lazy_static 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)", + "winapi 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "srml-democracy" -version = "2.0.0" +name = "schnorrkel" +version = "0.8.5" +source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "safe-mix 1.0.0 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", - "sr-io 2.0.0", - "sr-primitives 2.0.0", - "sr-std 2.0.0", - "srml-balances 2.0.0", - "srml-support 2.0.0", - "srml-system 2.0.0", - "substrate-primitives 2.0.0", + "curve25519-dalek 1.2.3 (registry+https://github.com/rust-lang/crates.io-index)", + "failure 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)", + "merlin 1.2.1 (registry+https://github.com/rust-lang/crates.io-index)", + "rand 0.6.5 (registry+https://github.com/rust-lang/crates.io-index)", + "rand_core 0.4.2 (registry+https://github.com/rust-lang/crates.io-index)", + "rand_os 0.1.3 (registry+https://github.com/rust-lang/crates.io-index)", + "sha2 0.8.0 (registry+https://github.com/rust-lang/crates.io-index)", + "subtle 2.2.1 (registry+https://github.com/rust-lang/crates.io-index)", + "zeroize 0.9.3 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "srml-elections" -version = "2.0.0" +name = "scopeguard" +version = "0.3.3" +source = "registry+https://github.com/rust-lang/crates.io-index" + +[[package]] +name = "scopeguard" +version = "1.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" + +[[package]] +name = "scroll" +version = "0.9.2" +source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "hex-literal 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)", - "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "safe-mix 1.0.0 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", - "sr-io 2.0.0", - "sr-primitives 2.0.0", - "sr-std 2.0.0", - "srml-balances 2.0.0", - "srml-support 2.0.0", - "srml-system 2.0.0", - "substrate-primitives 2.0.0", + "rustc_version 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)", + "scroll_derive 0.9.5 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "srml-elections-phragmen" -version = "2.0.0" +name = "scroll_derive" +version = "0.9.5" +source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "hex-literal 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)", - "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", - "sr-io 2.0.0", - "sr-primitives 2.0.0", - "sr-std 2.0.0", - "srml-balances 2.0.0", - "srml-support 2.0.0", - "srml-system 2.0.0", - "substrate-phragmen 2.0.0", - "substrate-primitives 2.0.0", + "proc-macro2 0.4.30 (registry+https://github.com/rust-lang/crates.io-index)", + "quote 0.6.13 (registry+https://github.com/rust-lang/crates.io-index)", + "syn 0.15.44 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "srml-example" -version = "2.0.0" +name = "sct" +version = "0.6.0" +source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", - "sr-io 2.0.0", - "sr-primitives 2.0.0", - "sr-std 2.0.0", - "srml-balances 2.0.0", - "srml-support 2.0.0", - "srml-system 2.0.0", - "substrate-primitives 2.0.0", + "ring 0.16.9 (registry+https://github.com/rust-lang/crates.io-index)", + "untrusted 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "srml-executive" -version = "2.0.0" +name = "security-framework" +version = "0.3.3" +source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "hex-literal 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)", - "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", - "sr-io 2.0.0", - "sr-primitives 2.0.0", - "sr-std 2.0.0", - "srml-balances 2.0.0", - "srml-indices 2.0.0", - "srml-support 2.0.0", - "srml-system 2.0.0", - "srml-transaction-payment 2.0.0", - "substrate-primitives 2.0.0", + "core-foundation 0.6.4 (registry+https://github.com/rust-lang/crates.io-index)", + "core-foundation-sys 0.6.2 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", + "security-framework-sys 0.3.3 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "srml-finality-tracker" -version = "2.0.0" +name = "security-framework-sys" +version = "0.3.3" +source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "impl-trait-for-tuples 0.1.3 (registry+https://github.com/rust-lang/crates.io-index)", - "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", - "sr-io 2.0.0", - "sr-primitives 2.0.0", - "sr-std 2.0.0", - "srml-support 2.0.0", - "srml-system 2.0.0", - "substrate-inherents 2.0.0", - "substrate-primitives 2.0.0", + "core-foundation-sys 0.6.2 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "srml-generic-asset" -version = "2.0.0" -dependencies = [ - "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", - "sr-io 2.0.0", - "sr-primitives 2.0.0", - "sr-std 2.0.0", - "srml-support 2.0.0", - "srml-system 2.0.0", - "substrate-primitives 2.0.0", -] +name = "semver" +version = "0.1.20" +source = "registry+https://github.com/rust-lang/crates.io-index" [[package]] -name = "srml-grandpa" -version = "2.0.0" +name = "semver" +version = "0.6.0" +source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", - "sr-io 2.0.0", - "sr-primitives 2.0.0", - "sr-staking-primitives 2.0.0", - "sr-std 2.0.0", - "srml-finality-tracker 2.0.0", - "srml-session 2.0.0", - "srml-support 2.0.0", - "srml-system 2.0.0", - "substrate-finality-grandpa-primitives 2.0.0", - "substrate-primitives 2.0.0", + "semver-parser 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "srml-im-online" -version = "0.1.0" +name = "semver" +version = "0.9.0" +source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", - "sr-io 2.0.0", - "sr-primitives 2.0.0", - "sr-staking-primitives 2.0.0", - "sr-std 2.0.0", - "srml-session 2.0.0", - "srml-support 2.0.0", - "srml-system 2.0.0", - "substrate-application-crypto 2.0.0", - "substrate-offchain 2.0.0", - "substrate-primitives 2.0.0", + "semver-parser 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "srml-indices" -version = "2.0.0" +name = "semver-parser" +version = "0.7.0" +source = "registry+https://github.com/rust-lang/crates.io-index" + +[[package]] +name = "send_wrapper" +version = "0.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" + +[[package]] +name = "serde" +version = "1.0.102" +source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "ref_thread_local 0.0.0 (registry+https://github.com/rust-lang/crates.io-index)", - "safe-mix 1.0.0 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", - "sr-io 2.0.0", - "sr-primitives 2.0.0", - "sr-std 2.0.0", - "srml-support 2.0.0", - "srml-system 2.0.0", - "substrate-keyring 2.0.0", - "substrate-primitives 2.0.0", + "serde_derive 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "srml-membership" -version = "2.0.0" +name = "serde_derive" +version = "1.0.102" +source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", - "sr-io 2.0.0", - "sr-primitives 2.0.0", - "sr-std 2.0.0", - "srml-support 2.0.0", - "srml-system 2.0.0", - "substrate-primitives 2.0.0", + "proc-macro2 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "quote 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)", + "syn 1.0.8 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "srml-metadata" -version = "2.0.0" +name = "serde_json" +version = "1.0.41" +source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", - "sr-std 2.0.0", - "substrate-primitives 2.0.0", + "itoa 0.4.4 (registry+https://github.com/rust-lang/crates.io-index)", + "ryu 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "srml-nicks" -version = "2.0.0" +name = "serde_urlencoded" +version = "0.5.5" +source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", - "sr-io 2.0.0", - "sr-primitives 2.0.0", - "sr-std 2.0.0", - "srml-balances 2.0.0", - "srml-support 2.0.0", - "srml-system 2.0.0", - "substrate-primitives 2.0.0", + "dtoa 0.4.4 (registry+https://github.com/rust-lang/crates.io-index)", + "itoa 0.4.4 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "url 1.7.2 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "srml-offences" -version = "1.0.0" +name = "sha-1" +version = "0.8.1" +source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", - "sr-io 2.0.0", - "sr-primitives 2.0.0", - "sr-staking-primitives 2.0.0", - "sr-std 2.0.0", - "srml-balances 2.0.0", - "srml-support 2.0.0", - "srml-system 2.0.0", - "substrate-primitives 2.0.0", + "block-buffer 0.7.3 (registry+https://github.com/rust-lang/crates.io-index)", + "digest 0.8.1 (registry+https://github.com/rust-lang/crates.io-index)", + "fake-simd 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)", + "opaque-debug 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "srml-randomness-collective-flip" -version = "2.0.0" +name = "sha1" +version = "0.6.0" +source = "registry+https://github.com/rust-lang/crates.io-index" + +[[package]] +name = "sha2" +version = "0.8.0" +source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "safe-mix 1.0.0 (registry+https://github.com/rust-lang/crates.io-index)", - "sr-io 2.0.0", - "sr-primitives 2.0.0", - "sr-std 2.0.0", - "srml-support 2.0.0", - "srml-system 2.0.0", - "substrate-primitives 2.0.0", + "block-buffer 0.7.3 (registry+https://github.com/rust-lang/crates.io-index)", + "digest 0.8.1 (registry+https://github.com/rust-lang/crates.io-index)", + "fake-simd 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)", + "opaque-debug 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "srml-scored-pool" -version = "1.0.0" +name = "sha3" +version = "0.8.2" +source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", - "sr-io 2.0.0", - "sr-primitives 2.0.0", - "sr-std 2.0.0", - "srml-balances 2.0.0", - "srml-support 2.0.0", - "srml-system 2.0.0", - "substrate-primitives 2.0.0", + "block-buffer 0.7.3 (registry+https://github.com/rust-lang/crates.io-index)", + "byte-tools 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", + "digest 0.8.1 (registry+https://github.com/rust-lang/crates.io-index)", + "keccak 0.1.0 (registry+https://github.com/rust-lang/crates.io-index)", + "opaque-debug 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "srml-session" -version = "2.0.0" +name = "shell32-sys" +version = "0.1.2" +source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "impl-trait-for-tuples 0.1.3 (registry+https://github.com/rust-lang/crates.io-index)", - "lazy_static 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)", - "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "safe-mix 1.0.0 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", - "sr-io 2.0.0", - "sr-primitives 2.0.0", - "sr-staking-primitives 2.0.0", - "sr-std 2.0.0", - "srml-support 2.0.0", - "srml-system 2.0.0", - "srml-timestamp 2.0.0", - "substrate-application-crypto 2.0.0", - "substrate-primitives 2.0.0", - "substrate-trie 2.0.0", + "winapi 0.2.8 (registry+https://github.com/rust-lang/crates.io-index)", + "winapi-build 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "srml-staking" -version = "2.0.0" +name = "slab" +version = "0.4.2" +source = "registry+https://github.com/rust-lang/crates.io-index" + +[[package]] +name = "slog" +version = "2.5.2" +source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "safe-mix 1.0.0 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", - "sr-io 2.0.0", - "sr-primitives 2.0.0", - "sr-staking-primitives 2.0.0", - "sr-std 2.0.0", - "srml-authorship 0.1.0", - "srml-balances 2.0.0", - "srml-session 2.0.0", - "srml-staking-reward-curve 2.0.0", - "srml-support 2.0.0", - "srml-system 2.0.0", - "srml-timestamp 2.0.0", - "substrate-keyring 2.0.0", - "substrate-phragmen 2.0.0", - "substrate-primitives 2.0.0", + "erased-serde 0.3.9 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "srml-staking-reward-curve" -version = "2.0.0" +name = "slog-async" +version = "2.3.0" +source = "git+https://github.com/paritytech/slog-async#107848e7ded5e80dc43f6296c2b96039eb92c0a5" dependencies = [ - "proc-macro-crate 0.1.4 (registry+https://github.com/rust-lang/crates.io-index)", - "proc-macro2 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "quote 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)", - "sr-primitives 2.0.0", - "syn 1.0.7 (registry+https://github.com/rust-lang/crates.io-index)", + "crossbeam-channel 0.3.9 (registry+https://github.com/rust-lang/crates.io-index)", + "slog 2.5.2 (registry+https://github.com/rust-lang/crates.io-index)", + "take_mut 0.2.2 (registry+https://github.com/rust-lang/crates.io-index)", + "thread_local 0.3.6 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "srml-sudo" -version = "2.0.0" +name = "slog-json" +version = "2.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", - "sr-io 2.0.0", - "sr-primitives 2.0.0", - "sr-std 2.0.0", - "srml-support 2.0.0", - "srml-system 2.0.0", - "substrate-primitives 2.0.0", + "chrono 0.4.9 (registry+https://github.com/rust-lang/crates.io-index)", + "erased-serde 0.3.9 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "serde_json 1.0.41 (registry+https://github.com/rust-lang/crates.io-index)", + "slog 2.5.2 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "srml-support" -version = "2.0.0" +name = "slog-scope" +version = "4.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "bitmask 0.5.0 (registry+https://github.com/rust-lang/crates.io-index)", - "impl-trait-for-tuples 0.1.3 (registry+https://github.com/rust-lang/crates.io-index)", - "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", - "once_cell 0.2.4 (registry+https://github.com/rust-lang/crates.io-index)", - "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "paste 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)", - "pretty_assertions 0.6.1 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", - "sr-io 2.0.0", - "sr-primitives 2.0.0", - "sr-std 2.0.0", - "srml-metadata 2.0.0", - "srml-support-procedural 2.0.0", - "srml-system 2.0.0", - "substrate-inherents 2.0.0", - "substrate-primitives 2.0.0", + "arc-swap 0.4.3 (registry+https://github.com/rust-lang/crates.io-index)", + "lazy_static 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)", + "slog 2.5.2 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "srml-support-procedural" -version = "2.0.0" +name = "slog_derive" +version = "0.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "proc-macro2 0.4.30 (registry+https://github.com/rust-lang/crates.io-index)", "quote 0.6.13 (registry+https://github.com/rust-lang/crates.io-index)", - "sr-api-macros 2.0.0", - "srml-support-procedural-tools 2.0.0", "syn 0.15.44 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "srml-support-procedural-tools" -version = "2.0.0" +name = "smallvec" +version = "0.6.13" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "maybe-uninit 2.0.0 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "snow" +version = "0.6.1" +source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "proc-macro-crate 0.1.4 (registry+https://github.com/rust-lang/crates.io-index)", - "proc-macro2 0.4.30 (registry+https://github.com/rust-lang/crates.io-index)", - "quote 0.6.13 (registry+https://github.com/rust-lang/crates.io-index)", - "srml-support-procedural-tools-derive 2.0.0", - "syn 0.15.44 (registry+https://github.com/rust-lang/crates.io-index)", + "arrayref 0.3.5 (registry+https://github.com/rust-lang/crates.io-index)", + "rand_core 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)", + "ring 0.16.9 (registry+https://github.com/rust-lang/crates.io-index)", + "rustc_version 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)", + "subtle 2.2.1 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "srml-support-procedural-tools-derive" -version = "2.0.0" +name = "soketto" +version = "0.2.3" +source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "proc-macro2 0.4.30 (registry+https://github.com/rust-lang/crates.io-index)", - "quote 0.6.13 (registry+https://github.com/rust-lang/crates.io-index)", - "syn 0.15.44 (registry+https://github.com/rust-lang/crates.io-index)", + "base64 0.10.1 (registry+https://github.com/rust-lang/crates.io-index)", + "bytes 0.4.12 (registry+https://github.com/rust-lang/crates.io-index)", + "flate2 1.0.13 (registry+https://github.com/rust-lang/crates.io-index)", + "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", + "http 0.1.19 (registry+https://github.com/rust-lang/crates.io-index)", + "httparse 1.3.4 (registry+https://github.com/rust-lang/crates.io-index)", + "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", + "rand 0.6.5 (registry+https://github.com/rust-lang/crates.io-index)", + "sha1 0.6.0 (registry+https://github.com/rust-lang/crates.io-index)", + "smallvec 0.6.13 (registry+https://github.com/rust-lang/crates.io-index)", + "tokio-codec 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)", + "tokio-io 0.1.12 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "srml-support-test" +name = "sourcefile" +version = "0.1.4" +source = "registry+https://github.com/rust-lang/crates.io-index" + +[[package]] +name = "spin" +version = "0.5.2" +source = "registry+https://github.com/rust-lang/crates.io-index" + +[[package]] +name = "sr-api" version = "2.0.0" dependencies = [ + "criterion 0.3.0 (registry+https://github.com/rust-lang/crates.io-index)", "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "pretty_assertions 0.6.1 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", - "sr-io 2.0.0", + "sr-api-proc-macro 2.0.0", "sr-primitives 2.0.0", - "srml-support 2.0.0", - "substrate-inherents 2.0.0", + "sr-std 2.0.0", + "sr-version 2.0.0", "substrate-primitives 2.0.0", - "trybuild 1.0.17 (registry+https://github.com/rust-lang/crates.io-index)", + "substrate-state-machine 2.0.0", + "substrate-test-runtime-client 2.0.0", ] [[package]] -name = "srml-system" +name = "sr-api-proc-macro" version = "2.0.0" dependencies = [ - "criterion 0.2.11 (registry+https://github.com/rust-lang/crates.io-index)", - "impl-trait-for-tuples 0.1.3 (registry+https://github.com/rust-lang/crates.io-index)", - "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "safe-mix 1.0.0 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", - "sr-io 2.0.0", + "blake2-rfc 0.2.18 (registry+https://github.com/rust-lang/crates.io-index)", + "proc-macro-crate 0.1.4 (registry+https://github.com/rust-lang/crates.io-index)", + "proc-macro2 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "quote 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-api 2.0.0", "sr-primitives 2.0.0", - "sr-std 2.0.0", "sr-version 2.0.0", - "srml-support 2.0.0", - "substrate-primitives 2.0.0", + "substrate-test-runtime-client 2.0.0", + "syn 1.0.8 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "srml-system-rpc" +name = "sr-api-test" version = "2.0.0" dependencies = [ - "env_logger 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)", - "futures-preview 0.3.0-alpha.19 (registry+https://github.com/rust-lang/crates.io-index)", - "jsonrpc-core 13.2.0 (registry+https://github.com/rust-lang/crates.io-index)", - "jsonrpc-core-client 13.2.0 (registry+https://github.com/rust-lang/crates.io-index)", - "jsonrpc-derive 13.2.0 (registry+https://github.com/rust-lang/crates.io-index)", - "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", + "rustversion 1.0.0 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-api 2.0.0", "sr-primitives 2.0.0", - "srml-system-rpc-runtime-api 2.0.0", - "substrate-client 2.0.0", - "substrate-primitives 2.0.0", + "sr-version 2.0.0", + "substrate-consensus-common 2.0.0", + "substrate-state-machine 2.0.0", "substrate-test-runtime-client 2.0.0", - "substrate-transaction-pool 2.0.0", + "trybuild 1.0.17 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "srml-system-rpc-runtime-api" +name = "sr-arithmetic" version = "2.0.0" dependencies = [ + "criterion 0.3.0 (registry+https://github.com/rust-lang/crates.io-index)", + "integer-sqrt 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)", + "num-traits 0.2.8 (registry+https://github.com/rust-lang/crates.io-index)", "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "substrate-client 2.0.0", + "primitive-types 0.6.1 (registry+https://github.com/rust-lang/crates.io-index)", + "rand 0.7.2 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-std 2.0.0", + "substrate-debug-derive 2.0.0", ] [[package]] -name = "srml-timestamp" +name = "sr-io" version = "2.0.0" dependencies = [ - "impl-trait-for-tuples 0.1.3 (registry+https://github.com/rust-lang/crates.io-index)", + "hash-db 0.15.2 (registry+https://github.com/rust-lang/crates.io-index)", + "libsecp256k1 0.3.2 (registry+https://github.com/rust-lang/crates.io-index)", + "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", - "sr-io 2.0.0", - "sr-primitives 2.0.0", "sr-std 2.0.0", - "srml-support 2.0.0", - "srml-system 2.0.0", - "substrate-inherents 2.0.0", + "substrate-externalities 2.0.0", "substrate-primitives 2.0.0", + "substrate-runtime-interface 2.0.0", + "substrate-state-machine 2.0.0", + "substrate-trie 2.0.0", ] [[package]] -name = "srml-transaction-payment" +name = "sr-primitives" version = "2.0.0" dependencies = [ + "impl-trait-for-tuples 0.1.3 (registry+https://github.com/rust-lang/crates.io-index)", + "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "paste 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)", + "rand 0.7.2 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "serde_json 1.0.41 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-arithmetic 2.0.0", "sr-io 2.0.0", - "sr-primitives 2.0.0", "sr-std 2.0.0", - "srml-balances 2.0.0", - "srml-support 2.0.0", - "srml-system 2.0.0", - "srml-transaction-payment-rpc-runtime-api 2.0.0", + "substrate-application-crypto 2.0.0", + "substrate-inherents 2.0.0", + "substrate-offchain 2.0.0", "substrate-primitives 2.0.0", ] [[package]] -name = "srml-transaction-payment-rpc" +name = "sr-sandbox" version = "2.0.0" dependencies = [ - "jsonrpc-core 13.2.0 (registry+https://github.com/rust-lang/crates.io-index)", - "jsonrpc-core-client 13.2.0 (registry+https://github.com/rust-lang/crates.io-index)", - "jsonrpc-derive 13.2.0 (registry+https://github.com/rust-lang/crates.io-index)", + "assert_matches 1.3.0 (registry+https://github.com/rust-lang/crates.io-index)", "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", - "sr-primitives 2.0.0", - "srml-transaction-payment-rpc-runtime-api 2.0.0", - "substrate-client 2.0.0", + "sr-io 2.0.0", + "sr-std 2.0.0", "substrate-primitives 2.0.0", - "substrate-rpc-primitives 2.0.0", + "wabt 0.9.2 (registry+https://github.com/rust-lang/crates.io-index)", + "wasmi 0.6.2 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] -name = "srml-transaction-payment-rpc-runtime-api" +name = "sr-staking-primitives" version = "2.0.0" dependencies = [ "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", "sr-primitives 2.0.0", "sr-std 2.0.0", - "substrate-client 2.0.0", ] [[package]] -name = "srml-treasury" +name = "sr-std" version = "2.0.0" -dependencies = [ - "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", - "sr-io 2.0.0", - "sr-primitives 2.0.0", - "sr-std 2.0.0", - "srml-balances 2.0.0", - "srml-support 2.0.0", - "srml-system 2.0.0", - "substrate-primitives 2.0.0", -] [[package]] -name = "srml-utility" +name = "sr-version" version = "2.0.0" dependencies = [ + "impl-serde 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)", "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", - "sr-io 2.0.0", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", "sr-primitives 2.0.0", "sr-std 2.0.0", - "srml-balances 2.0.0", - "srml-support 2.0.0", - "srml-system 2.0.0", - "substrate-primitives 2.0.0", ] [[package]] @@ -4691,12 +5435,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" [[package]] name = "static_assertions" -version = "0.2.5" -source = "registry+https://github.com/rust-lang/crates.io-index" - -[[package]] -name = "static_slice" -version = "0.0.3" +version = "1.1.0" source = "registry+https://github.com/rust-lang/crates.io-index" [[package]] @@ -4715,6 +5454,14 @@ dependencies = [ "bytes 0.4.12 (registry+https://github.com/rust-lang/crates.io-index)", ] +[[package]] +name = "string-interner" +version = "0.7.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", +] + [[package]] name = "strsim" version = "0.8.0" @@ -4722,23 +5469,23 @@ source = "registry+https://github.com/rust-lang/crates.io-index" [[package]] name = "structopt" -version = "0.3.3" +version = "0.3.4" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "clap 2.33.0 (registry+https://github.com/rust-lang/crates.io-index)", - "structopt-derive 0.3.3 (registry+https://github.com/rust-lang/crates.io-index)", + "structopt-derive 0.3.4 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] name = "structopt-derive" -version = "0.3.3" +version = "0.3.4" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "heck 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", "proc-macro-error 0.2.6 (registry+https://github.com/rust-lang/crates.io-index)", "proc-macro2 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", "quote 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)", - "syn 1.0.7 (registry+https://github.com/rust-lang/crates.io-index)", + "syn 1.0.8 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] @@ -4766,13 +5513,13 @@ dependencies = [ "hex-literal 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)", "node-primitives 2.0.0", "node-runtime 2.0.0", + "paint-balances 2.0.0", + "paint-system 2.0.0", + "paint-transaction-payment 2.0.0", "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", "rand 0.7.2 (registry+https://github.com/rust-lang/crates.io-index)", "rustc-hex 2.0.1 (registry+https://github.com/rust-lang/crates.io-index)", "sr-primitives 2.0.0", - "srml-balances 2.0.0", - "srml-system 2.0.0", - "srml-transaction-payment 2.0.0", "substrate-bip39 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", "substrate-primitives 2.0.0", "tiny-bip39 0.6.2 (registry+https://github.com/rust-lang/crates.io-index)", @@ -4783,7 +5530,7 @@ name = "substrate-application-crypto" version = "2.0.0" dependencies = [ "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", "sr-io 2.0.0", "sr-primitives 2.0.0", "sr-std 2.0.0", @@ -4797,23 +5544,24 @@ version = "2.0.0" dependencies = [ "bytes 0.4.12 (registry+https://github.com/rust-lang/crates.io-index)", "derive_more 0.15.0 (registry+https://github.com/rust-lang/crates.io-index)", - "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", - "libp2p 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)", + "futures-preview 0.3.0-alpha.19 (registry+https://github.com/rust-lang/crates.io-index)", + "futures-timer 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)", + "libp2p 0.13.0 (registry+https://github.com/rust-lang/crates.io-index)", "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", "parking_lot 0.9.0 (registry+https://github.com/rust-lang/crates.io-index)", "prost 0.5.0 (registry+https://github.com/rust-lang/crates.io-index)", "prost-build 0.5.0 (registry+https://github.com/rust-lang/crates.io-index)", "serde_json 1.0.41 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-api 2.0.0", "sr-primitives 2.0.0", "substrate-authority-discovery-primitives 2.0.0", - "substrate-client 2.0.0", + "substrate-client-api 2.0.0", + "substrate-keystore 2.0.0", "substrate-network 2.0.0", "substrate-peerset 2.0.0", "substrate-primitives 2.0.0", "substrate-test-runtime-client 2.0.0", - "tokio 0.1.22 (registry+https://github.com/rust-lang/crates.io-index)", - "tokio-timer 0.2.11 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] @@ -4821,9 +5569,10 @@ name = "substrate-authority-discovery-primitives" version = "2.0.0" dependencies = [ "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-api 2.0.0", "sr-primitives 2.0.0", "sr-std 2.0.0", - "substrate-client 2.0.0", + "substrate-application-crypto 2.0.0", ] [[package]] @@ -4834,7 +5583,9 @@ dependencies = [ "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", "sr-primitives 2.0.0", + "substrate-block-builder 2.0.0", "substrate-client 2.0.0", + "substrate-client-api 2.0.0", "substrate-consensus-common 2.0.0", "substrate-inherents 2.0.0", "substrate-primitives 2.0.0", @@ -4854,12 +5605,38 @@ dependencies = [ "sha2 0.8.0 (registry+https://github.com/rust-lang/crates.io-index)", ] +[[package]] +name = "substrate-block-builder" +version = "2.0.0" +dependencies = [ + "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-api 2.0.0", + "sr-primitives 2.0.0", + "substrate-block-builder-runtime-api 2.0.0", + "substrate-primitives 2.0.0", + "substrate-state-machine 2.0.0", +] + +[[package]] +name = "substrate-block-builder-runtime-api" +version = "2.0.0" +dependencies = [ + "sr-api 2.0.0", + "sr-primitives 2.0.0", + "sr-std 2.0.0", + "substrate-inherents 2.0.0", +] + +[[package]] +name = "substrate-build-script-utils" +version = "2.0.0" + [[package]] name = "substrate-chain-spec" version = "2.0.0" dependencies = [ "impl-trait-for-tuples 0.1.3 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", "serde_json 1.0.41 (registry+https://github.com/rust-lang/crates.io-index)", "sr-primitives 2.0.0", "substrate-chain-spec-derive 2.0.0", @@ -4875,7 +5652,7 @@ dependencies = [ "proc-macro-crate 0.1.4 (registry+https://github.com/rust-lang/crates.io-index)", "proc-macro2 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", "quote 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)", - "syn 1.0.7 (registry+https://github.com/rust-lang/crates.io-index)", + "syn 1.0.8 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] @@ -4887,7 +5664,7 @@ dependencies = [ "atty 0.2.13 (registry+https://github.com/rust-lang/crates.io-index)", "clap 2.33.0 (registry+https://github.com/rust-lang/crates.io-index)", "derive_more 0.15.0 (registry+https://github.com/rust-lang/crates.io-index)", - "env_logger 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)", + "env_logger 0.7.1 (registry+https://github.com/rust-lang/crates.io-index)", "exit-future 0.1.4 (registry+https://github.com/rust-lang/crates.io-index)", "fdlimit 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)", "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", @@ -4899,13 +5676,14 @@ dependencies = [ "rpassword 4.0.1 (registry+https://github.com/rust-lang/crates.io-index)", "serde_json 1.0.41 (registry+https://github.com/rust-lang/crates.io-index)", "sr-primitives 2.0.0", - "structopt 0.3.3 (registry+https://github.com/rust-lang/crates.io-index)", - "substrate-client 2.0.0", + "structopt 0.3.4 (registry+https://github.com/rust-lang/crates.io-index)", + "substrate-client-api 2.0.0", "substrate-header-metadata 2.0.0", "substrate-keyring 2.0.0", "substrate-network 2.0.0", "substrate-panic-handler 2.0.0", "substrate-primitives 2.0.0", + "substrate-prometheus 2.0.0", "substrate-service 2.0.0", "substrate-state-machine 2.0.0", "substrate-telemetry 2.0.0", @@ -4919,7 +5697,45 @@ name = "substrate-client" version = "2.0.0" dependencies = [ "derive_more 0.15.0 (registry+https://github.com/rust-lang/crates.io-index)", - "env_logger 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)", + "env_logger 0.7.1 (registry+https://github.com/rust-lang/crates.io-index)", + "fnv 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", + "futures-preview 0.3.0-alpha.19 (registry+https://github.com/rust-lang/crates.io-index)", + "hash-db 0.15.2 (registry+https://github.com/rust-lang/crates.io-index)", + "hex-literal 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)", + "kvdb 0.1.0 (git+https://github.com/paritytech/parity-common?rev=b0317f649ab2c665b7987b8475878fc4d2e1f81d)", + "kvdb-memorydb 0.1.0 (git+https://github.com/paritytech/parity-common?rev=b0317f649ab2c665b7987b8475878fc4d2e1f81d)", + "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", + "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "parking_lot 0.9.0 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-api 2.0.0", + "sr-primitives 2.0.0", + "sr-std 2.0.0", + "sr-version 2.0.0", + "substrate-block-builder 2.0.0", + "substrate-client-api 2.0.0", + "substrate-client-db 2.0.0", + "substrate-consensus-common 2.0.0", + "substrate-executor 2.0.0", + "substrate-header-metadata 2.0.0", + "substrate-inherents 2.0.0", + "substrate-keyring 2.0.0", + "substrate-panic-handler 2.0.0", + "substrate-primitives 2.0.0", + "substrate-prometheus 2.0.0", + "substrate-state-machine 2.0.0", + "substrate-telemetry 2.0.0", + "substrate-test-runtime-client 2.0.0", + "substrate-trie 2.0.0", + "tempfile 3.1.0 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "substrate-client-api" +version = "2.0.0" +dependencies = [ + "derive_more 0.15.0 (registry+https://github.com/rust-lang/crates.io-index)", + "env_logger 0.7.1 (registry+https://github.com/rust-lang/crates.io-index)", "fnv 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", "futures-preview 0.3.0-alpha.19 (registry+https://github.com/rust-lang/crates.io-index)", @@ -4930,10 +5746,11 @@ dependencies = [ "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", "parking_lot 0.9.0 (registry+https://github.com/rust-lang/crates.io-index)", - "sr-api-macros 2.0.0", + "sr-api 2.0.0", "sr-primitives 2.0.0", "sr-std 2.0.0", "sr-version 2.0.0", + "substrate-block-builder 2.0.0", "substrate-client-db 2.0.0", "substrate-consensus-common 2.0.0", "substrate-executor 2.0.0", @@ -4942,8 +5759,10 @@ dependencies = [ "substrate-keyring 2.0.0", "substrate-panic-handler 2.0.0", "substrate-primitives 2.0.0", + "substrate-prometheus 2.0.0", "substrate-state-machine 2.0.0", "substrate-telemetry 2.0.0", + "substrate-test-primitives 2.0.0", "substrate-test-runtime-client 2.0.0", "substrate-trie 2.0.0", "tempfile 3.1.0 (registry+https://github.com/rust-lang/crates.io-index)", @@ -4953,7 +5772,7 @@ dependencies = [ name = "substrate-client-db" version = "2.0.0" dependencies = [ - "env_logger 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)", + "env_logger 0.7.1 (registry+https://github.com/rust-lang/crates.io-index)", "hash-db 0.15.2 (registry+https://github.com/rust-lang/crates.io-index)", "kvdb 0.1.0 (git+https://github.com/paritytech/parity-common?rev=b0317f649ab2c665b7987b8475878fc4d2e1f81d)", "kvdb-memorydb 0.1.0 (git+https://github.com/paritytech/parity-common?rev=b0317f649ab2c665b7987b8475878fc4d2e1f81d)", @@ -4962,8 +5781,10 @@ dependencies = [ "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", "parking_lot 0.9.0 (registry+https://github.com/rust-lang/crates.io-index)", + "quickcheck 0.8.5 (registry+https://github.com/rust-lang/crates.io-index)", "sr-primitives 2.0.0", "substrate-client 2.0.0", + "substrate-client-api 2.0.0", "substrate-consensus-common 2.0.0", "substrate-executor 2.0.0", "substrate-header-metadata 2.0.0", @@ -4980,20 +5801,22 @@ name = "substrate-consensus-aura" version = "2.0.0" dependencies = [ "derive_more 0.15.0 (registry+https://github.com/rust-lang/crates.io-index)", - "env_logger 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)", + "env_logger 0.7.1 (registry+https://github.com/rust-lang/crates.io-index)", "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", "futures-preview 0.3.0-alpha.19 (registry+https://github.com/rust-lang/crates.io-index)", "futures-timer 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)", "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", + "paint-aura 2.0.0", "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", "parking_lot 0.9.0 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-api 2.0.0", "sr-io 2.0.0", "sr-primitives 2.0.0", "sr-version 2.0.0", - "srml-aura 2.0.0", - "srml-support 2.0.0", "substrate-application-crypto 2.0.0", + "substrate-block-builder-runtime-api 2.0.0", "substrate-client 2.0.0", + "substrate-client-api 2.0.0", "substrate-consensus-aura-primitives 2.0.0", "substrate-consensus-common 2.0.0", "substrate-consensus-slots 2.0.0", @@ -5003,6 +5826,7 @@ dependencies = [ "substrate-keystore 2.0.0", "substrate-network 2.0.0", "substrate-primitives 2.0.0", + "substrate-prometheus 2.0.0", "substrate-service 2.0.0", "substrate-telemetry 2.0.0", "substrate-test-runtime-client 2.0.0", @@ -5015,10 +5839,10 @@ name = "substrate-consensus-aura-primitives" version = "2.0.0" dependencies = [ "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-api 2.0.0", "sr-primitives 2.0.0", "sr-std 2.0.0", "substrate-application-crypto 2.0.0", - "substrate-client 2.0.0", ] [[package]] @@ -5026,7 +5850,7 @@ name = "substrate-consensus-babe" version = "2.0.0" dependencies = [ "derive_more 0.15.0 (registry+https://github.com/rust-lang/crates.io-index)", - "env_logger 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)", + "env_logger 0.7.1 (registry+https://github.com/rust-lang/crates.io-index)", "fork-tree 2.0.0", "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", "futures-preview 0.3.0-alpha.19 (registry+https://github.com/rust-lang/crates.io-index)", @@ -5036,18 +5860,21 @@ dependencies = [ "num-bigint 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)", "num-rational 0.2.2 (registry+https://github.com/rust-lang/crates.io-index)", "num-traits 0.2.8 (registry+https://github.com/rust-lang/crates.io-index)", + "paint-babe 2.0.0", "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", "parking_lot 0.9.0 (registry+https://github.com/rust-lang/crates.io-index)", "pdqselect 0.1.0 (registry+https://github.com/rust-lang/crates.io-index)", "rand 0.7.2 (registry+https://github.com/rust-lang/crates.io-index)", "schnorrkel 0.8.5 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-api 2.0.0", "sr-io 2.0.0", "sr-primitives 2.0.0", "sr-version 2.0.0", - "srml-babe 2.0.0", - "srml-support 2.0.0", "substrate-application-crypto 2.0.0", + "substrate-block-builder 2.0.0", + "substrate-block-builder-runtime-api 2.0.0", "substrate-client 2.0.0", + "substrate-client-api 2.0.0", "substrate-consensus-babe-primitives 2.0.0", "substrate-consensus-common 2.0.0", "substrate-consensus-slots 2.0.0", @@ -5072,10 +5899,10 @@ version = "2.0.0" dependencies = [ "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", "schnorrkel 0.8.5 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-api 2.0.0", "sr-primitives 2.0.0", "sr-std 2.0.0", "substrate-application-crypto 2.0.0", - "substrate-client 2.0.0", "substrate-consensus-slots 2.0.0", ] @@ -5086,7 +5913,7 @@ dependencies = [ "derive_more 0.15.0 (registry+https://github.com/rust-lang/crates.io-index)", "futures-preview 0.3.0-alpha.19 (registry+https://github.com/rust-lang/crates.io-index)", "futures-timer 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)", - "libp2p 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)", + "libp2p 0.13.0 (registry+https://github.com/rust-lang/crates.io-index)", "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", "parking_lot 0.9.0 (registry+https://github.com/rust-lang/crates.io-index)", @@ -5105,14 +5932,16 @@ dependencies = [ "derive_more 0.15.0 (registry+https://github.com/rust-lang/crates.io-index)", "futures-preview 0.3.0-alpha.19 (registry+https://github.com/rust-lang/crates.io-index)", "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", + "paint-timestamp 2.0.0", "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", "sr-primitives 2.0.0", - "srml-timestamp 2.0.0", - "substrate-client 2.0.0", + "substrate-block-builder-runtime-api 2.0.0", + "substrate-client-api 2.0.0", "substrate-consensus-common 2.0.0", "substrate-consensus-pow-primitives 2.0.0", "substrate-inherents 2.0.0", "substrate-primitives 2.0.0", + "substrate-prometheus 2.0.0", ] [[package]] @@ -5120,34 +5949,10 @@ name = "substrate-consensus-pow-primitives" version = "2.0.0" dependencies = [ "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-api 2.0.0", "sr-primitives 2.0.0", "sr-std 2.0.0", - "substrate-client 2.0.0", - "substrate-primitives 2.0.0", -] - -[[package]] -name = "substrate-consensus-rhd" -version = "2.0.0" -dependencies = [ - "derive_more 0.15.0 (registry+https://github.com/rust-lang/crates.io-index)", - "exit-future 0.1.4 (registry+https://github.com/rust-lang/crates.io-index)", - "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", - "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", - "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "parking_lot 0.9.0 (registry+https://github.com/rust-lang/crates.io-index)", - "rhododendron 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)", - "sr-io 2.0.0", - "sr-primitives 2.0.0", - "sr-version 2.0.0", - "srml-support 2.0.0", - "srml-system 2.0.0", - "substrate-client 2.0.0", - "substrate-consensus-common 2.0.0", - "substrate-keyring 2.0.0", "substrate-primitives 2.0.0", - "substrate-transaction-pool 2.0.0", - "tokio 0.1.22 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] @@ -5160,7 +5965,7 @@ dependencies = [ "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", "parking_lot 0.9.0 (registry+https://github.com/rust-lang/crates.io-index)", "sr-primitives 2.0.0", - "substrate-client 2.0.0", + "substrate-client-api 2.0.0", "substrate-consensus-common 2.0.0", "substrate-inherents 2.0.0", "substrate-primitives 2.0.0", @@ -5173,9 +5978,9 @@ name = "substrate-consensus-uncles" version = "2.0.0" dependencies = [ "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", + "paint-authorship 0.1.0", "sr-primitives 2.0.0", - "srml-authorship 0.1.0", - "substrate-client 2.0.0", + "substrate-client-api 2.0.0", "substrate-consensus-common 2.0.0", "substrate-inherents 2.0.0", "substrate-primitives 2.0.0", @@ -5187,7 +5992,7 @@ version = "2.0.0" dependencies = [ "proc-macro2 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", "quote 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)", - "syn 1.0.7 (registry+https://github.com/rust-lang/crates.io-index)", + "syn 1.0.8 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] @@ -5195,29 +6000,38 @@ name = "substrate-executor" version = "2.0.0" dependencies = [ "assert_matches 1.3.0 (registry+https://github.com/rust-lang/crates.io-index)", + "cranelift-codegen 0.46.1 (registry+https://github.com/rust-lang/crates.io-index)", + "cranelift-entity 0.46.1 (registry+https://github.com/rust-lang/crates.io-index)", + "cranelift-frontend 0.46.1 (registry+https://github.com/rust-lang/crates.io-index)", + "cranelift-native 0.46.1 (registry+https://github.com/rust-lang/crates.io-index)", + "cranelift-wasm 0.46.1 (registry+https://github.com/rust-lang/crates.io-index)", "derive_more 0.15.0 (registry+https://github.com/rust-lang/crates.io-index)", "hex-literal 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)", "lazy_static 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)", - "libsecp256k1 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", + "libsecp256k1 0.3.2 (registry+https://github.com/rust-lang/crates.io-index)", "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", "parity-wasm 0.40.3 (registry+https://github.com/rust-lang/crates.io-index)", "parking_lot 0.9.0 (registry+https://github.com/rust-lang/crates.io-index)", "sr-io 2.0.0", "sr-version 2.0.0", - "substrate-client 2.0.0", + "substrate-client-api 2.0.0", "substrate-externalities 2.0.0", "substrate-offchain 2.0.0", "substrate-panic-handler 2.0.0", "substrate-primitives 2.0.0", + "substrate-runtime-interface 2.0.0", "substrate-runtime-test 2.0.0", "substrate-serializer 2.0.0", "substrate-state-machine 2.0.0", "substrate-trie 2.0.0", "substrate-wasm-interface 2.0.0", - "tiny-keccak 1.5.0 (registry+https://github.com/rust-lang/crates.io-index)", + "test-case 0.3.3 (registry+https://github.com/rust-lang/crates.io-index)", "wabt 0.9.2 (registry+https://github.com/rust-lang/crates.io-index)", - "wasmi 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)", + "wasmi 0.6.2 (registry+https://github.com/rust-lang/crates.io-index)", + "wasmtime-environ 0.2.0 (git+https://github.com/CraneStation/wasmtime.git?rev=71dd73d6)", + "wasmtime-jit 0.2.0 (git+https://github.com/CraneStation/wasmtime.git?rev=71dd73d6)", + "wasmtime-runtime 0.2.0 (git+https://github.com/CraneStation/wasmtime.git?rev=71dd73d6)", ] [[package]] @@ -5225,7 +6039,7 @@ name = "substrate-externalities" version = "2.0.0" dependencies = [ "environmental 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)", - "primitive-types 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)", + "primitive-types 0.6.1 (registry+https://github.com/rust-lang/crates.io-index)", "sr-std 2.0.0", "substrate-primitives-storage 2.0.0", ] @@ -5234,19 +6048,21 @@ dependencies = [ name = "substrate-finality-grandpa" version = "2.0.0" dependencies = [ - "env_logger 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)", - "finality-grandpa 0.9.0 (registry+https://github.com/rust-lang/crates.io-index)", + "env_logger 0.7.1 (registry+https://github.com/rust-lang/crates.io-index)", + "finality-grandpa 0.9.1 (registry+https://github.com/rust-lang/crates.io-index)", "fork-tree 2.0.0", "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", "futures-preview 0.3.0-alpha.19 (registry+https://github.com/rust-lang/crates.io-index)", "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", + "paint-finality-tracker 2.0.0", "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", "parking_lot 0.9.0 (registry+https://github.com/rust-lang/crates.io-index)", "rand 0.7.2 (registry+https://github.com/rust-lang/crates.io-index)", "serde_json 1.0.41 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-api 2.0.0", "sr-primitives 2.0.0", - "srml-finality-tracker 2.0.0", "substrate-client 2.0.0", + "substrate-client-api 2.0.0", "substrate-consensus-babe-primitives 2.0.0", "substrate-consensus-common 2.0.0", "substrate-finality-grandpa-primitives 2.0.0", @@ -5256,6 +6072,8 @@ dependencies = [ "substrate-keystore 2.0.0", "substrate-network 2.0.0", "substrate-primitives 2.0.0", + "substrate-prometheus 2.0.0", + "substrate-state-machine 2.0.0", "substrate-telemetry 2.0.0", "substrate-test-runtime-client 2.0.0", "tempfile 3.1.0 (registry+https://github.com/rust-lang/crates.io-index)", @@ -5269,11 +6087,11 @@ name = "substrate-finality-grandpa-primitives" version = "2.0.0" dependencies = [ "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-api 2.0.0", "sr-primitives 2.0.0", "sr-std 2.0.0", "substrate-application-crypto 2.0.0", - "substrate-client 2.0.0", ] [[package]] @@ -5289,10 +6107,11 @@ dependencies = [ name = "substrate-inherents" version = "2.0.0" dependencies = [ + "derive_more 0.15.0 (registry+https://github.com/rust-lang/crates.io-index)", "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", "parking_lot 0.9.0 (registry+https://github.com/rust-lang/crates.io-index)", - "sr-primitives 2.0.0", "sr-std 2.0.0", + "substrate-primitives 2.0.0", ] [[package]] @@ -5325,18 +6144,18 @@ dependencies = [ name = "substrate-network" version = "2.0.0" dependencies = [ - "bitflags 1.2.0 (registry+https://github.com/rust-lang/crates.io-index)", + "bitflags 1.2.1 (registry+https://github.com/rust-lang/crates.io-index)", "bytes 0.4.12 (registry+https://github.com/rust-lang/crates.io-index)", "derive_more 0.15.0 (registry+https://github.com/rust-lang/crates.io-index)", "either 1.5.3 (registry+https://github.com/rust-lang/crates.io-index)", - "env_logger 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)", + "env_logger 0.7.1 (registry+https://github.com/rust-lang/crates.io-index)", "erased-serde 0.3.9 (registry+https://github.com/rust-lang/crates.io-index)", "fnv 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", "fork-tree 2.0.0", "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", "futures-preview 0.3.0-alpha.19 (registry+https://github.com/rust-lang/crates.io-index)", "futures-timer 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)", - "libp2p 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)", + "libp2p 0.13.0 (registry+https://github.com/rust-lang/crates.io-index)", "linked-hash-map 0.5.2 (registry+https://github.com/rust-lang/crates.io-index)", "linked_hash_set 0.1.3 (registry+https://github.com/rust-lang/crates.io-index)", "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", @@ -5346,13 +6165,16 @@ dependencies = [ "quickcheck 0.9.0 (registry+https://github.com/rust-lang/crates.io-index)", "rand 0.7.2 (registry+https://github.com/rust-lang/crates.io-index)", "rustc-hex 2.0.1 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", "serde_json 1.0.41 (registry+https://github.com/rust-lang/crates.io-index)", "slog 2.5.2 (registry+https://github.com/rust-lang/crates.io-index)", "slog_derive 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)", - "smallvec 0.6.10 (registry+https://github.com/rust-lang/crates.io-index)", + "smallvec 0.6.13 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-arithmetic 2.0.0", "sr-primitives 2.0.0", + "substrate-block-builder 2.0.0", "substrate-client 2.0.0", + "substrate-client-api 2.0.0", "substrate-consensus-babe-primitives 2.0.0", "substrate-consensus-common 2.0.0", "substrate-header-metadata 2.0.0", @@ -5375,20 +6197,21 @@ name = "substrate-offchain" version = "2.0.0" dependencies = [ "bytes 0.4.12 (registry+https://github.com/rust-lang/crates.io-index)", - "env_logger 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)", + "env_logger 0.7.1 (registry+https://github.com/rust-lang/crates.io-index)", "fnv 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", "futures-preview 0.3.0-alpha.19 (registry+https://github.com/rust-lang/crates.io-index)", "futures-timer 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)", "hyper 0.12.35 (registry+https://github.com/rust-lang/crates.io-index)", - "hyper-tls 0.3.2 (registry+https://github.com/rust-lang/crates.io-index)", + "hyper-rustls 0.17.1 (registry+https://github.com/rust-lang/crates.io-index)", "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", - "num_cpus 1.10.1 (registry+https://github.com/rust-lang/crates.io-index)", + "num_cpus 1.11.0 (registry+https://github.com/rust-lang/crates.io-index)", "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", "parking_lot 0.9.0 (registry+https://github.com/rust-lang/crates.io-index)", "rand 0.7.2 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-api 2.0.0", "sr-primitives 2.0.0", - "substrate-client 2.0.0", + "substrate-client-api 2.0.0", "substrate-client-db 2.0.0", "substrate-keystore 2.0.0", "substrate-network 2.0.0", @@ -5404,15 +6227,15 @@ dependencies = [ name = "substrate-offchain-primitives" version = "2.0.0" dependencies = [ + "sr-api 2.0.0", "sr-primitives 2.0.0", - "substrate-client 2.0.0", ] [[package]] name = "substrate-panic-handler" version = "2.0.0" dependencies = [ - "backtrace 0.3.38 (registry+https://github.com/rust-lang/crates.io-index)", + "backtrace 0.3.40 (registry+https://github.com/rust-lang/crates.io-index)", "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", ] @@ -5421,7 +6244,7 @@ name = "substrate-peerset" version = "2.0.0" dependencies = [ "futures-preview 0.3.0-alpha.19 (registry+https://github.com/rust-lang/crates.io-index)", - "libp2p 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)", + "libp2p 0.13.0 (registry+https://github.com/rust-lang/crates.io-index)", "linked-hash-map 0.5.2 (registry+https://github.com/rust-lang/crates.io-index)", "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", "lru-cache 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)", @@ -5433,11 +6256,12 @@ dependencies = [ name = "substrate-phragmen" version = "2.0.0" dependencies = [ + "paint-support 2.0.0", "rand 0.7.2 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", "sr-io 2.0.0", "sr-primitives 2.0.0", "sr-std 2.0.0", - "srml-support 2.0.0", ] [[package]] @@ -5451,33 +6275,34 @@ dependencies = [ "ed25519-dalek 0.9.1 (registry+https://github.com/rust-lang/crates.io-index)", "hash-db 0.15.2 (registry+https://github.com/rust-lang/crates.io-index)", "hash256-std-hasher 0.15.2 (registry+https://github.com/rust-lang/crates.io-index)", - "hex 0.3.2 (registry+https://github.com/rust-lang/crates.io-index)", + "hex 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)", "hex-literal 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)", "impl-serde 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)", "lazy_static 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)", - "libsecp256k1 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", + "libsecp256k1 0.3.2 (registry+https://github.com/rust-lang/crates.io-index)", "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", "num-traits 0.2.8 (registry+https://github.com/rust-lang/crates.io-index)", "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", "parking_lot 0.9.0 (registry+https://github.com/rust-lang/crates.io-index)", "pretty_assertions 0.6.1 (registry+https://github.com/rust-lang/crates.io-index)", - "primitive-types 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)", + "primitive-types 0.6.1 (registry+https://github.com/rust-lang/crates.io-index)", "rand 0.7.2 (registry+https://github.com/rust-lang/crates.io-index)", "regex 1.3.1 (registry+https://github.com/rust-lang/crates.io-index)", "rustc-hex 2.0.1 (registry+https://github.com/rust-lang/crates.io-index)", "schnorrkel 0.8.5 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", "sha2 0.8.0 (registry+https://github.com/rust-lang/crates.io-index)", "sr-std 2.0.0", "substrate-bip39 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", "substrate-debug-derive 2.0.0", "substrate-externalities 2.0.0", "substrate-primitives-storage 2.0.0", + "substrate-runtime-interface 2.0.0", "substrate-serializer 2.0.0", "tiny-bip39 0.6.2 (registry+https://github.com/rust-lang/crates.io-index)", - "tiny-keccak 1.5.0 (registry+https://github.com/rust-lang/crates.io-index)", + "tiny-keccak 2.0.1 (registry+https://github.com/rust-lang/crates.io-index)", "twox-hash 1.5.0 (registry+https://github.com/rust-lang/crates.io-index)", - "wasmi 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)", + "wasmi 0.6.2 (registry+https://github.com/rust-lang/crates.io-index)", "zeroize 0.10.1 (registry+https://github.com/rust-lang/crates.io-index)", ] @@ -5486,11 +6311,24 @@ name = "substrate-primitives-storage" version = "2.0.0" dependencies = [ "impl-serde 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", "sr-std 2.0.0", "substrate-debug-derive 2.0.0", ] +[[package]] +name = "substrate-prometheus" +version = "2.0.0" +dependencies = [ + "hyper 0.12.35 (registry+https://github.com/rust-lang/crates.io-index)", + "lazy_static 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)", + "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", + "prometheus 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)", + "reqwest 0.9.22 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-primitives 2.0.0", + "tokio 0.1.22 (registry+https://github.com/rust-lang/crates.io-index)", +] + [[package]] name = "substrate-rpc" version = "2.0.0" @@ -5499,17 +6337,19 @@ dependencies = [ "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", "futures-preview 0.3.0-alpha.19 (registry+https://github.com/rust-lang/crates.io-index)", "hash-db 0.15.2 (registry+https://github.com/rust-lang/crates.io-index)", - "jsonrpc-core 13.2.0 (registry+https://github.com/rust-lang/crates.io-index)", - "jsonrpc-pubsub 13.2.0 (registry+https://github.com/rust-lang/crates.io-index)", + "jsonrpc-core 14.0.3 (registry+https://github.com/rust-lang/crates.io-index)", + "jsonrpc-pubsub 14.0.3 (registry+https://github.com/rust-lang/crates.io-index)", "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", "parking_lot 0.9.0 (registry+https://github.com/rust-lang/crates.io-index)", "rustc-hex 2.0.1 (registry+https://github.com/rust-lang/crates.io-index)", "serde_json 1.0.41 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-api 2.0.0", "sr-io 2.0.0", "sr-primitives 2.0.0", "sr-version 2.0.0", "substrate-client 2.0.0", + "substrate-client-api 2.0.0", "substrate-executor 2.0.0", "substrate-keystore 2.0.0", "substrate-network 2.0.0", @@ -5529,14 +6369,14 @@ version = "2.0.0" dependencies = [ "derive_more 0.15.0 (registry+https://github.com/rust-lang/crates.io-index)", "futures-preview 0.3.0-alpha.19 (registry+https://github.com/rust-lang/crates.io-index)", - "jsonrpc-core 13.2.0 (registry+https://github.com/rust-lang/crates.io-index)", - "jsonrpc-core-client 13.2.0 (registry+https://github.com/rust-lang/crates.io-index)", - "jsonrpc-derive 13.2.0 (registry+https://github.com/rust-lang/crates.io-index)", - "jsonrpc-pubsub 13.2.0 (registry+https://github.com/rust-lang/crates.io-index)", + "jsonrpc-core 14.0.3 (registry+https://github.com/rust-lang/crates.io-index)", + "jsonrpc-core-client 14.0.3 (registry+https://github.com/rust-lang/crates.io-index)", + "jsonrpc-derive 14.0.3 (registry+https://github.com/rust-lang/crates.io-index)", + "jsonrpc-pubsub 14.0.3 (registry+https://github.com/rust-lang/crates.io-index)", "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", "parking_lot 0.9.0 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", "serde_json 1.0.41 (registry+https://github.com/rust-lang/crates.io-index)", "sr-version 2.0.0", "substrate-primitives 2.0.0", @@ -5548,7 +6388,7 @@ dependencies = [ name = "substrate-rpc-primitives" version = "2.0.0" dependencies = [ - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", "substrate-primitives 2.0.0", ] @@ -5556,16 +6396,62 @@ dependencies = [ name = "substrate-rpc-servers" version = "2.0.0" dependencies = [ - "jsonrpc-core 13.2.0 (registry+https://github.com/rust-lang/crates.io-index)", - "jsonrpc-http-server 13.2.0 (registry+https://github.com/rust-lang/crates.io-index)", - "jsonrpc-pubsub 13.2.0 (registry+https://github.com/rust-lang/crates.io-index)", - "jsonrpc-ws-server 13.2.0 (registry+https://github.com/rust-lang/crates.io-index)", + "jsonrpc-core 14.0.3 (registry+https://github.com/rust-lang/crates.io-index)", + "jsonrpc-http-server 14.0.3 (registry+https://github.com/rust-lang/crates.io-index)", + "jsonrpc-pubsub 14.0.3 (registry+https://github.com/rust-lang/crates.io-index)", + "jsonrpc-ws-server 14.0.3 (registry+https://github.com/rust-lang/crates.io-index)", "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", "serde_json 1.0.41 (registry+https://github.com/rust-lang/crates.io-index)", "sr-primitives 2.0.0", ] +[[package]] +name = "substrate-runtime-interface" +version = "2.0.0" +dependencies = [ + "environmental 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)", + "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "primitive-types 0.6.1 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-io 2.0.0", + "sr-std 2.0.0", + "static_assertions 1.1.0 (registry+https://github.com/rust-lang/crates.io-index)", + "substrate-executor 2.0.0", + "substrate-externalities 2.0.0", + "substrate-primitives 2.0.0", + "substrate-runtime-interface-proc-macro 2.0.0", + "substrate-runtime-interface-test-wasm 2.0.0", + "substrate-state-machine 2.0.0", + "substrate-wasm-interface 2.0.0", +] + +[[package]] +name = "substrate-runtime-interface-proc-macro" +version = "2.0.0" +dependencies = [ + "Inflector 0.11.4 (registry+https://github.com/rust-lang/crates.io-index)", + "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "proc-macro-crate 0.1.4 (registry+https://github.com/rust-lang/crates.io-index)", + "proc-macro2 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "quote 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)", + "rustversion 1.0.0 (registry+https://github.com/rust-lang/crates.io-index)", + "substrate-externalities 2.0.0", + "substrate-runtime-interface 2.0.0", + "syn 1.0.8 (registry+https://github.com/rust-lang/crates.io-index)", + "trybuild 1.0.17 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "substrate-runtime-interface-test-wasm" +version = "2.0.0" +dependencies = [ + "sr-io 2.0.0", + "sr-std 2.0.0", + "substrate-primitives 2.0.0", + "substrate-runtime-interface 2.0.0", + "substrate-wasm-builder-runner 1.0.4", +] + [[package]] name = "substrate-runtime-test" version = "2.0.0" @@ -5582,7 +6468,7 @@ dependencies = [ name = "substrate-serializer" version = "2.0.0" dependencies = [ - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", "serde_json 1.0.41 (registry+https://github.com/rust-lang/crates.io-index)", ] @@ -5599,17 +6485,19 @@ dependencies = [ "node-executor 2.0.0", "node-primitives 2.0.0", "node-runtime 2.0.0", - "parity-multiaddr 0.5.0 (registry+https://github.com/rust-lang/crates.io-index)", + "parity-multiaddr 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)", "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", "parking_lot 0.9.0 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", "serde_json 1.0.41 (registry+https://github.com/rust-lang/crates.io-index)", "slog 2.5.2 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-api 2.0.0", "sr-io 2.0.0", "sr-primitives 2.0.0", "substrate-application-crypto 2.0.0", "substrate-chain-spec 2.0.0", "substrate-client 2.0.0", + "substrate-client-api 2.0.0", "substrate-client-db 2.0.0", "substrate-consensus-babe-primitives 2.0.0", "substrate-consensus-common 2.0.0", @@ -5620,13 +6508,15 @@ dependencies = [ "substrate-network 2.0.0", "substrate-offchain 2.0.0", "substrate-primitives 2.0.0", + "substrate-prometheus 2.0.0", "substrate-rpc 2.0.0", "substrate-rpc-servers 2.0.0", "substrate-session 2.0.0", "substrate-telemetry 2.0.0", "substrate-test-runtime-client 2.0.0", "substrate-transaction-pool 2.0.0", - "sysinfo 0.9.5 (registry+https://github.com/rust-lang/crates.io-index)", + "substrate-transaction-pool-runtime-api 2.0.0", + "sysinfo 0.9.6 (registry+https://github.com/rust-lang/crates.io-index)", "target_info 0.1.0 (registry+https://github.com/rust-lang/crates.io-index)", "tokio 0.1.22 (registry+https://github.com/rust-lang/crates.io-index)", "tokio-executor 0.1.8 (registry+https://github.com/rust-lang/crates.io-index)", @@ -5637,7 +6527,7 @@ dependencies = [ name = "substrate-service-test" version = "2.0.0" dependencies = [ - "env_logger 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)", + "env_logger 0.7.1 (registry+https://github.com/rust-lang/crates.io-index)", "fdlimit 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)", "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", "futures-preview 0.3.0-alpha.19 (registry+https://github.com/rust-lang/crates.io-index)", @@ -5656,17 +6546,16 @@ dependencies = [ name = "substrate-session" version = "2.0.0" dependencies = [ + "sr-api 2.0.0", "sr-primitives 2.0.0", "sr-std 2.0.0", - "substrate-client 2.0.0", - "substrate-primitives 2.0.0", ] [[package]] name = "substrate-state-db" version = "2.0.0" dependencies = [ - "env_logger 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)", + "env_logger 0.7.1 (registry+https://github.com/rust-lang/crates.io-index)", "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", "parking_lot 0.9.0 (registry+https://github.com/rust-lang/crates.io-index)", @@ -5700,15 +6589,15 @@ dependencies = [ "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", "futures-preview 0.3.0-alpha.19 (registry+https://github.com/rust-lang/crates.io-index)", "futures-timer 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)", - "libp2p 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)", + "libp2p 0.13.0 (registry+https://github.com/rust-lang/crates.io-index)", "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", "parking_lot 0.9.0 (registry+https://github.com/rust-lang/crates.io-index)", "rand 0.7.2 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", "slog 2.5.2 (registry+https://github.com/rust-lang/crates.io-index)", "slog-async 2.3.0 (git+https://github.com/paritytech/slog-async)", "slog-json 2.3.0 (registry+https://github.com/rust-lang/crates.io-index)", - "slog-scope 4.1.2 (registry+https://github.com/rust-lang/crates.io-index)", + "slog-scope 4.3.0 (registry+https://github.com/rust-lang/crates.io-index)", "tokio-io 0.1.12 (registry+https://github.com/rust-lang/crates.io-index)", "void 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)", ] @@ -5722,6 +6611,7 @@ dependencies = [ "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", "sr-primitives 2.0.0", "substrate-client 2.0.0", + "substrate-client-api 2.0.0", "substrate-client-db 2.0.0", "substrate-consensus-common 2.0.0", "substrate-executor 2.0.0", @@ -5730,6 +6620,17 @@ dependencies = [ "substrate-state-machine 2.0.0", ] +[[package]] +name = "substrate-test-primitives" +version = "2.0.0" +dependencies = [ + "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-primitives 2.0.0", + "substrate-application-crypto 2.0.0", + "substrate-primitives 2.0.0", +] + [[package]] name = "substrate-test-runtime" version = "2.0.0" @@ -5737,19 +6638,21 @@ dependencies = [ "cfg-if 0.1.10 (registry+https://github.com/rust-lang/crates.io-index)", "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", "memory-db 0.15.2 (registry+https://github.com/rust-lang/crates.io-index)", + "paint-babe 2.0.0", + "paint-executive 2.0.0", + "paint-support 2.0.0", + "paint-system 2.0.0", + "paint-system-rpc-runtime-api 2.0.0", + "paint-timestamp 2.0.0", "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-api 2.0.0", "sr-io 2.0.0", "sr-primitives 2.0.0", "sr-std 2.0.0", "sr-version 2.0.0", - "srml-babe 2.0.0", - "srml-executive 2.0.0", - "srml-support 2.0.0", - "srml-system 2.0.0", - "srml-system-rpc-runtime-api 2.0.0", - "srml-timestamp 2.0.0", "substrate-application-crypto 2.0.0", + "substrate-block-builder-runtime-api 2.0.0", "substrate-client 2.0.0", "substrate-consensus-aura-primitives 2.0.0", "substrate-consensus-babe-primitives 2.0.0", @@ -5758,9 +6661,11 @@ dependencies = [ "substrate-keyring 2.0.0", "substrate-offchain-primitives 2.0.0", "substrate-primitives 2.0.0", + "substrate-runtime-interface 2.0.0", "substrate-session 2.0.0", "substrate-state-machine 2.0.0", "substrate-test-runtime-client 2.0.0", + "substrate-transaction-pool-runtime-api 2.0.0", "substrate-trie 2.0.0", "substrate-wasm-builder-runner 1.0.4", "trie-db 0.15.2 (registry+https://github.com/rust-lang/crates.io-index)", @@ -5772,6 +6677,9 @@ version = "2.0.0" dependencies = [ "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", "sr-primitives 2.0.0", + "substrate-block-builder 2.0.0", + "substrate-client 2.0.0", + "substrate-client-api 2.0.0", "substrate-primitives 2.0.0", "substrate-test-client 2.0.0", "substrate-test-runtime 2.0.0", @@ -5784,12 +6692,12 @@ dependencies = [ "assert_matches 1.3.0 (registry+https://github.com/rust-lang/crates.io-index)", "criterion 0.3.0 (registry+https://github.com/rust-lang/crates.io-index)", "derive_more 0.15.0 (registry+https://github.com/rust-lang/crates.io-index)", - "env_logger 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)", + "env_logger 0.7.1 (registry+https://github.com/rust-lang/crates.io-index)", "futures-preview 0.3.0-alpha.19 (registry+https://github.com/rust-lang/crates.io-index)", "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", "parking_lot 0.9.0 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", "sr-primitives 2.0.0", "substrate-primitives 2.0.0", "substrate-test-runtime 2.0.0", @@ -5800,16 +6708,26 @@ name = "substrate-transaction-pool" version = "2.0.0" dependencies = [ "derive_more 0.15.0 (registry+https://github.com/rust-lang/crates.io-index)", - "futures-preview 0.3.0-alpha.19 (registry+https://github.com/rust-lang/crates.io-index)", + "futures 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", "parking_lot 0.9.0 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-api 2.0.0", "sr-primitives 2.0.0", - "substrate-client 2.0.0", "substrate-keyring 2.0.0", "substrate-primitives 2.0.0", "substrate-test-runtime-client 2.0.0", "substrate-transaction-graph 2.0.0", + "substrate-transaction-pool-runtime-api 2.0.0", +] + +[[package]] +name = "substrate-transaction-pool-runtime-api" +version = "2.0.0" +dependencies = [ + "sr-api 2.0.0", + "sr-primitives 2.0.0", + "substrate-primitives 2.0.0", ] [[package]] @@ -5819,7 +6737,6 @@ dependencies = [ "criterion 0.2.11 (registry+https://github.com/rust-lang/crates.io-index)", "hash-db 0.15.2 (registry+https://github.com/rust-lang/crates.io-index)", "hex-literal 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)", - "keccak-hasher 0.15.2 (registry+https://github.com/rust-lang/crates.io-index)", "memory-db 0.15.2 (registry+https://github.com/rust-lang/crates.io-index)", "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", "sr-std 2.0.0", @@ -5834,11 +6751,12 @@ dependencies = [ name = "substrate-wasm-builder" version = "1.0.8" dependencies = [ + "atty 0.2.13 (registry+https://github.com/rust-lang/crates.io-index)", "build-helper 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)", - "cargo_metadata 0.8.2 (registry+https://github.com/rust-lang/crates.io-index)", + "cargo_metadata 0.9.0 (registry+https://github.com/rust-lang/crates.io-index)", "fs2 0.4.3 (registry+https://github.com/rust-lang/crates.io-index)", "tempfile 3.1.0 (registry+https://github.com/rust-lang/crates.io-index)", - "toml 0.5.3 (registry+https://github.com/rust-lang/crates.io-index)", + "toml 0.5.5 (registry+https://github.com/rust-lang/crates.io-index)", "walkdir 2.2.9 (registry+https://github.com/rust-lang/crates.io-index)", "wasm-gc-api 0.1.11 (registry+https://github.com/rust-lang/crates.io-index)", ] @@ -5847,16 +6765,12 @@ dependencies = [ name = "substrate-wasm-builder-runner" version = "1.0.4" -[[package]] -name = "substrate-wasm-builder-runner" -version = "1.0.4" -source = "registry+https://github.com/rust-lang/crates.io-index" - [[package]] name = "substrate-wasm-interface" version = "2.0.0" dependencies = [ - "wasmi 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)", + "impl-trait-for-tuples 0.1.3 (registry+https://github.com/rust-lang/crates.io-index)", + "wasmi 0.6.2 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] @@ -5881,7 +6795,7 @@ dependencies = [ [[package]] name = "syn" -version = "1.0.7" +version = "1.0.8" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "proc-macro2 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", @@ -5891,34 +6805,23 @@ dependencies = [ [[package]] name = "synstructure" -version = "0.10.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -dependencies = [ - "proc-macro2 0.4.30 (registry+https://github.com/rust-lang/crates.io-index)", - "quote 0.6.13 (registry+https://github.com/rust-lang/crates.io-index)", - "syn 0.15.44 (registry+https://github.com/rust-lang/crates.io-index)", - "unicode-xid 0.1.0 (registry+https://github.com/rust-lang/crates.io-index)", -] - -[[package]] -name = "synstructure" -version = "0.12.1" +version = "0.12.2" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "proc-macro2 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", "quote 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)", - "syn 1.0.7 (registry+https://github.com/rust-lang/crates.io-index)", + "syn 1.0.8 (registry+https://github.com/rust-lang/crates.io-index)", "unicode-xid 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] name = "sysinfo" -version = "0.9.5" +version = "0.9.6" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "cfg-if 0.1.10 (registry+https://github.com/rust-lang/crates.io-index)", "doc-comment 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", - "libc 0.2.62 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", "rayon 1.2.0 (registry+https://github.com/rust-lang/crates.io-index)", "winapi 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)", ] @@ -5928,6 +6831,16 @@ name = "take_mut" version = "0.2.2" source = "registry+https://github.com/rust-lang/crates.io-index" +[[package]] +name = "target-lexicon" +version = "0.8.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "failure 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)", + "failure_derive 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)", + "serde_json 1.0.41 (registry+https://github.com/rust-lang/crates.io-index)", +] + [[package]] name = "target_info" version = "0.1.0" @@ -5948,7 +6861,7 @@ version = "3.1.0" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "cfg-if 0.1.10 (registry+https://github.com/rust-lang/crates.io-index)", - "libc 0.2.62 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", "rand 0.7.2 (registry+https://github.com/rust-lang/crates.io-index)", "redox_syscall 0.1.56 (registry+https://github.com/rust-lang/crates.io-index)", "remove_dir_all 0.5.2 (registry+https://github.com/rust-lang/crates.io-index)", @@ -5963,6 +6876,18 @@ dependencies = [ "wincolor 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)", ] +[[package]] +name = "test-case" +version = "0.3.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "lazy_static 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)", + "proc-macro2 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "quote 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)", + "syn 1.0.8 (registry+https://github.com/rust-lang/crates.io-index)", + "version_check 0.9.1 (registry+https://github.com/rust-lang/crates.io-index)", +] + [[package]] name = "textwrap" version = "0.11.0" @@ -5984,7 +6909,7 @@ name = "threadpool" version = "1.7.1" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "num_cpus 1.10.1 (registry+https://github.com/rust-lang/crates.io-index)", + "num_cpus 1.11.0 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] @@ -5992,7 +6917,7 @@ name = "time" version = "0.1.42" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "libc 0.2.62 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", "redox_syscall 0.1.56 (registry+https://github.com/rust-lang/crates.io-index)", "winapi 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)", ] @@ -6019,12 +6944,20 @@ dependencies = [ "crunchy 0.2.2 (registry+https://github.com/rust-lang/crates.io-index)", ] +[[package]] +name = "tiny-keccak" +version = "2.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "crunchy 0.2.2 (registry+https://github.com/rust-lang/crates.io-index)", +] + [[package]] name = "tinytemplate" version = "1.0.2" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", "serde_json 1.0.41 (registry+https://github.com/rust-lang/crates.io-index)", ] @@ -6036,7 +6969,7 @@ dependencies = [ "bytes 0.4.12 (registry+https://github.com/rust-lang/crates.io-index)", "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", "mio 0.6.19 (registry+https://github.com/rust-lang/crates.io-index)", - "num_cpus 1.10.1 (registry+https://github.com/rust-lang/crates.io-index)", + "num_cpus 1.11.0 (registry+https://github.com/rust-lang/crates.io-index)", "tokio-codec 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)", "tokio-current-thread 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)", "tokio-executor 0.1.8 (registry+https://github.com/rust-lang/crates.io-index)", @@ -6130,7 +7063,7 @@ dependencies = [ "lazy_static 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)", "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", "mio 0.6.19 (registry+https://github.com/rust-lang/crates.io-index)", - "num_cpus 1.10.1 (registry+https://github.com/rust-lang/crates.io-index)", + "num_cpus 1.11.0 (registry+https://github.com/rust-lang/crates.io-index)", "parking_lot 0.9.0 (registry+https://github.com/rust-lang/crates.io-index)", "slab 0.4.2 (registry+https://github.com/rust-lang/crates.io-index)", "tokio-executor 0.1.8 (registry+https://github.com/rust-lang/crates.io-index)", @@ -6140,15 +7073,15 @@ dependencies = [ [[package]] name = "tokio-rustls" -version = "0.10.0-alpha.4" +version = "0.10.2" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "bytes 0.4.12 (registry+https://github.com/rust-lang/crates.io-index)", "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", "iovec 0.1.4 (registry+https://github.com/rust-lang/crates.io-index)", - "rustls 0.15.2 (registry+https://github.com/rust-lang/crates.io-index)", + "rustls 0.16.0 (registry+https://github.com/rust-lang/crates.io-index)", "tokio-io 0.1.12 (registry+https://github.com/rust-lang/crates.io-index)", - "webpki 0.19.1 (registry+https://github.com/rust-lang/crates.io-index)", + "webpki 0.21.0 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] @@ -6178,13 +7111,13 @@ name = "tokio-threadpool" version = "0.1.16" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "crossbeam-deque 0.7.1 (registry+https://github.com/rust-lang/crates.io-index)", + "crossbeam-deque 0.7.2 (registry+https://github.com/rust-lang/crates.io-index)", "crossbeam-queue 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)", "crossbeam-utils 0.6.6 (registry+https://github.com/rust-lang/crates.io-index)", "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", "lazy_static 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)", "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", - "num_cpus 1.10.1 (registry+https://github.com/rust-lang/crates.io-index)", + "num_cpus 1.11.0 (registry+https://github.com/rust-lang/crates.io-index)", "slab 0.4.2 (registry+https://github.com/rust-lang/crates.io-index)", "tokio-executor 0.1.8 (registry+https://github.com/rust-lang/crates.io-index)", ] @@ -6232,7 +7165,7 @@ dependencies = [ "bytes 0.4.12 (registry+https://github.com/rust-lang/crates.io-index)", "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", "iovec 0.1.4 (registry+https://github.com/rust-lang/crates.io-index)", - "libc 0.2.62 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", "mio 0.6.19 (registry+https://github.com/rust-lang/crates.io-index)", "mio-uds 0.6.7 (registry+https://github.com/rust-lang/crates.io-index)", @@ -6243,10 +7176,10 @@ dependencies = [ [[package]] name = "toml" -version = "0.5.3" +version = "0.5.5" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] @@ -6260,9 +7193,12 @@ version = "0.0.1" dependencies = [ "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", + "sr-api 2.0.0", "sr-primitives 2.0.0", + "substrate-block-builder-runtime-api 2.0.0", "substrate-cli 2.0.0", "substrate-client 2.0.0", + "substrate-client-api 2.0.0", "substrate-consensus-common 2.0.0", "substrate-primitives 2.0.0", "substrate-service 2.0.0", @@ -6290,7 +7226,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "elastic-array 0.10.2 (registry+https://github.com/rust-lang/crates.io-index)", "hash-db 0.15.2 (registry+https://github.com/rust-lang/crates.io-index)", - "hashbrown 0.6.1 (registry+https://github.com/rust-lang/crates.io-index)", + "hashbrown 0.6.3 (registry+https://github.com/rust-lang/crates.io-index)", "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", "rand 0.6.5 (registry+https://github.com/rust-lang/crates.io-index)", ] @@ -6317,6 +7253,14 @@ name = "try-lock" version = "0.2.2" source = "registry+https://github.com/rust-lang/crates.io-index" +[[package]] +name = "try_from" +version = "0.3.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "cfg-if 0.1.10 (registry+https://github.com/rust-lang/crates.io-index)", +] + [[package]] name = "trybuild" version = "1.0.17" @@ -6324,10 +7268,10 @@ source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "glob 0.3.0 (registry+https://github.com/rust-lang/crates.io-index)", "lazy_static 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)", - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", "serde_json 1.0.41 (registry+https://github.com/rust-lang/crates.io-index)", "termcolor 1.0.5 (registry+https://github.com/rust-lang/crates.io-index)", - "toml 0.5.3 (registry+https://github.com/rust-lang/crates.io-index)", + "toml 0.5.5 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] @@ -6360,12 +7304,13 @@ source = "registry+https://github.com/rust-lang/crates.io-index" [[package]] name = "uint" -version = "0.8.1" +version = "0.8.2" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "byteorder 1.3.2 (registry+https://github.com/rust-lang/crates.io-index)", "crunchy 0.2.2 (registry+https://github.com/rust-lang/crates.io-index)", "rustc-hex 2.0.1 (registry+https://github.com/rust-lang/crates.io-index)", + "static_assertions 1.1.0 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] @@ -6394,15 +7339,15 @@ dependencies = [ [[package]] name = "unicode-normalization" -version = "0.1.8" +version = "0.1.9" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "smallvec 0.6.10 (registry+https://github.com/rust-lang/crates.io-index)", + "smallvec 0.6.13 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] name = "unicode-segmentation" -version = "1.3.0" +version = "1.6.0" source = "registry+https://github.com/rust-lang/crates.io-index" [[package]] @@ -6431,7 +7376,7 @@ dependencies = [ [[package]] name = "untrusted" -version = "0.6.2" +version = "0.7.0" source = "registry+https://github.com/rust-lang/crates.io-index" [[package]] @@ -6454,6 +7399,14 @@ dependencies = [ "percent-encoding 2.1.0 (registry+https://github.com/rust-lang/crates.io-index)", ] +[[package]] +name = "uuid" +version = "0.7.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "rand 0.6.5 (registry+https://github.com/rust-lang/crates.io-index)", +] + [[package]] name = "vcpkg" version = "0.2.7" @@ -6469,7 +7422,7 @@ name = "vergen" version = "3.0.4" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "bitflags 1.2.0 (registry+https://github.com/rust-lang/crates.io-index)", + "bitflags 1.2.1 (registry+https://github.com/rust-lang/crates.io-index)", "chrono 0.4.9 (registry+https://github.com/rust-lang/crates.io-index)", "failure 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)", ] @@ -6479,6 +7432,11 @@ name = "version_check" version = "0.1.5" source = "registry+https://github.com/rust-lang/crates.io-index" +[[package]] +name = "version_check" +version = "0.9.1" +source = "registry+https://github.com/rust-lang/crates.io-index" + [[package]] name = "void" version = "1.0.2" @@ -6489,8 +7447,8 @@ name = "wabt" version = "0.9.2" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", - "serde_derive 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "serde_derive 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", "serde_json 1.0.41 (registry+https://github.com/rust-lang/crates.io-index)", "wabt-sys 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)", ] @@ -6500,7 +7458,7 @@ name = "wabt-sys" version = "0.7.0" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "cc 1.0.45 (registry+https://github.com/rust-lang/crates.io-index)", + "cc 1.0.47 (registry+https://github.com/rust-lang/crates.io-index)", "cmake 0.1.42 (registry+https://github.com/rust-lang/crates.io-index)", "glob 0.2.11 (registry+https://github.com/rust-lang/crates.io-index)", ] @@ -6532,16 +7490,16 @@ source = "registry+https://github.com/rust-lang/crates.io-index" [[package]] name = "wasm-bindgen" -version = "0.2.51" +version = "0.2.54" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "cfg-if 0.1.10 (registry+https://github.com/rust-lang/crates.io-index)", - "wasm-bindgen-macro 0.2.51 (registry+https://github.com/rust-lang/crates.io-index)", + "wasm-bindgen-macro 0.2.54 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] name = "wasm-bindgen-backend" -version = "0.2.51" +version = "0.2.54" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "bumpalo 2.6.0 (registry+https://github.com/rust-lang/crates.io-index)", @@ -6549,8 +7507,8 @@ dependencies = [ "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", "proc-macro2 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", "quote 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)", - "syn 1.0.7 (registry+https://github.com/rust-lang/crates.io-index)", - "wasm-bindgen-shared 0.2.51 (registry+https://github.com/rust-lang/crates.io-index)", + "syn 1.0.8 (registry+https://github.com/rust-lang/crates.io-index)", + "wasm-bindgen-shared 0.2.54 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] @@ -6560,49 +7518,49 @@ source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "cfg-if 0.1.10 (registry+https://github.com/rust-lang/crates.io-index)", "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", - "js-sys 0.3.28 (registry+https://github.com/rust-lang/crates.io-index)", - "wasm-bindgen 0.2.51 (registry+https://github.com/rust-lang/crates.io-index)", - "web-sys 0.3.28 (registry+https://github.com/rust-lang/crates.io-index)", + "js-sys 0.3.31 (registry+https://github.com/rust-lang/crates.io-index)", + "wasm-bindgen 0.2.54 (registry+https://github.com/rust-lang/crates.io-index)", + "web-sys 0.3.31 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] name = "wasm-bindgen-macro" -version = "0.2.51" +version = "0.2.54" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "quote 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)", - "wasm-bindgen-macro-support 0.2.51 (registry+https://github.com/rust-lang/crates.io-index)", + "wasm-bindgen-macro-support 0.2.54 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] name = "wasm-bindgen-macro-support" -version = "0.2.51" +version = "0.2.54" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "proc-macro2 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", "quote 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)", - "syn 1.0.7 (registry+https://github.com/rust-lang/crates.io-index)", - "wasm-bindgen-backend 0.2.51 (registry+https://github.com/rust-lang/crates.io-index)", - "wasm-bindgen-shared 0.2.51 (registry+https://github.com/rust-lang/crates.io-index)", + "syn 1.0.8 (registry+https://github.com/rust-lang/crates.io-index)", + "wasm-bindgen-backend 0.2.54 (registry+https://github.com/rust-lang/crates.io-index)", + "wasm-bindgen-shared 0.2.54 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] name = "wasm-bindgen-shared" -version = "0.2.51" +version = "0.2.54" source = "registry+https://github.com/rust-lang/crates.io-index" [[package]] name = "wasm-bindgen-webidl" -version = "0.2.51" +version = "0.2.54" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "failure 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)", + "anyhow 1.0.19 (registry+https://github.com/rust-lang/crates.io-index)", "heck 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)", "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", "proc-macro2 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)", "quote 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)", - "syn 1.0.7 (registry+https://github.com/rust-lang/crates.io-index)", - "wasm-bindgen-backend 0.2.51 (registry+https://github.com/rust-lang/crates.io-index)", + "syn 1.0.8 (registry+https://github.com/rust-lang/crates.io-index)", + "wasm-bindgen-backend 0.2.54 (registry+https://github.com/rust-lang/crates.io-index)", "weedle 0.10.0 (registry+https://github.com/rust-lang/crates.io-index)", ] @@ -6622,24 +7580,25 @@ version = "0.1.3" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", - "js-sys 0.3.28 (registry+https://github.com/rust-lang/crates.io-index)", + "js-sys 0.3.31 (registry+https://github.com/rust-lang/crates.io-index)", "send_wrapper 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)", "tokio-timer 0.2.11 (registry+https://github.com/rust-lang/crates.io-index)", - "wasm-bindgen 0.2.51 (registry+https://github.com/rust-lang/crates.io-index)", - "web-sys 0.3.28 (registry+https://github.com/rust-lang/crates.io-index)", + "wasm-bindgen 0.2.54 (registry+https://github.com/rust-lang/crates.io-index)", + "web-sys 0.3.31 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] name = "wasmi" -version = "0.5.1" +version = "0.6.2" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "libc 0.2.62 (registry+https://github.com/rust-lang/crates.io-index)", + "errno 0.2.4 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", "memory_units 0.3.0 (registry+https://github.com/rust-lang/crates.io-index)", "num-rational 0.2.2 (registry+https://github.com/rust-lang/crates.io-index)", "num-traits 0.2.8 (registry+https://github.com/rust-lang/crates.io-index)", - "parity-wasm 0.40.3 (registry+https://github.com/rust-lang/crates.io-index)", - "wasmi-validation 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)", + "parity-wasm 0.41.0 (registry+https://github.com/rust-lang/crates.io-index)", + "wasmi-validation 0.3.0 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] @@ -6651,33 +7610,137 @@ dependencies = [ ] [[package]] -name = "web-sys" -version = "0.3.28" +name = "wasmi-validation" +version = "0.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "parity-wasm 0.41.0 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "wasmparser" +version = "0.39.2" source = "registry+https://github.com/rust-lang/crates.io-index" + +[[package]] +name = "wasmtime-debug" +version = "0.2.0" +source = "git+https://github.com/CraneStation/wasmtime.git?rev=71dd73d6#71dd73d672deb325664e3c9cd4ee7acebed5fb95" +dependencies = [ + "cranelift-codegen 0.46.1 (registry+https://github.com/rust-lang/crates.io-index)", + "cranelift-entity 0.46.1 (registry+https://github.com/rust-lang/crates.io-index)", + "cranelift-wasm 0.46.1 (registry+https://github.com/rust-lang/crates.io-index)", + "faerie 0.11.0 (registry+https://github.com/rust-lang/crates.io-index)", + "failure 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)", + "failure_derive 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)", + "gimli 0.19.0 (registry+https://github.com/rust-lang/crates.io-index)", + "target-lexicon 0.8.1 (registry+https://github.com/rust-lang/crates.io-index)", + "wasmparser 0.39.2 (registry+https://github.com/rust-lang/crates.io-index)", + "wasmtime-environ 0.2.0 (git+https://github.com/CraneStation/wasmtime.git?rev=71dd73d6)", +] + +[[package]] +name = "wasmtime-environ" +version = "0.2.0" +source = "git+https://github.com/CraneStation/wasmtime.git?rev=71dd73d6#71dd73d672deb325664e3c9cd4ee7acebed5fb95" +dependencies = [ + "base64 0.10.1 (registry+https://github.com/rust-lang/crates.io-index)", + "bincode 1.2.0 (registry+https://github.com/rust-lang/crates.io-index)", + "cranelift-codegen 0.46.1 (registry+https://github.com/rust-lang/crates.io-index)", + "cranelift-entity 0.46.1 (registry+https://github.com/rust-lang/crates.io-index)", + "cranelift-wasm 0.46.1 (registry+https://github.com/rust-lang/crates.io-index)", + "directories 2.0.2 (registry+https://github.com/rust-lang/crates.io-index)", + "errno 0.2.4 (registry+https://github.com/rust-lang/crates.io-index)", + "failure 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)", + "failure_derive 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)", + "file-per-thread-logger 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)", + "indexmap 1.3.0 (registry+https://github.com/rust-lang/crates.io-index)", + "lazy_static 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", + "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", + "rayon 1.2.0 (registry+https://github.com/rust-lang/crates.io-index)", + "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)", + "sha2 0.8.0 (registry+https://github.com/rust-lang/crates.io-index)", + "spin 0.5.2 (registry+https://github.com/rust-lang/crates.io-index)", + "toml 0.5.5 (registry+https://github.com/rust-lang/crates.io-index)", + "winapi 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)", + "zstd 0.4.28+zstd.1.4.3 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "wasmtime-jit" +version = "0.2.0" +source = "git+https://github.com/CraneStation/wasmtime.git?rev=71dd73d6#71dd73d672deb325664e3c9cd4ee7acebed5fb95" dependencies = [ + "cranelift-codegen 0.46.1 (registry+https://github.com/rust-lang/crates.io-index)", + "cranelift-entity 0.46.1 (registry+https://github.com/rust-lang/crates.io-index)", + "cranelift-frontend 0.46.1 (registry+https://github.com/rust-lang/crates.io-index)", + "cranelift-wasm 0.46.1 (registry+https://github.com/rust-lang/crates.io-index)", "failure 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)", - "js-sys 0.3.28 (registry+https://github.com/rust-lang/crates.io-index)", + "failure_derive 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)", + "region 2.1.2 (registry+https://github.com/rust-lang/crates.io-index)", + "target-lexicon 0.8.1 (registry+https://github.com/rust-lang/crates.io-index)", + "wasmparser 0.39.2 (registry+https://github.com/rust-lang/crates.io-index)", + "wasmtime-debug 0.2.0 (git+https://github.com/CraneStation/wasmtime.git?rev=71dd73d6)", + "wasmtime-environ 0.2.0 (git+https://github.com/CraneStation/wasmtime.git?rev=71dd73d6)", + "wasmtime-runtime 0.2.0 (git+https://github.com/CraneStation/wasmtime.git?rev=71dd73d6)", +] + +[[package]] +name = "wasmtime-runtime" +version = "0.2.0" +source = "git+https://github.com/CraneStation/wasmtime.git?rev=71dd73d6#71dd73d672deb325664e3c9cd4ee7acebed5fb95" +dependencies = [ + "cc 1.0.47 (registry+https://github.com/rust-lang/crates.io-index)", + "cranelift-codegen 0.46.1 (registry+https://github.com/rust-lang/crates.io-index)", + "cranelift-entity 0.46.1 (registry+https://github.com/rust-lang/crates.io-index)", + "cranelift-wasm 0.46.1 (registry+https://github.com/rust-lang/crates.io-index)", + "failure 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)", + "failure_derive 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)", + "indexmap 1.3.0 (registry+https://github.com/rust-lang/crates.io-index)", + "lazy_static 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", + "memoffset 0.5.3 (registry+https://github.com/rust-lang/crates.io-index)", + "region 2.1.2 (registry+https://github.com/rust-lang/crates.io-index)", + "wasmtime-environ 0.2.0 (git+https://github.com/CraneStation/wasmtime.git?rev=71dd73d6)", + "winapi 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "web-sys" +version = "0.3.31" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "anyhow 1.0.19 (registry+https://github.com/rust-lang/crates.io-index)", + "js-sys 0.3.31 (registry+https://github.com/rust-lang/crates.io-index)", "sourcefile 0.1.4 (registry+https://github.com/rust-lang/crates.io-index)", - "wasm-bindgen 0.2.51 (registry+https://github.com/rust-lang/crates.io-index)", - "wasm-bindgen-webidl 0.2.51 (registry+https://github.com/rust-lang/crates.io-index)", + "wasm-bindgen 0.2.54 (registry+https://github.com/rust-lang/crates.io-index)", + "wasm-bindgen-webidl 0.2.54 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] name = "webpki" -version = "0.19.1" +version = "0.21.0" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "ring 0.14.6 (registry+https://github.com/rust-lang/crates.io-index)", - "untrusted 0.6.2 (registry+https://github.com/rust-lang/crates.io-index)", + "ring 0.16.9 (registry+https://github.com/rust-lang/crates.io-index)", + "untrusted 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] name = "webpki-roots" -version = "0.16.0" +version = "0.17.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "webpki 0.21.0 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "webpki-roots" +version = "0.18.0" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "untrusted 0.6.2 (registry+https://github.com/rust-lang/crates.io-index)", - "webpki 0.19.1 (registry+https://github.com/rust-lang/crates.io-index)", + "webpki 0.21.0 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] @@ -6686,7 +7749,7 @@ version = "0.23.0" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "base64 0.10.1 (registry+https://github.com/rust-lang/crates.io-index)", - "bitflags 1.2.0 (registry+https://github.com/rust-lang/crates.io-index)", + "bitflags 1.2.1 (registry+https://github.com/rust-lang/crates.io-index)", "byteorder 1.3.2 (registry+https://github.com/rust-lang/crates.io-index)", "bytes 0.4.12 (registry+https://github.com/rust-lang/crates.io-index)", "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", @@ -6717,7 +7780,7 @@ version = "2.0.1" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "failure 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)", - "libc 0.2.62 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] @@ -6766,9 +7829,17 @@ dependencies = [ "winapi-util 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)", ] +[[package]] +name = "winreg" +version = "0.6.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "winapi 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)", +] + [[package]] name = "ws" -version = "0.9.0" +version = "0.9.1" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "byteorder 1.3.2 (registry+https://github.com/rust-lang/crates.io-index)", @@ -6777,7 +7848,7 @@ dependencies = [ "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", "mio 0.6.19 (registry+https://github.com/rust-lang/crates.io-index)", "mio-extras 2.0.5 (registry+https://github.com/rust-lang/crates.io-index)", - "rand 0.6.5 (registry+https://github.com/rust-lang/crates.io-index)", + "rand 0.7.2 (registry+https://github.com/rust-lang/crates.io-index)", "sha-1 0.8.1 (registry+https://github.com/rust-lang/crates.io-index)", "slab 0.4.2 (registry+https://github.com/rust-lang/crates.io-index)", "url 2.1.0 (registry+https://github.com/rust-lang/crates.io-index)", @@ -6814,16 +7885,16 @@ source = "registry+https://github.com/rust-lang/crates.io-index" [[package]] name = "yamux" -version = "0.2.1" +version = "0.2.2" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "bytes 0.4.12 (registry+https://github.com/rust-lang/crates.io-index)", "futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)", "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)", "nohash-hasher 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)", - "parking_lot 0.6.4 (registry+https://github.com/rust-lang/crates.io-index)", - "quick-error 0.1.4 (registry+https://github.com/rust-lang/crates.io-index)", - "rand 0.6.5 (registry+https://github.com/rust-lang/crates.io-index)", + "parking_lot 0.9.0 (registry+https://github.com/rust-lang/crates.io-index)", + "quick-error 1.2.2 (registry+https://github.com/rust-lang/crates.io-index)", + "rand 0.7.2 (registry+https://github.com/rust-lang/crates.io-index)", "tokio-codec 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)", "tokio-io 0.1.12 (registry+https://github.com/rust-lang/crates.io-index)", ] @@ -6832,9 +7903,6 @@ dependencies = [ name = "zeroize" version = "0.9.3" source = "registry+https://github.com/rust-lang/crates.io-index" -dependencies = [ - "zeroize_derive 0.9.3 (registry+https://github.com/rust-lang/crates.io-index)", -] [[package]] name = "zeroize" @@ -6842,66 +7910,89 @@ version = "0.10.1" source = "registry+https://github.com/rust-lang/crates.io-index" [[package]] -name = "zeroize_derive" -version = "0.9.3" +name = "zeroize" +version = "1.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" + +[[package]] +name = "zstd" +version = "0.4.28+zstd.1.4.3" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "proc-macro2 0.4.30 (registry+https://github.com/rust-lang/crates.io-index)", - "quote 0.6.13 (registry+https://github.com/rust-lang/crates.io-index)", - "syn 0.15.44 (registry+https://github.com/rust-lang/crates.io-index)", - "synstructure 0.10.2 (registry+https://github.com/rust-lang/crates.io-index)", + "zstd-safe 1.4.13+zstd.1.4.3 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "zstd-safe" +version = "1.4.13+zstd.1.4.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", + "zstd-sys 1.4.13+zstd.1.4.3 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "zstd-sys" +version = "1.4.13+zstd.1.4.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "cc 1.0.47 (registry+https://github.com/rust-lang/crates.io-index)", + "glob 0.3.0 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)", ] [metadata] +"checksum Inflector 0.11.4 (registry+https://github.com/rust-lang/crates.io-index)" = "fe438c63458706e03479442743baae6c88256498e6431708f6dfc520a26515d3" "checksum adler32 1.0.4 (registry+https://github.com/rust-lang/crates.io-index)" = "5d2e7343e7fc9de883d1b0341e0b13970f764c14101234857d2ddafa1cb1cac2" "checksum aes-ctr 0.3.0 (registry+https://github.com/rust-lang/crates.io-index)" = "d2e5b0458ea3beae0d1d8c0f3946564f8e10f90646cf78c06b4351052058d1ee" "checksum aes-soft 0.3.3 (registry+https://github.com/rust-lang/crates.io-index)" = "cfd7e7ae3f9a1fb5c03b389fc6bb9a51400d0c13053f0dca698c832bfd893a0d" "checksum aesni 0.6.0 (registry+https://github.com/rust-lang/crates.io-index)" = "2f70a6b5f971e473091ab7cfb5ffac6cde81666c4556751d8d5620ead8abf100" -"checksum ahash 0.2.16 (registry+https://github.com/rust-lang/crates.io-index)" = "b35dfc96a657c1842b4eb73180b65e37152d4b94d0eb5cb51708aee7826950b4" +"checksum ahash 0.2.17 (registry+https://github.com/rust-lang/crates.io-index)" = "2f00e10d4814aa20900e7948174384f79f1317f24f0ba7494e735111653fc330" "checksum aho-corasick 0.7.6 (registry+https://github.com/rust-lang/crates.io-index)" = "58fb5e95d83b38284460a5fda7d6470aa0b8844d283a0b614b8535e880800d2d" -"checksum aio-limited 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)" = "c4dddf55b0b2da9acb7512f21c0a4f1c0871522ec4ab7fb919d0da807d1e32b3" "checksum ansi_term 0.11.0 (registry+https://github.com/rust-lang/crates.io-index)" = "ee49baf6cb617b853aa8d93bf420db2383fab46d314482ca2803b40d5fde979b" "checksum ansi_term 0.12.1 (registry+https://github.com/rust-lang/crates.io-index)" = "d52a9bb7ec0cf484c551830a7ce27bd20d67eac647e1befb56b0be4ee39a55d2" +"checksum anyhow 1.0.19 (registry+https://github.com/rust-lang/crates.io-index)" = "57114fc2a6cc374bce195d3482057c846e706d252ff3604363449695684d7a0d" "checksum app_dirs 1.2.1 (registry+https://github.com/rust-lang/crates.io-index)" = "e73a24bad9bd6a94d6395382a6c69fe071708ae4409f763c5475e14ee896313d" -"checksum arc-swap 0.3.11 (registry+https://github.com/rust-lang/crates.io-index)" = "bc4662175ead9cd84451d5c35070517777949a2ed84551764129cedb88384841" +"checksum arc-swap 0.4.3 (registry+https://github.com/rust-lang/crates.io-index)" = "f1a1eca3195b729bbd64e292ef2f5fff6b1c28504fed762ce2b1013dde4d8e92" "checksum arrayref 0.3.5 (registry+https://github.com/rust-lang/crates.io-index)" = "0d382e583f07208808f6b1249e60848879ba3543f57c32277bf52d69c2f0f0ee" "checksum arrayvec 0.4.12 (registry+https://github.com/rust-lang/crates.io-index)" = "cd9fd44efafa8690358b7408d253adf110036b88f55672a933f01d616ad9b1b9" +"checksum arrayvec 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)" = "cff77d8686867eceff3105329d4698d96c2391c176d5d03adc90c7389162b5b8" "checksum asn1_der 0.6.3 (registry+https://github.com/rust-lang/crates.io-index)" = "6fce6b6a0ffdafebd82c87e79e3f40e8d2c523e5fea5566ff6b90509bf98d638" "checksum asn1_der_derive 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)" = "0d0864d84b8e07b145449be9a8537db86bf9de5ce03b913214694643b4743502" "checksum assert_matches 1.3.0 (registry+https://github.com/rust-lang/crates.io-index)" = "7deb0a829ca7bcfaf5da70b073a8d128619259a7be8216a355e23f00763059e5" "checksum atty 0.2.13 (registry+https://github.com/rust-lang/crates.io-index)" = "1803c647a3ec87095e7ae7acfca019e98de5ec9a7d01343f611cf3152ed71a90" -"checksum autocfg 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)" = "b671c8fb71b457dd4ae18c4ba1e59aa81793daacc361d82fcd410cef0d491875" -"checksum backtrace 0.3.38 (registry+https://github.com/rust-lang/crates.io-index)" = "690a62be8920ccf773ee00ef0968649b0e724cda8bd5b12286302b4ae955fdf5" -"checksum backtrace-sys 0.1.31 (registry+https://github.com/rust-lang/crates.io-index)" = "82a830b4ef2d1124a711c71d263c5abdc710ef8e907bd508c88be475cebc422b" +"checksum autocfg 0.1.7 (registry+https://github.com/rust-lang/crates.io-index)" = "1d49d90015b3c36167a20fe2810c5cd875ad504b39cff3d4eae7977e6b7c1cb2" +"checksum backtrace 0.3.40 (registry+https://github.com/rust-lang/crates.io-index)" = "924c76597f0d9ca25d762c25a4d369d51267536465dc5064bdf0eb073ed477ea" +"checksum backtrace-sys 0.1.32 (registry+https://github.com/rust-lang/crates.io-index)" = "5d6575f128516de27e3ce99689419835fce9643a9b215a14d2b5b685be018491" "checksum base58 0.1.0 (registry+https://github.com/rust-lang/crates.io-index)" = "5024ee8015f02155eee35c711107ddd9a9bf3cb689cf2a9089c97e79b6e1ae83" "checksum base64 0.10.1 (registry+https://github.com/rust-lang/crates.io-index)" = "0b25d992356d2eb0ed82172f5248873db5560c4721f564b13cb5193bda5e668e" "checksum base64 0.9.3 (registry+https://github.com/rust-lang/crates.io-index)" = "489d6c0ed21b11d038c31b6ceccca973e65d73ba3bd8ecb9a2babf5546164643" +"checksum bincode 1.2.0 (registry+https://github.com/rust-lang/crates.io-index)" = "b8ab639324e3ee8774d296864fbc0dbbb256cf1a41c490b94cba90c082915f92" "checksum bindgen 0.47.3 (registry+https://github.com/rust-lang/crates.io-index)" = "df683a55b54b41d5ea8ebfaebb5aa7e6b84e3f3006a78f010dadc9ca88469260" -"checksum bitflags 1.2.0 (registry+https://github.com/rust-lang/crates.io-index)" = "8a606a02debe2813760609f57a64a2ffd27d9fdf5b2f133eaca0b248dd92cdd2" +"checksum bitflags 1.2.1 (registry+https://github.com/rust-lang/crates.io-index)" = "cf1de2fe8c75bc145a2f577add951f8134889b4795d47466a54a5c846d691693" "checksum bitmask 0.5.0 (registry+https://github.com/rust-lang/crates.io-index)" = "5da9b3d9f6f585199287a473f4f8dfab6566cf827d15c00c219f53c645687ead" "checksum bitvec 0.14.0 (registry+https://github.com/rust-lang/crates.io-index)" = "9633b74910e1870f50f5af189b08487195cdb83c0e27a71d6f64d5e09dd0538b" "checksum blake2 0.8.1 (registry+https://github.com/rust-lang/crates.io-index)" = "94cb07b0da6a73955f8fb85d24c466778e70cda767a568229b104f0264089330" "checksum blake2-rfc 0.2.18 (registry+https://github.com/rust-lang/crates.io-index)" = "5d6d530bdd2d52966a6d03b7a964add7ae1a288d25214066fd4b600f0f796400" -"checksum block-buffer 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)" = "1339a1042f5d9f295737ad4d9a6ab6bf81c84a933dba110b9200cd6d1448b814" +"checksum blake2b_simd 0.5.9 (registry+https://github.com/rust-lang/crates.io-index)" = "b83b7baab1e671718d78204225800d6b170e648188ac7dc992e9d6bddf87d0c0" "checksum block-buffer 0.7.3 (registry+https://github.com/rust-lang/crates.io-index)" = "c0940dc441f31689269e10ac70eb1002a3a1d3ad1390e030043662eb7fe4688b" "checksum block-cipher-trait 0.6.2 (registry+https://github.com/rust-lang/crates.io-index)" = "1c924d49bd09e7c06003acda26cd9742e796e34282ec6c1189404dee0c1f4774" -"checksum block-padding 0.1.4 (registry+https://github.com/rust-lang/crates.io-index)" = "6d4dc3af3ee2e12f3e5d224e5e1e3d73668abbeb69e566d361f7d5563a4fdf09" -"checksum bs58 0.2.5 (registry+https://github.com/rust-lang/crates.io-index)" = "c95ee6bba9d950218b6cc910cf62bc9e0a171d0f4537e3627b0f54d08549b188" +"checksum block-padding 0.1.5 (registry+https://github.com/rust-lang/crates.io-index)" = "fa79dedbb091f449f1f39e53edf88d5dbe95f895dae6135a8d7b881fb5af73f5" +"checksum bs58 0.3.0 (registry+https://github.com/rust-lang/crates.io-index)" = "b170cd256a3f9fa6b9edae3e44a7dfdfc77e8124dbc3e2612d75f9c3e2396dae" "checksum bstr 0.2.8 (registry+https://github.com/rust-lang/crates.io-index)" = "8d6c2c5b58ab920a4f5aeaaca34b4488074e8cc7596af94e6f8c6ff247c60245" "checksum build-helper 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)" = "bdce191bf3fa4995ce948c8c83b4640a1745457a149e73c6db75b4ffe36aad5f" "checksum bumpalo 2.6.0 (registry+https://github.com/rust-lang/crates.io-index)" = "ad807f2fc2bf185eeb98ff3a901bd46dc5ad58163d0fa4577ba0d25674d71708" -"checksum byte-slice-cast 0.3.2 (registry+https://github.com/rust-lang/crates.io-index)" = "7cbcbf18128ec71d8d4a0d054461ec59fff5b75b7d10a4c9b7c7cb1a379c3e77" -"checksum byte-tools 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)" = "560c32574a12a89ecd91f5e742165893f86e3ab98d21f8ea548658eb9eef5f40" +"checksum byte-slice-cast 0.3.4 (registry+https://github.com/rust-lang/crates.io-index)" = "f6209f3b2c1edea170002e016d5ead6903d3bb0a846477f53bbeb614967a52a9" "checksum byte-tools 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)" = "e3b5ca7a04898ad4bcd41c90c5285445ff5b791899bb1b0abdd2a2aa791211d7" "checksum byteorder 0.5.3 (registry+https://github.com/rust-lang/crates.io-index)" = "0fc10e8cc6b2580fda3f36eb6dc5316657f812a3df879a44a66fc9f0fdbc4855" "checksum byteorder 1.3.2 (registry+https://github.com/rust-lang/crates.io-index)" = "a7c3dd8985a7111efc5c80b44e23ecdd8c007de8ade3b96595387e812b957cf5" "checksum bytes 0.4.12 (registry+https://github.com/rust-lang/crates.io-index)" = "206fdffcfa2df7cbe15601ef46c813fce0965eb3286db6b56c583b814b51c81c" -"checksum c2-chacha 0.2.2 (registry+https://github.com/rust-lang/crates.io-index)" = "7d64d04786e0f528460fc884753cf8dddcc466be308f6026f8e355c41a0e4101" +"checksum c2-chacha 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)" = "214238caa1bf3a496ec3392968969cab8549f96ff30652c9e56885329315f6bb" "checksum c_linked_list 1.1.1 (registry+https://github.com/rust-lang/crates.io-index)" = "4964518bd3b4a8190e832886cdc0da9794f12e8e6c1613a9e90ff331c4c8724b" -"checksum cargo_metadata 0.8.2 (registry+https://github.com/rust-lang/crates.io-index)" = "700b3731fd7d357223d0000f4dbf1808401b694609035c3c411fbc0cd375c426" +"checksum cargo_metadata 0.9.0 (registry+https://github.com/rust-lang/crates.io-index)" = "8d2d1617e838936c0d2323a65cc151e03ae19a7678dd24f72bccf27119b90a5d" "checksum cast 0.2.2 (registry+https://github.com/rust-lang/crates.io-index)" = "926013f2860c46252efceabb19f4a6b308197505082c609025aa6706c011d427" -"checksum cc 1.0.45 (registry+https://github.com/rust-lang/crates.io-index)" = "4fc9a35e1f4290eb9e5fc54ba6cf40671ed2a2514c3eeb2b2a908dda2ea5a1be" -"checksum cexpr 0.3.5 (registry+https://github.com/rust-lang/crates.io-index)" = "a7fa24eb00d5ffab90eaeaf1092ac85c04c64aaf358ea6f84505b8116d24c6af" +"checksum cc 1.0.47 (registry+https://github.com/rust-lang/crates.io-index)" = "aa87058dce70a3ff5621797f1506cb837edd02ac4c0ae642b4542dce802908b8" +"checksum cexpr 0.3.6 (registry+https://github.com/rust-lang/crates.io-index)" = "fce5b5fb86b0c57c20c834c1b412fd09c77c8a59b9473f86272709e78874cd1d" "checksum cfg-if 0.1.10 (registry+https://github.com/rust-lang/crates.io-index)" = "4785bdd1c96b2a846b2bd7cc02e86b6b3dbf14e7e53446c4f54c92a361040822" "checksum chrono 0.4.9 (registry+https://github.com/rust-lang/crates.io-index)" = "e8493056968583b0193c1bb04d6f7684586f3726992d6c573261941a895dbd68" "checksum clang-sys 0.26.4 (registry+https://github.com/rust-lang/crates.io-index)" = "6ef0c1bcf2e99c649104bd7a7012d8f8802684400e03db0ec0af48583c6fa0e4" @@ -6909,26 +8000,39 @@ dependencies = [ "checksum clear_on_drop 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)" = "97276801e127ffb46b66ce23f35cc96bd454fa311294bced4bbace7baa8b1d17" "checksum cloudabi 0.0.3 (registry+https://github.com/rust-lang/crates.io-index)" = "ddfc5b9aa5d4507acaf872de71051dfd0e309860e88966e1051e462a077aac4f" "checksum cmake 0.1.42 (registry+https://github.com/rust-lang/crates.io-index)" = "81fb25b677f8bf1eb325017cb6bb8452f87969db0fedb4f757b297bee78a7c62" +"checksum console_error_panic_hook 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)" = "b8d976903543e0c48546a91908f21588a680a8c8f984df9a5d69feccb2b2a211" +"checksum console_log 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)" = "1e7871d2947441b0fdd8e2bd1ce2a2f75304f896582c0d572162d48290683c48" "checksum const-random 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)" = "7b641a8c9867e341f3295564203b1c250eb8ce6cb6126e007941f78c4d2ed7fe" "checksum const-random-macro 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)" = "c750ec12b83377637110d5a57f5ae08e895b06c4b16e2bdbf1a94ef717428c59" "checksum constant_time_eq 0.1.4 (registry+https://github.com/rust-lang/crates.io-index)" = "995a44c877f9212528ccc74b21a232f66ad69001e40ede5bcee2ac9ef2657120" +"checksum cookie 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)" = "888604f00b3db336d2af898ec3c1d5d0ddf5e6d462220f2ededc33a87ac4bbd5" +"checksum cookie_store 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)" = "46750b3f362965f197996c4448e4a0935e791bf7d6631bfce9ee0af3d24c919c" "checksum core-foundation 0.6.4 (registry+https://github.com/rust-lang/crates.io-index)" = "25b9e03f145fd4f2bf705e07b900cd41fc636598fe5dc452fd0db1441c3f496d" "checksum core-foundation-sys 0.6.2 (registry+https://github.com/rust-lang/crates.io-index)" = "e7ca8a5221364ef15ce201e8ed2f609fc312682a8f4e0e3d4aa5879764e0fa3b" +"checksum cranelift-bforest 0.46.1 (registry+https://github.com/rust-lang/crates.io-index)" = "18c97588946d3e5fe11f8e34ebf8cc65fd3fda50f3ffa2e80c98b2748058f00f" +"checksum cranelift-codegen 0.46.1 (registry+https://github.com/rust-lang/crates.io-index)" = "3255935da50302bcb0f7109f2fef27f44b46f1c797dfa7db971379261023adcd" +"checksum cranelift-codegen-meta 0.46.1 (registry+https://github.com/rust-lang/crates.io-index)" = "fd57265ef5e6ff253c378b6261ed8c2e6cb1b15e91624540dbd09b1e5a40e9ca" +"checksum cranelift-codegen-shared 0.46.1 (registry+https://github.com/rust-lang/crates.io-index)" = "c093398d21f9493ab29445191362592ef621f497e56a8efb15bdf80471978b7a" +"checksum cranelift-entity 0.46.1 (registry+https://github.com/rust-lang/crates.io-index)" = "e915fa58d2a75e3c4b768b7e4760282889915c3fcd9ccb2ad2b3ebec99654a78" +"checksum cranelift-frontend 0.46.1 (registry+https://github.com/rust-lang/crates.io-index)" = "46963952cda267bd0177b3f036e50038cd56e7b4c5b09a455b02df727e0f2a16" +"checksum cranelift-native 0.46.1 (registry+https://github.com/rust-lang/crates.io-index)" = "7ba8a2d69ddd4729199a321bc2f4020e1969a088b468ed6a29dc7a69350be76e" +"checksum cranelift-wasm 0.46.1 (registry+https://github.com/rust-lang/crates.io-index)" = "5a802357a6a016bf4c1dcdc6d73a650640eb3b613cc098a1a044a6c3731ca264" "checksum crc32fast 1.2.0 (registry+https://github.com/rust-lang/crates.io-index)" = "ba125de2af0df55319f41944744ad91c71113bf74a4646efff39afe1f6842db1" "checksum criterion 0.2.11 (registry+https://github.com/rust-lang/crates.io-index)" = "0363053954f3e679645fc443321ca128b7b950a6fe288cf5f9335cc22ee58394" "checksum criterion 0.3.0 (registry+https://github.com/rust-lang/crates.io-index)" = "938703e165481c8d612ea3479ac8342e5615185db37765162e762ec3523e2fc6" "checksum criterion-plot 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)" = "76f9212ddf2f4a9eb2d401635190600656a1f88a932ef53d06e7fa4c7e02fb8e" "checksum criterion-plot 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)" = "eccdc6ce8bbe352ca89025bee672aa6d24f4eb8c53e3a8b5d1bc58011da072a2" "checksum crossbeam-channel 0.3.9 (registry+https://github.com/rust-lang/crates.io-index)" = "c8ec7fcd21571dc78f96cc96243cab8d8f035247c3efd16c687be154c3fa9efa" -"checksum crossbeam-deque 0.7.1 (registry+https://github.com/rust-lang/crates.io-index)" = "b18cd2e169ad86297e6bc0ad9aa679aee9daa4f19e8163860faf7c164e4f5a71" -"checksum crossbeam-epoch 0.7.2 (registry+https://github.com/rust-lang/crates.io-index)" = "fedcd6772e37f3da2a9af9bf12ebe046c0dfe657992377b4df982a2b54cd37a9" +"checksum crossbeam-deque 0.7.2 (registry+https://github.com/rust-lang/crates.io-index)" = "c3aa945d63861bfe624b55d153a39684da1e8c0bc8fba932f7ee3a3c16cea3ca" +"checksum crossbeam-epoch 0.8.0 (registry+https://github.com/rust-lang/crates.io-index)" = "5064ebdbf05ce3cb95e45c8b086f72263f4166b29b97f6baff7ef7fe047b55ac" "checksum crossbeam-queue 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)" = "7c979cd6cfe72335896575c6b5688da489e420d36a27a0b9eb0c73db574b4a4b" "checksum crossbeam-utils 0.6.6 (registry+https://github.com/rust-lang/crates.io-index)" = "04973fa96e96579258a5091af6003abde64af786b860f18622b82e026cca60e6" +"checksum crossbeam-utils 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)" = "ce446db02cdc3165b94ae73111e570793400d0794e46125cc4056c81cbb039f4" "checksum crunchy 0.2.2 (registry+https://github.com/rust-lang/crates.io-index)" = "7a81dae078cea95a014a339291cec439d2f232ebe854a9d672b796c6afafa9b7" -"checksum crypto-mac 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)" = "779015233ac67d65098614aec748ac1c756ab6677fa2e14cf8b37c08dfed1198" "checksum crypto-mac 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)" = "4434400df11d95d556bac068ddfedd482915eb18fe8bea89bc80b6e4b1c179e5" "checksum csv 1.1.1 (registry+https://github.com/rust-lang/crates.io-index)" = "37519ccdfd73a75821cac9319d4fce15a81b9fcf75f951df5b9988aa3a0af87d" "checksum csv-core 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)" = "9b5cadb6b25c77aeff80ba701712494213f4a8418fcda2ee11b6560c3ad0bf4c" +"checksum ct-logs 0.6.0 (registry+https://github.com/rust-lang/crates.io-index)" = "4d3686f5fa27dbc1d76c751300376e167c5a43387f44bb451fd1c24776e49113" "checksum ctor 0.1.12 (registry+https://github.com/rust-lang/crates.io-index)" = "cd8ce37ad4184ab2ce004c33bf6379185d3b1c95801cab51026bd271bf68eedc" "checksum ctr 0.3.2 (registry+https://github.com/rust-lang/crates.io-index)" = "022cd691704491df67d25d006fe8eca083098253c4d43516c2206479c58c6736" "checksum ctrlc 3.1.3 (registry+https://github.com/rust-lang/crates.io-index)" = "c7dfd2d8b4c82121dfdff120f818e09fc4380b0b7e17a742081a89b94853e87f" @@ -6937,29 +8041,40 @@ dependencies = [ "checksum data-encoding 2.1.2 (registry+https://github.com/rust-lang/crates.io-index)" = "f4f47ca1860a761136924ddd2422ba77b2ea54fe8cc75b9040804a0d9d32ad97" "checksum derive_more 0.15.0 (registry+https://github.com/rust-lang/crates.io-index)" = "7a141330240c921ec6d074a3e188a7c7ef95668bb95e7d44fa0e5778ec2a7afe" "checksum difference 2.0.0 (registry+https://github.com/rust-lang/crates.io-index)" = "524cbf6897b527295dff137cec09ecf3a05f4fddffd7dfcd1585403449e74198" -"checksum digest 0.6.2 (registry+https://github.com/rust-lang/crates.io-index)" = "e5b29bf156f3f4b3c4f610a25ff69370616ae6e0657d416de22645483e72af0a" "checksum digest 0.8.1 (registry+https://github.com/rust-lang/crates.io-index)" = "f3d0c8c8752312f9713efd397ff63acb9f85585afbf179282e720e7704954dd5" +"checksum directories 2.0.2 (registry+https://github.com/rust-lang/crates.io-index)" = "551a778172a450d7fc12e629ca3b0428d00f6afa9a43da1b630d54604e97371c" +"checksum dirs-sys 0.3.4 (registry+https://github.com/rust-lang/crates.io-index)" = "afa0b23de8fd801745c471deffa6e12d248f962c9fd4b4c33787b055599bde7b" "checksum dns-parser 0.8.0 (registry+https://github.com/rust-lang/crates.io-index)" = "c4d33be9473d06f75f58220f71f7a9317aca647dc061dbd3c361b0bef505fbea" "checksum doc-comment 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)" = "923dea538cea0aa3025e8685b20d6ee21ef99c4f77e954a30febbaac5ec73a97" +"checksum dtoa 0.4.4 (registry+https://github.com/rust-lang/crates.io-index)" = "ea57b42383d091c85abcc2706240b94ab2a8fa1fc81c10ff23c4de06e2a90b5e" "checksum ed25519-dalek 0.9.1 (registry+https://github.com/rust-lang/crates.io-index)" = "2d07e8b8a8386c3b89a7a4b329fdfa4cb545de2545e9e2ebbc3dd3929253e426" "checksum ed25519-dalek 1.0.0-pre.2 (registry+https://github.com/rust-lang/crates.io-index)" = "845aaacc16f01178f33349e7c992ecd0cee095aa5e577f0f4dee35971bd36455" "checksum either 1.5.3 (registry+https://github.com/rust-lang/crates.io-index)" = "bb1f6b1ce1c140482ea30ddd3335fc0024ac7ee112895426e0a629a6c20adfe3" "checksum elastic-array 0.10.2 (registry+https://github.com/rust-lang/crates.io-index)" = "073be79b6538296faf81c631872676600616073817dd9a440c477ad09b408983" +"checksum encoding_rs 0.8.20 (registry+https://github.com/rust-lang/crates.io-index)" = "87240518927716f79692c2ed85bfe6e98196d18c6401ec75355760233a7e12e9" "checksum env_logger 0.6.2 (registry+https://github.com/rust-lang/crates.io-index)" = "aafcde04e90a5226a6443b7aabdb016ba2f8307c847d524724bd9b346dd1a2d3" -"checksum env_logger 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)" = "39ecdb7dd54465526f0a56d666e3b2dd5f3a218665a030b6e4ad9e70fa95d8fa" +"checksum env_logger 0.7.1 (registry+https://github.com/rust-lang/crates.io-index)" = "44533bbbb3bb3c1fa17d9f2e4e38bbbaf8396ba82193c4cb1b6445d711445d36" "checksum environmental 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)" = "34f8467a0284de039e6bd0e25c14519538462ba5beb548bb1f03e645097837a8" "checksum erased-serde 0.3.9 (registry+https://github.com/rust-lang/crates.io-index)" = "3beee4bc16478a1b26f2e80ad819a52d24745e292f521a63c16eea5f74b7eb60" +"checksum errno 0.2.4 (registry+https://github.com/rust-lang/crates.io-index)" = "c2a071601ed01b988f896ab14b95e67335d1eeb50190932a1320f7fe3cadc84e" +"checksum errno-dragonfly 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)" = "14ca354e36190500e1e1fb267c647932382b54053c50b14970856c0b00a35067" "checksum error-chain 0.12.1 (registry+https://github.com/rust-lang/crates.io-index)" = "3ab49e9dcb602294bc42f9a7dfc9bc6e936fca4418ea300dbfb84fe16de0b7d9" +"checksum evm 0.14.0 (registry+https://github.com/rust-lang/crates.io-index)" = "1138816a9b7f9a9d1fcabb1b8a7afed2687d035692baf297bd3fea122acdc96f" +"checksum evm-core 0.14.0 (registry+https://github.com/rust-lang/crates.io-index)" = "3bcde5af3d542874ddeb53de0919302d57586ea04b3f76f54d865f8a6cdc70ae" +"checksum evm-gasometer 0.14.0 (registry+https://github.com/rust-lang/crates.io-index)" = "b82bc9f275cb59d2bcc05d85c98736ddfaba003a7ef7b73893fa7c1c1fab29dc" +"checksum evm-runtime 0.14.1 (registry+https://github.com/rust-lang/crates.io-index)" = "0dbbc89d29618c3722c17ba78ddf432d40ace8ee27e3f8b28b52a85921112e4b" "checksum exit-future 0.1.4 (registry+https://github.com/rust-lang/crates.io-index)" = "d8013f441e38e31c670e7f34ec8f1d5d3a2bd9d303c1ff83976ca886005e8f48" +"checksum faerie 0.11.0 (registry+https://github.com/rust-lang/crates.io-index)" = "875d78b92b2a4d9e1e2c7eeccfa30a327d2ee6434db3beb8fd6fd92f41898bc4" "checksum failure 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)" = "f8273f13c977665c5db7eb2b99ae520952fe5ac831ae4cd09d80c4c7042b5ed9" "checksum failure_derive 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)" = "0bc225b78e0391e4b8683440bf2e63c2deeeb2ce5189eab46e2b68c6d3725d08" "checksum fake-simd 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)" = "e88a8acf291dafb59c2d96e8f59828f3838bb1a70398823ade51a84de6a6deed" +"checksum fallible-iterator 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)" = "4443176a9f2c162692bd3d352d745ef9413eec5782a80d8fd6f8a1ac692a07f7" "checksum fdlimit 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)" = "b1ee15a7050e5580b3712877157068ea713b245b080ff302ae2ca973cfcd9baa" -"checksum finality-grandpa 0.9.0 (registry+https://github.com/rust-lang/crates.io-index)" = "9681c1f75941ea47584573dd2bc10558b2067d460612945887e00744e43393be" -"checksum fixed-hash 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)" = "516877b7b9a1cc2d0293cbce23cd6203f0edbfd4090e6ca4489fecb5aa73050e" -"checksum fixed-hash 0.5.0 (registry+https://github.com/rust-lang/crates.io-index)" = "6357b15872f8126e4ea7cf79d579473f132ccd2de239494ad1bf4aa892faea68" +"checksum file-per-thread-logger 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)" = "8505b75b31ef7285168dd237c4a7db3c1f3e0927e7d314e670bc98e854272fe9" +"checksum finality-grandpa 0.9.1 (registry+https://github.com/rust-lang/crates.io-index)" = "34754852da8d86bc509715292c73140a5b678656d0b16132acd6737bdb5fd5f8" +"checksum fixed-hash 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)" = "72fe7539e2c5692c6989f2f9c0457e42f1e5768f96b85c87d273574670ae459f" "checksum fixedbitset 0.1.9 (registry+https://github.com/rust-lang/crates.io-index)" = "86d4de0081402f5e88cdac65c8dcdcc73118c1a7a465e2a05f0da05843a8ea33" -"checksum flate2 1.0.12 (registry+https://github.com/rust-lang/crates.io-index)" = "ad3c5233c9a940c8719031b423d7e6c16af66e031cb0420b0896f5245bf181d3" +"checksum flate2 1.0.13 (registry+https://github.com/rust-lang/crates.io-index)" = "6bd6d6f4752952feb71363cffc9ebac9411b75b87c6ab6058c40c8900cf43c0f" "checksum fnv 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)" = "2fad85553e09a6f881f739c29f0b00b0f01357c743266d478b68951ce23285f3" "checksum foreign-types 0.3.2 (registry+https://github.com/rust-lang/crates.io-index)" = "f6f339eb8adc052cd2ca78910fda869aefa38d22d5cb648e6485e4d3fc06f3b1" "checksum foreign-types-shared 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)" = "00b0228411908ca8685dba7fc2cdd70ec9990a6e753e89b6ac91a84c40fbaf4b" @@ -6969,68 +8084,79 @@ dependencies = [ "checksum fuchsia-zircon 0.3.3 (registry+https://github.com/rust-lang/crates.io-index)" = "2e9763c69ebaae630ba35f74888db465e49e259ba1bc0eda7d06f4a067615d82" "checksum fuchsia-zircon-sys 0.3.3 (registry+https://github.com/rust-lang/crates.io-index)" = "3dcaa9ae7725d12cdb85b3ad99a434db70b468c09ded17e012d86b5c1010f7a7" "checksum futures 0.1.29 (registry+https://github.com/rust-lang/crates.io-index)" = "1b980f2816d6ee8673b6517b52cb0e808a180efc92e5c19d02cdda79066703ef" +"checksum futures 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)" = "b6f16056ecbb57525ff698bb955162d0cd03bee84e6241c27ff75c08d8ca5987" +"checksum futures-channel 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)" = "fcae98ca17d102fd8a3603727b9259fcf7fa4239b603d2142926189bc8999b86" "checksum futures-channel-preview 0.3.0-alpha.19 (registry+https://github.com/rust-lang/crates.io-index)" = "d5e5f4df964fa9c1c2f8bddeb5c3611631cacd93baf810fc8bb2fb4b495c263a" +"checksum futures-core 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)" = "79564c427afefab1dfb3298535b21eda083ef7935b4f0ecbfcb121f0aec10866" "checksum futures-core-preview 0.3.0-alpha.19 (registry+https://github.com/rust-lang/crates.io-index)" = "b35b6263fb1ef523c3056565fa67b1d16f0a8604ff12b11b08c25f28a734c60a" "checksum futures-cpupool 0.1.8 (registry+https://github.com/rust-lang/crates.io-index)" = "ab90cde24b3319636588d0c35fe03b1333857621051837ed769faefb4c2162e4" +"checksum futures-executor 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)" = "1e274736563f686a837a0568b478bdabfeaec2dca794b5649b04e2fe1627c231" "checksum futures-executor-preview 0.3.0-alpha.19 (registry+https://github.com/rust-lang/crates.io-index)" = "75236e88bd9fe88e5e8bfcd175b665d0528fe03ca4c5207fabc028c8f9d93e98" +"checksum futures-io 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)" = "e676577d229e70952ab25f3945795ba5b16d63ca794ca9d2c860e5595d20b5ff" "checksum futures-io-preview 0.3.0-alpha.19 (registry+https://github.com/rust-lang/crates.io-index)" = "f4914ae450db1921a56c91bde97a27846287d062087d4a652efc09bb3a01ebda" +"checksum futures-macro 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)" = "52e7c56c15537adb4f76d0b7a76ad131cb4d2f4f32d3b0bcabcbe1c7c5e87764" "checksum futures-preview 0.3.0-alpha.19 (registry+https://github.com/rust-lang/crates.io-index)" = "3b1dce2a0267ada5c6ff75a8ba864b4e679a9e2aa44262af7a3b5516d530d76e" +"checksum futures-sink 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)" = "171be33efae63c2d59e6dbba34186fe0d6394fb378069a76dfd80fdcffd43c16" "checksum futures-sink-preview 0.3.0-alpha.19 (registry+https://github.com/rust-lang/crates.io-index)" = "86f148ef6b69f75bb610d4f9a2336d4fc88c4b5b67129d1a340dd0fd362efeec" +"checksum futures-task 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)" = "0bae52d6b29cf440e298856fec3965ee6fa71b06aa7495178615953fd669e5f9" "checksum futures-timer 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)" = "878f1d2fc31355fa02ed2372e741b0c17e58373341e6a122569b4623a14a7d33" +"checksum futures-util 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)" = "c0d66274fb76985d3c62c886d1da7ac4c0903a8c9f754e8fe0f35a6a6cc39e76" "checksum futures-util-preview 0.3.0-alpha.19 (registry+https://github.com/rust-lang/crates.io-index)" = "5ce968633c17e5f97936bd2797b6e38fb56cf16a7422319f7ec2e30d3c470e8d" "checksum gcc 0.3.55 (registry+https://github.com/rust-lang/crates.io-index)" = "8f5f3913fa0bfe7ee1fd8248b6b9f42a5af4b9d65ec2dd2c3c26132b950ecfc2" "checksum generic-array 0.12.3 (registry+https://github.com/rust-lang/crates.io-index)" = "c68f0274ae0e023facc3c97b2e00f076be70e254bc851d972503b328db79b2ec" -"checksum generic-array 0.8.3 (registry+https://github.com/rust-lang/crates.io-index)" = "fceb69994e330afed50c93524be68c42fa898c2d9fd4ee8da03bd7363acd26f2" "checksum get_if_addrs 0.5.3 (registry+https://github.com/rust-lang/crates.io-index)" = "abddb55a898d32925f3148bd281174a68eeb68bbfd9a5938a57b18f506ee4ef7" "checksum get_if_addrs-sys 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)" = "0d04f9fb746cf36b191c00f3ede8bde9c8e64f9f4b05ae2694a9ccf5e3f5ab48" -"checksum getrandom 0.1.12 (registry+https://github.com/rust-lang/crates.io-index)" = "473a1265acc8ff1e808cd0a1af8cee3c2ee5200916058a2ca113c29f2d903571" +"checksum getrandom 0.1.13 (registry+https://github.com/rust-lang/crates.io-index)" = "e7db7ca94ed4cd01190ceee0d8a8052f08a247aa1b469a7f68c6a3b71afcf407" +"checksum gimli 0.19.0 (registry+https://github.com/rust-lang/crates.io-index)" = "162d18ae5f2e3b90a993d202f1ba17a5633c2484426f8bcae201f86194bacd00" "checksum glob 0.2.11 (registry+https://github.com/rust-lang/crates.io-index)" = "8be18de09a56b60ed0edf84bc9df007e30040691af7acd1c41874faac5895bfb" "checksum glob 0.3.0 (registry+https://github.com/rust-lang/crates.io-index)" = "9b919933a397b79c37e33b77bb2aa3dc8eb6e165ad809e58ff75bc7db2e34574" "checksum globset 0.4.4 (registry+https://github.com/rust-lang/crates.io-index)" = "925aa2cac82d8834e2b2a4415b6f6879757fb5c0928fc445ae76461a12eed8f2" +"checksum goblin 0.0.24 (registry+https://github.com/rust-lang/crates.io-index)" = "e3fa261d919c1ae9d1e4533c4a2f99e10938603c4208d56c05bec7a872b661b0" "checksum h2 0.1.26 (registry+https://github.com/rust-lang/crates.io-index)" = "a5b34c246847f938a410a03c5458c7fee2274436675e76d8b903c08efc29c462" "checksum hash-db 0.15.2 (registry+https://github.com/rust-lang/crates.io-index)" = "d23bd4e7b5eda0d0f3a307e8b381fdc8ba9000f26fbe912250c0a4cc3956364a" "checksum hash256-std-hasher 0.15.2 (registry+https://github.com/rust-lang/crates.io-index)" = "92c171d55b98633f4ed3860808f004099b36c1cc29c42cfc53aa8591b21efcf2" "checksum hashbrown 0.1.8 (registry+https://github.com/rust-lang/crates.io-index)" = "3bae29b6653b3412c2e71e9d486db9f9df5d701941d86683005efb9f2d28e3da" -"checksum hashbrown 0.6.1 (registry+https://github.com/rust-lang/crates.io-index)" = "6587d09be37fb98a11cb08b9000a3f592451c1b1b613ca69d949160e313a430a" -"checksum hashmap_core 0.1.11 (registry+https://github.com/rust-lang/crates.io-index)" = "2d6852e5a86250521973b0c1d39677166d8a9c0047c908d7e04f1aa04177973c" +"checksum hashbrown 0.6.3 (registry+https://github.com/rust-lang/crates.io-index)" = "8e6073d0ca812575946eb5f35ff68dbe519907b25c42530389ff946dc84c6ead" "checksum heapsize 0.4.2 (registry+https://github.com/rust-lang/crates.io-index)" = "1679e6ea370dee694f91f1dc469bf94cf8f52051d147aec3e1f9497c6fc22461" "checksum heck 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)" = "20564e78d53d2bb135c343b3f47714a56af2061f1c928fdb541dc7b9fdd94205" +"checksum hermit-abi 0.1.3 (registry+https://github.com/rust-lang/crates.io-index)" = "307c3c9f937f38e3534b1d6447ecf090cafcc9744e4a6360e8b037b2cf5af120" "checksum hex 0.3.2 (registry+https://github.com/rust-lang/crates.io-index)" = "805026a5d0141ffc30abb3be3173848ad46a1b1664fe632428479619a3644d77" +"checksum hex 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)" = "023b39be39e3a2da62a94feb433e91e8bcd37676fbc8bea371daf52b7a769a3e" "checksum hex-literal 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)" = "961de220ec9a91af2e1e5bd80d02109155695e516771762381ef8581317066e0" "checksum hex-literal-impl 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)" = "9d4c5c844e2fee0bf673d54c2c177f1713b3d2af2ff6e666b49cb7572e6cf42d" -"checksum hmac 0.4.2 (registry+https://github.com/rust-lang/crates.io-index)" = "7a13f4163aa0c5ca1be584aace0e2212b2e41be5478218d4f657f5f778b2ae2a" "checksum hmac 0.7.1 (registry+https://github.com/rust-lang/crates.io-index)" = "5dcb5e64cda4c23119ab41ba960d1e170a774c8e4b9d9e6a9bc18aabf5e59695" -"checksum hmac-drbg 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)" = "4fe727d41d2eec0a6574d887914347e5ff96a3b87177817e2a9820c5c87fecc2" "checksum hmac-drbg 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)" = "c6e570451493f10f6581b48cdd530413b63ea9e780f544bfd3bdcaa0d89d1a7b" -"checksum http 0.1.18 (registry+https://github.com/rust-lang/crates.io-index)" = "372bcb56f939e449117fb0869c2e8fd8753a8223d92a172c6e808cf123a5b6e4" +"checksum http 0.1.19 (registry+https://github.com/rust-lang/crates.io-index)" = "d7e06e336150b178206af098a055e3621e8336027e2b4d126bda0bc64824baaf" "checksum http-body 0.1.0 (registry+https://github.com/rust-lang/crates.io-index)" = "6741c859c1b2463a423a1dbce98d418e6c3c3fc720fb0d45528657320920292d" "checksum httparse 1.3.4 (registry+https://github.com/rust-lang/crates.io-index)" = "cd179ae861f0c2e53da70d892f5f3029f9594be0c41dc5269cd371691b1dc2f9" "checksum humantime 1.3.0 (registry+https://github.com/rust-lang/crates.io-index)" = "df004cfca50ef23c36850aaaa59ad52cc70d0e90243c3c7737a4dd32dc7a3c4f" "checksum hyper 0.10.16 (registry+https://github.com/rust-lang/crates.io-index)" = "0a0652d9a2609a968c14be1a9ea00bf4b1d64e2e1f53a1b51b6fff3a6e829273" "checksum hyper 0.12.35 (registry+https://github.com/rust-lang/crates.io-index)" = "9dbe6ed1438e1f8ad955a4701e9a944938e9519f6888d12d8558b645e247d5f6" +"checksum hyper-rustls 0.17.1 (registry+https://github.com/rust-lang/crates.io-index)" = "719d85c7df4a7f309a77d145340a063ea929dcb2e025bae46a80345cffec2952" "checksum hyper-tls 0.3.2 (registry+https://github.com/rust-lang/crates.io-index)" = "3a800d6aa50af4b5850b2b0f659625ce9504df908e9733b635720483be26174f" "checksum idna 0.1.5 (registry+https://github.com/rust-lang/crates.io-index)" = "38f09e0f0b1fb55fdee1f17470ad800da77af5186a1a76c026b679358b7e844e" "checksum idna 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)" = "02e2673c30ee86b5b96a9cb52ad15718aa1f966f5ab9ad54a8b95d5ca33120a9" -"checksum impl-codec 0.4.1 (registry+https://github.com/rust-lang/crates.io-index)" = "3fa0086251524c50fd53b32e7b05eb6d79e2f97221eaf0c53c0ca9c3096f21d3" +"checksum impl-codec 0.4.2 (registry+https://github.com/rust-lang/crates.io-index)" = "1be51a921b067b0eaca2fad532d9400041561aa922221cc65f95a85641c6bf53" +"checksum impl-rlp 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)" = "8f7a72f11830b52333f36e3b09a288333888bf54380fd0ac0790a3c31ab0f3c5" "checksum impl-serde 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)" = "58e3cae7e99c7ff5a995da2cf78dd0a5383740eda71d98cf7b1910c301ac69b8" "checksum impl-trait-for-tuples 0.1.3 (registry+https://github.com/rust-lang/crates.io-index)" = "7ef5550a42e3740a0e71f909d4c861056a284060af885ae7aa6242820f920d9d" -"checksum indexmap 1.2.0 (registry+https://github.com/rust-lang/crates.io-index)" = "a61202fbe46c4a951e9404a720a0180bcf3212c750d735cb5c4ba4dc551299f3" +"checksum indexmap 1.3.0 (registry+https://github.com/rust-lang/crates.io-index)" = "712d7b3ea5827fcb9d4fda14bf4da5f136f0db2ae9c8f4bd4e2d1c6fde4e6db2" "checksum integer-sqrt 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)" = "ea155abb3ba6f382a75f1418988c05fe82959ed9ce727de427f9cfd425b0c903" "checksum interleaved-ordered 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)" = "141340095b15ed7491bd3d4ced9d20cebfb826174b6bb03386381f62b01e3d77" "checksum iovec 0.1.4 (registry+https://github.com/rust-lang/crates.io-index)" = "b2b3ea6ff95e175473f8ffe6a7eb7c00d054240321b84c57051175fe3c1e075e" -"checksum ipnet 2.0.1 (registry+https://github.com/rust-lang/crates.io-index)" = "6cc15ac2e0886d62ba078989ef6920ab23997ab0b04ca5687f1a9a7484296a48" -"checksum itertools 0.8.0 (registry+https://github.com/rust-lang/crates.io-index)" = "5b8467d9c1cebe26feb08c640139247fac215782d35371ade9a2136ed6085358" +"checksum ipnet 2.1.0 (registry+https://github.com/rust-lang/crates.io-index)" = "f2f4b06b21db0228860c8dfd17d2106c49c7c6bd07477a4036985347d84def04" +"checksum itertools 0.8.1 (registry+https://github.com/rust-lang/crates.io-index)" = "87fa75c9dea7b07be3138c49abbb83fd4bea199b5cdc76f9804458edc5da0d6e" "checksum itoa 0.4.4 (registry+https://github.com/rust-lang/crates.io-index)" = "501266b7edd0174f8530248f87f99c88fbe60ca4ef3dd486835b8d8d53136f7f" "checksum jobserver 0.1.17 (registry+https://github.com/rust-lang/crates.io-index)" = "f2b1d42ef453b30b7387e113da1c83ab1605d90c5b4e0eb8e96d016ed3b8c160" -"checksum js-sys 0.3.28 (registry+https://github.com/rust-lang/crates.io-index)" = "2cc9a97d7cec30128fd8b28a7c1f9df1c001ceb9b441e2b755e24130a6b43c79" -"checksum jsonrpc-client-transports 13.2.0 (registry+https://github.com/rust-lang/crates.io-index)" = "8dbf2466adbf6d5b4e618857f22be40b1e1cc6ed79d72751324358f6b539b06d" -"checksum jsonrpc-core 13.2.0 (registry+https://github.com/rust-lang/crates.io-index)" = "91d767c183a7e58618a609499d359ce3820700b3ebb4823a18c343b4a2a41a0d" -"checksum jsonrpc-core-client 13.2.0 (registry+https://github.com/rust-lang/crates.io-index)" = "161dc223549fa6fe4a4eda675de2d1d3cff5a7164e5c031cdf1e22c734700f8b" -"checksum jsonrpc-derive 13.2.0 (registry+https://github.com/rust-lang/crates.io-index)" = "4a76285ebba4515680fbfe4b62498ccb2a932384c8732eed68351b02fb7ae475" -"checksum jsonrpc-http-server 13.2.0 (registry+https://github.com/rust-lang/crates.io-index)" = "601fcc7bec888c7cbc7fd124d3d6744d72c0ebb540eca6fe2261b71f9cff6320" -"checksum jsonrpc-pubsub 13.2.0 (registry+https://github.com/rust-lang/crates.io-index)" = "64e0fb0664d8ce287e826940dafbb45379443c595bdd71d93655f3c8f25fd992" -"checksum jsonrpc-server-utils 13.2.0 (registry+https://github.com/rust-lang/crates.io-index)" = "4d415f51d016a4682878e19dd03e8c0b61cd4394912d7cd3dc48d4f19f061a4e" -"checksum jsonrpc-ws-server 13.2.0 (registry+https://github.com/rust-lang/crates.io-index)" = "4699433c1ac006d7df178b4c29c191e5bb6d81e2dca18c5c804a094592900101" +"checksum js-sys 0.3.31 (registry+https://github.com/rust-lang/crates.io-index)" = "d8657b7ca06a6044ece477f6900bf7670f8b5fd0cce177a1d7094eef51e0adf4" +"checksum jsonrpc-client-transports 14.0.3 (registry+https://github.com/rust-lang/crates.io-index)" = "d389a085cb2184604dff060390cadb8cba1f063c7fd0ad710272c163c88b9f20" +"checksum jsonrpc-core 14.0.3 (registry+https://github.com/rust-lang/crates.io-index)" = "34651edf3417637cc45e70ed0182ecfa9ced0b7e8131805fccf7400d989845ca" +"checksum jsonrpc-core-client 14.0.3 (registry+https://github.com/rust-lang/crates.io-index)" = "9dbaec1d57271ff952f24ca79d37d716cfd749c855b058d9aa5f053a6b8ae4ef" +"checksum jsonrpc-derive 14.0.3 (registry+https://github.com/rust-lang/crates.io-index)" = "d5d5c31575cc70a8b21542599028472c80a9248394aeea4d8918a045a0ab08a3" +"checksum jsonrpc-http-server 14.0.3 (registry+https://github.com/rust-lang/crates.io-index)" = "aa54c4c2d88cb5e04b251a5031ba0f2ee8c6ef30970e31228955b89a80c3b611" +"checksum jsonrpc-pubsub 14.0.3 (registry+https://github.com/rust-lang/crates.io-index)" = "3ee1b8da0b9219a231c4b7cbc7110bfdb457cbcd8d90a6224d0b3cab8aae8443" +"checksum jsonrpc-server-utils 14.0.3 (registry+https://github.com/rust-lang/crates.io-index)" = "87bc3c0a9a282211b2ec14abb3e977de33016bbec495332e9f7be858de7c5117" +"checksum jsonrpc-ws-server 14.0.3 (registry+https://github.com/rust-lang/crates.io-index)" = "af36a129cef77a9db8028ac7552d927e1bb7b6928cd96b23dd25cc38bff974ab" "checksum keccak 0.1.0 (registry+https://github.com/rust-lang/crates.io-index)" = "67c21572b4949434e4fc1e1978b99c5f77064153c59d998bf13ecd96fb5ecba7" "checksum keccak-hasher 0.15.2 (registry+https://github.com/rust-lang/crates.io-index)" = "3468207deea1359a0e921591ae9b4c928733d94eb9d6a2eeda994cfd59f42cf8" "checksum kernel32-sys 0.2.2 (registry+https://github.com/rust-lang/crates.io-index)" = "7507624b29483431c0ba2d82aece8ca6cdba9382bff4ddd0f7490560c056098d" @@ -7040,32 +8166,30 @@ dependencies = [ "checksum language-tags 0.2.2 (registry+https://github.com/rust-lang/crates.io-index)" = "a91d884b6667cd606bb5a69aa0c99ba811a115fc68915e7056ec08a46e93199a" "checksum lazy_static 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)" = "e2abad23fbc42b3700f2f279844dc832adb2b2eb069b2df918f455c4e18cc646" "checksum lazycell 1.2.1 (registry+https://github.com/rust-lang/crates.io-index)" = "b294d6fa9ee409a054354afc4352b0b9ef7ca222c69b8812cbea9e7d2bf3783f" -"checksum libc 0.2.62 (registry+https://github.com/rust-lang/crates.io-index)" = "34fcd2c08d2f832f376f4173a231990fa5aef4e99fb569867318a227ef4c06ba" +"checksum libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)" = "1a31a0627fdf1f6a39ec0dd577e101440b7db22672c0901fe00a9a6fbb5c24e8" "checksum libloading 0.5.2 (registry+https://github.com/rust-lang/crates.io-index)" = "f2b111a074963af1d37a139918ac6d49ad1d0d5e47f72fd55388619691a7d753" -"checksum libp2p 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)" = "4183fb4be621d97baebbbe0c499d6ae337e9e6ec955f9fa3cb29e55547dfacdb" -"checksum libp2p-core 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)" = "2a7ebd9d597299512e096cc1bd58e955c03ef28f33214a33b9c7e4ace109ff41" -"checksum libp2p-core-derive 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)" = "baffb3527eac95b717e5ebcd6539007152019a06b00548352cbd74474c07db27" -"checksum libp2p-deflate 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)" = "84bb91afe976893b9822103522cc178bd66eb7aa8e54c69ddd9e1825a3d894ab" -"checksum libp2p-dns 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)" = "b43d79936984b46a5ef4d7b070eaf786f6fab2d1a57e07646306b492e38b2d7f" -"checksum libp2p-floodsub 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)" = "798615b01761454818788dafe61b4fe2bda4306bfa5378cbe8715f57b752235f" -"checksum libp2p-identify 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)" = "a0a630d5ab928403e426672187514884a9ed0ea2065970ef0ec64971770be6d5" -"checksum libp2p-kad 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)" = "a66d2214dd47fa67878eaf0d76d19fd129eff65c45f83617829eb177b7285f97" -"checksum libp2p-mdns 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)" = "cbd443101542670935b6e6863b7bb88c10ac04393062e662201a3c104d80ae00" -"checksum libp2p-mplex 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)" = "59f283e603b078aa88e65c66c5d4f842f67bfbe4d016b0ae345b7e3bb78fe0af" -"checksum libp2p-noise 0.10.0 (registry+https://github.com/rust-lang/crates.io-index)" = "6ab3c7b36cde3bfe18a1d7a0a5693361115066365d32c60f210acc8224b88017" -"checksum libp2p-ping 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)" = "006bbfcb7d6ca7e617cb2924d99fff0e391d4c6e42e7047e226692c8c3e1f6a0" -"checksum libp2p-plaintext 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)" = "a4e673668e5ef47689ca832c33f2dc1e321ede245ee50b6084e4c45cce10fff6" -"checksum libp2p-ratelimit 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)" = "838538f6df5941626047903d14edc3112afb2807fc139535a8ca78469ccaf1ac" -"checksum libp2p-secio 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)" = "a99533cb55b9554d2927ad8a220c87b4e0bbfdec22b738eb6030b03e6a722fa1" -"checksum libp2p-swarm 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)" = "541f66cc794e522fb8072d35dba6be3fe4c3ffeadbed39bf4a6939d0695b4134" -"checksum libp2p-tcp 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)" = "b4e56f7c7e31d303898d51b293f8d95dcb99e6293fefebe184df03e82dd37571" -"checksum libp2p-uds 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)" = "180fa5ceb2f986786b4fca9582f6ffb98772db2e44df07c800693c97205e3310" -"checksum libp2p-wasm-ext 0.5.0 (registry+https://github.com/rust-lang/crates.io-index)" = "a806f0e4985ae2dbac2cbebadb72d586ffe2e1f62a265f5e019e57a3f02aa481" -"checksum libp2p-websocket 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)" = "e0dd3cb203aaa1736a38cdd157709153f90bfaed06b87f4dc3ebb62b5d79a643" -"checksum libp2p-yamux 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)" = "a37bed07c8ee0ceeecdfb90d703aa6b1cec99a69b4157e5f7f2c03acacbfca15" +"checksum libp2p 0.13.0 (registry+https://github.com/rust-lang/crates.io-index)" = "9aa3d728b96c06763b2e919b4c99a334d698303c49489671b5ffe3a4b0fd4c9c" +"checksum libp2p-core 0.13.0 (registry+https://github.com/rust-lang/crates.io-index)" = "07759706a4cb4a90903c67d92cb9575acd8df90f583dfdc46d57afdeaead4c82" +"checksum libp2p-core-derive 0.13.0 (registry+https://github.com/rust-lang/crates.io-index)" = "1eeb2704ac14c60f31967e351ed928b848526a5fc6db4104520020665012826f" +"checksum libp2p-deflate 0.5.0 (registry+https://github.com/rust-lang/crates.io-index)" = "ef2b0bf5d37692ac90e2bffa436bec26c0b0def6c0cab7ea85ff67a353d58aaa" +"checksum libp2p-dns 0.13.0 (registry+https://github.com/rust-lang/crates.io-index)" = "e3175fb0fc9016c95c8517a297bbdb5fb6bfbd5665bacd2eb23495d1cbdeb033" +"checksum libp2p-floodsub 0.13.0 (registry+https://github.com/rust-lang/crates.io-index)" = "92c11b95281e8cb87eb83c204b3ca4988fa665ed9351199b5bcc323056f49816" +"checksum libp2p-identify 0.13.0 (registry+https://github.com/rust-lang/crates.io-index)" = "4eba6103329e9a1a2aa940671efe5600c758a295e61172139d7a900166da0017" +"checksum libp2p-kad 0.13.0 (registry+https://github.com/rust-lang/crates.io-index)" = "84ceb0faa267b96560ef883dc5bc6dddd9de1662e35a4070208623b391deefca" +"checksum libp2p-mdns 0.13.0 (registry+https://github.com/rust-lang/crates.io-index)" = "ab1eec2958fc74883ed8ecb0c38324941a44195a58fea87fcfc2bd17da34d1fa" +"checksum libp2p-mplex 0.13.0 (registry+https://github.com/rust-lang/crates.io-index)" = "e2fe584816d993dc0f893396521a3c93191d78a6f28a892b150baa714a12c3e5" +"checksum libp2p-noise 0.11.0 (registry+https://github.com/rust-lang/crates.io-index)" = "a30ec2640262a7ad6b1a8b28f6cd8281e620a6802f700adf9ff26e61487c333a" +"checksum libp2p-ping 0.13.0 (registry+https://github.com/rust-lang/crates.io-index)" = "e4e1682cdae649394d2793758ded2bfd4d9d440f807e3b4d9f70981f377aa28a" +"checksum libp2p-plaintext 0.13.0 (registry+https://github.com/rust-lang/crates.io-index)" = "7f4fe82189f5c20e8f0a11deaa04d492703c501cefd2428ad68f4f64aefab76f" +"checksum libp2p-secio 0.13.0 (registry+https://github.com/rust-lang/crates.io-index)" = "7ee09e259ceb7633a52fd17f187bedf94e3545b1746487beedbd3a0a07d99817" +"checksum libp2p-swarm 0.3.0 (registry+https://github.com/rust-lang/crates.io-index)" = "cd55bc9f5f9eac2bb1ff24ca3c8a655810a566ac38c7a6ee1f30aced5a62905b" +"checksum libp2p-tcp 0.13.0 (registry+https://github.com/rust-lang/crates.io-index)" = "234a7093d05651ab5630db926a4a42ca8978a65bab8c27c2ce2b66b200c76989" +"checksum libp2p-uds 0.13.0 (registry+https://github.com/rust-lang/crates.io-index)" = "1e2fe0648967da3e56e4a55055c857c8c48326b66be0047d0e04c8ca60d34630" +"checksum libp2p-wasm-ext 0.6.0 (registry+https://github.com/rust-lang/crates.io-index)" = "3f7b8f2bd81fb356e81352d4513856bc21215ecf91502aa1f55b6449642a9acf" +"checksum libp2p-websocket 0.13.0 (registry+https://github.com/rust-lang/crates.io-index)" = "0d74d4fc229ad7e8d1a973178786bdcd5dadbdd7b9822c4477c8687df6f82f66" +"checksum libp2p-yamux 0.13.0 (registry+https://github.com/rust-lang/crates.io-index)" = "1913eb7dd6eb5515957b6f1770296f6921968db87bc9b985f0e974b6657e1003" "checksum librocksdb-sys 5.18.3 (registry+https://github.com/rust-lang/crates.io-index)" = "d19778314deaa7048f2ea7d07b8aa12e1c227acebe975a37eeab6d2f8c74e41b" -"checksum libsecp256k1 0.2.2 (registry+https://github.com/rust-lang/crates.io-index)" = "688e8d65e495567c2c35ea0001b26b9debf0b4ea11f8cccc954233b75fc3428a" -"checksum libsecp256k1 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)" = "63cc09b49bf0cc55885982347b174ad89855e97a12284d2c9dcc6da2e20c28f5" +"checksum libsecp256k1 0.3.2 (registry+https://github.com/rust-lang/crates.io-index)" = "2bd9a7c16c9487e710536b699c962f022266347c94201174aa0a7eb0546051aa" "checksum libz-sys 1.0.25 (registry+https://github.com/rust-lang/crates.io-index)" = "2eb5e43362e38e2bca2fd5f5134c4d4564a23a5c28e9b95411652021a8675ebe" "checksum linked-hash-map 0.5.2 (registry+https://github.com/rust-lang/crates.io-index)" = "ae91b68aebc4ddb91978b11a1b02ddd8602a05ec19002801c5666000e05e0f83" "checksum linked_hash_set 0.1.3 (registry+https://github.com/rust-lang/crates.io-index)" = "3c7c91c4c7bbeb4f2f7c4e5be11e6a05bd6830bc37249c47ce1ad86ad453ff9c" @@ -7075,51 +8199,66 @@ dependencies = [ "checksum log 0.3.9 (registry+https://github.com/rust-lang/crates.io-index)" = "e19e8d5c34a3e0e2223db8e060f9e8264aeeb5c5fc64a4ee9965c062211c024b" "checksum log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)" = "14b6052be84e6b71ab17edffc2eeabf5c2c3ae1fdb464aae35ac50c67a44e1f7" "checksum lru-cache 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)" = "31e24f1ad8321ca0e8a1e0ac13f23cb668e6f5466c2c57319f6a5cf1cc8e3b1c" -"checksum malloc_size_of_derive 0.1.0 (registry+https://github.com/rust-lang/crates.io-index)" = "35adee9ed962cf7d07d62cb58bc45029f3227f5b5b86246caa8632f06c187bc3" +"checksum mach 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)" = "86dd2487cdfea56def77b88438a2c915fb45113c5319bfe7e14306ca4cd0b0e1" +"checksum malloc_size_of_derive 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)" = "e37c5d4cd9473c5f4c9c111f033f15d4df9bd378fdf615944e360a4f55a05f0b" "checksum matches 0.1.8 (registry+https://github.com/rust-lang/crates.io-index)" = "7ffc5c5338469d4d3ea17d269fa8ea3512ad247247c30bd2df69e68309ed0a08" +"checksum maybe-uninit 2.0.0 (registry+https://github.com/rust-lang/crates.io-index)" = "60302e4db3a61da70c0cb7991976248362f30319e88850c487b9b95bbf059e00" "checksum memchr 2.2.1 (registry+https://github.com/rust-lang/crates.io-index)" = "88579771288728879b57485cc7d6b07d648c9f0141eb955f8ab7f9d45394468e" -"checksum memoffset 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)" = "ce6075db033bbbb7ee5a0bbd3a3186bbae616f57fb001c485c7ff77955f8177f" +"checksum memoffset 0.5.3 (registry+https://github.com/rust-lang/crates.io-index)" = "75189eb85871ea5c2e2c15abbdd541185f63b408415e5051f5cac122d8c774b9" "checksum memory-db 0.15.2 (registry+https://github.com/rust-lang/crates.io-index)" = "ef49315991403ba5fa225a70399df5e115f57b274cb0b1b4bcd6e734fa5bd783" "checksum memory_units 0.3.0 (registry+https://github.com/rust-lang/crates.io-index)" = "71d96e3f3c0b6325d8ccd83c33b28acb183edcb6c67938ba104ec546854b0882" "checksum merlin 1.2.1 (registry+https://github.com/rust-lang/crates.io-index)" = "de2d16d3b15fec5943d1144f861f61f279d165fdd60998ca262913b9bf1c8adb" "checksum mime 0.2.6 (registry+https://github.com/rust-lang/crates.io-index)" = "ba626b8a6de5da682e1caa06bdb42a335aee5a84db8e5046a3e8ab17ba0a3ae0" +<<<<<<< HEAD +<<<<<<< HEAD +"checksum mime 0.3.14 (registry+https://github.com/rust-lang/crates.io-index)" = "dd1d63acd1b78403cc0c325605908475dd9b9a3acbf65ed8bcab97e27014afcf" +"checksum mime_guess 2.0.1 (registry+https://github.com/rust-lang/crates.io-index)" = "1a0ed03949aef72dbdf3116a383d7b38b4768e6f960528cd6a6044aa9ed68599" "checksum miniz_oxide 0.3.3 (registry+https://github.com/rust-lang/crates.io-index)" = "304f66c19be2afa56530fa7c39796192eef38618da8d19df725ad7c6d6b2aaae" +======= +======= +"checksum mime 0.3.14 (registry+https://github.com/rust-lang/crates.io-index)" = "dd1d63acd1b78403cc0c325605908475dd9b9a3acbf65ed8bcab97e27014afcf" +"checksum mime_guess 2.0.1 (registry+https://github.com/rust-lang/crates.io-index)" = "1a0ed03949aef72dbdf3116a383d7b38b4768e6f960528cd6a6044aa9ed68599" +>>>>>>> 9e0aa9e5d... base-code(rebased) +"checksum miniz_oxide 0.3.5 (registry+https://github.com/rust-lang/crates.io-index)" = "6f3f74f726ae935c3f514300cc6773a0c9492abc5e972d42ba0c0ebb88757625" +>>>>>>> f76ccb03a... Bump dependencies with `cargo update` (#4087) "checksum mio 0.6.19 (registry+https://github.com/rust-lang/crates.io-index)" = "83f51996a3ed004ef184e16818edc51fadffe8e7ca68be67f9dee67d84d0ff23" "checksum mio-extras 2.0.5 (registry+https://github.com/rust-lang/crates.io-index)" = "46e73a04c2fa6250b8d802134d56d554a9ec2922bf977777c805ea5def61ce40" "checksum mio-uds 0.6.7 (registry+https://github.com/rust-lang/crates.io-index)" = "966257a94e196b11bb43aca423754d87429960a768de9414f3691d6957abf125" "checksum miow 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)" = "8c1f2f3b1cf331de6896aabf6e9d55dca90356cc9960cca7eaaf408a355ae919" "checksum multimap 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)" = "2eb04b9f127583ed176e163fb9ec6f3e793b87e21deedd5734a69386a18a0151" -"checksum multistream-select 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)" = "e8f3cb4c93f2d79811fc11fa01faab99d8b7b8cbe024b602c27434ff2b08a59d" +"checksum multistream-select 0.6.0 (registry+https://github.com/rust-lang/crates.io-index)" = "1242e4ecf2060b35fb58002988e4720fbb3a2cbd4c136d369c420fa028f69efe" "checksum names 0.11.0 (registry+https://github.com/rust-lang/crates.io-index)" = "ef320dab323286b50fb5cdda23f61c796a72a89998ab565ca32525c5c556f2da" "checksum native-tls 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)" = "4b2df1a4c22fd44a62147fd8f13dd0f95c9d8ca7b2610299b2a2f9cf8964274e" "checksum net2 0.2.33 (registry+https://github.com/rust-lang/crates.io-index)" = "42550d9fb7b6684a6d404d9fa7250c2eb2646df731d1c06afc06dcee9e1bcf88" "checksum nix 0.14.1 (registry+https://github.com/rust-lang/crates.io-index)" = "6c722bee1037d430d0f8e687bbdbf222f27cc6e4e68d5caf630857bb2b6dbdce" "checksum nodrop 0.1.14 (registry+https://github.com/rust-lang/crates.io-index)" = "72ef4a56884ca558e5ddb05a1d1e7e1bfd9a68d9ed024c21704cc98872dae1bb" "checksum nohash-hasher 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)" = "4e657a6ec97f9a3ba46f6f7034ea6db9fcd5b71d25ef1074b7bc03da49be0e8e" +"checksum nom 1.2.4 (registry+https://github.com/rust-lang/crates.io-index)" = "a5b8c256fd9471521bcb84c3cdba98921497f1a331cbc15b8030fc63b82050ce" "checksum nom 4.2.3 (registry+https://github.com/rust-lang/crates.io-index)" = "2ad2a91a8e869eeb30b9cb3119ae87773a8f4ae617f41b1eb9c154b2905f7bd6" "checksum num-bigint 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)" = "f9c3f34cdd24f334cb265d9bf8bfa8a241920d026916785747a92f0e55541a1a" "checksum num-integer 0.1.41 (registry+https://github.com/rust-lang/crates.io-index)" = "b85e541ef8255f6cf42bbfe4ef361305c6c135d10919ecc26126c4e5ae94bc09" "checksum num-rational 0.2.2 (registry+https://github.com/rust-lang/crates.io-index)" = "f2885278d5fe2adc2f75ced642d52d879bffaceb5a2e0b1d4309ffdfb239b454" "checksum num-traits 0.2.8 (registry+https://github.com/rust-lang/crates.io-index)" = "6ba9a427cfca2be13aa6f6403b0b7e7368fe982bfa16fccc450ce74c46cd9b32" -"checksum num_cpus 1.10.1 (registry+https://github.com/rust-lang/crates.io-index)" = "bcef43580c035376c0705c42792c294b66974abbfd2789b511784023f71f3273" +"checksum num_cpus 1.11.0 (registry+https://github.com/rust-lang/crates.io-index)" = "155394f924cdddf08149da25bfb932d226b4a593ca7468b08191ff6335941af5" "checksum ole32-sys 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)" = "5d2c49021782e5233cd243168edfa8037574afed4eba4bbaf538b3d8d1789d8c" "checksum once_cell 0.1.8 (registry+https://github.com/rust-lang/crates.io-index)" = "532c29a261168a45ce28948f9537ddd7a5dd272cc513b3017b1e82a88f962c37" "checksum once_cell 0.2.4 (registry+https://github.com/rust-lang/crates.io-index)" = "d584f08c2d717d5c23a6414fc2822b71c651560713e54fa7eace675f758a355e" "checksum opaque-debug 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)" = "2839e79665f131bdb5782e51f2c6c9599c133c6098982a54c794358bf432529c" "checksum openssl 0.10.25 (registry+https://github.com/rust-lang/crates.io-index)" = "2f372b2b53ce10fb823a337aaa674e3a7d072b957c6264d0f4ff0bd86e657449" "checksum openssl-probe 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)" = "77af24da69f9d9341038eba93a073b1fdaaa1b788221b00a69bce9e762cb32de" -"checksum openssl-sys 0.9.51 (registry+https://github.com/rust-lang/crates.io-index)" = "ba24190c8f0805d3bd2ce028f439fe5af1d55882bbe6261bed1dbc93b50dd6b1" +"checksum openssl-sys 0.9.52 (registry+https://github.com/rust-lang/crates.io-index)" = "c977d08e1312e2f7e4b86f9ebaa0ed3b19d1daff75fae88bbb88108afbd801fc" "checksum output_vt100 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)" = "53cdc5b785b7a58c5aad8216b3dfa114df64b0b06ae6e1501cef91df2fbdf8f9" "checksum owning_ref 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)" = "49a4b8ea2179e6a2e27411d3bca09ca6dd630821cf6894c6c7c8467a8ee7ef13" "checksum parity-bytes 0.1.0 (git+https://github.com/paritytech/parity-common?rev=b0317f649ab2c665b7987b8475878fc4d2e1f81d)" = "" -"checksum parity-multiaddr 0.5.0 (registry+https://github.com/rust-lang/crates.io-index)" = "045b3c7af871285146300da35b1932bb6e4639b66c7c98e85d06a32cbc4e8fa7" -"checksum parity-multihash 0.1.3 (registry+https://github.com/rust-lang/crates.io-index)" = "df3a17dc27848fd99e4f87eb0f8c9baba6ede0a6d555400c850ca45254ef4ce3" +"checksum parity-multiaddr 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)" = "7dbc379f41150dedda75cbbdb5b9beb2bf786a07e56c2c99ec89aeaaa894662c" +"checksum parity-multihash 0.1.4 (registry+https://github.com/rust-lang/crates.io-index)" = "340ed03f939e02e4cb71a5a127b5507ba4dab506e41a05f8f467e28d8ce529f4" "checksum parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)" = "001fbbb956d8593f321c7a784f64d16b2c99b2657823976eea729006ad2c3668" "checksum parity-scale-codec-derive 1.0.3 (registry+https://github.com/rust-lang/crates.io-index)" = "42af752f59119656fa3cb31e8852ed24e895b968c0bdb41847da7f0cea6d155f" "checksum parity-send-wrapper 0.1.0 (registry+https://github.com/rust-lang/crates.io-index)" = "aa9777aa91b8ad9dd5aaa04a9b6bcb02c7f1deb952fca5a66034d5e63afc5c6f" -"checksum parity-util-mem 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)" = "e2005637ccf93dbb60c85081ccaaf3f945f573da48dcc79f27f9646caa3ec1dc" +"checksum parity-util-mem 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)" = "570093f39f786beea92dcc09e45d8aae7841516ac19a50431953ac82a0e8f85c" "checksum parity-wasm 0.32.0 (registry+https://github.com/rust-lang/crates.io-index)" = "16ad52817c4d343339b3bc2e26861bd21478eda0b7509acf83505727000512ac" "checksum parity-wasm 0.40.3 (registry+https://github.com/rust-lang/crates.io-index)" = "1e39faaa292a687ea15120b1ac31899b13586446521df6c149e46f1584671e0f" +"checksum parity-wasm 0.41.0 (registry+https://github.com/rust-lang/crates.io-index)" = "ddfc878dac00da22f8f61e7af3157988424567ab01d9920b962ef7dcbd7cd865" "checksum parking_lot 0.6.4 (registry+https://github.com/rust-lang/crates.io-index)" = "f0802bff09003b291ba756dc7e79313e51cc31667e94afbe847def490424cde5" "checksum parking_lot 0.7.1 (registry+https://github.com/rust-lang/crates.io-index)" = "ab41b4aed082705d1056416ae4468b6ea99d52599ecf3169b00088d43113e337" "checksum parking_lot 0.8.0 (registry+https://github.com/rust-lang/crates.io-index)" = "fa7767817701cce701d5585b9c4db3cdd02086398322c1d7e8bf5094a96a2ce7" @@ -7137,24 +8276,32 @@ dependencies = [ "checksum percent-encoding 2.1.0 (registry+https://github.com/rust-lang/crates.io-index)" = "d4fd5641d01c8f18a23da7b6fe29298ff4b55afcccdf78973b24cf3175fee32e" "checksum petgraph 0.4.13 (registry+https://github.com/rust-lang/crates.io-index)" = "9c3659d1ee90221741f65dd128d9998311b0e40c5d3c23a62445938214abce4f" "checksum pin-utils 0.1.0-alpha.4 (registry+https://github.com/rust-lang/crates.io-index)" = "5894c618ce612a3fa23881b152b608bafb8c56cfc22f434a3ba3120b40f7b587" -"checksum pkg-config 0.3.16 (registry+https://github.com/rust-lang/crates.io-index)" = "72d5370d90f49f70bd033c3d75e87fc529fbfff9d6f7cccef07d6170079d91ea" -"checksum ppv-lite86 0.2.5 (registry+https://github.com/rust-lang/crates.io-index)" = "e3cbf9f658cdb5000fcf6f362b8ea2ba154b9f146a61c7a20d647034c6b6561b" +"checksum pkg-config 0.3.17 (registry+https://github.com/rust-lang/crates.io-index)" = "05da548ad6865900e60eaba7f589cc0783590a92e940c26953ff81ddbab2d677" +"checksum plain 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)" = "b4596b6d070b27117e987119b4dac604f3c58cfb0b191112e24771b2faeac1a6" +"checksum ppv-lite86 0.2.6 (registry+https://github.com/rust-lang/crates.io-index)" = "74490b50b9fbe561ac330df47c08f3f33073d2d00c150f719147d7c54522fa1b" "checksum pretty_assertions 0.6.1 (registry+https://github.com/rust-lang/crates.io-index)" = "3f81e1644e1b54f5a68959a29aa86cde704219254669da328ecfdf6a1f09d427" -"checksum primitive-types 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)" = "83ef7b3b965c0eadcb6838f34f827e1dfb2939bdd5ebd43f9647e009b12b0371" -"checksum primitive-types 0.6.0 (registry+https://github.com/rust-lang/crates.io-index)" = "97b5a08dda18910f056e5c2060c034e77cab18e0bd7d895e44f03207af4c71d5" +"checksum primitive-types 0.6.1 (registry+https://github.com/rust-lang/crates.io-index)" = "a0253db64c26d8b4e7896dd2063b516d2a1b9e0a5da26b5b78335f236d1e9522" "checksum proc-macro-crate 0.1.4 (registry+https://github.com/rust-lang/crates.io-index)" = "e10d4b51f154c8a7fb96fd6dad097cb74b863943ec010ac94b9fd1be8861fe1e" "checksum proc-macro-error 0.2.6 (registry+https://github.com/rust-lang/crates.io-index)" = "aeccfe4d5d8ea175d5f0e4a2ad0637e0f4121d63bd99d356fb1f39ab2e7c6097" -"checksum proc-macro-hack 0.5.10 (registry+https://github.com/rust-lang/crates.io-index)" = "114cdf1f426eb7f550f01af5f53a33c0946156f6814aec939b3bd77e844f9a9d" +"checksum proc-macro-hack 0.5.11 (registry+https://github.com/rust-lang/crates.io-index)" = "ecd45702f76d6d3c75a80564378ae228a85f0b59d2f3ed43c91b4a69eb2ebfc5" +"checksum proc-macro-nested 0.1.3 (registry+https://github.com/rust-lang/crates.io-index)" = "369a6ed065f249a159e06c45752c780bda2fb53c995718f9e484d08daa9eb42e" "checksum proc-macro2 0.4.30 (registry+https://github.com/rust-lang/crates.io-index)" = "cf3d2011ab5c909338f7887f4fc896d35932e29146c12c8d01da6b22a80ba759" "checksum proc-macro2 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)" = "9c9e470a8dc4aeae2dee2f335e8f533e2d4b347e1434e5671afc49b054592f27" +"checksum procinfo 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)" = "f42e8578852a3306838981aedad8c5642ba794929aa12af0c9eb6c072b77af6c" +"checksum prometheus 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)" = "5567486d5778e2c6455b1b90ff1c558f29e751fc018130fa182e15828e728af1" "checksum prost 0.5.0 (registry+https://github.com/rust-lang/crates.io-index)" = "96d14b1c185652833d24aaad41c5832b0be5616a590227c1fbff57c616754b23" "checksum prost-build 0.5.0 (registry+https://github.com/rust-lang/crates.io-index)" = "eb788126ea840817128183f8f603dce02cb7aea25c2a0b764359d8e20010702e" "checksum prost-derive 0.5.0 (registry+https://github.com/rust-lang/crates.io-index)" = "5e7dc378b94ac374644181a2247cebf59a6ec1c88b49ac77f3a94b86b79d0e11" "checksum prost-types 0.5.0 (registry+https://github.com/rust-lang/crates.io-index)" = "1de482a366941c8d56d19b650fac09ca08508f2a696119ee7513ad590c8bac6f" "checksum protobuf 2.8.1 (registry+https://github.com/rust-lang/crates.io-index)" = "40361836defdd5871ff7e84096c6f6444af7fc157f8ef1789f54f147687caa20" +<<<<<<< HEAD +"checksum publicsuffix 1.5.3 (registry+https://github.com/rust-lang/crates.io-index)" = "9bf259a81de2b2eb9850ec990ec78e6a25319715584fd7652b9b26f96fcb1510" +======= +"checksum publicsuffix 1.5.4 (registry+https://github.com/rust-lang/crates.io-index)" = "3bbaa49075179162b49acac1c6aa45fb4dafb5f13cf6794276d77bc7fd95757b" +>>>>>>> 9e0aa9e5d... base-code(rebased) "checksum pwasm-utils 0.11.0 (registry+https://github.com/rust-lang/crates.io-index)" = "1d473123ba135028544926f7aa6f34058d8bc6f120c4fcd3777f84af724280b3" -"checksum quick-error 0.1.4 (registry+https://github.com/rust-lang/crates.io-index)" = "5fb6ccf8db7bbcb9c2eae558db5ab4f3da1c2a87e4e597ed394726bc8ea6ca1d" "checksum quick-error 1.2.2 (registry+https://github.com/rust-lang/crates.io-index)" = "9274b940887ce9addde99c4eee6b5c44cc494b182b97e73dc8ffdcb3397fd3f0" +"checksum quickcheck 0.8.5 (registry+https://github.com/rust-lang/crates.io-index)" = "9c35d9c36a562f37eca96e79f66d5fd56eefbc22560dacc4a864cabd2d277456" "checksum quickcheck 0.9.0 (registry+https://github.com/rust-lang/crates.io-index)" = "d5ca504a2fdaa08d3517f442fbbba91ac24d1ec4c51ea68688a038765e3b2662" "checksum quote 0.6.13 (registry+https://github.com/rust-lang/crates.io-index)" = "6ce23b6b870e8f94f81fb0a363d65d86675884b34a09043c81e5562f11c1f8e1" "checksum quote 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)" = "053a8c8bcc71fcce321828dc897a98ab9760bef03a4fc36693c231e5b3216cfe" @@ -7178,46 +8325,72 @@ dependencies = [ "checksum rand_xorshift 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)" = "cbf7e9e623549b0e21f6e97cf8ecf247c1a8fd2e8a992ae265314300b2455d5c" "checksum rand_xoshiro 0.1.0 (registry+https://github.com/rust-lang/crates.io-index)" = "03b418169fb9c46533f326efd6eed2576699c44ca92d3052a066214a8d828929" "checksum rand_xoshiro 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)" = "0e18c91676f670f6f0312764c759405f13afb98d5d73819840cf72a518487bff" +"checksum raw-cpuid 6.1.0 (registry+https://github.com/rust-lang/crates.io-index)" = "30a9d219c32c9132f7be513c18be77c9881c7107d2ab5569d205a6a0f0e6dc7d" "checksum rayon 1.2.0 (registry+https://github.com/rust-lang/crates.io-index)" = "83a27732a533a1be0a0035a111fe76db89ad312f6f0347004c220c57f209a123" "checksum rayon-core 1.6.0 (registry+https://github.com/rust-lang/crates.io-index)" = "98dcf634205083b17d0861252431eb2acbfb698ab7478a2d20de07954f47ec7b" "checksum rdrand 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)" = "678054eb77286b51581ba43620cc911abf02758c91f93f479767aed0f90458b2" "checksum redox_syscall 0.1.56 (registry+https://github.com/rust-lang/crates.io-index)" = "2439c63f3f6139d1b57529d16bc3b8bb855230c8efcc5d3a896c8bea7c3b1e84" +"checksum redox_users 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)" = "4ecedbca3bf205f8d8f5c2b44d83cd0690e39ee84b951ed649e9f1841132b66d" "checksum ref_thread_local 0.0.0 (registry+https://github.com/rust-lang/crates.io-index)" = "d813022b2e00774a48eaf43caaa3c20b45f040ba8cbf398e2e8911a06668dbe6" "checksum regex 1.3.1 (registry+https://github.com/rust-lang/crates.io-index)" = "dc220bd33bdce8f093101afe22a037b8eb0e5af33592e6a9caafff0d4cb81cbd" "checksum regex-automata 0.1.8 (registry+https://github.com/rust-lang/crates.io-index)" = "92b73c2a1770c255c240eaa4ee600df1704a38dc3feaa6e949e7fcd4f8dc09f9" "checksum regex-syntax 0.6.12 (registry+https://github.com/rust-lang/crates.io-index)" = "11a7e20d1cce64ef2fed88b66d347f88bd9babb82845b2b858f3edbf59a4f716" +"checksum region 2.1.2 (registry+https://github.com/rust-lang/crates.io-index)" = "448e868c6e4cfddfa49b6a72c95906c04e8547465e9536575b95c70a4044f856" "checksum remove_dir_all 0.5.2 (registry+https://github.com/rust-lang/crates.io-index)" = "4a83fa3702a688b9359eccba92d153ac33fd2e8462f9e0e3fdf155239ea7792e" +<<<<<<< HEAD +<<<<<<< HEAD +"checksum reqwest 0.9.22 (registry+https://github.com/rust-lang/crates.io-index)" = "2c2064233e442ce85c77231ebd67d9eca395207dec2127fe0bbedde4bd29a650" "checksum rhododendron 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)" = "36542aafc2429a4c010fafa079a20dee953b663cb2427f51d86cf1d436846b4d" -"checksum ring 0.14.6 (registry+https://github.com/rust-lang/crates.io-index)" = "426bc186e3e95cac1e4a4be125a4aca7e84c2d616ffc02244eef36e2a60a093c" +======= +>>>>>>> f44873dce... Reorganising the repository - external renames and moves (#4074) +======= +"checksum reqwest 0.9.22 (registry+https://github.com/rust-lang/crates.io-index)" = "2c2064233e442ce85c77231ebd67d9eca395207dec2127fe0bbedde4bd29a650" +>>>>>>> 9e0aa9e5d... base-code(rebased) +"checksum ring 0.16.9 (registry+https://github.com/rust-lang/crates.io-index)" = "6747f8da1f2b1fabbee1aaa4eb8a11abf9adef0bf58a41cee45db5d59cecdfac" +"checksum rlp 0.4.3 (registry+https://github.com/rust-lang/crates.io-index)" = "8376a3f725ebb53f69263bbebb42196361fdfd551212409c8a721239aab4f09f" "checksum rocksdb 0.11.0 (registry+https://github.com/rust-lang/crates.io-index)" = "f1651697fefd273bfb4fd69466cc2a9d20de557a0213b97233b22b5e95924b5e" "checksum rpassword 4.0.1 (registry+https://github.com/rust-lang/crates.io-index)" = "f072d931f11a96546efd97642e1e75e807345aced86b947f9239102f262d0fcd" +"checksum rust-argon2 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)" = "4ca4eaef519b494d1f2848fc602d18816fed808a981aedf4f1f00ceb7c9d32cf" "checksum rustc-demangle 0.1.16 (registry+https://github.com/rust-lang/crates.io-index)" = "4c691c0e608126e00913e33f0ccf3727d5fc84573623b8d65b2df340b5201783" "checksum rustc-hex 2.0.1 (registry+https://github.com/rust-lang/crates.io-index)" = "403bb3a286107a04825a5f82e1270acc1e14028d3d554d7a1e08914549575ab8" +"checksum rustc_version 0.1.7 (registry+https://github.com/rust-lang/crates.io-index)" = "c5f5376ea5e30ce23c03eb77cbe4962b988deead10910c372b226388b594c084" "checksum rustc_version 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)" = "138e3e0acb6c9fb258b19b67cb8abd63c00679d2851805ea151465464fe9030a" -"checksum rustls 0.15.2 (registry+https://github.com/rust-lang/crates.io-index)" = "f271e3552cd835fa28c541c34a7e8fdd8cdff09d77fe4eb8f6c42e87a11b096e" +"checksum rustls 0.16.0 (registry+https://github.com/rust-lang/crates.io-index)" = "b25a18b1bf7387f0145e7f8324e700805aade3842dd3db2e74e4cdeb4677c09e" "checksum rustversion 1.0.0 (registry+https://github.com/rust-lang/crates.io-index)" = "c48f91977f4ef3be5358c15d131d3f663f6b4d7a112555bf3bf52ad23b6659e5" "checksum rw-stream-sink 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)" = "2f9cbe61c20455d3015b2bb7be39e1872310283b8e5a52f5b242b0ac7581fe78" -"checksum ryu 1.0.1 (registry+https://github.com/rust-lang/crates.io-index)" = "19d2271fa48eaf61e53cc88b4ad9adcbafa2d512c531e7fadb6dc11a4d3656c5" +"checksum ryu 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)" = "bfa8506c1de11c9c4e4c38863ccbe02a305c8188e85a05a784c9e11e1c3910c8" "checksum safe-mix 1.0.0 (registry+https://github.com/rust-lang/crates.io-index)" = "7f7bf422d23a88c16d5090d455f182bc99c60af4df6a345c63428acf5129e347" -"checksum safemem 0.3.2 (registry+https://github.com/rust-lang/crates.io-index)" = "d2b08423011dae9a5ca23f07cf57dac3857f5c885d352b76f6d95f4aea9434d0" +"checksum safemem 0.3.3 (registry+https://github.com/rust-lang/crates.io-index)" = "ef703b7cb59335eae2eb93ceb664c0eb7ea6bf567079d843e09420219668e072" "checksum same-file 1.0.5 (registry+https://github.com/rust-lang/crates.io-index)" = "585e8ddcedc187886a30fa705c47985c3fa88d06624095856b36ca0b82ff4421" "checksum schannel 0.1.16 (registry+https://github.com/rust-lang/crates.io-index)" = "87f550b06b6cba9c8b8be3ee73f391990116bf527450d2556e9b9ce263b9a021" "checksum schnorrkel 0.8.5 (registry+https://github.com/rust-lang/crates.io-index)" = "eacd8381b3c37840c9c9f40472af529e49975bdcbc24f83c31059fd6539023d3" "checksum scopeguard 0.3.3 (registry+https://github.com/rust-lang/crates.io-index)" = "94258f53601af11e6a49f722422f6e3425c52b06245a5cf9bc09908b174f5e27" "checksum scopeguard 1.0.0 (registry+https://github.com/rust-lang/crates.io-index)" = "b42e15e59b18a828bbf5c58ea01debb36b9b096346de35d941dcb89009f24a0d" -"checksum sct 0.5.0 (registry+https://github.com/rust-lang/crates.io-index)" = "2f5adf8fbd58e1b1b52699dc8bed2630faecb6d8c7bee77d009d6bbe4af569b9" +"checksum scroll 0.9.2 (registry+https://github.com/rust-lang/crates.io-index)" = "2f84d114ef17fd144153d608fba7c446b0145d038985e7a8cc5d08bb0ce20383" +"checksum scroll_derive 0.9.5 (registry+https://github.com/rust-lang/crates.io-index)" = "8f1aa96c45e7f5a91cb7fabe7b279f02fea7126239fc40b732316e8b6a2d0fcb" +"checksum sct 0.6.0 (registry+https://github.com/rust-lang/crates.io-index)" = "e3042af939fca8c3453b7af0f1c66e533a15a86169e39de2657310ade8f98d3c" +<<<<<<< HEAD "checksum security-framework 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)" = "eee63d0f4a9ec776eeb30e220f0bc1e092c3ad744b2a379e3993070364d3adc2" "checksum security-framework-sys 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)" = "9636f8989cbf61385ae4824b98c1aaa54c994d7d8b41f11c601ed799f0549a56" +"checksum semver 0.1.20 (registry+https://github.com/rust-lang/crates.io-index)" = "d4f410fedcf71af0345d7607d246e7ad15faaadd49d240ee3b24e5dc21a820ac" +======= +"checksum security-framework 0.3.3 (registry+https://github.com/rust-lang/crates.io-index)" = "301c862a6d0ee78f124c5e1710205965fc5c553100dcda6d98f13ef87a763f04" +"checksum security-framework-sys 0.3.3 (registry+https://github.com/rust-lang/crates.io-index)" = "e31493fc37615debb8c5090a7aeb4a9730bc61e77ab10b9af59f1a202284f895" +<<<<<<< HEAD +>>>>>>> f76ccb03a... Bump dependencies with `cargo update` (#4087) +======= +"checksum semver 0.1.20 (registry+https://github.com/rust-lang/crates.io-index)" = "d4f410fedcf71af0345d7607d246e7ad15faaadd49d240ee3b24e5dc21a820ac" +>>>>>>> 9e0aa9e5d... base-code(rebased) "checksum semver 0.6.0 (registry+https://github.com/rust-lang/crates.io-index)" = "7a3186ec9e65071a2095434b1f5bb24838d4e8e130f584c790f6033c79943537" "checksum semver 0.9.0 (registry+https://github.com/rust-lang/crates.io-index)" = "1d7eb9ef2c18661902cc47e535f9bc51b78acd254da71d375c2f6720d9a40403" "checksum semver-parser 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)" = "388a1df253eca08550bef6c72392cfe7c30914bf41df5269b68cbd6ff8f570a3" "checksum send_wrapper 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)" = "a0eddf2e8f50ced781f288c19f18621fa72a3779e3cb58dbf23b07469b0abeb4" -"checksum serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)" = "9796c9b7ba2ffe7a9ce53c2287dfc48080f4b2b362fcc245a259b3a7201119dd" -"checksum serde_derive 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)" = "4b133a43a1ecd55d4086bd5b4dc6c1751c68b1bfbeba7a5040442022c7e7c02e" +"checksum serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)" = "0c4b39bd9b0b087684013a792c59e3e07a46a01d2322518d8a1104641a0b1be0" +"checksum serde_derive 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)" = "ca13fc1a832f793322228923fbb3aba9f3f44444898f835d31ad1b74fa0a2bf8" "checksum serde_json 1.0.41 (registry+https://github.com/rust-lang/crates.io-index)" = "2f72eb2a68a7dc3f9a691bfda9305a1c017a6215e5a4545c258500d2099a37c2" +"checksum serde_urlencoded 0.5.5 (registry+https://github.com/rust-lang/crates.io-index)" = "642dd69105886af2efd227f75a520ec9b44a820d65bc133a9131f7d229fd165a" "checksum sha-1 0.8.1 (registry+https://github.com/rust-lang/crates.io-index)" = "23962131a91661d643c98940b20fcaffe62d776a823247be80a48fcb8b6fce68" "checksum sha1 0.6.0 (registry+https://github.com/rust-lang/crates.io-index)" = "2579985fda508104f7587689507983eadd6a6e84dd35d6d115361f530916fa0d" -"checksum sha2 0.6.0 (registry+https://github.com/rust-lang/crates.io-index)" = "7d963c78ce367df26d7ea8b8cc655c651b42e8a1e584e869c1e17dae3ccb116a" "checksum sha2 0.8.0 (registry+https://github.com/rust-lang/crates.io-index)" = "7b4d8bfd0e469f417657573d8451fb33d16cfe0989359b93baf3a1ffc639543d" "checksum sha3 0.8.2 (registry+https://github.com/rust-lang/crates.io-index)" = "dd26bc0e7a2e3a7c959bc494caf58b72ee0c71d67704e9520f736ca7e4853ecf" "checksum shell32-sys 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)" = "9ee04b46101f57121c9da2b151988283b6beb79b34f5bb29a58ee48cb695122c" @@ -7225,43 +8398,44 @@ dependencies = [ "checksum slog 2.5.2 (registry+https://github.com/rust-lang/crates.io-index)" = "1cc9c640a4adbfbcc11ffb95efe5aa7af7309e002adab54b185507dbf2377b99" "checksum slog-async 2.3.0 (git+https://github.com/paritytech/slog-async)" = "" "checksum slog-json 2.3.0 (registry+https://github.com/rust-lang/crates.io-index)" = "ddc0d2aff1f8f325ef660d9a0eb6e6dcd20b30b3f581a5897f58bf42d061c37a" -"checksum slog-scope 4.1.2 (registry+https://github.com/rust-lang/crates.io-index)" = "d1d3ec6214d46e57a7ec87c1972bbca66c59172a0cfffa5233c54726afb946bf" +"checksum slog-scope 4.3.0 (registry+https://github.com/rust-lang/crates.io-index)" = "7c44c89dd8b0ae4537d1ae318353eaf7840b4869c536e31c41e963d1ea523ee6" "checksum slog_derive 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)" = "9eff3b513cf2e0d1a60e1aba152dc72bedc5b05585722bb3cebd7bcb1e31b98f" -"checksum smallvec 0.6.10 (registry+https://github.com/rust-lang/crates.io-index)" = "ab606a9c5e214920bb66c458cd7be8ef094f813f20fe77a54cc7dbfff220d4b7" -"checksum snow 0.5.2 (registry+https://github.com/rust-lang/crates.io-index)" = "5a64f02fd208ef15bd2d1a65861df4707e416151e1272d02c8faafad1c138100" +"checksum smallvec 0.6.13 (registry+https://github.com/rust-lang/crates.io-index)" = "f7b0758c52e15a8b5e3691eae6cc559f08eee9406e548a4477ba4e67770a82b6" +"checksum snow 0.6.1 (registry+https://github.com/rust-lang/crates.io-index)" = "91eecae35b461ed26bda7a76bea2cc5bda2bf4b8dd06761879f19e6fdd50c2dd" "checksum soketto 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)" = "bceb1a3a15232d013d9a3b7cac9e5ce8e2313f348f01d4bc1097e5e53aa07095" "checksum sourcefile 0.1.4 (registry+https://github.com/rust-lang/crates.io-index)" = "4bf77cb82ba8453b42b6ae1d692e4cdc92f9a47beaf89a847c8be83f4e328ad3" "checksum spin 0.5.2 (registry+https://github.com/rust-lang/crates.io-index)" = "6e63cff320ae2c57904679ba7cb63280a3dc4613885beafb148ee7bf9aa9042d" "checksum stable_deref_trait 1.1.1 (registry+https://github.com/rust-lang/crates.io-index)" = "dba1a27d3efae4351c8051072d619e3ade2820635c3958d826bfea39d59b54c8" -"checksum static_assertions 0.2.5 (registry+https://github.com/rust-lang/crates.io-index)" = "c19be23126415861cb3a23e501d34a708f7f9b2183c5252d690941c2e69199d5" -"checksum static_slice 0.0.3 (registry+https://github.com/rust-lang/crates.io-index)" = "92a7e0c5e3dfb52e8fbe0e63a1b947bbb17b4036408b151353c4491374931362" +"checksum static_assertions 1.1.0 (registry+https://github.com/rust-lang/crates.io-index)" = "a2eb9349b6444b326872e140eb1cf5e7c522154d69e7a0ffb0fb81c06b37543f" "checksum stream-cipher 0.3.2 (registry+https://github.com/rust-lang/crates.io-index)" = "8131256a5896cabcf5eb04f4d6dacbe1aefda854b0d9896e09cb58829ec5638c" "checksum string 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)" = "d24114bfcceb867ca7f71a0d3fe45d45619ec47a6fbfa98cb14e14250bfa5d6d" +"checksum string-interner 0.7.1 (registry+https://github.com/rust-lang/crates.io-index)" = "fd710eadff449a1531351b0e43eb81ea404336fa2f56c777427ab0e32a4cf183" "checksum strsim 0.8.0 (registry+https://github.com/rust-lang/crates.io-index)" = "8ea5119cdb4c55b55d432abb513a0429384878c15dde60cc77b1c99de1a95a6a" -"checksum structopt 0.3.3 (registry+https://github.com/rust-lang/crates.io-index)" = "6d4f66a4c0ddf7aee4677995697366de0749b0139057342eccbb609b12d0affc" -"checksum structopt-derive 0.3.3 (registry+https://github.com/rust-lang/crates.io-index)" = "8fe0c13e476b4e21ff7f5c4ace3818b6d7bdc16897c31c73862471bc1663acae" +"checksum structopt 0.3.4 (registry+https://github.com/rust-lang/crates.io-index)" = "c167b61c7d4c126927f5346a4327ce20abf8a186b8041bbeb1ce49e5db49587b" +"checksum structopt-derive 0.3.4 (registry+https://github.com/rust-lang/crates.io-index)" = "519621841414165d2ad0d4c92be8f41844203f2b67e245f9345a5a12d40c69d7" "checksum strum 0.15.0 (registry+https://github.com/rust-lang/crates.io-index)" = "e5d1c33039533f051704951680f1adfd468fd37ac46816ded0d9ee068e60f05f" "checksum strum_macros 0.15.0 (registry+https://github.com/rust-lang/crates.io-index)" = "47cd23f5c7dee395a00fa20135e2ec0fffcdfa151c56182966d7a3261343432e" "checksum substrate-bip39 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)" = "3be511be555a3633e71739a79e4ddff6a6aaa6579fa6114182a51d72c3eb93c5" -"checksum substrate-wasm-builder-runner 1.0.4 (registry+https://github.com/rust-lang/crates.io-index)" = "bd48273fe9d7f92c1f7d6c1c537bb01c8068f925b47ad2cd8367e11dc32f8550" "checksum subtle 1.0.0 (registry+https://github.com/rust-lang/crates.io-index)" = "2d67a5a62ba6e01cb2192ff309324cb4875d0c451d55fe2319433abe7a05a8ee" "checksum subtle 2.2.1 (registry+https://github.com/rust-lang/crates.io-index)" = "ab3af2eb31c42e8f0ccf43548232556c42737e01a96db6e1777b0be108e79799" "checksum syn 0.15.44 (registry+https://github.com/rust-lang/crates.io-index)" = "9ca4b3b69a77cbe1ffc9e198781b7acb0c7365a883670e8f1c1bc66fba79a5c5" -"checksum syn 1.0.7 (registry+https://github.com/rust-lang/crates.io-index)" = "0e7bedb3320d0f3035594b0b723c8a28d7d336a3eda3881db79e61d676fb644c" -"checksum synstructure 0.10.2 (registry+https://github.com/rust-lang/crates.io-index)" = "02353edf96d6e4dc81aea2d8490a7e9db177bf8acb0e951c24940bf866cb313f" -"checksum synstructure 0.12.1 (registry+https://github.com/rust-lang/crates.io-index)" = "3f085a5855930c0441ca1288cf044ea4aecf4f43a91668abdb870b4ba546a203" -"checksum sysinfo 0.9.5 (registry+https://github.com/rust-lang/crates.io-index)" = "d5bd3b813d94552a8033c650691645f8dd5a63d614dddd62428a95d3931ef7b6" +"checksum syn 1.0.8 (registry+https://github.com/rust-lang/crates.io-index)" = "661641ea2aa15845cddeb97dad000d22070bb5c1fb456b96c1cba883ec691e92" +"checksum synstructure 0.12.2 (registry+https://github.com/rust-lang/crates.io-index)" = "575be94ccb86e8da37efb894a87e2b660be299b41d8ef347f9d6d79fbe61b1ba" +"checksum sysinfo 0.9.6 (registry+https://github.com/rust-lang/crates.io-index)" = "6f4b2468c629cffba39c0a4425849ab3cdb03d9dfacba69684609aea04d08ff9" "checksum take_mut 0.2.2 (registry+https://github.com/rust-lang/crates.io-index)" = "f764005d11ee5f36500a149ace24e00e3da98b0158b3e2d53a7495660d3f4d60" +"checksum target-lexicon 0.8.1 (registry+https://github.com/rust-lang/crates.io-index)" = "7975cb2c6f37d77b190bc5004a2bb015971464756fde9514651a525ada2a741a" "checksum target_info 0.1.0 (registry+https://github.com/rust-lang/crates.io-index)" = "c63f48baada5c52e65a29eef93ab4f8982681b67f9e8d29c7b05abcfec2b9ffe" "checksum tempdir 0.3.7 (registry+https://github.com/rust-lang/crates.io-index)" = "15f2b5fb00ccdf689e0149d1b1b3c03fead81c2b37735d812fa8bddbbf41b6d8" "checksum tempfile 3.1.0 (registry+https://github.com/rust-lang/crates.io-index)" = "7a6e24d9338a0a5be79593e2fa15a648add6138caa803e2d5bc782c371732ca9" "checksum termcolor 1.0.5 (registry+https://github.com/rust-lang/crates.io-index)" = "96d6098003bde162e4277c70665bd87c326f5a0c3f3fbfb285787fa482d54e6e" +"checksum test-case 0.3.3 (registry+https://github.com/rust-lang/crates.io-index)" = "a605baa797821796a751f4a959e1206079b24a4b7e1ed302b7d785d81a9276c9" "checksum textwrap 0.11.0 (registry+https://github.com/rust-lang/crates.io-index)" = "d326610f408c7a4eb6f51c37c330e496b08506c9457c9d34287ecc38809fb060" "checksum thread_local 0.3.6 (registry+https://github.com/rust-lang/crates.io-index)" = "c6b53e329000edc2b34dbe8545fd20e55a333362d0a321909685a19bd28c3f1b" "checksum threadpool 1.7.1 (registry+https://github.com/rust-lang/crates.io-index)" = "e2f0c90a5f3459330ac8bc0d2f879c693bb7a2f59689c1083fc4ef83834da865" "checksum time 0.1.42 (registry+https://github.com/rust-lang/crates.io-index)" = "db8dcfca086c1143c9270ac42a2bbd8a7ee477b78ac8e45b19abfb0cbede4b6f" "checksum tiny-bip39 0.6.2 (registry+https://github.com/rust-lang/crates.io-index)" = "c1c5676413eaeb1ea35300a0224416f57abc3bd251657e0fafc12c47ff98c060" "checksum tiny-keccak 1.5.0 (registry+https://github.com/rust-lang/crates.io-index)" = "1d8a021c69bb74a44ccedb824a046447e2c84a01df9e5c20779750acb38e11b2" +"checksum tiny-keccak 2.0.1 (registry+https://github.com/rust-lang/crates.io-index)" = "2953ca5148619bc99695c1274cb54c5275bbb913c6adad87e72eaf8db9787f69" "checksum tinytemplate 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)" = "4574b75faccaacddb9b284faecdf0b544b80b6b294f3d062d325c5726a209c20" "checksum tokio 0.1.22 (registry+https://github.com/rust-lang/crates.io-index)" = "5a09c0b5bb588872ab2f09afa13ee6e9dac11e10a0ec9e8e3ba39a5a5d530af6" "checksum tokio-buf 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)" = "8fb220f46c53859a4b7ec083e41dec9778ff0b1851c0942b211edb89e0ccdc46" @@ -7272,7 +8446,7 @@ dependencies = [ "checksum tokio-fs 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)" = "3fe6dc22b08d6993916647d108a1a7d15b9cd29c4f4496c62b92c45b5041b7af" "checksum tokio-io 0.1.12 (registry+https://github.com/rust-lang/crates.io-index)" = "5090db468dad16e1a7a54c8c67280c5e4b544f3d3e018f0b913b400261f85926" "checksum tokio-reactor 0.1.10 (registry+https://github.com/rust-lang/crates.io-index)" = "c56391be9805bc80163151c0b9e5164ee64f4b0200962c346fea12773158f22d" -"checksum tokio-rustls 0.10.0-alpha.4 (registry+https://github.com/rust-lang/crates.io-index)" = "3e5cebc3ca33110e460c4d2e7c5e863b159fadcbf125449d896720695b2af709" +"checksum tokio-rustls 0.10.2 (registry+https://github.com/rust-lang/crates.io-index)" = "1df2fa53ac211c136832f530ccb081af9af891af22d685a9493e232c7a359bc2" "checksum tokio-sync 0.1.7 (registry+https://github.com/rust-lang/crates.io-index)" = "d06554cce1ae4a50f42fba8023918afa931413aded705b560e29600ccf7c6d76" "checksum tokio-tcp 0.1.3 (registry+https://github.com/rust-lang/crates.io-index)" = "1d14b10654be682ac43efee27401d792507e30fd8d26389e1da3b185de2e4119" "checksum tokio-threadpool 0.1.16 (registry+https://github.com/rust-lang/crates.io-index)" = "2bd2c6a3885302581f4401c82af70d792bb9df1700e7437b0aeb4ada94d5388c" @@ -7280,55 +8454,65 @@ dependencies = [ "checksum tokio-tls 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)" = "354b8cd83825b3c20217a9dc174d6a0c67441a2fae5c41bcb1ea6679f6ae0f7c" "checksum tokio-udp 0.1.5 (registry+https://github.com/rust-lang/crates.io-index)" = "f02298505547f73e60f568359ef0d016d5acd6e830ab9bc7c4a5b3403440121b" "checksum tokio-uds 0.2.5 (registry+https://github.com/rust-lang/crates.io-index)" = "037ffc3ba0e12a0ab4aca92e5234e0dedeb48fddf6ccd260f1f150a36a9f2445" -"checksum toml 0.5.3 (registry+https://github.com/rust-lang/crates.io-index)" = "c7aabe75941d914b72bf3e5d3932ed92ce0664d49d8432305a8b547c37227724" +"checksum toml 0.5.5 (registry+https://github.com/rust-lang/crates.io-index)" = "01d1404644c8b12b16bfcffa4322403a91a451584daaaa7c28d3152e6cbc98cf" "checksum traitobject 0.1.0 (registry+https://github.com/rust-lang/crates.io-index)" = "efd1f82c56340fdf16f2a953d7bda4f8fdffba13d93b00844c25572110b26079" "checksum trie-bench 0.16.2 (registry+https://github.com/rust-lang/crates.io-index)" = "3073600c543ed001319d7e092c46dfd8c245af1a218ec5c75eb01582660a2b3e" "checksum trie-db 0.15.2 (registry+https://github.com/rust-lang/crates.io-index)" = "d0b62d27e8aa1c07414549ac872480ac82380bab39e730242ab08d82d7cc098a" "checksum trie-root 0.15.2 (registry+https://github.com/rust-lang/crates.io-index)" = "0b779f7c1c8fe9276365d9d5be5c4b5adeacf545117bb3f64c974305789c5c0b" "checksum trie-standardmap 0.15.2 (registry+https://github.com/rust-lang/crates.io-index)" = "c3161ba520ab28cd8e6b68e1126f1009f6e335339d1a73b978139011703264c8" "checksum try-lock 0.2.2 (registry+https://github.com/rust-lang/crates.io-index)" = "e604eb7b43c06650e854be16a2a03155743d3752dd1c943f6829e26b7a36e382" +"checksum try_from 0.3.2 (registry+https://github.com/rust-lang/crates.io-index)" = "283d3b89e1368717881a9d51dad843cc435380d8109c9e47d38780a324698d8b" "checksum trybuild 1.0.17 (registry+https://github.com/rust-lang/crates.io-index)" = "e6851bf8351876984fbab8a2391de6378947b898410d8714edd12164d2137127" "checksum twofish 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)" = "712d261e83e727c8e2dbb75dacac67c36e35db36a958ee504f2164fc052434e1" "checksum twox-hash 1.5.0 (registry+https://github.com/rust-lang/crates.io-index)" = "3bfd5b7557925ce778ff9b9ef90e3ade34c524b5ff10e239c69a42d546d2af56" "checksum typeable 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)" = "1410f6f91f21d1612654e7cc69193b0334f909dcf2c790c4826254fbb86f8887" "checksum typenum 1.11.2 (registry+https://github.com/rust-lang/crates.io-index)" = "6d2783fe2d6b8c1101136184eb41be8b1ad379e4657050b8aaff0c79ee7575f9" -"checksum uint 0.8.1 (registry+https://github.com/rust-lang/crates.io-index)" = "f8f0f47ed099f0db671ce82c66548c5de012e3c0cba3963514d1db15c7588701" +"checksum uint 0.8.2 (registry+https://github.com/rust-lang/crates.io-index)" = "e75a4cdd7b87b28840dba13c483b9a88ee6bbf16ba5c951ee1ecfcf723078e0d" "checksum unicase 1.4.2 (registry+https://github.com/rust-lang/crates.io-index)" = "7f4765f83163b74f957c797ad9253caf97f103fb064d3999aea9568d09fc8a33" "checksum unicase 2.5.1 (registry+https://github.com/rust-lang/crates.io-index)" = "2e2e6bd1e59e56598518beb94fd6db628ded570326f0a98c679a304bd9f00150" "checksum unicode-bidi 0.3.4 (registry+https://github.com/rust-lang/crates.io-index)" = "49f2bd0c6468a8230e1db229cff8029217cf623c767ea5d60bfbd42729ea54d5" -"checksum unicode-normalization 0.1.8 (registry+https://github.com/rust-lang/crates.io-index)" = "141339a08b982d942be2ca06ff8b076563cbe223d1befd5450716790d44e2426" -"checksum unicode-segmentation 1.3.0 (registry+https://github.com/rust-lang/crates.io-index)" = "1967f4cdfc355b37fd76d2a954fb2ed3871034eb4f26d60537d88795cfc332a9" +"checksum unicode-normalization 0.1.9 (registry+https://github.com/rust-lang/crates.io-index)" = "09c8070a9942f5e7cfccd93f490fdebd230ee3c3c9f107cb25bad5351ef671cf" +"checksum unicode-segmentation 1.6.0 (registry+https://github.com/rust-lang/crates.io-index)" = "e83e153d1053cbb5a118eeff7fd5be06ed99153f00dbcd8ae310c5fb2b22edc0" "checksum unicode-width 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)" = "7007dbd421b92cc6e28410fe7362e2e0a2503394908f417b68ec8d1c364c4e20" "checksum unicode-xid 0.1.0 (registry+https://github.com/rust-lang/crates.io-index)" = "fc72304796d0818e357ead4e000d19c9c174ab23dc11093ac919054d20a6a7fc" "checksum unicode-xid 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)" = "826e7639553986605ec5979c7dd957c7895e93eabed50ab2ffa7f6128a75097c" "checksum unsigned-varint 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)" = "a7f0023a96687fe169081e8adce3f65e3874426b7886e9234d490af2dc077959" -"checksum untrusted 0.6.2 (registry+https://github.com/rust-lang/crates.io-index)" = "55cd1f4b4e96b46aeb8d4855db4a7a9bd96eeeb5c6a1ab54593328761642ce2f" +"checksum untrusted 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)" = "60369ef7a31de49bcb3f6ca728d4ba7300d9a1658f94c727d4cab8c8d9f4aece" "checksum url 1.7.2 (registry+https://github.com/rust-lang/crates.io-index)" = "dd4e7c0d531266369519a4aa4f399d748bd37043b00bde1e4ff1f60a120b355a" "checksum url 2.1.0 (registry+https://github.com/rust-lang/crates.io-index)" = "75b414f6c464c879d7f9babf951f23bc3743fb7313c081b2e6ca719067ea9d61" +"checksum uuid 0.7.4 (registry+https://github.com/rust-lang/crates.io-index)" = "90dbc611eb48397705a6b0f6e917da23ae517e4d127123d2cf7674206627d32a" "checksum vcpkg 0.2.7 (registry+https://github.com/rust-lang/crates.io-index)" = "33dd455d0f96e90a75803cfeb7f948768c08d70a6de9a8d2362461935698bf95" "checksum vec_map 0.8.1 (registry+https://github.com/rust-lang/crates.io-index)" = "05c78687fb1a80548ae3250346c3db86a80a7cdd77bda190189f2d0a0987c81a" "checksum vergen 3.0.4 (registry+https://github.com/rust-lang/crates.io-index)" = "6aba5e34f93dc7051dfad05b98a18e9156f27e7b431fe1d2398cb6061c0a1dba" "checksum version_check 0.1.5 (registry+https://github.com/rust-lang/crates.io-index)" = "914b1a6776c4c929a602fafd8bc742e06365d4bcbe48c30f9cca5824f70dc9dd" +"checksum version_check 0.9.1 (registry+https://github.com/rust-lang/crates.io-index)" = "078775d0255232fb988e6fccf26ddc9d1ac274299aaedcedce21c6f72cc533ce" "checksum void 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)" = "6a02e4885ed3bc0f2de90ea6dd45ebcbb66dacffe03547fadbb0eeae2770887d" "checksum wabt 0.9.2 (registry+https://github.com/rust-lang/crates.io-index)" = "3c5c5c1286c6e578416982609f47594265f9d489f9b836157d403ad605a46693" "checksum wabt-sys 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)" = "af5d153dc96aad7dc13ab90835b892c69867948112d95299e522d370c4e13a08" "checksum walkdir 2.2.9 (registry+https://github.com/rust-lang/crates.io-index)" = "9658c94fa8b940eab2250bd5a457f9c48b748420d71293b165c8cdbe2f55f71e" "checksum want 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)" = "b6395efa4784b027708f7451087e647ec73cc74f5d9bc2e418404248d679a230" "checksum wasi 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)" = "b89c3ce4ce14bdc6fb6beaf9ec7928ca331de5df7e5ea278375642a2f478570d" -"checksum wasm-bindgen 0.2.51 (registry+https://github.com/rust-lang/crates.io-index)" = "cd34c5ba0d228317ce388e87724633c57edca3e7531feb4e25e35aaa07a656af" -"checksum wasm-bindgen-backend 0.2.51 (registry+https://github.com/rust-lang/crates.io-index)" = "927196b315c23eed2748442ba675a4c54a1a079d90d9bdc5ad16ce31cf90b15b" +"checksum wasm-bindgen 0.2.54 (registry+https://github.com/rust-lang/crates.io-index)" = "c4568ae1b4e07ca907b1a4de41174eaa3e5be4066c024475586b7842725f69a9" +"checksum wasm-bindgen-backend 0.2.54 (registry+https://github.com/rust-lang/crates.io-index)" = "5a00cfdce37367770062065fd3abb9278cbae86a0d918cacd0978a7acd51b481" "checksum wasm-bindgen-futures 0.3.27 (registry+https://github.com/rust-lang/crates.io-index)" = "83420b37346c311b9ed822af41ec2e82839bfe99867ec6c54e2da43b7538771c" -"checksum wasm-bindgen-macro 0.2.51 (registry+https://github.com/rust-lang/crates.io-index)" = "92c2442bf04d89792816650820c3fb407af8da987a9f10028d5317f5b04c2b4a" -"checksum wasm-bindgen-macro-support 0.2.51 (registry+https://github.com/rust-lang/crates.io-index)" = "9c075d27b7991c68ca0f77fe628c3513e64f8c477d422b859e03f28751b46fc5" -"checksum wasm-bindgen-shared 0.2.51 (registry+https://github.com/rust-lang/crates.io-index)" = "83d61fe986a7af038dd8b5ec660e5849cbd9f38e7492b9404cc48b2b4df731d1" -"checksum wasm-bindgen-webidl 0.2.51 (registry+https://github.com/rust-lang/crates.io-index)" = "9b979afb0535fe4749906a674082db1211de8aef466331d43232f63accb7c07c" +"checksum wasm-bindgen-macro 0.2.54 (registry+https://github.com/rust-lang/crates.io-index)" = "7c568f4d3cf6d7c1d72b165daf778fb0d6e09a24f96ac14fc8c4f66a96e86b72" +"checksum wasm-bindgen-macro-support 0.2.54 (registry+https://github.com/rust-lang/crates.io-index)" = "430d12539ae324d16097b399e9d07a6d5ce0173b2a61a2d02346ca7c198daffe" +"checksum wasm-bindgen-shared 0.2.54 (registry+https://github.com/rust-lang/crates.io-index)" = "8ae7167f0bbffd7fac2b12da0fa1f834c1d84671a1ae3c93ac8bde2e97179c39" +"checksum wasm-bindgen-webidl 0.2.54 (registry+https://github.com/rust-lang/crates.io-index)" = "3021567c515a746a64ad0b269d120d46e687c0c95702a4750623db935ae6b5e7" "checksum wasm-gc-api 0.1.11 (registry+https://github.com/rust-lang/crates.io-index)" = "d0c32691b6c7e6c14e7f8fd55361a9088b507aa49620fcd06c09b3a1082186b9" "checksum wasm-timer 0.1.3 (registry+https://github.com/rust-lang/crates.io-index)" = "aa3e01d234bb71760e685cfafa5e2c96f8ad877c161a721646356651069e26ac" -"checksum wasmi 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)" = "f31d26deb2d9a37e6cfed420edce3ed604eab49735ba89035e13c98f9a528313" +"checksum wasmi 0.6.2 (registry+https://github.com/rust-lang/crates.io-index)" = "bf617d864d25af3587aa745529f7aaa541066c876d57e050c0d0c85c61c92aff" "checksum wasmi-validation 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)" = "6bc0356e3df56e639fc7f7d8a99741915531e27ed735d911ed83d7e1339c8188" -"checksum web-sys 0.3.28 (registry+https://github.com/rust-lang/crates.io-index)" = "c84440699cd02ca23bed6f045ffb1497bc18a3c2628bd13e2093186faaaacf6b" -"checksum webpki 0.19.1 (registry+https://github.com/rust-lang/crates.io-index)" = "4f7e1cd7900a3a6b65a3e8780c51a3e6b59c0e2c55c6dc69578c288d69f7d082" -"checksum webpki-roots 0.16.0 (registry+https://github.com/rust-lang/crates.io-index)" = "c10fa4212003ba19a564f25cd8ab572c6791f99a03cc219c13ed35ccab00de0e" +"checksum wasmi-validation 0.3.0 (registry+https://github.com/rust-lang/crates.io-index)" = "ea78c597064ba73596099281e2f4cfc019075122a65cdda3205af94f0b264d93" +"checksum wasmparser 0.39.2 (registry+https://github.com/rust-lang/crates.io-index)" = "e5083b449454f7de0b15f131eee17de54b5a71dcb9adcf11df2b2f78fad0cd82" +"checksum wasmtime-debug 0.2.0 (git+https://github.com/CraneStation/wasmtime.git?rev=71dd73d6)" = "" +"checksum wasmtime-environ 0.2.0 (git+https://github.com/CraneStation/wasmtime.git?rev=71dd73d6)" = "" +"checksum wasmtime-jit 0.2.0 (git+https://github.com/CraneStation/wasmtime.git?rev=71dd73d6)" = "" +"checksum wasmtime-runtime 0.2.0 (git+https://github.com/CraneStation/wasmtime.git?rev=71dd73d6)" = "" +"checksum web-sys 0.3.31 (registry+https://github.com/rust-lang/crates.io-index)" = "ce8e893e021539beb87de8f06e77bdb390a3ab0db4cfeb569c4e377b55ed20de" +"checksum webpki 0.21.0 (registry+https://github.com/rust-lang/crates.io-index)" = "d7e664e770ac0110e2384769bcc59ed19e329d81f555916a6e072714957b81b4" +"checksum webpki-roots 0.17.0 (registry+https://github.com/rust-lang/crates.io-index)" = "a262ae37dd9d60f60dd473d1158f9fbebf110ba7b6a5051c8160460f6043718b" +"checksum webpki-roots 0.18.0 (registry+https://github.com/rust-lang/crates.io-index)" = "91cd5736df7f12a964a5067a12c62fa38e1bd8080aff1f80bc29be7c80d19ab4" "checksum websocket 0.23.0 (registry+https://github.com/rust-lang/crates.io-index)" = "b255b190f412e45000c35be7fe9b48b39a2ac5eb90d093d421694e5dae8b335c" "checksum weedle 0.10.0 (registry+https://github.com/rust-lang/crates.io-index)" = "3bb43f70885151e629e2a19ce9e50bd730fd436cfd4b666894c9ce4de9141164" "checksum which 2.0.1 (registry+https://github.com/rust-lang/crates.io-index)" = "b57acb10231b9493c8472b20cb57317d0679a49e0bdbee44b3b803a6473af164" @@ -7339,12 +8523,24 @@ dependencies = [ "checksum winapi-util 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)" = "7168bab6e1daee33b4557efd0e95d5ca70a03706d39fa5f3fe7a236f584b03c9" "checksum winapi-x86_64-pc-windows-gnu 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)" = "712e227841d057c1ee1cd2fb22fa7e5a5461ae8e48fa2ca79ec42cfc1931183f" "checksum wincolor 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)" = "96f5016b18804d24db43cebf3c77269e7569b8954a8464501c216cc5e070eaa9" +<<<<<<< HEAD +<<<<<<< HEAD +"checksum winreg 0.6.2 (registry+https://github.com/rust-lang/crates.io-index)" = "b2986deb581c4fe11b621998a5e53361efe6b48a151178d0cd9eeffa4dc6acc9" "checksum ws 0.9.0 (registry+https://github.com/rust-lang/crates.io-index)" = "8a6f5bb86663ff4d1639408410f50bf6050367a8525d644d49a6894cd618a631" +======= +======= +"checksum winreg 0.6.2 (registry+https://github.com/rust-lang/crates.io-index)" = "b2986deb581c4fe11b621998a5e53361efe6b48a151178d0cd9eeffa4dc6acc9" +>>>>>>> 9e0aa9e5d... base-code(rebased) +"checksum ws 0.9.1 (registry+https://github.com/rust-lang/crates.io-index)" = "c51a2c47b5798ccc774ffb93ff536aec7c4275d722fd9c740c83cdd1af1f2d94" +>>>>>>> f76ccb03a... Bump dependencies with `cargo update` (#4087) "checksum ws2_32-sys 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)" = "d59cefebd0c892fa2dd6de581e937301d8552cb44489cdff035c6187cb63fa5e" "checksum x25519-dalek 0.5.2 (registry+https://github.com/rust-lang/crates.io-index)" = "7ee1585dc1484373cbc1cee7aafda26634665cf449436fd6e24bfd1fad230538" "checksum xdg 2.2.0 (registry+https://github.com/rust-lang/crates.io-index)" = "d089681aa106a86fade1b0128fb5daf07d5867a509ab036d99988dec80429a57" "checksum yaml-rust 0.3.5 (registry+https://github.com/rust-lang/crates.io-index)" = "e66366e18dc58b46801afbf2ca7661a9f59cc8c5962c29892b6039b4f86fa992" -"checksum yamux 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)" = "01bd67889938c48f0049fc60a77341039e6c3eaf16cb7693e6ead7c0ba701295" +"checksum yamux 0.2.2 (registry+https://github.com/rust-lang/crates.io-index)" = "2758f29014c1cb7a6e74c1b1160ac8c8203be342d35b73462fc6a13cc6385423" "checksum zeroize 0.10.1 (registry+https://github.com/rust-lang/crates.io-index)" = "4090487fa66630f7b166fba2bbb525e247a5449f41c468cc1d98f8ae6ac03120" "checksum zeroize 0.9.3 (registry+https://github.com/rust-lang/crates.io-index)" = "45af6a010d13e4cf5b54c94ba5a2b2eba5596b9e46bf5875612d332a1f2b3f86" -"checksum zeroize_derive 0.9.3 (registry+https://github.com/rust-lang/crates.io-index)" = "080616bd0e31f36095288bb0acdf1f78ef02c2fa15527d7e993f2a6c7591643e" +"checksum zeroize 1.0.0 (registry+https://github.com/rust-lang/crates.io-index)" = "cdc979d9b5ead18184c357c4d8a3f81b579aae264e32507223032e64715462d3" +"checksum zstd 0.4.28+zstd.1.4.3 (registry+https://github.com/rust-lang/crates.io-index)" = "f4e716acaad66f2daf2526f37a1321674a8814c0b37a366ebe6c97a699f85ddc" +"checksum zstd-safe 1.4.13+zstd.1.4.3 (registry+https://github.com/rust-lang/crates.io-index)" = "bfe4d3b26a0790201848865663e8ffabf091e126e548bc9710ccfa95621ece48" +"checksum zstd-sys 1.4.13+zstd.1.4.3 (registry+https://github.com/rust-lang/crates.io-index)" = "fadc8ebe858f056ab82dffb9d93850b841603bdf663db7cf5e3dbd7f34cc55b2" diff --git a/Cargo.toml b/Cargo.toml index 31a08f4eafbe2..2570c37423f5c 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -1,5 +1,7 @@ [workspace] members = [ +<<<<<<< HEAD + "core/prometheus", "core/authority-discovery", "core/application-crypto", "core/chain-spec", @@ -15,6 +17,8 @@ members = [ "core/consensus/slots", "core/consensus/uncles", "core/consensus/pow", + "core/block-builder", + "core/block-builder/runtime-api", "core/executor", "core/executor/runtime-test", "core/externalities", @@ -29,11 +33,16 @@ members = [ "core/rpc", "core/rpc/primitives", "core/rpc-servers", + "core/runtime-interface", + "core/runtime-interface/proc-macro", + "core/runtime-interface/test-wasm", "core/serializer", "core/service", "core/service/test", "core/session", - "core/sr-api-macros", + "core/sr-api", + "core/sr-api/proc-macro", + "core/sr-api/test", "core/sr-arithmetic", "core/sr-io", "core/sr-primitives", @@ -48,12 +57,14 @@ members = [ "core/test-runtime/client", "core/transaction-pool", "core/transaction-pool/graph", + "core/transaction-pool/runtime-api", "core/trie", "core/utils/fork-tree", "core/utils/wasm-builder", "core/utils/wasm-builder-runner", "core/wasm-interface", "srml/support", + "srml/support/rpc", "srml/support/procedural", "srml/support/procedural/tools", "srml/support/procedural/tools/derive", @@ -93,6 +104,7 @@ members = [ "srml/transaction-payment", "srml/transaction-payment/rpc", "srml/utility", + "srml/evm", "node/cli", "node/executor", "node/primitives", @@ -103,6 +115,143 @@ members = [ "node-template", "subkey", "test-utils/chain-spec-builder", +======= + "bin/node-template", + "bin/node-template/runtime", + "bin/node/cli", + "bin/node/executor", + "bin/node/primitives", + "bin/node/rpc-client", + "bin/node/rpc", + "bin/node/runtime", + "bin/node/testing", + "bin/subkey", + "client", + "client/api", + "client/authority-discovery", + "client/basic-authorship", + "client/block-builder", + "client/chain-spec", + "client/chain-spec/derive", + "client/cli", + "client/consensus/aura", + "client/consensus/babe", + "client/consensus/pow", + "client/consensus/slots", + "client/consensus/uncles", + "client/db", + "client/executor", + "client/executor/runtime-test", + "client/finality-grandpa", + "client/header-metadata", + "client/keystore", + "client/network", + "client/offchain", + "client/rpc-servers", + "client/rpc", + "client/rpc/api", + "client/service", + "client/service/test", + "client/state-db", + "client/prometheus", + "client/telemetry", + "client/transaction-pool", + "client/transaction-pool/graph", + "client/utils/wasm-builder-runner", + "paint/assets", + "paint/aura", + "paint/authority-discovery", + "paint/authorship", + "paint/babe", + "paint/balances", + "paint/collective", + "paint/contracts", + "paint/contracts/rpc", + "paint/contracts/rpc/runtime-api", + "paint/democracy", + "paint/elections-phragmen", + "paint/elections", + "paint/evm", + "paint/example", + "paint/executive", + "paint/finality-tracker", + "paint/generic-asset", + "paint/grandpa", + "paint/im-online", + "paint/indices", + "paint/membership", + "paint/metadata", + "paint/nicks", + "paint/offences", + "paint/randomness-collective-flip", + "paint/scored-pool", + "paint/session", + "paint/staking", + "paint/staking/reward-curve", + "paint/sudo", + "paint/support", + "paint/support/procedural", + "paint/support/procedural/tools", + "paint/support/procedural/tools/derive", + "paint/support/rpc", + "paint/support/test", + "paint/system", + "paint/system/rpc", + "paint/system/rpc/runtime-api", + "paint/timestamp", + "paint/transaction-payment", + "paint/transaction-payment/rpc", + "paint/transaction-payment/rpc/runtime-api", + "paint/treasury", + "paint/utility", + "primitives/application-crypto", + "primitives/authority-discovery", + "primitives/block-builder/runtime-api", + "primitives/consensus/aura", + "primitives/consensus/babe", + "primitives/consensus/common", + "primitives/consensus/pow", + "primitives/core", + "primitives/core/debug-derive", + "primitives/core/storage", + "primitives/externalities", + "primitives/finality-grandpa", + "primitives/inherents", + "primitives/keyring", + "primitives/offchain", + "primitives/panic-handler", + "primitives/peerset", + "primitives/phragmen", + "primitives/rpc", + "primitives/runtime-interface", + "primitives/runtime-interface/proc-macro", + "primitives/runtime-interface/test-wasm", + "primitives/serializer", + "primitives/session", + "primitives/sr-api", + "primitives/sr-api/proc-macro", + "primitives/sr-api/test", + "primitives/sr-arithmetic", + "primitives/sr-io", + "primitives/sr-primitives", + "primitives/sr-sandbox", + "primitives/sr-staking-primitives", + "primitives/sr-std", + "primitives/sr-version", + "primitives/state-machine", + "primitives/transaction-pool/runtime-api", + "primitives/trie", + "primitives/wasm-interface", + "test/utils/chain-spec-builder", + "test/utils/client", + "test/utils/primitives", + "test/utils/runtime", + "test/utils/runtime/client", + "test/utils/transaction-factory", + "utils/build-script-utils", + "utils/fork-tree", + "utils/wasm-builder", +>>>>>>> f44873dce... Reorganising the repository - external renames and moves (#4074) ] [profile.release] diff --git a/README.adoc b/README.adoc index bb23958bced1e..62673f70e48c7 100644 --- a/README.adoc +++ b/README.adoc @@ -4,523 +4,20 @@ :toc: :sectnums: -== Intro in one sentence - Substrate is a next-generation framework for blockchain innovation. -== Description - -At its heart, Substrate is a combination of three technologies: https://webassembly.org/[WebAssembly], https://libp2p.io/[Libp2p] and GRANDPA Consensus. About GRANDPA, see this https://hackmd.io/Jd0byWX0RiqFiXUVC78Bdw?view#GRANDPA[definition], https://medium.com/polkadot-network/grandpa-block-finality-in-polkadot-an-introduction-part-1-d08a24a021b5[introduction] and https://github.com/w3f/consensus/blob/master/pdf/grandpa.pdf[formal specification]. It is both a library for building new blockchains and a "skeleton key" of a blockchain client, able to synchronize to any Substrate-based chain. - -Substrate chains have three distinct features that make them "next-generation": a dynamic, self-defining state-transition function; light-client functionality from day one; and a progressive consensus algorithm with fast block production and adaptive, definite finality. The STF, encoded in WebAssembly, is known as the "runtime". This defines the `execute_block` function, and can specify everything from the staking algorithm, transaction semantics, logging mechanisms and procedures for replacing any aspect of itself or of the blockchain's state ("governance"). Because the runtime is entirely dynamic all of these can be switched out or upgraded at any time. A Substrate chain is very much a "living organism". - -See also https://www.parity.io/what-is-substrate/. - -== Usage - -Substrate is still an early stage project, and while it has already been used as the basis of major projects like Polkadot, using it is still a significant undertaking. In particular, you should have a good knowledge of blockchain concepts and basic cryptography. Terminology like header, block, client, hash, transaction and signature should be familiar. At present you will need a working knowledge of Rust to be able to do anything interesting (though eventually, we aim for this not to be the case). - -Substrate is designed for use in one of three ways: - -**1. Trivial**: By running the Substrate binary `substrate` and configuring it with a genesis block that includes the current demonstration runtime. In this case, you just build Substrate, configure a JSON file, and launch your own blockchain. This affords you the least amount of customizability, primarily allowing you to change the genesis parameters of the various included runtime modules such as balances, staking, block-period, fees, and governance. - -**2. Modular**: By hacking together modules from the Substrate Runtime Module Library (SRML) into a new runtime and possibly altering or reconfiguring the Substrate client's block authoring logic. This affords you a very large amount of freedom over your blockchain's logic, letting you change datatypes, add or remove modules, and crucially, add your own modules. Much can be changed without touching the block authoring logic (since it is generic). If this is the case, then the existing Substrate binary can be used for block authoring and syncing. If the block authoring logic needs to be tweaked, then a new, altered block authoring binary must be built as a separate project and used by validators. This is how the Polkadot relay chain is built and should suffice for almost all circumstances in the near to mid-term. - -**3. Generic**: The entire SRML can be ignored and the entire runtime designed and implemented from scratch. If desired, this can be done in a language other than Rust, provided it can target WebAssembly. If the runtime can be made compatible with the existing client's block authoring logic, then you can simply construct a new genesis block from your Wasm blob and launch your chain with the existing Rust-based Substrate client. If not, then you'll need to alter the client's block authoring logic accordingly. This is probably a useless option for most projects right now, but provides complete flexibility allowing for a long-term, far-reaching upgrade path for the Substrate paradigm. - -=== The Basics of Substrate - -Substrate is a blockchain platform with a completely generic state transition function. That said, it does come with both standards and conventions (particularly regarding the Runtime Module Library) regarding underlying data structures. Roughly speaking, these core datatypes correspond to +trait+s in terms of the actual non-negotiable standard and generic +struct+s in terms of the convention. - -``` -Header := Parent + ExtrinsicsRoot + StorageRoot + Digest -Block := Header + Extrinsics + Justifications -``` - -=== Extrinsics - -Extrinsics in Substrate are pieces of information from "the outside world" that are contained in the blocks of the chain. You might think "ahh, that means *transactions*": in fact, no. Extrinsics fall into two broad categories of which only one is *transactions*. The other is known as *inherents*. The difference between these two is that transactions are signed and gossiped on the network and can be deemed useful *per se*. This fits the mold of what you would call transactions in Bitcoin or Ethereum. - -Inherents, meanwhile, are not passed on the network and are not signed. They represent data which describes the environment but which cannot call upon anything to prove it such as a signature. Rather they are assumed to be "true" simply because a sufficiently large number of validators have agreed on them being reasonable. - -To give an example, there is the timestamp inherent, which sets the current timestamp of the block. This is not a fixed part of Substrate, but does come as part of the Substrate Runtime Module Library to be used as desired. No signature could fundamentally prove that a block were authored at a given time in quite the same way that a signature can "prove" the desire to spend some particular funds. Rather, it is the business of each validator to ensure that they believe the timestamp is set to something reasonable before they agree that the block candidate is valid. - -Other examples include the parachain-heads extrinsic in Polkadot and the "note-missed-proposal" extrinsic used in the Substrate Runtime Module Library to determine and punish or deactivate offline validators. - - -=== Runtime and API - -Substrate chains all have a runtime. The runtime is a WebAssembly "blob" that includes a number of entry-points. Some entry-points are required as part of the underlying Substrate specification. Others are merely convention and required for the default implementation of the Substrate client to be able to author blocks. - -If you want to develop a chain with Substrate, you will need to implement the `Core` trait. This `Core` trait generates an API with the minimum necessary functionality to interact with your runtime. A special macro is provided called `impl_runtime_apis!` that help you implement runtime API traits. All runtime API trait implementations need to be done in one call of the `impl_runtime_apis!` macro. All parameters and return values need to implement https://crates.io/crates/parity-codec[`parity-codec`] to be encodable and decodable. - -Here's a snippet of the Polkadot API implementation as of PoC-3: - -```rust -impl_runtime_apis! { - impl client_api::Core for Runtime { - fn version() -> RuntimeVersion { - VERSION - } - - fn execute_block(block: Block) { - Executive::execute_block(block) - } - - fn initialize_block(header: ::Header) { - Executive::initialize_block(&header) - } - } - // ---snip--- -} -``` - - -=== Inherent Extrinsics - -The Substrate Runtime Module Library includes functionality for timestamps and slashing. If used, these rely on "trusted" external information being passed in via inherent extrinsics. The Substrate reference block authoring client software will expect to be able to call into the runtime API with collated data (in the case of the reference Substrate authoring client, this is merely the current timestamp and which nodes were offline) in order to return the appropriate extrinsics ready for inclusion. If new inherent extrinsic types and data are to be used in a modified runtime, then it is this function (and its argument type) that would change. - -=== Block-authoring Logic - -In Substrate, there is a major distinction between blockchain *syncing* and block *authoring* ("authoring" is a more general term for what is called "mining" in Bitcoin). The first case might be referred to as a "full node" (or "light node" - Substrate supports both): authoring necessarily requires a synced node and, therefore, all authoring clients must necessarily be able to synchronize. However, the reverse is not true. The primary functionality that authoring nodes have which is not in "sync nodes" is threefold: transaction queue logic, inherent transaction knowledge and BFT consensus logic. BFT consensus logic is provided as a core element of Substrate and can be ignored since it is only exposed in the SDK under the `authorities()` API entry. - -Transaction queue logic in Substrate is designed to be as generic as possible, allowing a runtime to express which transactions are fit for inclusion in a block through the `initialize_block` and `apply_extrinsic` calls. However, more subtle aspects like prioritization and replacement policy must currently be expressed "hard coded" as part of the blockchain's authoring code. That said, Substrate's reference implementation for a transaction queue should be sufficient for an initial chain implementation. - -Inherent extrinsic knowledge is again somewhat generic, and the actual construction of the extrinsics is, by convention, delegated to the "soft code" in the runtime. If ever there needs to be additional extrinsic information in the chain, then both the block authoring logic will need to be altered to provide it into the runtime and the runtime's `inherent_extrinsics` call will need to use this extra information in order to construct any additional extrinsic transactions for inclusion in the block. - -== Roadmap - -=== So far - -- 0.1 "PoC-1": PBFT consensus, Wasm runtime engine, basic runtime modules. -- 0.2 "PoC-2": Libp2p - -=== In progress - -- AfG consensus -- Improved PoS -- Smart contract runtime module - -=== The future - -- Splitting out runtime modules into separate repo -- Introduce substrate executable (the skeleton-key runtime) -- Introduce basic but extensible transaction queue and block-builder and place them in the executable. -- DAO runtime module -- Audit - -== Trying out Substrate Node - -Substrate Node is Substrate's pre-baked blockchain client. You can run a development node locally or configure a new chain and launch your own global testnet. - -=== On Mac and Ubuntu - -To get going as fast as possible, there is a simple script that installs all required dependencies and installs Substrate into your path. Just open a terminal and run: - -[source, shell] ----- -curl https://getsubstrate.io -sSf | bash ----- - -You can start a local Substrate development chain with running `substrate --dev`. - -To create your own global network/cryptocurrency, you'll need to make a new Substrate Node chain specification file ("chainspec"). - -First let's get a template chainspec that you can edit. We'll use the "staging" chain, a sort of default chain that the node comes pre-configured with: - -[source, shell] ----- -substrate build-spec --chain=staging > ~/chainspec.json ----- - -Now, edit `~/chainspec.json` in your editor. There are a lot of individual fields for each module, and one very large one which contains the WebAssembly code blob for this chain. The easiest field to edit is the block `period`. Change it to 10 (seconds): - -[source, json] ----- - "timestamp": { - "minimumPeriod": 10 - }, ----- - -Now with this new chainspec file, you can build a "raw" chain definition for your new chain: - -[source, shell] ----- -substrate build-spec --chain ~/chainspec.json --raw > ~/mychain.json ----- - -This can be fed into Substrate: - -[source, shell] ----- -substrate --chain ~/mychain.json ----- - -It won't do much until you start producing blocks though, so to do that you'll need to use the `--validator` option together with passing the seed for the account(s) that is configured to be the initial authorities: - -[source, shell] ----- -substrate --chain ~/mychain.json --validator ----- - -You can distribute `mychain.json` so that everyone can synchronize and (depending on your authorities list) validate on your chain. - - -== Building - -=== Hacking on Substrate - -If you'd actually like to hack on Substrate, you can just grab the source code and -build it. Ensure you have Rust and the support software installed: - -==== Linux and Mac - -For Unix-based operating systems, you should run the following commands: - -[source, shell] ----- -curl https://sh.rustup.rs -sSf | sh - -rustup update nightly -rustup target add wasm32-unknown-unknown --toolchain nightly -rustup update stable ----- - -You will also need to install the following packages: - - - Linux: -[source, shell] -sudo apt install cmake pkg-config libssl-dev git clang libclang-dev - -- Linux on ARM: -`rust-lld` is required for linking wasm, but is missing on non Tier 1 platforms. -So, use this https://github.com/Plume-org/Plume/blob/master/script/wasm-deps.sh[script] -to build `lld` and create the symlink `/usr/bin/rust-lld` to the build binary. - - - Mac: -[source, shell] -brew install cmake pkg-config openssl git llvm - -To finish installation of Substrate, jump down to <>. - -==== Windows - -If you are trying to set up Substrate on Windows, you should do the following: - -1. First, you will need to download and install "Build Tools for Visual Studio:" - - * You can get it at this link: https://aka.ms/buildtools - * Run the installation file: `vs_buildtools.exe` - * Please ensure the Windows 10 SDK component is included when installing the Visual C++ Build Tools. - * image:https://i.imgur.com/zayVLmu.png[image] - * Restart your computer. - -2. Next, you need to install Rust: - - * Detailed instructions are provided by the https://doc.rust-lang.org/book/ch01-01-installation.html#installing-rustup-on-windows[Rust Book]. - * Download from: https://www.rust-lang.org/tools/install - * Run the installation file: `rustup-init.exe` - > Note that it should not prompt you to install vs_buildtools since you did it in step 1. - * Choose "Default Installation." - * To get started, you need Cargo's bin directory (%USERPROFILE%\.cargo\bin) in your PATH environment variable. Future applications will automatically have the correct environment, but you may need to restart your current shell. - -3. Then, you will need to run some commands in CMD to set up your Wasm Build Environment: - - rustup update nightly - rustup update stable - rustup target add wasm32-unknown-unknown --toolchain nightly - -4. Then, you need to install LLVM: https://releases.llvm.org/download.html - -5. Next, you need to install OpenSSL, which we will do with `vcpkg`: - - mkdir \Tools - cd \Tools - git clone https://github.com/Microsoft/vcpkg.git - cd vcpkg - .\bootstrap-vcpkg.bat - .\vcpkg.exe install openssl:x64-windows-static - -6. After, you need to add OpenSSL to your System Variables. Note that in order for the following commands to work, you need to use Windows Powershell: - - $env:OPENSSL_DIR = 'C:\Tools\vcpkg\installed\x64-windows-static' - $env:OPENSSL_STATIC = 'Yes' - [System.Environment]::SetEnvironmentVariable('OPENSSL_DIR', $env:OPENSSL_DIR, [System.EnvironmentVariableTarget]::User) - [System.Environment]::SetEnvironmentVariable('OPENSSL_STATIC', $env:OPENSSL_STATIC, [System.EnvironmentVariableTarget]::User) - -7. Finally, you need to install `cmake`: https://cmake.org/download/ - -==== Shared Steps - -Then, grab the Substrate source code: - -[source, shell] ----- -git clone https://github.com/paritytech/substrate.git -cd substrate ----- - -Then build the code: - -[source, shell] ----- -cargo build # Builds all native code ----- - -You can run all the tests if you like: - -[source, shell] -cargo test --all - -Or just run the tests of a specific package (i.e. `cargo test -p srml-assets`) - -You can start a development chain with: - -[source, shell] -cargo run --release -- --dev - -Detailed logs may be shown by running the node with the following environment variables set: `RUST_LOG=debug RUST_BACKTRACE=1 cargo run --release \-- --dev`. - -If you want to see the multi-node consensus algorithm in action locally, then you can create a local testnet with two validator nodes for Alice and Bob, who are the initial authorities of the genesis chain specification that have been endowed with a testnet DOTs. We'll give each node a name and expose them so they are listed on link:https://telemetry.polkadot.io/#/Local%20Testnet[Telemetry]. You'll need two terminal windows open. - -We'll start Alice's Substrate node first on default TCP port 30333 with her chain database stored locally at `/tmp/alice`. The Bootnode ID of her node is `QmRpheLN4JWdAnY7HGJfWFNbfkQCb6tFf4vvA6hgjMZKrR`, which is generated from the `--node-key` value that we specify below: - -[source, shell] -cargo run --release \-- \ - --base-path /tmp/alice \ - --chain=local \ - --alice \ - --node-key 0000000000000000000000000000000000000000000000000000000000000001 \ - --telemetry-url ws://telemetry.polkadot.io:1024 \ - --validator - -In the second terminal, we'll run the following to start Bob's Substrate node on a different TCP port of 30334, and with his chain database stored locally at `/tmp/bob`. We'll specify a value for the `--bootnodes` option that will connect his node to Alice's Bootnode ID on TCP port 30333: - -[source, shell] -cargo run --release \-- \ - --base-path /tmp/bob \ - --bootnodes /ip4/127.0.0.1/tcp/30333/p2p/QmRpheLN4JWdAnY7HGJfWFNbfkQCb6tFf4vvA6hgjMZKrR \ - --chain=local \ - --bob \ - --port 30334 \ - --telemetry-url ws://telemetry.polkadot.io:1024 \ - --validator - -Additional Substrate CLI usage options are available and may be shown by running `cargo run \-- --help`. - -=== WASM binaries - -The WASM binaries are built during the normal `cargo build` process. To control the WASM binary building, -we support multiple environment variables: - -* `SKIP_WASM_BUILD` - Skips building any WASM binary. This is useful when only native should be recompiled. -* `BUILD_DUMMY_WASM_BINARY` - Builds dummy WASM binaries. These dummy binaries are empty and useful - for `cargo check` runs. -* `WASM_BUILD_TYPE` - Sets the build type for building WASM binaries. Supported values are `release` or `debug`. - By default the build type is equal to the build type used by the main build. -* `TRIGGER_WASM_BUILD` - Can be set to trigger a WASM build. On subsequent calls the value of the variable - needs to change. As WASM builder instructs `cargo` to watch for file changes - this environment variable should only be required in certain circumstances. -* `WASM_TARGET_DIRECTORY` - Will copy any build WASM binary to the given directory. The path needs - to be absolute. - -Each project can be skipped individually by using the environment variable `SKIP_PROJECT_NAME_WASM_BUILD`. -Where `PROJECT_NAME` needs to be replaced by the name of the cargo project, e.g. `node-runtime` will -be `NODE_RUNTIME`. - -[[flaming-fir]] -=== Joining the Flaming Fir Testnet - -Flaming Fir is the new testnet for Substrate master (2.0) to test the latest development features. Please note that master is not compatible with the BBQ Birch, Charred Cherry, Dried Danta or Emberic Elm testnets. Ensure you have the dependencies listed above before compiling. - -Since Flaming Fir is targeting the master branch we make absolutely no guarantees of stability and/or persistence of the network. We might reset the chain at any time if it is necessary to deploy new changes. Currently, the validators are running with a client built from `d013bd900`, if you build from this commit you should be able to successfully sync, later commits may not work as new breaking changes may be introduced in master. - -Latest known working version: `a2a0eb5398d6223e531455b4c155ef053a4a3a2b` - -[source, shell] ----- -git clone https://github.com/paritytech/substrate.git -cd substrate -git checkout -b flaming-fir a2a0eb5398d6223e531455b4c155ef053a4a3a2b ----- - -You can run the tests if you like: - -[source, shell] -cargo test --all - -Start your node: - -[source, shell] -cargo run --release \-- - -To see a list of command line options, enter: - -[source, shell] -cargo run --release \-- --help - -For example, you can choose a custom node name: - -[source, shell] -cargo run --release \-- --name my_custom_name - -If you are successful, you will see your node syncing at https://telemetry.polkadot.io/#/Flaming%20Fir - -=== Joining the Emberic Elm Testnet - -Emberic Elm is the testnet for Substrate 1.0. Please note that 1.0 is not compatible with the BBQ Birch, Charred Cherry, Dried Danta or Flaming Fir testnets. -In order to join the Emberic Elm testnet you should build from the `v1.0` branch. Ensure you have the dependencies listed above before compiling. - -[source, shell] ----- -git clone https://github.com/paritytech/substrate.git -cd substrate -git checkout -b v1.0 origin/v1.0 ----- - -You can then follow the same steps for building and running as described above in <>. - -== Key management - -Keys in Substrate are stored in the keystore in the file system. To store keys into this keystore, -you need to use one of the two provided RPC calls. If your keys are encrypted or should be encrypted -by the keystore, you need to provide the key using one of the cli arguments `--password`, -`--password-interactive` or `--password-filename`. - -=== Recommended RPC call - -For most users who want to run a validator node, the `author_rotateKeys` RPC call is sufficient. -The RPC call will generate `N` Session keys for you and return their public keys. `N` is the number -of session keys configured in the runtime. The output of the RPC call can be used as input for the -`session::set_keys` transaction. - -``` -curl -H 'Content-Type: application/json' --data '{ "jsonrpc":"2.0", "method":"author_rotateKeys", "id":1 }' localhost:9933 -``` - -=== Advanced RPC call - -If the Session keys need to match a fixed seed, they can be set individually key by key. The RPC call -expects the key seed and the key type. The key types supported by default in Substrate are listed -https://github.com/paritytech/substrate/blob/master/core/primitives/src/crypto.rs#L767[here], but the -user can declare any key type. - -``` -curl -H 'Content-Type: application/json' --data '{ "jsonrpc":"2.0", "method":"author_insertKey", "params":["KEY_TYPE", "SEED", "PUBLIC"],"id":1 }' localhost:9933 -``` - -`KEY_TYPE` - needs to be replaced with the 4-character key type identifier. -`SEED` - is the seed of the key. -`PUBLIC` - public key for the given key. - -== Documentation - -=== Viewing documentation for Substrate packages - -You can generate documentation for a Substrate Rust package and have it automatically open in your web browser using https://doc.rust-lang.org/rustdoc/what-is-rustdoc.html#using-rustdoc-with-cargo[rustdoc with Cargo], -(of the The Rustdoc Book), by running the following command: - -``` -cargo doc --package --open -``` - -Replacing `` with one of the following (i.e. `cargo doc --package substrate --open`): - -* All Substrate Packages -[source, shell] -substrate -* Substrate Core -[source, shell] -substrate, substrate-cli, substrate-client, substrate-client-db, -substrate-consensus-common, substrate-consensus-rhd, -substrate-executor, substrate-finality-grandpa, substrate-keyring, substrate-keystore, substrate-network, -substrate-network-libp2p, substrate-primitives, substrate-rpc, substrate-rpc-servers, -substrate-serializer, substrate-service, substrate-service-test, substrate-state-db, -substrate-state-machine, substrate-telemetry, substrate-test-client, -substrate-test-runtime, substrate-transaction-graph, substrate-transaction-pool, -substrate-trie -* Substrate Runtime -[source, shell] -sr-api, sr-io, sr-primitives, sr-sandbox, sr-std, sr-version -* Substrate Runtime Module Library (SRML) -[source, shell] -srml-assets, srml-balances, srml-consensus, srml-contracts, srml-council, srml-democracy, srml-example, -srml-executive, srml-metadata, srml-session, srml-staking, srml-support, srml-system, srml-timestamp, -srml-treasury -* Node -[source, shell] -node-cli, node-consensus, node-executor, node-network, node-primitives, node-runtime -* Subkey -[source, shell] -subkey - -=== Contributing to documentation for Substrate packages - -https://doc.rust-lang.org/1.9.0/book/documentation.html[Document source code] for Substrate packages by annotating the source code with documentation comments. - -Example (generic): -```markdown -/// Summary -/// -/// Description -/// -/// # Panics -/// -/// # Errors -/// -/// # Safety -/// -/// # Examples -/// -/// Summary of Example 1 -/// -/// ```rust -/// // insert example 1 code here -/// ``` -/// -``` - -* Important notes: -** Documentation comments must use annotations with a triple slash `///` -** Modules are documented using `//!` -``` -//! Summary (of module) -//! -//! Description (of module) -``` -* Special section header is indicated with a hash `#`. -** `Panics` section requires an explanation if the function triggers a panic -** `Errors` section is for describing conditions under which a function of method returns `Err(E)` if it returns a `Result` -** `Safety` section requires an explanation if the function is `unsafe` -** `Examples` section includes examples of using the function or method -* Code block annotations for examples are included between triple graves, as shown above. -Instead of including the programming language to use for syntax highlighting as the annotation -after the triple graves, alternative annotations include the `ignore`, `text`, `should_panic`, or `no_run`. -* Summary sentence is a short high level single sentence of its functionality -* Description paragraph is for details additional to the summary sentence -* Missing documentation annotations may be used to identify where to generate warnings with `#![warn(missing_docs)]` -or errors `#![deny(missing_docs)]` -* Hide documentation for items with `#[doc(hidden)]` - -=== Contributing to documentation (tests, extended examples, macros) for Substrate packages - -The code block annotations in the `# Example` section may be used as https://doc.rust-lang.org/1.9.0/book/documentation.html#documentation-as-tests[documentation as tests and for extended examples]. - -* Important notes: -** Rustdoc will automatically add a `main()` wrapper around the code block to test it -** https://doc.rust-lang.org/1.9.0/book/documentation.html#documenting-macros[Documenting macros]. -** Documentation as tests examples are included when running `cargo test` +== Trying it out -== Contributing +Simply go to [substrate.dev](https://substrate.dev) and follow the instructions as listed. -=== Contributing Guidelines +== Contributions & Code of Conduct -include::CONTRIBUTING.adoc[] +Please follow the contributions guidelines as outlined in [`docs/CONTRIBUTING.adoc`](docs/CONTRIBUTING.adoc). In all communications and contributions, this project follows the [Contributor Covenant Code of Conduct](docs/CODE_OF_CONDUCT.adoc). -=== Contributor Code of Conduct +== Security -include::CODE_OF_CONDUCT.adoc[] +The security policy and procedures can be found in [`docs/SECURITY.md`](docs/SECURITY.md). == License -https://github.com/paritytech/substrate/blob/master/LICENSE[LICENSE] +GPL 3.0 see `LICENSE`-file in the repo root. \ No newline at end of file diff --git a/bin/node-template/Cargo.toml b/bin/node-template/Cargo.toml new file mode 100644 index 0000000000000..99ea7af798f16 --- /dev/null +++ b/bin/node-template/Cargo.toml @@ -0,0 +1,41 @@ +[package] +name = "node-template" +version = "2.0.0" +authors = ["Anonymous"] +build = "build.rs" +edition = "2018" + +[[bin]] +name = "node-template" +path = "src/main.rs" + +[dependencies] +derive_more = "0.15.0" +futures = "0.1.29" +ctrlc = { version = "3.1.3", features = ["termination"] } +log = "0.4.8" +tokio = "0.1.22" +exit-future = "0.1.4" +parking_lot = "0.9.0" +codec = { package = "parity-scale-codec", version = "1.0.0" } +trie-root = "0.15.2" +sr-io = { path = "../../primitives/sr-io" } +substrate-cli = { path = "../../client/cli" } +primitives = { package = "substrate-primitives", path = "../../primitives/core" } +substrate-executor = { path = "../../client/executor" } +substrate-service = { path = "../../client/service" } +inherents = { package = "substrate-inherents", path = "../../primitives/inherents" } +transaction-pool = { package = "substrate-transaction-pool", path = "../../client/transaction-pool" } +network = { package = "substrate-network", path = "../../client/network" } +aura = { package = "substrate-consensus-aura", path = "../../client/consensus/aura" } +aura-primitives = { package = "substrate-consensus-aura-primitives", path = "../../primitives/consensus/aura" } +grandpa = { package = "substrate-finality-grandpa", path = "../../client/finality-grandpa" } +grandpa-primitives = { package = "substrate-finality-grandpa-primitives", path = "../../primitives/finality-grandpa" } +substrate-client = { path = "../../client/" } +runtime = { package = "node-template-runtime", path = "runtime" } +sr-primitives = { path = "../../primitives/sr-primitives" } +basic-authorship = { package = "substrate-basic-authorship", path = "../../client/basic-authorship"} + +[build-dependencies] +vergen = "3.0.4" +build-script-utils = { package = "substrate-build-script-utils", path = "../../utils/build-script-utils" } diff --git a/node-template/LICENSE b/bin/node-template/LICENSE similarity index 100% rename from node-template/LICENSE rename to bin/node-template/LICENSE diff --git a/node-template/README.md b/bin/node-template/README.md similarity index 100% rename from node-template/README.md rename to bin/node-template/README.md diff --git a/bin/node-template/build.rs b/bin/node-template/build.rs new file mode 100644 index 0000000000000..222cbb409285b --- /dev/null +++ b/bin/node-template/build.rs @@ -0,0 +1,9 @@ +use vergen::{ConstantsFlags, generate_cargo_keys}; + +const ERROR_MSG: &str = "Failed to generate metadata files"; + +fn main() { + generate_cargo_keys(ConstantsFlags::SHA_SHORT).expect(ERROR_MSG); + + build_script_utils::rerun_if_git_head_changed(); +} diff --git a/bin/node-template/runtime/Cargo.toml b/bin/node-template/runtime/Cargo.toml new file mode 100644 index 0000000000000..17b1c59106cd7 --- /dev/null +++ b/bin/node-template/runtime/Cargo.toml @@ -0,0 +1,67 @@ +[package] +name = "node-template-runtime" +version = "2.0.0" +authors = ["Anonymous"] +edition = "2018" + +[dependencies] +aura = { package = "paint-aura", path = "../../../paint/aura", default-features = false } +aura-primitives = { package = "substrate-consensus-aura-primitives", path = "../../../primitives/consensus/aura", default-features = false } +balances = { package = "paint-balances", path = "../../../paint/balances", default-features = false } +block-builder-api = { package = "substrate-block-builder-runtime-api", path = "../../../primitives/block-builder/runtime-api", default-features = false} +codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false, features = ["derive"] } +executive = { package = "paint-executive", path = "../../../paint/executive", default-features = false } +grandpa = { package = "paint-grandpa", path = "../../../paint/grandpa", default-features = false } +indices = { package = "paint-indices", path = "../../../paint/indices", default-features = false } +inherents = { package = "substrate-inherents", path = "../../../primitives/inherents", default-features = false} +offchain-primitives = { package = "substrate-offchain-primitives", path = "../../../primitives/offchain", default-features = false } +primitives = { package = "substrate-primitives", path = "../../../primitives/core", default-features = false } +randomness-collective-flip = { package = "paint-randomness-collective-flip", path = "../../../paint/randomness-collective-flip", default-features = false } +rstd = { package = "sr-std", path = "../../../primitives/sr-std", default-features = false } +runtime-io = { package = "sr-io", path = "../../../primitives/sr-io", default-features = false } +safe-mix = { version = "1.0.0", default-features = false } +serde = { version = "1.0.101", optional = true, features = ["derive"] } +sr-api = { path = "../../../primitives/sr-api", default-features = false } +sr-primitives = { path = "../../../primitives/sr-primitives", default-features = false } +substrate-session = { path = "../../../primitives/session", default-features = false } +sudo = { package = "paint-sudo", path = "../../../paint/sudo", default-features = false } +support = { package = "paint-support", path = "../../../paint/support", default-features = false } +system = { package = "paint-system", path = "../../../paint/system", default-features = false } +timestamp = { package = "paint-timestamp", path = "../../../paint/timestamp", default-features = false } +transaction-payment = { package = "paint-transaction-payment", path = "../../../paint/transaction-payment", default-features = false } +tx-pool-api = { package = "substrate-transaction-pool-runtime-api", path = "../../../primitives/transaction-pool/runtime-api", default-features = false } +version = { package = "sr-version", path = "../../../primitives/sr-version", default-features = false } + +[build-dependencies] +wasm-builder-runner = { package = "substrate-wasm-builder-runner", path = "../../../client/utils/wasm-builder-runner", version = "1.0.4" } + +[features] +default = ["std"] +std = [ + "aura-primitives/std", + "aura/std", + "balances/std", + "block-builder-api/std", + "codec/std", + "executive/std", + "grandpa/std", + "indices/std", + "inherents/std", + "offchain-primitives/std", + "primitives/std", + "randomness-collective-flip/std", + "rstd/std", + "runtime-io/std", + "safe-mix/std", + "serde", + "sr-api/std", + "sr-primitives/std", + "substrate-session/std", + "sudo/std", + "support/std", + "system/std", + "timestamp/std", + "transaction-payment/std", + "tx-pool-api/std", + "version/std", +] diff --git a/node-template/runtime/build.rs b/bin/node-template/runtime/build.rs similarity index 100% rename from node-template/runtime/build.rs rename to bin/node-template/runtime/build.rs diff --git a/node-template/runtime/src/lib.rs b/bin/node-template/runtime/src/lib.rs similarity index 93% rename from node-template/runtime/src/lib.rs rename to bin/node-template/runtime/src/lib.rs index c0cd2cf33770c..4b7a572da2337 100644 --- a/node-template/runtime/src/lib.rs +++ b/bin/node-template/runtime/src/lib.rs @@ -18,12 +18,9 @@ use sr_primitives::traits::{ NumberFor, BlakeTwo256, Block as BlockT, StaticLookup, Verify, ConvertInto, IdentifyAccount }; use sr_primitives::weights::Weight; -use client::{ - block_builder::api::{CheckInherentsResult, InherentData, self as block_builder_api}, - runtime_api as client_api, impl_runtime_apis -}; +use sr_api::impl_runtime_apis; use aura_primitives::sr25519::AuthorityId as AuraId; -use grandpa::{AuthorityId as GrandpaId, AuthorityWeight as GrandpaWeight}; +use grandpa::AuthorityList as GrandpaAuthorityList; use grandpa::fg_primitives; use version::RuntimeVersion; #[cfg(feature = "std")] @@ -283,7 +280,7 @@ pub type CheckedExtrinsic = generic::CheckedExtrinsic, Runtime, AllModules>; impl_runtime_apis! { - impl client_api::Core for Runtime { + impl sr_api::Core for Runtime { fn version() -> RuntimeVersion { VERSION } @@ -297,7 +294,7 @@ impl_runtime_apis! { } } - impl client_api::Metadata for Runtime { + impl sr_api::Metadata for Runtime { fn metadata() -> OpaqueMetadata { Runtime::metadata().into() } @@ -312,11 +309,14 @@ impl_runtime_apis! { Executive::finalize_block() } - fn inherent_extrinsics(data: InherentData) -> Vec<::Extrinsic> { + fn inherent_extrinsics(data: inherents::InherentData) -> Vec<::Extrinsic> { data.create_extrinsics() } - fn check_inherents(block: Block, data: InherentData) -> CheckInherentsResult { + fn check_inherents( + block: Block, + data: inherents::InherentData, + ) -> inherents::CheckInherentsResult { data.check_extrinsics(&block) } @@ -325,7 +325,7 @@ impl_runtime_apis! { } } - impl client_api::TaggedTransactionQueue for Runtime { + impl tx_pool_api::TaggedTransactionQueue for Runtime { fn validate_transaction(tx: ::Extrinsic) -> TransactionValidity { Executive::validate_transaction(tx) } @@ -349,13 +349,12 @@ impl_runtime_apis! { impl substrate_session::SessionKeys for Runtime { fn generate_session_keys(seed: Option>) -> Vec { - let seed = seed.as_ref().map(|s| rstd::str::from_utf8(&s).expect("Seed is an utf8 string")); opaque::SessionKeys::generate(seed) } } impl fg_primitives::GrandpaApi for Runtime { - fn grandpa_authorities() -> Vec<(GrandpaId, GrandpaWeight)> { + fn grandpa_authorities() -> GrandpaAuthorityList { Grandpa::grandpa_authorities() } } diff --git a/node-template/runtime/src/template.rs b/bin/node-template/runtime/src/template.rs similarity index 98% rename from node-template/runtime/src/template.rs rename to bin/node-template/runtime/src/template.rs index db7bd24b0e419..e28099d825e79 100644 --- a/node-template/runtime/src/template.rs +++ b/bin/node-template/runtime/src/template.rs @@ -6,7 +6,7 @@ /// For more guidance on Substrate modules, see the example module -/// https://github.com/paritytech/substrate/blob/master/srml/example/src/lib.rs +/// https://github.com/paritytech/substrate/blob/master/paint/example/src/lib.rs use support::{decl_module, decl_storage, decl_event, dispatch::Result}; use system::ensure_signed; diff --git a/scripts/init.sh b/bin/node-template/scripts/init.sh similarity index 100% rename from scripts/init.sh rename to bin/node-template/scripts/init.sh diff --git a/node-template/src/chain_spec.rs b/bin/node-template/src/chain_spec.rs similarity index 100% rename from node-template/src/chain_spec.rs rename to bin/node-template/src/chain_spec.rs diff --git a/node-template/src/cli.rs b/bin/node-template/src/cli.rs similarity index 97% rename from node-template/src/cli.rs rename to bin/node-template/src/cli.rs index ceb51504e29ba..ec463a236b2e0 100644 --- a/node-template/src/cli.rs +++ b/bin/node-template/src/cli.rs @@ -39,7 +39,7 @@ pub fn run(args: I, exit: E, version: VersionInfo) -> error::Result<()> ), } }), - ParseAndPrepare::BuildSpec(cmd) => cmd.run(load_spec), + ParseAndPrepare::BuildSpec(cmd) => cmd.run::(load_spec), ParseAndPrepare::ExportBlocks(cmd) => cmd.run_with_builder(|config: Config<_>| Ok(new_full_start!(config).0), load_spec, exit), ParseAndPrepare::ImportBlocks(cmd) => cmd.run_with_builder(|config: Config<_>| diff --git a/node-template/src/main.rs b/bin/node-template/src/main.rs similarity index 100% rename from node-template/src/main.rs rename to bin/node-template/src/main.rs diff --git a/node-template/src/service.rs b/bin/node-template/src/service.rs similarity index 97% rename from node-template/src/service.rs rename to bin/node-template/src/service.rs index 398795325fd04..ad517d956b4c7 100644 --- a/node-template/src/service.rs +++ b/bin/node-template/src/service.rs @@ -12,6 +12,7 @@ use substrate_executor::native_executor_instance; pub use substrate_executor::NativeExecutor; use aura_primitives::sr25519::{AuthorityPair as AuraPair}; use grandpa::{self, FinalityProofProvider as GrandpaFinalityProofProvider}; +use basic_authorship; // Our native executor instance. native_executor_instance!( @@ -48,7 +49,7 @@ macro_rules! new_full_start { .ok_or_else(|| substrate_service::Error::SelectChainRequired)?; let (grandpa_block_import, grandpa_link) = - grandpa::block_import::<_, _, _, runtime::RuntimeApi, _, _>( + grandpa::block_import::<_, _, _, runtime::RuntimeApi, _>( client.clone(), &*client, select_chain )?; @@ -197,8 +198,8 @@ pub fn new_light(config: Configuration( - client.clone(), backend, Arc::new(fetch_checker), client.clone() + let grandpa_block_import = grandpa::light_block_import::<_, _, _, RuntimeApi>( + client.clone(), backend, &*client.clone(), Arc::new(fetch_checker), )?; let finality_proof_import = grandpa_block_import.clone(); let finality_proof_request_builder = diff --git a/bin/node/cli/Cargo.toml b/bin/node/cli/Cargo.toml new file mode 100644 index 0000000000000..ba7568f571e83 --- /dev/null +++ b/bin/node/cli/Cargo.toml @@ -0,0 +1,141 @@ +[package] +name = "node-cli" +version = "2.0.0" +authors = ["Parity Technologies "] +description = "Substrate node implementation in Rust." +build = "build.rs" +edition = "2018" +default-run = "substrate" + +[badges] +travis-ci = { repository = "paritytech/substrate", branch = "master" } +maintenance = { status = "actively-developed" } +is-it-maintained-issue-resolution = { repository = "paritytech/substrate" } +is-it-maintained-open-issues = { repository = "paritytech/substrate" } + +[[bin]] +name = "substrate" +path = "bin/main.rs" +required-features = ["cli"] + +[lib] +crate-type = ["cdylib", "rlib"] + +[dependencies] +# third-party dependencies +codec = { package = "parity-scale-codec", version = "1.0.6" } +serde = { version = "1.0.102", features = [ "derive" ] } +futures = "0.1.29" +futures03 = { package = "futures-preview", version = "0.3.0-alpha.19", features = ["compat"] } +hex-literal = "0.2.1" +jsonrpc-core = "14.0.3" +log = "0.4.8" +rand = "0.7.2" +structopt = "0.3.3" + +# primitives +authority-discovery-primitives = { package = "substrate-authority-discovery-primitives", path = "../../../primitives/authority-discovery"} +babe-primitives = { package = "substrate-consensus-babe-primitives", path = "../../../primitives/consensus/babe" } +grandpa_primitives = { package = "substrate-finality-grandpa-primitives", path = "../../../primitives/finality-grandpa" } +primitives = { package = "substrate-primitives", path = "../../../primitives/core" } +sr-primitives = { path = "../../../primitives/sr-primitives" } + +# core dependencies +runtime-io = { package = "sr-io", path = "../../../primitives/sr-io" } +client-api = { package = "substrate-client-api", path = "../../../client/api" } +client = { package = "substrate-client", path = "../../../client/" } +inherents = { package = "substrate-inherents", path = "../../../primitives/inherents" } +chain-spec = { package = "substrate-chain-spec", path = "../../../client/chain-spec" } +transaction_pool = { package = "substrate-transaction-pool", path = "../../../client/transaction-pool" } +network = { package = "substrate-network", path = "../../../client/network" } +babe = { package = "substrate-consensus-babe", path = "../../../client/consensus/babe" } +grandpa = { package = "substrate-finality-grandpa", path = "../../../client/finality-grandpa" } +keyring = { package = "substrate-keyring", path = "../../../primitives/keyring" } +client_db = { package = "substrate-client-db", path = "../../../client/db", default-features = false } +offchain = { package = "substrate-offchain", path = "../../../client/offchain" } +substrate-rpc = { package = "substrate-rpc", path = "../../../client/rpc" } +substrate-basic-authorship = { path = "../../../client/basic-authorship" } +substrate-service = { path = "../../../client/service", default-features = false } +substrate-telemetry = { package = "substrate-telemetry", path = "../../../client/telemetry" } +authority-discovery = { package = "substrate-authority-discovery", path = "../../../client/authority-discovery"} + +# paint dependencies +indices = { package = "paint-indices", path = "../../../paint/indices" } +timestamp = { package = "paint-timestamp", path = "../../../paint/timestamp", default-features = false } +finality_tracker = { package = "paint-finality-tracker", path = "../../../paint/finality-tracker", default-features = false } +contracts = { package = "paint-contracts", path = "../../../paint/contracts" } +system = { package = "paint-system", path = "../../../paint/system" } +balances = { package = "paint-balances", path = "../../../paint/balances" } +transaction-payment = { package = "paint-transaction-payment", path = "../../../paint/transaction-payment" } +support = { package = "paint-support", path = "../../../paint/support", default-features = false } +im_online = { package = "paint-im-online", path = "../../../paint/im-online", default-features = false } +sr-authority-discovery = { package = "paint-authority-discovery", path = "../../../paint/authority-discovery"} + + +# node-specific dependencies +node-runtime = { path = "../runtime" } +node-rpc = { path = "../rpc" } +node-primitives = { path = "../primitives" } +node-executor = { path = "../executor" } + +# CLI-specific dependencies +tokio = { version = "0.1.22", optional = true } +exit-future = { version = "0.1.4", optional = true } +substrate-cli = { path = "../../../client/cli", optional = true } +transaction-factory = { path = "../../../test/utils/transaction-factory", optional = true } +ctrlc = { version = "3.1.3", features = ["termination"], optional = true } + +# WASM-specific dependencies +libp2p = { version = "0.13.0", default-features = false, optional = true } +clear_on_drop = { version = "0.2.3", features = ["no_cc"], optional = true } # Imported just for the `no_cc` feature +console_error_panic_hook = { version = "0.1.1", optional = true } +console_log = { version = "0.1.2", optional = true } +js-sys = { version = "0.3.22", optional = true } +wasm-bindgen = { version = "0.2.45", optional = true } +wasm-bindgen-futures = { version = "0.3.22", optional = true } +kvdb-memorydb = { git = "https://github.com/paritytech/parity-common", rev="b0317f649ab2c665b7987b8475878fc4d2e1f81d", optional = true } +rand6 = { package = "rand", version = "0.6", features = ["wasm-bindgen"], optional = true } # Imported just for the `wasm-bindgen` feature + +[dev-dependencies] +keystore = { package = "substrate-keystore", path = "../../../client/keystore" } +babe = { package = "substrate-consensus-babe", path = "../../../client/consensus/babe", features = ["test-helpers"] } +consensus-common = { package = "substrate-consensus-common", path = "../../../primitives/consensus/common" } +service-test = { package = "substrate-service-test", path = "../../../client/service/test" } +futures03 = { package = "futures-preview", version = "0.3.0-alpha.19" } +tempfile = "3.1.0" + +[build-dependencies] +substrate-cli = { package = "substrate-cli", path = "../../../client/cli" } +build-script-utils = { package = "substrate-build-script-utils", path = "../../../utils/build-script-utils" } +structopt = "0.3.3" +vergen = "3.0.4" + +[features] +default = ["cli"] +browser = [ + "clear_on_drop", + "console_error_panic_hook", + "console_log", + "js-sys", + "libp2p", + "wasm-bindgen", + "wasm-bindgen-futures", + "kvdb-memorydb", + "rand/wasm-bindgen", + "rand6" +] +cli = [ + "substrate-cli", + "transaction-factory", + "tokio", + "exit-future", + "ctrlc", + "substrate-service/rocksdb", + "node-executor/wasmi-errno", +] +wasmtime = [ + "cli", + "node-executor/wasmtime", + "substrate-cli/wasmtime", + "substrate-service/wasmtime", +] diff --git a/node/cli/bin/main.rs b/bin/node/cli/bin/main.rs similarity index 93% rename from node/cli/bin/main.rs rename to bin/node/cli/bin/main.rs index 4b07d8c8a4805..e4415a2a89e66 100644 --- a/node/cli/bin/main.rs +++ b/bin/node/cli/bin/main.rs @@ -18,15 +18,15 @@ #![warn(missing_docs)] -use cli::VersionInfo; use futures::sync::oneshot; use futures::{future, Future}; +use substrate_cli::VersionInfo; use std::cell::RefCell; // handles ctrl-c struct Exit; -impl cli::IntoExit for Exit { +impl substrate_cli::IntoExit for Exit { type Exit = future::MapErr, fn(oneshot::Canceled) -> ()>; fn into_exit(self) -> Self::Exit { // can't use signal directly here because CtrlC takes only `Fn`. @@ -43,7 +43,7 @@ impl cli::IntoExit for Exit { } } -fn main() -> Result<(), cli::error::Error> { +fn main() -> Result<(), substrate_cli::error::Error> { let version = VersionInfo { name: "Substrate Node", commit: env!("VERGEN_SHA_SHORT"), diff --git a/bin/node/cli/browser-demo/.gitignore b/bin/node/cli/browser-demo/.gitignore new file mode 100644 index 0000000000000..0c6117d9fb83b --- /dev/null +++ b/bin/node/cli/browser-demo/.gitignore @@ -0,0 +1 @@ +pkg \ No newline at end of file diff --git a/bin/node/cli/browser-demo/README.md b/bin/node/cli/browser-demo/README.md new file mode 100644 index 0000000000000..4faebcbc76df4 --- /dev/null +++ b/bin/node/cli/browser-demo/README.md @@ -0,0 +1,10 @@ +# How to run this demo + +```sh +cargo install wasm-pack # If necessary + +# From the `node/cli` directory (parent from this README) +wasm-pack build --target web --out-dir ./demo/pkg --no-typescript --release -- --no-default-features --features "browser" + +xdg-open index.html +``` diff --git a/bin/node/cli/browser-demo/build.sh b/bin/node/cli/browser-demo/build.sh new file mode 100755 index 0000000000000..c16100794ad2e --- /dev/null +++ b/bin/node/cli/browser-demo/build.sh @@ -0,0 +1,3 @@ +#!/usr/bin/env sh +wasm-pack build --target web --out-dir ./browser-demo/pkg --no-typescript --release ./.. -- --no-default-features --features "browser" +python -m SimpleHTTPServer 8000 diff --git a/bin/node/cli/browser-demo/favicon.png b/bin/node/cli/browser-demo/favicon.png new file mode 100644 index 0000000000000000000000000000000000000000..8a4548ce34dfa220f612080820cfde778a39cb8f GIT binary patch literal 10338 zcmYLP1yoaS+(wZQfk{a_5mcmyv>+u2qR5bj(d7VXk&w|jQbIxLPKhB5DS^?QA~Be> zGziGpzKj3wJKu9I=j@F4zQ5;ve)Zn>E>cVL2@Mq+6$uFmjk?-nZ4#18f){_3q`)_D zIm|2x$>n3MXF4jp|8W2QeLg-uetv!d0RcflK_MX_VPRnr5fKOg(F;6y@IXvVOdNoO zxP*j+q@<*jl$1088EF|AnTHP_$^wwPz@tZxgej~>gnm} z128Z!Ff=eUG&BNWY;0^|YyvZZ!CuI668y0eJoTwX?Ici?fT1iz@&(H#c`TcMo?D0G^(nZ{ED| zdgJxh>n#9pZ*LzTA739|KVLrp@7}%h_xDHmBLWbJfPjF&z`&rOpx~h3kl^5ukdV;O z(6G?3@UZZR@bHL;h{(vusK}`3sAyz#G!lu7iHV7giH(bmjf;zmkB?7CNJvabe4m*3 z{{8!;q@?8JF(<4>F)0B z>FMq5?d$FB@9P`r?;ji(7#tiN8XO!R8X6fM9vvAO9UU1P9UUJVn;0LToR|QvsmaOd zsi_~+(=$JQ0N2l%nc1H|=VoW;=jMQGVSXOFu&{{5E-qr178h|#OTTcqVf>&8^?Rx3{)-wzqe8cJ_96_xJV=_64TdJb+BN zWTYh5ft@eU`+KYNcD5M{XM?7``7t`@lo-?TqSWw8gC-HkVTpdIuznK{@L`HO%;X5( z5npN=E#+pjd4(c{(Js@-utc}8IBqBXov{8J_{6U^N!lLFkfj7X+1Fy7)$nv6ImVJx z?szijH$G>mSslenblh}AbvmYNFFMuu+U?xF-Dm5u5_I5ROAA7tIsQ2E6d|K5sIz3o$M{LPA|;*R$&mtt|s+o)K?&I~!SYvz&J zUjGn_sjC7^uk`#E;;Xr=-rLo|&?ll9?E9Uu(VJyPzH$BsT7&NGlLW6TR_SA&1=Hth zH8Lf7gWmae?sYz$?uc)d5dX5%++|DERP^5_&yI&);?}a0md`rJgIP==`wy$!v|#5c z&PU@1@^-S5*fhaDO>_Vkv{5(3*Cv`tK>c25fgh(yJ8=3!h%%D1ycF^bWcTT#@|8 z-)Tz2MKd0Ldz^G$1?7bGt;T#B60MjA<6Zc1=3kZ>m@KuSu(?x?<82xa9Q;AErED;w zyTu~iV6?Jgn<3iNnpJ<%N#3t@5v+M?f_LV56HWV`prv&-4)+-PDS%@`+qRQYcl{Ps zPLiMDcrs09FDO6)BPG!!%ryYCh&LY+fCHHupIijy~r}< zAD8;VY8TXy>PTNoj$4g2;qB1WLnb|kqx|;CVqKSEP2Q)G$ko3>H-0*IEU1)c(kVTY z`r_lC4}Zx|4D&L?cz&-I%w?enme1tvsMmYz?s-7Vb%XOItVPGO*z@VS+Mq;X`?wn| zad@{v6gqQF-fJdR$r1nVXvA$IS1)yn1(RRqAOx(Jm4L$Tg60jSnj8f!zB|eOnT8bIQ!mi=P?1()@I&* zOct9!6~1Ek-iT($r;vHIVNB2YTZDzl=fddy7Au0q&WMdte>-IKOJ&DuaJ6drr^Raq zSkvRL2AGLM;l?Wf|I;lQu4W8Gd(Xsf;xFzUnPJXkc8#igt6Ue_6d$Gp(dWA7hNTcvXf2m>OAJxw3PMweYvTYuO3RHmQD4iG7xToc z3Cu-aNpHyzn;@gG&aKumM(=??$YPP&43== z-J_r-ig=)MnPL;r{pNy1Q9Xz0%e3{Y2F%xH-%oamRD5!8rnBUXO`!d_WglwA^O5T+ z&j`muuj83ASAIr z{u6YDe`V*Ue!1pL@T%W^nMvKz_j{kLU>t<(K59S_wuR%|+)Ky=w?R{W)Ke{0_7~-M zLH)me*SifGe%&^~=+JsUaXF!|fqg(|odmZ+Mzas{^dfuhaYf0#iuWD5Jm0Fbh9lx9 z#mgB8#i@?gtf?JfKm)*VLK#vb%&2K~7E`kJE+>d|Tm2#?M$vURmUX)q*MP{t&Xn&r zw1H9M5-+%EudZwJvv6;{>k8eM))9n`ZpMEExv?=zPj-?VCwczU+eS)tX4Tm(o%A$(cELHx zsEws~$`57#yey??x~Evu^$pC7+T#DIYAk{-XL57Ys6~-j<&k3Uw=;g_(T`qMe|N)q z^eYFhI2;MZnlQV^h2kD;Q!3$Uxbwg&Lv^1C0Nw0}jjDU~KqV!eMP09zyfP}3E zraWBw0p|5R>3jwRuo!{F3UK6Apt&N%?H2yU3flg)4^4zrb!o zX2L-FSnL#odt_oX3iP&5={w%q;5NosK@TTYSW_>58GZ5!G=l~T$=SQp9ejW$&WP}Y zm~WXhg3;9&>R>%yY63ExdFBU^$YrKC86_oe?cG*zxlOz)iQYm^Rn-AB%8yG z)n*Q|^sKDch)zS&1~y$!#YhPF=XO@^P8k+tZVEYMwrZEx9uTuw=KZ0C8T!@ccZ_FP z2N+1+-OY{FDoq@HNxof9m(TFz><`am223(g%ac*nnbQAsiVe&%liM*7D405LHBwm2obNuo@ zc22Mm*Y(QtZ~PORR8~AO@)28+PyZUimthUkC!LSToCY{Ib7CBHY^r{1NIJnDBgfY= zx`L-X`^0al!b|tU>+#AQsD99(EkLuXBI@3{gn#n*?Lm$tPddz1!45S(+tbCk86;| z@OONiC{M^q_!px;8SiJ%lsIjL)a!lCOdN#8)Pv!~LHUS2o`b#ATsv*MAj{F7J7t_x z&6R9MKR|sbPdZ5AW%ReI1I;fe`@PElTKSn!dr1FhP}P>3IyHo5r(P)x>hDT>5!9de z8@Obmfb3Y%1H#H5H7*`VJ&`sXCd1H?{%tLJviJS{&D%(g%P>L!BoPBPN9u?f5`F)$ zL>5klKWQ&6f=$^p42$q#W*s(;{+do^IZ_(st_x;1?tWDkLCeab!u^2 z&)mX;Mj{%M+&n+?*7Y^o6N*_D!g#}rsnu^%2eI@#Y$Jov>ujPlUHIUkS6L#snNKjf z-hxDE<`=*n%hlb_P_d?YsUEjf_jduUZQ?EM>6LfTB(d4uazpil5mKcerzO4!-Bxuo z{Iy#@^~WRqDv+(e7S@DRT#!iyf~g%^xL4@XhTF%~IPF&5ztUL;^Y>eBA^BH-SMn3X zCz%Wvw;lr95h-09=#%J79=w4kjO|*h$FD#)HlmAB2kXI%WT5M-8`84B`9*`~aV?^(keUP2oy7TRE`)8|@aMN@)Ip5pHk3p6~-V?0sQ6JAb3BjF<&S{ zjXjelPp=iD;bfff3a{wc%yzS5?=(E*Kf3&p>$T~>P)iPx?T&6XJ|BNaN zezd57S8i|oKMI_|?J4_+-x^>lG_I+A4Y6Mk;VkTajIHiow16duH<r1S+d5iSMp8kekul)g>V zo^;v@D^e{x&wm&mF}QSwa{`DT)m-@7Y*k3*pB$p66u{J=PV-@wUSU6+w#uKk25lKZ z#$p};C*kk)f!$qvW}p!nIdS)qI45(b3ppGRl;T_H3w`IBez|%PmzqjEfk3rj5j&9qU?7EoZZQVed@X zk!v1)ZBkMU9XF8wzOUVQLi!8**Dr)t@NoO6n6JmL!;K0-sfk#B%{Z?&12<`4c8nS> znf*9K?C+YmFUKf$xF1)IgryKk3=0S5-F`TDtl5ImK z7JHbwt(v{*O{P1hA!B(*-viWjorqlo1t8)juWcx%jwqpC3tRWAMD&aoXiOcH^i2Dt zwNL2TcP<+-FM-)iYREW@)?8=lcSPuuMr`Bby?Mj(ApR@+T*x zxk!mBqt68gyA-e?{%};G_u%iAXOTu1QR{t}o47o+X*5$aA^+~Xj zdE|rod<0~m$f%wn6H;2i*_wJu*|;dr7#~^GWkTS#>OWQJxQkPM%}cbYm-^OIOa>tA z2@$K`ojV#;K`ruc`3!y)*lrJ}I`h_`OGE7LQ9L4EHxmSP zlAHuDLG}M`w)U@=X^6GW&z+rE@KOm7;gpEoQ;H?c<02}H zO`rZ6p;sp1%?|hK$^H0%7SqW5O_EiI+a^b^RepGmSN2A55JV{IHmTEQ)l21==wD7= z$zjy?I$bm+tZ5S=)Tt=1}hv&o0(7>d8$J?8pX+ob=(xmYl| zMNX)}=w{TVU5a2t)?T;=s%#T7g#M1vjfDIAboiJ=B#A!i5+mN?Z)Uj2aj_Uc8A=4D zt_t17I6ij$1rf+j8VCzE`g?Pa-kY(I4;Z+dNf>=TKuQP5^FpYsz9mPb)F4 z_~1Bs62juii;}xs+|D@46N&)&Jg1!862Ea5E1pum4~BzQzdc^Lef4g?iL{54d1)ekjGW%G=kBN{{>7f(A@AESh zws(It?_Z?=@)bg_7yUbeX&01L*zh%S-usEgU#i~kP1397xT$#5yLkk$JGVZoiYX#N6wTjDe0+<= zDqK1>Vw-h4P`qMIu3F~{|F+*ng?dDY*Sl|+4=2Nu+N&-)P25E37wk$$vOe{uxO(KH{M>V*J+k@h7LVHO8S(6?VA$k-hW- z)!xT~|B=Hhvk|a}#j5n$U({yCr>@4Ae_B2x!rfL`X5@+WS%4KUXSJV)%=3e23ErjF&4oT3II(28 zilyjGnL7?~tI|W>`%W&u4I{!zIouE5W?GxweB~Wp(eHt*s)f!)dR}T}DKljpTx+<) z*8|b#>7>(TtDBnW0?NVg16~z$qs6BM`AHb`5s}0v?SU`!14Mf!Xv}y^Rqy3b)2aD+ z)ASYatg4RdN%$2w=Q-Ckg!ysKF29e)?L-dL``(sEIcYP0&~>as#g9eVp}>}48t_H? z1MA~(zc_?~`yQW9W$(37GApS@k4#z-SnEFe?+K}W2b);VjK{0jC28YSCln-BF22zE zgp_nH@ox_;AJpDm;)eUe)c;%tSj0!oQ{t%9vY!I8 z`CQb2pzf4kV5-;i&^uRI>yg1fqc7_o{>AFQovw&o7nyMGc)AB7r&ZVo6_!^335fw~ z)5sAnp~Q~)e9Bq1=a+4uOkN}NOE{1Y_vCfB(mrr17t~cHfchXpzM(R5?&M_hhx&e$ zOWirE+dma9VRZi7yDTljKL29#gNfiUG6=A!fB43ylW@muFEb^%Ji1Mu#rR$n0vLjj z5INmooLwK>QVHbuNqpjXZB^^0P%%h0S3~~_(vPgJvfwE zI-C$7aJ6Fkz}$mN)cCf+X4{GzHOyqWIbY9zQQD+@V%=-m_OJiw3%yCjQZ6daxDmQi@*}hl#aTA3`tqL~8eA}EQe!+`xl-GoeeW0C)fbudU=5#2R`V^2@^57+drNm*UV*Rj5*M~cACs;27*+6qTy_~E*30| zqbTPqbez?^W0S3vjf+?K?rC_%0~MfS7aPr*G8R|-P5xqrWFPNTU15d_-yakO3Xz8; z%`U|aV!Rf>f~tiz^c^46G2zH_iwVV1YNBFOYngvSL5lb?pP@x|!Qe#C_6g*+vCXup z{unxfs)<~YUoR1{%b#$9SH6q_HYv#y(cg{l{xF6*V9c?>Bw9zsLERp?6A?sNrcRlYr&fNT{! z215#S8n2At+q1AYfm4`W(Yy}$mh~d&6;son5iXBEuMsdg(eI;{p-@$g<_c4!wPeMcg|MGg+Mf+#He`bkB0JK?yOdM)^>*-wWlKS~ZY^}CxTk=m+XrO$n#1d=KGAr}9DCGkX;n~cL zDF_M`q}N)K*tu=;Ql^lX zO*^w2q1caWX{NV__76zh{g^m~!e8g0&Hz?ehw%qmeXyNd__k&6?lVBMAa0kN*EhiI zpV!hPN9g_O#4Ler5x`s`H_I5B=SR3S__#%4d>}|w-d=}`106ZJ&79qO^#Bw(aY5(@ zlYQ2IgyOv?m+6l9M;z1+ZVL4ZQ{w|W<#}}aj4w=WsRr8RIQaDVnCA9D43tHkbL3k4 z;|=i2qNv#;+V5c53oBEHLZqzPM6q$q<%%>|)YTSBwyOQpa2~k#0B^=$*6Ecsd)A@H zooc@Zf(Cih68~7LNXlQGd#rI1Y6$OK0WXyEme7s_~KB=N(@`=eq zzYzbu6%3r{x=!p*KBPpr{HM0l*ai} zIxOfWkwl%ElRH83T3A`>vmc#b!nyf{lq5%H&TrdyJUsw4Ee!D-Tup-xPcGXTedy4F z(0kWp`JG`F|4X#LfZ|&C;M-!NT73?xfx8r6*~#ugJdlTu4!Z`BzWNW6BWx$6o$~do zE|WX!1JB6Cv9LquJ(T;m`4)zM|_~#mMGJ9-mtW|6|hKTHr%Hto$LtM zWBQKu>YUsZ`Yw!&wpS-`McjX&(H-FFJ;&>Ek4jfnW5uw-w*itk2b|cF0lDaJfdw%V z%~i3H6ElH(7ksuKS7q|$zRM?_HS1w{H8Jq#?ox&4Y2=MXW{gTLqX%|^YkkjN8)Xzn zdJ^0v->S)oF7P@fISIWJ?elf+!@{~xNlvO4L%A!RV2^yrFsCmz0~(dRlIx0$vj7N; zY0L{0g(DNz(#hH{%tN{iy6R1%)mkXY;{KG{pFb-5)GEh-GQkB& zx&jfu4{V*Sq0T>=vO*@m0HP=z#acB z8oz&y6p;m_qRm}oBbdH4zY!Y$nYY$k?%Jj|%D9^)ve{6?8kS`Cp^ZCRvA%=!Vy00% zDZv3S2Rs|n7>;YC9E6oe??-Fxs!60 zScE(sGsk9_2mj%jxlN zbW2lKz;~6dD|f0L0x90?TAa@#HIO}^k+wCg*c>U3ZEgBHF)H3feO>v+(QSO2S~u^1 zYs8J~A}?>J>1gbM*!O6fstcv<%L_3U)Gj*$xuF6UEZJ{DL|iH_p%{q@D~3M2gg`)O zN3$d|mW$L20ge~zrvi)yn1~9?K(440E-ixG4nGO=1(=1%MEs=To>gTc-xklF3>>Oq zgB}t{aG>7ijI!&(v|Q)k>myRckXq5Ez!c;6s)l%{Bb1KB5EbJ1LTlEou5<0qe^zW* z2y(S2+u@mxs|;JFiHI!c8{@bn-M_YNmm4owka3EM0Vm4!z`IxFsV}<}tlY`OfmJ-}<4~#C%tg;VaDcg59pyYHfOUbyeuwzW2E zg;`@kNfyAW`lD>(0L{&@pyFn5ZaUdH?UF*U%iPiT+>#CbVtCU(5xM}8kLy>Xw5G?q+ z=hO$LJ~osa53#is%tIu3edF+91j7VN?>*qAOxmKQg(9{Ictd=sJTJjQv@%#~2@L$> za1La6{QAgPc2VKmK*MVX4t$^}Qfp59-LdlIj$clfI{(N(P}{q0G2-=R%^NkAs2Amf zl~#C*TE*=jG~6a$&lk?M9!@0UNjt#QZgC?2d+U~t#*9ADb3~CYbKzcXOM> zkA2Oy&4tvpKWYL7{IsB&eO7?QB#NClnuE^Q+rE>oM^gZnhG&}tH=n_eU>w&F5r|jN z&)S>4Rajd^21HicpWl%WGY?(S-W24olWU%oE}Le>3JaF#^CRUWn17ykjP<}#-(#X5 zZ6CjX!J9rILttow&%CTmd4RsEHE>Gemslzw7k;P;)7p)oVksZ4oj z6yqEBqrS+rd_4c)?~QA-%jKAfUd@KIRhb3jdR@(!=wy_c35+n#+jzHDU#~B@SG8(m zIqHsHlTp=pNh*J=!I4`heW_`Mf#t*|lSZje!&0PfrM^J<22=3Ub9ueM)Z0O2WjVkv PrbyIPG#^(enFsw3U`DIs literal 0 HcmV?d00001 diff --git a/bin/node/cli/browser-demo/index.html b/bin/node/cli/browser-demo/index.html new file mode 100644 index 0000000000000..cf107e64568f2 --- /dev/null +++ b/bin/node/cli/browser-demo/index.html @@ -0,0 +1,39 @@ + + + + + Substrate node + + + + + diff --git a/bin/node/cli/browser-demo/ws.js b/bin/node/cli/browser-demo/ws.js new file mode 100644 index 0000000000000..fa7a499a8a7a3 --- /dev/null +++ b/bin/node/cli/browser-demo/ws.js @@ -0,0 +1,148 @@ +// Copyright 2019 Parity Technologies (UK) Ltd. +// This file is part of Substrate. + +// Substrate is free software: you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. + +// Substrate is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. + +// You should have received a copy of the GNU General Public License +// along with Substrate. If not, see . + +export default () => { + return { + dial: dial, + listen_on: (addr) => { + let err = new Error("Listening on WebSockets is not possible from within a browser"); + err.name = "NotSupportedError"; + throw err; + }, + }; +} + +/// Turns a string multiaddress into a WebSockets string URL. +// TODO: support dns addresses as well +const multiaddr_to_ws = (addr) => { + let parsed = addr.match(/^\/(ip4|ip6|dns4|dns6)\/(.*?)\/tcp\/(.*?)\/(ws|wss|x-parity-ws\/(.*)|x-parity-wss\/(.*))$/); + let proto = 'wss'; + if (parsed[4] == 'ws' || parsed[4] == 'x-parity-ws') { + proto = 'ws'; + } + let url = decodeURIComponent(parsed[5] || parsed[6] || ''); + if (parsed != null) { + if (parsed[1] == 'ip6') { + return proto + "://[" + parsed[2] + "]:" + parsed[3] + url; + } else { + return proto + "://" + parsed[2] + ":" + parsed[3] + url; + } + } + + let err = new Error("Address not supported: " + addr); + err.name = "NotSupportedError"; + throw err; +} + +// Attempt to dial a multiaddress. +const dial = (addr) => { + let ws = new WebSocket(multiaddr_to_ws(addr)); + let reader = read_queue(); + + return new Promise((resolve, reject) => { + // TODO: handle ws.onerror properly after dialing has happened + ws.onerror = (ev) => reject(ev); + ws.onmessage = (ev) => reader.inject_blob(ev.data); + ws.onclose = () => reader.inject_eof(); + ws.onopen = () => resolve({ + read: (function*() { while(ws.readyState == 1) { yield reader.next(); } })(), + write: (data) => { + if (ws.readyState == 1) { + ws.send(data); + return promise_when_ws_finished(ws); + } else { + return Promise.reject("WebSocket is closed"); + } + }, + shutdown: () => {}, + close: () => ws.close() + }); + }); +} + +// Takes a WebSocket object and returns a Promise that resolves when bufferedAmount is 0. +const promise_when_ws_finished = (ws) => { + if (ws.bufferedAmount == 0) { + return Promise.resolve(); + } + + return new Promise((resolve, reject) => { + setTimeout(function check() { + if (ws.bufferedAmount == 0) { + resolve(); + } else { + setTimeout(check, 100); + } + }, 2); + }) +} + +// Creates a queue reading system. +const read_queue = () => { + // State of the queue. + let state = { + // Array of promises resolving to `ArrayBuffer`s, that haven't been transmitted back with + // `next` yet. + queue: new Array(), + // If `resolve` isn't null, it is a "resolve" function of a promise that has already been + // returned by `next`. It should be called with some data. + resolve: null, + }; + + return { + // Inserts a new Blob in the queue. + inject_blob: (blob) => { + if (state.resolve != null) { + var resolve = state.resolve; + state.resolve = null; + + var reader = new FileReader(); + reader.addEventListener("loadend", () => resolve(reader.result)); + reader.readAsArrayBuffer(blob); + } else { + state.queue.push(new Promise((resolve, reject) => { + var reader = new FileReader(); + reader.addEventListener("loadend", () => resolve(reader.result)); + reader.readAsArrayBuffer(blob); + })); + } + }, + + // Inserts an EOF message in the queue. + inject_eof: () => { + if (state.resolve != null) { + var resolve = state.resolve; + state.resolve = null; + resolve(null); + } else { + state.queue.push(Promise.resolve(null)); + } + }, + + // Returns a Promise that yields the next entry as an ArrayBuffer. + next: () => { + if (state.queue.length != 0) { + return state.queue.shift(0); + } else { + if (state.resolve !== null) + throw "Internal error: already have a pending promise"; + return new Promise((resolve, reject) => { + state.resolve = resolve; + }); + } + } + }; +}; diff --git a/node/cli/build.rs b/bin/node/cli/build.rs similarity index 90% rename from node/cli/build.rs rename to bin/node/cli/build.rs index 48b57c4600e80..ae936ce4fbace 100644 --- a/node/cli/build.rs +++ b/bin/node/cli/build.rs @@ -14,16 +14,16 @@ // You should have received a copy of the GNU General Public License // along with Substrate. If not, see . -use cli::{NoCustom, CoreParams}; use std::{fs, env, path::Path}; use structopt::{StructOpt, clap::Shell}; +use substrate_cli::{NoCustom, CoreParams}; use vergen::{ConstantsFlags, generate_cargo_keys}; fn main() { build_shell_completion(); - generate_cargo_keys(ConstantsFlags::all()) - .expect("Failed to generate metadata files"); - println!("cargo:rerun-if-changed=.git/HEAD"); + generate_cargo_keys(ConstantsFlags::all()).expect("Failed to generate metadata files"); + + build_script_utils::rerun_if_git_head_changed(); } /// Build shell completion scripts for all known shells diff --git a/node/cli/doc/shell-completion.adoc b/bin/node/cli/doc/shell-completion.adoc similarity index 100% rename from node/cli/doc/shell-completion.adoc rename to bin/node/cli/doc/shell-completion.adoc diff --git a/node/cli/res/flaming-fir.json b/bin/node/cli/res/flaming-fir.json similarity index 100% rename from node/cli/res/flaming-fir.json rename to bin/node/cli/res/flaming-fir.json diff --git a/bin/node/cli/src/browser.rs b/bin/node/cli/src/browser.rs new file mode 100644 index 0000000000000..ada8a52e1e8b3 --- /dev/null +++ b/bin/node/cli/src/browser.rs @@ -0,0 +1,160 @@ +// Copyright 2019 Parity Technologies (UK) Ltd. +// This file is part of Substrate. + +// Substrate is free software: you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. + +// Substrate is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. + +// You should have received a copy of the GNU General Public License +// along with Substrate. If not, see . + +use crate::ChainSpec; +use futures::{prelude::*, sync::oneshot, sync::mpsc}; +use libp2p::wasm_ext; +use log::{debug, info}; +use std::sync::Arc; +use substrate_service::{AbstractService, RpcSession, Roles as ServiceRoles, Configuration, config::DatabaseConfig}; +use wasm_bindgen::prelude::*; + +/// Starts the client. +/// +/// You must pass a libp2p transport that supports . +#[wasm_bindgen] +pub fn start_client(wasm_ext: wasm_ext::ffi::Transport) -> Result { + start_inner(wasm_ext) + .map_err(|err| JsValue::from_str(&err.to_string())) +} + +fn start_inner(wasm_ext: wasm_ext::ffi::Transport) -> Result> { + console_error_panic_hook::set_once(); + console_log::init_with_level(log::Level::Info); + + // Build the configuration to pass to the service. + let config = { + let wasm_ext = wasm_ext::ExtTransport::new(wasm_ext); + let chain_spec = ChainSpec::FlamingFir.load().map_err(|e| format!("{:?}", e))?; + let mut config = Configuration::<(), _, _>::default_with_spec_and_base_path(chain_spec, None); + config.network.transport = network::config::TransportConfig::Normal { + wasm_external_transport: Some(wasm_ext.clone()), + allow_private_ipv4: true, + enable_mdns: false, + }; + config.telemetry_external_transport = Some(wasm_ext); + config.roles = ServiceRoles::LIGHT; + config.name = "Browser node".to_string(); + config.database = { + let db = Arc::new(kvdb_memorydb::create(10)); + DatabaseConfig::Custom(db) + }; + config + }; + + info!("Substrate browser node"); + info!(" version {}", config.full_version()); + info!(" by Parity Technologies, 2017-2019"); + info!("Chain specification: {}", config.chain_spec.name()); + info!("Node name: {}", config.name); + info!("Roles: {:?}", config.roles); + + // Create the service. This is the most heavy initialization step. + let mut service = crate::service::new_light(config).map_err(|e| format!("{:?}", e))?; + + // We now dispatch a background task responsible for processing the service. + // + // The main action performed by the code below consists in polling the service with + // `service.poll()`. + // The rest consists in handling RPC requests. + let (rpc_send_tx, mut rpc_send_rx) = mpsc::unbounded::(); + wasm_bindgen_futures::spawn_local(futures::future::poll_fn(move || { + loop { + match rpc_send_rx.poll() { + Ok(Async::Ready(Some(message))) => { + let fut = service.rpc_query(&message.session, &message.rpc_json); + let _ = message.send_back.send(Box::new(fut)); + }, + Ok(Async::NotReady) => break, + Err(_) | Ok(Async::Ready(None)) => return Ok(Async::Ready(())), + } + } + + loop { + match service.poll().map_err(|_| ())? { + Async::Ready(()) => return Ok(Async::Ready(())), + Async::NotReady => break + } + } + + Ok(Async::NotReady) + })); + + Ok(Client { + rpc_send_tx, + }) +} + +/// A running client. +#[wasm_bindgen] +pub struct Client { + rpc_send_tx: mpsc::UnboundedSender, +} + +struct RpcMessage { + rpc_json: String, + session: RpcSession, + send_back: oneshot::Sender, Error = ()>>>, +} + +#[wasm_bindgen] +impl Client { + /// Allows starting an RPC request. Returns a `Promise` containing the result of that request. + #[wasm_bindgen(js_name = "rpcSend")] + pub fn rpc_send(&mut self, rpc: &str) -> js_sys::Promise { + let rpc_session = RpcSession::new(mpsc::channel(1).0); + let (tx, rx) = oneshot::channel(); + let _ = self.rpc_send_tx.unbounded_send(RpcMessage { + rpc_json: rpc.to_owned(), + session: rpc_session, + send_back: tx, + }); + let fut = rx + .map_err(|_| ()) + .and_then(|fut| fut) + .map(|s| JsValue::from_str(&s.unwrap_or(String::new()))) + .map_err(|_| JsValue::NULL); + wasm_bindgen_futures::future_to_promise(fut) + } + + /// Subscribes to an RPC pubsub endpoint. + #[wasm_bindgen(js_name = "rpcSubscribe")] + pub fn rpc_subscribe(&mut self, rpc: &str, callback: js_sys::Function) { + let (tx, rx) = mpsc::channel(4); + let rpc_session = RpcSession::new(tx); + let (fut_tx, fut_rx) = oneshot::channel(); + let _ = self.rpc_send_tx.unbounded_send(RpcMessage { + rpc_json: rpc.to_owned(), + session: rpc_session.clone(), + send_back: fut_tx, + }); + let fut_rx = fut_rx + .map_err(|_| ()) + .and_then(|fut| fut); + wasm_bindgen_futures::spawn_local(fut_rx.then(|_| Ok(()))); + wasm_bindgen_futures::spawn_local(rx.for_each(move |s| { + match callback.call1(&callback, &JsValue::from_str(&s)) { + Ok(_) => Ok(()), + Err(_) => Err(()), + } + }).then(move |v| { + // We need to keep `rpc_session` alive. + debug!("RPC subscription has ended"); + drop(rpc_session); + v + })); + } +} diff --git a/node/cli/src/chain_spec.rs b/bin/node/cli/src/chain_spec.rs similarity index 88% rename from node/cli/src/chain_spec.rs rename to bin/node/cli/src/chain_spec.rs index 04fb41c211009..792dd237ac4dc 100644 --- a/node/cli/src/chain_spec.rs +++ b/bin/node/cli/src/chain_spec.rs @@ -20,9 +20,9 @@ use chain_spec::ChainSpecExtension; use primitives::{Pair, Public, crypto::UncheckedInto, sr25519}; use serde::{Serialize, Deserialize}; use node_runtime::{ - BabeConfig, BalancesConfig, ContractsConfig, CouncilConfig, DemocracyConfig, GrandpaConfig, - ImOnlineConfig, IndicesConfig, SessionConfig, SessionKeys, StakerStatus, StakingConfig, - SudoConfig, SystemConfig, TechnicalCommitteeConfig, WASM_BINARY, + AuthorityDiscoveryConfig, BabeConfig, BalancesConfig, ContractsConfig, CouncilConfig, DemocracyConfig, + GrandpaConfig, ImOnlineConfig, IndicesConfig, SessionConfig, SessionKeys, StakerStatus, StakingConfig, SudoConfig, + SystemConfig, TechnicalCommitteeConfig, WASM_BINARY, }; use node_runtime::Block; use node_runtime::constants::currency::*; @@ -32,6 +32,7 @@ use substrate_telemetry::TelemetryEndpoints; use grandpa_primitives::{AuthorityId as GrandpaId}; use babe_primitives::{AuthorityId as BabeId}; use im_online::sr25519::{AuthorityId as ImOnlineId}; +use authority_discovery_primitives::AuthorityId as AuthorityDiscoveryId; use sr_primitives::{Perbill, traits::{Verify, IdentifyAccount}}; pub use node_primitives::{AccountId, Balance, Signature}; @@ -61,8 +62,13 @@ pub fn flaming_fir_config() -> Result { ChainSpec::from_json_bytes(&include_bytes!("../res/flaming-fir.json")[..]) } -fn session_keys(grandpa: GrandpaId, babe: BabeId, im_online: ImOnlineId) -> SessionKeys { - SessionKeys { grandpa, babe, im_online, } +fn session_keys( + grandpa: GrandpaId, + babe: BabeId, + im_online: ImOnlineId, + authority_discovery: AuthorityDiscoveryId, +) -> SessionKeys { + SessionKeys { grandpa, babe, im_online, authority_discovery } } fn staging_testnet_config_genesis() -> GenesisConfig { @@ -72,7 +78,7 @@ fn staging_testnet_config_genesis() -> GenesisConfig { // and // for i in 1 2 3 4 ; do for j in session; do subkey --ed25519 inspect "$secret"//fir//$j//$i; done; done - let initial_authorities: Vec<(AccountId, AccountId, GrandpaId, BabeId, ImOnlineId)> = vec![( + let initial_authorities: Vec<(AccountId, AccountId, GrandpaId, BabeId, ImOnlineId, AuthorityDiscoveryId)> = vec![( // 5Fbsd6WXDGiLTxunqeK5BATNiocfCqu9bS1yArVjCgeBLkVy hex!["9c7a2ee14e565db0c69f78c7b4cd839fbf52b607d867e9e9c5a79042898a0d12"].into(), // 5EnCiV7wSHeNhjW3FSUwiJNkcc2SBkPLn5Nj93FmbLtBjQUq @@ -83,6 +89,8 @@ fn staging_testnet_config_genesis() -> GenesisConfig { hex!["6e7e4eb42cbd2e0ab4cae8708ce5509580b8c04d11f6758dbf686d50fe9f9106"].unchecked_into(), // 5EZaeQ8djPcq9pheJUhgerXQZt9YaHnMJpiHMRhwQeinqUW8 hex!["6e7e4eb42cbd2e0ab4cae8708ce5509580b8c04d11f6758dbf686d50fe9f9106"].unchecked_into(), + // 5EZaeQ8djPcq9pheJUhgerXQZt9YaHnMJpiHMRhwQeinqUW8 + hex!["6e7e4eb42cbd2e0ab4cae8708ce5509580b8c04d11f6758dbf686d50fe9f9106"].unchecked_into(), ),( // 5ERawXCzCWkjVq3xz1W5KGNtVx2VdefvZ62Bw1FEuZW4Vny2 hex!["68655684472b743e456907b398d3a44c113f189e56d1bbfd55e889e295dfde78"].into(), @@ -94,6 +102,8 @@ fn staging_testnet_config_genesis() -> GenesisConfig { hex!["482dbd7297a39fa145c570552249c2ca9dd47e281f0c500c971b59c9dcdcd82e"].unchecked_into(), // 5DhLtiaQd1L1LU9jaNeeu9HJkP6eyg3BwXA7iNMzKm7qqruQ hex!["482dbd7297a39fa145c570552249c2ca9dd47e281f0c500c971b59c9dcdcd82e"].unchecked_into(), + // 5DhLtiaQd1L1LU9jaNeeu9HJkP6eyg3BwXA7iNMzKm7qqruQ + hex!["482dbd7297a39fa145c570552249c2ca9dd47e281f0c500c971b59c9dcdcd82e"].unchecked_into(), ),( // 5DyVtKWPidondEu8iHZgi6Ffv9yrJJ1NDNLom3X9cTDi98qp hex!["547ff0ab649283a7ae01dbc2eb73932eba2fb09075e9485ff369082a2ff38d65"].into(), @@ -105,6 +115,8 @@ fn staging_testnet_config_genesis() -> GenesisConfig { hex!["482a3389a6cf42d8ed83888cfd920fec738ea30f97e44699ada7323f08c3380a"].unchecked_into(), // 5DhKqkHRkndJu8vq7pi2Q5S3DfftWJHGxbEUNH43b46qNspH hex!["482a3389a6cf42d8ed83888cfd920fec738ea30f97e44699ada7323f08c3380a"].unchecked_into(), + // 5DhKqkHRkndJu8vq7pi2Q5S3DfftWJHGxbEUNH43b46qNspH + hex!["482a3389a6cf42d8ed83888cfd920fec738ea30f97e44699ada7323f08c3380a"].unchecked_into(), ),( // 5HYZnKWe5FVZQ33ZRJK1rG3WaLMztxWrrNDb1JRwaHHVWyP9 hex!["f26cdb14b5aec7b2789fd5ca80f979cef3761897ae1f37ffb3e154cbcc1c2663"].into(), @@ -116,6 +128,8 @@ fn staging_testnet_config_genesis() -> GenesisConfig { hex!["00299981a2b92f878baaf5dbeba5c18d4e70f2a1fcd9c61b32ea18daf38f4378"].unchecked_into(), // 5C4vDQxA8LTck2xJEy4Yg1hM9qjDt4LvTQaMo4Y8ne43aU6x hex!["00299981a2b92f878baaf5dbeba5c18d4e70f2a1fcd9c61b32ea18daf38f4378"].unchecked_into(), + // 5C4vDQxA8LTck2xJEy4Yg1hM9qjDt4LvTQaMo4Y8ne43aU6x + hex!["00299981a2b92f878baaf5dbeba5c18d4e70f2a1fcd9c61b32ea18daf38f4378"].unchecked_into(), )]; // generated with secret: subkey inspect "$secret"/fir @@ -164,19 +178,27 @@ pub fn get_account_id_from_seed(seed: &str) -> AccountId where } /// Helper function to generate stash, controller and session key from seed -pub fn get_authority_keys_from_seed(seed: &str) -> (AccountId, AccountId, GrandpaId, BabeId, ImOnlineId) { +pub fn get_authority_keys_from_seed(seed: &str) -> ( + AccountId, + AccountId, + GrandpaId, + BabeId, + ImOnlineId, + AuthorityDiscoveryId, +) { ( get_account_id_from_seed::(&format!("{}//stash", seed)), get_account_id_from_seed::(seed), get_from_seed::(seed), get_from_seed::(seed), get_from_seed::(seed), + get_from_seed::(seed), ) } /// Helper function to create GenesisConfig for testing pub fn testnet_genesis( - initial_authorities: Vec<(AccountId, AccountId, GrandpaId, BabeId, ImOnlineId)>, + initial_authorities: Vec<(AccountId, AccountId, GrandpaId, BabeId, ImOnlineId, AuthorityDiscoveryId)>, root_key: AccountId, endowed_accounts: Option>, enable_println: bool, @@ -220,7 +242,7 @@ pub fn testnet_genesis( }), session: Some(SessionConfig { keys: initial_authorities.iter().map(|x| { - (x.0.clone(), session_keys(x.2.clone(), x.3.clone(), x.4.clone())) + (x.0.clone(), session_keys(x.2.clone(), x.3.clone(), x.4.clone(), x.5.clone())) }).collect::>(), }), staking: Some(StakingConfig { @@ -259,6 +281,9 @@ pub fn testnet_genesis( im_online: Some(ImOnlineConfig { keys: vec![], }), + authority_discovery: Some(AuthorityDiscoveryConfig { + keys: vec![], + }), grandpa: Some(GrandpaConfig { authorities: vec![], }), diff --git a/node/cli/src/lib.rs b/bin/node/cli/src/cli.rs similarity index 78% rename from node/cli/src/lib.rs rename to bin/node/cli/src/cli.rs index 5125ba216a2cc..4bddb50b4bcac 100644 --- a/node/cli/src/lib.rs +++ b/bin/node/cli/src/cli.rs @@ -14,41 +14,19 @@ // You should have received a copy of the GNU General Public License // along with Substrate. If not, see . -//! Substrate CLI library. - -#![warn(missing_docs)] -#![warn(unused_extern_crates)] - -pub use cli::error; -pub mod chain_spec; -#[macro_use] -mod service; -mod factory_impl; - +pub use substrate_cli::error; use tokio::prelude::Future; use tokio::runtime::{Builder as RuntimeBuilder, Runtime}; -pub use cli::{VersionInfo, IntoExit, NoCustom, SharedParams, ExecutionStrategyParam}; +pub use substrate_cli::{VersionInfo, IntoExit, NoCustom, SharedParams, ExecutionStrategyParam}; use substrate_service::{AbstractService, Roles as ServiceRoles, Configuration}; use log::info; use structopt::{StructOpt, clap::App}; -use cli::{display_role, parse_and_prepare, AugmentClap, GetLogFilter, ParseAndPrepare}; +use substrate_cli::{display_role, parse_and_prepare, AugmentClap, GetLogFilter, ParseAndPrepare}; +use crate::{service, ChainSpec, load_spec}; use crate::factory_impl::FactoryState; use transaction_factory::RuntimeAdapter; use client::ExecutionStrategies; -/// The chain specification option. -#[derive(Clone, Debug, PartialEq)] -pub enum ChainSpec { - /// Whatever the current runtime is, with just Alice as an auth. - Development, - /// Whatever the current runtime is, with simple Alice/Bob auths. - LocalTestnet, - /// The Flaming Fir testnet. - FlamingFir, - /// Whatever the current runtime is with the "global testnet" defaults. - StagingTestnet, -} - /// Custom subcommands. #[derive(Clone, Debug, StructOpt)] pub enum CustomSubcommands { @@ -121,37 +99,8 @@ impl AugmentClap for FactoryCmd { } } -/// Get a chain config from a spec setting. -impl ChainSpec { - pub(crate) fn load(self) -> Result { - Ok(match self { - ChainSpec::FlamingFir => chain_spec::flaming_fir_config()?, - ChainSpec::Development => chain_spec::development_config(), - ChainSpec::LocalTestnet => chain_spec::local_testnet_config(), - ChainSpec::StagingTestnet => chain_spec::staging_testnet_config(), - }) - } - - pub(crate) fn from(s: &str) -> Option { - match s { - "dev" => Some(ChainSpec::Development), - "local" => Some(ChainSpec::LocalTestnet), - "" | "fir" | "flaming-fir" => Some(ChainSpec::FlamingFir), - "staging" => Some(ChainSpec::StagingTestnet), - _ => None, - } - } -} - -fn load_spec(id: &str) -> Result, String> { - Ok(match ChainSpec::from(id) { - Some(spec) => Some(spec.load()?), - None => None, - }) -} - /// Parse command line arguments into service configuration. -pub fn run(args: I, exit: E, version: cli::VersionInfo) -> error::Result<()> where +pub fn run(args: I, exit: E, version: substrate_cli::VersionInfo) -> error::Result<()> where I: IntoIterator, T: Into + Clone, E: IntoExit, @@ -182,7 +131,7 @@ pub fn run(args: I, exit: E, version: cli::VersionInfo) -> error::Resul ), } }), - ParseAndPrepare::BuildSpec(cmd) => cmd.run(load_spec), + ParseAndPrepare::BuildSpec(cmd) => cmd.run::(load_spec), ParseAndPrepare::ExportBlocks(cmd) => cmd.run_with_builder(|config: Config<_, _>| Ok(new_full_start!(config).0), load_spec, exit), ParseAndPrepare::ImportBlocks(cmd) => cmd.run_with_builder(|config: Config<_, _>| @@ -191,7 +140,7 @@ pub fn run(args: I, exit: E, version: cli::VersionInfo) -> error::Resul ParseAndPrepare::RevertChain(cmd) => cmd.run_with_builder(|config: Config<_, _>| Ok(new_full_start!(config).0), load_spec), ParseAndPrepare::CustomCommand(CustomSubcommands::Factory(cli_args)) => { - let mut config: Config<_, _> = cli::create_config_with_db_path( + let mut config: Config<_, _> = substrate_cli::create_config_with_db_path( load_spec, &cli_args.shared_params, &version, @@ -238,7 +187,7 @@ where { let (exit_send, exit) = exit_future::signal(); - let informant = cli::informant::build(&service); + let informant = substrate_cli::informant::build(&service); runtime.executor().spawn(exit.until(informant).map(|_| ())); // we eagerly drop the service so that the internal exit future is fired, diff --git a/node/cli/src/factory_impl.rs b/bin/node/cli/src/factory_impl.rs similarity index 99% rename from node/cli/src/factory_impl.rs rename to bin/node/cli/src/factory_impl.rs index 48fb7b237f1e6..aaf74a0602082 100644 --- a/node/cli/src/factory_impl.rs +++ b/bin/node/cli/src/factory_impl.rs @@ -247,7 +247,7 @@ fn sign( let payload = (xt.function, extra.clone(), additional_signed); let signature = payload.using_encoded(|b| { if b.len() > 256 { - key.sign(&sr_io::blake2_256(b)) + key.sign(&runtime_io::hashing::blake2_256(b)) } else { key.sign(b) } diff --git a/bin/node/cli/src/lib.rs b/bin/node/cli/src/lib.rs new file mode 100644 index 0000000000000..78660ae92e3c5 --- /dev/null +++ b/bin/node/cli/src/lib.rs @@ -0,0 +1,88 @@ +// Copyright 2018-2019 Parity Technologies (UK) Ltd. +// This file is part of Substrate. + +// Substrate is free software: you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. + +// Substrate is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. + +// You should have received a copy of the GNU General Public License +// along with Substrate. If not, see . + +//! Substrate CLI library. +//! +//! This package has two Cargo features: +//! +//! - `cli` (default): exposes functions that parse command-line options, then start and run the +//! node as a CLI application. +//! +//! - `browser`: exposes the content of the `browser` module, which consists of exported symbols +//! that are meant to be passed through the `wasm-bindgen` utility and called from JavaScript. +//! Despite its name the produced WASM can theoretically also be used from NodeJS, although this +//! hasn't been tested. + +#![warn(missing_docs)] +#![warn(unused_extern_crates)] + +pub mod chain_spec; + +#[macro_use] +mod service; +#[cfg(feature = "browser")] +mod browser; +#[cfg(feature = "cli")] +mod cli; +#[cfg(feature = "cli")] +mod factory_impl; + +#[cfg(feature = "browser")] +pub use browser::*; +#[cfg(feature = "cli")] +pub use cli::*; + +/// The chain specification option. +#[derive(Clone, Debug, PartialEq)] +pub enum ChainSpec { + /// Whatever the current runtime is, with just Alice as an auth. + Development, + /// Whatever the current runtime is, with simple Alice/Bob auths. + LocalTestnet, + /// The Flaming Fir testnet. + FlamingFir, + /// Whatever the current runtime is with the "global testnet" defaults. + StagingTestnet, +} + +/// Get a chain config from a spec setting. +impl ChainSpec { + pub(crate) fn load(self) -> Result { + Ok(match self { + ChainSpec::FlamingFir => chain_spec::flaming_fir_config()?, + ChainSpec::Development => chain_spec::development_config(), + ChainSpec::LocalTestnet => chain_spec::local_testnet_config(), + ChainSpec::StagingTestnet => chain_spec::staging_testnet_config(), + }) + } + + pub(crate) fn from(s: &str) -> Option { + match s { + "dev" => Some(ChainSpec::Development), + "local" => Some(ChainSpec::LocalTestnet), + "" | "fir" | "flaming-fir" => Some(ChainSpec::FlamingFir), + "staging" => Some(ChainSpec::StagingTestnet), + _ => None, + } + } +} + +fn load_spec(id: &str) -> Result, String> { + Ok(match ChainSpec::from(id) { + Some(spec) => Some(spec.load()?), + None => None, + }) +} diff --git a/node/cli/src/service.rs b/bin/node/cli/src/service.rs similarity index 94% rename from node/cli/src/service.rs rename to bin/node/cli/src/service.rs index 47e36bd926275..f4c0ad69f1427 100644 --- a/node/cli/src/service.rs +++ b/bin/node/cli/src/service.rs @@ -69,10 +69,11 @@ macro_rules! new_full_start { .with_import_queue(|_config, client, mut select_chain, _transaction_pool| { let select_chain = select_chain.take() .ok_or_else(|| substrate_service::Error::SelectChainRequired)?; - let (grandpa_block_import, grandpa_link) = - grandpa::block_import::<_, _, _, node_runtime::RuntimeApi, _, _>( - client.clone(), &*client, select_chain - )?; + let (grandpa_block_import, grandpa_link) = grandpa::block_import( + client.clone(), + &*client, + select_chain, + )?; let justification_import = grandpa_block_import.clone(); let (block_import, babe_link) = babe::block_import( @@ -95,7 +96,7 @@ macro_rules! new_full_start { import_setup = Some((block_import, grandpa_link, babe_link)); Ok(import_queue) })? - .with_rpc_extensions(|client, pool| -> RpcExtension { + .with_rpc_extensions(|client, pool, _backend| -> RpcExtension { node_rpc::create(client, pool) })?; @@ -111,6 +112,11 @@ macro_rules! new_full { ($config:expr, $with_startup_data: expr) => {{ use futures::sync::mpsc; use network::DhtEvent; + use futures03::{ + compat::Stream01CompatExt, + stream::StreamExt, + future::{FutureExt, TryFutureExt}, + }; let ( is_authority, @@ -135,7 +141,7 @@ macro_rules! new_full { // back-pressure. Authority discovery is triggering one event per authority within the current authority set. // This estimates the authority set size to be somewhere below 10 000 thereby setting the channel buffer size to // 10 000. - let (dht_event_tx, _dht_event_rx) = + let (dht_event_tx, dht_event_rx) = mpsc::channel::(10_000); let service = builder.with_network_protocol(|_| Ok(crate::service::NodeProtocol::new()))? @@ -174,6 +180,19 @@ macro_rules! new_full { let babe = babe::start_babe(babe_config)?; service.spawn_essential_task(babe); + + let future03_dht_event_rx = dht_event_rx.compat() + .map(|x| x.expect(" never returns an error; qed")) + .boxed(); + let authority_discovery = authority_discovery::AuthorityDiscovery::new( + service.client(), + service.network(), + service.keystore(), + future03_dht_event_rx, + ); + let future01_authority_discovery = authority_discovery.map(|x| Ok(x)).compat(); + + service.spawn_task(future01_authority_discovery); } // if the node isn't actively participating in consensus then it doesn't @@ -264,7 +283,7 @@ pub fn new_full(config: NodeConfiguration) TransactionPool>, OffchainWorkers< ConcreteClient, - >::OffchainStorage, + >::OffchainStorage, ConcreteBlock, > >, @@ -291,8 +310,11 @@ pub fn new_light(config: NodeConfiguration) let fetch_checker = fetcher .map(|fetcher| fetcher.checker().clone()) .ok_or_else(|| "Trying to start light import queue without active fetch checker")?; - let grandpa_block_import = grandpa::light_block_import::<_, _, _, RuntimeApi, _>( - client.clone(), backend, Arc::new(fetch_checker), client.clone() + let grandpa_block_import = grandpa::light_block_import::<_, _, _, RuntimeApi>( + client.clone(), + backend, + &*client, + Arc::new(fetch_checker), )?; let finality_proof_import = grandpa_block_import.clone(); @@ -322,7 +344,7 @@ pub fn new_light(config: NodeConfiguration) .with_finality_proof_provider(|client, backend| Ok(Arc::new(GrandpaFinalityProofProvider::new(backend, client)) as _) )? - .with_rpc_extensions(|client, pool| -> RpcExtension { + .with_rpc_extensions(|client, pool, _backend| -> RpcExtension { node_rpc::create(client, pool) })? .build()?; @@ -386,7 +408,7 @@ mod tests { origin: BlockOrigin::File, justification: Vec::new(), internal_justification: Vec::new(), - finalized: true, + finalized: false, body: Some(block.extrinsics), header: block.header, auxiliary: Vec::new(), @@ -516,9 +538,10 @@ mod tests { justification: None, post_digests: vec![item], body: Some(new_body), - finalized: true, + finalized: false, auxiliary: Vec::new(), fork_choice: ForkChoiceStrategy::LongestChain, + allow_missing_state: false, }; block_import.import_block(params, Default::default()) diff --git a/bin/node/executor/Cargo.toml b/bin/node/executor/Cargo.toml new file mode 100644 index 0000000000000..5526b9691c07d --- /dev/null +++ b/bin/node/executor/Cargo.toml @@ -0,0 +1,48 @@ +[package] +name = "node-executor" +version = "2.0.0" +authors = ["Parity Technologies "] +description = "Substrate node implementation in Rust." +edition = "2018" + +[dependencies] +trie-root = "0.15.2" +codec = { package = "parity-scale-codec", version = "1.0.0" } +runtime_io = { package = "sr-io", path = "../../../primitives/sr-io" } +state_machine = { package = "substrate-state-machine", path = "../../../primitives/state-machine" } +substrate-executor = { path = "../../../client/executor" } +primitives = { package = "substrate-primitives", path = "../../../primitives/core" } +trie = { package = "substrate-trie", path = "../../../primitives/trie" } +node-primitives = { path = "../primitives" } +node-runtime = { path = "../runtime" } + +[dev-dependencies] +node-testing = { path = "../testing" } +test-client = { package = "substrate-test-client", path = "../../../test/utils/client" } +sr-primitives = { path = "../../../primitives/sr-primitives" } +runtime_support = { package = "paint-support", path = "../../../paint/support" } +balances = { package = "paint-balances", path = "../../../paint/balances" } +transaction-payment = { package = "paint-transaction-payment", path = "../../../paint/transaction-payment" } +session = { package = "paint-session", path = "../../../paint/session" } +system = { package = "paint-system", path = "../../../paint/system" } +timestamp = { package = "paint-timestamp", path = "../../../paint/timestamp" } +treasury = { package = "paint-treasury", path = "../../../paint/treasury" } +contracts = { package = "paint-contracts", path = "../../../paint/contracts" } +grandpa = { package = "paint-grandpa", path = "../../../paint/grandpa" } +indices = { package = "paint-indices", path = "../../../paint/indices" } +wabt = "0.9.2" +criterion = "0.3.0" + +[features] +wasmtime = [ + "substrate-executor/wasmtime", +] +wasmi-errno = [ + "substrate-executor/wasmi-errno", +] +stress-test = [] + +[[bench]] +name = "bench" +harness = false + diff --git a/bin/node/executor/benches/bench.rs b/bin/node/executor/benches/bench.rs new file mode 100644 index 0000000000000..e72c28467fa7f --- /dev/null +++ b/bin/node/executor/benches/bench.rs @@ -0,0 +1,196 @@ +// Copyright 2018-2019 Parity Technologies (UK) Ltd. +// This file is part of Substrate. + +// Substrate is free software: you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. + +// Substrate is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. + +// You should have received a copy of the GNU General Public License +// along with Substrate. If not, see . + +use codec::{Decode, Encode}; +use criterion::{BatchSize, Criterion, criterion_group, criterion_main}; +use node_executor::Executor; +use node_primitives::{BlockNumber, Hash}; +use node_runtime::{ + Block, BuildStorage, Call, CheckedExtrinsic, GenesisConfig, Header, UncheckedExtrinsic, +}; +use node_runtime::constants::currency::*; +use node_testing::keyring::*; +use primitives::{Blake2Hasher, NativeOrEncoded, NeverNativeValue}; +use primitives::storage::well_known_keys; +use primitives::traits::CodeExecutor; +use runtime_support::Hashable; +use state_machine::TestExternalities as CoreTestExternalities; +use substrate_executor::{NativeExecutor, RuntimeInfo, WasmExecutionMethod, Externalities}; + +criterion_group!(benches, bench_execute_block); +criterion_main!(benches); + +/// The wasm runtime code. +const COMPACT_CODE: &[u8] = node_runtime::WASM_BINARY; + +const GENESIS_HASH: [u8; 32] = [69u8; 32]; + +const VERSION: u32 = node_runtime::VERSION.spec_version; + +const HEAP_PAGES: u64 = 20; + +type TestExternalities = CoreTestExternalities; + +#[derive(Debug)] +enum ExecutionMethod { + Native, + Wasm(WasmExecutionMethod), +} + +fn sign(xt: CheckedExtrinsic) -> UncheckedExtrinsic { + node_testing::keyring::sign(xt, VERSION, GENESIS_HASH) +} + +fn new_test_ext(genesis_config: &GenesisConfig) -> TestExternalities { + let mut test_ext = TestExternalities::new_with_code( + COMPACT_CODE, + genesis_config.build_storage().unwrap(), + ); + test_ext.ext().place_storage(well_known_keys::HEAP_PAGES.to_vec(), Some(HEAP_PAGES.encode())); + test_ext +} + +fn construct_block( + executor: &NativeExecutor, + ext: &mut E, + number: BlockNumber, + parent_hash: Hash, + extrinsics: Vec, +) -> (Vec, Hash) { + use trie::{TrieConfiguration, trie_types::Layout}; + + // sign extrinsics. + let extrinsics = extrinsics.into_iter().map(sign).collect::>(); + + // calculate the header fields that we can. + let extrinsics_root = Layout::::ordered_trie_root( + extrinsics.iter().map(Encode::encode) + ).to_fixed_bytes() + .into(); + + let header = Header { + parent_hash, + number, + extrinsics_root, + state_root: Default::default(), + digest: Default::default(), + }; + + // execute the block to get the real header. + executor.call::<_, NeverNativeValue, fn() -> _>( + ext, + "Core_initialize_block", + &header.encode(), + true, + None, + ).0.unwrap(); + + for i in extrinsics.iter() { + executor.call::<_, NeverNativeValue, fn() -> _>( + ext, + "BlockBuilder_apply_extrinsic", + &i.encode(), + true, + None, + ).0.unwrap(); + } + + let header = match executor.call::<_, NeverNativeValue, fn() -> _>( + ext, + "BlockBuilder_finalize_block", + &[0u8;0], + true, + None, + ).0.unwrap() { + NativeOrEncoded::Native(_) => unreachable!(), + NativeOrEncoded::Encoded(h) => Header::decode(&mut &h[..]).unwrap(), + }; + + let hash = header.blake2_256(); + (Block { header, extrinsics }.encode(), hash.into()) +} + + +fn test_blocks(genesis_config: &GenesisConfig, executor: &NativeExecutor) + -> Vec<(Vec, Hash)> +{ + let mut test_ext = new_test_ext(genesis_config); + let mut block1_extrinsics = vec![ + CheckedExtrinsic { + signed: None, + function: Call::Timestamp(timestamp::Call::set(42 * 1000)), + }, + ]; + block1_extrinsics.extend((0..20).map(|i| { + CheckedExtrinsic { + signed: Some((alice(), signed_extra(i, 0))), + function: Call::Balances(balances::Call::transfer(bob().into(), 1 * DOLLARS)), + } + })); + let block1 = construct_block( + executor, + &mut test_ext.ext(), + 1, + GENESIS_HASH.into(), + block1_extrinsics, + ); + + vec![block1] +} + +fn bench_execute_block(c: &mut Criterion) { + c.bench_function_over_inputs( + "execute blocks", + |b, strategy| { + let genesis_config = node_testing::genesis::config(false, Some(COMPACT_CODE)); + let (use_native, wasm_method) = match strategy { + ExecutionMethod::Native => (true, WasmExecutionMethod::Interpreted), + ExecutionMethod::Wasm(wasm_method) => (false, *wasm_method), + }; + let executor = NativeExecutor::new(wasm_method, None); + + // Get the runtime version to initialize the runtimes cache. + { + let mut test_ext = new_test_ext(&genesis_config); + executor.runtime_version(&mut test_ext.ext()); + } + + let blocks = test_blocks(&genesis_config, &executor); + + b.iter_batched_ref( + || new_test_ext(&genesis_config), + |test_ext| { + for block in blocks.iter() { + executor.call::<_, NeverNativeValue, fn() -> _>( + &mut test_ext.ext(), + "Core_execute_block", + &block.0, + use_native, + None, + ).0.unwrap(); + } + }, + BatchSize::LargeInput, + ); + }, + vec![ + ExecutionMethod::Native, + ExecutionMethod::Wasm(WasmExecutionMethod::Interpreted), + #[cfg(feature = "wasmtime")] + ExecutionMethod::Wasm(WasmExecutionMethod::Compiled), + ], + ); +} diff --git a/node/executor/src/lib.rs b/bin/node/executor/src/lib.rs similarity index 98% rename from node/executor/src/lib.rs rename to bin/node/executor/src/lib.rs index 39727d1d5503e..ebff2da1a4b96 100644 --- a/node/executor/src/lib.rs +++ b/bin/node/executor/src/lib.rs @@ -17,10 +17,6 @@ //! A `CodeExecutor` specialization which uses natively compiled runtime when the wasm to be //! executed is equivalent to the natively compiled code. -#![cfg_attr(feature = "benchmarks", feature(test))] - -#[cfg(feature = "benchmarks")] extern crate test; - pub use substrate_executor::NativeExecutor; use substrate_executor::native_executor_instance; @@ -470,13 +466,18 @@ mod tests { event: Event::system(system::Event::ExtrinsicSuccess), topics: vec![], }, + EventRecord { + phase: Phase::ApplyExtrinsic(1), + event: Event::treasury(treasury::RawEvent::Deposit(1984800000000)), + topics: vec![], + }, EventRecord { phase: Phase::ApplyExtrinsic(1), event: Event::balances(balances::RawEvent::Transfer( alice().into(), bob().into(), 69 * DOLLARS, - 1 * CENTS + 1 * CENTS, )), topics: vec![], }, @@ -514,6 +515,11 @@ mod tests { event: Event::system(system::Event::ExtrinsicSuccess), topics: vec![], }, + EventRecord { + phase: Phase::ApplyExtrinsic(1), + event: Event::treasury(treasury::RawEvent::Deposit(1984780231392)), + topics: vec![], + }, EventRecord { phase: Phase::ApplyExtrinsic(1), event: Event::balances( @@ -531,6 +537,11 @@ mod tests { event: Event::system(system::Event::ExtrinsicSuccess), topics: vec![], }, + EventRecord { + phase: Phase::ApplyExtrinsic(2), + event: Event::treasury(treasury::RawEvent::Deposit(1984780231392)), + topics: vec![], + }, EventRecord { phase: Phase::ApplyExtrinsic(2), event: Event::balances( @@ -1208,21 +1219,4 @@ mod tests { block_number += 1; } } - - #[cfg(feature = "benchmarks")] - mod benches { - use super::*; - use test::Bencher; - - #[bench] - fn wasm_execute_block(b: &mut Bencher) { - let (block1, block2) = blocks(); - - b.iter(|| { - let mut t = new_test_ext(COMPACT_CODE, false); - WasmExecutor::new().call(&mut t, "Core_execute_block", &block1.0).unwrap(); - WasmExecutor::new().call(&mut t, "Core_execute_block", &block2.0).unwrap(); - }); - } - } } diff --git a/node/primitives/Cargo.toml b/bin/node/primitives/Cargo.toml similarity index 50% rename from node/primitives/Cargo.toml rename to bin/node/primitives/Cargo.toml index 25725449a3a5a..1f632dda8dd5c 100644 --- a/node/primitives/Cargo.toml +++ b/bin/node/primitives/Cargo.toml @@ -5,11 +5,11 @@ authors = ["Parity Technologies "] edition = "2018" [dependencies] -primitives = { package = "substrate-primitives", path = "../../core/primitives", default-features = false } -sr-primitives = { path = "../../core/sr-primitives", default-features = false } +primitives = { package = "substrate-primitives", path = "../../../primitives/core", default-features = false } +sr-primitives = { path = "../../../primitives/sr-primitives", default-features = false } [dev-dependencies] -substrate-serializer = { path = "../../core/serializer" } +substrate-serializer = { path = "../../../primitives/serializer" } pretty_assertions = "0.6.1" [features] diff --git a/node/primitives/src/lib.rs b/bin/node/primitives/src/lib.rs similarity index 100% rename from node/primitives/src/lib.rs rename to bin/node/primitives/src/lib.rs diff --git a/node/rpc-client/Cargo.toml b/bin/node/rpc-client/Cargo.toml similarity index 66% rename from node/rpc-client/Cargo.toml rename to bin/node/rpc-client/Cargo.toml index e377f89359582..882dd7ee2b629 100644 --- a/node/rpc-client/Cargo.toml +++ b/bin/node/rpc-client/Cargo.toml @@ -8,7 +8,7 @@ edition = "2018" env_logger = "0.7.0" futures = "0.1.29" hyper = "0.12.35" -jsonrpc-core-client = { version = "13.1.0", features = ["http", "ws"] } +jsonrpc-core-client = { version = "14.0.3", features = ["http", "ws"] } log = "0.4.8" node-primitives = { path = "../primitives" } -substrate-rpc = { path = "../../core/rpc", version = "2.0.0" } +substrate-rpc = { path = "../../../client/rpc", version = "2.0.0" } diff --git a/node/rpc-client/src/main.rs b/bin/node/rpc-client/src/main.rs similarity index 100% rename from node/rpc-client/src/main.rs rename to bin/node/rpc-client/src/main.rs diff --git a/bin/node/rpc/Cargo.toml b/bin/node/rpc/Cargo.toml new file mode 100644 index 0000000000000..18f63f6b40f0f --- /dev/null +++ b/bin/node/rpc/Cargo.toml @@ -0,0 +1,16 @@ +[package] +name = "node-rpc" +version = "2.0.0" +authors = ["Parity Technologies "] +edition = "2018" + +[dependencies] +client = { package = "substrate-client", path = "../../../client/" } +jsonrpc-core = "14.0.3" +node-primitives = { path = "../primitives" } +node-runtime = { path = "../runtime" } +sr-primitives = { path = "../../../primitives/sr-primitives" } +paint-contracts-rpc = { path = "../../../paint/contracts/rpc/" } +paint-transaction-payment-rpc = { path = "../../../paint/transaction-payment/rpc/" } +paint-system-rpc = { path = "../../../paint/system/rpc/" } +transaction_pool = { package = "substrate-transaction-pool", path = "../../../client/transaction-pool" } diff --git a/node/rpc/src/lib.rs b/bin/node/rpc/src/lib.rs similarity index 81% rename from node/rpc/src/lib.rs rename to bin/node/rpc/src/lib.rs index 99321373f8f8c..fd8dc2a5a7a6c 100644 --- a/node/rpc/src/lib.rs +++ b/bin/node/rpc/src/lib.rs @@ -19,7 +19,7 @@ //! Since `substrate` core functionality makes no assumptions //! about the modules used inside the runtime, so do //! RPC methods defined in `substrate-rpc` crate. -//! It means that `core/rpc` can't have any methods that +//! It means that `client/rpc` can't have any methods that //! need some strong assumptions about the particular runtime. //! //! The RPCs available in this crate however can make some assumptions @@ -41,15 +41,15 @@ pub fn create(client: Arc, pool: Arc>) -> jsonrpc_core::IoHa C: ProvideRuntimeApi, C: client::blockchain::HeaderBackend, C: Send + Sync + 'static, - C::Api: srml_system_rpc::AccountNonceApi, - C::Api: srml_contracts_rpc::ContractsRuntimeApi, - C::Api: srml_transaction_payment_rpc::TransactionPaymentRuntimeApi, + C::Api: paint_system_rpc::AccountNonceApi, + C::Api: paint_contracts_rpc::ContractsRuntimeApi, + C::Api: paint_transaction_payment_rpc::TransactionPaymentRuntimeApi, P: ChainApi + Sync + Send + 'static, M: jsonrpc_core::Metadata + Default, { - use srml_system_rpc::{System, SystemApi}; - use srml_contracts_rpc::{Contracts, ContractsApi}; - use srml_transaction_payment_rpc::{TransactionPayment, TransactionPaymentApi}; + use paint_system_rpc::{System, SystemApi}; + use paint_contracts_rpc::{Contracts, ContractsApi}; + use paint_transaction_payment_rpc::{TransactionPayment, TransactionPaymentApi}; let mut io = jsonrpc_core::IoHandler::default(); io.extend_with( diff --git a/bin/node/runtime/Cargo.toml b/bin/node/runtime/Cargo.toml new file mode 100644 index 0000000000000..2cc7f84e8574c --- /dev/null +++ b/bin/node/runtime/Cargo.toml @@ -0,0 +1,122 @@ +[package] +name = "node-runtime" +version = "2.0.0" +authors = ["Parity Technologies "] +edition = "2018" +build = "build.rs" + +[dependencies] +# third-party dependencies +codec = { package = "parity-scale-codec", version = "1.0.6", default-features = false, features = ["derive"] } +integer-sqrt = { version = "0.1.2" } +safe-mix = { version = "1.0", default-features = false } +rustc-hex = { version = "2.0", optional = true } +serde = { version = "1.0.102", optional = true } + +# primitives +authority-discovery-primitives = { package = "substrate-authority-discovery-primitives", path = "../../../primitives/authority-discovery", default-features = false } +babe-primitives = { package = "substrate-consensus-babe-primitives", path = "../../../primitives/consensus/babe", default-features = false } +block-builder-api = { package = "substrate-block-builder-runtime-api", path = "../../../primitives/block-builder/runtime-api", default-features = false} +inherents = { package = "substrate-inherents", path = "../../../primitives/inherents", default-features = false } +node-primitives = { path = "../primitives", default-features = false } +offchain-primitives = { package = "substrate-offchain-primitives", path = "../../../primitives/offchain", default-features = false } +primitives = { package = "substrate-primitives", path = "../../../primitives/core", default-features = false } +rstd = { package = "sr-std", path = "../../../primitives/sr-std", default-features = false } +sr-api = { path = "../../../primitives/sr-api", default-features = false } +sr-primitives = { path = "../../../primitives/sr-primitives", default-features = false } +sr-staking-primitives = { path = "../../../primitives/sr-staking-primitives", default-features = false } +substrate-keyring = { path = "../../../primitives/keyring", optional = true } +substrate-session = { path = "../../../primitives/session", default-features = false } +tx-pool-api = { package = "substrate-transaction-pool-runtime-api", path = "../../../primitives/transaction-pool/runtime-api", default-features = false } +version = { package = "sr-version", path = "../../../primitives/sr-version", default-features = false } + +# paint dependencies +authority-discovery = { package = "paint-authority-discovery", path = "../../../paint/authority-discovery", default-features = false } +authorship = { package = "paint-authorship", path = "../../../paint/authorship", default-features = false } +babe = { package = "paint-babe", path = "../../../paint/babe", default-features = false } +balances = { package = "paint-balances", path = "../../../paint/balances", default-features = false } +collective = { package = "paint-collective", path = "../../../paint/collective", default-features = false } +contracts = { package = "paint-contracts", path = "../../../paint/contracts", default-features = false } +contracts-rpc-runtime-api = { package = "paint-contracts-rpc-runtime-api", path = "../../../paint/contracts/rpc/runtime-api/", default-features = false } +democracy = { package = "paint-democracy", path = "../../../paint/democracy", default-features = false } +elections-phragmen = { package = "paint-elections-phragmen", path = "../../../paint/elections-phragmen", default-features = false } +executive = { package = "paint-executive", path = "../../../paint/executive", default-features = false } +finality-tracker = { package = "paint-finality-tracker", path = "../../../paint/finality-tracker", default-features = false } +grandpa = { package = "paint-grandpa", path = "../../../paint/grandpa", default-features = false } +im-online = { package = "paint-im-online", path = "../../../paint/im-online", default-features = false } +indices = { package = "paint-indices", path = "../../../paint/indices", default-features = false } +membership = { package = "paint-membership", path = "../../../paint/membership", default-features = false } +nicks = { package = "paint-nicks", path = "../../../paint/nicks", default-features = false } +offences = { package = "paint-offences", path = "../../../paint/offences", default-features = false } +randomness-collective-flip = { package = "paint-randomness-collective-flip", path = "../../../paint/randomness-collective-flip", default-features = false } +session = { package = "paint-session", path = "../../../paint/session", default-features = false, features = ["historical"] } +staking = { package = "paint-staking", path = "../../../paint/staking", default-features = false } +paint-staking-reward-curve = { path = "../../../paint/staking/reward-curve"} +sudo = { package = "paint-sudo", path = "../../../paint/sudo", default-features = false } +support = { package = "paint-support", path = "../../../paint/support", default-features = false } +system = { package = "paint-system", path = "../../../paint/system", default-features = false } +system-rpc-runtime-api = { package = "paint-system-rpc-runtime-api", path = "../../../paint/system/rpc/runtime-api/", default-features = false } +timestamp = { package = "paint-timestamp", path = "../../../paint/timestamp", default-features = false } +treasury = { package = "paint-treasury", path = "../../../paint/treasury", default-features = false } +utility = { package = "paint-utility", path = "../../../paint/utility", default-features = false } +transaction-payment = { package = "paint-transaction-payment", path = "../../../paint/transaction-payment", default-features = false } +transaction-payment-rpc-runtime-api = { package = "paint-transaction-payment-rpc-runtime-api", path = "../../../paint/transaction-payment/rpc/runtime-api/", default-features = false } + +[build-dependencies] +wasm-builder-runner = { package = "substrate-wasm-builder-runner", path = "../../../client/utils/wasm-builder-runner", version = "1.0.4" } + +[dev-dependencies] +runtime_io = { package = "sr-io", path = "../../../primitives/sr-io" } + +[features] +default = ["std"] +std = [ + "authority-discovery-primitives/std", + "authority-discovery/std", + "authorship/std", + "babe-primitives/std", + "babe/std", + "balances/std", + "block-builder-api/std", + "codec/std", + "collective/std", + "contracts-rpc-runtime-api/std", + "contracts/std", + "democracy/std", + "elections-phragmen/std", + "executive/std", + "finality-tracker/std", + "grandpa/std", + "im-online/std", + "indices/std", + "inherents/std", + "membership/std", + "nicks/std", + "node-primitives/std", + "offchain-primitives/std", + "offences/std", + "primitives/std", + "randomness-collective-flip/std", + "rstd/std", + "rustc-hex", + "safe-mix/std", + "serde", + "session/std", + "sr-api/std", + "sr-primitives/std", + "sr-staking-primitives/std", + "staking/std", + "substrate-keyring", + "substrate-session/std", + "sudo/std", + "support/std", + "system-rpc-runtime-api/std", + "system/std", + "timestamp/std", + "transaction-payment-rpc-runtime-api/std", + "transaction-payment/std", + "treasury/std", + "tx-pool-api/std", + "utility/std", + "version/std", +] diff --git a/core/executor/runtime-test/build.rs b/bin/node/runtime/build.rs similarity index 96% rename from core/executor/runtime-test/build.rs rename to bin/node/runtime/build.rs index 68e5205f0c7a8..d6b0ae626facb 100644 --- a/core/executor/runtime-test/build.rs +++ b/bin/node/runtime/build.rs @@ -20,7 +20,7 @@ fn main() { build_current_project_with_rustflags( "wasm_binary.rs", WasmBuilderSource::CratesOrPath { - path: "../../utils/wasm-builder", + path: "../../../utils/wasm-builder", version: "1.0.8", }, // This instructs LLD to export __heap_base as a global variable, which is used by the diff --git a/node/runtime/src/constants.rs b/bin/node/runtime/src/constants.rs similarity index 100% rename from node/runtime/src/constants.rs rename to bin/node/runtime/src/constants.rs diff --git a/node/runtime/src/impls.rs b/bin/node/runtime/src/impls.rs similarity index 99% rename from node/runtime/src/impls.rs rename to bin/node/runtime/src/impls.rs index 2e9bd38c8f822..f6d31cc22507b 100644 --- a/node/runtime/src/impls.rs +++ b/bin/node/runtime/src/impls.rs @@ -25,7 +25,7 @@ use crate::{Balances, System, Authorship, MaximumBlockWeight, NegativeImbalance} pub struct Author; impl OnUnbalanced for Author { - fn on_unbalanced(amount: NegativeImbalance) { + fn on_nonzero_unbalanced(amount: NegativeImbalance) { Balances::resolve_creating(&Authorship::author(), amount); } } @@ -61,7 +61,7 @@ impl> Convert for LinearWeightToFee { /// Update the given multiplier based on the following formula /// -/// diff = (target_weight - previous_block_weight) +/// diff = (previous_block_weight - target_weight) /// v = 0.00004 /// next_weight = weight * (1 + (v . diff) + (v . diff)^2 / 2) /// diff --git a/node/runtime/src/lib.rs b/bin/node/runtime/src/lib.rs similarity index 91% rename from node/runtime/src/lib.rs rename to bin/node/runtime/src/lib.rs index 4f261431462b3..26c8288a4c911 100644 --- a/node/runtime/src/lib.rs +++ b/bin/node/runtime/src/lib.rs @@ -25,15 +25,8 @@ use support::{ construct_runtime, parameter_types, traits::{SplitTwoWays, Currency, Randomness} }; use primitives::u32_trait::{_1, _2, _3, _4}; -use node_primitives::{ - AccountId, AccountIndex, Balance, BlockNumber, Hash, Index, - Moment, Signature, -}; -use grandpa::fg_primitives; -use client::{ - block_builder::api::{self as block_builder_api, InherentData, CheckInherentsResult}, - runtime_api as client_api, impl_runtime_apis -}; +use node_primitives::{AccountId, AccountIndex, Balance, BlockNumber, Hash, Index, Moment, Signature}; +use sr_api::impl_runtime_apis; use sr_primitives::{Permill, Perbill, ApplyResult, impl_opaque_keys, generic, create_runtime_str}; use sr_primitives::curve::PiecewiseLinear; use sr_primitives::transaction_validity::TransactionValidity; @@ -46,11 +39,14 @@ use version::RuntimeVersion; #[cfg(any(feature = "std", test))] use version::NativeVersion; use primitives::OpaqueMetadata; -use grandpa::{AuthorityId as GrandpaId, AuthorityWeight as GrandpaWeight}; +use grandpa::AuthorityList as GrandpaAuthorityList; +use grandpa::fg_primitives; use im_online::sr25519::{AuthorityId as ImOnlineId}; +use authority_discovery_primitives::AuthorityId as AuthorityDiscoveryId; use transaction_payment_rpc_runtime_api::RuntimeDispatchInfo; use contracts_rpc_runtime_api::ContractExecResult; use system::offchain::TransactionSubmitter; +use inherents::{InherentData, CheckInherentsResult}; #[cfg(any(feature = "std", test))] pub use sr_primitives::BuildStorage; @@ -81,8 +77,8 @@ pub const VERSION: RuntimeVersion = RuntimeVersion { // and set impl_version to equal spec_version. If only runtime // implementation changes and behavior does not, then leave spec_version as // is and increment impl_version. - spec_version: 190, - impl_version: 190, + spec_version: 194, + impl_version: 194, apis: RUNTIME_API_VERSIONS, }; @@ -206,7 +202,7 @@ impl authorship::Trait for Runtime { type FindAuthor = session::FindAccountFromAuthorIndex; type UncleGenerations = UncleGenerations; type FilterUncle = (); - type EventHandler = Staking; + type EventHandler = (Staking, ImOnline); } impl_opaque_keys! { @@ -214,6 +210,7 @@ impl_opaque_keys! { pub grandpa: Grandpa, pub babe: Babe, pub im_online: ImOnline, + pub authority_discovery: AuthorityDiscovery, } } @@ -238,7 +235,7 @@ impl session::historical::Trait for Runtime { type FullIdentificationOf = staking::ExposureOf; } -srml_staking_reward_curve::build! { +paint_staking_reward_curve::build! { const REWARD_CURVE: PiecewiseLinear<'static> = curve!( min_inflation: 0_025_000, max_inflation: 0_100_000, @@ -259,7 +256,7 @@ impl staking::Trait for Runtime { type Currency = Balances; type Time = Timestamp; type CurrencyToVote = CurrencyToVoteHandler; - type OnRewardMinted = Treasury; + type RewardRemainder = Treasury; type Event = Event; type Slash = Treasury; // send the slashed funds to the treasury. type Reward = (); // rewards are minted from the void @@ -419,12 +416,17 @@ impl sudo::Trait for Runtime { type SubmitTransaction = TransactionSubmitter; +parameter_types! { + pub const SessionDuration: BlockNumber = EPOCH_DURATION_IN_SLOTS as _; +} + impl im_online::Trait for Runtime { type AuthorityId = ImOnlineId; type Call = Call; type Event = Event; type SubmitTransaction = SubmitTransaction; type ReportUnresponsiveness = Offences; + type SessionDuration = SessionDuration; } impl offences::Trait for Runtime { @@ -433,6 +435,8 @@ impl offences::Trait for Runtime { type OnOffenceHandler = Staking; } +impl authority_discovery::Trait for Runtime {} + impl grandpa::Trait for Runtime { type Event = Event; } @@ -521,6 +525,7 @@ construct_runtime!( Contracts: contracts, Sudo: sudo, ImOnline: im_online::{Module, Call, Storage, Event, ValidateUnsigned, Config}, + AuthorityDiscovery: authority_discovery::{Module, Call, Config}, Offences: offences::{Module, Call, Storage, Event}, RandomnessCollectiveFlip: randomness_collective_flip::{Module, Call, Storage}, Nicks: nicks::{Module, Call, Storage, Event}, @@ -557,7 +562,7 @@ pub type CheckedExtrinsic = generic::CheckedExtrinsic, Runtime, AllModules>; impl_runtime_apis! { - impl client_api::Core for Runtime { + impl sr_api::Core for Runtime { fn version() -> RuntimeVersion { VERSION } @@ -571,7 +576,7 @@ impl_runtime_apis! { } } - impl client_api::Metadata for Runtime { + impl sr_api::Metadata for Runtime { fn metadata() -> OpaqueMetadata { Runtime::metadata().into() } @@ -599,7 +604,7 @@ impl_runtime_apis! { } } - impl client_api::TaggedTransactionQueue for Runtime { + impl tx_pool_api::TaggedTransactionQueue for Runtime { fn validate_transaction(tx: ::Extrinsic) -> TransactionValidity { Executive::validate_transaction(tx) } @@ -612,7 +617,7 @@ impl_runtime_apis! { } impl fg_primitives::GrandpaApi for Runtime { - fn grandpa_authorities() -> Vec<(GrandpaId, GrandpaWeight)> { + fn grandpa_authorities() -> GrandpaAuthorityList { Grandpa::grandpa_authorities() } } @@ -635,6 +640,12 @@ impl_runtime_apis! { } } + impl authority_discovery_primitives::AuthorityDiscoveryApi for Runtime { + fn authorities() -> Vec { + AuthorityDiscovery::authorities() + } + } + impl system_rpc_runtime_api::AccountNonceApi for Runtime { fn account_nonce(account: AccountId) -> Index { System::account_nonce(account) @@ -693,12 +704,11 @@ impl_runtime_apis! { impl substrate_session::SessionKeys for Runtime { fn generate_session_keys(seed: Option>) -> Vec { - let seed = seed.as_ref().map(|s| rstd::str::from_utf8(&s) - .expect("Seed is an utf8 string")); SessionKeys::generate(seed) } } } + #[cfg(test)] mod tests { use super::*; @@ -719,4 +729,36 @@ mod tests { let x = SubmitTransaction::default(); is_submit_signed_transaction(x); } + + #[test] + fn block_hooks_weight_should_not_exceed_limits() { + use sr_primitives::weights::WeighBlock; + let check_for_block = |b| { + let block_hooks_weight = + >::on_initialize(b) + + >::on_finalize(b); + + assert_eq!( + block_hooks_weight, + 0, + "This test might fail simply because the value being compared to has increased to a \ + module declaring a new weight for a hook or call. In this case update the test and \ + happily move on.", + ); + + // Invariant. Always must be like this to have a sane chain. + assert!(block_hooks_weight < MaximumBlockWeight::get()); + + // Warning. + if block_hooks_weight > MaximumBlockWeight::get() / 2 { + println!( + "block hooks weight is consuming more than a block's capacity. You probably want \ + to re-think this. This test will fail now." + ); + assert!(false); + } + }; + + let _ = (0..100_000).for_each(check_for_block); + } } diff --git a/bin/node/testing/Cargo.toml b/bin/node/testing/Cargo.toml new file mode 100644 index 0000000000000..1f585c19a9b30 --- /dev/null +++ b/bin/node/testing/Cargo.toml @@ -0,0 +1,31 @@ +[package] +name = "node-testing" +version = "2.0.0" +authors = ["Parity Technologies "] +description = "Test utilities for Substrate node." +edition = "2018" + +[dependencies] +balances = { package = "paint-balances", path = "../../../paint/balances" } +client = { package = "substrate-client", path = "../../../client/" } +codec = { package = "parity-scale-codec", version = "1.0.0" } +contracts = { package = "paint-contracts", path = "../../../paint/contracts" } +grandpa = { package = "paint-grandpa", path = "../../../paint/grandpa" } +indices = { package = "paint-indices", path = "../../../paint/indices" } +keyring = { package = "substrate-keyring", path = "../../../primitives/keyring" } +node-executor = { path = "../executor" } +node-primitives = { path = "../primitives" } +node-runtime = { path = "../runtime" } +primitives = { package = "substrate-primitives", path = "../../../primitives/core" } +runtime-io = { package = "sr-io", path = "../../../primitives/sr-io" } +runtime_support = { package = "paint-support", path = "../../../paint/support" } +session = { package = "paint-session", path = "../../../paint/session" } +sr-primitives = { path = "../../../primitives/sr-primitives" } +staking = { package = "paint-staking", path = "../../../paint/staking" } +substrate-executor = { path = "../../../client/executor" } +system = { package = "paint-system", path = "../../../paint/system" } +test-client = { package = "substrate-test-client", path = "../../../test/utils/client" } +timestamp = { package = "paint-timestamp", path = "../../../paint/timestamp" } +transaction-payment = { package = "paint-transaction-payment", path = "../../../paint/transaction-payment" } +treasury = { package = "paint-treasury", path = "../../../paint/treasury" } +wabt = "0.9.2" diff --git a/node/testing/src/client.rs b/bin/node/testing/src/client.rs similarity index 100% rename from node/testing/src/client.rs rename to bin/node/testing/src/client.rs diff --git a/node/testing/src/genesis.rs b/bin/node/testing/src/genesis.rs similarity index 98% rename from node/testing/src/genesis.rs rename to bin/node/testing/src/genesis.rs index 0b99052f25026..1531ba134892a 100644 --- a/node/testing/src/genesis.rs +++ b/bin/node/testing/src/genesis.rs @@ -89,6 +89,7 @@ pub fn config(support_changes_trie: bool, code: Option<&[u8]>) -> GenesisConfig authorities: vec![], }), im_online: Some(Default::default()), + authority_discovery: Some(Default::default()), democracy: Some(Default::default()), collective_Instance1: Some(Default::default()), collective_Instance2: Some(Default::default()), diff --git a/node/testing/src/keyring.rs b/bin/node/testing/src/keyring.rs similarity index 96% rename from node/testing/src/keyring.rs rename to bin/node/testing/src/keyring.rs index ca44a53880fc1..9c4f11fcf3110 100644 --- a/node/testing/src/keyring.rs +++ b/bin/node/testing/src/keyring.rs @@ -59,6 +59,7 @@ pub fn to_session_keys( grandpa: ed25519_keyring.to_owned().public().into(), babe: sr25519_keyring.to_owned().public().into(), im_online: sr25519_keyring.to_owned().public().into(), + authority_discovery: sr25519_keyring.to_owned().public().into(), } } @@ -83,7 +84,7 @@ pub fn sign(xt: CheckedExtrinsic, version: u32, genesis_hash: [u8; 32]) -> Unche let key = AccountKeyring::from_account_id(&signed).unwrap(); let signature = payload.using_encoded(|b| { if b.len() > 256 { - key.sign(&sr_io::blake2_256(b)) + key.sign(&runtime_io::hashing::blake2_256(b)) } else { key.sign(b) } @@ -99,4 +100,3 @@ pub fn sign(xt: CheckedExtrinsic, version: u32, genesis_hash: [u8; 32]) -> Unche }, } } - diff --git a/node/testing/src/lib.rs b/bin/node/testing/src/lib.rs similarity index 100% rename from node/testing/src/lib.rs rename to bin/node/testing/src/lib.rs diff --git a/subkey/Cargo.toml b/bin/subkey/Cargo.toml similarity index 61% rename from subkey/Cargo.toml rename to bin/subkey/Cargo.toml index e1c3ef64d26d4..2a3a8318c9e85 100644 --- a/subkey/Cargo.toml +++ b/bin/subkey/Cargo.toml @@ -5,10 +5,10 @@ authors = ["Parity Technologies "] edition = "2018" [dependencies] -primitives = { package = "substrate-primitives", version = "*", path = "../core/primitives" } +primitives = { package = "substrate-primitives", version = "*", path = "../../primitives/core" } node-runtime = { version = "*", path = "../node/runtime" } node-primitives = { version = "*", path = "../node/primitives" } -sr-primitives = { version = "*", path = "../core/sr-primitives" } +sr-primitives = { version = "*", path = "../../primitives/sr-primitives" } rand = "0.7.2" clap = { version = "2.33.0", features = ["yaml"] } tiny-bip39 = "0.6.2" @@ -17,9 +17,9 @@ substrate-bip39 = "0.3.1" hex = "0.3.2" hex-literal = "0.2.1" codec = { package = "parity-scale-codec", version = "1.0.0" } -system = { package = "srml-system", path = "../srml/system" } -balances = { package = "srml-balances", path = "../srml/balances" } -transaction-payment = { package = "srml-transaction-payment", path = "../srml/transaction-payment" } +system = { package = "paint-system", path = "../../paint/system" } +balances = { package = "paint-balances", path = "../../paint/balances" } +transaction-payment = { package = "paint-transaction-payment", path = "../../paint/transaction-payment" } [features] bench = [] diff --git a/subkey/README.adoc b/bin/subkey/README.adoc similarity index 100% rename from subkey/README.adoc rename to bin/subkey/README.adoc diff --git a/subkey/src/cli.yml b/bin/subkey/src/cli.yml similarity index 100% rename from subkey/src/cli.yml rename to bin/subkey/src/cli.yml diff --git a/subkey/src/main.rs b/bin/subkey/src/main.rs similarity index 100% rename from subkey/src/main.rs rename to bin/subkey/src/main.rs diff --git a/subkey/src/vanity.rs b/bin/subkey/src/vanity.rs similarity index 100% rename from subkey/src/vanity.rs rename to bin/subkey/src/vanity.rs diff --git a/ci/script.sh b/ci/script.sh deleted file mode 100755 index b1b7dd3edf356..0000000000000 --- a/ci/script.sh +++ /dev/null @@ -1,31 +0,0 @@ -#!/usr/bin/env bash - -set -eux - -# Enable warnings about unused extern crates -export RUSTFLAGS=" -W unused-extern-crates" - -# Install rustup and the specified rust toolchain. -curl https://sh.rustup.rs -sSf | sh -s -- --default-toolchain=$RUST_TOOLCHAIN -y - -# Load cargo environment. Specifically, put cargo into PATH. -source ~/.cargo/env - -rustc --version -rustup --version -cargo --version - -case $TARGET in - "native") - sudo apt-get -y update - sudo apt-get install -y cmake pkg-config libssl-dev - - cargo test --all --release --locked "$@" - ;; - - "wasm") - # Install prerequisites and build all wasm projects - ./scripts/init.sh - ./scripts/build.sh "$@" - ;; -esac diff --git a/core/client/Cargo.toml b/client/Cargo.toml similarity index 53% rename from core/client/Cargo.toml rename to client/Cargo.toml index 761b2a9da21b0..398fc5a09197b 100644 --- a/core/client/Cargo.toml +++ b/client/Cargo.toml @@ -5,6 +5,8 @@ authors = ["Parity Technologies "] edition = "2018" [dependencies] +<<<<<<< HEAD:core/client/Cargo.toml +<<<<<<< HEAD derive_more = { version = "0.15.0", optional = true } fnv = { version = "1.0.6", optional = true } log = { version = "0.4.8", optional = true } @@ -18,6 +20,7 @@ state-machine = { package = "substrate-state-machine", path = "../state-machine" keyring = { package = "substrate-keyring", path = "../keyring", optional = true } trie = { package = "substrate-trie", path = "../trie", optional = true } substrate-telemetry = { path = "../telemetry", optional = true } +substrate-prometheus = { path = "../prometheus",optional = true } hash-db = { version = "0.15.2", default-features = false } kvdb = { git = "https://github.com/paritytech/parity-common", optional = true, rev="b0317f649ab2c665b7987b8475878fc4d2e1f81d" } codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false, features = ["derive"] } @@ -28,15 +31,50 @@ rstd = { package = "sr-std", path = "../sr-std", default-features = false } inherents = { package = "substrate-inherents", path = "../inherents", default-features = false } sr-api-macros = { path = "../sr-api-macros" } header-metadata = { package = "substrate-header-metadata", path = "header-metadata", optional = true } +======= +======= +client-api = { package = "substrate-client-api", path = "api" } +block-builder = { package = "substrate-block-builder", path = "block-builder" } +codec = { package = "parity-scale-codec", version = "1.0.0", features = ["derive"] } +consensus = { package = "substrate-consensus-common", path = "../primitives/consensus/common" } +kvdb = { git = "https://github.com/paritytech/parity-common", rev="b0317f649ab2c665b7987b8475878fc4d2e1f81d" } +>>>>>>> f44873dce... Reorganising the repository - external renames and moves (#4074):client/Cargo.toml +derive_more = { version = "0.15.0" } +executor = { package = "substrate-executor", path = "executor" } +fnv = { version = "1.0.6" } +futures = { version = "0.1.29" } +futures03 = { package = "futures-preview", version = "0.3.0-alpha.19", features = ["compat"] } +hash-db = { version = "0.15.2" } +header-metadata = { package = "substrate-header-metadata", path = "header-metadata" } +<<<<<<< HEAD:core/client/Cargo.toml +block-builder = { package = "substrate-block-builder", path = "../block-builder" } +>>>>>>> d2c4b0dbd... Refactor sr-api to not depend on client anymore (#4086) +======= +hex-literal = { version = "0.2.1" } +inherents = { package = "substrate-inherents", path = "../primitives/inherents" } +keyring = { package = "substrate-keyring", path = "../primitives/keyring" } +log = { version = "0.4.8" } +parking_lot = { version = "0.9.0" } +primitives = { package = "substrate-primitives", path = "../primitives/core" } +rstd = { package = "sr-std", path = "../primitives/sr-std" } +runtime-version = { package = "sr-version", path = "../primitives/sr-version" } +sr-api = { path = "../primitives/sr-api" } +sr-primitives = { path = "../primitives/sr-primitives" } +state-machine = { package = "substrate-state-machine", path = "../primitives/state-machine" } +substrate-telemetry = { path = "telemetry" } +substrate-prometheus = { path = "prometheus" } +trie = { package = "substrate-trie", path = "../primitives/trie" } +>>>>>>> f44873dce... Reorganising the repository - external renames and moves (#4074):client/Cargo.toml [dev-dependencies] env_logger = "0.7.0" tempfile = "3.1.0" client-db = { package = "substrate-client-db", path = "./db", features = ["kvdb-rocksdb"] } -test-client = { package = "substrate-test-runtime-client", path = "../test-runtime/client" } +test-client = { package = "substrate-test-runtime-client", path = "../test/utils/runtime/client" } kvdb-memorydb = { git = "https://github.com/paritytech/parity-common", rev="b0317f649ab2c665b7987b8475878fc4d2e1f81d" } -panic-handler = { package = "substrate-panic-handler", path = "../panic-handler" } +panic-handler = { package = "substrate-panic-handler", path = "../primitives/panic-handler" } +<<<<<<< HEAD [features] default = ["std"] std = [ @@ -61,5 +99,8 @@ std = [ "keyring", "trie", "substrate-telemetry", + "substrate-prometheus", "kvdb" ] +======= +>>>>>>> d2c4b0dbd... Refactor sr-api to not depend on client anymore (#4086) diff --git a/client/api/Cargo.toml b/client/api/Cargo.toml new file mode 100644 index 0000000000000..9a122ad0f8591 --- /dev/null +++ b/client/api/Cargo.toml @@ -0,0 +1,41 @@ +[package] +name = "substrate-client-api" +version = "2.0.0" +authors = ["Parity Technologies "] +edition = "2018" + +[dependencies] +block-builder = { package = "substrate-block-builder", path = "../block-builder" } +codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false, features = ["derive"] } +consensus = { package = "substrate-consensus-common", path = "../../primitives/consensus/common" } +derive_more = { version = "0.15.0" } +executor = { package = "substrate-executor", path = "../executor" } +fnv = { version = "1.0.6" } +futures = { version = "0.1.29" } +futures03 = { package = "futures-preview", version = "0.3.0-alpha.19", features = ["compat"] } +hash-db = { version = "0.15.2", default-features = false } +header-metadata = { package = "substrate-header-metadata", path = "../header-metadata" } +hex-literal = { version = "0.2.1" } +inherents = { package = "substrate-inherents", path = "../../primitives/inherents", default-features = false } +keyring = { package = "substrate-keyring", path = "../../primitives/keyring" } +kvdb = { git = "https://github.com/paritytech/parity-common", rev="b0317f649ab2c665b7987b8475878fc4d2e1f81d" } +log = { version = "0.4.8" } +parking_lot = { version = "0.9.0" } +primitives = { package = "substrate-primitives", path = "../../primitives/core", default-features = false } +rstd = { package = "sr-std", path = "../../primitives/sr-std", default-features = false } +runtime-version = { package = "sr-version", path = "../../primitives/sr-version", default-features = false } +sr-api = { path = "../../primitives/sr-api" } +sr-primitives = { path = "../../primitives/sr-primitives", default-features = false } +state-machine = { package = "substrate-state-machine", path = "../../primitives/state-machine" } +substrate-telemetry = { path = "../telemetry" } +trie = { package = "substrate-trie", path = "../../primitives/trie" } + +[dev-dependencies] +env_logger = "0.7.0" +tempfile = "3.1.0" +client-db = { package = "substrate-client-db", path = ".././db", features = ["kvdb-rocksdb"] } +test-primitives = { package = "substrate-test-primitives", path = "../../test/utils/primitives" } +test-client = { package = "substrate-test-runtime-client", path = "../../test/utils/runtime/client" } +kvdb-memorydb = { git = "https://github.com/paritytech/parity-common", rev="b0317f649ab2c665b7987b8475878fc4d2e1f81d" } +panic-handler = { package = "substrate-panic-handler", path = "../../primitives/panic-handler" } + diff --git a/core/client/src/backend.rs b/client/api/src/backend.rs similarity index 82% rename from core/client/src/backend.rs rename to client/api/src/backend.rs index 9b6d9ce58fbfe..03cd183ad2c21 100644 --- a/core/client/src/backend.rs +++ b/client/api/src/backend.rs @@ -18,14 +18,19 @@ use std::sync::Arc; use std::collections::HashMap; -use crate::error; -use crate::light::blockchain::RemoteBlockchain; use primitives::ChangesTrieConfiguration; use sr_primitives::{generic::BlockId, Justification, StorageOverlay, ChildrenStorageOverlay}; use sr_primitives::traits::{Block as BlockT, NumberFor}; use state_machine::backend::Backend as StateBackend; use state_machine::{ChangesTrieStorage as StateChangesTrieStorage, ChangesTrieTransaction}; -use crate::blockchain::well_known_cache_keys; +use crate::{ + blockchain::{ + Backend as BlockchainBackend, well_known_cache_keys + }, + offchain::OffchainStorage, + error, + light::RemoteBlockchain, +}; use consensus::BlockOrigin; use hash_db::Hasher; use parking_lot::Mutex; @@ -36,13 +41,13 @@ pub type StorageCollection = Vec<(Vec, Option>)>; /// In memory arrays of storage values for multiple child tries. pub type ChildStorageCollection = Vec<(Vec, StorageCollection)>; -pub(crate) struct ImportSummary { - pub(crate) hash: Block::Hash, - pub(crate) origin: BlockOrigin, - pub(crate) header: Block::Header, - pub(crate) is_new_best: bool, - pub(crate) storage_changes: Option<(StorageCollection, ChildStorageCollection)>, - pub(crate) retracted: Vec, +pub struct ImportSummary { + pub hash: Block::Hash, + pub origin: BlockOrigin, + pub header: Block::Header, + pub is_new_best: bool, + pub storage_changes: Option<(StorageCollection, ChildStorageCollection)>, + pub retracted: Vec, } /// Import operation wrapper @@ -51,9 +56,9 @@ pub struct ClientImportOperation< H: Hasher, B: Backend, > { - pub(crate) op: B::BlockImportOperation, - pub(crate) notify_imported: Option>, - pub(crate) notify_finalized: Vec, + pub op: B::BlockImportOperation, + pub notify_imported: Option>, + pub notify_finalized: Vec, } /// State of a new block. @@ -85,7 +90,9 @@ impl NewBlockState { } } -/// Block insertion operation. Keeps hold if the inserted block state and data. +/// Block insertion operation. +/// +/// Keeps hold if the inserted block state and data. pub trait BlockImportOperation where Block: BlockT, H: Hasher, @@ -93,8 +100,11 @@ pub trait BlockImportOperation where /// Associated state backend type. type State: StateBackend; - /// Returns pending state. Returns None for backends with locally-unavailable state data. + /// Returns pending state. + /// + /// Returns None for backends with locally-unavailable state data. fn state(&self) -> error::Result>; + /// Append block data to the transaction. fn set_block_data( &mut self, @@ -106,21 +116,29 @@ pub trait BlockImportOperation where /// Update cached data. fn update_cache(&mut self, cache: HashMap>); + /// Inject storage data into the database. fn update_db_storage(&mut self, update: >::Transaction) -> error::Result<()>; + /// Inject storage data into the database replacing any existing data. fn reset_storage(&mut self, top: StorageOverlay, children: ChildrenStorageOverlay) -> error::Result; + /// Set storage changes. fn update_storage( &mut self, update: StorageCollection, child_update: ChildStorageCollection, ) -> error::Result<()>; + /// Inject changes trie data into the database. fn update_changes_trie(&mut self, update: ChangesTrieTransaction>) -> error::Result<()>; - /// Insert auxiliary keys. Values are `None` if should be deleted. + + /// Insert auxiliary keys. + /// + /// Values are `None` if should be deleted. fn insert_aux(&mut self, ops: I) -> error::Result<()> where I: IntoIterator, Option>)>; + /// Mark a block as finalized. fn mark_finalized(&mut self, id: BlockId, justification: Option) -> error::Result<()>; /// Mark a block as new head. If both block import and set head are specified, set head overrides block import's best block rule. @@ -129,8 +147,9 @@ pub trait BlockImportOperation where /// Finalize Facilities pub trait Finalizer, B: Backend> { - /// Mark all blocks up to given as finalized in operation. If a - /// justification is provided it is stored with the given finalized + /// Mark all blocks up to given as finalized in operation. + /// + /// If `justification` is provided it is stored with the given finalized /// block (any other finalized blocks are left unjustified). /// /// If the block being finalized is on a different fork from the current @@ -146,7 +165,9 @@ pub trait Finalizer, B: Backend error::Result<()>; - /// Finalize a block. This will implicitly finalize all blocks up to it and + /// Finalize a block. + /// + /// This will implicitly finalize all blocks up to it and /// fire finality notifications. /// /// If the block being finalized is on a different fork from the current @@ -168,7 +189,9 @@ pub trait Finalizer, B: Backend, D: IntoIterator, >(&self, insert: I, delete: D) -> error::Result<()>; + /// Query auxiliary data from key-value store. fn get_aux(&self, key: &[u8]) -> error::Result>>; } -/// Client backend. Manages the data layer. +/// Client backend. +/// +/// Manages the data layer. /// /// Note on state pruning: while an object from `state_at` is alive, the state /// should not be pruned. The backend should internally reference-count @@ -195,7 +221,7 @@ pub trait Backend: AuxStore + Send + Sync where /// Associated block insertion operation type. type BlockImportOperation: BlockImportOperation; /// Associated blockchain backend type. - type Blockchain: crate::blockchain::Backend; + type Blockchain: BlockchainBackend; /// Associated state backend type. type State: StateBackend; /// Changes trie storage. @@ -204,35 +230,49 @@ pub trait Backend: AuxStore + Send + Sync where type OffchainStorage: OffchainStorage; /// Begin a new block insertion transaction with given parent block id. + /// /// When constructing the genesis, this is called with all-zero hash. fn begin_operation(&self) -> error::Result; + /// Note an operation to contain state transition. fn begin_state_operation(&self, operation: &mut Self::BlockImportOperation, block: BlockId) -> error::Result<()>; + /// Commit block insertion. fn commit_operation(&self, transaction: Self::BlockImportOperation) -> error::Result<()>; - /// Finalize block with given Id. This should only be called if the parent of the given - /// block has been finalized. + + /// Finalize block with given Id. + /// + /// This should only be called if the parent of the given block has been finalized. fn finalize_block(&self, block: BlockId, justification: Option) -> error::Result<()>; + /// Returns reference to blockchain backend. fn blockchain(&self) -> &Self::Blockchain; + /// Returns the used state cache, if existent. fn used_state_cache_size(&self) -> Option; + /// Returns reference to changes trie storage. fn changes_trie_storage(&self) -> Option<&Self::ChangesTrieStorage>; + /// Returns a handle to offchain storage. fn offchain_storage(&self) -> Option; + /// Returns true if state for given block is available. fn have_state_at(&self, hash: &Block::Hash, _number: NumberFor) -> bool { self.state_at(BlockId::Hash(hash.clone())).is_ok() } + /// Returns state backend with post-state of given block. fn state_at(&self, block: BlockId) -> error::Result; + /// Destroy state and save any useful data, such as cache. fn destroy_state(&self, _state: Self::State) -> error::Result<()> { Ok(()) } - /// Attempts to revert the chain by `n` blocks. Returns the number of blocks that were - /// successfully reverted. + + /// Attempts to revert the chain by `n` blocks. + /// + /// Returns the number of blocks that were successfully reverted. fn revert(&self, n: NumberFor) -> error::Result>; /// Insert auxiliary data into key-value store. @@ -252,6 +292,7 @@ pub trait Backend: AuxStore + Send + Sync where } /// Gain access to the import lock around this backend. + /// /// _Note_ Backend isn't expected to acquire the lock by itself ever. Rather /// the using components should acquire and hold the lock whenever they do /// something that the import of a block would interfere with, e.g. importing @@ -259,26 +300,6 @@ pub trait Backend: AuxStore + Send + Sync where fn get_import_lock(&self) -> &Mutex<()>; } -/// Offchain workers local storage. -pub trait OffchainStorage: Clone + Send + Sync { - /// Persist a value in storage under given key and prefix. - fn set(&mut self, prefix: &[u8], key: &[u8], value: &[u8]); - - /// Retrieve a value from storage under given key and prefix. - fn get(&self, prefix: &[u8], key: &[u8]) -> Option>; - - /// Replace the value in storage if given old_value matches the current one. - /// - /// Returns `true` if the value has been set and false otherwise. - fn compare_and_set( - &mut self, - prefix: &[u8], - key: &[u8], - old_value: Option<&[u8]>, - new_value: &[u8], - ) -> bool; -} - /// Changes trie storage that supports pruning. pub trait PrunableStateChangesTrieStorage: StateChangesTrieStorage> @@ -306,7 +327,10 @@ where { /// Returns true if the state for given block is available locally. fn is_local_state_available(&self, block: &BlockId) -> bool; - /// Returns reference to blockchain backend that either resolves blockchain data + + /// Returns reference to blockchain backend. + /// + /// Returned backend either resolves blockchain data /// locally, or prepares request to fetch that data from remote node. fn remote_blockchain(&self) -> Arc>; } diff --git a/core/client/src/blockchain.rs b/client/api/src/blockchain.rs similarity index 100% rename from core/client/src/blockchain.rs rename to client/api/src/blockchain.rs diff --git a/client/api/src/call_executor.rs b/client/api/src/call_executor.rs new file mode 100644 index 0000000000000..c143999047d5e --- /dev/null +++ b/client/api/src/call_executor.rs @@ -0,0 +1,148 @@ +// Copyright 2017-2019 Parity Technologies (UK) Ltd. +// This file is part of Substrate. + +// Substrate is free software: you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. + +// Substrate is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. + +// You should have received a copy of the GNU General Public License +// along with Substrate. If not, see . + +use std::{cmp::Ord, panic::UnwindSafe, result, cell::RefCell}; +use codec::{Encode, Decode}; +use sr_primitives::{ + generic::BlockId, traits::Block as BlockT, traits::NumberFor, +}; +use state_machine::{ + self, OverlayedChanges, ExecutionManager, ExecutionStrategy, + ChangesTrieTransaction, StorageProof, +}; +use executor::{RuntimeVersion, NativeVersion}; +use hash_db::Hasher; +use primitives::{offchain::OffchainExt, Blake2Hasher, NativeOrEncoded}; + +use sr_api::{ProofRecorder, InitializeBlock}; +use crate::error; + +/// Method call executor. +pub trait CallExecutor +where + B: BlockT, + H: Hasher, + H::Out: Ord, +{ + /// Externalities error type. + type Error: state_machine::Error; + + /// Execute a call to a contract on top of state in a block of given hash. + /// + /// No changes are made. + fn call( + &self, + id: &BlockId, + method: &str, + call_data: &[u8], + strategy: ExecutionStrategy, + side_effects_handler: Option, + ) -> Result, error::Error>; + + /// Execute a contextual call on top of state in a block of a given hash. + /// + /// No changes are made. + /// Before executing the method, passed header is installed as the current header + /// of the execution context. + fn contextual_call< + 'a, + IB: Fn() -> error::Result<()>, + EM: Fn( + Result, Self::Error>, + Result, Self::Error> + ) -> Result, Self::Error>, + R: Encode + Decode + PartialEq, + NC: FnOnce() -> result::Result + UnwindSafe, + >( + &self, + initialize_block_fn: IB, + at: &BlockId, + method: &str, + call_data: &[u8], + changes: &RefCell, + initialize_block: InitializeBlock<'a, B>, + execution_manager: ExecutionManager, + native_call: Option, + side_effects_handler: Option, + proof_recorder: &Option>, + enable_keystore: bool, + ) -> error::Result> where ExecutionManager: Clone; + + /// Extract RuntimeVersion of given block + /// + /// No changes are made. + fn runtime_version(&self, id: &BlockId) -> Result; + + /// Execute a call to a contract on top of given state. + /// + /// No changes are made. + fn call_at_state< + S: state_machine::Backend, + F: FnOnce( + Result, Self::Error>, + Result, Self::Error>, + ) -> Result, Self::Error>, + R: Encode + Decode + PartialEq, + NC: FnOnce() -> result::Result + UnwindSafe, + >(&self, + state: &S, + overlay: &mut OverlayedChanges, + method: &str, + call_data: &[u8], + manager: ExecutionManager, + native_call: Option, + side_effects_handler: Option, + ) -> Result< + ( + NativeOrEncoded, + (S::Transaction, H::Out), + Option>> + ), + error::Error, + >; + + /// Execute a call to a contract on top of given state, gathering execution proof. + /// + /// No changes are made. + fn prove_at_state>( + &self, + mut state: S, + overlay: &mut OverlayedChanges, + method: &str, + call_data: &[u8] + ) -> Result<(Vec, StorageProof), error::Error> { + let trie_state = state.as_trie_backend() + .ok_or_else(|| + Box::new(state_machine::ExecutionError::UnableToGenerateProof) + as Box + )?; + self.prove_at_trie_state(trie_state, overlay, method, call_data) + } + + /// Execute a call to a contract on top of given trie state, gathering execution proof. + /// + /// No changes are made. + fn prove_at_trie_state>( + &self, + trie_state: &state_machine::TrieBackend, + overlay: &mut OverlayedChanges, + method: &str, + call_data: &[u8] + ) -> Result<(Vec, StorageProof), error::Error>; + + /// Get runtime version if supported. + fn native_runtime_version(&self) -> Option<&NativeVersion>; +} diff --git a/client/api/src/client.rs b/client/api/src/client.rs new file mode 100644 index 0000000000000..a51fc9f03f875 --- /dev/null +++ b/client/api/src/client.rs @@ -0,0 +1,141 @@ +// Copyright 2019 Parity Technologies (UK) Ltd. +// This file is part of Substrate. + +// Substrate is free software: you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. + +// Substrate is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. + +// You should have received a copy of the GNU General Public License +// along with Substrate. If not, see . + +use std::collections::HashMap; +use futures03::channel::mpsc; +use primitives::storage::StorageKey; +use state_machine::ExecutionStrategy; +use sr_primitives::{ + traits::{Block as BlockT, NumberFor}, + generic::BlockId +}; +use consensus::BlockOrigin; + +use crate::blockchain::Info; +use crate::notifications::StorageEventStream; +use crate::error; + +/// Type that implements `futures::Stream` of block import events. +pub type ImportNotifications = mpsc::UnboundedReceiver>; + +/// A stream of block finality notifications. +pub type FinalityNotifications = mpsc::UnboundedReceiver>; + +/// Expected hashes of blocks at given heights. +/// +/// This may be used as chain spec extension to filter out known, unwanted forks. +pub type ForkBlocks = Option, ::Hash>>; + +/// Execution strategies settings. +#[derive(Debug, Clone)] +pub struct ExecutionStrategies { + /// Execution strategy used when syncing. + pub syncing: ExecutionStrategy, + /// Execution strategy used when importing blocks. + pub importing: ExecutionStrategy, + /// Execution strategy used when constructing blocks. + pub block_construction: ExecutionStrategy, + /// Execution strategy used for offchain workers. + pub offchain_worker: ExecutionStrategy, + /// Execution strategy used in other cases. + pub other: ExecutionStrategy, +} + +impl Default for ExecutionStrategies { + fn default() -> ExecutionStrategies { + ExecutionStrategies { + syncing: ExecutionStrategy::NativeElseWasm, + importing: ExecutionStrategy::NativeElseWasm, + block_construction: ExecutionStrategy::AlwaysWasm, + offchain_worker: ExecutionStrategy::NativeWhenPossible, + other: ExecutionStrategy::NativeElseWasm, + } + } +} + +/// Figure out the block type for a given type (for now, just a `Client`). +pub trait BlockOf { + /// The type of the block. + type Type: BlockT; +} + +/// A source of blockchain events. +pub trait BlockchainEvents { + /// Get block import event stream. Not guaranteed to be fired for every + /// imported block. + fn import_notification_stream(&self) -> ImportNotifications; + + /// Get a stream of finality notifications. Not guaranteed to be fired for every + /// finalized block. + fn finality_notification_stream(&self) -> FinalityNotifications; + + /// Get storage changes event stream. + /// + /// Passing `None` as `filter_keys` subscribes to all storage changes. + fn storage_changes_notification_stream( + &self, + filter_keys: Option<&[StorageKey]>, + child_filter_keys: Option<&[(StorageKey, Option>)]>, + ) -> error::Result>; +} + +/// Fetch block body by ID. +pub trait BlockBody { + /// Get block body by ID. Returns `None` if the body is not stored. + fn block_body(&self, + id: &BlockId + ) -> error::Result::Extrinsic>>>; +} + +/// Provide a list of potential uncle headers for a given block. +pub trait ProvideUncles { + /// Gets the uncles of the block with `target_hash` going back `max_generation` ancestors. + fn uncles(&self, target_hash: Block::Hash, max_generation: NumberFor) + -> error::Result>; +} + +/// Client info +#[derive(Debug)] +pub struct ClientInfo { + /// Best block hash. + pub chain: Info, + /// State Cache Size currently used by the backend + pub used_state_cache_size: Option, +} + +/// Summary of an imported block +#[derive(Clone, Debug)] +pub struct BlockImportNotification { + /// Imported block header hash. + pub hash: Block::Hash, + /// Imported block origin. + pub origin: BlockOrigin, + /// Imported block header. + pub header: Block::Header, + /// Is this the new best block. + pub is_new_best: bool, + /// List of retracted blocks ordered by block number. + pub retracted: Vec, +} + +/// Summary of a finalized block. +#[derive(Clone, Debug)] +pub struct FinalityNotification { + /// Imported block header hash. + pub hash: Block::Hash, + /// Imported block header. + pub header: Block::Header, +} diff --git a/core/client/src/error.rs b/client/api/src/error.rs similarity index 96% rename from core/client/src/error.rs rename to client/api/src/error.rs index 922d122b42a8b..a695e17a4256c 100644 --- a/core/client/src/error.rs +++ b/client/api/src/error.rs @@ -124,6 +124,12 @@ impl<'a> From<&'a str> for Error { } } +impl From for Error { + fn from(err: block_builder::ApplyExtrinsicFailed) -> Self { + Self::ApplyExtrinsicFailed(err.0) + } +} + impl Error { /// Chain a blockchain error. pub fn from_blockchain(e: Box) -> Self { diff --git a/client/api/src/lib.rs b/client/api/src/lib.rs new file mode 100644 index 0000000000000..4163320546db4 --- /dev/null +++ b/client/api/src/lib.rs @@ -0,0 +1,84 @@ +// Copyright 2019 Parity Technologies (UK) Ltd. +// This file is part of Substrate. + +// Substrate is free software: you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. + +// Substrate is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. + +// You should have received a copy of the GNU General Public License +// along with Substrate. If not, see . + +//! Substrate client interfaces. + +// TODO: make internal +pub mod error; +pub mod backend; +pub mod blockchain; +pub mod light; +pub mod notifications; +pub mod call_executor; +pub mod client; +pub mod offchain; + +pub use error::*; +pub use backend::*; +pub use blockchain::*; +pub use light::*; +pub use notifications::*; +pub use call_executor::*; +pub use offchain::*; +pub use client::*; + +pub use state_machine::{StorageProof, ExecutionStrategy}; + + +/// Utility methods for the client. +pub mod utils { + use super::HeaderBackend; + use header_metadata::HeaderMetadata; + use crate::error; + use primitives::H256; + use sr_primitives::traits::{Block as BlockT}; + use std::borrow::Borrow; + + /// Returns a function for checking block ancestry, the returned function will + /// return `true` if the given hash (second parameter) is a descendent of the + /// base (first parameter). If the `current` parameter is defined, it should + /// represent the current block `hash` and its `parent hash`, if given the + /// function that's returned will assume that `hash` isn't part of the local DB + /// yet, and all searches in the DB will instead reference the parent. + pub fn is_descendent_of<'a, Block: BlockT, T, H: Borrow + 'a>( + client: &'a T, + current: Option<(H, H)>, + ) -> impl Fn(&H256, &H256) -> Result + 'a + where T: HeaderBackend + HeaderMetadata, + { + move |base, hash| { + if base == hash { return Ok(false); } + + let current = current.as_ref().map(|(c, p)| (c.borrow(), p.borrow())); + + let mut hash = hash; + if let Some((current_hash, current_parent_hash)) = current { + if base == current_hash { return Ok(false); } + if hash == current_hash { + if base == current_parent_hash { + return Ok(true); + } else { + hash = current_parent_hash; + } + } + } + + let ancestor = header_metadata::lowest_common_ancestor(client, *hash, *base)?; + + Ok(ancestor.hash == *base) + } + } +} diff --git a/client/api/src/light.rs b/client/api/src/light.rs new file mode 100644 index 0000000000000..b82f8ba745912 --- /dev/null +++ b/client/api/src/light.rs @@ -0,0 +1,335 @@ +// Copyright 2019 Parity Technologies (UK) Ltd. +// This file is part of Substrate. + +// Substrate is free software: you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. + +// Substrate is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. + +// You should have received a copy of the GNU General Public License +// along with Substrate. If not, see . + +//! Substrate light client interfaces + +use std::sync::Arc; +use std::collections::{BTreeMap, HashMap}; +use std::future::Future; + +use sr_primitives::{ + traits::{ + Block as BlockT, Header as HeaderT, NumberFor, + }, + generic::BlockId +}; +use primitives::{ChangesTrieConfiguration}; +use state_machine::StorageProof; +use header_metadata::HeaderMetadata; +use crate::{ + backend::{ + AuxStore, NewBlockState, + }, + blockchain::{ + well_known_cache_keys, HeaderBackend, Cache as BlockchainCache, + }, + error::{ Error as ClientError, Result as ClientResult }, +}; +/// Remote call request. +#[derive(Clone, Debug, PartialEq, Eq, Hash)] +pub struct RemoteCallRequest { + /// Call at state of given block. + pub block: Header::Hash, + /// Header of block at which call is performed. + pub header: Header, + /// Method to call. + pub method: String, + /// Call data. + pub call_data: Vec, + /// Number of times to retry request. None means that default RETRY_COUNT is used. + pub retry_count: Option, +} + +/// Remote canonical header request. +#[derive(Clone, Debug, Default, PartialEq, Eq, Hash)] +pub struct RemoteHeaderRequest { + /// The root of CHT this block is included in. + pub cht_root: Header::Hash, + /// Number of the header to query. + pub block: Header::Number, + /// Number of times to retry request. None means that default RETRY_COUNT is used. + pub retry_count: Option, +} + +/// Remote storage read request. +#[derive(Clone, Debug, PartialEq, Eq, Hash)] +pub struct RemoteReadRequest { + /// Read at state of given block. + pub block: Header::Hash, + /// Header of block at which read is performed. + pub header: Header, + /// Storage key to read. + pub keys: Vec>, + /// Number of times to retry request. None means that default RETRY_COUNT is used. + pub retry_count: Option, +} + +/// Remote storage read child request. +#[derive(Clone, Debug, PartialEq, Eq, Hash)] +pub struct RemoteReadChildRequest { + /// Read at state of given block. + pub block: Header::Hash, + /// Header of block at which read is performed. + pub header: Header, + /// Storage key for child. + pub storage_key: Vec, + /// Child storage key to read. + pub keys: Vec>, + /// Number of times to retry request. None means that default RETRY_COUNT is used. + pub retry_count: Option, +} + +/// Remote key changes read request. +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct RemoteChangesRequest { + /// Changes trie configuration. + pub changes_trie_config: ChangesTrieConfiguration, + /// Query changes from range of blocks, starting (and including) with this hash... + pub first_block: (Header::Number, Header::Hash), + /// ...ending (and including) with this hash. Should come after first_block and + /// be the part of the same fork. + pub last_block: (Header::Number, Header::Hash), + /// Only use digests from blocks up to this hash. Should be last_block OR come + /// after this block and be the part of the same fork. + pub max_block: (Header::Number, Header::Hash), + /// Known changes trie roots for the range of blocks [tries_roots.0..max_block]. + /// Proofs for roots of ascendants of tries_roots.0 are provided by the remote node. + pub tries_roots: (Header::Number, Header::Hash, Vec), + /// Optional Child Storage key to read. + pub storage_key: Option>, + /// Storage key to read. + pub key: Vec, + /// Number of times to retry request. None means that default RETRY_COUNT is used. + pub retry_count: Option, +} + +/// Key changes read proof. +#[derive(Debug, PartialEq, Eq)] +pub struct ChangesProof { + /// Max block that has been used in changes query. + pub max_block: Header::Number, + /// All touched nodes of all changes tries. + pub proof: Vec>, + /// All changes tries roots that have been touched AND are missing from + /// the requester' node. It is a map of block number => changes trie root. + pub roots: BTreeMap, + /// The proofs for all changes tries roots that have been touched AND are + /// missing from the requester' node. It is a map of CHT number => proof. + pub roots_proof: StorageProof, +} + +/// Remote block body request +#[derive(Clone, Default, Debug, PartialEq, Eq, Hash)] +pub struct RemoteBodyRequest { + /// Header of the requested block body + pub header: Header, + /// Number of times to retry request. None means that default RETRY_COUNT is used. + pub retry_count: Option, +} + +/// Light client data fetcher. Implementations of this trait must check if remote data +/// is correct (see FetchedDataChecker) and return already checked data. +pub trait Fetcher: Send + Sync { + /// Remote header future. + type RemoteHeaderResult: Future> + Send + 'static; + /// Remote storage read future. + type RemoteReadResult: Future, Option>>, ClientError>> + Send + 'static; + /// Remote call result future. + type RemoteCallResult: Future, ClientError>> + Send + 'static; + /// Remote changes result future. + type RemoteChangesResult: Future, u32)>, ClientError>> + Send + 'static; + /// Remote block body result future. + type RemoteBodyResult: Future, ClientError>> + Send + 'static; + + /// Fetch remote header. + fn remote_header(&self, request: RemoteHeaderRequest) -> Self::RemoteHeaderResult; + /// Fetch remote storage value. + fn remote_read( + &self, + request: RemoteReadRequest + ) -> Self::RemoteReadResult; + /// Fetch remote storage child value. + fn remote_read_child( + &self, + request: RemoteReadChildRequest + ) -> Self::RemoteReadResult; + /// Fetch remote call result. + fn remote_call(&self, request: RemoteCallRequest) -> Self::RemoteCallResult; + /// Fetch remote changes ((block number, extrinsic index)) where given key has been changed + /// at a given blocks range. + fn remote_changes(&self, request: RemoteChangesRequest) -> Self::RemoteChangesResult; + /// Fetch remote block body + fn remote_body(&self, request: RemoteBodyRequest) -> Self::RemoteBodyResult; +} + +/// Light client remote data checker. +/// +/// Implementations of this trait should not use any prunable blockchain data +/// except that is passed to its methods. +pub trait FetchChecker: Send + Sync { + /// Check remote header proof. + fn check_header_proof( + &self, + request: &RemoteHeaderRequest, + header: Option, + remote_proof: StorageProof, + ) -> ClientResult; + /// Check remote storage read proof. + fn check_read_proof( + &self, + request: &RemoteReadRequest, + remote_proof: StorageProof, + ) -> ClientResult, Option>>>; + /// Check remote storage read proof. + fn check_read_child_proof( + &self, + request: &RemoteReadChildRequest, + remote_proof: StorageProof, + ) -> ClientResult, Option>>>; + /// Check remote method execution proof. + fn check_execution_proof( + &self, + request: &RemoteCallRequest, + remote_proof: StorageProof, + ) -> ClientResult>; + /// Check remote changes query proof. + fn check_changes_proof( + &self, + request: &RemoteChangesRequest, + proof: ChangesProof + ) -> ClientResult, u32)>>; + /// Check remote body proof. + fn check_body_proof( + &self, + request: &RemoteBodyRequest, + body: Vec + ) -> ClientResult>; +} + + +/// Light client blockchain storage. +pub trait Storage: AuxStore + HeaderBackend + HeaderMetadata { + /// Store new header. Should refuse to revert any finalized blocks. + /// + /// Takes new authorities, the leaf state of the new block, and + /// any auxiliary storage updates to place in the same operation. + fn import_header( + &self, + header: Block::Header, + cache: HashMap>, + state: NewBlockState, + aux_ops: Vec<(Vec, Option>)>, + ) -> ClientResult<()>; + + /// Set an existing block as new best block. + fn set_head(&self, block: BlockId) -> ClientResult<()>; + + /// Mark historic header as finalized. + fn finalize_header(&self, block: BlockId) -> ClientResult<()>; + + /// Get last finalized header. + fn last_finalized(&self) -> ClientResult; + + /// Get headers CHT root for given block. Fails if the block is not pruned (not a part of any CHT). + fn header_cht_root( + &self, + cht_size: NumberFor, + block: NumberFor, + ) -> ClientResult; + + /// Get changes trie CHT root for given block. Fails if the block is not pruned (not a part of any CHT). + fn changes_trie_cht_root( + &self, + cht_size: NumberFor, + block: NumberFor, + ) -> ClientResult; + + /// Get storage cache. + fn cache(&self) -> Option>>; +} + +/// Remote header. +#[derive(Debug)] +pub enum LocalOrRemote { + /// When data is available locally, it is returned. + Local(Data), + /// When data is unavailable locally, the request to fetch it from remote node is returned. + Remote(Request), + /// When data is unknown. + Unknown, +} + +/// Futures-based blockchain backend that either resolves blockchain data +/// locally, or fetches required data from remote node. +pub trait RemoteBlockchain: Send + Sync { + /// Get block header. + fn header(&self, id: BlockId) -> ClientResult, + >>; +} + + + +#[cfg(test)] +pub mod tests { + use futures03::future::Ready; + use parking_lot::Mutex; + use crate::error::Error as ClientError; + use test_primitives::{Block, Header, Extrinsic}; + use super::*; + + pub type OkCallFetcher = Mutex>; + + fn not_implemented_in_tests() -> Ready> + where + E: std::convert::From<&'static str>, + { + futures03::future::ready(Err("Not implemented on test node".into())) + } + + impl Fetcher for OkCallFetcher { + type RemoteHeaderResult = Ready>; + type RemoteReadResult = Ready, Option>>, ClientError>>; + type RemoteCallResult = Ready, ClientError>>; + type RemoteChangesResult = Ready, u32)>, ClientError>>; + type RemoteBodyResult = Ready, ClientError>>; + + fn remote_header(&self, _request: RemoteHeaderRequest
) -> Self::RemoteHeaderResult { + not_implemented_in_tests() + } + + fn remote_read(&self, _request: RemoteReadRequest
) -> Self::RemoteReadResult { + not_implemented_in_tests() + } + + fn remote_read_child(&self, _request: RemoteReadChildRequest
) -> Self::RemoteReadResult { + not_implemented_in_tests() + } + + fn remote_call(&self, _request: RemoteCallRequest
) -> Self::RemoteCallResult { + futures03::future::ready(Ok((*self.lock()).clone())) + } + + fn remote_changes(&self, _request: RemoteChangesRequest
) -> Self::RemoteChangesResult { + not_implemented_in_tests() + } + + fn remote_body(&self, _request: RemoteBodyRequest
) -> Self::RemoteBodyResult { + not_implemented_in_tests() + } + } +} \ No newline at end of file diff --git a/core/client/src/notifications.rs b/client/api/src/notifications.rs similarity index 100% rename from core/client/src/notifications.rs rename to client/api/src/notifications.rs diff --git a/client/api/src/offchain.rs b/client/api/src/offchain.rs new file mode 100644 index 0000000000000..761241efe7996 --- /dev/null +++ b/client/api/src/offchain.rs @@ -0,0 +1,77 @@ +// Copyright 2019 Parity Technologies (UK) Ltd. +// This file is part of Substrate. + +// Substrate is free software: you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. + +// Substrate is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. + +// You should have received a copy of the GNU General Public License +// along with Substrate. If not, see . + +use std::collections::hash_map::{HashMap, Entry}; + +/// Offchain workers local storage. +pub trait OffchainStorage: Clone + Send + Sync { + /// Persist a value in storage under given key and prefix. + fn set(&mut self, prefix: &[u8], key: &[u8], value: &[u8]); + + /// Retrieve a value from storage under given key and prefix. + fn get(&self, prefix: &[u8], key: &[u8]) -> Option>; + + /// Replace the value in storage if given old_value matches the current one. + /// + /// Returns `true` if the value has been set and false otherwise. + fn compare_and_set( + &mut self, + prefix: &[u8], + key: &[u8], + old_value: Option<&[u8]>, + new_value: &[u8], + ) -> bool; +} + +/// In-memory storage for offchain workers. +#[derive(Debug, Clone, Default)] +pub struct InMemOffchainStorage { + storage: HashMap, Vec>, +} + +impl OffchainStorage for InMemOffchainStorage { + fn set(&mut self, prefix: &[u8], key: &[u8], value: &[u8]) { + let key = prefix.iter().chain(key).cloned().collect(); + self.storage.insert(key, value.to_vec()); + } + + fn get(&self, prefix: &[u8], key: &[u8]) -> Option> { + let key: Vec = prefix.iter().chain(key).cloned().collect(); + self.storage.get(&key).cloned() + } + + fn compare_and_set( + &mut self, + prefix: &[u8], + key: &[u8], + old_value: Option<&[u8]>, + new_value: &[u8], + ) -> bool { + let key = prefix.iter().chain(key).cloned().collect(); + + match self.storage.entry(key) { + Entry::Vacant(entry) => if old_value.is_none() { + entry.insert(new_value.to_vec()); + true + } else { false }, + Entry::Occupied(ref mut entry) if Some(entry.get().as_slice()) == old_value => { + entry.insert(new_value.to_vec()); + true + }, + _ => false, + } + } +} \ No newline at end of file diff --git a/client/authority-discovery/Cargo.toml b/client/authority-discovery/Cargo.toml new file mode 100644 index 0000000000000..e9c918500bb74 --- /dev/null +++ b/client/authority-discovery/Cargo.toml @@ -0,0 +1,32 @@ +[package] +name = "substrate-authority-discovery" +version = "2.0.0" +authors = ["Parity Technologies "] +edition = "2018" +build = "build.rs" + +[build-dependencies] +prost-build = "0.5.0" + +[dependencies] +authority-discovery-primitives = { package = "substrate-authority-discovery-primitives", path = "../../primitives/authority-discovery" } +bytes = "0.4.12" +client-api = { package = "substrate-client-api", path = "../api" } +codec = { package = "parity-scale-codec", default-features = false, version = "1.0.3" } +derive_more = "0.15.0" +futures-preview = "0.3.0-alpha.19" +futures-timer = "0.4" +keystore = { package = "substrate-keystore", path = "../keystore" } +libp2p = { version = "0.13.0", default-features = false, features = ["secp256k1", "libp2p-websocket"] } +log = "0.4.8" +network = { package = "substrate-network", path = "../network" } +primitives = { package = "substrate-primitives", path = "../../primitives/core" } +prost = "0.5.0" +serde_json = "1.0.41" +sr-primitives = { path = "../../primitives/sr-primitives" } + +[dev-dependencies] +parking_lot = "0.9.0" +peerset = { package = "substrate-peerset", path = "../../primitives/peerset" } +test-client = { package = "substrate-test-runtime-client", path = "../../test/utils/runtime/client" } +sr-api = { path = "../../primitives/sr-api" } diff --git a/core/authority-discovery/build.rs b/client/authority-discovery/build.rs similarity index 100% rename from core/authority-discovery/build.rs rename to client/authority-discovery/build.rs diff --git a/core/authority-discovery/src/error.rs b/client/authority-discovery/src/error.rs similarity index 84% rename from core/authority-discovery/src/error.rs rename to client/authority-discovery/src/error.rs index e8c1ad9705f0c..a05bc3b68005d 100644 --- a/core/authority-discovery/src/error.rs +++ b/client/authority-discovery/src/error.rs @@ -27,21 +27,19 @@ pub enum Error { /// Failed to hash the authority id to be used as a dht key. HashingAuthorityId(libp2p::core::multiaddr::multihash::EncodeError), /// Failed calling into the Substrate runtime. - CallingRuntime(client::error::Error), - /// Failed signing the dht payload via the Substrate runtime. - SigningDhtPayload, + CallingRuntime(client_api::error::Error), /// From the Dht we only get the hashed authority id. In order to retrieve the actual authority id and to ensure it /// is actually an authority, we match the hash against the hash of the authority id of all other authorities. This /// error is the result of the above failing. MatchingHashedAuthorityIdWithAuthorityId, /// Failed to set the authority discovery peerset priority group in the peerset module. SettingPeersetPriorityGroup(String), - /// Failed to encode a dht payload. - Encoding(prost::EncodeError), - /// Failed to decode a dht payload. - Decoding(prost::DecodeError), + /// Failed to encode a protobuf payload. + EncodingProto(prost::EncodeError), + /// Failed to decode a protobuf payload. + DecodingProto(prost::DecodeError), + /// Failed to encode or decode scale payload + EncodingDecodingScale(codec::Error), /// Failed to parse a libp2p multi address. ParsingMultiaddress(libp2p::core::multiaddr::Error), - /// Tokio timer error. - PollingTokioTimer(tokio_timer::Error) } diff --git a/core/authority-discovery/src/lib.rs b/client/authority-discovery/src/lib.rs similarity index 66% rename from core/authority-discovery/src/lib.rs rename to client/authority-discovery/src/lib.rs index 987169ead90b1..57805d0add18b 100644 --- a/core/authority-discovery/src/lib.rs +++ b/client/authority-discovery/src/lib.rs @@ -42,23 +42,31 @@ //! 3. Validates the signatures of the retrieved key value pairs. //! //! 4. Adds the retrieved external addresses as priority nodes to the peerset. +use std::collections::{HashMap, HashSet}; +use std::convert::TryInto; +use std::iter::FromIterator; +use std::marker::PhantomData; +use std::pin::Pin; +use std::sync::Arc; +use std::time::{Duration, Instant}; + +use futures::task::{Context, Poll}; +use futures::Future; +use futures_timer::Interval; +use futures::prelude::*; -use authority_discovery_primitives::{AuthorityDiscoveryApi, AuthorityId, Signature}; -use client::blockchain::HeaderBackend; +use authority_discovery_primitives::{AuthorityDiscoveryApi, AuthorityId, AuthoritySignature, AuthorityPair}; +use client_api::blockchain::HeaderBackend; +use codec::{Decode, Encode}; use error::{Error, Result}; -use futures::{prelude::*, sync::mpsc::Receiver}; use log::{debug, error, log_enabled, warn}; use network::specialization::NetworkSpecialization; use network::{DhtEvent, ExHashT}; +use primitives::crypto::{key_types, Pair}; +use primitives::traits::BareCryptoStorePtr; use prost::Message; use sr_primitives::generic::BlockId; use sr_primitives::traits::{Block as BlockT, ProvideRuntimeApi}; -use std::collections::{HashMap, HashSet}; -use std::convert::TryInto; -use std::iter::FromIterator; -use std::marker::PhantomData; -use std::sync::Arc; -use std::time::{Duration, Instant}; mod error; /// Dht payload schemas generated from Protobuf definitions via Prost crate in build.rs. @@ -66,6 +74,12 @@ mod schema { include!(concat!(env!("OUT_DIR"), "/authority_discovery.rs")); } +/// Upper bound estimation on how long one should wait before accessing the Kademlia DHT. +const LIBP2P_KADEMLIA_BOOTSTRAP_TIME: Duration = Duration::from_secs(30); + +/// Name of the Substrate peerset priority group for authorities discovered through the authority discovery module. +const AUTHORITIES_PRIORITY_GROUP_NAME: &'static str = "authorities"; + /// An `AuthorityDiscovery` makes a given authority discoverable and discovers other authorities. pub struct AuthorityDiscovery where @@ -73,17 +87,20 @@ where Network: NetworkProvider, Client: ProvideRuntimeApi + Send + Sync + 'static + HeaderBackend, ::Api: AuthorityDiscoveryApi, + { client: Arc, network: Arc, /// Channel we receive Dht events on. - dht_event_rx: Receiver, + dht_event_rx: Pin + Send>>, + + key_store: BareCryptoStorePtr, /// Interval to be proactive, publishing own addresses. - publish_interval: tokio_timer::Interval, + publish_interval: Interval, /// Interval on which to query for addresses of other authorities. - query_interval: tokio_timer::Interval, + query_interval: Interval, /// The network peerset interface for priority groups lets us only set an entire group, but we retrieve the /// addresses of other authorities one by one from the network. To use the peerset interface we need to cache the @@ -96,27 +113,33 @@ where impl AuthorityDiscovery where - Block: BlockT + 'static, + Block: BlockT + Unpin + 'static, Network: NetworkProvider, Client: ProvideRuntimeApi + Send + Sync + 'static + HeaderBackend, - ::Api: AuthorityDiscoveryApi, + ::Api: AuthorityDiscoveryApi, + Self: Future, { /// Return a new authority discovery. pub fn new( client: Arc, network: Arc, - dht_event_rx: futures::sync::mpsc::Receiver, - ) -> AuthorityDiscovery { + key_store: BareCryptoStorePtr, + dht_event_rx: Pin + Send>>, + ) -> Self { // Kademlia's default time-to-live for Dht records is 36h, republishing records every 24h. Given that a node // could restart at any point in time, one can not depend on the republishing process, thus publishing own // external addresses should happen on an interval < 36h. - let publish_interval = - tokio_timer::Interval::new(Instant::now(), Duration::from_secs(12 * 60 * 60)); + let publish_interval = Interval::new_at( + Instant::now() + LIBP2P_KADEMLIA_BOOTSTRAP_TIME, + Duration::from_secs(12 * 60 * 60), + ); // External addresses of other authorities can change at any given point in time. The interval on which to query // for external addresses of other authorities is a trade off between efficiency and performance. - let query_interval = - tokio_timer::Interval::new(Instant::now(), Duration::from_secs(10 * 60)); + let query_interval = Interval::new_at( + Instant::now() + LIBP2P_KADEMLIA_BOOTSTRAP_TIME, + Duration::from_secs(10 * 60), + ); let address_cache = HashMap::new(); @@ -124,6 +147,7 @@ where client, network, dht_event_rx, + key_store, publish_interval, query_interval, address_cache, @@ -132,8 +156,6 @@ where } fn publish_own_ext_addresses(&mut self) -> Result<()> { - let id = BlockId::hash(self.client.info().best_hash); - let addresses = self .network .external_addresses() @@ -149,27 +171,24 @@ where let mut serialized_addresses = vec![]; schema::AuthorityAddresses { addresses } .encode(&mut serialized_addresses) - .map_err(Error::Encoding)?; + .map_err(Error::EncodingProto)?; - let (signature, authority_id) = self - .client - .runtime_api() - .sign(&id, &serialized_addresses) - .map_err(Error::CallingRuntime)? - .ok_or(Error::SigningDhtPayload)?; + for key in self.get_priv_keys_within_authority_set()?.into_iter() { + let signature = key.sign(&serialized_addresses); - let mut signed_addresses = vec![]; - schema::SignedAuthorityAddresses { - addresses: serialized_addresses, - signature: signature.0, - } - .encode(&mut signed_addresses) - .map_err(Error::Encoding)?; + let mut signed_addresses = vec![]; + schema::SignedAuthorityAddresses { + addresses: serialized_addresses.clone(), + signature: signature.encode(), + } + .encode(&mut signed_addresses) + .map_err(Error::EncodingProto)?; - self.network.put_value( - hash_authority_id(authority_id.0.as_ref())?, - signed_addresses, - ); + self.network.put_value( + hash_authority_id(key.public().as_ref())?, + signed_addresses, + ); + } Ok(()) } @@ -185,14 +204,14 @@ where for authority_id in authorities.iter() { self.network - .get_value(&hash_authority_id(authority_id.0.as_ref())?); + .get_value(&hash_authority_id(authority_id.as_ref())?); } Ok(()) } - fn handle_dht_events(&mut self) -> Result<()> { - while let Ok(Async::Ready(Some(event))) = self.dht_event_rx.poll() { + fn handle_dht_events(&mut self, cx: &mut Context) -> Result<()> { + while let Poll::Ready(Some(event)) = self.dht_event_rx.poll_next_unpin(cx) { match event { DhtEvent::ValueFound(v) => { if log_enabled!(log::Level::Debug) { @@ -202,15 +221,17 @@ where self.handle_dht_value_found_event(v)?; } - DhtEvent::ValueNotFound(hash) => { - warn!(target: "sub-authority-discovery", "Value for hash '{:?}' not found on Dht.", hash) - } - DhtEvent::ValuePut(hash) => { - debug!(target: "sub-authority-discovery", "Successfully put hash '{:?}' on Dht.", hash) - } - DhtEvent::ValuePutFailed(hash) => { - warn!(target: "sub-authority-discovery", "Failed to put hash '{:?}' on Dht.", hash) - } + DhtEvent::ValueNotFound(hash) => warn!( + target: "sub-authority-discovery", + "Value for hash '{:?}' not found on Dht.", hash + ), + DhtEvent::ValuePut(hash) => debug!( + target: "sub-authority-discovery", + "Successfully put hash '{:?}' on Dht.", hash), + DhtEvent::ValuePutFailed(hash) => warn!( + target: "sub-authority-discovery", + "Failed to put hash '{:?}' on Dht.", hash + ), } } @@ -223,16 +244,16 @@ where ) -> Result<()> { debug!(target: "sub-authority-discovery", "Got Dht value from network."); - let id = BlockId::hash(self.client.info().best_hash); + let block_id = BlockId::hash(self.client.info().best_hash); // From the Dht we only get the hashed authority id. In order to retrieve the actual authority id and to ensure // it is actually an authority, we match the hash against the hash of the authority id of all other authorities. - let authorities = self.client.runtime_api().authorities(&id)?; + let authorities = self.client.runtime_api().authorities(&block_id)?; self.purge_old_authorities_from_cache(&authorities); let authorities = authorities .into_iter() - .map(|a| hash_authority_id(a.0.as_ref()).map(|h| (h, a))) + .map(|id| hash_authority_id(id.as_ref()).map(|h| (h, id))) .collect::>>()?; for (key, value) in values.iter() { @@ -244,22 +265,16 @@ where let schema::SignedAuthorityAddresses { signature, addresses, - } = schema::SignedAuthorityAddresses::decode(value).map_err(Error::Decoding)?; - let signature = Signature(signature); - - let is_verified = self - .client - .runtime_api() - .verify(&id, &addresses, &signature, &authority_id.clone()) - .map_err(Error::CallingRuntime)?; + } = schema::SignedAuthorityAddresses::decode(value).map_err(Error::DecodingProto)?; + let signature = AuthoritySignature::decode(&mut &signature[..]).map_err(Error::EncodingDecodingScale)?; - if !is_verified { + if !AuthorityPair::verify(&signature, &addresses, authority_id) { return Err(Error::VerifyingDhtPayload); } let addresses: Vec = schema::AuthorityAddresses::decode(addresses) .map(|a| a.addresses) - .map_err(Error::Decoding)? + .map_err(Error::DecodingProto)? .into_iter() .map(|a| a.try_into()) .collect::>() @@ -268,7 +283,7 @@ where self.address_cache.insert(authority_id.clone(), addresses); } - // Let's update the peerset priority group with the all the addresses we have in our cache. + // Let's update the peerset priority group with all the addresses we have in our cache. let addresses = HashSet::from_iter( self.address_cache @@ -279,7 +294,7 @@ where debug!(target: "sub-authority-discovery", "Applying priority group {:#?} to peerset.", addresses); self.network - .set_priority_group("authorities".to_string(), addresses) + .set_priority_group(AUTHORITIES_PRIORITY_GROUP_NAME.to_string(), addresses) .map_err(Error::SettingPeersetPriorityGroup)?; Ok(()) @@ -289,55 +304,84 @@ where self.address_cache .retain(|peer_id, _addresses| current_authorities.contains(peer_id)) } + + /// Retrieve all local authority discovery private keys that are within the current authority + /// set. + fn get_priv_keys_within_authority_set(&mut self) -> Result> { + let keys = self.get_own_public_keys_within_authority_set()? + .into_iter() + .map(std::convert::Into::into) + .filter_map(|pub_key| { + self.key_store.read().sr25519_key_pair(key_types::AUTHORITY_DISCOVERY, &pub_key) + }) + .map(std::convert::Into::into) + .collect(); + + Ok(keys) + } + + /// Retrieve our public keys within the current authority set. + // + // A node might have multiple authority discovery keys within its keystore, e.g. an old one and + // one for the upcoming session. In addition it could be participating in the current authority + // set with two keys. The function does not return all of the local authority discovery public + // keys, but only the ones intersecting with the current authority set. + fn get_own_public_keys_within_authority_set(&mut self) -> Result> { + let local_pub_keys = self.key_store + .read() + .sr25519_public_keys(key_types::AUTHORITY_DISCOVERY) + .into_iter() + .collect::>(); + + let id = BlockId::hash(self.client.info().best_hash); + let current_authorities = self + .client + .runtime_api() + .authorities(&id) + .map_err(Error::CallingRuntime)? + .into_iter() + .map(std::convert::Into::into) + .collect::>(); + + let intersection = local_pub_keys.intersection(¤t_authorities) + .cloned() + .map(std::convert::Into::into) + .collect(); + + Ok(intersection) + } } -impl futures::Future for AuthorityDiscovery +impl Future for AuthorityDiscovery where - Block: BlockT + 'static, + Block: BlockT + Unpin + 'static, Network: NetworkProvider, Client: ProvideRuntimeApi + Send + Sync + 'static + HeaderBackend, - ::Api: AuthorityDiscoveryApi, + ::Api: AuthorityDiscoveryApi, { - type Item = (); - type Error = (); + type Output = (); - fn poll(&mut self) -> futures::Poll { + fn poll(mut self: Pin<&mut Self>, cx: &mut Context) -> Poll { let mut inner = || -> Result<()> { // Process incoming events before triggering new ones. - self.handle_dht_events()?; + self.handle_dht_events(cx)?; - if let Async::Ready(_) = self - .publish_interval - .poll() - .map_err(Error::PollingTokioTimer)? - { + if let Poll::Ready(_) = self.publish_interval.poll_next_unpin(cx) { // Make sure to call interval.poll until it returns Async::NotReady once. Otherwise, in case one of the // function calls within this block do a `return`, we don't call `interval.poll` again and thereby the // underlying Tokio task is never registered with Tokio's Reactor to be woken up on the next interval // tick. - while let Async::Ready(_) = self - .publish_interval - .poll() - .map_err(Error::PollingTokioTimer)? - {} + while let Poll::Ready(_) = self.publish_interval.poll_next_unpin(cx) {} self.publish_own_ext_addresses()?; } - if let Async::Ready(_) = self - .query_interval - .poll() - .map_err(Error::PollingTokioTimer)? - { + if let Poll::Ready(_) = self.query_interval.poll_next_unpin(cx) { // Make sure to call interval.poll until it returns Async::NotReady once. Otherwise, in case one of the // function calls within this block do a `return`, we don't call `interval.poll` again and thereby the // underlying Tokio task is never registered with Tokio's Reactor to be woken up on the next interval // tick. - while let Async::Ready(_) = self - .query_interval - .poll() - .map_err(Error::PollingTokioTimer)? - {} + while let Poll::Ready(_) = self.query_interval.poll_next_unpin(cx) {} self.request_addresses_of_others()?; } @@ -351,7 +395,7 @@ where }; // Make sure to always return NotReady as this is a long running task with the same lifetime as the node itself. - Ok(futures::Async::NotReady) + Poll::Pending } } @@ -414,23 +458,26 @@ fn hash_authority_id(id: &[u8]) -> Result { #[cfg(test)] mod tests { use super::*; - use client::runtime_api::{ApiExt, Core, RuntimeVersion}; + use sr_api::{ApiExt, Core, RuntimeVersion, StorageProof}; + use futures::channel::mpsc::channel; + use futures::executor::block_on; use futures::future::poll_fn; - use primitives::{ExecutionContext, NativeOrEncoded}; + use primitives::{ExecutionContext, NativeOrEncoded, testing::KeyStore}; use sr_primitives::traits::Zero; use sr_primitives::traits::{ApiRef, Block as BlockT, NumberFor, ProvideRuntimeApi}; use std::sync::{Arc, Mutex}; use test_client::runtime::Block; - use tokio::runtime::current_thread; #[derive(Clone)] - struct TestApi {} + struct TestApi { + authorities: Vec, + } impl ProvideRuntimeApi for TestApi { type Api = RuntimeApi; fn runtime_api<'a>(&'a self) -> ApiRef<'a, Self::Api> { - RuntimeApi {}.into() + RuntimeApi{authorities: self.authorities.clone()}.into() } } @@ -439,12 +486,12 @@ mod tests { fn header( &self, _id: BlockId, - ) -> std::result::Result, client::error::Error> { + ) -> std::result::Result, client_api::error::Error> { Ok(None) } - fn info(&self) -> client::blockchain::Info { - client::blockchain::Info { + fn info(&self) -> client_api::blockchain::Info { + client_api::blockchain::Info { best_hash: Default::default(), best_number: Zero::zero(), finalized_hash: Default::default(), @@ -456,26 +503,28 @@ mod tests { fn status( &self, _id: BlockId, - ) -> std::result::Result { - Ok(client::blockchain::BlockStatus::Unknown) + ) -> std::result::Result { + Ok(client_api::blockchain::BlockStatus::Unknown) } fn number( &self, _hash: Block::Hash, - ) -> std::result::Result>, client::error::Error> { + ) -> std::result::Result>, client_api::error::Error> { Ok(None) } fn hash( &self, _number: NumberFor, - ) -> std::result::Result, client::error::Error> { + ) -> std::result::Result, client_api::error::Error> { Ok(None) } } - struct RuntimeApi {} + struct RuntimeApi { + authorities: Vec, + } impl Core for RuntimeApi { fn Core_version_runtime_api_impl( @@ -484,7 +533,7 @@ mod tests { _: ExecutionContext, _: Option<()>, _: Vec, - ) -> std::result::Result, client::error::Error> { + ) -> std::result::Result, client_api::error::Error> { unimplemented!("Not required for testing!") } @@ -494,7 +543,7 @@ mod tests { _: ExecutionContext, _: Option<(Block)>, _: Vec, - ) -> std::result::Result, client::error::Error> { + ) -> std::result::Result, client_api::error::Error> { unimplemented!("Not required for testing!") } @@ -504,12 +553,14 @@ mod tests { _: ExecutionContext, _: Option<&::Header>, _: Vec, - ) -> std::result::Result, client::error::Error> { + ) -> std::result::Result, client_api::error::Error> { unimplemented!("Not required for testing!") } } impl ApiExt for RuntimeApi { + type Error = client_api::error::Error; + fn map_api_result std::result::Result, R, E>( &self, _: F, @@ -520,7 +571,7 @@ mod tests { fn runtime_version_at( &self, _: &BlockId, - ) -> std::result::Result { + ) -> std::result::Result { unimplemented!("Not required for testing!") } @@ -528,7 +579,7 @@ mod tests { unimplemented!("Not required for testing!") } - fn extract_proof(&mut self) -> Option>> { + fn extract_proof(&mut self) -> Option { unimplemented!("Not required for testing!") } } @@ -540,38 +591,8 @@ mod tests { _: ExecutionContext, _: Option<()>, _: Vec, - ) -> std::result::Result>, client::error::Error> { - return Ok(NativeOrEncoded::Native(vec![ - AuthorityId("test-authority-id-1".as_bytes().to_vec()), - AuthorityId("test-authority-id-2".as_bytes().to_vec()), - ])); - } - fn AuthorityDiscoveryApi_sign_runtime_api_impl( - &self, - _: &BlockId, - _: ExecutionContext, - _: Option<&std::vec::Vec>, - _: Vec, - ) -> std::result::Result< - NativeOrEncoded>, - client::error::Error, - > { - return Ok(NativeOrEncoded::Native(Some(( - Signature("test-signature-1".as_bytes().to_vec()), - AuthorityId("test-authority-id-1".as_bytes().to_vec()), - )))); - } - fn AuthorityDiscoveryApi_verify_runtime_api_impl( - &self, - _: &BlockId, - _: ExecutionContext, - args: Option<(&Vec, &Signature, &AuthorityId)>, - _: Vec, - ) -> std::result::Result, client::error::Error> { - if *args.unwrap().1 == Signature("test-signature-1".as_bytes().to_vec()) { - return Ok(NativeOrEncoded::Native(true)); - } - return Ok(NativeOrEncoded::Native(false)); + ) -> std::result::Result>, client_api::error::Error> { + return Ok(NativeOrEncoded::Native(self.authorities.clone())); } } @@ -611,12 +632,14 @@ mod tests { #[test] fn publish_own_ext_addresses_puts_record_on_dht() { - let (_dht_event_tx, dht_event_rx) = futures::sync::mpsc::channel(1000); - let test_api = Arc::new(TestApi {}); + let (_dht_event_tx, dht_event_rx) = channel(1000); let network: Arc = Arc::new(Default::default()); + let key_store = KeyStore::new(); + let public = key_store.write().sr25519_generate_new(key_types::AUTHORITY_DISCOVERY, None).unwrap(); + let test_api = Arc::new(TestApi {authorities: vec![public.into()]}); let mut authority_discovery = - AuthorityDiscovery::new(test_api, network.clone(), dht_event_rx); + AuthorityDiscovery::new(test_api, network.clone(), key_store, dht_event_rx.boxed()); authority_discovery.publish_own_ext_addresses().unwrap(); @@ -626,12 +649,21 @@ mod tests { #[test] fn request_addresses_of_others_triggers_dht_get_query() { - let (_dht_event_tx, dht_event_rx) = futures::sync::mpsc::channel(1000); - let test_api = Arc::new(TestApi {}); + let (_dht_event_tx, dht_event_rx) = channel(1000); + + // Generate authority keys + let authority_1_key_pair = AuthorityPair::from_seed_slice(&[1; 32]).unwrap(); + let authority_2_key_pair = AuthorityPair::from_seed_slice(&[2; 32]).unwrap(); + + let test_api = Arc::new(TestApi { + authorities: vec![authority_1_key_pair.public(), authority_2_key_pair.public()], + }); + let network: Arc = Arc::new(Default::default()); + let key_store = KeyStore::new(); let mut authority_discovery = - AuthorityDiscovery::new(test_api, network.clone(), dht_event_rx); + AuthorityDiscovery::new(test_api, network.clone(), key_store, dht_event_rx.boxed()); authority_discovery.request_addresses_of_others().unwrap(); @@ -643,16 +675,18 @@ mod tests { fn handle_dht_events_with_value_found_should_call_set_priority_group() { // Create authority discovery. - let (mut dht_event_tx, dht_event_rx) = futures::sync::mpsc::channel(1000); - let test_api = Arc::new(TestApi {}); + let (mut dht_event_tx, dht_event_rx) = channel(1000); + let key_pair = AuthorityPair::from_seed_slice(&[1; 32]).unwrap(); + let test_api = Arc::new(TestApi {authorities: vec![key_pair.public()]}); let network: Arc = Arc::new(Default::default()); + let key_store = KeyStore::new(); let mut authority_discovery = - AuthorityDiscovery::new(test_api, network.clone(), dht_event_rx); + AuthorityDiscovery::new(test_api, network.clone(), key_store, dht_event_rx.boxed()); // Create sample dht event. - let authority_id_1 = hash_authority_id("test-authority-id-1".as_bytes()).unwrap(); + let authority_id_1 = hash_authority_id(key_pair.public().as_ref()).unwrap(); let address_1: libp2p::Multiaddr = "/ip6/2001:db8::".parse().unwrap(); let mut serialized_addresses = vec![]; @@ -662,10 +696,11 @@ mod tests { .encode(&mut serialized_addresses) .unwrap(); + let signature = key_pair.sign(serialized_addresses.as_ref()).encode(); let mut signed_addresses = vec![]; schema::SignedAuthorityAddresses { addresses: serialized_addresses, - signature: "test-signature-1".as_bytes().to_vec(), + signature: signature, } .encode(&mut signed_addresses) .unwrap(); @@ -674,9 +709,8 @@ mod tests { dht_event_tx.try_send(dht_event).unwrap(); // Make authority discovery handle the event. - - let f = || { - authority_discovery.handle_dht_events().unwrap(); + let f = |cx: &mut Context<'_>| -> Poll<()> { + authority_discovery.handle_dht_events(cx).unwrap(); // Expect authority discovery to set the priority set. assert_eq!(network.set_priority_group_call.lock().unwrap().len(), 1); @@ -689,10 +723,9 @@ mod tests { ) ); - Ok(Async::Ready(())) + Poll::Ready(()) }; - let mut runtime = current_thread::Runtime::new().unwrap(); - runtime.block_on(poll_fn::<(), (), _>(f)).unwrap(); + let _ = block_on(poll_fn(f)); } } diff --git a/core/authority-discovery/src/schema/dht.proto b/client/authority-discovery/src/schema/dht.proto similarity index 100% rename from core/authority-discovery/src/schema/dht.proto rename to client/authority-discovery/src/schema/dht.proto diff --git a/client/basic-authorship/Cargo.toml b/client/basic-authorship/Cargo.toml new file mode 100644 index 0000000000000..3104b70043b0c --- /dev/null +++ b/client/basic-authorship/Cargo.toml @@ -0,0 +1,22 @@ +[package] +name = "substrate-basic-authorship" +version = "2.0.0" +authors = ["Parity Technologies "] +edition = "2018" + +[dependencies] +log = "0.4.8" +futures-preview = "0.3.0-alpha.19" +codec = { package = "parity-scale-codec", version = "1.0.0" } +sr-primitives = { path = "../../primitives/sr-primitives" } +primitives = { package = "substrate-primitives", path = "../../primitives/core" } +client = { package = "substrate-client", path = "../" } +client-api = { package = "substrate-client-api", path = "../api" } +consensus_common = { package = "substrate-consensus-common", path = "../../primitives/consensus/common" } +inherents = { package = "substrate-inherents", path = "../../primitives/inherents" } +substrate-telemetry = { path = "../telemetry" } +transaction_pool = { package = "substrate-transaction-pool", path = "../transaction-pool" } +block-builder = { package = "substrate-block-builder", path = "../block-builder" } + +[dev-dependencies] +test-client = { package = "substrate-test-runtime-client", path = "../../test/utils/runtime/client" } diff --git a/core/basic-authorship/src/basic_authorship.rs b/client/basic-authorship/src/basic_authorship.rs similarity index 91% rename from core/basic-authorship/src/basic_authorship.rs rename to client/basic-authorship/src/basic_authorship.rs index 7f8b343f6512c..7d8956152c13f 100644 --- a/core/basic-authorship/src/basic_authorship.rs +++ b/client/basic-authorship/src/basic_authorship.rs @@ -20,10 +20,8 @@ // use std::{time, sync::Arc}; -use client::{ - error, Client as SubstrateClient, CallExecutor, - block_builder::api::BlockBuilder as BlockBuilderApi, -}; +use client_api::{error, CallExecutor}; +use client::Client as SubstrateClient; use codec::Decode; use consensus_common::{evaluation}; use inherents::InherentData; @@ -37,6 +35,7 @@ use sr_primitives::{ }; use transaction_pool::txpool::{self, Pool as TransactionPool}; use substrate_telemetry::{telemetry, CONSENSUS_INFO}; +use block_builder::BlockBuilderApi; /// Proposer factory. pub struct ProposerFactory where A: txpool::ChainApi { @@ -50,12 +49,13 @@ impl consensus_common::Environment for ProposerFactory, A> where A: txpool::ChainApi, - B: client::backend::Backend + Send + Sync + 'static, + B: client_api::backend::Backend + Send + Sync + 'static, E: CallExecutor + Send + Sync + Clone + 'static, Block: BlockT, RA: Send + Sync + 'static, SubstrateClient: ProvideRuntimeApi, - as ProvideRuntimeApi>::Api: BlockBuilderApi, + as ProvideRuntimeApi>::Api: + BlockBuilderApi, { type Proposer = Proposer, A>; type Error = error::Error; @@ -97,12 +97,13 @@ impl consensus_common::Proposer for Proposer, A> where A: txpool::ChainApi, - B: client::backend::Backend + Send + Sync + 'static, + B: client_api::backend::Backend + Send + Sync + 'static, E: CallExecutor + Send + Sync + Clone + 'static, Block: BlockT, RA: Send + Sync + 'static, SubstrateClient: ProvideRuntimeApi, - as ProvideRuntimeApi>::Api: BlockBuilderApi, + as ProvideRuntimeApi>::Api: + BlockBuilderApi, { type Create = futures::future::Ready>; type Error = error::Error; @@ -121,12 +122,13 @@ where impl Proposer, A> where A: txpool::ChainApi, - B: client::backend::Backend + Send + Sync + 'static, + B: client_api::backend::Backend + Send + Sync + 'static, E: CallExecutor + Send + Sync + Clone + 'static, Block: BlockT, RA: Send + Sync + 'static, SubstrateClient: ProvideRuntimeApi, - as ProvideRuntimeApi>::Api: BlockBuilderApi, + as ProvideRuntimeApi>::Api: + BlockBuilderApi, { fn propose_with( &self, @@ -167,7 +169,7 @@ impl Proposer, A> wh } trace!("[{:?}] Pushing to the block.", pending.hash); - match client::block_builder::BlockBuilder::push(&mut block_builder, pending.data.clone()) { + match block_builder::BlockBuilder::push(&mut block_builder, pending.data.clone()) { Ok(()) => { debug!("[{:?}] Pushed to the block.", pending.hash); } diff --git a/core/basic-authorship/src/lib.rs b/client/basic-authorship/src/lib.rs similarity index 100% rename from core/basic-authorship/src/lib.rs rename to client/basic-authorship/src/lib.rs diff --git a/client/block-builder/Cargo.toml b/client/block-builder/Cargo.toml new file mode 100644 index 0000000000000..34bfac17bf774 --- /dev/null +++ b/client/block-builder/Cargo.toml @@ -0,0 +1,14 @@ +[package] +name = "substrate-block-builder" +version = "2.0.0" +authors = ["Parity Technologies "] +edition = "2018" + +[dependencies] +state-machine = { package = "substrate-state-machine", path = "../../primitives/state-machine" } +sr-primitives = { path = "../../primitives/sr-primitives" } +primitives = { package = "substrate-primitives", path = "../../primitives/core" } +codec = { package = "parity-scale-codec", version = "1.0.6", features = ["derive"] } +runtime_api = { package = "substrate-block-builder-runtime-api", path = "../../primitives/block-builder/runtime-api" } +sr-api = { path = "../../primitives/sr-api" } + diff --git a/core/client/src/block_builder/block_builder.rs b/client/block-builder/src/lib.rs similarity index 61% rename from core/client/src/block_builder/block_builder.rs rename to client/block-builder/src/lib.rs index 08711469e9a1b..5a345cbeb62bf 100644 --- a/core/client/src/block_builder/block_builder.rs +++ b/client/block-builder/src/lib.rs @@ -14,58 +14,66 @@ // You should have received a copy of the GNU General Public License // along with Substrate. If not, see . -use super::api::BlockBuilder as BlockBuilderApi; -use std::vec::Vec; +//! Substrate block builder +//! +//! This crate provides the [`BlockBuilder`] utility and the corresponding runtime api +//! [`BlockBuilder`](api::BlockBuilder). +//! +//! The block builder utility is used in the node as an abstraction over the runtime api to +//! initialize a block, to push extrinsics and to finalize a block. + +#![warn(missing_docs)] + use codec::Encode; -use sr_primitives::generic::BlockId; -use sr_primitives::traits::{ - Header as HeaderT, Hash, Block as BlockT, One, HashFor, ProvideRuntimeApi, ApiRef, DigestFor, + +use sr_primitives::{ + generic::BlockId, + traits::{ + Header as HeaderT, Hash, Block as BlockT, HashFor, ProvideRuntimeApi, ApiRef, DigestFor, + NumberFor, One, + }, }; -use primitives::{H256, ExecutionContext}; -use crate::blockchain::HeaderBackend; -use crate::runtime_api::{Core, ApiExt}; -use crate::error; + +use primitives::ExecutionContext; + +use state_machine::StorageProof; + +use sr_api::{Core, ApiExt, ApiErrorFor}; + +pub use runtime_api::BlockBuilder as BlockBuilderApi; + +/// Error when the runtime failed to apply an extrinsic. +pub struct ApplyExtrinsicFailed(pub sr_primitives::ApplyError); /// Utility for building new (valid) blocks from a stream of extrinsics. -pub struct BlockBuilder<'a, Block, A: ProvideRuntimeApi> where Block: BlockT { - header: ::Header, - extrinsics: Vec<::Extrinsic>, +pub struct BlockBuilder<'a, Block: BlockT, A: ProvideRuntimeApi> { + header: Block::Header, + extrinsics: Vec, api: ApiRef<'a, A::Api>, block_id: BlockId, } impl<'a, Block, A> BlockBuilder<'a, Block, A> where - Block: BlockT, - A: ProvideRuntimeApi + HeaderBackend + 'a, + Block: BlockT, + A: ProvideRuntimeApi + 'a, A::Api: BlockBuilderApi, + ApiErrorFor: From, { - /// Create a new instance of builder from the given client, building on the - /// latest block. - pub fn new(api: &'a A, inherent_digests: DigestFor) -> error::Result { - Self::at_block(&BlockId::Hash(api.info().best_hash), api, false, inherent_digests) - } - - /// Create a new instance of builder from the given client using a - /// particular block's ID to build upon with optional proof recording enabled. + /// Create a new instance of builder based on the given `parent_hash` and `parent_number`. /// /// While proof recording is enabled, all accessed trie nodes are saved. /// These recorded trie nodes can be used by a third party to prove the /// output of this block builder without having access to the full storage. - pub fn at_block( - block_id: &BlockId, + pub fn new( api: &'a A, + parent_hash: Block::Hash, + parent_number: NumberFor, proof_recording: bool, inherent_digests: DigestFor, - ) -> error::Result { - let number = api.block_number_from_id(block_id)? - .ok_or_else(|| error::Error::UnknownBlock(format!("{}", block_id)))? - + One::one(); - - let parent_hash = api.block_hash_from_id(block_id)? - .ok_or_else(|| error::Error::UnknownBlock(format!("{}", block_id)))?; + ) -> Result> { let header = <::Header as HeaderT>::new( - number, + parent_number + One::one(), Default::default(), Default::default(), parent_hash, @@ -78,22 +86,24 @@ where api.record_proof(); } + let block_id = BlockId::Hash(parent_hash); + api.initialize_block_with_context( - block_id, ExecutionContext::BlockConstruction, &header, + &block_id, ExecutionContext::BlockConstruction, &header, )?; - Ok(BlockBuilder { + Ok(Self { header, extrinsics: Vec::new(), api, - block_id: *block_id, + block_id, }) } /// Push onto the block's list of extrinsics. /// /// This will ensure the extrinsic can be validly executed (by executing it); - pub fn push(&mut self, xt: ::Extrinsic) -> error::Result<()> { + pub fn push(&mut self, xt: ::Extrinsic) -> Result<(), ApiErrorFor> { let block_id = &self.block_id; let extrinsics = &mut self.extrinsics; @@ -108,19 +118,19 @@ where Ok(()) } Err(e) => { - Err(error::Error::ApplyExtrinsicFailed(e)) + Err(ApplyExtrinsicFailed(e))? } } }) } /// Consume the builder to return a valid `Block` containing all pushed extrinsics. - pub fn bake(mut self) -> error::Result { + pub fn bake(mut self) -> Result> { self.bake_impl()?; Ok(::new(self.header, self.extrinsics)) } - fn bake_impl(&mut self) -> error::Result<()> { + fn bake_impl(&mut self) -> Result<(), ApiErrorFor> { self.header = self.api.finalize_block_with_context( &self.block_id, ExecutionContext::BlockConstruction )?; @@ -140,7 +150,9 @@ where /// /// The proof will be `Some(_)`, if proof recording was enabled while creating /// the block builder. - pub fn bake_and_extract_proof(mut self) -> error::Result<(Block, Option>>)> { + pub fn bake_and_extract_proof(mut self) + -> Result<(Block, Option), ApiErrorFor> + { self.bake_impl()?; let proof = self.api.extract_proof(); diff --git a/client/chain-spec/Cargo.toml b/client/chain-spec/Cargo.toml new file mode 100644 index 0000000000000..da9543e398774 --- /dev/null +++ b/client/chain-spec/Cargo.toml @@ -0,0 +1,15 @@ +[package] +name = "substrate-chain-spec" +version = "2.0.0" +authors = ["Parity Technologies "] +edition = "2018" + +[dependencies] +substrate-chain-spec-derive = { path = "./derive" } +impl-trait-for-tuples = "0.1.3" +network = { package = "substrate-network", path = "../network" } +primitives = { package = "substrate-primitives", path = "../../primitives/core" } +serde = { version = "1.0.101", features = ["derive"] } +serde_json = "1.0.41" +sr-primitives = { path = "../../primitives/sr-primitives" } +tel = { package = "substrate-telemetry", path = "../telemetry" } diff --git a/core/chain-spec/derive/Cargo.toml b/client/chain-spec/derive/Cargo.toml similarity index 86% rename from core/chain-spec/derive/Cargo.toml rename to client/chain-spec/derive/Cargo.toml index 2e42ca0cf3a70..9fb8eabc60844 100644 --- a/core/chain-spec/derive/Cargo.toml +++ b/client/chain-spec/derive/Cargo.toml @@ -9,8 +9,8 @@ proc-macro = true [dependencies] proc-macro-crate = "0.1.4" -proc-macro2 = "1.0.4" +proc-macro2 = "1.0.6" quote = "1.0.2" -syn = "1.0.5" +syn = "1.0.7" [dev-dependencies] diff --git a/core/chain-spec/derive/src/impls.rs b/client/chain-spec/derive/src/impls.rs similarity index 100% rename from core/chain-spec/derive/src/impls.rs rename to client/chain-spec/derive/src/impls.rs diff --git a/core/chain-spec/derive/src/lib.rs b/client/chain-spec/derive/src/lib.rs similarity index 100% rename from core/chain-spec/derive/src/lib.rs rename to client/chain-spec/derive/src/lib.rs diff --git a/core/chain-spec/res/chain_spec.json b/client/chain-spec/res/chain_spec.json similarity index 100% rename from core/chain-spec/res/chain_spec.json rename to client/chain-spec/res/chain_spec.json diff --git a/core/chain-spec/res/chain_spec2.json b/client/chain-spec/res/chain_spec2.json similarity index 100% rename from core/chain-spec/res/chain_spec2.json rename to client/chain-spec/res/chain_spec2.json diff --git a/core/chain-spec/src/chain_spec.rs b/client/chain-spec/src/chain_spec.rs similarity index 100% rename from core/chain-spec/src/chain_spec.rs rename to client/chain-spec/src/chain_spec.rs diff --git a/core/chain-spec/src/extension.rs b/client/chain-spec/src/extension.rs similarity index 100% rename from core/chain-spec/src/extension.rs rename to client/chain-spec/src/extension.rs diff --git a/core/chain-spec/src/lib.rs b/client/chain-spec/src/lib.rs similarity index 100% rename from core/chain-spec/src/lib.rs rename to client/chain-spec/src/lib.rs diff --git a/core/cli/Cargo.toml b/client/cli/Cargo.toml similarity index 55% rename from core/cli/Cargo.toml rename to client/cli/Cargo.toml index a2723484ad4c1..caea2fa41dfad 100644 --- a/core/cli/Cargo.toml +++ b/client/cli/Cargo.toml @@ -22,19 +22,25 @@ futures03 = { package = "futures-preview", version = "=0.3.0-alpha.19", features fdlimit = "0.1.1" exit-future = "0.1.4" serde_json = "1.0.41" -panic-handler = { package = "substrate-panic-handler", path = "../../core/panic-handler" } -client = { package = "substrate-client", path = "../../core/client" } -header-metadata = { package = "substrate-header-metadata", path = "../../core/client/header-metadata" } -network = { package = "substrate-network", path = "../../core/network" } -sr-primitives = { path = "../../core/sr-primitives" } -primitives = { package = "substrate-primitives", path = "../../core/primitives" } -service = { package = "substrate-service", path = "../../core/service" } -state-machine = { package = "substrate-state-machine", path = "../../core/state-machine" } -substrate-telemetry = { path = "../../core/telemetry" } -keyring = { package = "substrate-keyring", path = "../keyring" } +panic-handler = { package = "substrate-panic-handler", path = "../../primitives/panic-handler" } +client-api = { package = "substrate-client-api", path = "../api" } +header-metadata = { package = "substrate-header-metadata", path = "../header-metadata" } +network = { package = "substrate-network", path = "../network" } +sr-primitives = { path = "../../primitives/sr-primitives" } +primitives = { package = "substrate-primitives", path = "../../primitives/core" } +service = { package = "substrate-service", path = "../service", default-features = false } +state-machine = { package = "substrate-state-machine", path = "../../primitives/state-machine" } +substrate-telemetry = { path = "../telemetry" } +substrate-prometheus = { path = "../prometheus" } +keyring = { package = "substrate-keyring", path = "../../primitives/keyring" } names = "0.11.0" structopt = "0.3.3" rpassword = "4.0.1" [dev-dependencies] tempdir = "0.3.7" + +[features] +wasmtime = [ + "service/wasmtime", +] diff --git a/core/cli/README.adoc b/client/cli/README.adoc similarity index 100% rename from core/cli/README.adoc rename to client/cli/README.adoc diff --git a/core/cli/src/error.rs b/client/cli/src/error.rs similarity index 97% rename from core/cli/src/error.rs rename to client/cli/src/error.rs index 600e73d44fe73..6b42773bb833a 100644 --- a/core/cli/src/error.rs +++ b/client/cli/src/error.rs @@ -16,7 +16,7 @@ //! Initialization errors. -use client; +use client_api; /// Result type alias for the CLI. pub type Result = std::result::Result; @@ -31,7 +31,7 @@ pub enum Error { /// Service error Service(service::Error), /// Client error - Client(client::error::Error), + Client(client_api::error::Error), /// Input error Input(String), /// Invalid listen multiaddress diff --git a/core/cli/src/execution_strategy.rs b/client/cli/src/execution_strategy.rs similarity index 100% rename from core/cli/src/execution_strategy.rs rename to client/cli/src/execution_strategy.rs diff --git a/core/cli/src/informant.rs b/client/cli/src/informant.rs similarity index 98% rename from core/cli/src/informant.rs rename to client/cli/src/informant.rs index f7c23ba47918a..4e9572522c331 100644 --- a/core/cli/src/informant.rs +++ b/client/cli/src/informant.rs @@ -16,7 +16,7 @@ //! Console informant. Prints sync progress and block events. Runs on the calling thread. -use client::BlockchainEvents; +use client_api::BlockchainEvents; use futures::{Future, Stream}; use futures03::{StreamExt as _, TryStreamExt as _}; use log::{info, warn}; diff --git a/core/cli/src/informant/display.rs b/client/cli/src/informant/display.rs similarity index 85% rename from core/cli/src/informant/display.rs rename to client/cli/src/informant/display.rs index c7cf9bfc930b9..bbe62af5b835e 100644 --- a/core/cli/src/informant/display.rs +++ b/client/cli/src/informant/display.rs @@ -15,13 +15,18 @@ // along with Substrate. If not, see . use ansi_term::Colour; -use client::ClientInfo; +use client_api::ClientInfo; use log::info; use network::SyncState; -use sr_primitives::traits::{Block as BlockT, CheckedDiv, NumberFor, Zero, Saturating}; use service::NetworkStatus; -use std::{convert::{TryFrom, TryInto}, fmt, time}; - +use sr_primitives::traits::{ + Block as BlockT, CheckedDiv, NumberFor, Saturating, Zero, +}; +use std::{ + convert::{TryFrom, TryInto}, + fmt, time, +}; +use substrate_prometheus::prometheus_gauge; /// State of the informant display system. /// /// This is the system that handles the line that gets regularly printed and that looks something @@ -63,9 +68,11 @@ impl InformantDisplay { let (status, target) = match (net_status.sync_state, net_status.best_seen_block) { (SyncState::Idle, _) => ("Idle".into(), "".into()), (SyncState::Downloading, None) => (format!("Syncing{}", speed), "".into()), - (SyncState::Downloading, Some(n)) => (format!("Syncing{}", speed), format!(", target=#{}", n)), + (SyncState::Downloading, Some(n)) => { + prometheus_gauge!(TARGET_NUM => n.saturated_into().try_into().unwrap()); + (format!("Syncing{}", speed), format!(", target=#{}", n)) + } }; - info!( target: "substrate", "{}{} ({} peers), best: #{} ({}), finalized #{} ({}), ⬇ {} ⬆ {}", @@ -87,7 +94,7 @@ impl InformantDisplay { fn speed( best_number: NumberFor, last_number: Option>, - last_update: time::Instant + last_update: time::Instant, ) -> String { // Number of milliseconds elapsed since last time. let elapsed_ms = { @@ -100,25 +107,29 @@ fn speed( // Number of blocks that have been imported since last time. let diff = match last_number { None => return String::new(), - Some(n) => best_number.saturating_sub(n) + Some(n) => best_number.saturating_sub(n), }; if let Ok(diff) = TryInto::::try_into(diff) { // If the number of blocks can be converted to a regular integer, then it's easy: just // do the math and turn it into a `f64`. - let speed = diff.saturating_mul(10_000).checked_div(u128::from(elapsed_ms)) - .map_or(0.0, |s| s as f64) / 10.0; + let speed = diff + .saturating_mul(10_000) + .checked_div(u128::from(elapsed_ms)) + .map_or(0.0, |s| s as f64) + / 10.0; format!(" {:4.1} bps", speed) - } else { // If the number of blocks can't be converted to a regular integer, then we need a more // algebraic approach and we stay within the realm of integers. let one_thousand = NumberFor::::from(1_000); let elapsed = NumberFor::::from( - >::try_from(elapsed_ms).unwrap_or(u32::max_value()) + >::try_from(elapsed_ms).unwrap_or(u32::max_value()), ); - let speed = diff.saturating_mul(one_thousand).checked_div(&elapsed) + let speed = diff + .saturating_mul(one_thousand) + .checked_div(&elapsed) .unwrap_or_else(Zero::zero); format!(" {} bps", speed) } @@ -131,17 +142,17 @@ impl fmt::Display for TransferRateFormat { fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { // Special case 0. if self.0 == 0 { - return write!(f, "0") + return write!(f, "0"); } // Under 0.1 kiB, display plain bytes. if self.0 < 100 { - return write!(f, "{} B/s", self.0) + return write!(f, "{} B/s", self.0); } // Under 1.0 MiB/sec, display the value in kiB/sec. if self.0 < 1024 * 1024 { - return write!(f, "{:.1}kiB/s", self.0 as f64 / 1024.0) + return write!(f, "{:.1}kiB/s", self.0 as f64 / 1024.0); } write!(f, "{:.1}MiB/s", self.0 as f64 / (1024.0 * 1024.0)) diff --git a/core/cli/src/lib.rs b/client/cli/src/lib.rs similarity index 92% rename from core/cli/src/lib.rs rename to client/cli/src/lib.rs index 7b96788433ed2..5cf55222c016c 100644 --- a/core/cli/src/lib.rs +++ b/client/cli/src/lib.rs @@ -26,7 +26,7 @@ mod execution_strategy; pub mod error; pub mod informant; -use client::ExecutionStrategies; +use client_api::ExecutionStrategies; use service::{ config::{Configuration, DatabaseConfig}, ServiceBuilderExport, ServiceBuilderImport, ServiceBuilderRevert, @@ -65,6 +65,13 @@ use lazy_static::lazy_static; use futures::Future; use substrate_telemetry::TelemetryEndpoints; +/// default sub directory to store network config +const DEFAULT_NETWORK_CONFIG_PATH : &'static str = "network"; +/// default sub directory to store database +const DEFAULT_DB_CONFIG_PATH : &'static str = "db"; +/// default sub directory for the key store +const DEFAULT_KEYSTORE_CONFIG_PATH : &'static str = "keystore"; + /// The maximum number of characters for a node name. const NODE_NAME_MAX_LENGTH: usize = 32; @@ -307,18 +314,36 @@ pub struct ParseAndPrepareBuildSpec<'a> { impl<'a> ParseAndPrepareBuildSpec<'a> { /// Runs the command and build the chain specs. - pub fn run( + pub fn run( self, spec_factory: S ) -> error::Result<()> where S: FnOnce(&str) -> Result>, String>, + C: Default, G: RuntimeGenesis, E: ChainSpecExtension, { info!("Building chain spec"); let raw_output = self.params.raw; let mut spec = load_spec(&self.params.shared_params, spec_factory)?; - with_default_boot_node(&mut spec, self.params, self.version)?; + + if spec.boot_nodes().is_empty() && !self.params.disable_default_bootnode { + let base_path = base_path(&self.params.shared_params, self.version); + let cfg = service::Configuration::::default_with_spec_and_base_path(spec.clone(), Some(base_path)); + let node_key = node_key_config( + self.params.node_key_params, + &Some(cfg.in_chain_config_dir(DEFAULT_NETWORK_CONFIG_PATH).expect("We provided a base_path")) + )?; + let keys = node_key.into_keypair()?; + let peer_id = keys.public().into_peer_id(); + let addr = build_multiaddr![ + Ip4([127, 0, 0, 1]), + Tcp(30333u16), + P2p(peer_id) + ]; + spec.add_boot_node(addr) + } + let json = service::chain_ops::build_spec(spec, raw_output)?; print!("{}", json); @@ -558,16 +583,13 @@ fn fill_transaction_pool_configuration( /// Fill the given `NetworkConfiguration` by looking at the cli parameters. fn fill_network_configuration( cli: NetworkConfigurationParams, - base_path: &Path, - chain_spec_id: &str, + config_path: PathBuf, config: &mut NetworkConfiguration, client_id: String, is_dev: bool, ) -> error::Result<()> { config.boot_nodes.extend(cli.bootnodes.into_iter()); - config.config_path = Some( - network_path(&base_path, chain_spec_id).to_string_lossy().into() - ); + config.config_path = Some(config_path.to_string_lossy().into()); config.net_config_path = config.config_path.clone(); config.reserved_nodes.extend(cli.reserved_nodes.into_iter()); @@ -603,9 +625,12 @@ fn fill_network_configuration( config.transport = TransportConfig::Normal { enable_mdns: !is_dev && !cli.no_mdns, + allow_private_ipv4: !cli.no_private_ipv4, wasm_external_transport: None, }; + config.max_parallel_downloads = cli.max_parallel_downloads; + Ok(()) } @@ -642,7 +667,8 @@ where S: FnOnce(&str) -> Result>, String>, { let spec = load_spec(&cli.shared_params, spec_factory)?; - let mut config = service::Configuration::default_with_spec(spec.clone()); + let base_path = base_path(&cli.shared_params, &version); + let mut config = service::Configuration::default_with_spec_and_base_path(spec.clone(), Some(base_path)); fill_config_keystore_password(&mut config, &cli)?; @@ -666,14 +692,10 @@ where )? } - let base_path = base_path(&cli.shared_params, version); - - config.keystore_path = cli.keystore_path.unwrap_or_else( - || keystore_path(&base_path, config.chain_spec.id()) - ); + config.keystore_path = cli.keystore_path.or_else(|| config.in_chain_config_dir(DEFAULT_KEYSTORE_CONFIG_PATH)); config.database = DatabaseConfig::Path { - path: db_path(&base_path, config.chain_spec.id()), + path: config.in_chain_config_dir(DEFAULT_DB_CONFIG_PATH).expect("We provided a base_path."), cache_size: cli.database_cache_size, }; config.state_cache_size = cli.state_cache_size; @@ -740,8 +762,7 @@ where let client_id = config.client_id(); fill_network_configuration( cli.network_config, - &base_path, - spec.id(), + config.in_chain_config_dir(DEFAULT_NETWORK_CONFIG_PATH).expect("We provided a basepath"), &mut config.network, client_id, is_dev, @@ -786,45 +807,25 @@ where config.telemetry_endpoints = Some(TelemetryEndpoints::new(cli.telemetry_endpoints)); } + match cli.prometheus_endpoint { + None => {config.prometheus_endpoint = None;}, + Some(x) => { + config.prometheus_endpoint = Some(parse_address(&format!("{}:{}", x, 33333), cli.prometheus_port)?); + } + } + + match cli.prometheus_endpoint { + None => {config.prometheus_endpoint = None;}, + Some(x) => { + config.prometheus_endpoint = Some(parse_address(&format!("{}:{}", x, 33333), cli.prometheus_port)?); + } + } // Imply forced authoring on --dev config.force_authoring = cli.shared_params.dev || cli.force_authoring; Ok(config) } -// -// IANA unassigned port ranges that we could use: -// 6717-6766 Unassigned -// 8504-8553 Unassigned -// 9556-9591 Unassigned -// 9803-9874 Unassigned -// 9926-9949 Unassigned - -fn with_default_boot_node( - spec: &mut ChainSpec, - cli: BuildSpecCmd, - version: &VersionInfo, -) -> error::Result<()> -where - G: RuntimeGenesis, - E: ChainSpecExtension, -{ - if spec.boot_nodes().is_empty() && !cli.disable_default_bootnode { - let base_path = base_path(&cli.shared_params, version); - let storage_path = network_path(&base_path, spec.id()); - let node_key = node_key_config(cli.node_key_params, &Some(storage_path))?; - let keys = node_key.into_keypair()?; - let peer_id = keys.public().into_peer_id(); - let addr = build_multiaddr![ - Ip4([127, 0, 0, 1]), - Tcp(30333u16), - P2p(peer_id) - ]; - spec.add_boot_node(addr) - } - Ok(()) -} - /// Creates a configuration including the database path. pub fn create_config_with_db_path( spec_factory: S, cli: &SharedParams, version: &VersionInfo, @@ -838,9 +839,9 @@ where let spec = load_spec(cli, spec_factory)?; let base_path = base_path(cli, version); - let mut config = service::Configuration::default_with_spec(spec.clone()); + let mut config = service::Configuration::default_with_spec_and_base_path(spec.clone(), Some(base_path)); config.database = DatabaseConfig::Path { - path: db_path(&base_path, spec.id()), + path: config.in_chain_config_dir(DEFAULT_DB_CONFIG_PATH).expect("We provided a base_path."), cache_size: None, }; @@ -866,30 +867,6 @@ fn parse_address( Ok(address) } -fn keystore_path(base_path: &Path, chain_id: &str) -> PathBuf { - let mut path = base_path.to_owned(); - path.push("chains"); - path.push(chain_id); - path.push("keystore"); - path -} - -fn db_path(base_path: &Path, chain_id: &str) -> PathBuf { - let mut path = base_path.to_owned(); - path.push("chains"); - path.push(chain_id); - path.push("db"); - path -} - -fn network_path(base_path: &Path, chain_id: &str) -> PathBuf { - let mut path = base_path.to_owned(); - path.push("chains"); - path.push(chain_id); - path.push("network"); - path -} - fn init_logger(pattern: &str) { use ansi_term::Colour; @@ -897,6 +874,7 @@ fn init_logger(pattern: &str) { // Disable info logging by default for some modules: builder.filter(Some("ws"), log::LevelFilter::Off); builder.filter(Some("hyper"), log::LevelFilter::Warn); + builder.filter(Some("cranelift_wasm"), log::LevelFilter::Warn); // Enable info for others. builder.filter(None, log::LevelFilter::Info); diff --git a/core/cli/src/params.rs b/client/cli/src/params.rs similarity index 91% rename from core/cli/src/params.rs rename to client/cli/src/params.rs index 007bdabf0b7f5..fcd617e73d6a9 100644 --- a/core/cli/src/params.rs +++ b/client/cli/src/params.rs @@ -32,13 +32,13 @@ macro_rules! impl_get_log_filter { } } -impl Into for ExecutionStrategy { - fn into(self) -> client::ExecutionStrategy { +impl Into for ExecutionStrategy { + fn into(self) -> client_api::ExecutionStrategy { match self { - ExecutionStrategy::Native => client::ExecutionStrategy::NativeWhenPossible, - ExecutionStrategy::Wasm => client::ExecutionStrategy::AlwaysWasm, - ExecutionStrategy::Both => client::ExecutionStrategy::Both, - ExecutionStrategy::NativeElseWasm => client::ExecutionStrategy::NativeElseWasm, + ExecutionStrategy::Native => client_api::ExecutionStrategy::NativeWhenPossible, + ExecutionStrategy::Wasm => client_api::ExecutionStrategy::AlwaysWasm, + ExecutionStrategy::Both => client_api::ExecutionStrategy::Both, + ExecutionStrategy::NativeElseWasm => client_api::ExecutionStrategy::NativeElseWasm, } } } @@ -50,6 +50,19 @@ arg_enum! { pub enum WasmExecutionMethod { // Uses an interpreter. Interpreted, + // Uses a compiled runtime. + Compiled, + } +} + +impl WasmExecutionMethod { + /// Returns list of variants that are not disabled by feature flags. + fn enabled_variants() -> Vec<&'static str> { + Self::variants() + .iter() + .cloned() + .filter(|&name| cfg!(feature = "wasmtime") || name != "Compiled") + .collect() } } @@ -57,6 +70,12 @@ impl Into for WasmExecutionMethod { fn into(self) -> service::config::WasmExecutionMethod { match self { WasmExecutionMethod::Interpreted => service::config::WasmExecutionMethod::Interpreted, + #[cfg(feature = "wasmtime")] + WasmExecutionMethod::Compiled => service::config::WasmExecutionMethod::Compiled, + #[cfg(not(feature = "wasmtime"))] + WasmExecutionMethod::Compiled => panic!( + "Substrate must be compiled with \"wasmtime\" feature for compiled Wasm execution" + ), } } } @@ -126,12 +145,18 @@ pub struct NetworkConfigurationParams { #[structopt(long = "port", value_name = "PORT")] pub port: Option, + /// Allow connecting to private IPv4 addresses (as specified in + /// [RFC1918](https://tools.ietf.org/html/rfc1918)), unless the address was passed with + /// `--reserved-nodes` or `--bootnodes`. + #[structopt(long = "no-private-ipv4")] + pub no_private_ipv4: bool, + /// Specify the number of outgoing connections we're trying to maintain. - #[structopt(long = "out-peers", value_name = "OUT_PEERS", default_value = "25")] + #[structopt(long = "out-peers", value_name = "COUNT", default_value = "25")] pub out_peers: u32, /// Specify the maximum number of incoming connections we're accepting. - #[structopt(long = "in-peers", value_name = "IN_PEERS", default_value = "25")] + #[structopt(long = "in-peers", value_name = "COUNT", default_value = "25")] pub in_peers: u32, /// Disable mDNS discovery. @@ -141,6 +166,13 @@ pub struct NetworkConfigurationParams { #[structopt(long = "no-mdns")] pub no_mdns: bool, + /// Maximum number of peers to ask the same blocks in parallel. + /// + /// This allows downlading announced blocks from multiple peers. Decrease to save + /// traffic and risk increased latency. + #[structopt(long = "max-parallel-downloads", value_name = "COUNT", default_value = "5")] + pub max_parallel_downloads: u32, + #[allow(missing_docs)] #[structopt(flatten)] pub node_key_params: NodeKeyParams @@ -354,10 +386,18 @@ pub struct RunCmd { #[structopt(long = "ws-external")] pub ws_external: bool, + #[structopt(long = "prometheus-port", value_name = "PORT")] + pub prometheus_port: Option, + #[structopt(long = "prometheus-addr", value_name = "Local IP address")] + pub prometheus_endpoint: Option, + /// Specify HTTP RPC server TCP port. #[structopt(long = "rpc-port", value_name = "PORT")] pub rpc_port: Option, + #[structopt(long = "prometheus-port", value_name = "PORT")] + pub prometheus_port: Option, + /// Specify WebSockets RPC server TCP port. #[structopt(long = "ws-port", value_name = "PORT")] pub ws_port: Option, @@ -413,6 +453,8 @@ pub struct RunCmd { #[structopt(long = "telemetry-url", value_name = "URL VERBOSITY", parse(try_from_str = parse_telemetry_endpoints))] pub telemetry_endpoints: Vec<(String, u8)>, + #[structopt(long = "prometheus-addr", value_name = "Local IP address")] + pub prometheus_endpoint: Option, /// Should execute offchain workers on every block. /// /// By default it's only enabled for nodes that are authoring new blocks. @@ -429,7 +471,7 @@ pub struct RunCmd { #[structopt( long = "wasm-execution", value_name = "METHOD", - possible_values = &WasmExecutionMethod::variants(), + possible_values = &WasmExecutionMethod::enabled_variants(), case_insensitive = true, default_value = "Interpreted" )] @@ -835,7 +877,7 @@ impl GetLogFilter for CoreParams where CC: GetLogFilter { /// A special commandline parameter that expands to nothing. /// Should be used as custom subcommand/run arguments if no custom values are required. -#[derive(Clone, Debug)] +#[derive(Clone, Debug, Default)] pub struct NoCustom {} impl StructOpt for NoCustom { diff --git a/core/cli/src/traits.rs b/client/cli/src/traits.rs similarity index 100% rename from core/cli/src/traits.rs rename to client/cli/src/traits.rs diff --git a/core/consensus/aura/Cargo.toml b/client/consensus/aura/Cargo.toml similarity index 53% rename from core/consensus/aura/Cargo.toml rename to client/consensus/aura/Cargo.toml index a09d51095018e..5975e2ab2a9dd 100644 --- a/core/consensus/aura/Cargo.toml +++ b/client/consensus/aura/Cargo.toml @@ -6,34 +6,37 @@ description = "Aura consensus algorithm for substrate" edition = "2018" [dependencies] +app-crypto = { package = "substrate-application-crypto", path = "../../../primitives/application-crypto" } +aura_primitives = { package = "substrate-consensus-aura-primitives", path = "../../../primitives/consensus/aura" } +block-builder-api = { package = "substrate-block-builder-runtime-api", path = "../../../primitives/block-builder/runtime-api" } +client = { package = "substrate-client", path = "../../" } +client-api = { package = "substrate-client-api", path = "../../api" } codec = { package = "parity-scale-codec", version = "1.0.0" } -primitives = { package = "substrate-primitives", path = "../../primitives" } -app-crypto = { package = "substrate-application-crypto", path = "../../application-crypto" } -runtime_support = { package = "srml-support", path = "../../../srml/support" } -runtime_version = { package = "sr-version", path = "../../sr-version" } -runtime_io = { package = "sr-io", path = "../../sr-io" } -slots = { package = "substrate-consensus-slots", path = "../slots" } -aura_primitives = { package = "substrate-consensus-aura-primitives", path = "primitives" } -inherents = { package = "substrate-inherents", path = "../../inherents" } -srml-aura = { path = "../../../srml/aura" } -client = { package = "substrate-client", path = "../../client" } -substrate-telemetry = { path = "../../telemetry" } -keystore = { package = "substrate-keystore", path = "../../keystore" } -consensus_common = { package = "substrate-consensus-common", path = "../common" } -sr-primitives = { path = "../../sr-primitives" } +consensus_common = { package = "substrate-consensus-common", path = "../../../primitives/consensus/common" } +derive_more = "0.15.0" futures-preview = { version = "0.3.0-alpha.19", features = ["compat"] } -futures01 = { package = "futures", version = "0.1" } futures-timer = "0.4.0" -parking_lot = "0.9.0" +futures01 = { package = "futures", version = "0.1" } +inherents = { package = "substrate-inherents", path = "../../../primitives/inherents" } +keystore = { package = "substrate-keystore", path = "../../keystore" } log = "0.4.8" -derive_more = "0.15.0" +parking_lot = "0.9.0" +primitives = { package = "substrate-primitives", path = "../../../primitives/core" } +runtime_io = { package = "sr-io", path = "../../../primitives/sr-io" } +runtime_version = { package = "sr-version", path = "../../../primitives/sr-version" } +slots = { package = "substrate-consensus-slots", path = "../slots" } +sr-api = { path = "../../../primitives/sr-api" } +sr-primitives = { path = "../../../primitives/sr-primitives" } +paint-aura = { path = "../../../paint/aura" } +substrate-telemetry = { path = "../../telemetry" } +substrate-prometheus = { path = "../../prometheus" } [dev-dependencies] -keyring = { package = "substrate-keyring", path = "../../keyring" } +keyring = { package = "substrate-keyring", path = "../../../primitives/keyring" } substrate-executor = { path = "../../executor" } network = { package = "substrate-network", path = "../../network", features = ["test-helpers"]} service = { package = "substrate-service", path = "../../service" } -test-client = { package = "substrate-test-runtime-client", path = "../../test-runtime/client" } +test-client = { package = "substrate-test-runtime-client", path = "../../../test/utils/runtime/client" } tokio = "0.1.22" env_logger = "0.7.0" tempfile = "3.1.0" diff --git a/core/consensus/aura/src/digest.rs b/client/consensus/aura/src/digest.rs similarity index 100% rename from core/consensus/aura/src/digest.rs rename to client/consensus/aura/src/digest.rs diff --git a/core/consensus/aura/src/lib.rs b/client/consensus/aura/src/lib.rs similarity index 73% rename from core/consensus/aura/src/lib.rs rename to client/consensus/aura/src/lib.rs index eb2d1dba20f1b..f350208389dc2 100644 --- a/core/consensus/aura/src/lib.rs +++ b/client/consensus/aura/src/lib.rs @@ -28,43 +28,58 @@ //! //! NOTE: Aura itself is designed to be generic over the crypto used. #![forbid(missing_docs, unsafe_code)] -use std::{sync::Arc, time::Duration, thread, marker::PhantomData, hash::Hash, fmt::Debug, pin::Pin}; +use std::{ + fmt::Debug, hash::Hash, marker::PhantomData, pin::Pin, sync::Arc, thread, time::Duration, +}; -use codec::{Encode, Decode, Codec}; -use consensus_common::{self, BlockImport, Environment, Proposer, - ForkChoiceStrategy, BlockImportParams, BlockOrigin, Error as ConsensusError, - SelectChain, +use client::{ + blockchain::ProvideCache, + well_known_cache_keys::{self, Id as CacheKeyId}, + BlockOf, }; +use client_api::{backend::AuxStore, error::Result as CResult}; +use codec::{Codec, Decode, Encode}; use consensus_common::import_queue::{ - Verifier, BasicQueue, BoxBlockImport, BoxJustificationImport, BoxFinalityProofImport, + BasicQueue, BoxBlockImport, BoxFinalityProofImport, BoxJustificationImport, Verifier, }; -use client::{ - block_builder::api::BlockBuilder as BlockBuilderApi, blockchain::ProvideCache, - runtime_api::ApiExt, error::Result as CResult, backend::AuxStore, BlockOf, - well_known_cache_keys::{self, Id as CacheKeyId}, +use consensus_common::{ + self, BlockImport, BlockImportParams, BlockOrigin, Environment, Error as ConsensusError, + ForkChoiceStrategy, Proposer, SelectChain, }; -use sr_primitives::{generic::{BlockId, OpaqueDigestItemId}, Justification}; -use sr_primitives::traits::{Block as BlockT, Header, DigestItemFor, ProvideRuntimeApi, Zero, Member}; +use block_builder_api::BlockBuilder as BlockBuilderApi; +use sr_primitives::traits::{ + Block as BlockT, DigestItemFor, Header, Member, ProvideRuntimeApi, Zero, +}; +use sr_primitives::{ + generic::{BlockId, OpaqueDigestItemId}, + Justification, +}; + +use inherents::{InherentData, InherentDataProviders}; use primitives::crypto::Pair; -use inherents::{InherentDataProviders, InherentData, RuntimeString}; use futures::prelude::*; -use parking_lot::Mutex; use log::{debug, info, trace}; +use parking_lot::Mutex; -use srml_aura::{ - InherentType as AuraInherent, AuraInherentData, - timestamp::{TimestampInherentData, InherentType as TimestampInherent, InherentError as TIError} +use paint_aura::{ + timestamp::{ + InherentError as TIError, InherentType as TimestampInherent, TimestampInherentData, + }, + AuraInherentData, InherentType as AuraInherent, }; -use substrate_telemetry::{telemetry, CONSENSUS_TRACE, CONSENSUS_DEBUG, CONSENSUS_INFO}; +use substrate_prometheus::prometheus_gauge; +use substrate_telemetry::{telemetry, CONSENSUS_DEBUG, CONSENSUS_INFO, CONSENSUS_TRACE}; -use slots::{CheckedHeader, SlotData, SlotWorker, SlotInfo, SlotCompatible}; use slots::check_equivocation; +use slots::{CheckedHeader, SlotCompatible, SlotData, SlotInfo, SlotWorker}; use keystore::KeyStorePtr; +use sr_api::ApiExt; + pub use aura_primitives::*; pub use consensus_common::SyncOracle; pub use digest::CompatibleDigestItem; @@ -85,7 +100,7 @@ impl SlotDuration { A: Codec, B: BlockT, C: AuxStore + ProvideRuntimeApi, - C::Api: AuraApi, + C::Api: AuraApi, { slots::SlotDuration::get_or_compute(client, |a, b| a.slot_duration(b)).map(Self) } @@ -98,7 +113,9 @@ impl SlotDuration { /// Get slot author for given block along with authorities. fn slot_author(slot_num: u64, authorities: &[AuthorityId

]) -> Option<&AuthorityId

> { - if authorities.is_empty() { return None } + if authorities.is_empty() { + return None; + } let idx = slot_num % (authorities.len() as u64); assert!( @@ -106,9 +123,10 @@ fn slot_author(slot_num: u64, authorities: &[AuthorityId

]) -> Option "It is impossible to have a vector with length beyond the address space; qed", ); - let current_author = authorities.get(idx as usize) - .expect("authorities not empty; index constrained to list length;\ - this is a valid index; qed"); + let current_author = authorities.get(idx as usize).expect( + "authorities not empty; index constrained to list length;\ + this is a valid index; qed", + ); Some(current_author) } @@ -119,7 +137,7 @@ struct AuraSlotCompatible; impl SlotCompatible for AuraSlotCompatible { fn extract_timestamp_and_slot( &self, - data: &InherentData + data: &InherentData, ) -> Result<(TimestampInherent, AuraInherent, std::time::Duration), consensus_common::Error> { data.timestamp_inherent_data() .and_then(|t| data.aura_inherent_data().map(|a| (t, a))) @@ -140,18 +158,19 @@ pub fn start_aura( inherent_data_providers: InherentDataProviders, force_authoring: bool, keystore: KeyStorePtr, -) -> Result, consensus_common::Error> where - B: BlockT, +) -> Result, consensus_common::Error> +where + B: BlockT

, C: ProvideRuntimeApi + BlockOf + ProvideCache + AuxStore + Send + Sync, C::Api: AuraApi>, SC: SelectChain, - E: Environment + Send + Sync + 'static, - E::Proposer: Proposer, + E: Environment + Send + Sync + 'static, + E::Proposer: Proposer, >::Create: Unpin + Send, P: Pair + Send + Sync, P::Public: Hash + Member + Encode + Decode, P::Signature: Hash + Member + Encode + Decode, - H: Header, + H: Header, I: BlockImport + Send + Sync + 'static, Error: ::std::error::Error + Send + From<::consensus_common::Error> + From + 'static, SO: SyncOracle + Send + Sync + Clone, @@ -167,16 +186,20 @@ pub fn start_aura( }; register_aura_inherent_data_provider( &inherent_data_providers, - slot_duration.0.slot_duration() + slot_duration.0.slot_duration(), )?; - Ok(slots::start_slot_worker::<_, _, _, _, _, AuraSlotCompatible>( - slot_duration.0, - select_chain, - worker, - sync_oracle, - inherent_data_providers, - AuraSlotCompatible, - ).map(|()| Ok::<(), ()>(())).compat()) + Ok( + slots::start_slot_worker::<_, _, _, _, _, AuraSlotCompatible>( + slot_duration.0, + select_chain, + worker, + sync_oracle, + inherent_data_providers, + AuraSlotCompatible, + ) + .map(|()| Ok::<(), ()>(())) + .compat(), + ) } struct AuraWorker { @@ -189,14 +212,15 @@ struct AuraWorker { _key_type: PhantomData

, } -impl slots::SimpleSlotWorker for AuraWorker where - B: BlockT, +impl slots::SimpleSlotWorker for AuraWorker +where + B: BlockT

, C: ProvideRuntimeApi + BlockOf + ProvideCache + Sync, C::Api: AuraApi>, - E: Environment, - E::Proposer: Proposer, + E: Environment, + E::Proposer: Proposer, >::Create: Unpin + Send, - H: Header, + H: Header, I: BlockImport + Send + Sync + 'static, P: Pair + Send + Sync, P::Public: Member + Encode + Decode + Hash, @@ -218,7 +242,11 @@ impl slots::SimpleSlotWorker for AuraWorker Result { + fn epoch_data( + &self, + header: &B::Header, + _slot_number: u64, + ) -> Result { authorities(self.client.as_ref(), &BlockId::Hash(header.hash())) } @@ -235,28 +263,38 @@ impl slots::SimpleSlotWorker for AuraWorker(slot_number, epoch_data); expected_author.and_then(|p| { - self.keystore.read() - .key_pair_by_type::

(&p, app_crypto::key_types::AURA).ok() + self.keystore + .read() + .key_pair_by_type::

(&p, app_crypto::key_types::AURA) + .ok() }) } - fn pre_digest_data(&self, slot_number: u64, _claim: &Self::Claim) -> Vec> { - vec![ - as CompatibleDigestItem

>::aura_pre_digest(slot_number), - ] + fn pre_digest_data( + &self, + slot_number: u64, + _claim: &Self::Claim, + ) -> Vec> { + vec![ as CompatibleDigestItem

>::aura_pre_digest(slot_number)] } - fn block_import_params(&self) -> Box, - Self::Claim, - ) -> consensus_common::BlockImportParams + Send> { + fn block_import_params( + &self, + ) -> Box< + dyn Fn( + B::Header, + &B::Hash, + Vec, + Self::Claim, + ) -> consensus_common::BlockImportParams + + Send, + > { Box::new(|header, header_hash, body, pair| { // sign the pre-sealed hash of the block and then // add it to a digest item. let signature = pair.sign(header_hash.as_ref()); - let signature_digest_item = as CompatibleDigestItem

>::aura_seal(signature); + let signature_digest_item = + as CompatibleDigestItem

>::aura_seal(signature); BlockImportParams { origin: BlockOrigin::Own, @@ -267,6 +305,7 @@ impl slots::SimpleSlotWorker for AuraWorker slots::SimpleSlotWorker for AuraWorker Result { - self.env.init(block).map_err(|e| { - consensus_common::Error::ClientImport(format!("{:?}", e)).into() - }) + self.env + .init(block) + .map_err(|e| consensus_common::Error::ClientImport(format!("{:?}", e)).into()) } } -impl SlotWorker for AuraWorker where - B: BlockT, +impl SlotWorker for AuraWorker +where + B: BlockT

, C: ProvideRuntimeApi + BlockOf + ProvideCache + Sync + Send, C::Api: AuraApi>, - E: Environment + Send + Sync, - E::Proposer: Proposer, + E: Environment + Send + Sync, + E::Proposer: Proposer, >::Create: Unpin + Send + 'static, - H: Header, + H: Header, I: BlockImport + Send + Sync + 'static, P: Pair + Send + Sync, P::Public: Member + Encode + Decode + Hash, @@ -331,13 +371,14 @@ enum Error { TooFarInFuture, Client(client::error::Error), DataProvider(String), - Runtime(RuntimeString) + Runtime(String), } fn find_pre_digest(header: &B::Header) -> Result> - where DigestItemFor: CompatibleDigestItem

, - P::Signature: Decode, - P::Public: Encode + Decode + PartialEq + Clone, +where + DigestItemFor: CompatibleDigestItem

, + P::Signature: Decode, + P::Public: Encode + Decode + PartialEq + Clone, { let mut pre_digest: Option = None; for log in header.digest().logs() { @@ -365,10 +406,11 @@ fn check_header( hash: B::Hash, authorities: &[AuthorityId

], _transaction_pool: Option<&T>, -) -> Result)>, Error> where +) -> Result)>, Error> +where DigestItemFor: CompatibleDigestItem

, P::Signature: Decode, - C: client::backend::AuxStore, + C: client_api::backend::AuxStore, P::Public: Encode + Decode + PartialEq + Clone, T: Send + Sync + 'static, { @@ -377,9 +419,9 @@ fn check_header( None => return Err(Error::HeaderUnsealed(hash)), }; - let sig = seal.as_aura_seal().ok_or_else(|| { - aura_err(Error::HeaderBadSeal(hash)) - })?; + let sig = seal + .as_aura_seal() + .ok_or_else(|| aura_err(Error::HeaderBadSeal(hash)))?; let slot_num = find_pre_digest::(&header)?; @@ -397,13 +439,10 @@ fn check_header( let pre_hash = header.hash(); if P::verify(&sig, pre_hash.as_ref(), expected_author) { - if let Some(equivocation_proof) = check_equivocation( - client, - slot_now, - slot_num, - &header, - expected_author, - ).map_err(Error::Client)? { + if let Some(equivocation_proof) = + check_equivocation(client, slot_now, slot_num, &header, expected_author) + .map_err(Error::Client)? + { info!( "Slot author is equivocating at slot {} with headers {:?} and {:?}", slot_num, @@ -428,7 +467,8 @@ pub struct AuraVerifier { } impl AuraVerifier - where P: Send + Sync + 'static +where + P: Send + Sync + 'static, { fn check_inherents( &self, @@ -437,15 +477,17 @@ impl AuraVerifier inherent_data: InherentData, timestamp_now: u64, ) -> Result<(), Error> - where C: ProvideRuntimeApi, C::Api: BlockBuilderApi + where + C: ProvideRuntimeApi, + C::Api: BlockBuilderApi, { const MAX_TIMESTAMP_DRIFT_SECS: u64 = 60; - let inherent_res = self.client.runtime_api().check_inherents( - &block_id, - block, - inherent_data, - ).map_err(Error::Client)?; + let inherent_res = self + .client + .runtime_api() + .check_inherents(&block_id, block, inherent_data) + .map_err(Error::Client)?; if !inherent_res.ok() { inherent_res @@ -467,12 +509,13 @@ impl AuraVerifier telemetry!(CONSENSUS_INFO; "aura.halting_for_future_block"; "diff" => ?diff ); + //prometheus_gauge!(BLOCK_INTERVAL_SECONDS => ?diff as u64); thread::sleep(Duration::from_secs(diff)); Ok(()) - }, - Some(TIError::Other(e)) => Err(Error::Runtime(e)), + } + Some(TIError::Other(e)) => Err(Error::Runtime(e.into())), None => Err(Error::DataProvider( - self.inherent_data_providers.error_to_string(&i, &e) + self.inherent_data_providers.error_to_string(&i, &e), )), }) } else { @@ -482,9 +525,11 @@ impl AuraVerifier } #[forbid(deprecated)] -impl Verifier for AuraVerifier where - C: ProvideRuntimeApi + Send + Sync + client::backend::AuxStore + ProvideCache + BlockOf, - C::Api: BlockBuilderApi + AuraApi>, +impl Verifier for AuraVerifier +where + C: ProvideRuntimeApi + Send + Sync + client_api::backend::AuxStore + ProvideCache + BlockOf, + C::Api: + BlockBuilderApi + AuraApi> + ApiExt, DigestItemFor: CompatibleDigestItem

, P: Pair + Send + Sync + 'static, P::Public: Send + Sync + Hash + Eq + Clone + Decode + Encode + Debug + 'static, @@ -498,8 +543,12 @@ impl Verifier for AuraVerifier where justification: Option, mut body: Option>, ) -> Result<(BlockImportParams, Option)>>), String> { - let mut inherent_data = self.inherent_data_providers.create_inherent_data().map_err(String::from)?; - let (timestamp_now, slot_now, _) = AuraSlotCompatible.extract_timestamp_and_slot(&inherent_data) + let mut inherent_data = self + .inherent_data_providers + .create_inherent_data() + .map_err(|e| e.into_string())?; + let (timestamp_now, slot_now, _) = AuraSlotCompatible + .extract_timestamp_and_slot(&inherent_data) .map_err(|e| format!("Could not extract timestamp and slot: {:?}", e))?; let hash = header.hash(); let parent_hash = *header.parent_hash(); @@ -516,7 +565,8 @@ impl Verifier for AuraVerifier where hash, &authorities[..], self.transaction_pool.as_ref().map(|x| &**x), - ).map_err(|e| e.to_string())?; + ) + .map_err(|e| e.to_string())?; match checked_header { CheckedHeader::Checked(pre_header, (slot_num, seal)) => { // if the body is passed through, we need to use the runtime @@ -527,9 +577,13 @@ impl Verifier for AuraVerifier where let block = B::new(pre_header.clone(), inner_body); // skip the inherents verification if the runtime API is old. - if self.client + if self + .client .runtime_api() - .has_api_with::, _>(&BlockId::Hash(parent_hash), |v| v >= 2) + .has_api_with::, _>( + &BlockId::Hash(parent_hash), + |v| v >= 2, + ) .map_err(|e| format!("{:?}", e))? { self.check_inherents( @@ -537,7 +591,8 @@ impl Verifier for AuraVerifier where BlockId::Hash(parent_hash), inherent_data, timestamp_now, - ).map_err(|e| e.to_string())?; + ) + .map_err(|e| e.to_string())?; } let (_, inner_body) = block.deconstruct(); @@ -546,18 +601,20 @@ impl Verifier for AuraVerifier where trace!(target: "aura", "Checked {:?}; importing.", pre_header); telemetry!(CONSENSUS_TRACE; "aura.checked_and_importing"; "pre_header" => ?pre_header); - // Look for an authorities-change log. - let maybe_keys = pre_header.digest() + let maybe_keys = pre_header + .digest() .logs() .iter() - .filter_map(|l| l.try_to::>>( - OpaqueDigestItemId::Consensus(&AURA_ENGINE_ID) - )) + .filter_map(|l| { + l.try_to::>>(OpaqueDigestItemId::Consensus( + &AURA_ENGINE_ID, + )) + }) .find_map(|l| match l { - ConsensusLog::AuthoritiesChange(a) => Some( - vec![(well_known_cache_keys::AUTHORITIES, a.encode())] - ), + ConsensusLog::AuthoritiesChange(a) => { + Some(vec![(well_known_cache_keys::AUTHORITIES, a.encode())]) + } _ => None, }); @@ -570,6 +627,7 @@ impl Verifier for AuraVerifier where justification, auxiliary: Vec::new(), fork_choice: ForkChoiceStrategy::LongestChain, + allow_missing_state: false, }; Ok((block_import_params, maybe_keys)) @@ -585,7 +643,8 @@ impl Verifier for AuraVerifier where } } -fn initialize_authorities_cache(client: &C) -> Result<(), ConsensusError> where +fn initialize_authorities_cache(client: &C) -> Result<(), ConsensusError> +where A: Codec, B: BlockT, C: ProvideRuntimeApi + BlockOf + ProvideCache, @@ -606,20 +665,26 @@ fn initialize_authorities_cache(client: &C) -> Result<(), ConsensusErro return Ok(()); } - let map_err = |error| consensus_common::Error::from(consensus_common::Error::ClientImport( - format!( + let map_err = |error| { + consensus_common::Error::from(consensus_common::Error::ClientImport(format!( "Error initializing authorities cache: {}", error, - ))); + ))) + }; let genesis_authorities = authorities(client, &genesis_id)?; - cache.initialize(&well_known_cache_keys::AUTHORITIES, genesis_authorities.encode()) + cache + .initialize( + &well_known_cache_keys::AUTHORITIES, + genesis_authorities.encode(), + ) .map_err(map_err)?; Ok(()) } #[allow(deprecated)] -fn authorities(client: &C, at: &BlockId) -> Result, ConsensusError> where +fn authorities(client: &C, at: &BlockId) -> Result, ConsensusError> +where A: Codec, B: BlockT, C: ProvideRuntimeApi + BlockOf + ProvideCache, @@ -627,10 +692,11 @@ fn authorities(client: &C, at: &BlockId) -> Result, Consensus { client .cache() - .and_then(|cache| cache - .get_at(&well_known_cache_keys::AUTHORITIES, at) - .and_then(|(_, _, v)| Decode::decode(&mut &v[..]).ok()) - ) + .and_then(|cache| { + cache + .get_at(&well_known_cache_keys::AUTHORITIES, at) + .and_then(|(_, _, v)| Decode::decode(&mut &v[..]).ok()) + }) .or_else(|| AuraApi::authorities(&*client.runtime_api(), at).ok()) .ok_or_else(|| consensus_common::Error::InvalidAuthoritiesSet.into()) } @@ -643,9 +709,9 @@ fn register_aura_inherent_data_provider( inherent_data_providers: &InherentDataProviders, slot_duration: u64, ) -> Result<(), consensus_common::Error> { - if !inherent_data_providers.has_provider(&srml_aura::INHERENT_IDENTIFIER) { + if !inherent_data_providers.has_provider(&paint_aura::INHERENT_IDENTIFIER) { inherent_data_providers - .register_provider(srml_aura::InherentDataProvider::new(slot_duration)) + .register_provider(paint_aura::InherentDataProvider::new(slot_duration)) .map_err(Into::into) .map_err(consensus_common::Error::InherentData) } else { @@ -662,10 +728,12 @@ pub fn import_queue( client: Arc, inherent_data_providers: InherentDataProviders, transaction_pool: Option>, -) -> Result, consensus_common::Error> where +) -> Result, consensus_common::Error> +where B: BlockT, C: 'static + ProvideRuntimeApi + BlockOf + ProvideCache + Send + Sync + AuxStore, - C::Api: BlockBuilderApi + AuraApi>, + C::Api: + BlockBuilderApi + AuraApi> + ApiExt, DigestItemFor: CompatibleDigestItem

, P: Pair + Send + Sync + 'static, P::Public: Clone + Eq + Send + Sync + Hash + Debug + Encode + Decode, @@ -692,17 +760,17 @@ pub fn import_queue( #[cfg(test)] mod tests { use super::*; + use aura_primitives::sr25519::AuthorityPair; + use client::BlockchainEvents; use consensus_common::NoNetwork as DummyOracle; + use keyring::sr25519::Keyring; + use network::config::ProtocolConfig; use network::test::*; use network::test::{Block as TestBlock, PeersClient, PeersFullClient}; - use sr_primitives::traits::{Block as BlockT, DigestFor}; - use network::config::ProtocolConfig; use parking_lot::Mutex; - use tokio::runtime::current_thread; - use keyring::sr25519::Keyring; - use client::BlockchainEvents; + use sr_primitives::traits::{Block as BlockT, DigestFor}; use test_client; - use aura_primitives::sr25519::AuthorityPair; + use tokio::runtime::current_thread; type Error = client::error::Error; @@ -710,7 +778,7 @@ mod tests { test_client::Backend, test_client::Executor, TestBlock, - test_client::runtime::RuntimeApi + test_client::runtime::RuntimeApi, >; struct DummyFactory(Arc); @@ -720,9 +788,10 @@ mod tests { type Proposer = DummyProposer; type Error = Error; - fn init(&mut self, parent_header: &::Header) - -> Result - { + fn init( + &mut self, + parent_header: &::Header, + ) -> Result { Ok(DummyProposer(parent_header.number + 1, self.0.clone())) } } @@ -737,7 +806,12 @@ mod tests { digests: DigestFor, _: Duration, ) -> Self::Create { - let r = self.1.new_block(digests).unwrap().bake().map_err(|e| e.into()); + let r = self + .1 + .new_block(digests) + .unwrap() + .bake() + .map_err(|e| e.into()); future::ready(r) } } @@ -755,23 +829,25 @@ mod tests { /// Create new test network with peers and given config. fn from_config(_config: &ProtocolConfig) -> Self { - AuraTestNet { - peers: Vec::new(), - } + AuraTestNet { peers: Vec::new() } } - fn make_verifier(&self, client: PeersClient, _cfg: &ProtocolConfig, _peer_data: &()) - -> Self::Verifier - { + fn make_verifier( + &self, + client: PeersClient, + _cfg: &ProtocolConfig, + _peer_data: &(), + ) -> Self::Verifier { match client { PeersClient::Full(client, _) => { - let slot_duration = SlotDuration::get_or_compute(&*client) - .expect("slot duration available"); + let slot_duration = + SlotDuration::get_or_compute(&*client).expect("slot duration available"); let inherent_data_providers = InherentDataProviders::new(); register_aura_inherent_data_provider( &inherent_data_providers, - slot_duration.get() - ).expect("Registers aura inherent data provider"); + slot_duration.get(), + ) + .expect("Registers aura inherent data provider"); assert_eq!(slot_duration.get(), SLOT_DURATION); AuraVerifier { @@ -780,7 +856,7 @@ mod tests { transaction_pool: Default::default(), phantom: Default::default(), } - }, + } PeersClient::Light(_, _) => unreachable!("No (yet) tests for light client + Aura"), } } @@ -793,7 +869,10 @@ mod tests { &self.peers } - fn mut_peers>)>(&mut self, closure: F) { + fn mut_peers>)>( + &mut self, + closure: F, + ) { closure(&mut self.peers); } } @@ -818,29 +897,38 @@ mod tests { for (peer_id, key) in peers { let mut net = net.lock(); let peer = net.peer(*peer_id); - let client = peer.client().as_full().expect("full clients are created").clone(); + let client = peer + .client() + .as_full() + .expect("full clients are created") + .clone(); let select_chain = peer.select_chain().expect("full client has a select chain"); let keystore_path = tempfile::tempdir().expect("Creates keystore path"); - let keystore = keystore::Store::open(keystore_path.path(), None).expect("Creates keystore."); + let keystore = + keystore::Store::open(keystore_path.path(), None).expect("Creates keystore."); - keystore.write().insert_ephemeral_from_seed::(&key.to_seed()) + keystore + .write() + .insert_ephemeral_from_seed::(&key.to_seed()) .expect("Creates authority key"); keystore_paths.push(keystore_path); let environ = DummyFactory(client.clone()); import_notifications.push( - client.import_notification_stream() - .take_while(|n| future::ready(!(n.origin != BlockOrigin::Own && n.header.number() < &5))) - .for_each(move |_| future::ready(())) + client + .import_notification_stream() + .take_while(|n| { + future::ready(!(n.origin != BlockOrigin::Own && n.header.number() < &5)) + }) + .for_each(move |_| future::ready(())), ); - let slot_duration = SlotDuration::get_or_compute(&*client) - .expect("slot duration available"); + let slot_duration = + SlotDuration::get_or_compute(&*client).expect("slot duration available"); let inherent_data_providers = InherentDataProviders::new(); - register_aura_inherent_data_provider( - &inherent_data_providers, slot_duration.get() - ).expect("Registers aura inherent data provider"); + register_aura_inherent_data_provider(&inherent_data_providers, slot_duration.get()) + .expect("Registers aura inherent data provider"); let aura = start_aura::<_, _, _, _, _, AuthorityPair, _, _, _>( slot_duration, @@ -852,7 +940,8 @@ mod tests { inherent_data_providers, false, keystore, - ).expect("Starts aura"); + ) + .expect("Starts aura"); runtime.spawn(aura); } @@ -862,8 +951,13 @@ mod tests { Ok::<_, ()>(futures01::Async::NotReady::<()>) })); - runtime.block_on(future::join_all(import_notifications) - .map(|_| Ok::<(), ()>(())).compat()).unwrap(); + runtime + .block_on( + future::join_all(import_notifications) + .map(|_| Ok::<(), ()>(())) + .compat(), + ) + .unwrap(); } #[test] @@ -871,10 +965,13 @@ mod tests { let client = test_client::new(); assert_eq!(client.info().chain.best_number, 0); - assert_eq!(authorities(&client, &BlockId::Number(0)).unwrap(), vec![ - Keyring::Alice.public().into(), - Keyring::Bob.public().into(), - Keyring::Charlie.public().into() - ]); + assert_eq!( + authorities(&client, &BlockId::Number(0)).unwrap(), + vec![ + Keyring::Alice.public().into(), + Keyring::Bob.public().into(), + Keyring::Charlie.public().into() + ] + ); } } diff --git a/core/consensus/babe/Cargo.toml b/client/consensus/babe/Cargo.toml similarity index 57% rename from core/consensus/babe/Cargo.toml rename to client/consensus/babe/Cargo.toml index 30b7c1f8d6b1c..a4a92f04f0247 100644 --- a/core/consensus/babe/Cargo.toml +++ b/client/consensus/babe/Cargo.toml @@ -7,26 +7,28 @@ edition = "2018" [dependencies] codec = { package = "parity-scale-codec", version = "1.0.0", features = ["derive"] } -babe_primitives = { package = "substrate-consensus-babe-primitives", path = "primitives" } -primitives = { package = "substrate-primitives", path = "../../primitives" } -app-crypto = { package = "substrate-application-crypto", path = "../../application-crypto" } +babe_primitives = { package = "substrate-consensus-babe-primitives", path = "../../../primitives/consensus/babe" } +primitives = { package = "substrate-primitives", path = "../../../primitives/core" } +app-crypto = { package = "substrate-application-crypto", path = "../../../primitives/application-crypto" } num-bigint = "0.2.3" num-rational = "0.2.2" num-traits = "0.2.8" -runtime-support = { package = "srml-support", path = "../../../srml/support" } -runtime-version = { package = "sr-version", path = "../../sr-version" } -runtime-io = { package = "sr-io", path = "../../sr-io" } -inherents = { package = "substrate-inherents", path = "../../inherents" } +runtime-version = { package = "sr-version", path = "../../../primitives/sr-version" } +runtime-io = { package = "sr-io", path = "../../../primitives/sr-io" } +inherents = { package = "substrate-inherents", path = "../../../primitives/inherents" } substrate-telemetry = { path = "../../telemetry" } keystore = { package = "substrate-keystore", path = "../../keystore" } -srml-babe = { path = "../../../srml/babe" } -client = { package = "substrate-client", path = "../../client" } -header-metadata = { package = "substrate-header-metadata", path = "../../client/header-metadata" } -consensus-common = { package = "substrate-consensus-common", path = "../common" } +paint-babe = { path = "../../../paint/babe" } +client-api = { package = "substrate-client-api", path = "../../api" } +client = { package = "substrate-client", path = "../../" } +sr-api = { path = "../../../primitives/sr-api" } +block-builder-api = { package = "substrate-block-builder-runtime-api", path = "../../../primitives/block-builder/runtime-api" } +header-metadata = { package = "substrate-header-metadata", path = "../../header-metadata" } +consensus-common = { package = "substrate-consensus-common", path = "../../../primitives/consensus/common" } uncles = { package = "substrate-consensus-uncles", path = "../uncles" } slots = { package = "substrate-consensus-slots", path = "../slots" } -sr-primitives = { path = "../../sr-primitives" } -fork-tree = { path = "../../utils/fork-tree" } +sr-primitives = { path = "../../../primitives/sr-primitives" } +fork-tree = { path = "../../../utils/fork-tree" } futures-preview = { version = "0.3.0-alpha.19", features = ["compat"] } futures01 = { package = "futures", version = "0.1" } futures-timer = "0.4.0" @@ -39,11 +41,12 @@ pdqselect = "0.1.0" derive_more = "0.15.0" [dev-dependencies] -keyring = { package = "substrate-keyring", path = "../../keyring" } +keyring = { package = "substrate-keyring", path = "../../../primitives/keyring" } substrate-executor = { path = "../../executor" } network = { package = "substrate-network", path = "../../network", features = ["test-helpers"]} service = { package = "substrate-service", path = "../../service" } -test-client = { package = "substrate-test-runtime-client", path = "../../test-runtime/client" } +test-client = { package = "substrate-test-runtime-client", path = "../../../test/utils/runtime/client" } +block-builder = { package = "substrate-block-builder", path = "../../block-builder" } tokio = "0.1.22" env_logger = "0.7.0" tempfile = "3.1.0" diff --git a/core/consensus/babe/src/authorship.rs b/client/consensus/babe/src/authorship.rs similarity index 100% rename from core/consensus/babe/src/authorship.rs rename to client/consensus/babe/src/authorship.rs diff --git a/core/consensus/babe/src/aux_schema.rs b/client/consensus/babe/src/aux_schema.rs similarity index 96% rename from core/consensus/babe/src/aux_schema.rs rename to client/consensus/babe/src/aux_schema.rs index 6290d5cf31643..04e9f81e5cffe 100644 --- a/core/consensus/babe/src/aux_schema.rs +++ b/client/consensus/babe/src/aux_schema.rs @@ -19,8 +19,10 @@ use log::info; use codec::{Decode, Encode}; -use client::backend::AuxStore; -use client::error::{Result as ClientResult, Error as ClientError}; +use client_api::{ + backend::AuxStore, + error::{Result as ClientResult, Error as ClientError}, +}; use sr_primitives::traits::Block as BlockT; use babe_primitives::BabeBlockWeight; diff --git a/core/consensus/babe/src/epoch_changes.rs b/client/consensus/babe/src/epoch_changes.rs similarity index 99% rename from core/consensus/babe/src/epoch_changes.rs rename to client/consensus/babe/src/epoch_changes.rs index 09a14d286426d..a5621efcd15a3 100644 --- a/core/consensus/babe/src/epoch_changes.rs +++ b/client/consensus/babe/src/epoch_changes.rs @@ -25,9 +25,11 @@ use fork_tree::ForkTree; use parking_lot::{Mutex, MutexGuard}; use sr_primitives::traits::{Block as BlockT, NumberFor, One, Zero}; use codec::{Encode, Decode}; -use client::error::Error as ClientError; -use client::utils as client_utils; -use client::blockchain::HeaderBackend; +use client_api::{ + error::Error as ClientError, + utils::is_descendent_of, + blockchain::HeaderBackend +}; use header_metadata::HeaderMetadata; use primitives::H256; use std::ops::Add; @@ -72,7 +74,7 @@ impl<'a, H, Block> IsDescendentOfBuilder fn build_is_descendent_of(&self, current: Option<(H256, H256)>) -> Self::IsDescendentOf { - Box::new(client_utils::is_descendent_of(self.0, current)) + Box::new(is_descendent_of(self.0, current)) } } diff --git a/core/consensus/babe/src/lib.rs b/client/consensus/babe/src/lib.rs similarity index 96% rename from core/consensus/babe/src/lib.rs rename to client/consensus/babe/src/lib.rs index 287b26a300f43..dbfd3dbfa4b54 100644 --- a/core/consensus/babe/src/lib.rs +++ b/client/consensus/babe/src/lib.rs @@ -66,7 +66,7 @@ use consensus_common::ImportResult; use consensus_common::import_queue::{ BoxJustificationImport, BoxFinalityProofImport, }; -use sr_primitives::{generic::{BlockId, OpaqueDigestItemId}, Justification, RuntimeString}; +use sr_primitives::{generic::{BlockId, OpaqueDigestItemId}, Justification}; use sr_primitives::traits::{ Block as BlockT, Header, DigestItemFor, ProvideRuntimeApi, Zero, @@ -75,27 +75,28 @@ use keystore::KeyStorePtr; use parking_lot::Mutex; use primitives::{Blake2Hasher, H256, Pair}; use inherents::{InherentDataProviders, InherentData}; -use substrate_telemetry::{ - telemetry, - CONSENSUS_TRACE, - CONSENSUS_DEBUG, -}; +use substrate_telemetry::{telemetry, CONSENSUS_TRACE, CONSENSUS_DEBUG}; use consensus_common::{ self, BlockImport, Environment, Proposer, BlockCheckParams, ForkChoiceStrategy, BlockImportParams, BlockOrigin, Error as ConsensusError, }; -use srml_babe::{ +use paint_babe::{ BabeInherentData, timestamp::{TimestampInherentData, InherentType as TimestampInherent} }; use consensus_common::SelectChain; use consensus_common::import_queue::{Verifier, BasicQueue, CacheKeyId}; -use client::{ - block_builder::api::BlockBuilder as BlockBuilderApi, - blockchain::{self, HeaderBackend, ProvideCache}, BlockchainEvents, CallExecutor, Client, - error::Result as ClientResult, error::Error as ClientError, backend::{AuxStore, Backend}, - ProvideUncles, +use client_api::{ + backend::{AuxStore, Backend}, + call_executor::CallExecutor, + error::{Result as ClientResult, Error as ClientError}, + blockchain::{self, HeaderBackend, ProvideCache}, + BlockchainEvents, ProvideUncles, }; +use client::Client; + +use block_builder_api::BlockBuilder as BlockBuilderApi; + use slots::{CheckedHeader, check_equivocation}; use futures::prelude::*; use log::{warn, debug, info, trace}; @@ -104,6 +105,8 @@ use epoch_changes::descendent_query; use header_metadata::HeaderMetadata; use schnorrkel::SignatureError; +use sr_api::ApiExt; + mod aux_schema; mod verification; mod epoch_changes; @@ -153,11 +156,11 @@ enum Error { #[display(fmt = "VRF verification failed: {:?}", _0)] VRFVerificationFailed(SignatureError), #[display(fmt = "Could not fetch parent header: {:?}", _0)] - FetchParentHeader(client::error::Error), + FetchParentHeader(client_api::error::Error), #[display(fmt = "Expected epoch change to happen at {:?}, s{}", _0, _1)] ExpectedEpochChange(B::Hash, u64), #[display(fmt = "Could not look up epoch: {:?}", _0)] - CouldNotLookUpEpoch(Box>), + CouldNotLookUpEpoch(Box>), #[display(fmt = "Block {} is not valid under any epoch.", _0)] BlockNotValid(B::Hash), #[display(fmt = "Unexpected epoch change")] @@ -166,9 +169,9 @@ enum Error { ParentBlockNoAssociatedWeight(B::Hash), #[display(fmt = "Checking inherents failed: {}", _0)] CheckInherents(String), - Client(client::error::Error), - Runtime(RuntimeString), - ForkTree(Box>), + Client(client_api::error::Error), + Runtime(inherents::Error), + ForkTree(Box>), } impl std::convert::From> for String { @@ -202,7 +205,7 @@ impl Config { /// Either fetch the slot duration from disk or compute it from the genesis /// state. pub fn get_or_compute(client: &C) -> ClientResult where - C: AuxStore + ProvideRuntimeApi, C::Api: BabeApi, + C: AuxStore + ProvideRuntimeApi, C::Api: BabeApi, { trace!(target: "babe", "Getting slot duration"); match slots::SlotDuration::get_or_compute(client, |a, b| a.configuration(b)).map(Self) { @@ -430,6 +433,7 @@ impl slots::SimpleSlotWorker for BabeWorker BabeVerifier { block_id: BlockId, inherent_data: InherentData, ) -> Result<(), Error> - where PRA: ProvideRuntimeApi, PRA::Api: BlockBuilderApi + where PRA: ProvideRuntimeApi, PRA::Api: BlockBuilderApi { let inherent_res = self.api.runtime_api().check_inherents( &block_id, @@ -622,7 +626,8 @@ impl Verifier for BabeVerifier + 'static + Clone + Send + Sync, RA: Send + Sync, PRA: ProvideRuntimeApi + Send + Sync + AuxStore + ProvideCache, - PRA::Api: BlockBuilderApi + BabeApi, + PRA::Api: BlockBuilderApi + + BabeApi, { fn verify( &mut self, @@ -741,6 +746,7 @@ impl Verifier for BabeVerifier Result<(), consensus_common::Error> { debug!(target: "babe", "Registering"); - if !inherent_data_providers.has_provider(&srml_babe::INHERENT_IDENTIFIER) { + if !inherent_data_providers.has_provider(&paint_babe::INHERENT_IDENTIFIER) { inherent_data_providers - .register_provider(srml_babe::InherentDataProvider::new(slot_duration)) + .register_provider(paint_babe::InherentDataProvider::new(slot_duration)) .map_err(Into::into) .map_err(consensus_common::Error::InherentData) } else { @@ -900,7 +906,7 @@ impl BlockImport for BabeBlockImport| ConsensusError::ChainLookup( + .map_err(|e: fork_tree::Error| ConsensusError::ChainLookup( babe_err(Error::::CouldNotLookUpEpoch(Box::new(e))).into() ))? .ok_or_else(|| ConsensusError::ClientImport( @@ -1139,7 +1145,7 @@ pub fn import_queue, I, RA, PRA>( E: CallExecutor + Clone + Send + Sync + 'static, RA: Send + Sync + 'static, PRA: ProvideRuntimeApi + ProvideCache + Send + Sync + AuxStore + 'static, - PRA::Api: BlockBuilderApi + BabeApi, + PRA::Api: BlockBuilderApi + BabeApi + ApiExt, { register_babe_inherent_data_provider(&inherent_data_providers, babe_link.config.slot_duration)?; diff --git a/core/consensus/babe/src/tests.rs b/client/consensus/babe/src/tests.rs similarity index 99% rename from core/consensus/babe/src/tests.rs rename to client/consensus/babe/src/tests.rs index 2e236f190b66a..19c5844511288 100644 --- a/core/consensus/babe/src/tests.rs +++ b/client/consensus/babe/src/tests.rs @@ -23,7 +23,7 @@ use super::*; use authorship::claim_slot; use babe_primitives::{AuthorityPair, SlotNumber}; -use client::block_builder::BlockBuilder; +use block_builder::BlockBuilder; use consensus_common::NoNetwork as DummyOracle; use consensus_common::import_queue::{ BoxBlockImport, BoxJustificationImport, BoxFinalityProofImport, @@ -34,7 +34,7 @@ use network::config::BoxFinalityProofRequestBuilder; use sr_primitives::{generic::DigestItem, traits::{Block as BlockT, DigestFor}}; use network::config::ProtocolConfig; use tokio::runtime::current_thread; -use client::BlockchainEvents; +use client_api::BlockchainEvents; use test_client; use log::debug; use std::{time::Duration, cell::RefCell}; @@ -578,6 +578,7 @@ fn propose_and_import_block( finalized: false, auxiliary: Vec::new(), fork_choice: ForkChoiceStrategy::LongestChain, + allow_missing_state: false, }, Default::default(), ).unwrap(); @@ -632,7 +633,7 @@ fn importing_block_one_sets_genesis_epoch() { #[test] fn importing_epoch_change_block_prunes_tree() { - use client::backend::Finalizer; + use client_api::Finalizer; let mut net = BabeTestNet::new(1); diff --git a/core/consensus/babe/src/verification.rs b/client/consensus/babe/src/verification.rs similarity index 100% rename from core/consensus/babe/src/verification.rs rename to client/consensus/babe/src/verification.rs diff --git a/client/consensus/pow/Cargo.toml b/client/consensus/pow/Cargo.toml new file mode 100644 index 0000000000000..6074b7fe30da9 --- /dev/null +++ b/client/consensus/pow/Cargo.toml @@ -0,0 +1,21 @@ +[package] +name = "substrate-consensus-pow" +version = "2.0.0" +authors = ["Parity Technologies "] +description = "PoW consensus algorithm for substrate" +edition = "2018" + +[dependencies] +codec = { package = "parity-scale-codec", version = "1.0.0", features = ["derive"] } +primitives = { package = "substrate-primitives", path = "../../../primitives/core" } +sr-primitives = { path = "../../../primitives/sr-primitives" } +client-api = { package = "substrate-client-api", path = "../../api" } +block-builder-api = { package = "substrate-block-builder-runtime-api", path = "../../../primitives/block-builder/runtime-api" } +paint-timestamp = { path = "../../../paint/timestamp" } +inherents = { package = "substrate-inherents", path = "../../../primitives/inherents" } +pow-primitives = { package = "substrate-consensus-pow-primitives", path = "../../../primitives/consensus/pow" } +consensus-common = { package = "substrate-consensus-common", path = "../../../primitives/consensus/common" } +log = "0.4.8" +futures-preview = { version = "0.3.0-alpha.19", features = ["compat"] } +derive_more = "0.15.0" +substrate-prometheus = { path = "../../prometheus" } \ No newline at end of file diff --git a/core/consensus/pow/src/lib.rs b/client/consensus/pow/src/lib.rs similarity index 94% rename from core/consensus/pow/src/lib.rs rename to client/consensus/pow/src/lib.rs index 040eb01d9c5f0..57d05fcc450be 100644 --- a/core/consensus/pow/src/lib.rs +++ b/client/consensus/pow/src/lib.rs @@ -32,15 +32,15 @@ use std::sync::Arc; use std::thread; use std::collections::HashMap; -use client::{ - BlockOf, blockchain::{HeaderBackend, ProvideCache}, - block_builder::api::BlockBuilder as BlockBuilderApi, backend::AuxStore, +use client_api::{ + BlockOf, blockchain::{HeaderBackend, ProvideCache}, backend::AuxStore, well_known_cache_keys::Id as CacheKeyId, }; +use block_builder_api::BlockBuilder as BlockBuilderApi; use sr_primitives::{Justification, RuntimeString}; use sr_primitives::generic::{BlockId, Digest, DigestItem}; use sr_primitives::traits::{Block as BlockT, Header as HeaderT, ProvideRuntimeApi}; -use srml_timestamp::{TimestampInherentData, InherentError as TIError}; +use paint_timestamp::{TimestampInherentData, InherentError as TIError}; use pow_primitives::{Seal, TotalDifficulty, POW_ENGINE_ID}; use primitives::H256; use inherents::{InherentDataProviders, InherentData}; @@ -50,6 +50,7 @@ use consensus_common::{ }; use consensus_common::import_queue::{BoxBlockImport, BasicQueue, Verifier}; use codec::{Encode, Decode}; +use client_api; use log::*; #[derive(derive_more::Display, Debug)] @@ -65,7 +66,7 @@ pub enum Error { #[display(fmt = "Fetching best header failed using select chain: {:?}", _0)] BestHeaderSelectChain(ConsensusError), #[display(fmt = "Fetching best header failed: {:?}", _0)] - BestHeader(client::error::Error), + BestHeader(client_api::error::Error), #[display(fmt = "Best header does not exist")] NoBestHeader, #[display(fmt = "Block proposing error: {:?}", _0)] @@ -75,10 +76,10 @@ pub enum Error { #[display(fmt = "Error with block built on {:?}: {:?}", _0, _1)] BlockBuiltError(B::Hash, ConsensusError), #[display(fmt = "Creating inherents failed: {}", _0)] - CreateInherents(RuntimeString), + CreateInherents(inherents::Error), #[display(fmt = "Checking inherents failed: {}", _0)] CheckInherents(String), - Client(client::error::Error), + Client(client_api::error::Error), Codec(codec::Error), Environment(String), Runtime(RuntimeString) @@ -210,7 +211,7 @@ impl, C, S, Algorithm> PowVerifier { inherent_data: InherentData, timestamp_now: u64, ) -> Result<(), Error> where - C: ProvideRuntimeApi, C::Api: BlockBuilderApi + C: ProvideRuntimeApi, C::Api: BlockBuilderApi { const MAX_TIMESTAMP_DRIFT_SECS: u64 = 60; @@ -248,7 +249,7 @@ impl, C, S, Algorithm> PowVerifier { impl, C, S, Algorithm> Verifier for PowVerifier where C: ProvideRuntimeApi + Send + Sync + HeaderBackend + AuxStore + ProvideCache + BlockOf, - C::Api: BlockBuilderApi, + C::Api: BlockBuilderApi, S: SelectChain, Algorithm: PowAlgorithm + Send + Sync, { @@ -260,8 +261,8 @@ impl, C, S, Algorithm> Verifier for PowVerifier>, ) -> Result<(BlockImportParams, Option)>>), String> { let inherent_data = self.inherent_data_providers - .create_inherent_data().map_err(String::from)?; - let timestamp_now = inherent_data.timestamp_inherent_data().map_err(String::from)?; + .create_inherent_data().map_err(|e| e.into_string())?; + let timestamp_now = inherent_data.timestamp_inherent_data().map_err(|e| e.into_string())?; let best_hash = match self.select_chain.as_ref() { Some(select_chain) => select_chain.best_chain() @@ -304,6 +305,7 @@ impl, C, S, Algorithm> Verifier for PowVerifier best_aux.total_difficulty), + allow_missing_state: false, }; Ok((import_block, None)) @@ -314,9 +316,9 @@ impl, C, S, Algorithm> Verifier for PowVerifier Result<(), consensus_common::Error> { - if !inherent_data_providers.has_provider(&srml_timestamp::INHERENT_IDENTIFIER) { + if !inherent_data_providers.has_provider(&paint_timestamp::INHERENT_IDENTIFIER) { inherent_data_providers - .register_provider(srml_timestamp::InherentDataProvider) + .register_provider(paint_timestamp::InherentDataProvider) .map_err(Into::into) .map_err(consensus_common::Error::InherentData) } else { @@ -339,7 +341,7 @@ pub fn import_queue( B: BlockT, C: ProvideRuntimeApi + HeaderBackend + BlockOf + ProvideCache + AuxStore, C: Send + Sync + AuxStore + 'static, - C::Api: BlockBuilderApi, + C::Api: BlockBuilderApi, Algorithm: PowAlgorithm + Send + Sync + 'static, S: SelectChain + 'static, { @@ -531,6 +533,7 @@ fn mine_loop, C, Algorithm, E, SO, S>( finalized: false, auxiliary: vec![(key, Some(aux.encode()))], fork_choice: ForkChoiceStrategy::Custom(true), + allow_missing_state: false, }; block_import.import_block(import_block, HashMap::default()) diff --git a/core/consensus/slots/Cargo.toml b/client/consensus/slots/Cargo.toml similarity index 59% rename from core/consensus/slots/Cargo.toml rename to client/consensus/slots/Cargo.toml index 4074242502b71..5e9af2f29a474 100644 --- a/core/consensus/slots/Cargo.toml +++ b/client/consensus/slots/Cargo.toml @@ -8,16 +8,16 @@ build = "build.rs" [dependencies] codec = { package = "parity-scale-codec", version = "1.0.0" } -client = { package = "substrate-client", path = "../../client" } -primitives = { package = "substrate-primitives", path = "../../primitives" } -sr-primitives = { path = "../../sr-primitives" } +client-api = { package = "substrate-client-api", path = "../../api" } +primitives = { package = "substrate-primitives", path = "../../../primitives/core" } +sr-primitives = { path = "../../../primitives/sr-primitives" } substrate-telemetry = { path = "../../telemetry" } -consensus_common = { package = "substrate-consensus-common", path = "../common" } -inherents = { package = "substrate-inherents", path = "../../inherents" } +consensus_common = { package = "substrate-consensus-common", path = "../../../primitives/consensus/common" } +inherents = { package = "substrate-inherents", path = "../../../primitives/inherents" } futures-preview = "0.3.0-alpha.19" futures-timer = "0.4.0" parking_lot = "0.9.0" log = "0.4.8" [dev-dependencies] -test-client = { package = "substrate-test-runtime-client", path = "../../test-runtime/client" } +test-client = { package = "substrate-test-runtime-client", path = "../../../test/utils/runtime/client" } diff --git a/core/consensus/slots/build.rs b/client/consensus/slots/build.rs similarity index 100% rename from core/consensus/slots/build.rs rename to client/consensus/slots/build.rs diff --git a/core/consensus/slots/src/aux_schema.rs b/client/consensus/slots/src/aux_schema.rs similarity index 98% rename from core/consensus/slots/src/aux_schema.rs rename to client/consensus/slots/src/aux_schema.rs index 186288c174432..1e777f35a17ed 100644 --- a/core/consensus/slots/src/aux_schema.rs +++ b/client/consensus/slots/src/aux_schema.rs @@ -17,8 +17,8 @@ //! Schema for slots in the aux-db. use codec::{Encode, Decode}; -use client::backend::AuxStore; -use client::error::{Result as ClientResult, Error as ClientError}; +use client_api::backend::AuxStore; +use client_api::error::{Result as ClientResult, Error as ClientError}; use sr_primitives::traits::Header; const SLOT_HEADER_MAP_KEY: &[u8] = b"slot_header_map"; diff --git a/core/consensus/slots/src/lib.rs b/client/consensus/slots/src/lib.rs similarity index 97% rename from core/consensus/slots/src/lib.rs rename to client/consensus/slots/src/lib.rs index e33d00d25531f..5897f88c77c29 100644 --- a/core/consensus/slots/src/lib.rs +++ b/client/consensus/slots/src/lib.rs @@ -41,6 +41,7 @@ use sr_primitives::traits::{ApiRef, Block as BlockT, Header, ProvideRuntimeApi}; use std::{fmt::Debug, ops::Deref, pin::Pin, sync::Arc}; use substrate_telemetry::{telemetry, CONSENSUS_DEBUG, CONSENSUS_WARN, CONSENSUS_INFO}; use parking_lot::Mutex; +use client_api; /// A worker that should be invoked at every new slot. pub trait SlotWorker { @@ -389,17 +390,17 @@ impl SlotDuration { /// /// `slot_key` is marked as `'static`, as it should really be a /// compile-time constant. - pub fn get_or_compute(client: &C, cb: CB) -> ::client::error::Result where - C: client::backend::AuxStore, + pub fn get_or_compute(client: &C, cb: CB) -> client_api::error::Result where + C: client_api::backend::AuxStore, C: ProvideRuntimeApi, - CB: FnOnce(ApiRef, &BlockId) -> ::client::error::Result, + CB: FnOnce(ApiRef, &BlockId) -> client_api::error::Result, T: SlotData + Encode + Decode + Debug, { match client.get_aux(T::SLOT_KEY)? { Some(v) => ::decode(&mut &v[..]) .map(SlotDuration) .map_err(|_| { - client::error::Error::Backend({ + client_api::error::Error::Backend({ error!(target: "slots", "slot duration kept in invalid format"); "slot duration kept in invalid format".to_string() }) @@ -410,7 +411,7 @@ impl SlotDuration { cb(client.runtime_api(), &BlockId::number(Zero::zero()))?; info!( - "Loaded block-time = {:?} seconds from genesis on first-launch", + "Loaded block-time = {:?} milliseconds from genesis on first-launch", genesis_slot_duration ); diff --git a/core/consensus/slots/src/slots.rs b/client/consensus/slots/src/slots.rs similarity index 99% rename from core/consensus/slots/src/slots.rs rename to client/consensus/slots/src/slots.rs index 98310bbf2e27c..0157bc70355d0 100644 --- a/core/consensus/slots/src/slots.rs +++ b/client/consensus/slots/src/slots.rs @@ -146,7 +146,7 @@ impl Stream for Slots { let inherent_data = match self.inherent_data_providers.create_inherent_data() { Ok(id) => id, - Err(err) => return Poll::Ready(Some(Err(consensus_common::Error::InherentData(err.into_owned())))), + Err(err) => return Poll::Ready(Some(Err(consensus_common::Error::InherentData(err)))), }; let result = self.timestamp_extractor.extract_timestamp_and_slot(&inherent_data); let (timestamp, slot_num, offset) = match result { diff --git a/client/consensus/uncles/Cargo.toml b/client/consensus/uncles/Cargo.toml new file mode 100644 index 0000000000000..1212449a55d60 --- /dev/null +++ b/client/consensus/uncles/Cargo.toml @@ -0,0 +1,15 @@ +[package] +name = "substrate-consensus-uncles" +version = "2.0.0" +authors = ["Parity Technologies "] +description = "Generic uncle inclusion utilities for consensus" +edition = "2018" + +[dependencies] +client-api = { package = "substrate-client-api", path = "../../api" } +primitives = { package = "substrate-primitives", path = "../../../primitives/core" } +sr-primitives = { path = "../../../primitives/sr-primitives" } +paint-authorship = { path = "../../../paint/authorship" } +consensus_common = { package = "substrate-consensus-common", path = "../../../primitives/consensus/common" } +inherents = { package = "substrate-inherents", path = "../../../primitives/inherents" } +log = "0.4.8" diff --git a/core/consensus/uncles/src/lib.rs b/client/consensus/uncles/src/lib.rs similarity index 91% rename from core/consensus/uncles/src/lib.rs rename to client/consensus/uncles/src/lib.rs index 5638a23175d56..63d4f584a31aa 100644 --- a/core/consensus/uncles/src/lib.rs +++ b/client/consensus/uncles/src/lib.rs @@ -22,7 +22,7 @@ use consensus_common::SelectChain; use inherents::{InherentDataProviders}; use log::warn; -use client::ProvideUncles; +use client_api::ProvideUncles; use sr_primitives::traits::{Block as BlockT, Header}; use std::sync::Arc; @@ -39,9 +39,9 @@ pub fn register_uncles_inherent_data_provider( C: ProvideUncles + Send + Sync + 'static, SC: SelectChain + 'static, { - if !inherent_data_providers.has_provider(&srml_authorship::INHERENT_IDENTIFIER) { + if !inherent_data_providers.has_provider(&paint_authorship::INHERENT_IDENTIFIER) { inherent_data_providers - .register_provider(srml_authorship::InherentDataProvider::new(move || { + .register_provider(paint_authorship::InherentDataProvider::new(move || { { let chain_head = match select_chain.best_chain() { Ok(x) => x, diff --git a/core/client/db/Cargo.toml b/client/db/Cargo.toml similarity index 67% rename from core/client/db/Cargo.toml rename to client/db/Cargo.toml index 7d88c39d7fd7e..f03fde576e21b 100644 --- a/core/client/db/Cargo.toml +++ b/client/db/Cargo.toml @@ -13,21 +13,23 @@ kvdb-rocksdb = { git = "https://github.com/paritytech/parity-common", rev="b0317 kvdb-memorydb = { git = "https://github.com/paritytech/parity-common", rev="b0317f649ab2c665b7987b8475878fc4d2e1f81d" } linked-hash-map = "0.5.2" hash-db = "0.15.2" -primitives = { package = "substrate-primitives", path = "../../primitives" } -sr-primitives = { path = "../../sr-primitives" } -client = { package = "substrate-client", path = "../../client" } -state-machine = { package = "substrate-state-machine", path = "../../state-machine" } +client-api = { package = "substrate-client-api", path = "../api" } +primitives = { package = "substrate-primitives", path = "../../primitives/core" } +sr-primitives = { path = "../../primitives/sr-primitives" } +client = { package = "substrate-client", path = "../" } +state-machine = { package = "substrate-state-machine", path = "../../primitives/state-machine" } codec = { package = "parity-scale-codec", version = "1.0.0", features = ["derive"] } -executor = { package = "substrate-executor", path = "../../executor" } -state_db = { package = "substrate-state-db", path = "../../state-db" } -trie = { package = "substrate-trie", path = "../../trie" } -consensus_common = { package = "substrate-consensus-common", path = "../../consensus/common" } +executor = { package = "substrate-executor", path = "../executor" } +state_db = { package = "substrate-state-db", path = "../state-db" } +trie = { package = "substrate-trie", path = "../../primitives/trie" } +consensus_common = { package = "substrate-consensus-common", path = "../../primitives/consensus/common" } header_metadata = { package = "substrate-header-metadata", path = "../header-metadata" } [dev-dependencies] -substrate-keyring = { path = "../../keyring" } -test-client = { package = "substrate-test-runtime-client", path = "../../test-runtime/client" } +substrate-keyring = { path = "../../primitives/keyring" } +test-client = { package = "substrate-test-runtime-client", path = "../../test/utils/runtime/client" } env_logger = "0.7.0" +quickcheck = "0.8" [features] default = [] diff --git a/core/client/db/src/cache/list_cache.rs b/client/db/src/cache/list_cache.rs similarity index 99% rename from core/client/db/src/cache/list_cache.rs rename to client/db/src/cache/list_cache.rs index 7a4fcc448e0e5..a51e6045c02fb 100644 --- a/core/client/db/src/cache/list_cache.rs +++ b/client/db/src/cache/list_cache.rs @@ -43,7 +43,7 @@ use std::collections::{BTreeSet, BTreeMap}; use log::warn; -use client::error::{Error as ClientError, Result as ClientResult}; +use client_api::error::{Error as ClientError, Result as ClientResult}; use sr_primitives::traits::{ Block as BlockT, NumberFor, Zero, Bounded, CheckedSub }; diff --git a/core/client/db/src/cache/list_entry.rs b/client/db/src/cache/list_entry.rs similarity index 99% rename from core/client/db/src/cache/list_entry.rs rename to client/db/src/cache/list_entry.rs index 6de369ecf56d2..afb2899362f91 100644 --- a/core/client/db/src/cache/list_entry.rs +++ b/client/db/src/cache/list_entry.rs @@ -16,7 +16,7 @@ //! List-cache storage entries. -use client::error::Result as ClientResult; +use client_api::error::Result as ClientResult; use sr_primitives::traits::{Block as BlockT, NumberFor}; use codec::{Encode, Decode}; diff --git a/core/client/db/src/cache/list_storage.rs b/client/db/src/cache/list_storage.rs similarity index 99% rename from core/client/db/src/cache/list_storage.rs rename to client/db/src/cache/list_storage.rs index 4fd9ecaced25e..d689619053d46 100644 --- a/core/client/db/src/cache/list_storage.rs +++ b/client/db/src/cache/list_storage.rs @@ -20,7 +20,7 @@ use std::sync::Arc; use kvdb::{KeyValueDB, DBTransaction}; -use client::error::{Error as ClientError, Result as ClientResult}; +use client_api::error::{Error as ClientError, Result as ClientResult}; use codec::{Encode, Decode}; use sr_primitives::generic::BlockId; use sr_primitives::traits::{Block as BlockT, Header as HeaderT, NumberFor}; diff --git a/core/client/db/src/cache/mod.rs b/client/db/src/cache/mod.rs similarity index 98% rename from core/client/db/src/cache/mod.rs rename to client/db/src/cache/mod.rs index 9aaea57b0cdb2..b3ff1a4ea6dbc 100644 --- a/core/client/db/src/cache/mod.rs +++ b/client/db/src/cache/mod.rs @@ -21,8 +21,8 @@ use parking_lot::RwLock; use kvdb::{KeyValueDB, DBTransaction}; -use client::blockchain::{well_known_cache_keys::{self, Id as CacheKeyId}, Cache as BlockchainCache}; -use client::error::Result as ClientResult; +use client_api::blockchain::{well_known_cache_keys::{self, Id as CacheKeyId}, Cache as BlockchainCache}; +use client_api::error::Result as ClientResult; use codec::{Encode, Decode}; use sr_primitives::generic::BlockId; use sr_primitives::traits::{Block as BlockT, Header as HeaderT, NumberFor, Zero}; @@ -211,7 +211,7 @@ impl<'a, Block: BlockT> DbCacheTransaction<'a, Block> { .cloned() .collect::>(); - let mut insert_op = |name: CacheKeyId, value: Option>| -> Result<(), client::error::Error> { + let mut insert_op = |name: CacheKeyId, value: Option>| -> Result<(), client_api::error::Error> { let cache = self.cache.get_cache(name); let op = cache.on_block_insert( &mut self::list_storage::DbStorageTransaction::new( diff --git a/core/client/src/children.rs b/client/db/src/children.rs similarity index 99% rename from core/client/src/children.rs rename to client/db/src/children.rs index 3128f860869ce..f6dd7299e43f5 100644 --- a/core/client/src/children.rs +++ b/client/db/src/children.rs @@ -18,7 +18,7 @@ use kvdb::{KeyValueDB, DBTransaction}; use codec::{Encode, Decode}; -use crate::error; +use client_api::error; use std::hash::Hash; diff --git a/core/client/db/src/lib.rs b/client/db/src/lib.rs similarity index 96% rename from core/client/db/src/lib.rs rename to client/db/src/lib.rs index 8bd0001981611..90d8aadccc1ae 100644 --- a/core/client/db/src/lib.rs +++ b/client/db/src/lib.rs @@ -29,6 +29,7 @@ pub mod light; pub mod offchain; +mod children; mod cache; mod storage_cache; mod utils; @@ -38,11 +39,11 @@ use std::path::PathBuf; use std::io; use std::collections::{HashMap, HashSet}; -use client::backend::NewBlockState; -use client::blockchain::{well_known_cache_keys, HeaderBackend}; -use client::{ForkBlocks, ExecutionStrategies}; -use client::backend::{StorageCollection, ChildStorageCollection}; -use client::error::{Result as ClientResult, Error as ClientError}; +use client_api::backend::NewBlockState; +use client_api::blockchain::{well_known_cache_keys, HeaderBackend}; +use client_api::{ForkBlocks, ExecutionStrategies}; +use client_api::backend::{StorageCollection, ChildStorageCollection}; +use client_api::error::{Result as ClientResult, Error as ClientError}; use codec::{Decode, Encode}; use hash_db::{Hasher, Prefix}; use kvdb::{KeyValueDB, DBTransaction}; @@ -64,7 +65,6 @@ use state_machine::{ }; use crate::utils::{Meta, db_err, meta_keys, read_db, read_meta}; use client::leaves::{LeafSet, FinalizationDisplaced}; -use client::children; use state_db::StateDb; use header_metadata::{CachedHeaderMetadata, HeaderMetadata, HeaderMetadataCache}; use crate::storage_cache::{CachingState, SharedCache, new_shared_cache}; @@ -457,6 +457,7 @@ pub struct BlockImportOperation { aux_ops: Vec<(Vec, Option>)>, finalized_blocks: Vec<(BlockId, Option)>, set_head: Option>, + commit_state: bool, } impl BlockImportOperation { @@ -470,7 +471,7 @@ impl BlockImportOperation { } } -impl client::backend::BlockImportOperation +impl client_api::backend::BlockImportOperation for BlockImportOperation where Block: BlockT, { type State = CachingState, Block>; @@ -531,6 +532,7 @@ impl client::backend::BlockImportOperation ); self.db_updates = transaction; + self.commit_state = true; Ok(root) } @@ -659,7 +661,7 @@ impl> DbChangesTrieStorage { } } -impl client::backend::PrunableStateChangesTrieStorage +impl client_api::backend::PrunableStateChangesTrieStorage for DbChangesTrieStorage where Block: BlockT, @@ -783,6 +785,7 @@ pub struct Backend { canonicalization_delay: u64, shared_cache: SharedCache, import_lock: Mutex<()>, + is_archive: bool, } impl> Backend { @@ -843,13 +846,14 @@ impl> Backend { config.state_cache_child_ratio.unwrap_or(DEFAULT_CHILD_RATIO), ), import_lock: Default::default(), + is_archive: is_archive_pruning, }) } /// Returns in-memory blockchain that contains the same set of blocks that the self. #[cfg(feature = "test-helpers")] pub fn as_in_memory(&self) -> InMemoryBackend { - use client::backend::{Backend as ClientBackend, BlockImportOperation}; + use client_api::backend::{Backend as ClientBackend, BlockImportOperation}; use client::blockchain::Backend as BlockchainBackend; let inmem = InMemoryBackend::::new(); @@ -894,6 +898,12 @@ impl> Backend { inmem } + /// Returns total numbet of blocks (headers) in the block DB. + #[cfg(feature = "test-helpers")] + pub fn blocks_count(&self) -> u64 { + self.blockchain.db.iter(columns::HEADER).count() as u64 + } + /// Read (from storage or cache) changes trie config. /// /// Currently changes tries configuration is set up once (at genesis) and could not @@ -903,7 +913,7 @@ impl> Backend { match cached_changes_trie_config.clone() { Some(cached_changes_trie_config) => Ok(cached_changes_trie_config), None => { - use client::backend::Backend; + use client_api::backend::Backend; let changes_trie_config = self .state_at(BlockId::Hash(block))? .storage(well_known_keys::CHANGES_TRIE_CONFIG)? @@ -1115,7 +1125,7 @@ impl> Backend { ); transaction.put(columns::HEADER, &lookup_key, &pending_block.header.encode()); - if let Some(body) = pending_block.body { + if let Some(body) = &pending_block.body { transaction.put(columns::BODY, &lookup_key, &body.encode()); } if let Some(justification) = pending_block.justification { @@ -1127,21 +1137,26 @@ impl> Backend { transaction.put(columns::META, meta_keys::GENESIS_HASH, hash.as_ref()); } - let mut changeset: state_db::ChangeSet> = state_db::ChangeSet::default(); - for (key, (val, rc)) in operation.db_updates.drain() { - if rc > 0 { - changeset.inserted.push((key, val.to_vec())); - } else if rc < 0 { - changeset.deleted.push(key); + let finalized = if operation.commit_state { + let mut changeset: state_db::ChangeSet> = state_db::ChangeSet::default(); + for (key, (val, rc)) in operation.db_updates.drain() { + if rc > 0 { + changeset.inserted.push((key, val.to_vec())); + } else if rc < 0 { + changeset.deleted.push(key); + } } - } - let number_u64 = number.saturated_into::(); - let commit = self.storage.state_db.insert_block(&hash, number_u64, &pending_block.header.parent_hash(), changeset) - .map_err(|e: state_db::Error| client::error::Error::from(format!("State database error: {:?}", e)))?; - apply_state_commit(&mut transaction, commit); - - // Check if need to finalize. Genesis is always finalized instantly. - let finalized = number_u64 == 0 || pending_block.leaf_state.is_final(); + let number_u64 = number.saturated_into::(); + let commit = self.storage.state_db.insert_block(&hash, number_u64, &pending_block.header.parent_hash(), changeset) + .map_err(|e: state_db::Error| client::error::Error::from(format!("State database error: {:?}", e)))?; + apply_state_commit(&mut transaction, commit); + + // Check if need to finalize. Genesis is always finalized instantly. + let finalized = number_u64 == 0 || pending_block.leaf_state.is_final(); + finalized + } else { + false + }; let header = &pending_block.header; let is_best = pending_block.leaf_state.is_best(); @@ -1303,7 +1318,7 @@ fn apply_state_commit(transaction: &mut DBTransaction, commit: state_db::CommitS } } -impl client::backend::AuxStore for Backend where Block: BlockT { +impl client_api::backend::AuxStore for Backend where Block: BlockT { fn insert_aux< 'a, 'b: 'a, @@ -1327,7 +1342,7 @@ impl client::backend::AuxStore for Backend where Block: BlockT client::backend::Backend for Backend where Block: BlockT { +impl client_api::backend::Backend for Backend where Block: BlockT { type BlockImportOperation = BlockImportOperation; type Blockchain = BlockchainDb; type State = CachingState, Block>; @@ -1347,6 +1362,7 @@ impl client::backend::Backend for Backend whe aux_ops: Vec::new(), finalized_blocks: Vec::new(), set_head: None, + commit_state: false, }) } @@ -1356,6 +1372,7 @@ impl client::backend::Backend for Backend whe block: BlockId, ) -> ClientResult<()> { operation.old_state = self.state_at(block)?; + operation.commit_state = true; Ok(()) } @@ -1478,6 +1495,9 @@ impl client::backend::Backend for Backend whe match self.blockchain.header(block) { Ok(Some(ref hdr)) => { let hash = hdr.hash(); + if !self.have_state_at(&hash, *hdr.number()) { + return Err(client::error::Error::UnknownBlock(format!("State already discarded for {:?}", block))) + } if let Ok(()) = self.storage.state_db.pin(&hash) { let root = H256::from_slice(hdr.state_root().as_ref()); let db_state = DbState::new(self.storage.clone(), root); @@ -1493,7 +1513,16 @@ impl client::backend::Backend for Backend whe } fn have_state_at(&self, hash: &Block::Hash, number: NumberFor) -> bool { - !self.storage.state_db.is_pruned(hash, number.saturated_into::()) + if self.is_archive { + match self.blockchain.header(BlockId::Hash(hash.clone())) { + Ok(Some(header)) => { + state_machine::Storage::get(self.storage.as_ref(), &header.state_root(), (&[], None)).unwrap_or(None).is_some() + }, + _ => false, + } + } else { + !self.storage.state_db.is_pruned(hash, number.saturated_into::()) + } } fn destroy_state(&self, state: Self::State) -> ClientResult<()> { @@ -1509,7 +1538,7 @@ impl client::backend::Backend for Backend whe } } -impl client::backend::LocalBackend for Backend +impl client_api::backend::LocalBackend for Backend where Block: BlockT {} /// TODO: remove me in #3201 @@ -1523,9 +1552,8 @@ mod tests { use hash_db::{HashDB, EMPTY_PREFIX}; use super::*; use crate::columns; - use client::backend::Backend as BTrait; + use client_api::backend::{Backend as BTrait, BlockImportOperation as Op}; use client::blockchain::Backend as BLBTrait; - use client::backend::BlockImportOperation as Op; use sr_primitives::testing::{Header, Block as RawBlock, ExtrinsicWrapper}; use sr_primitives::traits::{Hash, BlakeTwo256}; use state_machine::{TrieMut, TrieDBMut, ChangesTrieRootsStorage, ChangesTrieStorage}; @@ -1581,7 +1609,7 @@ mod tests { }; let mut op = backend.begin_operation().unwrap(); backend.begin_state_operation(&mut op, block_id).unwrap(); - op.set_block_data(header, None, None, NewBlockState::Best).unwrap(); + op.set_block_data(header, Some(Vec::new()), None, NewBlockState::Best).unwrap(); op.update_changes_trie((changes_trie_update, ChangesTrieCacheAction::Clear)).unwrap(); backend.commit_operation(op).unwrap(); diff --git a/core/client/db/src/light.rs b/client/db/src/light.rs similarity index 99% rename from core/client/db/src/light.rs rename to client/db/src/light.rs index 6e71b88ae73f9..bdce0e638428e 100644 --- a/core/client/db/src/light.rs +++ b/client/db/src/light.rs @@ -22,7 +22,7 @@ use parking_lot::RwLock; use kvdb::{KeyValueDB, DBTransaction}; -use client::backend::{AuxStore, NewBlockState}; +use client_api::backend::{AuxStore, NewBlockState}; use client::blockchain::{ BlockStatus, Cache as BlockchainCache, HeaderBackend as BlockchainHeaderBackend, Info as BlockchainInfo, diff --git a/core/client/db/src/offchain.rs b/client/db/src/offchain.rs similarity index 97% rename from core/client/db/src/offchain.rs rename to client/db/src/offchain.rs index 0640fb6c29bd6..09578f5f6b1d0 100644 --- a/core/client/db/src/offchain.rs +++ b/client/db/src/offchain.rs @@ -56,7 +56,7 @@ impl LocalStorage { } } -impl client::backend::OffchainStorage for LocalStorage { +impl client_api::OffchainStorage for LocalStorage { fn set(&mut self, prefix: &[u8], key: &[u8], value: &[u8]) { let key: Vec = prefix.iter().chain(key).cloned().collect(); let mut tx = self.db.transaction(); @@ -117,7 +117,7 @@ impl client::backend::OffchainStorage for LocalStorage { #[cfg(test)] mod tests { use super::*; - use client::backend::OffchainStorage; + use client_api::OffchainStorage; #[test] fn should_compare_and_set_and_clear_the_locks_map() { diff --git a/client/db/src/storage_cache.rs b/client/db/src/storage_cache.rs new file mode 100644 index 0000000000000..9888dfbe647a5 --- /dev/null +++ b/client/db/src/storage_cache.rs @@ -0,0 +1,1380 @@ +// Copyright 2019 Parity Technologies (UK) Ltd. +// This file is part of Substrate. + +// Substrate is free software: you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. + +// Substrate is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. + +// You should have received a copy of the GNU General Public License +// along with Substrate. If not, see . + +//! Global cache state. + +use std::collections::{VecDeque, HashSet, HashMap}; +use std::sync::Arc; +use parking_lot::{Mutex, RwLock, RwLockUpgradableReadGuard}; +use linked_hash_map::{LinkedHashMap, Entry}; +use hash_db::Hasher; +use sr_primitives::traits::{Block as BlockT, Header}; +use primitives::hexdisplay::HexDisplay; +use state_machine::{backend::Backend as StateBackend, TrieBackend}; +use log::trace; +use client_api::backend::{StorageCollection, ChildStorageCollection}; +use std::hash::Hash as StdHash; + +const STATE_CACHE_BLOCKS: usize = 12; + +type StorageKey = Vec; +type ChildStorageKey = (Vec, Vec); +type StorageValue = Vec; + +/// Shared canonical state cache. +pub struct Cache { + /// Storage cache. `None` indicates that key is known to be missing. + lru_storage: LRUMap>, + /// Storage hashes cache. `None` indicates that key is known to be missing. + lru_hashes: LRUMap>, + /// Storage cache for child trie. `None` indicates that key is known to be missing. + lru_child_storage: LRUMap>, + /// Information on the modifications in recently committed blocks; specifically which keys + /// changed in which block. Ordered by block number. + modifications: VecDeque>, +} + +struct LRUMap(LinkedHashMap, usize, usize); + +/// Internal trait similar to `heapsize` but using +/// a simply estimation. +/// +/// This should not be made public, it is implementation +/// detail trait. If it need to become public please +/// consider using `malloc_size_of`. +trait EstimateSize { + /// Return a size estimation of additional size needed + /// to cache this struct (in bytes). + fn estimate_size(&self) -> usize; +} + +impl EstimateSize for Vec { + fn estimate_size(&self) -> usize { + self.capacity() + } +} + +impl EstimateSize for Option> { + fn estimate_size(&self) -> usize { + self.as_ref().map(|v|v.capacity()).unwrap_or(0) + } +} + +struct OptionHOut>(Option); + +impl> EstimateSize for OptionHOut { + fn estimate_size(&self) -> usize { + // capacity would be better + self.0.as_ref().map(|v|v.as_ref().len()).unwrap_or(0) + } +} + +impl EstimateSize for (T, T) { + fn estimate_size(&self) -> usize { + self.0.estimate_size() + self.1.estimate_size() + } +} + +impl LRUMap { + fn remove(&mut self, k: &K) { + let map = &mut self.0; + let storage_used_size = &mut self.1; + if let Some(v) = map.remove(k) { + *storage_used_size -= k.estimate_size(); + *storage_used_size -= v.estimate_size(); + } + } + + fn add(&mut self, k: K, v: V) { + let lmap = &mut self.0; + let storage_used_size = &mut self.1; + let limit = self.2; + let klen = k.estimate_size(); + *storage_used_size += v.estimate_size(); + // TODO assert k v size fit into limit?? to avoid insert remove? + match lmap.entry(k) { + Entry::Occupied(mut entry) => { + // note that in this case we are not running pure lru as + // it would require to remove first + *storage_used_size -= entry.get().estimate_size(); + entry.insert(v); + }, + Entry::Vacant(entry) => { + *storage_used_size += klen; + entry.insert(v); + }, + }; + + while *storage_used_size > limit { + if let Some((k,v)) = lmap.pop_front() { + *storage_used_size -= k.estimate_size(); + *storage_used_size -= v.estimate_size(); + } else { + // can happen fairly often as we get value from multiple lru + // and only remove from a single lru + break; + } + } + } + + fn get(&mut self, k: &Q) -> Option<&mut V> + where K: std::borrow::Borrow, + Q: StdHash + Eq { + self.0.get_refresh(k) + } + + fn used_size(&self) -> usize { + self.1 + } + fn clear(&mut self) { + self.0.clear(); + self.1 = 0; + } + +} + +impl Cache { + /// Returns the used memory size of the storage cache in bytes. + pub fn used_storage_cache_size(&self) -> usize { + self.lru_storage.used_size() + + self.lru_child_storage.used_size() + // ignore small hashes storage and self.lru_hashes.used_size() + } + + /// Synchronize the shared cache with the best block state. + /// + /// This function updates the shared cache by removing entries + /// that are invalidated by chain reorganization. It should be called + /// externally when chain reorg happens without importing a new block. + pub fn sync(&mut self, enacted: &[B::Hash], retracted: &[B::Hash]) { + trace!("Syncing shared cache, enacted = {:?}, retracted = {:?}", enacted, retracted); + + // Purge changes from re-enacted and retracted blocks. + let mut clear = false; + for block in enacted { + clear = clear || { + if let Some(m) = self.modifications.iter_mut().find(|m| &m.hash == block) { + trace!("Reverting enacted block {:?}", block); + m.is_canon = true; + for a in &m.storage { + trace!("Reverting enacted key {:?}", HexDisplay::from(a)); + self.lru_storage.remove(a); + } + for a in &m.child_storage { + trace!("Reverting enacted child key {:?}", a); + self.lru_child_storage.remove(a); + } + false + } else { + true + } + }; + } + + for block in retracted { + clear = clear || { + if let Some(m) = self.modifications.iter_mut().find(|m| &m.hash == block) { + trace!("Retracting block {:?}", block); + m.is_canon = false; + for a in &m.storage { + trace!("Retracted key {:?}", HexDisplay::from(a)); + self.lru_storage.remove(a); + } + for a in &m.child_storage { + trace!("Retracted child key {:?}", a); + self.lru_child_storage.remove(a); + } + false + } else { + true + } + }; + } + if clear { + // We don't know anything about the block; clear everything + trace!("Wiping cache"); + self.lru_storage.clear(); + self.lru_child_storage.clear(); + self.lru_hashes.clear(); + self.modifications.clear(); + } + } +} + +pub type SharedCache = Arc>>; + +/// Fix lru storage size for hash (small 64ko). +const FIX_LRU_HASH_SIZE: usize = 65_536; + +/// Create a new shared cache instance with given max memory usage. +pub fn new_shared_cache( + shared_cache_size: usize, + child_ratio: (usize, usize), +) -> SharedCache { + let top = child_ratio.1.saturating_sub(child_ratio.0); + Arc::new(Mutex::new(Cache { + lru_storage: LRUMap(LinkedHashMap::new(), 0, + shared_cache_size * top / child_ratio.1), + lru_hashes: LRUMap(LinkedHashMap::new(), 0, FIX_LRU_HASH_SIZE), + lru_child_storage: LRUMap(LinkedHashMap::new(), 0, + shared_cache_size * child_ratio.0 / child_ratio.1), + modifications: VecDeque::new(), + })) +} + +#[derive(Debug)] +/// Accumulates a list of storage changed in a block. +struct BlockChanges { + /// Block number. + number: B::Number, + /// Block hash. + hash: B::Hash, + /// Parent block hash. + parent: B::Hash, + /// A set of modified storage keys. + storage: HashSet, + /// A set of modified child storage keys. + child_storage: HashSet, + /// Block is part of the canonical chain. + is_canon: bool, +} + +/// Cached values specific to a state. +struct LocalCache { + /// Storage cache. + /// + /// `None` indicates that key is known to be missing. + storage: HashMap>, + /// Storage hashes cache. + /// + /// `None` indicates that key is known to be missing. + hashes: HashMap>, + /// Child storage cache. + /// + /// `None` indicates that key is known to be missing. + child_storage: HashMap>, +} + +/// Cache changes. +pub struct CacheChanges { + /// Shared canonical state cache. + shared_cache: SharedCache, + /// Local cache of values for this state. + local_cache: RwLock>, + /// Hash of the block on top of which this instance was created or + /// `None` if cache is disabled + pub parent_hash: Option, +} + +/// State cache abstraction. +/// +/// Manages shared global state cache which reflects the canonical +/// state as it is on the disk. +/// +/// A instance of `CachingState` may be created as canonical or not. +/// For canonical instances local cache is accumulated and applied +/// in `sync_cache` along with the change overlay. +/// For non-canonical clones local cache and changes are dropped. +pub struct CachingState, B: BlockT> { + /// Backing state. + state: S, + /// Cache data. + pub cache: CacheChanges +} + +impl, B: BlockT> std::fmt::Debug for CachingState { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + write!(f, "Block {:?}", self.cache.parent_hash) + } +} + +impl CacheChanges { + /// Propagate local cache into the shared cache and synchronize + /// the shared cache with the best block state. + /// + /// This function updates the shared cache by removing entries + /// that are invalidated by chain reorganization. `sync_cache` + /// should be called after the block has been committed and the + /// blockchain route has been calculated. + pub fn sync_cache bool> ( + &mut self, + enacted: &[B::Hash], + retracted: &[B::Hash], + changes: StorageCollection, + child_changes: ChildStorageCollection, + commit_hash: Option, + commit_number: Option<::Number>, + is_best: F, + ) { + let mut cache = self.shared_cache.lock(); + let is_best = is_best(); + trace!("Syncing cache, id = (#{:?}, {:?}), parent={:?}, best={}", commit_number, commit_hash, self.parent_hash, is_best); + let cache = &mut *cache; + // Filter out commiting block if any. + let enacted: Vec<_> = enacted + .iter() + .filter(|h| commit_hash.as_ref().map_or(true, |p| *h != p)) + .cloned() + .collect(); + cache.sync(&enacted, retracted); + // Propagate cache only if committing on top of the latest canonical state + // blocks are ordered by number and only one block with a given number is marked as canonical + // (contributed to canonical state cache) + if let Some(_) = self.parent_hash { + let mut local_cache = self.local_cache.write(); + if is_best { + trace!( + "Committing {} local, {} hashes, {} modified root entries, {} modified child entries", + local_cache.storage.len(), + local_cache.hashes.len(), + changes.len(), + child_changes.iter().map(|v|v.1.len()).sum::(), + ); + for (k, v) in local_cache.storage.drain() { + cache.lru_storage.add(k, v); + } + for (k, v) in local_cache.child_storage.drain() { + cache.lru_child_storage.add(k, v); + } + for (k, v) in local_cache.hashes.drain() { + cache.lru_hashes.add(k, OptionHOut(v)); + } + } + } + + if let ( + Some(ref number), Some(ref hash), Some(ref parent)) + = (commit_number, commit_hash, self.parent_hash) + { + if cache.modifications.len() == STATE_CACHE_BLOCKS { + cache.modifications.pop_back(); + } + let mut modifications = HashSet::new(); + let mut child_modifications = HashSet::new(); + child_changes.into_iter().for_each(|(sk, changes)| + for (k, v) in changes.into_iter() { + let k = (sk.clone(), k); + if is_best { + cache.lru_child_storage.add(k.clone(), v); + } + child_modifications.insert(k); + } + ); + for (k, v) in changes.into_iter() { + if is_best { + cache.lru_hashes.remove(&k); + cache.lru_storage.add(k.clone(), v); + } + modifications.insert(k); + } + + // Save modified storage. These are ordered by the block number in reverse. + let block_changes = BlockChanges { + storage: modifications, + child_storage: child_modifications, + number: *number, + hash: hash.clone(), + is_canon: is_best, + parent: parent.clone(), + }; + let insert_at = cache.modifications.iter() + .enumerate() + .find(|(_, m)| m.number < *number) + .map(|(i, _)| i); + trace!("Inserting modifications at {:?}", insert_at); + if let Some(insert_at) = insert_at { + cache.modifications.insert(insert_at, block_changes); + } else { + cache.modifications.push_back(block_changes); + } + } + } + +} + +impl, B: BlockT> CachingState { + /// Create a new instance wrapping generic State and shared cache. + pub fn new(state: S, shared_cache: SharedCache, parent_hash: Option) -> CachingState { + CachingState { + state, + cache: CacheChanges { + shared_cache, + local_cache: RwLock::new(LocalCache { + storage: Default::default(), + hashes: Default::default(), + child_storage: Default::default(), + }), + parent_hash: parent_hash, + }, + } + } + + /// Check if the key can be returned from cache by matching current block parent hash against canonical + /// state and filtering out entries modified in later blocks. + fn is_allowed( + key: Option<&[u8]>, + child_key: Option<&ChildStorageKey>, + parent_hash: &Option, + modifications: &VecDeque> + ) -> bool + { + let mut parent = match *parent_hash { + None => { + trace!("Cache lookup skipped for {:?}: no parent hash", key.as_ref().map(HexDisplay::from)); + return false; + } + Some(ref parent) => parent, + }; + // Ignore all storage entries modified in later blocks. + // Modifications contains block ordered by the number + // We search for our parent in that list first and then for + // all its parents until we hit the canonical block, + // checking against all the intermediate modifications. + for m in modifications { + if &m.hash == parent { + if m.is_canon { + return true; + } + parent = &m.parent; + } + if let Some(key) = key { + if m.storage.contains(key) { + trace!("Cache lookup skipped for {:?}: modified in a later block", HexDisplay::from(&key)); + return false; + } + } + if let Some(child_key) = child_key { + if m.child_storage.contains(child_key) { + trace!("Cache lookup skipped for {:?}: modified in a later block", child_key); + return false; + } + } + } + trace!("Cache lookup skipped for {:?}: parent hash is unknown", key.as_ref().map(HexDisplay::from)); + false + } + + /// Dispose state and return cache data. + pub fn release(self) -> CacheChanges { + self.cache + } +} + +impl, B: BlockT> StateBackend for CachingState { + type Error = S::Error; + type Transaction = S::Transaction; + type TrieBackendStorage = S::TrieBackendStorage; + + fn storage(&self, key: &[u8]) -> Result>, Self::Error> { + let local_cache = self.cache.local_cache.upgradable_read(); + // Note that local cache makes that lru is not refreshed + if let Some(entry) = local_cache.storage.get(key).cloned() { + trace!("Found in local cache: {:?}", HexDisplay::from(&key)); + return Ok(entry) + } + let mut cache = self.cache.shared_cache.lock(); + if Self::is_allowed(Some(key), None, &self.cache.parent_hash, &cache.modifications) { + if let Some(entry) = cache.lru_storage.get(key).map(|a| a.clone()) { + trace!("Found in shared cache: {:?}", HexDisplay::from(&key)); + return Ok(entry) + } + } + trace!("Cache miss: {:?}", HexDisplay::from(&key)); + let value = self.state.storage(key)?; + RwLockUpgradableReadGuard::upgrade(local_cache).storage.insert(key.to_vec(), value.clone()); + Ok(value) + } + + fn storage_hash(&self, key: &[u8]) -> Result, Self::Error> { + let local_cache = self.cache.local_cache.upgradable_read(); + if let Some(entry) = local_cache.hashes.get(key).cloned() { + trace!("Found hash in local cache: {:?}", HexDisplay::from(&key)); + return Ok(entry) + } + let mut cache = self.cache.shared_cache.lock(); + if Self::is_allowed(Some(key), None, &self.cache.parent_hash, &cache.modifications) { + if let Some(entry) = cache.lru_hashes.get(key).map(|a| a.0.clone()) { + trace!("Found hash in shared cache: {:?}", HexDisplay::from(&key)); + return Ok(entry) + } + } + trace!("Cache hash miss: {:?}", HexDisplay::from(&key)); + let hash = self.state.storage_hash(key)?; + RwLockUpgradableReadGuard::upgrade(local_cache).hashes.insert(key.to_vec(), hash); + Ok(hash) + } + + fn child_storage(&self, storage_key: &[u8], key: &[u8]) -> Result>, Self::Error> { + let key = (storage_key.to_vec(), key.to_vec()); + let local_cache = self.cache.local_cache.upgradable_read(); + if let Some(entry) = local_cache.child_storage.get(&key).cloned() { + trace!("Found in local cache: {:?}", key); + return Ok(entry) + } + let mut cache = self.cache.shared_cache.lock(); + if Self::is_allowed(None, Some(&key), &self.cache.parent_hash, &cache.modifications) { + if let Some(entry) = cache.lru_child_storage.get(&key).map(|a| a.clone()) { + trace!("Found in shared cache: {:?}", key); + return Ok(entry) + } + } + trace!("Cache miss: {:?}", key); + let value = self.state.child_storage(storage_key, &key.1[..])?; + RwLockUpgradableReadGuard::upgrade(local_cache).child_storage.insert(key, value.clone()); + Ok(value) + } + + fn exists_storage(&self, key: &[u8]) -> Result { + Ok(self.storage(key)?.is_some()) + } + + fn exists_child_storage(&self, storage_key: &[u8], key: &[u8]) -> Result { + self.state.exists_child_storage(storage_key, key) + } + + fn for_keys_in_child_storage(&self, storage_key: &[u8], f: F) { + self.state.for_keys_in_child_storage(storage_key, f) + } + + fn for_keys_with_prefix(&self, prefix: &[u8], f: F) { + self.state.for_keys_with_prefix(prefix, f) + } + + fn for_key_values_with_prefix(&self, prefix: &[u8], f: F) { + self.state.for_key_values_with_prefix(prefix, f) + } + + fn for_child_keys_with_prefix(&self, storage_key: &[u8], prefix: &[u8], f: F) { + self.state.for_child_keys_with_prefix(storage_key, prefix, f) + } + + fn storage_root(&self, delta: I) -> (H::Out, Self::Transaction) + where + I: IntoIterator, Option>)>, + H::Out: Ord + { + self.state.storage_root(delta) + } + + fn child_storage_root(&self, storage_key: &[u8], delta: I) -> (Vec, bool, Self::Transaction) + where + I: IntoIterator, Option>)>, + H::Out: Ord + { + self.state.child_storage_root(storage_key, delta) + } + + fn pairs(&self) -> Vec<(Vec, Vec)> { + self.state.pairs() + } + + fn keys(&self, prefix: &[u8]) -> Vec> { + self.state.keys(prefix) + } + + fn child_keys(&self, child_key: &[u8], prefix: &[u8]) -> Vec> { + self.state.child_keys(child_key, prefix) + } + + fn as_trie_backend(&mut self) -> Option<&TrieBackend> { + self.state.as_trie_backend() + } +} + +#[cfg(test)] +mod tests { + use super::*; + use sr_primitives::testing::{H256, Block as RawBlock, ExtrinsicWrapper}; + use state_machine::backend::InMemory; + use primitives::Blake2Hasher; + + type Block = RawBlock>; + + #[test] + fn smoke() { + //init_log(); + let root_parent = H256::random(); + let key = H256::random()[..].to_vec(); + let h0 = H256::random(); + let h1a = H256::random(); + let h1b = H256::random(); + let h2a = H256::random(); + let h2b = H256::random(); + let h3a = H256::random(); + let h3b = H256::random(); + + let shared = new_shared_cache::(256*1024, (0,1)); + + // blocks [ 3a(c) 2a(c) 2b 1b 1a(c) 0 ] + // state [ 5 5 4 3 2 2 ] + let mut s = CachingState::new(InMemory::::default(), shared.clone(), Some(root_parent)); + s.cache.sync_cache(&[], &[], vec![(key.clone(), Some(vec![2]))], vec![], Some(h0), Some(0), || true); + + let mut s = CachingState::new(InMemory::::default(), shared.clone(), Some(h0)); + s.cache.sync_cache(&[], &[], vec![], vec![], Some(h1a), Some(1), || true); + + let mut s = CachingState::new(InMemory::::default(), shared.clone(), Some(h0)); + s.cache.sync_cache(&[], &[], vec![(key.clone(), Some(vec![3]))], vec![], Some(h1b), Some(1), || false); + + let mut s = CachingState::new(InMemory::::default(), shared.clone(), Some(h1b.clone())); + s.cache.sync_cache(&[], &[], vec![(key.clone(), Some(vec![4]))], vec![], Some(h2b), Some(2), || false); + + let mut s = CachingState::new(InMemory::::default(), shared.clone(), Some(h1a.clone())); + s.cache.sync_cache(&[], &[], vec![(key.clone(), Some(vec![5]))], vec![], Some(h2a), Some(2), || true); + + let mut s = CachingState::new(InMemory::::default(), shared.clone(), Some(h2a)); + s.cache.sync_cache(&[], &[], vec![], vec![], Some(h3a), Some(3), || true); + + let s = CachingState::new(InMemory::::default(), shared.clone(), Some(h3a)); + assert_eq!(s.storage(&key).unwrap().unwrap(), vec![5]); + + let s = CachingState::new(InMemory::::default(), shared.clone(), Some(h1a)); + assert!(s.storage(&key).unwrap().is_none()); + + let s = CachingState::new(InMemory::::default(), shared.clone(), Some(h2b)); + assert!(s.storage(&key).unwrap().is_none()); + + let s = CachingState::new(InMemory::::default(), shared.clone(), Some(h1b)); + assert!(s.storage(&key).unwrap().is_none()); + + // reorg to 3b + // blocks [ 3b(c) 3a 2a 2b(c) 1b 1a 0 ] + let mut s = CachingState::new(InMemory::::default(), shared.clone(), Some(h2b)); + s.cache.sync_cache( + &[h1b, h2b, h3b], + &[h1a, h2a, h3a], + vec![], + vec![], + Some(h3b), + Some(3), + || true, + ); + let s = CachingState::new(InMemory::::default(), shared.clone(), Some(h3a)); + assert!(s.storage(&key).unwrap().is_none()); + } + + #[test] + fn simple_fork() { + let _ = ::env_logger::try_init(); + + let root_parent = H256::random(); + let key = H256::random()[..].to_vec(); + let h1 = H256::random(); + let h2a = H256::random(); + let h2b = H256::random(); + let h3b = H256::random(); + + let shared = new_shared_cache::(256*1024, (0,1)); + + let mut s = CachingState::new(InMemory::::default(), shared.clone(), Some(root_parent)); + s.cache.sync_cache(&[], &[], vec![(key.clone(), Some(vec![2]))], vec![], Some(h1), Some(1), || true); + + let mut s = CachingState::new(InMemory::::default(), shared.clone(), Some(h1)); + s.cache.sync_cache(&[], &[], vec![], vec![], Some(h2a), Some(2), || true); + + let mut s = CachingState::new(InMemory::::default(), shared.clone(), Some(h1)); + s.cache.sync_cache(&[], &[], vec![(key.clone(), Some(vec![3]))], vec![], Some(h2b), Some(2), || false); + + let mut s = CachingState::new(InMemory::::default(), shared.clone(), Some(h2b)); + s.cache.sync_cache(&[], &[], vec![(key.clone(), Some(vec![3]))], vec![], Some(h3b), Some(2), || false); + + let s = CachingState::new(InMemory::::default(), shared.clone(), Some(h2a)); + assert_eq!(s.storage(&key).unwrap().unwrap(), vec![2]); + } + + #[test] + fn double_fork() { + let root_parent = H256::random(); + let key = H256::random()[..].to_vec(); + let h1 = H256::random(); + let h2a = H256::random(); + let h2b = H256::random(); + let h3a = H256::random(); + let h3b = H256::random(); + + let shared = new_shared_cache::(256*1024, (0,1)); + + let mut s = CachingState::new(InMemory::::default(), shared.clone(), Some(root_parent)); + s.cache.sync_cache(&[], &[], vec![], vec![], Some(h1), Some(1), || true); + + let mut s = CachingState::new(InMemory::::default(), shared.clone(), Some(h1)); + s.cache.sync_cache(&[], &[], vec![], vec![], Some(h2a), Some(2), || true); + + let mut s = CachingState::new(InMemory::::default(), shared.clone(), Some(h2a)); + s.cache.sync_cache(&[], &[], vec![(key.clone(), Some(vec![2]))], vec![], Some(h3a), Some(3), || true); + + let mut s = CachingState::new(InMemory::::default(), shared.clone(), Some(h1)); + s.cache.sync_cache(&[], &[], vec![], vec![], Some(h2b), Some(2), || false); + + let mut s = CachingState::new(InMemory::::default(), shared.clone(), Some(h2b)); + s.cache.sync_cache(&[], &[], vec![(key.clone(), Some(vec![3]))], vec![], Some(h3b), Some(3), || false); + + let s = CachingState::new(InMemory::::default(), shared.clone(), Some(h3a)); + assert_eq!(s.storage(&key).unwrap().unwrap(), vec![2]); + } + + #[test] + fn should_track_used_size_correctly() { + let root_parent = H256::random(); + let shared = new_shared_cache::(109, ((109-36), 109)); + let h0 = H256::random(); + + let mut s = CachingState::new(InMemory::::default(), shared.clone(), Some(root_parent)); + + let key = H256::random()[..].to_vec(); + let s_key = H256::random()[..].to_vec(); + s.cache.sync_cache( + &[], + &[], + vec![(key.clone(), Some(vec![1, 2, 3]))], + vec![], + Some(h0), + Some(0), + || true, + ); + // 32 key, 3 byte size + assert_eq!(shared.lock().used_storage_cache_size(), 35 /* bytes */); + + let key = H256::random()[..].to_vec(); + s.cache.sync_cache( + &[], + &[], + vec![], + vec![(s_key.clone(), vec![(key.clone(), Some(vec![1, 2]))])], + Some(h0), + Some(0), + || true, + ); + // 35 + (2 * 32) key, 2 byte size + assert_eq!(shared.lock().used_storage_cache_size(), 101 /* bytes */); + } + + #[test] + fn should_remove_lru_items_based_on_tracking_used_size() { + let root_parent = H256::random(); + let shared = new_shared_cache::(36*3, (2,3)); + let h0 = H256::random(); + + let mut s = CachingState::new(InMemory::::default(), shared.clone(), Some(root_parent)); + + let key = H256::random()[..].to_vec(); + s.cache.sync_cache( + &[], + &[], + vec![(key.clone(), Some(vec![1, 2, 3, 4]))], + vec![], + Some(h0), + Some(0), + || true, + ); + // 32 key, 4 byte size + assert_eq!(shared.lock().used_storage_cache_size(), 36 /* bytes */); + + let key = H256::random()[..].to_vec(); + s.cache.sync_cache( + &[], + &[], + vec![(key.clone(), Some(vec![1, 2]))], + vec![], + Some(h0), + Some(0), + || true, + ); + // 32 key, 2 byte size + assert_eq!(shared.lock().used_storage_cache_size(), 34 /* bytes */); + } + + #[test] + fn fix_storage_mismatch_issue() { + let _ = ::env_logger::try_init(); + let root_parent = H256::random(); + + let key = H256::random()[..].to_vec(); + + let h0 = H256::random(); + let h1 = H256::random(); + + let shared = new_shared_cache::(256*1024, (0, 1)); + let mut s = CachingState::new(InMemory::::default(), shared.clone(), Some(root_parent)); + s.cache.sync_cache(&[], &[], vec![(key.clone(), Some(vec![2]))], vec![], Some(h0), Some(0), || true); + + let mut s = CachingState::new(InMemory::::default(), shared.clone(), Some(h0)); + s.cache.sync_cache(&[], &[], vec![(key.clone(), Some(vec![3]))], vec![], Some(h1), Some(1), || true); + + let mut s = CachingState::new(InMemory::::default(), shared.clone(), Some(h1)); + assert_eq!(s.storage(&key).unwrap(), Some(vec![3])); + + // Restart (or unknown block?), clear caches. + { + let mut cache = s.cache.shared_cache.lock(); + let cache = &mut *cache; + cache.lru_storage.clear(); + cache.lru_hashes.clear(); + cache.lru_child_storage.clear(); + cache.modifications.clear(); + } + + // New value is written because of cache miss. + s.cache.local_cache.write().storage.insert(key.clone(), Some(vec![42])); + + // New value is propagated. + s.cache.sync_cache(&[], &[], vec![], vec![], None, None, || true); + + let s = CachingState::new(InMemory::::default(), shared.clone(), Some(h1)); + assert_eq!(s.storage(&key).unwrap(), None); + } +} + +#[cfg(test)] +mod qc { + use std::collections::{HashMap, hash_map::Entry}; + + use quickcheck::{quickcheck, TestResult, Arbitrary}; + + use super::*; + use sr_primitives::testing::{H256, Block as RawBlock, ExtrinsicWrapper}; + use state_machine::backend::InMemory; + use primitives::Blake2Hasher; + + type Block = RawBlock>; + + type KeySet = Vec<(Vec, Option>)>; + + type KeyMap = HashMap, Option>>; + + #[derive(Debug, Clone)] + struct Node { + hash: H256, + parent: H256, + state: KeyMap, + changes: KeySet, + } + + impl Node { + fn new_next(&self, hash: H256, changes: KeySet) -> Self { + let mut state = self.state.clone(); + + for (k, v) in self.state.iter() { state.insert(k.clone(), v.clone()); } + for (k, v) in changes.clone().into_iter() { state.insert(k, v); } + + Self { + hash, + parent: self.hash, + changes, + state, + } + } + + fn new(hash: H256, parent: H256, changes: KeySet) -> Self { + let mut state = KeyMap::new(); + + for (k, v) in changes.clone().into_iter() { state.insert(k, v); } + + Self { + hash, + parent, + state, + changes, + } + } + + fn purge(&mut self, other_changes: &KeySet) { + for (k, _) in other_changes.iter() { + self.state.remove(k); + } + } + } + + #[derive(Debug, Clone)] + enum Action { + Next { hash: H256, changes: KeySet }, + Fork { depth: usize, hash: H256, changes: KeySet }, + ReorgWithImport { depth: usize, hash: H256 }, + FinalizationReorg { fork_depth: usize, depth: usize }, + } + + impl Arbitrary for Action { + fn arbitrary(gen: &mut G) -> Self { + let path = gen.next_u32() as u8; + let mut buf = [0u8; 32]; + + match path { + 0..=175 => { + gen.fill_bytes(&mut buf[..]); + Action::Next { + hash: H256::from(&buf), + changes: { + let mut set = Vec::new(); + for _ in 0..gen.next_u32()/(64*256*256*256) { + set.push((vec![gen.next_u32() as u8], Some(vec![gen.next_u32() as u8]))); + } + set + } + } + }, + 176..=220 => { + gen.fill_bytes(&mut buf[..]); + Action::Fork { + hash: H256::from(&buf), + depth: ((gen.next_u32() as u8) / 32) as usize, + changes: { + let mut set = Vec::new(); + for _ in 0..gen.next_u32()/(64*256*256*256) { + set.push((vec![gen.next_u32() as u8], Some(vec![gen.next_u32() as u8]))); + } + set + } + } + }, + 221..=240 => { + gen.fill_bytes(&mut buf[..]); + Action::ReorgWithImport { + hash: H256::from(&buf), + depth: ((gen.next_u32() as u8) / 32) as usize, // 0-7 + } + }, + _ => { + gen.fill_bytes(&mut buf[..]); + Action::FinalizationReorg { + fork_depth: ((gen.next_u32() as u8) / 32) as usize, // 0-7 + depth: ((gen.next_u32() as u8) / 64) as usize, // 0-3 + } + }, + } + } + } + + struct Mutator { + shared: SharedCache, + canon: Vec, + forks: HashMap>, + } + + impl Mutator { + fn new_empty() -> Self { + let shared = new_shared_cache::(256*1024, (0,1)); + + Self { + shared, + canon: vec![], + forks: HashMap::new(), + } + } + + fn head_state(&self, hash: H256) -> CachingState, Block> { + CachingState::new( + InMemory::::default(), + self.shared.clone(), + Some(hash) + ) + } + + fn canon_head_state(&self) -> CachingState, Block> { + self.head_state(self.canon.last().expect("Expected to be one commit").hash) + } + + fn mutate_static(&mut self, action: Action) -> CachingState, Block> { + self.mutate(action).expect("Expected to provide only valid actions to the mutate_static") + } + + fn canon_len(&self) -> usize { + return self.canon.len(); + } + + fn head_storage_ref(&self) -> &KeyMap { + &self.canon.last().expect("Expected to be one commit").state + } + + fn key_permutations() -> Vec> { + (0u8..255).map(|x| vec![x]).collect() + } + + fn mutate(&mut self, action: Action) -> Result, Block>, ()> { + let state = match action { + Action::Fork { depth, hash, changes } => { + let pos = self.canon.len() as isize - depth as isize; + if pos < 0 || self.canon.len() == 0 || pos >= (self.canon.len()-1) as isize + // no fork on top also, thus len-1 + { + return Err(()); + } + + let pos = pos as usize; + + let fork_at = self.canon[pos].hash; + + let (total_h, parent) = match self.forks.entry(fork_at) { + Entry::Occupied(occupied) => { + let chain = occupied.into_mut(); + let parent = chain.last().expect("No empty forks are ever created").clone(); + let mut node = parent.new_next(hash, changes.clone()); + + for earlier in chain.iter() { + node.purge(&earlier.changes.clone()); + } + + chain.push(node); + + (pos + chain.len(), parent.hash) + }, + Entry::Vacant(vacant) => { + let canon_parent = &self.canon[pos]; + vacant.insert(vec![canon_parent.new_next(hash, changes.clone())]); + + (pos + 1, fork_at) + } + }; + + let mut state = CachingState::new( + InMemory::::default(), + self.shared.clone(), + Some(parent) + ); + + state.cache.sync_cache( + &[], + &[], + changes, + vec![], + Some(hash), + Some(total_h as u64), + || false, + ); + + state + }, + Action::Next { hash, changes } => { + let (next, parent_hash) = match self.canon.last() { + None => { + let parent_hash = H256::from(&[0u8; 32]); + (Node::new(hash, parent_hash, changes.clone()), parent_hash) + }, + Some(parent) => { + (parent.new_next(hash, changes.clone()), parent.hash) + } + }; + + // delete cache entries for earlier + for node in self.canon.iter_mut() { + node.purge(&next.changes); + if let Some(fork) = self.forks.get_mut(&node.hash) { + for node in fork.iter_mut() { + node.purge(&next.changes); + } + } + } + + let mut state = CachingState::new( + InMemory::::default(), + self.shared.clone(), + Some(parent_hash) + ); + + state.cache.sync_cache( + &[], + &[], + next.changes.clone(), + vec![], + Some(hash), + Some(self.canon.len() as u64 + 1), + || true, + ); + + self.canon.push(next); + + state + }, + Action::ReorgWithImport { depth, hash } => { + let pos = self.canon.len() as isize - depth as isize; + if pos < 0 || pos+1 >= self.canon.len() as isize { return Err(()); } + let fork_at = self.canon[pos as usize].hash; + let pos = pos as usize; + + match self.forks.get_mut(&fork_at) { + Some(chain) => { + let mut new_fork = self.canon.drain(pos+1..).collect::>(); + + let retracted: Vec = new_fork.iter().map(|node| node.hash).collect(); + let enacted: Vec = chain.iter().map(|node| node.hash).collect(); + + std::mem::swap(chain, &mut new_fork); + + let mut node = new_fork.last().map( + |node| node.new_next(hash, vec![]) + ).expect("No empty fork ever created!"); + + for invalidators in chain.iter().chain(new_fork.iter()) { + node.purge(&invalidators.changes); + } + + self.canon.extend(new_fork.into_iter()); + + self.canon.push(node); + + let mut state = CachingState::new( + InMemory::::default(), + self.shared.clone(), + Some(fork_at) + ); + + let height = pos as u64 + enacted.len() as u64 + 2; + state.cache.sync_cache( + &enacted[..], + &retracted[..], + vec![], + vec![], + Some(hash), + Some(height), + || true, + ); + + state + } + None => { + return Err(()); // no reorg without a fork atm! + }, + } + }, + Action::FinalizationReorg { fork_depth, depth } => { + let pos = self.canon.len() as isize - fork_depth as isize; + if pos < 0 || pos+1 >= self.canon.len() as isize { return Err(()); } + let fork_at = self.canon[pos as usize].hash; + let pos = pos as usize; + + match self.forks.get_mut(&fork_at) { + Some(fork_chain) => { + let sync_pos = fork_chain.len() as isize - fork_chain.len() as isize - depth as isize; + if sync_pos < 0 || sync_pos >= fork_chain.len() as isize { return Err (()); } + let sync_pos = sync_pos as usize; + + let mut new_fork = self.canon.drain(pos+1..).collect::>(); + + let retracted: Vec = new_fork.iter().map(|node| node.hash).collect(); + let enacted: Vec = fork_chain.iter().take(sync_pos+1).map(|node| node.hash).collect(); + + std::mem::swap(fork_chain, &mut new_fork); + + self.shared.lock().sync(&retracted, &enacted); + + self.head_state( + self.canon.last() + .expect("wasn't forking to emptiness so there shoud be one!") + .hash + ) + }, + None => { + return Err(()); // no reorg to nothing pls! + } + } + + }, + }; + + Ok(state) + } + } + + #[test] + fn smoke() { + let key = H256::random()[..].to_vec(); + let h0 = H256::random(); + let h1a = H256::random(); + let h1b = H256::random(); + let h2a = H256::random(); + let h2b = H256::random(); + let h3a = H256::random(); + let h3b = H256::random(); + + let mut mutator = Mutator::new_empty(); + mutator.mutate_static(Action::Next { hash: h0, changes: vec![(key.clone(), Some(vec![2]))] }); + mutator.mutate_static(Action::Next { hash: h1a, changes: vec![] }); + mutator.mutate_static(Action::Fork { depth: 2, hash: h1b, changes: vec![(key.clone(), Some(vec![3]))] }); + mutator.mutate_static(Action::Fork { depth: 2, hash: h2b, changes: vec![(key.clone(), Some(vec![4]))] }); + mutator.mutate_static(Action::Next { hash: h2a, changes: vec![(key.clone(), Some(vec![5]))] }); + mutator.mutate_static(Action::Next { hash: h3a, changes: vec![] }); + + assert_eq!(mutator.head_state(h3a).storage(&key).unwrap().expect("there should be a value"), vec![5]); + assert!(mutator.head_state(h1a).storage(&key).unwrap().is_none()); + assert!(mutator.head_state(h2b).storage(&key).unwrap().is_none()); + assert!(mutator.head_state(h1b).storage(&key).unwrap().is_none()); + + mutator.mutate_static(Action::ReorgWithImport { depth: 4, hash: h3b }); + assert!(mutator.head_state(h3a).storage(&key).unwrap().is_none()); + } + + fn is_head_match(mutator: &Mutator) -> bool { + let head_state = mutator.canon_head_state(); + + for key in Mutator::key_permutations() { + match (head_state.storage(&key).unwrap(), mutator.head_storage_ref().get(&key)) { + (Some(x), Some(y)) => { + if Some(&x) != y.as_ref() { + eprintln!("{:?} != {:?}", x, y); + return false; + } + }, + (None, Some(_y)) => { + // TODO: cache miss is not tracked atm + }, + (Some(x), None) => { + eprintln!("{:?} != ", x); + return false; + }, + _ => continue, + } + } + true + } + + fn is_canon_match(mutator: &Mutator) -> bool { + for node in mutator.canon.iter() { + let head_state = mutator.head_state(node.hash); + for key in Mutator::key_permutations() { + match (head_state.storage(&key).unwrap(), node.state.get(&key)) { + (Some(x), Some(y)) => { + if Some(&x) != y.as_ref() { + eprintln!("at [{}]: {:?} != {:?}", node.hash, x, y); + return false; + } + }, + (None, Some(_y)) => { + // cache miss is not tracked atm + }, + (Some(x), None) => { + eprintln!("at [{}]: {:?} != ", node.hash, x); + return false; + }, + _ => continue, + } + } + } + true + } + + #[test] + fn reorg() { + let key = H256::random()[..].to_vec(); + let h0 = H256::random(); + let h1 = H256::random(); + let h2 = H256::random(); + let h1b = H256::random(); + let h2b = H256::random(); + + let mut mutator = Mutator::new_empty(); + mutator.mutate_static(Action::Next { hash: h0, changes: vec![] }); + mutator.mutate_static(Action::Next { hash: h1, changes: vec![] }); + mutator.mutate_static(Action::Next { hash: h2, changes: vec![(key.clone(), Some(vec![2]))] }); + mutator.mutate_static(Action::Fork { depth: 2, hash: h1b, changes: vec![(key.clone(), Some(vec![3]))] }); + mutator.mutate_static(Action::ReorgWithImport { depth: 2, hash: h2b }); + + assert!(is_head_match(&mutator)) + } + + fn key(k: u8) -> Vec { vec![k] } + fn val(v: u8) -> Option> { Some(vec![v]) } + fn keyval(k: u8, v: u8) -> KeySet { vec![(key(k), val(v))] } + + #[test] + fn reorg2() { + let h0 = H256::random(); + let h1a = H256::random(); + let h1b = H256::random(); + let h2b = H256::random(); + let h2a = H256::random(); + let h3a = H256::random(); + + let mut mutator = Mutator::new_empty(); + mutator.mutate_static(Action::Next { hash: h0, changes: keyval(1, 1) }); + mutator.mutate_static(Action::Next { hash: h1a, changes: keyval(1, 1) }); + mutator.mutate_static(Action::Fork { depth: 2, hash: h1b, changes: keyval(2, 2 ) }); + + mutator.mutate_static(Action::Next { hash: h2a, changes: keyval(3, 3) }); + mutator.mutate_static(Action::Next { hash: h3a, changes: keyval(4, 4) }); + mutator.mutate_static(Action::ReorgWithImport { depth: 4, hash: h2b }); + + assert!(is_head_match(&mutator)) + } + + #[test] + fn fork2() { + let h1 = H256::random(); + let h2a = H256::random(); + let h2b = H256::random(); + let h3a = H256::random(); + let h3b = H256::random(); + + let mut mutator = Mutator::new_empty(); + mutator.mutate_static(Action::Next { hash: h1, changes: vec![] }); + mutator.mutate_static(Action::Next { hash: h2a, changes: vec![] }); + mutator.mutate_static(Action::Next { hash: h3a, changes: keyval(1, 1) }); + + mutator.mutate_static(Action::Fork { depth: 2, hash: h2b, changes: vec![] }); + mutator.mutate_static(Action::Fork { depth: 2, hash: h3b, changes: keyval(1, 2) }); + + assert!(is_head_match(&mutator)) + } + + #[test] + fn fork3() { + let h1 = H256::random(); + let h2a = H256::random(); + let h2b = H256::random(); + let h3a = H256::random(); + + let mut mutator = Mutator::new_empty(); + mutator.mutate_static(Action::Next { hash: h1, changes: keyval(1, 1) }); + mutator.mutate_static(Action::Next { hash: h2a, changes: keyval(2, 2) }); + mutator.mutate_static(Action::Next { hash: h3a, changes: keyval(3, 3) }); + + mutator.mutate_static(Action::Fork { depth: 2, hash: h2b, changes: keyval(1, 3) }); + + assert!(is_canon_match(&mutator)) + } + + quickcheck! { + fn head_complete(actions: Vec) -> TestResult { + let mut mutator = Mutator::new_empty(); + + for action in actions.into_iter() { + if let Err(_) = mutator.mutate(action) { + return TestResult::discard(); + } + } + + if mutator.canon_len() == 0 { + return TestResult::discard(); + } + + TestResult::from_bool(is_head_match(&mutator)) + } + + fn canon_complete(actions: Vec) -> TestResult { + let mut mutator = Mutator::new_empty(); + + for action in actions.into_iter() { + if let Err(_) = mutator.mutate(action) { + return TestResult::discard(); + } + } + + if mutator.canon_len() == 0 { + return TestResult::discard(); + } + + TestResult::from_bool(is_canon_match(&mutator)) + } + } +} diff --git a/core/client/db/src/utils.rs b/client/db/src/utils.rs similarity index 100% rename from core/client/db/src/utils.rs rename to client/db/src/utils.rs diff --git a/client/executor/Cargo.toml b/client/executor/Cargo.toml new file mode 100644 index 0000000000000..df023d9aa9a93 --- /dev/null +++ b/client/executor/Cargo.toml @@ -0,0 +1,60 @@ +[package] +name = "substrate-executor" +version = "2.0.0" +authors = ["Parity Technologies "] +edition = "2018" + +[dependencies] +derive_more = "0.15.0" +codec = { package = "parity-scale-codec", version = "1.0.0" } +runtime_io = { package = "sr-io", path = "../../primitives/sr-io" } +primitives = { package = "substrate-primitives", path = "../../primitives/core" } +trie = { package = "substrate-trie", path = "../../primitives/trie" } +serializer = { package = "substrate-serializer", path = "../../primitives/serializer" } +runtime_version = { package = "sr-version", path = "../../primitives/sr-version" } +panic-handler = { package = "substrate-panic-handler", path = "../../primitives/panic-handler" } +wasmi = "0.6.2" +parity-wasm = "0.40.3" +lazy_static = "1.4.0" +wasm-interface = { package = "substrate-wasm-interface", path = "../../primitives/wasm-interface" } +runtime-interface = { package = "substrate-runtime-interface", path = "../../primitives/runtime-interface" } +externalities = { package = "substrate-externalities", path = "../../primitives/externalities" } +parking_lot = "0.9.0" +log = "0.4.8" +libsecp256k1 = "0.3.2" + +cranelift-codegen = { version = "0.46.1", optional = true } +cranelift-entity = { version = "0.46.1", optional = true } +cranelift-frontend = { version = "0.46.1", optional = true } +cranelift-native = { version = "0.46.1", optional = true } +cranelift-wasm = { version = "0.46.1", optional = true } +wasmtime-environ = { version = "0.2", optional = true, git = "https://github.com/CraneStation/wasmtime.git", rev = "71dd73d6" } +wasmtime-jit = { version = "0.2", optional = true, git = "https://github.com/CraneStation/wasmtime.git", rev = "71dd73d6" } +wasmtime-runtime = { version = "0.2", optional = true, git = "https://github.com/CraneStation/wasmtime.git", rev = "71dd73d6" } + +[dev-dependencies] +assert_matches = "1.3.0" +wabt = "0.9.2" +hex-literal = "0.2.1" +runtime-test = { package = "substrate-runtime-test", path = "runtime-test" } +client-api = { package = "substrate-client-api", path = "../api" } +substrate-offchain = { path = "../offchain/" } +state_machine = { package = "substrate-state-machine", path = "../../primitives/state-machine" } +test-case = "0.3.3" + +[features] +default = [] +wasm-extern-trace = [] +wasmtime = [ + "cranelift-codegen", + "cranelift-entity", + "cranelift-frontend", + "cranelift-native", + "cranelift-wasm", + "wasmtime-environ", + "wasmtime-jit", + "wasmtime-runtime", +] +wasmi-errno = [ + "wasmi/errno" +] diff --git a/client/executor/runtime-test/Cargo.toml b/client/executor/runtime-test/Cargo.toml new file mode 100644 index 0000000000000..610db4545800e --- /dev/null +++ b/client/executor/runtime-test/Cargo.toml @@ -0,0 +1,20 @@ +[package] +name = "substrate-runtime-test" +version = "2.0.0" +authors = ["Parity Technologies "] +edition = "2018" +build = "build.rs" + +[dependencies] +rstd = { package = "sr-std", path = "../../../primitives/sr-std", default-features = false } +runtime_io = { package = "sr-io", path = "../../../primitives/sr-io", default-features = false } +sandbox = { package = "sr-sandbox", path = "../../../primitives/sr-sandbox", default-features = false } +primitives = { package = "substrate-primitives", path = "../../../primitives/core", default-features = false } +sr-primitives = { package = "sr-primitives", path = "../../../primitives/sr-primitives", default-features = false } + +[build-dependencies] +wasm-builder-runner = { package = "substrate-wasm-builder-runner", path = "../../../client/utils/wasm-builder-runner", version = "1.0.4" } + +[features] +default = [ "std" ] +std = ["runtime_io/std", "sandbox/std", "rstd/std"] diff --git a/node/runtime/build.rs b/client/executor/runtime-test/build.rs similarity index 96% rename from node/runtime/build.rs rename to client/executor/runtime-test/build.rs index f5c2f98a75c8a..d6b0ae626facb 100644 --- a/node/runtime/build.rs +++ b/client/executor/runtime-test/build.rs @@ -20,7 +20,7 @@ fn main() { build_current_project_with_rustflags( "wasm_binary.rs", WasmBuilderSource::CratesOrPath { - path: "../../core/utils/wasm-builder", + path: "../../../utils/wasm-builder", version: "1.0.8", }, // This instructs LLD to export __heap_base as a global variable, which is used by the diff --git a/core/executor/runtime-test/src/lib.rs b/client/executor/runtime-test/src/lib.rs similarity index 75% rename from core/executor/runtime-test/src/lib.rs rename to client/executor/runtime-test/src/lib.rs index 61eca8dd4e23f..16c6c353d9e37 100644 --- a/core/executor/runtime-test/src/lib.rs +++ b/client/executor/runtime-test/src/lib.rs @@ -10,8 +10,8 @@ use rstd::{vec::Vec, vec}; #[cfg(not(feature = "std"))] use runtime_io::{ - set_storage, storage, clear_prefix, blake2_128, blake2_256, - twox_128, twox_256, ed25519_verify, sr25519_verify, + storage, hashing::{blake2_128, blake2_256, twox_128, twox_256}, + crypto::{ed25519_verify, sr25519_verify}, }; #[cfg(not(feature = "std"))] use sr_primitives::{print, traits::{BlakeTwo256, Hash}}; @@ -21,20 +21,20 @@ use primitives::{ed25519, sr25519}; primitives::wasm_export_functions! { fn test_data_in(input: Vec) -> Vec { print("set_storage"); - set_storage(b"input", &input); + storage::set(b"input", &input); print("storage"); - let foo = storage(b"foo").unwrap(); + let foo = storage::get(b"foo").unwrap(); print("set_storage"); - set_storage(b"baz", &foo); + storage::set(b"baz", &foo); print("finished!"); b"all ok!".to_vec() } fn test_clear_prefix(input: Vec) -> Vec { - clear_prefix(&input); + storage::clear_prefix(&input); b"all ok!".to_vec() } @@ -142,40 +142,49 @@ primitives::wasm_export_functions! { fn test_offchain_local_storage() -> bool { let kind = primitives::offchain::StorageKind::PERSISTENT; - assert_eq!(runtime_io::local_storage_get(kind, b"test"), None); - runtime_io::local_storage_set(kind, b"test", b"asd"); - assert_eq!(runtime_io::local_storage_get(kind, b"test"), Some(b"asd".to_vec())); - - let res = runtime_io::local_storage_compare_and_set(kind, b"test", Some(b"asd"), b""); - assert_eq!(runtime_io::local_storage_get(kind, b"test"), Some(b"".to_vec())); + assert_eq!(runtime_io::offchain::local_storage_get(kind, b"test"), None); + runtime_io::offchain::local_storage_set(kind, b"test", b"asd"); + assert_eq!(runtime_io::offchain::local_storage_get(kind, b"test"), Some(b"asd".to_vec())); + + let res = runtime_io::offchain::local_storage_compare_and_set( + kind, + b"test", + Some(b"asd".to_vec()), + b"", + ); + assert_eq!(runtime_io::offchain::local_storage_get(kind, b"test"), Some(b"".to_vec())); res } fn test_offchain_local_storage_with_none() { let kind = primitives::offchain::StorageKind::PERSISTENT; - assert_eq!(runtime_io::local_storage_get(kind, b"test"), None); + assert_eq!(runtime_io::offchain::local_storage_get(kind, b"test"), None); - let res = runtime_io::local_storage_compare_and_set(kind, b"test", None, b"value"); + let res = runtime_io::offchain::local_storage_compare_and_set(kind, b"test", None, b"value"); assert_eq!(res, true); - assert_eq!(runtime_io::local_storage_get(kind, b"test"), Some(b"value".to_vec())); + assert_eq!(runtime_io::offchain::local_storage_get(kind, b"test"), Some(b"value".to_vec())); } fn test_offchain_http() -> bool { use primitives::offchain::HttpRequestStatus; let run = || -> Option<()> { - let id = runtime_io::http_request_start("POST", "http://localhost:12345", &[]).ok()?; - runtime_io::http_request_add_header(id, "X-Auth", "test").ok()?; - runtime_io::http_request_write_body(id, &[1, 2, 3, 4], None).ok()?; - runtime_io::http_request_write_body(id, &[], None).ok()?; - let status = runtime_io::http_response_wait(&[id], None); + let id = runtime_io::offchain::http_request_start( + "POST", + "http://localhost:12345", + &[], + ).ok()?; + runtime_io::offchain::http_request_add_header(id, "X-Auth", "test").ok()?; + runtime_io::offchain::http_request_write_body(id, &[1, 2, 3, 4], None).ok()?; + runtime_io::offchain::http_request_write_body(id, &[], None).ok()?; + let status = runtime_io::offchain::http_response_wait(&[id], None); assert!(status == vec![HttpRequestStatus::Finished(200)], "Expected Finished(200) status."); - let headers = runtime_io::http_response_headers(id); + let headers = runtime_io::offchain::http_response_headers(id); assert_eq!(headers, vec![(b"X-Auth".to_vec(), b"hello".to_vec())]); let mut buffer = vec![0; 64]; - let read = runtime_io::http_response_read_body(id, &mut buffer, None).ok()?; + let read = runtime_io::offchain::http_response_read_body(id, &mut buffer, None).ok()?; assert_eq!(read, 3); - assert_eq!(&buffer[0..read], &[1, 2, 3]); - let read = runtime_io::http_response_read_body(id, &mut buffer, None).ok()?; + assert_eq!(&buffer[0..read as usize], &[1, 2, 3]); + let read = runtime_io::offchain::http_response_read_body(id, &mut buffer, None).ok()?; assert_eq!(read, 0); Some(()) @@ -239,7 +248,7 @@ fn execute_sandboxed( }; let mut instance = sandbox::Instance::new(code, &env_builder, &mut state)?; - let result = instance.invoke(b"call", args, &mut state); + let result = instance.invoke("call", args, &mut state); result.map_err(|_| sandbox::HostError) } diff --git a/core/executor/src/allocator.rs b/client/executor/src/allocator.rs similarity index 100% rename from core/executor/src/allocator.rs rename to client/executor/src/allocator.rs diff --git a/client/executor/src/deprecated_host_interface.rs b/client/executor/src/deprecated_host_interface.rs new file mode 100644 index 0000000000000..223b13367aa87 --- /dev/null +++ b/client/executor/src/deprecated_host_interface.rs @@ -0,0 +1,1075 @@ +// Copyright 2017-2019 Parity Technologies (UK) Ltd. +// This file is part of Substrate. + +// Substrate is free software: you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. + +// Substrate is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. + +// You should have received a copy of the GNU General Public License +// along with Substrate. If not, see . + +//! Definition and implementation of the old and deprecated Substrate runtime interface for the host. + +use codec::Encode; +use std::{convert::TryFrom, str}; +use primitives::{ + blake2_128, blake2_256, twox_64, twox_128, twox_256, ed25519, sr25519, keccak_256, Blake2Hasher, Pair, + crypto::KeyTypeId, offchain, +}; +use trie::{TrieConfiguration, trie_types::Layout}; +use wasm_interface::{ + Pointer, WordSize, WritePrimitive, ReadPrimitive, FunctionContext, Result as WResult, +}; + +#[cfg(feature="wasm-extern-trace")] +macro_rules! debug_trace { + ( $( $x:tt )* ) => ( trace!( $( $x )* ) ) +} + +#[cfg(not(feature="wasm-extern-trace"))] +macro_rules! debug_trace { + ( $( $x:tt )* ) => () +} + +/// The old and deprecated Substrate externals. These are still required for backwards compatibility +/// reasons. +pub struct SubstrateExternals; + +enum RecoverResult { + Invalid(u32), + Valid(secp256k1::PublicKey), +} + +fn secp256k1_recover( + context: &mut dyn FunctionContext, + msg_data: Pointer, + sig_data: Pointer, +) -> WResult { + let mut sig = [0u8; 65]; + context.read_memory_into(sig_data, &mut sig[..]) + .map_err(|_| "Invalid attempt to get signature in ext_secp256k1_ecdsa_recover")?; + let rs = match secp256k1::Signature::parse_slice(&sig[0..64]) { + Ok(rs) => rs, + _ => return Ok(RecoverResult::Invalid(1)), + }; + + let recovery_id = if sig[64] > 26 { sig[64] - 27 } else { sig[64] } as u8; + let v = match secp256k1::RecoveryId::parse(recovery_id) { + Ok(v) => v, + _ => return Ok(RecoverResult::Invalid(2)), + }; + + let mut msg = [0u8; 32]; + context.read_memory_into(msg_data, &mut msg[..]) + .map_err(|_| "Invalid attempt to get message in ext_secp256k1_ecdsa_recover")?; + + Ok(match secp256k1::recover(&secp256k1::Message::parse(&msg), &rs, &v) { + Ok(pubkey) => RecoverResult::Valid(pubkey), + Err(_) => RecoverResult::Invalid(3), + }) +} + +impl_wasm_host_interface! { + impl SubstrateExternals where context { + ext_malloc(size: WordSize) -> Pointer { + let r = context.allocate_memory(size)?; + debug_trace!(target: "sr-io", "malloc {} bytes at {:?}", size, r); + Ok(r) + } + + ext_free(addr: Pointer) { + context.deallocate_memory(addr)?; + debug_trace!(target: "sr-io", "free {:?}", addr); + Ok(()) + } + + ext_sandbox_instantiate( + dispatch_thunk_idx: u32, + wasm_ptr: Pointer, + wasm_len: WordSize, + imports_ptr: Pointer, + imports_len: WordSize, + state: u32, + ) -> u32 { + let wasm = context.read_memory(wasm_ptr, wasm_len) + .map_err(|_| "OOB while ext_sandbox_instantiate: wasm")?; + let raw_env_def = context.read_memory(imports_ptr, imports_len) + .map_err(|_| "OOB while ext_sandbox_instantiate: imports")?; + + context.sandbox().instance_new(dispatch_thunk_idx, &wasm, &raw_env_def, state) + } + + ext_sandbox_instance_teardown(instance_idx: u32) { + context.sandbox().instance_teardown(instance_idx) + } + + ext_sandbox_invoke( + instance_idx: u32, + export_ptr: Pointer, + export_len: WordSize, + args_ptr: Pointer, + args_len: WordSize, + return_val_ptr: Pointer, + return_val_len: WordSize, + state: u32, + ) -> u32 { + let export = context.read_memory(export_ptr, export_len) + .map_err(|_| "OOB while ext_sandbox_invoke: export") + .and_then(|b| + String::from_utf8(b) + .map_err(|_| "Export name should be a valid utf-8 sequence") + )?; + + // Deserialize arguments and convert them into wasmi types. + let serialized_args = context.read_memory(args_ptr, args_len) + .map_err(|_| "OOB while ext_sandbox_invoke: args")?; + + context.sandbox().invoke( + instance_idx, + &export, + &serialized_args, + return_val_ptr, + return_val_len, + state, + ) + } + + ext_sandbox_memory_new(initial: WordSize, maximum: WordSize) -> u32 { + context.sandbox().memory_new(initial, maximum) + } + + ext_sandbox_memory_get( + memory_idx: u32, + offset: WordSize, + buf_ptr: Pointer, + buf_len: WordSize, + ) -> u32 { + context.sandbox().memory_get(memory_idx, offset, buf_ptr, buf_len) + } + + ext_sandbox_memory_set( + memory_idx: u32, + offset: WordSize, + val_ptr: Pointer, + val_len: WordSize, + ) -> u32 { + context.sandbox().memory_set(memory_idx, offset, val_ptr, val_len) + } + + ext_sandbox_memory_teardown(memory_idx: u32) { + context.sandbox().memory_teardown(memory_idx) + } + + ext_print_utf8(utf8_data: Pointer, utf8_len: WordSize) { + if let Ok(utf8) = context.read_memory(utf8_data, utf8_len) { + runtime_io::misc::print_utf8(&utf8); + } + Ok(()) + } + + ext_print_hex(data: Pointer, len: WordSize) { + if let Ok(hex) = context.read_memory(data, len) { + runtime_io::misc::print_hex(&hex); + } + Ok(()) + } + + ext_print_num(number: u64) { + runtime_io::misc::print_num(number); + Ok(()) + } + + ext_log( + level: u32, + target_data: Pointer, + target_len: WordSize, + message_data: Pointer, + message_len: WordSize, + ) { + let target = context.read_memory(target_data, target_len) + .map_err(|_| "Invalid attempt to determine target in ext_log")?; + let message = context.read_memory(message_data, message_len) + .map_err(|_| "Invalid attempt to determine message in ext_log")?; + + let target_str = std::str::from_utf8(&target) + .map_err(|_| "Target invalid utf8 in ext_log")?; + + runtime_io::logging::log(level.into(), &target_str, &message); + Ok(()) + } + + ext_set_storage( + key_data: Pointer, + key_len: WordSize, + value_data: Pointer, + value_len: WordSize, + ) { + let key = context.read_memory(key_data, key_len) + .map_err(|_| "Invalid attempt to determine key in ext_set_storage")?; + let value = context.read_memory(value_data, value_len) + .map_err(|_| "Invalid attempt to determine value in ext_set_storage")?; + Ok(runtime_io::storage::set(&key, &value)) + } + + ext_set_child_storage( + storage_key_data: Pointer, + storage_key_len: WordSize, + key_data: Pointer, + key_len: WordSize, + value_data: Pointer, + value_len: WordSize, + ) { + let storage_key = context.read_memory(storage_key_data, storage_key_len) + .map_err(|_| "Invalid attempt to determine storage_key in ext_set_child_storage")?; + let key = context.read_memory(key_data, key_len) + .map_err(|_| "Invalid attempt to determine key in ext_set_child_storage")?; + let value = context.read_memory(value_data, value_len) + .map_err(|_| "Invalid attempt to determine value in ext_set_child_storage")?; + + Ok(runtime_io::storage::child_set(&storage_key, &key, &value)) + } + + ext_clear_child_storage( + storage_key_data: Pointer, + storage_key_len: WordSize, + key_data: Pointer, + key_len: WordSize, + ) { + let storage_key = context.read_memory(storage_key_data, storage_key_len) + .map_err(|_| "Invalid attempt to determine storage_key in ext_clear_child_storage")?; + let key = context.read_memory(key_data, key_len) + .map_err(|_| "Invalid attempt to determine key in ext_clear_child_storage")?; + + Ok(runtime_io::storage::child_clear(&storage_key, &key)) + } + + ext_clear_storage(key_data: Pointer, key_len: WordSize) { + let key = context.read_memory(key_data, key_len) + .map_err(|_| "Invalid attempt to determine key in ext_clear_storage")?; + Ok(runtime_io::storage::clear(&key)) + } + + ext_exists_storage(key_data: Pointer, key_len: WordSize) -> u32 { + let key = context.read_memory(key_data, key_len) + .map_err(|_| "Invalid attempt to determine key in ext_exists_storage")?; + Ok(if runtime_io::storage::exists(&key) { 1 } else { 0 }) + } + + ext_exists_child_storage( + storage_key_data: Pointer, + storage_key_len: WordSize, + key_data: Pointer, + key_len: WordSize, + ) -> u32 { + let storage_key = context.read_memory(storage_key_data, storage_key_len) + .map_err(|_| "Invalid attempt to determine storage_key in ext_exists_child_storage")?; + let key = context.read_memory(key_data, key_len) + .map_err(|_| "Invalid attempt to determine key in ext_exists_child_storage")?; + + Ok(if runtime_io::storage::child_exists(&storage_key, &key) { 1 } else { 0 }) + } + + ext_clear_prefix(prefix_data: Pointer, prefix_len: WordSize) { + let prefix = context.read_memory(prefix_data, prefix_len) + .map_err(|_| "Invalid attempt to determine prefix in ext_clear_prefix")?; + Ok(runtime_io::storage::clear_prefix(&prefix)) + } + + ext_clear_child_prefix( + storage_key_data: Pointer, + storage_key_len: WordSize, + prefix_data: Pointer, + prefix_len: WordSize, + ) { + let storage_key = context.read_memory(storage_key_data, storage_key_len) + .map_err(|_| "Invalid attempt to determine storage_key in ext_clear_child_prefix")?; + let prefix = context.read_memory(prefix_data, prefix_len) + .map_err(|_| "Invalid attempt to determine prefix in ext_clear_child_prefix")?; + Ok(runtime_io::storage::child_clear_prefix(&storage_key, &prefix)) + } + + ext_kill_child_storage(storage_key_data: Pointer, storage_key_len: WordSize) { + let storage_key = context.read_memory(storage_key_data, storage_key_len) + .map_err(|_| "Invalid attempt to determine storage_key in ext_kill_child_storage")?; + Ok(runtime_io::storage::child_storage_kill(&storage_key)) + } + + ext_get_allocated_storage( + key_data: Pointer, + key_len: WordSize, + written_out: Pointer, + ) -> Pointer { + let key = context.read_memory(key_data, key_len) + .map_err(|_| "Invalid attempt to determine key in ext_get_allocated_storage")?; + + if let Some(value) = runtime_io::storage::get(&key) { + let offset = context.allocate_memory(value.len() as u32)?; + context.write_memory(offset, &value) + .map_err(|_| "Invalid attempt to set memory in ext_get_allocated_storage")?; + context.write_primitive(written_out, value.len() as u32) + .map_err(|_| "Invalid attempt to write written_out in ext_get_allocated_storage")?; + Ok(offset) + } else { + context.write_primitive(written_out, u32::max_value()) + .map_err(|_| "Invalid attempt to write failed written_out in ext_get_allocated_storage")?; + Ok(Pointer::null()) + } + } + + ext_get_allocated_child_storage( + storage_key_data: Pointer, + storage_key_len: WordSize, + key_data: Pointer, + key_len: WordSize, + written_out: Pointer, + ) -> Pointer { + let storage_key = context.read_memory(storage_key_data, storage_key_len) + .map_err(|_| "Invalid attempt to determine storage_key in ext_get_allocated_child_storage")?; + let key = context.read_memory(key_data, key_len) + .map_err(|_| "Invalid attempt to determine key in ext_get_allocated_child_storage")?; + + if let Some(value) = runtime_io::storage::child_get(&storage_key, &key) { + let offset = context.allocate_memory(value.len() as u32)?; + context.write_memory(offset, &value) + .map_err(|_| "Invalid attempt to set memory in ext_get_allocated_child_storage")?; + context.write_primitive(written_out, value.len() as u32) + .map_err(|_| "Invalid attempt to write written_out in ext_get_allocated_child_storage")?; + Ok(offset) + } else { + context.write_primitive(written_out, u32::max_value()) + .map_err(|_| "Invalid attempt to write failed written_out in ext_get_allocated_child_storage")?; + Ok(Pointer::null()) + } + } + + ext_get_storage_into( + key_data: Pointer, + key_len: WordSize, + value_data: Pointer, + value_len: WordSize, + value_offset: WordSize, + ) -> WordSize { + let key = context.read_memory(key_data, key_len) + .map_err(|_| "Invalid attempt to get key in ext_get_storage_into")?; + + if let Some(value) = runtime_io::storage::get(&key) { + let data = &value[value.len().min(value_offset as usize)..]; + let written = std::cmp::min(value_len as usize, data.len()); + context.write_memory(value_data, &data[..written]) + .map_err(|_| "Invalid attempt to set value in ext_get_storage_into")?; + Ok(value.len() as u32) + } else { + Ok(u32::max_value()) + } + } + + ext_get_child_storage_into( + storage_key_data: Pointer, + storage_key_len: WordSize, + key_data: Pointer, + key_len: WordSize, + value_data: Pointer, + value_len: WordSize, + value_offset: WordSize, + ) -> WordSize { + let storage_key = context.read_memory(storage_key_data, storage_key_len) + .map_err(|_| "Invalid attempt to determine storage_key in ext_get_child_storage_into")?; + let key = context.read_memory(key_data, key_len) + .map_err(|_| "Invalid attempt to get key in ext_get_child_storage_into")?; + + if let Some(value) = runtime_io::storage::child_get(&storage_key, &key) { + let data = &value[value.len().min(value_offset as usize)..]; + let written = std::cmp::min(value_len as usize, data.len()); + context.write_memory(value_data, &data[..written]) + .map_err(|_| "Invalid attempt to get value in ext_get_child_storage_into")?; + Ok(value.len() as u32) + } else { + Ok(u32::max_value()) + } + } + + ext_storage_root(result: Pointer) { + context.write_memory(result, runtime_io::storage::root().as_ref()) + .map_err(|_| "Invalid attempt to set memory in ext_storage_root".into()) + } + + ext_child_storage_root( + storage_key_data: Pointer, + storage_key_len: WordSize, + written_out: Pointer, + ) -> Pointer { + let storage_key = context.read_memory(storage_key_data, storage_key_len) + .map_err(|_| "Invalid attempt to determine storage_key in ext_child_storage_root")?; + let value = runtime_io::storage::child_root(&storage_key); + + let offset = context.allocate_memory(value.len() as u32)?; + context.write_memory(offset, &value) + .map_err(|_| "Invalid attempt to set memory in ext_child_storage_root")?; + context.write_primitive(written_out, value.len() as u32) + .map_err(|_| "Invalid attempt to write written_out in ext_child_storage_root")?; + Ok(offset) + } + + ext_storage_changes_root( + parent_hash_data: Pointer, + _len: WordSize, + result: Pointer, + ) -> u32 { + let mut parent_hash = [0u8; 32]; + context.read_memory_into(parent_hash_data, &mut parent_hash[..]) + .map_err(|_| "Invalid attempt to get parent_hash in ext_storage_changes_root")?; + + if let Some(r) = runtime_io::storage::changes_root(parent_hash) { + context.write_memory(result, &r[..]) + .map_err(|_| "Invalid attempt to set memory in ext_storage_changes_root")?; + Ok(1) + } else { + Ok(0) + } + } + + ext_blake2_256_enumerated_trie_root( + values_data: Pointer, + lens_data: Pointer, + lens_len: WordSize, + result: Pointer, + ) { + let values = (0..lens_len) + .map(|i| context.read_primitive(lens_data.offset(i).ok_or("Pointer overflow")?)) + .collect::, _>>()? + .into_iter() + .scan(0u32, |acc, v| { let o = *acc; *acc += v; Some((o, v)) }) + .map(|(offset, len)| + context.read_memory(values_data.offset(offset).ok_or("Pointer overflow")?, len) + .map_err(|_| + "Invalid attempt to get memory in ext_blake2_256_enumerated_trie_root" + ) + ) + .collect::, _>>()?; + let r = Layout::::ordered_trie_root(values.into_iter()); + context.write_memory(result, &r[..]) + .map_err(|_| "Invalid attempt to set memory in ext_blake2_256_enumerated_trie_root")?; + Ok(()) + } + + ext_chain_id() -> u64 { + Ok(runtime_io::misc::chain_id()) + } + + ext_twox_64(data: Pointer, len: WordSize, out: Pointer) { + let result: [u8; 8] = if len == 0 { + let hashed = twox_64(&[0u8; 0]); + hashed + } else { + let key = context.read_memory(data, len) + .map_err(|_| "Invalid attempt to get key in ext_twox_64")?; + let hashed_key = twox_64(&key); + hashed_key + }; + + context.write_memory(out, &result) + .map_err(|_| "Invalid attempt to set result in ext_twox_64")?; + Ok(()) + } + + ext_twox_128(data: Pointer, len: WordSize, out: Pointer) { + let result: [u8; 16] = if len == 0 { + let hashed = twox_128(&[0u8; 0]); + hashed + } else { + let key = context.read_memory(data, len) + .map_err(|_| "Invalid attempt to get key in ext_twox_128")?; + let hashed_key = twox_128(&key); + hashed_key + }; + + context.write_memory(out, &result) + .map_err(|_| "Invalid attempt to set result in ext_twox_128")?; + Ok(()) + } + + ext_twox_256(data: Pointer, len: WordSize, out: Pointer) { + let result: [u8; 32] = if len == 0 { + twox_256(&[0u8; 0]) + } else { + let mem = context.read_memory(data, len) + .map_err(|_| "Invalid attempt to get data in ext_twox_256")?; + twox_256(&mem) + }; + context.write_memory(out, &result) + .map_err(|_| "Invalid attempt to set result in ext_twox_256")?; + Ok(()) + } + + ext_blake2_128(data: Pointer, len: WordSize, out: Pointer) { + let result: [u8; 16] = if len == 0 { + let hashed = blake2_128(&[0u8; 0]); + hashed + } else { + let key = context.read_memory(data, len) + .map_err(|_| "Invalid attempt to get key in ext_blake2_128")?; + let hashed_key = blake2_128(&key); + hashed_key + }; + + context.write_memory(out, &result) + .map_err(|_| "Invalid attempt to set result in ext_blake2_128")?; + Ok(()) + } + + ext_blake2_256(data: Pointer, len: WordSize, out: Pointer) { + let result: [u8; 32] = if len == 0 { + blake2_256(&[0u8; 0]) + } else { + let mem = context.read_memory(data, len) + .map_err(|_| "Invalid attempt to get data in ext_blake2_256")?; + blake2_256(&mem) + }; + context.write_memory(out, &result) + .map_err(|_| "Invalid attempt to set result in ext_blake2_256")?; + Ok(()) + } + + ext_keccak_256(data: Pointer, len: WordSize, out: Pointer) { + let result: [u8; 32] = if len == 0 { + keccak_256(&[0u8; 0]) + } else { + let mem = context.read_memory(data, len) + .map_err(|_| "Invalid attempt to get data in ext_keccak_256")?; + keccak_256(&mem) + }; + context.write_memory(out, &result) + .map_err(|_| "Invalid attempt to set result in ext_keccak_256")?; + Ok(()) + } + + ext_ed25519_public_keys(id_data: Pointer, result_len: Pointer) -> Pointer { + let mut id = [0u8; 4]; + context.read_memory_into(id_data, &mut id[..]) + .map_err(|_| "Invalid attempt to get id in ext_ed25519_public_keys")?; + let key_type = KeyTypeId(id); + + let keys = runtime_io::crypto::ed25519_public_keys(key_type).encode(); + + let len = keys.len() as u32; + let offset = context.allocate_memory(len)?; + + context.write_memory(offset, keys.as_ref()) + .map_err(|_| "Invalid attempt to set memory in ext_ed25519_public_keys")?; + context.write_primitive(result_len, len) + .map_err(|_| "Invalid attempt to write result_len in ext_ed25519_public_keys")?; + + Ok(offset) + } + + ext_ed25519_verify( + msg_data: Pointer, + msg_len: WordSize, + sig_data: Pointer, + pubkey_data: Pointer, + ) -> u32 { + let mut sig = [0u8; 64]; + context.read_memory_into(sig_data, &mut sig[..]) + .map_err(|_| "Invalid attempt to get signature in ext_ed25519_verify")?; + let mut pubkey = [0u8; 32]; + context.read_memory_into(pubkey_data, &mut pubkey[..]) + .map_err(|_| "Invalid attempt to get pubkey in ext_ed25519_verify")?; + let msg = context.read_memory(msg_data, msg_len) + .map_err(|_| "Invalid attempt to get message in ext_ed25519_verify")?; + + Ok(if ed25519::Pair::verify_weak(&sig, &msg, &pubkey) { + 0 + } else { + 1 + }) + } + + ext_ed25519_generate( + id_data: Pointer, + seed: Pointer, + seed_len: WordSize, + out: Pointer, + ) { + let mut id = [0u8; 4]; + context.read_memory_into(id_data, &mut id[..]) + .map_err(|_| "Invalid attempt to get id in ext_ed25519_generate")?; + let key_type = KeyTypeId(id); + + let seed = if seed_len == 0 { + None + } else { + Some( + context.read_memory(seed, seed_len) + .map_err(|_| "Invalid attempt to get seed in ext_ed25519_generate")? + ) + }; + + let pubkey = runtime_io::crypto::ed25519_generate(key_type, seed); + + context.write_memory(out, pubkey.as_ref()) + .map_err(|_| "Invalid attempt to set out in ext_ed25519_generate".into()) + } + + ext_ed25519_sign( + id_data: Pointer, + pubkey_data: Pointer, + msg_data: Pointer, + msg_len: WordSize, + out: Pointer, + ) -> u32 { + let mut id = [0u8; 4]; + context.read_memory_into(id_data, &mut id[..]) + .map_err(|_| "Invalid attempt to get id in ext_ed25519_sign")?; + let key_type = KeyTypeId(id); + + let mut pubkey = [0u8; 32]; + context.read_memory_into(pubkey_data, &mut pubkey[..]) + .map_err(|_| "Invalid attempt to get pubkey in ext_ed25519_sign")?; + + let msg = context.read_memory(msg_data, msg_len) + .map_err(|_| "Invalid attempt to get message in ext_ed25519_sign")?; + + let pub_key = ed25519::Public::try_from(pubkey.as_ref()) + .map_err(|_| "Invalid `ed25519` public key")?; + + let signature = runtime_io::crypto::ed25519_sign(key_type, &pub_key, &msg); + + match signature { + Some(signature) => { + context.write_memory(out, signature.as_ref()) + .map_err(|_| "Invalid attempt to set out in ext_ed25519_sign")?; + Ok(0) + }, + None => Ok(1), + } + } + + ext_sr25519_public_keys(id_data: Pointer, result_len: Pointer) -> Pointer { + let mut id = [0u8; 4]; + context.read_memory_into(id_data, &mut id[..]) + .map_err(|_| "Invalid attempt to get id in ext_sr25519_public_keys")?; + let key_type = KeyTypeId(id); + + let keys = runtime_io::crypto::sr25519_public_keys(key_type).encode(); + + let len = keys.len() as u32; + let offset = context.allocate_memory(len)?; + + context.write_memory(offset, keys.as_ref()) + .map_err(|_| "Invalid attempt to set memory in ext_sr25519_public_keys")?; + context.write_primitive(result_len, len) + .map_err(|_| "Invalid attempt to write result_len in ext_sr25519_public_keys")?; + + Ok(offset) + } + + ext_sr25519_verify( + msg_data: Pointer, + msg_len: WordSize, + sig_data: Pointer, + pubkey_data: Pointer, + ) -> u32 { + let mut sig = [0u8; 64]; + context.read_memory_into(sig_data, &mut sig[..]) + .map_err(|_| "Invalid attempt to get signature in ext_sr25519_verify")?; + let mut pubkey = [0u8; 32]; + context.read_memory_into(pubkey_data, &mut pubkey[..]) + .map_err(|_| "Invalid attempt to get pubkey in ext_sr25519_verify")?; + let msg = context.read_memory(msg_data, msg_len) + .map_err(|_| "Invalid attempt to get message in ext_sr25519_verify")?; + + Ok(if sr25519::Pair::verify_weak(&sig, &msg, &pubkey) { + 0 + } else { + 1 + }) + } + + ext_sr25519_generate( + id_data: Pointer, + seed: Pointer, + seed_len: WordSize, + out: Pointer, + ) { + let mut id = [0u8; 4]; + context.read_memory_into(id_data, &mut id[..]) + .map_err(|_| "Invalid attempt to get id in ext_sr25519_generate")?; + let key_type = KeyTypeId(id); + let seed = if seed_len == 0 { + None + } else { + Some( + context.read_memory(seed, seed_len) + .map_err(|_| "Invalid attempt to get seed in ext_sr25519_generate")? + ) + }; + + let pubkey = runtime_io::crypto::sr25519_generate(key_type, seed); + + context.write_memory(out, pubkey.as_ref()) + .map_err(|_| "Invalid attempt to set out in ext_sr25519_generate".into()) + } + + ext_sr25519_sign( + id_data: Pointer, + pubkey_data: Pointer, + msg_data: Pointer, + msg_len: WordSize, + out: Pointer, + ) -> u32 { + let mut id = [0u8; 4]; + context.read_memory_into(id_data, &mut id[..]) + .map_err(|_| "Invalid attempt to get id in ext_sr25519_sign")?; + let key_type = KeyTypeId(id); + + let mut pubkey = [0u8; 32]; + context.read_memory_into(pubkey_data, &mut pubkey[..]) + .map_err(|_| "Invalid attempt to get pubkey in ext_sr25519_sign")?; + + let msg = context.read_memory(msg_data, msg_len) + .map_err(|_| "Invalid attempt to get message in ext_sr25519_sign")?; + + let pub_key = sr25519::Public::try_from(pubkey.as_ref()) + .map_err(|_| "Invalid `sr25519` public key")?; + + let signature = runtime_io::crypto::sr25519_sign(key_type, &pub_key, &msg); + + match signature { + Some(signature) => { + context.write_memory(out, signature.as_ref()) + .map_err(|_| "Invalid attempt to set out in ext_sr25519_sign")?; + Ok(0) + }, + None => Ok(1), + } + } + + ext_secp256k1_ecdsa_recover( + msg_data: Pointer, + sig_data: Pointer, + pubkey_data: Pointer, + ) -> u32 { + match secp256k1_recover(context, msg_data, sig_data)? { + RecoverResult::Invalid(c) => Ok(c), + RecoverResult::Valid(pubkey) => { + context.write_memory(pubkey_data, &pubkey.serialize()[1..65]) + .map_err(|_| "Invalid attempt to set pubkey in ext_secp256k1_ecdsa_recover")?; + Ok(0) + } + } + } + + ext_secp256k1_ecdsa_recover_compressed( + msg_data: Pointer, + sig_data: Pointer, + pubkey_data: Pointer, + ) -> u32 { + match secp256k1_recover(context, msg_data, sig_data)? { + RecoverResult::Invalid(c) => Ok(c), + RecoverResult::Valid(pubkey) => { + context.write_memory(pubkey_data, &pubkey.serialize_compressed()[..]) + .map_err(|_| "Invalid attempt to set pubkey in ext_secp256k1_ecdsa_recover")?; + Ok(0) + } + } + } + + ext_is_validator() -> u32 { + if runtime_io::offchain::is_validator() { Ok(1) } else { Ok(0) } + } + + ext_submit_transaction(msg_data: Pointer, len: WordSize) -> u32 { + let extrinsic = context.read_memory(msg_data, len) + .map_err(|_| "OOB while ext_submit_transaction: wasm")?; + + let res = runtime_io::offchain::submit_transaction(extrinsic); + + Ok(if res.is_ok() { 0 } else { 1 }) + } + + ext_network_state(written_out: Pointer) -> Pointer { + let res = runtime_io::offchain::network_state(); + + let encoded = res.encode(); + let len = encoded.len() as u32; + let offset = context.allocate_memory(len)?; + context.write_memory(offset, &encoded) + .map_err(|_| "Invalid attempt to set memory in ext_network_state")?; + + context.write_primitive(written_out, len) + .map_err(|_| "Invalid attempt to write written_out in ext_network_state")?; + + Ok(offset) + } + + ext_timestamp() -> u64 { + Ok(runtime_io::offchain::timestamp().unix_millis()) + } + + ext_sleep_until(deadline: u64) { + runtime_io::offchain::sleep_until(offchain::Timestamp::from_unix_millis(deadline)); + Ok(()) + } + + ext_random_seed(seed_data: Pointer) { + // NOTE the runtime as assumptions about seed size. + let seed = runtime_io::offchain::random_seed(); + + context.write_memory(seed_data, &seed) + .map_err(|_| "Invalid attempt to set value in ext_random_seed")?; + Ok(()) + } + + ext_local_storage_set( + kind: u32, + key: Pointer, + key_len: WordSize, + value: Pointer, + value_len: WordSize, + ) { + let kind = offchain::StorageKind::try_from(kind) + .map_err(|_| "storage kind OOB while ext_local_storage_set: wasm")?; + let key = context.read_memory(key, key_len) + .map_err(|_| "OOB while ext_local_storage_set: wasm")?; + let value = context.read_memory(value, value_len) + .map_err(|_| "OOB while ext_local_storage_set: wasm")?; + + runtime_io::offchain::local_storage_set(kind, &key, &value); + + Ok(()) + } + + ext_local_storage_get( + kind: u32, + key: Pointer, + key_len: WordSize, + value_len: Pointer, + ) -> Pointer { + let kind = offchain::StorageKind::try_from(kind) + .map_err(|_| "storage kind OOB while ext_local_storage_get: wasm")?; + let key = context.read_memory(key, key_len) + .map_err(|_| "OOB while ext_local_storage_get: wasm")?; + + let maybe_value = runtime_io::offchain::local_storage_get(kind, &key); + + let (offset, len) = if let Some(value) = maybe_value { + let offset = context.allocate_memory(value.len() as u32)?; + context.write_memory(offset, &value) + .map_err(|_| "Invalid attempt to set memory in ext_local_storage_get")?; + (offset, value.len() as u32) + } else { + (Pointer::null(), u32::max_value()) + }; + + context.write_primitive(value_len, len) + .map_err(|_| "Invalid attempt to write value_len in ext_local_storage_get")?; + + Ok(offset) + } + + ext_local_storage_compare_and_set( + kind: u32, + key: Pointer, + key_len: WordSize, + old_value: Pointer, + old_value_len: WordSize, + new_value: Pointer, + new_value_len: WordSize, + ) -> u32 { + let kind = offchain::StorageKind::try_from(kind) + .map_err(|_| "storage kind OOB while ext_local_storage_compare_and_set: wasm")?; + let key = context.read_memory(key, key_len) + .map_err(|_| "OOB while ext_local_storage_compare_and_set: wasm")?; + let new_value = context.read_memory(new_value, new_value_len) + .map_err(|_| "OOB while ext_local_storage_compare_and_set: wasm")?; + + let old_value = if old_value_len == u32::max_value() { + None + } else { + Some( + context.read_memory(old_value, old_value_len) + .map_err(|_| "OOB while ext_local_storage_compare_and_set: wasm")? + ) + }; + + let res = runtime_io::offchain::local_storage_compare_and_set( + kind, + &key, + old_value, + &new_value, + ); + + Ok(if res { 0 } else { 1 }) + } + + ext_http_request_start( + method: Pointer, + method_len: WordSize, + url: Pointer, + url_len: WordSize, + meta: Pointer, + meta_len: WordSize, + ) -> u32 { + let method = context.read_memory(method, method_len) + .map_err(|_| "OOB while ext_http_request_start: wasm")?; + let url = context.read_memory(url, url_len) + .map_err(|_| "OOB while ext_http_request_start: wasm")?; + let meta = context.read_memory(meta, meta_len) + .map_err(|_| "OOB while ext_http_request_start: wasm")?; + + let method_str = str::from_utf8(&method) + .map_err(|_| "invalid str while ext_http_request_start: wasm")?; + let url_str = str::from_utf8(&url) + .map_err(|_| "invalid str while ext_http_request_start: wasm")?; + + let id = runtime_io::offchain::http_request_start(method_str, url_str, &meta); + + if let Ok(id) = id { + Ok(id.into()) + } else { + Ok(u32::max_value()) + } + } + + ext_http_request_add_header( + request_id: u32, + name: Pointer, + name_len: WordSize, + value: Pointer, + value_len: WordSize, + ) -> u32 { + let name = context.read_memory(name, name_len) + .map_err(|_| "OOB while ext_http_request_add_header: wasm")?; + let value = context.read_memory(value, value_len) + .map_err(|_| "OOB while ext_http_request_add_header: wasm")?; + + let name_str = str::from_utf8(&name) + .map_err(|_| "Invalid str while ext_http_request_add_header: wasm")?; + let value_str = str::from_utf8(&value) + .map_err(|_| "Invalid str while ext_http_request_add_header: wasm")?; + + let res = runtime_io::offchain::http_request_add_header( + offchain::HttpRequestId(request_id as u16), + name_str, + value_str, + ); + + Ok(if res.is_ok() { 0 } else { 1 }) + } + + ext_http_request_write_body( + request_id: u32, + chunk: Pointer, + chunk_len: WordSize, + deadline: u64, + ) -> u32 { + let chunk = context.read_memory(chunk, chunk_len) + .map_err(|_| "OOB while ext_http_request_write_body: wasm")?; + + let res = runtime_io::offchain::http_request_write_body( + offchain::HttpRequestId(request_id as u16), + &chunk, + deadline_to_timestamp(deadline), + ); + + Ok(match res { + Ok(()) => 0, + Err(e) => e.into(), + }) + } + + ext_http_response_wait( + ids: Pointer, + ids_len: WordSize, + statuses: Pointer, + deadline: u64, + ) { + let ids = (0..ids_len) + .map(|i| + context.read_primitive(ids.offset(i).ok_or("Point overflow")?) + .map(|id: u32| offchain::HttpRequestId(id as u16)) + .map_err(|_| "OOB while ext_http_response_wait: wasm") + ) + .collect::, _>>()?; + + let res = runtime_io::offchain::http_response_wait(&ids, deadline_to_timestamp(deadline)) + .into_iter() + .map(|status| u32::from(status)) + .enumerate() + // make sure to take up to `ids_len` to avoid exceeding the mem. + .take(ids_len as usize); + + for (i, status) in res { + context.write_primitive(statuses.offset(i as u32).ok_or("Point overflow")?, status) + .map_err(|_| "Invalid attempt to set memory in ext_http_response_wait")?; + } + + Ok(()) + } + + ext_http_response_headers( + request_id: u32, + written_out: Pointer, + ) -> Pointer { + use codec::Encode; + + let headers = runtime_io::offchain::http_response_headers( + offchain::HttpRequestId(request_id as u16), + ); + + let encoded = headers.encode(); + let len = encoded.len() as u32; + let offset = context.allocate_memory(len)?; + + context.write_memory(offset, &encoded) + .map_err(|_| "Invalid attempt to set memory in ext_http_response_headers")?; + context.write_primitive(written_out, len) + .map_err(|_| "Invalid attempt to write written_out in ext_http_response_headers")?; + + Ok(offset) + } + + ext_http_response_read_body( + request_id: u32, + buffer: Pointer, + buffer_len: WordSize, + deadline: u64, + ) -> WordSize { + let mut internal_buffer = Vec::with_capacity(buffer_len as usize); + internal_buffer.resize(buffer_len as usize, 0); + + let res = runtime_io::offchain::http_response_read_body( + offchain::HttpRequestId(request_id as u16), + &mut internal_buffer, + deadline_to_timestamp(deadline), + ); + + Ok(match res { + Ok(read) => { + context.write_memory(buffer, &internal_buffer[..read as usize]) + .map_err(|_| "Invalid attempt to set memory in ext_http_response_read_body")?; + + read as u32 + }, + Err(err) => { + u32::max_value() - u32::from(err) + 1 + } + }) + } + } +} + +fn deadline_to_timestamp(deadline: u64) -> Option { + if deadline == 0 { + None + } else { + Some(offchain::Timestamp::from_unix_millis(deadline)) + } +} + diff --git a/core/executor/src/error.rs b/client/executor/src/error.rs similarity index 81% rename from core/executor/src/error.rs rename to client/executor/src/error.rs index e1221bea54aef..58a84e09902e9 100644 --- a/core/executor/src/error.rs +++ b/client/executor/src/error.rs @@ -18,6 +18,8 @@ use serializer; use wasmi; +#[cfg(feature = "wasmtime")] +use wasmtime_jit::{ActionError, SetupError}; /// Result type alias. pub type Result = std::result::Result; @@ -31,6 +33,9 @@ pub enum Error { Trap(wasmi::Trap), /// Wasmi loading/instantiating error Wasmi(wasmi::Error), + /// Wasmtime action error + #[cfg(feature = "wasmtime")] + Wasmtime(ActionError), /// Error in the API. Parameter is an error message. ApiError(String), /// Method is not found @@ -75,9 +80,9 @@ pub enum Error { /// Someone tried to allocate more memory than the allowed maximum per allocation. #[display(fmt="Requested allocation size is too large")] RequestedAllocationTooLarge, - /// Executing the given function failed with the given error. - #[display(fmt="Function execution failed with: {}", _0)] - FunctionExecution(String), + /// Execution of a host function failed. + #[display(fmt="Host function {} execution failed with: {}", _0, _1)] + FunctionExecution(String, String), } impl std::error::Error for Error { @@ -112,10 +117,24 @@ pub enum WasmError { CodeNotFound, /// Failure to reinitialize runtime instance from snapshot. ApplySnapshotFailed, + /// Failure to erase the wasm memory. + /// + /// Depending on the implementation might mean failure of allocating memory. + ErasingFailed(String), /// Wasm code failed validation. InvalidModule, /// Wasm code could not be deserialized. CantDeserializeWasm, + /// The module does not export a linear memory named `memory`. + InvalidMemory, + /// The number of heap pages requested is disallowed by the module. + InvalidHeapPages, /// Instantiation error. - Instantiation(Error), + Instantiation(String), + /// The compiler does not support the host machine as a target. + #[cfg(feature = "wasmtime")] + MissingCompilerSupport(&'static str), + /// Wasmtime setup error. + #[cfg(feature = "wasmtime")] + WasmtimeSetup(SetupError), } diff --git a/core/executor/src/host_interface.rs b/client/executor/src/host_interface.rs similarity index 89% rename from core/executor/src/host_interface.rs rename to client/executor/src/host_interface.rs index 4d1515d76999b..0499cad5663ce 100644 --- a/core/executor/src/host_interface.rs +++ b/client/executor/src/host_interface.rs @@ -14,9 +14,7 @@ // You should have received a copy of the GNU General Public License // along with Substrate. If not, see . -//! Definition and implementation of the Substrate Wasm host interface. -//! -//! These are the host functions callable from within the Substrate runtime. +//! Definition and implementation of the old and deprecated Substrate runtime interface for the host. use codec::Encode; use std::{convert::TryFrom, str}; @@ -25,7 +23,9 @@ use primitives::{ crypto::KeyTypeId, offchain, }; use trie::{TrieConfiguration, trie_types::Layout}; -use wasm_interface::{FunctionContext, Pointer, PointerType, Result as WResult, WordSize}; +use wasm_interface::{ + Pointer, WordSize, WritePrimitive, ReadPrimitive, FunctionContext, Result as WResult, +}; #[cfg(feature="wasm-extern-trace")] macro_rules! debug_trace { @@ -37,6 +37,8 @@ macro_rules! debug_trace { ( $( $x:tt )* ) => () } +/// The old and deprecated Substrate externals. These are still required for backwards compatibility +/// reasons. pub struct SubstrateExternals; enum RecoverResult { @@ -166,20 +168,20 @@ impl_wasm_host_interface! { ext_print_utf8(utf8_data: Pointer, utf8_len: WordSize) { if let Ok(utf8) = context.read_memory(utf8_data, utf8_len) { - runtime_io::print_utf8(&utf8); + runtime_io::misc::print_utf8(&utf8); } Ok(()) } ext_print_hex(data: Pointer, len: WordSize) { if let Ok(hex) = context.read_memory(data, len) { - runtime_io::print_hex(&hex); + runtime_io::misc::print_hex(&hex); } Ok(()) } ext_print_num(number: u64) { - runtime_io::print_num(number); + runtime_io::misc::print_num(number); Ok(()) } @@ -195,7 +197,10 @@ impl_wasm_host_interface! { let message = context.read_memory(message_data, message_len) .map_err(|_| "Invalid attempt to determine message in ext_log")?; - runtime_io::log(level.into(), &target, &message); + let target_str = std::str::from_utf8(&target) + .map_err(|_| "Target invalid utf8 in ext_log")?; + + runtime_io::logging::log(level.into(), &target_str, &message); Ok(()) } @@ -209,7 +214,7 @@ impl_wasm_host_interface! { .map_err(|_| "Invalid attempt to determine key in ext_set_storage")?; let value = context.read_memory(value_data, value_len) .map_err(|_| "Invalid attempt to determine value in ext_set_storage")?; - Ok(runtime_io::set_storage(&key, &value)) + Ok(runtime_io::storage::set(&key, &value)) } ext_set_child_storage( @@ -227,7 +232,7 @@ impl_wasm_host_interface! { let value = context.read_memory(value_data, value_len) .map_err(|_| "Invalid attempt to determine value in ext_set_child_storage")?; - Ok(runtime_io::set_child_storage(&storage_key, &key, &value)) + Ok(runtime_io::storage::child_set(&storage_key, &key, &value)) } ext_clear_child_storage( @@ -241,19 +246,19 @@ impl_wasm_host_interface! { let key = context.read_memory(key_data, key_len) .map_err(|_| "Invalid attempt to determine key in ext_clear_child_storage")?; - Ok(runtime_io::clear_child_storage(&storage_key, &key)) + Ok(runtime_io::storage::child_clear(&storage_key, &key)) } ext_clear_storage(key_data: Pointer, key_len: WordSize) { let key = context.read_memory(key_data, key_len) .map_err(|_| "Invalid attempt to determine key in ext_clear_storage")?; - Ok(runtime_io::clear_storage(&key)) + Ok(runtime_io::storage::clear(&key)) } ext_exists_storage(key_data: Pointer, key_len: WordSize) -> u32 { let key = context.read_memory(key_data, key_len) .map_err(|_| "Invalid attempt to determine key in ext_exists_storage")?; - Ok(if runtime_io::exists_storage(&key) { 1 } else { 0 }) + Ok(if runtime_io::storage::exists(&key) { 1 } else { 0 }) } ext_exists_child_storage( @@ -267,13 +272,13 @@ impl_wasm_host_interface! { let key = context.read_memory(key_data, key_len) .map_err(|_| "Invalid attempt to determine key in ext_exists_child_storage")?; - Ok(if runtime_io::exists_child_storage(&storage_key, &key) { 1 } else { 0 }) + Ok(if runtime_io::storage::child_exists(&storage_key, &key) { 1 } else { 0 }) } ext_clear_prefix(prefix_data: Pointer, prefix_len: WordSize) { let prefix = context.read_memory(prefix_data, prefix_len) .map_err(|_| "Invalid attempt to determine prefix in ext_clear_prefix")?; - Ok(runtime_io::clear_prefix(&prefix)) + Ok(runtime_io::storage::clear_prefix(&prefix)) } ext_clear_child_prefix( @@ -286,13 +291,13 @@ impl_wasm_host_interface! { .map_err(|_| "Invalid attempt to determine storage_key in ext_clear_child_prefix")?; let prefix = context.read_memory(prefix_data, prefix_len) .map_err(|_| "Invalid attempt to determine prefix in ext_clear_child_prefix")?; - Ok(runtime_io::clear_child_prefix(&storage_key, &prefix)) + Ok(runtime_io::storage::child_clear_prefix(&storage_key, &prefix)) } ext_kill_child_storage(storage_key_data: Pointer, storage_key_len: WordSize) { let storage_key = context.read_memory(storage_key_data, storage_key_len) .map_err(|_| "Invalid attempt to determine storage_key in ext_kill_child_storage")?; - Ok(runtime_io::kill_child_storage(&storage_key)) + Ok(runtime_io::storage::child_storage_kill(&storage_key)) } ext_get_allocated_storage( @@ -303,7 +308,7 @@ impl_wasm_host_interface! { let key = context.read_memory(key_data, key_len) .map_err(|_| "Invalid attempt to determine key in ext_get_allocated_storage")?; - if let Some(value) = runtime_io::storage(&key) { + if let Some(value) = runtime_io::storage::get(&key) { let offset = context.allocate_memory(value.len() as u32)?; context.write_memory(offset, &value) .map_err(|_| "Invalid attempt to set memory in ext_get_allocated_storage")?; @@ -329,7 +334,7 @@ impl_wasm_host_interface! { let key = context.read_memory(key_data, key_len) .map_err(|_| "Invalid attempt to determine key in ext_get_allocated_child_storage")?; - if let Some(value) = runtime_io::child_storage(&storage_key, &key) { + if let Some(value) = runtime_io::storage::child_get(&storage_key, &key) { let offset = context.allocate_memory(value.len() as u32)?; context.write_memory(offset, &value) .map_err(|_| "Invalid attempt to set memory in ext_get_allocated_child_storage")?; @@ -353,7 +358,7 @@ impl_wasm_host_interface! { let key = context.read_memory(key_data, key_len) .map_err(|_| "Invalid attempt to get key in ext_get_storage_into")?; - if let Some(value) = runtime_io::storage(&key) { + if let Some(value) = runtime_io::storage::get(&key) { let data = &value[value.len().min(value_offset as usize)..]; let written = std::cmp::min(value_len as usize, data.len()); context.write_memory(value_data, &data[..written]) @@ -378,7 +383,7 @@ impl_wasm_host_interface! { let key = context.read_memory(key_data, key_len) .map_err(|_| "Invalid attempt to get key in ext_get_child_storage_into")?; - if let Some(value) = runtime_io::child_storage(&storage_key, &key) { + if let Some(value) = runtime_io::storage::child_get(&storage_key, &key) { let data = &value[value.len().min(value_offset as usize)..]; let written = std::cmp::min(value_len as usize, data.len()); context.write_memory(value_data, &data[..written]) @@ -390,7 +395,7 @@ impl_wasm_host_interface! { } ext_storage_root(result: Pointer) { - context.write_memory(result, runtime_io::storage_root().as_ref()) + context.write_memory(result, runtime_io::storage::root().as_ref()) .map_err(|_| "Invalid attempt to set memory in ext_storage_root".into()) } @@ -401,7 +406,7 @@ impl_wasm_host_interface! { ) -> Pointer { let storage_key = context.read_memory(storage_key_data, storage_key_len) .map_err(|_| "Invalid attempt to determine storage_key in ext_child_storage_root")?; - let value = runtime_io::child_storage_root(&storage_key); + let value = runtime_io::storage::child_root(&storage_key); let offset = context.allocate_memory(value.len() as u32)?; context.write_memory(offset, &value) @@ -420,7 +425,7 @@ impl_wasm_host_interface! { context.read_memory_into(parent_hash_data, &mut parent_hash[..]) .map_err(|_| "Invalid attempt to get parent_hash in ext_storage_changes_root")?; - if let Some(r) = runtime_io::storage_changes_root(parent_hash) { + if let Some(r) = runtime_io::storage::changes_root(parent_hash) { context.write_memory(result, &r[..]) .map_err(|_| "Invalid attempt to set memory in ext_storage_changes_root")?; Ok(1) @@ -454,7 +459,7 @@ impl_wasm_host_interface! { } ext_chain_id() -> u64 { - Ok(runtime_io::chain_id()) + Ok(runtime_io::misc::chain_id()) } ext_twox_64(data: Pointer, len: WordSize, out: Pointer) { @@ -550,7 +555,7 @@ impl_wasm_host_interface! { .map_err(|_| "Invalid attempt to get id in ext_ed25519_public_keys")?; let key_type = KeyTypeId(id); - let keys = runtime_io::ed25519_public_keys(key_type).encode(); + let keys = runtime_io::crypto::ed25519_public_keys(key_type).encode(); let len = keys.len() as u32; let offset = context.allocate_memory(len)?; @@ -605,13 +610,7 @@ impl_wasm_host_interface! { ) }; - let seed = seed.as_ref() - .map(|seed| - std::str::from_utf8(&seed) - .map_err(|_| "Seed not a valid utf8 string in ext_sr25119_generate") - ).transpose()?; - - let pubkey = runtime_io::ed25519_generate(key_type, seed); + let pubkey = runtime_io::crypto::ed25519_generate(key_type, seed); context.write_memory(out, pubkey.as_ref()) .map_err(|_| "Invalid attempt to set out in ext_ed25519_generate".into()) @@ -639,7 +638,7 @@ impl_wasm_host_interface! { let pub_key = ed25519::Public::try_from(pubkey.as_ref()) .map_err(|_| "Invalid `ed25519` public key")?; - let signature = runtime_io::ed25519_sign(key_type, &pub_key, &msg); + let signature = runtime_io::crypto::ed25519_sign(key_type, &pub_key, &msg); match signature { Some(signature) => { @@ -657,7 +656,7 @@ impl_wasm_host_interface! { .map_err(|_| "Invalid attempt to get id in ext_sr25519_public_keys")?; let key_type = KeyTypeId(id); - let keys = runtime_io::sr25519_public_keys(key_type).encode(); + let keys = runtime_io::crypto::sr25519_public_keys(key_type).encode(); let len = keys.len() as u32; let offset = context.allocate_memory(len)?; @@ -711,14 +710,7 @@ impl_wasm_host_interface! { ) }; - let seed = seed.as_ref() - .map(|seed| - std::str::from_utf8(&seed) - .map_err(|_| "Seed not a valid utf8 string in ext_sr25119_generate") - ) - .transpose()?; - - let pubkey = runtime_io::sr25519_generate(key_type, seed); + let pubkey = runtime_io::crypto::sr25519_generate(key_type, seed); context.write_memory(out, pubkey.as_ref()) .map_err(|_| "Invalid attempt to set out in ext_sr25519_generate".into()) @@ -746,7 +738,7 @@ impl_wasm_host_interface! { let pub_key = sr25519::Public::try_from(pubkey.as_ref()) .map_err(|_| "Invalid `sr25519` public key")?; - let signature = runtime_io::sr25519_sign(key_type, &pub_key, &msg); + let signature = runtime_io::crypto::sr25519_sign(key_type, &pub_key, &msg); match signature { Some(signature) => { @@ -789,20 +781,20 @@ impl_wasm_host_interface! { } ext_is_validator() -> u32 { - if runtime_io::is_validator() { Ok(1) } else { Ok(0) } + if runtime_io::offchain::is_validator() { Ok(1) } else { Ok(0) } } ext_submit_transaction(msg_data: Pointer, len: WordSize) -> u32 { let extrinsic = context.read_memory(msg_data, len) .map_err(|_| "OOB while ext_submit_transaction: wasm")?; - let res = runtime_io::submit_transaction(extrinsic); + let res = runtime_io::offchain::submit_transaction(extrinsic); Ok(if res.is_ok() { 0 } else { 1 }) } ext_network_state(written_out: Pointer) -> Pointer { - let res = runtime_io::network_state(); + let res = runtime_io::offchain::network_state(); let encoded = res.encode(); let len = encoded.len() as u32; @@ -817,17 +809,17 @@ impl_wasm_host_interface! { } ext_timestamp() -> u64 { - Ok(runtime_io::timestamp().unix_millis()) + Ok(runtime_io::offchain::timestamp().unix_millis()) } ext_sleep_until(deadline: u64) { - runtime_io::sleep_until(offchain::Timestamp::from_unix_millis(deadline)); + runtime_io::offchain::sleep_until(offchain::Timestamp::from_unix_millis(deadline)); Ok(()) } ext_random_seed(seed_data: Pointer) { // NOTE the runtime as assumptions about seed size. - let seed = runtime_io::random_seed(); + let seed = runtime_io::offchain::random_seed(); context.write_memory(seed_data, &seed) .map_err(|_| "Invalid attempt to set value in ext_random_seed")?; @@ -848,7 +840,7 @@ impl_wasm_host_interface! { let value = context.read_memory(value, value_len) .map_err(|_| "OOB while ext_local_storage_set: wasm")?; - runtime_io::local_storage_set(kind, &key, &value); + runtime_io::offchain::local_storage_set(kind, &key, &value); Ok(()) } @@ -864,7 +856,7 @@ impl_wasm_host_interface! { let key = context.read_memory(key, key_len) .map_err(|_| "OOB while ext_local_storage_get: wasm")?; - let maybe_value = runtime_io::local_storage_get(kind, &key); + let maybe_value = runtime_io::offchain::local_storage_get(kind, &key); let (offset, len) = if let Some(value) = maybe_value { let offset = context.allocate_memory(value.len() as u32)?; @@ -906,10 +898,10 @@ impl_wasm_host_interface! { ) }; - let res = runtime_io::local_storage_compare_and_set( + let res = runtime_io::offchain::local_storage_compare_and_set( kind, &key, - old_value.as_ref().map(|v| v.as_ref()), + old_value, &new_value, ); @@ -936,7 +928,7 @@ impl_wasm_host_interface! { let url_str = str::from_utf8(&url) .map_err(|_| "invalid str while ext_http_request_start: wasm")?; - let id = runtime_io::http_request_start(method_str, url_str, &meta); + let id = runtime_io::offchain::http_request_start(method_str, url_str, &meta); if let Ok(id) = id { Ok(id.into()) @@ -962,7 +954,7 @@ impl_wasm_host_interface! { let value_str = str::from_utf8(&value) .map_err(|_| "Invalid str while ext_http_request_add_header: wasm")?; - let res = runtime_io::http_request_add_header( + let res = runtime_io::offchain::http_request_add_header( offchain::HttpRequestId(request_id as u16), name_str, value_str, @@ -980,7 +972,7 @@ impl_wasm_host_interface! { let chunk = context.read_memory(chunk, chunk_len) .map_err(|_| "OOB while ext_http_request_write_body: wasm")?; - let res = runtime_io::http_request_write_body( + let res = runtime_io::offchain::http_request_write_body( offchain::HttpRequestId(request_id as u16), &chunk, deadline_to_timestamp(deadline), @@ -1006,7 +998,7 @@ impl_wasm_host_interface! { ) .collect::, _>>()?; - let res = runtime_io::http_response_wait(&ids, deadline_to_timestamp(deadline)) + let res = runtime_io::offchain::http_response_wait(&ids, deadline_to_timestamp(deadline)) .into_iter() .map(|status| u32::from(status)) .enumerate() @@ -1027,7 +1019,9 @@ impl_wasm_host_interface! { ) -> Pointer { use codec::Encode; - let headers = runtime_io::http_response_headers(offchain::HttpRequestId(request_id as u16)); + let headers = runtime_io::offchain::http_response_headers( + offchain::HttpRequestId(request_id as u16), + ); let encoded = headers.encode(); let len = encoded.len() as u32; @@ -1050,7 +1044,7 @@ impl_wasm_host_interface! { let mut internal_buffer = Vec::with_capacity(buffer_len as usize); internal_buffer.resize(buffer_len as usize, 0); - let res = runtime_io::http_response_read_body( + let res = runtime_io::offchain::http_response_read_body( offchain::HttpRequestId(request_id as u16), &mut internal_buffer, deadline_to_timestamp(deadline), @@ -1058,7 +1052,7 @@ impl_wasm_host_interface! { Ok(match res { Ok(read) => { - context.write_memory(buffer, &internal_buffer[..read]) + context.write_memory(buffer, &internal_buffer[..read as usize]) .map_err(|_| "Invalid attempt to set memory in ext_http_response_read_body")?; read as u32 @@ -1071,29 +1065,6 @@ impl_wasm_host_interface! { } } -trait WritePrimitive { - fn write_primitive(&mut self, ptr: Pointer, t: T) -> WResult<()>; -} - -impl WritePrimitive for &mut dyn FunctionContext { - fn write_primitive(&mut self, ptr: Pointer, t: u32) -> WResult<()> { - let r = t.to_le_bytes(); - self.write_memory(ptr.cast(), &r) - } -} - -trait ReadPrimitive { - fn read_primitive(&self, offset: Pointer) -> WResult; -} - -impl ReadPrimitive for &mut dyn FunctionContext { - fn read_primitive(&self, ptr: Pointer) -> WResult { - let mut r = [0u8; 4]; - self.read_memory_into(ptr.cast(), &mut r)?; - Ok(u32::from_le_bytes(r)) - } -} - fn deadline_to_timestamp(deadline: u64) -> Option { if deadline == 0 { None diff --git a/client/executor/src/integration_tests/mod.rs b/client/executor/src/integration_tests/mod.rs new file mode 100644 index 0000000000000..71df0b8d8cc9e --- /dev/null +++ b/client/executor/src/integration_tests/mod.rs @@ -0,0 +1,459 @@ +// Copyright 2017-2019 Parity Technologies (UK) Ltd. +// This file is part of Substrate. + +// Substrate is free software: you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. + +// Substrate is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. + +// You should have received a copy of the GNU General Public License +// along with Substrate. If not, see . + +mod sandbox; + +use codec::{Encode, Decode}; +use hex_literal::hex; +use primitives::{ + Blake2Hasher, blake2_128, blake2_256, ed25519, sr25519, map, Pair, offchain::OffchainExt, + traits::Externalities, +}; +use runtime_test::WASM_BINARY; +use state_machine::TestExternalities as CoreTestExternalities; +use substrate_offchain::testing; +use test_case::test_case; +use trie::{TrieConfiguration, trie_types::Layout}; + +use crate::WasmExecutionMethod; + +pub type TestExternalities = CoreTestExternalities; + +fn call_in_wasm( + function: &str, + call_data: &[u8], + execution_method: WasmExecutionMethod, + ext: &mut E, + code: &[u8], + heap_pages: u64, +) -> crate::error::Result> { + crate::call_in_wasm::( + function, + call_data, + execution_method, + ext, + code, + heap_pages, + ) +} + +#[test_case(WasmExecutionMethod::Interpreted)] +#[cfg_attr(feature = "wasmtime", test_case(WasmExecutionMethod::Compiled))] +fn returning_should_work(wasm_method: WasmExecutionMethod) { + let mut ext = TestExternalities::default(); + let mut ext = ext.ext(); + let test_code = WASM_BINARY; + + let output = call_in_wasm( + "test_empty_return", + &[], + wasm_method, + &mut ext, + &test_code[..], + 8, + ).unwrap(); + assert_eq!(output, vec![0u8; 0]); +} + +#[test_case(WasmExecutionMethod::Interpreted)] +#[cfg_attr(feature = "wasmtime", test_case(WasmExecutionMethod::Compiled))] +fn panicking_should_work(wasm_method: WasmExecutionMethod) { + let mut ext = TestExternalities::default(); + let mut ext = ext.ext(); + let test_code = WASM_BINARY; + + let output = call_in_wasm( + "test_panic", + &[], + wasm_method, + &mut ext, + &test_code[..], + 8, + ); + assert!(output.is_err()); + + let output = call_in_wasm( + "test_conditional_panic", + &[0], + wasm_method, + &mut ext, + &test_code[..], + 8, + ); + assert_eq!(Decode::decode(&mut &output.unwrap()[..]), Ok(Vec::::new())); + + let output = call_in_wasm( + "test_conditional_panic", + &vec![2].encode(), + wasm_method, + &mut ext, + &test_code[..], + 8, + ); + assert!(output.is_err()); +} + +#[test_case(WasmExecutionMethod::Interpreted)] +#[cfg_attr(feature = "wasmtime", test_case(WasmExecutionMethod::Compiled))] +fn storage_should_work(wasm_method: WasmExecutionMethod) { + let mut ext = TestExternalities::default(); + + { + let mut ext = ext.ext(); + ext.set_storage(b"foo".to_vec(), b"bar".to_vec()); + let test_code = WASM_BINARY; + + let output = call_in_wasm( + "test_data_in", + &b"Hello world".to_vec().encode(), + wasm_method, + &mut ext, + &test_code[..], + 8, + ).unwrap(); + + assert_eq!(output, b"all ok!".to_vec().encode()); + } + + let expected = TestExternalities::new((map![ + b"input".to_vec() => b"Hello world".to_vec(), + b"foo".to_vec() => b"bar".to_vec(), + b"baz".to_vec() => b"bar".to_vec() + ], map![])); + assert_eq!(ext, expected); +} + +#[test_case(WasmExecutionMethod::Interpreted)] +#[cfg_attr(feature = "wasmtime", test_case(WasmExecutionMethod::Compiled))] +fn clear_prefix_should_work(wasm_method: WasmExecutionMethod) { + let mut ext = TestExternalities::default(); + { + let mut ext = ext.ext(); + ext.set_storage(b"aaa".to_vec(), b"1".to_vec()); + ext.set_storage(b"aab".to_vec(), b"2".to_vec()); + ext.set_storage(b"aba".to_vec(), b"3".to_vec()); + ext.set_storage(b"abb".to_vec(), b"4".to_vec()); + ext.set_storage(b"bbb".to_vec(), b"5".to_vec()); + let test_code = WASM_BINARY; + + // This will clear all entries which prefix is "ab". + let output = call_in_wasm( + "test_clear_prefix", + &b"ab".to_vec().encode(), + wasm_method, + &mut ext, + &test_code[..], + 8, + ).unwrap(); + + assert_eq!(output, b"all ok!".to_vec().encode()); + } + + let expected = TestExternalities::new((map![ + b"aaa".to_vec() => b"1".to_vec(), + b"aab".to_vec() => b"2".to_vec(), + b"bbb".to_vec() => b"5".to_vec() + ], map![])); + assert_eq!(expected, ext); +} + +#[test_case(WasmExecutionMethod::Interpreted)] +#[cfg_attr(feature = "wasmtime", test_case(WasmExecutionMethod::Compiled))] +fn blake2_256_should_work(wasm_method: WasmExecutionMethod) { + let mut ext = TestExternalities::default(); + let mut ext = ext.ext(); + let test_code = WASM_BINARY; + assert_eq!( + call_in_wasm( + "test_blake2_256", + &[0], + wasm_method, + &mut ext, + &test_code[..], + 8, + ).unwrap(), + blake2_256(&b""[..]).to_vec().encode(), + ); + assert_eq!( + call_in_wasm( + "test_blake2_256", + &b"Hello world!".to_vec().encode(), + wasm_method, + &mut ext, + &test_code[..], + 8, + ).unwrap(), + blake2_256(&b"Hello world!"[..]).to_vec().encode(), + ); +} + +#[test_case(WasmExecutionMethod::Interpreted)] +#[cfg_attr(feature = "wasmtime", test_case(WasmExecutionMethod::Compiled))] +fn blake2_128_should_work(wasm_method: WasmExecutionMethod) { + let mut ext = TestExternalities::default(); + let mut ext = ext.ext(); + let test_code = WASM_BINARY; + assert_eq!( + call_in_wasm( + "test_blake2_128", + &[0], + wasm_method, + &mut ext, + &test_code[..], + 8, + ).unwrap(), + blake2_128(&b""[..]).to_vec().encode(), + ); + assert_eq!( + call_in_wasm( + "test_blake2_128", + &b"Hello world!".to_vec().encode(), + wasm_method, + &mut ext, + &test_code[..], + 8, + ).unwrap(), + blake2_128(&b"Hello world!"[..]).to_vec().encode(), + ); +} + +#[test_case(WasmExecutionMethod::Interpreted)] +#[cfg_attr(feature = "wasmtime", test_case(WasmExecutionMethod::Compiled))] +fn twox_256_should_work(wasm_method: WasmExecutionMethod) { + let mut ext = TestExternalities::default(); + let mut ext = ext.ext(); + let test_code = WASM_BINARY; + assert_eq!( + call_in_wasm( + "test_twox_256", + &[0], + wasm_method, + &mut ext, + &test_code[..], + 8, + ).unwrap(), + hex!( + "99e9d85137db46ef4bbea33613baafd56f963c64b1f3685a4eb4abd67ff6203a" + ).to_vec().encode(), + ); + assert_eq!( + call_in_wasm( + "test_twox_256", + &b"Hello world!".to_vec().encode(), + wasm_method, + &mut ext, + &test_code[..], + 8, + ).unwrap(), + hex!( + "b27dfd7f223f177f2a13647b533599af0c07f68bda23d96d059da2b451a35a74" + ).to_vec().encode(), + ); +} + +#[test_case(WasmExecutionMethod::Interpreted)] +#[cfg_attr(feature = "wasmtime", test_case(WasmExecutionMethod::Compiled))] +fn twox_128_should_work(wasm_method: WasmExecutionMethod) { + let mut ext = TestExternalities::default(); + let mut ext = ext.ext(); + let test_code = WASM_BINARY; + assert_eq!( + call_in_wasm( + "test_twox_128", + &[0], + wasm_method, + &mut ext, + &test_code[..], + 8, + ).unwrap(), + hex!("99e9d85137db46ef4bbea33613baafd5").to_vec().encode(), + ); + assert_eq!( + call_in_wasm( + "test_twox_128", + &b"Hello world!".to_vec().encode(), + wasm_method, + &mut ext, + &test_code[..], + 8, + ).unwrap(), + hex!("b27dfd7f223f177f2a13647b533599af").to_vec().encode(), + ); +} + +#[test_case(WasmExecutionMethod::Interpreted)] +#[cfg_attr(feature = "wasmtime", test_case(WasmExecutionMethod::Compiled))] +fn ed25519_verify_should_work(wasm_method: WasmExecutionMethod) { + let mut ext = TestExternalities::default(); + let mut ext = ext.ext(); + let test_code = WASM_BINARY; + let key = ed25519::Pair::from_seed(&blake2_256(b"test")); + let sig = key.sign(b"all ok!"); + let mut calldata = vec![]; + calldata.extend_from_slice(key.public().as_ref()); + calldata.extend_from_slice(sig.as_ref()); + + assert_eq!( + call_in_wasm( + "test_ed25519_verify", + &calldata.encode(), + wasm_method, + &mut ext, + &test_code[..], + 8, + ).unwrap(), + true.encode(), + ); + + let other_sig = key.sign(b"all is not ok!"); + let mut calldata = vec![]; + calldata.extend_from_slice(key.public().as_ref()); + calldata.extend_from_slice(other_sig.as_ref()); + + assert_eq!( + call_in_wasm( + "test_ed25519_verify", + &calldata.encode(), + wasm_method, + &mut ext, + &test_code[..], + 8, + ).unwrap(), + false.encode(), + ); +} + +#[test_case(WasmExecutionMethod::Interpreted)] +#[cfg_attr(feature = "wasmtime", test_case(WasmExecutionMethod::Compiled))] +fn sr25519_verify_should_work(wasm_method: WasmExecutionMethod) { + let mut ext = TestExternalities::default(); + let mut ext = ext.ext(); + let test_code = WASM_BINARY; + let key = sr25519::Pair::from_seed(&blake2_256(b"test")); + let sig = key.sign(b"all ok!"); + let mut calldata = vec![]; + calldata.extend_from_slice(key.public().as_ref()); + calldata.extend_from_slice(sig.as_ref()); + + assert_eq!( + call_in_wasm( + "test_sr25519_verify", + &calldata.encode(), + wasm_method, + &mut ext, + &test_code[..], + 8, + ).unwrap(), + true.encode(), + ); + + let other_sig = key.sign(b"all is not ok!"); + let mut calldata = vec![]; + calldata.extend_from_slice(key.public().as_ref()); + calldata.extend_from_slice(other_sig.as_ref()); + + assert_eq!( + call_in_wasm( + "test_sr25519_verify", + &calldata.encode(), + wasm_method, + &mut ext, + &test_code[..], + 8, + ).unwrap(), + false.encode(), + ); +} + +#[test_case(WasmExecutionMethod::Interpreted)] +#[cfg_attr(feature = "wasmtime", test_case(WasmExecutionMethod::Compiled))] +fn ordered_trie_root_should_work(wasm_method: WasmExecutionMethod) { + let mut ext = TestExternalities::default(); + let mut ext = ext.ext(); + let trie_input = vec![b"zero".to_vec(), b"one".to_vec(), b"two".to_vec()]; + let test_code = WASM_BINARY; + assert_eq!( + call_in_wasm( + "test_ordered_trie_root", + &[0], + wasm_method, + &mut ext, + &test_code[..], + 8, + ).unwrap(), + Layout::::ordered_trie_root(trie_input.iter()).as_bytes().encode(), + ); +} + +#[test_case(WasmExecutionMethod::Interpreted)] +#[cfg_attr(feature = "wasmtime", test_case(WasmExecutionMethod::Compiled))] +fn offchain_local_storage_should_work(wasm_method: WasmExecutionMethod) { + use client_api::OffchainStorage; + + let mut ext = TestExternalities::default(); + let (offchain, state) = testing::TestOffchainExt::new(); + ext.register_extension(OffchainExt::new(offchain)); + let test_code = WASM_BINARY; + let mut ext = ext.ext(); + assert_eq!( + call_in_wasm( + "test_offchain_local_storage", + &[0], + wasm_method, + &mut ext, + &test_code[..], + 8, + ).unwrap(), + true.encode(), + ); + assert_eq!(state.read().persistent_storage.get(b"", b"test"), Some(vec![])); +} + +#[test_case(WasmExecutionMethod::Interpreted)] +#[cfg_attr(feature = "wasmtime", test_case(WasmExecutionMethod::Compiled))] +fn offchain_http_should_work(wasm_method: WasmExecutionMethod) { + let mut ext = TestExternalities::default(); + let (offchain, state) = testing::TestOffchainExt::new(); + ext.register_extension(OffchainExt::new(offchain)); + state.write().expect_request( + 0, + testing::PendingRequest { + method: "POST".into(), + uri: "http://localhost:12345".into(), + body: vec![1, 2, 3, 4], + headers: vec![("X-Auth".to_owned(), "test".to_owned())], + sent: true, + response: Some(vec![1, 2, 3]), + response_headers: vec![("X-Auth".to_owned(), "hello".to_owned())], + ..Default::default() + }, + ); + + let test_code = WASM_BINARY; + let mut ext = ext.ext(); + assert_eq!( + call_in_wasm( + "test_offchain_http", + &[0], + wasm_method, + &mut ext, + &test_code[..], + 8, + ).unwrap(), + true.encode(), + ); +} + diff --git a/client/executor/src/integration_tests/sandbox.rs b/client/executor/src/integration_tests/sandbox.rs new file mode 100644 index 0000000000000..c18b848acce7e --- /dev/null +++ b/client/executor/src/integration_tests/sandbox.rs @@ -0,0 +1,360 @@ +// Copyright 2018-2019 Parity Technologies (UK) Ltd. +// This file is part of Substrate. + +// Substrate is free software: you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. + +// Substrate is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. + +// You should have received a copy of the GNU General Public License +// along with Substrate. If not, see . + +use super::{TestExternalities, call_in_wasm}; +use crate::WasmExecutionMethod; + +use codec::Encode; +use runtime_test::WASM_BINARY; +use test_case::test_case; +use wabt; + +#[test_case(WasmExecutionMethod::Interpreted)] +#[cfg_attr(feature = "wasmtime", test_case(WasmExecutionMethod::Compiled))] +fn sandbox_should_work(wasm_method: WasmExecutionMethod) { + let mut ext = TestExternalities::default(); + let mut ext = ext.ext(); + let test_code = WASM_BINARY; + + let code = wabt::wat2wasm(r#" + (module + (import "env" "assert" (func $assert (param i32))) + (import "env" "inc_counter" (func $inc_counter (param i32) (result i32))) + (func (export "call") + (drop + (call $inc_counter (i32.const 5)) + ) + + (call $inc_counter (i32.const 3)) + ;; current counter value is on the stack + + ;; check whether current == 8 + i32.const 8 + i32.eq + + call $assert + ) + ) + "#).unwrap().encode(); + + assert_eq!( + call_in_wasm( + "test_sandbox", + &code, + wasm_method, + &mut ext, + &test_code[..], + 8, + ).unwrap(), + true.encode(), + ); +} + +#[test_case(WasmExecutionMethod::Interpreted)] +#[cfg_attr(feature = "wasmtime", test_case(WasmExecutionMethod::Compiled))] +fn sandbox_trap(wasm_method: WasmExecutionMethod) { + let mut ext = TestExternalities::default(); + let mut ext = ext.ext(); + let test_code = WASM_BINARY; + + let code = wabt::wat2wasm(r#" + (module + (import "env" "assert" (func $assert (param i32))) + (func (export "call") + i32.const 0 + call $assert + ) + ) + "#).unwrap(); + + assert_eq!( + call_in_wasm( + "test_sandbox", + &code, + wasm_method, + &mut ext, + &test_code[..], + 8, + ).unwrap(), + vec![0], + ); +} + +#[test_case(WasmExecutionMethod::Interpreted)] +#[cfg_attr(feature = "wasmtime", test_case(WasmExecutionMethod::Compiled))] +#[should_panic(expected = "Allocator ran out of space")] +fn sandbox_should_trap_when_heap_exhausted(wasm_method: WasmExecutionMethod) { + let mut ext = TestExternalities::default(); + let mut ext = ext.ext(); + let test_code = WASM_BINARY; + + let code = wabt::wat2wasm(r#" + (module + (import "env" "assert" (func $assert (param i32))) + (func (export "call") + i32.const 0 + call $assert + ) + ) + "#).unwrap().encode(); + + call_in_wasm( + "test_exhaust_heap", + &code, + wasm_method, + &mut ext, + &test_code[..], + 8, + ).unwrap(); +} + +#[test_case(WasmExecutionMethod::Interpreted)] +#[cfg_attr(feature = "wasmtime", test_case(WasmExecutionMethod::Compiled))] +fn start_called(wasm_method: WasmExecutionMethod) { + let mut ext = TestExternalities::default(); + let mut ext = ext.ext(); + let test_code = WASM_BINARY; + + let code = wabt::wat2wasm(r#" + (module + (import "env" "assert" (func $assert (param i32))) + (import "env" "inc_counter" (func $inc_counter (param i32) (result i32))) + + ;; Start function + (start $start) + (func $start + ;; Increment counter by 1 + (drop + (call $inc_counter (i32.const 1)) + ) + ) + + (func (export "call") + ;; Increment counter by 1. The current value is placed on the stack. + (call $inc_counter (i32.const 1)) + + ;; Counter is incremented twice by 1, once there and once in `start` func. + ;; So check the returned value is equal to 2. + i32.const 2 + i32.eq + call $assert + ) + ) + "#).unwrap().encode(); + + assert_eq!( + call_in_wasm( + "test_sandbox", + &code, + wasm_method, + &mut ext, + &test_code[..], + 8, + ).unwrap(), + true.encode(), + ); +} + +#[test_case(WasmExecutionMethod::Interpreted)] +#[cfg_attr(feature = "wasmtime", test_case(WasmExecutionMethod::Compiled))] +fn invoke_args(wasm_method: WasmExecutionMethod) { + let mut ext = TestExternalities::default(); + let mut ext = ext.ext(); + let test_code = WASM_BINARY; + + let code = wabt::wat2wasm(r#" + (module + (import "env" "assert" (func $assert (param i32))) + + (func (export "call") (param $x i32) (param $y i64) + ;; assert that $x = 0x12345678 + (call $assert + (i32.eq + (get_local $x) + (i32.const 0x12345678) + ) + ) + + (call $assert + (i64.eq + (get_local $y) + (i64.const 0x1234567887654321) + ) + ) + ) + ) + "#).unwrap().encode(); + + assert_eq!( + call_in_wasm( + "test_sandbox_args", + &code, + wasm_method, + &mut ext, + &test_code[..], + 8, + ).unwrap(), + true.encode(), + ); +} + +#[test_case(WasmExecutionMethod::Interpreted)] +#[cfg_attr(feature = "wasmtime", test_case(WasmExecutionMethod::Compiled))] +fn return_val(wasm_method: WasmExecutionMethod) { + let mut ext = TestExternalities::default(); + let mut ext = ext.ext(); + let test_code = WASM_BINARY; + + let code = wabt::wat2wasm(r#" + (module + (func (export "call") (param $x i32) (result i32) + (i32.add + (get_local $x) + (i32.const 1) + ) + ) + ) + "#).unwrap().encode(); + + assert_eq!( + call_in_wasm( + "test_sandbox_return_val", + &code, + wasm_method, + &mut ext, + &test_code[..], + 8, + ).unwrap(), + true.encode(), + ); +} + +#[test_case(WasmExecutionMethod::Interpreted)] +#[cfg_attr(feature = "wasmtime", test_case(WasmExecutionMethod::Compiled))] +fn unlinkable_module(wasm_method: WasmExecutionMethod) { + let mut ext = TestExternalities::default(); + let mut ext = ext.ext(); + let test_code = WASM_BINARY; + + let code = wabt::wat2wasm(r#" + (module + (import "env" "non-existent" (func)) + + (func (export "call") + ) + ) + "#).unwrap().encode(); + + assert_eq!( + call_in_wasm( + "test_sandbox_instantiate", + &code, + wasm_method, + &mut ext, + &test_code[..], + 8, + ).unwrap(), + 1u8.encode(), + ); +} + +#[test_case(WasmExecutionMethod::Interpreted)] +#[cfg_attr(feature = "wasmtime", test_case(WasmExecutionMethod::Compiled))] +fn corrupted_module(wasm_method: WasmExecutionMethod) { + let mut ext = TestExternalities::default(); + let mut ext = ext.ext(); + let test_code = WASM_BINARY; + + // Corrupted wasm file + let code = vec![0u8, 0, 0, 0, 1, 0, 0, 0].encode(); + + assert_eq!( + call_in_wasm( + "test_sandbox_instantiate", + &code, + wasm_method, + &mut ext, + &test_code[..], + 8, + ).unwrap(), + 1u8.encode(), + ); +} + +#[test_case(WasmExecutionMethod::Interpreted)] +#[cfg_attr(feature = "wasmtime", test_case(WasmExecutionMethod::Compiled))] +fn start_fn_ok(wasm_method: WasmExecutionMethod) { + let mut ext = TestExternalities::default(); + let mut ext = ext.ext(); + let test_code = WASM_BINARY; + + let code = wabt::wat2wasm(r#" + (module + (func (export "call") + ) + + (func $start + ) + + (start $start) + ) + "#).unwrap().encode(); + + assert_eq!( + call_in_wasm( + "test_sandbox_instantiate", + &code, + wasm_method, + &mut ext, + &test_code[..], + 8, + ).unwrap(), + 0u8.encode(), + ); +} + +#[test_case(WasmExecutionMethod::Interpreted)] +#[cfg_attr(feature = "wasmtime", test_case(WasmExecutionMethod::Compiled))] +fn start_fn_traps(wasm_method: WasmExecutionMethod) { + let mut ext = TestExternalities::default(); + let mut ext = ext.ext(); + let test_code = WASM_BINARY; + + let code = wabt::wat2wasm(r#" + (module + (func (export "call") + ) + + (func $start + unreachable + ) + + (start $start) + ) + "#).unwrap().encode(); + + assert_eq!( + call_in_wasm( + "test_sandbox_instantiate", + &code, + wasm_method, + &mut ext, + &test_code[..], + 8, + ).unwrap(), + 2u8.encode(), + ); +} diff --git a/core/executor/src/lib.rs b/client/executor/src/lib.rs similarity index 92% rename from core/executor/src/lib.rs rename to client/executor/src/lib.rs index ac98388cd7bbe..0638a71d1c873 100644 --- a/core/executor/src/lib.rs +++ b/client/executor/src/lib.rs @@ -36,8 +36,12 @@ mod wasmi_execution; mod native_executor; mod sandbox; mod allocator; -mod host_interface; +pub mod deprecated_host_interface; mod wasm_runtime; +#[cfg(feature = "wasmtime")] +mod wasmtime; +#[cfg(test)] +mod integration_tests; pub mod error; pub use wasmi; @@ -60,7 +64,7 @@ pub use wasm_runtime::WasmExecutionMethod; /// - `heap_pages`: The number of heap pages to allocate. /// /// Returns the `Vec` that contains the return value of the function. -pub fn call_in_wasm( +pub fn call_in_wasm( function: &str, call_data: &[u8], execution_method: WasmExecutionMethod, @@ -69,10 +73,10 @@ pub fn call_in_wasm( heap_pages: u64, ) -> error::Result> { let mut instance = wasm_runtime::create_wasm_runtime_with_code( - ext, execution_method, heap_pages, code, + HF::host_functions(), )?; instance.call(ext, function, call_data) } @@ -99,7 +103,7 @@ mod tests { fn call_in_interpreted_wasm_works() { let mut ext = TestExternalities::default(); let mut ext = ext.ext(); - let res = call_in_wasm( + let res = call_in_wasm::<_, runtime_io::SubstrateHostFunctions>( "test_empty_return", &[], WasmExecutionMethod::Interpreted, diff --git a/core/executor/src/native_executor.rs b/client/executor/src/native_executor.rs similarity index 88% rename from core/executor/src/native_executor.rs rename to client/executor/src/native_executor.rs index 082f0ba2adc33..3b33ee514dd39 100644 --- a/core/executor/src/native_executor.rs +++ b/client/executor/src/native_executor.rs @@ -14,15 +14,23 @@ // You should have received a copy of the GNU General Public License // along with Substrate. If not, see . -use std::{result, cell::RefCell, panic::{UnwindSafe, AssertUnwindSafe}}; -use crate::error::{Error, Result}; -use crate::wasm_runtime::{RuntimesCache, WasmExecutionMethod, WasmRuntime}; -use crate::RuntimeInfo; +use crate::{ + RuntimeInfo, error::{Error, Result}, + wasm_runtime::{RuntimesCache, WasmExecutionMethod, WasmRuntime}, +}; + use runtime_version::{NativeVersion, RuntimeVersion}; + use codec::{Decode, Encode}; + use primitives::{NativeOrEncoded, traits::{CodeExecutor, Externalities}}; + use log::{trace, warn}; +use std::{result, cell::RefCell, panic::{UnwindSafe, AssertUnwindSafe}}; + +use wasm_interface::{HostFunctions, Function}; + thread_local! { static RUNTIMES_CACHE: RefCell = RefCell::new(RuntimesCache::new()); } @@ -62,7 +70,6 @@ pub trait NativeExecutionDispatch: Send + Sync { /// A generic `CodeExecutor` implementation that uses a delegate to determine wasm code equivalence /// and dispatch to native code when possible, falling back on `WasmExecutor` when not. -#[derive(Debug)] pub struct NativeExecutor { /// Dummy field to avoid the compiler complaining about us not using `D`. _dummy: std::marker::PhantomData, @@ -72,6 +79,8 @@ pub struct NativeExecutor { native_version: NativeVersion, /// The number of 64KB pages to allocate for Wasm execution. default_heap_pages: u64, + /// The host functions registered with this instance. + host_functions: Vec<&'static dyn Function>, } impl NativeExecutor { @@ -84,11 +93,18 @@ impl NativeExecutor { /// `default_heap_pages` - Number of 64KB pages to allocate for Wasm execution. /// Defaults to `DEFAULT_HEAP_PAGES` if `None` is provided. pub fn new(fallback_method: WasmExecutionMethod, default_heap_pages: Option) -> Self { + let mut host_functions = runtime_io::SubstrateHostFunctions::host_functions(); + // Add the old and deprecated host functions as well, so that we support old wasm runtimes. + host_functions.extend( + crate::deprecated_host_interface::SubstrateExternals::host_functions(), + ); + NativeExecutor { _dummy: Default::default(), fallback_method, native_version: D::native_version(), default_heap_pages: default_heap_pages.unwrap_or(DEFAULT_HEAP_PAGES), + host_functions, } } @@ -110,21 +126,23 @@ impl NativeExecutor { ext: &mut E, f: impl for<'a> FnOnce( AssertUnwindSafe<&'a mut (dyn WasmRuntime + 'static)>, + &'a RuntimeVersion, AssertUnwindSafe<&'a mut E>, ) -> Result>, ) -> Result where E: Externalities { RUNTIMES_CACHE.with(|cache| { let mut cache = cache.borrow_mut(); - let (runtime, code_hash) = cache.fetch_runtime( + let (runtime, version, code_hash) = cache.fetch_runtime( ext, self.fallback_method, self.default_heap_pages, + &self.host_functions, )?; let runtime = AssertUnwindSafe(runtime); let ext = AssertUnwindSafe(ext); - match f(runtime, ext) { + match f(runtime, version, ext) { Ok(res) => res, Err(e) => { cache.invalidate_runtime(self.fallback_method, code_hash); @@ -142,6 +160,7 @@ impl Clone for NativeExecutor { fallback_method: self.fallback_method, native_version: D::native_version(), default_heap_pages: self.default_heap_pages, + host_functions: self.host_functions.clone(), } } } @@ -155,8 +174,8 @@ impl RuntimeInfo for NativeExecutor { &self, ext: &mut E, ) -> Option { - match self.with_runtime(ext, |runtime, _ext| Ok(Ok(runtime.version()))) { - Ok(version) => version, + match self.with_runtime(ext, |_runtime, version, _ext| Ok(Ok(version.clone()))) { + Ok(version) => Some(version), Err(e) => { warn!(target: "executor", "Failed to fetch runtime: {:?}", e); None @@ -182,13 +201,10 @@ impl CodeExecutor for NativeExecutor { native_call: Option, ) -> (Result>, bool){ let mut used_native = false; - let result = self.with_runtime(ext, |mut runtime, mut ext| { - let onchain_version = runtime.version(); + let result = self.with_runtime(ext, |mut runtime, onchain_version, mut ext| { match ( use_native, - onchain_version - .as_ref() - .map_or(false, |v| v.can_call_with(&self.native_version.runtime_version)), + onchain_version.can_call_with(&self.native_version.runtime_version), native_call, ) { (_, false, _) => { @@ -196,9 +212,7 @@ impl CodeExecutor for NativeExecutor { target: "executor", "Request for native execution failed (native: {}, chain: {})", self.native_version.runtime_version, - onchain_version - .as_ref() - .map_or_else(||"".into(), |v| format!("{}", v)) + onchain_version, ); safe_call( @@ -215,9 +229,7 @@ impl CodeExecutor for NativeExecutor { target: "executor", "Request for native execution with native call succeeded (native: {}, chain: {}).", self.native_version.runtime_version, - onchain_version - .as_ref() - .map_or_else(||"".into(), |v| format!("{}", v)) + onchain_version, ); used_native = true; @@ -234,7 +246,7 @@ impl CodeExecutor for NativeExecutor { target: "executor", "Request for native execution succeeded (native: {}, chain: {})", self.native_version.runtime_version, - onchain_version.as_ref().map_or_else(||"".into(), |v| format!("{}", v)) + onchain_version ); used_native = true; diff --git a/core/executor/src/sandbox.rs b/client/executor/src/sandbox.rs similarity index 74% rename from core/executor/src/sandbox.rs rename to client/executor/src/sandbox.rs index e1e9e0db95263..da71b06672465 100644 --- a/core/executor/src/sandbox.rs +++ b/client/executor/src/sandbox.rs @@ -211,7 +211,7 @@ pub trait SandboxCapabilities { /// Implementation of [`Externals`] that allows execution of guest module with /// [externals][`Externals`] that might refer functions defined by supervisor. /// -/// [`Externals`]: ../../wasmi/trait.Externals.html +/// [`Externals`]: ../wasmi/trait.Externals.html pub struct GuestExternals<'a, FE: SandboxCapabilities + 'a> { supervisor_externals: &'a mut FE, sandbox_instance: &'a SandboxInstance, @@ -432,7 +432,7 @@ fn decode_environment_definition( /// - `raw_env_def` can't be deserialized as a [`EnvironmentDefinition`]. /// - Module in `wasm` is invalid or couldn't be instantiated. /// -/// [`EnvironmentDefinition`]: ../../sandbox/struct.EnvironmentDefinition.html +/// [`EnvironmentDefinition`]: ../sandbox/struct.EnvironmentDefinition.html pub fn instantiate( supervisor_externals: &mut FE, dispatch_thunk: FE::SupervisorFuncRef, @@ -582,294 +582,3 @@ impl Store { instance_idx as u32 } } - -#[cfg(test)] -mod tests { - use super::*; - use primitives::{Blake2Hasher, traits::Externalities}; - use crate::wasm_runtime::WasmRuntime; - use crate::wasmi_execution; - use state_machine::TestExternalities as CoreTestExternalities; - use wabt; - use runtime_test::WASM_BINARY; - - type TestExternalities = CoreTestExternalities; - - fn call_wasm( - ext: &mut E, - heap_pages: u64, - code: &[u8], - method: &str, - data: &[u8], - ) -> Result> { - let mut instance = wasmi_execution::create_instance(ext, code, heap_pages) - .map_err(|err| err.to_string())?; - instance.call(ext, method, data) - } - - #[test] - fn sandbox_should_work() { - let mut ext = TestExternalities::default(); - let mut ext = ext.ext(); - let test_code = WASM_BINARY; - - let code = wabt::wat2wasm(r#" - (module - (import "env" "assert" (func $assert (param i32))) - (import "env" "inc_counter" (func $inc_counter (param i32) (result i32))) - (func (export "call") - (drop - (call $inc_counter (i32.const 5)) - ) - - (call $inc_counter (i32.const 3)) - ;; current counter value is on the stack - - ;; check whether current == 8 - i32.const 8 - i32.eq - - call $assert - ) - ) - "#).unwrap().encode(); - - assert_eq!( - call_wasm(&mut ext, 8, &test_code[..], "test_sandbox", &code).unwrap(), - true.encode(), - ); - } - - #[test] - fn sandbox_trap() { - let mut ext = TestExternalities::default(); - let mut ext = ext.ext(); - let test_code = WASM_BINARY; - - let code = wabt::wat2wasm(r#" - (module - (import "env" "assert" (func $assert (param i32))) - (func (export "call") - i32.const 0 - call $assert - ) - ) - "#).unwrap(); - - assert_eq!( - call_wasm(&mut ext, 8, &test_code[..], "test_sandbox", &code).unwrap(), - vec![0], - ); - } - - #[test] - fn sandbox_should_trap_when_heap_exhausted() { - let mut ext = TestExternalities::default(); - let mut ext = ext.ext(); - let test_code = WASM_BINARY; - - let code = wabt::wat2wasm(r#" - (module - (import "env" "assert" (func $assert (param i32))) - (func (export "call") - i32.const 0 - call $assert - ) - ) - "#).unwrap().encode(); - - let res = call_wasm(&mut ext, 8, &test_code[..], "test_exhaust_heap", &code); - assert_eq!(res.is_err(), true); - if let Err(err) = res { - assert_eq!( - format!("{}", err), - format!( - "{}", - wasmi::Error::Trap(Error::FunctionExecution("AllocatorOutOfSpace".into()).into()), - ), - ); - } - } - - #[test] - fn start_called() { - let mut ext = TestExternalities::default(); - let mut ext = ext.ext(); - let test_code = WASM_BINARY; - - let code = wabt::wat2wasm(r#" - (module - (import "env" "assert" (func $assert (param i32))) - (import "env" "inc_counter" (func $inc_counter (param i32) (result i32))) - - ;; Start function - (start $start) - (func $start - ;; Increment counter by 1 - (drop - (call $inc_counter (i32.const 1)) - ) - ) - - (func (export "call") - ;; Increment counter by 1. The current value is placed on the stack. - (call $inc_counter (i32.const 1)) - - ;; Counter is incremented twice by 1, once there and once in `start` func. - ;; So check the returned value is equal to 2. - i32.const 2 - i32.eq - call $assert - ) - ) - "#).unwrap().encode(); - - assert_eq!( - call_wasm(&mut ext, 8, &test_code[..], "test_sandbox", &code).unwrap(), - true.encode(), - ); - } - - #[test] - fn invoke_args() { - let mut ext = TestExternalities::default(); - let mut ext = ext.ext(); - let test_code = WASM_BINARY; - - let code = wabt::wat2wasm(r#" - (module - (import "env" "assert" (func $assert (param i32))) - - (func (export "call") (param $x i32) (param $y i64) - ;; assert that $x = 0x12345678 - (call $assert - (i32.eq - (get_local $x) - (i32.const 0x12345678) - ) - ) - - (call $assert - (i64.eq - (get_local $y) - (i64.const 0x1234567887654321) - ) - ) - ) - ) - "#).unwrap().encode(); - - assert_eq!( - call_wasm(&mut ext, 8, &test_code[..], "test_sandbox_args", &code).unwrap(), - true.encode(), - ); - } - - #[test] - fn return_val() { - let mut ext = TestExternalities::default(); - let mut ext = ext.ext(); - let test_code = WASM_BINARY; - - let code = wabt::wat2wasm(r#" - (module - (func (export "call") (param $x i32) (result i32) - (i32.add - (get_local $x) - (i32.const 1) - ) - ) - ) - "#).unwrap().encode(); - - assert_eq!( - call_wasm(&mut ext, 8, &test_code[..], "test_sandbox_return_val", &code).unwrap(), - true.encode(), - ); - } - - #[test] - fn unlinkable_module() { - let mut ext = TestExternalities::default(); - let mut ext = ext.ext(); - let test_code = WASM_BINARY; - - let code = wabt::wat2wasm(r#" - (module - (import "env" "non-existent" (func)) - - (func (export "call") - ) - ) - "#).unwrap().encode(); - - assert_eq!( - call_wasm(&mut ext, 8, &test_code[..], "test_sandbox_instantiate", &code).unwrap(), - 1u8.encode(), - ); - } - - #[test] - fn corrupted_module() { - let mut ext = TestExternalities::default(); - let mut ext = ext.ext(); - let test_code = WASM_BINARY; - - // Corrupted wasm file - let code = vec![0u8, 0, 0, 0, 1, 0, 0, 0].encode(); - - assert_eq!( - call_wasm(&mut ext, 8, &test_code[..], "test_sandbox_instantiate", &code).unwrap(), - 1u8.encode(), - ); - } - - #[test] - fn start_fn_ok() { - let mut ext = TestExternalities::default(); - let mut ext = ext.ext(); - let test_code = WASM_BINARY; - - let code = wabt::wat2wasm(r#" - (module - (func (export "call") - ) - - (func $start - ) - - (start $start) - ) - "#).unwrap().encode(); - - assert_eq!( - call_wasm(&mut ext, 8, &test_code[..], "test_sandbox_instantiate", &code).unwrap(), - 0u8.encode(), - ); - } - - #[test] - fn start_fn_traps() { - let mut ext = TestExternalities::default(); - let mut ext = ext.ext(); - let test_code = WASM_BINARY; - - let code = wabt::wat2wasm(r#" - (module - (func (export "call") - ) - - (func $start - unreachable - ) - - (start $start) - ) - "#).unwrap().encode(); - - assert_eq!( - call_wasm(&mut ext, 8, &test_code[..], "test_sandbox_instantiate", &code).unwrap(), - 2u8.encode(), - ); - } -} diff --git a/core/executor/src/wasm_runtime.rs b/client/executor/src/wasm_runtime.rs similarity index 63% rename from core/executor/src/wasm_runtime.rs rename to client/executor/src/wasm_runtime.rs index 29a6c51e39d3b..c58f63a1e03de 100644 --- a/core/executor/src/wasm_runtime.rs +++ b/client/executor/src/wasm_runtime.rs @@ -19,13 +19,19 @@ //! The primary means of accessing the runtimes is through a cache which saves the reusable //! components of the runtime that are expensive to initialize. -use crate::error::{Error, WasmError}; -use crate::wasmi_execution; +use crate::{wasmi_execution, error::{Error, WasmError}}; +#[cfg(feature = "wasmtime")] +use crate::wasmtime; use log::{trace, warn}; + use codec::Decode; + use primitives::{storage::well_known_keys, traits::Externalities, H256}; + use runtime_version::RuntimeVersion; -use std::{collections::hash_map::{Entry, HashMap}}; +use std::{collections::hash_map::{Entry, HashMap}, panic::AssertUnwindSafe}; + +use wasm_interface::Function; /// The Substrate Wasm runtime. pub trait WasmRuntime { @@ -35,15 +41,12 @@ pub trait WasmRuntime { /// the heap pages would not change from its current value. fn update_heap_pages(&mut self, heap_pages: u64) -> bool; + /// Return the host functions that are registered for this Wasm runtime. + fn host_functions(&self) -> &[&'static dyn Function]; + /// Call a method in the Substrate runtime by name. Returns the encoded result on success. fn call(&mut self, ext: &mut dyn Externalities, method: &str, data: &[u8]) -> Result, Error>; - - /// Returns the version of this runtime. - /// - /// Returns `None` if the runtime doesn't provide the information or there was an error - /// while fetching it. - fn version(&self) -> Option; } /// Specification of different methods of executing the runtime Wasm code. @@ -51,6 +54,16 @@ pub trait WasmRuntime { pub enum WasmExecutionMethod { /// Uses the Wasmi interpreter. Interpreted, + /// Uses the Wasmtime compiled runtime. + #[cfg(feature = "wasmtime")] + Compiled, +} + +/// A Wasm runtime object along with its cached runtime version. +struct VersionedRuntime { + runtime: Box, + /// Runtime version according to `Core_version`. + version: RuntimeVersion, } /// Cache for the runtimes. @@ -69,7 +82,7 @@ pub struct RuntimesCache { /// A cache of runtime instances along with metadata, ready to be reused. /// /// Instances are keyed by the Wasm execution method and the hash of their code. - instances: HashMap<(WasmExecutionMethod, [u8; 32]), Result, WasmError>>, + instances: HashMap<(WasmExecutionMethod, [u8; 32]), Result>, } impl RuntimesCache { @@ -92,11 +105,12 @@ impl RuntimesCache { /// # Parameters /// /// `ext` - Externalities to use for the runtime. This is used for setting - /// up an initial runtime instance. The parameter is only needed for calling - /// into the Wasm module to find out the `Core_version`. + /// up an initial runtime instance. /// /// `default_heap_pages` - Number of 64KB pages to allocate for Wasm execution. /// + /// `host_functions` - The host functions that should be registered for the Wasm runtime. + /// /// # Return value /// /// If no error occurred a tuple `(&mut WasmRuntime, H256)` is @@ -113,7 +127,8 @@ impl RuntimesCache { ext: &mut E, wasm_method: WasmExecutionMethod, default_heap_pages: u64, - ) -> Result<(&mut (dyn WasmRuntime + 'static), H256), Error> { + host_functions: &[&'static dyn Function], + ) -> Result<(&mut (dyn WasmRuntime + 'static), &RuntimeVersion, H256), Error> { let code_hash = ext .original_storage_hash(well_known_keys::CODE) .ok_or(Error::InvalidCode("`CODE` not found in storage.".into()))?; @@ -127,12 +142,27 @@ impl RuntimesCache { Entry::Occupied(o) => { let result = o.into_mut(); if let Ok(ref mut cached_runtime) = result { - if !cached_runtime.update_heap_pages(heap_pages) { + let heap_pages_changed = !cached_runtime.runtime.update_heap_pages(heap_pages); + let host_functions_changed = cached_runtime.runtime.host_functions() + != host_functions; + if heap_pages_changed || host_functions_changed { + let changed = if heap_pages_changed { + "heap_pages" + } else { + "host functions" + }; + trace!( target: "runtimes_cache", - "heap_pages were changed. Reinstantiating the instance", + "{} were changed. Reinstantiating the instance", + changed, + ); + *result = create_versioned_wasm_runtime( + ext, + wasm_method, + heap_pages, + host_functions.into(), ); - *result = create_wasm_runtime(ext, wasm_method, heap_pages); if let Err(ref err) = result { warn!(target: "runtimes_cache", "cannot create a runtime: {:?}", err); } @@ -142,7 +172,12 @@ impl RuntimesCache { }, Entry::Vacant(v) => { trace!(target: "runtimes_cache", "no instance found in cache, creating now."); - let result = create_wasm_runtime(ext, wasm_method, heap_pages); + let result = create_versioned_wasm_runtime( + ext, + wasm_method, + heap_pages, + host_functions.into(), + ); if let Err(ref err) = result { warn!(target: "runtimes_cache", "cannot create a runtime: {:?}", err); } @@ -151,7 +186,7 @@ impl RuntimesCache { }; result.as_mut() - .map(|runtime| (runtime.as_mut(), code_hash)) + .map(|entry| (entry.runtime.as_mut(), &entry.version, code_hash)) .map_err(|ref e| Error::InvalidCode(format!("{:?}", e))) } @@ -171,26 +206,66 @@ impl RuntimesCache { } /// Create a wasm runtime with the given `code`. -pub fn create_wasm_runtime_with_code( - ext: &mut E, +pub fn create_wasm_runtime_with_code( wasm_method: WasmExecutionMethod, heap_pages: u64, code: &[u8], + host_functions: Vec<&'static dyn Function>, ) -> Result, WasmError> { match wasm_method { WasmExecutionMethod::Interpreted => - wasmi_execution::create_instance(ext, code, heap_pages) + wasmi_execution::create_instance(code, heap_pages, host_functions) + .map(|runtime| -> Box { Box::new(runtime) }), + #[cfg(feature = "wasmtime")] + WasmExecutionMethod::Compiled => + wasmtime::create_instance(code, heap_pages, host_functions) .map(|runtime| -> Box { Box::new(runtime) }), } } -fn create_wasm_runtime( +fn create_versioned_wasm_runtime( ext: &mut E, wasm_method: WasmExecutionMethod, heap_pages: u64, -) -> Result, WasmError> { + host_functions: Vec<&'static dyn Function>, +) -> Result { let code = ext .original_storage(well_known_keys::CODE) .ok_or(WasmError::CodeNotFound)?; - create_wasm_runtime_with_code(ext, wasm_method, heap_pages, &code) + let mut runtime = create_wasm_runtime_with_code(wasm_method, heap_pages, &code, host_functions)?; + + // Call to determine runtime version. + let version_result = { + // `ext` is already implicitly handled as unwind safe, as we store it in a global variable. + let mut ext = AssertUnwindSafe(ext); + + // The following unwind safety assertion is OK because if the method call panics, the + // runtime will be dropped. + let mut runtime = AssertUnwindSafe(runtime.as_mut()); + crate::native_executor::safe_call( + move || runtime.call(&mut **ext, "Core_version", &[]) + ).map_err(|_| WasmError::Instantiation("panic in call to get runtime version".into()))? + }; + let encoded_version = version_result + .map_err(|e| WasmError::Instantiation(format!("failed to call \"Core_version\": {}", e)))?; + let version = RuntimeVersion::decode(&mut encoded_version.as_slice()) + .map_err(|_| WasmError::Instantiation("failed to decode \"Core_version\" result".into()))?; + + Ok(VersionedRuntime { + runtime, + version, + }) +} + +#[cfg(test)] +mod tests { + use wasm_interface::HostFunctions; + + #[test] + fn host_functions_are_equal() { + let host_functions = runtime_io::SubstrateHostFunctions::host_functions(); + + let equal = &host_functions[..] == &host_functions[..]; + assert!(equal, "Host functions are not equal"); + } } diff --git a/core/executor/src/wasm_utils.rs b/client/executor/src/wasm_utils.rs similarity index 88% rename from core/executor/src/wasm_utils.rs rename to client/executor/src/wasm_utils.rs index b217350ac6fc4..caa63ddbf2985 100644 --- a/core/executor/src/wasm_utils.rs +++ b/client/executor/src/wasm_utils.rs @@ -16,6 +16,8 @@ //! Utilities for defining the wasm host environment. +use wasm_interface::{Pointer, WordSize}; + /// Converts arguments into respective WASM types. #[macro_export] macro_rules! convert_args { @@ -45,7 +47,7 @@ macro_rules! gen_functions { { $( $generated:tt )* } $context:ident, ) => ( - &[ $( $generated )* ] + vec![ $( $generated )* ] ); (@INTERNAL { $( $generated:tt )* } @@ -162,7 +164,7 @@ macro_rules! impl_wasm_host_interface { ) => ( impl $crate::wasm_interface::HostFunctions for $interface_name { #[allow(non_camel_case_types)] - fn functions() -> &'static [&'static dyn $crate::wasm_interface::Function] { + fn host_functions() -> Vec<&'static dyn $crate::wasm_interface::Function> { gen_functions!( $context, $( $name( $( $names: $params ),* ) $( -> $returns )? { $( $body )* } )* @@ -171,3 +173,14 @@ macro_rules! impl_wasm_host_interface { } ); } + +/// Runtime API functions return an i64 which encodes a pointer in the least-significant 32 bits +/// and a length in the most-significant 32 bits. This interprets the returned value as a pointer, +/// length tuple. +pub fn interpret_runtime_api_result(retval: i64) -> (Pointer, WordSize) { + let ptr = >::new(retval as u32); + // The first cast to u64 is necessary so that the right shift does not sign-extend. + let len = ((retval as u64) >> 32) as WordSize; + (ptr, len) +} + diff --git a/core/executor/src/wasmi_execution.rs b/client/executor/src/wasmi_execution.rs similarity index 51% rename from core/executor/src/wasmi_execution.rs rename to client/executor/src/wasmi_execution.rs index a83729b4b6c8d..7f480695798fa 100644 --- a/core/executor/src/wasmi_execution.rs +++ b/client/executor/src/wasmi_execution.rs @@ -16,7 +16,7 @@ //! Implementation of a Wasm runtime using the Wasmi interpreter. -use std::{str, mem, panic::AssertUnwindSafe}; +use std::{str, mem}; use wasmi::{ Module, ModuleInstance, MemoryInstance, MemoryRef, TableRef, ImportsBuilder, ModuleRef, memory_units::Pages, RuntimeValue::{I32, I64, self}, @@ -24,36 +24,42 @@ use wasmi::{ use crate::error::{Error, WasmError}; use codec::{Encode, Decode}; use primitives::{sandbox as sandbox_primitives, traits::Externalities}; -use crate::host_interface::SubstrateExternals; use crate::sandbox; use crate::allocator; +use crate::wasm_utils::interpret_runtime_api_result; use crate::wasm_runtime::WasmRuntime; -use log::trace; +use log::{error, trace}; use parity_wasm::elements::{deserialize_buffer, DataSegment, Instruction, Module as RawModule}; -use runtime_version::RuntimeVersion; use wasm_interface::{ - FunctionContext, HostFunctions, Pointer, WordSize, Sandbox, MemoryId, Result as WResult, + FunctionContext, Pointer, WordSize, Sandbox, MemoryId, Result as WResult, Function, }; -struct FunctionExecutor { +struct FunctionExecutor<'a> { sandbox_store: sandbox::Store, heap: allocator::FreeingBumpHeapAllocator, memory: MemoryRef, table: Option, + host_functions: &'a [&'static dyn Function], } -impl FunctionExecutor { - fn new(m: MemoryRef, heap_base: u32, t: Option) -> Result { +impl<'a> FunctionExecutor<'a> { + fn new( + m: MemoryRef, + heap_base: u32, + t: Option, + host_functions: &'a [&'static dyn Function], + ) -> Result { Ok(FunctionExecutor { sandbox_store: sandbox::Store::new(), heap: allocator::FreeingBumpHeapAllocator::new(heap_base), memory: m, table: t, + host_functions, }) } } -impl sandbox::SandboxCapabilities for FunctionExecutor { +impl<'a> sandbox::SandboxCapabilities for FunctionExecutor<'a> { type SupervisorFuncRef = wasmi::FuncRef; fn store(&self) -> &sandbox::Store { @@ -108,26 +114,26 @@ impl sandbox::SandboxCapabilities for FunctionExecutor { } } -impl FunctionContext for FunctionExecutor { +impl<'a> FunctionContext for FunctionExecutor<'a> { fn read_memory_into(&self, address: Pointer, dest: &mut [u8]) -> WResult<()> { - self.memory.get_into(address.into(), dest).map_err(|e| format!("{:?}", e)) + self.memory.get_into(address.into(), dest).map_err(|e| e.to_string()) } fn write_memory(&mut self, address: Pointer, data: &[u8]) -> WResult<()> { - self.memory.set(address.into(), data).map_err(|e| format!("{:?}", e)) + self.memory.set(address.into(), data).map_err(|e| e.to_string()) } fn allocate_memory(&mut self, size: WordSize) -> WResult> { let heap = &mut self.heap; self.memory.with_direct_access_mut(|mem| { - heap.allocate(mem, size).map_err(|e| format!("{:?}", e)) + heap.allocate(mem, size).map_err(|e| e.to_string()) }) } fn deallocate_memory(&mut self, ptr: Pointer) -> WResult<()> { let heap = &mut self.heap; self.memory.with_direct_access_mut(|mem| { - heap.deallocate(mem, ptr).map_err(|e| format!("{:?}", e)) + heap.deallocate(mem, ptr).map_err(|e| e.to_string()) }) } @@ -136,15 +142,15 @@ impl FunctionContext for FunctionExecutor { } } -impl Sandbox for FunctionExecutor { +impl<'a> Sandbox for FunctionExecutor<'a> { fn memory_get( - &self, + &mut self, memory_id: MemoryId, offset: WordSize, buf_ptr: Pointer, buf_len: WordSize, ) -> WResult { - let sandboxed_memory = self.sandbox_store.memory(memory_id).map_err(|e| format!("{:?}", e))?; + let sandboxed_memory = self.sandbox_store.memory(memory_id).map_err(|e| e.to_string())?; match MemoryInstance::transfer( &sandboxed_memory, @@ -165,7 +171,7 @@ impl Sandbox for FunctionExecutor { val_ptr: Pointer, val_len: WordSize, ) -> WResult { - let sandboxed_memory = self.sandbox_store.memory(memory_id).map_err(|e| format!("{:?}", e))?; + let sandboxed_memory = self.sandbox_store.memory(memory_id).map_err(|e| e.to_string())?; match MemoryInstance::transfer( &self.memory, @@ -180,7 +186,7 @@ impl Sandbox for FunctionExecutor { } fn memory_teardown(&mut self, memory_id: MemoryId) -> WResult<()> { - self.sandbox_store.memory_teardown(memory_id).map_err(|e| format!("{:?}", e)) + self.sandbox_store.memory_teardown(memory_id).map_err(|e| e.to_string()) } fn memory_new( @@ -188,7 +194,7 @@ impl Sandbox for FunctionExecutor { initial: u32, maximum: u32, ) -> WResult { - self.sandbox_store.new_memory(initial, maximum).map_err(|e| format!("{:?}", e)) + self.sandbox_store.new_memory(initial, maximum).map_err(|e| e.to_string()) } fn invoke( @@ -209,7 +215,7 @@ impl Sandbox for FunctionExecutor { .map(Into::into) .collect::>(); - let instance = self.sandbox_store.instance(instance_id).map_err(|e| format!("{:?}", e))?; + let instance = self.sandbox_store.instance(instance_id).map_err(|e| e.to_string())?; let result = instance.invoke(export_name, &args, self, state); match result { @@ -229,7 +235,7 @@ impl Sandbox for FunctionExecutor { } fn instance_teardown(&mut self, instance_id: u32) -> WResult<()> { - self.sandbox_store.instance_teardown(instance_id).map_err(|e| format!("{:?}", e)) + self.sandbox_store.instance_teardown(instance_id).map_err(|e| e.to_string()) } fn instance_new( @@ -261,55 +267,51 @@ impl Sandbox for FunctionExecutor { } } -impl FunctionExecutor { - fn resolver() -> &'static dyn wasmi::ModuleImportResolver { - struct Resolver; - impl wasmi::ModuleImportResolver for Resolver { - fn resolve_func(&self, name: &str, signature: &wasmi::Signature) - -> std::result::Result - { - let signature = wasm_interface::Signature::from(signature); - - if let Some((index, func)) = SubstrateExternals::functions().iter() - .enumerate() - .find(|f| name == f.1.name()) - { - if signature == func.signature() { - Ok(wasmi::FuncInstance::alloc_host(signature.into(), index)) - } else { - Err(wasmi::Error::Instantiation( - format!( - "Invalid signature for function `{}` expected `{:?}`, got `{:?}`", - func.name(), - signature, - func.signature(), - ) - )) - } +struct Resolver<'a>(&'a[&'static dyn Function]); + +impl<'a> wasmi::ModuleImportResolver for Resolver<'a> { + fn resolve_func(&self, name: &str, signature: &wasmi::Signature) + -> std::result::Result + { + let signature = wasm_interface::Signature::from(signature); + for (function_index, function) in self.0.iter().enumerate() { + if name == function.name() { + if signature == function.signature() { + return Ok( + wasmi::FuncInstance::alloc_host(signature.into(), function_index), + ) } else { - Err(wasmi::Error::Instantiation( - format!("Export {} not found", name), + return Err(wasmi::Error::Instantiation( + format!( + "Invalid signature for function `{}` expected `{:?}`, got `{:?}`", + function.name(), + signature, + function.signature(), + ), )) } } } - &Resolver + + Err(wasmi::Error::Instantiation( + format!("Export {} not found", name), + )) } } -impl wasmi::Externals for FunctionExecutor { +impl<'a> wasmi::Externals for FunctionExecutor<'a> { fn invoke_index(&mut self, index: usize, args: wasmi::RuntimeArgs) -> Result, wasmi::Trap> { let mut args = args.as_ref().iter().copied().map(Into::into); - let function = SubstrateExternals::functions().get(index).ok_or_else(|| + let function = self.host_functions.get(index).ok_or_else(|| Error::from( format!("Could not find host function with index: {}", index), ) )?; function.execute(self, &mut args) - .map_err(Error::FunctionExecution) + .map_err(|msg| Error::FunctionExecution(function.name().to_string(), msg)) .map_err(wasmi::Trap::from) .map(|v| v.map(Into::into)) } @@ -346,39 +348,8 @@ fn call_in_wasm_module( module_instance: &ModuleRef, method: &str, data: &[u8], + host_functions: &[&'static dyn Function], ) -> Result, Error> { - call_in_wasm_module_with_custom_signature( - ext, - module_instance, - method, - |alloc| { - let offset = alloc(data)?; - Ok(vec![I32(offset as i32), I32(data.len() as i32)]) - }, - |res, memory| { - if let Some(I64(r)) = res { - let offset = r as u32; - let length = (r as u64 >> 32) as usize; - memory.get(offset, length).map_err(|_| Error::Runtime).map(Some) - } else { - Ok(None) - } - } - ) -} - -/// Call a given method in the given wasm-module runtime. -fn call_in_wasm_module_with_custom_signature< - F: FnOnce(&mut dyn FnMut(&[u8]) -> Result) -> Result, Error>, - FR: FnOnce(Option, &MemoryRef) -> Result, Error>, - R, ->( - ext: &mut dyn Externalities, - module_instance: &ModuleRef, - method: &str, - create_parameters: F, - filter_result: FR, -) -> Result { // extract a reference to a linear memory, optional reference to a table // and then initialize FunctionExecutor. let memory = get_mem_instance(module_instance)?; @@ -387,26 +358,25 @@ fn call_in_wasm_module_with_custom_signature< .and_then(|e| e.as_table().cloned()); let heap_base = get_heap_base(module_instance)?; - let mut fec = FunctionExecutor::new( - memory.clone(), - heap_base, - table, - )?; + let mut fec = FunctionExecutor::new(memory.clone(), heap_base, table, host_functions)?; - let parameters = create_parameters(&mut |data: &[u8]| { - let offset = fec.allocate_memory(data.len() as u32)?; - fec.write_memory(offset, data).map(|_| offset.into()).map_err(Into::into) - })?; + // Write the call data + let offset = fec.allocate_memory(data.len() as u32)?; + fec.write_memory(offset, data)?; let result = externalities::set_and_run_with_externalities( ext, - || module_instance.invoke_export(method, ¶meters, &mut fec), + || module_instance.invoke_export( + method, + &[I32(u32::from(offset) as i32), I32(data.len() as i32)], + &mut fec, + ), ); match result { - Ok(val) => match filter_result(val, &memory)? { - Some(val) => Ok(val), - None => Err(Error::InvalidReturn), + Ok(Some(I64(r))) => { + let (ptr, length) = interpret_runtime_api_result(r); + memory.get(ptr.into(), length as usize).map_err(|_| Error::Runtime) }, Err(e) => { trace!( @@ -416,6 +386,7 @@ fn call_in_wasm_module_with_custom_signature< ); Err(e.into()) }, + _ => Err(Error::InvalidReturn), } } @@ -423,12 +394,13 @@ fn call_in_wasm_module_with_custom_signature< fn instantiate_module( heap_pages: usize, module: &Module, + host_functions: &[&'static dyn Function], ) -> Result { + let resolver = Resolver(host_functions); // start module instantiation. Don't run 'start' function yet. let intermediate_instance = ModuleInstance::new( module, - &ImportsBuilder::new() - .with_resolver("env", FunctionExecutor::resolver()) + &ImportsBuilder::new().with_resolver("env", &resolver), )?; // Verify that the module has the heap base global variable. @@ -528,7 +500,7 @@ impl StateSnapshot { // First, erase the memory and copy the data segments into it. memory .erase() - .map_err(|_| WasmError::ApplySnapshotFailed)?; + .map_err(|e| WasmError::ErasingFailed(e.to_string()))?; for (offset, contents) in &self.data_segments { memory .set(*offset, contents) @@ -553,34 +525,15 @@ impl StateSnapshot { } } -/// A runtime along with its version and initial state snapshot. +/// A runtime along with its initial state snapshot. #[derive(Clone)] pub struct WasmiRuntime { /// A wasm module instance. instance: ModuleRef, - /// Runtime version according to `Core_version`. - /// - /// Can be `None` if the runtime doesn't expose this function. - version: Option, /// The snapshot of the instance's state taken just after the instantiation. state_snapshot: StateSnapshot, -} - -impl WasmiRuntime { - /// Perform an operation with the clean version of the runtime wasm instance. - fn with(&self, f: F) -> R - where - F: FnOnce(&ModuleRef) -> R, - { - self.state_snapshot.apply(&self.instance).expect( - "applying the snapshot can only fail if the passed instance is different - from the one that was used for creation of the snapshot; - we use the snapshot that is directly associated with the instance; - thus the snapshot was created using the instance; - qed", - ); - f(&self.instance) - } + /// The host functions registered for this instance. + host_functions: Vec<&'static dyn Function>, } impl WasmRuntime for WasmiRuntime { @@ -588,22 +541,33 @@ impl WasmRuntime for WasmiRuntime { self.state_snapshot.heap_pages == heap_pages } - fn call(&mut self, ext: &mut dyn Externalities, method: &str, data: &[u8]) - -> Result, Error> - { - self.with(|module| { - call_in_wasm_module(ext, module, method, data) - }) + fn host_functions(&self) -> &[&'static dyn Function] { + &self.host_functions } - fn version(&self) -> Option { - self.version.clone() + fn call( + &mut self, + ext: &mut dyn Externalities, + method: &str, + data: &[u8], + ) -> Result, Error> { + self.state_snapshot.apply(&self.instance) + .map_err(|e| { + // Snapshot restoration failed. This is pretty unexpected since this can happen + // if some invariant is broken or if the system is under extreme memory pressure + // (so erasing fails). + error!(target: "wasm-executor", "snapshot restoration failed: {}", e); + e + })?; + call_in_wasm_module(ext, &self.instance, method, data, &self.host_functions) } } -pub fn create_instance(ext: &mut E, code: &[u8], heap_pages: u64) - -> Result -{ +pub fn create_instance( + code: &[u8], + heap_pages: u64, + host_functions: Vec<&'static dyn Function>, +) -> Result { let module = Module::from_buffer(&code).map_err(|_| WasmError::InvalidModule)?; // Extract the data segments from the wasm code. @@ -613,8 +577,8 @@ pub fn create_instance(ext: &mut E, code: &[u8], heap_pages: u let data_segments = extract_data_segments(&code)?; // Instantiate this module. - let instance = instantiate_module(heap_pages as usize, &module) - .map_err(WasmError::Instantiation)?; + let instance = instantiate_module(heap_pages as usize, &module, &host_functions) + .map_err(|e| WasmError::Instantiation(e.to_string()))?; // Take state snapshot before executing anything. let state_snapshot = StateSnapshot::take(&instance, data_segments, heap_pages) @@ -625,18 +589,10 @@ pub fn create_instance(ext: &mut E, code: &[u8], heap_pages: u ", ); - let mut ext = AssertUnwindSafe(ext); - let call_instance = AssertUnwindSafe(&instance); - let version = crate::native_executor::safe_call( - move || call_in_wasm_module(&mut **ext, *call_instance, "Core_version", &[]) - .ok() - .and_then(|v| RuntimeVersion::decode(&mut v.as_slice()).ok()) - ).map_err(WasmError::Instantiation)?; - Ok(WasmiRuntime { instance, - version, state_snapshot, + host_functions, }) } @@ -654,315 +610,3 @@ fn extract_data_segments(wasm_code: &[u8]) -> Result, WasmError .to_vec(); Ok(segments) } - -#[cfg(test)] -mod tests { - use super::*; - - use state_machine::TestExternalities as CoreTestExternalities; - use hex_literal::hex; - use primitives::{ - Blake2Hasher, blake2_128, blake2_256, ed25519, sr25519, map, Pair, offchain::OffchainExt, - }; - use runtime_test::WASM_BINARY; - use substrate_offchain::testing; - use trie::{TrieConfiguration, trie_types::Layout}; - use codec::{Encode, Decode}; - - type TestExternalities = CoreTestExternalities; - - fn call( - ext: &mut E, - heap_pages: u64, - code: &[u8], - method: &str, - data: &[u8], - ) -> Result, Error> { - let mut instance = create_instance(ext, code, heap_pages) - .map_err(|err| err.to_string())?; - instance.call(ext, method, data) - } - - #[test] - fn returning_should_work() { - let mut ext = TestExternalities::default(); - let mut ext = ext.ext(); - let test_code = WASM_BINARY; - - let output = call(&mut ext, 8, &test_code[..], "test_empty_return", &[]).unwrap(); - assert_eq!(output, vec![0u8; 0]); - } - - #[test] - fn panicking_should_work() { - let mut ext = TestExternalities::default(); - let mut ext = ext.ext(); - let test_code = WASM_BINARY; - - let output = call(&mut ext, 8, &test_code[..], "test_panic", &[]); - assert!(output.is_err()); - - let output = call(&mut ext, 8, &test_code[..], "test_conditional_panic", &[0]); - assert_eq!(Decode::decode(&mut &output.unwrap()[..]), Ok(Vec::::new())); - - let output = call(&mut ext, 8, &test_code[..], "test_conditional_panic", &vec![2].encode()); - assert!(output.is_err()); - } - - #[test] - fn storage_should_work() { - let mut ext = TestExternalities::default(); - - { - let mut ext = ext.ext(); - ext.set_storage(b"foo".to_vec(), b"bar".to_vec()); - let test_code = WASM_BINARY; - - let output = call( - &mut ext, - 8, - &test_code[..], - "test_data_in", - &b"Hello world".to_vec().encode(), - ).unwrap(); - - assert_eq!(output, b"all ok!".to_vec().encode()); - } - - let expected = TestExternalities::new((map![ - b"input".to_vec() => b"Hello world".to_vec(), - b"foo".to_vec() => b"bar".to_vec(), - b"baz".to_vec() => b"bar".to_vec() - ], map![])); - assert_eq!(ext, expected); - } - - #[test] - fn clear_prefix_should_work() { - let mut ext = TestExternalities::default(); - { - let mut ext = ext.ext(); - ext.set_storage(b"aaa".to_vec(), b"1".to_vec()); - ext.set_storage(b"aab".to_vec(), b"2".to_vec()); - ext.set_storage(b"aba".to_vec(), b"3".to_vec()); - ext.set_storage(b"abb".to_vec(), b"4".to_vec()); - ext.set_storage(b"bbb".to_vec(), b"5".to_vec()); - let test_code = WASM_BINARY; - - // This will clear all entries which prefix is "ab". - let output = call( - &mut ext, - 8, - &test_code[..], - "test_clear_prefix", - &b"ab".to_vec().encode(), - ).unwrap(); - - assert_eq!(output, b"all ok!".to_vec().encode()); - } - - let expected = TestExternalities::new((map![ - b"aaa".to_vec() => b"1".to_vec(), - b"aab".to_vec() => b"2".to_vec(), - b"bbb".to_vec() => b"5".to_vec() - ], map![])); - assert_eq!(expected, ext); - } - - #[test] - fn blake2_256_should_work() { - let mut ext = TestExternalities::default(); - let mut ext = ext.ext(); - let test_code = WASM_BINARY; - assert_eq!( - call(&mut ext, 8, &test_code[..], "test_blake2_256", &[0]).unwrap(), - blake2_256(&b""[..]).to_vec().encode(), - ); - assert_eq!( - call( - &mut ext, - 8, - &test_code[..], - "test_blake2_256", - &b"Hello world!".to_vec().encode(), - ).unwrap(), - blake2_256(&b"Hello world!"[..]).to_vec().encode(), - ); - } - - #[test] - fn blake2_128_should_work() { - let mut ext = TestExternalities::default(); - let mut ext = ext.ext(); - let test_code = WASM_BINARY; - assert_eq!( - call(&mut ext, 8, &test_code[..], "test_blake2_128", &[0]).unwrap(), - blake2_128(&b""[..]).to_vec().encode(), - ); - assert_eq!( - call( - &mut ext, - 8, - &test_code[..], - "test_blake2_128", - &b"Hello world!".to_vec().encode(), - ).unwrap(), - blake2_128(&b"Hello world!"[..]).to_vec().encode(), - ); - } - - #[test] - fn twox_256_should_work() { - let mut ext = TestExternalities::default(); - let mut ext = ext.ext(); - let test_code = WASM_BINARY; - assert_eq!( - call(&mut ext, 8, &test_code[..], "test_twox_256", &[0]).unwrap(), - hex!( - "99e9d85137db46ef4bbea33613baafd56f963c64b1f3685a4eb4abd67ff6203a" - ).to_vec().encode(), - ); - assert_eq!( - call( - &mut ext, - 8, - &test_code[..], - "test_twox_256", - &b"Hello world!".to_vec().encode(), - ).unwrap(), - hex!( - "b27dfd7f223f177f2a13647b533599af0c07f68bda23d96d059da2b451a35a74" - ).to_vec().encode(), - ); - } - - #[test] - fn twox_128_should_work() { - let mut ext = TestExternalities::default(); - let mut ext = ext.ext(); - let test_code = WASM_BINARY; - assert_eq!( - call(&mut ext, 8, &test_code[..], "test_twox_128", &[0]).unwrap(), - hex!("99e9d85137db46ef4bbea33613baafd5").to_vec().encode(), - ); - assert_eq!( - call( - &mut ext, - 8, - &test_code[..], - "test_twox_128", - &b"Hello world!".to_vec().encode(), - ).unwrap(), - hex!("b27dfd7f223f177f2a13647b533599af").to_vec().encode(), - ); - } - - #[test] - fn ed25519_verify_should_work() { - let mut ext = TestExternalities::default(); - let mut ext = ext.ext(); - let test_code = WASM_BINARY; - let key = ed25519::Pair::from_seed(&blake2_256(b"test")); - let sig = key.sign(b"all ok!"); - let mut calldata = vec![]; - calldata.extend_from_slice(key.public().as_ref()); - calldata.extend_from_slice(sig.as_ref()); - - assert_eq!( - call(&mut ext, 8, &test_code[..], "test_ed25519_verify", &calldata.encode()).unwrap(), - true.encode(), - ); - - let other_sig = key.sign(b"all is not ok!"); - let mut calldata = vec![]; - calldata.extend_from_slice(key.public().as_ref()); - calldata.extend_from_slice(other_sig.as_ref()); - - assert_eq!( - call(&mut ext, 8, &test_code[..], "test_ed25519_verify", &calldata.encode()).unwrap(), - false.encode(), - ); - } - - #[test] - fn sr25519_verify_should_work() { - let mut ext = TestExternalities::default(); - let mut ext = ext.ext(); - let test_code = WASM_BINARY; - let key = sr25519::Pair::from_seed(&blake2_256(b"test")); - let sig = key.sign(b"all ok!"); - let mut calldata = vec![]; - calldata.extend_from_slice(key.public().as_ref()); - calldata.extend_from_slice(sig.as_ref()); - - assert_eq!( - call(&mut ext, 8, &test_code[..], "test_sr25519_verify", &calldata.encode()).unwrap(), - true.encode(), - ); - - let other_sig = key.sign(b"all is not ok!"); - let mut calldata = vec![]; - calldata.extend_from_slice(key.public().as_ref()); - calldata.extend_from_slice(other_sig.as_ref()); - - assert_eq!( - call(&mut ext, 8, &test_code[..], "test_sr25519_verify", &calldata.encode()).unwrap(), - false.encode(), - ); - } - - #[test] - fn ordered_trie_root_should_work() { - let mut ext = TestExternalities::default(); - let mut ext = ext.ext(); - let trie_input = vec![b"zero".to_vec(), b"one".to_vec(), b"two".to_vec()]; - let test_code = WASM_BINARY; - assert_eq!( - call(&mut ext, 8, &test_code[..], "test_ordered_trie_root", &[0]).unwrap(), - Layout::::ordered_trie_root(trie_input.iter()).as_bytes().encode(), - ); - } - - #[test] - fn offchain_local_storage_should_work() { - use substrate_client::backend::OffchainStorage; - - let mut ext = TestExternalities::default(); - let (offchain, state) = testing::TestOffchainExt::new(); - ext.register_extension(OffchainExt::new(offchain)); - let test_code = WASM_BINARY; - let mut ext = ext.ext(); - assert_eq!( - call(&mut ext, 8, &test_code[..], "test_offchain_local_storage", &[0]).unwrap(), - true.encode(), - ); - assert_eq!(state.read().persistent_storage.get(b"", b"test"), Some(vec![])); - } - - #[test] - fn offchain_http_should_work() { - let mut ext = TestExternalities::default(); - let (offchain, state) = testing::TestOffchainExt::new(); - ext.register_extension(OffchainExt::new(offchain)); - state.write().expect_request( - 0, - testing::PendingRequest { - method: "POST".into(), - uri: "http://localhost:12345".into(), - body: vec![1, 2, 3, 4], - headers: vec![("X-Auth".to_owned(), "test".to_owned())], - sent: true, - response: Some(vec![1, 2, 3]), - response_headers: vec![("X-Auth".to_owned(), "hello".to_owned())], - ..Default::default() - }, - ); - - let test_code = WASM_BINARY; - let mut ext = ext.ext(); - assert_eq!( - call(&mut ext, 8, &test_code[..], "test_offchain_http", &[0]).unwrap(), - true.encode(), - ); - } -} diff --git a/client/executor/src/wasmtime/function_executor.rs b/client/executor/src/wasmtime/function_executor.rs new file mode 100644 index 0000000000000..5dc8f42b280c4 --- /dev/null +++ b/client/executor/src/wasmtime/function_executor.rs @@ -0,0 +1,387 @@ +// Copyright 2019 Parity Technologies (UK) Ltd. +// This file is part of Substrate. + +// Substrate is free software: you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. + +// Substrate is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. + +// You should have received a copy of the GNU General Public License +// along with Substrate. If not, see . + +use crate::allocator::FreeingBumpHeapAllocator; +use crate::error::{Error, Result}; +use crate::sandbox::{self, SandboxCapabilities, SupervisorFuncIndex}; +use crate::wasmtime::util::{ + checked_range, cranelift_ir_signature, read_memory_into, write_memory_from, +}; + +use codec::{Decode, Encode}; +use cranelift_codegen::ir; +use cranelift_codegen::isa::TargetFrontendConfig; +use log::trace; +use primitives::sandbox as sandbox_primitives; +use std::{cmp, mem, ptr}; +use wasmtime_environ::translate_signature; +use wasmtime_jit::{ActionError, Compiler}; +use wasmtime_runtime::{Export, VMCallerCheckedAnyfunc, VMContext, wasmtime_call_trampoline}; +use wasm_interface::{ + FunctionContext, MemoryId, Pointer, Result as WResult, Sandbox, Signature, Value, ValueType, + WordSize, +}; + +/// Wrapper type for pointer to a Wasm table entry. +/// +/// The wrapper type is used to ensure that the function reference is valid as it must be unsafely +/// dereferenced from within the safe method `::invoke`. +#[derive(Clone, Copy)] +pub struct SupervisorFuncRef(*const VMCallerCheckedAnyfunc); + +/// The state required to construct a FunctionExecutor context. The context only lasts for one host +/// call, whereas the state is maintained for the duration of a Wasm runtime call, which may make +/// many different host calls that must share state. +/// +/// This is stored as part of the host state of the "env" Wasmtime instance. +pub struct FunctionExecutorState { + sandbox_store: sandbox::Store, + heap: FreeingBumpHeapAllocator, +} + +impl FunctionExecutorState { + /// Constructs a new `FunctionExecutorState`. + pub fn new(heap_base: u32) -> Self { + FunctionExecutorState { + sandbox_store: sandbox::Store::new(), + heap: FreeingBumpHeapAllocator::new(heap_base), + } + } + + /// Returns a mutable reference to the heap allocator. + pub fn heap(&mut self) -> &mut FreeingBumpHeapAllocator { + &mut self.heap + } +} + +/// A `FunctionExecutor` implements `FunctionContext` for making host calls from a Wasmtime +/// runtime. The `FunctionExecutor` exists only for the lifetime of the call and borrows state from +/// a longer-living `FunctionExecutorState`. +pub struct FunctionExecutor<'a> { + compiler: &'a mut Compiler, + sandbox_store: &'a mut sandbox::Store, + heap: &'a mut FreeingBumpHeapAllocator, + memory: &'a mut [u8], + table: Option<&'a [VMCallerCheckedAnyfunc]>, +} + +impl<'a> FunctionExecutor<'a> { + /// Construct a new `FunctionExecutor`. + /// + /// The vmctx MUST come from a call to a function in the "env" module. + /// The state MUST be looked up from the host state of the "env" module. + pub unsafe fn new( + vmctx: *mut VMContext, + compiler: &'a mut Compiler, + state: &'a mut FunctionExecutorState, + ) -> Result + { + let memory = match (*vmctx).lookup_global_export("memory") { + Some(Export::Memory { definition, vmctx: _, memory: _ }) => + std::slice::from_raw_parts_mut( + (*definition).base, + (*definition).current_length, + ), + _ => return Err(Error::InvalidMemoryReference), + }; + let table = match (*vmctx).lookup_global_export("__indirect_function_table") { + Some(Export::Table { definition, vmctx: _, table: _ }) => + Some(std::slice::from_raw_parts( + (*definition).base as *const VMCallerCheckedAnyfunc, + (*definition).current_elements as usize, + )), + _ => None, + }; + Ok(FunctionExecutor { + compiler, + sandbox_store: &mut state.sandbox_store, + heap: &mut state.heap, + memory, + table, + }) + } +} + +impl<'a> SandboxCapabilities for FunctionExecutor<'a> { + type SupervisorFuncRef = SupervisorFuncRef; + + fn store(&self) -> &sandbox::Store { + &self.sandbox_store + } + + fn store_mut(&mut self) -> &mut sandbox::Store { + &mut self.sandbox_store + } + + fn allocate(&mut self, len: WordSize) -> Result> { + self.heap.allocate(self.memory, len) + } + + fn deallocate(&mut self, ptr: Pointer) -> Result<()> { + self.heap.deallocate(self.memory, ptr) + } + + fn write_memory(&mut self, ptr: Pointer, data: &[u8]) -> Result<()> { + write_memory_from(self.memory, ptr, data) + } + + fn read_memory(&self, ptr: Pointer, len: WordSize) -> Result> { + let mut output = vec![0; len as usize]; + read_memory_into(self.memory, ptr, output.as_mut())?; + Ok(output) + } + + fn invoke( + &mut self, + dispatch_thunk: &Self::SupervisorFuncRef, + invoke_args_ptr: Pointer, + invoke_args_len: WordSize, + state: u32, + func_idx: SupervisorFuncIndex, + ) -> Result + { + let func_ptr = unsafe { (*dispatch_thunk.0).func_ptr }; + let vmctx = unsafe { (*dispatch_thunk.0).vmctx }; + + // The following code is based on the wasmtime_jit::Context::invoke. + let value_size = mem::size_of::(); + let (signature, mut values_vec) = generate_signature_and_args( + &[ + Value::I32(u32::from(invoke_args_ptr) as i32), + Value::I32(invoke_args_len as i32), + Value::I32(state as i32), + Value::I32(usize::from(func_idx) as i32), + ], + Some(ValueType::I64), + self.compiler.frontend_config(), + ); + + // Get the trampoline to call for this function. + let exec_code_buf = self.compiler + .get_published_trampoline(func_ptr, &signature, value_size) + .map_err(ActionError::Setup) + .map_err(Error::Wasmtime)?; + + // Call the trampoline. + if let Err(message) = unsafe { + wasmtime_call_trampoline( + vmctx, + exec_code_buf, + values_vec.as_mut_ptr() as *mut u8, + ) + } { + return Err(Error::Other(message)); + } + + // Load the return value out of `values_vec`. + Ok(unsafe { ptr::read(values_vec.as_ptr() as *const i64) }) + } +} + +impl<'a> FunctionContext for FunctionExecutor<'a> { + fn read_memory_into(&self, address: Pointer, dest: &mut [u8]) -> WResult<()> { + read_memory_into(self.memory, address, dest).map_err(|e| e.to_string()) + } + + fn write_memory(&mut self, address: Pointer, data: &[u8]) -> WResult<()> { + write_memory_from(self.memory, address, data).map_err(|e| e.to_string()) + } + + fn allocate_memory(&mut self, size: WordSize) -> WResult> { + self.heap.allocate(self.memory, size).map_err(|e| e.to_string()) + } + + fn deallocate_memory(&mut self, ptr: Pointer) -> WResult<()> { + self.heap.deallocate(self.memory, ptr).map_err(|e| e.to_string()) + } + + fn sandbox(&mut self) -> &mut dyn Sandbox { + self + } +} + +impl<'a> Sandbox for FunctionExecutor<'a> { + fn memory_get( + &mut self, + memory_id: MemoryId, + offset: WordSize, + buf_ptr: Pointer, + buf_len: WordSize, + ) -> WResult + { + let sandboxed_memory = self.sandbox_store.memory(memory_id) + .map_err(|e| e.to_string())?; + sandboxed_memory.with_direct_access(|memory| { + let len = buf_len as usize; + let src_range = match checked_range(offset as usize, len, memory.len()) { + Some(range) => range, + None => return Ok(sandbox_primitives::ERR_OUT_OF_BOUNDS), + }; + let dst_range = match checked_range(buf_ptr.into(), len, self.memory.len()) { + Some(range) => range, + None => return Ok(sandbox_primitives::ERR_OUT_OF_BOUNDS), + }; + &mut self.memory[dst_range].copy_from_slice(&memory[src_range]); + Ok(sandbox_primitives::ERR_OK) + }) + } + + fn memory_set( + &mut self, + memory_id: MemoryId, + offset: WordSize, + val_ptr: Pointer, + val_len: WordSize, + ) -> WResult + { + let sandboxed_memory = self.sandbox_store.memory(memory_id) + .map_err(|e| e.to_string())?; + sandboxed_memory.with_direct_access_mut(|memory| { + let len = val_len as usize; + let src_range = match checked_range(val_ptr.into(), len, self.memory.len()) { + Some(range) => range, + None => return Ok(sandbox_primitives::ERR_OUT_OF_BOUNDS), + }; + let dst_range = match checked_range(offset as usize, len, memory.len()) { + Some(range) => range, + None => return Ok(sandbox_primitives::ERR_OUT_OF_BOUNDS), + }; + &mut memory[dst_range].copy_from_slice(&self.memory[src_range]); + Ok(sandbox_primitives::ERR_OK) + }) + } + + fn memory_teardown(&mut self, memory_id: MemoryId) + -> WResult<()> + { + self.sandbox_store.memory_teardown(memory_id).map_err(|e| e.to_string()) + } + + fn memory_new(&mut self, initial: u32, maximum: MemoryId) -> WResult { + self.sandbox_store.new_memory(initial, maximum).map_err(|e| e.to_string()) + } + + fn invoke( + &mut self, + instance_id: u32, + export_name: &str, + args: &[u8], + return_val: Pointer, + return_val_len: u32, + state: u32, + ) -> WResult { + trace!(target: "sr-sandbox", "invoke, instance_idx={}", instance_id); + + // Deserialize arguments and convert them into wasmi types. + let args = Vec::::decode(&mut &args[..]) + .map_err(|_| "Can't decode serialized arguments for the invocation")? + .into_iter() + .map(Into::into) + .collect::>(); + + let instance = self.sandbox_store.instance(instance_id).map_err(|e| e.to_string())?; + let result = instance.invoke(export_name, &args, self, state); + + match result { + Ok(None) => Ok(sandbox_primitives::ERR_OK), + Ok(Some(val)) => { + // Serialize return value and write it back into the memory. + sandbox_primitives::ReturnValue::Value(val.into()).using_encoded(|val| { + if val.len() > return_val_len as usize { + Err("Return value buffer is too small")?; + } + FunctionContext::write_memory(self, return_val, val)?; + Ok(sandbox_primitives::ERR_OK) + }) + } + Err(_) => Ok(sandbox_primitives::ERR_EXECUTION), + } + } + + fn instance_teardown(&mut self, instance_id: u32) -> WResult<()> { + self.sandbox_store.instance_teardown(instance_id).map_err(|e| e.to_string()) + } + + fn instance_new(&mut self, dispatch_thunk_id: u32, wasm: &[u8], raw_env_def: &[u8], state: u32) + -> WResult + { + // Extract a dispatch thunk from instance's table by the specified index. + let dispatch_thunk = { + let table = self.table.as_ref() + .ok_or_else(|| "Runtime doesn't have a table; sandbox is unavailable")?; + let func_ref = table.get(dispatch_thunk_id as usize) + .ok_or_else(|| "dispatch_thunk_idx is out of the table bounds")?; + SupervisorFuncRef(func_ref) + }; + + let instance_idx_or_err_code = + match sandbox::instantiate(self, dispatch_thunk, wasm, raw_env_def, state) { + Ok(instance_idx) => instance_idx, + Err(sandbox::InstantiationError::StartTrapped) => + sandbox_primitives::ERR_EXECUTION, + Err(_) => sandbox_primitives::ERR_MODULE, + }; + + Ok(instance_idx_or_err_code as u32) + } +} + +// The storage for a Wasmtime invocation argument. +#[derive(Debug, Default, Copy, Clone)] +#[repr(C, align(8))] +struct VMInvokeArgument([u8; 8]); + +fn generate_signature_and_args( + args: &[Value], + result_type: Option, + frontend_config: TargetFrontendConfig, +) -> (ir::Signature, Vec) +{ + // This code is based on the wasmtime_jit::Context::invoke. + + let param_types = args.iter() + .map(|arg| arg.value_type()) + .collect::>(); + let signature = translate_signature( + cranelift_ir_signature( + Signature::new(param_types, result_type), + &frontend_config.default_call_conv + ), + frontend_config.pointer_type() + ); + + let mut values_vec = vec![ + VMInvokeArgument::default(); + cmp::max(args.len(), result_type.iter().len()) + ]; + + // Store the argument values into `values_vec`. + for (index, arg) in args.iter().enumerate() { + unsafe { + let ptr = values_vec.as_mut_ptr().add(index); + + match arg { + Value::I32(x) => ptr::write(ptr as *mut i32, *x), + Value::I64(x) => ptr::write(ptr as *mut i64, *x), + Value::F32(x) => ptr::write(ptr as *mut u32, *x), + Value::F64(x) => ptr::write(ptr as *mut u64, *x), + } + } + } + + (signature, values_vec) +} + diff --git a/core/client/src/block_builder/mod.rs b/client/executor/src/wasmtime/mod.rs similarity index 75% rename from core/client/src/block_builder/mod.rs rename to client/executor/src/wasmtime/mod.rs index 7f617044a42cf..7f442417ab849 100644 --- a/core/client/src/block_builder/mod.rs +++ b/client/executor/src/wasmtime/mod.rs @@ -1,4 +1,4 @@ -// Copyright 2018-2019 Parity Technologies (UK) Ltd. +// Copyright 2019 Parity Technologies (UK) Ltd. // This file is part of Substrate. // Substrate is free software: you can redistribute it and/or modify @@ -14,10 +14,11 @@ // You should have received a copy of the GNU General Public License // along with Substrate. If not, see . -//! Utility struct to build a block. +///! Defines a `WasmRuntime` that uses the Wasmtime JIT to execute. -#[cfg(feature = "std")] -mod block_builder; -#[cfg(feature = "std")] -pub use self::block_builder::*; -pub mod api; +mod function_executor; +mod runtime; +mod trampoline; +mod util; + +pub use runtime::create_instance; diff --git a/client/executor/src/wasmtime/runtime.rs b/client/executor/src/wasmtime/runtime.rs new file mode 100644 index 0000000000000..1a7385cc46025 --- /dev/null +++ b/client/executor/src/wasmtime/runtime.rs @@ -0,0 +1,372 @@ +// Copyright 2019 Parity Technologies (UK) Ltd. +// This file is part of Substrate. + +// Substrate is free software: you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. + +// Substrate is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. + +// You should have received a copy of the GNU General Public License +// along with Substrate. If not, see . + +//! Defines the compiled Wasm runtime that uses Wasmtime internally. + +use crate::error::{Error, Result, WasmError}; +use crate::wasm_runtime::WasmRuntime; +use crate::wasm_utils::interpret_runtime_api_result; +use crate::wasmtime::function_executor::FunctionExecutorState; +use crate::wasmtime::trampoline::{EnvState, make_trampoline}; +use crate::wasmtime::util::{cranelift_ir_signature, read_memory_into, write_memory_from}; +use crate::Externalities; + +use cranelift_codegen::ir; +use cranelift_codegen::isa::TargetIsa; +use cranelift_entity::{EntityRef, PrimaryMap}; +use cranelift_frontend::FunctionBuilderContext; +use cranelift_wasm::DefinedFuncIndex; +use std::cell::RefCell; +use std::collections::HashMap; +use std::convert::TryFrom; +use std::rc::Rc; +use wasm_interface::{Pointer, WordSize, Function}; +use wasmtime_environ::{Module, translate_signature}; +use wasmtime_jit::{ + ActionOutcome, ActionError, CodeMemory, CompilationStrategy, CompiledModule, Compiler, Context, + SetupError, RuntimeValue, +}; +use wasmtime_runtime::{Export, Imports, InstanceHandle, VMFunctionBody}; + +/// A `WasmRuntime` implementation using the Wasmtime JIT to compile the runtime module to native +/// and execute the compiled code. +pub struct WasmtimeRuntime { + module: CompiledModule, + context: Context, + max_heap_pages: Option, + heap_pages: u32, + /// The host functions registered for this instance. + host_functions: Vec<&'static dyn Function>, +} + +impl WasmRuntime for WasmtimeRuntime { + fn update_heap_pages(&mut self, heap_pages: u64) -> bool { + match heap_pages_valid(heap_pages, self.max_heap_pages) { + Some(heap_pages) => { + self.heap_pages = heap_pages; + true + } + None => false, + } + } + + fn host_functions(&self) -> &[&'static dyn Function] { + &self.host_functions + } + + fn call(&mut self, ext: &mut dyn Externalities, method: &str, data: &[u8]) -> Result> { + call_method( + &mut self.context, + &mut self.module, + ext, + method, + data, + self.heap_pages, + ) + } +} + +/// Create a new `WasmtimeRuntime` given the code. This function performs translation from Wasm to +/// machine code, which can be computationally heavy. +pub fn create_instance( + code: &[u8], + heap_pages: u64, + host_functions: Vec<&'static dyn Function>, +) -> std::result::Result { + let (compiled_module, context) = create_compiled_unit(code, &host_functions)?; + + // Inspect the module for the min and max memory sizes. + let (min_memory_size, max_memory_size) = { + let module = compiled_module.module_ref(); + let memory_index = match module.exports.get("memory") { + Some(wasmtime_environ::Export::Memory(memory_index)) => *memory_index, + _ => return Err(WasmError::InvalidMemory), + }; + let memory_plan = module.memory_plans.get(memory_index) + .expect("memory_index is retrieved from the module's exports map; qed"); + (memory_plan.memory.minimum, memory_plan.memory.maximum) + }; + + // Check that heap_pages is within the allowed range. + let max_heap_pages = max_memory_size.map(|max| max.saturating_sub(min_memory_size)); + let heap_pages = heap_pages_valid(heap_pages, max_heap_pages) + .ok_or_else(|| WasmError::InvalidHeapPages)?; + + Ok(WasmtimeRuntime { + module: compiled_module, + context, + max_heap_pages, + heap_pages, + host_functions, + }) +} + +fn create_compiled_unit( + code: &[u8], + host_functions: &[&'static dyn Function], +) -> std::result::Result<(CompiledModule, Context), WasmError> { + let compilation_strategy = CompilationStrategy::Cranelift; + + let compiler = new_compiler(compilation_strategy)?; + let mut context = Context::new(Box::new(compiler)); + + // Enable/disable producing of debug info. + context.set_debug_info(false); + + // Instantiate and link the env module. + let global_exports = context.get_global_exports(); + let compiler = new_compiler(compilation_strategy)?; + let env_module = instantiate_env_module(global_exports, compiler, host_functions)?; + context.name_instance("env".to_owned(), env_module); + + // Compile the wasm module. + let module = context.compile_module(&code) + .map_err(WasmError::WasmtimeSetup)?; + + Ok((module, context)) +} + +/// Call a function inside a precompiled Wasm module. +fn call_method( + context: &mut Context, + module: &mut CompiledModule, + ext: &mut dyn Externalities, + method: &str, + data: &[u8], + heap_pages: u32, +) -> Result> { + // Old exports get clobbered in `InstanceHandle::new` if we don't explicitly remove them first. + // + // The global exports mechanism is temporary in Wasmtime and expected to be removed. + // https://github.com/CraneStation/wasmtime/issues/332 + clear_globals(&mut *context.get_global_exports().borrow_mut()); + + let mut instance = module.instantiate() + .map_err(SetupError::Instantiate) + .map_err(ActionError::Setup) + .map_err(Error::Wasmtime)?; + + // Ideally there would be a way to set the heap pages during instantiation rather than + // growing the memory after the fact. Currently this may require an additional mmap and copy. + // However, the wasmtime API doesn't support modifying the size of memory on instantiation + // at this time. + grow_memory(&mut instance, heap_pages)?; + + // Initialize the function executor state. + let heap_base = get_heap_base(&instance)?; + let executor_state = FunctionExecutorState::new(heap_base); + reset_env_state_and_take_trap(context, Some(executor_state))?; + + // Write the input data into guest memory. + let (data_ptr, data_len) = inject_input_data(context, &mut instance, data)?; + let args = [RuntimeValue::I32(u32::from(data_ptr) as i32), RuntimeValue::I32(data_len as i32)]; + + // Invoke the function in the runtime. + let outcome = externalities::set_and_run_with_externalities(ext, || { + context + .invoke(&mut instance, method, &args[..]) + .map_err(Error::Wasmtime) + })?; + let trap_error = reset_env_state_and_take_trap(context, None)?; + let (output_ptr, output_len) = match outcome { + ActionOutcome::Returned { values } => match values.as_slice() { + [RuntimeValue::I64(retval)] => interpret_runtime_api_result(*retval), + _ => return Err(Error::InvalidReturn), + } + ActionOutcome::Trapped { message } => return Err(trap_error.unwrap_or_else( + || format!("Wasm execution trapped: {}", message).into() + )), + }; + + // Read the output data from guest memory. + let mut output = vec![0; output_len as usize]; + let memory = get_memory_mut(&mut instance)?; + read_memory_into(memory, output_ptr, &mut output)?; + Ok(output) +} + +/// The implementation is based on wasmtime_wasi::instantiate_wasi. +fn instantiate_env_module( + global_exports: Rc>>>, + compiler: Compiler, + host_functions: &[&'static dyn Function], +) -> std::result::Result +{ + let isa = target_isa()?; + let pointer_type = isa.pointer_type(); + let call_conv = isa.default_call_conv(); + + let mut fn_builder_ctx = FunctionBuilderContext::new(); + let mut module = Module::new(); + let mut finished_functions = >::new(); + let mut code_memory = CodeMemory::new(); + + for function in host_functions { + let sig = translate_signature( + cranelift_ir_signature(function.signature(), &call_conv), + pointer_type + ); + let sig_id = module.signatures.push(sig.clone()); + let func_id = module.functions.push(sig_id); + module + .exports + .insert(function.name().to_string(), wasmtime_environ::Export::Function(func_id)); + + let trampoline = make_trampoline( + isa.as_ref(), + &mut code_memory, + &mut fn_builder_ctx, + func_id.index() as u32, + &sig, + )?; + finished_functions.push(trampoline); + } + + code_memory.publish(); + + let imports = Imports::none(); + let data_initializers = Vec::new(); + let signatures = PrimaryMap::new(); + let env_state = EnvState::new(code_memory, compiler, host_functions); + + let result = InstanceHandle::new( + Rc::new(module), + global_exports, + finished_functions.into_boxed_slice(), + imports, + &data_initializers, + signatures.into_boxed_slice(), + None, + Box::new(env_state), + ); + result.map_err(|e| WasmError::WasmtimeSetup(SetupError::Instantiate(e))) +} + +/// Build a new TargetIsa for the host machine. +fn target_isa() -> std::result::Result, WasmError> { + let isa_builder = cranelift_native::builder() + .map_err(WasmError::MissingCompilerSupport)?; + let flag_builder = cranelift_codegen::settings::builder(); + Ok(isa_builder.finish(cranelift_codegen::settings::Flags::new(flag_builder))) +} + +fn new_compiler(strategy: CompilationStrategy) -> std::result::Result { + let isa = target_isa()?; + Ok(Compiler::new(isa, strategy)) +} + +fn clear_globals(global_exports: &mut HashMap>) { + global_exports.remove("memory"); + global_exports.remove("__heap_base"); + global_exports.remove("__indirect_function_table"); +} + +fn grow_memory(instance: &mut InstanceHandle, pages: u32) -> Result<()> { + // This is safe to wrap in an unsafe block as: + // - The result of the `lookup_immutable` call is not mutated + // - The definition pointer is returned by a lookup on a valid instance + let memory_index = unsafe { + match instance.lookup_immutable("memory") { + Some(Export::Memory { definition, vmctx: _, memory: _ }) => + instance.memory_index(&*definition), + _ => return Err(Error::InvalidMemoryReference), + } + }; + instance.memory_grow(memory_index, pages) + .map(|_| ()) + .ok_or_else(|| "requested heap_pages would exceed maximum memory size".into()) +} + +fn get_env_state(context: &mut Context) -> Result<&mut EnvState> { + let env_instance = context.get_instance("env") + .map_err(|err| format!("cannot find \"env\" module: {}", err))?; + env_instance + .host_state() + .downcast_mut::() + .ok_or_else(|| "cannot get \"env\" module host state".into()) +} + +fn reset_env_state_and_take_trap( + context: &mut Context, + executor_state: Option, +) -> Result> +{ + let env_state = get_env_state(context)?; + env_state.executor_state = executor_state; + Ok(env_state.take_trap()) +} + +fn inject_input_data( + context: &mut Context, + instance: &mut InstanceHandle, + data: &[u8], +) -> Result<(Pointer, WordSize)> { + let env_state = get_env_state(context)?; + let executor_state = env_state.executor_state + .as_mut() + .ok_or_else(|| "cannot get \"env\" module executor state")?; + + let memory = get_memory_mut(instance)?; + + let data_len = data.len() as WordSize; + let data_ptr = executor_state.heap().allocate(memory, data_len)?; + write_memory_from(memory, data_ptr, data)?; + Ok((data_ptr, data_len)) +} + +fn get_memory_mut(instance: &mut InstanceHandle) -> Result<&mut [u8]> { + match instance.lookup("memory") { + // This is safe to wrap in an unsafe block as: + // - The definition pointer is returned by a lookup on a valid instance and thus points to + // a valid memory definition + Some(Export::Memory { definition, vmctx: _, memory: _ }) => unsafe { + Ok(std::slice::from_raw_parts_mut( + (*definition).base, + (*definition).current_length, + )) + }, + _ => Err(Error::InvalidMemoryReference), + } +} + +fn get_heap_base(instance: &InstanceHandle) -> Result { + // This is safe to wrap in an unsafe block as: + // - The result of the `lookup_immutable` call is not mutated + // - The definition pointer is returned by a lookup on a valid instance + // - The defined value is checked to be an I32, which can be read safely as a u32 + unsafe { + match instance.lookup_immutable("__heap_base") { + Some(Export::Global { definition, vmctx: _, global }) + if global.ty == ir::types::I32 => + Ok(*(*definition).as_u32()), + _ => return Err(Error::HeapBaseNotFoundOrInvalid), + } + } +} + +/// Checks whether the heap_pages parameter is within the valid range and converts it to a u32. +/// Returns None if heaps_pages in not in range. +fn heap_pages_valid(heap_pages: u64, max_heap_pages: Option) + -> Option +{ + let heap_pages = u32::try_from(heap_pages).ok()?; + if let Some(max_heap_pages) = max_heap_pages { + if heap_pages > max_heap_pages { + return None; + } + } + Some(heap_pages) +} diff --git a/client/executor/src/wasmtime/trampoline.rs b/client/executor/src/wasmtime/trampoline.rs new file mode 100644 index 0000000000000..f6561f485d8e9 --- /dev/null +++ b/client/executor/src/wasmtime/trampoline.rs @@ -0,0 +1,347 @@ +// Copyright 2019 Parity Technologies (UK) Ltd. +// This file is part of Substrate. + +// Substrate is free software: you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. + +// Substrate is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. + +// You should have received a copy of the GNU General Public License +// along with Substrate. If not, see . + +//! The trampoline is the dynamically generated entry point to a runtime host call. +//! +//! This code is based on and large parts are copied from wasmtime's +//! wasmtime-api/src/trampoline/func.rs. + +use cranelift_codegen::{Context, binemit, ir, isa}; +use cranelift_codegen::ir::{InstBuilder, StackSlotData, StackSlotKind, TrapCode}; +use cranelift_frontend::{FunctionBuilder, FunctionBuilderContext}; +use wasmtime_jit::{CodeMemory, Compiler}; +use wasmtime_runtime::{VMContext, VMFunctionBody}; +use wasm_interface::{Function, Value, ValueType}; +use std::{cmp, panic::{self, AssertUnwindSafe}, ptr}; + +use crate::error::{Error, WasmError}; +use crate::wasmtime::function_executor::{FunctionExecutorState, FunctionExecutor}; + +const CALL_SUCCESS: u32 = 0; +const CALL_FAILED_WITH_ERROR: u32 = 1; +const CALL_WITH_BAD_HOST_STATE: u32 = 2; + +/// A code to trap with that indicates a host call error. +const TRAP_USER_CODE: u16 = 0; + +/// The only Wasm types allowed in host function signatures (I32, I64, F32, F64) are all +/// represented in at most 8 bytes. +const MAX_WASM_TYPE_SIZE: usize = 8; + +/// The top-level host state of the "env" module. This state is used by the trampoline function to +/// construct a `FunctionExecutor` which can execute the host call. +pub struct EnvState { + host_functions: Vec<&'static dyn Function>, + compiler: Compiler, + // The code memory must be kept around on the state to prevent it from being dropped. + #[allow(dead_code)] + code_memory: CodeMemory, + trap: Option, + /// The executor state stored across host calls during a single Wasm runtime call. + /// During a runtime call, this MUST be `Some`. + pub executor_state: Option, +} + +impl EnvState { + /// Construct a new `EnvState` which owns the given code memory. + pub fn new( + code_memory: CodeMemory, + compiler: Compiler, + host_functions: &[&'static dyn Function], + ) -> Self { + EnvState { + trap: None, + compiler, + code_memory, + executor_state: None, + host_functions: host_functions.to_vec(), + } + } + + /// Resets the trap error to None and returns the current value. + pub fn take_trap(&mut self) -> Option { + self.trap.take() + } +} + +/// This is called by the dynamically generated trampoline taking the function index and reference +/// to the call arguments on the stack as arguments. Returns zero on success and a non-zero value +/// on failure. +unsafe extern "C" fn stub_fn(vmctx: *mut VMContext, func_index: u32, values_vec: *mut i64) -> u32 { + if let Some(state) = (*vmctx).host_state().downcast_mut::() { + match stub_fn_inner( + vmctx, + &state.host_functions, + &mut state.compiler, + state.executor_state.as_mut(), + func_index, + values_vec, + ) { + Ok(()) => CALL_SUCCESS, + Err(err) => { + state.trap = Some(err); + CALL_FAILED_WITH_ERROR + } + } + } else { + // Well, we can't even set a trap message, so we'll just exit without one. + CALL_WITH_BAD_HOST_STATE + } +} + +/// Implements most of the logic in `stub_fn` but returning a `Result` instead of an integer error +/// for the sake of readability. +unsafe fn stub_fn_inner( + vmctx: *mut VMContext, + externals: &[&dyn Function], + compiler: &mut Compiler, + executor_state: Option<&mut FunctionExecutorState>, + func_index: u32, + values_vec: *mut i64, +) -> Result<(), Error> { + let func = externals.get(func_index as usize) + .ok_or_else(|| format!("call to undefined external function with index {}", func_index))?; + let executor_state = executor_state + .ok_or_else(|| "executor state is None during call to external function")?; + + // Build the external function context. + let mut context = FunctionExecutor::new(vmctx, compiler, executor_state)?; + let mut context = AssertUnwindSafe(&mut context); + + // Execute and write output back to the stack. + let return_val = panic::catch_unwind(move || { + let signature = func.signature(); + + // Read the arguments from the stack. + let mut args = signature.args.iter() + .enumerate() + .map(|(i, ¶m_type)| read_value_from(values_vec.offset(i as isize), param_type)); + + func.execute(&mut **context, &mut args) + }); + + match return_val { + Ok(ret_val) => { + if let Some(val) = ret_val + .map_err(|e| Error::FunctionExecution(func.name().to_string(), e))? { + write_value_to(values_vec, val); + } + + Ok(()) + }, + Err(e) => { + let message = if let Some(err) = e.downcast_ref::() { + err.to_string() + } else if let Some(err) = e.downcast_ref::<&str>() { + err.to_string() + } else { + "Panicked without any further information!".into() + }; + + Err(Error::FunctionExecution(func.name().to_string(), message)) + } + } +} + +/// Create a trampoline for invoking a host function. +/// +/// The trampoline is a dynamically generated entry point to a runtime host call. The function is +/// generated by manually constructing Cranelift IR and using the Cranelift compiler. The +/// trampoline embeds the function index as a constant and delegates to a stub function in Rust, +/// which takes the function index and a memory reference to the stack arguments and return value +/// slots. +/// +/// This code is of modified copy of wasmtime's wasmtime-api/src/trampoline/func.rs. +pub fn make_trampoline( + isa: &dyn isa::TargetIsa, + code_memory: &mut CodeMemory, + fn_builder_ctx: &mut FunctionBuilderContext, + func_index: u32, + signature: &ir::Signature, +) -> Result<*const VMFunctionBody, WasmError> { + // Mostly reverse copy of the similar method from wasmtime's + // wasmtime-jit/src/compiler.rs. + let pointer_type = isa.pointer_type(); + let mut stub_sig = ir::Signature::new(isa.frontend_config().default_call_conv); + + // Ensure that the first parameter of the generated function is the `VMContext` pointer. + assert_eq!( + signature.params[0], + ir::AbiParam::special(pointer_type, ir::ArgumentPurpose::VMContext) + ); + + // Add the `vmctx` parameter. + stub_sig.params.push(ir::AbiParam::special( + pointer_type, + ir::ArgumentPurpose::VMContext, + )); + + // Add the `func_index` parameter. + stub_sig.params.push(ir::AbiParam::new(ir::types::I32)); + + // Add the `values_vec` parameter. + stub_sig.params.push(ir::AbiParam::new(pointer_type)); + + // Add error/trap return. + stub_sig.returns.push(ir::AbiParam::new(ir::types::I32)); + + // Each parameter and return value gets a 64-bit (8-byte) wide slot on the stack, as that is + // large enough to fit all Wasm primitive types that can be used in host function signatures. + // The `VMContext` pointer, which is a parameter of the function signature, is excluded as it + // is passed directly to the stub function rather than being looked up on the caller stack from + // the `values_vec` pointer. + let values_vec_len = cmp::max(signature.params.len() - 1, signature.returns.len()); + let values_vec_size = (MAX_WASM_TYPE_SIZE * values_vec_len) as u32; + + let mut context = Context::new(); + context.func = + ir::Function::with_name_signature(ir::ExternalName::user(0, 0), signature.clone()); + + let ss = context.func.create_stack_slot(StackSlotData::new( + StackSlotKind::ExplicitSlot, + values_vec_size, + )); + + { + let mut builder = FunctionBuilder::new(&mut context.func, fn_builder_ctx); + let block0 = builder.create_ebb(); + + builder.append_ebb_params_for_function_params(block0); + builder.switch_to_block(block0); + builder.seal_block(block0); + + let values_vec_ptr_val = builder.ins().stack_addr(pointer_type, ss, 0); + let mflags = ir::MemFlags::trusted(); + for i in 1..signature.params.len() { + let val = builder.func.dfg.ebb_params(block0)[i]; + builder.ins().store( + mflags, + val, + values_vec_ptr_val, + ((i - 1) * MAX_WASM_TYPE_SIZE) as i32, + ); + } + + let vmctx_ptr_val = builder.func.dfg.ebb_params(block0)[0]; + let func_index_val = builder.ins().iconst(ir::types::I32, func_index as i64); + + let callee_args = vec![vmctx_ptr_val, func_index_val, values_vec_ptr_val]; + + let new_sig = builder.import_signature(stub_sig.clone()); + + let callee_value = builder + .ins() + .iconst(pointer_type, stub_fn as *const VMFunctionBody as i64); + let call = builder + .ins() + .call_indirect(new_sig, callee_value, &callee_args); + + let call_result = builder.func.dfg.inst_results(call)[0]; + builder.ins().trapnz(call_result, TrapCode::User(TRAP_USER_CODE)); + + let mflags = ir::MemFlags::trusted(); + let mut results = Vec::new(); + for (i, r) in signature.returns.iter().enumerate() { + let load = builder.ins().load( + r.value_type, + mflags, + values_vec_ptr_val, + (i * MAX_WASM_TYPE_SIZE) as i32, + ); + results.push(load); + } + builder.ins().return_(&results); + builder.finalize() + } + + let mut code_buf: Vec = Vec::new(); + let mut reloc_sink = RelocSink; + let mut trap_sink = binemit::NullTrapSink {}; + let mut stackmap_sink = binemit::NullStackmapSink {}; + context + .compile_and_emit( + isa, + &mut code_buf, + &mut reloc_sink, + &mut trap_sink, + &mut stackmap_sink, + ) + .map_err(|e| WasmError::Instantiation(format!("failed to compile trampoline: {}", e)))?; + + let func_ref = code_memory + .allocate_copy_of_byte_slice(&code_buf) + .map_err(|e| WasmError::Instantiation(format!("failed to allocate code memory: {}", e)))?; + + Ok(func_ref.as_ptr()) +} + +/// We don't expect trampoline compilation to produce any relocations, so +/// this `RelocSink` just asserts that it doesn't recieve any. +struct RelocSink; + +impl binemit::RelocSink for RelocSink { + fn reloc_ebb( + &mut self, + _offset: binemit::CodeOffset, + _reloc: binemit::Reloc, + _ebb_offset: binemit::CodeOffset, + ) { + panic!("trampoline compilation should not produce ebb relocs"); + } + fn reloc_external( + &mut self, + _offset: binemit::CodeOffset, + _reloc: binemit::Reloc, + _name: &ir::ExternalName, + _addend: binemit::Addend, + ) { + panic!("trampoline compilation should not produce external symbol relocs"); + } + fn reloc_constant( + &mut self, + _code_offset: binemit::CodeOffset, + _reloc: binemit::Reloc, + _constant_offset: ir::ConstantOffset, + ) { + panic!("trampoline compilation should not produce constant relocs"); + } + fn reloc_jt( + &mut self, + _offset: binemit::CodeOffset, + _reloc: binemit::Reloc, + _jt: ir::JumpTable, + ) { + panic!("trampoline compilation should not produce jump table relocs"); + } +} + +unsafe fn write_value_to(p: *mut i64, val: Value) { + match val { + Value::I32(i) => ptr::write(p as *mut i32, i), + Value::I64(i) => ptr::write(p as *mut i64, i), + Value::F32(u) => ptr::write(p as *mut u32, u), + Value::F64(u) => ptr::write(p as *mut u64, u), + } +} + +unsafe fn read_value_from(p: *const i64, ty: ValueType) -> Value { + match ty { + ValueType::I32 => Value::I32(ptr::read(p as *const i32)), + ValueType::I64 => Value::I64(ptr::read(p as *const i64)), + ValueType::F32 => Value::F32(ptr::read(p as *const u32)), + ValueType::F64 => Value::F64(ptr::read(p as *const u64)), + } +} diff --git a/client/executor/src/wasmtime/util.rs b/client/executor/src/wasmtime/util.rs new file mode 100644 index 0000000000000..874ccc8c85fbe --- /dev/null +++ b/client/executor/src/wasmtime/util.rs @@ -0,0 +1,113 @@ +// Copyright 2019 Parity Technologies (UK) Ltd. +// This file is part of Substrate. + +// Substrate is free software: you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. + +// Substrate is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. + +// You should have received a copy of the GNU General Public License +// along with Substrate. If not, see . + +use crate::error::{Error, Result}; + +use cranelift_codegen::{ir, isa}; +use std::ops::Range; +use wasm_interface::{Pointer, Signature, ValueType}; + +/// Read data from a slice of memory into a destination buffer. +/// +/// Returns an error if the read would go out of the memory bounds. +pub fn read_memory_into(memory: &[u8], address: Pointer, dest: &mut [u8]) -> Result<()> { + let range = checked_range(address.into(), dest.len(), memory.len()) + .ok_or_else(|| Error::Other("memory read is out of bounds".into()))?; + dest.copy_from_slice(&memory[range]); + Ok(()) +} + +/// Write data to a slice of memory. +/// +/// Returns an error if the write would go out of the memory bounds. +pub fn write_memory_from(memory: &mut [u8], address: Pointer, data: &[u8]) -> Result<()> { + let range = checked_range(address.into(), data.len(), memory.len()) + .ok_or_else(|| Error::Other("memory write is out of bounds".into()))?; + &mut memory[range].copy_from_slice(data); + Ok(()) +} + +/// Construct a range from an offset to a data length after the offset. +/// Returns None if the end of the range would exceed some maximum offset. +pub fn checked_range(offset: usize, len: usize, max: usize) -> Option> { + let end = offset.checked_add(len)?; + if end <= max { + Some(offset..end) + } else { + None + } +} + +/// Convert a wasm_interface Signature into a cranelift_codegen Signature. +pub fn cranelift_ir_signature(signature: Signature, call_conv: &isa::CallConv) -> ir::Signature { + ir::Signature { + params: signature.args.iter() + .map(cranelift_ir_type) + .map(ir::AbiParam::new) + .collect(), + returns: signature.return_value.iter() + .map(cranelift_ir_type) + .map(ir::AbiParam::new) + .collect(), + call_conv: call_conv.clone(), + } +} + +/// Convert a wasm_interface ValueType into a cranelift_codegen Type. +pub fn cranelift_ir_type(value_type: &ValueType) -> ir::types::Type { + match value_type { + ValueType::I32 => ir::types::I32, + ValueType::I64 => ir::types::I64, + ValueType::F32 => ir::types::F32, + ValueType::F64 => ir::types::F64, + } +} + +#[cfg(test)] +mod tests { + use super::*; + use assert_matches::assert_matches; + + #[test] + fn test_read_memory_into() { + let mut memory = [0; 20]; + let mut dest = [0; 5]; + + &mut memory[15..20].copy_from_slice(b"hello"); + + read_memory_into(&memory[..], Pointer::new(15), &mut dest[..]).unwrap(); + + // Test that out of bounds read fails. + assert_matches!( + read_memory_into(&memory[..], Pointer::new(16), &mut dest[..]), + Err(Error::Other(_)) + ) + } + + #[test] + fn test_write_memory_from() { + let mut memory = [0; 20]; + let data = b"hello"; + + write_memory_from(&mut memory[..], Pointer::new(15), data).unwrap(); + + // Test that out of bounds write fails. + assert_matches!( + write_memory_from(&mut memory[..], Pointer::new(16), data), + Err(Error::Other(_)) + ) + } +} diff --git a/core/finality-grandpa/Cargo.toml b/client/finality-grandpa/Cargo.toml similarity index 57% rename from core/finality-grandpa/Cargo.toml rename to client/finality-grandpa/Cargo.toml index 84178c8a78fd5..b42c81cf76a9f 100644 --- a/core/finality-grandpa/Cargo.toml +++ b/client/finality-grandpa/Cargo.toml @@ -5,7 +5,7 @@ authors = ["Parity Technologies "] edition = "2018" [dependencies] -fork-tree = { path = "../../core/utils/fork-tree" } +fork-tree = { path = "../../utils/fork-tree" } futures = "0.1.29" futures03 = { package = "futures-preview", version = "0.3.0-alpha.19", features = ["compat"] } log = "0.4.8" @@ -14,26 +14,32 @@ tokio-executor = "0.1.8" tokio-timer = "0.2.11" rand = "0.7.2" codec = { package = "parity-scale-codec", version = "1.0.0", features = ["derive"] } -sr-primitives = { path = "../sr-primitives" } -consensus_common = { package = "substrate-consensus-common", path = "../consensus/common" } -primitives = { package = "substrate-primitives", path = "../primitives" } +sr-primitives = { path = "../../primitives/sr-primitives" } +consensus_common = { package = "substrate-consensus-common", path = "../../primitives/consensus/common" } +primitives = { package = "substrate-primitives", path = "../../primitives/core" } substrate-telemetry = { path = "../telemetry" } +substrate-prometheus= { path = "../prometheus"} keystore = { package = "substrate-keystore", path = "../keystore" } serde_json = "1.0.41" -client = { package = "substrate-client", path = "../client" } -header-metadata = { package = "substrate-header-metadata", path = "../client/header-metadata" } -inherents = { package = "substrate-inherents", path = "../../core/inherents" } +client-api = { package = "substrate-client-api", path = "../api" } +client = { package = "substrate-client", path = "../" } +header-metadata = { package = "substrate-header-metadata", path = "../header-metadata" } +inherents = { package = "substrate-inherents", path = "../../primitives/inherents" } network = { package = "substrate-network", path = "../network" } -srml-finality-tracker = { path = "../../srml/finality-tracker" } -fg_primitives = { package = "substrate-finality-grandpa-primitives", path = "primitives" } +paint-finality-tracker = { path = "../../paint/finality-tracker" } +fg_primitives = { package = "substrate-finality-grandpa-primitives", path = "../../primitives/finality-grandpa" } grandpa = { package = "finality-grandpa", version = "0.9.0", features = ["derive-codec"] } +substrate-prometheus= { path = "../prometheus"} + [dev-dependencies] grandpa = { package = "finality-grandpa", version = "0.9.0", features = ["derive-codec", "test-helpers"] } network = { package = "substrate-network", path = "../network", features = ["test-helpers"] } -keyring = { package = "substrate-keyring", path = "../keyring" } -test-client = { package = "substrate-test-runtime-client", path = "../test-runtime/client"} -babe_primitives = { package = "substrate-consensus-babe-primitives", path = "../consensus/babe/primitives" } +keyring = { package = "substrate-keyring", path = "../../primitives/keyring" } +test-client = { package = "substrate-test-runtime-client", path = "../../test/utils/runtime/client"} +babe_primitives = { package = "substrate-consensus-babe-primitives", path = "../../primitives/consensus/babe" } +state_machine = { package = "substrate-state-machine", path = "../../primitives/state-machine" } env_logger = "0.7.0" tokio = "0.1.22" tempfile = "3.1.0" +sr-api = { path = "../../primitives/sr-api" } diff --git a/core/finality-grandpa/src/authorities.rs b/client/finality-grandpa/src/authorities.rs similarity index 99% rename from core/finality-grandpa/src/authorities.rs rename to client/finality-grandpa/src/authorities.rs index 9b83c9feb6871..263f2dc076e39 100644 --- a/core/finality-grandpa/src/authorities.rs +++ b/client/finality-grandpa/src/authorities.rs @@ -22,7 +22,7 @@ use grandpa::voter_set::VoterSet; use codec::{Encode, Decode}; use log::{debug, info}; use substrate_telemetry::{telemetry, CONSENSUS_INFO}; -use fg_primitives::AuthorityId; +use fg_primitives::{AuthorityId, AuthorityList}; use std::cmp::Ord; use std::fmt::Debug; @@ -86,7 +86,7 @@ pub(crate) struct Status { /// A set of authorities. #[derive(Debug, Clone, Encode, Decode, PartialEq)] pub(crate) struct AuthoritySet { - pub(crate) current_authorities: Vec<(AuthorityId, u64)>, + pub(crate) current_authorities: AuthorityList, pub(crate) set_id: u64, // Tree of pending standard changes across forks. Standard changes are // enacted on finality and must be enacted (i.e. finalized) in-order across @@ -103,7 +103,7 @@ where H: PartialEq, N: Ord, { /// Get a genesis set with given authorities. - pub(crate) fn genesis(initial: Vec<(AuthorityId, u64)>) -> Self { + pub(crate) fn genesis(initial: AuthorityList) -> Self { AuthoritySet { current_authorities: initial, set_id: 0, @@ -390,7 +390,7 @@ pub(crate) enum DelayKind { #[derive(Debug, Clone, Encode, PartialEq)] pub(crate) struct PendingChange { /// The new authorities and weights to apply. - pub(crate) next_authorities: Vec<(AuthorityId, u64)>, + pub(crate) next_authorities: AuthorityList, /// How deep in the chain the announcing block must be /// before the change is applied. pub(crate) delay: N, diff --git a/core/finality-grandpa/src/aux_schema.rs b/client/finality-grandpa/src/aux_schema.rs similarity index 97% rename from core/finality-grandpa/src/aux_schema.rs rename to client/finality-grandpa/src/aux_schema.rs index a2b05a0cd60e1..9857367dc99e6 100644 --- a/core/finality-grandpa/src/aux_schema.rs +++ b/client/finality-grandpa/src/aux_schema.rs @@ -19,13 +19,13 @@ use std::fmt::Debug; use std::sync::Arc; use codec::{Encode, Decode}; -use client::backend::AuxStore; -use client::error::{Result as ClientResult, Error as ClientError}; +use client_api::backend::AuxStore; +use client_api::error::{Result as ClientResult, Error as ClientError}; use fork_tree::ForkTree; use grandpa::round::State as RoundState; use sr_primitives::traits::{Block as BlockT, NumberFor}; use log::{info, warn}; -use fg_primitives::{AuthorityId, AuthorityWeight, SetId, RoundNumber}; +use fg_primitives::{AuthorityList, SetId, RoundNumber}; use crate::authorities::{AuthoritySet, SharedAuthoritySet, PendingChange, DelayKind}; use crate::consensus_changes::{SharedConsensusChanges, ConsensusChanges}; @@ -55,7 +55,7 @@ type V0VoterSetState = (RoundNumber, RoundState); #[derive(Debug, Clone, Encode, Decode, PartialEq)] struct V0PendingChange { - next_authorities: Vec<(AuthorityId, AuthorityWeight)>, + next_authorities: AuthorityList, delay: N, canon_height: N, canon_hash: H, @@ -63,7 +63,7 @@ struct V0PendingChange { #[derive(Debug, Clone, Encode, Decode, PartialEq)] struct V0AuthoritySet { - current_authorities: Vec<(AuthorityId, AuthorityWeight)>, + current_authorities: AuthorityList, set_id: SetId, pending_changes: Vec>, } @@ -266,7 +266,7 @@ pub(crate) fn load_persistent( -> ClientResult> where B: AuxStore, - G: FnOnce() -> ClientResult>, + G: FnOnce() -> ClientResult, { let version: Option = load_decode(backend, VERSION_KEY)?; let consensus_changes = load_decode(backend, CONSENSUS_CHANGES_KEY)? @@ -426,6 +426,7 @@ pub(crate) fn load_authorities(backend: &B) #[cfg(test)] mod test { + use fg_primitives::AuthorityId; use primitives::H256; use test_client; use super::*; diff --git a/core/finality-grandpa/src/communication/gossip.rs b/client/finality-grandpa/src/communication/gossip.rs similarity index 85% rename from core/finality-grandpa/src/communication/gossip.rs rename to client/finality-grandpa/src/communication/gossip.rs index efcd1d48c67f7..7758de6afa70d 100644 --- a/core/finality-grandpa/src/communication/gossip.rs +++ b/client/finality-grandpa/src/communication/gossip.rs @@ -92,6 +92,7 @@ use substrate_telemetry::{telemetry, CONSENSUS_DEBUG}; use log::{trace, debug, warn}; use futures::prelude::*; use futures::sync::mpsc; +use rand::Rng; use crate::{environment, CatchUp, CompactCommit, SignedMessage}; use super::{cost, benefit, Round, SetId}; @@ -483,6 +484,14 @@ impl Peers { fn peer<'a>(&'a self, who: &PeerId) -> Option<&'a PeerInfo> { self.inner.get(who) } + + fn authorities(&self) -> usize { + self.inner.iter().filter(|(_, info)| info.roles.is_authority()).count() + } + + fn non_authorities(&self) -> usize { + self.inner.iter().filter(|(_, info)| !info.roles.is_authority()).count() + } } #[derive(Debug, PartialEq)] @@ -980,6 +989,122 @@ impl Inner { (true, report) } + + /// The initial logic for filtering round messages follows the given state + /// transitions: + /// + /// - State 0: not allowed to anyone (only if our local node is not an authority) + /// - State 1: allowed to random `sqrt(authorities)` + /// - State 2: allowed to all authorities + /// - State 3: allowed to random `sqrt(non-authorities)` + /// - State 4: allowed to all non-authorities + /// + /// Transitions will be triggered on repropagation attempts by the + /// underlying gossip layer, which should happen every 30 seconds. + fn round_message_allowed(&self, peer: &PeerInfo, mut previous_attempts: usize) -> bool { + const MIN_AUTHORITIES: usize = 5; + + if !self.config.is_authority && previous_attempts == 0 { + // non-authority nodes don't gossip any messages right away. we + // assume that authorities (and sentries) are strongly connected, so + // it should be unnecessary for non-authorities to gossip all + // messages right away. + return false; + } + + if !self.config.is_authority { + // since the node is not an authority we skipped the initial attempt + // to gossip the message, therefore we decrement `previous_attempts` + // so that the state machine below works the same way it does for + // authority nodes. + previous_attempts -= 1; + } + + if peer.roles.is_authority() { + let authorities = self.peers.authorities(); + + // the target node is an authority, on the first attempt we start by + // sending the message to only `sqrt(authorities)` (if we're + // connected to at least `MIN_AUTHORITIES`). + if previous_attempts == 0 && authorities > MIN_AUTHORITIES { + let authorities = authorities as f64; + let p = (authorities.sqrt()).max(MIN_AUTHORITIES as f64) / authorities; + rand::thread_rng().gen_bool(p) + } else { + // otherwise we already went through the step above, so + // we won't filter the message and send it to all + // authorities for whom it is polite to do so + true + } + } else { + // the node is not an authority so we apply stricter filters + if previous_attempts >= 3 { + // if we previously tried to send this message 3 (or more) + // times, then it is allowed to be sent to all peers. + true + } else if previous_attempts == 2 { + // otherwise we only send it to `sqrt(non-authorities)`. + let non_authorities = self.peers.non_authorities() as f64; + let p = non_authorities.sqrt() / non_authorities; + rand::thread_rng().gen_bool(p) + } else { + false + } + } + } + + /// The initial logic for filtering global messages follows the given state + /// transitions: + /// + /// - State 0: send to `sqrt(authorities)` ++ `sqrt(non-authorities)`. + /// - State 1: send to all authorities + /// - State 2: send to all non-authorities + /// + /// We are more lenient with global messages since there should be a lot + /// less global messages than round messages (just commits), and we want + /// these to propagate to non-authorities fast enough so that they can + /// observe finality. + /// + /// Transitions will be triggered on repropagation attempts by the + /// underlying gossip layer, which should happen every 30 seconds. + fn global_message_allowed(&self, peer: &PeerInfo, previous_attempts: usize) -> bool { + const MIN_PEERS: usize = 5; + + if peer.roles.is_authority() { + let authorities = self.peers.authorities(); + + // the target node is an authority, on the first attempt we start by + // sending the message to only `sqrt(authorities)` (if we're + // connected to at least `MIN_PEERS`). + if previous_attempts == 0 && authorities > MIN_PEERS { + let authorities = authorities as f64; + let p = (authorities.sqrt()).max(MIN_PEERS as f64) / authorities; + rand::thread_rng().gen_bool(p) + } else { + // otherwise we already went through the step above, so + // we won't filter the message and send it to all + // authorities for whom it is polite to do so + true + } + } else { + let non_authorities = self.peers.non_authorities(); + + // the target node is not an authority, on the first and second + // attempt we start by sending the message to only + // `sqrt(non_authorities)` (if we're connected to at least + // `MIN_PEERS`). + if previous_attempts <= 1 && non_authorities > MIN_PEERS { + let non_authorities = non_authorities as f64; + let p = (non_authorities.sqrt()).max(MIN_PEERS as f64) / non_authorities ; + rand::thread_rng().gen_bool(p) + } else { + // otherwise we already went through the step above, so + // we won't filter the message and send it to all + // non-authorities for whom it is polite to do so + true + } + } + } } /// A validator for GRANDPA gossip messages. @@ -1190,6 +1315,20 @@ impl network_gossip::Validator for GossipValidator Some(x) => x, }; + if let MessageIntent::Broadcast { previous_attempts } = intent { + if maybe_round.is_some() { + if !inner.round_message_allowed(peer, previous_attempts) { + // early return if the vote message isn't allowed at this stage. + return false; + } + } else { + if !inner.global_message_allowed(peer, previous_attempts) { + // early return if the global message isn't allowed at this stage. + return false; + } + } + } + // if the topic is not something the peer accepts, discard. if let Some(round) = maybe_round { return peer.view.consider_vote(round, set_id) == Consider::Accept @@ -1209,8 +1348,8 @@ impl network_gossip::Validator for GossipValidator Ok(GossipMessage::Commit(full)) => { // we only broadcast our best commit and only if it's // better than last received by peer. - Some(full.message.target_number) == our_best_commit - && Some(full.message.target_number) > peer_best_commit + Some(full.message.target_number) == our_best_commit && + Some(full.message.target_number) > peer_best_commit } Ok(GossipMessage::Neighbor(_)) => false, Ok(GossipMessage::CatchUpRequest(_)) => false, @@ -1311,7 +1450,7 @@ mod tests { use super::environment::SharedVoterSetState; use network_gossip::Validator as GossipValidatorT; use network::test::Block; - use primitives::crypto::Public; + use primitives::{crypto::Public, H256}; // some random config (not really needed) fn config() -> crate::Config { @@ -1329,7 +1468,6 @@ mod tests { fn voter_set_state() -> SharedVoterSetState { use crate::authorities::AuthoritySet; use crate::environment::VoterSetState; - use primitives::H256; let base = (H256::zero(), 0); let voters = AuthoritySet::genesis(Vec::new()); @@ -1991,4 +2129,162 @@ mod tests { ) } } + + #[test] + fn progressively_gossips_to_more_peers() { + let (val, _) = GossipValidator::::new( + config(), + voter_set_state(), + ); + + // the validator start at set id 0 + val.note_set(SetId(0), Vec::new(), |_, _| {}); + + // add 60 peers, 30 authorities and 30 full nodes + let mut authorities = Vec::new(); + authorities.resize_with(30, || PeerId::random()); + + let mut full_nodes = Vec::new(); + full_nodes.resize_with(30, || PeerId::random()); + + for i in 0..30 { + val.inner.write().peers.new_peer(authorities[i].clone(), Roles::AUTHORITY); + val.inner.write().peers.new_peer(full_nodes[i].clone(), Roles::FULL); + } + + let test = |previous_attempts, peers| { + let mut message_allowed = val.message_allowed(); + + move || { + let mut allowed = 0; + for peer in peers { + if message_allowed( + peer, + MessageIntent::Broadcast { previous_attempts }, + &crate::communication::round_topic::(1, 0), + &[], + ) { + allowed += 1; + } + } + allowed + } + }; + + fn trial usize>(mut test: F) -> usize { + let mut results = Vec::new(); + let n = 1000; + + for _ in 0..n { + results.push(test()); + } + + let n = results.len(); + let sum: usize = results.iter().sum(); + + sum / n + } + + // on the first attempt we will only gossip to `sqrt(authorities)`, + // which should average out to 5 peers after a couple of trials + assert_eq!(trial(test(0, &authorities)), 5); + + // on the second (and subsequent attempts) we should gossip to all + // authorities we're connected to. + assert_eq!(trial(test(1, &authorities)), 30); + assert_eq!(trial(test(2, &authorities)), 30); + + // we should only gossip to non-authorities after the third attempt + assert_eq!(trial(test(0, &full_nodes)), 0); + assert_eq!(trial(test(1, &full_nodes)), 0); + + // and only to `sqrt(non-authorities)` + assert_eq!(trial(test(2, &full_nodes)), 5); + + // only on the fourth attempt should we gossip to all non-authorities + assert_eq!(trial(test(3, &full_nodes)), 30); + } + + #[test] + fn only_restricts_gossip_to_authorities_after_a_minimum_threshold() { + let (val, _) = GossipValidator::::new( + config(), + voter_set_state(), + ); + + // the validator start at set id 0 + val.note_set(SetId(0), Vec::new(), |_, _| {}); + + let mut authorities = Vec::new(); + for _ in 0..5 { + let peer_id = PeerId::random(); + val.inner.write().peers.new_peer(peer_id.clone(), Roles::AUTHORITY); + authorities.push(peer_id); + } + + let mut message_allowed = val.message_allowed(); + + // since we're only connected to 5 authorities, we should never restrict + // sending of gossip messages, and instead just allow them to all + // non-authorities on the first attempt. + for authority in &authorities { + assert!( + message_allowed( + authority, + MessageIntent::Broadcast { previous_attempts: 0 }, + &crate::communication::round_topic::(1, 0), + &[], + ) + ); + } + } + + #[test] + fn non_authorities_never_gossip_messages_on_first_attempt() { + let mut config = config(); + config.is_authority = false; + + let (val, _) = GossipValidator::::new( + config, + voter_set_state(), + ); + + // the validator start at set id 0 + val.note_set(SetId(0), Vec::new(), |_, _| {}); + + let mut authorities = Vec::new(); + for _ in 0..100 { + let peer_id = PeerId::random(); + val.inner.write().peers.new_peer(peer_id.clone(), Roles::AUTHORITY); + authorities.push(peer_id); + } + + let mut message_allowed = val.message_allowed(); + + // since our node is not an authority we should **never** gossip any + // messages on the first attempt. + for authority in &authorities { + assert!( + !message_allowed( + authority, + MessageIntent::Broadcast { previous_attempts: 0 }, + &crate::communication::round_topic::(1, 0), + &[], + ) + ); + } + + // on the third attempt we should allow messages to authorities + // (on the second attempt we would do `sqrt(authorities)`) + for authority in &authorities { + assert!( + message_allowed( + authority, + MessageIntent::Broadcast { previous_attempts: 2 }, + &crate::communication::round_topic::(1, 0), + &[], + ) + ); + } + } } diff --git a/core/finality-grandpa/src/communication/mod.rs b/client/finality-grandpa/src/communication/mod.rs similarity index 98% rename from core/finality-grandpa/src/communication/mod.rs rename to client/finality-grandpa/src/communication/mod.rs index 4cc772fe9d4a0..142fe85f5c102 100644 --- a/core/finality-grandpa/src/communication/mod.rs +++ b/client/finality-grandpa/src/communication/mod.rs @@ -276,7 +276,6 @@ pub(crate) struct NetworkBridge> { service: N, validator: Arc>, neighbor_sender: periodic::NeighborPacketSender, - announce_sender: periodic::BlockAnnounceSender, } impl> NetworkBridge { @@ -341,10 +340,9 @@ impl> NetworkBridge { } let (rebroadcast_job, neighbor_sender) = periodic::neighbor_packet_worker(service.clone()); - let (announce_job, announce_sender) = periodic::block_announce_worker(service.clone()); let reporting_job = report_stream.consume(service.clone()); - let bridge = NetworkBridge { service, validator, neighbor_sender, announce_sender }; + let bridge = NetworkBridge { service, validator, neighbor_sender }; let startup_work = futures::future::lazy(move || { // lazily spawn these jobs onto their own tasks. the lazy future has access @@ -352,8 +350,6 @@ impl> NetworkBridge { let mut executor = tokio_executor::DefaultExecutor::current(); executor.spawn(Box::new(rebroadcast_job.select(on_exit.clone()).then(|_| Ok(())))) .expect("failed to spawn grandpa rebroadcast job task"); - executor.spawn(Box::new(announce_job.select(on_exit.clone()).then(|_| Ok(())))) - .expect("failed to spawn grandpa block announce job task"); executor.spawn(Box::new(reporting_job.select(on_exit.clone()).then(|_| Ok(())))) .expect("failed to spawn grandpa reporting job task"); Ok(()) @@ -470,7 +466,6 @@ impl> NetworkBridge { network: self.service.clone(), locals, sender: tx, - announce_sender: self.announce_sender.clone(), has_voted, }; @@ -563,7 +558,7 @@ fn incoming_global>( "target_number" => ?msg.message.target_number.clone(), "target_hash" => ?msg.message.target_hash.clone(), ); - + if let Err(cost) = check_compact_commit::( &msg.message, voters, @@ -676,7 +671,6 @@ impl> Clone for NetworkBridge { service: self.service.clone(), validator: Arc::clone(&self.validator), neighbor_sender: self.neighbor_sender.clone(), - announce_sender: self.announce_sender.clone(), } } } @@ -723,7 +717,6 @@ struct OutgoingMessages> { set_id: SetIdNumber, locals: Option<(AuthorityPair, AuthorityId)>, sender: mpsc::UnboundedSender>, - announce_sender: periodic::BlockAnnounceSender, network: N, has_voted: HasVoted, } @@ -781,8 +774,8 @@ impl> Sink for OutgoingMessages "block" => ?target_hash, "round" => ?self.round, "set_id" => ?self.set_id, ); - // send the target block hash to the background block announcer - self.announce_sender.send(target_hash, Vec::new()); + // announce the block we voted on to our peers. + self.network.announce(target_hash, Vec::new()); // propagate the message to peers let topic = round_topic::(self.round, self.set_id); diff --git a/client/finality-grandpa/src/communication/periodic.rs b/client/finality-grandpa/src/communication/periodic.rs new file mode 100644 index 0000000000000..9dd662ce43461 --- /dev/null +++ b/client/finality-grandpa/src/communication/periodic.rs @@ -0,0 +1,110 @@ +// Copyright 2019 Parity Technologies (UK) Ltd. +// This file is part of Substrate. + +// Substrate is free software: you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. + +// Substrate is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. + +// You should have received a copy of the GNU General Public License +// along with Substrate. If not, see . + +//! Periodic rebroadcast of neighbor packets. + +use std::time::{Instant, Duration}; + +use codec::Encode; +use futures::prelude::*; +use futures::sync::mpsc; +use log::{debug, warn}; +use tokio_timer::Delay; + +use network::PeerId; +use sr_primitives::traits::{NumberFor, Block as BlockT}; +use super::{gossip::{NeighborPacket, GossipMessage}, Network}; + +// how often to rebroadcast, if no other +const REBROADCAST_AFTER: Duration = Duration::from_secs(2 * 60); + +fn rebroadcast_instant() -> Instant { + Instant::now() + REBROADCAST_AFTER +} + +/// A sender used to send neighbor packets to a background job. +#[derive(Clone)] +pub(super) struct NeighborPacketSender( + mpsc::UnboundedSender<(Vec, NeighborPacket>)> +); + +impl NeighborPacketSender { + /// Send a neighbor packet for the background worker to gossip to peers. + pub fn send( + &self, + who: Vec, + neighbor_packet: NeighborPacket>, + ) { + if let Err(err) = self.0.unbounded_send((who, neighbor_packet)) { + debug!(target: "afg", "Failed to send neighbor packet: {:?}", err); + } + } +} + +/// Does the work of sending neighbor packets, asynchronously. +/// +/// It may rebroadcast the last neighbor packet periodically when no +/// progress is made. +pub(super) fn neighbor_packet_worker(net: N) -> ( + impl Future + Send + 'static, + NeighborPacketSender, +) where + B: BlockT, + N: Network, +{ + let mut last = None; + let (tx, mut rx) = mpsc::unbounded::<(Vec, NeighborPacket>)>(); + let mut delay = Delay::new(rebroadcast_instant()); + + let work = futures::future::poll_fn(move || { + loop { + match rx.poll().expect("unbounded receivers do not error; qed") { + Async::Ready(None) => return Ok(Async::Ready(())), + Async::Ready(Some((to, packet))) => { + // send to peers. + net.send_message(to.clone(), GossipMessage::::from(packet.clone()).encode()); + + // rebroadcasting network. + delay.reset(rebroadcast_instant()); + last = Some((to, packet)); + } + Async::NotReady => break, + } + } + + // has to be done in a loop because it needs to be polled after + // re-scheduling. + loop { + match delay.poll() { + Err(e) => { + warn!(target: "afg", "Could not rebroadcast neighbor packets: {:?}", e); + delay.reset(rebroadcast_instant()); + } + Ok(Async::Ready(())) => { + delay.reset(rebroadcast_instant()); + + if let Some((ref to, ref packet)) = last { + // send to peers. + net.send_message(to.clone(), GossipMessage::::from(packet.clone()).encode()); + } + } + Ok(Async::NotReady) => return Ok(Async::NotReady), + } + } + }); + + (work, NeighborPacketSender(tx)) +} diff --git a/core/finality-grandpa/src/communication/tests.rs b/client/finality-grandpa/src/communication/tests.rs similarity index 87% rename from core/finality-grandpa/src/communication/tests.rs rename to client/finality-grandpa/src/communication/tests.rs index f918f47258d49..e8b399aef39b6 100644 --- a/core/finality-grandpa/src/communication/tests.rs +++ b/client/finality-grandpa/src/communication/tests.rs @@ -28,6 +28,7 @@ use codec::Encode; use sr_primitives::traits::NumberFor; use crate::environment::SharedVoterSetState; +use fg_primitives::AuthorityList; use super::gossip::{self, GossipValidator}; use super::{AuthorityId, VoterSet, Round, SetId}; @@ -200,7 +201,7 @@ fn make_test_network() -> ( ) } -fn make_ids(keys: &[Ed25519Keyring]) -> Vec<(AuthorityId, u64)> { +fn make_ids(keys: &[Ed25519Keyring]) -> AuthorityList { keys.iter() .map(|key| key.clone().public().into()) .map(|id| (id, 1)) @@ -503,79 +504,3 @@ fn peer_with_higher_view_leads_to_catch_up_request() { current_thread::block_on_all(test).unwrap(); } - -#[test] -fn periodically_reannounce_voted_blocks_on_stall() { - use futures::try_ready; - use std::collections::HashSet; - use std::sync::Arc; - use std::time::Duration; - use parking_lot::Mutex; - - let (tester, net) = make_test_network(); - let (announce_worker, announce_sender) = super::periodic::block_announce_worker_with_delay( - net, - Duration::from_secs(1), - ); - - let hashes = Arc::new(Mutex::new(Vec::new())); - - fn wait_all(tester: Tester, hashes: &[Hash]) -> impl Future { - struct WaitAll { - remaining_hashes: Arc>>, - events_fut: Box>, - } - - impl Future for WaitAll { - type Item = Tester; - type Error = (); - - fn poll(&mut self) -> Poll { - let tester = try_ready!(self.events_fut.poll()); - - if self.remaining_hashes.lock().is_empty() { - return Ok(Async::Ready(tester)); - } - - let remaining_hashes = self.remaining_hashes.clone(); - self.events_fut = Box::new(tester.filter_network_events(move |event| match event { - Event::Announce(h) => - remaining_hashes.lock().remove(&h) || panic!("unexpected announce"), - _ => false, - })); - - self.poll() - } - } - - WaitAll { - remaining_hashes: Arc::new(Mutex::new(hashes.iter().cloned().collect())), - events_fut: Box::new(futures::future::ok(tester)), - } - } - - let test = tester - .and_then(move |tester| { - current_thread::spawn(announce_worker); - Ok(tester) - }) - .and_then(|tester| { - // announce 12 blocks - for _ in 0..=12 { - let hash = Hash::random(); - hashes.lock().push(hash); - announce_sender.send(hash, Vec::new()); - } - - // we should see an event for each of those announcements - wait_all(tester, &hashes.lock()) - }) - .and_then(|tester| { - // after a period of inactivity we should see the last - // `LATEST_VOTED_BLOCKS_TO_ANNOUNCE` being rebroadcast - wait_all(tester, &hashes.lock()[7..=12]) - }); - - let mut runtime = current_thread::Runtime::new().unwrap(); - runtime.block_on(test).unwrap(); -} diff --git a/core/finality-grandpa/src/consensus_changes.rs b/client/finality-grandpa/src/consensus_changes.rs similarity index 95% rename from core/finality-grandpa/src/consensus_changes.rs rename to client/finality-grandpa/src/consensus_changes.rs index 4d942248920f7..e276b287dd18f 100644 --- a/core/finality-grandpa/src/consensus_changes.rs +++ b/client/finality-grandpa/src/consensus_changes.rs @@ -47,11 +47,11 @@ impl ConsensusChanges { /// Finalize all pending consensus changes that are finalized by given block. /// Returns true if there any changes were finalized. - pub(crate) fn finalize ::client::error::Result>>( + pub(crate) fn finalize ::client_api::error::Result>>( &mut self, block: (N, H), canonical_at_height: F, - ) -> ::client::error::Result<(bool, bool)> { + ) -> ::client_api::error::Result<(bool, bool)> { let (split_idx, has_finalized_changes) = self.pending_changes.iter() .enumerate() .take_while(|(_, &(at_height, _))| at_height <= block.0) diff --git a/core/finality-grandpa/src/environment.rs b/client/finality-grandpa/src/environment.rs similarity index 97% rename from core/finality-grandpa/src/environment.rs rename to client/finality-grandpa/src/environment.rs index 149b00e80f920..f3a082ec19a17 100644 --- a/core/finality-grandpa/src/environment.rs +++ b/client/finality-grandpa/src/environment.rs @@ -25,10 +25,21 @@ use futures::prelude::*; use tokio_timer::Delay; use parking_lot::RwLock; +<<<<<<< HEAD:core/finality-grandpa/src/environment.rs + + +======= +use client_api::{ + HeaderBackend, BlockchainEvents, + backend::{Backend}, + Finalizer, + call_executor::CallExecutor, + error::Error as ClientError, + utils::is_descendent_of, +}; +>>>>>>> f44873dce... Reorganising the repository - external renames and moves (#4074):client/finality-grandpa/src/environment.rs use client::{ - backend::Backend, apply_aux, BlockchainEvents, CallExecutor, - Client, error::Error as ClientError, utils::is_descendent_of, - blockchain::HeaderBackend, backend::Finalizer, + apply_aux, Client, }; use grandpa::{ BlockNumberOps, Equivocation, Error as GrandpaError, round::State as RoundState, @@ -37,10 +48,32 @@ use grandpa::{ use primitives::{Blake2Hasher, H256, Pair}; use sr_primitives::generic::BlockId; use sr_primitives::traits::{ - Block as BlockT, Header as HeaderT, NumberFor, One, Zero, + Block as BlockT, Header as HeaderT, NumberFor, One, Zero }; use substrate_telemetry::{telemetry, CONSENSUS_INFO}; - +<<<<<<< HEAD +<<<<<<< HEAD +<<<<<<< HEAD +<<<<<<< HEAD:client/finality-grandpa/src/environment.rs +use substrate_prometheus::{metrics}; +<<<<<<< HEAD +<<<<<<< HEAD + +======= +>>>>>>> 9e0aa9e5d... base-code(rebased) +======= +======= +//use crate::metrics; // for local metrics +>>>>>>> set global/local metric:core/finality-grandpa/src/environment.rs +>>>>>>> 5d52cd10e... set global/local metric +======= +//use crate::metrics; // for local metrics +>>>>>>> ece9dfd03... + +======= +use substrate_prometheus::{metrics}; +>>>>>>> 9cfda61a8... prometheus_v0.1 +======= +>>>>>>> e123ed09a... prres_num,best_heght,finallity_heght add use crate::{ CommandOrError, Commit, Config, Error, Network, Precommit, Prevote, PrimaryPropose, SignedMessage, NewAuthoritySet, VoterCommand, @@ -1103,4 +1136,4 @@ pub(crate) fn canonical_at_height, C: HeaderBackend: Send + Sync { - /// Call GrandpaApi::grandpa_authorities at given block. - fn authorities(&self, block: &BlockId) -> ClientResult>; - /// Prove call of GrandpaApi::grandpa_authorities at given block. - fn prove_authorities(&self, block: &BlockId) -> ClientResult>>; + /// Read GRANDPA_AUTHORITIES_KEY from storage at given block. + fn authorities(&self, block: &BlockId) -> ClientResult; + /// Prove storage read of GRANDPA_AUTHORITIES_KEY at given block. + fn prove_authorities(&self, block: &BlockId) -> ClientResult; } /// Client-based implementation of AuthoritySetForFinalityProver. @@ -72,33 +75,28 @@ impl, RA> AuthoritySetForFinalityProver fo E: CallExecutor + 'static + Clone + Send + Sync, RA: Send + Sync, { - fn authorities(&self, block: &BlockId) -> ClientResult> { - self.executor().call( - block, - "GrandpaApi_grandpa_authorities", - &[], - ExecutionStrategy::NativeElseWasm, - None, - ).and_then(|call_result| Decode::decode(&mut &call_result[..]) - .map_err(|err| ClientError::CallResultDecode( - "failed to decode GRANDPA authorities set proof".into(), err - ))) - } - - fn prove_authorities(&self, block: &BlockId) -> ClientResult>> { - self.execution_proof(block, "GrandpaApi_grandpa_authorities",&[]).map(|(_, proof)| proof) + fn authorities(&self, block: &BlockId) -> ClientResult { + let storage_key = StorageKey(GRANDPA_AUTHORITIES_KEY.to_vec()); + self.storage(block, &storage_key)? + .and_then(|encoded| VersionedAuthorityList::decode(&mut encoded.0.as_slice()).ok()) + .map(|versioned| versioned.into()) + .ok_or(ClientError::InvalidAuthoritiesSet) + } + + fn prove_authorities(&self, block: &BlockId) -> ClientResult { + self.read_proof(block, iter::once(GRANDPA_AUTHORITIES_KEY)) } } /// GRANDPA authority set related methods for the finality proof checker. pub trait AuthoritySetForFinalityChecker: Send + Sync { - /// Check execution proof of Grandpa::grandpa_authorities at given block. + /// Check storage read proof of GRANDPA_AUTHORITIES_KEY at given block. fn check_authorities_proof( &self, hash: Block::Hash, header: Block::Header, - proof: Vec>, - ) -> ClientResult>; + proof: StorageProof, + ) -> ClientResult; } /// FetchChecker-based implementation of AuthoritySetForFinalityChecker. @@ -107,23 +105,31 @@ impl AuthoritySetForFinalityChecker for Arc>, - ) -> ClientResult> { - let request = RemoteCallRequest { + proof: StorageProof, + ) -> ClientResult { + let storage_key = GRANDPA_AUTHORITIES_KEY.to_vec(); + let request = RemoteReadRequest { block: hash, header, - method: "GrandpaApi_grandpa_authorities".into(), - call_data: vec![], + keys: vec![storage_key.clone()], retry_count: None, }; - self.check_execution_proof(&request, proof) - .and_then(|authorities| { - let authorities: Vec<(AuthorityId, u64)> = Decode::decode(&mut &authorities[..]) - .map_err(|err| ClientError::CallResultDecode( - "failed to decode GRANDPA authorities set proof".into(), err - ))?; - Ok(authorities.into_iter().collect()) + self.check_read_proof(&request, proof) + .and_then(|results| { + let maybe_encoded = results.get(&storage_key) + .expect( + "storage_key is listed in the request keys; \ + check_read_proof must return a value for each requested key; + qed" + ); + maybe_encoded + .as_ref() + .and_then(|encoded| { + VersionedAuthorityList::decode(&mut encoded.as_slice()).ok() + }) + .map(|versioned| versioned.into()) + .ok_or(ClientError::InvalidAuthoritiesSet) }) } } @@ -189,7 +195,7 @@ pub struct FinalityEffects { /// New authorities set id that should be applied starting from block. pub new_set_id: u64, /// New authorities set that should be applied starting from block. - pub new_authorities: Vec<(AuthorityId, u64)>, + pub new_authorities: AuthorityList, } /// Single fragment of proof-of-finality. @@ -207,7 +213,7 @@ struct FinalityProofFragment { /// The set of headers in the range (U; F] that we believe are unknown to the caller. Ordered. pub unknown_headers: Vec

, /// Optional proof of execution of GRANDPA::authorities(). - pub authorities_proof: Option>>, + pub authorities_proof: Option, } /// Proof of finality is the ordered set of finality fragments, where: @@ -253,7 +259,7 @@ pub(crate) fn prove_finality, B: BlockchainBackend ::client::error::Result>> +) -> ::client_api::error::Result>> where J: ProvableJustification, { @@ -408,7 +414,7 @@ pub(crate) fn prove_finality, B: BlockchainBackend, B>( blockchain: &B, current_set_id: u64, - current_authorities: Vec<(AuthorityId, u64)>, + current_authorities: AuthorityList, authorities_provider: &dyn AuthoritySetForFinalityChecker, remote_proof: Vec, ) -> ClientResult> @@ -427,7 +433,7 @@ pub(crate) fn check_finality_proof, B>( fn do_check_finality_proof, B, J>( blockchain: &B, current_set_id: u64, - current_authorities: Vec<(AuthorityId, u64)>, + current_authorities: AuthorityList, authorities_provider: &dyn AuthoritySetForFinalityChecker, remote_proof: Vec, ) -> ClientResult> @@ -522,12 +528,12 @@ fn check_finality_proof_fragment, B, J>( /// Authorities set from initial authorities set or finality effects. enum AuthoritiesOrEffects { - Authorities(u64, Vec<(AuthorityId, u64)>), + Authorities(u64, AuthorityList), Effects(FinalityEffects
), } impl AuthoritiesOrEffects
{ - pub fn extract_authorities(self) -> (u64, Vec<(AuthorityId, u64)>) { + pub fn extract_authorities(self) -> (u64, AuthorityList) { match self { AuthoritiesOrEffects::Authorities(set_id, authorities) => (set_id, authorities), AuthoritiesOrEffects::Effects(effects) => (effects.new_set_id, effects.new_authorities), @@ -572,7 +578,7 @@ impl> ProvableJustification for GrandpaJ #[cfg(test)] pub(crate) mod tests { use test_client::runtime::{Block, Header, H256}; - use test_client::client::{backend::NewBlockState}; + use client_api::NewBlockState; use test_client::client::in_mem::Blockchain as InMemoryBlockchain; use super::*; use primitives::crypto::Public; @@ -581,14 +587,14 @@ pub(crate) mod tests { impl AuthoritySetForFinalityProver for (GetAuthorities, ProveAuthorities) where - GetAuthorities: Send + Sync + Fn(BlockId) -> ClientResult>, - ProveAuthorities: Send + Sync + Fn(BlockId) -> ClientResult>>, + GetAuthorities: Send + Sync + Fn(BlockId) -> ClientResult, + ProveAuthorities: Send + Sync + Fn(BlockId) -> ClientResult, { - fn authorities(&self, block: &BlockId) -> ClientResult> { + fn authorities(&self, block: &BlockId) -> ClientResult { self.0(*block) } - fn prove_authorities(&self, block: &BlockId) -> ClientResult>> { + fn prove_authorities(&self, block: &BlockId) -> ClientResult { self.1(*block) } } @@ -597,14 +603,14 @@ pub(crate) mod tests { impl AuthoritySetForFinalityChecker for ClosureAuthoritySetForFinalityChecker where - Closure: Send + Sync + Fn(H256, Header, Vec>) -> ClientResult>, + Closure: Send + Sync + Fn(H256, Header, StorageProof) -> ClientResult, { fn check_authorities_proof( &self, hash: H256, header: Header, - proof: Vec>, - ) -> ClientResult> { + proof: StorageProof + ) -> ClientResult { self.0(hash, header, proof) } } @@ -824,8 +830,8 @@ pub(crate) mod tests { _ => unreachable!("no other authorities should be fetched: {:?}", block_id), }, |block_id| match block_id { - BlockId::Number(4) => Ok(vec![vec![40]]), - BlockId::Number(6) => Ok(vec![vec![60]]), + BlockId::Number(4) => Ok(StorageProof::new(vec![vec![40]])), + BlockId::Number(6) => Ok(StorageProof::new(vec![vec![60]])), _ => unreachable!("no other authorities should be proved: {:?}", block_id), }, ), @@ -841,14 +847,14 @@ pub(crate) mod tests { block: header(5).hash(), justification: just5, unknown_headers: Vec::new(), - authorities_proof: Some(vec![vec![40]]), + authorities_proof: Some(StorageProof::new(vec![vec![40]])), }, // last fragment provides justification for #7 && unknown#7 FinalityProofFragment { block: header(7).hash(), justification: just7, unknown_headers: vec![header(7)], - authorities_proof: Some(vec![vec![60]]), + authorities_proof: Some(StorageProof::new(vec![vec![60]])), }, ]); } @@ -895,7 +901,7 @@ pub(crate) mod tests { block: header(4).hash(), justification: TestJustification(true, vec![7]).encode(), unknown_headers: vec![header(4)], - authorities_proof: Some(vec![vec![42]]), + authorities_proof: Some(StorageProof::new(vec![vec![42]])), }, FinalityProofFragment { block: header(5).hash(), justification: TestJustification(true, vec![8]).encode(), @@ -942,7 +948,7 @@ pub(crate) mod tests { block: header(2).hash(), justification: TestJustification(true, vec![7]).encode(), unknown_headers: Vec::new(), - authorities_proof: Some(vec![vec![42]]), + authorities_proof: Some(StorageProof::new(vec![vec![42]])), }, FinalityProofFragment { block: header(4).hash(), justification: TestJustification(true, vec![8]).encode(), diff --git a/core/finality-grandpa/src/import.rs b/client/finality-grandpa/src/import.rs similarity index 97% rename from core/finality-grandpa/src/import.rs rename to client/finality-grandpa/src/import.rs index 8fbe0791e8c05..d6a3ea6e94bbf 100644 --- a/core/finality-grandpa/src/import.rs +++ b/client/finality-grandpa/src/import.rs @@ -21,10 +21,12 @@ use codec::Encode; use futures::sync::mpsc; use parking_lot::RwLockWriteGuard; -use client::{blockchain, CallExecutor, Client, well_known_cache_keys}; -use client::blockchain::HeaderBackend; -use client::backend::Backend; -use client::utils::is_descendent_of; +use client_api::{ + backend::Backend, blockchain, + CallExecutor, blockchain::HeaderBackend, well_known_cache_keys, + utils::is_descendent_of, +}; +use client::Client; use consensus_common::{ BlockImport, Error as ConsensusError, BlockCheckParams, BlockImportParams, ImportResult, JustificationImport, @@ -34,7 +36,7 @@ use fg_primitives::{GRANDPA_ENGINE_ID, ScheduledChange, ConsensusLog}; use sr_primitives::Justification; use sr_primitives::generic::{BlockId, OpaqueDigestItemId}; use sr_primitives::traits::{ - Block as BlockT, DigestFor, Header as HeaderT, NumberFor, + Block as BlockT, DigestFor, Header as HeaderT, NumberFor, Zero, }; use primitives::{H256, Blake2Hasher}; @@ -97,10 +99,14 @@ impl, RA, SC> JustificationImport pending_change.effective_number() > chain_info.finalized_number && pending_change.effective_number() <= chain_info.best_number { - let effective_block_hash = self.select_chain.finality_target( - pending_change.canon_hash, - Some(pending_change.effective_number()), - ); + let effective_block_hash = if !pending_change.delay.is_zero() { + self.select_chain.finality_target( + pending_change.canon_hash, + Some(pending_change.effective_number()), + ) + } else { + Ok(Some(pending_change.canon_hash)) + }; if let Ok(Some(hash)) = effective_block_hash { if let Ok(Some(header)) = self.inner.header(&BlockId::Hash(hash)) { diff --git a/core/finality-grandpa/src/justification.rs b/client/finality-grandpa/src/justification.rs similarity index 98% rename from core/finality-grandpa/src/justification.rs rename to client/finality-grandpa/src/justification.rs index f5965df3e1228..eff06776bdb92 100644 --- a/core/finality-grandpa/src/justification.rs +++ b/client/finality-grandpa/src/justification.rs @@ -16,9 +16,8 @@ use std::collections::{HashMap, HashSet}; -use client::{CallExecutor, Client}; -use client::backend::Backend; -use client::error::Error as ClientError; +use client::Client; +use client_api::{CallExecutor, backend::Backend, error::Error as ClientError}; use codec::{Encode, Decode}; use grandpa::voter_set::VoterSet; use grandpa::{Error as GrandpaError}; diff --git a/core/finality-grandpa/src/lib.rs b/client/finality-grandpa/src/lib.rs similarity index 92% rename from core/finality-grandpa/src/lib.rs rename to client/finality-grandpa/src/lib.rs index 0decea58117b0..80d2350982481 100644 --- a/core/finality-grandpa/src/lib.rs +++ b/client/finality-grandpa/src/lib.rs @@ -55,16 +55,14 @@ use futures::prelude::*; use log::{debug, error, info}; use futures::sync::mpsc; -use client::{ - BlockchainEvents, CallExecutor, Client, backend::Backend, error::Error as ClientError, +use client_api::{ + BlockchainEvents, CallExecutor, backend::Backend, error::Error as ClientError, + ExecutionStrategy, HeaderBackend }; -use client::blockchain::HeaderBackend; -use codec::Encode; +use client::Client; +use codec::{Decode, Encode}; use sr_primitives::generic::BlockId; -use sr_primitives::traits::{ - NumberFor, Block as BlockT, DigestFor, ProvideRuntimeApi -}; -use fg_primitives::{GrandpaApi, AuthorityPair}; +use sr_primitives::traits::{NumberFor, Block as BlockT, DigestFor, Zero}; use keystore::KeyStorePtr; use inherents::InherentDataProviders; use consensus_common::SelectChain; @@ -72,7 +70,7 @@ use primitives::{H256, Blake2Hasher, Pair}; use substrate_telemetry::{telemetry, CONSENSUS_INFO, CONSENSUS_DEBUG, CONSENSUS_WARN}; use serde_json; -use srml_finality_tracker; +use paint_finality_tracker; use grandpa::Error as GrandpaError; use grandpa::{voter, BlockNumberOps, voter_set::VoterSet}; @@ -108,7 +106,7 @@ use environment::{Environment, VoterSetState}; use import::GrandpaBlockImport; use until_imported::UntilGlobalMessageBlocksImported; use communication::NetworkBridge; -use fg_primitives::{AuthoritySignature, SetId, AuthorityWeight}; +use fg_primitives::{AuthorityList, AuthorityPair, AuthoritySignature, SetId}; // Re-export these two because it's just so damn convenient. pub use fg_primitives::{AuthorityId, ScheduledChange}; @@ -295,7 +293,7 @@ pub(crate) struct NewAuthoritySet { pub(crate) canon_number: N, pub(crate) canon_hash: H, pub(crate) set_id: SetId, - pub(crate) authorities: Vec<(AuthorityId, AuthorityWeight)>, + pub(crate) authorities: AuthorityList, } /// Commands issued to the voter. @@ -367,11 +365,44 @@ pub struct LinkHalf, RA, SC> { voter_commands_rx: mpsc::UnboundedReceiver>>, } +/// Provider for the Grandpa authority set configured on the genesis block. +pub trait GenesisAuthoritySetProvider { + /// Get the authority set at the genesis block. + fn get(&self) -> Result; +} + +impl, RA> GenesisAuthoritySetProvider for Client + where + B: Backend + Send + Sync + 'static, + E: CallExecutor + 'static + Clone + Send + Sync, + RA: Send + Sync, +{ + fn get(&self) -> Result { + // This implementation uses the Grandpa runtime API instead of reading directly from the + // `GRANDPA_AUTHORITIES_KEY` as the data may have been migrated since the genesis block of + // the chain, whereas the runtime API is backwards compatible. + self.executor() + .call( + &BlockId::Number(Zero::zero()), + "GrandpaApi_grandpa_authorities", + &[], + ExecutionStrategy::NativeElseWasm, + None, + ) + .and_then(|call_result| { + Decode::decode(&mut &call_result[..]) + .map_err(|err| ClientError::CallResultDecode( + "failed to decode GRANDPA authorities set proof".into(), err + )) + }) + } +} + /// Make block importer and link half necessary to tie the background voter /// to it. -pub fn block_import, RA, PRA, SC>( +pub fn block_import, RA, SC>( client: Arc>, - api: &PRA, + genesis_authorities_provider: &dyn GenesisAuthoritySetProvider, select_chain: SC, ) -> Result<( GrandpaBlockImport, @@ -381,12 +412,8 @@ where B: Backend + 'static, E: CallExecutor + 'static + Clone + Send + Sync, RA: Send + Sync, - PRA: ProvideRuntimeApi, - PRA::Api: GrandpaApi, SC: SelectChain, { - use sr_primitives::traits::Zero; - let chain_info = client.info(); let genesis_hash = chain_info.chain.genesis_hash; @@ -395,12 +422,11 @@ where genesis_hash, >::zero(), || { - let genesis_authorities = api.runtime_api() - .grandpa_authorities(&BlockId::number(Zero::zero()))?; + let authorities = genesis_authorities_provider.get()?; telemetry!(CONSENSUS_DEBUG; "afg.loading_authorities"; - "authorities_len" => ?genesis_authorities.len() + "authorities_len" => ?authorities.len() ); - Ok(genesis_authorities) + Ok(authorities) } )?; @@ -479,9 +505,9 @@ fn register_finality_tracker_inherent_data_provider + Send + Sync + 'static, RA: Send + Sync + 'static, { - if !inherent_data_providers.has_provider(&srml_finality_tracker::INHERENT_IDENTIFIER) { + if !inherent_data_providers.has_provider(&paint_finality_tracker::INHERENT_IDENTIFIER) { inherent_data_providers - .register_provider(srml_finality_tracker::InherentDataProvider::new(move || { + .register_provider(paint_finality_tracker::InherentDataProvider::new(move || { #[allow(deprecated)] { let info = client.info().chain; @@ -520,7 +546,7 @@ pub struct GrandpaParams, N, RA, SC, VR, X> { /// block import worker that has already been instantiated with `block_import`. pub fn run_grandpa_voter, N, RA, SC, VR, X>( grandpa_params: GrandpaParams, -) -> client::error::Result + Send + 'static> where +) -> client_api::error::Result + Send + 'static> where Block::Hash: Ord, B: Backend + 'static, E: CallExecutor + Send + Sync + 'static, @@ -853,7 +879,7 @@ where #[deprecated(since = "1.1.0", note = "Please switch to run_grandpa_voter.")] pub fn run_grandpa, N, RA, SC, VR, X>( grandpa_params: GrandpaParams, -) -> ::client::error::Result + Send + 'static> where +) -> ::client_api::error::Result + Send + 'static> where Block::Hash: Ord, B: Backend + 'static, E: CallExecutor + Send + Sync + 'static, diff --git a/core/finality-grandpa/src/light_import.rs b/client/finality-grandpa/src/light_import.rs similarity index 93% rename from core/finality-grandpa/src/light_import.rs rename to client/finality-grandpa/src/light_import.rs index 30af3a06d3f76..5ab9082c90707 100644 --- a/core/finality-grandpa/src/light_import.rs +++ b/client/finality-grandpa/src/light_import.rs @@ -18,9 +18,9 @@ use std::collections::HashMap; use std::sync::Arc; use log::{info, trace, warn}; use parking_lot::RwLock; - -use client::{ - CallExecutor, Client, +use client::Client; +use client_api::{ + CallExecutor, backend::{AuxStore, Backend, Finalizer}, blockchain::HeaderBackend, error::Error as ClientError, @@ -34,17 +34,18 @@ use consensus_common::{ }; use network::config::{BoxFinalityProofRequestBuilder, FinalityProofRequestBuilder}; use sr_primitives::Justification; -use sr_primitives::traits::{ - NumberFor, Block as BlockT, Header as HeaderT, ProvideRuntimeApi, DigestFor, -}; -use fg_primitives::{self, GrandpaApi, AuthorityId}; +use sr_primitives::traits::{NumberFor, Block as BlockT, Header as HeaderT, DigestFor}; +use fg_primitives::{self, AuthorityList}; use sr_primitives::generic::BlockId; use primitives::{H256, Blake2Hasher}; +use crate::GenesisAuthoritySetProvider; use crate::aux_schema::load_decode; use crate::consensus_changes::ConsensusChanges; use crate::environment::canonical_at_height; -use crate::finality_proof::{AuthoritySetForFinalityChecker, ProvableJustification, make_finality_proof_request}; +use crate::finality_proof::{ + AuthoritySetForFinalityChecker, ProvableJustification, make_finality_proof_request, +}; use crate::justification::GrandpaJustification; /// LightAuthoritySet is saved under this key in aux storage. @@ -53,21 +54,23 @@ const LIGHT_AUTHORITY_SET_KEY: &[u8] = b"grandpa_voters"; const LIGHT_CONSENSUS_CHANGES_KEY: &[u8] = b"grandpa_consensus_changes"; /// Create light block importer. -pub fn light_block_import, RA, PRA>( +pub fn light_block_import, RA>( client: Arc>, backend: Arc, + genesis_authorities_provider: &dyn GenesisAuthoritySetProvider, authority_set_provider: Arc>, - api: Arc, ) -> Result, ClientError> where B: Backend + 'static, E: CallExecutor + 'static + Clone + Send + Sync, RA: Send + Sync, - PRA: ProvideRuntimeApi, - PRA::Api: GrandpaApi, { let info = client.info(); - let import_data = load_aux_import_data(info.chain.finalized_hash, &*client, api)?; + let import_data = load_aux_import_data( + info.chain.finalized_hash, + &*client, + genesis_authorities_provider, + )?; Ok(GrandpaLightBlockImport { client, backend, @@ -110,7 +113,7 @@ struct LightImportData> { #[derive(Debug, Encode, Decode)] struct LightAuthoritySet { set_id: u64, - authorities: Vec<(AuthorityId, u64)>, + authorities: AuthorityList, } impl, RA> GrandpaLightBlockImport { @@ -194,7 +197,7 @@ impl, RA> FinalityProofImport impl LightAuthoritySet { /// Get a genesis set with given authorities. - pub fn genesis(initial: Vec<(AuthorityId, u64)>) -> Self { + pub fn genesis(initial: AuthorityList) -> Self { LightAuthoritySet { set_id: fg_primitives::SetId::default(), authorities: initial, @@ -207,12 +210,12 @@ impl LightAuthoritySet { } /// Get latest authorities set. - pub fn authorities(&self) -> Vec<(AuthorityId, u64)> { + pub fn authorities(&self) -> AuthorityList { self.authorities.clone() } /// Set new authorities set. - pub fn update(&mut self, set_id: u64, authorities: Vec<(AuthorityId, u64)>) { + pub fn update(&mut self, set_id: u64, authorities: AuthorityList) { self.set_id = set_id; std::mem::replace(&mut self.authorities, authorities); } @@ -472,17 +475,14 @@ fn do_finalize_block>( } /// Load light import aux data from the store. -fn load_aux_import_data, PRA>( +fn load_aux_import_data>( last_finalized: Block::Hash, aux_store: &B, - api: Arc, + genesis_authorities_provider: &dyn GenesisAuthoritySetProvider, ) -> Result, ClientError> where B: AuxStore, - PRA: ProvideRuntimeApi, - PRA::Api: GrandpaApi, { - use sr_primitives::traits::Zero; let authority_set = match load_decode(aux_store, LIGHT_AUTHORITY_SET_KEY)? { Some(authority_set) => authority_set, None => { @@ -490,7 +490,7 @@ fn load_aux_import_data, PRA>( from genesis on what appears to be first startup."); // no authority set on disk: fetch authorities from genesis state - let genesis_authorities = api.runtime_api().grandpa_authorities(&BlockId::number(Zero::zero()))?; + let genesis_authorities = genesis_authorities_provider.get()?; let authority_set = LightAuthoritySet::genesis(genesis_authorities); let encoded = authority_set.encode(); @@ -546,6 +546,7 @@ fn on_post_finalization_error(error: ClientError, value_type: &str) -> Consensus pub mod tests { use super::*; use consensus_common::ForkChoiceStrategy; + use fg_primitives::AuthorityId; use primitives::{H256, crypto::Public}; use test_client::client::in_mem::Blockchain as InMemoryAuxStore; use test_client::runtime::{Block, Header}; @@ -622,20 +623,19 @@ pub mod tests { } /// Creates light block import that ignores justifications that came outside of finality proofs. - pub fn light_block_import_without_justifications, RA, PRA>( + pub fn light_block_import_without_justifications, RA>( client: Arc>, backend: Arc, + genesis_authorities_provider: &dyn GenesisAuthoritySetProvider, authority_set_provider: Arc>, - api: Arc, ) -> Result, ClientError> where B: Backend + 'static, E: CallExecutor + 'static + Clone + Send + Sync, RA: Send + Sync, - PRA: ProvideRuntimeApi, - PRA::Api: GrandpaApi, { - light_block_import(client, backend, authority_set_provider, api).map(NoJustificationsImport) + light_block_import(client, backend, genesis_authorities_provider, authority_set_provider) + .map(NoJustificationsImport) } fn import_block( @@ -663,6 +663,7 @@ pub mod tests { finalized: false, auxiliary: Vec::new(), fork_choice: ForkChoiceStrategy::LongestChain, + allow_missing_state: true, }; do_import_block::<_, _, _, TestJustification>( &client, @@ -680,6 +681,7 @@ pub mod tests { bad_justification: false, needs_finality_proof: false, is_new_best: true, + header_only: false, })); } @@ -692,6 +694,7 @@ pub mod tests { bad_justification: false, needs_finality_proof: false, is_new_best: true, + header_only: false, })); } @@ -705,6 +708,7 @@ pub mod tests { bad_justification: false, needs_finality_proof: true, is_new_best: true, + header_only: false, })); } @@ -721,6 +725,7 @@ pub mod tests { bad_justification: false, needs_finality_proof: true, is_new_best: false, + header_only: false, }, )); } @@ -729,14 +734,14 @@ pub mod tests { #[test] fn aux_data_updated_on_start() { let aux_store = InMemoryAuxStore::::new(); - let api = Arc::new(TestApi::new(vec![(AuthorityId::from_slice(&[1; 32]), 1)])); + let api = TestApi::new(vec![(AuthorityId::from_slice(&[1; 32]), 1)]); // when aux store is empty initially assert!(aux_store.get_aux(LIGHT_AUTHORITY_SET_KEY).unwrap().is_none()); assert!(aux_store.get_aux(LIGHT_CONSENSUS_CHANGES_KEY).unwrap().is_none()); // it is updated on importer start - load_aux_import_data(Default::default(), &aux_store, api).unwrap(); + load_aux_import_data(Default::default(), &aux_store, &api).unwrap(); assert!(aux_store.get_aux(LIGHT_AUTHORITY_SET_KEY).unwrap().is_some()); assert!(aux_store.get_aux(LIGHT_CONSENSUS_CHANGES_KEY).unwrap().is_some()); } @@ -744,7 +749,7 @@ pub mod tests { #[test] fn aux_data_loaded_on_restart() { let aux_store = InMemoryAuxStore::::new(); - let api = Arc::new(TestApi::new(vec![(AuthorityId::from_slice(&[1; 32]), 1)])); + let api = TestApi::new(vec![(AuthorityId::from_slice(&[1; 32]), 1)]); // when aux store is non-empty initially let mut consensus_changes = ConsensusChanges::::empty(); @@ -766,7 +771,7 @@ pub mod tests { ).unwrap(); // importer uses it on start - let data = load_aux_import_data(Default::default(), &aux_store, api).unwrap(); + let data = load_aux_import_data(Default::default(), &aux_store, &api).unwrap(); assert_eq!(data.authority_set.authorities(), vec![(AuthorityId::from_slice(&[42; 32]), 2)]); assert_eq!(data.consensus_changes.pending_changes(), &[(42, Default::default())]); } diff --git a/core/finality-grandpa/src/observer.rs b/client/finality-grandpa/src/observer.rs similarity index 98% rename from core/finality-grandpa/src/observer.rs rename to client/finality-grandpa/src/observer.rs index e4d90ddc22e77..b182eaaaa5e61 100644 --- a/core/finality-grandpa/src/observer.rs +++ b/client/finality-grandpa/src/observer.rs @@ -25,7 +25,8 @@ use grandpa::{ use log::{debug, info, warn}; use consensus_common::SelectChain; -use client::{CallExecutor, Client, backend::Backend}; +use client_api::{CallExecutor, backend::Backend}; +use client::Client; use sr_primitives::traits::{NumberFor, Block as BlockT}; use primitives::{H256, Blake2Hasher}; @@ -155,7 +156,7 @@ pub fn run_grandpa_observer, N, RA, SC>( link: LinkHalf, network: N, on_exit: impl Future + Clone + Send + 'static, -) -> ::client::error::Result + Send + 'static> where +) -> ::client_api::error::Result + Send + 'static> where B: Backend + 'static, E: CallExecutor + Send + Sync + 'static, N: Network + Send + Sync + 'static, diff --git a/core/finality-grandpa/src/tests.rs b/client/finality-grandpa/src/tests.rs similarity index 96% rename from core/finality-grandpa/src/tests.rs rename to client/finality-grandpa/src/tests.rs index 39173741711de..6d4d439a9fb34 100644 --- a/core/finality-grandpa/src/tests.rs +++ b/client/finality-grandpa/src/tests.rs @@ -25,11 +25,9 @@ use parking_lot::Mutex; use futures03::{StreamExt as _, TryStreamExt as _}; use tokio::runtime::current_thread; use keyring::Ed25519Keyring; -use client::{ - error::Result, - runtime_api::{Core, RuntimeVersion, ApiExt}, - LongestChain, -}; +use client::LongestChain; +use client_api::error::Result; +use sr_api::{Core, RuntimeVersion, ApiExt, StorageProof}; use test_client::{self, runtime::BlockNumber}; use consensus_common::{BlockOrigin, ForkChoiceStrategy, ImportedAux, BlockImportParams, ImportResult}; use consensus_common::import_queue::{BoxBlockImport, BoxJustificationImport, BoxFinalityProofImport}; @@ -39,7 +37,8 @@ use codec::Decode; use sr_primitives::traits::{ApiRef, ProvideRuntimeApi, Header as HeaderT}; use sr_primitives::generic::{BlockId, DigestItem}; use primitives::{NativeOrEncoded, ExecutionContext, crypto::Public}; -use fg_primitives::{GRANDPA_ENGINE_ID, AuthorityId}; +use fg_primitives::{GRANDPA_ENGINE_ID, AuthorityList, GrandpaApi}; +use state_machine::{backend::InMemory, prove_read, read_proof_check}; use authorities::AuthoritySet; use finality_proof::{FinalityProofProvider, AuthoritySetForFinalityProver, AuthoritySetForFinalityChecker}; @@ -93,9 +92,9 @@ impl TestNetFactory for GrandpaTestNet { fn default_config() -> ProtocolConfig { // the authority role ensures gossip hits all nodes here. - ProtocolConfig { - roles: Roles::AUTHORITY, - } + let mut config = ProtocolConfig::default(); + config.roles = Roles::AUTHORITY; + config } fn make_verifier( @@ -136,8 +135,8 @@ impl TestNetFactory for GrandpaTestNet { let import = light_block_import_without_justifications( client.clone(), backend.clone(), + &self.test_config, authorities_provider, - Arc::new(self.test_config.clone()) ).expect("Could not create block import for fresh peer."); let finality_proof_req_builder = import.0.create_finality_proof_request_builder(); let proof_import = Box::new(import.clone()); @@ -187,11 +186,11 @@ impl Future for Exit { #[derive(Default, Clone)] pub(crate) struct TestApi { - genesis_authorities: Vec<(AuthorityId, u64)>, + genesis_authorities: AuthorityList, } impl TestApi { - pub fn new(genesis_authorities: Vec<(AuthorityId, u64)>) -> Self { + pub fn new(genesis_authorities: AuthorityList) -> Self { TestApi { genesis_authorities, } @@ -243,6 +242,8 @@ impl Core for RuntimeApi { } impl ApiExt for RuntimeApi { + type Error = client_api::error::Error; + fn map_api_result result::Result, R, E>( &self, _: F @@ -258,7 +259,7 @@ impl ApiExt for RuntimeApi { unimplemented!("Not required for testing!") } - fn extract_proof(&mut self) -> Option>> { + fn extract_proof(&mut self) -> Option { unimplemented!("Not required for testing!") } } @@ -270,23 +271,30 @@ impl GrandpaApi for RuntimeApi { _: ExecutionContext, _: Option<()>, _: Vec, - ) -> Result>> { + ) -> Result> { Ok(self.inner.genesis_authorities.clone()).map(NativeOrEncoded::Native) } } +impl GenesisAuthoritySetProvider for TestApi { + fn get(&self) -> Result { + Ok(self.genesis_authorities.clone()) + } +} + impl AuthoritySetForFinalityProver for TestApi { - fn authorities(&self, block: &BlockId) -> Result> { - let runtime_api = RuntimeApi { inner: self.clone() }; - runtime_api.GrandpaApi_grandpa_authorities_runtime_api_impl(block, ExecutionContext::Syncing, None, Vec::new()) - .map(|v| match v { - NativeOrEncoded::Native(value) => value, - _ => unreachable!("only providing native values"), - }) + fn authorities(&self, _block: &BlockId) -> Result { + Ok(self.genesis_authorities.clone()) } - fn prove_authorities(&self, block: &BlockId) -> Result>> { - self.authorities(block).map(|auth| vec![auth.encode()]) + fn prove_authorities(&self, block: &BlockId) -> Result { + let authorities = self.authorities(block)?; + let backend = >::from(vec![ + (None, b"authorities".to_vec(), Some(authorities.encode())) + ]); + let proof = prove_read(backend, vec![b"authorities"]) + .expect("failure proving read from in-memory storage backend"); + Ok(proof) } } @@ -294,17 +302,26 @@ impl AuthoritySetForFinalityChecker for TestApi { fn check_authorities_proof( &self, _hash: ::Hash, - _header: ::Header, - proof: Vec>, - ) -> Result> { - Decode::decode(&mut &proof[0][..]) - .map_err(|_| unreachable!("incorrect value is passed as GRANDPA authorities proof")) + header: ::Header, + proof: StorageProof, + ) -> Result { + let results = read_proof_check::( + *header.state_root(), proof, vec![b"authorities"] + ) + .expect("failure checking read proof for authorities"); + let encoded = results.get(&b"authorities"[..]) + .expect("returned map must contain all proof keys") + .as_ref() + .expect("authorities in proof is None"); + let authorities = Decode::decode(&mut &encoded[..]) + .expect("failure decoding authorities read from proof"); + Ok(authorities) } } const TEST_GOSSIP_DURATION: Duration = Duration::from_millis(500); -fn make_ids(keys: &[Ed25519Keyring]) -> Vec<(AuthorityId, u64)> { +fn make_ids(keys: &[Ed25519Keyring]) -> AuthorityList { keys.iter().map(|key| key.clone().public().into()).map(|id| (id, 1)).collect() } @@ -957,6 +974,7 @@ fn allows_reimporting_change_blocks() { finalized: false, auxiliary: Vec::new(), fork_choice: ForkChoiceStrategy::LongestChain, + allow_missing_state: false, } }; @@ -968,6 +986,7 @@ fn allows_reimporting_change_blocks() { bad_justification: false, needs_finality_proof: false, is_new_best: true, + header_only: false, }), ); @@ -1008,6 +1027,7 @@ fn test_bad_justification() { finalized: false, auxiliary: Vec::new(), fork_choice: ForkChoiceStrategy::LongestChain, + allow_missing_state: false, } }; @@ -1704,6 +1724,7 @@ fn imports_justification_for_regular_blocks_on_import() { finalized: false, auxiliary: Vec::new(), fork_choice: ForkChoiceStrategy::LongestChain, + allow_missing_state: false, }; assert_eq!( diff --git a/core/finality-grandpa/src/until_imported.rs b/client/finality-grandpa/src/until_imported.rs similarity index 99% rename from core/finality-grandpa/src/until_imported.rs rename to client/finality-grandpa/src/until_imported.rs index 5fca476a82b6a..fca67ab195e54 100644 --- a/core/finality-grandpa/src/until_imported.rs +++ b/client/finality-grandpa/src/until_imported.rs @@ -29,7 +29,7 @@ use super::{ }; use log::{debug, warn}; -use client::{BlockImportNotification, ImportNotifications}; +use client_api::{BlockImportNotification, ImportNotifications}; use futures::prelude::*; use futures::stream::Fuse; use futures03::{StreamExt as _, TryStreamExt as _}; @@ -474,7 +474,7 @@ mod tests { use tokio_timer::Delay; use test_client::runtime::{Block, Hash, Header}; use consensus_common::BlockOrigin; - use client::BlockImportNotification; + use client_api::BlockImportNotification; use futures::future::Either; use futures03::channel::mpsc; use grandpa::Precommit; diff --git a/core/finality-grandpa/src/voting_rule.rs b/client/finality-grandpa/src/voting_rule.rs similarity index 99% rename from core/finality-grandpa/src/voting_rule.rs rename to client/finality-grandpa/src/voting_rule.rs index 355fa0cd2d080..7907515074466 100644 --- a/core/finality-grandpa/src/voting_rule.rs +++ b/client/finality-grandpa/src/voting_rule.rs @@ -22,7 +22,7 @@ use std::sync::Arc; -use client::blockchain::HeaderBackend; +use client_api::blockchain::HeaderBackend; use sr_primitives::generic::BlockId; use sr_primitives::traits::{Block as BlockT, Header, NumberFor, One, Zero}; diff --git a/core/client/header-metadata/Cargo.toml b/client/header-metadata/Cargo.toml similarity index 78% rename from core/client/header-metadata/Cargo.toml rename to client/header-metadata/Cargo.toml index 43fcebb25bf96..d9917c93f1178 100644 --- a/core/client/header-metadata/Cargo.toml +++ b/client/header-metadata/Cargo.toml @@ -7,4 +7,4 @@ edition = "2018" [dependencies] lru-cache = { version = "0.1.2" } parking_lot = { version = "0.9.0" } -sr-primitives = { path = "../../sr-primitives" } +sr-primitives = { path = "../../primitives/sr-primitives" } diff --git a/core/client/header-metadata/src/lib.rs b/client/header-metadata/src/lib.rs similarity index 100% rename from core/client/header-metadata/src/lib.rs rename to client/header-metadata/src/lib.rs diff --git a/core/keystore/Cargo.toml b/client/keystore/Cargo.toml similarity index 74% rename from core/keystore/Cargo.toml rename to client/keystore/Cargo.toml index cc491337cf83a..9923a7f357d1f 100644 --- a/core/keystore/Cargo.toml +++ b/client/keystore/Cargo.toml @@ -6,8 +6,8 @@ edition = "2018" [dependencies] derive_more = "0.15.0" -primitives = { package = "substrate-primitives", path = "../primitives" } -app-crypto = { package = "substrate-application-crypto", path = "../application-crypto" } +primitives = { package = "substrate-primitives", path = "../../primitives/core" } +app-crypto = { package = "substrate-application-crypto", path = "../../primitives/application-crypto" } hex = "0.3.2" rand = "0.7.2" serde_json = "1.0.41" diff --git a/core/keystore/src/lib.rs b/client/keystore/src/lib.rs similarity index 100% rename from core/keystore/src/lib.rs rename to client/keystore/src/lib.rs diff --git a/core/network/Cargo.toml b/client/network/Cargo.toml similarity index 61% rename from core/network/Cargo.toml rename to client/network/Cargo.toml index 71e7d95bfa56a..19c720c15011f 100644 --- a/core/network/Cargo.toml +++ b/client/network/Cargo.toml @@ -22,15 +22,18 @@ linked_hash_set = "0.1.3" lru-cache = "0.1.2" rustc-hex = "2.0.1" rand = "0.7.2" -libp2p = { version = "0.12.0", default-features = false, features = ["libp2p-websocket"] } -fork-tree = { path = "../../core/utils/fork-tree" } -consensus = { package = "substrate-consensus-common", path = "../../core/consensus/common" } -client = { package = "substrate-client", path = "../../core/client" } -header_metadata = { package = "substrate-header-metadata", path = "../../core/client/header-metadata" } -sr-primitives = { path = "../../core/sr-primitives" } -primitives = { package = "substrate-primitives", path = "../../core/primitives" } +libp2p = { version = "0.13.0", default-features = false, features = ["libp2p-websocket"] } +fork-tree = { path = "../../utils/fork-tree" } +consensus = { package = "substrate-consensus-common", path = "../../primitives/consensus/common" } +client = { package = "substrate-client", path = "../" } +client-api = { package = "substrate-client-api", path = "../api" } +header_metadata = { package = "substrate-header-metadata", path = "../header-metadata" } +sr-primitives = { path = "../../primitives/sr-primitives" } +sr-arithmetic = { path = "../../primitives/sr-arithmetic" } +primitives = { package = "substrate-primitives", path = "../../primitives/core" } +block-builder = { package = "substrate-block-builder", path = "../block-builder" } codec = { package = "parity-scale-codec", version = "1.0.0", features = ["derive"] } -peerset = { package = "substrate-peerset", path = "../../core/peerset" } +peerset = { package = "substrate-peerset", path = "../../primitives/peerset" } serde = { version = "1.0.101", features = ["derive"] } serde_json = "1.0.41" slog = { version = "2.5.2", features = ["nested-values"] } @@ -39,21 +42,21 @@ smallvec = "0.6.10" tokio-io = "0.1.12" tokio = { version = "0.1.22", optional = true } unsigned-varint = { version = "0.2.2", features = ["codec"] } -keyring = { package = "substrate-keyring", path = "../../core/keyring", optional = true } -test_client = { package = "substrate-test-client", path = "../../core/test-client", optional = true } -test-client = { package = "substrate-test-runtime-client", path = "../../core/test-runtime/client", optional = true } +keyring = { package = "substrate-keyring", path = "../../primitives/keyring", optional = true } +test_client = { package = "substrate-test-client", path = "../../test/utils/client", optional = true } +test-client = { package = "substrate-test-runtime-client", path = "../../test/utils/runtime/client", optional = true } erased-serde = "0.3.9" void = "1.0.2" zeroize = "0.10.1" -babe-primitives = { package = "substrate-consensus-babe-primitives", path = "../consensus/babe/primitives" } +babe-primitives = { package = "substrate-consensus-babe-primitives", path = "../../primitives/consensus/babe" } [dev-dependencies] env_logger = "0.7.0" -keyring = { package = "substrate-keyring", path = "../../core/keyring" } +keyring = { package = "substrate-keyring", path = "../../primitives/keyring" } quickcheck = "0.9.0" rand = "0.7.2" -test-client = { package = "substrate-test-runtime-client", path = "../../core/test-runtime/client" } -test_runtime = { package = "substrate-test-runtime", path = "../../core/test-runtime" } +test-client = { package = "substrate-test-runtime-client", path = "../../test/utils/runtime/client" } +test_runtime = { package = "substrate-test-runtime", path = "../../test/utils/runtime" } tempdir = "0.3.7" tokio = "0.1.22" diff --git a/core/network/src/behaviour.rs b/client/network/src/behaviour.rs similarity index 97% rename from core/network/src/behaviour.rs rename to client/network/src/behaviour.rs index 28830b326eaee..a0299bc340ce2 100644 --- a/core/network/src/behaviour.rs +++ b/client/network/src/behaviour.rs @@ -62,11 +62,17 @@ impl, H: ExHashT> Behaviour { local_public_key: PublicKey, known_addresses: Vec<(PeerId, Multiaddr)>, enable_mdns: bool, + allow_private_ipv4: bool, ) -> Self { Behaviour { substrate, debug_info: debug_info::DebugInfoBehaviour::new(user_agent, local_public_key.clone()), - discovery: DiscoveryBehaviour::new(local_public_key, known_addresses, enable_mdns), + discovery: DiscoveryBehaviour::new( + local_public_key, + known_addresses, + enable_mdns, + allow_private_ipv4 + ), events: Vec::new(), } } diff --git a/core/network/src/chain.rs b/client/network/src/chain.rs similarity index 90% rename from core/network/src/chain.rs rename to client/network/src/chain.rs index f68942fd96d38..5207a24bf9d30 100644 --- a/core/network/src/chain.rs +++ b/client/network/src/chain.rs @@ -16,9 +16,9 @@ //! Blockchain access trait -use client::{self, Client as SubstrateClient, ClientInfo, CallExecutor}; -use client::error::Error; -use client::light::fetcher::ChangesProof; +use client::Client as SubstrateClient; +use client_api::error::Error; +use client_api::{ChangesProof, StorageProof, ClientInfo, CallExecutor}; use consensus::{BlockImport, BlockStatus, Error as ConsensusError}; use sr_primitives::traits::{Block as BlockT, Header as HeaderT}; use sr_primitives::generic::{BlockId}; @@ -46,10 +46,11 @@ pub trait Client: Send + Sync { fn justification(&self, id: &BlockId) -> Result, Error>; /// Get block header proof. - fn header_proof(&self, block_number: ::Number) -> Result<(Block::Header, Vec>), Error>; + fn header_proof(&self, block_number: ::Number) + -> Result<(Block::Header, StorageProof), Error>; /// Get storage read execution proof. - fn read_proof(&self, block: &Block::Hash, keys: &[Vec]) -> Result>, Error>; + fn read_proof(&self, block: &Block::Hash, keys: &[Vec]) -> Result; /// Get child storage read execution proof. fn read_child_proof( @@ -57,10 +58,10 @@ pub trait Client: Send + Sync { block: &Block::Hash, storage_key: &[u8], keys: &[Vec], - ) -> Result>, Error>; + ) -> Result; /// Get method execution proof. - fn execution_proof(&self, block: &Block::Hash, method: &str, data: &[u8]) -> Result<(Vec, Vec>), Error>; + fn execution_proof(&self, block: &Block::Hash, method: &str, data: &[u8]) -> Result<(Vec, StorageProof), Error>; /// Get key changes proof. fn key_changes_proof( @@ -90,7 +91,7 @@ impl FinalityProofProvider for () { } impl Client for SubstrateClient where - B: client::backend::Backend + Send + Sync + 'static, + B: client_api::backend::Backend + Send + Sync + 'static, E: CallExecutor + Send + Sync + 'static, Self: BlockImport, Block: BlockT, @@ -120,11 +121,13 @@ impl Client for SubstrateClient where (self as &SubstrateClient).justification(id) } - fn header_proof(&self, block_number: ::Number) -> Result<(Block::Header, Vec>), Error> { + fn header_proof(&self, block_number: ::Number) + -> Result<(Block::Header, StorageProof), Error> + { (self as &SubstrateClient).header_proof(&BlockId::Number(block_number)) } - fn read_proof(&self, block: &Block::Hash, keys: &[Vec]) -> Result>, Error> { + fn read_proof(&self, block: &Block::Hash, keys: &[Vec]) -> Result { (self as &SubstrateClient).read_proof(&BlockId::Hash(block.clone()), keys) } @@ -133,12 +136,12 @@ impl Client for SubstrateClient where block: &Block::Hash, storage_key: &[u8], keys: &[Vec], - ) -> Result>, Error> { + ) -> Result { (self as &SubstrateClient) .read_child_proof(&BlockId::Hash(block.clone()), storage_key, keys) } - fn execution_proof(&self, block: &Block::Hash, method: &str, data: &[u8]) -> Result<(Vec, Vec>), Error> { + fn execution_proof(&self, block: &Block::Hash, method: &str, data: &[u8]) -> Result<(Vec, StorageProof), Error> { (self as &SubstrateClient).execution_proof(&BlockId::Hash(block.clone()), method, data) } diff --git a/core/network/src/config.rs b/client/network/src/config.rs similarity index 94% rename from core/network/src/config.rs rename to client/network/src/config.rs index 445d4427bd91c..d10345c2f3817 100644 --- a/core/network/src/config.rs +++ b/client/network/src/config.rs @@ -28,12 +28,11 @@ use crate::service::{ExHashT, TransactionPool}; use bitflags::bitflags; use consensus::{block_validation::BlockAnnounceValidator, import_queue::ImportQueue}; use sr_primitives::traits::{Block as BlockT}; -use std::sync::Arc; use libp2p::identity::{Keypair, ed25519}; use libp2p::wasm_ext; use libp2p::{PeerId, Multiaddr, multiaddr}; -use std::error::Error; -use std::{io::{self, Write}, iter, fmt, fs, net::Ipv4Addr, path::{Path, PathBuf}}; +use core::{fmt, iter}; +use std::{error::Error, fs, io::{self, Write}, net::Ipv4Addr, path::{Path, PathBuf}, sync::Arc}; use zeroize::Zeroize; /// Network initialization parameters. @@ -82,7 +81,7 @@ pub struct Params { pub specialization: S, /// Type to check incoming block announcements. - pub block_announce_validator: Box + Send> + pub block_announce_validator: Box + Send>, } bitflags! { @@ -234,7 +233,7 @@ impl From for ParseErr { } /// Network service configuration. -#[derive(Clone)] +#[derive(Clone, Debug)] pub struct NetworkConfiguration { /// Directory path to store general network configuration. None means nothing will be saved. pub config_path: Option, @@ -262,6 +261,8 @@ pub struct NetworkConfiguration { pub node_name: String, /// Configuration for the transport layer. pub transport: TransportConfig, + /// Maximum number of peers to ask the same blocks in parallel. + pub max_parallel_downloads: u32, } impl Default for NetworkConfiguration { @@ -281,8 +282,10 @@ impl Default for NetworkConfiguration { node_name: "unknown".into(), transport: TransportConfig::Normal { enable_mdns: false, + allow_private_ipv4: true, wasm_external_transport: None, }, + max_parallel_downloads: 5, } } } @@ -317,7 +320,7 @@ impl NetworkConfiguration { } /// Configuration for the transport layer. -#[derive(Clone)] +#[derive(Clone, Debug)] pub enum TransportConfig { /// Normal transport mode. Normal { @@ -325,6 +328,11 @@ pub enum TransportConfig { /// and connect to them if they support the same chain. enable_mdns: bool, + /// If true, allow connecting to private IPv4 addresses (as defined in + /// [RFC1918](https://tools.ietf.org/html/rfc1918)), unless the address has been passed in + /// [`NetworkConfiguration::reserved_nodes`] or [`NetworkConfiguration::boot_nodes`]. + allow_private_ipv4: bool, + /// Optional external implementation of a libp2p transport. Used in WASM contexts where we /// need some binding between the networking provided by the operating system or environment /// and libp2p. @@ -362,7 +370,7 @@ impl NonReservedPeerMode { /// The configuration of a node's secret key, describing the type of key /// and how it is obtained. A node's identity keypair is the result of /// the evaluation of the node key configuration. -#[derive(Clone)] +#[derive(Clone, Debug)] pub enum NodeKeyConfig { /// A Ed25519 secret key configuration. Ed25519(Secret) @@ -386,6 +394,16 @@ pub enum Secret { New } +impl fmt::Debug for Secret { + fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { + match self { + Secret::Input(_) => f.debug_tuple("Secret::Input").finish(), + Secret::File(path) => f.debug_tuple("Secret::File").field(path).finish(), + Secret::New => f.debug_tuple("Secret::New").finish(), + } + } +} + impl NodeKeyConfig { /// Evaluate a `NodeKeyConfig` to obtain an identity `Keypair`: /// diff --git a/core/network/src/debug_info.rs b/client/network/src/debug_info.rs similarity index 96% rename from core/network/src/debug_info.rs rename to client/network/src/debug_info.rs index 3b0d5513ef281..0283853a5fa14 100644 --- a/core/network/src/debug_info.rs +++ b/client/network/src/debug_info.rs @@ -21,7 +21,7 @@ use libp2p::Multiaddr; use libp2p::core::{ConnectedPoint, either::EitherOutput, PeerId, PublicKey}; use libp2p::swarm::{IntoProtocolsHandler, IntoProtocolsHandlerSelect, ProtocolsHandler}; use libp2p::swarm::{NetworkBehaviour, NetworkBehaviourAction, PollParameters}; -use libp2p::identify::{Identify, IdentifyEvent, protocol::IdentifyInfo}; +use libp2p::identify::{Identify, IdentifyEvent, IdentifyInfo}; use libp2p::ping::{Ping, PingConfig, PingEvent, PingSuccess}; use log::{debug, trace, error}; use std::collections::hash_map::Entry; @@ -287,16 +287,14 @@ where TSubstream: AsyncRead + AsyncWrite { Async::NotReady => break, Async::Ready(NetworkBehaviourAction::GenerateEvent(event)) => { match event { - IdentifyEvent::Identified { peer_id, info, .. } => { + IdentifyEvent::Received { peer_id, info, .. } => { self.handle_identify_report(&peer_id, &info); let event = DebugInfoEvent::Identified { peer_id, info }; return Async::Ready(NetworkBehaviourAction::GenerateEvent(event)); } - IdentifyEvent::Error { .. } => {} - IdentifyEvent::SendBack { result: Err(ref err), ref peer_id } => - debug!(target: "sub-libp2p", "Error when sending back identify info \ - to {:?} => {}", peer_id, err), - IdentifyEvent::SendBack { .. } => {} + IdentifyEvent::Error { peer_id, error } => + debug!(target: "sub-libp2p", "Identification with peer {:?} failed => {}", peer_id, error), + IdentifyEvent::Sent { .. } => {} } }, Async::Ready(NetworkBehaviourAction::DialAddress { address }) => diff --git a/core/network/src/discovery.rs b/client/network/src/discovery.rs similarity index 92% rename from core/network/src/discovery.rs rename to client/network/src/discovery.rs index a9cf61d040f0f..2e0a6fe2447ea 100644 --- a/core/network/src/discovery.rs +++ b/client/network/src/discovery.rs @@ -63,6 +63,7 @@ use libp2p::multiaddr::Protocol; use log::{debug, info, trace, warn}; use std::{cmp, collections::VecDeque, time::Duration}; use tokio_io::{AsyncRead, AsyncWrite}; +use primitives::hexdisplay::HexDisplay; /// Implementation of `NetworkBehaviour` that discovers the nodes on the network. pub struct DiscoveryBehaviour { @@ -84,6 +85,9 @@ pub struct DiscoveryBehaviour { local_peer_id: PeerId, /// Number of nodes we're currently connected to. num_connections: u64, + /// If false, `addresses_of_peer` won't return any private IPv4 address, except for the ones + /// stored in `user_defined`. + allow_private_ipv4: bool, } impl DiscoveryBehaviour { @@ -93,7 +97,8 @@ impl DiscoveryBehaviour { pub fn new( local_public_key: PublicKey, user_defined: Vec<(PeerId, Multiaddr)>, - enable_mdns: bool + enable_mdns: bool, + allow_private_ipv4: bool, ) -> Self { if enable_mdns { #[cfg(target_os = "unknown")] @@ -115,6 +120,7 @@ impl DiscoveryBehaviour { discoveries: VecDeque::new(), local_peer_id: local_public_key.into_peer_id(), num_connections: 0, + allow_private_ipv4, #[cfg(not(target_os = "unknown"))] mdns: if enable_mdns { match Mdns::new() { @@ -213,9 +219,27 @@ where let mut list = self.user_defined.iter() .filter_map(|(p, a)| if p == peer_id { Some(a.clone()) } else { None }) .collect::>(); - list.extend(self.kademlia.addresses_of_peer(peer_id)); - #[cfg(not(target_os = "unknown"))] - list.extend(self.mdns.addresses_of_peer(peer_id)); + + { + let mut list_to_filter = self.kademlia.addresses_of_peer(peer_id); + #[cfg(not(target_os = "unknown"))] + list_to_filter.extend(self.mdns.addresses_of_peer(peer_id)); + + if !self.allow_private_ipv4 { + list_to_filter.retain(|addr| { + if let Some(Protocol::Ip4(addr)) = addr.iter().next() { + if addr.is_private() { + return false; + } + } + + true + }); + } + + list.extend(list_to_filter); + } + trace!(target: "sub-libp2p", "Addresses of {:?} are {:?}", peer_id, list); if list.is_empty() { if self.kademlia.kbuckets_entries().any(|p| p == peer_id) { @@ -316,16 +340,16 @@ where KademliaEvent::GetClosestPeersResult(res) => { match res { Err(GetClosestPeersError::Timeout { key, peers }) => { - warn!(target: "sub-libp2p", - "Libp2p => Query for {:?} timed out with {:?} results", - key, peers.len()); + debug!(target: "sub-libp2p", + "Libp2p => Query for {:?} timed out with {} results", + HexDisplay::from(&key), peers.len()); }, Ok(ok) => { trace!(target: "sub-libp2p", "Libp2p => Query for {:?} yielded {:?} results", - ok.key, ok.peers.len()); + HexDisplay::from(&ok.key), ok.peers.len()); if ok.peers.is_empty() && self.num_connections != 0 { - warn!(target: "sub-libp2p", "Libp2p => Random Kademlia query has yielded empty \ + debug!(target: "sub-libp2p", "Libp2p => Random Kademlia query has yielded empty \ results"); } } @@ -436,19 +460,27 @@ mod tests { // Build swarms whose behaviour is `DiscoveryBehaviour`. let mut swarms = (0..25).map(|_| { let keypair = Keypair::generate_ed25519(); + let keypair2 = keypair.clone(); let transport = MemoryTransport - .with_upgrade(libp2p::secio::SecioConfig::new(keypair.clone())) .and_then(move |out, endpoint| { - let peer_id = out.remote_key.into_peer_id(); + let secio = libp2p::secio::SecioConfig::new(keypair2); + libp2p::core::upgrade::apply( + out, + secio, + endpoint, + libp2p::core::upgrade::Version::V1 + ) + }) + .and_then(move |(peer_id, stream), endpoint| { let peer_id2 = peer_id.clone(); let upgrade = libp2p::yamux::Config::default() .map_inbound(move |muxer| (peer_id, muxer)) .map_outbound(move |muxer| (peer_id2, muxer)); - upgrade::apply(out.stream, upgrade, endpoint) + upgrade::apply(stream, upgrade, endpoint, libp2p::core::upgrade::Version::V1) }); - let behaviour = DiscoveryBehaviour::new(keypair.public(), user_defined.clone(), false); + let behaviour = DiscoveryBehaviour::new(keypair.public(), user_defined.clone(), false, true); let mut swarm = Swarm::new(transport, behaviour, keypair.public().into_peer_id()); let listen_addr: Multiaddr = format!("/memory/{}", rand::random::()).parse().unwrap(); diff --git a/core/network/src/error.rs b/client/network/src/error.rs similarity index 97% rename from core/network/src/error.rs rename to client/network/src/error.rs index c3f89e43c1710..a33f0280abd42 100644 --- a/core/network/src/error.rs +++ b/client/network/src/error.rs @@ -16,7 +16,7 @@ //! Substrate network possible errors. -use client; +use client_api; use libp2p::{PeerId, Multiaddr}; @@ -31,7 +31,7 @@ pub enum Error { /// Io error Io(std::io::Error), /// Client error - Client(client::error::Error), + Client(client_api::error::Error), /// The same bootnode (based on address) is registered with two different peer ids. #[display( fmt = "The same bootnode (`{}`) is registered with two different peer ids: `{}` and `{}`", diff --git a/core/network/src/legacy_proto/behaviour.rs b/client/network/src/legacy_proto/behaviour.rs similarity index 97% rename from core/network/src/legacy_proto/behaviour.rs rename to client/network/src/legacy_proto/behaviour.rs index 1c83329ba0e10..d1d378174a21b 100644 --- a/core/network/src/legacy_proto/behaviour.rs +++ b/client/network/src/legacy_proto/behaviour.rs @@ -17,7 +17,7 @@ use crate::{DiscoveryNetBehaviour, config::ProtocolId}; use crate::legacy_proto::handler::{CustomProtoHandlerProto, CustomProtoHandlerOut, CustomProtoHandlerIn}; use crate::legacy_proto::upgrade::RegisteredProtocol; -use crate::protocol::message::Message; +use bytes::BytesMut; use fnv::FnvHashMap; use futures::prelude::*; use futures03::{compat::Compat, TryFutureExt as _, StreamExt as _, TryStreamExt as _}; @@ -25,7 +25,6 @@ use libp2p::core::{ConnectedPoint, Multiaddr, PeerId}; use libp2p::swarm::{NetworkBehaviour, NetworkBehaviourAction, PollParameters}; use log::{debug, error, trace, warn}; use rand::distributions::{Distribution as _, Uniform}; -use sr_primitives::traits::Block as BlockT; use smallvec::SmallVec; use std::{borrow::Cow, collections::hash_map::Entry, cmp, error, marker::PhantomData, mem, pin::Pin}; use std::time::{Duration, Instant}; @@ -61,9 +60,9 @@ use tokio_io::{AsyncRead, AsyncWrite}; /// Note that this "banning" system is not an actual ban. If a "banned" node tries to connect to /// us, we accept the connection. The "banning" system is only about delaying dialing attempts. /// -pub struct LegacyProto { +pub struct LegacyProto< TSubstream> { /// List of protocols to open with peers. Never modified. - protocol: RegisteredProtocol, + protocol: RegisteredProtocol, /// Receiver for instructions about who to connect to or disconnect from. peerset: peerset::Peerset, @@ -80,7 +79,7 @@ pub struct LegacyProto { next_incoming_index: peerset::IncomingIndex, /// Events to produce from `poll()`. - events: SmallVec<[NetworkBehaviourAction, LegacyProtoOut>; 4]>, + events: SmallVec<[NetworkBehaviourAction; 4]>, /// Marker to pin the generics. marker: PhantomData, @@ -189,7 +188,7 @@ struct IncomingPeer { /// Event that can be emitted by the `LegacyProto`. #[derive(Debug)] -pub enum LegacyProtoOut { +pub enum LegacyProtoOut { /// Opened a custom protocol with the remote. CustomProtocolOpen { /// Version of the protocol that has been opened. @@ -213,7 +212,7 @@ pub enum LegacyProtoOut { /// Id of the peer the message came from. peer_id: PeerId, /// Message that has been received. - message: Message, + message: BytesMut, }, /// The substream used by the protocol is pretty large. We should print avoid sending more @@ -222,11 +221,11 @@ pub enum LegacyProtoOut { /// Id of the peer which is clogged. peer_id: PeerId, /// Copy of the messages that are within the buffer, for further diagnostic. - messages: Vec>, + messages: Vec>, }, } -impl LegacyProto { +impl LegacyProto { /// Creates a `CustomProtos`. pub fn new( protocol: impl Into, @@ -350,8 +349,7 @@ impl LegacyProto { /// /// Also note that even we have a valid open substream, it may in fact be already closed /// without us knowing, in which case the packet will not be received. - pub fn send_packet(&mut self, target: &PeerId, message: Message) - where B: BlockT { + pub fn send_packet(&mut self, target: &PeerId, message: Vec) { if !self.is_open(target) { return; } @@ -607,7 +605,7 @@ impl LegacyProto { } } -impl DiscoveryNetBehaviour for LegacyProto { +impl DiscoveryNetBehaviour for LegacyProto { fn add_discovered_nodes(&mut self, peer_ids: impl Iterator) { self.peerset.discovered(peer_ids.into_iter().map(|peer_id| { debug!(target: "sub-libp2p", "PSM <= Discovered({:?})", peer_id); @@ -616,13 +614,12 @@ impl DiscoveryNetBehaviour for LegacyProto } } -impl NetworkBehaviour for LegacyProto +impl NetworkBehaviour for LegacyProto where TSubstream: AsyncRead + AsyncWrite, - B: BlockT, { - type ProtocolsHandler = CustomProtoHandlerProto; - type OutEvent = LegacyProtoOut; + type ProtocolsHandler = CustomProtoHandlerProto; + type OutEvent = LegacyProtoOut; fn new_handler(&mut self) -> Self::ProtocolsHandler { CustomProtoHandlerProto::new(self.protocol.clone()) @@ -825,7 +822,7 @@ where fn inject_node_event( &mut self, source: PeerId, - event: CustomProtoHandlerOut, + event: CustomProtoHandlerOut, ) { match event { CustomProtoHandlerOut::CustomProtocolClosed { reason } => { @@ -954,7 +951,7 @@ where _params: &mut impl PollParameters, ) -> Async< NetworkBehaviourAction< - CustomProtoHandlerIn, + CustomProtoHandlerIn, Self::OutEvent, >, > { diff --git a/core/network/src/legacy_proto/handler.rs b/client/network/src/legacy_proto/handler.rs similarity index 91% rename from core/network/src/legacy_proto/handler.rs rename to client/network/src/legacy_proto/handler.rs index 3fe88d3cfd410..7bdbe4a31ff7c 100644 --- a/core/network/src/legacy_proto/handler.rs +++ b/client/network/src/legacy_proto/handler.rs @@ -15,7 +15,7 @@ // along with Substrate. If not, see . use crate::legacy_proto::upgrade::{RegisteredProtocol, RegisteredProtocolEvent, RegisteredProtocolSubstream}; -use crate::protocol::message::Message; +use bytes::BytesMut; use futures::prelude::*; use futures03::{compat::Compat, TryFutureExt as _}; use futures_timer::Delay; @@ -29,7 +29,6 @@ use libp2p::swarm::{ SubstreamProtocol, }; use log::{debug, error}; -use sr_primitives::traits::Block as BlockT; use smallvec::{smallvec, SmallVec}; use std::{borrow::Cow, error, fmt, io, marker::PhantomData, mem, time::Duration}; use tokio_io::{AsyncRead, AsyncWrite}; @@ -88,21 +87,20 @@ use tokio_io::{AsyncRead, AsyncWrite}; /// We consider that we are now "closed" if the remote closes all the existing substreams. /// Re-opening it can then be performed by closing all active substream and re-opening one. /// -pub struct CustomProtoHandlerProto { +pub struct CustomProtoHandlerProto { /// Configuration for the protocol upgrade to negotiate. - protocol: RegisteredProtocol, + protocol: RegisteredProtocol, /// Marker to pin the generic type. marker: PhantomData, } -impl CustomProtoHandlerProto +impl CustomProtoHandlerProto where TSubstream: AsyncRead + AsyncWrite, - B: BlockT, { /// Builds a new `CustomProtoHandlerProto`. - pub fn new(protocol: RegisteredProtocol) -> Self { + pub fn new(protocol: RegisteredProtocol) -> Self { CustomProtoHandlerProto { protocol, marker: PhantomData, @@ -110,14 +108,13 @@ where } } -impl IntoProtocolsHandler for CustomProtoHandlerProto +impl IntoProtocolsHandler for CustomProtoHandlerProto where TSubstream: AsyncRead + AsyncWrite, - B: BlockT, { - type Handler = CustomProtoHandler; + type Handler = CustomProtoHandler; - fn inbound_protocol(&self) -> RegisteredProtocol { + fn inbound_protocol(&self) -> RegisteredProtocol { self.protocol.clone() } @@ -136,12 +133,12 @@ where } /// The actual handler once the connection has been established. -pub struct CustomProtoHandler { +pub struct CustomProtoHandler { /// Configuration for the protocol upgrade to negotiate. - protocol: RegisteredProtocol, + protocol: RegisteredProtocol, /// State of the communications with the remote. - state: ProtocolState, + state: ProtocolState, /// Identifier of the node we're talking to. Used only for logging purposes and shouldn't have /// any influence on the behaviour. @@ -155,15 +152,15 @@ pub struct CustomProtoHandler { /// /// This queue must only ever be modified to insert elements at the back, or remove the first /// element. - events_queue: SmallVec<[ProtocolsHandlerEvent, (), CustomProtoHandlerOut>; 16]>, + events_queue: SmallVec<[ProtocolsHandlerEvent; 16]>, } /// State of the handler. -enum ProtocolState { +enum ProtocolState { /// Waiting for the behaviour to tell the handler whether it is enabled or disabled. Init { /// List of substreams opened by the remote but that haven't been processed yet. - substreams: SmallVec<[RegisteredProtocolSubstream; 6]>, + substreams: SmallVec<[RegisteredProtocolSubstream; 6]>, /// Deadline after which the initialization is abnormally long. init_deadline: Compat, }, @@ -179,9 +176,9 @@ enum ProtocolState { /// If we are in this state, we have sent a `CustomProtocolOpen` message to the outside. Normal { /// The substreams where bidirectional communications happen. - substreams: SmallVec<[RegisteredProtocolSubstream; 4]>, + substreams: SmallVec<[RegisteredProtocolSubstream; 4]>, /// Contains substreams which are being shut down. - shutdown: SmallVec<[RegisteredProtocolSubstream; 4]>, + shutdown: SmallVec<[RegisteredProtocolSubstream; 4]>, }, /// We are disabled. Contains substreams that are being closed. @@ -189,7 +186,7 @@ enum ProtocolState { /// outside or we have never sent any `CustomProtocolOpen` in the first place. Disabled { /// List of substreams to shut down. - shutdown: SmallVec<[RegisteredProtocolSubstream; 6]>, + shutdown: SmallVec<[RegisteredProtocolSubstream; 6]>, /// If true, we should reactivate the handler after all the substreams in `shutdown` have /// been closed. @@ -210,7 +207,7 @@ enum ProtocolState { /// Event that can be received by a `CustomProtoHandler`. #[derive(Debug)] -pub enum CustomProtoHandlerIn { +pub enum CustomProtoHandlerIn { /// The node should start using custom protocols. Enable, @@ -220,13 +217,13 @@ pub enum CustomProtoHandlerIn { /// Sends a message through a custom protocol substream. SendCustomMessage { /// The message to send. - message: Message, + message: Vec, }, } /// Event that can be emitted by a `CustomProtoHandler`. #[derive(Debug)] -pub enum CustomProtoHandlerOut { +pub enum CustomProtoHandlerOut { /// Opened a custom protocol with the remote. CustomProtocolOpen { /// Version of the protocol that has been opened. @@ -242,14 +239,14 @@ pub enum CustomProtoHandlerOut { /// Receives a message on a custom protocol substream. CustomMessage { /// Message that has been received. - message: Message, + message: BytesMut, }, /// A substream to the remote is clogged. The send buffer is very large, and we should print /// a diagnostic message and/or avoid sending more data. Clogged { /// Copy of the messages that are within the buffer, for further diagnostic. - messages: Vec>, + messages: Vec>, }, /// An error has happened on the protocol level with this node. @@ -261,10 +258,9 @@ pub enum CustomProtoHandlerOut { }, } -impl CustomProtoHandler +impl CustomProtoHandler where TSubstream: AsyncRead + AsyncWrite, - B: BlockT, { /// Enables the handler. fn enable(&mut self) { @@ -342,7 +338,7 @@ where /// Polls the state for events. Optionally returns an event to produce. #[must_use] fn poll_state(&mut self) - -> Option, (), CustomProtoHandlerOut>> { + -> Option> { match mem::replace(&mut self.state, ProtocolState::Poisoned) { ProtocolState::Poisoned => { error!(target: "sub-libp2p", "Handler with {:?} is in poisoned state", @@ -471,7 +467,7 @@ where /// Called by `inject_fully_negotiated_inbound` and `inject_fully_negotiated_outbound`. fn inject_fully_negotiated( &mut self, - mut substream: RegisteredProtocolSubstream + mut substream: RegisteredProtocolSubstream ) { self.state = match mem::replace(&mut self.state, ProtocolState::Poisoned) { ProtocolState::Poisoned => { @@ -516,7 +512,7 @@ where } /// Sends a message to the remote. - fn send_message(&mut self, message: Message) { + fn send_message(&mut self, message: Vec) { match self.state { ProtocolState::Normal { ref mut substreams, .. } => substreams[0].send_message(message), @@ -527,14 +523,14 @@ where } } -impl ProtocolsHandler for CustomProtoHandler -where TSubstream: AsyncRead + AsyncWrite, B: BlockT { - type InEvent = CustomProtoHandlerIn; - type OutEvent = CustomProtoHandlerOut; +impl ProtocolsHandler for CustomProtoHandler +where TSubstream: AsyncRead + AsyncWrite { + type InEvent = CustomProtoHandlerIn; + type OutEvent = CustomProtoHandlerOut; type Substream = TSubstream; type Error = ConnectionKillError; - type InboundProtocol = RegisteredProtocol; - type OutboundProtocol = RegisteredProtocol; + type InboundProtocol = RegisteredProtocol; + type OutboundProtocol = RegisteredProtocol; type OutboundOpenInfo = (); fn listen_protocol(&self) -> SubstreamProtocol { @@ -556,7 +552,7 @@ where TSubstream: AsyncRead + AsyncWrite, B: BlockT { self.inject_fully_negotiated(proto); } - fn inject_event(&mut self, message: CustomProtoHandlerIn) { + fn inject_event(&mut self, message: CustomProtoHandlerIn) { match message { CustomProtoHandlerIn::Disable => self.disable(), CustomProtoHandlerIn::Enable => self.enable(), @@ -613,7 +609,7 @@ where TSubstream: AsyncRead + AsyncWrite, B: BlockT { } } -impl fmt::Debug for CustomProtoHandler +impl fmt::Debug for CustomProtoHandler where TSubstream: AsyncRead + AsyncWrite, { @@ -625,9 +621,9 @@ where /// Given a list of substreams, tries to shut them down. The substreams that have been successfully /// shut down are removed from the list. -fn shutdown_list - (list: &mut SmallVec>>) -where TSubstream: AsyncRead + AsyncWrite, B: BlockT { +fn shutdown_list + (list: &mut SmallVec>>) +where TSubstream: AsyncRead + AsyncWrite { 'outer: for n in (0..list.len()).rev() { let mut substream = list.swap_remove(n); loop { diff --git a/core/network/src/legacy_proto/mod.rs b/client/network/src/legacy_proto/mod.rs similarity index 100% rename from core/network/src/legacy_proto/mod.rs rename to client/network/src/legacy_proto/mod.rs diff --git a/core/network/src/legacy_proto/tests.rs b/client/network/src/legacy_proto/tests.rs similarity index 88% rename from core/network/src/legacy_proto/tests.rs rename to client/network/src/legacy_proto/tests.rs index 8fd47843df2e5..dc6d40eb040d7 100644 --- a/core/network/src/legacy_proto/tests.rs +++ b/client/network/src/legacy_proto/tests.rs @@ -17,6 +17,7 @@ #![cfg(test)] use futures::{future, prelude::*, try_ready}; +use codec::{Encode, Decode}; use libp2p::core::nodes::Substream; use libp2p::core::{ConnectedPoint, transport::boxed::Boxed, muxing::StreamMuxerBox}; use libp2p::swarm::{Swarm, ProtocolsHandler, IntoProtocolsHandler}; @@ -24,9 +25,9 @@ use libp2p::swarm::{PollParameters, NetworkBehaviour, NetworkBehaviourAction}; use libp2p::{PeerId, Multiaddr, Transport}; use rand::seq::SliceRandom; use std::{io, time::Duration, time::Instant}; -use test_client::runtime::Block; -use crate::message::generic::Message; +use crate::message::Message; use crate::legacy_proto::{LegacyProto, LegacyProtoOut}; +use test_client::runtime::Block; /// Builds two nodes that have each other as bootstrap nodes. /// This is to be used only for testing, and a panic will happen if something goes wrong. @@ -43,14 +44,27 @@ fn build_nodes() .collect(); for index in 0 .. 2 { + let keypair = keypairs[index].clone(); let transport = libp2p::core::transport::MemoryTransport - .with_upgrade(libp2p::secio::SecioConfig::new(keypairs[index].clone())) .and_then(move |out, endpoint| { - let peer_id = out.remote_key.into_peer_id(); - libp2p::core::upgrade::apply(out.stream, libp2p::yamux::Config::default(), endpoint) + let secio = libp2p::secio::SecioConfig::new(keypair); + libp2p::core::upgrade::apply( + out, + secio, + endpoint, + libp2p::core::upgrade::Version::V1 + ) + }) + .and_then(move |(peer_id, stream), endpoint| { + libp2p::core::upgrade::apply( + stream, + libp2p::yamux::Config::default(), + endpoint, + libp2p::core::upgrade::Version::V1 + ) .map(|muxer| (peer_id, libp2p::core::muxing::StreamMuxerBox::new(muxer))) }) - .with_timeout(Duration::from_secs(20)) + .timeout(Duration::from_secs(20)) .map_err(|err| io::Error::new(io::ErrorKind::Other, err)) .boxed(); @@ -101,12 +115,12 @@ fn build_nodes() /// Wraps around the `CustomBehaviour` network behaviour, and adds hardcoded node addresses to it. struct CustomProtoWithAddr { - inner: LegacyProto>, + inner: LegacyProto>, addrs: Vec<(PeerId, Multiaddr)>, } impl std::ops::Deref for CustomProtoWithAddr { - type Target = LegacyProto>; + type Target = LegacyProto>; fn deref(&self) -> &Self::Target { &self.inner @@ -121,8 +135,8 @@ impl std::ops::DerefMut for CustomProtoWithAddr { impl NetworkBehaviour for CustomProtoWithAddr { type ProtocolsHandler = - > as NetworkBehaviour>::ProtocolsHandler; - type OutEvent = > as NetworkBehaviour>::OutEvent; + > as NetworkBehaviour>::ProtocolsHandler; + type OutEvent = > as NetworkBehaviour>::OutEvent; fn new_handler(&mut self) -> Self::ProtocolsHandler { self.inner.new_handler() @@ -209,7 +223,7 @@ fn two_nodes_transfer_lots_of_packets() { for n in 0 .. NUM_PACKETS { service1.send_packet( &peer_id, - Message::ChainSpecific(vec![(n % 256) as u8]) + Message::::ChainSpecific(vec![(n % 256) as u8]).encode() ); } }, @@ -223,11 +237,16 @@ fn two_nodes_transfer_lots_of_packets() { loop { match try_ready!(service2.poll()) { Some(LegacyProtoOut::CustomProtocolOpen { .. }) => {}, - Some(LegacyProtoOut::CustomMessage { message: Message::ChainSpecific(message), .. }) => { - assert_eq!(message.len(), 1); - packet_counter += 1; - if packet_counter == NUM_PACKETS { - return Ok(Async::Ready(())) + Some(LegacyProtoOut::CustomMessage { message, .. }) => { + match Message::::decode(&mut &message[..]).unwrap() { + Message::::ChainSpecific(message) => { + assert_eq!(message.len(), 1); + packet_counter += 1; + if packet_counter == NUM_PACKETS { + return Ok(Async::Ready(())) + } + }, + _ => panic!(), } } _ => panic!(), @@ -248,7 +267,7 @@ fn basic_two_nodes_requests_in_parallel() { let mut to_send = Vec::new(); for _ in 0..200 { // Note: don't make that number too high or the CPU usage will explode. let msg = (0..10).map(|_| rand::random::()).collect::>(); - to_send.push(Message::ChainSpecific(msg)); + to_send.push(Message::::ChainSpecific(msg)); } to_send }; @@ -263,7 +282,7 @@ fn basic_two_nodes_requests_in_parallel() { match try_ready!(service1.poll()) { Some(LegacyProtoOut::CustomProtocolOpen { peer_id, .. }) => { for msg in to_send.drain(..) { - service1.send_packet(&peer_id, msg); + service1.send_packet(&peer_id, msg.encode()); } }, _ => panic!(), @@ -276,7 +295,7 @@ fn basic_two_nodes_requests_in_parallel() { match try_ready!(service2.poll()) { Some(LegacyProtoOut::CustomProtocolOpen { .. }) => {}, Some(LegacyProtoOut::CustomMessage { message, .. }) => { - let pos = to_receive.iter().position(|m| *m == message).unwrap(); + let pos = to_receive.iter().position(|m| m.encode() == message).unwrap(); to_receive.remove(pos); if to_receive.is_empty() { return Ok(Async::Ready(())) diff --git a/core/network/src/legacy_proto/upgrade.rs b/client/network/src/legacy_proto/upgrade.rs similarity index 82% rename from core/network/src/legacy_proto/upgrade.rs rename to client/network/src/legacy_proto/upgrade.rs index 8831d16f91636..fdf23ec351c8a 100644 --- a/core/network/src/legacy_proto/upgrade.rs +++ b/client/network/src/legacy_proto/upgrade.rs @@ -15,15 +15,11 @@ // along with Substrate. If not, see . use crate::config::ProtocolId; -use crate::protocol::message::Message; -use bytes::Bytes; +use bytes::{Bytes, BytesMut}; use libp2p::core::{Negotiated, Endpoint, UpgradeInfo, InboundUpgrade, OutboundUpgrade, upgrade::ProtocolName}; use libp2p::tokio_codec::Framed; -use log::debug; -use std::{collections::VecDeque, io, marker::PhantomData, vec::IntoIter as VecIntoIter}; +use std::{collections::VecDeque, io, vec::IntoIter as VecIntoIter}; use futures::{prelude::*, future, stream}; -use codec::{Decode, Encode}; -use sr_primitives::traits::Block as BlockT; use tokio_io::{AsyncRead, AsyncWrite}; use unsigned_varint::codec::UviBytes; @@ -31,7 +27,7 @@ use unsigned_varint::codec::UviBytes; /// /// Note that "a single protocol" here refers to `par` for example. However /// each protocol can have multiple different versions for networking purposes. -pub struct RegisteredProtocol { +pub struct RegisteredProtocol { /// Id of the protocol for API purposes. id: ProtocolId, /// Base name of the protocol as advertised on the network. @@ -40,11 +36,9 @@ pub struct RegisteredProtocol { /// List of protocol versions that we support. /// Ordered in descending order so that the best comes first. supported_versions: Vec, - /// Marker to pin the generic. - marker: PhantomData, } -impl RegisteredProtocol { +impl RegisteredProtocol { /// Creates a new `RegisteredProtocol`. The `custom_data` parameter will be /// passed inside the `RegisteredProtocolOutput`. pub fn new(protocol: impl Into, versions: &[u8]) @@ -62,24 +56,22 @@ impl RegisteredProtocol { tmp.sort_unstable_by(|a, b| b.cmp(&a)); tmp }, - marker: PhantomData, } } } -impl Clone for RegisteredProtocol { +impl Clone for RegisteredProtocol { fn clone(&self) -> Self { RegisteredProtocol { id: self.id.clone(), base_name: self.base_name.clone(), supported_versions: self.supported_versions.clone(), - marker: PhantomData, } } } /// Output of a `RegisteredProtocol` upgrade. -pub struct RegisteredProtocolSubstream { +pub struct RegisteredProtocolSubstream { /// If true, we are in the process of closing the sink. is_closing: bool, /// Whether the local node opened this substream (dialer), or we received this substream from @@ -96,11 +88,9 @@ pub struct RegisteredProtocolSubstream { /// If true, we have sent a "remote is clogged" event recently and shouldn't send another one /// unless the buffer empties then fills itself again. clogged_fuse: bool, - /// Marker to pin the generic. - marker: PhantomData, } -impl RegisteredProtocolSubstream { +impl RegisteredProtocolSubstream { /// Returns the version of the protocol that was negotiated. pub fn protocol_version(&self) -> u8 { self.protocol_version @@ -124,33 +114,32 @@ impl RegisteredProtocolSubstream { } /// Sends a message to the substream. - pub fn send_message(&mut self, data: Message) - where B: BlockT { + pub fn send_message(&mut self, data: Vec) { if self.is_closing { return } - self.send_queue.push_back(data.encode()); + self.send_queue.push_back(data); } } /// Event produced by the `RegisteredProtocolSubstream`. #[derive(Debug, Clone)] -pub enum RegisteredProtocolEvent { +pub enum RegisteredProtocolEvent { /// Received a message from the remote. - Message(Message), + Message(BytesMut), /// Diagnostic event indicating that the connection is clogged and we should avoid sending too /// many messages to it. Clogged { /// Copy of the messages that are within the buffer, for further diagnostic. - messages: Vec>, + messages: Vec>, }, } -impl Stream for RegisteredProtocolSubstream -where TSubstream: AsyncRead + AsyncWrite, B: BlockT { - type Item = RegisteredProtocolEvent; +impl Stream for RegisteredProtocolSubstream +where TSubstream: AsyncRead + AsyncWrite { + type Item = RegisteredProtocolEvent; type Error = io::Error; fn poll(&mut self) -> Poll, Self::Error> { @@ -179,8 +168,7 @@ where TSubstream: AsyncRead + AsyncWrite, B: BlockT { self.clogged_fuse = true; return Ok(Async::Ready(Some(RegisteredProtocolEvent::Clogged { messages: self.send_queue.iter() - .map(|m| Decode::decode(&mut &m[..])) - .filter_map(Result::ok) + .map(|m| m.clone()) .collect(), }))) } @@ -199,15 +187,7 @@ where TSubstream: AsyncRead + AsyncWrite, B: BlockT { // Note that `inner` is wrapped in a `Fuse`, therefore we can poll it forever. match self.inner.poll()? { Async::Ready(Some(data)) => { - let message = as Decode>::decode(&mut &data[..]) - .map_err(|err| { - debug!( - target: "sub-libp2p", - "Couldn't decode packet sent by the remote: {:?}: {}", data, err.what(), - ); - io::ErrorKind::InvalidData - })?; - Ok(Async::Ready(Some(RegisteredProtocolEvent::Message(message)))) + Ok(Async::Ready(Some(RegisteredProtocolEvent::Message(data)))) } Async::Ready(None) => if !self.requires_poll_complete && self.send_queue.is_empty() { @@ -220,7 +200,7 @@ where TSubstream: AsyncRead + AsyncWrite, B: BlockT { } } -impl UpgradeInfo for RegisteredProtocol { +impl UpgradeInfo for RegisteredProtocol { type Info = RegisteredProtocolName; type InfoIter = VecIntoIter; @@ -255,10 +235,10 @@ impl ProtocolName for RegisteredProtocolName { } } -impl InboundUpgrade for RegisteredProtocol +impl InboundUpgrade for RegisteredProtocol where TSubstream: AsyncRead + AsyncWrite, { - type Output = RegisteredProtocolSubstream; + type Output = RegisteredProtocolSubstream; type Future = future::FutureResult; type Error = io::Error; @@ -281,12 +261,11 @@ where TSubstream: AsyncRead + AsyncWrite, inner: framed.fuse(), protocol_version: info.version, clogged_fuse: false, - marker: PhantomData, }) } } -impl OutboundUpgrade for RegisteredProtocol +impl OutboundUpgrade for RegisteredProtocol where TSubstream: AsyncRead + AsyncWrite, { type Output = >::Output; @@ -308,7 +287,6 @@ where TSubstream: AsyncRead + AsyncWrite, inner: framed.fuse(), protocol_version: info.version, clogged_fuse: false, - marker: PhantomData, }) } } diff --git a/core/network/src/lib.rs b/client/network/src/lib.rs similarity index 100% rename from core/network/src/lib.rs rename to client/network/src/lib.rs diff --git a/core/network/src/on_demand_layer.rs b/client/network/src/on_demand_layer.rs similarity index 98% rename from core/network/src/on_demand_layer.rs rename to client/network/src/on_demand_layer.rs index cdcb99a25f99e..4a5ec4adcfb40 100644 --- a/core/network/src/on_demand_layer.rs +++ b/client/network/src/on_demand_layer.rs @@ -22,8 +22,8 @@ use std::sync::Arc; use futures::{prelude::*, sync::mpsc, sync::oneshot}; use futures03::compat::{Compat01As03, Future01CompatExt as _}; use parking_lot::Mutex; -use client::error::Error as ClientError; -use client::light::fetcher::{Fetcher, FetchChecker, RemoteHeaderRequest, +use client_api::error::Error as ClientError; +use client_api::{Fetcher, FetchChecker, RemoteHeaderRequest, RemoteCallRequest, RemoteReadRequest, RemoteChangesRequest, RemoteReadChildRequest, RemoteBodyRequest}; use sr_primitives::traits::{Block as BlockT, Header as HeaderT, NumberFor}; diff --git a/core/network/src/protocol.rs b/client/network/src/protocol.rs similarity index 86% rename from core/network/src/protocol.rs rename to client/network/src/protocol.rs index 2e036cf1183f1..780a071505a65 100644 --- a/core/network/src/protocol.rs +++ b/client/network/src/protocol.rs @@ -16,6 +16,7 @@ use crate::{DiscoveryNetBehaviour, config::ProtocolId}; use crate::legacy_proto::{LegacyProto, LegacyProtoOut}; +use bytes::BytesMut; use futures::prelude::*; use futures03::{StreamExt as _, TryStreamExt as _}; use libp2p::{Multiaddr, PeerId}; @@ -28,14 +29,14 @@ use consensus::{ block_validation::BlockAnnounceValidator, import_queue::{BlockImportResult, BlockImportError, IncomingBlock, Origin} }; +use codec::{Decode, Encode}; use sr_primitives::{generic::BlockId, ConsensusEngineId, Justification}; use sr_primitives::traits::{ - Block as BlockT, Header as HeaderT, NumberFor, One, Zero, - CheckedSub, SaturatedConversion + Block as BlockT, Header as HeaderT, NumberFor, One, Zero, CheckedSub }; +use sr_arithmetic::traits::SaturatedConversion; use message::{BlockAnnounce, BlockAttributes, Direction, FromBlock, Message, RequestId}; use message::generic::{Message as GenericMessage, ConsensusMessage}; -use event::Event; use consensus_gossip::{ConsensusGossip, MessageRecipient as GossipMessageRecipient}; use light_dispatch::{LightDispatch, LightDispatchNetwork, RequestData}; use specialization::NetworkSpecialization; @@ -45,10 +46,11 @@ use crate::config::{BoxFinalityProofRequestBuilder, Roles}; use rustc_hex::ToHex; use std::collections::{BTreeMap, HashMap}; use std::sync::Arc; +use std::fmt::Write; use std::{cmp, num::NonZeroUsize, time}; use log::{trace, debug, warn, error}; use crate::chain::{Client, FinalityProofProvider}; -use client::light::fetcher::{FetchChecker, ChangesProof}; +use client_api::{FetchChecker, ChangesProof, StorageProof}; use crate::error; use util::LruHashSet; @@ -66,13 +68,20 @@ const TICK_TIMEOUT: time::Duration = time::Duration::from_millis(1100); /// Interval at which we propagate exstrinsics; const PROPAGATE_TIMEOUT: time::Duration = time::Duration::from_millis(2900); +/// Maximim number of known block hashes to keep for a peer. +const MAX_KNOWN_BLOCKS: usize = 1024; // ~32kb per peer + LruHashSet overhead +/// Maximim number of known extrinsic hashes to keep for a peer. +const MAX_KNOWN_EXTRINSICS: usize = 4096; // ~128kb per peer + overhead + /// Current protocol version. -pub(crate) const CURRENT_VERSION: u32 = 4; +pub(crate) const CURRENT_VERSION: u32 = 5; /// Lowest version we support pub(crate) const MIN_VERSION: u32 = 3; // Maximum allowed entries in `BlockResponse` const MAX_BLOCK_DATA_RESPONSE: u32 = 128; +// Maximum allowed entries in `ConsensusBatch` +const MAX_CONSENSUS_MESSAGES: usize = 256; /// When light node connects to the full node and the full node is behind light node /// for at least `LIGHT_MAXIMAL_BLOCKS_DIFFERENCE` blocks, we consider it unuseful /// and disconnect to free connection slot. @@ -88,9 +97,13 @@ const UNEXPECTED_STATUS_REPUTATION_CHANGE: i32 = -(1 << 20); /// Reputation change when we are a light client and a peer is behind us. const PEER_BEHIND_US_LIGHT_REPUTATION_CHANGE: i32 = -(1 << 8); /// Reputation change when a peer sends us an extrinsic that we didn't know about. -const NEW_EXTRINSIC_REPUTATION_CHANGE: i32 = 1 << 7; +const GOOD_EXTRINSIC_REPUTATION_CHANGE: i32 = 1 << 7; +/// Reputation change when a peer sends us a bad extrinsic. +const BAD_EXTRINSIC_REPUTATION_CHANGE: i32 = -(1 << 12); /// We sent an RPC query to the given node, but it failed. const RPC_FAILED_REPUTATION_CHANGE: i32 = -(1 << 12); +/// We received a message that failed to decode. +const BAD_MESSAGE_REPUTATION_CHANGE: i32 = -(1 << 12); // Lock must always be taken in order declared here. pub struct Protocol, H: ExHashT> { @@ -114,7 +127,15 @@ pub struct Protocol, H: ExHashT> { /// When asked for a proof of finality, we use this struct to build one. finality_proof_provider: Option>>, /// Handles opening the unique substream and sending and receiving raw messages. - behaviour: LegacyProto>, + behaviour: LegacyProto>, +} + +#[derive(Default)] +struct PacketStats { + bytes_in: u64, + bytes_out: u64, + count_in: u64, + count_out: u64, } /// A peer that we are connected to @@ -152,12 +173,12 @@ pub struct PeerInfo { pub best_number: ::Number, } -struct LightDispatchIn<'a, B: BlockT> { - behaviour: &'a mut LegacyProto>, +struct LightDispatchIn<'a> { + behaviour: &'a mut LegacyProto>, peerset: peerset::PeersetHandle, } -impl<'a, B: BlockT> LightDispatchNetwork for LightDispatchIn<'a, B> { +impl<'a, B: BlockT> LightDispatchNetwork for LightDispatchIn<'a> { fn report_peer(&mut self, who: &PeerId, reputation: i32) { self.peerset.report_peer(who.clone(), reputation) } @@ -167,12 +188,12 @@ impl<'a, B: BlockT> LightDispatchNetwork for LightDispatchIn<'a, B> { } fn send_header_request(&mut self, who: &PeerId, id: RequestId, block: <::Header as HeaderT>::Number) { - let message = message::generic::Message::RemoteHeaderRequest(message::RemoteHeaderRequest { + let message: Message = message::generic::Message::RemoteHeaderRequest(message::RemoteHeaderRequest { id, block, }); - self.behaviour.send_packet(who, message) + self.behaviour.send_packet(who, message.encode()) } fn send_read_request( @@ -182,13 +203,13 @@ impl<'a, B: BlockT> LightDispatchNetwork for LightDispatchIn<'a, B> { block: ::Hash, keys: Vec>, ) { - let message = message::generic::Message::RemoteReadRequest(message::RemoteReadRequest { + let message: Message = message::generic::Message::RemoteReadRequest(message::RemoteReadRequest { id, block, keys, }); - self.behaviour.send_packet(who, message) + self.behaviour.send_packet(who, message.encode()) } fn send_read_child_request( @@ -199,14 +220,14 @@ impl<'a, B: BlockT> LightDispatchNetwork for LightDispatchIn<'a, B> { storage_key: Vec, keys: Vec>, ) { - let message = message::generic::Message::RemoteReadChildRequest(message::RemoteReadChildRequest { + let message: Message = message::generic::Message::RemoteReadChildRequest(message::RemoteReadChildRequest { id, block, storage_key, keys, }); - self.behaviour.send_packet(who, message) + self.behaviour.send_packet(who, message.encode()) } fn send_call_request( @@ -217,14 +238,14 @@ impl<'a, B: BlockT> LightDispatchNetwork for LightDispatchIn<'a, B> { method: String, data: Vec ) { - let message = message::generic::Message::RemoteCallRequest(message::RemoteCallRequest { + let message: Message = message::generic::Message::RemoteCallRequest(message::RemoteCallRequest { id, block, method, data, }); - self.behaviour.send_packet(who, message) + self.behaviour.send_packet(who, message.encode()) } fn send_changes_request( @@ -238,7 +259,7 @@ impl<'a, B: BlockT> LightDispatchNetwork for LightDispatchIn<'a, B> { storage_key: Option>, key: Vec, ) { - let message = message::generic::Message::RemoteChangesRequest(message::RemoteChangesRequest { + let message: Message = message::generic::Message::RemoteChangesRequest(message::RemoteChangesRequest { id, first, last, @@ -248,7 +269,7 @@ impl<'a, B: BlockT> LightDispatchNetwork for LightDispatchIn<'a, B> { key, }); - self.behaviour.send_packet(who, message) + self.behaviour.send_packet(who, message.encode()) } fn send_body_request( @@ -261,7 +282,7 @@ impl<'a, B: BlockT> LightDispatchNetwork for LightDispatchIn<'a, B> { direction: Direction, max: Option ) { - let message = message::generic::Message::BlockRequest(message::BlockRequest:: { + let message: Message = message::generic::Message::BlockRequest(message::BlockRequest:: { id, fields, from, @@ -270,7 +291,7 @@ impl<'a, B: BlockT> LightDispatchNetwork for LightDispatchIn<'a, B> { max, }); - self.behaviour.send_packet(who, message) + self.behaviour.send_packet(who, message.encode()) } } @@ -284,7 +305,7 @@ pub trait Context { fn disconnect_peer(&mut self, who: PeerId); /// Send a consensus message to a peer. - fn send_consensus(&mut self, who: PeerId, consensus: ConsensusMessage); + fn send_consensus(&mut self, who: PeerId, messages: Vec); /// Send a chain-specific message to a peer. fn send_chain_specific(&mut self, who: PeerId, message: Vec); @@ -292,7 +313,7 @@ pub trait Context { /// Protocol context. struct ProtocolContext<'a, B: 'a + BlockT, H: 'a + ExHashT> { - behaviour: &'a mut LegacyProto>, + behaviour: &'a mut LegacyProto>, context_data: &'a mut ContextData, peerset_handle: &'a peerset::PeersetHandle, } @@ -300,7 +321,7 @@ struct ProtocolContext<'a, B: 'a + BlockT, H: 'a + ExHashT> { impl<'a, B: BlockT + 'a, H: 'a + ExHashT> ProtocolContext<'a, B, H> { fn new( context_data: &'a mut ContextData, - behaviour: &'a mut LegacyProto>, + behaviour: &'a mut LegacyProto>, peerset_handle: &'a peerset::PeersetHandle, ) -> Self { ProtocolContext { context_data, peerset_handle, behaviour } @@ -316,20 +337,40 @@ impl<'a, B: BlockT + 'a, H: ExHashT + 'a> Context for ProtocolContext<'a, B, self.behaviour.disconnect_peer(&who) } - fn send_consensus(&mut self, who: PeerId, consensus: ConsensusMessage) { - send_message( - self.behaviour, - &mut self.context_data.peers, - who, - GenericMessage::Consensus(consensus) - ) + fn send_consensus(&mut self, who: PeerId, messages: Vec) { + if self.context_data.peers.get(&who).map_or(false, |peer| peer.info.protocol_version > 4) { + let mut batch = Vec::new(); + let len = messages.len(); + for (index, message) in messages.into_iter().enumerate() { + batch.reserve(MAX_CONSENSUS_MESSAGES); + batch.push(message); + if batch.len() == MAX_CONSENSUS_MESSAGES || index == len - 1 { + send_message:: ( + self.behaviour, + &mut self.context_data.stats, + &who, + GenericMessage::ConsensusBatch(std::mem::replace(&mut batch, Vec::new())), + ) + } + } + } else { + // Backwards compatibility + for message in messages { + send_message:: ( + self.behaviour, + &mut self.context_data.stats, + &who, + GenericMessage::Consensus(message) + ) + } + } } fn send_chain_specific(&mut self, who: PeerId, message: Vec) { - send_message( + send_message:: ( self.behaviour, - &mut self.context_data.peers, - who, + &mut self.context_data.stats, + &who, GenericMessage::ChainSpecific(message) ) } @@ -339,6 +380,7 @@ impl<'a, B: BlockT + 'a, H: ExHashT + 'a> Context for ProtocolContext<'a, B, struct ContextData { // All connected peers peers: HashMap>, + stats: HashMap<&'static str, PacketStats>, pub chain: Arc>, } @@ -347,12 +389,15 @@ struct ContextData { pub struct ProtocolConfig { /// Assigned roles. pub roles: Roles, + /// Maximum number of peers to ask the same blocks in parallel. + pub max_parallel_downloads: u32, } impl Default for ProtocolConfig { fn default() -> ProtocolConfig { ProtocolConfig { roles: Roles::FULL, + max_parallel_downloads: 5, } } } @@ -378,6 +423,7 @@ impl, H: ExHashT> Protocol { &info, finality_proof_request_builder, block_announce_validator, + config.max_parallel_downloads, ); let (peerset, peerset_handle) = peerset::Peerset::from_config(peerset_config); let versions = &((MIN_VERSION as u8)..=(CURRENT_VERSION as u8)).collect::>(); @@ -389,6 +435,7 @@ impl, H: ExHashT> Protocol { config, context_data: ContextData { peers: HashMap::new(), + stats: HashMap::new(), chain, }, light_dispatch: LightDispatch::new(checker), @@ -515,15 +562,25 @@ impl, H: ExHashT> Protocol { self.context_data.peers.iter().map(|(id, peer)| (id, &peer.info)) } - pub fn on_event(&mut self, event: Event) { - self.specialization.on_event(event); - } - pub fn on_custom_message( &mut self, who: PeerId, - message: Message, + data: BytesMut, ) -> CustomMessageOutcome { + + let message = match as Decode>::decode(&mut &data[..]) { + Ok(message) => message, + Err(err) => { + debug!(target: "sync", "Couldn't decode packet sent by {}: {:?}: {}", who, data, err.what()); + self.peerset_handle.report_peer(who.clone(), BAD_MESSAGE_REPUTATION_CHANGE); + return CustomMessageOutcome::None; + } + }; + + let mut stats = self.context_data.stats.entry(message.id()).or_default(); + stats.bytes_in += data.len() as u64; + stats.count_in += 1; + match message { GenericMessage::Status(s) => self.on_status_message(who, s), GenericMessage::BlockRequest(r) => self.on_block_request(who, r), @@ -568,13 +625,18 @@ impl, H: ExHashT> Protocol { GenericMessage::RemoteReadChildRequest(request) => self.on_remote_read_child_request(who, request), GenericMessage::Consensus(msg) => { - if self.context_data.peers.get(&who).map_or(false, |peer| peer.info.protocol_version > 2) { - self.consensus_gossip.on_incoming( - &mut ProtocolContext::new(&mut self.context_data, &mut self.behaviour, &self.peerset_handle), - who, - msg, - ); - } + self.consensus_gossip.on_incoming( + &mut ProtocolContext::new(&mut self.context_data, &mut self.behaviour, &self.peerset_handle), + who, + vec![msg], + ); + } + GenericMessage::ConsensusBatch(messages) => { + self.consensus_gossip.on_incoming( + &mut ProtocolContext::new(&mut self.context_data, &mut self.behaviour, &self.peerset_handle), + who, + messages, + ); } GenericMessage::ChainSpecific(msg) => self.specialization.on_message( &mut ProtocolContext::new(&mut self.context_data, &mut self.behaviour, &self.peerset_handle), @@ -586,15 +648,25 @@ impl, H: ExHashT> Protocol { CustomMessageOutcome::None } - fn send_message(&mut self, who: PeerId, message: Message) { - send_message::( + fn send_request(&mut self, who: &PeerId, message: Message) { + send_request::( &mut self.behaviour, + &mut self.context_data.stats, &mut self.context_data.peers, who, message, ); } + fn send_message(&mut self, who: &PeerId, message: Message) { + send_message::( + &mut self.behaviour, + &mut self.context_data.stats, + who, + message, + ); + } + /// Locks `self` and returns a context plus the `ConsensusGossip` struct. pub fn consensus_gossip_lock<'a>( &'a mut self, @@ -627,7 +699,7 @@ impl, H: ExHashT> Protocol { GossipMessageRecipient::BroadcastNew => self.consensus_gossip.multicast(&mut context, topic, message, false), GossipMessageRecipient::Peer(who) => - self.send_message(who, GenericMessage::Consensus(message)), + self.send_message(&who, GenericMessage::Consensus(message)), } } @@ -750,7 +822,7 @@ impl, H: ExHashT> Protocol { blocks: blocks, }; trace!(target: "sync", "Sending BlockResponse with {} blocks", response.blocks.len()); - self.send_message(peer, GenericMessage::BlockResponse(response)) + self.send_message(&peer, GenericMessage::BlockResponse(response)) } /// Adjusts the reputation of a node. @@ -797,7 +869,7 @@ impl, H: ExHashT> Protocol { Ok(sync::OnBlockData::Import(origin, blocks)) => CustomMessageOutcome::BlockImport(origin, blocks), Ok(sync::OnBlockData::Request(peer, req)) => { - self.send_message(peer, GenericMessage::BlockRequest(req)); + self.send_request(&peer, GenericMessage::BlockRequest(req)); CustomMessageOutcome::None } Err(sync::BadPeer(id, repu)) => { @@ -906,8 +978,6 @@ impl, H: ExHashT> Protocol { } } - let cache_limit = NonZeroUsize::new(1_000_000).expect("1_000_000 > 0; qed"); - let info = match self.handshaking_peers.remove(&who) { Some(_handshaking) => { PeerInfo { @@ -926,8 +996,10 @@ impl, H: ExHashT> Protocol { let peer = Peer { info, block_request: None, - known_extrinsics: LruHashSet::new(cache_limit), - known_blocks: LruHashSet::new(cache_limit), + known_extrinsics: LruHashSet::new(NonZeroUsize::new(MAX_KNOWN_EXTRINSICS) + .expect("Constant is nonzero")), + known_blocks: LruHashSet::new(NonZeroUsize::new(MAX_KNOWN_BLOCKS) + .expect("Constant is nonzero")), next_request_id: 0, obsolete_requests: HashMap::new(), }; @@ -942,12 +1014,14 @@ impl, H: ExHashT> Protocol { behaviour: &mut self.behaviour, peerset: self.peerset_handle.clone(), }, who.clone(), status.roles, status.best_number); - match self.sync.new_peer(who.clone(), info) { - Ok(None) => (), - Ok(Some(req)) => self.send_message(who.clone(), GenericMessage::BlockRequest(req)), - Err(sync::BadPeer(id, repu)) => { - self.behaviour.disconnect_peer(&id); - self.peerset_handle.report_peer(id, repu) + if info.roles.is_full() { + match self.sync.new_peer(who.clone(), info.best_hash, info.best_number) { + Ok(None) => (), + Ok(Some(req)) => self.send_request(&who, GenericMessage::BlockRequest(req)), + Err(sync::BadPeer(id, repu)) => { + self.behaviour.disconnect_peer(&id); + self.peerset_handle.report_peer(id, repu) + } } } let mut context = ProtocolContext::new(&mut self.context_data, &mut self.behaviour, &self.peerset_handle); @@ -985,7 +1059,8 @@ impl, H: ExHashT> Protocol { self.transaction_pool.import( self.peerset_handle.clone().into(), who.clone(), - NEW_EXTRINSIC_REPUTATION_CHANGE, + GOOD_EXTRINSIC_REPUTATION_CHANGE, + BAD_EXTRINSIC_REPUTATION_CHANGE, t, ); } @@ -1025,7 +1100,12 @@ impl, H: ExHashT> Protocol { .push(who.to_base58()); } trace!(target: "sync", "Sending {} transactions to {}", to_send.len(), who); - self.behaviour.send_packet(who, GenericMessage::Transactions(to_send)) + send_message:: ( + &mut self.behaviour, + &mut self.context_data.stats, + &who, + GenericMessage::Transactions(to_send) + ) } } @@ -1066,7 +1146,7 @@ impl, H: ExHashT> Protocol { trace!(target: "sync", "Announcing block {:?} to {}", hash, who); let inserted = peer.known_blocks.insert(hash); if inserted || force { - let message = GenericMessage::BlockAnnounce(message::BlockAnnounce { + let message: Message = GenericMessage::BlockAnnounce(message::BlockAnnounce { header: header.clone(), state: if peer.info.protocol_version >= 4 { if is_best { @@ -1084,7 +1164,12 @@ impl, H: ExHashT> Protocol { }, }); - self.behaviour.send_packet(who, message) + send_message:: ( + &mut self.behaviour, + &mut self.context_data.stats, + &who, + message, + ) } } } @@ -1102,7 +1187,7 @@ impl, H: ExHashT> Protocol { chain_status: self.specialization.status(), }; - self.send_message(who, GenericMessage::Status(status)) + self.send_message(&who, GenericMessage::Status(status)) } fn on_block_announce(&mut self, who: PeerId, announce: BlockAnnounce) -> CustomMessageOutcome { @@ -1162,7 +1247,7 @@ impl, H: ExHashT> Protocol { match blocks_to_import { Ok(sync::OnBlockData::Import(origin, blocks)) => CustomMessageOutcome::BlockImport(origin, blocks), Ok(sync::OnBlockData::Request(peer, req)) => { - self.send_message(peer, GenericMessage::BlockRequest(req)); + self.send_request(&peer, GenericMessage::BlockRequest(req)); CustomMessageOutcome::None } Err(sync::BadPeer(id, repu)) => { @@ -1226,12 +1311,12 @@ impl, H: ExHashT> Protocol { error ); self.peerset_handle.report_peer(who.clone(), RPC_FAILED_REPUTATION_CHANGE); - Default::default() + StorageProof::empty() } }; self.send_message( - who, + &who, GenericMessage::RemoteCallResponse(message::RemoteCallResponse { id: request.id, proof, @@ -1263,18 +1348,22 @@ impl, H: ExHashT> Protocol { count: usize, results: Vec<(Result>, BlockImportError>, B::Hash)> ) { - let peers = self.context_data.peers.clone(); let results = self.sync.on_blocks_processed( imported, count, results, - |peer_id| peers.get(peer_id).map(|i| i.info.clone()) ); for result in results { match result { Ok((id, req)) => { let msg = GenericMessage::BlockRequest(req); - send_message(&mut self.behaviour, &mut self.context_data.peers, id, msg) + send_request( + &mut self.behaviour, + &mut self.context_data.stats, + &mut self.context_data.peers, + &id, + msg + ) } Err(sync::BadPeer(id, repu)) => { self.behaviour.disconnect_peer(&id); @@ -1343,11 +1432,11 @@ impl, H: ExHashT> Protocol { request.block, error ); - Default::default() + StorageProof::empty() } }; self.send_message( - who, + &who, GenericMessage::RemoteReadResponse(message::RemoteReadResponse { id: request.id, proof, @@ -1386,11 +1475,11 @@ impl, H: ExHashT> Protocol { request.block, error ); - Default::default() + StorageProof::empty() } }; self.send_message( - who, + &who, GenericMessage::RemoteReadResponse(message::RemoteReadResponse { id: request.id, proof, @@ -1426,11 +1515,11 @@ impl, H: ExHashT> Protocol { request.block, error ); - (Default::default(), Default::default()) + (Default::default(), StorageProof::empty()) } }; self.send_message( - who, + &who, GenericMessage::RemoteHeaderResponse(message::RemoteHeaderResponse { id: request.id, header, @@ -1495,12 +1584,12 @@ impl, H: ExHashT> Protocol { max_block: Zero::zero(), proof: vec![], roots: BTreeMap::new(), - roots_proof: vec![], + roots_proof: StorageProof::empty(), } } }; self.send_message( - who, + &who, GenericMessage::RemoteChangesResponse(message::RemoteChangesResponse { id: request.id, max: proof.max_block, @@ -1550,7 +1639,7 @@ impl, H: ExHashT> Protocol { }, }; self.send_message( - who, + &who, GenericMessage::FinalityProofResponse(message::FinalityProofResponse { id: 0, block: request.block, @@ -1587,6 +1676,22 @@ impl, H: ExHashT> Protocol { peerset: self.peerset_handle.clone(), }, peer, response); } + + fn format_stats(&self) -> String { + let mut out = String::new(); + for (id, stats) in &self.context_data.stats { + let _ = writeln!( + &mut out, + "{}: In: {} bytes ({}), Out: {} bytes ({})", + id, + stats.bytes_in, + stats.count_in, + stats.bytes_out, + stats.count_out, + ); + } + out + } } /// Outcome of an incoming custom message. @@ -1598,14 +1703,15 @@ pub enum CustomMessageOutcome { None, } -fn send_message( - behaviour: &mut LegacyProto>, +fn send_request( + behaviour: &mut LegacyProto>, + stats: &mut HashMap<&'static str, PacketStats>, peers: &mut HashMap>, - who: PeerId, + who: &PeerId, mut message: Message, ) { if let GenericMessage::BlockRequest(ref mut r) = message { - if let Some(ref mut peer) = peers.get_mut(&who) { + if let Some(ref mut peer) = peers.get_mut(who) { r.id = peer.next_request_id; peer.next_request_id = peer.next_request_id + 1; if let Some((timestamp, request)) = peer.block_request.take() { @@ -1615,12 +1721,25 @@ fn send_message( peer.block_request = Some((time::Instant::now(), r.clone())); } } - behaviour.send_packet(&who, message); + send_message::(behaviour, stats, who, message) +} + +fn send_message( + behaviour: &mut LegacyProto>, + stats: &mut HashMap<&'static str, PacketStats>, + who: &PeerId, + message: Message, +) { + let encoded = message.encode(); + let mut stats = stats.entry(message.id()).or_default(); + stats.bytes_out += encoded.len() as u64; + stats.count_out += 1; + behaviour.send_packet(who, encoded); } impl, H: ExHashT> NetworkBehaviour for Protocol { - type ProtocolsHandler = > as NetworkBehaviour>::ProtocolsHandler; + type ProtocolsHandler = > as NetworkBehaviour>::ProtocolsHandler; type OutEvent = CustomMessageOutcome; fn new_handler(&mut self) -> Self::ProtocolsHandler { @@ -1665,13 +1784,30 @@ Protocol { } for (id, r) in self.sync.block_requests() { - send_message(&mut self.behaviour, &mut self.context_data.peers, id, GenericMessage::BlockRequest(r)) + send_request( + &mut self.behaviour, + &mut self.context_data.stats, + &mut self.context_data.peers, + &id, + GenericMessage::BlockRequest(r) + ) } for (id, r) in self.sync.justification_requests() { - send_message(&mut self.behaviour, &mut self.context_data.peers, id, GenericMessage::BlockRequest(r)) + send_request( + &mut self.behaviour, + &mut self.context_data.stats, + &mut self.context_data.peers, + &id, + GenericMessage::BlockRequest(r) + ) } for (id, r) in self.sync.finality_proof_requests() { - send_message(&mut self.behaviour, &mut self.context_data.peers, id, GenericMessage::FinalityProofRequest(r)) + send_request( + &mut self.behaviour, + &mut self.context_data.stats, + &mut self.context_data.peers, + &id, + GenericMessage::FinalityProofRequest(r)) } let event = match self.behaviour.poll(params) { @@ -1705,8 +1841,9 @@ Protocol { LegacyProtoOut::Clogged { peer_id, messages } => { debug!(target: "sync", "{} clogging messages:", messages.len()); for msg in messages.into_iter().take(5) { - debug!(target: "sync", "{:?}", msg); - self.on_clogged_peer(peer_id.clone(), Some(msg)); + let message: Option> = Decode::decode(&mut &msg[..]).ok(); + debug!(target: "sync", "{:?}", message); + self.on_clogged_peer(peer_id.clone(), message); } CustomMessageOutcome::None } @@ -1754,3 +1891,9 @@ impl, H: ExHashT> DiscoveryNetBehaviour f self.behaviour.add_discovered_nodes(peer_ids) } } + +impl, H: ExHashT> Drop for Protocol { + fn drop(&mut self) { + debug!(target: "sync", "Network stats:\n{}", self.format_stats()); + } +} diff --git a/core/network/src/protocol/consensus_gossip.rs b/client/network/src/protocol/consensus_gossip.rs similarity index 74% rename from core/network/src/protocol/consensus_gossip.rs rename to client/network/src/protocol/consensus_gossip.rs index f3d4e536a788d..0fd20092c1d84 100644 --- a/core/network/src/protocol/consensus_gossip.rs +++ b/client/network/src/protocol/consensus_gossip.rs @@ -73,6 +73,7 @@ const UNREGISTERED_TOPIC_REPUTATION_CHANGE: i32 = -(1 << 10); struct PeerConsensus { known_messages: HashSet, + filtered_messages: HashMap, roles: Roles, } @@ -104,9 +105,14 @@ pub enum MessageRecipient { /// The reason for sending out the message. #[derive(Eq, PartialEq, Copy, Clone)] +#[cfg_attr(test, derive(Debug))] pub enum MessageIntent { - /// Requested broadcast - Broadcast, + /// Requested broadcast. + Broadcast { + /// How many times this message was previously filtered by the gossip + /// validator when trying to propagate to a given peer. + previous_attempts: usize + }, /// Requested broadcast to all peers. ForcedBroadcast, /// Periodic rebroadcast of all messages to all peers. @@ -123,6 +129,12 @@ pub enum ValidationResult { Discard, } +impl MessageIntent { + fn broadcast() -> MessageIntent { + MessageIntent::Broadcast { previous_attempts: 0 } + } +} + /// Validation context. Allows reacting to incoming messages by sending out further messages. pub trait ValidatorContext { /// Broadcast all messages with given topic to peers that do not have it yet. @@ -159,10 +171,10 @@ impl<'g, 'p, B: BlockT> ValidatorContext for NetworkContext<'g, 'p, B> { /// Send addressed message to a peer. fn send_message(&mut self, who: &PeerId, message: Vec) { - self.protocol.send_consensus(who.clone(), ConsensusMessage { + self.protocol.send_consensus(who.clone(), vec![ConsensusMessage { engine_id: self.engine_id, data: message, - }); + }]); } /// Send all messages with given topic to a peer. @@ -178,7 +190,7 @@ fn propagate<'a, B: BlockT, I>( peers: &mut HashMap>, validators: &HashMap>>, ) - where I: IntoIterator, // (msg_hash, topic, message) + where I: Clone + IntoIterator, // (msg_hash, topic, message) { let mut check_fns = HashMap::new(); let mut message_allowed = move |who: &PeerId, intent: MessageIntent, topic: &B::Hash, message: &ConsensusMessage| { @@ -194,14 +206,20 @@ fn propagate<'a, B: BlockT, I>( (check_fn)(who, intent, topic, &message.data) }; - for (message_hash, topic, message) in messages { - for (id, ref mut peer) in peers.iter_mut() { + for (id, ref mut peer) in peers.iter_mut() { + let mut batch = Vec::new(); + for (message_hash, topic, message) in messages.clone() { + let previous_attempts = peer.filtered_messages + .get(&message_hash) + .cloned() + .unwrap_or(0); + let intent = match intent { - MessageIntent::Broadcast => + MessageIntent::Broadcast { .. } => if peer.known_messages.contains(&message_hash) { - continue + continue; } else { - MessageIntent::Broadcast + MessageIntent::Broadcast { previous_attempts } }, MessageIntent::PeriodicRebroadcast => if peer.known_messages.contains(&message_hash) { @@ -209,18 +227,28 @@ fn propagate<'a, B: BlockT, I>( } else { // peer doesn't know message, so the logic should treat it as an // initial broadcast. - MessageIntent::Broadcast + MessageIntent::Broadcast { previous_attempts } }, other => other, }; if !message_allowed(id, intent, &topic, &message) { - continue + let count = peer.filtered_messages + .entry(message_hash.clone()) + .or_insert(0); + + *count += 1; + + continue; } + + peer.filtered_messages.remove(message_hash); peer.known_messages.insert(message_hash.clone()); + trace!(target: "gossip", "Propagating to {}: {:?}", id, message); - protocol.send_consensus(id.clone(), message.clone()); + batch.push(message.clone()) } + protocol.send_consensus(id.clone(), batch); } } @@ -310,6 +338,7 @@ impl ConsensusGossip { trace!(target:"gossip", "Registering {:?} {}", roles, who); self.peers.insert(who.clone(), PeerConsensus { known_messages: HashSet::new(), + filtered_messages: HashMap::new(), roles, }); for (engine_id, v) in self.validators.clone() { @@ -379,7 +408,7 @@ impl ConsensusGossip { .filter_map(|entry| if entry.topic == topic { Some((&entry.message_hash, &entry.topic, &entry.message)) } else { None } ); - let intent = if force { MessageIntent::ForcedBroadcast } else { MessageIntent::Broadcast }; + let intent = if force { MessageIntent::ForcedBroadcast } else { MessageIntent::broadcast() }; propagate(protocol, messages, intent, &mut self.peers, &self.validators); } @@ -419,6 +448,7 @@ impl ConsensusGossip { for (_, ref mut peer) in self.peers.iter_mut() { peer.known_messages.retain(|h| known_messages.contains_key(h)); + peer.filtered_messages.retain(|h, _| known_messages.contains_key(h)); } } @@ -450,65 +480,68 @@ impl ConsensusGossip { &mut self, protocol: &mut dyn Context, who: PeerId, - message: ConsensusMessage, + messages: Vec, ) { - let message_hash = HashFor::::hash(&message.data[..]); - - if self.known_messages.contains_key(&message_hash) { - trace!(target:"gossip", "Ignored already known message from {}", who); - protocol.report_peer(who.clone(), DUPLICATE_GOSSIP_REPUTATION_CHANGE); - return; - } + trace!(target:"gossip", "Received {} messages from peer {}", messages.len(), who); + for message in messages { + let message_hash = HashFor::::hash(&message.data[..]); + + if self.known_messages.contains_key(&message_hash) { + trace!(target:"gossip", "Ignored already known message from {}", who); + protocol.report_peer(who.clone(), DUPLICATE_GOSSIP_REPUTATION_CHANGE); + continue; + } - let engine_id = message.engine_id; - // validate the message - let validation = self.validators.get(&engine_id) - .cloned() - .map(|v| { - let mut context = NetworkContext { gossip: self, protocol, engine_id }; - v.validate(&mut context, &who, &message.data) - }); + let engine_id = message.engine_id; + // validate the message + let validation = self.validators.get(&engine_id) + .cloned() + .map(|v| { + let mut context = NetworkContext { gossip: self, protocol, engine_id }; + v.validate(&mut context, &who, &message.data) + }); - let validation_result = match validation { - Some(ValidationResult::ProcessAndKeep(topic)) => Some((topic, true)), - Some(ValidationResult::ProcessAndDiscard(topic)) => Some((topic, false)), - Some(ValidationResult::Discard) => None, - None => { - trace!(target:"gossip", "Unknown message engine id {:?} from {}", engine_id, who); - protocol.report_peer(who.clone(), UNKNOWN_GOSSIP_REPUTATION_CHANGE); - protocol.disconnect_peer(who); - return; - } - }; + let validation_result = match validation { + Some(ValidationResult::ProcessAndKeep(topic)) => Some((topic, true)), + Some(ValidationResult::ProcessAndDiscard(topic)) => Some((topic, false)), + Some(ValidationResult::Discard) => None, + None => { + trace!(target:"gossip", "Unknown message engine id {:?} from {}", engine_id, who); + protocol.report_peer(who.clone(), UNKNOWN_GOSSIP_REPUTATION_CHANGE); + protocol.disconnect_peer(who.clone()); + continue; + } + }; - if let Some((topic, keep)) = validation_result { - protocol.report_peer(who.clone(), GOSSIP_SUCCESS_REPUTATION_CHANGE); - if let Some(ref mut peer) = self.peers.get_mut(&who) { - peer.known_messages.insert(message_hash); - if let Entry::Occupied(mut entry) = self.live_message_sinks.entry((engine_id, topic)) { - debug!(target: "gossip", "Pushing consensus message to sinks for {}.", topic); - entry.get_mut().retain(|sink| { - if let Err(e) = sink.unbounded_send(TopicNotification { - message: message.data.clone(), - sender: Some(who.clone()) - }) { - trace!(target: "gossip", "Error broadcasting message notification: {:?}", e); + if let Some((topic, keep)) = validation_result { + protocol.report_peer(who.clone(), GOSSIP_SUCCESS_REPUTATION_CHANGE); + if let Some(ref mut peer) = self.peers.get_mut(&who) { + peer.known_messages.insert(message_hash); + if let Entry::Occupied(mut entry) = self.live_message_sinks.entry((engine_id, topic)) { + debug!(target: "gossip", "Pushing consensus message to sinks for {}.", topic); + entry.get_mut().retain(|sink| { + if let Err(e) = sink.unbounded_send(TopicNotification { + message: message.data.clone(), + sender: Some(who.clone()) + }) { + trace!(target: "gossip", "Error broadcasting message notification: {:?}", e); + } + !sink.is_closed() + }); + if entry.get().is_empty() { + entry.remove_entry(); } - !sink.is_closed() - }); - if entry.get().is_empty() { - entry.remove_entry(); } - } - if keep { - self.register_message_hashed(message_hash, topic, message, Some(who.clone())); + if keep { + self.register_message_hashed(message_hash, topic, message, Some(who.clone())); + } + } else { + trace!(target:"gossip", "Ignored statement from unregistered peer {}", who); + protocol.report_peer(who.clone(), UNREGISTERED_TOPIC_REPUTATION_CHANGE); } } else { - trace!(target:"gossip", "Ignored statement from unregistered peer {}", who); - protocol.report_peer(who.clone(), UNREGISTERED_TOPIC_REPUTATION_CHANGE); + trace!(target:"gossip", "Handled valid one hop message from peer {}", who); } - } else { - trace!(target:"gossip", "Handled valid one hop message from peer {}", who); } } @@ -527,23 +560,44 @@ impl ConsensusGossip { Some(validator) => validator.message_allowed(), }; - let intent = if force { MessageIntent::ForcedBroadcast } else { MessageIntent::Broadcast }; - if let Some(ref mut peer) = self.peers.get_mut(who) { + let mut batch = Vec::new(); for entry in self.messages.iter().filter(|m| m.topic == topic && m.message.engine_id == engine_id) { + let intent = if force { + MessageIntent::ForcedBroadcast + } else { + let previous_attempts = peer.filtered_messages + .get(&entry.message_hash) + .cloned() + .unwrap_or(0); + + MessageIntent::Broadcast { previous_attempts } + }; + if !force && peer.known_messages.contains(&entry.message_hash) { - continue + continue; } + if !message_allowed(who, intent, &entry.topic, &entry.message.data) { - continue + let count = peer.filtered_messages + .entry(entry.message_hash) + .or_insert(0); + + *count += 1; + + continue; } + + peer.filtered_messages.remove(&entry.message_hash); peer.known_messages.insert(entry.message_hash.clone()); + trace!(target: "gossip", "Sending topic message to {}: {:?}", who, entry.message); - protocol.send_consensus(who.clone(), ConsensusMessage { + batch.push(ConsensusMessage { engine_id: engine_id.clone(), data: entry.message.data.clone(), }); } + protocol.send_consensus(who.clone(), batch); } } @@ -557,7 +611,7 @@ impl ConsensusGossip { ) { let message_hash = HashFor::::hash(&message.data); self.register_message_hashed(message_hash, topic, message.clone(), None); - let intent = if force { MessageIntent::ForcedBroadcast } else { MessageIntent::Broadcast }; + let intent = if force { MessageIntent::ForcedBroadcast } else { MessageIntent::broadcast() }; propagate(protocol, iter::once((&message_hash, &topic, &message)), intent, &mut self.peers, &self.validators); } @@ -578,8 +632,9 @@ impl ConsensusGossip { trace!(target: "gossip", "Sending direct to {}: {:?}", who, message); + peer.filtered_messages.remove(&message_hash); peer.known_messages.insert(message_hash); - protocol.send_consensus(who.clone(), message.clone()); + protocol.send_consensus(who.clone(), vec![message.clone()]); } } @@ -607,6 +662,8 @@ impl Validator for DiscardAll { #[cfg(test)] mod tests { + use std::sync::{Arc, atomic::{AtomicBool, Ordering}}; + use parking_lot::Mutex; use sr_primitives::testing::{H256, Block as RawBlock, ExtrinsicWrapper}; use futures03::executor::block_on_stream; @@ -657,7 +714,7 @@ mod tests { } fn message_expired<'a>(&'a self) -> Box bool + 'a> { - Box::new(move |_topic, data| data[0] != 1 ) + Box::new(move |_topic, data| data[0] != 1) } } @@ -755,4 +812,109 @@ mod tests { let _ = consensus.live_message_sinks.remove(&([0, 0, 0, 0], topic)); assert_eq!(stream.next(), None); } + + #[test] + fn keeps_track_of_broadcast_attempts() { + struct DummyNetworkContext; + impl Context for DummyNetworkContext { + fn report_peer(&mut self, _who: PeerId, _reputation: i32) {} + fn disconnect_peer(&mut self, _who: PeerId) {} + fn send_consensus(&mut self, _who: PeerId, _consensus: Vec) {} + fn send_chain_specific(&mut self, _who: PeerId, _message: Vec) {} + } + + // A mock gossip validator that never expires any message, allows + // setting whether messages should be allowed and keeps track of any + // messages passed to `message_allowed`. + struct MockValidator { + allow: AtomicBool, + messages: Arc, MessageIntent)>>>, + } + + impl MockValidator { + fn new() -> MockValidator { + MockValidator { + allow: AtomicBool::new(false), + messages: Arc::new(Mutex::new(Vec::new())), + } + } + } + + impl Validator for MockValidator { + fn validate( + &self, + _context: &mut dyn ValidatorContext, + _sender: &PeerId, + _data: &[u8], + ) -> ValidationResult { + ValidationResult::ProcessAndKeep(H256::default()) + } + + fn message_expired<'a>(&'a self) -> Box bool + 'a> { + Box::new(move |_topic, _data| false) + } + + fn message_allowed<'a>(&'a self) -> Box bool + 'a> { + let messages = self.messages.clone(); + Box::new(move |_, intent, _, data| { + messages.lock().push((data.to_vec(), intent)); + self.allow.load(Ordering::SeqCst) + }) + } + } + + // we setup an instance of the mock gossip validator, add a new peer to + // it and register a message. + let mut consensus = ConsensusGossip::::new(); + let validator = Arc::new(MockValidator::new()); + consensus.register_validator_internal([0, 0, 0, 0], validator.clone()); + consensus.new_peer( + &mut DummyNetworkContext, + PeerId::random(), + Roles::AUTHORITY, + ); + + let data = vec![1, 2, 3]; + let msg = ConsensusMessage { data: data.clone(), engine_id: [0, 0, 0, 0] }; + consensus.register_message(H256::default(), msg); + + // tick the gossip handler and make sure it triggers a message rebroadcast + let mut tick = || { + consensus.next_broadcast = std::time::Instant::now(); + consensus.tick(&mut DummyNetworkContext); + }; + + // by default we won't allow the message we registered, so everytime we + // tick the gossip handler, the message intent should be kept as + // `Broadcast` but the previous attempts should be incremented. + tick(); + assert_eq!( + validator.messages.lock().pop().unwrap(), + (data.clone(), MessageIntent::Broadcast { previous_attempts: 0 }), + ); + + tick(); + assert_eq!( + validator.messages.lock().pop().unwrap(), + (data.clone(), MessageIntent::Broadcast { previous_attempts: 1 }), + ); + + // we set the validator to allow the message to go through + validator.allow.store(true, Ordering::SeqCst); + + // we still get the same message intent but it should be delivered now + tick(); + assert_eq!( + validator.messages.lock().pop().unwrap(), + (data.clone(), MessageIntent::Broadcast { previous_attempts: 2 }), + ); + + // ticking the gossip handler again the message intent should change to + // `PeriodicRebroadcast` since it was sent. + tick(); + assert_eq!( + validator.messages.lock().pop().unwrap(), + (data.clone(), MessageIntent::PeriodicRebroadcast), + ); + } } diff --git a/core/network/src/protocol/event.rs b/client/network/src/protocol/event.rs similarity index 100% rename from core/network/src/protocol/event.rs rename to client/network/src/protocol/event.rs diff --git a/core/network/src/protocol/light_dispatch.rs b/client/network/src/protocol/light_dispatch.rs similarity index 98% rename from core/network/src/protocol/light_dispatch.rs rename to client/network/src/protocol/light_dispatch.rs index 33ff23c909d10..280495c0c41b6 100644 --- a/core/network/src/protocol/light_dispatch.rs +++ b/client/network/src/protocol/light_dispatch.rs @@ -25,10 +25,10 @@ use std::time::{Instant, Duration}; use log::{trace, info}; use futures::sync::oneshot::{Sender as OneShotSender}; use linked_hash_map::{Entry, LinkedHashMap}; -use client::error::Error as ClientError; -use client::light::fetcher::{FetchChecker, RemoteHeaderRequest, +use client_api::error::Error as ClientError; +use client_api::{FetchChecker, RemoteHeaderRequest, RemoteCallRequest, RemoteReadRequest, RemoteChangesRequest, ChangesProof, - RemoteReadChildRequest, RemoteBodyRequest}; + RemoteReadChildRequest, RemoteBodyRequest, StorageProof}; use crate::message::{self, BlockAttributes, Direction, FromBlock, RequestId}; use libp2p::PeerId; use crate::config::Roles; @@ -174,7 +174,7 @@ impl FetchChecker for AlwaysBadChecker { &self, _request: &RemoteHeaderRequest, _remote_header: Option, - _remote_proof: Vec> + _remote_proof: StorageProof, ) -> Result { Err(ClientError::Msg("AlwaysBadChecker".into())) } @@ -182,7 +182,7 @@ impl FetchChecker for AlwaysBadChecker { fn check_read_proof( &self, _request: &RemoteReadRequest, - _remote_proof: Vec> + _remote_proof: StorageProof, ) -> Result,Option>>, ClientError> { Err(ClientError::Msg("AlwaysBadChecker".into())) } @@ -190,7 +190,7 @@ impl FetchChecker for AlwaysBadChecker { fn check_read_child_proof( &self, _request: &RemoteReadChildRequest, - _remote_proof: Vec> + _remote_proof: StorageProof, ) -> Result, Option>>, ClientError> { Err(ClientError::Msg("AlwaysBadChecker".into())) } @@ -198,7 +198,7 @@ impl FetchChecker for AlwaysBadChecker { fn check_execution_proof( &self, _request: &RemoteCallRequest, - _remote_proof: Vec> + _remote_proof: StorageProof, ) -> Result, ClientError> { Err(ClientError::Msg("AlwaysBadChecker".into())) } @@ -677,14 +677,14 @@ pub mod tests { use std::time::Instant; use futures::{Future, sync::oneshot}; use sr_primitives::traits::{Block as BlockT, NumberFor, Header as HeaderT}; - use client::{error::{Error as ClientError, Result as ClientResult}}; - use client::light::fetcher::{FetchChecker, RemoteHeaderRequest, + use client_api::{error::{Error as ClientError, Result as ClientResult}}; + use client_api::{FetchChecker, RemoteHeaderRequest, ChangesProof, RemoteCallRequest, RemoteReadRequest, RemoteReadChildRequest, RemoteChangesRequest, RemoteBodyRequest}; use crate::config::Roles; use crate::message::{self, BlockAttributes, Direction, FromBlock, RequestId}; use libp2p::PeerId; - use super::{REQUEST_TIMEOUT, LightDispatch, LightDispatchNetwork, RequestData}; + use super::{REQUEST_TIMEOUT, LightDispatch, LightDispatchNetwork, RequestData, StorageProof}; use test_client::runtime::{changes_trie_config, Block, Extrinsic, Header}; struct DummyFetchChecker { ok: bool } @@ -694,7 +694,7 @@ pub mod tests { &self, _request: &RemoteHeaderRequest
, header: Option
, - _remote_proof: Vec> + _remote_proof: StorageProof, ) -> ClientResult
{ match self.ok { true if header.is_some() => Ok(header.unwrap()), @@ -705,7 +705,7 @@ pub mod tests { fn check_read_proof( &self, request: &RemoteReadRequest
, - _: Vec>, + _: StorageProof, ) -> ClientResult, Option>>> { match self.ok { true => Ok(request.keys @@ -721,7 +721,7 @@ pub mod tests { fn check_read_child_proof( &self, request: &RemoteReadChildRequest
, - _: Vec> + _: StorageProof, ) -> ClientResult, Option>>> { match self.ok { true => Ok(request.keys @@ -734,7 +734,7 @@ pub mod tests { } } - fn check_execution_proof(&self, _: &RemoteCallRequest
, _: Vec>) -> ClientResult> { + fn check_execution_proof(&self, _: &RemoteCallRequest
, _: StorageProof) -> ClientResult> { match self.ok { true => Ok(vec![42]), false => Err(ClientError::Backend("Test error".into())), @@ -780,7 +780,7 @@ pub mod tests { ) { light_dispatch.on_remote_call_response(network_interface, peer, message::RemoteCallResponse { id: id, - proof: vec![vec![2]], + proof: StorageProof::empty(), }); } @@ -943,7 +943,7 @@ pub mod tests { light_dispatch.on_remote_read_response(&mut network_interface, peer0.clone(), message::RemoteReadResponse { id: 0, - proof: vec![vec![2]], + proof: StorageProof::empty(), }); assert_disconnected_peer(&network_interface); assert_eq!(light_dispatch.pending_requests.len(), 1); @@ -1013,7 +1013,7 @@ pub mod tests { light_dispatch.on_remote_read_response(&mut network_interface, peer0.clone(), message::RemoteReadResponse { id: 0, - proof: vec![vec![2]], + proof: StorageProof::empty(), }); assert_eq!(response.wait().unwrap().unwrap().remove(b":key".as_ref()).unwrap(), Some(vec![42])); } @@ -1037,7 +1037,7 @@ pub mod tests { light_dispatch.on_remote_read_response(&mut network_interface, peer0.clone(), message::RemoteReadResponse { id: 0, - proof: vec![vec![2]], + proof: StorageProof::empty(), }); assert_eq!(response.wait().unwrap().unwrap().remove(b":key".as_ref()).unwrap(), Some(vec![42])); } @@ -1065,7 +1065,7 @@ pub mod tests { extrinsics_root: Default::default(), digest: Default::default(), }), - proof: vec![vec![2]], + proof: StorageProof::empty(), }); assert_eq!( response.wait().unwrap().unwrap().hash(), @@ -1097,7 +1097,7 @@ pub mod tests { max: 1000, proof: vec![vec![2]], roots: vec![], - roots_proof: vec![], + roots_proof: StorageProof::empty(), }); assert_eq!(response.wait().unwrap().unwrap(), vec![(100, 2)]); } @@ -1145,7 +1145,7 @@ pub mod tests { light_dispatch.on_remote_header_response(&mut network_interface, peer1.clone(), message::RemoteHeaderResponse { id: 0, header: Some(dummy_header()), - proof: vec![], + proof: StorageProof::empty(), }); assert!(!light_dispatch.idle_peers.iter().any(|_| true)); diff --git a/core/network/src/protocol/message.rs b/client/network/src/protocol/message.rs similarity index 89% rename from core/network/src/protocol/message.rs rename to client/network/src/protocol/message.rs index 6560ed0f13a52..847c03f680d7c 100644 --- a/core/network/src/protocol/message.rs +++ b/client/network/src/protocol/message.rs @@ -26,6 +26,7 @@ pub use self::generic::{ FinalityProofRequest, FinalityProofResponse, FromBlock, RemoteReadChildRequest, }; +use client_api::StorageProof; /// A unique ID of a request. pub type RequestId = u64; @@ -103,7 +104,7 @@ impl Decode for BlockAttributes { pub enum Direction { /// Enumerate in ascending order (from child to parent). Ascending = 0, - /// Enumerate in descendfing order (from parent to canonical child). + /// Enumerate in descending order (from parent to canonical child). Descending = 1, } @@ -122,7 +123,7 @@ pub struct RemoteCallResponse { /// Id of a request this response was made for. pub id: RequestId, /// Execution proof. - pub proof: Vec>, + pub proof: StorageProof, } #[derive(Debug, PartialEq, Eq, Clone, Encode, Decode)] @@ -131,7 +132,7 @@ pub struct RemoteReadResponse { /// Id of a request this response was made for. pub id: RequestId, /// Read proof. - pub proof: Vec>, + pub proof: StorageProof, } /// Generic types. @@ -142,7 +143,7 @@ pub mod generic { use super::{ RemoteReadResponse, Transactions, Direction, RequestId, BlockAttributes, RemoteCallResponse, ConsensusEngineId, - BlockState, + BlockState, StorageProof, }; /// Consensus is mostly opaque to us #[derive(Debug, PartialEq, Eq, Clone, Encode, Decode)] @@ -216,11 +217,40 @@ pub mod generic { FinalityProofRequest(FinalityProofRequest), /// Finality proof reponse. FinalityProofResponse(FinalityProofResponse), + /// Batch of consensus protocol messages. + ConsensusBatch(Vec), /// Chain-specific message. #[codec(index = "255")] ChainSpecific(Vec), } + impl Message { + /// Message id useful for logging. + pub fn id(&self) -> &'static str { + match self { + Message::Status(_) => "Status", + Message::BlockRequest(_) => "BlockRequest", + Message::BlockResponse(_) => "BlockResponse", + Message::BlockAnnounce(_) => "BlockAnnounce", + Message::Transactions(_) => "Transactions", + Message::Consensus(_) => "Consensus", + Message::RemoteCallRequest(_) => "RemoteCallRequest", + Message::RemoteCallResponse(_) => "RemoteCallResponse", + Message::RemoteReadRequest(_) => "RemoteReadRequest", + Message::RemoteReadResponse(_) => "RemoteReadResponse", + Message::RemoteHeaderRequest(_) => "RemoteHeaderRequest", + Message::RemoteHeaderResponse(_) => "RemoteHeaderResponse", + Message::RemoteChangesRequest(_) => "RemoteChangesRequest", + Message::RemoteChangesResponse(_) => "RemoteChangesResponse", + Message::RemoteReadChildRequest(_) => "RemoteReadChildRequest", + Message::FinalityProofRequest(_) => "FinalityProofRequest", + Message::FinalityProofResponse(_) => "FinalityProofResponse", + Message::ConsensusBatch(_) => "ConsensusBatch", + Message::ChainSpecific(_) => "ChainSpecific", + } + } + } + /// Status sent on connection. #[derive(Debug, PartialEq, Eq, Clone, Encode, Decode)] pub struct Status { @@ -359,7 +389,7 @@ pub mod generic { /// Header. None if proof generation has failed (e.g. header is unknown). pub header: Option
, /// Header proof. - pub proof: Vec>, + pub proof: StorageProof, } #[derive(Debug, PartialEq, Eq, Clone, Encode, Decode)] @@ -395,7 +425,7 @@ pub mod generic { /// Changes tries roots missing on the requester' node. pub roots: Vec<(N, H)>, /// Missing changes tries roots proof. - pub roots_proof: Vec>, + pub roots_proof: StorageProof, } #[derive(Debug, PartialEq, Eq, Clone, Encode, Decode)] diff --git a/core/network/src/protocol/specialization.rs b/client/network/src/protocol/specialization.rs similarity index 97% rename from core/network/src/protocol/specialization.rs rename to client/network/src/protocol/specialization.rs index e2c444ea889d6..1a15c47c87d3a 100644 --- a/core/network/src/protocol/specialization.rs +++ b/client/network/src/protocol/specialization.rs @@ -42,7 +42,8 @@ pub trait NetworkSpecialization: Send + Sync + 'static { ); /// Called when a network-specific event arrives. - fn on_event(&mut self, event: Event); + #[deprecated(note = "This method is never called; please use `with_dht_event_tx` when building the service")] + fn on_event(&mut self, _event: Event) {} /// Called on abort. #[deprecated(note = "This method is never called; aborting corresponds to dropping the object")] diff --git a/core/network/src/protocol/sync.rs b/client/network/src/protocol/sync.rs similarity index 88% rename from core/network/src/protocol/sync.rs rename to client/network/src/protocol/sync.rs index 4f08c942def1b..6f1255cd8ab9f 100644 --- a/core/network/src/protocol/sync.rs +++ b/client/network/src/protocol/sync.rs @@ -28,7 +28,7 @@ //! use blocks::BlockCollection; -use client::{ClientInfo, error::Error as ClientError}; +use client_api::{ClientInfo, error::Error as ClientError}; use consensus::{BlockOrigin, BlockStatus, block_validation::{BlockAnnounceValidator, Validation}, import_queue::{IncomingBlock, BlockImportResult, BlockImportError} @@ -37,7 +37,6 @@ use crate::{ config::{Roles, BoxFinalityProofRequestBuilder}, message::{self, generic::FinalityProofRequest, BlockAnnounce, BlockAttributes, BlockRequest, BlockResponse, FinalityProofResponse}, - protocol }; use either::Either; use extra_requests::ExtraRequests; @@ -59,6 +58,9 @@ const MAX_BLOCKS_TO_REQUEST: usize = 128; /// Maximum blocks to store in the import queue. const MAX_IMPORTING_BLOCKS: usize = 2048; +/// Maximum blocks to download ahead of any gap. +const MAX_DOWNLOAD_AHEAD: u32 = 2048; + /// We use a heuristic that with a high likelihood, by the time /// `MAJOR_SYNC_BLOCKS` have been imported we'll be on the same /// chain as (or at least closer to) the peer so we want to delay @@ -118,8 +120,9 @@ pub struct ChainSync { /// A set of hashes of blocks that are being downloaded or have been /// downloaded and are queued for import. queue_blocks: HashSet, - /// The best block number that we are currently importing. - best_importing_number: NumberFor, + /// The best block number that was successfully imported into the chain. + /// This can not decrease. + best_imported_number: NumberFor, /// Finality proof handler. request_builder: Option>, /// Fork sync targets. @@ -127,7 +130,9 @@ pub struct ChainSync { /// A flag that caches idle state with no pending requests. is_idle: bool, /// A type to check incoming block announcements. - block_announce_validator: Box + Send> + block_announce_validator: Box + Send>, + /// Maximum number of peers to ask the same blocks in parallel. + max_parallel_downloads: u32, } /// All the data we have about a Peer that we are trying to sync with @@ -282,7 +287,8 @@ impl ChainSync { client: Arc>, info: &ClientInfo, request_builder: Option>, - block_announce_validator: Box + Send> + block_announce_validator: Box + Send>, + max_parallel_downloads: u32, ) -> Self { let mut required_block_attributes = BlockAttributes::HEADER | BlockAttributes::JUSTIFICATION; @@ -296,16 +302,17 @@ impl ChainSync { blocks: BlockCollection::new(), best_queued_hash: info.chain.best_hash, best_queued_number: info.chain.best_number, + best_imported_number: info.chain.best_number, extra_finality_proofs: ExtraRequests::new(), extra_justifications: ExtraRequests::new(), role, required_block_attributes, queue_blocks: Default::default(), - best_importing_number: Zero::zero(), request_builder, fork_targets: Default::default(), is_idle: false, block_announce_validator, + max_parallel_downloads, } } @@ -343,23 +350,22 @@ impl ChainSync { /// Handle a new connected peer. /// /// Call this method whenever we connect to a new peer. - pub fn new_peer(&mut self, who: PeerId, info: protocol::PeerInfo) -> Result>, BadPeer> { + pub fn new_peer(&mut self, who: PeerId, best_hash: B::Hash, best_number: NumberFor) + -> Result>, BadPeer> + { // There is nothing sync can get from the node that has no blockchain data. - if !info.roles.is_full() { - return Ok(None) - } - match self.block_status(&info.best_hash) { + match self.block_status(&best_hash) { Err(e) => { debug!(target:"sync", "Error reading blockchain: {:?}", e); Err(BadPeer(who, BLOCKCHAIN_STATUS_READ_ERROR_REPUTATION_CHANGE)) } Ok(BlockStatus::KnownBad) => { - info!("New peer with known bad best block {} ({}).", info.best_hash, info.best_number); + info!("New peer with known bad best block {} ({}).", best_hash, best_number); Err(BadPeer(who, i32::min_value())) } Ok(BlockStatus::Unknown) => { - if info.best_number.is_zero() { - info!("New peer with unknown genesis hash {} ({}).", info.best_hash, info.best_number); + if best_number.is_zero() { + info!("New peer with unknown genesis hash {} ({}).", best_hash, best_number); return Err(BadPeer(who, i32::min_value())) } // If there are more than `MAJOR_SYNC_BLOCKS` in the import queue then we have @@ -374,8 +380,8 @@ impl ChainSync { ); self.peers.insert(who, PeerSync { common_number: self.best_queued_number, - best_hash: info.best_hash, - best_number: info.best_number, + best_hash, + best_number, state: PeerSyncState::Available, recently_announced: Default::default() }); @@ -384,29 +390,30 @@ impl ChainSync { // If we are at genesis, just start downloading. if self.best_queued_number.is_zero() { - debug!(target:"sync", "New peer with best hash {} ({}).", info.best_hash, info.best_number); + debug!(target:"sync", "New peer with best hash {} ({}).", best_hash, best_number); self.peers.insert(who.clone(), PeerSync { common_number: Zero::zero(), - best_hash: info.best_hash, - best_number: info.best_number, + best_hash, + best_number, state: PeerSyncState::Available, recently_announced: Default::default(), }); - return Ok(self.select_new_blocks(who).map(|(_, req)| req)) + self.is_idle = false; + return Ok(None) } - let common_best = std::cmp::min(self.best_queued_number, info.best_number); + let common_best = std::cmp::min(self.best_queued_number, best_number); debug!(target:"sync", "New peer with unknown best hash {} ({}), searching for common ancestor.", - info.best_hash, - info.best_number + best_hash, + best_number ); self.peers.insert(who, PeerSync { common_number: Zero::zero(), - best_hash: info.best_hash, - best_number: info.best_number, + best_hash, + best_number, state: PeerSyncState::AncestorSearch( common_best, AncestorSearchState::ExponentialBackoff(One::one()) @@ -418,11 +425,11 @@ impl ChainSync { Ok(Some(ancestry_request::(common_best))) } Ok(BlockStatus::Queued) | Ok(BlockStatus::InChainWithState) | Ok(BlockStatus::InChainPruned) => { - debug!(target:"sync", "New peer with known best hash {} ({}).", info.best_hash, info.best_number); + debug!(target:"sync", "New peer with known best hash {} ({}).", best_hash, best_number); self.peers.insert(who.clone(), PeerSync { - common_number: info.best_number, - best_hash: info.best_hash, - best_number: info.best_number, + common_number: best_number, + best_hash, + best_number, state: PeerSyncState::Available, recently_announced: Default::default(), }); @@ -479,13 +486,7 @@ impl ChainSync { return; } - let block_status = self.client.block_status(&BlockId::Number(number - One::one())) - .unwrap_or(BlockStatus::Unknown); - if block_status == BlockStatus::InChainPruned { - trace!(target: "sync", "Refusing to sync ancient block {:?}", hash); - return; - } - + trace!(target: "sync", "Downloading requested old fork {:?}", hash); self.is_idle = false; for peer_id in &peers { if let Some(peer) = self.peers.get_mut(peer_id) { @@ -567,14 +568,16 @@ impl ChainSync { trace!(target: "sync", "Too many blocks in the queue."); return Either::Left(std::iter::empty()) } + let major_sync = self.status().state == SyncState::Downloading; let blocks = &mut self.blocks; let attrs = &self.required_block_attributes; - let fork_targets = &self.fork_targets; + let fork_targets = &mut self.fork_targets; let mut have_requests = false; let last_finalized = self.client.info().chain.finalized_number; let best_queued = self.best_queued_number; let client = &self.client; let queue = &self.queue_blocks; + let max_parallel = if major_sync { 1 } else { self.max_parallel_downloads }; let iter = self.peers.iter_mut().filter_map(move |(id, peer)| { if !peer.state.is_available() { trace!(target: "sync", "Peer {} is busy", id); @@ -596,13 +599,26 @@ impl ChainSync { peer.state = PeerSyncState::DownloadingStale(hash); have_requests = true; Some((id.clone(), req)) - } else if let Some((range, req)) = peer_block_request(id, peer, blocks, attrs) { + } else if let Some((range, req)) = peer_block_request( + id, + peer, + blocks, + attrs, + max_parallel, + last_finalized + ) { peer.state = PeerSyncState::DownloadingNew(range.start); - trace!(target: "sync", "New block request for {}", id); + trace!( + target: "sync", + "New block request for {}, (best:{}, common:{}) {:?}", + id, + peer.best_number, + peer.common_number, + req, + ); have_requests = true; Some((id.clone(), req)) } else { - trace!(target: "sync", "No new block request for {}", id); None } }); @@ -644,6 +660,7 @@ impl ChainSync { body: block_data.block.body, justification: block_data.block.justification, origin: block_data.origin, + allow_missing_state: false, } }).collect() } @@ -656,14 +673,15 @@ impl ChainSync { body: b.body, justification: b.justification, origin: Some(who.clone()), + allow_missing_state: true, } }).collect() } PeerSyncState::AncestorSearch(num, state) => { - let block_hash_match = match (blocks.get(0), self.client.block_hash(*num)) { + let matching_hash = match (blocks.get(0), self.client.block_hash(*num)) { (Some(block), Ok(maybe_our_block_hash)) => { trace!(target: "sync", "Got ancestry block #{} ({}) from peer {}", num, block.hash, who); - maybe_our_block_hash.map_or(false, |x| x == block.hash) + maybe_our_block_hash.filter(|x| x == &block.hash) }, (None, _) => { debug!(target: "sync", "Invalid response when searching for ancestor from {}", who); @@ -674,27 +692,30 @@ impl ChainSync { return Err(BadPeer(who, ANCESTRY_BLOCK_ERROR_REPUTATION_CHANGE)) } }; - if block_hash_match && peer.common_number < *num { + if matching_hash.is_some() && peer.common_number < *num { peer.common_number = *num; } - if !block_hash_match && num.is_zero() { + if matching_hash.is_none() && num.is_zero() { trace!(target:"sync", "Ancestry search: genesis mismatch for peer {}", who); return Err(BadPeer(who, GENESIS_MISMATCH_REPUTATION_CHANGE)) } - if let Some((next_state, next_num)) = handle_ancestor_search_state(state, *num, block_hash_match) { + if let Some((next_state, next_num)) = handle_ancestor_search_state(state, *num, matching_hash.is_some()) { peer.state = PeerSyncState::AncestorSearch(next_num, next_state); return Ok(OnBlockData::Request(who, ancestry_request::(next_num))) } else { // Ancestry search is complete. Check if peer is on a stale fork unknown to us and // add it to sync targets if necessary. - trace!(target: "sync", "Ancestry search complete. Ours={} ({}), Theirs={} ({}), Common={}", + trace!(target: "sync", "Ancestry search complete. Ours={} ({}), Theirs={} ({}), Common={:?} ({})", self.best_queued_hash, self.best_queued_number, peer.best_hash, peer.best_number, - peer.common_number + matching_hash, + peer.common_number, ); - if peer.common_number < peer.best_number && peer.best_number < self.best_queued_number { + if peer.common_number < peer.best_number + && peer.best_number < self.best_queued_number + { trace!(target: "sync", "Added fork target {} for {}" , peer.best_hash, who); self.fork_targets .entry(peer.best_hash.clone()) @@ -736,14 +757,8 @@ impl ChainSync { self.on_block_queued(h, n) } - let new_best_importing_number = new_blocks.last() - .and_then(|b| b.header.as_ref().map(|h| *h.number())) - .unwrap_or_else(|| Zero::zero()); - self.queue_blocks.extend(new_blocks.iter().map(|b| b.hash)); - self.best_importing_number = std::cmp::max(new_best_importing_number, self.best_importing_number); - Ok(OnBlockData::Import(origin, new_blocks)) } @@ -830,7 +845,6 @@ impl ChainSync { imported: usize, count: usize, results: Vec<(Result>, BlockImportError>, B::Hash)>, - mut peer_info: impl FnMut(&PeerId) -> Option> ) -> impl Iterator), BadPeer>> + 'a { trace!(target: "sync", "Imported {} of {}", imported, count); @@ -878,32 +892,42 @@ impl ChainSync { trace!(target: "sync", "Block imported but requires finality proof {}: {:?}", number, hash); self.request_finality_proof(&hash, number); } + + if number > self.best_imported_number { + self.best_imported_number = number; + } }, Err(BlockImportError::IncompleteHeader(who)) => { if let Some(peer) = who { info!("Peer sent block with incomplete header to import"); output.push(Err(BadPeer(peer, INCOMPLETE_HEADER_REPUTATION_CHANGE))); - output.extend(self.restart(&mut peer_info)); + output.extend(self.restart()); } }, Err(BlockImportError::VerificationFailed(who, e)) => { if let Some(peer) = who { info!("Verification failed from peer: {}", e); output.push(Err(BadPeer(peer, VERIFICATION_FAIL_REPUTATION_CHANGE))); - output.extend(self.restart(&mut peer_info)); + output.extend(self.restart()); } }, Err(BlockImportError::BadBlock(who)) => { if let Some(peer) = who { info!("Bad block"); output.push(Err(BadPeer(peer, BAD_BLOCK_REPUTATION_CHANGE))); - output.extend(self.restart(&mut peer_info)); + output.extend(self.restart()); } }, + Err(BlockImportError::MissingState) => { + // This may happen if the chain we were requesting upon has been discarded + // in the meantime becasue other chain has been finalized. + // Don't mark it as bad as it still may be synced if explicitly requested. + trace!(target: "sync", "Obsolete block"); + }, Err(BlockImportError::UnknownParent) | Err(BlockImportError::Cancelled) | Err(BlockImportError::Other(_)) => { - output.extend(self.restart(&mut peer_info)); + output.extend(self.restart()); }, }; } @@ -911,9 +935,6 @@ impl ChainSync { for hash in hashes { self.queue_blocks.remove(&hash); } - if has_error { - self.best_importing_number = Zero::zero() - } self.is_idle = false; output.into_iter() @@ -958,34 +979,34 @@ impl ChainSync { /// Updates our internal state for best queued block and then goes /// through all peers to update our view of their state as well. fn on_block_queued(&mut self, hash: &B::Hash, number: NumberFor) { - if number > self.best_queued_number { - self.best_queued_number = number; - self.best_queued_hash = *hash; - } if let Some(_) = self.fork_targets.remove(&hash) { trace!(target: "sync", "Completed fork sync {:?}", hash); } - // Update common blocks - for (n, peer) in self.peers.iter_mut() { - if let PeerSyncState::AncestorSearch(_, _) = peer.state { - // Wait for ancestry search to complete first. - continue; + if number > self.best_queued_number { + self.best_queued_number = number; + self.best_queued_hash = *hash; + // Update common blocks + for (n, peer) in self.peers.iter_mut() { + if let PeerSyncState::AncestorSearch(_, _) = peer.state { + // Wait for ancestry search to complete first. + continue; + } + let new_common_number = if peer.best_number >= number { + number + } else { + peer.best_number + }; + trace!( + target: "sync", + "Updating peer {} info, ours={}, common={}->{}, their best={}", + n, + number, + peer.common_number, + new_common_number, + peer.best_number, + ); + peer.common_number = new_common_number; } - let new_common_number = if peer.best_number >= number { - number - } else { - peer.best_number - }; - trace!( - target: "sync", - "Updating peer {} info, ours={}, common={}->{}, their best={}", - n, - number, - peer.common_number, - new_common_number, - peer.best_number, - ); - peer.common_number = new_common_number; } self.is_idle = false; } @@ -1001,7 +1022,7 @@ impl ChainSync { { let header = &announce.header; let number = *header.number(); - debug!(target: "sync", "Received block announcement with number {:?}", number); + debug!(target: "sync", "Received block announcement {:?} with number {:?} from {}", hash, number, who); if number.is_zero() { warn!(target: "sync", "Ignored genesis block (#0) announcement from {}: {}", who, hash); return OnBlockAnnounce::Nothing @@ -1021,7 +1042,7 @@ impl ChainSync { peer.recently_announced.pop_front(); } peer.recently_announced.push_back(hash.clone()); - if is_best && number > peer.best_number { + if is_best { // update their best block peer.best_number = number; peer.best_hash = hash; @@ -1031,10 +1052,12 @@ impl ChainSync { } // If the announced block is the best they have seen, our common number // is either one further ahead or it's the one they just announced, if we know about it. - if known && is_best { - peer.common_number = number - } else if header.parent_hash() == &self.best_queued_hash || known_parent { - peer.common_number = number - One::one(); + if is_best { + if known { + peer.common_number = number + } else if header.parent_hash() == &self.best_queued_hash || known_parent { + peer.common_number = number - One::one(); + } } self.is_idle = false; @@ -1100,22 +1123,18 @@ impl ChainSync { } /// Restart the sync process. - fn restart<'a, F> - (&'a mut self, mut peer_info: F) -> impl Iterator), BadPeer>> + 'a - where F: FnMut(&PeerId) -> Option> + 'a + fn restart<'a>(&'a mut self) -> impl Iterator), BadPeer>> + 'a { self.queue_blocks.clear(); - self.best_importing_number = Zero::zero(); self.blocks.clear(); let info = self.client.info(); self.best_queued_hash = info.chain.best_hash; - self.best_queued_number = info.chain.best_number; + self.best_queued_number = std::cmp::max(info.chain.best_number, self.best_imported_number); self.is_idle = false; debug!(target:"sync", "Restarted with {} ({})", self.best_queued_number, self.best_queued_hash); let old_peers = std::mem::replace(&mut self.peers, HashMap::new()); - old_peers.into_iter().filter_map(move |(id, _)| { - let info = peer_info(&id)?; - match self.new_peer(id.clone(), info) { + old_peers.into_iter().filter_map(move |(id, p)| { + match self.new_peer(id.clone(), p.best_hash, p.best_number) { Ok(None) => None, Ok(Some(x)) => Some(Ok((id, x))), Err(e) => Some(Err(e)) @@ -1123,39 +1142,6 @@ impl ChainSync { }) } - /// Select a range of new blocks to download from the given peer. - fn select_new_blocks(&mut self, who: PeerId) -> Option<(Range>, BlockRequest)> { - // when there are too many blocks in the queue => do not try to download new blocks - if self.queue_blocks.len() > MAX_IMPORTING_BLOCKS { - trace!(target: "sync", "Too many blocks in the queue."); - return None - } - - let peer = self.peers.get_mut(&who)?; - - if !peer.state.is_available() { - trace!(target: "sync", "Peer {} is busy", who); - return None - } - - trace!( - target: "sync", - "Considering new block download from {}, common block is {}, best is {:?}", - who, - peer.common_number, - peer.best_number - ); - - if let Some((range, req)) = peer_block_request(&who, peer, &mut self.blocks, &self.required_block_attributes) { - trace!(target: "sync", "Requesting blocks from {}, ({} to {})", who, range.start, range.end); - peer.state = PeerSyncState::DownloadingNew(range.start); - Some((range, req)) - } else { - trace!(target: "sync", "Nothing to request from {}", who); - None - } - } - /// What is the status of the block corresponding to the given hash? fn block_status(&self, hash: &B::Hash) -> Result { if self.queue_blocks.contains(hash) { @@ -1254,8 +1240,20 @@ fn peer_block_request( peer: &PeerSync, blocks: &mut BlockCollection, attrs: &message::BlockAttributes, + max_parallel_downloads: u32, + finalized: NumberFor, ) -> Option<(Range>, BlockRequest)> { - if let Some(range) = blocks.needed_blocks(id.clone(), MAX_BLOCKS_TO_REQUEST, peer.best_number, peer.common_number) { + if peer.common_number < finalized { + return None; + } + if let Some(range) = blocks.needed_blocks( + id.clone(), + MAX_BLOCKS_TO_REQUEST, + peer.best_number, + peer.common_number, + max_parallel_downloads, + MAX_DOWNLOAD_AHEAD, + ) { let request = message::generic::BlockRequest { id: 0, fields: attrs.clone(), @@ -1273,25 +1271,31 @@ fn peer_block_request( /// Get pending fork sync targets for a peer. fn fork_sync_request( id: &PeerId, - targets: &HashMap>, + targets: &mut HashMap>, best_num: NumberFor, finalized: NumberFor, attributes: &message::BlockAttributes, check_block: impl Fn(&B::Hash) -> BlockStatus, ) -> Option<(B::Hash, BlockRequest)> { + targets.retain(|hash, r| if r.number > finalized { + true + } else { + trace!(target: "sync", "Removed expired fork sync request {:?} (#{})", hash, r.number); + false + }); for (hash, r) in targets { if !r.peers.contains(id) { continue } if r.number <= best_num { - trace!(target: "sync", "Downloading requested fork {:?} from {}", hash, id); let parent_status = r.parent_hash.as_ref().map_or(BlockStatus::Unknown, check_block); let mut count = (r.number - finalized).saturated_into::(); // up to the last finalized block if parent_status != BlockStatus::Unknown { // request only single block count = 1; } + trace!(target: "sync", "Downloading requested fork {:?} from {}, {} blocks", hash, id, count); return Some((hash.clone(), message::generic::BlockRequest { id: 0, fields: attributes.clone(), diff --git a/core/network/src/protocol/sync/blocks.rs b/client/network/src/protocol/sync/blocks.rs similarity index 86% rename from core/network/src/protocol/sync/blocks.rs rename to client/network/src/protocol/sync/blocks.rs index 90264249ea03c..70e9889e1e83f 100644 --- a/core/network/src/protocol/sync/blocks.rs +++ b/client/network/src/protocol/sync/blocks.rs @@ -24,8 +24,6 @@ use libp2p::PeerId; use sr_primitives::traits::{Block as BlockT, NumberFor, One}; use crate::message; -const MAX_PARALLEL_DOWNLOADS: u32 = 1; - /// Block data with origin. #[derive(Debug, Clone, PartialEq, Eq)] pub struct BlockData { @@ -84,9 +82,7 @@ impl BlockCollection { match self.blocks.get(&start) { Some(&BlockRangeState::Downloading { .. }) => { - trace!(target: "sync", "Ignored block data still marked as being downloaded: {}", start); - debug_assert!(false); - return; + trace!(target: "sync", "Inserting block data still marked as being downloaded: {}", start); }, Some(&BlockRangeState::Complete(ref existing)) if existing.len() >= blocks.len() => { trace!(target: "sync", "Ignored block data already downloaded: {}", start); @@ -100,8 +96,20 @@ impl BlockCollection { } /// Returns a set of block hashes that require a header download. The returned set is marked as being downloaded. - pub fn needed_blocks(&mut self, who: PeerId, count: usize, peer_best: NumberFor, common: NumberFor) - -> Option>> { + pub fn needed_blocks( + &mut self, + who: PeerId, + count: usize, + peer_best: NumberFor, + common: NumberFor, + max_parallel: u32, + max_ahead: u32, + ) -> Option>> + { + if peer_best <= common { + // Bail out early + return None; + } // First block number that we need to download let first_different = common + >::one(); let count = (count as u32).into(); @@ -112,7 +120,7 @@ impl BlockCollection { let next = downloading_iter.next(); break match &(prev, next) { &(Some((start, &BlockRangeState::Downloading { ref len, downloading })), _) - if downloading < MAX_PARALLEL_DOWNLOADS => + if downloading < max_parallel => (*start .. *start + *len, downloading), &(Some((start, r)), Some((next_start, _))) if *start + r.len() < *next_start => (*start + r.len() .. cmp::min(*next_start, *start + r.len() + count), 0), // gap @@ -135,6 +143,12 @@ impl BlockCollection { return None; } range.end = cmp::min(peer_best + One::one(), range.end); + + if self.blocks.iter().next().map_or(false, |(n, _)| range.start > *n + max_ahead.into()) { + trace!(target: "sync", "Too far ahead for peer {} ({})", who, range.start); + return None; + } + self.peer_requests.insert(who, range.start); self.blocks.insert(range.start, BlockRangeState::Downloading { len: range.end - range.start, @@ -185,7 +199,6 @@ impl BlockCollection { true }, _ => { - debug_assert!(false); false } }; @@ -242,18 +255,18 @@ mod test { let peer2 = PeerId::random(); let blocks = generate_blocks(150); - assert_eq!(bc.needed_blocks(peer0.clone(), 40, 150, 0), Some(1 .. 41)); - assert_eq!(bc.needed_blocks(peer1.clone(), 40, 150, 0), Some(41 .. 81)); - assert_eq!(bc.needed_blocks(peer2.clone(), 40, 150, 0), Some(81 .. 121)); + assert_eq!(bc.needed_blocks(peer0.clone(), 40, 150, 0, 1, 200), Some(1 .. 41)); + assert_eq!(bc.needed_blocks(peer1.clone(), 40, 150, 0, 1, 200), Some(41 .. 81)); + assert_eq!(bc.needed_blocks(peer2.clone(), 40, 150, 0, 1, 200), Some(81 .. 121)); bc.clear_peer_download(&peer1); bc.insert(41, blocks[41..81].to_vec(), peer1.clone()); assert_eq!(bc.drain(1), vec![]); - assert_eq!(bc.needed_blocks(peer1.clone(), 40, 150, 0), Some(121 .. 151)); + assert_eq!(bc.needed_blocks(peer1.clone(), 40, 150, 0, 1, 200), Some(121 .. 151)); bc.clear_peer_download(&peer0); bc.insert(1, blocks[1..11].to_vec(), peer0.clone()); - assert_eq!(bc.needed_blocks(peer0.clone(), 40, 150, 0), Some(11 .. 41)); + assert_eq!(bc.needed_blocks(peer0.clone(), 40, 150, 0, 1, 200), Some(11 .. 41)); assert_eq!(bc.drain(1), blocks[1..11].iter() .map(|b| BlockData { block: b.clone(), origin: Some(peer0.clone()) }).collect::>()); @@ -267,7 +280,7 @@ mod test { .map(|b| BlockData { block: b.clone(), origin: Some(peer1.clone()) }).collect::>()[..]); bc.clear_peer_download(&peer2); - assert_eq!(bc.needed_blocks(peer2.clone(), 40, 150, 80), Some(81 .. 121)); + assert_eq!(bc.needed_blocks(peer2.clone(), 40, 150, 80, 1, 200), Some(81 .. 121)); bc.clear_peer_download(&peer2); bc.insert(81, blocks[81..121].to_vec(), peer2.clone()); bc.clear_peer_download(&peer1); @@ -292,7 +305,8 @@ mod test { bc.blocks.insert(114305, BlockRangeState::Complete(blocks)); let peer0 = PeerId::random(); - assert_eq!(bc.needed_blocks(peer0.clone(), 128, 10000, 000), Some(1 .. 100)); - assert_eq!(bc.needed_blocks(peer0.clone(), 128, 10000, 600), Some(100 + 128 .. 100 + 128 + 128)); + assert_eq!(bc.needed_blocks(peer0.clone(), 128, 10000, 000, 1, 200), Some(1 .. 100)); + assert_eq!(bc.needed_blocks(peer0.clone(), 128, 10000, 600, 1, 200), None); // too far ahead + assert_eq!(bc.needed_blocks(peer0.clone(), 128, 10000, 600, 1, 200000), Some(100 + 128 .. 100 + 128 + 128)); } } diff --git a/core/network/src/protocol/sync/extra_requests.rs b/client/network/src/protocol/sync/extra_requests.rs similarity index 99% rename from core/network/src/protocol/sync/extra_requests.rs rename to client/network/src/protocol/sync/extra_requests.rs index 28ed5fed9f682..5bfeeba2a5c3a 100644 --- a/core/network/src/protocol/sync/extra_requests.rs +++ b/client/network/src/protocol/sync/extra_requests.rs @@ -14,7 +14,7 @@ // You should have received a copy of the GNU General Public License // along with Substrate. If not, see . -use client::error::Error as ClientError; +use client_api::error::Error as ClientError; use crate::protocol::sync::{PeerSync, PeerSyncState}; use fork_tree::ForkTree; use libp2p::PeerId; @@ -283,7 +283,7 @@ impl<'a, B: BlockT> Matcher<'a, B> { #[cfg(test)] mod tests { use crate::protocol::sync::PeerSync; - use client::error::Error as ClientError; + use client_api::error::Error as ClientError; use quickcheck::{Arbitrary, Gen, QuickCheck, StdThreadGen}; use rand::Rng; use std::collections::{HashMap, HashSet}; diff --git a/core/network/src/protocol/util.rs b/client/network/src/protocol/util.rs similarity index 100% rename from core/network/src/protocol/util.rs rename to client/network/src/protocol/util.rs diff --git a/core/network/src/service.rs b/client/network/src/service.rs similarity index 97% rename from core/network/src/service.rs rename to client/network/src/service.rs index 48ad51bde31a6..437e978f4bd47 100644 --- a/core/network/src/service.rs +++ b/client/network/src/service.rs @@ -71,7 +71,8 @@ pub trait TransactionPool: Send + Sync { &self, report_handle: ReportHandle, who: PeerId, - reputation_change: i32, + reputation_change_good: i32, + reputation_change_bad: i32, transaction: B::Extrinsic, ); /// Notify the pool about transactions broadcast. @@ -193,7 +194,10 @@ impl, H: ExHashT> NetworkWorker let num_connected = Arc::new(AtomicUsize::new(0)); let is_major_syncing = Arc::new(AtomicBool::new(false)); let (protocol, peerset_handle) = Protocol::new( - protocol::ProtocolConfig { roles: params.roles }, + protocol::ProtocolConfig { + roles: params.roles, + max_parallel_downloads: params.network_config.max_parallel_downloads, + }, params.chain, params.on_demand.as_ref().map(|od| od.checker().clone()) .unwrap_or(Arc::new(AlwaysBadChecker)), @@ -221,7 +225,11 @@ impl, H: ExHashT> NetworkWorker match params.network_config.transport { TransportConfig::MemoryOnly => false, TransportConfig::Normal { enable_mdns, .. } => enable_mdns, - } + }, + match params.network_config.transport { + TransportConfig::MemoryOnly => false, + TransportConfig::Normal { allow_private_ipv4, .. } => allow_private_ipv4, + }, ); let (transport, bandwidth) = { let (config_mem, config_wasm) = match params.network_config.transport { @@ -480,8 +488,8 @@ impl, H: ExHashT> NetworkServic /// Start getting a value from the DHT. /// - /// This will generate either a `ValueFound` or a `ValueNotFound` event and pass it to - /// `on_event` on the network specialization. + /// This will generate either a `ValueFound` or a `ValueNotFound` event and pass it as an + /// item on the [`NetworkWorker`] stream. pub fn get_value(&self, key: &record::Key) { let _ = self .to_worker @@ -490,8 +498,8 @@ impl, H: ExHashT> NetworkServic /// Start putting a value in the DHT. /// - /// This will generate either a `ValuePut` or a `ValuePutFailed` event and pass it to - /// `on_event` on the network specialization. + /// This will generate either a `ValuePut` or a `ValuePutFailed` event and pass it as an + /// item on the [`NetworkWorker`] stream. pub fn put_value(&self, key: record::Key, value: Vec) { let _ = self .to_worker @@ -718,12 +726,8 @@ impl, H: ExHashT> Stream for Ne let outcome = match poll_value { Ok(Async::NotReady) => break, Ok(Async::Ready(Some(BehaviourOut::SubstrateAction(outcome)))) => outcome, - Ok(Async::Ready(Some(BehaviourOut::Dht(ev)))) => { - self.network_service.user_protocol_mut() - .on_event(Event::Dht(ev.clone())); - - return Ok(Async::Ready(Some(Event::Dht(ev)))); - }, + Ok(Async::Ready(Some(BehaviourOut::Dht(ev)))) => + return Ok(Async::Ready(Some(Event::Dht(ev)))), Ok(Async::Ready(None)) => CustomMessageOutcome::None, Err(err) => { error!(target: "sync", "Error in the network: {:?}", err); diff --git a/core/network/src/test/block_import.rs b/client/network/src/test/block_import.rs similarity index 96% rename from core/network/src/test/block_import.rs rename to client/network/src/test/block_import.rs index f2830548a501a..6d077dcc6bba7 100644 --- a/core/network/src/test/block_import.rs +++ b/client/network/src/test/block_import.rs @@ -37,9 +37,10 @@ fn prepare_good_block() -> (TestClient, Hash, u64, PeerId, IncomingBlock) (client, hash, number, peer_id.clone(), IncomingBlock { hash, header, - body: None, + body: Some(Vec::new()), justification, - origin: Some(peer_id.clone()) + origin: Some(peer_id.clone()), + allow_missing_state: false, }) } @@ -53,7 +54,7 @@ fn import_single_good_block_works() { match import_single_block(&mut test_client::new(), BlockOrigin::File, block, &mut PassThroughVerifier(true)) { Ok(BlockImportResult::ImportedUnknown(ref num, ref aux, ref org)) if *num == number && *aux == expected_aux && *org == Some(peer_id) => {} - _ => panic!() + r @ _ => panic!("{:?}", r) } } diff --git a/core/network/src/test/mod.rs b/client/network/src/test/mod.rs similarity index 95% rename from core/network/src/test/mod.rs rename to client/network/src/test/mod.rs index 92e747280bbbb..964f27051a47d 100644 --- a/core/network/src/test/mod.rs +++ b/client/network/src/test/mod.rs @@ -27,15 +27,16 @@ use std::sync::Arc; use crate::config::build_multiaddr; use log::trace; use crate::chain::FinalityProofProvider; -use client::{ - self, ClientInfo, BlockchainEvents, BlockImportNotification, +use client_api::{ + ClientInfo, BlockchainEvents, BlockImportNotification, FinalityNotifications, ImportNotifications, - FinalityNotification, LongestChain, + FinalityNotification, error::Result as ClientResult, well_known_cache_keys::{self, Id as CacheKeyId}, + backend::{AuxStore, Backend, Finalizer} }; -use client::block_builder::BlockBuilder; -use client::backend::{AuxStore, Backend, Finalizer}; +use block_builder::BlockBuilder; +use client::LongestChain; use crate::config::Roles; use consensus::block_validation::DefaultBlockAnnounceValidator; use consensus::import_queue::BasicQueue; @@ -96,6 +97,7 @@ impl Verifier for PassThroughVerifier { post_digests: vec![], auxiliary: Vec::new(), fork_choice: ForkChoiceStrategy::LongestChain, + allow_missing_state: false, }, maybe_keys)) } } @@ -124,11 +126,6 @@ impl NetworkSpecialization for DummySpecialization { _peer_id: PeerId, _message: Vec, ) {} - - fn on_event( - &mut self, - _event: crate::specialization::Event - ) {} } pub type PeersFullClient = @@ -379,16 +376,10 @@ impl> Peer { } } - /// Count the current number of known blocks. Note that: - /// 1. this might be expensive as it creates an in-memory-copy of the chain - /// to count the blocks, thus if you have a different way of testing this - /// (e.g. `info.best_hash`) - use that. - /// 2. This is not always increasing nor accurate, as the - /// orphaned and proven-to-never-finalized blocks may be pruned at any time. - /// Therefore, this number can drop again. - pub fn blocks_count(&self) -> usize { + /// Count the total number of imported blocks. + pub fn blocks_count(&self) -> u64 { self.backend.as_ref().map( - |backend| backend.as_in_memory().blockchain().blocks_count() + |backend| backend.blocks_count() ).unwrap_or(0) } } @@ -404,7 +395,14 @@ impl TransactionPool for EmptyTransactionPool { Hash::default() } - fn import(&self, _report_handle: ReportHandle, _who: PeerId, _rep_change: i32, _transaction: Extrinsic) {} + fn import( + &self, + _report_handle: ReportHandle, + _who: PeerId, + _rep_change_good: i32, + _rep_change_bad: i32, + _transaction: Extrinsic + ) {} fn on_broadcasted(&self, _: HashMap>) {} } @@ -524,9 +522,16 @@ pub trait TestNetFactory: Sized { net } - /// Add a full peer. fn add_full_peer(&mut self, config: &ProtocolConfig) { - let test_client_builder = TestClientBuilder::with_default_backend(); + self.add_full_peer_with_states(config, None) + } + + /// Add a full peer. + fn add_full_peer_with_states(&mut self, config: &ProtocolConfig, keep_blocks: Option) { + let test_client_builder = match keep_blocks { + Some(keep_blocks) => TestClientBuilder::with_pruning_window(keep_blocks), + None => TestClientBuilder::with_default_backend(), + }; let backend = test_client_builder.backend(); let (c, longest_chain) = test_client_builder.build_with_longest_chain(); let client = Arc::new(c); @@ -684,7 +689,7 @@ pub trait TestNetFactory: Sized { if peer.is_major_syncing() || peer.network.num_queued_blocks() != 0 { return Async::NotReady } - match (highest, peer.client.info().chain.best_number) { + match (highest, peer.client.info().chain.best_hash) { (None, b) => highest = Some(b), (Some(ref a), ref b) if a == b => {}, (Some(_), _) => return Async::NotReady, diff --git a/core/network/src/test/sync.rs b/client/network/src/test/sync.rs similarity index 89% rename from core/network/src/test/sync.rs rename to client/network/src/test/sync.rs index b1b2b9d407262..9868bd0ed2bd9 100644 --- a/core/network/src/test/sync.rs +++ b/client/network/src/test/sync.rs @@ -236,7 +236,14 @@ fn sync_no_common_longer_chain_fails() { let mut net = TestNet::new(3); net.peer(0).push_blocks(20, true); net.peer(1).push_blocks(20, false); - net.block_until_sync(&mut runtime); + runtime.block_on(futures::future::poll_fn::<(), (), _>(|| -> Result<_, ()> { + net.poll(); + if net.peer(0).is_major_syncing() { + Ok(Async::NotReady) + } else { + Ok(Async::Ready(())) + } + })).unwrap(); let peer1 = &net.peers()[1]; assert!(!net.peers()[0].blockchain_canon_equals(peer1)); } @@ -592,3 +599,64 @@ fn can_sync_explicit_forks() { Ok(Async::Ready(())) })).unwrap(); } + +#[test] +fn syncs_header_only_forks() { + let _ = ::env_logger::try_init(); + let mut runtime = current_thread::Runtime::new().unwrap(); + let mut net = TestNet::new(0); + let config = ProtocolConfig::default(); + net.add_full_peer_with_states(&config, None); + net.add_full_peer_with_states(&config, Some(3)); + net.peer(0).push_blocks(2, false); + net.peer(1).push_blocks(2, false); + + net.peer(0).push_blocks(2, true); + let small_hash = net.peer(0).client().info().chain.best_hash; + let small_number = net.peer(0).client().info().chain.best_number; + net.peer(1).push_blocks(4, false); + + net.block_until_sync(&mut runtime); + // Peer 1 will sync the small fork even though common block state is missing + assert_eq!(9, net.peer(0).blocks_count()); + assert_eq!(9, net.peer(1).blocks_count()); + + // Request explicit header-only sync request for the ancient fork. + let first_peer_id = net.peer(0).id(); + net.peer(1).set_sync_fork_request(vec![first_peer_id], small_hash, small_number); + runtime.block_on(futures::future::poll_fn::<(), (), _>(|| -> Result<_, ()> { + net.poll(); + if net.peer(1).client().header(&BlockId::Hash(small_hash)).unwrap().is_none() { + return Ok(Async::NotReady) + } + Ok(Async::Ready(())) + })).unwrap(); +} + +#[test] +fn does_not_sync_announced_old_best_block() { + let _ = ::env_logger::try_init(); + let mut runtime = current_thread::Runtime::new().unwrap(); + let mut net = TestNet::new(3); + + let old_hash = net.peer(0).push_blocks(1, false); + let old_hash_with_parent = net.peer(0).push_blocks(1, false); + net.peer(0).push_blocks(18, true); + net.peer(1).push_blocks(20, true); + + net.peer(0).announce_block(old_hash, Vec::new()); + runtime.block_on(futures::future::poll_fn::<(), (), _>(|| -> Result<_, ()> { + // poll once to import announcement + net.poll(); + Ok(Async::Ready(())) + })).unwrap(); + assert!(!net.peer(1).is_major_syncing()); + + net.peer(0).announce_block(old_hash_with_parent, Vec::new()); + runtime.block_on(futures::future::poll_fn::<(), (), _>(|| -> Result<_, ()> { + // poll once to import announcement + net.poll(); + Ok(Async::Ready(())) + })).unwrap(); + assert!(!net.peer(1).is_major_syncing()); +} diff --git a/core/network/src/transport.rs b/client/network/src/transport.rs similarity index 89% rename from core/network/src/transport.rs rename to client/network/src/transport.rs index 901ec18581e1d..24c79bfa30fdb 100644 --- a/core/network/src/transport.rs +++ b/client/network/src/transport.rs @@ -22,8 +22,8 @@ use libp2p::{ #[cfg(not(target_os = "unknown"))] use libp2p::{tcp, dns, websocket, noise}; #[cfg(not(target_os = "unknown"))] -use libp2p::core::{upgrade, either::EitherError, either::EitherOutput}; -use libp2p::core::{self, transport::boxed::Boxed, transport::OptionalTransport, muxing::StreamMuxerBox}; +use libp2p::core::{either::EitherError, either::EitherOutput}; +use libp2p::core::{self, upgrade, transport::boxed::Boxed, transport::OptionalTransport, muxing::StreamMuxerBox}; use std::{io, sync::Arc, time::Duration, usize}; pub use self::bandwidth::BandwidthSinks; @@ -90,7 +90,7 @@ pub fn build_transport( #[cfg(not(target_os = "unknown"))] let transport = transport.and_then(move |stream, endpoint| { let upgrade = core::upgrade::SelectUpgrade::new(noise_config, secio_config); - core::upgrade::apply(stream, upgrade, endpoint) + core::upgrade::apply(stream, upgrade, endpoint, upgrade::Version::V1) .and_then(|out| match out { // We negotiated noise EitherOutput::First((remote_id, out)) => { @@ -101,16 +101,16 @@ pub fn build_transport( Ok((EitherOutput::First(out), remote_key.into_peer_id())) } // We negotiated secio - EitherOutput::Second(out) => - Ok((EitherOutput::Second(out.stream), out.remote_key.into_peer_id())) + EitherOutput::Second((remote_id, out)) => + Ok((EitherOutput::Second(out), remote_id)) }) }); // For WASM, we only support secio for now. #[cfg(target_os = "unknown")] let transport = transport.and_then(move |stream, endpoint| { - core::upgrade::apply(stream, secio_config, endpoint) - .and_then(|out| Ok((out.stream, out.remote_key.into_peer_id()))) + core::upgrade::apply(stream, secio_config, endpoint, upgrade::Version::V1) + .and_then(|(id, stream)| Ok((stream, id))) }); // Multiplexing @@ -120,11 +120,11 @@ pub fn build_transport( .map_inbound(move |muxer| (peer_id, muxer)) .map_outbound(move |muxer| (peer_id2, muxer)); - core::upgrade::apply(stream, upgrade, endpoint) + core::upgrade::apply(stream, upgrade, endpoint, upgrade::Version::V1) .map(|(id, muxer)| (id, core::muxing::StreamMuxerBox::new(muxer))) }) - .with_timeout(Duration::from_secs(20)) + .timeout(Duration::from_secs(20)) .map_err(|err| io::Error::new(io::ErrorKind::Other, err)) .boxed(); diff --git a/core/offchain/Cargo.toml b/client/offchain/Cargo.toml similarity index 63% rename from core/offchain/Cargo.toml rename to client/offchain/Cargo.toml index b52118aae7b70..29621d8502bc1 100644 --- a/core/offchain/Cargo.toml +++ b/client/offchain/Cargo.toml @@ -8,7 +8,8 @@ edition = "2018" [dependencies] bytes = "0.4.12" -client = { package = "substrate-client", path = "../../core/client" } +client-api = { package = "substrate-client-api", path = "../api" } +sr-api = { path = "../../primitives/sr-api" } fnv = "1.0.6" futures01 = { package = "futures", version = "0.1" } futures-preview = "0.3.0-alpha.19" @@ -16,24 +17,24 @@ futures-timer = "0.4.0" log = "0.4.8" threadpool = "1.7" num_cpus = "1.10" -offchain-primitives = { package = "substrate-offchain-primitives", path = "./primitives" } +offchain-primitives = { package = "substrate-offchain-primitives", path = "../../primitives/offchain" } codec = { package = "parity-scale-codec", version = "1.0.0", features = ["derive"] } parking_lot = "0.9.0" -primitives = { package = "substrate-primitives", path = "../../core/primitives" } +primitives = { package = "substrate-primitives", path = "../../primitives/core" } rand = "0.7.2" -sr-primitives = { path = "../../core/sr-primitives" } -transaction_pool = { package = "substrate-transaction-pool", path = "../../core/transaction-pool" } -network = { package = "substrate-network", path = "../../core/network" } +sr-primitives = { path = "../../primitives/sr-primitives" } +transaction_pool = { package = "substrate-transaction-pool", path = "../transaction-pool" } +network = { package = "substrate-network", path = "../network" } keystore = { package = "substrate-keystore", path = "../keystore" } [target.'cfg(not(target_os = "unknown"))'.dependencies] hyper = "0.12.35" -hyper-tls = "0.3.2" +hyper-rustls = "0.17.1" [dev-dependencies] env_logger = "0.7.0" -client-db = { package = "substrate-client-db", path = "../../core/client/db/", default-features = true } -test-client = { package = "substrate-test-runtime-client", path = "../../core/test-runtime/client" } +client-db = { package = "substrate-client-db", path = "../db/", default-features = true } +test-client = { package = "substrate-test-runtime-client", path = "../../test/utils/runtime/client" } tokio = "0.1.22" [features] diff --git a/core/offchain/src/api.rs b/client/offchain/src/api.rs similarity index 99% rename from core/offchain/src/api.rs rename to client/offchain/src/api.rs index 35b6e20df2b29..25d4fc267ae4d 100644 --- a/core/offchain/src/api.rs +++ b/client/offchain/src/api.rs @@ -21,7 +21,7 @@ use std::{ thread::sleep, }; -use client::backend::OffchainStorage; +use client_api::OffchainStorage; use futures::{StreamExt as _, Future, FutureExt as _, future, channel::mpsc}; use log::{info, debug, warn, error}; use network::{PeerId, Multiaddr, NetworkStateInfo}; @@ -30,6 +30,7 @@ use primitives::offchain::{ Externalities as OffchainExt, HttpRequestId, Timestamp, HttpRequestStatus, HttpError, OpaqueNetworkState, OpaquePeerId, OpaqueMultiaddr, StorageKind, }; +pub use offchain_primitives::STORAGE_PREFIX; use sr_primitives::{generic::BlockId, traits::{self, Extrinsic}}; use transaction_pool::txpool::{Pool, ChainApi}; @@ -71,7 +72,6 @@ fn unavailable_yet(name: &str) -> R { } const LOCAL_DB: &str = "LOCAL (fork-aware) DB"; -const STORAGE_PREFIX: &[u8] = b"storage"; impl OffchainExt for Api where diff --git a/core/offchain/src/api/http.rs b/client/offchain/src/api/http.rs similarity index 96% rename from core/offchain/src/api/http.rs rename to client/offchain/src/api/http.rs index 6744a42f959c7..30cadf091869f 100644 --- a/core/offchain/src/api/http.rs +++ b/client/offchain/src/api/http.rs @@ -29,7 +29,7 @@ use crate::api::timestamp; use bytes::Buf as _; use fnv::FnvHashMap; use futures::{prelude::*, channel::mpsc, compat::Compat01As03}; -use log::{warn, error}; +use log::error; use primitives::offchain::{HttpRequestId, Timestamp, HttpRequestStatus, HttpError}; use std::{fmt, io::Read as _, mem, pin::Pin, task::Context, task::Poll}; @@ -50,9 +50,7 @@ pub fn http() -> (HttpApi, HttpWorker) { let engine = HttpWorker { to_api, from_api, - // TODO: don't unwrap; we should fall back to the HttpConnector if we fail to create the - // Https one; there doesn't seem to be any built-in way to do this - http_client: HyperClient::new(), + http_client: hyper::Client::builder().build(hyper_rustls::HttpsConnector::new(1)), requests: Vec::new(), }; @@ -551,30 +549,6 @@ enum WorkerToApi { }, } -/// Wraps around a `hyper::Client` with either TLS enabled or disabled. -enum HyperClient { - /// Everything is ok and HTTPS is available. - Https(hyper::Client, hyper::Body>), - /// We failed to initialize HTTPS and therefore only allow HTTP. - Http(hyper::Client), -} - -impl HyperClient { - /// Creates new hyper client. - /// - /// By default we will try to initialize the `HttpsConnector`, - /// If that's not possible we'll fall back to `HttpConnector`. - pub fn new() -> Self { - match hyper_tls::HttpsConnector::new(1) { - Ok(tls) => HyperClient::Https(hyper::Client::builder().build(tls)), - Err(e) => { - warn!("Unable to initialize TLS client. Falling back to HTTP-only: {:?}", e); - HyperClient::Http(hyper::Client::new()) - }, - } - } -} - /// Must be continuously polled for the [`HttpApi`] to properly work. pub struct HttpWorker { /// Used to sends messages to the `HttpApi`. @@ -582,7 +556,7 @@ pub struct HttpWorker { /// Used to receive messages from the `HttpApi`. from_api: mpsc::UnboundedReceiver, /// The engine that runs HTTP requests. - http_client: HyperClient, + http_client: hyper::Client, hyper::Body>, /// HTTP requests that are being worked on by the engine. requests: Vec<(HttpRequestId, HttpWorkerRequest)>, } @@ -686,10 +660,7 @@ impl Future for HttpWorker { Poll::Pending => {}, Poll::Ready(None) => return Poll::Ready(()), // stops the worker Poll::Ready(Some(ApiToWorker::Dispatch { id, request })) => { - let future = Compat01As03::new(match me.http_client { - HyperClient::Http(ref mut c) => c.request(request), - HyperClient::Https(ref mut c) => c.request(request), - }); + let future = Compat01As03::new(me.http_client.request(request)); debug_assert!(me.requests.iter().all(|(i, _)| *i != id)); me.requests.push((id, HttpWorkerRequest::Dispatched(future))); cx.waker().wake_by_ref(); // reschedule the task to poll the request diff --git a/core/offchain/src/api/http_dummy.rs b/client/offchain/src/api/http_dummy.rs similarity index 100% rename from core/offchain/src/api/http_dummy.rs rename to client/offchain/src/api/http_dummy.rs diff --git a/core/offchain/src/api/timestamp.rs b/client/offchain/src/api/timestamp.rs similarity index 100% rename from core/offchain/src/api/timestamp.rs rename to client/offchain/src/api/timestamp.rs diff --git a/core/offchain/src/lib.rs b/client/offchain/src/lib.rs similarity index 94% rename from core/offchain/src/lib.rs rename to client/offchain/src/lib.rs index a335ca53807ab..e253a6d9234fb 100644 --- a/core/offchain/src/lib.rs +++ b/client/offchain/src/lib.rs @@ -33,27 +33,24 @@ #![warn(missing_docs)] -use std::{ - fmt, - marker::PhantomData, - sync::Arc, -}; +use std::{fmt, marker::PhantomData, sync::Arc}; use parking_lot::Mutex; use threadpool::ThreadPool; -use client::runtime_api::ApiExt; +use sr_api::ApiExt; use futures::future::Future; use log::{debug, warn}; use network::NetworkStateInfo; use primitives::{offchain, ExecutionContext}; use sr_primitives::{generic::BlockId, traits::{self, ProvideRuntimeApi}}; use transaction_pool::txpool::{Pool, ChainApi}; +use client_api::{OffchainStorage}; mod api; pub mod testing; -pub use offchain_primitives::OffchainWorkerApi; +pub use offchain_primitives::{OffchainWorkerApi, STORAGE_PREFIX}; /// An offchain workers manager. pub struct OffchainWorkers { @@ -93,7 +90,7 @@ impl OffchainWorkers< Block: traits::Block, Client: ProvideRuntimeApi + Send + Sync + 'static, Client::Api: OffchainWorkerApi, - Storage: client::backend::OffchainStorage + 'static, + Storage: OffchainStorage + 'static, { /// Start the offchain workers after given block. #[must_use] @@ -106,7 +103,7 @@ impl OffchainWorkers< ) -> impl Future where A: ChainApi + 'static { let runtime = self.client.runtime_api(); let at = BlockId::number(*number); - let has_api = runtime.has_api::>(&at); + let has_api = runtime.has_api::>(&at); debug!("Checking offchain workers at {:?}: {:?}", at, has_api); if has_api.unwrap_or(false) { diff --git a/core/offchain/src/testing.rs b/client/offchain/src/testing.rs similarity index 96% rename from core/offchain/src/testing.rs rename to client/offchain/src/testing.rs index e1cc7f71a3811..b1699a94ca37f 100644 --- a/core/offchain/src/testing.rs +++ b/client/offchain/src/testing.rs @@ -20,7 +20,7 @@ use std::{ collections::BTreeMap, sync::Arc, }; -use client::backend::OffchainStorage; +use client_api::{OffchainStorage, InMemOffchainStorage}; use parking_lot::RwLock; use primitives::offchain::{ self, @@ -64,9 +64,9 @@ pub struct State { pub requests: BTreeMap, expected_requests: BTreeMap, /// Persistent local storage - pub persistent_storage: client::in_mem::OffchainStorage, + pub persistent_storage: InMemOffchainStorage, /// Local storage - pub local_storage: client::in_mem::OffchainStorage, + pub local_storage: InMemOffchainStorage, /// A vector of transactions submitted from the runtime. pub transactions: Vec>, } @@ -119,7 +119,8 @@ impl State { impl Drop for State { fn drop(&mut self) { - if !self.expected_requests.is_empty() { + // If we panic! while we are already in a panic, the test dies with an illegal instruction. + if !self.expected_requests.is_empty() && !std::thread::panicking() { panic!("Unfulfilled expected requests: {:?}", self.expected_requests); } } diff --git a/client/prometheus/Cargo.toml b/client/prometheus/Cargo.toml new file mode 100644 index 0000000000000..7d258df409389 --- /dev/null +++ b/client/prometheus/Cargo.toml @@ -0,0 +1,17 @@ +[package] +name = "substrate-prometheus" +version = "2.0.0" +authors = ["Parity Technologies "] +description = "prometheus utils" +edition = "2018" + +[dependencies] +hyper = "0.12" +lazy_static = "1.0" +log = "0.4" +prometheus = { version = "0.7", features = ["nightly", "process"]} +tokio = "0.1" +sr-primitives = { path = "../../primitives/sr-primitives" } + +[dev-dependencies] +reqwest = "0.9" \ No newline at end of file diff --git a/client/prometheus/README.md b/client/prometheus/README.md new file mode 100644 index 0000000000000..fcf07efeae496 --- /dev/null +++ b/client/prometheus/README.md @@ -0,0 +1,661 @@ +# Substrate Prometheus Node Exporter + +## Introduction + +Prometheus is one of the most widely used monitoring tool for managing high availability services supported by [Cloud Native Computing Foundation](https://www.cncf.io/). By providing Prometheus exporter in substrate, node operators can easily adopt widely used display/alert tool such as Grafana without seting-up/operating external Prometheus agent through RPC connections. Easy access to such monitoring tools will benefit parachain develepers/operators and validators to have much higher availability quality of their services. + +## List of Contents + +Hack Prometheus in Substrate + - Prometheus starter + - CLI Config + - Metrics Add + +Metrics + - List of available metrics + +Start Prometheus + - Install prometheus + - Edit Prometheus config file + - Start Prometheus + +Start Grafana + - Install Grafana + +## Substrate Dev hack +### Prometheus starter + +Here is the entry point of prometheus core module in Parity Substrate. + +client/prometheus/src/lib.rs +```rust +#[macro_use] +extern crate lazy_static; +#[macro_use] +extern crate log; +use hyper::http::StatusCode; +use hyper::rt::Future; +use hyper::service::service_fn_ok; +use hyper::{Body, Request, Response, Server}; +pub use prometheus::{Encoder, HistogramOpts, Opts, TextEncoder}; +pub use prometheus::{Histogram, IntCounter, IntGauge, Result}; +pub use sr_primitives::traits::SaturatedConversion; +use std::net::SocketAddr; + +pub mod metrics; + +/// Initializes the metrics context, and starts an HTTP server +/// to serve metrics. +pub fn init_prometheus(prometheus_addr: SocketAddr) { + let addr = prometheus_addr; + let server = Server::bind(&addr) + .serve(|| { + // This is the `Service` that will handle the connection. + // `service_fn_ok` is a helper to convert a function that + // returns a Response into a `Service`. + service_fn_ok(move |req: Request| { + if req.uri().path() == "/metrics" { + let metric_families = prometheus::gather(); + let mut buffer = vec![]; + let encoder = TextEncoder::new(); + encoder.encode(&metric_families, &mut buffer).unwrap(); + Response::builder() + .status(StatusCode::OK) + .header("Content-Type", encoder.format_type()) + .body(Body::from(buffer)) + .expect("Sends OK(200) response with one or more data metrics") + } else { + Response::builder() + .status(StatusCode::NOT_FOUND) + .body(Body::from("Not found.")) + .expect("Sends NOT_FOUND(404) message with no data metric") + } + }) + }) + .map_err(|e| error!("server error: {}", e)); + + info!("Exporting metrics at http://{}/metrics", addr); + + let mut rt = tokio::runtime::Builder::new() + .core_threads(1) // one thread is sufficient + .build() + .expect("Builds one thread of tokio runtime exporter for prometheus"); + + std::thread::spawn(move || { + rt.spawn(server); + rt.shutdown_on_idle().wait().unwrap(); + }); +} + +#[macro_export] +macro_rules! prometheus_gauge( + ($($metric:expr => $value:expr),*) => { + use $crate::{metrics::*}; + $( + metrics::set_gauge(&$metric, $value); + )* + } +); + +#[macro_export] +macro_rules! prometheus_histogram( + ($($metric:expr => $value:expr),*) => { + use $crate::{metrics::*}; + $( + metrics::set_histogram(&$metric, $value); + )* + } +); + +#[macro_export] +macro_rules! prometheus_counter( + ($($metric:expr => $value:expr),*) => { + use $crate::{metrics::*}; + $( + metrics::set_counter(&$metric, $value); + )* + } +); + +/* +TODO: Make abstract type for all metrics(e.g. Gauge, Histogram, Counter) with generic traits so that all metrics can be set up with one function `set` +#[macro_export] +macro_rules! prometheus( + ($($a: expr; $metric:expr => $value:expr),*) => { + use $crate::{metrics::*}; + $( + metrics::set(#$a, &$metric, $value); + )* + } +); +*/ +``` + +Here is the dependancies of the module. +client/prometheus/Cargo.toml +```toml +[package] +name = "substrate-prometheus" +version = "2.0.0" +authors = ["Parity Technologies "] +description = "prometheus utils" +edition = "2018" + +[dependencies] +hyper = "0.12" +lazy_static = "1.0" +log = "0.4" +prometheus = { version = "0.7", features = ["nightly", "process"]} +tokio = "0.1" + +[dev-dependencies] +reqwest = "0.9" +``` + +**Abbreviation of the package in service manager of parity substrate** +client/service/Cargo.toml +```toml +.... +promet = { package = "substrate-prometheus", path="../../core/prometheus"} +.... +``` + +**Metrics builder as same as substrate-telemetry** +client/service/src/builder.rs +```rust + + ..... + let _ = to_spawn_tx.unbounded_send(Box::new(future + .select(exit.clone()) + .then(|_| Ok(())))); + telemetry + }); +---------------- +match config.prometheus_endpoint { + None => (), + Some(x) => {let _prometheus = promet::init_prometheus(x);} + } + +------------------- + Ok(NewService { + client, + network, + ..... +``` +substrate/Cargo.toml +```toml +<<<<<<< HEAD +<<<<<<< HEAD + .... +[workspace] +members = [ + "client/prometheus", + .... +======= +[workspace] +members = [ + "client/prometheus", +.... +>>>>>>> ed9d9a5fa... README.md add +======= + .... +[workspace] +members = [ + "client/prometheus", + .... +>>>>>>> 0c82bd7d8... README.md +``` +### CLI Config +client/cli/src/lib.rs +```rust +fn crate_run_node_config +... +} + + match cli.prometheus_endpoint { + None => {config.prometheus_endpoint = None;}, + Some(x) => { + config.prometheus_endpoint = Some(parse_address(&format!("{}:{}", x, 33333), cli.prometheus_port)?); + } + } +... + +``` + +client/cli/src/params.rs +```rust +pub struct RunCmd{ +/// Specify HTTP RPC server TCP port. +#[structopt(long = "prometheus-port", value_name = "PORT")] + pub prometheus_port: Option, +#[structopt(long = "prometheus-addr", value_name = "Local IP address")] + pub prometheus_endpoint: Option, +} +``` +client/service/src/config.rs +```rust +#[derive(Clone)] +pub struct Configuration { + ... + pub prometheus_endpoint: Option, + ... +} +impl Configuration where + C: Default, + G: RuntimeGenesis, + E: Extension, +{ + /// Create default config for given chain spec. + pub fn default_with_spec(chain_spec: ChainSpec) -> Self { + let mut configuration = Configuration { + ... + prometheus_endpoints: None, + ... + }; + configuration.network.boot_nodes = configuration.chain_spec.boot_nodes().to_vec(); + + configuration.telemetry_endpoints = configuration.chain_spec.telemetry_endpoints().clone(); + + configuration + } +``` + + + +### Metrics Add +ex) consensus_FINALITY_HEIGHT + +client/prometheus/src/metrics.rs + +```rust +pub use crate::*; + +pub fn try_create_int_gauge(name: &str, help: &str) -> Result { + let opts = Opts::new(name, help); + let gauge = IntGauge::with_opts(opts)?; + prometheus::register(Box::new(gauge.clone()))?; + Ok(gauge) +} + +pub fn set_gauge(gauge: &Result, value: u64) { + if let Ok(gauge) = gauge { + gauge.set(value as i64); + } +} + +lazy_static! { + pub static ref FINALITY_HEIGHT: Result = try_create_int_gauge( + "consensus_finality_block_height_number", + "block is finality HEIGHT" + + ); +} +``` +client/service/Cargo.toml +```rust +... +promet = { package = "substrate-prometheus", path="../prometheus"} +... +``` +client/service/src/builder.rs +```rust +..... +use promet::{prometheus_gauge}; +..... + let tel_task = state_rx.for_each(move |(net_status, _)| { + let info = client_.info(); + let best_number = info.chain.best_number.saturated_into::(); + let best_hash = info.chain.best_hash; + let num_peers = net_status.num_connected_peers; + let txpool_status = transaction_pool_.status(); + let finalized_number: u64 = info.chain.finalized_number.saturated_into::(); + let bandwidth_download = net_status.average_download_per_sec; + let bandwidth_upload = net_status.average_upload_per_sec; + + let used_state_cache_size = match info.used_state_cache_size { + Some(size) => size, + None => 0, + }; + + // get cpu usage and memory usage of this process + let (cpu_usage, memory) = if let Some(self_pid) = self_pid { + if sys.refresh_process(self_pid) { + let proc = sys.get_process(self_pid) + .expect("Above refresh_process succeeds, this should be Some(), qed"); + (proc.cpu_usage(), proc.memory()) + } else { (0.0, 0) } + } else { (0.0, 0) }; + + telemetry!( + SUBSTRATE_INFO; + "system.interval"; + "peers" => num_peers, + "height" => best_number, + "best" => ?best_hash, + "txcount" => txpool_status.ready, + "cpu" => cpu_usage, + "memory" => memory, + "finalized_height" => finalized_number, + "finalized_hash" => ?info.chain.finalized_hash, + "bandwidth_download" => bandwidth_download, + "bandwidth_upload" => bandwidth_upload, + "used_state_cache_size" => used_state_cache_size, + ); + + prometheus_gauge!( + MEMPOOL_SIZE => used_state_cache_size as u64, + NODE_MEMORY => memory as u64, + NODE_CPU => cpu_usage as u64, + TX_COUNT => txpool_status.ready as u64, + FINALITY_HEIGHT => finalized_number as u64, + BEST_HEIGHT => best_number as u64, + P2P_PEERS_NUM => num_peers as u64, + P2P_NODE_DOWNLOAD => net_status.average_download_per_sec as u64, + P2P_NODE_UPLOAD => net_status.average_upload_per_sec as u64 + ); +..... +``` +## Metrics + +substrate can report and serve the Prometheus metrics, which in their turn can be consumed by Prometheus collector(s). + +This functionality is disabled by default. + +<<<<<<< HEAD +<<<<<<< HEAD +<<<<<<< HEAD +======= +To enable the Prometheus metrics, set in your cli command (--prometheus-addr,--prometheus-port ). +Metrics will be served under /metrics on 33333 port by default. +>>>>>>> 9b05a7e8f... README.md + +### List of available metrics + +<<<<<<< HEAD +| **Name** | **Type** | **Tags** | **Description** | +|-----------------------------------------|-----------|----------|-----------------------------------------------------------------| +| consensus_FINALITY_HEIGHT | IntGauge | | finality Height of the chain | +| consensus_BEST_HEIGHT | IntGauge | | best Height of the chain | +| consensus_TARGET_HEIGHT | IntGauge | | syning Height target number | +<<<<<<< HEAD +| consensus_validators_power | IntGauge | | Total voting power of all validators | +<<<<<<< HEAD +======= +Application metrics, namespace: `substrate` + +| **Name** | **Type** | **Tags** | **Description** | +| ---- | ---- | ---- | ----------- | +| module_stake_bonded_token | Gauge | validator_address | Total bonded token by validator | +| module_stake_loosen_token | Gauge | | Total loose tokens | +| module_stake_burned_token | Gauge | | Total burned token | +| module_stake_slashed_token | Counter | validator_address | Total slashed token by validator | +| module_stake_jailed | Gauge | validator_address | Jailed status by validator, either 0 (not jailed) or 1 (jailed) | +| module_stake_power | Gauge | validator_address | Voting power by validator | +| module_upgrade_upgrade | Gauge | | Whether new software needs to be installed, either 0 (no) or 1 (yes) | +| module_upgrade_signal | Gauge | validator_address, version | Whether validator have run the new version software, either 0 (no) or 1 (yes)| +| module_service_active_requests | Gauge | | Number of active requests | +| module_gov_parameter | Gauge | parameter_key | Parameter of governance | +| module_gov_proposal_status | Gauge | proposal_id | Status of proposal, 0:DepositPeriod 1:VotingPeriod 2:Pass 3:Reject 4:Other | +| module_gov_vote | Gauge | proposal_id, validator_address | Validator vote result of a proposal, 0:Yes 1:No 2:NoWithVeto 3:Abstain | +| module_distribution_community_tax | Gauge | height | Community tax accumulation | +| v0_invariant_failure | counter | error | Invariant failure stats | + +Consensus metrics, namespace: `tendermint` +======= + +Consensus metrics, namespace: `substrate` +>>>>>>> 216ea4fbc... README.md add + +| **Name** | **Type** | **Tags** | **Description** | +|-----------------------------------------|-----------|----------|-----------------------------------------------------------------| +| consensus_height | Gauge | | Height of the chain | +| consensus_failure | counter | height | Consensus failure | +| consensus_validators | Gauge | | Number of validators | +| consensus_validators_power | Gauge | | Total voting power of all validators | +>>>>>>> ed9d9a5fa... README.md add +======= +>>>>>>> 0c82bd7d8... README.md +| consensus_missing_validators | Gauge | | Number of validators who did not sign | +| consensus_missing_validators_power | Gauge | | Total voting power of the missing validators | +| consensus_byzantine_validators | Gauge | | Number of validators who tried to double sign | +| consensus_byzantine_validators_power | Gauge | | Total voting power of the byzantine validators | +| consensus_block_interval_seconds | Histogram | | Time between this and last block (Block.Header.Time) in seconds | +| consensus_rounds | Gauge | | Number of rounds | +| consensus_num_txs | Gauge | | Number of transactions | +<<<<<<< HEAD +<<<<<<< HEAD +| consensus_block_parts | counter | peer_id | number of blockparts transmitted by peer | +| consensus_latest_block_height | gauge | | /status sync_info number | +| consensus_fast_syncing | gauge | | either 0 (not fast syncing) or 1 (syncing) | +| consensus_total_txs | Gauge | | Total number of transactions committed | +| consensus_block_size_bytes | Gauge | | Block size in bytes | +| p2p_peers_number | IntGauge | | Number of peers node's connected to | +| p2p_peer_receive_bytes_total | IntGauge | peer_id | number of bytes received from a given peer | +| p2p_peer_send_bytes_total | IntGauge | peer_id | number of bytes sent to a given peer | +| p2p_peer_pending_send_bytes | gauge | peer_id | number of pending bytes to be sent to a given peer | +| p2p_num_txs | gauge | peer_id | number of transactions submitted by each peer_id | +======= +| consensus_block_parts | counter | peer_id | number of blockparts transmitted by peer | +| consensus_latest_block_height | gauge | | /status sync_info number | +| consensus_fast_syncing | gauge | | either 0 (not fast syncing) or 1 (syncing) | +| consensus_total_txs | Gauge | | Total number of transactions committed | +| consensus_block_size_bytes | Gauge | | Block size in bytes | +| p2p_peers | Gauge | | Number of peers node's connected to | +| p2p_peer_receive_bytes_total | counter | peer_id | number of bytes received from a given peer | +| p2p_peer_send_bytes_total | counter | peer_id | number of bytes sent to a given peer | +| p2p_peer_pending_send_bytes | gauge | peer_id | number of pending bytes to be sent to a given peer | +| p2p_num_txs | gauge | peer_id | number of transactions submitted by each peer_id | +>>>>>>> ed9d9a5fa... README.md add +======= +| consensus_block_parts | counter | peer_id | number of blockparts transmitted by peer | +| consensus_latest_block_height | gauge | | /status sync_info number | +| consensus_fast_syncing | gauge | | either 0 (not fast syncing) or 1 (syncing) | +| consensus_total_txs | Gauge | | Total number of transactions committed | +| consensus_block_size_bytes | Gauge | | Block size in bytes | +| p2p_peers | IntGauge | | Number of peers node's connected to | +| p2p_peer_receive_bytes_total | counter | peer_id | number of bytes received from a given peer | +| p2p_peer_send_bytes_total | counter | peer_id | number of bytes sent to a given peer | +| p2p_peer_pending_send_bytes | gauge | peer_id | number of pending bytes to be sent to a given peer | +| p2p_num_txs | gauge | peer_id | number of transactions submitted by each peer_id | +>>>>>>> 0c82bd7d8... README.md +| mempool_size | Gauge | | Number of uncommitted transactions | +| mempool_tx_size_bytes | histogram | | transaction sizes in bytes | +| mempool_failed_txs | counter | | number of failed transactions | +| mempool_recheck_times | counter | | number of transactions rechecked in the mempool | +| state_block_processing_time | histogram | | time between BeginBlock and EndBlock in ms | +<<<<<<< HEAD +<<<<<<< HEAD +| state_recheck_time | histogram | | time cost on recheck in ms | +| state_app_hash_conflict | count | proposer, height | App hash conflict error | +======= +| consensus_validators_power(Add in the future) | IntGauge | | Total voting power of all validators | +| consensus_missing_validators(Add in the future) | Gauge | | Number of validators who did not sign | +| consensus_missing_validators_power(Add in the future) | Gauge | | Total voting power of the missing validators | +| consensus_byzantine_validators(Add in the future) | Gauge | | Number of validators who tried to double sign | +| consensus_byzantine_validators_power(Add in the future) | Gauge | | Total voting power of the byzantine validators | +| consensus_block_interval_seconds(Add in the future) | Histogram | | Time between this and last block (Block.Header.Time) in seconds | +| consensus_rounds(Add in the future) | Gauge | | Number of rounds | +| consensus_num_txs(Add in the future) | Gauge | | Number of transactions | +| consensus_block_parts(Add in the future) | counter | | number of blockparts transmitted by peer | +| consensus_total_txs(Add in the future) | Gauge | | Total number of transactions committed | +| consensus_block_size_bytes(Add in the future) | Gauge | | Block size in bytes | +| p2p_peers_number | IntGauge | | Number of peers node's connected to | +| p2p_peer_receive_bytes_total | IntGauge | | number of bytes received from a given peer | +| p2p_peer_send_bytes_total | IntGauge | | number of bytes sent to a given peer | +| mempool_size(Add in the future) | Gauge | | Number of uncommitted transactions | +| mempool_tx_size_bytes(Add in the future) | histogram | | transaction sizes in bytes | +| mempool_failed_txs(Add in the future) | counter | | number of failed transactions | +| mempool_recheck_times(Add in the future) | counter | | number of transactions rechecked in the mempool | +| state_block_processing_time(Add in the future) | histogram | | time between BeginBlock and EndBlock in ms | +| state_recheck_time(Add in the future) | histogram | | time cost on recheck in ms | +| state_app_hash_conflict(Add in the future) | count | | App hash conflict error | +>>>>>>> 9672809d2... README.md update +======= +>>>>>>> 9b05a7e8f... README.md + +Consensus metrics, namespace: `substrate` + +| **Name** | **Type** | **Tags** | **Description** | +| -------------------------------------- | --------- | -------- | --------------------------------------------------------------- | +| consensus_finality_block_height_number | IntGauge | | finality Height of the chain | +| consensus_best_block_height_number | IntGauge | | best Height of the chain | +| consensus_target_syn_number | IntGauge | | syning Height target number | +| consensus_block_interval_seconds | Histogram | | Time between this and last block (Block.Header.Time) in seconds | +| consensus_num_txs | Gauge | | Number of transactions | +| consensus_node_memory | IntGauge | | Node's primary memory | +| consensus_node_cpu | IntGauge | | Node's cpu load | +| p2p_peers_number | IntGauge | | Number of peers node's connected to | +| p2p_peer_receive_bytes_per_sec | IntGauge | | number of bytes received from a given peer | +| p2p_peer_send_bytes_per_sec | IntGauge | | number of bytes sent to a given peer | +| mempool_size | IntGauge | | Number of uncommitted transactions | + + +## Start Prometheus +### Install prometheus + +https://prometheus.io/download/ +```bash +wget +tar -zxvf +``` +======= +| state_recheck_time | histogram | | time cost on recheck in ms | +| state_app_hash_conflict | count | proposer, height | App hash conflict error | +======= +| state_recheck_time | histogram | | time cost on recheck in ms | +| state_app_hash_conflict | count | proposer, height | App hash conflict error | +>>>>>>> 0c82bd7d8... README.md + + + +## Start Prometheus +<<<<<<< HEAD +>>>>>>> ed9d9a5fa... README.md add +======= +### Install prometheus + +https://prometheus.io/download/ +```bash +wget +tar -zxvf +``` +>>>>>>> 0c82bd7d8... README.md + +### Edit Prometheus config file + +You can visit [prometheus.yml](https://github.com/prometheus/prometheus/blob/master/documentation/examples/prometheus.yml) to download default `prometheus.yml`. + +Then edit `prometheus.yml` and add `jobs` : + +```yaml +<<<<<<< HEAD +<<<<<<< HEAD + - job_name: kusama + static_configs: + - targets: ['localhost:33333'] + labels: + instance: local-validator +``` + +> Note：value of targets is ip:port which used by substrate monitor +======= + - job_name: ku +======= + - job_name: kusama +>>>>>>> 216ea4fbc... README.md add + static_configs: + - targets: ['localhost:33333'] + labels: + instance: local-validator +``` + +<<<<<<< HEAD +> Note：value of targets is ip:port which used by IRIS monitor +>>>>>>> ed9d9a5fa... README.md add +======= +> Note：value of targets is ip:port which used by substrate monitor +>>>>>>> 216ea4fbc... README.md add + +### Start Prometheus + +```bash +<<<<<<< HEAD +<<<<<<< HEAD +cd +./prometheus +======= +docker run -d --name=prometheus -p 9090:9090 -v ~/volumes/prometheus:/etc/prometheus prom/prometheus +>>>>>>> ed9d9a5fa... README.md add +======= +cd +./prometheus +>>>>>>> 0c82bd7d8... README.md +``` + +> The above example, you can save `prometheus.yml` at `~/volumes/prometheus` on your host machine + +You can visit `http://localhost:9090` to see prometheus data. + +<<<<<<< HEAD +<<<<<<< HEAD + + +## Start Grafana +### Install Grafana +https://grafana.com/docs/installation/debian/ + +```bash +apt-get install -y software-properties-common +sudo add-apt-repository "deb https://packages.grafana.com/oss/deb stable main" +wget -q -O - https://packages.grafana.com/gpg.key | sudo apt-key add - +sudo apt-get update +sudo apt-get install grafana +sudo service grafana-server start +./prometheus +======= +======= + + +>>>>>>> 0c82bd7d8... README.md +## Start Grafana +### Install Grafana +https://grafana.com/docs/installation/debian/ + +<<<<<<< HEAD +``` +docker run -d --name=grafana -p 3000:3000 grafana/grafana +>>>>>>> ed9d9a5fa... README.md add +======= +```bash +apt-get install -y software-properties-common +sudo add-apt-repository "deb https://packages.grafana.com/oss/deb stable main" +wget -q -O - https://packages.grafana.com/gpg.key | sudo apt-key add - +sudo apt-get update +sudo apt-get install grafana +sudo service grafana-server start +./prometheus +>>>>>>> 0c82bd7d8... README.md +``` + +You can visit `http://localhost:3000/` to open grafana and create your own dashboard. + +<<<<<<< HEAD +<<<<<<< HEAD +======= +>>>>>>> 0c82bd7d8... README.md +> Tips: The default username and password are both admin. We strongly recommend immediately changing your username & password after login + +### Seting Grafana + +<<<<<<< HEAD +<<<<<<< HEAD +Default ID:PW is admin. +======= +> Tips: The default username and password are both admin. We strongly recommend immediately changing your username & password after login +>>>>>>> ed9d9a5fa... README.md add +======= +Default ID:PW is admin. +>>>>>>> 0c82bd7d8... README.md +======= +Default ID:PW is admin. +>>>>>>> 9b05a7e8f... README.md diff --git a/client/prometheus/src/lib.rs b/client/prometheus/src/lib.rs new file mode 100644 index 0000000000000..e76f2e2d847f1 --- /dev/null +++ b/client/prometheus/src/lib.rs @@ -0,0 +1,116 @@ +#[macro_use] +extern crate lazy_static; +#[macro_use] +extern crate log; +use hyper::http::StatusCode; +use hyper::rt::Future; +use hyper::service::service_fn_ok; +use hyper::{Body, Request, Response, Server}; +pub use prometheus::{Encoder, HistogramOpts, Opts, TextEncoder}; +pub use prometheus::{Histogram, IntCounter, IntGauge, Result}; +pub use sr_primitives::traits::SaturatedConversion; +use std::net::SocketAddr; + +pub mod metrics; + +/// Initializes the metrics context, and starts an HTTP server +/// to serve metrics. +pub fn init_prometheus(prometheus_addr: SocketAddr) { + let addr = prometheus_addr; + let server = Server::bind(&addr) + .serve(|| { + // This is the `Service` that will handle the connection. + // `service_fn_ok` is a helper to convert a function that + // returns a Response into a `Service`. + service_fn_ok(move |req: Request| { + if req.uri().path() == "/metrics" { + let metric_families = prometheus::gather(); + let mut buffer = vec![]; + let encoder = TextEncoder::new(); + encoder.encode(&metric_families, &mut buffer).unwrap(); + Response::builder() + .status(StatusCode::OK) + .header("Content-Type", encoder.format_type()) + .body(Body::from(buffer)) + .expect("Sends OK(200) response with one or more data metrics") + } else { + Response::builder() + .status(StatusCode::NOT_FOUND) + .body(Body::from("Not found.")) + .expect("Sends NOT_FOUND(404) message with no data metric") + } + }) + }) + .map_err(|e| error!("server error: {}", e)); + + info!("Exporting metrics at http://{}/metrics", addr); + + let mut rt = tokio::runtime::Builder::new() + .core_threads(1) // one thread is sufficient + .build() + .expect("Builds one thread of tokio runtime exporter for prometheus"); + + std::thread::spawn(move || { + rt.spawn(server); + rt.shutdown_on_idle().wait().unwrap(); + }); +} + +#[macro_export] +macro_rules! prometheus_gauge( + ($($metric:expr => $value:expr),*) => { + use $crate::{metrics::*}; + $( + metrics::set_gauge(&$metric, $value); + )* + } +); + +#[macro_export] +macro_rules! prometheus_histogram( + ($($metric:expr => $value:expr),*) => { + use $crate::{metrics::*}; + $( + metrics::set_histogram(&$metric, $value); + )* + } +); + +<<<<<<< HEAD +<<<<<<< HEAD +#[macro_export] +macro_rules! prometheus_counter( + ($($metric:expr => $value:expr),*) => { + use $crate::{metrics::*}; + $( + metrics::set_counter(&$metric, $value); + )* + } +<<<<<<< HEAD +); +======= +); + +======= +>>>>>>> 4882334f7... fix: refactor prometheus-module +/* +TODO: Make abstract type for all metrics(e.g. Gauge, Histogram, Counter) with generic traits so that all metrics can be set up with one function `set` +#[macro_export] +macro_rules! prometheus( + ($($a: expr; $metric:expr => $value:expr),*) => { + use $crate::{metrics::*}; + $( + metrics::set(#$a, &$metric, $value); + )* + } +); +<<<<<<< HEAD +>>>>>>> aec43ec74... add prometheus command +======= +*/ +<<<<<<< HEAD +>>>>>>> 41a176425... chore: update README +======= +>>>>>>> 5842caed0... fix: justification and newline at EOF +======= +>>>>>>> 4882334f7... fix: refactor prometheus-module diff --git a/client/prometheus/src/metrics.rs b/client/prometheus/src/metrics.rs new file mode 100644 index 0000000000000..4215cf9cec675 --- /dev/null +++ b/client/prometheus/src/metrics.rs @@ -0,0 +1,77 @@ +pub use crate::*; + +pub fn try_create_int_gauge(name: &str, help: &str) -> Result { + let opts = Opts::new(name, help); + let gauge = IntGauge::with_opts(opts)?; + prometheus::register(Box::new(gauge.clone()))?; + Ok(gauge) +} + +pub fn try_create_histogram(name: &str, help: &str) -> Result { + let opts = HistogramOpts::new(name, help); + let histogram = Histogram::with_opts(opts)?; + prometheus::register(Box::new(histogram.clone()))?; + Ok(histogram) +} + +pub fn set_gauge(gauge: &Result, value: u64) { + if let Ok(gauge) = gauge { + gauge.set(value as i64); + } +} + +pub fn set_histogram(histogram: &Result, value: f64) { + if let Ok(histogram) = histogram { + histogram.observe(value) + } +} + +lazy_static! { + pub static ref FINALITY_HEIGHT: Result = try_create_int_gauge( + "consensus_finality_block_height_number", + "block is finality HEIGHT" + + ); + pub static ref BEST_HEIGHT: Result = try_create_int_gauge( + "consensus_best_block_height_number", + "block is best HEIGHT" + ); + + pub static ref BLOCK_INTERVAL_SECONDS: Result = try_create_histogram( + "consensus_block_interval_seconds", + "Time between this and last block(Block.Header.Time) in seconds" + ); + pub static ref P2P_PEERS_NUM: Result = try_create_int_gauge( + "p2p_peers_number", + "network gosip peers number" + ); + pub static ref TARGET_NUM: Result = try_create_int_gauge( + "consensus_target_syn_number", + "block syn target number" + ); + pub static ref TX_COUNT: Result = try_create_int_gauge( + "consensus_num_txs", + "Number of transactions" + ); + pub static ref NODE_MEMORY: Result = try_create_int_gauge( + "consensus_node_memory", + "node memory" + ); + pub static ref NODE_CPU: Result = try_create_int_gauge( + "consensus_node_cpu", + "node cpu" + ); + pub static ref MEMPOOL_SIZE: Result = try_create_int_gauge( + "mempool_size", + "Number of uncommitted transactions" + ); + //net_status.average_download_per_sec + pub static ref P2P_NODE_DOWNLOAD: Result = try_create_int_gauge( + "p2p_peer_receive_byte_per_sec", + "p2p_node_download_per_sec_byte" + ); + pub static ref P2P_NODE_UPLOAD: Result = try_create_int_gauge( + "p2p_peer_send_byte_per_sec", + "p2p_node_upload_per_sec_byte" + ); +} diff --git a/client/rpc-servers/Cargo.toml b/client/rpc-servers/Cargo.toml new file mode 100644 index 0000000000000..c3b9802f567cf --- /dev/null +++ b/client/rpc-servers/Cargo.toml @@ -0,0 +1,17 @@ +[package] +name = "substrate-rpc-servers" +version = "2.0.0" +authors = ["Parity Technologies "] +edition = "2018" + +[dependencies] +jsonrpc-core = "14.0.3" +pubsub = { package = "jsonrpc-pubsub", version = "14.0.3" } +log = "0.4.8" +serde = "1.0.101" +serde_json = "1.0.41" +sr-primitives = { path = "../../primitives/sr-primitives" } + +[target.'cfg(not(target_os = "unknown"))'.dependencies] +http = { package = "jsonrpc-http-server", version = "14.0.3" } +ws = { package = "jsonrpc-ws-server", version = "14.0.3" } diff --git a/core/rpc-servers/src/lib.rs b/client/rpc-servers/src/lib.rs similarity index 100% rename from core/rpc-servers/src/lib.rs rename to client/rpc-servers/src/lib.rs diff --git a/core/rpc/Cargo.toml b/client/rpc/Cargo.toml similarity index 59% rename from core/rpc/Cargo.toml rename to client/rpc/Cargo.toml index 85998feb1b887..4bd0f84b5f029 100644 --- a/core/rpc/Cargo.toml +++ b/client/rpc/Cargo.toml @@ -6,19 +6,21 @@ edition = "2018" [dependencies] api = { package = "substrate-rpc-api", path = "./api" } -client = { package = "substrate-client", path = "../client" } +client-api = { package = "substrate-client-api", path = "../api" } +client = { package = "substrate-client", path = "../" } +sr-api = { path = "../../primitives/sr-api" } codec = { package = "parity-scale-codec", version = "1.0.0" } futures03 = { package = "futures-preview", version = "0.3.0-alpha.19", features = ["compat"] } -jsonrpc-pubsub = "13.1.0" +jsonrpc-pubsub = "14.0.3" log = "0.4.8" -primitives = { package = "substrate-primitives", path = "../primitives" } -rpc = { package = "jsonrpc-core", version = "13.0.0" } -runtime_version = { package = "sr-version", path = "../sr-version" } +primitives = { package = "substrate-primitives", path = "../../primitives/core" } +rpc = { package = "jsonrpc-core", version = "14.0.3" } +runtime_version = { package = "sr-version", path = "../../primitives/sr-version" } serde_json = "1.0.41" -session = { package = "substrate-session", path = "../session" } -sr-primitives = { path = "../sr-primitives" } -rpc-primitives = { package = "substrate-rpc-primitives", path = "primitives" } -state_machine = { package = "substrate-state-machine", path = "../state-machine" } +session = { package = "substrate-session", path = "../../primitives/session" } +sr-primitives = { path = "../../primitives/sr-primitives" } +rpc-primitives = { package = "substrate-rpc-primitives", path = "../../primitives/rpc" } +state_machine = { package = "substrate-state-machine", path = "../../primitives/state-machine" } substrate-executor = { path = "../executor" } substrate-keystore = { path = "../keystore" } transaction_pool = { package = "substrate-transaction-pool", path = "../transaction-pool" } @@ -30,6 +32,6 @@ assert_matches = "1.3.0" futures = "0.1.29" network = { package = "substrate-network", path = "../network" } rustc-hex = "2.0.1" -sr-io = { path = "../sr-io" } -test-client = { package = "substrate-test-runtime-client", path = "../test-runtime/client" } +sr-io = { path = "../../primitives/sr-io" } +test-client = { package = "substrate-test-runtime-client", path = "../../test/utils/runtime/client" } tokio = "0.1.22" diff --git a/core/rpc/api/Cargo.toml b/client/rpc/api/Cargo.toml similarity index 65% rename from core/rpc/api/Cargo.toml rename to client/rpc/api/Cargo.toml index bccafc2a85dd7..053e58d031548 100644 --- a/core/rpc/api/Cargo.toml +++ b/client/rpc/api/Cargo.toml @@ -8,15 +8,15 @@ edition = "2018" codec = { package = "parity-scale-codec", version = "1.0.0" } derive_more = "0.15.0" futures03 = { package = "futures-preview", version = "0.3.0-alpha.19", features = ["compat"] } -jsonrpc-core = "13.2.0" -jsonrpc-core-client = "13.2.0" -jsonrpc-derive = "13.2.0" -jsonrpc-pubsub = "13.2.0" +jsonrpc-core = "14.0.3" +jsonrpc-core-client = "14.0.3" +jsonrpc-derive = "14.0.3" +jsonrpc-pubsub = "14.0.3" log = "0.4.8" parking_lot = "0.9.0" -primitives = { package = "substrate-primitives", path = "../../primitives" } -runtime_version = { package = "sr-version", path = "../../sr-version" } +primitives = { package = "substrate-primitives", path = "../../../primitives/core" } +runtime_version = { package = "sr-version", path = "../../../primitives/sr-version" } serde = { version = "1.0.101", features = ["derive"] } serde_json = "1.0.41" txpool = { package = "substrate-transaction-graph", path = "../../transaction-pool/graph" } -rpc-primitives = { package = "substrate-rpc-primitives", path = "../../rpc/primitives" } +rpc-primitives = { package = "substrate-rpc-primitives", path = "../../../primitives/rpc" } diff --git a/core/rpc/api/src/author/error.rs b/client/rpc/api/src/author/error.rs similarity index 100% rename from core/rpc/api/src/author/error.rs rename to client/rpc/api/src/author/error.rs diff --git a/core/rpc/api/src/author/hash.rs b/client/rpc/api/src/author/hash.rs similarity index 100% rename from core/rpc/api/src/author/hash.rs rename to client/rpc/api/src/author/hash.rs diff --git a/core/rpc/api/src/author/mod.rs b/client/rpc/api/src/author/mod.rs similarity index 100% rename from core/rpc/api/src/author/mod.rs rename to client/rpc/api/src/author/mod.rs diff --git a/core/rpc/api/src/chain/error.rs b/client/rpc/api/src/chain/error.rs similarity index 100% rename from core/rpc/api/src/chain/error.rs rename to client/rpc/api/src/chain/error.rs diff --git a/core/rpc/api/src/chain/mod.rs b/client/rpc/api/src/chain/mod.rs similarity index 100% rename from core/rpc/api/src/chain/mod.rs rename to client/rpc/api/src/chain/mod.rs diff --git a/core/rpc/api/src/errors.rs b/client/rpc/api/src/errors.rs similarity index 100% rename from core/rpc/api/src/errors.rs rename to client/rpc/api/src/errors.rs diff --git a/core/rpc/api/src/helpers.rs b/client/rpc/api/src/helpers.rs similarity index 100% rename from core/rpc/api/src/helpers.rs rename to client/rpc/api/src/helpers.rs diff --git a/core/rpc/api/src/lib.rs b/client/rpc/api/src/lib.rs similarity index 100% rename from core/rpc/api/src/lib.rs rename to client/rpc/api/src/lib.rs diff --git a/core/rpc/api/src/state/error.rs b/client/rpc/api/src/state/error.rs similarity index 100% rename from core/rpc/api/src/state/error.rs rename to client/rpc/api/src/state/error.rs diff --git a/core/rpc/api/src/state/mod.rs b/client/rpc/api/src/state/mod.rs similarity index 100% rename from core/rpc/api/src/state/mod.rs rename to client/rpc/api/src/state/mod.rs diff --git a/core/rpc/api/src/subscriptions.rs b/client/rpc/api/src/subscriptions.rs similarity index 100% rename from core/rpc/api/src/subscriptions.rs rename to client/rpc/api/src/subscriptions.rs diff --git a/core/rpc/api/src/system/error.rs b/client/rpc/api/src/system/error.rs similarity index 100% rename from core/rpc/api/src/system/error.rs rename to client/rpc/api/src/system/error.rs diff --git a/core/rpc/api/src/system/helpers.rs b/client/rpc/api/src/system/helpers.rs similarity index 100% rename from core/rpc/api/src/system/helpers.rs rename to client/rpc/api/src/system/helpers.rs diff --git a/core/rpc/api/src/system/mod.rs b/client/rpc/api/src/system/mod.rs similarity index 100% rename from core/rpc/api/src/system/mod.rs rename to client/rpc/api/src/system/mod.rs diff --git a/core/rpc/src/author/mod.rs b/client/rpc/src/author/mod.rs similarity index 93% rename from core/rpc/src/author/mod.rs rename to client/rpc/src/author/mod.rs index 82122dcf3d21f..e0d7cc10d7890 100644 --- a/core/rpc/src/author/mod.rs +++ b/client/rpc/src/author/mod.rs @@ -23,7 +23,9 @@ use std::{sync::Arc, convert::TryInto}; use futures03::future::{FutureExt, TryFutureExt}; use log::warn; -use client::{self, Client}; +use client::Client; +use client_api::error::Error as ClientError; + use rpc::futures::{ Sink, Future, future::result, @@ -80,14 +82,15 @@ impl Author where P: PoolChainApi + Sync + Send + 'sta } impl AuthorApi, BlockHash

> for Author where - B: client::backend::Backend<

::Block, Blake2Hasher> + Send + Sync + 'static, - E: client::CallExecutor<

::Block, Blake2Hasher> + Send + Sync + 'static, + B: client_api::backend::Backend<

::Block, Blake2Hasher> + Send + Sync + 'static, + E: client_api::CallExecutor<

::Block, Blake2Hasher> + Send + Sync + 'static, P: PoolChainApi + Sync + Send + 'static, P::Block: traits::Block, P::Error: 'static, RA: Send + Sync + 'static, Client: ProvideRuntimeApi, - as ProvideRuntimeApi>::Api: SessionKeys, + as ProvideRuntimeApi>::Api: + SessionKeys, { type Metadata = crate::metadata::Metadata; @@ -131,8 +134,9 @@ impl AuthorApi, BlockHash

> for Author whe Ok(self.pool.ready().map(|tx| tx.data.encode().into()).collect()) } - fn remove_extrinsic(&self, - bytes_or_hash: Vec>> + fn remove_extrinsic( + &self, + bytes_or_hash: Vec>>, ) -> Result>> { let hashes = bytes_or_hash.into_iter() .map(|x| match x { @@ -155,7 +159,7 @@ impl AuthorApi, BlockHash

> for Author whe fn watch_extrinsic(&self, _metadata: Self::Metadata, subscriber: Subscriber, BlockHash

>>, - xt: Bytes + xt: Bytes, ) { let submit = || -> Result<_> { let best_block_hash = self.client.info().chain.best_hash; diff --git a/core/rpc/src/author/tests.rs b/client/rpc/src/author/tests.rs similarity index 100% rename from core/rpc/src/author/tests.rs rename to client/rpc/src/author/tests.rs diff --git a/core/rpc/src/chain/chain_full.rs b/client/rpc/src/chain/chain_full.rs similarity index 97% rename from core/rpc/src/chain/chain_full.rs rename to client/rpc/src/chain/chain_full.rs index ad359a9300cdd..945ed19838db2 100644 --- a/core/rpc/src/chain/chain_full.rs +++ b/client/rpc/src/chain/chain_full.rs @@ -20,7 +20,8 @@ use std::sync::Arc; use rpc::futures::future::result; use api::Subscriptions; -use client::{backend::Backend, CallExecutor, Client}; +use client_api::{CallExecutor, backend::Backend}; +use client::Client; use primitives::{H256, Blake2Hasher}; use sr_primitives::{ generic::{BlockId, SignedBlock}, diff --git a/core/rpc/src/chain/chain_light.rs b/client/rpc/src/chain/chain_light.rs similarity index 97% rename from core/rpc/src/chain/chain_light.rs rename to client/rpc/src/chain/chain_light.rs index d969d6ca93702..5c2f1f803a438 100644 --- a/core/rpc/src/chain/chain_light.rs +++ b/client/rpc/src/chain/chain_light.rs @@ -68,7 +68,7 @@ impl> LightChain impl ChainBackend for LightChain where Block: BlockT + 'static, - B: client::backend::Backend + Send + Sync + 'static, + B: client_api::backend::Backend + Send + Sync + 'static, E: client::CallExecutor + Send + Sync + 'static, RA: Send + Sync + 'static, F: Fetcher + Send + Sync + 'static, diff --git a/core/rpc/src/chain/mod.rs b/client/rpc/src/chain/mod.rs similarity index 95% rename from core/rpc/src/chain/mod.rs rename to client/rpc/src/chain/mod.rs index 61ecf96711383..e846f7b59c441 100644 --- a/core/rpc/src/chain/mod.rs +++ b/client/rpc/src/chain/mod.rs @@ -51,7 +51,7 @@ pub use api::chain::*; trait ChainBackend: Send + Sync + 'static where Block: BlockT + 'static, - B: client::backend::Backend + Send + Sync + 'static, + B: client_api::backend::Backend + Send + Sync + 'static, E: client::CallExecutor + Send + Sync + 'static, { /// Get client reference. @@ -156,7 +156,7 @@ pub fn new_full( ) -> Chain where Block: BlockT + 'static, - B: client::backend::Backend + Send + Sync + 'static, + B: client_api::backend::Backend + Send + Sync + 'static, E: client::CallExecutor + Send + Sync + 'static + Clone, RA: Send + Sync + 'static, { @@ -174,7 +174,7 @@ pub fn new_light>( ) -> Chain where Block: BlockT + 'static, - B: client::backend::Backend + Send + Sync + 'static, + B: client_api::backend::Backend + Send + Sync + 'static, E: client::CallExecutor + Send + Sync + 'static + Clone, RA: Send + Sync + 'static, F: Send + Sync + 'static, @@ -196,7 +196,7 @@ pub struct Chain { impl ChainApi, Block::Hash, Block::Header, SignedBlock> for Chain where Block: BlockT + 'static, - B: client::backend::Backend + Send + Sync + 'static, + B: client_api::backend::Backend + Send + Sync + 'static, E: client::CallExecutor + Send + Sync + 'static, RA: Send + Sync + 'static { @@ -245,7 +245,7 @@ fn subscribe_headers( stream: F, ) where Block: BlockT + 'static, - B: client::backend::Backend + Send + Sync + 'static, + B: client_api::backend::Backend + Send + Sync + 'static, E: client::CallExecutor + Send + Sync + 'static, F: FnOnce() -> S, G: FnOnce() -> Block::Hash, diff --git a/core/rpc/src/chain/tests.rs b/client/rpc/src/chain/tests.rs similarity index 100% rename from core/rpc/src/chain/tests.rs rename to client/rpc/src/chain/tests.rs diff --git a/core/rpc/src/lib.rs b/client/rpc/src/lib.rs similarity index 100% rename from core/rpc/src/lib.rs rename to client/rpc/src/lib.rs diff --git a/core/rpc/src/metadata.rs b/client/rpc/src/metadata.rs similarity index 100% rename from core/rpc/src/metadata.rs rename to client/rpc/src/metadata.rs diff --git a/core/rpc/src/state/mod.rs b/client/rpc/src/state/mod.rs similarity index 93% rename from core/rpc/src/state/mod.rs rename to client/rpc/src/state/mod.rs index b922601b0a5b2..8e15c488070fd 100644 --- a/core/rpc/src/state/mod.rs +++ b/client/rpc/src/state/mod.rs @@ -24,17 +24,10 @@ mod tests; use std::sync::Arc; use jsonrpc_pubsub::{typed::Subscriber, SubscriptionId}; -use rpc::{ - Result as RpcResult, - futures::Future, -}; +use rpc::{Result as RpcResult, futures::Future}; use api::Subscriptions; -use client::{ - Client, CallExecutor, - runtime_api::Metadata, - light::{blockchain::RemoteBlockchain, fetcher::Fetcher}, -}; +use client::{Client, CallExecutor, light::{blockchain::RemoteBlockchain, fetcher::Fetcher}}; use primitives::{ Blake2Hasher, Bytes, H256, storage::{StorageKey, StorageData, StorageChangeSet}, @@ -44,6 +37,8 @@ use sr_primitives::{ traits::{Block as BlockT, ProvideRuntimeApi}, }; +use sr_api::Metadata; + use self::error::{Error, FutureResult}; pub use api::state::*; @@ -52,7 +47,7 @@ pub use api::state::*; pub trait StateBackend: Send + Sync + 'static where Block: BlockT + 'static, - B: client::backend::Backend + Send + Sync + 'static, + B: client_api::backend::Backend + Send + Sync + 'static, E: client::CallExecutor + Send + Sync + 'static, RA: Send + Sync + 'static, { @@ -184,11 +179,12 @@ pub fn new_full( ) -> State where Block: BlockT + 'static, - B: client::backend::Backend + Send + Sync + 'static, + B: client_api::backend::Backend + Send + Sync + 'static, E: CallExecutor + Send + Sync + 'static + Clone, RA: Send + Sync + 'static, Client: ProvideRuntimeApi, - as ProvideRuntimeApi>::Api: Metadata, + as ProvideRuntimeApi>::Api: + Metadata, { State { backend: Box::new(self::state_full::FullState::new(client, subscriptions)), @@ -204,7 +200,7 @@ pub fn new_light>( ) -> State where Block: BlockT + 'static, - B: client::backend::Backend + Send + Sync + 'static, + B: client_api::backend::Backend + Send + Sync + 'static, E: CallExecutor + Send + Sync + 'static + Clone, RA: Send + Sync + 'static, F: Send + Sync + 'static, @@ -227,7 +223,7 @@ pub struct State { impl StateApi for State where Block: BlockT + 'static, - B: client::backend::Backend + Send + Sync + 'static, + B: client_api::backend::Backend + Send + Sync + 'static, E: CallExecutor + Send + Sync + 'static + Clone, RA: Send + Sync + 'static, { diff --git a/core/rpc/src/state/state_full.rs b/client/rpc/src/state/state_full.rs similarity index 97% rename from core/rpc/src/state/state_full.rs rename to client/rpc/src/state/state_full.rs index ff4c5e5599a8a..7c4382210ce8f 100644 --- a/core/rpc/src/state/state_full.rs +++ b/client/rpc/src/state/state_full.rs @@ -28,9 +28,9 @@ use rpc::{ }; use api::Subscriptions; +use client_api::{backend::Backend, error::Result as ClientResult}; use client::{ - Client, CallExecutor, BlockchainEvents, runtime_api::Metadata, - backend::Backend, error::Result as ClientResult, + Client, CallExecutor, BlockchainEvents, }; use primitives::{ H256, Blake2Hasher, Bytes, storage::{well_known_keys, StorageKey, StorageData, StorageChangeSet}, @@ -42,6 +42,8 @@ use sr_primitives::{ traits::{Block as BlockT, Header, NumberFor, ProvideRuntimeApi, SaturatedConversion}, }; +use sr_api::Metadata; + use super::{StateBackend, error::{FutureResult, Error, Result}, client_err}; /// Ranges to query in state_queryStorage. @@ -229,7 +231,8 @@ impl StateBackend for FullState + Send + Sync + 'static + Clone, RA: Send + Sync + 'static, Client: ProvideRuntimeApi, - as ProvideRuntimeApi>::Api: Metadata, + as ProvideRuntimeApi>::Api: + Metadata, { fn call( &self, @@ -326,7 +329,9 @@ impl StateBackend for FullState) -> FutureResult { Box::new(result( self.block_or_best(block) - .and_then(|block| self.client.runtime_api().metadata(&BlockId::Hash(block)).map(Into::into)) + .and_then(|block| + self.client.runtime_api().metadata(&BlockId::Hash(block)).map(Into::into) + ) .map_err(client_err))) } diff --git a/core/rpc/src/state/state_light.rs b/client/rpc/src/state/state_light.rs similarity index 99% rename from core/rpc/src/state/state_light.rs rename to client/rpc/src/state/state_light.rs index 3d0c7979e3995..2c8e59c5b6aac 100644 --- a/core/rpc/src/state/state_light.rs +++ b/client/rpc/src/state/state_light.rs @@ -39,8 +39,9 @@ use rpc::{ }; use api::Subscriptions; +use client_api::backend::Backend; use client::{ - BlockchainEvents, Client, CallExecutor, backend::Backend, + BlockchainEvents, Client, CallExecutor, error::Error as ClientError, light::{ blockchain::{future_header, RemoteBlockchain}, diff --git a/core/rpc/src/state/tests.rs b/client/rpc/src/state/tests.rs similarity index 99% rename from core/rpc/src/state/tests.rs rename to client/rpc/src/state/tests.rs index 5dfa234337afa..2e3690f3058a2 100644 --- a/core/rpc/src/state/tests.rs +++ b/client/rpc/src/state/tests.rs @@ -22,7 +22,7 @@ use std::sync::Arc; use assert_matches::assert_matches; use futures::stream::Stream; use primitives::storage::well_known_keys; -use sr_io::blake2_256; +use sr_io::hashing::blake2_256; use test_client::{ prelude::*, consensus::BlockOrigin, diff --git a/core/rpc/src/system/mod.rs b/client/rpc/src/system/mod.rs similarity index 100% rename from core/rpc/src/system/mod.rs rename to client/rpc/src/system/mod.rs diff --git a/core/rpc/src/system/tests.rs b/client/rpc/src/system/tests.rs similarity index 100% rename from core/rpc/src/system/tests.rs rename to client/rpc/src/system/tests.rs diff --git a/client/service/Cargo.toml b/client/service/Cargo.toml new file mode 100644 index 0000000000000..7bd608487c40f --- /dev/null +++ b/client/service/Cargo.toml @@ -0,0 +1,63 @@ +[package] +name = "substrate-service" +version = "2.0.0" +authors = ["Parity Technologies "] +edition = "2018" + +[features] +default = ["rocksdb"] +# The RocksDB feature activates the RocksDB database backend. If it is not activated, and you pass +# a path to a database, an error will be produced at runtime. +rocksdb = ["client_db/kvdb-rocksdb"] +wasmtime = [ + "substrate-executor/wasmtime", +] + +[dependencies] +derive_more = "0.15.0" +futures = "0.1.29" +futures03 = { package = "futures-preview", version = "0.3.0-alpha.19", features = ["compat"] } +parking_lot = "0.9.0" +lazy_static = "1.4.0" +log = "0.4.8" +slog = { version = "2.5.2", features = ["nested-values"] } +tokio-executor = "0.1.8" +tokio-timer = "0.2.11" +exit-future = "0.1.4" +serde = "1.0.101" +serde_json = "1.0.41" +sysinfo = "0.9.5" +target_info = "0.1.0" +keystore = { package = "substrate-keystore", path = "../keystore" } +sr-io = { path = "../../primitives/sr-io" } +sr-primitives = { path = "../../primitives/sr-primitives" } +primitives = { package = "substrate-primitives", path = "../../primitives/core" } +session = { package = "substrate-session", path = "../../primitives/session" } +app-crypto = { package = "substrate-application-crypto", path = "../../primitives/application-crypto" } +consensus_common = { package = "substrate-consensus-common", path = "../../primitives/consensus/common" } +network = { package = "substrate-network", path = "../network" } +chain-spec = { package = "substrate-chain-spec", path = "../chain-spec" } +client-api = { package = "substrate-client-api", path = "../api" } +client = { package = "substrate-client", path = "../" } +sr-api = { path = "../../primitives/sr-api" } +tx-pool-api = { package = "substrate-transaction-pool-runtime-api", path = "../../primitives/transaction-pool/runtime-api" } +client_db = { package = "substrate-client-db", path = "../db" } +codec = { package = "parity-scale-codec", version = "1.0.0" } +substrate-executor = { path = "../executor" } +transaction_pool = { package = "substrate-transaction-pool", path = "../transaction-pool" } +rpc-servers = { package = "substrate-rpc-servers", path = "../rpc-servers" } +rpc = { package = "substrate-rpc", path = "../rpc" } +tel = { package = "substrate-telemetry", path = "../telemetry" } +promet = { package = "substrate-prometheus", path="../prometheus"} +offchain = { package = "substrate-offchain", path = "../offchain" } +parity-multiaddr = { package = "parity-multiaddr", version = "0.5.0" } + +[dev-dependencies] +substrate-test-runtime-client = { path = "../../test/utils/runtime/client" } +node-executor = { path = "../../bin/node/executor" } +node-primitives = { path = "../../bin/node/primitives" } +node-runtime = { path = "../../bin/node/runtime" } +babe-primitives = { package = "substrate-consensus-babe-primitives", path = "../../primitives/consensus/babe" } +grandpa = { package = "substrate-finality-grandpa", path = "../finality-grandpa" } +grandpa-primitives = { package = "substrate-finality-grandpa-primitives", path = "../../primitives/finality-grandpa" } +tokio = "0.1" diff --git a/core/service/src/builder.rs b/client/service/src/builder.rs similarity index 64% rename from core/service/src/builder.rs rename to client/service/src/builder.rs index e39610b702372..ea1039edb9dd6 100644 --- a/core/service/src/builder.rs +++ b/client/service/src/builder.rs @@ -14,37 +14,44 @@ // You should have received a copy of the GNU General Public License // along with Substrate. If not, see . -use crate::{Service, NetworkStatus, NetworkState, error::{self, Error}, DEFAULT_PROTOCOL_ID}; -use crate::{SpawnTaskHandle, start_rpc_servers, build_network_future, TransactionPoolAdapter}; -use crate::status_sinks; use crate::config::{Configuration, DatabaseConfig}; -use client::{ - BlockchainEvents, Client, runtime_api, - backend::RemoteBackend, light::blockchain::RemoteBlockchain, +use crate::status_sinks; +use crate::{build_network_future, start_rpc_servers, SpawnTaskHandle, TransactionPoolAdapter}; +use crate::{ + error::{self, Error}, + NetworkState, NetworkStatus, Service, DEFAULT_PROTOCOL_ID, }; -use chain_spec::{RuntimeGenesis, Extension}; +use chain_spec::{Extension, RuntimeGenesis}; +use client::Client; +use client_api::{self, backend::RemoteBackend, light::RemoteBlockchain, BlockchainEvents}; use codec::{Decode, Encode, IoReader}; use consensus_common::import_queue::ImportQueue; use futures::{prelude::*, sync::mpsc}; use futures03::{ - compat::Compat, - future::ready, - FutureExt as _, TryFutureExt as _, - StreamExt as _, TryStreamExt as _, + compat::Compat, future::ready, FutureExt as _, StreamExt as _, TryFutureExt as _, + TryStreamExt as _, }; -use keystore::{Store as Keystore}; +use keystore::Store as Keystore; use log::{info, warn}; -use network::{FinalityProofProvider, OnDemand, NetworkService, NetworkStateInfo, DhtEvent}; use network::{config::BoxFinalityProofRequestBuilder, specialization::NetworkSpecialization}; +use network::{DhtEvent, FinalityProofProvider, NetworkService, NetworkStateInfo, OnDemand}; use parking_lot::{Mutex, RwLock}; -use primitives::{Blake2Hasher, H256, Hasher}; +use primitives::{Blake2Hasher, Hasher, H256}; +use promet::prometheus_gauge; use rpc; use sr_primitives::generic::BlockId; use sr_primitives::traits::{ - Block as BlockT, Extrinsic, ProvideRuntimeApi, NumberFor, One, Zero, Header, SaturatedConversion + Block as BlockT, Extrinsic, Header, NumberFor, One, ProvideRuntimeApi, SaturatedConversion, + Zero, +}; +use std::{ + io::{Read, Seek, Write}, + marker::PhantomData, + sync::atomic::AtomicBool, + sync::Arc, + time::SystemTime, }; -use substrate_executor::{NativeExecutor, NativeExecutionDispatch}; -use std::{io::{Read, Write, Seek}, marker::PhantomData, sync::Arc, sync::atomic::AtomicBool}; +use substrate_executor::{NativeExecutionDispatch, NativeExecutor}; use sysinfo::{get_current_pid, ProcessExt, System, SystemExt}; use tel::{telemetry, SUBSTRATE_INFO}; use transaction_pool::txpool::{self, ChainApi, Pool as TransactionPool}; @@ -67,9 +74,23 @@ use transaction_pool::txpool::{self, ChainApi, Pool as TransactionPool}; /// The order in which the `with_*` methods are called doesn't matter, as the correct binding of /// generics is done when you call `build`. /// -pub struct ServiceBuilder -{ +pub struct ServiceBuilder< + TBl, + TRtApi, + TCfg, + TGen, + TCSExt, + TCl, + TFchr, + TSc, + TImpQu, + TFprb, + TFpp, + TNetP, + TExPool, + TRpc, + Backend, +> { config: Configuration, client: Arc, backend: Arc, @@ -88,81 +109,75 @@ pub struct ServiceBuilder = Client< - TFullBackend, - TFullCallExecutor, - TBl, - TRtApi, ->; +type TFullClient = + Client, TFullCallExecutor, TBl, TRtApi>; /// Full client backend type. type TFullBackend = client_db::Backend; /// Full client call executor type. -type TFullCallExecutor = client::LocalCallExecutor< - client_db::Backend, - NativeExecutor, ->; +type TFullCallExecutor = + client::LocalCallExecutor, NativeExecutor>; /// Light client type. -type TLightClient = Client< - TLightBackend, - TLightCallExecutor, - TBl, - TRtApi, ->; +type TLightClient = + Client, TLightCallExecutor, TBl, TRtApi>; /// Light client backend type. -type TLightBackend = client::light::backend::Backend< - client_db::light::LightStorage, - Blake2Hasher, ->; +type TLightBackend = + client::light::backend::Backend, Blake2Hasher>; /// Light call executor type. type TLightCallExecutor = client::light::call_executor::GenesisCallExecutor< - client::light::backend::Backend< - client_db::light::LightStorage, - Blake2Hasher - >, + client::light::backend::Backend, Blake2Hasher>, client::LocalCallExecutor< - client::light::backend::Backend< - client_db::light::LightStorage, - Blake2Hasher - >, - NativeExecutor + client::light::backend::Backend, Blake2Hasher>, + NativeExecutor, >, >; -impl ServiceBuilder<(), (), TCfg, TGen, TCSExt, (), (), (), (), (), (), (), (), (), ()> -where TGen: RuntimeGenesis, TCSExt: Extension { +impl + ServiceBuilder<(), (), TCfg, TGen, TCSExt, (), (), (), (), (), (), (), (), (), ()> +where + TGen: RuntimeGenesis, + TCSExt: Extension, +{ /// Start the service builder with a configuration. - pub fn new_full, TRtApi, TExecDisp: NativeExecutionDispatch>( - config: Configuration - ) -> Result, - Arc>, - (), - (), - BoxFinalityProofRequestBuilder, - Arc>, - (), - (), - (), - TFullBackend, - >, Error> { - let keystore = Keystore::open(config.keystore_path.clone(), config.keystore_password.clone())?; - - let executor = NativeExecutor::::new( - config.wasm_method, - config.default_heap_pages, - ); + pub fn new_full, TRtApi, TExecDisp: NativeExecutionDispatch>( + config: Configuration, + ) -> Result< + ServiceBuilder< + TBl, + TRtApi, + TCfg, + TGen, + TCSExt, + TFullClient, + Arc>, + (), + (), + BoxFinalityProofRequestBuilder, + Arc>, + (), + (), + (), + TFullBackend, + >, + Error, + > { + let keystore = Keystore::open( + config + .keystore_path + .clone() + .ok_or("No basepath configured")?, + config.keystore_password.clone(), + )?; - let fork_blocks = config.chain_spec + let executor = + NativeExecutor::::new(config.wasm_method, config.default_heap_pages); + + let fork_blocks = config + .chain_spec .extensions() .get::>() .cloned() @@ -171,20 +186,21 @@ where TGen: RuntimeGenesis, TCSExt: Extension { let (client, backend) = { let db_config = client_db::DatabaseSettings { state_cache_size: config.state_cache_size, - state_cache_child_ratio: - config.state_cache_child_ratio.map(|v| (v, 100)), + state_cache_child_ratio: config.state_cache_child_ratio.map(|v| (v, 100)), pruning: config.pruning.clone(), source: match &config.database { - DatabaseConfig::Path { path, cache_size } => + DatabaseConfig::Path { path, cache_size } => { client_db::DatabaseSettingsSrc::Path { path: path.clone(), cache_size: cache_size.clone().map(|u| u as usize), - }, - DatabaseConfig::Custom(db) => - client_db::DatabaseSettingsSrc::Custom(db.clone()), + } + } + DatabaseConfig::Custom(db) => { + client_db::DatabaseSettingsSrc::Custom(db.clone()) + } }, }; - + client_db::new_client( db_config, executor, @@ -217,52 +233,67 @@ where TGen: RuntimeGenesis, TCSExt: Extension { } /// Start the service builder with a configuration. - pub fn new_light, TRtApi, TExecDisp: NativeExecutionDispatch + 'static>( - config: Configuration - ) -> Result, TRtApi, - TCfg, - TGen, - TCSExt, - TLightClient, - Arc>, - (), - (), - BoxFinalityProofRequestBuilder, - Arc>, - (), - (), - (), - TLightBackend, - >, Error> { - let keystore = Keystore::open(config.keystore_path.clone(), config.keystore_password.clone())?; - - let executor = NativeExecutor::::new( - config.wasm_method, - config.default_heap_pages, - ); + TExecDisp: NativeExecutionDispatch + 'static, + >( + config: Configuration, + ) -> Result< + ServiceBuilder< + TBl, + TRtApi, + TCfg, + TGen, + TCSExt, + TLightClient, + Arc>, + (), + (), + BoxFinalityProofRequestBuilder, + Arc>, + (), + (), + (), + TLightBackend, + >, + Error, + > { + let keystore = Keystore::open( + config + .keystore_path + .clone() + .ok_or("No basepath configured")?, + config.keystore_password.clone(), + )?; + + let executor = + NativeExecutor::::new(config.wasm_method, config.default_heap_pages); let db_storage = { let db_settings = client_db::DatabaseSettings { state_cache_size: config.state_cache_size, - state_cache_child_ratio: - config.state_cache_child_ratio.map(|v| (v, 100)), + state_cache_child_ratio: config.state_cache_child_ratio.map(|v| (v, 100)), pruning: config.pruning.clone(), source: match &config.database { - DatabaseConfig::Path { path, cache_size } => + DatabaseConfig::Path { path, cache_size } => { client_db::DatabaseSettingsSrc::Path { path: path.clone(), cache_size: cache_size.clone().map(|u| u as usize), - }, - DatabaseConfig::Custom(db) => - client_db::DatabaseSettingsSrc::Custom(db.clone()), + } + } + DatabaseConfig::Custom(db) => { + client_db::DatabaseSettingsSrc::Custom(db.clone()) + } }, }; client_db::light::LightStorage::new(db_settings)? }; let light_blockchain = client::light::new_light_blockchain(db_storage); - let fetch_checker = Arc::new(client::light::new_fetch_checker(light_blockchain.clone(), executor.clone())); + let fetch_checker = Arc::new(client::light::new_fetch_checker( + light_blockchain.clone(), + executor.clone(), + )); let fetcher = Arc::new(network::OnDemand::new(fetch_checker)); let backend = client::light::new_light_backend(light_blockchain); let remote_blockchain = backend.remote_blockchain(); @@ -292,10 +323,41 @@ where TGen: RuntimeGenesis, TCSExt: Extension { } } -impl - ServiceBuilder { - +impl< + TBl, + TRtApi, + TCfg, + TGen, + TCSExt, + TCl, + TFchr, + TSc, + TImpQu, + TFprb, + TFpp, + TNetP, + TExPool, + TRpc, + Backend, + > + ServiceBuilder< + TBl, + TRtApi, + TCfg, + TGen, + TCSExt, + TCl, + TFchr, + TSc, + TImpQu, + TFprb, + TFpp, + TNetP, + TExPool, + TRpc, + Backend, + > +{ /// Returns a reference to the client that was stored in this builder. pub fn client(&self) -> &Arc { &self.client @@ -315,10 +377,29 @@ impl( self, select_chain_builder: impl FnOnce( - &Configuration, &Arc - ) -> Result, Error> - ) -> Result, Error> { + &Configuration, + &Arc, + ) -> Result, Error>, + ) -> Result< + ServiceBuilder< + TBl, + TRtApi, + TCfg, + TGen, + TCSExt, + TCl, + TFchr, + USc, + TImpQu, + TFprb, + TFpp, + TNetP, + TExPool, + TRpc, + Backend, + >, + Error, + > { let select_chain = select_chain_builder(&self.config, &self.backend)?; Ok(ServiceBuilder { @@ -343,25 +424,67 @@ impl( self, - builder: impl FnOnce(&Configuration, &Arc) -> Result - ) -> Result, Error> { + builder: impl FnOnce(&Configuration, &Arc) -> Result, + ) -> Result< + ServiceBuilder< + TBl, + TRtApi, + TCfg, + TGen, + TCSExt, + TCl, + TFchr, + USc, + TImpQu, + TFprb, + TFpp, + TNetP, + TExPool, + TRpc, + Backend, + >, + Error, + > { self.with_opt_select_chain(|cfg, b| builder(cfg, b).map(Option::Some)) } /// Defines which import queue to use. pub fn with_import_queue( self, - builder: impl FnOnce(&Configuration, Arc, Option, Arc) - -> Result - ) -> Result, Error> - where TSc: Clone { + builder: impl FnOnce( + &Configuration, + Arc, + Option, + Arc, + ) -> Result, + ) -> Result< + ServiceBuilder< + TBl, + TRtApi, + TCfg, + TGen, + TCSExt, + TCl, + TFchr, + TSc, + UImpQu, + TFprb, + TFpp, + TNetP, + TExPool, + TRpc, + Backend, + >, + Error, + > + where + TSc: Clone, + { let import_queue = builder( &self.config, self.client.clone(), self.select_chain.clone(), - self.transaction_pool.clone() + self.transaction_pool.clone(), )?; Ok(ServiceBuilder { @@ -386,9 +509,29 @@ impl( self, - network_protocol_builder: impl FnOnce(&Configuration) -> Result - ) -> Result, Error> { + network_protocol_builder: impl FnOnce( + &Configuration, + ) -> Result, + ) -> Result< + ServiceBuilder< + TBl, + TRtApi, + TCfg, + TGen, + TCSExt, + TCl, + TFchr, + TSc, + TImpQu, + TFprb, + TFpp, + UNetP, + TExPool, + TRpc, + Backend, + >, + Error, + > { let network_protocol = network_protocol_builder(&self.config)?; Ok(ServiceBuilder { @@ -413,24 +556,30 @@ impl, Arc) -> Result>>, Error> - ) -> Result>, - TNetP, - TExPool, - TRpc, - Backend, - >, Error> { + builder: impl FnOnce( + Arc, + Arc, + ) -> Result>>, Error>, + ) -> Result< + ServiceBuilder< + TBl, + TRtApi, + TCfg, + TGen, + TCSExt, + TCl, + TFchr, + TSc, + TImpQu, + TFprb, + Arc>, + TNetP, + TExPool, + TRpc, + Backend, + >, + Error, + > { let finality_proof_provider = builder(self.client.clone(), self.backend.clone())?; Ok(ServiceBuilder { @@ -455,25 +604,30 @@ impl, Arc) -> Result>, Error> - ) -> Result>, - TNetP, - TExPool, - TRpc, - Backend, - >, Error> { - self.with_opt_finality_proof_provider(|client, backend| build(client, backend).map(Option::Some)) + build: impl FnOnce(Arc, Arc) -> Result>, Error>, + ) -> Result< + ServiceBuilder< + TBl, + TRtApi, + TCfg, + TGen, + TCSExt, + TCl, + TFchr, + TSc, + TImpQu, + TFprb, + Arc>, + TNetP, + TExPool, + TRpc, + Backend, + >, + Error, + > { + self.with_opt_finality_proof_provider(|client, backend| { + build(client, backend).map(Option::Some) + }) } /// Defines which import queue to use. @@ -486,17 +640,38 @@ impl, Option, Arc, - ) -> Result<(UImpQu, Option), Error> - ) -> Result, Error> - where TSc: Clone, TFchr: Clone { + ) -> Result<(UImpQu, Option), Error>, + ) -> Result< + ServiceBuilder< + TBl, + TRtApi, + TCfg, + TGen, + TCSExt, + TCl, + TFchr, + TSc, + UImpQu, + UFprb, + TFpp, + TNetP, + TExPool, + TRpc, + Backend, + >, + Error, + > + where + TSc: Clone, + TFchr: Clone, + { let (import_queue, fprb) = builder( &self.config, self.client.clone(), self.backend.clone(), self.fetcher.clone(), self.select_chain.clone(), - self.transaction_pool.clone() + self.transaction_pool.clone(), )?; Ok(ServiceBuilder { @@ -528,23 +703,65 @@ impl, Option, Arc, - ) -> Result<(UImpQu, UFprb), Error> - ) -> Result, Error> - where TSc: Clone, TFchr: Clone { - self.with_import_queue_and_opt_fprb(|cfg, cl, b, f, sc, tx| - builder(cfg, cl, b, f, sc, tx) - .map(|(q, f)| (q, Some(f))) - ) + ) -> Result<(UImpQu, UFprb), Error>, + ) -> Result< + ServiceBuilder< + TBl, + TRtApi, + TCfg, + TGen, + TCSExt, + TCl, + TFchr, + TSc, + UImpQu, + UFprb, + TFpp, + TNetP, + TExPool, + TRpc, + Backend, + >, + Error, + > + where + TSc: Clone, + TFchr: Clone, + { + self.with_import_queue_and_opt_fprb(|cfg, cl, b, f, sc, tx| { + builder(cfg, cl, b, f, sc, tx).map(|(q, f)| (q, Some(f))) + }) } /// Defines which transaction pool to use. pub fn with_transaction_pool( self, - transaction_pool_builder: impl FnOnce(transaction_pool::txpool::Options, Arc) -> Result - ) -> Result, Error> { - let transaction_pool = transaction_pool_builder(self.config.transaction_pool.clone(), self.client.clone())?; + transaction_pool_builder: impl FnOnce( + transaction_pool::txpool::Options, + Arc, + ) -> Result, + ) -> Result< + ServiceBuilder< + TBl, + TRtApi, + TCfg, + TGen, + TCSExt, + TCl, + TFchr, + TSc, + TImpQu, + TFprb, + TFpp, + TNetP, + UExPool, + TRpc, + Backend, + >, + Error, + > { + let transaction_pool = + transaction_pool_builder(self.config.transaction_pool.clone(), self.client.clone())?; Ok(ServiceBuilder { config: self.config, @@ -568,10 +785,32 @@ impl( self, - rpc_ext_builder: impl FnOnce(Arc, Arc) -> URpc - ) -> Result, Error> { - let rpc_extensions = rpc_ext_builder(self.client.clone(), self.transaction_pool.clone()); + rpc_ext_builder: impl FnOnce(Arc, Arc, Arc) -> URpc, + ) -> Result< + ServiceBuilder< + TBl, + TRtApi, + TCfg, + TGen, + TCSExt, + TCl, + TFchr, + TSc, + TImpQu, + TFprb, + TFpp, + TNetP, + TExPool, + URpc, + Backend, + >, + Error, + > { + let rpc_extensions = rpc_ext_builder( + self.client.clone(), + self.transaction_pool.clone(), + self.backend.clone(), + ); Ok(ServiceBuilder { config: self.config, @@ -597,8 +836,26 @@ impl, - ) -> Result, Error> { + ) -> Result< + ServiceBuilder< + TBl, + TRtApi, + TCfg, + TGen, + TCSExt, + TCl, + TFchr, + TSc, + TImpQu, + TFprb, + TFpp, + TNetP, + TExPool, + TRpc, + Backend, + >, + Error, + > { Ok(ServiceBuilder { config: self.config, client: self.client, @@ -625,7 +882,7 @@ pub trait ServiceBuilderImport { /// Starts the process of importing blocks. fn import_blocks( self, - exit: impl Future + Send + 'static, + exit: impl Future + Send + 'static, input: impl Read + Seek, ) -> Result + Send>, Error>; } @@ -639,11 +896,11 @@ pub trait ServiceBuilderExport { /// Performs the blocks export. fn export_blocks( &self, - exit: impl Future + Send + 'static, + exit: impl Future + Send + 'static, output: impl Write, from: NumberFor, to: Option>, - json: bool + json: bool, ) -> Result<(), Error>; } @@ -654,110 +911,191 @@ pub trait ServiceBuilderRevert { type Block: BlockT; /// Performs a revert of `blocks` bocks. - fn revert_chain( - &self, - blocks: NumberFor - ) -> Result<(), Error>; + fn revert_chain(&self, blocks: NumberFor) -> Result<(), Error>; } impl< - TBl, TRtApi, TCfg, TGen, TCSExt, TBackend, - TExec, TFchr, TSc, TImpQu, TFprb, TFpp, TNetP, - TExPool, TRpc, Backend -> ServiceBuilderImport for ServiceBuilder< - TBl, TRtApi, TCfg, TGen, TCSExt, Client, - TFchr, TSc, TImpQu, TFprb, TFpp, TNetP, TExPool, TRpc, Backend -> where + TBl, + TRtApi, + TCfg, + TGen, + TCSExt, + TBackend, + TExec, + TFchr, + TSc, + TImpQu, + TFprb, + TFpp, + TNetP, + TExPool, + TRpc, + Backend, + > ServiceBuilderImport + for ServiceBuilder< + TBl, + TRtApi, + TCfg, + TGen, + TCSExt, + Client, + TFchr, + TSc, + TImpQu, + TFprb, + TFpp, + TNetP, + TExPool, + TRpc, + Backend, + > where TBl: BlockT::Out>, - TBackend: 'static + client::backend::Backend + Send, + TBackend: 'static + client_api::backend::Backend + Send, TExec: 'static + client::CallExecutor + Send + Sync + Clone, TImpQu: 'static + ImportQueue, TRtApi: 'static + Send + Sync, { fn import_blocks( self, - exit: impl Future + Send + 'static, + exit: impl Future + Send + 'static, input: impl Read + Seek, ) -> Result + Send>, Error> { let client = self.client; let mut queue = self.import_queue; - import_blocks!(TBl, client, queue, exit, input) - .map(|f| Box::new(f) as Box<_>) + import_blocks!(TBl, client, queue, exit, input).map(|f| Box::new(f) as Box<_>) } } -impl - ServiceBuilderExport for ServiceBuilder, - TFchr, TSc, TImpQu, TFprb, TFpp, TNetP, TExPool, TRpc, TBackend> -where +impl< + TBl, + TRtApi, + TCfg, + TGen, + TCSExt, + TBackend, + TExec, + TFchr, + TSc, + TImpQu, + TFprb, + TFpp, + TNetP, + TExPool, + TRpc, + > ServiceBuilderExport + for ServiceBuilder< + TBl, + TRtApi, + TCfg, + TGen, + TCSExt, + Client, + TFchr, + TSc, + TImpQu, + TFprb, + TFpp, + TNetP, + TExPool, + TRpc, + TBackend, + > where TBl: BlockT::Out>, - TBackend: 'static + client::backend::Backend + Send, - TExec: 'static + client::CallExecutor + Send + Sync + Clone + TBackend: 'static + client_api::backend::Backend + Send, + TExec: 'static + client::CallExecutor + Send + Sync + Clone, { type Block = TBl; fn export_blocks( &self, - exit: impl Future + Send + 'static, + exit: impl Future + Send + 'static, mut output: impl Write, from: NumberFor, to: Option>, - json: bool + json: bool, ) -> Result<(), Error> { let client = &self.client; export_blocks!(client, exit, output, from, to, json) } } -impl - ServiceBuilderRevert for ServiceBuilder, - TFchr, TSc, TImpQu, TFprb, TFpp, TNetP, TExPool, TRpc, TBackend> -where +impl< + TBl, + TRtApi, + TCfg, + TGen, + TCSExt, + TBackend, + TExec, + TFchr, + TSc, + TImpQu, + TFprb, + TFpp, + TNetP, + TExPool, + TRpc, + > ServiceBuilderRevert + for ServiceBuilder< + TBl, + TRtApi, + TCfg, + TGen, + TCSExt, + Client, + TFchr, + TSc, + TImpQu, + TFprb, + TFpp, + TNetP, + TExPool, + TRpc, + TBackend, + > where TBl: BlockT::Out>, - TBackend: 'static + client::backend::Backend + Send, - TExec: 'static + client::CallExecutor + Send + Sync + Clone + TBackend: 'static + client_api::backend::Backend + Send, + TExec: 'static + client::CallExecutor + Send + Sync + Clone, { type Block = TBl; - fn revert_chain( - &self, - blocks: NumberFor - ) -> Result<(), Error> { + fn revert_chain(&self, blocks: NumberFor) -> Result<(), Error> { let client = &self.client; revert_chain!(client, blocks) } } impl -ServiceBuilder< - TBl, - TRtApi, - TCfg, - TGen, - TCSExt, - Client, - Arc>, - TSc, - TImpQu, - BoxFinalityProofRequestBuilder, - Arc>, - TNetP, - TransactionPool, - TRpc, - TBackend, -> where + ServiceBuilder< + TBl, + TRtApi, + TCfg, + TGen, + TCSExt, + Client, + Arc>, + TSc, + TImpQu, + BoxFinalityProofRequestBuilder, + Arc>, + TNetP, + TransactionPool, + TRpc, + TBackend, + > where Client: ProvideRuntimeApi, as ProvideRuntimeApi>::Api: - runtime_api::Metadata + - offchain::OffchainWorkerApi + - runtime_api::TaggedTransactionQueue + - session::SessionKeys, + sr_api::Metadata + + offchain::OffchainWorkerApi + + tx_pool_api::TaggedTransactionQueue + + session::SessionKeys + + sr_api::ApiExt, TBl: BlockT::Out>, TRtApi: 'static + Send + Sync, TCfg: Default, TGen: RuntimeGenesis, TCSExt: Extension, - TBackend: 'static + client::backend::Backend + Send, + TBackend: 'static + client_api::backend::Backend + Send, TExec: 'static + client::CallExecutor + Send + Sync + Clone, TSc: Clone, TImpQu: 'static + ImportQueue, @@ -766,19 +1104,24 @@ ServiceBuilder< TRpc: rpc::RpcExtension + Clone, { /// Builds the service. - pub fn build(self) -> Result, - TSc, - NetworkStatus, - NetworkService::Hash>, - TransactionPool, - offchain::OffchainWorkers< + pub fn build( + self, + ) -> Result< + Service< + TBl, Client, - TBackend::OffchainStorage, - TBl + TSc, + NetworkStatus, + NetworkService::Hash>, + TransactionPool, + offchain::OffchainWorkers< + Client, + TBackend::OffchainStorage, + TBl, + >, >, - >, Error> { + Error, + > { let ServiceBuilder { marker: _, mut config, @@ -799,7 +1142,12 @@ ServiceBuilder< session::generate_initial_session_keys( client.clone(), - config.dev_key_seed.clone().map(|s| vec![s]).unwrap_or_default() + &BlockId::Hash(client.info().chain.best_hash), + config + .dev_key_seed + .clone() + .map(|s| vec![s]) + .unwrap_or_default(), )?; let (signal, exit) = exit_future::signal(); @@ -824,24 +1172,31 @@ ServiceBuilder< imports_external_transactions: !config.roles.is_light(), pool: transaction_pool.clone(), client: client.clone(), - executor: Arc::new(SpawnTaskHandle { sender: to_spawn_tx.clone(), on_exit: exit.clone() }), + executor: Arc::new(SpawnTaskHandle { + sender: to_spawn_tx.clone(), + on_exit: exit.clone(), + }), }); let protocol_id = { let protocol_id_full = match config.chain_spec.protocol_id() { Some(pid) => pid, None => { - warn!("Using default protocol ID {:?} because none is configured in the \ - chain specs", DEFAULT_PROTOCOL_ID + warn!( + "Using default protocol ID {:?} because none is configured in the \ + chain specs", + DEFAULT_PROTOCOL_ID ); DEFAULT_PROTOCOL_ID } - }.as_bytes(); + } + .as_bytes(); network::config::ProtocolId::from(protocol_id_full) }; - let block_announce_validator = - Box::new(consensus_common::block_validation::DefaultBlockAnnounceValidator::new(client.clone())); + let block_announce_validator = Box::new( + consensus_common::block_validation::DefaultBlockAnnounceValidator::new(client.clone()), + ); let network_params = network::config::Params { roles: config.roles, @@ -864,13 +1219,11 @@ ServiceBuilder< let offchain_storage = backend.offchain_storage(); let offchain_workers = match (config.offchain_worker, offchain_storage) { - (true, Some(db)) => { - Some(Arc::new(offchain::OffchainWorkers::new(client.clone(), db))) - }, + (true, Some(db)) => Some(Arc::new(offchain::OffchainWorkers::new(client.clone(), db))), (true, None) => { log::warn!("Offchain workers disabled, due to lack of offchain storage support in backend."); None - }, + } _ => None, }; @@ -883,8 +1236,10 @@ ServiceBuilder< let network_state_info: Arc = network.clone(); let is_validator = config.roles.is_authority(); - let events = client.import_notification_stream() - .map(|v| Ok::<_, ()>(v)).compat() + let events = client + .import_notification_stream() + .map(|v| Ok::<_, ()>(v)) + .compat() .for_each(move |notification| { let number = *notification.header.number(); let txpool = txpool.upgrade(); @@ -895,13 +1250,20 @@ ServiceBuilder< &client, &*txpool, ¬ification.retracted, - ).map_err(|e| warn!("Pool error processing new block: {:?}", e))?; + ) + .map_err(|e| warn!("Pool error processing new block: {:?}", e))?; let _ = to_spawn_tx_.unbounded_send(future); } let offchain = offchain.as_ref().and_then(|o| o.upgrade()); if let (Some(txpool), Some(offchain)) = (txpool, offchain) { - let future = offchain.on_block_imported(&number, &txpool, network_state_info.clone(), is_validator) + let future = offchain + .on_block_imported( + &number, + &txpool, + network_state_info.clone(), + is_validator, + ) .map(|()| Ok(())); let _ = to_spawn_tx_.unbounded_send(Box::new(Compat::new(future))); } @@ -917,8 +1279,10 @@ ServiceBuilder< // extrinsic notifications let network = Arc::downgrade(&network); let transaction_pool_ = transaction_pool.clone(); - let events = transaction_pool.import_notification_stream() - .map(|v| Ok::<_, ()>(v)).compat() + let events = transaction_pool + .import_notification_stream() + .map(|v| Ok::<_, ()>(v)) + .compat() .for_each(move |_| { if let Some(network) = network.upgrade() { network.trigger_repropagate(); @@ -942,68 +1306,95 @@ ServiceBuilder< let mut sys = System::new(); let self_pid = get_current_pid().ok(); let (state_tx, state_rx) = mpsc::unbounded::<(NetworkStatus<_>, NetworkState)>(); - network_status_sinks.lock().push(std::time::Duration::from_millis(5000), state_tx); - let tel_task = state_rx.for_each(move |(net_status, _)| { - let info = client_.info(); - let best_number = info.chain.best_number.saturated_into::(); - let best_hash = info.chain.best_hash; - let num_peers = net_status.num_connected_peers; - let txpool_status = transaction_pool_.status(); - let finalized_number: u64 = info.chain.finalized_number.saturated_into::(); - let bandwidth_download = net_status.average_download_per_sec; - let bandwidth_upload = net_status.average_upload_per_sec; - - let used_state_cache_size = match info.used_state_cache_size { - Some(size) => size, - None => 0, - }; - - // get cpu usage and memory usage of this process - let (cpu_usage, memory) = if let Some(self_pid) = self_pid { - if sys.refresh_process(self_pid) { - let proc = sys.get_process(self_pid) - .expect("Above refresh_process succeeds, this should be Some(), qed"); - (proc.cpu_usage(), proc.memory()) - } else { (0.0, 0) } - } else { (0.0, 0) }; - - telemetry!( - SUBSTRATE_INFO; - "system.interval"; - "peers" => num_peers, - "height" => best_number, - "best" => ?best_hash, - "txcount" => txpool_status.ready, - "cpu" => cpu_usage, - "memory" => memory, - "finalized_height" => finalized_number, - "finalized_hash" => ?info.chain.finalized_hash, - "bandwidth_download" => bandwidth_download, - "bandwidth_upload" => bandwidth_upload, - "used_state_cache_size" => used_state_cache_size, - ); + network_status_sinks + .lock() + .push(std::time::Duration::from_millis(5000), state_tx); + let tel_task = state_rx + .for_each(move |(net_status, _)| { + let info = client_.info(); + let best_number = info.chain.best_number.saturated_into::(); + let best_hash = info.chain.best_hash; + let num_peers = net_status.num_connected_peers; + let txpool_status = transaction_pool_.status(); + let finalized_number: u64 = info.chain.finalized_number.saturated_into::(); + let bandwidth_download = net_status.average_download_per_sec; + let bandwidth_upload = net_status.average_upload_per_sec; + + let used_state_cache_size = match info.used_state_cache_size { + Some(size) => size, + None => 0, + }; + + // get cpu usage and memory usage of this process + let (cpu_usage, memory) = if let Some(self_pid) = self_pid { + if sys.refresh_process(self_pid) { + let proc = sys + .get_process(self_pid) + .expect("Above refresh_process succeeds, this should be Some(), qed"); + (proc.cpu_usage(), proc.memory()) + } else { + (0.0, 0) + } + } else { + (0.0, 0) + }; + + telemetry!( + SUBSTRATE_INFO; + "system.interval"; + "peers" => num_peers, + "height" => best_number, + "best" => ?best_hash, + "txcount" => txpool_status.ready, + "cpu" => cpu_usage, + "memory" => memory, + "finalized_height" => finalized_number, + "finalized_hash" => ?info.chain.finalized_hash, + "bandwidth_download" => bandwidth_download, + "bandwidth_upload" => bandwidth_upload, + "used_state_cache_size" => used_state_cache_size, + ); - Ok(()) - }).select(exit.clone()).then(|_| Ok(())); + prometheus_gauge!( + MEMPOOL_SIZE => used_state_cache_size as u64, + NODE_MEMORY => memory as u64, + NODE_CPU => cpu_usage as u64, + TX_COUNT => txpool_status.ready as u64, + FINALITY_HEIGHT => finalized_number as u64, + BEST_HEIGHT => best_number as u64, + P2P_PEERS_NUM => num_peers as u64, + P2P_NODE_DOWNLOAD => net_status.average_download_per_sec as u64, + P2P_NODE_UPLOAD => net_status.average_upload_per_sec as u64 + ); + + Ok(()) + }) + .select(exit.clone()) + .then(|_| Ok(())); let _ = to_spawn_tx.unbounded_send(Box::new(tel_task)); // Periodically send the network state to the telemetry. let (netstat_tx, netstat_rx) = mpsc::unbounded::<(NetworkStatus<_>, NetworkState)>(); - network_status_sinks.lock().push(std::time::Duration::from_secs(30), netstat_tx); - let tel_task_2 = netstat_rx.for_each(move |(_, network_state)| { - telemetry!( - SUBSTRATE_INFO; - "system.network_state"; - "state" => network_state, - ); - Ok(()) - }).select(exit.clone()).then(|_| Ok(())); + network_status_sinks + .lock() + .push(std::time::Duration::from_secs(30), netstat_tx); + let tel_task_2 = netstat_rx + .for_each(move |(_, network_state)| { + telemetry!( + SUBSTRATE_INFO; + "system.network_state"; + "state" => network_state, + ); + Ok(()) + }) + .select(exit.clone()) + .then(|_| Ok(())); let _ = to_spawn_tx.unbounded_send(Box::new(tel_task_2)); // RPC let (system_rpc_tx, system_rpc_rx) = futures03::channel::mpsc::unbounded(); let gen_handler = || { - use rpc::{chain, state, author, system}; + use rpc::{author, chain, state, system}; let system_info = rpc::system::SystemInfo { chain_name: config.chain_spec.name().into(), @@ -1014,26 +1405,26 @@ ServiceBuilder< let subscriptions = rpc::Subscriptions::new(Arc::new(SpawnTaskHandle { sender: to_spawn_tx.clone(), - on_exit: exit.clone() + on_exit: exit.clone(), })); let (chain, state) = if let (Some(remote_backend), Some(on_demand)) = - (remote_backend.as_ref(), on_demand.as_ref()) { + (remote_backend.as_ref(), on_demand.as_ref()) + { // Light clients let chain = rpc::chain::new_light( client.clone(), subscriptions.clone(), remote_backend.clone(), - on_demand.clone() + on_demand.clone(), ); let state = rpc::state::new_light( client.clone(), subscriptions.clone(), remote_backend.clone(), - on_demand.clone() + on_demand.clone(), ); (chain, state) - } else { // Full nodes let chain = rpc::chain::new_full(client.clone(), subscriptions.clone()); @@ -1060,21 +1451,23 @@ ServiceBuilder< let rpc_handlers = gen_handler(); let rpc = start_rpc_servers(&config, gen_handler)?; - - let _ = to_spawn_tx.unbounded_send(Box::new(build_network_future( - config.roles, - network_mut, - client.clone(), - network_status_sinks.clone(), - system_rpc_rx, - has_bootnodes, - dht_event_tx, - ) + let _ = to_spawn_tx.unbounded_send(Box::new( + build_network_future( + config.roles, + network_mut, + client.clone(), + network_status_sinks.clone(), + system_rpc_rx, + has_bootnodes, + dht_event_tx, + ) .map_err(|_| ()) .select(exit.clone()) - .then(|_| Ok(())))); + .then(|_| Ok(())), + )); - let telemetry_connection_sinks: Arc>>> = Default::default(); + let telemetry_connection_sinks: Arc>>> = + Default::default(); // Telemetry let telemetry = config.telemetry_endpoints.clone().map(|endpoints| { @@ -1089,7 +1482,12 @@ ServiceBuilder< endpoints, wasm_external_transport: config.telemetry_external_transport.take(), }); - let future = telemetry.clone() + let startup_time = SystemTime::UNIX_EPOCH + .elapsed() + .map(|dur| dur.as_millis()) + .unwrap_or(0); + let future = telemetry + .clone() .map(|ev| Ok::<_, ()>(ev)) .compat() .for_each(move |event| { @@ -1103,20 +1501,29 @@ ServiceBuilder< "config" => "", "chain" => chain_name.clone(), "authority" => is_authority, + "startup_time" => startup_time, "network_id" => network_id.clone() ); - telemetry_connection_sinks_.lock().retain(|sink| { - sink.unbounded_send(()).is_ok() - }); + telemetry_connection_sinks_ + .lock() + .retain(|sink| sink.unbounded_send(()).is_ok()); Ok(()) }); - let _ = to_spawn_tx.unbounded_send(Box::new(future - .select(exit.clone()) - .then(|_| Ok(())))); + let _ = + to_spawn_tx.unbounded_send(Box::new(future.select(exit.clone()).then(|_| Ok(())))); telemetry }); - +<<<<<<< HEAD + +======= +>>>>>>> 9e0aa9e5d... base-code(rebased) + match config.prometheus_endpoint { + None => (), + Some(x) => { + let _prometheus = promet::init_prometheus(x); + } + } Ok(Service { client, network, @@ -1145,35 +1552,41 @@ pub(crate) fn maintain_transaction_pool( client: &Arc>, transaction_pool: &TransactionPool, retracted: &[Block::Hash], -) -> error::Result + Send>> where +) -> error::Result + Send>> +where Block: BlockT::Out>, - Backend: 'static + client::backend::Backend, + Backend: 'static + client_api::backend::Backend, Client: ProvideRuntimeApi, - as ProvideRuntimeApi>::Api: runtime_api::TaggedTransactionQueue, + as ProvideRuntimeApi>::Api: + tx_pool_api::TaggedTransactionQueue, Executor: 'static + client::CallExecutor, PoolApi: 'static + txpool::ChainApi, Api: 'static, { // Put transactions from retracted blocks back into the pool. let client_copy = client.clone(); - let retracted_transactions = retracted.to_vec().into_iter() + let retracted_transactions = retracted + .to_vec() + .into_iter() .filter_map(move |hash| client_copy.block(&BlockId::hash(hash)).ok().unwrap_or(None)) .flat_map(|block| block.block.deconstruct().1.into_iter()) .filter(|tx| tx.is_signed().unwrap_or(false)); let resubmit_future = transaction_pool .submit_at(id, retracted_transactions, true) - .then(|resubmit_result| ready(match resubmit_result { - Ok(_) => Ok(()), - Err(e) => { - warn!("Error re-submitting transactions: {:?}", e); - Ok(()) - } - })) + .then(|resubmit_result| { + ready(match resubmit_result { + Ok(_) => Ok(()), + Err(e) => { + warn!("Error re-submitting transactions: {:?}", e); + Ok(()) + } + }) + }) .compat(); // Avoid calling into runtime if there is nothing to prune from the pool anyway. if transaction_pool.status().is_empty() { - return Ok(Box::new(resubmit_future)) + return Ok(Box::new(resubmit_future)); } let block = client.block(id)?; @@ -1184,10 +1597,12 @@ pub(crate) fn maintain_transaction_pool( .prune(id, &parent_id, block.block.extrinsics()) .boxed() .compat() - .map_err(|e| { format!("{:?}", e); }); + .map_err(|e| { + format!("{:?}", e); + }); Box::new(resubmit_future.and_then(|_| prune_future)) - }, + } None => Box::new(resubmit_future), }) } @@ -1195,21 +1610,25 @@ pub(crate) fn maintain_transaction_pool( #[cfg(test)] mod tests { use super::*; - use futures03::executor::block_on; use consensus_common::{BlockOrigin, SelectChain}; + use futures03::executor::block_on; use substrate_test_runtime_client::{prelude::*, runtime::Transfer}; #[test] fn should_remove_transactions_from_the_pool() { let (client, longest_chain) = TestClientBuilder::new().build_with_longest_chain(); let client = Arc::new(client); - let pool = TransactionPool::new(Default::default(), ::transaction_pool::FullChainApi::new(client.clone())); + let pool = TransactionPool::new( + Default::default(), + ::transaction_pool::FullChainApi::new(client.clone()), + ); let transaction = Transfer { amount: 5, nonce: 0, from: AccountKeyring::Alice.into(), to: Default::default(), - }.into_signed_tx(); + } + .into_signed_tx(); let best = longest_chain.best_chain().unwrap(); // store the transaction in the pool @@ -1224,12 +1643,10 @@ mod tests { // fire notification - this should clean up the queue assert_eq!(pool.status().ready, 1); - maintain_transaction_pool( - &id, - &client, - &pool, - &[] - ).unwrap().wait().unwrap(); + maintain_transaction_pool(&id, &client, &pool, &[]) + .unwrap() + .wait() + .unwrap(); // then assert_eq!(pool.status().ready, 0); @@ -1240,13 +1657,17 @@ mod tests { fn should_add_reverted_transactions_to_the_pool() { let (client, longest_chain) = TestClientBuilder::new().build_with_longest_chain(); let client = Arc::new(client); - let pool = TransactionPool::new(Default::default(), ::transaction_pool::FullChainApi::new(client.clone())); + let pool = TransactionPool::new( + Default::default(), + ::transaction_pool::FullChainApi::new(client.clone()), + ); let transaction = Transfer { amount: 5, nonce: 0, from: AccountKeyring::Alice.into(), to: Default::default(), - }.into_signed_tx(); + } + .into_signed_tx(); let best = longest_chain.best_chain().unwrap(); // store the transaction in the pool @@ -1262,30 +1683,28 @@ mod tests { // fire notification - this should clean up the queue assert_eq!(pool.status().ready, 1); - maintain_transaction_pool( - &id, - &client, - &pool, - &[] - ).unwrap().wait().unwrap(); + maintain_transaction_pool(&id, &client, &pool, &[]) + .unwrap() + .wait() + .unwrap(); // then assert_eq!(pool.status().ready, 0); assert_eq!(pool.status().future, 0); // import second block - let builder = client.new_block_at(&BlockId::hash(best.hash()), Default::default()).unwrap(); + let builder = client + .new_block_at(&BlockId::hash(best.hash()), Default::default()) + .unwrap(); let block = builder.bake().unwrap(); let id = BlockId::hash(block.header().hash()); client.import(BlockOrigin::Own, block).unwrap(); // fire notification - this should add the transaction back to the pool. - maintain_transaction_pool( - &id, - &client, - &pool, - &[block1_hash] - ).unwrap().wait().unwrap(); + maintain_transaction_pool(&id, &client, &pool, &[block1_hash]) + .unwrap() + .wait() + .unwrap(); // then assert_eq!(pool.status().ready, 1); diff --git a/core/service/src/chain_ops.rs b/client/service/src/chain_ops.rs similarity index 99% rename from core/service/src/chain_ops.rs rename to client/service/src/chain_ops.rs index e6d7df33c2a1d..15d0189381358 100644 --- a/core/service/src/chain_ops.rs +++ b/client/service/src/chain_ops.rs @@ -156,6 +156,7 @@ macro_rules! import_blocks { body: block.body, justification: block.justification, origin: None, + allow_missing_state: false, } ]); } diff --git a/core/service/src/config.rs b/client/service/src/config.rs similarity index 87% rename from core/service/src/config.rs rename to client/service/src/config.rs index 21acae3cab051..a435195f9d8bc 100644 --- a/core/service/src/config.rs +++ b/client/service/src/config.rs @@ -43,8 +43,10 @@ pub struct Configuration { pub transaction_pool: transaction_pool::txpool::Options, /// Network configuration. pub network: NetworkConfiguration, + /// Path to the base configuration directory. + pub config_dir: Option, /// Path to key files. - pub keystore_path: PathBuf, + pub keystore_path: Option, /// Configuration for the database. pub database: DatabaseConfig, /// Size of internal state cache in Bytes @@ -71,7 +73,11 @@ pub struct Configuration { pub rpc_ws_max_connections: Option, /// CORS settings for HTTP & WS servers. `None` if all origins are allowed. pub rpc_cors: Option>, + /// prometheus service addr. `None` if disabled. + pub prometheus_endpoint: Option, /// Telemetry service URL. `None` if disabled. + pub prometheus_endpoint: Option, + /// prometheus exporter address. local. pub telemetry_endpoints: Option, /// External WASM transport for the telemetry. If `Some`, when connection to a telemetry /// endpoint, this transport will be tried in priority before all others. @@ -118,18 +124,19 @@ impl Configuration where G: RuntimeGenesis, E: Extension, { - /// Create default config for given chain spec. - pub fn default_with_spec(chain_spec: ChainSpec) -> Self { + /// Create a default config for given chain spec and path to configuration dir + pub fn default_with_spec_and_base_path(chain_spec: ChainSpec, config_dir: Option) -> Self { let mut configuration = Configuration { impl_name: "parity-substrate", impl_version: "0.0.0", impl_commit: "", chain_spec, + config_dir: config_dir.clone(), name: Default::default(), roles: Roles::FULL, transaction_pool: Default::default(), network: Default::default(), - keystore_path: Default::default(), + keystore_path: config_dir.map(|c| c.join("keystore")), database: DatabaseConfig::Path { path: Default::default(), cache_size: Default::default(), @@ -144,6 +151,7 @@ impl Configuration where rpc_ws: None, rpc_ws_max_connections: None, rpc_cors: Some(vec![]), + prometheus_endpoint: None, telemetry_endpoints: None, telemetry_external_transport: None, default_heap_pages: None, @@ -161,6 +169,9 @@ impl Configuration where configuration } +} + +impl Configuration { /// Returns full version string of this configuration. pub fn full_version(&self) -> String { full_version_from_strs(self.impl_version, self.impl_commit) @@ -170,6 +181,17 @@ impl Configuration where pub fn client_id(&self) -> String { format!("{}/v{}", self.impl_name, self.full_version()) } + + /// Generate a PathBuf to sub in the chain configuration directory + /// if given + pub fn in_chain_config_dir(&self, sub: &str) -> Option { + self.config_dir.clone().map(|mut path| { + path.push("chains"); + path.push(self.chain_spec.id()); + path.push(sub); + path + }) + } } /// Returns platform info diff --git a/core/service/src/error.rs b/client/service/src/error.rs similarity index 100% rename from core/service/src/error.rs rename to client/service/src/error.rs diff --git a/core/service/src/lib.rs b/client/service/src/lib.rs similarity index 96% rename from core/service/src/lib.rs rename to client/service/src/lib.rs index 3057b8ce4dcd4..2bc56647a6092 100644 --- a/core/service/src/lib.rs +++ b/client/service/src/lib.rs @@ -36,7 +36,7 @@ use std::time::{Duration, Instant}; use futures::sync::mpsc; use parking_lot::Mutex; -use client::{runtime_api::BlockT, Client}; +use client::Client; use exit_future::Signal; use futures::prelude::*; use futures03::{ @@ -51,7 +51,7 @@ use log::{log, warn, debug, error, Level}; use codec::{Encode, Decode}; use primitives::{Blake2Hasher, H256}; use sr_primitives::generic::BlockId; -use sr_primitives::traits::NumberFor; +use sr_primitives::traits::{NumberFor, Block as BlockT}; pub use self::error::Error; pub use self::builder::{ServiceBuilder, ServiceBuilderExport, ServiceBuilderImport, ServiceBuilderRevert}; @@ -136,7 +136,7 @@ pub trait AbstractService: 'static + Future + /// Type of block of this chain. type Block: BlockT; /// Backend storage for the client. - type Backend: 'static + client::backend::Backend; + type Backend: 'static + client_api::backend::Backend; /// How to execute calls towards the runtime. type CallExecutor: 'static + client::CallExecutor + Send + Sync + Clone; /// API that the runtime provides. @@ -203,7 +203,7 @@ impl AbstractServi NetworkService, TransactionPool, TOc> where TBl: BlockT, - TBackend: 'static + client::backend::Backend, + TBackend: 'static + client_api::backend::Backend, TExec: 'static + client::CallExecutor + Send + Sync + Clone, TRtApi: 'static + Send + Sync, TSc: consensus_common::SelectChain + 'static + Clone + Send, @@ -442,7 +442,7 @@ fn build_network_future< while let Ok(Async::Ready(Some(Event::Dht(event)))) = network.poll().map_err(|err| { warn!(target: "service", "Error in network: {:?}", err); }) { - // Given that core/authority-discovery is the only upper stack consumer of Dht events at the moment, all Dht + // Given that client/authority-discovery is the only upper stack consumer of Dht events at the moment, all Dht // events are being passed on to the authority-discovery module. In the future there might be multiple // consumers of these events. In that case this would need to be refactored to properly dispatch the events, // e.g. via a subscriber model. @@ -459,7 +459,7 @@ fn build_network_future< let polling_dur = before_polling.elapsed(); log!( target: "service", - if polling_dur >= Duration::from_millis(50) { Level::Debug } else { Level::Trace }, + if polling_dur >= Duration::from_secs(1) { Level::Warn } else { Level::Trace }, "Polling the network future took {:?}", polling_dur ); @@ -541,15 +541,16 @@ fn start_rpc_servers rpc_servers::RpcHandler components::RpcHandler>( +fn start_rpc_servers rpc_servers::RpcHandler>( _: &Configuration, _: H -) -> Result, error::Error> { +) -> Result, error::Error> { Ok(Box::new(())) } /// An RPC session. Used to perform in-memory RPC queries (ie. RPC queries that don't go through /// the HTTP or WebSockets server). +#[derive(Clone)] pub struct RpcSession { metadata: rpc::Metadata, } @@ -614,7 +615,14 @@ where self.pool.hash_of(transaction) } - fn import(&self, report_handle: ReportHandle, who: PeerId, reputation_change: i32, transaction: B::Extrinsic) { + fn import( + &self, + report_handle: ReportHandle, + who: PeerId, + reputation_change_good: i32, + reputation_change_bad: i32, + transaction: B::Extrinsic + ) { if !self.imports_external_transactions { debug!("Transaction rejected"); return; @@ -628,10 +636,13 @@ where let import_future = import_future .then(move |import_result| { match import_result { - Ok(_) => report_handle.report_peer(who, reputation_change), + Ok(_) => report_handle.report_peer(who, reputation_change_good), Err(e) => match e.into_pool_error() { Ok(txpool::error::Error::AlreadyImported(_)) => (), - Ok(e) => debug!("Error adding transaction to the pool: {:?}", e), + Ok(e) => { + report_handle.report_peer(who, reputation_change_bad); + debug!("Error adding transaction to the pool: {:?}", e) + } Err(e) => debug!("Error converting pool error: {:?}", e), } } diff --git a/core/service/src/status_sinks.rs b/client/service/src/status_sinks.rs similarity index 100% rename from core/service/src/status_sinks.rs rename to client/service/src/status_sinks.rs diff --git a/core/service/test/Cargo.toml b/client/service/test/Cargo.toml similarity index 58% rename from core/service/test/Cargo.toml rename to client/service/test/Cargo.toml index 872d63415f4e8..82b39ce767e38 100644 --- a/core/service/test/Cargo.toml +++ b/client/service/test/Cargo.toml @@ -12,9 +12,9 @@ log = "0.4.8" env_logger = "0.7.0" fdlimit = "0.1.1" futures03 = { package = "futures-preview", version = "=0.3.0-alpha.19", features = ["compat"] } -service = { package = "substrate-service", path = "../../../core/service" } -network = { package = "substrate-network", path = "../../../core/network" } -consensus = { package = "substrate-consensus-common", path = "../../../core/consensus/common" } -client = { package = "substrate-client", path = "../../../core/client" } -sr-primitives = { path = "../../../core/sr-primitives" } -primitives = { package = "substrate-primitives", path = "../../../core/primitives" } +service = { package = "substrate-service", path = "../../service", default-features = false } +network = { package = "substrate-network", path = "../../network" } +consensus = { package = "substrate-consensus-common", path = "../../../primitives/consensus/common" } +client = { package = "substrate-client", path = "../../" } +sr-primitives = { path = "../../../primitives/sr-primitives" } +primitives = { package = "substrate-primitives", path = "../../../primitives/core" } diff --git a/core/service/test/src/lib.rs b/client/service/test/src/lib.rs similarity index 98% rename from core/service/test/src/lib.rs rename to client/service/test/src/lib.rs index 27f03e09633b5..ea2275b47f960 100644 --- a/core/service/test/src/lib.rs +++ b/client/service/test/src/lib.rs @@ -158,8 +158,10 @@ fn node_config ( node_name: "unknown".to_owned(), transport: TransportConfig::Normal { enable_mdns: false, + allow_private_ipv4: true, wasm_external_transport: None, }, + max_parallel_downloads: NetworkConfiguration::default().max_parallel_downloads, }; Configuration { @@ -169,8 +171,9 @@ fn node_config ( roles: role, transaction_pool: Default::default(), network: network_config, - keystore_path: root.join("key"), + keystore_path: Some(root.join("key")), keystore_password: None, + config_dir: Some(root.clone()), database: DatabaseConfig::Path { path: root.join("db"), cache_size: None @@ -187,6 +190,7 @@ fn node_config ( rpc_ws: None, rpc_ws_max_connections: None, rpc_cors: None, + prometheus_endpoint: None, telemetry_endpoints: None, telemetry_external_transport: None, default_heap_pages: None, diff --git a/core/client/src/call_executor.rs b/client/src/call_executor.rs similarity index 65% rename from core/client/src/call_executor.rs rename to client/src/call_executor.rs index 05a2a8eba1c3e..9e7c89c8390dd 100644 --- a/core/client/src/call_executor.rs +++ b/client/src/call_executor.rs @@ -14,14 +14,14 @@ // You should have received a copy of the GNU General Public License // along with Substrate. If not, see . -use std::{sync::Arc, cmp::Ord, panic::UnwindSafe, result, cell::RefCell, rc::Rc}; +use std::{sync::Arc, panic::UnwindSafe, result, cell::RefCell}; use codec::{Encode, Decode}; use sr_primitives::{ generic::BlockId, traits::Block as BlockT, traits::NumberFor, }; use state_machine::{ self, OverlayedChanges, Ext, ExecutionManager, StateMachine, ExecutionStrategy, - backend::Backend as _, ChangesTrieTransaction, + backend::Backend as _, ChangesTrieTransaction, StorageProof, }; use executor::{RuntimeVersion, RuntimeInfo, NativeVersion}; use hash_db::Hasher; @@ -30,126 +30,10 @@ use primitives::{ traits::{CodeExecutor, KeystoreExt}, }; -use crate::runtime_api::{ProofRecorder, InitializeBlock}; -use crate::backend; -use crate::error; - -/// Method call executor. -pub trait CallExecutor -where - B: BlockT, - H: Hasher, - H::Out: Ord, -{ - /// Externalities error type. - type Error: state_machine::Error; - - /// Execute a call to a contract on top of state in a block of given hash. - /// - /// No changes are made. - fn call( - &self, - id: &BlockId, - method: &str, - call_data: &[u8], - strategy: ExecutionStrategy, - side_effects_handler: Option, - ) -> Result, error::Error>; - - /// Execute a contextual call on top of state in a block of a given hash. - /// - /// No changes are made. - /// Before executing the method, passed header is installed as the current header - /// of the execution context. - fn contextual_call< - 'a, - IB: Fn() -> error::Result<()>, - EM: Fn( - Result, Self::Error>, - Result, Self::Error> - ) -> Result, Self::Error>, - R: Encode + Decode + PartialEq, - NC: FnOnce() -> result::Result + UnwindSafe, - >( - &self, - initialize_block_fn: IB, - at: &BlockId, - method: &str, - call_data: &[u8], - changes: &RefCell, - initialize_block: InitializeBlock<'a, B>, - execution_manager: ExecutionManager, - native_call: Option, - side_effects_handler: Option, - proof_recorder: &Option>>>, - enable_keystore: bool, - ) -> error::Result> where ExecutionManager: Clone; - - /// Extract RuntimeVersion of given block - /// - /// No changes are made. - fn runtime_version(&self, id: &BlockId) -> Result; - - /// Execute a call to a contract on top of given state. - /// - /// No changes are made. - fn call_at_state< - S: state_machine::Backend, - F: FnOnce( - Result, Self::Error>, - Result, Self::Error>, - ) -> Result, Self::Error>, - R: Encode + Decode + PartialEq, - NC: FnOnce() -> result::Result + UnwindSafe, - >(&self, - state: &S, - overlay: &mut OverlayedChanges, - method: &str, - call_data: &[u8], - manager: ExecutionManager, - native_call: Option, - side_effects_handler: Option, - ) -> Result< - ( - NativeOrEncoded, - (S::Transaction, H::Out), - Option>> - ), - error::Error, - >; - - /// Execute a call to a contract on top of given state, gathering execution proof. - /// - /// No changes are made. - fn prove_at_state>( - &self, - mut state: S, - overlay: &mut OverlayedChanges, - method: &str, - call_data: &[u8] - ) -> Result<(Vec, Vec>), error::Error> { - let trie_state = state.as_trie_backend() - .ok_or_else(|| - Box::new(state_machine::ExecutionError::UnableToGenerateProof) - as Box - )?; - self.prove_at_trie_state(trie_state, overlay, method, call_data) - } - - /// Execute a call to a contract on top of given trie state, gathering execution proof. - /// - /// No changes are made. - fn prove_at_trie_state>( - &self, - trie_state: &state_machine::TrieBackend, - overlay: &mut OverlayedChanges, - method: &str, - call_data: &[u8] - ) -> Result<(Vec, Vec>), error::Error>; - - /// Get runtime version if supported. - fn native_runtime_version(&self) -> Option<&NativeVersion>; -} +use sr_api::{ProofRecorder, InitializeBlock}; +use client_api::{ + error, backend, call_executor::CallExecutor, +}; /// Call executor that executes methods locally, querying all required /// data from local backend. @@ -241,7 +125,7 @@ where execution_manager: ExecutionManager, native_call: Option, side_effects_handler: Option, - recorder: &Option>>>, + recorder: &Option>, enable_keystore: bool, ) -> Result, error::Error> where ExecutionManager: Clone { match initialize_block { @@ -377,7 +261,7 @@ where overlay: &mut OverlayedChanges, method: &str, call_data: &[u8] - ) -> Result<(Vec, Vec>), error::Error> { + ) -> Result<(Vec, StorageProof), error::Error> { state_machine::prove_execution_on_trie_backend( trie_state, overlay, diff --git a/core/client/src/cht.rs b/client/src/cht.rs similarity index 98% rename from core/client/src/cht.rs rename to client/src/cht.rs index 63a5b39c26904..874ea02c2a1a3 100644 --- a/core/client/src/cht.rs +++ b/client/src/cht.rs @@ -30,10 +30,10 @@ use trie; use primitives::{H256, convert_hash}; use sr_primitives::traits::{Header as HeaderT, SimpleArithmetic, Zero, One}; use state_machine::backend::InMemory as InMemoryState; -use state_machine::{MemoryDB, TrieBackend, Backend as StateBackend, +use state_machine::{MemoryDB, TrieBackend, Backend as StateBackend, StorageProof, prove_read_on_trie_backend, read_proof_check, read_proof_check_on_proving_backend}; -use crate::error::{Error as ClientError, Result as ClientResult}; +use client_api::error::{Error as ClientError, Result as ClientResult}; /// The size of each CHT. This value is passed to every CHT-related function from /// production code. Other values are passed from tests. @@ -88,7 +88,7 @@ pub fn build_proof( cht_num: Header::Number, blocks: BlocksI, hashes: HashesI -) -> ClientResult>> +) -> ClientResult where Header: HeaderT, Hasher: hash_db::Hasher, @@ -114,7 +114,7 @@ pub fn check_proof( local_root: Header::Hash, local_number: Header::Number, remote_hash: Header::Hash, - remote_proof: Vec> + remote_proof: StorageProof, ) -> ClientResult<()> where Header: HeaderT, diff --git a/core/client/src/client.rs b/client/src/client.rs similarity index 90% rename from core/client/src/client.rs rename to client/src/client.rs index 56a495e68383a..def038fb3e022 100644 --- a/core/client/src/client.rs +++ b/client/src/client.rs @@ -18,7 +18,7 @@ use std::{ marker::PhantomData, collections::{HashSet, BTreeMap, HashMap}, sync::Arc, - panic::UnwindSafe, result, cell::RefCell, rc::Rc, + panic::UnwindSafe, result, cell::RefCell, }; use log::{info, trace, warn}; use futures03::channel::mpsc; @@ -43,7 +43,7 @@ use state_machine::{ DBValue, Backend as StateBackend, ChangesTrieAnchorBlockId, ExecutionStrategy, ExecutionManager, prove_read, prove_child_read, ChangesTrieRootsStorage, ChangesTrieStorage, ChangesTrieTransaction, ChangesTrieConfigurationRange, key_changes, key_changes_proof, - OverlayedChanges, BackendTrustLevel, + OverlayedChanges, BackendTrustLevel, StorageProof, merge_storage_proofs, }; use executor::{RuntimeVersion, RuntimeInfo}; use consensus::{ @@ -53,33 +53,35 @@ use consensus::{ }; use header_metadata::{HeaderMetadata, CachedHeaderMetadata}; -use crate::{ - runtime_api::{ - CallRuntimeAt, ConstructRuntimeApi, Core as CoreApi, ProofRecorder, - InitializeBlock, - }, +use sr_api::{CallRuntimeAt, ConstructRuntimeApi, Core as CoreApi, ProofRecorder, InitializeBlock}; +use block_builder::BlockBuilderApi; + +pub use client_api::{ backend::{ self, BlockImportOperation, PrunableStateChangesTrieStorage, - ClientImportOperation, Finalizer, ImportSummary, + ClientImportOperation, Finalizer, ImportSummary, NewBlockState, }, blockchain::{ - self, Info as ChainInfo, Backend as ChainBackend, + self, Backend as ChainBackend, HeaderBackend as ChainHeaderBackend, ProvideCache, Cache, well_known_cache_keys::Id as CacheKeyId, }, - call_executor::{CallExecutor, LocalCallExecutor}, + client::{ + ImportNotifications, FinalityNotification, FinalityNotifications, BlockImportNotification, + ClientInfo, BlockchainEvents, BlockBody, ProvideUncles, ExecutionStrategies, ForkBlocks, + BlockOf, + }, notifications::{StorageNotifications, StorageEventStream}, - light::{call_executor::prove_execution, fetcher::ChangesProof}, - block_builder::{self, api::BlockBuilder as BlockBuilderAPI}, error::Error, - cht, error, in_mem, genesis + error, + CallExecutor }; -/// Type that implements `futures::Stream` of block import events. -pub type ImportNotifications = mpsc::UnboundedReceiver>; - -/// A stream of block finality notifications. -pub type FinalityNotifications = mpsc::UnboundedReceiver>; +use crate::{ + call_executor::LocalCallExecutor, + light::{call_executor::prove_execution, fetcher::ChangesProof}, + in_mem, genesis, cht, +}; type StorageUpdate = < < @@ -88,38 +90,6 @@ type StorageUpdate = < >::State as state_machine::Backend>::Transaction; type ChangesUpdate = ChangesTrieTransaction>; -/// Expected hashes of blocks at given heights. -/// -/// This may be used as chain spec extension to filter out known, unwanted forks. -pub type ForkBlocks = Option, ::Hash>>; - -/// Execution strategies settings. -#[derive(Debug, Clone)] -pub struct ExecutionStrategies { - /// Execution strategy used when syncing. - pub syncing: ExecutionStrategy, - /// Execution strategy used when importing blocks. - pub importing: ExecutionStrategy, - /// Execution strategy used when constructing blocks. - pub block_construction: ExecutionStrategy, - /// Execution strategy used for offchain workers. - pub offchain_worker: ExecutionStrategy, - /// Execution strategy used in other cases. - pub other: ExecutionStrategy, -} - -impl Default for ExecutionStrategies { - fn default() -> ExecutionStrategies { - ExecutionStrategies { - syncing: ExecutionStrategy::NativeElseWasm, - importing: ExecutionStrategy::NativeElseWasm, - block_construction: ExecutionStrategy::AlwaysWasm, - offchain_worker: ExecutionStrategy::NativeWhenPossible, - other: ExecutionStrategy::NativeElseWasm, - } - } -} - /// Substrate Client pub struct Client where Block: BlockT { backend: Arc, @@ -134,74 +104,6 @@ pub struct Client where Block: BlockT { _phantom: PhantomData, } -/// A source of blockchain events. -pub trait BlockchainEvents { - /// Get block import event stream. Not guaranteed to be fired for every - /// imported block. - fn import_notification_stream(&self) -> ImportNotifications; - - /// Get a stream of finality notifications. Not guaranteed to be fired for every - /// finalized block. - fn finality_notification_stream(&self) -> FinalityNotifications; - - /// Get storage changes event stream. - /// - /// Passing `None` as `filter_keys` subscribes to all storage changes. - fn storage_changes_notification_stream( - &self, - filter_keys: Option<&[StorageKey]>, - child_filter_keys: Option<&[(StorageKey, Option>)]>, - ) -> error::Result>; -} - -/// Fetch block body by ID. -pub trait BlockBody { - /// Get block body by ID. Returns `None` if the body is not stored. - fn block_body(&self, - id: &BlockId - ) -> error::Result::Extrinsic>>>; -} - -/// Provide a list of potential uncle headers for a given block. -pub trait ProvideUncles { - /// Gets the uncles of the block with `target_hash` going back `max_generation` ancestors. - fn uncles(&self, target_hash: Block::Hash, max_generation: NumberFor) - -> error::Result>; -} - -/// Client info -#[derive(Debug)] -pub struct ClientInfo { - /// Best block hash. - pub chain: ChainInfo, - /// State Cache Size currently used by the backend - pub used_state_cache_size: Option, -} - -/// Summary of an imported block -#[derive(Clone, Debug)] -pub struct BlockImportNotification { - /// Imported block header hash. - pub hash: Block::Hash, - /// Imported block origin. - pub origin: BlockOrigin, - /// Imported block header. - pub header: Block::Header, - /// Is this the new best block. - pub is_new_best: bool, - /// List of retracted blocks ordered by block number. - pub retracted: Vec, -} - -/// Summary of a finalized block. -#[derive(Clone, Debug)] -pub struct FinalityNotification { - /// Imported block header hash. - pub hash: Block::Hash, - /// Imported block header. - pub header: Block::Header, -} - // used in importing a block, where additional changes are made after the runtime // executed. enum PrePostHeader { @@ -273,12 +175,6 @@ pub fn new_with_backend( Client::new(backend, call_executor, build_genesis_storage, Default::default(), Default::default()) } -/// Figure out the block type for a given type (for now, just a `Client`). -pub trait BlockOf { - /// The type of the block. - type Type: BlockT; -} - impl BlockOf for Client where B: backend::Backend, E: CallExecutor, @@ -314,7 +210,7 @@ impl Client where genesis_block.deconstruct().0, Some(vec![]), None, - crate::backend::NewBlockState::Final + NewBlockState::Final )?; backend.commit_operation(op)?; } @@ -421,7 +317,7 @@ impl Client where } /// Reads storage value at a given block + key, returning read proof. - pub fn read_proof(&self, id: &BlockId, keys: I) -> error::Result>> where + pub fn read_proof(&self, id: &BlockId, keys: I) -> error::Result where I: IntoIterator, I::Item: AsRef<[u8]>, { @@ -437,7 +333,7 @@ impl Client where id: &BlockId, storage_key: &[u8], keys: I, - ) -> error::Result>> where + ) -> error::Result where I: IntoIterator, I::Item: AsRef<[u8]>, { @@ -454,14 +350,14 @@ impl Client where id: &BlockId, method: &str, call_data: &[u8] - ) -> error::Result<(Vec, Vec>)> { + ) -> error::Result<(Vec, StorageProof)> { let state = self.state_at(id)?; let header = self.prepare_environment_block(id)?; prove_execution(state, header, &self.executor, method, call_data) } /// Reads given header and generates CHT-based header proof. - pub fn header_proof(&self, id: &BlockId) -> error::Result<(Block::Header, Vec>)> { + pub fn header_proof(&self, id: &BlockId) -> error::Result<(Block::Header, StorageProof)> { self.header_proof_with_cht_size(id, cht::size()) } @@ -477,7 +373,7 @@ impl Client where &self, id: &BlockId, cht_size: NumberFor, - ) -> error::Result<(Block::Header, Vec>)> { + ) -> error::Result<(Block::Header, StorageProof)> { let proof_error = || error::Error::Backend(format!("Failed to generate header proof for {:?}", id)); let header = self.backend.blockchain().expect_header(*id)?; let block_num = *header.number(); @@ -497,7 +393,7 @@ impl Client where /// Get longest range within [first; last] that is possible to use in `key_changes` /// and `key_changes_proof` calls. /// Range could be shortened from the beginning if some changes tries have been pruned. - /// Returns Ok(None) if changes trues are not supported. + /// Returns Ok(None) if changes tries are not supported. pub fn max_key_changes_range( &self, first: NumberFor, @@ -696,18 +592,18 @@ impl Client where &self, cht_size: NumberFor, blocks: I - ) -> error::Result>> { + ) -> error::Result { // most probably we have touched several changes tries that are parts of the single CHT // => GroupBy changes tries by CHT number and then gather proof for the whole group at once - let mut proof = HashSet::new(); + let mut proofs = Vec::new(); cht::for_each_cht_group::(cht_size, blocks, |_, cht_num, cht_blocks| { let cht_proof = self.changes_trie_roots_proof_at_cht(cht_size, cht_num, cht_blocks)?; - proof.extend(cht_proof); + proofs.push(cht_proof); Ok(()) }, ())?; - Ok(proof.into_iter().collect()) + Ok(merge_storage_proofs(proofs)) } /// Generates CHT-based proof for roots of changes tries at given blocks (that are part of single CHT). @@ -716,7 +612,7 @@ impl Client where cht_size: NumberFor, cht_num: NumberFor, blocks: Vec> - ) -> error::Result>> { + ) -> error::Result { let cht_start = cht::start_number(cht_size, cht_num); let mut current_num = cht_start; let cht_range = ::std::iter::from_fn(|| { @@ -749,9 +645,16 @@ impl Client where E: Clone + Send + Sync, RA: Send + Sync, Self: ProvideRuntimeApi, - ::Api: BlockBuilderAPI + ::Api: BlockBuilderApi { - block_builder::BlockBuilder::new(self, inherent_digests) + let info = self.info(); + block_builder::BlockBuilder::new( + self, + info.chain.best_hash, + info.chain.best_number, + false, + inherent_digests, + ) } /// Create a new block, built on top of `parent`. @@ -763,9 +666,15 @@ impl Client where E: Clone + Send + Sync, RA: Send + Sync, Self: ProvideRuntimeApi, - ::Api: BlockBuilderAPI + ::Api: BlockBuilderApi { - block_builder::BlockBuilder::at_block(parent, &self, false, inherent_digests) + block_builder::BlockBuilder::new( + self, + self.expect_block_hash_from_id(parent)?, + self.expect_block_number_from_id(parent)?, + false, + inherent_digests, + ) } /// Create a new block, built on top of `parent` with proof recording enabled. @@ -781,9 +690,15 @@ impl Client where E: Clone + Send + Sync, RA: Send + Sync, Self: ProvideRuntimeApi, - ::Api: BlockBuilderAPI + ::Api: BlockBuilderApi { - block_builder::BlockBuilder::at_block(parent, &self, true, inherent_digests) + block_builder::BlockBuilder::new( + self, + self.expect_block_hash_from_id(parent)?, + self.expect_block_number_from_id(parent)?, + true, + inherent_digests, + ) } /// Lock the import lock, and run operations inside. @@ -838,15 +753,22 @@ impl Client where finalized, auxiliary, fork_choice, + allow_missing_state, } = import_block; assert!(justification.is_some() && finalized || justification.is_none()); let parent_hash = header.parent_hash().clone(); + let mut enact_state = true; - match self.backend.blockchain().status(BlockId::Hash(parent_hash))? { - blockchain::BlockStatus::InChain => {}, - blockchain::BlockStatus::Unknown => return Ok(ImportResult::UnknownParent), + match self.block_status(&BlockId::Hash(parent_hash))? { + BlockStatus::Unknown => return Ok(ImportResult::UnknownParent), + BlockStatus::InChainWithState | BlockStatus::Queued => {}, + BlockStatus::InChainPruned if allow_missing_state => { + enact_state = false; + }, + BlockStatus::InChainPruned => return Ok(ImportResult::MissingState), + BlockStatus::KnownBad => return Ok(ImportResult::KnownBad), } let import_headers = if post_digests.is_empty() { @@ -875,6 +797,7 @@ impl Client where finalized, auxiliary, fork_choice, + enact_state, ); if let Ok(ImportResult::Imported(ref aux)) = result { @@ -902,6 +825,7 @@ impl Client where finalized: bool, aux: Vec<(Vec, Option>)>, fork_choice: ForkChoiceStrategy, + enact_state: bool, ) -> error::Result where E: CallExecutor + Send + Sync + Clone, { @@ -927,22 +851,39 @@ impl Client where BlockOrigin::Genesis | BlockOrigin::NetworkInitialSync | BlockOrigin::File => false, }; - self.backend.begin_state_operation(&mut operation.op, BlockId::Hash(parent_hash))?; + let storage_changes = match &body { + Some(body) if enact_state => { + self.backend.begin_state_operation(&mut operation.op, BlockId::Hash(parent_hash))?; - // ensure parent block is finalized to maintain invariant that - // finality is called sequentially. - if finalized { - self.apply_finality_with_block_hash(operation, parent_hash, None, info.best_hash, make_notifications)?; - } + // ensure parent block is finalized to maintain invariant that + // finality is called sequentially. + if finalized { + self.apply_finality_with_block_hash(operation, parent_hash, None, info.best_hash, make_notifications)?; + } - // FIXME #1232: correct path logic for when to execute this function - let (storage_update, changes_update, storage_changes) = self.block_execution( - &operation.op, - &import_headers, - origin, - hash, - body.clone(), - )?; + // FIXME #1232: correct path logic for when to execute this function + let (storage_update, changes_update, storage_changes) = self.block_execution( + &operation.op, + &import_headers, + origin, + hash, + &body, + )?; + + operation.op.update_cache(new_cache); + if let Some(storage_update) = storage_update { + operation.op.update_db_storage(storage_update)?; + } + if let Some(storage_changes) = storage_changes.clone() { + operation.op.update_storage(storage_changes.0, storage_changes.1)?; + } + if let Some(Some(changes_update)) = changes_update { + operation.op.update_changes_trie(changes_update)?; + } + storage_changes + }, + _ => None, + }; let is_new_best = finalized || match fork_choice { ForkChoiceStrategy::LongestChain => import_headers.post().number() > &info.best_number, @@ -950,11 +891,11 @@ impl Client where }; let leaf_state = if finalized { - crate::backend::NewBlockState::Final + NewBlockState::Final } else if is_new_best { - crate::backend::NewBlockState::Best + NewBlockState::Best } else { - crate::backend::NewBlockState::Normal + NewBlockState::Normal }; let retracted = if is_new_best { @@ -977,17 +918,6 @@ impl Client where leaf_state, )?; - operation.op.update_cache(new_cache); - if let Some(storage_update) = storage_update { - operation.op.update_db_storage(storage_update)?; - } - if let Some(storage_changes) = storage_changes.clone() { - operation.op.update_storage(storage_changes.0, storage_changes.1)?; - } - if let Some(Some(changes_update)) = changes_update { - operation.op.update_changes_trie(changes_update)?; - } - operation.op.insert_aux(aux)?; if make_notifications { @@ -1014,7 +944,7 @@ impl Client where import_headers: &PrePostHeader, origin: BlockOrigin, hash: Block::Hash, - body: Option>, + body: &[Block::Extrinsic], ) -> error::Result<( Option>, Option>>, @@ -1052,7 +982,7 @@ impl Client where let encoded_block = ::encode_from( import_headers.pre(), - &body.unwrap_or_default() + body, ); let (_, storage_update, changes_update) = self.executor @@ -1352,7 +1282,7 @@ impl ChainHeaderBackend for Client wher B: backend::Backend, E: CallExecutor + Send + Sync, Block: BlockT, - RA: Send + Sync + RA: Send + Sync, { fn header(&self, id: BlockId) -> error::Result> { self.backend.blockchain().header(id) @@ -1375,6 +1305,23 @@ impl ChainHeaderBackend for Client wher } } +impl sr_primitives::traits::BlockIdTo for Client where + B: backend::Backend, + E: CallExecutor + Send + Sync, + Block: BlockT, + RA: Send + Sync, +{ + type Error = Error; + + fn to_hash(&self, block_id: &BlockId) -> error::Result> { + self.block_hash_from_id(block_id) + } + + fn to_number(&self, block_id: &BlockId) -> error::Result>> { + self.block_number_from_id(block_id) + } +} + impl ChainHeaderBackend for &Client where B: backend::Backend, E: CallExecutor + Send + Sync, @@ -1429,11 +1376,13 @@ impl CallRuntimeAt for Client where E: CallExecutor + Clone + Send + Sync, Block: BlockT, { + type Error = Error; + fn call_api_at< 'a, R: Encode + Decode + PartialEq, NC: FnOnce() -> result::Result + UnwindSafe, - C: CoreApi, + C: CoreApi, >( &self, core_api: &C, @@ -1444,7 +1393,7 @@ impl CallRuntimeAt for Client where initialize_block: InitializeBlock<'a, Block>, native_call: Option, context: ExecutionContext, - recorder: &Option>>>, + recorder: &Option>, ) -> error::Result> { let manager = match context { ExecutionContext::BlockConstruction => @@ -1523,7 +1472,7 @@ impl<'a, B, E, Block, RA> consensus::BlockImport for &'a Client, ) -> Result { - let BlockCheckParams { hash, number, parent_hash } = block; + let BlockCheckParams { hash, number, parent_hash, allow_missing_state } = block; if let Some(h) = self.fork_blocks.as_ref().and_then(|x| x.get(&number)) { if &hash != h { @@ -1537,22 +1486,27 @@ impl<'a, B, E, Block, RA> consensus::BlockImport for &'a Client {}, - BlockStatus::Unknown | BlockStatus::InChainPruned => return Ok(ImportResult::UnknownParent), - BlockStatus::KnownBad => return Ok(ImportResult::KnownBad), - } - + // Own status must be checked first. If the block and ancestry is pruned + // this function must return `AlreadyInChain` rather than `MissingState` match self.block_status(&BlockId::Hash(hash)) .map_err(|e| ConsensusError::ClientImport(e.to_string()))? { BlockStatus::InChainWithState | BlockStatus::Queued => return Ok(ImportResult::AlreadyInChain), - BlockStatus::Unknown | BlockStatus::InChainPruned => {}, + BlockStatus::InChainPruned => return Ok(ImportResult::AlreadyInChain), + BlockStatus::Unknown => {}, BlockStatus::KnownBad => return Ok(ImportResult::KnownBad), } + match self.block_status(&BlockId::Hash(parent_hash)) + .map_err(|e| ConsensusError::ClientImport(e.to_string()))? + { + BlockStatus::InChainWithState | BlockStatus::Queued => {}, + BlockStatus::Unknown => return Ok(ImportResult::UnknownParent), + BlockStatus::InChainPruned if allow_missing_state => {}, + BlockStatus::InChainPruned => return Ok(ImportResult::MissingState), + BlockStatus::KnownBad => return Ok(ImportResult::KnownBad), + } + Ok(ImportResult::imported(false)) } @@ -1762,7 +1716,7 @@ impl backend::AuxStore for Client } /// Query auxiliary data from key-value store. fn get_aux(&self, key: &[u8]) -> error::Result>> { - crate::backend::AuxStore::get_aux(&*self.backend, key) + backend::AuxStore::get_aux(&*self.backend, key) } } @@ -1809,49 +1763,6 @@ where ) } -/// Utility methods for the client. -pub mod utils { - use super::*; - use crate::error; - use primitives::H256; - use std::borrow::Borrow; - - /// Returns a function for checking block ancestry, the returned function will - /// return `true` if the given hash (second parameter) is a descendent of the - /// base (first parameter). If the `current` parameter is defined, it should - /// represent the current block `hash` and its `parent hash`, if given the - /// function that's returned will assume that `hash` isn't part of the local DB - /// yet, and all searches in the DB will instead reference the parent. - pub fn is_descendent_of<'a, Block: BlockT, T, H: Borrow + 'a>( - client: &'a T, - current: Option<(H, H)>, - ) -> impl Fn(&H256, &H256) -> Result + 'a - where T: ChainHeaderBackend + HeaderMetadata, - { - move |base, hash| { - if base == hash { return Ok(false); } - - let current = current.as_ref().map(|(c, p)| (c.borrow(), p.borrow())); - - let mut hash = hash; - if let Some((current_hash, current_parent_hash)) = current { - if base == current_hash { return Ok(false); } - if hash == current_hash { - if base == current_parent_hash { - return Ok(true); - } else { - hash = current_parent_hash; - } - } - } - - let ancestor = header_metadata::lowest_common_ancestor(client, *hash, *base)?; - - Ok(ancestor.hash == *base) - } - } -} - impl consensus::block_validation::Chain for Client where BE: backend::Backend, E: CallExecutor, @@ -1868,9 +1779,10 @@ pub(crate) mod tests { use super::*; use primitives::blake2_256; use sr_primitives::DigestItem; - use consensus::{BlockOrigin, SelectChain}; + use consensus::{BlockOrigin, SelectChain, BlockImport}; use test_client::{ prelude::*, + client_ext::ClientExt, client_db::{Backend, DatabaseSettings, DatabaseSettingsSrc, PruningMode}, runtime::{self, Block, Transfer, RuntimeApi, TestAPI}, }; @@ -2654,7 +2566,7 @@ pub(crate) mod tests { // we finalize block B1 which is on a different branch from current best // which should trigger a re-org. - client.finalize_block(BlockId::Hash(b1.hash()), None).unwrap(); + ClientExt::finalize_block(&client, BlockId::Hash(b1.hash()), None).unwrap(); // B1 should now be the latest finalized assert_eq!( @@ -2800,7 +2712,7 @@ pub(crate) mod tests { // we will finalize A2 which should make it impossible to import a new // B3 at the same height but that doesnt't include it - client.finalize_block(BlockId::Hash(a2.hash()), None).unwrap(); + ClientExt::finalize_block(&client, BlockId::Hash(a2.hash()), None).unwrap(); let b3 = client.new_block_at(&BlockId::Hash(b2.hash()), Default::default()) .unwrap().bake().unwrap(); @@ -2838,4 +2750,103 @@ pub(crate) mod tests { expected_err.to_string(), ); } + + #[test] + fn returns_status_for_pruned_blocks() { + let _ = env_logger::try_init(); + let tmp = tempfile::tempdir().unwrap(); + + // set to prune after 1 block + // states + let backend = Arc::new(Backend::new( + DatabaseSettings { + state_cache_size: 1 << 20, + state_cache_child_ratio: None, + pruning: PruningMode::keep_blocks(1), + source: DatabaseSettingsSrc::Path { + path: tmp.path().into(), + cache_size: None, + } + }, + u64::max_value(), + ).unwrap()); + + let mut client = TestClientBuilder::with_backend(backend).build(); + + let a1 = client.new_block_at(&BlockId::Number(0), Default::default()) + .unwrap().bake().unwrap(); + + let mut b1 = client.new_block_at(&BlockId::Number(0), Default::default()).unwrap(); + + // b1 is created, but not imported + b1.push_transfer(Transfer { + from: AccountKeyring::Alice.into(), + to: AccountKeyring::Ferdie.into(), + amount: 1, + nonce: 0, + }).unwrap(); + let b1 = b1.bake().unwrap(); + + let check_block_a1 = BlockCheckParams { + hash: a1.hash().clone(), + number: 0, + parent_hash: a1.header().parent_hash().clone(), + allow_missing_state: false + }; + + assert_eq!(client.check_block(check_block_a1.clone()).unwrap(), ImportResult::imported(false)); + assert_eq!(client.block_status(&BlockId::hash(check_block_a1.hash)).unwrap(), BlockStatus::Unknown); + + client.import_as_final(BlockOrigin::Own, a1.clone()).unwrap(); + + assert_eq!(client.check_block(check_block_a1.clone()).unwrap(), ImportResult::AlreadyInChain); + assert_eq!(client.block_status(&BlockId::hash(check_block_a1.hash)).unwrap(), BlockStatus::InChainWithState); + + let a2 = client.new_block_at(&BlockId::Hash(a1.hash()), Default::default()) + .unwrap().bake().unwrap(); + client.import_as_final(BlockOrigin::Own, a2.clone()).unwrap(); + + let check_block_a2 = BlockCheckParams { + hash: a2.hash().clone(), + number: 1, + parent_hash: a1.header().parent_hash().clone(), + allow_missing_state: false + }; + + assert_eq!(client.check_block(check_block_a1.clone()).unwrap(), ImportResult::AlreadyInChain); + assert_eq!(client.block_status(&BlockId::hash(check_block_a1.hash)).unwrap(), BlockStatus::InChainPruned); + assert_eq!(client.check_block(check_block_a2.clone()).unwrap(), ImportResult::AlreadyInChain); + assert_eq!(client.block_status(&BlockId::hash(check_block_a2.hash)).unwrap(), BlockStatus::InChainWithState); + + let a3 = client.new_block_at(&BlockId::Hash(a2.hash()), Default::default()) + .unwrap().bake().unwrap(); + + client.import_as_final(BlockOrigin::Own, a3.clone()).unwrap(); + let check_block_a3 = BlockCheckParams { + hash: a3.hash().clone(), + number: 2, + parent_hash: a2.header().parent_hash().clone(), + allow_missing_state: false + }; + + // a1 and a2 are both pruned at this point + assert_eq!(client.check_block(check_block_a1.clone()).unwrap(), ImportResult::AlreadyInChain); + assert_eq!(client.block_status(&BlockId::hash(check_block_a1.hash)).unwrap(), BlockStatus::InChainPruned); + assert_eq!(client.check_block(check_block_a2.clone()).unwrap(), ImportResult::AlreadyInChain); + assert_eq!(client.block_status(&BlockId::hash(check_block_a2.hash)).unwrap(), BlockStatus::InChainPruned); + assert_eq!(client.check_block(check_block_a3.clone()).unwrap(), ImportResult::AlreadyInChain); + assert_eq!(client.block_status(&BlockId::hash(check_block_a3.hash)).unwrap(), BlockStatus::InChainWithState); + + let mut check_block_b1 = BlockCheckParams { + hash: b1.hash().clone(), + number: 0, + parent_hash: b1.header().parent_hash().clone(), + allow_missing_state: false + }; + assert_eq!(client.check_block(check_block_b1.clone()).unwrap(), ImportResult::MissingState); + check_block_b1.allow_missing_state = true; + assert_eq!(client.check_block(check_block_b1.clone()).unwrap(), ImportResult::imported(false)); + check_block_b1.parent_hash = H256::random(); + assert_eq!(client.check_block(check_block_b1.clone()).unwrap(), ImportResult::UnknownParent); + } } diff --git a/core/client/src/genesis.rs b/client/src/genesis.rs similarity index 100% rename from core/client/src/genesis.rs rename to client/src/genesis.rs diff --git a/core/client/src/in_mem.rs b/client/src/in_mem.rs similarity index 94% rename from core/client/src/in_mem.rs rename to client/src/in_mem.rs index 5c35400d7743c..4c74943189020 100644 --- a/core/client/src/in_mem.rs +++ b/client/src/in_mem.rs @@ -29,13 +29,17 @@ use hash_db::{Hasher, Prefix}; use trie::MemoryDB; use header_metadata::{CachedHeaderMetadata, HeaderMetadata}; -use crate::error; -use crate::backend::{self, NewBlockState, StorageCollection, ChildStorageCollection}; -use crate::light; -use crate::leaves::LeafSet; -use crate::blockchain::{ - self, BlockStatus, HeaderBackend, well_known_cache_keys::Id as CacheKeyId +use client_api::{ + error, + backend::{self, NewBlockState, StorageCollection, ChildStorageCollection}, + blockchain::{ + self, BlockStatus, HeaderBackend, well_known_cache_keys::Id as CacheKeyId + }, + offchain::{ + InMemOffchainStorage as OffchainStorage + } }; +use crate::leaves::LeafSet; struct PendingBlock { block: StoredBlock, @@ -154,7 +158,7 @@ impl Blockchain { justification: Option, body: Option::Extrinsic>>, new_state: NewBlockState, - ) -> crate::error::Result<()> { + ) -> client_api::error::Result<()> { let number = header.number().clone(); if new_state.is_best() { self.apply_head(&header)?; @@ -395,7 +399,7 @@ impl backend::AuxStore for Blockchain { } } -impl light::blockchain::Storage for Blockchain +impl client_api::light::Storage for Blockchain where Block::Hash: From<[u8; 32]>, { @@ -805,50 +809,9 @@ pub fn check_genesis_storage(top: &StorageOverlay, children: &ChildrenStorageOve Ok(()) } -/// In-memory storage for offchain workers. -#[derive(Debug, Clone, Default)] -pub struct OffchainStorage { - storage: HashMap, Vec>, -} - -impl backend::OffchainStorage for OffchainStorage { - fn set(&mut self, prefix: &[u8], key: &[u8], value: &[u8]) { - let key = prefix.iter().chain(key).cloned().collect(); - self.storage.insert(key, value.to_vec()); - } - - fn get(&self, prefix: &[u8], key: &[u8]) -> Option> { - let key: Vec = prefix.iter().chain(key).cloned().collect(); - self.storage.get(&key).cloned() - } - - fn compare_and_set( - &mut self, - prefix: &[u8], - key: &[u8], - old_value: Option<&[u8]>, - new_value: &[u8], - ) -> bool { - use std::collections::hash_map::Entry; - let key = prefix.iter().chain(key).cloned().collect(); - - match self.storage.entry(key) { - Entry::Vacant(entry) => if old_value.is_none() { - entry.insert(new_value.to_vec()); - true - } else { false }, - Entry::Occupied(ref mut entry) if Some(entry.get().as_slice()) == old_value => { - entry.insert(new_value.to_vec()); - true - }, - _ => false, - } - } -} - #[cfg(test)] mod tests { - use super::*; + use client_api::offchain::{OffchainStorage, InMemOffchainStorage}; use std::sync::Arc; use test_client; use primitives::Blake2Hasher; @@ -871,9 +834,8 @@ mod tests { #[test] fn in_memory_offchain_storage() { - use crate::backend::OffchainStorage as _; - let mut storage = OffchainStorage::default(); + let mut storage = InMemOffchainStorage::default(); assert_eq!(storage.get(b"A", b"B"), None); assert_eq!(storage.get(b"B", b"A"), None); diff --git a/core/client/src/leaves.rs b/client/src/leaves.rs similarity index 99% rename from core/client/src/leaves.rs rename to client/src/leaves.rs index 825c1bf7c0bb5..bc42e83a66f57 100644 --- a/core/client/src/leaves.rs +++ b/client/src/leaves.rs @@ -21,7 +21,7 @@ use std::cmp::Reverse; use kvdb::{KeyValueDB, DBTransaction}; use sr_primitives::traits::SimpleArithmetic; use codec::{Encode, Decode}; -use crate::error; +use client_api::error; #[derive(Debug, Clone, PartialEq, Eq)] struct LeafSetItem { diff --git a/core/client/src/lib.rs b/client/src/lib.rs similarity index 76% rename from core/client/src/lib.rs rename to client/src/lib.rs index 5a2cb746f3864..21a1c415d54a5 100644 --- a/core/client/src/lib.rs +++ b/client/src/lib.rs @@ -73,60 +73,33 @@ //! ``` //! -#![cfg_attr(not(feature = "std"), no_std)] #![warn(missing_docs)] #![recursion_limit="128"] -#[macro_use] -pub mod runtime_api; -#[cfg(feature = "std")] -pub mod error; -#[cfg(feature = "std")] -pub mod blockchain; -#[cfg(feature = "std")] -pub mod backend; -#[cfg(feature = "std")] pub mod cht; -#[cfg(feature = "std")] pub mod in_mem; -#[cfg(feature = "std")] pub mod genesis; -pub mod block_builder; -#[cfg(feature = "std")] pub mod light; -#[cfg(feature = "std")] pub mod leaves; -#[cfg(feature = "std")] -pub mod children; -#[cfg(feature = "std")] mod call_executor; -#[cfg(feature = "std")] mod client; -#[cfg(feature = "std")] -mod notifications; - -#[cfg(feature = "std")] -pub use crate::blockchain::Info as ChainInfo; -#[cfg(feature = "std")] -pub use crate::call_executor::{CallExecutor, LocalCallExecutor}; -#[cfg(feature = "std")] +pub use client_api::{ + error, + blockchain, + blockchain::well_known_cache_keys, + blockchain::Info as ChainInfo, + notifications::{StorageEventStream, StorageChangeSet}, + call_executor::CallExecutor, + utils, +}; +pub use crate::call_executor::LocalCallExecutor; pub use crate::client::{ new_with_backend, new_in_mem, BlockBody, ImportNotifications, FinalityNotifications, BlockchainEvents, BlockImportNotification, Client, ClientInfo, ExecutionStrategies, FinalityNotification, - LongestChain, BlockOf, ProvideUncles, ForkBlocks, - utils, apply_aux, + LongestChain, BlockOf, ProvideUncles, ForkBlocks, apply_aux, }; -#[cfg(feature = "std")] -pub use crate::notifications::{StorageEventStream, StorageChangeSet}; -#[cfg(feature = "std")] -pub use state_machine::ExecutionStrategy; -#[cfg(feature = "std")] +pub use state_machine::{ExecutionStrategy, StorageProof}; pub use crate::leaves::LeafSet; -#[cfg(feature = "std")] -pub use crate::blockchain::well_known_cache_keys; - -#[doc(inline)] -pub use sr_api_macros::{decl_runtime_apis, impl_runtime_apis}; diff --git a/core/client/src/light/backend.rs b/client/src/light/backend.rs similarity index 96% rename from core/client/src/light/backend.rs rename to client/src/light/backend.rs index 300d140630d85..ec4543191896e 100644 --- a/core/client/src/light/backend.rs +++ b/client/src/light/backend.rs @@ -25,13 +25,21 @@ use sr_primitives::{generic::BlockId, Justification, StorageOverlay, ChildrenSto use state_machine::{Backend as StateBackend, TrieBackend, backend::InMemory as InMemoryState, ChangesTrieTransaction}; use sr_primitives::traits::{Block as BlockT, NumberFor, Zero, Header}; use crate::in_mem::{self, check_genesis_storage}; -use crate::backend::{ - AuxStore, Backend as ClientBackend, BlockImportOperation, RemoteBackend, NewBlockState, - StorageCollection, ChildStorageCollection, +use client_api::{ + backend::{ + AuxStore, Backend as ClientBackend, BlockImportOperation, RemoteBackend, NewBlockState, + StorageCollection, ChildStorageCollection, + }, + blockchain::{ + HeaderBackend as BlockchainHeaderBackend, well_known_cache_keys, + }, + error::{ + Error as ClientError, Result as ClientResult + }, + light::Storage as BlockchainStorage, + InMemOffchainStorage, }; -use crate::blockchain::{HeaderBackend as BlockchainHeaderBackend, well_known_cache_keys}; -use crate::error::{Error as ClientError, Result as ClientResult}; -use crate::light::blockchain::{Blockchain, Storage as BlockchainStorage}; +use crate::light::blockchain::{Blockchain}; use hash_db::Hasher; use trie::MemoryDB; @@ -107,7 +115,7 @@ impl ClientBackend for Backend where type Blockchain = Blockchain; type State = GenesisOrUnavailableState; type ChangesTrieStorage = in_mem::ChangesTrieStorage; - type OffchainStorage = in_mem::OffchainStorage; + type OffchainStorage = InMemOffchainStorage; fn begin_operation(&self) -> ClientResult { Ok(ImportOperation { @@ -442,7 +450,7 @@ impl StateBackend for GenesisOrUnavailableState mod tests { use primitives::Blake2Hasher; use test_client::{self, runtime::Block}; - use crate::backend::NewBlockState; + use client_api::backend::NewBlockState; use crate::light::blockchain::tests::{DummyBlockchain, DummyStorage}; use super::*; diff --git a/core/client/src/light/blockchain.rs b/client/src/light/blockchain.rs similarity index 78% rename from core/client/src/light/blockchain.rs rename to client/src/light/blockchain.rs index 202f94cd74fed..aa1866ae4d789 100644 --- a/core/client/src/light/blockchain.rs +++ b/client/src/light/blockchain.rs @@ -18,85 +18,31 @@ //! blocks. CHT roots are stored for headers of ancient blocks. use std::future::Future; -use std::{sync::Arc, collections::HashMap}; +use std::sync::Arc; use sr_primitives::{Justification, generic::BlockId}; use sr_primitives::traits::{Block as BlockT, Header as HeaderT, NumberFor, Zero}; use header_metadata::{HeaderMetadata, CachedHeaderMetadata}; - -use crate::backend::{AuxStore, NewBlockState}; -use crate::blockchain::{ - Backend as BlockchainBackend, BlockStatus, Cache as BlockchainCache, - HeaderBackend as BlockchainHeaderBackend, Info as BlockchainInfo, ProvideCache, - well_known_cache_keys +pub use client_api::{ + backend::{ + AuxStore, NewBlockState + }, + blockchain::{ + Backend as BlockchainBackend, BlockStatus, Cache as BlockchainCache, + HeaderBackend as BlockchainHeaderBackend, Info as BlockchainInfo, ProvideCache, + well_known_cache_keys, + }, + error::{ + Error as ClientError, Result as ClientResult + }, + light::{ + RemoteBlockchain, LocalOrRemote, Storage + } }; use crate::cht; -use crate::error::{Error as ClientError, Result as ClientResult}; use crate::light::fetcher::{Fetcher, RemoteHeaderRequest}; -/// Light client blockchain storage. -pub trait Storage: AuxStore + BlockchainHeaderBackend + HeaderMetadata { - /// Store new header. Should refuse to revert any finalized blocks. - /// - /// Takes new authorities, the leaf state of the new block, and - /// any auxiliary storage updates to place in the same operation. - fn import_header( - &self, - header: Block::Header, - cache: HashMap>, - state: NewBlockState, - aux_ops: Vec<(Vec, Option>)>, - ) -> ClientResult<()>; - - /// Set an existing block as new best block. - fn set_head(&self, block: BlockId) -> ClientResult<()>; - - /// Mark historic header as finalized. - fn finalize_header(&self, block: BlockId) -> ClientResult<()>; - - /// Get last finalized header. - fn last_finalized(&self) -> ClientResult; - - /// Get headers CHT root for given block. Fails if the block is not pruned (not a part of any CHT). - fn header_cht_root( - &self, - cht_size: NumberFor, - block: NumberFor, - ) -> ClientResult; - - /// Get changes trie CHT root for given block. Fails if the block is not pruned (not a part of any CHT). - fn changes_trie_cht_root( - &self, - cht_size: NumberFor, - block: NumberFor, - ) -> ClientResult; - - /// Get storage cache. - fn cache(&self) -> Option>>; -} - -/// Remote header. -#[derive(Debug)] -pub enum LocalOrRemote { - /// When data is available locally, it is returned. - Local(Data), - /// When data is unavailable locally, the request to fetch it from remote node is returned. - Remote(Request), - /// When data is unknown. - Unknown, -} - -/// Futures-based blockchain backend that either resolves blockchain data -/// locally, or fetches required data from remote node. -pub trait RemoteBlockchain: Send + Sync { - /// Get block header. - fn header(&self, id: BlockId) -> ClientResult, - >>; -} - /// Light client blockchain. pub struct Blockchain { storage: S, @@ -250,7 +196,7 @@ pub mod tests { use std::collections::HashMap; use parking_lot::Mutex; use test_client::runtime::{Hash, Block, Header}; - use crate::blockchain::Info; + use client_api::blockchain::Info; use super::*; pub type DummyBlockchain = Blockchain; diff --git a/core/client/src/light/call_executor.rs b/client/src/light/call_executor.rs similarity index 94% rename from core/client/src/light/call_executor.rs rename to client/src/light/call_executor.rs index 65776bcfe08f3..9d48cfebbe53d 100644 --- a/core/client/src/light/call_executor.rs +++ b/client/src/light/call_executor.rs @@ -17,8 +17,7 @@ //! Methods that light client could use to execute runtime calls. use std::{ - collections::HashSet, sync::Arc, panic::UnwindSafe, result, - cell::RefCell, rc::Rc, + sync::Arc, panic::UnwindSafe, result, cell::RefCell, }; use codec::{Encode, Decode}; @@ -31,15 +30,19 @@ use sr_primitives::{ }; use state_machine::{ self, Backend as StateBackend, OverlayedChanges, ExecutionStrategy, create_proof_check_backend, - execution_proof_check_on_trie_backend, ExecutionManager, ChangesTrieTransaction, + execution_proof_check_on_trie_backend, ExecutionManager, ChangesTrieTransaction, StorageProof, + merge_storage_proofs, }; use hash_db::Hasher; -use crate::runtime_api::{ProofRecorder, InitializeBlock}; -use crate::backend::RemoteBackend; -use crate::call_executor::CallExecutor; -use crate::error::{Error as ClientError, Result as ClientResult}; -use crate::light::fetcher::RemoteCallRequest; +use sr_api::{ProofRecorder, InitializeBlock}; + +use client_api::{ + backend::RemoteBackend, + error::{Error as ClientError, Result as ClientResult}, + light::RemoteCallRequest, + call_executor::CallExecutor +}; use executor::{RuntimeVersion, NativeVersion}; /// Call executor that is able to execute calls only on genesis state. @@ -109,7 +112,7 @@ impl CallExecutor for _manager: ExecutionManager, native_call: Option, side_effects_handler: Option, - recorder: &Option>>>, + recorder: &Option>, enable_keystore: bool, ) -> ClientResult> where ExecutionManager: Clone { // there's no actual way/need to specify native/wasm execution strategy on light node @@ -179,7 +182,7 @@ impl CallExecutor for _changes: &mut OverlayedChanges, _method: &str, _call_data: &[u8] - ) -> ClientResult<(Vec, Vec>)> { + ) -> ClientResult<(Vec, StorageProof)> { Err(ClientError::NotAvailableOnLightClient) } @@ -198,7 +201,7 @@ pub fn prove_execution( executor: &E, method: &str, call_data: &[u8], -) -> ClientResult<(Vec, Vec>)> +) -> ClientResult<(Vec, StorageProof)> where Block: BlockT, S: StateBackend, @@ -218,11 +221,7 @@ pub fn prove_execution( // execute method + record execution proof let (result, exec_proof) = executor.prove_at_trie_state(&trie_state, &mut changes, method, call_data)?; - let total_proof = init_proof.into_iter() - .chain(exec_proof.into_iter()) - .collect::>() - .into_iter() - .collect(); + let total_proof = merge_storage_proofs(vec![init_proof, exec_proof]); Ok((result, total_proof)) } @@ -234,7 +233,7 @@ pub fn prove_execution( pub fn check_execution_proof( executor: &E, request: &RemoteCallRequest

, - remote_proof: Vec>, + remote_proof: StorageProof, ) -> ClientResult> where Header: HeaderT, @@ -258,7 +257,7 @@ pub fn check_execution_proof( fn check_execution_proof_with_make_header Header>( executor: &E, request: &RemoteCallRequest
, - remote_proof: Vec>, + remote_proof: StorageProof, make_next_header: MakeNextHeader, ) -> ClientResult> where @@ -300,7 +299,7 @@ mod tests { use test_client::{self, runtime::{Header, Digest, Block}, ClientExt, TestClient}; use executor::{NativeExecutor, WasmExecutionMethod}; use primitives::Blake2Hasher; - use crate::backend::{Backend, NewBlockState}; + use client_api::backend::{Backend, NewBlockState}; use crate::in_mem::Backend as InMemBackend; struct DummyCallExecutor; @@ -339,7 +338,7 @@ mod tests { _execution_manager: ExecutionManager, _native_call: Option, _side_effects_handler: Option, - _proof_recorder: &Option>>>, + _proof_recorder: &Option>, _enable_keystore: bool, ) -> ClientResult> where ExecutionManager: Clone { unreachable!() @@ -382,7 +381,7 @@ mod tests { _overlay: &mut OverlayedChanges, _method: &str, _call_data: &[u8] - ) -> Result<(Vec, Vec>), ClientError> { + ) -> Result<(Vec, StorageProof), ClientError> { unreachable!() } @@ -455,7 +454,7 @@ mod tests { ), ); match execution_result { - Err(crate::error::Error::Execution(_)) => (), + Err(client_api::error::Error::Execution(_)) => (), _ => panic!("Unexpected execution result: {:?}", execution_result), } } diff --git a/core/client/src/light/fetcher.rs b/client/src/light/fetcher.rs similarity index 74% rename from core/client/src/light/fetcher.rs rename to client/src/light/fetcher.rs index dbf6d41c38f16..e8b88e93b9bab 100644 --- a/core/client/src/light/fetcher.rs +++ b/client/src/light/fetcher.rs @@ -19,11 +19,10 @@ use std::sync::Arc; use std::collections::{BTreeMap, HashMap}; use std::marker::PhantomData; -use std::future::Future; use hash_db::{HashDB, Hasher, EMPTY_PREFIX}; use codec::{Decode, Encode}; -use primitives::{ChangesTrieConfiguration, convert_hash, traits::CodeExecutor, H256}; +use primitives::{convert_hash, traits::CodeExecutor, H256}; use sr_primitives::traits::{ Block as BlockT, Header as HeaderT, Hash, HashFor, NumberFor, SimpleArithmetic, CheckedConversion, Zero, @@ -33,193 +32,22 @@ use state_machine::{ TrieBackend, read_proof_check, key_changes_proof_check, create_proof_check_backend_storage, read_child_proof_check, }; +pub use state_machine::StorageProof; use crate::cht; -use crate::error::{Error as ClientError, Result as ClientResult}; -use crate::light::blockchain::{Blockchain, Storage as BlockchainStorage}; +pub use client_api::{ + error::{ + Error as ClientError, Result as ClientResult + }, + light::{ + RemoteCallRequest, RemoteHeaderRequest, RemoteReadRequest, RemoteReadChildRequest, + RemoteChangesRequest, ChangesProof, RemoteBodyRequest, Fetcher, FetchChecker, + Storage as BlockchainStorage, + }, +}; +use crate::light::blockchain::{Blockchain}; use crate::light::call_executor::check_execution_proof; -/// Remote call request. -#[derive(Clone, Debug, PartialEq, Eq, Hash)] -pub struct RemoteCallRequest { - /// Call at state of given block. - pub block: Header::Hash, - /// Header of block at which call is performed. - pub header: Header, - /// Method to call. - pub method: String, - /// Call data. - pub call_data: Vec, - /// Number of times to retry request. None means that default RETRY_COUNT is used. - pub retry_count: Option, -} - -/// Remote canonical header request. -#[derive(Clone, Debug, Default, PartialEq, Eq, Hash)] -pub struct RemoteHeaderRequest { - /// The root of CHT this block is included in. - pub cht_root: Header::Hash, - /// Number of the header to query. - pub block: Header::Number, - /// Number of times to retry request. None means that default RETRY_COUNT is used. - pub retry_count: Option, -} - -/// Remote storage read request. -#[derive(Clone, Debug, PartialEq, Eq, Hash)] -pub struct RemoteReadRequest { - /// Read at state of given block. - pub block: Header::Hash, - /// Header of block at which read is performed. - pub header: Header, - /// Storage key to read. - pub keys: Vec>, - /// Number of times to retry request. None means that default RETRY_COUNT is used. - pub retry_count: Option, -} - -/// Remote storage read child request. -#[derive(Clone, Debug, PartialEq, Eq, Hash)] -pub struct RemoteReadChildRequest { - /// Read at state of given block. - pub block: Header::Hash, - /// Header of block at which read is performed. - pub header: Header, - /// Storage key for child. - pub storage_key: Vec, - /// Child storage key to read. - pub keys: Vec>, - /// Number of times to retry request. None means that default RETRY_COUNT is used. - pub retry_count: Option, -} - -/// Remote key changes read request. -#[derive(Clone, Debug, PartialEq, Eq)] -pub struct RemoteChangesRequest { - /// Changes trie configuration. - pub changes_trie_config: ChangesTrieConfiguration, - /// Query changes from range of blocks, starting (and including) with this hash... - pub first_block: (Header::Number, Header::Hash), - /// ...ending (and including) with this hash. Should come after first_block and - /// be the part of the same fork. - pub last_block: (Header::Number, Header::Hash), - /// Only use digests from blocks up to this hash. Should be last_block OR come - /// after this block and be the part of the same fork. - pub max_block: (Header::Number, Header::Hash), - /// Known changes trie roots for the range of blocks [tries_roots.0..max_block]. - /// Proofs for roots of ascendants of tries_roots.0 are provided by the remote node. - pub tries_roots: (Header::Number, Header::Hash, Vec), - /// Optional Child Storage key to read. - pub storage_key: Option>, - /// Storage key to read. - pub key: Vec, - /// Number of times to retry request. None means that default RETRY_COUNT is used. - pub retry_count: Option, -} - -/// Key changes read proof. -#[derive(Debug, PartialEq, Eq)] -pub struct ChangesProof { - /// Max block that has been used in changes query. - pub max_block: Header::Number, - /// All touched nodes of all changes tries. - pub proof: Vec>, - /// All changes tries roots that have been touched AND are missing from - /// the requester' node. It is a map of block number => changes trie root. - pub roots: BTreeMap, - /// The proofs for all changes tries roots that have been touched AND are - /// missing from the requester' node. It is a map of CHT number => proof. - pub roots_proof: Vec>, -} - -/// Remote block body request -#[derive(Clone, Default, Debug, PartialEq, Eq, Hash)] -pub struct RemoteBodyRequest { - /// Header of the requested block body - pub header: Header, - /// Number of times to retry request. None means that default RETRY_COUNT is used. - pub retry_count: Option, -} - -/// Light client data fetcher. Implementations of this trait must check if remote data -/// is correct (see FetchedDataChecker) and return already checked data. -pub trait Fetcher: Send + Sync { - /// Remote header future. - type RemoteHeaderResult: Future> + Send + 'static; - /// Remote storage read future. - type RemoteReadResult: Future, Option>>, ClientError>> + Send + 'static; - /// Remote call result future. - type RemoteCallResult: Future, ClientError>> + Send + 'static; - /// Remote changes result future. - type RemoteChangesResult: Future, u32)>, ClientError>> + Send + 'static; - /// Remote block body result future. - type RemoteBodyResult: Future, ClientError>> + Send + 'static; - - /// Fetch remote header. - fn remote_header(&self, request: RemoteHeaderRequest) -> Self::RemoteHeaderResult; - /// Fetch remote storage value. - fn remote_read( - &self, - request: RemoteReadRequest - ) -> Self::RemoteReadResult; - /// Fetch remote storage child value. - fn remote_read_child( - &self, - request: RemoteReadChildRequest - ) -> Self::RemoteReadResult; - /// Fetch remote call result. - fn remote_call(&self, request: RemoteCallRequest) -> Self::RemoteCallResult; - /// Fetch remote changes ((block number, extrinsic index)) where given key has been changed - /// at a given blocks range. - fn remote_changes(&self, request: RemoteChangesRequest) -> Self::RemoteChangesResult; - /// Fetch remote block body - fn remote_body(&self, request: RemoteBodyRequest) -> Self::RemoteBodyResult; -} - -/// Light client remote data checker. -/// -/// Implementations of this trait should not use any prunable blockchain data -/// except that is passed to its methods. -pub trait FetchChecker: Send + Sync { - /// Check remote header proof. - fn check_header_proof( - &self, - request: &RemoteHeaderRequest, - header: Option, - remote_proof: Vec> - ) -> ClientResult; - /// Check remote storage read proof. - fn check_read_proof( - &self, - request: &RemoteReadRequest, - remote_proof: Vec> - ) -> ClientResult, Option>>>; - /// Check remote storage read proof. - fn check_read_child_proof( - &self, - request: &RemoteReadChildRequest, - remote_proof: Vec> - ) -> ClientResult, Option>>>; - /// Check remote method execution proof. - fn check_execution_proof( - &self, - request: &RemoteCallRequest, - remote_proof: Vec> - ) -> ClientResult>; - /// Check remote changes query proof. - fn check_changes_proof( - &self, - request: &RemoteChangesRequest, - proof: ChangesProof - ) -> ClientResult, u32)>>; - /// Check remote body proof. - fn check_body_proof( - &self, - request: &RemoteBodyRequest, - body: Vec - ) -> ClientResult>; -} - /// Remote data checker. pub struct LightDataChecker> { blockchain: Arc>, @@ -318,7 +146,7 @@ impl> LightDataChecker { &self, cht_size: NumberFor, remote_roots: &BTreeMap, B::Hash>, - remote_roots_proof: Vec>, + remote_roots_proof: StorageProof, ) -> ClientResult<()> where H: Hasher, @@ -378,7 +206,7 @@ impl FetchChecker for LightDataChecker &self, request: &RemoteHeaderRequest, remote_header: Option, - remote_proof: Vec> + remote_proof: StorageProof, ) -> ClientResult { let remote_header = remote_header.ok_or_else(|| ClientError::from(ClientError::InvalidCHTProof))?; @@ -394,7 +222,7 @@ impl FetchChecker for LightDataChecker fn check_read_proof( &self, request: &RemoteReadRequest, - remote_proof: Vec>, + remote_proof: StorageProof, ) -> ClientResult, Option>>> { read_proof_check::( convert_hash(request.header.state_root()), @@ -406,7 +234,7 @@ impl FetchChecker for LightDataChecker fn check_read_child_proof( &self, request: &RemoteReadChildRequest, - remote_proof: Vec> + remote_proof: StorageProof, ) -> ClientResult, Option>>> { read_child_proof_check::( convert_hash(request.header.state_root()), @@ -419,7 +247,7 @@ impl FetchChecker for LightDataChecker fn check_execution_proof( &self, request: &RemoteCallRequest, - remote_proof: Vec> + remote_proof: StorageProof, ) -> ClientResult> { check_execution_proof::<_, _, H>(&self.executor, request, remote_proof) } @@ -498,12 +326,13 @@ impl<'a, H, Number, Hash> ChangesTrieRootsStorage for RootsStorage<'a #[cfg(test)] pub mod tests { - use futures03::future::Ready; - use parking_lot::Mutex; use codec::Decode; use crate::client::tests::prepare_client_with_key_changes; use executor::{NativeExecutor, WasmExecutionMethod}; - use crate::error::Error as ClientError; + use client_api::{ + backend::NewBlockState, + error::Error as ClientError, + }; use test_client::{ self, ClientExt, blockchain::HeaderBackend, AccountKeyring, runtime::{self, Hash, Block, Header, Extrinsic} @@ -511,8 +340,7 @@ pub mod tests { use consensus::BlockOrigin; use crate::in_mem::{Blockchain as InMemoryBlockchain}; - use crate::light::fetcher::{Fetcher, FetchChecker, LightDataChecker, - RemoteCallRequest, RemoteHeaderRequest}; + use crate::light::fetcher::{FetchChecker, LightDataChecker, RemoteHeaderRequest}; use crate::light::blockchain::tests::{DummyStorage, DummyBlockchain}; use primitives::{blake2_256, Blake2Hasher, H256}; use primitives::storage::{well_known_keys, StorageKey}; @@ -520,47 +348,6 @@ pub mod tests { use state_machine::Backend; use super::*; - pub type OkCallFetcher = Mutex>; - - fn not_implemented_in_tests() -> Ready> - where - E: std::convert::From<&'static str>, - { - futures03::future::ready(Err("Not implemented on test node".into())) - } - - impl Fetcher for OkCallFetcher { - type RemoteHeaderResult = Ready>; - type RemoteReadResult = Ready, Option>>, ClientError>>; - type RemoteCallResult = Ready, ClientError>>; - type RemoteChangesResult = Ready, u32)>, ClientError>>; - type RemoteBodyResult = Ready, ClientError>>; - - fn remote_header(&self, _request: RemoteHeaderRequest
) -> Self::RemoteHeaderResult { - not_implemented_in_tests() - } - - fn remote_read(&self, _request: RemoteReadRequest
) -> Self::RemoteReadResult { - not_implemented_in_tests() - } - - fn remote_read_child(&self, _request: RemoteReadChildRequest
) -> Self::RemoteReadResult { - not_implemented_in_tests() - } - - fn remote_call(&self, _request: RemoteCallRequest
) -> Self::RemoteCallResult { - futures03::future::ready(Ok((*self.lock()).clone())) - } - - fn remote_changes(&self, _request: RemoteChangesRequest
) -> Self::RemoteChangesResult { - not_implemented_in_tests() - } - - fn remote_body(&self, _request: RemoteBodyRequest
) -> Self::RemoteBodyResult { - not_implemented_in_tests() - } - } - type TestChecker = LightDataChecker< NativeExecutor, Blake2Hasher, @@ -572,7 +359,7 @@ pub mod tests { NativeExecutor::new(WasmExecutionMethod::Interpreted, None) } - fn prepare_for_read_proof_check() -> (TestChecker, Header, Vec>, u32) { + fn prepare_for_read_proof_check() -> (TestChecker, Header, StorageProof, u32) { // prepare remote client let remote_client = test_client::new(); let remote_block_id = BlockId::Number(0); @@ -597,7 +384,7 @@ pub mod tests { remote_block_header.clone(), None, None, - crate::backend::NewBlockState::Final, + NewBlockState::Final, ).unwrap(); let local_checker = LightDataChecker::new( Arc::new(DummyBlockchain::new(DummyStorage::new())), @@ -606,7 +393,7 @@ pub mod tests { (local_checker, remote_block_header, remote_read_proof, heap_pages) } - fn prepare_for_read_child_proof_check() -> (TestChecker, Header, Vec>, Vec) { + fn prepare_for_read_child_proof_check() -> (TestChecker, Header, StorageProof, Vec) { use test_client::DefaultTestClientBuilderExt; use test_client::TestClientBuilderExt; // prepare remote client @@ -639,7 +426,7 @@ pub mod tests { remote_block_header.clone(), None, None, - crate::backend::NewBlockState::Final, + NewBlockState::Final, ).unwrap(); let local_checker = LightDataChecker::new( Arc::new(DummyBlockchain::new(DummyStorage::new())), @@ -648,7 +435,7 @@ pub mod tests { (local_checker, remote_block_header, remote_read_proof, child_value) } - fn prepare_for_header_proof_check(insert_cht: bool) -> (TestChecker, Hash, Header, Vec>) { + fn prepare_for_header_proof_check(insert_cht: bool) -> (TestChecker, Hash, Header, StorageProof) { // prepare remote client let remote_client = test_client::new(); let mut local_headers_hashes = Vec::new(); @@ -899,13 +686,13 @@ pub mod tests { max_block: remote_proof.max_block, proof: remote_proof.proof.clone(), roots: vec![(begin - 1, Default::default())].into_iter().collect(), - roots_proof: vec![], + roots_proof: StorageProof::empty(), }).is_err()); assert!(local_checker.check_changes_proof(&request, ChangesProof { max_block: remote_proof.max_block, proof: remote_proof.proof.clone(), roots: vec![(end + 1, Default::default())].into_iter().collect(), - roots_proof: vec![], + roots_proof: StorageProof::empty(), }).is_err()); } @@ -945,7 +732,10 @@ pub mod tests { Arc::new(DummyBlockchain::new(local_storage)), local_executor(), ); - assert!(local_checker.check_changes_tries_proof(4, &remote_proof.roots, vec![]).is_err()); + let result = local_checker.check_changes_tries_proof( + 4, &remote_proof.roots, StorageProof::empty() + ); + assert!(result.is_err()); } #[test] diff --git a/core/client/src/light/mod.rs b/client/src/light/mod.rs similarity index 95% rename from core/client/src/light/mod.rs rename to client/src/light/mod.rs index d06a9ae9dd93c..7db1bff58900f 100644 --- a/core/client/src/light/mod.rs +++ b/client/src/light/mod.rs @@ -30,9 +30,12 @@ use sr_primitives::traits::Block as BlockT; use crate::call_executor::LocalCallExecutor; use crate::client::Client; -use crate::error::Result as ClientResult; +use client_api::{ + error::Result as ClientResult, + light::Storage as BlockchainStorage, +}; use crate::light::backend::Backend; -use crate::light::blockchain::{Blockchain, Storage as BlockchainStorage}; +use crate::light::blockchain::Blockchain; use crate::light::call_executor::GenesisCallExecutor; use crate::light::fetcher::LightDataChecker; diff --git a/core/state-db/Cargo.toml b/client/state-db/Cargo.toml similarity index 78% rename from core/state-db/Cargo.toml rename to client/state-db/Cargo.toml index d271a0e179d6d..d2752c14dd9c8 100644 --- a/core/state-db/Cargo.toml +++ b/client/state-db/Cargo.toml @@ -7,7 +7,7 @@ edition = "2018" [dependencies] parking_lot = "0.9.0" log = "0.4.8" -primitives = { package = "substrate-primitives", path = "../../core/primitives" } +primitives = { package = "substrate-primitives", path = "../../primitives/core" } codec = { package = "parity-scale-codec", version = "1.0.0", features = ["derive"] } [dev-dependencies] diff --git a/core/state-db/src/lib.rs b/client/state-db/src/lib.rs similarity index 100% rename from core/state-db/src/lib.rs rename to client/state-db/src/lib.rs diff --git a/core/state-db/src/noncanonical.rs b/client/state-db/src/noncanonical.rs similarity index 100% rename from core/state-db/src/noncanonical.rs rename to client/state-db/src/noncanonical.rs diff --git a/core/state-db/src/pruning.rs b/client/state-db/src/pruning.rs similarity index 100% rename from core/state-db/src/pruning.rs rename to client/state-db/src/pruning.rs diff --git a/core/state-db/src/test.rs b/client/state-db/src/test.rs similarity index 100% rename from core/state-db/src/test.rs rename to client/state-db/src/test.rs diff --git a/core/telemetry/Cargo.toml b/client/telemetry/Cargo.toml similarity index 93% rename from core/telemetry/Cargo.toml rename to client/telemetry/Cargo.toml index 90150ad7490df..82096a2cbf788 100644 --- a/core/telemetry/Cargo.toml +++ b/client/telemetry/Cargo.toml @@ -11,7 +11,7 @@ parking_lot = "0.9.0" futures01 = { package = "futures", version = "0.1" } futures-preview = { version = "0.3.0-alpha.19", features = ["compat"] } futures-timer = "0.4.0" -libp2p = { version = "0.12.0", default-features = false, features = ["libp2p-websocket"] } +libp2p = { version = "0.13.0", default-features = false, features = ["libp2p-websocket"] } log = "0.4.8" rand = "0.7.2" serde = { version = "1.0.101", features = ["derive"] } diff --git a/core/telemetry/src/lib.rs b/client/telemetry/src/lib.rs similarity index 100% rename from core/telemetry/src/lib.rs rename to client/telemetry/src/lib.rs diff --git a/core/telemetry/src/worker.rs b/client/telemetry/src/worker.rs similarity index 99% rename from core/telemetry/src/worker.rs rename to client/telemetry/src/worker.rs index 24a1de8ec4a31..37b21fa73eb24 100644 --- a/core/telemetry/src/worker.rs +++ b/client/telemetry/src/worker.rs @@ -109,7 +109,7 @@ impl TelemetryWorker { let transport = transport .map((|inner, _| Compat01As03Sink::new(inner)) as fn(_, _) -> _) - .with_timeout(CONNECT_TIMEOUT); + .timeout(CONNECT_TIMEOUT); TelemetryWorker { nodes: endpoints.into_iter().map(|(addr, verbosity)| { diff --git a/core/telemetry/src/worker/node.rs b/client/telemetry/src/worker/node.rs similarity index 100% rename from core/telemetry/src/worker/node.rs rename to client/telemetry/src/worker/node.rs diff --git a/client/transaction-pool/Cargo.toml b/client/transaction-pool/Cargo.toml new file mode 100644 index 0000000000000..1060dcd2a3ed3 --- /dev/null +++ b/client/transaction-pool/Cargo.toml @@ -0,0 +1,21 @@ +[package] +name = "substrate-transaction-pool" +version = "2.0.0" +authors = ["Parity Technologies "] +edition = "2018" + +[dependencies] +codec = { package = "parity-scale-codec", version = "1.0.0" } +derive_more = "0.15.0" +futures = { version = "0.3.0", features = ["thread-pool"] } +log = "0.4.8" +parking_lot = "0.9.0" +primitives = { package = "substrate-primitives", path = "../../primitives/core" } +sr-api = { path = "../../primitives/sr-api" } +sr-primitives = { path = "../../primitives/sr-primitives" } +tx-runtime-api = { package = "substrate-transaction-pool-runtime-api", path = "../../primitives/transaction-pool/runtime-api" } +txpool = { package = "substrate-transaction-graph", path = "./graph" } + +[dev-dependencies] +keyring = { package = "substrate-keyring", path = "../../primitives/keyring" } +test-client = { package = "substrate-test-runtime-client", path = "../../test/utils/runtime/client" } diff --git a/core/transaction-pool/graph/Cargo.toml b/client/transaction-pool/graph/Cargo.toml similarity index 84% rename from core/transaction-pool/graph/Cargo.toml rename to client/transaction-pool/graph/Cargo.toml index 4b628079cd3ce..93068c514add0 100644 --- a/core/transaction-pool/graph/Cargo.toml +++ b/client/transaction-pool/graph/Cargo.toml @@ -10,14 +10,14 @@ futures-preview = "0.3.0-alpha.19" log = "0.4.8" parking_lot = "0.9.0" serde = { version = "1.0.101", features = ["derive"] } -primitives = { package = "substrate-primitives", path = "../../primitives" } -sr-primitives = { path = "../../sr-primitives" } +primitives = { package = "substrate-primitives", path = "../../../primitives/core" } +sr-primitives = { path = "../../../primitives/sr-primitives" } [dev-dependencies] assert_matches = "1.3.0" env_logger = "0.7.0" codec = { package = "parity-scale-codec", version = "1.0.0" } -test_runtime = { package = "substrate-test-runtime", path = "../../test-runtime" } +test_runtime = { package = "substrate-test-runtime", path = "../../../test/utils/runtime" } criterion = "0.3" [[bench]] diff --git a/core/transaction-pool/graph/benches/basics.rs b/client/transaction-pool/graph/benches/basics.rs similarity index 100% rename from core/transaction-pool/graph/benches/basics.rs rename to client/transaction-pool/graph/benches/basics.rs diff --git a/core/transaction-pool/graph/src/base_pool.rs b/client/transaction-pool/graph/src/base_pool.rs similarity index 100% rename from core/transaction-pool/graph/src/base_pool.rs rename to client/transaction-pool/graph/src/base_pool.rs diff --git a/core/transaction-pool/graph/src/error.rs b/client/transaction-pool/graph/src/error.rs similarity index 100% rename from core/transaction-pool/graph/src/error.rs rename to client/transaction-pool/graph/src/error.rs diff --git a/core/transaction-pool/graph/src/future.rs b/client/transaction-pool/graph/src/future.rs similarity index 100% rename from core/transaction-pool/graph/src/future.rs rename to client/transaction-pool/graph/src/future.rs diff --git a/core/transaction-pool/graph/src/lib.rs b/client/transaction-pool/graph/src/lib.rs similarity index 100% rename from core/transaction-pool/graph/src/lib.rs rename to client/transaction-pool/graph/src/lib.rs diff --git a/core/transaction-pool/graph/src/listener.rs b/client/transaction-pool/graph/src/listener.rs similarity index 86% rename from core/transaction-pool/graph/src/listener.rs rename to client/transaction-pool/graph/src/listener.rs index 335ff8a0537e9..a96c31544fc75 100644 --- a/core/transaction-pool/graph/src/listener.rs +++ b/client/transaction-pool/graph/src/listener.rs @@ -17,12 +17,13 @@ use std::{ collections::HashMap, + fmt, hash, }; use serde::Serialize; use crate::watcher; use sr_primitives::traits; -use log::warn; +use log::{debug, trace, warn}; /// Extrinsic pool default listener. pub struct Listener { @@ -37,7 +38,7 @@ impl Default for Listener { } } -impl Listener { +impl Listener { fn fire(&mut self, hash: &H, fun: F) where F: FnOnce(&mut watcher::Sender) { let clean = if let Some(h) = self.watchers.get_mut(hash) { fun(h); @@ -61,11 +62,13 @@ impl Listener { /// Notify the listeners about extrinsic broadcast. pub fn broadcasted(&mut self, hash: &H, peers: Vec) { + trace!(target: "txpool", "[{:?}] Broadcasted", hash); self.fire(hash, |watcher| watcher.broadcast(peers)); } /// New transaction was added to the ready pool or promoted from the future pool. pub fn ready(&mut self, tx: &H, old: Option<&H>) { + trace!(target: "txpool", "[{:?}] Ready (replaced: {:?})", tx, old); self.fire(tx, |watcher| watcher.ready()); if let Some(old) = old { self.fire(old, |watcher| watcher.usurped(tx.clone())); @@ -74,11 +77,13 @@ impl Listener { /// New transaction was added to the future pool. pub fn future(&mut self, tx: &H) { + trace!(target: "txpool", "[{:?}] Future", tx); self.fire(tx, |watcher| watcher.future()); } /// Transaction was dropped from the pool because of the limit. pub fn dropped(&mut self, tx: &H, by: Option<&H>) { + trace!(target: "txpool", "[{:?}] Dropped (replaced by {:?})", tx, by); self.fire(tx, |watcher| match by { Some(t) => watcher.usurped(t.clone()), None => watcher.dropped(), @@ -93,6 +98,7 @@ impl Listener { /// Transaction was pruned from the pool. pub fn pruned(&mut self, header_hash: H2, tx: &H) { + debug!(target: "txpool", "[{:?}] Pruned at {:?}", tx, header_hash); self.fire(tx, |watcher| watcher.finalized(header_hash)) } } diff --git a/core/transaction-pool/graph/src/pool.rs b/client/transaction-pool/graph/src/pool.rs similarity index 98% rename from core/transaction-pool/graph/src/pool.rs rename to client/transaction-pool/graph/src/pool.rs index dcb54f710f175..c6e33223282ff 100644 --- a/core/transaction-pool/graph/src/pool.rs +++ b/client/transaction-pool/graph/src/pool.rs @@ -177,6 +177,12 @@ impl Pool { parent: &BlockId, extrinsics: &[ExtrinsicFor], ) -> impl Future> { + log::debug!( + target: "txpool", + "Starting pruning of block {:?} (extrinsics: {})", + at, + extrinsics.len() + ); // Get details of all extrinsics that are already in the pool let (in_pool_hashes, in_pool_tags) = self.validated_pool.extrinsics_tags(extrinsics); @@ -342,14 +348,14 @@ impl Pool { ) -> impl Future> { let (hash, bytes) = self.validated_pool.api().hash_and_length(&xt); if !force && self.validated_pool.is_banned(&hash) { - return Either::Left(ready(ValidatedTransaction::Invalid(error::Error::TemporarilyBanned.into()))) + return Either::Left(ready(ValidatedTransaction::Invalid(hash, error::Error::TemporarilyBanned.into()))) } Either::Right(self.validated_pool.api().validate_transaction(block_id, xt.clone()) .then(move |validation_result| ready(match validation_result { Ok(validity) => match validity { Ok(validity) => if validity.provides.is_empty() { - ValidatedTransaction::Invalid(error::Error::NoTagsProvided.into()) + ValidatedTransaction::Invalid(hash, error::Error::NoTagsProvided.into()) } else { ValidatedTransaction::Valid(base::Transaction { data: xt, @@ -365,11 +371,11 @@ impl Pool { }) }, Err(TransactionValidityError::Invalid(e)) => - ValidatedTransaction::Invalid(error::Error::InvalidTransaction(e).into()), + ValidatedTransaction::Invalid(hash, error::Error::InvalidTransaction(e).into()), Err(TransactionValidityError::Unknown(e)) => ValidatedTransaction::Unknown(hash, error::Error::UnknownTransaction(e).into()), }, - Err(e) => ValidatedTransaction::Invalid(e), + Err(e) => ValidatedTransaction::Invalid(hash, e), }))) } } diff --git a/core/transaction-pool/graph/src/ready.rs b/client/transaction-pool/graph/src/ready.rs similarity index 100% rename from core/transaction-pool/graph/src/ready.rs rename to client/transaction-pool/graph/src/ready.rs diff --git a/core/transaction-pool/graph/src/rotator.rs b/client/transaction-pool/graph/src/rotator.rs similarity index 100% rename from core/transaction-pool/graph/src/rotator.rs rename to client/transaction-pool/graph/src/rotator.rs diff --git a/core/transaction-pool/graph/src/validated_pool.rs b/client/transaction-pool/graph/src/validated_pool.rs similarity index 96% rename from core/transaction-pool/graph/src/validated_pool.rs rename to client/transaction-pool/graph/src/validated_pool.rs index 9bf1012628645..7317d41f42e97 100644 --- a/core/transaction-pool/graph/src/validated_pool.rs +++ b/client/transaction-pool/graph/src/validated_pool.rs @@ -16,6 +16,7 @@ use std::{ collections::{HashSet, HashMap}, + fmt, hash, time, }; @@ -45,7 +46,7 @@ pub enum ValidatedTransaction { /// Transaction that has been validated successfully. Valid(base::Transaction), /// Transaction that is invalid. - Invalid(Error), + Invalid(Hash, Error), /// Transaction which validity can't be determined. /// /// We're notifying watchers about failure, if 'unknown' transaction is submitted. @@ -125,7 +126,10 @@ impl ValidatedPool { fire_events(&mut *listener, &imported); Ok(imported.hash().clone()) } - ValidatedTransaction::Invalid(err) => Err(err.into()), + ValidatedTransaction::Invalid(hash, err) => { + self.rotator.ban(&std::time::Instant::now(), std::iter::once(hash)); + Err(err.into()) + }, ValidatedTransaction::Unknown(hash, err) => { self.listener.write().invalid(&hash); Err(err.into()) @@ -177,7 +181,10 @@ impl ValidatedPool { .expect("One extrinsic passed; one result returned; qed") .map(|_| watcher) }, - ValidatedTransaction::Invalid(err) => Err(err.into()), + ValidatedTransaction::Invalid(hash, err) => { + self.rotator.ban(&std::time::Instant::now(), std::iter::once(hash)); + Err(err.into()) + }, ValidatedTransaction::Unknown(_, err) => Err(err.into()), } } @@ -349,7 +356,7 @@ fn fire_events( imported: &base::Imported, ) where H: hash::Hash + Eq + traits::Member + Serialize, - H2: Clone, + H2: Clone + fmt::Debug, { match *imported { base::Imported::Ready { ref promoted, ref failed, ref removed, ref hash } => { diff --git a/core/transaction-pool/graph/src/watcher.rs b/client/transaction-pool/graph/src/watcher.rs similarity index 100% rename from core/transaction-pool/graph/src/watcher.rs rename to client/transaction-pool/graph/src/watcher.rs diff --git a/client/transaction-pool/src/api.rs b/client/transaction-pool/src/api.rs new file mode 100644 index 0000000000000..28681fb11ba8a --- /dev/null +++ b/client/transaction-pool/src/api.rs @@ -0,0 +1,112 @@ +// Copyright 2018-2019 Parity Technologies (UK) Ltd. +// This file is part of Substrate. + +// Substrate is free software: you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. + +// Substrate is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. + +// You should have received a copy of the GNU General Public License +// along with Substrate. If not, see . + +//! Chain api required for the transaction pool. + +use std::{marker::PhantomData, pin::Pin, sync::Arc}; + +use codec::Encode; + +use futures::{channel::oneshot, executor::{ThreadPool, ThreadPoolBuilder}, future::Future}; + +use primitives::{H256, Blake2Hasher, Hasher}; + +use sr_primitives::{generic::BlockId, traits, transaction_validity::TransactionValidity}; + +use tx_runtime_api::TaggedTransactionQueue; + +use crate::error::{self, Error}; + +/// The transaction pool logic +pub struct FullChainApi { + client: Arc, + pool: ThreadPool, + _marker: PhantomData, +} + +impl FullChainApi where + Block: traits::Block, + T: traits::ProvideRuntimeApi + traits::BlockIdTo { + /// Create new transaction pool logic. + pub fn new(client: Arc) -> Self { + FullChainApi { + client, + pool: ThreadPoolBuilder::new() + .pool_size(2) + .name_prefix("txpool-verifier") + .create() + .expect("Failed to spawn verifier threads, that are critical for node operation."), + _marker: Default::default() + } + } +} + +impl txpool::ChainApi for FullChainApi where + Block: traits::Block, + T: traits::ProvideRuntimeApi + traits::BlockIdTo + 'static + Send + Sync, + T::Api: TaggedTransactionQueue, + sr_api::ApiErrorFor: Send, +{ + type Block = Block; + type Hash = H256; + type Error = error::Error; + type ValidationFuture = Pin> + Send>>; + + fn validate_transaction( + &self, + at: &BlockId, + uxt: txpool::ExtrinsicFor, + ) -> Self::ValidationFuture { + let (tx, rx) = oneshot::channel(); + let client = self.client.clone(); + let at = at.clone(); + + self.pool.spawn_ok(async move { + let res = client.runtime_api().validate_transaction(&at, uxt) + .map_err(|e| Error::RuntimeApi(format!("{:?}", e))); + if let Err(e) = tx.send(res) { + log::warn!("Unable to send a validate transaction result: {:?}", e); + } + }); + + Box::pin(async move { + match rx.await { + Ok(r) => r, + Err(_) => Err(Error::RuntimeApi("Validation was canceled".into())), + } + }) + } + + fn block_id_to_number( + &self, + at: &BlockId, + ) -> error::Result>> { + self.client.to_number(at).map_err(|e| Error::BlockIdConversion(format!("{:?}", e))) + } + + fn block_id_to_hash( + &self, + at: &BlockId, + ) -> error::Result>> { + self.client.to_hash(at).map_err(|e| Error::BlockIdConversion(format!("{:?}", e))) + } + + fn hash_and_length(&self, ex: &txpool::ExtrinsicFor) -> (Self::Hash, usize) { + ex.using_encoded(|x| { + (Blake2Hasher::hash(x), x.len()) + }) + } +} diff --git a/core/transaction-pool/src/error.rs b/client/transaction-pool/src/error.rs similarity index 86% rename from core/transaction-pool/src/error.rs rename to client/transaction-pool/src/error.rs index f3641aa8ecee3..ae0a058e5cc37 100644 --- a/core/transaction-pool/src/error.rs +++ b/client/transaction-pool/src/error.rs @@ -16,26 +16,26 @@ //! Transaction pool error. -use client; -use txpool; - /// Transaction pool result. pub type Result = std::result::Result; /// Transaction pool error type. #[derive(Debug, derive_more::Display, derive_more::From)] pub enum Error { - /// Client error. - Client(client::error::Error), /// Pool error. Pool(txpool::error::Error), + /// Error while converting a `BlockId`. + BlockIdConversion(String), + /// Error while calling the runtime api. + RuntimeApi(String), } impl std::error::Error for Error { fn source(&self) -> Option<&(dyn std::error::Error + 'static)> { match self { - Error::Client(ref err) => Some(err), Error::Pool(ref err) => Some(err), + Error::BlockIdConversion(_) => None, + Error::RuntimeApi(_) => None, } } } diff --git a/core/transaction-pool/src/lib.rs b/client/transaction-pool/src/lib.rs similarity index 99% rename from core/transaction-pool/src/lib.rs rename to client/transaction-pool/src/lib.rs index 6938166299d85..d7703de65285d 100644 --- a/core/transaction-pool/src/lib.rs +++ b/client/transaction-pool/src/lib.rs @@ -20,10 +20,9 @@ #![warn(unused_extern_crates)] mod api; +pub mod error; #[cfg(test)] mod tests; -pub mod error; - pub use api::FullChainApi; pub use txpool; diff --git a/core/transaction-pool/src/tests.rs b/client/transaction-pool/src/tests.rs similarity index 100% rename from core/transaction-pool/src/tests.rs rename to client/transaction-pool/src/tests.rs diff --git a/core/utils/wasm-builder-runner/Cargo.toml b/client/utils/wasm-builder-runner/Cargo.toml similarity index 100% rename from core/utils/wasm-builder-runner/Cargo.toml rename to client/utils/wasm-builder-runner/Cargo.toml diff --git a/core/utils/wasm-builder-runner/README.md b/client/utils/wasm-builder-runner/README.md similarity index 100% rename from core/utils/wasm-builder-runner/README.md rename to client/utils/wasm-builder-runner/README.md diff --git a/core/utils/wasm-builder-runner/src/lib.rs b/client/utils/wasm-builder-runner/src/lib.rs similarity index 98% rename from core/utils/wasm-builder-runner/src/lib.rs rename to client/utils/wasm-builder-runner/src/lib.rs index 1739c5eff211e..706b623fc3445 100644 --- a/core/utils/wasm-builder-runner/src/lib.rs +++ b/client/utils/wasm-builder-runner/src/lib.rs @@ -149,9 +149,9 @@ pub fn build_current_project(file_name: &str, wasm_builder_source: WasmBuilderSo build_current_project_with_rustflags(file_name, wasm_builder_source, ""); } -/// Returns the root path of the wasm-builder-runner workspace. +/// Returns the root path of the wasm-builder workspace. /// -/// The wasm-builder-runner workspace contains all wasm-builder-runner's projects. +/// The wasm-builder workspace contains all wasm-builder's projects. fn get_workspace_root() -> PathBuf { let out_dir_env = env::var("OUT_DIR").expect("`OUT_DIR` is set by cargo!"); let mut out_dir = PathBuf::from(&out_dir_env); diff --git a/core/authority-discovery/Cargo.toml b/core/authority-discovery/Cargo.toml deleted file mode 100644 index 7283e07f89ca7..0000000000000 --- a/core/authority-discovery/Cargo.toml +++ /dev/null @@ -1,31 +0,0 @@ -[package] -name = "substrate-authority-discovery" -version = "2.0.0" -authors = ["Parity Technologies "] -edition = "2018" -build = "build.rs" - -[build-dependencies] -prost-build = "0.5.0" - -[dependencies] -authority-discovery-primitives = { package = "substrate-authority-discovery-primitives", path = "./primitives" } -bytes = "0.4.12" -client = { package = "substrate-client", path = "../../core/client" } -codec = { package = "parity-scale-codec", default-features = false, version = "1.0.3" } -derive_more = "0.15.0" -futures = "0.1.29" -libp2p = { version = "0.12.0", default-features = false, features = ["secp256k1", "libp2p-websocket"] } -log = "0.4.8" -network = { package = "substrate-network", path = "../../core/network" } -primitives = { package = "substrate-primitives", path = "../primitives" } -prost = "0.5.0" -serde_json = "1.0.41" -sr-primitives = { path = "../../core/sr-primitives" } -tokio-timer = "0.2.11" - -[dev-dependencies] -parking_lot = "0.9.0" -peerset = { package = "substrate-peerset", path = "../../core/peerset" } -test-client = { package = "substrate-test-runtime-client", path = "../../core/test-runtime/client" } -tokio = "0.1.22" diff --git a/core/authority-discovery/primitives/Cargo.toml b/core/authority-discovery/primitives/Cargo.toml deleted file mode 100644 index 8e9e465a299f6..0000000000000 --- a/core/authority-discovery/primitives/Cargo.toml +++ /dev/null @@ -1,21 +0,0 @@ -[package] -name = "substrate-authority-discovery-primitives" -version = "2.0.0" -authors = ["Parity Technologies "] -description = "Authority discovery primitives" -edition = "2018" - -[dependencies] -codec = { package = "parity-scale-codec", default-features = false, version = "1.0.3" } -client = { package = "substrate-client", path = "../../client", default-features = false } -sr-primitives = { path = "../../sr-primitives", default-features = false } -rstd = { package = "sr-std", path = "../../sr-std", default-features = false } - -[features] -default = ["std"] -std = [ - "rstd/std", - "client/std", - "codec/std", - "sr-primitives/std" -] diff --git a/core/basic-authorship/Cargo.toml b/core/basic-authorship/Cargo.toml deleted file mode 100644 index 0a98c151d6eaf..0000000000000 --- a/core/basic-authorship/Cargo.toml +++ /dev/null @@ -1,20 +0,0 @@ -[package] -name = "substrate-basic-authorship" -version = "2.0.0" -authors = ["Parity Technologies "] -edition = "2018" - -[dependencies] -log = "0.4.8" -futures-preview = "0.3.0-alpha.19" -codec = { package = "parity-scale-codec", version = "1.0.0" } -sr-primitives = { path = "../../core/sr-primitives" } -primitives = { package = "substrate-primitives", path = "../../core/primitives" } -client = { package = "substrate-client", path = "../../core/client" } -consensus_common = { package = "substrate-consensus-common", path = "../../core/consensus/common" } -inherents = { package = "substrate-inherents", path = "../inherents" } -substrate-telemetry = { path = "../telemetry" } -transaction_pool = { package = "substrate-transaction-pool", path = "../../core/transaction-pool" } - -[dev-dependencies] -test-client = { package = "substrate-test-runtime-client", path = "../../core/test-runtime/client" } diff --git a/core/chain-spec/Cargo.toml b/core/chain-spec/Cargo.toml deleted file mode 100644 index b4d9d004e4c3f..0000000000000 --- a/core/chain-spec/Cargo.toml +++ /dev/null @@ -1,17 +0,0 @@ -[package] -name = "substrate-chain-spec" -version = "2.0.0" -authors = ["Parity Technologies "] -edition = "2018" - -[dependencies] -substrate-chain-spec-derive = { path = "./derive" } -impl-trait-for-tuples = "0.1.2" -network = { package = "substrate-network", path = "../../core/network" } -primitives = { package = "substrate-primitives", path = "../primitives" } -serde = { version = "1.0.101", features = ["derive"] } -serde_json = "1.0.41" -sr-primitives = { path = "../../core/sr-primitives" } -tel = { package = "substrate-telemetry", path = "../../core/telemetry" } - -[dev-dependencies] diff --git a/core/client/db/src/storage_cache.rs b/core/client/db/src/storage_cache.rs deleted file mode 100644 index af8c9e379c4c1..0000000000000 --- a/core/client/db/src/storage_cache.rs +++ /dev/null @@ -1,731 +0,0 @@ -// Copyright 2019 Parity Technologies (UK) Ltd. -// This file is part of Substrate. - -// Substrate is free software: you can redistribute it and/or modify -// it under the terms of the GNU General Public License as published by -// the Free Software Foundation, either version 3 of the License, or -// (at your option) any later version. - -// Substrate is distributed in the hope that it will be useful, -// but WITHOUT ANY WARRANTY; without even the implied warranty of -// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -// GNU General Public License for more details. - -// You should have received a copy of the GNU General Public License -// along with Substrate. If not, see . - -//! Global cache state. - -use std::collections::{VecDeque, HashSet, HashMap}; -use std::sync::Arc; -use parking_lot::{Mutex, RwLock, RwLockUpgradableReadGuard}; -use linked_hash_map::{LinkedHashMap, Entry}; -use hash_db::Hasher; -use sr_primitives::traits::{Block as BlockT, Header}; -use state_machine::{backend::Backend as StateBackend, TrieBackend}; -use log::trace; -use super::{StorageCollection, ChildStorageCollection}; -use std::hash::Hash as StdHash; -const STATE_CACHE_BLOCKS: usize = 12; - -type StorageKey = Vec; -type ChildStorageKey = (Vec, Vec); -type StorageValue = Vec; - -/// Shared canonical state cache. -pub struct Cache { - /// Storage cache. `None` indicates that key is known to be missing. - lru_storage: LRUMap>, - /// Storage hashes cache. `None` indicates that key is known to be missing. - lru_hashes: LRUMap>, - /// Storage cache for child trie. `None` indicates that key is known to be missing. - lru_child_storage: LRUMap>, - /// Information on the modifications in recently committed blocks; specifically which keys - /// changed in which block. Ordered by block number. - modifications: VecDeque>, -} - -struct LRUMap(LinkedHashMap, usize, usize); - -/// Internal trait similar to `heapsize` but using -/// a simply estimation. -/// -/// This should not be made public, it is implementation -/// detail trait. If it need to become public please -/// consider using `malloc_size_of`. -trait EstimateSize { - /// Return a size estimation of additional size needed - /// to cache this struct (in bytes). - fn estimate_size(&self) -> usize; -} - -impl EstimateSize for Vec { - fn estimate_size(&self) -> usize { - self.capacity() - } -} - -impl EstimateSize for Option> { - fn estimate_size(&self) -> usize { - self.as_ref().map(|v|v.capacity()).unwrap_or(0) - } -} - -struct OptionHOut>(Option); - -impl> EstimateSize for OptionHOut { - fn estimate_size(&self) -> usize { - // capacity would be better - self.0.as_ref().map(|v|v.as_ref().len()).unwrap_or(0) - } -} - -impl EstimateSize for (T, T) { - fn estimate_size(&self) -> usize { - self.0.estimate_size() + self.1.estimate_size() - } -} - -impl LRUMap { - fn remove(&mut self, k: &K) { - let map = &mut self.0; - let storage_used_size = &mut self.1; - if let Some(v) = map.remove(k) { - *storage_used_size -= k.estimate_size(); - *storage_used_size -= v.estimate_size(); - } - } - - fn add(&mut self, k: K, v: V) { - let lmap = &mut self.0; - let storage_used_size = &mut self.1; - let limit = self.2; - let klen = k.estimate_size(); - *storage_used_size += v.estimate_size(); - // TODO assert k v size fit into limit?? to avoid insert remove? - match lmap.entry(k) { - Entry::Occupied(mut entry) => { - // note that in this case we are not running pure lru as - // it would require to remove first - *storage_used_size -= entry.get().estimate_size(); - entry.insert(v); - }, - Entry::Vacant(entry) => { - *storage_used_size += klen; - entry.insert(v); - }, - }; - - while *storage_used_size > limit { - if let Some((k,v)) = lmap.pop_front() { - *storage_used_size -= k.estimate_size(); - *storage_used_size -= v.estimate_size(); - } else { - // can happen fairly often as we get value from multiple lru - // and only remove from a single lru - break; - } - } - } - - fn get(&mut self, k: &Q) -> Option<&mut V> - where K: std::borrow::Borrow, - Q: StdHash + Eq { - self.0.get_refresh(k) - } - - fn used_size(&self) -> usize { - self.1 - } - fn clear(&mut self) { - self.0.clear(); - self.1 = 0; - } - -} - -impl Cache { - /// Returns the used memory size of the storage cache in bytes. - pub fn used_storage_cache_size(&self) -> usize { - self.lru_storage.used_size() - + self.lru_child_storage.used_size() - // ignore small hashes storage and self.lru_hashes.used_size() - } - - /// Synchronize the shared cache with the best block state. - /// This function updates the shared cache by removing entries - /// that are invalidated by chain reorganization. It should be - /// be called when chain reorg happens without importing a new block. - pub fn sync(&mut self, enacted: &[B::Hash], retracted: &[B::Hash]) { - trace!("Syncing shared cache, enacted = {:?}, retracted = {:?}", enacted, retracted); - - // Purge changes from re-enacted and retracted blocks. - // Filter out commiting block if any. - let mut clear = false; - for block in enacted { - clear = clear || { - if let Some(ref mut m) = self.modifications.iter_mut().find(|m| &m.hash == block) { - trace!("Reverting enacted block {:?}", block); - m.is_canon = true; - for a in &m.storage { - trace!("Reverting enacted key {:?}", a); - self.lru_storage.remove(a); - } - for a in &m.child_storage { - trace!("Reverting enacted child key {:?}", a); - self.lru_child_storage.remove(a); - } - false - } else { - true - } - }; - } - - for block in retracted { - clear = clear || { - if let Some(ref mut m) = self.modifications.iter_mut().find(|m| &m.hash == block) { - trace!("Retracting block {:?}", block); - m.is_canon = false; - for a in &m.storage { - trace!("Retracted key {:?}", a); - self.lru_storage.remove(a); - } - for a in &m.child_storage { - trace!("Retracted child key {:?}", a); - self.lru_child_storage.remove(a); - } - false - } else { - true - } - }; - } - if clear { - // We don't know anything about the block; clear everything - trace!("Wiping cache"); - self.lru_storage.clear(); - self.lru_child_storage.clear(); - self.lru_hashes.clear(); - self.modifications.clear(); - } - } -} - -pub type SharedCache = Arc>>; - -/// Fix lru storage size for hash (small 64ko). -const FIX_LRU_HASH_SIZE: usize = 65_536; - -/// Create a new shared cache instance with given max memory usage. -pub fn new_shared_cache( - shared_cache_size: usize, - child_ratio: (usize, usize), -) -> SharedCache { - let top = child_ratio.1.saturating_sub(child_ratio.0); - Arc::new(Mutex::new(Cache { - lru_storage: LRUMap(LinkedHashMap::new(), 0, - shared_cache_size * top / child_ratio.1), - lru_hashes: LRUMap(LinkedHashMap::new(), 0, FIX_LRU_HASH_SIZE), - lru_child_storage: LRUMap(LinkedHashMap::new(), 0, - shared_cache_size * child_ratio.0 / child_ratio.1), - modifications: VecDeque::new(), - })) -} - -#[derive(Debug)] -/// Accumulates a list of storage changed in a block. -struct BlockChanges { - /// Block number. - number: B::Number, - /// Block hash. - hash: B::Hash, - /// Parent block hash. - parent: B::Hash, - /// A set of modified storage keys. - storage: HashSet, - /// A set of modified child storage keys. - child_storage: HashSet, - /// Block is part of the canonical chain. - is_canon: bool, -} - -/// Cached values specific to a state. -struct LocalCache { - /// Storage cache. `None` indicates that key is known to be missing. - storage: HashMap>, - /// Storage hashes cache. `None` indicates that key is known to be missing. - hashes: HashMap>, - /// Child storage cache. `None` indicates that key is known to be missing. - child_storage: HashMap>, -} - -/// Cache changes. -pub struct CacheChanges { - /// Shared canonical state cache. - shared_cache: SharedCache, - /// Local cache of values for this state. - local_cache: RwLock>, - /// Hash of the block on top of which this instance was created or - /// `None` if cache is disabled - pub parent_hash: Option, -} - -/// State abstraction. -/// Manages shared global state cache which reflects the canonical -/// state as it is on the disk. -/// A instance of `CachingState` may be created as canonical or not. -/// For canonical instances local cache is accumulated and applied -/// in `sync_cache` along with the change overlay. -/// For non-canonical clones local cache and changes are dropped. -pub struct CachingState, B: BlockT> { - /// Backing state. - state: S, - /// Cache data. - pub cache: CacheChanges -} - -impl, B: BlockT> std::fmt::Debug for CachingState { - fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { - write!(f, "Block {:?}", self.cache.parent_hash) - } -} - -impl CacheChanges { - /// Propagate local cache into the shared cache and synchronize - /// the shared cache with the best block state. - /// This function updates the shared cache by removing entries - /// that are invalidated by chain reorganization. `sync_cache` - /// should be called after the block has been committed and the - /// blockchain route has been calculated. - pub fn sync_cache bool> ( - &mut self, - enacted: &[B::Hash], - retracted: &[B::Hash], - changes: StorageCollection, - child_changes: ChildStorageCollection, - commit_hash: Option, - commit_number: Option<::Number>, - is_best: F, - ) { - let mut cache = self.shared_cache.lock(); - let is_best = is_best(); - trace!("Syncing cache, id = (#{:?}, {:?}), parent={:?}, best={}", commit_number, commit_hash, self.parent_hash, is_best); - let cache = &mut *cache; - let enacted: Vec<_> = enacted - .iter() - .filter(|h| commit_hash.as_ref().map_or(true, |p| *h != p)) - .cloned() - .collect(); - cache.sync(&enacted, retracted); - // Propagate cache only if committing on top of the latest canonical state - // blocks are ordered by number and only one block with a given number is marked as canonical - // (contributed to canonical state cache) - if let Some(_) = self.parent_hash { - let mut local_cache = self.local_cache.write(); - if is_best { - trace!( - "Committing {} local, {} hashes, {} modified root entries, {} modified child entries", - local_cache.storage.len(), - local_cache.hashes.len(), - changes.len(), - child_changes.iter().map(|v|v.1.len()).sum::(), - ); - for (k, v) in local_cache.storage.drain() { - cache.lru_storage.add(k, v); - } - for (k, v) in local_cache.child_storage.drain() { - cache.lru_child_storage.add(k, v); - } - for (k, v) in local_cache.hashes.drain() { - cache.lru_hashes.add(k, OptionHOut(v)); - } - } - } - - if let ( - Some(ref number), Some(ref hash), Some(ref parent)) - = (commit_number, commit_hash, self.parent_hash) - { - if cache.modifications.len() == STATE_CACHE_BLOCKS { - cache.modifications.pop_back(); - } - let mut modifications = HashSet::new(); - let mut child_modifications = HashSet::new(); - child_changes.into_iter().for_each(|(sk, changes)| - for (k, v) in changes.into_iter() { - let k = (sk.clone(), k); - if is_best { - cache.lru_child_storage.add(k.clone(), v); - } - child_modifications.insert(k); - } - ); - for (k, v) in changes.into_iter() { - if is_best { - cache.lru_hashes.remove(&k); - cache.lru_storage.add(k.clone(), v); - } - modifications.insert(k); - } - - // Save modified storage. These are ordered by the block number. - let block_changes = BlockChanges { - storage: modifications, - child_storage: child_modifications, - number: *number, - hash: hash.clone(), - is_canon: is_best, - parent: parent.clone(), - }; - let insert_at = cache.modifications.iter() - .enumerate() - .find(|&(_, m)| m.number < *number) - .map(|(i, _)| i); - trace!("Inserting modifications at {:?}", insert_at); - if let Some(insert_at) = insert_at { - cache.modifications.insert(insert_at, block_changes); - } else { - cache.modifications.push_back(block_changes); - } - } - } - -} - -impl, B: BlockT> CachingState { - /// Create a new instance wrapping generic State and shared cache. - pub fn new(state: S, shared_cache: SharedCache, parent_hash: Option) -> CachingState { - CachingState { - state, - cache: CacheChanges { - shared_cache, - local_cache: RwLock::new(LocalCache { - storage: Default::default(), - hashes: Default::default(), - child_storage: Default::default(), - }), - parent_hash: parent_hash, - }, - } - } - - /// Check if the key can be returned from cache by matching current block parent hash against canonical - /// state and filtering out entries modified in later blocks. - fn is_allowed( - key: Option<&[u8]>, - child_key: Option<&ChildStorageKey>, - parent_hash: &Option, - modifications: - &VecDeque> - ) -> bool - { - let mut parent = match *parent_hash { - None => { - trace!("Cache lookup skipped for {:?}: no parent hash", key); - return false; - } - Some(ref parent) => parent, - }; - if modifications.is_empty() { - trace!("Cache lookup allowed for {:?}", key); - return true; - } - // Ignore all storage modified in later blocks - // Modifications contains block ordered by the number - // We search for our parent in that list first and then for - // all its parent until we hit the canonical block, - // checking against all the intermediate modifications. - for m in modifications { - if &m.hash == parent { - if m.is_canon { - return true; - } - parent = &m.parent; - } - if let Some(key) = key { - if m.storage.contains(key) { - trace!("Cache lookup skipped for {:?}: modified in a later block", key); - return false; - } - } - if let Some(child_key) = child_key { - if m.child_storage.contains(child_key) { - trace!("Cache lookup skipped for {:?}: modified in a later block", child_key); - return false; - } - } - } - trace!("Cache lookup skipped for {:?}: parent hash is unknown", key); - false - } - - /// Dispose state and return cache data. - pub fn release(self) -> CacheChanges { - self.cache - } -} - -impl, B: BlockT> StateBackend for CachingState { - type Error = S::Error; - type Transaction = S::Transaction; - type TrieBackendStorage = S::TrieBackendStorage; - - fn storage(&self, key: &[u8]) -> Result>, Self::Error> { - let local_cache = self.cache.local_cache.upgradable_read(); - // Note that local cache makes that lru is not refreshed - if let Some(entry) = local_cache.storage.get(key).cloned() { - trace!("Found in local cache: {:?}", key); - return Ok(entry) - } - let mut cache = self.cache.shared_cache.lock(); - if Self::is_allowed(Some(key), None, &self.cache.parent_hash, &cache.modifications) { - if let Some(entry) = cache.lru_storage.get(key).map(|a| a.clone()) { - trace!("Found in shared cache: {:?}", key); - return Ok(entry) - } - } - trace!("Cache miss: {:?}", key); - let value = self.state.storage(key)?; - RwLockUpgradableReadGuard::upgrade(local_cache).storage.insert(key.to_vec(), value.clone()); - Ok(value) - } - - fn storage_hash(&self, key: &[u8]) -> Result, Self::Error> { - let local_cache = self.cache.local_cache.upgradable_read(); - if let Some(entry) = local_cache.hashes.get(key).cloned() { - trace!("Found hash in local cache: {:?}", key); - return Ok(entry) - } - let mut cache = self.cache.shared_cache.lock(); - if Self::is_allowed(Some(key), None, &self.cache.parent_hash, &cache.modifications) { - if let Some(entry) = cache.lru_hashes.get(key).map(|a| a.0.clone()) { - trace!("Found hash in shared cache: {:?}", key); - return Ok(entry) - } - } - trace!("Cache hash miss: {:?}", key); - let hash = self.state.storage_hash(key)?; - RwLockUpgradableReadGuard::upgrade(local_cache).hashes.insert(key.to_vec(), hash.clone()); - Ok(hash) - } - - fn child_storage(&self, storage_key: &[u8], key: &[u8]) -> Result>, Self::Error> { - let key = (storage_key.to_vec(), key.to_vec()); - let local_cache = self.cache.local_cache.upgradable_read(); - if let Some(entry) = local_cache.child_storage.get(&key).cloned() { - trace!("Found in local cache: {:?}", key); - return Ok(entry) - } - let mut cache = self.cache.shared_cache.lock(); - if Self::is_allowed(None, Some(&key), &self.cache.parent_hash, &cache.modifications) { - if let Some(entry) = cache.lru_child_storage.get(&key).map(|a| a.clone()) { - trace!("Found in shared cache: {:?}", key); - return Ok(entry) - } - } - trace!("Cache miss: {:?}", key); - let value = self.state.child_storage(storage_key, &key.1[..])?; - RwLockUpgradableReadGuard::upgrade(local_cache).child_storage.insert(key, value.clone()); - Ok(value) - } - - fn exists_storage(&self, key: &[u8]) -> Result { - Ok(self.storage(key)?.is_some()) - } - - fn exists_child_storage(&self, storage_key: &[u8], key: &[u8]) -> Result { - self.state.exists_child_storage(storage_key, key) - } - - fn for_keys_with_prefix(&self, prefix: &[u8], f: F) { - self.state.for_keys_with_prefix(prefix, f) - } - - fn for_key_values_with_prefix(&self, prefix: &[u8], f: F) { - self.state.for_key_values_with_prefix(prefix, f) - } - - fn for_keys_in_child_storage(&self, storage_key: &[u8], f: F) { - self.state.for_keys_in_child_storage(storage_key, f) - } - - fn for_child_keys_with_prefix(&self, storage_key: &[u8], prefix: &[u8], f: F) { - self.state.for_child_keys_with_prefix(storage_key, prefix, f) - } - - fn storage_root(&self, delta: I) -> (H::Out, Self::Transaction) - where - I: IntoIterator, Option>)>, - H::Out: Ord - { - self.state.storage_root(delta) - } - - fn child_storage_root(&self, storage_key: &[u8], delta: I) -> (Vec, bool, Self::Transaction) - where - I: IntoIterator, Option>)>, - H::Out: Ord - { - self.state.child_storage_root(storage_key, delta) - } - - fn pairs(&self) -> Vec<(Vec, Vec)> { - self.state.pairs() - } - - fn keys(&self, prefix: &[u8]) -> Vec> { - self.state.keys(prefix) - } - - fn child_keys(&self, child_key: &[u8], prefix: &[u8]) -> Vec> { - self.state.child_keys(child_key, prefix) - } - - fn as_trie_backend(&mut self) -> Option<&TrieBackend> { - self.state.as_trie_backend() - } -} - -#[cfg(test)] -mod tests { - use super::*; - use sr_primitives::testing::{H256, Block as RawBlock, ExtrinsicWrapper}; - use state_machine::backend::InMemory; - use primitives::Blake2Hasher; - - type Block = RawBlock>; - #[test] - fn smoke() { - //init_log(); - let root_parent = H256::random(); - let key = H256::random()[..].to_vec(); - let h0 = H256::random(); - let h1a = H256::random(); - let h1b = H256::random(); - let h2a = H256::random(); - let h2b = H256::random(); - let h3a = H256::random(); - let h3b = H256::random(); - - let shared = new_shared_cache::(256*1024, (0,1)); - - // blocks [ 3a(c) 2a(c) 2b 1b 1a(c) 0 ] - // state [ 5 5 4 3 2 2 ] - let mut s = CachingState::new(InMemory::::default(), shared.clone(), Some(root_parent.clone())); - s.cache.sync_cache(&[], &[], vec![(key.clone(), Some(vec![2]))], vec![], Some(h0.clone()), Some(0), || true); - - let mut s = CachingState::new(InMemory::::default(), shared.clone(), Some(h0.clone())); - s.cache.sync_cache(&[], &[], vec![], vec![], Some(h1a.clone()), Some(1), || true); - - let mut s = CachingState::new(InMemory::::default(), shared.clone(), Some(h0.clone())); - s.cache.sync_cache(&[], &[], vec![(key.clone(), Some(vec![3]))], vec![], Some(h1b.clone()), Some(1), || false); - - let mut s = CachingState::new(InMemory::::default(), shared.clone(), Some(h1b.clone())); - s.cache.sync_cache(&[], &[], vec![(key.clone(), Some(vec![4]))], vec![], Some(h2b.clone()), Some(2), || false); - - let mut s = CachingState::new(InMemory::::default(), shared.clone(), Some(h1a.clone())); - s.cache.sync_cache(&[], &[], vec![(key.clone(), Some(vec![5]))], vec![], Some(h2a.clone()), Some(2), || true); - - let mut s = CachingState::new(InMemory::::default(), shared.clone(), Some(h2a.clone())); - s.cache.sync_cache(&[], &[], vec![], vec![], Some(h3a.clone()), Some(3), || true); - - let s = CachingState::new(InMemory::::default(), shared.clone(), Some(h3a.clone())); - assert_eq!(s.storage(&key).unwrap().unwrap(), vec![5]); - - let s = CachingState::new(InMemory::::default(), shared.clone(), Some(h1a.clone())); - assert!(s.storage(&key).unwrap().is_none()); - - let s = CachingState::new(InMemory::::default(), shared.clone(), Some(h2b.clone())); - assert!(s.storage(&key).unwrap().is_none()); - - let s = CachingState::new(InMemory::::default(), shared.clone(), Some(h1b.clone())); - assert!(s.storage(&key).unwrap().is_none()); - - // reorg to 3b - // blocks [ 3b(c) 3a 2a 2b(c) 1b 1a 0 ] - let mut s = CachingState::new(InMemory::::default(), shared.clone(), Some(h2b.clone())); - s.cache.sync_cache( - &[h1b.clone(), h2b.clone(), h3b.clone()], - &[h1a.clone(), h2a.clone(), h3a.clone()], - vec![], - vec![], - Some(h3b.clone()), - Some(3), - || true, - ); - let s = CachingState::new(InMemory::::default(), shared.clone(), Some(h3a.clone())); - assert!(s.storage(&key).unwrap().is_none()); - } - - #[test] - fn should_track_used_size_correctly() { - let root_parent = H256::random(); - let shared = new_shared_cache::(109, ((109-36), 109)); - let h0 = H256::random(); - - let mut s = CachingState::new(InMemory::::default(), shared.clone(), Some(root_parent.clone())); - - let key = H256::random()[..].to_vec(); - let s_key = H256::random()[..].to_vec(); - s.cache.sync_cache( - &[], - &[], - vec![(key.clone(), Some(vec![1, 2, 3]))], - vec![], - Some(h0.clone()), - Some(0), - || true, - ); - // 32 key, 3 byte size - assert_eq!(shared.lock().used_storage_cache_size(), 35 /* bytes */); - - let key = H256::random()[..].to_vec(); - s.cache.sync_cache( - &[], - &[], - vec![], - vec![(s_key.clone(), vec![(key.clone(), Some(vec![1, 2]))])], - Some(h0.clone()), - Some(0), - || true, - ); - // 35 + (2 * 32) key, 2 byte size - assert_eq!(shared.lock().used_storage_cache_size(), 101 /* bytes */); - } - - #[test] - fn should_remove_lru_items_based_on_tracking_used_size() { - let root_parent = H256::random(); - let shared = new_shared_cache::(36*3, (2,3)); - let h0 = H256::random(); - - let mut s = CachingState::new(InMemory::::default(), shared.clone(), Some(root_parent.clone())); - - let key = H256::random()[..].to_vec(); - s.cache.sync_cache( - &[], - &[], - vec![(key.clone(), Some(vec![1, 2, 3, 4]))], - vec![], - Some(h0.clone()), - Some(0), - || true, - ); - // 32 key, 4 byte size - assert_eq!(shared.lock().used_storage_cache_size(), 36 /* bytes */); - - let key = H256::random()[..].to_vec(); - s.cache.sync_cache( - &[], - &[], - vec![(key.clone(), Some(vec![1, 2]))], - vec![], - Some(h0.clone()), - Some(0), - || true, - ); - // 32 key, 2 byte size - assert_eq!(shared.lock().used_storage_cache_size(), 34 /* bytes */); - } -} diff --git a/core/consensus/aura/primitives/Cargo.toml b/core/consensus/aura/primitives/Cargo.toml deleted file mode 100644 index 7fd3f3d05d9bf..0000000000000 --- a/core/consensus/aura/primitives/Cargo.toml +++ /dev/null @@ -1,23 +0,0 @@ -[package] -name = "substrate-consensus-aura-primitives" -version = "2.0.0" -authors = ["Parity Technologies "] -description = "Primitives for Aura consensus" -edition = "2018" - -[dependencies] -codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false } -substrate-client = { path = "../../../client", default-features = false } -app-crypto = { package = "substrate-application-crypto", path = "../../../application-crypto", default-features = false } -rstd = { package = "sr-std", path = "../../../sr-std", default-features = false } -sr-primitives = { path = "../../../sr-primitives", default-features = false } - -[features] -default = ["std"] -std = [ - "rstd/std", - "codec/std", - "sr-primitives/std", - "substrate-client/std", - "app-crypto/std", -] diff --git a/core/consensus/pow/Cargo.toml b/core/consensus/pow/Cargo.toml deleted file mode 100644 index 86efcbb95da2b..0000000000000 --- a/core/consensus/pow/Cargo.toml +++ /dev/null @@ -1,19 +0,0 @@ -[package] -name = "substrate-consensus-pow" -version = "2.0.0" -authors = ["Parity Technologies "] -description = "PoW consensus algorithm for substrate" -edition = "2018" - -[dependencies] -codec = { package = "parity-scale-codec", version = "1.0.0", features = ["derive"] } -primitives = { package = "substrate-primitives", path = "../../primitives" } -sr-primitives = { path = "../../sr-primitives" } -client = { package = "substrate-client", path = "../../client" } -srml-timestamp = { path = "../../../srml/timestamp" } -inherents = { package = "substrate-inherents", path = "../../inherents" } -pow-primitives = { package = "substrate-consensus-pow-primitives", path = "primitives" } -consensus-common = { package = "substrate-consensus-common", path = "../common" } -log = "0.4.8" -futures-preview = { version = "0.3.0-alpha.19", features = ["compat"] } -derive_more = "0.15.0" diff --git a/core/consensus/rhd/Cargo.toml b/core/consensus/rhd/Cargo.toml deleted file mode 100644 index 48d0840b5592d..0000000000000 --- a/core/consensus/rhd/Cargo.toml +++ /dev/null @@ -1,37 +0,0 @@ -[package] -name = "substrate-consensus-rhd" -version = "2.0.0" -authors = ["Parity Technologies "] -description = "Rhododendron Round-Based consensus-algorithm for substrate" -edition = "2018" - -[dependencies] -derive_more = "0.15.0" -futures = "0.1.29" -codec = { package = "parity-scale-codec", version = "1.0.0", features = ["derive"] } -primitives = { package = "substrate-primitives", path = "../../primitives" } -consensus = { package = "substrate-consensus-common", path = "../common" } -client = { package = "substrate-client", path = "../../client" } -transaction_pool = { package = "substrate-transaction-pool", path = "../../transaction-pool" } -runtime_support = { package = "srml-support", path = "../../../srml/support" } -srml-system = { path = "../../../srml/system" } -sr-primitives = { path = "../../sr-primitives" } -runtime_version = { package = "sr-version", path = "../../sr-version" } -runtime_io = { package = "sr-io", path = "../../sr-io" } -tokio = "0.1.22" -parking_lot = "0.9.0" -log = "0.4.8" -rhododendron = { version = "0.7.0", features = ["codec"] } -exit-future = "0.1.4" - -[dev-dependencies] -keyring = { package = "substrate-keyring", path = "../../keyring" } - -[features] -default = ["std"] -std = [ - "primitives/std", - "runtime_support/std", - "sr-primitives/std", - "runtime_version/std", -] diff --git a/core/consensus/rhd/src/error.rs b/core/consensus/rhd/src/error.rs deleted file mode 100644 index 601cf1c963a58..0000000000000 --- a/core/consensus/rhd/src/error.rs +++ /dev/null @@ -1,50 +0,0 @@ -// Copyright 2018-2019 Parity Technologies (UK) Ltd. -// This file is part of Substrate. - -// Substrate is free software: you can redistribute it and/or modify -// it under the terms of the GNU General Public License as published by -// the Free Software Foundation, either version 3 of the License, or -// (at your option) any later version. - -// Substrate is distributed in the hope that it will be useful, -// but WITHOUT ANY WARRANTY; without even the implied warranty of -// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -// GNU General Public License for more details. - -// You should have received a copy of the GNU General Public License -// along with Substrate. If not, see . - -//! Error types in the rhododendron Consensus service. -use consensus::error::{Error as CommonError}; -use primitives::AuthorityId; -use client; - -/// A result alias. -pub type Result = std::result::Result; - -/// A RHD error type. -#[derive(Debug, derive_more::Display, derive_more::From)] -pub enum Error { - /// Client error. - Client(client::error::Error), - /// Consensus error. - Common(CommonError), - /// Local account ID not a validator at this block. - #[display(fmt="Local account ID ({:?}) not a validator at this block.", _0)] - NotValidator(AuthorityId), - /// Proposer destroyed before finishing proposing or evaluating - #[display(fmt="Proposer destroyed before finishing proposing or evaluating")] - PrematureDestruction, - /// Failed to register or resolve async timer. - #[display(fmt="Timer failed: {}", _0)] - Timer(tokio::timer::Error), - /// Unable to dispatch agreement future - #[display(fmt="Unable to dispatch agreement future: {:?}", _0)] - Executor(futures::future::ExecuteErrorKind), -} - -impl From<::rhododendron::InputStreamConcluded> for Error { - fn from(_: ::rhododendron::InputStreamConcluded) -> Self { - CommonError::IoTerminated.into() - } -} diff --git a/core/consensus/rhd/src/lib.rs b/core/consensus/rhd/src/lib.rs deleted file mode 100644 index 3bcd1346d47c9..0000000000000 --- a/core/consensus/rhd/src/lib.rs +++ /dev/null @@ -1,1699 +0,0 @@ -// Copyright 2017-2019 Parity Technologies (UK) Ltd. -// This file is part of Substrate. - -// Substrate is free software: you can redistribute it and/or modify -// it under the terms of the GNU General Public License as published by -// the Free Software Foundation, either version 3 of the License, or -// (at your option) any later version. - -// Substrate is distributed in the hope that it will be useful, -// but WITHOUT ANY WARRANTY; without even the implied warranty of -// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -// GNU General Public License for more details. - -// You should have received a copy of the GNU General Public License -// along with Substrate. If not, see . - -//! BFT Agreement based on a rotating proposer in different rounds. -//! -//! Where this crate refers to input stream, should never logically conclude. -//! The logic in this crate assumes that messages flushed to the output stream -//! will eventually reach other nodes and that our own messages are not included -//! in the input stream. -//! -//! Note that it is possible to witness agreement being reached without ever -//! seeing the candidate. Any candidates seen will be checked for validity. -//! -//! Although technically the agreement will always complete (given the eventual -//! delivery of messages), in practice it is possible for this future to -//! conclude without having witnessed the conclusion. -//! In general, this future should be pre-empted by the import of a justification -//! set for this block height. - -#![cfg(feature="rhd")] -// FIXME #1020 doesn't compile -// NOTE: this is the legacy constant used for transaction size. No longer used except -// for the rhd code which is not updated. Placed here for compatibility. -const MAX_TRANSACTIONS_SIZE: u32 = 4 * 1024 * 1024; - -use std::sync::Arc; -use std::sync::atomic::{AtomicUsize, Ordering}; -use std::time::{self, Instant, Duration}; - -use codec::{Decode, Encode}; -use consensus::offline_tracker::OfflineTracker; -use consensus::error::{ErrorKind as CommonErrorKind}; -use consensus::{Authorities, BlockImport, Environment, Proposer as BaseProposer}; -use client::{Client as SubstrateClient, CallExecutor}; -use client::runtime_api::{Core, BlockBuilder as BlockBuilderAPI, OldTxQueue, BlockBuilderError}; -use sr_primitives::generic::{BlockId, Era, ImportResult, BlockImportParams, BlockOrigin}; -use sr_primitives::traits::{Block, Header}; -use sr_primitives::traits::{ - Block as BlockT, Hash as HashT, Header as HeaderT, - BlockNumberToHash, SaturatedConversion -}; -use sr_primitives::Justification; -use primitives::{AuthorityId, ed25519, Blake2Hasher, ed25519::LocalizedSignature}; -use srml_system::Trait as SystemT; - -use node_runtime::Runtime; -use transaction_pool::txpool::{self, Pool as TransactionPool}; - -use futures::prelude::*; -use futures::future; -use futures::sync::oneshot; -use tokio::runtime::TaskExecutor; -use tokio::timer::Delay; -use parking_lot::{RwLock, Mutex}; - -pub use rhododendron::{ - self, InputStreamConcluded, AdvanceRoundReason, Message as RhdMessage, - Vote as RhdMessageVote, Communication as RhdCommunication, -}; -pub use self::error::{Error, ErrorKind}; - -// pub mod misbehavior_check; -mod error; -mod service; - -// statuses for an agreement -mod status { - pub const LIVE: usize = 0; - pub const BAD: usize = 1; - pub const GOOD: usize = 2; -} - -pub type Timestamp = u64; - -pub type AccountId = ::primitives::H256; - -/// Localized message type. -pub type LocalizedMessage = rhododendron::LocalizedMessage< - B, - ::Hash, - AuthorityId, - LocalizedSignature ->; - -/// Justification of some hash. -pub struct RhdJustification(rhododendron::Justification); - -/// Justification of a prepare message. -pub struct PrepareJustification(rhododendron::PrepareJustification); - -/// Unchecked justification. -#[derive(Encode, Decode)] -pub struct UncheckedJustification(rhododendron::UncheckedJustification); - -impl UncheckedJustification { - /// Create a new, unchecked justification. - pub fn new(digest: H, signatures: Vec, round_number: u32) -> Self { - UncheckedJustification(rhododendron::UncheckedJustification { - digest, - signatures, - round_number, - }) - } -} - -impl UncheckedJustification { - /// Decode a justification. - pub fn decode_justification(justification: Justification) -> Option { - let inner: rhododendron::UncheckedJustification<_, _> = Decode::decode(&mut &justification[..])?; - - Some(UncheckedJustification(inner)) - } -} - -impl Into for UncheckedJustification { - fn into(self) -> Justification { - self.0.encode() - } -} - -impl From> for UncheckedJustification { - fn from(inner: rhododendron::UncheckedJustification) -> Self { - UncheckedJustification(inner) - } -} - -/// Result of a committed round of BFT -pub type Committed = rhododendron::Committed::Hash, LocalizedSignature>; - -/// Communication between BFT participants. -pub type Communication = rhododendron::Communication::Hash, AuthorityId, LocalizedSignature>; - -/// Misbehavior observed from BFT participants. -pub type Misbehavior = rhododendron::Misbehavior; - -/// Shared offline validator tracker. -pub type SharedOfflineTracker = Arc>; - -/// A proposer for a rhododendron instance. This must implement the base proposer logic. -pub trait LocalProposer: BaseProposer { - /// Import witnessed rhododendron misbehavior. - fn import_misbehavior(&self, misbehavior: Vec<(AuthorityId, Misbehavior)>); - - /// Determine the proposer for a given round. This should be a deterministic function - /// with consistent results across all authorities. - fn round_proposer(&self, round_number: u32, authorities: &[AuthorityId]) -> AuthorityId; - - /// Hook called when a BFT round advances without a proposal. - fn on_round_end(&self, _round_number: u32, _proposed: bool) { } -} - - -/// Build new blocks. -pub trait BlockBuilder { - /// Push an extrinsic onto the block. Fails if the extrinsic is invalid. - fn push_extrinsic(&mut self, extrinsic: ::Extrinsic) -> Result<(), Error>; -} - -/// Local client abstraction for the consensus. -pub trait AuthoringApi: - Send - + Sync - + BlockBuilderAPI<::Block, InherentData, Error=::Error> - + Core<::Block, AuthorityId, Error=::Error> - + OldTxQueue<::Block, Error=::Error> -{ - /// The block used for this API type. - type Block: BlockT; - /// The error used by this API type. - type Error: std::error::Error; - - /// Build a block on top of the given, with inherent extrinsics pre-pushed. - fn build_block) -> ()>( - &self, - at: &BlockId, - inherent_data: InherentData, - build_ctx: F, - ) -> Result; -} - -/// A long-lived network which can create BFT message routing processes on demand. -pub trait Network { - /// The block used for this API type. - type Block: BlockT; - /// The input stream of BFT messages. Should never logically conclude. - type Input: Stream,Error=Error>; - /// The output sink of BFT messages. Messages sent here should eventually pass to all - /// current authorities. - type Output: Sink,SinkError=Error>; - - /// Instantiate input and output streams. - fn communication_for( - &self, - validators: &[AuthorityId], - local_id: AuthorityId, - parent_hash: ::Hash, - task_executor: TaskExecutor - ) -> (Self::Input, Self::Output); -} - - -// caches the round number to start at if we end up with BFT consensus on the same -// parent hash more than once (happens if block is bad). -// -// this will force a committed but locally-bad block to be considered analogous to -// a round advancement vote. -#[derive(Debug)] -struct RoundCache { - hash: Option, - start_round: u32, -} - -/// Instance of BFT agreement. -struct BftInstance { - key: Arc, - authorities: Vec, - parent_hash: B::Hash, - round_timeout_multiplier: u64, - cache: Arc>>, - proposer: P, -} - -impl> BftInstance - where - B: Clone + Eq, - B::Hash: ::std::hash::Hash - -{ - fn round_timeout_duration(&self, round: u32) -> Duration { - // 2^(min(6, x/8)) * 10 - // Grows exponentially starting from 10 seconds, capped at 640 seconds. - const ROUND_INCREMENT_STEP: u32 = 8; - - let round = round / ROUND_INCREMENT_STEP; - let round = ::std::cmp::min(6, round); - - let timeout = 1u64.checked_shl(round) - .unwrap_or_else(u64::max_value) - .saturating_mul(self.round_timeout_multiplier); - - Duration::from_secs(timeout) - } - - fn update_round_cache(&self, current_round: u32) { - let mut cache = self.cache.lock(); - if cache.hash.as_ref() == Some(&self.parent_hash) { - cache.start_round = current_round + 1; - } - } -} - -impl> rhododendron::Context for BftInstance - where - B: Clone + Eq, - B::Hash: ::std::hash::Hash, -{ - type Error = P::Error; - type AuthorityId = AuthorityId; - type Digest = B::Hash; - type Signature = LocalizedSignature; - type Candidate = B; - type RoundTimeout = Box>; - type CreateProposal = ::Future; - type EvaluateProposal = ::Future; - - fn local_id(&self) -> AuthorityId { - self.key.public().into() - } - - fn proposal(&self) -> Self::CreateProposal { - self.proposer.propose().into_future() - } - - fn candidate_digest(&self, proposal: &B) -> B::Hash { - proposal.hash() - } - - fn sign_local(&self, message: RhdMessage) -> LocalizedMessage { - sign_message(message, &*self.key, self.parent_hash.clone()) - } - - fn round_proposer(&self, round: u32) -> AuthorityId { - self.proposer.round_proposer(round, &self.authorities[..]) - } - - fn proposal_valid(&self, proposal: &B) -> Self::EvaluateProposal { - self.proposer.evaluate(proposal).into_future() - } - - fn begin_round_timeout(&self, round: u32) -> Self::RoundTimeout { - let timeout = self.round_timeout_duration(round); - let fut = Delay::new(Instant::now() + timeout) - .map_err(|e| Error::from(CommonErrorKind::FaultyTimer(e))) - .map_err(Into::into); - - Box::new(fut) - } - - fn on_advance_round( - &self, - accumulator: &rhododendron::Accumulator, - round: u32, - next_round: u32, - reason: AdvanceRoundReason, - ) { - use std::collections::HashSet; - - let collect_pubkeys = |participants: HashSet<&Self::AuthorityId>| participants.into_iter() - .map(|p| ::ed25519::Public::from_raw(p.0)) - .collect::>(); - - let round_timeout = self.round_timeout_duration(next_round); - debug!(target: "rhd", "Advancing to round {} from {}", next_round, round); - debug!(target: "rhd", "Participating authorities: {:?}", - collect_pubkeys(accumulator.participants())); - debug!(target: "rhd", "Voting authorities: {:?}", - collect_pubkeys(accumulator.voters())); - debug!(target: "rhd", "Round {} should end in at most {} seconds from now", next_round, round_timeout.as_secs()); - - self.update_round_cache(next_round); - - if let AdvanceRoundReason::Timeout = reason { - self.proposer.on_round_end(round, accumulator.proposal().is_some()); - } - } -} - -/// A future that resolves either when canceled (witnessing a block from the network at same height) -/// or when agreement completes. -pub struct BftFuture where - B: Block + Clone + Eq, - B::Hash: ::std::hash::Hash, - P: LocalProposer, - P: BaseProposer, - InStream: Stream, Error=Error>, - OutSink: Sink, SinkError=Error>, -{ - inner: rhododendron::Agreement, InStream, OutSink>, - status: Arc, - cancel: oneshot::Receiver<()>, - import: Arc, -} - -impl Future for BftFuture where - B: Block + Clone + Eq, - B::Hash: ::std::hash::Hash, - P: LocalProposer, - P: BaseProposer, - I: BlockImport, - InStream: Stream, Error=Error>, - OutSink: Sink, SinkError=Error>, -{ - type Item = (); - type Error = (); - - fn poll(&mut self) -> ::futures::Poll<(), ()> { - // service has canceled the future. bail - let cancel = match self.cancel.poll() { - Ok(Async::Ready(())) | Err(_) => true, - Ok(Async::NotReady) => false, - }; - - let committed = match self.inner.poll().map_err(|_| ()) { - Ok(Async::Ready(x)) => x, - Ok(Async::NotReady) => - return Ok(if cancel { Async::Ready(()) } else { Async::NotReady }), - Err(()) => return Err(()), - }; - - // if something was committed, the round leader must have proposed. - self.inner.context().proposer.on_round_end(committed.round_number, true); - - // If we didn't see the proposal (very unlikely), - // we will get the block from the network later. - if let Some(justified_block) = committed.candidate { - let hash = justified_block.hash(); - info!(target: "rhd", "Importing block #{} ({}) directly from BFT consensus", - justified_block.header().number(), hash); - let just: Justification = UncheckedJustification(committed.justification.uncheck()).into(); - let (header, body) = justified_block.deconstruct(); - let import_block = BlockImportParams { - origin: BlockOrigin::ConsensusBroadcast, - header: header, - justification: Some(just), - body: Some(body), - finalized: true, - post_digests: Default::default(), - auxiliary: Default::default() - }; - - let new_status = match self.import.import_block(import_block, None) { - Err(e) => { - warn!(target: "rhd", "Error importing block {:?} in round #{}: {:?}", - hash, committed.round_number, e); - status::BAD - } - Ok(ImportResult::KnownBad) => { - warn!(target: "rhd", "{:?} was bad block agreed on in round #{}", - hash, committed.round_number); - status::BAD - } - _ => status::GOOD - }; - - self.status.store(new_status, Ordering::Release); - - } else { - // assume good unless we received the proposal. - self.status.store(status::GOOD, Ordering::Release); - } - - self.inner.context().update_round_cache(committed.round_number); - - Ok(Async::Ready(())) - } -} - -impl Drop for BftFuture where - B: Block + Clone + Eq, - B::Hash: ::std::hash::Hash, - P: LocalProposer, - P: BaseProposer, - InStream: Stream, Error=Error>, - OutSink: Sink, SinkError=Error>, -{ - fn drop(&mut self) { - let misbehavior = self.inner.drain_misbehavior().collect::>(); - self.inner.context().proposer.import_misbehavior(misbehavior); - } -} - -struct AgreementHandle { - status: Arc, - send_cancel: Option>, -} - -impl AgreementHandle { - fn status(&self) -> usize { - self.status.load(Ordering::Acquire) - } -} - -impl Drop for AgreementHandle { - fn drop(&mut self) { - if let Some(sender) = self.send_cancel.take() { - let _ = sender.send(()); - } - } -} - -/// The BftService kicks off the agreement process on top of any blocks it -/// is notified of. -/// -/// This assumes that it is being run in the context of a tokio runtime. -pub struct BftService { - client: Arc, - live_agreement: Mutex>, - round_cache: Arc>>, - round_timeout_multiplier: u64, - key: Arc, - factory: P, -} - -impl BftService - where - B: Block + Clone + Eq, - P: Environment, - P::Proposer: LocalProposer, - P::Proposer: BaseProposer, - I: BlockImport + Authorities, -{ - /// Create a new service instance. - pub fn new(client: Arc, key: Arc, factory: P) -> BftService { - BftService { - client: client, - live_agreement: Mutex::new(None), - round_cache: Arc::new(Mutex::new(RoundCache { - hash: None, - start_round: 0, - })), - round_timeout_multiplier: 10, - key: key, - factory, - } - } - - /// Get the local Authority ID. - pub fn local_id(&self) -> AuthorityId { - self.key.public().into() - } - - /// Signal that a valid block with the given header has been imported. - /// Provide communication streams that are localized to this block. - /// It's recommended to use the communication primitives provided by this - /// module for signature checking and decoding. See `CheckedStream` and - /// `SigningSink` for more details. - /// - /// Messages received on the stream that don't match the expected format - /// will be dropped. - /// - /// If the local signing key is an authority, this will begin the consensus process to build a - /// block on top of it. If the executor fails to run the future, an error will be returned. - /// Returns `None` if the agreement on the block with given parent is already in progress. - pub fn build_upon(&self, header: &B::Header, input: In, output: Out) - -> Result>::Proposer, - I, - In, - Out, - >>, P::Error> - where - In: Stream, Error=Error>, - Out: Sink, SinkError=Error>, - { - let hash = header.hash(); - - let mut live_agreement = self.live_agreement.lock(); - let can_build = live_agreement.as_ref() - .map_or(true, |x| self.can_build_on_inner(header, x)); - - if !can_build { - return Ok(None) - } - - let authorities = self.client.authorities(&BlockId::Hash(hash.clone())) - .map_err(|e| CommonErrorKind::Other(Box::new(e)).into())?; - - let n = authorities.len(); - let max_faulty = max_faulty_of(n); - trace!(target: "rhd", "Initiating agreement on top of #{}, {:?}", header.number(), hash); - trace!(target: "rhd", "max_faulty_of({})={}", n, max_faulty); - - let local_id = self.local_id(); - - if !authorities.contains(&local_id) { - // cancel current agreement - live_agreement.take(); - Err(CommonErrorKind::InvalidAuthority(local_id).into())?; - } - - let proposer = self.factory.init(header, &authorities, self.key.clone())?; - - let bft_instance = BftInstance { - proposer, - parent_hash: hash.clone(), - cache: self.round_cache.clone(), - round_timeout_multiplier: self.round_timeout_multiplier, - key: self.key.clone(), - authorities: authorities, - }; - - let mut agreement = rhododendron::agree( - bft_instance, - n, - max_faulty, - input, - output, - ); - - // fast forward round number if necessary. - { - let mut cache = self.round_cache.lock(); - trace!(target: "rhd", "Round cache: {:?}", &*cache); - if cache.hash.as_ref() == Some(&hash) { - trace!(target: "rhd", "Fast-forwarding to round {}", cache.start_round); - let start_round = cache.start_round; - cache.start_round += 1; - - drop(cache); - agreement.fast_forward(start_round); - } else { - *cache = RoundCache { - hash: Some(hash.clone()), - start_round: 1, - }; - } - } - - let status = Arc::new(AtomicUsize::new(status::LIVE)); - let (tx, rx) = oneshot::channel(); - - // cancel current agreement. - *live_agreement = Some((header.clone(), AgreementHandle { - send_cancel: Some(tx), - status: status.clone(), - })); - - Ok(Some(BftFuture { - inner: agreement, - status: status, - cancel: rx, - import: self.client.clone(), - })) - } - - /// Cancel current agreement if any. - pub fn cancel_agreement(&self) { - self.live_agreement.lock().take(); - } - - /// Whether we can build using the given header. - pub fn can_build_on(&self, header: &B::Header) -> bool { - self.live_agreement.lock().as_ref() - .map_or(true, |x| self.can_build_on_inner(header, x)) - } - - /// Get a reference to the underlying client. - pub fn client(&self) -> &I { &*self.client } - - fn can_build_on_inner(&self, header: &B::Header, live: &(B::Header, AgreementHandle)) -> bool { - let hash = header.hash(); - let &(ref live_header, ref handle) = live; - match handle.status() { - _ if *header != *live_header && *live_header.parent_hash() != hash => true, // can always follow with next block. - status::BAD => hash == live_header.hash(), // bad block can be re-agreed on. - _ => false, // canceled won't appear since we overwrite the handle before returning. - } - } -} - -/// Stream that decodes rhododendron messages and checks signatures. -/// -/// This stream is localized to a specific parent block-hash, as all messages -/// will be signed in a way that accounts for it. When using this with -/// `BftService::build_upon`, the user should take care to use the same hash as for that. -pub struct CheckedStream { - inner: S, - local_id: AuthorityId, - authorities: Vec, - parent_hash: B::Hash, -} - -impl CheckedStream { - /// Construct a new checked stream. - pub fn new( - inner: S, - local_id: AuthorityId, - authorities: Vec, - parent_hash: B::Hash, - ) -> Self { - CheckedStream { - inner, - local_id, - authorities, - parent_hash, - } - } -} - -impl>> Stream for CheckedStream - where S::Error: From, -{ - type Item = Communication; - type Error = S::Error; - - fn poll(&mut self) -> Poll, Self::Error> { - use rhododendron::LocalizedMessage as RhdLocalized; - loop { - match self.inner.poll()? { - Async::Ready(Some(item)) => { - let comms: Communication = match Decode::decode(&mut &item[..]) { - Some(x) => x, - None => continue, - }; - - match comms { - RhdCommunication::Auxiliary(prepare_just) => { - let checked = check_prepare_justification::( - &self.authorities, - self.parent_hash, - UncheckedJustification(prepare_just.uncheck()), - ); - if let Ok(checked) = checked { - return Ok(Async::Ready( - Some(RhdCommunication::Auxiliary(checked.0)) - )); - } - } - RhdCommunication::Consensus(RhdLocalized::Propose(p)) => { - if p.sender == self.local_id { continue } - - let checked = check_proposal::( - &self.authorities, - &self.parent_hash, - &p, - ); - - if let Ok(()) = checked { - return Ok(Async::Ready( - Some(RhdCommunication::Consensus(RhdLocalized::Propose(p))) - )); - } - } - RhdCommunication::Consensus(RhdLocalized::Vote(v)) => { - if v.sender == self.local_id { continue } - - let checked = check_vote::( - &self.authorities, - &self.parent_hash, - &v, - ); - - if let Ok(()) = checked { - return Ok(Async::Ready( - Some(RhdCommunication::Consensus(RhdLocalized::Vote(v))) - )); - } - } - } - } - Async::Ready(None) => return Ok(Async::Ready(None)), - Async::NotReady => return Ok(Async::NotReady), - } - } - } -} - -/// Given a total number of authorities, yield the maximum faulty that would be allowed. -/// This will always be under 1/3. -pub fn max_faulty_of(n: usize) -> usize { - n.saturating_sub(1) / 3 -} - -/// Given a total number of authorities, yield the minimum required signatures. -/// This will always be over 2/3. -pub fn bft_threshold(n: usize) -> usize { - n - max_faulty_of(n) -} - -// actions in the signature scheme. -#[derive(Encode)] -enum Action { - Prepare(u32, H), - Commit(u32, H), - AdvanceRound(u32), - // signatures of header hash and full candidate are both included. - ProposeHeader(u32, H), - Propose(u32, B), -} - -// encode something in a way which is localized to a specific parent-hash -fn localized_encode(parent_hash: H, value: E) -> Vec { - (parent_hash, value).encode() -} - -fn check_justification_signed_message( - authorities: &[AuthorityId], - message: &[u8], - just: UncheckedJustification) --> Result, UncheckedJustification> { - // additional error information could be useful here. - just.0.check(authorities.len() - max_faulty_of(authorities.len()), |_, _, sig| { - let auth_id = sig.signer.clone().into(); - if !authorities.contains(&auth_id) { return None } - - if ed25519::Pair::verify(&sig.signature, message, &sig.signer) { - Some(sig.signer.0) - } else { - None - } - }).map(RhdJustification).map_err(UncheckedJustification) -} - -/// Check a full justification for a header hash. -/// Provide all valid authorities. -/// -/// On failure, returns the justification back. -pub fn check_justification( - authorities: &[AuthorityId], - parent: B::Hash, - just: UncheckedJustification -) -> Result, UncheckedJustification> { - let vote: Action = Action::Commit(just.0.round_number as u32, just.0.digest.clone()); - let message = localized_encode(parent, vote); - - check_justification_signed_message(authorities, &message[..], just) -} - -/// Check a prepare justification for a header hash. -/// Provide all valid authorities. -/// -/// On failure, returns the justification back. -pub fn check_prepare_justification( - authorities: &[AuthorityId], - parent: B::Hash, - just: UncheckedJustification -) -> Result, UncheckedJustification> { - let vote: Action = Action::Prepare(just.0.round_number as u32, just.0.digest.clone()); - let message = localized_encode(parent, vote); - - check_justification_signed_message(authorities, &message[..], just).map(|e| PrepareJustification(e.0)) -} - -/// Check proposal message signatures and authority. -/// Provide all valid authorities. -pub fn check_proposal( - authorities: &[AuthorityId], - parent_hash: &B::Hash, - propose: &rhododendron::LocalizedProposal) - -> Result<(), Error> -{ - if !authorities.contains(&propose.sender) { - return Err(CommonErrorKind::InvalidAuthority(propose.sender.into()).into()); - } - - let action_header = Action::ProposeHeader(propose.round_number as u32, propose.digest.clone()); - let action_propose = Action::Propose(propose.round_number as u32, propose.proposal.clone()); - check_action::(action_header, parent_hash, &propose.digest_signature)?; - check_action::(action_propose, parent_hash, &propose.full_signature) -} - -/// Check vote message signatures and authority. -/// Provide all valid authorities. -pub fn check_vote( - authorities: &[AuthorityId], - parent_hash: &B::Hash, - vote: &rhododendron::LocalizedVote) - -> Result<(), Error> -{ - if !authorities.contains(&vote.sender) { - return Err(CommonErrorKind::InvalidAuthority(vote.sender.into()).into()); - } - - let action = match vote.vote { - rhododendron::Vote::Prepare(r, ref h) => Action::Prepare(r as u32, h.clone()), - rhododendron::Vote::Commit(r, ref h) => Action::Commit(r as u32, h.clone()), - rhododendron::Vote::AdvanceRound(r) => Action::AdvanceRound(r as u32), - }; - check_action::(action, parent_hash, &vote.signature) -} - -fn check_action( - action: Action, - parent_hash: &B::Hash, - sig: &LocalizedSignature -) -> Result<(), Error> { - let message = localized_encode(*parent_hash, action); - if ed25519::Pair::verify(&sig.signature, &message, &sig.signer) { - Ok(()) - } else { - Err(CommonErrorKind::InvalidSignature(sig.signature.into(), sig.signer.clone().into()).into()) - } -} - -/// Sign a BFT message with the given key. -pub fn sign_message( - message: RhdMessage, - key: &ed25519::Pair, - parent_hash: B::Hash -) -> LocalizedMessage { - let signer = key.public(); - - let sign_action = |action: Action| { - let to_sign = localized_encode(parent_hash.clone(), action); - - LocalizedSignature { - signer: signer.clone(), - signature: key.sign(&to_sign), - } - }; - - match message { - RhdMessage::Propose(r, proposal) => { - let header_hash = proposal.hash(); - let action_header = Action::ProposeHeader(r as u32, header_hash.clone()); - let action_propose = Action::Propose(r as u32, proposal.clone()); - - rhododendron::LocalizedMessage::Propose(rhododendron::LocalizedProposal { - round_number: r, - proposal, - digest: header_hash, - sender: signer.clone().into(), - digest_signature: sign_action(action_header), - full_signature: sign_action(action_propose), - }) - } - RhdMessage::Vote(vote) => rhododendron::LocalizedMessage::Vote({ - let action = match vote { - RhdMessageVote::Prepare(r, h) => Action::Prepare(r as u32, h), - RhdMessageVote::Commit(r, h) => Action::Commit(r as u32, h), - RhdMessageVote::AdvanceRound(r) => Action::AdvanceRound(r as u32), - }; - - rhododendron::LocalizedVote { - vote: vote, - sender: signer.clone().into(), - signature: sign_action(action), - } - }) - } -} - - -impl<'a, B, E, Block> BlockBuilder for client::block_builder::BlockBuilder<'a, B, E, Block, Blake2Hasher> where - B: client::backend::Backend + Send + Sync + 'static, - E: CallExecutor + Send + Sync + Clone + 'static, - Block: BlockT -{ - fn push_extrinsic(&mut self, extrinsic: ::Extrinsic) -> Result<(), Error> { - client::block_builder::BlockBuilder::push(self, extrinsic).map_err(Into::into) - } -} - -impl<'a, B, E, Block> AuthoringApi for SubstrateClient where - B: client::backend::Backend + Send + Sync + 'static, - E: CallExecutor + Send + Sync + Clone + 'static, - Block: BlockT, -{ - type Block = Block; - type Error = client::error::Error; - - fn build_block) -> ()>( - &self, - at: &BlockId, - inherent_data: InherentData, - mut build_ctx: F, - ) -> Result { - let runtime_version = self.runtime_version_at(at)?; - - let mut block_builder = self.new_block_at(at)?; - if runtime_version.has_api(*b"blkbuild", 1) { - for inherent in self.inherent_extrinsics(at, &inherent_data)? { - block_builder.push(inherent)?; - } - } - - build_ctx(&mut block_builder); - - block_builder.bake().map_err(Into::into) - } -} - - -/// Proposer factory. -pub struct ProposerFactory where - C: AuthoringApi, - A: txpool::ChainApi, -{ - /// The client instance. - pub client: Arc, - /// The transaction pool. - pub transaction_pool: Arc>, - /// The backing network handle. - pub network: N, - /// handle to remote task executor - pub handle: TaskExecutor, - /// Offline-tracker. - pub offline: SharedOfflineTracker, - /// Force delay in evaluation this long. - pub force_delay: u64, -} - -impl consensus::Environment<::Block> for ProposerFactory where - N: Network::Block>, - C: AuthoringApi + BlockNumberToHash, - A: txpool::ChainApi::Block>, - // <::Block as BlockT>::Hash: - // Into<::Hash> + PartialEq + Into, - Error: From<::Error> -{ - type Proposer = Proposer; - type Error = Error; - - fn init( - &self, - parent_header: &<::Block as BlockT>::Header, - authorities: &[AuthorityId], - sign_with: Arc, - ) -> Result { - use sr_primitives::traits::Hash as HashT; - let parent_hash = parent_header.hash(); - - let id = BlockId::hash(parent_hash); - let random_seed = self.client.random_seed(&id)?; - let random_seed = <<::Block as BlockT>::Header as HeaderT> - ::Hashing::hash(random_seed.as_ref()); - - let validators = self.client.validators(&id)?; - self.offline.write().note_new_block(&validators[..]); - - info!("Starting consensus session on top of parent {:?}", parent_hash); - - let local_id = sign_with.public().0.into(); - let (input, output) = self.network.communication_for( - authorities, - local_id, - parent_hash.clone(), - self.handle.clone(), - ); - let now = Instant::now(); - let proposer = Proposer { - client: self.client.clone(), - start: now, - local_key: sign_with, - parent_hash, - parent_id: id, - parent_number: *parent_header.number(), - random_seed, - transaction_pool: self.transaction_pool.clone(), - offline: self.offline.clone(), - validators, - minimum_timestamp: current_timestamp() + self.force_delay, - network: self.network.clone() - }; - - Ok(proposer) - } -} - -/// The proposer logic. -pub struct Proposer { - client: Arc, - start: Instant, - local_key: Arc, - parent_hash: <::Block as BlockT>::Hash, - parent_id: BlockId<::Block>, - parent_number: <<::Block as BlockT>::Header as HeaderT>::Number, - random_seed: <::Block as BlockT>::Hash, - transaction_pool: Arc>, - offline: SharedOfflineTracker, - validators: Vec, - minimum_timestamp: u64, - network: N, -} - -impl Proposer { - fn primary_index(&self, round_number: u32, len: usize) -> usize { - use primitives::uint::U256; - - let big_len = U256::from(len); - let offset = U256::from_big_endian(self.random_seed.as_ref()) % big_len; - let offset = offset.low_u64() as usize + round_number as usize; - offset % len - } -} - -impl BaseProposer<::Block> for Proposer where - C: AuthoringApi + BlockNumberToHash, - A: txpool::ChainApi::Block>, - <::Block as BlockT>::Hash: - Into<::Hash> + PartialEq + Into, - error::Error: From<::Error> -{ - type Create = Result<::Block, Error>; - type Error = Error; - type Evaluate = Box>; - - fn propose(&self) -> Self::Create { - use sr_primitives::traits::BlakeTwo256; - - const MAX_VOTE_OFFLINE_SECONDS: Duration = Duration::from_secs(60); - - let timestamp = ::std::cmp::max(self.minimum_timestamp, current_timestamp()); - - let elapsed_since_start = self.start.elapsed(); - let offline_indices = if elapsed_since_start > MAX_VOTE_OFFLINE_SECONDS { - Vec::new() - } else { - self.offline.read().reports(&self.validators[..]) - }; - - if !offline_indices.is_empty() { - info!( - "Submitting offline validators {:?} for slash-vote", - offline_indices.iter().map(|&i| self.validators[i as usize]).collect::>(), - ) - } - - let inherent_data = InherentData { - timestamp, - offline_indices, - }; - - let block = self.client.build_block( - &self.parent_id, - inherent_data, - |block_builder| { - let mut unqueue_invalid = Vec::new(); - self.transaction_pool.ready(|pending_iterator| { - let mut pending_size = 0; - for pending in pending_iterator { - let encoded_size = pending.data.encode().len(); - if pending_size + encoded_size >= MAX_TRANSACTIONS_SIZE { break } - - match block_builder.push_extrinsic(pending.data.clone()) { - Ok(()) => { - pending_size += encoded_size; - } - Err(e) => { - trace!(target: "transaction-pool", "Invalid transaction: {}", e); - unqueue_invalid.push(pending.hash.clone()); - } - } - } - }); - - self.transaction_pool.remove_invalid(&unqueue_invalid); - })?; - - info!("Proposing block [number: {}; hash: {}; parent_hash: {}; extrinsics: [{}]]", - block.header().number(), - <::Block as BlockT>::Hash::from(block.header().hash()), - block.header().parent_hash(), - block.extrinsics().iter() - .map(|xt| format!("{}", BlakeTwo256::hash_of(xt))) - .collect::>() - .join(", ") - ); - - let substrate_block = Decode::decode(&mut block.encode().as_slice()) - .expect("blocks are defined to serialize to substrate blocks correctly; qed"); - - assert!(evaluation::evaluate_initial( - &substrate_block, - &self.parent_hash, - self.parent_number, - ).is_ok()); - - Ok(substrate_block) - } - - fn evaluate(&self, unchecked_proposal: &::Block) -> Self::Evaluate { - debug!(target: "rhd", "evaluating block on top of parent ({}, {:?})", self.parent_number, self.parent_hash); - - // do initial serialization and structural integrity checks. - if let Err(e) = evaluation::evaluate_initial( - unchecked_proposal, - &self.parent_hash, - self.parent_number, - ) { - debug!(target: "rhd", "Invalid proposal: {:?}", e); - return Box::new(future::ok(false)); - }; - - let current_timestamp = current_timestamp(); - let inherent = InherentData::new( - current_timestamp, - self.offline.read().reports(&self.validators) - ); - let proposed_timestamp = match self.client.check_inherents( - &self.parent_id, - &unchecked_proposal, - &inherent, - ) { - Ok(Ok(())) => None, - Ok(Err(BlockBuilderError::ValidAtTimestamp(timestamp))) => Some(timestamp), - Ok(Err(e)) => { - debug!(target: "rhd", "Invalid proposal (check_inherents): {:?}", e); - return Box::new(future::ok(false)); - }, - Err(e) => { - debug!(target: "rhd", "Could not call into runtime: {:?}", e); - return Box::new(future::ok(false)); - } - }; - - let vote_delays = { - - // the duration until the given timestamp is current - let proposed_timestamp = ::std::cmp::max(self.minimum_timestamp, proposed_timestamp.unwrap_or(0)); - let timestamp_delay = if proposed_timestamp > current_timestamp { - let delay_s = proposed_timestamp - current_timestamp; - debug!(target: "rhd", "Delaying evaluation of proposal for {} seconds", delay_s); - Some(Instant::now() + Duration::from_secs(delay_s)) - } else { - None - }; - - match timestamp_delay { - Some(duration) => future::Either::A( - Delay::new(duration).map_err(|e| ErrorKind::Timer(e).into()) - ), - None => future::Either::B(future::ok(())), - } - }; - - // evaluate whether the block is actually valid. - // it may be better to delay this until the delays are finished - let evaluated = match self.client.execute_block(&self.parent_id, &unchecked_proposal.clone()) - .map_err(Error::from) { - Ok(()) => Ok(true), - Err(err) => match err.kind() { - error::ErrorKind::Client(client::error::ErrorKind::Execution(_)) => Ok(false), - _ => Err(err) - } - }; - - let future = future::result(evaluated).and_then(move |good| { - let end_result = future::ok(good); - if good { - // delay a "good" vote. - future::Either::A(vote_delays.and_then(|_| end_result)) - } else { - // don't delay a "bad" evaluation. - future::Either::B(end_result) - } - }); - - Box::new(future) as Box<_> - } -} - -impl LocalProposer<::Block> for Proposer where - C: AuthoringApi + BlockNumberToHash, - A: txpool::ChainApi::Block>, - Self: BaseProposer<::Block, Error=Error>, - <::Block as BlockT>::Hash: - Into<::Hash> + PartialEq + Into, - error::Error: From<::Error> -{ - - fn round_proposer(&self, round_number: u32, authorities: &[AuthorityId]) -> AuthorityId { - let offset = self.primary_index(round_number, authorities.len()); - let proposer = authorities[offset as usize].clone(); - trace!(target: "rhd", "proposer for round {} is {}", round_number, proposer); - - proposer - } - - fn import_misbehavior( - &self, - _misbehavior: Vec<(AuthorityId, Misbehavior<<::Block as BlockT>::Hash>)> - ) { - use rhododendron::Misbehavior as GenericMisbehavior; - use sr_primitives::bft::{MisbehaviorKind, MisbehaviorReport}; - use node_runtime::{Call, UncheckedExtrinsic, ConsensusCall}; - - let mut next_index = { - let local_id = self.local_key.public().0; - let cur_index = self.transaction_pool.cull_and_get_pending(&BlockId::hash(self.parent_hash), |pending| pending - .filter(|tx| tx.verified.sender == local_id) - .last() - .map(|tx| Ok(tx.verified.index())) - .unwrap_or_else(|| self.client.account_nonce(&self.parent_id, local_id)) - .map_err(Error::from) - ); - - match cur_index { - Ok(cur_index) => cur_index + 1, - Err(e) => { - warn!(target: "consensus", "Error computing next transaction index: {:?}", e); - return; - } - } - }; - - for (target, misbehavior) in misbehavior { - let report = MisbehaviorReport { - parent_hash: self.parent_hash.into(), - parent_number: self.parent_number.saturated_into::(), - target, - misbehavior: match misbehavior { - GenericMisbehavior::ProposeOutOfTurn(_, _, _) => continue, - GenericMisbehavior::DoublePropose(_, _, _) => continue, - GenericMisbehavior::DoublePrepare(round, (h1, s1), (h2, s2)) - => MisbehaviorKind::BftDoublePrepare(round as u32, (h1.into(), s1.signature), (h2.into(), s2.signature)), - GenericMisbehavior::DoubleCommit(round, (h1, s1), (h2, s2)) - => MisbehaviorKind::BftDoubleCommit(round as u32, (h1.into(), s1.signature), (h2.into(), s2.signature)), - } - }; - let payload = ( - next_index, - Call::Consensus(ConsensusCall::report_misbehavior(report)), - Era::immortal(), - self.client.genesis_hash() - ); - let signature = self.local_key.sign(&payload.encode()).into(); - next_index += 1; - - let local_id = self.local_key.public().0.into(); - let extrinsic = UncheckedExtrinsic { - signature: Some((node_runtime::RawAddress::Id(local_id), signature, payload.0, Era::immortal())), - function: payload.1, - }; - let uxt: <::Block as BlockT>::Extrinsic = Decode::decode( - &mut extrinsic.encode().as_slice()).expect("Encoded extrinsic is valid"); - let hash = BlockId::<::Block>::hash(self.parent_hash); - if let Err(e) = self.transaction_pool.submit_one(&hash, uxt) { - warn!("Error importing misbehavior report: {:?}", e); - } - } - } - - fn on_round_end(&self, round_number: u32, was_proposed: bool) { - let primary_validator = self.validators[ - self.primary_index(round_number, self.validators.len()) - ]; - - // alter the message based on whether we think the empty proposer was forced to skip the round. - // this is determined by checking if our local validator would have been forced to skip the round. - if !was_proposed { - let public = ed25519::Public::from_raw(primary_validator.0); - info!( - "Potential Offline Validator: {} failed to propose during assigned slot: {}", - public, - round_number, - ); - } - - self.offline.write().note_round_end(primary_validator, was_proposed); - } -} - -fn current_timestamp() -> u64 { - time::SystemTime::now().duration_since(time::UNIX_EPOCH) - .expect("now always later than unix epoch; qed") - .as_secs() -} - - -#[cfg(test)] -mod tests { - use super::*; - use std::collections::HashSet; - use std::marker::PhantomData; - - use sr_primitives::testing::{Block as GenericTestBlock, Header as TestHeader}; - use primitives::H256; - use keyring::Ed25519Keyring; - - type TestBlock = GenericTestBlock<()>; - - struct FakeClient { - authorities: Vec, - imported_heights: Mutex> - } - - impl BlockImport for FakeClient { - type Error = Error; - - fn import_block(&self, - block: BlockImportParams, - _new_authorities: Option> - ) -> Result { - assert!(self.imported_heights.lock().insert(block.header.number)); - Ok(ImportResult::Queued) - } - } - - impl Authorities for FakeClient { - type Error = Error; - - fn authorities(&self, _at: &BlockId) -> Result, Self::Error> { - Ok(self.authorities.clone()) - } - } - - // "black hole" output sink. - struct Comms(::std::marker::PhantomData); - - impl Sink for Comms { - type SinkItem = Communication; - type SinkError = E; - - fn start_send(&mut self, _item: Communication) -> ::futures::StartSend, E> { - Ok(::futures::AsyncSink::Ready) - } - - fn poll_complete(&mut self) -> ::futures::Poll<(), E> { - Ok(Async::Ready(())) - } - } - - impl Stream for Comms { - type Item = Communication; - type Error = E; - - fn poll(&mut self) -> ::futures::Poll, Self::Error> { - Ok(::futures::Async::NotReady) - } - } - - struct DummyFactory; - struct DummyProposer(u64); - - impl Environment for DummyFactory { - type Proposer = DummyProposer; - type Error = Error; - - fn init(&mut self, parent_header: &TestHeader, _authorities: &[AuthorityId], _sign_with: Arc) - -> Result - { - Ok(DummyProposer(parent_header.number + 1)) - } - } - - impl BaseProposer for DummyProposer { - type Error = Error; - type Create = Result; - type Evaluate = Result; - - fn propose(&self) -> Result { - - Ok(TestBlock { - header: from_block_number(self.0), - extrinsics: Default::default() - }) - } - - fn evaluate(&self, proposal: &TestBlock) -> Result { - Ok(proposal.header.number == self.0) - } - } - - impl LocalProposer for DummyProposer { - fn import_misbehavior(&self, _misbehavior: Vec<(AuthorityId, Misbehavior)>) {} - - fn round_proposer(&self, round_number: u32, authorities: &[AuthorityId]) -> AuthorityId { - authorities[(round_number as usize) % authorities.len()].clone() - } - } - - fn make_service(client: FakeClient) - -> BftService - { - BftService { - client: Arc::new(client), - live_agreement: Mutex::new(None), - round_cache: Arc::new(Mutex::new(RoundCache { - hash: None, - start_round: 0, - })), - round_timeout_multiplier: 10, - key: Arc::new(Ed25519Keyring::One.into()), - factory: DummyFactory - } - } - - fn sign_vote(vote: rhododendron::Vote, key: &ed25519::Pair, parent_hash: H256) -> LocalizedSignature { - match sign_message::(vote.into(), key, parent_hash) { - rhododendron::LocalizedMessage::Vote(vote) => vote.signature, - _ => panic!("signing vote leads to signed vote"), - } - } - - fn from_block_number(num: u64) -> TestHeader { - TestHeader::new( - num, - Default::default(), - Default::default(), - Default::default(), - Default::default(), - ) - } - - #[test] - fn future_gets_preempted() { - let client = FakeClient { - authorities: vec![ - Ed25519Keyring::One.into(), - Ed25519Keyring::Two.into(), - Ed25519Keyring::Alice.into(), - Ed25519Keyring::Eve.into(), - ], - imported_heights: Mutex::new(HashSet::new()), - }; - - let service = make_service(client); - - let first = from_block_number(2); - let first_hash = first.hash(); - - let mut second = from_block_number(3); - second.parent_hash = first_hash; - let _second_hash = second.hash(); - - let mut first_bft = service.build_upon(&first, Comms(PhantomData), Comms(PhantomData)).unwrap().unwrap(); - assert!(service.live_agreement.lock().as_ref().unwrap().0 == first); - - let _second_bft = service.build_upon(&second, Comms(PhantomData), Comms(PhantomData)).unwrap(); - assert!(service.live_agreement.lock().as_ref().unwrap().0 != first); - assert!(service.live_agreement.lock().as_ref().unwrap().0 == second); - - // first_bft has been cancelled. need to swap out so we can check it. - let (_tx, mut rx) = oneshot::channel(); - ::std::mem::swap(&mut rx, &mut first_bft.cancel); - - assert!(rx.wait().is_ok()); - } - - #[test] - fn max_faulty() { - assert_eq!(max_faulty_of(3), 0); - assert_eq!(max_faulty_of(4), 1); - assert_eq!(max_faulty_of(100), 33); - assert_eq!(max_faulty_of(0), 0); - assert_eq!(max_faulty_of(11), 3); - assert_eq!(max_faulty_of(99), 32); - } - - #[test] - fn justification_check_works() { - let parent_hash = Default::default(); - let hash = [0xff; 32].into(); - - let authorities = vec![ - Ed25519Keyring::One.into(), - Ed25519Keyring::Two.into(), - Ed25519Keyring::Alice.into(), - Ed25519Keyring::Eve.into(), - ]; - - let authorities_keys = vec![ - Ed25519Keyring::One.into(), - Ed25519Keyring::Two.into(), - Ed25519Keyring::Alice.into(), - Ed25519Keyring::Eve.into(), - ]; - - let unchecked = UncheckedJustification(rhododendron::UncheckedJustification { - digest: hash, - round_number: 1, - signatures: authorities_keys.iter().take(3).map(|key| { - sign_vote(rhododendron::Vote::Commit(1, hash).into(), key, parent_hash) - }).collect(), - }); - - assert!(check_justification::(&authorities, parent_hash, unchecked).is_ok()); - - let unchecked = UncheckedJustification(rhododendron::UncheckedJustification { - digest: hash, - round_number: 0, // wrong round number (vs. the signatures) - signatures: authorities_keys.iter().take(3).map(|key| { - sign_vote(rhododendron::Vote::Commit(1, hash).into(), key, parent_hash) - }).collect(), - }); - - assert!(check_justification::(&authorities, parent_hash, unchecked).is_err()); - - // not enough signatures. - let unchecked = UncheckedJustification(rhododendron::UncheckedJustification { - digest: hash, - round_number: 1, - signatures: authorities_keys.iter().take(2).map(|key| { - sign_vote(rhododendron::Vote::Commit(1, hash).into(), key, parent_hash) - }).collect(), - }); - - assert!(check_justification::(&authorities, parent_hash, unchecked).is_err()); - - // wrong hash. - let unchecked = UncheckedJustification(rhododendron::UncheckedJustification { - digest: [0xfe; 32].into(), - round_number: 1, - signatures: authorities_keys.iter().take(3).map(|key| { - sign_vote(rhododendron::Vote::Commit(1, hash).into(), key, parent_hash) - }).collect(), - }); - - assert!(check_justification::(&authorities, parent_hash, unchecked).is_err()); - } - - #[test] - fn propose_check_works() { - let parent_hash = Default::default(); - - let authorities = vec![ - Ed25519Keyring::Alice.into(), - Ed25519Keyring::Eve.into(), - ]; - - let block = TestBlock { - header: from_block_number(1), - extrinsics: Default::default() - }; - - let proposal = sign_message( - rhododendron::Message::Propose(1, block.clone()), - &Ed25519Keyring::Alice.pair(), - parent_hash, - ); - if let rhododendron::LocalizedMessage::Propose(proposal) = proposal { - assert!(check_proposal(&authorities, &parent_hash, &proposal).is_ok()); - let mut invalid_round = proposal.clone(); - invalid_round.round_number = 0; - assert!(check_proposal(&authorities, &parent_hash, &invalid_round).is_err()); - let mut invalid_digest = proposal.clone(); - invalid_digest.digest = [0xfe; 32].into(); - assert!(check_proposal(&authorities, &parent_hash, &invalid_digest).is_err()); - } else { - assert!(false); - } - - // Not an authority - let proposal = sign_message::( - rhododendron::Message::Propose(1, block), - &Ed25519Keyring::Bob.pair(), - parent_hash, - ); - if let rhododendron::LocalizedMessage::Propose(proposal) = proposal { - assert!(check_proposal(&authorities, &parent_hash, &proposal).is_err()); - } else { - assert!(false); - } - } - - #[test] - fn vote_check_works() { - let parent_hash: H256 = Default::default(); - let hash: H256 = [0xff; 32].into(); - - let authorities = vec![ - Ed25519Keyring::Alice.into(), - Ed25519Keyring::Eve.into(), - ]; - - let vote = sign_message::(rhododendron::Message::Vote(rhododendron::Vote::Prepare(1, hash)), &Keyring::Alice.pair(), parent_hash);; - if let rhododendron::LocalizedMessage::Vote(vote) = vote { - assert!(check_vote::(&authorities, &parent_hash, &vote).is_ok()); - let mut invalid_sender = vote.clone(); - invalid_sender.signature.signer = Keyring::Eve.into(); - assert!(check_vote::(&authorities, &parent_hash, &invalid_sender).is_err()); - } else { - assert!(false); - } - - // Not an authority - let vote = sign_message::(rhododendron::Message::Vote(rhododendron::Vote::Prepare(1, hash)), &Keyring::Bob.pair(), parent_hash);; - if let rhododendron::LocalizedMessage::Vote(vote) = vote { - assert!(check_vote::(&authorities, &parent_hash, &vote).is_err()); - } else { - assert!(false); - } - } - - #[test] - fn drop_bft_future_does_not_deadlock() { - let client = FakeClient { - authorities: vec![ - Ed25519Keyring::One.into(), - Ed25519Keyring::Two.into(), - Ed25519Keyring::Alice.into(), - Ed25519Keyring::Eve.into(), - ], - imported_heights: Mutex::new(HashSet::new()), - }; - - let service = make_service(client); - - let first = from_block_number(2); - let first_hash = first.hash(); - - let mut second = from_block_number(3); - second.parent_hash = first_hash; - - let _ = service.build_upon(&first, Comms(PhantomData), Comms(PhantomData)).unwrap(); - assert!(service.live_agreement.lock().as_ref().unwrap().0 == first); - service.live_agreement.lock().take(); - } - - #[test] - fn bft_can_build_though_skipped() { - let client = FakeClient { - authorities: vec![ - Ed25519Keyring::One.into(), - Ed25519Keyring::Two.into(), - Ed25519Keyring::Alice.into(), - Ed25519Keyring::Eve.into(), - ], - imported_heights: Mutex::new(HashSet::new()), - }; - - let service = make_service(client); - - let first = from_block_number(2); - let first_hash = first.hash(); - - let mut second = from_block_number(3); - second.parent_hash = first_hash; - - let mut third = from_block_number(4); - third.parent_hash = second.hash(); - - let _ = service.build_upon(&first, Comms(PhantomData), Comms(PhantomData)).unwrap(); - assert!(service.live_agreement.lock().as_ref().unwrap().0 == first); - // BFT has not seen second, but will move forward on third - service.build_upon(&third, Comms(PhantomData), Comms(PhantomData)).unwrap(); - assert!(service.live_agreement.lock().as_ref().unwrap().0 == third); - - // but we are not walking backwards - service.build_upon(&second, Comms(PhantomData), Comms(PhantomData)).unwrap(); - assert!(service.live_agreement.lock().as_ref().unwrap().0 == third); - } -} diff --git a/core/consensus/rhd/src/misbehaviour_check.rs b/core/consensus/rhd/src/misbehaviour_check.rs deleted file mode 100644 index bde3976c19b09..0000000000000 --- a/core/consensus/rhd/src/misbehaviour_check.rs +++ /dev/null @@ -1,191 +0,0 @@ -// Copyright 2017-2019 Parity Technologies (UK) Ltd. -// This file is part of Substrate. - -// Substrate is free software: you can redistribute it and/or modify -// it under the terms of the GNU General Public License as published by -// the Free Software Foundation, either version 3 of the License, or -// (at your option) any later version. - -// Substrate is distributed in the hope that it will be useful, -// but WITHOUT ANY WARRANTY; without even the implied warranty of -// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -// GNU General Public License for more details. - -// You should have received a copy of the GNU General Public License -// along with Substrate. If not, see . - -//! Utility for substrate-based runtimes that want to check misbehavior reports. - -use codec::{Codec, Encode}; -use primitives::{AuthorityId, Signature}; - -use rhododendron::messages::{Action, Message, MisbehaviorKind}; -use runtime_io; - -// check a message signature. returns true if signed by that authority. -fn check_message_sig( - message: Message, - signature: &Signature, - from: &AuthorityId, -) -> bool { - let msg: Vec = message.encode(); - runtime_io::ed25519_verify(&signature.0, &msg, from) -} - -fn prepare(parent: H, round_number: u32, hash: H) -> Message { - Message { - parent, - action: Action::Prepare(round_number, hash), - } -} - -fn commit(parent: H, round_number: u32, hash: H) -> Message { - Message { - parent, - action: Action::Commit(round_number, hash), - } -} - -/// Evaluate misbehavior. -/// -/// Doesn't check that the header hash in question is -/// valid or whether the misbehaving authority was part of -/// the set at that block. -pub fn evaluate_misbehavior( - misbehaved: &AuthorityId, - parent_hash: H, - kind: &MisbehaviorKind, -) -> bool { - match *kind { - MisbehaviorKind::BftDoublePrepare(round, (h_1, ref s_1), (h_2, ref s_2)) => { - s_1 != s_2 && - check_message_sig::(prepare::(parent_hash, round, h_1), s_1, misbehaved) && - check_message_sig::(prepare::(parent_hash, round, h_2), s_2, misbehaved) - } - MisbehaviorKind::BftDoubleCommit(round, (h_1, ref s_1), (h_2, ref s_2)) => { - s_1 != s_2 && - check_message_sig::(commit::(parent_hash, round, h_1), s_1, misbehaved) && - check_message_sig::(commit::(parent_hash, round, h_2), s_2, misbehaved) - } - } -} - -#[cfg(test)] -mod tests { - use super::*; - - use keyring::Ed25519Keyring; - use rhododendron; - - use sr_primitives::testing::{H256, Block as RawBlock}; - - type Block = RawBlock; - - fn sign_prepare(key: &ed25519::Pair, round: u32, hash: H256, parent_hash: H256) -> (H256, Signature) { - let msg = ::sign_message::( - rhododendron::Message::Vote(rhododendron::Vote::Prepare(round as _, hash)), - key, - parent_hash - ); - - match msg { - rhododendron::LocalizedMessage::Vote(vote) => (hash, vote.signature.signature), - _ => panic!("signing vote leads to signed vote"), - } - } - - fn sign_commit(key: &ed25519::Pair, round: u32, hash: H256, parent_hash: H256) -> (H256, Signature) { - let msg = ::sign_message::( - rhododendron::Message::Vote(rhododendron::Vote::Commit(round as _, hash)), - key, - parent_hash - ); - - match msg { - rhododendron::LocalizedMessage::Vote(vote) => (hash, vote.signature.signature), - _ => panic!("signing vote leads to signed vote"), - } - } - - #[test] - fn evaluates_double_prepare() { - let key = Ed25519Keyring::One.pair(); - let parent_hash = [0xff; 32].into(); - let hash_1 = [0; 32].into(); - let hash_2 = [1; 32].into(); - - assert!(evaluate_misbehavior::( - &key.public().into(), - parent_hash, - &MisbehaviorKind::BftDoublePrepare( - 1, - sign_prepare(&key, 1, hash_1, parent_hash), - sign_prepare(&key, 1, hash_2, parent_hash) - ) - )); - - // same signature twice is not misbehavior. - let signed = sign_prepare(&key, 1, hash_1, parent_hash); - assert!(!evaluate_misbehavior::( - &key.public().into(), - parent_hash, - &MisbehaviorKind::BftDoublePrepare( - 1, - signed, - signed, - ) - )); - - // misbehavior has wrong target. - assert!(!evaluate_misbehavior::( - &Ed25519Keyring::Two.into(), - parent_hash, - &MisbehaviorKind::BftDoublePrepare( - 1, - sign_prepare(&key, 1, hash_1, parent_hash), - sign_prepare(&key, 1, hash_2, parent_hash), - ) - )); - } - - #[test] - fn evaluates_double_commit() { - let key = Ed25519Keyring::One.pair(); - let parent_hash = [0xff; 32].into(); - let hash_1 = [0; 32].into(); - let hash_2 = [1; 32].into(); - - assert!(evaluate_misbehavior::( - &key.public().into(), - parent_hash, - &MisbehaviorKind::BftDoubleCommit( - 1, - sign_commit(&key, 1, hash_1, parent_hash), - sign_commit(&key, 1, hash_2, parent_hash) - ) - )); - - // same signature twice is not misbehavior. - let signed = sign_commit(&key, 1, hash_1, parent_hash); - assert!(!evaluate_misbehavior::( - &key.public().into(), - parent_hash, - &MisbehaviorKind::BftDoubleCommit( - 1, - signed, - signed, - ) - )); - - // misbehavior has wrong target. - assert!(!evaluate_misbehavior::( - &Ed25519Keyring::Two.into(), - parent_hash, - &MisbehaviorKind::BftDoubleCommit( - 1, - sign_commit(&key, 1, hash_1, parent_hash), - sign_commit(&key, 1, hash_2, parent_hash), - ) - )); - } -} diff --git a/core/consensus/rhd/src/service.rs b/core/consensus/rhd/src/service.rs deleted file mode 100644 index 641a97fe06e9b..0000000000000 --- a/core/consensus/rhd/src/service.rs +++ /dev/null @@ -1,181 +0,0 @@ -// Copyright 2018-2019 Parity Technologies (UK) Ltd. -// This file is part of Substrate. - -// Substrate is free software: you can redistribute it and/or modify -// it under the terms of the GNU General Public License as published by -// the Free Software Foundation, either version 3 of the License, or -// (at your option) any later version. - -// Substrate is distributed in the hope that it will be useful, -// but WITHOUT ANY WARRANTY; without even the implied warranty of -// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -// GNU General Public License for more details. - -// You should have received a copy of the GNU General Public License -// along with Substrate. If not, see . - -//! Consensus service. - -/// Consensus service. A long running service that manages BFT agreement -/// the network. -use std::thread; -use std::time::{Duration, Instant}; -use std::sync::Arc; - -use client::{BlockchainEvents, BlockBody}; -use futures::prelude::*; -use transaction_pool::txpool::{Pool as TransactionPool, ChainApi as PoolChainApi}; -use sr_primitives::traits::{Block as BlockT, Header as HeaderT, BlockNumberToHash}; - -use tokio::executor::current_thread::TaskExecutor as LocalThreadHandle; -use tokio::runtime::TaskExecutor as ThreadPoolHandle; -use tokio::runtime::current_thread::Runtime as LocalRuntime; -use tokio::timer::Interval; - -use parking_lot::RwLock; -use consensus::{self, offline_tracker::OfflineTracker}; - -use super::{Network, ProposerFactory, AuthoringApi}; -use {consensus, primitives, ed25519, error, BftService, LocalProposer}; - -const TIMER_DELAY_MS: u64 = 5000; -const TIMER_INTERVAL_MS: u64 = 500; - -// spin up an instance of BFT agreement on the current thread's executor. -// panics if there is no current thread executor. -fn start_bft( - header: ::Header, - bft_service: Arc>, -) where - F: consensus::Environment + 'static, - C: consensus::BlockImport + consensus::Authorities + 'static, - F::Error: ::std::fmt::Debug, - >::Error: ::std::fmt::Display + Into, - >::Proposer : LocalProposer, - >::Error: ::std::fmt::Display, - Block: BlockT, -{ - let mut handle = LocalThreadHandle::current(); - match bft_service.build_upon(&header) { - Ok(Some(bft_work)) => if let Err(e) = handle.spawn_local(Box::new(bft_work)) { - warn!(target: "bft", "Couldn't initialize BFT agreement: {:?}", e); - } - Ok(None) => trace!(target: "bft", "Could not start agreement on top of {}", header.hash()), - Err(e) => warn!(target: "bft", "BFT agreement error: {}", e), - } -} - -/// Consensus service. Starts working when created. -pub struct Service { - thread: Option>, - exit_signal: Option<::exit_future::Signal>, -} - -impl Service { - /// Create and start a new instance. - pub fn new( - client: Arc, - api: Arc, - network: N, - transaction_pool: Arc>, - thread_pool: ThreadPoolHandle, - key: ed25519::Pair, - block_delay: u64, - ) -> Service - where - error::Error: From<::Error>, - A: AuthoringApi + BlockNumberToHash + 'static, - P: PoolChainApi::Block> + 'static, - C: BlockchainEvents<::Block> - + BlockBody<::Block> - + consensus::SelectChain<::Block> - + consensus::BlockImport<::Block> - + consensus::Authorities<::Block> + Send + Sync + 'static, - primitives::H256: From<<::Block as BlockT>::Hash>, - <::Block as BlockT>::Hash: PartialEq + PartialEq, - N: Network::Block> + Send + 'static, - { - - let (signal, exit) = ::exit_future::signal(); - let thread = thread::spawn(move || { - let mut runtime = LocalRuntime::new().expect("Could not create local runtime"); - let key = Arc::new(key); - - let factory = ProposerFactory { - client: api.clone(), - transaction_pool: transaction_pool.clone(), - network, - handle: thread_pool.clone(), - offline: Arc::new(RwLock::new(OfflineTracker::new())), - force_delay: block_delay, - }; - let bft_service = Arc::new(BftService::new(client.clone(), key, factory)); - - let notifications = { - let client = client.clone(); - let bft_service = bft_service.clone(); - - client.import_notification_stream().for_each(move |notification| { - if notification.is_new_best { - start_bft(notification.header, bft_service.clone()); - } - Ok(()) - }) - }; - - let interval = Interval::new( - Instant::now() + Duration::from_millis(TIMER_DELAY_MS), - Duration::from_millis(TIMER_INTERVAL_MS), - ); - - let mut prev_best = match client.best_block_header() { - Ok(header) => header.hash(), - Err(e) => { - warn!("Cant's start consensus service. Error reading best block header: {:?}", e); - return; - } - }; - - let timed = { - let c = client.clone(); - let s = bft_service.clone(); - - interval.map_err(|e| debug!(target: "bft", "Timer error: {:?}", e)).for_each(move |_| { - if let Ok(best_block) = c.best_block_header() { - let hash = best_block.hash(); - - if hash == prev_best { - debug!(target: "bft", "Starting consensus round after a timeout"); - start_bft(best_block, s.clone()); - } - prev_best = hash; - } - Ok(()) - }) - }; - - runtime.spawn(notifications); - runtime.spawn(timed); - - if let Err(e) = runtime.block_on(exit) { - debug!("BFT event loop error {:?}", e); - } - }); - Service { - thread: Some(thread), - exit_signal: Some(signal), - } - } -} - -impl Drop for Service { - fn drop(&mut self) { - if let Some(signal) = self.exit_signal.take() { - signal.fire(); - } - - if let Some(thread) = self.thread.take() { - thread.join().expect("The service thread has panicked"); - } - } -} diff --git a/core/consensus/uncles/Cargo.toml b/core/consensus/uncles/Cargo.toml deleted file mode 100644 index 4820324870929..0000000000000 --- a/core/consensus/uncles/Cargo.toml +++ /dev/null @@ -1,15 +0,0 @@ -[package] -name = "substrate-consensus-uncles" -version = "2.0.0" -authors = ["Parity Technologies "] -description = "Generic uncle inclusion utilities for consensus" -edition = "2018" - -[dependencies] -client = { package = "substrate-client", path = "../../client" } -primitives = { package = "substrate-primitives", path = "../../primitives" } -sr-primitives = { path = "../../sr-primitives" } -srml-authorship = { path = "../../../srml/authorship" } -consensus_common = { package = "substrate-consensus-common", path = "../common" } -inherents = { package = "substrate-inherents", path = "../../inherents" } -log = "0.4.8" diff --git a/core/executor/Cargo.toml b/core/executor/Cargo.toml deleted file mode 100644 index cf3ea4ff7a6e7..0000000000000 --- a/core/executor/Cargo.toml +++ /dev/null @@ -1,37 +0,0 @@ -[package] -name = "substrate-executor" -version = "2.0.0" -authors = ["Parity Technologies "] -edition = "2018" - -[dependencies] -derive_more = "0.15.0" -codec = { package = "parity-scale-codec", version = "1.0.0" } -runtime_io = { package = "sr-io", path = "../sr-io" } -primitives = { package = "substrate-primitives", path = "../primitives" } -trie = { package = "substrate-trie", path = "../trie" } -serializer = { package = "substrate-serializer", path = "../serializer" } -runtime_version = { package = "sr-version", path = "../sr-version" } -panic-handler = { package = "substrate-panic-handler", path = "../panic-handler" } -wasmi = "0.5.1" -parity-wasm = "0.40.3" -lazy_static = "1.4.0" -wasm-interface = { package = "substrate-wasm-interface", path = "../wasm-interface" } -externalities = { package = "substrate-externalities", path = "../externalities" } -parking_lot = "0.9.0" -log = "0.4.8" -libsecp256k1 = "0.3.0" -tiny-keccak = "1.5.0" - -[dev-dependencies] -assert_matches = "1.3.0" -wabt = "0.9.2" -hex-literal = "0.2.1" -runtime-test = { package = "substrate-runtime-test", path = "runtime-test" } -substrate-client = { path = "../client" } -substrate-offchain = { path = "../offchain/" } -state_machine = { package = "substrate-state-machine", path = "../state-machine" } - -[features] -default = [] -wasm-extern-trace = [] diff --git a/core/executor/runtime-test/Cargo.toml b/core/executor/runtime-test/Cargo.toml deleted file mode 100644 index 72041218bf9f2..0000000000000 --- a/core/executor/runtime-test/Cargo.toml +++ /dev/null @@ -1,20 +0,0 @@ -[package] -name = "substrate-runtime-test" -version = "2.0.0" -authors = ["Parity Technologies "] -edition = "2018" -build = "build.rs" - -[dependencies] -rstd = { package = "sr-std", path = "../../sr-std", default-features = false } -runtime_io = { package = "sr-io", path = "../../sr-io", default-features = false } -sandbox = { package = "sr-sandbox", path = "../../sr-sandbox", default-features = false } -primitives = { package = "substrate-primitives", path = "../../primitives", default-features = false } -sr-primitives = { package = "sr-primitives", path = "../../sr-primitives", default-features = false } - -[build-dependencies] -wasm-builder-runner = { package = "substrate-wasm-builder-runner", version = "1.0.4", path = "../../utils/wasm-builder-runner" } - -[features] -default = [ "std" ] -std = ["runtime_io/std", "sandbox/std", "rstd/std"] diff --git a/core/finality-grandpa/src/communication/periodic.rs b/core/finality-grandpa/src/communication/periodic.rs deleted file mode 100644 index 81c18891d03b5..0000000000000 --- a/core/finality-grandpa/src/communication/periodic.rs +++ /dev/null @@ -1,252 +0,0 @@ -// Copyright 2019 Parity Technologies (UK) Ltd. -// This file is part of Substrate. - -// Substrate is free software: you can redistribute it and/or modify -// it under the terms of the GNU General Public License as published by -// the Free Software Foundation, either version 3 of the License, or -// (at your option) any later version. - -// Substrate is distributed in the hope that it will be useful, -// but WITHOUT ANY WARRANTY; without even the implied warranty of -// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -// GNU General Public License for more details. - -// You should have received a copy of the GNU General Public License -// along with Substrate. If not, see . - -//! Periodic rebroadcast of neighbor packets. - -use std::collections::VecDeque; -use std::time::{Instant, Duration}; - -use codec::Encode; -use futures::prelude::*; -use futures::sync::mpsc; -use log::{debug, warn}; -use tokio_timer::Delay; - -use network::PeerId; -use sr_primitives::traits::{NumberFor, Block as BlockT}; -use super::{gossip::{NeighborPacket, GossipMessage}, Network}; - -// how often to rebroadcast, if no other -const REBROADCAST_AFTER: Duration = Duration::from_secs(2 * 60); - -/// The number of block hashes that we have previously voted on that we should -/// keep around for announcement. The current value should be enough for 3 -/// rounds assuming we have prevoted and precommited on different blocks. -const LATEST_VOTED_BLOCKS_TO_ANNOUNCE: usize = 6; - -fn rebroadcast_instant() -> Instant { - Instant::now() + REBROADCAST_AFTER -} - -/// A sender used to send neighbor packets to a background job. -#[derive(Clone)] -pub(super) struct NeighborPacketSender( - mpsc::UnboundedSender<(Vec, NeighborPacket>)> -); - -impl NeighborPacketSender { - /// Send a neighbor packet for the background worker to gossip to peers. - pub fn send( - &self, - who: Vec, - neighbor_packet: NeighborPacket>, - ) { - if let Err(err) = self.0.unbounded_send((who, neighbor_packet)) { - debug!(target: "afg", "Failed to send neighbor packet: {:?}", err); - } - } -} - -/// Does the work of sending neighbor packets, asynchronously. -/// -/// It may rebroadcast the last neighbor packet periodically when no -/// progress is made. -pub(super) fn neighbor_packet_worker(net: N) -> ( - impl Future + Send + 'static, - NeighborPacketSender, -) where - B: BlockT, - N: Network, -{ - let mut last = None; - let (tx, mut rx) = mpsc::unbounded::<(Vec, NeighborPacket>)>(); - let mut delay = Delay::new(rebroadcast_instant()); - - let work = futures::future::poll_fn(move || { - loop { - match rx.poll().expect("unbounded receivers do not error; qed") { - Async::Ready(None) => return Ok(Async::Ready(())), - Async::Ready(Some((to, packet))) => { - // send to peers. - net.send_message(to.clone(), GossipMessage::::from(packet.clone()).encode()); - - // rebroadcasting network. - delay.reset(rebroadcast_instant()); - last = Some((to, packet)); - } - Async::NotReady => break, - } - } - - // has to be done in a loop because it needs to be polled after - // re-scheduling. - loop { - match delay.poll() { - Err(e) => { - warn!(target: "afg", "Could not rebroadcast neighbor packets: {:?}", e); - delay.reset(rebroadcast_instant()); - } - Ok(Async::Ready(())) => { - delay.reset(rebroadcast_instant()); - - if let Some((ref to, ref packet)) = last { - // send to peers. - net.send_message(to.clone(), GossipMessage::::from(packet.clone()).encode()); - } - } - Ok(Async::NotReady) => return Ok(Async::NotReady), - } - } - }); - - (work, NeighborPacketSender(tx)) -} - -/// A background worker for performing block announcements. -struct BlockAnnouncer { - net: N, - block_rx: mpsc::UnboundedReceiver<(B::Hash, Vec)>, - latest_voted_blocks: VecDeque, - reannounce_after: Duration, - delay: Delay, -} - -/// A background worker for announcing block hashes to peers. The worker keeps -/// track of `LATEST_VOTED_BLOCKS_TO_ANNOUNCE` and periodically announces these -/// blocks to all peers if no new blocks to announce are noted (i.e. presumably -/// GRANDPA progress is stalled). -pub(super) fn block_announce_worker>(net: N) -> ( - impl Future, - BlockAnnounceSender, -) { - block_announce_worker_aux(net, REBROADCAST_AFTER) -} - -#[cfg(test)] -pub(super) fn block_announce_worker_with_delay>( - net: N, - reannounce_after: Duration, -) -> ( - impl Future, - BlockAnnounceSender, -) { - block_announce_worker_aux(net, reannounce_after) -} - -fn block_announce_worker_aux>( - net: N, - reannounce_after: Duration, -) -> ( - impl Future, - BlockAnnounceSender, -) { - let latest_voted_blocks = VecDeque::with_capacity(LATEST_VOTED_BLOCKS_TO_ANNOUNCE); - - let (block_tx, block_rx) = mpsc::unbounded(); - - let announcer = BlockAnnouncer { - net, - block_rx, - latest_voted_blocks, - reannounce_after, - delay: Delay::new(Instant::now() + reannounce_after), - }; - - (announcer, BlockAnnounceSender(block_tx)) -} - - -impl BlockAnnouncer { - fn note_block(&mut self, block: B::Hash) -> bool { - if !self.latest_voted_blocks.contains(&block) { - if self.latest_voted_blocks.len() >= LATEST_VOTED_BLOCKS_TO_ANNOUNCE { - self.latest_voted_blocks.pop_front(); - } - - self.latest_voted_blocks.push_back(block); - - true - } else { - false - } - } - - fn reset_delay(&mut self) { - self.delay.reset(Instant::now() + self.reannounce_after); - } -} - -impl> Future for BlockAnnouncer { - type Item = (); - type Error = (); - - fn poll(&mut self) -> Poll { - // note any new blocks to announce and announce them - loop { - match self.block_rx.poll().expect("unbounded receivers do not error; qed") { - Async::Ready(None) => return Ok(Async::Ready(())), - Async::Ready(Some(block)) => { - if self.note_block(block.0) { - self.net.announce(block.0, block.1); - self.reset_delay(); - } - }, - Async::NotReady => break, - } - } - - // check the reannouncement delay timer, has to be done in a loop - // because it needs to be polled after re-scheduling. - loop { - match self.delay.poll() { - Err(e) => { - warn!(target: "afg", "Error in periodic block announcer timer: {:?}", e); - self.reset_delay(); - }, - // after the delay fires announce all blocks that we have - // stored. note that this only happens if we don't receive any - // new blocks above for the duration of `reannounce_after`. - Ok(Async::Ready(())) => { - self.reset_delay(); - - debug!( - target: "afg", - "Re-announcing latest voted blocks due to lack of progress: {:?}", - self.latest_voted_blocks, - ); - - for block in self.latest_voted_blocks.iter() { - self.net.announce(*block, Vec::new()); - } - }, - Ok(Async::NotReady) => return Ok(Async::NotReady), - } - } - } -} - -/// A sender used to send block hashes to announce to a background job. -#[derive(Clone)] -pub(super) struct BlockAnnounceSender(mpsc::UnboundedSender<(B::Hash, Vec)>); - -impl BlockAnnounceSender { - /// Send a block hash for the background worker to announce. - pub fn send(&self, block: B::Hash, associated_data: Vec) { - if let Err(err) = self.0.unbounded_send((block, associated_data)) { - debug!(target: "afg", "Failed to send block to background announcer: {:?}", err); - } - } -} diff --git a/core/prometheus/Cargo.toml b/core/prometheus/Cargo.toml new file mode 100644 index 0000000000000..bc3e7fd41d0d4 --- /dev/null +++ b/core/prometheus/Cargo.toml @@ -0,0 +1,16 @@ +[package] +name = "substrate-prometheus" +version = "2.0.0" +authors = ["Parity Technologies "] +description = "prometheus utils" +edition = "2018" + +[dependencies] +hyper = "0.12" +lazy_static = "1.0" +log = "0.4" +prometheus = { version = "0.7", features = ["nightly", "process"]} +tokio = "0.1" + +[dev-dependencies] +reqwest = "0.9" diff --git a/core/prometheus/src/lib.rs b/core/prometheus/src/lib.rs new file mode 100644 index 0000000000000..a36724ba15ac6 --- /dev/null +++ b/core/prometheus/src/lib.rs @@ -0,0 +1,62 @@ + +#[macro_use] +extern crate lazy_static; +#[macro_use] +extern crate log; +use hyper::http::StatusCode; +use hyper::rt::Future; +use hyper::service::service_fn_ok; +use hyper::{Body, Request, Response, Server}; +use std::{net::{ SocketAddr}}; +pub use prometheus::{Histogram, IntCounter, IntGauge,Opts,Encoder, TextEncoder,Result}; + +pub mod metrics; + +/// Initializes the metrics context, and starts an HTTP server +/// to serve metrics. +pub fn init_prometheus(prometheus_addr: SocketAddr) { + //let addr = &std::net::SocketAddr::V4; + //let addr = ([127, 0, 0, 1], 9898).into(); + let addr = prometheus_addr; + //let parsed_addr = addr.parse().unwrap(); + //prometheus::register_int_counter!("meh", "foo"); + let server = Server::bind(&addr) + .serve(|| { + // This is the `Service` that will handle the connection. + // `service_fn_ok` is a helper to convert a function that + // returns a Response into a `Service`. + service_fn_ok(move |req: Request| { + + if req.uri().path() == "/metrics" { + let metric_families = prometheus::gather(); + let mut buffer = vec![]; + let encoder = TextEncoder::new(); + encoder.encode(&metric_families, &mut buffer).unwrap(); + + Response::builder() + .status(StatusCode::OK) + .header("Content-Type", encoder.format_type()) + .body(Body::from(buffer)) + .expect("Error constructing response") + } else { + Response::builder() + .status(StatusCode::NOT_FOUND) + .body(Body::from("Not found.")) + .expect("Error constructing response") + } + }) + }) + .map_err(|e| error!("server error: {}", e)); + + info!("Exporting metrics at http://{}/metrics", addr); + + let mut rt = tokio::runtime::Builder::new() + .core_threads(1) // one thread is sufficient + .build() + .expect("Unable to build metrics exporter tokio runtime"); + + std::thread::spawn(move || { + rt.spawn(server); + rt.shutdown_on_idle().wait().unwrap(); + }); +} \ No newline at end of file diff --git a/core/prometheus/src/metrics.rs b/core/prometheus/src/metrics.rs new file mode 100644 index 0000000000000..e8779c4caecad --- /dev/null +++ b/core/prometheus/src/metrics.rs @@ -0,0 +1,25 @@ +pub use prometheus::*; + +pub fn try_create_int_gauge(name: &str, help: &str) -> Result { + let opts = Opts::new(name, help); + let gauge = IntGauge::with_opts(opts)?; + prometheus::register(Box::new(gauge.clone()))?; + Ok(gauge) +} + +pub fn set_gauge(gauge: &Result, value: u64) { + if let Ok(gauge) = gauge { + gauge.set(value as i64); + } +} + +lazy_static! { + pub static ref FINALITY_HEIGHT: Result = try_create_int_gauge( + "finality_block_height_number", + "block is finality HEIGHT" + ); + pub static ref BEST_HEIGHT: Result = try_create_int_gauge( + "best_block_height_number", + "block is best HEIGHT" + ); +} diff --git a/core/rpc-servers/Cargo.toml b/core/rpc-servers/Cargo.toml deleted file mode 100644 index 80e16bc5ae5be..0000000000000 --- a/core/rpc-servers/Cargo.toml +++ /dev/null @@ -1,17 +0,0 @@ -[package] -name = "substrate-rpc-servers" -version = "2.0.0" -authors = ["Parity Technologies "] -edition = "2018" - -[dependencies] -jsonrpc-core = "13.2.0" -pubsub = { package = "jsonrpc-pubsub", version = "13.2.0" } -log = "0.4.8" -serde = "1.0.101" -serde_json = "1.0.41" -sr-primitives = { path = "../sr-primitives" } - -[target.'cfg(not(target_os = "unknown"))'.dependencies] -http = { package = "jsonrpc-http-server", version = "13.2.0" } -ws = { package = "jsonrpc-ws-server", version = "13.2.0" } diff --git a/core/service/Cargo.toml b/core/service/Cargo.toml deleted file mode 100644 index 0d6f27936244f..0000000000000 --- a/core/service/Cargo.toml +++ /dev/null @@ -1,50 +0,0 @@ -[package] -name = "substrate-service" -version = "2.0.0" -authors = ["Parity Technologies "] -edition = "2018" - -[dependencies] -derive_more = "0.15.0" -futures = "0.1.29" -futures03 = { package = "futures-preview", version = "0.3.0-alpha.19", features = ["compat"] } -parking_lot = "0.9.0" -lazy_static = "1.4.0" -log = "0.4.8" -slog = { version = "2.5.2", features = ["nested-values"] } -tokio-executor = "0.1.8" -tokio-timer = "0.2.11" -exit-future = "0.1.4" -serde = "1.0.101" -serde_json = "1.0.41" -sysinfo = "0.9.5" -target_info = "0.1.0" -keystore = { package = "substrate-keystore", path = "../../core/keystore" } -sr-io = { path = "../../core/sr-io" } -sr-primitives = { path = "../../core/sr-primitives" } -primitives = { package = "substrate-primitives", path = "../primitives" } -session = { package = "substrate-session", path = "../session" } -app-crypto = { package = "substrate-application-crypto", path = "../application-crypto" } -consensus_common = { package = "substrate-consensus-common", path = "../../core/consensus/common" } -network = { package = "substrate-network", path = "../../core/network" } -chain-spec = { package = "substrate-chain-spec", path = "../chain-spec" } -client = { package = "substrate-client", path = "../../core/client" } -client_db = { package = "substrate-client-db", path = "../../core/client/db", features = ["kvdb-rocksdb"] } -codec = { package = "parity-scale-codec", version = "1.0.0" } -substrate-executor = { path = "../../core/executor" } -transaction_pool = { package = "substrate-transaction-pool", path = "../../core/transaction-pool" } -rpc-servers = { package = "substrate-rpc-servers", path = "../../core/rpc-servers" } -rpc = { package = "substrate-rpc", path = "../../core/rpc" } -tel = { package = "substrate-telemetry", path = "../../core/telemetry" } -offchain = { package = "substrate-offchain", path = "../../core/offchain" } -parity-multiaddr = { package = "parity-multiaddr", version = "0.5.0" } - -[dev-dependencies] -substrate-test-runtime-client = { path = "../test-runtime/client" } -node-executor = { path = "../../node/executor" } -node-primitives = { path = "../../node/primitives" } -node-runtime = { path = "../../node/runtime" } -babe-primitives = { package = "substrate-consensus-babe-primitives", path = "../../core/consensus/babe/primitives" } -grandpa = { package = "substrate-finality-grandpa", path = "../../core/finality-grandpa" } -grandpa-primitives = { package = "substrate-finality-grandpa-primitives", path = "../../core/finality-grandpa/primitives" } -tokio = "0.1" diff --git a/core/sr-api-macros/Cargo.toml b/core/sr-api-macros/Cargo.toml deleted file mode 100644 index 022536136b80e..0000000000000 --- a/core/sr-api-macros/Cargo.toml +++ /dev/null @@ -1,37 +0,0 @@ -[package] -name = "sr-api-macros" -version = "2.0.0" -authors = ["Parity Technologies "] -edition = "2018" - -[lib] -proc-macro = true - -[dependencies] -quote = "0.6.12" -syn = { version = "0.15.44", features = [ "full", "fold", "extra-traits", "visit" ] } -proc-macro2 = "0.4.27" -blake2-rfc = "0.2.18" -proc-macro-crate = "0.1.4" - -[dev-dependencies] -client = { package = "substrate-client", path = "../client" } -test-client = { package = "substrate-test-runtime-client", path = "../test-runtime/client" } -state_machine = { package = "substrate-state-machine", path = "../state-machine" } -sr-primitives = { path = "../sr-primitives" } -sr-version = { path = "../sr-version" } -primitives = { package = "substrate-primitives", path = "../primitives" } -criterion = "0.3.0" -consensus_common = { package = "substrate-consensus-common", path = "../consensus/common" } -codec = { package = "parity-scale-codec", version = "1.0.0" } -trybuild = "1.0.17" -rustversion = "1.0.0" - -[[bench]] -name = "bench" -harness = false - -# We actually don't need the `std` feature in this crate, but the tests require it. -[features] -default = [ "std" ] -std = [] diff --git a/core/sr-api-macros/tests/trybuild.rs b/core/sr-api-macros/tests/trybuild.rs deleted file mode 100644 index 9baea83196e9b..0000000000000 --- a/core/sr-api-macros/tests/trybuild.rs +++ /dev/null @@ -1,11 +0,0 @@ -use std::env; - -#[rustversion::attr(not(stable), ignore)] -#[test] -fn ui() { - // As trybuild is using `cargo check`, we don't need the real WASM binaries. - env::set_var("BUILD_DUMMY_WASM_BINARY", "1"); - - let t = trybuild::TestCases::new(); - t.compile_fail("tests/ui/*.rs"); -} diff --git a/core/sr-api-macros/tests/ui/adding_at_parameter.rs b/core/sr-api-macros/tests/ui/adding_at_parameter.rs deleted file mode 100644 index d4757e256f024..0000000000000 --- a/core/sr-api-macros/tests/ui/adding_at_parameter.rs +++ /dev/null @@ -1,9 +0,0 @@ -use client::decl_runtime_apis; - -decl_runtime_apis! { - pub trait Api { - fn test(at: u64); - } -} - -fn main() {} diff --git a/core/sr-api-macros/tests/ui/adding_at_parameter.stderr b/core/sr-api-macros/tests/ui/adding_at_parameter.stderr deleted file mode 100644 index 1c7e07a418c0a..0000000000000 --- a/core/sr-api-macros/tests/ui/adding_at_parameter.stderr +++ /dev/null @@ -1,5 +0,0 @@ -error: `decl_runtime_apis!` adds automatically a parameter `at: &BlockId`. Please rename/remove your parameter. - --> $DIR/adding_at_parameter.rs:5:11 - | -5 | fn test(at: u64); - | ^^ diff --git a/core/sr-api-macros/tests/ui/adding_self_parameter.stderr b/core/sr-api-macros/tests/ui/adding_self_parameter.stderr deleted file mode 100644 index e7249e9f732f5..0000000000000 --- a/core/sr-api-macros/tests/ui/adding_self_parameter.stderr +++ /dev/null @@ -1,5 +0,0 @@ -error: Self values are not supported. - --> $DIR/adding_self_parameter.rs:5:11 - | -5 | fn test(&self); - | ^ diff --git a/core/sr-api-macros/tests/ui/empty_impl_runtime_apis_call.stderr b/core/sr-api-macros/tests/ui/empty_impl_runtime_apis_call.stderr deleted file mode 100644 index 61527a3480361..0000000000000 --- a/core/sr-api-macros/tests/ui/empty_impl_runtime_apis_call.stderr +++ /dev/null @@ -1,5 +0,0 @@ -error: No api implementation given! - --> $DIR/empty_impl_runtime_apis_call.rs:18:1 - | -18 | impl_runtime_apis! {} - | ^^^^^^^^^^^^^^^^^^^^^ diff --git a/core/sr-api-macros/tests/ui/impl_incorrect_method_signature.stderr b/core/sr-api-macros/tests/ui/impl_incorrect_method_signature.stderr deleted file mode 100644 index 025ca60c48095..0000000000000 --- a/core/sr-api-macros/tests/ui/impl_incorrect_method_signature.stderr +++ /dev/null @@ -1,51 +0,0 @@ -error[E0053]: method `test` has an incompatible type for trait - --> $DIR/impl_incorrect_method_signature.rs:20:17 - | -14 | fn test(data: u64); - | --- type in trait -... -20 | fn test(data: String) {} - | ^^^^^^ expected u64, found struct `std::string::String` - | - = note: expected type `fn(u64)` - found type `fn(std::string::String)` - -error[E0053]: method `Api_test_runtime_api_impl` has an incompatible type for trait - --> $DIR/impl_incorrect_method_signature.rs:18:1 - | -12 | / decl_runtime_apis! { -13 | | pub trait Api { -14 | | fn test(data: u64); -15 | | } -16 | | } - | |_- type in trait -17 | -18 | impl_runtime_apis! { - | ^^^^^^^^^^^^^^^^^^ expected u64, found struct `std::string::String` - | - = note: expected type `fn(&RuntimeApiImpl, &sr_primitives::generic::block::BlockId, substrate_test_runtime::Extrinsic>>, sr_api_hidden_includes_DECL_RUNTIME_APIS::sr_api_client::runtime_api::ExecutionContext, std::option::Option, std::vec::Vec) -> std::result::Result, sr_api_hidden_includes_DECL_RUNTIME_APIS::sr_api_client::error::Error>` - found type `fn(&RuntimeApiImpl, &sr_primitives::generic::block::BlockId, substrate_test_runtime::Extrinsic>>, sr_api_hidden_includes_DECL_RUNTIME_APIS::sr_api_client::runtime_api::ExecutionContext, std::option::Option, std::vec::Vec) -> std::result::Result, sr_api_hidden_includes_DECL_RUNTIME_APIS::sr_api_client::error::Error>` - -error[E0308]: mismatched types - --> $DIR/impl_incorrect_method_signature.rs:18:1 - | -18 | / impl_runtime_apis! { -19 | | impl self::Api for Runtime { -20 | | fn test(data: String) {} -21 | | } -... | -33 | | } -34 | | } - | |_^ expected u64, found struct `std::string::String` - | - = note: expected type `u64` - found type `std::string::String` - -error[E0308]: mismatched types - --> $DIR/impl_incorrect_method_signature.rs:20:11 - | -20 | fn test(data: String) {} - | ^^^^ expected u64, found struct `std::string::String` - | - = note: expected type `u64` - found type `std::string::String` diff --git a/core/sr-api-macros/tests/ui/impl_two_traits_with_same_name.stderr b/core/sr-api-macros/tests/ui/impl_two_traits_with_same_name.stderr deleted file mode 100644 index 4c37b6b716a0a..0000000000000 --- a/core/sr-api-macros/tests/ui/impl_two_traits_with_same_name.stderr +++ /dev/null @@ -1,17 +0,0 @@ -error: Two traits with the same name detected! The trait name is used to generate its ID. Please rename one trait at the declaration! - --> $DIR/impl_two_traits_with_same_name.rs:33:15 - | -33 | impl second::Api for Runtime { - | ^^^ - -error[E0277]: the trait bound `RuntimeApiImpl: sr_api_hidden_includes_DECL_RUNTIME_APIS::sr_api_client::runtime_api::Core, substrate_test_runtime::Extrinsic>>` is not satisfied - --> $DIR/impl_two_traits_with_same_name.rs:29:7 - | -29 | impl self::Api for Runtime { - | ^^^^^^^^^^^^^^^^ the trait `sr_api_hidden_includes_DECL_RUNTIME_APIS::sr_api_client::runtime_api::Core, substrate_test_runtime::Extrinsic>>` is not implemented for `RuntimeApiImpl` - -error[E0277]: the trait bound `RuntimeApiImpl: sr_api_hidden_includes_DECL_RUNTIME_APIS::sr_api_client::runtime_api::Core, substrate_test_runtime::Extrinsic>>` is not satisfied - --> $DIR/impl_two_traits_with_same_name.rs:33:7 - | -33 | impl second::Api for Runtime { - | ^^^^^^^^^^^^^^^^^^ the trait `sr_api_hidden_includes_DECL_RUNTIME_APIS::sr_api_client::runtime_api::Core, substrate_test_runtime::Extrinsic>>` is not implemented for `RuntimeApiImpl` diff --git a/core/sr-api-macros/tests/ui/invalid_api_version.stderr b/core/sr-api-macros/tests/ui/invalid_api_version.stderr deleted file mode 100644 index e7d6aa0ed133f..0000000000000 --- a/core/sr-api-macros/tests/ui/invalid_api_version.stderr +++ /dev/null @@ -1,29 +0,0 @@ -error: can't qualify macro invocation with `pub` - --> $DIR/invalid_api_version.rs:3:1 - | -3 | / decl_runtime_apis! { -4 | | #[api_version] -5 | | pub trait Api { -6 | | fn test(data: u64); -7 | | } -8 | | } - | |_^ - | - = help: try adjusting the macro to put `pub` inside the invocation - -error: Unexpected `api_version` attribute. The supported format is `api_version(1)` - --> $DIR/invalid_api_version.rs:3:1 - | -3 | / decl_runtime_apis! { -4 | | #[api_version] -5 | | pub trait Api { -6 | | fn test(data: u64); -7 | | } -8 | | } - | |_^ - -error: Unexpected `api_version` attribute. The supported format is `api_version(1)` - --> $DIR/invalid_api_version.rs:4:4 - | -4 | #[api_version] - | ^^^^^^^^^^^ diff --git a/core/sr-api-macros/tests/ui/invalid_api_version_2.stderr b/core/sr-api-macros/tests/ui/invalid_api_version_2.stderr deleted file mode 100644 index 3e46efb258052..0000000000000 --- a/core/sr-api-macros/tests/ui/invalid_api_version_2.stderr +++ /dev/null @@ -1,29 +0,0 @@ -error: can't qualify macro invocation with `pub` - --> $DIR/invalid_api_version_2.rs:3:1 - | -3 | / decl_runtime_apis! { -4 | | #[api_version("1")] -5 | | pub trait Api { -6 | | fn test(data: u64); -7 | | } -8 | | } - | |_^ - | - = help: try adjusting the macro to put `pub` inside the invocation - -error: Unexpected `api_version` attribute. The supported format is `api_version(1)` - --> $DIR/invalid_api_version_2.rs:3:1 - | -3 | / decl_runtime_apis! { -4 | | #[api_version("1")] -5 | | pub trait Api { -6 | | fn test(data: u64); -7 | | } -8 | | } - | |_^ - -error: Unexpected `api_version` attribute. The supported format is `api_version(1)` - --> $DIR/invalid_api_version_2.rs:4:4 - | -4 | #[api_version("1")] - | ^^^^^^^^^^^ diff --git a/core/sr-api-macros/tests/ui/invalid_api_version_3.stderr b/core/sr-api-macros/tests/ui/invalid_api_version_3.stderr deleted file mode 100644 index 661221f28e892..0000000000000 --- a/core/sr-api-macros/tests/ui/invalid_api_version_3.stderr +++ /dev/null @@ -1,29 +0,0 @@ -error: can't qualify macro invocation with `pub` - --> $DIR/invalid_api_version_3.rs:3:1 - | -3 | / decl_runtime_apis! { -4 | | #[api_version()] -5 | | pub trait Api { -6 | | fn test(data: u64); -7 | | } -8 | | } - | |_^ - | - = help: try adjusting the macro to put `pub` inside the invocation - -error: Unexpected `api_version` attribute. The supported format is `api_version(1)` - --> $DIR/invalid_api_version_3.rs:3:1 - | -3 | / decl_runtime_apis! { -4 | | #[api_version()] -5 | | pub trait Api { -6 | | fn test(data: u64); -7 | | } -8 | | } - | |_^ - -error: Unexpected `api_version` attribute. The supported format is `api_version(1)` - --> $DIR/invalid_api_version_3.rs:4:4 - | -4 | #[api_version()] - | ^^^^^^^^^^^ diff --git a/core/sr-api-macros/tests/ui/missing_block_generic_parameter.stderr b/core/sr-api-macros/tests/ui/missing_block_generic_parameter.stderr deleted file mode 100644 index 5c8563a8b89d0..0000000000000 --- a/core/sr-api-macros/tests/ui/missing_block_generic_parameter.stderr +++ /dev/null @@ -1,13 +0,0 @@ -error: Missing `Block` generic parameter. - --> $DIR/missing_block_generic_parameter.rs:19:13 - | -19 | impl self::Api for Runtime { - | ^^^ - -error[E0107]: wrong number of type arguments: expected 1, found 0 - --> $DIR/missing_block_generic_parameter.rs:19:7 - | -19 | impl self::Api for Runtime { - | ^^^^^^^^^ expected 1 type argument - -For more information about this error, try `rustc --explain E0107`. diff --git a/core/sr-api-macros/tests/ui/type_reference_in_impl_runtime_apis_call.stderr b/core/sr-api-macros/tests/ui/type_reference_in_impl_runtime_apis_call.stderr deleted file mode 100644 index f3abaddd6ea9d..0000000000000 --- a/core/sr-api-macros/tests/ui/type_reference_in_impl_runtime_apis_call.stderr +++ /dev/null @@ -1,42 +0,0 @@ -error[E0053]: method `test` has an incompatible type for trait - --> $DIR/type_reference_in_impl_runtime_apis_call.rs:20:17 - | -14 | fn test(data: u64); - | --- type in trait -... -20 | fn test(data: &u64) { - | ^^^^ expected u64, found &u64 - | - = note: expected type `fn(u64)` - found type `fn(&u64)` - -error[E0053]: method `Api_test_runtime_api_impl` has an incompatible type for trait - --> $DIR/type_reference_in_impl_runtime_apis_call.rs:18:1 - | -12 | / decl_runtime_apis! { -13 | | pub trait Api { -14 | | fn test(data: u64); -15 | | } -16 | | } - | |_- type in trait -17 | -18 | impl_runtime_apis! { - | ^^^^^^^^^^^^^^^^^^ expected u64, found &u64 - | - = note: expected type `fn(&RuntimeApiImpl, &sr_primitives::generic::block::BlockId, substrate_test_runtime::Extrinsic>>, sr_api_hidden_includes_DECL_RUNTIME_APIS::sr_api_client::runtime_api::ExecutionContext, std::option::Option, std::vec::Vec) -> std::result::Result, sr_api_hidden_includes_DECL_RUNTIME_APIS::sr_api_client::error::Error>` - found type `fn(&RuntimeApiImpl, &sr_primitives::generic::block::BlockId, substrate_test_runtime::Extrinsic>>, sr_api_hidden_includes_DECL_RUNTIME_APIS::sr_api_client::runtime_api::ExecutionContext, std::option::Option<&u64>, std::vec::Vec) -> std::result::Result, sr_api_hidden_includes_DECL_RUNTIME_APIS::sr_api_client::error::Error>` - -error[E0308]: mismatched types - --> $DIR/type_reference_in_impl_runtime_apis_call.rs:18:1 - | -18 | / impl_runtime_apis! { -19 | | impl self::Api for Runtime { -20 | | fn test(data: &u64) { -21 | | unimplemented!() -... | -35 | | } -36 | | } - | |_^ expected u64, found &u64 - | - = note: expected type `u64` - found type `&u64` diff --git a/core/sr-io/build.rs b/core/sr-io/build.rs deleted file mode 100644 index 5b5d06b65a253..0000000000000 --- a/core/sr-io/build.rs +++ /dev/null @@ -1,13 +0,0 @@ -//! Set a nightly feature - -use rustc_version::{version, version_meta, Channel}; - -fn main() { - // Assert we haven't traveled back in time - assert!(version().unwrap().major >= 1); - - // Set cfg flags depending on release channel - if let Channel::Nightly = version_meta().unwrap().channel { - println!("cargo:rustc-cfg=feature=\"nightly\""); - } -} diff --git a/core/sr-io/src/lib.rs b/core/sr-io/src/lib.rs deleted file mode 100644 index fe5e50b3eda3e..0000000000000 --- a/core/sr-io/src/lib.rs +++ /dev/null @@ -1,397 +0,0 @@ -// Copyright 2017-2019 Parity Technologies (UK) Ltd. -// This file is part of Substrate. - -// Substrate is free software: you can redistribute it and/or modify -// it under the terms of the GNU General Public License as published by -// the Free Software Foundation, either version 3 of the License, or -// (at your option) any later version. - -// Substrate is distributed in the hope that it will be useful, -// but WITHOUT ANY WARRANTY; without even the implied warranty of -// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -// GNU General Public License for more details. - -// You should have received a copy of the GNU General Public License -// along with Substrate. If not, see . - -//! This is part of the Substrate runtime. - -#![warn(missing_docs)] - -#![cfg_attr(not(feature = "std"), no_std)] -#![cfg_attr(not(feature = "std"), feature(lang_items))] -#![cfg_attr(not(feature = "std"), feature(alloc_error_handler))] -#![cfg_attr(not(feature = "std"), feature(core_intrinsics))] - -#![cfg_attr(feature = "std", doc = "Substrate runtime standard library as compiled when linked with Rust's standard library.")] -#![cfg_attr(not(feature = "std"), doc = "Substrate's runtime standard library as compiled without Rust's standard library.")] - -use rstd::vec::Vec; - -use primitives::{ - crypto::KeyTypeId, ed25519, sr25519, H256, - offchain::{ - Timestamp, HttpRequestId, HttpRequestStatus, HttpError, StorageKind, OpaqueNetworkState, - }, - LogLevel, -}; - -/// Error verifying ECDSA signature -pub enum EcdsaVerifyError { - /// Incorrect value of R or S - BadRS, - /// Incorrect value of V - BadV, - /// Invalid signature - BadSignature, -} - -/// Converts a public trait definition into a private trait and set of public functions -/// that assume the trait is implemented for `()` for ease of calling. -macro_rules! export_api { - ( - $( #[$trait_attr:meta] )* - pub(crate) trait $trait_name:ident { - $( - $( #[$attr:meta] )* - fn $name:ident - ( $( $arg:ident : $arg_ty:ty ),* $(,)? ) - $( -> $ret:ty )? - $( where $( $w_name:path : $w_ty:path ),+ )?; - )* - } - ) => { - $( #[$trait_attr] )* - pub(crate) trait $trait_name { - $( - $( #[$attr] )* - fn $name ( $($arg : $arg_ty ),* ) $( -> $ret )? - $( where $( $w_name : $w_ty ),+ )?; - )* - } - - $( - $( #[$attr] )* - pub fn $name ( $($arg : $arg_ty ),* ) $( -> $ret )? - $( where $( $w_name : $w_ty ),+ )? - { - #[allow(deprecated)] - <()>:: $name ( $( $arg ),* ) - } - )* - } -} - -export_api! { - pub(crate) trait StorageApi { - /// Get `key` from storage and return a `Vec`, empty if there's a problem. - fn storage(key: &[u8]) -> Option>; - - /// Get `key` from child storage and return a `Vec`, empty if there's a problem. - fn child_storage(storage_key: &[u8], key: &[u8]) -> Option>; - - /// Get `key` from storage, placing the value into `value_out` and return the number of - /// bytes that the entry in storage has beyond the offset or `None` if the storage entry - /// doesn't exist at all. - /// If `value_out` length is smaller than the returned length, only `value_out` length bytes - /// are copied into `value_out`. - fn read_storage(key: &[u8], value_out: &mut [u8], value_offset: usize) -> Option; - - /// Get `key` from child storage, placing the value into `value_out` and return the number - /// of bytes that the entry in storage has beyond the offset or `None` if the storage entry - /// doesn't exist at all. - /// If `value_out` length is smaller than the returned length, only `value_out` length bytes - /// are copied into `value_out`. - fn read_child_storage(storage_key: &[u8], key: &[u8], value_out: &mut [u8], value_offset: usize) -> Option; - - /// Set the storage of some particular key to Some value. - fn set_storage(key: &[u8], value: &[u8]); - - /// Set the child storage of some particular key to Some value. - fn set_child_storage(storage_key: &[u8], key: &[u8], value: &[u8]); - - /// Clear the storage of a key. - fn clear_storage(key: &[u8]); - - /// Clear the storage of a key. - fn clear_child_storage(storage_key: &[u8], key: &[u8]); - - /// Clear an entire child storage. - fn kill_child_storage(storage_key: &[u8]); - - /// Check whether a given `key` exists in storage. - fn exists_storage(key: &[u8]) -> bool; - - /// Check whether a given `key` exists in storage. - fn exists_child_storage(storage_key: &[u8], key: &[u8]) -> bool; - - /// Clear the storage entries with a key that starts with the given prefix. - fn clear_prefix(prefix: &[u8]); - - /// Clear the child storage entries with a key that starts with the given prefix. - fn clear_child_prefix(storage_key: &[u8], prefix: &[u8]); - - /// "Commit" all existing operations and compute the resultant storage root. - fn storage_root() -> [u8; 32]; - - /// "Commit" all existing operations and compute the resultant child storage root. - fn child_storage_root(storage_key: &[u8]) -> Vec; - - /// "Commit" all existing operations and get the resultant storage change root. - fn storage_changes_root(parent_hash: [u8; 32]) -> Option<[u8; 32]>; - - /// A trie root formed from the iterated items. - fn blake2_256_trie_root(input: Vec<(Vec, Vec)>) -> H256; - - /// A trie root formed from the enumerated items. - fn blake2_256_ordered_trie_root(input: Vec>) -> H256; - } -} - -export_api! { - pub(crate) trait OtherApi { - /// The current relay chain identifier. - fn chain_id() -> u64; - - /// Print a number. - fn print_num(val: u64); - /// Print any valid `utf8` buffer. - fn print_utf8(utf8: &[u8]); - /// Print any `u8` slice as hex. - fn print_hex(data: &[u8]); - - /// Request to print a log message (stderr) on the host. - /// - /// Note that this will be only displayed if the host - /// is enabed to display log messages with given - /// level and target. - /// - /// Instead of using directly, prefer setting up `RuntimeLogger` - /// and using `log` macros. - fn log( - level: LogLevel, - target: &[u8], - message: &[u8] - ); - } -} - -export_api! { - pub(crate) trait CryptoApi { - /// Returns all ed25519 public keys for the given key id from the keystore. - fn ed25519_public_keys(id: KeyTypeId) -> Vec; - /// Generate an ed22519 key for the given key type and store it in the keystore. - /// - /// Returns the raw public key. - fn ed25519_generate(id: KeyTypeId, seed: Option<&str>) -> ed25519::Public; - /// Sign the given `msg` with the ed25519 key that corresponds to the given public key and - /// key type in the keystore. - /// - /// Returns the raw signature. - fn ed25519_sign( - id: KeyTypeId, - pubkey: &ed25519::Public, - msg: &[u8], - ) -> Option; - /// Verify an ed25519 signature. - /// - /// Returns `true` when the verification in successful. - fn ed25519_verify(sig: &ed25519::Signature, msg: &[u8], pubkey: &ed25519::Public) -> bool; - - /// Returns all sr25519 public keys for the given key id from the keystore. - fn sr25519_public_keys(id: KeyTypeId) -> Vec; - /// Generate an sr22519 key for the given key type and store it in the keystore. - /// - /// Returns the raw public key. - fn sr25519_generate(id: KeyTypeId, seed: Option<&str>) -> sr25519::Public; - /// Sign the given `msg` with the sr25519 key that corresponds to the given public key and - /// key type in the keystore. - /// - /// Returns the raw signature. - fn sr25519_sign( - id: KeyTypeId, - pubkey: &sr25519::Public, - msg: &[u8], - ) -> Option; - /// Verify an sr25519 signature. - /// - /// Returns `true` when the verification in successful. - fn sr25519_verify(sig: &sr25519::Signature, msg: &[u8], pubkey: &sr25519::Public) -> bool; - - /// Verify and recover a SECP256k1 ECDSA signature. - /// - `sig` is passed in RSV format. V should be either 0/1 or 27/28. - /// - returns `Err` if the signature is bad, otherwise the 64-byte raw pubkey (doesn't include the 0x04 prefix). - fn secp256k1_ecdsa_recover(sig: &[u8; 65], msg: &[u8; 32]) -> Result<[u8; 64], EcdsaVerifyError>; - - /// Verify and recover a SECP256k1 ECDSA signature. - /// - `sig` is passed in RSV format. V should be either 0/1 or 27/28. - /// - returns `Err` if the signature is bad, otherwise the 33-byte compressed pubkey. - fn secp256k1_ecdsa_recover_compressed(sig: &[u8; 65], msg: &[u8; 32]) -> Result<[u8; 33], EcdsaVerifyError>; - } -} - -export_api! { - pub(crate) trait HashingApi { - /// Conduct a 256-bit Keccak hash. - fn keccak_256(data: &[u8]) -> [u8; 32]; - - /// Conduct a 128-bit Blake2 hash. - fn blake2_128(data: &[u8]) -> [u8; 16]; - - /// Conduct a 256-bit Blake2 hash. - fn blake2_256(data: &[u8]) -> [u8; 32]; - - /// Conduct four XX hashes to give a 256-bit result. - fn twox_256(data: &[u8]) -> [u8; 32]; - - /// Conduct two XX hashes to give a 128-bit result. - fn twox_128(data: &[u8]) -> [u8; 16]; - - /// Conduct two XX hashes to give a 64-bit result. - fn twox_64(data: &[u8]) -> [u8; 8]; - } -} - -export_api! { - pub(crate) trait OffchainApi { - /// Returns if the local node is a potential validator. - /// - /// Even if this function returns `true`, it does not mean that any keys are configured - /// and that the validator is registered in the chain. - fn is_validator() -> bool; - - /// Submit transaction to the pool. - /// - /// The transaction will end up in the pool. - fn submit_transaction(data: Vec) -> Result<(), ()>; - - /// Returns information about the local node's network state. - fn network_state() -> Result; - - /// Returns current UNIX timestamp (in millis) - fn timestamp() -> Timestamp; - - /// Pause the execution until `deadline` is reached. - fn sleep_until(deadline: Timestamp); - - /// Returns a random seed. - /// - /// This is a trully random non deterministic seed generated by host environment. - /// Obviously fine in the off-chain worker context. - fn random_seed() -> [u8; 32]; - - /// Sets a value in the local storage. - /// - /// Note this storage is not part of the consensus, it's only accessible by - /// offchain worker tasks running on the same machine. It IS persisted between runs. - fn local_storage_set(kind: StorageKind, key: &[u8], value: &[u8]); - - /// Sets a value in the local storage if it matches current value. - /// - /// Since multiple offchain workers may be running concurrently, to prevent - /// data races use CAS to coordinate between them. - /// - /// Returns `true` if the value has been set, `false` otherwise. - /// - /// Note this storage is not part of the consensus, it's only accessible by - /// offchain worker tasks running on the same machine. It IS persisted between runs. - fn local_storage_compare_and_set( - kind: StorageKind, - key: &[u8], - old_value: Option<&[u8]>, - new_value: &[u8], - ) -> bool; - - /// Gets a value from the local storage. - /// - /// If the value does not exist in the storage `None` will be returned. - /// Note this storage is not part of the consensus, it's only accessible by - /// offchain worker tasks running on the same machine. It IS persisted between runs. - fn local_storage_get(kind: StorageKind, key: &[u8]) -> Option>; - - /// Initiates a http request given HTTP verb and the URL. - /// - /// Meta is a future-reserved field containing additional, parity-scale-codec encoded parameters. - /// Returns the id of newly started request. - fn http_request_start( - method: &str, - uri: &str, - meta: &[u8], - ) -> Result; - - /// Append header to the request. - fn http_request_add_header( - request_id: HttpRequestId, - name: &str, - value: &str, - ) -> Result<(), ()>; - - /// Write a chunk of request body. - /// - /// Writing an empty chunks finalises the request. - /// Passing `None` as deadline blocks forever. - /// - /// Returns an error in case deadline is reached or the chunk couldn't be written. - fn http_request_write_body( - request_id: HttpRequestId, - chunk: &[u8], - deadline: Option, - ) -> Result<(), HttpError>; - - /// Block and wait for the responses for given requests. - /// - /// Returns a vector of request statuses (the len is the same as ids). - /// Note that if deadline is not provided the method will block indefinitely, - /// otherwise unready responses will produce `DeadlineReached` status. - /// - /// Passing `None` as deadline blocks forever. - fn http_response_wait( - ids: &[HttpRequestId], - deadline: Option, - ) -> Vec; - - /// Read all response headers. - /// - /// Returns a vector of pairs `(HeaderKey, HeaderValue)`. - /// NOTE response headers have to be read before response body. - fn http_response_headers(request_id: HttpRequestId) -> Vec<(Vec, Vec)>; - - /// Read a chunk of body response to given buffer. - /// - /// Returns the number of bytes written or an error in case a deadline - /// is reached or server closed the connection. - /// If `0` is returned it means that the response has been fully consumed - /// and the `request_id` is now invalid. - /// NOTE this implies that response headers must be read before draining the body. - /// Passing `None` as a deadline blocks forever. - fn http_response_read_body( - request_id: HttpRequestId, - buffer: &mut [u8], - deadline: Option, - ) -> Result; - } -} - -/// API trait that should cover all other APIs. -/// -/// Implement this to make sure you implement all APIs. -trait Api: StorageApi + OtherApi + CryptoApi + HashingApi + OffchainApi {} - -mod imp { - use super::*; - - #[cfg(feature = "std")] - include!("../with_std.rs"); - - #[cfg(not(feature = "std"))] - include!("../without_std.rs"); -} - -#[cfg(feature = "std")] -pub use self::imp::{StorageOverlay, ChildrenStorageOverlay, with_storage}; -#[cfg(not(feature = "std"))] -pub use self::imp::ext::*; - -/// Type alias for Externalities implementation used in tests. -#[cfg(feature = "std")] -pub type TestExternalities = self::imp::TestExternalities; diff --git a/core/sr-sandbox/build.rs b/core/sr-sandbox/build.rs deleted file mode 100755 index 5b5d06b65a253..0000000000000 --- a/core/sr-sandbox/build.rs +++ /dev/null @@ -1,13 +0,0 @@ -//! Set a nightly feature - -use rustc_version::{version, version_meta, Channel}; - -fn main() { - // Assert we haven't traveled back in time - assert!(version().unwrap().major >= 1); - - // Set cfg flags depending on release channel - if let Channel::Nightly = version_meta().unwrap().channel { - println!("cargo:rustc-cfg=feature=\"nightly\""); - } -} diff --git a/core/sr-std/build.rs b/core/sr-std/build.rs deleted file mode 100644 index af9c91db877dd..0000000000000 --- a/core/sr-std/build.rs +++ /dev/null @@ -1,13 +0,0 @@ -//! Set a nightly feature - -use rustc_version::{version, version_meta, Channel}; - -fn main() { - // Assert we haven't traveled back in time - assert!(version().unwrap().major >= 1); - - // Set cfg flags depending on release channel - if let Channel::Nightly = version_meta().unwrap().channel { - println!("cargo:rustc-cfg=feature=\"nightly\""); - } -} diff --git a/core/test-client/Cargo.toml b/core/test-client/Cargo.toml deleted file mode 100644 index de13b34d69753..0000000000000 --- a/core/test-client/Cargo.toml +++ /dev/null @@ -1,18 +0,0 @@ -[package] -name = "substrate-test-client" -version = "2.0.0" -authors = ["Parity Technologies "] -edition = "2018" - -[dependencies] -client = { package = "substrate-client", path = "../client" } -client-db = { package = "substrate-client-db", path = "../client/db", features = ["test-helpers"] } -consensus = { package = "substrate-consensus-common", path = "../consensus/common" } -executor = { package = "substrate-executor", path = "../executor" } -futures-preview = "0.3.0-alpha.19" -hash-db = "0.15.2" -keyring = { package = "substrate-keyring", path = "../keyring" } -codec = { package = "parity-scale-codec", version = "1.0.0" } -primitives = { package = "substrate-primitives", path = "../primitives" } -sr-primitives = { path = "../sr-primitives" } -state_machine = { package = "substrate-state-machine", path = "../state-machine" } diff --git a/core/test-runtime/Cargo.toml b/core/test-runtime/Cargo.toml deleted file mode 100644 index 4e7c3f8bca450..0000000000000 --- a/core/test-runtime/Cargo.toml +++ /dev/null @@ -1,75 +0,0 @@ -[package] -name = "substrate-test-runtime" -version = "2.0.0" -authors = ["Parity Technologies "] -edition = "2018" -build = "build.rs" - -[dependencies] -log = { version = "0.4.8", optional = true } -serde = { version = "1.0.101", optional = true, features = ["derive"] } -codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false, features = ["derive"] } -keyring = { package = "substrate-keyring", path = "../keyring", optional = true } -substrate-client = { path = "../client", default-features = false } -primitives = { package = "substrate-primitives", path = "../primitives", default-features = false } -app-crypto = { package = "substrate-application-crypto", path = "../application-crypto", default-features = false } -inherents = { package = "substrate-inherents", path = "../inherents", default-features = false } -aura-primitives = { package = "substrate-consensus-aura-primitives", path = "../consensus/aura/primitives", default-features = false } -babe-primitives = { package = "substrate-consensus-babe-primitives", path = "../consensus/babe/primitives", default-features = false } -rstd = { package = "sr-std", path = "../sr-std", default-features = false } -runtime_io = { package = "sr-io", path = "../sr-io", default-features = false } -sr-primitives = { path = "../sr-primitives", default-features = false } -session = { package = "substrate-session", path = "../session", default-features = false } -runtime_version = { package = "sr-version", path = "../sr-version", default-features = false } -runtime_support = { package = "srml-support", path = "../../srml/support", default-features = false } -substrate-trie = { path = "../trie", default-features = false } -trie-db = { version = "0.15.2", default-features = false } -memory-db = { version = "0.15.2", default-features = false } -offchain-primitives = { package = "substrate-offchain-primitives", path = "../offchain/primitives", default-features = false} -executive = { package = "srml-executive", path = "../../srml/executive", default-features = false } -cfg-if = "0.1.10" -srml-babe = { path = "../../srml/babe", default-features = false } -srml-timestamp = { path = "../../srml/timestamp", default-features = false } -srml-system = { path = "../../srml/system", default-features = false } -srml-system-rpc-runtime-api = { path = "../../srml/system/rpc/runtime-api", default-features = false } - -[dev-dependencies] -substrate-executor = { path = "../executor" } -substrate-test-runtime-client = { path = "./client" } -state_machine = { package = "substrate-state-machine", path = "../state-machine" } - -[build-dependencies] -wasm-builder-runner = { package = "substrate-wasm-builder-runner", version = "1.0.4", path = "../utils/wasm-builder-runner" } - -[features] -default = [ - "std", -] -std = [ - "log", - "serde", - "substrate-client/std", - "keyring", - "codec/std", - "rstd/std", - "runtime_io/std", - "runtime_support/std", - "primitives/std", - "inherents/std", - "sr-primitives/std", - "runtime_version/std", - "aura-primitives/std", - "babe-primitives/std", - "primitives/std", - "substrate-trie/std", - "trie-db/std", - "memory-db/std", - "offchain-primitives/std", - "executive/std", - "srml-babe/std", - "srml-timestamp/std", - "srml-system/std", - "srml-system-rpc-runtime-api/std", - "app-crypto/std", - "session/std", -] diff --git a/core/test-runtime/client/Cargo.toml b/core/test-runtime/client/Cargo.toml deleted file mode 100644 index 6545048759687..0000000000000 --- a/core/test-runtime/client/Cargo.toml +++ /dev/null @@ -1,20 +0,0 @@ -[package] -name = "substrate-test-runtime-client" -version = "2.0.0" -authors = ["Parity Technologies "] -edition = "2018" - -[dependencies] -generic-test-client = { package = "substrate-test-client", path = "../../test-client" } -primitives = { package = "substrate-primitives", path = "../../primitives" } -runtime = { package = "substrate-test-runtime", path = "../../test-runtime", default-features = false } -sr-primitives = { path = "../../sr-primitives" } -codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false } - -[features] -default = [ - "std", -] -std = [ - "runtime/std", -] diff --git a/core/transaction-pool/Cargo.toml b/core/transaction-pool/Cargo.toml deleted file mode 100644 index 33ec9e9ece176..0000000000000 --- a/core/transaction-pool/Cargo.toml +++ /dev/null @@ -1,20 +0,0 @@ -[package] -name = "substrate-transaction-pool" -version = "2.0.0" -authors = ["Parity Technologies "] -edition = "2018" - -[dependencies] -derive_more = "0.15.0" -log = "0.4.8" -futures-preview = "0.3.0-alpha.19" -codec = { package = "parity-scale-codec", version = "1.0.0" } -parking_lot = "0.9.0" -sr-primitives = { path = "../sr-primitives" } -client = { package = "substrate-client", path = "../client" } -primitives = { package = "substrate-primitives", path = "../primitives" } -txpool = { package = "substrate-transaction-graph", path = "./graph" } - -[dev-dependencies] -keyring = { package = "substrate-keyring", path = "../../core/keyring" } -test-client = { package = "substrate-test-runtime-client", path = "../../core/test-runtime/client" } diff --git a/core/transaction-pool/src/api.rs b/core/transaction-pool/src/api.rs deleted file mode 100644 index 96403bd3f876f..0000000000000 --- a/core/transaction-pool/src/api.rs +++ /dev/null @@ -1,88 +0,0 @@ -// Copyright 2018-2019 Parity Technologies (UK) Ltd. -// This file is part of Substrate. - -// Substrate is free software: you can redistribute it and/or modify -// it under the terms of the GNU General Public License as published by -// the Free Software Foundation, either version 3 of the License, or -// (at your option) any later version. - -// Substrate is distributed in the hope that it will be useful, -// but WITHOUT ANY WARRANTY; without even the implied warranty of -// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -// GNU General Public License for more details. - -// You should have received a copy of the GNU General Public License -// along with Substrate. If not, see . - -//! Chain api required for the transaction pool. - -use std::{ - sync::Arc, - marker::PhantomData, -}; -use client::{runtime_api::TaggedTransactionQueue, blockchain::HeaderBackend}; -use codec::Encode; -use txpool; -use primitives::{ - H256, - Blake2Hasher, - Hasher, -}; -use sr_primitives::{ - generic::BlockId, - traits, - transaction_validity::TransactionValidity, -}; - -use crate::error; - -/// The transaction pool logic -pub struct FullChainApi { - client: Arc, - _marker: PhantomData, -} - -impl FullChainApi where - Block: traits::Block, - T: traits::ProvideRuntimeApi + HeaderBackend { - /// Create new transaction pool logic. - pub fn new(client: Arc) -> Self { - FullChainApi { - client, - _marker: Default::default() - } - } -} - -impl txpool::ChainApi for FullChainApi where - Block: traits::Block, - T: traits::ProvideRuntimeApi + HeaderBackend, - T::Api: TaggedTransactionQueue -{ - type Block = Block; - type Hash = H256; - type Error = error::Error; - type ValidationFuture = futures::future::Ready>; - - fn validate_transaction( - &self, - at: &BlockId, - uxt: txpool::ExtrinsicFor, - ) -> Self::ValidationFuture { - futures::future::ready(self.client.runtime_api().validate_transaction(at, uxt).map_err(Into::into)) - } - - fn block_id_to_number(&self, at: &BlockId) -> error::Result>> { - Ok(self.client.block_number_from_id(at)?) - } - - fn block_id_to_hash(&self, at: &BlockId) -> error::Result>> { - Ok(self.client.block_hash_from_id(at)?) - } - - fn hash_and_length(&self, ex: &txpool::ExtrinsicFor) -> (Self::Hash, usize) { - ex.using_encoded(|x| { - (Blake2Hasher::hash(x), x.len()) - }) - } -} diff --git a/CODEOWNERS b/docs/CODEOWNERS similarity index 85% rename from CODEOWNERS rename to docs/CODEOWNERS index 866f16522316b..f61da029bc8ab 100644 --- a/CODEOWNERS +++ b/docs/CODEOWNERS @@ -41,22 +41,22 @@ /core/rpc-servers/ @tomusdrw # GRANDPA, BABE, consensus stuff -/srml/babe/ @andresilva @DemiMarie-parity -/srml/grandpa/ @andresilva @DemiMarie-parity +/paint/babe/ @andresilva @DemiMarie-parity +/paint/grandpa/ @andresilva @DemiMarie-parity /core/finality-grandpa/ @andresilva @DemiMarie-parity /core/consensus/babe/ @andresilva @DemiMarie-parity /core/consensus/slots/ @andresilva @DemiMarie-parity # Contracts -/srml/contracts/ @pepyakin @thiolliere @jimpo -/srml/contracts/src/wasm/runtime.rs @Robbepop +/paint/contracts/ @pepyakin @thiolliere @jimpo +/paint/contracts/src/wasm/runtime.rs @Robbepop # Inflation points -/srml/staking/src/inflation.rs @thiolliere +/paint/staking/src/inflation.rs @thiolliere # NPoS and Governance -/srml/staking/ @kianenigma -/srml/election/ @kianenigma +/paint/staking/ @kianenigma +/paint/election/ @kianenigma # End to end testing of substrate node /node/executor/ @kianenigma @@ -65,4 +65,4 @@ /core/sr-primitives/weights.rs @kianenigma # Support crates -/srml/support/ @thiolliere @kianenigma +/paint/support/ @thiolliere @kianenigma diff --git a/CODE_OF_CONDUCT.adoc b/docs/CODE_OF_CONDUCT.adoc similarity index 100% rename from CODE_OF_CONDUCT.adoc rename to docs/CODE_OF_CONDUCT.adoc diff --git a/CONTRIBUTING.adoc b/docs/CONTRIBUTING.adoc similarity index 100% rename from CONTRIBUTING.adoc rename to docs/CONTRIBUTING.adoc diff --git a/PULL_REQUEST_TEMPLATE.md b/docs/PULL_REQUEST_TEMPLATE.md similarity index 100% rename from PULL_REQUEST_TEMPLATE.md rename to docs/PULL_REQUEST_TEMPLATE.md diff --git a/docs/README.adoc b/docs/README.adoc new file mode 100644 index 0000000000000..05eb7e3f814ae --- /dev/null +++ b/docs/README.adoc @@ -0,0 +1,528 @@ += Substrate +:Author: Substrate developers +:Revision: 0.2.0 +:toc: +:sectnums: + +== Intro in one sentence + +Substrate is a next-generation framework for blockchain innovation. + +== Description + +At its heart, Substrate is a combination of three technologies: https://webassembly.org/[WebAssembly], https://libp2p.io/[Libp2p] and GRANDPA Consensus. About GRANDPA, see this https://hackmd.io/Jd0byWX0RiqFiXUVC78Bdw?view#GRANDPA[definition], https://medium.com/polkadot-network/grandpa-block-finality-in-polkadot-an-introduction-part-1-d08a24a021b5[introduction] and https://github.com/w3f/consensus/blob/master/pdf/grandpa.pdf[formal specification]. It is both a library for building new blockchains and a "skeleton key" of a blockchain client, able to synchronize to any Substrate-based chain. + +Substrate chains have three distinct features that make them "next-generation": a dynamic, self-defining state-transition function; light-client functionality from day one; and a progressive consensus algorithm with fast block production and adaptive, definite finality. The STF, encoded in WebAssembly, is known as the "runtime". This defines the `execute_block` function, and can specify everything from the staking algorithm, transaction semantics, logging mechanisms and procedures for replacing any aspect of itself or of the blockchain's state ("governance"). Because the runtime is entirely dynamic all of these can be switched out or upgraded at any time. A Substrate chain is very much a "living organism". + +See also https://www.parity.io/what-is-substrate/. + +== Usage + +Substrate is still an early stage project, and while it has already been used as the basis of major projects like Polkadot, using it is still a significant undertaking. In particular, you should have a good knowledge of blockchain concepts and basic cryptography. Terminology like header, block, client, hash, transaction and signature should be familiar. At present you will need a working knowledge of Rust to be able to do anything interesting (though eventually, we aim for this not to be the case). + +Substrate is designed for use in one of three ways: + +**1. Trivial**: By running the Substrate binary `substrate` and configuring it with a genesis block that includes the current demonstration runtime. In this case, you just build Substrate, configure a JSON file, and launch your own blockchain. This affords you the least amount of customizability, primarily allowing you to change the genesis parameters of the various included runtime modules such as balances, staking, block-period, fees, and governance. + +**2. Modular**: By hacking together modules from the Substrate Runtime Module Library (SRML) into a new runtime and possibly altering or reconfiguring the Substrate client's block authoring logic. This affords you a very large amount of freedom over your blockchain's logic, letting you change datatypes, add or remove modules, and crucially, add your own modules. Much can be changed without touching the block authoring logic (since it is generic). If this is the case, then the existing Substrate binary can be used for block authoring and syncing. If the block authoring logic needs to be tweaked, then a new, altered block authoring binary must be built as a separate project and used by validators. This is how the Polkadot relay chain is built and should suffice for almost all circumstances in the near to mid-term. + +**3. Generic**: The entire SRML can be ignored and the entire runtime designed and implemented from scratch. If desired, this can be done in a language other than Rust, provided it can target WebAssembly. If the runtime can be made compatible with the existing client's block authoring logic, then you can simply construct a new genesis block from your Wasm blob and launch your chain with the existing Rust-based Substrate client. If not, then you'll need to alter the client's block authoring logic accordingly. This is probably a useless option for most projects right now, but provides complete flexibility allowing for a long-term, far-reaching upgrade path for the Substrate paradigm. + +=== The Basics of Substrate + +Substrate is a blockchain platform with a completely generic state transition function. That said, it does come with both standards and conventions (particularly regarding the Runtime Module Library) regarding underlying data structures. Roughly speaking, these core datatypes correspond to +trait+s in terms of the actual non-negotiable standard and generic +struct+s in terms of the convention. + +``` +Header := Parent + ExtrinsicsRoot + StorageRoot + Digest +Block := Header + Extrinsics + Justifications +``` + +=== Extrinsics + +Extrinsics in Substrate are pieces of information from "the outside world" that are contained in the blocks of the chain. You might think "ahh, that means *transactions*": in fact, no. Extrinsics fall into two broad categories of which only one is *transactions*. The other is known as *inherents*. The difference between these two is that transactions are signed and gossiped on the network and can be deemed useful *per se*. This fits the mold of what you would call transactions in Bitcoin or Ethereum. + +Inherents, meanwhile, are not passed on the network and are not signed. They represent data which describes the environment but which cannot call upon anything to prove it such as a signature. Rather they are assumed to be "true" simply because a sufficiently large number of validators have agreed on them being reasonable. + +To give an example, there is the timestamp inherent, which sets the current timestamp of the block. This is not a fixed part of Substrate, but does come as part of the Substrate Runtime Module Library to be used as desired. No signature could fundamentally prove that a block were authored at a given time in quite the same way that a signature can "prove" the desire to spend some particular funds. Rather, it is the business of each validator to ensure that they believe the timestamp is set to something reasonable before they agree that the block candidate is valid. + +Other examples include the parachain-heads extrinsic in Polkadot and the "note-missed-proposal" extrinsic used in the Substrate Runtime Module Library to determine and punish or deactivate offline validators. + + +=== Runtime and API + +Substrate chains all have a runtime. The runtime is a WebAssembly "blob" that includes a number of entry-points. Some entry-points are required as part of the underlying Substrate specification. Others are merely convention and required for the default implementation of the Substrate client to be able to author blocks. + +If you want to develop a chain with Substrate, you will need to implement the `Core` trait. This `Core` trait generates an API with the minimum necessary functionality to interact with your runtime. A special macro is provided called `impl_runtime_apis!` that help you implement runtime API traits. All runtime API trait implementations need to be done in one call of the `impl_runtime_apis!` macro. All parameters and return values need to implement https://crates.io/crates/parity-codec[`parity-codec`] to be encodable and decodable. + +Here's a snippet of the Polkadot API implementation as of PoC-3: + +```rust +impl_runtime_apis! { + impl client_api::Core for Runtime { + fn version() -> RuntimeVersion { + VERSION + } + + fn execute_block(block: Block) { + Executive::execute_block(block) + } + + fn initialize_block(header: ::Header) { + Executive::initialize_block(&header) + } + } + // ---snip--- +} +``` + + +=== Inherent Extrinsics + +The Substrate Runtime Module Library includes functionality for timestamps and slashing. If used, these rely on "trusted" external information being passed in via inherent extrinsics. The Substrate reference block authoring client software will expect to be able to call into the runtime API with collated data (in the case of the reference Substrate authoring client, this is merely the current timestamp and which nodes were offline) in order to return the appropriate extrinsics ready for inclusion. If new inherent extrinsic types and data are to be used in a modified runtime, then it is this function (and its argument type) that would change. + +=== Block-authoring Logic + +In Substrate, there is a major distinction between blockchain *syncing* and block *authoring* ("authoring" is a more general term for what is called "mining" in Bitcoin). The first case might be referred to as a "full node" (or "light node" - Substrate supports both): authoring necessarily requires a synced node and, therefore, all authoring clients must necessarily be able to synchronize. However, the reverse is not true. The primary functionality that authoring nodes have which is not in "sync nodes" is threefold: transaction queue logic, inherent transaction knowledge and BFT consensus logic. BFT consensus logic is provided as a core element of Substrate and can be ignored since it is only exposed in the SDK under the `authorities()` API entry. + +Transaction queue logic in Substrate is designed to be as generic as possible, allowing a runtime to express which transactions are fit for inclusion in a block through the `initialize_block` and `apply_extrinsic` calls. However, more subtle aspects like prioritization and replacement policy must currently be expressed "hard coded" as part of the blockchain's authoring code. That said, Substrate's reference implementation for a transaction queue should be sufficient for an initial chain implementation. + +Inherent extrinsic knowledge is again somewhat generic, and the actual construction of the extrinsics is, by convention, delegated to the "soft code" in the runtime. If ever there needs to be additional extrinsic information in the chain, then both the block authoring logic will need to be altered to provide it into the runtime and the runtime's `inherent_extrinsics` call will need to use this extra information in order to construct any additional extrinsic transactions for inclusion in the block. + +== Roadmap + +=== So far + +- 0.1 "PoC-1": PBFT consensus, Wasm runtime engine, basic runtime modules. +- 0.2 "PoC-2": Libp2p + +=== In progress + +- AfG consensus +- Improved PoS +- Smart contract runtime module + +=== The future + +- Splitting out runtime modules into separate repo +- Introduce substrate executable (the skeleton-key runtime) +- Introduce basic but extensible transaction queue and block-builder and place them in the executable. +- DAO runtime module +- Audit + +== Trying out Substrate Node + +Substrate Node is Substrate's pre-baked blockchain client. You can run a development node locally or configure a new chain and launch your own global testnet. + +=== On Mac and Ubuntu + +To get going as fast as possible, there is a simple script that installs all required dependencies and installs Substrate into your path. Just open a terminal and run: + +[source, shell] +---- +curl https://getsubstrate.io -sSf | bash +---- + +You can start a local Substrate development chain with running `substrate --dev`. + +To create your own global network/cryptocurrency, you'll need to make a new Substrate Node chain specification file ("chainspec"). + +First let's get a template chainspec that you can edit. We'll use the "staging" chain, a sort of default chain that the node comes pre-configured with: + +[source, shell] +---- +substrate build-spec --chain=staging > ~/chainspec.json +---- + +Now, edit `~/chainspec.json` in your editor. There are a lot of individual fields for each module, and one very large one which contains the WebAssembly code blob for this chain. The easiest field to edit is the block `period`. Change it to 10 (seconds): + +[source, json] +---- + "timestamp": { + "minimumPeriod": 10 + }, +---- + +Now with this new chainspec file, you can build a "raw" chain definition for your new chain: + +[source, shell] +---- +substrate build-spec --chain ~/chainspec.json --raw > ~/mychain.json +---- + +This can be fed into Substrate: + +[source, shell] +---- +substrate --chain ~/mychain.json +---- + +It won't do much until you start producing blocks though, so to do that you'll need to use the `--validator` option together with passing the seed for the account(s) that is configured to be the initial authorities: + +[source, shell] +---- +substrate --chain ~/mychain.json --validator +---- + +You can distribute `mychain.json` so that everyone can synchronize and (depending on your authorities list) validate on your chain. + + +== Building + +=== Hacking on Substrate + +If you'd actually like to hack on Substrate, you can just grab the source code and +build it. Ensure you have Rust and the support software installed: + +==== Linux and Mac + +For Unix-based operating systems, you should run the following commands: + +[source, shell] +---- +curl https://sh.rustup.rs -sSf | sh + +rustup update nightly +rustup target add wasm32-unknown-unknown --toolchain nightly +rustup update stable +---- + +You will also need to install the following packages: + + - Linux: +[source, shell] +sudo apt install cmake pkg-config libssl-dev git clang libclang-dev + +- Linux on ARM: +`rust-lld` is required for linking wasm, but is missing on non Tier 1 platforms. +So, use this https://github.com/Plume-org/Plume/blob/master/script/wasm-deps.sh[script] +to build `lld` and create the symlink `/usr/bin/rust-lld` to the build binary. + + - Mac: +[source, shell] +brew install cmake pkg-config openssl git llvm + +To finish installation of Substrate, jump down to <>. + +==== Windows + +If you are trying to set up Substrate on Windows, you should do the following: + +1. First, you will need to download and install "Build Tools for Visual Studio:" + + * You can get it at this link: https://aka.ms/buildtools + * Run the installation file: `vs_buildtools.exe` + * Please ensure the Windows 10 SDK component is included when installing the Visual C++ Build Tools. + * image:https://i.imgur.com/zayVLmu.png[image] + * Restart your computer. + +2. Next, you need to install Rust: + + * Detailed instructions are provided by the https://doc.rust-lang.org/book/ch01-01-installation.html#installing-rustup-on-windows[Rust Book]. + * Download from: https://www.rust-lang.org/tools/install + * Run the installation file: `rustup-init.exe` + > Note that it should not prompt you to install vs_buildtools since you did it in step 1. + * Choose "Default Installation." + * To get started, you need Cargo's bin directory (%USERPROFILE%\.cargo\bin) in your PATH environment variable. Future applications will automatically have the correct environment, but you may need to restart your current shell. + +3. Then, you will need to run some commands in CMD to set up your Wasm Build Environment: + + rustup update nightly + rustup update stable + rustup target add wasm32-unknown-unknown --toolchain nightly + +4. Then, you need to install LLVM: https://releases.llvm.org/download.html + +5. Next, you need to install OpenSSL, which we will do with `vcpkg`: + + mkdir \Tools + cd \Tools + git clone https://github.com/Microsoft/vcpkg.git + cd vcpkg + .\bootstrap-vcpkg.bat + .\vcpkg.exe install openssl:x64-windows-static + +6. After, you need to add OpenSSL to your System Variables. Note that in order for the following commands to work, you need to use Windows Powershell: + + $env:OPENSSL_DIR = 'C:\Tools\vcpkg\installed\x64-windows-static' + $env:OPENSSL_STATIC = 'Yes' + [System.Environment]::SetEnvironmentVariable('OPENSSL_DIR', $env:OPENSSL_DIR, [System.EnvironmentVariableTarget]::User) + [System.Environment]::SetEnvironmentVariable('OPENSSL_STATIC', $env:OPENSSL_STATIC, [System.EnvironmentVariableTarget]::User) + +7. Finally, you need to install `cmake`: https://cmake.org/download/ + +==== Shared Steps + +Then, grab the Substrate source code: + +[source, shell] +---- +git clone https://github.com/paritytech/substrate.git +cd substrate +---- + +Then build the code: + +[source, shell] +---- +cargo build # Builds all native code +---- + +You can run all the tests if you like: + +[source, shell] +cargo test --all + +Or just run the tests of a specific package (i.e. `cargo test -p paint-assets`) + +You can start a development chain with: + +[source, shell] +cargo run --release -- --dev + +Detailed logs may be shown by running the node with the following environment variables set: `RUST_LOG=debug RUST_BACKTRACE=1 cargo run --release \-- --dev`. + +If you want to see the multi-node consensus algorithm in action locally, then you can create a local testnet with two validator nodes for Alice and Bob, who are the initial authorities of the genesis chain specification that have been endowed with a testnet DOTs. We'll give each node a name and expose them so they are listed on link:https://telemetry.polkadot.io/#/Local%20Testnet[Telemetry]. You'll need two terminal windows open. + +We'll start Alice's Substrate node first on default TCP port 30333 with her chain database stored locally at `/tmp/alice`. The Bootnode ID of her node is `QmRpheLN4JWdAnY7HGJfWFNbfkQCb6tFf4vvA6hgjMZKrR`, which is generated from the `--node-key` value that we specify below: + +[source, shell] +cargo run --release \-- \ + --base-path /tmp/alice \ + --chain=local \ + --alice \ + --node-key 0000000000000000000000000000000000000000000000000000000000000001 \ + --telemetry-url ws://telemetry.polkadot.io:1024 \ + --validator + +In the second terminal, we'll run the following to start Bob's Substrate node on a different TCP port of 30334, and with his chain database stored locally at `/tmp/bob`. We'll specify a value for the `--bootnodes` option that will connect his node to Alice's Bootnode ID on TCP port 30333: + +[source, shell] +cargo run --release \-- \ + --base-path /tmp/bob \ + --bootnodes /ip4/127.0.0.1/tcp/30333/p2p/QmRpheLN4JWdAnY7HGJfWFNbfkQCb6tFf4vvA6hgjMZKrR \ + --chain=local \ + --bob \ + --port 30334 \ + --telemetry-url ws://telemetry.polkadot.io:1024 \ + --validator + +Additional Substrate CLI usage options are available and may be shown by running `cargo run \-- --help`. + +=== WASM binaries + +The WASM binaries are built during the normal `cargo build` process. To control the WASM binary building, +we support multiple environment variables: + +* `SKIP_WASM_BUILD` - Skips building any WASM binary. This is useful when only native should be recompiled. +* `BUILD_DUMMY_WASM_BINARY` - Builds dummy WASM binaries. These dummy binaries are empty and useful + for `cargo check` runs. +* `WASM_BUILD_TYPE` - Sets the build type for building WASM binaries. Supported values are `release` or `debug`. + By default the build type is equal to the build type used by the main build. +* `TRIGGER_WASM_BUILD` - Can be set to trigger a WASM build. On subsequent calls the value of the variable + needs to change. As WASM builder instructs `cargo` to watch for file changes + this environment variable should only be required in certain circumstances. +* `WASM_TARGET_DIRECTORY` - Will copy any build WASM binary to the given directory. The path needs + to be absolute. +* `WASM_BUILD_RUSTFLAGS` - Extend `RUSTFLAGS` given to `cargo build` while building the wasm binary. +* `WASM_BUILD_NO_COLOR` - Disable color output of the wasm build. + +Each project can be skipped individually by using the environment variable `SKIP_PROJECT_NAME_WASM_BUILD`. +Where `PROJECT_NAME` needs to be replaced by the name of the cargo project, e.g. `node-runtime` will +be `NODE_RUNTIME`. + +[[flaming-fir]] +=== Joining the Flaming Fir Testnet + +Flaming Fir is the new testnet for Substrate master (2.0) to test the latest development features. Please note that master is not compatible with the BBQ Birch, Charred Cherry, Dried Danta or Emberic Elm testnets. Ensure you have the dependencies listed above before compiling. + +Since Flaming Fir is targeting the master branch we make absolutely no guarantees of stability and/or persistence of the network. We might reset the chain at any time if it is necessary to deploy new changes. Currently, the validators are running with a client built from `d013bd900`, if you build from this commit you should be able to successfully sync, later commits may not work as new breaking changes may be introduced in master. + +Latest known working version: `a2a0eb5398d6223e531455b4c155ef053a4a3a2b` + +[source, shell] +---- +git clone https://github.com/paritytech/substrate.git +cd substrate +git checkout -b flaming-fir a2a0eb5398d6223e531455b4c155ef053a4a3a2b +---- + +You can run the tests if you like: + +[source, shell] +cargo test --all + +Start your node: + +[source, shell] +cargo run --release \-- + +To see a list of command line options, enter: + +[source, shell] +cargo run --release \-- --help + +For example, you can choose a custom node name: + +[source, shell] +cargo run --release \-- --name my_custom_name + +If you are successful, you will see your node syncing at https://telemetry.polkadot.io/#/Flaming%20Fir + +=== Joining the Emberic Elm Testnet + +Emberic Elm is the testnet for Substrate 1.0. Please note that 1.0 is not compatible with the BBQ Birch, Charred Cherry, Dried Danta or Flaming Fir testnets. +In order to join the Emberic Elm testnet you should build from the `v1.0` branch. Ensure you have the dependencies listed above before compiling. + +[source, shell] +---- +git clone https://github.com/paritytech/substrate.git +cd substrate +git checkout -b v1.0 origin/v1.0 +---- + +You can then follow the same steps for building and running as described above in <>. + +== Key management + +Keys in Substrate are stored in the keystore in the file system. To store keys into this keystore, +you need to use one of the two provided RPC calls. If your keys are encrypted or should be encrypted +by the keystore, you need to provide the key using one of the cli arguments `--password`, +`--password-interactive` or `--password-filename`. + +=== Recommended RPC call + +For most users who want to run a validator node, the `author_rotateKeys` RPC call is sufficient. +The RPC call will generate `N` Session keys for you and return their public keys. `N` is the number +of session keys configured in the runtime. The output of the RPC call can be used as input for the +`session::set_keys` transaction. + +``` +curl -H 'Content-Type: application/json' --data '{ "jsonrpc":"2.0", "method":"author_rotateKeys", "id":1 }' localhost:9933 +``` + +=== Advanced RPC call + +If the Session keys need to match a fixed seed, they can be set individually key by key. The RPC call +expects the key seed and the key type. The key types supported by default in Substrate are listed +https://github.com/paritytech/substrate/blob/master/core/primitives/src/crypto.rs#L767[here], but the +user can declare any key type. + +``` +curl -H 'Content-Type: application/json' --data '{ "jsonrpc":"2.0", "method":"author_insertKey", "params":["KEY_TYPE", "SEED", "PUBLIC"],"id":1 }' localhost:9933 +``` + +`KEY_TYPE` - needs to be replaced with the 4-character key type identifier. +`SEED` - is the seed of the key. +`PUBLIC` - public key for the given key. + +== Documentation + +=== Viewing documentation for Substrate packages + +You can generate documentation for a Substrate Rust package and have it automatically open in your web browser using https://doc.rust-lang.org/rustdoc/what-is-rustdoc.html#using-rustdoc-with-cargo[rustdoc with Cargo], +(of the The Rustdoc Book), by running the following command: + +``` +cargo doc --package --open +``` + +Replacing `` with one of the following (i.e. `cargo doc --package substrate --open`): + +* All Substrate Packages +[source, shell] +substrate +* Substrate Core +[source, shell] +substrate, substrate-cli, substrate-client, substrate-client-db, +substrate-consensus-common, substrate-consensus-rhd, +substrate-executor, substrate-finality-grandpa, substrate-keyring, substrate-keystore, substrate-network, +substrate-network-libp2p, substrate-primitives, substrate-rpc, substrate-rpc-servers, +substrate-serializer, substrate-service, substrate-service-test, substrate-state-db, +substrate-state-machine, substrate-telemetry, substrate-test-client, +substrate-test-runtime, substrate-transaction-graph, substrate-transaction-pool, +substrate-trie +* Substrate Runtime +[source, shell] +sr-api, sr-io, sr-primitives, sr-sandbox, sr-std, sr-version +* Substrate Runtime Module Library (SRML) +[source, shell] +paint-assets, paint-balances, paint-consensus, paint-contracts, paint-council, paint-democracy, paint-example, +paint-executive, paint-metadata, paint-session, paint-staking, paint-support, paint-system, paint-timestamp, +paint-treasury +* Node +[source, shell] +node-cli, node-consensus, node-executor, node-network, node-primitives, node-runtime +* Subkey +[source, shell] +subkey + +=== Contributing to documentation for Substrate packages + +https://doc.rust-lang.org/1.9.0/book/documentation.html[Document source code] for Substrate packages by annotating the source code with documentation comments. + +Example (generic): +```markdown +/// Summary +/// +/// Description +/// +/// # Panics +/// +/// # Errors +/// +/// # Safety +/// +/// # Examples +/// +/// Summary of Example 1 +/// +/// ```rust +/// // insert example 1 code here +/// ``` +/// +``` + +* Important notes: +** Documentation comments must use annotations with a triple slash `///` +** Modules are documented using `//!` +``` +//! Summary (of module) +//! +//! Description (of module) +``` +* Special section header is indicated with a hash `#`. +** `Panics` section requires an explanation if the function triggers a panic +** `Errors` section is for describing conditions under which a function of method returns `Err(E)` if it returns a `Result` +** `Safety` section requires an explanation if the function is `unsafe` +** `Examples` section includes examples of using the function or method +* Code block annotations for examples are included between triple graves, as shown above. +Instead of including the programming language to use for syntax highlighting as the annotation +after the triple graves, alternative annotations include the `ignore`, `text`, `should_panic`, or `no_run`. +* Summary sentence is a short high level single sentence of its functionality +* Description paragraph is for details additional to the summary sentence +* Missing documentation annotations may be used to identify where to generate warnings with `#![warn(missing_docs)]` +or errors `#![deny(missing_docs)]` +* Hide documentation for items with `#[doc(hidden)]` + +=== Contributing to documentation (tests, extended examples, macros) for Substrate packages + +The code block annotations in the `# Example` section may be used as https://doc.rust-lang.org/1.9.0/book/documentation.html#documentation-as-tests[documentation as tests and for extended examples]. + +* Important notes: +** Rustdoc will automatically add a `main()` wrapper around the code block to test it +** https://doc.rust-lang.org/1.9.0/book/documentation.html#documenting-macros[Documenting macros]. +** Documentation as tests examples are included when running `cargo test` + +== Contributing + +=== Contributing Guidelines + +include::CONTRIBUTING.adoc[] + +=== Contributor Code of Conduct + +include::CODE_OF_CONDUCT.adoc[] + +== License + +https://github.com/paritytech/substrate/blob/master/LICENSE[LICENSE] diff --git a/SECURITY.md b/docs/SECURITY.md similarity index 100% rename from SECURITY.md rename to docs/SECURITY.md diff --git a/docs/Structure.adoc b/docs/Structure.adoc new file mode 100644 index 0000000000000..0e9606e0a900a --- /dev/null +++ b/docs/Structure.adoc @@ -0,0 +1,110 @@ += Structure +:Author: Substrate developers +:Revision: 0.3.0 +:toc: +:sectnums: + + +== Overview + +Substrate is split into multiple levels with increasing opinion and decreasing flexibility: + +* primitives +* client +* PAINT (formerly `paint`) + +Putting all these components together we have: + +* Integration Tests +* Node +* Node template +* Subkey + +=== Runtime + +* _found in_: `/primitives` +* _crates prefix_: `sp-` +* _constrains_: +** must be `[no_std]` +** crates may not (dev-)depend on crates in other subfolders of this repo + +In the lowest level, substrate defines primitives, interfaces and traits to implement any on-chain substrate transition system and its interactions with the outside world. This is the lowest level of abstraction and opinion everything else builds upon. + +=== Client + +* _found in_: `/client` +* _crates prefix_: `substrate-` +* _constrains_: +** crates may not (dev-)depend on any `paint-`-crates + +In the client you can find a set of crates to construct the outer substrate-node, implementing outer runtime interfaces, thus it depends on `runtime`. It provides the outer building blocks like transaction queue, networking layer, database backend, full* and light-client support. + +=== PAINT (formerly `paint`) + +* _found in_: `/paint` +* _crates prefix_: `paint-` +* _constrains_: +** all crates that go on chain, must be `[no_std]` +** must not (dev-)depend on anything in `/client` + +PAINT, the Parity Application Interface for Network-wide Transitions, is a set of modules implementing specific transition functions and features one (might) want to have in their runtime. + +=== integration tests + +* _found in_: `/test` +* _crates prefix_: `substrate-test` +* _constrains_: +** only helpers may be published +** purely testing crates must be `publish = false` + +All tests that have to pull (dev)-dependencies out of their substree and would thus break the dependency rules, are considered intergration tests and should be stored in here. Only helper-crates in here shall be published, everything else is expected to be non-publish. + +=== Binaries and template + +* _found in_: `/bin` + +We also provide some binaries pulling from the components creating full applications. + +==== Node + +* _found in_: `/bin/node` + +The default (testing) application pulling together our recommended setup of substrate-client with a wasm-contracts-supporting paint-runtime. The node pulls it all together, constructs the (upgradable) runtime and wires up the client around it. You can find an example client, which includes a full wasm-contracts chain in `node`. This is also what is being build and run if you do `cargo run`. + + +==== Node Template + +* _found in_: `/bin/node-template` + +We also provide a template to get you started building your own node. + +==== Subkey + +* _found in_: `/bin/subkey` + +Subkey is a client library to generate keys and sign transactions to send to a substrate node. + +== Internal Dependency Tree + +[ditaa] +.... ++---------------+ +----------------+ +| | | | +| runtime +<------+ paint | +| | | | ++------+-----+--+ +-------------+--+ + ^ ^ ^ + | +----------------+ | + | | | ++------+--------+ | | +| | | | +| client | +--+-------+--------+ +| +<---------+ | ++---------------+ | | + | test /bin/* | + | | + | | + +-------------------+ + +.... + diff --git a/license_header.txt b/docs/license_header.txt similarity index 100% rename from license_header.txt rename to docs/license_header.txt diff --git a/node-template/Cargo.toml b/node-template/Cargo.toml deleted file mode 100644 index ba8c4caaf8562..0000000000000 --- a/node-template/Cargo.toml +++ /dev/null @@ -1,40 +0,0 @@ -[package] -name = "node-template" -version = "2.0.0" -authors = ["Anonymous"] -build = "build.rs" -edition = "2018" - -[[bin]] -name = "node-template" -path = "src/main.rs" - -[dependencies] -derive_more = "0.15.0" -futures = "0.1.29" -ctrlc = { version = "3.1.3", features = ["termination"] } -log = "0.4.8" -tokio = "0.1.22" -exit-future = "0.1.4" -parking_lot = "0.9.0" -codec = { package = "parity-scale-codec", version = "1.0.0" } -trie-root = "0.15.2" -sr-io = { path = "../core/sr-io" } -substrate-cli = { path = "../core/cli" } -primitives = { package = "substrate-primitives", path = "../core/primitives" } -substrate-executor = { path = "../core/executor" } -substrate-service = { path = "../core/service" } -inherents = { package = "substrate-inherents", path = "../core/inherents" } -transaction-pool = { package = "substrate-transaction-pool", path = "../core/transaction-pool" } -network = { package = "substrate-network", path = "../core/network" } -aura = { package = "substrate-consensus-aura", path = "../core/consensus/aura" } -aura-primitives = { package = "substrate-consensus-aura-primitives", path = "../core/consensus/aura/primitives" } -grandpa = { package = "substrate-finality-grandpa", path = "../core/finality-grandpa" } -grandpa-primitives = { package = "substrate-finality-grandpa-primitives", path = "../core/finality-grandpa/primitives" } -substrate-client = { path = "../core/client" } -basic-authorship = { package = "substrate-basic-authorship", path = "../core/basic-authorship" } -runtime = { package = "node-template-runtime", path = "runtime" } -sr-primitives = { path = "../core/sr-primitives" } - -[build-dependencies] -vergen = "3.0.4" diff --git a/node-template/build.rs b/node-template/build.rs deleted file mode 100644 index bab46f579d068..0000000000000 --- a/node-template/build.rs +++ /dev/null @@ -1,24 +0,0 @@ -use std::{env, path::PathBuf}; - -use vergen::{ConstantsFlags, generate_cargo_keys}; - -const ERROR_MSG: &str = "Failed to generate metadata files"; - -fn main() { - generate_cargo_keys(ConstantsFlags::SHA_SHORT).expect(ERROR_MSG); - - let mut manifest_dir = PathBuf::from( - env::var("CARGO_MANIFEST_DIR").expect("`CARGO_MANIFEST_DIR` is always set by cargo.") - ); - - while manifest_dir.parent().is_some() { - if manifest_dir.join(".git/HEAD").exists() { - println!("cargo:rerun-if-changed={}", manifest_dir.join(".git/HEAD").display()); - return - } - - manifest_dir.pop(); - } - - println!("cargo:warning=Could not find `.git/HEAD` from manifest dir!"); -} diff --git a/node-template/runtime/Cargo.toml b/node-template/runtime/Cargo.toml deleted file mode 100644 index ff2e3eb2b1676..0000000000000 --- a/node-template/runtime/Cargo.toml +++ /dev/null @@ -1,61 +0,0 @@ -[package] -name = "node-template-runtime" -version = "2.0.0" -authors = ["Anonymous"] -edition = "2018" - -[dependencies] -serde = { version = "1.0.101", optional = true, features = ["derive"] } -safe-mix = { version = "1.0.0", default-features = false } -codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false, features = ["derive"] } -rstd = { package = "sr-std", path = "../../core/sr-std", default_features = false } -runtime-io = { package = "sr-io", path = "../../core/sr-io", default_features = false } -version = { package = "sr-version", path = "../../core/sr-version", default_features = false } -support = { package = "srml-support", path = "../../srml/support", default_features = false } -primitives = { package = "substrate-primitives", path = "../../core/primitives", default_features = false } -substrate-session = { path = "../../core/session", default-features = false } -balances = { package = "srml-balances", path = "../../srml/balances", default_features = false } -aura = { package = "srml-aura", path = "../../srml/aura", default_features = false } -aura-primitives = { package = "substrate-consensus-aura-primitives", path = "../../core/consensus/aura/primitives", default_features = false } -grandpa = { package = "srml-grandpa", path = "../../srml/grandpa", default_features = false } -executive = { package = "srml-executive", path = "../../srml/executive", default_features = false } -indices = { package = "srml-indices", path = "../../srml/indices", default_features = false } -randomness-collective-flip = { package = "srml-randomness-collective-flip", path = "../../srml/randomness-collective-flip", default_features = false } -system = { package = "srml-system", path = "../../srml/system", default_features = false } -timestamp = { package = "srml-timestamp", path = "../../srml/timestamp", default_features = false } -sudo = { package = "srml-sudo", path = "../../srml/sudo", default_features = false } -transaction-payment = { package = "srml-transaction-payment", path = "../../srml/transaction-payment", default_features = false } -sr-primitives = { path = "../../core/sr-primitives", default_features = false } -client = { package = "substrate-client", path = "../../core/client", default_features = false } -offchain-primitives = { package = "substrate-offchain-primitives", path = "../../core/offchain/primitives", default-features = false } - -[build-dependencies] -wasm-builder-runner = { package = "substrate-wasm-builder-runner", version = "1.0.4" } - -[features] -default = ["std"] -std = [ - "codec/std", - "client/std", - "rstd/std", - "runtime-io/std", - "support/std", - "balances/std", - "aura/std", - "aura-primitives/std", - "grandpa/std", - "executive/std", - "indices/std", - "primitives/std", - "sr-primitives/std", - "randomness-collective-flip/std", - "system/std", - "timestamp/std", - "sudo/std", - "transaction-payment/std", - "version/std", - "serde", - "safe-mix/std", - "offchain-primitives/std", - "substrate-session/std", -] diff --git a/node/cli/Cargo.toml b/node/cli/Cargo.toml deleted file mode 100644 index b85014eeef06a..0000000000000 --- a/node/cli/Cargo.toml +++ /dev/null @@ -1,78 +0,0 @@ -[package] -name = "node-cli" -version = "2.0.0" -authors = ["Parity Technologies "] -description = "Substrate node implementation in Rust." -build = "build.rs" -edition = "2018" -default-run = "substrate" - -[badges] -travis-ci = { repository = "paritytech/substrate", branch = "master" } -maintenance = { status = "actively-developed" } -is-it-maintained-issue-resolution = { repository = "paritytech/substrate" } -is-it-maintained-open-issues = { repository = "paritytech/substrate" } - -[[bin]] -name = "substrate" -path = "bin/main.rs" - -[dependencies] -log = "0.4.8" -tokio = "0.1.22" -futures = "0.1.29" -exit-future = "0.1.4" -jsonrpc-core = "13.2.0" -cli = { package = "substrate-cli", path = "../../core/cli" } -codec = { package = "parity-scale-codec", version = "1.0.0" } -sr-io = { path = "../../core/sr-io" } -client = { package = "substrate-client", path = "../../core/client" } -primitives = { package = "substrate-primitives", path = "../../core/primitives" } -inherents = { package = "substrate-inherents", path = "../../core/inherents" } -node-runtime = { path = "../runtime" } -node-rpc = { path = "../rpc" } -node-primitives = { path = "../primitives" } -hex-literal = "0.2.1" -substrate-rpc = { package = "substrate-rpc", path = "../../core/rpc" } -substrate-basic-authorship = { path = "../../core/basic-authorship" } -substrate-service = { path = "../../core/service" } -chain-spec = { package = "substrate-chain-spec", path = "../../core/chain-spec" } -transaction_pool = { package = "substrate-transaction-pool", path = "../../core/transaction-pool" } -network = { package = "substrate-network", path = "../../core/network" } -babe = { package = "substrate-consensus-babe", path = "../../core/consensus/babe" } -babe-primitives = { package = "substrate-consensus-babe-primitives", path = "../../core/consensus/babe/primitives" } -grandpa = { package = "substrate-finality-grandpa", path = "../../core/finality-grandpa" } -grandpa_primitives = { package = "substrate-finality-grandpa-primitives", path = "../../core/finality-grandpa/primitives" } -sr-primitives = { path = "../../core/sr-primitives" } -node-executor = { path = "../executor" } -substrate-telemetry = { package = "substrate-telemetry", path = "../../core/telemetry" } -structopt = "0.3.3" -transaction-factory = { path = "../../test-utils/transaction-factory" } -keyring = { package = "substrate-keyring", path = "../../core/keyring" } -indices = { package = "srml-indices", path = "../../srml/indices" } -timestamp = { package = "srml-timestamp", path = "../../srml/timestamp", default-features = false } -rand = "0.7.2" -finality_tracker = { package = "srml-finality-tracker", path = "../../srml/finality-tracker", default-features = false } -contracts = { package = "srml-contracts", path = "../../srml/contracts" } -system = { package = "srml-system", path = "../../srml/system" } -balances = { package = "srml-balances", path = "../../srml/balances" } -transaction-payment = { package = "srml-transaction-payment", path = "../../srml/transaction-payment" } -support = { package = "srml-support", path = "../../srml/support", default-features = false } -im_online = { package = "srml-im-online", path = "../../srml/im-online", default-features = false } -serde = { version = "1.0.101", features = [ "derive" ] } -client_db = { package = "substrate-client-db", path = "../../core/client/db", features = ["kvdb-rocksdb"] } -offchain = { package = "substrate-offchain", path = "../../core/offchain" } -ctrlc = { version = "3.1.3", features = ["termination"] } - -[dev-dependencies] -keystore = { package = "substrate-keystore", path = "../../core/keystore" } -babe = { package = "substrate-consensus-babe", path = "../../core/consensus/babe", features = ["test-helpers"] } -consensus-common = { package = "substrate-consensus-common", path = "../../core/consensus/common" } -service-test = { package = "substrate-service-test", path = "../../core/service/test" } -futures03 = { package = "futures-preview", version = "0.3.0-alpha.19" } -tempfile = "3.1.0" - -[build-dependencies] -cli = { package = "substrate-cli", path = "../../core/cli" } -structopt = "0.3.3" -vergen = "3.0.4" diff --git a/node/executor/Cargo.toml b/node/executor/Cargo.toml deleted file mode 100644 index 1908443ca9519..0000000000000 --- a/node/executor/Cargo.toml +++ /dev/null @@ -1,37 +0,0 @@ -[package] -name = "node-executor" -version = "2.0.0" -authors = ["Parity Technologies "] -description = "Substrate node implementation in Rust." -edition = "2018" - -[dependencies] -trie-root = "0.15.2" -codec = { package = "parity-scale-codec", version = "1.0.0" } -runtime_io = { package = "sr-io", path = "../../core/sr-io" } -state_machine = { package = "substrate-state-machine", path = "../../core/state-machine" } -substrate-executor = { path = "../../core/executor" } -primitives = { package = "substrate-primitives", path = "../../core/primitives" } -trie = { package = "substrate-trie", path = "../../core/trie" } -node-primitives = { path = "../primitives" } -node-runtime = { path = "../runtime" } - -[dev-dependencies] -node-testing = { path = "../testing" } -test-client = { package = "substrate-test-client", path = "../../core/test-client" } -sr-primitives = { path = "../../core/sr-primitives" } -runtime_support = { package = "srml-support", path = "../../srml/support" } -balances = { package = "srml-balances", path = "../../srml/balances" } -transaction-payment = { package = "srml-transaction-payment", path = "../../srml/transaction-payment" } -session = { package = "srml-session", path = "../../srml/session" } -system = { package = "srml-system", path = "../../srml/system" } -timestamp = { package = "srml-timestamp", path = "../../srml/timestamp" } -treasury = { package = "srml-treasury", path = "../../srml/treasury" } -contracts = { package = "srml-contracts", path = "../../srml/contracts" } -grandpa = { package = "srml-grandpa", path = "../../srml/grandpa" } -indices = { package = "srml-indices", path = "../../srml/indices" } -wabt = "0.9.2" - -[features] -benchmarks = [] -stress-test = [] diff --git a/node/rpc/Cargo.toml b/node/rpc/Cargo.toml deleted file mode 100644 index 5e8b7614892ba..0000000000000 --- a/node/rpc/Cargo.toml +++ /dev/null @@ -1,16 +0,0 @@ -[package] -name = "node-rpc" -version = "2.0.0" -authors = ["Parity Technologies "] -edition = "2018" - -[dependencies] -client = { package = "substrate-client", path = "../../core/client" } -jsonrpc-core = "13.2.0" -node-primitives = { path = "../primitives" } -node-runtime = { path = "../runtime" } -sr-primitives = { path = "../../core/sr-primitives" } -srml-contracts-rpc = { path = "../../srml/contracts/rpc/" } -srml-transaction-payment-rpc = { path = "../../srml/transaction-payment/rpc/" } -srml-system-rpc = { path = "../../srml/system/rpc/" } -transaction_pool = { package = "substrate-transaction-pool", path = "../../core/transaction-pool" } diff --git a/node/runtime/Cargo.toml b/node/runtime/Cargo.toml deleted file mode 100644 index 0aa2dc551ef3a..0000000000000 --- a/node/runtime/Cargo.toml +++ /dev/null @@ -1,109 +0,0 @@ -[package] -name = "node-runtime" -version = "2.0.0" -authors = ["Parity Technologies "] -edition = "2018" -build = "build.rs" - -[dependencies] -codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false, features = ["derive"] } -integer-sqrt = { version = "0.1.2" } -rustc-hex = { version = "2.0", optional = true } -safe-mix = { version = "1.0", default-features = false } -serde = { version = "1.0.101", optional = true } - -babe-primitives = { package = "substrate-consensus-babe-primitives", path = "../../core/consensus/babe/primitives", default-features = false } -client = { package = "substrate-client", path = "../../core/client", default-features = false } -node-primitives = { path = "../primitives", default-features = false } -offchain-primitives = { package = "substrate-offchain-primitives", path = "../../core/offchain/primitives", default-features = false } -primitives = { package = "substrate-primitives", path = "../../core/primitives", default-features = false } -rstd = { package = "sr-std", path = "../../core/sr-std", default-features = false } -sr-primitives = { path = "../../core/sr-primitives", default-features = false } -sr-staking-primitives = { path = "../../core/sr-staking-primitives", default-features = false } -substrate-keyring = { path = "../../core/keyring", optional = true } -substrate-session = { path = "../../core/session", default-features = false } -version = { package = "sr-version", path = "../../core/sr-version", default-features = false } - -authorship = { package = "srml-authorship", path = "../../srml/authorship", default-features = false } -babe = { package = "srml-babe", path = "../../srml/babe", default-features = false } -balances = { package = "srml-balances", path = "../../srml/balances", default-features = false } -collective = { package = "srml-collective", path = "../../srml/collective", default-features = false } -contracts = { package = "srml-contracts", path = "../../srml/contracts", default-features = false } -contracts-rpc-runtime-api = { package = "srml-contracts-rpc-runtime-api", path = "../../srml/contracts/rpc/runtime-api/", default-features = false } -democracy = { package = "srml-democracy", path = "../../srml/democracy", default-features = false } -elections-phragmen = { package = "srml-elections-phragmen", path = "../../srml/elections-phragmen", default-features = false } -executive = { package = "srml-executive", path = "../../srml/executive", default-features = false } -finality-tracker = { package = "srml-finality-tracker", path = "../../srml/finality-tracker", default-features = false } -grandpa = { package = "srml-grandpa", path = "../../srml/grandpa", default-features = false } -im-online = { package = "srml-im-online", path = "../../srml/im-online", default-features = false } -indices = { package = "srml-indices", path = "../../srml/indices", default-features = false } -membership = { package = "srml-membership", path = "../../srml/membership", default-features = false } -nicks = { package = "srml-nicks", path = "../../srml/nicks", default-features = false } -offences = { package = "srml-offences", path = "../../srml/offences", default-features = false } -randomness-collective-flip = { package = "srml-randomness-collective-flip", path = "../../srml/randomness-collective-flip", default-features = false } -session = { package = "srml-session", path = "../../srml/session", default-features = false, features = ["historical"] } -staking = { package = "srml-staking", path = "../../srml/staking", default-features = false } -srml-staking-reward-curve = { path = "../../srml/staking/reward-curve"} -sudo = { package = "srml-sudo", path = "../../srml/sudo", default-features = false } -support = { package = "srml-support", path = "../../srml/support", default-features = false } -system = { package = "srml-system", path = "../../srml/system", default-features = false } -system-rpc-runtime-api = { package = "srml-system-rpc-runtime-api", path = "../../srml/system/rpc/runtime-api/", default-features = false } -timestamp = { package = "srml-timestamp", path = "../../srml/timestamp", default-features = false } -treasury = { package = "srml-treasury", path = "../../srml/treasury", default-features = false } -utility = { package = "srml-utility", path = "../../srml/utility", default-features = false } -transaction-payment = { package = "srml-transaction-payment", path = "../../srml/transaction-payment", default-features = false } -transaction-payment-rpc-runtime-api = { package = "srml-transaction-payment-rpc-runtime-api", path = "../../srml/transaction-payment/rpc/runtime-api/", default-features = false } - -[build-dependencies] -wasm-builder-runner = { package = "substrate-wasm-builder-runner", version = "1.0.4", path = "../../core/utils/wasm-builder-runner" } - -[dev-dependencies] -runtime_io = { package = "sr-io", path = "../../core/sr-io" } - -[features] -default = ["std"] -std = [ - "authorship/std", - "babe-primitives/std", - "babe/std", - "balances/std", - "client/std", - "codec/std", - "collective/std", - "contracts/std", - "contracts-rpc-runtime-api/std", - "democracy/std", - "elections-phragmen/std", - "executive/std", - "finality-tracker/std", - "grandpa/std", - "im-online/std", - "indices/std", - "membership/std", - "nicks/std", - "node-primitives/std", - "offchain-primitives/std", - "offences/std", - "primitives/std", - "randomness-collective-flip/std", - "rstd/std", - "rustc-hex", - "safe-mix/std", - "serde", - "session/std", - "sr-primitives/std", - "sr-staking-primitives/std", - "staking/std", - "substrate-keyring", - "substrate-session/std", - "sudo/std", - "support/std", - "system/std", - "system-rpc-runtime-api/std", - "timestamp/std", - "treasury/std", - "utility/std", - "transaction-payment/std", - "transaction-payment-rpc-runtime-api/std", - "version/std", -] diff --git a/node/testing/Cargo.toml b/node/testing/Cargo.toml deleted file mode 100644 index 8a4c08ed11bd2..0000000000000 --- a/node/testing/Cargo.toml +++ /dev/null @@ -1,31 +0,0 @@ -[package] -name = "node-testing" -version = "2.0.0" -authors = ["Parity Technologies "] -description = "Test utilities for Substrate node." -edition = "2018" - -[dependencies] -balances = { package = "srml-balances", path = "../../srml/balances" } -client = { package = "substrate-client", path = "../../core/client" } -contracts = { package = "srml-contracts", path = "../../srml/contracts" } -grandpa = { package = "srml-grandpa", path = "../../srml/grandpa" } -indices = { package = "srml-indices", path = "../../srml/indices" } -keyring = { package = "substrate-keyring", path = "../../core/keyring" } -node-executor = { path = "../executor" } -node-primitives = { path = "../primitives" } -node-runtime = { path = "../runtime" } -codec = { package = "parity-scale-codec", version = "1.0.0" } -primitives = { package = "substrate-primitives", path = "../../core/primitives" } -sr-io = { path = "../../core/sr-io" } -sr-primitives = { path = "../../core/sr-primitives" } -runtime_support = { package = "srml-support", path = "../../srml/support" } -session = { package = "srml-session", path = "../../srml/session" } -staking = { package = "srml-staking", path = "../../srml/staking" } -substrate-executor = { path = "../../core/executor" } -system = { package = "srml-system", path = "../../srml/system" } -test-client = { package = "substrate-test-client", path = "../../core/test-client" } -timestamp = { package = "srml-timestamp", path = "../../srml/timestamp" } -treasury = { package = "srml-treasury", path = "../../srml/treasury" } -transaction-payment = { package = "srml-transaction-payment", path = "../../srml/transaction-payment" } -wabt = "0.9.2" diff --git a/srml/assets/Cargo.toml b/paint/assets/Cargo.toml similarity index 62% rename from srml/assets/Cargo.toml rename to paint/assets/Cargo.toml index 052a3045b2164..d8fa259484dbc 100644 --- a/srml/assets/Cargo.toml +++ b/paint/assets/Cargo.toml @@ -1,5 +1,5 @@ [package] -name = "srml-assets" +name = "paint-assets" version = "2.0.0" authors = ["Parity Technologies "] edition = "2018" @@ -8,16 +8,16 @@ edition = "2018" serde = { version = "1.0.101", optional = true } codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false } # Needed for various traits. In our case, `OnFinalize`. -sr-primitives = { path = "../../core/sr-primitives", default-features = false } +sr-primitives = { path = "../../primitives/sr-primitives", default-features = false } # Needed for type-safe access to storage DB. -support = { package = "srml-support", path = "../support", default-features = false } +support = { package = "paint-support", path = "../support", default-features = false } # `system` module provides us with all sorts of useful stuff and macros depend on it being around. -system = { package = "srml-system", path = "../system", default-features = false } +system = { package = "paint-system", path = "../system", default-features = false } [dev-dependencies] -primitives = { package = "substrate-primitives", path = "../../core/primitives" } -rstd = { package = "sr-std", path = "../../core/sr-std" } -runtime-io = { package = "sr-io", path = "../../core/sr-io" } +primitives = { package = "substrate-primitives", path = "../../primitives/core" } +rstd = { package = "sr-std", path = "../../primitives/sr-std" } +runtime-io = { package = "sr-io", path = "../../primitives/sr-io" } [features] default = ["std"] diff --git a/srml/assets/src/lib.rs b/paint/assets/src/lib.rs similarity index 99% rename from srml/assets/src/lib.rs rename to paint/assets/src/lib.rs index 5c8b1bbd610a3..4f91cd19198a4 100644 --- a/srml/assets/src/lib.rs +++ b/paint/assets/src/lib.rs @@ -124,8 +124,8 @@ //! //! ## Related Modules //! -//! * [`System`](../srml_system/index.html) -//! * [`Support`](../srml_support/index.html) +//! * [`System`](../paint_system/index.html) +//! * [`Support`](../paint_support/index.html) // Ensure we're `no_std` when compiling for Wasm. #![cfg_attr(not(feature = "std"), no_std)] diff --git a/paint/aura/Cargo.toml b/paint/aura/Cargo.toml new file mode 100644 index 0000000000000..23250ac312ee6 --- /dev/null +++ b/paint/aura/Cargo.toml @@ -0,0 +1,41 @@ +[package] +name = "paint-aura" +version = "2.0.0" +authors = ["Parity Technologies "] +edition = "2018" + +[dependencies] +app-crypto = { package = "substrate-application-crypto", path = "../../primitives/application-crypto", default-features = false } +codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false, features = ["derive"] } +inherents = { package = "substrate-inherents", path = "../../primitives/inherents", default-features = false } +primitives = { package = "substrate-primitives", path = "../../primitives/core", default-features = false } +rstd = { package = "sr-std", path = "../../primitives/sr-std", default-features = false } +serde = { version = "1.0.101", optional = true } +session = { package = "paint-session", path = "../session", default-features = false } +sr-primitives = { path = "../../primitives/sr-primitives", default-features = false } +runtime-io ={ package = "sr-io", path = "../../primitives/sr-io", default-features = false } +support = { package = "paint-support", path = "../support", default-features = false } +substrate-consensus-aura-primitives = { path = "../../primitives/consensus/aura", default-features = false} +system = { package = "paint-system", path = "../system", default-features = false } +timestamp = { package = "paint-timestamp", path = "../timestamp", default-features = false } + +[dev-dependencies] +lazy_static = "1.4.0" +parking_lot = "0.9.0" + +[features] +default = ["std"] +std = [ + "app-crypto/std", + "codec/std", + "inherents/std", + "runtime-io/std", + "primitives/std", + "rstd/std", + "serde", + "sr-primitives/std", + "support/std", + "substrate-consensus-aura-primitives/std", + "system/std", + "timestamp/std", +] diff --git a/srml/aura/src/lib.rs b/paint/aura/src/lib.rs similarity index 90% rename from srml/aura/src/lib.rs rename to paint/aura/src/lib.rs index f27cb98c0131d..4b14c5b6485fa 100644 --- a/srml/aura/src/lib.rs +++ b/paint/aura/src/lib.rs @@ -31,15 +31,15 @@ //! //! ## Related Modules //! -//! - [Timestamp](../srml_timestamp/index.html): The Timestamp module is used in Aura to track +//! - [Timestamp](../paint_timestamp/index.html): The Timestamp module is used in Aura to track //! consensus rounds (via `slots`). -//! - [Consensus](../srml_consensus/index.html): The Consensus module does not relate directly to Aura, +//! - [Consensus](../paint_consensus/index.html): The Consensus module does not relate directly to Aura, //! but serves to manage offline reporting by implementing `ProvideInherent` in a similar way. //! //! ## References //! //! If you're interested in hacking on this module, it is useful to understand the interaction with -//! `substrate/core/inherents/src/lib.rs` and, specifically, the required implementation of +//! `substrate/primitives/inherents/src/lib.rs` and, specifically, the required implementation of //! [`ProvideInherent`](../substrate_inherents/trait.ProvideInherent.html) and //! [`ProvideInherentData`](../substrate_inherents/trait.ProvideInherentData.html) to create and check inherents. @@ -60,7 +60,7 @@ use sr_primitives::{ use timestamp::OnTimestampSet; #[cfg(feature = "std")] use timestamp::TimestampInherentData; -use inherents::{RuntimeString, InherentIdentifier, InherentData, ProvideInherent, MakeFatalError}; +use inherents::{InherentIdentifier, InherentData, ProvideInherent, MakeFatalError}; #[cfg(feature = "std")] use inherents::{InherentDataProviders, ProvideInherentData}; use substrate_consensus_aura_primitives::{AURA_ENGINE_ID, ConsensusLog, AuthorityIndex}; @@ -77,13 +77,13 @@ pub type InherentType = u64; /// Auxiliary trait to extract Aura inherent data. pub trait AuraInherentData { /// Get aura inherent data. - fn aura_inherent_data(&self) -> result::Result; + fn aura_inherent_data(&self) -> result::Result; /// Replace aura inherent data. fn aura_replace_inherent_data(&mut self, new: InherentType); } impl AuraInherentData for InherentData { - fn aura_inherent_data(&self) -> result::Result { + fn aura_inherent_data(&self) -> result::Result { self.get_data(&INHERENT_IDENTIFIER) .and_then(|r| r.ok_or_else(|| "Aura inherent data not found".into())) } @@ -113,7 +113,7 @@ impl ProvideInherentData for InherentDataProvider { fn on_register( &self, providers: &InherentDataProviders, - ) -> result::Result<(), RuntimeString> { + ) -> result::Result<(), inherents::Error> { if !providers.has_provider(×tamp::INHERENT_IDENTIFIER) { // Add the timestamp inherent data provider, as we require it. providers.register_provider(timestamp::InherentDataProvider) @@ -129,14 +129,14 @@ impl ProvideInherentData for InherentDataProvider { fn provide_inherent_data( &self, inherent_data: &mut InherentData, - ) -> result::Result<(), RuntimeString> { + ) -> result::Result<(), inherents::Error> { let timestamp = inherent_data.timestamp_inherent_data()?; let slot_num = timestamp / self.slot_duration; inherent_data.put_data(INHERENT_IDENTIFIER, &slot_num) } fn error_to_string(&self, error: &[u8]) -> Option { - RuntimeString::decode(&mut &error[..]).map(Into::into).ok() + inherents::Error::decode(&mut &error[..]).map(|e| e.into_string()).ok() } } @@ -279,7 +279,7 @@ impl OnTimestampSet for Module { impl ProvideInherent for Module { type Call = timestamp::Call; - type Error = MakeFatalError; + type Error = MakeFatalError; const INHERENT_IDENTIFIER: InherentIdentifier = INHERENT_IDENTIFIER; fn create_inherent(_: &InherentData) -> Option { @@ -300,7 +300,7 @@ impl ProvideInherent for Module { if timestamp_based_slot == seal_slot { Ok(()) } else { - Err(RuntimeString::from("timestamp set in block doesn't match slot in seal").into()) + Err(inherents::Error::from("timestamp set in block doesn't match slot in seal").into()) } } } diff --git a/srml/aura/src/mock.rs b/paint/aura/src/mock.rs similarity index 100% rename from srml/aura/src/mock.rs rename to paint/aura/src/mock.rs diff --git a/srml/aura/src/tests.rs b/paint/aura/src/tests.rs similarity index 100% rename from srml/aura/src/tests.rs rename to paint/aura/src/tests.rs diff --git a/paint/authority-discovery/Cargo.toml b/paint/authority-discovery/Cargo.toml new file mode 100644 index 0000000000000..923017d9a4b41 --- /dev/null +++ b/paint/authority-discovery/Cargo.toml @@ -0,0 +1,37 @@ +[package] +name = "paint-authority-discovery" +version = "0.1.0" +authors = ["Parity Technologies "] +edition = "2018" + +[dependencies] +authority-discovery-primitives = { package = "substrate-authority-discovery-primitives", path = "../../primitives/authority-discovery", default-features = false } +app-crypto = { package = "substrate-application-crypto", path = "../../primitives/application-crypto", default-features = false } +codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false, features = ["derive"] } +primitives = { package = "substrate-primitives", path = "../../primitives/core", default-features = false } +rstd = { package = "sr-std", path = "../../primitives/sr-std", default-features = false } +serde = { version = "1.0.101", optional = true } +runtime-io = { package = "sr-io", path = "../../primitives/sr-io", default-features = false } +session = { package = "paint-session", path = "../session", default-features = false } +sr-primitives = { path = "../../primitives/sr-primitives", default-features = false } +support = { package = "paint-support", path = "../support", default-features = false } +system = { package = "paint-system", path = "../system", default-features = false } + +[dev-dependencies] +sr-staking-primitives = { path = "../../primitives/sr-staking-primitives", default-features = false } + +[features] +default = ["std"] +std = [ + "app-crypto/std", + "authority-discovery-primitives/std", + "codec/std", + "primitives/std", + "rstd/std", + "runtime-io/std", + "serde", + "session/std", + "sr-primitives/std", + "support/std", + "system/std", +] diff --git a/paint/authority-discovery/src/lib.rs b/paint/authority-discovery/src/lib.rs new file mode 100644 index 0000000000000..9bae70a797942 --- /dev/null +++ b/paint/authority-discovery/src/lib.rs @@ -0,0 +1,266 @@ +// Copyright 2019 Parity Technologies (UK) Ltd. +// This file is part of Substrate. + +// Substrate is free software: you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. + +// Substrate is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. + +// You should have received a copy of the GNU General Public License +// along with Substrate. If not, see . + +//! # Authority discovery module. +//! +//! This module is used by the `core/authority-discovery` to retrieve the +//! current set of authorities. + +// Ensure we're `no_std` when compiling for Wasm. +#![cfg_attr(not(feature = "std"), no_std)] + +use rstd::prelude::*; +use support::{decl_module, decl_storage}; +use authority_discovery_primitives::AuthorityId; + +/// The module's config trait. +pub trait Trait: system::Trait + session::Trait {} + +decl_storage! { + trait Store for Module as AuthorityDiscovery { + /// Keys of the current authority set. + Keys get(fn keys): Vec; + } + add_extra_genesis { + config(keys): Vec; + build(|config| Module::::initialize_keys(&config.keys)) + } +} + +decl_module! { + pub struct Module for enum Call where origin: T::Origin { + } +} + +impl Module { + /// Retrieve authority identifiers of the current authority set. + pub fn authorities() -> Vec { + Keys::get() + } + + fn initialize_keys(keys: &[AuthorityId]) { + if !keys.is_empty() { + assert!(Keys::get().is_empty(), "Keys are already initialized!"); + Keys::put(keys); + } + } +} + +impl sr_primitives::BoundToRuntimeAppPublic for Module { + type Public = AuthorityId; +} + +impl session::OneSessionHandler for Module { + type Key = AuthorityId; + + fn on_genesis_session<'a, I: 'a>(authorities: I) + where + I: Iterator, + { + let keys = authorities.map(|x| x.1).collect::>(); + Self::initialize_keys(&keys); + } + + fn on_new_session<'a, I: 'a>(changed: bool, validators: I, _queued_validators: I) + where + I: Iterator, + { + // Remember who the authorities are for the new session. + if changed { + Keys::put(validators.map(|x| x.1).collect::>()); + } + } + + fn on_disabled(_i: usize) { + // ignore + } +} + +#[cfg(test)] +mod tests { + use super::*; + use authority_discovery_primitives::{AuthorityPair}; + use app_crypto::Pair; + use primitives::{crypto::key_types, H256}; + use runtime_io::TestExternalities; + use sr_primitives::{ + testing::{Header, UintAuthorityId}, traits::{ConvertInto, IdentityLookup, OpaqueKeys}, + Perbill, KeyTypeId, + }; + use support::{impl_outer_origin, parameter_types}; + + type AuthorityDiscovery = Module; + type SessionIndex = u32; + + #[derive(Clone, Eq, PartialEq)] + pub struct Test; + impl Trait for Test {} + + pub struct TestOnSessionEnding; + impl session::OnSessionEnding for TestOnSessionEnding { + fn on_session_ending(_: SessionIndex, _: SessionIndex) -> Option> { + None + } + } + + parameter_types! { + pub const DisabledValidatorsThreshold: Perbill = Perbill::from_percent(33); + } + + impl session::Trait for Test { + type OnSessionEnding = TestOnSessionEnding; + type Keys = UintAuthorityId; + type ShouldEndSession = session::PeriodicSessions; + type SessionHandler = TestSessionHandler; + type Event = (); + type ValidatorId = AuthorityId; + type ValidatorIdOf = ConvertInto; + type SelectInitialValidators = (); + type DisabledValidatorsThreshold = DisabledValidatorsThreshold; + } + + impl session::historical::Trait for Test { + type FullIdentification = (); + type FullIdentificationOf = (); + } + + pub type BlockNumber = u64; + + parameter_types! { + pub const Period: BlockNumber = 1; + pub const Offset: BlockNumber = 0; + pub const UncleGenerations: u64 = 0; + pub const BlockHashCount: u64 = 250; + pub const MaximumBlockWeight: u32 = 1024; + pub const MaximumBlockLength: u32 = 2 * 1024; + pub const AvailableBlockRatio: Perbill = Perbill::one(); + } + + impl system::Trait for Test { + type Origin = Origin; + type Index = u64; + type BlockNumber = BlockNumber; + type Call = (); + type Hash = H256; + type Hashing = ::sr_primitives::traits::BlakeTwo256; + type AccountId = AuthorityId; + type Lookup = IdentityLookup; + type Header = Header; + type Event = (); + type BlockHashCount = BlockHashCount; + type MaximumBlockWeight = MaximumBlockWeight; + type AvailableBlockRatio = AvailableBlockRatio; + type MaximumBlockLength = MaximumBlockLength; + type Version = (); + } + + impl_outer_origin! { + pub enum Origin for Test {} + } + + pub struct TestSessionHandler; + impl session::SessionHandler for TestSessionHandler { + const KEY_TYPE_IDS: &'static [KeyTypeId] = &[key_types::DUMMY]; + + fn on_new_session( + _changed: bool, + _validators: &[(AuthorityId, Ks)], + _queued_validators: &[(AuthorityId, Ks)], + ) { + } + + fn on_disabled(_validator_index: usize) {} + + fn on_genesis_session(_validators: &[(AuthorityId, Ks)]) {} + } + + #[test] + fn authorities_returns_current_authority_set() { + // The whole authority discovery module ignores account ids, but we still need it for + // `session::OneSessionHandler::on_new_session`, thus its safe to use the same value everywhere. + let account_id = AuthorityPair::from_seed_slice(vec![10; 32].as_ref()).unwrap().public(); + + let first_authorities: Vec = vec![0, 1].into_iter() + .map(|i| AuthorityPair::from_seed_slice(vec![i; 32].as_ref()).unwrap().public()) + .map(AuthorityId::from) + .collect(); + + // Needed for `session::OneSessionHandler::on_new_session`. + let first_authorities_and_account_ids: Vec<(&AuthorityId, AuthorityId)> = first_authorities.clone() + .into_iter() + .map(|id| (&account_id, id)) + .collect(); + + let second_authorities: Vec = vec![2, 3].into_iter() + .map(|i| AuthorityPair::from_seed_slice(vec![i; 32].as_ref()).unwrap().public()) + .map(AuthorityId::from) + .collect(); + + // Needed for `session::OneSessionHandler::on_new_session`. + let second_authorities_and_account_ids: Vec<(&AuthorityId, AuthorityId)> = second_authorities.clone() + .into_iter() + .map(|id| (&account_id, id)) + .collect(); + + // Build genesis. + let mut t = system::GenesisConfig::default() + .build_storage::() + .unwrap(); + + GenesisConfig { + keys: vec![], + } + .assimilate_storage::(&mut t) + .unwrap(); + + // Create externalities. + let mut externalities = TestExternalities::new(t); + + externalities.execute_with(|| { + use session::OneSessionHandler; + + AuthorityDiscovery::on_genesis_session( + first_authorities.iter().map(|id| (id, id.clone())) + ); + assert_eq!( + first_authorities, + AuthorityDiscovery::authorities() + ); + + // When `changed` set to false, the authority set should not be updated. + AuthorityDiscovery::on_new_session( + false, + second_authorities_and_account_ids.clone().into_iter(), + vec![].into_iter(), + ); + assert_eq!( + first_authorities, + AuthorityDiscovery::authorities() + ); + + // When `changed` set to true, the authority set should be updated. + AuthorityDiscovery::on_new_session( + true, + second_authorities_and_account_ids.into_iter(), + vec![].into_iter(), + ); + assert_eq!( + second_authorities, + AuthorityDiscovery::authorities() + ); + }); + } +} diff --git a/srml/authorship/Cargo.toml b/paint/authorship/Cargo.toml similarity index 50% rename from srml/authorship/Cargo.toml rename to paint/authorship/Cargo.toml index e860be2f64484..b53c059c3a01c 100644 --- a/srml/authorship/Cargo.toml +++ b/paint/authorship/Cargo.toml @@ -1,20 +1,20 @@ [package] -name = "srml-authorship" +name = "paint-authorship" version = "0.1.0" description = "Block and Uncle Author tracking for the SRML" authors = ["Parity Technologies "] edition = "2018" [dependencies] -primitives = { package = "substrate-primitives", path = "../../core/primitives", default-features = false } +primitives = { package = "substrate-primitives", path = "../../primitives/core", default-features = false } codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false, features = ["derive"] } -inherents = { package = "substrate-inherents", path = "../../core/inherents", default-features = false } -rstd = { package = "sr-std", path = "../../core/sr-std", default-features = false } -sr-primitives = { path = "../../core/sr-primitives", default-features = false } -support = { package = "srml-support", path = "../support", default-features = false } -system = { package = "srml-system", path = "../system", default-features = false } -runtime-io ={ package = "sr-io", path = "../../core/sr-io", default-features = false } -impl-trait-for-tuples = "0.1.2" +inherents = { package = "substrate-inherents", path = "../../primitives/inherents", default-features = false } +rstd = { package = "sr-std", path = "../../primitives/sr-std", default-features = false } +sr-primitives = { path = "../../primitives/sr-primitives", default-features = false } +support = { package = "paint-support", path = "../support", default-features = false } +system = { package = "paint-system", path = "../system", default-features = false } +runtime-io ={ package = "sr-io", path = "../../primitives/sr-io", default-features = false } +impl-trait-for-tuples = "0.1.3" [features] default = ["std"] diff --git a/srml/authorship/src/lib.rs b/paint/authorship/src/lib.rs similarity index 98% rename from srml/authorship/src/lib.rs rename to paint/authorship/src/lib.rs index 173af5f729cbb..795b0f7940e11 100644 --- a/srml/authorship/src/lib.rs +++ b/paint/authorship/src/lib.rs @@ -29,10 +29,7 @@ use codec::{Encode, Decode}; use system::ensure_none; use sr_primitives::traits::{Header as HeaderT, One, Zero}; use sr_primitives::weights::SimpleDispatchInfo; -use inherents::{ - RuntimeString, InherentIdentifier, ProvideInherent, - InherentData, MakeFatalError, -}; +use inherents::{InherentIdentifier, ProvideInherent, InherentData, MakeFatalError}; /// The identifier for the `uncles` inherent. pub const INHERENT_IDENTIFIER: InherentIdentifier = *b"uncles00"; @@ -40,11 +37,11 @@ pub const INHERENT_IDENTIFIER: InherentIdentifier = *b"uncles00"; /// Auxiliary trait to extract uncles inherent data. pub trait UnclesInherentData { /// Get uncles. - fn uncles(&self) -> Result, RuntimeString>; + fn uncles(&self) -> Result, inherents::Error>; } impl UnclesInherentData for InherentData { - fn uncles(&self) -> Result, RuntimeString> { + fn uncles(&self) -> Result, inherents::Error> { Ok(self.get_data(&INHERENT_IDENTIFIER)?.unwrap_or_default()) } } @@ -71,7 +68,7 @@ where F: Fn() -> Vec &INHERENT_IDENTIFIER } - fn provide_inherent_data(&self, inherent_data: &mut InherentData) -> Result<(), RuntimeString> { + fn provide_inherent_data(&self, inherent_data: &mut InherentData) -> Result<(), inherents::Error> { let uncles = (self.inner)(); if !uncles.is_empty() { inherent_data.put_data(INHERENT_IDENTIFIER, &uncles) diff --git a/paint/babe/Cargo.toml b/paint/babe/Cargo.toml new file mode 100644 index 0000000000000..cdbee72216dde --- /dev/null +++ b/paint/babe/Cargo.toml @@ -0,0 +1,44 @@ +[package] +name = "paint-babe" +version = "2.0.0" +authors = ["Parity Technologies "] +edition = "2018" + +[dependencies] +hex-literal = "0.2.1" +codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false, features = ["derive"] } +serde = { version = "1.0.101", optional = true } +inherents = { package = "substrate-inherents", path = "../../primitives/inherents", default-features = false } +rstd = { package = "sr-std", path = "../../primitives/sr-std", default-features = false } +sr-primitives = { path = "../../primitives/sr-primitives", default-features = false } +sr-staking-primitives = { path = "../../primitives/sr-staking-primitives", default-features = false } +support = { package = "paint-support", path = "../support", default-features = false } +system = { package = "paint-system", path = "../system", default-features = false } +timestamp = { package = "paint-timestamp", path = "../timestamp", default-features = false } +session = { package = "paint-session", path = "../session", default-features = false } +babe-primitives = { package = "substrate-consensus-babe-primitives", path = "../../primitives/consensus/babe", default-features = false } +runtime-io ={ package = "sr-io", path = "../../primitives/sr-io", default-features = false } + +[dev-dependencies] +lazy_static = "1.4.0" +parking_lot = "0.9.0" +sr-version = { path = "../../primitives/sr-version", default-features = false } +primitives = { package = "substrate-primitives", path = "../../primitives/core" } +test-runtime = { package = "substrate-test-runtime", path = "../../test/utils/runtime" } + +[features] +default = ["std"] +std = [ + "serde", + "codec/std", + "rstd/std", + "support/std", + "sr-primitives/std", + "sr-staking-primitives/std", + "system/std", + "timestamp/std", + "inherents/std", + "babe-primitives/std", + "session/std", + "runtime-io/std", +] diff --git a/srml/babe/src/lib.rs b/paint/babe/src/lib.rs similarity index 96% rename from srml/babe/src/lib.rs rename to paint/babe/src/lib.rs index e12e123daf6f2..5bdae6ce9b1b0 100644 --- a/srml/babe/src/lib.rs +++ b/paint/babe/src/lib.rs @@ -34,7 +34,7 @@ use sr_staking_primitives::{ #[cfg(feature = "std")] use timestamp::TimestampInherentData; use codec::{Encode, Decode}; -use inherents::{RuntimeString, InherentIdentifier, InherentData, ProvideInherent, MakeFatalError}; +use inherents::{InherentIdentifier, InherentData, ProvideInherent, MakeFatalError}; #[cfg(feature = "std")] use inherents::{InherentDataProviders, ProvideInherentData}; use babe_primitives::{ @@ -57,13 +57,13 @@ pub type InherentType = u64; /// Auxiliary trait to extract BABE inherent data. pub trait BabeInherentData { /// Get BABE inherent data. - fn babe_inherent_data(&self) -> result::Result; + fn babe_inherent_data(&self) -> result::Result; /// Replace BABE inherent data. fn babe_replace_inherent_data(&mut self, new: InherentType); } impl BabeInherentData for InherentData { - fn babe_inherent_data(&self) -> result::Result { + fn babe_inherent_data(&self) -> result::Result { self.get_data(&INHERENT_IDENTIFIER) .and_then(|r| r.ok_or_else(|| "BABE inherent data not found".into())) } @@ -94,7 +94,7 @@ impl ProvideInherentData for InherentDataProvider { fn on_register( &self, providers: &InherentDataProviders, - ) -> result::Result<(), RuntimeString> { + ) -> result::Result<(), inherents::Error> { if !providers.has_provider(×tamp::INHERENT_IDENTIFIER) { // Add the timestamp inherent data provider, as we require it. providers.register_provider(timestamp::InherentDataProvider) @@ -110,14 +110,14 @@ impl ProvideInherentData for InherentDataProvider { fn provide_inherent_data( &self, inherent_data: &mut InherentData, - ) -> result::Result<(), RuntimeString> { + ) -> result::Result<(), inherents::Error> { let timestamp = inherent_data.timestamp_inherent_data()?; let slot_number = timestamp / self.slot_duration; inherent_data.put_data(INHERENT_IDENTIFIER, &slot_number) } fn error_to_string(&self, error: &[u8]) -> Option { - RuntimeString::decode(&mut &error[..]).map(Into::into).ok() + inherents::Error::decode(&mut &error[..]).map(|e| e.into_string()).ok() } } @@ -148,7 +148,7 @@ pub trait EpochChangeTrigger { } /// A type signifying to BABE that an external trigger -/// for epoch changes (e.g. srml-session) is used. +/// for epoch changes (e.g. paint-session) is used. pub struct ExternalTrigger; impl EpochChangeTrigger for ExternalTrigger { @@ -390,7 +390,7 @@ impl Module { /// and the caller is the only caller of this function. /// /// Typically, this is not handled directly by the user, but by higher-level validator-set manager logic like - /// `srml-session`. + /// `paint-session`. pub fn enact_epoch_change( authorities: Vec<(AuthorityId, BabeAuthorityWeight)>, next_authorities: Vec<(AuthorityId, BabeAuthorityWeight)>, @@ -597,12 +597,12 @@ fn compute_randomness( s.extend_from_slice(&vrf_output[..]); } - runtime_io::blake2_256(&s) + runtime_io::hashing::blake2_256(&s) } impl ProvideInherent for Module { type Call = timestamp::Call; - type Error = MakeFatalError; + type Error = MakeFatalError; const INHERENT_IDENTIFIER: InherentIdentifier = INHERENT_IDENTIFIER; fn create_inherent(_: &InherentData) -> Option { @@ -621,7 +621,7 @@ impl ProvideInherent for Module { if timestamp_based_slot == seal_slot { Ok(()) } else { - Err(RuntimeString::from("timestamp set in block doesn't match slot in seal").into()) + Err(inherents::Error::from("timestamp set in block doesn't match slot in seal").into()) } } } diff --git a/srml/babe/src/mock.rs b/paint/babe/src/mock.rs similarity index 100% rename from srml/babe/src/mock.rs rename to paint/babe/src/mock.rs diff --git a/srml/babe/src/tests.rs b/paint/babe/src/tests.rs similarity index 100% rename from srml/babe/src/tests.rs rename to paint/babe/src/tests.rs diff --git a/srml/balances/Cargo.toml b/paint/balances/Cargo.toml similarity index 51% rename from srml/balances/Cargo.toml rename to paint/balances/Cargo.toml index f7f6041c6bb16..be00ed30c7660 100644 --- a/srml/balances/Cargo.toml +++ b/paint/balances/Cargo.toml @@ -1,5 +1,5 @@ [package] -name = "srml-balances" +name = "paint-balances" version = "2.0.0" authors = ["Parity Technologies "] edition = "2018" @@ -8,16 +8,16 @@ edition = "2018" serde = { version = "1.0.101", optional = true } safe-mix = { version = "1.0.0", default-features = false } codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false, features = ["derive"] } -substrate-keyring = { path = "../../core/keyring", optional = true } -rstd = { package = "sr-std", path = "../../core/sr-std", default-features = false } -sr-primitives = { path = "../../core/sr-primitives", default-features = false } -support = { package = "srml-support", path = "../support", default-features = false } -system = { package = "srml-system", path = "../system", default-features = false } +substrate-keyring = { path = "../../primitives/keyring", optional = true } +rstd = { package = "sr-std", path = "../../primitives/sr-std", default-features = false } +sr-primitives = { path = "../../primitives/sr-primitives", default-features = false } +support = { package = "paint-support", path = "../support", default-features = false } +system = { package = "paint-system", path = "../system", default-features = false } [dev-dependencies] -runtime-io = { package = "sr-io", path = "../../core/sr-io" } -primitives = { package = "substrate-primitives", path = "../../core/primitives" } -transaction-payment = { package = "srml-transaction-payment", path = "../transaction-payment" } +runtime-io = { package = "sr-io", path = "../../primitives/sr-io" } +primitives = { package = "substrate-primitives", path = "../../primitives/core" } +transaction-payment = { package = "paint-transaction-payment", path = "../transaction-payment" } [features] default = ["std"] diff --git a/srml/balances/src/lib.rs b/paint/balances/src/lib.rs similarity index 95% rename from srml/balances/src/lib.rs rename to paint/balances/src/lib.rs index 71f37cb8f8193..37869345c2ca4 100644 --- a/srml/balances/src/lib.rs +++ b/paint/balances/src/lib.rs @@ -63,16 +63,16 @@ //! The Balances module provides implementations for the following traits. If these traits provide the functionality //! that you need, then you can avoid coupling with the Balances module. //! -//! - [`Currency`](../srml_support/traits/trait.Currency.html): Functions for dealing with a +//! - [`Currency`](../paint_support/traits/trait.Currency.html): Functions for dealing with a //! fungible assets system. -//! - [`ReservableCurrency`](../srml_support/traits/trait.ReservableCurrency.html): +//! - [`ReservableCurrency`](../paint_support/traits/trait.ReservableCurrency.html): //! Functions for dealing with assets that can be reserved from an account. -//! - [`LockableCurrency`](../srml_support/traits/trait.LockableCurrency.html): Functions for +//! - [`LockableCurrency`](../paint_support/traits/trait.LockableCurrency.html): Functions for //! dealing with accounts that allow liquidity restrictions. -//! - [`Imbalance`](../srml_support/traits/trait.Imbalance.html): Functions for handling +//! - [`Imbalance`](../paint_support/traits/trait.Imbalance.html): Functions for handling //! imbalances between total issuance in the system and account balances. Must be used when a function //! creates new funds (e.g. a reward) or destroys some funds (e.g. a system fee). -//! - [`IsDeadAccount`](../srml_system/trait.IsDeadAccount.html): Determiner to say whether a +//! - [`IsDeadAccount`](../paint_system/trait.IsDeadAccount.html): Determiner to say whether a //! given account is unused. //! //! ## Interface @@ -153,7 +153,7 @@ use codec::{Codec, Encode, Decode}; use support::{ StorageValue, Parameter, decl_event, decl_storage, decl_module, traits::{ - UpdateBalanceOutcome, Currency, OnFreeBalanceZero, OnUnbalanced, + UpdateBalanceOutcome, Currency, OnFreeBalanceZero, OnUnbalanced, TryDrop, WithdrawReason, WithdrawReasons, LockIdentifier, LockableCurrency, ExistenceRequirement, Imbalance, SignedImbalance, ReservableCurrency, Get, }, @@ -397,6 +397,8 @@ decl_module! { /// `T::OnNewAccount::on_new_account` to be called. /// - Removing enough funds from an account will trigger /// `T::DustRemoval::on_unbalanced` and `T::OnFreeBalanceZero::on_free_balance_zero`. + /// - `transfer_keep_alive` works the same way as `transfer`, but has an additional + /// check that the transfer will not kill the origin account. /// /// # #[weight = SimpleDispatchInfo::FixedNormal(1_000_000)] @@ -407,7 +409,7 @@ decl_module! { ) { let transactor = ensure_signed(origin)?; let dest = T::Lookup::lookup(dest)?; - >::transfer(&transactor, &dest, value)?; + >::transfer(&transactor, &dest, value, ExistenceRequirement::AllowDeath)?; } /// Set the balances of a given account. @@ -462,8 +464,26 @@ decl_module! { ensure_root(origin)?; let source = T::Lookup::lookup(source)?; let dest = T::Lookup::lookup(dest)?; - >::transfer(&source, &dest, value)?; + >::transfer(&source, &dest, value, ExistenceRequirement::AllowDeath)?; } + + /// Same as the [`transfer`] call, but with a check that the transfer will not kill the + /// origin account. + /// + /// 99% of the time you want [`transfer`] instead. + /// + /// [`transfer`]: struct.Module.html#method.transfer + #[weight = SimpleDispatchInfo::FixedNormal(1_000_000)] + pub fn transfer_keep_alive( + origin, + dest: ::Source, + #[compact] value: T::Balance + ) { + let transactor = ensure_signed(origin)?; + let dest = T::Lookup::lookup(dest)?; + >::transfer(&transactor, &dest, value, ExistenceRequirement::KeepAlive)?; + } + } } @@ -583,7 +603,7 @@ impl, I: Instance> Module { mod imbalances { use super::{ result, Subtrait, DefaultInstance, Imbalance, Trait, Zero, Instance, Saturating, - StorageValue, + StorageValue, TryDrop, }; use rstd::mem; @@ -611,6 +631,12 @@ mod imbalances { } } + impl, I: Instance> TryDrop for PositiveImbalance { + fn try_drop(self) -> result::Result<(), Self> { + self.drop_zero() + } + } + impl, I: Instance> Imbalance for PositiveImbalance { type Opposite = NegativeImbalance; @@ -656,6 +682,12 @@ mod imbalances { } } + impl, I: Instance> TryDrop for NegativeImbalance { + fn try_drop(self) -> result::Result<(), Self> { + self.drop_zero() + } + } + impl, I: Instance> Imbalance for NegativeImbalance { type Opposite = PositiveImbalance; @@ -851,7 +883,12 @@ where } } - fn transfer(transactor: &T::AccountId, dest: &T::AccountId, value: Self::Balance) -> Result { + fn transfer( + transactor: &T::AccountId, + dest: &T::AccountId, + value: Self::Balance, + existence_requirement: ExistenceRequirement, + ) -> Result { let from_balance = Self::free_balance(transactor); let to_balance = Self::free_balance(dest); let would_create = to_balance.is_zero(); @@ -878,6 +915,12 @@ where }; if transactor != dest { + if existence_requirement == ExistenceRequirement::KeepAlive { + if new_from_balance < Self::minimum_balance() { + return Err("transfer would kill account"); + } + } + Self::set_free_balance(transactor, new_from_balance); if !>::exists(dest) { Self::new_account(dest, new_to_balance); diff --git a/srml/balances/src/mock.rs b/paint/balances/src/mock.rs similarity index 100% rename from srml/balances/src/mock.rs rename to paint/balances/src/mock.rs diff --git a/srml/balances/src/tests.rs b/paint/balances/src/tests.rs similarity index 92% rename from srml/balances/src/tests.rs rename to paint/balances/src/tests.rs index 839ac67991c83..8afec6f697de1 100644 --- a/srml/balances/src/tests.rs +++ b/paint/balances/src/tests.rs @@ -24,7 +24,7 @@ use sr_primitives::traits::SignedExtension; use support::{ assert_noop, assert_ok, assert_err, traits::{LockableCurrency, LockIdentifier, WithdrawReason, WithdrawReasons, - Currency, ReservableCurrency} + Currency, ReservableCurrency, ExistenceRequirement::AllowDeath} }; use transaction_payment::ChargeTransactionPayment; use system::RawOrigin; @@ -39,7 +39,7 @@ fn basic_locking_should_work() { assert_eq!(Balances::free_balance(&1), 10); Balances::set_lock(ID_1, &1, 9, u64::max_value(), WithdrawReasons::all()); assert_noop!( - >::transfer(&1, &2, 5), + >::transfer(&1, &2, 5, AllowDeath), "account liquidity restrictions prevent withdrawal" ); }); @@ -49,7 +49,7 @@ fn basic_locking_should_work() { fn partial_locking_should_work() { ExtBuilder::default().existential_deposit(1).monied(true).build().execute_with(|| { Balances::set_lock(ID_1, &1, 5, u64::max_value(), WithdrawReasons::all()); - assert_ok!(>::transfer(&1, &2, 1)); + assert_ok!(>::transfer(&1, &2, 1, AllowDeath)); }); } @@ -58,7 +58,7 @@ fn lock_removal_should_work() { ExtBuilder::default().existential_deposit(1).monied(true).build().execute_with(|| { Balances::set_lock(ID_1, &1, u64::max_value(), u64::max_value(), WithdrawReasons::all()); Balances::remove_lock(ID_1, &1); - assert_ok!(>::transfer(&1, &2, 1)); + assert_ok!(>::transfer(&1, &2, 1, AllowDeath)); }); } @@ -67,7 +67,7 @@ fn lock_replacement_should_work() { ExtBuilder::default().existential_deposit(1).monied(true).build().execute_with(|| { Balances::set_lock(ID_1, &1, u64::max_value(), u64::max_value(), WithdrawReasons::all()); Balances::set_lock(ID_1, &1, 5, u64::max_value(), WithdrawReasons::all()); - assert_ok!(>::transfer(&1, &2, 1)); + assert_ok!(>::transfer(&1, &2, 1, AllowDeath)); }); } @@ -76,7 +76,7 @@ fn double_locking_should_work() { ExtBuilder::default().existential_deposit(1).monied(true).build().execute_with(|| { Balances::set_lock(ID_1, &1, 5, u64::max_value(), WithdrawReasons::all()); Balances::set_lock(ID_2, &1, 5, u64::max_value(), WithdrawReasons::all()); - assert_ok!(>::transfer(&1, &2, 1)); + assert_ok!(>::transfer(&1, &2, 1, AllowDeath)); }); } @@ -86,7 +86,7 @@ fn combination_locking_should_work() { Balances::set_lock(ID_1, &1, u64::max_value(), 0, WithdrawReasons::none()); Balances::set_lock(ID_2, &1, 0, u64::max_value(), WithdrawReasons::none()); Balances::set_lock(ID_3, &1, 0, 0, WithdrawReasons::all()); - assert_ok!(>::transfer(&1, &2, 1)); + assert_ok!(>::transfer(&1, &2, 1, AllowDeath)); }); } @@ -95,17 +95,17 @@ fn lock_value_extension_should_work() { ExtBuilder::default().existential_deposit(1).monied(true).build().execute_with(|| { Balances::set_lock(ID_1, &1, 5, u64::max_value(), WithdrawReasons::all()); assert_noop!( - >::transfer(&1, &2, 6), + >::transfer(&1, &2, 6, AllowDeath), "account liquidity restrictions prevent withdrawal" ); Balances::extend_lock(ID_1, &1, 2, u64::max_value(), WithdrawReasons::all()); assert_noop!( - >::transfer(&1, &2, 6), + >::transfer(&1, &2, 6, AllowDeath), "account liquidity restrictions prevent withdrawal" ); Balances::extend_lock(ID_1, &1, 8, u64::max_value(), WithdrawReasons::all()); assert_noop!( - >::transfer(&1, &2, 3), + >::transfer(&1, &2, 3, AllowDeath), "account liquidity restrictions prevent withdrawal" ); }); @@ -120,7 +120,7 @@ fn lock_reasons_should_work() { .execute_with(|| { Balances::set_lock(ID_1, &1, 10, u64::max_value(), WithdrawReason::Transfer.into()); assert_noop!( - >::transfer(&1, &2, 1), + >::transfer(&1, &2, 1, AllowDeath), "account liquidity restrictions prevent withdrawal" ); assert_ok!(>::reserve(&1, 1)); @@ -134,7 +134,7 @@ fn lock_reasons_should_work() { ).is_ok()); Balances::set_lock(ID_1, &1, 10, u64::max_value(), WithdrawReason::Reserve.into()); - assert_ok!(>::transfer(&1, &2, 1)); + assert_ok!(>::transfer(&1, &2, 1, AllowDeath)); assert_noop!( >::reserve(&1, 1), "account liquidity restrictions prevent withdrawal" @@ -148,7 +148,7 @@ fn lock_reasons_should_work() { ).is_ok()); Balances::set_lock(ID_1, &1, 10, u64::max_value(), WithdrawReason::TransactionPayment.into()); - assert_ok!(>::transfer(&1, &2, 1)); + assert_ok!(>::transfer(&1, &2, 1, AllowDeath)); assert_ok!(>::reserve(&1, 1)); assert!( as SignedExtension>::pre_dispatch( ChargeTransactionPayment::from(1), @@ -165,12 +165,12 @@ fn lock_block_number_should_work() { ExtBuilder::default().existential_deposit(1).monied(true).build().execute_with(|| { Balances::set_lock(ID_1, &1, 10, 2, WithdrawReasons::all()); assert_noop!( - >::transfer(&1, &2, 1), + >::transfer(&1, &2, 1, AllowDeath), "account liquidity restrictions prevent withdrawal" ); System::set_block_number(2); - assert_ok!(>::transfer(&1, &2, 1)); + assert_ok!(>::transfer(&1, &2, 1, AllowDeath)); }); } @@ -179,18 +179,18 @@ fn lock_block_number_extension_should_work() { ExtBuilder::default().existential_deposit(1).monied(true).build().execute_with(|| { Balances::set_lock(ID_1, &1, 10, 2, WithdrawReasons::all()); assert_noop!( - >::transfer(&1, &2, 6), + >::transfer(&1, &2, 6, AllowDeath), "account liquidity restrictions prevent withdrawal" ); Balances::extend_lock(ID_1, &1, 10, 1, WithdrawReasons::all()); assert_noop!( - >::transfer(&1, &2, 6), + >::transfer(&1, &2, 6, AllowDeath), "account liquidity restrictions prevent withdrawal" ); System::set_block_number(2); Balances::extend_lock(ID_1, &1, 10, 8, WithdrawReasons::all()); assert_noop!( - >::transfer(&1, &2, 3), + >::transfer(&1, &2, 3, AllowDeath), "account liquidity restrictions prevent withdrawal" ); }); @@ -201,17 +201,17 @@ fn lock_reasons_extension_should_work() { ExtBuilder::default().existential_deposit(1).monied(true).build().execute_with(|| { Balances::set_lock(ID_1, &1, 10, 10, WithdrawReason::Transfer.into()); assert_noop!( - >::transfer(&1, &2, 6), + >::transfer(&1, &2, 6, AllowDeath), "account liquidity restrictions prevent withdrawal" ); Balances::extend_lock(ID_1, &1, 10, 10, WithdrawReasons::none()); assert_noop!( - >::transfer(&1, &2, 6), + >::transfer(&1, &2, 6, AllowDeath), "account liquidity restrictions prevent withdrawal" ); Balances::extend_lock(ID_1, &1, 10, 10, WithdrawReason::Reserve.into()); assert_noop!( - >::transfer(&1, &2, 6), + >::transfer(&1, &2, 6, AllowDeath), "account liquidity restrictions prevent withdrawal" ); }); @@ -746,3 +746,17 @@ fn burn_must_work() { assert_eq!(Balances::total_issuance(), init_total_issuance); }); } + +#[test] +fn transfer_keep_alive_works() { + ExtBuilder::default().existential_deposit(1).build().execute_with(|| { + let _ = Balances::deposit_creating(&1, 100); + assert_err!( + Balances::transfer_keep_alive(Some(1).into(), 2, 100), + "transfer would kill account" + ); + assert_eq!(Balances::is_dead_account(&1), false); + assert_eq!(Balances::total_balance(&1), 100); + assert_eq!(Balances::total_balance(&2), 0); + }); +} diff --git a/srml/collective/Cargo.toml b/paint/collective/Cargo.toml similarity index 55% rename from srml/collective/Cargo.toml rename to paint/collective/Cargo.toml index 347a289b57b89..3f629f25c1e94 100644 --- a/srml/collective/Cargo.toml +++ b/paint/collective/Cargo.toml @@ -1,5 +1,5 @@ [package] -name = "srml-collective" +name = "paint-collective" version = "2.0.0" authors = ["Parity Technologies "] edition = "2018" @@ -8,16 +8,16 @@ edition = "2018" serde = { version = "1.0.101", optional = true } safe-mix = { version = "1.0.0", default-features = false } codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false, features = ["derive"] } -primitives = { package = "substrate-primitives", path = "../../core/primitives", default-features = false } -rstd = { package = "sr-std", path = "../../core/sr-std", default-features = false } -runtime-io = { package = "sr-io", path = "../../core/sr-io", default-features = false } -sr-primitives = { path = "../../core/sr-primitives", default-features = false } -support = { package = "srml-support", path = "../support", default-features = false } -system = { package = "srml-system", path = "../system", default-features = false } +primitives = { package = "substrate-primitives", path = "../../primitives/core", default-features = false } +rstd = { package = "sr-std", path = "../../primitives/sr-std", default-features = false } +runtime-io = { package = "sr-io", path = "../../primitives/sr-io", default-features = false } +sr-primitives = { path = "../../primitives/sr-primitives", default-features = false } +support = { package = "paint-support", path = "../support", default-features = false } +system = { package = "paint-system", path = "../system", default-features = false } [dev-dependencies] hex-literal = "0.2.1" -balances = { package = "srml-balances", path = "../balances" } +balances = { package = "paint-balances", path = "../balances" } [features] default = ["std"] diff --git a/srml/collective/src/lib.rs b/paint/collective/src/lib.rs similarity index 100% rename from srml/collective/src/lib.rs rename to paint/collective/src/lib.rs diff --git a/srml/contracts/COMPLEXITY.md b/paint/contracts/COMPLEXITY.md similarity index 100% rename from srml/contracts/COMPLEXITY.md rename to paint/contracts/COMPLEXITY.md diff --git a/srml/contracts/Cargo.toml b/paint/contracts/Cargo.toml similarity index 53% rename from srml/contracts/Cargo.toml rename to paint/contracts/Cargo.toml index 365566cfb547e..728cf72e11d49 100644 --- a/srml/contracts/Cargo.toml +++ b/paint/contracts/Cargo.toml @@ -1,5 +1,5 @@ [package] -name = "srml-contracts" +name = "paint-contracts" version = "2.0.0" authors = ["Parity Technologies "] edition = "2018" @@ -10,22 +10,22 @@ pwasm-utils = { version = "0.11.0", default-features = false } codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false, features = ["derive"] } parity-wasm = { version = "0.40.3", default-features = false } wasmi-validation = { version = "0.2.0", default-features = false } -primitives = { package = "substrate-primitives", path = "../../core/primitives", default-features = false } -sr-primitives = { path = "../../core/sr-primitives", default-features = false } -runtime-io = { package = "sr-io", path = "../../core/sr-io", default-features = false } -rstd = { package = "sr-std", path = "../../core/sr-std", default-features = false } -sandbox = { package = "sr-sandbox", path = "../../core/sr-sandbox", default-features = false } -support = { package = "srml-support", path = "../support", default-features = false } -system = { package = "srml-system", path = "../system", default-features = false } +primitives = { package = "substrate-primitives", path = "../../primitives/core", default-features = false } +sr-primitives = { path = "../../primitives/sr-primitives", default-features = false } +runtime-io = { package = "sr-io", path = "../../primitives/sr-io", default-features = false } +rstd = { package = "sr-std", path = "../../primitives/sr-std", default-features = false } +sandbox = { package = "sr-sandbox", path = "../../primitives/sr-sandbox", default-features = false } +support = { package = "paint-support", path = "../support", default-features = false } +system = { package = "paint-system", path = "../system", default-features = false } [dev-dependencies] wabt = "0.9.2" assert_matches = "1.3.0" hex-literal = "0.2.1" -balances = { package = "srml-balances", path = "../balances" } +balances = { package = "paint-balances", path = "../balances" } hex = "0.3.2" -timestamp = { package = "srml-timestamp", path = "../timestamp" } -randomness-collective-flip = { package = "srml-randomness-collective-flip", path = "../randomness-collective-flip" } +timestamp = { package = "paint-timestamp", path = "../timestamp" } +randomness-collective-flip = { package = "paint-randomness-collective-flip", path = "../randomness-collective-flip" } [features] default = ["std"] diff --git a/srml/contracts/rpc/Cargo.toml b/paint/contracts/rpc/Cargo.toml similarity index 50% rename from srml/contracts/rpc/Cargo.toml rename to paint/contracts/rpc/Cargo.toml index 90bf34bec1fd9..723b51a38bdaa 100644 --- a/srml/contracts/rpc/Cargo.toml +++ b/paint/contracts/rpc/Cargo.toml @@ -1,17 +1,17 @@ [package] -name = "srml-contracts-rpc" +name = "paint-contracts-rpc" version = "2.0.0" authors = ["Parity Technologies "] edition = "2018" [dependencies] -client = { package = "substrate-client", path = "../../../core/client" } +client = { package = "substrate-client", path = "../../../client/" } codec = { package = "parity-scale-codec", version = "1.0.0" } -jsonrpc-core = "13.2.0" -jsonrpc-core-client = "13.2.0" -jsonrpc-derive = "13.2.0" -primitives = { package = "substrate-primitives", path = "../../../core/primitives" } -rpc-primitives = { package = "substrate-rpc-primitives", path = "../../../core/rpc/primitives" } +jsonrpc-core = "14.0.3" +jsonrpc-core-client = "14.0.3" +jsonrpc-derive = "14.0.3" +primitives = { package = "substrate-primitives", path = "../../../primitives/core" } +rpc-primitives = { package = "substrate-rpc-primitives", path = "../../../primitives/rpc" } serde = { version = "1.0.101", features = ["derive"] } -sr-primitives = { path = "../../../core/sr-primitives" } -srml-contracts-rpc-runtime-api = { path = "./runtime-api" } +sr-primitives = { path = "../../../primitives/sr-primitives" } +paint-contracts-rpc-runtime-api = { path = "./runtime-api" } diff --git a/srml/contracts/rpc/runtime-api/Cargo.toml b/paint/contracts/rpc/runtime-api/Cargo.toml similarity index 54% rename from srml/contracts/rpc/runtime-api/Cargo.toml rename to paint/contracts/rpc/runtime-api/Cargo.toml index 2a36ed2c96cc4..ac5c2be224713 100644 --- a/srml/contracts/rpc/runtime-api/Cargo.toml +++ b/paint/contracts/rpc/runtime-api/Cargo.toml @@ -1,20 +1,20 @@ [package] -name = "srml-contracts-rpc-runtime-api" +name = "paint-contracts-rpc-runtime-api" version = "2.0.0" authors = ["Parity Technologies "] edition = "2018" [dependencies] -client = { package = "substrate-client", path = "../../../../core/client", default-features = false } +sr-api = { path = "../../../../primitives/sr-api", default-features = false } codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false, features = ["derive"] } -rstd = { package = "sr-std", path = "../../../../core/sr-std", default-features = false } +rstd = { package = "sr-std", path = "../../../../primitives/sr-std", default-features = false } serde = { version = "1.0.101", optional = true, features = ["derive"] } -sr-primitives = { path = "../../../../core/sr-primitives", default-features = false } +sr-primitives = { path = "../../../../primitives/sr-primitives", default-features = false } [features] default = ["std"] std = [ - "client/std", + "sr-api/std", "codec/std", "rstd/std", "serde", diff --git a/srml/contracts/rpc/runtime-api/src/lib.rs b/paint/contracts/rpc/runtime-api/src/lib.rs similarity index 99% rename from srml/contracts/rpc/runtime-api/src/lib.rs rename to paint/contracts/rpc/runtime-api/src/lib.rs index 054f110beb4d0..6d9c4a27b1feb 100644 --- a/srml/contracts/rpc/runtime-api/src/lib.rs +++ b/paint/contracts/rpc/runtime-api/src/lib.rs @@ -59,7 +59,7 @@ pub enum GetStorageError { IsTombstone, } -client::decl_runtime_apis! { +sr_api::decl_runtime_apis! { /// The API to interact with contracts without using executive. pub trait ContractsApi where AccountId: Codec, diff --git a/srml/contracts/rpc/src/lib.rs b/paint/contracts/rpc/src/lib.rs similarity index 99% rename from srml/contracts/rpc/src/lib.rs rename to paint/contracts/rpc/src/lib.rs index 91783df99617e..e226949d54a64 100644 --- a/srml/contracts/rpc/src/lib.rs +++ b/paint/contracts/rpc/src/lib.rs @@ -31,7 +31,7 @@ use sr_primitives::{ }; pub use self::gen_client::Client as ContractsClient; -pub use srml_contracts_rpc_runtime_api::{ +pub use paint_contracts_rpc_runtime_api::{ self as runtime_api, ContractExecResult, ContractsApi as ContractsRuntimeApi, GetStorageResult, }; diff --git a/srml/contracts/src/account_db.rs b/paint/contracts/src/account_db.rs similarity index 99% rename from srml/contracts/src/account_db.rs rename to paint/contracts/src/account_db.rs index 50bd1fd40e97d..5aa3a64fd9bc8 100644 --- a/srml/contracts/src/account_db.rs +++ b/paint/contracts/src/account_db.rs @@ -24,7 +24,7 @@ use crate::exec::StorageKey; use rstd::cell::RefCell; use rstd::collections::btree_map::{BTreeMap, Entry}; use rstd::prelude::*; -use runtime_io::blake2_256; +use runtime_io::hashing::blake2_256; use sr_primitives::traits::{Bounded, Zero}; use support::traits::{Currency, Get, Imbalance, SignedImbalance, UpdateBalanceOutcome}; use support::{storage::child, StorageMap}; diff --git a/srml/contracts/src/exec.rs b/paint/contracts/src/exec.rs similarity index 100% rename from srml/contracts/src/exec.rs rename to paint/contracts/src/exec.rs diff --git a/srml/contracts/src/gas.rs b/paint/contracts/src/gas.rs similarity index 100% rename from srml/contracts/src/gas.rs rename to paint/contracts/src/gas.rs diff --git a/srml/contracts/src/lib.rs b/paint/contracts/src/lib.rs similarity index 99% rename from srml/contracts/src/lib.rs rename to paint/contracts/src/lib.rs index df38747cc5d52..a4b59226dcdf4 100644 --- a/srml/contracts/src/lib.rs +++ b/paint/contracts/src/lib.rs @@ -84,7 +84,7 @@ //! //! ## Related Modules //! -//! * [Balances](../srml_balances/index.html) +//! * [Balances](../paint_balances/index.html) #![cfg_attr(not(feature = "std"), no_std)] @@ -111,7 +111,7 @@ use serde::{Serialize, Deserialize}; use primitives::crypto::UncheckedFrom; use rstd::{prelude::*, marker::PhantomData, fmt::Debug}; use codec::{Codec, Encode, Decode}; -use runtime_io::blake2_256; +use runtime_io::hashing::blake2_256; use sr_primitives::{ traits::{Hash, StaticLookup, Zero, MaybeSerializeDeserialize, Member, SignedExtension}, weights::DispatchInfo, @@ -803,7 +803,7 @@ impl Module { let tombstone = >::new( // This operation is cheap enough because last_write (delta not included) // is not this block as it has been checked earlier. - &runtime_io::child_storage_root(&origin_contract.trie_id)[..], + &runtime_io::storage::child_root(&origin_contract.trie_id)[..], code_hash, ); diff --git a/srml/contracts/src/rent.rs b/paint/contracts/src/rent.rs similarity index 97% rename from srml/contracts/src/rent.rs rename to paint/contracts/src/rent.rs index 6647f8963165d..e286ce307fcc8 100644 --- a/srml/contracts/src/rent.rs +++ b/paint/contracts/src/rent.rs @@ -99,7 +99,7 @@ fn try_evict_or_and_pay_rent( if balance < subsistence_threshold { // The contract cannot afford to leave a tombstone, so remove the contract info altogether. >::remove(account); - runtime_io::kill_child_storage(&contract.trie_id); + runtime_io::storage::child_storage_kill(&contract.trie_id); return (RentOutcome::Evicted, None); } @@ -146,7 +146,7 @@ fn try_evict_or_and_pay_rent( // threshold, so it leaves a tombstone. // Note: this operation is heavy. - let child_storage_root = runtime_io::child_storage_root(&contract.trie_id); + let child_storage_root = runtime_io::storage::child_root(&contract.trie_id); let tombstone = >::new( &child_storage_root[..], @@ -155,7 +155,7 @@ fn try_evict_or_and_pay_rent( let tombstone_info = ContractInfo::Tombstone(tombstone); >::insert(account, &tombstone_info); - runtime_io::kill_child_storage(&contract.trie_id); + runtime_io::storage::child_storage_kill(&contract.trie_id); return (RentOutcome::Evicted, Some(tombstone_info)); } diff --git a/srml/contracts/src/tests.rs b/paint/contracts/src/tests.rs similarity index 100% rename from srml/contracts/src/tests.rs rename to paint/contracts/src/tests.rs diff --git a/srml/contracts/src/wasm/code_cache.rs b/paint/contracts/src/wasm/code_cache.rs similarity index 100% rename from srml/contracts/src/wasm/code_cache.rs rename to paint/contracts/src/wasm/code_cache.rs diff --git a/srml/contracts/src/wasm/env_def/macros.rs b/paint/contracts/src/wasm/env_def/macros.rs similarity index 100% rename from srml/contracts/src/wasm/env_def/macros.rs rename to paint/contracts/src/wasm/env_def/macros.rs diff --git a/srml/contracts/src/wasm/env_def/mod.rs b/paint/contracts/src/wasm/env_def/mod.rs similarity index 100% rename from srml/contracts/src/wasm/env_def/mod.rs rename to paint/contracts/src/wasm/env_def/mod.rs diff --git a/srml/contracts/src/wasm/mod.rs b/paint/contracts/src/wasm/mod.rs similarity index 99% rename from srml/contracts/src/wasm/mod.rs rename to paint/contracts/src/wasm/mod.rs index 4fa9412bc1769..42ee6ee0a2c5a 100644 --- a/srml/contracts/src/wasm/mod.rs +++ b/paint/contracts/src/wasm/mod.rs @@ -58,7 +58,7 @@ pub struct PrefabWasmModule { /// Wasm executable loaded by `WasmLoader` and executed by `WasmVm`. pub struct WasmExecutable { - entrypoint_name: &'static [u8], + entrypoint_name: &'static str, prefab_module: PrefabWasmModule, } @@ -79,14 +79,14 @@ impl<'a, T: Trait> crate::exec::Loader for WasmLoader<'a> { fn load_init(&self, code_hash: &CodeHash) -> Result { let prefab_module = load_code::(code_hash, self.schedule)?; Ok(WasmExecutable { - entrypoint_name: b"deploy", + entrypoint_name: "deploy", prefab_module, }) } fn load_main(&self, code_hash: &CodeHash) -> Result { let prefab_module = load_code::(code_hash, self.schedule)?; Ok(WasmExecutable { - entrypoint_name: b"call", + entrypoint_name: "call", prefab_module, }) } @@ -406,7 +406,7 @@ mod tests { let exec = WasmExecutable { // Use a "call" convention. - entrypoint_name: b"call", + entrypoint_name: "call", prefab_module, }; diff --git a/srml/contracts/src/wasm/prepare.rs b/paint/contracts/src/wasm/prepare.rs similarity index 100% rename from srml/contracts/src/wasm/prepare.rs rename to paint/contracts/src/wasm/prepare.rs diff --git a/srml/contracts/src/wasm/runtime.rs b/paint/contracts/src/wasm/runtime.rs similarity index 100% rename from srml/contracts/src/wasm/runtime.rs rename to paint/contracts/src/wasm/runtime.rs diff --git a/srml/democracy/Cargo.toml b/paint/democracy/Cargo.toml similarity index 56% rename from srml/democracy/Cargo.toml rename to paint/democracy/Cargo.toml index 761e56ceda6cb..1f28c013c5d1c 100644 --- a/srml/democracy/Cargo.toml +++ b/paint/democracy/Cargo.toml @@ -1,5 +1,5 @@ [package] -name = "srml-democracy" +name = "paint-democracy" version = "2.0.0" authors = ["Parity Technologies "] edition = "2018" @@ -8,15 +8,15 @@ edition = "2018" serde = { version = "1.0.101", optional = true, features = ["derive"] } safe-mix = { version = "1.0.0", default-features = false } codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false, features = ["derive"] } -rstd = { package = "sr-std", path = "../../core/sr-std", default-features = false } -runtime-io = { package = "sr-io", path = "../../core/sr-io", default-features = false } -sr-primitives = { path = "../../core/sr-primitives", default-features = false } -support = { package = "srml-support", path = "../support", default-features = false } -system = { package = "srml-system", path = "../system", default-features = false } +rstd = { package = "sr-std", path = "../../primitives/sr-std", default-features = false } +runtime-io = { package = "sr-io", path = "../../primitives/sr-io", default-features = false } +sr-primitives = { path = "../../primitives/sr-primitives", default-features = false } +support = { package = "paint-support", path = "../support", default-features = false } +system = { package = "paint-system", path = "../system", default-features = false } [dev-dependencies] -primitives = { package = "substrate-primitives", path = "../../core/primitives" } -balances = { package = "srml-balances", path = "../balances" } +primitives = { package = "substrate-primitives", path = "../../primitives/core" } +balances = { package = "paint-balances", path = "../balances" } [features] default = ["std"] diff --git a/srml/democracy/src/lib.rs b/paint/democracy/src/lib.rs similarity index 100% rename from srml/democracy/src/lib.rs rename to paint/democracy/src/lib.rs diff --git a/srml/democracy/src/vote_threshold.rs b/paint/democracy/src/vote_threshold.rs similarity index 100% rename from srml/democracy/src/vote_threshold.rs rename to paint/democracy/src/vote_threshold.rs diff --git a/paint/elections-phragmen/Cargo.toml b/paint/elections-phragmen/Cargo.toml new file mode 100644 index 0000000000000..0fb1f6246567d --- /dev/null +++ b/paint/elections-phragmen/Cargo.toml @@ -0,0 +1,31 @@ +[package] +name = "paint-elections-phragmen" +version = "2.0.0" +authors = ["Parity Technologies "] +edition = "2018" + +[dependencies] +codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false, features = ["derive"] } +sr-primitives = { path = "../../primitives/sr-primitives", default-features = false } +phragmen = { package = "substrate-phragmen", path = "../../primitives/phragmen", default-features = false } +paint-support = { path = "../support", default-features = false } +system = { package = "paint-system", path = "../system", default-features = false } +rstd = { package = "sr-std", path = "../../primitives/sr-std", default-features = false } + +[dev-dependencies] +runtime_io = { package = "sr-io", path = "../../primitives/sr-io" } +hex-literal = "0.2.1" +balances = { package = "paint-balances", path = "../balances" } +primitives = { package = "substrate-primitives", path = "../../primitives/core" } +serde = { version = "1.0.101" } + +[features] +default = ["std"] +std = [ + "codec/std", + "paint-support/std", + "sr-primitives/std", + "phragmen/std", + "system/std", + "rstd/std", +] diff --git a/srml/elections-phragmen/src/lib.rs b/paint/elections-phragmen/src/lib.rs similarity index 99% rename from srml/elections-phragmen/src/lib.rs rename to paint/elections-phragmen/src/lib.rs index 9b277815f385a..0a615bb8c98a2 100644 --- a/srml/elections-phragmen/src/lib.rs +++ b/paint/elections-phragmen/src/lib.rs @@ -80,7 +80,7 @@ use rstd::prelude::*; use codec::Decode; use sr_primitives::{print, traits::{Zero, StaticLookup, Bounded, Convert}}; use sr_primitives::weights::SimpleDispatchInfo; -use srml_support::{ +use paint_support::{ decl_storage, decl_event, ensure, decl_module, dispatch, storage::unhashed, traits::{ @@ -542,7 +542,6 @@ impl Module { candidates, voters_and_votes, Self::locked_stake_of, - false, ); let mut to_release_bond: Vec = Vec::with_capacity(desired_seats); @@ -563,7 +562,6 @@ impl Module { &new_set, &phragmen_result.assignments, Self::locked_stake_of, - false, ); let to_balance = |e: ExtendedBalance| @@ -669,7 +667,7 @@ impl Module { mod tests { use super::*; use std::cell::RefCell; - use srml_support::{assert_ok, assert_noop, parameter_types, assert_eq_uvec}; + use paint_support::{assert_ok, assert_noop, parameter_types, assert_eq_uvec}; use primitives::H256; use sr_primitives::{ Perbill, testing::Header, BuildStorage, @@ -785,7 +783,7 @@ mod tests { pub type Block = sr_primitives::generic::Block; pub type UncheckedExtrinsic = sr_primitives::generic::UncheckedExtrinsic; - srml_support::construct_runtime!( + paint_support::construct_runtime!( pub enum Test where Block = Block, NodeBlock = Block, @@ -865,7 +863,7 @@ mod tests { #[test] fn temp_migration_works() { ExtBuilder::default().build().execute_with(|| { - use srml_support::storage::unhashed; + use paint_support::storage::unhashed; use codec::Encode; let old_members = vec![1u64, 2]; diff --git a/srml/elections/Cargo.toml b/paint/elections/Cargo.toml similarity index 55% rename from srml/elections/Cargo.toml rename to paint/elections/Cargo.toml index 88d147ac087aa..1e8e7702f557d 100644 --- a/srml/elections/Cargo.toml +++ b/paint/elections/Cargo.toml @@ -1,5 +1,5 @@ [package] -name = "srml-elections" +name = "paint-elections" version = "2.0.0" authors = ["Parity Technologies "] edition = "2018" @@ -8,16 +8,16 @@ edition = "2018" serde = { version = "1.0.101", optional = true } safe-mix = { version = "1.0.0", default-features = false } codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false, features = ["derive"] } -primitives = { package = "substrate-primitives", path = "../../core/primitives", default-features = false } -rstd = { package = "sr-std", path = "../../core/sr-std", default-features = false } -runtime-io = { package = "sr-io", path = "../../core/sr-io", default-features = false } -sr-primitives = { path = "../../core/sr-primitives", default-features = false } -support = { package = "srml-support", path = "../support", default-features = false } -system = { package = "srml-system", path = "../system", default-features = false } +primitives = { package = "substrate-primitives", path = "../../primitives/core", default-features = false } +rstd = { package = "sr-std", path = "../../primitives/sr-std", default-features = false } +runtime-io = { package = "sr-io", path = "../../primitives/sr-io", default-features = false } +sr-primitives = { path = "../../primitives/sr-primitives", default-features = false } +support = { package = "paint-support", path = "../support", default-features = false } +system = { package = "paint-system", path = "../system", default-features = false } [dev-dependencies] hex-literal = "0.2.1" -balances = { package = "srml-balances", path = "../balances" } +balances = { package = "paint-balances", path = "../balances" } [features] default = ["std"] diff --git a/srml/elections/src/lib.rs b/paint/elections/src/lib.rs similarity index 100% rename from srml/elections/src/lib.rs rename to paint/elections/src/lib.rs diff --git a/srml/elections/src/mock.rs b/paint/elections/src/mock.rs similarity index 100% rename from srml/elections/src/mock.rs rename to paint/elections/src/mock.rs diff --git a/srml/elections/src/tests.rs b/paint/elections/src/tests.rs similarity index 100% rename from srml/elections/src/tests.rs rename to paint/elections/src/tests.rs diff --git a/paint/evm/Cargo.toml b/paint/evm/Cargo.toml new file mode 100644 index 0000000000000..45cb990861d27 --- /dev/null +++ b/paint/evm/Cargo.toml @@ -0,0 +1,40 @@ +[package] +name = "paint-evm" +version = "2.0.0" +authors = ["Parity Technologies "] +edition = "2018" + +[dependencies] +serde = { version = "1.0.101", optional = true, features = ["derive"] } +codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false } +support = { package = "paint-support", path = "../support", default-features = false } +system = { package = "paint-system", path = "../system", default-features = false } +timestamp = { package = "paint-timestamp", path = "../timestamp", default-features = false } +balances = { package = "paint-balances", path = "../balances", default-features = false } +primitives = { package = "substrate-primitives", path = "../../primitives/core", default-features = false } +sr-primitives = { path = "../../primitives/sr-primitives", default-features = false } +rstd = { package = "sr-std", path = "../../primitives/sr-std", default-features = false } +runtime-io = { package = "sr-io", path = "../../primitives/sr-io", default-features = false } +primitive-types = { version = "0.6", default-features = false, features = ["rlp"] } +rlp = { version = "0.4", default-features = false } +evm = { version = "0.14", default-features = false } +sha3 = { version = "0.8", default-features = false } + +[features] +default = ["std"] +std = [ + "serde", + "codec/std", + "primitives/std", + "sr-primitives/std", + "support/std", + "system/std", + "balances/std", + "runtime-io/std", + "rstd/std", + "sha3/std", + "rlp/std", + "primitive-types/std", + "evm/std", + "timestamp/std", +] diff --git a/paint/evm/src/backend.rs b/paint/evm/src/backend.rs new file mode 100644 index 0000000000000..1f3dfe309b4ee --- /dev/null +++ b/paint/evm/src/backend.rs @@ -0,0 +1,187 @@ +use rstd::marker::PhantomData; +use rstd::vec::Vec; +#[cfg(feature = "std")] +use serde::{Serialize, Deserialize}; +use codec::{Encode, Decode}; +use primitives::{U256, H256, H160}; +use sr_primitives::traits::UniqueSaturatedInto; +use support::storage::{StorageMap, StorageDoubleMap}; +use sha3::{Keccak256, Digest}; +use evm::Config; +use evm::backend::{Backend as BackendT, ApplyBackend, Apply}; +use crate::{Trait, Accounts, AccountStorages, AccountCodes, Module, Event}; + +#[derive(Clone, Eq, PartialEq, Encode, Decode, Default)] +#[cfg_attr(feature = "std", derive(Debug, Serialize, Deserialize))] +/// Ethereum account nonce, balance and code. Used by storage. +pub struct Account { + /// Account nonce. + pub nonce: U256, + /// Account balance. + pub balance: U256, +} + +#[derive(Clone, Eq, PartialEq, Encode, Decode)] +#[cfg_attr(feature = "std", derive(Debug, Serialize, Deserialize))] +/// Ethereum log. Used for `deposit_event`. +pub struct Log { + /// Source address of the log. + pub address: H160, + /// Topics of the log. + pub topics: Vec, + /// Bytearray data of the log. + pub data: Vec, +} + +#[derive(Clone, Eq, PartialEq, Encode, Decode, Default)] +#[cfg_attr(feature = "std", derive(Debug, Serialize, Deserialize))] +/// External input from the transaction. +pub struct Vicinity { + /// Current transaction gas price. + pub gas_price: U256, + /// Origin of the transaction. + pub origin: H160, +} + +/// Gasometer config used for executor. Currently this is hard-coded to +/// Istanbul hard fork. +pub static GASOMETER_CONFIG: Config = Config::istanbul(); + +/// Substrate backend for EVM. +pub struct Backend<'vicinity, T> { + vicinity: &'vicinity Vicinity, + _marker: PhantomData, +} + +impl<'vicinity, T> Backend<'vicinity, T> { + /// Create a new backend with given vicinity. + pub fn new(vicinity: &'vicinity Vicinity) -> Self { + Self { vicinity, _marker: PhantomData } + } +} + +impl<'vicinity, T: Trait> BackendT for Backend<'vicinity, T> { + fn gas_price(&self) -> U256 { self.vicinity.gas_price } + fn origin(&self) -> H160 { self.vicinity.origin } + + fn block_hash(&self, number: U256) -> H256 { + if number > U256::from(u32::max_value()) { + H256::default() + } else { + let number = T::BlockNumber::from(number.as_u32()); + H256::from_slice(system::Module::::block_hash(number).as_ref()) + } + } + + fn block_number(&self) -> U256 { + let number: u128 = system::Module::::block_number().unique_saturated_into(); + U256::from(number) + } + + fn block_coinbase(&self) -> H160 { + H160::default() + } + + fn block_timestamp(&self) -> U256 { + let now: u128 = timestamp::Module::::get().unique_saturated_into(); + U256::from(now) + } + + fn block_difficulty(&self) -> U256 { + U256::zero() + } + + fn block_gas_limit(&self) -> U256 { + U256::zero() + } + + fn chain_id(&self) -> U256 { + U256::from(runtime_io::misc::chain_id()) + } + + fn exists(&self, _address: H160) -> bool { + true + } + + fn basic(&self, address: H160) -> evm::backend::Basic { + let account = Accounts::get(&address); + + evm::backend::Basic { + balance: account.balance, + nonce: account.nonce, + } + } + + fn code_size(&self, address: H160) -> usize { + AccountCodes::decode_len(&address).unwrap_or(0) + } + + fn code_hash(&self, address: H160) -> H256 { + H256::from_slice(Keccak256::digest(&AccountCodes::get(&address)).as_slice()) + } + + fn code(&self, address: H160) -> Vec { + AccountCodes::get(&address) + } + + fn storage(&self, address: H160, index: H256) -> H256 { + AccountStorages::get(address, index) + } +} + +impl<'vicinity, T: Trait> ApplyBackend for Backend<'vicinity, T> { + fn apply( + &mut self, + values: A, + logs: L, + delete_empty: bool, + ) where + A: IntoIterator>, + I: IntoIterator, + L: IntoIterator, + { + for apply in values { + match apply { + Apply::Modify { + address, basic, code, storage, reset_storage, + } => { + Accounts::mutate(&address, |account| { + account.balance = basic.balance; + account.nonce = basic.nonce; + }); + + if let Some(code) = code { + AccountCodes::insert(address, code); + } + + if reset_storage { + AccountStorages::remove_prefix(address); + } + + for (index, value) in storage { + if value == H256::default() { + AccountStorages::remove(address, index); + } else { + AccountStorages::insert(address, index, value); + } + } + + if delete_empty { + Module::::remove_account_if_empty(&address); + } + }, + Apply::Delete { address } => { + Module::::remove_account(&address) + }, + } + } + + for log in logs { + Module::::deposit_event(Event::Log(Log { + address: log.address, + topics: log.topics, + data: log.data, + })); + } + } +} diff --git a/paint/evm/src/lib.rs b/paint/evm/src/lib.rs new file mode 100644 index 0000000000000..30835986e2014 --- /dev/null +++ b/paint/evm/src/lib.rs @@ -0,0 +1,299 @@ +// Copyright 2017-2019 Parity Technologies (UK) Ltd. +// This file is part of Substrate. + +// Substrate is free software: you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. + +// Substrate is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. + +// You should have received a copy of the GNU General Public License +// along with Substrate. If not, see . + +//! EVM execution module for Substrate + +// Ensure we're `no_std` when compiling for Wasm. +#![cfg_attr(not(feature = "std"), no_std)] + +mod backend; + +pub use crate::backend::{Account, Log, Vicinity, Backend}; + +use rstd::vec::Vec; +use support::{dispatch::Result, decl_module, decl_storage, decl_event}; +use support::traits::{Currency, WithdrawReason, ExistenceRequirement}; +use system::ensure_signed; +use sr_primitives::ModuleId; +use sr_primitives::weights::SimpleDispatchInfo; +use sr_primitives::traits::{UniqueSaturatedInto, AccountIdConversion}; +use primitives::{U256, H256, H160}; +use evm::{ExitReason, ExitSucceed, ExitError}; +use evm::executor::StackExecutor; +use evm::backend::ApplyBackend; + +const MODULE_ID: ModuleId = ModuleId(*b"py/ethvm"); + +/// Type alias for currency balance. +pub type BalanceOf = <::Currency as Currency<::AccountId>>::Balance; + +/// Trait that outputs the current transaction gas price. +pub trait FeeCalculator { + /// Return the current gas price. + fn gas_price() -> U256; +} + +/// Trait for converting account ids of `balances` module into +/// `H160` for EVM module. +/// +/// Accounts and contracts of this module are stored in its own +/// storage, in an Ethereum-compatible format. In order to communicate +/// with the rest of Substrate module, we require an one-to-one +/// mapping of Substrate account to Ethereum address. +pub trait ConvertAccountId { + /// Given a Substrate address, return the corresponding Ethereum address. + fn convert_account_id(account_id: &A) -> H160; +} + +/// Custom precompiles to be used by EVM engine. +pub trait Precompiles { + /// Try to execute the code address as precompile. If the code address is not + /// a precompile or the precompile is not yet available, return `None`. + /// Otherwise, calculate the amount of gas needed with given `input` and + /// `target_gas`. Return `Some(Ok(status, output, gas_used))` if the execution + /// is successful. Otherwise return `Some(Err(_))`. + fn execute( + address: H160, + input: &[u8], + target_gas: Option + ) -> Option, usize), ExitError>>; +} + +impl Precompiles for () { + fn execute( + _address: H160, + _input: &[u8], + _target_gas: Option + ) -> Option, usize), ExitError>> { + None + } +} + +/// EVM module trait +pub trait Trait: system::Trait + timestamp::Trait { + /// Calculator for current gas price. + type FeeCalculator: FeeCalculator; + /// Convert account ID to H160; + type ConvertAccountId: ConvertAccountId; + /// Currency type for deposit and withdraw. + type Currency: Currency; + /// The overarching event type. + type Event: From + Into<::Event>; + /// Precompiles associated with this EVM engine. + type Precompiles: Precompiles; +} + +decl_storage! { + trait Store for Module as Example { + Accounts get(fn accounts) config(): map H160 => Account; + AccountCodes: map H160 => Vec; + AccountStorages: double_map H160, blake2_256(H256) => H256; + } +} + +decl_event!( + /// EVM events + pub enum Event { + /// Ethereum events from contracts. + Log(Log), + } +); + +decl_module! { + pub struct Module for enum Call where origin: T::Origin { + fn deposit_event() = default; + + #[weight = SimpleDispatchInfo::FixedNormal(10_000)] + fn deposit_balance(origin, value: BalanceOf) -> Result { + let sender = ensure_signed(origin)?; + + let imbalance = T::Currency::withdraw( + &sender, + value, + WithdrawReason::Reserve.into(), + ExistenceRequirement::AllowDeath, + )?; + T::Currency::resolve_creating(&Self::account_id(), imbalance); + + let bvalue = U256::from(UniqueSaturatedInto::::unique_saturated_into(value)); + let address = T::ConvertAccountId::convert_account_id(&sender); + Accounts::mutate(&address, |account| { + account.balance += bvalue; + }); + + Ok(()) + } + + #[weight = SimpleDispatchInfo::FixedNormal(10_000)] + fn withdraw_balance(origin, value: BalanceOf) -> Result { + let sender = ensure_signed(origin)?; + let address = T::ConvertAccountId::convert_account_id(&sender); + let bvalue = U256::from(UniqueSaturatedInto::::unique_saturated_into(value)); + + let mut account = Accounts::get(&address); + account.balance = account.balance.checked_sub(bvalue) + .ok_or("Not enough balance to withdraw")?; + + let imbalance = T::Currency::withdraw( + &Self::account_id(), + value, + WithdrawReason::Reserve.into(), + ExistenceRequirement::AllowDeath + )?; + + Accounts::insert(&address, account); + + T::Currency::resolve_creating(&sender, imbalance); + + Ok(()) + } + + #[weight = SimpleDispatchInfo::FixedNormal(10_000)] + fn call(origin, target: H160, input: Vec, value: U256, gas_limit: u32) -> Result { + let sender = ensure_signed(origin)?; + let source = T::ConvertAccountId::convert_account_id(&sender); + let gas_price = T::FeeCalculator::gas_price(); + + let vicinity = Vicinity { + gas_price, + origin: source, + }; + + let mut backend = Backend::::new(&vicinity); + let mut executor = StackExecutor::new_with_precompile( + &backend, + gas_limit as usize, + &backend::GASOMETER_CONFIG, + T::Precompiles::execute, + ); + + let total_fee = gas_price.checked_mul(U256::from(gas_limit)) + .ok_or("Calculating total fee overflowed")?; + if Accounts::get(&source).balance < + value.checked_add(total_fee).ok_or("Calculating total payment overflowed")? + { + return Err("Not enough balance to pay transaction fee") + } + executor.withdraw(source, total_fee).map_err(|_| "Withdraw fee failed")?; + + let reason = executor.transact_call( + source, + target, + value, + input, + gas_limit as usize, + ); + + let ret = match reason { + ExitReason::Succeed(_) => Ok(()), + ExitReason::Error(_) => Err("Execute message call failed"), + ExitReason::Revert(_) => Err("Execute message call reverted"), + ExitReason::Fatal(_) => Err("Execute message call returned VM fatal error"), + }; + let actual_fee = executor.fee(gas_price); + executor.deposit(source, total_fee.saturating_sub(actual_fee)); + + let (values, logs) = executor.deconstruct(); + backend.apply(values, logs, true); + + ret + } + + #[weight = SimpleDispatchInfo::FixedNormal(10_000)] + fn create(origin, init: Vec, value: U256, gas_limit: u32) -> Result { + let sender = ensure_signed(origin)?; + let source = T::ConvertAccountId::convert_account_id(&sender); + let gas_price = T::FeeCalculator::gas_price(); + + let vicinity = Vicinity { + gas_price, + origin: source, + }; + + let mut backend = Backend::::new(&vicinity); + let mut executor = StackExecutor::new_with_precompile( + &backend, + gas_limit as usize, + &backend::GASOMETER_CONFIG, + T::Precompiles::execute, + ); + + let total_fee = gas_price.checked_mul(U256::from(gas_limit)) + .ok_or("Calculating total fee overflowed")?; + if Accounts::get(&source).balance < + value.checked_add(total_fee).ok_or("Calculating total payment overflowed")? + { + return Err("Not enough balance to pay transaction fee") + } + executor.withdraw(source, total_fee).map_err(|_| "Withdraw fee failed")?; + + let reason = executor.transact_create( + source, + value, + init, + gas_limit as usize, + ); + + let ret = match reason { + ExitReason::Succeed(_) => Ok(()), + ExitReason::Error(_) => Err("Execute contract creation failed"), + ExitReason::Revert(_) => Err("Execute contract creation reverted"), + ExitReason::Fatal(_) => Err("Execute contract creation returned VM fatal error"), + }; + let actual_fee = executor.fee(gas_price); + executor.deposit(source, total_fee.saturating_sub(actual_fee)); + + let (values, logs) = executor.deconstruct(); + backend.apply(values, logs, true); + + ret + } + } +} + +impl Module { + /// The account ID of the EVM module. + /// + /// This actually does computation. If you need to keep using it, then make sure you cache the + /// value and only call this once. + pub fn account_id() -> T::AccountId { + MODULE_ID.into_account() + } + + /// Check whether an account is empty. + pub fn is_account_empty(address: &H160) -> bool { + let account = Accounts::get(address); + let code_len = AccountCodes::decode_len(address).unwrap_or(0); + + account.nonce == U256::zero() && + account.balance == U256::zero() && + code_len == 0 + } + + /// Remove an account if its empty. + pub fn remove_account_if_empty(address: &H160) { + if Self::is_account_empty(address) { + Self::remove_account(address) + } + } + + /// Remove an account from state. + fn remove_account(address: &H160) { + Accounts::remove(address); + AccountCodes::remove(address); + AccountStorages::remove_prefix(address); + } +} diff --git a/srml/example/Cargo.toml b/paint/example/Cargo.toml similarity index 50% rename from srml/example/Cargo.toml rename to paint/example/Cargo.toml index 695e0602fb327..c25e91235f67b 100644 --- a/srml/example/Cargo.toml +++ b/paint/example/Cargo.toml @@ -1,5 +1,5 @@ [package] -name = "srml-example" +name = "paint-example" version = "2.0.0" authors = ["Parity Technologies "] edition = "2018" @@ -7,15 +7,15 @@ edition = "2018" [dependencies] serde = { version = "1.0.101", optional = true } codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false } -support = { package = "srml-support", path = "../support", default-features = false } -system = { package = "srml-system", path = "../system", default-features = false } -balances = { package = "srml-balances", path = "../balances", default-features = false } -sr-primitives = { path = "../../core/sr-primitives", default-features = false } -rstd = { package = "sr-std", path = "../../core/sr-std", default-features = false } -runtime-io = { package = "sr-io", path = "../../core/sr-io", default-features = false } +support = { package = "paint-support", path = "../support", default-features = false } +system = { package = "paint-system", path = "../system", default-features = false } +balances = { package = "paint-balances", path = "../balances", default-features = false } +sr-primitives = { path = "../../primitives/sr-primitives", default-features = false } +rstd = { package = "sr-std", path = "../../primitives/sr-std", default-features = false } +runtime-io = { package = "sr-io", path = "../../primitives/sr-io", default-features = false } [dev-dependencies] -primitives = { package = "substrate-primitives", path = "../../core/primitives" } +primitives = { package = "substrate-primitives", path = "../../primitives/core" } [features] default = ["std"] diff --git a/srml/example/src/lib.rs b/paint/example/src/lib.rs similarity index 97% rename from srml/example/src/lib.rs rename to paint/example/src/lib.rs index fb05e731bdd4d..2fdee498e06bb 100644 --- a/srml/example/src/lib.rs +++ b/paint/example/src/lib.rs @@ -20,7 +20,7 @@ //! The Example: A simple example of a runtime module demonstrating //! concepts, APIs and structures common to most runtime modules. //! -//! Run `cargo doc --package srml-example --open` to view this module's documentation. +//! Run `cargo doc --package paint-example --open` to view this module's documentation. //! //! ### Documentation Guidelines: //! @@ -49,8 +49,8 @@ //! //! ### Documentation Template:
//! -//! Copy and paste this template from srml/example/src/lib.rs into file -//! `srml//src/lib.rs` of your own custom module and complete it. +//! Copy and paste this template from paint/example/src/lib.rs into file +//! `paint//src/lib.rs` of your own custom module and complete it. //! 

 //! // Add heading with custom module name
 //!
@@ -489,7 +489,7 @@ decl_module! {
 		// calls to be executed - we don't need to care why. Because it's privileged, we can
 		// assume it's a one-off operation and substantial processing/storage/memory can be used
 		// without worrying about gameability or attack scenarios.
-		// If you not specify `Result` explicitly as return value, it will be added automatically
+		// If you do not specify `Result` explicitly as return value, it will be added automatically
 		// for you and `Ok(())` will be returned.
 		#[weight = WeightForSetDummy::(>::from(100u32))]
 		fn set_dummy(origin, #[compact] new_value: T::Balance) {
@@ -498,13 +498,18 @@ decl_module! {
 			>::put(new_value);
 		}
 
-		// The signature could also look like: `fn on_initialize()`
+		// The signature could also look like: `fn on_initialize()`.
+		// This function could also very well have a weight annotation, similar to any other. The
+		// only difference being that if it is not annotated, the default is
+		// `SimpleDispatchInfo::zero()`, which resolves into no weight.
+		#[weight = SimpleDispatchInfo::FixedNormal(1000)]
 		fn on_initialize(_n: T::BlockNumber) {
 			// Anything that needs to be done at the start of the block.
 			// We don't do anything here.
 		}
 
 		// The signature could also look like: `fn on_finalize()`
+		#[weight = SimpleDispatchInfo::FixedNormal(2000)]
 		fn on_finalize(_n: T::BlockNumber) {
 			// Anything that needs to be done at the end of the block.
 			// We just kill our dummy storage item.
diff --git a/paint/executive/Cargo.toml b/paint/executive/Cargo.toml
new file mode 100644
index 0000000000000..9dbea514d20ec
--- /dev/null
+++ b/paint/executive/Cargo.toml
@@ -0,0 +1,33 @@
+[package]
+name = "paint-executive"
+version = "2.0.0"
+authors = ["Parity Technologies "]
+edition = "2018"
+
+[dependencies]
+serde = { version = "1.0.101", optional = true }
+codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false, features = ["derive"] }
+rstd = { package = "sr-std", path = "../../primitives/sr-std", default-features = false }
+runtime-io ={ package = "sr-io", path = "../../primitives/sr-io", default-features = false }
+sr-primitives = { path = "../../primitives/sr-primitives", default-features = false }
+support = { package = "paint-support", path = "../support", default-features = false }
+system = { package = "paint-system", path = "../system", default-features = false }
+
+[dev-dependencies]
+hex-literal = "0.2.1"
+primitives = { package = "substrate-primitives",  path = "../../primitives/core" }
+paint-indices = { path = "../indices" }
+balances = { package = "paint-balances", path = "../balances" }
+transaction-payment = { package = "paint-transaction-payment", path = "../transaction-payment" }
+
+[features]
+default = ["std"]
+std = [
+	"rstd/std",
+	"support/std",
+	"serde",
+	"codec/std",
+	"sr-primitives/std",
+	"runtime-io/std",
+	"system/std",
+]
diff --git a/srml/executive/src/lib.rs b/paint/executive/src/lib.rs
similarity index 87%
rename from srml/executive/src/lib.rs
rename to paint/executive/src/lib.rs
index 927a7fef25581..2f0a418ab089b 100644
--- a/srml/executive/src/lib.rs
+++ b/paint/executive/src/lib.rs
@@ -23,7 +23,7 @@
 //!
 //! The executive module is not a typical SRML module providing functionality around a specific feature.
 //! It is a cross-cutting framework component for the SRML. It works in conjunction with the
-//! [SRML System module](../srml_system/index.html) to perform these cross-cutting functions.
+//! [SRML System module](../paint_system/index.html) to perform these cross-cutting functions.
 //!
 //! The Executive module provides functions to:
 //!
@@ -51,7 +51,7 @@
 //!
 //! ```
 //! # use sr_primitives::generic;
-//! # use srml_executive as executive;
+//! # use paint_executive as executive;
 //! # pub struct UncheckedExtrinsic {};
 //! # pub struct Header {};
 //! # type Context = system::ChainContext;
@@ -60,7 +60,9 @@
 //! # pub type AllModules = u64;
 //! # pub enum Runtime {};
 //! # use sr_primitives::transaction_validity::{TransactionValidity, UnknownTransaction};
+//!	# #[allow(deprecated)]
 //! # use sr_primitives::traits::ValidateUnsigned;
+//!	# #[allow(deprecated)]
 //! # impl ValidateUnsigned for Runtime {
 //! # 	type Call = ();
 //! #
@@ -76,13 +78,16 @@
 
 use rstd::{prelude::*, marker::PhantomData};
 use sr_primitives::{
-	generic::Digest, ApplyResult, weights::GetDispatchInfo,
+	generic::Digest, ApplyResult,
+	weights::{GetDispatchInfo, WeighBlock},
 	traits::{
 		self, Header, Zero, One, Checkable, Applyable, CheckEqual, OnFinalize, OnInitialize,
-		NumberFor, Block as BlockT, OffchainWorker, ValidateUnsigned, Dispatchable
+		NumberFor, Block as BlockT, OffchainWorker, Dispatchable,
 	},
 	transaction_validity::TransactionValidity,
 };
+#[allow(deprecated)]
+use sr_primitives::traits::ValidateUnsigned;
 use codec::{Codec, Encode};
 use system::{extrinsics_root, DigestOf};
 
@@ -100,12 +105,17 @@ pub struct Executive(
 	PhantomData<(System, Block, Context, UnsignedValidator, AllModules)>
 );
 
+#[allow(deprecated)] // Allow ValidateUnsigned, remove the attribute when the trait is removed.
 impl<
 	System: system::Trait,
 	Block: traits::Block,
 	Context: Default,
 	UnsignedValidator,
-	AllModules: OnInitialize + OnFinalize + OffchainWorker,
+	AllModules:
+		OnInitialize +
+		OnFinalize +
+		OffchainWorker +
+		WeighBlock,
 > ExecuteBlock for Executive
 where
 	Block::Extrinsic: Checkable + Codec,
@@ -119,12 +129,17 @@ where
 	}
 }
 
+#[allow(deprecated)] // Allow ValidateUnsigned, remove the attribute when the trait is removed.
 impl<
 	System: system::Trait,
 	Block: traits::Block,
 	Context: Default,
 	UnsignedValidator,
-	AllModules: OnInitialize + OnFinalize + OffchainWorker,
+	AllModules:
+		OnInitialize +
+		OnFinalize +
+		OffchainWorker +
+		WeighBlock,
 > Executive
 where
 	Block::Extrinsic: Checkable + Codec,
@@ -148,6 +163,12 @@ where
 	) {
 		>::initialize(block_number, parent_hash, extrinsics_root, digest);
 		>::on_initialize(*block_number);
+		>::register_extra_weight_unchecked(
+			>::on_initialize(*block_number)
+		);
+		>::register_extra_weight_unchecked(
+			>::on_finalize(*block_number)
+		);
 	}
 
 	fn initial_checks(block: &Block) {
@@ -242,7 +263,7 @@ where
 
 		// Decode parameters and dispatch
 		let dispatch_info = xt.get_dispatch_info();
-		let r = Applyable::apply(xt, dispatch_info, encoded_len)?;
+		let r = Applyable::apply::(xt, dispatch_info, encoded_len)?;
 
 		>::note_applied_extrinsic(&r, encoded_len as u32);
 
@@ -303,12 +324,48 @@ mod tests {
 		impl_outer_event, impl_outer_origin, parameter_types, impl_outer_dispatch,
 		traits::{Currency, LockIdentifier, LockableCurrency, WithdrawReasons, WithdrawReason},
 	};
-	use system::Call as SystemCall;
+	use system::{Call as SystemCall, ChainContext};
 	use balances::Call as BalancesCall;
 	use hex_literal::hex;
 
+	mod custom {
+		use sr_primitives::weights::SimpleDispatchInfo;
+
+		pub trait Trait: system::Trait {}
+
+		support::decl_module! {
+			pub struct Module for enum Call where origin: T::Origin {
+				#[weight = SimpleDispatchInfo::FixedNormal(100)]
+				fn some_function(origin) {
+					// NOTE: does not make any different.
+					let _ = system::ensure_signed(origin);
+				}
+				#[weight = SimpleDispatchInfo::FixedOperational(200)]
+				fn some_root_operation(origin) {
+					let _ = system::ensure_root(origin);
+				}
+				#[weight = SimpleDispatchInfo::FreeNormal]
+				fn some_unsigned_message(origin) {
+					let _ = system::ensure_none(origin);
+				}
+
+				// module hooks.
+				// one with block number arg and one without
+				#[weight = SimpleDispatchInfo::FixedNormal(25)]
+				fn on_initialize(n: T::BlockNumber) {
+					println!("on_initialize({})", n);
+				}
+				#[weight = SimpleDispatchInfo::FixedNormal(150)]
+				fn on_finalize() {
+					println!("on_finalize(?)");
+				}
+			}
+		}
+	}
+
 	type System = system::Module;
 	type Balances = balances::Module;
+	type Custom = custom::Module;
 
 	impl_outer_origin! {
 		pub enum Origin for Runtime { }
@@ -326,7 +383,6 @@ mod tests {
 		}
 	}
 
-	// Workaround for https://github.com/rust-lang/rust/issues/26925 . Remove when sorted.
 	#[derive(Clone, Eq, PartialEq)]
 	pub struct Runtime;
 	parameter_types! {
@@ -381,10 +437,16 @@ mod tests {
 		type WeightToFee = ConvertInto;
 		type FeeMultiplierUpdate = ();
 	}
+	impl custom::Trait for Runtime {}
 
+	#[allow(deprecated)]
 	impl ValidateUnsigned for Runtime {
 		type Call = Call;
 
+		fn pre_dispatch(_call: &Self::Call) -> Result<(), ApplyError> {
+			Ok(())
+		}
+
 		fn validate_unsigned(call: &Self::Call) -> TransactionValidity {
 			match call {
 				Call::Balances(BalancesCall::set_balance(_, _, _)) => Ok(Default::default()),
@@ -399,8 +461,9 @@ mod tests {
 		system::CheckWeight,
 		transaction_payment::ChargeTransactionPayment
 	);
+	type AllModules = (System, Balances, Custom);
 	type TestXt = sr_primitives::testing::TestXt;
-	type Executive = super::Executive, system::ChainContext, Runtime, ()>;
+	type Executive = super::Executive, ChainContext, Runtime, AllModules>;
 
 	fn extra(nonce: u64, fee: u64) -> SignedExtra {
 		(
@@ -524,7 +587,7 @@ mod tests {
 		let xt = sr_primitives::testing::TestXt(sign_extra(1, 0, 0), Call::Balances(BalancesCall::transfer(33, 0)));
 		let encoded = xt.encode();
 		let encoded_len = encoded.len() as Weight;
-		let limit = AvailableBlockRatio::get() * MaximumBlockWeight::get();
+		let limit = AvailableBlockRatio::get() * MaximumBlockWeight::get() - 175;
 		let num_to_exhaust_block = limit / encoded_len;
 		t.execute_with(|| {
 			Executive::initialize_block(&Header::new(
@@ -534,7 +597,8 @@ mod tests {
 				[69u8; 32].into(),
 				Digest::default(),
 			));
-			assert_eq!(>::all_extrinsics_weight(), 0);
+			// Initial block weight form the custom module.
+			assert_eq!(>::all_extrinsics_weight(), 175);
 
 			for nonce in 0..=num_to_exhaust_block {
 				let xt = sr_primitives::testing::TestXt(
@@ -545,7 +609,7 @@ mod tests {
 					assert!(res.is_ok());
 					assert_eq!(
 						>::all_extrinsics_weight(),
-						encoded_len * (nonce + 1),
+						encoded_len * (nonce + 1) + 175,
 					);
 					assert_eq!(>::extrinsic_index(), Some(nonce as u32 + 1));
 				} else {
@@ -642,4 +706,15 @@ mod tests {
 		execute_with_lock(WithdrawReasons::all());
 		execute_with_lock(WithdrawReasons::except(WithdrawReason::TransactionPayment));
 	}
+
+	#[test]
+	fn block_hooks_weight_is_stored() {
+		new_test_ext(0).execute_with(|| {
+
+			Executive::initialize_block(&Header::new_from_number(1));
+			// NOTE: might need updates over time if system and balance introduce new weights. For
+			// now only accounts for the custom module.
+			assert_eq!(>::all_extrinsics_weight(), 150 + 25);
+		})
+	}
 }
diff --git a/paint/finality-tracker/Cargo.toml b/paint/finality-tracker/Cargo.toml
new file mode 100644
index 0000000000000..f11fa2c4d9159
--- /dev/null
+++ b/paint/finality-tracker/Cargo.toml
@@ -0,0 +1,31 @@
+[package]
+name = "paint-finality-tracker"
+version = "2.0.0"
+authors = ["Parity Technologies "]
+edition = "2018"
+
+[dependencies]
+serde = { version = "1.0.101", default-features = false, features = ["derive"] }
+codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false }
+inherents = { package = "substrate-inherents", path = "../../primitives/inherents", default-features = false }
+rstd = { package = "sr-std", path = "../../primitives/sr-std", default-features = false }
+sr-primitives = { path = "../../primitives/sr-primitives", default-features = false }
+support = { package = "paint-support", path = "../support", default-features = false }
+paint-system = { path = "../system", default-features = false }
+impl-trait-for-tuples = "0.1.3"
+
+[dev-dependencies]
+primitives = { package = "substrate-primitives",  path = "../../primitives/core", default-features = false }
+runtime-io = { package = "sr-io", path = "../../primitives/sr-io", default-features = false }
+
+[features]
+default = ["std"]
+std = [
+	"serde/std",
+	"codec/std",
+	"rstd/std",
+	"support/std",
+	"sr-primitives/std",
+	"paint-system/std",
+	"inherents/std",
+]
diff --git a/srml/finality-tracker/src/lib.rs b/paint/finality-tracker/src/lib.rs
similarity index 95%
rename from srml/finality-tracker/src/lib.rs
rename to paint/finality-tracker/src/lib.rs
index 5e973cd43235d..e0721e5a1419f 100644
--- a/srml/finality-tracker/src/lib.rs
+++ b/paint/finality-tracker/src/lib.rs
@@ -18,16 +18,13 @@
 
 #![cfg_attr(not(feature = "std"), no_std)]
 
-use inherents::{
-	RuntimeString, InherentIdentifier, ProvideInherent,
-	InherentData, MakeFatalError,
-};
+use inherents::{InherentIdentifier, ProvideInherent, InherentData, MakeFatalError};
 use sr_primitives::traits::{One, Zero, SaturatedConversion};
 use rstd::{prelude::*, result, cmp, vec};
 use codec::Decode;
 use support::{decl_module, decl_storage};
 use support::traits::Get;
-use srml_system::{ensure_none, Trait as SystemTrait};
+use paint_system::{ensure_none, Trait as SystemTrait};
 
 #[cfg(feature = "std")]
 use codec::Encode;
@@ -38,11 +35,11 @@ pub const INHERENT_IDENTIFIER: InherentIdentifier = *b"finalnum";
 /// Auxiliary trait to extract finalized inherent data.
 pub trait FinalizedInherentData {
 	/// Get finalized inherent data.
-	fn finalized_number(&self) -> Result;
+	fn finalized_number(&self) -> Result;
 }
 
 impl FinalizedInherentData for InherentData {
-	fn finalized_number(&self) -> Result {
+	fn finalized_number(&self) -> Result {
 		self.get_data(&INHERENT_IDENTIFIER)
 			.and_then(|r| r.ok_or_else(|| "Finalized number inherent data not found".into()))
 	}
@@ -64,13 +61,16 @@ impl InherentDataProvider {
 
 #[cfg(feature = "std")]
 impl inherents::ProvideInherentData for InherentDataProvider
-	where F: Fn() -> Result
+	where F: Fn() -> Result
 {
 	fn inherent_identifier(&self) -> &'static InherentIdentifier {
 		&INHERENT_IDENTIFIER
 	}
 
-	fn provide_inherent_data(&self, inherent_data: &mut InherentData) -> Result<(), RuntimeString> {
+	fn provide_inherent_data(
+		&self,
+		inherent_data: &mut InherentData,
+	) -> Result<(), inherents::Error> {
 		(self.inner)()
 			.and_then(|n| inherent_data.put_data(INHERENT_IDENTIFIER, &n))
 	}
@@ -124,7 +124,7 @@ decl_module! {
 			ensure_none(origin)?;
 			assert!(!::Update::exists(), "Final hint must be updated only once in the block");
 			assert!(
-				srml_system::Module::::block_number() >= hint,
+				paint_system::Module::::block_number() >= hint,
 				"Finalized height above block number",
 			);
 			::Update::put(hint);
@@ -199,7 +199,7 @@ impl Module {
 		::Median::put(median);
 
 		if T::BlockNumber::from(our_window_size) == window_size {
-			let now = srml_system::Module::::block_number();
+			let now = paint_system::Module::::block_number();
 			let latency = T::ReportLatency::get();
 
 			// the delay is the latency plus half the window size.
@@ -254,7 +254,7 @@ mod tests {
 		traits::{BlakeTwo256, IdentityLookup, OnFinalize, Header as HeaderT},
 	};
 	use support::{assert_ok, impl_outer_origin, parameter_types};
-	use srml_system as system;
+	use paint_system as system;
 	use std::cell::RefCell;
 
 	#[derive(Clone, PartialEq, Debug)]
diff --git a/srml/generic-asset/Cargo.toml b/paint/generic-asset/Cargo.toml
similarity index 57%
rename from srml/generic-asset/Cargo.toml
rename to paint/generic-asset/Cargo.toml
index 49a9db4c0ecd8..db0bae7a46bff 100644
--- a/srml/generic-asset/Cargo.toml
+++ b/paint/generic-asset/Cargo.toml
@@ -1,5 +1,5 @@
 [package]
-name = "srml-generic-asset"
+name = "paint-generic-asset"
 version = "2.0.0"
 authors = ["Centrality Developers "]
 edition = "2018"
@@ -7,14 +7,14 @@ edition = "2018"
 [dependencies]
 serde = { version = "1.0.101", optional = true }
 codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false, features = ["derive"] }
-rstd = { package = "sr-std", path = "../../core/sr-std", default-features = false }
-sr-primitives = { path = "../../core/sr-primitives", default-features = false }
-support = { package = "srml-support", path = "../support", default-features = false }
-system = { package = "srml-system", path = "../system", default-features = false }
+rstd = { package = "sr-std", path = "../../primitives/sr-std", default-features = false }
+sr-primitives = { path = "../../primitives/sr-primitives", default-features = false }
+support = { package = "paint-support", path = "../support", default-features = false }
+system = { package = "paint-system", path = "../system", default-features = false }
 
 [dev-dependencies]
-runtime-io ={ package = "sr-io", path = "../../core/sr-io" }
-primitives = { package = "substrate-primitives",  path = "../../core/primitives" }
+runtime-io ={ package = "sr-io", path = "../../primitives/sr-io" }
+primitives = { package = "substrate-primitives",  path = "../../primitives/core" }
 
 [features]
 default = ["std"]
diff --git a/srml/generic-asset/src/lib.rs b/paint/generic-asset/src/lib.rs
similarity index 98%
rename from srml/generic-asset/src/lib.rs
rename to paint/generic-asset/src/lib.rs
index 38bff08e12d0b..a552880a34b62 100644
--- a/srml/generic-asset/src/lib.rs
+++ b/paint/generic-asset/src/lib.rs
@@ -155,7 +155,8 @@ use codec::{Decode, Encode, HasCompact, Input, Output, Error};
 
 use sr_primitives::RuntimeDebug;
 use sr_primitives::traits::{
-	CheckedAdd, CheckedSub, MaybeSerializeDeserialize, Member, One, Saturating, SimpleArithmetic, Zero, Bounded
+	CheckedAdd, CheckedSub, MaybeSerializeDeserialize, Member, One, Saturating, SimpleArithmetic,
+	Zero, Bounded,
 };
 
 use rstd::prelude::*;
@@ -165,7 +166,7 @@ use support::{
 	decl_event, decl_module, decl_storage, ensure,
 	traits::{
 		Currency, ExistenceRequirement, Imbalance, LockIdentifier, LockableCurrency, ReservableCurrency,
-		SignedImbalance, UpdateBalanceOutcome, WithdrawReason, WithdrawReasons,
+		SignedImbalance, UpdateBalanceOutcome, WithdrawReason, WithdrawReasons, TryDrop,
 	},
 	Parameter, StorageMap,
 };
@@ -862,7 +863,9 @@ pub trait AssetIdProvider {
 // wrapping these imbalanes in a private module is necessary to ensure absolute privacy
 // of the inner member.
 mod imbalances {
-	use super::{result, AssetIdProvider, Imbalance, Saturating, StorageMap, Subtrait, Zero};
+	use super::{
+		result, AssetIdProvider, Imbalance, Saturating, StorageMap, Subtrait, Zero, TryDrop
+	};
 	use rstd::mem;
 
 	/// Opaque, move-only struct with private fields that serves as a token denoting that
@@ -899,6 +902,16 @@ mod imbalances {
 		}
 	}
 
+	impl TryDrop for PositiveImbalance
+	where
+		T: Subtrait,
+		U: AssetIdProvider,
+	{
+		fn try_drop(self) -> result::Result<(), Self> {
+			self.drop_zero()
+		}
+	}
+
 	impl Imbalance for PositiveImbalance
 	where
 		T: Subtrait,
@@ -948,6 +961,16 @@ mod imbalances {
 		}
 	}
 
+	impl TryDrop for NegativeImbalance
+	where
+		T: Subtrait,
+		U: AssetIdProvider,
+	{
+		fn try_drop(self) -> result::Result<(), Self> {
+			self.drop_zero()
+		}
+	}
+
 	impl Imbalance for NegativeImbalance
 	where
 		T: Subtrait,
@@ -1096,7 +1119,12 @@ where
 		Zero::zero()
 	}
 
-	fn transfer(transactor: &T::AccountId, dest: &T::AccountId, value: Self::Balance) -> Result {
+	fn transfer(
+		transactor: &T::AccountId,
+		dest: &T::AccountId,
+		value: Self::Balance,
+		_: ExistenceRequirement, // no existential deposit policy for generic asset
+	) -> Result {
 		>::make_transfer(&U::asset_id(), transactor, dest, value)
 	}
 
diff --git a/srml/generic-asset/src/mock.rs b/paint/generic-asset/src/mock.rs
similarity index 100%
rename from srml/generic-asset/src/mock.rs
rename to paint/generic-asset/src/mock.rs
diff --git a/srml/generic-asset/src/tests.rs b/paint/generic-asset/src/tests.rs
similarity index 100%
rename from srml/generic-asset/src/tests.rs
rename to paint/generic-asset/src/tests.rs
diff --git a/paint/grandpa/Cargo.toml b/paint/grandpa/Cargo.toml
new file mode 100644
index 0000000000000..956478b41af45
--- /dev/null
+++ b/paint/grandpa/Cargo.toml
@@ -0,0 +1,38 @@
+[package]
+name = "paint-grandpa"
+version = "2.0.0"
+authors = ["Parity Technologies "]
+edition = "2018"
+
+[dependencies]
+serde = { version = "1.0.101", optional = true, features = ["derive"] }
+codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false, features = ["derive"] }
+primitives = { package = "substrate-primitives",  path = "../../primitives/core", default-features = false }
+substrate-finality-grandpa-primitives = { path = "../../primitives/finality-grandpa", default-features = false }
+rstd = { package = "sr-std", path = "../../primitives/sr-std", default-features = false }
+sr-primitives = { path = "../../primitives/sr-primitives", default-features = false }
+sr-staking-primitives = { path = "../../primitives/sr-staking-primitives", default-features = false }
+support = { package = "paint-support", path = "../support", default-features = false }
+system = { package = "paint-system", path = "../system", default-features = false }
+session = { package = "paint-session", path = "../session", default-features = false }
+finality-tracker = { package = "paint-finality-tracker", path = "../finality-tracker", default-features = false }
+
+[dev-dependencies]
+runtime-io ={ package = "sr-io", path = "../../primitives/sr-io" }
+
+[features]
+default = ["std"]
+std = [
+	"serde",
+	"codec/std",
+	"primitives/std",
+	"substrate-finality-grandpa-primitives/std",
+	"rstd/std",
+	"support/std",
+	"sr-primitives/std",
+	"sr-staking-primitives/std",
+	"system/std",
+	"session/std",
+	"finality-tracker/std",
+]
+migrate-authorities = []
diff --git a/srml/grandpa/src/lib.rs b/paint/grandpa/src/lib.rs
similarity index 90%
rename from srml/grandpa/src/lib.rs
rename to paint/grandpa/src/lib.rs
index f3e876f2c4e0e..877521c9746d9 100644
--- a/srml/grandpa/src/lib.rs
+++ b/paint/grandpa/src/lib.rs
@@ -32,9 +32,7 @@ pub use substrate_finality_grandpa_primitives as fg_primitives;
 
 use rstd::prelude::*;
 use codec::{self as codec, Encode, Decode, Error};
-use support::{
-	decl_event, decl_storage, decl_module, dispatch::Result,
-};
+use support::{decl_event, decl_storage, decl_module, dispatch::Result, storage};
 use sr_primitives::{
 	generic::{DigestItem, OpaqueDigestItemId}, traits::Zero, Perbill,
 };
@@ -42,8 +40,10 @@ use sr_staking_primitives::{
 	SessionIndex,
 	offence::{Offence, Kind},
 };
-use fg_primitives::{GRANDPA_ENGINE_ID, ScheduledChange, ConsensusLog, SetId, RoundNumber};
-pub use fg_primitives::{AuthorityId, AuthorityWeight};
+use fg_primitives::{
+	GRANDPA_AUTHORITIES_KEY, GRANDPA_ENGINE_ID, ScheduledChange, ConsensusLog, SetId, RoundNumber,
+};
+pub use fg_primitives::{AuthorityId, AuthorityList, AuthorityWeight, VersionedAuthorityList};
 use system::{ensure_signed, DigestOf};
 
 mod mock;
@@ -64,7 +64,7 @@ pub struct OldStoredPendingChange {
 	/// The delay in blocks until it will be applied.
 	pub delay: N,
 	/// The next authority set.
-	pub next_authorities: Vec<(AuthorityId, AuthorityWeight)>,
+	pub next_authorities: AuthorityList,
 }
 
 /// A stored pending change.
@@ -75,7 +75,7 @@ pub struct StoredPendingChange {
 	/// The delay in blocks until it will be applied.
 	pub delay: N,
 	/// The next authority set.
-	pub next_authorities: Vec<(AuthorityId, AuthorityWeight)>,
+	pub next_authorities: AuthorityList,
 	/// If defined it means the change was forced and the given block number
 	/// indicates the median last finalized block when the change was signaled.
 	pub forced: Option,
@@ -126,7 +126,7 @@ pub enum StoredState {
 decl_event!(
 	pub enum Event {
 		/// New authority set has been applied.
-		NewAuthorities(Vec<(AuthorityId, AuthorityWeight)>),
+		NewAuthorities(AuthorityList),
 		/// Current authority set has been paused.
 		Paused,
 		/// Current authority set has been resumed.
@@ -136,8 +136,12 @@ decl_event!(
 
 decl_storage! {
 	trait Store for Module as GrandpaFinality {
-		/// The current authority set.
-		Authorities get(fn authorities): Vec<(AuthorityId, AuthorityWeight)>;
+		/// DEPRECATED
+		///
+		/// This used to store the current authority set, which has been migrated to the well-known
+		/// GRANDPA_AUTHORITES_KEY unhashed key.
+		#[cfg(feature = "migrate-authorities")]
+		pub(crate) Authorities get(fn authorities): AuthorityList;
 
 		/// State of the current authority set.
 		State get(fn state): StoredState = StoredState::Live;
@@ -159,7 +163,7 @@ decl_storage! {
 		SetIdSession get(fn session_for_set): map SetId => Option;
 	}
 	add_extra_genesis {
-		config(authorities): Vec<(AuthorityId, AuthorityWeight)>;
+		config(authorities): AuthorityList;
 		build(|config| Module::::initialize_authorities(&config.authorities))
 	}
 }
@@ -174,6 +178,11 @@ decl_module! {
 			// FIXME: https://github.com/paritytech/substrate/issues/1112
 		}
 
+		fn on_initialize() {
+			#[cfg(feature = "migrate-authorities")]
+			Self::migrate_authorities();
+		}
+
 		fn on_finalize(block_number: T::BlockNumber) {
 			// check for scheduled pending authority set changes
 			if let Some(pending_change) = >::get() {
@@ -199,7 +208,7 @@ decl_module! {
 
 				// enact the change if we've reached the enacting block
 				if block_number == pending_change.scheduled_at + pending_change.delay {
-					Authorities::put(&pending_change.next_authorities);
+					Self::set_grandpa_authorities(&pending_change.next_authorities);
 					Self::deposit_event(
 						Event::NewAuthorities(pending_change.next_authorities)
 					);
@@ -241,8 +250,16 @@ decl_module! {
 
 impl Module {
 	/// Get the current set of authorities, along with their respective weights.
-	pub fn grandpa_authorities() -> Vec<(AuthorityId, AuthorityWeight)> {
-		Authorities::get()
+	pub fn grandpa_authorities() -> AuthorityList {
+		storage::unhashed::get_or_default::(GRANDPA_AUTHORITIES_KEY).into()
+	}
+
+	/// Set the current set of authorities, along with their respective weights.
+	fn set_grandpa_authorities(authorities: &AuthorityList) {
+		storage::unhashed::put(
+			GRANDPA_AUTHORITIES_KEY,
+			&VersionedAuthorityList::from(authorities),
+		);
 	}
 
 	/// Schedule GRANDPA to pause starting in the given number of blocks.
@@ -293,7 +310,7 @@ impl Module {
 	/// No change should be signaled while any change is pending. Returns
 	/// an error if a change is already pending.
 	pub fn schedule_change(
-		next_authorities: Vec<(AuthorityId, AuthorityWeight)>,
+		next_authorities: AuthorityList,
 		in_blocks: T::BlockNumber,
 		forced: Option,
 	) -> Result {
@@ -329,10 +346,20 @@ impl Module {
 		>::deposit_log(log.into());
 	}
 
-	fn initialize_authorities(authorities: &[(AuthorityId, AuthorityWeight)]) {
+	fn initialize_authorities(authorities: &AuthorityList) {
 		if !authorities.is_empty() {
-			assert!(Authorities::get().is_empty(), "Authorities are already initialized!");
-			Authorities::put(authorities);
+			assert!(
+				Self::grandpa_authorities().is_empty(),
+				"Authorities are already initialized!"
+			);
+			Self::set_grandpa_authorities(authorities);
+		}
+	}
+
+	#[cfg(feature = "migrate-authorities")]
+	fn migrate_authorities() {
+		if Authorities::exists() {
+			Self::set_grandpa_authorities(&Authorities::take());
 		}
 	}
 }
diff --git a/srml/grandpa/src/mock.rs b/paint/grandpa/src/mock.rs
similarity index 94%
rename from srml/grandpa/src/mock.rs
rename to paint/grandpa/src/mock.rs
index fcacbade20490..c6ea2075575c5 100644
--- a/srml/grandpa/src/mock.rs
+++ b/paint/grandpa/src/mock.rs
@@ -23,7 +23,7 @@ use runtime_io;
 use support::{impl_outer_origin, impl_outer_event, parameter_types};
 use primitives::H256;
 use codec::{Encode, Decode};
-use crate::{AuthorityId, GenesisConfig, Trait, Module, ConsensusLog};
+use crate::{AuthorityId, AuthorityList, GenesisConfig, Trait, Module, ConsensusLog};
 use substrate_finality_grandpa_primitives::GRANDPA_ENGINE_ID;
 
 impl_outer_origin!{
@@ -75,7 +75,7 @@ impl_outer_event!{
 	}
 }
 
-pub fn to_authorities(vec: Vec<(u64, u64)>) -> Vec<(AuthorityId, u64)> {
+pub fn to_authorities(vec: Vec<(u64, u64)>) -> AuthorityList {
 	vec.into_iter()
 		.map(|(id, weight)| (UintAuthorityId(id).to_public_key::(), weight))
 		.collect()
diff --git a/srml/grandpa/src/tests.rs b/paint/grandpa/src/tests.rs
similarity index 95%
rename from srml/grandpa/src/tests.rs
rename to paint/grandpa/src/tests.rs
index 2efeb4b5bf3cf..3d6a8752c5de3 100644
--- a/srml/grandpa/src/tests.rs
+++ b/paint/grandpa/src/tests.rs
@@ -308,3 +308,21 @@ fn time_slot_have_sane_ord() {
 	];
 	assert!(FIXTURE.windows(2).all(|f| f[0] < f[1]));
 }
+
+#[test]
+#[cfg(feature = "migrate-authorities")]
+fn authorities_migration() {
+	use sr_primitives::traits::OnInitialize;
+
+	with_externalities(&mut new_test_ext(vec![]), || {
+		let authorities = to_authorities(vec![(1, 1), (2, 1), (3, 1)]);
+
+		Authorities::put(authorities.clone());
+		assert!(Grandpa::grandpa_authorities().is_empty());
+
+		Grandpa::on_initialize(1);
+
+		assert!(!Authorities::exists());
+		assert_eq!(Grandpa::grandpa_authorities(), authorities);
+	});
+}
diff --git a/paint/im-online/Cargo.toml b/paint/im-online/Cargo.toml
new file mode 100644
index 0000000000000..e509620f6e037
--- /dev/null
+++ b/paint/im-online/Cargo.toml
@@ -0,0 +1,39 @@
+[package]
+name = "paint-im-online"
+version = "0.1.0"
+authors = ["Parity Technologies "]
+edition = "2018"
+
+[dependencies]
+app-crypto = { package = "substrate-application-crypto", path = "../../primitives/application-crypto", default-features = false }
+authorship = { package = "paint-authorship", path = "../authorship", default-features = false }
+codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false, features = ["derive"] }
+primitives = { package="substrate-primitives", path = "../../primitives/core", default-features = false }
+rstd = { package = "sr-std", path = "../../primitives/sr-std", default-features = false }
+serde = { version = "1.0.101", optional = true }
+session = { package = "paint-session", path = "../session", default-features = false }
+runtime-io = { package = "sr-io", path = "../../primitives/sr-io", default-features = false }
+sr-primitives = { path = "../../primitives/sr-primitives", default-features = false }
+sr-staking-primitives = { path = "../../primitives/sr-staking-primitives", default-features = false }
+support = { package = "paint-support", path = "../support", default-features = false }
+system = { package = "paint-system", path = "../system", default-features = false }
+
+[dev-dependencies]
+offchain = { package = "substrate-offchain", path = "../../client/offchain" }
+
+[features]
+default = ["std", "session/historical"]
+std = [
+	"app-crypto/std",
+	"authorship/std",
+	"codec/std",
+	"primitives/std",
+	"rstd/std",
+	"serde",
+	"session/std",
+	"runtime-io/std",
+	"sr-primitives/std",
+	"sr-staking-primitives/std",
+	"support/std",
+	"system/std",
+]
diff --git a/srml/im-online/src/lib.rs b/paint/im-online/src/lib.rs
similarity index 72%
rename from srml/im-online/src/lib.rs
rename to paint/im-online/src/lib.rs
index aeeaf74dc6551..4df8f442e4da4 100644
--- a/srml/im-online/src/lib.rs
+++ b/paint/im-online/src/lib.rs
@@ -26,7 +26,7 @@
 //!
 //! The heartbeat is a signed transaction, which was signed using the session key
 //! and includes the recent best block number of the local validators chain as well
-//! as the [NetworkState](../../core/offchain/struct.NetworkState.html).
+//! as the [NetworkState](../../client/offchain/struct.NetworkState.html).
 //! It is submitted as an Unsigned Transaction via off-chain workers.
 //!
 //! - [`im_online::Trait`](./trait.Trait.html)
@@ -37,14 +37,14 @@
 //!
 //! ### Public Functions
 //!
-//! - `is_online_in_current_session` - True if the validator sent a heartbeat in the current session.
+//! - `is_online` - True if the validator sent a heartbeat in the current session.
 //!
 //! ## Usage
 //!
 //! ```
 //! use support::{decl_module, dispatch::Result};
 //! use system::ensure_signed;
-//! use srml_im_online::{self as im_online};
+//! use paint_im_online::{self as im_online};
 //!
 //! pub trait Trait: im_online::Trait {}
 //!
@@ -52,7 +52,7 @@
 //! 	pub struct Module for enum Call where origin: T::Origin {
 //! 		pub fn is_online(origin, authority_index: u32) -> Result {
 //! 			let _sender = ensure_signed(origin)?;
-//! 			let _is_online = >::is_online_in_current_session(authority_index);
+//! 			let _is_online = >::is_online(authority_index);
 //! 			Ok(())
 //! 		}
 //! 	}
@@ -62,7 +62,7 @@
 //!
 //! ## Dependencies
 //!
-//! This module depends on the [Session module](../srml_session/index.html).
+//! This module depends on the [Session module](../paint_session/index.html).
 
 // Ensure we're `no_std` when compiling for Wasm.
 #![cfg_attr(not(feature = "std"), no_std)]
@@ -74,12 +74,14 @@ use app_crypto::RuntimeAppPublic;
 use codec::{Encode, Decode};
 use primitives::offchain::{OpaqueNetworkState, StorageKind};
 use rstd::prelude::*;
+use rstd::convert::TryInto;
 use session::historical::IdentificationTuple;
 use sr_primitives::{
 	RuntimeDebug,
 	traits::{Convert, Member, Printable, Saturating}, Perbill,
 	transaction_validity::{
-		TransactionValidity, TransactionLongevity, ValidTransaction, InvalidTransaction,
+		TransactionValidity, ValidTransaction, InvalidTransaction,
+		TransactionPriority,
 	},
 };
 use sr_staking_primitives::{
@@ -87,7 +89,8 @@ use sr_staking_primitives::{
 	offence::{ReportOffence, Offence, Kind},
 };
 use support::{
-	decl_module, decl_event, decl_storage, print, ensure, Parameter, debug
+	decl_module, decl_event, decl_storage, print, Parameter, debug,
+	traits::Get,
 };
 use system::ensure_none;
 use system::offchain::SubmitUnsignedTransaction;
@@ -128,7 +131,7 @@ pub mod ed25519 {
 
 /// The local storage database key under which the worker progress status
 /// is tracked.
-const DB_KEY: &[u8] = b"srml/im-online-worker-status";
+const DB_KEY: &[u8] = b"paint/im-online-worker-status";
 
 /// It's important to persist the worker state, since e.g. the
 /// server could be restarted while starting the gossip process, but before
@@ -187,6 +190,12 @@ pub trait Trait: system::Trait + session::historical::Trait {
 	/// A transaction submitter.
 	type SubmitTransaction: SubmitUnsignedTransaction::Call>;
 
+	/// An expected duration of the session.
+	///
+	/// This parameter is used to determine the longevity of `heartbeat` transaction
+	/// and a rough time when the heartbeat should be sent.
+	type SessionDuration: Get;
+
 	/// A type that gives us the ability to submit unresponsiveness offence reports.
 	type ReportUnresponsiveness:
 		ReportOffence<
@@ -199,9 +208,14 @@ pub trait Trait: system::Trait + session::historical::Trait {
 decl_event!(
 	pub enum Event where
 		::AuthorityId,
+		IdentificationTuple = IdentificationTuple,
 	{
 		/// A new heartbeat was received from `AuthorityId`
 		HeartbeatReceived(AuthorityId),
+		/// At the end of the session, no offence was committed.
+		AllGood,
+		/// At the end of the session, at least once validator was found to be offline.
+		SomeOffline(Vec),
 	}
 );
 
@@ -213,10 +227,15 @@ decl_storage! {
 		/// The current set of keys that may issue a heartbeat.
 		Keys get(fn keys): Vec;
 
-		/// For each session index we keep a mapping of `AuthorityId`
+		/// For each session index, we keep a mapping of `AuthIndex`
 		/// to `offchain::OpaqueNetworkState`.
 		ReceivedHeartbeats get(fn received_heartbeats): double_map SessionIndex,
-			blake2_256(AuthIndex) => Vec;
+			blake2_256(AuthIndex) => Option>;
+
+		/// For each session index, we keep a mapping of `T::ValidatorId` to the
+		/// number of blocks authored by the given authority.
+		AuthoredBlocks get(fn authored_blocks): double_map SessionIndex,
+			blake2_256(T::ValidatorId) => u32;
 	}
 	add_extra_genesis {
 		config(keys): Vec;
@@ -232,24 +251,20 @@ decl_module! {
 		fn heartbeat(
 			origin,
 			heartbeat: Heartbeat,
-			signature: ::Signature
+			// since signature verification is done in `validate_unsigned`
+			// we can skip doing it here again.
+			_signature: ::Signature
 		) {
 			ensure_none(origin)?;
 
 			let current_session = >::current_index();
-			ensure!(current_session == heartbeat.session_index, "Outdated heartbeat received.");
 			let exists = ::exists(
 				¤t_session,
 				&heartbeat.authority_index
 			);
 			let keys = Keys::::get();
 			let public = keys.get(heartbeat.authority_index as usize);
-			if let (true, Some(public)) = (!exists, public) {
-				let signature_valid = heartbeat.using_encoded(|encoded_heartbeat| {
-					public.verify(&encoded_heartbeat, &signature)
-				});
-				ensure!(signature_valid, "Invalid heartbeat signature.");
-
+			if let (false, Some(public)) = (exists, public) {
 				Self::deposit_event(Event::::HeartbeatReceived(public.clone()));
 
 				let network_state = heartbeat.network_state.encode();
@@ -270,21 +285,70 @@ decl_module! {
 			debug::RuntimeLogger::init();
 
 			// Only send messages if we are a potential validator.
-			if runtime_io::is_validator() {
+			if runtime_io::offchain::is_validator() {
 				Self::offchain(now);
 			}
 		}
 	}
 }
 
+/// Keep track of number of authored blocks per authority, uncles are counted as
+/// well since they're a valid proof of onlineness.
+impl authorship::EventHandler for Module {
+	fn note_author(author: T::ValidatorId) {
+		Self::note_authorship(author);
+	}
+
+	fn note_uncle(author: T::ValidatorId, _age: T::BlockNumber) {
+		Self::note_authorship(author);
+	}
+}
+
 impl Module {
+	/// Returns `true` if a heartbeat has been received for the authority at
+	/// `authority_index` in the authorities series or if the authority has
+	/// authored at least one block, during the current session. Otherwise
+	/// `false`.
+	pub fn is_online(authority_index: AuthIndex) -> bool {
+		let current_validators = >::validators();
+
+		if authority_index >= current_validators.len() as u32 {
+			return false;
+		}
+
+		let authority = ¤t_validators[authority_index as usize];
+
+		Self::is_online_aux(authority_index, authority)
+	}
+
+	fn is_online_aux(authority_index: AuthIndex, authority: &T::ValidatorId) -> bool {
+		let current_session = >::current_index();
+
+		::exists(¤t_session, &authority_index) ||
+			>::get(
+				¤t_session,
+				authority,
+			) != 0
+	}
+
 	/// Returns `true` if a heartbeat has been received for the authority at `authority_index` in
 	/// the authorities series, during the current session. Otherwise `false`.
-	pub fn is_online_in_current_session(authority_index: AuthIndex) -> bool {
+	pub fn received_heartbeat_in_current_session(authority_index: AuthIndex) -> bool {
 		let current_session = >::current_index();
 		::exists(¤t_session, &authority_index)
 	}
 
+	/// Note that the given authority has authored a block in the current session.
+	fn note_authorship(author: T::ValidatorId) {
+		let current_session = >::current_index();
+
+		>::mutate(
+			¤t_session,
+			author,
+			|authored| *authored += 1,
+		);
+	}
+
 	pub(crate) fn offchain(now: T::BlockNumber) {
 		let next_gossip = >::get();
 		let check = Self::check_not_yet_gossipped(now, next_gossip);
@@ -309,7 +373,7 @@ impl Module {
 				Err(err) => print(err),
 			}
 		} else {
-			debug::native::trace!(
+			debug::native::debug!(
 				target: "imonline",
 				"Skipping gossip at: {:?} >= {:?} || {:?}",
 				next_gossip,
@@ -322,6 +386,7 @@ impl Module {
 	fn do_gossip_at(block_number: T::BlockNumber) -> Result<(), OffchainErr> {
 		// we run only when a local authority key is configured
 		let authorities = Keys::::get();
+		let mut results = Vec::new();
 		let mut local_keys = T::AuthorityId::all();
 		local_keys.sort();
 
@@ -333,7 +398,18 @@ impl Module {
 					.map(|location| (index as u32, &local_keys[location]))
 			})
 		{
-			let network_state = runtime_io::network_state().map_err(|_| OffchainErr::NetworkState)?;
+			if Self::is_online(authority_index) {
+				debug::native::info!(
+					target: "imonline",
+					"[index: {:?}] Skipping sending heartbeat at block: {:?}. Already online.",
+					authority_index,
+					block_number
+				);
+				continue;
+			}
+
+			let network_state = runtime_io::offchain::network_state()
+				.map_err(|_| OffchainErr::NetworkState)?;
 			let heartbeat_data = Heartbeat {
 				block_number,
 				network_state,
@@ -350,14 +426,21 @@ impl Module {
 				authority_index,
 				block_number
 			);
-			T::SubmitTransaction::submit_unsigned(call)
-				.map_err(|_| OffchainErr::SubmitTransaction)?;
 
-			// once finished we set the worker status without comparing
-			// if the existing value changed in the meantime. this is
-			// because at this point the heartbeat was definitely submitted.
-			Self::set_worker_status(block_number, true);
+			results.push(
+				T::SubmitTransaction::submit_unsigned(call)
+					.map_err(|_| OffchainErr::SubmitTransaction)
+			);
 		}
+
+		// fail only after trying all keys.
+		results.into_iter().collect::, OffchainErr>>()?;
+
+		// once finished we set the worker status without comparing
+		// if the existing value changed in the meantime. this is
+		// because at this point the heartbeat was definitely submitted.
+		Self::set_worker_status(block_number, true);
+
 		Ok(())
 	}
 
@@ -370,10 +453,10 @@ impl Module {
 			done,
 			gossipping_at,
 		};
-		runtime_io::local_storage_compare_and_set(
+		runtime_io::offchain::local_storage_compare_and_set(
 			StorageKind::PERSISTENT,
 			DB_KEY,
-			curr_worker_status.as_ref().map(Vec::as_slice),
+			curr_worker_status,
 			&enc.encode()
 		)
 	}
@@ -386,8 +469,7 @@ impl Module {
 			done,
 			gossipping_at,
 		};
-		runtime_io::local_storage_set(
-			StorageKind::PERSISTENT, DB_KEY, &enc.encode());
+		runtime_io::offchain::local_storage_set(StorageKind::PERSISTENT, DB_KEY, &enc.encode());
 	}
 
 	// Checks if a heartbeat gossip already occurred at this block number.
@@ -397,7 +479,7 @@ impl Module {
 		now: T::BlockNumber,
 		next_gossip: T::BlockNumber,
 	) -> Result<(Option>, bool), OffchainErr> {
-		let last_gossip = runtime_io::local_storage_get(StorageKind::PERSISTENT, DB_KEY);
+		let last_gossip = runtime_io::offchain::local_storage_get(StorageKind::PERSISTENT, DB_KEY);
 		match last_gossip {
 			Some(last) => {
 				let worker_status: WorkerStatus = Decode::decode(&mut &last[..])
@@ -445,53 +527,43 @@ impl session::OneSessionHandler for Module {
 		where I: Iterator
 	{
 		// Tell the offchain worker to start making the next session's heartbeats.
-		>::put(>::block_number());
+		// Since we consider producing blocks as being online,
+		// the hearbeat is defered a bit to prevent spaming.
+		let block_number = >::block_number();
+		let half_session = T::SessionDuration::get() / 2.into();
+		>::put(block_number + half_session);
 
 		// Remember who the authorities are for the new session.
 		Keys::::put(validators.map(|x| x.1).collect::>());
 	}
 
 	fn on_before_session_ending() {
-		let mut unresponsive = vec![];
-
-		let current_session = >::current_index();
-
+		let session_index = >::current_index();
 		let keys = Keys::::get();
 		let current_validators = >::validators();
 
-		for (auth_idx, validator_id) in current_validators.into_iter().enumerate() {
-			let auth_idx = auth_idx as u32;
-			let exists = ::exists(¤t_session, &auth_idx);
-			if !exists {
-				let full_identification = T::FullIdentificationOf::convert(validator_id.clone())
-					.expect(
-						"we got the validator_id from current_validators;
-						current_validators is set of currently acting validators;
-						the mapping between the validator id and its full identification should be valid;
-						thus `FullIdentificationOf::convert` can't return `None`;
-						qed",
-					);
-
-				unresponsive.push((validator_id, full_identification));
-			}
-		}
-
-		if unresponsive.is_empty() {
-			return;
-		}
+		let offenders = current_validators.into_iter().enumerate()
+			.filter(|(index, id)|
+				!Self::is_online_aux(*index as u32, id)
+			).filter_map(|(_, id)|
+				T::FullIdentificationOf::convert(id.clone()).map(|full_id| (id, full_id))
+			).collect::>>();
 
-		let validator_set_count = keys.len() as u32;
-		let offence = UnresponsivenessOffence {
-			session_index: current_session,
-			validator_set_count,
-			offenders: unresponsive,
-		};
+		// Remove all received heartbeats and number of authored blocks from the
+		// current session, they have already been processed and won't be needed
+		// anymore.
+		::remove_prefix(&>::current_index());
+		>::remove_prefix(&>::current_index());
 
-		T::ReportUnresponsiveness::report_offence(vec![], offence);
+		if offenders.is_empty() {
+			Self::deposit_event(RawEvent::AllGood);
+		} else {
+			Self::deposit_event(RawEvent::SomeOffline(offenders.clone()));
 
-		// Remove all received heartbeats from the current session, they have
-		// already been processed and won't be needed anymore.
-		::remove_prefix(&>::current_index());
+			let validator_set_count = keys.len() as u32;
+			let offence = UnresponsivenessOffence { session_index, validator_set_count, offenders };
+			T::ReportUnresponsiveness::report_offence(vec![], offence);
+		}
 	}
 
 	fn on_disabled(_i: usize) {
@@ -499,12 +571,13 @@ impl session::OneSessionHandler for Module {
 	}
 }
 
+#[allow(deprecated)]
 impl support::unsigned::ValidateUnsigned for Module {
 	type Call = Call;
 
 	fn validate_unsigned(call: &Self::Call) -> TransactionValidity {
 		if let Call::heartbeat(heartbeat, signature) = call {
-			if >::is_online_in_current_session(heartbeat.authority_index) {
+			if >::is_online(heartbeat.authority_index) {
 				// we already received a heartbeat for this authority
 				return InvalidTransaction::Stale.into();
 			}
@@ -532,10 +605,10 @@ impl support::unsigned::ValidateUnsigned for Module {
 			}
 
 			Ok(ValidTransaction {
-				priority: 0,
+				priority: TransactionPriority::max_value(),
 				requires: vec![],
 				provides: vec![(current_session, authority_id).encode()],
-				longevity: TransactionLongevity::max_value(),
+				longevity: TryInto::::try_into(T::SessionDuration::get() / 2.into()).unwrap_or(64_u64),
 				propagate: true,
 			})
 		} else {
diff --git a/srml/im-online/src/mock.rs b/paint/im-online/src/mock.rs
similarity index 95%
rename from srml/im-online/src/mock.rs
rename to paint/im-online/src/mock.rs
index e50e7779e9fcd..4be33c44ed59e 100644
--- a/srml/im-online/src/mock.rs
+++ b/paint/im-online/src/mock.rs
@@ -145,12 +145,24 @@ impl session::historical::Trait for Runtime {
 	type FullIdentificationOf = ConvertInto;
 }
 
+parameter_types! {
+	pub const UncleGenerations: u32 = 5;
+}
+
+impl authorship::Trait for Runtime {
+	type FindAuthor = ();
+	type UncleGenerations = UncleGenerations;
+	type FilterUncle = ();
+	type EventHandler = ImOnline;
+}
+
 impl Trait for Runtime {
 	type AuthorityId = UintAuthorityId;
 	type Event = ();
 	type Call = Call;
 	type SubmitTransaction = SubmitTransaction;
 	type ReportUnresponsiveness = OffenceHandler;
+	type SessionDuration = Period;
 }
 
 /// Im Online module.
diff --git a/srml/im-online/src/tests.rs b/paint/im-online/src/tests.rs
similarity index 66%
rename from srml/im-online/src/tests.rs
rename to paint/im-online/src/tests.rs
index 652d751281294..382eb4f1d1f04 100644
--- a/srml/im-online/src/tests.rs
+++ b/paint/im-online/src/tests.rs
@@ -103,6 +103,9 @@ fn heartbeat(
 	authority_index: u32,
 	id: UintAuthorityId,
 ) -> dispatch::Result {
+	#[allow(deprecated)]
+	use support::unsigned::ValidateUnsigned;
+
 	let heartbeat = Heartbeat {
 		block_number,
 		network_state: OpaqueNetworkState {
@@ -114,6 +117,8 @@ fn heartbeat(
 	};
 	let signature = id.sign(&heartbeat.encode()).unwrap();
 
+	#[allow(deprecated)] // Allow ValidateUnsigned
+	ImOnline::pre_dispatch(&crate::Call::heartbeat(heartbeat.clone(), signature.clone()))?;
 	ImOnline::heartbeat(
 		Origin::system(system::RawOrigin::None),
 		heartbeat,
@@ -134,25 +139,25 @@ fn should_mark_online_validator_when_heartbeat_is_received() {
 		assert_eq!(Session::current_index(), 2);
 		assert_eq!(Session::validators(), vec![1, 2, 3]);
 
-		assert!(!ImOnline::is_online_in_current_session(0));
-		assert!(!ImOnline::is_online_in_current_session(1));
-		assert!(!ImOnline::is_online_in_current_session(2));
+		assert!(!ImOnline::is_online(0));
+		assert!(!ImOnline::is_online(1));
+		assert!(!ImOnline::is_online(2));
 
 		// when
 		let _ = heartbeat(1, 2, 0, 1.into()).unwrap();
 
 		// then
-		assert!(ImOnline::is_online_in_current_session(0));
-		assert!(!ImOnline::is_online_in_current_session(1));
-		assert!(!ImOnline::is_online_in_current_session(2));
+		assert!(ImOnline::is_online(0));
+		assert!(!ImOnline::is_online(1));
+		assert!(!ImOnline::is_online(2));
 
 		// and when
 		let _ = heartbeat(1, 2, 2, 3.into()).unwrap();
 
 		// then
-		assert!(ImOnline::is_online_in_current_session(0));
-		assert!(!ImOnline::is_online_in_current_session(1));
-		assert!(ImOnline::is_online_in_current_session(2));
+		assert!(ImOnline::is_online(0));
+		assert!(!ImOnline::is_online(1));
+		assert!(ImOnline::is_online(2));
 	});
 }
 
@@ -170,8 +175,8 @@ fn late_heartbeat_should_fail() {
 		assert_eq!(Session::validators(), vec![1, 2, 3]);
 
 		// when
-		assert_noop!(heartbeat(1, 3, 0, 1.into()), "Outdated heartbeat received.");
-		assert_noop!(heartbeat(1, 1, 0, 1.into()), "Outdated heartbeat received.");
+		assert_noop!(heartbeat(1, 3, 0, 1.into()), "Transaction is outdated");
+		assert_noop!(heartbeat(1, 1, 0, 1.into()), "Transaction is outdated");
 	});
 }
 
@@ -208,7 +213,7 @@ fn should_generate_heartbeats() {
 
 		assert_eq!(heartbeat, Heartbeat {
 			block_number: 2,
-			network_state: runtime_io::network_state().unwrap(),
+			network_state: runtime_io::offchain::network_state().unwrap(),
 			session_index: 2,
 			authority_index: 2,
 		});
@@ -233,13 +238,87 @@ fn should_cleanup_received_heartbeats_on_session_end() {
 		let _ = heartbeat(1, 2, 0, 1.into()).unwrap();
 
 		// the heartbeat is stored
-		assert!(!ImOnline::received_heartbeats(&2, &0).is_empty());
+		assert!(!ImOnline::received_heartbeats(&2, &0).is_none());
 
 		advance_session();
 
 		// after the session has ended we have already processed the heartbeat
 		// message, so any messages received on the previous session should have
 		// been pruned.
-		assert!(ImOnline::received_heartbeats(&2, &0).is_empty());
+		assert!(ImOnline::received_heartbeats(&2, &0).is_none());
+	});
+}
+
+#[test]
+fn should_mark_online_validator_when_block_is_authored() {
+	use authorship::EventHandler;
+
+	new_test_ext().execute_with(|| {
+		advance_session();
+		// given
+		VALIDATORS.with(|l| *l.borrow_mut() = Some(vec![1, 2, 3, 4, 5, 6]));
+		assert_eq!(Session::validators(), Vec::::new());
+		// enact the change and buffer another one
+		advance_session();
+
+		assert_eq!(Session::current_index(), 2);
+		assert_eq!(Session::validators(), vec![1, 2, 3]);
+
+		for i in 0..3 {
+			assert!(!ImOnline::is_online(i));
+		}
+
+		// when
+		ImOnline::note_author(1);
+		ImOnline::note_uncle(2, 0);
+
+		// then
+		assert!(ImOnline::is_online(0));
+		assert!(ImOnline::is_online(1));
+		assert!(!ImOnline::is_online(2));
+	});
+}
+
+#[test]
+fn should_not_send_a_report_if_already_online() {
+	use authorship::EventHandler;
+
+	let mut ext = new_test_ext();
+	let (offchain, state) = TestOffchainExt::new();
+	ext.register_extension(OffchainExt::new(offchain));
+
+	ext.execute_with(|| {
+		advance_session();
+		// given
+		VALIDATORS.with(|l| *l.borrow_mut() = Some(vec![1, 2, 3, 4, 5, 6]));
+		assert_eq!(Session::validators(), Vec::::new());
+		// enact the change and buffer another one
+		advance_session();
+		assert_eq!(Session::current_index(), 2);
+		assert_eq!(Session::validators(), vec![1, 2, 3]);
+		ImOnline::note_author(2);
+		ImOnline::note_uncle(3, 0);
+
+		// when
+		UintAuthorityId::set_all_keys(vec![0]); // all authorities use session key 0
+		ImOnline::offchain(4);
+
+		// then
+		let transaction = state.write().transactions.pop().unwrap();
+		// All validators have `0` as their session key, but we should only produce 1 hearbeat.
+		assert_eq!(state.read().transactions.len(), 0);
+		// check stuff about the transaction.
+		let ex: Extrinsic = Decode::decode(&mut &*transaction).unwrap();
+		let heartbeat = match ex.1 {
+			crate::mock::Call::ImOnline(crate::Call::heartbeat(h, _)) => h,
+			e => panic!("Unexpected call: {:?}", e),
+		};
+
+		assert_eq!(heartbeat, Heartbeat {
+			block_number: 4,
+			network_state: runtime_io::offchain::network_state().unwrap(),
+			session_index: 2,
+			authority_index: 0,
+		});
 	});
 }
diff --git a/srml/indices/Cargo.toml b/paint/indices/Cargo.toml
similarity index 56%
rename from srml/indices/Cargo.toml
rename to paint/indices/Cargo.toml
index b8b560c3f9b4d..443d28ba878d8 100644
--- a/srml/indices/Cargo.toml
+++ b/paint/indices/Cargo.toml
@@ -1,5 +1,5 @@
 [package]
-name = "srml-indices"
+name = "paint-indices"
 version = "2.0.0"
 authors = ["Parity Technologies "]
 edition = "2018"
@@ -8,13 +8,13 @@ edition = "2018"
 serde = { version = "1.0.101", optional = true }
 safe-mix = { version = "1.0.0", default-features = false }
 codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false, features = ["derive"] }
-substrate-keyring = { path = "../../core/keyring", optional = true }
-rstd = { package = "sr-std", path = "../../core/sr-std", default-features = false }
-runtime-io = { package = "sr-io", path = "../../core/sr-io", default-features = false }
-sr-primitives = { path = "../../core/sr-primitives", default-features = false }
-primitives = { package = "substrate-primitives",  path = "../../core/primitives", default-features = false }
-support = { package = "srml-support", path = "../support", default-features = false }
-system = { package = "srml-system", path = "../system", default-features = false }
+substrate-keyring = { path = "../../primitives/keyring", optional = true }
+rstd = { package = "sr-std", path = "../../primitives/sr-std", default-features = false }
+runtime-io = { package = "sr-io", path = "../../primitives/sr-io", default-features = false }
+sr-primitives = { path = "../../primitives/sr-primitives", default-features = false }
+primitives = { package = "substrate-primitives",  path = "../../primitives/core", default-features = false }
+support = { package = "paint-support", path = "../support", default-features = false }
+system = { package = "paint-system", path = "../system", default-features = false }
 
 [dev-dependencies]
 ref_thread_local = "0.0.0"
diff --git a/srml/indices/src/address.rs b/paint/indices/src/address.rs
similarity index 100%
rename from srml/indices/src/address.rs
rename to paint/indices/src/address.rs
diff --git a/srml/indices/src/lib.rs b/paint/indices/src/lib.rs
similarity index 100%
rename from srml/indices/src/lib.rs
rename to paint/indices/src/lib.rs
diff --git a/srml/indices/src/mock.rs b/paint/indices/src/mock.rs
similarity index 100%
rename from srml/indices/src/mock.rs
rename to paint/indices/src/mock.rs
diff --git a/srml/indices/src/tests.rs b/paint/indices/src/tests.rs
similarity index 100%
rename from srml/indices/src/tests.rs
rename to paint/indices/src/tests.rs
diff --git a/srml/membership/Cargo.toml b/paint/membership/Cargo.toml
similarity index 54%
rename from srml/membership/Cargo.toml
rename to paint/membership/Cargo.toml
index a8cb73055563e..fc75bc4c69ea5 100644
--- a/srml/membership/Cargo.toml
+++ b/paint/membership/Cargo.toml
@@ -1,5 +1,5 @@
 [package]
-name = "srml-membership"
+name = "paint-membership"
 version = "2.0.0"
 authors = ["Parity Technologies "]
 edition = "2018"
@@ -7,14 +7,14 @@ edition = "2018"
 [dependencies]
 serde = { version = "1.0.101", optional = true }
 codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false }
-rstd = { package = "sr-std", path = "../../core/sr-std", default-features = false }
-runtime-io = { package = "sr-io", path = "../../core/sr-io", default-features = false }
-support = { package = "srml-support", path = "../support", default-features = false }
-system = { package = "srml-system", path = "../system", default-features = false }
-sr-primitives = { path = "../../core/sr-primitives", default-features = false }
+rstd = { package = "sr-std", path = "../../primitives/sr-std", default-features = false }
+runtime-io = { package = "sr-io", path = "../../primitives/sr-io", default-features = false }
+support = { package = "paint-support", path = "../support", default-features = false }
+system = { package = "paint-system", path = "../system", default-features = false }
+sr-primitives = { path = "../../primitives/sr-primitives", default-features = false }
 
 [dev-dependencies]
-primitives = { package = "substrate-primitives",  path = "../../core/primitives" }
+primitives = { package = "substrate-primitives",  path = "../../primitives/core" }
 
 [features]
 default = ["std"]
diff --git a/srml/membership/src/lib.rs b/paint/membership/src/lib.rs
similarity index 100%
rename from srml/membership/src/lib.rs
rename to paint/membership/src/lib.rs
diff --git a/srml/metadata/Cargo.toml b/paint/metadata/Cargo.toml
similarity index 63%
rename from srml/metadata/Cargo.toml
rename to paint/metadata/Cargo.toml
index e9381f68bbbf9..1d7185cc1060a 100644
--- a/srml/metadata/Cargo.toml
+++ b/paint/metadata/Cargo.toml
@@ -1,5 +1,5 @@
 [package]
-name = "srml-metadata"
+name = "paint-metadata"
 version = "2.0.0"
 authors = ["Parity Technologies "]
 edition = "2018"
@@ -7,8 +7,8 @@ edition = "2018"
 [dependencies]
 codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false, features = ["derive"] }
 serde = { version = "1.0.101", optional = true, features = ["derive"] }
-rstd = { package = "sr-std", path = "../../core/sr-std", default-features = false }
-primitives = { package = "substrate-primitives", path = "../../core/primitives", default-features = false }
+rstd = { package = "sr-std", path = "../../primitives/sr-std", default-features = false }
+primitives = { package = "substrate-primitives", path = "../../primitives/core", default-features = false }
 
 [features]
 default = ["std"]
diff --git a/srml/metadata/src/lib.rs b/paint/metadata/src/lib.rs
similarity index 100%
rename from srml/metadata/src/lib.rs
rename to paint/metadata/src/lib.rs
diff --git a/srml/nicks/Cargo.toml b/paint/nicks/Cargo.toml
similarity index 50%
rename from srml/nicks/Cargo.toml
rename to paint/nicks/Cargo.toml
index 4b8fabe9effe5..9e78254332451 100644
--- a/srml/nicks/Cargo.toml
+++ b/paint/nicks/Cargo.toml
@@ -1,5 +1,5 @@
 [package]
-name = "srml-nicks"
+name = "paint-nicks"
 version = "2.0.0"
 authors = ["Parity Technologies "]
 edition = "2018"
@@ -7,15 +7,15 @@ edition = "2018"
 [dependencies]
 serde = { version = "1.0.101", optional = true }
 codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false, features = ["derive"] }
-rstd = { package = "sr-std", path = "../../core/sr-std", default-features = false }
-runtime-io = { package = "sr-io", path = "../../core/sr-io", default-features = false }
-sr-primitives = { path = "../../core/sr-primitives", default-features = false }
-support = { package = "srml-support", path = "../support", default-features = false }
-system = { package = "srml-system", path = "../system", default-features = false }
+rstd = { package = "sr-std", path = "../../primitives/sr-std", default-features = false }
+runtime-io = { package = "sr-io", path = "../../primitives/sr-io", default-features = false }
+sr-primitives = { path = "../../primitives/sr-primitives", default-features = false }
+support = { package = "paint-support", path = "../support", default-features = false }
+system = { package = "paint-system", path = "../system", default-features = false }
 
 [dev-dependencies]
-primitives = { package = "substrate-primitives",  path = "../../core/primitives" }
-balances = { package = "srml-balances", path = "../balances", default-features = false }
+primitives = { package = "substrate-primitives",  path = "../../primitives/core" }
+balances = { package = "paint-balances", path = "../balances", default-features = false }
 
 [features]
 default = ["std"]
diff --git a/srml/nicks/src/lib.rs b/paint/nicks/src/lib.rs
similarity index 100%
rename from srml/nicks/src/lib.rs
rename to paint/nicks/src/lib.rs
diff --git a/paint/offences/Cargo.toml b/paint/offences/Cargo.toml
new file mode 100644
index 0000000000000..8eb217c8b3265
--- /dev/null
+++ b/paint/offences/Cargo.toml
@@ -0,0 +1,32 @@
+[package]
+name = "paint-offences"
+version = "1.0.0"
+authors = ["Parity Technologies "]
+edition = "2018"
+
+[dependencies]
+balances = { package = "paint-balances", path = "../balances", default-features = false }
+codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false, features = ["derive"] }
+rstd = { package = "sr-std", path = "../../primitives/sr-std", default-features = false }
+serde = { version = "1.0.101", optional = true }
+sr-primitives = { path = "../../primitives/sr-primitives", default-features = false }
+sr-staking-primitives = { path = "../../primitives/sr-staking-primitives", default-features = false }
+support = { package = "paint-support", path = "../support", default-features = false }
+system = { package = "paint-system", path = "../system", default-features = false }
+
+[dev-dependencies]
+runtime-io = { package = "sr-io", path = "../../primitives/sr-io" }
+substrate-primitives = { path = "../../primitives/core" }
+
+[features]
+default = ["std"]
+std = [
+	"balances/std",
+	"codec/std",
+	"rstd/std",
+	"serde",
+	"sr-primitives/std",
+	"sr-staking-primitives/std",
+	"support/std",
+	"system/std",
+]
diff --git a/srml/offences/src/lib.rs b/paint/offences/src/lib.rs
similarity index 100%
rename from srml/offences/src/lib.rs
rename to paint/offences/src/lib.rs
diff --git a/srml/offences/src/mock.rs b/paint/offences/src/mock.rs
similarity index 97%
rename from srml/offences/src/mock.rs
rename to paint/offences/src/mock.rs
index f9c79390819e0..6ae0e35b9bd58 100644
--- a/srml/offences/src/mock.rs
+++ b/paint/offences/src/mock.rs
@@ -148,7 +148,7 @@ impl offence::Offence for Offence {
 	}
 
 	fn session_index(&self) -> SessionIndex {
-		// session index is not used by the srml-offences directly, but rather it exists only for
+		// session index is not used by the paint-offences directly, but rather it exists only for
 		// filtering historical reports.
 		unimplemented!()
 	}
diff --git a/srml/offences/src/tests.rs b/paint/offences/src/tests.rs
similarity index 100%
rename from srml/offences/src/tests.rs
rename to paint/offences/src/tests.rs
diff --git a/srml/randomness-collective-flip/Cargo.toml b/paint/randomness-collective-flip/Cargo.toml
similarity index 55%
rename from srml/randomness-collective-flip/Cargo.toml
rename to paint/randomness-collective-flip/Cargo.toml
index 5be9aad50f0b3..97e56d1a83a41 100644
--- a/srml/randomness-collective-flip/Cargo.toml
+++ b/paint/randomness-collective-flip/Cargo.toml
@@ -1,5 +1,5 @@
 [package]
-name = "srml-randomness-collective-flip"
+name = "paint-randomness-collective-flip"
 version = "2.0.0"
 authors = ["Parity Technologies "]
 edition = "2018"
@@ -7,14 +7,14 @@ edition = "2018"
 [dependencies]
 safe-mix = { version = "1.0", default-features = false }
 codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false, features = ["derive"] }
-sr-primitives = { path = "../../core/sr-primitives", default-features = false }
-support = { package = "srml-support", path = "../support", default-features = false }
-system = { package = "srml-system", path = "../system", default-features = false }
-rstd = { package = "sr-std", path = "../../core/sr-std", default-features = false }
+sr-primitives = { path = "../../primitives/sr-primitives", default-features = false }
+support = { package = "paint-support", path = "../support", default-features = false }
+system = { package = "paint-system", path = "../system", default-features = false }
+rstd = { package = "sr-std", path = "../../primitives/sr-std", default-features = false }
 
 [dev-dependencies]
-primitives = { package = "substrate-primitives",  path = "../../core/primitives" }
-runtime-io = { package = "sr-io", path = "../../core/sr-io" }
+primitives = { package = "substrate-primitives",  path = "../../primitives/core" }
+runtime-io = { package = "sr-io", path = "../../primitives/sr-io" }
 
 [features]
 default = ["std"]
diff --git a/srml/randomness-collective-flip/src/lib.rs b/paint/randomness-collective-flip/src/lib.rs
similarity index 99%
rename from srml/randomness-collective-flip/src/lib.rs
rename to paint/randomness-collective-flip/src/lib.rs
index 3644b8898c2ab..62170e8a8842c 100644
--- a/srml/randomness-collective-flip/src/lib.rs
+++ b/paint/randomness-collective-flip/src/lib.rs
@@ -42,7 +42,7 @@
 //! decl_module! {
 //! 	pub struct Module for enum Call where origin: T::Origin {
 //! 		pub fn random_module_example(origin) -> Result {
-//! 			let _random_seed = >::random_seed();
+//! 			let _random_seed = >::random_seed();
 //! 			Ok(())
 //! 		}
 //! 	}
diff --git a/srml/scored-pool/Cargo.toml b/paint/scored-pool/Cargo.toml
similarity index 51%
rename from srml/scored-pool/Cargo.toml
rename to paint/scored-pool/Cargo.toml
index e65f71a10d13e..35da18437bb51 100644
--- a/srml/scored-pool/Cargo.toml
+++ b/paint/scored-pool/Cargo.toml
@@ -1,5 +1,5 @@
 [package]
-name = "srml-scored-pool"
+name = "paint-scored-pool"
 version = "1.0.0"
 authors = ["Parity Technologies "]
 edition = "2018"
@@ -7,15 +7,15 @@ edition = "2018"
 [dependencies]
 codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false, features = ["derive"] }
 serde = { version = "1.0.101", optional = true }
-runtime-io = { package = "sr-io", path = "../../core/sr-io", default-features = false }
-sr-primitives = { path = "../../core/sr-primitives", default-features = false }
-rstd = { package = "sr-std", path = "../../core/sr-std", default-features = false }
-support = { package = "srml-support", path = "../support", default-features = false }
-system = { package = "srml-system", path = "../system", default-features = false }
+runtime-io = { package = "sr-io", path = "../../primitives/sr-io", default-features = false }
+sr-primitives = { path = "../../primitives/sr-primitives", default-features = false }
+rstd = { package = "sr-std", path = "../../primitives/sr-std", default-features = false }
+support = { package = "paint-support", path = "../support", default-features = false }
+system = { package = "paint-system", path = "../system", default-features = false }
 
 [dev-dependencies]
-balances = { package = "srml-balances", path = "../balances" }
-primitives = { package = "substrate-primitives",  path = "../../core/primitives" }
+balances = { package = "paint-balances", path = "../balances" }
+primitives = { package = "substrate-primitives",  path = "../../primitives/core" }
 
 [features]
 default = ["std"]
diff --git a/srml/scored-pool/src/lib.rs b/paint/scored-pool/src/lib.rs
similarity index 99%
rename from srml/scored-pool/src/lib.rs
rename to paint/scored-pool/src/lib.rs
index 5fde1e9c450cf..3ed4b8f21fa49 100644
--- a/srml/scored-pool/src/lib.rs
+++ b/paint/scored-pool/src/lib.rs
@@ -55,7 +55,7 @@
 //! ```
 //! use support::{decl_module, dispatch::Result};
 //! use system::ensure_signed;
-//! use srml_scored_pool::{self as scored_pool};
+//! use paint_scored_pool::{self as scored_pool};
 //!
 //! pub trait Trait: scored_pool::Trait {}
 //!
@@ -77,7 +77,7 @@
 //!
 //! ## Dependencies
 //!
-//! This module depends on the [System module](../srml_system/index.html).
+//! This module depends on the [System module](../paint_system/index.html).
 
 // Ensure we're `no_std` when compiling for Wasm.
 #![cfg_attr(not(feature = "std"), no_std)]
diff --git a/srml/scored-pool/src/mock.rs b/paint/scored-pool/src/mock.rs
similarity index 100%
rename from srml/scored-pool/src/mock.rs
rename to paint/scored-pool/src/mock.rs
diff --git a/srml/scored-pool/src/tests.rs b/paint/scored-pool/src/tests.rs
similarity index 100%
rename from srml/scored-pool/src/tests.rs
rename to paint/scored-pool/src/tests.rs
diff --git a/srml/session/Cargo.toml b/paint/session/Cargo.toml
similarity index 53%
rename from srml/session/Cargo.toml
rename to paint/session/Cargo.toml
index 1346867910313..3a5b1476a455f 100644
--- a/srml/session/Cargo.toml
+++ b/paint/session/Cargo.toml
@@ -1,5 +1,5 @@
 [package]
-name = "srml-session"
+name = "paint-session"
 version = "2.0.0"
 authors = ["Parity Technologies "]
 edition =  "2018"
@@ -8,19 +8,19 @@ edition =  "2018"
 serde = { version = "1.0.101", optional = true }
 safe-mix = { version = "1.0.0", default-features = false }
 codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false, features = ["derive"] }
-rstd = { package = "sr-std", path = "../../core/sr-std", default-features = false }
-sr-primitives = { path = "../../core/sr-primitives", default-features = false }
-sr-staking-primitives = { path = "../../core/sr-staking-primitives", default-features = false }
-support = { package = "srml-support", path = "../support", default-features = false }
-system = { package = "srml-system", path = "../system", default-features = false }
-timestamp = { package = "srml-timestamp", path = "../timestamp", default-features = false }
-substrate-trie = { path = "../../core/trie", default-features = false, optional = true }
-runtime-io ={ package = "sr-io", path = "../../core/sr-io", default-features = false }
+rstd = { package = "sr-std", path = "../../primitives/sr-std", default-features = false }
+sr-primitives = { path = "../../primitives/sr-primitives", default-features = false }
+sr-staking-primitives = { path = "../../primitives/sr-staking-primitives", default-features = false }
+support = { package = "paint-support", path = "../support", default-features = false }
+system = { package = "paint-system", path = "../system", default-features = false }
+timestamp = { package = "paint-timestamp", path = "../timestamp", default-features = false }
+substrate-trie = { path = "../../primitives/trie", default-features = false, optional = true }
+runtime-io ={ package = "sr-io", path = "../../primitives/sr-io", default-features = false }
 impl-trait-for-tuples = "0.1.3"
 
 [dev-dependencies]
-primitives = { package = "substrate-primitives",  path = "../../core/primitives" }
-app-crypto = { package = "substrate-application-crypto",  path = "../../core/application-crypto" }
+primitives = { package = "substrate-primitives",  path = "../../primitives/core" }
+app-crypto = { package = "substrate-application-crypto",  path = "../../primitives/application-crypto" }
 lazy_static = "1.4.0"
 
 [features]
diff --git a/srml/session/src/historical.rs b/paint/session/src/historical.rs
similarity index 100%
rename from srml/session/src/historical.rs
rename to paint/session/src/historical.rs
diff --git a/srml/session/src/lib.rs b/paint/session/src/lib.rs
similarity index 98%
rename from srml/session/src/lib.rs
rename to paint/session/src/lib.rs
index 42324d387f69e..95209c9ab6bd3 100644
--- a/srml/session/src/lib.rs
+++ b/paint/session/src/lib.rs
@@ -45,7 +45,7 @@
 //! a block author or a validator. The session keys of accounts are removed once their account balance is zero.
 //! - **Validator set session key configuration process:** Each session we iterate through the current
 //! set of validator account IDs to check if a session key was created for it in the previous session
-//! using `set_key`. If it was then we call `set_authority` from the [Consensus module](../srml_consensus/index.html)
+//! using `set_key`. If it was then we call `set_authority` from the [Consensus module](../paint_consensus/index.html)
 //! and pass it a set of session keys (each associated with an account ID) as the session keys for the new
 //! validator set. Lastly, if the session key of the current authority does not match any session keys stored under
 //! its validator index in the `AuthorityStorageVec` mapping, then we update the mapping with its session
@@ -100,10 +100,10 @@
 //!
 //! ### Example from the SRML
 //!
-//! The [Staking module](../srml_staking/index.html) uses the Session module to get the validator set.
+//! The [Staking module](../paint_staking/index.html) uses the Session module to get the validator set.
 //!
 //! ```
-//! use srml_session as session;
+//! use paint_session as session;
 //! # fn not_executed() {
 //!
 //! let validators = >::validators();
@@ -113,9 +113,9 @@
 //!
 //! ## Related Modules
 //!
-//! - [Consensus](../srml_consensus/index.html)
-//! - [Staking](../srml_staking/index.html)
-//! - [Timestamp](../srml_timestamp/index.html)
+//! - [Consensus](../paint_consensus/index.html)
+//! - [Staking](../paint_staking/index.html)
+//! - [Timestamp](../paint_timestamp/index.html)
 
 #![cfg_attr(not(feature = "std"), no_std)]
 
@@ -125,10 +125,7 @@ use sr_primitives::{KeyTypeId, Perbill, RuntimeAppPublic, BoundToRuntimeAppPubli
 use sr_primitives::weights::SimpleDispatchInfo;
 use sr_primitives::traits::{Convert, Zero, Member, OpaqueKeys};
 use sr_staking_primitives::SessionIndex;
-use support::{
-	dispatch::Result, ConsensusEngineId, decl_module, decl_event,
-	decl_storage,
-};
+use support::{dispatch::Result, ConsensusEngineId, decl_module, decl_event, decl_storage};
 use support::{ensure, traits::{OnFreeBalanceZero, Get, FindAuthor}, Parameter};
 use system::{self, ensure_signed};
 
@@ -361,7 +358,7 @@ pub trait Trait: system::Trait {
 	/// The fraction of validators set that is safe to be disabled.
 	///
 	/// After the threshold is reached `disabled` method starts to return true,
-	/// which in combination with `srml_staking` forces a new era.
+	/// which in combination with `paint_staking` forces a new era.
 	type DisabledValidatorsThreshold: Get;
 
 	/// Select initial validators.
diff --git a/srml/session/src/mock.rs b/paint/session/src/mock.rs
similarity index 100%
rename from srml/session/src/mock.rs
rename to paint/session/src/mock.rs
diff --git a/paint/staking/Cargo.toml b/paint/staking/Cargo.toml
new file mode 100644
index 0000000000000..7632ae38c4d78
--- /dev/null
+++ b/paint/staking/Cargo.toml
@@ -0,0 +1,45 @@
+[package]
+name = "paint-staking"
+version = "2.0.0"
+authors = ["Parity Technologies "]
+edition = "2018"
+
+[dependencies]
+serde = { version = "1.0.101", optional = true }
+safe-mix = { version = "1.0.0", default-features = false }
+codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false, features = ["derive"] }
+substrate-keyring = { path = "../../primitives/keyring", optional = true }
+rstd = { package = "sr-std", path = "../../primitives/sr-std", default-features = false }
+phragmen = { package = "substrate-phragmen", path = "../../primitives/phragmen", default-features = false }
+runtime-io ={ package = "sr-io", path = "../../primitives/sr-io", default-features = false }
+sr-primitives = { path = "../../primitives/sr-primitives", default-features = false }
+sr-staking-primitives = { path = "../../primitives/sr-staking-primitives", default-features = false }
+support = { package = "paint-support", path = "../support", default-features = false }
+system = { package = "paint-system", path = "../system", default-features = false }
+session = { package = "paint-session", path = "../session", default-features = false, features = ["historical"] }
+authorship = { package = "paint-authorship", path = "../authorship", default-features = false }
+
+[dev-dependencies]
+primitives = { package = "substrate-primitives",  path = "../../primitives/core" }
+balances = { package = "paint-balances", path = "../balances" }
+timestamp = { package = "paint-timestamp", path = "../timestamp" }
+paint-staking-reward-curve = { path = "../staking/reward-curve"}
+
+[features]
+equalize = []
+default = ["std", "equalize"]
+std = [
+	"serde",
+	"safe-mix/std",
+	"substrate-keyring",
+	"codec/std",
+	"rstd/std",
+	"phragmen/std",
+	"runtime-io/std",
+	"support/std",
+	"sr-primitives/std",
+	"sr-staking-primitives/std",
+	"session/std",
+	"system/std",
+	"authorship/std",
+]
diff --git a/srml/staking/reward-curve/Cargo.toml b/paint/staking/reward-curve/Cargo.toml
similarity index 52%
rename from srml/staking/reward-curve/Cargo.toml
rename to paint/staking/reward-curve/Cargo.toml
index 4fb0ab1672ce0..03b6ab8955de1 100644
--- a/srml/staking/reward-curve/Cargo.toml
+++ b/paint/staking/reward-curve/Cargo.toml
@@ -1,5 +1,5 @@
 [package]
-name = "srml-staking-reward-curve"
+name = "paint-staking-reward-curve"
 version = "2.0.0"
 authors = ["Parity Technologies "]
 edition = "2018"
@@ -8,10 +8,10 @@ edition = "2018"
 proc-macro = true
 
 [dependencies]
-syn = { version = "1.0", features = [ "full", "visit" ] }
+syn = { version = "1.0.7", features = [ "full", "visit" ] }
 quote = "1.0"
-proc-macro2 = "1.0.4"
+proc-macro2 = "1.0.6"
 proc-macro-crate = "0.1.4"
 
 [dev-dependencies]
-sr-primitives = { path = "../../../core/sr-primitives" }
+sr-primitives = { path = "../../../primitives/sr-primitives" }
diff --git a/srml/staking/reward-curve/src/lib.rs b/paint/staking/reward-curve/src/lib.rs
similarity index 97%
rename from srml/staking/reward-curve/src/lib.rs
rename to paint/staking/reward-curve/src/lib.rs
index 7e1f1e6aa945f..947a9a84d0cb3 100644
--- a/srml/staking/reward-curve/src/lib.rs
+++ b/paint/staking/reward-curve/src/lib.rs
@@ -44,7 +44,7 @@ use syn::parse::{Parse, ParseStream};
 /// # fn main() {}
 /// use sr_primitives::curve::PiecewiseLinear;
 ///
-/// srml_staking_reward_curve::build! {
+/// paint_staking_reward_curve::build! {
 /// 	const I_NPOS: PiecewiseLinear<'static> = curve!(
 /// 		min_inflation: 0_025_000,
 /// 		max_inflation: 0_100_000,
@@ -323,6 +323,14 @@ fn compute_points(input: &INposInput) -> Vec<(u32, u32)> {
 fn generate_piecewise_linear(points: Vec<(u32, u32)>) -> TokenStream2 {
 	let mut points_tokens = quote!();
 
+	let max = points.iter()
+		.map(|&(_, x)| x)
+		.max()
+		.unwrap_or(0)
+		.checked_mul(1_000)
+		// clip at 1.0 for sanity only since it'll panic later if too high.
+		.unwrap_or(1_000_000_000);
+
 	for (x, y) in points {
 		let error = || panic!(format!(
 			"Generated reward curve approximation doesn't fit into [0, 1] -> [0, 1] \
@@ -346,6 +354,7 @@ fn generate_piecewise_linear(points: Vec<(u32, u32)>) -> TokenStream2 {
 	quote!(
 		_sr_primitives::curve::PiecewiseLinear::<'static> {
 			points: & [ #points_tokens ],
+			maximum: _sr_primitives::Perbill::from_parts(#max),
 		}
 	)
 }
@@ -364,7 +373,7 @@ fn generate_test_module(input: &INposInput) -> TokenStream2 {
 
 	quote!(
 		#[cfg(test)]
-		mod __srml_staking_reward_curve_test_module {
+		mod __paint_staking_reward_curve_test_module {
 			fn i_npos(x: f64) -> f64 {
 				if x <= #x_ideal {
 					#i_0 + x * (#i_ideal - #i_0 / #x_ideal)
diff --git a/srml/staking/reward-curve/src/log.rs b/paint/staking/reward-curve/src/log.rs
similarity index 100%
rename from srml/staking/reward-curve/src/log.rs
rename to paint/staking/reward-curve/src/log.rs
diff --git a/srml/staking/reward-curve/tests/test.rs b/paint/staking/reward-curve/tests/test.rs
similarity index 89%
rename from srml/staking/reward-curve/tests/test.rs
rename to paint/staking/reward-curve/tests/test.rs
index 55a3b7d383e9e..be7f0cbd1883c 100644
--- a/srml/staking/reward-curve/tests/test.rs
+++ b/paint/staking/reward-curve/tests/test.rs
@@ -14,11 +14,11 @@
 // You should have received a copy of the GNU General Public License
 // along with Substrate.  If not, see .
 
-//! Test crate for srml-staking-reward-curve. Allows to test for procedural macro.
+//! Test crate for paint-staking-reward-curve. Allows to test for procedural macro.
 //! See tests directory.
 
 mod test_small_falloff {
-	srml_staking_reward_curve::build! {
+	paint_staking_reward_curve::build! {
 		const REWARD_CURVE: sr_primitives::curve::PiecewiseLinear<'static> = curve!(
 			min_inflation: 0_020_000,
 			max_inflation: 0_200_000,
@@ -31,7 +31,7 @@ mod test_small_falloff {
 }
 
 mod test_big_falloff {
-	srml_staking_reward_curve::build! {
+	paint_staking_reward_curve::build! {
 		const REWARD_CURVE: sr_primitives::curve::PiecewiseLinear<'static> = curve!(
 			min_inflation: 0_100_000,
 			max_inflation: 0_400_000,
diff --git a/srml/staking/src/inflation.rs b/paint/staking/src/inflation.rs
similarity index 79%
rename from srml/staking/src/inflation.rs
rename to paint/staking/src/inflation.rs
index 89326b92c0fd1..11cf3151f6599 100644
--- a/srml/staking/src/inflation.rs
+++ b/paint/staking/src/inflation.rs
@@ -32,20 +32,24 @@ pub fn compute_total_payout(
 	npos_token_staked: N,
 	total_tokens: N,
 	era_duration: u64
-) -> N where N: SimpleArithmetic + Clone
-{
+) -> (N, N) where N: SimpleArithmetic + Clone {
 	// Milliseconds per year for the Julian year (365.25 days).
 	const MILLISECONDS_PER_YEAR: u64 = 1000 * 3600 * 24 * 36525 / 100;
 
-	Perbill::from_rational_approximation(era_duration as u64, MILLISECONDS_PER_YEAR)
-		* yearly_inflation.calculate_for_fraction_times_denominator(npos_token_staked, total_tokens)
+	let portion = Perbill::from_rational_approximation(era_duration as u64, MILLISECONDS_PER_YEAR);
+	let payout = portion * yearly_inflation.calculate_for_fraction_times_denominator(
+		npos_token_staked,
+		total_tokens.clone(),
+	);
+	let maximum = portion * (yearly_inflation.maximum * total_tokens);
+	(payout, maximum)
 }
 
 #[cfg(test)]
 mod test {
 	use sr_primitives::curve::PiecewiseLinear;
 
-	srml_staking_reward_curve::build! {
+	paint_staking_reward_curve::build! {
 		const I_NPOS: PiecewiseLinear<'static> = curve!(
 			min_inflation: 0_025_000,
 			max_inflation: 0_100_000,
@@ -59,26 +63,31 @@ mod test {
 	#[test]
 	fn npos_curve_is_sensible() {
 		const YEAR: u64 = 365 * 24 * 60 * 60 * 1000;
+
+		// check maximum inflation.
+		// not 10_000 due to rounding error.
+		assert_eq!(super::compute_total_payout(&I_NPOS, 0, 100_000u64, YEAR).1, 9_993);
+
 		//super::I_NPOS.calculate_for_fraction_times_denominator(25, 100)
-		assert_eq!(super::compute_total_payout(&I_NPOS, 0, 100_000u64, YEAR), 2_498);
-		assert_eq!(super::compute_total_payout(&I_NPOS, 5_000, 100_000u64, YEAR), 3_248);
-		assert_eq!(super::compute_total_payout(&I_NPOS, 25_000, 100_000u64, YEAR), 6_246);
-		assert_eq!(super::compute_total_payout(&I_NPOS, 40_000, 100_000u64, YEAR), 8_494);
-		assert_eq!(super::compute_total_payout(&I_NPOS, 50_000, 100_000u64, YEAR), 9_993);
-		assert_eq!(super::compute_total_payout(&I_NPOS, 60_000, 100_000u64, YEAR), 4_379);
-		assert_eq!(super::compute_total_payout(&I_NPOS, 75_000, 100_000u64, YEAR), 2_733);
-		assert_eq!(super::compute_total_payout(&I_NPOS, 95_000, 100_000u64, YEAR), 2_513);
-		assert_eq!(super::compute_total_payout(&I_NPOS, 100_000, 100_000u64, YEAR), 2_505);
+		assert_eq!(super::compute_total_payout(&I_NPOS, 0, 100_000u64, YEAR).0, 2_498);
+		assert_eq!(super::compute_total_payout(&I_NPOS, 5_000, 100_000u64, YEAR).0, 3_248);
+		assert_eq!(super::compute_total_payout(&I_NPOS, 25_000, 100_000u64, YEAR).0, 6_246);
+		assert_eq!(super::compute_total_payout(&I_NPOS, 40_000, 100_000u64, YEAR).0, 8_494);
+		assert_eq!(super::compute_total_payout(&I_NPOS, 50_000, 100_000u64, YEAR).0, 9_993);
+		assert_eq!(super::compute_total_payout(&I_NPOS, 60_000, 100_000u64, YEAR).0, 4_379);
+		assert_eq!(super::compute_total_payout(&I_NPOS, 75_000, 100_000u64, YEAR).0, 2_733);
+		assert_eq!(super::compute_total_payout(&I_NPOS, 95_000, 100_000u64, YEAR).0, 2_513);
+		assert_eq!(super::compute_total_payout(&I_NPOS, 100_000, 100_000u64, YEAR).0, 2_505);
 
 		const DAY: u64 = 24 * 60 * 60 * 1000;
-		assert_eq!(super::compute_total_payout(&I_NPOS, 25_000, 100_000u64, DAY), 17);
-		assert_eq!(super::compute_total_payout(&I_NPOS, 50_000, 100_000u64, DAY), 27);
-		assert_eq!(super::compute_total_payout(&I_NPOS, 75_000, 100_000u64, DAY), 7);
+		assert_eq!(super::compute_total_payout(&I_NPOS, 25_000, 100_000u64, DAY).0, 17);
+		assert_eq!(super::compute_total_payout(&I_NPOS, 50_000, 100_000u64, DAY).0, 27);
+		assert_eq!(super::compute_total_payout(&I_NPOS, 75_000, 100_000u64, DAY).0, 7);
 
 		const SIX_HOURS: u64 = 6 * 60 * 60 * 1000;
-		assert_eq!(super::compute_total_payout(&I_NPOS, 25_000, 100_000u64, SIX_HOURS), 4);
-		assert_eq!(super::compute_total_payout(&I_NPOS, 50_000, 100_000u64, SIX_HOURS), 7);
-		assert_eq!(super::compute_total_payout(&I_NPOS, 75_000, 100_000u64, SIX_HOURS), 2);
+		assert_eq!(super::compute_total_payout(&I_NPOS, 25_000, 100_000u64, SIX_HOURS).0, 4);
+		assert_eq!(super::compute_total_payout(&I_NPOS, 50_000, 100_000u64, SIX_HOURS).0, 7);
+		assert_eq!(super::compute_total_payout(&I_NPOS, 75_000, 100_000u64, SIX_HOURS).0, 2);
 
 		const HOUR: u64 = 60 * 60 * 1000;
 		assert_eq!(
@@ -87,7 +96,7 @@ mod test {
 				2_500_000_000_000_000_000_000_000_000u128,
 				5_000_000_000_000_000_000_000_000_000u128,
 				HOUR
-			),
+			).0,
 			57_038_500_000_000_000_000_000
 		);
 	}
diff --git a/srml/staking/src/lib.rs b/paint/staking/src/lib.rs
similarity index 97%
rename from srml/staking/src/lib.rs
rename to paint/staking/src/lib.rs
index 23f0d7715cc2e..3ef7032783465 100644
--- a/srml/staking/src/lib.rs
+++ b/paint/staking/src/lib.rs
@@ -138,7 +138,7 @@
 //! ```
 //! use support::{decl_module, dispatch::Result};
 //! use system::ensure_signed;
-//! use srml_staking::{self as staking};
+//! use paint_staking::{self as staking};
 //!
 //! pub trait Trait: staking::Trait {}
 //!
@@ -179,7 +179,7 @@
 //! [`reward_by_indices`](./enum.Call.html#variant.reward_by_indices).
 //!
 //! [`Module`](./struct.Module.html) implements
-//! [`authorship::EventHandler`](../srml_authorship/trait.EventHandler.html) to add reward points
+//! [`authorship::EventHandler`](../paint_authorship/trait.EventHandler.html) to add reward points
 //! to block producer and block producer of referenced uncles.
 //!
 //! The validator and its nominator split their reward as following:
@@ -237,8 +237,8 @@
 //!
 //! ## Related Modules
 //!
-//! - [Balances](../srml_balances/index.html): Used to manage values at stake.
-//! - [Session](../srml_session/index.html): Used to manage sessions. Also, a list of new validators
+//! - [Balances](../paint_balances/index.html): Used to manage values at stake.
+//! - [Session](../paint_session/index.html): Used to manage sessions. Also, a list of new validators
 //! is stored in the Session module's `Validators` at the end of each era.
 
 #![recursion_limit="128"]
@@ -256,7 +256,7 @@ use codec::{HasCompact, Encode, Decode};
 use support::{
 	decl_module, decl_event, decl_storage, ensure,
 	traits::{
-		Currency, OnFreeBalanceZero, OnDilution, LockIdentifier, LockableCurrency,
+		Currency, OnFreeBalanceZero, LockIdentifier, LockableCurrency,
 		WithdrawReasons, OnUnbalanced, Imbalance, Get, Time
 	}
 };
@@ -501,8 +501,8 @@ pub trait Trait: system::Trait {
 	/// The post-processing needs it but will be moved to off-chain. TODO: #2908
 	type CurrencyToVote: Convert, u64> + Convert>;
 
-	/// Some tokens minted.
-	type OnRewardMinted: OnDilution>;
+	/// Tokens have been minted and are unused for validator-reward.
+	type RewardRemainder: OnUnbalanced>;
 
 	/// The overarching event type.
 	type Event: From> + Into<::Event>;
@@ -652,8 +652,9 @@ decl_storage! {
 
 decl_event!(
 	pub enum Event where Balance = BalanceOf, ::AccountId {
-		/// All validators have been rewarded by the given balance.
-		Reward(Balance),
+		/// All validators have been rewarded by the first balance; the second is the remainder
+		/// from the maximum amount of reward.
+		Reward(Balance, Balance),
 		/// One validator (and its nominators) has been slashed by the given amount.
 		Slash(AccountId, Balance),
 		/// An old slashing report from a prior era was discarded because it could
@@ -1198,7 +1199,7 @@ impl Module {
 			let validator_len: BalanceOf = (validators.len() as u32).into();
 			let total_rewarded_stake = Self::slot_stake() * validator_len;
 
-			let total_payout = inflation::compute_total_payout(
+			let (total_payout, max_payout) = inflation::compute_total_payout(
 				&T::RewardCurve::get(),
 				total_rewarded_stake.clone(),
 				T::Currency::total_issuance(),
@@ -1215,12 +1216,14 @@ impl Module {
 				}
 			}
 
-			let total_reward = total_imbalance.peek();
-			// assert!(total_reward <= total_payout)
+			// assert!(total_imbalance.peek() == total_payout)
+			let total_payout = total_imbalance.peek();
+
+			let rest = max_payout.saturating_sub(total_payout);
+			Self::deposit_event(RawEvent::Reward(total_payout, rest));
 
-			Self::deposit_event(RawEvent::Reward(total_reward));
 			T::Reward::on_unbalanced(total_imbalance);
-			T::OnRewardMinted::on_dilution(total_reward, total_rewarded_stake);
+			T::RewardRemainder::on_unbalanced(T::Currency::issue(rest));
 		}
 
 		// Increment current era.
@@ -1262,13 +1265,21 @@ impl Module {
 	///
 	/// Returns the new `SlotStake` value and a set of newly selected _stash_ IDs.
 	fn select_validators() -> (BalanceOf, Option>) {
+		let mut all_nominators: Vec<(T::AccountId, Vec)> = Vec::new();
+		let all_validator_candidates_iter = >::enumerate();
+		let all_validators = all_validator_candidates_iter.map(|(who, _pref)| {
+			let self_vote = (who.clone(), vec![who.clone()]);
+			all_nominators.push(self_vote);
+			who
+		}).collect::>();
+		all_nominators.extend(>::enumerate());
+
 		let maybe_phragmen_result = elect::<_, _, _, T::CurrencyToVote>(
 			Self::validator_count() as usize,
 			Self::minimum_validator_count().max(1) as usize,
-			>::enumerate().map(|(who, _)| who).collect::>(),
-			>::enumerate().collect(),
+			all_validators,
+			all_nominators,
 			Self::slashable_balance_of,
-			true,
 		);
 
 		if let Some(phragmen_result) = maybe_phragmen_result {
@@ -1286,7 +1297,6 @@ impl Module {
 				&elected_stashes,
 				&assignments,
 				Self::slashable_balance_of,
-				true,
 			);
 
 			if cfg!(feature = "equalize") {
@@ -1297,6 +1307,13 @@ impl Module {
 					let mut staked_assignment
 						: Vec>
 						= Vec::with_capacity(assignment.len());
+
+					// If this is a self vote, then we don't need to equalise it at all. While the
+					// staking system does not allow nomination and validation at the same time,
+					// this must always be 100% support.
+					if assignment.len() == 1 && assignment[0].0 == *n {
+						continue;
+					}
 					for (c, per_thing) in assignment.iter() {
 						let nominator_stake = to_votes(Self::slashable_balance_of(n));
 						let other_stake = *per_thing * nominator_stake;
diff --git a/srml/staking/src/mock.rs b/paint/staking/src/mock.rs
similarity index 99%
rename from srml/staking/src/mock.rs
rename to paint/staking/src/mock.rs
index b0ad771435ac4..ff9bfe50a5df1 100644
--- a/srml/staking/src/mock.rs
+++ b/paint/staking/src/mock.rs
@@ -178,7 +178,7 @@ impl timestamp::Trait for Test {
 	type OnTimestampSet = ();
 	type MinimumPeriod = MinimumPeriod;
 }
-srml_staking_reward_curve::build! {
+paint_staking_reward_curve::build! {
 	const I_NPOS: PiecewiseLinear<'static> = curve!(
 		min_inflation: 0_025_000,
 		max_inflation: 0_100_000,
@@ -197,7 +197,7 @@ impl Trait for Test {
 	type Currency = balances::Module;
 	type Time = timestamp::Module;
 	type CurrencyToVote = CurrencyToVoteHandler;
-	type OnRewardMinted = ();
+	type RewardRemainder = ();
 	type Event = ();
 	type Slash = ();
 	type Reward = ();
@@ -432,14 +432,12 @@ pub fn start_era(era_index: EraIndex) {
 }
 
 pub fn current_total_payout_for_duration(duration: u64) -> u64 {
-	let res = inflation::compute_total_payout(
+	inflation::compute_total_payout(
 		::RewardCurve::get(),
 		>::slot_stake() * 2,
 		Balances::total_issuance(),
 		duration,
-	);
-
-	res
+	).0
 }
 
 pub fn reward_all_elected() {
diff --git a/srml/staking/src/tests.rs b/paint/staking/src/tests.rs
similarity index 100%
rename from srml/staking/src/tests.rs
rename to paint/staking/src/tests.rs
diff --git a/srml/sudo/Cargo.toml b/paint/sudo/Cargo.toml
similarity index 55%
rename from srml/sudo/Cargo.toml
rename to paint/sudo/Cargo.toml
index 152f32ab89363..2bd1e83585ff7 100644
--- a/srml/sudo/Cargo.toml
+++ b/paint/sudo/Cargo.toml
@@ -1,5 +1,5 @@
 [package]
-name = "srml-sudo"
+name = "paint-sudo"
 version = "2.0.0"
 authors = ["Parity Technologies "]
 edition = "2018"
@@ -7,14 +7,14 @@ edition = "2018"
 [dependencies]
 serde = { version = "1.0.101", optional = true }
 codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false, features = ["derive"] }
-rstd = { package = "sr-std", path = "../../core/sr-std", default-features = false }
-runtime-io = { package = "sr-io", path = "../../core/sr-io", default-features = false }
-sr-primitives = { path = "../../core/sr-primitives", default-features = false }
-support = { package = "srml-support", path = "../support", default-features = false }
-system = { package = "srml-system", path = "../system", default-features = false }
+rstd = { package = "sr-std", path = "../../primitives/sr-std", default-features = false }
+runtime-io = { package = "sr-io", path = "../../primitives/sr-io", default-features = false }
+sr-primitives = { path = "../../primitives/sr-primitives", default-features = false }
+support = { package = "paint-support", path = "../support", default-features = false }
+system = { package = "paint-system", path = "../system", default-features = false }
 
 [dev-dependencies]
-primitives = { package = "substrate-primitives",  path = "../../core/primitives" }
+primitives = { package = "substrate-primitives",  path = "../../primitives/core" }
 
 [features]
 default = ["std"]
diff --git a/srml/sudo/src/lib.rs b/paint/sudo/src/lib.rs
similarity index 98%
rename from srml/sudo/src/lib.rs
rename to paint/sudo/src/lib.rs
index 7801384abcc2f..326c07cf8f436 100644
--- a/srml/sudo/src/lib.rs
+++ b/paint/sudo/src/lib.rs
@@ -77,8 +77,8 @@
 //!
 //! ## Related Modules
 //!
-//! * [Consensus](../srml_consensus/index.html)
-//! * [Democracy](../srml_democracy/index.html)
+//! * [Consensus](../paint_consensus/index.html)
+//! * [Democracy](../paint_democracy/index.html)
 //!
 //! [`Call`]: ./enum.Call.html
 //! [`Trait`]: ./trait.Trait.html
diff --git a/paint/support/Cargo.toml b/paint/support/Cargo.toml
new file mode 100644
index 0000000000000..17f0ddebc9c27
--- /dev/null
+++ b/paint/support/Cargo.toml
@@ -0,0 +1,43 @@
+[package]
+name = "paint-support"
+version = "2.0.0"
+authors = ["Parity Technologies "]
+edition = "2018"
+
+[dependencies]
+log = "0.4"
+serde = { version = "1.0.101", optional = true, features = ["derive"] }
+codec = { package = "parity-scale-codec", version = "1.0.6", default-features = false, features = ["derive"] }
+paint-metadata = { path = "../metadata", default-features = false }
+rstd = { package = "sr-std", path = "../../primitives/sr-std", default-features = false }
+runtime-io ={ package = "sr-io", path = "../../primitives/sr-io", default-features = false }
+sr-primitives = { path = "../../primitives/sr-primitives", default-features = false }
+primitives = { package = "substrate-primitives",  path = "../../primitives/core", default-features = false }
+inherents = { package = "substrate-inherents", path = "../../primitives/inherents", default-features = false }
+paint-support-procedural = { package = "paint-support-procedural", path = "./procedural" }
+paste = "0.1.6"
+once_cell = { version = "0.2.4", default-features = false, optional = true }
+state-machine = { package = "substrate-state-machine", path = "../../primitives/state-machine", optional = true }
+bitmask = { version = "0.5.0", default-features = false }
+impl-trait-for-tuples = "0.1.3"
+
+[dev-dependencies]
+pretty_assertions = "0.6.1"
+paint-system = { path = "../system" }
+
+[features]
+default = ["std"]
+std = [
+	"once_cell",
+	"bitmask/std",
+	"serde",
+	"runtime-io/std",
+	"codec/std",
+	"rstd/std",
+	"sr-primitives/std",
+	"paint-metadata/std",
+	"inherents/std",
+	"state-machine",
+]
+nightly = []
+strict = []
diff --git a/paint/support/procedural/Cargo.toml b/paint/support/procedural/Cargo.toml
new file mode 100644
index 0000000000000..96839c7d1fe55
--- /dev/null
+++ b/paint/support/procedural/Cargo.toml
@@ -0,0 +1,15 @@
+[package]
+name = "paint-support-procedural"
+version = "2.0.0"
+authors = ["Parity Technologies "]
+edition = "2018"
+
+[lib]
+proc-macro = true
+
+[dependencies]
+paint-support-procedural-tools = { package = "paint-support-procedural-tools", path = "./tools" }
+
+proc-macro2 = "1.0.6"
+quote = "1.0.2"
+syn = { version = "1.0.7", features = ["full"] }
diff --git a/srml/support/procedural/src/lib.rs b/paint/support/procedural/src/lib.rs
similarity index 86%
rename from srml/support/procedural/src/lib.rs
rename to paint/support/procedural/src/lib.rs
index 792f739f1c357..ee704617fe297 100644
--- a/srml/support/procedural/src/lib.rs
+++ b/paint/support/procedural/src/lib.rs
@@ -53,16 +53,16 @@ use proc_macro::TokenStream;
 /// Basic storage consists of a name and a type; supported types are:
 ///
 /// * Value: `Foo: type`: Implements the
-///   [`StorageValue`](../srml_support/storage/trait.StorageValue.html) trait using the
-///   [`StorageValue generator`](../srml_support/storage/generator/trait.StorageValue.html).
+///   [`StorageValue`](../paint_support/storage/trait.StorageValue.html) trait using the
+///   [`StorageValue generator`](../paint_support/storage/generator/trait.StorageValue.html).
 ///   The generator `unhashed_key` is `$module_prefix ++ " " ++ $storage_name`
 ///
 /// * Map: `Foo: map hasher($hash) type => type`: Implements the
-///   [`StorageMap`](../srml_support/storage/trait.StorageMap.html) trait using the
-///   [`StorageMap generator`](../srml_support/storage/generator/trait.StorageMap.html).
+///   [`StorageMap`](../paint_support/storage/trait.StorageMap.html) trait using the
+///   [`StorageMap generator`](../paint_support/storage/generator/trait.StorageMap.html).
 ///
 ///   `$hash` representing a choice of hashing algorithms available in the
-///   [`Hashable`](../srml_support/trait.Hashable.html) trait.
+///   [`Hashable`](../paint_support/trait.Hashable.html) trait.
 ///
 ///   `hasher($hash)` is optional and its default is `blake2_256`. One should use another hasher
 ///   with care, see generator documentation.
@@ -72,11 +72,11 @@ use proc_macro::TokenStream;
 ///   * `Hasher`: $hash
 ///
 /// * Linked map: `Foo: linked_map hasher($hash) type => type`: Implements the
-///   [`StorageLinkedMap`](../srml_support/storage/trait.StorageLinkedMap.html) trait using the
-///   [`StorageLinkedMap generator`](../srml_support/storage/generator/trait.StorageLinkedMap.html).
+///   [`StorageLinkedMap`](../paint_support/storage/trait.StorageLinkedMap.html) trait using the
+///   [`StorageLinkedMap generator`](../paint_support/storage/generator/trait.StorageLinkedMap.html).
 ///
 ///   `$hash` representing a choice of hashing algorithms available in the
-///   [`Hashable`](../srml_support/trait.Hashable.html) trait.
+///   [`Hashable`](../paint_support/trait.Hashable.html) trait.
 ///
 ///   `hasher($hash)` is optional and its default is `blake2_256`. One should use another hasher
 ///   with care, see generator documentation.
@@ -86,12 +86,16 @@ use proc_macro::TokenStream;
 ///   * `head_key`: `"head of " ++ $module_prefix ++ " " ++ $storage_name`
 ///   * `Hasher`: $hash
 ///
+///   All key formatting logic can be accessed in a type-agnostic format via the
+///   [`KeyFormat`](../srml_support/storage/generator/trait.KeyFormat.html) trait, which
+///   is implemented for the storage linked map type as well.
+///
 /// * Double map: `Foo: double_map hasher($hash1) u32, $hash2(u32) => u32`: Implements the
-///   [`StorageDoubleMap`](../srml_support/storage/trait.StorageDoubleMap.html) trait using the
-///   [`StorageDoubleMap generator`](../srml_support/storage/generator/trait.StorageDoubleMap.html).
+///   [`StorageDoubleMap`](../paint_support/storage/trait.StorageDoubleMap.html) trait using the
+///   [`StorageDoubleMap generator`](../paint_support/storage/generator/trait.StorageDoubleMap.html).
 ///
 ///   `$hash1` and `$hash2` representing choices of hashing algorithms available in the
-///   [`Hashable`](../srml_support/trait.Hashable.html) trait. They must be choosen with care, see
+///   [`Hashable`](../paint_support/trait.Hashable.html) trait. They must be choosen with care, see
 ///   generator documentation.
 ///
 ///   `hasher($hash)` is optional and its default is `blake2_256`.
diff --git a/srml/support/procedural/src/storage/genesis_config/builder_def.rs b/paint/support/procedural/src/storage/genesis_config/builder_def.rs
similarity index 98%
rename from srml/support/procedural/src/storage/genesis_config/builder_def.rs
rename to paint/support/procedural/src/storage/genesis_config/builder_def.rs
index fc425e4a6d532..12694e1f0610e 100644
--- a/srml/support/procedural/src/storage/genesis_config/builder_def.rs
+++ b/paint/support/procedural/src/storage/genesis_config/builder_def.rs
@@ -16,7 +16,7 @@
 
 //! Builder logic definition used to build genesis storage.
 
-use srml_support_procedural_tools::syn_ext as ext;
+use paint_support_procedural_tools::syn_ext as ext;
 use proc_macro2::TokenStream;
 use syn::spanned::Spanned;
 use quote::{quote, quote_spanned};
diff --git a/srml/support/procedural/src/storage/genesis_config/genesis_config_def.rs b/paint/support/procedural/src/storage/genesis_config/genesis_config_def.rs
similarity index 83%
rename from srml/support/procedural/src/storage/genesis_config/genesis_config_def.rs
rename to paint/support/procedural/src/storage/genesis_config/genesis_config_def.rs
index f9d2f8abe8055..b63c2993b25f6 100644
--- a/srml/support/procedural/src/storage/genesis_config/genesis_config_def.rs
+++ b/paint/support/procedural/src/storage/genesis_config/genesis_config_def.rs
@@ -16,16 +16,16 @@
 
 //! Genesis config defintion.
 
-use srml_support_procedural_tools::syn_ext as ext;
+use paint_support_procedural_tools::syn_ext as ext;
 use proc_macro2::TokenStream;
-use syn::parse_quote;
+use syn::{spanned::Spanned, parse_quote};
 use quote::quote;
 use super::super::{DeclStorageDefExt, StorageLineTypeDef};
 
 pub struct GenesisConfigFieldDef {
-	pub doc: Vec,
 	pub name: syn::Ident,
 	pub typ: syn::Type,
+	pub attrs: Vec,
 	pub default: TokenStream,
 }
 
@@ -43,8 +43,8 @@ pub struct GenesisConfigDef {
 }
 
 impl GenesisConfigDef {
-	pub fn from_def(def: &DeclStorageDefExt) -> Self {
-		let fields = Self::get_genesis_config_field_defs(def);
+	pub fn from_def(def: &DeclStorageDefExt) -> syn::Result {
+		let fields = Self::get_genesis_config_field_defs(def)?;
 
 		let is_generic = fields.iter()
 			.any(|field| ext::type_contains_ident(&field.typ, &def.module_runtime_generic));
@@ -71,17 +71,19 @@ impl GenesisConfigDef {
 			(quote!(), quote!(), quote!(), None)
 		};
 
-		Self {
+		Ok(Self {
 			is_generic,
 			fields,
 			genesis_struct_decl,
 			genesis_struct,
 			genesis_impl,
 			genesis_where_clause,
-		}
+		})
 	}
 
-	fn get_genesis_config_field_defs(def: &DeclStorageDefExt) -> Vec {
+	fn get_genesis_config_field_defs(def: &DeclStorageDefExt)
+		-> syn::Result>
+	{
 		let mut config_field_defs = Vec::new();
 
 		for (config_field, line) in def.storage_lines.iter()
@@ -114,31 +116,39 @@ impl GenesisConfigDef {
 				.unwrap_or_else(|| quote!( Default::default() ));
 
 			config_field_defs.push(GenesisConfigFieldDef {
-				doc: line.doc_attrs.clone(),
 				name: config_field,
 				typ,
+				attrs: line.doc_attrs.clone(),
 				default,
 			});
 		}
 
 		for line in &def.extra_genesis_config_lines {
-			let doc = line.attrs.iter()
-				.filter_map(|a| a.parse_meta().ok())
-				.filter(|m| m.name() == "doc")
-				.collect();
+			let attrs = line.attrs.iter()
+				.map(|attr| {
+					let meta = attr.parse_meta()?;
+					if meta.path().is_ident("cfg") {
+						return Err(syn::Error::new(
+							meta.span(),
+							"extra genesis config items do not support `cfg` attribute"
+						));
+					}
+					Ok(meta)
+				})
+				.collect::>()?;
 
 			let default = line.default.as_ref().map(|e| quote!( #e ))
 				.unwrap_or_else(|| quote!( Default::default() ));
 
 
 			config_field_defs.push(GenesisConfigFieldDef {
-				doc,
 				name: line.name.clone(),
 				typ: line.typ.clone(),
+				attrs,
 				default,
 			});
 		}
 
-		config_field_defs
+		Ok(config_field_defs)
 	}
 }
diff --git a/srml/support/procedural/src/storage/genesis_config/mod.rs b/paint/support/procedural/src/storage/genesis_config/mod.rs
similarity index 90%
rename from srml/support/procedural/src/storage/genesis_config/mod.rs
rename to paint/support/procedural/src/storage/genesis_config/mod.rs
index 2d4d4af3861ed..c222644f78ae3 100644
--- a/srml/support/procedural/src/storage/genesis_config/mod.rs
+++ b/paint/support/procedural/src/storage/genesis_config/mod.rs
@@ -33,13 +33,13 @@ fn decl_genesis_config_and_impl_default(
 	genesis_config: &GenesisConfigDef,
 ) -> TokenStream {
 	let config_fields = genesis_config.fields.iter().map(|field| {
-		let (name, typ, doc) = (&field.name, &field.typ, &field.doc);
-		quote!( #( #[ #doc] )* pub #name: #typ, )
+		let (name, typ, attrs) = (&field.name, &field.typ, &field.attrs);
+		quote!( #( #[ #attrs] )* pub #name: #typ, )
 	});
 
 	let config_field_defaults = genesis_config.fields.iter().map(|field| {
-		let (name, default, doc) = (&field.name, &field.default, &field.doc);
-		quote!( #( #[ #doc] )* #name: #default, )
+		let (name, default) = (&field.name, &field.default);
+		quote!( #name: #default, )
 	});
 
 	let serde_bug_bound = if !genesis_config.fields.is_empty() {
@@ -158,7 +158,7 @@ fn impl_build_storage(
 					#scrate::sr_primitives::ChildrenStorageOverlay,
 				),
 			) -> std::result::Result<(), String> #fn_where_clause {
-				#scrate::with_storage(tuple_storage, || {
+				#scrate::BasicExternalities::execute_with_storage(tuple_storage, || {
 					#( #builder_blocks )*
 					Ok(())
 				})
@@ -188,10 +188,13 @@ pub fn genesis_config_and_build_storage(
 ) -> TokenStream {
 	let builders = BuilderDef::from_def(scrate, def);
 	if !builders.blocks.is_empty() {
-		let genesis_config = &GenesisConfigDef::from_def(def);
+		let genesis_config = match GenesisConfigDef::from_def(def) {
+			Ok(genesis_config) => genesis_config,
+			Err(err) => return err.to_compile_error(),
+		};
 		let decl_genesis_config_and_impl_default =
-			decl_genesis_config_and_impl_default(scrate, genesis_config);
-		let impl_build_storage = impl_build_storage(scrate, def, genesis_config, &builders);
+			decl_genesis_config_and_impl_default(scrate, &genesis_config);
+		let impl_build_storage = impl_build_storage(scrate, def, &genesis_config, &builders);
 
 		quote!{
 			#decl_genesis_config_and_impl_default
diff --git a/srml/support/procedural/src/storage/getters.rs b/paint/support/procedural/src/storage/getters.rs
similarity index 100%
rename from srml/support/procedural/src/storage/getters.rs
rename to paint/support/procedural/src/storage/getters.rs
diff --git a/srml/support/procedural/src/storage/instance_trait.rs b/paint/support/procedural/src/storage/instance_trait.rs
similarity index 100%
rename from srml/support/procedural/src/storage/instance_trait.rs
rename to paint/support/procedural/src/storage/instance_trait.rs
diff --git a/srml/support/procedural/src/storage/metadata.rs b/paint/support/procedural/src/storage/metadata.rs
similarity index 98%
rename from srml/support/procedural/src/storage/metadata.rs
rename to paint/support/procedural/src/storage/metadata.rs
index e280c7d8a20c6..e02e02c3ce875 100644
--- a/srml/support/procedural/src/storage/metadata.rs
+++ b/paint/support/procedural/src/storage/metadata.rs
@@ -16,7 +16,7 @@
 
 //! Implementation of `storage_metadata` on module structure, used by construct_runtime.
 
-use srml_support_procedural_tools::clean_type_string;
+use paint_support_procedural_tools::clean_type_string;
 use proc_macro2::TokenStream;
 use quote::quote;
 use super::{DeclStorageDefExt, StorageLineDefExt, StorageLineTypeDef};
@@ -175,7 +175,7 @@ pub fn impl_metadata(scrate: &TokenStream, def: &DeclStorageDefExt) -> TokenStre
 		let mut docs = TokenStream::new();
 		for attr in line.attrs.iter().filter_map(|v| v.parse_meta().ok()) {
 			if let syn::Meta::NameValue(meta) = attr {
-				if meta.ident == "doc" {
+				if meta.path.is_ident("doc") {
 					let lit = meta.lit;
 					docs.extend(quote!(#lit,));
 				}
diff --git a/srml/support/procedural/src/storage/mod.rs b/paint/support/procedural/src/storage/mod.rs
similarity index 98%
rename from srml/support/procedural/src/storage/mod.rs
rename to paint/support/procedural/src/storage/mod.rs
index 9a6931d87e9c8..ca03f6d3d5626 100644
--- a/srml/support/procedural/src/storage/mod.rs
+++ b/paint/support/procedural/src/storage/mod.rs
@@ -25,7 +25,7 @@ mod instance_trait;
 mod genesis_config;
 
 use quote::quote;
-use srml_support_procedural_tools::{
+use paint_support_procedural_tools::{
 	generate_crate_access, generate_hidden_includes, syn_ext as ext
 };
 
@@ -307,7 +307,7 @@ impl StorageLineDefExt {
 
 		let doc_attrs = storage_def.attrs.iter()
 			.filter_map(|a| a.parse_meta().ok())
-			.filter(|m| m.name() == "doc")
+			.filter(|m| m.path().is_ident("doc"))
 			.collect();
 
 		Self {
@@ -403,8 +403,8 @@ pub fn decl_storage_impl(input: proc_macro::TokenStream) -> proc_macro::TokenStr
 	let hidden_crate_name = def_ext.hidden_crate.as_ref().map(|i| i.to_string())
 		.unwrap_or_else(|| "decl_storage".to_string());
 
-	let scrate = generate_crate_access(&hidden_crate_name, "srml-support");
-	let scrate_decl = generate_hidden_includes(&hidden_crate_name, "srml-support");
+	let scrate = generate_crate_access(&hidden_crate_name, "paint-support");
+	let scrate_decl = generate_hidden_includes(&hidden_crate_name, "paint-support");
 
 	let store_trait = store_trait::decl_and_impl(&def_ext);
 	let getters = getters::impl_getters(&scrate, &def_ext);
diff --git a/srml/support/procedural/src/storage/parse.rs b/paint/support/procedural/src/storage/parse.rs
similarity index 99%
rename from srml/support/procedural/src/storage/parse.rs
rename to paint/support/procedural/src/storage/parse.rs
index e428fbe24f295..a680d5e9ee8b2 100644
--- a/srml/support/procedural/src/storage/parse.rs
+++ b/paint/support/procedural/src/storage/parse.rs
@@ -16,7 +16,7 @@
 
 //! Parsing of decl_storage input.
 
-use srml_support_procedural_tools::{ToTokens, Parse, syn_ext as ext};
+use paint_support_procedural_tools::{ToTokens, Parse, syn_ext as ext};
 use syn::{Ident, Token, spanned::Spanned};
 
 mod keyword {
diff --git a/srml/support/procedural/src/storage/storage_struct.rs b/paint/support/procedural/src/storage/storage_struct.rs
similarity index 97%
rename from srml/support/procedural/src/storage/storage_struct.rs
rename to paint/support/procedural/src/storage/storage_struct.rs
index e195fb53e8a2c..5267876a4416c 100644
--- a/srml/support/procedural/src/storage/storage_struct.rs
+++ b/paint/support/procedural/src/storage/storage_struct.rs
@@ -162,15 +162,12 @@ pub fn decl_and_impl(scrate: &TokenStream, def: &DeclStorageDefExt) -> TokenStre
 					{
 						type Query = #query_type;
 						type Hasher = #scrate::#hasher;
+						type KeyFormat = Self;
 
 						fn prefix() -> &'static [u8] {
 							#final_prefix
 						}
 
-						fn head_key() -> &'static [u8] {
-							#head_key
-						}
-
 						fn from_optional_value_to_query(v: Option<#value_type>) -> Self::Query {
 							#from_optional_value_to_query
 						}
@@ -179,6 +176,14 @@ pub fn decl_and_impl(scrate: &TokenStream, def: &DeclStorageDefExt) -> TokenStre
 							#from_query_to_optional_value
 						}
 					}
+
+					impl<#impl_trait> #scrate::storage::generator::LinkedMapKeyFormat for #storage_struct {
+						type Hasher = #scrate::#hasher;
+
+						fn head_key() -> &'static [u8] {
+							#head_key
+						}
+					}
 				)
 			},
 			StorageLineTypeDef::DoubleMap(map) => {
diff --git a/srml/support/procedural/src/storage/store_trait.rs b/paint/support/procedural/src/storage/store_trait.rs
similarity index 100%
rename from srml/support/procedural/src/storage/store_trait.rs
rename to paint/support/procedural/src/storage/store_trait.rs
diff --git a/paint/support/procedural/tools/Cargo.toml b/paint/support/procedural/tools/Cargo.toml
new file mode 100644
index 0000000000000..7fb906b6ffcca
--- /dev/null
+++ b/paint/support/procedural/tools/Cargo.toml
@@ -0,0 +1,12 @@
+[package]
+name = "paint-support-procedural-tools"
+version = "2.0.0"
+authors = ["Parity Technologies "]
+edition = "2018"
+
+[dependencies]
+paint-support-procedural-tools-derive = { package = "paint-support-procedural-tools-derive", path = "./derive" }
+proc-macro2 = "1.0.6"
+quote = "1.0.2"
+syn = { version = "1.0.7", features = ["full"] }
+proc-macro-crate = "0.1.4"
diff --git a/paint/support/procedural/tools/derive/Cargo.toml b/paint/support/procedural/tools/derive/Cargo.toml
new file mode 100644
index 0000000000000..6b81ee829d1cb
--- /dev/null
+++ b/paint/support/procedural/tools/derive/Cargo.toml
@@ -0,0 +1,13 @@
+[package]
+name = "paint-support-procedural-tools-derive"
+version = "2.0.0"
+authors = ["Parity Technologies "]
+edition = "2018"
+
+[lib]
+proc-macro = true
+
+[dependencies]
+proc-macro2 = "1.0.6"
+quote = { version = "1.0.2", features = ["proc-macro"] }
+syn = { version = "1.0.7", features = ["proc-macro" ,"full", "extra-traits", "parsing"] }
diff --git a/srml/support/procedural/tools/derive/src/lib.rs b/paint/support/procedural/tools/derive/src/lib.rs
similarity index 100%
rename from srml/support/procedural/tools/derive/src/lib.rs
rename to paint/support/procedural/tools/derive/src/lib.rs
diff --git a/srml/support/procedural/tools/src/lib.rs b/paint/support/procedural/tools/src/lib.rs
similarity index 94%
rename from srml/support/procedural/tools/src/lib.rs
rename to paint/support/procedural/tools/src/lib.rs
index 2b08104ca71ba..d76ef24f19d7e 100644
--- a/srml/support/procedural/tools/src/lib.rs
+++ b/paint/support/procedural/tools/src/lib.rs
@@ -19,7 +19,7 @@
 // end::description[]
 
 // reexport proc macros
-pub use srml_support_procedural_tools_derive::*;
+pub use paint_support_procedural_tools_derive::*;
 
 use proc_macro_crate::crate_name;
 use syn::parse::Error;
@@ -35,10 +35,10 @@ fn generate_hidden_includes_mod_name(unique_id: &str) -> Ident {
 	Ident::new(&format!("sr_api_hidden_includes_{}", unique_id), Span::call_site())
 }
 
-/// Generates the access to the `srml-support` crate.
+/// Generates the access to the `paint-support` crate.
 pub fn generate_crate_access(unique_id: &str, def_crate: &str) -> TokenStream {
 	if std::env::var("CARGO_PKG_NAME").unwrap() == def_crate {
-		quote::quote!( srml_support )
+		quote::quote!( paint_support )
 	} else {
 		let mod_name = generate_hidden_includes_mod_name(unique_id);
 		quote::quote!( self::#mod_name::hidden_include )
diff --git a/srml/support/procedural/tools/src/syn_ext.rs b/paint/support/procedural/tools/src/syn_ext.rs
similarity index 93%
rename from srml/support/procedural/tools/src/syn_ext.rs
rename to paint/support/procedural/tools/src/syn_ext.rs
index 1658a6b4aea25..a3f6d52c4c643 100644
--- a/srml/support/procedural/tools/src/syn_ext.rs
+++ b/paint/support/procedural/tools/src/syn_ext.rs
@@ -20,9 +20,9 @@
 
 use syn::{visit::{Visit, self}, parse::{Parse, ParseStream, Result}, Ident};
 use proc_macro2::{TokenStream, TokenTree};
-use quote::{ToTokens, quote};
+use quote::ToTokens;
 use std::iter::once;
-use srml_support_procedural_tools_derive::{ToTokens, Parse};
+use paint_support_procedural_tools_derive::{ToTokens, Parse};
 
 /// stop parsing here getting remaining token as content
 /// Warn duplicate stream (part of)
@@ -124,13 +124,7 @@ impl Parse for Meta {
 impl ToTokens for Meta {
 	fn to_tokens(&self, tokens: &mut TokenStream) {
 		match self.inner {
-			syn::Meta::Word(ref ident) => {
-				let ident = ident.clone();
-				let toks = quote!{
-					#[#ident]
-				};
-				tokens.extend(toks);
-			},
+			syn::Meta::Path(ref path) => path.to_tokens(tokens),
 			syn::Meta::List(ref l) => l.to_tokens(tokens),
 			syn::Meta::NameValue(ref n) => n.to_tokens(tokens),
 		}
@@ -187,10 +181,10 @@ impl ToTokens for Opt
 {
 pub fn extract_type_option(typ: &syn::Type) -> Option {
 	if let syn::Type::Path(ref path) = typ {
 		let v = path.path.segments.last()?;
-		if v.value().ident == "Option" {
+		if v.ident == "Option" {
 			// Option has only one type argument in angle bracket.
-			if let syn::PathArguments::AngleBracketed(a) = &v.value().arguments {
-				if let syn::GenericArgument::Type(typ) = a.args.last()?.value() {
+			if let syn::PathArguments::AngleBracketed(a) = &v.arguments {
+				if let syn::GenericArgument::Type(typ) = a.args.last()? {
 					return Some(typ.clone())
 				}
 			}
@@ -230,7 +224,7 @@ impl<'ast> Visit<'ast> for ContainsIdent<'ast> {
 	}
 
 	fn visit_macro(&mut self, input: &'ast syn::Macro) {
-		self.visit_tokenstream(input.tts.clone());
+		self.visit_tokenstream(input.tokens.clone());
 		visit::visit_macro(self, input);
 	}
 }
diff --git a/paint/support/rpc/Cargo.toml b/paint/support/rpc/Cargo.toml
new file mode 100644
index 0000000000000..17533950102b2
--- /dev/null
+++ b/paint/support/rpc/Cargo.toml
@@ -0,0 +1,19 @@
+[package]
+name = "paint-support-rpc"
+version = "2.0.0"
+authors = ["Parity Technologies ", "Andrew Dirksen "]
+edition = "2018"
+
+[dependencies]
+futures = { version = "0.3.0", features = ["compat"] }
+jsonrpc-client-transports = "14"
+jsonrpc-core = "14"
+parity-scale-codec = "1"
+serde = "1"
+paint-support = { path = "../" }
+substrate-primitives-storage = { path = "../../../primitives/core/storage" }
+substrate-rpc-api = { path = "../../../client/rpc/api" }
+
+[dev-dependencies]
+paint-system = { path = "../../system" }
+tokio = "0.1"
diff --git a/paint/support/rpc/src/lib.rs b/paint/support/rpc/src/lib.rs
new file mode 100644
index 0000000000000..a09fbd694c872
--- /dev/null
+++ b/paint/support/rpc/src/lib.rs
@@ -0,0 +1,155 @@
+// Copyright 2019 Parity Technologies (UK) Ltd.
+// This file is part of Substrate.
+
+// Substrate is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+
+// Substrate is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+
+// You should have received a copy of the GNU General Public License
+// along with Substrate.  If not, see .
+
+//! Combines [substrate_rpc_api::state::StateClient] with [paint_support::storage::generator] traits
+//! to provide strongly typed chain state queries over rpc.
+
+#![warn(missing_docs)]
+
+use core::marker::PhantomData;
+use futures::compat::Future01CompatExt;
+use jsonrpc_client_transports::RpcError;
+use parity_scale_codec::{DecodeAll, FullCodec, FullEncode};
+use serde::{de::DeserializeOwned, Serialize};
+use paint_support::storage::generator::{
+	StorageDoubleMap, StorageLinkedMap, StorageMap, StorageValue
+};
+use substrate_primitives_storage::{StorageData, StorageKey};
+use substrate_rpc_api::state::StateClient;
+
+/// A typed query on chain state usable from an RPC client.
+///
+/// ```no_run
+/// # use futures::compat::Compat;
+/// # use futures::compat::Future01CompatExt;
+/// # use futures::future::FutureExt;
+/// # use jsonrpc_client_transports::RpcError;
+/// # use jsonrpc_client_transports::transports::http;
+/// # use parity_scale_codec::Encode;
+/// # use paint_support::{decl_storage, decl_module};
+/// # use paint_support_rpc::StorageQuery;
+/// # use paint_system::Trait;
+/// # use substrate_rpc_api::state::StateClient;
+/// #
+/// # // Hash would normally be ::Hash, but we don't have
+/// # // paint_system::Trait implemented for TestRuntime. Here we just pretend.
+/// # type Hash = ();
+/// #
+/// # fn main() -> Result<(), RpcError> {
+/// #     tokio::runtime::Runtime::new().unwrap().block_on(Compat::new(test().boxed()))
+/// # }
+/// #
+/// # struct TestRuntime;
+/// #
+/// # decl_module! {
+///	#     pub struct Module for enum Call where origin: T::Origin {}
+/// # }
+/// #
+/// pub type Loc = (i64, i64, i64);
+/// pub type Block = u8;
+///
+/// // Note that all fields are marked pub.
+/// decl_storage! {
+///     trait Store for Module as TestRuntime {
+///         pub LastActionId: u64;
+///         pub Voxels: map Loc => Block;
+///         pub Actions: linked_map u64 => Loc;
+///         pub Prefab: double_map u128, blake2_256((i8, i8, i8)) => Block;
+///     }
+/// }
+///
+/// # async fn test() -> Result<(), RpcError> {
+/// let conn = http::connect("http://[::1]:9933").compat().await?;
+/// let cl = StateClient::::new(conn);
+///
+/// let q = StorageQuery::value::();
+/// let _: Option = q.get(&cl, None).await?;
+///
+/// let q = StorageQuery::map::((0, 0, 0));
+/// let _: Option = q.get(&cl, None).await?;
+///
+/// let q = StorageQuery::linked_map::(12);
+/// let _: Option = q.get(&cl, None).await?;
+///
+/// let q = StorageQuery::double_map::(3, (0, 0, 0));
+/// let _: Option = q.get(&cl, None).await?;
+/// #
+/// # Ok(())
+/// # }
+/// ```
+#[derive(Clone, Eq, PartialEq, Ord, PartialOrd, Hash, Debug)]
+pub struct StorageQuery {
+	key: StorageKey,
+	_spook: PhantomData,
+}
+
+impl StorageQuery {
+	/// Create a storage query for a StorageValue.
+	pub fn value>() -> Self {
+		Self {
+			key: StorageKey(St::storage_value_final_key().to_vec()),
+			_spook: PhantomData,
+		}
+	}
+
+	/// Create a storage query for a value in a StorageMap.
+	pub fn map, K: FullEncode>(key: K) -> Self {
+		Self {
+			key: StorageKey(St::storage_map_final_key(key).as_ref().to_vec()),
+			_spook: PhantomData,
+		}
+	}
+
+	/// Create a storage query for a value in a StorageLinkedMap.
+	pub fn linked_map, K: FullCodec>(key: K) -> Self {
+		Self {
+			key: StorageKey(St::storage_linked_map_final_key(key).as_ref().to_vec()),
+			_spook: PhantomData,
+		}
+	}
+
+	/// Create a storage query for a value in a StorageDoubleMap.
+	pub fn double_map, K1: FullEncode, K2: FullEncode>(
+		key1: K1,
+		key2: K2,
+	) -> Self {
+		Self {
+			key: StorageKey(St::storage_double_map_final_key(key1, key2)),
+			_spook: PhantomData,
+		}
+	}
+
+	/// Send this query over RPC, await the typed result.
+	///
+	/// Hash should be ::Hash.
+	///
+	/// # Arguments
+	///
+	/// state_client represents a connection to the RPC server.
+	///
+	/// block_index indicates the block for which state will be queried. A value of None indicates
+	/// the latest block.
+	pub async fn get(
+		self,
+		state_client: &StateClient,
+		block_index: Option,
+	) -> Result, RpcError> {
+		let opt: Option = state_client.storage(self.key, block_index).compat().await?;
+		opt.map(|encoded| V::decode_all(&encoded.0))
+			.transpose()
+			.map_err(|decode_err| RpcError::Other(decode_err.into()))
+	}
+}
diff --git a/srml/support/src/debug.rs b/paint/support/src/debug.rs
similarity index 96%
rename from srml/support/src/debug.rs
rename to paint/support/src/debug.rs
index 1c4e463bf1234..8904d3defef63 100644
--- a/srml/support/src/debug.rs
+++ b/paint/support/src/debug.rs
@@ -35,7 +35,7 @@
 //! that WASM debug formatting of structs will be empty.
 //!
 //! ```rust,no_run
-//!	use srml_support::debug;
+//!	use paint_support::debug;
 //!
 //! #[derive(primitives::RuntimeDebug)]
 //!	struct MyStruct {
@@ -66,7 +66,7 @@
 //! logs conditionally and strips out logs in WASM.
 //!
 //! ```rust,no_run
-//!	use srml_support::debug::native;
+//!	use paint_support::debug::native;
 //!
 //! #[derive(primitives::RuntimeDebug)]
 //!	struct MyStruct {
@@ -120,6 +120,12 @@ pub mod native {
 }
 
 /// Print out a formatted message.
+///
+/// # Example
+///
+/// ```
+/// paint_support::runtime_print!("my value is {}", 3);
+/// ```
 #[macro_export]
 macro_rules! runtime_print {
 	($($arg:tt)+) => {
@@ -149,7 +155,7 @@ impl fmt::Write for Writer {
 impl Writer {
 	/// Print the content of this `Writer` out.
 	pub fn print(&self) {
-		runtime_io::print_utf8(&self.0)
+		runtime_io::misc::print_utf8(&self.0)
 	}
 }
 
@@ -198,9 +204,9 @@ impl log::Log for RuntimeLogger {
 		let mut w = Writer::default();
 		let _ = core::write!(&mut w, "{}", record.args());
 
-		runtime_io::log(
+		runtime_io::logging::log(
 			record.level().into(),
-			record.target().as_bytes(),
+			record.target(),
 			&w.0,
 		);
 	}
diff --git a/srml/support/src/dispatch.rs b/paint/support/src/dispatch.rs
similarity index 90%
rename from srml/support/src/dispatch.rs
rename to paint/support/src/dispatch.rs
index df86f436117a1..d985ce80cd62d 100644
--- a/srml/support/src/dispatch.rs
+++ b/paint/support/src/dispatch.rs
@@ -19,14 +19,14 @@
 
 pub use crate::rstd::{result, fmt, prelude::{Vec, Clone, Eq, PartialEq}, marker};
 pub use crate::codec::{Codec, EncodeLike, Decode, Encode, Input, Output, HasCompact, EncodeAsRef};
-pub use srml_metadata::{
+pub use paint_metadata::{
 	FunctionMetadata, DecodeDifferent, DecodeDifferentArray, FunctionArgumentMetadata,
 	ModuleConstantMetadata, DefaultByte, DefaultByteGetter, ModuleErrorMetadata, ErrorMetadata
 };
 pub use sr_primitives::{
 	weights::{
 		SimpleDispatchInfo, GetDispatchInfo, DispatchInfo, WeighData, ClassifyDispatch,
-		TransactionPriority
+		TransactionPriority, Weight, WeighBlock,
 	},
 	traits::{Dispatchable, DispatchResult, ModuleDispatchError},
 	DispatchError,
@@ -57,9 +57,9 @@ impl Parameter for T where T: Codec + EncodeLike + Clone + Eq + fmt::Debug {}
 ///
 /// ```
 /// # #[macro_use]
-/// # extern crate srml_support;
-/// # use srml_support::dispatch::Result;
-/// # use srml_system::{self as system, Trait, ensure_signed};
+/// # extern crate paint_support;
+/// # use paint_support::dispatch::Result;
+/// # use paint_system::{self as system, Trait, ensure_signed};
 /// decl_module! {
 /// 	pub struct Module for enum Call where origin: T::Origin {
 ///
@@ -97,9 +97,9 @@ impl Parameter for T where T: Codec + EncodeLike + Clone + Eq + fmt::Debug {}
 ///
 /// ```
 /// # #[macro_use]
-/// # extern crate srml_support;
-/// # use srml_support::dispatch::Result;
-/// # use srml_system::{self as system, Trait, ensure_signed};
+/// # extern crate paint_support;
+/// # use paint_support::dispatch::Result;
+/// # use paint_system::{self as system, Trait, ensure_signed};
 /// decl_module! {
 /// 	pub struct Module for enum Call where origin: T::Origin {
 ///
@@ -122,9 +122,9 @@ impl Parameter for T where T: Codec + EncodeLike + Clone + Eq + fmt::Debug {}
 ///
 /// ```
 /// # #[macro_use]
-/// # extern crate srml_support;
-/// # use srml_support::dispatch::Result;
-/// # use srml_system::{self as system, Trait, ensure_signed, ensure_root};
+/// # extern crate paint_support;
+/// # use paint_support::dispatch::Result;
+/// # use paint_system::{self as system, Trait, ensure_signed, ensure_root};
 /// decl_module! {
 /// 	pub struct Module for enum Call where origin: T::Origin {
 ///			fn my_privileged_function(origin) -> Result {
@@ -140,15 +140,15 @@ impl Parameter for T where T: Codec + EncodeLike + Clone + Eq + fmt::Debug {}
 /// ## Multiple Module Instances Example
 ///
 /// A Substrate module can be built such that multiple instances of the same module can be used within a single
-/// runtime. For example, the [Balances module](../srml_balances/index.html) can be added multiple times to your
+/// runtime. For example, the [Balances module](../paint_balances/index.html) can be added multiple times to your
 /// runtime in order to support multiple, independent currencies for your blockchain. Here is an example of how
 /// you would declare such a module using the `decl_module!` macro:
 ///
 /// ```
 /// # #[macro_use]
-/// # extern crate srml_support;
-/// # use srml_support::dispatch::Result;
-/// # use srml_system::{self as system, ensure_signed};
+/// # extern crate paint_support;
+/// # use paint_support::dispatch::Result;
+/// # use paint_system::{self as system, ensure_signed};
 /// # pub struct DefaultInstance;
 /// # pub trait Instance {}
 /// # impl Instance for DefaultInstance {}
@@ -174,9 +174,9 @@ impl Parameter for T where T: Codec + EncodeLike + Clone + Eq + fmt::Debug {}
 ///
 /// ```
 /// # #[macro_use]
-/// # extern crate srml_support;
-/// # use srml_support::dispatch::Result;
-/// # use srml_system::{self as system, ensure_signed};
+/// # extern crate paint_support;
+/// # use paint_support::dispatch::Result;
+/// # use paint_system::{self as system, ensure_signed};
 /// pub trait Trait: system::Trait where Self::AccountId: From {}
 ///
 /// decl_module! {
@@ -192,7 +192,7 @@ impl Parameter for T where T: Codec + EncodeLike + Clone + Eq + fmt::Debug {}
 /// The following are reserved function signatures:
 ///
 /// * `deposit_event`: Helper function for depositing an [event](https://docs.substrate.dev/docs/event-enum).
-/// The default behavior is to call `deposit_event` from the [System module](../srml_system/index.html).
+/// The default behavior is to call `deposit_event` from the [System module](../paint_system/index.html).
 /// However, you can write your own implementation for events in your runtime. To use the default behavior,
 /// add `fn deposit_event() = default;` to your `Module`.
 ///
@@ -320,6 +320,7 @@ macro_rules! decl_module {
 			"`deposit_event` function is reserved and must follow the syntax: `$vis:vis fn deposit_event() = default;`"
 		);
 	};
+	// Add on_finalize, without a given weight.
 	(@normalize
 		$(#[$attr:meta])*
 		pub struct $mod_type:ident<$trait_instance:ident: $trait_name:ident$(, I: $instantiable:path $(= $module_default_instance:path)?)?>
@@ -343,7 +344,10 @@ macro_rules! decl_module {
 			{ $( $other_where_bounds )* }
 			{ $( $deposit_event )* }
 			{ $( $on_initialize )* }
-			{ fn on_finalize( $( $param_name : $param ),* ) { $( $impl )* } }
+			{
+				#[weight = $crate::dispatch::SimpleDispatchInfo::zero()]
+				fn on_finalize( $( $param_name : $param ),* ) { $( $impl )* }
+			}
 			{ $( $offchain )* }
 			{ $( $constants )* }
 			{ $( $error_type )* }
@@ -351,12 +355,51 @@ macro_rules! decl_module {
 			$($rest)*
 		);
 	};
+	// Add on_finalize, given weight.
 	(@normalize
 		$(#[$attr:meta])*
 		pub struct $mod_type:ident<$trait_instance:ident: $trait_name:ident$(, I: $instantiable:path $(= $module_default_instance:path)?)?>
 		for enum $call_type:ident where origin: $origin_type:ty, system = $system:ident
 		{ $( $other_where_bounds:tt )* }
 		{ $( $deposit_event:tt )* }
+		{ $( $on_initialize:tt )* }
+		{}
+		{ $( $offchain:tt )* }
+		{ $( $constants:tt )* }
+		{ $( $error_type:tt )* }
+		[ $( $dispatchables:tt )* ]
+		$(#[doc = $doc_attr:tt])*
+		#[weight = $weight:expr]
+		fn on_finalize( $( $param_name:ident : $param:ty ),* $(,)? ) { $( $impl:tt )* }
+		$($rest:tt)*
+	) => {
+		$crate::decl_module!(@normalize
+			$(#[$attr])*
+			pub struct $mod_type<$trait_instance: $trait_name$(, I: $instantiable $(= $module_default_instance)?)?>
+			for enum $call_type where origin: $origin_type, system = $system
+			{ $( $other_where_bounds )* }
+			{ $( $deposit_event )* }
+			{ $( $on_initialize )* }
+			{
+				#[weight = $weight]
+				fn on_finalize( $( $param_name : $param ),* ) { $( $impl )* }
+			}
+			{ $( $offchain )* }
+			{ $( $constants )* }
+			{ $( $error_type )* }
+			[ $( $dispatchables )* ]
+			$($rest)*
+		);
+	};
+	// Add on_initialize, without a given weight.
+	(@normalize
+		$(#[$attr:meta])*
+		pub struct $mod_type:ident<
+			$trait_instance:ident: $trait_name:ident$(, I: $instantiable:path $(= $module_default_instance:path)?)?
+		>
+		for enum $call_type:ident where origin: $origin_type:ty, system = $system:ident
+		{ $( $other_where_bounds:tt )* }
+		{ $( $deposit_event:tt )* }
 		{}
 		{ $( $on_finalize:tt )* }
 		{ $( $offchain:tt )* }
@@ -373,7 +416,48 @@ macro_rules! decl_module {
 			for enum $call_type where origin: $origin_type, system = $system
 			{ $( $other_where_bounds )* }
 			{ $( $deposit_event )* }
-			{ fn on_initialize( $( $param_name : $param ),* ) { $( $impl )* } }
+			{
+				#[weight = $crate::dispatch::SimpleDispatchInfo::zero()]
+				fn on_initialize( $( $param_name : $param ),* ) { $( $impl )* }
+			}
+			{ $( $on_finalize )* }
+			{ $( $offchain )* }
+			{ $( $constants )* }
+			{ $( $error_type )* }
+			[ $( $dispatchables )* ]
+			$($rest)*
+		);
+	};
+	// Add on_initialize, given weight.
+	(@normalize
+		$(#[$attr:meta])*
+		pub struct $mod_type:ident<
+			$trait_instance:ident: $trait_name:ident$(, I: $instantiable:path $(= $module_default_instance:path)?)?
+		>
+		for enum $call_type:ident where origin: $origin_type:ty, system = $system:ident
+		{ $( $other_where_bounds:tt )* }
+		{ $( $deposit_event:tt )* }
+		{}
+		{ $( $on_finalize:tt )* }
+		{ $( $offchain:tt )* }
+		{ $( $constants:tt )* }
+		{ $( $error_type:tt )* }
+		[ $( $dispatchables:tt )* ]
+		$(#[doc = $doc_attr:tt])*
+		#[weight = $weight:expr]
+		fn on_initialize( $( $param_name:ident : $param:ty ),* $(,)? ) { $( $impl:tt )* }
+		$($rest:tt)*
+	) => {
+		$crate::decl_module!(@normalize
+			$(#[$attr])*
+			pub struct $mod_type<$trait_instance: $trait_name$(, I: $instantiable $(= $module_default_instance)?)?>
+			for enum $call_type where origin: $origin_type, system = $system
+			{ $( $other_where_bounds )* }
+			{ $( $deposit_event )* }
+			{
+				#[weight = $weight]
+				fn on_initialize( $( $param_name : $param ),* ) { $( $impl )* }
+			}
 			{ $( $on_finalize )* }
 			{ $( $offchain )* }
 			{ $( $constants )* }
@@ -770,6 +854,7 @@ macro_rules! decl_module {
 	(@impl_on_initialize
 		$module:ident<$trait_instance:ident: $trait_name:ident$(, $instance:ident: $instantiable:path)?>;
 		{ $( $other_where_bounds:tt )* }
+		#[weight = $weight:expr]
 		fn on_initialize() { $( $impl:tt )* }
 	) => {
 		impl<$trait_instance: $trait_name$(, $instance: $instantiable)?>
@@ -783,6 +868,7 @@ macro_rules! decl_module {
 	(@impl_on_initialize
 		$module:ident<$trait_instance:ident: $trait_name:ident$(, $instance:ident: $instantiable:path)?>;
 		{ $( $other_where_bounds:tt )* }
+		#[weight = $weight:expr]
 		fn on_initialize($param:ident : $param_ty:ty) { $( $impl:tt )* }
 	) => {
 		impl<$trait_instance: $trait_name$(, $instance: $instantiable)?>
@@ -806,6 +892,7 @@ macro_rules! decl_module {
 	(@impl_on_finalize
 		$module:ident<$trait_instance:ident: $trait_name:ident$(, $instance:ident: $instantiable:path)?>;
 		{ $( $other_where_bounds:tt )* }
+		#[weight = $weight:expr]
 		fn on_finalize() { $( $impl:tt )* }
 	) => {
 		impl<$trait_instance: $trait_name$(, $instance: $instantiable)?>
@@ -819,6 +906,7 @@ macro_rules! decl_module {
 	(@impl_on_finalize
 		$module:ident<$trait_instance:ident: $trait_name:ident$(, $instance:ident: $instantiable:path)?>;
 		{ $( $other_where_bounds:tt )* }
+		#[weight = $weight:expr]
 		fn on_finalize($param:ident : $param_ty:ty) { $( $impl:tt )* }
 	) => {
 		impl<$trait_instance: $trait_name$(, $instance: $instantiable)?>
@@ -840,6 +928,35 @@ macro_rules! decl_module {
 		}
 	};
 
+	(@impl_block_hooks_weight
+		$module:ident<$trait_instance:ident: $trait_name:ident$(, $instance:ident: $instantiable:path)?>;
+		{ $( $other_where_bounds:tt )* }
+		@init $(
+			#[weight = $weight_initialize:expr]
+			fn on_initialize($( $param_initialize:ident : $param_ty_initialize:ty )*) { $( $impl_initialize:tt )* }
+		)?
+		@fin $(
+			#[weight = $weight_finalize:expr]
+			fn on_finalize($( $param_finalize:ident : $param_ty_finalize:ty )*) { $( $impl_finalize:tt )* }
+		)?
+	) => {
+		impl<$trait_instance: $trait_name$(, $instance: $instantiable)?>
+		$crate::dispatch::WeighBlock<$trait_instance::BlockNumber> for $module<$trait_instance$(, $instance)?> where
+			$( $other_where_bounds )*
+		{
+			$(
+				fn on_initialize(n: $trait_instance::BlockNumber) -> $crate::dispatch::Weight {
+					>::weigh_data(&$weight_initialize, n)
+				}
+			)?
+			$(
+				fn on_finalize(n: $trait_instance::BlockNumber) -> $crate::dispatch::Weight {
+					>::weigh_data(&$weight_finalize, n)
+				}
+			)?
+		}
+	};
+
 	(@impl_offchain
 		$module:ident<$trait_instance:ident: $trait_name:ident$(, $instance:ident: $instantiable:path)?>;
 		{ $( $other_where_bounds:tt )* }
@@ -1083,6 +1200,14 @@ macro_rules! decl_module {
 			$( $on_finalize )*
 		}
 
+		$crate::decl_module! {
+			@impl_block_hooks_weight
+			$mod_type<$trait_instance: $trait_name $(, $instance: $instantiable)?>;
+			{ $( $other_where_bounds )* }
+			@init $( $on_initialize )*
+			@fin $( $on_finalize )*
+		}
+
 		$crate::decl_module! {
 			@impl_offchain
 			$mod_type<$trait_instance: $trait_name $(, $instance: $instantiable)?>;
@@ -1727,6 +1852,14 @@ mod tests {
 		}
 	}
 
+	struct BLockWeight;
+	impl> WeighData for BLockWeight {
+		fn weigh_data(&self, target: BlockNumber) -> Weight {
+			let target: u32 = target.into();
+			if target % 2 == 0 { 10 } else { 0 }
+		}
+	}
+
 	decl_module! {
 		pub struct Module for enum Call where origin: T::Origin, T::AccountId: From {
 			/// Hi, this is a comment.
@@ -1738,7 +1871,9 @@ mod tests {
 			fn aux_4(_origin, _data: i32) -> Result { unreachable!() }
 			fn aux_5(_origin, _data: i32, #[compact] _data2: u32,) -> Result { unreachable!() }
 
+			#[weight = SimpleDispatchInfo::FixedNormal(7)]
 			fn on_initialize(n: T::BlockNumber,) { if n.into() == 42 { panic!("on_initialize") } }
+			#[weight = BLockWeight]
 			fn on_finalize(n: T::BlockNumber) { if n.into() == 42 { panic!("on_finalize") } }
 			fn offchain_worker() {}
 
@@ -1899,4 +2034,16 @@ mod tests {
 			DispatchInfo { weight: 3, class: DispatchClass::Normal },
 		);
 	}
+
+	#[test]
+	fn weight_for_block_hooks() {
+		// independent of block number
+		assert_eq!(>::on_initialize(0), 7);
+		assert_eq!(>::on_initialize(10), 7);
+		assert_eq!(>::on_initialize(100), 7);
+
+		// dependent
+		assert_eq!(>::on_finalize(2), 10);
+		assert_eq!(>::on_finalize(3), 0);
+	}
 }
diff --git a/srml/support/src/error.rs b/paint/support/src/error.rs
similarity index 97%
rename from srml/support/src/error.rs
rename to paint/support/src/error.rs
index 9aa13713daafb..4b2e0a232d4d3 100644
--- a/srml/support/src/error.rs
+++ b/paint/support/src/error.rs
@@ -18,7 +18,7 @@
 
 #[doc(hidden)]
 pub use sr_primitives::traits::LookupError;
-pub use srml_metadata::{ModuleErrorMetadata, ErrorMetadata, DecodeDifferent};
+pub use paint_metadata::{ModuleErrorMetadata, ErrorMetadata, DecodeDifferent};
 
 /// Declare an error type for a runtime module.
 ///
@@ -31,7 +31,7 @@ pub use srml_metadata::{ModuleErrorMetadata, ErrorMetadata, DecodeDifferent};
 /// # Usage
 ///
 /// ```
-/// # use srml_support::decl_error;
+/// # use paint_support::decl_error;
 /// decl_error! {
 ///     /// Errors that can occur in my module.
 ///     pub enum MyError {
diff --git a/srml/support/src/event.rs b/paint/support/src/event.rs
similarity index 98%
rename from srml/support/src/event.rs
rename to paint/support/src/event.rs
index 3411c93922e2b..139db1f609b9a 100644
--- a/srml/support/src/event.rs
+++ b/paint/support/src/event.rs
@@ -17,14 +17,14 @@
 // You should have received a copy of the GNU General Public License
 // along with Substrate.  If not, see .
 
-pub use srml_metadata::{EventMetadata, DecodeDifferent, OuterEventMetadata, FnEncode};
+pub use paint_metadata::{EventMetadata, DecodeDifferent, OuterEventMetadata, FnEncode};
 
 /// Implement the `Event` for a module.
 ///
 /// # Simple Event Example:
 ///
 /// ```rust
-/// srml_support::decl_event!(
+/// paint_support::decl_event!(
 ///    pub enum Event {
 ///       Success,
 ///       Failure(String),
@@ -44,7 +44,7 @@ pub use srml_metadata::{EventMetadata, DecodeDifferent, OuterEventMetadata, FnEn
 ///
 /// mod event1 {
 ///     // Event that specifies the generic parameter explicitly (`Balance`).
-///     srml_support::decl_event!(
+///     paint_support::decl_event!(
 ///        pub enum Event where Balance = ::Balance {
 ///           Message(Balance),
 ///        }
@@ -55,7 +55,7 @@ pub use srml_metadata::{EventMetadata, DecodeDifferent, OuterEventMetadata, FnEn
 ///     // Event that uses the generic parameter `Balance`.
 ///     // If no name for the generic parameter is specified explicitly,
 ///     // the name will be taken from the type name of the trait.
-///     srml_support::decl_event!(
+///     paint_support::decl_event!(
 ///        pub enum Event where ::Balance {
 ///           Message(Balance),
 ///        }
@@ -64,7 +64,7 @@ pub use srml_metadata::{EventMetadata, DecodeDifferent, OuterEventMetadata, FnEn
 ///
 /// mod event3 {
 ///     // And we even support declaring multiple generic parameters!
-///     srml_support::decl_event!(
+///     paint_support::decl_event!(
 ///        pub enum Event where ::Balance, ::Token {
 ///           Message(Balance, Token),
 ///        }
@@ -88,7 +88,7 @@ pub use srml_metadata::{EventMetadata, DecodeDifferent, OuterEventMetadata, FnEn
 /// }
 ///
 /// // For module with instances, DefaultInstance is optional
-/// srml_support::decl_event!(
+/// paint_support::decl_event!(
 ///    pub enum Event where
 ///       ::Balance,
 ///       ::Token
diff --git a/srml/support/src/hash.rs b/paint/support/src/hash.rs
similarity index 94%
rename from srml/support/src/hash.rs
rename to paint/support/src/hash.rs
index cbd78f603241e..122fefab70e96 100644
--- a/srml/support/src/hash.rs
+++ b/paint/support/src/hash.rs
@@ -18,9 +18,9 @@
 
 use codec::Codec;
 use rstd::prelude::Vec;
-use runtime_io::{blake2_128, blake2_256, twox_64, twox_128, twox_256};
+use runtime_io::hashing::{blake2_128, blake2_256, twox_64, twox_128, twox_256};
 
-// This trait must be kept coherent with srml-support-procedural HasherKind usage
+// This trait must be kept coherent with paint-support-procedural HasherKind usage
 pub trait Hashable: Sized {
 	fn blake2_128(&self) -> [u8; 16];
 	fn blake2_256(&self) -> [u8; 32];
diff --git a/srml/support/src/inherent.rs b/paint/support/src/inherent.rs
similarity index 100%
rename from srml/support/src/inherent.rs
rename to paint/support/src/inherent.rs
diff --git a/srml/support/src/lib.rs b/paint/support/src/lib.rs
similarity index 94%
rename from srml/support/src/lib.rs
rename to paint/support/src/lib.rs
index 2a9f66bd526e2..37dd2b4f9d47a 100644
--- a/srml/support/src/lib.rs
+++ b/paint/support/src/lib.rs
@@ -18,8 +18,8 @@
 
 #![cfg_attr(not(feature = "std"), no_std)]
 
-/// Export ourself as `srml_support` to make tests happy.
-extern crate self as srml_support;
+/// Export ourself as `paint_support` to make tests happy.
+extern crate self as paint_support;
 
 #[macro_use]
 extern crate bitmask;
@@ -37,9 +37,9 @@ pub use once_cell;
 pub use paste;
 #[cfg(feature = "std")]
 #[doc(hidden)]
-pub use runtime_io::with_storage;
+pub use state_machine::BasicExternalities;
 #[doc(hidden)]
-pub use runtime_io::storage_root;
+pub use runtime_io::storage::root as storage_root;
 #[doc(hidden)]
 pub use sr_primitives::RuntimeDebug;
 
@@ -47,7 +47,6 @@ pub use sr_primitives::RuntimeDebug;
 pub mod debug;
 #[macro_use]
 pub mod dispatch;
-#[macro_use]
 pub mod storage;
 mod hash;
 #[macro_use]
@@ -114,7 +113,7 @@ macro_rules! parameter_types {
 }
 
 #[doc(inline)]
-pub use srml_support_procedural::decl_storage;
+pub use paint_support_procedural::decl_storage;
 
 /// Return Err of the expression: `return Err($expression);`.
 ///
@@ -191,8 +190,8 @@ macro_rules! assert_ok {
 ///
 /// ```rust
 /// #[macro_use]
-/// # extern crate srml_support;
-/// # use srml_support::{assert_eq_uvec};
+/// # extern crate paint_support;
+/// # use paint_support::{assert_eq_uvec};
 /// # fn main() {
 /// assert_eq_uvec!(vec![1,2], vec![2,1]);
 /// # }
@@ -200,8 +199,8 @@ macro_rules! assert_ok {
 ///
 /// ```rust,should_panic
 /// #[macro_use]
-/// # extern crate srml_support;
-/// # use srml_support::{assert_eq_uvec};
+/// # extern crate paint_support;
+/// # use paint_support::{assert_eq_uvec};
 /// # fn main() {
 /// assert_eq_uvec!(vec![1,2,3], vec![2,1]);
 /// # }
@@ -239,7 +238,7 @@ pub use serde::{Serialize, Deserialize};
 mod tests {
 	use super::*;
 	use codec::{Codec, EncodeLike};
-	use srml_metadata::{
+	use paint_metadata::{
 		DecodeDifferent, StorageEntryMetadata, StorageMetadata, StorageEntryType,
 		StorageEntryModifier, DefaultByteGetter, StorageHasher,
 	};
@@ -263,16 +262,21 @@ mod tests {
 
 	decl_storage! {
 		trait Store for Module as Example {
-			pub Data get(fn data) build(|_| vec![(15u32, 42u64)]): linked_map hasher(twox_64_concat) u32 => u64;
+			pub Data get(fn data) build(|_| vec![(15u32, 42u64)]):
+				linked_map hasher(twox_64_concat) u32 => u64;
 			pub OptionLinkedMap: linked_map u32 => Option;
-			pub GenericData get(fn generic_data): linked_map hasher(twox_128) T::BlockNumber => T::BlockNumber;
-			pub GenericData2 get(fn generic_data2): linked_map T::BlockNumber => Option;
+			pub GenericData get(fn generic_data):
+				linked_map hasher(twox_128) T::BlockNumber => T::BlockNumber;
+			pub GenericData2 get(fn generic_data2):
+				linked_map T::BlockNumber => Option;
 			pub GetterNoFnKeyword get(no_fn): Option;
 
 			pub DataDM config(test_config) build(|_| vec![(15u32, 16u32, 42u64)]):
 				double_map hasher(twox_64_concat) u32, blake2_256(u32) => u64;
-			pub GenericDataDM: double_map T::BlockNumber, twox_128(T::BlockNumber) => T::BlockNumber;
-			pub GenericData2DM: double_map T::BlockNumber, twox_256(T::BlockNumber) => Option;
+			pub GenericDataDM:
+				double_map T::BlockNumber, twox_128(T::BlockNumber) => T::BlockNumber;
+			pub GenericData2DM:
+				double_map T::BlockNumber, twox_256(T::BlockNumber) => Option;
 			pub AppendableDM: double_map u32, blake2_256(T::BlockNumber) => Vec;
 		}
 	}
diff --git a/srml/support/src/metadata.rs b/paint/support/src/metadata.rs
similarity index 98%
rename from srml/support/src/metadata.rs
rename to paint/support/src/metadata.rs
index a223a14f9e405..f55cddea9914e 100644
--- a/srml/support/src/metadata.rs
+++ b/paint/support/src/metadata.rs
@@ -14,7 +14,7 @@
 // You should have received a copy of the GNU General Public License
 // along with Substrate.  If not, see .
 
-pub use srml_metadata::{
+pub use paint_metadata::{
 	DecodeDifferent, FnEncode, RuntimeMetadata, ModuleMetadata, RuntimeMetadataLastVersion,
 	DefaultByteGetter, RuntimeMetadataPrefixed, StorageEntryMetadata, StorageMetadata,
 	StorageEntryType, StorageEntryModifier, DefaultByte, StorageHasher, ModuleErrorMetadata
@@ -29,11 +29,11 @@ pub use srml_metadata::{
 ///#        type Origin;
 ///#        type BlockNumber;
 ///#    }
-///#    srml_support::decl_module! {
+///#    paint_support::decl_module! {
 ///#        pub struct Module for enum Call where origin: T::Origin {}
 ///#    }
 ///#
-///#    srml_support::decl_storage! {
+///#    paint_support::decl_storage! {
 ///#        trait Store for Module as TestStorage {}
 ///#    }
 ///# }
@@ -45,7 +45,7 @@ pub use srml_metadata::{
 ///# }
 ///
 /// struct Runtime;
-/// srml_support::impl_runtime_metadata! {
+/// paint_support::impl_runtime_metadata! {
 ///     for Runtime with modules
 ///         module0::Module as Module0 with,
 ///         module1::Module as Module1 with,
@@ -229,7 +229,7 @@ macro_rules! __runtime_modules_to_metadata_calls_storage {
 #[allow(dead_code)]
 mod tests {
 	use super::*;
-	use srml_metadata::{
+	use paint_metadata::{
 		EventMetadata, StorageEntryModifier, StorageEntryType, FunctionMetadata, StorageEntryMetadata,
 		ModuleMetadata, RuntimeMetadataPrefixed, DefaultByte, ModuleConstantMetadata, DefaultByteGetter,
 		ErrorMetadata,
diff --git a/srml/support/src/origin.rs b/paint/support/src/origin.rs
similarity index 100%
rename from srml/support/src/origin.rs
rename to paint/support/src/origin.rs
diff --git a/srml/support/src/runtime.rs b/paint/support/src/runtime.rs
similarity index 100%
rename from srml/support/src/runtime.rs
rename to paint/support/src/runtime.rs
diff --git a/srml/support/src/storage/child.rs b/paint/support/src/storage/child.rs
similarity index 85%
rename from srml/support/src/storage/child.rs
rename to paint/support/src/storage/child.rs
index 1d6ee7a6f1a61..d43c2e896f3d8 100644
--- a/srml/support/src/storage/child.rs
+++ b/paint/support/src/storage/child.rs
@@ -26,7 +26,7 @@ use codec::{Codec, Encode, Decode};
 
 /// Return the value of the item in storage under `key`, or `None` if there is no explicit entry.
 pub fn get(storage_key: &[u8], key: &[u8]) -> Option {
-	runtime_io::child_storage(storage_key, key).map(|v| {
+	runtime_io::storage::child_get(storage_key, key).map(|v| {
 		Decode::decode(&mut &v[..]).expect("storage is not null, therefore must be a valid type")
 	})
 }
@@ -45,13 +45,17 @@ pub fn get_or(storage_key: &[u8], key: &[u8], default_value:
 
 /// Return the value of the item in storage under `key`, or `default_value()` if there is no
 /// explicit entry.
-pub fn get_or_else T>(storage_key: &[u8], key: &[u8], default_value: F) -> T {
+pub fn get_or_else T>(
+	storage_key: &[u8],
+	key: &[u8],
+	default_value: F,
+) -> T {
 	get(storage_key, key).unwrap_or_else(default_value)
 }
 
 /// Put `value` in storage under `key`.
 pub fn put(storage_key: &[u8], key: &[u8], value: &T) {
-	value.using_encoded(|slice| runtime_io::set_child_storage(storage_key, key, slice));
+	value.using_encoded(|slice| runtime_io::storage::child_set(storage_key, key, slice));
 }
 
 /// Remove `key` from storage, returning its value if it had an explicit entry or `None` otherwise.
@@ -77,31 +81,35 @@ pub fn take_or(storage_key: &[u8],key: &[u8], default_value: T
 
 /// Return the value of the item in storage under `key`, or `default_value()` if there is no
 /// explicit entry. Ensure there is no explicit entry on return.
-pub fn take_or_else T>(storage_key: &[u8], key: &[u8], default_value: F) -> T {
+pub fn take_or_else T>(
+	storage_key: &[u8],
+	key: &[u8],
+	default_value: F,
+) -> T {
 	take(storage_key, key).unwrap_or_else(default_value)
 }
 
 /// Check to see if `key` has an explicit entry in storage.
 pub fn exists(storage_key: &[u8], key: &[u8]) -> bool {
-	runtime_io::read_child_storage(storage_key, key, &mut [0;0][..], 0).is_some()
+	runtime_io::storage::child_read(storage_key, key, &mut [0;0][..], 0).is_some()
 }
 
 /// Remove all `storage_key` key/values
 pub fn kill_storage(storage_key: &[u8]) {
-	runtime_io::kill_child_storage(storage_key)
+	runtime_io::storage::child_storage_kill(storage_key)
 }
 
 /// Ensure `key` has no explicit entry in storage.
 pub fn kill(storage_key: &[u8], key: &[u8]) {
-	runtime_io::clear_child_storage(storage_key, key);
+	runtime_io::storage::child_clear(storage_key, key);
 }
 
 /// Get a Vec of bytes from storage.
 pub fn get_raw(storage_key: &[u8], key: &[u8]) -> Option> {
-	runtime_io::child_storage(storage_key, key)
+	runtime_io::storage::child_get(storage_key, key)
 }
 
 /// Put a raw byte slice into storage.
 pub fn put_raw(storage_key: &[u8], key: &[u8], value: &[u8]) {
-	runtime_io::set_child_storage(storage_key, key, value)
+	runtime_io::storage::child_set(storage_key, key, value)
 }
diff --git a/srml/support/src/storage/generator/double_map.rs b/paint/support/src/storage/generator/double_map.rs
similarity index 100%
rename from srml/support/src/storage/generator/double_map.rs
rename to paint/support/src/storage/generator/double_map.rs
diff --git a/srml/support/src/storage/generator/linked_map.rs b/paint/support/src/storage/generator/linked_map.rs
similarity index 61%
rename from srml/support/src/storage/generator/linked_map.rs
rename to paint/support/src/storage/generator/linked_map.rs
index 6643a37f4f50c..9e2722d5fc496 100644
--- a/srml/support/src/storage/generator/linked_map.rs
+++ b/paint/support/src/storage/generator/linked_map.rs
@@ -47,16 +47,21 @@ pub trait StorageLinkedMap {
 	/// Hasher used to insert into storage.
 	type Hasher: StorageHasher;
 
+	/// The family of key formats used for this map.
+	type KeyFormat: KeyFormat;
+
 	/// Prefix used to prepend each key.
 	fn prefix() -> &'static [u8];
 
-	/// Key used to store linked map head.
-	fn head_key() -> &'static [u8];
+	/// The head key of the linked-map.
+	fn head_key() -> &'static [u8] {
+		::head_key()
+	}
 
-	/// Convert an optionnal value retrieved from storage to the type queried.
+	/// Convert an optional value retrieved from storage to the type queried.
 	fn from_optional_value_to_query(v: Option) -> Self::Query;
 
-	/// Convert a query to an optionnal value into storage.
+	/// Convert a query to an optional value into storage.
 	fn from_query_to_optional_value(v: Self::Query) -> Option;
 
 	/// Generate the full key used in top storage.
@@ -64,14 +69,37 @@ pub trait StorageLinkedMap {
 	where
 		KeyArg: EncodeLike,
 	{
-		let mut final_key = Self::prefix().to_vec();
-		key.encode_to(&mut final_key);
-		Self::Hasher::hash(&final_key)
+		::storage_linked_map_final_key::(Self::prefix(), &key)
 	}
 
 	/// Generate the hashed key for head
 	fn storage_linked_map_final_head_key() -> ::Output {
-		Self::Hasher::hash(Self::head_key())
+		::storage_linked_map_final_head_key()
+	}
+}
+
+/// A type-abstracted key format used for a family of linked-map types.
+pub trait KeyFormat {
+	type Hasher: StorageHasher;
+
+	/// Key used to store linked map head.
+	fn head_key() -> &'static [u8];
+
+	/// Generate the full key used in top storage.
+	fn storage_linked_map_final_key(prefix: &[u8], key: &K)
+		-> ::Output
+	where
+		K: Encode,
+	{
+		let mut final_key = prefix.to_vec();
+		key.encode_to(&mut final_key);
+		::hash(&final_key)
+	}
+
+	fn storage_linked_map_final_head_key()
+		-> ::Output
+	{
+		::hash(Self::head_key())
 	}
 }
 
@@ -105,25 +133,39 @@ struct EncodeLikeLinkage, NKey: EncodeLike, Key: Enco
 }
 
 /// A key-value pair iterator for enumerable map.
-pub struct Enumerator> {
+pub struct Enumerator {
 	next: Option,
-	_phantom: PhantomData<(G, V)>,
+	prefix: &'static [u8],
+	_phantom: PhantomData<(V, F)>,
+}
+
+impl Enumerator {
+	/// Create an explicit enumerator for testing.
+	#[cfg(test)]
+	pub fn from_head(head: K, prefix: &'static [u8]) -> Self {
+		Enumerator {
+			next: Some(head),
+			prefix,
+			_phantom: Default::default(),
+		}
+	}
 }
 
-impl Iterator for Enumerator
+impl Iterator for Enumerator
 where
 	K: FullCodec,
 	V: FullCodec,
-	G: StorageLinkedMap,
+	F: KeyFormat,
 {
 	type Item = (K, V);
 
 	fn next(&mut self) -> Option {
 		let next = self.next.take()?;
+
 		let (val, linkage): (V, Linkage) = {
-			let next_full_key = G::storage_linked_map_final_key(&next);
-			unhashed::get(next_full_key.as_ref())
-				.expect("previous/next only contain existing entires;
+			let next_full_key = F::storage_linked_map_final_key(self.prefix, &next);
+			read_with_linkage::(next_full_key.as_ref())
+				.expect("previous/next only contains existing entries;
 						we enumerate using next; entry exists; qed")
 		};
 
@@ -136,33 +178,33 @@ where
 ///
 /// Takes care of updating previous and next elements points
 /// as well as updates head if the element is first or last.
-fn remove_linkage(linkage: Linkage)
+fn remove_linkage(linkage: Linkage, prefix: &[u8])
 where
 	K: FullCodec,
 	V: FullCodec,
-	G: StorageLinkedMap,
+	F: KeyFormat,
 {
 	let next_key = linkage.next.as_ref()
-		.map(G::storage_linked_map_final_key)
+		.map(|k| F::storage_linked_map_final_key(prefix, k))
 		.map(|x| x.as_ref().to_vec());
 	let prev_key = linkage.previous.as_ref()
-		.map(G::storage_linked_map_final_key)
+		.map(|k| F::storage_linked_map_final_key(prefix, k))
 		.map(|x| x.as_ref().to_vec());
 
 	if let Some(prev_key) = prev_key {
 		// Retrieve previous element and update `next`
-		let mut res = read_with_linkage::<_, _, G>(prev_key.as_ref())
+		let mut res = read_with_linkage::(prev_key.as_ref())
 			.expect("Linkage is updated in case entry is removed;
 					it always points to existing keys; qed");
 		res.1.next = linkage.next;
 		unhashed::put(prev_key.as_ref(), &res);
 	} else {
 		// we were first so let's update the head
-		write_head::<_, _, _, G>(linkage.next.as_ref());
+		write_head::<&K, K, F>(linkage.next.as_ref());
 	}
 	if let Some(next_key) = next_key {
 		// Update previous of next element
-		let mut res = read_with_linkage::<_, _, G>(next_key.as_ref())
+		let mut res = read_with_linkage::(next_key.as_ref())
 			.expect("Linkage is updated in case entry is removed;
 					it always points to existing keys; qed");
 		res.1.previous = linkage.previous;
@@ -170,12 +212,11 @@ where
 	}
 }
 
-/// Read the contained data and it's linkage.
-fn read_with_linkage(key: &[u8]) -> Option<(V, Linkage)>
+/// Read the contained data and its linkage.
+pub(super) fn read_with_linkage(key: &[u8]) -> Option<(V, Linkage)>
 where
-	K: FullCodec,
-	V: FullCodec,
-	G: StorageLinkedMap,
+	K: Decode,
+	V: Decode,
 {
 	unhashed::get(key)
 }
@@ -183,18 +224,18 @@ where
 /// Generate linkage for newly inserted element.
 ///
 /// Takes care of updating head and previous head's pointer.
-fn new_head_linkage(key: KeyArg) -> Linkage
+pub(super) fn new_head_linkage(key: KeyArg, prefix: &[u8]) -> Linkage
 where
 	KeyArg: EncodeLike,
 	K: FullCodec,
 	V: FullCodec,
-	G: StorageLinkedMap,
+	F: KeyFormat,
 {
-	if let Some(head) = read_head::<_, _, G>() {
+	if let Some(head) = read_head::() {
 		// update previous head predecessor
 		{
-			let head_key = G::storage_linked_map_final_key(&head);
-			let (data, linkage) = read_with_linkage::<_, _, G>(head_key.as_ref())
+			let head_key = F::storage_linked_map_final_key(prefix, &head);
+			let (data, linkage) = read_with_linkage::(head_key.as_ref())
 				.expect("head is set when first element is inserted
 						and unset when last element is removed;
 						if head is Some then it points to existing key; qed");
@@ -206,41 +247,39 @@ where
 			unhashed::put(head_key.as_ref(), &(data, new_linkage));
 		}
 		// update to current head
-		write_head::<_, _, _, G>(Some(key));
+		write_head::<_, _, F>(Some(key));
 		// return linkage with pointer to previous head
 		let mut linkage = Linkage::default();
 		linkage.next = Some(head);
 		linkage
 	} else {
 		// we are first - update the head and produce empty linkage
-		write_head::<_, _, _, G>(Some(key));
+		write_head::<_, _, F>(Some(key));
 		Linkage::default()
 	}
 }
 
 /// Read current head pointer.
-fn read_head() -> Option
+pub(crate) fn read_head() -> Option
 where
-	K: FullCodec,
-	V: FullCodec,
-	G: StorageLinkedMap,
+	K: Decode,
+	F: KeyFormat,
 {
-	unhashed::get(G::storage_linked_map_final_head_key().as_ref())
+	unhashed::get(F::storage_linked_map_final_head_key().as_ref())
 }
 
 /// Overwrite current head pointer.
 ///
 /// If `None` is given head is removed from storage.
-fn write_head(head: Option)
+pub(super) fn write_head(head: Option)
 where
 	KeyArg: EncodeLike,
 	K: FullCodec,
-	V: FullCodec,
-	G: StorageLinkedMap,
+	F: KeyFormat,
 {
 	match head.as_ref() {
-		Some(head) => unhashed::put(G::storage_linked_map_final_head_key().as_ref(), head),
-		None => unhashed::kill(G::storage_linked_map_final_head_key().as_ref()),
+		Some(head) => unhashed::put(F::storage_linked_map_final_head_key().as_ref(), head),
+		None => unhashed::kill(F::storage_linked_map_final_head_key().as_ref()),
 	}
 }
 
@@ -252,7 +291,7 @@ where
 {
 	type Query = G::Query;
 
-	type Enumerator = Enumerator;
+	type Enumerator = Enumerator;
 
 	fn exists>(key: KeyArg) -> bool {
 		unhashed::exists(Self::storage_linked_map_final_key(key).as_ref())
@@ -264,10 +303,11 @@ where
 	}
 
 	fn swap, KeyArg2: EncodeLike>(key1: KeyArg1, key2: KeyArg2) {
+		let prefix = Self::prefix();
 		let final_key1 = Self::storage_linked_map_final_key(Ref::from(&key1));
 		let final_key2 = Self::storage_linked_map_final_key(Ref::from(&key2));
-		let full_value_1 = read_with_linkage::<_, _, G>(final_key1.as_ref());
-		let full_value_2 = read_with_linkage::<_, _, G>(final_key2.as_ref());
+		let full_value_1 = read_with_linkage::(final_key1.as_ref());
+		let full_value_2 = read_with_linkage::(final_key2.as_ref());
 
 		match (full_value_1, full_value_2) {
 			// Just keep linkage in order and only swap values.
@@ -278,13 +318,13 @@ where
 			// Remove key and insert the new one.
 			(Some((value, _linkage)), None) => {
 				Self::remove(key1);
-				let linkage = new_head_linkage::<_, _, _, G>(key2);
+				let linkage = new_head_linkage::<_, _, V, G::KeyFormat>(key2, prefix);
 				unhashed::put(final_key2.as_ref(), &(value, linkage));
 			}
 			// Remove key and insert the new one.
 			(None, Some((value, _linkage))) => {
 				Self::remove(key2);
-				let linkage = new_head_linkage::<_, _, _, G>(key1);
+				let linkage = new_head_linkage::<_, _, V, G::KeyFormat>(key1, prefix);
 				unhashed::put(final_key1.as_ref(), &(value, linkage));
 			}
 			// No-op.
@@ -294,11 +334,11 @@ where
 
 	fn insert, ValArg: EncodeLike>(key: KeyArg, val: ValArg) {
 		let final_key = Self::storage_linked_map_final_key(Ref::from(&key));
-		let linkage = match read_with_linkage::<_, _, G>(final_key.as_ref()) {
+		let linkage = match read_with_linkage::<_, V>(final_key.as_ref()) {
 			// overwrite but reuse existing linkage
 			Some((_data, linkage)) => linkage,
 			// create new linkage
-			None => new_head_linkage::<_, _, _, G>(key),
+			None => new_head_linkage::<_, _, V, G::KeyFormat>(key, Self::prefix()),
 		};
 		unhashed::put(final_key.as_ref(), &(val, linkage))
 	}
@@ -310,7 +350,7 @@ where
 	fn mutate, R, F: FnOnce(&mut Self::Query) -> R>(key: KeyArg, f: F) -> R {
 		let final_key = Self::storage_linked_map_final_key(Ref::from(&key));
 
-		let (mut val, _linkage) = read_with_linkage::<_, _, G>(final_key.as_ref())
+		let (mut val, _linkage) = read_with_linkage::(final_key.as_ref())
 			.map(|(data, linkage)| (G::from_optional_value_to_query(Some(data)), Some(linkage)))
 			.unwrap_or_else(|| (G::from_optional_value_to_query(None), None));
 
@@ -328,7 +368,7 @@ where
 		let full_value: Option<(V, Linkage)> = unhashed::take(final_key.as_ref());
 
 		let value = full_value.map(|(data, linkage)| {
-			remove_linkage::<_, _, G>(linkage);
+			remove_linkage::(linkage, Self::prefix());
 			data
 		});
 
@@ -336,14 +376,15 @@ where
 	}
 
 	fn enumerate() -> Self::Enumerator {
-		Enumerator::<_, _, G> {
-			next: read_head::<_, _, G>(),
+		Enumerator::<_, _, G::KeyFormat> {
+			next: read_head::<_, G::KeyFormat>(),
+			prefix: Self::prefix(),
 			_phantom: Default::default(),
 		}
 	}
 
 	fn head() -> Option {
-		read_head::<_, _, G>()
+		read_head::<_, G::KeyFormat>()
 	}
 
 	fn decode_len>(key: KeyArg) -> Result
@@ -360,4 +401,67 @@ where
 			Ok(len)
 		}
 	}
+
+	fn translate(translate_key: TK, translate_val: TV) -> Result<(), Option>
+		where K2: FullCodec + Clone, V2: Decode, TK: Fn(K2) -> K, TV: Fn(V2) -> V
+	{
+		let head_key = read_head::().ok_or(None)?;
+		let prefix = G::prefix();
+
+		let mut last_key = None;
+		let mut current_key = head_key.clone();
+
+		write_head::<&K, K, G::KeyFormat>(Some(&translate_key(head_key)));
+
+		let translate_linkage = |old: Linkage| -> Linkage {
+			Linkage {
+				previous: old.previous.map(&translate_key),
+				next: old.next.map(&translate_key),
+			}
+		};
+
+		loop {
+			let old_raw_key = G::KeyFormat::storage_linked_map_final_key(prefix, ¤t_key);
+			let x = unhashed::take(old_raw_key.as_ref());
+			let (val, linkage): (V2, Linkage) = match x {
+				Some(v) => v,
+				None => {
+					// we failed to read value and linkage. Update the last key's linkage
+					// to end the map early, since it's impossible to iterate further.
+					if let Some(last_key) = last_key {
+						let last_raw_key = G::storage_linked_map_final_key(&last_key);
+						if let Some((val, mut linkage))
+							= read_with_linkage::(last_raw_key.as_ref())
+						{
+							// defensive: should always happen, since it was just written
+							// in the last iteration of the loop.
+							linkage.next = None;
+							unhashed::put(last_raw_key.as_ref(), &(&val, &linkage));
+						}
+					}
+
+					return Err(Some(current_key));
+				}
+			};
+			let next = linkage.next.clone();
+
+			let val = translate_val(val);
+			let linkage = translate_linkage(linkage);
+
+			// and write in the value and linkage under the new key.
+			let new_key = translate_key(current_key.clone());
+			let new_raw_key = G::storage_linked_map_final_key(&new_key);
+			unhashed::put(new_raw_key.as_ref(), &(&val, &linkage));
+
+			match next {
+				None => break,
+				Some(next) => {
+					last_key = Some(new_key);
+					current_key = next
+				},
+			}
+		}
+
+		Ok(())
+	}
 }
diff --git a/srml/support/src/storage/generator/map.rs b/paint/support/src/storage/generator/map.rs
similarity index 100%
rename from srml/support/src/storage/generator/map.rs
rename to paint/support/src/storage/generator/map.rs
diff --git a/paint/support/src/storage/generator/mod.rs b/paint/support/src/storage/generator/mod.rs
new file mode 100644
index 0000000000000..f546546dc7209
--- /dev/null
+++ b/paint/support/src/storage/generator/mod.rs
@@ -0,0 +1,132 @@
+// Copyright 2019 Parity Technologies (UK) Ltd.
+// This file is part of Substrate.
+
+// Substrate is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+
+// Substrate is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+
+// You should have received a copy of the GNU General Public License
+// along with Substrate.  If not, see .
+
+//! Generators are a set of trait on which storage traits are implemented.
+//!
+//! (i.e. implementing the generator for StorageValue on a type will automatically derive the
+//! implementation of StorageValue for this type).
+//!
+//! They are used by `decl_storage`.
+
+mod linked_map;
+mod map;
+mod double_map;
+mod value;
+
+pub use linked_map::{StorageLinkedMap, Enumerator, Linkage, KeyFormat as LinkedMapKeyFormat};
+pub use map::StorageMap;
+pub use double_map::StorageDoubleMap;
+pub use value::StorageValue;
+
+
+#[cfg(test)]
+#[allow(dead_code)]
+mod tests {
+	use runtime_io::TestExternalities;
+	use codec::{Encode, Decode};
+	use crate::storage::{unhashed, generator::{StorageValue, StorageLinkedMap}};
+
+	struct Runtime {}
+	pub trait Trait {
+		type Origin;
+		type BlockNumber;
+	}
+
+	impl Trait for Runtime {
+		type Origin = u32;
+		type BlockNumber = u32;
+	}
+
+	decl_module! {
+		pub struct Module for enum Call where origin: T::Origin {}
+	}
+
+	#[derive(Encode, Decode, Clone, Debug, Eq, PartialEq)]
+	struct NumberNumber {
+		a: u32,
+		b: u32,
+	}
+
+	crate::decl_storage! {
+		trait Store for Module as Runtime {
+			Value get(fn value) config(): (u64, u64);
+			NumberMap: linked_map NumberNumber => u64;
+		}
+	}
+
+	#[test]
+	fn value_translate_works() {
+		let t = GenesisConfig::default().build_storage().unwrap();
+		TestExternalities::new(t).execute_with(|| {
+			// put the old value `1111u32` in the storage.
+			let key = Value::storage_value_final_key();
+			unhashed::put_raw(&key, &1111u32.encode());
+
+			// translate
+			let translate_fn = |old: Option| -> Option<(u64, u64)> {
+				old.map(|o| (o.into(), (o*2).into()))
+			};
+			let _ = Value::translate(translate_fn);
+
+			// new storage should be `(1111, 1111 * 2)`
+			assert_eq!(Value::get(), (1111, 2222));
+		})
+	}
+
+	#[test]
+	fn linked_map_translate_works() {
+		use super::linked_map::{self, Enumerator, KeyFormat};
+
+		type Format = >::KeyFormat;
+
+		let t = GenesisConfig::default().build_storage().unwrap();
+		TestExternalities::new(t).execute_with(|| {
+			let prefix = NumberMap::prefix();
+
+			// start with a map of u32 -> u32.
+			for i in 0u32..100u32 {
+				let final_key = ::storage_linked_map_final_key(
+					prefix, &i,
+				);
+
+				let linkage = linked_map::new_head_linkage::<_, u32, u32, Format>(&i, prefix);
+				unhashed::put(final_key.as_ref(), &(&i, linkage));
+			}
+
+			let head = linked_map::read_head::().unwrap();
+
+			assert_eq!(
+				Enumerator::::from_head(head, prefix).collect::>(),
+				(0..100).rev().map(|x| (x, x)).collect::>(),
+			);
+
+			// do translation.
+			NumberMap::translate(
+				|k: u32| NumberNumber { a: k, b: k },
+				|v: u32| (v as u64) << 32 | v as u64,
+			).unwrap();
+
+			assert!(linked_map::read_head::().is_some());
+			assert_eq!(
+				NumberMap::enumerate().collect::>(),
+				(0..100u32).rev().map(|x| (
+					NumberNumber { a: x, b: x },
+					(x as u64) << 32 | x as u64,
+				)).collect::>(),
+			);
+		})
+	}
+}
diff --git a/srml/support/src/storage/generator/value.rs b/paint/support/src/storage/generator/value.rs
similarity index 88%
rename from srml/support/src/storage/generator/value.rs
rename to paint/support/src/storage/generator/value.rs
index 8423503dde20e..5ebc25a70af2a 100644
--- a/srml/support/src/storage/generator/value.rs
+++ b/paint/support/src/storage/generator/value.rs
@@ -16,7 +16,7 @@
 
 #[cfg(not(feature = "std"))]
 use rstd::prelude::*;
-use codec::{FullCodec, Encode, EncodeAppend, EncodeLike};
+use codec::{FullCodec, Encode, EncodeAppend, EncodeLike, Decode};
 use crate::{storage::{self, unhashed}, hash::{Twox128, StorageHasher}, traits::Len};
 
 /// Generator for `StorageValue` used by `decl_storage`.
@@ -60,6 +60,23 @@ impl> storage::StorageValue for G {
 		G::from_optional_value_to_query(value)
 	}
 
+	fn translate) -> Option>(f: F) -> Result, ()> {
+		let key = Self::storage_value_final_key();
+
+		// attempt to get the length directly.
+		let maybe_old = match unhashed::get_raw(&key) {
+			Some(old_data) => Some(O::decode(&mut &old_data[..]).map_err(|_| ())?),
+			None => None,
+		};
+		let maybe_new = f(maybe_old);
+		if let Some(new) = maybe_new.as_ref() {
+			new.using_encoded(|d| unhashed::put_raw(&key, d));
+		} else {
+			unhashed::kill(&key);
+		}
+		Ok(maybe_new)
+	}
+
 	fn put>(val: Arg) {
 		unhashed::put(&Self::storage_value_final_key(), &val)
 	}
diff --git a/srml/support/src/storage/hashed.rs b/paint/support/src/storage/hashed.rs
similarity index 100%
rename from srml/support/src/storage/hashed.rs
rename to paint/support/src/storage/hashed.rs
diff --git a/srml/support/src/storage/mod.rs b/paint/support/src/storage/mod.rs
similarity index 79%
rename from srml/support/src/storage/mod.rs
rename to paint/support/src/storage/mod.rs
index 648009b470e42..af64d2a179002 100644
--- a/srml/support/src/storage/mod.rs
+++ b/paint/support/src/storage/mod.rs
@@ -17,17 +17,18 @@
 //! Stuff to do with the runtime's storage.
 
 use rstd::prelude::*;
-use codec::{FullCodec, FullEncode, Encode, EncodeAppend, EncodeLike};
+use codec::{FullCodec, FullEncode, Encode, EncodeAppend, EncodeLike, Decode};
 use crate::traits::Len;
 
-#[macro_use]
-pub mod storage_items;
 pub mod unhashed;
 pub mod hashed;
 pub mod child;
 pub mod generator;
 
 /// A trait for working with macro-generated storage values under the substrate storage API.
+///
+/// Details on implementation can be found at
+/// [`generator::StorageValue`]
 pub trait StorageValue {
 	/// The type that get/take return.
 	type Query;
@@ -41,6 +42,28 @@ pub trait StorageValue {
 	/// Load the value from the provided storage instance.
 	fn get() -> Self::Query;
 
+	/// Translate a value from some previous type (`O`) to the current type.
+	///
+	/// `f: F` is the translation function.
+	///
+	/// Returns `Err` if the storage item could not be interpreted as the old type, and Ok, along
+	/// with the new value if it could.
+	///
+	/// NOTE: This operates from and to `Option<_>` types; no effort is made to respect the default
+	/// value of the original type.
+	///
+	/// # Warning
+	///
+	/// This function must be used with care, before being updated the storage still contains the
+	/// old type, thus other calls (such as `get`) will fail at decoding it.
+	///
+	/// # Usage
+	///
+	/// This would typically be called inside the module implementation of on_initialize, while
+	/// ensuring **no usage of this storage are made before the call to `on_initialize`**. (More
+	/// precisely prior initialized modules doesn't make use of this storage).
+	fn translate) -> Option>(f: F) -> Result, ()>;
+
 	/// Store a value under this key into the provided storage instance.
 	fn put>(val: Arg);
 
@@ -91,6 +114,9 @@ pub trait StorageValue {
 }
 
 /// A strongly-typed map in storage.
+///
+/// Details on implementation can be found at
+/// [`generator::StorageMap`]
 pub trait StorageMap {
 	/// The type that get/take return.
 	type Query;
@@ -158,6 +184,9 @@ pub trait StorageMap {
 /// A strongly-typed linked map in storage.
 ///
 /// Similar to `StorageMap` but allows to enumerate other elements and doesn't implement append.
+///
+/// Details on implementation can be found at
+/// [`generator::StorageLinkedMap`]
 pub trait StorageLinkedMap {
 	/// The type that get/take return.
 	type Query;
@@ -201,12 +230,36 @@ pub trait StorageLinkedMap {
 	/// function for this purpose.
 	fn decode_len>(key: KeyArg) -> Result
 		where V: codec::DecodeLength + Len;
+
+	/// Translate the keys and values from some previous `(K2, V2)` to the current type.
+	///
+	/// `TK` translates keys from the old type, and `TV` translates values.
+	///
+	/// Returns `Err` if the map could not be interpreted as the old type, and Ok if it could.
+	/// The `Err` contains the first key which could not be migrated, or `None` if the
+	/// head of the list could not be read.
+	///
+	/// # Warning
+	///
+	/// This function must be used with care, before being updated the storage still contains the
+	/// old type, thus other calls (such as `get`) will fail at decoding it.
+	///
+	/// # Usage
+	///
+	/// This would typically be called inside the module implementation of on_initialize, while
+	/// ensuring **no usage of this storage are made before the call to `on_initialize`**. (More
+	/// precisely prior initialized modules doesn't make use of this storage).
+	fn translate(translate_key: TK, translate_val: TV) -> Result<(), Option>
+		where K2: FullCodec + Clone, V2: Decode, TK: Fn(K2) -> K, TV: Fn(V2) -> V;
 }
 
 /// An implementation of a map with a two keys.
 ///
 /// It provides an important ability to efficiently remove all entries
 /// that have a common first key.
+///
+/// Details on implementation can be found at
+/// [`generator::StorageDoubleMap`]
 pub trait StorageDoubleMap {
 	/// The type that get/take returns.
 	type Query;
diff --git a/srml/support/src/storage/unhashed.rs b/paint/support/src/storage/unhashed.rs
similarity index 91%
rename from srml/support/src/storage/unhashed.rs
rename to paint/support/src/storage/unhashed.rs
index 6397fd39fcd2d..803a512b2a2c4 100644
--- a/srml/support/src/storage/unhashed.rs
+++ b/paint/support/src/storage/unhashed.rs
@@ -21,7 +21,7 @@ use codec::{Encode, Decode};
 
 /// Return the value of the item in storage under `key`, or `None` if there is no explicit entry.
 pub fn get(key: &[u8]) -> Option {
-	runtime_io::storage(key).map(|val| {
+	runtime_io::storage::get(key).map(|val| {
 		Decode::decode(&mut &val[..]).expect("storage is not null, therefore must be a valid type")
 	})
 }
@@ -46,7 +46,7 @@ pub fn get_or_else T>(key: &[u8], default_valu
 
 /// Put `value` in storage under `key`.
 pub fn put(key: &[u8], value: &T) {
-	value.using_encoded(|slice| runtime_io::set_storage(key, slice));
+	value.using_encoded(|slice| runtime_io::storage::set(key, slice));
 }
 
 /// Remove `key` from storage, returning its value if it had an explicit entry or `None` otherwise.
@@ -78,25 +78,25 @@ pub fn take_or_else T>(key: &[u8], default_val
 
 /// Check to see if `key` has an explicit entry in storage.
 pub fn exists(key: &[u8]) -> bool {
-	runtime_io::read_storage(key, &mut [0;0][..], 0).is_some()
+	runtime_io::storage::read(key, &mut [0;0][..], 0).is_some()
 }
 
 /// Ensure `key` has no explicit entry in storage.
 pub fn kill(key: &[u8]) {
-	runtime_io::clear_storage(key);
+	runtime_io::storage::clear(key);
 }
 
 /// Ensure keys with the given `prefix` have no entries in storage.
 pub fn kill_prefix(prefix: &[u8]) {
-	runtime_io::clear_prefix(prefix);
+	runtime_io::storage::clear_prefix(prefix);
 }
 
 /// Get a Vec of bytes from storage.
 pub fn get_raw(key: &[u8]) -> Option> {
-	runtime_io::storage(key)
+	runtime_io::storage::get(key)
 }
 
 /// Put a raw byte slice into storage.
 pub fn put_raw(key: &[u8], value: &[u8]) {
-	runtime_io::set_storage(key, value)
+	runtime_io::storage::set(key, value)
 }
diff --git a/srml/support/src/traits.rs b/paint/support/src/traits.rs
similarity index 96%
rename from srml/support/src/traits.rs
rename to paint/support/src/traits.rs
index b321aeaeb9fe9..b723860739d3d 100644
--- a/srml/support/src/traits.rs
+++ b/paint/support/src/traits.rs
@@ -69,17 +69,6 @@ pub trait OnFreeBalanceZero {
 	fn on_free_balance_zero(who: &AccountId);
 }
 
-/// Trait for a hook to get called when some balance has been minted, causing dilution.
-pub trait OnDilution {
-	/// Some `portion` of the total balance just "grew" by `minted`. `portion` is the pre-growth
-	/// amount (it doesn't take account of the recent growth).
-	fn on_dilution(minted: Balance, portion: Balance);
-}
-
-impl OnDilution for () {
-	fn on_dilution(_minted: Balance, _portion: Balance) {}
-}
-
 /// Outcome of a balance update.
 pub enum UpdateBalanceOutcome {
 	/// Account balance was simply updated.
@@ -143,13 +132,19 @@ pub trait KeyOwnerProofSystem {
 ///
 /// - Someone got slashed.
 /// - Someone paid for a transaction to be included.
-pub trait OnUnbalanced {
+pub trait OnUnbalanced {
 	/// Handler for some imbalance. Infallible.
-	fn on_unbalanced(amount: Imbalance);
+	fn on_unbalanced(amount: Imbalance) {
+		amount.try_drop().unwrap_or_else(Self::on_nonzero_unbalanced)
+	}
+
+	/// Actually handle a non-zero imbalance. You probably want to implement this rather than
+	/// `on_unbalanced`.
+	fn on_nonzero_unbalanced(amount: Imbalance);
 }
 
-impl OnUnbalanced for () {
-	fn on_unbalanced(amount: Imbalance) { drop(amount); }
+impl OnUnbalanced for () {
+	fn on_nonzero_unbalanced(amount: Imbalance) { drop(amount); }
 }
 
 /// Simple boolean for whether an account needs to be kept in existence.
@@ -164,6 +159,12 @@ pub enum ExistenceRequirement {
 	AllowDeath,
 }
 
+/// A type for which some values make sense to be able to drop without further consideration.
+pub trait TryDrop: Sized {
+	/// Drop an instance cleanly. Only works if its value represents "no-operation".
+	fn try_drop(self) -> Result<(), Self>;
+}
+
 /// A trait for a not-quite Linear Type that tracks an imbalance.
 ///
 /// Functions that alter account balances return an object of this trait to
@@ -193,14 +194,14 @@ pub enum ExistenceRequirement {
 ///
 /// You can always retrieve the raw balance value using `peek`.
 #[must_use]
-pub trait Imbalance: Sized {
+pub trait Imbalance: Sized + TryDrop {
 	/// The oppositely imbalanced type. They come in pairs.
 	type Opposite: Imbalance;
 
 	/// The zero imbalance. Can be destroyed with `drop_zero`.
 	fn zero() -> Self;
 
-	/// Drop an instance cleanly. Only works if its `value()` is zero.
+	/// Drop an instance cleanly. Only works if its `self.value()` is zero.
 	fn drop_zero(self) -> Result<(), Self>;
 
 	/// Consume `self` and return two independent instances; the first
@@ -308,7 +309,7 @@ impl<
 	Target2: OnUnbalanced,
 > OnUnbalanced for SplitTwoWays
 {
-	fn on_unbalanced(amount: I) {
+	fn on_nonzero_unbalanced(amount: I) {
 		let total: u32 = Part1::VALUE + Part2::VALUE;
 		let amount1 = amount.peek().saturating_mul(Part1::VALUE.into()) / total.into();
 		let (imb1, imb2) = amount.split(amount1);
@@ -395,6 +396,7 @@ pub trait Currency {
 		source: &AccountId,
 		dest: &AccountId,
 		value: Self::Balance,
+		existence_requirement: ExistenceRequirement,
 	) -> result::Result<(), &'static str>;
 
 	/// Deducts up to `value` from the combined balance of `who`, preferring to deduct from the
@@ -629,7 +631,7 @@ impl WithdrawReasons {
 	/// Choose all variants except for `one`.
 	///
 	/// ```rust
-	/// # use srml_support::traits::{WithdrawReason, WithdrawReasons};
+	/// # use paint_support::traits::{WithdrawReason, WithdrawReasons};
 	/// # fn main() {
 	/// assert_eq!(
 	/// 	WithdrawReason::Fee | WithdrawReason::Transfer | WithdrawReason::Reserve | WithdrawReason::Tip,
diff --git a/srml/support/src/unsigned.rs b/paint/support/src/unsigned.rs
similarity index 81%
rename from srml/support/src/unsigned.rs
rename to paint/support/src/unsigned.rs
index 4d2ceddd79f4a..690ea4947129e 100644
--- a/srml/support/src/unsigned.rs
+++ b/paint/support/src/unsigned.rs
@@ -15,6 +15,7 @@
 // along with Substrate.  If not, see .
 
 #[doc(hidden)]
+#[allow(deprecated)]
 pub use crate::sr_primitives::traits::ValidateUnsigned;
 #[doc(hidden)]
 pub use crate::sr_primitives::transaction_validity::{TransactionValidity, UnknownTransaction};
@@ -31,10 +32,10 @@ pub use crate::sr_primitives::ApplyError;
 /// # mod timestamp {
 /// # 	pub struct Module;
 /// #
-/// # 	impl srml_support::unsigned::ValidateUnsigned for Module {
+/// # 	impl paint_support::unsigned::ValidateUnsigned for Module {
 /// # 		type Call = Call;
 /// #
-/// # 		fn validate_unsigned(call: &Self::Call) -> srml_support::unsigned::TransactionValidity {
+/// # 		fn validate_unsigned(call: &Self::Call) -> paint_support::unsigned::TransactionValidity {
 /// # 			unimplemented!();
 /// # 		}
 /// # 	}
@@ -52,7 +53,7 @@ pub use crate::sr_primitives::ApplyError;
 /// # #[allow(unused)]
 /// pub struct Runtime;
 ///
-/// srml_support::impl_outer_validate_unsigned! {
+/// paint_support::impl_outer_validate_unsigned! {
 /// 	impl ValidateUnsigned for Runtime {
 /// 		Timestamp
 /// 	}
@@ -65,9 +66,20 @@ macro_rules! impl_outer_validate_unsigned {
 			$( $module:ident )*
 		}
 	) => {
+		#[allow(deprecated)] // Allow ValidateUnsigned
 		impl $crate::unsigned::ValidateUnsigned for $runtime {
 			type Call = Call;
 
+			fn pre_dispatch(call: &Self::Call) -> Result<(), $crate::unsigned::ApplyError> {
+				#[allow(unreachable_patterns)]
+				match call {
+					$( Call::$module(inner_call) => $module::pre_dispatch(inner_call), )*
+					// pre-dispatch should not stop inherent extrinsics, validation should prevent
+					// including arbitrary (non-inherent) extrinsics to blocks.
+					_ => Ok(()),
+				}
+			}
+
 			fn validate_unsigned(call: &Self::Call) -> $crate::unsigned::TransactionValidity {
 				#[allow(unreachable_patterns)]
 				match call {
@@ -97,6 +109,7 @@ mod test_partial_and_full_call {
 	pub mod timestamp {
 		pub struct Module;
 
+		#[allow(deprecated)] // Allow ValidateUnsigned
 		impl super::super::ValidateUnsigned for Module {
 			type Call = Call;
 
diff --git a/srml/support/test/Cargo.toml b/paint/support/test/Cargo.toml
similarity index 53%
rename from srml/support/test/Cargo.toml
rename to paint/support/test/Cargo.toml
index a5e375b5c6b93..19ed73566090e 100644
--- a/srml/support/test/Cargo.toml
+++ b/paint/support/test/Cargo.toml
@@ -1,5 +1,5 @@
 [package]
-name = "srml-support-test"
+name = "paint-support-test"
 version = "2.0.0"
 authors = ["Parity Technologies "]
 edition = "2018"
@@ -7,11 +7,12 @@ edition = "2018"
 [dependencies]
 serde = { version = "1.0.101", default-features = false, features = ["derive"] }
 codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false, features = ["derive"] }
-runtime-io ={ package = "sr-io", path = "../../../core/sr-io", default-features = false }
-support = { package = "srml-support", version = "2", path = "../", default-features = false }
-inherents = { package = "substrate-inherents", path = "../../../core/inherents", default-features = false }
-sr-primitives = { package = "sr-primitives", path = "../../../core/sr-primitives", default-features = false }
-primitives = { package = "substrate-primitives", path = "../../../core/primitives", default-features = false }
+runtime-io ={ package = "sr-io", path = "../../../primitives/sr-io", default-features = false }
+state-machine ={ package = "substrate-state-machine", path = "../../../primitives/state-machine", optional = true }
+support = { package = "paint-support", version = "2", path = "../", default-features = false }
+inherents = { package = "substrate-inherents", path = "../../../primitives/inherents", default-features = false }
+sr-primitives = { package = "sr-primitives", path = "../../../primitives/sr-primitives", default-features = false }
+primitives = { package = "substrate-primitives", path = "../../../primitives/core", default-features = false }
 trybuild = "1.0.17"
 pretty_assertions = "0.6.1"
 
@@ -25,4 +26,5 @@ std = [
 	"inherents/std",
 	"primitives/std",
 	"sr-primitives/std",
+	"state-machine",
 ]
diff --git a/srml/support/test/src/lib.rs b/paint/support/test/src/lib.rs
similarity index 90%
rename from srml/support/test/src/lib.rs
rename to paint/support/test/src/lib.rs
index b65bc43cbb5a8..fa5c8cf1ea31d 100644
--- a/srml/support/test/src/lib.rs
+++ b/paint/support/test/src/lib.rs
@@ -14,5 +14,5 @@
 // You should have received a copy of the GNU General Public License
 // along with Substrate.  If not, see .
 
-//! Test crate for srml_support. Allow to make use of `support::decl_storage`.
+//! Test crate for paint_support. Allow to make use of `support::decl_storage`.
 //! See tests directory.
diff --git a/srml/support/src/storage/storage_items.rs b/paint/support/test/tests/decl_storage.rs
similarity index 67%
rename from srml/support/src/storage/storage_items.rs
rename to paint/support/test/tests/decl_storage.rs
index 1edf9c03db61a..396288d3be31d 100644
--- a/srml/support/src/storage/storage_items.rs
+++ b/paint/support/test/tests/decl_storage.rs
@@ -1,4 +1,4 @@
-// Copyright 2017-2019 Parity Technologies (UK) Ltd.
+// Copyright 2019 Parity Technologies (UK) Ltd.
 // This file is part of Substrate.
 
 // Substrate is free software: you can redistribute it and/or modify
@@ -14,276 +14,16 @@
 // You should have received a copy of the GNU General Public License
 // along with Substrate.  If not, see .
 
-//! Strongly typed wrappers around values in storage.
-//!
-//! This crate exports a macro `storage_items!` and traits describing behavior of generated
-//! structs.
-//!
-//! Two kinds of data types are currently supported:
-//!   - values
-//!   - maps
-//!
-//! # Examples:
-//!
-//! ```rust
-//! #[macro_use]
-//! extern crate srml_support;
-//!
-//! type AuthorityId = [u8; 32];
-//! type Balance = u64;
-//! pub type SessionKey = [u8; 32];
-//!
-//! storage_items! {
-//!     // public value
-//!     pub Value: b"putd_key" => SessionKey;
-//!     // private map.
-//!     Balances: b"private_map:" => map [AuthorityId => Balance];
-//! }
-//!
-//!# fn main() { }
-//! ```
-
-#[doc(hidden)]
-pub use crate::rstd::borrow::Borrow;
-#[doc(hidden)]
-pub use crate::rstd::marker::PhantomData;
-#[doc(hidden)]
-pub use crate::rstd::boxed::Box;
-
-#[doc(hidden)]
-pub fn id(t: T) -> T {
-	t
-}
-
-#[doc(hidden)]
-pub use Some;
-
-#[doc(hidden)]
-pub fn unwrap_or_default(t: Option) -> T {
-	t.unwrap_or_else(|| Default::default())
-}
-
-#[doc(hidden)]
-pub fn require(t: Option) -> T {
-	t.expect("Required values must be in storage")
-}
-
-// FIXME #1466 Remove this in favor of `decl_storage` macro.
-/// Declares strongly-typed wrappers around codec-compatible types in storage.
-#[macro_export]
-macro_rules! storage_items {
-	// simple values
-	($name:ident : $key:expr => $ty:ty; $($t:tt)*) => {
-		$crate::__storage_items_internal!(() () (OPTION_TYPE Option<$ty>) (id) (id) $name: $key => $ty);
-		storage_items!($($t)*);
-	};
-	(pub $name:ident : $key:expr => $ty:ty; $($t:tt)*) => {
-		$crate::__storage_items_internal!((pub) () (OPTION_TYPE Option<$ty>) (id) (id) $name: $key => $ty);
-		storage_items!($($t)*);
-	};
-	($name:ident : $key:expr => default $ty:ty; $($t:tt)*) => {
-		$crate::__storage_items_internal!(() () (RAW_TYPE $ty) (unwrap_or_default) (Some) $name: $key => $ty);
-		storage_items!($($t)*);
-	};
-	(pub $name:ident : $key:expr => default $ty:ty; $($t:tt)*) => {
-		$crate::__storage_items_internal!((pub) () (RAW_TYPE $ty) (unwrap_or_default) (Some) $name: $key => $ty);
-		storage_items!($($t)*);
-	};
-	($name:ident : $key:expr => required $ty:ty; $($t:tt)*) => {
-		$crate::__storage_items_internal!(() () (RAW_TYPE $ty) (require) (Some) $name: $key => $ty);
-		storage_items!($($t)*);
-	};
-	(pub $name:ident : $key:expr => required $ty:ty; $($t:tt)*) => {
-		$crate::__storage_items_internal!((pub) () (RAW_TYPE $ty) (require) (Some) $name: $key => $ty);
-		storage_items!($($t)*);
-	};
-
-	($name:ident get($getfn:ident) : $key:expr => $ty:ty; $($t:tt)*) => {
-		$crate::__storage_items_internal!(() ($getfn) (OPTION_TYPE Option<$ty>) (id) (id) $name: $key => $ty);
-		storage_items!($($t)*);
-	};
-	(pub $name:ident get($getfn:ident) : $key:expr => $ty:ty; $($t:tt)*) => {
-		$crate::__storage_items_internal!((pub) ($getfn) (OPTION_TYPE Option<$ty>) (id) (id) $name: $key => $ty);
-		storage_items!($($t)*);
-	};
-	($name:ident get($getfn:ident) : $key:expr => default $ty:ty; $($t:tt)*) => {
-		$crate::__storage_items_internal!(() ($getfn) (RAW_TYPE $ty) (unwrap_or_default) (Some) $name: $key => $ty);
-		storage_items!($($t)*);
-	};
-	(pub $name:ident get($getfn:ident) : $key:expr => default $ty:ty; $($t:tt)*) => {
-		$crate::__storage_items_internal!((pub) ($getfn) (RAW_TYPE $ty) (unwrap_or_default) (Some) $name: $key => $ty);
-		storage_items!($($t)*);
-	};
-	($name:ident get($getfn:ident) : $key:expr => required $ty:ty; $($t:tt)*) => {
-		$crate::__storage_items_internal!(() ($getfn) (RAW_TYPE $ty) (require) (Some) $name: $key => $ty);
-		storage_items!($($t)*);
-	};
-	(pub $name:ident get($getfn:ident) : $key:expr => required $ty:ty; $($t:tt)*) => {
-		$crate::__storage_items_internal!((pub) ($getfn) (RAW_TYPE $ty) (require) (Some) $name: $key => $ty);
-		storage_items!($($t)*);
-	};
-
-	// maps
-	($name:ident : $prefix:expr => map [$kty:ty => $ty:ty]; $($t:tt)*) => {
-		$crate::__storage_items_internal!(() () (OPTION_TYPE Option<$ty>) (id) (id) $name: $prefix => map [$kty => $ty]);
-		storage_items!($($t)*);
-	};
-	(pub $name:ident : $prefix:expr => map [$kty:ty => $ty:ty]; $($t:tt)*) => {
-		$crate::__storage_items_internal!((pub) () (OPTION_TYPE Option<$ty>) (id) (id) $name: $prefix => map [$kty => $ty]);
-		storage_items!($($t)*);
-	};
-	($name:ident : $prefix:expr => default map [$kty:ty => $ty:ty]; $($t:tt)*) => {
-		$crate::__storage_items_internal!(() () (RAW_TYPE $ty) (unwrap_or_default) (Some) $name: $prefix => map [$kty => $ty]);
-		storage_items!($($t)*);
-	};
-	(pub $name:ident : $prefix:expr => default map [$kty:ty => $ty:ty]; $($t:tt)*) => {
-		$crate::__storage_items_internal!((pub) () (RAW_TYPE $ty) (unwrap_or_default) (Some) $name: $prefix => map [$kty => $ty]);
-		storage_items!($($t)*);
-	};
-	($name:ident : $prefix:expr => required map [$kty:ty => $ty:ty]; $($t:tt)*) => {
-		$crate::__storage_items_internal!(() () (RAW_TYPE $ty) (require) (Some) $name: $prefix => map [$kty => $ty]);
-		storage_items!($($t)*);
-	};
-	(pub $name:ident : $prefix:expr => required map [$kty:ty => $ty:ty]; $($t:tt)*) => {
-		$crate::__storage_items_internal!((pub) () (RAW_TYPE $ty) (require) (Some) $name: $prefix => map [$kty => $ty]);
-		storage_items!($($t)*);
-	};
-
-	($name:ident get($getfn:ident) : $prefix:expr => map [$kty:ty => $ty:ty]; $($t:tt)*) => {
-		$crate::__storage_items_internal!(() ($getfn) (OPTION_TYPE Option<$ty>) (id) (id) $name: $prefix => map [$kty => $ty]);
-		storage_items!($($t)*);
-	};
-	(pub $name:ident get($getfn:ident) : $prefix:expr => map [$kty:ty => $ty:ty]; $($t:tt)*) => {
-		$crate::__storage_items_internal!((pub) ($getfn) (OPTION_TYPE Option<$ty>) (id) (id) $name: $prefix => map [$kty => $ty]);
-		storage_items!($($t)*);
-	};
-	($name:ident get($getfn:ident) : $prefix:expr => default map [$kty:ty => $ty:ty]; $($t:tt)*) => {
-		$crate::__storage_items_internal!(() ($getfn) (RAW_TYPE $ty) (unwrap_or_default) (Some) $name: $prefix => map [$kty => $ty]);
-		storage_items!($($t)*);
-	};
-	(pub $name:ident get($getfn:ident) : $prefix:expr => default map [$kty:ty => $ty:ty]; $($t:tt)*) => {
-		$crate::__storage_items_internal!((pub) ($getfn) (RAW_TYPE $ty) (unwrap_or_default) (Some) $name: $prefix => map [$kty => $ty]);
-		storage_items!($($t)*);
-	};
-	($name:ident get($getfn:ident) : $prefix:expr => required map [$kty:ty => $ty:ty]; $($t:tt)*) => {
-		$crate::__storage_items_internal!(() ($getfn) (RAW_TYPE $ty) (require) (Some) $name: $prefix => map [$kty => $ty]);
-		storage_items!($($t)*);
-	};
-	(pub $name:ident get($getfn:ident) : $prefix:expr => required map [$kty:ty => $ty:ty]; $($t:tt)*) => {
-		$crate::__storage_items_internal!((pub) ($getfn) (RAW_TYPE $ty) (require) (Some) $name: $prefix => map [$kty => $ty]);
-		storage_items!($($t)*);
-	};
-
-	() => ()
-}
-
-#[macro_export]
-#[doc(hidden)]
-macro_rules! __storage_items_internal {
-	// generator for values.
-	(($($vis:tt)*) ($get_fn:ident) ($wraptype:ident $gettype:ty) ($into_query:ident) ($into_opt_val:ident) $name:ident : $key:expr => $ty:ty) => {
-		$crate::__storage_items_internal!{ ($($vis)*) () ($wraptype $gettype) ($into_query) ($into_opt_val) $name : $key => $ty }
-		pub fn $get_fn() -> $gettype { <$name as $crate::storage::StorageValue<$ty>> :: get() }
-	};
-	(($($vis:tt)*) () ($wraptype:ident $gettype:ty) ($into_query:ident) ($into_opt_val:ident) $name:ident : $key:expr => $ty:ty) => {
-		$($vis)* struct $name;
-
-		impl $crate::storage::generator::StorageValue<$ty> for $name {
-			type Query = $gettype;
-
-			fn unhashed_key() -> &'static [u8] {
-				$key
-			}
-
-			fn from_optional_value_to_query(v: Option<$ty>) -> Self::Query {
-				$crate::storage::storage_items::$into_query(v)
-			}
-
-			fn from_query_to_optional_value(v: Self::Query) -> Option<$ty> {
-				$crate::storage::storage_items::$into_opt_val(v)
-			}
-		}
-	};
-	// generator for maps.
-	(($($vis:tt)*) ($get_fn:ident) ($wraptype:ident $gettype:ty) ($into_query:ident) ($into_opt_val:ident) $name:ident : $prefix:expr => map [$kty:ty => $ty:ty]) => {
-		$crate::__storage_items_internal!{ ($($vis)*) () ($wraptype $gettype) ($into_query) ($into_opt_val) $name : $prefix => map [$kty => $ty] }
-		pub fn $get_fn>(key: K) -> $gettype {
-			<$name as $crate::storage::StorageMap<$kty, $ty>> :: get(key.borrow())
-		}
-	};
-	(($($vis:tt)*) () ($wraptype:ident $gettype:ty) ($into_query:ident) ($into_opt_val:ident) $name:ident : $prefix:expr => map [$kty:ty => $ty:ty]) => {
-		$($vis)* struct $name;
-
-		impl $crate::storage::generator::StorageMap<$kty, $ty> for $name {
-			type Query = $gettype;
-			type Hasher = $crate::Blake2_256;
-
-			fn prefix() -> &'static [u8] {
-				$prefix
-			}
-
-			fn from_optional_value_to_query(v: Option<$ty>) -> Self::Query {
-				$crate::storage::storage_items::$into_query(v)
-			}
-
-			fn from_query_to_optional_value(v: Self::Query) -> Option<$ty> {
-				$crate::storage::storage_items::$into_opt_val(v)
-			}
-		}
-	};
-}
-
-#[macro_export]
-#[doc(hidden)]
-macro_rules! __handle_wrap_internal {
-	(RAW_TYPE { $($raw:tt)* } { $($option:tt)* }) => {
-		$($raw)*;
-	};
-	(OPTION_TYPE { $($raw:tt)* } { $($option:tt)* }) => {
-		$($option)*;
-	};
-}
-
-// FIXME: revisit this idiom once we get `type`s in `impl`s.
-/*impl Module {
-	type Now = super::Now;
-}*/
-
 #[cfg(test)]
 // Do not complain about unused `dispatch` and `dispatch_aux`.
 #[allow(dead_code)]
 mod tests {
-	use crate::metadata::*;
-	use crate::metadata::StorageHasher;
-	use crate::rstd::marker::PhantomData;
-	use crate::codec::{Encode, Decode, EncodeLike};
-
-	storage_items! {
-		Value: b"a" => u32;
-		Map: b"c:" => map [u32 => [u8; 32]];
-	}
+	use support::metadata::*;
+	use std::marker::PhantomData;
+	use codec::{Encode, Decode, EncodeLike};
 
-	#[test]
-	fn value() {
-		runtime_io::with_storage(&mut Default::default(), || {
-			assert!(Value::get().is_none());
-			Value::put(&100_000);
-			assert_eq!(Value::get(), Some(100_000));
-			Value::kill();
-			assert!(Value::get().is_none());
-		})
-	}
-
-	#[test]
-	fn map() {
-		runtime_io::with_storage(&mut Default::default(), || {
-			assert!(Map::get(&5).is_none());
-			Map::insert(&5, &[1; 32]);
-			assert_eq!(Map::get(&5), Some([1; 32]));
-			assert_eq!(Map::take(&5), Some([1; 32]));
-			assert!(Map::get(&5).is_none());
-			assert!(Map::get(&999).is_none());
-		})
+	support::decl_module! {
+		pub struct Module for enum Call where origin: T::Origin {}
 	}
 
 	pub trait Trait {
@@ -291,11 +31,7 @@ mod tests {
 		type BlockNumber;
 	}
 
-	decl_module! {
-		pub struct Module for enum Call where origin: T::Origin {}
-	}
-
-	crate::decl_storage! {
+	support::decl_storage! {
 		trait Store for Module as TestStorage {
 			// non-getters: pub / $default
 
@@ -701,13 +437,13 @@ mod test2 {
 		type BlockNumber;
 	}
 
-	decl_module! {
+	support::decl_module! {
 		pub struct Module for enum Call where origin: T::Origin {}
 	}
 
 	type PairOf = (T, T);
 
-	crate::decl_storage! {
+	support::decl_storage! {
 		trait Store for Module as TestStorage {
 			SingleDef : u32;
 			PairDef : PairOf;
@@ -736,10 +472,10 @@ mod test3 {
 		type Origin;
 		type BlockNumber;
 	}
-	decl_module! {
+	support::decl_module! {
 		pub struct Module for enum Call where origin: T::Origin {}
 	}
-	crate::decl_storage! {
+	support::decl_storage! {
 		trait Store for Module as Test {
 			Foo get(fn foo) config(initial_foo): u32;
 		}
@@ -766,14 +502,14 @@ mod test_append_and_len {
 		type BlockNumber;
 	}
 
-	decl_module! {
+	support::decl_module! {
 		pub struct Module for enum Call where origin: T::Origin {}
 	}
 
 	#[derive(PartialEq, Eq, Clone, Encode, Decode)]
 	struct NoDef(u32);
 
-	crate::decl_storage! {
+	support::decl_storage! {
 		trait Store for Module as Test {
 			NoDefault: Option;
 
diff --git a/srml/support/test/tests/decl_storage_ui.rs b/paint/support/test/tests/decl_storage_ui.rs
similarity index 100%
rename from srml/support/test/tests/decl_storage_ui.rs
rename to paint/support/test/tests/decl_storage_ui.rs
diff --git a/srml/support/test/tests/decl_storage_ui/config_duplicate.rs b/paint/support/test/tests/decl_storage_ui/config_duplicate.rs
similarity index 100%
rename from srml/support/test/tests/decl_storage_ui/config_duplicate.rs
rename to paint/support/test/tests/decl_storage_ui/config_duplicate.rs
diff --git a/srml/support/test/tests/decl_storage_ui/config_duplicate.stderr b/paint/support/test/tests/decl_storage_ui/config_duplicate.stderr
similarity index 100%
rename from srml/support/test/tests/decl_storage_ui/config_duplicate.stderr
rename to paint/support/test/tests/decl_storage_ui/config_duplicate.stderr
diff --git a/srml/support/test/tests/decl_storage_ui/config_get_duplicate.rs b/paint/support/test/tests/decl_storage_ui/config_get_duplicate.rs
similarity index 100%
rename from srml/support/test/tests/decl_storage_ui/config_get_duplicate.rs
rename to paint/support/test/tests/decl_storage_ui/config_get_duplicate.rs
diff --git a/srml/support/test/tests/decl_storage_ui/config_get_duplicate.stderr b/paint/support/test/tests/decl_storage_ui/config_get_duplicate.stderr
similarity index 100%
rename from srml/support/test/tests/decl_storage_ui/config_get_duplicate.stderr
rename to paint/support/test/tests/decl_storage_ui/config_get_duplicate.stderr
diff --git a/srml/support/test/tests/decl_storage_ui/get_duplicate.rs b/paint/support/test/tests/decl_storage_ui/get_duplicate.rs
similarity index 100%
rename from srml/support/test/tests/decl_storage_ui/get_duplicate.rs
rename to paint/support/test/tests/decl_storage_ui/get_duplicate.rs
diff --git a/srml/support/test/tests/decl_storage_ui/get_duplicate.stderr b/paint/support/test/tests/decl_storage_ui/get_duplicate.stderr
similarity index 100%
rename from srml/support/test/tests/decl_storage_ui/get_duplicate.stderr
rename to paint/support/test/tests/decl_storage_ui/get_duplicate.stderr
diff --git a/srml/support/test/tests/final_keys.rs b/paint/support/test/tests/final_keys.rs
similarity index 70%
rename from srml/support/test/tests/final_keys.rs
rename to paint/support/test/tests/final_keys.rs
index 44a6b540a7a0e..c1cf5c651a545 100644
--- a/srml/support/test/tests/final_keys.rs
+++ b/paint/support/test/tests/final_keys.rs
@@ -14,10 +14,10 @@
 // You should have received a copy of the GNU General Public License
 // along with Substrate.  If not, see .
 
-use runtime_io::with_storage;
 use support::storage::unhashed;
 use codec::Encode;
 use support::{StorageDoubleMap, StorageLinkedMap, StorageMap, StorageValue};
+use runtime_io::{TestExternalities, hashing};
 
 mod no_instance {
 	use codec::{Encode, Decode, EncodeLike};
@@ -87,141 +87,141 @@ mod instance {
 
 #[test]
 fn final_keys_no_instance() {
-	with_storage(&mut Default::default(), || {
+	TestExternalities::default().execute_with(|| {
 		no_instance::Value::put(1);
-		assert_eq!(unhashed::get::(&runtime_io::twox_128(b"FinalKeysNone Value")), Some(1u32));
+		assert_eq!(unhashed::get::(&hashing::twox_128(b"FinalKeysNone Value")), Some(1u32));
 
 		no_instance::Map::insert(1, 2);
 		let mut k = b"FinalKeysNone Map".to_vec();
 		k.extend(1u32.encode());
-		assert_eq!(unhashed::get::(&runtime_io::blake2_256(&k)), Some(2u32));
+		assert_eq!(unhashed::get::(&hashing::blake2_256(&k)), Some(2u32));
 
 		no_instance::Map2::insert(1, 2);
 		let mut k = b"FinalKeysNone Map2".to_vec();
 		k.extend(1u32.encode());
-		assert_eq!(unhashed::get::(&runtime_io::twox_128(&k)), Some(2u32));
+		assert_eq!(unhashed::get::(&hashing::twox_128(&k)), Some(2u32));
 
 		let head = b"head of FinalKeysNone LinkedMap".to_vec();
-		assert_eq!(unhashed::get::(&runtime_io::blake2_256(&head)), None);
+		assert_eq!(unhashed::get::(&hashing::blake2_256(&head)), None);
 
 		no_instance::LinkedMap::insert(1, 2);
 		let mut k = b"FinalKeysNone LinkedMap".to_vec();
 		k.extend(1u32.encode());
-		assert_eq!(unhashed::get::(&runtime_io::blake2_256(&k)), Some(2u32));
-		assert_eq!(unhashed::get::(&runtime_io::blake2_256(&head)), Some(1u32));
+		assert_eq!(unhashed::get::(&hashing::blake2_256(&k)), Some(2u32));
+		assert_eq!(unhashed::get::(&hashing::blake2_256(&head)), Some(1u32));
 
 		no_instance::LinkedMap2::insert(1, 2);
 		let mut k = b"FinalKeysNone LinkedMap2".to_vec();
 		k.extend(1u32.encode());
-		assert_eq!(unhashed::get::(&runtime_io::twox_128(&k)), Some(2u32));
+		assert_eq!(unhashed::get::(&hashing::twox_128(&k)), Some(2u32));
 
 		no_instance::DoubleMap::insert(&1, &2, &3);
 		let mut k = b"FinalKeysNone DoubleMap".to_vec();
 		k.extend(1u32.encode());
-		let mut k = runtime_io::blake2_256(&k).to_vec();
-		k.extend(&runtime_io::blake2_256(&2u32.encode()));
+		let mut k = hashing::blake2_256(&k).to_vec();
+		k.extend(&hashing::blake2_256(&2u32.encode()));
 		assert_eq!(unhashed::get::(&k), Some(3u32));
 
 		no_instance::DoubleMap2::insert(&1, &2, &3);
 		let mut k = b"FinalKeysNone DoubleMap2".to_vec();
 		k.extend(1u32.encode());
-		let mut k = runtime_io::twox_128(&k).to_vec();
-		k.extend(&runtime_io::blake2_128(&2u32.encode()));
+		let mut k = hashing::twox_128(&k).to_vec();
+		k.extend(&hashing::blake2_128(&2u32.encode()));
 		assert_eq!(unhashed::get::(&k), Some(3u32));
 	});
 }
 
 #[test]
 fn final_keys_default_instance() {
-	with_storage(&mut Default::default(), || {
+	TestExternalities::default().execute_with(|| {
 		>::put(1);
-		assert_eq!(unhashed::get::(&runtime_io::twox_128(b"FinalKeysSome Value")), Some(1u32));
+		assert_eq!(unhashed::get::(&hashing::twox_128(b"FinalKeysSome Value")), Some(1u32));
 
 		>::insert(1, 2);
 		let mut k = b"FinalKeysSome Map".to_vec();
 		k.extend(1u32.encode());
-		assert_eq!(unhashed::get::(&runtime_io::blake2_256(&k)), Some(2u32));
+		assert_eq!(unhashed::get::(&hashing::blake2_256(&k)), Some(2u32));
 
 		>::insert(1, 2);
 		let mut k = b"FinalKeysSome Map2".to_vec();
 		k.extend(1u32.encode());
-		assert_eq!(unhashed::get::(&runtime_io::twox_128(&k)), Some(2u32));
+		assert_eq!(unhashed::get::(&hashing::twox_128(&k)), Some(2u32));
 
 		let head = b"head of FinalKeysSome LinkedMap".to_vec();
-		assert_eq!(unhashed::get::(&runtime_io::blake2_256(&head)), None);
+		assert_eq!(unhashed::get::(&hashing::blake2_256(&head)), None);
 
 		>::insert(1, 2);
 		let mut k = b"FinalKeysSome LinkedMap".to_vec();
 		k.extend(1u32.encode());
-		assert_eq!(unhashed::get::(&runtime_io::blake2_256(&k)), Some(2u32));
-		assert_eq!(unhashed::get::(&runtime_io::blake2_256(&head)), Some(1u32));
+		assert_eq!(unhashed::get::(&hashing::blake2_256(&k)), Some(2u32));
+		assert_eq!(unhashed::get::(&hashing::blake2_256(&head)), Some(1u32));
 
-<		instance::LinkedMap2>::insert(1, 2);
+		>::insert(1, 2);
 		let mut k = b"FinalKeysSome LinkedMap2".to_vec();
 		k.extend(1u32.encode());
-		assert_eq!(unhashed::get::(&runtime_io::twox_128(&k)), Some(2u32));
+		assert_eq!(unhashed::get::(&hashing::twox_128(&k)), Some(2u32));
 
 		>::insert(&1, &2, &3);
 		let mut k = b"FinalKeysSome DoubleMap".to_vec();
 		k.extend(1u32.encode());
-		let mut k = runtime_io::blake2_256(&k).to_vec();
-		k.extend(&runtime_io::blake2_256(&2u32.encode()));
+		let mut k = hashing::blake2_256(&k).to_vec();
+		k.extend(&hashing::blake2_256(&2u32.encode()));
 		assert_eq!(unhashed::get::(&k), Some(3u32));
 
 		>::insert(&1, &2, &3);
 		let mut k = b"FinalKeysSome DoubleMap2".to_vec();
 		k.extend(1u32.encode());
-		let mut k = runtime_io::twox_128(&k).to_vec();
-		k.extend(&runtime_io::blake2_128(&2u32.encode()));
+		let mut k = hashing::twox_128(&k).to_vec();
+		k.extend(&hashing::blake2_128(&2u32.encode()));
 		assert_eq!(unhashed::get::(&k), Some(3u32));
 	});
 }
 
 #[test]
 fn final_keys_instance_2() {
-	with_storage(&mut Default::default(), || {
+	TestExternalities::default().execute_with(|| {
 		>::put(1);
 		assert_eq!(
-			unhashed::get::(&runtime_io::twox_128(b"Instance2FinalKeysSome Value")),
+			unhashed::get::(&hashing::twox_128(b"Instance2FinalKeysSome Value")),
 			Some(1u32)
 		);
 
 		>::insert(1, 2);
 		let mut k = b"Instance2FinalKeysSome Map".to_vec();
 		k.extend(1u32.encode());
-		assert_eq!(unhashed::get::(&runtime_io::blake2_256(&k)), Some(2u32));
+		assert_eq!(unhashed::get::(&hashing::blake2_256(&k)), Some(2u32));
 
 		>::insert(1, 2);
 		let mut k = b"Instance2FinalKeysSome Map2".to_vec();
 		k.extend(1u32.encode());
-		assert_eq!(unhashed::get::(&runtime_io::twox_128(&k)), Some(2u32));
+		assert_eq!(unhashed::get::(&hashing::twox_128(&k)), Some(2u32));
 
 		let head = b"head of Instance2FinalKeysSome LinkedMap".to_vec();
-		assert_eq!(unhashed::get::(&runtime_io::blake2_256(&head)), None);
+		assert_eq!(unhashed::get::(&hashing::blake2_256(&head)), None);
 
 		>::insert(1, 2);
 		let mut k = b"Instance2FinalKeysSome LinkedMap".to_vec();
 		k.extend(1u32.encode());
-		assert_eq!(unhashed::get::(&runtime_io::blake2_256(&k)), Some(2u32));
-		assert_eq!(unhashed::get::(&runtime_io::blake2_256(&head)), Some(1u32));
+		assert_eq!(unhashed::get::(&hashing::blake2_256(&k)), Some(2u32));
+		assert_eq!(unhashed::get::(&hashing::blake2_256(&head)), Some(1u32));
 
 		>::insert(1, 2);
 		let mut k = b"Instance2FinalKeysSome LinkedMap2".to_vec();
 		k.extend(1u32.encode());
-		assert_eq!(unhashed::get::(&runtime_io::twox_128(&k)), Some(2u32));
+		assert_eq!(unhashed::get::(&hashing::twox_128(&k)), Some(2u32));
 
 		>::insert(&1, &2, &3);
 		let mut k = b"Instance2FinalKeysSome DoubleMap".to_vec();
 		k.extend(1u32.encode());
-		let mut k = runtime_io::blake2_256(&k).to_vec();
-		k.extend(&runtime_io::blake2_256(&2u32.encode()));
+		let mut k = hashing::blake2_256(&k).to_vec();
+		k.extend(&hashing::blake2_256(&2u32.encode()));
 		assert_eq!(unhashed::get::(&k), Some(3u32));
 
 		>::insert(&1, &2, &3);
 		let mut k = b"Instance2FinalKeysSome DoubleMap2".to_vec();
 		k.extend(1u32.encode());
-		let mut k = runtime_io::twox_128(&k).to_vec();
-		k.extend(&runtime_io::blake2_128(&2u32.encode()));
+		let mut k = hashing::twox_128(&k).to_vec();
+		k.extend(&hashing::blake2_128(&2u32.encode()));
 		assert_eq!(unhashed::get::(&k), Some(3u32));
 	});
 }
diff --git a/srml/support/test/tests/genesisconfig.rs b/paint/support/test/tests/genesisconfig.rs
similarity index 100%
rename from srml/support/test/tests/genesisconfig.rs
rename to paint/support/test/tests/genesisconfig.rs
diff --git a/srml/support/test/tests/instance.rs b/paint/support/test/tests/instance.rs
similarity index 98%
rename from srml/support/test/tests/instance.rs
rename to paint/support/test/tests/instance.rs
index 282fb9a6e299f..dc4fded17edc7 100644
--- a/srml/support/test/tests/instance.rs
+++ b/paint/support/test/tests/instance.rs
@@ -25,9 +25,7 @@ use support::{
 	},
 	StorageValue, StorageMap, StorageLinkedMap, StorageDoubleMap,
 };
-use inherents::{
-	ProvideInherent, InherentData, InherentIdentifier, RuntimeString, MakeFatalError
-};
+use inherents::{ProvideInherent, InherentData, InherentIdentifier, MakeFatalError};
 use primitives::{H256, sr25519};
 
 mod system;
@@ -100,7 +98,7 @@ mod module1 {
 		T::BlockNumber: From
 	{
 		type Call = Call;
-		type Error = MakeFatalError;
+		type Error = MakeFatalError;
 		const INHERENT_IDENTIFIER: InherentIdentifier = INHERENT_IDENTIFIER;
 
 		fn create_inherent(_data: &InherentData) -> Option {
@@ -160,7 +158,7 @@ mod module2 {
 
 	impl, I: Instance> ProvideInherent for Module {
 		type Call = Call;
-		type Error = MakeFatalError;
+		type Error = MakeFatalError;
 		const INHERENT_IDENTIFIER: InherentIdentifier = INHERENT_IDENTIFIER;
 
 		fn create_inherent(_data: &InherentData) -> Option {
@@ -303,7 +301,7 @@ fn new_test_ext() -> runtime_io::TestExternalities {
 #[test]
 fn storage_instance_independance() {
 	let mut storage = (std::collections::HashMap::new(), std::collections::HashMap::new());
-	runtime_io::with_storage(&mut storage, || {
+	state_machine::BasicExternalities::execute_with_storage(&mut storage, || {
 		module2::Value::::put(0);
 		module2::Value::::put(0);
 		module2::Value::::put(0);
diff --git a/srml/support/test/tests/issue2219.rs b/paint/support/test/tests/issue2219.rs
similarity index 100%
rename from srml/support/test/tests/issue2219.rs
rename to paint/support/test/tests/issue2219.rs
diff --git a/srml/support/test/tests/reserved_keyword.rs b/paint/support/test/tests/reserved_keyword.rs
similarity index 100%
rename from srml/support/test/tests/reserved_keyword.rs
rename to paint/support/test/tests/reserved_keyword.rs
diff --git a/srml/support/test/tests/reserved_keyword/on_initialize.rs b/paint/support/test/tests/reserved_keyword/on_initialize.rs
similarity index 100%
rename from srml/support/test/tests/reserved_keyword/on_initialize.rs
rename to paint/support/test/tests/reserved_keyword/on_initialize.rs
diff --git a/srml/support/test/tests/reserved_keyword/on_initialize.stderr b/paint/support/test/tests/reserved_keyword/on_initialize.stderr
similarity index 100%
rename from srml/support/test/tests/reserved_keyword/on_initialize.stderr
rename to paint/support/test/tests/reserved_keyword/on_initialize.stderr
diff --git a/srml/support/test/tests/system.rs b/paint/support/test/tests/system.rs
similarity index 100%
rename from srml/support/test/tests/system.rs
rename to paint/support/test/tests/system.rs
diff --git a/srml/system/Cargo.toml b/paint/system/Cargo.toml
similarity index 61%
rename from srml/system/Cargo.toml
rename to paint/system/Cargo.toml
index a9ce6b3f8c830..9fa98937b9d36 100644
--- a/srml/system/Cargo.toml
+++ b/paint/system/Cargo.toml
@@ -1,5 +1,5 @@
 [package]
-name = "srml-system"
+name = "paint-system"
 version = "2.0.0"
 authors = ["Parity Technologies "]
 edition = "2018"
@@ -8,13 +8,13 @@ edition = "2018"
 serde = { version = "1.0.101", optional = true, features = ["derive"] }
 safe-mix = { version = "1.0.0", default-features = false }
 codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false, features = ["derive"] }
-primitives = { package = "substrate-primitives",  path = "../../core/primitives", default-features = false }
-rstd = { package = "sr-std", path = "../../core/sr-std", default-features = false }
-runtime-io ={ package = "sr-io", path = "../../core/sr-io", default-features = false }
-sr-primitives = { path = "../../core/sr-primitives", default-features = false }
-sr-version = { path = "../../core/sr-version", default-features = false }
-support = { package = "srml-support", path = "../support", default-features = false }
-impl-trait-for-tuples = "0.1.2"
+primitives = { package = "substrate-primitives",  path = "../../primitives/core", default-features = false }
+rstd = { package = "sr-std", path = "../../primitives/sr-std", default-features = false }
+runtime-io ={ package = "sr-io", path = "../../primitives/sr-io", default-features = false }
+sr-primitives = { path = "../../primitives/sr-primitives", default-features = false }
+sr-version = { path = "../../primitives/sr-version", default-features = false }
+support = { package = "paint-support", path = "../support", default-features = false }
+impl-trait-for-tuples = "0.1.3"
 
 [dev-dependencies]
 criterion = "0.2.11"
diff --git a/srml/system/benches/bench.rs b/paint/system/benches/bench.rs
similarity index 99%
rename from srml/system/benches/bench.rs
rename to paint/system/benches/bench.rs
index 6da02cf9c8cea..2ff794521d4e5 100644
--- a/srml/system/benches/bench.rs
+++ b/paint/system/benches/bench.rs
@@ -15,7 +15,7 @@
 // along with Substrate.  If not, see .
 
 use criterion::{Criterion, criterion_group, criterion_main, black_box};
-use srml_system as system;
+use paint_system as system;
 use support::{decl_module, decl_event, impl_outer_origin, impl_outer_event};
 use primitives::H256;
 use sr_primitives::{Perbill, traits::{BlakeTwo256, IdentityLookup}, testing::Header};
diff --git a/srml/system/rpc/Cargo.toml b/paint/system/rpc/Cargo.toml
similarity index 52%
rename from srml/system/rpc/Cargo.toml
rename to paint/system/rpc/Cargo.toml
index 04856a817f681..6f71b9af51b14 100644
--- a/srml/system/rpc/Cargo.toml
+++ b/paint/system/rpc/Cargo.toml
@@ -1,23 +1,23 @@
 [package]
-name = "srml-system-rpc"
+name = "paint-system-rpc"
 version = "2.0.0"
 authors = ["Parity Technologies "]
 edition = "2018"
 
 [dependencies]
-client = { package = "substrate-client", path = "../../../core/client" }
+client = { package = "substrate-client", path = "../../../client/" }
 codec = { package = "parity-scale-codec", version = "1.0.0" }
-jsonrpc-core = "13.2.0"
-jsonrpc-core-client = "13.2.0"
-jsonrpc-derive = "13.2.0"
+jsonrpc-core = "14.0.3"
+jsonrpc-core-client = "14.0.3"
+jsonrpc-derive = "14.0.3"
 log = "0.4.8"
 serde = { version = "1.0.101", features = ["derive"] }
-sr-primitives = { path = "../../../core/sr-primitives" }
-srml-system-rpc-runtime-api = { path = "./runtime-api" }
-substrate-primitives = { path = "../../../core/primitives" }
-transaction_pool = { package = "substrate-transaction-pool", path = "../../../core/transaction-pool" }
+sr-primitives = { path = "../../../primitives/sr-primitives" }
+paint-system-rpc-runtime-api = { path = "./runtime-api" }
+substrate-primitives = { path = "../../../primitives/core" }
+transaction_pool = { package = "substrate-transaction-pool", path = "../../../client/transaction-pool" }
 
 [dev-dependencies]
-test-client = { package = "substrate-test-runtime-client", path = "../../../core/test-runtime/client" }
+test-client = { package = "substrate-test-runtime-client", path = "../../../test/utils/runtime/client" }
 env_logger = "0.7.0"
 futures03 = { package = "futures-preview", version = "=0.3.0-alpha.19" }
diff --git a/srml/system/rpc/runtime-api/Cargo.toml b/paint/system/rpc/runtime-api/Cargo.toml
similarity index 62%
rename from srml/system/rpc/runtime-api/Cargo.toml
rename to paint/system/rpc/runtime-api/Cargo.toml
index fc525d8fce2ad..e0e90e79658ba 100644
--- a/srml/system/rpc/runtime-api/Cargo.toml
+++ b/paint/system/rpc/runtime-api/Cargo.toml
@@ -1,16 +1,16 @@
 [package]
-name = "srml-system-rpc-runtime-api"
+name = "paint-system-rpc-runtime-api"
 version = "2.0.0"
 authors = ["Parity Technologies "]
 edition = "2018"
 
 [dependencies]
-client = { package = "substrate-client", path = "../../../../core/client", default-features = false }
+sr-api = { path = "../../../../primitives/sr-api", default-features = false }
 codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false  }
 
 [features]
 default = ["std"]
 std = [
-	"client/std",
+	"sr-api/std",
 	"codec/std",
 ]
diff --git a/srml/system/rpc/runtime-api/src/lib.rs b/paint/system/rpc/runtime-api/src/lib.rs
similarity index 97%
rename from srml/system/rpc/runtime-api/src/lib.rs
rename to paint/system/rpc/runtime-api/src/lib.rs
index 45af0241e0836..5c8c64902b0da 100644
--- a/srml/system/rpc/runtime-api/src/lib.rs
+++ b/paint/system/rpc/runtime-api/src/lib.rs
@@ -22,7 +22,7 @@
 
 #![cfg_attr(not(feature = "std"), no_std)]
 
-client::decl_runtime_apis! {
+sr_api::decl_runtime_apis! {
 	/// The API to query account nonce (aka transaction index).
 	pub trait AccountNonceApi where
 		AccountId: codec::Codec,
diff --git a/srml/system/rpc/src/lib.rs b/paint/system/rpc/src/lib.rs
similarity index 98%
rename from srml/system/rpc/src/lib.rs
rename to paint/system/rpc/src/lib.rs
index c6c0a658fba53..e89e9b7592cb8 100644
--- a/srml/system/rpc/src/lib.rs
+++ b/paint/system/rpc/src/lib.rs
@@ -29,7 +29,7 @@ use sr_primitives::{
 use substrate_primitives::hexdisplay::HexDisplay;
 use transaction_pool::txpool::{self, Pool};
 
-pub use srml_system_rpc_runtime_api::AccountNonceApi;
+pub use paint_system_rpc_runtime_api::AccountNonceApi;
 pub use self::gen_client::Client as SystemClient;
 
 /// System RPC methods.
diff --git a/srml/system/src/lib.rs b/paint/system/src/lib.rs
similarity index 96%
rename from srml/system/src/lib.rs
rename to paint/system/src/lib.rs
index e07b937751267..9a4bf2297a46f 100644
--- a/srml/system/src/lib.rs
+++ b/paint/system/src/lib.rs
@@ -69,7 +69,7 @@
 //!
 //! ```
 //! use support::{decl_module, dispatch::Result};
-//! use srml_system::{self as system, ensure_signed};
+//! use paint_system::{self as system, ensure_signed};
 //!
 //! pub trait Trait: system::Trait {}
 //!
@@ -424,11 +424,11 @@ decl_storage! {
 		build(|config: &GenesisConfig| {
 			use codec::Encode;
 
-			runtime_io::set_storage(well_known_keys::CODE, &config.code);
-			runtime_io::set_storage(well_known_keys::EXTRINSIC_INDEX, &0u32.encode());
+			runtime_io::storage::set(well_known_keys::CODE, &config.code);
+			runtime_io::storage::set(well_known_keys::EXTRINSIC_INDEX, &0u32.encode());
 
 			if let Some(ref changes_trie_config) = config.changes_trie_config {
-				runtime_io::set_storage(
+				runtime_io::storage::set(
 					well_known_keys::CHANGES_TRIE_CONFIG,
 					&changes_trie_config.encode(),
 				);
@@ -606,6 +606,29 @@ impl Module {
 		AllExtrinsicsLen::get().unwrap_or_default()
 	}
 
+	/// Inform the system module of some additional weight that should be accounted for, in the
+	/// current block.
+	///
+	/// NOTE: use with extra care; this function is made public only be used for certain modules
+	/// that need it. A runtime that does not have dynamic calls should never need this and should
+	/// stick to static weights. A typical use case for this is inner calls or smart contract calls.
+	/// Furthermore, it only makes sense to use this when it is presumably  _cheap_ to provide the
+	/// argument `weight`; In other words, if this function is to be used to account for some
+	/// unknown, user provided call's weight, it would only make sense to use it if you are sure you
+	/// can rapidly compute the weight of the inner call.
+	///
+	/// Even more dangerous is to note that this function does NOT take any action, if the new sum
+	/// of block weight is more than the block weight limit. This is what the _unchecked_.
+	///
+	/// Another potential use-case could be for the `on_initialise` and `on_finalize` hooks.
+	///
+	/// If no previous weight exists, the function initializes the weight to zero.
+	pub fn register_extra_weight_unchecked(weight: Weight) {
+		let current_weight = AllExtrinsicsWeight::get().unwrap_or_default();
+		let next_weight = current_weight.saturating_add(weight).min(T::MaximumBlockWeight::get());
+		AllExtrinsicsWeight::put(next_weight);
+	}
+
 	/// Start the execution of a particular block.
 	pub fn initialize(
 		number: &T::BlockNumber,
diff --git a/srml/system/src/offchain.rs b/paint/system/src/offchain.rs
similarity index 98%
rename from srml/system/src/offchain.rs
rename to paint/system/src/offchain.rs
index 11f7e234f7197..3d44746bfd1f4 100644
--- a/srml/system/src/offchain.rs
+++ b/paint/system/src/offchain.rs
@@ -114,7 +114,7 @@ where
 			::create_transaction::(call, public, id, expected)
 			.ok_or(())?;
 		let xt = Self::Extrinsic::new(call, Some(signature_data)).ok_or(())?;
-		runtime_io::submit_transaction(xt.encode())
+		runtime_io::offchain::submit_transaction(xt.encode())
 	}
 }
 
@@ -129,7 +129,7 @@ pub trait SubmitUnsignedTransaction {
 	/// and `Err` if transaction was rejected from the pool.
 	fn submit_unsigned(call: impl Into) -> Result<(), ()> {
 		let xt = Self::Extrinsic::new(call.into(), None).ok_or(())?;
-		runtime_io::submit_transaction(xt.encode())
+		runtime_io::offchain::submit_transaction(xt.encode())
 	}
 }
 
diff --git a/paint/timestamp/Cargo.toml b/paint/timestamp/Cargo.toml
new file mode 100644
index 0000000000000..73485f96e12bb
--- /dev/null
+++ b/paint/timestamp/Cargo.toml
@@ -0,0 +1,31 @@
+[package]
+name = "paint-timestamp"
+version = "2.0.0"
+authors = ["Parity Technologies "]
+edition = "2018"
+
+[dependencies]
+serde = { version = "1.0.101", optional = true }
+codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false, features = ["derive"] }
+rstd = { package = "sr-std", path = "../../primitives/sr-std", default-features = false }
+sr-primitives = { path = "../../primitives/sr-primitives", default-features = false }
+inherents = { package = "substrate-inherents", path = "../../primitives/inherents", default-features = false }
+support = { package = "paint-support", path = "../support", default-features = false }
+system = { package = "paint-system", path = "../system", default-features = false }
+impl-trait-for-tuples = "0.1.3"
+
+[dev-dependencies]
+runtime-io ={ package = "sr-io", path = "../../primitives/sr-io" }
+primitives = { package = "substrate-primitives",  path = "../../primitives/core" }
+
+[features]
+default = ["std"]
+std = [
+	"inherents/std",
+	"codec/std",
+	"rstd/std",
+	"sr-primitives/std",
+	"support/std",
+	"serde",
+	"system/std",
+]
diff --git a/srml/timestamp/src/lib.rs b/paint/timestamp/src/lib.rs
similarity index 94%
rename from srml/timestamp/src/lib.rs
rename to paint/timestamp/src/lib.rs
index 2ab751088e8c7..d6123e54a9b38 100644
--- a/srml/timestamp/src/lib.rs
+++ b/paint/timestamp/src/lib.rs
@@ -62,7 +62,7 @@
 //!
 //! ```
 //! use support::{decl_module, dispatch::Result};
-//! # use srml_timestamp as timestamp;
+//! # use paint_timestamp as timestamp;
 //! use system::ensure_signed;
 //!
 //! pub trait Trait: timestamp::Trait {}
@@ -81,12 +81,12 @@
 //!
 //! ### Example from the SRML
 //!
-//! The [Session module](https://github.com/paritytech/substrate/blob/master/srml/session/src/lib.rs) uses
+//! The [Session module](https://github.com/paritytech/substrate/blob/master/paint/session/src/lib.rs) uses
 //! the Timestamp module for session management.
 //!
 //! ## Related Modules
 //!
-//! * [Session](../srml_session/index.html)
+//! * [Session](../paint_session/index.html)
 
 #![cfg_attr(not(feature = "std"), no_std)]
 
@@ -98,12 +98,15 @@ use codec::Decode;
 use inherents::ProvideInherentData;
 use support::{Parameter, decl_storage, decl_module};
 use support::traits::{Time, Get};
-use sr_primitives::traits::{
-	SimpleArithmetic, Zero, SaturatedConversion, Scale
+use sr_primitives::{
+	RuntimeString,
+	traits::{
+		SimpleArithmetic, Zero, SaturatedConversion, Scale
+	}
 };
 use sr_primitives::weights::SimpleDispatchInfo;
 use system::ensure_none;
-use inherents::{RuntimeString, InherentIdentifier, ProvideInherent, IsFatalError, InherentData};
+use inherents::{InherentIdentifier, ProvideInherent, IsFatalError, InherentData};
 
 /// The identifier for the `timestamp` inherent.
 pub const INHERENT_IDENTIFIER: InherentIdentifier = *b"timstap0";
@@ -145,11 +148,11 @@ impl InherentError {
 /// Auxiliary trait to extract timestamp inherent data.
 pub trait TimestampInherentData {
 	/// Get timestamp inherent data.
-	fn timestamp_inherent_data(&self) -> Result;
+	fn timestamp_inherent_data(&self) -> Result;
 }
 
 impl TimestampInherentData for InherentData {
-	fn timestamp_inherent_data(&self) -> Result {
+	fn timestamp_inherent_data(&self) -> Result {
 		self.get_data(&INHERENT_IDENTIFIER)
 			.and_then(|r| r.ok_or_else(|| "Timestamp inherent data not found".into()))
 	}
@@ -164,7 +167,10 @@ impl ProvideInherentData for InherentDataProvider {
 		&INHERENT_IDENTIFIER
 	}
 
-	fn provide_inherent_data(&self, inherent_data: &mut InherentData) -> Result<(), RuntimeString> {
+	fn provide_inherent_data(
+		&self,
+		inherent_data: &mut InherentData,
+	) -> Result<(), inherents::Error> {
 		use std::time::SystemTime;
 
 		let now = SystemTime::now();
@@ -288,7 +294,7 @@ impl ProvideInherent for Module {
 	}
 
 	fn check_inherent(call: &Self::Call, data: &InherentData) -> result::Result<(), Self::Error> {
-		const MAX_TIMESTAMP_DRIFT: u64 = 60;
+		const MAX_TIMESTAMP_DRIFT_MILLIS: u64 = 30 * 1000;
 
 		let t: u64 = match call {
 			Call::set(ref t) => t.clone().saturated_into::(),
@@ -298,7 +304,7 @@ impl ProvideInherent for Module {
 		let data = extract_inherent_data(data).map_err(|e| InherentError::Other(e))?;
 
 		let minimum = (Self::now() + T::MinimumPeriod::get()).saturated_into::();
-		if t > data + MAX_TIMESTAMP_DRIFT {
+		if t > data + MAX_TIMESTAMP_DRIFT_MILLIS {
 			Err(InherentError::Other("Timestamp too far in future to accept".into()))
 		} else if t < minimum {
 			Err(InherentError::ValidAtTimestamp(minimum))
diff --git a/paint/transaction-payment/Cargo.toml b/paint/transaction-payment/Cargo.toml
new file mode 100644
index 0000000000000..535df1ed95660
--- /dev/null
+++ b/paint/transaction-payment/Cargo.toml
@@ -0,0 +1,29 @@
+[package]
+name = "paint-transaction-payment"
+version = "2.0.0"
+authors = ["Parity Technologies "]
+edition = "2018"
+
+[dependencies]
+codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false, features = ["derive"] }
+rstd = { package = "sr-std", path = "../../primitives/sr-std", default-features = false }
+sr-primitives = { path = "../../primitives/sr-primitives", default-features = false }
+support = { package = "paint-support", path = "../support", default-features = false }
+system = { package = "paint-system", path = "../system", default-features = false }
+transaction-payment-rpc-runtime-api = { package = "paint-transaction-payment-rpc-runtime-api", path = "./rpc/runtime-api", default-features = false }
+
+[dev-dependencies]
+runtime-io = { package = "sr-io", path = "../../primitives/sr-io" }
+primitives = { package = "substrate-primitives",  path = "../../primitives/core" }
+balances = { package = "paint-balances", path = "../balances" }
+
+[features]
+default = ["std"]
+std = [
+	"codec/std",
+	"rstd/std",
+	"sr-primitives/std",
+	"support/std",
+	"system/std",
+	"transaction-payment-rpc-runtime-api/std"
+]
diff --git a/paint/transaction-payment/rpc/Cargo.toml b/paint/transaction-payment/rpc/Cargo.toml
new file mode 100644
index 0000000000000..22d68f8a6c7ea
--- /dev/null
+++ b/paint/transaction-payment/rpc/Cargo.toml
@@ -0,0 +1,17 @@
+[package]
+name = "paint-transaction-payment-rpc"
+version = "2.0.0"
+authors = ["Parity Technologies "]
+edition = "2018"
+
+[dependencies]
+client = { package = "substrate-client", path = "../../../client/" }
+codec = { package = "parity-scale-codec", version = "1.0.0" }
+jsonrpc-core = "14.0.3"
+jsonrpc-core-client = "14.0.3"
+jsonrpc-derive = "14.0.3"
+primitives = { package = "substrate-primitives",  path = "../../../primitives/core" }
+rpc-primitives = { package = "substrate-rpc-primitives", path = "../../../primitives/rpc" }
+serde = { version = "1.0.101", features = ["derive"] }
+sr-primitives = { path = "../../../primitives/sr-primitives" }
+paint-transaction-payment-rpc-runtime-api = { path = "./runtime-api" }
diff --git a/srml/transaction-payment/rpc/runtime-api/Cargo.toml b/paint/transaction-payment/rpc/runtime-api/Cargo.toml
similarity index 53%
rename from srml/transaction-payment/rpc/runtime-api/Cargo.toml
rename to paint/transaction-payment/rpc/runtime-api/Cargo.toml
index c26f295f4bec3..05ba940d675f6 100644
--- a/srml/transaction-payment/rpc/runtime-api/Cargo.toml
+++ b/paint/transaction-payment/rpc/runtime-api/Cargo.toml
@@ -1,21 +1,21 @@
 [package]
-name = "srml-transaction-payment-rpc-runtime-api"
+name = "paint-transaction-payment-rpc-runtime-api"
 version = "2.0.0"
 authors = ["Parity Technologies "]
 edition = "2018"
 
 [dependencies]
 serde = { version = "1.0.101", optional = true, features = ["derive"] }
-client = { package = "substrate-client", path = "../../../../core/client", default-features = false }
+sr-api = { path = "../../../../primitives/sr-api", default-features = false }
 codec = { package = "parity-scale-codec", version = "1.0.6", default-features = false, features = ["derive"] }
-rstd = { package = "sr-std", path = "../../../../core/sr-std", default-features = false }
-sr-primitives = { path = "../../../../core/sr-primitives", default-features = false }
+rstd = { package = "sr-std", path = "../../../../primitives/sr-std", default-features = false }
+sr-primitives = { path = "../../../../primitives/sr-primitives", default-features = false }
 
 [features]
 default = ["std"]
 std = [
 	"serde",
-	"client/std",
+	"sr-api/std",
 	"codec/std",
 	"rstd/std",
 	"sr-primitives/std",
diff --git a/srml/transaction-payment/rpc/runtime-api/src/lib.rs b/paint/transaction-payment/rpc/runtime-api/src/lib.rs
similarity index 98%
rename from srml/transaction-payment/rpc/runtime-api/src/lib.rs
rename to paint/transaction-payment/rpc/runtime-api/src/lib.rs
index dc6360ca88d76..1254d5acb98f6 100644
--- a/srml/transaction-payment/rpc/runtime-api/src/lib.rs
+++ b/paint/transaction-payment/rpc/runtime-api/src/lib.rs
@@ -37,7 +37,7 @@ pub struct RuntimeDispatchInfo {
 	pub partial_fee: Balance,
 }
 
-client::decl_runtime_apis! {
+sr_api::decl_runtime_apis! {
 	pub trait TransactionPaymentApi where
 		Balance: Codec,
 		Extrinsic: Codec,
diff --git a/srml/transaction-payment/rpc/src/lib.rs b/paint/transaction-payment/rpc/src/lib.rs
similarity index 94%
rename from srml/transaction-payment/rpc/src/lib.rs
rename to paint/transaction-payment/rpc/src/lib.rs
index 6ee3c7eddbc55..8b59263b57959 100644
--- a/srml/transaction-payment/rpc/src/lib.rs
+++ b/paint/transaction-payment/rpc/src/lib.rs
@@ -26,8 +26,8 @@ use sr_primitives::{
 	traits::{Block as BlockT, ProvideRuntimeApi},
 };
 use primitives::Bytes;
-use srml_transaction_payment_rpc_runtime_api::RuntimeDispatchInfo;
-pub use srml_transaction_payment_rpc_runtime_api::TransactionPaymentApi as TransactionPaymentRuntimeApi;
+use paint_transaction_payment_rpc_runtime_api::RuntimeDispatchInfo;
+pub use paint_transaction_payment_rpc_runtime_api::TransactionPaymentApi as TransactionPaymentRuntimeApi;
 pub use self::gen_client::Client as TransactionPaymentClient;
 
 #[rpc]
diff --git a/srml/transaction-payment/src/lib.rs b/paint/transaction-payment/src/lib.rs
similarity index 100%
rename from srml/transaction-payment/src/lib.rs
rename to paint/transaction-payment/src/lib.rs
diff --git a/srml/treasury/Cargo.toml b/paint/treasury/Cargo.toml
similarity index 52%
rename from srml/treasury/Cargo.toml
rename to paint/treasury/Cargo.toml
index f19fa2c2188eb..7f917fb22e50b 100644
--- a/srml/treasury/Cargo.toml
+++ b/paint/treasury/Cargo.toml
@@ -1,5 +1,5 @@
 [package]
-name = "srml-treasury"
+name = "paint-treasury"
 version = "2.0.0"
 authors = ["Parity Technologies "]
 edition = "2018"
@@ -7,15 +7,15 @@ edition = "2018"
 [dependencies]
 serde = { version = "1.0.101", optional = true, features = ["derive"] }
 codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false, features = ["derive"] }
-rstd = { package = "sr-std", path = "../../core/sr-std", default-features = false }
-sr-primitives = { path = "../../core/sr-primitives", default-features = false }
-support = { package = "srml-support", path = "../support", default-features = false }
-system = { package = "srml-system", path = "../system", default-features = false }
-balances = { package = "srml-balances", path = "../balances", default-features = false }
+rstd = { package = "sr-std", path = "../../primitives/sr-std", default-features = false }
+sr-primitives = { path = "../../primitives/sr-primitives", default-features = false }
+support = { package = "paint-support", path = "../support", default-features = false }
+system = { package = "paint-system", path = "../system", default-features = false }
+balances = { package = "paint-balances", path = "../balances", default-features = false }
 
 [dev-dependencies]
-runtime-io ={ package = "sr-io", path = "../../core/sr-io" }
-primitives = { package = "substrate-primitives",  path = "../../core/primitives" }
+runtime-io ={ package = "sr-io", path = "../../primitives/sr-io" }
+primitives = { package = "substrate-primitives",  path = "../../primitives/core" }
 
 [features]
 default = ["std"]
diff --git a/srml/treasury/src/lib.rs b/paint/treasury/src/lib.rs
similarity index 87%
rename from srml/treasury/src/lib.rs
rename to paint/treasury/src/lib.rs
index 7c38115bfbab1..4a613dfb7859c 100644
--- a/srml/treasury/src/lib.rs
+++ b/paint/treasury/src/lib.rs
@@ -41,16 +41,6 @@
 //! respectively.
 //! - **Pot:** Unspent funds accumulated by the treasury module.
 //!
-//! ### Implementations
-//!
-//! The treasury module provides an implementation for the following trait:
-//!
-//! - `OnDilution` - When new funds are minted to reward the deployment of other existing funds,
-//! a corresponding amount of tokens are minted into the treasury so that the tokens being rewarded
-//! do not represent a higher portion of total supply. For example, in the default substrate node,
-//! when validators are rewarded new tokens for staking, they do not hold a higher portion of total
-//! tokens. Rather, tokens are added to the treasury to keep the portion of tokens staked constant.
-//!
 //! ## Interface
 //!
 //! ### Dispatchable Functions
@@ -72,12 +62,12 @@ use serde::{Serialize, Deserialize};
 use rstd::prelude::*;
 use support::{decl_module, decl_storage, decl_event, ensure, print};
 use support::traits::{
-	Currency, ExistenceRequirement, Get, Imbalance, OnDilution, OnUnbalanced,
+	Currency, ExistenceRequirement, Get, Imbalance, OnUnbalanced,
 	ReservableCurrency, WithdrawReason
 };
-use sr_primitives::{Permill, Perbill, ModuleId};
+use sr_primitives::{Permill, ModuleId};
 use sr_primitives::traits::{
-	Zero, EnsureOrigin, StaticLookup, AccountIdConversion, CheckedSub, Saturating
+	Zero, EnsureOrigin, StaticLookup, AccountIdConversion, Saturating
 };
 use sr_primitives::weights::SimpleDispatchInfo;
 use codec::{Encode, Decode};
@@ -257,6 +247,8 @@ decl_event!(
 		Burnt(Balance),
 		/// Spending has finished; this is the amount that rolls over until next spend.
 		Rollover(Balance),
+		/// Some funds have been deposited.
+		Deposit(Balance),
 	}
 );
 
@@ -345,30 +337,13 @@ impl Module {
 }
 
 impl OnUnbalanced> for Module {
-	fn on_unbalanced(amount: NegativeImbalanceOf) {
+	fn on_nonzero_unbalanced(amount: NegativeImbalanceOf) {
+		let numeric_amount = amount.peek();
+
 		// Must resolve into existing but better to be safe.
 		let _ = T::Currency::resolve_creating(&Self::account_id(), amount);
-	}
-}
 
-/// Mint extra funds for the treasury to keep the ratio of portion to total_issuance equal
-/// pre dilution and post-dilution.
-///
-/// i.e.
-/// ```nocompile
-/// portion / total_issuance_before_dilution ==
-///   (portion + minted) / (total_issuance_before_dilution + minted_to_treasury + minted)
-/// ```
-impl OnDilution> for Module {
-	fn on_dilution(minted: BalanceOf, portion: BalanceOf) {
-		if !minted.is_zero() && !portion.is_zero() {
-			let total_issuance = T::Currency::total_issuance();
-			if let Some(funding) = total_issuance.checked_sub(&portion) {
-				let increase_ratio = Perbill::from_rational_approximation(minted, portion);
-				let funding = increase_ratio * funding;
-				Self::on_unbalanced(T::Currency::issue(funding));
-			}
-		}
+		Self::deposit_event(RawEvent::Deposit(numeric_amount));
 	}
 }
 
@@ -379,7 +354,7 @@ mod tests {
 	use support::{assert_noop, assert_ok, impl_outer_origin, parameter_types};
 	use primitives::H256;
 	use sr_primitives::{
-		traits::{BlakeTwo256, OnFinalize, IdentityLookup}, testing::Header, assert_eq_error_rate,
+		traits::{BlakeTwo256, OnFinalize, IdentityLookup}, testing::Header, Perbill
 	};
 
 	impl_outer_origin! {
@@ -470,37 +445,11 @@ mod tests {
 	fn minting_works() {
 		new_test_ext().execute_with(|| {
 			// Check that accumulate works when we have Some value in Dummy already.
-			Treasury::on_dilution(100, 100);
+			Balances::make_free_balance_be(&Treasury::account_id(), 101);
 			assert_eq!(Treasury::pot(), 100);
 		});
 	}
 
-	#[test]
-	fn minting_works_2() {
-		let tests = [(1, 10), (1, 20), (40, 130), (2, 66), (2, 67), (2, 100), (2, 101), (2, 134)];
-		for &(minted, portion) in &tests {
-			new_test_ext().execute_with(|| {
-				let init_total_issuance = Balances::total_issuance();
-				Treasury::on_dilution(minted, portion);
-
-				assert_eq!(
-					Treasury::pot(),
-					(((init_total_issuance - portion) * minted) as f32 / portion as f32)
-						.round() as u64
-				);
-
-				// Assert:
-				// portion / init_total_issuance
-				// == (portion + minted) / (init_total_issuance + Treasury::pot() + minted),
-				assert_eq_error_rate!(
-					portion * 1_000 / init_total_issuance,
-					(portion + minted) * 1_000 / (init_total_issuance + Treasury::pot() + minted),
-					2,
-				);
-			});
-		}
-	}
-
 	#[test]
 	fn spend_proposal_takes_min_deposit() {
 		new_test_ext().execute_with(|| {
@@ -529,7 +478,7 @@ mod tests {
 	#[test]
 	fn accepted_spend_proposal_ignored_outside_spend_period() {
 		new_test_ext().execute_with(|| {
-			Treasury::on_dilution(100, 100);
+			Balances::make_free_balance_be(&Treasury::account_id(), 101);
 
 			assert_ok!(Treasury::propose_spend(Origin::signed(0), 100, 3));
 			assert_ok!(Treasury::approve_proposal(Origin::ROOT, 0));
@@ -544,7 +493,7 @@ mod tests {
 	fn unused_pot_should_diminish() {
 		new_test_ext().execute_with(|| {
 			let init_total_issuance = Balances::total_issuance();
-			Treasury::on_dilution(100, 100);
+			Balances::make_free_balance_be(&Treasury::account_id(), 101);
 			assert_eq!(Balances::total_issuance(), init_total_issuance + 100);
 
 			>::on_finalize(2);
@@ -556,7 +505,7 @@ mod tests {
 	#[test]
 	fn rejected_spend_proposal_ignored_on_spend_period() {
 		new_test_ext().execute_with(|| {
-			Treasury::on_dilution(100, 100);
+			Balances::make_free_balance_be(&Treasury::account_id(), 101);
 
 			assert_ok!(Treasury::propose_spend(Origin::signed(0), 100, 3));
 			assert_ok!(Treasury::reject_proposal(Origin::ROOT, 0));
@@ -570,7 +519,7 @@ mod tests {
 	#[test]
 	fn reject_already_rejected_spend_proposal_fails() {
 		new_test_ext().execute_with(|| {
-			Treasury::on_dilution(100, 100);
+			Balances::make_free_balance_be(&Treasury::account_id(), 101);
 
 			assert_ok!(Treasury::propose_spend(Origin::signed(0), 100, 3));
 			assert_ok!(Treasury::reject_proposal(Origin::ROOT, 0));
@@ -595,7 +544,7 @@ mod tests {
 	#[test]
 	fn accept_already_rejected_spend_proposal_fails() {
 		new_test_ext().execute_with(|| {
-			Treasury::on_dilution(100, 100);
+			Balances::make_free_balance_be(&Treasury::account_id(), 101);
 
 			assert_ok!(Treasury::propose_spend(Origin::signed(0), 100, 3));
 			assert_ok!(Treasury::reject_proposal(Origin::ROOT, 0));
@@ -606,7 +555,7 @@ mod tests {
 	#[test]
 	fn accepted_spend_proposal_enacted_on_spend_period() {
 		new_test_ext().execute_with(|| {
-			Treasury::on_dilution(100, 100);
+			Balances::make_free_balance_be(&Treasury::account_id(), 101);
 			assert_eq!(Treasury::pot(), 100);
 
 			assert_ok!(Treasury::propose_spend(Origin::signed(0), 100, 3));
@@ -621,7 +570,7 @@ mod tests {
 	#[test]
 	fn pot_underflow_should_not_diminish() {
 		new_test_ext().execute_with(|| {
-			Treasury::on_dilution(100, 100);
+			Balances::make_free_balance_be(&Treasury::account_id(), 101);
 			assert_eq!(Treasury::pot(), 100);
 
 			assert_ok!(Treasury::propose_spend(Origin::signed(0), 150, 3));
@@ -630,10 +579,10 @@ mod tests {
 			>::on_finalize(2);
 			assert_eq!(Treasury::pot(), 100); // Pot hasn't changed
 
-			Treasury::on_dilution(100, 100);
+			let _ = Balances::deposit_into_existing(&Treasury::account_id(), 100).unwrap();
 			>::on_finalize(4);
 			assert_eq!(Balances::free_balance(&3), 150); // Fund has been spent
-			assert_eq!(Treasury::pot(), 75); // Pot has finally changed
+			assert_eq!(Treasury::pot(), 25); // Pot has finally changed
 		});
 	}
 
@@ -642,7 +591,7 @@ mod tests {
 	#[test]
 	fn treasury_account_doesnt_get_deleted() {
 		new_test_ext().execute_with(|| {
-			Treasury::on_dilution(100, 100);
+			Balances::make_free_balance_be(&Treasury::account_id(), 101);
 			assert_eq!(Treasury::pot(), 100);
 			let treasury_balance = Balances::free_balance(&Treasury::account_id());
 
@@ -685,7 +634,7 @@ mod tests {
 			assert_eq!(Treasury::pot(), 0); // Pot hasn't changed
 			assert_eq!(Balances::free_balance(&3), 0); // Balance of `3` hasn't changed
 
-			Treasury::on_dilution(100, 100);
+			Balances::make_free_balance_be(&Treasury::account_id(), 100);
 			assert_eq!(Treasury::pot(), 99); // Pot now contains funds
 			assert_eq!(Balances::free_balance(&Treasury::account_id()), 100); // Account does exist
 
diff --git a/srml/utility/Cargo.toml b/paint/utility/Cargo.toml
similarity index 50%
rename from srml/utility/Cargo.toml
rename to paint/utility/Cargo.toml
index 5e92bcf88ad48..1415c3e92245a 100644
--- a/srml/utility/Cargo.toml
+++ b/paint/utility/Cargo.toml
@@ -1,5 +1,5 @@
 [package]
-name = "srml-utility"
+name = "paint-utility"
 version = "2.0.0"
 authors = ["Parity Technologies "]
 edition = "2018"
@@ -7,15 +7,15 @@ edition = "2018"
 [dependencies]
 serde = { version = "1.0.101", optional = true }
 codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false }
-support = { package = "srml-support", path = "../support", default-features = false }
-system = { package = "srml-system", path = "../system", default-features = false }
-sr-primitives = { path = "../../core/sr-primitives", default-features = false }
-rstd = { package = "sr-std", path = "../../core/sr-std", default-features = false }
-runtime-io = { package = "sr-io", path = "../../core/sr-io", default-features = false }
+support = { package = "paint-support", path = "../support", default-features = false }
+system = { package = "paint-system", path = "../system", default-features = false }
+sr-primitives = { path = "../../primitives/sr-primitives", default-features = false }
+rstd = { package = "sr-std", path = "../../primitives/sr-std", default-features = false }
+runtime-io = { package = "sr-io", path = "../../primitives/sr-io", default-features = false }
 
 [dev-dependencies]
-primitives = { package = "substrate-primitives",  path = "../../core/primitives" }
-balances = { package = "srml-balances", path = "../balances" }
+primitives = { package = "substrate-primitives",  path = "../../primitives/core" }
+balances = { package = "paint-balances", path = "../balances" }
 
 [features]
 default = ["std"]
diff --git a/srml/utility/src/lib.rs b/paint/utility/src/lib.rs
similarity index 100%
rename from srml/utility/src/lib.rs
rename to paint/utility/src/lib.rs
diff --git a/core/application-crypto/Cargo.toml b/primitives/application-crypto/Cargo.toml
similarity index 51%
rename from core/application-crypto/Cargo.toml
rename to primitives/application-crypto/Cargo.toml
index 3a48446696058..598f74ac15bd4 100644
--- a/core/application-crypto/Cargo.toml
+++ b/primitives/application-crypto/Cargo.toml
@@ -6,16 +6,23 @@ edition = "2018"
 description = "Provides facilities for generating application specific crypto wrapper types."
 
 [dependencies]
-primitives = { package = "substrate-primitives", path = "../primitives", default-features = false }
+primitives = { package = "substrate-primitives", path = "../core", default-features = false }
 codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false, features = ["derive"] }
 serde = { version = "1.0.101", optional = true, features = ["derive"] }
 rstd = { package = "sr-std", path = "../sr-std", default-features = false }
-runtime-io = { package = "sr-io", path = "../sr-io", default-features = false }
+runtime-io = { package = "sr-io", path = "../../primitives/sr-io", default-features = false }
 
 [dev-dependencies]
-test-client = { package = "substrate-test-runtime-client", path = "../test-runtime/client" }
-sr-primitives = { path = "../sr-primitives" }
+test-client = { package = "substrate-test-runtime-client", path = "../../test/utils/runtime/client" }
+sr-primitives = { path = "../../primitives/sr-primitives" }
 
 [features]
 default = [ "std" ]
-std = [ "primitives/std", "codec/std", "serde", "rstd/std", "runtime-io/std" ]
+std = [ "full_crypto", "primitives/std", "codec/std", "serde", "rstd/std", "runtime-io/std" ]
+
+# This feature enables all crypto primitives for `no_std` builds like microcontrollers 
+# or Intel SGX. 
+# For the regular wasm runtime builds this should not be used.
+full_crypto = [
+    "primitives/full_crypto"
+]
\ No newline at end of file
diff --git a/core/application-crypto/src/ed25519.rs b/primitives/application-crypto/src/ed25519.rs
similarity index 83%
rename from core/application-crypto/src/ed25519.rs
rename to primitives/application-crypto/src/ed25519.rs
index ac01cef61b1c5..468907a5c40a4 100644
--- a/core/application-crypto/src/ed25519.rs
+++ b/primitives/application-crypto/src/ed25519.rs
@@ -18,6 +18,8 @@
 
 use crate::{RuntimePublic, KeyTypeId};
 
+use rstd::vec::Vec;
+
 pub use primitives::ed25519::*;
 
 mod app {
@@ -29,28 +31,27 @@ mod app {
 	}
 }
 
-pub use app::Public as AppPublic;
-pub use app::Signature as AppSignature;
-#[cfg(feature="std")]
+pub use app::{Public as AppPublic, Signature as AppSignature};
+#[cfg(feature = "full_crypto")]
 pub use app::Pair as AppPair;
 
 impl RuntimePublic for Public {
 	type Signature = Signature;
 
 	fn all(key_type: KeyTypeId) -> crate::Vec {
-		runtime_io::ed25519_public_keys(key_type)
+		runtime_io::crypto::ed25519_public_keys(key_type)
 	}
 
-	fn generate_pair(key_type: KeyTypeId, seed: Option<&str>) -> Self {
-		runtime_io::ed25519_generate(key_type, seed)
+	fn generate_pair(key_type: KeyTypeId, seed: Option>) -> Self {
+		runtime_io::crypto::ed25519_generate(key_type, seed)
 	}
 
 	fn sign>(&self, key_type: KeyTypeId, msg: &M) -> Option {
-		runtime_io::ed25519_sign(key_type, self, msg.as_ref())
+		runtime_io::crypto::ed25519_sign(key_type, self, msg.as_ref())
 	}
 
 	fn verify>(&self, msg: &M, signature: &Self::Signature) -> bool {
-		runtime_io::ed25519_verify(&signature, msg.as_ref(), self)
+		runtime_io::crypto::ed25519_verify(&signature, msg.as_ref(), self)
 	}
 }
 
diff --git a/core/application-crypto/src/lib.rs b/primitives/application-crypto/src/lib.rs
similarity index 64%
rename from core/application-crypto/src/lib.rs
rename to primitives/application-crypto/src/lib.rs
index b4ada5425b400..a02b1003b2ff4 100644
--- a/core/application-crypto/src/lib.rs
+++ b/primitives/application-crypto/src/lib.rs
@@ -23,7 +23,7 @@
 #[doc(hidden)]
 pub use primitives::{self, crypto::{CryptoType, Public, Derive, IsWrappedBy, Wraps}, RuntimeDebug};
 #[doc(hidden)]
-#[cfg(feature = "std")]
+#[cfg(feature = "full_crypto")]
 pub use primitives::crypto::{SecretStringError, DeriveJunction, Ss58Codec, Pair};
 pub use primitives::{crypto::{KeyTypeId, key_types}};
 
@@ -50,17 +50,43 @@ pub use traits::*;
 /// // of value `b"fuba"`.
 /// app_crypto!(ed25519, KeyTypeId(*b"_uba"));
 /// ```
+#[cfg(feature = "full_crypto")]
 #[macro_export]
 macro_rules! app_crypto {
 	($module:ident, $key_type:expr) => {
-		#[cfg(feature="std")]
-		$crate::app_crypto!($module::Pair, $module::Public, $module::Signature, $key_type);
-		#[cfg(not(feature="std"))]
-		$crate::app_crypto!($module::Public, $module::Signature, $key_type);
+		$crate::app_crypto_public_full_crypto!($module::Public, $key_type);
+		$crate::app_crypto_public_common!($module::Public, $module::Signature, $key_type);
+		$crate::app_crypto_signature_full_crypto!($module::Signature, $key_type);
+		$crate::app_crypto_signature_common!($module::Signature, $key_type);
+		$crate::app_crypto_pair!($module::Pair, $key_type);
 	};
-	($pair:ty, $public:ty, $sig:ty, $key_type:expr) => {
-		$crate::app_crypto!($public, $sig, $key_type);
+}
 
+/// Declares Public, Pair, Signature types which are functionally equivalent to `$pair`, but are new
+/// Application-specific types whose identifier is `$key_type`.
+///
+/// ```rust
+///# use substrate_application_crypto::{app_crypto, wrap, ed25519, KeyTypeId};
+/// // Declare a new set of crypto types using Ed25519 logic that identifies as `KeyTypeId`
+/// // of value `b"fuba"`.
+/// app_crypto!(ed25519, KeyTypeId(*b"_uba"));
+/// ```
+#[cfg(not(feature = "full_crypto"))]
+#[macro_export]
+macro_rules! app_crypto {
+	($module:ident, $key_type:expr) => {
+		$crate::app_crypto_public_not_full_crypto!($module::Public, $key_type);
+		$crate::app_crypto_public_common!($module::Public, $module::Signature, $key_type);
+		$crate::app_crypto_signature_not_full_crypto!($module::Signature, $key_type);
+		$crate::app_crypto_signature_common!($module::Signature, $key_type);
+	};
+}
+
+/// Declares Pair type which is functionally equivalent to `$pair`, but is new
+/// Application-specific type whose identifier is `$key_type`.
+#[macro_export]
+macro_rules! app_crypto_pair {
+	($pair:ty, $key_type:expr) => {
 		$crate::wrap!{
 			/// A generic `AppPublic` wrapper type over $pair crypto; this has no specific App.
 			#[derive(Clone)]
@@ -71,16 +97,18 @@ macro_rules! app_crypto {
 			type Pair = Pair;
 		}
 
-		#[cfg(feature = "std")]
 		impl $crate::Pair for Pair {
 			type Public = Public;
 			type Seed = <$pair as $crate::Pair>::Seed;
 			type Signature = Signature;
 			type DeriveError = <$pair as $crate::Pair>::DeriveError;
+			
+			#[cfg(feature = "std")]
 			fn generate_with_phrase(password: Option<&str>) -> (Self, String, Self::Seed) {
 				let r = <$pair>::generate_with_phrase(password);
 				(Self(r.0), r.1, r.2)
 			}
+			#[cfg(feature = "std")]
 			fn from_phrase(phrase: &str, password: Option<&str>)
 				-> Result<(Self, Self::Seed), $crate::SecretStringError>
 			{
@@ -115,6 +143,7 @@ macro_rules! app_crypto {
 			fn public(&self) -> Self::Public { Public(self.0.public()) }
 			fn to_raw_vec(&self) -> Vec { self.0.to_raw_vec() }
 		}
+
 		impl $crate::AppKey for Pair {
 			type UntypedGeneric = $pair;
 			type Public = Public;
@@ -122,11 +151,20 @@ macro_rules! app_crypto {
 			type Signature = Signature;
 			const ID: $crate::KeyTypeId = $key_type;
 		}
+
 		impl $crate::AppPair for Pair {
 			type Generic = $pair;
 		}
 	};
-	($public:ty, $sig:ty, $key_type:expr) => {
+}
+
+/// Declares Public type which is functionally equivalent to `$public`, but is new
+/// Application-specific type whose identifier is `$key_type`.
+/// can only be used together with `full_crypto` feature
+/// For full functionality, app_crypto_public_common! must be called too.
+#[macro_export]
+macro_rules! app_crypto_public_full_crypto {
+	($public:ty, $key_type:expr) => {
 		$crate::wrap!{
 			/// A generic `AppPublic` wrapper type over $public crypto; this has no specific App.
 			#[derive(
@@ -135,10 +173,59 @@ macro_rules! app_crypto {
 				$crate::codec::Decode,
 				$crate::RuntimeDebug,
 			)]
-			#[cfg_attr(feature = "std", derive(Hash))]
+			#[derive(Hash)]
 			pub struct Public($public);
 		}
 
+		impl $crate::CryptoType for Public {
+			type Pair = Pair;
+		}
+
+		impl $crate::AppKey for Public {
+			type UntypedGeneric = $public;
+			type Public = Public;
+			type Pair = Pair;
+			type Signature = Signature;
+			const ID: $crate::KeyTypeId = $key_type;
+		}
+	}
+}
+
+/// Declares Public type which is functionally equivalent to `$public`, but is new
+/// Application-specific type whose identifier is `$key_type`.
+/// can only be used without `full_crypto` feature
+/// For full functionality, app_crypto_public_common! must be called too.
+#[macro_export]
+macro_rules! app_crypto_public_not_full_crypto {
+	($public:ty, $key_type:expr) => {
+		$crate::wrap!{
+			/// A generic `AppPublic` wrapper type over $public crypto; this has no specific App.
+			#[derive(
+				Clone, Default, Eq, PartialEq, Ord, PartialOrd,
+				$crate::codec::Encode,
+				$crate::codec::Decode,
+				$crate::RuntimeDebug,
+			)]
+			pub struct Public($public);
+		}
+
+		impl $crate::CryptoType for Public {}
+
+		impl $crate::AppKey for Public {
+			type UntypedGeneric = $public;
+			type Public = Public;
+			type Signature = Signature;
+			const ID: $crate::KeyTypeId = $key_type;
+		}
+	}
+}
+
+/// Declares Public type which is functionally equivalent to `$public`, but is new
+/// Application-specific type whose identifier is `$key_type`.
+/// For full functionality, app_crypto_public_(not)_full_crypto! must be called too.
+#[macro_export]
+macro_rules! app_crypto_public_common {
+	($public:ty, $sig:ty, $key_type:expr) => {
 		impl $crate::Derive for Public {
 			#[cfg(feature = "std")]
 			fn derive>(&self,
@@ -183,24 +270,10 @@ macro_rules! app_crypto {
 			fn as_mut(&mut self) -> &mut [u8] { self.0.as_mut() }
 		}
 
-		impl $crate::CryptoType for Public {
-			#[cfg(feature="std")]
-			type Pair = Pair;
-		}
-
 		impl $crate::Public for Public {
 			fn from_slice(x: &[u8]) -> Self { Self(<$public>::from_slice(x)) }
 		}
 
-		impl $crate::AppKey for Public {
-			type UntypedGeneric = $public;
-			type Public = Public;
-			#[cfg(feature="std")]
-			type Pair = Pair;
-			type Signature = Signature;
-			const ID: $crate::KeyTypeId = $key_type;
-		}
-
 		impl $crate::AppPublic for Public {
 			type Generic = $public;
 		}
@@ -213,7 +286,7 @@ macro_rules! app_crypto {
 				<$public as $crate::RuntimePublic>::all($key_type).into_iter().map(Self).collect()
 			}
 
-			fn generate_pair(seed: Option<&str>) -> Self {
+			fn generate_pair(seed: Option<$crate::Vec>) -> Self {
 				Self(<$public as $crate::RuntimePublic>::generate_pair($key_type, seed))
 			}
 
@@ -229,7 +302,16 @@ macro_rules! app_crypto {
 				<$public as $crate::RuntimePublic>::verify(self.as_ref(), msg, &signature.as_ref())
 			}
 		}
+	}
+}
 
+/// Declares Signature type which is functionally equivalent to `$sig`, but is new
+/// Application-specific type whose identifier is `$key_type`.
+/// can only be used together with `full_crypto` feature
+/// For full functionality, app_crypto_public_common! must be called too.
+#[macro_export]
+macro_rules! app_crypto_signature_full_crypto {
+	($sig:ty, $key_type:expr) => {
 		$crate::wrap! {
 			/// A generic `AppPublic` wrapper type over $public crypto; this has no specific App.
 			#[derive(Clone, Default, Eq, PartialEq,
@@ -237,33 +319,67 @@ macro_rules! app_crypto {
 				$crate::codec::Decode,
 				$crate::RuntimeDebug,
 			)]
-			#[cfg_attr(feature = "std", derive(Hash))]
+			#[derive(Hash)]
 			pub struct Signature($sig);
 		}
 
-		impl $crate::Deref for Signature {
-			type Target = [u8];
-
-			fn deref(&self) -> &Self::Target { self.0.as_ref() }
+		impl $crate::CryptoType for Signature {
+			type Pair = Pair;
 		}
 
-		impl AsRef<[u8]> for Signature {
-			fn as_ref(&self) -> &[u8] { self.0.as_ref() }
+		impl $crate::AppKey for Signature {
+			type UntypedGeneric = $sig;
+			type Public = Public;
+			type Pair = Pair;
+			type Signature = Signature;
+			const ID: $crate::KeyTypeId = $key_type;
 		}
+	}
+}
 
-		impl $crate::CryptoType for Signature {
-			#[cfg(feature="std")]
-			type Pair = Pair;
+/// Declares Signature type which is functionally equivalent to `$sig`, but is new
+/// Application-specific type whose identifier is `$key_type`.
+/// can only be used without `full_crypto` feature
+/// For full functionality, app_crypto_public_common! must be called too.
+#[macro_export]
+macro_rules! app_crypto_signature_not_full_crypto {
+	($sig:ty, $key_type:expr) => {
+		$crate::wrap! {
+			/// A generic `AppPublic` wrapper type over $public crypto; this has no specific App.
+			#[derive(Clone, Default, Eq, PartialEq,
+				$crate::codec::Encode,
+				$crate::codec::Decode,
+				$crate::RuntimeDebug,
+			)]
+			pub struct Signature($sig);
 		}
+		
+		impl $crate::CryptoType for Signature {}
 
 		impl $crate::AppKey for Signature {
 			type UntypedGeneric = $sig;
 			type Public = Public;
-			#[cfg(feature="std")]
-			type Pair = Pair;
 			type Signature = Signature;
 			const ID: $crate::KeyTypeId = $key_type;
 		}
+	}
+}
+
+/// Declares Signature type which is functionally equivalent to `$sig`, but is new
+/// Application-specific type whose identifier is `$key_type`.
+/// For full functionality, app_crypto_public_(not)_full_crypto! must be called too.
+#[macro_export]
+macro_rules! app_crypto_signature_common {
+	($sig:ty, $key_type:expr) => {
+		impl $crate::Deref for Signature {
+			type Target = [u8];
+
+			fn deref(&self) -> &Self::Target { self.0.as_ref() }
+		}
+
+		impl AsRef<[u8]> for Signature {
+			fn as_ref(&self) -> &[u8] { self.0.as_ref() }
+		}
 
 		impl $crate::AppSignature for Signature {
 			type Generic = $sig;
diff --git a/core/application-crypto/src/sr25519.rs b/primitives/application-crypto/src/sr25519.rs
similarity index 83%
rename from core/application-crypto/src/sr25519.rs
rename to primitives/application-crypto/src/sr25519.rs
index c343a2e1676b5..2ad279a6bfd42 100644
--- a/core/application-crypto/src/sr25519.rs
+++ b/primitives/application-crypto/src/sr25519.rs
@@ -18,6 +18,8 @@
 
 use crate::{RuntimePublic, KeyTypeId};
 
+use rstd::vec::Vec;
+
 pub use primitives::sr25519::*;
 
 mod app {
@@ -29,28 +31,27 @@ mod app {
 	}
 }
 
-pub use app::Public as AppPublic;
-pub use app::Signature as AppSignature;
-#[cfg(feature="std")]
+pub use app::{Public as AppPublic, Signature as AppSignature};
+#[cfg(feature = "full_crypto")]
 pub use app::Pair as AppPair;
 
 impl RuntimePublic for Public {
 	type Signature = Signature;
 
 	fn all(key_type: KeyTypeId) -> crate::Vec {
-		runtime_io::sr25519_public_keys(key_type)
+		runtime_io::crypto::sr25519_public_keys(key_type)
 	}
 
-	fn generate_pair(key_type: KeyTypeId, seed: Option<&str>) -> Self {
-		runtime_io::sr25519_generate(key_type, seed)
+	fn generate_pair(key_type: KeyTypeId, seed: Option>) -> Self {
+		runtime_io::crypto::sr25519_generate(key_type, seed)
 	}
 
 	fn sign>(&self, key_type: KeyTypeId, msg: &M) -> Option {
-		runtime_io::sr25519_sign(key_type, self, msg.as_ref())
+		runtime_io::crypto::sr25519_sign(key_type, self, msg.as_ref())
 	}
 
 	fn verify>(&self, msg: &M, signature: &Self::Signature) -> bool {
-		runtime_io::sr25519_verify(&signature, msg.as_ref(), self)
+		runtime_io::crypto::sr25519_verify(&signature, msg.as_ref(), self)
 	}
 }
 
diff --git a/core/application-crypto/src/traits.rs b/primitives/application-crypto/src/traits.rs
similarity index 81%
rename from core/application-crypto/src/traits.rs
rename to primitives/application-crypto/src/traits.rs
index db71f3072af58..0c99d49ce3bff 100644
--- a/core/application-crypto/src/traits.rs
+++ b/primitives/application-crypto/src/traits.rs
@@ -14,12 +14,12 @@
 // You should have received a copy of the GNU General Public License
 // along with Substrate.  If not, see .
 
-#[cfg(feature = "std")]
+#[cfg(feature = "full_crypto")]
 use primitives::crypto::Pair;
 
 use codec::Codec;
 use primitives::crypto::{KeyTypeId, CryptoType, IsWrappedBy, Public};
-use rstd::fmt::Debug;
+use rstd::{fmt::Debug, vec::Vec};
 
 /// An application-specific key.
 pub trait AppKey: 'static + Send + Sync + Sized + CryptoType + Clone {
@@ -30,7 +30,7 @@ pub trait AppKey: 'static + Send + Sync + Sized + CryptoType + Clone {
 	type Public: AppPublic;
 
 	/// The corresponding key pair type in this application scheme.
-	#[cfg(feature="std")]
+	#[cfg(feature = "full_crypto")]
 	type Pair: AppPair;
 
 	/// The corresponding signature type in this application scheme.
@@ -42,16 +42,22 @@ pub trait AppKey: 'static + Send + Sync + Sized + CryptoType + Clone {
 
 /// Type which implements Hash in std, not when no-std (std variant).
 #[cfg(feature = "std")]
-pub trait MaybeHash: std::hash::Hash {}
+pub trait MaybeHash: rstd::hash::Hash {}
 #[cfg(feature = "std")]
-impl MaybeHash for T {}
+impl MaybeHash for T {}
 
 /// Type which implements Hash in std, not when no-std (no-std variant).
-#[cfg(not(feature = "std"))]
+#[cfg(all(not(feature = "std"), not(feature = "full_crypto")))]
 pub trait MaybeHash {}
-#[cfg(not(feature = "std"))]
+#[cfg(all(not(feature = "std"), not(feature = "full_crypto")))]
 impl MaybeHash for T {}
 
+/// Type which implements Debug and Hash in std, not when no-std (no-std variant with crypto).
+#[cfg(all(not(feature = "std"), feature = "full_crypto"))]
+pub trait MaybeDebugHash: rstd::hash::Hash  {}
+#[cfg(all(not(feature = "std"), feature = "full_crypto"))]
+impl MaybeDebugHash for T {}
+
 /// A application's public key.
 pub trait AppPublic:
 	AppKey + Public + Ord + PartialOrd + Eq + PartialEq + Debug + MaybeHash + codec::Codec
@@ -62,7 +68,7 @@ pub trait AppPublic:
 }
 
 /// A application's key pair.
-#[cfg(feature = "std")]
+#[cfg(feature = "full_crypto")]
 pub trait AppPair: AppKey + Pair::Public> {
 	/// The wrapped type which is just a plain instance of `Pair`.
 	type Generic: IsWrappedBy + Pair::Public as AppPublic>::Generic>;
@@ -82,10 +88,13 @@ pub trait RuntimePublic: Sized {
 	/// Returns all public keys for the given key type in the keystore.
 	fn all(key_type: KeyTypeId) -> crate::Vec;
 
-	/// Generate a public/private pair for the given key type and store it in the keystore.
+	/// Generate a public/private pair for the given key type with an optional `seed` and
+	/// store it in the keystore.
+	///
+	/// The `seed` needs to be valid utf8.
 	///
 	/// Returns the generated public key.
-	fn generate_pair(key_type: KeyTypeId, seed: Option<&str>) -> Self;
+	fn generate_pair(key_type: KeyTypeId, seed: Option>) -> Self;
 
 	/// Sign the given message with the corresponding private key of this public key.
 	///
@@ -110,10 +119,12 @@ pub trait RuntimeAppPublic: Sized  {
 	/// Returns all public keys for this application in the keystore.
 	fn all() -> crate::Vec;
 
-	/// Generate a public/private pair and store it in the keystore.
+	/// Generate a public/private pair with an optional `seed` and store it in the keystore.
+	///
+	/// The `seed` needs to be valid utf8.
 	///
 	/// Returns the generated public key.
-	fn generate_pair(seed: Option<&str>) -> Self;
+	fn generate_pair(seed: Option>) -> Self;
 
 	/// Sign the given message with the corresponding private key of this public key.
 	///
diff --git a/primitives/authority-discovery/Cargo.toml b/primitives/authority-discovery/Cargo.toml
new file mode 100644
index 0000000000000..1e975b5680105
--- /dev/null
+++ b/primitives/authority-discovery/Cargo.toml
@@ -0,0 +1,23 @@
+[package]
+name = "substrate-authority-discovery-primitives"
+version = "2.0.0"
+authors = ["Parity Technologies "]
+description = "Authority discovery primitives"
+edition = "2018"
+
+[dependencies]
+app-crypto = { package = "substrate-application-crypto", path = "../application-crypto", default-features = false }
+codec = { package = "parity-scale-codec", default-features = false, version = "1.0.3" }
+rstd = { package = "sr-std", path = "../sr-std", default-features = false }
+sr-api = { path = "../sr-api", default-features = false }
+sr-primitives = { path = "../sr-primitives", default-features = false }
+
+[features]
+default = ["std"]
+std = [
+	"app-crypto/std",
+	"codec/std",
+	"rstd/std",
+	"sr-api/std",
+	"sr-primitives/std"
+]
diff --git a/core/authority-discovery/primitives/src/lib.rs b/primitives/authority-discovery/src/lib.rs
similarity index 53%
rename from core/authority-discovery/primitives/src/lib.rs
rename to primitives/authority-discovery/src/lib.rs
index 7c56dc6ca4ca6..0116ca02aad1f 100644
--- a/core/authority-discovery/primitives/src/lib.rs
+++ b/primitives/authority-discovery/src/lib.rs
@@ -18,33 +18,30 @@
 
 #![cfg_attr(not(feature = "std"), no_std)]
 
-use client::decl_runtime_apis;
 use rstd::vec::Vec;
-use sr_primitives::RuntimeDebug;
 
-#[derive(codec::Encode, codec::Decode, Eq, PartialEq, Clone, RuntimeDebug)]
-#[cfg_attr(feature = "std", derive(Hash))]
-pub struct Signature(pub Vec);
-#[derive(codec::Encode, codec::Decode, Eq, PartialEq, Clone, RuntimeDebug)]
-#[cfg_attr(feature = "std", derive(Hash))]
-pub struct AuthorityId(pub Vec);
+mod app {
+	use app_crypto::{app_crypto, key_types::AUTHORITY_DISCOVERY, sr25519};
+	app_crypto!(sr25519, AUTHORITY_DISCOVERY);
+}
+
+/// An authority discovery authority keypair.
+#[cfg(feature = "std")]
+pub type AuthorityPair = app::Pair;
+
+/// An authority discovery authority identifier.
+pub type AuthorityId = app::Public;
 
-decl_runtime_apis! {
+/// An authority discovery authority signature.
+pub type AuthoritySignature = app::Signature;
+
+sr_api::decl_runtime_apis! {
 	/// The authority discovery api.
 	///
-	/// This api is used by the `core/authority-discovery` module to retrieve our
-	/// own authority identifier, to retrieve identifiers of the current authority
-	/// set, as well as sign and verify Kademlia Dht external address payloads
-	/// from and to other authorities.
+	/// This api is used by the `core/authority-discovery` module to retrieve identifiers
+	/// of the current authority set.
 	pub trait AuthorityDiscoveryApi {
 		/// Retrieve authority identifiers of the current authority set.
 		fn authorities() -> Vec;
-
-		/// Sign the given payload with the private key corresponding to the given authority id.
-		fn sign(payload: &Vec) -> Option<(Signature, AuthorityId)>;
-
-		/// Verify the given signature for the given payload with the given
-		/// authority identifier.
-		fn verify(payload: &Vec, signature: &Signature, authority_id: &AuthorityId) -> bool;
 	}
 }
diff --git a/primitives/block-builder/runtime-api/Cargo.toml b/primitives/block-builder/runtime-api/Cargo.toml
new file mode 100644
index 0000000000000..48f94ce0ee8e4
--- /dev/null
+++ b/primitives/block-builder/runtime-api/Cargo.toml
@@ -0,0 +1,20 @@
+[package]
+name = "substrate-block-builder-runtime-api"
+version = "2.0.0"
+authors = ["Parity Technologies "]
+edition = "2018"
+
+[dependencies]
+sr-primitives = { path = "../../sr-primitives", default-features = false }
+sr-api = { path = "../../sr-api", default-features = false }
+rstd = { package = "sr-std", path = "../../sr-std", default-features = false }
+inherents = { package = "substrate-inherents", path = "../../inherents", default-features = false }
+
+[features]
+default = [ "std" ]
+std = [
+	"sr-primitives/std",
+	"inherents/std",
+	"sr-api/std",
+	"rstd/std",
+]
diff --git a/core/client/src/block_builder/api.rs b/primitives/block-builder/runtime-api/src/lib.rs
similarity index 76%
rename from core/client/src/block_builder/api.rs
rename to primitives/block-builder/runtime-api/src/lib.rs
index 5bf742a4560d4..6469ac3d9ec62 100644
--- a/core/client/src/block_builder/api.rs
+++ b/primitives/block-builder/runtime-api/src/lib.rs
@@ -1,4 +1,4 @@
-// Copyright 2018-2019 Parity Technologies (UK) Ltd.
+// Copyright 2019 Parity Technologies (UK) Ltd.
 // This file is part of Substrate.
 
 // Substrate is free software: you can redistribute it and/or modify
@@ -14,15 +14,16 @@
 // You should have received a copy of the GNU General Public License
 // along with Substrate.  If not, see .
 
-//! The runtime api for building blocks.
+//! The block builder runtime api.
+
+#![cfg_attr(not(feature = "std"), no_std)]
 
 use sr_primitives::{traits::Block as BlockT, ApplyResult};
-use rstd::vec::Vec;
-use sr_api_macros::decl_runtime_apis;
-pub use inherents::{InherentData, CheckInherentsResult};
 
-decl_runtime_apis! {
-	/// The `BlockBuilder` api trait that provides required functions for building a block for a runtime.
+use inherents::{InherentData, CheckInherentsResult};
+
+sr_api::decl_runtime_apis! {
+	/// The `BlockBuilder` api trait that provides the required functionality for building a block.
 	#[api_version(3)]
 	pub trait BlockBuilder {
 		/// Apply the given extrinsics.
@@ -31,7 +32,9 @@ decl_runtime_apis! {
 		#[renamed("finalise_block", 3)]
 		fn finalize_block() -> ::Header;
 		/// Generate inherent extrinsics. The inherent data will vary from chain to chain.
-		fn inherent_extrinsics(inherent: InherentData) -> Vec<::Extrinsic>;
+		fn inherent_extrinsics(
+			inherent: InherentData,
+		) -> rstd::vec::Vec<::Extrinsic>;
 		/// Check that the inherents are valid. The inherent data will vary from chain to chain.
 		fn check_inherents(block: Block, data: InherentData) -> CheckInherentsResult;
 		/// Generate a random seed.
diff --git a/core/finality-grandpa/primitives/Cargo.toml b/primitives/consensus/aura/Cargo.toml
similarity index 66%
rename from core/finality-grandpa/primitives/Cargo.toml
rename to primitives/consensus/aura/Cargo.toml
index 02439d4150d81..20a98f9f7a617 100644
--- a/core/finality-grandpa/primitives/Cargo.toml
+++ b/primitives/consensus/aura/Cargo.toml
@@ -1,24 +1,23 @@
 [package]
-name = "substrate-finality-grandpa-primitives"
+name = "substrate-consensus-aura-primitives"
 version = "2.0.0"
 authors = ["Parity Technologies "]
+description = "Primitives for Aura consensus"
 edition = "2018"
 
 [dependencies]
-client = { package = "substrate-client", path = "../../client", default-features = false }
 app-crypto = { package = "substrate-application-crypto", path = "../../application-crypto", default-features = false }
-codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false, features = ["derive"] }
-sr-primitives = { path = "../../sr-primitives", default-features = false }
+codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false }
 rstd = { package = "sr-std", path = "../../sr-std", default-features = false }
-serde = { version = "1.0.101", optional = true, features = ["derive"] }
+sr-api = { path = "../../sr-api", default-features = false }
+sr-primitives = { path = "../../sr-primitives", default-features = false }
 
 [features]
 default = ["std"]
 std = [
-	"client/std",
+	"app-crypto/std",
 	"codec/std",
-	"sr-primitives/std",
 	"rstd/std",
-	"serde",
-	"app-crypto/std",
+	"sr-api/std",
+	"sr-primitives/std",
 ]
diff --git a/core/consensus/aura/primitives/src/lib.rs b/primitives/consensus/aura/src/lib.rs
similarity index 97%
rename from core/consensus/aura/primitives/src/lib.rs
rename to primitives/consensus/aura/src/lib.rs
index e4620fcdbfdd4..ccf2d8e5fd52a 100644
--- a/core/consensus/aura/primitives/src/lib.rs
+++ b/primitives/consensus/aura/src/lib.rs
@@ -19,7 +19,6 @@
 #![cfg_attr(not(feature = "std"), no_std)]
 
 use codec::{Encode, Decode, Codec};
-use substrate_client::decl_runtime_apis;
 use rstd::vec::Vec;
 use sr_primitives::ConsensusEngineId;
 
@@ -74,7 +73,7 @@ pub enum ConsensusLog {
 	OnDisabled(AuthorityIndex),
 }
 
-decl_runtime_apis! {
+sr_api::decl_runtime_apis! {
 	/// API necessary for block authorship with aura.
 	pub trait AuraApi {
 		/// Return the slot duration in seconds for Aura.
diff --git a/core/consensus/babe/primitives/Cargo.toml b/primitives/consensus/babe/Cargo.toml
similarity index 60%
rename from core/consensus/babe/primitives/Cargo.toml
rename to primitives/consensus/babe/Cargo.toml
index 79e817d081028..2ee36bd6c39f7 100644
--- a/core/consensus/babe/primitives/Cargo.toml
+++ b/primitives/consensus/babe/Cargo.toml
@@ -6,22 +6,22 @@ description = "Primitives for BABE consensus"
 edition = "2018"
 
 [dependencies]
-substrate-client = { path = "../../../client", default-features = false }
-rstd = { package = "sr-std", path = "../../../sr-std", default-features = false }
-sr-primitives = {  path = "../../../sr-primitives", default-features = false }
-app-crypto = { package = "substrate-application-crypto", path = "../../../application-crypto", default-features = false }
-slots = { package = "substrate-consensus-slots", path = "../../slots", optional = true }
-schnorrkel = { version = "0.8.5", features = ["preaudit_deprecated"], optional = true }
+app-crypto = { package = "substrate-application-crypto", path = "../../application-crypto", default-features = false }
 codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false }
+rstd = { package = "sr-std", path = "../../sr-std", default-features = false }
+schnorrkel = { version = "0.8.5", features = ["preaudit_deprecated"], optional = true }
+slots = { package = "substrate-consensus-slots", path = "../../../client/consensus/slots", optional = true }
+sr-api = { path = "../../sr-api", default-features = false }
+sr-primitives = {  path = "../../sr-primitives", default-features = false }
 
 [features]
 default = ["std"]
 std = [
-	"rstd/std",
-	"sr-primitives/std",
-	"substrate-client/std",
+	"app-crypto/std",
 	"codec/std",
+	"rstd/std",
 	"schnorrkel",
 	"slots",
-	"app-crypto/std",
+	"sr-api/std",
+	"sr-primitives/std",
 ]
diff --git a/core/consensus/babe/primitives/src/digest.rs b/primitives/consensus/babe/src/digest.rs
similarity index 100%
rename from core/consensus/babe/primitives/src/digest.rs
rename to primitives/consensus/babe/src/digest.rs
diff --git a/core/consensus/babe/primitives/src/lib.rs b/primitives/consensus/babe/src/lib.rs
similarity index 98%
rename from core/consensus/babe/primitives/src/lib.rs
rename to primitives/consensus/babe/src/lib.rs
index c464e797d8d58..6ebd2969613b3 100644
--- a/core/consensus/babe/primitives/src/lib.rs
+++ b/primitives/consensus/babe/src/lib.rs
@@ -24,7 +24,6 @@ mod digest;
 use codec::{Encode, Decode};
 use rstd::vec::Vec;
 use sr_primitives::{ConsensusEngineId, RuntimeDebug};
-use substrate_client::decl_runtime_apis;
 
 #[cfg(feature = "std")]
 pub use digest::{BabePreDigest, CompatibleDigestItem};
@@ -165,7 +164,7 @@ impl slots::SlotData for BabeConfiguration {
 	const SLOT_KEY: &'static [u8] = b"babe_configuration";
 }
 
-decl_runtime_apis! {
+sr_api::decl_runtime_apis! {
 	/// API necessary for block authorship with BABE.
 	pub trait BabeApi {
 		/// Return the configuration for BABE. Currently,
diff --git a/core/consensus/common/Cargo.toml b/primitives/consensus/common/Cargo.toml
similarity index 81%
rename from core/consensus/common/Cargo.toml
rename to primitives/consensus/common/Cargo.toml
index 73a74c7ced30d..6901bf09b70dc 100644
--- a/core/consensus/common/Cargo.toml
+++ b/primitives/consensus/common/Cargo.toml
@@ -7,9 +7,9 @@ edition = "2018"
 
 [dependencies]
 derive_more = "0.15.0"
-libp2p = { version = "0.12.0", default-features = false }
+libp2p = { version = "0.13.0", default-features = false }
 log = "0.4.8"
-primitives = { package = "substrate-primitives", path= "../../primitives" }
+primitives = { package = "substrate-primitives", path= "../../core" }
 inherents = { package = "substrate-inherents", path = "../../inherents" }
 futures-preview = "0.3.0-alpha.19"
 futures-timer = "0.4.0"
@@ -20,7 +20,7 @@ codec = { package = "parity-scale-codec", version = "1.0.0", features = ["derive
 parking_lot = "0.9.0"
 
 [dev-dependencies]
-test-client = { package = "substrate-test-runtime-client", path = "../../test-runtime/client" }
+test-client = { package = "substrate-test-runtime-client", path = "../../../test/utils/runtime/client" }
 
 [features]
 default = []
diff --git a/core/consensus/common/src/block_import.rs b/primitives/consensus/common/src/block_import.rs
similarity index 95%
rename from core/consensus/common/src/block_import.rs
rename to primitives/consensus/common/src/block_import.rs
index 4342ee38df10a..79d9be7b84e23 100644
--- a/core/consensus/common/src/block_import.rs
+++ b/primitives/consensus/common/src/block_import.rs
@@ -35,11 +35,15 @@ pub enum ImportResult {
 	KnownBad,
 	/// Block parent is not in the chain.
 	UnknownParent,
+	/// Parent state is missing.
+	MissingState,
 }
 
 /// Auxiliary data associated with an imported block result.
 #[derive(Debug, Default, PartialEq, Eq)]
 pub struct ImportedAux {
+	/// Only the header has been imported. Block body verification was skipped.
+	pub header_only: bool,
 	/// Clear all pending justification requests.
 	pub clear_justification_requests: bool,
 	/// Request a justification for the given block.
@@ -91,6 +95,7 @@ pub enum ForkChoiceStrategy {
 }
 
 /// Data required to check validity of a Block.
+#[derive(Debug, PartialEq, Eq, Clone)]
 pub struct BlockCheckParams {
 	/// Hash of the block that we verify.
 	pub hash: Block::Hash,
@@ -98,6 +103,8 @@ pub struct BlockCheckParams {
 	pub number: NumberFor,
 	/// Parent hash of the block that we verify.
 	pub parent_hash: Block::Hash,
+	/// Allow importing the block skipping state verification if parent state is missing.
+	pub allow_missing_state: bool,
 }
 
 /// Data required to import a Block.
@@ -133,6 +140,8 @@ pub struct BlockImportParams {
 	/// Fork choice strategy of this import. This should only be set by a
 	/// synchronous import, otherwise it may race against other imports.
 	pub fork_choice: ForkChoiceStrategy,
+	/// Allow importing the block skipping state verification if parent state is missing.
+	pub allow_missing_state: bool,
 }
 
 impl BlockImportParams {
diff --git a/core/consensus/common/src/block_validation.rs b/primitives/consensus/common/src/block_validation.rs
similarity index 100%
rename from core/consensus/common/src/block_validation.rs
rename to primitives/consensus/common/src/block_validation.rs
diff --git a/core/consensus/common/src/error.rs b/primitives/consensus/common/src/error.rs
similarity index 98%
rename from core/consensus/common/src/error.rs
rename to primitives/consensus/common/src/error.rs
index cb57bb915eb2d..16781b04ff27f 100644
--- a/core/consensus/common/src/error.rs
+++ b/primitives/consensus/common/src/error.rs
@@ -36,7 +36,7 @@ pub enum Error {
 	FaultyTimer(std::io::Error),
 	/// Error while working with inherent data.
 	#[display(fmt="InherentData error: {}", _0)]
-	InherentData(String),
+	InherentData(inherents::Error),
 	/// Unable to propose a block.
 	#[display(fmt="Unable to create block proposal.")]
 	CannotPropose,
diff --git a/core/consensus/common/src/evaluation.rs b/primitives/consensus/common/src/evaluation.rs
similarity index 100%
rename from core/consensus/common/src/evaluation.rs
rename to primitives/consensus/common/src/evaluation.rs
diff --git a/core/consensus/common/src/import_queue.rs b/primitives/consensus/common/src/import_queue.rs
similarity index 93%
rename from core/consensus/common/src/import_queue.rs
rename to primitives/consensus/common/src/import_queue.rs
index dc1678fcf189d..4bc986e1a3ee6 100644
--- a/core/consensus/common/src/import_queue.rs
+++ b/primitives/consensus/common/src/import_queue.rs
@@ -63,6 +63,8 @@ pub struct IncomingBlock {
 	pub justification: Option,
 	/// The peer, we received this from
 	pub origin: Option,
+	/// Allow importing the block skipping state verification if parent state is missing.
+	pub allow_missing_state: bool,
 }
 
 /// Type of keys in the blockchain cache that consensus module could use for its needs.
@@ -161,6 +163,8 @@ pub enum BlockImportError {
 	VerificationFailed(Option, String),
 	/// Block is known to be Bad
 	BadBlock(Option),
+	/// Parent state is missing.
+	MissingState,
 	/// Block has an unknown parent
 	UnknownParent,
 	/// Block import has been cancelled. This can happen if the parent block fails to be imported.
@@ -203,6 +207,10 @@ pub fn import_single_block>(
 				Ok(BlockImportResult::ImportedKnown(number))
 			},
 			Ok(ImportResult::Imported(aux)) => Ok(BlockImportResult::ImportedUnknown(number, aux, peer.clone())),
+			Ok(ImportResult::MissingState) => {
+				debug!(target: "sync", "Parent state is missing for {}: {:?}, parent: {:?}", number, hash, parent_hash);
+				Err(BlockImportError::MissingState)
+			},
 			Ok(ImportResult::UnknownParent) => {
 				debug!(target: "sync", "Block with unknown parent {}: {:?}, parent: {:?}", number, hash, parent_hash);
 				Err(BlockImportError::UnknownParent)
@@ -217,12 +225,17 @@ pub fn import_single_block>(
 			}
 		}
 	};
-	match import_error(import_handle.check_block(BlockCheckParams { hash, number, parent_hash }))? {
+	match import_error(import_handle.check_block(BlockCheckParams {
+		hash,
+		number,
+		parent_hash,
+		allow_missing_state: block.allow_missing_state,
+	}))? {
 		BlockImportResult::ImportedUnknown { .. } => (),
 		r => return Ok(r), // Any other successful result means that the block is already imported.
 	}
 
-	let (import_block, maybe_keys) = verifier.verify(block_origin, header, justification, block.body)
+	let (mut import_block, maybe_keys) = verifier.verify(block_origin, header, justification, block.body)
 		.map_err(|msg| {
 			if let Some(ref peer) = peer {
 				trace!(target: "sync", "Verifying {}({}) from {} failed: {}", number, hash, peer, msg);
@@ -236,6 +249,7 @@ pub fn import_single_block>(
 	if let Some(keys) = maybe_keys {
 		cache.extend(keys.into_iter());
 	}
+	import_block.allow_missing_state = block.allow_missing_state;
 
 	import_error(import_handle.import_block(import_block, cache))
 }
diff --git a/core/consensus/common/src/import_queue/basic_queue.rs b/primitives/consensus/common/src/import_queue/basic_queue.rs
similarity index 100%
rename from core/consensus/common/src/import_queue/basic_queue.rs
rename to primitives/consensus/common/src/import_queue/basic_queue.rs
diff --git a/core/consensus/common/src/import_queue/buffered_link.rs b/primitives/consensus/common/src/import_queue/buffered_link.rs
similarity index 100%
rename from core/consensus/common/src/import_queue/buffered_link.rs
rename to primitives/consensus/common/src/import_queue/buffered_link.rs
diff --git a/core/consensus/common/src/lib.rs b/primitives/consensus/common/src/lib.rs
similarity index 100%
rename from core/consensus/common/src/lib.rs
rename to primitives/consensus/common/src/lib.rs
diff --git a/core/consensus/common/src/offline_tracker.rs b/primitives/consensus/common/src/offline_tracker.rs
similarity index 100%
rename from core/consensus/common/src/offline_tracker.rs
rename to primitives/consensus/common/src/offline_tracker.rs
diff --git a/core/consensus/common/src/select_chain.rs b/primitives/consensus/common/src/select_chain.rs
similarity index 100%
rename from core/consensus/common/src/select_chain.rs
rename to primitives/consensus/common/src/select_chain.rs
diff --git a/core/consensus/pow/primitives/Cargo.toml b/primitives/consensus/pow/Cargo.toml
similarity index 53%
rename from core/consensus/pow/primitives/Cargo.toml
rename to primitives/consensus/pow/Cargo.toml
index 021e4c80a754c..99aa00da8ee8e 100644
--- a/core/consensus/pow/primitives/Cargo.toml
+++ b/primitives/consensus/pow/Cargo.toml
@@ -6,17 +6,17 @@ description = "Primitives for Aura consensus"
 edition = "2018"
 
 [dependencies]
-substrate-client = { path = "../../../client", default-features = false }
-rstd = { package = "sr-std", path = "../../../sr-std", default-features = false }
-sr-primitives = { path = "../../../sr-primitives", default-features = false }
-primitives = { package = "substrate-primitives", path = "../../../primitives", default-features = false }
+sr-api = { path = "../../sr-api", default-features = false }
+rstd = { package = "sr-std", path = "../../sr-std", default-features = false }
+sr-primitives = { path = "../../sr-primitives", default-features = false }
+primitives = { package = "substrate-primitives", path = "../../core", default-features = false }
 codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false, features = ["derive"] }
 
 [features]
 default = ["std"]
 std = [
 	"rstd/std",
-	"substrate-client/std",
+	"sr-api/std",
 	"sr-primitives/std",
 	"primitives/std",
 	"codec/std",
diff --git a/core/consensus/pow/primitives/src/lib.rs b/primitives/consensus/pow/src/lib.rs
similarity index 97%
rename from core/consensus/pow/primitives/src/lib.rs
rename to primitives/consensus/pow/src/lib.rs
index 2079b1cbe7e88..400ed0594a026 100644
--- a/core/consensus/pow/primitives/src/lib.rs
+++ b/primitives/consensus/pow/src/lib.rs
@@ -21,7 +21,6 @@
 use rstd::vec::Vec;
 use sr_primitives::ConsensusEngineId;
 use codec::Decode;
-use substrate_client::decl_runtime_apis;
 
 /// The `ConsensusEngineId` of PoW.
 pub const POW_ENGINE_ID: ConsensusEngineId = [b'p', b'o', b'w', b'_'];
@@ -48,7 +47,7 @@ impl TotalDifficulty for u128 {
 	}
 }
 
-decl_runtime_apis! {
+sr_api::decl_runtime_apis! {
 	/// API necessary for timestamp-based difficulty adjustment algorithms.
 	pub trait TimestampApi {
 		/// Return the timestamp in the current block.
diff --git a/core/primitives/Cargo.toml b/primitives/core/Cargo.toml
similarity index 62%
rename from core/primitives/Cargo.toml
rename to primitives/core/Cargo.toml
index 30c1aab29fd7d..f9888928dc13d 100644
--- a/core/primitives/Cargo.toml
+++ b/primitives/core/Cargo.toml
@@ -10,32 +10,33 @@ codec = { package = "parity-scale-codec", version = "1.0.0", default-features =
 rustc-hex = { version = "2.0.1", default-features = false }
 log = { version = "0.4.8", default-features = false }
 serde = { version = "1.0.101", optional = true, features = ["derive"] }
-twox-hash = { version = "1.5.0", optional = true }
+twox-hash = { version = "1.5.0", default-features = false, optional = true }
 byteorder = { version = "1.3.2", default-features = false }
-primitive-types = { version = "0.5.1", default-features = false, features = ["codec"] }
+primitive-types = { version = "0.6", default-features = false, features = ["codec"] }
 impl-serde = { version = "0.2.3", optional = true }
-wasmi = { version = "0.5.1", optional = true }
+wasmi = { version = "0.6.2", optional = true }
 hash-db = { version = "0.15.2", default-features = false }
 hash256-std-hasher = { version = "0.15.2", default-features = false }
-ed25519-dalek = { version = "0.9.1", optional = true }
+ed25519-dalek = { version = "0.9.1",  default-features = false, features = ["u64_backend"], optional = true }
 base58 = { version = "0.1.0", optional = true }
-blake2-rfc = { version = "0.2.18", optional = true }
-schnorrkel = { version = "0.8.5", features = ["preaudit_deprecated"], optional = true }
+blake2-rfc = { version = "0.2.18", default-features = false, optional = true }
+schnorrkel = { version = "0.8.5", features = ["preaudit_deprecated"], default-features = false, optional = true }
 rand = { version = "0.7.2", optional = true }
-sha2 = { version = "0.8.0", optional = true }
+sha2 = { version = "0.8.0", default-features = false, optional = true }
 substrate-bip39 = { version = "0.3.1", optional = true }
 tiny-bip39 = { version = "0.6.2", optional = true }
-hex = { version = "0.3.2", optional = true }
+hex = { version = "0.4", default-features = false, optional = true }
 regex = { version = "1.3.1", optional = true }
 num-traits = { version = "0.2.8", default-features = false }
-zeroize = "0.10.1"
-lazy_static = { version = "1.4.0", optional = true }
+zeroize = { version = "0.10.1", default-features = false }
+lazy_static = { version = "1.4.0", default-features = false, optional = true }
 parking_lot = { version = "0.9.0", optional = true }
-libsecp256k1 = { version = "0.3.0", optional = true }
-tiny-keccak = { version = "1.5.0", optional = true }
+libsecp256k1 = { version = "0.3.0", default-features = false, optional = true }
+tiny-keccak = { version = "2.0.1", features = ["keccak"], optional = true }
 substrate-debug-derive = { version = "2.0.0", path = "./debug-derive" }
 externalities = { package = "substrate-externalities", path = "../externalities", optional = true }
 primitives-storage = { package = "substrate-primitives-storage", path = "storage", default-features = false }
+runtime-interface = { package = "substrate-runtime-interface", path = "../runtime-interface", default-features = false }
 
 [dev-dependencies]
 substrate-serializer = { path = "../serializer" }
@@ -54,6 +55,7 @@ bench = false
 [features]
 default = ["std"]
 std = [
+	"full_crypto",
 	"log/std",
 	"wasmi",
 	"lazy_static",
@@ -70,18 +72,18 @@ std = [
 	"rstd/std",
 	"serde",
 	"rustc-hex/std",
-	"twox-hash",
-	"blake2-rfc",
-	"ed25519-dalek",
-	"hex",
+	"twox-hash/std",
+	"blake2-rfc/std",
+	"ed25519-dalek/std",
+	"hex/std",
 	"base58",
 	"substrate-bip39",
 	"tiny-bip39",
 	"serde",
 	"byteorder/std",
 	"rand",
-	"sha2",
-	"schnorrkel",
+	"sha2/std",
+	"schnorrkel/std",
 	"regex",
 	"num-traits/std",
 	"libsecp256k1",
@@ -89,4 +91,20 @@ std = [
 	"substrate-debug-derive/std",
 	"externalities",
 	"primitives-storage/std",
+	"runtime-interface/std",
+]
+
+# This feature enables all crypto primitives for `no_std` builds like microcontrollers
+# or Intel SGX.
+# For the regular wasm runtime builds this should not be used.
+full_crypto = [
+	"ed25519-dalek",
+	"blake2-rfc",
+	"tiny-keccak",
+	"schnorrkel",
+	"libsecp256k1",
+	"hex",
+	"sha2",
+	"twox-hash",
+	"runtime-interface/disable_target_static_assertions",
 ]
diff --git a/core/primitives/benches/bench.rs b/primitives/core/benches/bench.rs
similarity index 100%
rename from core/primitives/benches/bench.rs
rename to primitives/core/benches/bench.rs
diff --git a/core/primitives/debug-derive/Cargo.toml b/primitives/core/debug-derive/Cargo.toml
similarity index 94%
rename from core/primitives/debug-derive/Cargo.toml
rename to primitives/core/debug-derive/Cargo.toml
index 9c7b7aa1bad49..a7bf90695ab93 100644
--- a/core/primitives/debug-derive/Cargo.toml
+++ b/primitives/core/debug-derive/Cargo.toml
@@ -9,7 +9,7 @@ proc-macro = true
 
 [dependencies]
 quote = "1.0.2"
-syn = "1.0.5"
+syn = "1.0.7"
 proc-macro2 = "1.0"
 
 [features]
diff --git a/core/primitives/debug-derive/src/impls.rs b/primitives/core/debug-derive/src/impls.rs
similarity index 100%
rename from core/primitives/debug-derive/src/impls.rs
rename to primitives/core/debug-derive/src/impls.rs
diff --git a/core/primitives/debug-derive/src/lib.rs b/primitives/core/debug-derive/src/lib.rs
similarity index 100%
rename from core/primitives/debug-derive/src/lib.rs
rename to primitives/core/debug-derive/src/lib.rs
diff --git a/core/primitives/debug-derive/tests/tests.rs b/primitives/core/debug-derive/tests/tests.rs
similarity index 100%
rename from core/primitives/debug-derive/tests/tests.rs
rename to primitives/core/debug-derive/tests/tests.rs
diff --git a/core/primitives/src/changes_trie.rs b/primitives/core/src/changes_trie.rs
similarity index 100%
rename from core/primitives/src/changes_trie.rs
rename to primitives/core/src/changes_trie.rs
diff --git a/core/primitives/src/crypto.rs b/primitives/core/src/crypto.rs
similarity index 96%
rename from core/primitives/src/crypto.rs
rename to primitives/core/src/crypto.rs
index 93f499b230d5a..f5484efed7a38 100644
--- a/core/primitives/src/crypto.rs
+++ b/primitives/core/src/crypto.rs
@@ -18,6 +18,7 @@
 //! Cryptographic utilities.
 // end::description[]
 
+use rstd::{vec::Vec, hash::Hash};
 #[cfg(feature = "std")]
 use rstd::convert::TryInto;
 use rstd::convert::TryFrom;
@@ -30,11 +31,11 @@ use codec::{Encode, Decode};
 use regex::Regex;
 #[cfg(feature = "std")]
 use base58::{FromBase58, ToBase58};
-#[cfg(feature = "std")]
-use std::hash::Hash;
+
 use zeroize::Zeroize;
 #[doc(hidden)]
 pub use rstd::ops::Deref;
+use runtime_interface::pass_by::PassByInner;
 
 /// The root phrase for our publicly known keys.
 pub const DEV_PHRASE: &str = "bottom drive obey lake curtain smoke basket hold race lonely fit walk";
@@ -48,7 +49,7 @@ pub enum Infallible {}
 
 /// The length of the junction identifier. Note that this is also referred to as the
 /// `CHAIN_CODE_LENGTH` in the context of Schnorrkel.
-#[cfg(feature = "std")]
+#[cfg(feature = "full_crypto")]
 pub const JUNCTION_ID_LEN: usize = 32;
 
 /// Similar to `From`, except that the onus is on the part of the caller to ensure
@@ -120,7 +121,7 @@ impl Drop for Protected {
 
 /// An error with the interpretation of a secret.
 #[derive(Debug, Clone, PartialEq, Eq)]
-#[cfg(feature = "std")]
+#[cfg(feature = "full_crypto")]
 pub enum SecretStringError {
 	/// The overall format was invalid (e.g. the seed phrase contained symbols).
 	InvalidFormat,
@@ -140,7 +141,7 @@ pub enum SecretStringError {
 /// a new secret key from an existing secret key and, in the case of `SoftRaw` and `SoftIndex`
 /// a new public key from an existing public key.
 #[derive(Copy, Clone, Eq, PartialEq, Hash, Debug, Encode, Decode)]
-#[cfg(feature = "std")]
+#[cfg(feature = "full_crypto")]
 pub enum DeriveJunction {
 	/// Soft (vanilla) derivation. Public keys have a correspondent derivation.
 	Soft([u8; JUNCTION_ID_LEN]),
@@ -148,7 +149,7 @@ pub enum DeriveJunction {
 	Hard([u8; JUNCTION_ID_LEN]),
 }
 
-#[cfg(feature = "std")]
+#[cfg(feature = "full_crypto")]
 impl DeriveJunction {
 	/// Consume self to return a soft derive junction with the same chain code.
 	pub fn soften(self) -> Self { DeriveJunction::Soft(self.unwrap_inner()) }
@@ -209,7 +210,7 @@ impl DeriveJunction {
 	}
 }
 
-#[cfg(feature = "std")]
+#[cfg(feature = "full_crypto")]
 impl> From for DeriveJunction {
 	fn from(j: T) -> DeriveJunction {
 		let j = j.as_ref();
@@ -236,7 +237,7 @@ impl> From for DeriveJunction {
 }
 
 /// An error type for SS58 decoding.
-#[cfg(feature = "std")]
+#[cfg(feature = "full_crypto")]
 #[derive(Clone, Copy, Eq, PartialEq, Debug)]
 pub enum PublicError {
 	/// Bad alphabet.
@@ -254,9 +255,10 @@ pub enum PublicError {
 }
 
 /// Key that can be encoded to/from SS58.
-#[cfg(feature = "std")]
+#[cfg(feature = "full_crypto")]
 pub trait Ss58Codec: Sized + AsMut<[u8]> + AsRef<[u8]> + Default {
 	/// Some if the string is a properly encoded SS58Check address.
+	#[cfg(feature = "std")]
 	fn from_ss58check(s: &str) -> Result {
 		Self::from_ss58check_with_version(s)
 			.and_then(|(r, v)| match v {
@@ -269,6 +271,7 @@ pub trait Ss58Codec: Sized + AsMut<[u8]> + AsRef<[u8]> + Default {
 			})
 	}
 	/// Some if the string is a properly encoded SS58Check address.
+	#[cfg(feature = "std")]
 	fn from_ss58check_with_version(s: &str) -> Result<(Self, Ss58AddressFormat), PublicError> {
 		let mut res = Self::default();
 		let len = res.as_mut().len();
@@ -288,6 +291,7 @@ pub trait Ss58Codec: Sized + AsMut<[u8]> + AsRef<[u8]> + Default {
 	}
 	/// Some if the string is a properly encoded SS58Check address, optionally with
 	/// a derivation path following.
+	#[cfg(feature = "std")]
 	fn from_string(s: &str) -> Result {
 		Self::from_string_with_version(s)
 			.and_then(|(r, v)| match v {
@@ -301,6 +305,8 @@ pub trait Ss58Codec: Sized + AsMut<[u8]> + AsRef<[u8]> + Default {
 	}
 
 	/// Return the ss58-check string for this key.
+
+	#[cfg(feature = "std")]
 	fn to_ss58check_with_version(&self, version: Ss58AddressFormat) -> String {
 		let mut v = vec![version.into()];
 		v.extend(self.as_ref());
@@ -309,9 +315,11 @@ pub trait Ss58Codec: Sized + AsMut<[u8]> + AsRef<[u8]> + Default {
 		v.to_base58()
 	}
 	/// Return the ss58-check string for this key.
+	#[cfg(feature = "std")]
 	fn to_ss58check(&self) -> String { self.to_ss58check_with_version(*DEFAULT_VERSION.lock()) }
 	/// Some if the string is a properly encoded SS58Check address, optionally with
 	/// a derivation path following.
+	#[cfg(feature = "std")]
 	fn from_string_with_version(s: &str) -> Result<(Self, Ss58AddressFormat), PublicError> {
 		Self::from_ss58check_with_version(s)
 	}
@@ -346,7 +354,7 @@ lazy_static::lazy_static! {
 }
 
 /// A known address (sub)format/network ID for SS58.
-#[cfg(feature = "std")]
+#[cfg(feature = "full_crypto")]
 #[derive(Copy, Clone, PartialEq, Eq)]
 pub enum Ss58AddressFormat {
 	/// Any Substrate network, direct checksum, standard account (*25519).
@@ -361,7 +369,7 @@ pub enum Ss58AddressFormat {
 	Custom(u8),
 }
 
-#[cfg(feature = "std")]
+#[cfg(feature = "full_crypto")]
 impl From for u8 {
 	fn from(x: Ss58AddressFormat) -> u8 {
 		match x {
@@ -374,7 +382,7 @@ impl From for u8 {
 	}
 }
 
-#[cfg(feature = "std")]
+#[cfg(feature = "full_crypto")]
 impl TryFrom for Ss58AddressFormat {
 	type Error = ();
 	fn try_from(x: u8) -> Result {
@@ -388,7 +396,7 @@ impl TryFrom for Ss58AddressFormat {
 	}
 }
 
-#[cfg(feature = "std")]
+#[cfg(feature = "full_crypto")]
 impl<'a> TryFrom<&'a str> for Ss58AddressFormat {
 	type Error = ();
 	fn try_from(x: &'a str) -> Result {
@@ -640,7 +648,9 @@ mod dummy {
 		type Seed = Dummy;
 		type Signature = Dummy;
 		type DeriveError = ();
+		#[cfg(feature = "std")]
 		fn generate_with_phrase(_: Option<&str>) -> (Self, String, Self::Seed) { Default::default() }
+		#[cfg(feature = "std")]
 		fn from_phrase(_: &str, _: Option<&str>)
 			-> Result<(Self, Self::Seed), SecretStringError>
 		{
@@ -662,7 +672,7 @@ mod dummy {
 /// Trait suitable for typical cryptographic PKI key pair type.
 ///
 /// For now it just specifies how to create a key from a phrase and derivation path.
-#[cfg(feature = "std")]
+#[cfg(feature = "full_crypto")]
 pub trait Pair: CryptoType + Sized + Clone + Send + Sync + 'static {
 	/// The type which is used to encode a public key.
 	type Public: Public + Hash;
@@ -682,6 +692,7 @@ pub trait Pair: CryptoType + Sized + Clone + Send + Sync + 'static {
 	///
 	/// This is only for ephemeral keys really, since you won't have access to the secret key
 	/// for storage. If you want a persistent key pair, use `generate_with_phrase` instead.
+	#[cfg(feature = "std")]
 	fn generate() -> (Self, Self::Seed) {
 		let mut seed = Self::Seed::default();
 		OsRng.fill_bytes(seed.as_mut());
@@ -694,9 +705,11 @@ pub trait Pair: CryptoType + Sized + Clone + Send + Sync + 'static {
 	///
 	/// This is generally slower than `generate()`, so prefer that unless you need to persist
 	/// the key from the current session.
+	#[cfg(feature = "std")]
 	fn generate_with_phrase(password: Option<&str>) -> (Self, String, Self::Seed);
 
 	/// Returns the KeyPair from the English BIP39 seed `phrase`, or `None` if it's invalid.
+	#[cfg(feature = "std")]
 	fn from_phrase(phrase: &str, password: Option<&str>) -> Result<(Self, Self::Seed), SecretStringError>;
 
 	/// Derive a child key from a series of given junctions.
@@ -758,7 +771,10 @@ pub trait Pair: CryptoType + Sized + Clone + Send + Sync + 'static {
 	/// be equivalent to no password at all.
 	///
 	/// `None` is returned if no matches are found.
-	fn from_string_with_seed(s: &str, password_override: Option<&str>) -> Result<(Self, Option), SecretStringError> {
+	#[cfg(feature = "std")]
+	fn from_string_with_seed(s: &str, password_override: Option<&str>) 
+		-> Result<(Self, Option), SecretStringError> 
+	{
 		let re = Regex::new(r"^(?P[\d\w ]+)?(?P(//?[^/]+)*)(///(?P.*))?$")
 			.expect("constructed from known-good static value; qed");
 		let cap = re.captures(s).ok_or(SecretStringError::InvalidFormat)?;
@@ -793,6 +809,7 @@ pub trait Pair: CryptoType + Sized + Clone + Send + Sync + 'static {
 	/// Interprets the string `s` in order to generate a key pair.
 	///
 	/// See [`from_string_with_seed`](Self::from_string_with_seed) for more extensive documentation.
+	#[cfg(feature = "std")]
 	fn from_string(s: &str, password_override: Option<&str>) -> Result {
 		Self::from_string_with_seed(s, password_override).map(|x| x.0)
 	}
@@ -839,7 +856,7 @@ impl UncheckedFrom for Outer where
 /// Type which has a particular kind of crypto associated with it.
 pub trait CryptoType {
 	/// The pair key type of this crypto.
-	#[cfg(feature="std")]
+	#[cfg(feature = "full_crypto")]
 	type Pair: Pair;
 }
 
@@ -850,8 +867,10 @@ pub trait CryptoType {
 ///
 /// Values whose first character is `_` are reserved for private use and won't conflict with any
 /// public modules.
-#[derive(Copy, Clone, Default, PartialEq, Eq, PartialOrd, Ord, Hash, Encode, Decode)]
-#[derive(crate::RuntimeDebug)]
+#[derive(
+	Copy, Clone, Default, PartialEq, Eq, PartialOrd, Ord, Hash, Encode, Decode, PassByInner,
+	crate::RuntimeDebug
+)]
 pub struct KeyTypeId(pub [u8; 4]);
 
 impl From for KeyTypeId {
@@ -897,6 +916,8 @@ pub mod key_types {
 	pub const AURA: KeyTypeId = KeyTypeId(*b"aura");
 	/// Key type for ImOnline module, built-in.
 	pub const IM_ONLINE: KeyTypeId = KeyTypeId(*b"imon");
+	/// Key type for AuthorityDiscovery module, built-in.
+	pub const AUTHORITY_DISCOVERY: KeyTypeId = KeyTypeId(*b"audi");
 	/// A key type ID useful for tests.
 	pub const DUMMY: KeyTypeId = KeyTypeId(*b"dumy");
 }
diff --git a/core/primitives/src/ecdsa.rs b/primitives/core/src/ecdsa.rs
similarity index 94%
rename from core/primitives/src/ecdsa.rs
rename to primitives/core/src/ecdsa.rs
index abff460c91034..f1d4d2446aa13 100644
--- a/core/primitives/src/ecdsa.rs
+++ b/primitives/core/src/ecdsa.rs
@@ -18,27 +18,32 @@
 //! Simple ECDSA API.
 // end::description[]
 
+#[cfg(feature = "full_crypto")]
+use rstd::vec::Vec;
+
 use rstd::cmp::Ordering;
 use codec::{Encode, Decode};
 
-#[cfg(feature = "std")]
-use std::convert::{TryFrom, TryInto};
+#[cfg(feature = "full_crypto")]
+use core::convert::{TryFrom, TryInto};
 #[cfg(feature = "std")]
 use substrate_bip39::seed_from_entropy;
 #[cfg(feature = "std")]
 use bip39::{Mnemonic, Language, MnemonicType};
+#[cfg(feature = "full_crypto")]
+use crate::{hashing::blake2_256, crypto::{Pair as TraitPair, DeriveJunction, SecretStringError}};
 #[cfg(feature = "std")]
-use crate::{hashing::blake2_256, crypto::{Pair as TraitPair, DeriveJunction, SecretStringError, Ss58Codec}};
+use crate::crypto::Ss58Codec;
 #[cfg(feature = "std")]
 use serde::{de, Serializer, Serialize, Deserializer, Deserialize};
 use crate::crypto::{Public as TraitPublic, UncheckedFrom, CryptoType, Derive};
-#[cfg(feature = "std")]
+#[cfg(feature = "full_crypto")]
 use secp256k1::{PublicKey, SecretKey};
 
 /// A secret seed (which is bytewise essentially equivalent to a SecretKey).
 ///
 /// We need it as a different type because `Seed` is expected to be AsRef<[u8]>.
-#[cfg(feature = "std")]
+#[cfg(feature = "full_crypto")]
 type Seed = [u8; 32];
 
 /// The ECDSA 33-byte compressed public key.
@@ -72,7 +77,7 @@ impl Default for Public {
 }
 
 /// A key pair.
-#[cfg(feature = "std")]
+#[cfg(feature = "full_crypto")]
 #[derive(Clone)]
 pub struct Pair {
 	public: PublicKey,
@@ -117,7 +122,7 @@ impl From for [u8; 33] {
 	}
 }
 
-#[cfg(feature = "std")]
+#[cfg(feature = "full_crypto")]
 impl From for Public {
 	fn from(x: Pair) -> Self {
 		x.public()
@@ -160,9 +165,9 @@ impl<'de> Deserialize<'de> for Public {
 	}
 }
 
-#[cfg(feature = "std")]
-impl std::hash::Hash for Public {
-	fn hash(&self, state: &mut H) {
+#[cfg(feature = "full_crypto")]
+impl rstd::hash::Hash for Public {
+	fn hash(&self, state: &mut H) {
 		self.0.hash(state);
 	}
 }
@@ -238,10 +243,10 @@ impl std::fmt::Debug for Signature {
 	}
 }
 
-#[cfg(feature = "std")]
-impl std::hash::Hash for Signature {
-	fn hash(&self, state: &mut H) {
-		std::hash::Hash::hash(&self.0[..], state);
+#[cfg(feature = "full_crypto")]
+impl rstd::hash::Hash for Signature {
+	fn hash(&self, state: &mut H) {
+		rstd::hash::Hash::hash(&self.0[..], state);
 	}
 }
 
@@ -255,7 +260,7 @@ impl Signature {
 	}
 
 	/// Recover the public key from this signature and a message.
-	#[cfg(feature = "std")]
+	#[cfg(feature = "full_crypto")]
 	pub fn recover>(&self, message: M) -> Option {
 		let message = secp256k1::Message::parse(&blake2_256(message.as_ref()));
 		let sig: (_, _) = self.try_into().ok()?;
@@ -264,7 +269,7 @@ impl Signature {
 	}
 }
 
-#[cfg(feature = "std")]
+#[cfg(feature = "full_crypto")]
 impl From<(secp256k1::Signature, secp256k1::RecoveryId)> for Signature {
 	fn from(x: (secp256k1::Signature, secp256k1::RecoveryId)) -> Signature {
 		let mut r = Self::default();
@@ -274,7 +279,7 @@ impl From<(secp256k1::Signature, secp256k1::RecoveryId)> for Signature {
 	}
 }
 
-#[cfg(feature = "std")]
+#[cfg(feature = "full_crypto")]
 impl<'a> TryFrom<&'a Signature> for (secp256k1::Signature, secp256k1::RecoveryId) {
 	type Error = ();
 	fn try_from(x: &'a Signature) -> Result<(secp256k1::Signature, secp256k1::RecoveryId), Self::Error> {
@@ -329,7 +334,7 @@ impl TraitPublic for Public {
 impl Derive for Public {}
 
 /// Derive a single hard junction.
-#[cfg(feature = "std")]
+#[cfg(feature = "full_crypto")]
 fn derive_hard_junction(secret_seed: &Seed, cc: &[u8; 32]) -> Seed {
 	("Secp256k1HDKD", secret_seed, cc).using_encoded(|data| {
 		let mut res = [0u8; 32];
@@ -339,13 +344,13 @@ fn derive_hard_junction(secret_seed: &Seed, cc: &[u8; 32]) -> Seed {
 }
 
 /// An error when deriving a key.
-#[cfg(feature = "std")]
+#[cfg(feature = "full_crypto")]
 pub enum DeriveError {
 	/// A soft key was found in the path (and is unsupported).
 	SoftKeyInPath,
 }
 
-#[cfg(feature = "std")]
+#[cfg(feature = "full_crypto")]
 impl TraitPair for Pair {
 	type Public = Public;
 	type Seed = Seed;
@@ -355,6 +360,7 @@ impl TraitPair for Pair {
 	/// Generate new secure (random) key pair and provide the recovery phrase.
 	///
 	/// You can recover the same key later with `from_phrase`.
+	#[cfg(feature = "std")]
 	fn generate_with_phrase(password: Option<&str>) -> (Pair, String, Seed) {
 		let mnemonic = Mnemonic::new(MnemonicType::Words12, Language::English);
 		let phrase = mnemonic.phrase();
@@ -368,6 +374,7 @@ impl TraitPair for Pair {
 	}
 
 	/// Generate key pair from given recovery phrase and password.
+	#[cfg(feature = "std")]
 	fn from_phrase(phrase: &str, password: Option<&str>) -> Result<(Pair, Seed), SecretStringError> {
 		let big_seed = seed_from_entropy(
 			Mnemonic::from_phrase(phrase, Language::English)
@@ -454,7 +461,7 @@ impl TraitPair for Pair {
 	}
 }
 
-#[cfg(feature = "std")]
+#[cfg(feature = "full_crypto")]
 impl Pair {
 	/// Get the seed for this key.
 	pub fn seed(&self) -> Seed {
@@ -463,6 +470,7 @@ impl Pair {
 
 	/// Exactly as `from_string` except that if no matches are found then, the the first 32
 	/// characters are taken (padded with spaces as necessary) and used as the MiniSecretKey.
+	#[cfg(feature = "std")]
 	pub fn from_legacy_string(s: &str, password_override: Option<&str>) -> Pair {
 		Self::from_string(s, password_override).unwrap_or_else(|_| {
 			let mut padded_seed: Seed = [' ' as u8; 32];
@@ -474,16 +482,16 @@ impl Pair {
 }
 
 impl CryptoType for Public {
-	#[cfg(feature="std")]
+	#[cfg(feature="full_crypto")]
 	type Pair = Pair;
 }
 
 impl CryptoType for Signature {
-	#[cfg(feature="std")]
+	#[cfg(feature="full_crypto")]
 	type Pair = Pair;
 }
 
-#[cfg(feature = "std")]
+#[cfg(feature="full_crypto")]
 impl CryptoType for Pair {
 	type Pair = Pair;
 }
diff --git a/core/primitives/src/ed25519.rs b/primitives/core/src/ed25519.rs
similarity index 92%
rename from core/primitives/src/ed25519.rs
rename to primitives/core/src/ed25519.rs
index 8c3aa9f89dbdc..2b79f92d80cbf 100644
--- a/core/primitives/src/ed25519.rs
+++ b/primitives/core/src/ed25519.rs
@@ -18,38 +18,44 @@
 //! Simple Ed25519 API.
 // end::description[]
 
+#[cfg(feature = "full_crypto")]
+use rstd::vec::Vec;
 
 use crate::{hash::H256, hash::H512};
 use codec::{Encode, Decode};
 
-#[cfg(feature = "std")]
+#[cfg(feature = "full_crypto")]
 use blake2_rfc;
 #[cfg(feature = "std")]
 use substrate_bip39::seed_from_entropy;
 #[cfg(feature = "std")]
 use bip39::{Mnemonic, Language, MnemonicType};
+#[cfg(feature = "full_crypto")]
+use crate::crypto::{Pair as TraitPair, DeriveJunction, SecretStringError};
 #[cfg(feature = "std")]
-use crate::crypto::{Pair as TraitPair, DeriveJunction, SecretStringError, Ss58Codec};
+use crate::crypto::Ss58Codec;
 #[cfg(feature = "std")]
 use serde::{de, Serializer, Serialize, Deserializer, Deserialize};
 use crate::{crypto::{Public as TraitPublic, UncheckedFrom, CryptoType, Derive}};
+use runtime_interface::pass_by::PassByInner;
+use rstd::ops::Deref;
 
 /// A secret seed. It's not called a "secret key" because ring doesn't expose the secret keys
 /// of the key pair (yeah, dumb); as such we're forced to remember the seed manually if we
 /// will need it later (such as for HDKD).
-#[cfg(feature = "std")]
+#[cfg(feature = "full_crypto")]
 type Seed = [u8; 32];
 
 /// A public key.
-#[cfg_attr(feature = "std", derive(Hash))]
-#[derive(PartialEq, Eq, PartialOrd, Ord, Clone, Encode, Decode, Default)]
+#[cfg_attr(feature = "full_crypto", derive(Hash))]
+#[derive(PartialEq, Eq, PartialOrd, Ord, Clone, Copy, Encode, Decode, Default, PassByInner)]
 pub struct Public(pub [u8; 32]);
 
 /// A key pair.
-#[cfg(feature = "std")]
+#[cfg(feature = "full_crypto")]
 pub struct Pair(ed25519_dalek::Keypair);
 
-#[cfg(feature = "std")]
+#[cfg(feature = "full_crypto")]
 impl Clone for Pair {
 	fn clone(&self) -> Self {
 		Pair(ed25519_dalek::Keypair {
@@ -78,6 +84,14 @@ impl AsMut<[u8]> for Public {
 	}
 }
 
+impl Deref for Public {
+    type Target = [u8];
+
+    fn deref(&self) -> &Self::Target {
+        &self.0
+    }
+}
+
 impl rstd::convert::TryFrom<&[u8]> for Public {
 	type Error = ();
 
@@ -98,7 +112,7 @@ impl From for [u8; 32] {
 	}
 }
 
-#[cfg(feature = "std")]
+#[cfg(feature = "full_crypto")]
 impl From for Public {
 	fn from(x: Pair) -> Self {
 		x.public()
@@ -111,6 +125,15 @@ impl From for H256 {
 	}
 }
 
+#[cfg(feature = "std")]
+impl std::str::FromStr for Public {
+    type Err = crate::crypto::PublicError;
+
+    fn from_str(s: &str) -> Result {
+        Self::from_ss58check(s)
+    }
+}
+
 impl UncheckedFrom<[u8; 32]> for Public {
 	fn unchecked_from(x: [u8; 32]) -> Self {
 		Public::from_raw(x)
@@ -159,7 +182,7 @@ impl<'de> Deserialize<'de> for Public {
 }
 
 /// A signature (a 512-bit value).
-#[derive(Encode, Decode)]
+#[derive(Encode, Decode, PassByInner)]
 pub struct Signature(pub [u8; 64]);
 
 impl rstd::convert::TryFrom<&[u8]> for Signature {
@@ -240,10 +263,10 @@ impl rstd::fmt::Debug for Signature {
 	}
 }
 
-#[cfg(feature = "std")]
-impl std::hash::Hash for Signature {
-	fn hash(&self, state: &mut H) {
-		std::hash::Hash::hash(&self.0[..], state);
+#[cfg(feature = "full_crypto")]
+impl rstd::hash::Hash for Signature {
+	fn hash(&self, state: &mut H) {
+		rstd::hash::Hash::hash(&self.0[..], state);
 	}
 }
 
@@ -337,7 +360,7 @@ impl TraitPublic for Public {
 impl Derive for Public {}
 
 /// Derive a single hard junction.
-#[cfg(feature = "std")]
+#[cfg(feature = "full_crypto")]
 fn derive_hard_junction(secret_seed: &Seed, cc: &[u8; 32]) -> Seed {
 	("Ed25519HDKD", secret_seed, cc).using_encoded(|data| {
 		let mut res = [0u8; 32];
@@ -347,13 +370,13 @@ fn derive_hard_junction(secret_seed: &Seed, cc: &[u8; 32]) -> Seed {
 }
 
 /// An error when deriving a key.
-#[cfg(feature = "std")]
+#[cfg(feature = "full_crypto")]
 pub enum DeriveError {
 	/// A soft key was found in the path (and is unsupported).
 	SoftKeyInPath,
 }
 
-#[cfg(feature = "std")]
+#[cfg(feature = "full_crypto")]
 impl TraitPair for Pair {
 	type Public = Public;
 	type Seed = Seed;
@@ -363,6 +386,7 @@ impl TraitPair for Pair {
 	/// Generate new secure (random) key pair and provide the recovery phrase.
 	///
 	/// You can recover the same key later with `from_phrase`.
+	#[cfg(feature = "std")]
 	fn generate_with_phrase(password: Option<&str>) -> (Pair, String, Seed) {
 		let mnemonic = Mnemonic::new(MnemonicType::Words12, Language::English);
 		let phrase = mnemonic.phrase();
@@ -376,6 +400,7 @@ impl TraitPair for Pair {
 	}
 
 	/// Generate key pair from given recovery phrase and password.
+	#[cfg(feature = "std")]
 	fn from_phrase(phrase: &str, password: Option<&str>) -> Result<(Pair, Seed), SecretStringError> {
 		let big_seed = seed_from_entropy(
 			Mnemonic::from_phrase(phrase, Language::English)
@@ -466,7 +491,7 @@ impl TraitPair for Pair {
 	}
 }
 
-#[cfg(feature = "std")]
+#[cfg(feature = "full_crypto")]
 impl Pair {
 	/// Get the seed for this key.
 	pub fn seed(&self) -> &Seed {
@@ -475,6 +500,7 @@ impl Pair {
 
 	/// Exactly as `from_string` except that if no matches are found then, the the first 32
 	/// characters are taken (padded with spaces as necessary) and used as the MiniSecretKey.
+	#[cfg(feature = "std")]
 	pub fn from_legacy_string(s: &str, password_override: Option<&str>) -> Pair {
 		Self::from_string(s, password_override).unwrap_or_else(|_| {
 			let mut padded_seed: Seed = [' ' as u8; 32];
@@ -486,16 +512,16 @@ impl Pair {
 }
 
 impl CryptoType for Public {
-	#[cfg(feature="std")]
+	#[cfg(feature = "full_crypto")]
 	type Pair = Pair;
 }
 
 impl CryptoType for Signature {
-	#[cfg(feature="std")]
+	#[cfg(feature = "full_crypto")]
 	type Pair = Pair;
 }
 
-#[cfg(feature = "std")]
+#[cfg(feature = "full_crypto")]
 impl CryptoType for Pair {
 	type Pair = Pair;
 }
diff --git a/core/primitives/src/hash.rs b/primitives/core/src/hash.rs
similarity index 100%
rename from core/primitives/src/hash.rs
rename to primitives/core/src/hash.rs
diff --git a/core/primitives/src/hasher.rs b/primitives/core/src/hasher.rs
similarity index 100%
rename from core/primitives/src/hasher.rs
rename to primitives/core/src/hasher.rs
diff --git a/core/primitives/src/hashing.rs b/primitives/core/src/hashing.rs
similarity index 93%
rename from core/primitives/src/hashing.rs
rename to primitives/core/src/hashing.rs
index 87312ce6e46e9..c3f7ddf9f5a59 100644
--- a/core/primitives/src/hashing.rs
+++ b/primitives/core/src/hashing.rs
@@ -17,6 +17,7 @@
 //! Hashing functions.
 
 use blake2_rfc;
+use tiny_keccak::{Hasher, Keccak};
 use twox_hash;
 
 /// Do a Blake2 512-bit hash and place result in `dest`.
@@ -121,3 +122,12 @@ pub fn twox_256(data: &[u8]) -> [u8; 32] {
 	twox_256_into(data, &mut r);
 	r
 }
+
+/// Do a keccak 256 hash and return result.
+pub fn keccak_256(data: &[u8]) -> [u8; 32] {
+	let mut keccak = Keccak::v256();
+	keccak.update(data);
+	let mut output = [0u8; 32];
+	keccak.finalize(&mut output);
+	output
+}
diff --git a/core/primitives/src/hexdisplay.rs b/primitives/core/src/hexdisplay.rs
similarity index 100%
rename from core/primitives/src/hexdisplay.rs
rename to primitives/core/src/hexdisplay.rs
diff --git a/core/primitives/src/lib.rs b/primitives/core/src/lib.rs
similarity index 97%
rename from core/primitives/src/lib.rs
rename to primitives/core/src/lib.rs
index 02f28c797c002..900728afbfb44 100644
--- a/core/primitives/src/lib.rs
+++ b/primitives/core/src/lib.rs
@@ -38,19 +38,19 @@ use std::borrow::Cow;
 #[cfg(feature = "std")]
 use serde::{Serialize, Deserialize};
 #[cfg(feature = "std")]
-pub use serde;// << for macro
+pub use serde;
 #[doc(hidden)]
-pub use codec::{Encode, Decode};// << for macro
+pub use codec::{Encode, Decode};
 
 pub use substrate_debug_derive::RuntimeDebug;
 
 #[cfg(feature = "std")]
 pub use impl_serde::serialize as bytes;
 
-#[cfg(feature = "std")]
+#[cfg(feature = "full_crypto")]
 pub mod hashing;
-#[cfg(feature = "std")]
-pub use hashing::{blake2_128, blake2_256, twox_64, twox_128, twox_256};
+#[cfg(feature = "full_crypto")]
+pub use hashing::{blake2_128, blake2_256, twox_64, twox_128, twox_256, keccak_256};
 #[cfg(feature = "std")]
 pub mod hexdisplay;
 pub mod crypto;
@@ -76,7 +76,7 @@ mod tests;
 pub use self::hash::{H160, H256, H512, convert_hash};
 pub use self::uint::U256;
 pub use changes_trie::ChangesTrieConfiguration;
-#[cfg(feature = "std")]
+#[cfg(feature = "full_crypto")]
 pub use crypto::{DeriveJunction, Pair, Public};
 
 pub use hash_db::Hasher;
@@ -236,7 +236,7 @@ pub trait TypeId {
 /// A log level matching the one from `log` crate.
 ///
 /// Used internally by `runtime_io::log` method.
-#[repr(u32)]
+#[derive(Encode, Decode, runtime_interface::pass_by::PassByEnum, Copy, Clone)]
 pub enum LogLevel {
 	/// `Error` log level.
 	Error = 1,
diff --git a/core/primitives/src/offchain.rs b/primitives/core/src/offchain.rs
similarity index 97%
rename from core/primitives/src/offchain.rs
rename to primitives/core/src/offchain.rs
index 27bd29a00df8d..7b24e5ee72c98 100644
--- a/core/primitives/src/offchain.rs
+++ b/primitives/core/src/offchain.rs
@@ -19,11 +19,12 @@
 use codec::{Encode, Decode};
 use rstd::{prelude::{Vec, Box}, convert::TryFrom};
 use crate::RuntimeDebug;
+use runtime_interface::pass_by::{PassByCodec, PassByInner, PassByEnum};
 
 pub use crate::crypto::KeyTypeId;
 
 /// A type of supported crypto.
-#[derive(Clone, Copy, PartialEq, Eq, Encode, Decode, RuntimeDebug)]
+#[derive(Clone, Copy, PartialEq, Eq, Encode, Decode, RuntimeDebug, PassByEnum)]
 #[repr(C)]
 pub enum StorageKind {
 	/// Persistent storage is non-revertible and not fork-aware. It means that any value
@@ -59,7 +60,7 @@ impl From for u32 {
 }
 
 /// Opaque type for offchain http requests.
-#[derive(Clone, Copy, PartialEq, Eq, PartialOrd, Ord, RuntimeDebug)]
+#[derive(Clone, Copy, PartialEq, Eq, PartialOrd, Ord, RuntimeDebug, Encode, Decode, PassByInner)]
 #[cfg_attr(feature = "std", derive(Hash))]
 pub struct HttpRequestId(pub u16);
 
@@ -70,7 +71,7 @@ impl From for u32 {
 }
 
 /// An error enum returned by some http methods.
-#[derive(Clone, Copy, PartialEq, Eq, RuntimeDebug)]
+#[derive(Clone, Copy, PartialEq, Eq, RuntimeDebug, Encode, Decode, PassByEnum)]
 #[repr(C)]
 pub enum HttpError {
 	/// The requested action couldn't been completed within a deadline.
@@ -101,7 +102,7 @@ impl From for u32 {
 }
 
 /// Status of the HTTP request
-#[derive(Clone, Copy, PartialEq, Eq, RuntimeDebug)]
+#[derive(Clone, Copy, PartialEq, Eq, RuntimeDebug, Encode, Decode, PassByCodec)]
 pub enum HttpRequestStatus {
 	/// Deadline was reached while we waited for this request to finish.
 	///
@@ -147,7 +148,7 @@ impl TryFrom for HttpRequestStatus {
 
 /// A blob to hold information about the local node's network state
 /// without committing to its format.
-#[derive(Clone, Eq, PartialEq, Encode, Decode, RuntimeDebug)]
+#[derive(Clone, Eq, PartialEq, Encode, Decode, RuntimeDebug, PassByCodec)]
 pub struct OpaqueNetworkState {
 	/// PeerId of the local node.
 	pub peer_id: OpaquePeerId,
@@ -156,7 +157,7 @@ pub struct OpaqueNetworkState {
 }
 
 /// Simple blob to hold a `PeerId` without committing to its format.
-#[derive(Default, Clone, Eq, PartialEq, Encode, Decode, RuntimeDebug)]
+#[derive(Default, Clone, Eq, PartialEq, Encode, Decode, RuntimeDebug, PassByInner)]
 pub struct OpaquePeerId(pub Vec);
 
 impl OpaquePeerId {
@@ -167,7 +168,7 @@ impl OpaquePeerId {
 }
 
 /// Simple blob to hold a `Multiaddr` without committing to its format.
-#[derive(Clone, Eq, PartialEq, Encode, Decode, RuntimeDebug)]
+#[derive(Clone, Eq, PartialEq, Encode, Decode, RuntimeDebug, PassByInner)]
 pub struct OpaqueMultiaddr(pub Vec);
 
 impl OpaqueMultiaddr {
@@ -178,11 +179,11 @@ impl OpaqueMultiaddr {
 }
 
 /// Opaque timestamp type
-#[derive(Clone, Copy, PartialEq, Eq, Ord, PartialOrd, Default, RuntimeDebug)]
+#[derive(Clone, Copy, PartialEq, Eq, Ord, PartialOrd, Default, RuntimeDebug, PassByInner, Encode, Decode)]
 pub struct Timestamp(u64);
 
 /// Duration type
-#[derive(Clone, Copy, PartialEq, Eq, Ord, PartialOrd, Default, RuntimeDebug)]
+#[derive(Clone, Copy, PartialEq, Eq, Ord, PartialOrd, Default, RuntimeDebug, PassByInner, Encode, Decode)]
 pub struct Duration(u64);
 
 impl Duration {
@@ -289,7 +290,7 @@ impl<'a> From<&'a [Capability]> for Capabilities {
 }
 
 /// An extended externalities for offchain workers.
-pub trait Externalities {
+pub trait Externalities: Send {
 	/// Returns if the local node is a potential validator.
 	///
 	/// Even if this function returns `true`, it does not mean that any keys are configured
diff --git a/core/primitives/src/sandbox.rs b/primitives/core/src/sandbox.rs
similarity index 100%
rename from core/primitives/src/sandbox.rs
rename to primitives/core/src/sandbox.rs
diff --git a/core/primitives/src/sr25519.rs b/primitives/core/src/sr25519.rs
similarity index 93%
rename from core/primitives/src/sr25519.rs
rename to primitives/core/src/sr25519.rs
index ba7f480128081..668bf1f0b42ee 100644
--- a/core/primitives/src/sr25519.rs
+++ b/primitives/core/src/sr25519.rs
@@ -20,8 +20,9 @@
 //! Note: `CHAIN_CODE_LENGTH` must be equal to `crate::crypto::JUNCTION_ID_LEN`
 //! for this to work.
 // end::description[]
-
-#[cfg(feature = "std")]
+#[cfg(feature = "full_crypto")]
+use rstd::vec::Vec;
+#[cfg(feature = "full_crypto")]
 use schnorrkel::{signing_context, ExpansionMode, Keypair, SecretKey, MiniSecretKey, PublicKey,
 	derive::{Derivation, ChainCode, CHAIN_CODE_LENGTH}
 };
@@ -29,33 +30,38 @@ use schnorrkel::{signing_context, ExpansionMode, Keypair, SecretKey, MiniSecretK
 use substrate_bip39::mini_secret_from_entropy;
 #[cfg(feature = "std")]
 use bip39::{Mnemonic, Language, MnemonicType};
-#[cfg(feature = "std")]
+#[cfg(feature = "full_crypto")]
 use crate::crypto::{
-	Pair as TraitPair, DeriveJunction, Infallible, SecretStringError, Ss58Codec
+	Pair as TraitPair, DeriveJunction, Infallible, SecretStringError
 };
+#[cfg(feature = "std")]
+use crate::crypto::Ss58Codec;
+
 use crate::{crypto::{Public as TraitPublic, UncheckedFrom, CryptoType, Derive}};
 use crate::hash::{H256, H512};
 use codec::{Encode, Decode};
+use rstd::ops::Deref;
 
 #[cfg(feature = "std")]
 use serde::{de, Deserialize, Deserializer, Serialize, Serializer};
-#[cfg(feature = "std")]
+#[cfg(feature = "full_crypto")]
 use schnorrkel::keys::{MINI_SECRET_KEY_LENGTH, SECRET_KEY_LENGTH};
+use runtime_interface::pass_by::PassByInner;
 
 // signing context
-#[cfg(feature = "std")]
+#[cfg(feature = "full_crypto")]
 const SIGNING_CTX: &[u8] = b"substrate";
 
 /// An Schnorrkel/Ristretto x25519 ("sr25519") public key.
-#[cfg_attr(feature = "std", derive(Hash))]
-#[derive(PartialEq, Eq, PartialOrd, Ord, Clone, Encode, Decode, Default)]
+#[cfg_attr(feature = "full_crypto", derive(Hash))]
+#[derive(PartialEq, Eq, PartialOrd, Ord, Clone, Copy, Encode, Decode, Default, PassByInner)]
 pub struct Public(pub [u8; 32]);
 
 /// An Schnorrkel/Ristretto x25519 ("sr25519") key pair.
-#[cfg(feature = "std")]
+#[cfg(feature = "full_crypto")]
 pub struct Pair(Keypair);
 
-#[cfg(feature = "std")]
+#[cfg(feature = "full_crypto")]
 impl Clone for Pair {
 	fn clone(&self) -> Self {
 		Pair(schnorrkel::Keypair {
@@ -84,6 +90,14 @@ impl AsMut<[u8]> for Public {
 	}
 }
 
+impl Deref for Public {
+    type Target = [u8];
+
+    fn deref(&self) -> &Self::Target {
+        &self.0
+    }
+}
+
 impl From for [u8; 32] {
 	fn from(x: Public) -> [u8; 32] {
 		x.0
@@ -96,6 +110,15 @@ impl From for H256 {
 	}
 }
 
+#[cfg(feature = "std")]
+impl std::str::FromStr for Public {
+    type Err = crate::crypto::PublicError;
+
+    fn from_str(s: &str) -> Result {
+        Self::from_ss58check(s)
+    }
+}
+
 impl rstd::convert::TryFrom<&[u8]> for Public {
 	type Error = ();
 
@@ -160,7 +183,7 @@ impl<'de> Deserialize<'de> for Public {
 /// An Schnorrkel/Ristretto x25519 ("sr25519") signature.
 ///
 /// Instead of importing it for the local module, alias it to be available as a public type
-#[derive(Encode, Decode)]
+#[derive(Encode, Decode, PassByInner)]
 pub struct Signature(pub [u8; 64]);
 
 impl rstd::convert::TryFrom<&[u8]> for Signature {
@@ -229,7 +252,7 @@ impl AsMut<[u8]> for Signature {
 	}
 }
 
-#[cfg(feature = "std")]
+#[cfg(feature = "full_crypto")]
 impl From for Signature {
 	fn from(s: schnorrkel::Signature) -> Signature {
 		Signature(s.to_bytes())
@@ -248,10 +271,10 @@ impl rstd::fmt::Debug for Signature {
 	}
 }
 
-#[cfg(feature = "std")]
-impl std::hash::Hash for Signature {
-	fn hash(&self, state: &mut H) {
-		std::hash::Hash::hash(&self.0[..], state);
+#[cfg(feature = "full_crypto")]
+impl rstd::hash::Hash for Signature {
+	fn hash(&self, state: &mut H) {
+		rstd::hash::Hash::hash(&self.0[..], state);
 	}
 }
 
@@ -362,21 +385,21 @@ impl From for Pair {
 	}
 }
 
-#[cfg(feature = "std")]
+#[cfg(feature = "full_crypto")]
 impl From for Pair {
 	fn from(p: schnorrkel::Keypair) -> Pair {
 		Pair(p)
 	}
 }
 
-#[cfg(feature = "std")]
+#[cfg(feature = "full_crypto")]
 impl From for schnorrkel::Keypair {
 	fn from(p: Pair) -> schnorrkel::Keypair {
 		p.0
 	}
 }
 
-#[cfg(feature = "std")]
+#[cfg(feature = "full_crypto")]
 impl AsRef for Pair {
 	fn as_ref(&self) -> &schnorrkel::Keypair {
 		&self.0
@@ -384,16 +407,16 @@ impl AsRef for Pair {
 }
 
 /// Derive a single hard junction.
-#[cfg(feature = "std")]
+#[cfg(feature = "full_crypto")]
 fn derive_hard_junction(secret: &SecretKey, cc: &[u8; CHAIN_CODE_LENGTH]) -> MiniSecretKey {
 	secret.hard_derive_mini_secret_key(Some(ChainCode(cc.clone())), b"").0
 }
 
 /// The raw secret seed, which can be used to recreate the `Pair`.
-#[cfg(feature = "std")]
+#[cfg(feature = "full_crypto")]
 type Seed = [u8; MINI_SECRET_KEY_LENGTH];
 
-#[cfg(feature = "std")]
+#[cfg(feature = "full_crypto")]
 impl TraitPair for Pair {
 	type Public = Public;
 	type Seed = Seed;
@@ -440,7 +463,7 @@ impl TraitPair for Pair {
 			_ => Err(SecretStringError::InvalidSeedLength)
 		}
 	}
-
+	#[cfg(feature = "std")]
 	fn generate_with_phrase(password: Option<&str>) -> (Pair, String, Seed) {
 		let mnemonic = Mnemonic::new(MnemonicType::Words12, Language::English);
 		let phrase = mnemonic.phrase();
@@ -452,7 +475,7 @@ impl TraitPair for Pair {
 			seed,
 		)
 	}
-
+	#[cfg(feature = "std")]
 	fn from_phrase(phrase: &str, password: Option<&str>) -> Result<(Pair, Seed), SecretStringError> {
 		Mnemonic::from_phrase(phrase, Language::English)
 			.map_err(|_| SecretStringError::InvalidPhrase)
@@ -527,16 +550,16 @@ impl Pair {
 }
 
 impl CryptoType for Public {
-	#[cfg(feature="std")]
+	#[cfg(feature = "full_crypto")]
 	type Pair = Pair;
 }
 
 impl CryptoType for Signature {
-	#[cfg(feature="std")]
+	#[cfg(feature = "full_crypto")]
 	type Pair = Pair;
 }
 
-#[cfg(feature = "std")]
+#[cfg(feature = "full_crypto")]
 impl CryptoType for Pair {
 	type Pair = Pair;
 }
diff --git a/core/primitives/src/testing.rs b/primitives/core/src/testing.rs
similarity index 100%
rename from core/primitives/src/testing.rs
rename to primitives/core/src/testing.rs
diff --git a/core/primitives/src/tests.rs b/primitives/core/src/tests.rs
similarity index 100%
rename from core/primitives/src/tests.rs
rename to primitives/core/src/tests.rs
diff --git a/core/primitives/src/traits.rs b/primitives/core/src/traits.rs
similarity index 100%
rename from core/primitives/src/traits.rs
rename to primitives/core/src/traits.rs
diff --git a/core/primitives/src/u32_trait.rs b/primitives/core/src/u32_trait.rs
similarity index 100%
rename from core/primitives/src/u32_trait.rs
rename to primitives/core/src/u32_trait.rs
diff --git a/core/primitives/src/uint.rs b/primitives/core/src/uint.rs
similarity index 97%
rename from core/primitives/src/uint.rs
rename to primitives/core/src/uint.rs
index b41596a910326..c835cf5773518 100644
--- a/core/primitives/src/uint.rs
+++ b/primitives/core/src/uint.rs
@@ -91,7 +91,9 @@ mod tests {
 			"\"0xffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff\""
 		);
 		assert!(
-			ser::from_str::("\"0x1ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff\"").unwrap_err().is_data()
+			ser::from_str::("\"0x1ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff\"")
+			.unwrap_err()
+			.is_data()
 		);
 	}
 }
diff --git a/core/primitives/storage/Cargo.toml b/primitives/core/storage/Cargo.toml
similarity index 100%
rename from core/primitives/storage/Cargo.toml
rename to primitives/core/storage/Cargo.toml
diff --git a/core/primitives/storage/src/lib.rs b/primitives/core/storage/src/lib.rs
similarity index 94%
rename from core/primitives/storage/src/lib.rs
rename to primitives/core/storage/src/lib.rs
index dcdc223994e26..ba36e2c80f81e 100644
--- a/core/primitives/storage/src/lib.rs
+++ b/primitives/core/storage/src/lib.rs
@@ -40,6 +40,14 @@ pub struct StorageData(
 	pub Vec,
 );
 
+/// A set of key value pairs for storage.
+#[cfg(feature = "std")]
+pub type StorageOverlay = std::collections::HashMap, Vec>;
+
+/// A set of key value pairs for children storage;
+#[cfg(feature = "std")]
+pub type ChildrenStorageOverlay = std::collections::HashMap, StorageOverlay>;
+
 /// Storage change set
 #[derive(RuntimeDebug)]
 #[cfg_attr(feature = "std", derive(Serialize, Deserialize, PartialEq, Eq))]
diff --git a/core/externalities/Cargo.toml b/primitives/externalities/Cargo.toml
similarity index 76%
rename from core/externalities/Cargo.toml
rename to primitives/externalities/Cargo.toml
index 97ece5439ac99..0b806ecee0494 100644
--- a/core/externalities/Cargo.toml
+++ b/primitives/externalities/Cargo.toml
@@ -6,7 +6,7 @@ authors = ["Parity Technologies "]
 edition = "2018"
 
 [dependencies]
-primitive-types = { version = "0.5.1", features = ["codec"] }
-primitives-storage = { package = "substrate-primitives-storage", path = "../primitives/storage" }
+primitive-types = { version = "0.6", features = ["codec"] }
+primitives-storage = { package = "substrate-primitives-storage", path = "../core/storage" }
 rstd = { package = "sr-std", path = "../sr-std" }
 environmental = { version = "1.0.2" }
diff --git a/core/externalities/src/extensions.rs b/primitives/externalities/src/extensions.rs
similarity index 87%
rename from core/externalities/src/extensions.rs
rename to primitives/externalities/src/extensions.rs
index c7d7bc48538ac..a1a83cb197d4f 100644
--- a/core/externalities/src/extensions.rs
+++ b/primitives/externalities/src/extensions.rs
@@ -27,7 +27,12 @@ use std::{collections::HashMap, any::{Any, TypeId}, ops::DerefMut};
 ///
 /// As extensions are stored as `Box`, this trait should give more confidence that the correct
 /// type is registered and requested.
-pub trait Extension: Sized {}
+pub trait Extension: Send + Any {
+	/// Return the extension as `&mut dyn Any`.
+	///
+	/// This is a trick to make the trait type castable into an `Any`.
+	fn as_mut_any(&mut self) -> &mut dyn Any;
+}
 
 /// Macro for declaring an extension that usable with [`Extensions`].
 ///
@@ -51,7 +56,11 @@ macro_rules! decl_extension {
 		$( #[ $attr ] )*
 		$vis struct $ext_name (pub $inner);
 
-		impl $crate::Extension for $ext_name {}
+		impl $crate::Extension for $ext_name {
+			fn as_mut_any(&mut self) -> &mut dyn std::any::Any {
+				self
+			}
+		}
 
 		impl std::ops::Deref for $ext_name {
 			type Target = $inner;
@@ -83,7 +92,7 @@ pub trait ExtensionStore {
 /// Stores extensions that should be made available through the externalities.
 #[derive(Default)]
 pub struct Extensions {
-	extensions: HashMap>,
+	extensions: HashMap>,
 }
 
 impl Extensions {
@@ -93,13 +102,13 @@ impl Extensions {
 	}
 
 	/// Register the given extension.
-	pub fn register(&mut self, ext: E) {
+	pub fn register(&mut self, ext: E) {
 		self.extensions.insert(ext.type_id(), Box::new(ext));
 	}
 
 	/// Return a mutable reference to the requested extension.
 	pub fn get_mut(&mut self, ext_type_id: TypeId) -> Option<&mut dyn Any> {
-		self.extensions.get_mut(&ext_type_id).map(DerefMut::deref_mut)
+		self.extensions.get_mut(&ext_type_id).map(DerefMut::deref_mut).map(Extension::as_mut_any)
 	}
 }
 
@@ -107,11 +116,12 @@ impl Extensions {
 mod tests {
 	use super::*;
 
-	struct DummyExt(u32);
-	impl Extension for DummyExt {}
-
-	struct DummyExt2(u32);
-	impl Extension for DummyExt2 {}
+	decl_extension! {
+		struct DummyExt(u32);
+	}
+	decl_extension! {
+		struct DummyExt2(u32);
+	}
 
 	#[test]
 	fn register_and_retrieve_extension() {
diff --git a/core/externalities/src/lib.rs b/primitives/externalities/src/lib.rs
similarity index 96%
rename from core/externalities/src/lib.rs
rename to primitives/externalities/src/lib.rs
index ecd5f7a7a751d..4efbc54a4edc9 100644
--- a/core/externalities/src/lib.rs
+++ b/primitives/externalities/src/lib.rs
@@ -132,8 +132,8 @@ pub trait ExternalitiesExt {
 	fn extension(&mut self) -> Option<&mut T>;
 }
 
-impl ExternalitiesExt for T {
-	fn extension(&mut self) -> Option<&mut A> {
-		self.extension_by_type_id(TypeId::of::()).and_then(Any::downcast_mut)
+impl ExternalitiesExt for &mut dyn Externalities {
+	fn extension(&mut self) -> Option<&mut T> {
+		self.extension_by_type_id(TypeId::of::()).and_then(Any::downcast_mut)
 	}
 }
diff --git a/core/externalities/src/scope_limited.rs b/primitives/externalities/src/scope_limited.rs
similarity index 100%
rename from core/externalities/src/scope_limited.rs
rename to primitives/externalities/src/scope_limited.rs
diff --git a/primitives/finality-grandpa/Cargo.toml b/primitives/finality-grandpa/Cargo.toml
new file mode 100644
index 0000000000000..e9a166d2875bb
--- /dev/null
+++ b/primitives/finality-grandpa/Cargo.toml
@@ -0,0 +1,24 @@
+[package]
+name = "substrate-finality-grandpa-primitives"
+version = "2.0.0"
+authors = ["Parity Technologies "]
+edition = "2018"
+
+[dependencies]
+app-crypto = { package = "substrate-application-crypto", path = "../application-crypto", default-features = false }
+codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false, features = ["derive"] }
+rstd = { package = "sr-std", path = "../sr-std", default-features = false }
+serde = { version = "1.0.101", optional = true, features = ["derive"] }
+sr-api = { path = "../sr-api", default-features = false }
+sr-primitives = { path = "../sr-primitives", default-features = false }
+
+[features]
+default = ["std"]
+std = [
+	"app-crypto/std",
+	"codec/std",
+	"rstd/std",
+	"serde",
+	"sr-api/std",
+	"sr-primitives/std",
+]
diff --git a/core/finality-grandpa/primitives/src/lib.rs b/primitives/finality-grandpa/src/lib.rs
similarity index 73%
rename from core/finality-grandpa/primitives/src/lib.rs
rename to primitives/finality-grandpa/src/lib.rs
index 27139bbeeffa8..ff26b6a68ac2d 100644
--- a/core/finality-grandpa/primitives/src/lib.rs
+++ b/primitives/finality-grandpa/src/lib.rs
@@ -23,9 +23,9 @@ extern crate alloc;
 
 #[cfg(feature = "std")]
 use serde::Serialize;
-use codec::{Encode, Decode, Codec};
+use codec::{Encode, Decode, Input, Codec};
 use sr_primitives::{ConsensusEngineId, RuntimeDebug};
-use client::decl_runtime_apis;
+use rstd::borrow::Cow;
 use rstd::vec::Vec;
 
 mod app {
@@ -46,6 +46,10 @@ pub type AuthoritySignature = app::Signature;
 /// The `ConsensusEngineId` of GRANDPA.
 pub const GRANDPA_ENGINE_ID: ConsensusEngineId = *b"FRNK";
 
+/// The storage key for the current set of weighted Grandpa authorities.
+/// The value stored is an encoded VersionedAuthorityList.
+pub const GRANDPA_AUTHORITIES_KEY: &'static [u8] = b":grandpa_authorities";
+
 /// The weight of an authority.
 pub type AuthorityWeight = u64;
 
@@ -58,12 +62,15 @@ pub type SetId = u64;
 /// The round indicator.
 pub type RoundNumber = u64;
 
+/// A list of Grandpa authorities with associated weights.
+pub type AuthorityList = Vec<(AuthorityId, AuthorityWeight)>;
+
 /// A scheduled change of authority set.
 #[cfg_attr(feature = "std", derive(Serialize))]
 #[derive(Clone, Eq, PartialEq, Encode, Decode, RuntimeDebug)]
 pub struct ScheduledChange {
 	/// The new authorities after the change, along with their respective weights.
-	pub next_authorities: Vec<(AuthorityId, AuthorityWeight)>,
+	pub next_authorities: AuthorityList,
 	/// The number of blocks to delay.
 	pub delay: N,
 }
@@ -154,7 +161,56 @@ pub const PENDING_CHANGE_CALL: &str = "grandpa_pending_change";
 /// WASM function call to get current GRANDPA authorities.
 pub const AUTHORITIES_CALL: &str = "grandpa_authorities";
 
-decl_runtime_apis! {
+/// The current version of the stored AuthorityList type. The encoding version MUST be updated any
+/// time the AuthorityList type changes.
+const AUTHORITIES_VERISON: u8 = 1;
+
+/// An AuthorityList that is encoded with a version specifier. The encoding version is updated any
+/// time the AuthorityList type changes. This ensures that encodings of different versions of an
+/// AuthorityList are differentiable. Attempting to decode an authority list with an unknown
+/// version will fail.
+#[derive(Default)]
+pub struct VersionedAuthorityList<'a>(Cow<'a, AuthorityList>);
+
+impl<'a> From for VersionedAuthorityList<'a> {
+	fn from(authorities: AuthorityList) -> Self {
+		VersionedAuthorityList(Cow::Owned(authorities))
+	}
+}
+
+impl<'a> From<&'a AuthorityList> for VersionedAuthorityList<'a> {
+	fn from(authorities: &'a AuthorityList) -> Self {
+		VersionedAuthorityList(Cow::Borrowed(authorities))
+	}
+}
+
+impl<'a> Into for VersionedAuthorityList<'a> {
+	fn into(self) -> AuthorityList {
+		self.0.into_owned()
+	}
+}
+
+impl<'a> Encode for VersionedAuthorityList<'a> {
+	fn size_hint(&self) -> usize {
+		(AUTHORITIES_VERISON, self.0.as_ref()).size_hint()
+	}
+
+	fn using_encoded R>(&self, f: F) -> R {
+		(AUTHORITIES_VERISON, self.0.as_ref()).using_encoded(f)
+	}
+}
+
+impl<'a> Decode for VersionedAuthorityList<'a> {
+	fn decode(value: &mut I) -> Result {
+		let (version, authorities): (u8, AuthorityList) = Decode::decode(value)?;
+		if version != AUTHORITIES_VERISON {
+			return Err("unknown Grandpa authorities version".into());
+		}
+		Ok(authorities.into())
+	}
+}
+
+sr_api::decl_runtime_apis! {
 	/// APIs for integrating the GRANDPA finality gadget into runtimes.
 	/// This should be implemented on the runtime side.
 	///
@@ -172,6 +228,6 @@ decl_runtime_apis! {
 		/// When called at block B, it will return the set of authorities that should be
 		/// used to finalize descendants of this block (B+1, B+2, ...). The block B itself
 		/// is finalized by the authorities from block B-1.
-		fn grandpa_authorities() -> Vec<(AuthorityId, AuthorityWeight)>;
+		fn grandpa_authorities() -> AuthorityList;
 	}
 }
diff --git a/core/inherents/Cargo.toml b/primitives/inherents/Cargo.toml
similarity index 62%
rename from core/inherents/Cargo.toml
rename to primitives/inherents/Cargo.toml
index 45e0b9e828ec7..108cb15e9cce5 100644
--- a/core/inherents/Cargo.toml
+++ b/primitives/inherents/Cargo.toml
@@ -7,8 +7,9 @@ edition = "2018"
 [dependencies]
 parking_lot = { version = "0.9.0", optional = true }
 rstd = { package = "sr-std", path = "../sr-std", default-features = false }
-codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false, features = ["derive"] }
-sr-primitives = {  path = "../sr-primitives", default-features = false }
+primitives = { package = "substrate-primitives", path = "../core", default-features = false }
+codec = { package = "parity-scale-codec", version = "1.0.6", default-features = false, features = ["derive"] }
+derive_more = { version = "0.15.0", optional = true }
 
 [features]
 default = [ "std" ]
@@ -16,5 +17,6 @@ std = [
 	"parking_lot",
 	"rstd/std",
 	"codec/std",
-	"sr-primitives/std",
+	"primitives/std",
+	"derive_more",
 ]
diff --git a/core/inherents/src/lib.rs b/primitives/inherents/src/lib.rs
similarity index 94%
rename from core/inherents/src/lib.rs
rename to primitives/inherents/src/lib.rs
index f7363b483bfec..25512f4fcae2a 100644
--- a/core/inherents/src/lib.rs
+++ b/primitives/inherents/src/lib.rs
@@ -43,7 +43,37 @@ use parking_lot::RwLock;
 #[cfg(feature = "std")]
 use std::{sync::Arc, format};
 
-pub use sr_primitives::RuntimeString;
+/// An error that can occur within the inherent data system.
+#[derive(Debug, Encode, Decode, derive_more::Display)]
+#[cfg(feature = "std")]
+pub struct Error(String);
+
+#[cfg(feature = "std")]
+impl> From for Error {
+	fn from(data: T) -> Error {
+		Self(data.into())
+	}
+}
+
+#[cfg(feature = "std")]
+impl Error {
+	/// Convert this error into a `String`.
+	pub fn into_string(self) -> String {
+		self.0
+	}
+}
+
+/// An error that can occur within the inherent data system.
+#[derive(Encode, primitives::RuntimeDebug)]
+#[cfg(not(feature = "std"))]
+pub struct Error(&'static str);
+
+#[cfg(not(feature = "std"))]
+impl From<&'static str> for Error {
+	fn from(data: &'static str) -> Error {
+		Self(data)
+	}
+}
 
 /// An identifier for an inherent.
 pub type InherentIdentifier = [u8; 8];
@@ -73,7 +103,7 @@ impl InherentData {
 		&mut self,
 		identifier: InherentIdentifier,
 		inherent: &I,
-	) -> Result<(), RuntimeString> {
+	) -> Result<(), Error> {
 		match self.data.entry(identifier) {
 			Entry::Vacant(entry) => {
 				entry.insert(inherent.encode());
@@ -106,7 +136,7 @@ impl InherentData {
 	pub fn get_data(
 		&self,
 		identifier: &InherentIdentifier,
-	) -> Result, RuntimeString> {
+	) -> Result, Error> {
 		match self.data.get(identifier) {
 			Some(inherent) =>
 				I::decode(&mut &inherent[..])
@@ -163,7 +193,7 @@ impl CheckInherentsResult {
 		&mut self,
 		identifier: InherentIdentifier,
 		error: &E,
-	) -> Result<(), RuntimeString> {
+	) -> Result<(), Error> {
 		// Don't accept any other error
 		if self.fatal_error {
 			return Err("No other errors are accepted after an hard error!".into())
@@ -191,7 +221,7 @@ impl CheckInherentsResult {
 	pub fn get_error(
 		&self,
 		identifier: &InherentIdentifier,
-	) -> Result, RuntimeString> {
+	) -> Result, Error> {
 		self.errors.get_data(identifier)
 	}
 
@@ -245,7 +275,7 @@ impl InherentDataProviders {
 	pub fn register_provider(
 		&self,
 		provider: P,
-	) -> Result<(), RuntimeString> {
+	) -> Result<(), Error> {
 		if self.has_provider(&provider.inherent_identifier()) {
 			Err(
 				format!(
@@ -266,7 +296,7 @@ impl InherentDataProviders {
 	}
 
 	/// Create inherent data.
-	pub fn create_inherent_data(&self) -> Result {
+	pub fn create_inherent_data(&self) -> Result {
 		let mut data = InherentData::new();
 		self.providers.read().iter().try_for_each(|p| {
 			p.provide_inherent_data(&mut data)
@@ -305,7 +335,7 @@ impl InherentDataProviders {
 pub trait ProvideInherentData {
 	/// Is called when this inherent data provider is registered at the given
 	/// `InherentDataProviders`.
-	fn on_register(&self, _: &InherentDataProviders) -> Result<(), RuntimeString> {
+	fn on_register(&self, _: &InherentDataProviders) -> Result<(), Error> {
 		Ok(())
 	}
 
@@ -315,7 +345,7 @@ pub trait ProvideInherentData {
 	/// Provide inherent data that should be included in a block.
 	///
 	/// The data should be stored in the given `InherentData` structure.
-	fn provide_inherent_data(&self, inherent_data: &mut InherentData) -> Result<(), RuntimeString>;
+	fn provide_inherent_data(&self, inherent_data: &mut InherentData) -> Result<(), Error>;
 
 	/// Convert the given encoded error to a string.
 	///
@@ -445,7 +475,7 @@ mod tests {
 	const ERROR_TO_STRING: &str = "Found error!";
 
 	impl ProvideInherentData for TestInherentDataProvider {
-		fn on_register(&self, _: &InherentDataProviders) -> Result<(), RuntimeString> {
+		fn on_register(&self, _: &InherentDataProviders) -> Result<(), Error> {
 			*self.registered.write() = true;
 			Ok(())
 		}
@@ -454,7 +484,7 @@ mod tests {
 			&TEST_INHERENT_0
 		}
 
-		fn provide_inherent_data(&self, data: &mut InherentData) -> Result<(), RuntimeString> {
+		fn provide_inherent_data(&self, data: &mut InherentData) -> Result<(), Error> {
 			data.put_data(TEST_INHERENT_0, &42)
 		}
 
diff --git a/core/keyring/Cargo.toml b/primitives/keyring/Cargo.toml
similarity index 76%
rename from core/keyring/Cargo.toml
rename to primitives/keyring/Cargo.toml
index 0147689985fa9..cc0e2836bacea 100644
--- a/core/keyring/Cargo.toml
+++ b/primitives/keyring/Cargo.toml
@@ -5,7 +5,7 @@ authors = ["Parity Technologies "]
 edition = "2018"
 
 [dependencies]
-primitives = { package = "substrate-primitives",  path = "../primitives" }
+primitives = { package = "substrate-primitives",  path = "../core" }
 sr-primitives = { path = "../sr-primitives" }
 lazy_static = "1.4.0"
 strum = "0.15.0"
diff --git a/core/keyring/src/ed25519.rs b/primitives/keyring/src/ed25519.rs
similarity index 100%
rename from core/keyring/src/ed25519.rs
rename to primitives/keyring/src/ed25519.rs
diff --git a/core/keyring/src/lib.rs b/primitives/keyring/src/lib.rs
similarity index 100%
rename from core/keyring/src/lib.rs
rename to primitives/keyring/src/lib.rs
diff --git a/core/keyring/src/sr25519.rs b/primitives/keyring/src/sr25519.rs
similarity index 100%
rename from core/keyring/src/sr25519.rs
rename to primitives/keyring/src/sr25519.rs
diff --git a/core/offchain/primitives/Cargo.toml b/primitives/offchain/Cargo.toml
similarity index 61%
rename from core/offchain/primitives/Cargo.toml
rename to primitives/offchain/Cargo.toml
index c96a579c4446d..5b451ee6d1053 100644
--- a/core/offchain/primitives/Cargo.toml
+++ b/primitives/offchain/Cargo.toml
@@ -7,12 +7,12 @@ authors = ["Parity Technologies "]
 edition = "2018"
 
 [dependencies]
-client = { package = "substrate-client", path = "../../client", default-features = false }
-sr-primitives = {  path = "../../sr-primitives", default-features = false }
+sr-api = { path = "../sr-api", default-features = false }
+sr-primitives = {  path = "../sr-primitives", default-features = false }
 
 [features]
 default = ["std"]
 std = [
-	"client/std",
+	"sr-api/std",
 	"sr-primitives/std"
 ]
diff --git a/core/offchain/primitives/src/lib.rs b/primitives/offchain/src/lib.rs
similarity index 88%
rename from core/offchain/primitives/src/lib.rs
rename to primitives/offchain/src/lib.rs
index dda08ae43f570..876fcf49a2e74 100644
--- a/core/offchain/primitives/src/lib.rs
+++ b/primitives/offchain/src/lib.rs
@@ -19,10 +19,12 @@
 #![cfg_attr(not(feature = "std"), no_std)]
 #![warn(missing_docs)]
 
-use client::decl_runtime_apis;
 use sr_primitives::traits::NumberFor;
 
-decl_runtime_apis! {
+/// Local Storage Prefix used by the Offchain Worker API to
+pub const STORAGE_PREFIX: &[u8] = b"storage";
+
+sr_api::decl_runtime_apis! {
 	/// The offchain worker api.
 	pub trait OffchainWorkerApi {
 		/// Starts the off-chain task for given block number.
diff --git a/core/panic-handler/Cargo.toml b/primitives/panic-handler/Cargo.toml
similarity index 100%
rename from core/panic-handler/Cargo.toml
rename to primitives/panic-handler/Cargo.toml
diff --git a/core/panic-handler/src/lib.rs b/primitives/panic-handler/src/lib.rs
similarity index 100%
rename from core/panic-handler/src/lib.rs
rename to primitives/panic-handler/src/lib.rs
diff --git a/core/peerset/Cargo.toml b/primitives/peerset/Cargo.toml
similarity index 87%
rename from core/peerset/Cargo.toml
rename to primitives/peerset/Cargo.toml
index 96b721b41caf0..1b46737d2ac44 100644
--- a/core/peerset/Cargo.toml
+++ b/primitives/peerset/Cargo.toml
@@ -9,7 +9,7 @@ edition = "2018"
 
 [dependencies]
 futures-preview = "0.3.0-alpha.19"
-libp2p = { version = "0.12.0", default-features = false }
+libp2p = { version = "0.13.0", default-features = false }
 linked-hash-map = "0.5.2"
 log = "0.4.8"
 lru-cache = "0.1.2"
diff --git a/core/peerset/src/lib.rs b/primitives/peerset/src/lib.rs
similarity index 100%
rename from core/peerset/src/lib.rs
rename to primitives/peerset/src/lib.rs
diff --git a/core/peerset/src/peersstate.rs b/primitives/peerset/src/peersstate.rs
similarity index 100%
rename from core/peerset/src/peersstate.rs
rename to primitives/peerset/src/peersstate.rs
diff --git a/core/peerset/tests/fuzz.rs b/primitives/peerset/tests/fuzz.rs
similarity index 100%
rename from core/peerset/tests/fuzz.rs
rename to primitives/peerset/tests/fuzz.rs
diff --git a/core/phragmen/Cargo.toml b/primitives/phragmen/Cargo.toml
similarity index 51%
rename from core/phragmen/Cargo.toml
rename to primitives/phragmen/Cargo.toml
index e10be1d92d908..250a09dd34a06 100644
--- a/core/phragmen/Cargo.toml
+++ b/primitives/phragmen/Cargo.toml
@@ -5,17 +5,19 @@ authors = ["Parity Technologies "]
 edition = "2018"
 
 [dependencies]
-sr-primitives = { path = "../sr-primitives", default-features = false }
+serde = { version = "1.0.101", optional = true, features = ["derive"] }
 rstd = { package = "sr-std", path = "../sr-std", default-features = false }
+sr-primitives = { path = "../../primitives/sr-primitives", default-features = false }
 
 [dev-dependencies]
-runtime-io ={ package = "sr-io", path = "../sr-io" }
-support = { package = "srml-support", path = "../../srml/support" }
+runtime-io ={ package = "sr-io", path = "../../primitives/sr-io" }
+support = { package = "paint-support", path = "../../paint/support" }
 rand = "0.7.2"
 
 [features]
 default = ["std"]
 std = [
+	"serde",
 	"rstd/std",
 	"sr-primitives/std",
 ]
diff --git a/core/phragmen/benches/phragmen.rs b/primitives/phragmen/benches/phragmen.rs
similarity index 100%
rename from core/phragmen/benches/phragmen.rs
rename to primitives/phragmen/benches/phragmen.rs
diff --git a/core/phragmen/src/lib.rs b/primitives/phragmen/src/lib.rs
similarity index 86%
rename from core/phragmen/src/lib.rs
rename to primitives/phragmen/src/lib.rs
index 8c8401bed6511..67ea6077adf58 100644
--- a/core/phragmen/src/lib.rs
+++ b/primitives/phragmen/src/lib.rs
@@ -116,6 +116,7 @@ pub struct PhragmenResult {
 /// This, at the current version, resembles the `Exposure` defined in the staking SRML module, yet
 /// they do not necessarily have to be the same.
 #[derive(Default, RuntimeDebug)]
+#[cfg_attr(feature = "std", derive(serde::Serialize, serde::Deserialize))]
 pub struct Support {
 	/// The amount of support as the effect of self-vote.
 	pub own: ExtendedBalance,
@@ -139,16 +140,17 @@ pub type SupportMap = BTreeMap>;
 /// * `initial_candidates`: candidates list to be elected from.
 /// * `initial_voters`: voters list.
 /// * `stake_of`: something that can return the stake stake of a particular candidate or voter.
-/// * `self_vote`. If true, then each candidate will automatically vote for themselves with the a
-///   weight indicated by their stake. Note that when this is `true` candidates are filtered by
-/// having at least some backed stake from themselves.
+///
+/// This function does not strip out candidates who do not have any backing stake. It is the
+/// responsibility of the caller to make sure only those candidates who have a sensible economic
+/// value are passed in. From the perspective of this function, a candidate can easily be among the
+/// winner with no backing stake.
 pub fn elect(
 	candidate_count: usize,
 	minimum_candidate_count: usize,
 	initial_candidates: Vec,
 	initial_voters: Vec<(AccountId, Vec)>,
 	stake_of: FS,
-	self_vote: bool,
 ) -> Option> where
 	AccountId: Default + Ord + Member,
 	Balance: Default + Copy + SimpleArithmetic,
@@ -168,36 +170,16 @@ pub fn elect(
 	let num_voters = initial_candidates.len() + initial_voters.len();
 	let mut voters: Vec> = Vec::with_capacity(num_voters);
 
-	// collect candidates. self vote or filter might apply
-	let mut candidates = if self_vote {
-		// self vote. filter.
-		initial_candidates.into_iter().map(|who| {
-			let stake = stake_of(&who);
-			Candidate { who, approval_stake: to_votes(stake), ..Default::default() }
-		})
-		.filter(|c| !c.approval_stake.is_zero())
+	// Iterate once to create a cache of candidates indexes. This could be optimized by being
+	// provided by the call site.
+	let mut candidates = initial_candidates
+		.into_iter()
 		.enumerate()
-		.map(|(i, c)| {
-			voters.push(Voter {
-				who: c.who.clone(),
-				edges: vec![Edge { who: c.who.clone(), candidate_index: i, ..Default::default() }],
-				budget: c.approval_stake,
-				load: Rational128::zero(),
-			});
-			c_idx_cache.insert(c.who.clone(), i);
-			c
+		.map(|(idx, who)| {
+			c_idx_cache.insert(who.clone(), idx);
+			Candidate { who, ..Default::default() }
 		})
-		.collect::>>()
-	} else {
-		// no self vote. just collect.
-		initial_candidates.into_iter()
-			.enumerate()
-			.map(|(idx, who)| {
-				c_idx_cache.insert(who.clone(), idx);
-				Candidate { who, ..Default::default() }
-			})
-			.collect::>>()
-	};
+		.collect::>>();
 
 	// early return if we don't have enough candidates
 	if candidates.len() < minimum_candidate_count { return None; }
@@ -289,37 +271,33 @@ pub fn elect(
 	for n in &mut voters {
 		let mut assignment = (n.who.clone(), vec![]);
 		for e in &mut n.edges {
-			if let Some(c) = elected_candidates.iter().cloned().find(|(c, _)| *c == e.who) {
-				// if self_vote == false, this branch should always be executed as we want to
-				// collect all nominations
-				if c.0 != n.who || !self_vote  {
-					let per_bill_parts =
-					{
-						if n.load == e.load {
-							// Full support. No need to calculate.
-							Perbill::accuracy().into()
+			if elected_candidates.iter().position(|(ref c, _)| *c == e.who).is_some() {
+				let per_bill_parts =
+				{
+					if n.load == e.load {
+						// Full support. No need to calculate.
+						Perbill::accuracy().into()
+					} else {
+						if e.load.d() == n.load.d() {
+							// return e.load / n.load.
+							let desired_scale: u128 = Perbill::accuracy().into();
+							multiply_by_rational(
+								desired_scale,
+								e.load.n(),
+								n.load.n(),
+							).unwrap_or(Bounded::max_value())
 						} else {
-							if e.load.d() == n.load.d() {
-								// return e.load / n.load.
-								let desired_scale: u128 = Perbill::accuracy().into();
-								multiply_by_rational(
-									desired_scale,
-									e.load.n(),
-									n.load.n(),
-								).unwrap_or(Bounded::max_value())
-							} else {
-								// defensive only. Both edge and nominator loads are built from
-								// scores, hence MUST have the same denominator.
-								Zero::zero()
-							}
+							// defensive only. Both edge and nominator loads are built from
+							// scores, hence MUST have the same denominator.
+							Zero::zero()
 						}
-					};
-					// safer to .min() inside as well to argue as u32 is safe.
-					let per_thing = Perbill::from_parts(
-						per_bill_parts.min(Perbill::accuracy().into()) as u32
-					);
-					assignment.1.push((e.who.clone(), per_thing));
-				}
+					}
+				};
+				// safer to .min() inside as well to argue as u32 is safe.
+				let per_thing = Perbill::from_parts(
+					per_bill_parts.min(Perbill::accuracy().into()) as u32
+				);
+				assignment.1.push((e.who.clone(), per_thing));
 			}
 		}
 
@@ -367,7 +345,6 @@ pub fn build_support_map(
 	elected_stashes: &Vec,
 	assignments: &Vec<(AccountId, Vec>)>,
 	stake_of: FS,
-	assume_self_vote: bool,
 ) -> SupportMap where
 	AccountId: Default + Ord + Member,
 	Balance: Default + Copy + SimpleArithmetic,
@@ -379,11 +356,7 @@ pub fn build_support_map(
 	let mut supports = >::new();
 	elected_stashes
 		.iter()
-		.map(|e| (e, if assume_self_vote { to_votes(stake_of(e)) } else { Zero::zero() } ))
-		.for_each(|(e, s)| {
-			let item = Support { own: s, total: s, ..Default::default() };
-			supports.insert(e.clone(), item);
-		});
+		.for_each(|e| { supports.insert(e.clone(), Default::default()); });
 
 	// build support struct.
 	for (n, assignment) in assignments.iter() {
@@ -393,10 +366,20 @@ pub fn build_support_map(
 			// per-things to be sound.
 			let other_stake = *per_thing * nominator_stake;
 			if let Some(support) = supports.get_mut(c) {
-				// For an astronomically rich validator with more astronomically rich
-				// set of nominators, this might saturate.
-				support.total = support.total.saturating_add(other_stake);
-				support.others.push((n.clone(), other_stake));
+				if c == n {
+					// This is a nomination from `n` to themselves. This will increase both the
+					// `own` and `total` field.
+					debug_assert!(*per_thing == Perbill::one()); // TODO: deal with this: do we want it?
+					support.own = support.own.saturating_add(other_stake);
+					support.total = support.total.saturating_add(other_stake);
+				} else {
+					// This is a nomination from `n` to someone else. Increase `total` and add an entry
+					// inside `others`.
+					// For an astronomically rich validator with more astronomically rich
+					// set of nominators, this might saturate.
+					support.total = support.total.saturating_add(other_stake);
+					support.others.push((n.clone(), other_stake));
+				}
 			}
 		}
 	}
diff --git a/core/phragmen/src/mock.rs b/primitives/phragmen/src/mock.rs
similarity index 92%
rename from core/phragmen/src/mock.rs
rename to primitives/phragmen/src/mock.rs
index ee4e1eb4bbb32..a0ab23db34040 100644
--- a/core/phragmen/src/mock.rs
+++ b/primitives/phragmen/src/mock.rs
@@ -75,13 +75,16 @@ pub(crate) struct _PhragmenResult {
 	pub assignments: Vec<(A, Vec<_PhragmenAssignment>)>
 }
 
+pub(crate) fn auto_generate_self_voters(candidates: &[A]) -> Vec<(A, Vec)> {
+	candidates.iter().map(|c| (c.clone(), vec![c.clone()])).collect()
+}
+
 pub(crate) fn elect_float(
 	candidate_count: usize,
 	minimum_candidate_count: usize,
 	initial_candidates: Vec,
 	initial_voters: Vec<(A, Vec)>,
 	stake_of: FS,
-	self_vote: bool,
 ) -> Option<_PhragmenResult> where
 	A: Default + Ord + Member + Copy,
 	for<'r> FS: Fn(&'r A) -> Balance,
@@ -92,36 +95,14 @@ pub(crate) fn elect_float(
 	let num_voters = initial_candidates.len() + initial_voters.len();
 	let mut voters: Vec<_Voter> = Vec::with_capacity(num_voters);
 
-	let mut candidates = if self_vote {
-		initial_candidates.into_iter().map(|who| {
-			let stake = stake_of(&who) as f64;
-			_Candidate { who, approval_stake: stake, ..Default::default() }
-		})
-		.filter(|c| c.approval_stake != 0f64)
+	let mut candidates = initial_candidates
+		.into_iter()
 		.enumerate()
-		.map(|(i, c)| {
-			let who = c.who;
-			voters.push(_Voter {
-				who: who.clone(),
-				edges: vec![
-					_Edge { who: who.clone(), candidate_index: i, ..Default::default() }
-				],
-				budget: c.approval_stake,
-				load: 0f64,
-			});
-			c_idx_cache.insert(c.who.clone(), i);
-			c
+		.map(|(idx, who)| {
+			c_idx_cache.insert(who.clone(), idx);
+			_Candidate { who, ..Default::default() }
 		})
-		.collect::>>()
-	} else {
-		initial_candidates.into_iter()
-			.enumerate()
-			.map(|(idx, who)| {
-				c_idx_cache.insert(who.clone(), idx);
-				_Candidate { who, ..Default::default() }
-			})
-			.collect::>>()
-	};
+		.collect::>>();
 
 	if candidates.len() < minimum_candidate_count {
 		return None;
@@ -359,7 +340,6 @@ pub(crate) fn run_and_compare(
 	stake_of: Box Balance>,
 	to_elect: usize,
 	min_to_elect: usize,
-	self_vote: bool,
 ) {
 	// run fixed point code.
 	let PhragmenResult { winners, assignments } = elect::<_, _, _, TestCurrencyToVote>(
@@ -368,7 +348,6 @@ pub(crate) fn run_and_compare(
 		candidates.clone(),
 		voters.clone(),
 		&stake_of,
-		self_vote,
 	).unwrap();
 
 	// run float poc code.
@@ -378,7 +357,6 @@ pub(crate) fn run_and_compare(
 		candidates,
 		voters,
 		&stake_of,
-		self_vote,
 	).unwrap();
 
 	assert_eq!(winners, truth_value.winners);
diff --git a/core/phragmen/src/tests.rs b/primitives/phragmen/src/tests.rs
similarity index 89%
rename from core/phragmen/src/tests.rs
rename to primitives/phragmen/src/tests.rs
index d3c5b1168cc4e..aaeac27c1411c 100644
--- a/core/phragmen/src/tests.rs
+++ b/primitives/phragmen/src/tests.rs
@@ -32,7 +32,7 @@ fn float_phragmen_poc_works() {
 		(30, vec![2, 3]),
 	];
 	let stake_of = create_stake_of(&[(10, 10), (20, 20), (30, 30), (1, 0), (2, 0), (3, 0)]);
-	let mut phragmen_result = elect_float(2, 2, candidates, voters, &stake_of, false).unwrap();
+	let mut phragmen_result = elect_float(2, 2, candidates, voters, &stake_of).unwrap();
 	let winners = phragmen_result.clone().winners;
 	let assignments = phragmen_result.clone().assignments;
 
@@ -84,7 +84,6 @@ fn phragmen_poc_works() {
 		candidates,
 		voters,
 		create_stake_of(&[(10, 10), (20, 20), (30, 30)]),
-		false,
 	).unwrap();
 
 	assert_eq_uvec!(winners, vec![(2, 40), (3, 50)]);
@@ -114,7 +113,7 @@ fn phragmen_poc_2_works() {
 		(4, 500),
 	]);
 
-	run_and_compare(candidates, voters, stake_of, 2, 2, true);
+	run_and_compare(candidates, voters, stake_of, 2, 2);
 }
 
 #[test]
@@ -132,7 +131,7 @@ fn phragmen_poc_3_works() {
 		(4, 1000),
 	]);
 
-	run_and_compare(candidates, voters, stake_of, 2, 2, true);
+	run_and_compare(candidates, voters, stake_of, 2, 2);
 }
 
 #[test]
@@ -151,24 +150,24 @@ fn phragmen_accuracy_on_large_scale_only_validators() {
 	let PhragmenResult { winners, assignments } = elect::<_, _, _, TestCurrencyToVote>(
 		2,
 		2,
-		candidates,
-		vec![],
+		candidates.clone(),
+		auto_generate_self_voters(&candidates),
 		stake_of,
-		true,
 	).unwrap();
 
 	assert_eq_uvec!(winners, vec![(1, 18446744073709551614u128), (5, 18446744073709551613u128)]);
-	assert_eq!(assignments.len(), 0);
+	assert_eq!(assignments.len(), 2);
 	check_assignments(assignments);
 }
 
 #[test]
 fn phragmen_accuracy_on_large_scale_validators_and_nominators() {
 	let candidates = vec![1, 2, 3, 4, 5];
-	let voters = vec![
+	let mut voters = vec![
 		(13, vec![1, 3, 5]),
 		(14, vec![2, 4]),
 	];
+	voters.extend(auto_generate_self_voters(&candidates));
 	let stake_of = create_stake_of(&[
 		(1,  (u64::max_value() - 1).into()),
 		(2,  (u64::max_value() - 4).into()),
@@ -185,13 +184,17 @@ fn phragmen_accuracy_on_large_scale_validators_and_nominators() {
 		candidates,
 		voters,
 		stake_of,
-		true,
 	).unwrap();
 
 	assert_eq_uvec!(winners, vec![(2, 36893488147419103226u128), (1, 36893488147419103219u128)]);
 	assert_eq!(
 		assignments,
-		vec![(13, vec![(1, Perbill::one())]), (14, vec![(2, Perbill::one())])]
+		vec![
+			(13, vec![(1, Perbill::one())]),
+			(14, vec![(2, Perbill::one())]),
+			(1, vec![(1, Perbill::one())]),
+			(2, vec![(2, Perbill::one())]),
+		]
 	);
 	check_assignments(assignments);
 }
@@ -199,7 +202,7 @@ fn phragmen_accuracy_on_large_scale_validators_and_nominators() {
 #[test]
 fn phragmen_accuracy_on_small_scale_self_vote() {
 	let candidates = vec![40, 10, 20, 30];
-	let voters = vec![];
+	let voters = auto_generate_self_voters(&candidates);
 	let stake_of = create_stake_of(&[
 		(40, 0),
 		(10, 1),
@@ -213,7 +216,6 @@ fn phragmen_accuracy_on_small_scale_self_vote() {
 		candidates,
 		voters,
 		stake_of,
-		true,
 	).unwrap();
 
 	assert_eq_uvec!(winners, vec![(20, 2), (10, 1), (30, 1)]);
@@ -245,7 +247,6 @@ fn phragmen_accuracy_on_small_scale_no_self_vote() {
 		candidates,
 		voters,
 		stake_of,
-		false,
 	).unwrap();
 
 	assert_eq_uvec!(winners, vec![(20, 2), (10, 1), (30, 1)]);
@@ -254,9 +255,10 @@ fn phragmen_accuracy_on_small_scale_no_self_vote() {
 #[test]
 fn phragmen_large_scale_test() {
 	let candidates = vec![2, 4, 6, 8, 10, 12, 14, 16 ,18, 20, 22, 24];
-	let voters = vec![
+	let mut voters = vec![
 		(50, vec![2, 4, 6, 8, 10, 12, 14, 16 ,18, 20, 22, 24]),
 	];
+	voters.extend(auto_generate_self_voters(&candidates));
 	let stake_of = create_stake_of(&[
 		(2,  1),
 		(4,  100),
@@ -279,7 +281,6 @@ fn phragmen_large_scale_test() {
 		candidates,
 		voters,
 		stake_of,
-		true,
 	).unwrap();
 
 	assert_eq_uvec!(winners, vec![(24, 1490000000000200000u128), (22, 1490000000000100000u128)]);
@@ -292,7 +293,8 @@ fn phragmen_large_scale_test_2() {
 	let c_budget: u64 = 4_000_000;
 
 	let candidates = vec![2, 4];
-	let voters = vec![(50, vec![2, 4])];
+	let mut voters = vec![(50, vec![2, 4])];
+	voters.extend(auto_generate_self_voters(&candidates));
 
 	let stake_of = create_stake_of(&[
 		(2,  c_budget.into()),
@@ -306,13 +308,16 @@ fn phragmen_large_scale_test_2() {
 		candidates,
 		voters,
 		stake_of,
-		true,
 	).unwrap();
 
 	assert_eq_uvec!(winners, vec![(2, 1000000000004000000u128), (4, 1000000000004000000u128)]);
 	assert_eq!(
 		assignments,
-		vec![(50, vec![(2, Perbill::from_parts(500000001)), (4, Perbill::from_parts(499999999))])],
+		vec![
+			(50, vec![(2, Perbill::from_parts(500000001)), (4, Perbill::from_parts(499999999))]),
+			(2, vec![(2, Perbill::one())]),
+			(4, vec![(4, Perbill::one())]),
+		],
 	);
 	check_assignments(assignments);
 }
@@ -347,5 +352,5 @@ fn phragmen_linear_equalize() {
 		(130, 1000),
 	]);
 
-	run_and_compare(candidates, voters, stake_of, 2, 2, true);
+	run_and_compare(candidates, voters, stake_of, 2, 2);
 }
diff --git a/core/rpc/primitives/Cargo.toml b/primitives/rpc/Cargo.toml
similarity index 72%
rename from core/rpc/primitives/Cargo.toml
rename to primitives/rpc/Cargo.toml
index 4044428112780..5216882b1a22e 100644
--- a/core/rpc/primitives/Cargo.toml
+++ b/primitives/rpc/Cargo.toml
@@ -6,4 +6,4 @@ edition = "2018"
 
 [dependencies]
 serde = { version = "1.0.101", features = ["derive"] }
-primitives = { package = "substrate-primitives", path = "../../primitives" }
+primitives = { package = "substrate-primitives", path = "../core" }
diff --git a/core/rpc/primitives/src/lib.rs b/primitives/rpc/src/lib.rs
similarity index 100%
rename from core/rpc/primitives/src/lib.rs
rename to primitives/rpc/src/lib.rs
diff --git a/core/rpc/primitives/src/number.rs b/primitives/rpc/src/number.rs
similarity index 100%
rename from core/rpc/primitives/src/number.rs
rename to primitives/rpc/src/number.rs
diff --git a/primitives/runtime-interface/Cargo.toml b/primitives/runtime-interface/Cargo.toml
new file mode 100644
index 0000000000000..7af9b8b0887e5
--- /dev/null
+++ b/primitives/runtime-interface/Cargo.toml
@@ -0,0 +1,41 @@
+[package]
+name = "substrate-runtime-interface"
+version = "2.0.0"
+authors = ["Parity Technologies "]
+edition = "2018"
+
+[dependencies]
+wasm-interface = { package = "substrate-wasm-interface", path = "../wasm-interface", optional = true }
+rstd = { package = "sr-std", path = "../sr-std", default-features = false }
+substrate-runtime-interface-proc-macro = { path = "proc-macro" }
+externalities = { package = "substrate-externalities", path = "../externalities", optional = true }
+codec = { package = "parity-scale-codec", version = "1.0.6", default-features = false }
+environmental = { version = "1.0.2", optional = true }
+static_assertions = "1.0.0"
+primitive-types = { version = "0.6.1", default-features = false }
+
+[dev-dependencies]
+executor = { package = "substrate-executor", path = "../../client/executor" }
+test-wasm = { package = "substrate-runtime-interface-test-wasm", path = "test-wasm" }
+state_machine = { package = "substrate-state-machine", path = "../../primitives/state-machine"  }
+primitives = { package = "substrate-primitives", path = "../core" }
+runtime-io = { package = "sr-io", path = "../sr-io" }
+
+[features]
+default = [ "std" ]
+std = [
+	"wasm-interface",
+	"rstd/std",
+	"codec/std",
+	"externalities",
+	"environmental",
+	"primitive-types/std",
+]
+
+# ATTENTION
+#
+# Only use when you know what you are doing.
+#
+# Disables static assertions in `impls.rs` that checks the word size. To prevent any footgun, the
+# check is changed into a runtime check.
+disable_target_static_assertions = []
diff --git a/primitives/runtime-interface/proc-macro/Cargo.toml b/primitives/runtime-interface/proc-macro/Cargo.toml
new file mode 100644
index 0000000000000..0b073b854749d
--- /dev/null
+++ b/primitives/runtime-interface/proc-macro/Cargo.toml
@@ -0,0 +1,27 @@
+[package]
+name = "substrate-runtime-interface-proc-macro"
+version = "2.0.0"
+authors = ["Parity Technologies "]
+edition = "2018"
+
+[lib]
+proc-macro = true
+
+[dependencies]
+syn = { version = "1.0.5", features = [ "full", "visit", "fold", "extra-traits" ] }
+quote = "1.0.2"
+proc-macro2 = "1.0.3"
+Inflector = "0.11.4"
+proc-macro-crate = "0.1.4"
+
+[dev-dependencies]
+runtime-interface = { package = "substrate-runtime-interface", path = ".." }
+codec = { package = "parity-scale-codec", version = "1.0.6", features = [ "derive" ] }
+externalities = { package = "substrate-externalities", path = "../../externalities" }
+rustversion = "1.0.0"
+trybuild = "1.0.17"
+
+# We actually don't need the `std` feature in this crate, but the tests require it.
+[features]
+default = [ "std" ]
+std = []
diff --git a/primitives/runtime-interface/proc-macro/src/lib.rs b/primitives/runtime-interface/proc-macro/src/lib.rs
new file mode 100644
index 0000000000000..991fab7569388
--- /dev/null
+++ b/primitives/runtime-interface/proc-macro/src/lib.rs
@@ -0,0 +1,265 @@
+// Copyright 2019 Parity Technologies (UK) Ltd.
+// This file is part of Substrate.
+
+// Substrate is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+
+// Substrate is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+
+// You should have received a copy of the GNU General Public License
+// along with Substrate.  If not, see .
+
+//! This crate provides procedural macros for usage within the context of the Substrate runtime
+//! interface.
+//!
+//! The following macros are provided:
+//!
+//! 1. The [`#[runtime_interface]`](attr.runtime_interface.html) attribute macro for generating the
+//!    runtime interfaces.
+//! 2. The [`PassByCodec`](derive.PassByCodec.html) derive macro for implementing `PassBy` with `Codec`.
+//! 3. The [`PassByEnum`](derive.PassByInner.html) derive macro for implementing `PassBy` with `Enum`.
+//! 4. The [`PassByInner`](derive.PassByInner.html) derive macro for implementing `PassBy` with `Inner`.
+
+extern crate proc_macro;
+
+use syn::{parse_macro_input, ItemTrait, DeriveInput};
+
+mod pass_by;
+mod runtime_interface;
+mod utils;
+
+/// Attribute macro for transforming a trait declaration into a runtime interface.
+///
+/// A runtime interface is a fixed interface between a Substrate compatible runtime and the native
+/// node. This interface is callable from a native and a wasm runtime. The macro will generate the
+/// corresponding code for the native implementation and the code for calling from the wasm
+/// side to the native implementation.
+///
+/// The macro expects the runtime interface declaration as trait declaration:
+///
+/// ```
+/// # use runtime_interface::runtime_interface;
+///
+/// #[runtime_interface]
+/// trait Interface {
+///     /// A function that can be called from native/wasm.
+///     ///
+///     /// The implementation given to this function is only compiled on native.
+///     fn call_some_complex_code(data: &[u8]) -> Vec {
+///         // Here you could call some rather complex code that only compiles on native or
+///         // is way faster in native than executing it in wasm.
+///         Vec::new()
+///     }
+///
+///     /// A function can take a `&self` or `&mut self` argument to get access to the
+///     /// `Externalities`. (The generated method does not require
+///     /// this argument, so the function can be called just with the `optional` argument)
+///     fn set_or_clear(&mut self, optional: Option>) {
+///         match optional {
+///             Some(value) => self.set_storage([1, 2, 3, 4].to_vec(), value),
+///             None => self.clear_storage(&[1, 2, 3, 4]),
+///         }
+///     }
+/// }
+/// ```
+///
+///
+/// The given example will generate roughly the following code for native:
+///
+/// ```
+/// // The name of the trait is converted to snake case and used as mod name.
+/// //
+/// // Be aware that this module is not `public`, the visibility of the module is determined based
+/// // on the visibility of the trait declaration.
+/// mod interface {
+///     trait Interface {
+///         fn call_some_complex_code(data: &[u8]) -> Vec;
+///         fn set_or_clear(&mut self, optional: Option>);
+///     }
+///
+///     impl Interface for &mut dyn externalities::Externalities {
+///         fn call_some_complex_code(data: &[u8]) -> Vec { Vec::new() }
+///         fn set_or_clear(&mut self, optional: Option>) {
+///             match optional {
+///                 Some(value) => self.set_storage([1, 2, 3, 4].to_vec(), value),
+///                 None => self.clear_storage(&[1, 2, 3, 4]),
+///             }
+///         }
+///     }
+///
+///     pub fn call_some_complex_code(data: &[u8]) -> Vec {
+///         <&mut dyn externalities::Externalities as Interface>::call_some_complex_code(data)
+///     }
+///
+///     pub fn set_or_clear(optional: Option>) {
+///         externalities::with_externalities(|mut ext| Interface::set_or_clear(&mut ext, optional))
+///             .expect("`set_or_clear` called outside of an Externalities-provided environment.")
+///     }
+///
+///     /// This type implements the `HostFunctions` trait (from `substrate-wasm-interface`) and
+///     /// provides the host implementation for the wasm side. The host implementation converts the
+///     /// arguments from wasm to native and calls the corresponding native function.
+///     ///
+///     /// This type needs to be passed to the wasm executor, so that the host functions will be
+///     /// registered in the executor.
+///     pub struct HostFunctions;
+/// }
+/// ```
+///
+///
+/// The given example will generate roughly the following code for wasm:
+///
+/// ```
+/// mod interface {
+///     mod extern_host_functions_impls {
+///         extern "C" {
+///             /// Every function is exported as `ext_TRAIT_NAME_FUNCTION_NAME_version_VERSION`.
+///             ///
+///             /// `TRAIT_NAME` is converted into snake case.
+///             ///
+///             /// The type for each argument of the exported function depends on
+///             /// `::FFIType`.
+///             ///
+///             /// `data` holds the pointer and the length to the `[u8]` slice.
+///             pub fn ext_Interface_call_some_complex_code_version_1(data: u64) -> u64;
+///             /// `optional` holds the pointer and the length of the encoded value.
+///             pub fn ext_Interface_set_or_clear_version_1(optional: u64);
+///         }
+///     }
+///
+///     /// The type is actually `ExchangeableFunction` (from `substrate-runtime-interface`).
+///     ///
+///     /// This can be used to replace the implementation of the `call_some_complex_code` function.
+///     /// Instead of calling into the host, the callee will automatically call the other
+///     /// implementation.
+///     ///
+///     /// To replace the implementation:
+///     ///
+///     /// `host_call_some_complex_code.replace_implementation(some_other_impl)`
+///     pub static host_call_some_complex_code: () = ();
+///     pub static host_set_or_clear: () = ();
+///
+///     pub fn call_some_complex_code(data: &[u8]) -> Vec {
+///         // This is the actual call: `host_call_some_complex_code.get()(data)`
+///         //
+///         // But that does not work for several reasons in this example, so we just return an
+///         // empty vector.
+///         Vec::new()
+///     }
+///
+///     pub fn set_or_clear(optional: Option>) {
+///         // Same as above
+///     }
+/// }
+/// ```
+///
+/// # Argument types
+///
+/// The macro supports any kind of argument type, as long as it implements `RIType` and the required
+/// `FromFFIValue`/`IntoFFIValue` from `substrate-runtime-interface`. The macro will convert each
+/// argument to the corresponding FFI representation and will call into the host using this FFI
+/// representation. On the host each argument is converted back to the native representation and
+/// the native implementation is called. Any return value is handled in the same way.
+///
+/// # Wasm only interfaces
+///
+/// Some interfaces are only required from within the wasm runtime e.g. the allocator interface.
+/// To support this, the macro can be called like `#[runtime_interface(wasm_only)]`. This instructs
+/// the macro to make two significant changes to the generated code:
+///
+/// 1. The generated functions are not callable from the native side.
+/// 2. The trait as shown above is not implemented for `Externalities` and is instead implemented
+///    for `FunctionExecutor` (from `substrate-wasm-interface`).
+#[proc_macro_attribute]
+pub fn runtime_interface(
+	attrs: proc_macro::TokenStream,
+	input: proc_macro::TokenStream,
+) -> proc_macro::TokenStream {
+	let trait_def = parse_macro_input!(input as ItemTrait);
+	let wasm_only = parse_macro_input!(attrs as Option);
+
+	runtime_interface::runtime_interface_impl(trait_def, wasm_only.is_some())
+		.unwrap_or_else(|e| e.to_compile_error())
+		.into()
+}
+
+/// Derive macro for implementing `PassBy` with the `Codec` strategy.
+///
+/// This requires that the type implements `Encode` and `Decode` from `parity-scale-codec`.
+///
+/// # Example
+///
+/// ```
+/// # use runtime_interface::pass_by::PassByCodec;
+/// # use codec::{Encode, Decode};
+/// #[derive(PassByCodec, Encode, Decode)]
+/// struct EncodableType {
+///     name: Vec,
+///     param: u32,
+/// }
+/// ```
+#[proc_macro_derive(PassByCodec)]
+pub fn pass_by_codec(input: proc_macro::TokenStream) -> proc_macro::TokenStream {
+	let input = parse_macro_input!(input as DeriveInput);
+	pass_by::codec_derive_impl(input).unwrap_or_else(|e| e.to_compile_error()).into()
+}
+
+/// Derive macro for implementing `PassBy` with the `Inner` strategy.
+///
+/// Besides implementing `PassBy`, this derive also implements the helper trait `PassByInner`.
+///
+/// The type is required to be a struct with just one field. The field type needs to implement
+/// the required traits to pass it between the wasm and the native side. (See the runtime interface
+/// crate for more information about these traits.)
+///
+/// # Example
+///
+/// ```
+/// # use runtime_interface::pass_by::PassByInner;
+/// #[derive(PassByInner)]
+/// struct Data([u8; 32]);
+/// ```
+///
+/// ```
+/// # use runtime_interface::pass_by::PassByInner;
+/// #[derive(PassByInner)]
+/// struct Data {
+///     data: [u8; 32],
+/// }
+/// ```
+#[proc_macro_derive(PassByInner)]
+pub fn pass_by_inner(input: proc_macro::TokenStream) -> proc_macro::TokenStream {
+	let input = parse_macro_input!(input as DeriveInput);
+	pass_by::inner_derive_impl(input).unwrap_or_else(|e| e.to_compile_error()).into()
+}
+
+/// Derive macro for implementing `PassBy` with the `Enum` strategy.
+///
+/// Besides implementing `PassBy`, this derive also implements `TryFrom` and `From for u8`
+/// for the type.
+///
+/// The type is required to be an enum with only unit variants and at maximum `256` variants. Also
+/// it is required that the type implements `Copy`.
+///
+/// # Example
+///
+/// ```
+/// # use runtime_interface::pass_by::PassByEnum;
+/// #[derive(PassByEnum, Copy, Clone)]
+/// enum Data {
+///     Okay,
+///     NotOkay,
+///     // This will not work with the derive.
+///     //Why(u32),
+/// }
+/// ```
+#[proc_macro_derive(PassByEnum)]
+pub fn pass_by_enum(input: proc_macro::TokenStream) -> proc_macro::TokenStream {
+	let input = parse_macro_input!(input as DeriveInput);
+	pass_by::enum_derive_impl(input).unwrap_or_else(|e| e.to_compile_error()).into()
+}
diff --git a/primitives/runtime-interface/proc-macro/src/pass_by/codec.rs b/primitives/runtime-interface/proc-macro/src/pass_by/codec.rs
new file mode 100644
index 0000000000000..c5c6980d2c652
--- /dev/null
+++ b/primitives/runtime-interface/proc-macro/src/pass_by/codec.rs
@@ -0,0 +1,58 @@
+// Copyright 2019 Parity Technologies (UK) Ltd.
+// This file is part of Substrate.
+
+// Substrate is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+
+// Substrate is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+
+// You should have received a copy of the GNU General Public License
+// along with Substrate.  If not, see .
+
+//! Derive macro implementation of `PassBy` with the associated type set to `Codec`.
+//!
+//! It is required that the type implements `Encode` and `Decode` from the `parity-scale-codec`
+//! crate.
+
+use crate::utils::{generate_crate_access, generate_runtime_interface_include};
+
+use syn::{DeriveInput, Result, Generics, parse_quote};
+
+use quote::quote;
+
+use proc_macro2::TokenStream;
+
+/// The derive implementation for `PassBy` with `Codec`.
+pub fn derive_impl(mut input: DeriveInput) -> Result {
+	add_trait_bounds(&mut input.generics);
+	let (impl_generics, ty_generics, where_clause) = input.generics.split_for_impl();
+	let crate_include = generate_runtime_interface_include();
+	let crate_ = generate_crate_access();
+	let ident = input.ident;
+
+	let res = quote! {
+		const _: () = {
+			#crate_include
+
+			impl #impl_generics #crate_::pass_by::PassBy for #ident #ty_generics #where_clause {
+				type PassBy = #crate_::pass_by::Codec<#ident>;
+			}
+		};
+	};
+
+	Ok(res)
+}
+
+/// Add the `codec::Codec` trait bound to every type parameter.
+fn add_trait_bounds(generics: &mut Generics) {
+	let crate_ = generate_crate_access();
+
+	generics.type_params_mut()
+		.for_each(|type_param| type_param.bounds.push(parse_quote!(#crate_::codec::Codec)));
+}
+
diff --git a/primitives/runtime-interface/proc-macro/src/pass_by/enum_.rs b/primitives/runtime-interface/proc-macro/src/pass_by/enum_.rs
new file mode 100644
index 0000000000000..ac5e67552083b
--- /dev/null
+++ b/primitives/runtime-interface/proc-macro/src/pass_by/enum_.rs
@@ -0,0 +1,101 @@
+// Copyright 2019 Parity Technologies (UK) Ltd.
+// This file is part of Substrate.
+
+// Substrate is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+
+// Substrate is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+
+// You should have received a copy of the GNU General Public License
+// along with Substrate.  If not, see .
+
+//! Derive macro implementation of `PassBy` with the associated type set to `Enum`.
+//!
+//! Besides `PassBy`, `TryFrom` and `From for u8` are implemented for the type.
+
+use crate::utils::{generate_crate_access, generate_runtime_interface_include};
+
+use syn::{DeriveInput, Result, Data, Fields, Error, Ident};
+
+use quote::quote;
+
+use proc_macro2::{TokenStream, Span};
+
+/// The derive implementation for `PassBy` with `Enum`.
+pub fn derive_impl(input: DeriveInput) -> Result {
+	let crate_include = generate_runtime_interface_include();
+	let crate_ = generate_crate_access();
+	let ident = input.ident;
+	let enum_fields = get_enum_field_idents(&input.data)?
+		.enumerate()
+		.map(|(i, v)| {
+			let i = i as u8;
+
+			v.map(|v| (quote!(#i => Ok(#ident::#v)), quote!(#ident::#v => #i)))
+		})
+		.collect::>>()?;
+	let try_from_variants = enum_fields.iter().map(|i| &i.0);
+	let into_variants = enum_fields.iter().map(|i| &i.1);
+
+	let res = quote! {
+		const _: () = {
+			#crate_include
+
+			impl #crate_::pass_by::PassBy for #ident {
+				type PassBy = #crate_::pass_by::Enum<#ident>;
+			}
+
+			impl #crate_::rstd::convert::TryFrom for #ident {
+				type Error = ();
+
+				fn try_from(inner: u8) -> #crate_::rstd::result::Result {
+					match inner {
+						#( #try_from_variants, )*
+						_ => Err(()),
+					}
+				}
+			}
+
+			impl From<#ident> for u8 {
+				fn from(var: #ident) -> u8 {
+					match var {
+						#( #into_variants ),*
+					}
+				}
+			}
+		};
+	};
+
+	Ok(res)
+}
+
+/// Get the enum fields idents of the given `data` object as iterator.
+///
+/// Returns an error if the number of variants is greater than `256`, the given `data` is not an
+/// enum or a variant is not an unit.
+fn get_enum_field_idents<'a>(data: &'a Data) -> Result>> {
+	match data {
+		Data::Enum(d) => {
+			if d.variants.len() <= 256 {
+				Ok(
+					d.variants.iter().map(|v| if let Fields::Unit = v.fields {
+						Ok(&v.ident)
+					} else {
+						Err(Error::new(
+							Span::call_site(),
+							"`PassByEnum` only supports unit variants.",
+						))
+					})
+				)
+			} else {
+				Err(Error::new(Span::call_site(), "`PassByEnum` only supports `256` variants."))
+			}
+		},
+		_ => Err(Error::new(Span::call_site(), "`PassByEnum` only supports enums as input type."))
+	}
+}
diff --git a/primitives/runtime-interface/proc-macro/src/pass_by/inner.rs b/primitives/runtime-interface/proc-macro/src/pass_by/inner.rs
new file mode 100644
index 0000000000000..0ca8cd0b5cd91
--- /dev/null
+++ b/primitives/runtime-interface/proc-macro/src/pass_by/inner.rs
@@ -0,0 +1,110 @@
+// Copyright 2019 Parity Technologies (UK) Ltd.
+// This file is part of Substrate.
+
+// Substrate is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+
+// Substrate is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+
+// You should have received a copy of the GNU General Public License
+// along with Substrate.  If not, see .
+
+//! Derive macro implementation of `PassBy` with the associated type set to `Inner` and of the
+//! helper trait `PassByInner`.
+//!
+//! It is required that the type is a newtype struct, otherwise an error is generated.
+
+use crate::utils::{generate_crate_access, generate_runtime_interface_include};
+
+use syn::{DeriveInput, Result, Generics, parse_quote, Type, Data, Error, Fields, Ident};
+
+use quote::quote;
+
+use proc_macro2::{TokenStream, Span};
+
+/// The derive implementation for `PassBy` with `Inner` and `PassByInner`.
+pub fn derive_impl(mut input: DeriveInput) -> Result {
+	add_trait_bounds(&mut input.generics);
+	let (impl_generics, ty_generics, where_clause) = input.generics.split_for_impl();
+	let crate_include = generate_runtime_interface_include();
+	let crate_ = generate_crate_access();
+	let ident = input.ident;
+	let (inner_ty, inner_name) = extract_inner_ty_and_name(&input.data)?;
+
+	let access_inner = match inner_name {
+		Some(ref name) => quote!(self.#name),
+		None => quote!(self.0),
+	};
+
+	let from_inner = match inner_name {
+		Some(name) => quote!(Self { #name: inner }),
+		None => quote!(Self(inner)),
+	};
+
+	let res = quote! {
+		const _: () = {
+			#crate_include
+
+			impl #impl_generics #crate_::pass_by::PassBy for #ident #ty_generics #where_clause {
+				type PassBy = #crate_::pass_by::Inner<#ident, #inner_ty>;
+			}
+
+			impl #impl_generics #crate_::pass_by::PassByInner for #ident #ty_generics #where_clause {
+				type Inner = #inner_ty;
+
+				fn into_inner(self) -> Self::Inner {
+					#access_inner
+				}
+
+				fn inner(&self) -> &Self::Inner {
+					&#access_inner
+				}
+
+				fn from_inner(inner: Self::Inner) -> Self {
+					#from_inner
+				}
+			}
+		};
+	};
+
+	Ok(res)
+}
+
+/// Add the `RIType` trait bound to every type parameter.
+fn add_trait_bounds(generics: &mut Generics) {
+	let crate_ = generate_crate_access();
+
+	generics.type_params_mut()
+		.for_each(|type_param| type_param.bounds.push(parse_quote!(#crate_::RIType)));
+}
+
+/// Extract the inner type and optional name from given input data.
+///
+/// It also checks that the input data is a newtype struct.
+fn extract_inner_ty_and_name(data: &Data) -> Result<(Type, Option)> {
+	if let Data::Struct(ref struct_data) = data {
+		match struct_data.fields {
+			Fields::Named(ref named) if named.named.len() == 1 => {
+				let field = &named.named[0];
+				return Ok((field.ty.clone(), field.ident.clone()))
+			},
+			Fields::Unnamed(ref unnamed) if unnamed.unnamed.len() == 1 => {
+				let field = &unnamed.unnamed[0];
+				return Ok((field.ty.clone(), field.ident.clone()))
+			}
+			_ => {},
+		}
+	}
+
+	Err(
+		Error::new(
+			Span::call_site(),
+			"Only newtype/one field structs are supported by `PassByInner`!",
+		)
+	)
+}
diff --git a/primitives/runtime-interface/proc-macro/src/pass_by/mod.rs b/primitives/runtime-interface/proc-macro/src/pass_by/mod.rs
new file mode 100644
index 0000000000000..23d511183293c
--- /dev/null
+++ b/primitives/runtime-interface/proc-macro/src/pass_by/mod.rs
@@ -0,0 +1,25 @@
+// Copyright 2019 Parity Technologies (UK) Ltd.
+// This file is part of Substrate.
+
+// Substrate is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+
+// Substrate is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+
+// You should have received a copy of the GNU General Public License
+// along with Substrate.  If not, see .
+
+//! All the `PassBy*` derive implementations.
+
+mod codec;
+mod enum_;
+mod inner;
+
+pub use self::codec::derive_impl as codec_derive_impl;
+pub use enum_::derive_impl as enum_derive_impl;
+pub use inner::derive_impl as inner_derive_impl;
diff --git a/primitives/runtime-interface/proc-macro/src/runtime_interface/bare_function_interface.rs b/primitives/runtime-interface/proc-macro/src/runtime_interface/bare_function_interface.rs
new file mode 100644
index 0000000000000..dbedba000e262
--- /dev/null
+++ b/primitives/runtime-interface/proc-macro/src/runtime_interface/bare_function_interface.rs
@@ -0,0 +1,186 @@
+// Copyright 2019 Parity Technologies (UK) Ltd.
+// This file is part of Substrate.
+
+// Substrate is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+
+// Substrate is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+
+// You should have received a copy of the GNU General Public License
+// along with Substrate.  If not, see .
+
+//! Generates the bare function interface for a given trait definition.
+//!
+//! The bare functions are the ones that will be called by the user. On the native/host side, these
+//! functions directly execute the provided implementation. On the wasm side, these
+//! functions will prepare the parameters for the FFI boundary, call the external host function
+//! exported into wasm and convert back the result.
+//!
+//! [`generate`](bare_function_interface::generate) is the entry point for generating for each
+//! trait method one bare function.
+//!
+//! [`function_for_method`](bare_function_interface::function_for_method) generates the bare
+//! function per trait method. Each bare function contains both implementations. The implementations
+//! are feature-gated, so that one is compiled for the native and the other for the wasm side.
+
+use crate::utils::{
+	generate_crate_access, create_exchangeable_host_function_ident, get_function_arguments,
+	get_function_argument_names, get_trait_methods,
+};
+
+use syn::{
+	Ident, ItemTrait, TraitItemMethod, FnArg, Signature, Result, spanned::Spanned, parse_quote,
+};
+
+use proc_macro2::{TokenStream, Span};
+
+use quote::{quote, quote_spanned};
+
+use std::iter;
+
+/// Generate one bare function per trait method. The name of the bare function is equal to the name
+/// of the trait method.
+pub fn generate(trait_def: &ItemTrait, is_wasm_only: bool) -> Result {
+	let trait_name = &trait_def.ident;
+	get_trait_methods(trait_def).try_fold(TokenStream::new(), |mut t, m| {
+		t.extend(function_for_method(trait_name, m, is_wasm_only)?);
+		Ok(t)
+	})
+}
+
+/// Generates the bare function implementation for the given method for the host and wasm side.
+fn function_for_method(
+	trait_name: &Ident,
+	method: &TraitItemMethod,
+	is_wasm_only: bool,
+) -> Result {
+	let std_impl = function_std_impl(trait_name, method, is_wasm_only)?;
+	let no_std_impl = function_no_std_impl(method)?;
+
+	Ok(
+		quote! {
+			#std_impl
+
+			#no_std_impl
+		}
+	)
+}
+
+/// Generates the bare function implementation for `cfg(not(feature = "std"))`.
+fn function_no_std_impl(method: &TraitItemMethod) -> Result {
+	let function_name = &method.sig.ident;
+	let host_function_name = create_exchangeable_host_function_ident(&method.sig.ident);
+	let args = get_function_arguments(&method.sig);
+	let arg_names = get_function_argument_names(&method.sig);
+	let return_value = &method.sig.output;
+	let attrs = &method.attrs;
+
+	Ok(
+		quote! {
+			#[cfg(not(feature = "std"))]
+			#( #attrs )*
+			pub fn #function_name( #( #args, )* ) #return_value {
+				// Call the host function
+				#host_function_name.get()( #( #arg_names, )* )
+			}
+		}
+	)
+}
+
+/// Generates the bare function implementation for `cfg(feature = "std")`.
+fn function_std_impl(
+	trait_name: &Ident,
+	method: &TraitItemMethod,
+	is_wasm_only: bool,
+) -> Result {
+	let function_name = &method.sig.ident;
+	let crate_ = generate_crate_access();
+	let args = get_function_arguments(&method.sig).map(FnArg::Typed).chain(
+		// Add the function context as last parameter when this is a wasm only interface.
+		iter::from_fn(||
+			if is_wasm_only {
+				Some(
+					parse_quote!(
+						mut __function_context__: &mut dyn #crate_::wasm_interface::FunctionContext
+					)
+				)
+			} else {
+				None
+			}
+		).take(1),
+	);
+	let return_value = &method.sig.output;
+	let attrs = &method.attrs;
+	// Don't make the function public accessible when this is a wasm only interface.
+	let vis = if is_wasm_only { quote!() } else { quote!(pub) };
+	let call_to_trait = generate_call_to_trait(trait_name, method, is_wasm_only);
+
+	Ok(
+		quote_spanned! { method.span() =>
+			#[cfg(feature = "std")]
+			#( #attrs )*
+			#vis fn #function_name( #( #args, )* ) #return_value {
+				#call_to_trait
+			}
+		}
+	)
+}
+
+/// Generate the call to the interface trait.
+fn generate_call_to_trait(
+	trait_name: &Ident,
+	method: &TraitItemMethod,
+	is_wasm_only: bool,
+) -> TokenStream {
+	let crate_ = generate_crate_access();
+	let method_name = &method.sig.ident;
+	let expect_msg = format!(
+		"`{}` called outside of an Externalities-provided environment.",
+		method_name,
+	);
+	let arg_names = get_function_argument_names(&method.sig);
+
+	if takes_self_argument(&method.sig) {
+		let instance = if is_wasm_only {
+			Ident::new("__function_context__", Span::call_site())
+		} else {
+			Ident::new("__externalities__", Span::call_site())
+		};
+
+		let impl_ = quote!( #trait_name::#method_name(&mut #instance, #( #arg_names, )*) );
+
+		if is_wasm_only {
+			quote_spanned! { method.span() => #impl_ }
+		} else {
+			quote_spanned! { method.span() =>
+				#crate_::with_externalities(|mut #instance| #impl_).expect(#expect_msg)
+			}
+		}
+	} else {
+		// The name of the trait the interface trait is implemented for
+		let impl_trait_name = if is_wasm_only {
+			quote!( #crate_::wasm_interface::FunctionContext )
+		} else {
+			quote!( #crate_::Externalities )
+		};
+
+		quote_spanned! { method.span() =>
+			<&mut dyn #impl_trait_name as #trait_name>::#method_name(
+				#( #arg_names, )*
+			)
+		}
+	}
+}
+
+/// Returns if the given `Signature` takes a `self` argument.
+fn takes_self_argument(sig: &Signature) -> bool {
+	match sig.inputs.first() {
+		Some(FnArg::Receiver(_)) => true,
+		_ => false,
+	}
+}
diff --git a/primitives/runtime-interface/proc-macro/src/runtime_interface/host_function_interface.rs b/primitives/runtime-interface/proc-macro/src/runtime_interface/host_function_interface.rs
new file mode 100644
index 0000000000000..f710e9b60cbf4
--- /dev/null
+++ b/primitives/runtime-interface/proc-macro/src/runtime_interface/host_function_interface.rs
@@ -0,0 +1,415 @@
+// Copyright 2019 Parity Technologies (UK) Ltd.
+// This file is part of Substrate.
+
+// Substrate is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+
+// Substrate is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+
+// You should have received a copy of the GNU General Public License
+// along with Substrate.  If not, see .
+
+//! Generates the extern host functions and the implementation for these host functions.
+//!
+//! The extern host functions will be called by the bare function interface from the Wasm side.
+//! The implementation of these host functions will be called on the host side from the Wasm
+//! executor. These implementations call the bare function interface.
+
+use crate::utils::{
+	generate_crate_access, create_host_function_ident, get_function_argument_names,
+	get_function_argument_types_without_ref, get_function_argument_types_ref_and_mut,
+	get_function_argument_names_and_types_without_ref, get_trait_methods, get_function_arguments,
+	get_function_argument_types, create_exchangeable_host_function_ident,
+};
+
+use syn::{
+	ItemTrait, TraitItemMethod, Result, ReturnType, Ident, TraitItem, Pat, Error, Signature,
+	spanned::Spanned,
+};
+
+use proc_macro2::{TokenStream, Span};
+
+use quote::{quote, ToTokens};
+
+use inflector::Inflector;
+
+use std::iter::{Iterator, self};
+
+/// Generate the extern host functions for wasm and the `HostFunctions` struct that provides the
+/// implementations for the host functions on the host.
+pub fn generate(trait_def: &ItemTrait, is_wasm_only: bool) -> Result {
+	let trait_name = &trait_def.ident;
+	let extern_host_function_impls = get_trait_methods(trait_def)
+		.try_fold(TokenStream::new(), |mut t, m| {
+			t.extend(generate_extern_host_function(m, trait_name)?);
+			Ok::<_, Error>(t)
+		})?;
+	let exchangeable_host_functions = get_trait_methods(trait_def)
+		.try_fold(TokenStream::new(), |mut t, m| {
+			t.extend(generate_exchangeable_host_function(m)?);
+			Ok::<_, Error>(t)
+		})?;
+	let host_functions_struct = generate_host_functions_struct(trait_def, is_wasm_only)?;
+
+	Ok(
+		quote! {
+			/// The implementations of the extern host functions. This special implementation module
+			/// is required to change the extern host functions signature to
+			/// `unsafe fn name(args) -> ret` to make the function implementations exchangeable.
+			#[cfg(not(feature = "std"))]
+			mod extern_host_function_impls {
+				use super::*;
+
+				#extern_host_function_impls
+			}
+
+			#exchangeable_host_functions
+
+			#host_functions_struct
+		}
+	)
+}
+
+/// Generate the extern host function for the given method.
+fn generate_extern_host_function(method: &TraitItemMethod, trait_name: &Ident) -> Result {
+	let crate_ = generate_crate_access();
+	let args = get_function_arguments(&method.sig);
+	let arg_types = get_function_argument_types_without_ref(&method.sig);
+	let arg_types2 = get_function_argument_types_without_ref(&method.sig);
+	let arg_names = get_function_argument_names(&method.sig);
+	let arg_names2 = get_function_argument_names(&method.sig);
+	let arg_names3 = get_function_argument_names(&method.sig);
+	let function = &method.sig.ident;
+	let ext_function = create_host_function_ident(&method.sig.ident, trait_name);
+	let doc_string = format!(
+		" Default extern host function implementation for [`super::{}`].",
+		method.sig.ident,
+	);
+	let return_value = &method.sig.output;
+
+	let ffi_return_value = match method.sig.output {
+		ReturnType::Default => quote!(),
+		ReturnType::Type(_, ref ty) => quote! {
+			-> <#ty as #crate_::RIType>::FFIType
+		},
+	};
+
+	let convert_return_value = match return_value {
+		ReturnType::Default => quote!(),
+		ReturnType::Type(_, ref ty) => quote! {
+			<#ty as #crate_::wasm::FromFFIValue>::from_ffi_value(result)
+		}
+	};
+
+	Ok(
+		quote! {
+			#[doc = #doc_string]
+			pub fn #function ( #( #args ),* ) #return_value {
+				extern "C" {
+					/// The extern function.
+					pub fn #ext_function (
+						#( #arg_names: <#arg_types as #crate_::RIType>::FFIType ),*
+					) #ffi_return_value;
+				}
+
+				// Generate all wrapped ffi values.
+				#(
+					let #arg_names2 = <#arg_types2 as #crate_::wasm::IntoFFIValue>::into_ffi_value(
+						&#arg_names2,
+					);
+				)*
+
+				let result = unsafe { #ext_function( #( #arg_names3.get() ),* ) };
+
+				#convert_return_value
+			}
+		}
+	)
+}
+
+/// Generate the host exchangeable function for the given method.
+fn generate_exchangeable_host_function(method: &TraitItemMethod) -> Result {
+	let crate_ = generate_crate_access();
+	let arg_types = get_function_argument_types(&method.sig);
+	let function = &method.sig.ident;
+	let exchangeable_function = create_exchangeable_host_function_ident(&method.sig.ident);
+	let doc_string = format!(" Exchangeable host function used by [`{}`].", method.sig.ident);
+	let output = &method.sig.output;
+
+	Ok(
+		quote! {
+			#[cfg(not(feature = "std"))]
+			#[allow(non_upper_case_globals)]
+			#[doc = #doc_string]
+			pub static #exchangeable_function : #crate_::wasm::ExchangeableFunction<
+				fn ( #( #arg_types ),* ) #output
+			> = #crate_::wasm::ExchangeableFunction::new(extern_host_function_impls::#function);
+		}
+	)
+}
+
+/// Generate the `HostFunctions` struct that implements `wasm-interface::HostFunctions` to provide
+/// implementations for the extern host functions.
+fn generate_host_functions_struct(trait_def: &ItemTrait, is_wasm_only: bool) -> Result {
+	let crate_ = generate_crate_access();
+	let host_functions = trait_def
+		.items
+		.iter()
+		.filter_map(|i| match i {
+			TraitItem::Method(ref method) => Some(method),
+			_ => None,
+		})
+		.map(|m| generate_host_function_implementation(&trait_def.ident, m, is_wasm_only))
+		.collect::>>()?;
+
+	Ok(
+		quote! {
+			/// Provides implementations for the extern host functions.
+			#[cfg(feature = "std")]
+			pub struct HostFunctions;
+
+			#[cfg(feature = "std")]
+			impl #crate_::wasm_interface::HostFunctions for HostFunctions {
+				fn host_functions() -> Vec<&'static dyn #crate_::wasm_interface::Function> {
+					vec![ #( #host_functions ),* ]
+				}
+			}
+		}
+	)
+}
+
+/// Generates the host function struct that implements `wasm_interface::Function` and returns a static
+/// reference to this struct.
+///
+/// When calling from wasm into the host, we will call the `execute` function that calls the native
+/// implementation of the function.
+fn generate_host_function_implementation(
+	trait_name: &Ident,
+	method: &TraitItemMethod,
+	is_wasm_only: bool,
+) -> Result {
+	let name = create_host_function_ident(&method.sig.ident, trait_name).to_string();
+	let struct_name = Ident::new(&name.to_pascal_case(), Span::call_site());
+	let crate_ = generate_crate_access();
+	let signature = generate_wasm_interface_signature_for_host_function(&method.sig)?;
+	let wasm_to_ffi_values = generate_wasm_to_ffi_values(
+		&method.sig,
+		trait_name,
+	).collect::>>()?;
+	let ffi_to_host_values = generate_ffi_to_host_value(&method.sig).collect::>>()?;
+	let host_function_call = generate_host_function_call(&method.sig, is_wasm_only);
+	let into_preallocated_ffi_value = generate_into_preallocated_ffi_value(&method.sig)?;
+	let convert_return_value = generate_return_value_into_wasm_value(&method.sig);
+
+	Ok(
+		quote! {
+			{
+				struct #struct_name;
+
+				#[allow(unused)]
+				impl #crate_::wasm_interface::Function for #struct_name {
+					fn name(&self) -> &str {
+						#name
+					}
+
+					fn signature(&self) -> #crate_::wasm_interface::Signature {
+						#signature
+					}
+
+					fn execute(
+						&self,
+						__function_context__: &mut dyn #crate_::wasm_interface::FunctionContext,
+						args: &mut dyn Iterator,
+					) -> std::result::Result, String> {
+						#( #wasm_to_ffi_values )*
+						#( #ffi_to_host_values )*
+						#host_function_call
+						#into_preallocated_ffi_value
+						#convert_return_value
+					}
+				}
+
+				&#struct_name as &dyn #crate_::wasm_interface::Function
+			}
+		}
+	)
+}
+
+/// Generate the `wasm_interface::Signature` for the given host function `sig`.
+fn generate_wasm_interface_signature_for_host_function(sig: &Signature) -> Result {
+	let crate_ = generate_crate_access();
+	let return_value = match &sig.output {
+		ReturnType::Type(_, ty) =>
+			quote! {
+				Some( <<#ty as #crate_::RIType>::FFIType as #crate_::wasm_interface::IntoValue>::VALUE_TYPE )
+			},
+		ReturnType::Default => quote!( None ),
+	};
+	let arg_types = get_function_argument_types_without_ref(sig)
+		.map(|ty| quote! {
+			<<#ty as #crate_::RIType>::FFIType as #crate_::wasm_interface::IntoValue>::VALUE_TYPE
+		});
+
+	Ok(
+		quote! {
+			#crate_::wasm_interface::Signature {
+				args: std::borrow::Cow::Borrowed(&[ #( #arg_types ),* ][..]),
+				return_value: #return_value,
+			}
+		}
+	)
+}
+
+/// Generate the code that converts the wasm values given to `HostFunctions::execute` into the FFI
+/// values.
+fn generate_wasm_to_ffi_values<'a>(
+	sig: &'a Signature,
+	trait_name: &'a Ident,
+) -> impl Iterator> + 'a {
+	let crate_ = generate_crate_access();
+	let function_name = &sig.ident;
+	let error_message = format!(
+		"Number of arguments given to `{}` does not match the expected number of arguments!",
+		function_name,
+	);
+
+	get_function_argument_names_and_types_without_ref(sig)
+		.map(move |(name, ty)| {
+			let try_from_error = format!(
+				"Could not instantiate `{}` from wasm value while executing `{}` from interface `{}`!",
+				name.to_token_stream(),
+				function_name,
+				trait_name,
+			);
+
+			let var_name = generate_ffi_value_var_name(&name)?;
+
+			Ok(quote! {
+				let val = args.next().ok_or_else(|| #error_message)?;
+				let #var_name = <
+					<#ty as #crate_::RIType>::FFIType as #crate_::wasm_interface::TryFromValue
+				>::try_from_value(val).ok_or_else(|| #try_from_error)?;
+			})
+		})
+}
+
+/// Generate the code to convert the ffi values on the host to the host values using `FromFFIValue`.
+fn generate_ffi_to_host_value<'a>(
+	sig: &'a Signature,
+) -> impl Iterator> + 'a {
+	let mut_access = get_function_argument_types_ref_and_mut(sig);
+	let crate_ = generate_crate_access();
+
+	get_function_argument_names_and_types_without_ref(sig)
+		.zip(mut_access.map(|v| v.and_then(|m| m.1)))
+		.map(move |((name, ty), mut_access)| {
+			let ffi_value_var_name = generate_ffi_value_var_name(&name)?;
+
+			Ok(
+				quote! {
+					let #mut_access #name = <#ty as #crate_::host::FromFFIValue>::from_ffi_value(
+						__function_context__,
+						#ffi_value_var_name,
+					)?;
+				}
+			)
+		})
+}
+
+/// Generate the code to call the host function and the ident that stores the result.
+fn generate_host_function_call(sig: &Signature, is_wasm_only: bool) -> TokenStream {
+	let host_function_name = &sig.ident;
+	let result_var_name = generate_host_function_result_var_name(&sig.ident);
+	let ref_and_mut = get_function_argument_types_ref_and_mut(sig).map(|ram|
+		ram.map(|(vr, vm)| quote!(#vr #vm))
+	);
+	let names = get_function_argument_names(sig);
+
+	let var_access = names.zip(ref_and_mut)
+		.map(|(n, ref_and_mut)| {
+			quote!( #ref_and_mut #n )
+		})
+		// If this is a wasm only interface, we add the function context as last parameter.
+		.chain(
+			iter::from_fn(|| if is_wasm_only { Some(quote!(__function_context__)) } else { None })
+				.take(1)
+		);
+
+	quote! {
+		let #result_var_name = #host_function_name ( #( #var_access ),* );
+	}
+}
+
+/// Generate the variable name that stores the result of the host function.
+fn generate_host_function_result_var_name(name: &Ident) -> Ident {
+	Ident::new(&format!("{}_result", name), Span::call_site())
+}
+
+/// Generate the variable name that stores the FFI value.
+fn generate_ffi_value_var_name(pat: &Pat) -> Result {
+	match pat {
+		Pat::Ident(pat_ident) => {
+			if let Some(by_ref) = pat_ident.by_ref {
+				Err(Error::new(by_ref.span(), "`ref` not supported!"))
+			} else if let Some(sub_pattern) = &pat_ident.subpat {
+				Err(Error::new(sub_pattern.0.span(), "Not supported!"))
+			} else {
+				Ok(Ident::new(&format!("{}_ffi_value", pat_ident.ident), Span::call_site()))
+			}
+		}
+		_ => Err(Error::new(pat.span(), "Not supported as variable name!"))
+	}
+}
+
+/// Generate code that copies data from the host back to preallocated wasm memory.
+///
+/// Any argument that is given as `&mut` is interpreted as preallocated memory and it is expected
+/// that the type implements `IntoPreAllocatedFFIValue`.
+fn generate_into_preallocated_ffi_value(sig: &Signature) -> Result {
+	let crate_ = generate_crate_access();
+	let ref_and_mut = get_function_argument_types_ref_and_mut(sig).map(|ram|
+		ram.and_then(|(vr, vm)| vm.map(|v| (vr, v)))
+	);
+	let names_and_types = get_function_argument_names_and_types_without_ref(sig);
+
+	ref_and_mut.zip(names_and_types)
+		.filter_map(|(ram, (name, ty))| ram.map(|_| (name, ty)))
+		.map(|(name, ty)| {
+			let ffi_var_name = generate_ffi_value_var_name(&name)?;
+
+			Ok(
+				quote! {
+					<#ty as #crate_::host::IntoPreallocatedFFIValue>::into_preallocated_ffi_value(
+						#name,
+						__function_context__,
+						#ffi_var_name,
+					)?;
+				}
+			)
+		})
+		.collect()
+}
+
+/// Generate the code that converts the return value into the appropriate wasm value.
+fn generate_return_value_into_wasm_value(sig: &Signature) -> TokenStream {
+	let crate_ = generate_crate_access();
+
+	match &sig.output {
+		ReturnType::Default => quote!( Ok(None) ),
+		ReturnType::Type(_, ty) => {
+			let result_var_name = generate_host_function_result_var_name(&sig.ident);
+
+			quote! {
+				<#ty as #crate_::host::IntoFFIValue>::into_ffi_value(
+					#result_var_name,
+					__function_context__,
+				).map(#crate_::wasm_interface::IntoValue::into_value).map(Some)
+			}
+		}
+	}
+}
diff --git a/primitives/runtime-interface/proc-macro/src/runtime_interface/mod.rs b/primitives/runtime-interface/proc-macro/src/runtime_interface/mod.rs
new file mode 100644
index 0000000000000..142fff646ed18
--- /dev/null
+++ b/primitives/runtime-interface/proc-macro/src/runtime_interface/mod.rs
@@ -0,0 +1,65 @@
+// Copyright 2019 Parity Technologies (UK) Ltd.
+// This file is part of Substrate.
+
+// Substrate is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+
+// Substrate is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+
+// You should have received a copy of the GNU General Public License
+// along with Substrate.  If not, see .
+
+use crate::utils::generate_runtime_interface_include;
+
+use proc_macro2::{Span, TokenStream};
+
+use syn::{Ident, ItemTrait, Result};
+
+use inflector::Inflector;
+
+use quote::quote;
+
+mod bare_function_interface;
+mod host_function_interface;
+mod trait_decl_impl;
+
+/// Custom keywords supported by the `runtime_interface` attribute.
+pub mod keywords {
+	// Custom keyword `wasm_only` that can be given as attribute to [`runtime_interface`].
+	syn::custom_keyword!(wasm_only);
+}
+
+/// Implementation of the `runtime_interface` attribute.
+///
+/// It expects the trait definition the attribute was put above and if this should be an wasm only
+/// interface.
+pub fn runtime_interface_impl(trait_def: ItemTrait, is_wasm_only: bool) -> Result {
+	let bare_functions = bare_function_interface::generate(&trait_def, is_wasm_only)?;
+	let crate_include = generate_runtime_interface_include();
+	let mod_name = Ident::new(&trait_def.ident.to_string().to_snake_case(), Span::call_site());
+	let trait_decl_impl = trait_decl_impl::process(&trait_def, is_wasm_only)?;
+	let host_functions = host_function_interface::generate(&trait_def, is_wasm_only)?;
+	let vis = trait_def.vis;
+	let attrs = &trait_def.attrs;
+
+	let res = quote! {
+		#( #attrs )*
+		#vis mod #mod_name {
+			use super::*;
+			#crate_include
+
+			#bare_functions
+
+			#trait_decl_impl
+
+			#host_functions
+		}
+	};
+
+	Ok(res)
+}
diff --git a/primitives/runtime-interface/proc-macro/src/runtime_interface/trait_decl_impl.rs b/primitives/runtime-interface/proc-macro/src/runtime_interface/trait_decl_impl.rs
new file mode 100644
index 0000000000000..0e5ae906ab680
--- /dev/null
+++ b/primitives/runtime-interface/proc-macro/src/runtime_interface/trait_decl_impl.rs
@@ -0,0 +1,146 @@
+// Copyright 2019 Parity Technologies (UK) Ltd.
+// This file is part of Substrate.
+
+// Substrate is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+
+// Substrate is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+
+// You should have received a copy of the GNU General Public License
+// along with Substrate.  If not, see .
+
+//! Checks the trait declaration, makes the trait declaration module local, removes all method
+//! default implementations and implements the trait for `&mut dyn Externalities`.
+
+use crate::utils::{generate_crate_access, get_function_argument_types_without_ref};
+
+use syn::{
+	ItemTrait, TraitItemMethod, Result, TraitItem, Error, fold::{self, Fold}, spanned::Spanned,
+	Visibility, Receiver, Type, Generics,
+};
+
+use proc_macro2::TokenStream;
+
+use quote::quote;
+
+/// Process the given trait definition, by checking that the definition is valid, fold it to the
+/// essential definition and implement this essential definition for `dyn Externalities`.
+pub fn process(trait_def: &ItemTrait, is_wasm_only: bool) -> Result {
+	let impl_trait = impl_trait_for_externalities(trait_def, is_wasm_only)?;
+	let essential_trait_def = ToEssentialTraitDef::convert(trait_def.clone())?;
+
+	Ok(
+		quote! {
+			#impl_trait
+
+			#essential_trait_def
+		}
+	)
+}
+
+/// Converts the given trait definition into the essential trait definition without method
+/// default implementations and visibility set to inherited.
+struct ToEssentialTraitDef {
+	/// All errors found while doing the conversion.
+	errors: Vec,
+}
+
+impl ToEssentialTraitDef {
+	/// Convert the given trait definition to the essential trait definition.
+	fn convert(trait_def: ItemTrait) -> Result {
+		let mut folder = ToEssentialTraitDef {
+			errors: Vec::new(),
+		};
+
+		let res = folder.fold_item_trait(trait_def);
+
+		if let Some(first_error) = folder.errors.pop() {
+			Err(
+				folder.errors.into_iter().fold(first_error, |mut o, n| {
+					o.combine(n);
+					o
+				})
+			)
+		} else {
+			Ok(res)
+		}
+	}
+
+	fn push_error(&mut self, span: &S, msg: &str) {
+		self.errors.push(Error::new(span.span(), msg));
+	}
+
+	fn error_on_generic_parameters(&mut self, generics: &Generics) {
+		if let Some(param) = generics.params.first() {
+			self.push_error(param, "Generic parameters not supported.");
+		}
+	}
+}
+
+impl Fold for ToEssentialTraitDef {
+	fn fold_trait_item_method(&mut self, mut method: TraitItemMethod) -> TraitItemMethod {
+		if method.default.take().is_none() {
+			self.push_error(&method, "Methods need to have an implementation.");
+		}
+
+		let arg_types = get_function_argument_types_without_ref(&method.sig);
+		arg_types.filter_map(|ty|
+			match *ty {
+				Type::ImplTrait(impl_trait) => Some(impl_trait),
+				_ => None
+			}
+		).for_each(|invalid| self.push_error(&invalid, "`impl Trait` syntax not supported."));
+
+		self.error_on_generic_parameters(&method.sig.generics);
+
+		fold::fold_trait_item_method(self, method)
+	}
+
+	fn fold_item_trait(&mut self, mut trait_def: ItemTrait) -> ItemTrait {
+		self.error_on_generic_parameters(&trait_def.generics);
+
+		trait_def.vis = Visibility::Inherited;
+		fold::fold_item_trait(self, trait_def)
+	}
+
+	fn fold_receiver(&mut self, receiver: Receiver) -> Receiver {
+		if receiver.reference.is_none() {
+			self.push_error(&receiver, "Taking `Self` by value is not allowed.");
+		}
+
+		fold::fold_receiver(self, receiver)
+	}
+}
+
+/// Implements the given trait definition for `dyn Externalities`.
+fn impl_trait_for_externalities(trait_def: &ItemTrait, is_wasm_only: bool) -> Result {
+	let trait_ = &trait_def.ident;
+	let crate_ = generate_crate_access();
+	let methods = trait_def
+		.items
+		.iter()
+		.filter_map(|i| match i {
+			TraitItem::Method(ref method) => Some(method),
+			_ => None,
+		});
+
+	let impl_type = if is_wasm_only {
+		quote!( &mut dyn #crate_::wasm_interface::FunctionContext )
+	} else {
+		quote!( &mut dyn #crate_::Externalities )
+	};
+
+	Ok(
+		quote! {
+			#[cfg(feature = "std")]
+			impl #trait_ for #impl_type {
+				#( #methods )*
+			}
+		}
+	)
+}
diff --git a/primitives/runtime-interface/proc-macro/src/utils.rs b/primitives/runtime-interface/proc-macro/src/utils.rs
new file mode 100644
index 0000000000000..d868452dcec2a
--- /dev/null
+++ b/primitives/runtime-interface/proc-macro/src/utils.rs
@@ -0,0 +1,162 @@
+// Copyright 2019 Parity Technologies (UK) Ltd.
+// This file is part of Substrate.
+
+// Substrate is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+
+// Substrate is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+
+// You should have received a copy of the GNU General Public License
+// along with Substrate.  If not, see 
+
+//! Util function used by this crate.
+
+use proc_macro2::{TokenStream, Span};
+
+use syn::{
+	Ident, Error, Signature, Pat, PatType, FnArg, Type, token, TraitItemMethod, ItemTrait,
+	TraitItem, parse_quote, spanned::Spanned,
+};
+
+use proc_macro_crate::crate_name;
+
+use std::env;
+
+use quote::quote;
+
+use inflector::Inflector;
+
+/// Generates the include for the runtime-interface crate.
+pub fn generate_runtime_interface_include() -> TokenStream {
+	if env::var("CARGO_PKG_NAME").unwrap() == "substrate-runtime-interface" {
+		TokenStream::new()
+	} else {
+		match crate_name("substrate-runtime-interface") {
+			Ok(crate_name) => {
+				let crate_name = Ident::new(&crate_name, Span::call_site());
+				quote!(
+					#[doc(hidden)]
+					extern crate #crate_name as proc_macro_runtime_interface;
+				)
+			},
+			Err(e) => {
+				let err = Error::new(Span::call_site(), &e).to_compile_error();
+				quote!( #err )
+			}
+		}
+	}
+}
+
+/// Generates the access to the `substrate-runtime-interface` crate.
+pub fn generate_crate_access() -> TokenStream {
+	if env::var("CARGO_PKG_NAME").unwrap() == "substrate-runtime-interface" {
+		quote!( substrate_runtime_interface )
+	} else {
+		quote!( proc_macro_runtime_interface )
+	}
+}
+
+/// Create the exchangeable host function identifier for the given function name.
+pub fn create_exchangeable_host_function_ident(name: &Ident) -> Ident {
+	Ident::new(&format!("host_{}", name), Span::call_site())
+}
+
+/// Create the host function identifier for the given function name.
+pub fn create_host_function_ident(name: &Ident, trait_name: &Ident) -> Ident {
+	Ident::new(
+		&format!(
+			"ext_{}_{}_version_1",
+			trait_name.to_string().to_snake_case(),
+			name,
+		),
+		Span::call_site(),
+	)
+}
+
+/// Returns the function arguments of the given `Signature`, minus any `self` arguments.
+pub fn get_function_arguments<'a>(sig: &'a Signature) -> impl Iterator + 'a {
+	sig.inputs
+		.iter()
+		.filter_map(|a| match a {
+			FnArg::Receiver(_) => None,
+			FnArg::Typed(pat_type) => Some(pat_type),
+		})
+		.enumerate()
+		.map(|(i, arg)| {
+			let mut res = arg.clone();
+			if let Pat::Wild(wild) = &*arg.pat {
+				let ident = Ident::new(
+					&format!("__runtime_interface_generated_{}_", i),
+					wild.span(),
+				);
+
+				res.pat = Box::new(parse_quote!( #ident ))
+			}
+
+			res
+		})
+}
+
+/// Returns the function argument names of the given `Signature`, minus any `self`.
+pub fn get_function_argument_names<'a>(sig: &'a Signature) -> impl Iterator> + 'a {
+	get_function_arguments(sig).map(|pt| pt.pat)
+}
+
+/// Returns the function argument types of the given `Signature`, minus any `Self` type.
+pub fn get_function_argument_types<'a>(sig: &'a Signature) -> impl Iterator> + 'a {
+	get_function_arguments(sig).map(|pt| pt.ty)
+}
+
+/// Returns the function argument types, minus any `Self` type. If any of the arguments
+/// is a reference, the underlying type without the ref is returned.
+pub fn get_function_argument_types_without_ref<'a>(
+	sig: &'a Signature,
+) -> impl Iterator> + 'a {
+	get_function_arguments(sig)
+		.map(|pt| pt.ty)
+		.map(|ty| match *ty {
+			Type::Reference(type_ref) => type_ref.elem,
+			_ => ty,
+		})
+}
+
+/// Returns the function argument names and types, minus any `self`. If any of the arguments
+/// is a reference, the underlying type without the ref is returned.
+pub fn get_function_argument_names_and_types_without_ref<'a>(
+	sig: &'a Signature,
+) -> impl Iterator, Box)> + 'a {
+	get_function_arguments(sig)
+		.map(|pt| match *pt.ty {
+			Type::Reference(type_ref) => (pt.pat, type_ref.elem),
+			_ => (pt.pat, pt.ty),
+		})
+}
+
+/// Returns the `&`/`&mut` for all function argument types, minus the `self` arg. If a function
+/// argument is not a reference, `None` is returned.
+pub fn get_function_argument_types_ref_and_mut<'a>(
+	sig: &'a Signature,
+) -> impl Iterator)>> + 'a {
+	get_function_arguments(sig)
+		.map(|pt| pt.ty)
+		.map(|ty| match *ty {
+			Type::Reference(type_ref) => Some((type_ref.and_token, type_ref.mutability)),
+			_ => None,
+		})
+}
+
+/// Returns an iterator over all trait methods for the given trait definition.
+pub fn get_trait_methods<'a>(trait_def: &'a ItemTrait) -> impl Iterator {
+	trait_def
+		.items
+		.iter()
+		.filter_map(|i| match i {
+			TraitItem::Method(ref method) => Some(method),
+			_ => None,
+		})
+}
diff --git a/primitives/runtime-interface/proc-macro/tests/ui.rs b/primitives/runtime-interface/proc-macro/tests/ui.rs
new file mode 100644
index 0000000000000..5b14ee81e8e8a
--- /dev/null
+++ b/primitives/runtime-interface/proc-macro/tests/ui.rs
@@ -0,0 +1,27 @@
+// Copyright 2019 Parity Technologies (UK) Ltd.
+// This file is part of Substrate.
+
+// Substrate is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+
+// Substrate is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+
+// You should have received a copy of the GNU General Public License
+// along with Substrate.  If not, see .
+
+use std::env;
+
+#[rustversion::attr(not(stable), ignore)]
+#[test]
+fn ui() {
+	// As trybuild is using `cargo check`, we don't need the real WASM binaries.
+	env::set_var("BUILD_DUMMY_WASM_BINARY", "1");
+
+	let t = trybuild::TestCases::new();
+	t.compile_fail("tests/ui/*.rs");
+}
diff --git a/primitives/runtime-interface/proc-macro/tests/ui/no_generic_parameters.rs b/primitives/runtime-interface/proc-macro/tests/ui/no_generic_parameters.rs
new file mode 100644
index 0000000000000..489fe5d9b4f91
--- /dev/null
+++ b/primitives/runtime-interface/proc-macro/tests/ui/no_generic_parameters.rs
@@ -0,0 +1,8 @@
+use runtime_interface::runtime_interface;
+
+#[runtime_interface]
+trait Test {
+	fn test() {}
+}
+
+fn main() {}
diff --git a/primitives/runtime-interface/proc-macro/tests/ui/no_generic_parameters.stderr b/primitives/runtime-interface/proc-macro/tests/ui/no_generic_parameters.stderr
new file mode 100644
index 0000000000000..c3e46655e5be7
--- /dev/null
+++ b/primitives/runtime-interface/proc-macro/tests/ui/no_generic_parameters.stderr
@@ -0,0 +1,11 @@
+error: Generic parameters not supported.
+ --> $DIR/no_generic_parameters.rs:5:10
+  |
+5 |     fn test() {}
+  |             ^
+
+error: Generic parameters not supported.
+ --> $DIR/no_generic_parameters.rs:4:12
+  |
+4 | trait Test {
+  |            ^
diff --git a/primitives/runtime-interface/proc-macro/tests/ui/no_method_implementation.rs b/primitives/runtime-interface/proc-macro/tests/ui/no_method_implementation.rs
new file mode 100644
index 0000000000000..5291942420fd5
--- /dev/null
+++ b/primitives/runtime-interface/proc-macro/tests/ui/no_method_implementation.rs
@@ -0,0 +1,8 @@
+use runtime_interface::runtime_interface;
+
+#[runtime_interface]
+trait Test {
+	fn test();
+}
+
+fn main() {}
diff --git a/primitives/runtime-interface/proc-macro/tests/ui/no_method_implementation.stderr b/primitives/runtime-interface/proc-macro/tests/ui/no_method_implementation.stderr
new file mode 100644
index 0000000000000..31b2d39762340
--- /dev/null
+++ b/primitives/runtime-interface/proc-macro/tests/ui/no_method_implementation.stderr
@@ -0,0 +1,5 @@
+error: Methods need to have an implementation.
+ --> $DIR/no_method_implementation.rs:5:2
+  |
+5 |     fn test();
+  |     ^^
diff --git a/primitives/runtime-interface/proc-macro/tests/ui/pass_by_enum_with_struct.rs b/primitives/runtime-interface/proc-macro/tests/ui/pass_by_enum_with_struct.rs
new file mode 100644
index 0000000000000..a729e0a99adca
--- /dev/null
+++ b/primitives/runtime-interface/proc-macro/tests/ui/pass_by_enum_with_struct.rs
@@ -0,0 +1,6 @@
+use runtime_interface::pass_by::PassByEnum;
+
+#[derive(PassByEnum)]
+struct Test;
+
+fn main() {}
diff --git a/primitives/runtime-interface/proc-macro/tests/ui/pass_by_enum_with_struct.stderr b/primitives/runtime-interface/proc-macro/tests/ui/pass_by_enum_with_struct.stderr
new file mode 100644
index 0000000000000..6502a36fc18b7
--- /dev/null
+++ b/primitives/runtime-interface/proc-macro/tests/ui/pass_by_enum_with_struct.stderr
@@ -0,0 +1,5 @@
+error: `PassByEnum` only supports enums as input type.
+ --> $DIR/pass_by_enum_with_struct.rs:3:10
+  |
+3 | #[derive(PassByEnum)]
+  |          ^^^^^^^^^^
diff --git a/primitives/runtime-interface/proc-macro/tests/ui/pass_by_enum_with_value_variant.rs b/primitives/runtime-interface/proc-macro/tests/ui/pass_by_enum_with_value_variant.rs
new file mode 100644
index 0000000000000..d2558e797770b
--- /dev/null
+++ b/primitives/runtime-interface/proc-macro/tests/ui/pass_by_enum_with_value_variant.rs
@@ -0,0 +1,8 @@
+use runtime_interface::pass_by::PassByEnum;
+
+#[derive(PassByEnum)]
+enum Test {
+	Var0(u32),
+}
+
+fn main() {}
diff --git a/primitives/runtime-interface/proc-macro/tests/ui/pass_by_enum_with_value_variant.stderr b/primitives/runtime-interface/proc-macro/tests/ui/pass_by_enum_with_value_variant.stderr
new file mode 100644
index 0000000000000..1f03436d4e007
--- /dev/null
+++ b/primitives/runtime-interface/proc-macro/tests/ui/pass_by_enum_with_value_variant.stderr
@@ -0,0 +1,5 @@
+error: `PassByEnum` only supports unit variants.
+ --> $DIR/pass_by_enum_with_value_variant.rs:3:10
+  |
+3 | #[derive(PassByEnum)]
+  |          ^^^^^^^^^^
diff --git a/primitives/runtime-interface/proc-macro/tests/ui/pass_by_inner_with_two_fields.rs b/primitives/runtime-interface/proc-macro/tests/ui/pass_by_inner_with_two_fields.rs
new file mode 100644
index 0000000000000..eab79eae1910c
--- /dev/null
+++ b/primitives/runtime-interface/proc-macro/tests/ui/pass_by_inner_with_two_fields.rs
@@ -0,0 +1,9 @@
+use runtime_interface::pass_by::PassByInner;
+
+#[derive(PassByInner)]
+struct Test {
+	data: u32,
+	data2: u32,
+}
+
+fn main() {}
diff --git a/primitives/runtime-interface/proc-macro/tests/ui/pass_by_inner_with_two_fields.stderr b/primitives/runtime-interface/proc-macro/tests/ui/pass_by_inner_with_two_fields.stderr
new file mode 100644
index 0000000000000..7f576a69f0e50
--- /dev/null
+++ b/primitives/runtime-interface/proc-macro/tests/ui/pass_by_inner_with_two_fields.stderr
@@ -0,0 +1,5 @@
+error: Only newtype/one field structs are supported by `PassByInner`!
+ --> $DIR/pass_by_inner_with_two_fields.rs:3:10
+  |
+3 | #[derive(PassByInner)]
+  |          ^^^^^^^^^^^
diff --git a/primitives/runtime-interface/proc-macro/tests/ui/take_self_by_value.rs b/primitives/runtime-interface/proc-macro/tests/ui/take_self_by_value.rs
new file mode 100644
index 0000000000000..f01c2de21ef61
--- /dev/null
+++ b/primitives/runtime-interface/proc-macro/tests/ui/take_self_by_value.rs
@@ -0,0 +1,8 @@
+use runtime_interface::runtime_interface;
+
+#[runtime_interface]
+trait Test {
+	fn test(self) {}
+}
+
+fn main() {}
diff --git a/primitives/runtime-interface/proc-macro/tests/ui/take_self_by_value.stderr b/primitives/runtime-interface/proc-macro/tests/ui/take_self_by_value.stderr
new file mode 100644
index 0000000000000..9b17a63a35cbc
--- /dev/null
+++ b/primitives/runtime-interface/proc-macro/tests/ui/take_self_by_value.stderr
@@ -0,0 +1,5 @@
+error: Taking `Self` by value is not allowed.
+ --> $DIR/take_self_by_value.rs:5:10
+  |
+5 |     fn test(self) {}
+  |             ^^^^
diff --git a/primitives/runtime-interface/src/host.rs b/primitives/runtime-interface/src/host.rs
new file mode 100644
index 0000000000000..313aba3d855d6
--- /dev/null
+++ b/primitives/runtime-interface/src/host.rs
@@ -0,0 +1,62 @@
+// Copyright 2019 Parity Technologies (UK) Ltd.
+// This file is part of Substrate.
+
+// Substrate is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+
+// Substrate is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+
+// You should have received a copy of the GNU General Public License
+// along with Substrate.  If not, see .
+
+//! Traits required by the runtime interface from the host side.
+
+use crate::RIType;
+
+use wasm_interface::{FunctionContext, Result};
+
+/// Something that can be converted into a ffi value.
+pub trait IntoFFIValue: RIType {
+	/// Convert `self` into a ffi value.
+	fn into_ffi_value(self, context: &mut dyn FunctionContext) -> Result;
+}
+
+/// Something that can be converted into a preallocated ffi value.
+///
+/// Every type parameter that should be given as `&mut` into a runtime interface function, needs
+/// to implement this trait. After executing the host implementation of the runtime interface
+/// function, the value is copied into the preallocated wasm memory.
+///
+/// This should only be used for types which have a fixed size, like slices. Other types like a vec
+/// do not work with this interface, as we can not call into wasm to reallocate memory. So, this
+/// trait should be implemented carefully.
+pub trait IntoPreallocatedFFIValue: RIType {
+	/// As `Self` can be an unsized type, it needs to be represented by a sized type at the host.
+	/// This `SelfInstance` is the sized type.
+	type SelfInstance;
+
+	/// Convert `self_instance` into the given preallocated ffi value.
+	fn into_preallocated_ffi_value(
+		self_instance: Self::SelfInstance,
+		context: &mut dyn FunctionContext,
+		allocated: Self::FFIType,
+	) -> Result<()>;
+}
+
+/// Something that can be created from a ffi value.
+pub trait FromFFIValue: RIType {
+	/// As `Self` can be an unsized type, it needs to be represented by a sized type at the host.
+	/// This `SelfInstance` is the sized type.
+	type SelfInstance;
+
+	/// Create `SelfInstance` from the given
+	fn from_ffi_value(
+		context: &mut dyn FunctionContext,
+		arg: Self::FFIType,
+	) -> Result;
+}
diff --git a/primitives/runtime-interface/src/impls.rs b/primitives/runtime-interface/src/impls.rs
new file mode 100644
index 0000000000000..7a6adc90c96ea
--- /dev/null
+++ b/primitives/runtime-interface/src/impls.rs
@@ -0,0 +1,491 @@
+// Copyright 2019 Parity Technologies (UK) Ltd.
+// This file is part of Substrate.
+
+// Substrate is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+
+// Substrate is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+
+// You should have received a copy of the GNU General Public License
+// along with Substrate.  If not, see .
+
+//! Provides implementations for the runtime interface traits.
+
+use crate::{RIType, Pointer, pass_by::{PassBy, Codec, Inner, PassByInner}};
+#[cfg(feature = "std")]
+use crate::host::*;
+#[cfg(not(feature = "std"))]
+use crate::wasm::*;
+
+#[cfg(all(not(feature = "std"), not(feature = "disable_target_static_assertions")))]
+use static_assertions::assert_eq_size;
+
+#[cfg(feature = "std")]
+use wasm_interface::{FunctionContext, Result};
+
+use codec::{Encode, Decode};
+
+use rstd::{any::TypeId, mem, vec::Vec};
+
+#[cfg(feature = "std")]
+use rstd::borrow::Cow;
+
+#[cfg(not(feature = "std"))]
+use rstd::{slice, boxed::Box};
+
+// Make sure that our assumptions for storing a pointer + its size in `u64` is valid.
+#[cfg(all(not(feature = "std"), not(feature = "disable_target_static_assertions")))]
+assert_eq_size!(usize, u32);
+#[cfg(all(not(feature = "std"), not(feature = "disable_target_static_assertions")))]
+assert_eq_size!(*const u8, u32);
+
+/// Converts a pointer and length into an `u64`.
+pub fn pointer_and_len_to_u64(ptr: u32, len: u32) -> u64 {
+	// The static assertions from above are changed into a runtime check.
+	#[cfg(all(feature = "std", not(feature = "disable_target_static_assertions")))]
+	assert_eq!(4, rstd::mem::size_of::());
+
+	(u64::from(len) << 32) | u64::from(ptr)
+}
+
+/// Splits an `u64` into the pointer and length.
+pub fn pointer_and_len_from_u64(val: u64) -> (u32, u32) {
+	// The static assertions from above are changed into a runtime check.
+	#[cfg(all(feature = "std", not(feature = "disable_target_static_assertions")))]
+	assert_eq!(4, rstd::mem::size_of::());
+
+	let ptr = (val & (!0u32 as u64)) as u32;
+	let len = (val >> 32) as u32;
+
+	(ptr, len)
+}
+
+/// Implement the traits for the given primitive traits.
+macro_rules! impl_traits_for_primitives {
+	(
+		$(
+			$rty:ty, $fty:ty,
+		)*
+	) => {
+		$(
+			/// The type is passed directly.
+			impl RIType for $rty {
+				type FFIType = $fty;
+			}
+
+			#[cfg(not(feature = "std"))]
+			impl IntoFFIValue for $rty {
+				type Owned = ();
+
+				fn into_ffi_value(&self) -> WrappedFFIValue<$fty> {
+					(*self as $fty).into()
+				}
+			}
+
+			#[cfg(not(feature = "std"))]
+			impl FromFFIValue for $rty {
+				fn from_ffi_value(arg: $fty) -> $rty {
+					arg as $rty
+				}
+			}
+
+			#[cfg(feature = "std")]
+			impl FromFFIValue for $rty {
+				type SelfInstance = $rty;
+
+				fn from_ffi_value(_: &mut dyn FunctionContext, arg: $fty) -> Result<$rty> {
+					Ok(arg as $rty)
+				}
+			}
+
+			#[cfg(feature = "std")]
+			impl IntoFFIValue for $rty {
+				fn into_ffi_value(self, _: &mut dyn FunctionContext) -> Result<$fty> {
+					Ok(self as $fty)
+				}
+			}
+		)*
+	}
+}
+
+impl_traits_for_primitives! {
+	u8, u8,
+	u16, u16,
+	u32, u32,
+	u64, u64,
+	i8, i8,
+	i16, i16,
+	i32, i32,
+	i64, i64,
+}
+
+/// `bool` is passed as `u8`.
+///
+/// - `1`: true
+/// - `0`: false
+impl RIType for bool {
+	type FFIType = u8;
+}
+
+#[cfg(not(feature = "std"))]
+impl IntoFFIValue for bool {
+	type Owned = ();
+
+	fn into_ffi_value(&self) -> WrappedFFIValue {
+		if *self { 1 } else { 0 }.into()
+	}
+}
+
+#[cfg(not(feature = "std"))]
+impl FromFFIValue for bool {
+	fn from_ffi_value(arg: u8) -> bool {
+		arg == 1
+	}
+}
+
+#[cfg(feature = "std")]
+impl FromFFIValue for bool {
+	type SelfInstance = bool;
+
+	fn from_ffi_value(_: &mut dyn FunctionContext, arg: u8) -> Result {
+		Ok(arg == 1)
+	}
+}
+
+#[cfg(feature = "std")]
+impl IntoFFIValue for bool {
+	fn into_ffi_value(self, _: &mut dyn FunctionContext) -> Result {
+		Ok(if self { 1 } else { 0 })
+	}
+}
+
+/// The type is passed as `u64`.
+///
+/// The `u64` value is build by `length 32bit << 32 | pointer 32bit`
+///
+/// If `T == u8` the length and the pointer are taken directly from the `Self`.
+/// Otherwise `Self` is encoded and the length and the pointer are taken from the encoded vector.
+impl RIType for Vec {
+	type FFIType = u64;
+}
+
+#[cfg(feature = "std")]
+impl IntoFFIValue for Vec {
+	fn into_ffi_value(self, context: &mut dyn FunctionContext) -> Result {
+		let vec: Cow<'_, [u8]> = if TypeId::of::() == TypeId::of::() {
+			unsafe { Cow::Borrowed(mem::transmute(&self[..])) }
+		} else {
+			Cow::Owned(self.encode())
+		};
+
+		let ptr = context.allocate_memory(vec.as_ref().len() as u32)?;
+		context.write_memory(ptr, &vec)?;
+
+		Ok(pointer_and_len_to_u64(ptr.into(), vec.len() as u32))
+	}
+}
+
+#[cfg(feature = "std")]
+impl FromFFIValue for Vec {
+	type SelfInstance = Vec;
+
+	fn from_ffi_value(context: &mut dyn FunctionContext, arg: u64) -> Result> {
+		<[T] as FromFFIValue>::from_ffi_value(context, arg)
+	}
+}
+
+#[cfg(not(feature = "std"))]
+impl IntoFFIValue for Vec {
+	type Owned = Vec;
+
+	fn into_ffi_value(&self) -> WrappedFFIValue> {
+		self[..].into_ffi_value()
+	}
+}
+
+#[cfg(not(feature = "std"))]
+impl FromFFIValue for Vec {
+	fn from_ffi_value(arg: u64) -> Vec {
+		let (ptr, len) = pointer_and_len_from_u64(arg);
+		let len = len as usize;
+
+		if TypeId::of::() == TypeId::of::() {
+			unsafe { mem::transmute(Vec::from_raw_parts(ptr as *mut u8, len, len)) }
+		} else {
+			let slice = unsafe { slice::from_raw_parts(ptr as *const u8, len) };
+			Self::decode(&mut &slice[..]).expect("Host to wasm values are encoded correctly; qed")
+		}
+	}
+}
+
+/// The type is passed as `u64`.
+///
+/// The `u64` value is build by `length 32bit << 32 | pointer 32bit`
+///
+/// If `T == u8` the length and the pointer are taken directly from the `Self`.
+/// Otherwise `Self` is encoded and the length and the pointer are taken from the encoded vector.
+impl RIType for [T] {
+	type FFIType = u64;
+}
+
+#[cfg(feature = "std")]
+impl FromFFIValue for [T] {
+	type SelfInstance = Vec;
+
+	fn from_ffi_value(context: &mut dyn FunctionContext, arg: u64) -> Result> {
+		let (ptr, len) = pointer_and_len_from_u64(arg);
+
+		let vec = context.read_memory(Pointer::new(ptr), len)?;
+
+		if TypeId::of::() == TypeId::of::() {
+			Ok(unsafe { mem::transmute(vec) })
+		} else {
+			Ok(Vec::::decode(&mut &vec[..]).expect("Wasm to host values are encoded correctly; qed"))
+		}
+	}
+}
+
+#[cfg(feature = "std")]
+impl IntoPreallocatedFFIValue for [u8] {
+	type SelfInstance = Vec;
+
+	fn into_preallocated_ffi_value(
+		self_instance: Self::SelfInstance,
+		context: &mut dyn FunctionContext,
+		allocated: u64,
+	) -> Result<()> {
+		let (ptr, len) = pointer_and_len_from_u64(allocated);
+
+		if (len as usize) < self_instance.len() {
+			Err(
+				format!(
+					"Preallocated buffer is not big enough (given {} vs needed {})!",
+					len,
+					self_instance.len()
+				)
+			)
+		} else {
+			context.write_memory(Pointer::new(ptr), &self_instance)
+		}
+	}
+}
+
+#[cfg(not(feature = "std"))]
+impl IntoFFIValue for [T] {
+	type Owned = Vec;
+
+	fn into_ffi_value(&self) -> WrappedFFIValue> {
+		if TypeId::of::() == TypeId::of::() {
+			let slice = unsafe { mem::transmute::<&[T], &[u8]>(self) };
+			pointer_and_len_to_u64(slice.as_ptr() as u32, slice.len() as u32).into()
+		} else {
+			let data = self.encode();
+			let ffi_value = pointer_and_len_to_u64(data.as_ptr() as u32, data.len() as u32);
+			(ffi_value, data).into()
+		}
+	}
+}
+
+/// Implement the traits for the `[u8; N]` arrays, where `N` is the input to this macro.
+macro_rules! impl_traits_for_arrays {
+	(
+		$(
+			$n:expr
+		),*
+		$(,)?
+	) => {
+		$(
+			/// The type is passed as `u32`.
+			///
+			/// The `u32` is the pointer to the array.
+			impl RIType for [u8; $n] {
+				type FFIType = u32;
+			}
+
+			#[cfg(not(feature = "std"))]
+			impl IntoFFIValue for [u8; $n] {
+				type Owned = ();
+
+				fn into_ffi_value(&self) -> WrappedFFIValue {
+					(self.as_ptr() as u32).into()
+				}
+			}
+
+			#[cfg(not(feature = "std"))]
+			impl FromFFIValue for [u8; $n] {
+				fn from_ffi_value(arg: u32) -> [u8; $n] {
+					let mut res = unsafe { mem::MaybeUninit::<[u8; $n]>::zeroed().assume_init() };
+					res.copy_from_slice(unsafe { slice::from_raw_parts(arg as *const u8, $n) });
+
+					// Make sure we free the pointer.
+					let _ = unsafe { Box::from_raw(arg as *mut u8) };
+
+					res
+				}
+			}
+
+			#[cfg(feature = "std")]
+			impl FromFFIValue for [u8; $n] {
+				type SelfInstance = [u8; $n];
+
+				fn from_ffi_value(context: &mut dyn FunctionContext, arg: u32) -> Result<[u8; $n]> {
+					let data = context.read_memory(Pointer::new(arg), $n)?;
+					let mut res = unsafe { mem::MaybeUninit::<[u8; $n]>::zeroed().assume_init() };
+					res.copy_from_slice(&data);
+					Ok(res)
+				}
+			}
+
+			#[cfg(feature = "std")]
+			impl IntoFFIValue for [u8; $n] {
+				fn into_ffi_value(self, context: &mut dyn FunctionContext) -> Result {
+					let addr = context.allocate_memory($n)?;
+					context.write_memory(addr, &self)?;
+					Ok(addr.into())
+				}
+			}
+
+			#[cfg(feature = "std")]
+			impl IntoPreallocatedFFIValue for [u8; $n] {
+				type SelfInstance = [u8; $n];
+
+				fn into_preallocated_ffi_value(
+					self_instance: Self::SelfInstance,
+					context: &mut dyn FunctionContext,
+					allocated: u32,
+				) -> Result<()> {
+					context.write_memory(Pointer::new(allocated), &self_instance)
+				}
+			}
+		)*
+	}
+}
+
+impl_traits_for_arrays! {
+	1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26,
+	27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50,
+	51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74,
+	75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96,
+}
+
+impl PassBy for rstd::result::Result {
+	type PassBy = Codec;
+}
+
+impl PassBy for Option {
+	type PassBy = Codec;
+}
+
+/// Implement `PassBy` with `Inner` for the given fixed sized hash types.
+macro_rules! for_primitive_types {
+	{ $( $hash:ident $n:expr ),* $(,)? } => {
+		$(
+			impl PassBy for primitive_types::$hash {
+				type PassBy = Inner;
+			}
+
+			impl PassByInner for primitive_types::$hash {
+				type Inner = [u8; $n];
+
+				fn inner(&self) -> &Self::Inner {
+					&self.0
+				}
+
+				fn into_inner(self) -> Self::Inner {
+					self.0
+				}
+
+				fn from_inner(inner: Self::Inner) -> Self {
+					Self(inner)
+				}
+			}
+		)*
+	}
+}
+
+for_primitive_types! {
+	H160 20,
+	H256 32,
+	H512 64,
+}
+
+/// The type is passed as `u64`.
+///
+/// The `u64` value is build by `length 32bit << 32 | pointer 32bit`
+///
+/// The length and the pointer are taken directly from the `Self`.
+impl RIType for str {
+	type FFIType = u64;
+}
+
+#[cfg(feature = "std")]
+impl FromFFIValue for str {
+	type SelfInstance = String;
+
+	fn from_ffi_value(context: &mut dyn FunctionContext, arg: u64) -> Result {
+		let (ptr, len) = pointer_and_len_from_u64(arg);
+
+		let vec = context.read_memory(Pointer::new(ptr), len)?;
+
+		// The data is valid utf8, as it is stored as `&str` in wasm.
+		String::from_utf8(vec).map_err(|_| "Invalid utf8 data provided".into())
+	}
+}
+
+#[cfg(not(feature = "std"))]
+impl IntoFFIValue for str {
+	type Owned = ();
+
+	fn into_ffi_value(&self) -> WrappedFFIValue {
+		let bytes = self.as_bytes();
+		pointer_and_len_to_u64(bytes.as_ptr() as u32, bytes.len() as u32).into()
+	}
+}
+
+#[cfg(feature = "std")]
+impl RIType for Pointer {
+	type FFIType = u32;
+}
+
+/// The type is passed as `u32`.
+#[cfg(not(feature = "std"))]
+impl RIType for Pointer {
+	type FFIType = u32;
+}
+
+#[cfg(not(feature = "std"))]
+impl IntoFFIValue for Pointer {
+	type Owned = ();
+
+	fn into_ffi_value(&self) -> WrappedFFIValue {
+		(*self as u32).into()
+	}
+}
+
+#[cfg(not(feature = "std"))]
+impl FromFFIValue for Pointer {
+	fn from_ffi_value(arg: u32) -> Self {
+		arg as _
+	}
+}
+
+#[cfg(feature = "std")]
+impl FromFFIValue for Pointer {
+	type SelfInstance = Self;
+
+	fn from_ffi_value(_: &mut dyn FunctionContext, arg: u32) -> Result {
+		Ok(Pointer::new(arg))
+	}
+}
+
+#[cfg(feature = "std")]
+impl IntoFFIValue for Pointer {
+	fn into_ffi_value(self, _: &mut dyn FunctionContext) -> Result {
+		Ok(self.into())
+	}
+}
diff --git a/primitives/runtime-interface/src/lib.rs b/primitives/runtime-interface/src/lib.rs
new file mode 100644
index 0000000000000..fb70d252a6772
--- /dev/null
+++ b/primitives/runtime-interface/src/lib.rs
@@ -0,0 +1,212 @@
+// Copyright 2019 Parity Technologies (UK) Ltd.
+// This file is part of Substrate.
+
+// Substrate is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+
+// Substrate is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+
+// You should have received a copy of the GNU General Public License
+// along with Substrate.  If not, see .
+
+//! Substrate runtime interface
+//!
+//! This crate provides types, traits and macros around runtime interfaces. A runtime interface is
+//! a fixed interface between a Substrate runtime and a Substrate node. For a native runtime the
+//! interface maps to a direct function call of the implementation. For a wasm runtime the interface
+//! maps to an external function call. These external functions are exported by the wasm executor
+//! and they map to the same implementation as the native calls.
+//!
+//! # Using a type in a runtime interface
+//!
+//! Any type that should be used in a runtime interface as argument or return value needs to
+//! implement [`RIType`]. The associated type `FFIType` is the type that is used in the FFI
+//! function to represent the actual type. For example `[T]` is represented by an `u64`. The slice
+//! pointer and the length will be mapped to an `u64` value. For more information, see the
+//! implementation of [`RIType`] for [`T`]. The FFI function definition is used when calling from
+//! the wasm runtime into the node.
+//!
+//! Traits are used to convert from a type to the corresponding [`RIType::FFIType`].
+//! Depending on where and how a type should be used in a function signature, a combination of the
+//! following traits need to be implemented:
+//!
+//! 1. Pass as function argument: [`wasm::IntoFFIValue`] and [`host::FromFFIValue`]
+//! 2. As function return value: [`wasm::FromFFIValue`] and [`host::IntoFFIValue`]
+//! 3. Pass as mutable function argument: [`host::IntoPreallocatedFFIValue`]
+//!
+//! The traits are implemented for most of the common types like `[T]`, `Vec`, arrays and
+//! primitive types.
+//!
+//! For custom types, we provide the [`PassBy`](pass_by::PassBy) trait and strategies that define
+//! how a type is passed between the wasm runtime and the node. Each strategy also provides a derive
+//! macro to simplify the implementation.
+//!
+//! # Performance
+//!
+//! To not waste any more performance when calling into the node, not all types are SCALE encoded
+//! when being passed as arguments between the wasm runtime and the node. For most types that
+//! are raw bytes like `Vec`, `[u8]` or `[u8; N]` we pass them directly, without SCALE encoding
+//! them in front of. The implementation of [`RIType`] each type provides more information on how
+//! the data is passed.
+//!
+//! # Declaring a runtime interface
+//!
+//! Declaring a runtime interface is similar to declaring a trait in Rust:
+//!
+//! ```
+//! #[substrate_runtime_interface::runtime_interface]
+//! trait RuntimeInterface {
+//!     fn some_function(value: &[u8]) -> bool {
+//!         value.iter().all(|v| *v > 125)
+//!     }
+//! }
+//! ```
+//!
+//! For more information on declaring a runtime interface, see
+//! [`#[runtime_interface]`](attr.runtime_interface.html).
+
+#![cfg_attr(not(feature = "std"), no_std)]
+
+#[doc(hidden)]
+#[cfg(feature = "std")]
+pub use wasm_interface;
+
+#[doc(hidden)]
+pub use rstd;
+
+pub use substrate_runtime_interface_proc_macro::runtime_interface;
+
+#[doc(hidden)]
+#[cfg(feature = "std")]
+pub use externalities::{
+	set_and_run_with_externalities, with_externalities, Externalities, ExternalitiesExt, ExtensionStore,
+};
+
+#[doc(hidden)]
+pub use codec;
+
+pub(crate) mod impls;
+#[cfg(feature = "std")]
+pub mod host;
+#[cfg(not(feature = "std"))]
+pub mod wasm;
+pub mod pass_by;
+
+/// Something that can be used by the runtime interface as type to communicate between wasm and the
+/// host.
+///
+/// Every type that should be used in a runtime interface function signature needs to implement
+/// this trait.
+pub trait RIType {
+	/// The ffi type that is used to represent `Self`.
+	#[cfg(feature = "std")]
+	type FFIType: wasm_interface::IntoValue + wasm_interface::TryFromValue;
+	#[cfg(not(feature = "std"))]
+	type FFIType;
+}
+
+/// A pointer that can be used in a runtime interface function signature.
+#[cfg(not(feature = "std"))]
+pub type Pointer = *mut T;
+
+/// A pointer that can be used in a runtime interface function signature.
+#[cfg(feature = "std")]
+pub type Pointer = wasm_interface::Pointer;
+
+#[cfg(test)]
+mod tests {
+	use super::*;
+	use test_wasm::{WASM_BINARY, test_api::HostFunctions};
+	use wasm_interface::HostFunctions as HostFunctionsT;
+
+	type TestExternalities = state_machine::TestExternalities;
+
+	fn call_wasm_method(method: &str) -> TestExternalities {
+		let mut ext = TestExternalities::default();
+		let mut ext_ext = ext.ext();
+
+		executor::call_in_wasm::<
+			_,
+			(
+				HF,
+				runtime_io::SubstrateHostFunctions,
+				executor::deprecated_host_interface::SubstrateExternals
+			)
+		>(
+			method,
+			&[],
+			executor::WasmExecutionMethod::Interpreted,
+			&mut ext_ext,
+			&WASM_BINARY[..],
+			8,
+		).expect(&format!("Executes `{}`", method));
+
+		ext
+	}
+
+	#[test]
+	fn test_return_data() {
+		call_wasm_method::("test_return_data");
+	}
+
+	#[test]
+	fn test_return_option_data() {
+		call_wasm_method::("test_return_option_data");
+	}
+
+	#[test]
+	fn test_set_storage() {
+		let mut ext = call_wasm_method::("test_set_storage");
+
+		let expected = "world";
+		assert_eq!(expected.as_bytes(), &ext.ext().storage("hello".as_bytes()).unwrap()[..]);
+	}
+
+	#[test]
+	fn test_return_value_into_mutable_reference() {
+		call_wasm_method::("test_return_value_into_mutable_reference");
+	}
+
+	#[test]
+	fn test_get_and_return_array() {
+		call_wasm_method::("test_get_and_return_array");
+	}
+
+	#[test]
+	fn test_array_as_mutable_reference() {
+		call_wasm_method::("test_array_as_mutable_reference");
+	}
+
+	#[test]
+	fn test_return_input_public_key() {
+		call_wasm_method::("test_return_input_public_key");
+	}
+
+	#[test]
+	#[should_panic(
+		expected = "Other(\"Instantiation: Export ext_test_api_return_input_version_1 not found\")"
+	)]
+	fn host_function_not_found() {
+		call_wasm_method::<()>("test_return_data");
+	}
+
+	#[test]
+	#[should_panic(
+		expected =
+			"FunctionExecution(\"ext_test_api_invalid_utf8_data_version_1\", \
+			\"Invalid utf8 data provided\")"
+	)]
+	fn test_invalid_utf8_data_should_return_an_error() {
+		call_wasm_method::("test_invalid_utf8_data_should_return_an_error");
+	}
+
+	#[test]
+	fn test_overwrite_native_function_implementation() {
+		call_wasm_method::("test_overwrite_native_function_implementation");
+	}
+}
diff --git a/primitives/runtime-interface/src/pass_by.rs b/primitives/runtime-interface/src/pass_by.rs
new file mode 100644
index 0000000000000..46265237c0c47
--- /dev/null
+++ b/primitives/runtime-interface/src/pass_by.rs
@@ -0,0 +1,384 @@
+// Copyright 2019 Parity Technologies (UK) Ltd.
+// This file is part of Substrate.
+
+// Substrate is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+
+// Substrate is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+
+// You should have received a copy of the GNU General Public License
+// along with Substrate.  If not, see .
+
+//! Provides the [`PassBy`](pass_by::PassBy) trait to simplify the implementation of the
+//! runtime interface traits for custom types.
+//!
+//! [`Codec`](pass_by::Codec), [`Inner`](pass_by::Inner) and [`Enum`](pass_by::Enum) are the
+//! provided strategy implementations.
+
+use crate::{RIType, impls::{pointer_and_len_from_u64, pointer_and_len_to_u64}};
+
+#[cfg(feature = "std")]
+use crate::host::*;
+#[cfg(not(feature = "std"))]
+use crate::wasm::*;
+
+#[cfg(feature = "std")]
+use wasm_interface::{FunctionContext, Pointer, Result};
+
+use rstd::{marker::PhantomData, convert::TryFrom};
+
+#[cfg(not(feature = "std"))]
+use rstd::{slice, vec::Vec};
+
+pub use substrate_runtime_interface_proc_macro::{PassByCodec, PassByInner, PassByEnum};
+
+/// Something that should be passed between wasm and the host using the given strategy.
+///
+/// See [`Codec`], [`Inner`] or [`Enum`] for more information about the provided strategies.
+pub trait PassBy: Sized {
+	/// The strategy that should be used to pass the type.
+	type PassBy: PassByImpl;
+}
+
+/// Something that provides a strategy for passing a type between wasm and the host.
+///
+/// This trait exposes the same functionality as [`crate::host::IntoFFIValue`] and
+/// [`crate::host::FromFFIValue`] to delegate the implementation for a type to a different type.
+///
+/// This trait is used for the host implementation.
+#[cfg(feature = "std")]
+pub trait PassByImpl: RIType {
+	/// Convert the given instance to the ffi value.
+	///
+	/// For more information see: [`crate::host::IntoFFIValue::into_ffi_value`]
+	fn into_ffi_value(
+		instance: T,
+		context: &mut dyn FunctionContext,
+	) -> Result;
+
+	/// Create `T` from the given ffi value.
+	///
+	/// For more information see: [`crate::host::FromFFIValue::from_ffi_value`]
+	fn from_ffi_value(
+		context: &mut dyn FunctionContext,
+		arg: Self::FFIType,
+	) -> Result;
+}
+
+/// Something that provides a strategy for passing a type between wasm and the host.
+///
+/// This trait exposes the same functionality as [`crate::wasm::IntoFFIValue`] and
+/// [`crate::wasm::FromFFIValue`] to delegate the implementation for a type to a different type.
+///
+/// This trait is used for the wasm implementation.
+#[cfg(not(feature = "std"))]
+pub trait PassByImpl: RIType {
+	/// The owned rust type that is stored with the ffi value in [`crate::wasm::WrappedFFIValue`].
+	type Owned;
+
+	/// Convert the given `instance` into [`crate::wasm::WrappedFFIValue`].
+	///
+	/// For more information see: [`crate::wasm::IntoFFIValue::into_ffi_value`]
+	fn into_ffi_value(instance: &T) -> WrappedFFIValue;
+
+	/// Create `T` from the given ffi value.
+	///
+	/// For more information see: [`crate::wasm::FromFFIValue::from_ffi_value`]
+	fn from_ffi_value(arg: Self::FFIType) -> T;
+}
+
+impl RIType for T {
+	type FFIType = ::FFIType;
+}
+
+#[cfg(feature = "std")]
+impl IntoFFIValue for T {
+	fn into_ffi_value(
+		self,
+		context: &mut dyn FunctionContext,
+	) -> Result<::FFIType> {
+		T::PassBy::into_ffi_value(self, context)
+	}
+}
+
+#[cfg(feature = "std")]
+impl FromFFIValue for T {
+	type SelfInstance = Self;
+
+	fn from_ffi_value(
+		context: &mut dyn FunctionContext,
+		arg: ::FFIType,
+	) -> Result {
+		T::PassBy::from_ffi_value(context, arg)
+	}
+}
+
+#[cfg(not(feature = "std"))]
+impl IntoFFIValue for T {
+	type Owned = >::Owned;
+
+	fn into_ffi_value(&self) -> WrappedFFIValue<::FFIType, Self::Owned> {
+		T::PassBy::into_ffi_value(self)
+	}
+}
+
+#[cfg(not(feature = "std"))]
+impl FromFFIValue for T {
+	fn from_ffi_value(arg: ::FFIType) -> Self {
+		T::PassBy::from_ffi_value(arg)
+	}
+}
+
+/// The implementation of the pass by codec strategy. This strategy uses a SCALE encoded
+/// representation of the type between wasm and the host.
+///
+/// Use this type as associated type for [`PassBy`] to implement this strategy for a type.
+///
+/// This type expects the type that wants to implement this strategy as generic parameter.
+///
+/// [`PassByCodec`](derive.PassByCodec.html) is a derive macro to implement this strategy.
+///
+/// # Example
+/// ```
+/// # use substrate_runtime_interface::pass_by::{PassBy, Codec};
+/// #[derive(codec::Encode, codec::Decode)]
+/// struct Test;
+///
+/// impl PassBy for Test {
+///     type PassBy = Codec;
+/// }
+/// ```
+pub struct Codec(PhantomData);
+
+#[cfg(feature = "std")]
+impl PassByImpl for Codec {
+	fn into_ffi_value(
+		instance: T,
+		context: &mut dyn FunctionContext,
+	) -> Result {
+		let vec = instance.encode();
+		let ptr = context.allocate_memory(vec.len() as u32)?;
+		context.write_memory(ptr, &vec)?;
+
+		Ok(pointer_and_len_to_u64(ptr.into(), vec.len() as u32))
+	}
+
+	fn from_ffi_value(
+		context: &mut dyn FunctionContext,
+		arg: Self::FFIType,
+	) -> Result {
+		let (ptr, len) = pointer_and_len_from_u64(arg);
+		let vec = context.read_memory(Pointer::new(ptr), len)?;
+		T::decode(&mut &vec[..])
+			.map_err(|e| format!("Could not decode value from wasm: {}", e.what()))
+	}
+}
+
+#[cfg(not(feature = "std"))]
+impl PassByImpl for Codec {
+	type Owned = Vec;
+
+	fn into_ffi_value(instance: &T) -> WrappedFFIValue {
+		let data = instance.encode();
+		let ffi_value = pointer_and_len_to_u64(data.as_ptr() as u32, data.len() as u32);
+		(ffi_value, data).into()
+	}
+
+	fn from_ffi_value(arg: Self::FFIType) -> T {
+		let (ptr, len) = pointer_and_len_from_u64(arg);
+		let len = len as usize;
+
+		let slice = unsafe { slice::from_raw_parts(ptr as *const u8, len) };
+		T::decode(&mut &slice[..]).expect("Host to wasm values are encoded correctly; qed")
+	}
+}
+
+/// The type is passed as `u64`.
+///
+/// The `u64` value is build by `length 32bit << 32 | pointer 32bit`
+///
+/// `Self` is encoded and the length and the pointer are taken from the encoded vector.
+impl RIType for Codec {
+	type FFIType = u64;
+}
+
+/// Trait that needs to be implemented by a type that should be passed between wasm and the host,
+/// by using the inner type. See [`Inner`] for more information.
+pub trait PassByInner: Sized {
+	/// The inner type that is wrapped by `Self`.
+	type Inner: RIType;
+
+	/// Consumes `self` and returns the inner type.
+	fn into_inner(self) -> Self::Inner;
+
+	/// Returns the reference to the inner type.
+	fn inner(&self) -> &Self::Inner;
+
+	/// Construct `Self` from the given `inner`.
+	fn from_inner(inner: Self::Inner) -> Self;
+}
+
+/// The implementation of the pass by inner type strategy. The type that uses this strategy will be
+/// passed between wasm and the host by using the wrapped inner type. So, this strategy is only
+/// usable by newtype structs.
+///
+/// Use this type as associated type for [`PassBy`] to implement this strategy for a type. Besides
+/// that the `PassByInner` trait need to be implemented as well.
+///
+/// This type expects the type that wants to use this strategy as generic parameter `T` and the
+/// inner type as generic parameter `I`.
+///
+/// [`PassByInner`](derive.PassByInner.html) is a derive macro to implement this strategy.
+///
+/// # Example
+/// ```
+/// # use substrate_runtime_interface::pass_by::{PassBy, Inner, PassByInner};
+/// struct Test([u8; 32]);
+///
+/// impl PassBy for Test {
+///     type PassBy = Inner;
+/// }
+///
+/// impl PassByInner for Test {
+///     type Inner = [u8; 32];
+///
+///     fn into_inner(self) -> [u8; 32] {
+///         self.0
+///     }
+///     fn inner(&self) -> &[u8; 32] {
+///         &self.0
+///     }
+///     fn from_inner(inner: [u8; 32]) -> Self {
+///         Self(inner)
+///     }
+/// }
+/// ```
+pub struct Inner, I: RIType>(PhantomData<(T, I)>);
+
+#[cfg(feature = "std")]
+impl, I: RIType> PassByImpl for Inner
+	where I: IntoFFIValue + FromFFIValue
+{
+	fn into_ffi_value(
+		instance: T,
+		context: &mut dyn FunctionContext,
+	) -> Result {
+		instance.into_inner().into_ffi_value(context)
+	}
+
+	fn from_ffi_value(
+		context: &mut dyn FunctionContext,
+		arg: Self::FFIType,
+	) -> Result {
+		I::from_ffi_value(context, arg).map(T::from_inner)
+	}
+}
+
+#[cfg(not(feature = "std"))]
+impl, I: RIType> PassByImpl for Inner
+	where I: IntoFFIValue + FromFFIValue
+{
+	type Owned = I::Owned;
+
+	fn into_ffi_value(instance: &T) -> WrappedFFIValue {
+ 		instance.inner().into_ffi_value()
+	}
+
+	fn from_ffi_value(arg: Self::FFIType) -> T {
+ 		T::from_inner(I::from_ffi_value(arg))
+	}
+}
+
+/// The type is passed as the inner type.
+impl, I: RIType> RIType for Inner {
+	type FFIType = I::FFIType;
+}
+
+/// The implementation of the pass by enum strategy. This strategy uses an `u8` internally to pass
+/// the enum between wasm and the host. So, this strategy only supports enums with unit variants.
+///
+/// Use this type as associated type for [`PassBy`] to implement this strategy for a type.
+///
+/// This type expects the type that wants to implement this strategy as generic parameter. Besides
+/// that the type needs to implement `TryFrom` and `From for u8`.
+///
+/// [`PassByEnum`](derive.PassByEnum.html) is a derive macro to implement this strategy.
+///
+/// # Example
+/// ```
+/// # use substrate_runtime_interface::pass_by::{PassBy, Enum};
+/// #[derive(Clone, Copy)]
+/// enum Test {
+///     Test1,
+///     Test2,
+/// }
+///
+/// impl From for u8 {
+///     fn from(val: Test) -> u8 {
+///         match val {
+///             Test::Test1 => 0,
+///             Test::Test2 => 1,
+///         }
+///     }
+/// }
+///
+/// impl std::convert::TryFrom for Test {
+///     type Error = ();
+///
+///     fn try_from(val: u8) -> Result {
+///         match val {
+///             0 => Ok(Test::Test1),
+///             1 => Ok(Test::Test2),
+///             _ => Err(()),
+///         }
+///     }
+/// }
+///
+/// impl PassBy for Test {
+///     type PassBy = Enum;
+/// }
+/// ```
+pub struct Enum + TryFrom>(PhantomData);
+
+#[cfg(feature = "std")]
+impl + TryFrom> PassByImpl for Enum {
+	fn into_ffi_value(
+		instance: T,
+		_: &mut dyn FunctionContext,
+	) -> Result {
+		Ok(instance.into())
+	}
+
+	fn from_ffi_value(
+		_: &mut dyn FunctionContext,
+		arg: Self::FFIType,
+	) -> Result {
+		T::try_from(arg).map_err(|_| format!("Invalid enum discriminant: {}", arg))
+	}
+}
+
+#[cfg(not(feature = "std"))]
+impl + TryFrom> PassByImpl for Enum {
+	type Owned = ();
+
+	fn into_ffi_value(instance: &T) -> WrappedFFIValue {
+		let value: u8 = (*instance).into();
+		value.into()
+	}
+
+	fn from_ffi_value(arg: Self::FFIType) -> T {
+		T::try_from(arg).expect("Host to wasm provides a valid enum discriminant; qed")
+	}
+}
+
+/// The type is passed as `u8`.
+///
+/// The value is corresponds to the discriminant of the variant.
+impl + TryFrom> RIType for Enum {
+	type FFIType = u8;
+}
diff --git a/primitives/runtime-interface/src/wasm.rs b/primitives/runtime-interface/src/wasm.rs
new file mode 100644
index 0000000000000..7ac890a3ca3ba
--- /dev/null
+++ b/primitives/runtime-interface/src/wasm.rs
@@ -0,0 +1,142 @@
+// Copyright 2019 Parity Technologies (UK) Ltd.
+// This file is part of Substrate.
+
+// Substrate is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+
+// Substrate is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+
+// You should have received a copy of the GNU General Public License
+// along with Substrate.  If not, see .
+
+//! Traits required by the runtime interface from the wasm side.
+
+use crate::RIType;
+
+use rstd::cell::Cell;
+
+/// Something that can be created from a ffi value.
+///
+/// # Safety
+///
+/// It is unsafe behavior to call `Something::into_ffi_value().get()` and take this as input for
+/// `from_ffi_value`. Implementations are safe to assume that the `arg` given to `from_ffi_value`
+/// is only generated by the corresponding `host::IntoFFIValue` implementation.
+pub trait FromFFIValue: Sized + RIType {
+	/// Create `Self` from the given ffi value.
+	fn from_ffi_value(arg: Self::FFIType) -> Self;
+}
+
+/// Something that can be converted into a ffi value.
+pub trait IntoFFIValue: RIType {
+	/// The owned rust type that is stored with the ffi value in [`WrappedFFIValue`].
+	///
+	/// If no owned value is required, `()` can be used as a type.
+	type Owned;
+
+	/// Convert `self` into a [`WrappedFFIValue`].
+	fn into_ffi_value(&self) -> WrappedFFIValue;
+}
+
+/// Represents a wrapped ffi value.
+///
+/// It is either the ffi value itself or the ffi value plus some other owned value. By providing
+/// support for storing another owned value besides the actual ffi value certain performance
+/// optimizations can be applied. For example using the pointer to a `Vec`, while using the
+/// pointer to a SCALE encoded `Vec` that is stored in this wrapper for any other `Vec`.
+pub enum WrappedFFIValue {
+	Wrapped(T),
+	WrappedAndOwned(T, O),
+}
+
+impl WrappedFFIValue {
+	/// Returns the wrapped ffi value.
+	pub fn get(&self) -> T {
+		match self {
+			Self::Wrapped(data) | Self::WrappedAndOwned(data, _) => *data,
+		}
+	}
+}
+
+impl From for WrappedFFIValue {
+	fn from(val: T) -> Self {
+		WrappedFFIValue::Wrapped(val)
+	}
+}
+
+impl From<(T, O)> for WrappedFFIValue {
+	fn from(val: (T, O)) -> Self {
+		WrappedFFIValue::WrappedAndOwned(val.0, val.1)
+	}
+}
+
+/// The state of an exchangeable function.
+#[derive(Clone, Copy)]
+enum ExchangeableFunctionState {
+	/// Original function is present
+	Original,
+	/// The function has been replaced.
+	Replaced,
+}
+
+/// A function which implementation can be exchanged.
+///
+/// Internally this works by swapping function pointers.
+pub struct ExchangeableFunction(Cell<(T, ExchangeableFunctionState)>);
+
+impl ExchangeableFunction {
+	/// Create a new instance of `ExchangeableFunction`.
+	pub const fn new(impl_: T) -> Self {
+		Self(Cell::new((impl_, ExchangeableFunctionState::Original)))
+	}
+}
+
+impl ExchangeableFunction {
+	/// Replace the implementation with `new_impl`.
+	///
+	/// # Panics
+	///
+	/// Panics when trying to replace an already replaced implementation.
+	///
+	/// # Returns
+	///
+	/// Returns the original implementation wrapped in [`RestoreImplementation`].
+	pub fn replace_implementation(&'static self, new_impl: T)  -> RestoreImplementation {
+		if let ExchangeableFunctionState::Replaced = self.0.get().1 {
+			panic!("Trying to replace an already replaced implementation!")
+		}
+
+		let old = self.0.replace((new_impl, ExchangeableFunctionState::Replaced));
+
+		RestoreImplementation(self, Some(old.0))
+	}
+
+	/// Restore the original implementation.
+	fn restore_orig_implementation(&self, orig: T) {
+		self.0.set((orig, ExchangeableFunctionState::Original));
+	}
+
+	/// Returns the internal function pointer.
+	pub fn get(&self) -> T {
+		self.0.get().0
+	}
+}
+
+// Wasm does not support threads, so this is safe; qed.
+unsafe impl Sync for ExchangeableFunction {}
+
+/// Restores a function implementation on drop.
+///
+/// Stores a static reference to the function object and the original implementation.
+pub struct RestoreImplementation(&'static ExchangeableFunction, Option);
+
+impl Drop for RestoreImplementation {
+	fn drop(&mut self) {
+		self.0.restore_orig_implementation(self.1.take().expect("Value is only taken on drop; qed"));
+	}
+}
diff --git a/primitives/runtime-interface/test-wasm/Cargo.toml b/primitives/runtime-interface/test-wasm/Cargo.toml
new file mode 100644
index 0000000000000..13d6e2591421b
--- /dev/null
+++ b/primitives/runtime-interface/test-wasm/Cargo.toml
@@ -0,0 +1,19 @@
+[package]
+name = "substrate-runtime-interface-test-wasm"
+version = "2.0.0"
+authors = ["Parity Technologies "]
+edition = "2018"
+build = "build.rs"
+
+[dependencies]
+runtime-interface = { package = "substrate-runtime-interface", path = "../", default-features = false }
+rstd = { package = "sr-std", path = "../../sr-std", default-features = false }
+runtime-io = { package = "sr-io", path = "../../sr-io", default-features = false }
+primitives = { package = "substrate-primitives", path = "../../core", default-features = false }
+
+[build-dependencies]
+wasm-builder-runner = { package = "substrate-wasm-builder-runner", version = "1.0.3", path = "../../../client/utils/wasm-builder-runner" }
+
+[features]
+default = [ "std" ]
+std = [ "runtime-interface/std", "rstd/std", "primitives/std", "runtime-io/std" ]
diff --git a/primitives/runtime-interface/test-wasm/build.rs b/primitives/runtime-interface/test-wasm/build.rs
new file mode 100644
index 0000000000000..ef2650dd5afe8
--- /dev/null
+++ b/primitives/runtime-interface/test-wasm/build.rs
@@ -0,0 +1,30 @@
+// Copyright 2019 Parity Technologies (UK) Ltd.
+// This file is part of Substrate.
+
+// Substrate is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+
+// Substrate is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+
+// You should have received a copy of the GNU General Public License
+// along with Substrate.  If not, see .
+
+use wasm_builder_runner::{build_current_project_with_rustflags, WasmBuilderSource};
+
+fn main() {
+	build_current_project_with_rustflags(
+		"wasm_binary.rs",
+		WasmBuilderSource::CratesOrPath {
+			path: "../../../utils/wasm-builder",
+			version: "1.0.6",
+		},
+		// This instructs LLD to export __heap_base as a global variable, which is used by the
+		// external memory allocator.
+		"-Clink-arg=--export=__heap_base",
+	);
+}
diff --git a/primitives/runtime-interface/test-wasm/src/lib.rs b/primitives/runtime-interface/test-wasm/src/lib.rs
new file mode 100644
index 0000000000000..d61315c521b68
--- /dev/null
+++ b/primitives/runtime-interface/test-wasm/src/lib.rs
@@ -0,0 +1,194 @@
+// Copyright 2019 Parity Technologies (UK) Ltd.
+// This file is part of Substrate.
+
+// Substrate is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+
+// Substrate is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+
+// You should have received a copy of the GNU General Public License
+// along with Substrate.  If not, see .
+
+//! Tests for the runtime interface traits and proc macros.
+
+#![cfg_attr(not(feature = "std"), no_std)]
+
+use runtime_interface::runtime_interface;
+
+#[cfg(not(feature = "std"))]
+use rstd::{vec, vec::Vec, mem, convert::TryFrom};
+
+use primitives::{sr25519::Public, wasm_export_functions};
+
+// Inlucde the WASM binary
+#[cfg(feature = "std")]
+include!(concat!(env!("OUT_DIR"), "/wasm_binary.rs"));
+
+/// Used in the `test_array_as_mutable_reference` test.
+const TEST_ARRAY: [u8; 16] = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16];
+
+#[runtime_interface]
+pub trait TestApi {
+	/// Returns the input data as result.
+	fn return_input(data: Vec) -> Vec {
+		data
+	}
+
+	/// Set the storage at key with value.
+	fn set_storage(&mut self, key: &[u8], data: &[u8]) {
+		self.place_storage(key.to_vec(), Some(data.to_vec()));
+	}
+
+	/// Copy `hello` into the given mutable reference
+	fn return_value_into_mutable_reference(&self, data: &mut [u8]) {
+		let res = "hello";
+		data[..res.as_bytes().len()].copy_from_slice(res.as_bytes());
+	}
+
+	/// Returns the input data wrapped in an `Option` as result.
+	fn return_option_input(data: Vec) -> Option> {
+		Some(data)
+	}
+
+	/// Get an array as input and returns a subset of this array.
+	fn get_and_return_array(data: [u8; 34]) -> [u8; 16] {
+		let mut res = [0u8; 16];
+		res.copy_from_slice(&data[..16]);
+		res
+	}
+
+	/// Take and fill mutable array.
+	fn array_as_mutable_reference(data: &mut [u8; 16]) {
+		data.copy_from_slice(&TEST_ARRAY);
+	}
+
+	/// Returns the given public key as result.
+	fn return_input_public_key(key: Public) -> Public {
+		key
+	}
+
+	/// A function that is called with invalid utf8 data from the runtime.
+	///
+	/// This also checks that we accept `_` (wild card) argument names.
+	fn invalid_utf8_data(_: &str) {}
+
+	/// Overwrite the native implementation in wasm. The native implementation always returns
+	/// `false` and the replacement function will return always `true`.
+	fn overwrite_native_function_implementation() -> bool {
+		false
+	}
+}
+
+/// Two random external functions from the old runtime interface.
+/// This ensures that we still inherently export these functions from the host and that we are still
+/// compatible with old wasm runtimes.
+extern "C" {
+	pub fn ext_clear_storage(key_data: *const u8, key_len: u32);
+	pub fn ext_keccak_256(data: *const u8, len: u32, out: *mut u8);
+}
+
+/// Make sure the old runtime interface needs to be imported.
+#[no_mangle]
+pub fn force_old_runtime_interface_import() {
+	unsafe { ext_clear_storage(rstd::ptr::null(), 0); }
+	unsafe { ext_keccak_256(rstd::ptr::null(), 0, rstd::ptr::null_mut()); }
+}
+
+/// This function is not used, but we require it for the compiler to include `runtime-io`.
+/// `runtime-io` is required for its panic and oom handler.
+#[no_mangle]
+pub fn import_runtime_io() {
+	runtime_io::misc::print_utf8(&[]);
+}
+
+wasm_export_functions! {
+	fn test_return_data() {
+		let input = vec![1, 2, 3, 4, 5, 6];
+		let res = test_api::return_input(input.clone());
+
+		assert_eq!(input, res);
+	}
+
+	fn test_return_option_data() {
+		let input = vec![1, 2, 3, 4, 5, 6];
+		let res = test_api::return_option_input(input.clone());
+
+		assert_eq!(Some(input), res);
+	}
+
+	fn test_set_storage() {
+		let key = "hello";
+		let value = "world";
+
+		test_api::set_storage(key.as_bytes(), value.as_bytes());
+	}
+
+	fn test_return_value_into_mutable_reference() {
+		let mut data = vec![1, 2, 3, 4, 5, 6];
+
+		test_api::return_value_into_mutable_reference(&mut data);
+
+		let expected = "hello";
+		assert_eq!(expected.as_bytes(), &data[..expected.len()]);
+	}
+
+	fn test_get_and_return_array() {
+		let mut input = unsafe { mem::MaybeUninit::<[u8; 34]>::zeroed().assume_init() };
+		input.copy_from_slice(&[
+			24, 3, 23, 20, 2, 16, 32, 1, 12, 26, 27, 8, 29, 31, 6, 5, 4, 19, 10, 28, 34, 21, 18, 33, 9,
+			13, 22, 25, 15, 11, 30, 7, 14, 17,
+		]);
+
+		let res = test_api::get_and_return_array(input);
+
+		assert_eq!(&res, &input[..16]);
+	}
+
+	fn test_array_as_mutable_reference() {
+		let mut array = [0u8; 16];
+		test_api::array_as_mutable_reference(&mut array);
+
+		assert_eq!(array, TEST_ARRAY);
+	}
+
+	fn test_return_input_public_key() {
+		let key = Public::try_from(
+			&[
+				1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24,
+				25, 26, 27, 28, 29, 30, 31, 32,
+			][..],
+		).unwrap();
+		let ret_key = test_api::return_input_public_key(key.clone());
+
+		let key_data: &[u8] = key.as_ref();
+		let ret_key_data: &[u8] = ret_key.as_ref();
+		assert_eq!(key_data, ret_key_data);
+	}
+
+	fn test_invalid_utf8_data_should_return_an_error() {
+		let data = vec![0, 159, 146, 150];
+		// I'm an evil hacker, trying to hack!
+		let data_str = unsafe { rstd::str::from_utf8_unchecked(&data) };
+
+		test_api::invalid_utf8_data(data_str);
+	}
+
+	fn test_overwrite_native_function_implementation() {
+		fn new_implementation() -> bool {
+			true
+		}
+
+		// Check native implementation
+		assert!(!test_api::overwrite_native_function_implementation());
+
+		let _guard = test_api::host_overwrite_native_function_implementation
+			.replace_implementation(new_implementation);
+
+		assert!(test_api::overwrite_native_function_implementation());
+	}
+}
diff --git a/core/serializer/Cargo.toml b/primitives/serializer/Cargo.toml
similarity index 100%
rename from core/serializer/Cargo.toml
rename to primitives/serializer/Cargo.toml
diff --git a/core/serializer/src/lib.rs b/primitives/serializer/src/lib.rs
similarity index 100%
rename from core/serializer/src/lib.rs
rename to primitives/serializer/src/lib.rs
diff --git a/core/session/Cargo.toml b/primitives/session/Cargo.toml
similarity index 55%
rename from core/session/Cargo.toml
rename to primitives/session/Cargo.toml
index 5d8cb3f0001ba..d1490905c200b 100644
--- a/core/session/Cargo.toml
+++ b/primitives/session/Cargo.toml
@@ -5,11 +5,10 @@ authors = ["Parity Technologies "]
 edition = "2018"
 
 [dependencies]
-client = { package = "substrate-client", path = "../client", default-features = false }
+sr-api = { path = "../sr-api", default-features = false }
 rstd = { package = "sr-std", path = "../sr-std", default-features = false }
 sr-primitives = { path = "../sr-primitives", optional = true }
-primitives = { package = "substrate-primitives", path = "../primitives", optional = true }
 
 [features]
 default = [ "std" ]
-std = [ "client/std", "rstd/std", "sr-primitives", "primitives" ]
+std = [ "sr-api/std", "rstd/std", "sr-primitives" ]
diff --git a/core/session/src/lib.rs b/primitives/session/src/lib.rs
similarity index 63%
rename from core/session/src/lib.rs
rename to primitives/session/src/lib.rs
index 1b40d2d9ba815..adc7629c36816 100644
--- a/core/session/src/lib.rs
+++ b/primitives/session/src/lib.rs
@@ -21,11 +21,9 @@
 use rstd::vec::Vec;
 
 #[cfg(feature = "std")]
-use sr_primitives::traits::{ProvideRuntimeApi, Block as BlockT};
-#[cfg(feature = "std")]
-use primitives::{H256, Blake2Hasher};
+use sr_primitives::{generic::BlockId, traits::{ProvideRuntimeApi, Block as BlockT}};
 
-client::decl_runtime_apis! {
+sr_api::decl_runtime_apis! {
 	/// Session keys runtime api.
 	pub trait SessionKeys {
 		/// Generate a set of session keys with optionally using the given seed.
@@ -39,28 +37,23 @@ client::decl_runtime_apis! {
 	}
 }
 
-/// Generate the initial session keys with the given seeds and store them in
+/// Generate the initial session keys with the given seeds, at the given block and store them in
 /// the client's keystore.
 #[cfg(feature = "std")]
-pub fn generate_initial_session_keys(
-	client: std::sync::Arc>,
+pub fn generate_initial_session_keys(
+	client: std::sync::Arc,
+	at: &BlockId,
 	seeds: Vec,
-) -> Result<(), client::error::Error>
+) -> Result<(), <::Api as sr_api::ApiExt>::Error>
 where
-	B: client::backend::Backend,
-	E: client::CallExecutor,
-	Block: BlockT,
-	client::Client: ProvideRuntimeApi,
-	 as ProvideRuntimeApi>::Api: SessionKeys,
+	Block: BlockT,
+	T: ProvideRuntimeApi,
+	::Api: SessionKeys,
 {
-	let info = client.info().chain;
 	let runtime_api = client.runtime_api();
 
 	for seed in seeds {
-		runtime_api.generate_session_keys(
-			&sr_primitives::generic::BlockId::Number(info.best_number),
-			Some(seed.as_bytes().to_vec()),
-		)?;
+		runtime_api.generate_session_keys(at, Some(seed.as_bytes().to_vec()))?;
 	}
 
 	Ok(())
diff --git a/primitives/sr-api/Cargo.toml b/primitives/sr-api/Cargo.toml
new file mode 100644
index 0000000000000..a23a36284d8cb
--- /dev/null
+++ b/primitives/sr-api/Cargo.toml
@@ -0,0 +1,33 @@
+[package]
+name = "sr-api"
+version = "2.0.0"
+authors = ["Parity Technologies "]
+edition = "2018"
+
+[dependencies]
+codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false }
+sr-api-proc-macro = { path = "proc-macro" }
+primitives = { package = "substrate-primitives", path = "../core", default-features = false }
+rstd = { package = "sr-std", path = "../sr-std", default-features = false }
+sr-primitives = { path = "../sr-primitives", default-features = false }
+sr-version = { path = "../sr-version", default-features = false }
+state-machine = { package = "substrate-state-machine", path = "../../primitives/state-machine", optional = true }
+
+[dev-dependencies]
+criterion = "0.3.0"
+test-client = { package = "substrate-test-runtime-client", path = "../../test/utils/runtime/client" }
+
+[[bench]]
+name = "bench"
+harness = false
+
+[features]
+default = [ "std" ]
+std = [
+	"codec/std",
+	"primitives/std",
+	"rstd/std",
+	"sr-primitives/std",
+	"state-machine",
+	"sr-version/std",
+]
diff --git a/core/sr-api-macros/benches/bench.rs b/primitives/sr-api/benches/bench.rs
similarity index 96%
rename from core/sr-api-macros/benches/bench.rs
rename to primitives/sr-api/benches/bench.rs
index 9aba38c2d1820..49c8e1e380415 100644
--- a/core/sr-api-macros/benches/bench.rs
+++ b/primitives/sr-api/benches/bench.rs
@@ -16,8 +16,8 @@
 
 use criterion::{Criterion, criterion_group, criterion_main};
 use test_client::{
-    DefaultTestClientBuilderExt, TestClientBuilder,
-    TestClientBuilderExt, runtime::TestAPI,
+	DefaultTestClientBuilderExt, TestClientBuilder,
+	TestClientBuilderExt, runtime::TestAPI,
 };
 use sr_primitives::{generic::BlockId, traits::ProvideRuntimeApi};
 use state_machine::ExecutionStrategy;
diff --git a/primitives/sr-api/proc-macro/Cargo.toml b/primitives/sr-api/proc-macro/Cargo.toml
new file mode 100644
index 0000000000000..bd6418c458f0a
--- /dev/null
+++ b/primitives/sr-api/proc-macro/Cargo.toml
@@ -0,0 +1,26 @@
+[package]
+name = "sr-api-proc-macro"
+version = "2.0.0"
+authors = ["Parity Technologies "]
+edition = "2018"
+
+[lib]
+proc-macro = true
+
+[dependencies]
+quote = "1.0.2"
+syn = { version = "1.0.8", features = [ "full", "fold", "extra-traits", "visit" ] }
+proc-macro2 = "1.0.6"
+blake2-rfc = "0.2.18"
+proc-macro-crate = "0.1.4"
+
+[dev-dependencies]
+sr-api = { path = ".." }
+sr-primitives = { path = "../../sr-primitives" }
+sr-version = { path = "../../sr-version" }
+test-client = { package = "substrate-test-runtime-client", path = "../../../test/utils/runtime/client" }
+
+# Required for the doc tests
+[features]
+default = [ "std" ]
+std = []
diff --git a/core/sr-api-macros/src/decl_runtime_apis.rs b/primitives/sr-api/proc-macro/src/decl_runtime_apis.rs
similarity index 85%
rename from core/sr-api-macros/src/decl_runtime_apis.rs
rename to primitives/sr-api/proc-macro/src/decl_runtime_apis.rs
index 12639bd1c1f17..b57cc739975c4 100644
--- a/core/sr-api-macros/src/decl_runtime_apis.rs
+++ b/primitives/sr-api/proc-macro/src/decl_runtime_apis.rs
@@ -19,6 +19,7 @@ use crate::utils::{
 	fold_fn_decl_for_client_side, unwrap_or_error, extract_parameter_names_types_and_borrows,
 	generate_native_call_generator_fn_name, return_type_extract_type,
 	generate_method_runtime_api_impl_name, generate_call_api_at_fn_name, prefix_function_with_trait,
+	replace_wild_card_parameter_names,
 };
 
 use proc_macro2::{TokenStream, Span};
@@ -27,9 +28,8 @@ use quote::quote;
 
 use syn::{
 	spanned::Spanned, parse_macro_input, parse::{Parse, ParseStream, Result, Error}, ReturnType,
-	fold::{self, Fold}, parse_quote, ItemTrait, Generics, GenericParam, Attribute, FnArg,
-	visit::{Visit, self}, Pat, TraitBound, Meta, NestedMeta, Lit, TraitItem, Ident, Type,
-	TraitItemMethod
+	fold::{self, Fold}, parse_quote, ItemTrait, Generics, GenericParam, Attribute, FnArg, Type,
+	visit::{Visit, self}, TraitBound, Meta, NestedMeta, Lit, TraitItem, Ident, TraitItemMethod,
 };
 
 use std::collections::HashMap;
@@ -95,9 +95,9 @@ impl Parse for RuntimeApiDecls {
 fn extend_generics_with_block(generics: &mut Generics) {
 	let c = generate_crate_access(HIDDEN_INCLUDES_ID);
 
-	generics.lt_token = Some(parse_quote!(<));
-	generics.params.insert(0, parse_quote!( Block: #c::runtime_api::BlockT ));
-	generics.gt_token = Some(parse_quote!(>));
+	generics.lt_token = Some(Default::default());
+	generics.params.insert(0, parse_quote!( Block: #c::BlockT ));
+	generics.gt_token = Some(Default::default());
 }
 
 /// Remove all attributes from the vector that are supported by us in the declaration of a runtime
@@ -182,26 +182,26 @@ fn generate_native_call_generators(decl: &ItemTrait) -> Result {
 	let crate_ = generate_crate_access(HIDDEN_INCLUDES_ID);
 
 	// Auxiliary function that is used to convert between types that use different block types.
-	// The function expects that both a convertable by encoding the one and decoding the other.
+	// The function expects that both are convertible by encoding the one and decoding the other.
 	result.push(quote!(
 		#[cfg(any(feature = "std", test))]
 		fn convert_between_block_types
-			(
+			(
 				input: &I, error_desc: &'static str,
 			) -> std::result::Result
 		{
-			::decode(
-				&mut &#crate_::runtime_api::Encode::encode(input)[..],
+			::decode(
+				&mut &#crate_::Encode::encode(input)[..],
 			).map_err(|e| format!("{} {}", error_desc, e.what()))
 		}
 	));
 
 	// Generate a native call generator for each function of the given trait.
 	for fn_ in fns {
-		let params = extract_parameter_names_types_and_borrows(&fn_.decl)?;
+		let params = extract_parameter_names_types_and_borrows(&fn_)?;
 		let trait_fn_name = &fn_.ident;
 		let fn_name = generate_native_call_generator_fn_name(&fn_.ident);
-		let output = return_type_replace_block_with_node_block(fn_.decl.output.clone());
+		let output = return_type_replace_block_with_node_block(fn_.output.clone());
 		let output_ty = return_type_extract_type(&output);
 		let output = quote!( std::result::Result<#output_ty, String> );
 
@@ -217,7 +217,7 @@ fn generate_native_call_generators(decl: &ItemTrait) -> Result {
 		});
 		// Same as for the input types, we need to check if we also need to convert the output,
 		// before returning it.
-		let output_conversion = if return_type_is_using_block(&fn_.decl.output) {
+		let output_conversion = if return_type_is_using_block(&fn_.output) {
 			quote!(
 				convert_between_block_types(
 					&res,
@@ -234,22 +234,21 @@ fn generate_native_call_generators(decl: &ItemTrait) -> Result {
 		// the user. Otherwise if it is not using the block, we don't need to add anything.
 		let input_borrows = params
 			.iter()
-			.map(|v| if type_is_using_block(&v.1) { v.2.clone() } else { quote!() });
+			.map(|v| if type_is_using_block(&v.1) { v.2.clone() } else { None });
 
 		// Replace all `Block` with `NodeBlock`, add `'a` lifetime to references and collect
 		// all the function inputs.
 		let fn_inputs = fn_
-			.decl
 			.inputs
 			.iter()
 			.map(|v| fn_arg_replace_block_with_node_block(v.clone()))
 			.map(|v| match v {
-				FnArg::Captured(ref arg) => {
+				FnArg::Typed(ref arg) => {
 					let mut arg = arg.clone();
-					if let Type::Reference(ref mut r) = arg.ty {
+					if let Type::Reference(ref mut r) = *arg.ty {
 						r.lifetime = Some(parse_quote!( 'a ));
 					}
-					FnArg::Captured(arg)
+					FnArg::Typed(arg)
 				},
 				r => r.clone(),
 			});
@@ -273,7 +272,7 @@ fn generate_native_call_generators(decl: &ItemTrait) -> Result {
 		result.push(quote!(
 			#[cfg(any(feature = "std", test))]
 			pub fn #fn_name<
-				'a, ApiImpl: #trait_ #ty_generics, NodeBlock: #crate_::runtime_api::BlockT
+				'a, ApiImpl: #trait_ #ty_generics, NodeBlock: #crate_::BlockT
 				#(, #impl_generics_params)*
 			>(
 				#( #fn_inputs ),*
@@ -310,15 +309,15 @@ fn parse_renamed_attribute(renamed: &Attribute) -> Result<(String, u32)> {
 			} else {
 				let mut itr = list.nested.iter();
 				let old_name = match itr.next() {
-					Some(NestedMeta::Literal(Lit::Str(i))) => {
+					Some(NestedMeta::Lit(Lit::Str(i))) => {
 						i.value()
 					},
 					_ => return err,
 				};
 
 				let version = match itr.next() {
-					Some(NestedMeta::Literal(Lit::Int(i))) => {
-						i.value() as u32
+					Some(NestedMeta::Lit(Lit::Int(i))) => {
+						i.base10_parse()?
 					},
 					_ => return err,
 				};
@@ -397,24 +396,24 @@ fn generate_call_api_at_calls(decl: &ItemTrait) -> Result {
 		result.push(quote!(
 			#[cfg(any(feature = "std", test))]
 			pub fn #fn_name<
-				R: #crate_::runtime_api::Encode + #crate_::runtime_api::Decode + PartialEq,
+				R: #crate_::Encode + #crate_::Decode + PartialEq,
 				NC: FnOnce() -> std::result::Result + std::panic::UnwindSafe,
-				Block: #crate_::runtime_api::BlockT,
-				T: #crate_::runtime_api::CallRuntimeAt,
-				C: #crate_::runtime_api::Core,
+				Block: #crate_::BlockT,
+				T: #crate_::CallRuntimeAt,
+				C: #crate_::Core,
 			>(
 				call_runtime_at: &T,
 				core_api: &C,
-				at: &#crate_::runtime_api::BlockId,
+				at: &#crate_::BlockId,
 				args: Vec,
-				changes: &std::cell::RefCell<#crate_::runtime_api::OverlayedChanges>,
-				initialized_block: &std::cell::RefCell>>,
+				changes: &std::cell::RefCell<#crate_::OverlayedChanges>,
+				initialized_block: &std::cell::RefCell>>,
 				native_call: Option,
-				context: #crate_::runtime_api::ExecutionContext,
-				recorder: &Option>>>,
-			) -> #crate_::error::Result<#crate_::runtime_api::NativeOrEncoded> {
+				context: #crate_::ExecutionContext,
+				recorder: &Option<#crate_::ProofRecorder>,
+			) -> std::result::Result<#crate_::NativeOrEncoded, T::Error> {
 				let version = call_runtime_at.runtime_version_at(at)?;
-				use #crate_::runtime_api::InitializeBlock;
+				use #crate_::InitializeBlock;
 				let initialize_block = if #skip_initialize_block {
 					InitializeBlock::Skip
 				} else {
@@ -488,6 +487,8 @@ fn generate_runtime_decls(decls: &[ItemTrait]) -> TokenStream {
 				if remove_supported_attributes(&mut method.attrs).contains_key(CHANGED_IN_ATTRIBUTE) {
 					None
 				} else {
+					// Make sure we replace all the wild card parameter names.
+					replace_wild_card_parameter_names(&mut method.sig);
 					Some(TraitItem::Method(method.clone()))
 				}
 			}
@@ -552,7 +553,7 @@ impl<'a> ToClientSideDecl<'a> {
 	fn fold_trait_item_method(&mut self, method: TraitItemMethod)
 		-> (TraitItemMethod, Option, TraitItemMethod) {
 		let crate_ = self.crate_;
-		let context = quote!( #crate_::runtime_api::ExecutionContext::OffchainCall(None) );
+		let context = quote!( #crate_::ExecutionContext::OffchainCall(None) );
 		let fn_impl = self.create_method_runtime_api_impl(method.clone());
 		let fn_decl = self.create_method_decl(method.clone(), context);
 		let fn_decl_ctx = self.create_method_decl_with_context(method);
@@ -562,10 +563,10 @@ impl<'a> ToClientSideDecl<'a> {
 
 	fn create_method_decl_with_context(&mut self, method: TraitItemMethod) -> TraitItemMethod {
 		let crate_ = self.crate_;
-		let context_arg: syn::FnArg = parse_quote!( context: #crate_::runtime_api::ExecutionContext );
+		let context_arg: syn::FnArg = parse_quote!( context: #crate_::ExecutionContext );
 		let mut fn_decl_ctx = self.create_method_decl(method, quote!( context ));
 		fn_decl_ctx.sig.ident = Ident::new(&format!("{}_with_context", &fn_decl_ctx.sig.ident), Span::call_site());
-		fn_decl_ctx.sig.decl.inputs.insert(2, context_arg);
+		fn_decl_ctx.sig.inputs.insert(2, context_arg);
 
 		fn_decl_ctx
 	}
@@ -577,12 +578,12 @@ impl<'a> ToClientSideDecl<'a> {
 			return None;
 		}
 
-		let fn_decl = &method.sig.decl;
-		let ret_type = return_type_extract_type(&fn_decl.output);
+		let fn_sig = &method.sig;
+		let ret_type = return_type_extract_type(&fn_sig.output);
 
 		// Get types and if the value is borrowed from all parameters.
 		// If there is an error, we push it as the block to the user.
-		let param_types = match extract_parameter_names_types_and_borrows(fn_decl) {
+		let param_types = match extract_parameter_names_types_and_borrows(fn_sig) {
 			Ok(res) => res.into_iter().map(|v| {
 				let ty = v.1;
 				let borrow = v.2;
@@ -603,10 +604,10 @@ impl<'a> ToClientSideDecl<'a> {
 				fn #name(
 					&self,
 					at: &#block_id,
-					context: #crate_::runtime_api::ExecutionContext,
+					context: #crate_::ExecutionContext,
 					params: Option<( #( #param_types ),* )>,
 					params_encoded: Vec,
-				) -> #crate_::error::Result<#crate_::runtime_api::NativeOrEncoded<#ret_type>>;
+				) -> std::result::Result<#crate_::NativeOrEncoded<#ret_type>, Self::Error>;
 			}
 		)
 	}
@@ -614,8 +615,12 @@ impl<'a> ToClientSideDecl<'a> {
 	/// Takes the method declared by the user and creates the declaration we require for the runtime
 	/// api client side. This method will call by default the `method_runtime_api_impl` for doing
 	/// the actual call into the runtime.
-	fn create_method_decl(&mut self, mut method: TraitItemMethod, context: TokenStream) -> TraitItemMethod {
-		let params = match extract_parameter_names_types_and_borrows(&method.sig.decl) {
+	fn create_method_decl(
+		&mut self,
+		mut method: TraitItemMethod,
+		context: TokenStream,
+	) -> TraitItemMethod {
+		let params = match extract_parameter_names_types_and_borrows(&method.sig) {
 			Ok(res) => res.into_iter().map(|v| v.0).collect::>(),
 			Err(e) => {
 				self.errors.push(e.to_compile_error());
@@ -623,13 +628,10 @@ impl<'a> ToClientSideDecl<'a> {
 			}
 		};
 		let params2 = params.clone();
-		let ret_type = return_type_extract_type(&method.sig.decl.output);
+		let ret_type = return_type_extract_type(&method.sig.output);
+
+		fold_fn_decl_for_client_side(&mut method.sig, &self.block_id);
 
-		method.sig.decl = fold_fn_decl_for_client_side(
-			method.sig.decl.clone(),
-			&self.block_id,
-			&self.crate_
-		);
 		let name_impl = generate_method_runtime_api_impl_name(&self.trait_, &method.sig.ident);
 		let crate_ = self.crate_;
 
@@ -650,7 +652,7 @@ impl<'a> ToClientSideDecl<'a> {
 
 				let ident = Ident::new(
 					&format!("{}_before_version_{}", method.sig.ident, version),
-					method.sig.ident.span()
+					method.sig.ident.span(),
 				);
 				method.sig.ident = ident;
 				method.attrs.push(parse_quote!( #[deprecated] ));
@@ -672,24 +674,30 @@ impl<'a> ToClientSideDecl<'a> {
 			parse_quote! {
 				{
 					let runtime_api_impl_params_encoded =
-						#crate_::runtime_api::Encode::encode(&( #( &#params ),* ));
-
-					self.#name_impl(at, #context, #param_tuple, runtime_api_impl_params_encoded)
-						.and_then(|r|
-							match r {
-								#crate_::runtime_api::NativeOrEncoded::Native(n) => {
-									#native_handling
-								},
-								#crate_::runtime_api::NativeOrEncoded::Encoded(r) => {
-									<#ret_type as #crate_::runtime_api::Decode>::decode(&mut &r[..])
-										.map_err(|err|
-											#crate_::error::Error::CallResultDecode(
-												#function_name, err
-											).into()
-										)
-								}
+						#crate_::Encode::encode(&( #( &#params ),* ));
+
+					self.#name_impl(
+						__runtime_api_at_param__,
+						#context,
+						#param_tuple,
+						runtime_api_impl_params_encoded,
+					).and_then(|r|
+						match r {
+							#crate_::NativeOrEncoded::Native(n) => {
+								#native_handling
+							},
+							#crate_::NativeOrEncoded::Encoded(r) => {
+								<#ret_type as #crate_::Decode>::decode(&mut &r[..])
+									.map_err(|err|
+										format!(
+											"Failed to decode result of `{}`: {}",
+											#function_name,
+											err.what(),
+										).into()
+									)
 							}
-						)
+						}
+					)
 				}
 			}
 		);
@@ -714,12 +722,12 @@ impl<'a> Fold for ToClientSideDecl<'a> {
 				'static
 				+ Send
 				+ Sync
-				+ #crate_::runtime_api::ApiExt<#block_ident>
+				+ #crate_::ApiExt<#block_ident>
 			);
 		} else {
 			// Add the `Core` runtime api as super trait.
 			let crate_ = &self.crate_;
-			input.supertraits.push(parse_quote!( #crate_::runtime_api::Core<#block_ident> ));
+			input.supertraits.push(parse_quote!( #crate_::Core<#block_ident> ));
 		}
 
 		// The client side trait is only required when compiling with the feature `std` or `test`.
@@ -745,15 +753,12 @@ fn parse_runtime_api_version(version: &Attribute) -> Result {
 
 	match meta {
 		Meta::List(list) => {
-			if list.nested.len() > 1 && list.nested.is_empty() {
+			if list.nested.len() != 1 {
 				err
+			} else if let Some(NestedMeta::Lit(Lit::Int(i))) = list.nested.first() {
+				i.base10_parse()
 			} else {
-				match list.nested.first().as_ref().map(|v| v.value()) {
-					Some(NestedMeta::Literal(Lit::Int(i))) => {
-						Ok(i.value())
-					},
-					_ => err,
-				}
+				err
 			}
 		},
 		_ => err,
@@ -780,12 +785,24 @@ fn generate_runtime_info_impl(trait_: &ItemTrait, version: u64) -> TokenStream {
 	let crate_ = generate_crate_access(HIDDEN_INCLUDES_ID);
 	let id = generate_runtime_api_id(&trait_name.to_string());
 	let version = generate_runtime_api_version(version as u32);
-	let (impl_generics, ty_generics, where_clause) = trait_.generics.split_for_impl();
+
+	let impl_generics = trait_.generics.type_params().map(|t| {
+		let ident = &t.ident;
+		let colon_token = &t.colon_token;
+		let bounds = &t.bounds;
+
+		quote! { #ident #colon_token #bounds }
+	}).chain(std::iter::once(quote! { __Sr_Api_Error__ }));
+
+	let ty_generics = trait_.generics.type_params().map(|t| {
+		let ident = &t.ident;
+		quote! { #ident }
+	}).chain(std::iter::once(quote! { Error = __Sr_Api_Error__ }));
 
 	quote!(
 		#[cfg(any(feature = "std", test))]
-		impl #impl_generics #crate_::runtime_api::RuntimeApiInfo
-			for #trait_name #ty_generics #where_clause
+		impl < #( #impl_generics, )* > #crate_::RuntimeApiInfo
+			for #trait_name < #( #ty_generics, )* >
 		{
 			#id
 			#version
@@ -813,7 +830,7 @@ fn generate_client_side_decls(decls: &[ItemTrait]) -> TokenStream {
 		let decl = decl.clone();
 
 		let crate_ = generate_crate_access(HIDDEN_INCLUDES_ID);
-		let block_id = quote!( #crate_::runtime_api::BlockId );
+		let block_id = quote!( #crate_::BlockId );
 		let mut found_attributes = HashMap::new();
 		let mut errors = Vec::new();
 		let trait_ = decl.ident.clone();
@@ -848,32 +865,8 @@ struct CheckTraitDecl {
 
 impl<'ast> Visit<'ast> for CheckTraitDecl {
 	fn visit_fn_arg(&mut self, input: &'ast FnArg) {
-		match input {
-			FnArg::Captured(ref arg) => {
-				match arg.pat {
-					Pat::Ident(ref pat) if pat.ident == "at" => {
-						self.errors.push(
-							Error::new(
-								pat.span(),
-								"`decl_runtime_apis!` adds automatically a parameter \
-								`at: &BlockId`. Please rename/remove your parameter."
-							)
-						)
-					},
-					_ => {}
-				}
-			},
-			FnArg::SelfRef(_) | FnArg::SelfValue(_) => {
-				self.errors.push(Error::new(input.span(), "Self values are not supported."))
-			}
-			_ => {
-				self.errors.push(
-					Error::new(
-						input.span(),
-						"Only function arguments in the form `pat: type` are supported."
-					)
-				)
-			}
+		if let FnArg::Receiver(_) = input {
+			self.errors.push(Error::new(input.span(), "`self` as argument not supported."))
 		}
 
 		visit::visit_fn_arg(self, input);
@@ -897,7 +890,7 @@ impl<'ast> Visit<'ast> for CheckTraitDecl {
 	}
 
 	fn visit_trait_bound(&mut self, input: &'ast TraitBound) {
-		if let Some(last_ident) = input.path.segments.last().map(|v| &v.value().ident) {
+		if let Some(last_ident) = input.path.segments.last().map(|v| &v.ident) {
 			if last_ident == "BlockT" || last_ident == BLOCK_GENERIC_IDENT {
 				self.errors.push(
 					Error::new(
diff --git a/core/sr-api-macros/src/impl_runtime_apis.rs b/primitives/sr-api/proc-macro/src/impl_runtime_apis.rs
similarity index 81%
rename from core/sr-api-macros/src/impl_runtime_apis.rs
rename to primitives/sr-api/proc-macro/src/impl_runtime_apis.rs
index 28eb5c60729ed..6a18651655f32 100644
--- a/core/sr-api-macros/src/impl_runtime_apis.rs
+++ b/primitives/sr-api/proc-macro/src/impl_runtime_apis.rs
@@ -26,7 +26,7 @@ use proc_macro2::{Span, TokenStream};
 use quote::quote;
 
 use syn::{
-	spanned::Spanned, parse_macro_input, Ident, Type, ItemImpl, MethodSig, Path,
+	spanned::Spanned, parse_macro_input, Ident, Type, ItemImpl, Path, Signature,
 	ImplItem, parse::{Parse, ParseStream, Result, Error}, PathArguments, GenericArgument, TypePath,
 	fold::{self, Fold}, parse_quote
 };
@@ -56,12 +56,12 @@ impl Parse for RuntimeApiImpls {
 /// Generates the call to the implementation of the requested function.
 /// The generated code includes decoding of the input arguments and encoding of the output.
 fn generate_impl_call(
-	signature: &MethodSig,
+	signature: &Signature,
 	runtime: &Type,
 	input: &Ident,
 	impl_trait: &Path
 ) -> Result {
-	let params = extract_parameter_names_types_and_borrows(&signature.decl)?;
+	let params = extract_parameter_names_types_and_borrows(signature)?;
 
 	let c = generate_crate_access(HIDDEN_INCLUDES_ID);
 	let c_iter = iter::repeat(&c);
@@ -76,7 +76,7 @@ fn generate_impl_call(
 	Ok(
 		quote!(
 			#(
-				let #pnames : #ptypes = match #c_iter::runtime_api::Decode::decode(&mut #input) {
+				let #pnames : #ptypes = match #c_iter::Decode::decode(&mut #input) {
 					Ok(input) => input,
 					Err(e) => panic!("Bad input data provided to {}: {}", #fn_name_str, e.what()),
 				};
@@ -110,23 +110,20 @@ fn extract_impl_trait<'a>(impl_: &'a ItemImpl) -> Result<&'a Path> {
 /// Extracts the runtime block identifier.
 fn extract_runtime_block_ident(trait_: &Path) -> Result<&TypePath> {
 	let span = trait_.span();
-	let segment = trait_
+	let generics = trait_
 		.segments
 		.last()
-		.ok_or_else(
-			|| Error::new(span, "Empty path not supported")
-		)?;
-	let generics = segment.value();
+		.ok_or_else(|| Error::new(span, "Empty path not supported"))?;
 
 	match &generics.arguments {
 		PathArguments::AngleBracketed(ref args) => {
-			args.args.first().and_then(|v| match v.value() {
-			GenericArgument::Type(Type::Path(block)) => Some(block),
+			args.args.first().and_then(|v| match v {
+			GenericArgument::Type(Type::Path(ref block)) => Some(block),
 				_ => None
 			}).ok_or_else(|| Error::new(args.span(), "Missing `Block` generic parameter."))
 		},
 		PathArguments::None => {
-			let span = trait_.segments.last().as_ref().unwrap().value().span();
+			let span = trait_.segments.last().as_ref().unwrap().span();
 			Err(Error::new(span, "Missing `Block` generic parameter."))
 		},
 		PathArguments::Parenthesized(_) => {
@@ -149,7 +146,6 @@ fn generate_impl_calls(
 			.segments
 			.last()
 			.ok_or_else(|| Error::new(impl_trait_path.span(), "Empty trait path not possible!"))?
-			.value()
 			.ident;
 
 		for item in &impl_.items {
@@ -179,7 +175,7 @@ fn generate_dispatch_function(impls: &[ItemImpl]) -> Result {
 		.into_iter()
 		.map(|(trait_, fn_name, impl_)| {
 			let name = prefix_function_with_trait(&trait_, &fn_name);
-			quote!( #name => Some(#c::runtime_api::Encode::encode(&{ #impl_ })), )
+			quote!( #name => Some(#c::Encode::encode(&{ #impl_ })), )
 		});
 
 	Ok(quote!(
@@ -213,12 +209,12 @@ fn generate_wasm_interface(impls: &[ItemImpl]) -> Result {
 						&[0u8; 0]
 					} else {
 						unsafe {
-							#c::runtime_api::slice::from_raw_parts(input_data, input_len)
+							#c::slice::from_raw_parts(input_data, input_len)
 						}
 					};
 
 					let output = { #impl_ };
-					#c::runtime_api::to_substrate_wasm_fn_return_value(&output)
+					#c::to_substrate_wasm_fn_return_value(&output)
 				}
 			)
 		});
@@ -235,8 +231,8 @@ fn generate_block_and_block_id_ty(
 	let trait_ = Ident::new(trait_, Span::call_site());
 	let assoc_type = Ident::new(assoc_type, Span::call_site());
 
-	let block = quote!( <#runtime as #crate_::runtime_api::#trait_>::#assoc_type );
-	let block_id = quote!( #crate_::runtime_api::BlockId<#block> );
+	let block = quote!( <#runtime as #crate_::#trait_>::#assoc_type );
+	let block_id = quote!( #crate_::BlockId<#block> );
 
 	(block, block_id)
 }
@@ -256,30 +252,30 @@ fn generate_runtime_api_base_structures(impls: &[ItemImpl]) -> Result + 'static> {
+		pub struct RuntimeApiImpl + 'static> {
 			call: &'static C,
 			commit_on_success: std::cell::RefCell,
 			initialized_block: std::cell::RefCell>,
-			changes: std::cell::RefCell<#crate_::runtime_api::OverlayedChanges>,
-			recorder: Option>>>,
+			changes: std::cell::RefCell<#crate_::OverlayedChanges>,
+			recorder: Option<#crate_::ProofRecorder<#block>>,
 		}
 
 		// `RuntimeApi` itself is not threadsafe. However, an instance is only available in a
 		// `ApiRef` object and `ApiRef` also has an associated lifetime. This lifetimes makes it
 		// impossible to move `RuntimeApi` into another thread.
 		#[cfg(any(feature = "std", test))]
-		unsafe impl> Send for RuntimeApiImpl {}
+		unsafe impl> Send for RuntimeApiImpl {}
 		#[cfg(any(feature = "std", test))]
-		unsafe impl> Sync for RuntimeApiImpl {}
+		unsafe impl> Sync for RuntimeApiImpl {}
 
 		#[cfg(any(feature = "std", test))]
-		impl> #crate_::runtime_api::ApiExt<#block>
-			for RuntimeApiImpl
-		{
+		impl> #crate_::ApiExt<#block> for RuntimeApiImpl {
+			type Error = C::Error;
+
 			fn map_api_result std::result::Result, R, E>(
 				&self,
 				map_call: F
-			) -> ::std::result::Result where Self: Sized {
+			) -> std::result::Result where Self: Sized {
 				*self.commit_on_success.borrow_mut() = false;
 				let res = map_call(self);
 				*self.commit_on_success.borrow_mut() = true;
@@ -292,7 +288,7 @@ fn generate_runtime_api_base_structures(impls: &[ItemImpl]) -> Result #crate_::error::Result<#crate_::runtime_api::RuntimeVersion> {
+			) -> std::result::Result<#crate_::RuntimeVersion, C::Error> {
 				self.call.runtime_version_at(at)
 			}
 
@@ -300,30 +296,30 @@ fn generate_runtime_api_base_structures(impls: &[ItemImpl]) -> Result Option>> {
+			fn extract_proof(&mut self) -> Option<#crate_::StorageProof> {
 				self.recorder
 					.take()
-					.map(|r| {
-						r.borrow_mut()
-							.drain()
-							.into_iter()
-							.map(|n| n.data.to_vec())
-							.collect()
+					.map(|recorder| {
+						let trie_nodes = recorder.read()
+							.iter()
+							.filter_map(|(_k, v)| v.as_ref().map(|v| v.to_vec()))
+							.collect();
+						#crate_::StorageProof::new(trie_nodes)
 					})
 			}
 		}
 
 		#[cfg(any(feature = "std", test))]
-		impl + 'static>
-			#crate_::runtime_api::ConstructRuntimeApi<#block, C> for RuntimeApi
+		impl + 'static> #crate_::ConstructRuntimeApi<#block, C>
+			for RuntimeApi
 		{
 			type RuntimeApi = RuntimeApiImpl;
 
 			fn construct_runtime_api<'a>(
 				call: &'a C,
-			) -> #crate_::runtime_api::ApiRef<'a, Self::RuntimeApi> {
+			) -> #crate_::ApiRef<'a, Self::RuntimeApi> {
 				RuntimeApiImpl {
-					call: unsafe { ::std::mem::transmute(call) },
+					call: unsafe { std::mem::transmute(call) },
 					commit_on_success: true.into(),
 					initialized_block: None.into(),
 					changes: Default::default(),
@@ -333,35 +329,34 @@ fn generate_runtime_api_base_structures(impls: &[ItemImpl]) -> Result> RuntimeApiImpl {
+		impl> RuntimeApiImpl {
 			fn call_api_at<
-				R: #crate_::runtime_api::Encode + #crate_::runtime_api::Decode + PartialEq,
+				R: #crate_::Encode + #crate_::Decode + PartialEq,
 				F: FnOnce(
 					&C,
 					&Self,
-					&std::cell::RefCell<#crate_::runtime_api::OverlayedChanges>,
-					&std::cell::RefCell>>,
-					&Option>>>,
-				) -> #crate_::error::Result<#crate_::runtime_api::NativeOrEncoded>,
+					&std::cell::RefCell<#crate_::OverlayedChanges>,
+					&std::cell::RefCell>>,
+					&Option<#crate_::ProofRecorder<#block>>,
+				) -> std::result::Result<#crate_::NativeOrEncoded, E>,
+				E,
 			>(
 				&self,
 				call_api_at: F,
-			) -> #crate_::error::Result<#crate_::runtime_api::NativeOrEncoded> {
-				let res = unsafe {
-					call_api_at(
-						&self.call,
-						self,
-						&self.changes,
-						&self.initialized_block,
-						&self.recorder,
-					)
-				};
+			) -> std::result::Result<#crate_::NativeOrEncoded, E> {
+				let res = call_api_at(
+					&self.call,
+					self,
+					&self.changes,
+					&self.initialized_block,
+					&self.recorder,
+				);
 
 				self.commit_on_ok(&res);
 				res
 			}
 
-			fn commit_on_ok(&self, res: &::std::result::Result) {
+			fn commit_on_ok(&self, res: &std::result::Result) {
 				if *self.commit_on_success.borrow() {
 					if res.is_err() {
 						self.changes.borrow_mut().discard_prospective();
@@ -383,7 +378,6 @@ fn extend_with_runtime_decl_path(mut trait_: Path) -> Path {
 			.last()
 			.as_ref()
 			.expect("Trait path should always contain at least one item; qed")
-			.value()
 			.ident;
 
 		generate_runtime_mod_name_for_trait(trait_name)
@@ -453,16 +447,16 @@ impl<'a> Fold for ApiRuntimeImplToApiRuntimeApiImpl<'a> {
 			let block_id = self.node_block_id;
 
 			// Generate the access to the native parameters
-			let param_tuple_access = if input.sig.decl.inputs.len() == 1 {
+			let param_tuple_access = if input.sig.inputs.len() == 1 {
 				vec![ quote!( p ) ]
 			} else {
-				input.sig.decl.inputs.iter().enumerate().map(|(i, _)| {
+				input.sig.inputs.iter().enumerate().map(|(i, _)| {
 					let i = syn::Index::from(i);
 					quote!( p.#i )
 				}).collect::>()
 			};
 
-			let (param_types, error) = match extract_parameter_names_types_and_borrows(&input.sig.decl) {
+			let (param_types, error) = match extract_parameter_names_types_and_borrows(&input.sig) {
 				Ok(res) => (
 					res.into_iter().map(|v| {
 						let ty = v.1;
@@ -474,26 +468,31 @@ impl<'a> Fold for ApiRuntimeImplToApiRuntimeApiImpl<'a> {
 				Err(e) => (Vec::new(), Some(e.to_compile_error())),
 			};
 
-			let context_arg: syn::FnArg = parse_quote!( context: #crate_::runtime_api::ExecutionContext );
-
 			// Rewrite the input parameters.
-			input.sig.decl.inputs = parse_quote! {
-				&self, at: &#block_id, #context_arg, params: Option<( #( #param_types ),* )>, params_encoded: Vec
+			input.sig.inputs = parse_quote! {
+				&self,
+				at: &#block_id,
+				context: #crate_::ExecutionContext,
+				params: Option<( #( #param_types ),* )>,
+				params_encoded: Vec,
 			};
 
-			input.sig.ident = generate_method_runtime_api_impl_name(&self.impl_trait, &input.sig.ident);
-			let ret_type = return_type_extract_type(&input.sig.decl.output);
+			input.sig.ident = generate_method_runtime_api_impl_name(
+				&self.impl_trait,
+				&input.sig.ident,
+			);
+			let ret_type = return_type_extract_type(&input.sig.output);
 
 			// Generate the correct return type.
-			input.sig.decl.output = parse_quote!(
-				-> #crate_::error::Result<#crate_::runtime_api::NativeOrEncoded<#ret_type>>
+			input.sig.output = parse_quote!(
+				-> std::result::Result<#crate_::NativeOrEncoded<#ret_type>, RuntimeApiImplCall::Error>
 			);
 
 			// Generate the new method implementation that calls into the runtime.
 			parse_quote!(
 				{
 					// Get the error to the user (if we have one).
-					#( #error )*
+					#error
 
 					self.call_api_at(
 						|call_runtime_at, core_api, changes, initialized_block, recorder| {
@@ -533,7 +532,7 @@ impl<'a> Fold for ApiRuntimeImplToApiRuntimeApiImpl<'a> {
 		let crate_ = generate_crate_access(HIDDEN_INCLUDES_ID);
 		let block = self.node_block;
 		input.generics.params.push(
-			parse_quote!( RuntimeApiImplCall: #crate_::runtime_api::CallRuntimeAt<#block> + 'static )
+			parse_quote!( RuntimeApiImplCall: #crate_::CallRuntimeAt<#block> + 'static )
 		);
 
 		// The implementation for the `RuntimeApiImpl` is only required when compiling with
@@ -554,7 +553,7 @@ fn generate_api_impl_for_runtime_api(impls: &[ItemImpl]) -> Result
 			.segments
 			.last()
 			.ok_or_else(|| Error::new(impl_trait_path.span(), "Empty trait path not possible!"))?
-			.into_value();
+			.clone();
 		let runtime_block = extract_runtime_block_ident(impl_trait_path)?;
 		let (node_block, node_block_id) = generate_node_block_and_block_id_ty(&impl_.self_ty);
 		let runtime_type = &impl_.self_ty;
@@ -619,8 +618,7 @@ fn generate_runtime_api_versions(impls: &[ItemImpl]) -> Result {
 	let c = generate_crate_access(HIDDEN_INCLUDES_ID);
 
 	Ok(quote!(
-		const RUNTIME_API_VERSIONS: #c::runtime_api::ApisVec =
-			#c::runtime_api::create_apis_vec!([ #( #result ),* ]);
+		const RUNTIME_API_VERSIONS: #c::ApisVec = #c::create_apis_vec!([ #( #result ),* ]);
 	))
 }
 
diff --git a/core/sr-api-macros/src/lib.rs b/primitives/sr-api/proc-macro/src/lib.rs
similarity index 91%
rename from core/sr-api-macros/src/lib.rs
rename to primitives/sr-api/proc-macro/src/lib.rs
index d88fb09d1b11a..913e6e9d04bfb 100644
--- a/core/sr-api-macros/src/lib.rs
+++ b/primitives/sr-api/proc-macro/src/lib.rs
@@ -44,14 +44,7 @@ mod utils;
 /// # Example
 ///
 /// ```rust
-/// #[macro_use]
-/// extern crate client;
-/// extern crate sr_version as version;
-///
-/// use version::create_runtime_str;
-/// # extern crate test_client;
-/// # extern crate sr_primitives;
-/// # extern crate primitives;
+/// use sr_version::create_runtime_str;
 /// #
 /// # use sr_primitives::traits::GetNodeBlockType;
 /// # use test_client::runtime::{Block, Header};
@@ -63,7 +56,7 @@ mod utils;
 /// #     type NodeBlock = Block;
 /// # }
 /// #
-/// # decl_runtime_apis! {
+/// # sr_api::decl_runtime_apis! {
 /// #     /// Declare the api trait.
 /// #     pub trait Balance {
 /// #         /// Get the balance.
@@ -77,9 +70,9 @@ mod utils;
 /// # }
 ///
 /// /// All runtime api implementations need to be done in one call of the macro!
-/// impl_runtime_apis! {
-/// #   impl client::runtime_api::Core for Runtime {
-/// #       fn version() -> client::runtime_api::RuntimeVersion {
+/// sr_api::impl_runtime_apis! {
+/// #   impl sr_api::Core for Runtime {
+/// #       fn version() -> sr_version::RuntimeVersion {
 /// #           unimplemented!()
 /// #       }
 /// #       fn execute_block(_block: Block) {}
@@ -103,7 +96,7 @@ mod utils;
 /// }
 ///
 /// /// Runtime version. This needs to be declared for each runtime.
-/// pub const VERSION: version::RuntimeVersion = version::RuntimeVersion {
+/// pub const VERSION: sr_version::RuntimeVersion = sr_version::RuntimeVersion {
 ///     spec_name: create_runtime_str!("node"),
 ///     impl_name: create_runtime_str!("test-node"),
 ///     authoring_version: 1,
@@ -134,10 +127,7 @@ pub fn impl_runtime_apis(input: TokenStream) -> TokenStream {
 /// # Example
 ///
 /// ```rust
-/// #[macro_use]
-/// extern crate client;
-///
-/// decl_runtime_apis! {
+/// sr_api::decl_runtime_apis! {
 ///     /// Declare the api trait.
 ///     pub trait Balance {
 ///         /// Get the balance.
@@ -169,10 +159,7 @@ pub fn impl_runtime_apis(input: TokenStream) -> TokenStream {
 /// spec version!). Such a method also does not need to be implemented in the runtime.
 ///
 /// ```rust
-/// #[macro_use]
-/// extern crate client;
-///
-/// decl_runtime_apis! {
+/// sr_api::decl_runtime_apis! {
 ///     /// Declare the api trait.
 ///     #[api_version(2)]
 ///     pub trait Balance {
diff --git a/core/sr-api-macros/src/utils.rs b/primitives/sr-api/proc-macro/src/utils.rs
similarity index 72%
rename from core/sr-api-macros/src/utils.rs
rename to primitives/sr-api/proc-macro/src/utils.rs
index 21000f431b815..a46397be1b849 100644
--- a/core/sr-api-macros/src/utils.rs
+++ b/primitives/sr-api/proc-macro/src/utils.rs
@@ -15,9 +15,15 @@
 // along with Substrate.  If not, see .
 
 use proc_macro2::{TokenStream, Span};
-use syn::{Result, Ident, FnDecl, parse_quote, Type, Pat, spanned::Spanned, FnArg, Error};
+
+use syn::{
+	Result, Ident, Signature, parse_quote, Type, Pat, spanned::Spanned, FnArg, Error, token::And,
+};
+
 use quote::quote;
+
 use std::env;
+
 use proc_macro_crate::crate_name;
 
 /// Unwrap the given result, if it is an error, `compile_error!` will be generated.
@@ -31,17 +37,17 @@ fn generate_hidden_includes_mod_name(unique_id: &'static str) -> Ident {
 
 /// Generates the hidden includes that are required to make the macro independent from its scope.
 pub fn generate_hidden_includes(unique_id: &'static str) -> TokenStream {
-	if env::var("CARGO_PKG_NAME").unwrap() == "substrate-client" {
+	if env::var("CARGO_PKG_NAME").unwrap() == "sr-api" {
 		TokenStream::new()
 	} else {
 		let mod_name = generate_hidden_includes_mod_name(unique_id);
-		match crate_name("substrate-client") {
+		match crate_name("sr-api") {
 			Ok(client_name) => {
 				let client_name = Ident::new(&client_name, Span::call_site());
 				quote!(
 					#[doc(hidden)]
 					mod #mod_name {
-						pub extern crate #client_name as sr_api_client;
+						pub extern crate #client_name as sr_api;
 					}
 				)
 			},
@@ -56,11 +62,11 @@ pub fn generate_hidden_includes(unique_id: &'static str) -> TokenStream {
 
 /// Generates the access to the `substrate_client` crate.
 pub fn generate_crate_access(unique_id: &'static str) -> TokenStream {
-	if env::var("CARGO_PKG_NAME").unwrap() == "substrate-client" {
+	if env::var("CARGO_PKG_NAME").unwrap() == "sr-api" {
 		quote!( crate )
 	} else {
 		let mod_name = generate_hidden_includes_mod_name(unique_id);
-		quote!( self::#mod_name::sr_api_client )
+		quote!( self::#mod_name::sr_api )
 	}.into()
 }
 
@@ -82,23 +88,32 @@ pub fn return_type_extract_type(rt: &syn::ReturnType) -> Type {
 	}
 }
 
-/// Fold the given `FnDecl` to make it usable on the client side.
+/// Replace the `_` (wild card) parameter names in the given signature with unique identifiers.
+pub fn replace_wild_card_parameter_names(input: &mut Signature) {
+	let mut generated_pattern_counter = 0;
+	input.inputs.iter_mut().for_each(|arg| if let FnArg::Typed(arg) = arg {
+		arg.pat = Box::new(
+			generate_unique_pattern((*arg.pat).clone(), &mut generated_pattern_counter),
+		);
+	});
+}
+
+/// Fold the given `Signature` to make it usable on the client side.
 pub fn fold_fn_decl_for_client_side(
-	mut input: FnDecl,
+	input: &mut Signature,
 	block_id: &TokenStream,
-	crate_: &TokenStream
-) -> FnDecl {
+) {
+	replace_wild_card_parameter_names(input);
+
 	// Add `&self, at:& BlockId` as parameters to each function at the beginning.
-	input.inputs.insert(0, parse_quote!( at: &#block_id ));
+	input.inputs.insert(0, parse_quote!( __runtime_api_at_param__: &#block_id ));
 	input.inputs.insert(0, parse_quote!( &self ));
 
 	// Wrap the output in a `Result`
 	input.output = {
 		let ty = return_type_extract_type(&input.output);
-		parse_quote!( -> ::std::result::Result<#ty, #crate_::error::Error> )
+		parse_quote!( -> std::result::Result<#ty, Self::Error> )
 	};
-
-	input
 }
 
 /// Generate an unique pattern based on the given counter, if the given pattern is a `_`.
@@ -106,8 +121,8 @@ pub fn generate_unique_pattern(pat: Pat, counter: &mut u32) -> Pat {
 	match pat {
 		Pat::Wild(_) => {
 			let generated_name = Ident::new(
-				&format!("runtime_api_generated_name_{}", counter),
-				pat.span()
+				&format!("__runtime_api_generated_name_{}__", counter),
+				pat.span(),
 			);
 			*counter += 1;
 
@@ -115,38 +130,31 @@ pub fn generate_unique_pattern(pat: Pat, counter: &mut u32) -> Pat {
 		},
 		_ => pat,
 	}
-}
+ }
 
 /// Extracts the name, the type and `&` or ``(if it is a reference or not)
-/// for each parameter in the given function declaration.
-pub fn extract_parameter_names_types_and_borrows(fn_decl: &FnDecl)
-	-> Result>
+/// for each parameter in the given function signature.
+pub fn extract_parameter_names_types_and_borrows(sig: &Signature)
+	-> Result)>>
 {
 	let mut result = Vec::new();
 	let mut generated_pattern_counter = 0;
-	for input in fn_decl.inputs.iter() {
+	for input in sig.inputs.iter() {
 		match input {
-			FnArg::Captured(arg) => {
-				let (ty, borrow) = match &arg.ty {
+			FnArg::Typed(arg) => {
+				let (ty, borrow) = match &*arg.ty {
 					Type::Reference(t) => {
-						let ty = &t.elem;
-						(parse_quote!( #ty ), quote!( & ))
+						((*t.elem).clone(), Some(t.and_token))
 					},
-					t => { (t.clone(), quote!()) },
+					t => { (t.clone(), None) },
 				};
 
 				let name =
-					generate_unique_pattern(arg.pat.clone(), &mut generated_pattern_counter);
+					generate_unique_pattern((*arg.pat).clone(), &mut generated_pattern_counter);
 				result.push((name, ty, borrow));
 			},
-			_ => {
-				return Err(
-					Error::new(
-						input.span(),
-						"Only function arguments with the following \
-						pattern are accepted: `name: type`!"
-					)
-				)
+			FnArg::Receiver(_) => {
+				return Err(Error::new(input.span(), "`self` parameter not supported!"))
 			}
 		}
 	}
diff --git a/core/client/src/runtime_api.rs b/primitives/sr-api/src/lib.rs
similarity index 70%
rename from core/client/src/runtime_api.rs
rename to primitives/sr-api/src/lib.rs
index 68b49e5910b45..4a5cbf513f13e 100644
--- a/core/client/src/runtime_api.rs
+++ b/primitives/sr-api/src/lib.rs
@@ -1,4 +1,4 @@
-// Copyright 2018-2019 Parity Technologies (UK) Ltd.
+// Copyright 2019 Parity Technologies (UK) Ltd.
 // This file is part of Substrate.
 
 // Substrate is free software: you can redistribute it and/or modify
@@ -14,11 +14,25 @@
 // You should have received a copy of the GNU General Public License
 // along with Substrate.  If not, see .
 
-//! All the functionality required for declaring and implementing runtime apis.
+//! Substrate runtime api
+//!
+//! The Substrate runtime api is the crucial interface between the node and the runtime.
+//! Every call that goes into the runtime is done with a runtime api. The runtime apis are not fixed.
+//! Every Substrate user can define its own apis with
+//! [`decl_runtime_apis`](macro.decl_runtime_apis.html) and implement them in
+//! the runtime with [`impl_runtime_apis`](macro.impl_runtime_apis.html).
+//!
+//! Every Substrate runtime needs to implement the [`Core`] runtime api. This api provides the basic
+//! functionality that every runtime needs to export.
+//!
+//! Besides the macros and the [`Core`] runtime api, this crates provides the [`Metadata`] runtime
+//! api, the [`ApiExt`] trait, the [`CallRuntimeAt`] trait and the [`ConstructRuntimeApi`] trait.
+
+#![cfg_attr(not(feature = "std"), no_std)]
 
 #[doc(hidden)]
 #[cfg(feature = "std")]
-pub use state_machine::OverlayedChanges;
+pub use state_machine::{OverlayedChanges, StorageProof};
 #[doc(hidden)]
 #[cfg(feature = "std")]
 pub use primitives::NativeOrEncoded;
@@ -36,26 +50,23 @@ pub use sr_primitives::{
 #[doc(hidden)]
 pub use primitives::{offchain, ExecutionContext};
 #[doc(hidden)]
-pub use runtime_version::{ApiId, RuntimeVersion, ApisVec, create_apis_vec};
+pub use sr_version::{ApiId, RuntimeVersion, ApisVec, create_apis_vec};
 #[doc(hidden)]
 pub use rstd::{slice, mem};
 #[cfg(feature = "std")]
 use rstd::result;
 #[doc(hidden)]
 pub use codec::{Encode, Decode};
-#[cfg(feature = "std")]
-use crate::error;
-use sr_api_macros::decl_runtime_apis;
 use primitives::OpaqueMetadata;
 #[cfg(feature = "std")]
-use std::{panic::UnwindSafe, cell::RefCell, rc::Rc};
-#[cfg(feature = "std")]
-use primitives::Hasher as HasherT;
+use std::{panic::UnwindSafe, cell::RefCell};
+
+pub use sr_api_proc_macro::{decl_runtime_apis, impl_runtime_apis};
 
 #[cfg(feature = "std")]
 /// A type that records all accessed trie nodes and generates a proof out of it.
 pub type ProofRecorder = state_machine::ProofRecorder<
-	<<<::Header as HeaderT>::Hashing as HashT>::Hasher as HasherT>::Out
+	<<::Header as HeaderT>::Hashing as HashT>::Hasher
 >;
 
 /// Something that can be constructed to a runtime api.
@@ -71,6 +82,9 @@ pub trait ConstructRuntimeApi> {
 /// An extension for the `RuntimeApi`.
 #[cfg(feature = "std")]
 pub trait ApiExt {
+	/// Error type used by the interface.
+	type Error: std::fmt::Debug + From;
+
 	/// The given closure will be called with api instance. Inside the closure any api call is
 	/// allowed. After doing the api call, the closure is allowed to map the `Result` to a
 	/// different `Result` type. This can be important, as the internal data structure that keeps
@@ -85,7 +99,7 @@ pub trait ApiExt {
 	fn has_api(
 		&self,
 		at: &BlockId
-	) -> error::Result where Self: Sized {
+	) -> Result where Self: Sized {
 		self.runtime_version_at(at).map(|v| v.has_api::())
 	}
 
@@ -94,19 +108,19 @@ pub trait ApiExt {
 		&self,
 		at: &BlockId,
 		pred: P,
-	) -> error::Result where Self: Sized {
+	) -> Result where Self: Sized {
 		self.runtime_version_at(at).map(|v| v.has_api_with::(pred))
 	}
 
 	/// Returns the runtime version at the given block id.
-	fn runtime_version_at(&self, at: &BlockId) -> error::Result;
+	fn runtime_version_at(&self, at: &BlockId) -> Result;
 
 	/// Start recording all accessed trie nodes for generating proofs.
 	fn record_proof(&mut self);
 
 	/// Extract the recorded proof.
 	/// This stops the proof recording.
-	fn extract_proof(&mut self) -> Option>>;
+	fn extract_proof(&mut self) -> Option;
 }
 
 /// Before calling any runtime api function, the runtime need to be initialized
@@ -121,26 +135,27 @@ pub trait ApiExt {
 pub enum InitializeBlock<'a, Block: BlockT> {
 	/// Skip initializing the runtime for a given block.
 	///
-	/// This is used by functions who do the initialization by themself or don't
-	/// require it.
+	/// This is used by functions who do the initialization by themselves or don't require it.
 	Skip,
 	/// Initialize the runtime for a given block.
 	///
-	/// If the stored `BlockId` is `Some(_)`, the runtime is currently initialized
-	/// at this block.
+	/// If the stored `BlockId` is `Some(_)`, the runtime is currently initialized at this block.
 	Do(&'a RefCell>>),
 }
 
 /// Something that can call into the runtime at a given block.
 #[cfg(feature = "std")]
 pub trait CallRuntimeAt {
-	/// Calls the given api function with the given encoded arguments at the given block
-	/// and returns the encoded result.
+	/// Error type used by the interface.
+	type Error: std::fmt::Debug + From;
+
+	/// Calls the given api function with the given encoded arguments at the given block and returns
+	/// the encoded result.
 	fn call_api_at<
 		'a,
 		R: Encode + Decode + PartialEq,
 		NC: FnOnce() -> result::Result + UnwindSafe,
-		C: Core,
+		C: Core,
 	>(
 		&self,
 		core_api: &C,
@@ -151,15 +166,21 @@ pub trait CallRuntimeAt {
 		initialize_block: InitializeBlock<'a, Block>,
 		native_call: Option,
 		context: ExecutionContext,
-		recorder: &Option>>>,
-	) -> error::Result>;
+		recorder: &Option>,
+	) -> Result, Self::Error>;
 
 	/// Returns the runtime version at the given block.
-	fn runtime_version_at(&self, at: &BlockId) -> error::Result;
+	fn runtime_version_at(&self, at: &BlockId) -> Result;
 }
 
+/// Extracts the `Api::Error` for a type that provides a runtime api.
+#[cfg(feature = "std")]
+pub type ApiErrorFor = <
+	::Api as ApiExt
+>::Error;
+
 decl_runtime_apis! {
-	/// The `Core` api trait that is mandatory for each runtime.
+	/// The `Core` runtime api that every Substrate runtime needs to implement.
 	#[core_trait]
 	#[api_version(2)]
 	pub trait Core {
@@ -180,11 +201,4 @@ decl_runtime_apis! {
 		/// Returns the metadata of a runtime.
 		fn metadata() -> OpaqueMetadata;
 	}
-
-	/// The `TaggedTransactionQueue` api trait for interfering with the new transaction queue.
-	pub trait TaggedTransactionQueue {
-		/// Validate the given transaction.
-		fn validate_transaction(tx: ::Extrinsic) -> TransactionValidity;
-	}
 }
-
diff --git a/primitives/sr-api/test/Cargo.toml b/primitives/sr-api/test/Cargo.toml
new file mode 100644
index 0000000000000..7915266cca107
--- /dev/null
+++ b/primitives/sr-api/test/Cargo.toml
@@ -0,0 +1,21 @@
+[package]
+name = "sr-api-test"
+version = "2.0.0"
+authors = ["Parity Technologies "]
+edition = "2018"
+
+[dependencies]
+sr-api = { path = "../" }
+test-client = { package = "substrate-test-runtime-client", path = "../../../test/utils/runtime/client" }
+sr-version = { path = "../../sr-version" }
+sr-primitives = { path = "../../sr-primitives" }
+consensus_common = { package = "substrate-consensus-common", path = "../../../primitives/consensus/common" }
+codec = { package = "parity-scale-codec", version = "1.0.0" }
+state-machine = { package = "substrate-state-machine", path = "../../../primitives/state-machine" }
+trybuild = "1.0.17"
+rustversion = "1.0.0"
+
+# We only need this to generate the correct code.
+[features]
+default = [ "std" ]
+std = []
diff --git a/core/sr-api-macros/tests/decl_and_impl.rs b/primitives/sr-api/test/tests/decl_and_impl.rs
similarity index 70%
rename from core/sr-api-macros/tests/decl_and_impl.rs
rename to primitives/sr-api/test/tests/decl_and_impl.rs
index a539d838221be..91863e186521f 100644
--- a/core/sr-api-macros/tests/decl_and_impl.rs
+++ b/primitives/sr-api/test/tests/decl_and_impl.rs
@@ -14,11 +14,11 @@
 // You should have received a copy of the GNU General Public License
 // along with Substrate.  If not, see .
 
-use sr_primitives::traits::{GetNodeBlockType, Block as BlockT};
-use sr_primitives::generic::BlockId;
-use client::runtime_api::{self, RuntimeApiInfo};
-use client::{error::Result, decl_runtime_apis, impl_runtime_apis};
-use test_client::runtime::Block;
+use sr_api::{RuntimeApiInfo, decl_runtime_apis, impl_runtime_apis};
+
+use sr_primitives::{traits::{GetNodeBlockType, Block as BlockT}, generic::BlockId};
+
+use test_client::{client::error::Result, runtime::Block};
 
 /// The declaration of the `Runtime` type and the implementation of the `GetNodeBlockType`
 /// trait are done by the `construct_runtime!` macro in a real runtime.
@@ -33,6 +33,7 @@ decl_runtime_apis! {
 		fn something_with_block(block: Block) -> Block;
 		fn function_with_two_args(data: u64, block: Block);
 		fn same_name();
+		fn wild_card(_: u32);
 	}
 
 	#[api_version(2)]
@@ -58,14 +59,16 @@ impl_runtime_apis! {
 		}
 
 		fn same_name() {}
+
+		fn wild_card(_: u32) {}
 	}
 
 	impl self::ApiWithCustomVersion for Runtime {
 		fn same_name() {}
 	}
 
-	impl runtime_api::Core for Runtime {
-		fn version() -> runtime_api::RuntimeVersion {
+	impl sr_api::Core for Runtime {
+		fn version() -> sr_version::RuntimeVersion {
 			unimplemented!()
 		}
 		fn execute_block(_: Block) {
@@ -77,7 +80,9 @@ impl_runtime_apis! {
 	}
 }
 
-type TestClient = client::Client;
+type TestClient = test_client::client::Client<
+	test_client::Backend, test_client::Executor, Block, RuntimeApi
+>;
 
 #[test]
 fn test_client_side_function_signature() {
@@ -95,15 +100,19 @@ fn test_client_side_function_signature() {
 
 #[test]
 fn check_runtime_api_info() {
-	assert_eq!(&Api::::ID, &runtime_decl_for_Api::ID);
-	assert_eq!(Api::::VERSION, runtime_decl_for_Api::VERSION);
-	assert_eq!(Api::::VERSION, 1);
+	assert_eq!(&Api::::ID, &runtime_decl_for_Api::ID);
+	assert_eq!(Api::::VERSION, runtime_decl_for_Api::VERSION);
+	assert_eq!(Api::::VERSION, 1);
 
 	assert_eq!(
-		ApiWithCustomVersion::::VERSION, runtime_decl_for_ApiWithCustomVersion::VERSION
+		ApiWithCustomVersion::::VERSION,
+		runtime_decl_for_ApiWithCustomVersion::VERSION,
+	);
+	assert_eq!(
+		&ApiWithCustomVersion::::ID,
+		&runtime_decl_for_ApiWithCustomVersion::ID,
 	);
-	assert_eq!(&ApiWithCustomVersion::::ID, &runtime_decl_for_ApiWithCustomVersion::ID);
-	assert_eq!(ApiWithCustomVersion::::VERSION, 2);
+	assert_eq!(ApiWithCustomVersion::::VERSION, 2);
 }
 
 fn check_runtime_api_versions_contains() {
@@ -112,7 +121,7 @@ fn check_runtime_api_versions_contains() {
 
 #[test]
 fn check_runtime_api_versions() {
-	check_runtime_api_versions_contains::>();
-	check_runtime_api_versions_contains::>();
-	check_runtime_api_versions_contains::>();
+	check_runtime_api_versions_contains::>();
+	check_runtime_api_versions_contains::>();
+	check_runtime_api_versions_contains::>();
 }
diff --git a/core/sr-api-macros/tests/runtime_calls.rs b/primitives/sr-api/test/tests/runtime_calls.rs
similarity index 100%
rename from core/sr-api-macros/tests/runtime_calls.rs
rename to primitives/sr-api/test/tests/runtime_calls.rs
diff --git a/primitives/sr-api/test/tests/trybuild.rs b/primitives/sr-api/test/tests/trybuild.rs
new file mode 100644
index 0000000000000..5b14ee81e8e8a
--- /dev/null
+++ b/primitives/sr-api/test/tests/trybuild.rs
@@ -0,0 +1,27 @@
+// Copyright 2019 Parity Technologies (UK) Ltd.
+// This file is part of Substrate.
+
+// Substrate is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+
+// Substrate is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+
+// You should have received a copy of the GNU General Public License
+// along with Substrate.  If not, see .
+
+use std::env;
+
+#[rustversion::attr(not(stable), ignore)]
+#[test]
+fn ui() {
+	// As trybuild is using `cargo check`, we don't need the real WASM binaries.
+	env::set_var("BUILD_DUMMY_WASM_BINARY", "1");
+
+	let t = trybuild::TestCases::new();
+	t.compile_fail("tests/ui/*.rs");
+}
diff --git a/core/sr-api-macros/tests/ui/adding_self_parameter.rs b/primitives/sr-api/test/tests/ui/adding_self_parameter.rs
similarity index 50%
rename from core/sr-api-macros/tests/ui/adding_self_parameter.rs
rename to primitives/sr-api/test/tests/ui/adding_self_parameter.rs
index fb048211adac3..9195598b5a437 100644
--- a/core/sr-api-macros/tests/ui/adding_self_parameter.rs
+++ b/primitives/sr-api/test/tests/ui/adding_self_parameter.rs
@@ -1,6 +1,4 @@
-use client::decl_runtime_apis;
-
-decl_runtime_apis! {
+sr_api::decl_runtime_apis! {
 	pub trait Api {
 		fn test(&self);
 	}
diff --git a/primitives/sr-api/test/tests/ui/adding_self_parameter.stderr b/primitives/sr-api/test/tests/ui/adding_self_parameter.stderr
new file mode 100644
index 0000000000000..894713d35eef8
--- /dev/null
+++ b/primitives/sr-api/test/tests/ui/adding_self_parameter.stderr
@@ -0,0 +1,5 @@
+error: `self` as argument not supported.
+ --> $DIR/adding_self_parameter.rs:3:11
+  |
+3 |         fn test(&self);
+  |                 ^
diff --git a/core/sr-api-macros/tests/ui/changed_in_unknown_version.rs b/primitives/sr-api/test/tests/ui/changed_in_unknown_version.rs
similarity index 89%
rename from core/sr-api-macros/tests/ui/changed_in_unknown_version.rs
rename to primitives/sr-api/test/tests/ui/changed_in_unknown_version.rs
index 127236200517f..1fcb5d4be1e5a 100644
--- a/core/sr-api-macros/tests/ui/changed_in_unknown_version.rs
+++ b/primitives/sr-api/test/tests/ui/changed_in_unknown_version.rs
@@ -1,6 +1,5 @@
 use sr_primitives::traits::GetNodeBlockType;
 use test_client::runtime::Block;
-use client::decl_runtime_apis;
 
 /// The declaration of the `Runtime` type and the implementation of the `GetNodeBlockType`
 /// trait are done by the `construct_runtime!` macro in a real runtime.
@@ -9,7 +8,7 @@ impl GetNodeBlockType for Runtime {
 	type NodeBlock = Block;
 }
 
-decl_runtime_apis! {
+sr_api::decl_runtime_apis! {
 	pub trait Api {
 		#[changed_in(2)]
 		fn test(data: u64);
diff --git a/core/sr-api-macros/tests/ui/changed_in_unknown_version.stderr b/primitives/sr-api/test/tests/ui/changed_in_unknown_version.stderr
similarity index 53%
rename from core/sr-api-macros/tests/ui/changed_in_unknown_version.stderr
rename to primitives/sr-api/test/tests/ui/changed_in_unknown_version.stderr
index c62befbab362a..cf03ee4530ab7 100644
--- a/core/sr-api-macros/tests/ui/changed_in_unknown_version.stderr
+++ b/primitives/sr-api/test/tests/ui/changed_in_unknown_version.stderr
@@ -1,5 +1,5 @@
 error: `changed_in` version can not be greater than the `api_version`
-  --> $DIR/changed_in_unknown_version.rs:15:3
+  --> $DIR/changed_in_unknown_version.rs:14:3
    |
-15 |         fn test(data: u64);
+14 |         fn test(data: u64);
    |         ^^
diff --git a/core/sr-api-macros/tests/ui/declaring_old_block.rs b/primitives/sr-api/test/tests/ui/declaring_old_block.rs
similarity index 67%
rename from core/sr-api-macros/tests/ui/declaring_old_block.rs
rename to primitives/sr-api/test/tests/ui/declaring_old_block.rs
index 78d35579fae0c..962aae4506642 100644
--- a/core/sr-api-macros/tests/ui/declaring_old_block.rs
+++ b/primitives/sr-api/test/tests/ui/declaring_old_block.rs
@@ -1,7 +1,6 @@
 use sr_primitives::traits::Block as BlockT;
-use client::decl_runtime_apis;
 
-decl_runtime_apis! {
+sr_api::decl_runtime_apis! {
 	pub trait Api {
 		fn test();
 	}
diff --git a/core/sr-api-macros/tests/ui/declaring_old_block.stderr b/primitives/sr-api/test/tests/ui/declaring_old_block.stderr
similarity index 80%
rename from core/sr-api-macros/tests/ui/declaring_old_block.stderr
rename to primitives/sr-api/test/tests/ui/declaring_old_block.stderr
index 999a50cc96977..e27294692b3e0 100644
--- a/core/sr-api-macros/tests/ui/declaring_old_block.stderr
+++ b/primitives/sr-api/test/tests/ui/declaring_old_block.stderr
@@ -1,13 +1,13 @@
 error: `Block: BlockT` generic parameter will be added automatically by the `decl_runtime_apis!` macro!
- --> $DIR/declaring_old_block.rs:5:16
+ --> $DIR/declaring_old_block.rs:4:16
   |
-5 |     pub trait Api {
+4 |     pub trait Api {
   |                   ^^^^^
 
 error: `Block: BlockT` generic parameter will be added automatically by the `decl_runtime_apis!` macro! If you try to use a different trait than the substrate `Block` trait, please rename it locally.
- --> $DIR/declaring_old_block.rs:5:23
+ --> $DIR/declaring_old_block.rs:4:23
   |
-5 |     pub trait Api {
+4 |     pub trait Api {
   |                          ^^^^^^
 
 warning: unused import: `sr_primitives::traits::Block as BlockT`
diff --git a/core/sr-api-macros/tests/ui/declaring_own_block_with_different_name.rs b/primitives/sr-api/test/tests/ui/declaring_own_block_with_different_name.rs
similarity index 66%
rename from core/sr-api-macros/tests/ui/declaring_own_block_with_different_name.rs
rename to primitives/sr-api/test/tests/ui/declaring_own_block_with_different_name.rs
index d63eadc1e4b73..9a47148256493 100644
--- a/core/sr-api-macros/tests/ui/declaring_own_block_with_different_name.rs
+++ b/primitives/sr-api/test/tests/ui/declaring_own_block_with_different_name.rs
@@ -1,7 +1,6 @@
 use sr_primitives::traits::Block as BlockT;
-use client::decl_runtime_apis;
 
-decl_runtime_apis! {
+sr_api::decl_runtime_apis! {
 	pub trait Api {
 		fn test();
 	}
diff --git a/core/sr-api-macros/tests/ui/declaring_own_block_with_different_name.stderr b/primitives/sr-api/test/tests/ui/declaring_own_block_with_different_name.stderr
similarity index 84%
rename from core/sr-api-macros/tests/ui/declaring_own_block_with_different_name.stderr
rename to primitives/sr-api/test/tests/ui/declaring_own_block_with_different_name.stderr
index ec033f2e09d23..88359f19afc72 100644
--- a/core/sr-api-macros/tests/ui/declaring_own_block_with_different_name.stderr
+++ b/primitives/sr-api/test/tests/ui/declaring_own_block_with_different_name.stderr
@@ -1,7 +1,7 @@
 error: `Block: BlockT` generic parameter will be added automatically by the `decl_runtime_apis!` macro! If you try to use a different trait than the substrate `Block` trait, please rename it locally.
- --> $DIR/declaring_own_block_with_different_name.rs:5:19
+ --> $DIR/declaring_own_block_with_different_name.rs:4:19
   |
-5 |     pub trait Api {
+4 |     pub trait Api {
   |                      ^^^^^^
 
 warning: unused import: `sr_primitives::traits::Block as BlockT`
diff --git a/core/sr-api-macros/tests/ui/empty_impl_runtime_apis_call.rs b/primitives/sr-api/test/tests/ui/empty_impl_runtime_apis_call.rs
similarity index 80%
rename from core/sr-api-macros/tests/ui/empty_impl_runtime_apis_call.rs
rename to primitives/sr-api/test/tests/ui/empty_impl_runtime_apis_call.rs
index c9c334f6fdcb5..fee4e475e39aa 100644
--- a/core/sr-api-macros/tests/ui/empty_impl_runtime_apis_call.rs
+++ b/primitives/sr-api/test/tests/ui/empty_impl_runtime_apis_call.rs
@@ -1,6 +1,5 @@
 use sr_primitives::traits::GetNodeBlockType;
 use test_client::runtime::Block;
-use client::{decl_runtime_apis, impl_runtime_apis};
 
 /// The declaration of the `Runtime` type and the implementation of the `GetNodeBlockType`
 /// trait are done by the `construct_runtime!` macro in a real runtime.
@@ -9,12 +8,12 @@ impl GetNodeBlockType for Runtime {
 	type NodeBlock = Block;
 }
 
-decl_runtime_apis! {
+sr_api::decl_runtime_apis! {
 	pub trait Api {
 		fn test(data: u64);
 	}
 }
 
-impl_runtime_apis! {}
+sr_api::impl_runtime_apis! {}
 
 fn main() {}
diff --git a/primitives/sr-api/test/tests/ui/empty_impl_runtime_apis_call.stderr b/primitives/sr-api/test/tests/ui/empty_impl_runtime_apis_call.stderr
new file mode 100644
index 0000000000000..e7bf3b8563f93
--- /dev/null
+++ b/primitives/sr-api/test/tests/ui/empty_impl_runtime_apis_call.stderr
@@ -0,0 +1,5 @@
+error: No api implementation given!
+  --> $DIR/empty_impl_runtime_apis_call.rs:17:1
+   |
+17 | sr_api::impl_runtime_apis! {}
+   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ in this macro invocation
diff --git a/core/sr-api-macros/tests/ui/impl_incorrect_method_signature.rs b/primitives/sr-api/test/tests/ui/impl_incorrect_method_signature.rs
similarity index 81%
rename from core/sr-api-macros/tests/ui/impl_incorrect_method_signature.rs
rename to primitives/sr-api/test/tests/ui/impl_incorrect_method_signature.rs
index b85431f3ba049..08c3ce8320fb5 100644
--- a/core/sr-api-macros/tests/ui/impl_incorrect_method_signature.rs
+++ b/primitives/sr-api/test/tests/ui/impl_incorrect_method_signature.rs
@@ -1,6 +1,5 @@
 use sr_primitives::traits::{GetNodeBlockType, Block as BlockT};
 use test_client::runtime::Block;
-use client::{decl_runtime_apis, impl_runtime_apis, runtime_api};
 
 /// The declaration of the `Runtime` type and the implementation of the `GetNodeBlockType`
 /// trait are done by the `construct_runtime!` macro in a real runtime.
@@ -9,18 +8,18 @@ impl GetNodeBlockType for Runtime {
 	type NodeBlock = Block;
 }
 
-decl_runtime_apis! {
+sr_api::decl_runtime_apis! {
 	pub trait Api {
 		fn test(data: u64);
 	}
 }
 
-impl_runtime_apis! {
+sr_api::impl_runtime_apis! {
 	impl self::Api for Runtime {
 		fn test(data: String) {}
 	}
 
-	impl runtime_api::Core for Runtime {
+	impl sr_api::Core for Runtime {
 		fn version() -> runtime_api::RuntimeVersion {
 			unimplemented!()
 		}
diff --git a/primitives/sr-api/test/tests/ui/impl_incorrect_method_signature.stderr b/primitives/sr-api/test/tests/ui/impl_incorrect_method_signature.stderr
new file mode 100644
index 0000000000000..2bf8da343fe6b
--- /dev/null
+++ b/primitives/sr-api/test/tests/ui/impl_incorrect_method_signature.stderr
@@ -0,0 +1,70 @@
+error[E0433]: failed to resolve: use of undeclared type or module `runtime_api`
+  --> $DIR/impl_incorrect_method_signature.rs:23:19
+   |
+23 |         fn version() -> runtime_api::RuntimeVersion {
+   |                         ^^^^^^^^^^^ use of undeclared type or module `runtime_api`
+
+error[E0053]: method `test` has an incompatible type for trait
+  --> $DIR/impl_incorrect_method_signature.rs:19:17
+   |
+13 |         fn test(data: u64);
+   |                       --- type in trait
+...
+19 |         fn test(data: String) {}
+   |                       ^^^^^^ expected u64, found struct `std::string::String`
+   |
+   = note: expected type `fn(u64)`
+              found type `fn(std::string::String)`
+
+error[E0053]: method `Api_test_runtime_api_impl` has an incompatible type for trait
+  --> $DIR/impl_incorrect_method_signature.rs:17:1
+   |
+11 | / sr_api::decl_runtime_apis! {
+12 | |     pub trait Api {
+13 | |         fn test(data: u64);
+14 | |     }
+15 | | }
+   | |_- type in trait
+16 | 
+17 |   sr_api::impl_runtime_apis! {
+   |   -^^^^^^^^^^^^^^^^^^^^^^^^^
+   |   |
+   |  _expected u64, found struct `std::string::String`
+   | |
+18 | |     impl self::Api for Runtime {
+19 | |         fn test(data: String) {}
+20 | |     }
+...  |
+32 | |     }
+33 | | }
+   | |_- in this macro invocation
+   |
+   = note: expected type `fn(&RuntimeApiImpl, &sr_api_hidden_includes_DECL_RUNTIME_APIS::sr_api::BlockId, substrate_test_runtime::Extrinsic>>, sr_api_hidden_includes_DECL_RUNTIME_APIS::sr_api::ExecutionContext, std::option::Option, std::vec::Vec) -> std::result::Result, , substrate_test_runtime::Extrinsic>>>::Error>`
+              found type `fn(&RuntimeApiImpl, &sr_api_hidden_includes_DECL_RUNTIME_APIS::sr_api::BlockId, substrate_test_runtime::Extrinsic>>, sr_api_hidden_includes_DECL_RUNTIME_APIS::sr_api::ExecutionContext, std::option::Option, std::vec::Vec) -> std::result::Result, , substrate_test_runtime::Extrinsic>>>::Error>`
+
+error[E0308]: mismatched types
+  --> $DIR/impl_incorrect_method_signature.rs:17:1
+   |
+17 | / sr_api::impl_runtime_apis! {
+18 | |     impl self::Api for Runtime {
+19 | |         fn test(data: String) {}
+20 | |     }
+...  |
+32 | |     }
+33 | | }
+   | | ^
+   | | |
+   | |_expected u64, found struct `std::string::String`
+   |   in this macro invocation
+   |
+   = note: expected type `u64`
+              found type `std::string::String`
+
+error[E0308]: mismatched types
+  --> $DIR/impl_incorrect_method_signature.rs:19:11
+   |
+19 |         fn test(data: String) {}
+   |                 ^^^^ expected u64, found struct `std::string::String`
+   |
+   = note: expected type `u64`
+              found type `std::string::String`
diff --git a/core/sr-api-macros/tests/ui/impl_two_traits_with_same_name.rs b/primitives/sr-api/test/tests/ui/impl_two_traits_with_same_name.rs
similarity index 86%
rename from core/sr-api-macros/tests/ui/impl_two_traits_with_same_name.rs
rename to primitives/sr-api/test/tests/ui/impl_two_traits_with_same_name.rs
index 1664bec577b57..6aee0ec6c2bb4 100644
--- a/core/sr-api-macros/tests/ui/impl_two_traits_with_same_name.rs
+++ b/primitives/sr-api/test/tests/ui/impl_two_traits_with_same_name.rs
@@ -1,6 +1,5 @@
 use sr_primitives::traits::GetNodeBlockType;
 use test_client::runtime::Block;
-use client::{decl_runtime_apis, impl_runtime_apis};
 
 /// The declaration of the `Runtime` type and the implementation of the `GetNodeBlockType`
 /// trait are done by the `construct_runtime!` macro in a real runtime.
@@ -9,7 +8,7 @@ impl GetNodeBlockType for Runtime {
 	type NodeBlock = Block;
 }
 
-decl_runtime_apis! {
+sr_api::decl_runtime_apis! {
 	pub trait Api {
 		fn test(data: u64);
 	}
@@ -25,7 +24,7 @@ mod second {
 	}
 }
 
-impl_runtime_apis! {
+sr_api::impl_runtime_apis! {
 	impl self::Api for Runtime {
 		fn test(data: u64) {}
 	}
diff --git a/primitives/sr-api/test/tests/ui/impl_two_traits_with_same_name.stderr b/primitives/sr-api/test/tests/ui/impl_two_traits_with_same_name.stderr
new file mode 100644
index 0000000000000..9aa38805b940f
--- /dev/null
+++ b/primitives/sr-api/test/tests/ui/impl_two_traits_with_same_name.stderr
@@ -0,0 +1,77 @@
+error: Two traits with the same name detected! The trait name is used to generate its ID. Please rename one trait at the declaration!
+  --> $DIR/impl_two_traits_with_same_name.rs:32:15
+   |
+32 |     impl second::Api for Runtime {
+   |                  ^^^
+
+error: cannot find macro `decl_runtime_apis` in this scope
+  --> $DIR/impl_two_traits_with_same_name.rs:20:2
+   |
+20 |     decl_runtime_apis! {
+   |     ^^^^^^^^^^^^^^^^^
+
+error[E0425]: cannot find function `test2_call_api_at` in `second::runtime_decl_for_Api`
+  --> $DIR/impl_two_traits_with_same_name.rs:27:1
+   |
+27 | / sr_api::impl_runtime_apis! {
+28 | |     impl self::Api for Runtime {
+29 | |         fn test(data: u64) {}
+30 | |     }
+...  |
+34 | |     }
+35 | | }
+   | | ^
+   | | |
+   | |_not found in `second::runtime_decl_for_Api`
+   |   in this macro invocation
+
+error[E0425]: cannot find function `test2_native_call_generator` in `second::runtime_decl_for_Api`
+  --> $DIR/impl_two_traits_with_same_name.rs:27:1
+   |
+27 | / sr_api::impl_runtime_apis! {
+28 | |     impl self::Api for Runtime {
+29 | |         fn test(data: u64) {}
+30 | |     }
+...  |
+34 | |     }
+35 | | }
+   | | ^
+   | | |
+   | |_not found in `second::runtime_decl_for_Api`
+   |   in this macro invocation
+
+error[E0576]: cannot find method or associated constant `test2` in `second::runtime_decl_for_Api::Api`
+  --> $DIR/impl_two_traits_with_same_name.rs:33:6
+   |
+33 |         fn test2(data: u64) {}
+   |            ^^^^^ not found in `second::runtime_decl_for_Api::Api`
+
+error[E0603]: module `runtime_decl_for_Api` is private
+  --> $DIR/impl_two_traits_with_same_name.rs:27:1
+   |
+27 | / sr_api::impl_runtime_apis! {
+28 | |     impl self::Api for Runtime {
+29 | |         fn test(data: u64) {}
+30 | |     }
+...  |
+34 | |     }
+35 | | }
+   | |_^
+
+error[E0119]: conflicting implementations of trait `runtime_decl_for_Api::Api, substrate_test_runtime::Extrinsic>>` for type `Runtime`:
+  --> $DIR/impl_two_traits_with_same_name.rs:32:2
+   |
+28 |     impl self::Api for Runtime {
+   |     --------------------------------- first implementation here
+...
+32 |     impl second::Api for Runtime {
+   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ conflicting implementation for `Runtime`
+
+error[E0119]: conflicting implementations of trait `Api, substrate_test_runtime::Extrinsic>>` for type `RuntimeApiImpl<_>`:
+  --> $DIR/impl_two_traits_with_same_name.rs:32:2
+   |
+28 |     impl self::Api for Runtime {
+   |     --------------------------------- first implementation here
+...
+32 |     impl second::Api for Runtime {
+   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ conflicting implementation for `RuntimeApiImpl<_>`
diff --git a/core/sr-api-macros/tests/ui/invalid_api_version.rs b/primitives/sr-api/test/tests/ui/invalid_api_version.rs
similarity index 58%
rename from core/sr-api-macros/tests/ui/invalid_api_version.rs
rename to primitives/sr-api/test/tests/ui/invalid_api_version.rs
index b5afa1d6998ef..0b7f5e88ff74b 100644
--- a/core/sr-api-macros/tests/ui/invalid_api_version.rs
+++ b/primitives/sr-api/test/tests/ui/invalid_api_version.rs
@@ -1,6 +1,4 @@
-use client::decl_runtime_apis;
-
-decl_runtime_apis! {
+sr_api::decl_runtime_apis! {
 	#[api_version]
 	pub trait Api {
 		fn test(data: u64);
diff --git a/primitives/sr-api/test/tests/ui/invalid_api_version.stderr b/primitives/sr-api/test/tests/ui/invalid_api_version.stderr
new file mode 100644
index 0000000000000..7e63eb8ebf8a4
--- /dev/null
+++ b/primitives/sr-api/test/tests/ui/invalid_api_version.stderr
@@ -0,0 +1,33 @@
+error: can't qualify macro invocation with `pub`
+ --> $DIR/invalid_api_version.rs:1:1
+  |
+1 | / sr_api::decl_runtime_apis! {
+2 | |     #[api_version]
+3 | |     pub trait Api {
+4 | |         fn test(data: u64);
+5 | |     }
+6 | | }
+  | | ^ in this macro invocation
+  | |_|
+  | 
+  |
+  = help: try adjusting the macro to put `pub` inside the invocation
+
+error: Unexpected `api_version` attribute. The supported format is `api_version(1)`
+ --> $DIR/invalid_api_version.rs:1:1
+  |
+1 | / sr_api::decl_runtime_apis! {
+2 | |     #[api_version]
+3 | |     pub trait Api {
+4 | |         fn test(data: u64);
+5 | |     }
+6 | | }
+  | | ^ in this macro invocation
+  | |_|
+  | 
+
+error: Unexpected `api_version` attribute. The supported format is `api_version(1)`
+ --> $DIR/invalid_api_version.rs:2:4
+  |
+2 |     #[api_version]
+  |       ^^^^^^^^^^^
diff --git a/core/sr-api-macros/tests/ui/invalid_api_version_2.rs b/primitives/sr-api/test/tests/ui/invalid_api_version_2.rs
similarity index 59%
rename from core/sr-api-macros/tests/ui/invalid_api_version_2.rs
rename to primitives/sr-api/test/tests/ui/invalid_api_version_2.rs
index b8870838009bf..4e29d36e1ba2f 100644
--- a/core/sr-api-macros/tests/ui/invalid_api_version_2.rs
+++ b/primitives/sr-api/test/tests/ui/invalid_api_version_2.rs
@@ -1,6 +1,4 @@
-use client::decl_runtime_apis;
-
-decl_runtime_apis! {
+sr_api::decl_runtime_apis! {
 	#[api_version("1")]
 	pub trait Api {
 		fn test(data: u64);
diff --git a/primitives/sr-api/test/tests/ui/invalid_api_version_2.stderr b/primitives/sr-api/test/tests/ui/invalid_api_version_2.stderr
new file mode 100644
index 0000000000000..e080b2dd1a0d6
--- /dev/null
+++ b/primitives/sr-api/test/tests/ui/invalid_api_version_2.stderr
@@ -0,0 +1,33 @@
+error: can't qualify macro invocation with `pub`
+ --> $DIR/invalid_api_version_2.rs:1:1
+  |
+1 | / sr_api::decl_runtime_apis! {
+2 | |     #[api_version("1")]
+3 | |     pub trait Api {
+4 | |         fn test(data: u64);
+5 | |     }
+6 | | }
+  | | ^ in this macro invocation
+  | |_|
+  | 
+  |
+  = help: try adjusting the macro to put `pub` inside the invocation
+
+error: Unexpected `api_version` attribute. The supported format is `api_version(1)`
+ --> $DIR/invalid_api_version_2.rs:1:1
+  |
+1 | / sr_api::decl_runtime_apis! {
+2 | |     #[api_version("1")]
+3 | |     pub trait Api {
+4 | |         fn test(data: u64);
+5 | |     }
+6 | | }
+  | | ^ in this macro invocation
+  | |_|
+  | 
+
+error: Unexpected `api_version` attribute. The supported format is `api_version(1)`
+ --> $DIR/invalid_api_version_2.rs:2:4
+  |
+2 |     #[api_version("1")]
+  |       ^^^^^^^^^^^
diff --git a/core/sr-api-macros/tests/ui/invalid_api_version_3.rs b/primitives/sr-api/test/tests/ui/invalid_api_version_3.rs
similarity index 58%
rename from core/sr-api-macros/tests/ui/invalid_api_version_3.rs
rename to primitives/sr-api/test/tests/ui/invalid_api_version_3.rs
index 6f365b146b222..bafe566840d25 100644
--- a/core/sr-api-macros/tests/ui/invalid_api_version_3.rs
+++ b/primitives/sr-api/test/tests/ui/invalid_api_version_3.rs
@@ -1,6 +1,4 @@
-use client::decl_runtime_apis;
-
-decl_runtime_apis! {
+sr_api::decl_runtime_apis! {
 	#[api_version()]
 	pub trait Api {
 		fn test(data: u64);
diff --git a/primitives/sr-api/test/tests/ui/invalid_api_version_3.stderr b/primitives/sr-api/test/tests/ui/invalid_api_version_3.stderr
new file mode 100644
index 0000000000000..fd6e15852f74b
--- /dev/null
+++ b/primitives/sr-api/test/tests/ui/invalid_api_version_3.stderr
@@ -0,0 +1,33 @@
+error: can't qualify macro invocation with `pub`
+ --> $DIR/invalid_api_version_3.rs:1:1
+  |
+1 | / sr_api::decl_runtime_apis! {
+2 | |     #[api_version()]
+3 | |     pub trait Api {
+4 | |         fn test(data: u64);
+5 | |     }
+6 | | }
+  | | ^ in this macro invocation
+  | |_|
+  | 
+  |
+  = help: try adjusting the macro to put `pub` inside the invocation
+
+error: Unexpected `api_version` attribute. The supported format is `api_version(1)`
+ --> $DIR/invalid_api_version_3.rs:1:1
+  |
+1 | / sr_api::decl_runtime_apis! {
+2 | |     #[api_version()]
+3 | |     pub trait Api {
+4 | |         fn test(data: u64);
+5 | |     }
+6 | | }
+  | | ^ in this macro invocation
+  | |_|
+  | 
+
+error: Unexpected `api_version` attribute. The supported format is `api_version(1)`
+ --> $DIR/invalid_api_version_3.rs:2:4
+  |
+2 |     #[api_version()]
+  |       ^^^^^^^^^^^
diff --git a/core/sr-api-macros/tests/ui/missing_block_generic_parameter.rs b/primitives/sr-api/test/tests/ui/missing_block_generic_parameter.rs
similarity index 83%
rename from core/sr-api-macros/tests/ui/missing_block_generic_parameter.rs
rename to primitives/sr-api/test/tests/ui/missing_block_generic_parameter.rs
index 99755144f7556..d35253a7219a4 100644
--- a/core/sr-api-macros/tests/ui/missing_block_generic_parameter.rs
+++ b/primitives/sr-api/test/tests/ui/missing_block_generic_parameter.rs
@@ -1,6 +1,5 @@
 use sr_primitives::traits::GetNodeBlockType;
 use test_client::runtime::Block;
-use client::{decl_runtime_apis, impl_runtime_apis};
 
 /// The declaration of the `Runtime` type and the implementation of the `GetNodeBlockType`
 /// trait are done by the `construct_runtime!` macro in a real runtime.
@@ -9,13 +8,13 @@ impl GetNodeBlockType for Runtime {
 	type NodeBlock = Block;
 }
 
-decl_runtime_apis! {
+sr_api::decl_runtime_apis! {
 	pub trait Api {
 		fn test(data: u64);
 	}
 }
 
-impl_runtime_apis! {
+sr_api::impl_runtime_apis! {
 	impl self::Api for Runtime {
 		fn test(data: u64) {
 			unimplemented!()
diff --git a/primitives/sr-api/test/tests/ui/missing_block_generic_parameter.stderr b/primitives/sr-api/test/tests/ui/missing_block_generic_parameter.stderr
new file mode 100644
index 0000000000000..61311c14b882d
--- /dev/null
+++ b/primitives/sr-api/test/tests/ui/missing_block_generic_parameter.stderr
@@ -0,0 +1,11 @@
+error: Missing `Block` generic parameter.
+  --> $DIR/missing_block_generic_parameter.rs:18:13
+   |
+18 |     impl self::Api for Runtime {
+   |                ^^^
+
+error[E0107]: wrong number of type arguments: expected 1, found 0
+  --> $DIR/missing_block_generic_parameter.rs:18:7
+   |
+18 |     impl self::Api for Runtime {
+   |          ^^^^^^^^^ expected 1 type argument
diff --git a/core/sr-api-macros/tests/ui/missing_path_for_trait.rs b/primitives/sr-api/test/tests/ui/missing_path_for_trait.rs
similarity index 83%
rename from core/sr-api-macros/tests/ui/missing_path_for_trait.rs
rename to primitives/sr-api/test/tests/ui/missing_path_for_trait.rs
index f6f6e3dfb3c43..fb78374ebdf70 100644
--- a/core/sr-api-macros/tests/ui/missing_path_for_trait.rs
+++ b/primitives/sr-api/test/tests/ui/missing_path_for_trait.rs
@@ -1,6 +1,5 @@
 use sr_primitives::traits::GetNodeBlockType;
 use test_client::runtime::Block;
-use client::{decl_runtime_apis, impl_runtime_apis};
 
 /// The declaration of the `Runtime` type and the implementation of the `GetNodeBlockType`
 /// trait are done by the `construct_runtime!` macro in a real runtime.
@@ -9,13 +8,13 @@ impl GetNodeBlockType for Runtime {
 	type NodeBlock = Block;
 }
 
-decl_runtime_apis! {
+sr_api::decl_runtime_apis! {
 	pub trait Api {
 		fn test(data: u64);
 	}
 }
 
-impl_runtime_apis! {
+sr_api::impl_runtime_apis! {
 	impl Api for Runtime {
 		fn test(data: u64) {
 			unimplemented!()
diff --git a/core/sr-api-macros/tests/ui/missing_path_for_trait.stderr b/primitives/sr-api/test/tests/ui/missing_path_for_trait.stderr
similarity index 60%
rename from core/sr-api-macros/tests/ui/missing_path_for_trait.stderr
rename to primitives/sr-api/test/tests/ui/missing_path_for_trait.stderr
index 4018712e3f5b3..729ff0bad18d8 100644
--- a/core/sr-api-macros/tests/ui/missing_path_for_trait.stderr
+++ b/primitives/sr-api/test/tests/ui/missing_path_for_trait.stderr
@@ -1,5 +1,5 @@
 error: The implemented trait has to be referenced with a path, e.g. `impl client::Core for Runtime`.
-  --> $DIR/missing_path_for_trait.rs:19:7
+  --> $DIR/missing_path_for_trait.rs:18:7
    |
-19 |     impl Api for Runtime {
+18 |     impl Api for Runtime {
    |          ^^^
diff --git a/core/sr-api-macros/tests/ui/type_reference_in_impl_runtime_apis_call.rs b/primitives/sr-api/test/tests/ui/type_reference_in_impl_runtime_apis_call.rs
similarity index 82%
rename from core/sr-api-macros/tests/ui/type_reference_in_impl_runtime_apis_call.rs
rename to primitives/sr-api/test/tests/ui/type_reference_in_impl_runtime_apis_call.rs
index 20f114c6bb29e..41bbd8a9eee53 100644
--- a/core/sr-api-macros/tests/ui/type_reference_in_impl_runtime_apis_call.rs
+++ b/primitives/sr-api/test/tests/ui/type_reference_in_impl_runtime_apis_call.rs
@@ -1,6 +1,5 @@
 use sr_primitives::traits::{GetNodeBlockType, Block as BlockT};
 use test_client::runtime::Block;
-use client::{decl_runtime_apis, impl_runtime_apis, runtime_api};
 
 /// The declaration of the `Runtime` type and the implementation of the `GetNodeBlockType`
 /// trait are done by the `construct_runtime!` macro in a real runtime.
@@ -9,20 +8,20 @@ impl GetNodeBlockType for Runtime {
 	type NodeBlock = Block;
 }
 
-decl_runtime_apis! {
+sr_api::decl_runtime_apis! {
 	pub trait Api {
 		fn test(data: u64);
 	}
 }
 
-impl_runtime_apis! {
+sr_api::impl_runtime_apis! {
 	impl self::Api for Runtime {
 		fn test(data: &u64) {
 			unimplemented!()
 		}
 	}
 
-	impl runtime_api::Core for Runtime {
+	impl sr_api::Core for Runtime {
 		fn version() -> runtime_api::RuntimeVersion {
 			unimplemented!()
 		}
diff --git a/primitives/sr-api/test/tests/ui/type_reference_in_impl_runtime_apis_call.stderr b/primitives/sr-api/test/tests/ui/type_reference_in_impl_runtime_apis_call.stderr
new file mode 100644
index 0000000000000..4614fe89b8cea
--- /dev/null
+++ b/primitives/sr-api/test/tests/ui/type_reference_in_impl_runtime_apis_call.stderr
@@ -0,0 +1,73 @@
+error[E0433]: failed to resolve: use of undeclared type or module `runtime_api`
+  --> $DIR/type_reference_in_impl_runtime_apis_call.rs:25:19
+   |
+25 |         fn version() -> runtime_api::RuntimeVersion {
+   |                         ^^^^^^^^^^^ use of undeclared type or module `runtime_api`
+
+error[E0053]: method `test` has an incompatible type for trait
+  --> $DIR/type_reference_in_impl_runtime_apis_call.rs:19:17
+   |
+13 |         fn test(data: u64);
+   |                       --- type in trait
+...
+19 |         fn test(data: &u64) {
+   |                       ^^^^ expected u64, found &u64
+   |
+   = note: expected type `fn(u64)`
+              found type `fn(&u64)`
+
+error[E0053]: method `Api_test_runtime_api_impl` has an incompatible type for trait
+  --> $DIR/type_reference_in_impl_runtime_apis_call.rs:17:1
+   |
+11 | / sr_api::decl_runtime_apis! {
+12 | |     pub trait Api {
+13 | |         fn test(data: u64);
+14 | |     }
+15 | | }
+   | |_- type in trait
+16 | 
+17 |   sr_api::impl_runtime_apis! {
+   |   -^^^^^^^^^^^^^^^^^^^^^^^^^
+   |   |
+   |  _expected u64, found &u64
+   | |
+18 | |     impl self::Api for Runtime {
+19 | |         fn test(data: &u64) {
+20 | |             unimplemented!()
+...  |
+34 | |     }
+35 | | }
+   | |_- in this macro invocation
+   |
+   = note: expected type `fn(&RuntimeApiImpl, &sr_api_hidden_includes_DECL_RUNTIME_APIS::sr_api::BlockId, substrate_test_runtime::Extrinsic>>, sr_api_hidden_includes_DECL_RUNTIME_APIS::sr_api::ExecutionContext, std::option::Option, std::vec::Vec) -> std::result::Result, , substrate_test_runtime::Extrinsic>>>::Error>`
+              found type `fn(&RuntimeApiImpl, &sr_api_hidden_includes_DECL_RUNTIME_APIS::sr_api::BlockId, substrate_test_runtime::Extrinsic>>, sr_api_hidden_includes_DECL_RUNTIME_APIS::sr_api::ExecutionContext, std::option::Option<&u64>, std::vec::Vec) -> std::result::Result, , substrate_test_runtime::Extrinsic>>>::Error>`
+
+error[E0308]: mismatched types
+  --> $DIR/type_reference_in_impl_runtime_apis_call.rs:17:1
+   |
+17 | / sr_api::impl_runtime_apis! {
+18 | |     impl self::Api for Runtime {
+19 | |         fn test(data: &u64) {
+20 | |             unimplemented!()
+...  |
+34 | |     }
+35 | | }
+   | | ^
+   | | |
+   | |_expected u64, found &u64
+   |   in this macro invocation
+   |
+   = note: expected type `u64`
+              found type `&u64`
+
+error[E0308]: mismatched types
+  --> $DIR/type_reference_in_impl_runtime_apis_call.rs:19:11
+   |
+19 |         fn test(data: &u64) {
+   |                 ^^^^^^^
+   |                 |
+   |                 expected u64, found &u64
+   |                 help: consider removing the borrow: `data`
+   |
+   = note: expected type `u64`
+              found type `&u64`
diff --git a/core/sr-arithmetic/Cargo.toml b/primitives/sr-arithmetic/Cargo.toml
similarity index 88%
rename from core/sr-arithmetic/Cargo.toml
rename to primitives/sr-arithmetic/Cargo.toml
index fd484671dd848..47b3315e86193 100644
--- a/core/sr-arithmetic/Cargo.toml
+++ b/primitives/sr-arithmetic/Cargo.toml
@@ -10,7 +10,7 @@ integer-sqrt = "0.1.2"
 num-traits = { version = "0.2.8", default-features = false }
 rstd = { package = "sr-std", path = "../sr-std", default-features = false }
 serde = { version = "1.0.101", optional = true, features = ["derive"] }
-substrate-debug-derive = { path = "../primitives/debug-derive", default-features = false }
+substrate-debug-derive = { path = "../../primitives/core/debug-derive", default-features = false }
 
 [dev-dependencies]
 primitive-types = "0.6.0"
diff --git a/core/sr-arithmetic/benches/bench.rs b/primitives/sr-arithmetic/benches/bench.rs
similarity index 100%
rename from core/sr-arithmetic/benches/bench.rs
rename to primitives/sr-arithmetic/benches/bench.rs
diff --git a/core/sr-arithmetic/fuzzer/.gitignore b/primitives/sr-arithmetic/fuzzer/.gitignore
similarity index 100%
rename from core/sr-arithmetic/fuzzer/.gitignore
rename to primitives/sr-arithmetic/fuzzer/.gitignore
diff --git a/core/sr-arithmetic/fuzzer/Cargo.lock b/primitives/sr-arithmetic/fuzzer/Cargo.lock
similarity index 81%
rename from core/sr-arithmetic/fuzzer/Cargo.lock
rename to primitives/sr-arithmetic/fuzzer/Cargo.lock
index f6ce6f1d4797d..83fcd3db3d1c4 100644
--- a/core/sr-arithmetic/fuzzer/Cargo.lock
+++ b/primitives/sr-arithmetic/fuzzer/Cargo.lock
@@ -25,7 +25,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
 
 [[package]]
 name = "byte-slice-cast"
-version = "0.3.2"
+version = "0.3.4"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 
 [[package]]
@@ -53,18 +53,18 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
 
 [[package]]
 name = "fixed-hash"
-version = "0.5.0"
+version = "0.5.1"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 dependencies = [
  "byteorder 1.3.2 (registry+https://github.com/rust-lang/crates.io-index)",
  "rand 0.7.2 (registry+https://github.com/rust-lang/crates.io-index)",
  "rustc-hex 2.0.1 (registry+https://github.com/rust-lang/crates.io-index)",
- "static_assertions 0.2.5 (registry+https://github.com/rust-lang/crates.io-index)",
+ "static_assertions 1.1.0 (registry+https://github.com/rust-lang/crates.io-index)",
 ]
 
 [[package]]
 name = "getrandom"
-version = "0.1.12"
+version = "0.1.13"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 dependencies = [
  "cfg-if 0.1.10 (registry+https://github.com/rust-lang/crates.io-index)",
@@ -84,7 +84,7 @@ dependencies = [
 
 [[package]]
 name = "impl-codec"
-version = "0.4.1"
+version = "0.4.2"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 dependencies = [
  "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)",
@@ -153,9 +153,9 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
 dependencies = [
  "arrayvec 0.4.12 (registry+https://github.com/rust-lang/crates.io-index)",
  "bitvec 0.14.0 (registry+https://github.com/rust-lang/crates.io-index)",
- "byte-slice-cast 0.3.2 (registry+https://github.com/rust-lang/crates.io-index)",
+ "byte-slice-cast 0.3.4 (registry+https://github.com/rust-lang/crates.io-index)",
  "parity-scale-codec-derive 1.0.3 (registry+https://github.com/rust-lang/crates.io-index)",
- "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)",
+ "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)",
 ]
 
 [[package]]
@@ -176,12 +176,12 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
 
 [[package]]
 name = "primitive-types"
-version = "0.6.0"
+version = "0.6.1"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 dependencies = [
- "fixed-hash 0.5.0 (registry+https://github.com/rust-lang/crates.io-index)",
- "impl-codec 0.4.1 (registry+https://github.com/rust-lang/crates.io-index)",
- "uint 0.8.1 (registry+https://github.com/rust-lang/crates.io-index)",
+ "fixed-hash 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)",
+ "impl-codec 0.4.2 (registry+https://github.com/rust-lang/crates.io-index)",
+ "uint 0.8.2 (registry+https://github.com/rust-lang/crates.io-index)",
 ]
 
 [[package]]
@@ -189,7 +189,7 @@ name = "proc-macro-crate"
 version = "0.1.4"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 dependencies = [
- "toml 0.5.3 (registry+https://github.com/rust-lang/crates.io-index)",
+ "toml 0.5.5 (registry+https://github.com/rust-lang/crates.io-index)",
 ]
 
 [[package]]
@@ -229,7 +229,7 @@ name = "rand"
 version = "0.7.2"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 dependencies = [
- "getrandom 0.1.12 (registry+https://github.com/rust-lang/crates.io-index)",
+ "getrandom 0.1.13 (registry+https://github.com/rust-lang/crates.io-index)",
  "libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)",
  "rand_chacha 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)",
  "rand_core 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)",
@@ -250,7 +250,7 @@ name = "rand_core"
 version = "0.5.1"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 dependencies = [
- "getrandom 0.1.12 (registry+https://github.com/rust-lang/crates.io-index)",
+ "getrandom 0.1.13 (registry+https://github.com/rust-lang/crates.io-index)",
 ]
 
 [[package]]
@@ -266,43 +266,22 @@ name = "rustc-hex"
 version = "2.0.1"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 
-[[package]]
-name = "rustc_version"
-version = "0.2.3"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-dependencies = [
- "semver 0.9.0 (registry+https://github.com/rust-lang/crates.io-index)",
-]
-
-[[package]]
-name = "semver"
-version = "0.9.0"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-dependencies = [
- "semver-parser 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)",
-]
-
-[[package]]
-name = "semver-parser"
-version = "0.7.0"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-
 [[package]]
 name = "serde"
-version = "1.0.101"
+version = "1.0.102"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 dependencies = [
- "serde_derive 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)",
+ "serde_derive 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)",
 ]
 
 [[package]]
 name = "serde_derive"
-version = "1.0.101"
+version = "1.0.102"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 dependencies = [
  "proc-macro2 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)",
  "quote 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)",
- "syn 1.0.5 (registry+https://github.com/rust-lang/crates.io-index)",
+ "syn 1.0.8 (registry+https://github.com/rust-lang/crates.io-index)",
 ]
 
 [[package]]
@@ -312,8 +291,9 @@ dependencies = [
  "integer-sqrt 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)",
  "num-traits 0.2.8 (registry+https://github.com/rust-lang/crates.io-index)",
  "parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)",
- "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)",
+ "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)",
  "sr-std 2.0.0",
+ "substrate-debug-derive 2.0.0",
 ]
 
 [[package]]
@@ -323,22 +303,28 @@ dependencies = [
  "honggfuzz 0.5.45 (registry+https://github.com/rust-lang/crates.io-index)",
  "num-bigint 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)",
  "num-traits 0.2.8 (registry+https://github.com/rust-lang/crates.io-index)",
- "primitive-types 0.6.0 (registry+https://github.com/rust-lang/crates.io-index)",
+ "primitive-types 0.6.1 (registry+https://github.com/rust-lang/crates.io-index)",
  "sr-arithmetic 2.0.0",
 ]
 
 [[package]]
 name = "sr-std"
 version = "2.0.0"
-dependencies = [
- "rustc_version 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)",
-]
 
 [[package]]
 name = "static_assertions"
-version = "0.2.5"
+version = "1.1.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 
+[[package]]
+name = "substrate-debug-derive"
+version = "2.0.0"
+dependencies = [
+ "proc-macro2 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)",
+ "quote 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)",
+ "syn 1.0.8 (registry+https://github.com/rust-lang/crates.io-index)",
+]
+
 [[package]]
 name = "syn"
 version = "0.15.44"
@@ -351,7 +337,7 @@ dependencies = [
 
 [[package]]
 name = "syn"
-version = "1.0.5"
+version = "1.0.8"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 dependencies = [
  "proc-macro2 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)",
@@ -361,20 +347,21 @@ dependencies = [
 
 [[package]]
 name = "toml"
-version = "0.5.3"
+version = "0.5.5"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 dependencies = [
- "serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)",
+ "serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)",
 ]
 
 [[package]]
 name = "uint"
-version = "0.8.1"
+version = "0.8.2"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 dependencies = [
  "byteorder 1.3.2 (registry+https://github.com/rust-lang/crates.io-index)",
  "crunchy 0.2.2 (registry+https://github.com/rust-lang/crates.io-index)",
  "rustc-hex 2.0.1 (registry+https://github.com/rust-lang/crates.io-index)",
+ "static_assertions 1.1.0 (registry+https://github.com/rust-lang/crates.io-index)",
 ]
 
 [[package]]
@@ -416,15 +403,15 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
 "checksum arrayvec 0.4.12 (registry+https://github.com/rust-lang/crates.io-index)" = "cd9fd44efafa8690358b7408d253adf110036b88f55672a933f01d616ad9b1b9"
 "checksum autocfg 0.1.7 (registry+https://github.com/rust-lang/crates.io-index)" = "1d49d90015b3c36167a20fe2810c5cd875ad504b39cff3d4eae7977e6b7c1cb2"
 "checksum bitvec 0.14.0 (registry+https://github.com/rust-lang/crates.io-index)" = "9633b74910e1870f50f5af189b08487195cdb83c0e27a71d6f64d5e09dd0538b"
-"checksum byte-slice-cast 0.3.2 (registry+https://github.com/rust-lang/crates.io-index)" = "7cbcbf18128ec71d8d4a0d054461ec59fff5b75b7d10a4c9b7c7cb1a379c3e77"
+"checksum byte-slice-cast 0.3.4 (registry+https://github.com/rust-lang/crates.io-index)" = "f6209f3b2c1edea170002e016d5ead6903d3bb0a846477f53bbeb614967a52a9"
 "checksum byteorder 1.3.2 (registry+https://github.com/rust-lang/crates.io-index)" = "a7c3dd8985a7111efc5c80b44e23ecdd8c007de8ade3b96595387e812b957cf5"
 "checksum c2-chacha 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)" = "214238caa1bf3a496ec3392968969cab8549f96ff30652c9e56885329315f6bb"
 "checksum cfg-if 0.1.10 (registry+https://github.com/rust-lang/crates.io-index)" = "4785bdd1c96b2a846b2bd7cc02e86b6b3dbf14e7e53446c4f54c92a361040822"
 "checksum crunchy 0.2.2 (registry+https://github.com/rust-lang/crates.io-index)" = "7a81dae078cea95a014a339291cec439d2f232ebe854a9d672b796c6afafa9b7"
-"checksum fixed-hash 0.5.0 (registry+https://github.com/rust-lang/crates.io-index)" = "6357b15872f8126e4ea7cf79d579473f132ccd2de239494ad1bf4aa892faea68"
-"checksum getrandom 0.1.12 (registry+https://github.com/rust-lang/crates.io-index)" = "473a1265acc8ff1e808cd0a1af8cee3c2ee5200916058a2ca113c29f2d903571"
+"checksum fixed-hash 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)" = "72fe7539e2c5692c6989f2f9c0457e42f1e5768f96b85c87d273574670ae459f"
+"checksum getrandom 0.1.13 (registry+https://github.com/rust-lang/crates.io-index)" = "e7db7ca94ed4cd01190ceee0d8a8052f08a247aa1b469a7f68c6a3b71afcf407"
 "checksum honggfuzz 0.5.45 (registry+https://github.com/rust-lang/crates.io-index)" = "24c27b4aa3049d6d10d8e33d52c9d03ca9aec18f8a449b246f8c4a5b0c10fb34"
-"checksum impl-codec 0.4.1 (registry+https://github.com/rust-lang/crates.io-index)" = "3fa0086251524c50fd53b32e7b05eb6d79e2f97221eaf0c53c0ca9c3096f21d3"
+"checksum impl-codec 0.4.2 (registry+https://github.com/rust-lang/crates.io-index)" = "1be51a921b067b0eaca2fad532d9400041561aa922221cc65f95a85641c6bf53"
 "checksum integer-sqrt 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)" = "ea155abb3ba6f382a75f1418988c05fe82959ed9ce727de427f9cfd425b0c903"
 "checksum lazy_static 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)" = "e2abad23fbc42b3700f2f279844dc832adb2b2eb069b2df918f455c4e18cc646"
 "checksum libc 0.2.65 (registry+https://github.com/rust-lang/crates.io-index)" = "1a31a0627fdf1f6a39ec0dd577e101440b7db22672c0901fe00a9a6fbb5c24e8"
@@ -436,7 +423,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
 "checksum parity-scale-codec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)" = "001fbbb956d8593f321c7a784f64d16b2c99b2657823976eea729006ad2c3668"
 "checksum parity-scale-codec-derive 1.0.3 (registry+https://github.com/rust-lang/crates.io-index)" = "42af752f59119656fa3cb31e8852ed24e895b968c0bdb41847da7f0cea6d155f"
 "checksum ppv-lite86 0.2.6 (registry+https://github.com/rust-lang/crates.io-index)" = "74490b50b9fbe561ac330df47c08f3f33073d2d00c150f719147d7c54522fa1b"
-"checksum primitive-types 0.6.0 (registry+https://github.com/rust-lang/crates.io-index)" = "97b5a08dda18910f056e5c2060c034e77cab18e0bd7d895e44f03207af4c71d5"
+"checksum primitive-types 0.6.1 (registry+https://github.com/rust-lang/crates.io-index)" = "a0253db64c26d8b4e7896dd2063b516d2a1b9e0a5da26b5b78335f236d1e9522"
 "checksum proc-macro-crate 0.1.4 (registry+https://github.com/rust-lang/crates.io-index)" = "e10d4b51f154c8a7fb96fd6dad097cb74b863943ec010ac94b9fd1be8861fe1e"
 "checksum proc-macro2 0.4.30 (registry+https://github.com/rust-lang/crates.io-index)" = "cf3d2011ab5c909338f7887f4fc896d35932e29146c12c8d01da6b22a80ba759"
 "checksum proc-macro2 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)" = "9c9e470a8dc4aeae2dee2f335e8f533e2d4b347e1434e5671afc49b054592f27"
@@ -447,16 +434,13 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
 "checksum rand_core 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)" = "90bde5296fc891b0cef12a6d03ddccc162ce7b2aff54160af9338f8d40df6d19"
 "checksum rand_hc 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)" = "ca3129af7b92a17112d59ad498c6f81eaf463253766b90396d39ea7a39d6613c"
 "checksum rustc-hex 2.0.1 (registry+https://github.com/rust-lang/crates.io-index)" = "403bb3a286107a04825a5f82e1270acc1e14028d3d554d7a1e08914549575ab8"
-"checksum rustc_version 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)" = "138e3e0acb6c9fb258b19b67cb8abd63c00679d2851805ea151465464fe9030a"
-"checksum semver 0.9.0 (registry+https://github.com/rust-lang/crates.io-index)" = "1d7eb9ef2c18661902cc47e535f9bc51b78acd254da71d375c2f6720d9a40403"
-"checksum semver-parser 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)" = "388a1df253eca08550bef6c72392cfe7c30914bf41df5269b68cbd6ff8f570a3"
-"checksum serde 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)" = "9796c9b7ba2ffe7a9ce53c2287dfc48080f4b2b362fcc245a259b3a7201119dd"
-"checksum serde_derive 1.0.101 (registry+https://github.com/rust-lang/crates.io-index)" = "4b133a43a1ecd55d4086bd5b4dc6c1751c68b1bfbeba7a5040442022c7e7c02e"
-"checksum static_assertions 0.2.5 (registry+https://github.com/rust-lang/crates.io-index)" = "c19be23126415861cb3a23e501d34a708f7f9b2183c5252d690941c2e69199d5"
+"checksum serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)" = "0c4b39bd9b0b087684013a792c59e3e07a46a01d2322518d8a1104641a0b1be0"
+"checksum serde_derive 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)" = "ca13fc1a832f793322228923fbb3aba9f3f44444898f835d31ad1b74fa0a2bf8"
+"checksum static_assertions 1.1.0 (registry+https://github.com/rust-lang/crates.io-index)" = "a2eb9349b6444b326872e140eb1cf5e7c522154d69e7a0ffb0fb81c06b37543f"
 "checksum syn 0.15.44 (registry+https://github.com/rust-lang/crates.io-index)" = "9ca4b3b69a77cbe1ffc9e198781b7acb0c7365a883670e8f1c1bc66fba79a5c5"
-"checksum syn 1.0.5 (registry+https://github.com/rust-lang/crates.io-index)" = "66850e97125af79138385e9b88339cbcd037e3f28ceab8c5ad98e64f0f1f80bf"
-"checksum toml 0.5.3 (registry+https://github.com/rust-lang/crates.io-index)" = "c7aabe75941d914b72bf3e5d3932ed92ce0664d49d8432305a8b547c37227724"
-"checksum uint 0.8.1 (registry+https://github.com/rust-lang/crates.io-index)" = "f8f0f47ed099f0db671ce82c66548c5de012e3c0cba3963514d1db15c7588701"
+"checksum syn 1.0.8 (registry+https://github.com/rust-lang/crates.io-index)" = "661641ea2aa15845cddeb97dad000d22070bb5c1fb456b96c1cba883ec691e92"
+"checksum toml 0.5.5 (registry+https://github.com/rust-lang/crates.io-index)" = "01d1404644c8b12b16bfcffa4322403a91a451584daaaa7c28d3152e6cbc98cf"
+"checksum uint 0.8.2 (registry+https://github.com/rust-lang/crates.io-index)" = "e75a4cdd7b87b28840dba13c483b9a88ee6bbf16ba5c951ee1ecfcf723078e0d"
 "checksum unicode-xid 0.1.0 (registry+https://github.com/rust-lang/crates.io-index)" = "fc72304796d0818e357ead4e000d19c9c174ab23dc11093ac919054d20a6a7fc"
 "checksum unicode-xid 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)" = "826e7639553986605ec5979c7dd957c7895e93eabed50ab2ffa7f6128a75097c"
 "checksum wasi 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)" = "b89c3ce4ce14bdc6fb6beaf9ec7928ca331de5df7e5ea278375642a2f478570d"
diff --git a/core/sr-arithmetic/fuzzer/Cargo.toml b/primitives/sr-arithmetic/fuzzer/Cargo.toml
similarity index 100%
rename from core/sr-arithmetic/fuzzer/Cargo.toml
rename to primitives/sr-arithmetic/fuzzer/Cargo.toml
diff --git a/core/sr-arithmetic/fuzzer/src/biguint.rs b/primitives/sr-arithmetic/fuzzer/src/biguint.rs
similarity index 100%
rename from core/sr-arithmetic/fuzzer/src/biguint.rs
rename to primitives/sr-arithmetic/fuzzer/src/biguint.rs
diff --git a/core/sr-arithmetic/fuzzer/src/rational128.rs b/primitives/sr-arithmetic/fuzzer/src/rational128.rs
similarity index 100%
rename from core/sr-arithmetic/fuzzer/src/rational128.rs
rename to primitives/sr-arithmetic/fuzzer/src/rational128.rs
diff --git a/core/sr-arithmetic/src/biguint.rs b/primitives/sr-arithmetic/src/biguint.rs
similarity index 100%
rename from core/sr-arithmetic/src/biguint.rs
rename to primitives/sr-arithmetic/src/biguint.rs
diff --git a/core/sr-arithmetic/src/fixed64.rs b/primitives/sr-arithmetic/src/fixed64.rs
similarity index 100%
rename from core/sr-arithmetic/src/fixed64.rs
rename to primitives/sr-arithmetic/src/fixed64.rs
diff --git a/core/sr-arithmetic/src/helpers_128bit.rs b/primitives/sr-arithmetic/src/helpers_128bit.rs
similarity index 100%
rename from core/sr-arithmetic/src/helpers_128bit.rs
rename to primitives/sr-arithmetic/src/helpers_128bit.rs
diff --git a/core/sr-arithmetic/src/lib.rs b/primitives/sr-arithmetic/src/lib.rs
similarity index 100%
rename from core/sr-arithmetic/src/lib.rs
rename to primitives/sr-arithmetic/src/lib.rs
diff --git a/core/sr-arithmetic/src/per_things.rs b/primitives/sr-arithmetic/src/per_things.rs
similarity index 100%
rename from core/sr-arithmetic/src/per_things.rs
rename to primitives/sr-arithmetic/src/per_things.rs
diff --git a/core/sr-arithmetic/src/rational128.rs b/primitives/sr-arithmetic/src/rational128.rs
similarity index 100%
rename from core/sr-arithmetic/src/rational128.rs
rename to primitives/sr-arithmetic/src/rational128.rs
diff --git a/core/sr-arithmetic/src/traits.rs b/primitives/sr-arithmetic/src/traits.rs
similarity index 100%
rename from core/sr-arithmetic/src/traits.rs
rename to primitives/sr-arithmetic/src/traits.rs
diff --git a/core/sr-io/Cargo.toml b/primitives/sr-io/Cargo.toml
similarity index 57%
rename from core/sr-io/Cargo.toml
rename to primitives/sr-io/Cargo.toml
index 4d140d289dafe..52e175d2b80cd 100644
--- a/core/sr-io/Cargo.toml
+++ b/primitives/sr-io/Cargo.toml
@@ -2,21 +2,17 @@
 name = "sr-io"
 version = "2.0.0"
 authors = ["Parity Technologies "]
-build = "build.rs"
 edition = "2018"
 
-[build-dependencies]
-rustc_version = "0.2.3"
-
 [dependencies]
-codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false }
+codec = { package = "parity-scale-codec", version = "1.0.6", default-features = false }
 hash-db = { version = "0.15.2", default-features = false }
-primitives = { package = "substrate-primitives", path = "../primitives", default-features = false }
+primitives = { package = "substrate-primitives", path = "../core", default-features = false }
 rstd = { package = "sr-std", path = "../sr-std", default-features = false }
 libsecp256k1 = { version = "0.3.0", optional = true }
-tiny-keccak = { version = "1.5.0", optional = true }
-substrate-state-machine = { path = "../state-machine", optional = true }
-trie = { package = "substrate-trie", path = "../trie", optional = true }
+substrate-state-machine = { path = "../../primitives/state-machine", optional = true }
+runtime-interface = { package = "substrate-runtime-interface", path = "../runtime-interface", default-features = false }
+trie = { package = "substrate-trie", path = "../../primitives/trie", optional = true }
 externalities = { package = "substrate-externalities", path = "../externalities", optional = true }
 log = { version = "0.4.8", optional = true }
 
@@ -30,16 +26,15 @@ std = [
 	"trie",
 	"substrate-state-machine",
 	"libsecp256k1",
-	"tiny-keccak",
+	"runtime-interface/std",
 	"externalities",
 	"log",
 ]
-nightly = []
-strict = []
 
 # These two features are used for `no_std` builds for the environments which already provides
-# `#[panic_handler]` and `#[alloc_error_handler]`.
+# `#[panic_handler]`, `#[alloc_error_handler]` and `#[global_allocator]`.
 #
 # For the regular wasm runtime builds those are not used.
-no_panic_handler = []
-no_oom = []
+disable_panic_handler = []
+disable_oom = []
+disable_allocator = []
diff --git a/primitives/sr-io/src/lib.rs b/primitives/sr-io/src/lib.rs
new file mode 100644
index 0000000000000..080d0b245b738
--- /dev/null
+++ b/primitives/sr-io/src/lib.rs
@@ -0,0 +1,840 @@
+// Copyright 2017-2019 Parity Technologies (UK) Ltd.
+// This file is part of Substrate.
+
+// Substrate is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+
+// Substrate is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+
+// You should have received a copy of the GNU General Public License
+// along with Substrate.  If not, see .
+
+//! This is part of the Substrate runtime.
+
+#![warn(missing_docs)]
+
+#![cfg_attr(not(feature = "std"), no_std)]
+#![cfg_attr(not(feature = "std"), feature(lang_items))]
+#![cfg_attr(not(feature = "std"), feature(alloc_error_handler))]
+#![cfg_attr(not(feature = "std"), feature(core_intrinsics))]
+
+#![cfg_attr(feature = "std",
+   doc = "Substrate runtime standard library as compiled when linked with Rust's standard library.")]
+#![cfg_attr(not(feature = "std"),
+   doc = "Substrate's runtime standard library as compiled without Rust's standard library.")]
+
+use rstd::vec::Vec;
+
+#[cfg(feature = "std")]
+use rstd::ops::Deref;
+
+#[cfg(feature = "std")]
+use primitives::{
+	crypto::Pair, traits::KeystoreExt, offchain::OffchainExt, hexdisplay::HexDisplay,
+	storage::ChildStorageKey,
+};
+
+use primitives::{
+	crypto::KeyTypeId, ed25519, sr25519, H256, LogLevel,
+	offchain::{
+		Timestamp, HttpRequestId, HttpRequestStatus, HttpError, StorageKind, OpaqueNetworkState,
+	},
+};
+
+#[cfg(feature = "std")]
+use trie::{TrieConfiguration, trie_types::Layout};
+
+use runtime_interface::{runtime_interface, Pointer};
+
+use codec::{Encode, Decode};
+
+#[cfg(feature = "std")]
+use externalities::{ExternalitiesExt, Externalities};
+
+/// Error verifying ECDSA signature
+#[derive(Encode, Decode)]
+pub enum EcdsaVerifyError {
+	/// Incorrect value of R or S
+	BadRS,
+	/// Incorrect value of V
+	BadV,
+	/// Invalid signature
+	BadSignature,
+}
+
+/// Returns a `ChildStorageKey` if the given `storage_key` slice is a valid storage
+/// key or panics otherwise.
+///
+/// Panicking here is aligned with what the `without_std` environment would do
+/// in the case of an invalid child storage key.
+#[cfg(feature = "std")]
+fn child_storage_key_or_panic(storage_key: &[u8]) -> ChildStorageKey {
+	match ChildStorageKey::from_slice(storage_key) {
+		Some(storage_key) => storage_key,
+		None => panic!("child storage key is invalid"),
+	}
+}
+
+/// Interface for accessing the storage from within the runtime.
+#[runtime_interface]
+pub trait Storage {
+	/// Returns the data for `key` in the storage or `None` if the key can not be found.
+	fn get(&self, key: &[u8]) -> Option> {
+		self.storage(key).map(|s| s.to_vec())
+	}
+
+	/// Returns the data for `key` in the child storage or `None` if the key can not be found.
+	fn child_get(&self, child_storage_key: &[u8], key: &[u8]) -> Option> {
+		let storage_key = child_storage_key_or_panic(child_storage_key);
+		self.child_storage(storage_key, key).map(|s| s.to_vec())
+	}
+
+	/// Get `key` from storage, placing the value into `value_out` and return the number of
+	/// bytes that the entry in storage has beyond the offset or `None` if the storage entry
+	/// doesn't exist at all.
+	/// If `value_out` length is smaller than the returned length, only `value_out` length bytes
+	/// are copied into `value_out`.
+	fn read(&self, key: &[u8], value_out: &mut [u8], value_offset: u32) -> Option {
+		self.storage(key).map(|value| {
+			let value_offset = value_offset as usize;
+			let data = &value[value_offset.min(value.len())..];
+			let written = std::cmp::min(data.len(), value_out.len());
+			value_out[..written].copy_from_slice(&data[..written]);
+			value.len() as u32
+		})
+	}
+
+	/// Get `key` from child storage, placing the value into `value_out` and return the number
+	/// of bytes that the entry in storage has beyond the offset or `None` if the storage entry
+	/// doesn't exist at all.
+	/// If `value_out` length is smaller than the returned length, only `value_out` length bytes
+	/// are copied into `value_out`.
+	fn child_read(
+		&self,
+		child_storage_key: &[u8],
+		key: &[u8],
+		value_out: &mut [u8],
+		value_offset: u32,
+	) -> Option {
+		let storage_key = child_storage_key_or_panic(child_storage_key);
+		self.child_storage(storage_key, key)
+			.map(|value| {
+				let value_offset = value_offset as usize;
+				let data = &value[value_offset.min(value.len())..];
+				let written = std::cmp::min(data.len(), value_out.len());
+				value_out[..written].copy_from_slice(&data[..written]);
+				value.len() as u32
+			})
+	}
+
+	/// Set `key` to `value` in the storage.
+	fn set(&mut self, key: &[u8], value: &[u8]) {
+		self.set_storage(key.to_vec(), value.to_vec());
+	}
+
+	/// Set `key` to `value` in the child storage denoted by `child_storage_key`.
+	fn child_set(&mut self, child_storage_key: &[u8], key: &[u8], value: &[u8]) {
+		let storage_key = child_storage_key_or_panic(child_storage_key);
+		self.set_child_storage(storage_key, key.to_vec(), value.to_vec());
+	}
+
+	/// Clear the storage of the given `key` and its value.
+	fn clear(&mut self, key: &[u8]) {
+		self.clear_storage(key)
+	}
+
+	/// Clear the given child storage of the given `key` and its value.
+	fn child_clear(&mut self, child_storage_key: &[u8], key: &[u8]) {
+		let storage_key = child_storage_key_or_panic(child_storage_key);
+		self.clear_child_storage(storage_key, key);
+	}
+
+	/// Clear an entire child storage.
+	fn child_storage_kill(&mut self, child_storage_key: &[u8]) {
+		let storage_key = child_storage_key_or_panic(child_storage_key);
+		self.kill_child_storage(storage_key);
+	}
+
+	/// Check whether the given `key` exists in storage.
+	fn exists(&self, key: &[u8]) -> bool {
+		self.exists_storage(key)
+	}
+
+	/// Check whether the given `key` exists in storage.
+	fn child_exists(&self, child_storage_key: &[u8], key: &[u8]) -> bool {
+		let storage_key = child_storage_key_or_panic(child_storage_key);
+		self.exists_child_storage(storage_key, key)
+	}
+
+	/// Clear the storage of each key-value pair where the key starts with the given `prefix`.
+	fn clear_prefix(&mut self, prefix: &[u8]) {
+		Externalities::clear_prefix(*self, prefix)
+	}
+
+	/// Clear the child storage of each key-value pair where the key starts with the given `prefix`.
+	fn child_clear_prefix(&mut self, child_storage_key: &[u8], prefix: &[u8]) {
+		let storage_key = child_storage_key_or_panic(child_storage_key);
+		self.clear_child_prefix(storage_key, prefix);
+	}
+
+	/// "Commit" all existing operations and compute the resulting storage root.
+	fn root(&mut self) -> H256 {
+		self.storage_root()
+	}
+
+	/// "Commit" all existing operations and compute the resulting child storage root.
+	fn child_root(&mut self, child_storage_key: &[u8]) -> Vec {
+		let storage_key = child_storage_key_or_panic(child_storage_key);
+		self.child_storage_root(storage_key)
+	}
+
+	/// "Commit" all existing operations and get the resulting storage change root.
+	fn changes_root(&mut self, parent_hash: [u8; 32]) -> Option {
+		self.storage_changes_root(parent_hash.into()).ok().and_then(|h| h)
+	}
+
+	/// A trie root formed from the iterated items.
+	fn blake2_256_trie_root(input: Vec<(Vec, Vec)>) -> H256 {
+		Layout::::trie_root(input)
+	}
+
+	/// A trie root formed from the enumerated items.
+	fn blake2_256_ordered_trie_root(input: Vec>) -> H256 {
+		Layout::::ordered_trie_root(input)
+	}
+}
+
+/// Interface that provides miscellaneous functions for communicating between the runtime and the node.
+#[runtime_interface]
+pub trait Misc {
+	/// The current relay chain identifier.
+	fn chain_id(&self) -> u64 {
+		externalities::Externalities::chain_id(*self)
+	}
+
+	/// Print a number.
+	fn print_num(val: u64) {
+		log::debug!(target: "runtime", "{}", val);
+	}
+
+	/// Print any valid `utf8` buffer.
+	fn print_utf8(utf8: &[u8]) {
+		if let Ok(data) = std::str::from_utf8(utf8) {
+			log::debug!(target: "runtime", "{}", data)
+		}
+	}
+
+	/// Print any `u8` slice as hex.
+	fn print_hex(data: &[u8]) {
+		log::debug!(target: "runtime", "{}", HexDisplay::from(&data));
+	}
+}
+
+/// Interfaces for working with crypto related types from within the runtime.
+#[runtime_interface]
+pub trait Crypto {
+	/// Returns all `ed25519` public keys for the given key id from the keystore.
+	fn ed25519_public_keys(&mut self, id: KeyTypeId) -> Vec {
+		self.extension::()
+			.expect("No `keystore` associated for the current context!")
+			.read()
+			.ed25519_public_keys(id)
+	}
+
+	/// Generate an `ed22519` key for the given key type using an optional `seed` and
+	/// store it in the keystore.
+	///
+	/// The `seed` needs to be a valid utf8.
+	///
+	/// Returns the public key.
+	fn ed25519_generate(&mut self, id: KeyTypeId, seed: Option>) -> ed25519::Public {
+		let seed = seed.as_ref().map(|s| std::str::from_utf8(&s).expect("Seed is valid utf8!"));
+		self.extension::()
+			.expect("No `keystore` associated for the current context!")
+			.write()
+			.ed25519_generate_new(id, seed)
+			.expect("`ed25519_generate` failed")
+	}
+
+	/// Sign the given `msg` with the `ed25519` key that corresponds to the given public key and
+	/// key type in the keystore.
+	///
+	/// Returns the signature.
+	fn ed25519_sign(
+		&mut self,
+		id: KeyTypeId,
+		pub_key: &ed25519::Public,
+		msg: &[u8],
+	) -> Option {
+		self.extension::()
+			.expect("No `keystore` associated for the current context!")
+			.read()
+			.ed25519_key_pair(id, &pub_key)
+			.map(|k| k.sign(msg))
+	}
+
+	/// Verify an `ed25519` signature.
+	///
+	/// Returns `true` when the verification in successful.
+	fn ed25519_verify(
+		&self,
+		sig: &ed25519::Signature,
+		msg: &[u8],
+		pub_key: &ed25519::Public,
+	) -> bool {
+		ed25519::Pair::verify(sig, msg, pub_key)
+	}
+
+	/// Returns all `sr25519` public keys for the given key id from the keystore.
+	fn sr25519_public_keys(&mut self, id: KeyTypeId) -> Vec {
+		self.extension::()
+			.expect("No `keystore` associated for the current context!")
+			.read()
+			.sr25519_public_keys(id)
+	}
+
+	/// Generate an `sr22519` key for the given key type using an optional seed and
+	/// store it in the keystore.
+	///
+	/// The `seed` needs to be a valid utf8.
+	///
+	/// Returns the public key.
+	fn sr25519_generate(&mut self, id: KeyTypeId, seed: Option>) -> sr25519::Public {
+		let seed = seed.as_ref().map(|s| std::str::from_utf8(&s).expect("Seed is valid utf8!"));
+		self.extension::()
+			.expect("No `keystore` associated for the current context!")
+			.write()
+			.sr25519_generate_new(id, seed)
+			.expect("`sr25519_generate` failed")
+	}
+
+	/// Sign the given `msg` with the `sr25519` key that corresponds to the given public key and
+	/// key type in the keystore.
+	///
+	/// Returns the signature.
+	fn sr25519_sign(
+		&mut self,
+		id: KeyTypeId,
+		pub_key: &sr25519::Public,
+		msg: &[u8],
+	) -> Option {
+		self.extension::()
+			.expect("No `keystore` associated for the current context!")
+			.read()
+			.sr25519_key_pair(id, &pub_key)
+			.map(|k| k.sign(msg))
+	}
+
+	/// Verify an `sr25519` signature.
+	///
+	/// Returns `true` when the verification in successful.
+	fn sr25519_verify(sig: &sr25519::Signature, msg: &[u8], pubkey: &sr25519::Public) -> bool {
+		sr25519::Pair::verify(sig, msg, pubkey)
+	}
+
+	/// Verify and recover a SECP256k1 ECDSA signature.
+	/// - `sig` is passed in RSV format. V should be either 0/1 or 27/28.
+	/// Returns `Err` if the signature is bad, otherwise the 64-byte pubkey
+	/// (doesn't include the 0x04 prefix).
+	fn secp256k1_ecdsa_recover(
+		sig: &[u8; 65],
+		msg: &[u8; 32],
+	) -> Result<[u8; 64], EcdsaVerifyError> {
+		let rs = secp256k1::Signature::parse_slice(&sig[0..64])
+			.map_err(|_| EcdsaVerifyError::BadRS)?;
+		let v = secp256k1::RecoveryId::parse(if sig[64] > 26 { sig[64] - 27 } else { sig[64] } as u8)
+			.map_err(|_| EcdsaVerifyError::BadV)?;
+		let pubkey = secp256k1::recover(&secp256k1::Message::parse(msg), &rs, &v)
+			.map_err(|_| EcdsaVerifyError::BadSignature)?;
+		let mut res = [0u8; 64];
+		res.copy_from_slice(&pubkey.serialize()[1..65]);
+		Ok(res)
+	}
+
+	/// Verify and recover a SECP256k1 ECDSA signature.
+	/// - `sig` is passed in RSV format. V should be either 0/1 or 27/28.
+	/// - returns `Err` if the signature is bad, otherwise the 33-byte compressed pubkey.
+	fn secp256k1_ecdsa_recover_compressed(
+		sig: &[u8; 65],
+		msg: &[u8; 32],
+	) -> Result<[u8; 33], EcdsaVerifyError> {
+		let rs = secp256k1::Signature::parse_slice(&sig[0..64])
+			.map_err(|_| EcdsaVerifyError::BadRS)?;
+		let v = secp256k1::RecoveryId::parse(if sig[64] > 26 { sig[64] - 27 } else { sig[64] } as u8)
+			.map_err(|_| EcdsaVerifyError::BadV)?;
+		let pubkey = secp256k1::recover(&secp256k1::Message::parse(msg), &rs, &v)
+			.map_err(|_| EcdsaVerifyError::BadSignature)?;
+		Ok(pubkey.serialize_compressed())
+	}
+}
+
+/// Interface that provides functions for hashing with different algorithms.
+#[runtime_interface]
+pub trait Hashing {
+	/// Conduct a 256-bit Keccak hash.
+	fn keccak_256(data: &[u8]) -> [u8; 32] {
+		primitives::hashing::keccak_256(data)
+	}
+
+	/// Conduct a 128-bit Blake2 hash.
+	fn blake2_128(data: &[u8]) -> [u8; 16] {
+		primitives::hashing::blake2_128(data)
+	}
+
+	/// Conduct a 256-bit Blake2 hash.
+	fn blake2_256(data: &[u8]) -> [u8; 32] {
+		primitives::hashing::blake2_256(data)
+	}
+
+	/// Conduct four XX hashes to give a 256-bit result.
+	fn twox_256(data: &[u8]) -> [u8; 32] {
+		primitives::hashing::twox_256(data)
+	}
+
+	/// Conduct two XX hashes to give a 128-bit result.
+	fn twox_128(data: &[u8]) -> [u8; 16] {
+		primitives::hashing::twox_128(data)
+	}
+
+	/// Conduct two XX hashes to give a 64-bit result.
+	fn twox_64(data: &[u8]) -> [u8; 8] {
+		primitives::hashing::twox_64(data)
+	}
+}
+
+/// Interface that provides functions to access the offchain functionality.
+#[runtime_interface]
+pub trait Offchain {
+	/// Returns if the local node is a potential validator.
+	///
+	/// Even if this function returns `true`, it does not mean that any keys are configured
+	/// and that the validator is registered in the chain.
+	fn is_validator(&mut self) -> bool {
+		self.extension::()
+			.expect("is_validator can be called only in the offchain worker context")
+			.is_validator()
+	}
+
+	/// Submit an encoded transaction to the pool.
+	///
+	/// The transaction will end up in the pool.
+	fn submit_transaction(&mut self, data: Vec) -> Result<(), ()> {
+		self.extension::()
+			.expect("submit_transaction can be called only in the offchain worker context")
+			.submit_transaction(data)
+	}
+
+	/// Returns information about the local node's network state.
+	fn network_state(&mut self) -> Result {
+		self.extension::()
+			.expect("network_state can be called only in the offchain worker context")
+			.network_state()
+	}
+
+	/// Returns current UNIX timestamp (in millis)
+	fn timestamp(&mut self) -> Timestamp {
+		self.extension::()
+			.expect("timestamp can be called only in the offchain worker context")
+			.timestamp()
+	}
+
+	/// Pause the execution until `deadline` is reached.
+	fn sleep_until(&mut self, deadline: Timestamp) {
+		self.extension::()
+			.expect("sleep_until can be called only in the offchain worker context")
+			.sleep_until(deadline)
+	}
+
+	/// Returns a random seed.
+	///
+	/// This is a trully random non deterministic seed generated by host environment.
+	/// Obviously fine in the off-chain worker context.
+	fn random_seed(&mut self) -> [u8; 32] {
+		self.extension::()
+			.expect("random_seed can be called only in the offchain worker context")
+			.random_seed()
+	}
+
+	/// Sets a value in the local storage.
+	///
+	/// Note this storage is not part of the consensus, it's only accessible by
+	/// offchain worker tasks running on the same machine. It IS persisted between runs.
+	fn local_storage_set(&mut self, kind: StorageKind, key: &[u8], value: &[u8]) {
+		self.extension::()
+			.expect("random_seed can be called only in the offchain worker context")
+			.local_storage_set(kind, key, value)
+	}
+
+	/// Sets a value in the local storage if it matches current value.
+	///
+	/// Since multiple offchain workers may be running concurrently, to prevent
+	/// data races use CAS to coordinate between them.
+	///
+	/// Returns `true` if the value has been set, `false` otherwise.
+	///
+	/// Note this storage is not part of the consensus, it's only accessible by
+	/// offchain worker tasks running on the same machine. It IS persisted between runs.
+	fn local_storage_compare_and_set(
+		&mut self,
+		kind: StorageKind,
+		key: &[u8],
+		old_value: Option>,
+		new_value: &[u8],
+	) -> bool {
+		self.extension::()
+			.expect("random_seed can be called only in the offchain worker context")
+			.local_storage_compare_and_set(kind, key, old_value.as_ref().map(|v| v.deref()), new_value)
+	}
+
+	/// Gets a value from the local storage.
+	///
+	/// If the value does not exist in the storage `None` will be returned.
+	/// Note this storage is not part of the consensus, it's only accessible by
+	/// offchain worker tasks running on the same machine. It IS persisted between runs.
+	fn local_storage_get(&mut self, kind: StorageKind, key: &[u8]) -> Option> {
+		self.extension::()
+			.expect("random_seed can be called only in the offchain worker context")
+			.local_storage_get(kind, key)
+	}
+
+	/// Initiates a http request given HTTP verb and the URL.
+	///
+	/// Meta is a future-reserved field containing additional, parity-scale-codec encoded parameters.
+	/// Returns the id of newly started request.
+	fn http_request_start(
+		&mut self,
+		method: &str,
+		uri: &str,
+		meta: &[u8],
+	) -> Result {
+		self.extension::()
+			.expect("random_seed can be called only in the offchain worker context")
+			.http_request_start(method, uri, meta)
+	}
+
+	/// Append header to the request.
+	fn http_request_add_header(
+		&mut self,
+		request_id: HttpRequestId,
+		name: &str,
+		value: &str,
+	) -> Result<(), ()> {
+		self.extension::()
+			.expect("random_seed can be called only in the offchain worker context")
+			.http_request_add_header(request_id, name, value)
+	}
+
+	/// Write a chunk of request body.
+	///
+	/// Writing an empty chunks finalizes the request.
+	/// Passing `None` as deadline blocks forever.
+	///
+	/// Returns an error in case deadline is reached or the chunk couldn't be written.
+	fn http_request_write_body(
+		&mut self,
+		request_id: HttpRequestId,
+		chunk: &[u8],
+		deadline: Option,
+	) -> Result<(), HttpError> {
+		self.extension::()
+			.expect("random_seed can be called only in the offchain worker context")
+			.http_request_write_body(request_id, chunk, deadline)
+	}
+
+	/// Block and wait for the responses for given requests.
+	///
+	/// Returns a vector of request statuses (the len is the same as ids).
+	/// Note that if deadline is not provided the method will block indefinitely,
+	/// otherwise unready responses will produce `DeadlineReached` status.
+	///
+	/// Passing `None` as deadline blocks forever.
+	fn http_response_wait(
+		&mut self,
+		ids: &[HttpRequestId],
+		deadline: Option,
+	) -> Vec {
+		self.extension::()
+			.expect("random_seed can be called only in the offchain worker context")
+			.http_response_wait(ids, deadline)
+	}
+
+	/// Read all response headers.
+	///
+	/// Returns a vector of pairs `(HeaderKey, HeaderValue)`.
+	/// NOTE response headers have to be read before response body.
+	fn http_response_headers(&mut self, request_id: HttpRequestId) -> Vec<(Vec, Vec)> {
+		self.extension::()
+			.expect("random_seed can be called only in the offchain worker context")
+			.http_response_headers(request_id)
+	}
+
+	/// Read a chunk of body response to given buffer.
+	///
+	/// Returns the number of bytes written or an error in case a deadline
+	/// is reached or server closed the connection.
+	/// If `0` is returned it means that the response has been fully consumed
+	/// and the `request_id` is now invalid.
+	/// NOTE this implies that response headers must be read before draining the body.
+	/// Passing `None` as a deadline blocks forever.
+	fn http_response_read_body(
+		&mut self,
+		request_id: HttpRequestId,
+		buffer: &mut [u8],
+		deadline: Option,
+	) -> Result {
+		self.extension::()
+			.expect("random_seed can be called only in the offchain worker context")
+			.http_response_read_body(request_id, buffer, deadline)
+			.map(|r| r as u32)
+	}
+}
+
+/// Wasm only interface that provides functions for calling into the allocator.
+#[runtime_interface(wasm_only)]
+trait Allocator {
+	/// Malloc the given number of bytes and return the pointer to the allocated memory location.
+	fn malloc(&mut self, size: u32) -> Pointer {
+		self.allocate_memory(size).expect("Failed to allocate memory")
+	}
+
+	/// Free the given pointer.
+	fn free(&mut self, ptr: Pointer) {
+		self.deallocate_memory(ptr).expect("Failed to deallocate memory")
+	}
+}
+
+/// Interface that provides functions for logging from within the runtime.
+#[runtime_interface]
+pub trait Logging {
+	/// Request to print a log message on the host.
+	///
+	/// Note that this will be only displayed if the host is enabled to display log messages with
+	/// given level and target.
+	///
+	/// Instead of using directly, prefer setting up `RuntimeLogger` and using `log` macros.
+	fn log(level: LogLevel, target: &str, message: &[u8]) {
+		if let Ok(message) = std::str::from_utf8(message) {
+			log::log!(
+				target: target,
+				log::Level::from(level),
+				"{}",
+				message,
+			)
+		}
+	}
+}
+
+/// Wasm-only interface that provides functions for interacting with the sandbox.
+#[runtime_interface(wasm_only)]
+pub trait Sandbox {
+	/// Instantiate a new sandbox instance with the given `wasm_code`.
+	fn instantiate(
+		&mut self,
+		dispatch_thunk: u32,
+		wasm_code: &[u8],
+		env_def: &[u8],
+		state_ptr: Pointer,
+	) -> u32 {
+		self.sandbox()
+			.instance_new(dispatch_thunk, wasm_code, env_def, state_ptr.into())
+			.expect("Failed to instantiate a new sandbox")
+	}
+
+	/// Invoke `function` in the sandbox with `sandbox_idx`.
+	fn invoke(
+		&mut self,
+		instance_idx: u32,
+		function: &str,
+		args: &[u8],
+		return_val_ptr: Pointer,
+		return_val_len: u32,
+		state_ptr: Pointer,
+	) -> u32 {
+		self.sandbox().invoke(
+			instance_idx,
+			&function,
+			&args,
+			return_val_ptr,
+			return_val_len,
+			state_ptr.into(),
+		).expect("Failed to invoke function with sandbox")
+	}
+
+	/// Create a new memory instance with the given `initial` and `maximum` size.
+	fn memory_new(&mut self, initial: u32, maximum: u32) -> u32 {
+		self.sandbox()
+			.memory_new(initial, maximum)
+			.expect("Failed to create new memory with sandbox")
+	}
+
+	/// Get the memory starting at `offset` from the instance with `memory_idx` into the buffer.
+	fn memory_get(
+		&mut self,
+		memory_idx: u32,
+		offset: u32,
+		buf_ptr: Pointer,
+		buf_len: u32,
+	) -> u32 {
+		self.sandbox()
+			.memory_get(memory_idx, offset, buf_ptr, buf_len)
+			.expect("Failed to get memory with sandbox")
+	}
+
+	/// Set the memory in the given `memory_idx` to the given value at `offset`.
+	fn memory_set(
+		&mut self,
+		memory_idx: u32,
+		offset: u32,
+		val_ptr: Pointer,
+		val_len: u32,
+	) -> u32 {
+		self.sandbox()
+			.memory_set(memory_idx, offset, val_ptr, val_len)
+			.expect("Failed to set memory with sandbox")
+	}
+
+	/// Teardown the memory instance with the given `memory_idx`.
+	fn memory_teardown(&mut self, memory_idx: u32) {
+		self.sandbox().memory_teardown(memory_idx).expect("Failed to teardown memory with sandbox")
+	}
+
+	/// Teardown the sandbox instance with the given `instance_idx`.
+	fn instance_teardown(&mut self, instance_idx: u32) {
+		self.sandbox().instance_teardown(instance_idx).expect("Failed to teardown sandbox instance")
+	}
+}
+
+/// Allocator used by Substrate when executing the Wasm runtime.
+#[cfg(not(feature = "std"))]
+struct WasmAllocator;
+
+#[cfg(all(not(feature = "disable_global_allocator"), not(feature = "std")))]
+#[global_allocator]
+static ALLOCATOR: WasmAllocator = WasmAllocator;
+
+#[cfg(not(feature = "std"))]
+mod allocator_impl {
+	use super::*;
+	use core::alloc::{GlobalAlloc, Layout};
+
+	unsafe impl GlobalAlloc for WasmAllocator {
+		unsafe fn alloc(&self, layout: Layout) -> *mut u8 {
+			allocator::malloc(layout.size() as u32)
+		}
+
+		unsafe fn dealloc(&self, ptr: *mut u8, _: Layout) {
+			allocator::free(ptr)
+		}
+	}
+}
+
+#[cfg(all(not(feature = "disable_panic_handler"), not(feature = "std")))]
+#[panic_handler]
+#[no_mangle]
+pub fn panic(info: &core::panic::PanicInfo) -> ! {
+	unsafe {
+		let message = rstd::alloc::format!("{}", info);
+		misc::print_utf8(message.as_bytes());
+		core::intrinsics::abort()
+	}
+}
+
+#[cfg(all(not(feature = "disable_oom"), not(feature = "std")))]
+#[alloc_error_handler]
+pub extern fn oom(_: core::alloc::Layout) -> ! {
+	static OOM_MSG: &str = "Runtime memory exhausted. Aborting";
+
+	unsafe {
+		misc::print_utf8(OOM_MSG.as_bytes());
+		core::intrinsics::abort();
+	}
+}
+
+/// Type alias for Externalities implementation used in tests.
+#[cfg(feature = "std")]
+pub type TestExternalities = substrate_state_machine::TestExternalities;
+
+/// The host functions Substrate provides for the Wasm runtime environment.
+///
+/// All these host functions will be callable from inside the Wasm environment.
+#[cfg(feature = "std")]
+pub type SubstrateHostFunctions = (
+	storage::HostFunctions,
+	misc::HostFunctions,
+	offchain::HostFunctions,
+	crypto::HostFunctions,
+	hashing::HostFunctions,
+	allocator::HostFunctions,
+	logging::HostFunctions,
+	sandbox::HostFunctions,
+);
+
+#[cfg(test)]
+mod tests {
+	use super::*;
+	use primitives::map;
+	use substrate_state_machine::BasicExternalities;
+
+	#[test]
+	fn storage_works() {
+		let mut t = BasicExternalities::default();
+		t.execute_with(|| {
+			assert_eq!(storage::get(b"hello"), None);
+			storage::set(b"hello", b"world");
+			assert_eq!(storage::get(b"hello"), Some(b"world".to_vec()));
+			assert_eq!(storage::get(b"foo"), None);
+			storage::set(b"foo", &[1, 2, 3][..]);
+		});
+
+		t = BasicExternalities::new(map![b"foo".to_vec() => b"bar".to_vec()], map![]);
+
+		t.execute_with(|| {
+			assert_eq!(storage::get(b"hello"), None);
+			assert_eq!(storage::get(b"foo"), Some(b"bar".to_vec()));
+		});
+	}
+
+	#[test]
+	fn read_storage_works() {
+		let mut t = BasicExternalities::new(
+			map![b":test".to_vec() => b"\x0b\0\0\0Hello world".to_vec()],
+			map![],
+		);
+
+		t.execute_with(|| {
+			let mut v = [0u8; 4];
+			assert!(storage::read(b":test", &mut v[..], 0).unwrap() >= 4);
+			assert_eq!(v, [11u8, 0, 0, 0]);
+			let mut w = [0u8; 11];
+			assert!(storage::read(b":test", &mut w[..], 4).unwrap() >= 11);
+			assert_eq!(&w, b"Hello world");
+		});
+	}
+
+	#[test]
+	fn clear_prefix_works() {
+		let mut t = BasicExternalities::new(
+			map![
+				b":a".to_vec() => b"\x0b\0\0\0Hello world".to_vec(),
+				b":abcd".to_vec() => b"\x0b\0\0\0Hello world".to_vec(),
+				b":abc".to_vec() => b"\x0b\0\0\0Hello world".to_vec(),
+				b":abdd".to_vec() => b"\x0b\0\0\0Hello world".to_vec()
+			],
+			map![],
+		);
+
+		t.execute_with(|| {
+			storage::clear_prefix(b":abc");
+
+			assert!(storage::get(b":a").is_some());
+			assert!(storage::get(b":abdd").is_some());
+			assert!(storage::get(b":abcd").is_none());
+			assert!(storage::get(b":abc").is_none());
+		});
+	}
+}
diff --git a/core/sr-io/with_std.rs b/primitives/sr-io/with_std.rs
similarity index 99%
rename from core/sr-io/with_std.rs
rename to primitives/sr-io/with_std.rs
index fdd32124c129c..7e0504c37aafc 100644
--- a/core/sr-io/with_std.rs
+++ b/primitives/sr-io/with_std.rs
@@ -180,17 +180,17 @@ impl OtherApi for () {
 	}
 
 	fn print_num(val: u64) {
-		println!("{}", val);
+		log::debug!(target: "runtime", "{}", val);
 	}
 
 	fn print_utf8(utf8: &[u8]) {
 		if let Ok(data) = std::str::from_utf8(utf8) {
-			println!("{}", data)
+			log::debug!(target: "runtime", "{}", data)
 		}
 	}
 
 	fn print_hex(data: &[u8]) {
-		println!("{}", HexDisplay::from(&data));
+		log::debug!(target: "runtime", "{}", HexDisplay::from(&data));
 	}
 
 	fn log(
diff --git a/core/sr-io/without_std.rs b/primitives/sr-io/without_std.rs
similarity index 100%
rename from core/sr-io/without_std.rs
rename to primitives/sr-io/without_std.rs
diff --git a/core/sr-primitives/Cargo.toml b/primitives/sr-primitives/Cargo.toml
similarity index 77%
rename from core/sr-primitives/Cargo.toml
rename to primitives/sr-primitives/Cargo.toml
index d3303014260f4..e1412cf53407e 100644
--- a/core/sr-primitives/Cargo.toml
+++ b/primitives/sr-primitives/Cargo.toml
@@ -7,7 +7,7 @@ edition = "2018"
 [dependencies]
 serde = { version = "1.0.101", optional = true, features = ["derive"] }
 codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false, features = ["derive"] }
-primitives = { package = "substrate-primitives",  path = "../primitives", default-features = false }
+primitives = { package = "substrate-primitives",  path = "../core", default-features = false }
 app-crypto = { package = "substrate-application-crypto",  path = "../application-crypto", default-features = false }
 arithmetic = { package = "sr-arithmetic",  path = "../sr-arithmetic", default-features = false }
 rstd = { package = "sr-std", path = "../sr-std", default-features = false }
@@ -15,12 +15,13 @@ runtime_io = { package = "sr-io", path = "../sr-io", default-features = false }
 log = { version = "0.4.8", optional = true }
 paste = "0.1.6"
 rand = { version = "0.7.2", optional = true }
-impl-trait-for-tuples = "0.1.2"
+impl-trait-for-tuples = "0.1.3"
+inherents = { package = "substrate-inherents", path = "../inherents", default-features = false }
 
 [dev-dependencies]
 serde_json = "1.0.41"
 rand = "0.7.2"
-substrate-offchain = { path = "../offchain" }
+substrate-offchain = { path = "../../client/offchain" }
 
 [features]
 bench = []
@@ -35,4 +36,5 @@ std = [
 	"rstd/std",
 	"runtime_io/std",
 	"serde",
+	"inherents/std",
 ]
diff --git a/core/sr-primitives/src/curve.rs b/primitives/sr-primitives/src/curve.rs
similarity index 97%
rename from core/sr-primitives/src/curve.rs
rename to primitives/sr-primitives/src/curve.rs
index 6f25af3e10b6e..52a6ddd33b8c4 100644
--- a/core/sr-primitives/src/curve.rs
+++ b/primitives/sr-primitives/src/curve.rs
@@ -23,7 +23,9 @@ use core::ops::Sub;
 #[derive(PartialEq, Eq, primitives::RuntimeDebug)]
 pub struct PiecewiseLinear<'a> {
 	/// Array of points. Must be in order from the lowest abscissas to the highest.
-	pub points: &'a [(Perbill, Perbill)]
+	pub points: &'a [(Perbill, Perbill)],
+	/// The maximum value that can be returned.
+	pub maximum: Perbill,
 }
 
 fn abs_sub + Clone>(a: N, b: N) -> N where {
@@ -135,7 +137,8 @@ fn test_calculate_for_fraction_times_denominator() {
 			(Perbill::from_parts(0_000_000_000), Perbill::from_parts(0_500_000_000)),
 			(Perbill::from_parts(0_500_000_000), Perbill::from_parts(1_000_000_000)),
 			(Perbill::from_parts(1_000_000_000), Perbill::from_parts(0_000_000_000)),
-		]
+		],
+		maximum: Perbill::from_parts(1_000_000_000),
 	};
 
 	pub fn formal_calculate_for_fraction_times_denominator(n: u64, d: u64) -> u64 {
diff --git a/core/sr-primitives/src/generic/block.rs b/primitives/sr-primitives/src/generic/block.rs
similarity index 100%
rename from core/sr-primitives/src/generic/block.rs
rename to primitives/sr-primitives/src/generic/block.rs
diff --git a/core/sr-primitives/src/generic/checked_extrinsic.rs b/primitives/sr-primitives/src/generic/checked_extrinsic.rs
similarity index 85%
rename from core/sr-primitives/src/generic/checked_extrinsic.rs
rename to primitives/sr-primitives/src/generic/checked_extrinsic.rs
index 1e030ea1d8769..3fc7711ccc451 100644
--- a/core/sr-primitives/src/generic/checked_extrinsic.rs
+++ b/primitives/sr-primitives/src/generic/checked_extrinsic.rs
@@ -18,8 +18,10 @@
 //! stage.
 
 use crate::traits::{
-	self, Member, MaybeDisplay, SignedExtension, Dispatchable, ValidateUnsigned,
+	self, Member, MaybeDisplay, SignedExtension, Dispatchable,
 };
+#[allow(deprecated)]
+use crate::traits::ValidateUnsigned;
 use crate::weights::{GetDispatchInfo, DispatchInfo};
 use crate::transaction_validity::TransactionValidity;
 
@@ -36,8 +38,7 @@ pub struct CheckedExtrinsic {
 	pub function: Call,
 }
 
-impl traits::Applyable
-for
+impl traits::Applyable for
 	CheckedExtrinsic
 where
 	AccountId: Member + MaybeDisplay,
@@ -52,6 +53,7 @@ where
 		self.signed.as_ref().map(|x| &x.0)
 	}
 
+	#[allow(deprecated)] // Allow ValidateUnsigned
 	fn validate>(
 		&self,
 		info: DispatchInfo,
@@ -61,11 +63,13 @@ where
 			Extra::validate(extra, id, &self.function, info, len)
 		} else {
 			let valid = Extra::validate_unsigned(&self.function, info, len)?;
-			Ok(valid.combine_with(U::validate_unsigned(&self.function)?))
+			let unsigned_validation = U::validate_unsigned(&self.function)?;
+			Ok(valid.combine_with(unsigned_validation))
 		}
 	}
 
-	fn apply(
+	#[allow(deprecated)] // Allow ValidateUnsigned
+	fn apply>(
 		self,
 		info: DispatchInfo,
 		len: usize,
@@ -75,6 +79,7 @@ where
 			(Some(id), pre)
 		} else {
 			let pre = Extra::pre_dispatch_unsigned(&self.function, info, len)?;
+			U::pre_dispatch(&self.function)?;
 			(None, pre)
 		};
 		let res = self.function.dispatch(Origin::from(maybe_who));
diff --git a/core/sr-primitives/src/generic/digest.rs b/primitives/sr-primitives/src/generic/digest.rs
similarity index 100%
rename from core/sr-primitives/src/generic/digest.rs
rename to primitives/sr-primitives/src/generic/digest.rs
diff --git a/core/sr-primitives/src/generic/era.rs b/primitives/sr-primitives/src/generic/era.rs
similarity index 100%
rename from core/sr-primitives/src/generic/era.rs
rename to primitives/sr-primitives/src/generic/era.rs
diff --git a/core/sr-primitives/src/generic/header.rs b/primitives/sr-primitives/src/generic/header.rs
similarity index 100%
rename from core/sr-primitives/src/generic/header.rs
rename to primitives/sr-primitives/src/generic/header.rs
diff --git a/core/sr-primitives/src/generic/mod.rs b/primitives/sr-primitives/src/generic/mod.rs
similarity index 100%
rename from core/sr-primitives/src/generic/mod.rs
rename to primitives/sr-primitives/src/generic/mod.rs
diff --git a/core/sr-primitives/src/generic/tests.rs b/primitives/sr-primitives/src/generic/tests.rs
similarity index 100%
rename from core/sr-primitives/src/generic/tests.rs
rename to primitives/sr-primitives/src/generic/tests.rs
diff --git a/core/sr-primitives/src/generic/unchecked_extrinsic.rs b/primitives/sr-primitives/src/generic/unchecked_extrinsic.rs
similarity index 99%
rename from core/sr-primitives/src/generic/unchecked_extrinsic.rs
rename to primitives/sr-primitives/src/generic/unchecked_extrinsic.rs
index befa857dffd64..79d86a0756d3c 100644
--- a/core/sr-primitives/src/generic/unchecked_extrinsic.rs
+++ b/primitives/sr-primitives/src/generic/unchecked_extrinsic.rs
@@ -16,9 +16,8 @@
 
 //! Generic implementation of an unchecked (pre-verification) extrinsic.
 
-use rstd::prelude::*;
-use rstd::fmt;
-use runtime_io::blake2_256;
+use rstd::{fmt, prelude::*};
+use runtime_io::hashing::blake2_256;
 use codec::{Decode, Encode, EncodeLike, Input, Error};
 use crate::{
 	traits::{self, Member, MaybeDisplay, SignedExtension, Checkable, Extrinsic, IdentifyAccount},
@@ -295,7 +294,7 @@ where
 #[cfg(test)]
 mod tests {
 	use super::*;
-	use runtime_io::blake2_256;
+	use runtime_io::hashing::blake2_256;
 	use crate::codec::{Encode, Decode};
 	use crate::traits::{SignedExtension, IdentifyAccount, IdentityLookup};
 	use serde::{Serialize, Deserialize};
diff --git a/core/sr-primitives/src/lib.rs b/primitives/sr-primitives/src/lib.rs
similarity index 95%
rename from core/sr-primitives/src/lib.rs
rename to primitives/sr-primitives/src/lib.rs
index fce9f7def05b3..01ace2c004d0b 100644
--- a/core/sr-primitives/src/lib.rs
+++ b/primitives/sr-primitives/src/lib.rs
@@ -38,19 +38,18 @@ pub use paste;
 pub use app_crypto;
 
 #[cfg(feature = "std")]
-pub use runtime_io::{StorageOverlay, ChildrenStorageOverlay};
+pub use primitives::storage::{StorageOverlay, ChildrenStorageOverlay};
 
 use rstd::prelude::*;
 use rstd::convert::TryFrom;
 use primitives::{crypto, ed25519, sr25519, ecdsa, hash::{H256, H512}};
 use codec::{Encode, Decode};
 
-#[cfg(feature = "std")]
-pub mod testing;
-
 pub mod curve;
 pub mod generic;
 pub mod offchain;
+#[cfg(feature = "std")]
+pub mod testing;
 pub mod traits;
 pub mod transaction_validity;
 pub mod weights;
@@ -66,10 +65,7 @@ pub use app_crypto::{RuntimeAppPublic, BoundToRuntimeAppPublic};
 pub use primitives::RuntimeDebug;
 
 /// Re-export top-level arithmetic stuff.
-pub use arithmetic::{
-	Perquintill, Perbill, Permill, Percent,
-	Rational128, Fixed64
-};
+pub use arithmetic::{Perquintill, Perbill, Permill, Percent, Rational128, Fixed64};
 /// Re-export 128 bit helpers.
 pub use arithmetic::helpers_128bit;
 /// Re-export big_uint stuff.
@@ -96,19 +92,19 @@ impl TypeId for ModuleId {
 
 /// A String that is a `&'static str` on `no_std` and a `Cow<'static, str>` on `std`.
 #[cfg(feature = "std")]
-pub type RuntimeString = ::std::borrow::Cow<'static, str>;
+pub type RuntimeString = std::borrow::Cow<'static, str>;
 /// A String that is a `&'static str` on `no_std` and a `Cow<'static, str>` on `std`.
 #[cfg(not(feature = "std"))]
 pub type RuntimeString = &'static str;
 
-/// Create a const [RuntimeString].
+/// Create a const [`RuntimeString`].
 #[cfg(feature = "std")]
 #[macro_export]
 macro_rules! create_runtime_str {
-	( $y:expr ) => {{ ::std::borrow::Cow::Borrowed($y) }}
+	( $y:expr ) => {{ std::borrow::Cow::Borrowed($y) }}
 }
 
-/// Create a const [RuntimeString].
+/// Create a const [`RuntimeString`].
 #[cfg(not(feature = "std"))]
 #[macro_export]
 macro_rules! create_runtime_str {
@@ -244,7 +240,7 @@ impl traits::IdentifyAccount for MultiSigner {
 		match self {
 			MultiSigner::Ed25519(who) => <[u8; 32]>::from(who).into(),
 			MultiSigner::Sr25519(who) => <[u8; 32]>::from(who).into(),
-			MultiSigner::Ecdsa(who) => runtime_io::blake2_256(who.as_ref()).into(),
+			MultiSigner::Ecdsa(who) => runtime_io::hashing::blake2_256(who.as_ref()).into(),
 		}
 	}
 }
@@ -307,9 +303,11 @@ impl Verify for MultiSignature {
 			(MultiSignature::Ed25519(ref sig), who) => sig.verify(msg, &ed25519::Public::from_slice(who.as_ref())),
 			(MultiSignature::Sr25519(ref sig), who) => sig.verify(msg, &sr25519::Public::from_slice(who.as_ref())),
 			(MultiSignature::Ecdsa(ref sig), who) => {
-				let m = runtime_io::blake2_256(msg.get());
-				match runtime_io::secp256k1_ecdsa_recover_compressed(sig.as_ref(), &m) {
-					Ok(pubkey) => &runtime_io::blake2_256(pubkey.as_ref()) == >::as_ref(who),
+				let m = runtime_io::hashing::blake2_256(msg.get());
+				match runtime_io::crypto::secp256k1_ecdsa_recover_compressed(sig.as_ref(), &m) {
+					Ok(pubkey) =>
+						&runtime_io::hashing::blake2_256(pubkey.as_ref())
+							== >::as_ref(who),
 					_ => false,
 				}
 			}
@@ -455,7 +453,11 @@ impl From<&'static str> for DispatchError {
 
 /// Verify a signature on an encoded value in a lazy manner. This can be
 /// an optimization if the signature scheme has an "unsigned" escape hash.
-pub fn verify_encoded_lazy(sig: &V, item: &T, signer: &::AccountId) -> bool {
+pub fn verify_encoded_lazy(
+	sig: &V,
+	item: &T,
+	signer: &::AccountId
+) -> bool {
 	// The `Lazy` trait expresses something like `X: FnMut &'a T>`.
 	// unfortunately this is a lifetime relationship that can't
 	// be expressed without generic associated types, better unification of HRTBs in type position,
diff --git a/core/sr-primitives/src/offchain/http.rs b/primitives/sr-primitives/src/offchain/http.rs
similarity index 95%
rename from core/sr-primitives/src/offchain/http.rs
rename to primitives/sr-primitives/src/offchain/http.rs
index 77e514d65342b..8024437075924 100644
--- a/core/sr-primitives/src/offchain/http.rs
+++ b/primitives/sr-primitives/src/offchain/http.rs
@@ -221,11 +221,15 @@ impl<'a, I: AsRef<[u8]>, T: IntoIterator> Request<'a, T> {
 		let meta = &[];
 
 		// start an http request.
-		let id = runtime_io::http_request_start(self.method.as_ref(), self.url, meta).map_err(|_| HttpError::IoError)?;
+		let id = runtime_io::offchain::http_request_start(
+			self.method.as_ref(),
+			self.url,
+			meta,
+		).map_err(|_| HttpError::IoError)?;
 
 		// add custom headers
 		for header in &self.headers {
-			runtime_io::http_request_add_header(
+			runtime_io::offchain::http_request_add_header(
 				id,
 				header.name(),
 				header.value(),
@@ -234,11 +238,11 @@ impl<'a, I: AsRef<[u8]>, T: IntoIterator> Request<'a, T> {
 
 		// write body
 		for chunk in self.body {
-			runtime_io::http_request_write_body(id, chunk.as_ref(), self.deadline)?;
+			runtime_io::offchain::http_request_write_body(id, chunk.as_ref(), self.deadline)?;
 		}
 
 		// finalise the request
-		runtime_io::http_request_write_body(id, &[], self.deadline)?;
+		runtime_io::offchain::http_request_write_body(id, &[], self.deadline)?;
 
 		Ok(PendingRequest {
 			id,
@@ -303,7 +307,7 @@ impl PendingRequest {
 		deadline: impl Into>
 	) -> Vec> {
 		let ids = requests.iter().map(|r| r.id).collect::>();
-		let statuses = runtime_io::http_response_wait(&ids, deadline.into());
+		let statuses = runtime_io::offchain::http_response_wait(&ids, deadline.into());
 
 		statuses
 			.into_iter()
@@ -341,7 +345,9 @@ impl Response {
 	/// Retrieve the headers for this response.
 	pub fn headers(&mut self) -> &Headers {
 		if self.headers.is_none() {
-			self.headers = Some(Headers { raw: runtime_io::http_response_headers(self.id) });
+			self.headers = Some(
+				Headers { raw: runtime_io::offchain::http_response_headers(self.id) },
+			);
 		}
 		self.headers.as_ref().expect("Headers were just set; qed")
 	}
@@ -420,7 +426,10 @@ impl Iterator for ResponseBody {
 		}
 
 		if self.filled_up_to.is_none() {
-			let result = runtime_io::http_response_read_body(self.id, &mut self.buffer, self.deadline);
+			let result = runtime_io::offchain::http_response_read_body(
+				self.id,
+				&mut self.buffer,
+				self.deadline);
 			match result {
 				Err(e) => {
 					self.error = Some(e);
@@ -431,7 +440,7 @@ impl Iterator for ResponseBody {
 				}
 				Ok(size) => {
 					self.position = 0;
-					self.filled_up_to = Some(size);
+					self.filled_up_to = Some(size as usize);
 				}
 			}
 		}
diff --git a/core/sr-primitives/src/offchain/mod.rs b/primitives/sr-primitives/src/offchain/mod.rs
similarity index 100%
rename from core/sr-primitives/src/offchain/mod.rs
rename to primitives/sr-primitives/src/offchain/mod.rs
diff --git a/core/sr-primitives/src/testing.rs b/primitives/sr-primitives/src/testing.rs
similarity index 93%
rename from core/sr-primitives/src/testing.rs
rename to primitives/sr-primitives/src/testing.rs
index d60e58bab1a5c..bd3f673cd5c43 100644
--- a/core/sr-primitives/src/testing.rs
+++ b/primitives/sr-primitives/src/testing.rs
@@ -20,9 +20,11 @@ use serde::{Serialize, Serializer, Deserialize, de::Error as DeError, Deserializ
 use std::{fmt::Debug, ops::Deref, fmt, cell::RefCell};
 use crate::codec::{Codec, Encode, Decode};
 use crate::traits::{
-	self, Checkable, Applyable, BlakeTwo256, OpaqueKeys, ValidateUnsigned,
+	self, Checkable, Applyable, BlakeTwo256, OpaqueKeys,
 	SignedExtension, Dispatchable,
 };
+#[allow(deprecated)]
+use crate::traits::ValidateUnsigned;
 use crate::{generic, KeyTypeId, ApplyResult};
 use crate::weights::{GetDispatchInfo, DispatchInfo};
 pub use primitives::{H256, sr25519};
@@ -88,7 +90,7 @@ impl app_crypto::RuntimeAppPublic for UintAuthorityId {
 		ALL_KEYS.with(|l| l.borrow().clone())
 	}
 
-	fn generate_pair(_: Option<&str>) -> Self {
+	fn generate_pair(_: Option>) -> Self {
 		use rand::RngCore;
 		UintAuthorityId(rand::thread_rng().next_u64())
 	}
@@ -143,7 +145,7 @@ pub type DigestItem = generic::DigestItem;
 pub type Digest = generic::Digest;
 
 /// Block Header
-#[derive(PartialEq, Eq, Clone, Serialize, Debug, Encode, Decode)]
+#[derive(PartialEq, Eq, Clone, Serialize, Debug, Encode, Decode, Default)]
 #[serde(rename_all = "camelCase")]
 #[serde(deny_unknown_fields)]
 pub struct Header {
@@ -196,6 +198,16 @@ impl traits::Header for Header {
 	}
 }
 
+impl Header {
+	/// A new header with the given number and default hash for all other fields.
+	pub fn new_from_number(number: ::Number) -> Self {
+		Self {
+			number,
+			..Default::default()
+		}
+	}
+}
+
 impl<'a> Deserialize<'a> for Header {
 	fn deserialize>(de: D) -> Result {
 		let r = >::deserialize(de)?;
@@ -246,7 +258,9 @@ pub struct Block {
 	pub extrinsics: Vec,
 }
 
-impl traits::Block for Block {
+impl traits::Block
+	for Block
+{
 	type Extrinsic = Xt;
 	type Header = Header;
 	type Hash = 
::Hash;
@@ -323,6 +337,7 @@ impl Applyable for TestXt where
 	fn sender(&self) -> Option<&Self::AccountId> { self.0.as_ref().map(|x| &x.0) }
 
 	/// Checks to see if this is a valid *transaction*. It returns information on it if so.
+	#[allow(deprecated)] // Allow ValidateUnsigned
 	fn validate>(
 		&self,
 		_info: DispatchInfo,
@@ -333,7 +348,8 @@ impl Applyable for TestXt where
 
 	/// Executes all necessary logic needed prior to dispatch and deconstructs into function call,
 	/// index and sender.
-	fn apply(
+	#[allow(deprecated)] // Allow ValidateUnsigned
+	fn apply>(
 		self,
 		info: DispatchInfo,
 		len: usize,
diff --git a/core/sr-primitives/src/traits.rs b/primitives/sr-primitives/src/traits.rs
similarity index 94%
rename from core/sr-primitives/src/traits.rs
rename to primitives/sr-primitives/src/traits.rs
index ecaf6a78fcfe4..edddaf035aa95 100644
--- a/core/sr-primitives/src/traits.rs
+++ b/primitives/sr-primitives/src/traits.rs
@@ -85,21 +85,24 @@ pub trait Verify {
 impl Verify for primitives::ed25519::Signature {
 	type Signer = primitives::ed25519::Public;
 	fn verify>(&self, mut msg: L, signer: &primitives::ed25519::Public) -> bool {
-		runtime_io::ed25519_verify(self, msg.get(), signer)
+		runtime_io::crypto::ed25519_verify(self, msg.get(), signer)
 	}
 }
 
 impl Verify for primitives::sr25519::Signature {
 	type Signer = primitives::sr25519::Public;
 	fn verify>(&self, mut msg: L, signer: &primitives::sr25519::Public) -> bool {
-		runtime_io::sr25519_verify(self, msg.get(), signer)
+		runtime_io::crypto::sr25519_verify(self, msg.get(), signer)
 	}
 }
 
 impl Verify for primitives::ecdsa::Signature {
 	type Signer = primitives::ecdsa::Public;
 	fn verify>(&self, mut msg: L, signer: &primitives::ecdsa::Public) -> bool {
-		match runtime_io::secp256k1_ecdsa_recover_compressed(self.as_ref(), &runtime_io::blake2_256(msg.get())) {
+		match runtime_io::crypto::secp256k1_ecdsa_recover_compressed(
+			self.as_ref(),
+			&runtime_io::hashing::blake2_256(msg.get()),
+		) {
 			Ok(pubkey) => >::as_ref(signer) == &pubkey[..],
 			_ => false,
 		}
@@ -399,23 +402,23 @@ impl Hash for BlakeTwo256 {
 	type Output = primitives::H256;
 	type Hasher = Blake2Hasher;
 	fn hash(s: &[u8]) -> Self::Output {
-		runtime_io::blake2_256(s).into()
+		runtime_io::hashing::blake2_256(s).into()
 	}
 
 	fn trie_root(input: Vec<(Vec, Vec)>) -> Self::Output {
-		runtime_io::blake2_256_trie_root(input)
+		runtime_io::storage::blake2_256_trie_root(input)
 	}
 
 	fn ordered_trie_root(input: Vec>) -> Self::Output {
-		runtime_io::blake2_256_ordered_trie_root(input)
+		runtime_io::storage::blake2_256_ordered_trie_root(input)
 	}
 
 	fn storage_root() -> Self::Output {
-		runtime_io::storage_root().into()
+		runtime_io::storage::root().into()
 	}
 
 	fn storage_changes_root(parent_hash: Self::Output) -> Option {
-		runtime_io::storage_changes_root(parent_hash.into()).map(Into::into)
+		runtime_io::storage::changes_root(parent_hash.into()).map(Into::into)
 	}
 }
 
@@ -755,9 +758,6 @@ pub trait SignedExtension: Codec + Debug + Sync + Send + Clone + Eq + PartialEq
 
 	/// Validate an unsigned transaction for the transaction queue.
 	///
-	/// Normally the default implementation is fine since `ValidateUnsigned`
-	/// is a better way of recognising and validating unsigned transactions.
-	///
 	/// This function can be called frequently by the transaction queue,
 	/// to obtain transaction validity against current state.
 	/// It should perform all checks that determine a valid unsigned transaction,
@@ -889,6 +889,7 @@ pub trait Applyable: Sized + Send + Sync {
 	fn sender(&self) -> Option<&Self::AccountId>;
 
 	/// Checks to see if this is a valid *transaction*. It returns information on it if so.
+	#[allow(deprecated)] // Allow ValidateUnsigned
 	fn validate>(
 		&self,
 		info: DispatchInfo,
@@ -897,7 +898,8 @@ pub trait Applyable: Sized + Send + Sync {
 
 	/// Executes all necessary logic needed prior to dispatch and deconstructs into function call,
 	/// index and sender.
-	fn apply(
+	#[allow(deprecated)] // Allow ValidateUnsigned
+	fn apply>(
 		self,
 		info: DispatchInfo,
 		len: usize,
@@ -966,10 +968,27 @@ pub trait RuntimeApiInfo {
 /// the transaction for the transaction pool.
 /// During block execution phase one need to perform the same checks anyway,
 /// since this function is not being called.
+#[deprecated(note = "Use SignedExtensions instead.")]
 pub trait ValidateUnsigned {
 	/// The call to validate
 	type Call;
 
+	/// Validate the call right before dispatch.
+	///
+	/// This method should be used to prevent transactions already in the pool
+	/// (i.e. passing `validate_unsigned`) from being included in blocks
+	/// in case we know they now became invalid.
+	///
+	/// By default it's a good idea to call `validate_unsigned` from within
+	/// this function again to make sure we never include an invalid transaction.
+	///
+	/// Changes made to storage WILL be persisted if the call returns `Ok`.
+	fn pre_dispatch(call: &Self::Call) -> Result<(), crate::ApplyError> {
+		Self::validate_unsigned(call)
+			.map(|_| ())
+			.map_err(Into::into)
+	}
+
 	/// Return the validity of the call
 	///
 	/// This doesn't execute any side-effects; it merely checks
@@ -1136,14 +1155,14 @@ macro_rules! impl_opaque_keys {
 			/// The generated key pairs are stored in the keystore.
 			///
 			/// Returns the concatenated SCALE encoded public keys.
-			pub fn generate(seed: Option<&str>) -> $crate::rstd::vec::Vec {
+			pub fn generate(seed: Option<$crate::rstd::vec::Vec>) -> $crate::rstd::vec::Vec {
 				let keys = Self{
 					$(
 						$field: <
 							<
 								$type as $crate::BoundToRuntimeAppPublic
 							>::Public as $crate::RuntimeAppPublic
-						>::generate_pair(seed),
+						>::generate_pair(seed.clone()),
 					)*
 				};
 				$crate::codec::Encode::encode(&keys)
@@ -1208,19 +1227,19 @@ impl Printable for usize {
 
 impl Printable for u64 {
 	fn print(&self) {
-		runtime_io::print_num(*self);
+		runtime_io::misc::print_num(*self);
 	}
 }
 
 impl Printable for &[u8] {
 	fn print(&self) {
-		runtime_io::print_hex(self);
+		runtime_io::misc::print_hex(self);
 	}
 }
 
 impl Printable for &str {
 	fn print(&self) {
-		runtime_io::print_utf8(self.as_bytes());
+		runtime_io::misc::print_utf8(self.as_bytes());
 	}
 }
 
@@ -1231,6 +1250,25 @@ impl Printable for Tuple {
 	}
 }
 
+/// Something that can convert a [`BlockId`] to a number or a hash.
+#[cfg(feature = "std")]
+pub trait BlockIdTo {
+	/// The error type that will be returned by the functions.
+	type Error: std::fmt::Debug;
+
+	/// Convert the given `block_id` to the corresponding block hash.
+	fn to_hash(
+		&self,
+		block_id: &crate::generic::BlockId,
+	) -> Result, Self::Error>;
+
+	/// Convert the given `block_id` to the corresponding block number.
+	fn to_number(
+		&self,
+		block_id: &crate::generic::BlockId,
+	) -> Result>, Self::Error>;
+}
+
 #[cfg(test)]
 mod tests {
 	use super::AccountIdConversion;
diff --git a/core/sr-primitives/src/transaction_validity.rs b/primitives/sr-primitives/src/transaction_validity.rs
similarity index 100%
rename from core/sr-primitives/src/transaction_validity.rs
rename to primitives/sr-primitives/src/transaction_validity.rs
diff --git a/core/sr-primitives/src/weights.rs b/primitives/sr-primitives/src/weights.rs
similarity index 84%
rename from core/sr-primitives/src/weights.rs
rename to primitives/sr-primitives/src/weights.rs
index 088f13244eb31..6f2a8676798a4 100644
--- a/core/sr-primitives/src/weights.rs
+++ b/primitives/sr-primitives/src/weights.rs
@@ -37,8 +37,9 @@
 
 #[cfg(feature = "std")]
 use serde::{Serialize, Deserialize};
+use impl_trait_for_tuples::impl_for_tuples;
 use codec::{Encode, Decode};
-use arithmetic::traits::Bounded;
+use arithmetic::traits::{Bounded, Zero};
 use crate::RuntimeDebug;
 
 /// Re-export priority as type
@@ -62,6 +63,35 @@ pub trait ClassifyDispatch {
 	fn classify_dispatch(&self, target: T) -> DispatchClass;
 }
 
+/// Means of determining the weight of a block's lifecycle hooks: on_initialize, on_finalize and
+/// such.
+pub trait WeighBlock {
+	/// Return the weight of the block's on_initialize hook.
+	fn on_initialize(_: BlockNumber) -> Weight { Zero::zero() }
+	/// Return the weight of the block's on_finalize hook.
+	fn on_finalize(_: BlockNumber) -> Weight { Zero::zero() }
+}
+
+/// Maybe I can do something to remove the duplicate code here.
+#[impl_for_tuples(30)]
+impl WeighBlock for SingleModule {
+	fn on_initialize(n: BlockNumber) -> Weight {
+		let mut accumulated_weight: Weight = Zero::zero();
+		for_tuples!(
+			#( accumulated_weight = accumulated_weight.saturating_add(SingleModule::on_initialize(n)); )*
+		);
+		accumulated_weight
+	}
+
+	fn on_finalize(n: BlockNumber) -> Weight {
+		let mut accumulated_weight: Weight = Zero::zero();
+		for_tuples!(
+			#( accumulated_weight = accumulated_weight.saturating_add(SingleModule::on_finalize(n)); )*
+		);
+		accumulated_weight
+	}
+}
+
 /// A generalized group of dispatch types. This is only distinguishing normal, user-triggered transactions
 /// (`Normal`) and anything beyond which serves a higher purpose to the system (`Operational`).
 #[cfg_attr(feature = "std", derive(Serialize, Deserialize))]
@@ -181,3 +211,10 @@ impl Default for SimpleDispatchInfo {
 		SimpleDispatchInfo::FixedNormal(10_000)
 	}
 }
+
+impl SimpleDispatchInfo {
+	/// An _additive zero_ variant of SimpleDispatchInfo.
+	pub fn zero() -> Self {
+		Self::FixedNormal(0)
+	}
+}
diff --git a/core/sr-sandbox/Cargo.toml b/primitives/sr-sandbox/Cargo.toml
similarity index 65%
rename from core/sr-sandbox/Cargo.toml
rename to primitives/sr-sandbox/Cargo.toml
index 87b4e742a0413..97d391d794d65 100755
--- a/core/sr-sandbox/Cargo.toml
+++ b/primitives/sr-sandbox/Cargo.toml
@@ -2,16 +2,13 @@
 name = "sr-sandbox"
 version = "2.0.0"
 authors = ["Parity Technologies "]
-build = "build.rs"
 edition = "2018"
 
-[build-dependencies]
-rustc_version = "0.2.3"
-
 [dependencies]
-wasmi = { version = "0.5.1", optional = true }
-primitives = { package = "substrate-primitives", path = "../primitives", default-features = false }
+wasmi = { version = "0.6.2", optional = true }
+primitives = { package = "substrate-primitives", path = "../core", default-features = false }
 rstd = { package = "sr-std", path = "../sr-std", default-features = false }
+runtime-io = { package = "sr-io", path = "../sr-io", default-features = false }
 codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false }
 
 [dev-dependencies]
@@ -25,6 +22,6 @@ std = [
 	"primitives/std",
 	"rstd/std",
 	"codec/std",
+	"runtime-io/std",
 ]
-nightly = []
 strict = []
diff --git a/core/sr-sandbox/src/lib.rs b/primitives/sr-sandbox/src/lib.rs
similarity index 98%
rename from core/sr-sandbox/src/lib.rs
rename to primitives/sr-sandbox/src/lib.rs
index c9f9135661586..17b2cb7c1ebda 100755
--- a/core/sr-sandbox/src/lib.rs
+++ b/primitives/sr-sandbox/src/lib.rs
@@ -178,7 +178,9 @@ impl Instance {
 	/// be returned.
 	///
 	/// [`EnvironmentDefinitionBuilder`]: struct.EnvironmentDefinitionBuilder.html
-	pub fn new(code: &[u8], env_def_builder: &EnvironmentDefinitionBuilder, state: &mut T) -> Result, Error> {
+	pub fn new(code: &[u8], env_def_builder: &EnvironmentDefinitionBuilder, state: &mut T)
+		-> Result, Error>
+	{
 		Ok(Instance {
 			inner: imp::Instance::new(code, &env_def_builder.inner, state)?,
 		})
@@ -197,7 +199,7 @@ impl Instance {
 	/// - Trap occured at the execution time.
 	pub fn invoke(
 		&mut self,
-		name: &[u8],
+		name: &str,
 		args: &[TypedValue],
 		state: &mut T,
 	) -> Result {
diff --git a/core/sr-sandbox/with_std.rs b/primitives/sr-sandbox/with_std.rs
similarity index 97%
rename from core/sr-sandbox/with_std.rs
rename to primitives/sr-sandbox/with_std.rs
index ea7ce818350d0..afc092686eeca 100755
--- a/core/sr-sandbox/with_std.rs
+++ b/primitives/sr-sandbox/with_std.rs
@@ -257,7 +257,11 @@ pub struct Instance {
 }
 
 impl Instance {
-	pub fn new(code: &[u8], env_def_builder: &EnvironmentDefinitionBuilder, state: &mut T) -> Result, Error> {
+	pub fn new(
+		code: &[u8],
+		env_def_builder: &EnvironmentDefinitionBuilder,
+		state: &mut T,
+	) -> Result, Error> {
 		let module = Module::from_buffer(code).map_err(|_| Error::Module)?;
 		let not_started_instance = ModuleInstance::new(&module, env_def_builder)
 			.map_err(|_| Error::Module)?;
@@ -269,7 +273,8 @@ impl Instance {
 				state,
 				defined_host_functions: &defined_host_functions,
 			};
-			let instance = not_started_instance.run_start(&mut externals).map_err(|_| Error::Execution)?;
+			let instance = not_started_instance.run_start(&mut externals)
+				.map_err(|_| Error::Execution)?;
 			instance
 		};
 
@@ -282,13 +287,12 @@ impl Instance {
 
 	pub fn invoke(
 		&mut self,
-		name: &[u8],
+		name: &str,
 		args: &[TypedValue],
 		state: &mut T,
 	) -> Result {
 		let args = args.iter().cloned().map(Into::into).collect::>();
 
-		let name = ::std::str::from_utf8(name).map_err(|_| Error::Execution)?;
 		let mut externals = GuestExternals {
 			state,
 			defined_host_functions: &self.defined_host_functions,
@@ -350,7 +354,7 @@ mod tests {
 		env_builder.add_host_func("env", "polymorphic_id", env_polymorphic_id);
 
 		let mut instance = Instance::new(code, &env_builder, &mut state)?;
-		let result = instance.invoke(b"call", args, &mut state);
+		let result = instance.invoke("call", args, &mut state);
 
 		result.map_err(|_| HostError)
 	}
@@ -474,7 +478,7 @@ mod tests {
 
 		// But this fails since we imported a function that returns i32 as if it returned i64.
 		assert_matches!(
-			instance.invoke(b"call", &[], &mut ()),
+			instance.invoke("call", &[], &mut ()),
 			Err(Error::Execution)
 		);
 	}
diff --git a/core/sr-sandbox/without_std.rs b/primitives/sr-sandbox/without_std.rs
similarity index 74%
rename from core/sr-sandbox/without_std.rs
rename to primitives/sr-sandbox/without_std.rs
index ee5f7697fe71d..d7fffbf88b27f 100755
--- a/core/sr-sandbox/without_std.rs
+++ b/primitives/sr-sandbox/without_std.rs
@@ -14,12 +14,11 @@
 // You should have received a copy of the GNU General Public License
 // along with Substrate.  If not, see .
 
-use rstd::prelude::*;
-use rstd::{slice, marker, mem, vec};
-use rstd::rc::Rc;
+use rstd::{prelude::*, slice, marker, mem, vec, rc::Rc};
 use codec::{Decode, Encode};
 use primitives::sandbox as sandbox_primitives;
 use super::{Error, TypedValue, ReturnValue, HostFuncType};
+use runtime_io::sandbox;
 
 mod ffi {
 	use rstd::mem;
@@ -43,51 +42,6 @@ mod ffi {
 		assert!(mem::size_of::() == mem::size_of::>());
 		mem::transmute::>(idx)
 	}
-
-	extern "C" {
-		pub fn ext_sandbox_instantiate(
-			dispatch_thunk: extern "C" fn(
-				serialized_args_ptr: *const u8,
-				serialized_args_len: usize,
-				state: usize,
-				f: HostFuncIndex,
-			) -> u64,
-			wasm_ptr: *const u8,
-			wasm_len: usize,
-			imports_ptr: *const u8,
-			imports_len: usize,
-			state: usize,
-		) -> u32;
-		pub fn ext_sandbox_invoke(
-			instance_idx: u32,
-			export_ptr: *const u8,
-			export_len: usize,
-			args_ptr: *const u8,
-			args_len: usize,
-			return_val_ptr: *mut u8,
-			return_val_len: usize,
-			state: usize,
-		) -> u32;
-		pub fn ext_sandbox_memory_new(initial: u32, maximum: u32) -> u32;
-		pub fn ext_sandbox_memory_get(
-			memory_idx: u32,
-			offset: u32,
-			buf_ptr: *mut u8,
-			buf_len: usize,
-		) -> u32;
-		pub fn ext_sandbox_memory_set(
-			memory_idx: u32,
-			offset: u32,
-			val_ptr: *const u8,
-			val_len: usize,
-		) -> u32;
-		pub fn ext_sandbox_memory_teardown(
-			memory_idx: u32,
-		);
-		pub fn ext_sandbox_instance_teardown(
-			instance_idx: u32,
-		);
-	}
 }
 
 struct MemoryHandle {
@@ -96,9 +50,7 @@ struct MemoryHandle {
 
 impl Drop for MemoryHandle {
 	fn drop(&mut self) {
-		unsafe {
-			ffi::ext_sandbox_memory_teardown(self.memory_idx);
-		}
+		sandbox::memory_teardown(self.memory_idx);
 	}
 }
 
@@ -111,15 +63,13 @@ pub struct Memory {
 
 impl Memory {
 	pub fn new(initial: u32, maximum: Option) -> Result {
-		let result = unsafe {
-			let maximum = if let Some(maximum) = maximum {
-				maximum
-			} else {
-				sandbox_primitives::MEM_UNLIMITED
-			};
-			ffi::ext_sandbox_memory_new(initial, maximum)
+		let maximum = if let Some(maximum) = maximum {
+			maximum
+		} else {
+			sandbox_primitives::MEM_UNLIMITED
 		};
-		match result {
+
+		match sandbox::memory_new(initial, maximum) {
 			sandbox_primitives::ERR_MODULE => Err(Error::Module),
 			memory_idx => Ok(Memory {
 				handle: Rc::new(MemoryHandle { memory_idx, }),
@@ -128,7 +78,12 @@ impl Memory {
 	}
 
 	pub fn get(&self, offset: u32, buf: &mut [u8]) -> Result<(), Error> {
-		let result = unsafe { ffi::ext_sandbox_memory_get(self.handle.memory_idx, offset, buf.as_mut_ptr(), buf.len()) };
+		let result = sandbox::memory_get(
+			self.handle.memory_idx,
+			offset,
+			buf.as_mut_ptr(),
+			buf.len() as u32,
+		);
 		match result {
 			sandbox_primitives::ERR_OK => Ok(()),
 			sandbox_primitives::ERR_OUT_OF_BOUNDS => Err(Error::OutOfBounds),
@@ -137,7 +92,12 @@ impl Memory {
 	}
 
 	pub fn set(&self, offset: u32, val: &[u8]) -> Result<(), Error> {
-		let result = unsafe { ffi::ext_sandbox_memory_set(self.handle.memory_idx, offset, val.as_ptr(), val.len()) };
+		let result = sandbox::memory_set(
+			self.handle.memory_idx,
+			offset,
+			val.as_ptr() as _ ,
+			val.len() as u32,
+		);
 		match result {
 			sandbox_primitives::ERR_OK => Ok(()),
 			sandbox_primitives::ERR_OUT_OF_BOUNDS => Err(Error::OutOfBounds),
@@ -251,26 +211,27 @@ extern "C" fn dispatch_thunk(
 }
 
 impl Instance {
-	pub fn new(code: &[u8], env_def_builder: &EnvironmentDefinitionBuilder, state: &mut T) -> Result, Error> {
+	pub fn new(
+		code: &[u8],
+		env_def_builder: &EnvironmentDefinitionBuilder,
+		state: &mut T,
+	) -> Result, Error> {
 		let serialized_env_def: Vec = env_def_builder.env_def.encode();
-		let result = unsafe {
-			// It's very important to instantiate thunk with the right type.
-			let dispatch_thunk = dispatch_thunk::;
-
-			ffi::ext_sandbox_instantiate(
-				dispatch_thunk,
-				code.as_ptr(),
-				code.len(),
-				serialized_env_def.as_ptr(),
-				serialized_env_def.len(),
-				state as *const T as usize,
-			)
-		};
+		// It's very important to instantiate thunk with the right type.
+		let dispatch_thunk = dispatch_thunk::;
+		let result = sandbox::instantiate(
+			dispatch_thunk as u32,
+			code,
+			&serialized_env_def,
+			state as *const T as _,
+		);
+
 		let instance_idx = match result {
 			sandbox_primitives::ERR_MODULE => return Err(Error::Module),
 			sandbox_primitives::ERR_EXECUTION => return Err(Error::Execution),
 			instance_idx => instance_idx,
 		};
+
 		// We need to retain memories to keep them alive while the Instance is alive.
 		let retained_memories = env_def_builder.retained_memories.clone();
 		Ok(Instance {
@@ -282,25 +243,22 @@ impl Instance {
 
 	pub fn invoke(
 		&mut self,
-		name: &[u8],
+		name: &str,
 		args: &[TypedValue],
 		state: &mut T,
 	) -> Result {
 		let serialized_args = args.to_vec().encode();
 		let mut return_val = vec![0u8; sandbox_primitives::ReturnValue::ENCODED_MAX_SIZE];
 
-		let result = unsafe {
-			ffi::ext_sandbox_invoke(
-				self.instance_idx,
-				name.as_ptr(),
-				name.len(),
-				serialized_args.as_ptr(),
-				serialized_args.len(),
-				return_val.as_mut_ptr(),
-				return_val.len(),
-				state as *const T as usize,
-			)
-		};
+		let result = sandbox::invoke(
+			self.instance_idx,
+			name,
+			&serialized_args,
+			return_val.as_mut_ptr() as _,
+			return_val.len() as u32,
+			state as *const T as _,
+		);
+
 		match result {
 			sandbox_primitives::ERR_OK => {
 				let return_val = sandbox_primitives::ReturnValue::decode(&mut &return_val[..])
@@ -315,8 +273,6 @@ impl Instance {
 
 impl Drop for Instance {
 	fn drop(&mut self) {
-		unsafe {
-			ffi::ext_sandbox_instance_teardown(self.instance_idx);
-		}
+		sandbox::instance_teardown(self.instance_idx);
 	}
 }
diff --git a/core/sr-staking-primitives/Cargo.toml b/primitives/sr-staking-primitives/Cargo.toml
similarity index 100%
rename from core/sr-staking-primitives/Cargo.toml
rename to primitives/sr-staking-primitives/Cargo.toml
diff --git a/core/sr-staking-primitives/src/lib.rs b/primitives/sr-staking-primitives/src/lib.rs
similarity index 100%
rename from core/sr-staking-primitives/src/lib.rs
rename to primitives/sr-staking-primitives/src/lib.rs
diff --git a/core/sr-staking-primitives/src/offence.rs b/primitives/sr-staking-primitives/src/offence.rs
similarity index 100%
rename from core/sr-staking-primitives/src/offence.rs
rename to primitives/sr-staking-primitives/src/offence.rs
diff --git a/core/sr-std/Cargo.toml b/primitives/sr-std/Cargo.toml
similarity index 56%
rename from core/sr-std/Cargo.toml
rename to primitives/sr-std/Cargo.toml
index 2a8b7d37ca272..77021af935ae1 100644
--- a/core/sr-std/Cargo.toml
+++ b/primitives/sr-std/Cargo.toml
@@ -2,15 +2,8 @@
 name = "sr-std"
 version = "2.0.0"
 authors = ["Parity Technologies "]
-build = "build.rs"
 edition = "2018"
 
-[build-dependencies]
-rustc_version = "0.2.3"
-
 [features]
 default = ["std"]
 std = []
-nightly = []
-strict = []
-no_global_allocator = []
diff --git a/core/sr-std/src/lib.rs b/primitives/sr-std/src/lib.rs
similarity index 86%
rename from core/sr-std/src/lib.rs
rename to primitives/sr-std/src/lib.rs
index f369d3908ba83..5aa8e82235247 100644
--- a/core/sr-std/src/lib.rs
+++ b/primitives/sr-std/src/lib.rs
@@ -15,13 +15,15 @@
 // along with Substrate.  If not, see .
 
 //! Lowest-abstraction level for the Substrate runtime: just exports useful primitives from std
-//! or core/alloc to be used with any code that depends on the runtime.
+//! or client/alloc to be used with any code that depends on the runtime.
 
 #![cfg_attr(not(feature = "std"), no_std)]
 #![cfg_attr(not(feature = "std"), feature(core_intrinsics))]
 
-#![cfg_attr(feature = "std", doc = "Substrate runtime standard library as compiled when linked with Rust's standard library.")]
-#![cfg_attr(not(feature = "std"), doc = "Substrate's runtime standard library as compiled without Rust's standard library.")]
+#![cfg_attr(feature = "std",
+   doc = "Substrate runtime standard library as compiled when linked with Rust's standard library.")]
+#![cfg_attr(not(feature = "std"),
+   doc = "Substrate's runtime standard library as compiled without Rust's standard library.")]
 
 #[macro_export]
 macro_rules! map {
diff --git a/core/sr-std/with_std.rs b/primitives/sr-std/with_std.rs
similarity index 97%
rename from core/sr-std/with_std.rs
rename to primitives/sr-std/with_std.rs
index 0d5ee04ed51ab..e41a9d7ad6c5c 100644
--- a/core/sr-std/with_std.rs
+++ b/primitives/sr-std/with_std.rs
@@ -14,6 +14,8 @@
 // You should have received a copy of the GNU General Public License
 // along with Substrate.  If not, see .
 
+pub use std::alloc;
+pub use std::any;
 pub use std::borrow;
 pub use std::boxed;
 pub use std::cell;
diff --git a/core/sr-std/without_std.rs b/primitives/sr-std/without_std.rs
similarity index 73%
rename from core/sr-std/without_std.rs
rename to primitives/sr-std/without_std.rs
index 9762c74367198..a35e1395a543e 100755
--- a/core/sr-std/without_std.rs
+++ b/primitives/sr-std/without_std.rs
@@ -14,40 +14,12 @@
 // You should have received a copy of the GNU General Public License
 // along with Substrate.  If not, see .
 
-#[doc(hidden)]
 pub extern crate alloc;
 
-extern "C" {
-	fn ext_malloc(size: u32) -> *mut u8;
-	fn ext_free(ptr: *mut u8);
-}
-
-/// Wasm allocator
-pub struct WasmAllocator;
-
-#[cfg(not(feature = "no_global_allocator"))]
-#[global_allocator]
-static ALLOCATOR: WasmAllocator = WasmAllocator;
-
-mod __impl {
-	use core::alloc::{GlobalAlloc, Layout};
-
-	use super::WasmAllocator;
-
-	unsafe impl GlobalAlloc for WasmAllocator {
-		unsafe fn alloc(&self, layout: Layout) -> *mut u8 {
-			super::ext_malloc(layout.size() as u32) as *mut u8
-		}
-
-		unsafe fn dealloc(&self, ptr: *mut u8, _layout: Layout) {
-			super::ext_free(ptr as *mut u8)
-		}
-	}
-}
-
 pub use alloc::boxed;
 pub use alloc::rc;
 pub use alloc::vec;
+pub use core::any;
 pub use core::cell;
 pub use core::clone;
 pub use core::cmp;
diff --git a/core/sr-version/Cargo.toml b/primitives/sr-version/Cargo.toml
similarity index 100%
rename from core/sr-version/Cargo.toml
rename to primitives/sr-version/Cargo.toml
diff --git a/core/sr-version/src/lib.rs b/primitives/sr-version/src/lib.rs
similarity index 99%
rename from core/sr-version/src/lib.rs
rename to primitives/sr-version/src/lib.rs
index 24c54a739a2e0..f342c25217926 100644
--- a/core/sr-version/src/lib.rs
+++ b/primitives/sr-version/src/lib.rs
@@ -49,7 +49,7 @@ pub type ApisVec = &'static [(ApiId, u32)];
 #[macro_export]
 #[cfg(feature = "std")]
 macro_rules! create_apis_vec {
-	( $y:expr ) => { ::std::borrow::Cow::Borrowed(& $y) }
+	( $y:expr ) => { std::borrow::Cow::Borrowed(& $y) }
 }
 #[macro_export]
 #[cfg(not(feature = "std"))]
diff --git a/core/state-machine/Cargo.toml b/primitives/state-machine/Cargo.toml
similarity index 90%
rename from core/state-machine/Cargo.toml
rename to primitives/state-machine/Cargo.toml
index 7cd8601a3b636..3f02056647a00 100644
--- a/core/state-machine/Cargo.toml
+++ b/primitives/state-machine/Cargo.toml
@@ -12,7 +12,7 @@ hash-db = "0.15.2"
 trie-db = "0.15.2"
 trie-root = "0.15.2"
 trie = { package = "substrate-trie", path = "../trie" }
-primitives = { package = "substrate-primitives", path = "../primitives" }
+primitives = { package = "substrate-primitives", path = "../core" }
 panic-handler = { package = "substrate-panic-handler", path = "../panic-handler" }
 codec = { package = "parity-scale-codec", version = "1.0.0" }
 num-traits = "0.2.8"
diff --git a/core/state-machine/src/backend.rs b/primitives/state-machine/src/backend.rs
similarity index 94%
rename from core/state-machine/src/backend.rs
rename to primitives/state-machine/src/backend.rs
index e2f398ef7ccae..5fbda1aa32f45 100644
--- a/core/state-machine/src/backend.rs
+++ b/primitives/state-machine/src/backend.rs
@@ -246,10 +246,11 @@ impl error::Error for Void {
 	fn description(&self) -> &str { "unreachable error" }
 }
 
-/// In-memory backend. Fully recomputes tries on each commit but useful for
-/// tests.
+/// In-memory backend. Fully recomputes tries each time `as_trie_backend` is called but useful for
+/// tests and proof checking.
 pub struct InMemory {
 	inner: HashMap>, HashMap, Vec>>,
+	// This field is only needed for returning reference in `as_trie_backend`.
 	trie: Option, H>>,
 	_hasher: PhantomData,
 }
@@ -467,7 +468,6 @@ impl Backend for InMemory {
 
 	fn as_trie_backend(&mut self)-> Option<&TrieBackend> {
 		let mut mdb = MemoryDB::default();
-		let mut root = None;
 		let mut new_child_roots = Vec::new();
 		let mut root_map = None;
 		for (storage_key, map) in &self.inner {
@@ -478,16 +478,15 @@ impl Backend for InMemory {
 				root_map = Some(map);
 			}
 		}
-		// root handling
-		if let Some(map) = root_map.take() {
-			root = Some(insert_into_memory_db::(
+		let root = match root_map {
+			Some(map) => insert_into_memory_db::(
 				&mut mdb,
-				map.clone().into_iter().chain(new_child_roots.into_iter())
-			)?);
-		}
-		let root = match root {
-			Some(root) => root,
-			None => insert_into_memory_db::(&mut mdb, ::std::iter::empty())?,
+				map.clone().into_iter().chain(new_child_roots.into_iter()),
+			)?,
+			None => insert_into_memory_db::(
+				&mut mdb,
+				new_child_roots.into_iter(),
+			)?,
 		};
 		self.trie = Some(TrieBackend::new(mdb, root));
 		self.trie.as_ref()
@@ -513,3 +512,20 @@ pub(crate) fn insert_into_memory_db(mdb: &mut MemoryDB, input: I) -> Op
 
 	Some(root)
 }
+
+#[cfg(test)]
+mod tests {
+	use super::*;
+
+	/// Assert in memory backend with only child trie keys works as trie backend.
+	#[test]
+	fn in_memory_with_child_trie_only() {
+		let storage = InMemory::::default();
+		let mut storage = storage.update(
+			vec![(Some(b"1".to_vec()), b"2".to_vec(), Some(b"3".to_vec()))]
+		);
+		let trie_backend = storage.as_trie_backend().unwrap();
+		assert_eq!(trie_backend.child_storage(b"1", b"2").unwrap(), Some(b"3".to_vec()));
+		assert!(trie_backend.storage(b"1").unwrap().is_some());
+	}
+}
diff --git a/core/state-machine/src/basic.rs b/primitives/state-machine/src/basic.rs
similarity index 89%
rename from core/state-machine/src/basic.rs
rename to primitives/state-machine/src/basic.rs
index c2d1a0e3950d0..e758b3dd3b814 100644
--- a/core/state-machine/src/basic.rs
+++ b/primitives/state-machine/src/basic.rs
@@ -22,7 +22,10 @@ use hash_db::Hasher;
 use trie::{TrieConfiguration, default_child_trie_root};
 use trie::trie_types::Layout;
 use primitives::{
-	storage::{well_known_keys::is_child_storage_key, ChildStorageKey},
+	storage::{
+		well_known_keys::is_child_storage_key, ChildStorageKey, StorageOverlay,
+		ChildrenStorageOverlay
+	},
 	traits::Externalities, Blake2Hasher, hash::H256,
 };
 use log::warn;
@@ -30,16 +33,13 @@ use log::warn;
 /// Simple HashMap-based Externalities impl.
 #[derive(Debug)]
 pub struct BasicExternalities {
-	top: HashMap, Vec>,
-	children: HashMap, HashMap, Vec>>,
+	top: StorageOverlay,
+	children: ChildrenStorageOverlay,
 }
 
 impl BasicExternalities {
 	/// Create a new instance of `BasicExternalities`
-	pub fn new(
-		top: HashMap, Vec>,
-		children: HashMap, HashMap, Vec>>,
-	) -> Self {
+	pub fn new(top: StorageOverlay, children: ChildrenStorageOverlay) -> Self {
 		BasicExternalities {
 			top,
 			children,
@@ -58,6 +58,32 @@ impl BasicExternalities {
 	) {
 		(self.top, self.children)
 	}
+
+	/// Execute the given closure `f` with the externalities set and initialized with `storage`.
+	///
+	/// Returns the result of the closure and updates `storage` with all changes.
+	pub fn execute_with_storage(
+		storage: &mut (StorageOverlay, ChildrenStorageOverlay),
+		f: impl FnOnce() -> R,
+	) -> R {
+		let mut ext = Self {
+			top: storage.0.drain().collect(),
+			children: storage.1.drain().collect(),
+		};
+
+		let r = ext.execute_with(f);
+
+		*storage = ext.into_storages();
+
+		r
+	}
+
+	/// Execute the given closure while `self` is set as externalities.
+	///
+	/// Returns the result of the given closure.
+	pub fn execute_with(&mut self, f: impl FnOnce() -> R) -> R {
+		externalities::set_and_run_with_externalities(self, f)
+	}
 }
 
 impl PartialEq for BasicExternalities {
diff --git a/core/state-machine/src/changes_trie/build.rs b/primitives/state-machine/src/changes_trie/build.rs
similarity index 100%
rename from core/state-machine/src/changes_trie/build.rs
rename to primitives/state-machine/src/changes_trie/build.rs
diff --git a/core/state-machine/src/changes_trie/build_cache.rs b/primitives/state-machine/src/changes_trie/build_cache.rs
similarity index 100%
rename from core/state-machine/src/changes_trie/build_cache.rs
rename to primitives/state-machine/src/changes_trie/build_cache.rs
diff --git a/core/state-machine/src/changes_trie/build_iterator.rs b/primitives/state-machine/src/changes_trie/build_iterator.rs
similarity index 100%
rename from core/state-machine/src/changes_trie/build_iterator.rs
rename to primitives/state-machine/src/changes_trie/build_iterator.rs
diff --git a/core/state-machine/src/changes_trie/changes_iterator.rs b/primitives/state-machine/src/changes_trie/changes_iterator.rs
similarity index 99%
rename from core/state-machine/src/changes_trie/changes_iterator.rs
rename to primitives/state-machine/src/changes_trie/changes_iterator.rs
index e8730a1bddedb..2f61b87fe2199 100644
--- a/core/state-machine/src/changes_trie/changes_iterator.rs
+++ b/primitives/state-machine/src/changes_trie/changes_iterator.rs
@@ -28,7 +28,7 @@ use crate::changes_trie::input::{DigestIndex, ExtrinsicIndex, DigestIndexValue,
 use crate::changes_trie::storage::{TrieBackendAdapter, InMemoryStorage};
 use crate::changes_trie::input::ChildIndex;
 use crate::changes_trie::surface_iterator::{surface_iterator, SurfaceIterator};
-use crate::proving_backend::ProvingBackendEssence;
+use crate::proving_backend::ProvingBackendRecorder;
 use crate::trie_backend_essence::{TrieBackendEssence};
 
 /// Return changes of given key at given blocks range.
@@ -366,7 +366,7 @@ impl<'a, H, Number> Iterator for ProvingDrilldownIterator<'a, H, Number>
 		let proof_recorder = &mut *self.proof_recorder.try_borrow_mut()
 			.expect("only fails when already borrowed; storage() is non-reentrant; qed");
 		self.essence.next(|storage, root, key|
-			ProvingBackendEssence::<_, H> {
+			ProvingBackendRecorder::<_, H> {
 				backend: &TrieBackendEssence::new(TrieBackendAdapter::new(storage), root),
 				proof_recorder,
 			}.storage(key))
diff --git a/core/state-machine/src/changes_trie/input.rs b/primitives/state-machine/src/changes_trie/input.rs
similarity index 100%
rename from core/state-machine/src/changes_trie/input.rs
rename to primitives/state-machine/src/changes_trie/input.rs
diff --git a/core/state-machine/src/changes_trie/mod.rs b/primitives/state-machine/src/changes_trie/mod.rs
similarity index 100%
rename from core/state-machine/src/changes_trie/mod.rs
rename to primitives/state-machine/src/changes_trie/mod.rs
diff --git a/core/state-machine/src/changes_trie/prune.rs b/primitives/state-machine/src/changes_trie/prune.rs
similarity index 94%
rename from core/state-machine/src/changes_trie/prune.rs
rename to primitives/state-machine/src/changes_trie/prune.rs
index 1dc7c3e6c07b2..fccd177e60a2f 100644
--- a/core/state-machine/src/changes_trie/prune.rs
+++ b/primitives/state-machine/src/changes_trie/prune.rs
@@ -20,7 +20,7 @@ use hash_db::Hasher;
 use trie::Recorder;
 use log::warn;
 use num_traits::{One, Zero};
-use crate::proving_backend::ProvingBackendEssence;
+use crate::proving_backend::ProvingBackendRecorder;
 use crate::trie_backend_essence::TrieBackendEssence;
 use crate::changes_trie::{AnchorBlockId, Configuration, Storage, BlockNumber};
 use crate::changes_trie::storage::TrieBackendAdapter;
@@ -122,7 +122,7 @@ fn prune_trie, H: Hasher, Number: BlockNumber, F: FnMut(H:
 	// (effectively - all changes trie nodes)
 	let mut proof_recorder: Recorder = Default::default();
 	{
-		let mut trie = ProvingBackendEssence::<_, H> {
+		let mut trie = ProvingBackendRecorder::<_, H> {
 			backend: &TrieBackendEssence::new(TrieBackendAdapter::new(storage), root),
 			proof_recorder: &mut proof_recorder,
 		};
@@ -168,7 +168,9 @@ fn pruning_range(
 
 	// last block for which changes trie is pruned
 	let last_block_to_prune = blocks_to_keep.and_then(|b| block.checked_sub(&b));
-	let first_block_to_prune = last_block_to_prune.clone().and_then(|b| b.checked_sub(&prune_interval.into()));
+	let first_block_to_prune = last_block_to_prune
+		.clone()
+		.and_then(|b| b.checked_sub(&prune_interval.into()));
 
 	last_block_to_prune
 		.and_then(|last| first_block_to_prune.map(|first| (first + One::one(), last)))
@@ -221,7 +223,10 @@ mod tests {
 		current_block: u64,
 	) -> HashSet {
 		let mut pruned_trie_nodes = HashSet::new();
-		prune(config, storage, min_blocks_to_keep, &AnchorBlockId { hash: Default::default(), number: current_block },
+		prune(config,
+			storage,
+			min_blocks_to_keep,
+			&AnchorBlockId { hash: Default::default(), number: current_block },
 			|node| { pruned_trie_nodes.insert(node); });
 		pruned_trie_nodes
 	}
@@ -232,18 +237,22 @@ mod tests {
 
 			let child_key = ChildIndex { block: 67u64, storage_key: b"1".to_vec() }.encode();
 			let mut mdb1 = MemoryDB::::default();
-			let root1 = insert_into_memory_db::(&mut mdb1, vec![(vec![10], vec![20])]).unwrap();
+			let root1 = insert_into_memory_db::(
+				&mut mdb1, vec![(vec![10], vec![20])]).unwrap();
 			let mut mdb2 = MemoryDB::::default();
-			let root2 = insert_into_memory_db::(&mut mdb2, vec![(vec![11], vec![21]), (vec![12], vec![22])]).unwrap();
+			let root2 = insert_into_memory_db::(
+				&mut mdb2, vec![(vec![11], vec![21]), (vec![12], vec![22])]).unwrap();
 			let mut mdb3 = MemoryDB::::default();
-			let ch_root3 = insert_into_memory_db::(&mut mdb3, vec![(vec![110], vec![120])]).unwrap();
+			let ch_root3 = insert_into_memory_db::(
+				&mut mdb3, vec![(vec![110], vec![120])]).unwrap();
 			let root3 = insert_into_memory_db::(&mut mdb3, vec![
 				(vec![13], vec![23]),
 				(vec![14], vec![24]),
 				(child_key, ch_root3.as_ref().encode()),
 			]).unwrap();
 			let mut mdb4 = MemoryDB::::default();
-			let root4 = insert_into_memory_db::(&mut mdb4, vec![(vec![15], vec![25])]).unwrap();
+			let root4 = insert_into_memory_db::(
+				&mut mdb4, vec![(vec![15], vec![25])]).unwrap();
 			let storage = InMemoryStorage::new();
 			storage.insert(65, root1, mdb1);
 			storage.insert(66, root2, mdb2);
diff --git a/core/state-machine/src/changes_trie/storage.rs b/primitives/state-machine/src/changes_trie/storage.rs
similarity index 100%
rename from core/state-machine/src/changes_trie/storage.rs
rename to primitives/state-machine/src/changes_trie/storage.rs
diff --git a/core/state-machine/src/changes_trie/surface_iterator.rs b/primitives/state-machine/src/changes_trie/surface_iterator.rs
similarity index 100%
rename from core/state-machine/src/changes_trie/surface_iterator.rs
rename to primitives/state-machine/src/changes_trie/surface_iterator.rs
diff --git a/core/state-machine/src/error.rs b/primitives/state-machine/src/error.rs
similarity index 100%
rename from core/state-machine/src/error.rs
rename to primitives/state-machine/src/error.rs
diff --git a/core/state-machine/src/ext.rs b/primitives/state-machine/src/ext.rs
similarity index 100%
rename from core/state-machine/src/ext.rs
rename to primitives/state-machine/src/ext.rs
diff --git a/core/state-machine/src/lib.rs b/primitives/state-machine/src/lib.rs
similarity index 98%
rename from core/state-machine/src/lib.rs
rename to primitives/state-machine/src/lib.rs
index c3092367f0646..85fa02f7ffe8b 100644
--- a/core/state-machine/src/lib.rs
+++ b/primitives/state-machine/src/lib.rs
@@ -59,8 +59,8 @@ pub use changes_trie::{
 };
 pub use overlayed_changes::OverlayedChanges;
 pub use proving_backend::{
-	create_proof_check_backend, create_proof_check_backend_storage,
-	Recorder as ProofRecorder, ProvingBackend,
+	create_proof_check_backend, create_proof_check_backend_storage, merge_storage_proofs,
+	ProofRecorder, ProvingBackend, ProvingBackendRecorder, StorageProof,
 };
 pub use trie_backend_essence::{TrieBackendStorage, Storage};
 pub use trie_backend::TrieBackend;
@@ -83,7 +83,7 @@ pub enum ExecutionStrategy {
 	NativeWhenPossible,
 	/// Use the given wasm module.
 	AlwaysWasm,
-	/// Run with both the wasm and the native variant (if compatible). Report any discrepency as an error.
+	/// Run with both the wasm and the native variant (if compatible). Report any discrepancy as an error.
 	Both,
 	/// First native, then if that fails or is not possible, wasm.
 	NativeElseWasm,
@@ -109,7 +109,7 @@ pub enum ExecutionManager {
 	/// trusted to provide all storage or not (i.e. the light client cannot be trusted to provide
 	/// for all storage queries since the storage entries it has come from an external node).
 	AlwaysWasm(BackendTrustLevel),
-	/// Run with both the wasm and the native variant (if compatible). Call `F` in the case of any discrepency.
+	/// Run with both the wasm and the native variant (if compatible). Call `F` in the case of any discrepancy.
 	Both(F),
 	/// First native, then if that fails or is not possible, wasm.
 	NativeElseWasm,
@@ -463,7 +463,7 @@ pub fn prove_execution(
 	method: &str,
 	call_data: &[u8],
 	keystore: Option,
-) -> Result<(Vec, Vec>), Box>
+) -> Result<(Vec, StorageProof), Box>
 where
 	B: Backend,
 	H: Hasher,
@@ -490,7 +490,7 @@ pub fn prove_execution_on_trie_backend(
 	method: &str,
 	call_data: &[u8],
 	keystore: Option,
-) -> Result<(Vec, Vec>), Box>
+) -> Result<(Vec, StorageProof), Box>
 where
 	S: trie_backend_essence::TrieBackendStorage,
 	H: Hasher,
@@ -513,7 +513,7 @@ where
 /// Check execution proof, generated by `prove_execution` call.
 pub fn execution_proof_check(
 	root: H::Out,
-	proof: Vec>,
+	proof: StorageProof,
 	overlay: &mut OverlayedChanges,
 	exec: &Exec,
 	method: &str,
@@ -557,7 +557,7 @@ where
 pub fn prove_read(
 	mut backend: B,
 	keys: I,
-) -> Result>, Box>
+) -> Result>
 where
 	B: Backend,
 	H: Hasher,
@@ -577,7 +577,7 @@ pub fn prove_child_read(
 	mut backend: B,
 	storage_key: &[u8],
 	keys: I,
-) -> Result>, Box>
+) -> Result>
 where
 	B: Backend,
 	H: Hasher,
@@ -594,7 +594,7 @@ where
 pub fn prove_read_on_trie_backend(
 	trie_backend: &TrieBackend,
 	keys: I,
-) -> Result>, Box>
+) -> Result>
 where
 	S: trie_backend_essence::TrieBackendStorage,
 	H: Hasher,
@@ -616,7 +616,7 @@ pub fn prove_child_read_on_trie_backend(
 	trie_backend: &TrieBackend,
 	storage_key: &[u8],
 	keys: I,
-) -> Result>, Box>
+) -> Result>
 where
 	S: trie_backend_essence::TrieBackendStorage,
 	H: Hasher,
@@ -636,7 +636,7 @@ where
 /// Check storage read proof, generated by `prove_read` call.
 pub fn read_proof_check(
 	root: H::Out,
-	proof: Vec>,
+	proof: StorageProof,
 	keys: I,
 ) -> Result, Option>>, Box>
 where
@@ -657,7 +657,7 @@ where
 /// Check child storage read proof, generated by `prove_child_read` call.
 pub fn read_child_proof_check(
 	root: H::Out,
-	proof: Vec>,
+	proof: StorageProof,
 	storage_key: &[u8],
 	keys: I,
 ) -> Result, Option>>, Box>
diff --git a/core/state-machine/src/overlayed_changes.rs b/primitives/state-machine/src/overlayed_changes.rs
similarity index 92%
rename from core/state-machine/src/overlayed_changes.rs
rename to primitives/state-machine/src/overlayed_changes.rs
index 53a66dc49ee05..73cb8b604a1c2 100644
--- a/core/state-machine/src/overlayed_changes.rs
+++ b/primitives/state-machine/src/overlayed_changes.rs
@@ -359,7 +359,9 @@ mod tests {
 	use crate::ext::Ext;
 	use super::*;
 
-	fn strip_extrinsic_index(map: &HashMap, OverlayedValue>) -> HashMap, OverlayedValue> {
+	fn strip_extrinsic_index(map: &HashMap, OverlayedValue>)
+		-> HashMap, OverlayedValue>
+	{
 		let mut clone = map.clone();
 		clone.remove(&EXTRINSIC_INDEX.to_vec());
 		clone
@@ -455,7 +457,8 @@ mod tests {
 		assert_eq!(
 			strip_extrinsic_index(&overlay.prospective.top),
 			vec![
-				(vec![1], OverlayedValue { value: Some(vec![2]), extrinsics: Some(vec![0].into_iter().collect()) }),
+				(vec![1], OverlayedValue { value: Some(vec![2]),
+				 extrinsics: Some(vec![0].into_iter().collect()) }),
 			].into_iter().collect(),
 		);
 	}
@@ -491,9 +494,12 @@ mod tests {
 
 		assert_eq!(strip_extrinsic_index(&overlay.prospective.top),
 			vec![
-				(vec![1], OverlayedValue { value: Some(vec![6]), extrinsics: Some(vec![0, 2].into_iter().collect()) }),
-				(vec![3], OverlayedValue { value: Some(vec![4]), extrinsics: Some(vec![1].into_iter().collect()) }),
-				(vec![100], OverlayedValue { value: Some(vec![101]), extrinsics: Some(vec![NO_EXTRINSIC_INDEX].into_iter().collect()) }),
+				(vec![1], OverlayedValue { value: Some(vec![6]),
+				 extrinsics: Some(vec![0, 2].into_iter().collect()) }),
+				(vec![3], OverlayedValue { value: Some(vec![4]),
+				 extrinsics: Some(vec![1].into_iter().collect()) }),
+				(vec![100], OverlayedValue { value: Some(vec![101]),
+				 extrinsics: Some(vec![NO_EXTRINSIC_INDEX].into_iter().collect()) }),
 			].into_iter().collect());
 
 		overlay.commit_prospective();
@@ -506,24 +512,32 @@ mod tests {
 
 		assert_eq!(strip_extrinsic_index(&overlay.committed.top),
 			vec![
-				(vec![1], OverlayedValue { value: Some(vec![6]), extrinsics: Some(vec![0, 2].into_iter().collect()) }),
-				(vec![3], OverlayedValue { value: Some(vec![4]), extrinsics: Some(vec![1].into_iter().collect()) }),
-				(vec![100], OverlayedValue { value: Some(vec![101]), extrinsics: Some(vec![NO_EXTRINSIC_INDEX].into_iter().collect()) }),
+				(vec![1], OverlayedValue { value: Some(vec![6]),
+				 extrinsics: Some(vec![0, 2].into_iter().collect()) }),
+				(vec![3], OverlayedValue { value: Some(vec![4]),
+				 extrinsics: Some(vec![1].into_iter().collect()) }),
+				(vec![100], OverlayedValue { value: Some(vec![101]),
+				 extrinsics: Some(vec![NO_EXTRINSIC_INDEX].into_iter().collect()) }),
 			].into_iter().collect());
 
 		assert_eq!(strip_extrinsic_index(&overlay.prospective.top),
 			vec![
-				(vec![1], OverlayedValue { value: Some(vec![8]), extrinsics: Some(vec![4].into_iter().collect()) }),
-				(vec![3], OverlayedValue { value: Some(vec![7]), extrinsics: Some(vec![3].into_iter().collect()) }),
+				(vec![1], OverlayedValue { value: Some(vec![8]),
+				 extrinsics: Some(vec![4].into_iter().collect()) }),
+				(vec![3], OverlayedValue { value: Some(vec![7]),
+				 extrinsics: Some(vec![3].into_iter().collect()) }),
 			].into_iter().collect());
 
 		overlay.commit_prospective();
 
 		assert_eq!(strip_extrinsic_index(&overlay.committed.top),
 			vec![
-				(vec![1], OverlayedValue { value: Some(vec![8]), extrinsics: Some(vec![0, 2, 4].into_iter().collect()) }),
-				(vec![3], OverlayedValue { value: Some(vec![7]), extrinsics: Some(vec![1, 3].into_iter().collect()) }),
-				(vec![100], OverlayedValue { value: Some(vec![101]), extrinsics: Some(vec![NO_EXTRINSIC_INDEX].into_iter().collect()) }),
+				(vec![1], OverlayedValue { value: Some(vec![8]),
+				 extrinsics: Some(vec![0, 2, 4].into_iter().collect()) }),
+				(vec![3], OverlayedValue { value: Some(vec![7]),
+				 extrinsics: Some(vec![1, 3].into_iter().collect()) }),
+				(vec![100], OverlayedValue { value: Some(vec![101]),
+				 extrinsics: Some(vec![NO_EXTRINSIC_INDEX].into_iter().collect()) }),
 			].into_iter().collect());
 
 		assert_eq!(overlay.prospective,
diff --git a/core/state-machine/src/proving_backend.rs b/primitives/state-machine/src/proving_backend.rs
similarity index 66%
rename from core/state-machine/src/proving_backend.rs
rename to primitives/state-machine/src/proving_backend.rs
index c6c4ef1ec8c26..2ebd82f0c43c0 100644
--- a/core/state-machine/src/proving_backend.rs
+++ b/primitives/state-machine/src/proving_backend.rs
@@ -16,9 +16,11 @@
 
 //! Proving state machine backend.
 
-use std::{cell::RefCell, rc::Rc};
+use std::sync::Arc;
+use parking_lot::RwLock;
+use codec::{Decode, Encode};
 use log::debug;
-use hash_db::{Hasher, HashDB, EMPTY_PREFIX};
+use hash_db::{Hasher, HashDB, EMPTY_PREFIX, Prefix};
 use trie::{
 	MemoryDB, PrefixedMemoryDB, default_child_trie_root,
 	read_trie_value_with, read_child_trie_value_with, record_all_keys
@@ -28,19 +30,97 @@ pub use trie::trie_types::{Layout, TrieError};
 use crate::trie_backend::TrieBackend;
 use crate::trie_backend_essence::{Ephemeral, TrieBackendEssence, TrieBackendStorage};
 use crate::{Error, ExecutionError, Backend};
+use std::collections::{HashMap, HashSet};
+use crate::DBValue;
 
-/// Patricia trie-based backend essence which also tracks all touched storage trie values.
-/// These can be sent to remote node and used as a proof of execution.
-pub struct ProvingBackendEssence<'a, S: 'a + TrieBackendStorage, H: 'a + Hasher> {
+/// Patricia trie-based backend specialized in get value proofs.
+pub struct ProvingBackendRecorder<'a, S: 'a + TrieBackendStorage, H: 'a + Hasher> {
 	pub(crate) backend: &'a TrieBackendEssence,
 	pub(crate) proof_recorder: &'a mut Recorder,
 }
 
-impl<'a, S, H> ProvingBackendEssence<'a, S, H>
+/// A proof that some set of key-value pairs are included in the storage trie. The proof contains
+/// the storage values so that the partial storage backend can be reconstructed by a verifier that
+/// does not already have access to the key-value pairs.
+///
+/// The proof consists of the set of serialized nodes in the storage trie accessed when looking up
+/// the keys covered by the proof. Verifying the proof requires constructing the partial trie from
+/// the serialized nodes and performing the key lookups.
+#[derive(Debug, PartialEq, Eq, Clone, Encode, Decode)]
+pub struct StorageProof {
+	trie_nodes: Vec>,
+}
+
+impl StorageProof {
+	/// Constructs a storage proof from a subset of encoded trie nodes in a storage backend.
+	pub fn new(trie_nodes: Vec>) -> Self {
+		StorageProof { trie_nodes }
+	}
+
+	/// Returns a new empty proof.
+	///
+	/// An empty proof is capable of only proving trivial statements (ie. that an empty set of
+	/// key-value pairs exist in storage).
+	pub fn empty() -> Self {
+		StorageProof {
+			trie_nodes: Vec::new(),
+		}
+	}
+
+	/// Returns whether this is an empty proof.
+	pub fn is_empty(&self) -> bool {
+		self.trie_nodes.is_empty()
+	}
+
+	/// Create an iterator over trie nodes constructed from the proof. The nodes are not guaranteed
+	/// to be traversed in any particular order.
+	pub fn iter_nodes(self) -> StorageProofNodeIterator {
+		StorageProofNodeIterator::new(self)
+	}
+}
+
+/// An iterator over trie nodes constructed from a storage proof. The nodes are not guaranteed to
+/// be traversed in any particular order.
+pub struct StorageProofNodeIterator {
+	inner: > as IntoIterator>::IntoIter,
+}
+
+impl StorageProofNodeIterator {
+	fn new(proof: StorageProof) -> Self {
+		StorageProofNodeIterator {
+			inner: proof.trie_nodes.into_iter(),
+		}
+	}
+}
+
+impl Iterator for StorageProofNodeIterator {
+	type Item = Vec;
+
+	fn next(&mut self) -> Option {
+		self.inner.next()
+	}
+}
+
+/// Merges multiple storage proofs covering potentially different sets of keys into one proof
+/// covering all keys. The merged proof output may be smaller than the aggregate size of the input
+/// proofs due to deduplication of trie nodes.
+pub fn merge_storage_proofs(proofs: I) -> StorageProof
+	where I: IntoIterator
+{
+	let trie_nodes = proofs.into_iter()
+		.flat_map(|proof| proof.iter_nodes())
+		.collect::>()
+		.into_iter()
+		.collect();
+	StorageProof { trie_nodes }
+}
+
+impl<'a, S, H> ProvingBackendRecorder<'a, S, H>
 	where
 		S: TrieBackendStorage,
 		H: Hasher,
 {
+	/// Produce proof for a key query.
 	pub fn storage(&mut self, key: &[u8]) -> Result>, String> {
 		let mut read_overlay = S::Overlay::default();
 		let eph = Ephemeral::new(
@@ -58,6 +138,7 @@ impl<'a, S, H> ProvingBackendEssence<'a, S, H>
 		).map_err(map_e)
 	}
 
+	/// Produce proof for a child key query.
 	pub fn child_storage(
 		&mut self,
 		storage_key: &[u8],
@@ -83,6 +164,7 @@ impl<'a, S, H> ProvingBackendEssence<'a, S, H>
 		).map_err(map_e)
 	}
 
+	/// Produce proof for the whole backend.
 	pub fn record_all_keys(&mut self) {
 		let mut read_overlay = S::Overlay::default();
 		let eph = Ephemeral::new(
@@ -101,41 +183,64 @@ impl<'a, S, H> ProvingBackendEssence<'a, S, H>
 	}
 }
 
+/// Global proof recorder, act as a layer over a hash db for recording queried
+/// data.
+pub type ProofRecorder = Arc::Out, Option>>>;
+
 /// Patricia trie-based backend which also tracks all touched storage trie values.
 /// These can be sent to remote node and used as a proof of execution.
-pub struct ProvingBackend<'a, S: 'a + TrieBackendStorage, H: 'a + Hasher> {
-	backend: &'a TrieBackend,
-	proof_recorder: Rc>>,
+pub struct ProvingBackend<'a, S: 'a + TrieBackendStorage, H: 'a + Hasher> (
+	TrieBackend, H>,
+);
+
+/// Trie backend storage with its proof recorder.
+pub struct ProofRecorderBackend<'a, S: 'a + TrieBackendStorage, H: 'a + Hasher> {
+	backend: &'a S,
+	proof_recorder: ProofRecorder,
 }
 
 impl<'a, S: 'a + TrieBackendStorage, H: 'a + Hasher> ProvingBackend<'a, S, H> {
 	/// Create new proving backend.
 	pub fn new(backend: &'a TrieBackend) -> Self {
-		ProvingBackend {
-			backend,
-			proof_recorder: Rc::new(RefCell::new(Recorder::new())),
-		}
+		let proof_recorder = Default::default();
+		Self::new_with_recorder(backend, proof_recorder)
 	}
 
 	/// Create new proving backend with the given recorder.
 	pub fn new_with_recorder(
 		backend: &'a TrieBackend,
-		proof_recorder: Rc>>,
+		proof_recorder: ProofRecorder,
 	) -> Self {
-		ProvingBackend {
-			backend,
-			proof_recorder,
-		}
+		let essence = backend.essence();
+		let root = essence.root().clone();
+		let recorder = ProofRecorderBackend {
+			backend: essence.backend_storage(),
+			proof_recorder: proof_recorder,
+		};
+		ProvingBackend(TrieBackend::new(recorder, root))
 	}
 
-	/// Consume the backend, extracting the gathered proof in lexicographical order by value.
-	pub fn extract_proof(&self) -> Vec> {
-		self.proof_recorder
-			.borrow_mut()
-			.drain()
-			.into_iter()
-			.map(|n| n.data.to_vec())
-			.collect()
+	/// Extracting the gathered unordered proof.
+	pub fn extract_proof(&self) -> StorageProof {
+		let trie_nodes = self.0.essence().backend_storage().proof_recorder
+			.read()
+			.iter()
+			.filter_map(|(_k, v)| v.as_ref().map(|v| v.to_vec()))
+			.collect();
+		StorageProof::new(trie_nodes)
+	}
+}
+
+impl<'a, S: 'a + TrieBackendStorage, H: 'a + Hasher> TrieBackendStorage for ProofRecorderBackend<'a, S, H> {
+	type Overlay = S::Overlay;
+
+	fn get(&self, key: &H::Out, prefix: Prefix) -> Result, String> {
+		if let Some(v) = self.proof_recorder.read().get(key) {
+			return Ok(v.clone());
+		}
+		let backend_value =  self.backend.get(key, prefix)?;
+		self.proof_recorder.write().insert(key.clone(), backend_value.clone());
+		Ok(backend_value)
 	}
 }
 
@@ -156,53 +261,45 @@ impl<'a, S, H> Backend for ProvingBackend<'a, S, H>
 	type TrieBackendStorage = PrefixedMemoryDB;
 
 	fn storage(&self, key: &[u8]) -> Result>, Self::Error> {
-		ProvingBackendEssence {
-			backend: self.backend.essence(),
-			proof_recorder: &mut *self.proof_recorder.try_borrow_mut()
-				.expect("only fails when already borrowed; storage() is non-reentrant; qed"),
-		}.storage(key)
+		self.0.storage(key)
 	}
 
 	fn child_storage(&self, storage_key: &[u8], key: &[u8]) -> Result>, Self::Error> {
-		ProvingBackendEssence {
-			backend: self.backend.essence(),
-			proof_recorder: &mut *self.proof_recorder.try_borrow_mut()
-				.expect("only fails when already borrowed; child_storage() is non-reentrant; qed"),
-		}.child_storage(storage_key, key)
+		self.0.child_storage(storage_key, key)
 	}
 
 	fn for_keys_in_child_storage(&self, storage_key: &[u8], f: F) {
-		self.backend.for_keys_in_child_storage(storage_key, f)
+		self.0.for_keys_in_child_storage(storage_key, f)
 	}
 
 	fn for_keys_with_prefix(&self, prefix: &[u8], f: F) {
-		self.backend.for_keys_with_prefix(prefix, f)
+		self.0.for_keys_with_prefix(prefix, f)
 	}
 
 	fn for_key_values_with_prefix(&self, prefix: &[u8], f: F) {
-		self.backend.for_key_values_with_prefix(prefix, f)
+		self.0.for_key_values_with_prefix(prefix, f)
 	}
 
 	fn for_child_keys_with_prefix(&self, storage_key: &[u8], prefix: &[u8], f: F) {
-		self.backend.for_child_keys_with_prefix(storage_key, prefix, f)
+		self.0.for_child_keys_with_prefix(storage_key, prefix, f)
 	}
 
 	fn pairs(&self) -> Vec<(Vec, Vec)> {
-		self.backend.pairs()
+		self.0.pairs()
 	}
 
 	fn keys(&self, prefix: &[u8]) -> Vec> {
-		self.backend.keys(prefix)
+		self.0.keys(prefix)
 	}
 
 	fn child_keys(&self, child_storage_key: &[u8], prefix: &[u8]) -> Vec> {
-		self.backend.child_keys(child_storage_key, prefix)
+		self.0.child_keys(child_storage_key, prefix)
 	}
 
 	fn storage_root(&self, delta: I) -> (H::Out, Self::Transaction)
 		where I: IntoIterator, Option>)>
 	{
-		self.backend.storage_root(delta)
+		self.0.storage_root(delta)
 	}
 
 	fn child_storage_root(&self, storage_key: &[u8], delta: I) -> (Vec, bool, Self::Transaction)
@@ -210,14 +307,14 @@ impl<'a, S, H> Backend for ProvingBackend<'a, S, H>
 		I: IntoIterator, Option>)>,
 		H::Out: Ord
 	{
-		self.backend.child_storage_root(storage_key, delta)
+		self.0.child_storage_root(storage_key, delta)
 	}
 }
 
 /// Create proof check backend.
 pub fn create_proof_check_backend(
 	root: H::Out,
-	proof: Vec>
+	proof: StorageProof,
 ) -> Result, H>, Box>
 where
 	H: Hasher,
@@ -233,13 +330,13 @@ where
 
 /// Create in-memory storage of proof check backend.
 pub fn create_proof_check_backend_storage(
-	proof: Vec>
+	proof: StorageProof,
 ) -> MemoryDB
 where
 	H: Hasher,
 {
 	let mut db = MemoryDB::default();
-	for item in proof {
+	for item in proof.iter_nodes() {
 		db.insert(EMPTY_PREFIX, &item);
 	}
 	db
@@ -251,6 +348,7 @@ mod tests {
 	use crate::trie_backend::tests::test_trie;
 	use super::*;
 	use primitives::{Blake2Hasher, storage::ChildStorageKey};
+	use crate::proving_backend::create_proof_check_backend;
 
 	fn test_proving<'a>(
 		trie_backend: &'a TrieBackend,Blake2Hasher>,
@@ -275,7 +373,11 @@ mod tests {
 	#[test]
 	fn proof_is_invalid_when_does_not_contains_root() {
 		use primitives::H256;
-		assert!(create_proof_check_backend::(H256::from_low_u64_be(1), vec![]).is_err());
+		let result = create_proof_check_backend::(
+			H256::from_low_u64_be(1),
+			StorageProof::empty()
+		);
+		assert!(result.is_err());
 	}
 
 	#[test]
diff --git a/core/state-machine/src/testing.rs b/primitives/state-machine/src/testing.rs
similarity index 98%
rename from core/state-machine/src/testing.rs
rename to primitives/state-machine/src/testing.rs
index 16ff62020b594..61b338bc81acd 100644
--- a/core/state-machine/src/testing.rs
+++ b/primitives/state-machine/src/testing.rs
@@ -46,7 +46,6 @@ pub struct TestExternalities=Blake2Hasher, N: ChangesTrieBlo
 }
 
 impl, N: ChangesTrieBlockNumber> TestExternalities {
-
 	/// Get externalities implementation.
 	pub fn ext(&mut self) -> Ext, ChangesTrieInMemoryStorage> {
 		Ext::new(
@@ -192,4 +191,10 @@ mod tests {
 
 		assert_eq!(&ext.storage(CODE).unwrap(), &code);
 	}
+
+	#[test]
+	fn check_send() {
+		fn assert_send() {}
+		assert_send::>();
+	}
 }
diff --git a/core/state-machine/src/trie_backend.rs b/primitives/state-machine/src/trie_backend.rs
similarity index 100%
rename from core/state-machine/src/trie_backend.rs
rename to primitives/state-machine/src/trie_backend.rs
diff --git a/core/state-machine/src/trie_backend_essence.rs b/primitives/state-machine/src/trie_backend_essence.rs
similarity index 100%
rename from core/state-machine/src/trie_backend_essence.rs
rename to primitives/state-machine/src/trie_backend_essence.rs
diff --git a/primitives/transaction-pool/runtime-api/Cargo.toml b/primitives/transaction-pool/runtime-api/Cargo.toml
new file mode 100644
index 0000000000000..e002208502592
--- /dev/null
+++ b/primitives/transaction-pool/runtime-api/Cargo.toml
@@ -0,0 +1,14 @@
+[package]
+name = "substrate-transaction-pool-runtime-api"
+version = "2.0.0"
+authors = ["Parity Technologies "]
+edition = "2018"
+
+[dependencies]
+primitives = { package = "substrate-primitives",  path = "../../core", default-features = false }
+sr-api = { path = "../../sr-api", default-features = false }
+sr-primitives = { path = "../../sr-primitives", default-features = false }
+
+[features]
+default = [ "std" ]
+std = [ "sr-primitives/std", "primitives/std", "sr-api/std" ]
diff --git a/primitives/transaction-pool/runtime-api/src/lib.rs b/primitives/transaction-pool/runtime-api/src/lib.rs
new file mode 100644
index 0000000000000..48d0f8a85dd31
--- /dev/null
+++ b/primitives/transaction-pool/runtime-api/src/lib.rs
@@ -0,0 +1,29 @@
+// Copyright 2018-2019 Parity Technologies (UK) Ltd.
+// This file is part of Substrate.
+
+// Substrate is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+
+// Substrate is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+
+// You should have received a copy of the GNU General Public License
+// along with Substrate.  If not, see .
+
+//! Substrate runtime api for the transaction queue.
+
+#![cfg_attr(not(feature = "std"), no_std)]
+
+use sr_primitives::{transaction_validity::TransactionValidity, traits::Block as BlockT};
+
+sr_api::decl_runtime_apis! {
+	/// The `TaggedTransactionQueue` api trait for interfering with the transaction queue.
+	pub trait TaggedTransactionQueue {
+		/// Validate the given transaction.
+		fn validate_transaction(tx: ::Extrinsic) -> TransactionValidity;
+	}
+}
diff --git a/core/trie/Cargo.toml b/primitives/trie/Cargo.toml
similarity index 88%
rename from core/trie/Cargo.toml
rename to primitives/trie/Cargo.toml
index 50498a17ba65d..202c9066d0917 100644
--- a/core/trie/Cargo.toml
+++ b/primitives/trie/Cargo.toml
@@ -18,12 +18,11 @@ hash-db = { version = "0.15.2", default-features = false }
 trie-db = { version = "0.15.2", default-features = false }
 trie-root = { version = "0.15.2", default-features = false }
 memory-db = { version = "0.15.2", default-features = false }
-primitives = { package = "substrate-primitives",  path = "../primitives", default-features = false }
+primitives = { package = "substrate-primitives",  path = "../core", default-features = false }
 
 [dev-dependencies]
 trie-bench = "0.16.2"
 trie-standardmap = "0.15.2"
-keccak-hasher = "0.15.2"
 criterion = "0.2.11"
 hex-literal = "0.2.1"
 
diff --git a/core/trie/benches/bench.rs b/primitives/trie/benches/bench.rs
similarity index 100%
rename from core/trie/benches/bench.rs
rename to primitives/trie/benches/bench.rs
diff --git a/core/trie/src/error.rs b/primitives/trie/src/error.rs
similarity index 100%
rename from core/trie/src/error.rs
rename to primitives/trie/src/error.rs
diff --git a/core/trie/src/lib.rs b/primitives/trie/src/lib.rs
similarity index 100%
rename from core/trie/src/lib.rs
rename to primitives/trie/src/lib.rs
diff --git a/core/trie/src/node_codec.rs b/primitives/trie/src/node_codec.rs
similarity index 100%
rename from core/trie/src/node_codec.rs
rename to primitives/trie/src/node_codec.rs
diff --git a/core/trie/src/node_header.rs b/primitives/trie/src/node_header.rs
similarity index 100%
rename from core/trie/src/node_header.rs
rename to primitives/trie/src/node_header.rs
diff --git a/core/trie/src/trie_stream.rs b/primitives/trie/src/trie_stream.rs
similarity index 100%
rename from core/trie/src/trie_stream.rs
rename to primitives/trie/src/trie_stream.rs
diff --git a/core/wasm-interface/Cargo.toml b/primitives/wasm-interface/Cargo.toml
similarity index 75%
rename from core/wasm-interface/Cargo.toml
rename to primitives/wasm-interface/Cargo.toml
index b8169031d0bb6..90dc7b812d9f9 100644
--- a/core/wasm-interface/Cargo.toml
+++ b/primitives/wasm-interface/Cargo.toml
@@ -5,4 +5,5 @@ authors = ["Parity Technologies "]
 edition = "2018"
 
 [dependencies]
-wasmi = "0.5.1"
+wasmi = "0.6.2"
+impl-trait-for-tuples = "0.1.2"
diff --git a/core/wasm-interface/src/lib.rs b/primitives/wasm-interface/src/lib.rs
similarity index 78%
rename from core/wasm-interface/src/lib.rs
rename to primitives/wasm-interface/src/lib.rs
index 1e67260638900..b2d57d080d533 100644
--- a/core/wasm-interface/src/lib.rs
+++ b/primitives/wasm-interface/src/lib.rs
@@ -49,6 +49,18 @@ pub enum Value {
 	F64(u64),
 }
 
+impl Value {
+	/// Returns the type of this value.
+	pub fn value_type(&self) -> ValueType {
+		match self {
+			Value::I32(_) => ValueType::I32,
+			Value::I64(_) => ValueType::I64,
+			Value::F32(_) => ValueType::F32,
+			Value::F64(_) => ValueType::F64,
+		}
+	}
+}
+
 /// Provides `Sealed` trait to prevent implementing trait `PointerType` outside of this crate.
 mod private {
 	pub trait Sealed {}
@@ -119,6 +131,12 @@ impl From> for u32 {
 	}
 }
 
+impl From> for u64 {
+	fn from(ptr: Pointer) -> Self {
+		u64::from(ptr.ptr)
+	}
+}
+
 impl From> for usize {
 	fn from(ptr: Pointer) -> Self {
 		ptr.ptr as _
@@ -171,7 +189,7 @@ impl Signature {
 }
 
 /// Something that provides a function implementation on the host for a wasm function.
-pub trait Function {
+pub trait Function: std::panic::RefUnwindSafe + Send + Sync {
 	/// Returns the name of this function.
 	fn name(&self) -> &str;
 	/// Returns the signature of this function.
@@ -184,6 +202,12 @@ pub trait Function {
 	) -> Result>;
 }
 
+impl PartialEq for dyn Function {
+	fn eq(&self, other: &Self) -> bool {
+		other.name() == self.name() && other.signature() == self.signature()
+	}
+}
+
 /// Context used by `Function` to interact with the allocator and the memory of the wasm instance.
 pub trait FunctionContext {
 	/// Read memory from `address` into a vector.
@@ -212,7 +236,7 @@ pub type MemoryId = u32;
 pub trait Sandbox {
 	/// Get sandbox memory from the `memory_id` instance at `offset` into the given buffer.
 	fn memory_get(
-		&self,
+		&mut self,
 		memory_id: MemoryId,
 		offset: WordSize,
 		buf_ptr: Pointer,
@@ -254,9 +278,20 @@ pub trait Sandbox {
 }
 
 /// Something that provides implementations for host functions.
-pub trait HostFunctions {
-	/// Returns all host functions.
-	fn functions() -> &'static [&'static dyn Function];
+pub trait HostFunctions: 'static {
+	/// Returns the host functions `Self` provides.
+	fn host_functions() -> Vec<&'static dyn Function>;
+}
+
+#[impl_trait_for_tuples::impl_for_tuples(30)]
+impl HostFunctions for Tuple {
+	fn host_functions() -> Vec<&'static dyn Function> {
+		let mut host_functions = Vec::new();
+
+		for_tuples!( #( host_functions.extend(Tuple::host_functions()); )* );
+
+		host_functions
+	}
 }
 
 /// Something that can be converted into a wasm compatible `Value`.
@@ -299,12 +334,58 @@ macro_rules! impl_into_and_from_value {
 }
 
 impl_into_and_from_value! {
+	u8, I32,
+	u16, I32,
 	u32, I32,
-	i32, I32,
 	u64, I64,
+	i8, I32,
+	i16, I32,
+	i32, I32,
 	i64, I64,
 }
 
+/// Something that can write a primitive to wasm memory location.
+pub trait WritePrimitive {
+	/// Write the given value `t` to the given memory location `ptr`.
+	fn write_primitive(&mut self, ptr: Pointer, t: T) -> Result<()>;
+}
+
+impl WritePrimitive for &mut dyn FunctionContext {
+	fn write_primitive(&mut self, ptr: Pointer, t: u32) -> Result<()> {
+		let r = t.to_le_bytes();
+		self.write_memory(ptr.cast(), &r)
+	}
+}
+
+impl WritePrimitive for &mut dyn FunctionContext {
+	fn write_primitive(&mut self, ptr: Pointer, t: u64) -> Result<()> {
+		let r = t.to_le_bytes();
+		self.write_memory(ptr.cast(), &r)
+	}
+}
+
+/// Something that can read a primitive from a wasm memory location.
+pub trait ReadPrimitive {
+	/// Read a primitive from the given memory location `ptr`.
+	fn read_primitive(&self, ptr: Pointer) -> Result;
+}
+
+impl ReadPrimitive for &mut dyn FunctionContext {
+	fn read_primitive(&self, ptr: Pointer) -> Result {
+		let mut r = [0u8; 4];
+		self.read_memory_into(ptr.cast(), &mut r)?;
+		Ok(u32::from_le_bytes(r))
+	}
+}
+
+impl ReadPrimitive for &mut dyn FunctionContext {
+	fn read_primitive(&self, ptr: Pointer) -> Result {
+		let mut r = [0u8; 8];
+		self.read_memory_into(ptr.cast(), &mut r)?;
+		Ok(u64::from_le_bytes(r))
+	}
+}
+
 #[cfg(test)]
 mod tests {
 	use super::*;
diff --git a/core/wasm-interface/src/wasmi_impl.rs b/primitives/wasm-interface/src/wasmi_impl.rs
similarity index 100%
rename from core/wasm-interface/src/wasmi_impl.rs
rename to primitives/wasm-interface/src/wasmi_impl.rs
diff --git a/srml/README.adoc b/srml/README.adoc
deleted file mode 100644
index 05da2de0a0357..0000000000000
--- a/srml/README.adoc
+++ /dev/null
@@ -1,26 +0,0 @@
-
-= SRML
-
-The Substrate Runtime Module Library (SRML) is a collection of runtime modules.
-
-== What are runtime modules?
-
-A Substrate runtime can be composed of several smaller components for separation of concerns. These components are called runtime _modules_. Each runtime module packages together a set of functions (dispatchable extrinsic calls, public or private, mutable or immutable), storage items, and events.
-
-There are four primary components that support runtime modules:
-
-=== system module
-
-https://github.com/paritytech/substrate/tree/master/srml/system[`system`] provides low-level APIs and utilities for other modules. https://github.com/paritytech/substrate/tree/master/srml/system[`system`] also defines all core types and extrinsic events for the Substrate runtime. *All modules depend on the system module.*
-
-=== executive module
-
-https://github.com/paritytech/substrate/tree/master/srml/executive[`executive`] dispatches incoming extrinsic calls to the respective modules in the runtime.
-
-=== support macros
-
-https://github.com/paritytech/substrate/tree/master/srml/support[`support` macros] are a collection of Rust macros to facilitate the implementation of common module components. https://github.com/paritytech/substrate/tree/master/srml/support[`support` macros] expand at runtime to generate types (e.g. `Module`, `Call`, `Store`, `Event`) which are thereafter used by the runtime to communicate with the modules. Common support macros include https://crates.parity.io/srml_support/macro.decl_module.html[`decl_module`], https://crates.parity.io/srml_support_procedural/macro.decl_storage.html[`decl_storage`], https://crates.parity.io/srml_support/macro.decl_event.html[`decl_event`], and https://crates.parity.io/srml_support/macro.ensure.html[`ensure`].
-
-=== runtime
-
-The runtime expands the support macros to get type and trait implementations for each module before calling https://github.com/paritytech/substrate/tree/master/srml/executive[`executive`] to dispatch calls to the individual modules. To see an example of how this might look, see https://github.com/paritytech/substrate/blob/master/node/runtime/src/lib.rs[`../node/runtime`].
\ No newline at end of file
diff --git a/srml/aura/Cargo.toml b/srml/aura/Cargo.toml
deleted file mode 100644
index f1c37b7c08d45..0000000000000
--- a/srml/aura/Cargo.toml
+++ /dev/null
@@ -1,41 +0,0 @@
-[package]
-name = "srml-aura"
-version = "2.0.0"
-authors = ["Parity Technologies "]
-edition = "2018"
-
-[dependencies]
-app-crypto = { package = "substrate-application-crypto",  path = "../../core/application-crypto", default-features = false }
-codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false, features = ["derive"] }
-inherents = { package = "substrate-inherents", path = "../../core/inherents", default-features = false }
-primitives = { package = "substrate-primitives",  path = "../../core/primitives", default-features = false }
-rstd = { package = "sr-std", path = "../../core/sr-std", default-features = false }
-serde = { version = "1.0.101", optional = true }
-session = { package = "srml-session", path = "../session", default-features = false }
-sr-primitives = { path = "../../core/sr-primitives", default-features = false }
-runtime-io ={ package = "sr-io", path = "../../core/sr-io", default-features = false }
-support = { package = "srml-support", path = "../support", default-features = false }
-substrate-consensus-aura-primitives = { path = "../../core/consensus/aura/primitives", default-features = false}
-system = { package = "srml-system", path = "../system", default-features = false }
-timestamp = { package = "srml-timestamp", path = "../timestamp", default-features = false }
-
-[dev-dependencies]
-lazy_static = "1.4.0"
-parking_lot = "0.9.0"
-
-[features]
-default = ["std"]
-std = [
-	"app-crypto/std",
-	"codec/std",
-	"inherents/std",
-	"runtime-io/std",
-	"primitives/std",
-	"rstd/std",
-	"serde",
-	"sr-primitives/std",
-	"support/std",
-	"substrate-consensus-aura-primitives/std",
-	"system/std",
-	"timestamp/std",
-]
diff --git a/srml/authority-discovery/Cargo.toml b/srml/authority-discovery/Cargo.toml
deleted file mode 100644
index 3968ab52aec66..0000000000000
--- a/srml/authority-discovery/Cargo.toml
+++ /dev/null
@@ -1,36 +0,0 @@
-[package]
-name = "srml-authority-discovery"
-version = "0.1.0"
-authors = ["Parity Technologies "]
-edition = "2018"
-
-[dependencies]
-app-crypto = { package = "substrate-application-crypto", path = "../../core/application-crypto", default-features = false }
-codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false, features = ["derive"] }
-primitives = { package = "substrate-primitives",  path = "../../core/primitives", default-features = false }
-rstd = { package = "sr-std", path = "../../core/sr-std", default-features = false }
-serde = { version = "1.0.101", optional = true }
-runtime-io = { package = "sr-io", path = "../../core/sr-io", default-features = false }
-session = { package = "srml-session", path = "../session", default-features = false }
-sr-primitives = { path = "../../core/sr-primitives", default-features = false }
-support = { package = "srml-support", path = "../support", default-features = false }
-system = { package = "srml-system", path = "../system", default-features = false }
-
-[dev-dependencies]
-babe-primitives = { package = "substrate-consensus-babe-primitives", path = "../../core/consensus/babe/primitives", default-features = false }
-sr-staking-primitives = { path = "../../core/sr-staking-primitives", default-features = false }
-
-[features]
-default = ["std"]
-std = [
-	"app-crypto/std",
-	"codec/std",
-	"primitives/std",
-	"rstd/std",
-	"runtime-io/std",
-	"serde",
-	"session/std",
-	"sr-primitives/std",
-	"support/std",
-	"system/std",
-]
diff --git a/srml/authority-discovery/src/lib.rs b/srml/authority-discovery/src/lib.rs
deleted file mode 100644
index 8c2951f3acbcc..0000000000000
--- a/srml/authority-discovery/src/lib.rs
+++ /dev/null
@@ -1,362 +0,0 @@
-// Copyright 2019 Parity Technologies (UK) Ltd.
-// This file is part of Substrate.
-
-// Substrate is free software: you can redistribute it and/or modify
-// it under the terms of the GNU General Public License as published by
-// the Free Software Foundation, either version 3 of the License, or
-// (at your option) any later version.
-
-// Substrate is distributed in the hope that it will be useful,
-// but WITHOUT ANY WARRANTY; without even the implied warranty of
-// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-// GNU General Public License for more details.
-
-// You should have received a copy of the GNU General Public License
-// along with Substrate.  If not, see .
-
-//! # Authority discovery module.
-//!
-//! This module is used by the `core/authority-discovery` to retrieve the
-//! current set of authorities, learn its own authority id as well as sign and
-//! verify messages to and from other authorities.
-//!
-//! ## Dependencies
-//!
-//! This module depends on an externally defined session key type, specified via
-//! `Trait::AuthorityId` in the respective node runtime implementation.
-
-// Ensure we're `no_std` when compiling for Wasm.
-#![cfg_attr(not(feature = "std"), no_std)]
-
-use app_crypto::RuntimeAppPublic;
-use codec::FullCodec;
-use rstd::prelude::*;
-use support::{decl_module, decl_storage};
-
-/// The module's config trait.
-pub trait Trait: system::Trait + session::Trait {
-	type AuthorityId: RuntimeAppPublic + Default + FullCodec + PartialEq;
-}
-
-decl_storage! {
-	trait Store for Module as AuthorityDiscovery {
-		/// The current set of keys that may issue a heartbeat.
-		Keys get(fn keys): Vec;
-	}
-	add_extra_genesis {
-		config(keys): Vec;
-		build(|config| Module::::initialize_keys(&config.keys))
-	}
-}
-
-decl_module! {
-	pub struct Module for enum Call where origin: T::Origin {
-	}
-}
-
-impl Module {
-	/// Returns own authority identifier iff it is part of the current authority
-	/// set, otherwise this function returns None. The restriction might be
-	/// softened in the future in case a consumer needs to learn own authority
-	/// identifier.
-	fn authority_id() -> Option {
-		let authorities = Keys::::get();
-
-		let local_keys = T::AuthorityId::all();
-
-		authorities.into_iter().find_map(|authority| {
-			if local_keys.contains(&authority) {
-				Some(authority)
-			} else {
-				None
-			}
-		})
-	}
-
-	/// Retrieve authority identifiers of the current authority set.
-	pub fn authorities() -> Vec {
-		Keys::::get()
-	}
-
-	/// Sign the given payload with the private key corresponding to the given authority id.
-	pub fn sign(
-		payload: &Vec,
-	) -> Option<(
-		<::AuthorityId as RuntimeAppPublic>::Signature,
-		T::AuthorityId,
-	)> {
-		let authority_id = Module::::authority_id()?;
-		authority_id.sign(payload).map(|s| (s, authority_id))
-	}
-
-	/// Verify the given signature for the given payload with the given
-	/// authority identifier.
-	pub fn verify(
-		payload: &Vec,
-		signature: <::AuthorityId as RuntimeAppPublic>::Signature,
-		authority_id: T::AuthorityId,
-	) -> bool {
-		authority_id.verify(payload, &signature)
-	}
-
-	fn initialize_keys(keys: &[T::AuthorityId]) {
-		if !keys.is_empty() {
-			assert!(Keys::::get().is_empty(), "Keys are already initialized!");
-			Keys::::put(keys);
-		}
-	}
-}
-
-impl sr_primitives::BoundToRuntimeAppPublic for Module {
-	type Public = T::AuthorityId;
-}
-
-impl session::OneSessionHandler for Module {
-	type Key = T::AuthorityId;
-
-	fn on_genesis_session<'a, I: 'a>(validators: I)
-	where
-		I: Iterator,
-	{
-		let keys = validators.map(|x| x.1).collect::>();
-		Self::initialize_keys(&keys);
-	}
-
-	fn on_new_session<'a, I: 'a>(_changed: bool, _validators: I, next_validators: I)
-	where
-		I: Iterator,
-	{
-		// Remember who the authorities are for the new session.
-		Keys::::put(next_validators.map(|x| x.1).collect::>());
-	}
-
-	fn on_disabled(_i: usize) {
-		// ignore
-	}
-}
-
-#[cfg(test)]
-mod tests {
-	use super::*;
-	use app_crypto::Pair;
-	use primitives::{testing::KeyStore, crypto::key_types, sr25519, H256, traits::KeystoreExt};
-	use runtime_io::TestExternalities;
-	use sr_primitives::{
-		testing::{Header, UintAuthorityId}, traits::{ConvertInto, IdentityLookup, OpaqueKeys},
-		Perbill, KeyTypeId,
-	};
-	use support::{impl_outer_origin, parameter_types};
-
-	type AuthorityDiscovery = Module;
-	type SessionIndex = u32;
-
-	#[derive(Clone, Eq, PartialEq)]
-	pub struct Test;
-	impl Trait for Test {
-		type AuthorityId = babe_primitives::AuthorityId;
-	}
-
-	type AuthorityId = babe_primitives::AuthorityId;
-
-	pub struct TestOnSessionEnding;
-	impl session::OnSessionEnding for TestOnSessionEnding {
-		fn on_session_ending(_: SessionIndex, _: SessionIndex) -> Option> {
-			None
-		}
-	}
-
-	parameter_types! {
-		pub const DisabledValidatorsThreshold: Perbill = Perbill::from_percent(33);
-	}
-
-	impl session::Trait for Test {
-		type OnSessionEnding = TestOnSessionEnding;
-		type Keys = UintAuthorityId;
-		type ShouldEndSession = session::PeriodicSessions;
-		type SessionHandler = TestSessionHandler;
-		type Event = ();
-		type ValidatorId = AuthorityId;
-		type ValidatorIdOf = ConvertInto;
-		type SelectInitialValidators = ();
-		type DisabledValidatorsThreshold = DisabledValidatorsThreshold;
-	}
-
-	impl session::historical::Trait for Test {
-		type FullIdentification = ();
-		type FullIdentificationOf = ();
-	}
-
-	pub type BlockNumber = u64;
-
-	parameter_types! {
-		pub const Period: BlockNumber = 1;
-		pub const Offset: BlockNumber = 0;
-		pub const UncleGenerations: u64 = 0;
-		pub const BlockHashCount: u64 = 250;
-		pub const MaximumBlockWeight: u32 = 1024;
-		pub const MaximumBlockLength: u32 = 2 * 1024;
-		pub const AvailableBlockRatio: Perbill = Perbill::one();
-	}
-
-	impl system::Trait for Test {
-		type Origin = Origin;
-		type Index = u64;
-		type BlockNumber = BlockNumber;
-		type Call = ();
-		type Hash = H256;
-		type Hashing = ::sr_primitives::traits::BlakeTwo256;
-		type AccountId = AuthorityId;
-		type Lookup = IdentityLookup;
-		type Header = Header;
-		type Event = ();
-		type BlockHashCount = BlockHashCount;
-		type MaximumBlockWeight = MaximumBlockWeight;
-		type AvailableBlockRatio = AvailableBlockRatio;
-		type MaximumBlockLength = MaximumBlockLength;
-		type Version = ();
-	}
-
-	impl_outer_origin! {
-		pub enum Origin for Test {}
-	}
-
-	pub struct TestSessionHandler;
-	impl session::SessionHandler for TestSessionHandler {
-		const KEY_TYPE_IDS: &'static [KeyTypeId] = &[key_types::DUMMY];
-
-		fn on_new_session(
-			_changed: bool,
-			_validators: &[(AuthorityId, Ks)],
-			_queued_validators: &[(AuthorityId, Ks)],
-		) {
-		}
-
-		fn on_disabled(_validator_index: usize) {}
-
-		fn on_genesis_session(_validators: &[(AuthorityId, Ks)]) {}
-	}
-
-	#[test]
-	fn authority_id_fn_returns_intersection_of_current_authorities_and_keys_in_key_store() {
-		// Create keystore and generate key.
-		let key_store = KeyStore::new();
-		key_store
-			.write()
-			.sr25519_generate_new(key_types::BABE, None)
-			.expect("Generates key.");
-
-		// Retrieve key to later check if we got the right one.
-		let public_key = key_store
-			.read()
-			.sr25519_public_keys(key_types::BABE)
-			.pop()
-			.unwrap();
-		let authority_id = AuthorityId::from(public_key);
-
-		// Build genesis.
-		let mut t = system::GenesisConfig::default()
-			.build_storage::()
-			.unwrap();
-
-		GenesisConfig:: {
-			keys: vec![authority_id.clone()],
-		}
-		.assimilate_storage(&mut t)
-		.unwrap();
-
-		// Create externalities.
-		let mut externalities = TestExternalities::new(t);
-		externalities.register_extension(KeystoreExt(key_store));
-
-		externalities.execute_with(|| {
-			assert_eq!(
-				authority_id,
-				AuthorityDiscovery::authority_id().expect("Retrieving public key.")
-			);
-		});
-	}
-
-	#[test]
-	fn authority_id_fn_does_not_return_key_outside_current_authority_set() {
-		// Create keystore and generate key.
-		let key_store = KeyStore::new();
-		key_store
-			.write()
-			.sr25519_generate_new(key_types::BABE, None)
-			.expect("Generates key.");
-
-		// Build genesis.
-		let mut t = system::GenesisConfig::default()
-			.build_storage::()
-			.unwrap();
-
-		// Generate random authority set.
-		let keys = vec![(); 5]
-			.iter()
-			.map(|_x| sr25519::Pair::generate_with_phrase(None).0.public())
-			.map(AuthorityId::from)
-			.collect();
-
-		GenesisConfig:: { keys: keys }
-			.assimilate_storage(&mut t)
-			.unwrap();
-
-		// Create externalities.
-		let mut externalities = TestExternalities::new(t);
-		externalities.register_extension(KeystoreExt(key_store));
-
-		externalities.execute_with(|| {
-			assert_eq!(None, AuthorityDiscovery::authority_id());
-		});
-	}
-
-	#[test]
-	fn sign_and_verify_workflow() {
-		// Create keystore and generate key.
-		let key_store = KeyStore::new();
-		key_store
-			.write()
-			.sr25519_generate_new(key_types::BABE, None)
-			.expect("Generates key.");
-
-		// Retrieve key to later check if we got the right one.
-		let public_key = key_store
-			.read()
-			.sr25519_public_keys(key_types::BABE)
-			.pop()
-			.unwrap();
-		let authority_id = AuthorityId::from(public_key);
-
-		// Build genesis.
-		let mut t = system::GenesisConfig::default()
-			.build_storage::()
-			.unwrap();
-
-		GenesisConfig:: {
-			keys: vec![authority_id.clone()],
-		}
-		.assimilate_storage(&mut t)
-		.unwrap();
-
-		// Create externalities.
-		let mut externalities = TestExternalities::new(t);
-		externalities.register_extension(KeystoreExt(key_store));
-
-		externalities.execute_with(|| {
-			let payload = String::from("test payload").into_bytes();
-			let (sig, authority_id) = AuthorityDiscovery::sign(&payload).expect("signature");
-
-			assert!(AuthorityDiscovery::verify(
-				&payload,
-				sig.clone(),
-				authority_id.clone(),
-			));
-
-			assert!(!AuthorityDiscovery::verify(
-				&String::from("other payload").into_bytes(),
-				sig,
-				authority_id,
-			))
-		});
-	}
-}
diff --git a/srml/babe/Cargo.toml b/srml/babe/Cargo.toml
deleted file mode 100644
index dc82906650bef..0000000000000
--- a/srml/babe/Cargo.toml
+++ /dev/null
@@ -1,44 +0,0 @@
-[package]
-name = "srml-babe"
-version = "2.0.0"
-authors = ["Parity Technologies "]
-edition = "2018"
-
-[dependencies]
-hex-literal = "0.2.1"
-codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false, features = ["derive"] }
-serde = { version = "1.0.101", optional = true }
-inherents = { package = "substrate-inherents", path = "../../core/inherents", default-features = false }
-rstd = { package = "sr-std", path = "../../core/sr-std", default-features = false }
-sr-primitives = { path = "../../core/sr-primitives", default-features = false }
-sr-staking-primitives = { path = "../../core/sr-staking-primitives", default-features = false }
-support = { package = "srml-support", path = "../support", default-features = false }
-system = { package = "srml-system", path = "../system", default-features = false }
-timestamp = { package = "srml-timestamp", path = "../timestamp", default-features = false }
-session = { package = "srml-session", path = "../session", default-features = false }
-babe-primitives = { package = "substrate-consensus-babe-primitives", path = "../../core/consensus/babe/primitives", default-features = false }
-runtime-io ={ package = "sr-io", path = "../../core/sr-io", default-features = false }
-
-[dev-dependencies]
-lazy_static = "1.4.0"
-parking_lot = "0.9.0"
-sr-version = { path = "../../core/sr-version", default-features = false }
-primitives = { package = "substrate-primitives",  path = "../../core/primitives" }
-test-runtime = { package = "substrate-test-runtime", path = "../../core/test-runtime" }
-
-[features]
-default = ["std"]
-std = [
-	"serde",
-	"codec/std",
-	"rstd/std",
-	"support/std",
-	"sr-primitives/std",
-	"sr-staking-primitives/std",
-	"system/std",
-	"timestamp/std",
-	"inherents/std",
-	"babe-primitives/std",
-	"session/std",
-	"runtime-io/std",
-]
diff --git a/srml/elections-phragmen/Cargo.toml b/srml/elections-phragmen/Cargo.toml
deleted file mode 100644
index 85532849ecc00..0000000000000
--- a/srml/elections-phragmen/Cargo.toml
+++ /dev/null
@@ -1,31 +0,0 @@
-[package]
-name = "srml-elections-phragmen"
-version = "2.0.0"
-authors = ["Parity Technologies "]
-edition = "2018"
-
-[dependencies]
-codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false, features = ["derive"] }
-sr-primitives = { path = "../../core/sr-primitives", default-features = false }
-phragmen = { package = "substrate-phragmen", path = "../../core/phragmen", default-features = false }
-srml-support = { path = "../support", default-features = false }
-system = { package = "srml-system", path = "../system", default-features = false }
-rstd = { package = "sr-std", path = "../../core/sr-std", default-features = false }
-
-[dev-dependencies]
-runtime_io = { package = "sr-io", path = "../../core/sr-io" }
-hex-literal = "0.2.1"
-balances = { package = "srml-balances", path = "../balances" }
-primitives = { package = "substrate-primitives",  path = "../../core/primitives" }
-serde = { version = "1.0.101" }
-
-[features]
-default = ["std"]
-std = [
-	"codec/std",
-	"srml-support/std",
-	"sr-primitives/std",
-	"phragmen/std",
-	"system/std",
-	"rstd/std",
-]
diff --git a/srml/executive/Cargo.toml b/srml/executive/Cargo.toml
deleted file mode 100644
index 92191a7aac4db..0000000000000
--- a/srml/executive/Cargo.toml
+++ /dev/null
@@ -1,33 +0,0 @@
-[package]
-name = "srml-executive"
-version = "2.0.0"
-authors = ["Parity Technologies "]
-edition = "2018"
-
-[dependencies]
-serde = { version = "1.0.101", optional = true }
-codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false, features = ["derive"] }
-rstd = { package = "sr-std", path = "../../core/sr-std", default-features = false }
-runtime-io ={ package = "sr-io", path = "../../core/sr-io", default-features = false }
-sr-primitives = { path = "../../core/sr-primitives", default-features = false }
-support = { package = "srml-support", path = "../support", default-features = false }
-system = { package = "srml-system", path = "../system", default-features = false }
-
-[dev-dependencies]
-hex-literal = "0.2.1"
-primitives = { package = "substrate-primitives",  path = "../../core/primitives" }
-srml-indices = { path = "../indices" }
-balances = { package = "srml-balances", path = "../balances" }
-transaction-payment = { package = "srml-transaction-payment", path = "../transaction-payment" }
-
-[features]
-default = ["std"]
-std = [
-	"rstd/std",
-	"support/std",
-	"serde",
-	"codec/std",
-	"sr-primitives/std",
-	"runtime-io/std",
-	"system/std",
-]
diff --git a/srml/finality-tracker/Cargo.toml b/srml/finality-tracker/Cargo.toml
deleted file mode 100644
index 7e3439e0fecea..0000000000000
--- a/srml/finality-tracker/Cargo.toml
+++ /dev/null
@@ -1,31 +0,0 @@
-[package]
-name = "srml-finality-tracker"
-version = "2.0.0"
-authors = ["Parity Technologies "]
-edition = "2018"
-
-[dependencies]
-serde = { version = "1.0.101", default-features = false, features = ["derive"] }
-codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false }
-inherents = { package = "substrate-inherents", path = "../../core/inherents", default-features = false }
-rstd = { package = "sr-std", path = "../../core/sr-std", default-features = false }
-sr-primitives = { path = "../../core/sr-primitives", default-features = false }
-support = { package = "srml-support", path = "../support", default-features = false }
-srml-system = { path = "../system", default-features = false }
-impl-trait-for-tuples = "0.1.2"
-
-[dev-dependencies]
-primitives = { package = "substrate-primitives",  path = "../../core/primitives", default-features = false }
-runtime-io = { package = "sr-io", path = "../../core/sr-io", default-features = false }
-
-[features]
-default = ["std"]
-std = [
-	"serde/std",
-	"codec/std",
-	"rstd/std",
-	"support/std",
-	"sr-primitives/std",
-	"srml-system/std",
-	"inherents/std",
-]
diff --git a/srml/grandpa/Cargo.toml b/srml/grandpa/Cargo.toml
deleted file mode 100644
index 4b494cfeff8d1..0000000000000
--- a/srml/grandpa/Cargo.toml
+++ /dev/null
@@ -1,37 +0,0 @@
-[package]
-name = "srml-grandpa"
-version = "2.0.0"
-authors = ["Parity Technologies "]
-edition = "2018"
-
-[dependencies]
-serde = { version = "1.0.101", optional = true, features = ["derive"] }
-codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false, features = ["derive"] }
-primitives = { package = "substrate-primitives",  path = "../../core/primitives", default-features = false }
-substrate-finality-grandpa-primitives = { path = "../../core/finality-grandpa/primitives", default-features = false }
-rstd = { package = "sr-std", path = "../../core/sr-std", default-features = false }
-sr-primitives = { path = "../../core/sr-primitives", default-features = false }
-sr-staking-primitives = { path = "../../core/sr-staking-primitives", default-features = false }
-support = { package = "srml-support", path = "../support", default-features = false }
-system = { package = "srml-system", path = "../system", default-features = false }
-session = { package = "srml-session", path = "../session", default-features = false }
-finality-tracker = { package = "srml-finality-tracker", path = "../finality-tracker", default-features = false }
-
-[dev-dependencies]
-runtime-io ={ package = "sr-io", path = "../../core/sr-io" }
-
-[features]
-default = ["std"]
-std = [
-	"serde",
-	"codec/std",
-	"primitives/std",
-	"substrate-finality-grandpa-primitives/std",
-	"rstd/std",
-	"support/std",
-	"sr-primitives/std",
-	"sr-staking-primitives/std",
-	"system/std",
-	"session/std",
-	"finality-tracker/std",
-]
diff --git a/srml/im-online/Cargo.toml b/srml/im-online/Cargo.toml
deleted file mode 100644
index b62cc2a8f9615..0000000000000
--- a/srml/im-online/Cargo.toml
+++ /dev/null
@@ -1,37 +0,0 @@
-[package]
-name = "srml-im-online"
-version = "0.1.0"
-authors = ["Parity Technologies "]
-edition = "2018"
-
-[dependencies]
-app-crypto = { package = "substrate-application-crypto", path = "../../core/application-crypto", default-features = false }
-codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false, features = ["derive"] }
-primitives = { package="substrate-primitives", path = "../../core/primitives", default-features = false }
-rstd = { package = "sr-std", path = "../../core/sr-std", default-features = false }
-serde = { version = "1.0.101", optional = true }
-session = { package = "srml-session", path = "../session", default-features = false }
-runtime-io = { package = "sr-io", path = "../../core/sr-io", default-features = false }
-sr-primitives = { path = "../../core/sr-primitives", default-features = false }
-sr-staking-primitives = { path = "../../core/sr-staking-primitives", default-features = false }
-support = { package = "srml-support", path = "../support", default-features = false }
-system = { package = "srml-system", path = "../system", default-features = false }
-
-[dev-dependencies]
-offchain = { package = "substrate-offchain", path = "../../core/offchain" }
-
-[features]
-default = ["std", "session/historical"]
-std = [
-	"app-crypto/std",
-	"codec/std",
-	"primitives/std",
-	"rstd/std",
-	"serde",
-	"session/std",
-	"runtime-io/std",
-	"sr-primitives/std",
-	"sr-staking-primitives/std",
-	"support/std",
-	"system/std",
-]
diff --git a/srml/offences/Cargo.toml b/srml/offences/Cargo.toml
deleted file mode 100644
index 90a3396d11228..0000000000000
--- a/srml/offences/Cargo.toml
+++ /dev/null
@@ -1,32 +0,0 @@
-[package]
-name = "srml-offences"
-version = "1.0.0"
-authors = ["Parity Technologies "]
-edition = "2018"
-
-[dependencies]
-balances = { package = "srml-balances", path = "../balances", default-features = false }
-codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false, features = ["derive"] }
-rstd = { package = "sr-std", path = "../../core/sr-std", default-features = false }
-serde = { version = "1.0.101", optional = true }
-sr-primitives = { path = "../../core/sr-primitives", default-features = false }
-sr-staking-primitives = { path = "../../core/sr-staking-primitives", default-features = false }
-support = { package = "srml-support", path = "../support", default-features = false }
-system = { package = "srml-system", path = "../system", default-features = false }
-
-[dev-dependencies]
-runtime-io = { package = "sr-io", path = "../../core/sr-io" }
-substrate-primitives = { path = "../../core/primitives" }
-
-[features]
-default = ["std"]
-std = [
-	"balances/std",
-	"codec/std",
-	"rstd/std",
-	"serde",
-	"sr-primitives/std",
-	"sr-staking-primitives/std",
-	"support/std",
-	"system/std",
-]
diff --git a/srml/staking/Cargo.toml b/srml/staking/Cargo.toml
deleted file mode 100644
index 67f3ab41d6583..0000000000000
--- a/srml/staking/Cargo.toml
+++ /dev/null
@@ -1,45 +0,0 @@
-[package]
-name = "srml-staking"
-version = "2.0.0"
-authors = ["Parity Technologies "]
-edition = "2018"
-
-[dependencies]
-serde = { version = "1.0.101", optional = true }
-safe-mix = { version = "1.0.0", default-features = false }
-codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false, features = ["derive"] }
-substrate-keyring = { path = "../../core/keyring", optional = true }
-rstd = { package = "sr-std", path = "../../core/sr-std", default-features = false }
-phragmen = { package = "substrate-phragmen", path = "../../core/phragmen", default-features = false }
-runtime-io ={ package = "sr-io", path = "../../core/sr-io", default-features = false }
-sr-primitives = { path = "../../core/sr-primitives", default-features = false }
-sr-staking-primitives = { path = "../../core/sr-staking-primitives", default-features = false }
-support = { package = "srml-support", path = "../support", default-features = false }
-system = { package = "srml-system", path = "../system", default-features = false }
-session = { package = "srml-session", path = "../session", default-features = false, features = ["historical"] }
-authorship = { package = "srml-authorship", path = "../authorship", default-features = false }
-
-[dev-dependencies]
-primitives = { package = "substrate-primitives",  path = "../../core/primitives" }
-balances = { package = "srml-balances", path = "../balances" }
-timestamp = { package = "srml-timestamp", path = "../timestamp" }
-srml-staking-reward-curve = { path = "../staking/reward-curve"}
-
-[features]
-equalize = []
-default = ["std", "equalize"]
-std = [
-	"serde",
-	"safe-mix/std",
-	"substrate-keyring",
-	"codec/std",
-	"rstd/std",
-	"phragmen/std",
-	"runtime-io/std",
-	"support/std",
-	"sr-primitives/std",
-	"sr-staking-primitives/std",
-	"session/std",
-	"system/std",
-	"authorship/std",
-]
diff --git a/srml/support/Cargo.toml b/srml/support/Cargo.toml
deleted file mode 100644
index 2b8c4cb9f98b8..0000000000000
--- a/srml/support/Cargo.toml
+++ /dev/null
@@ -1,41 +0,0 @@
-[package]
-name = "srml-support"
-version = "2.0.0"
-authors = ["Parity Technologies "]
-edition = "2018"
-
-[dependencies]
-log = "0.4"
-serde = { version = "1.0.101", optional = true, features = ["derive"] }
-codec = { package = "parity-scale-codec", version = "1.0.6", default-features = false, features = ["derive"] }
-srml-metadata = { path = "../metadata", default-features = false }
-rstd = { package = "sr-std", path = "../../core/sr-std", default-features = false }
-runtime-io ={ package = "sr-io", path = "../../core/sr-io", default-features = false }
-sr-primitives = { path = "../../core/sr-primitives", default-features = false }
-primitives = { package = "substrate-primitives",  path = "../../core/primitives", default-features = false }
-inherents = { package = "substrate-inherents", path = "../../core/inherents", default-features = false }
-srml-support-procedural = { package = "srml-support-procedural", path = "./procedural" }
-paste = "0.1.6"
-once_cell = { version = "0.2.4", default-features = false, optional = true }
-bitmask = { version = "0.5.0", default-features = false }
-impl-trait-for-tuples = "0.1.2"
-
-[dev-dependencies]
-pretty_assertions = "0.6.1"
-srml-system = { path = "../system" }
-
-[features]
-default = ["std"]
-std = [
-	"once_cell",
-	"bitmask/std",
-	"serde",
-	"runtime-io/std",
-	"codec/std",
-	"rstd/std",
-	"sr-primitives/std",
-	"srml-metadata/std",
-	"inherents/std",
-]
-nightly = []
-strict = []
diff --git a/srml/support/procedural/Cargo.toml b/srml/support/procedural/Cargo.toml
deleted file mode 100644
index 91b4ca507497e..0000000000000
--- a/srml/support/procedural/Cargo.toml
+++ /dev/null
@@ -1,16 +0,0 @@
-[package]
-name = "srml-support-procedural"
-version = "2.0.0"
-authors = ["Parity Technologies "]
-edition = "2018"
-
-[lib]
-proc-macro = true
-
-[dependencies]
-srml-support-procedural-tools = { package = "srml-support-procedural-tools", path = "./tools" }
-sr-api-macros = { path = "../../../core/sr-api-macros" }
-
-proc-macro2 = "0.4.27"
-quote = "0.6.12"
-syn = { version = "0.15.44", features = ["full"] }
diff --git a/srml/support/procedural/tools/Cargo.toml b/srml/support/procedural/tools/Cargo.toml
deleted file mode 100644
index 62c55a703f4ed..0000000000000
--- a/srml/support/procedural/tools/Cargo.toml
+++ /dev/null
@@ -1,12 +0,0 @@
-[package]
-name = "srml-support-procedural-tools"
-version = "2.0.0"
-authors = ["Parity Technologies "]
-edition = "2018"
-
-[dependencies]
-srml-support-procedural-tools-derive = { package = "srml-support-procedural-tools-derive", path = "./derive" }
-proc-macro2 = "0.4.27"
-quote = "0.6.12"
-syn = { version = "0.15.44", features = ["full"] }
-proc-macro-crate = "0.1.4"
diff --git a/srml/support/procedural/tools/derive/Cargo.toml b/srml/support/procedural/tools/derive/Cargo.toml
deleted file mode 100644
index 54bfaba838434..0000000000000
--- a/srml/support/procedural/tools/derive/Cargo.toml
+++ /dev/null
@@ -1,13 +0,0 @@
-[package]
-name = "srml-support-procedural-tools-derive"
-version = "2.0.0"
-authors = ["Parity Technologies "]
-edition = "2018"
-
-[lib]
-proc-macro = true
-
-[dependencies]
-proc-macro2 = "0.4.27"
-quote = { version = "0.6.12", features = ["proc-macro"] }
-syn = { version = "0.15.44", features = ["proc-macro" ,"full", "extra-traits", "parsing"] }
diff --git a/srml/support/src/storage/generator/mod.rs b/srml/support/src/storage/generator/mod.rs
deleted file mode 100644
index ab7616158afd8..0000000000000
--- a/srml/support/src/storage/generator/mod.rs
+++ /dev/null
@@ -1,32 +0,0 @@
-// Copyright 2019 Parity Technologies (UK) Ltd.
-// This file is part of Substrate.
-
-// Substrate is free software: you can redistribute it and/or modify
-// it under the terms of the GNU General Public License as published by
-// the Free Software Foundation, either version 3 of the License, or
-// (at your option) any later version.
-
-// Substrate is distributed in the hope that it will be useful,
-// but WITHOUT ANY WARRANTY; without even the implied warranty of
-// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-// GNU General Public License for more details.
-
-// You should have received a copy of the GNU General Public License
-// along with Substrate.  If not, see .
-
-//! Generators are a set of trait on which storage traits are implemented.
-//!
-//! (i.e. implementing the generator for StorageValue on a type will automatically derive the
-//! implementation of StorageValue for this type).
-//!
-//! They are used by `decl_storage`.
-
-mod linked_map;
-mod map;
-mod double_map;
-mod value;
-
-pub use linked_map::{StorageLinkedMap, Enumerator, Linkage};
-pub use map::StorageMap;
-pub use double_map::StorageDoubleMap;
-pub use value::StorageValue;
diff --git a/srml/timestamp/Cargo.toml b/srml/timestamp/Cargo.toml
deleted file mode 100644
index 939fb72566608..0000000000000
--- a/srml/timestamp/Cargo.toml
+++ /dev/null
@@ -1,31 +0,0 @@
-[package]
-name = "srml-timestamp"
-version = "2.0.0"
-authors = ["Parity Technologies "]
-edition = "2018"
-
-[dependencies]
-serde = { version = "1.0.101", optional = true }
-codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false, features = ["derive"] }
-rstd = { package = "sr-std", path = "../../core/sr-std", default-features = false }
-sr-primitives = { path = "../../core/sr-primitives", default-features = false }
-inherents = { package = "substrate-inherents", path = "../../core/inherents", default-features = false }
-support = { package = "srml-support", path = "../support", default-features = false }
-system = { package = "srml-system", path = "../system", default-features = false }
-impl-trait-for-tuples = "0.1.2"
-
-[dev-dependencies]
-runtime-io ={ package = "sr-io", path = "../../core/sr-io" }
-primitives = { package = "substrate-primitives",  path = "../../core/primitives" }
-
-[features]
-default = ["std"]
-std = [
-	"inherents/std",
-	"codec/std",
-	"rstd/std",
-	"sr-primitives/std",
-	"support/std",
-	"serde",
-	"system/std",
-]
diff --git a/srml/transaction-payment/Cargo.toml b/srml/transaction-payment/Cargo.toml
deleted file mode 100644
index cbce1e945e69b..0000000000000
--- a/srml/transaction-payment/Cargo.toml
+++ /dev/null
@@ -1,29 +0,0 @@
-[package]
-name = "srml-transaction-payment"
-version = "2.0.0"
-authors = ["Parity Technologies "]
-edition = "2018"
-
-[dependencies]
-codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false, features = ["derive"] }
-rstd = { package = "sr-std", path = "../../core/sr-std", default-features = false }
-sr-primitives = { path = "../../core/sr-primitives", default-features = false }
-support = { package = "srml-support", path = "../support", default-features = false }
-system = { package = "srml-system", path = "../system", default-features = false }
-transaction-payment-rpc-runtime-api = { package = "srml-transaction-payment-rpc-runtime-api", path = "./rpc/runtime-api", default-features = false }
-
-[dev-dependencies]
-runtime-io = { package = "sr-io", path = "../../core/sr-io" }
-primitives = { package = "substrate-primitives",  path = "../../core/primitives" }
-balances = { package = "srml-balances", path = "../balances" }
-
-[features]
-default = ["std"]
-std = [
-	"codec/std",
-	"rstd/std",
-	"sr-primitives/std",
-	"support/std",
-	"system/std",
-	"transaction-payment-rpc-runtime-api/std"
-]
diff --git a/srml/transaction-payment/rpc/Cargo.toml b/srml/transaction-payment/rpc/Cargo.toml
deleted file mode 100644
index e3dc6f553fc0c..0000000000000
--- a/srml/transaction-payment/rpc/Cargo.toml
+++ /dev/null
@@ -1,17 +0,0 @@
-[package]
-name = "srml-transaction-payment-rpc"
-version = "2.0.0"
-authors = ["Parity Technologies "]
-edition = "2018"
-
-[dependencies]
-client = { package = "substrate-client", path = "../../../core/client" }
-codec = { package = "parity-scale-codec", version = "1.0.0" }
-jsonrpc-core = "13.2.0"
-jsonrpc-core-client = "13.2.0"
-jsonrpc-derive = "13.2.0"
-primitives = { package = "substrate-primitives",  path = "../../../core/primitives" }
-rpc-primitives = { package = "substrate-rpc-primitives", path = "../../../core/rpc/primitives" }
-serde = { version = "1.0.101", features = ["derive"] }
-sr-primitives = { path = "../../../core/sr-primitives" }
-srml-transaction-payment-rpc-runtime-api = { path = "./runtime-api" }
diff --git a/test-utils/transaction-factory/Cargo.toml b/test-utils/transaction-factory/Cargo.toml
deleted file mode 100644
index e53972eec65c5..0000000000000
--- a/test-utils/transaction-factory/Cargo.toml
+++ /dev/null
@@ -1,22 +0,0 @@
-[package]
-name = "transaction-factory"
-version = "0.0.1"
-authors = ["Parity Technologies "]
-edition = "2018"
-
-[dependencies]
-cli = { package = "substrate-cli", path = "../../core/cli" }
-client = { package = "substrate-client", path = "../../core/client" }
-consensus_common = { package = "substrate-consensus-common", path = "../../core/consensus/common" }
-log = "0.4.8"
-codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false, features = ["derive"] }
-primitives = { package = "substrate-primitives", path = "../../core/primitives", default-features = false }
-sr-primitives = { path = "../../core/sr-primitives", default-features = false }
-substrate-service = { path = "../../core/service" }
-
-[features]
-default = ["std"]
-std = [
-	"codec/std",
-	"sr-primitives/std",
-]
diff --git a/test-utils/chain-spec-builder/Cargo.toml b/test/utils/chain-spec-builder/Cargo.toml
similarity index 65%
rename from test-utils/chain-spec-builder/Cargo.toml
rename to test/utils/chain-spec-builder/Cargo.toml
index e8ca8d354025a..7498880e0f566 100644
--- a/test-utils/chain-spec-builder/Cargo.toml
+++ b/test/utils/chain-spec-builder/Cargo.toml
@@ -7,8 +7,8 @@ build = "build.rs"
 
 [dependencies]
 ansi_term = "0.12.1"
-keystore = { package = "substrate-keystore", path = "../../core/keystore" }
-node-cli = { path = "../../node/cli" }
-primitives = { package = "substrate-primitives",  path = "../../core/primitives" }
+keystore = { package = "substrate-keystore", path = "../../../client/keystore" }
+node-cli = { path = "../../../bin/node/cli" }
+primitives = { package = "substrate-primitives",  path = "../../../primitives/core" }
 rand = "0.7.2"
 structopt = "0.3.3"
diff --git a/test-utils/chain-spec-builder/build.rs b/test/utils/chain-spec-builder/build.rs
similarity index 100%
rename from test-utils/chain-spec-builder/build.rs
rename to test/utils/chain-spec-builder/build.rs
diff --git a/test-utils/chain-spec-builder/src/main.rs b/test/utils/chain-spec-builder/src/main.rs
similarity index 97%
rename from test-utils/chain-spec-builder/src/main.rs
rename to test/utils/chain-spec-builder/src/main.rs
index d46f4b7f0bc98..b277987e1ea5e 100644
--- a/test-utils/chain-spec-builder/src/main.rs
+++ b/test/utils/chain-spec-builder/src/main.rs
@@ -141,7 +141,7 @@ fn generate_authority_keys_and_store(
 			None,
 		).map_err(|err| err.to_string())?;
 
-		let (_, _, grandpa, babe, im_online) =
+		let (_, _, grandpa, babe, im_online, authority_discovery) =
 			chain_spec::get_authority_keys_from_seed(seed);
 
 		let insert_key = |key_type, public| {
@@ -166,6 +166,11 @@ fn generate_authority_keys_and_store(
 			primitives::crypto::key_types::IM_ONLINE,
 			im_online.as_slice(),
 		)?;
+
+		insert_key(
+			primitives::crypto::key_types::AUTHORITY_DISCOVERY,
+			authority_discovery.as_slice(),
+		)?;
 	}
 
 	Ok(())
diff --git a/test/utils/client/Cargo.toml b/test/utils/client/Cargo.toml
new file mode 100644
index 0000000000000..a000d8186f8e6
--- /dev/null
+++ b/test/utils/client/Cargo.toml
@@ -0,0 +1,19 @@
+[package]
+name = "substrate-test-client"
+version = "2.0.0"
+authors = ["Parity Technologies "]
+edition = "2018"
+
+[dependencies]
+client-api = { package = "substrate-client-api", path = "../../../client/api" }
+client = { package = "substrate-client", path = "../../../client/" }
+client-db = { package = "substrate-client-db", path = "../../../client//db", features = ["test-helpers"] }
+consensus = { package = "substrate-consensus-common", path = "../../../primitives/consensus/common" }
+executor = { package = "substrate-executor", path = "../../../client/executor" }
+futures-preview = "0.3.0-alpha.19"
+hash-db = "0.15.2"
+keyring = { package = "substrate-keyring", path = "../../../primitives/keyring" }
+codec = { package = "parity-scale-codec", version = "1.0.0" }
+primitives = { package = "substrate-primitives", path = "../../../primitives/core" }
+sr-primitives = {  path = "../../../primitives/sr-primitives" }
+state_machine = { package = "substrate-state-machine", path = "../../../primitives/state-machine" }
diff --git a/core/test-client/src/client_ext.rs b/test/utils/client/src/client_ext.rs
similarity index 82%
rename from core/test-client/src/client_ext.rs
rename to test/utils/client/src/client_ext.rs
index 7d3d7301c55d3..d7ae5097a3196 100644
--- a/core/test-client/src/client_ext.rs
+++ b/test/utils/client/src/client_ext.rs
@@ -16,7 +16,8 @@
 
 //! Client extension for tests.
 
-use client::{self, Client, backend::Finalizer};
+use client::{self, Client};
+use client_api::backend::Finalizer;
 use consensus::{
 	BlockImportParams, BlockImport, BlockOrigin, Error as ConsensusError,
 	ForkChoiceStrategy,
@@ -38,6 +39,10 @@ pub trait ClientExt: Sized {
 	fn import_as_best(&self, origin: BlockOrigin, block: Block)
 		-> Result<(), ConsensusError>;
 
+	/// Import a block and finalize it.
+	fn import_as_final(&self, origin: BlockOrigin, block: Block)
+		-> Result<(), ConsensusError>;
+
 	/// Import block with justification, finalizes block.
 	fn import_justified(
 		&self,
@@ -59,7 +64,7 @@ pub trait ClientExt: Sized {
 
 impl ClientExt for Client
 	where
-		B: client::backend::Backend,
+		B: client_api::backend::Backend,
 		E: client::CallExecutor,
 		for<'r> &'r Self: BlockImport,
 		Block: BlockT::Out>,
@@ -77,6 +82,7 @@ impl ClientExt for Client
 			finalized: false,
 			auxiliary: Vec::new(),
 			fork_choice: ForkChoiceStrategy::LongestChain,
+			allow_missing_state: false,
 		};
 
 		BlockImport::import_block(&mut (&*self), import, HashMap::new()).map(|_| ())
@@ -95,6 +101,26 @@ impl ClientExt for Client
 			finalized: false,
 			auxiliary: Vec::new(),
 			fork_choice: ForkChoiceStrategy::Custom(true),
+			allow_missing_state: false,
+		};
+
+		BlockImport::import_block(&mut (&*self), import, HashMap::new()).map(|_| ())
+	}
+
+	fn import_as_final(&self, origin: BlockOrigin, block: Block)
+		-> Result<(), ConsensusError>
+	{
+		let (header, extrinsics) = block.deconstruct();
+		let import = BlockImportParams {
+			origin,
+			header,
+			justification: None,
+			post_digests: vec![],
+			body: Some(extrinsics),
+			finalized: true,
+			auxiliary: Vec::new(),
+			fork_choice: ForkChoiceStrategy::Custom(true),
+			allow_missing_state: false,
 		};
 
 		BlockImport::import_block(&mut (&*self), import, HashMap::new()).map(|_| ())
@@ -116,6 +142,7 @@ impl ClientExt for Client
 			finalized: true,
 			auxiliary: Vec::new(),
 			fork_choice: ForkChoiceStrategy::LongestChain,
+			allow_missing_state: false,
 		};
 
 		BlockImport::import_block(&mut (&*self), import, HashMap::new()).map(|_| ())
diff --git a/core/test-client/src/lib.rs b/test/utils/client/src/lib.rs
similarity index 93%
rename from core/test-client/src/lib.rs
rename to test/utils/client/src/lib.rs
index dbe4431456a74..7143b67599199 100644
--- a/core/test-client/src/lib.rs
+++ b/test/utils/client/src/lib.rs
@@ -20,7 +20,7 @@
 
 pub mod client_ext;
 
-pub use client::{ExecutionStrategies, blockchain, backend, self};
+pub use client::{ExecutionStrategies, blockchain, self};
 pub use client_db::{Backend, self};
 pub use client_ext::ClientExt;
 pub use consensus;
@@ -98,6 +98,12 @@ impl TestClientBuilder<
 	pub fn backend(&self) -> Arc> {
 		self.backend.clone()
 	}
+
+	/// Create new `TestClientBuilder` with default backend and pruning window size
+	pub fn with_pruning_window(keep_blocks: u32) -> Self {
+		let backend = Arc::new(Backend::new_test(keep_blocks, 0));
+		Self::with_backend(backend)
+	}
 }
 
 impl TestClientBuilder {
@@ -165,7 +171,7 @@ impl TestClientBuilder
 		client::LongestChain,
 	) where
 		Executor: client::CallExecutor,
-		Backend: client::backend::Backend,
+		Backend: client_api::backend::Backend,
 		Block: BlockT::Out>,
 	{
 
@@ -217,7 +223,7 @@ impl TestClientBuilder<
 	) where
 		I: Into>>,
 		E: executor::NativeExecutionDispatch,
-		Backend: client::backend::Backend,
+		Backend: client_api::backend::Backend,
 		Block: BlockT::Out>,
 	{
 		let executor = executor.into().unwrap_or_else(||
diff --git a/test/utils/primitives/Cargo.toml b/test/utils/primitives/Cargo.toml
new file mode 100644
index 0000000000000..b3d200f95defb
--- /dev/null
+++ b/test/utils/primitives/Cargo.toml
@@ -0,0 +1,71 @@
+[package]
+name = "substrate-test-primitives"
+version = "2.0.0"
+authors = ["Parity Technologies "]
+edition = "2018"
+
+[dependencies]
+app-crypto = { package = "substrate-application-crypto", path = "../../../primitives/application-crypto", default-features = false }
+codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false, features = ["derive"] }
+# aura-primitives = { package = "substrate-consensus-aura-primitives", path = "../../../primitives/consensus/aura", default-features = false }
+# babe-primitives = { package = "substrate-consensus-babe-primitives", path = "../../../primitives/consensus/babe", default-features = false }
+# block-builder-api = { package = "substrate-block-builder-runtime-api", path = "../../../primitives/block-builder/runtime-api", default-features = false }
+# inherents = { package = "substrate-inherents", path = "../../../primitives/inherents", default-features = false }
+# log = { version = "0.4.8", optional = true }
+# memory-db = { version = "0.15.2", default-features = false }
+# offchain-primitives = { package = "substrate-offchain-primitives", path = "../../../primitives/offchain", default-features = false}
+primitives = { package = "substrate-primitives", path = "../../../primitives/core", default-features = false }
+# rstd = { package = "sr-std", path = "../../../primitives/sr-std", default-features = false }
+# runtime-interface = { package = "substrate-runtime-interface", path = "../../../primitives/runtime-interface", default-features = false}
+# runtime_io = { package = "sr-io", path = "../../../primitives/sr-io", default-features = false }
+# runtime_support = { package = "paint-support", path = "../../../paint/support", default-features = false }
+# runtime_version = { package = "sr-version", path = "../../../primitives/sr-version", default-features = false }
+serde = { version = "1.0.101", optional = true, features = ["derive"] }
+# session = { package = "substrate-session", path = "../../../primitives/session", default-features = false }
+# sr-api = { path = "../../../primitives/sr-api", default-features = false }
+sr-primitives = {  path = "../../../primitives/sr-primitives", default-features = false }
+# paint-babe = { path = "../../../paint/babe", default-features = false }
+# paint-system = { path = "../../../paint/system", default-features = false }
+# paint-system-rpc-runtime-api = { path = "../../../paint/system/rpc/runtime-api", default-features = false }
+# paint-timestamp = { path = "../../../paint/timestamp", default-features = false }
+# substrate-client = { path = "../../../client", optional = true }
+# substrate-trie = { path = "../../../primitives/trie", default-features = false }
+# transaction-pool-api = { package = "substrate-transaction-pool-runtime-api", path = "../../../primitives/transaction-pool/runtime-api", default-features = false }
+# trie-db = { version = "0.15.2", default-features = false }
+
+[features]
+default = [
+	"std",
+]
+std = [
+	"app-crypto/std",
+	# "aura-primitives/std",
+	# "babe-primitives/std",
+	# "block-builder-api/std",
+	# "codec/std",
+	# "executive/std",
+	# "inherents/std",
+	# "keyring",
+	# "log",
+	# "memory-db/std",
+	# "offchain-primitives/std",
+	# "primitives/std",
+	# "primitives/std",
+	# "rstd/std",
+	# "runtime-interface/std",
+	# "runtime_io/std",
+	# "runtime_support/std",
+	# "runtime_version/std",
+	"serde",
+	# "session/std",
+	# "sr-api/std",
+	# "sr-primitives/std",
+	# "paint-babe/std",
+	# "paint-system-rpc-runtime-api/std",
+	# "paint-system/std",
+	# "paint-timestamp/std",
+	# "substrate-client",
+	# "substrate-trie/std",
+	# "transaction-pool-api/std",
+	# "trie-db/std",
+]
diff --git a/test/utils/primitives/src/lib.rs b/test/utils/primitives/src/lib.rs
new file mode 100644
index 0000000000000..d30b9eabf6fa8
--- /dev/null
+++ b/test/utils/primitives/src/lib.rs
@@ -0,0 +1,78 @@
+// Copyright 2017-2019 Parity Technologies (UK) Ltd.
+// This file is part of Substrate.
+
+// Substrate is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+
+// Substrate is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+
+// You should have received a copy of the GNU General Public License
+// along with Substrate.  If not, see .
+
+//! The Substrate test primitives to share 
+
+#![cfg_attr(not(feature = "std"), no_std)]
+
+use codec::{Encode, Decode};
+
+use app_crypto::sr25519;
+pub use app_crypto;
+
+pub use primitives::{hash::H256, RuntimeDebug};
+use sr_primitives::traits::{BlakeTwo256, Verify, Extrinsic as ExtrinsicT,};
+
+/// Extrinsic for test-runtime.
+#[derive(Clone, PartialEq, Eq, Encode, Decode, RuntimeDebug)]
+pub enum Extrinsic {
+	IncludeData(Vec),
+	StorageChange(Vec, Option>),
+}
+
+#[cfg(feature = "std")]
+impl serde::Serialize for Extrinsic {
+	fn serialize(&self, seq: S) -> Result where S: ::serde::Serializer {
+		self.using_encoded(|bytes| seq.serialize_bytes(bytes))
+	}
+}
+
+
+impl ExtrinsicT for Extrinsic {
+	type Call = Extrinsic;
+	type SignaturePayload = ();
+
+	fn is_signed(&self) -> Option {
+		if let Extrinsic::IncludeData(_) = *self {
+			Some(false)
+		} else {
+			Some(true)
+		}
+	}
+
+	fn new(call: Self::Call, _signature_payload: Option) -> Option {
+		Some(call)
+	}
+}
+
+/// The signature type used by accounts/transactions.
+pub type AccountSignature = sr25519::Signature;
+/// An identifier for an account on this system.
+pub type AccountId = ::Signer;
+/// A simple hash type for all our hashing.
+pub type Hash = H256;
+/// The block number type used in this runtime.
+pub type BlockNumber = u64;
+/// Index of a transaction.
+pub type Index = u64;
+/// The item of a block digest.
+pub type DigestItem = sr_primitives::generic::DigestItem;
+/// The digest of a block.
+pub type Digest = sr_primitives::generic::Digest;
+/// A test block.
+pub type Block = sr_primitives::generic::Block;
+/// A test block's header.
+pub type Header = sr_primitives::generic::Header;
diff --git a/test/utils/runtime/Cargo.toml b/test/utils/runtime/Cargo.toml
new file mode 100644
index 0000000000000..c405451fa8602
--- /dev/null
+++ b/test/utils/runtime/Cargo.toml
@@ -0,0 +1,83 @@
+[package]
+name = "substrate-test-runtime"
+version = "2.0.0"
+authors = ["Parity Technologies "]
+edition = "2018"
+build = "build.rs"
+
+[dependencies]
+app-crypto = { package = "substrate-application-crypto", path = "../../../primitives/application-crypto", default-features = false }
+aura-primitives = { package = "substrate-consensus-aura-primitives", path = "../../../primitives/consensus/aura", default-features = false }
+babe-primitives = { package = "substrate-consensus-babe-primitives", path = "../../../primitives/consensus/babe", default-features = false }
+block-builder-api = { package = "substrate-block-builder-runtime-api", path = "../../../primitives/block-builder/runtime-api", default-features = false }
+cfg-if = "0.1.10"
+codec = { package = "parity-scale-codec", version = "1.0.0", default-features = false, features = ["derive"] }
+executive = { package = "paint-executive", path = "../../../paint/executive", default-features = false }
+inherents = { package = "substrate-inherents", path = "../../../primitives/inherents", default-features = false }
+keyring = { package = "substrate-keyring", path = "../../../primitives/keyring", optional = true }
+log = { version = "0.4.8", optional = true }
+memory-db = { version = "0.15.2", default-features = false }
+offchain-primitives = { package = "substrate-offchain-primitives", path = "../../../primitives/offchain", default-features = false}
+primitives = { package = "substrate-primitives", path = "../../../primitives/core", default-features = false }
+rstd = { package = "sr-std", path = "../../../primitives/sr-std", default-features = false }
+runtime-interface = { package = "substrate-runtime-interface", path = "../../../primitives/runtime-interface", default-features = false}
+runtime_io = { package = "sr-io", path = "../../../primitives/sr-io", default-features = false }
+runtime_support = { package = "paint-support", path = "../../../paint/support", default-features = false }
+runtime_version = { package = "sr-version", path = "../../../primitives/sr-version", default-features = false }
+serde = { version = "1.0.101", optional = true, features = ["derive"] }
+session = { package = "substrate-session", path = "../../../primitives/session", default-features = false }
+sr-api = { path = "../../../primitives/sr-api", default-features = false }
+sr-primitives = {  path = "../../../primitives/sr-primitives", default-features = false }
+paint-babe = { path = "../../../paint/babe", default-features = false }
+paint-system = { path = "../../../paint/system", default-features = false }
+paint-system-rpc-runtime-api = { path = "../../../paint/system/rpc/runtime-api", default-features = false }
+paint-timestamp = { path = "../../../paint/timestamp", default-features = false }
+substrate-client = { path = "../../../client", optional = true }
+substrate-trie = { path = "../../../primitives/trie", default-features = false }
+transaction-pool-api = { package = "substrate-transaction-pool-runtime-api", path = "../../../primitives/transaction-pool/runtime-api", default-features = false }
+trie-db = { version = "0.15.2", default-features = false }
+
+[dev-dependencies]
+substrate-executor = { path = "../../../client/executor" }
+substrate-test-runtime-client = { path = "./client" }
+state_machine = { package = "substrate-state-machine", path = "../../../primitives/state-machine" }
+
+[build-dependencies]
+wasm-builder-runner = { package = "substrate-wasm-builder-runner", path = "../../../client/utils/wasm-builder-runner", version = "1.0.4" }
+
+[features]
+default = [
+	"std",
+]
+std = [
+	"app-crypto/std",
+	"aura-primitives/std",
+	"babe-primitives/std",
+	"block-builder-api/std",
+	"codec/std",
+	"executive/std",
+	"inherents/std",
+	"keyring",
+	"log",
+	"memory-db/std",
+	"offchain-primitives/std",
+	"primitives/std",
+	"primitives/std",
+	"rstd/std",
+	"runtime-interface/std",
+	"runtime_io/std",
+	"runtime_support/std",
+	"runtime_version/std",
+	"serde",
+	"session/std",
+	"sr-api/std",
+	"sr-primitives/std",
+	"paint-babe/std",
+	"paint-system-rpc-runtime-api/std",
+	"paint-system/std",
+	"paint-timestamp/std",
+	"substrate-client",
+	"substrate-trie/std",
+	"transaction-pool-api/std",
+	"trie-db/std",
+]
diff --git a/core/test-runtime/build.rs b/test/utils/runtime/build.rs
similarity index 97%
rename from core/test-runtime/build.rs
rename to test/utils/runtime/build.rs
index 200cd6d42c2d7..cd2c502e78c45 100644
--- a/core/test-runtime/build.rs
+++ b/test/utils/runtime/build.rs
@@ -20,7 +20,7 @@ fn main() {
 	build_current_project_with_rustflags(
 		"wasm_binary.rs",
 		WasmBuilderSource::CratesOrPath {
-			path: "../utils/wasm-builder",
+			path: "../../../utils/wasm-builder",
 			version: "1.0.8",
 		},
 		// Note that we set the stack-size to 1MB explicitly even though it is set
diff --git a/test/utils/runtime/client/Cargo.toml b/test/utils/runtime/client/Cargo.toml
new file mode 100644
index 0000000000000..68f5820062100
--- /dev/null
+++ b/test/utils/runtime/client/Cargo.toml
@@ -0,0 +1,15 @@
+[package]
+name = "substrate-test-runtime-client"
+version = "2.0.0"
+authors = ["Parity Technologies "]
+edition = "2018"
+
+[dependencies]
+block-builder = { package = "substrate-block-builder", path = "../../../../client/block-builder" }
+generic-test-client = { package = "substrate-test-client", path = "../../client" }
+primitives = { package = "substrate-primitives", path = "../../../../primitives/core" }
+runtime = { package = "substrate-test-runtime", path = "../../runtime" }
+sr-primitives = { path = "../../../../primitives/sr-primitives" }
+codec = { package = "parity-scale-codec", version = "1.0.0" }
+client-api = { package = "substrate-client-api", path = "../../../../client/api" }
+client = { package = "substrate-client", path = "../../../../client/" }
diff --git a/core/test-runtime/client/src/block_builder_ext.rs b/test/utils/runtime/client/src/block_builder_ext.rs
similarity index 72%
rename from core/test-runtime/client/src/block_builder_ext.rs
rename to test/utils/runtime/client/src/block_builder_ext.rs
index c389a946bab86..6bd3ed18ebeb6 100644
--- a/core/test-runtime/client/src/block_builder_ext.rs
+++ b/test/utils/runtime/client/src/block_builder_ext.rs
@@ -19,25 +19,34 @@
 use runtime;
 use sr_primitives::traits::ProvideRuntimeApi;
 use generic_test_client::client;
-use generic_test_client::client::block_builder::api::BlockBuilder;
+
+use block_builder::BlockBuilderApi;
 
 /// Extension trait for test block builder.
 pub trait BlockBuilderExt {
 	/// Add transfer extrinsic to the block.
 	fn push_transfer(&mut self, transfer: runtime::Transfer) -> Result<(), client::error::Error>;
 	/// Add storage change extrinsic to the block.
-	fn push_storage_change(&mut self, key: Vec, value: Option>) -> Result<(), client::error::Error>;
+	fn push_storage_change(
+		&mut self,
+		key: Vec,
+		value: Option>,
+	) -> Result<(), client::error::Error>;
 }
 
-impl<'a, A> BlockBuilderExt for client::block_builder::BlockBuilder<'a, runtime::Block, A> where
-	A: ProvideRuntimeApi + client::blockchain::HeaderBackend + 'a,
-	A::Api: BlockBuilder
+impl<'a, A> BlockBuilderExt for block_builder::BlockBuilder<'a, runtime::Block, A> where
+	A: ProvideRuntimeApi + 'a,
+	A::Api: BlockBuilderApi,
 {
 	fn push_transfer(&mut self, transfer: runtime::Transfer) -> Result<(), client::error::Error> {
 		self.push(transfer.into_signed_tx())
 	}
 
-	fn push_storage_change(&mut self, key: Vec, value: Option>) -> Result<(), client::error::Error> {
+	fn push_storage_change(
+		&mut self,
+		key: Vec,
+		value: Option>,
+	) -> Result<(), client::error::Error> {
 		self.push(runtime::Extrinsic::StorageChange(key, value))
 	}
 }
diff --git a/core/test-runtime/client/src/lib.rs b/test/utils/runtime/client/src/lib.rs
similarity index 99%
rename from core/test-runtime/client/src/lib.rs
rename to test/utils/runtime/client/src/lib.rs
index affbae62c22e7..b5f3833e52f07 100644
--- a/core/test-runtime/client/src/lib.rs
+++ b/test/utils/runtime/client/src/lib.rs
@@ -205,7 +205,7 @@ impl TestClientBuilderExt for TestClientBuilder<
 	client::LocalCallExecutor>,
 	B
 > where
-	B: client::backend::Backend,
+	B: client_api::backend::Backend,
 {
 	fn set_heap_pages(mut self, heap_pages: u64) -> Self {
 		self.genesis_init_mut().heap_pages_override = Some(heap_pages);
diff --git a/core/test-runtime/client/src/trait_tests.rs b/test/utils/runtime/client/src/trait_tests.rs
similarity index 97%
rename from core/test-runtime/client/src/trait_tests.rs
rename to test/utils/runtime/client/src/trait_tests.rs
index 37fad9c7a1bf1..bfdab4d942ed2 100644
--- a/core/test-runtime/client/src/trait_tests.rs
+++ b/test/utils/runtime/client/src/trait_tests.rs
@@ -21,9 +21,9 @@
 
 use std::sync::Arc;
 
-use crate::backend;
+use client_api::backend::LocalBackend;
 use crate::block_builder_ext::BlockBuilderExt;
-use crate::blockchain::{Backend as BlockChainBackendT, HeaderBackend};
+use client_api::blockchain::{Backend as BlockChainBackendT, HeaderBackend};
 use crate::{AccountKeyring, ClientExt, TestClientBuilder, TestClientBuilderExt};
 use generic_test_client::consensus::BlockOrigin;
 use primitives::Blake2Hasher;
@@ -33,7 +33,7 @@ use sr_primitives::traits::Block as BlockT;
 
 /// helper to test the `leaves` implementation for various backends
 pub fn test_leaves_for_backend(backend: Arc) where
-	B: backend::LocalBackend,
+	B: LocalBackend,
 {
 	// block tree:
 	// G -> A1 -> A2 -> A3 -> A4 -> A5
@@ -149,7 +149,7 @@ pub fn test_leaves_for_backend(backend: Arc) where
 
 /// helper to test the `children` implementation for various backends
 pub fn test_children_for_backend(backend: Arc) where
-	B: backend::LocalBackend,
+	B: LocalBackend,
 {
 	// block tree:
 	// G -> A1 -> A2 -> A3 -> A4 -> A5
@@ -240,7 +240,7 @@ pub fn test_children_for_backend(backend: Arc) where
 }
 
 pub fn test_blockchain_query_by_number_gets_canonical(backend: Arc) where
-	B: backend::LocalBackend,
+	B: LocalBackend,
 {
 	// block tree:
 	// G -> A1 -> A2 -> A3 -> A4 -> A5
diff --git a/core/test-runtime/src/genesismap.rs b/test/utils/runtime/src/genesismap.rs
similarity index 89%
rename from core/test-runtime/src/genesismap.rs
rename to test/utils/runtime/src/genesismap.rs
index a3abf235ff1ca..79cba52323e58 100644
--- a/core/test-runtime/src/genesismap.rs
+++ b/test/utils/runtime/src/genesismap.rs
@@ -17,8 +17,8 @@
 //! Tool for creating the genesis block.
 
 use std::collections::HashMap;
-use runtime_io::{blake2_256, twox_128};
-use super::{AuthorityId, AccountId, WASM_BINARY};
+use runtime_io::hashing::{blake2_256, twox_128};
+use super::{AuthorityId, AccountId, WASM_BINARY, system};
 use codec::{Encode, KeyedVec, Joiner};
 use primitives::{ChangesTrieConfiguration, map, storage::well_known_keys};
 use sr_primitives::traits::{Block as BlockT, Hash as HashT, Header as HeaderT};
@@ -77,10 +77,16 @@ impl GenesisConfig {
 			map.insert(well_known_keys::CHANGES_TRIE_CONFIG.to_vec(), changes_trie_config.encode());
 		}
 		map.insert(twox_128(&b"sys:auth"[..])[..].to_vec(), self.authorities.encode());
-		// Finally, add the extra storage entries.
+		// Add the extra storage entries.
 		map.extend(self.extra_storage.clone().into_iter());
 
-		(map, self.child_extra_storage.clone())
+		// Assimilate the system genesis config.
+		let mut storage = (map, self.child_extra_storage.clone());
+		let mut config = system::GenesisConfig::default();
+		config.authorities = self.authorities.clone();
+		config.assimilate_storage(&mut storage).expect("Adding `system::GensisConfig` to the genesis");
+
+		storage
 	}
 }
 
diff --git a/core/test-runtime/src/lib.rs b/test/utils/runtime/src/lib.rs
similarity index 94%
rename from core/test-runtime/src/lib.rs
rename to test/utils/runtime/src/lib.rs
index 7fac0e702e6ca..2ec109938889d 100644
--- a/core/test-runtime/src/lib.rs
+++ b/test/utils/runtime/src/lib.rs
@@ -32,10 +32,7 @@ use trie_db::{TrieMut, Trie};
 use substrate_trie::PrefixedMemoryDB;
 use substrate_trie::trie_types::{TrieDB, TrieDBMut};
 
-use substrate_client::{
-	runtime_api as client_api, block_builder::api as block_builder_api, decl_runtime_apis,
-	impl_runtime_apis,
-};
+use sr_api::{decl_runtime_apis, impl_runtime_apis};
 use sr_primitives::{
 	ApplyResult, create_runtime_str, Perbill, impl_opaque_keys,
 	transaction_validity::{
@@ -341,14 +338,14 @@ impl GetRuntimeBlockType for Runtime {
 }
 
 impl_outer_origin!{
-	pub enum Origin for Runtime where system = srml_system {}
+	pub enum Origin for Runtime where system = paint_system {}
 }
 
 #[derive(Clone, Encode, Decode, Eq, PartialEq, RuntimeDebug)]
 pub struct Event;
 
-impl From for Event {
-	fn from(_evt: srml_system::Event) -> Self {
+impl From for Event {
+	fn from(_evt: paint_system::Event) -> Self {
 		unimplemented!("Not required in tests!")
 	}
 }
@@ -361,7 +358,7 @@ parameter_types! {
 	pub const AvailableBlockRatio: Perbill = Perbill::from_percent(75);
 }
 
-impl srml_system::Trait for Runtime {
+impl paint_system::Trait for Runtime {
 	type Origin = Origin;
 	type Call = Extrinsic;
 	type Index = u64;
@@ -379,7 +376,7 @@ impl srml_system::Trait for Runtime {
 	type Version = ();
 }
 
-impl srml_timestamp::Trait for Runtime {
+impl paint_timestamp::Trait for Runtime {
 	/// A timestamp: milliseconds since the unix epoch.
 	type Moment = u64;
 	type OnTimestampSet = ();
@@ -391,13 +388,13 @@ parameter_types! {
 	pub const ExpectedBlockTime: u64 = 10_000;
 }
 
-impl srml_babe::Trait for Runtime {
+impl paint_babe::Trait for Runtime {
 	type EpochDuration = EpochDuration;
 	type ExpectedBlockTime = ExpectedBlockTime;
 	// there is no actual runtime in this test-runtime, so testing crates
 	// are manually adding the digests. normally in this situation you'd use
-	// srml_babe::SameAuthoritiesForever.
-	type EpochChangeTrigger = srml_babe::ExternalTrigger;
+	// paint_babe::SameAuthoritiesForever.
+	type EpochChangeTrigger = paint_babe::ExternalTrigger;
 }
 
 /// Adds one to the given input and returns the final result.
@@ -408,7 +405,8 @@ fn benchmark_add_one(i: u64) -> u64 {
 
 /// The `benchmark_add_one` function as function pointer.
 #[cfg(not(feature = "std"))]
-static BENCHMARK_ADD_ONE: runtime_io::ExchangeableFunction u64> = runtime_io::ExchangeableFunction::new(benchmark_add_one);
+static BENCHMARK_ADD_ONE: runtime_interface::wasm::ExchangeableFunction u64> =
+	runtime_interface::wasm::ExchangeableFunction::new(benchmark_add_one);
 
 fn code_using_trie() -> u64 {
 	let pairs = [
@@ -459,7 +457,7 @@ static mut MUTABLE_STATIC: u64 = 32;
 cfg_if! {
 	if #[cfg(feature = "std")] {
 		impl_runtime_apis! {
-			impl client_api::Core for Runtime {
+			impl sr_api::Core for Runtime {
 				fn version() -> RuntimeVersion {
 					version()
 				}
@@ -473,13 +471,13 @@ cfg_if! {
 				}
 			}
 
-			impl client_api::Metadata for Runtime {
+			impl sr_api::Metadata for Runtime {
 				fn metadata() -> OpaqueMetadata {
 					unimplemented!()
 				}
 			}
 
-			impl client_api::TaggedTransactionQueue for Runtime {
+			impl transaction_pool_api::TaggedTransactionQueue for Runtime {
 				fn validate_transaction(utx: ::Extrinsic) -> TransactionValidity {
 					if let Extrinsic::IncludeData(data) = utx {
 						return Ok(ValidTransaction {
@@ -617,7 +615,7 @@ cfg_if! {
 						c: (3, 10),
 						genesis_authorities: system::authorities()
 							.into_iter().map(|x|(x, 1)).collect(),
-						randomness: >::randomness(),
+						randomness: >::randomness(),
 						secondary_slots: true,
 					}
 				}
@@ -626,7 +624,7 @@ cfg_if! {
 			impl offchain_primitives::OffchainWorkerApi for Runtime {
 				fn offchain_worker(block: u64) {
 					let ex = Extrinsic::IncludeData(block.encode());
-					runtime_io::submit_transaction(ex.encode()).unwrap();
+					runtime_io::offchain::submit_transaction(ex.encode()).unwrap();
 				}
 			}
 
@@ -636,7 +634,7 @@ cfg_if! {
 				}
 			}
 
-			impl srml_system_rpc_runtime_api::AccountNonceApi for Runtime {
+			impl paint_system_rpc_runtime_api::AccountNonceApi for Runtime {
 				fn account_nonce(_account: AccountId) -> Index {
 					0
 				}
@@ -644,7 +642,7 @@ cfg_if! {
 		}
 	} else {
 		impl_runtime_apis! {
-			impl client_api::Core for Runtime {
+			impl sr_api::Core for Runtime {
 				fn version() -> RuntimeVersion {
 					version()
 				}
@@ -658,13 +656,13 @@ cfg_if! {
 				}
 			}
 
-			impl client_api::Metadata for Runtime {
+			impl sr_api::Metadata for Runtime {
 				fn metadata() -> OpaqueMetadata {
 					unimplemented!()
 				}
 			}
 
-			impl client_api::TaggedTransactionQueue for Runtime {
+			impl transaction_pool_api::TaggedTransactionQueue for Runtime {
 				fn validate_transaction(utx: ::Extrinsic) -> TransactionValidity {
 					if let Extrinsic::IncludeData(data) = utx {
 						return Ok(ValidTransaction{
@@ -833,7 +831,7 @@ cfg_if! {
 						c: (3, 10),
 						genesis_authorities: system::authorities()
 							.into_iter().map(|x|(x, 1)).collect(),
-						randomness: >::randomness(),
+						randomness: >::randomness(),
 						secondary_slots: true,
 					}
 				}
@@ -842,7 +840,7 @@ cfg_if! {
 			impl offchain_primitives::OffchainWorkerApi for Runtime {
 				fn offchain_worker(block: u64) {
 					let ex = Extrinsic::IncludeData(block.encode());
-					runtime_io::submit_transaction(ex.encode()).unwrap()
+					runtime_io::offchain::submit_transaction(ex.encode()).unwrap()
 				}
 			}
 
@@ -852,7 +850,7 @@ cfg_if! {
 				}
 			}
 
-			impl srml_system_rpc_runtime_api::AccountNonceApi for Runtime {
+			impl paint_system_rpc_runtime_api::AccountNonceApi for Runtime {
 				fn account_nonce(_account: AccountId) -> Index {
 					0
 				}
@@ -893,10 +891,10 @@ fn test_sr25519_crypto() -> (sr25519::AppSignature, sr25519::AppPublic) {
 
 fn test_read_storage() {
 	const KEY: &[u8] = b":read_storage";
-	runtime_io::set_storage(KEY, b"test");
+	runtime_io::storage::set(KEY, b"test");
 
 	let mut v = [0u8; 4];
-	let r = runtime_io::read_storage(
+	let r = runtime_io::storage::read(
 		KEY,
 		&mut v,
 		0
@@ -905,7 +903,7 @@ fn test_read_storage() {
 	assert_eq!(&v, b"test");
 
 	let mut v = [0u8; 4];
-	let r = runtime_io::read_storage(KEY, &mut v, 8);
+	let r = runtime_io::storage::read(KEY, &mut v, 8);
 	assert_eq!(r, Some(4));
 	assert_eq!(&v, &[0, 0, 0, 0]);
 }
@@ -913,10 +911,10 @@ fn test_read_storage() {
 fn test_read_child_storage() {
 	const CHILD_KEY: &[u8] = b":child_storage:default:read_child_storage";
 	const KEY: &[u8] = b":read_child_storage";
-	runtime_io::set_child_storage(CHILD_KEY, KEY, b"test");
+	runtime_io::storage::child_set(CHILD_KEY, KEY, b"test");
 
 	let mut v = [0u8; 4];
-	let r = runtime_io::read_child_storage(
+	let r = runtime_io::storage::child_read(
 		CHILD_KEY,
 		KEY,
 		&mut v,
@@ -926,7 +924,7 @@ fn test_read_child_storage() {
 	assert_eq!(&v, b"test");
 
 	let mut v = [0u8; 4];
-	let r = runtime_io::read_child_storage(CHILD_KEY, KEY, &mut v, 8);
+	let r = runtime_io::storage::child_read(CHILD_KEY, KEY, &mut v, 8);
 	assert_eq!(r, Some(4));
 	assert_eq!(&v, &[0, 0, 0, 0]);
 }
diff --git a/core/test-runtime/src/system.rs b/test/utils/runtime/src/system.rs
similarity index 94%
rename from core/test-runtime/src/system.rs
rename to test/utils/runtime/src/system.rs
index fc4de0ce4b911..44743be200840 100644
--- a/core/test-runtime/src/system.rs
+++ b/test/utils/runtime/src/system.rs
@@ -18,14 +18,18 @@
 //! and depositing logs.
 
 use rstd::prelude::*;
-use runtime_io::{storage_root, storage_changes_root, blake2_256};
-use runtime_support::storage::{self, StorageValue, StorageMap};
-use runtime_support::storage_items;
+use runtime_io::{
+	storage::root as storage_root, storage::changes_root as storage_changes_root,
+	hashing::blake2_256,
+};
+use runtime_support::storage;
+use runtime_support::{decl_storage, decl_module};
 use sr_primitives::{
 	traits::{Hash as HashT, BlakeTwo256, Header as _}, generic, ApplyError, ApplyResult,
 	transaction_validity::{TransactionValidity, ValidTransaction, InvalidTransaction},
 };
 use codec::{KeyedVec, Encode};
+use paint_system::Trait;
 use crate::{
 	AccountId, BlockNumber, Extrinsic, Transfer, H256 as Hash, Block, Header, Digest, AuthorityId
 };
@@ -34,14 +38,20 @@ use primitives::storage::well_known_keys;
 const NONCE_OF: &[u8] = b"nonce:";
 const BALANCE_OF: &[u8] = b"balance:";
 
-storage_items! {
-	ExtrinsicData: b"sys:xtd" => required map [ u32 => Vec ];
-	// The current block number being processed. Set by `execute_block`.
-	Number: b"sys:num" => BlockNumber;
-	ParentHash: b"sys:pha" => required Hash;
-	NewAuthorities: b"sys:new_auth" => Vec;
-	StorageDigest: b"sys:digest" => Digest;
-	Authorities get(authorities): b"sys:auth" => default Vec;
+decl_module! {
+	pub struct Module for enum Call where origin: T::Origin {}
+}
+
+decl_storage! {
+	trait Store for Module as TestRuntime {
+		ExtrinsicData: map u32 => Vec;
+		// The current block number being processed. Set by `execute_block`.
+		Number get(fn number): Option;
+		ParentHash get(fn parent_hash): Hash;
+		NewAuthorities get(fn new_authorities): Option>;
+		StorageDigest get(fn storage_digest): Option;
+		Authorities get(fn authorities) config(): Vec;
+	}
 }
 
 pub fn balance_of_key(who: AccountId) -> Vec {
@@ -70,6 +80,10 @@ pub fn initialize_block(header: &Header) {
 	}
 }
 
+pub fn authorities() -> Vec {
+	Authorities::get()
+}
+
 pub fn get_block_number() -> Option {
 	Number::get()
 }
@@ -316,7 +330,7 @@ mod tests {
 	use crate::{Header, Transfer, WASM_BINARY};
 	use primitives::{NeverNativeValue, map, traits::CodeExecutor};
 	use substrate_executor::{NativeExecutor, WasmExecutionMethod, native_executor_instance};
-	use runtime_io::twox_128;
+	use runtime_io::hashing::twox_128;
 
 	// Declare an instance of the native executor dispatch for the test runtime.
 	native_executor_instance!(
diff --git a/test/utils/transaction-factory/Cargo.toml b/test/utils/transaction-factory/Cargo.toml
new file mode 100644
index 0000000000000..fdd948a6f2d29
--- /dev/null
+++ b/test/utils/transaction-factory/Cargo.toml
@@ -0,0 +1,19 @@
+[package]
+name = "transaction-factory"
+version = "0.0.1"
+authors = ["Parity Technologies "]
+edition = "2018"
+
+[dependencies]
+block-builder-api = { package = "substrate-block-builder-runtime-api", path = "../../../primitives/block-builder/runtime-api" }
+cli = { package = "substrate-cli", path = "../../../client/cli" }
+client-api = { package = "substrate-client-api", path = "../../../client/api" }
+client = { package = "substrate-client", path = "../../../client" }
+codec = { package = "parity-scale-codec", version = "1.0.0", features = ["derive"] }
+consensus_common = { package = "substrate-consensus-common", path = "../../../primitives/consensus/common" }
+log = "0.4.8"
+primitives = { package = "substrate-primitives", path = "../../../primitives/core" }
+sr-api = { path = "../../../primitives/sr-api" }
+sr-primitives = { path = "../../../primitives/sr-primitives" }
+substrate-service = { path = "../../../client/service" }
+
diff --git a/test-utils/transaction-factory/src/complex_mode.rs b/test/utils/transaction-factory/src/complex_mode.rs
similarity index 96%
rename from test-utils/transaction-factory/src/complex_mode.rs
rename to test/utils/transaction-factory/src/complex_mode.rs
index ed76a66b09083..086698ffc3ac9 100644
--- a/test-utils/transaction-factory/src/complex_mode.rs
+++ b/test/utils/transaction-factory/src/complex_mode.rs
@@ -42,8 +42,8 @@ use std::sync::Arc;
 
 use log::info;
 use client::Client;
-use client::block_builder::api::BlockBuilder;
-use client::runtime_api::ConstructRuntimeApi;
+use block_builder_api::BlockBuilder;
+use sr_api::ConstructRuntimeApi;
 use primitives::{Blake2Hasher, Hasher};
 use sr_primitives::generic::BlockId;
 use sr_primitives::traits::{Block as BlockT, ProvideRuntimeApi, One, Zero};
@@ -61,9 +61,10 @@ pub fn next(
 where
 	Block: BlockT::Out>,
 	Exec: client::CallExecutor + Send + Sync + Clone,
-	Backend: client::backend::Backend + Send,
+	Backend: client_api::backend::Backend + Send,
 	Client: ProvideRuntimeApi,
-	 as ProvideRuntimeApi>::Api: BlockBuilder,
+	 as ProvideRuntimeApi>::Api:
+		BlockBuilder,
 	RtApi: ConstructRuntimeApi> + Send + Sync,
 	RA: RuntimeAdapter,
 {
diff --git a/test-utils/transaction-factory/src/lib.rs b/test/utils/transaction-factory/src/lib.rs
similarity index 93%
rename from test-utils/transaction-factory/src/lib.rs
rename to test/utils/transaction-factory/src/lib.rs
index 067c75c3fc344..357e8067f37b9 100644
--- a/test-utils/transaction-factory/src/lib.rs
+++ b/test/utils/transaction-factory/src/lib.rs
@@ -26,7 +26,9 @@ use std::fmt::Display;
 
 use log::info;
 
-use client::{Client, block_builder::api::BlockBuilder, runtime_api::ConstructRuntimeApi};
+use client::Client;
+use block_builder_api::BlockBuilder;
+use sr_api::ConstructRuntimeApi;
 use consensus_common::{
 	BlockOrigin, BlockImportParams, InherentData, ForkChoiceStrategy,
 	SelectChain
@@ -100,9 +102,10 @@ pub fn factory(
 where
 	Block: BlockT::Out>,
 	Exec: client::CallExecutor + Send + Sync + Clone,
-	Backend: client::backend::Backend + Send,
+	Backend: client_api::backend::Backend + Send,
 	Client: ProvideRuntimeApi,
-	 as ProvideRuntimeApi>::Api: BlockBuilder,
+	 as ProvideRuntimeApi>::Api:
+		BlockBuilder,
 	RtApi: ConstructRuntimeApi> + Send + Sync,
 	Sc: SelectChain,
 	RA: RuntimeAdapter,
@@ -158,10 +161,11 @@ pub fn create_block(
 where
 	Block: BlockT::Out>,
 	Exec: client::CallExecutor + Send + Sync + Clone,
-	Backend: client::backend::Backend + Send,
+	Backend: client_api::backend::Backend + Send,
 	Client: ProvideRuntimeApi,
 	RtApi: ConstructRuntimeApi> + Send + Sync,
-	 as ProvideRuntimeApi>::Api: BlockBuilder,
+	 as ProvideRuntimeApi>::Api:
+		BlockBuilder,
 	RA: RuntimeAdapter,
 {
 	let mut block = client.new_block(Default::default()).expect("Failed to create new block");
@@ -180,10 +184,10 @@ where
 fn import_block(
 	client: &Arc>,
 	block: Block
-) -> () where 
+) -> () where
 	Block: BlockT::Out>,
 	Exec: client::CallExecutor + Send + Sync + Clone,
-	Backend: client::backend::Backend + Send,
+	Backend: client_api::backend::Backend + Send,
 {
 	let import = BlockImportParams {
 		origin: BlockOrigin::File,
@@ -194,6 +198,7 @@ fn import_block(
 		justification: None,
 		auxiliary: Vec::new(),
 		fork_choice: ForkChoiceStrategy::LongestChain,
+		allow_missing_state: false,
 	};
 	(&**client).import_block(import, HashMap::new()).expect("Failed to import block");
 }
diff --git a/test-utils/transaction-factory/src/modes.rs b/test/utils/transaction-factory/src/modes.rs
similarity index 100%
rename from test-utils/transaction-factory/src/modes.rs
rename to test/utils/transaction-factory/src/modes.rs
diff --git a/test-utils/transaction-factory/src/simple_modes.rs b/test/utils/transaction-factory/src/simple_modes.rs
similarity index 94%
rename from test-utils/transaction-factory/src/simple_modes.rs
rename to test/utils/transaction-factory/src/simple_modes.rs
index bcbb91200657f..f869201b46cd6 100644
--- a/test-utils/transaction-factory/src/simple_modes.rs
+++ b/test/utils/transaction-factory/src/simple_modes.rs
@@ -37,8 +37,8 @@ use std::sync::Arc;
 
 use log::info;
 use client::Client;
-use client::block_builder::api::BlockBuilder;
-use client::runtime_api::ConstructRuntimeApi;
+use block_builder_api::BlockBuilder;
+use sr_api::ConstructRuntimeApi;
 use primitives::{Blake2Hasher, Hasher};
 use sr_primitives::traits::{Block as BlockT, ProvideRuntimeApi, One};
 use sr_primitives::generic::BlockId;
@@ -56,9 +56,10 @@ pub fn next(
 where
 	Block: BlockT::Out>,
 	Exec: client::CallExecutor + Send + Sync + Clone,
-	Backend: client::backend::Backend + Send,
+	Backend: client_api::backend::Backend + Send,
 	Client: ProvideRuntimeApi,
-	 as ProvideRuntimeApi>::Api: BlockBuilder,
+	 as ProvideRuntimeApi>::Api:
+		BlockBuilder,
 	RtApi: ConstructRuntimeApi> + Send + Sync,
 	RA: RuntimeAdapter,
 {
diff --git a/utils/build-script-utils/Cargo.toml b/utils/build-script-utils/Cargo.toml
new file mode 100644
index 0000000000000..36703ab838f06
--- /dev/null
+++ b/utils/build-script-utils/Cargo.toml
@@ -0,0 +1,7 @@
+[package]
+name = "substrate-build-script-utils"
+version = "2.0.0"
+authors = ["Parity Technologies "]
+edition = "2018"
+
+[dependencies]
diff --git a/utils/build-script-utils/src/lib.rs b/utils/build-script-utils/src/lib.rs
new file mode 100644
index 0000000000000..be2f4636b6746
--- /dev/null
+++ b/utils/build-script-utils/src/lib.rs
@@ -0,0 +1,44 @@
+// Copyright 2019 Parity Technologies (UK) Ltd.
+// This file is part of Substrate.
+
+// Substrate is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+
+// Substrate is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+
+// You should have received a copy of the GNU General Public License
+// along with Substrate.  If not, see .
+
+//! Crate with utility functions for `build.rs` scripts.
+
+use std::{env, path::PathBuf};
+
+/// Make sure the calling `build.rs` script is rerun when `.git/HEAD` changed.
+///
+/// The file is searched from the `CARGO_MANIFEST_DIR` upwards. If the file can not be found,
+/// a warning is generated.
+pub fn rerun_if_git_head_changed() {
+	let mut manifest_dir = PathBuf::from(
+		env::var("CARGO_MANIFEST_DIR").expect("`CARGO_MANIFEST_DIR` is always set by cargo.")
+	);
+	let manifest_dir_copy = manifest_dir.clone();
+
+	while manifest_dir.parent().is_some() {
+		if manifest_dir.join(".git/HEAD").exists() {
+			println!("cargo:rerun-if-changed={}", manifest_dir.join(".git/HEAD").display());
+			return
+		}
+
+		manifest_dir.pop();
+	}
+
+	println!(
+		"cargo:warning=Could not find `.git/HEAD` searching from `{}` upwards!",
+		manifest_dir_copy.display(),
+	);
+}
diff --git a/core/utils/fork-tree/Cargo.toml b/utils/fork-tree/Cargo.toml
similarity index 100%
rename from core/utils/fork-tree/Cargo.toml
rename to utils/fork-tree/Cargo.toml
diff --git a/core/utils/fork-tree/src/lib.rs b/utils/fork-tree/src/lib.rs
similarity index 100%
rename from core/utils/fork-tree/src/lib.rs
rename to utils/fork-tree/src/lib.rs
diff --git a/core/utils/wasm-builder/Cargo.toml b/utils/wasm-builder/Cargo.toml
similarity index 87%
rename from core/utils/wasm-builder/Cargo.toml
rename to utils/wasm-builder/Cargo.toml
index e9d38f2bd23f6..66d94c32ab330 100644
--- a/core/utils/wasm-builder/Cargo.toml
+++ b/utils/wasm-builder/Cargo.toml
@@ -10,9 +10,10 @@ license = "GPL-3.0"
 
 [dependencies]
 build-helper = "0.1.1"
-cargo_metadata = "0.8.2"
+cargo_metadata = "0.9.0"
 tempfile = "3.1.0"
-toml = "0.5.3"
+toml = "0.5.4"
 walkdir = "2.2.9"
 fs2 = "0.4.3"
 wasm-gc-api = "0.1.11"
+atty = "0.2.13"
diff --git a/core/utils/wasm-builder/README.md b/utils/wasm-builder/README.md
similarity index 73%
rename from core/utils/wasm-builder/README.md
rename to utils/wasm-builder/README.md
index b15d2ebfab2a0..b21c588ac383c 100644
--- a/core/utils/wasm-builder/README.md
+++ b/utils/wasm-builder/README.md
@@ -1,14 +1,14 @@
-## WASM builder is a utility for building a project as a WASM binary
+# WASM builder is a utility for building a project as a WASM binary
 
 The WASM builder is a tool that integrates the process of building the WASM binary of your project into the main
 `cargo` build process.
 
-### Project setup
+## Project setup
 
 A project that should be compiled as a WASM binary needs to:
 
 1. Add a `build.rs` file.
-2. Add `substrate-wasm-builder-runner` as dependency into `build-dependencies`.
+2. Add `substrate-wasm-builder` as dependency into `build-dependencies`.
 
 The `build.rs` file needs to contain the following code:
 
@@ -34,25 +34,28 @@ include!(concat!(env!("OUT_DIR"), "/wasm_binary.rs"));
 This will include the generated WASM binary as two constants `WASM_BINARY` and `WASM_BINARY_BLOATY`.
 The former is a compact WASM binary and the latter is not compacted.
 
-### Environment variables
+## Environment variables
 
 By using environment variables, you can configure which WASM binaries are built and how:
 
-- `SKIP_WASM_BUILD` - Skips building any WASM binary. This is useful when only native should be recompiled.
-- `BUILD_DUMMY_WASM_BINARY` - Builds dummy WASM binaries. These dummy binaries are empty and useful
+- `SKIP_WASM_BUILD` - Skips building any wasm binary. This is useful when only native should be recompiled.
+- `BUILD_DUMMY_WASM_BINARY` - Builds dummy wasm binaries. These dummy binaries are empty and useful
                              for `cargo check` runs.
-- `WASM_BUILD_TYPE` - Sets the build type for building WASM binaries. Supported values are `release` or `debug`.
+- `WASM_BUILD_TYPE` - Sets the build type for building wasm binaries. Supported values are `release` or `debug`.
                       By default the build type is equal to the build type used by the main build.
-- `TRIGGER_WASM_BUILD` - Can be set to trigger a WASM build. On subsequent calls the value of the variable
+- `TRIGGER_WASM_BUILD` - Can be set to trigger a wasm build. On subsequent calls the value of the variable
                          needs to change. As WASM builder instructs `cargo` to watch for file changes
                          this environment variable should only be required in certain circumstances.
-- `WASM_BUILD_RUSTFLAGS` - Extend `RUSTFLAGS` given to `cargo build` while building the WASM binary.
+- `WASM_BUILD_RUSTFLAGS` - Extend `RUSTFLAGS` given to `cargo build` while building the wasm binary.
+- `WASM_BUILD_NO_COLOR` - Disable color output of the wasm build.
+- `WASM_TARGET_DIRECTORY` - Will copy any build wasm binary to the given directory. The path needs
+                           to be absolute.
 
 Each project can be skipped individually by using the environment variable `SKIP_PROJECT_NAME_WASM_BUILD`.
 Where `PROJECT_NAME` needs to be replaced by the name of the cargo project, e.g. `node-runtime` will
 be `NODE_RUNTIME`.
 
-### Prerequisites:
+## Prerequisites:
 
 WASM builder requires the following prerequisities for building the WASM binary:
 
diff --git a/core/utils/wasm-builder/src/lib.rs b/utils/wasm-builder/src/lib.rs
similarity index 90%
rename from core/utils/wasm-builder/src/lib.rs
rename to utils/wasm-builder/src/lib.rs
index 93e1700792ab4..a689f15b55418 100644
--- a/core/utils/wasm-builder/src/lib.rs
+++ b/utils/wasm-builder/src/lib.rs
@@ -24,7 +24,7 @@
 //! A project that should be compiled as a WASM binary needs to:
 //!
 //! 1. Add a `build.rs` file.
-//! 2. Add `substrate-wasm-builder-runner` as dependency into `build-dependencies`.
+//! 2. Add `substrate-wasm-builder` as dependency into `build-dependencies`.
 //!
 //! The `build.rs` file needs to contain the following code:
 //!
@@ -54,16 +54,17 @@
 //!
 //! By using environment variables, you can configure which WASM binaries are built and how:
 //!
-//! - `SKIP_WASM_BUILD` - Skips building any WASM binary. This is useful when only native should be recompiled.
-//! - `BUILD_DUMMY_WASM_BINARY` - Builds dummy WASM binaries. These dummy binaries are empty and useful
+//! - `SKIP_WASM_BUILD` - Skips building any wasm binary. This is useful when only native should be recompiled.
+//! - `BUILD_DUMMY_WASM_BINARY` - Builds dummy wasm binaries. These dummy binaries are empty and useful
 //!                              for `cargo check` runs.
-//! - `WASM_BUILD_TYPE` - Sets the build type for building WASM binaries. Supported values are `release` or `debug`.
+//! - `WASM_BUILD_TYPE` - Sets the build type for building wasm binaries. Supported values are `release` or `debug`.
 //!                       By default the build type is equal to the build type used by the main build.
-//! - `TRIGGER_WASM_BUILD` - Can be set to trigger a WASM build. On subsequent calls the value of the variable
+//! - `TRIGGER_WASM_BUILD` - Can be set to trigger a wasm build. On subsequent calls the value of the variable
 //!                          needs to change. As WASM builder instructs `cargo` to watch for file changes
 //!                          this environment variable should only be required in certain circumstances.
-//! - `WASM_BUILD_RUSTFLAGS` - Extend `RUSTFLAGS` given to `cargo build` while building the WASM binary.
-//! - `WASM_TARGET_DIRECTORY` - Will copy any build WASM binary to the given directory. The path needs
+//! - `WASM_BUILD_RUSTFLAGS` - Extend `RUSTFLAGS` given to `cargo build` while building the wasm binary.
+//! - `WASM_BUILD_NO_COLOR` - Disable color output of the wasm build.
+//! - `WASM_TARGET_DIRECTORY` - Will copy any build wasm binary to the given directory. The path needs
 //!                            to be absolute.
 //!
 //! Each project can be skipped individually by using the environment variable `SKIP_PROJECT_NAME_WASM_BUILD`.
@@ -82,24 +83,27 @@ use std::{env, fs, path::PathBuf, process::{Command, Stdio, self}};
 mod prerequisites;
 mod wasm_project;
 
-/// Environment variable that tells us to skip building the WASM binary.
+/// Environment variable that tells us to skip building the wasm binary.
 const SKIP_BUILD_ENV: &str = "SKIP_WASM_BUILD";
 
-/// Environment variable to force a certain build type when building the WASM binary.
+/// Environment variable to force a certain build type when building the wasm binary.
 /// Expects "debug" or "release" as value.
 ///
 /// By default the WASM binary uses the same build type as the main cargo build.
 const WASM_BUILD_TYPE_ENV: &str = "WASM_BUILD_TYPE";
 
-/// Environment variable to extend the `RUSTFLAGS` variable given to the WASM build.
+/// Environment variable to extend the `RUSTFLAGS` variable given to the wasm build.
 const WASM_BUILD_RUSTFLAGS_ENV: &str = "WASM_BUILD_RUSTFLAGS";
 
-/// Environment variable to set the target directory to copy the final WASM binary.
+/// Environment variable to set the target directory to copy the final wasm binary.
 ///
 /// The directory needs to be an absolute path.
 const WASM_TARGET_DIRECTORY: &str = "WASM_TARGET_DIRECTORY";
 
-/// Build the currently built project as WASM binary.
+/// Environment variable to disable color output of the wasm build.
+const WASM_BUILD_NO_COLOR: &str = "WASM_BUILD_NO_COLOR";
+
+/// Build the currently built project as wasm binary.
 ///
 /// The current project is determined by using the `CARGO_MANIFEST_DIR` environment variable.
 ///
@@ -110,7 +114,7 @@ pub fn build_project(file_name: &str, cargo_manifest: &str) {
 	build_project_with_default_rustflags(file_name, cargo_manifest, "");
 }
 
-/// Build the currently built project as WASM binary.
+/// Build the currently built project as wasm binary.
 ///
 /// The current project is determined by using the `CARGO_MANIFEST_DIR` environment variable.
 ///
diff --git a/core/utils/wasm-builder/src/prerequisites.rs b/utils/wasm-builder/src/prerequisites.rs
similarity index 100%
rename from core/utils/wasm-builder/src/prerequisites.rs
rename to utils/wasm-builder/src/prerequisites.rs
diff --git a/core/utils/wasm-builder/src/wasm_project.rs b/utils/wasm-builder/src/wasm_project.rs
similarity index 98%
rename from core/utils/wasm-builder/src/wasm_project.rs
rename to utils/wasm-builder/src/wasm_project.rs
index 1651642ee0a7b..701a50fe1c9c7 100644
--- a/core/utils/wasm-builder/src/wasm_project.rs
+++ b/utils/wasm-builder/src/wasm_project.rs
@@ -320,12 +320,16 @@ fn build_project(project: &Path, default_rustflags: &str) {
 		env::var(crate::WASM_BUILD_RUSTFLAGS_ENV).unwrap_or_default(),
 	);
 
-	build_cmd.args(&["build", "--target=wasm32-unknown-unknown"])
+	build_cmd.args(&["rustc", "--target=wasm32-unknown-unknown"])
 		.arg(format!("--manifest-path={}", manifest_path.display()))
 		.env("RUSTFLAGS", rustflags)
 		// We don't want to call ourselves recursively
 		.env(crate::SKIP_BUILD_ENV, "");
 
+	if env::var(crate::WASM_BUILD_NO_COLOR).is_err() {
+		build_cmd.arg("--color=always");
+	}
+
 	if is_release_build() {
 		build_cmd.arg("--release");
 	};