KOS & CLI #236
Merged
KOS & CLI #236
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commit adds `page_size` as a parameter in several function definitions in `builder.rs`. It also adds counters to track `resident_set_size_bytes` and `virtual_memory_size_bytes` in `counters.rs`. These changes are designed to enable accurate memory monitoring and provide additional details about memory usage. The new `page_size` parameter allows us to associate memory usage with the page size, which will be helpful for monitoring and troubleshooting. The `page_size` crate has also been added to `Cargo.toml` to manage the retrieval of system's memory page size. This addition is part of the effort to accurately track and report memory usage in bytes.
In this commit, we have integrated the performance monitoring module into the RPC layer of our application. The PerfMonitor module is now added as a dependency in various Cargo.toml files, and methods related to performance monitoring have been added/updated in several .rs files. This allows us to collect various metrics related to our application's performance, including memory usage, CPU usage, disc I/O, etc. which will help in evaluating our software's efficiency. We chose to collect these specific metrics as they give a comprehensive overview of the software's performance and can help us detect potential bottlenecks or issues. We have also updated the ProcessMetrics structure to include these new metrics.
Get metrics
kos cli (wizards), rearrange cli commands, usage of term().help() where appropriate, metrics (testing), misc wallet operations bugfixes.
… into wallet-store
Common-use case impl and move addresses to main
michaelsutton
approved these changes
Aug 22, 2023
smartgoo
pushed a commit
to smartgoo/rusty-kaspa
that referenced
this pull request
Jun 18, 2024
* wip - generator change & fees * wip - propagate generator to send() and estimate() * yield_executor in account send() * Add page size and memory usage in bytes to perf_monitor This commit adds `page_size` as a parameter in several function definitions in `builder.rs`. It also adds counters to track `resident_set_size_bytes` and `virtual_memory_size_bytes` in `counters.rs`. These changes are designed to enable accurate memory monitoring and provide additional details about memory usage. The new `page_size` parameter allows us to associate memory usage with the page size, which will be helpful for monitoring and troubleshooting. The `page_size` crate has also been added to `Cargo.toml` to manage the retrieval of system's memory page size. This addition is part of the effort to accurately track and report memory usage in bytes. * Add performance monitoring to RPC In this commit, we have integrated the performance monitoring module into the RPC layer of our application. The PerfMonitor module is now added as a dependency in various Cargo.toml files, and methods related to performance monitoring have been added/updated in several .rs files. This allows us to collect various metrics related to our application's performance, including memory usage, CPU usage, disc I/O, etc. which will help in evaluating our software's efficiency. We chose to collect these specific metrics as they give a comprehensive overview of the software's performance and can help us detect potential bottlenecks or issues. We have also updated the ProcessMetrics structure to include these new metrics. * wip - metrics * is_standard_output_amount_dust() updated using static/const values * rename get_connection_info to get_server_info * wip - kos metrics * wip - metrics * wip - tx generator summary struct * clippy * wip - metrics (add 2hrs span) * wip - fix wallet naming during creation * cleanup * wip - graph duration (cleanup) * wip - help now uses term().help() * wip - general refactoring kos cli (wizards), rearrange cli commands, usage of term().help() where appropriate, metrics (testing), misc wallet operations bugfixes. * wip - account ident renaming * wip - utxo groupting for tx * wip - fix node sync monito * fix "no transactions found" error * disable background console * cli - fix commit() during account rename * mutation lock issue * cli - fix account selection * rearrange mutexes in mass calculator * working on tx generator * wip - generator signer * sign_with_multiple_v2, tx generator wip * wip - cli rpc handler * update metrics to use a published workflow-perf-monitor fork of the perf-monitor crate * Remove "page_size" and related counters from "perf_monitor" The "page_size" attribute and its related attributes "resident_set_size_bytes" and "virutal_memory_size_bytes" have been removed from the performance monitor (perf_monitor) module. These counters were found to be unnecessary and their removal improves the simplicity and readability of the module. This change has also led to the removal of the "page_size" package from the dependencies. * perf-monitor - rename resident_set_size_bytes to resident_set_size * wip: tx generator * cli - monitor command * cli - disable default tracking on mute * generator docs * cli - transfer, monitor, generator summary, cleanup * wip - isolate consensus WASM primitives under a separate module * native cli - cleanup logs * cli - refactor help output to use help() helper that helps to output help * cli - cleanup, parsing helper functions * wip - address derivation scan fn (for legacy accounts) * kos - miner throttle setting * wip - legacy account scans * wip - imports * tx mass calculator for WASM * fix main declaration in kaspa-cli * wip - wallet imports * metrics - basic tps indicator * wallet - switch account names to Option<String> * wip - metrics retention * fix filling virtual memory size * wallet - utxo processor - fix misc issues in utxo processor handling; extend pending transactions; fix issues in utxo iterator; improve event formatting; * wallet - fix network_id reference; cleanup * add RPC API versioning provision (handled by get-server-info) * cli - misc output formatting + select_private_key() handler * wip - kos README * kos - metrics toolbar styling * cleanup * cleanup * cleanup * cleanup * cleanup and comments * refactor tx storage to use outgoing transactions, refactor metrics data submission to use raw f64 values, history list vs details; propagate additional data through PendingTransaction and GeneratorSummary. * cli - cleanup details cmd output * cli - cleanup account selection output * fix storage collision between outgoing tx record and change UTXO * guide and general cleanup * wallet storage - version * cleanup * kos - guide * kos - README * main workspace README update * cleanup * published workflow-rs 0.4.0 * kos - improve autoconnect * wallet framework event docs * cli - fix typo * kos - fix set_duration error handler * kos - cleanup logs * fix lints * update check script to use --tests and --benches * fix wallet storage unit test * js-api - initial Generator harness * js-api - JsValue for UtxoEntryRference conversion * js-api - estimateTransactions * js-api - fix Keypair::toAddress() * add build script argument propagation * wallet - add basic user input checks * WIP: address range with keys * Add test for transaction bincode serialization Added a new test function, `test_transaction_bincode`, in `consensus/core/src/tx.rs`. The test case covers the serialization of a transaction structure into bincode and subsequent deserialization. This is beneficial for ensuring the correctness and consistency of the serialization and deserialization processes within our blockchain consensus mechanism. * Refactor Hash serialization and deserialization Updated how Hash is serialized and deserialized in crypto/hashes/src/lib.rs. * Add serde_bytes module for handling byte arrays This commit introduces a new module called `serde_bytes`. The new module contains two inner modules: `ser` and `de`. These are for Serde's `Serialize` and `Deserialize`traits respectively. The `ser` module has a `serialize` function for human-readable and non-human-readable versions of byte arrays. The `de` module essentially does the opposite and contains a function to deserialize the byte arrays. They both utilize faster_hex library for efficient conversion. This was done to abstract away byte array serialization/deserialization logic to a separate unit of the project, enhancing modularity and code organization. * Improve error handling in serde_bytes implementation The trait Deserialize in utils/src/serde_bytes/de.rs has been updated to use TryFrom rather than From. The implementation has been updated accordingly. * Update serde serialization and remove unnecessary dependencies use utils::serde_bytes instead of external crate serde_bytes * tmp * add forceful shutdown on 3 quit attempts * replace all ToString impls with Display trait impl * "Implement improved serialization and deserialization for fixed-size bytes" This update implements a more efficient and concise way to serialize and deserialize fixed-size bytes. It replaces previous code with a new approach that leverages macros and generic traits. The changes were made across multiple files including "subnets.rs", "tx.rs" and several in the "serde_bytes_fixed" directory. The new approach uses the macro "serde_impl_ser_fixed_bytes" and the trait "serialize", which together streamline the process of serialization. For deserialization, the "deserialize" trait and the macro "serde_impl_deser_fixed_bytes" were used. These new implementations are designed to support both human-readable and binary formats. This allows for more flexibility and efficiency in handling different types of data. In addition, they improve code coherence and maintainability, making the codebase easier to understand and modify in the future. * "Implement serialization of SubnetworkId with fixed bytes" SubnetworkId is now serialized using newly imported serde utilities, serde_impl_ser_fixed_bytes and serde_impl_deser_fixed_bytes instead of serde_bytes_fixed attribute. * Update formatting of deserialization error message Modified the "expecting" function in serde_bytes_fixed/de.rs to provide clearer error messages when deserializing byte arrays. The error message now includes the expected data type in addition to the array size. * wip - refactoring JS APIs * wip - account refactoring * wasm - refactor UtxoEntryPointReference TryFrom<&JsValue> * Refactor deserialization of hexadecimal strings Reduced code redundancy in serde_bytes_fixed/de.rs and serde_bytes/de.rs by introducing a FromHexVisitor struct which handles converting hexadecimal strings to their respective types. This makes the code more maintainable and easier to test. Moreover, implemented FromHex trait to ScriptPublicKey, which allows us to use our new visitor in deserialization of ScriptPublicKey. * Update deserialization for fixed-size byte arrays Fixed-size byte arrays now should implement AsRef<[u8; N]> in addition to FromHex and From<[u8; N]> to fit the constraints. This modification is found in the kaspa-utils library. This also extends the functionality of the Deserialize trait in kaspa-utils which will now support the deserialization of byte arrays of a fixed size. By doing so, the code maintaining the 'serde_impl_deser_fixed_bytes' and 'serde_impl_ser_fixed_bytes' macros to implement serialization and deserialization for byte arrays of a certain size is simplified. The method works by referencing the byte array as the fixed-size array which makes it possible to use a reference implementation instead of the usual implementation. Added new Deserialize trait implementation for byte arrays of length 0 to 32 and replaced fixed_bytes with fixed_bytes_ref for module hashes and tx in crypto and consensus/core respectively. Modified the implementation of the Serialize trait for types that can be referenced as a fixed-sized byte array which is now possible for byte arrays from a length of 0 to 32. An extended set of examples demonstrating how to use the proposed serialization/deserialization has been included. The purpose of these changes is to simplify the implementation and usage of serialization/deserialization for byte arrays of a fixed size, and to extend this functionality to a larger range of possible array sizes. * WIP: gen 0 address manager * wip - address refactoring * wip - small utils snippet for argument parsing * wasm utils script, minor fix for network type * js-api - generator example * js-api estimate * Modified utils.js to allow for more customisation, fixed issues. Updated simple-transaction.js to use `parseArgs`. * bugfix potentially undefined addressArg * update utils to use 'node:util' and use address as destination address * update rpc.js to use parseArgs * addresses and hd-wallet * modify header js * formatting * refactor state.js * refactor storage.js * refactor test.js * refactor tx-create.js and introduce guard issynced * refactor tx-script-sign.js and add guard rpc synced * network type not used * refactor tx-send.js and add guard rpc is synced * refactor utxo-split.js and add guard rpc synced * add guard to simple-transaction.js * storage use networktype from args * rpc - implement get_server_info_call and get_sync_status_call for gRPC proxy * update Cargo.lock following merge * Add end-to-end demo Testing both `createTransaction` and `createTransactions` * Simplify simple-transactions * Move mining references * unit tests: gen0 kaspatest addresses * wip - runtime account refactoring * wip - account refactoring * wallet decrypt error message * WIP: storage structure update * wip - account refactoring * updated demo, tx-create with more comments and use rpc parse url/args added comments to utils * a better rpc parse url for tx-script-sign tx-send and utxo-split updated utils to have a better help message * dont include null * dont directly use paymount output * wip - account refactoring (metadata storage, import scaffolding, account derivation indexes) * utxo processing settings * wallet - update stored metadata * clippy * Add WASM serialization/deserialization tests and improve error handling This commit adds unit tests for WASM serialization and deserialization in 'wasm.rs' and also updates relevant dependencies in 'Cargo.toml'. In addition, this commit enhances error handling in 'lib.rs' by making 'TryFrom<&str> for Version' fallible, meaning it can return an 'AddressError' if an invalid string is converted. This should improve robustness and ease debugging efforts. Finally, this commit modifies the Deserializer implementation for 'Address' to handle more input data types and give more detailed error messages. This should enhance error handling capability and improve user/dev experience. * Move test cases from rpc/wrpc/client to crypto/addresses The test cases related to the functionality of Address in wasm (WebAssembly) were moved from rpc/wrpc/client to the crypto/addresses module. This move improves the modularity and coherence of the tests by placing them directly in the module they're testing. Additionally, dependencies related to these test cases were added to crypto/addresses module and removed from rpc/wrpc/client, further improving the modular structure of the project. * Configure wasm tests to run in browser A line was added to make the WebAssembly tests run in a browser context. This is implemented using the `wasm_bindgen_test_configure!` macro from the `wasm_bindgen_test` crate. This change is necessary to ensure accuracy of WebAssembly functionality in the browser environment. * Refactor AddressVisitor to simplify default implementation The Default implementation for AddressVisitor has been condensed for simplification. Instead of manually implementing the Default trait with a function, the built-in derive annotation is used. This refactoring makes code cleaner and easier to understand. * Remove run_in_browser configuration in wasm test The 'run_in_browser' configuration that was previously set up in the wasm_bindgen_test has been removed as it is unnecessary. Additionally, an assertion was added to check the type of JavaScript value generated upon serializing Address, to ensure type consistency across serialization and deserialization. * Add assert for object type in lib.rs A new assertion has been added in the lib.rs file to ensure that the object type is correctly set to "object". * wallet core - address scan for keypair and resident addresss * WIP: account storage structure * wip - refactoring async * wip - remove async from some derivation trait fns * progress working on parse host function * parse host with tests * bugfix only port and added more tests * added docs * Small changes to parse * update parse_url to use parse_host function * add brackets to ipv6 and remove previous impl for parse_url * use ok or else * back to .map(Ok) * parse host to check server if valid input * wip - migrating parts of the JS SDK into kaspa-consensus-wasm * wip - cleanup + migrate wasm traits to extensions * cleanup * wasm - rename modules * wip - cleanup * Update wasm handling in lib.rs Added special handling for wasm target architecture in lib.rs. The changes include specifying different formats for the formatter, new methods to handle different types of input(particularly: conversion from pointer to Address struct) and added new tests. These changes were necessary to ensure compatibility with the wasm32 architecture * Update wasm-bindgen-test and web-sys dependency scope Moved wasm-bindgen-test and web-sys dependencies from specific versions to workspace in crypto/addresses/Cargo.toml file. Also, added wasm-bindgen-test to the parent Cargo.toml. This change was necessary to ensure all dependencies are fetched from the workspace, creating a unified build and test environment. * Remove unnecessary whitespace in wasm32 function Whitespace on line 379 between the #[cfg] attribute and the visit_f32 function definition was removed. The change was made to clean up the code and maintain consistency with the rest of the codebase in crypto/addresses/src/lib.rs. * adress - post-merge fixes and cleanup * "Refactor ScriptPublicKey implementation in tx.rs" Moved the implementation of ScriptPublicKey from tx.rs to a new file script_public_key.rs to make the structure of the codebase more organized. Also updated the tx.rs file to import the ScriptPublicKey related types from this new file. This will improve the readability of the tx.rs file and make it easier for developers to navigate through the codebase. Future improvements and changes related to ScriptPublicKey will now be done in script_public_key.rs, thus making these changes more isolated and easier to manage. * Add wasm support for ScriptPublicKey Added WebAssembly (wasm) support to the ScriptPublicKey module in the consensus core, and updated dependencies accordingly. This change allows for ScriptPublicKey usage within a wasm context. Additionally, updated dependencies in Cargo.toml and Cargo.lock to reflect changes needed for wasm support. This improvement is essential to expanding the consensus core's platform interoperability. The ScriptPublicKeyVisitor struct was added to the src/tx/script_public_key.rs file to support the wasm target. Also, the new FromHexVisitor and TryFrom traits are implemented to transform Value into u16 and Vec<u8> respectively. This is necessary for conversion between numeric and text representations. Wasm tests were also added to ensure proper working of the updates. The consensus core's interoperability and functionality are improved with this update. Therefore, it expands the platform's usage potential. This is a crucial update for working with wasm environments, especially for web-based projects. * wasm - expose UtxoProcessor and UtxoContext * preparing merge into kos PR * add /test script * wasm - fix duplics wasm symbol error * cli - different prompt on shutdown * kos - update terminal for clipboard copy/paste fix on windows * wallet core - utxo recovery period * rustdoc * wallet storage - stored data version check * kos 0.0.2 * Carog.lock update * kos - rename kaspa miner binary during redistributable build * fix for terminal unicode handling issues * Common-use case impl and move addresses to main * WIP: Start implementing a PublicKey class for binding * Implement toAddress of compressed public key * Export JSPublicKey from rust as PublicKey in js * Implement x-only JS pubkey support * Full DER Public Key support * wasm - interfaces and prv keydata * wasm - updates for the keypair merge * wasm - cleanup, update keypair to use XOnlyPublicKey instead of &serialize[1..] * wrpc - websocket defaults (tungstenite update) * workflow-rs 0.6.0 * wip - update JavaScript demo scripts for the latest WASM API changes * wallet - fix unit tests * add integration tests and improve /test script * wasm - UtxoProcessor and UtxoContext APIs, improving demos * code cleanup * /test - fix pushd arguments * wasm - utxo context support for wasm tx generator * wip - js examples, padding for fees * cleanup (addressing consensus-related github review topics) * github review issues (phase II) * github review (phase 3) * fix /test script that was failing due to wasm-pack warning (wasm-pack issue) --------- Co-authored-by: max143672 <[email protected]> Co-authored-by: Surinder Singh Matoo <[email protected]> Co-authored-by: Maxim <[email protected]> Co-authored-by: Dennis Kievits <[email protected]> Co-authored-by: coderofstuff <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
KOS & CLI - platform-agnostic Rust-based Kaspa wallet framework, custom native shell, and desktop applications.
These are various application and framework components built on top of and integrated with the Rusty Kaspa project.
KOS is a desktop application with a terminal interface, and CLI is a custom shell (same as KOS) that can be run as a native application in a computer terminal or in a server environment.
This project is in the early alpha release stages and is being PRed to help test it (as well as help test Rusty Kaspa) with the upcoming
testnet-11reboot. It is also being PRed at this stage to prevent code divergence and unify as well as simplify the development process as some components of this stack are beneficial for Kaspa developers and integrators.KOS is built on top of multiple components - the core of which is the Rusty Kaspa Wallet framework that is built in Rust and can operate in WASM. The entire framework is being built as an API that allows direct integration within 3rd-party Rust and JavaScript/TypeScript applications. KOS is a test environment for this framework.
/kos- an installable desktop application capable of running on Windows, MacOS, and Linux operating systems (powered by NWJS)./cli- a command-line terminal application capable of running as a native command-line application./wallet/wasm- a web application, capable of running in a browser environment.All 3 variants are the same application/framework built in Rust on top of the Rusty Kaspa Wallet framework. (Enabled by WASM, the wallet framework is platform and environment agnostic, allowing for this kind of unique unified model)
The framework is comprised of the following key components:
CLI handlers
Command-line handlers (implement Handler trait) receive terminal commands and execute internal operations on the framework. Each instance has a startup and shutdown entrypoints, allowing each module to act as a plugin/service (making it very easy to introduce new or customized existing commands).
https://github.com/aspectron/rusty-kaspa/tree/kos/cli/src/modules
The CLI module can be started on its own as a command-line application. CLI also serves as a terminal handler within the KOS and the web wallet.
Wallet Runtime
https://github.com/aspectron/rusty-kaspa/tree/kos/wallet/core/src/runtime
The wallet runtime module contains structures that can be used by applications to interface with the wallet. They represent in-memory instances of data structures and allow for different operations on the wallet, accounts, etc., tracking their operational state.
Wallet Storage
https://github.com/aspectron/rusty-kaspa/tree/kos/wallet/core/src/storage
Offers the wallet data storage subsystem. Runtime components interface with the storage subsystem to store wallet and related data, such as transactions.
The wallet storage subsystem is abstracted by a number of storage interface traits
https://github.com/aspectron/rusty-kaspa/blob/kos/wallet/core/src/storage/interface.rs
These traits include: Private Key Storage, Account Data Storage, Transaction Record Storage
The framework implementation uses a "database access pattern" when interfacing with these storage traits, making it possible to interface them with an internal or an external database (i.e. allow a systems integrator to provide his own storage backend in Rust or in JavaScript).
The built-in storage interface called "local storage" offers file-backed storage which falls back on the browser's local storage when the browser environment is detected.
UTXO Processing Subsystem
https://github.com/aspectron/rusty-kaspa/tree/kos/wallet/core/src/utxo
UTXO processing subsystem is comprised of 2 main components: UtxoProcessor and UtxoContext. Both components are designed to work in tandem with the Utxo event notification system offered by the Rusty Kaspa framework. They aggregate and track the lifetime of incoming UTXOs making them available for generating transactions.
UtxoProcessor is the main "coordinator" while UtxoContext represents an account-like structure (but is designed to be general-purpose, so it does not know anything about "accounts", it only operates on sets of addresses).
Each
runtime::Accountowns UtxoContext and uses it to track UTXO sets that belong to it.UtxoContext tracks UTXO maturity as well as offers mature UTXOs as an iterator.
Mass Calculator
https://github.com/aspectron/rusty-kaspa/blob/kos/wallet/core/src/tx/mass.rs
The mass calculator in the wallet subsystem is refactored to remove the separation between byte size calculations and other variables that affect mass. You now have functions that only return mass - no other components. However, these functions break mass down into components such as "blank transaction", "input", and "output" (as well as fees).
This allows for a progressive creation of transactions, where you can measure the mass of each input while adding it to a transaction until you reach the transaction maximum.
Transaction Generator
https://github.com/aspectron/rusty-kaspa/tree/kos/wallet/core/src/tx/generator
UTXO Generator sources from a UTXO iterator (such as UtxoContext) and generates transactions. Transactions are generated progressively by iterating over the UTXO entry iterator and accumulating mass. If the mass threshold is reached, the current transaction is converted into a "batch" (a.k.a. sweep/compound) transaction going to the change address, and the next transaction is then set up for further processing.
Transaction Generator offers transactions as an Iterator or an async Stream.
So on one side, you have a UTXO iterator intake, and on the other side, you have a transaction stream/iterator that will produce transactions until the desired amount is reached (or various conditions occur).
Reasons and limitations of the design:
While the iterator/stream-based approach taken by the UtxoContext and Transaction Generator comes across as quite elegant (at least I think it does), the real reason for this design is constraints imposed by the web environment, where you are not allowed to block execution for extended periods of time or consume large amounts of memory in case of large UTXO sets. Having data processed one transaction at a time and having these transactions incoming as an async stream (or an iterator, which ends up being the same, as long as you can make async calls after each iteration) allows us to pace the execution while retaining the transaction generation context.
Here is a sample transaction generation process:
https://github.com/aspectron/rusty-kaspa/blob/kos/wallet/core/src/runtime/account.rs#L598-L608
The initial design that was attempting to chunk UTXO sets has proven to be ineffective and complex to work with. (OTOH the current Transaction Generator implementation is also quite complex as it ends up being a large loop accumulating transaction inputs).
Events
https://github.com/aspectron/rusty-kaspa/blob/kos/wallet/core/src/events.rs
Events are emitted by various framework components and provide state updates as well as notifications of transaction activity. This makes wallet and 3rd-party integration very simple as you can simply submit a transaction and receive activity details from the framework. It reduces the complexity because while you can get access to all information, you no longer need to track UTXOs. When a transaction comes in, you will be notified and you will then be notified when the transaction maturity was reached.
This is especially important when the network increases BPS because high-BPS processing will impose different logic for tx confirmation processing and this logic will be implemented within the framework by core developers. (or additional tools will be added to the framework to facilitate this).
JS/WASM32 API
KOS is meant to test-drive the above-mentioned components while we create JS/WASM32 bindings for them. The existing WASM API is fully functional (RPC + Transaction Primitives), however, transaction creation and signing are currently disabled as this needs additional refactoring. UtxoContext, UtxoProcessor, and TransactionGenerator will be made available as a part of the WASM32 framework. MassCalculator has already been exposed.
Strategical Considerations
Both CLI and KOS have been developed as terminal applications on purpose. This eliminates the UI/UX considerations (which can be an inhibitor to developers), while allowing any Rust developer very easily integrate and test any functionality on native and web platforms.
3rd-party integrations are meant to capitalize on the framework by integrating with it and using its components for various purposes. Every component of the framework is meant to function in isolation. I.e. they are not directly dependent on one another. You can use UtxoProcessor and Transaction Generator without the account or wallet structure, providing your own, or use the wallet and accounts, but provide your own storage backend.
The wallet is designed around the concept of active and inactive accounts, allowing the wallet to hold thousands of accounts, while on-demand activating or deactivating them. Such integrations would require additional effort, but they are possible from an architectural standpoint.
Other notes:
General
The most important element is that KOS represents a 100% rust-based alternative to KDX capable of running Rusty Kaspa. Other applications like KNG will effectively be a skin over this functionality.
A challenging issue, to date, has been that system architects - developers that work on the node have been, to a certain extent isolated from the application layer. As this framework is based on Rust, it allows everyone to use the same tools, breaking the client/server boundaries, so to speak.
Policies & Security Considerations
Due to various technical and security considerations, JavaScript code or toolchain components like NPM are strictly disallowed. The few JavaScript components involved have been embedded directly within Rust and are loaded by the Rust codebase at runtime.
Down the road, work will be done to reduce crate dependencies as well. In order to build the project, I had to use a variety of crates that are easy to implement in-house. These will be internalized.
Wallet Cache
When the wallet is open, it creates an in-memory cache and re-encrypts sensitive data. This allows to minimize storage access.
Transaction Storage
Due to a variety of technical considerations transaction history for CLI and KOS is stored as files. Each transaction is a separate file named with a hex transaction id.
Web IndexDb
Web transactions storage is not currently implemented. It will be backed by the browser IndexDb interface, but for now, the web wallet does not have a transaction history.
Unit and Integration Tests
A lot of work remains in this area and will be gradually addressed over the coming weeks and months.
Especially the challenge has been handling mutations within JS API. There are some ideas and plans to automate JS API testing via Rust integration tests. Just a question of priorities.
Polling and Subscriptions
Some limitations of Rusty Kaspa are currently bypassed using polling mechanisms as well as monitoring Kaspad logs. These will be improved and optimized in the future using the Rusty Kaspa notification subscription mechanism.
Intersection with the main node codebase
An effort has been made to avoid touching the main node codebase, with the exception of already integrated WASM bindings. The exception to this is 3 new RPC commands (get_system_metrics, get_server_status, and get_sync_status), which have been added as a part of the service but bound only to the wRPC pipeline. Their use will be reviewed by core devs in the coming weeks and perhaps analogous command will be added to gRPC.
Please also note that due to this wRPC/gRPC proxy project is deprecated (although KOS has a feature gate that allows it to be built with legacy RPC compatibility).