Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fastpath] add a basic TransactionDriver #20088

Closed
wants to merge 49 commits into from

Conversation

mwtian
Copy link
Contributor

@mwtian mwtian commented Oct 30, 2024

Description

Describe the changes or additions included in this PR.

Test plan

How did you test the new or updated feature?


Release notes

Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required.

For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates.

  • Protocol:
  • Nodes (Validators and Full nodes):
  • Indexer:
  • JSON-RPC:
  • GraphQL:
  • CLI:
  • Rust SDK:
  • REST API:

Copy link

vercel bot commented Oct 30, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
sui-docs ✅ Ready (Inspect) Visit Preview 💬 Add feedback Nov 1, 2024 1:39am
3 Skipped Deployments
Name Status Preview Comments Updated (UTC)
multisig-toolkit ⬜️ Ignored (Inspect) Visit Preview Nov 1, 2024 1:39am
sui-kiosk ⬜️ Ignored (Inspect) Visit Preview Nov 1, 2024 1:39am
sui-typescript-docs ⬜️ Ignored (Inspect) Visit Preview Nov 1, 2024 1:39am

aschran and others added 13 commits October 30, 2024 16:14
---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol: 
- [ ] Nodes (Validators and Full nodes): 
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [ ] GraphQL: 
- [ ] CLI: 
- [ ] Rust SDK:
- [ ] REST API:
## Description 

Renames the `gas-profiler` feature flag to `tracing`. Otherwise
everything else is unchanged.

If you were previously building with `--features gas-profiler` you
should use `--features tracing` and everything should work and behave as
before.

Renamings performed:
1. `gas-profiler` => `tracing`
2. `gas_profiler_feature` => `tracing_feature`

## Test plan 

CI

---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol: 
- [ ] Nodes (Validators and Full nodes): 
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [ ] GraphQL: 
- [X] CLI: The `gas-profiler` Rust feature flag was renamed to
`tracing`. If you were previously building the Sui CLI from source with
`--features gas-profiler` this will no longer work, and you should
instead use `--features tracing`. This will enable the same features as
before.
- [ ] Rust SDK:
- [ ] REST API:
## Description 

Updates to docs renaming the `gas-profiler` feature flag to `tracing`
## Description 

Reuse the work within `move-compiler/src/diagnostics/` instead of
reimplementing code span reporting usage directly.

## Test plan 

Snapshots

---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol: 
- [ ] Nodes (Validators and Full nodes): 
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [ ] GraphQL: 
- [ ] CLI: 
- [ ] Rust SDK:
- [ ] REST API:
## Description 

Describe the changes or additions included in this PR.

## Test plan 

How did you test the new or updated feature?

---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol: 
- [ ] Nodes (Validators and Full nodes): 
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [ ] GraphQL: 
- [ ] CLI: 
- [ ] Rust SDK:
- [ ] REST API:
## Description 
Add emoji to slack error messages

## Test plan 
👀
…he VM (#20084)

## Description 

This unifies the old tracing under the `tracing` flag, and removes the
`debugging` feature flag from the VM (one less feature flag 🎉).
Otherwise behavior is kept the exact same.

## Test plan 

Tested manually + CI

---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol: 
- [ ] Nodes (Validators and Full nodes): 
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [ ] GraphQL: 
- [ ] CLI: 
- [ ] Rust SDK:
- [ ] REST API:
## Description 

This adds support for raw `abort`s in a similar manner to raw
`assert!`ions. It will introduce the line number that the `abort` was
called on (and behaves in the same manner as `assert!` with no error
codes for macro line number remapping).

## Test plan 

Added parser tests as well as additional transactional tests, and
verified that they produce the exact same output as the derived line
number assertion tests that they copy.
## Description 
This will be used by the new edge proxy for routing execute transaction
requests to fullnodes with lower consensus latency

## Test plan 

unit tests

---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol: 
- [ ] Nodes (Validators and Full nodes): 
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [ ] GraphQL: 
- [ ] CLI: 
- [ ] Rust SDK:
- [ ] REST API:
## Description 

This PR fixes a minor inefficiency in the coin metadata api.
It does not need to fetch the transaction, as it only needs the effects.
This likely avoids a remote KV fetch.

## Test plan 

CI

---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol: 
- [ ] Nodes (Validators and Full nodes): 
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [ ] GraphQL: 
- [ ] CLI: 
- [ ] Rust SDK:
- [ ] REST API:
…20080)

## Description 

Adds support for native functions where the `CloseFrame` event happens
right after `OpenFrame` event and both need to be skipped to skip over
the native function. Also added support for handling global locations -
they should be parsed properly but largely ignored in the trace as they
cannot directly affect values of local variables

## Test plan 

All old and new tests must pass
## Description 

Disallow new modules from being added to packages in deps-only mode. 

## Test plan 

Added new tests to check for both previous and new behavior.

---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [X] Protocol: Updated restrictions on deps-only packages to not allow
adding new modules to the package.
- [ ] Nodes (Validators and Full nodes): 
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [ ] GraphQL: 
- [ ] CLI: 
- [ ] Rust SDK:
- [ ] REST API:
#20101)

## Description 
enables periodic compaction of individual SST files by default

---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol: 
- [ ] Nodes (Validators and Full nodes): 
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [ ] GraphQL: 
- [ ] CLI: 
- [ ] Rust SDK:
- [ ] REST API:
lxfind and others added 7 commits October 30, 2024 22:25
## Description 

Describe the changes or additions included in this PR.

## Test plan 

How did you test the new or updated feature?

---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol: 
- [ ] Nodes (Validators and Full nodes): 
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [ ] GraphQL: 
- [ ] CLI: 
- [ ] Rust SDK:
- [ ] REST API:
Remove the ListFullCheckpoint api as its a bit too heavy-weight and
instead we'll look to introduce a streaming api in the future.
Revert the client change to request protobuf as a response type, and
instead request bcs, because the support for protobuf hasn't rolled out
to other networks yet.
wlmyng and others added 12 commits October 31, 2024 10:18
## Description 

Describe the changes or additions included in this PR.

## Test plan 

How did you test the new or updated feature?

---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol: 
- [ ] Nodes (Validators and Full nodes): 
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [ ] GraphQL: 
- [ ] CLI: 
- [ ] Rust SDK:
- [ ] REST API:
## Description 

Updates the RPC render to include component schemas. Also updates a few
comments.

## Test plan 

How did you test the new or updated feature?

---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol: 
- [ ] Nodes (Validators and Full nodes): 
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [ ] GraphQL: 
- [ ] CLI: 
- [ ] Rust SDK:
- [ ] REST API:
This PR was opened by the [Changesets
release](https://github.com/changesets/action) GitHub action. When
you're ready to do a release, you can merge this and publish to npm
yourself or [setup this action to publish
automatically](https://github.com/changesets/action#with-publishing). If
you're not ready to do a release yet, that's fine, whenever you add more
changesets to main, this PR will be updated.


# Releases
## @mysten/[email protected]

### Minor Changes

- c24814b: Adds a custom header; 'Client-Request-Method' which will
contain the method name used in
    each outgoing jsonrpc request

## @mysten/[email protected]

### Patch Changes

-   Updated dependencies [c24814b]
    -   @mysten/[email protected]
    -   @mysten/[email protected]

## @mysten/[email protected]

### Patch Changes

-   Updated dependencies [c24814b]
    -   @mysten/[email protected]
    -   @mysten/[email protected]
    -   @mysten/[email protected]

## @mysten/[email protected]

### Patch Changes

-   Updated dependencies [c24814b]
    -   @mysten/[email protected]

## @mysten/[email protected]

### Patch Changes

-   Updated dependencies [c24814b]
    -   @mysten/[email protected]

## @mysten/[email protected]

### Patch Changes

-   Updated dependencies [c24814b]
    -   @mysten/[email protected]
    -   @mysten/[email protected]

## @mysten/[email protected]

### Patch Changes

-   Updated dependencies [c24814b]
    -   @mysten/[email protected]

## @mysten/[email protected]

### Patch Changes

-   4166d71: Fix doc comment on `getKiosk` command
-   Updated dependencies [c24814b]
    -   @mysten/[email protected]

## @mysten/[email protected]

### Patch Changes

- b3f3925: Introduces @mysten/kms which initially exposes a Sui AWS KMS
signer
-   Updated dependencies [c24814b]
    -   @mysten/[email protected]

## @mysten/[email protected]

### Patch Changes

-   Updated dependencies [c24814b]
    -   @mysten/[email protected]

## @mysten/[email protected]

### Patch Changes

-   Updated dependencies [c24814b]
    -   @mysten/[email protected]

## @mysten/[email protected]

### Patch Changes

-   Updated dependencies [c24814b]
    -   @mysten/[email protected]

## @mysten/[email protected]

### Patch Changes

-   Updated dependencies [c24814b]
    -   @mysten/[email protected]
    -   @mysten/[email protected]

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
…PC interface (#19796)

Client side use of TLS will be enabled-by-default in a future PR.

---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol: 
- [x] Nodes (Validators and Full nodes): Adds support for TLS on
validator gRPC interface.
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [ ] GraphQL: 
- [ ] CLI: 
- [ ] Rust SDK:
- [ ] REST API:
## Description 

Introduce a system invariant violation macro.
We can then set up alerts whenever the counter value changes.
Used it in one place as an example.

## Test plan 

How did you test the new or updated feature?

---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol: 
- [ ] Nodes (Validators and Full nodes): 
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [ ] GraphQL: 
- [ ] CLI: 
- [ ] Rust SDK:
- [ ] REST API:
## Description 

Describe the changes or additions included in this PR.

## Test plan 

How did you test the new or updated feature?

---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol: 
- [ ] Nodes (Validators and Full nodes): 
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [ ] GraphQL: 
- [ ] CLI: 
- [ ] Rust SDK:
- [ ] REST API:
…0018)

## Description 

adding BigTable KV as a variant to main binary that manages internal
workflows

---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol: 
- [ ] Nodes (Validators and Full nodes): 
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [ ] GraphQL: 
- [ ] CLI: 
- [ ] Rust SDK:
- [ ] REST API:
## Description 

Introduce a new payload type into the benchmark tool -
`ExpectedFailurePayload`.

This payload type is configured with some expected failure type
(currently only one implemented - user signature failure - but others
can be added with minimal scaffolding) such that
`payload.make_transaction()` generates a transaction that will fail in
this manner. Note that the failures that this payload type is concerned
with are failures to execute the transaction itself, rather than during
execution by the MoveVM. In other words, it is expected that the failure
mode will not consume gas or produce effects.

Note that for this reason, its failure mode is inverted. It will be
tallied for metrics purposes as an `error` if it succeeds and tallied as
an `expected_error` (which is a success) if it fails.

Also note that transaction responses for this type are handled by
producing a `NextOp::Retry` (with some additional logging/metrics) since
it is functionally equivalent to a retryable error.

## Test plan 

Ran the following:
```
SIM_STRESS_TEST_QPS=200 SIM_STRESS_TEST_WORKERS=20 RUST_LOG=debug cargo simtest --nocapture test_simulated_load_expected_failure_traffic_control
```

Observed the following in logs
```
2022-01-03T02:05:47.734507Z  INFO node{id=1 name="client"}: simtest::test: crates/sui-benchmark/tests/simtest.rs:1088: end of test BenchmarkStats { duration: 50.000002242s, num_error_txes: 0, num_expected_error_txes: 8049, num_success_txes: 348, num_success_cmds: 1977, total_gas_used: 3215776400, latency_ms: HistogramWrapper { histogram: Histogram { auto_resize: false, highest_trackable_value: 120000, lowest_discernible_value: 1, significant_value_digits: 3, bucket_count: 7, sub_bucket_count: 2048, sub_bucket_half_count: 1024, sub_bucket_half_count_magnitude: 10, sub_bucket_mask: 2047, leading_zero_count_base: 53, unit_magnitude: 0, unit_magnitude_mask: 0, max_value: 14359, min_non_zero_value: 72, total_count: 348, counts: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 2, 8, 12, 9, 20, 25, 22, 19
...
```

[Also ran in
PTN](https://metrics.sui.io/d/adl51ctsvmkg0a/traffic-control-dos-protection-dashboard?from=2024-10-31T19:00:50.548Z&to=2024-10-31T20:20:03.673Z&timezone=browser&var-network=private-testnet&var-host=$__all&var-fullnode=$__all)

---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol: 
- [ ] Nodes (Validators and Full nodes): 
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [ ] GraphQL: 
- [ ] CLI: 
- [ ] Rust SDK:
- [ ] REST API:
## Description 

Build will always treat `src` as the base for outputs when building.

## Test plan 

How did you test the new or updated feature?

---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol: 
- [ ] Nodes (Validators and Full nodes): 
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [ ] GraphQL: 
- [ ] CLI: 
- [ ] Rust SDK:
- [ ] REST API:
This PR was opened by the [Changesets
release](https://github.com/changesets/action) GitHub action. When
you're ready to do a release, you can merge this and publish to npm
yourself or [setup this action to publish
automatically](https://github.com/changesets/action#with-publishing). If
you're not ready to do a release yet, that's fine, whenever you add more
changesets to main, this PR will be updated.


# Releases
## @mysten/[email protected]

### Patch Changes

-   02c9e46: Fix exports on the bundled package

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
… not work (#20111)

## Description 

fix `sui keytool import` --alias command and default alias does not work

Before Fix:
<img width="1295" alt="image"
src="https://github.com/user-attachments/assets/4cb40d92-e266-4b6e-9c39-c05ac52d7a9d">

After Fix:

- **--alias**
<img width="1301" alt="image"
src="https://github.com/user-attachments/assets/b5c711e8-2a4d-4861-9965-30cde260e93f">

- **default alias**
<img width="1295" alt="image"
src="https://github.com/user-attachments/assets/4513db65-0e10-4179-b5f0-de188871cf78">

## Test plan 

How did you test the new or updated feature?

---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol: 
- [ ] Nodes (Validators and Full nodes): 
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [ ] GraphQL: 
- [x] CLI: `sui keytool import --alias` is now fixed and correctly
records the provided alias.
- [ ] Rust SDK:
- [ ] REST API:
…20122)

## Description 

Adds support for querying checkpoint by digest

---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol: 
- [ ] Nodes (Validators and Full nodes): 
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [ ] GraphQL: 
- [ ] CLI: 
- [ ] Rust SDK:
- [ ] REST API:
gegaowp and others added 13 commits October 31, 2024 23:30
## Description 

title, so that we can replace hard-coded field count in pipelines.
a crate did very similar things but it's archived, so prob better to
have our own impl, also b/c the functionality is relatively simple
https://github.com/discosultan/field-count

## Test plan 

added tests in models/

---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol: 
- [ ] Nodes (Validators and Full nodes): 
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [ ] GraphQL: 
- [ ] CLI: 
- [ ] Rust SDK:
- [ ] REST API:
@mwtian mwtian force-pushed the tmw/consensus-txn-client branch from 4cc50ef to d79675a Compare November 1, 2024 01:36
@github-actions github-actions bot added the Type: Documentation Improvements or additions to documentation label Nov 1, 2024
@mwtian mwtian closed this Nov 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Type: Documentation Improvements or additions to documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.