Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pull changes from upstream master #629

Closed
wants to merge 313 commits into from

Conversation

evan-forbes
Copy link
Member

@evan-forbes evan-forbes commented Feb 9, 2022

Description

This PR pulls changes from upstream, including ABCI++ changes. leaving as a draft for now as there is a lot of active work going on upstream, and we will want to pull those changes soon.

dependabot bot and others added 30 commits February 2, 2022 15:37
…287)

Bumps [github.com/tendermint/tm-db](https://github.com/tendermint/tm-db) from 0.6.4 to 0.6.6.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/tendermint/tm-db/releases">github.com/tendermint/tm-db's releases</a>.</em></p>
<blockquote>
<h2>Release v0.6.6</h2>
<p>This release is the same API as v0.6.4, but bypasses the accidental v0.6.5
release that exported a different API. Any existing users of v0.6.5 may continue
to use that tag.</p>
<p><a href="https://github.com/tendermint/tm-db/blob/v0.6.6/CHANGELOG.md">https://github.com/tendermint/tm-db/blob/v0.6.6/CHANGELOG.md</a></p>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href="https://github.com/tendermint/tm-db/blob/master/CHANGELOG.md">github.com/tendermint/tm-db's changelog</a>.</em></p>
<blockquote>
<h2>0.6.6</h2>
<p><strong>2021-11-08</strong></p>
<p><strong>Important note:</strong> Version v0.6.5 was accidentally tagged and should be
avoided.  This version is identical to v0.6.4 in package structure and API, but
has updated the version marker so that normal <code>go get</code> upgrades will not
require modifying existing use of v0.6.4.</p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="https://github.com/tendermint/tm-db/commit/0e6efdda1b15ef02cdb8f66b36bac942943ab7d5"><code>0e6efdd</code></a> Prepare a new compatibility release v0.6.6. (<a href="https://github-redirect.dependabot.com/tendermint/tm-db/issues/203">#203</a>)</li>
<li><a href="https://github.com/tendermint/tm-db/commit/8f92601b6539a3611063709f65256150e198cfd9"><code>8f92601</code></a> testing: tools/Dockerfile: Upgrade RocksDB to 6.24.2 (<a href="https://github-redirect.dependabot.com/tendermint/tm-db/issues/197">#197</a>)</li>
<li><a href="https://github.com/tendermint/tm-db/commit/16415bfbd73b4a28e46c8c4f72f3285a5f8b6358"><code>16415bf</code></a> build(deps): bump google.golang.org/grpc from 1.38.0 to 1.42.0 (<a href="https://github-redirect.dependabot.com/tendermint/tm-db/issues/193">#193</a>)</li>
<li><a href="https://github.com/tendermint/tm-db/commit/11c5e90c12f9151ebe03386af4b650c1438fb39e"><code>11c5e90</code></a> Update mergify configuration. (<a href="https://github-redirect.dependabot.com/tendermint/tm-db/issues/198">#198</a>)</li>
<li><a href="https://github.com/tendermint/tm-db/commit/d3744d52e8bb46b3bbe76ea48e850267eb893a29"><code>d3744d5</code></a> Retract the accidentally-released v0.6.5. (<a href="https://github-redirect.dependabot.com/tendermint/tm-db/issues/194">#194</a>)</li>
<li><a href="https://github.com/tendermint/tm-db/commit/6dc0379403b860e475b9b05d0baef15eaa7ba5f6"><code>6dc0379</code></a> Update CODEOWNERS file. (<a href="https://github-redirect.dependabot.com/tendermint/tm-db/issues/196">#196</a>)</li>
<li><a href="https://github.com/tendermint/tm-db/commit/63a566e76533e635270fa8a860614de0623c2533"><code>63a566e</code></a> Enable cgo for testing. (<a href="https://github-redirect.dependabot.com/tendermint/tm-db/issues/200">#200</a>)</li>
<li><a href="https://github.com/tendermint/tm-db/commit/451c8444fb1732632a7a9b1f2ccdc491b70d6d81"><code>451c844</code></a> Revert rocksdb version at tip. (<a href="https://github-redirect.dependabot.com/tendermint/tm-db/issues/202">#202</a>)</li>
<li><a href="https://github.com/tendermint/tm-db/commit/d65205892e9e5ab5606161acc86c75a5cdc8233d"><code>d652058</code></a> build: fix the image to work with golangci-lint (<a href="https://github-redirect.dependabot.com/tendermint/tm-db/issues/201">#201</a>)</li>
<li><a href="https://github.com/tendermint/tm-db/commit/8882f2706633b2944b4084961fab1aaeb7ec3c09"><code>8882f27</code></a> Update Go version in the build container. (<a href="https://github-redirect.dependabot.com/tendermint/tm-db/issues/195">#195</a>)</li>
<li>Additional commits viewable in <a href="https://github.com/tendermint/tm-db/compare/v0.6.4...v0.6.6">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=github.com/tendermint/tm-db&package-manager=go_modules&previous-version=0.6.4&new-version=0.6.6)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


</details>
* proto: rebuild the proto files from the spec repository

* remove proto checks
The event examples for the query filter language were not updated after the
change of key and value types from []byte to string. Also, the attributes need
to be a slice not a bare value.
Rework the implementation of event query parsing and execution to
improve performance and reduce memory usage.

Previous memory and CPU profiles of the pubsub service showed query
processing as a significant hotspot. While we don't have evidence that
this is visibly hurting users, fixing it is fairly easy and self-contained.

Updates #6439.

Typical benchmark results comparing the original implementation (PEG) with the reworked implementation (Custom):
```
TEST                        TIME/OP  BYTES/OP  ALLOCS/OP  SPEEDUP   MEM SAVING
BenchmarkParsePEG-12       51716 ns  526832    27
BenchmarkParseCustom-12     2167 ns    4616    17         23.8x     99.1%
BenchmarkMatchPEG-12        3086 ns    1097    22
BenchmarkMatchCustom-12    294.2 ns      64     3         10.5x     94.1%
```

Components:
* Add a basic parsing benchmark.
* Move the original query implementation to a subdirectory.
* Add lexical scanner for Query expressions.
* Add a parser for Query expressions.
* Implement query compiler.
* Add test cases based on OpenAPI examples.
* Add MustCompile to replace the original MustParse, and update usage.
Bumps [watchpack](https://github.com/webpack/watchpack) from 2.2.0 to 2.3.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/webpack/watchpack/releases">watchpack's releases</a>.</em></p>
<blockquote>
<h2>v2.3.0</h2>
<h1>Features</h1>
<ul>
<li>allow to grab separate file and directory time info objects</li>
<li>allow functions passed to the <code>ignored</code> option</li>
</ul>
<h1>Bugfixes</h1>
<ul>
<li>ignore EACCESS errors during initial scan</li>
</ul>
<h1>Performance</h1>
<ul>
<li>improve performance of watcher update</li>
</ul>
<h1>Contributing</h1>
<ul>
<li>CI tests node.js 17 too</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="https://github.com/webpack/watchpack/commit/b903dd384da582ae5f1e6ca71586fed3d991de7f"><code>b903dd3</code></a> 2.3.0</li>
<li><a href="https://github.com/webpack/watchpack/commit/c82a595128e75efb3b0b8f0d1a819b4bf95119bf"><code>c82a595</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/webpack/watchpack/issues/211">#211</a> from webpack/bugfix/missing-info</li>
<li><a href="https://github.com/webpack/watchpack/commit/1e560268d3409760c6a68779a157bf30893484c2"><code>1e56026</code></a> fix missing time info in files</li>
<li><a href="https://github.com/webpack/watchpack/commit/89d5f4884c414aafede5ca26a5fe0ecfe9c6cb11"><code>89d5f48</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/webpack/watchpack/issues/205">#205</a> from markjm/markjm/split</li>
<li><a href="https://github.com/webpack/watchpack/commit/f1f35863303c1938d8c74f57160e61f9de94ff8d"><code>f1f3586</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/webpack/watchpack/issues/210">#210</a> from webpack/ci/no-macos-polling</li>
<li><a href="https://github.com/webpack/watchpack/commit/8d14e94274843d30a6c95a0bd602ad79fe58b720"><code>8d14e94</code></a> Merge branch 'main' into markjm/split</li>
<li><a href="https://github.com/webpack/watchpack/commit/e71b62ba1f3d9be6d7b95083e85f79b811e2e094"><code>e71b62b</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/webpack/watchpack/issues/197">#197</a> from markjm/markjm/watch-change</li>
<li><a href="https://github.com/webpack/watchpack/commit/a3b2b82ec7df1e86119653cea113a5a2bb78030c"><code>a3b2b82</code></a> provide additional method instead of changing existing one</li>
<li><a href="https://github.com/webpack/watchpack/commit/aab36974181d6971c0d7a81499f1df87c7802e88"><code>aab3697</code></a> disable testing polling for macos ci</li>
<li><a href="https://github.com/webpack/watchpack/commit/523793ef5dcdc38cc62045079a9500068c6908bd"><code>523793e</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/webpack/watchpack/issues/208">#208</a> from webpack/perf/update-watchers</li>
<li>Additional commits viewable in <a href="https://github.com/webpack/watchpack/compare/v2.2.0...v2.3.0">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=watchpack&package-manager=npm_and_yarn&previous-version=2.2.0&new-version=2.3.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


</details>
* eventbus: plumb contexts

* fix lint
Prior to this change, shutting down the pubsub server could cause
any laggard publishers to race with the shutdown plumbing.

Fix that race condition, and plumb in the service context to the
runner so that it will respect the external signal directly.
Remove now-redundant local shutdown plumbing.
This is probably belt and suspenders, but might help the apphash clarity.
This is a more clear version of the stable parts of the apphash tests.
I saw a race detected in a test here that I think would be better
handled by just wiring up these threads.
This might be a source of non-determinism in the e2e test.
Some of our tests were creating a psql event sink and expecting
it to report (or not report) certain kinds of errors. These tests
were ill-founded in a couple of ways:

1. Tests that required the Postgres driver were not loading it.
   This led to spurious successes on tests that wanted "some error"
   from the sink constructor, but didn't exercise the right path.

2. Tests that wanted a Postgres sink to succeed without a database.
   These tests "passed" because they weren't actually establishing a
   connection to the database, but if they had would have failed for
   the lack of one.

To fix this:
- Load the postgres driver in tests that need it.
- Verify connectivity before reporting successful creation of a PSQL event sink.
- Remove tests that wanted a psql sink without a database, since that case
  is already tested elsewhere.
…s in test (#7362)

The current testing for this tool relies on hardcoding a set of configs into the tests. This means that when the config structure changes, the tests break. I'm not sure that this makes sense since we are separately testing our ability to read and validate the configuration file format. Hardcoding bytes into a different file duplicates this. Using the structs themselves should therefore be preferred.
tychoish and others added 20 commits February 2, 2022 15:51
* rpc: simplify the handling of JSON-RPC request and response IDs

Replace the ID wrapper interface with plain JSON. Internally, the client
libraries use only integer IDs, and the server does not care about the ID
structure apart from checking its validity.

Basic structure of this change:

- Remove the jsonrpcid interface and its helpers.
- Unexport the ID field of request and response.
- Add helpers for constructing requests and responses.
- Fix up usage and tests.
* rpc: fix layout of endpoint list

The output of the default endpoint-list query was not correctly segregating
methods with and without arguments. Fix this, and also clean up the output to
be easier to read (both in code and in generated source).

Fixes #3618.
* add proto, add boilerplates

* add canonical

* fix tests

* add vote signing test

* Update internal/consensus/msgs_test.go

* modify state execution in progress

* add extension signing

* VoteExtension -> ExtendVote

* apply review

* update data structures

* Add comments

* Apply suggestions from code review

Co-authored-by: Aleksandr Bezobchuk <[email protected]>

* *Signed -> *ToSign

* add Vote to RequestExtendVote

* apply reviews

* Apply suggestions from code review

Co-authored-by: Dev Ojha <[email protected]>
Co-authored-by: Aleksandr Bezobchuk <[email protected]>

* fix typo, modify proto

Co-authored-by: Aleksandr Bezobchuk <[email protected]>
Co-authored-by: Dev Ojha <[email protected]>
* add proto, add boilerplates

* add canonical

* fix tests

* add vote signing test

* Update internal/consensus/msgs_test.go

* modify state execution in progress

* add extension signing

* add extension signing

* VoteExtension -> ExtendVote

* modify state execution in progress

* add extension signing

* verify in progress

* modify CommitSig

* fix test

* apply review

* update data structures

* Apply suggestions from code review

* Add comments

* fix test

* VoteExtensionSigned => VoteExtensionToSigned

* Apply suggestions from code review

Co-authored-by: Aleksandr Bezobchuk <[email protected]>

* *Signed -> *ToSign

* add Vote to RequestExtendVote

* add example VoteExtension

* apply reviews

* fix vote

* Apply suggestions from code review

Co-authored-by: Dev Ojha <[email protected]>
Co-authored-by: Aleksandr Bezobchuk <[email protected]>

* fix typo, modify proto

* add abcipp_kvstore.go

* add extension test

* fix test

* fix test

* fix test

* fit lint

* uncomment test

* refactor test in progress

* gofmt

* apply review

* fix lint

Co-authored-by: Aleksandr Bezobchuk <[email protected]>
Co-authored-by: Dev Ojha <[email protected]>
… of EvidenceList (#7700)

* evidence: Refactored the evidence message to send/recieve Evidence instead of EvidenceList

Peers send evidences one by one. The refactored code reflects this behaviour.

* Update internal/evidence/reactor.go

Co-authored-by: M. J. Fromberger <[email protected]>

Co-authored-by: M. J. Fromberger <[email protected]>
Co-authored-by: William Banfield <[email protected]>
This change adds logic to double the message delay bound after every 10 rounds. Alternatives to this somewhat magic number were discussed. Specifically, whether or not to make '10' modifiable as a parameter was discussed. Since this behavior only exists to ensure liveness in the case that these values were poorly chosen to begin with, a method to configure this value was not created. Chains that notice many 'untimely' rounds per the [relevant metric](tendermint/tendermint#7709) are expected to take action to increase the configured message delay to more accurately match the conditions of the network. 

closes: tendermint/spec#371
* rpc/client: remove the placeholder RunState type.

I added the RunState type in #6971 to disconnect clients from the service
plumbing, which they do not need. Now that we have more complete context
plumbing, the lifecycle of a client no longer depends on this type: It serves
as a carrier for a logger, and a Boolean flag for "running" status, neither of
which is used outside of tests.

Logging in particular is defaulted to a no-op logger in all production use.
Arguably we could just remove the logging calls, since they are never invoked
except in tests. To defer the question of whether we should do that or make the
logging go somewhere more productive, I've preserved the existing use here.

Remove use of the IsRunning method that was provided by the RunState, and use
the Start method and context to govern client lifecycle.

Remove the one test that exercised "unstarted" clients. I would like to remove
that method entirely, but that will require updating the constructors for all
the client types to plumb a context and possibly other options. I have deferred
that for now.
@evan-forbes evan-forbes self-assigned this Feb 9, 2022
@evan-forbes
Copy link
Member Author

evan-forbes commented Feb 16, 2022

While at first I was hoping that we would be able to build features of the app using tendmerint master, this will not be possible without quite a bit of work to the sdk and the app. Instead, #631 pulls only the two new methods that we need into the v0.34.x branch, which makes them trivial to use with our fork of the sdk and the app.

This PR can still be used, as we need to pull from upstream anyway.

@evan-forbes
Copy link
Member Author

closing as we replaced our default branch with v0.34.x-celestia

@rootulp rootulp deleted the evan/rebase-upstream-master branch August 12, 2022 19:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
Archived in project
Development

Successfully merging this pull request may close these issues.