Releases: celestiaorg/celestia-node
v0.21.2-arabica
This release introduces two notable features:
- Bridge node <> core endpoint sync over gRPC
- Archival trimming of historic block files
Bridge node sync over gRPC
--core.grpc.port
flag.
Additionally, it's possible to configure TLS for the gRPC connection via the following new flags:
--core.tls Specifies whether TLS is enabled or not. Default: false
--core.xtoken.path string specifies the file path to the JSON file containing the X-Token for gRPC authentication. The JSON file should have a key-value pair where the key is 'x-token' and the value is the authentication token. NOTE: the path is parsed only if coreTLS enabled.If left empty, the client will not include the X-Token in its requests.
Archival trimming of historic block files
For archival node runners (both full node and bridge node), archival trimming will now be enabled by default, cutting archival node disk use for historic blocks in roughly half via #4028.
Note that the previous shwap release contains a condition to prevent nodes freshly syncing from storing the fourth quadrant files (.q4
files) for historic blocks that are outside the sampling window, but did not contain the functionality to clean up .q4
files once a block becomes freshly outdated, so disk use reductions are noticed only for the period between the completed full sync --> now.
What's Changed
- feat!(core): remove dependency of RPC in favour of gRPC by @vgonkivs in #3998
- deps(go.mod): bump app by @renaynay in #4056
- fix(share/shwap/p2p/bitswap): set archivalPool ctx by @renaynay in #4052
- fix(goreleaser): tag_sort by @cristaloleg in #3972
- nodebuilder/pruner: Enforce light pruning by @renaynay in #3912
- feat(availability): env var to disable pruning by @Wondertan in #3949
- fix(ci): add param to specify tag by @cristaloleg in #3979
- fix(share/availability): add missing protection for autobatch by @walldiss in #3988
- refactor(share/availability/light | share/availability/full): Availability implementations are aware of sampling window, removed from DASer by @renaynay in #3957
- fix(api): add time.Duration and rsmt2d.Axis types by @cristaloleg in #3994
- fix(cmd/blob): fix namespace parsing in cli by @vgonkivs in #4003
- chore/make: add celestia-node make file for arabica testnet by @Bidon15 in #3992
- fix(sync/test): fix TestSyncStartStopLightWithBridge by @vgonkivs in #3990
- perf(shwap): cache Both Row sides by @Wondertan in #4005
- refactor(docgen): simplify init by @cristaloleg in #3997
- fix(nodebuilder/da): remove random print by @renaynay in #4029
- feat(modshare): implement GetRow by @vgonkivs in #4002
- Revert "fix(p2p): disable quic (#3937)" by @Wondertan in #4039
- chore(pruner): remove SpacedHeaderGenerator by @ronething-bot in #4032
- feat(modp2p): Ping and ConnectionState APIs by @Wondertan in #3989
- feat(share/availability/full): Introduce Q4 trimming for archival nodes by @renaynay in #4028
- chore(deps): upgrade celestia-app by @rootulp in #4024
- feat: lunar use dnsaddr by @smuu in #3741
Full Changelog: v0.21.1-arabica...v0.21.2-arabica
v0.21.1-arabica
What's Changed
Full Changelog: v0.21.0-arabica...v0.21.1-arabica
v0.21.0-arabica
What's Changed
- fix(goreleaser): tag_sort by @cristaloleg in #3972
- nodebuilder/pruner: Enforce light pruning by @renaynay in #3912
- feat(availability): env var to disable pruning by @Wondertan in #3949
- fix(ci): add param to specify tag by @cristaloleg in #3979
- fix(share/availability): add missing protection for autobatch by @walldiss in #3988
- refactor(share/availability/light | share/availability/full): Availability implementations are aware of sampling window, removed from DASer by @renaynay in #3957
- fix(api): add time.Duration and rsmt2d.Axis types by @cristaloleg in #3994
- fix(cmd/blob): fix namespace parsing in cli by @vgonkivs in #4003
- chore/make: add celestia-node make file for arabica testnet by @Bidon15 in #3992
- fix(sync/test): fix TestSyncStartStopLightWithBridge by @vgonkivs in #3990
- perf(shwap): cache Both Row sides by @Wondertan in #4005
- refactor(docgen): simplify init by @cristaloleg in #3997
- fix(nodebuilder/da): remove random print by @renaynay in #4029
- feat(modshare): implement GetRow by @vgonkivs in #4002
- Revert "fix(p2p): disable quic (#3937)" by @Wondertan in #4039
- chore(pruner): remove SpacedHeaderGenerator by @ronething-bot in #4032
- feat(modp2p): Ping and ConnectionState APIs by @Wondertan in #3989
- feat(share/availability/full): Introduce Q4 trimming for archival nodes by @renaynay in #4028
- chore(deps): upgrade celestia-app by @rootulp in #4024
- feat: lunar use dnsaddr by @smuu in #3741
New Contributors
- @ronething-bot made their first contribution in #4032
Full Changelog: v0.20.4-arabica...v0.21.0-arabica
v0.20.4
This is a security patch release, please upgrade as soon as possible.
What's Changed
- refactor(share): GetShare -> GetSamples by @Wondertan in #3891
- deps(go.mod): bump app by @renaynay in #3968
- fix: jwt token nonce and expiration time by @cristaloleg in #3967
- chore(deps): upgrade to celestia-app v3.0.2 by @rootulp in #3969
Full Changelog: v0.20.3...v0.20.4
v0.20.4-mocha
This is a security patch release for the mocha network, please upgrade as soon as possible.
What's Changed
- fix(openrpc-gen): add go-square blob to examples by @vgonkivs in #3962
- fix(.github/workflows): no automatic tagging of releases for CI by @renaynay in #3963
- refactor(share): GetShare -> GetSamples by @Wondertan in #3891
- deps(go.mod): bump app by @renaynay in #3968
- fix: jwt token nonce and expiration time by @cristaloleg in #3967
- chore(deps): upgrade to celestia-app v3.0.2 by @rootulp in #3969
Full Changelog: v0.20.3-mocha...v0.20.4-mocha
v0.20.4-arabica
This is a security patch release for the arabica network, please upgrade as soon as possible.
What's Changed
- fix(openrpc-gen): add go-square blob to examples by @vgonkivs in #3962
- fix(.github/workflows): no automatic tagging of releases for CI by @renaynay in #3963
- refactor(share): GetShare -> GetSamples by @Wondertan in #3891
- deps(go.mod): bump app by @renaynay in #3968
- fix: jwt token nonce and expiration time by @cristaloleg in #3967
- chore(deps): upgrade to celestia-app v3.0.2 by @rootulp in #3969
Full Changelog: v0.20.3-arabica...v0.20.4-arabica
v0.20.3
This patch release fixes a JSON serialisation error with libshare.Blob via #3958, fixing the state.SubmitPayForBlob
endpoint.
What's Changed
- chore(deps): Bump cosmossdk.io/math from 1.3.0 to 1.4.0 by @dependabot in #3955
- chore(go.mod) bump go-square by @vgonkivs in #3958
- fix(openrpc-gen): add go-square blob to examples by @vgonkivs in #3962
- fix(.github/workflows): no automatic tagging of releases for CI by @renaynay in #3963
Full Changelog: v0.20.2...v0.20.3
v0.20.3-mocha
This patch release fixes a JSON serialisation error with libshare.Blob
via #3958.
What's Changed
- refactor: remove blob size check by @rootulp in #3934
- fix(core): change fill block mode by @cristaloleg in #3936
- hotfix(core): don't hold on Data by @Wondertan in #3926
- fix(p2p): disable quic by @walldiss in #3937
- chore(deps): temporary fork boxo by @Wondertan in #3939
- refactor(share/availability): Move window and constants to share/availability pkg by @renaynay in #3906
- fix(shwap/bitswap): tune bitswap params by @Wondertan in #3940
- fix(shwap/bitswap): Blockstore.GetSize: getting size with no compute by @Wondertan in #3894
- chore(logs): cleanup logging by @Wondertan in #3945
- fix(shwap/bitswap): update boxo and config by @Wondertan in #3946
- chore(go.mod): bump boxo fork by @Wondertan in #3948
- fix(share/availability/light): Allow retries on ODS size 1 by @renaynay in #3944
- perf(shwap/bitswap): add session pools inside Getter by @Wondertan in #3947
- chore(deps): Bump cosmossdk.io/math from 1.3.0 to 1.4.0 by @dependabot in #3955
- chore(go.mod) bump go-square by @vgonkivs in #3958
Full Changelog: v0.20.2-mocha...v0.20.3-mocha
v0.20.3-arabica
This patch release fixes a JSON serialisation error with libshare.Blob
via #3958.
What's Changed
- refactor: remove blob size check by @rootulp in #3934
- fix(core): change fill block mode by @cristaloleg in #3936
- hotfix(core): don't hold on Data by @Wondertan in #3926
- fix(p2p): disable quic by @walldiss in #3937
- chore(deps): temporary fork boxo by @Wondertan in #3939
- refactor(share/availability): Move window and constants to share/availability pkg by @renaynay in #3906
- fix(shwap/bitswap): tune bitswap params by @Wondertan in #3940
- fix(shwap/bitswap): Blockstore.GetSize: getting size with no compute by @Wondertan in #3894
- chore(logs): cleanup logging by @Wondertan in #3945
- fix(shwap/bitswap): update boxo and config by @Wondertan in #3946
- chore(go.mod): bump boxo fork by @Wondertan in #3948
- fix(share/availability/light): Allow retries on ODS size 1 by @renaynay in #3944
- perf(shwap/bitswap): add session pools inside Getter by @Wondertan in #3947
- chore(deps): Bump cosmossdk.io/math from 1.3.0 to 1.4.0 by @dependabot in #3955
- chore(go.mod) bump go-square by @vgonkivs in #3958
Full Changelog: v0.20.2-arabica...v0.20.3-arabica
v0.20.2
NOTE: v0.18 and v0.19 are skipped for mainnet
Hey everyone, this is quite a mammoth of a release, so please hang tight. We have two major changes packaged in this release:
- Shwap (protocol-breaking)
- Celestia v3 compatibility (API-breaking)
The migration period for this release is quite tight (a bit longer than 2 weeks), so we urge all node operators to upgrade to this version and re-sync against the network by December 2nd.
As this release is API-breaking, we also urge all integrations to migrate to this new version by December 2nd. (The list of API breaks can be found below).
Shwap
This release features Shwap(CIP-19), a protocol-breaking feature that brings massive optimizations to disk usage(16x) and data access latencies. The new protocol is not backwards compatible with the currently deployed protocol and software.
Both protocols will run side-by-side for about a two-week transition period, from November 20 to December 2. During this period, node operators should migrate to the new version. At the end of the period, posted historical and recent data will no longer be available on the old protocol.
Migration Notes
Migration requires resyncing the node from scratch, as it now contains a new storage format.
Migration Steps over the old node:
- Remove old data:
- In home directory(default
.celestia-<type>
) =>rm -rf inverted_index index data transients blocks
- Keep
keys
directory to preserve peer and chain identity keys in place
- In home directory(default
- Config updates
- If defaults were in use, remove the old config and re-init in the next step
- If you want to preserve the old config - move
BlockstoreCacheSize
entry from[Share.EDSStoreParams]
section to just[Share]
and removeGCInterval
,RoutingTableRefreshPeriod
- Re-init node for the Mocha p2p network
celestia init
- Start the node and monitor syncing
celestia start
If the old node has to be kept running - initiate and start the new version in a different directory:
celestia init --node.store <new_path>
celestia start --node.store <new_path>
Celestia V3 compatibility
This release contains an upgrade to the celestia-app version (to v3). v3 will only be live on the mainnet in mid-December, but this release contains the requirements for compatibility with the upgrade.
Notable API breaks include:
- simplification of the
blob
share
andstate
modules via the deduplication of types (via [#3782](#3782)) share
module now takesheight
instead of the fullExtendedHeader
(via [#3870](#3870))
Comprehensive list of API breaks:
GetSharesByNamespace
renamed toGetNamespaceData
- the
share
module returnsshwap.NamespaceData
for theGetNamespaceData
method - the
share
module now takesheight (uint64)
instead of the fullExtendedHeader
for all methods - both the
share
andblob
modules now use the go-square v2libshare.Namespace
(from https://github.com/celestiaorg/go-square/tree/v2.0.0) instead of the oldshare.Namespace
for all methods - the
state
module now uses the go-square v2libshare.Blob
type (from https://github.com/celestiaorg/go-square/tree/v2.0.0) instead of the old one from the state module - the
blob
package now embeds the new go-square v2libshare.Blob
type mentioned above in theblob.Blob
struct instead of the old v1 type.
Miscellaneous
What to look out for in the following releases:
- the
-experimental-pruning
flag will soon be deprecated in favour of pruning becoming the default mode for all node types, requiring users to pass-archival
to retain block history beyond the sampling window - the API gateway will be deprecated
What's Changed
Features
- feat(shwap): Add eds store by @walldiss in #3545
- feat(shwap): Add namespace data shwap id by @walldiss in #3587
- feat(shwap): Add axis roots to Accessor by @walldiss in #3586
- feat(shwap): Add bufferisation for writing ods/Q1Q4 files by @walldiss in #3592
- feat(shwap/p2p): Bitswap Getter by @Wondertan in #3584
- feat(shwap): Shwap add readFrom Writeto methods to shwap ids by @walldiss in #3605
- feat(shwap): Integrate shrex into shwap by @walldiss in #3554
- feat(shwap): Shwap core integration by @walldiss in #3598
- feat(shwap): Shwap cascade and mock getters by @walldiss in #3599
- feat(shwap): Update availability to work with shwap by @walldiss in #3600
- feat(shwap): add shwap to blob module by @walldiss in #3613
- feat(shwap): update daser for shwap by @walldiss in #3614
- feat(shwap): add shwap store getter by @walldiss in #3617
- feat(shwap): Add caching to blockstore by @walldiss in #3615
- feat(shwap/bitswap): Shwap optimized Bitswap constructors by @Wondertan in #3536
- feat(shwap): Improve eds store cache refcounter forgot Close() protection by @walldiss in #3638
- feat(shwap): shwap nodebuilder and cleanup by @walldiss in #3637
- feat(shwap/ods_file): tail padding trim by @Wondertan in #3620
- feat(shwap): Add discovery version by @walldiss in #3616
- feat(cmd/cel-shed): shwap by @Wondertan in #3720
- feat(shwap/store): Add q4 trimming support in store by @walldiss in #3680
- feat(nodebuilder): Blockstore cache params by @walldiss in #3739
- feat(share/availability): Dont store q4 on archival by @walldiss in #3751
- feat(store/cache): add Has methods by @Wondertan in #3757
- feat(cel-shed/p2p): bootstrapper probe tool by @smuu in #3780
- feat!: Live, laugh, Shwap by @Wondertan in #3675
- feat(cmd): adding timeout flag by @sebasti810 in #3580
- feat(nodebuilder/da): implement go-da v0.6.1 changes by @tzdybal in #3750
- feat(modp2p): close peer connections after successful blocking by @notlelouch in #3838
- feat(shwap/bitswap): use new option for optimized Has check by @Wondertan in #3813
- feat(store): Test all put funcs by @walldiss in #3847
- feat(store): use use relative path for empty blocks symlinks by @walldiss in #3861
- feat(store): correct corrupted files on write by @walldiss in #3859
- feat(share/availability)!: Store success results for light availability by @walldiss in #3887
- feat(pruner/light): add light pruning for shwap by @walldiss in #3896
Fixes
- fix(shwap/bitswap): close acccessor in Blockstore by @Wondertan in #3535
- fix(shwap/store): ensure EDS is accessible during writes by @Wondertan in #3596
- fix(shwap): don't cache empty eds and close recent accessor by @walldiss in #3608
- fix(store/files): ODSQ4 Closing audit by @Wondertan in #3691
- fix(shwap/cahce): use local copy of axis in proofs cache by @walldiss in #3657
- fix(shwap): io.ReadFull for read helpers by @Wondertan in #3696
- fix(shwap): io.ReadFull for NamespaceDataID by @walldiss in #3697
- fix(shwap/bitswap): correct protocol id by @Wondertan in #3722
- fix(store/odsq4): fix order of opened flag stored by @walldiss in #3735
- fix(store/ods): fix full ods caching by @walldiss in #3736
- fix(core): fix flaky generateNonEmptyBlocks by @walldiss in #3740
- fix(blob): fix blobsub subscription cancellation test by @walldiss in #3746
- fix(store): use symlinks for empty blocks heighs by @walldiss in #3747
- fix(header): soft failure is only for not enough voting power by @cristaloleg in #3769...