chore(deps): update dependency llvm_zstd to v1.5.7 #469
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![renovate[bot]](https://avatars.githubusercontent.com/in/2740?s=60&v=4){kind=small}
renovate
bot
force-pushed
the
renovate/llvm_zstd-1.x
branch
from
98f1402 to
f5b51ee
Compare
renovate
bot
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR contains the following updates:
Test plan: CI should pass with updated dependencies. No review required: this is an automated dependency update PR.
Release Notes
facebook/zstd (llvm_zstd)
v1.5.7: Zstandard v1.5.7Compare Source
Zstandard
v1.5.7is a significant release, featuring over 500 commits accumulated over the past year. This update brings enhancements across various domains, including performance, stability, and functionality, and is particularly recommended for 32-bit users due to a bug fix detailed below.Performance Enhancements
Enhanced Compression Speed for Small Data Blocks
The compression speed for small data blocks has been notably improved at fast compression levels, thanks to contributions from @TocarIP, further extended in #4165. Below are benchmark results illustrating the performance improvements for level 1 compression on the Silesia corpus, segmented into different block sizes:
These improvements are particularly beneficial in data centers and databases, where data is often divided into small segments to enhance seekability. A notable example is RocksDB, which is frequently configured with block sizes around 16 KB. Such systems will experience an immediate performance boost by updating their compression library to
v1.5.7.Additionally, the speed improvements partially extend to dictionary compression, with observed gains ranging from none to +15%.
Substantial
--patch-fromimprovementsThe
--patch-fromfunctionality of thezstdCLI allows for differential compression of a file given an older version, similar tobsdiff. While compression levels impact processing speed and compression ratio as expected, at high compression levels (18+), speed was previously reduced to uncomfortable levels.v1.5.7largely mitigates the speed impact of high compression levels 18+, as illustrated in the following scenario, which compresseslinux-6.13.taras a difference fromlinux-6.12.tarusing 6 threads:Additionally, compression ratio of
--patch-fromis also substantially improved across the board, including higher compression levels:--maxCompression ratio improvements for large files
The compression ratio has been enhanced slightly for large data across all compression levels, thanks to a refined approach in selecting block boundaries. This improvement is more pronounced at lower compression levels (≤ 15), where the previous boundary selection method was relatively basic. In contrast, higher compression levels (16+) already employed some analysis, but still benefit from the introduction of the new heuristic, resulting in marginally better compression ratios.
A comparison of the compression ratios between
v1.5.6andv1.5.7is presented below:v1.5.6v1.5.7Command-Line Interface Updates
Recognizing the prevalence of multi-core systems,
zstdnow employs multiple threads by default, thanks to #4211 by @daniellerozenblitIn
v1.5.7, systems will see default threading scaled according to capability, but remains cautious to mitigate risks of overloading surprises on existing user base, with a maximum of 4 threads by default.For example, a typical "low-end" system with 4 cores and hyperthreading will employ 2 threads by default. A "higher-end" system with 8 cores and hyperthreading will employ 4 threads by default.
These modifications only impact default settings. As usual, the number of threads that
zstduses can also be set manually, either via command line or via environment variables.Additionally, a new
zstdcommand--maxis available, designed to maximize compression ratios by leveraging all parameters, at the expense of time and memory. It's slower and more resource hungry than--ultra -22, but achieves denser compressed archives if that's the goal. For instance, compressingenwik9(1,000,000,000 bytes) at--maxlevel on a i7-9700k platform yields:--ultra -22--maxBug Fixes
v1.5.7resolves a long standing and very rare compression issue in 32-bit mode, which can be triggered during long-lasting sessions (sameZSTD_CCtx*being continuously reused). This issue required several conditions to be met and a large amount of data to occur, but at the scale ofzstdusage, it has been observed and reported (#4292).In summary, if you use
libzstdin 32-bit mode, this update is highly recommended.Miscellaneous
The release also brings its lot of various improvements, impacting build scripts (cmake, meson, Apple Framework, Visual Studio), documentation and portability (QNX, FreeBSD, Hurd, icc/icx).
Notably, there are measurable performance gains for binaries produced with Visual Studio, as a cumulative result of several contributions.
Changelog:
--patch-fromat high compression levels (#4276)--patch-fromcompression ratios, notably at high levels (#4288)dfast, aka levels 3 and 4 (#4171)--maxcommand (#4290)ZSTD_compressSequencesAndLiterals()(#4217, #4232)ZSTD_getFrameHeader()now works on skippable frames (#4228)msbuildversion autodetection, support VS2022, by @ManuelBlancmesonbuild by @artem and @Victor-C-Zhang, and on Windows by @bgilbertNew Contributors
Full Changelog: facebook/zstd@v1.5.6...v1.5.7
v1.5.6: Zstandard v1.5.6 - Chrome EditionCompare Source
This release highlights the deployment of Google Chrome 123, introducing
zstd-encodingfor Web traffic, offered as a preferable option for compression of dynamic contents. With limited web server support forzstd-encodingdue to its novelty, we are launching an updated Zstandard version to facilitate broader adoption.Improved latency (time to first byte) for web pages
Using
zstdcompression for large documents over the Internet, data is segmented into smaller blocks of up to 128 KB, for incremental updates. This is crucial for applications like Chrome that process parts of documents as they arrive. However, on slow or congested networks, there can be some brief unresponsiveness in the middle of a block transmission, delaying update. To mitigate such scenarios,libzstdintroduces the new parameterZSTD_c_targetCBlockSize, enabling the division of blocks into even smaller segments to enhance initial byte delivery speed. Activating this feature incurs a cost, both runtime (equivalent to -2% speed at level 8) and a slight compression efficiency decrease (<0.1%), but offers some desirable latency reduction, notably beneficial in areas with more congested network infrastructure.Improved compression ratio at high levels
Highest compression levels (typically 18+) receive some compression ratio improvement. The improvement is really noticeable for 32-bit structures, like arrays of
intfor example. A real-world example would the.debug_str_offsetssection ofDWARFdebug info withinELFexecutables, mentioned in #2832, for which the compression effectiveness increases by +35%. It's not rare for many files or objects to contain sections of 32-bit structures, resulting in corresponding compression ratio improvements.Granular binary size selection
libzstdprovides build customization, including options to compile only the compression or decompression modules, minimizing binary size. Enhanced inv1.5.6(source), it now allows for even finer control by enabling selective inclusion or exclusion of specific components within these modules. This advancement aids applications needing precise binary size management.Miscellaneous Enhancements
This release includes various minor enhancements and bug fixes to enhance user experience. Key updates include an expanded list of recognized compressed file suffixes for the
--exclude-compressedflag, improving efficiency by skipping presumed incompressible content. Furthermore, compatibility has been broadened to include additional chipsets (sparc64,ARM64EC,risc-v) and operating systems (QNX,AIX,Solaris,HP-UX).Change Log
api: Promote
ZSTD_c_targetCBlockSizeto Stable API by @felixhandteapi: new experimental
ZSTD_d_maxBlockSizeparameter, to reduce streaming decompression memory, by @terrellnperf: improve performance of param
ZSTD_c_targetCBlockSize, by @Cyan4973perf: improved compression of arrays of integers at high compression, by @Cyan4973
lib: reduce binary size with selective built-time exclusion, by @felixhandte
lib: improved huffman speed on small data and linux kernel, by @terrelln
lib: accept dictionaries with partial literal tables, by @terrelln
lib: fix CCtx size estimation with external sequence producer, by @embg
lib: fix corner case decoder behaviors, by @Cyan4973 and @aimuz
lib: fix zdict prototype mismatch in static_only mode, by @ldv-alt
lib: fix several bugs in magicless-format decoding, by @embg
cli: add common compressed file types to
--exclude-compressedby @daniellerozenblit (requested by @dcog989)cli: fix mixing
-cand-ocommands with--rm, by @Cyan4973cli: fix erroneous exclusion of hidden files with
--output-dir-mirrorby @felixhandtecli: improved time accuracy on BSD, by @felixhandte
cli: better errors on argument parsing, by @KapJI
tests: better compatibility with older versions of
grep, by @Cyan4973tests: lorem ipsum generator as default content generator, by @Cyan4973
build: cmake improvements by @terrelln, @sighingnow, @gjasny, @JohanMabille, @Saverio976, @gruenich, @teo-tsirpanis
build: bazel support, by @jondo2010
build: fix cross-compiling for AArch64 with lld by @jcelerier
build: fix Apple platform compatibility, by @nidhijaju
build: fix Visual 2012 and lower compatibility, by @Cyan4973
build: improve win32 support, by @DimitriPapadopoulos
build: better C90 compliance for zlibWrapper, by @emaste
port: make: fat binaries on macos, by @mredig
port: ARM64EC compatibility for Windows, by @dunhor
port: QNX support by @klausholstjacobsen
port: MSYS2 and Cygwin makefile installation and test support, by @QBos07
port: risc-v support validation in CI, by @Cyan4973
port: sparc64 support validation in CI, by @Cyan4973
port: AIX compatibility, by @likema
port: HP-UX compatibility, by @likema
doc: Improved specification accuracy, by @elasota
bug: Fix and deprecate ZSTD_generateSequences (#3981), by @terrelln
Full change list (auto-generated)
https://github.com/facebook/zstd/pull/36343634
DESTDIRandBINDIRseparately by @paulmenzel in https://github.com/facebook/zstd/pull/3752do { } while (0)by @terrelln in https://github.com/facebook/zstd/pull/38315921623by @Cyan4973 in https://github.com/facebook/zstd/pull/39005131069by @Cyan4973 in https://github.com/facebook/zstd/pull/3902targetCBlockSizeby @Cyan4973 in https://github.com/facebook/zstd/pull/3917zstd::libzstdCMake target if only static or dynamic linkage is specified. by @teo-tsirpanis in https://github.com/facebook/zstd/pull/3811ZSTD_RowFindBestMatchcomment by @yoniko in https://github.com/facebook/zstd/pull/3947utimensat()on FreeBSD by @felixhandte in https://github.com/facebook/zstd/pull/3960--output-dir-mirrorby @felixhandte in https://github.com/facebook/zstd/pull/3963ZSTD_c_targetCBlockSizeParameter to Stable API by @felixhandte in https://github.com/facebook/zstd/pull/3964New Contributors
Configuration
📅 Schedule: Branch creation - "on the 1st through 7th day of the month" in timezone America/Los_Angeles, Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.
This PR was generated by Mend Renovate. View the repository job log.