updated README #876
Merged
updated README #876
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
danking
reviewed
Sep 19, 2024
README.md
Outdated
| Vortex is a toolkit for working with compressed Apache Arrow arrays in-memory, on-disk, and over-the-wire. | ||
|
|
||
| Vortex is designed to be to columnar file formats what Apache DataFusion is to query engines (or, analogously, | ||
| what LLVM + Clang are to compilers): a highly extensible & extremely performant *framework* for building a modern |
There was a problem hiding this comment.
I prefer fast to performant personally.
README.md
Outdated
|
|
||
| Vinyl is an aspiring successor to Apache Parquet, with dramatically faster random access reads (100-200x faster) | ||
| and scans (2-10x faster), while preserving approximately the same compression ratio and write throughput. It will also support very wide | ||
| tables (at least 10s of thousands of columns) and (in the future) on-device decompression on GPUs. |
There was a problem hiding this comment.
I’m hoping for millions; is that too speculative to include here?
There was a problem hiding this comment.
We already lazy deserialize schema so might be worth trying to benchmark the overhead of arbirtrary number of columns vs read of selected few. I think layouts get bigger and we have to read them all.
README.md
Outdated
| Optimized for random access reads and extremely fast scans; an aspiring successor to Apache Parquet. | ||
|
|
||
|
|
||
| [1]: Because like its predecessor, it's a type of [flooring](https://en.wikipedia.org/wiki/Parquet), but it is much better for |
robert3005
reviewed
Sep 19, 2024
README.md
Outdated
| Vortex is a toolkit for working with compressed Apache Arrow arrays in-memory, on-disk, and over-the-wire. | ||
|
|
||
| Vortex is designed to be to columnar file formats what Apache DataFusion is to query engines (or, analogously, | ||
| what LLVM + Clang are to compilers): a highly extensible & extremely performant *framework* for building a modern |
There was a problem hiding this comment.
Clang is the C++ compiler using LLVM. In this analogy it's just the LLVM
robert3005
approved these changes
Sep 23, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.