feat: track compressed size & compare to parquet(zstd)? & canonical #880

danking · 2024-09-19T19:29:52Z

We now track these six values:

Compression time (s).
Compression throughput (bytes/s).
Compressed size (bytes).
Compressed size as fraction of a Vortex Canonical array.
Compressed Layout size as fraction of Parquet without block compression.
Compressed Layout size as fraction of Parquet with Zstd.

It's a bit janky: I just unconditionally compute these values for several datasets. I couldn't figure out how to ask criterion which benchmark regex is currently in use so, for example, cargo bench taxi will still run all the size benchmarks for every other dataset.

I also had to do some janky jq parsing to convert from Criterion's JSON output to the style expected by the benchmark-action GitHub action that we use.

Nevertheless, now, for each commit to develop, we should get all six numbers for the Taxi, Airline Sentiment, Arade, Bimbo, CMSprovider, Euro2016, Food, HashTags, and TPC-H l_comment datasets. They'll be displayed under Vortex
Compression at the benchmarks site.

I might need to delete some old data form the gh-pages-bench branch since I changed some benchmarknames, but after a few commits, those plots should become useful measures of our compression performance in space and time.

github-actions

Vortex bytes_at

Benchmark suite	Current: `64904a2`	Previous: `a96ff2c`	Ratio
`bytes_at/array_data`	`608.0382304849801` ns (`0.19519179892182592`)	`613` ns/iter (`± 8`)	`0.99`
`bytes_at/array_data #2`	`1036.2175205854583` ns (`2.691644106922922`)	`1043` ns/iter (`± 4`)	`0.99`

This comment was automatically generated by workflow using github-action-benchmark.

github-actions

Vortex random_access

Benchmark suite	Current: `64904a2`	Previous: `a96ff2c`	Ratio
`vortex/tokio local disk`	`1298462.6652337688` ns (`11119.263724996708`)	`1308917` ns/iter (`± 29650`)	`0.99`
`vortex/localfs`	`1427657.5326669263` ns (`5988.312219878775`)	`1457592` ns/iter (`± 32225`)	`0.98`
`parquet/tokio local disk`	`202354071.56666666` ns (`6067911.224166647`)	`178158170` ns/iter (`± 2466099`)	`1.14`

This comment was automatically generated by workflow using github-action-benchmark.

github-actions

Vortex DataFusion

Benchmark suite	Current: `64904a2`	Previous: `a96ff2c`	Ratio
`arrow/planning`	`827662.4524491443` ns (`5103.659116700001`)	`813880` ns/iter (`± 4517`)	`1.02`
`arrow/exec`	`1772940.2969369683` ns (`1176.9243653769372`)	`1774262` ns/iter (`± 18680`)	`1.00`
`vortex-pushdown-compressed/planning`	`519177.2428005217` ns (`298.5027583242045`)	`516095` ns/iter (`± 1831`)	`1.01`
`vortex-pushdown-compressed/exec`	`3048774.367647059` ns (`2989.4164779412095`)	`3209669` ns/iter (`± 141970`)	`0.95`
`vortex-pushdown-uncompressed/planning`	`524839.169970599` ns (`3274.487221657735`)	`514579` ns/iter (`± 1971`)	`1.02`
`vortex-pushdown-uncompressed/exec`	`2939029.1035294123` ns (`1245.3141838239972`)	`3336867` ns/iter (`± 9294`)	`0.88`
`vortex-nopushdown-compressed/planning`	`721176.4743120342` ns (`677.4496210361831`)	`710291` ns/iter (`± 5322`)	`1.02`
`vortex-nopushdown-compressed/exec`	`8552565.399999999` ns (`126292.94252083264`)	`14988542` ns/iter (`± 251952`)	`0.57`
`vortex-nopushdown-uncompressed/planning`	`719793.2418384501` ns (`631.538322867942`)	`715546` ns/iter (`± 3807`)	`1.01`
`vortex-nopushdown-uncompressed/exec`	`2013752.8291999998` ns (`1724.1680299995933`)	`2001661` ns/iter (`± 82038`)	`1.01`

This comment was automatically generated by workflow using github-action-benchmark.

github-actions

Vortex Compression

Benchmark suite	Current: `64904a2`	Previous: `a96ff2c`	Ratio
`Yellow Taxi Trip Data Compression Time/taxi compression`	`2474817918.8` ns (`4587512.808749914`)
`Yellow Taxi Trip Data Compression Time/taxi compression throughput`	`470808924` bytes
`Yellow Taxi Trip Data Vortex-to-ParquetZstd Ratio/taxi`	`0.9502797751619019` ratio
`Yellow Taxi Trip Data Vortex-to-ParquetUncompressed Ratio/taxi`	`0.6100036654712709` ratio
`Yellow Taxi Trip Data Compression Ratio/taxi`	`0.10728657726122455` ratio
`Yellow Taxi Trip Data Compression Size/taxi`	`50511478` ratio
`Public BI Compression Time/AirlineSentiment compression`	`414561.17468779447` ns (`197.2479264921858`)
`Public BI Compression Time/AirlineSentiment compression throughput`	`2020` bytes
`Public BI Vortex-to-ParquetZstd Ratio/AirlineSentiment`	`6.400830737279335` ratio
`Public BI Vortex-to-ParquetUncompressed Ratio/AirlineSentiment`	`4.353107344632768` ratio
`Public BI Compression Ratio/AirlineSentiment`	`0.6207920792079208` ratio
`Public BI Compression Size/AirlineSentiment`	`1254` ratio
`Public BI Compression Time/Arade compression`	`3158825482.2` ns (`3128658.201249838`)
`Public BI Compression Time/Arade compression throughput`	`787023760` bytes
`Public BI Vortex-to-ParquetZstd Ratio/Arade`	`0.49135399238694333` ratio
`Public BI Vortex-to-ParquetUncompressed Ratio/Arade`	`0.4385731803671133` ratio
`Public BI Compression Ratio/Arade`	`0.18567306150960422` ratio
`Public BI Compression Size/Arade`	`146129111` ratio
`Public BI Compression Time/Bimbo compression`	`22797650985.2` ns (`14045274.374998093`)
`Public BI Compression Time/Bimbo compression throughput`	`7121333608` bytes
`Public BI Vortex-to-ParquetZstd Ratio/Bimbo`	`1.3017830104422772` ratio
`Public BI Vortex-to-ParquetUncompressed Ratio/Bimbo`	`0.8826521636227171` ratio
`Public BI Compression Ratio/Bimbo`	`0.0646312220906138` ratio
`Public BI Compression Size/Bimbo`	`460260494` ratio
`Public BI Compression Time/CMSprovider compression`	`13119410162.4` ns (`14299149.900000572`)
`Public BI Compression Time/CMSprovider compression throughput`	`5149123964` bytes
`Public BI Vortex-to-ParquetZstd Ratio/CMSprovider`	`1.2011142354620599` ratio
`Public BI Vortex-to-ParquetUncompressed Ratio/CMSprovider`	`0.7755509255966841` ratio
`Public BI Compression Ratio/CMSprovider`	`0.17581527874049063` ratio
`Public BI Compression Size/CMSprovider`	`905294665` ratio
`Public BI Compression Time/Euro2016 compression`	`2226019552.2` ns (`21705941.84249997`)
`Public BI Compression Time/Euro2016 compression throughput`	`393253221` bytes
`Public BI Vortex-to-ParquetZstd Ratio/Euro2016`	`1.4663903701389807` ratio
`Public BI Vortex-to-ParquetUncompressed Ratio/Euro2016`	`0.622157628400298` ratio
`Public BI Compression Ratio/Euro2016`	`0.4350781274338246` ratio
`Public BI Compression Size/Euro2016`	`171095875` ratio
`Public BI Compression Time/Food compression`	`1089726169.3` ns (`3004860.2287499905`)
`Public BI Compression Time/Food compression throughput`	`332718229` bytes
`Public BI Vortex-to-ParquetZstd Ratio/Food`	`1.2271002122949906` ratio
`Public BI Vortex-to-ParquetUncompressed Ratio/Food`	`0.6938323589538802` ratio
`Public BI Compression Ratio/Food`	`0.12973288878620473` ratio
`Public BI Compression Size/Food`	`43164497` ratio
`Public BI Compression Time/HashTags compression`	`2927684783.8` ns (`11641658.546249866`)
`Public BI Compression Time/HashTags compression throughput`	`804495592` bytes
`Public BI Vortex-to-ParquetZstd Ratio/HashTags`	`1.6557846810450216` ratio
`Public BI Vortex-to-ParquetUncompressed Ratio/HashTags`	`0.4707428540382966` ratio
`Public BI Compression Ratio/HashTags`	`0.2656995490411587` ratio
`Public BI Compression Size/HashTags`	`213754116` ratio
`TPC-H l_comment Compression Time/chunked-without-fsst compression`	`192593209.5506548` ns (`159774.29374924302`)
`TPC-H l_comment Compression Time/chunked-without-fsst compression throughput`	`183010921` bytes
`TPC-H l_comment Vortex-to-ParquetZstd Ratio/chunked-without-fsst`	`3.215587952596748` ratio
`TPC-H l_comment Vortex-to-ParquetUncompressed Ratio/chunked-without-fsst`	`0.9983803631502735` ratio
`TPC-H l_comment Compression Ratio/chunked-without-fsst`	`0.999965750677797` ratio
`TPC-H l_comment Compression Size/chunked-without-fsst`	`183004653` ratio
`TPC-H l_comment Compression Time/chunked-with-fsst compression`	`1135326748.05` ns (`1407376.4031250477`)
`TPC-H l_comment Compression Time/chunked-with-fsst compression throughput`	`183010921` bytes
`TPC-H l_comment Vortex-to-ParquetZstd Ratio/chunked-with-fsst`	`1.504836821707564` ratio
`TPC-H l_comment Vortex-to-ParquetUncompressed Ratio/chunked-with-fsst`	`0.467223896433944` ratio
`TPC-H l_comment Compression Ratio/chunked-with-fsst`	`0.44243102847397836` ratio
`TPC-H l_comment Compression Size/chunked-with-fsst`	`80969710` ratio
`TPC-H l_comment Compression Time/canonical-with-fsst compression`	`1132259446.9` ns (`852477.966250062`)
`TPC-H l_comment Compression Time/canonical-with-fsst compression throughput`	`183010937` bytes
`TPC-H l_comment Vortex-to-ParquetZstd Ratio/canonical-with-fsst`	`1.5026112658946886` ratio
`TPC-H l_comment Vortex-to-ParquetUncompressed Ratio/canonical-with-fsst`	`0.46653220871703877` ratio
`TPC-H l_comment Compression Ratio/canonical-with-fsst`	`0.4418543248046427` ratio
`TPC-H l_comment Compression Size/canonical-with-fsst`	`80864174` ratio

This comment was automatically generated by workflow using github-action-benchmark.

We now track these six values: 1. Compression time (s). 2. Compression throughput (bytes/s). 3. Compressed size (bytes). 4. Compressed size as fraction of a Vortex Canonical array. 5. Compressed Layout size as fraction of Parquet without block compression. 6. Compressed Layout size as fraction of Parquet with Zstd. It's a bit janky: I just unconditionally compute these values for several datasets. I couldn't figure out how to ask criterion which benchmark regex is currently in use so, for example, `cargo bench taxi` will still run all the size benchmarks for every other dataset. I also had to do some janky jq parsing to convert from Criterion's JSON output to the style expected by the benchmark-action GitHub action that we use. Nevertheless, now, for each commit to `develop`, we should get all six numbers for the Taxi, Airline Sentiment, Arade, Bimbo, CMSprovider, Euro2016, Food, HashTags, and TPC-H l_comment datasets. They'll be displayed under [Vortex Compression](https://spiraldb.github.io/vortex/dev/bench/#Vortex_Compression) at the benchmarks site. I might need to delete some old data form the gh-pages-bench branch since I changed some benchmark names, but after a few commits, those plots should become useful measures of our compression performance in space and time.

github-actions

Vortex benchmarks

Benchmark suite	Current: `64904a2`	Previous: `a96ff2c`	Ratio
`tpch_q1/vortex-in-memory-no-pushdown`	`464304077.85` ns (`1533560.2999999821`)	`456752113` ns/iter (`± 3867547`)	`1.02`
`tpch_q1/vortex-in-memory-pushdown`	`533009655.9` ns (`5378226.188749999`)	`532735558` ns/iter (`± 1607247`)	`1.00`
`tpch_q1/arrow`	`443722504.6` ns (`1115956.6900000274`)	`443097274` ns/iter (`± 626790`)	`1.00`
`tpch_q1/parquet`	`660632551.2` ns (`19461738.899999976`)	`653884935` ns/iter (`± 2663691`)	`1.01`
`tpch_q1/vortex-file-compressed`	`621017522.8` ns (`793126.0587499738`)	`625869224` ns/iter (`± 2721859`)	`0.99`
`tpch_q1/vortex-file-uncompressed`	`624409927.3` ns (`2489514.1650000215`)	`631514808` ns/iter (`± 9245882`)	`0.99`
`tpch_q2/vortex-in-memory-no-pushdown`	`142459030.68797618` ns (`1162015.545416668`)	`146858416` ns/iter (`± 2531945`)	`0.97`
`tpch_q2/vortex-in-memory-pushdown`	`147246475.67940477` ns (`5193940.229345232`)	`143317300` ns/iter (`± 2886213`)	`1.03`
`tpch_q2/arrow`	`122143122.7840873` ns (`613386.064764373`)	`122568135` ns/iter (`± 397231`)	`1.00`
`tpch_q2/parquet`	`155868631.64071432` ns (`1300941.583431542`)	`159787707` ns/iter (`± 5185870`)	`0.98`
`tpch_q2/vortex-file-compressed`	`156560477.174246` ns (`1469067.645724222`)	`156466031` ns/iter (`± 1277797`)	`1.00`
`tpch_q2/vortex-file-uncompressed`	`166912482.05730158` ns (`2307440.765142843`)	`162139847` ns/iter (`± 3160753`)	`1.03`
`tpch_q3/vortex-in-memory-no-pushdown`	`149115138.0123016` ns (`2159670.5297618955`)	`153229410` ns/iter (`± 1562047`)	`0.97`
`tpch_q3/vortex-in-memory-pushdown`	`180973752.20900792` ns (`389101.39488494396`)	`186029987` ns/iter (`± 1669213`)	`0.97`
`tpch_q3/arrow`	`144574885.88309523` ns (`370286.6344047636`)	`147328590` ns/iter (`± 1417356`)	`0.98`
`tpch_q3/parquet`	`329729923.15` ns (`555769.0787500143`)	`336975915` ns/iter (`± 2025628`)	`0.98`
`tpch_q3/vortex-file-compressed`	`287385699.85` ns (`757747.8312499821`)	`309258408` ns/iter (`± 2757350`)	`0.93`
`tpch_q3/vortex-file-uncompressed`	`365948211.4` ns (`682112.4850000143`)	`375568417` ns/iter (`± 4807910`)	`0.97`
`tpch_q4/vortex-in-memory-no-pushdown`	`104982279.28626983` ns (`496512.5886696279`)	`106376311` ns/iter (`± 304925`)	`0.99`
`tpch_q4/vortex-in-memory-pushdown`	`136728891.9136111` ns (`265040.5339965224`)	`141994138` ns/iter (`± 1248613`)	`0.96`
`tpch_q4/arrow`	`101308723.68607143` ns (`139306.25577379763`)	`101065390` ns/iter (`± 370932`)	`1.00`
`tpch_q4/parquet`	`212498857.93333334` ns (`342192.60916668177`)	`214248841` ns/iter (`± 2228222`)	`0.99`
`tpch_q4/vortex-file-compressed`	`256142642.7` ns (`528195.4406249821`)	`262684851` ns/iter (`± 1417453`)	`0.98`
`tpch_q4/vortex-file-uncompressed`	`311832024.4` ns (`3928716.9337500334`)	`322371836` ns/iter (`± 4220226`)	`0.97`
`tpch_q5/vortex-in-memory-no-pushdown`	`279659733.25` ns (`1001442.896874994`)	`296691840` ns/iter (`± 6168057`)	`0.94`
`tpch_q5/vortex-in-memory-pushdown`	`295151300.65` ns (`1038721.4293749928`)	`321813589` ns/iter (`± 5635245`)	`0.92`
`tpch_q5/arrow`	`279926912.9` ns (`707757.1999999881`)	`289107585` ns/iter (`± 2924141`)	`0.97`
`tpch_q5/parquet`	`427239058.65` ns (`404816.5618750155`)	`449018047` ns/iter (`± 2489275`)	`0.95`
`tpch_q5/vortex-file-compressed`	`322193807.3` ns (`1654653.376875013`)	`341880037` ns/iter (`± 8522037`)	`0.94`
`tpch_q5/vortex-file-uncompressed`	`337214643.4` ns (`1440883.1493749917`)	`356647316` ns/iter (`± 5958948`)	`0.95`
`tpch_q6/vortex-in-memory-no-pushdown`	`36775646.094126984` ns (`19476.342416666448`)	`40138218` ns/iter (`± 630500`)	`0.92`
`tpch_q6/vortex-in-memory-pushdown`	`84810398.53295635` ns (`275279.34296923876`)	`92149267` ns/iter (`± 303889`)	`0.92`
`tpch_q6/arrow`	`35653509.05183862` ns (`447290.3801309541`)	`36334469` ns/iter (`± 211591`)	`0.98`
`tpch_q6/parquet`	`150169413.98996028` ns (`2069685.8542474955`)	`151921473` ns/iter (`± 1264234`)	`0.99`
`tpch_q6/vortex-file-compressed`	`81733112.0160119` ns (`184630.87145832926`)	`78859071` ns/iter (`± 1115685`)	`1.04`
`tpch_q6/vortex-file-uncompressed`	`167585822.39670637` ns (`380986.36732143164`)	`167141882` ns/iter (`± 1751525`)	`1.00`
`tpch_q7/vortex-in-memory-no-pushdown`	`545986895.3` ns (`1253510.7362499833`)	`562119306` ns/iter (`± 3476977`)	`0.97`
`tpch_q7/vortex-in-memory-pushdown`	`598615119.6` ns (`8397887.399999976`)	`611059188` ns/iter (`± 6446587`)	`0.98`
`tpch_q7/arrow`	`545806379.9` ns (`2369302.7562499642`)	`553024994` ns/iter (`± 2909226`)	`0.99`
`tpch_q7/parquet`	`694387265.3` ns (`7450212.17749995`)	`710209548` ns/iter (`± 5017550`)	`0.98`
`tpch_q7/vortex-file-compressed`	`669958979.8` ns (`2182435.5`)	`672453257` ns/iter (`± 5566775`)	`1.00`
`tpch_q7/vortex-file-uncompressed`	`737341433.7` ns (`5431513.763749957`)	`744071550` ns/iter (`± 5659596`)	`0.99`
`tpch_q8/vortex-in-memory-no-pushdown`	`208854667.96666667` ns (`516034.6899999976`)	`216237880` ns/iter (`± 504152`)	`0.97`
`tpch_q8/vortex-in-memory-pushdown`	`225333089.06666666` ns (`371392.93291670084`)	`230296027` ns/iter (`± 963193`)	`0.98`
`tpch_q8/arrow`	`209062804.00000003` ns (`305187.71666666865`)	`215487494` ns/iter (`± 822806`)	`0.97`
`tpch_q8/parquet`	`473784336.15` ns (`3235365.916874975`)	`482558982` ns/iter (`± 1927968`)	`0.98`
`tpch_q8/vortex-file-compressed`	`261757067.9` ns (`4436969.673125014`)	`272225347` ns/iter (`± 3218905`)	`0.96`
`tpch_q8/vortex-file-uncompressed`	`300842027.75` ns (`1131956.550000012`)	`307092746` ns/iter (`± 4647118`)	`0.98`
`tpch_q9/vortex-in-memory-no-pushdown`	`396433296.3` ns (`2034854.732499987`)	`405778945` ns/iter (`± 3408198`)	`0.98`
`tpch_q9/vortex-in-memory-pushdown`	`387813627.35` ns (`371879.91624999046`)	`409784837` ns/iter (`± 8637477`)	`0.95`
`tpch_q9/arrow`	`391656983.3` ns (`5264873.551249981`)	`400998246` ns/iter (`± 7870465`)	`0.98`
`tpch_q9/parquet`	`682841437.5` ns (`2939525.367500007`)	`687723525` ns/iter (`± 2769586`)	`0.99`
`tpch_q9/vortex-file-compressed`	`444211767.45` ns (`2201231.9731250107`)	`449724976` ns/iter (`± 6436082`)	`0.99`
`tpch_q9/vortex-file-uncompressed`	`478970853.95` ns (`5516594.525000006`)	`482884495` ns/iter (`± 6141352`)	`0.99`
`tpch_q10/vortex-in-memory-no-pushdown`	`225716306.7` ns (`6366994.144166663`)	`224740155` ns/iter (`± 1207852`)	`1.00`
`tpch_q10/vortex-in-memory-pushdown`	`260860365.7` ns (`4925735.380624995`)	`265222009` ns/iter (`± 4544285`)	`0.98`
`tpch_q10/arrow`	`221600023.16666666` ns (`2282484.8899999857`)	`219076345` ns/iter (`± 7024828`)	`1.01`
`tpch_q10/parquet`	`469986581.9` ns (`840855.0293750167`)	`481426698` ns/iter (`± 4462077`)	`0.98`
`tpch_q10/vortex-file-compressed`	`465704735.25` ns (`2336274.956249982`)	`474019593` ns/iter (`± 4032038`)	`0.98`
`tpch_q10/vortex-file-uncompressed`	`399096131.4` ns (`818911.8899999857`)	`408859777` ns/iter (`± 3938984`)	`0.98`
`tpch_q11/vortex-in-memory-no-pushdown`	`215081483.9` ns (`1018470.2733333111`)	`219129162` ns/iter (`± 1832776`)	`0.98`
`tpch_q11/vortex-in-memory-pushdown`	`217861051.46666664` ns (`4088130.2970833033`)	`220793553` ns/iter (`± 918518`)	`0.99`
`tpch_q11/arrow`	`174634397.18031746` ns (`2131647.687620029`)	`175455464` ns/iter (`± 1125682`)	`1.00`
`tpch_q11/parquet`	`179670336.71075398` ns (`424755.4409375042`)	`185576140` ns/iter (`± 2442270`)	`0.97`
`tpch_q11/vortex-file-compressed`	`224431755.86666664` ns (`1886778.0008333623`)	`229509379` ns/iter (`± 1644525`)	`0.98`
`tpch_q11/vortex-file-uncompressed`	`229292836.2` ns (`1640592.134166658`)	`232738732` ns/iter (`± 1873655`)	`0.99`
`tpch_q12/vortex-in-memory-no-pushdown`	`177642417.6570635` ns (`116093.88803078234`)	`179897756` ns/iter (`± 1962967`)	`0.99`
`tpch_q12/vortex-in-memory-pushdown`	`254785644.8` ns (`289834.97375001013`)	`268815014` ns/iter (`± 1804665`)	`0.95`
`tpch_q12/arrow`	`168644879.84781748` ns (`418898.60029760003`)	`170395809` ns/iter (`± 844024`)	`0.99`
`tpch_q12/parquet`	`352627738.25` ns (`170870.7018749714`)	`365760882` ns/iter (`± 5113024`)	`0.96`
`tpch_q12/vortex-file-compressed`	`614294864.4` ns (`7913363.078750014`)	`611089999` ns/iter (`± 3516355`)	`1.01`
`tpch_q12/vortex-file-uncompressed`	`358325235.95` ns (`2365733.775000006`)	`363970552` ns/iter (`± 2594091`)	`0.98`
`tpch_q13/vortex-in-memory-no-pushdown`	`156144794.07916665` ns (`615453.301458329`)	`171007772` ns/iter (`± 4051193`)	`0.91`
`tpch_q13/vortex-in-memory-pushdown`	`156774092.7992857` ns (`913640.3976190388`)	`169154998` ns/iter (`± 6178477`)	`0.93`
`tpch_q13/arrow`	`157783842.0720635` ns (`3918559.6385198534`)	`179394695` ns/iter (`± 11628817`)	`0.88`
`tpch_q13/parquet`	`299416647.65` ns (`5516799.510625005`)	`343672528` ns/iter (`± 12687514`)	`0.87`
`tpch_q13/vortex-file-compressed`	`198547266.23333332` ns (`1326709.9075000137`)	`221913395` ns/iter (`± 3831446`)	`0.89`
`tpch_q13/vortex-file-uncompressed`	`202978341.36666664` ns (`2833577.7666666657`)	`212642135` ns/iter (`± 1772424`)	`0.95`
`tpch_q14/vortex-in-memory-no-pushdown`	`37004936.319986776` ns (`200027.29246990755`)	`37691250` ns/iter (`± 577567`)	`0.98`
`tpch_q14/vortex-in-memory-pushdown`	`79191076.4785516` ns (`967225.2857051119`)	`90997812` ns/iter (`± 1433438`)	`0.87`
`tpch_q14/arrow`	`38392207.505198404` ns (`1580528.3829379939`)	`39535216` ns/iter (`± 535028`)	`0.97`
`tpch_q14/parquet`	`219139935.63333336` ns (`1768794.9208333194`)	`227708765` ns/iter (`± 1731670`)	`0.96`
`tpch_q14/vortex-file-compressed`	`87327637.15424605` ns (`397347.65172816813`)	`90305017` ns/iter (`± 631935`)	`0.97`
`tpch_q14/vortex-file-uncompressed`	`142977717.5492857` ns (`417165.495714277`)	`144470755` ns/iter (`± 708935`)	`0.99`
`tpch_q15/vortex-in-memory-no-pushdown`	`62704249.051765874` ns (`440493.7101354115`)	`71237883` ns/iter (`± 1431115`)	`0.88`
`tpch_q15/vortex-in-memory-pushdown`	`107315237.3897619` ns (`109767.07269345224`)	`124403185` ns/iter (`± 854376`)	`0.86`
`tpch_q15/arrow`	`63173515.51857142` ns (`130043.41368303075`)	`66195092` ns/iter (`± 1472663`)	`0.95`
`tpch_q15/parquet`	`286461936.65` ns (`153364.6262499988`)	`295437640` ns/iter (`± 1150294`)	`0.97`
`tpch_q15/vortex-file-compressed`	`152947559.96718255` ns (`883461.6438819468`)	`157382540` ns/iter (`± 411484`)	`0.97`
`tpch_q15/vortex-file-uncompressed`	`265955967.95` ns (`4291999.850000009`)	`275891348` ns/iter (`± 6001720`)	`0.96`
`tpch_q16/vortex-in-memory-no-pushdown`	`117989455.33039685` ns (`135413.09100990742`)	`118867963` ns/iter (`± 629106`)	`0.99`
`tpch_q16/vortex-in-memory-pushdown`	`123431369.27507937` ns (`470388.2658293471`)	`124703683` ns/iter (`± 1081895`)	`0.99`
`tpch_q16/arrow`	`105423126.90357143` ns (`720029.0482202321`)	`107392480` ns/iter (`± 705642`)	`0.98`
`tpch_q16/parquet`	`120799820.03285715` ns (`1525781.411666654`)	`123485091` ns/iter (`± 3669376`)	`0.98`
`tpch_q16/vortex-file-compressed`	`136384045.53698412` ns (`338577.32264880836`)	`138265217` ns/iter (`± 832715`)	`0.99`
`tpch_q16/vortex-file-uncompressed`	`138515182.4557143` ns (`912874.0090535581`)	`137767991` ns/iter (`± 578019`)	`1.01`
`tpch_q17/vortex-in-memory-no-pushdown`	`672455884.3` ns (`12530350.91624999`)	`649086157` ns/iter (`± 16184725`)	`1.04`
`tpch_q17/vortex-in-memory-pushdown`	`697313734.7` ns (`4168258`)	`654515157` ns/iter (`± 14489515`)	`1.07`
`tpch_q17/arrow`	`575081203.6` ns (`20533376.753749967`)	`567239351` ns/iter (`± 11560937`)	`1.01`
`tpch_q17/parquet`	`580241261.7` ns (`3004441.0212500095`)	`595915976` ns/iter (`± 6246602`)	`0.97`
`tpch_q17/vortex-file-compressed`	`615235622.5` ns (`3011760.279999912`)	`612595783` ns/iter (`± 2547793`)	`1.00`
`tpch_q17/vortex-file-uncompressed`	`674247544` ns (`4147862.6450000405`)	`667861291` ns/iter (`± 8580372`)	`1.01`
`tpch_q18/vortex-in-memory-no-pushdown`	`1004634512` ns (`7720466.849999964`)	`1034223912` ns/iter (`± 23067942`)	`0.97`
`tpch_q18/vortex-in-memory-pushdown`	`994246936.3` ns (`10559742.503749967`)	`994376340` ns/iter (`± 5989420`)	`1.00`
`tpch_q18/arrow`	`1004813963.2` ns (`11608545.568749964`)	`1004004887` ns/iter (`± 4588695`)	`1.00`
`tpch_q18/parquet`	`1184250124.5` ns (`11291440.860000014`)	`1186490542` ns/iter (`± 18651939`)	`1.00`
`tpch_q18/vortex-file-compressed`	`1078406485.1` ns (`12838050.699999928`)	`1065012633` ns/iter (`± 14258649`)	`1.01`
`tpch_q18/vortex-file-uncompressed`	`1130648562.8` ns (`11869234.07375002`)	`1135720332` ns/iter (`± 32401940`)	`1.00`
`tpch_q19/vortex-in-memory-no-pushdown`	`166777510.5570238` ns (`1200974.8047023714`)	`165874289` ns/iter (`± 732945`)	`1.01`
`tpch_q19/vortex-in-memory-pushdown`	`257057079.65` ns (`588456.9068749994`)	`260523501` ns/iter (`± 1589056`)	`0.99`
`tpch_q19/arrow`	`153318864.10369048` ns (`282869.1049389988`)	`153540134` ns/iter (`± 536959`)	`1.00`
`tpch_q19/parquet`	`478838708.95` ns (`588495.7287499905`)	`477195361` ns/iter (`± 3004902`)	`1.00`
`tpch_q19/vortex-file-compressed`	`771806227.5` ns (`5484281.650000036`)	`757301083` ns/iter (`± 6091638`)	`1.02`
`tpch_q19/vortex-file-uncompressed`	`364354112.65` ns (`1578555.4806250334`)	`374268649` ns/iter (`± 1709655`)	`0.97`
`tpch_q20/vortex-in-memory-no-pushdown`	`250806156.25` ns (`1518240.724999994`)	`267235874` ns/iter (`± 6059222`)	`0.94`
`tpch_q20/vortex-in-memory-pushdown`	`272585024.25` ns (`546909.599999994`)	`299117478` ns/iter (`± 6408411`)	`0.91`
`tpch_q20/arrow`	`236673056.2333333` ns (`1285820.670416668`)	`256806267` ns/iter (`± 8213627`)	`0.92`
`tpch_q20/parquet`	`377605482.15` ns (`7201988.625`)	`377456624` ns/iter (`± 5175006`)	`1.00`
`tpch_q20/vortex-file-compressed`	`322460870.5` ns (`3395283.550625026`)	`327554400` ns/iter (`± 7045439`)	`0.98`
`tpch_q20/vortex-file-uncompressed`	`402837567.6` ns (`3890327.069999993`)	`416259584` ns/iter (`± 5453969`)	`0.97`
`tpch_q21/vortex-in-memory-no-pushdown`	`818311449.3` ns (`6314055.379999995`)	`839899381` ns/iter (`± 9947739`)	`0.97`
`tpch_q21/vortex-in-memory-pushdown`	`884564635.4` ns (`5190742.611249924`)	`904150556` ns/iter (`± 17125645`)	`0.98`
`tpch_q21/arrow`	`827220509.6` ns (`10521542.986249983`)	`834172632` ns/iter (`± 6626356`)	`0.99`
`tpch_q21/parquet`	`954563781.2` ns (`1878705.1499999762`)	`987756274` ns/iter (`± 14734132`)	`0.97`
`tpch_q21/vortex-file-compressed`	`1168647396.9` ns (`2101206.2999999523`)	`1173609856` ns/iter (`± 5160528`)	`1.00`
`tpch_q21/vortex-file-uncompressed`	`1319313857.2` ns (`14216709.648749948`)	`1328981880` ns/iter (`± 9120789`)	`0.99`
`tpch_q22/vortex-in-memory-no-pushdown`	`93444799.94087301` ns (`634413.6074444428`)	`97935522` ns/iter (`± 453887`)	`0.95`
`tpch_q22/vortex-in-memory-pushdown`	`96680416.32436508` ns (`373293.2604900822`)	`97710962` ns/iter (`± 829935`)	`0.99`
`tpch_q22/arrow`	`65895319.629285716` ns (`297365.3176458366`)	`69526644` ns/iter (`± 272226`)	`0.95`
`tpch_q22/parquet`	`94113476.82996032` ns (`265220.52212747186`)	`96659104` ns/iter (`± 1019394`)	`0.97`
`tpch_q22/vortex-file-compressed`	`100671637.59269843` ns (`330471.35942261666`)	`103294285` ns/iter (`± 932885`)	`0.97`
`tpch_q22/vortex-file-uncompressed`	`109431716.5527381` ns (`214143.68898808956`)	`111618839` ns/iter (`± 1236391`)	`0.98`

This comment was automatically generated by workflow using github-action-benchmark.

danking added the benchmark Run benchmarks on this branch label Sep 19, 2024