Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[wip] sparse array take golf #1163

Closed
wants to merge 1 commit into from
Closed

Conversation

danking
Copy link
Member

@danking danking commented Oct 29, 2024

No description provided.

@danking danking added the benchmark Run benchmarks on this branch label Oct 29, 2024
@github-actions github-actions bot removed the benchmark Run benchmarks on this branch label Oct 29, 2024
Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Vortex bytes_at

Benchmark suite Current: 8291090 Previous: 5636f85 Ratio
bytes_at/array_data 731.314136672123 ns (1.8464498522530448) 720.496247419828 ns (1.1007934728878581) 1.02
bytes_at/array_view 524.0486199215779 ns (1.2935292283790432) 530.0457101689611 ns (0.8522450779692008) 0.99

This comment was automatically generated by workflow using github-action-benchmark.

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Random Access

Benchmark suite Current: 8291090 Previous: 5636f85 Ratio
random-access/vortex-tokio-local-disk 1007832.0399768251 ns (7318.807201628864) 995206.0391717573 ns (6707.452124927251) 1.01
random-access/vortex-local-fs 1105025.9624478824 ns (8884.05411276198) 1112941.3555277025 ns (6694.403028444387) 0.99
random-access/parquet-tokio-local-disk 227972110.1333333 ns (3429472.6474999785) 229612739.66666666 ns (5429392.983333334) 0.99

This comment was automatically generated by workflow using github-action-benchmark.

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DataFusion

Benchmark suite Current: 8291090 Previous: 5636f85 Ratio
arrow/planning 770352.4537814201 ns (2330.8181985333795) 763836.4758681321 ns (1680.301121970173) 1.01
arrow/exec 1384122.9208033138 ns (6564.3421325832605) 1373398.9731945803 ns (6515.883443722385) 1.01
vortex-pushdown-compressed/planning 481518.3174955507 ns (1592.0475983389188) 483451.44097852125 ns (1449.6424518436834) 1.00
vortex-pushdown-compressed/exec 2600095.9910000004 ns (17149.666381249437) 2665648.684736842 ns (13132.727631578688) 0.98
vortex-pushdown-uncompressed/planning 482507.07666069904 ns (3080.8834048921417) 482127.41689707054 ns (2610.6961696482613) 1.00
vortex-pushdown-uncompressed/exec 2916998.0427777786 ns (8205.799270833144) 2566211.9730000007 ns (5604.806543750921) 1.14
vortex-nopushdown-compressed/planning 788335.5736834352 ns (2717.826278977969) 791262.6610104984 ns (1850.4956722183852) 1.00
vortex-nopushdown-compressed/exec 3312345.10875 ns (76590.1633671876) 2922020.1361111104 ns (30457.5752916669) 1.13
vortex-nopushdown-uncompressed/planning 779930.8372493386 ns (1534.8806002760539) 779657.3431898197 ns (1959.953749693639) 1.00
vortex-nopushdown-uncompressed/exec 4711996.149999997 ns (41738.04128408898) 4667871.163636361 ns (46788.92796590878) 1.01

This comment was automatically generated by workflow using github-action-benchmark.

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TPC-H

Benchmark suite Current: 8291090 Previous: 5636f85 Ratio
tpch_q1/vortex-in-memory-no-pushdown 583222582 ns (4121437.088749945) 576776766 ns (3718324.46875) 1.01
tpch_q1/vortex-in-memory-pushdown 467730901.5 ns (6856234.2193749845) 454041824.1 ns (1077692.5193750262) 1.03
tpch_q1/arrow 558871677.9 ns (2491920.727499962) 549607339.9 ns (2553559.399999976) 1.02
tpch_q1/parquet 700512918.7 ns (1954896.217500031) 709135483.5 ns (2314483.84375) 0.99
tpch_q1/vortex-file-compressed 515337444.2 ns (1900790.1124999821) 526073634.4 ns (8049624.800000012) 0.98
tpch_q1/vortex-file-uncompressed 544328892 ns (6173652.787500024) 538314026.1 ns (2039703.530000031) 1.01
tpch_q2/vortex-in-memory-no-pushdown 122656784.29650792 ns (884223.7083611265) 121120604.40170634 ns (1512738.5384156704) 1.01
tpch_q2/vortex-in-memory-pushdown 121774656.79865082 ns (394916.044822447) 118961692.89670637 ns (479563.2986512929) 1.02
tpch_q2/arrow 117296564.02599207 ns (751777.7918571383) 120855278.21158728 ns (565297.9031646699) 0.97
tpch_q2/parquet 149308715.55238098 ns (1141967.3615595251) 150784125.16289684 ns (1332485.3849166483) 0.99
tpch_q2/vortex-file-compressed 173360140.06551588 ns (1252825.4094662815) 172122137.75527778 ns (2410229.600781247) 1.01
tpch_q2/vortex-file-uncompressed 170584772.32559526 ns (1856042.8569642901) 174793382.0556746 ns (1602746.5173412561) 0.98
tpch_q3/vortex-in-memory-no-pushdown 160834483.61440474 ns (1087492.1295089126) 161497172.15428573 ns (1192845.440083325) 1.00
tpch_q3/vortex-in-memory-pushdown 179471775.80480158 ns (846882.5131696463) 180126691.03757936 ns (2350925.495448917) 1.00
tpch_q3/arrow 149828945.3660714 ns (1299847.46409674) 152645389.08936507 ns (1782058.1251865178) 0.98
tpch_q3/parquet 337949736.15 ns (1714919.0787500143) 341190179.15 ns (1971287.131249994) 0.99
tpch_q3/vortex-file-compressed 349288257.85 ns (2756526.4200000167) 291925456.6 ns (1904870.0099999905) 1.20
tpch_q3/vortex-file-uncompressed 245873986.95 ns (1325447.7206249982) 242766469.29999995 ns (2090913.9975000322) 1.01
tpch_q4/vortex-in-memory-no-pushdown 112204159.07634921 ns (794116.5880238116) 110755341.93884921 ns (1186064.265790671) 1.01
tpch_q4/vortex-in-memory-pushdown 132609727.4545635 ns (599952.8607132956) 131369111.62218253 ns (1384881.7269245982) 1.01
tpch_q4/arrow 188844457.23333332 ns (1874755.323333338) 187292208.4333333 ns (880440.0333333313) 1.01
tpch_q4/parquet 205048239.2 ns (413280.2204166651) 205780879.96666667 ns (917832.2166666538) 1.00
tpch_q4/vortex-file-compressed 312102498.55 ns (1096574.8493749797) 248976270.1 ns (1598828.0499999821) 1.25
tpch_q4/vortex-file-uncompressed 203414813.63333333 ns (727906.1741666645) 205028089.3 ns (1800348.3633333445) 0.99
tpch_q5/vortex-in-memory-no-pushdown 283634407.6 ns (1878687.4074999988) 289207852.3 ns (1831083.359375) 0.98
tpch_q5/vortex-in-memory-pushdown 298550741.8 ns (3432544.1587499976) 296686112.9 ns (2041088.2599999905) 1.01
tpch_q5/arrow 273614252.45 ns (3687554.1762500107) 269113715.55 ns (1721201.5) 1.02
tpch_q5/parquet 449534787.4 ns (3321734.574999988) 457269912.35 ns (1470666) 0.98
tpch_q5/vortex-file-compressed 367092335.6 ns (6669153.84375) 365100971.05 ns (4015416.525000006) 1.01
tpch_q5/vortex-file-uncompressed 356164304.65 ns (7667515.784999996) 340411518.85 ns (1439864.2668749988) 1.05
tpch_q6/vortex-in-memory-no-pushdown 36410102.54609789 ns (221630.52818452567) 35763461.88890211 ns (600199.9655477852) 1.02
tpch_q6/vortex-in-memory-pushdown 75844098.68819444 ns (102177.76710069925) 69762017.26160714 ns (134302.5719479248) 1.09
tpch_q6/arrow 27185905.674811512 ns (441140.51153273694) 26110596.14990079 ns (150563.85426612198) 1.04
tpch_q6/parquet 137552054.43801588 ns (419053.52872914076) 138079293.30376983 ns (847280.2482093275) 1.00
tpch_q6/vortex-file-compressed 73008147.2025992 ns (457755.25757168233) 71393553.00978175 ns (190363.72685019672) 1.02
tpch_q6/vortex-file-uncompressed 174400505.6367857 ns (1517488.0627455413) 166351082.40444443 ns (702424.3472360969) 1.05
tpch_q7/vortex-in-memory-no-pushdown 556117870 ns (6389196.723749995) 560478984.4 ns (5940847.25000006) 0.99
tpch_q7/vortex-in-memory-pushdown 584004340.8 ns (5465194.813750029) 598829294 ns (6258142.057500005) 0.98
tpch_q7/arrow 537495196.7 ns (3723721.399999976) 539882106 ns (5041368.60375002) 1.00
tpch_q7/parquet 679897893.8 ns (2284433.226249993) 683992093.7 ns (9059936.998750031) 0.99
tpch_q7/vortex-file-compressed 745495779.3 ns (7678057.972500026) 688911562.8 ns (3283493.9850000143) 1.08
tpch_q7/vortex-file-uncompressed 689781758.5 ns (3050645.012499988) 671978054 ns (5718932.793749988) 1.03
tpch_q8/vortex-in-memory-no-pushdown 226754215.5333333 ns (1895501.5779166818) 222440856.2 ns (1380318.8150000125) 1.02
tpch_q8/vortex-in-memory-pushdown 236457409.1333333 ns (2481198.599999979) 226806871.4666667 ns (1248196.122083336) 1.04
tpch_q8/arrow 213040288.6666667 ns (1309280.6520833224) 211053624.93333334 ns (1950827.449999988) 1.01
tpch_q8/parquet 492595338.1 ns (1879286.3937499821) 496373086.75 ns (3664885.668124974) 0.99
tpch_q8/vortex-file-compressed 313630927.55 ns (2050512.025000006) 313949752.4 ns (2764663.1268749833) 1.00
tpch_q8/vortex-file-uncompressed 303220914.55 ns (3269532.6037499905) 298757702 ns (2860656.362499982) 1.01
tpch_q9/vortex-in-memory-no-pushdown 438179244.6 ns (4074084.731249988) 432334016.6 ns (2653556.401250005) 1.01
tpch_q9/vortex-in-memory-pushdown 415727785.35 ns (6101931.400000006) 426497038.9 ns (2227775.6181249917) 0.97
tpch_q9/arrow 398653300.5 ns (4059189.1037499905) 403588551 ns (2626345.2387500107) 0.99
tpch_q9/parquet 710689667.5 ns (3274875.649999976) 718676727.2 ns (3352052.778749943) 0.99
tpch_q9/vortex-file-compressed 505969991.5 ns (5849181.5) 518180911.3 ns (6160943.838749975) 0.98
tpch_q9/vortex-file-uncompressed 485160658.6 ns (4646625.525000006) 489292559.5 ns (6092718.65625) 0.99
tpch_q10/vortex-in-memory-no-pushdown 291770065.05 ns (5044689.868750006) 280928025.4 ns (4938427.326874971) 1.04
tpch_q10/vortex-in-memory-pushdown 322882320.7 ns (1573548.800000012) 307877272.9 ns (2100010.600000024) 1.05
tpch_q10/arrow 275452203.95 ns (2619023.925000012) 265422862.5 ns (1994390.1974999905) 1.04
tpch_q10/parquet 503882135.8 ns (1635222.099999994) 501146757.3 ns (1583243) 1.01
tpch_q10/vortex-file-compressed 444601204.1 ns (3260419.3918750286) 412329714.6 ns (1493152.625) 1.08
tpch_q10/vortex-file-uncompressed 397795352.25 ns (2612098.6662500203) 392573727.5 ns (1349789.1293750107) 1.01
tpch_q11/vortex-in-memory-no-pushdown 179296196.5 ns (1249540.6116666496) 178673151.9290873 ns (1081868.3603561372) 1.00
tpch_q11/vortex-in-memory-pushdown 183602270.47988096 ns (1444128.8054836243) 180961588.3333333 ns (1617904.9004166722) 1.01
tpch_q11/arrow 178375050 ns (2077512.371666655) 176290423.7150397 ns (1663925.979171142) 1.01
tpch_q11/parquet 193101926.09999996 ns (2704531.4025000036) 189923196.26666665 ns (1299698.1570833325) 1.02
tpch_q11/vortex-file-compressed 266393741.55 ns (3046520.498750001) 270440301.15 ns (2039187.371875003) 0.99
tpch_q11/vortex-file-uncompressed 271806809.6 ns (1624077.6568749547) 262619557.55 ns (1463805.140625) 1.03
tpch_q12/vortex-in-memory-no-pushdown 233959888.0333333 ns (779931.2583333105) 230217596.6333333 ns (721423.1033333242) 1.02
tpch_q12/vortex-in-memory-pushdown 257365532.75 ns (676802.1875000149) 248187362.7 ns (688598.9050000012) 1.04
tpch_q12/arrow 187810104.93333334 ns (789284.6379166842) 187881974.13333336 ns (545755.982916683) 1.00
tpch_q12/parquet 340960641.25 ns (2275996.876249999) 339610900.55 ns (936049.2931250036) 1.00
tpch_q12/vortex-file-compressed 407957419.7 ns (708482.6331250072) 397825849.8 ns (2012932.921875) 1.03
tpch_q12/vortex-file-uncompressed 392972891.05 ns (1572533.4750000238) 387996696.75 ns (1349010.125) 1.01
tpch_q13/vortex-in-memory-no-pushdown 174398878.19630954 ns (4658590.331678569) 184129295.46666664 ns (1415497.000000015) 0.95
tpch_q13/vortex-in-memory-pushdown 189971399.46666667 ns (3046424.1666666716) 184812051.86666664 ns (1360214.1670833528) 1.03
tpch_q13/arrow 172493398.33218256 ns (2446495.580203846) 192349939.96666664 ns (1848171.4225000143) 0.90
tpch_q13/parquet 331562820.15 ns (4637440.091874987) 341723275.6 ns (5988858.72875002) 0.97
tpch_q13/vortex-file-compressed 209942178.53333336 ns (2577557.490833327) 200226129.23333335 ns (3234616.261250004) 1.05
tpch_q13/vortex-file-uncompressed 208154369.6 ns (2254844.3979166746) 204869485.36666667 ns (766142.0349999964) 1.02
tpch_q14/vortex-in-memory-no-pushdown 45478788.12055555 ns (963990.6199027784) 44510646.2255291 ns (175435.24638888985) 1.02
tpch_q14/vortex-in-memory-pushdown 78162763.65180555 ns (219938.22145139426) 73095095.52793652 ns (610358.9941626936) 1.07
tpch_q14/arrow 35168504.3851455 ns (287179.2313812822) 36962877.06898148 ns (1201558.3972222246) 0.95
tpch_q14/parquet 226802038.6 ns (1020547.4333333522) 227257151.0666667 ns (772438.3995833248) 1.00
tpch_q14/vortex-file-compressed 124580638.45158729 ns (653525.242738083) 118976021.2970635 ns (992903.7644295767) 1.05
tpch_q14/vortex-file-uncompressed 137824979.43079364 ns (573732.808595255) 135580530.32460317 ns (877380.3837549537) 1.02
tpch_q15/vortex-in-memory-no-pushdown 72169907.08599207 ns (1275477.8752728254) 72459822.07113096 ns (435016.8275572881) 1.00
tpch_q15/vortex-in-memory-pushdown 110428790.3409524 ns (667850.9241190553) 105805283.62396826 ns (597209.844404757) 1.04
tpch_q15/arrow 60025666.97067461 ns (549357.8140863068) 59607225.30992063 ns (368366.824243553) 1.01
tpch_q15/parquet 303066744.95 ns (1921054.732499987) 303891068.85 ns (1182899.525000006) 1.00
tpch_q15/vortex-file-compressed 233113834.2666667 ns (1180940.576666668) 219414293 ns (676430.200000003) 1.06
tpch_q15/vortex-file-uncompressed 256580470.25 ns (1326328.9712499976) 249478835.95 ns (935836.690624997) 1.03
tpch_q16/vortex-in-memory-no-pushdown 110607948.64908728 ns (1269905.7075744048) 110834743.53027777 ns (1242840.5001388788) 1.00
tpch_q16/vortex-in-memory-pushdown 120804576.83944443 ns (1214919.0552916601) 121815582.92607144 ns (1198923.131309539) 0.99
tpch_q16/arrow 106612166.6429365 ns (963941.551061511) 105778766.15821429 ns (464661.22792857885) 1.01
tpch_q16/parquet 119322062.97099206 ns (1420752.8160128891) 117867167.93400793 ns (1191134.821845241) 1.01
tpch_q16/vortex-file-compressed 134870748.11210316 ns (510158.03234125674) 133272139.26825397 ns (382128.0395238027) 1.01
tpch_q16/vortex-file-uncompressed 134141546.76571429 ns (920501.4050000012) 136310660.57424602 ns (1443746.0874206424) 0.98
tpch_q17/vortex-in-memory-no-pushdown 542519119.9 ns (16411146.38625002) 545464700 ns (7964284.870000005) 0.99
tpch_q17/vortex-in-memory-pushdown 636982924.5 ns (5765566.097499967) 641718874 ns (11872963.897499979) 0.99
tpch_q17/arrow 537636475.3 ns (5253141.04125002) 514454748 ns (4482152.650000006) 1.05
tpch_q17/parquet 639058416.3 ns (5056639.099999964) 662550012.7 ns (11364559.607500017) 0.96
tpch_q17/vortex-file-compressed 634293938.4 ns (8459521.038749993) 633995049 ns (3216880.675000012) 1.00
tpch_q17/vortex-file-uncompressed 626176557.4 ns (6853305.69749999) 615618096.3 ns (2802774.643750012) 1.02
tpch_q18/vortex-in-memory-no-pushdown 1057846633.6 ns (13121622.284999967) 1073213329.4 ns (7945393.50000006) 0.99
tpch_q18/vortex-in-memory-pushdown 1081167870.1 ns (29247764.09374994) 1140092518.1 ns (11099154.95749998) 0.95
tpch_q18/arrow 1103781380.8 ns (13640428.71875) 1093035967.2 ns (16368066.723749995) 1.01
tpch_q18/parquet 1204456123.1 ns (7770803.899999976) 1223777686 ns (9803162.299999952) 0.98
tpch_q18/vortex-file-compressed 1172213058.3 ns (17749995.67874992) 1165393808 ns (14644021.591250062) 1.01
tpch_q18/vortex-file-uncompressed 1106659764.2 ns (9283531.944999933) 1107502360.4 ns (11900343.649999976) 1.00
tpch_q19/vortex-in-memory-no-pushdown 182843291.9 ns (475380.75) 183223062.96666667 ns (655586.6645833254) 1.00
tpch_q19/vortex-in-memory-pushdown 261987271.65 ns (951881.2362499833) 250657159.8333333 ns (579262.9566666782) 1.05
tpch_q19/arrow 166530226.83730158 ns (906416.0011547506) 165713756.08869046 ns (388431.5725208521) 1.00
tpch_q19/parquet 455744470.25 ns (1010289.1956250072) 456595107.65 ns (1322088.0618749857) 1.00
tpch_q19/vortex-file-compressed 358821287.2 ns (2670827.3849999905) 394603307.95 ns (1248630.8893750012) 0.91
tpch_q19/vortex-file-uncompressed 359591100.85 ns (2921059.6412500143) 354229189.05 ns (2130517.9131250083) 1.02
tpch_q20/vortex-in-memory-no-pushdown 250773042.8333333 ns (6379467.151666686) 262031756.6 ns (6378110.206874996) 0.96
tpch_q20/vortex-in-memory-pushdown 274010789.7 ns (1636061.6906249821) 279575175.7 ns (1874100.7987499833) 0.98
tpch_q20/arrow 233665979.0666667 ns (2401565.9370833486) 239461636.63333336 ns (3099680.351666674) 0.98
tpch_q20/parquet 357240699.2 ns (3189531.7756249905) 367592791.8 ns (4995399.125) 0.97
tpch_q20/vortex-file-compressed 364717477.65 ns (6637447.948125005) 362345102.1 ns (2794695.6662499905) 1.01
tpch_q20/vortex-file-uncompressed 371013935.35 ns (2323163.3849999905) 369557418 ns (2225105.9693750143) 1.00
tpch_q21/vortex-in-memory-no-pushdown 897228628.7 ns (5593186.714999974) 916069501.9 ns (8035357.165000021) 0.98
tpch_q21/vortex-in-memory-pushdown 920522300.8 ns (4631744.204999983) 914132277.6 ns (9441826.944999993) 1.01
tpch_q21/arrow 883812654.6 ns (9792075.849999964) 876117620.5 ns (11208191.412500024) 1.01
tpch_q21/parquet 1005426941.7 ns (13652051.226250052) 1002644522.8 ns (11856263.600000024) 1.00
tpch_q21/vortex-file-compressed 1303726465.6 ns (9950060.6049999) 1213663163.9 ns (9727712.59499991) 1.07
tpch_q21/vortex-file-uncompressed 1088834480.9 ns (11314672.151250005) 1094638657.1 ns (9375014.84375) 0.99
tpch_q22/vortex-in-memory-no-pushdown 76965309.9696627 ns (387695.1810602769) 76361548.48295635 ns (252917.45179340988) 1.01
tpch_q22/vortex-in-memory-pushdown 75907087.56990078 ns (460357.04316963255) 76312438.91928571 ns (495238.02847322077) 0.99
tpch_q22/arrow 74487736.26335317 ns (220208.564725928) 75359925.60329364 ns (274550.27212302387) 0.99
tpch_q22/parquet 95394049.29119048 ns (1357483.0547261834) 94805988.47313493 ns (293522.458370544) 1.01
tpch_q22/vortex-file-compressed 120736865.79059525 ns (511243.8942381069) 120465852.82067461 ns (285851.7604861036) 1.00
tpch_q22/vortex-file-uncompressed 118558231.53912696 ns (905838.2744613141) 120306895.75980158 ns (514701.57652777433) 0.99

This comment was automatically generated by workflow using github-action-benchmark.

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Vortex Compression

Benchmark suite Current: 8291090 Previous: 5636f85 Ratio
compress time/taxi 1187378772.8 ns (1132781.8537499905) 1236401094.6 ns (3029662.758749962) 0.96
compress time/taxi throughput 470808924 bytes 470808924 bytes 1
parquet_rs-zstd compress time/taxi 1819510557.8 ns (8468425.357500076) 1778411207.1 ns (4130951.383750081) 1.02
parquet_rs-zstd compress time/taxi throughput 470808924 bytes 470808924 bytes 1
decompress time/taxi 419064877.3 ns (1855340.440625012) 449261255.45 ns (3973687.540624976) 0.93
decompress time/taxi throughput 470808924 bytes 470808924 bytes 1
parquet_rs-zstd decompress time/taxi 307453407.5 ns (721745.2506250143) 311385598.15 ns (1043086.0512500107) 0.99
parquet_rs-zstd decompress time/taxi throughput 470808924 bytes 470808924 bytes 1
vortex:parquet-zstd size/taxi 0.9370724976997296 ratio 0.9451684491343139 ratio 0.99
vortex:raw size/taxi 0.11138135520982606 ratio 0.11234527916467446 ratio 0.99
vortex size/taxi 52439336 bytes 52893160 bytes 0.99
compress time/AirlineSentiment 676602.1841096415 ns (1015.7545805408736) 669811.0422487746 ns (804.5701811428298) 1.01
compress time/AirlineSentiment throughput 2020 bytes 2020 bytes 1
parquet_rs-zstd compress time/AirlineSentiment 55932.80369001505 ns (184.50126725341033) 56507.95333693635 ns (109.00224058218373) 0.99
parquet_rs-zstd compress time/AirlineSentiment throughput 2020 bytes 2020 bytes 1
decompress time/AirlineSentiment 39567.25250315873 ns (72.11470566912976) 39737.18837127999 ns (67.9021309055097) 1.00
decompress time/AirlineSentiment throughput 2020 bytes 2020 bytes 1
parquet_rs-zstd decompress time/AirlineSentiment 31966.85171397939 ns (116.41174261452397) 32370.369280702645 ns (124.73807489636783) 0.99
parquet_rs-zstd decompress time/AirlineSentiment throughput 2020 bytes 2020 bytes 1
vortex:parquet-zstd size/AirlineSentiment 6.196483971044468 ratio 6.196483971044468 ratio 1
vortex:raw size/AirlineSentiment 2.9663366336633663 ratio 2.9663366336633663 ratio 1
vortex size/AirlineSentiment 5992 bytes 5992 bytes 1
compress time/Arade 2180983744.2 ns (7622264.950000048) 2299788388.7 ns (28120555.84624982) 0.95
compress time/Arade throughput 787023760 bytes 787023760 bytes 1
parquet_rs-zstd compress time/Arade 3107856773.8 ns (7323405.899999857) 3187907213.2 ns (28827734.26249981) 0.97
parquet_rs-zstd compress time/Arade throughput 787023760 bytes 787023760 bytes 1
decompress time/Arade 771616127.7 ns (2543458) 803594803.8 ns (27112118.637499988) 0.96
decompress time/Arade throughput 787023760 bytes 787023760 bytes 1
parquet_rs-zstd decompress time/Arade 644272927.8 ns (2375164.649999976) 679046725.6 ns (13080117.290000021) 0.95
parquet_rs-zstd decompress time/Arade throughput 787023760 bytes 787023760 bytes 1
vortex:parquet-zstd size/Arade 0.47890649124129325 ratio 0.4789069103731621 ratio 1.00
vortex:raw size/Arade 0.18583271234403392 ratio 0.18583271234403392 ratio 1
vortex size/Arade 146254760 bytes 146254760 bytes 1
compress time/Bimbo 10401789316.1 ns (7294201.17250061) 11194660241 ns (74964142.07625008) 0.93
compress time/Bimbo throughput 7121333608 bytes 7121333608 bytes 1
parquet_rs-zstd compress time/Bimbo 21598160778.1 ns (23034997.984998703) 20983458446.5 ns (302764710.80000114) 1.03
parquet_rs-zstd compress time/Bimbo throughput 7121333608 bytes 7121333608 bytes 1
decompress time/Bimbo 4820232981.4 ns (71950121.3199997) 4910427622 ns (32914328.60375023) 0.98
decompress time/Bimbo throughput 7121333608 bytes 7121333608 bytes 1
parquet_rs-zstd decompress time/Bimbo 2620335138.4 ns (12937741.121249914) 2644695718.5 ns (10614766.44875002) 0.99
parquet_rs-zstd decompress time/Bimbo throughput 7121333608 bytes 7121333608 bytes 1
vortex:parquet-zstd size/Bimbo 1.2342855358573495 ratio 1.1815029394361685 ratio 1.04
vortex:raw size/Bimbo 0.06727519792947187 ratio 0.06439826263507918 ratio 1.04
vortex size/Bimbo 479089128 bytes 458601512 bytes 1.04
compress time/CMSprovider 11980285239 ns (11682956.386249542) 11870790433.5 ns (72411845.59999943) 1.01
compress time/CMSprovider throughput 5149123964 bytes 5149123964 bytes 1
parquet_rs-zstd compress time/CMSprovider 20274434754.5 ns (36963329.42625046) 20000916103.8 ns (66648796.849998474) 1.01
parquet_rs-zstd compress time/CMSprovider throughput 5149123964 bytes 5149123964 bytes 1
decompress time/CMSprovider 7199909282.3 ns (11617312.42624998) 7525145859.2 ns (34238933.819999695) 0.96
decompress time/CMSprovider throughput 5149123964 bytes 5149123964 bytes 1
parquet_rs-zstd decompress time/CMSprovider 4890598696.3 ns (41916946.35999966) 4914856443.2 ns (39417281.49000025) 1.00
parquet_rs-zstd decompress time/CMSprovider throughput 5149123964 bytes 5149123964 bytes 1
vortex:parquet-zstd size/CMSprovider 1.201647685298353 ratio 1.2016519269342556 ratio 1.00
vortex:raw size/CMSprovider 0.17958134984998003 ratio 0.17958143685507122 ratio 1.00
vortex size/CMSprovider 924686632 bytes 924687080 bytes 1.00
compress time/Euro2016 2631492351.8 ns (3135038.5687499046) 2640425739.9 ns (6252775.75) 1.00
compress time/Euro2016 throughput 393253221 bytes 393253221 bytes 1
parquet_rs-zstd compress time/Euro2016 1592372837 ns (2805533.2624999285) 1628456520.7 ns (5819239.563749909) 0.98
parquet_rs-zstd compress time/Euro2016 throughput 393253221 bytes 393253221 bytes 1
decompress time/Euro2016 299089806.7 ns (1156383.599999994) 304345849.55 ns (2266060.6587500274) 0.98
decompress time/Euro2016 throughput 393253221 bytes 393253221 bytes 1
parquet_rs-zstd decompress time/Euro2016 484269584.25 ns (1242018.294999987) 496979190.8 ns (2755683.3306249976) 0.97
parquet_rs-zstd decompress time/Euro2016 throughput 393253221 bytes 393253221 bytes 1
vortex:parquet-zstd size/Euro2016 1.4383522286595696 ratio 1.4383522286595696 ratio 1
vortex:raw size/Euro2016 0.4348484255644533 ratio 0.4348484255644533 ratio 1
vortex size/Euro2016 171005544 bytes 171005544 bytes 1
compress time/Food 1084963807.2 ns (3850257.25) 1085592823.7 ns (4039805.411249995) 1.00
compress time/Food throughput 332718229 bytes 332718229 bytes 1
parquet_rs-zstd compress time/Food 1140969891 ns (2937502.649999976) 1114696833.2 ns (4420492.882500052) 1.02
parquet_rs-zstd compress time/Food throughput 332718229 bytes 332718229 bytes 1
decompress time/Food 203058562.50000003 ns (799653.25) 202844954.23333335 ns (1074651.9533333182) 1.00
decompress time/Food throughput 332718229 bytes 332718229 bytes 1
parquet_rs-zstd decompress time/Food 214595071.03333333 ns (770394.6774999797) 218380613.46666664 ns (1138057.493749991) 0.98
parquet_rs-zstd decompress time/Food throughput 332718229 bytes 332718229 bytes 1
vortex:parquet-zstd size/Food 1.238933256800101 ratio 1.2389367897825485 ratio 1.00
vortex:raw size/Food 0.1349086526906225 ratio 0.1349086526906225 ratio 1
vortex size/Food 44886568 bytes 44886568 bytes 1
compress time/HashTags 2602235103.4 ns (2770637.4375) 2589608003.3 ns (10941123.115000248) 1.00
compress time/HashTags throughput 804495592 bytes 804495592 bytes 1
parquet_rs-zstd compress time/HashTags 2539069092.1 ns (3714098.9500000477) 2504294950.2 ns (5331431.299999952) 1.01
parquet_rs-zstd compress time/HashTags throughput 804495592 bytes 804495592 bytes 1
decompress time/HashTags 560231260.3 ns (1911566.4887500405) 556757095.9 ns (4279235.596249998) 1.01
decompress time/HashTags throughput 804495592 bytes 804495592 bytes 1
parquet_rs-zstd decompress time/HashTags 776268990.9 ns (2861046.3187499642) 808291763.6 ns (8834606.172500014) 0.96
parquet_rs-zstd decompress time/HashTags throughput 804495592 bytes 804495592 bytes 1
vortex:parquet-zstd size/HashTags 1.7048629043277994 ratio 1.6582904353477639 ratio 1.03
vortex:raw size/HashTags 0.2838956611710061 ratio 0.2761403619971606 ratio 1.03
vortex size/HashTags 228392808 bytes 222153704 bytes 1.03
compress time/TPC-H l_comment chunked without fsst 3896681568.9 ns (10722078.900000095) 4067864898.5 ns (78055659.44000006) 0.96
compress time/TPC-H l_comment chunked without fsst throughput 249197090 bytes 249197090 bytes 1
parquet_rs-zstd compress time/TPC-H l_comment chunked without fsst 905777266.5 ns (1721995.7049999833) 921526090 ns (3279127.9599999785) 0.98
parquet_rs-zstd compress time/TPC-H l_comment chunked without fsst throughput 249197090 bytes 249197090 bytes 1
decompress time/TPC-H l_comment chunked without fsst 112174807.40492065 ns (1337156.0219166651) 118990993.8170238 ns (652056.620133914) 0.94
decompress time/TPC-H l_comment chunked without fsst throughput 249197090 bytes 249197090 bytes 1
parquet_rs-zstd decompress time/TPC-H l_comment chunked without fsst 251623514.9 ns (702640.575000003) 252112013.55 ns (815309.053124994) 1.00
parquet_rs-zstd decompress time/TPC-H l_comment chunked without fsst throughput 249197090 bytes 249197090 bytes 1
vortex:parquet-zstd size/TPC-H l_comment chunked without fsst 4.607634271473487 ratio 4.607903277997382 ratio 1.00
vortex:raw size/TPC-H l_comment chunked without fsst 1.0527089541856207 ratio 1.0527140906821986 ratio 1.00
vortex size/TPC-H l_comment chunked without fsst 262332008 bytes 262333288 bytes 1.00
compress time/TPC-H l_comment chunked 915590445.6 ns (1769360.332499981) 930497622.8 ns (5774079.307499945) 0.98
compress time/TPC-H l_comment chunked throughput 249197090 bytes 249197090 bytes 1
parquet_rs-zstd compress time/TPC-H l_comment chunked 911086422.2 ns (1172052.6175000072) 922210107.8 ns (4227995.511250019) 0.99
parquet_rs-zstd compress time/TPC-H l_comment chunked throughput 249197090 bytes 249197090 bytes 1
decompress time/TPC-H l_comment chunked 122473964.23353174 ns (485723.78482142836) 126075318.9602778 ns (1595210.3755555525) 0.97
decompress time/TPC-H l_comment chunked throughput 249197090 bytes 249197090 bytes 1
parquet_rs-zstd decompress time/TPC-H l_comment chunked 251169330.8 ns (355345.97937500477) 251854087.1 ns (710769.1250000149) 1.00
parquet_rs-zstd decompress time/TPC-H l_comment chunked throughput 249197090 bytes 249197090 bytes 1
vortex:parquet-zstd size/TPC-H l_comment chunked 1.347976246547594 ratio 1.3480449951118159 ratio 1.00
vortex:raw size/TPC-H l_comment chunked 0.30797293820726396 ratio 0.3079721677327773 ratio 1.00
vortex size/TPC-H l_comment chunked 76745960 bytes 76745768 bytes 1.00
compress time/TPC-H l_comment canonical 914484645.55 ns (1024409.0818750262) 914679795.95 ns (2200726.809374988) 1.00
compress time/TPC-H l_comment canonical throughput 249197106 bytes 249197106 bytes 1
parquet_rs-zstd compress time/TPC-H l_comment canonical 912493564.75 ns (1586575.7487499714) 926804426.75 ns (3575661.278124988) 0.98
parquet_rs-zstd compress time/TPC-H l_comment canonical throughput 249197106 bytes 249197106 bytes 1
decompress time/TPC-H l_comment canonical 122172131.93354496 ns (508737.905941464) 123706878.63112435 ns (569467.7860152125) 0.99
decompress time/TPC-H l_comment canonical throughput 249197106 bytes 249197106 bytes 1
parquet_rs-zstd decompress time/TPC-H l_comment canonical 248546287.10430557 ns (490854.95581768453) 251446034.9734326 ns (1112960.2260776162) 0.99
parquet_rs-zstd decompress time/TPC-H l_comment canonical throughput 249197106 bytes 249197106 bytes 1
vortex:parquet-zstd size/TPC-H l_comment canonical 1.347975323182899 ratio 1.3480380810228385 ratio 1.00
vortex:raw size/TPC-H l_comment canonical 0.30797291843349095 ratio 0.30797214795905375 ratio 1.00
vortex size/TPC-H l_comment canonical 76745960 bytes 76745768 bytes 1.00

This comment was automatically generated by workflow using github-action-benchmark.

@danking
Copy link
Member Author

danking commented Oct 29, 2024

not clearly an improvement.

@danking danking closed this Oct 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant