feat: use hashbrown::hashmap everywhere (modestly faster decompress) #1160
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
I saw a modest (~5%) reduction in decompress time on a string-heavy dataset in which SparseArray take occupies ~10% of total time.
AdamGS
reviewed
Oct 29, 2024
|
You have stumbled upon something I had complained about before #1050 |
robert3005
reviewed
Oct 29, 2024
AdamGS
reviewed
Oct 29, 2024
| @@ -0,0 +1 @@ | |||
| pub mod hash_map; | |||
There was a problem hiding this comment.
aliases feels a bit weird, but I don't have a better idea and it does make sense to have a module to contain the type aliases.
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
Vortex bytes_at
| Benchmark suite | Current: c73c36d | Previous: d884d9f | Ratio |
|---|---|---|---|
bytes_at/array_data |
729.6063958426961 ns (2.223190886287682) |
732.2359535172372 ns (1.0561040854057637) |
1.00 |
bytes_at/array_view |
524.634320678985 ns (0.9293849780604546) |
522.6535023687583 ns (2.7973229528466845) |
1.00 |
This comment was automatically generated by workflow using github-action-benchmark.
AdamGS
approved these changes
Oct 29, 2024
There was a problem hiding this comment.
DataFusion
| Benchmark suite | Current: c73c36d | Previous: d884d9f | Ratio |
|---|---|---|---|
arrow/planning |
765006.7657071784 ns (3233.9540993381524) |
781365.0053680907 ns (2373.744954989059) |
0.98 |
arrow/exec |
1365989.0146346989 ns (5744.469187291106) |
1353230.6847115327 ns (3518.7806924759643) |
1.01 |
vortex-pushdown-compressed/planning |
480700.4065757988 ns (2066.5046185236424) |
489169.92450935557 ns (1319.8334024686774) |
0.98 |
vortex-pushdown-compressed/exec |
2586195.4619999994 ns (11860.0614249988) |
2589140.966000001 ns (9767.356000000378) |
1.00 |
vortex-pushdown-uncompressed/planning |
478716.07856790064 ns (1300.370169368689) |
490178.3531981086 ns (950.6232835749979) |
0.98 |
vortex-pushdown-uncompressed/exec |
2910380.089444444 ns (3561.1752222224604) |
2912878.927222222 ns (2529.9935000000987) |
1.00 |
vortex-nopushdown-compressed/planning |
789410.538473084 ns (2899.0688644893235) |
797489.6971976915 ns (1319.491780355922) |
0.99 |
vortex-nopushdown-compressed/exec |
3305846.761875 ns (29478.721140624955) |
3147787.345 ns (22989.452062499942) |
1.05 |
vortex-nopushdown-uncompressed/planning |
779061.5799868611 ns (1295.288770509942) |
795337.5191684244 ns (2851.1112864858587) |
0.98 |
vortex-nopushdown-uncompressed/exec |
4686358.697272729 ns (23094.642897726037) |
4526307.259090909 ns (7238.793500000611) |
1.04 |
This comment was automatically generated by workflow using github-action-benchmark.
There was a problem hiding this comment.
Random Access
| Benchmark suite | Current: c73c36d | Previous: d884d9f | Ratio |
|---|---|---|---|
random-access/vortex-tokio-local-disk |
968298.2402401173 ns (4921.310236137651) |
1008554.957505605 ns (5723.423690343858) |
0.96 |
random-access/vortex-local-fs |
1077612.2534337488 ns (4379.724454871612) |
1123010.3675028072 ns (5866.917831303203) |
0.96 |
random-access/parquet-tokio-local-disk |
228281859.63333336 ns (3555858.864166692) |
228036596.7333333 ns (3931433.62833333) |
1.00 |
This comment was automatically generated by workflow using github-action-benchmark.
|
no pushdown becomes weirdly slower? 1.38x and 1.28x |
There was a problem hiding this comment.
TPC-H
| Benchmark suite | Current: c73c36d | Previous: d884d9f | Ratio |
|---|---|---|---|
tpch_q1/vortex-in-memory-no-pushdown |
573350796 ns (2228749.75) |
558564063.8 ns (3545931) |
1.03 |
tpch_q1/vortex-in-memory-pushdown |
455225734 ns (1698653.8993749917) |
448235561.6 ns (1755886.4987500012) |
1.02 |
tpch_q1/arrow |
550826617 ns (3379406.25) |
552512084.9 ns (4793511.908749998) |
1.00 |
tpch_q1/parquet |
691675310.2 ns (2041635.8499999642) |
684866980.6 ns (3869860.605000019) |
1.01 |
tpch_q1/vortex-file-compressed |
519159150.4 ns (2169561.71874997) |
505282794.85 ns (2693140.0412499905) |
1.03 |
tpch_q1/vortex-file-uncompressed |
527415376.4 ns (2772586.3325000107) |
526624468.9 ns (5679470.006249994) |
1.00 |
tpch_q2/vortex-in-memory-no-pushdown |
121089925.07583332 ns (448892.52583333105) |
117804539.86194444 ns (631736.4481944442) |
1.03 |
tpch_q2/vortex-in-memory-pushdown |
117946789.85996032 ns (673352.2658333331) |
116749759.89111109 ns (341612.9177777767) |
1.01 |
tpch_q2/arrow |
117590425.29519841 ns (503000.5645238012) |
118971397.22416668 ns (641978.839062497) |
0.99 |
tpch_q2/parquet |
148474400.3035714 ns (773383.080053553) |
146458325.72075397 ns (753462.1352375895) |
1.01 |
tpch_q2/vortex-file-compressed |
169403062.501627 ns (1572801.2679677457) |
163916311.92309523 ns (4458349.4163214415) |
1.03 |
tpch_q2/vortex-file-uncompressed |
173011056.59694442 ns (1928115.9292951524) |
169352064.82234126 ns (1363369.0011681616) |
1.02 |
tpch_q3/vortex-in-memory-no-pushdown |
160460384.6755159 ns (613816.0702961236) |
156544036.2773016 ns (1305407.8551666886) |
1.03 |
tpch_q3/vortex-in-memory-pushdown |
177336997.59361112 ns (1297355.4963611215) |
180355393.5570635 ns (1559006.5351240188) |
0.98 |
tpch_q3/arrow |
149235118.045 ns (935956.1100625098) |
147413113.75396824 ns (1655608.8119156659) |
1.01 |
tpch_q3/parquet |
340896921.65 ns (1902948.8393749893) |
332425979.4 ns (1960303.071875006) |
1.03 |
tpch_q3/vortex-file-compressed |
293206277.2 ns (1025835.1606250107) |
289986098.75 ns (1193350.4687499702) |
1.01 |
tpch_q3/vortex-file-uncompressed |
245833816.9 ns (885466.8991666585) |
238915518.13333336 ns (552723.8137500137) |
1.03 |
tpch_q4/vortex-in-memory-no-pushdown |
111185315.71857142 ns (564521.959023796) |
109975214.21158728 ns (1282448.4029047564) |
1.01 |
tpch_q4/vortex-in-memory-pushdown |
128567055.69746032 ns (480738.62328173965) |
128869274.23357141 ns (645338.1214196533) |
1.00 |
tpch_q4/arrow |
182927663.2666667 ns (655666.4120833427) |
177504619.36666664 ns (514998.7441666573) |
1.03 |
tpch_q4/parquet |
205907505.23333335 ns (840745.3612499982) |
203731358.56666666 ns (758459.3333333284) |
1.01 |
tpch_q4/vortex-file-compressed |
249120896.06666666 ns (821163.509166643) |
247234356.66666666 ns (910489.5666666627) |
1.01 |
tpch_q4/vortex-file-uncompressed |
203925002.6 ns (1029856.2999999821) |
199001290.56666666 ns (1135144.9529166669) |
1.02 |
tpch_q5/vortex-in-memory-no-pushdown |
289085431.5 ns (2022876.2881249785) |
274158289.4 ns (3993771.2037499845) |
1.05 |
tpch_q5/vortex-in-memory-pushdown |
295823214.5 ns (1268253.001875013) |
289628736.1 ns (5048047.69312501) |
1.02 |
tpch_q5/arrow |
278402013.5 ns (1993315.5118749738) |
273459685.35 ns (2862193.699999988) |
1.02 |
tpch_q5/parquet |
452850784.6 ns (2641687.320625007) |
436438403.1 ns (4633727.629999995) |
1.04 |
tpch_q5/vortex-file-compressed |
365923922.75 ns (4161458.471249968) |
364005021.5 ns (5019916.53687501) |
1.01 |
tpch_q5/vortex-file-uncompressed |
350843616.15 ns (2285823.801249981) |
337427234.55 ns (3019015.239375025) |
1.04 |
tpch_q6/vortex-in-memory-no-pushdown |
35454954.246613756 ns (228806.24093121663) |
34527867.92838624 ns (481778.37615343556) |
1.03 |
tpch_q6/vortex-in-memory-pushdown |
68256552.76369047 ns (230877.14197172225) |
69344154.32069443 ns (226543.53943403065) |
0.98 |
tpch_q6/arrow |
25566472.822271824 ns (266344.3175626211) |
26123065.564216267 ns (187293.37580977194) |
0.98 |
tpch_q6/parquet |
138069784.48809522 ns (350091.5839583576) |
137178315.97265872 ns (349936.3251284808) |
1.01 |
tpch_q6/vortex-file-compressed |
71524117.51448414 ns (212555.78109077364) |
72895281.80755952 ns (493377.9763281122) |
0.98 |
tpch_q6/vortex-file-uncompressed |
168819573.5542857 ns (767734.3192857057) |
163968901.9956746 ns (409931.28347469866) |
1.03 |
tpch_q7/vortex-in-memory-no-pushdown |
555099531.7 ns (5536850.350000024) |
525449852.6 ns (3851838.882499993) |
1.06 |
tpch_q7/vortex-in-memory-pushdown |
586868462.4 ns (4538182.045000017) |
564329974.9 ns (7245849.647499979) |
1.04 |
tpch_q7/arrow |
541716572.9 ns (2334979.69750005) |
534840248.9 ns (5545517.638750017) |
1.01 |
tpch_q7/parquet |
677491812 ns (3785415.1812500358) |
660516396.7 ns (6521969.445000052) |
1.03 |
tpch_q7/vortex-file-compressed |
687399512.6 ns (3178592.3412500024) |
675861651.6 ns (3225491.8600000143) |
1.02 |
tpch_q7/vortex-file-uncompressed |
672997981.4 ns (3594680.817499995) |
648966157.6 ns (3508225.9187500477) |
1.04 |
tpch_q8/vortex-in-memory-no-pushdown |
221485701.0333333 ns (1253546.3833333254) |
218136800.5 ns (1746469.3333333582) |
1.02 |
tpch_q8/vortex-in-memory-pushdown |
226532436.83333334 ns (1764167.806249991) |
221736321.9666667 ns (727223.1808333397) |
1.02 |
tpch_q8/arrow |
208027136.86666664 ns (1372562.6933333278) |
208984738.23333332 ns (1055382.9233333468) |
1.00 |
tpch_q8/parquet |
488941424.55 ns (1958248.9212500155) |
479573434.7 ns (3288678.305000007) |
1.02 |
tpch_q8/vortex-file-compressed |
308265707.75 ns (3079104.0618750155) |
304548830.45 ns (2334500.133125007) |
1.01 |
tpch_q8/vortex-file-uncompressed |
298478848.55 ns (2107991.453125) |
302837009.3 ns (3097024.393750012) |
0.99 |
tpch_q9/vortex-in-memory-no-pushdown |
427865939.5 ns (2615791.1131249964) |
419212128.7 ns (3705499.324999988) |
1.02 |
tpch_q9/vortex-in-memory-pushdown |
422697539.35 ns (3299021.7931250334) |
420322744.5 ns (2459712.5525000095) |
1.01 |
tpch_q9/arrow |
410902867.85 ns (3485396.221875012) |
392234792.8 ns (3082428.768124968) |
1.05 |
tpch_q9/parquet |
710238228.2 ns (3537441.089999974) |
703530395.3 ns (4167020.3849999905) |
1.01 |
tpch_q9/vortex-file-compressed |
506628348.3 ns (3830142.349999994) |
494295406.95 ns (3218398.9631249905) |
1.02 |
tpch_q9/vortex-file-uncompressed |
487266660.2 ns (2380490.868750006) |
477135686.55 ns (5665862.039999992) |
1.02 |
tpch_q10/vortex-in-memory-no-pushdown |
277433209.8 ns (853324.6693750024) |
273911302.8 ns (747665.8000000119) |
1.01 |
tpch_q10/vortex-in-memory-pushdown |
302004619.8 ns (1050039.6631250381) |
297101239.55 ns (1564350.2556250095) |
1.02 |
tpch_q10/arrow |
269608795.45 ns (2318738.026875004) |
267344823.8 ns (2820922.16125001) |
1.01 |
tpch_q10/parquet |
506008026.6 ns (2021888.894999981) |
496088060.3 ns (1412060.2450000048) |
1.02 |
tpch_q10/vortex-file-compressed |
412069064.75 ns (2042005.650000006) |
406989667.45 ns (1799184.763749987) |
1.01 |
tpch_q10/vortex-file-uncompressed |
395219151.7 ns (1046631.7118749917) |
392971064.75 ns (2930525.25) |
1.01 |
tpch_q11/vortex-in-memory-no-pushdown |
181033791.5056349 ns (778302.3134156615) |
177013950.8859524 ns (997269.1148809493) |
1.02 |
tpch_q11/vortex-in-memory-pushdown |
180347131.48984128 ns (603764.407980144) |
177281745.90670633 ns (766677.9942232221) |
1.02 |
tpch_q11/arrow |
178825190.48115078 ns (852592.9741751105) |
172167266.58392853 ns (1516000.7932291627) |
1.04 |
tpch_q11/parquet |
188575823.56666666 ns (1223598.4833333194) |
184572961.13333336 ns (1353496.8191666752) |
1.02 |
tpch_q11/vortex-file-compressed |
273552221.65 ns (1369895.821875006) |
260119102 ns (3071929.450000003) |
1.05 |
tpch_q11/vortex-file-uncompressed |
272635073.35 ns (2287395.053124994) |
257468440.55 ns (2401063.6712500006) |
1.06 |
tpch_q12/vortex-in-memory-no-pushdown |
231264275.3 ns (565009.1162499785) |
229577569.06666666 ns (449513.92874999344) |
1.01 |
tpch_q12/vortex-in-memory-pushdown |
249987882.53333336 ns (408090.0324999988) |
247962939.49999994 ns (389190.66666665673) |
1.01 |
tpch_q12/arrow |
187782571.46666667 ns (569532.3833333254) |
185701374.00000003 ns (299127.1591666788) |
1.01 |
tpch_q12/parquet |
338698480.5 ns (624391.8662500083) |
337664105.75 ns (1306508.4856249988) |
1.00 |
tpch_q12/vortex-file-compressed |
400062883.65 ns (1089362.7056249976) |
394857975.9 ns (1063580.949999988) |
1.01 |
tpch_q12/vortex-file-uncompressed |
388026070 ns (1357890.5368750095) |
384441916.8 ns (1093032.676249981) |
1.01 |
tpch_q13/vortex-in-memory-no-pushdown |
182339632.9 ns (2188312.8191666454) |
182892440.50718254 ns (6502707.078591287) |
1.00 |
tpch_q13/vortex-in-memory-pushdown |
187292834.1666667 ns (1550828.0166666508) |
166715759.7431746 ns (1404004.481089294) |
1.12 |
tpch_q13/arrow |
182042599.24559525 ns (1197788.9441086352) |
165306228.15392858 ns (1459099.2227976173) |
1.10 |
tpch_q13/parquet |
340733116 ns (1810177.5075000226) |
318413596.85 ns (2182454.0268749893) |
1.07 |
tpch_q13/vortex-file-compressed |
210179050.1666667 ns (1733262.8833333254) |
194083824.63333336 ns (1490987.7999999821) |
1.08 |
tpch_q13/vortex-file-uncompressed |
212121672.63333333 ns (1287277.7454166561) |
207539422.8666667 ns (1605506.3333333284) |
1.02 |
tpch_q14/vortex-in-memory-no-pushdown |
45589063.529265866 ns (341827.9733134918) |
44108887.86488095 ns (893986.2320238054) |
1.03 |
tpch_q14/vortex-in-memory-pushdown |
73980087.55404761 ns (510231.45438391715) |
73525071.09551588 ns (681553.2880357206) |
1.01 |
tpch_q14/arrow |
36995139.153703704 ns (576395.8299490698) |
36830390.92600529 ns (505471.5605539009) |
1.00 |
tpch_q14/parquet |
229061196.76666665 ns (837320.1216666847) |
223798567.9 ns (667855.1675000191) |
1.02 |
tpch_q14/vortex-file-compressed |
118365300.42694445 ns (373672.42372915894) |
118040570.00650795 ns (619896.8822123036) |
1.00 |
tpch_q14/vortex-file-uncompressed |
134980066.166627 ns (441068.9933010936) |
133862377.52940476 ns (775777.3873139769) |
1.01 |
tpch_q15/vortex-in-memory-no-pushdown |
72707121.40373015 ns (303713.98769741505) |
69346688.32615079 ns (320665.5250148922) |
1.05 |
tpch_q15/vortex-in-memory-pushdown |
103773384.0209127 ns (544577.3140644804) |
102719891.24742064 ns (766481.2544320524) |
1.01 |
tpch_q15/arrow |
59298775.20815476 ns (476281.68191666156) |
56849488.87132935 ns (137749.30199255422) |
1.04 |
tpch_q15/parquet |
301947189.35 ns (789681.0381250083) |
296179248.85 ns (1233123.724999994) |
1.02 |
tpch_q15/vortex-file-compressed |
219935462.86666664 ns (1239692.0000000298) |
222005446.06666666 ns (1037689.4670833349) |
0.99 |
tpch_q15/vortex-file-uncompressed |
249882818.4 ns (991258.25) |
251285842.1 ns (1259095.034374997) |
0.99 |
tpch_q16/vortex-in-memory-no-pushdown |
107503134.85107143 ns (510977.73739881814) |
107816069.54900794 ns (915716.617193453) |
1.00 |
tpch_q16/vortex-in-memory-pushdown |
121077701.20753968 ns (450097.7194752097) |
117815889.2234524 ns (287421.62530209124) |
1.03 |
tpch_q16/arrow |
107536336.50035715 ns (700848.2207142711) |
106735130.51281746 ns (578748.1502435431) |
1.01 |
tpch_q16/parquet |
118464151.98448412 ns (406873.05442460626) |
118428579.35246031 ns (524180.67143253237) |
1.00 |
tpch_q16/vortex-file-compressed |
136020861.29174605 ns (435300.2962023914) |
134889272.05682537 ns (488325.01718747616) |
1.01 |
tpch_q16/vortex-file-uncompressed |
136221792.84305558 ns (631950.9261111021) |
133948531.89333332 ns (910672.769749999) |
1.02 |
tpch_q17/vortex-in-memory-no-pushdown |
581208880.6 ns (9909734.73999995) |
554986046.3 ns (28498782.65874991) |
1.05 |
tpch_q17/vortex-in-memory-pushdown |
662147737.6 ns (6176657.569999933) |
612914717.8 ns (4735961.872500062) |
1.08 |
tpch_q17/arrow |
587081708.2 ns (7562808.524999976) |
524517255.7 ns (6674411.25) |
1.12 |
tpch_q17/parquet |
656291254.8 ns (1684712.4962500334) |
637498558.4 ns (5681296.699999988) |
1.03 |
tpch_q17/vortex-file-compressed |
652016127.7 ns (3767542.926249981) |
621822814 ns (4273943.378750026) |
1.05 |
tpch_q17/vortex-file-uncompressed |
621568812 ns (2166622.050000012) |
606289323.2 ns (3596925.344999969) |
1.03 |
tpch_q18/vortex-in-memory-no-pushdown |
1077839446.1 ns (9680204.772499919) |
1072756464.3 ns (8626274.399999976) |
1.00 |
tpch_q18/vortex-in-memory-pushdown |
1057627882.6 ns (6821160.953750014) |
1068887077.8 ns (8298793.360000074) |
0.99 |
tpch_q18/arrow |
1081469697.7 ns (9311121.81250006) |
1068608247.6 ns (17363991.52999997) |
1.01 |
tpch_q18/parquet |
1248477462.5 ns (10349204.923750043) |
1202489809.4 ns (15585252.186249971) |
1.04 |
tpch_q18/vortex-file-compressed |
1162824677.9 ns (5956334.825000048) |
1145156324.4 ns (19160548.56124997) |
1.02 |
tpch_q18/vortex-file-uncompressed |
1122760727.8 ns (8583400.566249967) |
1080620499.9 ns (12256010.043750048) |
1.04 |
tpch_q19/vortex-in-memory-no-pushdown |
181046603.36666667 ns (411327.59999999404) |
182210607.00194445 ns (333948.61780902743) |
0.99 |
tpch_q19/vortex-in-memory-pushdown |
249709521.7 ns (621126.5166666508) |
251200799.15 ns (324334.4487499893) |
0.99 |
tpch_q19/arrow |
165424895.62210315 ns (385646.25553670526) |
166540374.5692857 ns (264525.33207142353) |
0.99 |
tpch_q19/parquet |
456721056.6 ns (1326635.099999994) |
454096059.3 ns (1723064.4174999893) |
1.01 |
tpch_q19/vortex-file-compressed |
400962029.9 ns (1309068.6781250238) |
391325287 ns (1106968.2849999964) |
1.02 |
tpch_q19/vortex-file-uncompressed |
350205767.25 ns (2045679.6381250024) |
352917745.05 ns (931579.599999994) |
0.99 |
tpch_q20/vortex-in-memory-no-pushdown |
255065138.05 ns (3427891.650000006) |
254395809.45 ns (1646583.8981249928) |
1.00 |
tpch_q20/vortex-in-memory-pushdown |
267109893.95 ns (2488374.8693750054) |
256363260.15 ns (1189079.593750015) |
1.04 |
tpch_q20/arrow |
243364483.2 ns (2395390.9395833313) |
241432686.5 ns (5560481.14791666) |
1.01 |
tpch_q20/parquet |
362389861.2 ns (2282421.459374994) |
359106969.55 ns (5172230.152500004) |
1.01 |
tpch_q20/vortex-file-compressed |
358834888.75 ns (2748470.5849999785) |
346643670.95 ns (3145925.336874962) |
1.04 |
tpch_q20/vortex-file-uncompressed |
371185322.95 ns (1720465.2637500167) |
364575517.4 ns (4130506.200000018) |
1.02 |
tpch_q21/vortex-in-memory-no-pushdown |
888074419.9 ns (3212344.207499981) |
861047843.7 ns (5582780.29124999) |
1.03 |
tpch_q21/vortex-in-memory-pushdown |
905438800.3 ns (5412015.362500012) |
895484797.6 ns (8873623.44749999) |
1.01 |
tpch_q21/arrow |
860533368.7 ns (5374147.25) |
857314840 ns (3686318.6549999714) |
1.00 |
tpch_q21/parquet |
994694988.8 ns (7016188.539999962) |
960161993.9 ns (6083477.408749998) |
1.04 |
tpch_q21/vortex-file-compressed |
1186182304.1 ns (2695942.4299999475) |
1189511356.3 ns (14376843.286250114) |
1.00 |
tpch_q21/vortex-file-uncompressed |
1064271790.3 ns (6608628.70875001) |
1061928587.3 ns (6006301.941250026) |
1.00 |
tpch_q22/vortex-in-memory-no-pushdown |
77060006.10333334 ns (191110.88791666925) |
76382901.66676587 ns (252633.43997892737) |
1.01 |
tpch_q22/vortex-in-memory-pushdown |
76222341.79984125 ns (151841.67416270077) |
75204930.35837302 ns (249676.23247618973) |
1.01 |
tpch_q22/arrow |
75009766.51136905 ns (184363.8086822927) |
74516127.71563491 ns (233837.8145367056) |
1.01 |
tpch_q22/parquet |
95789532.37178572 ns (681046.7194404751) |
94661465.93972223 ns (273294.7573055625) |
1.01 |
tpch_q22/vortex-file-compressed |
120129156.74749999 ns (568996.455802083) |
119438599.44706349 ns (521044.83635317534) |
1.01 |
tpch_q22/vortex-file-uncompressed |
119341824.68575397 ns (421617.39947172254) |
116310499.07321426 ns (498364.3532976061) |
1.03 |
This comment was automatically generated by workflow using github-action-benchmark.
There was a problem hiding this comment.
Vortex Compression
| Benchmark suite | Current: c73c36d | Previous: d884d9f | Ratio |
|---|---|---|---|
compress time/taxi |
1247839416.5 ns (4611116.25) |
1238906390.4 ns (3853518.971249819) |
1.01 |
compress time/taxi throughput |
470808924 bytes |
470808924 bytes |
1 |
parquet_rs-zstd compress time/taxi |
1834113084.7 ns (2655601.774999976) |
1780488477.9 ns (5497947.5) |
1.03 |
parquet_rs-zstd compress time/taxi throughput |
470808924 bytes |
470808924 bytes |
1 |
decompress time/taxi |
444297455 ns (2159410.334374994) |
420656176.9 ns (2925202.526250005) |
1.06 |
decompress time/taxi throughput |
470808924 bytes |
470808924 bytes |
1 |
parquet_rs-zstd decompress time/taxi |
310350634.9 ns (1236750.0437500179) |
312223791.15 ns (1292003.8556250036) |
0.99 |
parquet_rs-zstd decompress time/taxi throughput |
470808924 bytes |
470808924 bytes |
1 |
vortex:parquet-zstd size/taxi |
0.9417431949251628 ratio |
0.9417958031701114 ratio |
1.00 |
vortex:raw size/taxi |
0.11193651885833837 ratio |
0.11194277192587794 ratio |
1.00 |
vortex size/taxi |
52700712 bytes |
52703656 bytes |
1.00 |
compress time/AirlineSentiment |
671510.1602207159 ns (1777.8507977059344) |
682766.9353565505 ns (2587.1516616913723) |
0.98 |
compress time/AirlineSentiment throughput |
2020 bytes |
2020 bytes |
1 |
parquet_rs-zstd compress time/AirlineSentiment |
56256.8227433846 ns (69.87736685746495) |
56149.97527175934 ns (180.3787937670022) |
1.00 |
parquet_rs-zstd compress time/AirlineSentiment throughput |
2020 bytes |
2020 bytes |
1 |
decompress time/AirlineSentiment |
38773.17940804633 ns (160.72868709450267) |
39643.895187626214 ns (94.62242937222982) |
0.98 |
decompress time/AirlineSentiment throughput |
2020 bytes |
2020 bytes |
1 |
parquet_rs-zstd decompress time/AirlineSentiment |
31795.679028048435 ns (134.69289186611968) |
32718.40360604191 ns (79.32330090688265) |
0.97 |
parquet_rs-zstd decompress time/AirlineSentiment throughput |
2020 bytes |
2020 bytes |
1 |
vortex:parquet-zstd size/AirlineSentiment |
6.196483971044468 ratio |
6.196483971044468 ratio |
1 |
vortex:raw size/AirlineSentiment |
2.9663366336633663 ratio |
2.9663366336633663 ratio |
1 |
vortex size/AirlineSentiment |
5992 bytes |
5992 bytes |
1 |
compress time/Arade |
2195900022.3 ns (10909053.896250248) |
2239841682.6 ns (8720170.722500086) |
0.98 |
compress time/Arade throughput |
787023760 bytes |
787023760 bytes |
1 |
parquet_rs-zstd compress time/Arade |
3138480332.7 ns (17643332.952499866) |
3059614537.6 ns (10287822.48124981) |
1.03 |
parquet_rs-zstd compress time/Arade throughput |
787023760 bytes |
787023760 bytes |
1 |
decompress time/Arade |
818829595.7 ns (3159728.9975000024) |
787106878 ns (5704453.25) |
1.04 |
decompress time/Arade throughput |
787023760 bytes |
787023760 bytes |
1 |
parquet_rs-zstd decompress time/Arade |
648748375.1 ns (2820001.043749988) |
656764498.4 ns (3695007.2024999857) |
0.99 |
parquet_rs-zstd decompress time/Arade throughput |
787023760 bytes |
787023760 bytes |
1 |
vortex:parquet-zstd size/Arade |
0.47890649124129325 ratio |
0.4789069103731621 ratio |
1.00 |
vortex:raw size/Arade |
0.1858328749820717 ratio |
0.1858328749820717 ratio |
1 |
vortex size/Arade |
146254888 bytes |
146254888 bytes |
1 |
compress time/Bimbo |
10383061708.6 ns (18977457.198750496) |
10726879240.3 ns (33382094.35000038) |
0.97 |
compress time/Bimbo throughput |
7121333608 bytes |
7121333608 bytes |
1 |
parquet_rs-zstd compress time/Bimbo |
21734284329.5 ns (103467919.75) |
20699989222.1 ns (69348320.96624947) |
1.05 |
parquet_rs-zstd compress time/Bimbo throughput |
7121333608 bytes |
7121333608 bytes |
1 |
decompress time/Bimbo |
4833486924.6 ns (49986019.94999981) |
4937909037 ns (37691776.09625006) |
0.98 |
decompress time/Bimbo throughput |
7121333608 bytes |
7121333608 bytes |
1 |
parquet_rs-zstd decompress time/Bimbo |
2628512381.7 ns (8408954.657500029) |
2649494619.7 ns (4174373.1999998093) |
0.99 |
parquet_rs-zstd decompress time/Bimbo throughput |
7121333608 bytes |
7121333608 bytes |
1 |
vortex:parquet-zstd size/Bimbo |
1.2368870803028007 ratio |
1.1815029394361685 ratio |
1.05 |
vortex:raw size/Bimbo |
0.06741699608914038 ratio |
0.06439826263507918 ratio |
1.05 |
vortex size/Bimbo |
480098920 bytes |
458601512 bytes |
1.05 |
compress time/CMSprovider |
11879685663.4 ns (22820559.703749657) |
11931634951.5 ns (55207927.65000057) |
1.00 |
compress time/CMSprovider throughput |
5149123964 bytes |
5149123964 bytes |
1 |
parquet_rs-zstd compress time/CMSprovider |
20062412096.8 ns (61420407.403749466) |
20120265067.1 ns (82400494.50374985) |
1.00 |
parquet_rs-zstd compress time/CMSprovider throughput |
5149123964 bytes |
5149123964 bytes |
1 |
decompress time/CMSprovider |
7613742577.5 ns (118899888.75499964) |
8115959459.1 ns (63919239.851249695) |
0.94 |
decompress time/CMSprovider throughput |
5149123964 bytes |
5149123964 bytes |
1 |
parquet_rs-zstd decompress time/CMSprovider |
4850528278.4 ns (34478627.27750015) |
5020099672.4 ns (63415094.58500004) |
0.97 |
parquet_rs-zstd decompress time/CMSprovider throughput |
5149123964 bytes |
5149123964 bytes |
1 |
vortex:parquet-zstd size/CMSprovider |
1.2143092179759043 ratio |
1.2144312273851054 ratio |
1.00 |
vortex:raw size/CMSprovider |
0.18147308911827162 ratio |
0.1814911116014452 ratio |
1.00 |
vortex size/CMSprovider |
934427432 bytes |
934520232 bytes |
1.00 |
compress time/Euro2016 |
2667078440.4 ns (14869181.720000029) |
2634861345.7 ns (3167414.477499962) |
1.01 |
compress time/Euro2016 throughput |
393253221 bytes |
393253221 bytes |
1 |
parquet_rs-zstd compress time/Euro2016 |
1624679916.3 ns (6303943.006250024) |
1577430425.3 ns (6865771.857500076) |
1.03 |
parquet_rs-zstd compress time/Euro2016 throughput |
393253221 bytes |
393253221 bytes |
1 |
decompress time/Euro2016 |
306656393 ns (2844148.5024999976) |
301404712.55 ns (2567039.2224999964) |
1.02 |
decompress time/Euro2016 throughput |
393253221 bytes |
393253221 bytes |
1 |
parquet_rs-zstd decompress time/Euro2016 |
509677020.2 ns (6422605.591250002) |
508410266.55 ns (5079228.667499989) |
1.00 |
parquet_rs-zstd decompress time/Euro2016 throughput |
393253221 bytes |
393253221 bytes |
1 |
vortex:parquet-zstd size/Euro2016 |
1.4383489987803337 ratio |
1.4383489987803337 ratio |
1 |
vortex:raw size/Euro2016 |
0.4348474490943839 ratio |
0.4348474490943839 ratio |
1 |
vortex size/Euro2016 |
171005160 bytes |
171005160 bytes |
1 |
compress time/Food |
1108542842.8 ns (5966918.118750095) |
1131883143.6 ns (4009900.25) |
0.98 |
compress time/Food throughput |
332718229 bytes |
332718229 bytes |
1 |
parquet_rs-zstd compress time/Food |
1166043266.2 ns (7371553.598749876) |
1115385661.7 ns (3876845.718749881) |
1.05 |
parquet_rs-zstd compress time/Food throughput |
332718229 bytes |
332718229 bytes |
1 |
decompress time/Food |
209886477.9 ns (1010333.1408333182) |
212033647.09999996 ns (963261.71875) |
0.99 |
decompress time/Food throughput |
332718229 bytes |
332718229 bytes |
1 |
parquet_rs-zstd decompress time/Food |
217885425.6 ns (811036.7783333212) |
218582790.46666664 ns (1211018) |
1.00 |
parquet_rs-zstd decompress time/Food throughput |
332718229 bytes |
332718229 bytes |
1 |
vortex:parquet-zstd size/Food |
1.2389367897825485 ratio |
1.2386506182043049 ratio |
1.00 |
vortex:raw size/Food |
0.1349090374005327 ratio |
0.13487787589780662 ratio |
1.00 |
vortex size/Food |
44886696 bytes |
44876328 bytes |
1.00 |
compress time/HashTags |
2625099124.9 ns (12476636.505000114) |
2601287602.4 ns (17833982.950000048) |
1.01 |
compress time/HashTags throughput |
804495592 bytes |
804495592 bytes |
1 |
parquet_rs-zstd compress time/HashTags |
2601316225.4 ns (12801645.216250181) |
2540956099.9 ns (10280427.105000257) |
1.02 |
parquet_rs-zstd compress time/HashTags throughput |
804495592 bytes |
804495592 bytes |
1 |
decompress time/HashTags |
635733108.9 ns (8955415.587499976) |
653038982.3 ns (5991007.28125) |
0.97 |
decompress time/HashTags throughput |
804495592 bytes |
804495592 bytes |
1 |
parquet_rs-zstd decompress time/HashTags |
846543196.4 ns (10203674.074999988) |
829965254.8 ns (13047694.268750012) |
1.02 |
parquet_rs-zstd decompress time/HashTags throughput |
804495592 bytes |
804495592 bytes |
1 |
vortex:parquet-zstd size/HashTags |
1.6563035357058875 ratio |
1.6563035357058875 ratio |
1 |
vortex:raw size/HashTags |
0.2758095012657322 ratio |
0.2758095012657322 ratio |
1 |
vortex size/HashTags |
221887528 bytes |
221887528 bytes |
1 |
compress time/TPC-H l_comment chunked without fsst |
4215416418.4 ns (33738029.9775002) |
4142602128.7 ns (94850583.66000009) |
1.02 |
compress time/TPC-H l_comment chunked without fsst throughput |
249197090 bytes |
249197090 bytes |
1 |
parquet_rs-zstd compress time/TPC-H l_comment chunked without fsst |
939196337.9 ns (4988044.5) |
920175450 ns (3333363.050000012) |
1.02 |
parquet_rs-zstd compress time/TPC-H l_comment chunked without fsst throughput |
249197090 bytes |
249197090 bytes |
1 |
decompress time/TPC-H l_comment chunked without fsst |
117351042.84396824 ns (1532876.777759932) |
117056858.70055556 ns (1063581.4461458474) |
1.00 |
decompress time/TPC-H l_comment chunked without fsst throughput |
249197090 bytes |
249197090 bytes |
1 |
parquet_rs-zstd decompress time/TPC-H l_comment chunked without fsst |
258108863.4 ns (1069178.474999994) |
254290945.6 ns (969345.9793749899) |
1.02 |
parquet_rs-zstd decompress time/TPC-H l_comment chunked without fsst throughput |
249197090 bytes |
249197090 bytes |
1 |
vortex:parquet-zstd size/TPC-H l_comment chunked without fsst |
4.60767197511081 ratio |
4.607619442861625 ratio |
1.00 |
vortex:raw size/TPC-H l_comment chunked without fsst |
1.0527068995869895 ratio |
1.0527071564118184 ratio |
1.00 |
vortex size/TPC-H l_comment chunked without fsst |
262331496 bytes |
262331560 bytes |
1.00 |
compress time/TPC-H l_comment chunked |
954960852.3 ns (3368876.0537499785) |
942934350.1 ns (4054982.307500005) |
1.01 |
compress time/TPC-H l_comment chunked throughput |
249197090 bytes |
249197090 bytes |
1 |
parquet_rs-zstd compress time/TPC-H l_comment chunked |
935138820.7 ns (4559791.99000001) |
926292169.9 ns (5513548.400000036) |
1.01 |
parquet_rs-zstd compress time/TPC-H l_comment chunked throughput |
249197090 bytes |
249197090 bytes |
1 |
decompress time/TPC-H l_comment chunked |
136400134.4818254 ns (847028.6606101245) |
136563601.22234127 ns (1406311.590615064) |
1.00 |
decompress time/TPC-H l_comment chunked throughput |
249197090 bytes |
249197090 bytes |
1 |
parquet_rs-zstd decompress time/TPC-H l_comment chunked |
258411983.1 ns (1461219.428124994) |
254511649.25 ns (901226.9762499928) |
1.02 |
parquet_rs-zstd decompress time/TPC-H l_comment chunked throughput |
249197090 bytes |
249197090 bytes |
1 |
vortex:parquet-zstd size/TPC-H l_comment chunked |
1.347992155989489 ratio |
1.3479787068229914 ratio |
1.00 |
vortex:raw size/TPC-H l_comment chunked |
0.30797345185692176 ratio |
0.3079739655065796 ratio |
1.00 |
vortex size/TPC-H l_comment chunked |
76746088 bytes |
76746216 bytes |
1.00 |
compress time/TPC-H l_comment canonical |
947654513.4 ns (2665584.1356250644) |
928654297.25 ns (3339421.2568749785) |
1.02 |
compress time/TPC-H l_comment canonical throughput |
249197106 bytes |
249197106 bytes |
1 |
parquet_rs-zstd compress time/TPC-H l_comment canonical |
949756182.55 ns (3460838.241250038) |
929155648.45 ns (2684684.106249988) |
1.02 |
parquet_rs-zstd compress time/TPC-H l_comment canonical throughput |
249197106 bytes |
249197106 bytes |
1 |
decompress time/TPC-H l_comment canonical |
135014010.9255423 ns (455158.2384201437) |
135845756.44195768 ns (911322.9764946997) |
0.99 |
decompress time/TPC-H l_comment canonical throughput |
249197106 bytes |
249197106 bytes |
1 |
parquet_rs-zstd decompress time/TPC-H l_comment canonical |
258502341.58244047 ns (1051512.714176327) |
253532402.38039678 ns (896869.5840615034) |
1.02 |
parquet_rs-zstd decompress time/TPC-H l_comment canonical throughput |
249197106 bytes |
249197106 bytes |
1 |
vortex:parquet-zstd size/TPC-H l_comment canonical |
1.3479946420321833 ratio |
1.3479744214731058 ratio |
1.00 |
vortex:raw size/TPC-H l_comment canonical |
0.3079734320831158 ratio |
0.30797394573274056 ratio |
1.00 |
vortex size/TPC-H l_comment canonical |
76746088 bytes |
76746216 bytes |
1.00 |
This comment was automatically generated by workflow using github-action-benchmark.
robert3005
approved these changes
Oct 29, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Resolves #1050 .
I saw a modest (~5%) reduction in decompress time on a string-heavy dataset in which SparseArray take occupies ~10% of total time.