Run super command perf tests in vector runtime that couldn't before #5552
Conversation
| This code path in `super` is not multi-threaded so not particularly performant, | ||
| but on our test machine it runs a bit faster than both the `duckdb` method of | ||
| creating a schema-fused table or loading the data to the `clickhouse` beta JSON type. |
There was a problem hiding this comment.
Even though I used the same DuckDB release version and AWS instance type as the last time I ran these tests, for some reason DuckDB did happen to run this "table creation" step substantially faster than last time (328 seconds now vs. 513 seconds last time). I can't explain it offhand and I'm not inspired to take a detour and study it deeply, but in the interest of science I've corrected the summary here to reflect that. I'll keep an eye on it in future runs and maybe the operation just turns out to have high variance.
| ClickHouse's beta JSON type_ | ||
| Since DuckDB with its native format could successfully run all queries with | ||
| decent performance, we used it as the baseline for all of the speed-up factors. | ||
| To summarize, | ||
| `super` with Super Binary is substantially faster than multiple relational systems for | ||
| the search use cases and performs on par with the others for traditional OLAP queries, | ||
| the search use cases, and with Parquet performs on par with the others for traditional OLAP queries, |
There was a problem hiding this comment.
Especially now that we have across-the-boards Parquet results, I figured I might as well clarify that those are the ones where super perf comes closest with OLAP queries. Looking forward to seeing CSUP showing the best performance. 🤞
philrz
changed the title
What's Changing
Query performance results are added to the
supercommand doc for tests that couldn't be run previously, thanks to the merge of recent PRs that enabled new functionality in vector runtime.Why
This allows us to get rid of some cells in the results summary where we previously had disclaimers saying we couldn't run these yet.
Details
When making these updates I hoped to show vector CSUP performance with these queries as well, but I bumped into new issue #5550. Therefore I've left the relevant parts of the scripts commented out for now, but will revisit once #5550 is addressed.