-
Notifications
You must be signed in to change notification settings - Fork 30
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Lucene query improvements #311
Conversation
This comment was marked as outdated.
This comment was marked as outdated.
16644d9
to
e82bddd
Compare
kaldb/src/main/java/com/slack/kaldb/logstore/search/LogIndexSearcherImpl.java
Outdated
Show resolved
Hide resolved
kaldb/src/main/java/com/slack/kaldb/logstore/search/LogIndexSearcherImpl.java
Show resolved
Hide resolved
kaldb/src/main/java/com/slack/kaldb/logstore/LuceneIndexStoreImpl.java
Outdated
Show resolved
Hide resolved
kaldb/src/main/java/com/slack/kaldb/logstore/search/LogIndexSearcherImpl.java
Outdated
Show resolved
Hide resolved
kaldb/src/main/java/com/slack/kaldb/logstore/LuceneIndexStoreConfig.java
Outdated
Show resolved
Hide resolved
kaldb/src/main/java/com/slack/kaldb/logstore/search/LogIndexSearcherImpl.java
Outdated
Show resolved
Hide resolved
A few questions about the benchmarks:
But calculating both histogram and topK takes less time than either: I am curious how we can explain the speed up? Could the speed up be due to a bug in our code? Can you ensure that we have unit tests(existing or new) that test all of these outputs to make sure the outputs match the expected result? |
6d94ce2
to
72c32e8
Compare
Updated benchmarks:
|
kaldb/src/main/java/com/slack/kaldb/logstore/LuceneIndexStoreConfig.java
Outdated
Show resolved
Hide resolved
kaldb/src/main/java/com/slack/kaldb/logstore/search/LogIndexSearcherImpl.java
Outdated
Show resolved
Hide resolved
kaldb/src/main/java/com/slack/kaldb/logstore/search/LogIndexSearcherImpl.java
Outdated
Show resolved
Hide resolved
kaldb/src/test/java/com/slack/kaldb/chunk/IndexingChunkImplTest.java
Outdated
Show resolved
Hide resolved
21ca60b
to
9705487
Compare
Reworks collectors to use executor service, removes unnecceary collectors depending on provided params
This reverts commit e82bddd.
9705487
to
84e5a75
Compare
SimpleCollector
, matching the built-in collectors, https://lucene.apache.org/core/8_0_0/core/org/apache/lucene/search/SimpleCollector.html