-
Notifications
You must be signed in to change notification settings - Fork 4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Caching fix #282
Caching fix #282
Conversation
private final LRUQueryCache queryCache; | ||
private final IndexReader indexReader; | ||
|
||
public CacheEntry(LRUQueryCache queryCache, IndexReader indexReader) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I dont know anything about lucene, but is caching a Reader a good idea? Does this keep file handles open?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
and if we cache them, should we be caching one reader instance per lucene index, rather than one per cache result?
This moves to caching one IndexSearcher per lucene index path instead of one IndexReader. I moved the error handling to the caching function instead of the base of the API call which is why there are so many line changes in the most recent patch. If this passes (except for that postgres issue) we should reindex. |
@hextraza: i think this is probably fine. there's one thing i think we need and one question i have:
as i write this: open file handles could be a problem if unrelated code tried to update the lucene index files, right? |
Caching was getting aggressively emptied because Lucene has a callback running on a loop internally that empties the cache if the IndexReader gets garbage collected. This patch caches the IndexReader per-session alongside the LRUQueryCache and slightly modifies the caching policy, which should improve the cache's behavior in prod.