-
Notifications
You must be signed in to change notification settings - Fork 47
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Full text search #318
Comments
I love the idea, but I don't know if we can keep an index in memory, so this might mean using something like ElasticSearch unfortunately. |
Wouldn't need to go all the way to ElasticSearch, do we? I think databases like sqlite and postgres have text-search. sqlite fts Would require moving from filesystem-as-database to sqlite-as-database. |
We could store the index on disk ourselves perhaps. I really like the way we use the filesystem right now, it makes maintenance very simple. |
How about routinely merging all json files in "data/verified" into a single file containing all the comments? In terms of performance, we either search the file by scanning it or simply keeping it entirely in memory. For the former, we would just do something line-based and for the latter use json. |
This just showed up in my github feed: https://github.com/well-typed/full-text-search and might be worth looking into. It comes with an example that coincidentally also searches packages. |
Include comments in the search index, so that e.g. if I search for
Kleisli
I getStar
from Data.Profunctor. Note that the documentation uses the word "Kleisli" but the word "Kleisli" doesn't appear in any of the types, which is why this query currently returns no results.The text was updated successfully, but these errors were encountered: