-
Notifications
You must be signed in to change notification settings - Fork 1
References
Matthew Jadud edited this page Jan 25, 2025
·
11 revisions
- https://til.simonwillison.net/sqlite
- https://avi.im/blag/2021/fast-sqlite-inserts/
- https://simonwillison.net/2021/Mar/15/sqlite-spellfix/
- https://observablehq.com/@asg017/scrape-json-html-zip-with-sqlite
- https://til.simonwillison.net/sqlite/copy-tables-between-databases
- https://github.com/rqlite/rqlite?tab=readme-ov-file
- https://phiresky.github.io/blog/2021/hosting-sqlite-databases-on-github-pages/
- https://til.simonwillison.net/sqlite/database-file-size
- https://til.simonwillison.net/sqlite/json-audit-log
- https://github.com/jtarchie/sqlitezstd
- https://www.reddit.com/r/golang/comments/1beqri3/use_zstd_seekable_compressed_readonly_sqlite_db/
- https://github.com/phiresky/sqlite-zstd?tab=readme-ov-file
- https://phiresky.github.io/blog/2022/sqlite-zstd/
- https://trunc.org/learning/compressing-sqlite-with-zfs
- https://kimsereylam.com/sqlite/2020/03/06/full-text-search-with-sqlite.html
- https://stackoverflow.com/questions/70847617/populate-virtual-sqlite-fts5-full-text-search-table-from-content-table
- https://blog.ouseful.info/2022/04/06/compiling-full-text-search-fts5-into-sqlite-wasm-build/
- https://abdus.dev/posts/quick-full-text-search-using-sqlite/
- https://darksi.de/13.sqlite-fts5-structure/
- https://sqlite.org/forum/info/00d53dbed15f5e5a
- https://charlesleifer.com/blog/using-sqlite-full-text-search-with-python/
- https://sqlite.org/fts5.html#synonym_support
- https://www.sqlite.org/spellfix1.html
- https://stackoverflow.com/questions/52803014/sqlite-with-real-full-text-search-and-spelling-mistakes-ftsspellfix-together
- https://stackoverflow.com/questions/49779281/string-similarity-with-python-sqlite-levenshtein-distance-edit-distance
- https://resources.gtaf.org/Technology/Sqlite-FTS-Aarabic-Tokeniser-e82e2f2d5db04f139e160e1ae4964638/
-
https://github.com/asg017/sqlite-html
- Extract HTML w/ CSS selectors (directly)
- Raises questions about where to store HTML...
-
https://llm.datasette.io/en/stable/embeddings/index.html
- Some python tools from Datasette that are for extracting text embeddings.
- See also: https://til.simonwillison.net/sqlite/sqlite-vec
-
https://github.com/asg017/sqlite-vec
- https://alexgarcia.xyz/blog/2024/building-new-vector-search-sqlite/index.html
- https://www.nomic.ai/blog/posts/nomic-embed-matryoshka
- This could serve as the starting point for "find pages similar to this on this domain." Or... any number of things. (How many "similar" pages are there on a .gov? Probably a lot.)
- https://github.com/asg017/sqlite-ecosystem
- https://github.com/asg017/sqlite-rembed
- Leverages language models to try and find related text in the DB
- Probably a safe(ish) use of LLMs, as it takes your text and tries to find related articles. No content is being generated, but instead we're doing a best-guess similarity to the text in the DB.
- https://github.com/asg017/sqlite-lembed?tab=readme-ov-file
- https://github.com/ggerganov/llama.cpp
- https://til.simonwillison.net/llms/openai-embeddings-related-content
It might be that more work can be done by SQLite for me. For example, I'm currently snowballing in the code.
perhaps I can compile/load the extension in the app?
-
https://shot-scraper.datasette.io/en/stable/index.html
- This is a utility that can grab screencaps of web pages as well as run Javascript against a page in a virtual browser. That means we could extract content directly, or automate tests, or...
- https://www.zenrows.com/blog/goquery
- https://pkg.go.dev/github.com/anonimy/go-spell-checker
-
https://github.com/riyaz-ali/sqlite
- Build SQlite extensions in Go
- https://stackoverflow.com/questions/70847617/populate-virtual-sqlite-fts5-full-text-search-table-from-content-table
- https://blog.ouseful.info/2022/04/06/compiling-full-text-search-fts5-into-sqlite-wasm-build/
- https://phiresky.github.io/blog/2021/hosting-sqlite-databases-on-github-pages/
- https://abdus.dev/posts/quick-full-text-search-using-sqlite/
- https://darksi.de/13.sqlite-fts5-structure/
- https://sqlite.org/forum/info/00d53dbed15f5e5a
- https://charlesleifer.com/blog/using-sqlite-full-text-search-with-python/
- https://sqlite.org/fts5.html#synonym_support
- https://www.sqlite.org/spellfix1.html
- https://stackoverflow.com/questions/52803014/sqlite-with-real-full-text-search-and-spelling-mistakes-ftsspellfix-together
- https://stackoverflow.com/questions/49779281/string-similarity-with-python-sqlite-levenshtein-distance-edit-distance
- https://resources.gtaf.org/Technology/Sqlite-FTS-Aarabic-Tokeniser-e82e2f2d5db04f139e160e1ae4964638/
- https://lunrjs.com/docs/index.html