-
Notifications
You must be signed in to change notification settings - Fork 6
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Deployed 0952bbd with MkDocs version: 1.5.3
- Loading branch information
Showing
18 changed files
with
303 additions
and
20 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,187 @@ | ||
| <!DOCTYPE html> | ||
| <html class="writer-html5" lang="en" > | ||
| <head> | ||
| <meta charset="utf-8" /> | ||
| <meta http-equiv="X-UA-Compatible" content="IE=edge" /> | ||
| <meta name="viewport" content="width=device-width, initial-scale=1.0" /><link rel="canonical" href="https://ossirytk.github.io/llama-cpp-chat-memory/named_entity_recognition/" /> | ||
| <link rel="shortcut icon" href="../img/favicon.ico" /> | ||
| <title>Named Entity Recognition(NER) - Llama.cpp chat</title> | ||
| <link rel="stylesheet" href="../css/theme.css" /> | ||
| <link rel="stylesheet" href="../css/theme_extra.css" /> | ||
| <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.8.0/styles/github.min.css" /> | ||
|
|
||
| <script> | ||
| // Current page data | ||
| var mkdocs_page_name = "Named Entity Recognition(NER)"; | ||
| var mkdocs_page_input_path = "named_entity_recognition.md"; | ||
| var mkdocs_page_url = "/llama-cpp-chat-memory/named_entity_recognition/"; | ||
| </script> | ||
|
|
||
| <!--[if lt IE 9]> | ||
| <script src="../js/html5shiv.min.js"></script> | ||
| <![endif]--> | ||
| <script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.8.0/highlight.min.js"></script> | ||
| <script>hljs.highlightAll();</script> | ||
| </head> | ||
|
|
||
| <body class="wy-body-for-nav" role="document"> | ||
|
|
||
| <div class="wy-grid-for-nav"> | ||
| <nav data-toggle="wy-nav-shift" class="wy-nav-side stickynav"> | ||
| <div class="wy-side-scroll"> | ||
| <div class="wy-side-nav-search"> | ||
| <a href=".." class="icon icon-home"> Llama.cpp chat | ||
| </a><div role="search"> | ||
| <form id ="rtd-search-form" class="wy-form" action="../search.html" method="get"> | ||
| <input type="text" name="q" placeholder="Search docs" aria-label="Search docs" title="Type search term here" /> | ||
| </form> | ||
| </div> | ||
| </div> | ||
|
|
||
| <div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="Navigation menu"> | ||
| <ul> | ||
| <li class="toctree-l1"><a class="reference internal" href="..">Home</a> | ||
| </li> | ||
| </ul> | ||
| <p class="caption"><span class="caption-text">Quickstart</span></p> | ||
| <ul> | ||
| <li class="toctree-l1"><a class="reference internal" href="../getting_started/">Getting started</a> | ||
| </li> | ||
| <li class="toctree-l1"><a class="reference internal" href="../prompt_support/">Prompt Support</a> | ||
| </li> | ||
| <li class="toctree-l1"><a class="reference internal" href="../card_format/">Card Format</a> | ||
| </li> | ||
| <li class="toctree-l1"><a class="reference internal" href="../configs/">Configs</a> | ||
| </li> | ||
| <li class="toctree-l1"><a class="reference internal" href="../preparing_the_env/">Preparing the env</a> | ||
| </li> | ||
| </ul> | ||
| <p class="caption"><span class="caption-text">The chatbot</span></p> | ||
| <ul> | ||
| <li class="toctree-l1"><a class="reference internal" href="../running_the_env/">Running the env</a> | ||
| </li> | ||
| <li class="toctree-l1"><a class="reference internal" href="../running_the_chatbot/">Running the chatbot</a> | ||
| </li> | ||
| </ul> | ||
| <p class="caption"><span class="caption-text">Working with documents and the vectorstore</span></p> | ||
| <ul class="current"> | ||
| <li class="toctree-l1"><a class="reference internal" href="../webscraping/">Webscraping</a> | ||
| </li> | ||
| <li class="toctree-l1 current"><a class="reference internal current" href="./">Named Entity Recognition(NER)</a> | ||
| <ul class="current"> | ||
| </ul> | ||
| </li> | ||
| <li class="toctree-l1"><a class="reference internal" href="../creating_embeddings/">Creating embeddings</a> | ||
| </li> | ||
| </ul> | ||
| <p class="caption"><span class="caption-text">Extras</span></p> | ||
| <ul> | ||
| <li class="toctree-l1"><a class="reference internal" href="../examples/">Some Examples</a> | ||
| </li> | ||
| <li class="toctree-l1"><a class="reference internal" href="../UNLICENSE/">License</a> | ||
| </li> | ||
| </ul> | ||
| </div> | ||
| </div> | ||
| </nav> | ||
|
|
||
| <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap"> | ||
| <nav class="wy-nav-top" role="navigation" aria-label="Mobile navigation menu"> | ||
| <i data-toggle="wy-nav-top" class="fa fa-bars"></i> | ||
| <a href="..">Llama.cpp chat</a> | ||
|
|
||
| </nav> | ||
| <div class="wy-nav-content"> | ||
| <div class="rst-content"><div role="navigation" aria-label="breadcrumbs navigation"> | ||
| <ul class="wy-breadcrumbs"> | ||
| <li><a href=".." class="icon icon-home" aria-label="Docs"></a></li> | ||
| <li class="breadcrumb-item">Working with documents and the vectorstore</li> | ||
| <li class="breadcrumb-item active">Named Entity Recognition(NER)</li> | ||
| <li class="wy-breadcrumbs-aside"> | ||
| <a href="https://github.com/ossirytk/llama-cpp-chat-memory/edit/master/docs/named_entity_recognition.md" class="icon icon-github"> Edit on GitHub</a> | ||
| </li> | ||
| </ul> | ||
| <hr/> | ||
| </div> | ||
| <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article"> | ||
| <div class="section" itemprop="articleBody"> | ||
|
|
||
| <h3 id="named-entity-recognitionner">Named Entity Recognition(NER)</h3> | ||
| <p>You can use textacy_parsing script for generating document metadata keys automatically. The scripts are a modified version of textacy code updated to run with the current spacy version. The script uses a spacy embeddings model to process a text document for a json metadata keyfile. The include positions are: "PROPN", "NOUN", "ADJ". The includes entities are: "PRODUCT", "EVENT", "FAC", "NORP", "PERSON", "ORG", "GPE", "LOC", "DATE", "TIME", "WORK_OF_ART". For details see <a href="https://spacy.io/usage/linguistic-features">Spacy linguistic features</a> and <a href="https://spacy.io/models/en">Model NER labels</a>. The instructions expect en model, but spacy supports a wide range of models.</p> | ||
| <p>You can create ner metadata list with</p> | ||
| <pre><code>python -m document_parsing.textacy_parsing | ||
| </code></pre> | ||
| <table> | ||
| <thead> | ||
| <tr> | ||
| <th>Optional param</th> | ||
| <th>Description</th> | ||
| </tr> | ||
| </thead> | ||
| <tbody> | ||
| <tr> | ||
| <td>--data-directory</td> | ||
| <td>The directory where your text files are stored. Default "./documents/skynet"</td> | ||
| </tr> | ||
| <tr> | ||
| <td>--collection-name</td> | ||
| <td>The name of the collection Will be used as name and location for the keyfile. Default "skynet"</td> | ||
| </tr> | ||
| <tr> | ||
| <td>--key-storage</td> | ||
| <td>The directory for the collection metadata keys. Default "./key_storage/"</td> | ||
| </tr> | ||
| </tbody> | ||
| </table> | ||
|
|
||
| </div> | ||
| </div><footer> | ||
| <div class="rst-footer-buttons" role="navigation" aria-label="Footer Navigation"> | ||
| <a href="../webscraping/" class="btn btn-neutral float-left" title="Webscraping"><span class="icon icon-circle-arrow-left"></span> Previous</a> | ||
| <a href="../creating_embeddings/" class="btn btn-neutral float-right" title="Creating embeddings">Next <span class="icon icon-circle-arrow-right"></span></a> | ||
| </div> | ||
|
|
||
| <hr/> | ||
|
|
||
| <div role="contentinfo"> | ||
| <!-- Copyright etc --> | ||
| </div> | ||
|
|
||
| Built with <a href="https://www.mkdocs.org/">MkDocs</a> using a <a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a> provided by <a href="https://readthedocs.org">Read the Docs</a>. | ||
| </footer> | ||
|
|
||
| </div> | ||
| </div> | ||
|
|
||
| </section> | ||
|
|
||
| </div> | ||
|
|
||
| <div class="rst-versions" role="note" aria-label="Versions"> | ||
| <span class="rst-current-version" data-toggle="rst-current-version"> | ||
|
|
||
| <span> | ||
| <a href="https://github.com/ossirytk/llama-cpp-chat-memory" class="fa fa-github" style="color: #fcfcfc"> GitHub</a> | ||
| </span> | ||
|
|
||
|
|
||
| <span><a href="../webscraping/" style="color: #fcfcfc">« Previous</a></span> | ||
|
|
||
|
|
||
| <span><a href="../creating_embeddings/" style="color: #fcfcfc">Next »</a></span> | ||
|
|
||
| </span> | ||
| </div> | ||
| <script src="../js/jquery-3.6.0.min.js"></script> | ||
| <script>var base_url = "..";</script> | ||
| <script src="../js/theme_extra.js"></script> | ||
| <script src="../js/theme.js"></script> | ||
| <script src="../search/main.js"></script> | ||
| <script> | ||
| jQuery(function () { | ||
| SphinxRtdTheme.Navigation.enable(true); | ||
| }); | ||
| </script> | ||
|
|
||
| </body> | ||
| </html> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Large diffs are not rendered by default.
Oops, something went wrong.
Oops, something went wrong.