From 47252b772aecb9d7490c84cb8e4efce55266cec7 Mon Sep 17 00:00:00 2001 From: Daksh Sharma Date: Tue, 29 Aug 2023 10:57:08 +0530 Subject: [PATCH 1/2] Draft Nuclia Documentation --- nuclia.md | 85 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 85 insertions(+) create mode 100644 nuclia.md diff --git a/nuclia.md b/nuclia.md new file mode 100644 index 000000000..f12f6796f --- /dev/null +++ b/nuclia.md @@ -0,0 +1,85 @@ +# Replacing Sphinx Search with Nuclia Search in Plone + +## Overview + +This documentation outlines the process of replacing Sphinx search in a Plone site with Nuclia search. The goal is to explain the approach taken and provide instructions for replicating the process in another project. + +## Approach + +### Data Indexing and Processing + +To achieve the transition, the following steps were taken: + +1. **Generate Nuclia Sync Data:** + - Iterate through `.md` files in the repository [docs](https://github.com/plone/documentation/tree/6.0/docs) . + - Calculate the hash of each file and store the mapping in `nuclia_sync.json`. + +2. **Extract Headings:** + - Use regular expressions to extract headings from Markdown files. + - Store the extracted headings and slugs for document URLs. + +3. **Upload Documents:** + - Utilize the `NucliaUpload` class from the `nuclia` SDK. + - Generated slugs, Extracted headings, Nuclia `API_KEY` and constructed URLs for `.md` can be passed in the class method. + +4. **Synchronize Documents:** + - Compare hashes to determine whether to upload, update, or delete documents, when changes are made. + - Handle document deletions using the `NucliaKB` class. + +5. **GitHub Actions Workflow:** + - Created a synchronization workflow triggered by a `push` event. + - Defined the steps for checking out code, setting up the Python environment, running the sync script, and committing changes back to the repository. + +## Replication Steps + +Follow these steps to replicate the process in your own project: + +### Set Up + +Copy this in your terminal to clone plone/documentation + +```bash + git clone `` + + pip install -q -r requirements-initial.txt + + pip install -q -r requirements.txt +``` + +### Global Configuration + +1. **PUBLIC_URL:** + - Set the URL of your website +2. **API Key:** + - Obtain the API key from your nuclia Knowledge Box. +3. **Knowledge Base URL:** + - Set up the variable to define the Nuclia knowledge box URL. + +### Indexing and Syncing + +1. **Generate Nuclia Sync Data:** + + ```bash + python3 upload.py + ``` + +2. **Sync Documents:** + +- Run the `sync` function to upload, update, or delete documents in Nuclia. + +### GitHub Actions + +- **Workflow Setup:** + Modify the GitHub Actions workflow `nuclia_sync.yml` to match your repository structure. + +>## Usage Notes +> +>- Ensure that the API key and knowledge base URL are correctly configured. +>- Regularly update the sync process to keep the knowledge base up to date. +>- Troubleshoot issues by checking API key validity and document URLs. + +### Conclusion + +Replacing Sphinx search with Nuclia search brings improved search functionality to Plone sites. Feel free to reach out for assistance or clarification on any aspect of this documentation. + +Happy syncing and searching! From ad1d0df6db60a907b3050e01596d01439ade0dc6 Mon Sep 17 00:00:00 2001 From: justdaksh Date: Wed, 18 Oct 2023 00:15:09 +0530 Subject: [PATCH 2/2] Required Changes made to nuclia.md documentation --- nuclia.md | 113 +++++++++++++++++++++++++++++++----------------------- 1 file changed, 65 insertions(+), 48 deletions(-) diff --git a/nuclia.md b/nuclia.md index f12f6796f..2cc9f05d3 100644 --- a/nuclia.md +++ b/nuclia.md @@ -6,80 +6,97 @@ This documentation outlines the process of replacing Sphinx search in a Plone si ## Approach -### Data Indexing and Processing +To achieve the transition from Sphinx to Nuclia search, we took an approach as shown in the following generalized outline. -To achieve the transition, the following steps were taken: +1. Generate Nuclia sync data -1. **Generate Nuclia Sync Data:** - - Iterate through `.md` files in the repository [docs](https://github.com/plone/documentation/tree/6.0/docs) . - - Calculate the hash of each file and store the mapping in `nuclia_sync.json`. + - Iterate through `.md` files in the repository [docs](https://github.com/plone/documentation/tree/6.0/docs) . + - Calculate the hash of each file and store the mapping in `nuclia_sync.json`. -2. **Extract Headings:** - - Use regular expressions to extract headings from Markdown files. - - Store the extracted headings and slugs for document URLs. +2. Extract Headings -3. **Upload Documents:** - - Utilize the `NucliaUpload` class from the `nuclia` SDK. - - Generated slugs, Extracted headings, Nuclia `API_KEY` and constructed URLs for `.md` can be passed in the class method. + - Use regular expressions to extract headings from Markdown files. + - Store the extracted headings and slugs for document URLs. -4. **Synchronize Documents:** - - Compare hashes to determine whether to upload, update, or delete documents, when changes are made. - - Handle document deletions using the `NucliaKB` class. +3. Upload Documents -5. **GitHub Actions Workflow:** - - Created a synchronization workflow triggered by a `push` event. - - Defined the steps for checking out code, setting up the Python environment, running the sync script, and committing changes back to the repository. + - Utilize the `NucliaUpload` class from the `nuclia` SDK. + - Generated slugs, Extracted headings, Nuclia `API_KEY` and constructed URLs for `.md` can be passed in the class method. -## Replication Steps +4. Synchronize Documents + + - Compare hashes to determine whether to upload, update, or delete documents, when changes are made. + - Handle document deletions using the `NucliaKB` class. + +5. GitHub Actions Workflow + + - Created a synchronization workflow triggered by a `push` event. + - Defined the steps for checking out code, setting up the Python environment, running the sync script, and committing changes back to the repository. + +## Replication steps Follow these steps to replicate the process in your own project: -### Set Up +### Pre-requisites + +- See [installation](https://6.docs.plone.org/install/index.html) for requirements to build Plone 6 Documentation. +- See `Training ` for requirements to build Plone Training. + +### Global configuration for `upload.py` + +1. `PUBLIC_URL` + - Set the URL of your website. + +2. `API Key` + - Set this variable with the API key of your knowledge box. + +3. `KB` + - Set up this variable with your knowledge box URL. + +### Indexing and syncing + +1. Generate Sync Document -Copy this in your terminal to clone plone/documentation + - Make sure to create a `json` file and replace each path of `json` file being read or written inside `upload` script with the path of your `json` file. + - Fill the `json` with content as given below this is essential for syncing old and new data. -```bash - git clone `` + ```json + { + "docs" : { - pip install -q -r requirements-initial.txt + } + } + ``` - pip install -q -r requirements.txt -``` +2. Populate Sync Document -### Global Configuration + - Running the `upload` script will now eventually handle the syncing and indexing to the knowledge box. -1. **PUBLIC_URL:** - - Set the URL of your website -2. **API Key:** - - Obtain the API key from your nuclia Knowledge Box. -3. **Knowledge Base URL:** - - Set up the variable to define the Nuclia knowledge box URL. + ```bash + python3 upload.py + ``` -### Indexing and Syncing +### Integrating the widget -1. **Generate Nuclia Sync Data:** +- Nuclia provides with it's own widget code for our use. - ```bash - python3 upload.py - ``` +- Choose the functionalities you want your widget to have from the widget generator tab. -2. **Sync Documents:** +- You will be provided with a code snippet from nuclia which can be used for integration. -- Run the `sync` function to upload, update, or delete documents in Nuclia. +- To perform style changes to the widget you may have to use CSS or other alternatives. ### GitHub Actions -- **Workflow Setup:** - Modify the GitHub Actions workflow `nuclia_sync.yml` to match your repository structure. +Modify the GitHub Actions workflow `nuclia_sync.yml` to match your repository structure. ->## Usage Notes -> ->- Ensure that the API key and knowledge base URL are correctly configured. ->- Regularly update the sync process to keep the knowledge base up to date. ->- Troubleshoot issues by checking API key validity and document URLs. +## Usage notes -### Conclusion +- Ensure that the API key and knowledge box URL are correctly configured. +- Troubleshoot issues by checking the API key validity and document URLs. -Replacing Sphinx search with Nuclia search brings improved search functionality to Plone sites. Feel free to reach out for assistance or clarification on any aspect of this documentation. +## Conclusion +Replacing Sphinx search with Nuclia search brings improved search functionality to Plone sites. +Feel free to reach out for assistance or clarification on any aspect of this documentation. Happy syncing and searching!