Merge pull request #412 from aws-solutions/develop

 Update to version v5.1.1

.eslintignore

@@ -0,0 +1,11 @@
+node_modules
+tmp
+# don't lint build output (make sure it's set to your correct build folder name)
+dist
+# don't lint test folders
+test
+# don't lint coverage output
+coverage
+*.config.js
+# dont lint eslint config
+*.eslint*

.eslintrc.js

@@ -0,0 +1,14 @@
+module.exports = {
+    root: true,
+    parserOptions: {
+        ecmaVersion: 2021
+    },
+    env: {
+        node: true
+    },
+    extends: ['eslint:recommended'],
+    rules: {
+        indent: ['warn', 4],
+        quotes: ['warn', 'single']
+    }
+};

.prettierignore

@@ -0,0 +1,2 @@
+node_modules
+tmp

.prettierrc.yml

@@ -0,0 +1,8 @@
+# .prettierrc or .prettierrc.yaml
+proseWrap: 'preserve'
+trailingComma: 'none'
+tabWidth: 4
+semi: true
+singleQuote: true
+quoteProps: 'preserve'
+printWidth: 120

.vscode/spellright.dict

@@ -40,3 +40,11 @@ cardtitle
 cardimageurl
 buttonvalue
 py_lambda_hooks
+ERRORMESSAGE
+qna
+qnabot
+postprocessing
+py
+js
+nodejs
+Opensearch

CHANGELOG.md

@@ -5,6 +5,21 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+
+## [5.1.1] - 2022-02-04
+
+### Added
+- Expanded language support for voice and text interactions. Also included support for Neural voices for Lex language locales. See [supported languages](docs/multilanguage_support.md#supported-languages).
+- Expanded `config.json` to support `LexV2BotLocaleIds` parameter.
+- Updated `LexV2BotLocaleIds` parameter in CloudFormation template to include link to supported languages.
+- Updated [Multi Language Support readme](docs/multilanguage_support.md#supported-languages) and added supported languages section.
+- Updated `ENABLE_MULTI_LANGUAGE_SUPPORT` setting in [Settings readme](docs/settings.md) with link to supported languages.
+
+### Fixed
+- Fixed Kendra Webcrawler data source sync issue by adding support to check for Kendra webcrawler data source {status} before initiating sync
+- Fixed issue where the Kendra Webcrawler data source {settings} page in Kendra console was resulting in a blank page - by updating configuraton parameters for data source create and update steps
+
+
 ## [5.1.0] - 2021-12-09
 
 ### Added

assets/default-utterances.json

@@ -37,7 +37,6 @@
 "what is normal blood pressure",
 "how many people live in Ireland",
 "how do I measure my volume",
-"what car is the fastest from zero to sixty",
 "tell me the benefits of green vegetables",
 "when should I call emergency services",
 "where is the nearest computer repair store",

bin/config.js

@@ -11,6 +11,7 @@ module.exports={
     "devPublicOrPrivate": "PRIVATE",
     "namespace":"dev",
     "LexBotVersion":"LexV2 Only",
+    "LexV2BotLocaleIds": "en_US,es_US,fr_CA",
     "stackNamePrefix":"QNA",
     "skipCheckTemplate": false,
     "noStackOutput": false,

docs/PII_Detection/README.md → docs/PII_Detection_And_Redaction/README.md

@@ -1,6 +1,6 @@
 # Personally Identifiable Information Rejection and redaction
 
-QnABot can now detect Personally Identifiable Information (PII) using [Amazon Comprehend](https://docs.aws.amazon.com/comprehend/latest/dg/how-pii.html) and regular expressions.
+QnABot can now detect and redact Personally Identifiable Information (PII) using [Amazon Comprehend](https://docs.aws.amazon.com/comprehend/latest/dg/how-pii.html) and regular expressions.
 
 If ENABLE_REDACTION is set to "true", the Comprehend detected PII entities will also be redacted from Amazon CloudWatch logs and Amazon Opensearch logs.
 
@@ -10,9 +10,12 @@ If ENABLE_REDACTION is set to "true", the Comprehend detected PII entities will
 --------|---------------|-------------|
 | ENABLE_REDACTING | true or false | Enable the system to redact log output
 | REDACTING_REGEX | regex expression | Redacts expressions matching regex from logs
+| ENABLE_REDACTING_WITH_COMPREHEND | true or false | Enables [Amazon Comprehend based PII Redacting](https://aws.amazon.com/blogs/machine-learning/detecting-and-redacting-pii-using-amazon-comprehend/)
+| COMPREHEND_REDACTING_CONFIDENCE_SCORE | number (0 to 0.99) | Only redact PII where Amazon Comprehend's confidence score is greater than this number
+| COMPREHEND_REDACTING_ENTITY_TYPES | comma separated list of [PII Entity Categories](https://aws.amazon.com/blogs/machine-learning/detecting-and-redacting-pii-using-amazon-comprehend/) | Only recognize PII entity types in the list for redaction
 | PII_REJECTION_ENABLED | true or false | Enables PII Rejection
 | PII_REJECTION_QUESTION | text  | If PII is found, the user's request (question) will change to this phrase
-| PII_REJECTION_WITH_COMPREHEND: | true or false | Enable's [Personal Identifiable Information](https://aws.amazon.com/blogs/machine-learning/detecting-and-redacting-pii-using-amazon-comprehend/) detection with Amazon Comprehend
+| PII_REJECTION_CONFIDENCE_SCORE | number (0 to 0.99) | Only reject PII where Amazon Comprehend's confidence score is greater than this number
 | PII_REJECTION_REGEX | regex expression | Used to find PII based on a regex
 | PII_REJECTION_ENTITY_TYPES | comma separated list of [PII Entity Categories](https://aws.amazon.com/blogs/machine-learning/detecting-and-redacting-pii-using-amazon-comprehend/) | Only recognize PII entity types in the list
 | DISABLE_CLOUDWATCH_LOGGING | true or false | Disable all logging in fulfillment es query handler lambda. does not disable logging from Lambda Hooks or Conditional Chaining Lambda functions

docs/client_filters.md

@@ -0,0 +1,5 @@
+# Client Filters
+
+Client filtering allows you to set a parameter for each question stored in QnABot. The answer to a question with a specific client filter will only be returned if the associated client filter value is sent in with the query. This will allow you to store the same question multiple times in the same QnABot instance and control which answer is returned based on the client filter parameter you send in.
+
+For more information, watch [QnABot Client Filter and Amazon Connect Live Chat](https://www.youtube.com/watch?app=desktop&v=M_V-3NkMhfE&ab_channel=AmazonWebServices)

docs/excel_import/README.md

@@ -1,6 +1,6 @@
 # Excel Workbook Import
 
-QnABot now supports importing questions and answers from Microsoft Excel (xlsx) workbooks.
+QnABot supports importing questions and answers from Microsoft Excel (xlsx) workbooks.
 
 A [sample](./sample.xlsx) Excel workbook is included. Each column in the Excel sheet maps to a field in the Content Designer.
 

docs/kendra_crawler_guide/README.md

@@ -2,7 +2,7 @@
 
 Please read the [Amazon Kendra Integration](../../workshops/kendra/README.md) for instructions about configuring Kendra
 
-QnABot can now crawl your website with Kendra and answer questions based on what it has found.
+QnABot can crawl your website with Kendra and answer questions based on what it has found.
 
 Go to the Settings option in the Tools menu
 

docs/lambda_hooks/README.md

@@ -0,0 +1,70 @@
+# Extending QnABot with Lambda hook functions
+
+Content Designer gives you the ability to dynamically generate answers by letting you specify your own Lambda ‘hook’ function for any item. When you specify the name, or ARN, of a Lambda function in the Lambda Hook field for an item, QnABot will call your function any time that item is matched to a user’s question. Your Lambda function can run code to integrate with other services, perform actions, and generate dynamic answers.
+
+QnABot comes with a simple Lambda Hook function example that you can explore:
+
+1. Log in to the Content Designer, and choose **Import** from the tools menu ( ☰ ).
+2. Open Examples/Extensions, and choose **LOAD** from the ‘GreetingHook’ example.
+
+![Greeting Hook Example](./images/GreetingHookExample.jpg)
+3. When the import job has completed, return to the edit page, and examine the item “GreetingHookExample.” Note that the Lambda Hook field is populated with a Lambda function name.
+4. Use the Web UI to say “_What are lambda hooks?_”. Note that the answer is prepended with a dynamic greeting based on the current time of day – in this case ‘_good afternoon_’:
+    ![Greeting Hook Web UI Example](./images/WebUiGreetingHook.jpg)
+5. Inspect the example function (ExampleJSLambdahook) using the [AWS Lambda console](https://console.aws.amazon.com/lambda/home?region=us-east-1#/functions/qna-QnABot-hello?tab=graph).
+
+Choose **Lambda Hooks** from the Content Designer tools menu ( **☰**) to display additional information to help you create your own Lambda hook functions.
+
+---------
+
+The Lambda hook receives the current QnABot response in the `res` attribute of the input event of your Lambda handler.
+You can override values from this event in your Lambda hook code.
+For example, you can pass a dynamically retrieved answer by overriding the value of the incoming `event.res.message` attribute from the incoming event.
+You can then send the modified response in the Lambda return value.
+Here is a simplistic code snippet:
+
+```python
+def handler(event, context):
+    event['res']['message']="Hi! This is your Custom Python Hook speaking!"
+    return event
+```
+
+## Adding your own Lambda hooks from the repository
+
+1. Navigate to the `aws-ai-qna-bot/templates/examples/extensions/` folder in your local repository.
+2. Create javascript (node.js) lambda hooks under `./js_lambda_hooks`.
+3. Create python3 lambda hooks under `./py_lambda_hooks`.
+4. Create a new subdirectory for your new lambda hook, e.g MyLambdaHook
+5. Create a code file with your lambda function source code in the new subdirectory:
+
+    - the file name should be the same as your directory name, with .py or .js suffix as appropriate, eg MyLambdaHook.py
+    - the code must contain a lambda function named 'handler', and handle standard lambda parameters and returns.
+    - if your lambda code relies on any packages not provided by Lambda, you can bundle these with your functions by creating a requirements.txt (python) or a package.json (javascript) file in the same directory. During the build process the listed packages will be downloaded and packaged/installed with your function.
+
+## [](#reference-your-lambda-hooks-from-content-designer)Reference your Lambda hooks from Content Designer
+
+In Content Designer, use the following syntax to reference your Lambda hook function:
+
+- QNA:EXT*LambdaHookName (e.g _QNA:EXTMyLambdaHook_)
+
+The ARN of your installed lambda hook will be referenced at runtime by the QnABot Fulfillment function using environment variables. I.e. the fulfillment function is set up (during installation) with environment variable `EXTMyLambdaHook` and the value is the ARN of your installed function.
+Using the environment variable indirection is preferable to using your function ARN, since you can maintain separate function instances for different QnABot stacks / environments, and you can easily export/import content that does not contain ARN references to specific function instances in specific accounts and regions.
+
+## [](#add-new-content-packages-for-content-designer-import-examplesextensions-listing)Add new content packages for Content Designer Import Examples/Extensions listing
+
+Add importable content packages in the `./ui_imports/content` folder using two files as follows:
+
+- \<name>.json -- the JSON representation of the QnA documents to be imported (can be a file that was previous exported from Content Designer.
+- \<name>.txt -- a short tagline description of the content that will be displayed in the Content Designer listing.
+
+## Preprocessing and postprocessing Lambda hooks
+
+You can also add Lambda hooks globally that run before (preprocessing) and after every question is run via the settings page.
+
+![settings hooks](./images/pre_post_hook.png)
+
+## [](#notes)NOTES
+
+- The `Makefile` residing in the extensions folder creates separate zip packages for each separate Lambda hook function
+- Lambda hook functions use nodejs12.x or python3.6 only at this time
+- Lambda hook functions will be allocated 2048MB memory (defined in index.js)