Skip to content

Commit

Permalink
Merge pull request #384 from aws-solutions/develop
Browse files Browse the repository at this point in the history
Update to version v5.1.0
  • Loading branch information
aassadza authored Dec 20, 2021
2 parents c0f3827 + fad087c commit e3fb839
Show file tree
Hide file tree
Showing 142 changed files with 16,617 additions and 6,518 deletions.
20 changes: 16 additions & 4 deletions BotRoutingREADME.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Bot Routing - Preview Mode
(version 1.0 - December 2020)
(version 1.1 - November 2021)

Bots exist to perform a variety of automation tasks. Usually they take
as input a message from a human and respond performing
Expand Down Expand Up @@ -54,13 +54,25 @@ BotRouter for the question being edited.**
![Configuration](./docs/botroutingconfig.png)

* Bot Name or Lambda - You can configure and existing Lex Bot or configure
a specialty BotRouter implemented via a Lambda function.
a specialty BotRouter implemented via a Lambda function. QnABot route
requests to a Lex Version 1 or Version 2 bot. The syntax is different
depending on the target version. For Lex version 2 bots use the syntax
lexv2::BotId/BotAliasId/LocaleId. For Lex version 1 bots use just the
BotName. Note that QnABot can be installed as either Lex V1 or Lex V2
bot. It is important to check your QnABot configuration and select the
appropriate identifiers when you want to use QnABot as a specialty bot.

* Simple name - A short string that we expect web User Interfaces to use as
a breadcrumb to identify where in an enterprise the user is interacting.

* Lex Alias - If a target bot is configured, the alias used to communicate
with the target bot.
* Lex Alias - If your specialty bot is a Lex version 1 bot, specify the Lex version 1
alias used to communicate with the target bot.

* Lex session attributes to forward to the specialty bot. A comma separated
list of session attribute names can be specified. The session attributes
will be passed on each request to the specialty bot. They will override
any session attributes which might have been returned from the specialty bot
on the prior request.

The example image shows an integration we've developed which communicates
with the Nutritionix Bot.
Expand Down
25 changes: 25 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,31 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [5.1.0] - 2021-12-09

### Added
- Tags to questions in Content Designer and ability to create reports in Kibana.
- Integration with Genesys call center platform.
- Client Filtering with Session Attributes (i.e., Support to allow the same set of questions to be answered differently based on a session attribute).
- Intelligent redaction of Personally Identifiable Information in logs with Amazon Comprehend.
- A QnABot client (e.g. a Connect contact flow) can now optionally provide a value for session attribute, `qnabotUserId`. When this session attribute is set, QnABot tracks user activity based on the provided value. If not set, QnABot continues to track user activity based on the request userId (LexV1) or sessionId (LexV2). NOTE: `qnabotUserId` value is not used when user authentication using JWTs is enabled - in this case users are securely identified and verified from the JWT.
- Support for pre and post processing AWS Lambda Hooks.
- Setting that determines whether Amazon Kendra responses are abbreviated when sent via SSML (`ALT_SEARCH_KENDRA_ABBREVIATE_MESSAGE_FOR_SSML`)
- Setting that determines the types of responses that Amazon Kendra returns (`ALT_SEARCH_KENDRA_RESPONSE_TYPES`: `ANSWER`, `DOCUMENT`, `QUESTION_ANSWER`)

### Fixed
- Test tab in Content Designer to show same results as the web client when using Kendra FAQ.
- Broken link in documentation for downloading CloudFormation template.
- Integration with Slack on Amazon LexV2 bots.
- QnABot will set the sessionAttribute from CONNECT_NEXT_PROMPT_VARNAME to an empty string if QnABot is in a response bot in a voice channel. This will prevent QnABot from saying the next prompt in the middle of a response bot flow.
- Kendra FAQ sync where export Lambda was missing the Layer containing qnabot log.
- Bug with response bots with Alexa where QnABot was filling in a malformed reprompt.text in the response.
- Excel import improvements and bug fixes.

### Changed
- Bot routing capability to have multiple-bot architecture (e.g., General bot routing questions to specialty bots).
- The built-in Amazon Kendra “web page indexer” with the new Kendra Web Crawler Datasource.

## [5.0.1] - 2021-10-20

### Added
Expand Down
13 changes: 7 additions & 6 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,13 +1,14 @@
TEMPLATES=$(shell for l in $$(ls ./templates | grep -v util | grep -v README.md);do echo templates/$$l;done)
TEMPLATES=$(shell for l in $$(ls ./templates | egrep -v "util|lib|README.md");do echo templates/$$l;done)

All: assets templates lambda website make_directories

build: All

make_directories:
mkdir -p build; mkdir -p build/lambda; mkdir -p build/templates/test;mkdir -p build/templates;mkdir -p build/documents; mkdir -p build/templates/dev
mkdir -p build/lambda build/documents build/templates/test build/templates/dev

.PHONY: lambda templates upload website test bootstrap assets config.aws-solutions.json
.PHONY: $(TEMPLATES)

config.json:
node bin/config.js > config.json
Expand All @@ -21,10 +22,10 @@ lambda: make_directories
bootstrap: make_directories
$(MAKE) ../../build/templates/dev/bootstrap.json -C templates/dev

templates: $(TEMPLATES) make_directories
for l in $(TEMPLATES); do \
$(MAKE) -C $$l; \
done;
templates: $(TEMPLATES)

$(TEMPLATES): make_directories
$(MAKE) -C $@

website: make_directories
$(MAKE) -C ./website
Expand Down
2 changes: 0 additions & 2 deletions NOTICE.txt
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,6 @@ This software includes third party software subject to the following copyrights:
@babel/preset-stage-2 under the Massachusetts Institute of Technology (MIT) license
ajv under the Massachusetts Institute of Technology (MIT) license
alexa-sdk under the Apache License Version 2.0
amazon-cognito-identity-js under the Apache License Version 2.0
ask-cli under the Apache License Version 2.0
async-mutex under the Massachusetts Institute of Technology (MIT) license
autosize under the Massachusetts Institute of Technology (MIT) license
aws-lex-web-ui under the Amazon Software License
Expand Down
14 changes: 11 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,19 @@ This repository contains code for the QnABot, described in the AWS AI blog post

Refer to [Getting Started](#getting-started) to launch your own QnABot, or [Automated Deployment](https://docs.aws.amazon.com/solutions/latest/aws-qnabot/automated-deployment.html) for detailed implementation instructions.

**See all the new features list for 4.7.3** [Fulfillment Lambda Provisioned Concurrency](CHANGELOG.md#473---2021-08-04)
The 5.1.0 release of QnABot provides support for a new mechanism where a Question can set a Lex session attribute.

**New features in 4.6.0** [Improved Kendra integration and Kibana dashboards. Additional settings to filter Kendra responses based on confidence levels](CHANGELOG.md#460---2021-04-30).
In prior versions, the use of handlebars in an answer could set a session attribute. For example, the following

**New features in 4.5.0** [Kendra Web Crawler, Comprehend PII Detection, Translate Custom Terminology, Increased deployment regions](CHANGELOG.md#450---2021-03-07).
"{{setSessionAttr ‘attributeName' ‘attributeValue’}}""

will set the an attribute called “attributeName” to the value “attributeValue”. The drawback in this approach is that the attribute value could not itself contain nested handle bars.

In 5.1.0, a Question can now define a set of name/value pairs to set as session attributes when the answer is returned. You’ll see a new field to set a name/value pair plus an “Add” button to define additional name/value pairs and a button to delete a name/value pair. The Value field can contain handlebar statements as well.

The attribute name can be a simple name like “myAttribute” or a complex name like “myAttribute.subAttribute”. Using the dot notation an attribute several levels deep can be set. Note that one can overwrite reserved session attribute names. Avoid using “appContext” or “qnabotcontext” as attribute names. Setting these may have adverse effects on the system.

Note, the handlebar syntax still works just as it did before. The question configuration just has a new option.

## Architecture Overview
![Architecture](docs/architecture.png)
Expand Down
2 changes: 1 addition & 1 deletion bin/check.js
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ async function run(stack,options={}){
var Bucket=exp.Bucket
var prefix=exp.Prefix
var Key=`${prefix}/templates/${stack}.json`
var TemplateURL=`http://${Bucket}.s3.${region}.amazonaws.com/${Key}`
var TemplateURL=`https://${Bucket}.s3.${region}.amazonaws.com/${Key}`
console.log(TemplateURL)
await s3.putObject({Bucket,Key,Body:template}).promise()
return cf.validateTemplate({TemplateURL}).promise()
Expand Down
2 changes: 1 addition & 1 deletion bin/json.js
Original file line number Diff line number Diff line change
Expand Up @@ -261,7 +261,7 @@ function printHelp() {
w(' -4 => -o jsony-4');
w(' -j0 => -o json-0');
w('');
w('See <http://trentm.com/json> for more docs and ');
w('See <https://trentm.com/json> for more docs and ');
w('<https://github.com/trentm/json> for project details.');
/* END JSSTYLED */
}
Expand Down
4 changes: 2 additions & 2 deletions bin/launch.js
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,7 @@ async function up(stack,options){
var exp=await bootstrap()
var bucket=exp.Bucket
var prefix=exp.Prefix
var url=`http://${bucket}.s3.${region}.amazonaws.com/${prefix}/templates/${stack}.json`
var url=`https://${bucket}.s3.${region}.amazonaws.com/${prefix}/templates/${stack}.json`
await s3.putObject({
Bucket:bucket,
Key:`${prefix}/templates/${stack}.json`,
Expand Down Expand Up @@ -171,7 +171,7 @@ function update(stack,options){
var start=bootstrap().then(function(exp){
var bucket=exp.Bucket
var prefix=exp.Prefix
var url=`http://${bucket}.s3.${region}.amazonaws.com/${prefix}/templates/${stack}.json`
var url=`https://${bucket}.s3.${region}.amazonaws.com/${prefix}/templates/${stack}.json`
console.log(url)
return s3.putObject({
Bucket:bucket,
Expand Down
26 changes: 14 additions & 12 deletions docs/PII_Detection/README.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,18 @@
# Personally Identifiable Information Rejection
# Personally Identifiable Information Rejection and redaction

QnABot can now detect Personally Identifiable Information (PII) using [Amazon Comprehend](https://docs.aws.amazon.com/comprehend/latest/dg/how-pii.html) and regular expressions
QnABot can now detect Personally Identifiable Information (PII) using [Amazon Comprehend](https://docs.aws.amazon.com/comprehend/latest/dg/how-pii.html) and regular expressions.

If ENABLE_REDACTION is set to "true", the Comprehend detected PII entities will also be redacted from Amazon CloudWatch logs and Amazon Opensearch logs.

![](./settings.png)


Setting | Description |
---------|----------|
PII_REJECTION_ENABLED | set to "true" to enable pii rejection |
PII_REJECTION_QUESTION | If PII is found, the user's request (question) will change to this phrase|
PII_REJECTION_WITH_COMPREHEND | Enable's Personal Identifiable Information detection with Amazon Comprehend |
PII_REJECTION_REGEX | Used to find PII based on a regular expression |
PII_REJECTION_IGNORE_TYPES | Do not detect the specified list of [entity types](https://aws.amazon.com/blogs/machine-learning/detecting-and-redacting-pii-using-amazon-comprehend/) |
![settings image](./settings.png)

|Setting | Type of Value | Description |
--------|---------------|-------------|
| ENABLE_REDACTING | true or false | Enable the system to redact log output
| REDACTING_REGEX | regex expression | Redacts expressions matching regex from logs
| PII_REJECTION_ENABLED | true or false | Enables PII Rejection
| PII_REJECTION_QUESTION | text | If PII is found, the user's request (question) will change to this phrase
| PII_REJECTION_WITH_COMPREHEND: | true or false | Enable's [Personal Identifiable Information](https://aws.amazon.com/blogs/machine-learning/detecting-and-redacting-pii-using-amazon-comprehend/) detection with Amazon Comprehend
| PII_REJECTION_REGEX | regex expression | Used to find PII based on a regex
| PII_REJECTION_ENTITY_TYPES | comma separated list of [PII Entity Categories](https://aws.amazon.com/blogs/machine-learning/detecting-and-redacting-pii-using-amazon-comprehend/) | Only recognize PII entity types in the list
| DISABLE_CLOUDWATCH_LOGGING | true or false | Disable all logging in fulfillment es query handler lambda. does not disable logging from Lambda Hooks or Conditional Chaining Lambda functions
Binary file modified docs/PII_Detection/settings.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/botroutingconfig.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
23 changes: 23 additions & 0 deletions docs/excel_import/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,29 @@ Excel Column | Content Designer Field| Description
displaytext1...displaytext(n) | Display Text | The text of the button. Each button text should be in a separate column numbered button1, button2, ...
buttonvalue...buttonvalue(n) | Button Value| The value of the button. Each button value should be in a separate column numbered button1, button2, ...

## Support for other Content Designer fields

While special affordances have been made to allow importing the fields above, any "string" field in the [Content Designer schema](../../lambda/schema/qna.js)
can be imported by specifying the JSON path as the column.

### Example of a field on the root level

![Schema snippet](schema2.png)

Excel Column | Content Designer Field
-------------|-----------------------
clientFilterValues | Client Filter: Values


### Examples of nested fields

![Schema snippet](schema.png)

Excel Column | Content Designer Field (title in the schema.json)
-------------|------------------------
elicitResponse.response_hook | Elicit Response: Responsebot Hook
elicitResponse.response_sessionattr_namespace | Response Session Attribute Namespace

## Importing the Microsoft Excel file

Choose *Import* from the Menu and then choose *From File*
Expand Down
Binary file modified docs/excel_import/sample.xlsx
Binary file not shown.
Binary file added docs/excel_import/schema.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/excel_import/schema2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/kendra_crawler_guide/IndexDialog.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/kendra_crawler_guide/NoIndexDialog.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
34 changes: 22 additions & 12 deletions docs/kendra_crawler_guide/README.md
Original file line number Diff line number Diff line change
@@ -1,29 +1,39 @@
# Kendra Web Indexer Integration
# Amazon Kendra Web Crawler Integration

Please read the [Kendra Integration](../../workshops/kendra/README.md) for instructions about configuring Kendra
Please read the [Amazon Kendra Integration](../../workshops/kendra/README.md) for instructions about configuring Kendra

QnABot can now index your website with Kendra and answer questions based on what it has found.
QnABot can now crawl your website with Kendra and answer questions based on what it has found.

Go to the Settings option in the Tools menu

![Setting](./settings.png)

1. Enable the indexer by setting _ENABLE_WEB_PAGE_INDEXER_ to true
1. Tell QnABot which web pages to index by specifying a comma separated list of addresses in _KENDRA_INDEXER_URLS_
1. The indexer can be optionally configured to run on a schedule by setting the _KENDRA_INDEXER_SCHEDULE_
It supports standard [CloudWatch Events rate syntax](https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/ScheduledEvents.html#RateExpressions)
1. Tell QnABot which Kendra index to use by adding your Kendra IndexId to the setting _KENDRA_WEB_PAGE_INDEX_
The built in "Web Page Indexer" has been replaced with Kendra's native web crawler data source.
The settings names have been maintained for compatibility.

After you save your settings, go back to the Tools Menu and then choose **Kendra Web Page Indexing**
| Setting | Valid Values | Description |
|---------|--------------|-------------|
| ENABLE_KENDRA_WEB_INDEXER | true or false | Enables the web crawler
| KENDRA_INDEXER_URLS | comma separated list | List of web addresses QnABot should crawl and [index with Kendra](./kendra_crawler_guide/README.md)
| KENDRA_INDEXER_SCHEDULE | [CloudWatch Rate Syntax](https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/ScheduledEvents.html) | Interval Indexer should crawl
| KENDRA_INDEXER_CRAWL_DEPTH | number | Sets the depth to the number of levels in a website from the seed level that you want to crawl
| KENDRA_INDEXER_CRAWL_MODE | HOST_ONLY \| SUBDOMAINS \| EVERYTHING | Determines which addresses should be crawled
| KENDRA_WEB_PAGE_INDEX | Kendra Index Id | The index to use for the web crawler, a [custom data source](https://docs.aws.amazon.com/kendra/latest/dg/data-source-custom.html) will automatically be added to the specified index.

![Tools](./tools.png)
After you save your settings, go back to the Tools Menu and then choose **Kendra Web Page Crawler**

![No Index Dialog](./NoIndexDialog.png)
![Tools](./tools.png)

If your settings are correct, the first time you choose the option, you should see the following dialog.
![No Index Dialog](./NoIndexDialog.png)

Just press **Start Indexing** to index your web pages to Kendra.
Just press **Start Crawling** to index your web pages to Amazon Kendra.

The Content Designer will show you a history of your indexing.

![Index Dialog](./IndexDialog.png)

If you are logged into your AWS Account, you can click on "View Web Crawling Errors in CloudWatch" to get the details of
any pages that Amazon Kendra couldn't crawl.

![CloudWatch](cloudwatch.png)
Binary file added docs/kendra_crawler_guide/cloudwatch.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/kendra_crawler_guide/settings.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/kendra_crawler_guide/tools.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading

0 comments on commit e3fb839

Please sign in to comment.