All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog, markdownlint, and this project adheres to Semantic Versioning.
The entity-search-webapp
now REQUIRES the use of the senzing-poc-server to provide it's senzing api access. This is a change from before where you could use either the senzing-api-server OR the senzing-poc-server.
- Dependency updates.
- Searches can now survive a page reload
- Route Resolvers changed from Class to Fn
- New Landing page
- Donut widget displaying how many records have been loader for each datasource.
- License widget displays how much of your current license is being used and it's expiration date.
- Cross Source Comparison widget. Allows selection of a data source or two and selection of the type of matches or relations for sample browsing.
- Sample browsing Data Table widget. Is populated from the user's selection from the Cross Source Comparison widget and allows the user to browse through a sample-set that matches the parameters selected.
- Removed 'SSO' integration variables. SSO authententication/redirects should be handled at the network/formation level instead of by the individual container.
- Dependency updates.
- Create arm64 Docker image
- In
Dockerfile
, updated FROM instruction to:node:16-bullseye-slim@sha256:503446c15c6236291222f8192513c2eb56a02a8949cbadf4fe78cce19815c734
node:16-alpine3.18@sha256:a1f9d027912b58a7c75be7716c97cfbc6d3099f3a97ed84aa490be9dee20e787
- new graph icons.
- match keys are now shown by default in the graph.
- new why and why not report column formatting to match g2Explorer.
- @senzing/sdk-components-ng updated to 6.1.0
- "How" entity report component(s) added.
- Added
SzHowEntityComponent
component which is the only component that should be used at an implementation level. The other components listed below are components that this component uses to create things like different color cards, nested trees, navigation, groups etc.
- Added
- "How" entity report component(s) added.
- Added new
/how/${entityId}
route to application to display a how report for a specific entity. - Update dependencies to latest compatible versions
- In
Dockerfile
, updated FROM instruction to:node:16-bullseye-slim@sha256:726698a073f984efd26cb31c176a35b39200c4a82f4dc6f933c7cc957403b567
node:16-alpine3.17@sha256:fb4c5fcefe7cf706ab7f9eed97258641f33ca318b03f072119180133ab3ef334
- Update dependencies
- socket.io-parser:4.2.3
- In
Dockerfile
, updated FROM instruction to:node:16-bullseye-slim@sha256:7ee0b958bd5f47f54b58f2b9932b1975a4d98d8f332bd2134c2b65514cadb6c6
node:16-alpine3.17@sha256:f1657204d3463bce763cefa5b25e48c28af6fe0cdb0f68b354f0f8225ef61be7
- Update dependencies
- http-cache-semantics:4.1.1
- node_modules/http-cache-semantics:4.1.1
- node_modules/ua-parser-js:0.7.33
- ua-parser-js:0.7.33
- Angular framework updated to @15.x.x
- Material toolkit updated to @15.x.x
- D3 charting toolkit updated to @7.x.x
@senzing/rest-api-client-ng
updated to @6.0.0@senzing/sdk-components-ng
updated to @6.0.0- Dependency Securtity updates (various)
- In
Dockerfile
, updated FROM instruction to:node:16-bullseye-slim@sha256:cfb2b5e2b39f341056ac624d32fae00ba0ab94145364111b7edfd9db703526e0
node:16-alpine3.16@sha256:e023c53915c5e20df594809477dc249dd595a8e731a6e214c4dbdcf7431bf942
- In
Dockerfile
, updated FROM instruction to:node:16-bullseye-slim@sha256:5c9f79e11b4f867582241b5e7db96bbe1893fad8c1f523261690c743d0950667
node:16-alpine3.16@sha256:f16544bc93cf1a36d213c8e2efecf682e9f4df28429a629a37aaf38ecfc25cf4
- In
Dockerfile
, updated FROM instruction to:node:16-bullseye-slim@sha256:18ae6567b623f8c1caada3fefcc8746f8e84ad5c832abd909e129f6b13df25b4
node:16-alpine3.15@sha256:a60b681e1c28f60ea63f8394dea5384c69bdc464b9655e880f74aafaa5945665
- added new radio switch for graph match key filtering scope
- added "unlimited" UI options for buildOut and maxEntities graph filtering parameters
- bugfixes for large graph match key filtering
- bugfixes for large graph FOC
- graph match key filters now de-select on entity id(s) change
- bugfixes for graph lifecycle progress/busy indicators.
relevant tickets: #299 #321 #302 #324
This update adds the long-awaited Expand/Collapse graph node edge feature. Now nodes in the graph will display a bubble with the number of relationships not currently visible. Relationships can be explored by simply expanding relationships down a path. The Why Not report can now be run from specific integration points. The why not report will run a side-by-side table of two entities information in order to help illustrate why two entities did not come together.
- New graph features:
- On clicking the bubble the hidden related entities will be drawn around the node.
- On clicking the expand/collapse bubble again the related nodes that are not related to any other nodes on the canvas will be hidden.
- On right clicking the node there is a new option to hide just that specific entity from view(to eliminate clutter)
- On right clicking the relationship line match key there will be a menu option for running a Why NOT report between the two connected entities
- There is a new Multi-Select mode button in the upper right-corner of search results. Once the button is clicked a user can click on two search results and click on "Compare" to bring up a "Why NOT" report for why those two entities did not come together.
- Match keys are now displayed in the search result cards by default. (they can be turned off through preferences)
This update brings the models in line with the changes for the 3.0.0
release of the
senzing rest api server
and rest api specification.
- "Filter By Match Key" added back to the large graph display using a more user-friendly approach of a _"Tag Cloud" of match key tokens.
- Why Entity feature that adds a button to the entity detail page that opens a draggable/resizable window with a horizontal side-by-side view of why records in an entity came together
- Why Entity feature that adds a button to the entity detail page's record rows that opens a draggable/resizable window with a horizontal side-by-side view of why that particular record in an entity came together
- Major refactoring done under the hood inside the admin area and data importing functionality due to the removal of entity types/classes.
@senzing/rest-api-client-ng
updated to4.0.0
@senzing/sdk-graph-components
updated to4.1.0-beta.1
@senzing/sdk-components-ng
updated to4.0.0-beta.7
- UI fixes (various)
- Any and all functionality surrounding Entity Type's and Entity Classes. Management of these features has been removed in the 3.0 version of the senzing sdk.
relevant tickets: #265 #263 #261 #258
- dependency updates to resolve security bulletins
- bugfix for json/json-lines analyze functions switched from
streamAnalyzeByRecords
tostreamAnalyzeByChunks
- bugfix for json/json-lines loading functions switched from
streamLoadByRecords
tostreamLoadByChunks
- Optional Web Console Access to the senzing-web-app-console container. Users can now run popular shell tools like G2Explorer.py, G2Snapshot.py and G2Audit.py from within the context of the container through the web interface.
- "Filter By Match Key" removed from the large graph display due to usability issues on large datasets. (It will return once a better approach can be implemented)
relevant tickets: #242 #243 #246 #252 #256
/health
route that returns a model shape of{"isProxyAlive": true, "isApiServerAlive": true, "isWebserverAlive": true}
/health/proxy
route that returns the status of the proxy- HealthCheckerUtility that provides state management for health status checks.
- Angular Framework Updated to version 13. As of 3.0.0 release this package now requires
@angular@~13.0.0
to compile and run. Updating to angular 13 resolves major dependency compatibility issues so we can apply the latest security patches. - Small bugfix to properly clip the graph scale ui control when graph section in entity detail is collapsed.
- All dependencies updated to latest versions
- Docker container now based off of
lts-alpine
(nodejs@16) instead of14-alpine
(nodejs@14) - Docker container now gets
npm
itself deleted from container after build phase since it is not needed for runtime to minimize security exposure. - Dockerfile updated to wire HEALTHCHECK's to the new /health/check script
relevant tickets: #230 #231 #235 #195
- The ability to filter graph elements by "match keys" present on the links between graph entities. #215
- The ability to import CSV files when using SQS stream loading. #217
- Zoom and Pan controls added to embedded and standalone Network Graph components. #216
- Ability to filter by Match Keys present on the links between graph entities in the dataset added to standalone Network Graph component. #215
- List of ALL command line flags and environment variables for configuration of the docker container. See docs/Environment Variables #218
- "Search By Record Id" now returns full entity NOT just "record". #220
- bugfix for #222. introduced by #215
- bugfix for #226. introduced by #220
- dependencies updated
relevant tickets #226 #222 #220 #218 #216 #217 #215
- Added multi-stage build to
Dockerfile
so that only production dependencies and compiled code is present in immutable container. - Removed Google Chrome from container (was used for container e2e testing)
- Removed
travis.yml
from project. - Added test runner(s) back via Github Action workflows.
- removed the re-mapping UI interaction in admin/load when an import contains explicitly specified datasources on the record level. If the record contains an explicit datasource that is what the record will be loaded in to. If the datasource specified does not exist it will be created. If the file contains records that have NO datasource specified the user will be prompted to enter one before load.
- removed the UI interaction in admin/load around custom entity types.
Now if a imported file specifies a specific entity type that does not exist it is created.
If no entity type is specified it is automatically assumed
GENERIC
. - Added a "Admin" menu option in the upper-left site menu WHEN the poc or api server has "-adminEnabled" set.
- bugfixes related to stream loading (various, see relevant tickets)
relevant tickets #189 #190 #192 #196 #198 #200 #202 #204
The configuration options/setup surrounding the stream loading feature released in 2.3.0 has been simplified. Now stream loading is automatically enabled for the user IF the webapp is running against a POC Server that has SQS configured. If not configured properly or running against the API server(with adminMode=true set) the loading feature will fallback to the non-stream method. As a result of this streamlining the UI toggle switch and "connection configuration" modal has been removed from the UI.
The following ENV vars introduced in 2.3.0
have been removed:
SENZING_STREAM_SERVER_URL
SENZING_STREAM_SERVER_PORT
SENZING_STREAM_SERVER_DEST_URL
SENZING_WEB_SERVER_CSP_STREAM_SERVER_URL
The following ENV var has been introduced:
SENZING_WEB_SERVER_URL
- The fully qualified url to the root of where the webapp is being served from. This is used to define in the webapp security policy that outbound socket connections are allowed to this address.http://my.public.domain/webapp
http://my.specific.domain:specificport
The following cmdline args have been introduced:
webServerUrl
- see ENV varSENZING_WEB_SERVER_URL
description
relevant tickets #180 #183 #185
This release is primarily to support stream loading which enables stream ingestion through a web interface to the POC Server which hands off to the configured SQS provider. The stream interface is capable of reading records in a file and sending them out in small chunked batches either as quickly as they can be read or a fast as configured upload rate.
These features should be cross compatible with the standard API Server backend. If the stack is set up with SQS ingestion and the backend is the POC Server then stream loading will be available. If the backend is the standard API server then stream loading features will be hidden from the UI.
relevant tickets: #143 #179
- support for runtime configuration of API path that client connects to through env vars and cmdline switches.
- bugfix for virtual directories
- dependency security updates
- docker image base is now
lts-buster-slim
relevant tickets: #163 #166 #173
- support for virtual directories
- support for AWS Cloud Formation
- webfonts now referenced locally
- dependency security updates
- bugfix for download as pdf functionality
relevant tickets: #147 #154, #156, #157
- updated @senzing/sdk-graph-components to 2.1.3
- updated @senzing/sdk-components-ng to 2.2.1
- fixed bug where entity detail would not render when entity had 0 relationships.
- updated senzing libs to 2.2.0
- bug in route resolver to only accept numbers for record paths fixed
- Select Identifiers in Search Form feature added.
- updated senzing libs to 2.2.0
- bug in route resolver to only accept numbers for record paths fixed
Bugfixes for:
- Large graph filtering
- Large graph color highlights by datasource
- Detail Graph lifecycle issue on in-component navigation
- Graph tooltips showing null for address and phone numbers
- Enitity Id added to detail report
- Search by Attribute option pinned to menu options
- Best name logic fix
Compatibility release for framework upgrade to Angular 10: See https://blog.angular.io/version-10-of-angular-now-available-78960babd41
Major updates to most dependency versions have also been made which should improve file sizes, security, and stability.
The following Senzing sponsored projects have also been updated to operate on Angular 10, see the following links for associated tickets:
The scripts no longer write any configuration or runtime configuration information to the filesystem. Instead, the options are read from either ENV vars or Command Line arguements in to a data store class, which is then accessed by scripts and endpoints that need access to these arguements. This was done to support immutable containers. See Ensure that your containers are stateless and immutable.
Added:
- run/runtime.datastore.js
- run/runtime.datastore.config.js
- src/app/services/config.service.ts
- Interactive TTY "type any key to quit" web server prompt
Removed:
- proxy.conf.tmpl.json
- auth/auth.conf.json
- auth/auth.conf.tmpl.admin.json
- auth/auth.conf.tmpl.full.json
- auth/auth.conf.tmpl.json
- auth/cors.conf.tmpl.json
- any direct references to static files ie: /auth/auth.conf.json in ts files
Moved:
- auth-server.js → run/authserver/index.js
- auth/auth.js → run/authserver/auth.js
- webserver.js → run/webserver/index.js
There are several NEW env vars/command args that now are required for full operation, your docker formations or scripts should be updated.
Compatibility release for interacting with the 2.0.0 senzing-rest-api-spec and senzing-api-server. For information on specifics of endpoint changes see below:
- Show version information (for diagnostics)
- Open search results directly in Graph view
- Allow the user to collapse/expand the filters/detail drawer in the Graph view.
- Added MSSQL support
- Added Search by record Id or entity Id form interfaces
- Added Record JSON viewer
- Added Admin functionality. (see Readme for more details)
- added ability to view datasources available to an instance
- added ability to do bulk import or csv or json files in to a datasource
- added randomly generated JWT token to
/admin
routes by default. other authentication options include external SSO relay and NONE.
- Implemented CORS support for allowing the container to tell the client to directly request api resources. Default is disabled.
- Implemented CSP (Content Security Policy) - enabled by default. see #96 for more details.
- relevant tickets: #66, #67, #68, #73, #74, #76, #83, #84, #89, #96
- Standalone graph.
- graph filtering.
- embedded rail-format entity detail viewer (for graph)
- search result(s) directly displayed in standalone graph.
- graph color(s) by datasource membership.
- SSL support for docker image.
- basic auth support (alpha) for docker image.
- premature loading indicator disapearing fix.
- relevant tickets: #34, #67, #69, #71
- sdk-components-ng updated to 1.0.9 and sdk-graph-components updated to 0.0.6
- Preferences UI interface added to top ribbon
- Added SzPrefsService
- Added SzPreferencesComponent
- CSS for responsive breakpoint(s) and/or reflow on narrow width
- Various UI/UX layout bugfixes
- Graph should reload on entityIdChange
- Graph should collapse on 0 results
- Include "other data" in records area.
- Text highlighting no longer triggers click-thru
- Search identifiers drop-down should auto-update on api config change
- identifier dropdown issue
- proxy.conf perm issue
- entity icon fixes
- graph overflow issue
- relevant tickets: #32, #36, #47, #51, #53
- e2e tooling
- wonky graph styling issue
- basic unit tests fixes
- sdk-components-ng version bump. 1.0.4 -> 1.0.5
- adds licence badge to readme.
- adds
npm run e2e
,npm run e2e:headless
,npm run test
, andnpm run test:headless
scripts - added .travis.yml for CI build/test integration
- angular.json configuration changes to support path refactoring
- Initial release of the entity search web app.