This log documents the significant changes for each release. This project follows Semantic Versioning.
- Checking for an issue with the "not" modifier during initialization might not work correctly. The problem appears when a patient for the very first Observation in the list only has Observations with the same single code.
- Updating the selected observation codes and data type when changing a variable name was performed for all variable values in the advanced criteria, even if they were not connected. This resulted in incorrect display of units of measurement for variable values.
- Updated Angular Material to version 17.
- An issue where part of a table was not visible when scrolling after it was expanded to full screen.
- Removed deprecated RouterTestingModule.
- Replaced legacy Angular Material components.
- Passing a unit system when filtering by observation values. Previously saved criteria work as they worked before (but the issue of not being able to pass the unit system for an observation value persists because they do not contain units of measurement).
- Exception when selecting variable values in the cart using autocomplete.
- An issue with updating the current value of autocompleter when changing the search parameter name (without breaking the loading of saved criteria).
- Added loading possible search parameter values for value sets with the required binding strength from the FHIR server. This allows us to display only existing values and fixes the fallback used to load values from the server when we can't get them from spec definitions.
- Updated Angular to version 17.
- Updated Angular Material to version 16.
- Styles for autocomplete fields in the cart.
- Update autocomplete-lhc version and disable its progress bar.
- Updated Cypress to version 13.
- An input field to set "max recent Observations per Patient to check" for EvidenceVariables.
- "Id", "Status", "Publisher", and "Patient Id" columns to the EvidenceVariable table.
- The option to show only unique EvidenceVariables.
- Removing a resource type tab in the pull data step did not stop the loading process on this tab or clear the already loaded data.
- Removed duplicate requests for EvidenceVariable.
- loadSharedNavLinks.js link from lforms-app.
- Use shared navigation links from lforms-app.
- Issues with tooltips, reading column headers and filter dialogs by a screen reader. Also, for sighted users, a column description will be displayed when they hover over the info button, and for screen reader users it will be read when they click on the button.
- Removed a separate column without a heading for the "Add record to cart" button.
- Changed the initialization spinner to a non-blocking one.
- Made buttons and menus more obvious.
- Added showing number of people in studies for R5 servers (ResearchStudy.recruitment.actualNumber).
- Support for rdf-server 1.0.0, which is ready for application-based stickiness for AWS load balancer.
- Issue of missing logout button and checkbox after SMART on FHIR login.
- Observation.interpretation should be searchable if available on server.
- Align search parameter name fields in query builder.
- Added baseR5 server to the list and made it default.
- When a query fails, limit the retry count to 3 instead of retrying for 90 seconds.
- Combined requests for patients with the "_id" parameter to reduce the number of requests.
- Bug in the search for patients when nested ORed criteria produce requests for different resource types therefore in the output stream, we may have different resource types.
- Improved performance of loading variables from Observations.
- Disable next step header if RAS not logged in.
- Updated limit on the number of _has parameters per request for known servers.
- The alpha version became the current version, current version became the previous version.
- Infinite loop when loading variables in the "Select records" step if the server returns observations without codes.
- Inform user if the search patient or pull data request failed.
- Navigate to alpha version after Immport OAuth2 login.
- Ability to login to Immport server through OAuth2.
- When user types in the Observation Name autocomplete search input field, wait 200ms before sending out the query, so that previous queries are not sent if user keeps typing.
- After logging in through RAS, a new TST token is received but the subsequent queries are not sent with the new token.
- Issue of focus left behind the dialog after the dialog of "session expired" or "disable batch" opens.
- Names of search parameter value fields.
- Target javascript version: es2017 -> es2015.
- An issue where the search parameter value was updated with the value of the previous control when the search parameter name was changed to a search parameter of a different data type.
- Improved search performance when using ResearchStudy criteria.
- Improved search performance when using criteria that can be converted to the _has search parameter for the Patient resource.
- Bypass initialization queries when there is a setting for the result.
- Show "browse public data" option only if it's useful.
- Remove Immport dev server from list.
- Sorting CTSS variables did not work.
- The selected studies were not considered in the patient search when variables were selected.
- Go to "Additional Criteria" tab and show errors, if user search for patients from another tab while "Additional Criteria" tab has validation errors.
- Tooltip for the "add to cart" button blocks next row button.
- Handle RAS error in RDF callback.
- OAuth2 support.
- Using the "advanced search" UI for additional criteria.
- A bug in the search algorithm when ANDed criteria are ORed in the parent node.
- Issues with saving and loading a cohort and criteria.
- Special case of sorting only the newly loaded items of the list of variables obtained from observations.
- After selecting variables with the same display name but different codes or systems in the pull data step, we only saw one selected observation code item.
- "No records were found" message for empty tables not in "browse public data" mode.
- CSP issue when a https server returns next links with http.
- Next page preloading for resource tables and using it to load variables from observations.
- Use CTSS for Observation code lookup in the pull data step.
- Issue with incomplete data loading in the pull data step when checking 10000 observations for each of 1000 patients in a cohort by removing the XMLHttpRequest timeout and disabling response caching for requests in the pull data step.
- Support for basic authentication.
- Server version detection: 4.0.0 is now defined as R4.
- Download button on all tables.
- Preserve Advanced Settings after RAS login.
- Option to convert Observation table into Variable-Patient table in pull data step.
- Display failed requests message in Pull Data tables.
- Request priority management.
- Loading data in the pull data step blocked other requests and made it impossible to create a new cohort.
- Display of incorrect amount of studies in the select records step.
- R5 support.
- Issue with empty list of variables in the select records step for an R4 server with ResearchStudies.
- Replaced
subject:Patient=<id1>,<id2>withsubject=Patient/<id1>,Patient/<id2>because baseR5 doesn't support it.
- Resend requests separately and notify the user of problems if a batch request fails.
- Show the list of Observation variables with the same name in define cohort step.
- "Browse public data" option for other FHIR servers
- Issue with incorrect caching of loaded studies
- Do not show session expired message on "browse public data".
- Replaced use of ":not=zzz" in autocomplete with ":missing=false" if supported.
- Do not cache failed requests that have status 401 or 403.
- Renamed "Code" column to "Variable Name" in Observation table.
- A new "Code" column to display raw code in Observation table.
- Do not retry GET requests aborted by the server.
- Showing the total number of variables from CTSS.
- Save/load/restore cohort with cart data.
- On dbGaP 4xx errors, direct user to re-login and restore the step and data afterwards.
- Make initialization requests only when necessary. Remove unnecessary requests before login.
- Handled cases when a server returns less than the expected number of results.
- Match Observation table Code column to selected Observation codes, if any, in "pull data" step.
- Error handling for dbGaP 4xx and 5xx responses.
- Split column of values into columns of values and units in export.
- Added study name to the variables table for non-dbGaP servers.
- Limited variables list to user's studies for dbGap servers.
- Made the alpha version the default for the dbGap server.
- when a SMART on FHIR connection was used, batch processing and query caching didn't work, and initialization requests were issued without authorization.
- when there are ResearchStudies but no ResearchSubjects, we should not display the study tab and should not use queries for ResearchSubjects
- Filter observation codes by search text, while taking into account synonyms.
- Show proper error message for unsupported FHIR version.
- Ability to add the other end of the range in a second line for Observation variable constraints, if the selected comparator in the first line is '>', '>=', '<' or '<='.
- The list for selecting codes in the "pull data for cohort" step is limited to those codes that exist for the patient cohort.
- Filter certain columns by a prefix in dbGaP Variables table.
- Ability to create a cohort of patients by browsing and selecting records for non-dbGap FHIR servers.
- Variables tab for non-dbGap FHIR servers.
- Cart UI changes:
- Replaced the checkboxes with small "move to cart" buttons.
- Moved the cart part of the page above the list of studies/variables and hid it until there is something in the cart.
- Revised row range selection.
- Moved the download button to the table header.
- An issue where the switching between FHIR servers was not followed by clearing loaded data.
- An issue where the switching to the "View cohort" step was allowed without searching for patients.
- An exception when trying to pull data for an empty patient cohort.
- Show message inside table when no record matches filter.
- Close the server select list after selection.
- Issue with browser back button after RAS login.
- Updated to Node.js 18.
- Hid an unnecessary option to view the full list of research studies from the select records step.
- Added aborting of HTTP requests if we no longer need a response to them.
- An input field for the maximum number of recent Observations per Patient to check when no code is specified in the criteria.
- Temporarily disabled the use of the _security URL parameter.
- Enabled use of RAS for dbGap by default.
- Temporarily enable RAS with a URL parameter until RAS goes live.
- Added a column with the LOINC short name to the table of variables.
- Added search for synonyms when searching for variables by display name.
- Removed access to the FHIR server from unit tests.
- Editing the "server" URL parameter did not result in a logout from RAS.
- Replaced calls to LiveAnnouncer with calls to Def.ScreenReaderLog from autocomplete-lhc.
- Issues with initialization spinner:
- there were no announcements for the screen reader,
- didn't block keyboard input.
- Duplication of Observations when pulling data for a cohort.
- Updated website header and footer and logo.
- Removed downloading the full CapabilityStatement with /metadata.
- Removed the manual show of study 2410.
- Upgrade Angular to Version 14. Removed support for IE11.
- Integration with dbGaP power user portal to allow user to login/logout.
- Do not cache responses to aborted requests.
- Caching of requests which check whether SMART on FHIR connection is available.
- A bug due to which the stepper could not progress past the first step after logout from SMART on FHIR without changing the server URL.
- Removed Patient.deceased from dbGaP fields.
- Narrator read "The field's value does not match any items in the list." when the page loads.
- Added caching of response to a query for the full list of ResearchStudies.
- Added caching of responses to initialization requests when selecting the FHIR server.
- Option to launch a SMART on FHIR connection.
- Search for patients by records in the cart.
- Using the UCUM data from fhirpath.js to display a list of eligible units for variables of type Quantity.
- Incorrect stepper state after selecting an action and switching to another dbGap server.
- Bug that could cause the patient search to end unexpectedly without any notification.
- Optimized patient search by combining ORed code criteria for Observation and eliminating unnecessary queries for ANDed criteria.
- Basic RAS flow without actually talking to dbGaP login portal.
Columns in the table of variables:
- Added a column with the LOINC long common name to the table of variables.
- Renamed column "Variable Name" to "Variable Display Text" in the table of variables.
- Loss of focus on resource table header cell when sorting table.
- Added server-side study filtering for the title column of the ResearchStudy table.
- No variables after selecting studies.
- Adding studies to the cart affected the list of studies when browsing public data.
- Option to limit the ResearchStudy resources to ones the user can access.
- The validation status of the Query Builder component was not updated when its configuration was changed. Therefore, the search for patients could not be started without changing the criteria.
- Button to download the Research Study or Variable tables into .csv files.
- Grouping/ungrouping variables in the cart.
- Pressing the Tab key on the keyboard while the resource table filter button had focus was ignored.
- Constraints for variables in the cart.
- Names of studies and datasets to the table of variables.
- A component for displaying text that can be truncated.
- Loading variables when scrolling the table.
- Unnecessary too fast loading of studies in an inactive tab after a 2-minute pause.
- Writing a value to AutocompleteParameterValueComponent.
- Options to variables tab for CTSS API.
- Updating the position of the sticky table header when changing the content of the header.
- Replace mat-autocomplete with autocomplete-lhc controls.
- Match list value on search parameter value autocomplete controls.
- The ability to select multiple rows in a table with a mouse click while holding down the Shift key.
- The ability to deselect all selected records.
- Sorting to the variable table (for the alpha version).
- Simple prototype of the component for records in cart for the alpha version.
- The ability to switch to the alpha version using the menu on the version number link.
- Incorrect query parameter _type replaced with _format.
- Removed prev folder.
- Do not sort the column when user clicks on header info icon.
- Use url pattern https://dbgap-api.ncbi.nlm.nih.gov/fhir* for dbGap.
- Alpha version of creating a cohort of patients by browsing and selecting records.
- Update criteria data from an old cohort to match current format.
- Add version number to downloaded cohort.
- Use comparators for "string" type search parameters.
- Show "display | code | system" for Observation codes that have duplicate display.
- In Observation codes query, include code system.
- Filter out Observation codings with no code in autocomplete.
- Display code if a coding has code but no display.
- If user selects Observation "Variable Value" search parameter without "Variable Name", show the complete list of numeric and string comparators.
- Reinitialize autocomplete field for "variable value" when "variable name" changes.
- Incorrect query in some cases:
- when multiple values are specified in the "variable name" field, and there is a "variable value" field.
- when multiple values are specified in the autocomplete field for "variable value".
- Simplified code of resource table component and components that use it.
- valueString Observations are not handled.
- Fixed exception "expression has changed after it was checked" (https://angular.io/errors/NG0100).
- Simplified component code by moving code from components to services.
- Disabled "Save the cohort and criteria for later" button when loading a cohort of Patients.
- % complete based on the number of requests processed.
- the ability to set default values for "Requests per batch" and "Maximum active requests" in settings.json5 per server and set the default value for dbGap to 50.
- When requesting Observations, replaced the use of two separate parameters "combo-code" and "combo-value-quantity" with a single parameter "combo-code-value-quantity".
- Default filter for the "Pull data for the cohort" step.
- Removed duplicate observation codes from requests. Duplication occurs when different autocomplete items in the observation lookup component have the same codes.
- Fixed a bug in the webpack loader that caused the application to build on Windows incorrectly.
- Hide resource table columns without data after loading data.
- The use of a number range filter for the Count column type of the ResearchStudy table has been restored.
- Replaced incorrect "dateTime" data type for search parameters with "date" to use the correct controls.
- Replaced hyphens in search parameter names with spaces.
- Hide shared header/footer until page loads.
- Evidence Variable search parameters in define cohort.
- Ability to pull Evidence Variable data for a cohort.
- get code list for DocumentReference.contenttype via autocomplete search.
- Partial hiding of expanded filter lists for research study table.
- Load shared header and footer from CTSS.
- Show tooltip if the resource table cell text has been truncated.
- Removed the option to wrap text in resource table cells.
- Sorted the dropdown list items for the filter fields in the resource table alphabetically.
- Reduced time to get the first page of research studies.
- Provided lists for valueCodeableConcept Observations.
- Enabled source map generation.
- "Keywords" and "Id" for the ResearchStudy table is hidden by default.
- Additional columns with numbers for ResearchStudy table are shown by default and renamed.
- Use acronym from Organization reference as a cell value in resource table. when available and when it matches the display text.
- Provided the option not to wrap the text in the field.
- Forced "Title" column wider than others for the ResearchStudy table.
- Columns are hidden when we know there is no data for them.
- Migrated some of the e2e tests to Cypress.
- Program to update the .xlsx configuration file and updated the .xlsx configuration file.
- Missing spaces between words in autocomplete dropdown list.
- Support for interpretation search parameter when applicable.
- Moved comparator, value, and unit to a separate search parameter.
- Show "Value" column in Observation table by default.
- Query Builder widget:
- Add color to the AND/OR lines.
- Radio buttons to switch between AND/OR operators.
- Remove "OR" option for single-resource criteria.
- Added AND/OR above connecting lines.
- Hide AND/OR toggle when there is only one condition.
- Program to store lists with non-required binding.
- Observation.category search parameter in default server now uses a autocomplete-lhc Prefetch control with values updated by above program.
- Until authentication is in place for dbGaP, we include the consent groups as values for _security.
- Query Builder: search criteria in Define Cohort step now can combine using Boolean operators.
- Detecting of the possibility of using the "age-at-event" sorting parameter.
- Rename "resource type" to "record type".
- Text changes in define cohort page.
- Removed Observation.identifier from search parameters.
- Left match word boundaries when filtering resource type or search parameter.
- Announce to the user that a new field has appeared after selecting a search parameter.
- a "code text" search parameter for each of the resource types with a main "code" field.
- Turned search parameter value autocomplete prefetch into search, for lists whose binding is not required, e.g. Observation.category.
- Show example items when the empty control is focused.
- Program to update the .xlsx configuration file for the 'show/hide' value (and row colors) of columns, based on the 'show/hide' values of matching search parameters.
- Script for start-public task.
- The wrong tab highlighted when the dbGaP server was selected.
- Ability for user to use TAB key on a dropdown list to select an item and move to next form control.
- Cursor will be focused on the input control of the newly added line after user hits "add resource type" or "add search criterion" buttons.
- Simplified pull data step search criteria, keeping only observation code selection criterion.
- A program to automatically determine whether search parameters have data on corresponding server and update the .xlsx configuration file for the 'show/hide' column.
- Current tab of the stepper will stay highlighted.
- Include word synonyms when querying server for autocompletion.
- Adding a new resource tab in the Pull data for cohort step after or when loading Observations data threw exceptions that blocked further operation of the application.
- Issue with displaying an empty table when changing the active tab in the Pull data for cohort step.
- Using XLSX file(s) to configure search parameters and resource table columns.
- Switching to the View Cohort step after loading criteria and cohort.
- Sort number columns by number in resource table.
- Updated to Angular version 12.
- UI updates to make resource table more readable.
- Preserve filter values when changing column selections, unless the column with filters is removed.
- Trigger table filtering after column change.
- Ability to filter number columns by range.
- Only Research Studies user has access to can be selected.
- By default, all selectable Research Study rows are checked and displayed at the beginning of the table.
- Removed "Skip this step" radio button.
- Reduced number of columns shown by default in Research Study table.
- Not all criteria were saved while saving cohort and criteria.
- Not all characters were escaped correctly when downloading a CSV file.
- Descriptions of not all criteria were correctly extracted from the specification.
- Quantity units should not have units in single quotes.
- Autocomplete-lhc dropdown styles.
- Ability to filter column using autocomplete control with a list of possible column values to choose from.
- Moved the filter into popup modals that you can open by clicking the filter icon on table headers.
- Validation for criteria.
- Disabled download button until the resource table data has been loaded.
- Slow evaluation of the contents of the resource table cells.
- Observation search parameter of type "Quantity" now uses the composite test value controls.
- List of units for LOINC codes from CTSS.
- Added descriptions for search parameter names.
- Renamed "Test value prefix" and "Test value modifier" to "Comparator".
- The default comparator for the test value is "=".
- Patient Id column to ResearchStudy table in the step of pulling data for cohort
- Search criteria in Define Cohort step now has categorized structure based on resource types.
- "Observation by Test" is now combined into "Observation" resource type as parameter name "code text".
- Removed the date fields in "code text" search parameter.
- Hide Research Study step if server has no Research Study data.
- Automatically fallback to single requests if batching is not supported.
- Show Patient id (dbGaP)
- Store visible table columns for each service base URL separately
- Limit list of resources to what has data (dbGaP).
- It was possible to switch to the View cohort step without searching for Patients.
- README.md
- Functionality to save cohort data and criteria into file.
- Functionality to load cohort data and criteria from file.
- Slowness to load Patients from a ResearchStudy.
- When searching for observation codes, the system now tries to match code as well as text. A code will be returned if user input matches exactly the code, or part of the code text.
- User can now pull data for Patient the same way as other resource types.
- After user enters a date range value, the application will use that as default value for later date range controls. Refreshing page will clear the default value.
- User input to define number of resources per patient for all resource types in Pull Data step.
- When user searches for active patients, the application now tries to also return patients without the 'active' field defined.
- Filtering of values in the ResearchStudy.condition column by preferred code system.
- Columns for research study content (dbGap only).
- Option to filter ResearchStudies if the user has access to their data.
- Detection of columns visible by default.
- Requests to the FHIR server without endpoint - they cannot be combined into a batch request.
- Title of the configure columns dialog.
- The application has been rewritten to use Angular.
- The new application uses a wizard-like workflow - content is divided into logical steps.
- Speed up initialization by replacing "_sort=date" with "date=gt1000-01-01" in the init query which is used to check if sorting Observations by date is supported.
- $lastn lookup of Observation codes, when supported
- It was possible to select the same criteria after loading the cohort criteria
- Displaying Observation component values
- ResearchStudy for cohort selection
- Spinner when loading data
- Support for age-at-event search parameter extension
- Display Observation value from component property
- The ability to enter a floating point Observation value in search parameters
- Advanced settings section
- Sort Observations by date only if possible
- Handle 429 responses with rate limiting
- Resource type field value is empty by default
- Column selection for resource tables
- Allowed non-LOINC codes to be entered in Patient selection criteria
- Added description for each step
- Reuse selection criteria values from the Patient selection area
- Notes to screen reader log when adding/removing elements
- Observation list does not match term after selection
- Save/Load Cohort criteria
- Version number to html page
- Build/Load Cohort (instead of Patient selection)
- Allow patients to be selected by Observation codes which do not have values provided for them
- Changed the Patient.active parameter input control from a checkbox to a radio button group
- Exception that can be thrown after removing a resource type tab
- Results of not all queries were considered when filtering patient resources
- Extra commas in http requests for data needed for resource tabs
- Allowance to choose which resource types users want to see for selected patients
- Display of minimum and maximum values that exist in the database for a date range criterion
- npm task "show-webpack-treemap" to run webpack-bundle-analyzer
- Bundle size reduced by removing moment locales
- Revised Observation criteria for Patient selection
- Issue with parse batch response
- Issue with partial support of Promise in some Edge versions
- Duplication of the search criteria component when changing the server
- Batch tuning applies only after server change
- Display that criteria are combined with logical AND
- Issue with value sets that are used more than once
- Allow other resources to be used to select patients
- Resource ValueSets are built from the FHIR spec downloads
- Allowed searching of any searchable Condition, Observation or MedicationDispense field
- Separate Patient selection section
- Statistical data loading information
- Allowed searching of any searchable Encounter field
- Allowed searching of any searchable Patient field
- Added display of data loading time
- Allowed selection of patients by gender and age
- Issues queries per each selected patient
- Added the ability to automatically combine requests in a batch
- No cache used for http errors
- separate configurations for production and development
- Added output fields
- Added button to download observations in CSV format.
- Some markup issues fixed
- Set the default test/category radio button selection to 'test'
- Support for searching by categories. The category list includes a mixture of categories from the Observation category list and the DiagnosticReport category list, with a few modifications.
- A cache for the AJAX requests.