All changes to the MarkLogic (backend) portion of LUX capable of impacting the runtime experience will be documented in this file. These are to include software, configuration, and environment changes.
- Optimized semantic facets to produce results faster (#365).
- Add work.aboutEvent and event.subjectOfEvent back to the advanced search config. Use updated predicate for these search terms (#387).
- Removed the semantic facet timeout override of 59 seconds. Semantic facet requests are now subject to the application server's default timeout of 20 seconds. Associated build property:
viaSearchFacetTimeout. (Part of #365) - Removed the ability to filter searches related to semantic facet requests. Associated facets endpoint parameter:
filterResults. Associated build property:filterSemanticFacetSearchResults. (Part of #365)
- Updated search terms config for predicate names that changed in the 2025-01-10 dataset (#420).
- Removed the advanced search config for Works About an Event (#387).
- Added search term config for Works About an Event (#387).
- Improved performance of queries when a Hop Inverse term is nested inside a Hop With Field term (#304).
- Enable Sorting By Semantic Criteria and Provide Various Semantic Sort Options (#343).
- Add new Non-Semantic Sorts (#342).
- Fixed Part of Set search term for Works, which was modified while creating Part of Work search term (#234).
- Modified logic for quoting terms with colons (#272).
- Added Part Of Work and Contains Work search terms to Works (#234).
- Enabled searching and sorting across multiple scopes (#318).
- Enable searching for identifiers without quoting them (#272).
- Exclude Collection Sets from the stats for Works (#283).
- Enabled searching for Works whose creation was caused by an Event and vice-versa (#269).
-
Updated Work Is Online facet to include Sets (#250).
-
Change Work Created By to include People & Groups that are listed as being creators in part (#278).
-
Update AAT for archive sorting - used to be sort titles (http://vocab.getty.edu/aat/300451544), now it is sort values (http://vocab.getty.edu/aat/300456575) (#325).
-
Revert workaround in #337 - triple has been restored (#337).
- Removed "Is the Place of Publication of..." and "...Published At this Place" relationships from Related Lists (#284).
- Fixed issues getting docs with the 'name' profile and primary names (#290).
-
Workaround for missing 'setClassifiedAs' triple on Collection sets. Sets will use the 'referenceClassifiedAs' triple in the meantime. (#337).
-
LUX is no longer dependent on generating data constants after data is loaded into the content database. (#290).
- Fixed '<=' and '>=' comparators for dates (#332)
- Added the 'Professional Activity Categorized As' and 'Professional Activity of' search terms to Advanced Search (#248)
- Added versionInfo endpoint to make the Marklogic Code Version, Date Version, Server Version, Database (Tenant) Name available (#264)
- Added IPCH user for local deployments (#314)
- Updated help text for ID fields so that they don't prescribe a specific entity type (#303)
- Fixed incorrectly configured fields - idIndexReferences should be an array (#301)
- Update date facets such that they can be sorted more accurately(#256)
- Updated triples used by search terms so that there are not overlapping triples used for different search scopes. This prevents records of the wrong scope showing up in related lists. (e.g. the Exhibitions concept showing up in Related People & Groups for Pablo Picasso) (#309)
- Added reader roles for ILS, IPCH, and PMC. Added reader roles for a couple other values that may be present in the
admin.sourcesarray: 'create' and 'update'. Added endpoint consumer role for IPCH. Removed endpoint consumer roles for YCBA and YUAG. #280
- Fixed tokenization issue that #273's original implementation introduced in v1.23.0.
- Fixed the triple referenced by work.aboutItem so that it can return results (#74).
- Changed units lib to parse units based on %%mlAppName%% instead of 'lux' (#277)
- Added the Objects/Works about/subject of other Objects/Works relationship (#74).
- Added the ability to vary the search term, advanced search, and related lists configurations by unit, or more precisely, endpoint consumer. This has no impact on https://lux.collections.yale.edu/. For more information, see Unit Portals, an upcoming feature of LUX. (#277)
- Fixed search criteria processing for terms containing a phrase and either another phrase or word, such as
"physical measurements" analysis. The phrase was being split into words when it should have been. This only impacted advanced search. (#273)
- Changed exception handling for invalid search requests, enabling endpoints to conditionally process invalid search requests. The
facets,searchEstimate, andsearchWillMatchendpoints now log different messages when failing due to invalid search requests, enabling more targeted system monitoring alerts. Other than the wording of some messages, there is no impact to endpoint consumers. (#236) - Search no longer calculates search result scores when sorting by an index (#123).
- Changed help text for [Record Type] Class advanced search fields to explain what each Class is (#251).
- Changed the "Occupation" field in Advanced Search to be called "Occupation/Role". Also updated the help text accordingly. (#40).
- The default user on the request-group-1 and request-group-2 application servers is now explicitly set to
nobodyduring deployment (#252). - Replaced the
copyRestApiOptionsdeployment task withimportRestApiOptions. The tenant's deployer role was unable to execute thecopyRestApiOptionstask but is able to execute theimportRestApiOptionstask (#243).
- Added the ability to switch from filtered to unfiltered results in the search, semantic facet, and related list contexts. The default for each context is controlled by a dedicated build property:
filterSearchResults,filterSemanticFacetSearchResults, andfilterRelatedListSearchResults, respectively. The initial intent is to continue filtering results by default. Thesearch,facets, andrelatedListsendpoints now accept thefilterResultsparameter, which may be used to override the default or otherwise explicitly specify the filtering behavior. (#223) - Added facets and advanced search configuration for agent's professional activity (#156)
- Changed log messages for failed facet and search estimate requests when due to insufficient search criteria in order to prevent email alerts (#236).
- Changed the People or Group advanced search option config to People or Group Class (#201).
- Changed event used field to use a field index instead of triples - this doesn't change functionality, but keeps patterns used in the code consistent (#217).
- Fixed Has Digital Image facet and advanced search field for Works - they now include Sets (#52).
- Fixed Work Class facet and advanced search field for Works - they now include Sets (#222).
- Added range indexes for AutoComplete fields without range indexes. AutoComplete is not used at the moment, but this ensures it is functional if that endpoint is consumed by the frontend / middle tier (#198).
- Changed security roles and how document permissions are set in support of tenants and unit portals. Includes roles for YCBA, YPM, and YUAG. For details, see LUX Backend: Security and Software and LUX Backend: Importing Data. (Security portion of #73)
- The REST API options of both application servers used by the middle tier are now identical (#212).
- Error handler no longer masks the underlying error when
external.erroris undefined (#207). - Additional details will be logged for failed
searchEstimaterequests (#183). - Additional details will be logged for failed
searchWillMatchrequests (#177). - The
facetsendpoint now supports pagination. Given maximum page lengths, it may be necessary to make multiple facet requests to retrieve the desired number of a facet's values. The maximum page length varies by facet type (semantic vs. non-semantic). One may also wish to override the default sort order, for non-semantic facets. For details, please see thefacetsendpoint documentation. (#160 and #161) - Increased support for multi-tenant environments (#149).
- Name search criteria now resolves against primary and alternative names within select contexts (#100):
- Keyword search criteria resolves in related documents using primary and alternative reference names.
- Less the
setsearch scope, allnamesearch terms resolve against their respective primary and alternative names.
- Removed the
personRolesendpoint. A data change made long ago resigned this endpoint to returning an empty array (#203). - Moved the query plan viewer (developer tool) into its own ML Gradle project (#192).
- Removed obsolete depict-related configuration (#190).
- The event scope's
usedsearch term now works with an ID child field (#186).
- Added facets and search terms for record types for Objects, Works, Concepts, and Events (#138,#139,#140,#141)
- Added search terms for creation/production influenced by an agent (#65)
- Added build time data constant for 'collection' (#167)
- Added search term for works that are/are not in the public domain (#103)
- Changed indexes to use Getty AAT values instead of data constants (#54)
- Added config file for modules database (#124)
- Update member of / containing fields to remove duplicate and allow traversing hiearchical sets (#135)
- Added concept relatedToEvent related list (#96)
- Added support to sort items by their archive sort values (#90)
- Changed logic for via search facets (Responsible Units contain departments or curate sets directly. Responsible Collections are collection sets.) (#68)
- Updated pipelineDataConstants.json for new dataset
- Added agent recordType (Person or Group) to the user facing config (so it shows up in advanced search) (#75)
- Added relation names for agent relatedToEvent related list (#67)
- Added place relatedToEvent related list (#95)
- Fixed the search scope specified throughout the search response body when the search request specifies the search scope using the search criteria's
_scopeproperty (#81)
- Changed via search facets to order by number of results descending (#57)
- Fixed incorrect ID URIs in Activity Streams responses (#60)
- Cleaned up code for release to public repo (#1095).
- Updated documentation on setting up a local Marklogic envrionment (#1071).
- Added LICENSE.txt and NOTICE.txt (#60).
- Changed search estimate endpoint to accept scope as a query parameter (#1161).
- Changed Context link returned in Activity Streams endpoints to be correct (https://linked.art/ns/v1/search.json) (#1120).
- Changed facets and related list endpoints to use Activity Streams format (#1164).
- Removed search estimates endpoint. It is replaced by multiple requests to the search estimate (singular) endpoint. (#1118).
- Added a new data constant for visitor statements (#1143).
- Updated data constants for 1/25 dataset (#1150).
- Updated data constants for 1/13 dataset (#1137).
- Removed cluster-formation directory and cluster-formation documentation, as it is migrating to a new repository (#1130).
- Reverted date search change (#1032).
- Changed date search to use new JSON format (#1032).
- Added Search Estimate Endpoint (single estimate) which uses Activity Streams (#1104).
- Added Events To Agents Related List (agent relatedToEvent search term) (#899).
- Changed Search Endpoint to use Activity Streams (#1103).
- Changed Search Estimates Endpoint to to use Activity Streams (#1105).
- fix typo in pagination error (#1106).
- Updated documentation to make navigation easier (#1071).
- fix typo in pagination error (#1106).
- Changed advanced search labels to all use plural verbs - e.g. 'Have Digital Image' instead of 'Has Digital Image' (#1085).
- Add New Data Constants For Copyright Licensing Statements, First, and Specimens (#1064).
- Delete indexes associated to the deleted depictedBy and depicted[Scope] search terms (#934).
- Drop the any search terms and the *AnyId indexing configuration (#812).
- Drop event 'hasDigitalImage' search terms and indexing configuration (#777).
- Enable faceting by agent record type (Person vs Group) (#1011).
- Add new item profile 'rights' for getting rights information from works (#1059).
- Increased the maximum page length from 20 to 100 for search, while keeping the default at 20 (#1035).
- Removed event's hasDigitalImage facet as events never have digital images (#1037).
- Added the search request ID to the request log (#997).
- Changed the default request timeout from 59 seconds to 20 seconds.
- Changed simple search to no longer treat
NEARas a search operator (#974). - Changed the database index configuration generation script to no longer include field-level properties when those values match the database-level defaults (#904).
- Removed obsolete IRI code from the facets library (#1012).
- Fixed search parsing error messages that were being suppressed (#1016).
Increased the default request timeout from 20 seconds to 59 seconds.
Reduced the maximum number of concurrent threads on the lux application server (port 8003) from 32 to 24 (#920).
- Added visual items to the work about indexes (#961).
- Added a sort binding for sorting by relevance (#960).
- Changed the URI of several examples within the advanced search help text (#1004).
- Changed the advanced search term labels from being singular to plural (#975).
- Changed the search endpoint to only estimate the requested search scope (#953).
- Changed facet calculation to by faster by using
.toArray(#952). - Changed search to use
cts.falseQueryfor ignored criteria (#950). - Changed advanced search help text to indicate
has All of,has Any of, andhas None ofare the advanced search equivalents of simple search'sAND,OR, and-operators (#946).
For changes that predate v1.0.0 or for non-runtime tickets, please search for closed tickets with or without a version label. For example, there were over two dozen Backend v1.0.1 tickets.