From 08956f1edf9518c3237ba944d22bddbcadd099d9 Mon Sep 17 00:00:00 2001 From: milanmajchrak <90026355+milanmajchrak@users.noreply.github.com> Date: Wed, 20 Mar 2024 13:45:42 +0100 Subject: [PATCH] customer/uk-it-20 (#561) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * ufal/downloading-restricted-bitstreams-not-working-properly (#457) * The admin could download restricted bitstreams. * The user cannot download bitstream if he/she is not signed in. * Cannot obtain context with user. * The user is already fetched from the context. * Added docs. * Do not use endpoint for downloading the single file because it already exists - remove it. * Fixed AuthorizationRestControllerIT tests. * ufal/be-cannot-download-and-preview-files-after-migration (#454) * Find bitstream format using mimetype not ID. * Updated tests according to fix. * ufal/update-canonical-prefix-to-hdl (#460) * Updated `handle.canonical.prefix` to hdl handle. * Integration test cannot have changed handle.canonical.prefix. * ufal/be-user-registration-missing (#463) * The user registration is added when the eperson is created by the ui. * try using newer checkout, same as upstream, since error is with git --------- Co-authored-by: MajoBerger <88670521+MajoBerger@users.noreply.github.com> * ufal/be-license-download-statistics (#462) * Add logging of downloading restricted bitstreams. * Log downloading of every bitstream not only restricted ones. * Added docs * Refactored logging. * ufal/be-shibboleth-headers-encoding (#464) * givenname and last name values are converted to UTF-8 encoding * Delete eperson in tests * using our dspace-dependencies because of https://github.com/dataquest-dev/DSpace/issues/466 * ufal/be-fix-email-parameters Fixed emails sending - added arguments and cfg properties that are sent in the email. (#470) * ufal/be-provenance-subbmitter-missing (#469) * #8585 Add submitter information to provenance metadata * Cherry picked fix from vanilla and added test to check if the Item provenance metadata is added --------- Co-authored-by: Adán Román Ruiz * ufal/be-get-user-ip-address (#468) * Created endpoint for fetching IP address with tests. * Made the code more understable * ufal/publisher-ok-fix (#473) * fixes ufal/clarin-dspace#1096 (#471) * Added publisher filter in the integration test check --------- Co-authored-by: Ondřej Košarko * ufal/zip-preview-configurable (#475) * Made previewing of the file configurable. * The default value of the previewing the file must be true in the integration tests. * ufal/fe-oversized-file-upload-message Exposed max file limit for upload from the cfg. (#477) * ufal/be-email-restricted-download * Fixed comments in the clarin emails (#484) * ufal/be-not-show-shib-welcome-page (#485) * Added cfg property to enable/disable showing of the page with attributes passed from the idp. * ufal/be-s3-customization (#481) * The bitstream data are stored in to local store after uploading to S3 * The bitstream is removed from the S3 and local assetstore * Store number is specific if all storages are synchronized. * Return store number in the BitstreamRest object * Find out which store is used - it could be synchronized stores number. * Constant is moved to the Bitstream class * Synchronization of storages is not allowed by default - set up it in the test environment. * Added docs * Removed constant from the Bitstream class - it wasn't consistent * Overriden BitstreamStorageServiceImpl by custom SyncBitstreamStorageServiceImpl * Removed ClarinS3BitStoreService.java to SyncS3BitStoreService * Added doc and refactoring. * ufal/be-s3-checker-sync (#486) * Fixed UserCheck - There was created a wrong select * Fixed ChecksumCheck - the session was closed but there were some requests do the database then * internal/load-version-from-file (#488) * Load version from the specific file and show content in the root endpoint. * Fixed checkstyle issue * DO not log anything if the VERSION file does not exist. Added VERSION file path into cfg property. * Append every line into final String. * ufal/be-s3-checksum-fix (#487) * Cherry picked fix S3 optimalization fixes from Vanilla * Cherry picked adding of pagination on cleanup * Updated Sync services according to S3 optimization fix * The checksum value is compared between S3 and local assetstore and the new result is inserted to the database. * Added docs and refactoring * The checksum values are exposed by REST API * Fixed failing tests --------- Co-authored-by: Tim Donohue Co-authored-by: Luca Giamminonni * internal/versioning (#491) * add version to docker * Allow shell because it is using double quotes in the command string. * removed cat and ls commands --------- Co-authored-by: MajoBerger * ufal/comment-ever-failing-test * Commented still failing tests (#490) * internal/verion-update-path (#492) * Updated `build.version.file.path` because the server cannot find `VERSION_D.txt` file * Added VERSION_D.txt info `.gitignore` * User ${dspace.dir} instead of absolute path * internal/be-upgrade-clarin-dspace-7.6 (#497) * Split docker image builds into separate jobs to allow them to run in parallel. * Ensure 'main' code is tagged as 'latest' in DockerHub * Fixes #8558 - set Solr UTC timezone Set the timezone of the Solr date formatter to UTC * remove obsolete code fragments * Add action to automatically create a port PR when specified (cherry picked from commit f6a898c3d13360286c416b2588ab0447d9e3d81b) * Run PR Port action as 'dspace-bot' to allow new PRs to trigger CI checks * Minor update to label_merge_conflicts to ignore any errors (seem random at this time) * Fix typo. Config must be a valid regex (cherry picked from commit 799528963e3c0391852ecbaf82ef21ec7d477342) * Bump h2 from 2.1.214 to 2.2.220 Bumps [h2](https://github.com/h2database/h2database) from 2.1.214 to 2.2.220. - [Release notes](https://github.com/h2database/h2database/releases) - [Commits](https://github.com/h2database/h2database/compare/version-2.1.214...version-2.2.220) --- updated-dependencies: - dependency-name: com.h2database:h2 dependency-type: direct:production ... Signed-off-by: dependabot[bot] * Update FullTextContentStreams.java Fix NPE if bitstream is null * Update FullTextContentStreams.java Add additional NPE checks * Update ItemUtils.java Prevent npe if bitstream is null * dspace.cfg: remove old webui.itemlist properties These properties are no longer used in DSpace 7: webui.itemlist.widths webui.itemlist.*.widths webui.itemlist.tablewidth (cherry picked from commit 16c46c49797cf4a15b8ef51efdd365610bdf73ab) * dspace.cfg: remove old webui.browse.thumbnail.show property The webui.browse.thumbnail.show property is no longer used as of DSpace 7. Also remove subsequent references to adding thumbnails to item view columns since these are from legacy DSpace. (cherry picked from commit 213a546486073f09e73d91d014c489ed300bf59d) * dspace.cfg: remove old webui.itemlist.browse.* property The webui.itemlist.browse.* properties are no longer used in DSpace 7. (cherry picked from commit 35f72bc9d0c1a01aa0b9313216bbbe63e6960a81) * Fix #8963: Remove deletion constraint from Groomer * Enable entity type to submission form mapping by default (cherry picked from commit b71eee89c1e1dd7569e800e13eb8878548853ce6) * [DURACOM-179] replaced 'null' value with exception actual value in sendErrorResponse method calls having 'null' (cherry picked from commit aa35a47add5565a9302d276da2ceb22b8dbc320f) * DS-8935. webui.browse.link CrossLinks - Fix for multiple exact matches Fixes #8935 when multiple exact match "webui.browse.link" configuration entries are present that point to different indexes. Modified the code to return the index associated with the given metadata (which is used as the key in the hash map), instead of the key from the keySet (which may not actually be the metadata value being searched for). https://github.com/DSpace/DSpace/issues/8935 (cherry picked from commit b846c53baaeae1e19fbbafa3dc7ca724bcaf32c1) * #9006 fix referenced configuration file (cherry picked from commit 29a88d7e2dcfc36d2cd7991de3b84ef5f5623b63) * #9006 fix referenced configuration file (Test) (cherry picked from commit 309b0b355e4bffd6a1be3e6341dd6d17f99892c8) * Fix #8933: Only add the base statistic core if it hasn't already been added * Remove duplicate code * On failure log the name of the assetstore file and trace causes of exception. (cherry picked from commit 22974e982c99b7faa9d287ddc5bef4715f19849a) * Report Throwable's type too. (cherry picked from commit d6b612fc5cf84fe6b7226649451b7b927ded8997) * More description on OutOfMemoryError too. (cherry picked from commit bbe5df3f7dd4a33423fdf47702e23f3eb9ef821f) * Remove useless log.info (cherry picked from commit 1f3ad993cc4d10694112227245be3de1ec7b3762) * 103837: Refactor GA config to list bundles * README.md: Fix typo (cherry picked from commit ca8abddff1230e581501b482623966e64016d609) * Bump up versions of buildnumber-maven-plugin & build-helper-maven-plugin. add configuration for SCM failure (cherry picked from commit 78ea9e86df5e6737a4ce129120a6e1e6c74a371c) * 3331 - remove the --optimize feature of 'dspace stats-util' (cherry picked from commit 08c650921d7ce5210906db846ff29a053c2155a7) * fix MissingOptionException on help (cherry picked from commit 8ae5ffbf9b41fe2ad4e1146eceeff47e6ba985a0) * fix stylecheck (cherry picked from commit b1377ca1ef82d80f2ece9b48b8f1571e786c4525) * ingore unrecognized arguments on help (cherry picked from commit 82c9b6fc9baee9f4f5d8b4cc967b5d12b63cdd39) * add DSpaceSkipUnknownArgumentsParser as parser to ignore/skip unknown arguments in cli by help, fix not necessary ParseException in help (cherry picked from commit 86285d78aa0844b9811dcebdefa897ceeb944226) * remove not necessary else (cherry picked from commit 064e2caa37dfa283c3c08dee0e7321e36073bfa2) * Enable new skip merge commit feature * dspace-api: remove unnecessary trailing whitespace * 103818 ItemServiceImpl#inheritCollectionDefaultPolicies now clears item READ policies if new parent collection has a default READ policy * 103818 Add boolean parameter to ItemServiceImpl#inheritCollectionDefaultPolicies to decide whether to override item read policies * 103818 Add boolean parameters to ItemServiceImpl methodds to decide whether to override read policies * 103818 Extend ItemServiceTest#testMoveItemToCollectionWithMoreRestrictiveReadPolicy * 104878 Fix error in ItemServiceTest related to inheriting collection policies upon item move * 104878 Adjust ItemServiceTest to expect correct behavior of bundles when item is migrated * Fix failing IT in BulkAccessControlIT * 103837: Fix isContentBitstream() in GoogleAsyncEventListener * Rename test suites that are really integration testing. (cherry picked from commit f66ca33b0627c1b0789c9c3ce407463f5dc3356e) * Bump org.eclipse.jetty:jetty-xml Bumps [org.eclipse.jetty:jetty-xml](https://github.com/eclipse/jetty.project) from 9.4.51.v20230217 to 9.4.52.v20230823. - [Release notes](https://github.com/eclipse/jetty.project/releases) - [Commits](https://github.com/eclipse/jetty.project/compare/jetty-9.4.51.v20230217...jetty-9.4.52.v20230823) --- updated-dependencies: - dependency-name: org.eclipse.jetty:jetty-xml dependency-type: direct:production ... Signed-off-by: dependabot[bot] (cherry picked from commit 2bcc0b38a9436b0abc4c54e419f0fa6ae194269c) * Add note that rebooting Tomcat required * Add websvc.opensearch.autolink and websvc.opensearch.shortname to exposed REST configuration properties (cherry picked from commit 80b35c9650ec2f40bea3b497b65ce77d0c97bfcf) * Add a "container friendly" log4j2 cfg and output compose dspace log to console (cherry picked from commit 9eefd56cd7013730ff43969b321d6cbee2a3ae64) * Fix ClassCastException (Collection cannot be cast to Item) in Handle identifier classes (cherry picked from commit 1271374d37a3f7d9cec270e54ec106895aa934bc) * Fix checkstyle. Correct grammar of comment while doing so. (cherry picked from commit ffa2683c632cfab59b0432c203434ac15a6eb85f) * Avoid double slashes in sitemap paths. (cherry picked from commit eae4463eaa5916bd9b20f4e4132398aceeba1f02) * Remove 'cross join' from count query. Updates "countHandlesByPrefix" to use a query similar to existing "findByPrefix" (cherry picked from commit 14223bd712ce91cf97096f2201924baea8456814) * fix logical bug when checking if field is controlled authority * remove optimize option from oai import * added authorization check for license bitstream in OAI import * OAI: add support to extract embargo from bitstreams and expose it in OAI metadata (cherry picked from commit db81d758a947a9bdbb63fea9e872bc9b52a377ff) * ItemUtils.java: added method doc (cherry picked from commit 51e60fbcf92ea731c4e355c9cf080d251ffbf68f) * ItemUtils.java: improved method to account for multiple embargo policies and select the longest embargo (cherry picked from commit 538be7f09ba790a4ab7099e7027e1e8f6a9c62ea) * Refactored access-status to include embargo date based on the DefaultAccessStatusHelper logic (look at primary or first bitstream) (cherry picked from commit 895926f021a355181faef47b5c41e78031700475) * Remove unused imports (cherry picked from commit 4bd2cfdf0f931aec7a05db42f255423fe806ea77) * Fix style issues (cherry picked from commit 724a4ffb0ed9ffefb2866930655767590b462bb5) * Fix style issues (cherry picked from commit 6e2c8a4ae0068d844d0fc796001c170c8849babf) * Add null check (cherry picked from commit 0de4c3945ed7f30d41841cda4bf01acf9ffc130f) * ItemUtils.java: refactored addEmbargoField (cherry picked from commit 291afa765d29836a67727fdd2f82ac0c9f9310c4) * uketd_dc.xsl: also expose access-status if embargo or restricted (cherry picked from commit 4b40872a6d5a3934c1f79c6babf439a21ce25f66) * DefaultAccessStatusHelper: fix logic to take shortest embargo (cherry picked from commit d17ef09082aa237cffdc928d9560667487c2c976) * Remove currently unused customisation of ItemUtils (cherry picked from commit 490a982e8055991a6b8cbacece22b924466e22df) * DefaultAccessStatusHelper: getEmbargoFromItem return null embargo if status than embargo (cherry picked from commit e05e73a112ce60bd0689ce68af442382712bd5fc) * Additional Item class cast fixes in handle providers DSOs were not properly checked if they were instanceof Item before attempting the cast in HandleIdentifierProvider and VersionedHandleIdentifierProviderWithCanonicalHandles * Remove Oracle script that accidentally made it in via #8800 (cherry picked from commit 5e04edf41e452cd383597680da9c3101211156b8) * 8968 - request-a-copy email: non ASCII characters are encoded as HTML character entity references (cherry picked from commit db36d5eeae3e76b61178c2c7ac4243bc2fc20a97) * unused import (cherry picked from commit bf6e042085140e305d43d61ddce564fbfe819c7f) * 8968 - added custom StringEscapper (cherry picked from commit 103c8ee75771d3d9e58e530b8855d07cc14598c2) * checkstyle (cherry picked from commit 2c2b3b18dc781054539add48ca74e4bf688c400c) * 8968 - implementated using HtmlUtils scaping (cherry picked from commit 090beedb6f692df29d1a61d4c2e6fde09d4b4c1d) * checkstiye (cherry picked from commit d12fbe2c340e18e42dba4380ee9976bccb4ca421) * Define _version_ * #8585 Add submitter information to provenance metadata (cherry picked from commit c15ac0eb4a3d39a0de47adbfa5260a6f3b396837) * 8585 - added provenance to metadata-import and itemImport (cherry picked from commit ea6307dcc68a75c935049a02022145691693cff4) * dspace-api: fix misaligned comment (cherry picked from commit 4fba787322803cc36ef267f0d6913b92c1eaeca4) * oai_openaire.xsl : change resourceTypeGeneral for thesis Thesis are "Literature" resource type (resourceTypeGeneral), not "other research product" ref: https://github.com/openaire/guidelines-literature-repositories/issues/43#issuecomment-1318262914 and https://api.openaire.eu/vocabularies/dnet:result_typologies/publication (cherry picked from commit 669ff343503539aa6fc8b23600989ab958a403b9) * Bump org.eclipse.jetty:jetty-http Bumps [org.eclipse.jetty:jetty-http](https://github.com/eclipse/jetty.project) from 9.4.52.v20230823 to 9.4.53.v20231009. - [Release notes](https://github.com/eclipse/jetty.project/releases) - [Commits](https://github.com/eclipse/jetty.project/compare/jetty-9.4.52.v20230823...jetty-9.4.53.v20231009) --- updated-dependencies: - dependency-name: org.eclipse.jetty:jetty-http dependency-type: direct:production ... Signed-off-by: dependabot[bot] * Bump org.json:json from 20230227 to 20231013 in /dspace-api Bumps [org.json:json](https://github.com/douglascrockford/JSON-java) from 20230227 to 20231013. - [Release notes](https://github.com/douglascrockford/JSON-java/releases) - [Changelog](https://github.com/stleary/JSON-java/blob/master/docs/RELEASES.md) - [Commits](https://github.com/douglascrockford/JSON-java/commits) --- updated-dependencies: - dependency-name: org.json:json dependency-type: direct:production ... Signed-off-by: dependabot[bot] (cherry picked from commit 11a08f1ac0a9b75bf3f2869d3760b2f0e229aefe) * Reading localized license file. * Reading localized license file. * Applying commit 6108c98d82f3cbf00e248f83535b06da4040ece4 * quote Pattern for thumbnail resolution constructed from bitstream filename (cherry picked from commit a1248074681a7bc4603176fb3e7d989b91edcbcd) * check null value of bitstream name before quoting name for regex (cherry picked from commit a9bcc0c223d0219f464d986d7b7c66b3c4cbc39c) * extract bitstream thumbnail name pattern into own function (cherry picked from commit c5466c2249c092f6638a7072b57c934d1d3581b5) * Make workflow curation tasks actually work. When curation runs, there was no "current user" and no claimed task, so the code broke when trying to find people to notify about curation failures. (cherry picked from commit a76af35a0cd4f0c0e8737c736578b17bcc349691) * Community request: fake EPerson from configuration. (cherry picked from commit bb9e88d1bb452d0865f4827134baf907e6d34044) * Correct some documentation. (cherry picked from commit be22790aad7f627e2ac027773e272b703986f589) * Handle missing role. (cherry picked from commit 2e62fa3fd1f264aac0bb4a12953b6385211e5656) * Fix "Site cannot be indexed" error by ignoring ADD/REMOVE events on Site object (cherry picked from commit ef7f02fe81bc570353c0bf6a43706c77909e30e3) * Add a null check when assigning ldap groups Prevent NullReferenceException by checking if the group list is null Fixes #8920 (cherry picked from commit bb6498ed5e4696201d3e45bd377faa407dca277f) * 9043 use Templates for compiled XSLT instead of Transformer - use Templates are thread-safe and NOT Transformer (cherry picked from commit 1160341cb2a2c163c8fddc04ddab46de9041e1b8) * Update to newly released XOAI 3.4.0 (cherry picked from commit 160ebbd791c0545db6516403da40cb191a2c8b99) * add test and fix (cherry picked from commit 48b0b71c6301b6eb46c387c47b71d0729cc2f889) * unset primary bitstream on bitstream service (cherry picked from commit 47ca74bc4220249b95de9b8e71186277c9ac31ca) * adding sql expression to fix deleted primary bitstreams from bundle (cherry picked from commit 8a531ad0c7e8fdf09fa9a3870024687e6708a9a1) * add bundle remove authorization (cherry picked from commit 3255e073fa110a3354f1265853bbf531c677f6ea) * adding missing bundle REMOVE authorization (cherry picked from commit 4a05600194fb9be7e19084f3a9106a0152fd0d80) * add missing head style check (cherry picked from commit caba4bbb96f56c103c4dd8ac9f9fa5863b40e04c) * fix style errors (cherry picked from commit 74605f159af5e53a3e890f578732a858cef12e51) * new testDeleteBitstreamAndUnsetPrimaryBitstreamID test for primary bitstream verification (cherry picked from commit e6d108a94e41e58d6d701f3ef0429fda438e6555) * new testDeleteBitstreamAndUnsetPrimaryBitstreamID test for primary bitstream verification (cherry picked from commit ad0d22a13a35a2167557efeb5ddea7a3a504424d) * new testDeleteBitstreamAndUnsetPrimaryBitstreamID remove unnecessary stubs (cherry picked from commit a3e506c7f452133e3cc973705d671dba61a469d6) * make comments more clear to understand (cherry picked from commit c0bbd9d91f894fbe26f8cf7c4f166da8ba1cefd3) * typo (cherry picked from commit 74cce86afcc163c52502892556679e6175fa1948) * Add basic pagination to /groups/[uuid]/epersons endpoint (cherry picked from commit 74c72354b405ed266b65cdd50b594d25bea0e87f) * Bug fix. Only use pageSize and offset if >0 (cherry picked from commit 15de2d0074b56f421b3bbb9f3955814497985aef) * Add missing pagination test for /groups/[uuid]/epersons (cherry picked from commit 457dd9ae441fa084ff7cc3eaf9213e5497a2b298) * Add pagination to /groups/[uuid]/subgroups endpoint, along with tests (cherry picked from commit e7c4b9eba2d8148e07543c3b6c61dde359018da2) * Add basic unit test for new EpersonService methods (cherry picked from commit c000e54116498030261d988f87a496beef7d21d1) * Minor unit test fix. Use isEqualCollection to compare list with Hibernate results (cherry picked from commit cdb68a6fdc925fcbb76f9265e64771497b3f78bc) * Add countAllMembers() with tests. Update tests to use try/catch (cherry picked from commit 58a15b72975940d48ae450e6b46557b4443f2978) * Replace several usages of allMembers() with count methods to avoid performance issues (cherry picked from commit 2c9165afb08126189ee3367347e7011f89227b7c) * Fix bug in logic for determining whether a workflow group will be left empty. Need to check *both* EPerson and subgroup counts. (cherry picked from commit 9832259aa06d9fe140407ed54c4687989e98f7b2) * Use join instead of subquery as join seems slightly faster. (cherry picked from commit 9c0bf08cf4c3ab7e941ebe1bae66cf2aea720697) * Address feedback. Initialize HashSet sizes to avoid resizing. Correct comment about indeterminante ordering. (cherry picked from commit f011a5a5dbcd2def47dde7830981cf282ca660aa) * Allow users with write permission to view hidden metadata (cherry picked from commit 65a17d4390aeab69c191fb75559646aec9dda512) * Test modification: allow users with write rights to see hidden metadata (cherry picked from commit df7f6e9f4082e5aef3392932f8a87177ac202655) * Add test to check that user with read rights can see hidden metadata (cherry picked from commit 03496c36d4d47138bcd51badf8daca720d4cc484) * Verify optional message is not missing or a literal "null" value (cherry picked from commit 534ee3a699937eedd11aa5cb54f97b081bcda621) * Change the database mode to READ_ONLY during the indexing by discovery consumer (IndexEventConsumer) (cherry picked from commit 94822b50af4098d990d63e27bb3906cfa9c0ec37) * Add functions to do a manual flush of the db session and call flush before change to READ_ONLY mode to be sure we index the current object (cherry picked from commit c33d3fa87d6c29533d379939bd23b29ff3d9b5c9) * Flush database changes after switching to READONLY mode (cherry picked from commit 00a65312ccb52481cd72653b4c5465b7d16c760e) * Add test to check retrieving of policies after changing mode to READ_ONLY (cherry picked from commit d19a9599b5f08a567c93d2e167e219673518fb78) * Change class name to ContextIT and correct a test (cherry picked from commit a5567992bbe456cd33c68f695a2364f507149e7a) * dspace/config: update spider agent list Update list of spider user agents from the COUNTER-Robots project. See: https://github.com/atmire/COUNTER-Robots (cherry picked from commit 7566a79d906b5050bef01d22c5f4b3e4ab6e4b58) * removed options to ping search engines when generating sitemaps (cherry picked from commit f8f88060408c30314cdcf38ba5bbac0f367ee3fd) * XmlWorkflowCuratorServiceImpl: add check to queue task if configured; Curation: remove obsolete code preventing curation running on workflow tasks as #3157 is now implemented * Return both user and operational LDAP attributes Explicitly request both user and operation attributes for LDAP group search as the default searching does not include operational attributes. This is required to fetch the memberOf attribute when checking LDAP group membership. Fixes #9151 (cherry picked from commit 56b7cbf4dbcc4a1ec201518f291c119470cc4e93) * [DURACOM-200] improvement of checker script (cherry picked from commit 5a7c7956cd4e8b47f6a6f53adbc646adeddb0f88) * 108055: isClosed method should use xml configuration (cherry picked from commit d800d800d5346ea9a526ba2a880fc93a6892da98) * [DURACOM-192] Authentication Method related special groups are put in claim set even if a different authentication method is used (cherry picked from commit 6504d749b91508096300e4545069a0554eb5934b) * [DURACOM-192] Added test (cherry picked from commit fa39251071156a6eeb1030000f50761663e128e2) * 107891: Cache administrator group (cherry picked from commit 1e82ca7998c45bd628cd84cefce9ae3f0a0ce046) * DURACOM-199 improved test to show bug related to restricted content (cherry picked from commit dac4df9c1a0b813d2b7578a17c79dd1e9f798a55) * DURACOM-199 fix sitemap generator for restricted content and improve performance (cherry picked from commit 6d9ca388dac3a632530eccbdeda955f7842aae84) * [Port dspace-7_x] Event consumer for submission config reloading when a collection changes (#9196) * initialization with refactoring (cherry picked from commit e93dc1cf4586f684bb441eb0341b4f14393abde5) * also consider SubmissionConfigReaderException (cherry picked from commit 8a04b8775088a82ded154e325a6cb0ce250f1a2e) * rename consumer file (cherry picked from commit f6c92a4c8b25fccc6920646f99e09dd967a2ccf2) * init submission service factory (cherry picked from commit fcc52390b98e3a92927d7a99d9fcf57579a8ed97) * set submissionconfig config settings by default (cherry picked from commit e343d515198950dae7ddb705fee5b34cc7ee0eec) * renaming SubmissionConfigReaderService (cherry picked from commit 9ea7c321c374a0058935efa33e1c2941d6aeb602) * support for SubmissionConfigService (cherry picked from commit 61389fb7876a0757eaac497737aa2cfcd4d4c55a) * fixing style errors and renaming submissionConfigService (cherry picked from commit 5f49491b53c2570fa12bf725a7324e7d7338fc9c) * fixing style errors and unused imports (cherry picked from commit 31d92519508f676659c2efb9e6b9a9089305016d) * set default submission event configs (cherry picked from commit 578198c588f5e7f0da6448f1b4497a041df7fddd) * adding force indexing action to Consumer (cherry picked from commit b91236434f38e8d3510aae816f6e2e871517fb70) * stylecheck fixes (cherry picked from commit 89e89c3d8a7ef8b77a8388cde9905c7fdc1b4bba) * undo event.dispatcher.noindex.consumers (cherry picked from commit ae9dc5f0058f8c7ceb4fb960fc3ca3343fac3763) --------- Co-authored-by: Paulo Graça * Add isNotMemberOf for groups, including unit and integration tests (cherry picked from commit 9d271b24b9721741a53142a690b86287efb738fe) * Implement searchNonMembers for EPersonService. Add tests to prove it works (and tests for search()). Requires minor bug fix to AbstractHibernateDSODAO to allow for additional OR/AND clauses to be appended. (cherry picked from commit f186dcf4ca17f56478ce27946acdc2c269d8bd50) * Add /epersons/search/isNotMemberOf endpoint to REST API along with integration tests (cherry picked from commit 5208a355d69c86dc7cb3ea372656c6959664fd9a) * Bug fix to EPersonDAOImpl. Correctly determine if excluded group needs to be preceded by AND or WHERE (cherry picked from commit e5e0eaa9999a96f499c131e02877d4280f7b5263) * Updated IIIF Controller IT to text bitstream and bundle exclusions (cherry picked from commit e92b4b7bfdc08efab9aee9b8f07506273ee2bfcb) * Simplified the process of fixing the tests after adding new sidebar facets/search filters and sort options to discover.xml (cherry picked from commit b40ad0dfc23040f335d6c6be0fcd0ae6e68a318f) * 107671: Expose the handle.canonical.prefix to the frontend (cherry picked from commit 6d86e65b720b5108e94b1df85e6038394c183214) * Remove line breaks from default.license because they are being rendered in the frontend (cherry picked from commit 56aae347c2a7012af912a8893142fc04809e7ff3) * [Port dspace-7_x] subscription email: do not send email if nothing has changed (#9204) * improved subscriptions email template (cherry picked from commit 6e7b32795930ea3c0758875c654e95b8602cf9b3) * do not send emails without content (cherry picked from commit 926b2421302587d69318f208b48e334cab57b204) * fixed coding style violations (cherry picked from commit fdacec08df8f930ff7c45745a20297a10eff3ad2) * removed unnecessary isEmpty check as suggested by reviewer (cherry picked from commit 30a837f85403332c31761880a0ed936996ba1b5a) * moved null check on indexableObjects in generateBodyMail (cherry picked from commit b43c340b182a62ddc51e3fd7e294101f6d355e90) * fixed unhandled IOException (cherry picked from commit 9b3f7b698c3efbff165fb727ac91609dbe569218) * fixed typo in bodyCommunities (cherry picked from commit ac3d02eb1ce83f9e1fee55f69f1c0fd2c0e03332) * do not use != to compare strings (cherry picked from commit e46018333508215fce7225b75a5af50d4d7beb59) * fixed improper handling of empty list (cherry picked from commit ac72aae44b5ce49ec1a8ddfa2b7986d0d580a8ac) --------- Co-authored-by: Sascha Szott * [maven-release-plugin] prepare release dspace-7.6.1 * Fixed checkstyle issues * Fixed compilation errors * Removed redundant service definitions in the `core-factory-services.xml` because unit tests was failing * Removed redundant bean in the `discovery.xml` * Removed redundant `latestVersion` from `schema.xml` * Fixed wrong resolved conflicts - the app cannot start running * Used ClarinVersionedHandleIdentifierProvider instead of Vanilla one in the `identifier-service.xml` * Fixed tests - the user has an authorization to remove bundle * Fixed failing test in the `MetadataExportSearchIT` it was trying to test dateIssued filter, but the CLARIN DSpace doesn't use that. * controlled.authority.dc.relation is changed in the configuration by CLARIN so the test VersioningWithRelationshipsIT is failing because of that. * Fixed wrong license validation class * Removed test which was accidentaly added durign resolving conflicts * The ACL tag restricted loading of some input field which was missing in the assert * The test cannot convert `Oct` from the xml to normal number because of Slovak default locale which doesn't know what `Oct` means in the date. I set up EN locale for the current test * Added clarin set up of search Facets into integration tests - because src/test/java/org/dspace/app/rest/DiscoveryScopeBasedRestControllerIT.java was failing * Added method `areSpecialGroupsApplicable` into ClarinShibAuthentication because after shib authentication the user wasn't assinged into special group because of some dspace new feature * Change DEFAULT_AMOUNT_FORMATS in the BitstreamFormatRestRepositoryIT.java because vanila has added a new format. * Ignored still failing RegistrationRestRepositoryIT test - it is ignored also in the 7.5 version * fix: Failure of org.dspace.app.rest.SitemapRestControllerIT when running locally * Ignore pubmedImportMetadataGetRecordsTest or allow ssl * Let the pubmedImportMetadataGetRecordsTests ignored * Fixed CurationIT - it failed on the Stream because Stream has null param. Null param was because of null handle on the Site object - the site handle was removed in the `ClarinHandleImportControllerIT.java#createHandleWithoutDSpaceObjectTest`. * Do not delete site handle in the HandleRestRepositoryIT * Updated scripts for fast building 7.6.1 * Moved flyway script for updating checksum_results into separate file * Added missing headers --------- Signed-off-by: dependabot[bot] Co-authored-by: Tim Donohue Co-authored-by: Sean Kalynuk Co-authored-by: Sascha Szott Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Agustina Martinez Co-authored-by: Alan Orth Co-authored-by: Martin Walk Co-authored-by: corrado lombardi Co-authored-by: David P. Steelman Co-authored-by: Christian Bethge <54576195+ChrisBethgster@users.noreply.github.com> Co-authored-by: max.nuding Co-authored-by: Max Nuding Co-authored-by: Mark H. Wood Co-authored-by: Nona Luypaert Co-authored-by: Christian Clauss Co-authored-by: Hrafn Malmquist Co-authored-by: aroman-arvo Co-authored-by: Christian Bethge Co-authored-by: Koen Pauwels Co-authored-by: Mark Cooper Co-authored-by: nwoodward Co-authored-by: Kim Shepherd Co-authored-by: Marie-Hélène Vézina Co-authored-by: damian Co-authored-by: Gantner, Florian Klaus Co-authored-by: wwuck Co-authored-by: Paulo Graça Co-authored-by: Toni Prieto Co-authored-by: mohamed eskander Co-authored-by: Jens Vannerum Co-authored-by: Luca Giamminonni Co-authored-by: Andrea Bollini Co-authored-by: DSpace Bot <68393067+dspace-bot@users.noreply.github.com> Co-authored-by: Michael Spalti Co-authored-by: Alexandre Vryghem Co-authored-by: Shankeerthan Kasilingam * The cas.init() method is called in the VocabularyRestRepositoryIT#setup() method * Update docker.yml (#502) dspace-dependencies job was missing * Update docker.yml (#503) Fixed dspace-dependencies error * Update docker.yml (#504) * Empty commit - run actions * use checkstyle before commit * remove fix duplicate dependency * remove discofeed related fetching from obsolete discojuice servers (#508) * remove discofeed related fetching from obsolete discojuice servers * check style, turned off in testing, turned of by default * fix failing test --------- Co-authored-by: jm * ufal/cannot-upload-big-file (#533) * Do not compute localChecksum because it takes too long for the big files. The checksum is computed using DigestInputStream object. Copy content into sync local assetstore using buffered copying. * Fixed checkstyle issue * Updated comment about creating required directories * ufal/license-agreement-wrong-behaviour (#534) * The all transactions are not updated in the user metadata after a new download.. * Return the user metadata depending on user registration, bitstream, mapping and the last translation. * Fixed checkstyle issues * Fixed failing integration tests - do not fetch user metadata for testing by ID where there are more tests which creates the user metadata * internal/discofeed-it (#535) * Disable SSL check for specific discofeed requests * Fixed checkstyle issues * Do not throw error in private constructor * Fixed failing IT - ssl check wasn't disabled * Ignore integration test which is trying to connect to our private network. * allow running from branch rework-actions for testing * ufal/s3-check-etag (#537) * Temp commit - upload a file by uploading 50MB parts * Added uploading the file by parts - for every part is computed checksum and compared with UploadPartResult's ETag. This feature could be enabled or disabled by cfg. * Undo S3BitStoreService changes * Fixed checkstyle issues * Prettify the code * Changed cfg property to be better understandable and file size is converted into constant. * ufal/allow-impersonating Allow impersonating by default (#539) * correct branches for main gh actions * Rework actions (#540) * run correct actions on correct repos * remove old docker job, separating it to several * reusable build yml from upstream * upstream docker.yml version * first test run * steppify docker.yml action to be able to add more steps * added python version to reusable build * fix incorrect usage of steps in gh action * corect the type in reusable wf * correct condition in python version script * corrected the branch * corrected image dockerhub owner * no linux/arm64 * comment out unused tasks; minor corrections * do not redepoy becuse jm said so * do not use codecov * disable deploy differently * extend possibilities for using version script (#542) * parametrized output of python version script * RUUUN * correct version placement * correct branches for run of gh action * Fix all mistakes in the /home page (#543) * Configured browsing by language * Updated tests following the browsing cfg * reusable run provides links to run and it's ID * first attempt at tag release action * also release images under github.sha * Improve version info, add release-tag action add run link to version info correct image for tests (remove dspace-dependencies from migrate to 7.5) add action to release on tagging * add necessary option to tag-action * Fixed all mistakes in the Item View page (#551) * Remove the xml info and bibtex tag from the citation refbox * Show handle and DOI in the Item View following the new cfg property * Made updating xml string more generic * Updated the method name for the getting text content from the bibtex xml. * Fixed all mistakes in the Search page (#549) * File size in the MetadataBitstreamWrapperRest changed to `long` type instead of String, the FE transfer all bytes into readable form. * Changed default sorting options * Added a new search configuration for the homepage, because searching by `dc.date.accesioned` must be removed because of clarin requirements, but the homepage uses searching by `dc.date.accesioned` * Configured searching by Item Type in the search page. * Updated community indexing - added _filter and _keyword because they are needed to work as facet and filter * Updated `Rights` and `Community` configuration. Changed to `text` type. * Allow Site usage reports for the Anonymous user * Fixed MetadataBitstreamRestRepositoryIT - the Matcher has a problem to compare long types. * Fixed DiscoveryScopeBasedRestControllerIT - the expected facets and Matchers did must be updated. * Fixed ClarinDiscoveryRestControllerIT - the expected facets and Matchers did must be updated. * Use `dc.contributor.other` metadata field for mapping the authors (#557) * Added `edm.xml` registry (#559) * Added `edm.xml` registry * Fixed cfg for `edm.xml` * Updated shibboleth cfg properties (firstname, lastname, netid, email) (#558) * Updated shibboleth cfg properties (firstname, lastname, netid, email) * Keep vanilla shib headers for the integration tests * Removed duplicated code --------- Signed-off-by: dependabot[bot] Co-authored-by: MajoBerger <88670521+MajoBerger@users.noreply.github.com> Co-authored-by: MajoBerger Co-authored-by: Adán Román Ruiz Co-authored-by: Ondřej Košarko Co-authored-by: Tim Donohue Co-authored-by: Luca Giamminonni Co-authored-by: Sean Kalynuk Co-authored-by: Sascha Szott Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Agustina Martinez Co-authored-by: Alan Orth Co-authored-by: Martin Walk Co-authored-by: corrado lombardi Co-authored-by: David P. Steelman Co-authored-by: Christian Bethge <54576195+ChrisBethgster@users.noreply.github.com> Co-authored-by: max.nuding Co-authored-by: Max Nuding Co-authored-by: Mark H. Wood Co-authored-by: Nona Luypaert Co-authored-by: Christian Clauss Co-authored-by: Hrafn Malmquist Co-authored-by: Christian Bethge Co-authored-by: Koen Pauwels Co-authored-by: Mark Cooper Co-authored-by: nwoodward Co-authored-by: Kim Shepherd Co-authored-by: Marie-Hélène Vézina Co-authored-by: damian Co-authored-by: Gantner, Florian Klaus Co-authored-by: wwuck Co-authored-by: Paulo Graça Co-authored-by: Toni Prieto Co-authored-by: mohamed eskander Co-authored-by: Jens Vannerum Co-authored-by: Andrea Bollini Co-authored-by: DSpace Bot <68393067+dspace-bot@users.noreply.github.com> Co-authored-by: Michael Spalti Co-authored-by: Alexandre Vryghem Co-authored-by: Shankeerthan Kasilingam Co-authored-by: jm Co-authored-by: Jozef Misutka <332350+vidiecan@users.noreply.github.com> --- .../ClarinSolrItemsCommunityIndexPlugin.java | 4 + .../test/data/dspaceFolder/config/local.cfg | 4 + .../app/rest/ClarinRefBoxController.java | 74 +++++- .../MetadataBitstreamWrapperConverter.java | 4 +- .../model/MetadataBitstreamWrapperRest.java | 8 +- ...geReportRestPermissionEvaluatorPlugin.java | 10 + .../rest/ClarinDiscoveryRestControllerIT.java | 99 ++++--- .../DiscoveryScopeBasedRestControllerIT.java | 9 +- .../MetadataBitstreamRestRepositoryIT.java | 9 +- .../app/rest/StatisticsRestRepositoryIT.java | 19 ++ .../app/rest/matcher/FacetEntryMatcher.java | 4 +- .../app/rest/matcher/SearchFilterMatcher.java | 6 +- dspace/config/clarin-dspace.cfg | 8 + dspace/config/dspace.cfg | 1 + .../modules/authentication-shibboleth.cfg | 8 +- dspace/config/modules/rest.cfg | 1 + dspace/config/registries/edm.xml | 17 ++ dspace/config/spring/api/discovery.xml | 242 ++++++++++++++++-- 18 files changed, 444 insertions(+), 83 deletions(-) create mode 100644 dspace/config/registries/edm.xml diff --git a/dspace-api/src/main/java/org/dspace/discovery/ClarinSolrItemsCommunityIndexPlugin.java b/dspace-api/src/main/java/org/dspace/discovery/ClarinSolrItemsCommunityIndexPlugin.java index fe419cb6d2ff..36e60e3af816 100644 --- a/dspace-api/src/main/java/org/dspace/discovery/ClarinSolrItemsCommunityIndexPlugin.java +++ b/dspace-api/src/main/java/org/dspace/discovery/ClarinSolrItemsCommunityIndexPlugin.java @@ -39,7 +39,11 @@ public void additionalIndex(Context context, IndexableObject indexableObject, So Community owningCommunity = clarinItemService.getOwningCommunity(context, item); String communityName = Objects.isNull(owningCommunity) ? " " : owningCommunity.getName(); + // _keyword and _filter because + // they are needed in order to work as a facet and filter. document.addField("items_owning_community", communityName); + document.addField("items_owning_community_keyword", communityName); + document.addField("items_owning_community_filter", communityName); } } } diff --git a/dspace-api/src/test/data/dspaceFolder/config/local.cfg b/dspace-api/src/test/data/dspaceFolder/config/local.cfg index 8a9decfa3318..a842905c53ab 100644 --- a/dspace-api/src/test/data/dspaceFolder/config/local.cfg +++ b/dspace-api/src/test/data/dspaceFolder/config/local.cfg @@ -219,6 +219,10 @@ featured.service.teitok.description = A web-based platform for viewing, creating ##### Shibboleth ##### +authentication-shibboleth.netid-header = SHIB-NETID +authentication-shibboleth.email-header = SHIB-MAIL +authentication-shibboleth.firstname-header = SHIB-GIVENNAME +authentication-shibboleth.lastname-header = SHIB-SURNAME # Turn off the discofeed, it is allowed by default shibboleth.discofeed.allowed = false # File where is DiscoJuiceFeed response diff --git a/dspace-server-webapp/src/main/java/org/dspace/app/rest/ClarinRefBoxController.java b/dspace-server-webapp/src/main/java/org/dspace/app/rest/ClarinRefBoxController.java index 8c8490f1d53e..710a96648152 100644 --- a/dspace-server-webapp/src/main/java/org/dspace/app/rest/ClarinRefBoxController.java +++ b/dspace-server-webapp/src/main/java/org/dspace/app/rest/ClarinRefBoxController.java @@ -21,10 +21,15 @@ import java.util.Map; import java.util.Objects; import java.util.UUID; +import java.util.regex.Matcher; +import java.util.regex.Pattern; import javax.servlet.ServletException; import javax.servlet.http.HttpServletRequest; import javax.servlet.http.HttpServletResponse; import javax.ws.rs.NotFoundException; +import javax.xml.parsers.DocumentBuilder; +import javax.xml.parsers.DocumentBuilderFactory; +import javax.xml.parsers.ParserConfigurationException; import javax.xml.stream.XMLStreamException; import com.hp.hpl.jena.rdf.model.Model; @@ -67,6 +72,9 @@ import org.springframework.web.bind.annotation.RequestMethod; import org.springframework.web.bind.annotation.RequestParam; import org.springframework.web.bind.annotation.RestController; +import org.w3c.dom.Document; +import org.w3c.dom.Node; +import org.xml.sax.SAXException; /** * A Controller for fetching the data for the ref-box in the Item View (FE). @@ -77,6 +85,9 @@ @RestController @RequestMapping("/api/core/refbox") public class ClarinRefBoxController { + + private final static String BIBTEX_TYPE = "bibtex"; + private final Logger log = org.apache.logging.log4j.LogManager.getLogger(ClarinRefBoxController.class); @Autowired @@ -228,7 +239,7 @@ public ResponseEntity getCitationText(@RequestParam(name = "type") String type, XmlOutputContext xmlOutContext = XmlOutputContext.emptyContext(output); xmlOutContext.getWriter().writeStartDocument(); - //Try to obtain just the metadata, if that fails return "normal" response + // Try to obtain just the metadata, if that fails return "normal" response try { oaipmh.getInfo().getGetRecord().getRecord().getMetadata().write(xmlOutContext); } catch (Exception e) { @@ -246,8 +257,10 @@ public ResponseEntity getCitationText(@RequestParam(name = "type") String type, response.sendError(HttpServletResponse.SC_INTERNAL_SERVER_ERROR, "Unexpected error while writing the output. For more information visit the log files."); } catch (XOAIManagerResolverException e) { - throw new ServletException("OAI 2.0 wasn't correctly initialized," + - " please check the log for previous errors", e); + String errMessage = "OAI 2.0 wasn't correctly initialized, please check the log for previous errors. " + + "Error message: " + e.getMessage(); + log.error(errMessage); + throw new ServletException(errMessage); } catch (OAIException e) { response.sendError(HttpServletResponse.SC_INTERNAL_SERVER_ERROR, "Unexpected error. For more information visit the log files."); @@ -263,8 +276,11 @@ public ResponseEntity getCitationText(@RequestParam(name = "type") String type, HttpStatus.valueOf(HttpServletResponse.SC_NO_CONTENT)); } + // Update the output string and remove the unwanted parts. + String outputString = updateOutput(type, output.toString()); + // Wrap the String output to the class for better parsing in the FE - OaiMetadataWrapper oaiMetadataWrapper = new OaiMetadataWrapper(output.toString()); + OaiMetadataWrapper oaiMetadataWrapper = new OaiMetadataWrapper(StringUtils.defaultIfEmpty(outputString, "")); return new ResponseEntity<>(oaiMetadataWrapper, HttpStatus.valueOf(SC_OK)); } @@ -331,6 +347,56 @@ private List mapFeaturedServiceLinks(List tag from the string. + return getXmlTextContent(output); + } else { + // Remove the XML header tag from the string. + return removeXmlHeaderTag(output); + } + } catch (Exception e) { + log.error("Cannot update the xml string for citation because of: " + e.getMessage()); + return null; + } + } + + /** + * Remove the XML header tag from the string. + * + * @param xml + * @return + */ + private String removeXmlHeaderTag(String xml) { + String xmlHeaderPattern = "<\\?xml[^>]*\\?>"; + Pattern xmlHeaderRegex = Pattern.compile(xmlHeaderPattern); + Matcher xmlHeaderMatcher = xmlHeaderRegex.matcher(xml); + if (xmlHeaderMatcher.find()) { + return xml.replaceFirst(xmlHeaderPattern, ""); + } else { + return xml; + } + } + + /** + * Get the text content from the xml string. + */ + private String getXmlTextContent(String xml) throws ParserConfigurationException, IOException, SAXException { + // Parse the XML string + DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance(); + DocumentBuilder builder = factory.newDocumentBuilder(); + Document document = builder.parse(new org.xml.sax.InputSource(new java.io.StringReader(xml))); + + // Get the root element + Node root = document.getDocumentElement(); + + // Get the text content of the root element + return root.getTextContent().trim(); + } } /** diff --git a/dspace-server-webapp/src/main/java/org/dspace/app/rest/converter/MetadataBitstreamWrapperConverter.java b/dspace-server-webapp/src/main/java/org/dspace/app/rest/converter/MetadataBitstreamWrapperConverter.java index 49728da0b899..e42c023f72a5 100644 --- a/dspace-server-webapp/src/main/java/org/dspace/app/rest/converter/MetadataBitstreamWrapperConverter.java +++ b/dspace-server-webapp/src/main/java/org/dspace/app/rest/converter/MetadataBitstreamWrapperConverter.java @@ -10,7 +10,6 @@ import org.dspace.app.rest.model.MetadataBitstreamWrapperRest; import org.dspace.app.rest.model.wrapper.MetadataBitstreamWrapper; import org.dspace.app.rest.projection.Projection; -import org.dspace.util.FileTreeViewGenerator; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.context.annotation.Lazy; import org.springframework.stereotype.Component; @@ -41,8 +40,7 @@ public MetadataBitstreamWrapperRest convert(MetadataBitstreamWrapper modelObject bitstreamWrapperRest.setId(modelObject.getBitstream().getID().toString()); bitstreamWrapperRest.setDescription(modelObject.getDescription()); bitstreamWrapperRest.setChecksum(modelObject.getBitstream().getChecksum()); - bitstreamWrapperRest.setFileSize(FileTreeViewGenerator.humanReadableFileSize( - modelObject.getBitstream().getSizeBytes())); + bitstreamWrapperRest.setFileSize(modelObject.getBitstream().getSizeBytes()); bitstreamWrapperRest.setFileInfo(modelObject.getFileInfo()); bitstreamWrapperRest.setHref(modelObject.getHref()); bitstreamWrapperRest.setFormat(modelObject.getFormat()); diff --git a/dspace-server-webapp/src/main/java/org/dspace/app/rest/model/MetadataBitstreamWrapperRest.java b/dspace-server-webapp/src/main/java/org/dspace/app/rest/model/MetadataBitstreamWrapperRest.java index 7e3c1c03568e..8e4d60d4bfab 100644 --- a/dspace-server-webapp/src/main/java/org/dspace/app/rest/model/MetadataBitstreamWrapperRest.java +++ b/dspace-server-webapp/src/main/java/org/dspace/app/rest/model/MetadataBitstreamWrapperRest.java @@ -23,14 +23,14 @@ public class MetadataBitstreamWrapperRest extends BaseObjectRest { private String name; private String description; - private String fileSize; + private long fileSize; private String checksum; private List fileInfo; private String format; private String href; private boolean canPreview; - public MetadataBitstreamWrapperRest(String name, String description, String fileSize, String checksum, + public MetadataBitstreamWrapperRest(String name, String description, long fileSize, String checksum, List fileInfo, String format, String href, boolean canPreview) { this.name = name; this.description = description; @@ -93,11 +93,11 @@ public void setDescription(String description) { this.description = description; } - public String getFileSize() { + public long getFileSize() { return fileSize; } - public void setFileSize(String fileSize) { + public void setFileSize(long fileSize) { this.fileSize = fileSize; } diff --git a/dspace-server-webapp/src/main/java/org/dspace/app/rest/security/UsageReportRestPermissionEvaluatorPlugin.java b/dspace-server-webapp/src/main/java/org/dspace/app/rest/security/UsageReportRestPermissionEvaluatorPlugin.java index 06254d561ace..8e0b57e6791a 100644 --- a/dspace-server-webapp/src/main/java/org/dspace/app/rest/security/UsageReportRestPermissionEvaluatorPlugin.java +++ b/dspace-server-webapp/src/main/java/org/dspace/app/rest/security/UsageReportRestPermissionEvaluatorPlugin.java @@ -18,6 +18,7 @@ import org.dspace.app.rest.utils.DSpaceObjectUtils; import org.dspace.authorize.service.AuthorizeService; import org.dspace.content.DSpaceObject; +import org.dspace.content.Site; import org.dspace.core.Context; import org.dspace.services.ConfigurationService; import org.dspace.services.RequestService; @@ -93,6 +94,15 @@ public boolean hasDSpacePermission(Authentication authentication, Serializable t if (Objects.isNull(dso)) { return true; } + + // Show Site usage reports to anonymous users if the configuration is set to do so + // This Site usage reports are used in the home page and are not sensitive + if (dso instanceof Site) { + if (!configurationService.getBooleanProperty("site.usage-reports.enable.auth.anonymous", + false)) { + return true; + } + } return authorizeService.authorizeActionBoolean(context, dso, restPermission.getDspaceApiActionId()); } catch (SQLException e) { log.error(e.getMessage(), e); diff --git a/dspace-server-webapp/src/test/java/org/dspace/app/rest/ClarinDiscoveryRestControllerIT.java b/dspace-server-webapp/src/test/java/org/dspace/app/rest/ClarinDiscoveryRestControllerIT.java index 72b12c7962af..460373091d27 100644 --- a/dspace-server-webapp/src/test/java/org/dspace/app/rest/ClarinDiscoveryRestControllerIT.java +++ b/dspace-server-webapp/src/test/java/org/dspace/app/rest/ClarinDiscoveryRestControllerIT.java @@ -122,7 +122,8 @@ public void discoverFacetsTestWithoutParameters() throws Exception { FacetEntryMatcher.clarinLicenseRightsFacet(false), FacetEntryMatcher.clarinItemsLanguageFacet(false), FacetEntryMatcher.clarinItemsCommunityFacet(false), - FacetEntryMatcher.hasContentInOriginalBundleFacet(false))) + FacetEntryMatcher.hasContentInOriginalBundleFacet(false), + FacetEntryMatcher.typeFacet(false))) ); } @@ -1006,9 +1007,11 @@ public void discoverSearchTest() throws Exception { SortOptionMatcher.sortOptionMatcher( "dc.title", DiscoverySortFieldConfiguration.SORT_ORDER.asc.name()), SortOptionMatcher.sortOptionMatcher( - "dc.date.issued", DiscoverySortFieldConfiguration.SORT_ORDER.desc.name()), + "dc.title", DiscoverySortFieldConfiguration.SORT_ORDER.desc.name()), + SortOptionMatcher.sortOptionMatcher( + "dc.date.issued", DiscoverySortFieldConfiguration.SORT_ORDER.asc.name()), SortOptionMatcher.sortOptionMatcher( - "dc.date.accessioned", DiscoverySortFieldConfiguration.SORT_ORDER.desc.name()) + "dc.date.issued", DiscoverySortFieldConfiguration.SORT_ORDER.desc.name()) ))); } @@ -1146,7 +1149,8 @@ public void discoverSearchObjectsTest() throws Exception { FacetEntryMatcher.clarinLicenseRightsFacet(false), FacetEntryMatcher.clarinItemsLanguageFacet(false), FacetEntryMatcher.clarinItemsCommunityFacet(false), - FacetEntryMatcher.hasContentInOriginalBundleFacet(false) + FacetEntryMatcher.hasContentInOriginalBundleFacet(false), + FacetEntryMatcher.typeFacet(false) ))) //There always needs to be a self link .andExpect(jsonPath("$._links.self.href", containsString("/api/discover/search/objects"))) @@ -1287,7 +1291,8 @@ public void discoverSearchObjectsTestHasMoreAuthorFacet() throws Exception { FacetEntryMatcher.clarinLicenseRightsFacet(false), FacetEntryMatcher.clarinItemsLanguageFacet(false), FacetEntryMatcher.clarinItemsCommunityFacet(false), - FacetEntryMatcher.hasContentInOriginalBundleFacet(false) + FacetEntryMatcher.hasContentInOriginalBundleFacet(false), + FacetEntryMatcher.typeFacet(false) ))) //There always needs to be a self link available .andExpect(jsonPath("$._links.self.href", containsString("/api/discover/search/objects"))) @@ -1379,7 +1384,8 @@ public void discoverSearchObjectsTestHasMoreSubjectFacet() throws Exception { FacetEntryMatcher.clarinLicenseRightsFacet(false), FacetEntryMatcher.clarinItemsLanguageFacet(false), FacetEntryMatcher.clarinItemsCommunityFacet(false), - FacetEntryMatcher.hasContentInOriginalBundleFacet(false) + FacetEntryMatcher.hasContentInOriginalBundleFacet(false), + FacetEntryMatcher.typeFacet(false) ))) //There always needs to be a self link available .andExpect(jsonPath("$._links.self.href", containsString("/api/discover/search/objects"))) @@ -1461,7 +1467,8 @@ public void discoverSearchObjectsTestWithBasicQuery() throws Exception { FacetEntryMatcher.clarinLicenseRightsFacet(false), FacetEntryMatcher.clarinItemsLanguageFacet(false), FacetEntryMatcher.clarinItemsCommunityFacet(false), - FacetEntryMatcher.hasContentInOriginalBundleFacet(false) + FacetEntryMatcher.hasContentInOriginalBundleFacet(false), + FacetEntryMatcher.typeFacet(false) ))) //There always needs to be a self link available .andExpect(jsonPath("$._links.self.href", containsString("/api/discover/search/objects"))) @@ -1571,7 +1578,8 @@ public void discoverSearchObjectsTestWithScope() throws Exception { FacetEntryMatcher.clarinLicenseRightsFacet(false), FacetEntryMatcher.clarinItemsLanguageFacet(false), FacetEntryMatcher.clarinItemsCommunityFacet(false), - FacetEntryMatcher.hasContentInOriginalBundleFacet(false) + FacetEntryMatcher.hasContentInOriginalBundleFacet(false), + FacetEntryMatcher.typeFacet(false) ))) //There always needs to be a self link available .andExpect(jsonPath("$._links.self.href", containsString("/api/discover/search/objects"))) @@ -1653,7 +1661,8 @@ public void discoverSearchObjectsTestWithDsoType() throws Exception { FacetEntryMatcher.clarinLicenseRightsFacet(false), FacetEntryMatcher.clarinItemsLanguageFacet(false), FacetEntryMatcher.clarinItemsCommunityFacet(false), - FacetEntryMatcher.hasContentInOriginalBundleFacet(false) + FacetEntryMatcher.hasContentInOriginalBundleFacet(false), + FacetEntryMatcher.typeFacet(false) ))) //There always needs to be a self link available .andExpect(jsonPath("$._links.self.href", containsString("/api/discover/search/objects"))); @@ -1690,7 +1699,8 @@ public void discoverSearchObjectsTestWithDsoType() throws Exception { FacetEntryMatcher.clarinLicenseRightsFacet(false), FacetEntryMatcher.clarinItemsLanguageFacet(false), FacetEntryMatcher.clarinItemsCommunityFacet(false), - FacetEntryMatcher.hasContentInOriginalBundleFacet(false) + FacetEntryMatcher.hasContentInOriginalBundleFacet(false), + FacetEntryMatcher.typeFacet(false) ))) //There always needs to be a self link available .andExpect(jsonPath("$._links.self.href", containsString("/api/discover/search/objects"))); @@ -1728,7 +1738,8 @@ public void discoverSearchObjectsTestWithDsoType() throws Exception { FacetEntryMatcher.clarinLicenseRightsFacet(false), FacetEntryMatcher.clarinItemsLanguageFacet(false), FacetEntryMatcher.clarinItemsCommunityFacet(false), - FacetEntryMatcher.hasContentInOriginalBundleFacet(false) + FacetEntryMatcher.hasContentInOriginalBundleFacet(false), + FacetEntryMatcher.typeFacet(false) ))) //There always needs to be a self link available .andExpect(jsonPath("$._links.self.href", containsString("/api/discover/search/objects"))); @@ -1770,7 +1781,8 @@ public void discoverSearchObjectsTestWithDsoType() throws Exception { FacetEntryMatcher.clarinLicenseRightsFacet(false), FacetEntryMatcher.clarinItemsLanguageFacet(false), FacetEntryMatcher.clarinItemsCommunityFacet(false), - FacetEntryMatcher.hasContentInOriginalBundleFacet(false) + FacetEntryMatcher.hasContentInOriginalBundleFacet(false), + FacetEntryMatcher.typeFacet(false) ))) //There always needs to be a self link available .andExpect(jsonPath("$._links.self.href", containsString("/api/discover/search/objects"))); @@ -1859,7 +1871,8 @@ public void discoverSearchObjectsTestWithDsoTypeAndSort() throws Exception { FacetEntryMatcher.clarinLicenseRightsFacet(false), FacetEntryMatcher.clarinItemsLanguageFacet(false), FacetEntryMatcher.clarinItemsCommunityFacet(false), - FacetEntryMatcher.hasContentInOriginalBundleFacet(false) + FacetEntryMatcher.hasContentInOriginalBundleFacet(false), + FacetEntryMatcher.typeFacet(false) ))) //We want to get the sort that's been used as well in the response .andExpect(jsonPath("$.sort", is( @@ -2073,7 +2086,8 @@ public void discoverSearchObjectsTestForPaginationAndNextLinks() throws Exceptio FacetEntryMatcher.clarinItemsLanguageFacet(false), FacetEntryMatcher.clarinItemsCommunityFacet(false), FacetEntryMatcher.hasContentInOriginalBundleFacet(false), - FacetEntryMatcher.entityTypeFacet(false) + FacetEntryMatcher.entityTypeFacet(false), + FacetEntryMatcher.typeFacet(false) ))) //There always needs to be a self link available .andExpect(jsonPath("$._links.self.href", containsString("/api/discover/search/objects"))) @@ -2165,7 +2179,8 @@ public void discoverSearchObjectsTestWithContentInABitstream() throws Exception FacetEntryMatcher.clarinLicenseRightsFacet(false), FacetEntryMatcher.clarinItemsLanguageFacet(false), FacetEntryMatcher.clarinItemsCommunityFacet(false), - FacetEntryMatcher.hasContentInOriginalBundleFacet(false) + FacetEntryMatcher.hasContentInOriginalBundleFacet(false), + FacetEntryMatcher.typeFacet(false) ))) //There always needs to be a self link available .andExpect(jsonPath("$._links.self.href", containsString("/api/discover/search/objects"))) @@ -2257,7 +2272,8 @@ public void discoverSearchObjectsTestForEmbargoedItemsAndPrivateItems() throws E FacetEntryMatcher.clarinLicenseRightsFacet(false), FacetEntryMatcher.clarinItemsLanguageFacet(false), FacetEntryMatcher.clarinItemsCommunityFacet(false), - FacetEntryMatcher.hasContentInOriginalBundleFacet(false) + FacetEntryMatcher.hasContentInOriginalBundleFacet(false), + FacetEntryMatcher.typeFacet(false) ))) //There always needs to be a self link available .andExpect(jsonPath("$._links.self.href", containsString("/api/discover/search/objects"))) @@ -2343,7 +2359,8 @@ public void discoverSearchObjectsTestWithContentInAPrivateBitstream() throws Exc FacetEntryMatcher.clarinLicenseRightsFacet(false), FacetEntryMatcher.clarinItemsLanguageFacet(false), FacetEntryMatcher.clarinItemsCommunityFacet(false), - FacetEntryMatcher.hasContentInOriginalBundleFacet(false) + FacetEntryMatcher.hasContentInOriginalBundleFacet(false), + FacetEntryMatcher.typeFacet(false) ))) //There always needs to be a self link available .andExpect(jsonPath("$._links.self.href", containsString("/api/discover/search/objects"))) @@ -2424,7 +2441,8 @@ public void discoverSearchObjectsTestForScope() throws Exception { FacetEntryMatcher.clarinLicenseRightsFacet(false), FacetEntryMatcher.clarinItemsLanguageFacet(false), FacetEntryMatcher.clarinItemsCommunityFacet(false), - FacetEntryMatcher.hasContentInOriginalBundleFacet(false) + FacetEntryMatcher.hasContentInOriginalBundleFacet(false), + FacetEntryMatcher.typeFacet(false) ))) //There always needs to be a self link available .andExpect(jsonPath("$._links.self.href", containsString("/api/discover/search/objects"))) @@ -2511,7 +2529,8 @@ public void discoverSearchObjectsTestForScopeWithPrivateItem() throws Exception FacetEntryMatcher.clarinLicenseRightsFacet(false), FacetEntryMatcher.clarinItemsLanguageFacet(false), FacetEntryMatcher.clarinItemsCommunityFacet(false), - FacetEntryMatcher.hasContentInOriginalBundleFacet(false) + FacetEntryMatcher.hasContentInOriginalBundleFacet(false), + FacetEntryMatcher.typeFacet(false) ))) //There always needs to be a self link available .andExpect(jsonPath("$._links.self.href", containsString("/api/discover/search/objects"))) @@ -2693,7 +2712,8 @@ public void discoverSearchObjectsTestForHitHighlights() throws Exception { FacetEntryMatcher.clarinLicenseRightsFacet(false), FacetEntryMatcher.clarinItemsLanguageFacet(false), FacetEntryMatcher.clarinItemsCommunityFacet(false), - FacetEntryMatcher.hasContentInOriginalBundleFacet(false) + FacetEntryMatcher.hasContentInOriginalBundleFacet(false), + FacetEntryMatcher.typeFacet(false) ))) //There always needs to be a self link available .andExpect(jsonPath("$._links.self.href", containsString("/api/discover/search/objects"))) @@ -2839,7 +2859,8 @@ public void discoverSearchObjectsWithQueryOperatorContains_query() throws Except FacetEntryMatcher.clarinLicenseRightsFacet(false), FacetEntryMatcher.clarinItemsLanguageFacet(false), FacetEntryMatcher.clarinItemsCommunityFacet(false), - FacetEntryMatcher.hasContentInOriginalBundleFacet(false) + FacetEntryMatcher.hasContentInOriginalBundleFacet(false), + FacetEntryMatcher.typeFacet(false) ))) //There always needs to be a self link available .andExpect(jsonPath("$._links.self.href", containsString("/api/discover/search/objects"))) @@ -2917,7 +2938,8 @@ public void discoverSearchObjectsWithQueryOperatorContains() throws Exception { FacetEntryMatcher.clarinLicenseRightsFacet(false), FacetEntryMatcher.clarinItemsLanguageFacet(false), FacetEntryMatcher.clarinItemsCommunityFacet(false), - FacetEntryMatcher.hasContentInOriginalBundleFacet(false) + FacetEntryMatcher.hasContentInOriginalBundleFacet(false), + FacetEntryMatcher.typeFacet(false) ))) //There always needs to be a self link available .andExpect(jsonPath("$._links.self.href", containsString("/api/discover/search/objects"))) @@ -2993,7 +3015,8 @@ public void discoverSearchObjectsWithQueryOperatorNotContains_query() throws Exc FacetEntryMatcher.clarinLicenseRightsFacet(false), FacetEntryMatcher.clarinItemsLanguageFacet(false), FacetEntryMatcher.clarinItemsCommunityFacet(false), - FacetEntryMatcher.hasContentInOriginalBundleFacet(false) + FacetEntryMatcher.hasContentInOriginalBundleFacet(false), + FacetEntryMatcher.typeFacet(false) ))) //There always needs to be a self link available .andExpect(jsonPath("$._links.self.href", containsString("/api/discover/search/objects"))) @@ -3070,7 +3093,8 @@ public void discoverSearchObjectsWithQueryOperatorNotContains() throws Exception FacetEntryMatcher.clarinLicenseRightsFacet(false), FacetEntryMatcher.clarinItemsLanguageFacet(false), FacetEntryMatcher.clarinItemsCommunityFacet(false), - FacetEntryMatcher.hasContentInOriginalBundleFacet(false) + FacetEntryMatcher.hasContentInOriginalBundleFacet(false), + FacetEntryMatcher.typeFacet(false) ))) //There always needs to be a self link available .andExpect(jsonPath("$._links.self.href", containsString("/api/discover/search/objects"))) @@ -3157,7 +3181,8 @@ public void discoverSearchObjectsTestForMinMaxValues() throws Exception { FacetEntryMatcher.clarinLicenseRightsFacet(false), FacetEntryMatcher.clarinItemsLanguageFacet(false), FacetEntryMatcher.clarinItemsCommunityFacet(false), - FacetEntryMatcher.hasContentInOriginalBundleFacet(false) + FacetEntryMatcher.hasContentInOriginalBundleFacet(false), + FacetEntryMatcher.typeFacet(false) ))) //There always needs to be a self link available .andExpect(jsonPath("$._links.self.href", containsString("/api/discover/search/objects"))) @@ -3228,7 +3253,8 @@ public void discoverSearchFacetsTestForMinMaxValues() throws Exception { FacetEntryMatcher.clarinLicenseRightsFacet(false), FacetEntryMatcher.clarinItemsLanguageFacet(false), FacetEntryMatcher.clarinItemsCommunityFacet(false), - FacetEntryMatcher.hasContentInOriginalBundleFacet(false) + FacetEntryMatcher.hasContentInOriginalBundleFacet(false), + FacetEntryMatcher.typeFacet(false) ))) //There always needs to be a self link available .andExpect(jsonPath("$._links.self.href", containsString("/api/discover/search/facets"))) @@ -3304,7 +3330,8 @@ public void discoverSearchObjectsWithQueryOperatorEquals_query() throws Exceptio FacetEntryMatcher.clarinLicenseRightsFacet(false), FacetEntryMatcher.clarinItemsLanguageFacet(false), FacetEntryMatcher.clarinItemsCommunityFacet(false), - FacetEntryMatcher.hasContentInOriginalBundleFacet(false) + FacetEntryMatcher.hasContentInOriginalBundleFacet(false), + FacetEntryMatcher.typeFacet(false) ))) //There always needs to be a self link available .andExpect(jsonPath("$._links.self.href", containsString("/api/discover/search/objects"))) @@ -3381,7 +3408,8 @@ public void discoverSearchObjectsWithQueryOperatorEquals() throws Exception { FacetEntryMatcher.clarinLicenseRightsFacet(false), FacetEntryMatcher.clarinItemsLanguageFacet(false), FacetEntryMatcher.clarinItemsCommunityFacet(false), - FacetEntryMatcher.hasContentInOriginalBundleFacet(false) + FacetEntryMatcher.hasContentInOriginalBundleFacet(false), + FacetEntryMatcher.typeFacet(false) ))) //There always needs to be a self link available .andExpect(jsonPath("$._links.self.href", containsString("/api/discover/search/objects"))) @@ -3458,7 +3486,8 @@ public void discoverSearchObjectsWithQueryOperatorNotEquals_query() throws Excep FacetEntryMatcher.clarinLicenseRightsFacet(false), FacetEntryMatcher.clarinItemsLanguageFacet(false), FacetEntryMatcher.clarinItemsCommunityFacet(false), - FacetEntryMatcher.hasContentInOriginalBundleFacet(false) + FacetEntryMatcher.hasContentInOriginalBundleFacet(false), + FacetEntryMatcher.typeFacet(false) ))) //There always needs to be a self link available .andExpect(jsonPath("$._links.self.href", containsString("/api/discover/search/objects"))) @@ -3536,7 +3565,8 @@ public void discoverSearchObjectsWithQueryOperatorNotEquals() throws Exception { FacetEntryMatcher.clarinLicenseRightsFacet(false), FacetEntryMatcher.clarinItemsLanguageFacet(false), FacetEntryMatcher.clarinItemsCommunityFacet(false), - FacetEntryMatcher.hasContentInOriginalBundleFacet(false) + FacetEntryMatcher.hasContentInOriginalBundleFacet(false), + FacetEntryMatcher.typeFacet(false) ))) //There always needs to be a self link available .andExpect(jsonPath("$._links.self.href", containsString("/api/discover/search/objects"))) @@ -3612,7 +3642,8 @@ public void discoverSearchObjectsWithQueryOperatorNotAuthority_query() throws Ex FacetEntryMatcher.clarinLicenseRightsFacet(false), FacetEntryMatcher.clarinItemsLanguageFacet(false), FacetEntryMatcher.clarinItemsCommunityFacet(false), - FacetEntryMatcher.hasContentInOriginalBundleFacet(false) + FacetEntryMatcher.hasContentInOriginalBundleFacet(false), + FacetEntryMatcher.typeFacet(false) ))) //There always needs to be a self link available .andExpect(jsonPath("$._links.self.href", containsString("/api/discover/search/objects"))) @@ -3689,7 +3720,8 @@ public void discoverSearchObjectsWithQueryOperatorNotAuthority() throws Exceptio FacetEntryMatcher.clarinLicenseRightsFacet(false), FacetEntryMatcher.clarinItemsLanguageFacet(false), FacetEntryMatcher.clarinItemsCommunityFacet(false), - FacetEntryMatcher.hasContentInOriginalBundleFacet(false) + FacetEntryMatcher.hasContentInOriginalBundleFacet(false), + FacetEntryMatcher.typeFacet(false) ))) //There always needs to be a self link available .andExpect(jsonPath("$._links.self.href", containsString("/api/discover/search/objects"))) @@ -4138,7 +4170,8 @@ public void discoverSearchObjectsWithInProgressSubmissionTest() throws Exception FacetEntryMatcher.clarinItemsLanguageFacet(false), FacetEntryMatcher.clarinItemsCommunityFacet(false), FacetEntryMatcher.hasContentInOriginalBundleFacet(false), - FacetEntryMatcher.entityTypeFacet(false) + FacetEntryMatcher.entityTypeFacet(false), + FacetEntryMatcher.typeFacet(false) ))) //There always needs to be a self link .andExpect(jsonPath("$._links.self.href", containsString("/api/discover/search/objects"))) diff --git a/dspace-server-webapp/src/test/java/org/dspace/app/rest/DiscoveryScopeBasedRestControllerIT.java b/dspace-server-webapp/src/test/java/org/dspace/app/rest/DiscoveryScopeBasedRestControllerIT.java index 67e514d5084d..4a83a0ea593b 100644 --- a/dspace-server-webapp/src/test/java/org/dspace/app/rest/DiscoveryScopeBasedRestControllerIT.java +++ b/dspace-server-webapp/src/test/java/org/dspace/app/rest/DiscoveryScopeBasedRestControllerIT.java @@ -509,7 +509,8 @@ public void ScopeBasedIndexingAndSearchTestParentCommunity2() throws Exception { FacetEntryMatcher.entityTypeFacet(false), FacetEntryMatcher.clarinLicenseRightsFacet(false), FacetEntryMatcher.clarinItemsCommunityFacet(false), - FacetEntryMatcher.clarinItemsLanguageFacet(false) + FacetEntryMatcher.clarinItemsLanguageFacet(false), + FacetEntryMatcher.typeFacet(false) )) ); } @@ -624,7 +625,8 @@ public void ScopeBasedIndexingAndSearchTestSubcommunity22() throws Exception { FacetEntryMatcher.entityTypeFacet(false), FacetEntryMatcher.clarinLicenseRightsFacet(false), FacetEntryMatcher.clarinItemsCommunityFacet(false), - FacetEntryMatcher.clarinItemsLanguageFacet(false) + FacetEntryMatcher.clarinItemsLanguageFacet(false), + FacetEntryMatcher.typeFacet(false) )) ); } @@ -677,7 +679,8 @@ public void ScopeBasedIndexingAndSearchTestCollection222() throws Exception { FacetEntryMatcher.entityTypeFacet(false), FacetEntryMatcher.clarinLicenseRightsFacet(false), FacetEntryMatcher.clarinItemsCommunityFacet(false), - FacetEntryMatcher.clarinItemsLanguageFacet(false) + FacetEntryMatcher.clarinItemsLanguageFacet(false), + FacetEntryMatcher.typeFacet(false) )) ); } diff --git a/dspace-server-webapp/src/test/java/org/dspace/app/rest/MetadataBitstreamRestRepositoryIT.java b/dspace-server-webapp/src/test/java/org/dspace/app/rest/MetadataBitstreamRestRepositoryIT.java index 59e39285bea1..c966ecfa58e3 100644 --- a/dspace-server-webapp/src/test/java/org/dspace/app/rest/MetadataBitstreamRestRepositoryIT.java +++ b/dspace-server-webapp/src/test/java/org/dspace/app/rest/MetadataBitstreamRestRepositoryIT.java @@ -7,6 +7,7 @@ */ package org.dspace.app.rest; +import static org.hamcrest.Matchers.hasItem; import static org.hamcrest.Matchers.is; import static org.hamcrest.Matchers.notNullValue; import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.get; @@ -34,7 +35,6 @@ import org.dspace.content.service.clarin.ClarinLicenseResourceMappingService; import org.dspace.core.Constants; import org.dspace.services.ConfigurationService; -import org.dspace.util.FileTreeViewGenerator; import org.hamcrest.Matchers; import org.junit.Before; import org.junit.Test; @@ -115,9 +115,9 @@ public void findByHandle() throws Exception { .andExpect(jsonPath("$._embedded.metadatabitstreams[*].format") .value(Matchers.containsInAnyOrder(Matchers.containsString( bts.getFormat(context).getMIMEType())))) + // Convert the long into int because Marchers has a problem to compare long format .andExpect(jsonPath("$._embedded.metadatabitstreams[*].fileSize") - .value(Matchers.containsInAnyOrder(Matchers.containsString( - FileTreeViewGenerator.humanReadableFileSize(bts.getSizeBytes()))))) + .value(hasItem(is((int) bts.getSizeBytes())))) .andExpect(jsonPath("$._embedded.metadatabitstreams[*].canPreview") .value(Matchers.containsInAnyOrder(Matchers.is(canPreview)))) .andExpect(jsonPath("$._embedded.metadatabitstreams[*].fileInfo").exists()) @@ -150,8 +150,7 @@ public void previewingIsDisabledByCfg() throws Exception { .value(Matchers.containsInAnyOrder(Matchers.containsString( bts.getFormat(context).getMIMEType())))) .andExpect(jsonPath("$._embedded.metadatabitstreams[*].fileSize") - .value(Matchers.containsInAnyOrder(Matchers.containsString( - FileTreeViewGenerator.humanReadableFileSize(bts.getSizeBytes()))))) + .value(hasItem(is((int) bts.getSizeBytes())))) .andExpect(jsonPath("$._embedded.metadatabitstreams[*].canPreview") .value(Matchers.containsInAnyOrder(Matchers.is(false)))) .andExpect(jsonPath("$._embedded.metadatabitstreams[*].fileInfo").exists()) diff --git a/dspace-server-webapp/src/test/java/org/dspace/app/rest/StatisticsRestRepositoryIT.java b/dspace-server-webapp/src/test/java/org/dspace/app/rest/StatisticsRestRepositoryIT.java index 5544ecdb032b..2a07dc479807 100644 --- a/dspace-server-webapp/src/test/java/org/dspace/app/rest/StatisticsRestRepositoryIT.java +++ b/dspace-server-webapp/src/test/java/org/dspace/app/rest/StatisticsRestRepositoryIT.java @@ -53,6 +53,7 @@ import org.dspace.content.DSpaceObject; import org.dspace.content.Item; import org.dspace.content.Site; +import org.dspace.content.service.SiteService; import org.dspace.core.Constants; import org.dspace.eperson.EPerson; import org.dspace.services.ConfigurationService; @@ -76,6 +77,8 @@ public class StatisticsRestRepositoryIT extends AbstractControllerIntegrationTes ConfigurationService configurationService; @Autowired protected AuthorizeService authorizeService; + @Autowired + protected SiteService siteService; private Community communityNotVisited; private Community communityVisited; @@ -1532,6 +1535,22 @@ public void usageReportsSearch_Bitstream_Visited() throws Exception { ))); } + // Show usage reports for the Anonymous user - it could be configured by cfg property + // `site.usage-reports.enable.auth.anonymous` + @Test + public void usageReportsSearch_Site_For_Anonymous() throws Exception { + // This property is set to `true` before each test + configurationService.setProperty("usage-statistics.authorization.admin.usage", false); + + // Get the site object UUID + Site site = siteService.findSite(context); + // Allow accessing Site usage reports for anonymous + getClient() + .perform(get("/api/statistics/usagereports/search/object?uri=http://localhost:8080/server/api/core" + + "/sites/" + site.getID())) + .andExpect(status().isOk()); + } + // Create expected points from -6 months to now, with given number of views in current month private List getListOfVisitsPerMonthsPoints(int viewsLastMonth) { List expectedPoints = new ArrayList<>(); diff --git a/dspace-server-webapp/src/test/java/org/dspace/app/rest/matcher/FacetEntryMatcher.java b/dspace-server-webapp/src/test/java/org/dspace/app/rest/matcher/FacetEntryMatcher.java index 9176a42616f9..23d66754dcc6 100644 --- a/dspace-server-webapp/src/test/java/org/dspace/app/rest/matcher/FacetEntryMatcher.java +++ b/dspace-server-webapp/src/test/java/org/dspace/app/rest/matcher/FacetEntryMatcher.java @@ -124,7 +124,7 @@ public static Matcher matchFacet(boolean hasNext, String name, S public static Matcher clarinLicenseRightsFacet(boolean hasNext) { return allOf( hasJsonPath("$.name", is("rights")), - hasJsonPath("$.facetType", is("standard")), + hasJsonPath("$.facetType", is("text")), hasJsonPath("$.facetLimit", any(Integer.class)), hasJsonPath("$._links.self.href", containsString("api/discover/facets/rights")), hasJsonPath("$._links", matchNextLink(hasNext, "api/discover/facets/rights")) @@ -144,7 +144,7 @@ public static Matcher clarinItemsLanguageFacet(boolean hasNext) public static Matcher clarinItemsCommunityFacet(boolean hasNext) { return allOf( hasJsonPath("$.name", is("items_owning_community")), - hasJsonPath("$.facetType", is("standard")), + hasJsonPath("$.facetType", is("text")), hasJsonPath("$.facetLimit", any(Integer.class)), hasJsonPath("$._links.self.href", containsString("api/discover/facets/items_owning_community")), diff --git a/dspace-server-webapp/src/test/java/org/dspace/app/rest/matcher/SearchFilterMatcher.java b/dspace-server-webapp/src/test/java/org/dspace/app/rest/matcher/SearchFilterMatcher.java index 7700d5291249..b60f26940ff8 100644 --- a/dspace-server-webapp/src/test/java/org/dspace/app/rest/matcher/SearchFilterMatcher.java +++ b/dspace-server-webapp/src/test/java/org/dspace/app/rest/matcher/SearchFilterMatcher.java @@ -160,7 +160,7 @@ public static Matcher clarinLicenseRightsFilter() { return allOf( hasJsonPath("$.filter", is("rights")), hasJsonPath("$.hasFacets", is(true)), - hasJsonPath("$.type", is("standard")), + hasJsonPath("$.type", is("text")), hasJsonPath("$.openByDefault", is(false)), checkOperators() ); @@ -180,7 +180,7 @@ public static Matcher clarinItemsCommunityFilter() { return allOf( hasJsonPath("$.filter", is("items_owning_community")), hasJsonPath("$.hasFacets", is(true)), - hasJsonPath("$.type", is("standard")), + hasJsonPath("$.type", is("text")), hasJsonPath("$.openByDefault", is(false)), checkOperators() ); @@ -189,7 +189,7 @@ public static Matcher clarinItemsCommunityFilter() { public static Matcher clarinItemsTypeFilter() { return allOf( hasJsonPath("$.filter", is("itemtype")), - hasJsonPath("$.hasFacets", is(false)), + hasJsonPath("$.hasFacets", is(true)), hasJsonPath("$.type", is("text")), hasJsonPath("$.openByDefault", is(false)), checkOperators() diff --git a/dspace/config/clarin-dspace.cfg b/dspace/config/clarin-dspace.cfg index be485ad54184..8d8ae7898433 100644 --- a/dspace/config/clarin-dspace.cfg +++ b/dspace/config/clarin-dspace.cfg @@ -156,6 +156,9 @@ matomo.tracker.bitstream.site_id = 1 matomo.tracker.oai.site_id = 1 statistics.cache-server.uri = http://cache-server.none +#### Statistic usage reports #### +# site.usage-reports.enable.auth.anonymous = true + ##### Citacepro config ##### # citace.pro.url = https://www.citacepro.com/api/dspace/citace/oai @@ -252,3 +255,8 @@ s3.upload.by.parts.enabled = true ### The build version is stored in the specific file ### build.version.file.path = ${dspace.dir}/config/VERSION_D.txt + + +#### Item View #### +# Show handle and doi as identifiers - show only DOI if it exists instead of handle by default +item-page.show-handle-and-doi = false \ No newline at end of file diff --git a/dspace/config/dspace.cfg b/dspace/config/dspace.cfg index 60e46607c21f..4cf68c17f82f 100644 --- a/dspace/config/dspace.cfg +++ b/dspace/config/dspace.cfg @@ -931,6 +931,7 @@ registry.metadata.load = iiif-types.xml ### CLARIN ### # This property cannot be added in the `clarin-dspace.cfg` because then some Unit tests are failing.. registry.metadata.load = metashare-schema.xml +registry.metadata.load = edm.xml #---------------------------------------------------------------# #-----------------UI-Related CONFIGURATIONS---------------------# diff --git a/dspace/config/modules/authentication-shibboleth.cfg b/dspace/config/modules/authentication-shibboleth.cfg index 4034d7e2c326..7e176f71b718 100644 --- a/dspace/config/modules/authentication-shibboleth.cfg +++ b/dspace/config/modules/authentication-shibboleth.cfg @@ -90,8 +90,8 @@ authentication-shibboleth.lazysession.secure = true # Authentication headers for Mail, NetID, and Tomcat's Remote User. # Supply all parameters possible. -authentication-shibboleth.netid-header = SHIB-NETID -authentication-shibboleth.email-header = SHIB-MAIL +authentication-shibboleth.netid-header = eppn +authentication-shibboleth.email-header = mail authentication-shibboleth.email-use-tomcat-remote-user = false # Should we allow new users to be registered automatically? @@ -127,8 +127,8 @@ authentication-shibboleth.sword.compatibility = false # Metadata Headers # Shibboleth-based headers for the first and last name attirbutes -authentication-shibboleth.firstname-header = SHIB-GIVENNAME -authentication-shibboleth.lastname-header = SHIB-SURNAME +authentication-shibboleth.firstname-header = givenName +authentication-shibboleth.lastname-header = sn # Additional user attributes mapping, multiple attributes may be stored # for each user. The left side is the Shibboleth-based metadata Header diff --git a/dspace/config/modules/rest.cfg b/dspace/config/modules/rest.cfg index d17c6dc6bf2c..32dcc0fb1e67 100644 --- a/dspace/config/modules/rest.cfg +++ b/dspace/config/modules/rest.cfg @@ -70,6 +70,7 @@ rest.properties.exposed = themed.by.company.name rest.properties.exposed = identifier.doi.resolver rest.properties.exposed = spring.servlet.multipart.max-file-size rest.properties.exposed = authentication-shibboleth.show.idp-attributes +rest.properties.exposed = item-page.show-handle-and-doi # TUL rest.properties.exposed = dspace.ui.url diff --git a/dspace/config/registries/edm.xml b/dspace/config/registries/edm.xml new file mode 100644 index 000000000000..1c49505e9c20 --- /dev/null +++ b/dspace/config/registries/edm.xml @@ -0,0 +1,17 @@ + + + + edm + http://www.europeana.eu/schemas/edm/ + + + edm + type + + The value must be one of the types accepted by Europeana as it will support portal functionality: TEXT, VIDEO, SOUND, IMAGE, 3D. + There's a huge overlap with values we allow in metashare's mediaType; eventually this might get merged. + + + + + diff --git a/dspace/config/spring/api/discovery.xml b/dspace/config/spring/api/discovery.xml index c373051b848a..138c771dedb4 100644 --- a/dspace/config/spring/api/discovery.xml +++ b/dspace/config/spring/api/discovery.xml @@ -64,6 +64,7 @@ + @@ -171,6 +172,7 @@ + @@ -207,8 +209,168 @@ - - + + + + + + + + + + + + + (search.resourcetype:Item AND latestVersion:true) OR search.resourcetype:Collection OR search.resourcetype:Community + -withdrawn:true AND -discoverable:false + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + dc.title + dc.contributor.author + dc.creator + dc.contributor.other + dc.subject + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + @@ -303,6 +465,7 @@ dc.title dc.contributor.author dc.creator + dc.contributor.other dc.subject @@ -355,8 +518,10 @@ - - + + + + @@ -440,6 +605,7 @@ dc.title dc.contributor.author dc.creator + dc.contributor.other dc.subject @@ -494,8 +660,10 @@ - - + + + + @@ -579,6 +747,7 @@ dc.title dc.contributor.author dc.creator + dc.contributor.other dc.subject @@ -636,8 +805,10 @@ - - + + + + @@ -719,6 +890,7 @@ dc.title dc.contributor.author dc.creator + dc.contributor.other dc.subject @@ -765,8 +937,10 @@ - - + + + + @@ -916,8 +1090,10 @@ - - + + + + @@ -991,8 +1167,10 @@ - - + + + + @@ -1066,8 +1244,10 @@ - - + + + + @@ -1150,8 +1330,10 @@ - - + + + + @@ -1220,8 +1402,10 @@ - - + + + + @@ -2308,6 +2492,7 @@ dc.contributor.author dc.creator + dc.contributor.other @@ -2325,7 +2510,7 @@ dc.rights.label - + @@ -2350,7 +2535,7 @@ - + @@ -2360,6 +2545,8 @@ dc.type + edm.type + metashare.ResourceInfo#ContentInfo.mediaType @@ -2995,6 +3182,17 @@ + + + + + + + + + + +