Customer/lindat #833
Open
Customer/lindat #833
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…larin-dspace.cfg: (#80) * Separated config files, reverted dspace.cfg to original: * Must be called dspace.name = DSpace at My University or tests will fail * cleared up clarin-dspace.cfg * added link to future changes from the past clarin config
Problem description For some items the date of creation is unknown or in date range. This information is stored in the approximateDate metadata. If approximateDate metadata is not null -> show value from this metadata instead of date issued.
…ts. Suggestions are showed in the complex input field. ## Problem description Show funding suggestions for EU projects - openAIRE and non-EU projects. Suggestions are showed in the complex input field. - [x] If should return suggestions for the non eu sponsors do not return eu suggestions - [x] distinct suggestions - if the sponsor exist in more items the API return the same suggestion several times - [x] openAIRE oai-pmh return only the items with eu-identifier **!!!** This pull request contains pull request for _complex-input-types_ and _autocomplete_ **!!!** **complex-input-types:** #44 **autocomplete:** #50 **Commits:** * Added configuration for the complex input type, complex input type is NOT send to the request. * Fixed checkstyle issues * Return empty complexDefinition in the response * complex definition is processed * COMPLEX_DEFINITION_REF added to the formDefns * Return complexDefinition as JSON string * Created unit tests for the new local metadata types * Created unit tests for the DCInput.java * Fixed checkstyle issues * Moved a new files to the dspace packages * Added docs * Added docs * Edited docs * removed empty row * Commented complex input type using in the submission-forms.xml * Removed empty row * Added metadata validation for the complex input definition and transformed complex definition into json is ordered * Fixed checkstyle issues * set series to not required * fixed issue after resolved conflicts * fixed checkstyle issue * Defined autocomplete tag * Autocomplete definition is exposed in the form definition response * Created example new REST API * Endpoint is returning searched metadata values * created endpoint /metadatavalues but it throws strange error * Created wrapper for MetadataValue with converter, resource, repository * Created functionality to return all metadataValues * Added some docs * Added some docs and removed autocomplete definition from the submission-forms.dtd * Removed autocomplete tag * added empty row * Refactoring * Created IT for MetadataValueRestRepository * Removed unused code * Changed docs * Removed autocomplete comment * changed method which create json from complex definition. * refactoring * fixed checkstyle violations * changed expected json in the test * added funding complex type * Fixed empty N/A stored value * EU sponsor is added to the dc.relation * Added doc * fixed difference error between add and replace operation * fixed eu prefix null error * Fixed error: cannot call replace method on empty metadata * Created some tests for OpenAIRE * Created some tests for OpenAIRE and fixed some errors * Finish open-aire complex definition tests for storing information in more metadata fields * Removed duplicities in suggestions and do not return eu sponsors as suggestions. * Fixed local.bistream.redirectsToURL test error - schema tag was removed in the conflict * Fixed checkstyle violations * One test should throw exception but didn't - fixed * Fixed change requests * Refactoring Co-authored-by: MilanMajchrák <[email protected]>
* Added generating test scripts * fixed misleading comment * added maven source to our readme * fixed minor incoviniences while generating tests * added pause to scripts * Added newline to end of two scripts * improved description, changed generated filenames
* Loaded code from the upstream * Loaded code from the upstream to fix errors * Loaded another code from the upstream to fix errors * Revert changes * Copied type-bind code from the upstream main * Fixed wrong formatting * Copied test correctly * Fixed test errors * Loaded submission forms from main * Some fixes * Fixed missing ACL * Created tests to check if is not converted admin input field for the user. Co-authored-by: MilanMajchrák <[email protected]>
…property, hints * Uploaded big file hint and label * The uploaded big file is deleted after upload and added property to decide if should be deleted uploaded file * Update dspace.cfg * Changed property name * property added to the clarin-dspace.cfg Co-authored-by: MilanMajchrák <[email protected]>
* Added a new local metadata type - hasCMDI * CMDI file is uploaded to the right bundle * From method o maintain cmdi file was created Util class * Changed values of input field for local metadata hasCMDI * Fixed checkstyle issues * Created tests for the CMDIFileBundleMaintainer * Fixed checkstyle issues and removed unnecessary code. * Added documentation and removed some unnecessary code * Fixed checkstyle issues * Updated cmdi file maintaining method because it failed when the hasCMDI metadata was null * fixed error of getting item from the empty array * Removed empty row * Changed hasUploadFiles method and upload validation * Light code refactoring * Fixed error which was created by resolving conflicts * Fixed error which was created by resolving conflicts * Done review changes * changed sign * Refactor if statement Co-authored-by: MilanMajchrák <[email protected]>
Co-authored-by: MilanMajchrák <[email protected]>
…rch expression consist of nums and chars * Initial commit * Separating characters and numbers from the searchValue. * Fixed wrong autocomplete suggestions Co-authored-by: MilanMajchrák <[email protected]>
…earched only in Items not WorkspaceItems Co-authored-by: MilanMajchrák <[email protected]>
* script for deleting test environment * implementation of comments after review * edited example file
* Initial commit * Created helpDesk Controller and added help-desk mail to the configuration * Updated ItemConverter to return necessary metadata for the tombstone. * Removed HelpDesk Controller the mail is exposed by the ConfigurationController * Uncomment complex input fields Co-authored-by: MilanMajchrák <[email protected]>
* Added openaire context and data format from clarin-dspace * added distinct * added test support for OAI-PMH (MockSolrOAICore) * added OAIMPH test * attempted transforming XOAI.java into bean * modify IT for Openaire OAI * added files for cmdi format * adding cmdi format - rewriting java functions * fixed functions for clarin cmdi format * new url, correct document for cmdi * sacrifice to the checkstyle god * added /cmdi/oai-metadata endpoint * sacrifice to the checkstyle god * replaced distinct with custom function * build speed-up maven build speed Co-authored-by: kshepherd <[email protected]> * Initial commit * added olac format * added type and test * fixed generating proper unit tests * local.hidden support * changed local.hidden to local.hiddenButHarvestable; adjusted local.hasMetadata to local.hasCMDI, set deleted to false in xoai on localButHidde * modified hidden input box * changes in oai_dc.xsl * edits after tests * safety check, changes after tests * replaced distinct function with working version * replace fn:distinct with distinct-values * fixed tests * cleaning useless files * removing comments and useless classes * PR clean-up 4 * fixes to metasharev2 OAI metadataFormat * removed redundant spaces, added newlines at ends of files * fixes, PR * PR fixes 2 * added last missing source * fixed error - whitespace at end of line * skip overwrite prompt in our script * used wrong method Co-authored-by: kshepherd <[email protected]> Co-authored-by: MilanMajchrák <[email protected]>
* User community or collection handle `com_/col_` instead of v5 prefix `hdl_` because it cannot match handle of the community which should not be exposed. * Throw error when the dso object is not Community or Collection in the ColComFilter.java.
* separate emails and send * formatting * multiple admins * remove redundant and reformat * fixes from PR * comment on old property * separated function * separate function to add arguments to admin email * clarification of using 'anonymous user' twice
…etRestControllerIT (#700) Co-authored-by: Tim Donohue <[email protected]>
* Added a new tag into submission-forms.xml called `default-value`. This default value is assigned into Item during sending a sectionData to the FE. * Cherry picked fixed randomly failing IT: ManageGroupsFeatureIT, LinksetRestControllerIT (#701) Co-authored-by: Tim Donohue <[email protected]> --------- Co-authored-by: Tim Donohue <[email protected]>
* Added a new Step for the Note with czech translation * Updated script for copying config to not copy solr cores
The `{params[5]}` in the message body was not interpolated due to missing `$`
* Added a new Step for the Note with czech translation * Updated script for copying config to not copy solr cores * Defined new special fields and moved `note` into this step. * Added czech translation for the new submission inputs. * Cherry picked fixed randomly failing IT: ManageGroupsFeatureIT, LinksetRestControllerIT (#702) Co-authored-by: Tim Donohue <[email protected]> --------- Co-authored-by: Tim Donohue <[email protected]>
* work in progress * only use first available value from demo uri * reorder funding to be correct * WIP * remove forgotten script * restore old version * restore old version for real * restore old version for real fr * restore old version for real fr2 * comment of changed order * add issues and todo comments * use old and correct version * comment change * sacrifice to the checkstyle god * reformat TODOs
…roperty is corretly loaded. (#716) * The handle configuration from the `lr.pid.community.configurations` property is corretly loaded. * The new version of the Item does not have a version subprefix e.g., `2-<NEW_ID>` * Fixed a problem with integration tests - the pid configuration wasn't initialized after setting the property. * Fixed failing tests * Prettyfied the code
* updated comments * added new databased entity preview content * not works: map fileinfo to preview content * get data from database if it exists * create preview content from file info * Failing tests - temp commit * changed name varchar length * added _ to sql script names
* send email when SEND_TOKEN is required to EXTRA_EMAIL address * removed unwanted line
Update sourceversion.py
* copy functionality from UFAL repo * cleanup * comment from upstream * checkstyle
* control if owning collection is null
Co-authored-by: Tim Donohue <[email protected]>
* Do not commit a session during processing the previews. Catch the error when the ZIP is broken because it wont render files in the UI. * Fixed checkstyle issues
* Use $JAVA_OPTS_HANDLE to define handle server memory. * Do not add another slash in the handle URL (#737) * The scheduled tasks are available in the `/scheduledtasks` endpoint. (#741) * TUL/ The context user is set to current subscriber (#747) * The context user is set to current subscriber * Updated comment * Fixed checktyle issue * Cherry picked fix about integration tests (#751) Co-authored-by: Tim Donohue <[email protected]> --------- Co-authored-by: Tim Donohue <[email protected]>
* sort file_downloads.log by date * removed log which could generate noise
* UFAL/Removed duplicities of the bitstreams in the cmdi (#766) * Removed duplicities of the bitstreams in the cmdi. * Fixed checkstyle violation * used lindat code instead of vanilla. * Ufal/Preview issues (#764) * Ensure the content preview doesn't overload maximum length of the column in the database. And encode the input stream into UTF-8. * Do not store HTML content in the database because it could be longer than the limit of the database column. * UFAL/Encoded the UTF-8 characters from the redirect URL to UTF (#758) * Encoded the UTF-8 characters from the redirect URL to UTF * Moved ClarinUtils into Utils class * Added a new `dq` package into ComponentScan * Moved dq.Utils into DSpace utils.Utils because the components with the same name causes conflicts. * Removed *.dq component scan from the App * Merge pull request DSpace#9790 from DSpace/backport-9775-to-dspace-7_x (#769) [Port dspace-7_x] Make statistics autocommit much more frequently Co-authored-by: Tim Donohue <[email protected]> * test for bitstream with null value of sizebytes * Update README.md * UFAL/Shibboleth - load more net-id headers e.g. persistent-id (#772) * Load netid from more than one header. authentication-shibboleth.netid-header could be list, not only single value * Shibboleth login - sort the emails passed in the shibboleth email header and get the first one. * The user is redirected to the login page when it is trying to update eperson email which is already assigned to another eperson. * Sorting emails is moved into specific method and ShibbolethLoginFilter is updated following the ShibAuthentication changes * Fixed failing tests * The ClarinShibbolethLoginFilter and ClarinShibAuthentication has duplicate code, I put it into static method. * Propagate the verification token to the DB after the email is successfully sent. (#786) * UFAL/Enhanced type-bind feature (#762) * type bind is correctly rendered in the FE, but BE is still not working * Synchronized the `submission-forms_cs.xml` * Added doc into `submission-forms` about enhanced type-bind `field` * Updated `local.cfg` for tests - added type-bind property * Updated docs for the customized type-bind configuration property. * Updated MetadataValidation following the type-bind customization. * Added isAllowed function for multiple type-bind definitions * Added some docs for the new method * The values of the input wasn't loaded. * Allowed fields could be empty when they should have values. * Used isEmpty function and created constant for the `=>`. * create preview content for tar files (#759) * create preview content for tar files * Added right logs * devided extractFile funs into several separated smaller funs * added comment and removed empty line * added empty lines and removed unwanted comments * removed empty line * used consts * try incorrect identification level * log errors and removed unneeded consts --------- Co-authored-by: milanmajchrak <[email protected]> * Internal/fix failing Clarin integration test (#796) * Initial commit * Ignore the test class from where the tests has started failing. * Ignored half of tests in the ClarinShibbolethLoginFilterIT file * Ignored all tests * unignore some tests * 3 tests ignored 3 allowed * Maybe the problematic test is hidden between 3 unignored tests * two candidates * The last candidate * Ignore just wrong test and all test should pass * Clean up object created in the test. * Removed unused import. * Check the user which is going to be deleted is not null. * Rest api for handle resolution with metadata * decoded rawvalues and response json modification * used static extractMetadata funct in HandlePlugin * return dict: * removed property for test from local * Add default licenses - from ZCU update (#801) * Added flyway file to insert default licenses with license labels and mappings * Added required header * UFAL/share submission by email (#780) * Updated table workspace with share token, created endpoint to generate share token and it is sent via email * Added method to get workspaceitem via share token. * Added an endpoint for changing the submission's owner. * Added license headers * Added test for fetching item with share Token * Added tests to check the owner is changed * Added better explanation why the BE must return Page object in the search endpoint * Validate the user in the SubmissionController, it cannot be null * Updated email - some values are fetched directly from the configuration property * Updated preAuthorization method to ADD instead of WRITE (write is used for controlling authorization for modifying the Item) and updated shareURL * Authorize the submitter which is trying to take sharing item via shareToken. * Update integration test following the feature update * Import default licenses only if the license tables are empty. (#808) * Oai elg crosswalk (#798) * problem with language code * fix amount and sizeUnitOther rest-tests errors * added language coding * new language and funding project mishmash array position * added isoCodes * removed handle from item submission (#812) * UFAL/Shibboleth - netid-header should use getArrayProperty everywhere (#807) * Fetch netid as array from the cfg. Now netid as array is used everywhere. Added integration test to ask for an email when the user send only persistent-id in the shib header. * Fixed checkstyle issue * The user is not signed in without using link with the verification token from the email/ (#809) * UFAL/Shibboleth - show error in the UI when shibboleth authentication is failed (#810) * The user is not signed in without using link with the verification token from the email/ * Send a redirect to UI with specific parameter that the Shibboleth authorization wasn't successful * UFAL/Autocomplete enhancement (#768) * Added solr index `handle_title_ac` and `_comp` for the Item * Added support for searching results from specific solr indexes. 1. Updated submission-forms autocomplete definition to specify a specific index. 2. Updated configuration is provided via REST API. 3. Create a new `/suggestions` endpoint for searching values from custom solr index - it returns VocabularyEntry page. * Supported searching Item byHandle when passed a handle as parameter without handle canonical prefix. * Added autocompleteCustom `solr-subject_ac` and `handle_title_ac`. * Added autocompleteCustom `solr-publisher_ac`. * Added cfg property to define a separator from the solr value to get only display value. * Added autocompleteCustom `solr-dataProvider_ac` * Refactored code and created integration test for the SuggestionRestController * Updated suggestion integration tests because it has had a conflict with another IT * Added doc and changed `autocomplete.custom.format` to `autocomplete.custom.separator` for proper naming. * Added support for loading suggestions from the json file - need to refactor. * Refactored and added docs. * Created tests and fixed failing ones due to updated solr definition * Synchronized submission-forms_cs.xml with the original-english one * Added docs about magic constants * Added doc why the handle is updated to canonical form in the searchbyHandle endpoint * Allow searching only within the solr indexes or JSON files permitted by the configuration. * Removed normalization of handle prefix because there could be more prefixes. Expect only handle value. * Fixed if condition and config property default value. * Fixed integration tests. Allowed autocomplete custom was missing in the test cfg. * The suggestion endpoint is allowed only for authorized users * Refactored method for the normalizing the query for the discoverQuery and added unit tests. * UFAL/Changed position of rows in submission-forms.xml following v5 (#802) * Changed position of rows in submission-forms.xml following v5 * Fixed the alignment of some text * Removed rows which are not in the v5 * Removed license selector from the `teaching` and `clariah-data` collections * Updated input differences in the submission-forms.xml following the v5. --------- Co-authored-by: Juraj Roka <[email protected]> Co-authored-by: milanmajchrak <[email protected]> * Show db connection statistics in the log file or the `dbstatistics` endpoint (#815) * Show db statistics in the log file or the `dbstatistics` endpoint * Finding out why github checks are failed - undo hibernate.cfg * Disabled automatic logging * Use scheduled CRON job instead of PostConstruct * hibernate generating property true --------- Co-authored-by: Paurikova2 <[email protected]> * Translation of submission-forms to _cs (#816) * Translation of submission-forms to _cs * Translated bitstream metadata and complex input fields * Translated the rest of submission-froms_cs.xml * Fixed regex... it must contain regex value, not the message. --------- Co-authored-by: Juraj Roka <[email protected]> Co-authored-by: milanmajchrak <[email protected]> * Updated cfg to pretify the author suggestions (#819) * crosswalk-embargo (#821) * added fn for embargo * using of res policy end_date and added comments * fix string format problem with %s * integration tests are falling down * checkstyle violations * removed findHandle duplicity * added deleted line * checkstyle violations * For now the complex input field is without autocomplete for the size and contact person (#823) * Send the custom type bind `field` to the FE configuration (#822) * fix date converting to string (#825) * fix date converting to string * made const from format * checkstyle * cherry-pick clarin v7 into dtq dev (#820) * cherry-picked DataCite related changes from customer/uk * Add a script that adds a file from url to an item intended for large file workflows * Add ways to influence the bitstream name * add more options to specify an item * Expose resourceId (DSpace#1134) A BE part of DSpace#1127 - this exposes the resourceId so it can be used in the handle mgmt table * fixes ufal#1135 - findEpersonByNetId should stop searching when it finds an eperson - moved the `return eperson` inside the for cycle (after eperson non null check). - removed the eperson param (both callers were passing in `null`) * Test release without db logs (#827) * UFAL/Matomo statistics with dimension (#813) * Updated the version of matomo dependency and tried to change request from Custom Variables to Dimension * Added a custom dimension with item's handle URL * Send custom dimension also in oai tracker * Use only IPv4 address, the Matomo tracker has a problem with IPv6 * Do not change custom dimension when the Item is null * First custom dimension should have ID '1'. * Use a valid URL for Matomo tracker in the IT * Configure handle custom dimension ID in the clarin-dspace.cfg * Refactored ipv4 method to be more readable - return null --------- Co-authored-by: Juraj Roka <[email protected]> Co-authored-by: milanmajchrak <[email protected]> Co-authored-by: milanmajchrak <[email protected]> * The `dspace.name.short` is not working properly in the email, use `dspace.shortname` instead (#828) --------- Co-authored-by: Tim Donohue <[email protected]> Co-authored-by: Paurikova2 <[email protected]> Co-authored-by: Jozef Misutka <[email protected]> Co-authored-by: jurinecko <[email protected]> Co-authored-by: Juraj Roka <[email protected]> Co-authored-by: Paurikova2 <[email protected]> Co-authored-by: Ondřej Košarko <[email protected]>
* commented license import script * deleted script for default licenses insertion
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem description
We