harvest_responsible_organizations setting in harvester configuration crashes harvester if set as boolean value

[sjbruce]

The harvester configuration is a JSON object, however, if harvest_responsible_organizations is set as a boolean value like the rest of the boolean options in the configuration the ckan_fetch_harvester container will crash because it is expecting a string value.

For consistency this should be updated to expect a boolean and either test for type or have a try/except block to catch the error and respond accordingly.

This snippet is from handle_groups() function in plugin.py, the line in question is 125

        harvest_responsible_organizations = (source_config.get('harvest_responsible_organizations') or toolkit.config.get('ckan.harvest_responsible_organizations') or 'true').lower()
        if harvest_responsible_organizations == "true":
            # Handle org mapping using metadata cited-responsible-party
            log.info(':::::::::::::-Handle Groups-::::::::::::: %r ', cats)
        else:
            log.debug(':::::::::::::-Skipping Handle Groups-::::::::::::: %r ', cats)
            return validated_groups

[fostermh]

note that it can also be set in the ckan.ini / .env file so any solution should work for both the config file and the harvester config. Matthew Foster Hakai Institute, Calvert Island, BC, Canada / www.hakai.org Tula Foundation, Quadra Island, BC, Canada / www.tula.org

…

On Fri, Jul 19, 2024 at 8:11 AM Scott Bruce ***@***.***> wrote: The harvester configuration is a JSON object, however, if harvest_responsible_organizations is set as a boolean value like the rest of the boolean options in the configuration the ckan_fetch_harvester container will crash because it is expecting a string value. For consistency this should be updated to expect a boolean and either test for type or have a try/except block to catch the error and respond accordingly. This snippet is from handle_groups() function in plugin.py, the line in question is 125 harvest_responsible_organizations = (source_config.get('harvest_responsible_organizations') or toolkit.config.get('ckan.harvest_responsible_organizations') or 'true').lower() if harvest_responsible_organizations == "true": # Handle org mapping using metadata cited-responsible-party log.info(':::::::::::::-Handle Groups-::::::::::::: %r ', cats) else: log.debug(':::::::::::::-Skipping Handle Groups-::::::::::::: %r ', cats) return validated_groups — Reply to this email directly, view it on GitHub <#18>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABQDJYHW7C3B7VDMJABLN2TZNEUDBAVCNFSM6AAAAABLEYTLL6VHI2DSMVQWIX3LMV43ASLTON2WKOZSGQYTSMJXHAYTANI> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[sjbruce]

Right. Am I correct that in assuming that the settings from .env and ckan.ini are basically funneled into the toolkit.config object? Does it behave like JSON as well or is it a different beast?

-- Scott From: Matthew Foster ***@***.***> Sent: Friday, July 19, 2024 4:40 PM To: cioos-siooc/ckanext-cioos_harvest ***@***.***> Cc: Scott Bruce ***@***.***>; Author ***@***.***> Subject: Re: [cioos-siooc/ckanext-cioos_harvest] harvest_responsible_organizations setting in harvester configuration crashes harvester if set as boolean value (Issue #18) note that it can also be set in the ckan.ini / .env file so any solution should work for both the config file and the harvester config. Matthew Foster Hakai Institute, Calvert Island, BC, Canada / www.hakai.org<http://www.hakai.org> Tula Foundation, Quadra Island, BC, Canada / www.tula.org<http://www.tula.org>

On Fri, Jul 19, 2024 at 8:11 AM Scott Bruce ***@***.***<mailto:***@***.***>> wrote: The harvester configuration is a JSON object, however, if harvest_responsible_organizations is set as a boolean value like the rest of the boolean options in the configuration the ckan_fetch_harvester container will crash because it is expecting a string value. For consistency this should be updated to expect a boolean and either test for type or have a try/except block to catch the error and respond accordingly. This snippet is from handle_groups() function in plugin.py, the line in question is 125 harvest_responsible_organizations = (source_config.get('harvest_responsible_organizations') or toolkit.config.get('ckan.harvest_responsible_organizations') or 'true').lower() if harvest_responsible_organizations == "true": # Handle org mapping using metadata cited-responsible-party log.info(':::::::::::::-Handle Groups-::::::::::::: %r ', cats) else: log.debug(':::::::::::::-Skipping Handle Groups-::::::::::::: %r ', cats) return validated_groups — Reply to this email directly, view it on GitHub <#18>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABQDJYHW7C3B7VDMJABLN2TZNEUDBAVCNFSM6AAAAABLEYTLL6VHI2DSMVQWIX3LMV43ASLTON2WKOZSGQYTSMJXHAYTANI> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***<mailto:***@***.***>>

— Reply to this email directly, view it on GitHub<#18 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AMGVME237CEUHTZFVHFE7Z3ZNFP67AVCNFSM6AAAAABLEYTLL6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMZZHE2TINBYHE>. You are receiving this because you authored the thread.Message ID: ***@***.******@***.***>> External Sender: Use caution when clicking links, opening attachments, or responding. This email is governed by the Terms and Conditions found in our Disclaimer<http://www.mi.mun.ca/ict/disclaimer>.

[fostermh]

they are funnelled into the config object yes. they are treated as strings. For example, a value of "true" in the config is the string "true" not a boolean value.