diff --git a/plugins/classical_extras/Readme.md b/plugins/classical_extras/Readme.md index 79871d76..e860c3ce 100644 --- a/plugins/classical_extras/Readme.md +++ b/plugins/classical_extras/Readme.md @@ -1,7 +1,5 @@ # General Information -This is version 0.8.8 of "classical_extras". It has only been tested with FLAC and mp3 files. It does work with m4a files, but Picard does not write all m4a tags (see further notes for iTunes users at the end of the "works and parts tab" section). -It populates hidden variables in Picard with information from the MusicBrainz database about the recording, artists and work(s), and of any containing works, passing up through mutiple work-part levels until the top is reached. -The "Options" page (Options->Options->Plugins->Classical Extras) allows the user to determine how these hidden variables are written to file tags, as well as a variety of other options. +This is version 0.8.9 of "classical_extras". It has only been tested with FLAC and mp3 files. It does work with m4a files, but Picard does not write all m4a tags (see further notes for iTunes users at the end of the "works and parts tab" section). It populates hidden variables in Picard with information from the MusicBrainz database about the recording, artists and work(s), and of any containing works, passing up through mutiple work-part levels until the top is reached. The "Options" page (Options->Options->Plugins->Classical Extras) allows the user to determine how these hidden variables are written to file tags, as well as a variety of other options. This plugin is particularly designed to assist with tagging of classical music so that player or library manager software which can display multiple work levels and different artist types, can have access to these details. It has two main components "Extra Artists" and "Work Parts" which can be used independently or together. "Work Parts" will take at least as many seconds to process as there are works to look up (owing to MB throttling) so users who only want the extra artist information and not the work details may turn it off (e.g. perhaps for 'popular' music). @@ -11,6 +9,8 @@ Tags are output depending on the choices specified by the user in the Options Pa If the Options Page does not provide sufficient flexibility, users familiar with scripting can write Tagger Scripts to access the hidden variables directly. ## Updates +Version 0.8.9: Provide option (in advanced tab) to disable Classical Extras processing if no file is present; this enables (for example) single discs from box sets to be loaded without incurring the additional processing overhead for all the other discs. The settings of the main Picard options "translate_artist_names" and "standardize_artists" is now saved along with the Classical Extras options so that they can be used to over-ride the displayed options. This is because they interact with the Classical Extras options in certain cases. Also:- graceful recovery from authentication failure; improved UI - more scalable; minor bug fixes. + Version 0.8.8: Fixes to allow for (1) disabling of 'use_cache' across releases which may share works and (2) works which might appear in their own right and also have arrangements. Also, re-set certain important options to defaults on starting Picard, namely: 'use_cache' set to True, 'log_debug', 'log_info' and 'options_overwrite' set to False; the user will need to deliberately re-set these on starting Picard if required - this is to prevent inadvertently leaving these flags in an abnormal state. Version 0.8.7: Revised treatment of "conditional" tag mapping. Previously, if multiple sources were specified for a tag mapping and the "conditional" flag set, only the first non-empty source was used. Now all sources will be mapped to a tag if it was empty before executing the current tag mapping line. This is considered to be more intuitive and leads to less complex mapping lines. However, it some cases it may be necessary to split a line from a previous version if the previous behaviour was specifically desired. Improved algorithms for extending metadata with title info. Bug fixes. @@ -308,6 +308,8 @@ If it is important that only whole words are to be matched, be sure to include a The tag contents are in dict format. These tags can then be used to over-ride the displayed options subsequently (see below). + N.B. The "Tag name for artist/misc. options" also saves the Picard options for 'translate_artist_names' and 'standardize_artists' as these interact with the Classical Extras options. + 5. "Over-ride plugin options displayed in UI with options from local file tags". If options have previously been saved (see above), selecting these will cause the saved options to be used in preference to the displayed options. The displayed options will not be affected and will be used if no saved options are present. The default is for no over-ride. ***Note that very occasionally (if the tag containing the options has been corrupted) use of this option may cause an error. In such a case you will need to deselect the "over-ride" option and set the required options manually; then save the resulting tags and the corrupted tag should be over-written*** The last checkbox, "Overwrite options in Options Pages", is for **VERY CAREFUL USE ONLY**. It will cause any options read from the saved tags (if the relevant box has been ticked) to over-write the options on the plugin Options Page UI. The intended use of this is if for some reason the user's preferred options have been erased/reverted to default - by using this option, the previously-used choices from a reliable filed album can be used to populate the Options Page. The box will automatically be unticked after loading/refreshing one album, and will always be turned off when starting Picard, to prevent inadvertant use. Far better is to make a **backup copy** of the picard.ini file. diff --git a/plugins/classical_extras/__init__.py b/plugins/classical_extras/__init__.py index ac99da2a..fbbaa534 100644 --- a/plugins/classical_extras/__init__.py +++ b/plugins/classical_extras/__init__.py @@ -17,7 +17,7 @@

See Readme file for full details. """ -PLUGIN_VERSION = '0.8.8' +PLUGIN_VERSION = '0.8.9' PLUGIN_API_VERSIONS = ["1.4.0", "1.4.2"] PLUGIN_LICENSE = "GPL-2.0" PLUGIN_LICENSE_URL = "https://www.gnu.org/licenses/gpl-2.0.html" @@ -108,16 +108,16 @@ def get_options(album, track): set_options[track] = option_settings( album.tagger.config.setting) # make a copy - # Also use some of the main Picard options - set_options[track]['translate_artist_names'] = config.setting['translate_artist_names'] - set_options[track]['standardize_artists'] = config.setting['standardize_artists'] + # As we use some of the main Picard options and may over-write them, save them here + # set_options[track]['translate_artist_names'] = config.setting['translate_artist_names'] + # set_options[track]['standardize_artists'] = config.setting['standardize_artists'] options = set_options[track] tm = track.metadata orig_metadata = None # Only look up files if needed file_options = {} - music_file = None + music_file_found = None found_file = False for music_file in album.tagger.files: new_metadata = album.tagger.files[music_file].metadata @@ -174,6 +174,7 @@ def get_options(album, track): ' options cannot be read. Using current settings') found_file = True + music_file_found = music_file break # we've found the file and don't want any more! else: if 'musicbrainz_trackid' not in new_metadata: @@ -210,7 +211,7 @@ def get_options(album, track): for opt in opt_dict: opt_value = opt_dict[opt] if section == 'artists': - addn = plugin_options('tag') + addn = plugin_options('tag') + plugin_options('picard') else: addn = [] for ea_opt in plugin_options(section) + addn: @@ -247,9 +248,9 @@ def get_options(album, track): else: options_dict = options tm['~ce_options'] = options_dict - tm['~ce_file'] = music_file + tm['~ce_file'] = music_file_found if options['log_info']: - log.info('Get_options is returning options shown below and file: %s', music_file) + log.info('Get_options is returning options shown below and file: %s', music_file_found) log.info(options_dict) @@ -718,6 +719,20 @@ def plugin_options(type): } ] + # Picard options which are also saved + picard_options = [ + {'option': 'standardize_artists', + 'name': 'standardize artists', + 'type': 'Boolean', + 'default': False + }, + {'option': 'translate_artist_names', + 'name': 'translate artist names', + 'type': 'Boolean', + 'default': True + }, + ] + # other options (not saved in file tags) other_options = [ {'option': 'use_cache', @@ -819,7 +834,12 @@ def plugin_options(type): }, {'option': 'ce_options_overwrite', 'type': 'Boolean', - 'default': False} + 'default': False + }, + {'option': 'ce_no_run', + 'type': 'Boolean', + 'default': False + } ] if type == 'artists': @@ -828,6 +848,8 @@ def plugin_options(type): return tag_options elif type == 'workparts': return workparts_options + elif type == 'picard': + return picard_options elif type == 'other': return other_options else: @@ -841,7 +863,7 @@ def option_settings(config_settings): """ options = {} for option in plugin_options('artists') + plugin_options('tag') \ - + plugin_options('workparts') + plugin_options('other'): + + plugin_options('workparts') + plugin_options('picard') + plugin_options('other'): options[option['option']] = copy.deepcopy( config_settings[option['option']]) return options @@ -1051,6 +1073,56 @@ def get_artists(log_options, relations, relation_type): log.info('sorted artists = %s', artists) return artists +def apply_artist_style(options, lang, a_list, name_style, name_tag, sort_tag, names_tag, names_sort_tag): + # Get artist and apply style + for acs in a_list: + for ncs in acs: + artistlist = parse_data( + options, ncs, [], 'artist', 'name', 'text') + sortlist = parse_data( + options, ncs, [], 'artist', 'sort_name', 'text') + names = {} + if lang: + names['alias'] = parse_data( + options, + ncs, + [], + 'artist', + 'alias_list', + 'alias', + 'attribs.locale:' + + lang, + 'attribs.primary:primary', + 'text') + else: + names['alias'] = [] + names['credit'] = parse_data( + options, ncs, [], 'name', 'text') + pairslist = zip(artistlist, sortlist) + names['sort'] = [ + translate_from_sortname( + *pair) for pair in pairslist] + for style in name_style: + if names[style]: + artistlist = names[style] + break + joinlist = parse_data( + options, ncs, [], 'attribs.joinphrase') + + if artistlist: + name_tag.append(artistlist[0]) + sort_tag.append(sortlist[0]) + names_tag.append(artistlist[0]) + names_sort_tag.append(sortlist[0]) + + if joinlist: + name_tag.append(joinlist[0]) + sort_tag.append(joinlist[0]) + + name_tag_str = ''.join(name_tag) + sort_tag_str = ''.join(sort_tag) + + return {'artists': names_tag, 'artists_sort': names_sort_tag, 'artist': name_tag_str, 'artistsort': sort_tag_str} def set_work_artists(self, album, track, writerList, tm, count): """ @@ -1228,7 +1300,6 @@ def set_work_artists(self, album, track, writerList, tm, count): # fix cyrillic names if not already fixed if options['cea_cyrillic']: if not only_roman_chars(name): - # if not only_roman_chars(tm[tag]): name = remove_middle(unsort(sort_name)) # Only remove middle name where the existing # performer is in non-latin script @@ -1402,25 +1473,11 @@ def remove_middle(performer): # Sorting etc. -# def sort_field(performer): -# """ -# To create a sort -# No longer used as all sort names are now sourced from XML lookup -# """ -# sorter = re.compile(r'(.*)\s(.*)$') -# match = sorter.search(performer) -# if match: -# return match.group(2) + ", " + match.group(1) -# else: -# return performer - - def unsort(performer): """ To take a sort field and recreate the name Only now used for last-ditch cyrillic translation - superseded by 'translate_from_sortname' """ - sorted_list = performer.split(', ') sorted_list.reverse() for i, item in enumerate(sorted_list): @@ -1733,7 +1790,7 @@ def map_tags(options, tm): def sort_suffix(tag): """To determine what sort suffix is appropriate for a given tag""" - if tag == "composer" or tag == "artist" or tag == "albumartist" or tag == "trackartist": + if tag == "composer" or tag == "artist" or tag == "albumartist" or tag == "trackartist" or tag == "~cea_MB_artist": sort = "sort" else: sort = "_sort" @@ -2043,28 +2100,6 @@ def composer_last_names(self, tm, album): '~cea_warning', 'No composer for this track, but checking parent work.') -# def substitute_name(credit_list, name, sort_name=None): -# """ -# NOT CURRENTLY USED -# :param credit_list: -# :param name: -# :param sort_name: -# :return: -# """ -# new_name = None -# for artist_credit in credit_list: -# if not isinstance(artist_credit[0], list): -# if name == artist_credit[1] or ( -# sort_name and sort_name == artist_credit[2]): -# new_name = artist_credit[0] -# else: -# if artist_credit[0]: -# for i, n in enumerate(artist_credit[0]): -# if name == artist_credit[1][i] or ( -# sort_name and sort_name == artist_credit[2][i]): -# new_name = n -# return new_name - def add_list_uniquely(list_to, list_from): """ @@ -2202,6 +2237,7 @@ def add_artist_info( track = album._new_tracks[-1] tm = track.metadata + # OPTIONS - OVER-RIDE IF REQUIRED if '~ce_options' not in tm: # log.error('Artists gets track first...') get_options(album, track) @@ -2216,10 +2252,6 @@ def add_artist_info( options = config.setting self.options[track] = options - # OPTIONS - OVER-RIDE IF REQUIRED - - if not options["classical_extra_artists"]: - return # CONSTANTS self.ERROR = options["log_error"] self.WARNING = options["log_warning"] @@ -2231,226 +2263,233 @@ def add_artist_info( self.ENSEMBLE_TYPES = self.ORCHESTRAS + self.CHOIRS + self.GROUPS self.SEPARATORS = ['; ', '/ ', ';', '/'] - if self.DEBUG: - log.debug("%s: add_artist_info", PLUGIN_NAME) - - self.track_listing.append(track) - # fix odd hyphens in names for consistency - field_types = ['~albumartists', '~albumartists_sort'] - for field_type in field_types: - if field_type in tm: - field = tm[field_type] - if isinstance(field, list): - for x, it in enumerate(field): - field[x] = it.replace(u'\u2010', u'-') - elif isinstance(field, basestring): - field = field.replace(u'\u2010', u'-') - else: - pass - tm[field_type] = field + # continue? + if not options["classical_extra_artists"]: + return + if not (options["ce_no_run"] and (not tm['~ce_file'] or tm['~ce_file'] == "None")): + # continue + if self.DEBUG: + log.debug("%s: add_artist_info", PLUGIN_NAME) + + self.track_listing.append(track) + # fix odd hyphens in names for consistency + field_types = ['~albumartists', '~albumartists_sort'] + for field_type in field_types: + if field_type in tm: + field = tm[field_type] + if isinstance(field, list): + for x, it in enumerate(field): + field[x] = it.replace(u'\u2010', u'-') + elif isinstance(field, basestring): + field = field.replace(u'\u2010', u'-') + else: + pass + tm[field_type] = field - # log.error('disc = %s, track = %s', tm['discnumber'], tm['tracknumber']) # not currently used - # first time for this album (reloads each refresh) - if tm['discnumber'] == '1' and tm['tracknumber'] == '1': - # get artist aliases - these are cached so can be re-used across - # releases, but are reloaded with each refresh - get_aliases(self, album, options, releaseXmlNode) + # first time for this album (reloads each refresh) + if tm['discnumber'] == '1' and tm['tracknumber'] == '1': + # get artist aliases - these are cached so can be re-used across + # releases, but are reloaded with each refresh + get_aliases(self, album, options, releaseXmlNode) - # xml_type = 'release' - # get performers etc who are related at the release level - relation_list = parse_data( - options, releaseXmlNode, [], 'relation_list') - album_performerList = get_artists( - options, relation_list, 'release') - self.album_performers[album] = album_performerList + # xml_type = 'release' + # get performers etc who are related at the release level + relation_list = parse_data( + options, releaseXmlNode, [], 'relation_list') + album_performerList = get_artists( + options, relation_list, 'release') + self.album_performers[album] = album_performerList - else: - if album in self.album_performers: - album_performerList = self.album_performers[album] else: - album_performerList = [] - - if 'recording' in trackXmlNode.children: - self.globals[track]['is_recording'] = True - for record in trackXmlNode.children['recording']: - # Note that the lists below reflect https://musicbrainz.org/relationships/artist-recording - # Any changes to that DB structure will require changes here - - # get recording artists data - recording_artist_list = parse_data( - options, record, [], 'artist_credit', 'name_credit') - if recording_artist_list: - recording_artist = [] - recording_artistsort = [] - recording_artists = [] - recording_artists_sort = [] - locale = config.setting["artist_locale"] - # NB this is the Picard code in /util - lang = locale.split("_")[0] - - # Set naming option - # Put naming style into preferential list + if album in self.album_performers: + album_performerList = self.album_performers[album] + else: + album_performerList = [] - # naming as for vanilla Picard for track artists - if options['cea_ra_trackartist']: - if config.setting['translate_artist_names'] and lang: - if config.setting['standardize_artists']: - name_style = ['alias', 'sort'] - else: - name_style = ['alias', 'credit', 'sort'] - else: - if not config.setting['standardize_artists']: - name_style = ['credit'] + track_artist_list = parse_data( + options, + trackXmlNode, + [], + 'artist_credit', + 'name_credit' + ) + if track_artist_list: + track_artist = [] + track_artistsort = [] + track_artists = [] + track_artists_sort = [] + locale = config.setting["artist_locale"] + # NB this is the Picard code in /util + lang = locale.split("_")[0] + + # Set naming option + # Put naming style into preferential list + + # naming as for vanilla Picard for track artists + + if options['translate_artist_names'] and lang: + name_style = ['alias', 'sort'] + # documentation indicates that processing should be as below, + # but processing above appears to reflect what vanilla Picard actually does + # if options['standardize_artists']: + # name_style = ['alias', 'sort'] + # else: + # name_style = ['alias', 'credit', 'sort'] + else: + if not options['standardize_artists']: + name_style = ['credit'] + else: + name_style = [] + log.info( + 'Priority order of naming style for track artists = %s', + name_style) + styled_artists = apply_artist_style(options, lang, track_artist_list, name_style, track_artist, track_artistsort, track_artists, track_artists_sort) + tm['artists'] = styled_artists['artists'] + tm['~artists_sort'] = styled_artists['artists_sort'] + tm['artist'] = styled_artists['artist'] + tm['artistsort'] = styled_artists['artistsort'] + + if 'recording' in trackXmlNode.children: + self.globals[track]['is_recording'] = True + for record in trackXmlNode.children['recording']: + # Note that the lists below reflect https://musicbrainz.org/relationships/artist-recording + # Any changes to that DB structure will require changes here + + # get recording artists data + recording_artist_list = parse_data( + options, record, [], 'artist_credit', 'name_credit') + if recording_artist_list: + recording_artist = [] + recording_artistsort = [] + recording_artists = [] + recording_artists_sort = [] + locale = config.setting["artist_locale"] + # NB this is the Picard code in /util + lang = locale.split("_")[0] + + # Set naming option + # Put naming style into preferential list + + # naming as for vanilla Picard for track artists (per documentation rather than actual?) + if options['cea_ra_trackartist']: + if options['translate_artist_names'] and lang: + if options['standardize_artists']: + name_style = ['alias', 'sort'] + else: + name_style = ['alias', 'credit', 'sort'] else: - name_style = [] - # naming as for performers in classical extras - elif options['cea_ra_performer']: - # if config.setting['standardize_artists']: - if options['cea_aliases']: - if options['cea_alias_overrides']: - name_style = ['alias', 'credit'] + if not options['standardize_artists']: + name_style = ['credit'] + else: + name_style = [] + # naming as for performers in classical extras + elif options['cea_ra_performer']: + if options['cea_aliases']: + if options['cea_alias_overrides']: + name_style = ['alias', 'credit'] + else: + name_style = ['credit', 'alias'] else: - name_style = ['credit', 'alias'] + name_style = ['credit'] + else: - name_style = ['credit'] + name_style = [] + if self.INFO: + log.info( + 'Priority order of naming style for recording artists = %s', + name_style) + + styled_artists = apply_artist_style(options, lang, recording_artist_list, name_style, recording_artist, recording_artistsort, recording_artists, recording_artists_sort) + self.append_tag( + tm, '~cea_recording_artists', styled_artists['artists']) + self.append_tag( + tm, + '~cea_recording_artists_sort', + styled_artists['artists_sort']) + self.append_tag( + tm, '~cea_recording_artist', styled_artists['artist']) + self.append_tag( + tm, + '~cea_recording_artistsort', + styled_artists['artistsort']) else: - name_style = [] - if self.INFO: - log.info( - 'Priority order of naming style for recording artists = %s', - name_style) - # Get recording artist and apply style - for acs in recording_artist_list: - for ncs in acs: - artistlist = parse_data( - options, ncs, [], 'artist', 'name', 'text') - sortlist = parse_data( - options, ncs, [], 'artist', 'sort_name', 'text') - names = {} - if lang: - names['alias'] = parse_data( - options, - ncs, - [], - 'artist', - 'alias_list', - 'alias', - 'attribs.locale:' + - lang, - 'attribs.primary:primary', - 'text') - else: - names['alias'] = [] - names['credit'] = parse_data( - options, ncs, [], 'name', 'text') - pairslist = zip(artistlist, sortlist) - names['sort'] = [ - translate_from_sortname( - *pair) for pair in pairslist] - for style in name_style: - if names[style]: - artistlist = names[style] - break - joinlist = parse_data( - options, ncs, [], 'attribs.joinphrase') - - if artistlist: - recording_artist.append(artistlist[0]) - recording_artistsort.append(sortlist[0]) - recording_artists.append(artistlist[0]) - recording_artists_sort.append(sortlist[0]) - - if joinlist: - recording_artist.append(joinlist[0]) - recording_artistsort.append(joinlist[0]) - - recording_artist_str = ''.join(recording_artist) - recording_artistsort_str = ''.join(recording_artistsort) - self.append_tag( - tm, '~cea_recording_artists', recording_artists) - self.append_tag( - tm, - '~cea_recording_artists_sort', - recording_artists_sort) - self.append_tag( - tm, '~cea_recording_artist', recording_artist_str) - self.append_tag( - tm, - '~cea_recording_artistsort', - recording_artistsort_str) - else: - tm['~cea_recording_artists'] = '' - tm['~cea_recording_artists_sort'] = '' - tm['~cea_recording_artist'] = '' - tm['~cea_recording_artistsort'] = '' - - # use recording artist options - tm['~cea_MB_artist'] = tm['artist'] - tm['~cea_MB_artistsort'] = tm['artistsort'] - tm['~cea_MB_artists'] = tm['artists'] - tm['~cea_MB_artists_sort'] = tm['~artists_sort'] - - if options['cea_ra_use']: - if options['cea_ra_replace_ta']: - if tm['~cea_recording_artist']: - tm['artist'] = tm['~cea_recording_artist'] - tm['artistsort'] = tm['~cea_recording_artistsort'] - tm['artists'] = tm['~cea_recording_artists'] - tm['~artists_sort'] = tm['~cea_recording_artists_sort'] - elif not options['cea_ra_noblank_ta']: - tm['artist'] = '' - tm['artistsort'] = '' - tm['artists'] = '' - tm['~artists_sort'] = '' - elif options['cea_ra_merge_ta']: - if tm['~cea_recording_artist']: - tm['artists'] = add_list_uniquely( - tm['artists'], tm['~cea_recording_artists']) - tm['~artists_sort'] = add_list_uniquely( - tm['~artists_sort'], tm['~cea_recording_artists_sort']) - if tm['artist'] != tm['~cea_recording_artist']: - tm['artist'] = tm['artist'] + \ - ' (' + tm['~cea_recording_artist'] + ')' - tm['artistsort'] = tm['artistsort'] + \ - ' (' + tm['~cea_recording_artistsort'] + ')' - - # xml_type = 'recording' - relation_list = parse_data( - options, record, [], 'relation_list') - performerList = album_performerList + \ - get_artists(options, relation_list, 'recording') - # returns [(artist type, instrument or None, artist name, artist sort name, instrument sort, type sort)] - # where instrument sort places solo ahead of additional etc. and type sort applies a custom sequencing - # to the artist types - if performerList: - if self.DEBUG: - log.debug( - "%s: Performers: %s", - PLUGIN_NAME, - performerList) - self.set_performer(album, track, performerList, tm) + tm['~cea_recording_artists'] = '' + tm['~cea_recording_artists_sort'] = '' + tm['~cea_recording_artist'] = '' + tm['~cea_recording_artistsort'] = '' + + # use recording artist options + tm['~cea_MB_artist'] = tm['artist'] + tm['~cea_MB_artistsort'] = tm['artistsort'] + tm['~cea_MB_artists'] = tm['artists'] + tm['~cea_MB_artists_sort'] = tm['~artists_sort'] + + if options['cea_ra_use']: + if options['cea_ra_replace_ta']: + if tm['~cea_recording_artist']: + tm['artist'] = tm['~cea_recording_artist'] + tm['artistsort'] = tm['~cea_recording_artistsort'] + tm['artists'] = tm['~cea_recording_artists'] + tm['~artists_sort'] = tm['~cea_recording_artists_sort'] + elif not options['cea_ra_noblank_ta']: + tm['artist'] = '' + tm['artistsort'] = '' + tm['artists'] = '' + tm['~artists_sort'] = '' + elif options['cea_ra_merge_ta']: + if tm['~cea_recording_artist']: + tm['artists'] = add_list_uniquely( + tm['artists'], tm['~cea_recording_artists']) + tm['~artists_sort'] = add_list_uniquely( + tm['~artists_sort'], tm['~cea_recording_artists_sort']) + if tm['artist'] != tm['~cea_recording_artist']: + tm['artist'] = tm['artist'] + \ + ' (' + tm['~cea_recording_artist'] + ')' + tm['artistsort'] = tm['artistsort'] + \ + ' (' + tm['~cea_recording_artistsort'] + ')' + + # xml_type = 'recording' + relation_list = parse_data( + options, record, [], 'relation_list') + performerList = album_performerList + \ + get_artists(options, relation_list, 'recording') + # returns [(artist type, instrument or None, artist name, artist sort name, instrument sort, type sort)] + # where instrument sort places solo ahead of additional etc. and type sort applies a custom sequencing + # to the artist types + if performerList: + if self.DEBUG: + log.debug( + "%s: Performers: %s", + PLUGIN_NAME, + performerList) + self.set_performer(album, track, performerList, tm) - if not options['classical_work_parts']: - work_artist_list = parse_data( - options, - record, - [], - 'relation_list', - 'attribs.target_type:work', - 'relation', - 'attribs.type:performance', - 'work', - 'relation_list', - 'attribs.target_type:artist') - work_artists = get_artists( - options, work_artist_list, 'work') - set_work_artists(self, album, track, work_artists, tm, 0) - # otherwise composers etc. will be set in work parts + if not options['classical_work_parts']: + work_artist_list = parse_data( + options, + record, + [], + 'relation_list', + 'attribs.target_type:work', + 'relation', + 'attribs.type:performance', + 'work', + 'relation_list', + 'attribs.target_type:artist') + work_artists = get_artists( + options, work_artist_list, 'work') + set_work_artists(self, album, track, work_artists, tm, 0) + # otherwise composers etc. will be set in work parts + else: + tm['000_warning'] = "WARNING: Classical Extras not run for this track as no file present - " \ + "deselect the option on the advanced tab to run. If there is a file, then try 'Refresh'." if track_metadata['tracknumber'] == track_metadata['totaltracks'] and track_metadata[ 'discnumber'] == track_metadata['totaldiscs']: # last track self.process_album(album) + + # Checks for ensembles def ensemble_type(self, performer): """ @@ -2517,14 +2556,13 @@ def process_album(self, album): if common: for track in self.track_listing: options = self.options[track] - if options['cea_split_lyrics']: + if options['cea_split_lyrics'] and options['cea_lyrics_tag']: tm = track.metadata lcs = longest_common_substring( tmlyrics_list[track], common) start = lcs['start'] length = lcs['length'] end = start + length - # log.error('lyrics start: %s, end: %s', start, end) unique = tmlyrics_list[track][:start] + \ tmlyrics_list[track][end:] tm['~cea_track_lyrics'] = ' '.join(unique) @@ -2671,7 +2709,7 @@ def process_album(self, album): lambda: collections.defaultdict( lambda: collections.defaultdict(dict))) - for opt in plugin_options('artists'): + for opt in plugin_options('artists') + plugin_options('picard'): if 'name' in opt: if 'value' in opt: if options[opt['option']]: @@ -2686,6 +2724,7 @@ def process_album(self, album): self.cea_options['Classical Extras']['Artists options'][name_list[0] ][name_list[1]] = options[opt['option']] + if options['ce_version_tag'] and options['ce_version_tag'] != "": self.append_tag( tm, @@ -2719,48 +2758,6 @@ def append_tag(self, tm, tag, source): log.info("Extra Artists - appending %s to %s", source, tag) append_tag(tm, tag, source, self.SEPARATORS) - # def remove_tag(self, tm, tag, source): - # """ - # NO LONGER USED - # :param tm: - # :param tag: - # :param source: - # :return: - # """ - # if self.INFO: - # log.info("Extra Artists - removing %s from %s", source, tag) - # if tag in tm: - # if isinstance(source, basestring): - # source = source.replace(u'\u2010', u'-') - # if source in tm[tag]: - # if isinstance(tm[tag], list): - # old_tag = tm[tag] - # else: - # old_tag = re.split('|'.join(self.SEPARATORS), tm[tag]) - # new_tag = old_tag - # for i, tag_item in enumerate(old_tag): - # if tag_item == source: - # new_tag.pop(i) - # tm[tag] = new_tag - - # def update_tag(self, tm, tag, old_source, new_source): - # """ - # NO LONGER USED - # :param tm: - # :param tag: - # :param old_source: - # :param new_source: - # :return: - # """ - # # if old_source does not exist, it will just append new_source - # if self.INFO: - # log.info( - # "Extra Artists - updating %s from %s to %s", - # tag, - # old_source, - # new_source) - # self.remove_tag(tm, tag, old_source) - # self.append_tag(tm, tag, new_source) def set_performer(self, album, track, performerList, tm): """ @@ -2864,7 +2861,7 @@ def set_performer(self, album, track, performerList, tm): if True and artist_type in [ 'instrument', 'vocal', - 'performing orchestra']: # There may be an option here, + 'performing orchestra']: # There may be an option here (to replace 'True') # Currently groups instruments by artist - alternative has been # tested if required instrument = artist_inst_list[tuple(performer[3])] @@ -2957,7 +2954,6 @@ def set_performer(self, album, track, performerList, tm): # fix cyrillic names if not already fixed if options['cea_cyrillic']: if not only_roman_chars(name): - # if not only_roman_chars(tm[tag]): name = remove_middle(unsort(sort_name)) # Only remove middle name where the existing # performer is in non-latin script @@ -3260,7 +3256,7 @@ def add_work_info( # now process works workIds = dict.get(track_metadata, 'musicbrainz_workid', []) - if workIds: + if workIds and not (options["ce_no_run"] and (not tm['~ce_file'] or tm['~ce_file'] == "None")): # works = dict.get(track_metadata, 'work', []) work_list_info = [] keyed_workIds = {} @@ -3300,6 +3296,7 @@ def add_work_info( else: key = 'no key - id seq: ' + unicode(i) keyed_workIds[key] = workId + partial = False for key in sorted(keyed_workIds.iterkeys()): workId = keyed_workIds[key] work_rels = parse_data( @@ -3323,12 +3320,6 @@ def add_work_info( workId, 'title', 'text') - # if self.INFO: - # log.info( - # 'Work details. Rels: %s \n Attributes: %s \n Titles: %s', - # work_rels, - # work_attributes, - # work_titles) work_list_info_item = { 'id': workId, 'attributes': work_attributes, @@ -3338,7 +3329,6 @@ def add_work_info( for title in work_titles: work.append(title) - partial = False if options['cwp_partial']: # treat the recording as work level 0 and the work of which it # is a partial recording as work level 1 @@ -3732,17 +3722,24 @@ def work_process(self, workId, tries, response, reply, error): PLUGIN_NAME, album._requests) if tries < self.MAX_RETRIES: + user_data = True if self.DEBUG: log.debug("REQUEUEING...") - self.work_add_track(album, track, workId, tries + 1) + if str(error) == '204': # Authentication error + if self.DEBUG: + log.debug("... without user authentication") + user_data = False + self.append_tag(track.metadata, '~cwp_error', + 'Authentication failure - data retrieval omits user-specific requests') + self.work_add_track(album, track, workId, tries + 1, user_data) else: if self.ERROR: log.error( "%s: EXHAUSTED MAX RE-TRIES for XML lookup for track %s", PLUGIN_NAME, track) - track.metadata[ - '~cwp_error'] = "ERROR: MISSING METADATA due to network errors. Re-try or fix manually." + self.append_tag(track.metadata, '~cwp_error', + "ERROR: MISSING METADATA due to network errors. Re-try or fix manually.") self.album_remove_request(album) return tuples = self.works_queue.remove(workId) @@ -3837,10 +3834,18 @@ def work_process(self, workId, tries, response, reply, error): tuple(parentIds)) # de-duplicate the parent names - self.parts[tuple(parentIds)]['name'] = list( - collections.OrderedDict.fromkeys(self.parts[tuple(parentIds)]['name'])) - # list(set()) won't work as need to retain - # order + self.parts[tuple(parentIds)]['name'] = list( + collections.OrderedDict.fromkeys(self.parts[tuple(parentIds)]['name'])) + # list(set()) won't work as need to retain + # order + + # de-duplicate the parent ids also, otherwise they will be treated as a separate parent + # in the trackback structure + self.parts[wid]['parent'] = list( + collections.OrderedDict.fromkeys(self.parts[wid]['parent'])) + self.works_cache[wid] = list( + collections.OrderedDict.fromkeys(self.works_cache[wid])) + if self.DEBUG: log.debug( '%s: added parent ids to work_listing: %s, [Requests = %s]', @@ -3949,8 +3954,10 @@ def work_process_metadata(self, workId, wid, tuples, response): alias) relation_list = parse_data( log_options, response.metadata[0].work, [], 'relation_list') + rep_track = None for track, _ in tuples: - rep_track = track # Representative track for option ident only + rep_track = track # Representative track for option ident only, + # otherwise rep_track = None will yield Picard.ini settings return self.work_process_relations( rep_track, workId, wid, relation_list) @@ -3986,7 +3993,10 @@ def work_process_relations(self, track, workId, wid, relations): "%s In work_process_relations. Relations--> %s", PLUGIN_NAME, relations) - options = self.options[track] + if track: + options = self.options[track] + else: + options = config.setting log_options = {'log_debug': self.DEBUG, 'log_info': self.INFO} new_workIds = [] new_works = [] @@ -4898,9 +4908,17 @@ def set_metadata(self, part_level, workId, parentId, parent, track): diff = "" strip = [diff, parent] # but don't leave name of arrangement blank unless it is virtually the same as the parent... - if not diff and 'arrangement' in self.parts[workId] and self.parts[workId]['arrangement']: - strip = self.strip_parent_from_work( - work, parent, part_level, False) + clean_work = re.sub("(?u)[\W]", ' ', work) + clean_work_list = clean_work.split() + extra_words = False + for work_word in clean_work_list: + if work_word not in parent: + extra_words = True + break + if extra_words: + if not diff and 'arrangement' in self.parts[workId] and self.parts[workId]['arrangement']: + strip = self.strip_parent_from_work( + work, parent, part_level, False) else: extend = True strip = self.strip_parent_from_work( @@ -5090,7 +5108,6 @@ def extend_metadata(self, top_info, track, ref_height, depth): part.append(tm['~cwp_part_' + unicode(level)]) work.append(tm['~cwp_work_' + unicode(level)]) work.append(tm['~cwp_work_' + unicode(part_levels)]) - # log.error('part list = %s', part) # Use level_0-derived names if applicable if options["cwp_level0_works"]: @@ -5169,10 +5186,7 @@ def extend_metadata(self, top_info, track, ref_height, depth): log.info("TW_STR = %s", tw_str) if tw_str in tm: title_tag.append(tm[tw_str]) - # if title_tag[d] == title_tag[d - 1]: - # title_work = '' - # else: - title_work = title_tag[d] # indent if re-instate else + title_work = title_tag[d] work_main = work[d] diff_work[d - 1] = self.diff_pair(track, tm, work_main, title_work) @@ -5578,48 +5592,7 @@ def append_tag(self, tm, tag, source, sep=None): tm[tag] ) - # def remove_tag(self, tm, tag, source): - # """ - # NO LONGER USED - # :param tm: - # :param tag: - # :param source: - # :return: - # """ - # if self.INFO: - # log.info("Work Partss - removing %s from %s", source, tag) - # if tag in tm: - # if isinstance(source, basestring): - # source = source.replace(u'\u2010', u'-') - # if source in tm[tag]: - # if isinstance(tm[tag], list): - # old_tag = tm[tag] - # else: - # old_tag = re.split('|'.join(self.SEPARATORS), tm[tag]) - # new_tag = old_tag - # for i, tag_item in enumerate(old_tag): - # if tag_item == source: - # new_tag.pop(i) - # tm[tag] = new_tag - - # def update_tag(self, tm, tag, old_source, new_source): - # """ - # NO LONGER USED - # :param tm: - # :param tag: - # :param old_source: - # :param new_source: - # :return: - # """ - # # if old_source does not exist, it will just append new_source - # if self.INFO: - # log.info( - # "Work Parts - updating %s from %s to %s", - # tag, - # old_source, - # new_source) - # self.remove_tag(tm, tag, old_source) - # self.append_tag(tm, tag, new_source) + ################################################ # SECTION 6 - Common string handling functions # ################################################ @@ -5640,7 +5613,7 @@ def strip_parent_from_work( :param parentId: :return: """ - # extend=True is used [ NO LONGER to find "full_parent" names] and also (with parentId) to trigger recursion if unable to strip parent name from work + # extend=True is used [ NO LONGER to find "full_parent" names] + (with parentId) to trigger recursion if unable to strip parent name from work # extend=False is used when this routine is called for other purposes # than strict work: parent relationships if self.DEBUG: @@ -5663,7 +5636,7 @@ def strip_parent_from_work( pattern_parent = "(.*\s|^)(\W*" + \ pattern_parent + "\w*)(\W*\s)(.*)" else: - pattern_parent = "(.*\s|^)(\W*" + pattern_parent + "w*\W?)(.*)" + pattern_parent = "(.*\s|^)(\W*" + pattern_parent + "\w*\W?)(.*)" if self.INFO: log.info("Pattern parent: %s, Work: %s", pattern_parent, work) p = re.compile(pattern_parent, re.IGNORECASE | re.UNICODE) @@ -5686,6 +5659,7 @@ def strip_parent_from_work( # HOWEVER, this next section has been removed, because it can cause incorrect answers if lower level # works are inconsistently named. Use of level_0 naming can achieve result better and # We want top work to be MB-canonical, regardless + # Nevertheless, this code is left in comments in case it proves useful again # if m.group(3) != ": " and extend: # # no. of colons is consistent with "work: part" structure # if work.count(": ") >= part_level: @@ -5995,7 +5969,6 @@ def diff_pair(self, track, tm, mb_item, title_item): ti_zip_list = zip(ti_list, ti_list_punc) # len(ti_list) should be = len(ti_test_list) as only difference should be synonyms which are each one word - # ti_list = ti_test_list mb_list2 = re.findall(r"\b\w+?\b|\B\&\B", mb_test, re.UNICODE) for index, mb_bit2 in enumerate(mb_list2): mb_list2[index] = self.boil(mb_bit2) @@ -6396,10 +6369,12 @@ def save(self): # MAIN ROUTINE # ################# +# set defaults for certain options that MUST be manually changed by the user each time they are to be over-ridden config.setting['use_cache'] = True config.setting['log_debug'] = False config.setting['log_info'] = False config.setting['ce_options_overwrite'] = False + register_track_metadata_processor(PartLevels().add_work_info) register_track_metadata_processor(ExtraArtists().add_artist_info) register_options_page(ClassicalExtrasOptionsPage) diff --git a/plugins/classical_extras/options_classical_extras.ui b/plugins/classical_extras/options_classical_extras.ui index 3b03a84a..e85e0a37 100644 --- a/plugins/classical_extras/options_classical_extras.ui +++ b/plugins/classical_extras/options_classical_extras.ui @@ -78,7 +78,7 @@ 0 - 0 + -162 1157 1025 @@ -4129,7 +4129,7 @@ alternate-background-color: rgb(135, 157, 255); 0 0 - 1022 + 1157 1012 @@ -5038,11 +5038,30 @@ text-decoration: underline; 0 0 - 1022 - 877 + 1157 + 933 + + + + background-color: rgb(170, 170, 164); + + + General + + + + + + Do not run Classical Extras for tracks where no pre-existing file is detected (warning tag will be written) + + + + + + diff --git a/plugins/classical_extras/ui_options_classical_extras.py b/plugins/classical_extras/ui_options_classical_extras.py index 8e966da1..300be438 100644 --- a/plugins/classical_extras/ui_options_classical_extras.py +++ b/plugins/classical_extras/ui_options_classical_extras.py @@ -2,7 +2,7 @@ # Form implementation generated from reading ui file 'C:\Users\Mark\Documents\Mark's documents\Music\Picard\Classical Extras development\classical_extras\options_classical_extras.ui' # -# Created: Fri Dec 29 23:20:23 2017 +# Created: Wed Jan 03 23:04:16 2018 # by: PyQt4 UI code generator 4.10 # # WARNING! All changes made in this file will be lost! @@ -1420,7 +1420,7 @@ def setupUi(self, ClassicalExtrasOptionsPage): self.scrollArea_3.setWidgetResizable(True) self.scrollArea_3.setObjectName(_fromUtf8("scrollArea_3")) self.scrollAreaWidgetContents_3 = QtGui.QWidget() - self.scrollAreaWidgetContents_3.setGeometry(QtCore.QRect(0, 0, 1022, 1012)) + self.scrollAreaWidgetContents_3.setGeometry(QtCore.QRect(0, 0, 1157, 1012)) sizePolicy = QtGui.QSizePolicy(QtGui.QSizePolicy.Preferred, QtGui.QSizePolicy.Preferred) sizePolicy.setHorizontalStretch(0) sizePolicy.setVerticalStretch(0) @@ -1819,10 +1819,19 @@ def setupUi(self, ClassicalExtrasOptionsPage): self.scrollArea_2.setWidgetResizable(True) self.scrollArea_2.setObjectName(_fromUtf8("scrollArea_2")) self.scrollAreaWidgetContents_2 = QtGui.QWidget() - self.scrollAreaWidgetContents_2.setGeometry(QtCore.QRect(0, 0, 1022, 877)) + self.scrollAreaWidgetContents_2.setGeometry(QtCore.QRect(0, 0, 1157, 933)) self.scrollAreaWidgetContents_2.setObjectName(_fromUtf8("scrollAreaWidgetContents_2")) self.verticalLayout_18 = QtGui.QVBoxLayout(self.scrollAreaWidgetContents_2) self.verticalLayout_18.setObjectName(_fromUtf8("verticalLayout_18")) + self.groupBox_3 = QtGui.QGroupBox(self.scrollAreaWidgetContents_2) + self.groupBox_3.setStyleSheet(_fromUtf8("background-color: rgb(170, 170, 164);")) + self.groupBox_3.setObjectName(_fromUtf8("groupBox_3")) + self.verticalLayout_5 = QtGui.QVBoxLayout(self.groupBox_3) + self.verticalLayout_5.setObjectName(_fromUtf8("verticalLayout_5")) + self.ce_no_run = QtGui.QCheckBox(self.groupBox_3) + self.ce_no_run.setObjectName(_fromUtf8("ce_no_run")) + self.verticalLayout_5.addWidget(self.ce_no_run) + self.verticalLayout_18.addWidget(self.groupBox_3) self.advanced_artists = QtGui.QGroupBox(self.scrollAreaWidgetContents_2) self.advanced_artists.setStyleSheet(_fromUtf8("background-color: rgb(170, 170, 164);")) self.advanced_artists.setObjectName(_fromUtf8("advanced_artists")) @@ -2127,7 +2136,7 @@ def setupUi(self, ClassicalExtrasOptionsPage): self.label_8.setBuddy(self.cwp_retries) self.retranslateUi(ClassicalExtrasOptionsPage) - self.tabWidget.setCurrentIndex(0) + self.tabWidget.setCurrentIndex(3) QtCore.QObject.connect(self.cwp_titles, QtCore.SIGNAL(_fromUtf8("clicked(bool)")), self.groupBox_16.setDisabled) QtCore.QObject.connect(self.cwp_works, QtCore.SIGNAL(_fromUtf8("clicked(bool)")), self.groupBox_16.setEnabled) QtCore.QObject.connect(self.cwp_extended, QtCore.SIGNAL(_fromUtf8("clicked(bool)")), self.groupBox_16.setEnabled) @@ -2816,6 +2825,8 @@ def retranslateUi(self, ClassicalExtrasOptionsPage): self.cwp_write_sk.setText(_translate("ClassicalExtrasOptionsPage", "Write SongKong-compatible work tags*", None)) self.label_82.setText(_translate("ClassicalExtrasOptionsPage", "* ASTERISKED OPTIONS ARE NOT SAVED IN FILE TAGS", None)) self.tabWidget.setTabText(self.tabWidget.indexOf(self.Works), _translate("ClassicalExtrasOptionsPage", "Works and parts", None)) + self.groupBox_3.setTitle(_translate("ClassicalExtrasOptionsPage", "General", None)) + self.ce_no_run.setText(_translate("ClassicalExtrasOptionsPage", "Do not run Classical Extras for tracks where no pre-existing file is detected (warning tag will be written)", None)) self.advanced_artists.setToolTip(_translate("ClassicalExtrasOptionsPage", "

Separate multiple names by commas. Do not use any quotation marks.

", None)) self.advanced_artists.setWhatsThis(_translate("ClassicalExtrasOptionsPage", "

Permits the listing of strings by which ensembles of different types may be identified. This is used by the plugin to place performer details in the relevant hidden variables and thus make them available for use in the "Tag mapping" tab as sources for any required tags.

If it is important that only whole words are to be matched, be sure to include a space after the string.

", None)) self.advanced_artists.setTitle(_translate("ClassicalExtrasOptionsPage", "Artists", None))