Merge pull request #29 from Moonbase59/master

Add changes in master to v5.0.0-dev

.github/FUNDING.yml

@@ -1,3 +1,3 @@
 # These are supported funding model platforms
 
-custom: ["https://www.paypal.me/Moonbase59"]
+custom: ["https://www.paypal.com/donate/?hosted_button_id=PBPR63362LDEU"]

CHANGELOG.md

@@ -1,5 +1,12 @@
 # autocue changelog
 
+### 2024-07-01 – v4.0.4
+
+- Allow to override results via JSON even _after_ having done a fresh analysis (automatic or forced), for ultimate flexibility when using `cue_file` for pre-processing. You can now add fades, ramp or hook points, or do other calculations and feed the results into `cue_file` for tagging. **Use with care**, because some values are dependent on others. In any case, `cue_file` will ever _only_ write tags beginning with `liq_` and (if requested) `replaygain_`.
+- Prevent some strange errors that could happen when piping something into `cue_file` and JSON input was not `stdin`. We now use `ffmpeg -nostdin` to prevent it reading input that was meant for `cue_file`.
+- Don’t write _all_ `liq_*` tags (could have side effects), but only those _known_ (see `cue_file --help` for current list).
+- Streamlined tag conversion code a little.
+
 ### 2024-06-18 – v4.0.3
 
 - Changed default of `-x`/`--extra` and `settings.autocue.cue_file.overlay_longtail` from `-15.0` LU to `-12.0` LU, requested by @RM-FM and the community. Together with the `-d`/`--drop` default change from `60.0` to `40.0`, this makes for a "tighter" playout and doesn’t lose too much of long or sustained endings.

FAQ.md

@@ -1,5 +1,5 @@
 ---
-date: 2024-06-23
+date: 2024-07-01
 author: Matthias C. Hormann (Moonbase59)
 ---
 # FAQ – Frequently Asked Questions
@@ -16,6 +16,7 @@ author: Matthias C. Hormann (Moonbase59)
 - [How to pre-process more than one file at a time ("mass tagging")?](#how-to-pre-process-more-than-one-file-at-a-time-mass-tagging)
 - [What tagging software to use?](#what-tagging-software-to-use)
 - [Can I use `cue_file` to replaygain my files?](#can-i-use-cue_file-to-replaygain-my-files)
+- [Can I use `cue_file` to _manually_ add/overwrite tags when pre-processing?](#can-i-use-cue_file-to-manually-addoverwrite-tags-when-pre-processing)
 - [How to make transitions _tighter_, i.e. overlay earlier?](#how-to-make-transitions-tighter-ie-overlay-earlier)
 - [How to make transitions _longer_, i.e. overlay later and keep every bit of a song ending?](#how-to-make-transitions-longer-ie-overlay-later-and-keep-every-bit-of-a-song-ending)
 - [Can I completely _disable_ the "sustained endings" feature?](#can-i-completely-disable-the-sustained-endings-feature)
@@ -25,6 +26,7 @@ author: Matthias C. Hormann (Moonbase59)
 - [Why should I use `check_autocue_setup`, and what does it do?](#why-should-i-use-check_autocue_setup-and-what-does-it-do)
 - [What are good first steps to use Autocue in my own Liquidsoap script?](#what-are-good-first-steps-to-use-autocue-in-my-own-liquidsoap-script)
 - [What will happen when I switch to the upcoming Liquidsoap 2.3.0?](#what-will-happen-when-i-switch-to-the-upcoming-liquidsoap-230)
+- [What can I do if some files or a remote playlist simply _don’t play?_](#what-can-i-do-if-some-files-or-a-remote-playlist-simply-dont-play)
 <!-- Generated by gh-toc, https://moonbase59.github.io/gh-toc/ -->
 <!-- ToC end -->
 
@@ -155,6 +157,36 @@ As always, you should _know what you’re doing_, and set up these tools appropr
   |replaygain_track_range|dB|
 
 
+## <a name="can-i-use-cue_file-to-manually-addoverwrite-tags-when-pre-processing"></a>Can I use `cue_file` to _manually_ add/overwrite tags when pre-processing? <a href="#toc" class="goToc">⇧</a>
+
+- _Yes_, you can, even _after_ forcing a re-analysis (v4.0.4+).
+- _Only_ tags from `cue_file`’s list of _known_ tags (see `cue_file --help`) will ever been written by `cue_file`. So no overriding artist or title here—that’s what should have been done in an earlier step, using a good tagging software.
+- **Use with care!** Some values are dependent on others, you could easily mess up something.
+- You **must** create _well-formed JSON_ and can then use `cue_file` with the `-j`/`--json` switch to let your tags override or add to what’s already there.
+
+#### Example: Adding fade-in and fade-out, using `echo` and `stdin`
+
+```bash
+echo '{"liq_fade_in": 0.1, "liq_fade_out": 0.1}' | cue_file -j - -fwr "filename.ext"
+```
+- `-j -` — sets JSON input to `stdin`
+- `-fwr` — _force_ re-analysis, _write_ tags, write _replaygain_
+
+#### Example: using a JSON file `fades.json`
+
+```json
+{
+  "liq_fade_in": 0.10,
+  "liq_fade_out": 0.10
+}
+```
+```bash
+cue_file -j fades.json -fwr "filename.ext"
+```
+- `-j fades.json` — read JSON data from file `fades.json`
+- `-fwr` — _force_ re-analysis, _write_ tags, write _replaygain_
+
+
 ## <a name="how-to-make-transitions-tighter-ie-overlay-earlier"></a>How to make transitions _tighter_, i.e. overlay earlier? <a href="#toc" class="goToc">⇧</a>
 
 - First, try to _decrease_ `-d`/`--drop`/`settings.autocue.cue_file.sustained_loudness_drop` _gradually_. The default is `40.0`%, so maybe go down in 10% increments and see how you like it.
@@ -307,7 +339,7 @@ Nothing is definite yet, Liquidsoap 2.3.0 is still under heavy development.
 
 Most certainly, as of 2024-06-23…
 
-- you will need a _new version_ 5.x.x of both `autocue.cue_file` and `cue_file`, because Liquidsoap will change the tags and API.
+- you will need a _new version_ of both `autocue.cue_file` and `cue_file`, because Liquidsoap will change the tags and API.
 - you’ll need to _pre-process your files again_, if you have used that feature.
 - I will see that `cue_file` will be able to _remove obsolete tags_.
 - if possible, I’ll do some _checking_ so that you don’t run `autocue.cue_file` (and thus `cue_file`) under an incompatible Liquidsoap version. The `check_autocue_setup()` function will take care of that.
@@ -322,3 +354,33 @@ Most certainly, as of 2024-06-23…
 See also:
 
 - https://github.com/savonet/liquidsoap/pull/3931#issuecomment-2184937173
+
+
+## <a name="what-can-i-do-if-some-files-or-a-remote-playlist-simply-dont-play"></a>What can I do if some files or a remote playlist simply _don’t play?_ <a href="#toc" class="goToc">⇧</a>
+
+This most often happens with _large files_, _large remote files_, and files that are _hard to analyse_ because they’re already clipping (too loud).
+
+Check your Liquidsoap log for entries like
+
+```
+2024/06/26 10:01:43 [request:2] Time limit exceeded by 16.00 secs!
+```
+
+This can happen if the _downloading/preparing is slow_, or the _files are large_ (recordings, prerecorded shows, DJ sets, podcasts) or _hard to analyse_. These factors are out of our control. If a request timeout happens, the file will simply _not be played_, and the _next available used_.
+
+In such cases, I suggest to _pre-tag_ files using the `cue_file` tool, but this might not always be possible.
+
+In the above case, I’d suggest _increasing the Autocue timeout slightly_ from the default `60.0` seconds to `60` + timeout shown (`16.0`) + a small "safety margin", say another `10` seconds, making for `86` seconds. Let’s round that up to `90` seconds.
+
+In AzuraCast’s _Edit Liquidsoap Configuration_, in the _second input box_, add this parameter:
+```
+settings.autocue.cue_file.timeout := 90.0
+```
+
+**Note:** Please do _not_ use arbitrary large values like `480.0` "just to prevent it happening"! Doing so could well have adverse side effects.
+
+_Save Changes_ and _Restart Broadcasting_.
+
+Keep watching your logs for timeouts, just in case even larger files arrive. You can search for `Time limit exceeded` to quickly find such entries in the log.
+
+If the above should _not_ help, you should look for a _faster/higher bandwidth connection_ to the source you download the files from. To be able to _work_ with a file, Liquidsoap has to download a complete copy first, before autocueing and playout can begin.

README.md

@@ -1,5 +1,7 @@
 # <a name="autocue"></a>autocue <a href="#toc" class="goToc">⇧</a>
 
+**If you like what you got, please consider to [![Donate with Paypal](https://www.paypalobjects.com/en_US/i/btn/btn_donate_LG.gif)](https://www.paypal.com/donate/?hosted_button_id=PBPR63362LDEU). Thank you! ❤️**
+
 On-the-fly JSON song cue-in, cue-out, overlay, replaygain calculation for Liquidsoap, AzuraCast and other AutoDJ software.
 
 **Check out the [presentation](https://moonbase59.github.io/autocue/presentation/autocue.html) for an introduction, the [FAQ – Frequently Asked Questions](FAQ.md), and the [changelog](CHANGELOG.md) for latest information!**

autocue.cue_file.liq

@@ -37,6 +37,7 @@
 # 2024-06-16 - Moonbase59 - v4.0.2 - Allow `-8.33dB` type values with no blank
 # 2024-06-18 - Moonbase59 - v4.0.3 - Changed overlay_longtail from -15 to -12,
 #                           most people seem to want transitions a bit tighter
+# 2024-07-01 - Moonbase59 - v4.0.4 - Sync with cue_file version
 # 2024-07-01 - Moonbase59 - v5.0.0-dev - New JSON API
 
 # Lots of debugging output for AzuraCast in this, will be removed eventually.

cue_file

@@ -58,6 +58,11 @@
 #                         for slightly tighter/denser playout (community wish)
 # 2024-06-18 Moonbase59 - v4.0.3 Change LONGTAIL_EXTRA_LU from -15 to -12,
 #                         most people seem to want transitions a bit tighter
+# 2024-07-01 Moonbase59 - v4.0.4 Fix JSON override after analysis
+#                       - Add `-nostdin` to ffmpeg commands, prevents strange
+#                         errors when piping something to cue_file
+#                       - streamline tag conversion code a little
+#                       - only write known tags, not all `liq_*`
 # 2024-07-01 Moonbase59 - v5.0.0-dev
 #                       - New more compatible JSON output
 #
@@ -252,6 +257,56 @@ def amplify_correct(target, loudness, true_peak_dB, noclip):
     return amplify, amplify_correction
 
 
+# remove " dB", " LU", " dBFS", " dBTP" and " LUFS" suffixes from
+# tags_found
+def remove_suffix(tags):
+    suffixed_tags = [
+        "liq_amplify", "liq_amplify_adjustment",
+        "liq_loudness", "liq_loudness_range", "liq_reference_loudness",
+        "replaygain_track_gain", "replaygain_track_range",
+        "replaygain_reference_loudness",
+        "liq_true_peak_db",
+        "liq_true_peak",  # in case old " dBFS" values were stored in v1.2.3
+    ]
+    for tag in suffixed_tags:
+        if tag in tags and isinstance(tags[tag], str):
+            # No need to check for unit name, only using defined tags
+            m = re.search(r'([+-]?\d*\.?\d+)', tags[tag])
+            if m is not None:
+                tags[tag] = m.group()
+
+    return tags
+
+
+# convert tags into their typed variants, ready for calculations
+def convert_tags(tags):
+    items = tags.items()
+
+    # make keys lowercase, include only tags in tags_to_check
+    tags = {
+        k.lower(): v for k,
+        v in items if k.lower() in tags_to_check}
+
+    # remove suffixes from several tags
+    tags = remove_suffix(tags)
+
+    # convert tag string values to the correct types, listed in tags_to_check
+    tags = {k: tags_to_check[k](v) for k, v in tags.items()}
+
+    return tags
+
+# override a (typed) result with JSON overrides
+def override_from_JSON(tags, tags_json={}):
+    # get tags in JSON override file
+    tags_in_json = convert_tags(tags_json)
+
+    # unify, right overwrites left if key in both
+    # tags_found = tags_in_stream | tags_in_format | tags_in_json
+    tags = {**tags, **tags_in_json}
+
+    return tags
+
+
 def read_tags(
         filename,
         tags_json={},
@@ -284,28 +339,20 @@ def read_tags(
 
     # get tags in stream #0 (mka, opus, etc.)
     try:
-        stream_items = result['streams'][0]['tags'].items()
+        stream_tags = result['streams'][0]['tags']
     except KeyError:
-        stream_items = {}
+        stream_tags = {}
 
     # get tags in format (flac, mp3, etc.)
     try:
-        format_items = result['format']['tags'].items()
+        format_tags = result['format']['tags']
     except KeyError:
-        format_items = {}
+        format_tags = {}
 
-    # get tags in JSON override file
-    json_items = tags_json.items()
+    tags_in_stream = convert_tags(stream_tags)
+    tags_in_format = convert_tags(format_tags)
+    tags_in_json = convert_tags(tags_json)
 
-    tags_in_stream = {
-        k.lower(): v for k,
-        v in stream_items if k.lower() in tags_to_check}
-    tags_in_format = {
-        k.lower(): v for k,
-        v in format_items if k.lower() in tags_to_check}
-    tags_in_json = {
-        k.lower(): v for k,
-        v in json_items if k.lower() in tags_to_check}
     # unify, right overwrites left if key in both
     # tags_found = tags_in_stream | tags_in_format | tags_in_json
     tags_found = {**tags_in_stream, **tags_in_format, **tags_in_json}
@@ -322,29 +369,6 @@ def read_tags(
         except KeyError:
             pass
 
-    # remove " dB", " LU", " dBFS", " dBTP" and " LUFS" suffixes from
-    # tags_found
-    def remove_suffix(tags):
-        suffixed_tags = [
-            "liq_amplify", "liq_amplify_adjustment",
-            "liq_loudness", "liq_loudness_range", "liq_reference_loudness",
-            "replaygain_track_gain", "replaygain_track_range",
-            "replaygain_reference_loudness",
-            "liq_true_peak_db",
-            "liq_true_peak",  # in case old " dBFS" values were stored in v1.2.3
-        ]
-        for tag in suffixed_tags:
-            if tag in tags and isinstance(tags[tag], str):
-                # No need to check for unit name, only using defined tags
-                m = re.search(r'([+-]?\d*\.?\d+)', tags[tag])
-                if m is not None:
-                    tags[tag] = m.group()
-
-        return tags
-
-    # remove suffixes from several tags
-    tags_found = remove_suffix(tags_found)
-
     # create replaygain_track_gain from Opus R128_TRACK_GAIN (ref: -23 LUFS)
     if "r128_track_gain" in tags_found:
         rg = float(tags_found["r128_track_gain"]) / 256 + (target - -23.0)
@@ -355,9 +379,6 @@ def read_tags(
             "replaygain_track_gain" in tags_found):
         tags_found["liq_amplify"] = tags_found["replaygain_track_gain"]
 
-    # convert tag string values to the correct types, listed in tags_to_check
-    tags_found = {k: tags_to_check[k](v) for k, v in tags_found.items()}
-
     # Handle old RG1/mp3gain positive loudness reference
     # "89 dB" (SPL) should actually be -14 LUFS, but as a reference
     # it is usually set equal to the RG2 -18 LUFS reference point
@@ -447,7 +468,7 @@ def add_missing(tags_found, target=TARGET_LUFS, blankskip=0.0, noclip=False):
         tags_found["liq_amplify"] = tags_found["replaygain_track_gain"]
 
     if "liq_amplify_adjustment" not in tags_found:
-        tags_found["liq_amplify_adjustment"] = "0.00 dB"
+        tags_found["liq_amplify_adjustment"] = 0.00  # dB
 
     if "liq_loudness" not in tags_found:
         tags_found["liq_loudness"] = target - \
@@ -503,6 +524,7 @@ def analyse(
         FFMPEG,
         "-v",
         "quiet",
+        "-nostdin",
         "-y",
         "-i",
         filename,
@@ -843,7 +865,7 @@ def write_tags(filename, tags={}, replaygain=False):
         # copy only `liq_*`, float with 2 decimals, bools and strings lowercase
         tags_new = {k: "{:.2f}".format(v)
                     if isinstance(v, float) else str(v).lower()
-                    for k, v in tags.items() if k.startswith("liq_") or k in rg_tags
+                    for k, v in tags.items() if k in tags_to_check or k in rg_tags
                     }
         # liq_true_peak & replaygain_track_peak have 6 decimals, fix it
         if "liq_true_peak" in tags_new:
@@ -957,6 +979,7 @@ def write_tags(filename, tags={}, replaygain=False):
             args = [
                 FFMPEG,
                 '-v', 'quiet',
+                '-nostdin',
                 '-y',
                 '-i', str(filename.absolute()),
                 '-map_metadata', '0',
@@ -1199,6 +1222,9 @@ if args.force or not skip_analysis:
         nice=args.nice,
         noclip=args.noclip
     )
+    # allow to override even the analysis results
+    if args.json:
+        result = override_from_JSON(result, tags_json)
 else:
     result = add_missing(tags_found, args.target, args.blankskip, args.noclip)
 

docs/presentation/autocue.html

@@ -959,8 +959,11 @@ <h1>Links</h1>
 <section id="thanks" class="title-slide slide level1">
 <h1>Thanks!</h1>
 <p><img data-src="images/matthias.png" /><br />
-Matthias C. Hormann<br />
-(“Moonbase59”)</p>
+Matthias C. Hormann (“Moonbase59”)</p>
+<p><strong>If you like what you got, please consider to <a
+href="https://www.paypal.com/donate/?hosted_button_id=PBPR63362LDEU"><img
+data-src="https://www.paypalobjects.com/en_US/i/btn/btn_donate_LG.gif"
+alt="Donate with Paypal" /></a>. Thank you! ❤️</strong></p>
 </section>
     </div>
   </div>

docs/presentation/autocue.md

@@ -745,5 +745,6 @@ GitHub repo:
 # Thanks!
 
 ![ ](images/matthias.png)  
-Matthias C. Hormann  
-(“Moonbase59”)
+Matthias C. Hormann (“Moonbase59”)
+
+**If you like what you got, please consider to [![Donate with Paypal](https://www.paypalobjects.com/en_US/i/btn/btn_donate_LG.gif)](https://www.paypal.com/donate/?hosted_button_id=PBPR63362LDEU). Thank you! ❤️**