tool-v0.18.1

[tool-v0.18.1] - 2024-11-30: Hotfixes

Fixed

tool-v0.18.0 introduced a bunch of issues:

• organize:

  • Fixed organize --symlink --latest sometimes overwriting (plain WRR) source files with symlinks to their newer versions.

    The good news is that this bug was only triggered when organize --symlink --latest was run with some newly archived data and, for each updated URL, it only overwrote the second to last WRR file with a symlink to the latest WRR file.
    Unfortunately, this error was self-propagating, so those files could then get overwritten again by the next invocation of organize --symlink --latest with some more new data.

    You can check if you were affected by running:

    cd ~/web/raw ; find . -type l

    The paths it outputs will be the paths of lost WRR files.

    A reminder that it is good to do daily backups, I suppose.

    The next version will have a test for this, but I'm releasing this hotfix an hour after I discovered this.

  • Fixed it assert-crashing sometimes when running with --symlink.

  • Improved memory consumption a bit.

• export mirror:

  • Fixed overly large memory consumption.

tool-v0.18.0

[tool-v0.18.0] - 2024-11-20: Incremental improvements

Added

• export mirror:

  • Implemented the --boring option, which allows you to load some input PATHs without adding them as roots, even when no --root-* options are specified.

    This make CLI a bit more convenient to use.
    The README.md has a new example showcasing it.

• export mirror, scrub:

  • Implemented support for @import CSS rules using a string token in place of a URL.

    As far as I can see, this syntax is rarely used in practice.
    But the spec allows this, so.

  • Implemented interpret_noscript option, which enables inlining of noscript tags when scrub is running with -scripts.

    That is, export mirror will now use this feature by default.

    This is needed because some websites put link tags with CSS under noscript, thus making such pages look broken when scrubbed with -scripts (which is the default) and then opened in a browser with scripts enabled.

Changed

• *: Refactored/reworked a large chunk of internals, as a result:

  • organize can now take WRR bundles as inputs too,
  • export mirror became much faster at indexing inputs that contain archives of the same URLs, repeatedly.

  In general, these changes are aimed towards making hoardy-web completely input-agnostic.
  That is, wouldn't it be nice if you could feed mitmproxy files to export mirror directly, instead of going through import mitmproxy first?

• export mirror, scrub:

  • From now on, it will stop generating link tags with void URLs, it will simply censor them out instead.

  • scrub with +verbose set will now also show original rel attr values for censored out tags.

  • Also, in general, the outputs of scrub with +verbose set are much prettier now.

• Improved documentation.

tool-v0.17.0

See CHANGELOG.md.

extension-v1.17.2

[extension-v1.17.2] - 2024-11-09: Documentation fixes, mostly

Changed

• The Help page:

  • Rewrote "Conventions" and "'Work offline' mode" sections of to be much more readable.

• *:

  • Improved contrast when running with a light CSS color scheme.

Fixed

• Documentation:

  • Fixed some typos.

• *:

  • Fixed some potential state display inconsistency bugs and improved UI pages' init performance when the core is very busy.

extension-v1.17.1

[extension-v1.17.1] - 2024-11-01: Annoyance fixes

Changed

• Popup UI:

  • Reverted most of the block reordering bit of popup UI rework of extension-v1.17.0.

    The "Globally" block is near the top again.

  • Edited the "Persistence" block a bit more.

    Mainly, to stop graying out always-useful stat lines, even when the associated features are disabled, to prevent possible confusion there.

  • Renamed some options and stat lines, mostly to make their names shorter to make popup UI on Fenix more readable.

• Toolbar button:

  • Edited its title format to be much shorter, especially on Fenix.

  • Reverted the ordering of parts there to how it was before extension-v1.17.0.

    The (much shorter now) "globally" part is at the front again because otherwise the badge being at the front there too without an explanation of its format is kind of confusing.

• Core + All internal pages:

  • Improved message handling infrastructure.

  • Used it to improve initialization functions of all internal pages, improving efficiency and making the resulting UI much less flaky.

• The Help page:

  • Documented what webNavigation permission is used for, improved the rest a bit.

• *:

  • Renamed build.sh firefox target to firefox-mv2, for consistency.

Fixed

• UI:

  • Fixed flaky rendering of Help and Changelog pages on Fenix.

    They render properly now the very first time you load them, no reloads needed.

  • Fixed duplication of history entries when navigating internal links.

  • Fixed source links sometimes failing to being highlighted when pressing the browser's "Back" button.

  • Fixed some small CSS nitpicks.

• Popup UI + Documentation:

  • Realigned some help strings with reality.

• Fixed some more mostly inconsequential things.

extension-v1.17.0

[extension-v1.17.0] - 2024-10-30: Halloween special: major UI and state display improvements, fine-grained Work offline mode, add-on reloading with its state preserved, new options, etc

In related news, I have 💸☕ a Patreon account now.

Fixed: Possibly important

• Core:

  • Fixed a bug in upgradeConfig that was resetting bucket settings to their default values or upgrade to extension-v1.13.0.
    So, this is no longer relevant, but still.
    Also, refactored code there to prevent such errors in the future.

    However, just in case, if you previously set bucket settings to something other than their default values and those settings are important to you, you should probably check your settings to ensure everything there is set as you expect it to be.

Changed: Important UI

• Core + Popup UI + Documentation:

  • Renamed failed state and related failed* stats to unarchived state and unarchived* stats.
    Introduced a new failed stat that is now a sum of unstashed and unarchived stats.
    Edited the popup UI and the other pages appropriately.

    This makes documentation's terminology more consistent, and simplifies UI a bit.

    In particular, the Retry button of Queued/Failed stat line will both retry stashing unstashed and archiving unarchived reqres now.

• Popup UI:

  • Reworked the whole thing quite a bit:

    • Improved option names and help strings.
    • Sorted sections and options to follow a more logically consistent order.
    • Improved layout.
    • Fixed some typos there.

  • From now on, setting Bucket for the current tab will set Bucket for its new children too, similar to how the rest of those settings work.

  • From now on, setting any of the Bucket settings to nothing will reset it to the parent/default value.
    I.e.:

    • Setting Bucket of This tab's new children to nothing will reset it to Bucket value of This tab.
    • Setting Bucket of This tab to nothing will reset it to Bucket value of New root tabs.
    • Setting Bucket of New root tabs to nothing will reset it to default.

• The Help page:

  • The previous "Desktop" JavaScript-generated layout became columns CSS layout and JS-operation mode, while the "Mobile" JavaScript-generated layout became linear CSS layout and JS-operation mode.
    The page will now automatically switch between these two layouts and modes synchronously, depending on viewport width.

    (As before, in linear mode hovering over a link does nothing, but in columns mode, hovering over a link referring to a target in popup UI scrolls the popup UI column to that target and highlights it.)

    I.e., this means that on a Desktop browser, you can now zoom the Help page to arbitrary zoom levels and it will just switch between layouts and link-hover behaviors depending on available viewport width.

  • Greatly improved the styling of all links and documented it in the "Conventions" section.

• All internal pages:

  • All internal pages now color-code links depending on where they point to, using exactly the same CSS as the Help page.

  • All pages now use the same history state handling behaviour.

    I.e., using the "Back" button of your browser will now not only go back, but also highlight the last link you clicked.

  • All documentation pages now set viewport width to device-width, set content's max-width to 900px and width to 100% - padding, preventing horizontal scroll, when possible.

  • Improved the CSS styling in general.

• Core + Popup UI + General UI:

  • Implemented a new popup UI tristate toggle named Color scheme which allows Hoardy-Web's color-scheme to be different from the browser's default.

  • Implemented a mechanism and popup UI settings for applying additional themes and experimental features.

  • And then I looked at the date. Which is why ◥▅◤◢▅◣◥▅◤ Hoardy-Web now has 🦇 Halloween mode. ◥▅◤◢▅◣◥▅◤.

  • Also, from now on, the neutral states of tristate toggles are displayed with toggle knobs being in the middle of the things, not on their left.
    This is not a political statement.
    This mans that all tristate toggles, from left to right, now go false -> null -> true both internally (exactly as they did before) and externally (which is new).

Changed: State display

• Core + Toolbar button + Icons:

  • Replaced toolbar button's icons representing Cartesian products of other icons with animations.

    In other words, the previous "this tab has limbo mode enabled while this tab's children do not" icon will now instead be represented with an animation that switches between "this tab has limbo mode enabled" and "this tab is idle" icons instead.

    This both takes less space in the XPI/CRX, makes for a cuter UI, and is the only reasonable solution when the core wants to display more than two icons at the same time.

  • Improved toolbar button's badge and title format a bit.

    "This tab" part goes first now, then "its new children", then "globally".

    Also, the order of sub-parts of those strings is more consistent now.

  • From now on, internal UI updater will generate icon animation frames for all important statuses and setting states.

    • When per-tab and per-tab's-new-children animation frames are equal, the repeated part will be elided.
    • When per-tab and per-tab's-new-children animation frames differ, the main icon will be inserted at the end to make it obvious when the animation loop restarts (otherwise, it's easy to interpret such animation loops incorrectly).

  • The update frequency of toolbar button's icon, badge, and title now depends on the amount of not yet done stuff still queued in the core.

    I.e., from now on, when the core has a lot of stuff to do (like when re-archiving thousands of reqres at the same time), it will start updating toolbar button's properties less to trade update latency for improved performance, and vice versa.

  • Greatly improved performance of state display updates. It's uses 2-1000x less CPU now, depending on what the core is doing.

• Icons:

  • Renamed the error icon to failed and added a new error icon.

    From now on, the failed icon will only be used for archival/stashing errors, while the error icon will only be used for internal errors (i.e. bugs).

  • Improved all icons to make them more visually distinct when they are being rendered at 48x48 or less, both in light and dark mode.

  • On Chromium, all icons are now rendered with transparent backgrounds, so now they will look nice in the dark mode too.

Added: State display

• Core + Popup UI + Toolbar button:

  • From now on, popup UI and toolbar button's badge and title will display information about currently running internal actions.

    (Implementing this took a surprising amount of effort in improvements to infrastructure code.)

• Core + Toolbar button + Icons:

  • Added a new in_limbo icon for "this tab has data in limbo" status.
    Unlike most other icons, this icon will never be used alone, it will always be an animation frame of something longer.

• Core + Popup UI + Toolbar button:

  • Implemented Animate toolbar icon every setting for controlling toolbar icon animation speed.

Fixed: State display

• Core + Toolbar button:

  • Fixed a bunch of bugs that prevented updates to toolbar button's icon and badge in some cases.

  • The icon and the badge will no longer get stuck when the core is very busy, like when re-archiving a lot of stuff all at once.

Added: Work offline mode

• Core + Popup UI + Toolbar button + Icons + Documentation:

  • Implemented Work offline mode, options, their popup UI, shortcuts, and icons.

    This mode does the same thing as File > Work Offline checkbox of Firefox, except it supports per-tab/per-other-origin operation, not just the whole-browser one.
    Also, enabling any these options will not break requests that are still in flight, and the requests they do cancel can be logged.

    That is, enabling Work offline in a tab will start canceling all new requests that tab generates, and the resulting canceled reqres will get logged if Track new requests option is enabled in the same tab.
    Similarly for background tasks and other origins.

    This can be generally useful for debugging your own websites with dynamic responsive CSS, or if you just want to prevent a tab from accessing the network for some reason.

    However, the main reason this exists is that the files generated by hoardy-web export mirror do not get scrubbed absolutely correctly at the moment, and the resulting pages can end up with some references to remote resources (in cases when an exported page uses some rare HTML and CSS tag combinations, or lazy-load images via JavaScript, but still).
    With Work offline options enabled in a tab, you can now be sure that opening pages generated by hoardy-web export mirror won't send any requests to the network.

    In fact, from now on, by default, Hoardy-Web will enable Work offline in all tabs pointing to file: URLs.
    This can be disabled in the settings.

  • Documented it in more detain on the Help page.

  • Added a new offline toolbar icon to display the above state.

Added: Reloading with state preserved

• Core + Popup UI + Documentation:

  • Implemented reloadSelf action that reloads the add-on while preserving its state.

    This action is different from similar Reload buttons in browser's own UI in that trigge...

tool-v0.16.0

See CHANGELOG.md.

extension-v1.16.1

[extension-v1.16.1] - 2024-10-15

Fixed

• On Chromium, fixed request tracking being frequently broken since extension-v1.15.0.

• Fixed reqres without responses but with networking errors having "Responded at" field set in the logs.

tool-v0.15.5+simple_server-v1.7.0

See CHANGELOG.md.

tool-v0.15.4

See CHANGELOG.md.