Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rewrite the media processing pipeline #84

Open
12 of 20 tasks
ungive opened this issue Nov 15, 2024 · 0 comments
Open
12 of 20 tasks

Rewrite the media processing pipeline #84

ungive opened this issue Nov 15, 2024 · 0 comments
Labels
high priority This is important going forward milestone In progress milestone
Milestone

Comments

@ungive
Copy link
Owner

ungive commented Nov 15, 2024

Right now everything's a bit all over the place with how media is handled internally, from detection, to sanitizing it, fixing badly formatted metadata from some media players, making API requests, to getting a URL for a locally available cover image and more. This needs to be reworked such that that media processing pipeline is well-structured and can be extended with more than just one media source (system media) and more than just one entity that consumes the media (the Discord status).

Reworking this lays the foundation for:

  • Getting media from additional sources, like web browsers, next to system media alone
  • Consuming media for more than just one purpose, e.g. scrobbling, next to just a Discord status
  • Improving maintainability and extensibility for the future and reducing code complexity
  • Less code redundancy with system media detection
  • Improved performance

This is required for #53, #5 and #62.

Mandatory todos:

  • Implement data structures for a media pipeline including different types of hooks that can manipulate and work with media in the pipeline and that can be registered at different points.
  • Implement a universal algorithm to detect changes in media properties between two media instances which compares media based on a selectable subset of properties (e.g. only title and artist and/or other properties)
  • Write tests to ensure existing pipeline data structures work as intended
  • Modify the Media data structure to have any number of unique source identifiers (platform/player ID/disambiguation tuples, collectively called "source ID") and optional source metadata (player display name) per identifier and implement comparison and hashing functions so that the source ID of each Media instance can be used as map keys
  • Modify the underlying system media detection code so it uses the new media source data structures
  • Modify the Linux media detection code to report two source IDs for media from browsers, the MPRIS player ID and the domain name that is retrieved from the MPRIS url field, in order to test that multiple source IDs work correctly and make sure that websites are identified properly
  • Now implement change detection between consecutive pipeline runs, where media from the first pipeline run is compared with media from the second pipeline run in such a way, that media with the same source ID is compared to each other for equality. This is only possible after implementing the above source ID type
  • Allow each consumer (Discord status, scrobbling, etc.) to determine the attributes that need to be compared for changes (using the comparison algorithm from before) for each media instance that passes through the pipeline (i.e. they select which attributes are important to them at any given point in time and per media instance). That way the Discord status consumer can e.g. control that comparing playing media involves the current playback position of the song, but paused media does not (since showing paused media in the status never shows the current playback position). This abstracts away change detection from the consumers and makes the pipeline a lot more efficient since consumers are only executed when the data they care about actually changed and additional expensive operations, like calls to music APIs are only made when really needed.
  • Fix the pipeline to have a copy of the processed media for each consumer and for selectable amender hooks to operate on these copies. Fix amenders selected by one consumer modifying media for another consumer
  • Run each consumer on a separate thread and join those threads during a pipeline run
  • Allow retrieving all registered hooks from the pipeline which will be very useful when embedding toggle checkboxes for them in the UI. That way transformer hooks like the fix for Apple Music do not need to be hardcoded the same way as before
  • Allow to disable each registered pipeline hook individually
  • Allow consumer-specific hooks that are executed before media/consumer/amender selection for consumers that are selected using the API of the pipeline. Unlike amenders these hooks are not selected by consumers but by the user of the pipeline which allows certain hooks (e.g. only allow podcasts from Spotify) to be shown for certain consumers only (e.g. only the Discord status). "only allow podcasts from Spotify" doesn't make sense for scrobbling e.g.
  • Migrate the Discord status to be a pipeline consumer
  • Migrate all "amenders" (components that add media metadata using external sources, e.g. the TIDAL API)
  • Migrate all filters/transformers to the pipeline (e.g. Spotify ads filters, Apple Music fixes, etc.)
  • ...

Further todos:

  • Refactor system media detection to use the universal comparison algorithm to replace the previous copy-pasted and redundant comparison code (which is identical on all platforms) - not mandatory for things to work, but reduces redundancy and can improve performance slightly by detecting changes more robustly and with less false-positives
  • Allow cancellation of pipeline runs, which is desirable when a new media update is coming in and a previous pipeline run is still active
  • ...
@ungive ungive added the milestone In progress milestone label Nov 15, 2024
@ungive ungive added this to the Music Presence 2.2.8 milestone Nov 24, 2024
@ungive ungive added the high priority This is important going forward label Dec 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
high priority This is important going forward milestone In progress milestone
Projects
Status: In Progress
Development

No branches or pull requests

1 participant