Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature Request] Creating list of releases on label's page #106

Open
viviancels opened this issue Oct 5, 2024 · 5 comments
Open

[Feature Request] Creating list of releases on label's page #106

viviancels opened this issue Oct 5, 2024 · 5 comments

Comments

@viviancels
Copy link

viviancels commented Oct 5, 2024

Hello. Recently, the Qobuz introduced limitation on the number of releases in the list on the label page =1000. Example:
https://www.qobuz.com/se-en/label/the-dance-division/download-streaming-albums/287307
If you copy-past the link and download it using your Swedish account, you will receive 1406 releases
For some reason, some releases did not download automatically when downloading the label, I downloaded manualy, also inserting links with "se-en" and using Swedish account :
https://www.qobuz.com/se-en/album/gonna-be-alright-gregoir-cruz/hlt9f1gly5l5a
https://www.qobuz.com/se-en/album/scooby-doo-pa-pa-don-lore-v/duiggmbjp9nrb
https://www.qobuz.com/se-en/album/memories-dj-jump/a28ys78et29ec
https://www.qobuz.com/se-en/album/secrets-kery-fay/hdpyntt15gp5a

Another example 1000 releases limitation:
https://www.qobuz.com/se-en/label/armada-music/download-streaming-albums/248166
In search shows 5416 releases on Label, in fact, there were much fewer releases in the summer

I always used to create a list of releases manually using regular expressions in source code page:
.*?.*?href="(.*?)" .*?tooltip">(.*?)(.*?).*?|.*
$3 $2 https://www.qobuz.com$1\n
But now, due to the limitation of the number of releases on label's page, this makes no sense. However, QobuzDownloaderX-MOD somehow learns about the rest of the releases that are not visible on the label's page (even if it misses some releases).

Describe the solution you'd like

Creating a list (album artist - album title (full date) link) would allow you to find out which releases are available

@DJDoubleD
Copy link
Owner

There is a difference between the main qobuz.com website, which is basically a storefront and the Qobuz streaming platforms (web app and mobile apps).
I don't know what the main site uses to fetch the max 1000 albums from a label (I'm guessing they are using the catalog/search endpoint as it seems to be limited to 1000 results as well).

In any case, QobuzDownloaderX-MOD uses the same API as the web player's label page, which is still limited to 10000 results (see #101). I can only assume the results from that API are accurate and have no way of automatically comparing them to what the main site shows.

Implementing a list feature in the application would be theoretically possible, but there is no way that winforms will be able to display thousands of results in a performant way (at least that I know of).
Rendering performance is the reason why I limit the search results to 15 items.

In any case, as there is only 1 API endpoint that's used to retrieve the albums from a label, it is fairly simple to emulate what the application does to retrieve the albums for download.
As an example, I've attached a small python script you can use to generate a json file with all (up to 10000) albums from a label. (had to zip it because github doesn't allow .py files, apparently)
The output from this list is what QobuzDownloaderX-MOD will use when you download a label.

Maybe this will help you in finding the discrepancies and get the albums that are on the main site, but not in the downloaded items.

@viviancels
Copy link
Author

@DJDoubleD thank you. The script requests app_id, authorization token and label id

@DJDoubleD
Copy link
Owner

@DJDoubleD thank you. The script requests app_id, authorization token and label id

@viviancels Yes, I made those input variables deliberate as I try not to publish the app_id on GitHub.
You can get the app_id using this method (I just tried it and still works). Normally it doesn't change, so you can hard code it in the script yourself if you want.

The authorization token is just the token for the account you use, see my wiki for instructions if you need to get it from your personal account.
The label id is in the label url that you use to download the label.
So in your example https://www.qobuz.com/se-en/label/the-dance-division/download-streaming-albums/287307, the label id is 287307.

@viviancels
Copy link
Author

viviancels commented Oct 6, 2024

@DJDoubleD thank you. It works.

  1. For those who use regular expressions to clear code and create a spreadsheet for excel, I will leave the code that I used to create a list of label releases:
    find: .*?media_count.*?name": "(.*?)",.*?title": "(.*?)",.*?url": "(.*?)",.*?release_date_download": "(.*?)",.*?
    replace: $1\t$2\t$4\thttps://www.qobuz.com/$3\n
    . matches newline (single line)
    Example: https://regex101.com/r/KFqh6J/1

For data release I choosed release_date_download but there are also release_date_original and release_date_stream, I don't know what the difference is

      "label": {
        "name": "The Dance Division",
        "id": 287307,
        "albums_count": 1537,
        "supplier_id": 17,
        "slug": "the-dance-division"

It is unclear why parameter albums_count is variable: 1568, 1537, 1557, 1406. Number 1406 for the latest release iat the bottom of the list and the actual number of label releases, which is obtained if the se-en link and the Sweden account. Although, as I wrote above, some releases that are not on the list are also available for Sweden, this is still a mystery. 1568 is the number that the Qobuz site shows if you insert the label name into the search bar, apparently this is the total number of releases available in all countries. I used 25 tokens and corresponding country codes in label's link. the result was 1422 releases. In some countries, downloads were interrupted by an error on the few same releases, something related to the api. Like this:
[ERROR] Communication problem with Qobuz API. Details saved to error log
[ERROR] Label Download Task ERROR. Details saved to error log.

Fortunately, there was no mistake with Sweden and a couple of other countries.

  1. I saw the json code and noticed that albums (releases) have two identifiers:
  1.   "url": "/se-en/album/-/m31zcug4bfd3b",
    
  2.   "id": "m31zcug4bfd3b",
    

I believe that QobuzDownloaderX-MOD uses the second identifier and adds the beginning of the address to it https://www.qobuz.com/fr-fr/album/. Perhaps it would be more accurate to use the first identifier (/se-en/album/-/m31zcug4bfd3b), due to the fact that when a release has territorial restrictions, the repository page cannot be opened if its address (link) contains the code of another country (se-en). The first release identifier contains the country code specified in the label reference. I am writing this here so as not to create a new topic.

@DJDoubleD
Copy link
Owner

I believe that QobuzDownloaderX-MOD uses the second identifier and adds the beginning of the address to it https://www.qobuz.com/fr-fr/album/. Perhaps it would be more accurate to use the first identifier (/se-en/album/-/m31zcug4bfd3b), due to the fact that when a release has territorial restrictions, the repository page cannot be opened if its address (link) contains the code of another country (se-en). The first release identifier contains the country code specified in the label reference. I am writing this here so as not to create a new topic.

Just to avoid confusion, QobuzDownloaderX-MOD does not generate links like https://www.qobuz.com/fr-fr/album/... to get the album details.
It uses the Qobuz API directly (via the API library I made) to get the album data.
Just like in the script I added to this issue, it's the user_auth_token that determines the region (region of the account) and only the albums that are available for that region will be returned and attempted to be downloaded.

There is no way to explain the discrepancies between the numbers that the main www.qobuz.com website gives and what the API returns. I can only work with what the API returns...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants