Skip to content
This repository was archived by the owner on Mar 28, 2024. It is now read-only.

iPlayer Doctor Who (1963-1996) all one season? #38

Open
JohnVeness opened this issue Dec 23, 2023 · 8 comments
Open

iPlayer Doctor Who (1963-1996) all one season? #38

JohnVeness opened this issue Dec 23, 2023 · 8 comments
Labels
bug Something isn't working

Comments

@JohnVeness
Copy link

This might just be a problem on the BBC site that you can't fix, but for info:

python3 freevine.py get --titles https://www.bbc.co.uk/iplayer/episodes/p0ggwr8l

suggests everything is lumped into one season.


18:12:03.619 INFO : BBC iPlayer
18:12:05.796 INFO : Doctor Who (1963–1996): 1 Season(s), 651 Episode(s)

18:12:05.796 INFO : Doctor Who (1963–1996) The Daleks in Colour
18:12:05.796 INFO : Doctor Who (1963–1996) The Daleks: The Dead Planet
18:12:05.796 INFO : Doctor Who (1963–1996) The Daleks: The Survivors
18:12:05.796 INFO : Doctor Who (1963–1996) The Daleks: The Escape
18:12:05.797 INFO : Doctor Who (1963–1996) The Daleks: The Ambush
18:12:05.797 INFO : Doctor Who (1963–1996) The Daleks: The Expedition
18:12:05.797 INFO : Doctor Who (1963–1996) The Daleks: The Ordeal
18:12:05.797 INFO : Doctor Who (1963–1996) The Daleks: The Rescue
18:12:05.797 INFO : Doctor Who (1963–1996) The Edge of Destruction: The Edge of Destruction
18:12:05.797 INFO : Doctor Who (1963–1996) The Edge of Destruction: The Brink of Disaster
18:12:05.797 INFO : Doctor Who (1963–1996) The Keys of Marinus: The Sea of Death
18:12:05.797 INFO : Doctor Who (1963–1996) The Keys of Marinus: The Velvet Web
18:12:05.797 INFO : Doctor Who (1963–1996) The Keys of Marinus: The Screaming Jungle
18:12:05.798 INFO : Doctor Who (1963–1996) The Keys of Marinus: The Snows of Terror
18:12:05.798 INFO : Doctor Who (1963–1996) The Keys of Marinus: Sentence of Death
18:12:05.798 INFO : Doctor Who (1963–1996) The Keys of Marinus: The Keys of Marinus
18:12:05.798 INFO : Doctor Who (1963–1996) The Aztecs: The Temple of Evil
18:12:05.798 INFO : Doctor Who (1963–1996) The Aztecs: The Warriors of Death
18:12:05.798 INFO : Doctor Who (1963–1996) The Aztecs: The Bride of Sacrifice
18:12:05.798 INFO : Doctor Who (1963–1996) The Aztecs: The Day of Darkness
18:12:05.798 INFO : Doctor Who (1963–1996) The Sensorites: Strangers in Space
18:12:05.798 INFO : Doctor Who (1963–1996) The Sensorites: The Unwilling Warriors
18:12:05.798 INFO : Doctor Who (1963–1996) The Sensorites: Hidden Danger
18:12:05.799 INFO : Doctor Who (1963–1996) The Sensorites: A Race Against Death
18:12:05.799 INFO : Doctor Who (1963–1996) The Sensorites: Kidnap
18:12:05.799 INFO : Doctor Who (1963–1996) The Sensorites: A Desperate Venture
18:12:05.799 INFO : Doctor Who (1963–1996) The Reign of Terror: A Land of Fear
18:12:05.799 INFO : Doctor Who (1963–1996) The Reign of Terror:  Guests of Madame Guillotine
18:12:05.799 INFO : Doctor Who (1963–1996) The Reign of Terror: A Change of Identity
18:12:05.799 INFO : Doctor Who (1963–1996) The Reign of Terror: The Tyrant of France (Animated)
18:12:05.800 INFO : Doctor Who (1963–1996) The Reign of Terror: A Bargain of Necessity (Animated)
18:12:05.800 INFO : Doctor Who (1963–1996) The Reign of Terror: Prisoners of Conciergerie
18:12:05.800 INFO : Doctor Who (1963–1996) Planet of Giants: Planet of Giants
18:12:05.800 INFO : Doctor Who (1963–1996) Planet of Giants: Dangerous Journey
18:12:05.800 INFO : Doctor Who (1963–1996) Planet of Giants: Crisis
18:12:05.800 INFO : Doctor Who (1963–1996) The Dalek Invasion of Earth: World's End
18:12:05.800 INFO : Doctor Who (1963–1996) The Dalek Invasion of Earth: The Daleks
18:12:05.800 INFO : Doctor Who (1963–1996) The Dalek Invasion of Earth: Day of Reckoning
18:12:05.800 INFO : Doctor Who (1963–1996) The Dalek Invasion of Earth: The End of Tomorrow
18:12:05.800 INFO : Doctor Who (1963–1996) The Dalek Invasion of Earth: The Waking Ally
18:12:05.800 INFO : Doctor Who (1963–1996) The Dalek Invasion of Earth: Flashpoint
18:12:05.801 INFO : Doctor Who (1963–1996) The Rescue: The Powerful Enemy
18:12:05.801 INFO : Doctor Who (1963–1996) The Rescue: Desperate Measures
18:12:05.801 INFO : Doctor Who (1963–1996) The Romans: The Slave Traders
18:12:05.801 INFO : Doctor Who (1963–1996) The Romans: All Roads Lead to Rome
18:12:05.801 INFO : Doctor Who (1963–1996) The Romans: Conspiracy
18:12:05.801 INFO : Doctor Who (1963–1996) The Romans: Inferno
18:12:05.801 INFO : Doctor Who (1963–1996) The Web Planet: The Web Planet
18:12:05.801 INFO : Doctor Who (1963–1996) The Web Planet: The Zarbi
18:12:05.801 INFO : Doctor Who (1963–1996) The Web Planet: Escape to Danger
18:12:05.801 INFO : Doctor Who (1963–1996) The Web Planet: Crater of Needles
18:12:05.802 INFO : Doctor Who (1963–1996) The Web Planet: Invasion
18:12:05.802 INFO : Doctor Who (1963–1996) The Web Planet: The Centre
18:12:05.802 INFO : Doctor Who (1963–1996) The Crusade: The Lion
18:12:05.802 INFO : Doctor Who (1963–1996) The Crusade: The Wheel of Fortune
18:12:05.802 INFO : Doctor Who (1963–1996) The Space Museum: The Space Museum
18:12:05.802 INFO : Doctor Who (1963–1996) The Space Museum: The Dimensions of Time
18:12:05.802 INFO : Doctor Who (1963–1996) The Space Museum: The Search
18:12:05.802 INFO : Doctor Who (1963–1996) The Space Museum: The Final Phase
18:12:05.802 INFO : Doctor Who (1963–1996) The Chase: The Executioners
18:12:05.802 INFO : Doctor Who (1963–1996) The Chase: The Death of Time
18:12:05.802 INFO : Doctor Who (1963–1996) The Chase: Flight through Eternity
18:12:05.803 INFO : Doctor Who (1963–1996) The Chase: Journey into Terror
18:12:05.803 INFO : Doctor Who (1963–1996) The Chase: The Death of Dr Who
18:12:05.803 INFO : Doctor Who (1963–1996) The Chase: The Planet of Decision
18:12:05.803 INFO : Doctor Who (1963–1996) The Time Meddler: The Watcher
18:12:05.803 INFO : Doctor Who (1963–1996) The Time Meddler: The Meddling Monk
18:12:05.803 INFO : Doctor Who (1963–1996) The Time Meddler: A Battle of Wits
18:12:05.803 INFO : Doctor Who (1963–1996) The Time Meddler: Checkmate
18:12:05.803 INFO : Doctor Who (1963–1996) Galaxy 4: Four Hundred Dawns (Animated)
18:12:05.803 INFO : Doctor Who (1963–1996) Galaxy 4: Trap of Steel (Animated)
18:12:05.803 INFO : Doctor Who (1963–1996) Galaxy 4: Air Lock (Animated)
18:12:05.803 INFO : Doctor Who (1963–1996) Galaxy 4: The Exploding Planet (Animated)
18:12:05.803 INFO : Doctor Who (1963–1996) The Daleks' Master Plan: Day of Armageddon
18:12:05.804 INFO : Doctor Who (1963–1996) The Daleks' Master Plan: Counter Plot
18:12:05.804 INFO : Doctor Who (1963–1996) The Daleks' Master Plan: Escape Switch
18:12:05.804 INFO : Doctor Who (1963–1996) The Ark: The Steel Sky
18:12:05.804 INFO : Doctor Who (1963–1996) The Ark: The Plague
18:12:05.804 INFO : Doctor Who (1963–1996) The Ark: The Return
18:12:05.804 INFO : Doctor Who (1963–1996) The Ark: The Bomb

etc.

If I download an episode, it will have "Season.xx" as part of the filename, but no "SxxEyy".
@JohnVeness JohnVeness added the bug Something isn't working label Dec 23, 2023
@stabbedbybrick
Copy link
Owner

BBC is wildly inconsistent with how they label their shows, so it's always a work in progress. This show doesn't have any episode numbers, only names, but I've added a fix to main that adds their season numbering: #39

@JohnVeness
Copy link
Author

Thank you, I've tested that and that seems to work :)

@JohnVeness
Copy link
Author

I don't know how it's getting the data but I notice that get_iplayer is able to obtain episode numbers for at least some episodes that freevine isn't, if that's something you would like to investigate. (There are also plenty of problems where there are multiple episode 1s for season 4 etc., but at least some seasons look a bit better).

I've attached the results of doing get_iplayer https://www.bbc.co.uk/iplayer/episodes/p0ggwr8l --pid-recursive --get --metadata=json --metadata-only --future --refresh-future --refresh --file-prefix "<name> - <senum> <seriesnum> <episodenum>. <episodeshort>"
Doctor_Who_1963-1996_.zip

@stabbedbybrick
Copy link
Owner

get_iplayer isn't actually getting the data in this case, it's appending the SXXEXX to the episodes as they are listed on iPlayer site. For example, in the data you posted, the TV movie is listed as S28E01, which isn't an actual season. It's a decent solution, though, because iPlayer API is a huge mess behind the scenes and adding ordered numbers is sometimes the only way. I used to have it the same way a few versions back, but it caused issues with other shows so I lost patience and removed it.

Keep in mind that you can also download episodes by name:
freevine.py get -e "The Faceless Ones: Episode 1 (Animated)" https://www.bbc.co.uk/iplayer/episodes/p0ggwr8l

But yeah, it's definitely a work in progress.

@JohnVeness
Copy link
Author

Thanks. Yeah, I can imagine it must be really frustrating dealing with these messy services.

@sleaux-meaux
Copy link

sleaux-meaux commented Dec 25, 2023 •
edited
Loading

get_iplayer isn't actually getting the data in this case

Not true, but get_iplayer's approach won't help you. Unless the programme titles say otherwise, it uses whatever season numbers it finds in the BBC metadata, which don't necessarily agree with series labels or external sources. But allowing programme titles to override metadata is what causes the problematic episode numbering in season 4. Some a priori knowledge is required if you want to fix up the numbering post-download.

@stabbedbybrick
Copy link
Owner

stabbedbybrick commented Dec 27, 2023 •
edited
Loading

I've just added a new option, -fn/--force-numbering, to basically force episode numbering. It's not very sophisticated, but it works pretty well for these outliers where you just need clear labels in order to download properly.

python freevine.py get -t -fn https://www.bbc.co.uk/iplayer/episodes/p0ggwr8l


14:51:42.571 INFO : BBC iPlayer
14:51:45.604 INFO : Doctor Who (1963–1996): 27 Season(s), 651 Episode(s)

14:51:45.604 INFO : Doctor Who (1963–1996) E01 Special
14:51:45.604 INFO : Doctor Who (1963–1996) E02 Doctor Who
14:51:45.607 INFO : Doctor Who (1963–1996) E03 Galaxy 4: Air Lock
14:51:45.607 INFO : Doctor Who (1963–1996) E04 The Faceless Ones: Episode 1
14:51:45.607 INFO : Doctor Who (1963–1996) E05 The Faceless Ones: Episode 3
14:51:45.607 INFO : Doctor Who (1963–1996) E06 The Evil of the Daleks: Episode 2
14:51:45.607 INFO : Doctor Who (1963–1996) S01E01 The Daleks in Colour
14:51:45.607 INFO : Doctor Who (1963–1996) S01E02 The Daleks: The Dead Planet
14:51:45.607 INFO : Doctor Who (1963–1996) S01E03 The Daleks: The Survivors
14:51:45.607 INFO : Doctor Who (1963–1996) S01E04 The Daleks: The Escape
14:51:45.610 INFO : Doctor Who (1963–1996) S01E05 The Daleks: The Ambush
14:51:45.610 INFO : Doctor Who (1963–1996) S01E06 The Daleks: The Expedition
14:51:45.610 INFO : Doctor Who (1963–1996) S01E07 The Daleks: The Ordeal
14:51:45.610 INFO : Doctor Who (1963–1996) S01E08 The Daleks: The Rescue
14:51:45.610 INFO : Doctor Who (1963–1996) S01E09 The Edge of Destruction: The Edge of Destruction
14:51:45.610 INFO : Doctor Who (1963–1996) S01E10 The Edge of Destruction: The Brink of Disaster
14:51:45.610 INFO : Doctor Who (1963–1996) S01E11 The Keys of Marinus: The Sea of Death
14:51:45.610 INFO : Doctor Who (1963–1996) S01E12 The Keys of Marinus: The Velvet Web
14:51:45.610 INFO : Doctor Who (1963–1996) S01E13 The Keys of Marinus: The Screaming Jungle
14:51:45.610 INFO : Doctor Who (1963–1996) S01E14 The Keys of Marinus: The Snows of Terror
14:51:45.610 INFO : Doctor Who (1963–1996) S01E15 The Keys of Marinus: Sentence of Death
14:51:45.610 INFO : Doctor Who (1963–1996) S01E16 The Keys of Marinus: The Keys of Marinus
14:51:45.610 INFO : Doctor Who (1963–1996) S01E17 The Aztecs: The Temple of Evil
14:51:45.610 INFO : Doctor Who (1963–1996) S01E18 The Aztecs: The Warriors of Death
14:51:45.610 INFO : Doctor Who (1963–1996) S01E19 The Aztecs: The Bride of Sacrifice
14:51:45.610 INFO : Doctor Who (1963–1996) S01E20 The Aztecs: The Day of Darkness

@JohnVeness
Copy link
Author

JohnVeness commented Dec 27, 2023 •
edited
Loading

Many thanks, that looks good, and is probably as good as we can get for now :). Short of interfacing with TMDB or something, anyway.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@JohnVeness @sleaux-meaux @stabbedbybrick