Non-parallel GitHub downloads, one progress bar per download

[HebaruSan]

Problem

If you try to install lots of mods that you haven't previously downloaded, you'll likely get errors like this (possibly in a different format since we've changed them a bunch recently):

Failed to download "https://github.com/jrossignol/ContractPack-AnomalySurveyor/releases/download/1.6.0/ContractPack-AnomalySurveyor_1.6.0.zip" - error: The remote server returned an error: (403) Forbidden.

Cause

GitHub doesn't like it when you try to download multiple things from it at the same time, which CKAN aggressively does. GitHub sometimes responds to such attempts with 403-Forbidden statuses (the precise rules are not publicly known).

The GitHub API works similarly, except it has defined (and queryable) throttling rules and explicitly says not to download from it in parallel:

Make requests for a single user or client ID serially. Do not make requests for a single user or client ID concurrently.

Motivation

Since we're planning a pre-release anyway, why not go nuts?

Changes

Now if multiple GitHub downloads are requested, only one of them will be started, and the rest will wait. Non-GitHub downloads will still be performed in parallel as before. When the current GitHub download finishes, one of the others will be started, and so on until completion.

Now each active download has its own progress bar in GUI, so the user can tell what's happening:

Fixes #1817.
Fixes #2210. (URLs haven't been migrated to API format, but that's probably not needed.)
Probably fixes #2400.
Fixes #1085. (The description and conversation went all over the place, so it may not be possible to even define what the issue is about, but the original complaint was about 403 statuses during downloads.)

[HebaruSan]

In latest commit, we create a 3-second timer to remove completed progress bars and add new ones, so the display won't fill up with full bars, but it also won't be constantly bouncing around.

[DasSkelett]

Since the timer patch, I get this exception from time to time:

System.InvalidOperationException: Collection was modified; enumeration operation may not execute.
  at System.Collections.ArrayList+ArrayListEnumeratorSimple.MoveNext () [0x00013] in <b2b5eee6b3734e9586bbc3ab32e40bc7>:0
  at System.Windows.Forms.XplatUIX11.NextTimeout (System.Collections.ArrayList timers, System.DateTime now) [0x0003a] in <8637a6a20d8848c3b0482a80613e13f8>:0
  at System.Windows.Forms.XplatUIX11.UpdateMessageQueue (System.Windows.Forms.XEventQueue queue, System.Boolean allowIdle) [0x0009f] in <8637a6a20d8848c3b0482a80613e13f8>:0
  at System.Windows.Forms.XplatUIX11.UpdateMessageQueue (System.Windows.Forms.XEventQueue queue) [0x00000] in <8637a6a20d8848c3b0482a80613e13f8>:0
  at System.Windows.Forms.XplatUIX11.GetMessage (System.Object queue_id, System.Windows.Forms.MSG& msg, System.IntPtr handle, System.Int32 wFilterMin, System.Int32 wFilterMax) [0x0001c] in <8637a6a20d8848c3b0482a80613e13f8>:0
  at System.Windows.Forms.XplatUI.GetMessage (System.Object queue_id, System.Windows.Forms.MSG& msg, System.IntPtr hWnd, System.Int32 wFilterMin, System.Int32 wFilterMax) [0x00000] in <8637a6a20d8848c3b0482a80613e13f8>:0
  at System.Windows.Forms.Application.RunLoop (System.Boolean Modal, System.Windows.Forms.ApplicationContext context) [0x0034e] in <8637a6a20d8848c3b0482a80613e13f8>:0
  at System.Windows.Forms.Application.Run (System.Windows.Forms.ApplicationContext context) [0x00011] in <8637a6a20d8848c3b0482a80613e13f8>:0
  at System.Windows.Forms.Application.Run (System.Windows.Forms.Form mainForm) [0x00006] in <8637a6a20d8848c3b0482a80613e13f8>:0
  at CKAN.Main..ctor (System.String[] cmdlineArgs, CKAN.KSPManager mgr, System.Boolean showConsole) [0x002bc] in /home/kilian/git/CKAN/GUI/Main/Main.cs:164
  at at (wrapper remoting-invoke-with-check) CKAN.Main..ctor(string[],CKAN.KSPManager,bool)
  at CKAN.GUI.Main_ (System.String[] args, CKAN.KSPManager manager, System.Boolean showConsole) [0x00047] in /git/CKAN/GUI/Program.cs:35
  at CKAN.GUI.Main (System.String[] args) [0x00001] in /git/CKAN/GUI/Program.cs:15

Unfortunately the stack trace doesn't really help locating the cause. But I suspect it's one of the two foreach loops. Maybe I can track it down further.

[HebaruSan]

Added a ToList to the first loop to maybe avoid that enumerator problem.

[DasSkelett]

Just wanted to ask if this is intentional or not, but I like it:
Progress bars for downloads that complete during the first three seconds are not shown since the timer also waits 3 seconds for the first trigger.

But I think that's quite nice, because you probably aren't interested in the progress bars of downloads which complete that fast, only for ones that take longer.

[HebaruSan]

Progress bars for downloads that complete during the first three seconds are not shown since the timer also waits 3 seconds for the first trigger.

Partially intentional. :)

To show all the downloads in the first 3 seconds, we would have to restore the code that displays the progress bars on the fly, and the annoying flickering would be back (even if just for 3 seconds). And the first 3 seconds aren't quite special—if a download both starts and ends in any interval, it won't get a progress bar.

[DasSkelett]

Error seems to be gone, and otherwise works fine too.
Let's hope that this comforts GitHub a bit.