Merge missytake (#5)

* xtream1101#41 Clean up alignment of progress percentage.

* Use a temporary file to prevent cache file corruption when interrupted.

* Add timeouts/retries

Add timeouts to prevent downloads from timing out in minutes.
Add retries to all downloads (the only way I was able to
download my entire library).

* Suppress "No url found" warning for 0 byte entries

This commit suppresses the "No url found" warning for entries
which contain a "human_size" field set to "0 bytes".

When downloading a product, occasionally an entry in the
"download_struct" won't have a viable URL to download.
In every case I observed, this was harmless; the entry
was extra information, like a link to a web page where
you could download the soundtrack for free ("World Of Goo"),
or a link to a help page on which APK to choose ("Costume Quest"),
or some sort of... Javascript soundtrack player? ("Dustforce").

Also, in every case I observed, the entry in the "download_struct"
contained a "human_size" field set to the exact string "0 bytes".
This seems like a hint in the JSON blob saying: yeah, this entry
doesn't have anything for you to download, you can ignore it.

It's a little worrying to the casual user of the tool when it prints
these error messages.  But it seems like it's harmless.  So let's
suppress these error messages when "human_size" is "0 bytes".

* Content-type argument. Possible to download bittorrent files.

* Fixed Changes that got wiped
- don't cancel a rebase.

* multi-processing.
- lambda to be pedantic about the type map expects.
- added exception package with InvalidCookieException.py
- added multiprocess package with exorcise_daemons.py
- exorcise_daemons#ExorcistPool creates multiprocessing pools without creating daemons
- mapped self._process_order_id to multiprocess pool of purchase_keys.

* multiprocessing
- Downloading: basename now prints to avoid so much console spam.
- join the pool to finalize it properly.

* removed unnecessary import to base __init__.py that I pushed accidentally.

* Process Safe File Writing
- created CacheData class in cache.py
- read cache data in for every write.

* _get_trove_products
- updated trove_base_url

* Caching Update: (#4)

- caching with json is pretty crazy, so I've switched it to a csv.
- added _strtobool
- added _strtonone will be useful when converting old json cache
- cache object inheriting list.
- file operations moved to file_ops.py
- readability changes... sorry
- changes for consistency across trove and non-trove cache

* Fixed Changes that got wiped
- don't cancel a rebase.

* Rebased on missytake/main
- multi-processing
- file safety
- no tmp cache anymore
- cache is a visible file.
- intellij .idea to .gitignore

* Rebased on missytake/main
- multi-processing
- file safety
- no tmp cache anymore
- cache is a visible file.
- intellij .idea to .gitignore

* changed uploaded_at to string again

---------

Co-authored-by: Thomas Aglassinger <[email protected]>
Co-authored-by: yakovlevtx <[email protected]>
Co-authored-by: Robert Xu <[email protected]>
Co-authored-by: Larry Hastings <[email protected]>
Co-authored-by: Valtteri-Jokinen <[email protected]>
Co-authored-by: missytake <[email protected]>

.gitignore

@@ -2,3 +2,6 @@
 __pycache__
 *cookies.txt
 .tox/
+build/
+dist/
+.idea

humblebundle_downloader/cli.py

@@ -50,6 +50,13 @@ def parse_args(args):
         help=("Only get content in a platform. Values can be seen in your "
               "humble bundle's library dropdown. Ex: -p ebook video"),
     )
+    parser.add_argument(
+        '--content-types',
+        type=str, nargs='*',
+        help="Whether to download only direct content (web) and/or the bittorrent file. Default is only direct content. "
+        "Available variables: 'web', 'bittorrent' "
+        "Ex: --content-types web bittorrent ",
+    )
     parser.add_argument(
         '--progress',
         action='store_true',
@@ -92,6 +99,7 @@ def cli():
         purchase_keys=cli_args.keys,
         trove=cli_args.trove,
         update=cli_args.update,
+        content_types=cli_args.content_types
     ).start()
 
 

humblebundle_downloader/data/cache.py

@@ -32,14 +32,6 @@ def _strtonone(val):
     else:
         raise ValueError("value was none 'None' %r" % (val,))
 
-class CacheDataJson:
-    key: str
-    value: dict
-
-    def __init__(self, key: str, value: dict):
-        self.key = key
-        self.value = value
-
 
 class CsvCacheData: