Skip to content

Commit

Permalink
🎁 Bug Fix v2.1.1 (thanks to @nemuruibai)
Browse files Browse the repository at this point in the history
Release v2.1.1
- Partial Windows support is back

Changes:
- wget disabled on windows currently, until it fixed

Fixes:
- gdown & curl bug fixed
- utf-8 as default encoding for queue checker (fix bug in Windows)
  • Loading branch information
etherealxx committed Mar 8, 2023
1 parent 58b4d0c commit 6f8c65f
Show file tree
Hide file tree
Showing 4 changed files with 53 additions and 19 deletions.
20 changes: 16 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -244,6 +244,17 @@ On Windows, this extension will install [MEGAcmd](https://github.com/meganz/MEGA
MacOS is not supported.

## Latest release: v2.1.0
#### Release v2.1.1
- Partial Windows support is back

Changes:
- wget disabled on windows currently, until it fixed

Fixes:
- gdown & curl bug fixed
- utf-8 as default encoding for queue checker (fix bug in Windows)

### Release v2.1.0
Features:
- Supports renaming downloaded file with `>` (for example: `https://files.catbox.moe/uarze8.safetensors > neurosama.safetensors`)
- Supports extension usage without `--gradio-queue` (ported from [onedotsix](https://github.com/etherealxx/batchlinks-webui/tree/onedotsix))
Expand Down Expand Up @@ -283,13 +294,14 @@ When `globaldebug = True`, it will give an output for how long a single download
## Roadmap

- [ ] Add checker for downloaded models (so that it won't download again after the model is downloaded)
- [ ] Logo change
- [ ] Different UI for mobile
- [ ] Gradio progress bar
- [ ] Logo change
- [ ] Moving most of the content of this Readme.md to Wiki instead
- [ ] Other download sites (s-ul.eu, gitgud, Google Drive)
- [ ] Gradio progress bar
- [ ] Support customizable hashtag from the UI
- [ ] UI overhaul
- [ ] (Windows) wget & aria2 support
<br/>_
- [x] aria2 for huggingface download method
- [x] Cleaning the code from unnecesarry comments
Expand All @@ -300,12 +312,12 @@ When `globaldebug = True`, it will give an output for how long a single download
## Known Bugs

- Progress bar (the yellow bar) doesn't progress as expected
- Sometimes notification sound doesn't play when downloading same file twice in a row
- Sometimes colab cannot be shut down with a single click on the stop button. Hitting the button several times will raise a KeyboardInterrupt and forcely stopping the cell.
- Sometimes notification sound doesn't play when downloading same file twice in a row
- Sometimes notification sound shows up when starting download, instead of when the download process is completed.
- There's still a chance that the UI of non `--gradio-queue` session and/or onedotsix freezes after a download session
- Windows: The delay between file is downloaded and the output shows is pretty long, and even sometimes the notification comes at the wrong time.
<br/>_
- Sometimes the notification bell shows up when starting download, instead of when the download process is completed.
- ~~Links that has bracket in it needs to be 'escaped' (For example, `Baka-DiffusionV1(Fp16).safetensors` must be typed `Baka-DiffusionV1\(Fp16\).safetensors`)~~ Fixed in [v2.0.0](dbb2adb3d07e41654244076b8ef4e851c3bb1f0c)
- ~~The delay between file is downloaded and the output shows is really long (1min+) on [camenduru's v1.6 colab](https://github.com/camenduru/stable-diffusion-webui-colab) (Gradio related?)~~ Seems like fixed in [v1.1.0](fe6feafc07fbbe3efd2883b33855f8d66b5f89ea)
- ~~File downloaded from MEGA will not listed on the output, as it use different download method. There is some delay between the transfare() function complete until it writes the file. I don't know how long the delay is.~~ Fixed in [v1.1.0](fe6feafc07fbbe3efd2883b33855f8d66b5f89ea)
Expand Down
6 changes: 3 additions & 3 deletions scripts/a_gradiocheck.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
batchlinks_dir = os.path.join(script_dir, "batchlinks-downloader.py")

# remove the progress bar if queue off
with open(batchlinks_dir, 'r') as f:
with open(batchlinks_dir, 'r', encoding='utf-8') as f:
contents = f.read()

if cmd_opts.gradio_queue:
Expand All @@ -14,7 +14,7 @@
new_contents = re.sub(r'^(\s*)#progress\(', r'\1progress(', contents, flags=re.MULTILINE)
contents = re.sub(r'choosedowner\):', 'choosedowner, progress=gr.Progress()):', new_contents, flags=re.MULTILINE)

with open(batchlinks_dir, 'w') as f:
with open(batchlinks_dir, 'w', encoding='utf-8') as f:
f.write(contents)

else:
Expand All @@ -23,5 +23,5 @@
new_contents = re.sub(r'^(\s*)progress\(', r'\1#progress(', contents, flags=re.MULTILINE)
contents = re.sub(r'choosedowner,\s*progress=gr\.Progress\(\)\):$', 'choosedowner):', new_contents, flags=re.MULTILINE)

with open(batchlinks_dir, 'w') as f:
with open(batchlinks_dir, 'w', encoding='utf-8') as f:
f.write(contents)
34 changes: 23 additions & 11 deletions scripts/batchlinks-downloader.py
Original file line number Diff line number Diff line change
Expand Up @@ -563,13 +563,15 @@ def hfdown(todownload, folder, downloader, mode='default', torename=''):
if mode=='civit' or mode=='civitdebugevery':
filename = pathlib.Path(folder).name
filename_s = shlex.quote(filename)
filepath = shlex.quote(folder)
filepath = folder
filepath_s = shlex.quote(folder)
todownload_s = todownload
folder_s = pathlib.Path(folder).parent.resolve()
else:
filename = todownload.rsplit('/', 1)[-1]
filename_s = shlex.quote(filename)
filepath = shlex.quote(os.path.join(folder, filename))
filepath = os.path.join(folder, filename)
filepath_s = shlex.quote(filepath)
todownload_s = shlex.quote(todownload)
folder_s = shlex.quote(folder)
#savestate_folder(folder_s)
Expand All @@ -582,15 +584,15 @@ def hfdown(todownload, folder, downloader, mode='default', torename=''):
import wget
wget.download(todownload, filepath)
elif downloader=='curl':
runwithsubprocess(f"curl -Lo {filepath} {todownload_s}")
os.system("curl -Lo " + filepath + " " + todownload_s)
else:
if downloader=='gdown':
printdebug(f"debug gdown {todownload_s} -O {filepath}")
runwithsubprocess(f"gdown {todownload_s} -O {filepath}", folder_s)
printdebug(f"debug gdown {todownload_s} -O {filepath_s}")
runwithsubprocess(f"gdown {todownload_s} -O {filepath_s}", folder_s)
elif downloader=='wget':
runwithsubprocess(f"wget -O {filepath} {todownload_s} --progress=bar:force", folder_s)
runwithsubprocess(f"wget -O {filepath_s} {todownload_s} ", folder_s)
elif downloader=='curl':
runwithsubprocess(f"curl -Lo {filepath} {todownload_s}", folder_s)
runwithsubprocess(f"curl -Lo {filepath_s} {todownload_s}", folder_s)
# curdir = os.getcwd()
# os.rename(os.path.join(curdir, filename), filepath)
elif downloader=='aria2':
Expand Down Expand Up @@ -621,7 +623,7 @@ def hfdown(todownload, folder, downloader, mode='default', torename=''):
time.sleep(2)
printdebug("debug filename: " + str(filename))
printdebug("debug filename_s: " + str(filename_s))
printdebug("debug filepath: " + str(filepath))
printdebug("debug filepath: " + str(filepath_s))
printdebug("debug todownload_s: " + str(todownload_s))
printdebug("debug folder_s: " + str(folder_s))
try:
Expand Down Expand Up @@ -964,6 +966,10 @@ def run(command, choosedowner, progress=gr.Progress()):
os.kill(tokill, signal.SIGTERM)
except ProcessLookupError:
pass
except PermissionError:
pass
except OSError:
pass
print()
print('BatchLinks Downloads finished!')
print('')
Expand Down Expand Up @@ -1053,8 +1059,14 @@ def cancelrun():
global prockilled
printdebug("debug processid: " + str(processid))
if not processid == '':

os.kill(processid, signal.SIGTERM)
try:
os.kill(processid, signal.SIGTERM)
except ProcessLookupError:
pass
except PermissionError:
pass
except OSError:
pass
#os.killpg(os.getpgid(processid.pid), signal.SIGTERM)
prockilled = True
if prockilled == True and globaldebug == True:
Expand Down Expand Up @@ -1122,7 +1134,7 @@ def on_ui_tabs():
out_text = gr.Textbox("(If this text disappear, that means a download session is in progress.)", label="Output")

if platform.system() == "Windows":
choose_downloader = gr.Radio(["gdown", "wget", "curl"], value="gdown", label="Download method")
choose_downloader = gr.Radio(["gdown", "curl"], value="gdown", label="Download method")
else:
if gradiostate == True:
choose_downloader = gr.Radio(["gdown", "wget", "curl", "aria2"], value="gdown", label="Download method")
Expand Down
12 changes: 11 additions & 1 deletion version.txt
Original file line number Diff line number Diff line change
@@ -1,8 +1,18 @@
v2.1.0
v2.1.1
(^current version^)

Release notes:

#### Release v2.1.1
- Partial Windows support is back

Changes:
- wget disabled on windows currently, until it fixed

Fixes:
- gdown & curl bug fixed
- utf-8 as default encoding for queue checker (fix bug in Windows)

## v2.1.0
Features:
- Supports renaming downloaded file with `>` (for example: `https://files.catbox.moe/uarze8.safetensors > neurosama.safetensors`)
Expand Down

0 comments on commit 6f8c65f

Please sign in to comment.