Add browsing scanning feature through mitmproxy

.github/workflows/build-release.yaml

@@ -1,6 +1,11 @@
 name: Build artifacts to link to release
 
 on:
+  workflow_dispatch:
+    inputs:
+      version:
+        description: 'Version number'
+        required: true
   push:
     tags:
       - '*'
@@ -9,7 +14,7 @@ jobs:
   build-linux:
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v2
+      - uses: actions/checkout@v4
       - uses: actions/setup-node@v2
         with:
           node-version: '20'
@@ -28,7 +33,7 @@ jobs:
   build-windows:
     runs-on: windows-latest
     steps:
-      - uses: actions/checkout@v2
+      - uses: actions/checkout@v4
       - uses: actions/setup-node@v2
         with:
           node-version: '20'
@@ -43,7 +48,7 @@ jobs:
         env:
           GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
   build-mac:
-    runs-on: macos-11
+    runs-on: macos-13
     steps:
       - uses: actions/checkout@v2
       - uses: actions/setup-node@v2

.github/workflows/update-winget.yaml

@@ -21,4 +21,4 @@ jobs:
         shell: cmd
         run: |
           curl https://aka.ms/wingetcreate/latest -L --output wingetcreate.exe
-          wingetcreate.exe update StefanLobbenmeier.youtube-dl-gui -s true -u https://github.com/StefanLobbenmeier/youtube-dl-gui/releases/download/v%APP_VERSION%/Open-Video-Downloader-Setup-%APP_VERSION%.exe -v %APP_VERSION%  -t %WINGET_TOKEN%
+          wingetcreate.exe update mp3butcher.youtube-dl-gui -s true -u https://github.com/mp3butcher/youtube-dl-gui/releases/download/v%APP_VERSION%/Open-Video-Downloader-Setup-%APP_VERSION%.exe -v %APP_VERSION%  -t %WINGET_TOKEN%

README.md

@@ -1,16 +1,22 @@
 <img src="https://raw.githubusercontent.com/StefanLobbenmeier/youtube-dl-gui/v2.0.0/renderer/img/icon.png" alt="logo" align="left" height="100"/>
 
 # Open Video Downloader (youtube-dl-gui)
-[![version badge](https://img.shields.io/github/v/release/StefanLobbenmeier/youtube-dl-gui?label=latest-release)](https://github.com/StefanLobbenmeier/youtube-dl-gui/releases/latest)
+[![version badge](https://img.shields.io/github/v/release/mp3butcher/youtube-dl-gui?label=latest-release)](https://github.com/mp3butcher/youtube-dl-gui/releases/latest)
 [![license](https://img.shields.io/github/license/StefanLobbenmeier/youtube-dl-gui)](https://github.com/StefanLobbenmeier/youtube-dl-gui/blob/master/LICENSE)
-[![coverage badge](https://img.shields.io/codecov/c/github/StefanLobbenmeier/youtube-dl-gui)](https://app.codecov.io/gh/StefanLobbenmeier/youtube-dl-gui)
-[![downloads](https://img.shields.io/github/downloads/StefanLobbenmeier/youtube-dl-gui/total)](https://github.com/StefanLobbenmeier/youtube-dl-gui/releases/latest)
-[![CI badge](https://img.shields.io/github/actions/workflow/status/StefanLobbenmeier/youtube-dl-gui/continuous-integration.yaml)](https://github.com/StefanLobbenmeier/youtube-dl-gui/actions)
+[![coverage badge](https://img.shields.io/codecov/c/github/mp3butcher/youtube-dl-gui)](https://app.codecov.io/gh/mp3butcher/youtube-dl-gui)
+[![downloads](https://img.shields.io/github/downloads/mp3butcher/youtube-dl-gui/total)](https://github.com/mp3butcher/youtube-dl-gui/releases/latest)
+[![CI badge](https://img.shields.io/github/actions/workflow/status/mp3butcher/youtube-dl-gui/continuous-integration.yaml)](https://github.com/mp3butcher/youtube-dl-gui/actions)
 
-https://github.com/StefanLobbenmeier/youtube-dl-gui
 
 A cross-platform GUI for youtube-dl made in Electron and node.js
 
+based on https://github.com/StefanLobbenmeier/youtube-dl-gui but adding a traffic scanning feature (using mitmproxy)
+
+- I] When scanner activated the first time don't forget to install mitmproxy authority certificate (cf image) 
+![Screenshot from 2024-10-07 16-37-04](https://github.com/user-attachments/assets/3fb3bb86-d5a8-44d8-ab8e-05429bd8fb4c)
+- II] I recommend to use browser proxy switcher extension such as [this extension](https://chromewebstore.google.com/detail/proxy-switcher/iejkjpdckomcjdhmkemlfdapjodcpgih?hl=fr) to easily enable/disable proxy usage while browsing (set it up proxy=localhost and port=15930)
+![Screenshot from 2024-10-07 16-43-32](https://github.com/user-attachments/assets/a4650d10-a2f0-4059-b72c-9a241e550338)
+- III] Browse to a streaming content and let the scanner capture the video
 
 ### Features:
 - Download from all kind of platforms: YouTube, vimeo, twitter & many more
@@ -20,25 +26,32 @@ A cross-platform GUI for youtube-dl made in Electron and node.js
 - Multithreaded, up to 32 videos can be downloaded synchronously
 - Shows how much size the download will use up on your system
 - The app automatically keeps ytdl up-to-date
+- The app include a network scanner through a binding with mitmproxy 
 
 Be sure to check out [a demo gif of the application](#Demo-gif)!
 
 ## How to use
-1. Download the [applicable installer or executable](https://github.com/StefanLobbenmeier/youtube-dl-gui/releases/latest) for your system.
+1. Download the [applicable installer or executable](https://github.com/mp3butcher/youtube-dl-gui/releases/latest) for your system.
 2. If you are on windows, make sure that the [Microsoft Visual C++ 2010 Redistributable Package (x86)](https://download.microsoft.com/download/1/6/5/165255E7-1014-4D0A-B094-B6A430A6BFFC/vcredist_x86.exe) is installed. 
 3. Paste a link into the box up top.
 4. Wait for the app to gather all required metadata.
 5. Press download, and the video(s) will get downloaded to your downloads folder.
 
 Want to know more about the features this app offers? Head over to the [wiki](https://github.com/StefanLobbenmeier/youtube-dl-gui/wiki/).
 
+## Supporting the project
+I appreciate any help. That may include:
+- testing and raising issues if you encounter any
+- contributing via a pull request
+- donating to the project ([ko-fi](https://ko-fi.com/stefanlobbenmeier))
+
 ## Something is not working!
 Please see if the answer is in the [frequently answered questions](https://github.com/StefanLobbenmeier/youtube-dl-gui/wiki/FAQ) or in the [wiki](https://github.com/StefanLobbenmeier/youtube-dl-gui/wiki/).
 
 Still haven't found your answer? [Open up an issue](https://github.com/StefanLobbenmeier/youtube-dl-gui/issues), and describe the problem you're facing.
 
 ## Building from source
-First, clone the repository using `git clone https://github.com/StefanLobbenmeier/youtube-dl-gui.git`.
+First, clone the repository using `git clone https://github.com/mp3butcher/youtube-dl-gui.git`.
 
 Then navigate to the directory and install the npm modules by executing: `npm install`.
 
@@ -49,7 +62,6 @@ Please be aware that this app is only tested on windows, linux and macOS. If you
 ## Planned features
 - Select individual audio and video codecs (advanced mode)
 - List all audio qualities
-- Support for downloading livestreams
 
 Feel free to [request a new feature](https://github.com/StefanLobbenmeier/youtube-dl-gui/issues).
 

binaries/send_traffic_to_videodownloader.py

@@ -0,0 +1,100 @@
+import asyncio
+import base64
+import socket, threading
+import queue
+import time
+
+from mitmproxy import ctx
+from mitmproxy import http
+
+LISTENING_PORT = 12000
+
+# Queue to hold requests data to be sent
+data_queue = queue.Queue(maxsize=15)
+
+def sendTrafficLoop(trafficLoggerAddon):
+  data=''
+  deadguard=0
+  try:
+    while data=='':
+        try:
+          data=trafficLoggerAddon.socket_connection.recv(1)
+          print(data)
+          trafficLoggerAddon.socket_connection.close()
+          trafficLoggerAddon.socket_instance.close()
+          ctx.master.shutdown()
+        except:
+          pass
+        if data_queue.qsize()>0:
+          msg = data_queue.get()        
+          if msg is not None:
+            trafficLoggerAddon.socket_connection.send(msg.encode())
+        else:
+          deadguard=deadguard+1
+          if deadguard>10000000:
+            #Dirty connection test to exit even if OpenVideoDownloader prematurally die
+            trafficLoggerAddon.socket_connection.send('testconnection'.encode())
+            deadguard=0
+  except:
+    trafficLoggerAddon.socket_connection.close()
+    trafficLoggerAddon.socket_instance.close()
+    ctx.master.shutdown()
+  trafficLoggerAddon.socket_connection.close()
+  trafficLoggerAddon.socket_instance.close()
+  ctx.master.shutdown()
+
+class TrafficLogger:
+  def __init__(self):
+    self.socket_instance = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+    #self.socket_instance.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
+    self.socket_instance.bind(('127.0.0.1', LISTENING_PORT))
+    self.socket_instance.listen(1) #limit to one client
+    print('Proxy traffic server running on port 12000: Waiting for OpenVideoDownloader')
+    # Accept client connection    
+    self.socket_connection, self.address = self.socket_instance.accept()
+    self.socket_connection.setblocking(False)
+    print('OpenVideoDownloader connected: Ready to send traffic')
+
+    # Launch a thread to send traffic
+    thread = threading.Thread(target=sendTrafficLoop, args=([self]))
+    thread.start()
+
+
+  def request(self, flow):
+    self.checkflow(flow, False)
+
+  def response(self, flow):
+    self.checkflow(flow, True)
+    return
+
+  def checkflow(self,flow,isrep):
+    url='"'    + flow.request.url+  '"'
+    headers="["
+    for k, v in flow.request.headers.items():
+        if k!='If-Range': #TODO: Headers should be filtered in OpenVideoDownloader instead of here
+           headers=headers+'{"k":"'+k+'"'+',"v":'+'"'+v.replace('"','\\"')+'"'+'},'
+    if headers=="[":
+       headers=headers+"]"
+    else:
+       headers=headers[:-1]+"]"
+    rheaders="[]"
+    if isrep:    
+      rheaders="["
+      for k, v in flow.response.headers.items():
+        rheaders=rheaders+'{"k":"'+k+'"'+',"v":'+'"'+v.replace('"','\\"')+'"'+'},'
+      if rheaders=="[":
+         rheaders=rheaders+"]"
+      else:
+         rheaders=rheaders[:-1]+"]"
+    data=""
+    if isrep:
+      #Don't send large response    
+      if len(flow.response.text)<20000000:
+       try:
+         data=base64.b64encode(flow.response.content[:10000]).decode("ascii")
+       except:
+         data=""
+    msg='{"url":'+url+',"headers":'+headers+',"rheaders":'+rheaders+',"response":"'+data+'"}'
+    data_queue.put(msg)
+
+addons = [TrafficLogger()]