quiet errors on download, update readme

NotJoeMartinez · Sep 6, 2024 · 0df6ced · 0df6ced
1 parent 7f9f792
commit 0df6ced
Show file tree

Hide file tree

Showing 4 changed files with 36 additions and 6 deletions.
diff --git a/README.md b/README.md
@@ -8,6 +8,7 @@ It also supports semantic search via the [OpenAI embeddings API](https://beta.op
 
 - [Blog Post](https://notjoemartinez.com/blog/youtube_full_text_search/)
 - [LLM/RAG Chat Bot](#llm-chat-bot)
+- [Video Summaries](#summarize)
 - [Semantic Search](#vsearch-semantic-search)
 - [CHANGELOG](CHANGELOG.md)
 
@@ -24,8 +25,9 @@ pip install yt-fts
 ## `download`
 Download subtitles for a channel. 
 
-Takes a channel url as an argument. Specify the number of jobs to parallelize the download with the `--jobs` option. Use the `--cookies-from-browser` to use cookies from your browser in the requests, will help if you're 
-getting errors that request you to sign in.
+Takes a channel url as an argument. Specify the number of jobs to parallelize the download with the `--jobs` flag. 
+Use the `--cookies-from-browser` to use cookies from your browser in the requests, will help if you're getting errors 
+that request you to sign in. You can also run the `update` command several times to gradually get more videos into the database. 
 
 ```bash
 yt-fts download --jobs 5 "https://www.youtube.com/@3blue1brown"
@@ -124,10 +126,34 @@ to answer questions. If it can't answer your question, it has a
 mechanism to update the context by running targeted query based 
 off the conversation. The channel must have semantic search enabled.
 
-```sh
+```bash
 yt-fts llm --channel "3Blue1Brown" "How does back propagation work?"
 ```
 
+## `summarize`
+Summarizes a YouTube video transcript, providing time stamped URLS. 
+Requires a valid YouTube video URL or video ID as argument. If the 
+trancript is not in the database it will try to scrape it.
+
+```bash
+yt-fts summarize "https://www.youtube.com/watch?v=9-Jl0dxWQs8"
+# or
+yt-fts summarize "9-Jl0dxWQs8"
+```
+output:
+```
+In this video, 3Blue1Brown explores how large language models (LLMs) like GPT-3 
+might store facts within their vast...                                                         
+
+ 1 Introduction to Fact Storage in LLMs:                                                                                     
+    • The video starts by questioning how LLMs store specific facts and                                                      
+      introduces the idea that these facts might be stored in a particular part of the                                       
+      network known as multi-layer perceptrons (MLPs).                                                                       
+    • 0:00                                                                                                                   
+ 2 Overview of Transformers and MLPs:                                                                                        
+    • Provides a refresher on transformers and explains that the video will focus                                            
+```
+
 ## `vsearch` (Semantic Search)
 `vsearch` is for "Vector search". This requires that you enable semantic 
 search for a channel with `embeddings`. It has the same options as 

diff --git a/pyproject.toml b/pyproject.toml
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "yt-fts"
-version = "0.1.56"
+version = "0.1.57"
 description = "Search all of a YouTube channel from the command line"
 readme = "README.md"
 requires-python = ">=3.8"

diff --git a/yt_fts/download.py b/yt_fts/download.py
@@ -6,6 +6,7 @@
 import json
 import requests
 
+from pathlib import Path
 from concurrent.futures import ThreadPoolExecutor
 from bs4 import BeautifulSoup
 from urllib.parse import urlparse
@@ -251,8 +252,10 @@ def download_vtts(self):
             futures[i].result()
 
     def quiet_progress_hook(self, d):
+        console = self.console
         if d['status'] == 'finished':
-            print(f" -> {d['filename']}")
+            file_name = Path(d['filename']).name
+            console.print(f" -> \"{file_name}\"")
 
     def get_vtt(self, tmp_dir, video_url, language):
         try:
@@ -264,6 +267,7 @@ def get_vtt(self, tmp_dir, video_url, language):
                 'skip_download': True,
                 'subtitleslangs': [language, '-live_chat'],
                 'quiet': True,
+                'no_warnings': True,
                 'progress_hooks': [self.quiet_progress_hook],
             }
 

diff --git a/yt_fts/yt_fts.py b/yt_fts/yt_fts.py
@@ -23,7 +23,7 @@
     delete_channel
 )
 
-YT_FTS_VERSION = "0.1.56"
+YT_FTS_VERSION = "0.1.57"
 console = Console()