Skip to content

Commit

Permalink
Update requirements.txt and setup.py
Browse files Browse the repository at this point in the history
  • Loading branch information
bropines committed Oct 18, 2024
1 parent 48382e1 commit b8a9f21
Show file tree
Hide file tree
Showing 8 changed files with 195 additions and 78 deletions.
56 changes: 56 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -339,6 +339,62 @@ These values are used to accurately place, scale, and display the text on the im

</details>

<details>
<summary><b>Debugging and Logging</b></summary>

When using the CLI tool `lens_scan`, you can control the logging level using the `--debug` flag. There are two levels available:

- `--debug=info`: Enables logging of informational messages, which include general information about the processing steps.
- `--debug=debug`: Enables detailed debugging messages, including verbose output and the saving of the raw response from the API to a file named `response_debug.txt` in the current directory.

**Example Usage:**

- To run with informational logging:

```bash
lens_scan path/to/image.jpg all --debug=info
```

- To run with detailed debugging logging:

```bash
lens_scan path/to/image.jpg all --debug=debug
```

When using `--debug=debug`, the library will save the raw response from the API to `response_debug.txt` in the current working directory. This can be useful for deep debugging and understanding the exact response from the API.

#### Programmatic Debugging

When using the API in your Python scripts, you can control the logging level by configuring the logging module and by passing the `logging_level` parameter when instantiating the `LensAPI` class.

**Example Usage:**

```python
import logging
from chrome_lens_py import LensAPI

# Configure logging
logging.basicConfig(level=logging.DEBUG)

# Instantiate the API with the desired logging level
api = LensAPI(logging_level=logging.DEBUG)

# Process an image
result = api.get_all_data('path/to/image.jpg')
print(result)
```

The `logging_level` parameter accepts standard logging levels from the `logging` module, such as `logging.INFO`, `logging.DEBUG`, etc.

When the logging level is set to `DEBUG`, the library will output detailed debugging information and save the raw API response to `response_debug.txt` in the current directory.

#### Notes on Logging Levels

- **INFO** level: Provides general information about the process, such as when requests are sent and responses are received.
- **DEBUG** level: Provides detailed information useful for debugging, including internal state and saved responses.

</details>

## Project Structure

```plain
Expand Down
56 changes: 56 additions & 0 deletions README_RU.md
Original file line number Diff line number Diff line change
Expand Up @@ -355,6 +355,62 @@ from chrome_lens_py import LensAPI

</details>

<details>
<summary><b>Отладка и Логирование</b></summary>

При использовании инструмента командной строки `lens_scan` вы можете управлять уровнем логирования с помощью флага `--debug`. Доступны два уровня:

- `--debug=info`: Включает логирование информационных сообщений, которые содержат общую информацию о этапах обработки.
- `--debug=debug`: Включает подробные отладочные сообщения, включая детальный вывод и сохранение сырого ответа от API в файл `response_debug.txt` в текущей директории.

**Примеры использования:**

- Запуск с информационным логированием:

```bash
lens_scan path/to/image.jpg all --debug=info
```

- Запуск с подробным отладочным логированием:

```bash
lens_scan path/to/image.jpg all --debug=debug
```

При использовании `--debug=debug` библиотека сохранит сырой ответ от API в файл `response_debug.txt` в текущей рабочей директории. Это может быть полезно для глубокой отладки и понимания точного ответа от API.

#### Программная Отладка

При использовании API в ваших Python-скриптах вы можете управлять уровнем логирования, настраивая модуль `logging` и передавая параметр `logging_level` при создании экземпляра класса `LensAPI`.

**Пример использования:**

```python
import logging
from chrome_lens_py import LensAPI

# Настройка логирования
logging.basicConfig(level=logging.DEBUG)

# Создаем экземпляр API с нужным уровнем логирования
api = LensAPI(logging_level=logging.DEBUG)

# Обрабатываем изображение
result = api.get_all_data('path/to/image.jpg')
print(result)
```

Параметр `logging_level` принимает стандартные уровни логирования из модуля `logging`, такие как `logging.INFO`, `logging.DEBUG` и т.д.

Когда уровень логирования установлен на `DEBUG`, библиотека будет выводить подробную отладочную информацию и сохранять сырой ответ от API в файл `response_debug.txt` в текущей директории.

#### Примечания об Уровнях Логирования

- Уровень **INFO**: Предоставляет общую информацию о процессе, такую как отправка запросов и получение ответов.
- Уровень **DEBUG**: Предоставляет подробную информацию, полезную для отладки, включая внутреннее состояние и сохраненные ответы.

</details>

## Структура проекта

```plain text
Expand Down
3 changes: 1 addition & 2 deletions requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -4,5 +4,4 @@ filetype
lxml
json5
PySocks
httpx
logging
httpx
3 changes: 1 addition & 2 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@

setup(
name='chrome_lens_py',
version='1.1.0',
version='1.1.1',
packages=find_packages(where="src"),
package_dir={"": "src"},
install_requires=[
Expand All @@ -21,7 +21,6 @@
'rich',
'PySocks',
'httpx',
'logging',
],
entry_points={
'console_scripts': [
Expand Down
74 changes: 33 additions & 41 deletions src/chrome_lens_py/cookies_manager.py
Original file line number Diff line number Diff line change
@@ -1,40 +1,31 @@
import time
import logging # Импортируем модуль логирования
import logging
from http.cookies import SimpleCookie
import os
import pickle
from email.utils import parsedate_to_datetime
from datetime import datetime
from .exceptions import LensCookieError


class CookiesManager:
def __init__(self, config=None, cookie_file='cookies.pkl', enable_logging=False):
def __init__(self, config=None, cookie_file='cookies.pkl', logging_level=logging.WARNING):
self.cookies = {}
self.config = config or {}
self.cookie_file = os.path.join(os.path.dirname(__file__), cookie_file)
# Получаем информацию о том, нужно ли логировать
self.enable_logging = enable_logging
self.logging_level = logging_level
logging.getLogger().setLevel(self.logging_level)
logging.debug(f"Initialized CookiesManager with cookie file: {self.cookie_file}")
self.load_cookies()

if self.enable_logging:
logging.info(f"Initialized CookiesManager with cookie file: {self.cookie_file}")

def log(self, message, level=logging.INFO):
"""Вспомогательный метод для логирования, который учитывает флаг enable_logging.
Logs a message if logging is enabled.
"""
if self.enable_logging:
logging.log(level, message)

def load_cookies(self):
"""Loads cookies from a file or from config."""
if os.path.exists(self.cookie_file):
try:
with open(self.cookie_file, 'rb') as f:
self.cookies = pickle.load(f)
self.log(f"Loaded cookies from file: {self.cookie_file}")
logging.debug(f"Loaded cookies from file: {self.cookie_file}")
except (FileNotFoundError, pickle.PickleError) as e:
self.log(f"Error loading cookies from file: {e}", level=logging.WARNING)
logging.warning(f"Error loading cookies from file: {e}")

if 'headers' in self.config and 'cookie' in self.config['headers']:
cookie_data = self.config['headers']['cookie']
Expand All @@ -46,13 +37,12 @@ def load_cookies(self):
elif isinstance(cookie_data, dict):
self.parse_cookie_dict(cookie_data)
else:
self.log(f"Unexpected cookie data type in config: {type(cookie_data)}",
level=logging.WARNING)
logging.warning(f"Unexpected cookie data type in config: {type(cookie_data)}")
self.save_cookies()

def parse_cookie_string(self, cookie_string):
"""Parses cookie string and stores it."""
self.log(f"Parsing cookie string: {cookie_string}", logging.DEBUG)
logging.debug(f"Parsing cookie string: {cookie_string}")
cookie = SimpleCookie(cookie_string)
for key, morsel in cookie.items():
self.cookies[key] = {
Expand All @@ -63,7 +53,7 @@ def parse_cookie_string(self, cookie_string):

def parse_cookie_dict(self, cookie_dict):
"""Parses cookie dict and stores it."""
self.log("Parsing cookie dictionary.", logging.DEBUG)
logging.debug("Parsing cookie dictionary.")
for key, cookie_data in cookie_dict.items():
self.cookies[key] = {
'name': cookie_data.get('name', key),
Expand All @@ -73,51 +63,48 @@ def parse_cookie_dict(self, cookie_dict):

def parse_netscape_cookie_file(self, file_path):
"""Parses a Netscape format cookie file and imports all cookies."""
self.log(f"Parsing Netscape cookie file: {file_path}")
logging.debug(f"Parsing Netscape cookie file: {file_path}")
try:
with open(file_path, 'r') as file:
for line in file:
if not line.startswith('#') and line.strip():
parts = line.split('\t')
parts = line.strip().split('\t')
if len(parts) >= 7:
try:
domain, _, _, _, expires, name, value = parts
domain, flag, path, secure, expires, name, value = parts
self.cookies[name] = {
'name': name,
'value': value.strip(),
'value': value,
'expires': self.ensure_timestamp(expires)
}
except IndexError as e:
self.log(f"Error parsing cookie line: {line.strip()} - {e}", logging.ERROR)

logging.error(f"Error parsing cookie line: {line.strip()} - {e}")
self.save_cookies()
except (FileNotFoundError, IOError) as e:
raise LensCookieError(
f"Error reading Netscape cookie file: {e}") from e
raise LensCookieError(f"Error reading Netscape cookie file: {e}") from e

def generate_cookie_header(self):
"""Generates a cookie header for requests."""
self.filter_expired_cookies()
cookie_header = '; '.join(
[f"{cookie['name']}={cookie['value']}" for cookie in self.cookies.values()])
self.log(f"Generated cookie header: {cookie_header}", logging.DEBUG)
logging.debug(f"Generated cookie header: {cookie_header}")
return cookie_header

def filter_expired_cookies(self):
"""Filters out expired cookies."""
self.log("Filtering expired cookies.")
logging.debug("Filtering expired cookies.")
current_time = time.time()
try:
self.cookies = {k: v for k, v in self.cookies.items(
) if not v['expires'] or v['expires'] > current_time}
self.cookies = {k: v for k, v in self.cookies.items()
if not v['expires'] or v['expires'] > current_time}
except ValueError as e:
self.log(f"Error filtering cookies: {e}. Rewriting cookies.", logging.ERROR)
logging.error(f"Error filtering cookies: {e}. Rewriting cookies.")
self.rewrite_cookies()

def update_cookies(self, set_cookie_header):
"""Updates cookies from the Set-Cookie header and saves them."""
self.log(
f"Updating cookies from Set-Cookie header: {set_cookie_header}")
logging.debug(f"Updating cookies from Set-Cookie header: {set_cookie_header}")
if set_cookie_header:
cookie = SimpleCookie(set_cookie_header)
for key, morsel in cookie.items():
Expand All @@ -132,12 +119,11 @@ def save_cookies(self):
"""Saves cookies to a file."""
with open(self.cookie_file, 'wb') as f:
pickle.dump(self.cookies, f)
self.log(f"Cookies saved to file: {self.cookie_file}")
logging.debug(f"Cookies saved to file: {self.cookie_file}")

def rewrite_cookies(self):
"""Rewrites cookies from the original config or file if an error occurs."""
self.log("Rewriting cookies from config or original file.",
logging.WARNING)
logging.warning("Rewriting cookies from config or original file.")
self.cookies = {}
self.load_cookies()
self.save_cookies()
Expand All @@ -151,6 +137,12 @@ def ensure_timestamp(expires):
try:
return float(expires)
except ValueError:
raise LensCookieError(
f"Failed to convert expires '{expires}' to timestamp: Invalid date value or format")
logging.debug(f"Failed to convert expires '{expires}' to float. Trying to parse date string.")
try:
dt = datetime.strptime(expires, '%a, %d-%b-%Y %H:%M:%S GMT')
timestamp = dt.timestamp()
return float(timestamp)
except ValueError as e:
logging.error(f"Failed to parse expires '{expires}' as datetime: {e}")
raise LensCookieError(f"Failed to convert expires '{expires}' to timestamp: {e}")
return expires
7 changes: 4 additions & 3 deletions src/chrome_lens_py/lens_api.py
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
from .request_handler import Lens
from .text_processing import simplify_output, extract_full_text
from .exceptions import LensAPIError, LensParsingError
from .utils import is_url # Import the is_url function
from .utils import is_url
import logging

class LensAPI:
def __init__(self, config=None, sleep_time=1000):
self.lens = Lens(config=config, sleep_time=sleep_time)
def __init__(self, config=None, sleep_time=1000, logging_level=logging.WARNING):
self.lens = Lens(config=config, sleep_time=sleep_time, logging_level=logging_level)

def get_all_data(self, image_source):
"""Returns all data for the image, accepting either a file path or a URL."""
Expand Down
20 changes: 17 additions & 3 deletions src/chrome_lens_py/main.py
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
import sys
import argparse
import logging
from .lens_api import LensAPI
from rich.console import Console
from .exceptions import LensAPIError, LensParsingError, LensCookieError
from .utils import is_url # Import the is_url function

console = Console()

Expand All @@ -13,6 +13,7 @@ def print_help():
console.print("[b]-h, --help[/b] Show this help message and exit")
console.print("[b]-c, --cookie-file[/b] Path to the Netscape cookie file")
console.print("[b]-p, --proxy[/b] Specify proxy server (e.g., socks5://user:pass@host:port)")
console.print("[b]--debug=(info|debug)[/b] Enable logging at the specified level")
console.print("\n[b]<data_type>[/b] options:")
console.print("[b]all[/b] Get all data (full text, coordinates, and stitched text)")
console.print("[b]full_text_default[/b] Get only the default full text")
Expand All @@ -32,20 +33,33 @@ def main():
help="Path to the Netscape cookie file")
parser.add_argument(
'-p', '--proxy', help="Proxy server (e.g., socks5://user:pass@host:port)")
parser.add_argument('--debug', choices=['info', 'debug'],
help="Enable logging at the specified level")

args = parser.parse_args()

if args.help or not args.image_source or not args.data_type:
print_help()
sys.exit(1)

# Настраиваем уровень логирования на основе параметра debug
if args.debug == 'debug':
logging_level = logging.DEBUG
elif args.debug == 'info':
logging_level = logging.INFO
else:
logging_level = logging.WARNING

logging.basicConfig(level=logging_level)

config = {}
if args.cookie_file:
config['headers'] = {'cookie': args.cookie_file}
if args.proxy:
config['proxy'] = args.proxy

api = LensAPI(config=config)
# Передаем уровень логирования в LensAPI
api = LensAPI(config=config, logging_level=logging_level)

image_source = args.image_source
data_type = args.data_type
Expand All @@ -65,7 +79,7 @@ def main():
console.print("[red]Invalid data type specified.[/red]")
sys.exit(1)

# Output the result
# Выводим результат
console.print(result)

except (LensAPIError, LensParsingError, LensCookieError) as e:
Expand Down
Loading

0 comments on commit b8a9f21

Please sign in to comment.