NSFW boards/threads which require cookies can't be loaded, and if provided python still throws an error

[r3538987]

Введите название доски: e
Введите номер треда: 527494

Traceback (most recent call last):
  File "C:\Users\user\Desktop\2ch-main\media.py", line 22, in <module>
    thread.update_posts()
  File "C:\Users\user\Desktop\2ch-main\dvach.py", line 167, in update_posts
    self.get_posts(json_posts)
  File "C:\Users\user\Desktop\2ch-main\dvach.py", line 172, in get_posts
    self.unique_posters = int(posts_json[0]['unique_posters'])
KeyError: 0
>>> 

Strange is that some boards like "gg" work, but others - in example "fet" - doesn't.

[diademoff]

Strange is that some boards like "gg" work, but others - in example "fet" - doesn't.

Every user has usercode_auth. It is required to get access to some boards. I have found usercode_auth here: https://2ch.hk/b/arch/2018-08-06/res/180783789.html and just copy-paste it. You can find it in dvach.py and replace to yours.

To get your usercode_auth press F12 go to Network.

Then replace 3c86e2be7602c264ffddd9723be0688b to yours.

headers = {
    "Accept": "image/webp,*/*",
    "User-Agent": "Mozilla/5.0 (X11; Linux x86_64; rv:88.0) Gecko/20100101 Firefox/88.0",
    "Accept-Encoding": "gzip, deflate, br",
    "Accept-Language": "ru-RU,ru;q=0.8,en-US;q=0.5,en;q=0.3",
    "Cookie": "usercode_auth=MY-USER-CODE; wakabastyle=Futaba;"
}

[r3538987]

Argh, forgot to add it. I did found out thing about usercode_auth in dvach.py file. Replaced that but result is exactly same.
Strange is that other similar scripts work. Just fine.

When you open private browser session, you get 404 on GG and other pages.
When you enter code into script, GG gets loaded, but others - don't that's to me is confusing.

[diademoff]

I can not catch this error. I tried every nsfw board, but it works for me. Try:

• Download the latest version.
• Set delay more.
• Check your keyboard layout (English e and Russian е are similar)
• Reset headers to default value

If it doesn't help, create new file in repository root directory and paste this code and replace board name:

import dvach
import time

board_name = 'TYPE BOARD NAME HERE'

board = dvach.Board.from_json(dvach.Board.json_download(board_name))

board.update_threads()

for key in list(board.threads.keys()):
    thread = board.threads[key]
    thread.update_posts()
    print(f'Загружен тред: {thread.num}, постов: {len(thread.posts)}')
    time.sleep(2)

This script downloads threads one by one.

It catches the error. If scripts throws exception let me know.

[diademoff]

When you open private browser session, you get 404 on GG and other pages.
When you enter code into script, GG gets loaded, but others - don't that's to me is confusing.

@r3538987
Got the same. I typed in console (F12):

document.cookie = "usercode_auth=3c86e2be7602c264ffddd9723be0688b; path=/; expires=${new Date(new Date().getTime() + 60 * 60 * 60 * 1000).toUTCString()}";

And it worked.

[r3538987]

Assuming code you provided will extent cookie expiring date. But I doubt it will help.
Default or mine usercode:

Python 3.9.5 (tags/v3.9.5:0a7dcbd, May  3 2021, 17:27:52) [MSC v.1928 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license()" for more information.
>>> 
============== RESTART: C:\Users\user\Desktop\2ch-main\media.py =============
Введите название доски: e
Введите номер треда: 529343
Traceback (most recent call last):
  File "C:\Users\user\Desktop\2ch-main\media.py", line 22, in <module>
    thread.update_posts()
  File "C:\Users\user\Desktop\2ch-main\dvach.py", line 167, in update_posts
    self.get_posts(json_posts)
  File "C:\Users\user\Desktop\2ch-main\dvach.py", line 172, in get_posts
    self.unique_posters = int(posts_json[0]['unique_posters'])
KeyError: 0
>>> 
============== RESTART: C:\Users\user\Desktop\2ch-main\media.py =============
Введите название доски: gg
Введите номер треда: 1047871
Тред скачен
Скачан файл: 16222051872430.jpg
Скачан файл: 16222051872441.jpg
Скачан файл: 16222051872472.jpg

Now will check piece of code you offered above.
FET/E - \2ch-main\dvach.py", line 172, in get_posts self.unique_posters = int(posts_json[0]['unique_posters']) KeyError: 0
HC/GG - loads just fine. 😄

Does this mean that when asking certain boards I get empty JSON responses ?

[diademoff]

media.py

media.py is deprecated, download latest version with thread_saver.py. This may help as there are a lot of improvements.

Does this mean that when asking certain boards I get empty JSON responses ?

Yes. You got empty response for some reason.

[r3538987]

========== RESTART: C:\Users\user\Desktop\2ch-main\thread_saver.py ==========
Введите название доски: e
Введите номер треда: 526336
Не удалось получить тред, повтор
Не удалось получить тред, повтор

Okay, later I will try to debug this and see where issue is.

[Notrum666]

to anyone wondering here with this problem, solved it via modifying the
requests.get
method call at the very end of dvach.py file, i've added
cookies={"usercode_auth": "my actual auth code"}
argument (with real auth code i found in my browser cookies, read the replies above to figure out how to find it)

[r3538987]

to anyone wondering here with this problem, solved it via modifying the requests.get method call at the very end of dvach.py file, i've added cookies={"usercode_auth": "my actual auth code"} argument (with real auth code i found in my browser cookies, read the replies above to figure out how to find it)

2 years later, don't think this is actual, at least for me. :)
So you just hardcoded auth code instead of asking for user input. Back then I think* I tried this as well, and result was same.
Could that years after site got more polished and fixed problems.

[Notrum666]

the point is not that i hardcoded it, the point is that i at least passed it to the method, othervise json was always empty (for boards that require this code), passing headers also didn't work (as in commented-out method call) so i googled how to pass cookies to requests.get and did just that