Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

爬虫做种的时候报错 #10

Open
q1192589000 opened this issue May 23, 2018 · 14 comments
Open

爬虫做种的时候报错 #10

q1192589000 opened this issue May 23, 2018 · 14 comments

Comments

@q1192589000
Copy link

success to fetched hot users: 24
Traceback (most recent call last):
File "spider.py", line 475, in
spider.seedUsers()
File "spider.py", line 328, in seedUsers
self.db.commit()
File "spider.py", line 101, in commit
self.dbconn.commit()
AttributeError: 'NoneType' object has no attribute 'commit'

请问有没有什么解决方法呢?
操作系统是用的 Centos 7X64
Python版本是:2.7.5

@saaak
Copy link

saaak commented Apr 27, 2019

seedhotuser里有段代码有问题

@q1192589000
Copy link
Author

现在估计也爬不了百度了

@k1995
Copy link
Owner

k1995 commented Apr 30, 2019

新版本正在开发中,dev分支。旧版本因百度API失效了,爬虫不可用

@q1192589000
Copy link
Author

期待!!

@JackyKen
Copy link

现在新版本可用了吗

@k1995
Copy link
Owner

k1995 commented Jun 21, 2019

可以

@saaak
Copy link

saaak commented Jun 21, 2019

可以

现在只能手动添加地址,不能自动爬?

@k1995
Copy link
Owner

k1995 commented Jun 21, 2019

emm.....开源版的目前是这样的

@saaak
Copy link

saaak commented Jun 22, 2019

emm.....开源版的目前是这样的

还有闭源版的吗

@saaak
Copy link

saaak commented Jun 22, 2019

emm.....开源版的目前是这样的

貌似百度把接口封了

@k1995
Copy link
Owner

k1995 commented Jun 22, 2019

有闭源版的,现在功能不太稳定,数据较少。百度把接口封了,只能自己做全网URL发现或者手动提交URL。

@saaak
Copy link

saaak commented Jun 22, 2019

有闭源版的,现在功能不太稳定,数据较少。百度把接口封了,只能自己做全网URL发现或者手动提交URL。

闭源版以后会开源吗

@bytesniffer
Copy link

bytesniffer commented Jun 30, 2019

全网url 发现,大概怎么弄,效率不高吧

@wwmxs
Copy link

wwmxs commented Feb 20, 2023

1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants