Skip to content

Commit

Permalink
Update 基于nodejs的分布式爬虫客户端使用.md
Browse files Browse the repository at this point in the history
  • Loading branch information
nintha authored Feb 24, 2018
1 parent becd6c5 commit bb7cc35
Showing 1 changed file with 3 additions and 2 deletions.
5 changes: 3 additions & 2 deletions 基于nodejs的分布式爬虫客户端使用.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ npm install moment

在文件夹里新建一个名为 `main.js` 的文件,编辑输入
```javascript
//version 20180225-1
const superagent = require('superagent');
var moment = require('moment');
moment.locale('zh-cn');
Expand Down Expand Up @@ -94,12 +95,12 @@ const packageFetchInsertAsync = async (pid, mids) => {
break // 结束本次任务,尝试下个任务
}
}
await sleep(12000)
await sleep(10000)
if (cardList.length === midSize) {
await uploadPackageAsync(pid, cardList)
console.log(`${nowstr()} Send package ${pid}`);
} else {
console.error(`${nowstr()} failed to fetch info, mids=${mids}`);
console.error(`${nowstr()} failed to fetch info,finished/all=${cardList.length}/${midSize}, mids=${mids}`);
}
}

Expand Down

0 comments on commit bb7cc35

Please sign in to comment.