node-web-crawler

A web crawler with a web user interface which shows crawling stats in realtime. Uses Node.JS, jQuery, Socket.IO, Express, Twitter Bootstrap.

Install

To use node web crawler, you need to have installed Node.js and MongoDB. When you have them both installed, just clone the repo:

git clone [email protected]:ecdeveloper/node-web-crawler.git

And then install dependencies:

npm install

That's it. Run the app:

node app

Name		Name	Last commit message	Last commit date
Latest commit History 54 Commits
public		public
routes		routes
views		views
.gitignore		.gitignore
LICENSE.txt		LICENSE.txt
README.md		README.md
app.js		app.js
config.json		config.json
crawling-daemon.js		crawling-daemon.js
package.json		package.json