Skip to content

Latest commit

 

History

History
21 lines (17 loc) · 1.92 KB

Scrapy.md

File metadata and controls

21 lines (17 loc) · 1.92 KB

Problem Statements

1. JOB crawler to scrape jobs / resources across all the job portal in India and store the data to a CSV file.

2. Build a scraper for an actual freelance gig where the client wants a Python program to scrape data from Stack Overflow to grab new questions (question title and URL). Scraped data should then be stored in MongoDB.

3. Build a scraper to generate CloudCoder problems using ideone.com , Hackerrank, CodeChef and HackerEarth public submissions

4. Build a crawler which searches across website and provide information regadring Manufacturing companies in India.

5. Search accorss all the news feed and provide latest update abot technology updates and Scraped data should be send to developers via mail.

6. Eduaction crawler to scrape all the college websites and provide information including Placement, admission, companies visited.

Use cases from Lazer

1. Scrape products from retailer or manufacturer websites to show on their own website or provide specs/price comparison

2. Scrape business profiles and reviews to track online presence and reputation

3. Scrape people profiles from social networks for tracking online reputation

4. Scrape search engine results for SEO tracking monitor specific company pages from social networks to gather what people are saying about certain companies and their products

5. Scrape health physicians from their clinic websites to provide a catalog of available doctors per specialization and region

6. Scrape job ads from many ATS (Application Tracking System) to build online directories or provide better targeted ads to their customers

7. Scrape product reviews from retailers to detect fraudulent reviews

8.Scrape news websites to apply custom analysis and curation (manual or automatic), often with the goal of providing better targeted news to their audience