Skip to content

Akulavenkatesh/google_url_scraper

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

7 Commits
 
 
 
 
 
 
 
 

Repository files navigation

Python script: Scrape top Google search results for a given keyword

The script extracts only wikipedia pages URL's from Google search results for a given keyword. This script will be very useful if you want to find the wikipedia pages of Personalities, Celbrities, Sports persons etc

I'm resticting Google to show only wikipedia pages URL's by giving this "site:en.wikipedia.org/wiki/" rule while searching Google. Modify the url as per your needs.

Inside the script in the place of keywords enter your own keywords.

I'm using inbuilt python Google Search module. The URL's of the wikipedia pages will be output to the python console

You can continuously ping Google and extract search results of 100 keywords without getting blocked. Post this limit Google server stops responding to the script thus you get an error in the console.

I'm using a data card, so I will disconnect and reconnect to internet (new IP) again after extracting search results for 100 keywords.

If you are using a router just restart it.

About

Scrape top Google search results for a given keyword

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages