ECE-Jobs-Board-Analysis

Scrape jobs found at http://ece.ucsd.edu/jobs and find patterns Helpful tutorial at http://newcoder.io/scrape/

Features

• Organize all job posting into a CSV file
• Find common connection between postings (skills, languages, etc)
• Classify types of jobs and experience needed
• Relate it back to ECE classes

Stack

Python (Version 2):

• Scrapy: web scraping framework
• SQLAlchemy: interact with databases (Postgres, MySQL, MS SQL, etc) without needing to write raw SQL code
• Pandas: high-performance, easy-to-use data structures and data analysis tools ("Excel" but in Python)
• Numpy: fundamental package for scientific computing ("MATLAB" but in python)
• Jupyter/IPython: interactive computing, useful for data visualization

Database:

• Postgres: popular database to store data

Task Manager:

• Cronjobs: job scheduler for Unix-like computers