ETL
Folders and files
Name | Name | Last commit date | ||
---|---|---|---|---|
parent directory.. | ||||
Installation of MySQL Server: MySQL Community Server http://dev.mysql.com/downloads/ On Mac, download the .dmg file Starting MySQL Server (on Mac) - In Spotlight search for MySQL - MySQL window should open up with “Start MySQL Server” button on it - MySQL Server Instance should be in “stopped” status - Hit the “Start MySQL Server” button; you may be prompted for a password - Enter the password; MySQL server should be up MySQL Clients - Command line - /usr/local/mysql/bin/mysql – if you install the .dmg from above link - MySQL Workbench - SQuirrel SQL - http://squirrel-sql.sourceforge.net/ Local MySQL Server 127.0.0.1 Port number 3306 Usename: root Setting up database: 1) Run mysql client - /usr/local/mysql/bin/mysql -u root 2) Create Database - create database student_courses; 3) Create User - Example: create user 'devdatta'@'localhost’; 4) Grant Privileges GRANT ALL ON student_courses.* TO ‘devdatta’@’localhost’; 5) Logout - On mysql prompt: “quit” ETL Example: The example in this directory model the situation of a course having one or more assignments. There are two entities corresponding to a course and a assignment. These are defined in assign.domain package. There is a foreign key relationship between assignment entity and course entity. This relationship is expressed in the assignment entity using @ManyToOne and @JoinColumn annotations, and in the course entity via the @OneToMany annotation. Both the entities have an autogenerated Id column identified by @Id annotation. Hibernate configuration is provided in the file: src/main/resources/hibernate.cfg.xml There is a section in it to specify the database settings. You will need to modify it based on the DB creds that you created above. The value of the property hibernate.hbm2ddl.auto is set to create. This means that everytime you run the tests, the tables will be created fresh (the data from the previous run will be removed). You can comment out this property to prevent tables from recreating everytime. The property show_sql is set to true. This causes the SQL statements generated by Hibernate to be output. You can use these statements to create the tables and set up foreign key reference. In assign.etl package there are following classes: - ETLController.java - EavesdropReader.java - Transformer.java - DBLoader.java The ETLController has a main method and performETLActions method. You can execute the main method from Eclipse. The performETLActions executes the steps of reading the data, transforming it, and loading it into the database by invoking methods on the corresponding classs' objects. Currently these methods are empty. You need to implement these methods. Also, feel free to change the signature of the methods (input and return types) as seem appropriate to your design. In TestETLHandler class has test methods which show example of using Hibernate for performing database calls. You can run these test cases by importing the project in Eclipse.