Final project for ELEC 825 Machine Learning/ Deep Learning. A research project into using cGANs to remove snow distortion from input images.
The contribution of this repository is to continue exploring methods of performing the task of weather removal. Most other generator structures such as DesnowGAN, have components designed for snow removal, followed by residual generation to fill the image. The pix2pix GAN is designed for general image translations and learns the translation necessary based on the provided images. The results show that simply learning the translation provides decent performance and further research into learning complex image translations could be combined with existing methods to further improve results.
Main analysis and training was performed using the Snow100K dataset. The dataset can be accessed here. Includes 50K training and 50K testing images of synthetically created snowy images and their groud truth images.
a) synthetic input b) generated snow removal c) ground truth
Main file used to train the cGAN, runs in the standard python console. Uses the .npz files generated by "npz_data_generator.py". Contains an experiemntal section to change many charcateristics of how the layers of the GAN are organized, including batch size, epoch count, downsampling method, batch normalization use, and dropout. This script will output network weight checkpoint files and example generated images on regular intervals.
File used to generate .npz files for input data into the cGAN. The synthetic and ground truth images are both taken from their stored directories and loaded into a numpy .npz file for quicker and easier loading into the cGAN. This way image files can be loaded once and accesses quicker.
Jupyter notebook that will calculate psnr and ssim results for the network weight files generated by "main_GAN_trainer". Will generate its own test images from a given file path containing images never seen before by the cGAN. Outputs the results to .xlsx files for easier analysis using excel.
A folder containing various scripts that were used to pre-process or create data to be used by the cGAN. Each file contains an outline at the top of the file and is well commented.
Riley Cooper - Electrical and Computer Engineering
Jason Harris - Electrical and Computer Engineering
Liam Horton - Mechanical and Materials Engineering
Francesco Marrato - Electrical and Computer Engineering
Full report is included which cover ours groups research, tests, and analysis.