Team Members: Amol Khanna and Connie-Chang Chien
Link to App: WAIVED. Please note that Dr. Caffo waived the requirement for our team to submit a link to a live app. This is because our project required TensorFlow, and when we tried to host our project on shinyapps.io, it would tell us that our project used too much memory and we would have to upgrade our plan to a paid subscription.
Link to Documentation: Instructions. These instructions are also displayed on our app.
Project Roles: Amol wrote the backend, documentation, and README. Connie wrote the frontend.
Federated learning is an algorithm proposed by scientists at Google which allows data scientists to combine neural networks concurrently trained on data located in separate, private locations by sending neural network parameters to a central server, which can conduct a weighted averaging of the parameters. The central server can then send these new weights back to its clients, which can update their models and continue training in an iterative fashion.
Our team designed an RShiny app which can simulate federated learning in the binary prediction setting with a dense neural network. The application requires the user to upload 4 CSV files: 2 data files and 2 corresponding results files. All of these files must not contain any row names or data labels. Since this application is designed to be used after scientists perform some data processing, we thought it was reasonable to assume that users could put their data in this format. After uploading the data, users must input the architecture of their network, excluding their data input layer. For this basic application, we have included ReLU and sigmoidal activation functions for each layer. Once the user finishes updating the architecture, they need to choose the number of iterations which they want to run globally and locally. This means that a total of (global iterations x local iterations) epochs will run in this program. Note that after users hit run, they may have to wait for a long time (up to 10 minutes) for the program to produce an output, as this program initializes and trains multiple neural networks.
Once the program runs, it will output 4 graphs. 2 graphs will demonstrate the local loss on each client. Graphs demonstrating a generally decreasing loss trend indicate that the model trains locally on each client. The other 2 graphs will demonstrate the federated loss after each global iteration. A decreasing loss on these graphs indicates that the federated model indeed improves after the weights from the locally trained models are averaged, indicating that federated learning produces a model which can predict trends from the data.
To provide the graders with an example, we have uploaded two data and results files which match the data format which our application requires to our GitHub submission. If the graders upload these files to their corresponding locations within our application, and then choose a reasonable architecture for binary prediction, say 10 (ReLU) 1 (Sigmoid), and sufficiently large values for global and local iterations, such as 5 and 5, they should see generally decreasing loss with iterations. As stated before, expect to wait between 2 and 10 minutes for the application to produce an output, since it is training multiple neural networks and TensorFlow for R is quite slow.
The last thing we want to mention is that using TensorFlow and Keras from R can be quite finicky. In our situation, we created an Anaconda environment named r-reticulate, and installed Shiny, Tidyverse, Reticulate, TensorFlow for R, Keras for R, and virtualenv. We also installed and ran our code through Rstudio in our r-reticulate Anaconda environment. Please note that you must use the latest version of r-keras (2.4), which is not what Anaconda installs by default. You must also name your environment r-reticulate, because otherwise TensorFlow for R will not run. You must also download and include the .Rprofile in our GitHub submission in the directory with server.R and ui.R. Note that we have attached the file r-reticulate.yml to our submission. This file contains the state of our directory at the time of submission. Feel free to reach out to us if your environment isn’t working; it took us a few days to figure out how to run ours. While this project was certainly interesting to write, the sheer number of dependencies and very specific way that the dependencies need to be installed would certainly be a reason why users could not be expected to run it on their own machines.