Distributed Neural Network Training in Shifu

Neural Network model is supported in Shifu based on encog framework. While encog is single machine based. In Shifu, encog code is changed to do distributed model training.

Distributed Neural Network Training

• Data are split into workers.
• Each worker collects gradients on each connection of neural network settings and send back to master
• Master accumulates all gradients and get a updated model based on global gradients and send new model to workers
• Workers continue working on collecting gradients based on current model
• Model training will be stopped according to max iterations.

Configurations in Neural Network Model Training

TODO

How to Tune Parameters to Accelerate NN Model Training

TODO

How to Set Hidden Layer in Neural Network (Shallow)

• TODO
• TODO