Skip to content

Shifu 0.2.7 Train Step Improvement

Jump to bottom Edit New page
Zhang Pengshan (David) edited this page Aug 31, 2015 · 2 revisions

  1. Support Re-Try Failed Job Feature on D-Training Sometimes one of multiple bagging jobs is failed, we need restart all jobs to get enough models. Add re-try to make it more stable for some cases of failed jobs. https://github.com/ShifuML/shifu/issues/106

  2. Change Double to Float for MemoryDiskMLDataSet to Save Training Memory Space All data saved in memory now is float type instead of double type, this saves memory for model training. https://github.com/ShifuML/shifu/issues/159

  3. Tune Parameter for Training Tune some parameters according to Hadoop 2.0 (some configurations are not used in Hadoop 2.0 like mapred.map.java.opts). https://github.com/ShifuML/shifu/issues/166

  4. Add regularization for NN (Experimental) Should be tested more in Shifu 0.2.8 https://github.com/ShifuML/shifu/issues/66

Shifu: A Distributed Model Training Framework on Hadoop

DOWNLOAD

Clone this wiki locally