Merge pull request #615 from ShifuML/develop

Release 0.12.0

CHANGES.txt

@@ -16,6 +16,18 @@
 
 Shifu Change Log
 
+Changes for Shifu-0.13.0
+	* Integration of Tensorflow
+
+Changes for Shifu-0.11.2
+    * Bug on GBT version when loading GBT model: if use static version, it will cause issue when loading multi GBT 
+      models
+       https://github.com/ShifuML/shifu/issues/487
+
+Changes for Shifu-0.11.1
+    * TreeModel Load Failed for Category > 10K
+       https://github.com/ShifuML/shifu/issues/482
+
 Changes for Shifu-0.11.0
     * Python Module Support
        https://github.com/ShifuML/pyshifu
@@ -39,7 +51,7 @@ Changes for Shifu-0.11.0
        https://github.com/ShifuML/shifu/issues?q=is%3Aissue+is%3Aclosed
 
 Changes for Shifu-0.10.5
-    * Optimize IndependetTreeModel by decreaseing model memory to 70% and CPU time to 90%
+    * Optimize IndependetTreeModel by decreasing model memory to 70% and CPU time to 90%
     * Upgrade guagua to 0.7.0 t fix a bug on empty gzip files in one worker
 
 Changes for Shifu-0.10.4

README.md

@@ -1,9 +1,9 @@
-[<img src="images/logo/shifu.png" alt="Shifu" align="left">](http://shifu.ml)<div align="right"><div>[![Build Status](https://travis-ci.org/ShifuML/shifu.svg)](https://travis-ci.org/ShifuML/shifu)</div><div>[![Maven Central](https://maven-badges.herokuapp.com/maven-central/ml.shifu/shifu/badge.svg)](https://maven-badges.herokuapp.com/maven-central/ml.shifu/shifu)</div></div>
+[<img src="images/logo/shifu.png" alt="Shifu" align="left">](http://shifu.ml)<div align="right"><div>[![Build Status](https://travis-ci.org/ShifuML/shifu.svg)](https://travis-ci.org/ShifuML/shifu?branch=develop)</div><div>[![Maven Central](https://maven-badges.herokuapp.com/maven-central/ml.shifu/shifu/badge.svg)](https://maven-badges.herokuapp.com/maven-central/ml.shifu/shifu)</div></div>
 
 #
 
 ## Download
-Please [download](https://github.com/ShifuML/shifu/wiki/shifu-0.10.5-hdp-yarn.tar.gz) latest shifu [here](https://github.com/ShifuML/shifu/wiki/shifu-0.10.5-hdp-yarn.tar.gz).
+Please [download](https://github.com/ShifuML/shifu/wiki/shifu-0.11.1-hdp-yarn.tar.gz) latest shifu [here](https://github.com/ShifuML/shifu/wiki/shifu-0.11.1-hdp-yarn.tar.gz).
 
 ## Getting Started
 After shifu downloading, build your first model with Shifu [tutorial](https://github.com/ShifuML/shifu/wiki/Tutorial---Build-Your-First-ML-Model). More details about shifu can be found in our [wiki pages](https://github.com/ShifuML/shifu/wiki).
@@ -23,6 +23,7 @@ Shifu provides a simple command-line interface for each step of the model buildi
 * [Distributed neural network model training](https://github.com/ShifuML/shifu/wiki/Distributed%20Neural%20Network%20Training%20in%20Shifu)
 * [Distributed tree ensemble model training](https://github.com/ShifuML/shifu/wiki/Distributed%20Tree%20Ensemble%20Model%20Training%20in%20Shifu)
 * Post training analysis & model evaluation
+* [Distributed Tensorflow on SHIFU] {https://github.com/ShifuML/shifu/wiki/Distributed-Tensorflow-Support-On-SHIFU)
 
 Shifu’s fast Hadoop-based, distributed neural network / logistic regression / gradient boosted trees training can reduce model training time from days to hours on TB data sets. Shifu integrates with Pig workflows on Hadoop, and Shifu-trained models can be integrated into production code with a simple Java API. Shifu leverages Pig, Akka, Encog and other open source projects.
 
@@ -50,11 +51,12 @@ More details about shifu can be found in our [wiki pages](https://github.com/Shi
  - Sharma Nitin ([email protected])
  - Wayne Zhu ([email protected])
  - Devin Wu ([email protected])
+ - Fred Bai ([email protected])
 
 ## Google Group
 
 Please join [Shifu group](https://groups.google.com/forum/#!forum/shifuml) if questions, bugs or anything else.
 
 ## Copyright and License
 
-Copyright 2012-2017, PayPal Software Foundation under the [Apache License](LICENSE.txt).
+Copyright 2012-2019, PayPal Software Foundation under the [Apache License](LICENSE.txt).

assembly-hadoop1.xml

@@ -147,6 +147,7 @@
                 <include>*:automaton:*</include>
                 <include>*:jpmml:*</include>
                 <include>*:stream:*</include>
+                <include>*:reflections:*</include>
                 <include>*:parquet-pig:*</include>
                 <include>*:parquet-column:*</include>
                 <include>*:parquet-common:*</include>
@@ -156,6 +157,8 @@
                 <include>*:parquet-format:*</include>
                 <include>*:parquet-jackson:*</include>
                 <include>*:snappy-java:*</include>
+                <include>*:zip4j:*</include>
+                <include>*:shifu-tensorflow-on-yarn:*</include>
             </includes>
         </dependencySet>
         <dependencySet>

assembly-hadoop2.xml

@@ -21,6 +21,13 @@
                 <include>*.pig</include>
             </includes>
         </fileSet>
+        <fileSet>
+            <directory>${project.basedir}/src/main/python</directory>
+            <outputDirectory>/scripts</outputDirectory>
+            <includes>
+                <include>*.py</include>
+            </includes>
+        </fileSet>
         <fileSet>
             <directory>${project.basedir}/src/main/pig/stats/hadoop2</directory>
             <outputDirectory>/scripts</outputDirectory>
@@ -133,17 +140,18 @@
                 <include>*:liblinear:*</include>
                 <include>*:log4j:*</include>
                 <include>*:netty:*</include>
-                <include>*:pig:*</include>
+                <include>*:pig:*:h2:*</include>
+                <include>*:piggybank:*</include>
                 <include>*:scala-library:*</include>
                 <include>*:shifu:*</include>
                 <include>*:slf4j-api:*</include>
                 <include>*:stringtemplate:*</include>
                 <include>*:xmlenc:*</include>
-                <include>*:xz:*</include>
                 <include>*:zookeeper:*</include>
                 <include>*:joda-time:*</include>
                 <include>*:automaton:*</include>
                 <include>*:stream:*</include>
+                <include>*:reflections:*</include>
                 <include>*:parquet-pig:*</include>
                 <include>*:parquet-column:*</include>
                 <include>*:parquet-common:*</include>
@@ -165,4 +173,4 @@
             </includes>
         </dependencySet>
     </dependencySets>
-</assembly>
+</assembly>