v0.17

Highlights

• LightGBM evaluation 3-4x faster!
• Spark Serving v2
• LightGBM training supports early stopping and regularization
• LIME on Spark significantly faster

New Features

Spark Serving v2:

• Both Microbatch and Continuous mode have sub-millisecond latency
• Supports fault tolerance
• Can reply from anywhere in the pipeline
• Fail fast modes for warning callers of bad JSON parsing
• Fully based on DataSource API v2

LightGBM:

• 3-4x evaluation performance improvement
• Add early stopping capabilities
• Added L1 and L2 Regularization parameters
• Made network init more robust
• Fixed bug caused by empty partitions

LIME on Spark:

• LIME Parallelization significantly faster for large datasets
• Tabular Lime now supported

Other:

• Added UnicodeNormalizer for working with complex text
• Recognize Text exposes parameters for its polling handlers

Acknowledgements

We would like to acknowledge the developers and contributors, both internal and external who helped create this version of MMLSpark.

• Ilya Matiach, Markus Cozowicz, Scott Graham, Daniel Ciborowski, Jeremy Reynolds, Miguel Fierro, Robert Alexander, Tao Wu, Sudarshan Raghunathan, Anand Raman,Casey Hong, Karthik Rajendran, Dalitso Banda, Manon Knoertzer, Lars Ahlfors, The Microsoft AI Development Acceleration Program, Cognitive Search Team, Azure Search Team