Machine Learning

This is the combination of some algorithms and codes from scratch.

Please use Chrome with MathJax plugin.

Email: [email protected]

Topics	Introductions	Codes
1. Math	1. Bayesian Estimation, MLE, MAP 2. Exponential Family Distribution 3. Quadratic Form & Quadratic Matrix Defferentiating 4. Jacobian & Hessian Matrix 5. Gradient Descent 6. Newton's Method & Quasi-Newton 7. EM
2. Evaluation Metrics	1. Classification 中文 2. Regression
3. Perceptron	Perceptron	Perceptron Code
4. Linear Regression(Not finished)
5. KNN	KNN	KNN Classifier: mnist handwriting recognition
6. Decision Tree	Decision Tree: Feature Selection, Build Tree, Pruning	1. Decision Tree Classifier 2. Decision Tree Regressor
7. Logistic Regression	Logistic Regression	Logistic Regression Classifier
8. Naive Bayes	Naive Bayes Intro	Naive Bayes Classifier: GaussianNB, MultinomialNB, BernoulliNB
9. Bagging	Random Forest
10. Boosting	1. Adaboost Classifier 2. GBDT 3. XgBoost(Not Finished)	Adaboost Classifier Code
11. SVM	SVM: Duality, KKT, Hard Margin, Soft Margin, SMO	Linear SVM Code
12. Kernel Methods	Kernels (Not Finished)	Kernel SVM (Not Finished)
13. Dimensionality Reduction	1. PCA, SVD, PCoA, PPCA 2. Linear/Fisher Discriminant Analysis	1. PCA 2. SVD
14. L1, L2 Regularization	L1, L2 Regularization
15. HMM	HMM
16. CRF	CRF
17. NLP	1. Word Embedding 2. Language model, RNNs, GRU & LSTM

References

1. James, G., Witten, D., Hastie, T., & Tibshirani, R. (2017). An introduction to statistical learning: with applications in R. New York: Springer.
2. Bishop, Christopher M. Pattern Recognition and Machine Learning. Springer New York, 2016.
3. StatQuest with Josh Starmer. https://statquest.org
4. Jie Zhou: https://github.com/shuhuai007/Machine-Learning-Session