Implemented in emlearn
Simple generative model. Very effective at some classification problems. Quick and easy to train, just basic descriptive statistics. Making predictions also quick, amounts to calculating probabilities for each class.
Variations
- Gaussian,Multinomial,Bernouilli
- Adaptive Naive Bayes (ANBC)
- Fuzzy Naive Bayes
- Rough Gaussian Naive Bayes
- Non-naive Bayes. Actually takes covariance into account
Naive Bayes classifier implementations
- Python. Discrete Naive Bayes. AppliedMachineLearning
- Gaussian Naive Bayes in Python. MachineLearningMastery
- Discrete Naive Bayes in C++ codeforge
- Discrete,Gaussian,Multinomial,etc in Python scikit-learn
- Gaussian Naive Bayes from scratch, in Python. chrisalbon. Good walkthrough
- Multinomial, Bernoulli and Gaussian. In Python, with sklearn APIs. kenzotakahashi
Techniques for improvement
- Compensate for naive assumption of indepenence. Introduce covariance.
- Bagging, one-against-many
References
- Binary LNS-based Naive Bayes Hardware Classifier for Spam Control. Naive Bayes on FGPA. Not using a gaussian.
Gaussian Naive Bayes
- Naive Bayes Models for Probability Estimation Proposes naive Bayes models as an alternative to Bayesian networks for general probability estimation tasks. Compared on 50 UCI repo datasets. The two take similar time to learn and are similarly accurate, but naive Bayes inference is orders of magnitude faster. Code
- Comparing fuzzy naive bayes and gaussian naive bayes for decision making in robocup 3d Fuzzy Naive Bayes classifier just a little better than the Gaussian Naive Bayes. Beat Decision Trees.
Prior art for embayes optimization
- Fast Gaussian Naïve Bayes for searchlight classification analysis Called M-GNB / Massive-GNB. Equation (2) has the simplfied quadratic equation also found in embayes. Also uses a sparse computation. Was 34 times faster than libSVM. Code in MATLAB/C++.
- Learning with Mixtures of Trees.
- The Likelihood, the prior and Bayes Theorem. Derives equation close to embayes via minus log likelyhood of Gaussian distribution.
- GENERATIVE AND DISCRIMINATIVE CLASSIFIERS: NAIVE BAYES AND LOGISTIC REGRESSION, chapter 2.4, 3.1. maximum likelihood estimator (MLE) and minimum variance unbiased estimator (MVUE) which is very similar. Explains relationship between Gaussian Naive Bayes and Logistic Regression. "if the GNB assumptions hold, then asymptotically (with training examples) the GNB and Logistic Regression converge toward identical classifiers"
Baysian Networks
- Learning of Bayesian Network Classifiers Under Computational Constraints. Online learning of Bayesian network classifiers (BNCs). Using low bit-width fixed-point numbers.