Machine Learning algorithm implementation in python without scikit-learn

This repository contains implementation for basic Machine Learning algorithms in Python from scratch. People normally use different python libraries(e.g scikit-learn) for ML related tasks. But this repository helps to understand the algorithm and the concepts from scratch.

Table of contents;

---

1. Linear Regression:

Program that implements a (batch) linear regression using the gradient descent method in Python 3.

2. Decision Tree:

Program that implements a decision tree using the ID3 algorithm. Information Gain as your decision measure and treat all features as discrete multinomial distributions. There are the two data sets named car and nursery as csv files. Both data sets can be learned perfectly, i.e. all leaves have an entropy of 0. The output of algorithm is tree in XML format

3. Perceptron:

Program that implements a single perceptron using the delta rule.

4. Gaussian Naive Bayes:

Program that implements a 2-class Naive Bayes algorithm with an apriori decision rule using a multinomial estimation for the classes and a gaussian estimation for the attributes.

5. KNN:

Program that implements a 2-class kNN classifier with case base created by IB2 algorithm. The assignment for the point is then determined by the sum of the weights per class, whereas the maximal sum is the "winner".

6. KMeans:

program that implements the k-Means clustering. The optimization criteria can be seen as a kind of Sum of Squared Errors.The goal is to minimize J, which basically minimizes the sum of intra-cluster distances.

---

Citation:

If this repository helps you or you use some of its code, please refer the following archive as citation in your codes/publications.

Contact:

For any issues, clarification, bug or improvement suggestion, please send an email to [email protected]. I will reach back as soon as possible.