ML_models_with_Wine.ipynb

{"nbformat":4,"nbformat_minor":0,"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.8.3"},"colab":{"name":"DM_HW3_20161641_Wine.ipynb","provenance":[],"collapsed_sections":[]}},"cells":[{"cell_type":"code","metadata":{"id":"nBp11-xouqUD","executionInfo":{"status":"ok","timestamp":1619796851434,"user_tz":-540,"elapsed":823,"user":{"displayName":"SungYub Jung","photoUrl":"","userId":"06395096588766753020"}}},"source":["import numpy as np\n","import pandas as pd"],"execution_count":1,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"yx7eJIJNuqUJ"},"source":["# Wine dataset 불러오기"]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"nNyujJ9EuqUJ","executionInfo":{"status":"ok","timestamp":1619796852735,"user_tz":-540,"elapsed":2119,"user":{"displayName":"SungYub Jung","photoUrl":"","userId":"06395096588766753020"}},"outputId":"861f6b03-ef5b-4acb-9209-e6b49032353d"},"source":["from sklearn.datasets import load_wine\n","from sklearn.model_selection import train_test_split\n","\n","wine = load_wine()\n","X_train, X_test, Y_train, Y_test = train_test_split(wine.data, wine.target, random_state=777, test_size=0.2)\n","print(X_train.shape)\n","print(X_test.shape)\n","print(Y_train.shape)\n","print(Y_test.shape)"],"execution_count":2,"outputs":[{"output_type":"stream","text":["(142, 13)\n","(36, 13)\n","(142,)\n","(36,)\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"KL918dteuqUK"},"source":["# Wine dataset (참고용)"]},{"cell_type":"code","metadata":{"scrolled":true,"colab":{"base_uri":"https://localhost:8080/","height":1000},"id":"G4wfdreNuqUK","executionInfo":{"status":"ok","timestamp":1619796852736,"user_tz":-540,"elapsed":2116,"user":{"displayName":"SungYub Jung","photoUrl":"","userId":"06395096588766753020"}},"outputId":"9cd10b41-ed71-4a82-c9c1-02e58f8a6699"},"source":["df = pd.DataFrame(wine.data, columns=wine.feature_names)\n","sy = pd.Series(wine.target, dtype=\"category\")\n","sy = sy.cat.rename_categories(wine.target_names)\n","df['class'] = sy\n","print(wine.DESCR)\n","df.head()"],"execution_count":3,"outputs":[{"output_type":"stream","text":[".. _wine_dataset:\n","\n","Wine recognition dataset\n","------------------------\n","\n","**Data Set Characteristics:**\n","\n","    :Number of Instances: 178 (50 in each of three classes)\n","    :Number of Attributes: 13 numeric, predictive attributes and the class\n","    :Attribute Information:\n"," \t\t- Alcohol\n"," \t\t- Malic acid\n"," \t\t- Ash\n","\t\t- Alcalinity of ash  \n"," \t\t- Magnesium\n","\t\t- Total phenols\n"," \t\t- Flavanoids\n"," \t\t- Nonflavanoid phenols\n"," \t\t- Proanthocyanins\n","\t\t- Color intensity\n"," \t\t- Hue\n"," \t\t- OD280/OD315 of diluted wines\n"," \t\t- Proline\n","\n","    - class:\n","            - class_0\n","            - class_1\n","            - class_2\n","\t\t\n","    :Summary Statistics:\n","    \n","    ============================= ==== ===== ======= =====\n","                                   Min   Max   Mean     SD\n","    ============================= ==== ===== ======= =====\n","    Alcohol:                      11.0  14.8    13.0   0.8\n","    Malic Acid:                   0.74  5.80    2.34  1.12\n","    Ash:                          1.36  3.23    2.36  0.27\n","    Alcalinity of Ash:            10.6  30.0    19.5   3.3\n","    Magnesium:                    70.0 162.0    99.7  14.3\n","    Total Phenols:                0.98  3.88    2.29  0.63\n","    Flavanoids:                   0.34  5.08    2.03  1.00\n","    Nonflavanoid Phenols:         0.13  0.66    0.36  0.12\n","    Proanthocyanins:              0.41  3.58    1.59  0.57\n","    Colour Intensity:              1.3  13.0     5.1   2.3\n","    Hue:                          0.48  1.71    0.96  0.23\n","    OD280/OD315 of diluted wines: 1.27  4.00    2.61  0.71\n","    Proline:                       278  1680     746   315\n","    ============================= ==== ===== ======= =====\n","\n","    :Missing Attribute Values: None\n","    :Class Distribution: class_0 (59), class_1 (71), class_2 (48)\n","    :Creator: R.A. Fisher\n","    :Donor: Michael Marshall (MARSHALL%PLU@io.arc.nasa.gov)\n","    :Date: July, 1988\n","\n","This is a copy of UCI ML Wine recognition datasets.\n","https://archive.ics.uci.edu/ml/machine-learning-databases/wine/wine.data\n","\n","The data is the results of a chemical analysis of wines grown in the same\n","region in Italy by three different cultivators. There are thirteen different\n","measurements taken for different constituents found in the three types of\n","wine.\n","\n","Original Owners: \n","\n","Forina, M. et al, PARVUS - \n","An Extendible Package for Data Exploration, Classification and Correlation. \n","Institute of Pharmaceutical and Food Analysis and Technologies,\n","Via Brigata Salerno, 16147 Genoa, Italy.\n","\n","Citation:\n","\n","Lichman, M. (2013). UCI Machine Learning Repository\n","[https://archive.ics.uci.edu/ml]. Irvine, CA: University of California,\n","School of Information and Computer Science. \n","\n",".. topic:: References\n","\n","  (1) S. Aeberhard, D. Coomans and O. de Vel, \n","  Comparison of Classifiers in High Dimensional Settings, \n","  Tech. Rep. no. 92-02, (1992), Dept. of Computer Science and Dept. of  \n","  Mathematics and Statistics, James Cook University of North Queensland. \n","  (Also submitted to Technometrics). \n","\n","  The data was used with many others for comparing various \n","  classifiers. The classes are separable, though only RDA \n","  has achieved 100% correct classification. \n","  (RDA : 100%, QDA 99.4%, LDA 98.9%, 1NN 96.1% (z-transformed data)) \n","  (All results using the leave-one-out technique) \n","\n","  (2) S. Aeberhard, D. Coomans and O. de Vel, \n","  \"THE CLASSIFICATION PERFORMANCE OF RDA\" \n","  Tech. Rep. no. 92-01, (1992), Dept. of Computer Science and Dept. of \n","  Mathematics and Statistics, James Cook University of North Queensland. \n","  (Also submitted to Journal of Chemometrics).\n","\n"],"name":"stdout"},{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>alcohol</th>\n","      <th>malic_acid</th>\n","      <th>ash</th>\n","      <th>alcalinity_of_ash</th>\n","      <th>magnesium</th>\n","      <th>total_phenols</th>\n","      <th>flavanoids</th>\n","      <th>nonflavanoid_phenols</th>\n","      <th>proanthocyanins</th>\n","      <th>color_intensity</th>\n","      <th>hue</th>\n","      <th>od280/od315_of_diluted_wines</th>\n","      <th>proline</th>\n","      <th>class</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>14.23</td>\n","      <td>1.71</td>\n","      <td>2.43</td>\n","      <td>15.6</td>\n","      <td>127.0</td>\n","      <td>2.80</td>\n","      <td>3.06</td>\n","      <td>0.28</td>\n","      <td>2.29</td>\n","      <td>5.64</td>\n","      <td>1.04</td>\n","      <td>3.92</td>\n","      <td>1065.0</td>\n","      <td>class_0</td>\n","    </tr>\n","    <tr>\n","      <th>1</th>\n","      <td>13.20</td>\n","      <td>1.78</td>\n","      <td>2.14</td>\n","      <td>11.2</td>\n","      <td>100.0</td>\n","      <td>2.65</td>\n","      <td>2.76</td>\n","      <td>0.26</td>\n","      <td>1.28</td>\n","      <td>4.38</td>\n","      <td>1.05</td>\n","      <td>3.40</td>\n","      <td>1050.0</td>\n","      <td>class_0</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>13.16</td>\n","      <td>2.36</td>\n","      <td>2.67</td>\n","      <td>18.6</td>\n","      <td>101.0</td>\n","      <td>2.80</td>\n","      <td>3.24</td>\n","      <td>0.30</td>\n","      <td>2.81</td>\n","      <td>5.68</td>\n","      <td>1.03</td>\n","      <td>3.17</td>\n","      <td>1185.0</td>\n","      <td>class_0</td>\n","    </tr>\n","    <tr>\n","      <th>3</th>\n","      <td>14.37</td>\n","      <td>1.95</td>\n","      <td>2.50</td>\n","      <td>16.8</td>\n","      <td>113.0</td>\n","      <td>3.85</td>\n","      <td>3.49</td>\n","      <td>0.24</td>\n","      <td>2.18</td>\n","      <td>7.80</td>\n","      <td>0.86</td>\n","      <td>3.45</td>\n","      <td>1480.0</td>\n","      <td>class_0</td>\n","    </tr>\n","    <tr>\n","      <th>4</th>\n","      <td>13.24</td>\n","      <td>2.59</td>\n","      <td>2.87</td>\n","      <td>21.0</td>\n","      <td>118.0</td>\n","      <td>2.80</td>\n","      <td>2.69</td>\n","      <td>0.39</td>\n","      <td>1.82</td>\n","      <td>4.32</td>\n","      <td>1.04</td>\n","      <td>2.93</td>\n","      <td>735.0</td>\n","      <td>class_0</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["   alcohol  malic_acid   ash  ...  od280/od315_of_diluted_wines  proline    class\n","0    14.23        1.71  2.43  ...                          3.92   1065.0  class_0\n","1    13.20        1.78  2.14  ...                          3.40   1050.0  class_0\n","2    13.16        2.36  2.67  ...                          3.17   1185.0  class_0\n","3    14.37        1.95  2.50  ...                          3.45   1480.0  class_0\n","4    13.24        2.59  2.87  ...                          2.93    735.0  class_0\n","\n","[5 rows x 14 columns]"]},"metadata":{"tags":[]},"execution_count":3}]},{"cell_type":"markdown","metadata":{"id":"0P9ONhQ4uqUL"},"source":["# # Problem 1\n","CART decision Tree classifier.\n","- scikit-learn의 decision tree는 CART 기반 구현체이다.\n","- Problem 1과 같은 방법으로 CART classifier 모델을 학습시키고 test데이터의 정확도를 출력하시오."]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"XVrCaoakuqUL","executionInfo":{"status":"ok","timestamp":1619796853068,"user_tz":-540,"elapsed":2444,"user":{"displayName":"SungYub Jung","photoUrl":"","userId":"06395096588766753020"}},"outputId":"bf518660-3647-44b7-aa4a-f629afa0aec5"},"source":["from sklearn.tree import DecisionTreeClassifier\n","\n","CART_model = None\n","accuracy = None\n","############# Put your code here ################\n","CART_model = DecisionTreeClassifier()\n","CART_model.fit(X_train,Y_train)\n","accuracy = CART_model.score(X_test,Y_test)\n","################################################# \n","print('Test data prediction accuracy (CART) : %f'%accuracy)"],"execution_count":4,"outputs":[{"output_type":"stream","text":["Test data prediction accuracy (CART) : 0.972222\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"8kt8-9W0uqUL"},"source":["# # Problem 2\n","Random Forest classifier.\n","- scikit-learn의 random forest 패키지를 이용하여 모델을 학습시키고 test데이터의 정확도를 출력하시오."]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"90e01vJHuqUM","executionInfo":{"status":"ok","timestamp":1619796853068,"user_tz":-540,"elapsed":2441,"user":{"displayName":"SungYub Jung","photoUrl":"","userId":"06395096588766753020"}},"outputId":"359e6e05-fbe1-47bc-e37f-70255d565c2b"},"source":["from sklearn.ensemble import RandomForestClassifier\n","\n","Random_Forest_model = None\n","accuracy = None\n","############# Put your code here ################\n","Random_Forest_model = RandomForestClassifier()\n","Random_Forest_model.fit(X_train,Y_train)\n","accuracy = Random_Forest_model.score(X_test,Y_test)\n","################################################# \n","print('Test data prediction accuracy (Random Forest) : %f'%accuracy)"],"execution_count":5,"outputs":[{"output_type":"stream","text":["Test data prediction accuracy (Random Forest) : 1.000000\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"iRNbhsIbuqUM"},"source":["# # Problem 3\n","XGBoost classifier.\n","- XGBoost 패키지를 pip install을 통해 설치하시오.\n","- 패키지를 불러와 모델을 학습시키고 test데이터의 정확도를 출력하시오."]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"zkbedcZhuqUM","executionInfo":{"status":"ok","timestamp":1619796857153,"user_tz":-540,"elapsed":6523,"user":{"displayName":"SungYub Jung","photoUrl":"","userId":"06395096588766753020"}},"outputId":"5a8f3705-a8ec-4655-d980-f8697e9e4480"},"source":["##########Put your installation code here###############\n","!pip install xgboost\n","########################################################"],"execution_count":6,"outputs":[{"output_type":"stream","text":["Requirement already satisfied: xgboost in /usr/local/lib/python3.7/dist-packages (0.90)\n","Requirement already satisfied: scipy in /usr/local/lib/python3.7/dist-packages (from xgboost) (1.4.1)\n","Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from xgboost) (1.19.5)\n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"CRXq_G2zuqUN","executionInfo":{"status":"ok","timestamp":1619796857864,"user_tz":-540,"elapsed":7232,"user":{"displayName":"SungYub Jung","photoUrl":"","userId":"06395096588766753020"}},"outputId":"73720e1a-5ff4-4920-81f0-8f9454d87a37"},"source":["import xgboost as xgb\n","\n","XGB_model = None\n","accuracy = None\n","############# Put your code here ################\n","XGB_model = xgb.XGBClassifier()\n","XGB_model.fit(X_train,Y_train)\n","accuracy = XGB_model.score(X_test,Y_test)\n","################################################# \n","print('Test data prediction accuracy (XGBoost) : %f'%accuracy)"],"execution_count":7,"outputs":[{"output_type":"stream","text":["Test data prediction accuracy (XGBoost) : 0.972222\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"WNVnYteMuqUN"},"source":["# # Problem 4\n","LightGBM classifier.\n","- LighGBM 패키지를 pip install을 통해 설치하시오.\n","- 패키지를 불러와 모델을 학습시키고 test데이터의 정확도를 출력하시오."]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"EnM4u28huqUN","executionInfo":{"status":"ok","timestamp":1619796860585,"user_tz":-540,"elapsed":9948,"user":{"displayName":"SungYub Jung","photoUrl":"","userId":"06395096588766753020"}},"outputId":"ef862352-237f-40e9-bb1f-e25c270f5d18"},"source":["##########Put your installation code here###############\n","!pip install lightgbm\n","########################################################"],"execution_count":8,"outputs":[{"output_type":"stream","text":["Requirement already satisfied: lightgbm in /usr/local/lib/python3.7/dist-packages (2.2.3)\n","Requirement already satisfied: scikit-learn in /usr/local/lib/python3.7/dist-packages (from lightgbm) (0.22.2.post1)\n","Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from lightgbm) (1.19.5)\n","Requirement already satisfied: scipy in /usr/local/lib/python3.7/dist-packages (from lightgbm) (1.4.1)\n","Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.7/dist-packages (from scikit-learn->lightgbm) (1.0.1)\n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"z-mcy6PYuqUN","executionInfo":{"status":"ok","timestamp":1619796860921,"user_tz":-540,"elapsed":10281,"user":{"displayName":"SungYub Jung","photoUrl":"","userId":"06395096588766753020"}},"outputId":"df692ff4-f904-498b-9b37-d3ca9785eeeb"},"source":["import lightgbm as lgb\n","\n","LGBM_model = None\n","accuracy = None\n","############# Put your code here ################\n","LGBM_model = lgb.LGBMClassifier()\n","LGBM_model.fit(X_train,Y_train)\n","accuracy = LGBM_model.score(X_test,Y_test)      \n","################################################# \n","print('Test data prediction accuracy (LightGBM) : %f'%accuracy)"],"execution_count":9,"outputs":[{"output_type":"stream","text":["Test data prediction accuracy (LightGBM) : 0.944444\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"n41N7zu0uqUN"},"source":["# # Problem 5\n","Naive Bayes classifier.\n","- Scikit-learn의 Gaussian Naive Bayes classifier를 불러와 학습시키고, test데이터의 정확도를 출력하시오."]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"SVdNAZ3fuqUO","executionInfo":{"status":"ok","timestamp":1619796860921,"user_tz":-540,"elapsed":10278,"user":{"displayName":"SungYub Jung","photoUrl":"","userId":"06395096588766753020"}},"outputId":"6847e3a0-593c-4d21-a03b-a6be7664e430"},"source":["from sklearn.naive_bayes import GaussianNB\n","\n","NB_model = None\n","accuracy = None\n","############# Put your code here ################\n","NB_model = GaussianNB()\n","NB_model.fit(X_train,Y_train)\n","accuracy = NB_model.score(X_test,Y_test) \n","################################################# \n","print('Test data prediction accuracy (NaiveBayes) : %f'%accuracy)"],"execution_count":10,"outputs":[{"output_type":"stream","text":["Test data prediction accuracy (NaiveBayes) : 0.972222\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"yInZqnVOuqUO"},"source":["# # Problem 6\n","SVM Bayes classifier.\n","- Scikit-learn의 SVM classifier를 불러와 학습시키고, test데이터의 정확도를 출력하시오."]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"5SiF8jIsuqUO","executionInfo":{"status":"ok","timestamp":1619796860921,"user_tz":-540,"elapsed":10276,"user":{"displayName":"SungYub Jung","photoUrl":"","userId":"06395096588766753020"}},"outputId":"9f7e2c70-3cb8-48c5-83aa-9d40ff676ce2"},"source":["from sklearn.svm import SVC\n","\n","SVM_model = None\n","accuracy = None\n","############# Put your code here ################\n","SVM_model = SVC()\n","SVM_model.fit(X_train,Y_train)\n","accuracy = SVM_model.score(X_test,Y_test) \n","#################################################\n","print('Test data prediction accuracy (SVC) : %f'%accuracy)"],"execution_count":11,"outputs":[{"output_type":"stream","text":["Test data prediction accuracy (SVC) : 0.694444\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"AOZ8srGruqUO"},"source":["# # Problem 7\n","MLP classifier.\n","- Scikit-learn의 MLP classifier를 불러와 학습시키고, test데이터의 정확도를 출력하시오."]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"AhQPkJEFuqUO","executionInfo":{"status":"ok","timestamp":1619796884483,"user_tz":-540,"elapsed":942,"user":{"displayName":"SungYub Jung","photoUrl":"","userId":"06395096588766753020"}},"outputId":"4b842a7d-9783-4bcb-f279-9b6efdc86eb3"},"source":["from sklearn.neural_network import MLPClassifier\n","\n","MLP_model = None\n","accuracy = None\n","############# Put your code here ################\n","MLP_model = MLPClassifier()\n","MLP_model.fit(X_train,Y_train)\n","accuracy = MLP_model.score(X_test,Y_test) \n","#################################################\n","print('Test data prediction accuracy (MLPClassifier) : %f'%accuracy)"],"execution_count":14,"outputs":[{"output_type":"stream","text":["Test data prediction accuracy (MLPClassifier) : 0.416667\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"anULz3vxuqUP"},"source":["# # Problem 8\n","Ensemble classifier.\n","- 위의 7개의 모델(CART, Random Forest, XGBoost, LightGBM, Naive Bayes, SVM, MLP)을 이용하여 Voting(7개의 예측 label중 가장 많이 예측된 label을 최종 prediction label로 택하는것)방법으로 최종 label을 택한 뒤, 예측 정확도를 출력하시오.\n","- Scikit-learn의 VotingClassifier 패키지를 사용하여 작성.\n","- 위에서 학습시킨 모델 변수명 사용 가능."]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"uUvo-sKNuqUP","executionInfo":{"status":"ok","timestamp":1619796890728,"user_tz":-540,"elapsed":1453,"user":{"displayName":"SungYub Jung","photoUrl":"","userId":"06395096588766753020"}},"outputId":"e5fb7754-d306-460f-d53a-81b086b222de"},"source":["from sklearn.ensemble import VotingClassifier\n","\n","ensemble_classifier = None\n","accuracy = None\n","############# Put your code here ################\n","ensemble_classifier = VotingClassifier(estimators = [('CART',CART_model),('Random Forest',Random_Forest_model),('XGBoost',XGB_model),('LightGBM',LGBM_model),('Naive Bayes',NB_model),('SVM',SVM_model),('MLP',MLP_model)])\n","ensemble_classifier.fit(X_train,Y_train)\n","accuracy = ensemble_classifier.score(X_test,Y_test) \n","################################################# \n","print('Test data prediction accuracy (Ensemble) : %f'%accuracy)"],"execution_count":15,"outputs":[{"output_type":"stream","text":["Test data prediction accuracy (Ensemble) : 0.972222\n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"id":"bBRNx1sB1jEj","executionInfo":{"status":"ok","timestamp":1619796861448,"user_tz":-540,"elapsed":10795,"user":{"displayName":"SungYub Jung","photoUrl":"","userId":"06395096588766753020"}}},"source":[""],"execution_count":13,"outputs":[]}]}