10 Steps to Become a Data Scientist.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "9f550639a0754f42a243e5785d895d24ba655515"
   },
   "source": [
    " ## <div align=\"center\">  10 Steps to Become a Data Scientist +20Q</div>\n",
    " <div align=\"center\">**quite practical and far from any theoretical concepts**</div>\n",
    "<div style=\"text-align:center\">last update: <b>15/01/2019</b></div>\n",
    "<img src=\"http://s9.picofile.com/file/8338833934/DS.png\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "e02d495da0fb0ad24e0341e91848f4c4cfc35bdb"
   },
   "source": [
    "\n",
    "\n",
    "---------------------------------------------------------------------\n",
    "Fork and Run this course on GitHub:\n",
    "> #### [ GitHub](https://github.com/mjbahmani/10-steps-to-become-a-data-scientist)\n",
    "\n",
    "\n",
    "-------------------------------------------------------------------------------------------------------------\n",
    " <b>I hope you find this kernel helpful and some <font color=\"red\"> UPVOTES</font> would be very much appreciated.</b>\n",
    " \n",
    " -----------\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "85b27cf82d3023fd69c338df2be7afb2d7afaf32"
   },
   "source": [
    " <a id=\"top\"></a> <br>\n",
    "**Notebook Content**\n",
    "\n",
    " [Introduction](#Introduction)\n",
    "1. [Python](#Python)\n",
    "1. [Python Packages](#PythonPackages)\n",
    "1. [Mathematics and Linear Algebra](#Algebra)\n",
    "1. [Programming & Analysis Tools](#Programming)\n",
    "1. [Big Data](#BigData)\n",
    "1. [Data visualization](#Datavisualization)\n",
    "1. [Data Cleaning](#DataCleaning)\n",
    "1. [How to solve Problem?](#Howto)\n",
    "1. [Machine Learning](#MachineLearning)\n",
    "1. [Deep Learning](#DeepLearning)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "2a77b410b99632c4d99b652c226178cb1ff10b51"
   },
   "source": [
    " <a id=\"Introduction\"></a> <br>\n",
    "# Introduction\n",
    "If you Read and Follow **Job Ads** to hire a machine learning expert or a data scientist, you find that some skills you should have to get the job. In this Kernel, I want to review **10 skills** that are essentials to get the job. In fact, this kernel is a reference for **10 other kernels**, which you can learn with them,  all of the skills that you need. \n",
    "\n",
    "We have used two well-known DataSets **Titanic** and **House prices** for starting but  when you learned python and python packages, you can start using other datasets too. \n",
    "\n",
    "**Ready to learn**! you will learn 10 skills as data scientist: \n",
    "\n",
    "1. [Learn Python](https://www.kaggle.com/mjbahmani/the-data-scientist-s-toolbox-tutorial-1)\n",
    "1. [Learn python packages](https://www.kaggle.com/mjbahmani/the-data-scientist-s-toolbox-tutorial-2) \n",
    "1. [Linear Algebra for Data Scientists](https://www.kaggle.com/mjbahmani/linear-algebra-for-data-scientists)\n",
    "1. [Programming & Analysis Tools](https://www.kaggle.com/mjbahmani/machine-learning-workflow-for-house-prices)\n",
    "1. [Big Data](https://www.kaggle.com/mjbahmani/a-data-science-framework-for-quora)\n",
    "1. [Top 5 Data Visualization Libraries Tutorial](https://www.kaggle.com/mjbahmani/top-5-data-visualization-libraries-tutorial)\n",
    "1. [How to solve Problem?](https://www.kaggle.com/mjbahmani/a-data-science-framework-for-quora)\n",
    "1. [Data Cleaning](https://www.kaggle.com/mjbahmani/some-eda-for-elo)\n",
    "1. [Machine Learning](https://www.kaggle.com/mjbahmani/a-comprehensive-ml-workflow-with-python)\n",
    "1. [Deep Learning](https://www.kaggle.com/mjbahmani/top-5-deep-learning-frameworks-tutorial) \n",
    "\n",
    "Thanks to **Kaggle team** due to provide a great professional community for Data Scientists.\n",
    "###### [go to top](#top)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "5efeff35ad9951e40551d0763eaf26f08bb4119e"
   },
   "source": [
    " <a id=\"1\"></a> <br>\n",
    "# 1-Python\n",
    "The first step in this course for beginners is python learning. Just takes **10 hours** to learn python.\n",
    "\n",
    "for reading this section **please** fork and run  the following kernel:\n",
    "\n",
    "[Learn Python](https://www.kaggle.com/mjbahmani/the-data-scientist-s-toolbox-tutorial-1)\n",
    " \n",
    "  ###### [go to top](#top)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "1a8697f93952e076f6f949997676d40518d7b5a6"
   },
   "source": [
    "<a id=\"PythonPackages\"></a> <br>\n",
    "# 2-Python Packages\n",
    "In the second step, we will learn  the necessary libraries that are essential for any data scietist.\n",
    "1. Numpy\n",
    "1. Pandas\n",
    "1. Matplotlib\n",
    "1. Seaborn\n",
    "1. TensorFlow\n",
    "1. NLTK\n",
    "1. Sklearn\n",
    "and so on\n",
    "\n",
    "<img src=\"http://s8.picofile.com/file/8338227868/packages.png\">\n",
    "\n",
    "for Reading this section **please** fork and run  this kernel:\n",
    "\n",
    "\n",
    "\n",
    "1. [The data scientist's toolbox tutorial 1](https://www.kaggle.com/mjbahmani/the-data-scientist-s-toolbox-tutorial-1)\n",
    "\n",
    "1. [The data scientist's toolbox tutorial 2](https://www.kaggle.com/mjbahmani/the-data-scientist-s-toolbox-tutorial-2)\n",
    "\n",
    "###### [go to top](#top)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "ad8fa54ba57aa4a336080eb044109702c743d7a0"
   },
   "source": [
    "<a id=\"Algebra\"></a> <br>\n",
    "##  3- Mathematics and Linear Algebra\n",
    "Linear algebra is the branch of mathematics that deals with vector spaces. good understanding of Linear Algebra is intrinsic to analyze Machine Learning algorithms, especially for Deep Learning where so much happens behind the curtain.you have my word that I will try to keep mathematical formulas & derivations out of this completely mathematical topic and I try to cover all of subject that you need as data scientist.\n",
    "\n",
    "<img src=\" https://s3.amazonaws.com/www.mathnasium.com/upload/824/images/algebra.jpg \" height=\"300\" width=\"300\">\n",
    "\n",
    "for Reading this section **please** fork and run  this kernel:\n",
    "\n",
    "[Linear Algebra for Data Scientists](https://www.kaggle.com/mjbahmani/linear-algebra-for-data-scientists)\n",
    "###### [go to top](#top)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "697ba206ad7adf4d99814cb1d89375b745eaba19"
   },
   "source": [
    "<a id=\"Programming\"></a> <br>\n",
    "## 4- Programming & Analysis Tools\n",
    "\n",
    "This section is not completed yet but for reading an alternative for it **please** fork and run  this kernel:\n",
    "\n",
    "[Programming & Analysis Tools](https://www.kaggle.com/mjbahmani/machine-learning-workflow-for-house-prices)\n",
    "\n",
    "###### [go to top](#top)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "00f5c5ce80c7e302e83f0ea9b451dfaae7aa52cf"
   },
   "source": [
    "<a id=\"BigData\"></a> <br>\n",
    "## 5- Big Data\n",
    "\n",
    "This section is not completed yet but for reading an alternative for it **please** fork and run  this kernel:\n",
    "\n",
    "[Big Data](https://www.kaggle.com/mjbahmani/a-data-science-framework-for-quora)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "33bb9c265bef5e4474dcac0638cc632b5532f1ce"
   },
   "source": [
    "<a id=\"Datavisualization\"></a> <br>\n",
    "## 6- Data Visualization\n",
    "For Reading this section **please** fork and upvote  this kernel:\n",
    "\n",
    "[Top 5 Data Visualization Libraries Tutorial](https://www.kaggle.com/mjbahmani/top-5-data-visualization-libraries-tutorial)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "9bf1d9444651e2756c4fa4d71914ec20d621305e"
   },
   "source": [
    "<a id=\"DataCleaning\"></a> <br>\n",
    "## 7- Data Cleaning\n",
    "Certainly another important step in the way of specialization is learning how to clean the data.\n",
    "In this section, we will do this on Elo data set.\n",
    "for Reading this section **please** fork and upvote  this kernel:\n",
    "\n",
    "[Data Cleaning](https://www.kaggle.com/mjbahmani/some-eda-for-elo)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "8720a4ddaab64e4bff226bed9e4e200dc9b94913"
   },
   "source": [
    "<a id=\"Howto\"></a> <br>\n",
    "## 8- How to Solve Problem?\n",
    "If you have already read some [machine learning books](https://github.com/mjbahmani/10-steps-to-become-a-data-scientist/tree/master/Ebooks). You have noticed that there are different ways to stream data into machine learning.\n",
    "\n",
    "most of these books share the following steps (checklist):\n",
    "*   Define the Problem(Look at the big picture)\n",
    "*   Specify Inputs & Outputs\n",
    "*   Data Collection\n",
    "*   Exploratory data analysis\n",
    "*   Data Preprocessing\n",
    "*   Model Design, Training, and Offline Evaluation\n",
    "*   Model Deployment, Online Evaluation, and Monitoring\n",
    "*   Model Maintenance, Diagnosis, and Retraining\n",
    "\n",
    "**You can see my workflow in the below image** :\n",
    " <img src=\"http://s9.picofile.com/file/8338227634/workflow.png\" />\n",
    "## 8-1 Real world Application Vs Competitions\n",
    "Just a simple comparison between real-world apps with competitions:\n",
    "<img src=\"http://s9.picofile.com/file/8339956300/reallife.png\" height=\"600\" width=\"500\" />\n",
    "**you should\tfeel free\tto\tadapt \tthis\tchecklist \tto\tyour needs**\n",
    " \n",
    "## 8-2 Problem Definition\n",
    "I think one of the important things when you start a new machine learning project is Defining your problem. that means you should understand business problem.( **Problem Formalization**)\n",
    "\n",
    "Problem Definition has four steps that have illustrated in the picture below:\n",
    "<img src=\"http://s8.picofile.com/file/8338227734/ProblemDefination.png\">\n",
    " \n",
    "### 8-2-1 Problem Feature\n",
    "The sinking of the Titanic is one of the most infamous shipwrecks in history. **On April 15, 1912**, during her maiden voyage, the Titanic sank after colliding with an iceberg, killing **1502 out of 2224** passengers and crew. That's why the name DieTanic. This is a very unforgetable disaster that no one in the world can forget.\n",
    "\n",
    "It took about $7.5 million to build the Titanic and it sunk under the ocean due to collision. The Titanic Dataset is a very good dataset for begineers to start a journey in data science and participate in competitions in Kaggle.\n",
    "\n",
    "we will use the classic titanic data set. This dataset contains information about **11 different variables**:\n",
    "<img src=\"http://s9.picofile.com/file/8340453092/Titanic_feature.png\" height=\"500\" width=\"500\">\n",
    "\n",
    "* Survival\n",
    "* Pclass\n",
    "* Name\n",
    "* Sex\n",
    "* Age\n",
    "* SibSp\n",
    "* Parch\n",
    "* Ticket\n",
    "* Fare\n",
    "* Cabin\n",
    "* Embarked\n",
    "\n",
    "<font color='red'><b>Question</b></font>\n",
    "1. It's your train what's House Price Data sets feature?\n",
    "\n",
    "### 8-2-2 Aim\n",
    "\n",
    "It is your job to predict if a passenger survived the sinking of the Titanic or not.  For each PassengerId in the test set, you must predict a 0 or 1 value for the Survived variable.\n",
    "\n",
    " \n",
    "### 8-2-3 Variables\n",
    "\n",
    "1.  **Age** ==>> Age is fractional if less than 1. If the age is estimated, is it in the form of xx.5\n",
    "\n",
    "2. **Sibsp** ==>> The dataset defines family relations in this way...\n",
    "\n",
    "    a. Sibling = brother, sister, stepbrother, stepsister\n",
    "\n",
    "    b. Spouse = husband, wife (mistresses and fiancés were ignored)\n",
    "\n",
    "3. **Parch** ==>> The dataset defines family relations in this way...\n",
    "\n",
    "    a. Parent = mother, father\n",
    "\n",
    "    b. Child = daughter, son, stepdaughter, stepson\n",
    "\n",
    "    c. Some children travelled only with a nanny, therefore parch=0 for them.\n",
    "\n",
    "4. **Pclass** ==>> A proxy for socio-economic status (SES)\n",
    "\n",
    "    * 1st = Upper\n",
    "    * 2nd = Middle\n",
    "    * 3rd = Lower\n",
    "    \n",
    "5. **Embarked** ==>> nominal datatype \n",
    "6. **Name** ==>> nominal datatype . It could be used in feature engineering to derive the gender from title\n",
    "7. **Sex** ==>>  nominal datatype \n",
    "8. **Ticket** ==>> that have no impact on the outcome variable. Thus, they will be excluded from analysis\n",
    "9. **Cabin** ==>>  is a nominal datatype that can be used in feature engineering\n",
    "11. **Fare** ==>>  Indicating the fare\n",
    "12. **PassengerID ** ==>> have no impact on the outcome variable. Thus, it will be excluded from analysis\n",
    "11. **Survival** is ==>> **[dependent variable](http://www.dailysmarty.com/posts/difference-between-independent-and-dependent-variables-in-machine-learning)** , 0 or 1\n",
    "\n",
    "\n",
    "**<< Note >>**\n",
    "\n",
    "> You must answer the following question:\n",
    "How does your company expact to use and benfit from your model.\n",
    "\n",
    "for Reading this section **please** fork and upvote  this kernel:\n",
    "\n",
    "[How to solve Problem?](https://www.kaggle.com/mjbahmani/a-data-science-framework-for-quora)\n",
    "###### [Go to top](#top)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "d4f8718cc7e1a8fc60a3815b55a2ab9a5eeef4f9"
   },
   "source": [
    "<a id=\"MachineLearning\"></a> <br>\n",
    "## 9- Machine learning  \n",
    "for reading this section **please** fork and upvote  this kernel:\n",
    "\n",
    "[A Comprehensive ML Workflow with Python](https://www.kaggle.com/mjbahmani/a-comprehensive-ml-workflow-with-python)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "3544d2fd1490f646f2f1c0fd4271f9a8745d2e36"
   },
   "source": [
    "<a id=\"DeepLearning\"></a> <br>\n",
    "##  10- Deep Learning\n",
    "\n",
    "for Reading this section **please** fork and upvote  this kernel:\n",
    "\n",
    "[A-Comprehensive-Deep-Learning-Workflow-with-Python](https://www.kaggle.com/mjbahmani/a-comprehensive-deep-learning-workflow-with-python)\n",
    "\n",
    "---------------------------\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "b49f503c61c5763f8f46ef10e115f6ffd808bae4"
   },
   "source": [
    "#  <div align=\"center\">  50 machine learning  questions & answers for Beginners </div>\n",
    "If you are studying this kernel, you're probably at beginning of this journey. Here are some useful Python codes you need to get started."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "_kg_hide-input": true,
    "_uuid": "0e4b7ea75492288738d63fd487dd5665aee4a945"
   },
   "outputs": [],
   "source": [
    "import matplotlib.animation as animation\n",
    "from matplotlib.figure import Figure\n",
    "import plotly.figure_factory as ff\n",
    "import matplotlib.pylab as pylab\n",
    "from ipywidgets import interact\n",
    "import plotly.graph_objs as go\n",
    "import plotly.offline as py\n",
    "from random import randint\n",
    "from plotly import tools\n",
    "import seaborn as sns\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib\n",
    "import warnings\n",
    "import string\n",
    "import numpy\n",
    "import csv\n",
    "import os"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "ffb6edf54635217f41255c7cf1d4eaef5cb33a74"
   },
   "source": [
    "## 1-how to import your data?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "9459b02f920bafd3c0a1ce3e516cedfd86fc3806"
   },
   "source": [
    "What you have in your data folder?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "_uuid": "2d230abd1d2b1e638a535c06380caa1def7b5abf"
   },
   "outputs": [],
   "source": [
    "print(os.listdir(\"../input/\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "2fa91020db4f946f3eb27235b25a21133eea369a"
   },
   "source": [
    "import all of your data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "_uuid": "1b75fd89050e48450c6a08ae495e0d7607ecfb23"
   },
   "outputs": [],
   "source": [
    "titanic_train=pd.read_csv('../input/train.csv')\n",
    "titanic_test=pd.read_csv('../input/test.csv')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "e5890dcd5c76c05fd532170c737986d2c5030e68"
   },
   "source": [
    "Or import just %10 of your data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "_uuid": "ff07b0f1a5bba6e27e41051f8b7f03a744427f68"
   },
   "outputs": [],
   "source": [
    "titanic_train2=pd.read_csv('../input/train.csv',nrows=1000)\n",
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "cf8a6ef896748398d8791312b52bfd0d99c2e875"
   },
   "source": [
    "How to see the size of your data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "_uuid": "659775fa6159a23d5e1b5e3c6ab3ceda05fe1149"
   },
   "outputs": [],
   "source": [
    "print(\"Train: rows:{} columns:{}\".format(titanic_train.shape[0], titanic_train.shape[1]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "923af15ecbadcbca2ee91b7cd5a1e903c71e454c"
   },
   "source": [
    "**For reading more about how to import your data you can visit: **[this Kernel](https://www.kaggle.com/dansbecker/finding-your-files-in-kaggle-kernels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "e59033f12042184182dd1df791f0da04e50a5511"
   },
   "source": [
    "## 2- How to check missed data?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "_uuid": "df10c29fd5d5d14ed7a3cea2e52e97b4c61d0766"
   },
   "outputs": [],
   "source": [
    "titanic_train.isna().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "b35a48a3e0a4bc4c2883df2f4dac18563ea89ecb"
   },
   "source": [
    "or you can use below code"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "_uuid": "aa78be75200487a5175cb94e6a7e74bb55707155"
   },
   "outputs": [],
   "source": [
    "total = titanic_train.isnull().sum().sort_values(ascending=False)\n",
    "percent = (titanic_train.isnull().sum()/titanic_train.isnull().count()).sort_values(ascending=False)\n",
    "missing_data = pd.concat([total, percent], axis=1, keys=['Total', 'Percent'])\n",
    "missing_data.head(20)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "bf0579ada55ee69eb3082a87f74afad711764358"
   },
   "source": [
    "## 3- How to view the statistical characteristics of the data?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "_uuid": "450ef7914220ca0fad004a561595ea0d8d98cd6d"
   },
   "outputs": [],
   "source": [
    "titanic_train.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "23954ce7b3166c58a9e121b42883d8f2382cc156"
   },
   "source": [
    "or just for one column"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "_uuid": "35094dbeff8dfecca5307b74203cfac6e37b1837"
   },
   "outputs": [],
   "source": [
    "titanic_train['Age'].describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "536a58050ceae5e9e4f3dd155d0a1a33f1d1700d"
   },
   "source": [
    "with a another shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "_uuid": "4e3d0803b3ffc457946c206e5f7d4bd2367c7109"
   },
   "outputs": [],
   "source": [
    "titanic_train.Age.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "43fb867cf20ce5a424ee1574aca0e84f7d15f0ec"
   },
   "source": [
    "## 4- How check the column's name?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "_uuid": "14d5ea4d91ffee1c63a358276e9b6897718ecde7"
   },
   "outputs": [],
   "source": [
    "titanic_train.columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "b605639fae498c4f159649cea103dea5ac515950"
   },
   "source": [
    "or you can the check the column name with another ways too"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "_uuid": "e9ed54f66abf8e61762415222a8fc72e5429b31f"
   },
   "outputs": [],
   "source": [
    "titanic_train.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "1d59ce0b7038ecc0709ea125b6343d07dbf4ebaf"
   },
   "source": [
    "## 5- how to view randomly your data set ?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "_uuid": "05bb29269315f38a961db2b93ce67e5e145645e9"
   },
   "outputs": [],
   "source": [
    "titanic_train.sample(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "1720cd820761133563be081702ab70297613eb5d"
   },
   "source": [
    "## 6-How  random row selection in Pandas dataframe?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "_uuid": "5d605f1460feaa4796306a5bd4dd73f4907061de"
   },
   "outputs": [],
   "source": [
    "titanic_train.sample(frac=0.007)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "a3205cc9eaeb7fb78b19a8ddd56fc2444407f5f5"
   },
   "source": [
    "## 7- How to copy a column and drop it ?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "_uuid": "ab6b884f287bf1f188b9884730e0867f829882e3"
   },
   "outputs": [],
   "source": [
    "PassengerId=titanic_train['PassengerId'].copy()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "_uuid": "20a88f1cf2e33a1149383d9d3f88aab714f070d8"
   },
   "outputs": [],
   "source": [
    "PassengerId.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "_uuid": "149e3ac42f577b1d0ba18adf6a85ed3fc4977585"
   },
   "outputs": [],
   "source": [
    "type(PassengerId)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "_uuid": "e0afa45e89306282a9bca9b9b428f81063758c36"
   },
   "outputs": [],
   "source": [
    "titanic_train=titanic_train.drop('PassengerId',1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "_uuid": "41246d7270f3db045f5aacea992edce3c45444f2"
   },
   "outputs": [],
   "source": [
    "titanic_train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "_uuid": "1f6b167d47cfbee1b5a3fe37d4e1b089fc09b688"
   },
   "outputs": [],
   "source": [
    "titanic_train=pd.read_csv('../input/train.csv')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "d48e8df93e98d36afebf0cb01ccbb3c1f55c9902"
   },
   "source": [
    "## 8- How to check out last 5 row of the dataset?\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "705583dde374fa2ceadf488242bb08d36c885e1e"
   },
   "source": [
    "we use tail() function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "_uuid": "32c07ff14def9c8b16b0feb178f7a4b3ddc853c1"
   },
   "outputs": [],
   "source": [
    "titanic_train.tail() "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "369652d95b5399f70ca6487c552283af41d00d45"
   },
   "source": [
    "\n",
    "# 9- How to concatenation operations along an axis?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "_uuid": "b44ae4b87d92b110a4d9bdb464ed20bbef47004b"
   },
   "outputs": [],
   "source": [
    "all_data = pd.concat((titanic_train.loc[:,'Pclass':'Embarked'],\n",
    "                      titanic_test.loc[:,'Pclass':'Embarked']))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "_uuid": "9cdcc6f188a908c275fbdbea2c038ec28843db35"
   },
   "outputs": [],
   "source": [
    "all_data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "_uuid": "d5740b3e180eca78034ec0810f425e5e79bf10bf"
   },
   "outputs": [],
   "source": [
    "titanic_train.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "_uuid": "906c205c9f8ab8ccddfb75cab53b86e09413f718"
   },
   "outputs": [],
   "source": [
    "titanic_test.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "_uuid": "9a36ac64cf2bc9ac5c815bc0f3e42e8b3ee7896c"
   },
   "outputs": [],
   "source": [
    "all_data.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "e454c7b0970a332f6107766e9231c9fa1234f5e5"
   },
   "source": [
    "## 10- How to see unique values for a culomns?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "_uuid": "af962f4944c2781147afde7cb0ae09b8c9b035dc"
   },
   "outputs": [],
   "source": [
    "titanic_train['Sex'].unique()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "_uuid": "86085674515bbc28056fd77ed200ca5ff8de6376"
   },
   "outputs": [],
   "source": [
    "titanic_train['Cabin'].unique()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "_uuid": "a5f56d9175fc549e342812470c574becd54b4300"
   },
   "outputs": [],
   "source": [
    "titanic_train['Pclass'].unique()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "301884e424d34ee8c601b392e68feeef1f6c55cb"
   },
   "source": [
    "## 11- How to perform some query on your datasets?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "_uuid": "9c7b548593117af9091738ef8421306a5e5fd47d"
   },
   "outputs": [],
   "source": [
    "titanic_train[titanic_train['Age']>70]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "_uuid": "e5c37ba74d228f3f8392f724e5696f858680b866"
   },
   "outputs": [],
   "source": [
    "titanic_train[titanic_train['Pclass']==1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "edb768e0b3390ec29acab20593948c3f3bbf5bba",
    "collapsed": true
   },
   "source": [
    "---------------------------------------------------------------------\n",
    "Fork and Run this kernel on GitHub:\n",
    "> ###### [ GitHub](https://github.com/mjbahmani/10-steps-to-become-a-data-scientist)\n",
    "\n",
    " \n",
    "\n",
    "-------------------------------------------------------------------------------------------------------------\n",
    " <b>I hope you find this kernel helpful and some <font color=\"red\">UPVOTES</font> would be very much appreciated</b>\n",
    " \n",
    " -----------"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}