Sparkify.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1. Sparkify Project Workspace\n",
    "\n",
    "This is done based on the analysis with Pandas. So some detailed steps are ommited here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# import libraries\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt \n",
    "% matplotlib inline\n",
    "import seaborn as sns\n",
    "\n",
    "import pyspark\n",
    "from pyspark import SparkConf, SparkContext\n",
    "from pyspark.sql import SparkSession\n",
    "from pyspark.sql import Row\n",
    "from pyspark.sql.functions import udf\n",
    "from pyspark.sql.types import *\n",
    "from pyspark.sql import functions as F\n",
    "\n",
    "import datetime"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('spark.master', 'local'),\n",
       " ('spark.app.id', 'local-1614466796548'),\n",
       " ('spark.app.name', 'sparkify'),\n",
       " ('spark.driver.port', '33655'),\n",
       " ('spark.rdd.compress', 'True'),\n",
       " ('spark.serializer.objectStreamReset', '100'),\n",
       " ('spark.driver.host', 'e8b259294fcb'),\n",
       " ('spark.executor.id', 'driver'),\n",
       " ('spark.submit.deployMode', 'client'),\n",
       " ('spark.ui.showConsoleProgress', 'true')]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# create a Spark session\n",
    "configure = SparkConf().setAppName(\"sparkify\").setMaster(\"local\")\n",
    "sc = SparkContext(conf = configure)\n",
    "spark = SparkSession.builder.appName('sparkify').getOrCreate()\n",
    "spark.sparkContext.getConf().getAll()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2. Load and Clean Dataset\n",
    "In this workspace, the mini-dataset file is `mini_sparkify_event_data.json`. Load and clean the dataset, checking for invalid or missing data - for example, records without userids or sessionids. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Row(artist='Martha Tilston', auth='Logged In', firstName='Colin', gender='M', itemInSession=50, lastName='Freeman', length=277.89016, level='paid', location='Bakersfield, CA', method='PUT', page='NextSong', registration=1538173362000, sessionId=29, song='Rockpools', status=200, ts=1538352117000, userAgent='Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Firefox/31.0', userId='30')]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "path = 'mini_sparkify_event_data.json'\n",
    "df_total = spark.read.json(path)\n",
    "df_total.take(1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_total_backup = df_total"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.1 Remove undesired columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Following columns are removed:\n",
    "\n",
    "- **`artist`**, **`song`** and **`location`**\n",
    "\n",
    "    The behaviour of a R&B fan and a classic music lover can certainly be different. But considering this variance requirs a    database with labeled artists and songs, which is beyond the scope of this project. \n",
    "\n",
    "    Similarly, user from different locations may also behaviour differenetly related with for example the local personal income level. This feature is also left for investigation in future.  \n",
    "    \n",
    "\n",
    "- **`firstName`** and **`lastName`**\n",
    "\n",
    "    They are not related with user behaviour.\n",
    "    \n",
    "\n",
    "- **`itemInSession`**, **`method`**, **`sessionId`**, **`status`**\n",
    "\n",
    "    These parameters are simply technical parameters. They are not related with user behaviour."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_total = df_total.drop('artist','song','location','firstName','lastName','itemInSession','method','sessionId','status')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Row(auth='Logged In', gender='M', length=277.89016, level='paid', page='NextSong', registration=1538173362000, ts=1538352117000, userAgent='Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Firefox/31.0', userId='30')]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_total.take(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.2 Understanding and clean the data "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2.1 Overview"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "root\n",
      " |-- auth: string (nullable = true)\n",
      " |-- gender: string (nullable = true)\n",
      " |-- length: double (nullable = true)\n",
      " |-- level: string (nullable = true)\n",
      " |-- page: string (nullable = true)\n",
      " |-- registration: long (nullable = true)\n",
      " |-- ts: long (nullable = true)\n",
      " |-- userAgent: string (nullable = true)\n",
      " |-- userId: string (nullable = true)\n",
      "\n"
     ]
    }
   ],
   "source": [
    "df_total.printSchema()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Row(summary='count', auth='286500', gender='278154', length='228108', level='286500', page='286500', registration='278154', ts='286500', userAgent='278154', userId='286500'),\n",
       " Row(summary='mean', auth=None, gender=None, length='249.1171819778458', level=None, page=None, registration='1.5353588340844272E12', ts='1.5409568898104834E12', userAgent=None, userId='59682.02278593872'),\n",
       " Row(summary='stddev', auth=None, gender=None, length='99.23517921058361', level=None, page=None, registration='3.291321616327586E9', ts='1.5075439608226302E9', userAgent=None, userId='109091.9499991047'),\n",
       " Row(summary='min', auth='Cancelled', gender='F', length='0.78322', level='free', page='About', registration='1521380675000', ts='1538352117000', userAgent='\"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10) AppleWebKit/600.1.3 (KHTML, like Gecko) Version/8.0 Safari/600.1.3\"', userId=''),\n",
       " Row(summary='max', auth='Logged Out', gender='M', length='3024.66567', level='paid', page='Upgrade', registration='1543247354000', ts='1543799476000', userAgent='Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)', userId='99')]"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_total.describe().collect()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "8346\n",
      "58392\n",
      "8346\n",
      "8346\n"
     ]
    }
   ],
   "source": [
    "print(df_total.where(df_total.gender.isNull()).count())\n",
    "print(df_total.where(df_total.length.isNull()).count())\n",
    "print(df_total.where(df_total.registration.isNull()).count())\n",
    "print(df_total.where(df_total.userAgent.isNull()).count())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "They are all referring to unregistered users, see the results below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "8346"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_total[df_total.userId == ''].count()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Those columns are removed. Statistics of the dataframe with remaining columns is shown below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_total = df_total[df_total.userId != '']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2.3 Clean Column `auth`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Unique values of `auth`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_total = df_total.drop('auth')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2.4. Column `gender`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Row(gender='F'), Row(gender='M')]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# check against null \n",
    "df_total.select('gender').distinct().collect()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There is no empty value. It will be transformed to numeric value using OneHotEnconder later."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2.5 Clean Column `length`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When not visiting page 'Next song', the value `length` is null. These nulls are as expected, so they are \"right\" nulls, and do not require removing or imputing. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+--------------------+-----+\n",
      "|                page|count|\n",
      "+--------------------+-----+\n",
      "|              Cancel|   52|\n",
      "|    Submit Downgrade|   63|\n",
      "|         Thumbs Down| 2546|\n",
      "|                Home|10082|\n",
      "|           Downgrade| 2055|\n",
      "|         Roll Advert| 3933|\n",
      "|              Logout| 3226|\n",
      "|       Save Settings|  310|\n",
      "|Cancellation Conf...|   52|\n",
      "|               About|  495|\n",
      "|            Settings| 1514|\n",
      "|     Add to Playlist| 6526|\n",
      "|          Add Friend| 4277|\n",
      "|           Thumbs Up|12551|\n",
      "|                Help| 1454|\n",
      "|             Upgrade|  499|\n",
      "|               Error|  252|\n",
      "|      Submit Upgrade|  159|\n",
      "+--------------------+-----+\n",
      "\n"
     ]
    }
   ],
   "source": [
    "df_total.select('length','page').where(df_total.length.isNull()).groupBy('page').count().show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2.6 Clean Column `level`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# check against null\n",
    "df_total.where(df_total.level.isNull()).count()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This column has no nulls now. It is used to generate the UBC."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2.7 Clean column `page`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Unique values:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['Cancel',\n",
       " 'Submit Downgrade',\n",
       " 'Thumbs Down',\n",
       " 'Home',\n",
       " 'Downgrade',\n",
       " 'Roll Advert',\n",
       " 'Logout',\n",
       " 'Save Settings',\n",
       " 'Cancellation Confirmation',\n",
       " 'About',\n",
       " 'Settings',\n",
       " 'Add to Playlist',\n",
       " 'Add Friend',\n",
       " 'NextSong',\n",
       " 'Thumbs Up',\n",
       " 'Help',\n",
       " 'Upgrade',\n",
       " 'Error',\n",
       " 'Submit Upgrade']"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list(df_total.select('page').distinct().toPandas().page.values)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2.8 clean column `registration`, 2.2.9 clean column `ts`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I leave these two columns as they are to save computation cost. After all, I only care about the difference. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2.10 clean column `userAgent`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I devide the values into three groups:\n",
    "\n",
    "- when it contains 'Windows', then it is category w\n",
    "- when it contains 'Linux', then it is category l\n",
    "- the rest are category m (ios, mac etc.)\n",
    "\n",
    "For this purpose, a function is defined: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "function = udf(lambda x: 'w' if 'Windows' in x else 'l' if 'Linux' in x else 'm')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_total = df_total.withColumn('userAgentNew', function('userAgent'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+------+---------+-----+--------+-------------+-------------+--------------------+------+------------+\n",
      "|gender|   length|level|    page| registration|           ts|           userAgent|userId|userAgentNew|\n",
      "+------+---------+-----+--------+-------------+-------------+--------------------+------+------------+\n",
      "|     M|277.89016| paid|NextSong|1538173362000|1538352117000|Mozilla/5.0 (Wind...|    30|           w|\n",
      "+------+---------+-----+--------+-------------+-------------+--------------------+------+------------+\n",
      "only showing top 1 row\n",
      "\n"
     ]
    }
   ],
   "source": [
    "df_total.show(1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+------------+\n",
      "|userAgentNew|\n",
      "+------------+\n",
      "|           l|\n",
      "|           m|\n",
      "|           w|\n",
      "+------------+\n",
      "\n"
     ]
    }
   ],
   "source": [
    "df_total = df_total.drop('userAgent')\n",
    "df_total.select('userAgentNew').distinct().show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2.11 clean column `userId`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Nothing need to be done."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3. Feature Engineering"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.1 Generate user behaviour capsule (UBC) and new features"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Churn can be identified by the column `level`, which shows the current level of the user (whether 'paid' or 'free'), and the column `page`, which shows the pages the user visits. If a paid user visits 'Submit Downgrade' or 'Cancellation Confirmation', that means the users has churned. \n",
    "\n",
    "As discussed above, the investigation here focus on the categorize following two groups of users:\n",
    "- 1. paided user and churn the service \n",
    "- 2. paided user and does not churn the service\n",
    "\n",
    "For this purpose, I define the so-called \"user behaviour capsule\" for this two groups of users as below:\n",
    "- 1. from the first row of level \"paid\" to the first row of page 'Submit Downgrade' or 'Cancellation Confirmation'\n",
    "- 2. from the first row of level \"paid\" to the end of the data without churn\n",
    "\n",
    "The bahaviour of free user and their upgrade activities are not the focus in this project. For the users who have downgrades the service several times, only the first time is considered. \n",
    "\n",
    "To generate UBC, a function is defined below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "# sort the data according to `userId` and `ts`\n",
    "df_total = df_total.sort(['userId','ts'])\n",
    "\n",
    "# Generate the list of userIds\n",
    "\n",
    "userId_list = list(df_total.select('userId').distinct().toPandas().userId.values)\n",
    "\n",
    "\n",
    "# columns of the old data frame \n",
    "all_col = ['gender','length','level','page','registration','ts','userAgentNew','userId']\n",
    "\n",
    "# columns of the to generate data frame \n",
    "new_col = ['userId','PlayedTime','PaidTime','PlayNumber','PosEva','NegEva','Gender','Device','y']\n",
    "\n",
    "# schema of the to generate data frame \n",
    "schema = StructType([\n",
    "    StructField('userId', StringType(), True),\n",
    "    StructField('PlayedTime',IntegerType(), True),\n",
    "    StructField('PaidTime',IntegerType(), True), \n",
    "    StructField('PlayNumber',IntegerType(), True),\n",
    "    StructField('PosEva',IntegerType(), True),\n",
    "    StructField('NegEva',IntegerType(), True), \n",
    "    StructField('Gender',StringType(), True),\n",
    "    StructField('Device',StringType(),True),\n",
    "    StructField('y',IntegerType(),True)\n",
    "])\n",
    "\n",
    "df_ubc = spark.createDataFrame(data = '', schema = schema ).toDF(*new_col) # the empty dataframe to store the data\n",
    "df_ubc_list = [] # the empty list used to store data "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Due to the disconnection setting of Udacity, I have to perform the calculation in two steps\n",
    "\n",
    "userId_length = len(userId_list)\n",
    "for userId in userId_list[0:int(userId_length/2)]:\n",
    "    \n",
    "    value_temp = []\n",
    "\n",
    "    stay_user = 0\n",
    "\n",
    "    df_temp_paid = df_total.select(*all_col).where((df_total.level == 'paid') & (df_total.userId == userId))\n",
    "\n",
    "    if len(df_temp_paid.head(1)) == 0:\n",
    "        # print(\"Free users\")\n",
    "        pass\n",
    "\n",
    "    else: \n",
    "        first_paid_time = df_temp_paid.take(1)[0].__getitem__('ts')\n",
    "\n",
    "        # NO Free users \n",
    "        churn_time = df_temp_paid.select(\"ts\").where((df_temp_paid.level == 'paid') & (df_temp_paid.page == 'Cancellation Confirmation')|(df_temp_paid.level == 'Submit Downgrade'))\n",
    "\n",
    "        if churn_time.head(1) == []:\n",
    "            # stay paid user \n",
    "            stay_user = 1\n",
    "            first_churn_time = df_temp_paid.select('ts').groupby().max().collect()[0].__getitem__('max(ts)')\n",
    "\n",
    "\n",
    "        else:\n",
    "            # churn paid user \n",
    "            first_churn_time = churn_time.take(1)[0].__getitem__('ts')\n",
    "\n",
    "        df_final = df_temp_paid.select(*all_col).where((df_temp_paid.ts >= first_paid_time) & (df_temp_paid.ts <= first_churn_time))\n",
    "\n",
    "        # add userId \n",
    "        value_temp.append(userId)\n",
    "\n",
    "        # add new column total played time \n",
    "        try:\n",
    "            value_temp.append(int(df_final.select('length').groupby().sum().collect()[0].__getitem__('sum(length)')))\n",
    "        except:\n",
    "            value_temp.append(None)\n",
    "\n",
    "        # total paid time   \n",
    "        try:\n",
    "            value_temp.append(int((df_final.select('ts').groupby().max().collect()[0].__getitem__('max(ts)')\\\n",
    "                                -df_final.select('registration').collect()[0].__getitem__('registration'))/3600/1000))\n",
    "        except:\n",
    "            value_temp.append(None)\n",
    "\n",
    "        # total played number\n",
    "        try:\n",
    "            value_temp.append(int(df_final.select('page').where(df_final.page == 'NextSong').groupby().count().collect()[0]\\\n",
    "                                .__getitem__('count')))\n",
    "        except:\n",
    "            value_temp.append(None)\n",
    "            \n",
    "        # add to list and thumb up\n",
    "        try: \n",
    "            value_temp.append(int(df_final.select('page').where((df_final.page == 'Add to Playlist')\\\n",
    "                                                            |(df_final.page == 'Thumbs Up')).\\\n",
    "                                groupby().count().collect()[0].__getitem__('count')))        \n",
    "        except: \n",
    "            value_temp.append(None)\n",
    "            \n",
    "        # negative evaluated\n",
    "        \n",
    "        try: \n",
    "            value_temp.append(int(df_final.select('page').where((df_final.page == 'Downgrade')\\\n",
    "                                                            |(df_final.page == 'Thumbs Down')\\\n",
    "                                                            |(df_final.page == 'Roll Advert')\\\n",
    "                                                            |(df_final.page == 'Cancel')).\\\n",
    "                                groupby().count().collect()[0].__getitem__('count')))          \n",
    "        except:\n",
    "            value_temp.append(None)\n",
    "            \n",
    "        # gender \n",
    "        value_temp.append(df_final.select('gender').collect()[0].__getitem__('gender'))\n",
    "\n",
    "        # device \n",
    "        value_temp.append(df_final.select('userAgentNew').collect()[0].__getitem__('userAgentNew'))\n",
    "\n",
    "        value_temp.append(stay_user)\n",
    "        \n",
    "        df_ubc_list.append(tuple(value_temp))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "# save temperory results \n",
    "\n",
    "import csv\n",
    "with open('df_ubc_totallist', 'w') as f: \n",
    "      \n",
    "    # using csv.writer method from CSV package \n",
    "    write = csv.writer(f) \n",
    "      \n",
    "    write.writerow(df_ubc_list) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "# run the second half \n",
    "\n",
    "for userId in userId_list[int(userId_length/2):-1]:\n",
    "    \n",
    "    value_temp = []\n",
    "\n",
    "    stay_user = 0\n",
    "\n",
    "    df_temp_paid = df_total.select(*all_col).where((df_total.level == 'paid') & (df_total.userId == userId))\n",
    "\n",
    "    if len(df_temp_paid.head(1)) == 0:\n",
    "        # print(\"Free users\")\n",
    "        pass\n",
    "\n",
    "    else: \n",
    "        first_paid_time = df_temp_paid.take(1)[0].__getitem__('ts')\n",
    "\n",
    "        # NO Free users \n",
    "        churn_time = df_temp_paid.select(\"ts\").where((df_temp_paid.level == 'paid') & (df_temp_paid.page == 'Cancellation Confirmation')|(df_temp_paid.level == 'Submit Downgrade'))\n",
    "\n",
    "        if churn_time.head(1) == []:\n",
    "            # stay paid user \n",
    "            stay_user = 1\n",
    "            first_churn_time = df_temp_paid.select('ts').groupby().max().collect()[0].__getitem__('max(ts)')\n",
    "\n",
    "\n",
    "        else:\n",
    "            # churn paid user \n",
    "            first_churn_time = churn_time.take(1)[0].__getitem__('ts')\n",
    "\n",
    "        df_final = df_temp_paid.select(*all_col).where((df_temp_paid.ts >= first_paid_time) & (df_temp_paid.ts <= first_churn_time))\n",
    "\n",
    "        # add userId \n",
    "        value_temp.append(userId)\n",
    "\n",
    "        # add new column total played time \n",
    "        try:\n",
    "            value_temp.append(int(df_final.select('length').groupby().sum().collect()[0].__getitem__('sum(length)')))\n",
    "        except:\n",
    "            value_temp.append(None)\n",
    "\n",
    "        # total paid time   \n",
    "        try:\n",
    "            value_temp.append(int((df_final.select('ts').groupby().max().collect()[0].__getitem__('max(ts)')\\\n",
    "                                -df_final.select('registration').collect()[0].__getitem__('registration'))/3600/1000))\n",
    "        except:\n",
    "            value_temp.append(None)\n",
    "\n",
    "        # total played number\n",
    "        try:\n",
    "            value_temp.append(int(df_final.select('page').where(df_final.page == 'NextSong').groupby().count().collect()[0]\\\n",
    "                                .__getitem__('count')))\n",
    "        except:\n",
    "            value_temp.append(None)\n",
    "            \n",
    "        # add to list and thumb up\n",
    "        try: \n",
    "            value_temp.append(int(df_final.select('page').where((df_final.page == 'Add to Playlist')\\\n",
    "                                                            |(df_final.page == 'Thumbs Up')).\\\n",
    "                                groupby().count().collect()[0].__getitem__('count')))        \n",
    "        except: \n",
    "            value_temp.append(None)\n",
    "            \n",
    "        # negative evaluated\n",
    "        \n",
    "        try: \n",
    "            value_temp.append(int(df_final.select('page').where((df_final.page == 'Downgrade')\\\n",
    "                                                            |(df_final.page == 'Thumbs Down')\\\n",
    "                                                            |(df_final.page == 'Roll Advert')\\\n",
    "                                                            |(df_final.page == 'Cancel')).\\\n",
    "                                groupby().count().collect()[0].__getitem__('count')))          \n",
    "        except:\n",
    "            value_temp.append(None)\n",
    "            \n",
    "        # gender \n",
    "        value_temp.append(df_final.select('gender').collect()[0].__getitem__('gender'))\n",
    "\n",
    "        # device \n",
    "        value_temp.append(df_final.select('userAgentNew').collect()[0].__getitem__('userAgentNew'))\n",
    "\n",
    "        value_temp.append(stay_user)\n",
    "        \n",
    "        df_ubc_list.append(tuple(value_temp))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "import csv\n",
    "with open('df_ubc_totallist', 'w') as f: \n",
    "      \n",
    "    # using csv.writer method from CSV package \n",
    "    write = csv.writer(f) \n",
    "      \n",
    "    write.writerow(df_ubc_list) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+------+----------+--------+----------+------+------+------+------+---+\n",
      "|userId|PlayedTime|PaidTime|PlayNumber|PosEva|NegEva|Gender|Device|  y|\n",
      "+------+----------+--------+----------+------+------+------+------+---+\n",
      "|200002|     72934|    1681|       300|    20|    10|     M|     m|  1|\n",
      "|   124|   1012312|    3157|      4079|   289|    86|     F|     m|  1|\n",
      "|    51|    523275|     466|      2111|   152|    45|     M|     w|  0|\n",
      "|    15|    477307|    1356|      1914|   140|    43|     M|     w|  1|\n",
      "|    54|    600432|    2658|      2402|   201|    64|     F|     w|  0|\n",
      "|   155|    174342|     565|       719|    72|    14|     F|     w|  1|\n",
      "|100014|     67703|    2042|       257|    24|     9|     M|     w|  0|\n",
      "|   132|    475673|    1605|      1900|   134|    38|     F|     m|  1|\n",
      "|   101|    433239|    1295|      1743|   144|    39|     M|     w|  0|\n",
      "|    11|     47660|    2987|       192|    16|     8|     F|     w|  1|\n",
      "|   138|    463125|    1599|      1868|   142|    47|     M|     m|  1|\n",
      "|300017|    897406|    1784|      3632|   416|    64|     F|     m|  1|\n",
      "|    29|    676516|    1442|      2715|   212|    47|     M|     m|  0|\n",
      "|    69|    279030|    1714|      1095|   103|    19|     F|     w|  1|\n",
      "|    42|    857297|    1610|      3474|   264|    62|     F|     w|  1|\n",
      "|    73|     90961|    1194|       365|    25|     9|     F|     w|  0|\n",
      "|    87|     34184|    1419|       136|     2|     3|     M|     w|  0|\n",
      "|     3|     54424|    1941|       214|    18|     7|     M|     w|  0|\n",
      "|   113|    380538|    2964|      1510|   127|    26|     F|     m|  1|\n",
      "|    30|    145731|    1513|       575|    43|    15|     M|     w|  1|\n",
      "+------+----------+--------+----------+------+------+------+------+---+\n",
      "only showing top 20 rows\n",
      "\n"
     ]
    }
   ],
   "source": [
    "df_ubc = df_ubc.union(spark.createDataFrame(data = df_ubc_list, schema = schema ).toDF(*new_col))\n",
    "# save to local \n",
    "df_ubc.write.format('json').save('./final/ubc.json')\n",
    "df_ubc.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.2 Study generated features"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.2.1 `Gender`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5,1,'Gender distribution in group \"Stay user\"')"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEICAYAAABPgw/pAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAFUlJREFUeJzt3XucZGV95/HPFwYU8QI4DctFHI2It6yYTNCsiYsSEjVZYXcVYdWMBiXGNdGNG0V3NWrcjW5M1H2tmuB1vERgURckaiQouiYKDmoUggYlCOMM0CgooAHBX/44T0PRdE9V3+iZZz7v16tfXeec55zzq9OnvvXUc6qqU1VIknZ8u6x2AZKk5WGgS1InDHRJ6oSBLkmdMNAlqRMGuiR1wkDfTiV5dpLPr+D2X53kA+32wUluSLLrMm37z5O8st0+Isnm5dhu294vJ/nmcm1vZLvLegyk1WCgL0CS45Kcl+TGJFe32y9IktWubSmq6vKqumdV3bqtdpM+yVTV86vqj5ajtiSV5EEj2/7/VXXocmx71KTHYEfTnlDPbbdrZP7Dk3wqybVJrktyQZInj6yzbE/CqyXJe9s5++wk713teu4KBvqEkrwEeAvwJ8C/AvYDng88Fth9FUu7k9XsZdrDHW87OUYfA85mOI/3BX4P+OGqVrSMtpNjfJcz0CeQ5D7Aa4EXVNXpVXV9Db5SVc+oqptau7sleWOSy5Nc1YYe9mjLjkiyOclLWu9+a5LnjOzjvknOTPLDJOcDPzOrhockOTvJ95N8M8mxI8vem+TtST6e5Ebg8XPchwck+WyS65OcDawdWbau9YTXtOlnJ7m0tf2nJM9I8lDgz4FfbEMT18237zbvdbP2/4ok1yS5LMkzRuafm+S5I9O3vQpI8rk2++/bPp8+u/eY5KFtG9cluSjJU2Ydl7cm+at2X85Lcofjuo1jcG6SP0ryt23dTyVZO9e6rf1L2990S5Lnjr6ymOcY3SfJ+5JMJ/lOkv+eZJfW/rbhsG3U9sdJzk/ygyRnJNlnvtrmqHUt8ADgHVV1c/v526r6fJI9gU8AB7RjfkOSA5IcnuQL7ThvTfJ/kuzetvfWJH86ax8fS/Liccd55P48t91+UDtPf9DOl1NH2i3pMbBTqCp/xvwATwRuAdaMafdm4ExgH+BeDL2gP27LjmjbeC2wG/Bk4EfA3m35KcBpwJ7AI4DvAp9vy/YErgCeA6wBfg64Bnh4W/5e4AcMrxZ2Ae4+R21fAP4MuBvwOOB64ANt2Tqg2rb3ZOipHdqW7T+yn2fP1DSy3Tvtu8173az7PbPvfwvcOLL9c4HnjmzvDvtodT1oZPoIYHO7vRvwLeAVDK+SntDu16EjtX0fOLzdtw8Cp8zzt7vtGIzU9W3gwcAebfr12zg/rgQeDtwDeP9o3fMco/cBZzCcJ+uAfwROaO1fPfO32UZt32U4T/YEPjzafoLzOcAlwFnAMcB+s5bfdoxH5v088Jh2HNcBFwMvbssOB7YAu7TptQzn9n5z7PsO92X2OQB8CPhvI8fpl5brMbAz/NhDn8xa4JqqumVmRpK/a72VHyd5XJIAzwP+S1V9v6quB/4ncNzIdn4CvLaqflJVHwduAA7N8PLwPwKvqqobq+pCYOPIer8BXFZV76mqW6rqywwP4qeOtDmjhl7WT6vqn0eLT3Iw8AvAK6vqpqr6HMOTzXx+CjwiyR5VtbWqLhpzfObd94iZfX8W+Cvg2HnaLcRjgHsyBO3NVfVphpA6fqTNR6rq/Pa3+yBw2AK2/56q+seq+jHDk+186x7b2l5UVT8CXjNHm9uOEcN58HTg5TW82rsM+FPgWQuo7f1VdWFV3Qi8Ejg2Ew4z1JCAjwdm9rs1yeeSHLKNdS6oqi+28+8y4C8YnpypqvMZwvTI1vw44NyqumoB92fGT4D7AwdU1T9X1cw1myU9BnYWBvpkvgesHX2ZWFX/pqr2ast2AaYYemcXtKC/Dvhkm3/bdkafFBh6MfdsbdYw9EBmfGfk9v2BR89st237GQxj+TNG153tAODa9uCfa/u3aW2eznB9YGsbrnjINrY9bt/Ms+8DxqwziQOAK1pIjm77wJHpK0duzxzvSU267gHc8RjMdTxG561leEUx+jeYXfc4s8+V3RgZRhunqjZX1Qur6mcYzq8bGV41zCnJg5OcleTKJD9k6KyM7m8j8Mx2+5kMr1IW46UMryDOb0Nov9XmL/UxsFMw0CfzBeAm4OhttLkG+DHDS8C92s99qmqSAJlmGJa438i8g0duXwF8dmS7e9XwjozfGWmzra/N3Ars3cZH59r+HVTVX1fVUQzDLd8A3jFmH+O+snOufW9pt29keCKcMfoAHWcLcL+ZseeRbX93AdtYDluBg0am7zdHm9FjdA2390RnjNY9yTGZfa78pG13warqCuCtDEM4s2ud8XaGc+GQqro3wzDX6Lu7PgAcneSRwEOB/zfP7mae2Oe8f1V1ZVU9r6oOAH4beFu7FrHUx8BOwUCfQFVdx/Ay+m1Jnprknkl2SXIYw9gerZf4DuBNSfYFSHJgkl+bYPu3Ah8BXp3kHkkeBmwYaXIW8OAkz0qyW/v5hQwXKiep/zvAJuA1SXZP8kvAv5urbZL9kjylBfBNDMNCM2/luwo4aOZi2ALN7PuXGV4+/982/6vAf2j3+0HACbPWuwp44DzbPI8hIF7ajskR7X6dsoj6luI04DkZLtDeA3jVthq3v/dpwP9Icq8k9wd+nyEUYTgmj8vw3vj7AC+fYzPPTPKwtr/XAqfXhG+5TLJ3kte0C5C7tIukvwV8sTW5Crhv2/eMezFcW7mhvWIbDVKqajPwJYae+YfbMNVc932a4YnrmUl2bT3w2y5UJ3lakpknx2sZQvpWlvgY2FkY6BOqqv/F8KB7KXA1w0n/F8DLgL9rzV7GcJHui+1l6d8Ak75n+oUML+mvZLjA856RfV8P/CrD2OSW1uYNDBcZJ/WfgEczXCT8Q+Z/eb0L8JK2n+8zjJO+oC37NHARcGWShfQGr2R4cG5hGMd+flV9oy17E3Azw/Hc2JaPejWwsb3MvsO4e1XdDDwFeBJD7/RtwG+ObPsuUVWfAP438BmGv/8X2qKbtrHa7zI8GV0KfB74S+DdbXtnA6cCXwMuYAiz2d7PcJ5cyXDx8PcWUPLNDBcn/4YhpC9stT677f8bDBcnL23H/QDgvzKcQ9czdFxOvdNWh7/fzzJ+uOV5wB8wDFc+nNsfPzBc6zkvyQ0MbzB4UVX90zI9BrqX4fqIpOXSeo0XAnebdc1kubZ/LsO7Wt653NteiiSPY3iVsW7WdQ3dReyhS8sgyb9vQ0p7M/QcP7YSYb69SrIb8CLgnYb56jHQpeXx2wwXt7/NMOb7O9tu3o/2iuQ6hovob17lcnZqDrlIUifsoUtSJ9aMb7J81q5dW+vWrbsrdylJO7wLLrjgmqqaGtdubKAnOZQ7vkXpgQzvs31fm7+O4SPEx1bVtdva1rp169i0adO4XUqSRiSZ85Pds40dcqmqb1bVYVV1GMMX9PwI+ChwEnBOVR0CnNOmJUmrZKFj6EcC326fPDya279AaiPDt7ZJklbJQgP9OIZPkMHw1ZhbAdrvfZezMEnSwkwc6O37O57C7d/BMel6JybZlGTT9PT0QuuTJE1oIT30JwFfHvmO46uS7A/Qfl8910pVdXJVra+q9VNTYy/SSpIWaSGBfjy3D7fA8MU5M98IuIHhv69IklbJRIHevqLzKIaveJ3xeuCoJJe0Za9f/vIkSZOa6INF7d9q3XfWvO9x+7+ckiStMj/6L0mduEs/+r8cfv4P5v23h9pJXfAnv7naJUjbBXvoktQJA12SOmGgS1InDHRJ6oSBLkmd2OHe5SJtry5/7c+udgnaDh38qq/fZfuyhy5JnTDQJakTBrokdcJAl6ROGOiS1AkDXZI6YaBLUicMdEnqhIEuSZ0w0CWpEwa6JHXCQJekThjoktQJA12SOjFRoCfZK8npSb6R5OIkv5hknyRnJ7mk/d57pYuVJM1v0h76W4BPVtVDgEcCFwMnAedU1SHAOW1akrRKxgZ6knsDjwPeBVBVN1fVdcDRwMbWbCNwzEoVKUkab5Ie+gOBaeA9Sb6S5J1J9gT2q6qtAO33vnOtnOTEJJuSbJqenl62wiVJdzRJoK8Bfg54e1U9CriRBQyvVNXJVbW+qtZPTU0tskxJ0jiTBPpmYHNVndemT2cI+KuS7A/Qfl+9MiVKkiYxNtCr6krgiiSHtllHAv8AnAlsaPM2AGesSIWSpImsmbDd7wIfTLI7cCnwHIYng9OSnABcDjxtZUqUJE1iokCvqq8C6+dYdOTyliNJWiw/KSpJnTDQJakTBrokdcJAl6ROGOiS1AkDXZI6YaBLUicMdEnqhIEuSZ0w0CWpEwa6JHXCQJekThjoktQJA12SOmGgS1InDHRJ6oSBLkmdMNAlqRMGuiR1wkCXpE4Y6JLUiTWTNEpyGXA9cCtwS1WtT7IPcCqwDrgMOLaqrl2ZMiVJ4yykh/74qjqsqta36ZOAc6rqEOCcNi1JWiVLGXI5GtjYbm8Ejll6OZKkxZo00Av4VJILkpzY5u1XVVsB2u99V6JASdJkJhpDBx5bVVuS7AucneQbk+6gPQGcCHDwwQcvokRJ0iQm6qFX1Zb2+2rgo8DhwFVJ9gdov6+eZ92Tq2p9Va2fmppanqolSXcyNtCT7JnkXjO3gV8FLgTOBDa0ZhuAM1aqSEnSeJMMuewHfDTJTPu/rKpPJvkScFqSE4DLgaetXJmSpHHGBnpVXQo8co753wOOXImiJEkL5ydFJakTBrokdcJAl6ROGOiS1AkDXZI6YaBLUicMdEnqhIEuSZ0w0CWpEwa6JHXCQJekThjoktQJA12SOmGgS1InDHRJ6oSBLkmdMNAlqRMGuiR1wkCXpE4Y6JLUCQNdkjphoEtSJyYO9CS7JvlKkrPa9AOSnJfkkiSnJtl95cqUJI2zkB76i4CLR6bfALypqg4BrgVOWM7CJEkLM1GgJzkI+HXgnW06wBOA01uTjcAxK1GgJGkyk/bQ3wy8FPhpm74vcF1V3dKmNwMHzrVikhOTbEqyaXp6eknFSpLmNzbQk/wGcHVVXTA6e46mNdf6VXVyVa2vqvVTU1OLLFOSNM6aCdo8FnhKkicDdwfuzdBj3yvJmtZLPwjYsnJlSpLGGdtDr6qXV9VBVbUOOA74dFU9A/gM8NTWbANwxopVKUkaaynvQ38Z8PtJvsUwpv6u5SlJkrQYkwy53KaqzgXObbcvBQ5f/pIkSYvhJ0UlqRMGuiR1wkCXpE4Y6JLUCQNdkjphoEtSJwx0SeqEgS5JnTDQJakTBrokdcJAl6ROGOiS1AkDXZI6YaBLUicMdEnqhIEuSZ0w0CWpEwa6JHXCQJekThjoktQJA12SOjE20JPcPcn5Sf4+yUVJXtPmPyDJeUkuSXJqkt1XvlxJ0nwm6aHfBDyhqh4JHAY8McljgDcAb6qqQ4BrgRNWrkxJ0jhjA70GN7TJ3dpPAU8ATm/zNwLHrEiFkqSJTDSGnmTXJF8FrgbOBr4NXFdVt7Qmm4ED51n3xCSbkmyanp5ejpolSXOYKNCr6taqOgw4CDgceOhczeZZ9+SqWl9V66emphZfqSRpmxb0Lpequg44F3gMsFeSNW3RQcCW5S1NkrQQk7zLZSrJXu32HsCvABcDnwGe2pptAM5YqSIlSeOtGd+E/YGNSXZleAI4rarOSvIPwClJXgd8BXjXCtYpSRpjbKBX1deAR80x/1KG8XRJ0nbAT4pKUicMdEnqhIEuSZ0w0CWpEwa6JHXCQJekThjoktQJA12SOmGgS1InDHRJ6oSBLkmdMNAlqRMGuiR1wkCXpE4Y6JLUCQNdkjphoEtSJwx0SeqEgS5JnTDQJakTBrokdcJAl6ROjA30JPdL8pkkFye5KMmL2vx9kpyd5JL2e++VL1eSNJ9Jeui3AC+pqocCjwH+c5KHAScB51TVIcA5bVqStErGBnpVba2qL7fb1wMXAwcCRwMbW7ONwDErVaQkabwFjaEnWQc8CjgP2K+qtsIQ+sC+86xzYpJNSTZNT08vrVpJ0rwmDvQk9wQ+DLy4qn446XpVdXJVra+q9VNTU4upUZI0gYkCPcluDGH+war6SJt9VZL92/L9gatXpkRJ0iQmeZdLgHcBF1fVn40sOhPY0G5vAM5Y/vIkSZNaM0GbxwLPAr6e5Ktt3iuA1wOnJTkBuBx42sqUKEmaxNhAr6rPA5ln8ZHLW44kabH8pKgkdcJAl6ROGOiS1AkDXZI6YaBLUicMdEnqhIEuSZ0w0CWpEwa6JHXCQJekThjoktQJA12SOmGgS1InDHRJ6oSBLkmdMNAlqRMGuiR1wkCXpE4Y6JLUCQNdkjphoEtSJ8YGepJ3J7k6yYUj8/ZJcnaSS9rvvVe2TEnSOJP00N8LPHHWvJOAc6rqEOCcNi1JWkVjA72qPgd8f9bso4GN7fZG4JhlrkuStECLHUPfr6q2ArTf+87XMMmJSTYl2TQ9Pb3I3UmSxlnxi6JVdXJVra+q9VNTUyu9O0naaS020K9Ksj9A+3318pUkSVqMxQb6mcCGdnsDcMbylCNJWqxJ3rb4IeALwKFJNic5AXg9cFSSS4Cj2rQkaRWtGdegqo6fZ9GRy1yLJGkJ/KSoJHXCQJekThjoktQJA12SOmGgS1InDHRJ6oSBLkmdMNAlqRMGuiR1wkCXpE4Y6JLUCQNdkjphoEtSJwx0SeqEgS5JnTDQJakTBrokdcJAl6ROGOiS1AkDXZI6YaBLUicMdEnqxJICPckTk3wzybeSnLRcRUmSFm7RgZ5kV+CtwJOAhwHHJ3nYchUmSVqYpfTQDwe+VVWXVtXNwCnA0ctTliRpodYsYd0DgStGpjcDj57dKMmJwIlt8oYk31zCPnVHa4FrVruI1ZY3bljtEnRnnpsz/jDLsZX7T9JoKYE+V5V1pxlVJwMnL2E/mkeSTVW1frXrkGbz3FwdSxly2Qzcb2T6IGDL0sqRJC3WUgL9S8AhSR6QZHfgOODM5SlLkrRQix5yqapbkrwQ+GtgV+DdVXXRslWmSTiUpe2V5+YqSNWdhr0lSTsgPykqSZ0w0CWpEwb6DiRJJXn/yPSaJNNJzlrNuqQZSW5N8tWRn3WrXdPOZCnvQ9dd70bgEUn2qKofA0cB313lmqRRP66qw1a7iJ2VPfQdzyeAX2+3jwc+tIq1SNqOGOg7nlOA45LcHfjXwHmrXI80ao+R4ZaPrnYxOxuHXHYwVfW1Ni55PPDx1a1GuhOHXFaRgb5jOhN4I3AEcN/VLUXS9sJA3zG9G/hBVX09yRGrXYyk7YOBvgOqqs3AW1a7DknbFz/6L0md8F0uktQJA12SOmGgS1InDHRJ6oSBLkmdMNAlqRMGuiR14l8ArjqgnBq0yvsAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f2e94866a20>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.barplot(['M','F'], [df_ubc.select('Gender').where((df_ubc.Gender =='M') & (df_ubc.y ==1)).count(),df_ubc.select('Gender').where((df_ubc.Gender =='F') & (df_ubc.y ==1)).count()])\n",
    "plt.title('Gender distribution in group \"Stay user\"')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5,1,'Gender distribution in group \"Churn user\"')"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEICAYAAABYoZ8gAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAFk9JREFUeJzt3XucZGV95/HPVwaMFwR0WpSLjBsNim5gzQRiVIKrIhAjm4SXzugKKmbES3azceMlWUExycao8RVFZUdF0ACiRCJRVNgQRBNEG5brKpGwKOOM0FzkJlFHf/tHncaiqe6uqaqZHub5vF+vevW5POd5nj516lunnnOqO1WFJKkdD1rqDkiStiyDX5IaY/BLUmMMfklqjMEvSY0x+CWpMQb/A0iSlyf56mas/21J/qabflySu5JsN6G6T0zy1m76oCTrJlFvV9+zklwzqfr66p3oPnig29zHn7Ycg39MSVYluTjJ3Ulu6qZfmyRL3bdxVNV3q+rhVfXThcoNGwZVdUxVvWMSfUtSSZ7QV/dXqmrvSdTdb9h98EDTvfFe0E3XnHXPT3JhkjuTzCT5cpIXLklHl8hC+2dbYfCPIckbgL8G3gU8BtgVOAZ4BrDDEnbtfpbyrNUz5sVtDfsoyRHAp4GPA3vQO56PBX5rM7S1bNJ1TsLW2q9JM/hHlGQn4HjgtVV1ZlXdWT3/p6peWlU/6so9OMm7k3w3yY3dkMdDunUHJVmX5A3dp4UNSV7R18ajkpyd5I4kXwd+cU4fnpTkvCS3JrkmyYv61p2c5ENJzklyN/DsAb/D47szujuTnAcs71u3ojuzXtbNvzzJdV3Z/5fkpUmeDJwIPL0bEvnBfG13y/50Tvt/nOTmJNcneWnf8guSvKpv/t5PFUku7BZf3rX54rlDR0me3NXxgyRX95+xdv34QJLPd7/LxUnus18X2AcXJHlHkn/qtj03yfJB23bl39g9p+uTvKr/k8o8+2inJB/vzrS/k+R/JHlQV/7eYbgF+vY/k3w9ye1JPpvkkfP1bUBfA/wV8I6q+khV3V5VP6uqL1fV780p++4kt3XHwaF9y69P8ty++f6hw9n+Hp3ku8D5fcuO6l4fNyf5kwX6uNBxkSTvTe91dHuSK5I8tVs3zGvwTUm+D3xs2H32QGbwj+7pwIOBzy5S7p3ALwH7AU8Adqd3FjXrMcBO3fKjgQ8k2aVb9wHg34DHAq/sHgAkeRhwHnAa8GhgNfDBJE/pq/slwJ8BOwKDhmNOAy6hF/jvAI4a9At0bb0POLSqdgR+Hbisqr5J7xPORd2QyM6b0PZjunZ379pdm2TR4ZqqOrCb3Ldr84w5fd0e+HvgXHr75feBU+fUvRp4O7ALcG3Xz2G9BHhFV/cOwH8fVCjJIcAfAs+l97z/xjx19e+j99M7Fv5dV/7Irq1hHUnvGNkN2EjvObufqrqgqg7qpmeHJPcG9gTOXKSNA4Br6D13fwl8tHvTGNZvAE8Gnt+37Jld+88Bju1OKDbVwcCB9F5rOwMvBm7p1g3zGnwksBewZp79s00x+Ee3HLi5qjbOLkjyz91Z5j1JDuxeEL8H/LequrWq7gT+HFjVV89PgOOr6idVdQ5wF7B3eh/9fxc4tqrurqqrgFP6tnsBcH1VfayqNlbVpcDfAkf0lflsVf1Td+b2b/2dT/I44FeBt1bVj6rqQnqBOZ+fAU9N8pCq2lBVVy+yf+Ztu89s218GPg+8aJ5ym+LXgIcDf1FVP66q84HP0Qv7WZ+pqq93z92p9AJhWB+rqn+pqnuATy2w7Yu6sldX1Q/pvdHMde8+onccvBh4S/fp8XrgPcDLNqFvn6iqq6rqbuCtwIsy/BDSo7qfGxYp952q+nB33eMUeiclu25CH9/WHc/39C17e1XdU1WXA5cD+25CfbN+Qu8N9ElAquqbVbVhyNfgz4DjumPxnvvVvA0y+Ed3C7A8fWOCVfXr3VnvLfT27RTwUOCS7g3hB8AXu+X31tP/5gH8kF5wTQHLgBv61n2nb3ov4IDZeru6X0rv7GVW/7Zz7Qbc1oXEoPrv1ZV5Mb2z+w3dMMmTFqh7sbaZp+3dFtlmGLsBN3Rh2l/37n3z3++bnt3fwxp229247z4YtD/6ly2n9wmi/zmY2+/FzD1Wtqdv+G4Rs2fHj12k3L2/f/eGBpu2/wbth3Gej9m+nA+cQO9T8o1J1iZ5BMO9BmcWODnZJhn8o7sI+BFw+AJlbgbuAZ5SVTt3j52qapgDe4bex/U9+5Y9rm/6BuDLffXu3A19vKavzEJ3JGwAdumGcQbVfx9V9aWqeh69YPgW8OFF2ljsbohBba/vpu+m92Kd1f9mtpj1wJ6zY+N9dX9vE+qYhA30LpDO2nNAmf59dDO9s9a9+pb193uYfTL3WPlJV+8wrqF3TP3ukOUHGaaP49wls2D9VfW+qvoV4Cn0hnb+iOFeg9vknTsLMfhHVFU/oPfx/YNJjkjy8CQPSrIf8LCuzM/oBeR7kzwaIMnuSZ4/b8U/r/+nwGeAtyV5aJJ9uO8Y/OeAX0rysiTbd49fHXZ8tKq+A0wDb0+yQ5JnMs/dG0l2TfLCLqh/RG84avYWxxuBPZKMchfTbNvPojd09elu+WXA73S/9xPoXfvodyO9cfBBLqYXEG/s9slB3e/1yRH6N45PAa9I70LzQ7nvmPL9dM/3p4A/S7Jjkr3oXSOYvaB7GXBget8t2Al4y4Bq/nOSfbr2jgfOHPZW1Or9ffY/BN6a5BVJHtEdz89MsnaYOro+rur2+0ruO+w4CfMeF92xf0B3jeduetfGfjrOa3BbZvCPoar+kt6L5Y3ATfQC6X8BbwL+uSv2JnoXEL+W5A7gf9O7kDWM19P72Pt94GT67jjoxioPpjdWub4r8056F5yH9RJ6F+tuBY6jdxvfIA8C3tC1cyu9C3Sv7dadD1wNfD/JsGeXdP29ravzVOCYqvpWt+69wI/p7c9TuvX93gac0n10v891gar6MfBC4FB6Z3sfBI7sq3uLqKov0Lu4+o/0nv+LulU/WmCz36cXWtfRu9h7GnBSV995wBnAFfQuyH9uwPafoHecfB/4BeC/bGKfz6Q3pPdKes/LjcCfsvgNDLPeSu/Os9vonRSdtintD2Gh4+IR9AL+NnrDXLcA7+7WjfMa3Cal/Ecs0mbXfRK7CnjwnGs6k6r/AuBvquojk65b2x7P+KXNJMlvd0NZu9D7NPb3myP0pU1l8Eubz6vpXaT/V3rXRF6zcHFpy3CoR5Ia4xm/JDVmq/yDRMuXL68VK1YsdTck6QHjkksuubmqphYvuZUG/4oVK5ienl7qbkjSA0aSgd+8H8ShHklqjMEvSY0x+CWpMQa/JDXG4Jekxhj8ktQYg1+SGmPwS1JjDH5JasxW+c3dcf3KH833/0TUskvedeRSd0HaKnjGL0mNWfSMP8lJ9P4f6k1V9dRu2Rn8/F+X7Qz8oKr2G7Dt9cCd9P4W+caqWjmhfkuSRjTMUM/JwAn0/T/Wqnrx7HSS9wC3L7D9s6tqU/4XqyRpM1o0+KvqwiQrBq1LEuBFwH+cbLckSZvLuGP8zwJurKpvz7O+gHOTXJJkzUIVJVmTZDrJ9MzMzJjdkiTNZ9zgXw2cvsD6Z1TV04BDgdclOXC+glW1tqpWVtXKqamh/peAJGkEIwd/kmXA7wBnzFemqtZ3P28CzgL2H7U9SdJkjHPG/1zgW1W1btDKJA9LsuPsNHAwcNUY7UmSJmDR4E9yOnARsHeSdUmO7latYs4wT5LdkpzTze4KfDXJ5cDXgc9X1Rcn13VJ0iiGuatn9TzLXz5g2XrgsG76OmDfMfsnSZowv7krSY0x+CWpMQa/JDXG4Jekxhj8ktQYg1+SGmPwS1JjDH5JaozBL0mNMfglqTEGvyQ1xuCXpMYY/JLUGINfkhpj8EtSYwx+SWqMwS9JjTH4JakxBr8kNcbgl6TGLBr8SU5KclOSq/qWvS3J95Jc1j0Om2fbQ5Jck+TaJG+eZMclSaMZ5oz/ZOCQAcvfW1X7dY9z5q5Msh3wAeBQYB9gdZJ9xumsJGl8iwZ/VV0I3DpC3fsD11bVdVX1Y+CTwOEj1CNJmqBxxvhfn+SKbiholwHrdwdu6Jtf1y0bKMmaJNNJpmdmZsboliRpIaMG/4eAXwT2AzYA7xlQJgOW1XwVVtXaqlpZVSunpqZG7JYkaTEjBX9V3VhVP62qnwEfpjesM9c6YM+++T2A9aO0J0manJGCP8lj+2Z/G7hqQLFvAE9M8vgkOwCrgLNHaU+SNDnLFiuQ5HTgIGB5knXAccBBSfajN3RzPfDqruxuwEeq6rCq2pjk9cCXgO2Ak6rq6s3yW0iShrZo8FfV6gGLPzpP2fXAYX3z5wD3u9VTkrR0/OauJDXG4Jekxhj8ktQYg1+SGmPwS1JjDH5JaozBL0mNMfglqTEGvyQ1xuCXpMYY/JLUGINfkhpj8EtSYwx+SWqMwS9JjTH4JakxBr8kNWbR/8AlabK+e/y/X+ouaCv0uGOv3GJtecYvSY0x+CWpMYsGf5KTktyU5Kq+Ze9K8q0kVyQ5K8nO82x7fZIrk1yWZHqSHZckjWaYM/6TgUPmLDsPeGpV/TLwL8BbFtj+2VW1X1WtHK2LkqRJWjT4q+pC4NY5y86tqo3d7NeAPTZD3yRJm8EkxvhfCXxhnnUFnJvkkiRrFqokyZok00mmZ2ZmJtAtSdIgYwV/kj8BNgKnzlPkGVX1NOBQ4HVJDpyvrqpaW1Urq2rl1NTUON2SJC1g5OBPchTwAuClVVWDylTV+u7nTcBZwP6jtidJmoyRgj/JIcCbgBdW1Q/nKfOwJDvOTgMHA1cNKitJ2nKGuZ3zdOAiYO8k65IcDZwA7Aic192qeWJXdrck53Sb7gp8NcnlwNeBz1fVFzfLbyFJGtqif7KhqlYPWPzRecquBw7rpq8D9h2rd5KkifObu5LUGINfkhpj8EtSYwx+SWqMwS9JjTH4JakxBr8kNcbgl6TGGPyS1BiDX5IaY/BLUmMMfklqjMEvSY0x+CWpMQa/JDXG4Jekxhj8ktQYg1+SGmPwS1JjDH5JasxQwZ/kpCQ3Jbmqb9kjk5yX5Nvdz13m2faorsy3kxw1qY5LkkYz7Bn/ycAhc5a9GfiHqnoi8A/d/H0keSRwHHAAsD9w3HxvEJKkLWOo4K+qC4Fb5yw+HDilmz4F+E8DNn0+cF5V3VpVtwHncf83EEnSFjTOGP+uVbUBoPv56AFldgdu6Jtf1y27nyRrkkwnmZ6ZmRmjW5KkhWzui7sZsKwGFayqtVW1sqpWTk1NbeZuSVK7xgn+G5M8FqD7edOAMuuAPfvm9wDWj9GmJGlM4wT/2cDsXTpHAZ8dUOZLwMFJduku6h7cLZMkLZFhb+c8HbgI2DvJuiRHA38BPC/Jt4HndfMkWZnkIwBVdSvwDuAb3eP4bpkkaYksG6ZQVa2eZ9VzBpSdBl7VN38ScNJIvZMkTZzf3JWkxhj8ktQYg1+SGmPwS1JjDH5JaozBL0mNMfglqTEGvyQ1xuCXpMYY/JLUGINfkhpj8EtSYwx+SWqMwS9JjTH4JakxBr8kNcbgl6TGGPyS1BiDX5IaM3LwJ9k7yWV9jzuS/MGcMgclub2vzLHjd1mSNI6h/tn6IFV1DbAfQJLtgO8BZw0o+pWqesGo7UiSJmtSQz3PAf61qr4zofokSZvJpIJ/FXD6POuenuTyJF9I8pQJtSdJGtHYwZ9kB+CFwKcHrL4U2Kuq9gXeD/zdAvWsSTKdZHpmZmbcbkmS5jGJM/5DgUur6sa5K6rqjqq6q5s+B9g+yfJBlVTV2qpaWVUrp6amJtAtSdIgkwj+1cwzzJPkMUnSTe/ftXfLBNqUJI1o5Lt6AJI8FHge8Oq+ZccAVNWJwBHAa5JsBO4BVlVVjdOmJGk8YwV/Vf0QeNScZSf2TZ8AnDBOG5KkyfKbu5LUGINfkhpj8EtSYwx+SWqMwS9JjTH4JakxBr8kNcbgl6TGGPyS1BiDX5IaY/BLUmMMfklqjMEvSY0x+CWpMQa/JDXG4Jekxhj8ktQYg1+SGmPwS1JjDH5JaszYwZ/k+iRXJrksyfSA9UnyviTXJrkiydPGbVOSNLplE6rn2VV18zzrDgWe2D0OAD7U/ZQkLYEtMdRzOPDx6vkasHOSx26BdiVJA0wi+As4N8klSdYMWL87cEPf/Lpu2X0kWZNkOsn0zMzMBLolSRpkEsH/jKp6Gr0hndclOXDO+gzYpu63oGptVa2sqpVTU1MT6JYkaZCxg7+q1nc/bwLOAvafU2QdsGff/B7A+nHblSSNZqzgT/KwJDvOTgMHA1fNKXY2cGR3d8+vAbdX1YZx2pUkjW7cu3p2Bc5KMlvXaVX1xSTHAFTVicA5wGHAtcAPgVeM2aYkaQxjBX9VXQfsO2D5iX3TBbxunHYkSZPjN3clqTEGvyQ1xuCXpMYY/JLUGINfkhpj8EtSYwx+SWqMwS9JjTH4JakxBr8kNcbgl6TGGPyS1BiDX5IaY/BLUmMMfklqjMEvSY0x+CWpMQa/JDXG4Jekxhj8ktSYkYM/yZ5J/jHJN5NcneS/DihzUJLbk1zWPY4dr7uSpHEtG2PbjcAbqurSJDsClyQ5r6r+75xyX6mqF4zRjiRpgkY+46+qDVV1aTd9J/BNYPdJdUyStHlMZIw/yQrgPwAXD1j99CSXJ/lCkqcsUMeaJNNJpmdmZibRLUnSAGMHf5KHA38L/EFV3TFn9aXAXlW1L/B+4O/mq6eq1lbVyqpaOTU1NW63JEnzGCv4k2xPL/RPrarPzF1fVXdU1V3d9DnA9kmWj9OmJGk849zVE+CjwDer6q/mKfOYrhxJ9u/au2XUNiVJ4xvnrp5nAC8DrkxyWbfsj4HHAVTVicARwGuSbATuAVZVVY3RpiRpTCMHf1V9FcgiZU4AThi1DUnS5PnNXUlqjMEvSY0x+CWpMQa/JDXG4Jekxhj8ktQYg1+SGmPwS1JjDH5JaozBL0mNMfglqTEGvyQ1xuCXpMYY/JLUGINfkhpj8EtSYwx+SWqMwS9JjTH4JakxBr8kNWas4E9ySJJrklyb5M0D1j84yRnd+ouTrBinPUnS+EYO/iTbAR8ADgX2AVYn2WdOsaOB26rqCcB7gXeO2p4kaTLGOePfH7i2qq6rqh8DnwQOn1PmcOCUbvpM4DlJMkabkqQxLRtj292BG/rm1wEHzFemqjYmuR14FHDz3MqSrAHWdLN3JblmjL7p55YzYH+3KO8+aqm7oPvz+Jx13NjnxHsNW3Cc4B/UyxqhTG9h1Vpg7Rj90QBJpqtq5VL3QxrE43NpjDPUsw7Ys29+D2D9fGWSLAN2Am4do01J0pjGCf5vAE9M8vgkOwCrgLPnlDkbmP18fQRwflUNPOOXJG0ZIw/1dGP2rwe+BGwHnFRVVyc5HpiuqrOBjwKfSHItvTP9VZPotDaJw2famnl8LoF4Ai5JbfGbu5LUGINfkhpj8G+DklSST/TNL0syk+RzS9kvaVaSnya5rO+xYqn71JJx7uPX1utu4KlJHlJV9wDPA763xH2S+t1TVfstdSda5Rn/tusLwG9206uB05ewL5K2Igb/tuuTwKokvwD8MnDxEvdH6veQvmGes5a6M61xqGcbVVVXdOOmq4FzlrY30v041LOEDP5t29nAu4GD6P1xPEky+LdxJwG3V9WVSQ5a6s5I2joY/NuwqloH/PVS90PS1sU/2SBJjfGuHklqjMEvSY0x+CWpMQa/JDXG4Jekxhj8ktQYg1+SGvP/AQdVwQkoEWu9AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f2e9470d128>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.barplot(['M','F'], [df_ubc.select('Gender').where((df_ubc.Gender =='M') & (df_ubc.y ==0)).count(),df_ubc.select('Gender').where((df_ubc.Gender =='F') & (df_ubc.y ==0)).count()])\n",
    "plt.title('Gender distribution in group \"Churn user\"')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.2.2 `Device`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5,1,'Device distribution in group \"Stay user\"')"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEICAYAAABPgw/pAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAF9ZJREFUeJzt3Xm0ZGV97vHvwyQqYIM2hLmJEkQ0IrYoEpWIGkQZYnBAg61BEVecckEk5i4vcRmDSa7kOt0Eh9CJqHBRBDUasRWcAO1GBAQUggikCTQCAk4I/O4f+z1QHM+h6kx96H2+n7V61Z73r/be9dS736o6napCkrTuW2++C5AkzQ4DXZJ6wkCXpJ4w0CWpJwx0SeoJA12SesJAf5BIskOSO5KsP8f7OTvJa9rwK5J8eRa3/YMk+7Th45J8fBa3/fYkH5mt7Q1sd1aPgTSfDPQpSHJ1kl8muT3JrUm+neTIJDM+jlV1TVVtUlV3z0atI+7z5Kp63rDlkpyU5F0jbG+3qjp7pnUl2SfJdeO2/e6qes1Mtz3eqMdgXdPeUI9rx/LsgekHJbkwyW1JbkqyIsmSgXVm7U14vrTX6ZJ23b5qvutZmwz0qTugqjYFdgSOB94GfHR+S5pfSTaY7xoe7B4MxyjJY4B/BY4CHgHsBHwIuGc+65otD4ZjPN8M9Gmqqp9V1ZnAS4FlSR4PkOQhSf4hyTVJbkjyT0ke2uZdluSFY9tIskFrJe3RWhQ1dlEm2SLJvyRZneSWJJ8dWO+FrZU1dpfw+5PVmeS5SS5P8rMkHwAyMO9VSb7ZhpPkhCQ3tmUvSvL4JEcArwCOaV1Cn2vLX53kbUkuAn7ensvVSZ4zsPuNk5zS7mguSPLEgX1XC5ix8ZOSvCvJw4EvAtu0/d2RZJvxrcckB7YunltbN9KuA/OuTnJ0ew4/azVsPMnxufcYDNR1ZJIr2nH/YJJMsu5Dkyxvy12W5JjBO4tJjtGurd5bW/0HDix/b3fYA9T2piRXtevm7zO1u8PdgR9X1Yrq3F5Vn66qa5LsB7wdeGk75t9v+3x1e263t/2+bqCeS5IcMDC+Yatr92HHeeD5PKYN75/k0raf/0py9MByk17vEx3jKRyP/qkq/434D7gaeM4E068BXt+G/xE4E9gC2BT4HPC3bd47gJMH1nsBcHkbXgIUsEEb/wJwCrA5sCHwrDZ9D+BG4KnA+sCyVtdDJqjrUcBtwCFtG38B3AW8ps1/FfDNNvxHwCpgEV3o7wps3eadBLxrgmNxIbA98NDxxwc4DvjNwL6PBn4MbNjmF/CYge3duw9gH+C6cfs7Dvh4G/494OfAc9u2jwGuBDYaqOM7wDbtPFwGHDnJOb33GAzU9fl2HHYA1gD7TbLu8cA57RxtB1w0WPf4Y9RqvZIuODcCng3cDuzSlj977Nw8QG1fa89pB+BHg8uPcP3+LvAr4ATgD4FNJjvG467RR7dr4lnAL4A92rxjgFMGlj0IuHiU4zz+GgCuB57Rhjcf2McDXu/jj/F8Z8R8/7OFPjtWA1u0ltxrgb+oqpur6nbg3cDL2nKfAA5M8rA2/vI27X6SbA08ny6Ebqmq31TVOW32a4F/rqrzq+ruqloO/Bp42gR17Q9cWlWnVdVv6N5s/nuS5/AbujegxwKpqsuq6vohz/t9VXVtVf1ykvmrBvb9XmDjSeqcqpcCX6iqs9q2/4EuMJ8+rrbVVXUz3Zvqb7UaH8DxVXVrVV1DF6CTrfsS4N3tHF0HvG+CZQaP0dOATdr276yqr9K9eRw6hdre066ta+jO58jrVtVVdG+W2wKnAje1O6NNHmCdL1TVf1bnHODLwDPa7I8D+yfZrI0fBvzbFJ7LoN8Aj0uyWTueF7Tpo1zvw67DBcNAnx3bAjcDi4GHAava7eGtwJfadKrqSrrW4gEt1A9kgkCna23cXFW3TDBvR+Cose23fWxP1xodbxvg2rGR6po0106wHC1cPgB8ELghyYkDL9TJTLitieZX1T3AdZPUOVXbAD8Zt+1r6c7DmME3rl/QBemoRl33fseXiY/H4LRtgGtbvWN+wv3rHmZwez9hisezqs6rqpdU1WK6YH4m8FeTLZ/k+UnOS3Jzu9b2p7vzo6pWA98C/iTJIrpGyMlTqWfAn7Rt/yTJOUn2atNHud6HXYcLhoE+Q0meQveC/CZwE/BLYLeqWtT+PaKqBgPhk3StqoPoWs9XTrDZa+la/Ismmfc3A9tfVFUPq6pPTrDs9XQX/1itGRwfr6reV1VPBnaj69Z469isyVaZbFvN4L7Xo+uWWN0m/YLuzW/M70xhu6vpXuhj2x57Xv81ZL3Zdj3dcxoz0bEdfC6rge3H9XvvwH11/5zJj8lE+9iB+47nlFXVd4HPAI+foFaSPAT4NN0d0FZVtQj4dwY+hwGWA38KvBg4t6omOwf3e25J7vfcquq7VXUQsCXwWbo7CBjtevdPxjYG+jQl2SzdB5yfout3vLi1vD4MnJBky7bctkn+aGDVTwHPA17PxK1zWlfHF4EPJdm8fdj0zDb7w8CRSZ6azsOTvCDJphNs6gvAbkle1D4sehMThwRJntK2uSHdi+9XwNhXKG+g63+dqicP7PstdLfK57V5FwIvT7J++0DuWQPr3QA8MskjJtnuqcALkuzb6j2qbfvb06hxJk4F/rKdo22BNwxZ/ny6Y3tMO6f7AAfQXRPQHZMXJXlY+7Dw8Am28da2v+2BN9N9zjKSJH+Q5LUD1+Zj6e4Sx87JDcCSgTecjYCH0H2OcFeS59Ndu4M+S9fP/Wa6b9BM5vt01+Lu6T6gPm6gro3S/R7gEa0L7Tbuu/amcr0veAb61H0uye10LYe/ousbfvXA/LfRffB1XpLbgK8Au4zNbGF9Ll1/7wO9GA+j61e8nO5Dobe09VfS9St+ALil7etVE22gqm6iazkdD/wU2JnuFnkim9G9eG6hu5X/KV3LDLqvZT6u3fJ+dpL1J3IGXX/3Le35vKi9YKELgAOAW+m+RXPvdqvqcro7mavaPu/XrVBVP6RrFb6f7q7oALqvk945hdpmwzvpupF+THeeT6N7Y5lQq+9Auq6Jm+i+MvjK9nyh+7DyTrpgXc7E3Rdn0H14fSHdG/ZUvjJ7a9v/xUnuoOsOPB34uzb//7XHnya5oH0G9Ca6N65b6D7zOXPcc/olXSt+J7rW/oSq6kd0x+srwBV0d7SDDgOubq+ZI+nO75Sud3Uffs13DVIvJHk98LKqetbQhae3/QJ2nqSbbt4keQfwe1X1p/Ndy0JnC12apiRbJ9k7yXpJdqHr+jl9vutam5JsQdc1dOJ81yIDXZqJjYB/pvsu+VfpukM+NK8VrUVJXkvX9fjFqvr6fNcju1wkqTdsoUtST6zVv3vwqEc9qpYsWbI2dylJ67xVq1bd1H4M9oDWaqAvWbKElStXrs1dStI6L8lPhi9ll4sk9YaBLkk9YaBLUk8Y6JLUEwa6JPWEgS5JPWGgS1JPGOiS1BMGuiT1xFr9pehUPPmtD/Sfn2g2rPr7V853CZJmkS10SeoJA12SesJAl6SeMNAlqScMdEnqCQNdknrCQJeknjDQJaknDHRJ6gkDXZJ6wkCXpJ540P4tF63brnnnE+a7hN7b4R0Xz3cJepCxhS5JPWGgS1JPjBToSRYlOS3J5UkuS7JXki2SnJXkiva4+VwXK0ma3Kgt9P8DfKmqHgs8EbgMOBZYUVU7AyvauCRpngwN9CSbAc8EPgpQVXdW1a3AQcDytthy4OC5KlKSNNwoLfTfBdYA/5Lke0k+kuThwFZVdT1Ae9xyDuuUJA0xSqBvAOwB/N+qehLwc6bQvZLkiCQrk6xcs2bNNMuUJA0zSqBfB1xXVee38dPoAv6GJFsDtMcbJ1q5qk6sqqVVtXTx4sWzUbMkaQJDA72q/hu4NskubdK+wKXAmcCyNm0ZcMacVChJGsmovxR9I3Byko2Aq4BX070ZnJrkcOAa4MVzU6IkaRQjBXpVXQgsnWDWvrNbjiRpuvylqCT1hIEuST1hoEtSTxjoktQTBrok9YSBLkk9YaBLUk8Y6JLUEwa6JPWEgS5JPWGgS1JPGOiS1BMGuiT1hIEuST1hoEtSTxjoktQTBrok9YSBLkk9YaBLUk8Y6JLUEwa6JPWEgS5JPWGgS1JPbDDKQkmuBm4H7gbuqqqlSbYATgGWAFcDL6mqW+amTEnSMFNpof9hVe1eVUvb+LHAiqraGVjRxiVJ82QmXS4HAcvb8HLg4JmXI0marlEDvYAvJ1mV5Ig2bauquh6gPW450YpJjkiyMsnKNWvWzLxiSdKERupDB/auqtVJtgTOSnL5qDuoqhOBEwGWLl1a06hRkjSCkVroVbW6Pd4InA7sCdyQZGuA9njjXBUpSRpuaKAneXiSTceGgecBlwBnAsvaYsuAM+aqSEnScKN0uWwFnJ5kbPlPVNWXknwXODXJ4cA1wIvnrkxJ0jBDA72qrgKeOMH0nwL7zkVRkqSp85eiktQTBrok9YSBLkk9YaBLUk8Y6JLUEwa6JPWEgS5JPWGgS1JPGOiS1BMGuiT1hIEuST1hoEtSTxjoktQTBrok9YSBLkk9YaBLUk8Y6JLUEwa6JPWEgS5JPWGgS1JPGOiS1BMGuiT1xMiBnmT9JN9L8vk2vlOS85NckeSUJBvNXZmSpGGm0kJ/M3DZwPh7gBOqamfgFuDw2SxMkjQ1IwV6ku2AFwAfaeMBng2c1hZZDhw8FwVKkkYzagv9H4FjgHva+COBW6vqrjZ+HbDtRCsmOSLJyiQr16xZM6NiJUmTGxroSV4I3FhVqwYnT7BoTbR+VZ1YVUuraunixYunWaYkaZgNRlhmb+DAJPsDGwOb0bXYFyXZoLXStwNWz12ZkqRhhrbQq+ovq2q7qloCvAz4alW9AvgacEhbbBlwxpxVKUkaaibfQ38b8D+SXEnXp/7R2SlJkjQdo3S53KuqzgbObsNXAXvOfkmSpOnwl6KS1BMGuiT1hIEuST1hoEtSTxjoktQTBrok9YSBLkk9YaBLUk8Y6JLUEwa6JPWEgS5JPWGgS1JPGOiS1BMGuiT1hIEuST1hoEtSTxjoktQTBrok9YSBLkk9YaBLUk8Y6JLUEwa6JPWEgS5JPTE00JNsnOQ7Sb6f5AdJ/rpN3ynJ+UmuSHJKko3mvlxJ0mRGaaH/Gnh2VT0R2B3YL8nTgPcAJ1TVzsAtwOFzV6YkaZihgV6dO9rohu1fAc8GTmvTlwMHz0mFkqSRjNSHnmT9JBcCNwJnAf8J3FpVd7VFrgO2nWTdI5KsTLJyzZo1s1GzJGkCIwV6Vd1dVbsD2wF7ArtOtNgk655YVUuraunixYunX6kk6QFN6VsuVXUrcDbwNGBRkg3arO2A1bNbmiRpKkb5lsviJIva8EOB5wCXAV8DDmmLLQPOmKsiJUnDbTB8EbYGlidZn+4N4NSq+nySS4FPJXkX8D3go3NYpyRpiKGBXlUXAU+aYPpVdP3pkqQHAX8pKkk9YaBLUk8Y6JLUEwa6JPWEgS5JPWGgS1JPGOiS1BMGuiT1hIEuST1hoEtSTxjoktQTBrok9YSBLkk9YaBLUk8Y6JLUEwa6JPWEgS5JPWGgS1JPGOiS1BMGuiT1hIEuST1hoEtSTwwN9CTbJ/laksuS/CDJm9v0LZKcleSK9rj53JcrSZrMKC30u4CjqmpX4GnAnyd5HHAssKKqdgZWtHFJ0jwZGuhVdX1VXdCGbwcuA7YFDgKWt8WWAwfPVZGSpOGm1IeeZAnwJOB8YKuquh660Ae2nGSdI5KsTLJyzZo1M6tWkjSpkQM9ySbAp4G3VNVto65XVSdW1dKqWrp48eLp1ChJGsFIgZ5kQ7owP7mqPtMm35Bk6zZ/a+DGuSlRkjSKUb7lEuCjwGVV9d6BWWcCy9rwMuCM2S9PkjSqDUZYZm/gMODiJBe2aW8HjgdOTXI4cA3w4rkpUZI0iqGBXlXfBDLJ7H1ntxxJ0nT5S1FJ6gkDXZJ6wkCXpJ4w0CWpJwx0SeoJA12SesJAl6SeMNAlqScMdEnqCQNdknrCQJeknjDQJaknDHRJ6gkDXZJ6wkCXpJ4w0CWpJwx0SeoJA12SesJAl6SeMNAlqScMdEnqCQNdknpiaKAn+ViSG5NcMjBtiyRnJbmiPW4+t2VKkoYZpYV+ErDfuGnHAiuqamdgRRuXJM2joYFeVV8Hbh43+SBgeRteDhw8y3VJkqZoun3oW1XV9QDtccvJFkxyRJKVSVauWbNmmruTJA0z5x+KVtWJVbW0qpYuXrx4rncnSQvWdAP9hiRbA7THG2evJEnSdEw30M8ElrXhZcAZs1OOJGm6Rvna4ieBc4FdklyX5HDgeOC5Sa4AntvGJUnzaINhC1TVoZPM2neWa5EkzYC/FJWknjDQJaknDHRJ6gkDXZJ6wkCXpJ4w0CWpJwx0SeqJod9Dl7Sw7P3+vee7hN771hu/NSfbtYUuST1hoEtSTxjoktQTBrok9YSBLkk9YaBLUk8Y6JLUEwa6JPWEgS5JPWGgS1JPGOiS1BMGuiT1hIEuST1hoEtSTxjoktQTMwr0JPsl+WGSK5McO1tFSZKmbtqBnmR94IPA84HHAYcmedxsFSZJmpqZtND3BK6sqquq6k7gU8BBs1OWJGmqUlXTWzE5BNivql7Txg8DnlpVbxi33BHAEW10F+CH0y/3Qe9RwE3zXYSmxXO3buv7+duxqhYPW2gm/6doJpj2W+8OVXUicOIM9rPOSLKyqpbOdx2aOs/dus3z15lJl8t1wPYD49sBq2dWjiRpumYS6N8Fdk6yU5KNgJcBZ85OWZKkqZp2l0tV3ZXkDcB/AOsDH6uqH8xaZeumBdG11FOeu3Wb548ZfCgqSXpw8ZeiktQTBrok9cSCDPQkJyR5y8D4fyT5yMD4/07y9iSnTXG7r0rygdmsVdOT5I+TVJLHzmAbJ7XfW2geJLmjPW4z1dfiQrUgAx34NvB0gCTr0f0oYbeB+U8HVlSVL+Z116HAN+m+faV1WFWt9rU4moUa6N+iBTpdkF8C3J5k8yQPAXYFbklyCdzb8v5Mki8luSLJ341tKMmrk/woyTnA3gPTd0yyIslF7XGHJOsnuSqdRUnuSfLMtvw3kjwmybOSXNj+fS/JpmvpmPRGkk3ozsXhtEBPsk+Sryc5PcmlSf6pvZmT5I52V3ZBO1e/9Yu8JE9Ock6SVe2Obuu1+qQWsCRLRnwt3jEwfEiSk9rwGUle2YZfl+TktfwU1poFGehVtRq4K8kOdMF+LnA+sBewFLgIuHPcarsDLwWeALw0yfbtRf3XdOHxXLo/UjbmA8C/VtXvAycD76uqu4EfteX+AFgFPKO9iWxXVVcCRwN/XlW7A88Afjnbz38BOBj4UlX9CLg5yR5t+p7AUXTn8NHAi9r0hwMXVNUewDnA/xrcWJINgfcDh1TVk4GPAX8z589Ck/mt1+KQ5Y8A3pHkGXTn/41zXN+8WZCB3oy10scC/dyB8W9PsPyKqvpZVf0KuBTYEXgqcHZVrWl/oOyUgeX3Aj7Rhv+NLsABvgE8s/372zb9KXQ/1Bqr671J3gQsqqq7ZuG5LjSH0v2xONrjoW34O+2Pyd0NfJL7zsk93HfuPj4wfcwuwOOBs5JcCPxPul9Ga35M9FqcVFXdALwD+BpwVFXdvBZqnBcz+Vsu67qxfvQn0HW5XEv37n0bXQtsvF8PDN/Nfcdu1C/yjy33DeBIYBu6i+ytwD7A1wGq6vgkXwD2B85L8pyqunzEfSx4SR4JPBt4fJKi+9FbAf/Ob5+ryc7d+OkBflBVe81mrZq2UV6LG49b5wnAT+led7210FvoLwRurqq727v2IrqW9bkjbuN8YJ8kj2y35S8emPdt7vtA7hV0H9CNrfN04J7WwrgQeB1d0JPk0VV1cVW9B1gJTPtbGgvUIXRdXTtW1ZKq2h74MV2re8/2pyrWo7tlHzsn67X1AF4+MH3MD4HFSfaCrgsmyW7oweaGJLu28/vHYxOT7En3/zY8CTg6yU7zVeBcW8iBfjHdt1vOGzftZ1U10p/hrKrrgePo3gC+AlwwMPtNwKuTXAQcBry5rfNruruBsf1+A9i07RvgLUkuSfJ9uv7zL075mS1shwKnj5v2abqgPhc4nu6O7McDy/0c2C3JKrrW/TsHV27daYcA72nn5ULu+1BdDx7HAp8HvgpcD9A+n/ow8Gfts7OjgI8lmeivxa7z/Om/FoQk+wBHV9ULJ5h3R1VtsvarkmbXQm6hS1Kv2EKXpJ6whS5JPWGgS1JPGOiS1BMGuiT1hIEuST3x/wFadm+0BuqONgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f2e946b9ac8>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.barplot(['Windows','Apple','linux'], [df_ubc.select('Device').where((df_ubc.Device =='w') & (df_ubc.y ==1)).count(), \\\n",
    "                                         df_ubc.select('Device').where((df_ubc.Device =='m') & (df_ubc.y ==1)).count(),\\\n",
    "                                         df_ubc.select('Device').where((df_ubc.Device =='l') & (df_ubc.y ==1)).count()])\n",
    "plt.title('Device distribution in group \"Stay user\"')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5,1,'Device distribution in group \"Churn user\"')"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEICAYAAABYoZ8gAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAGgNJREFUeJzt3Xu4HFWd7vHvCwERiAbIFgmEhAFk5CKZsAkiB4yDIMQAAw8jiY6CogEfFJmDF4aZBxhm9IA3zoHoMAFyQMXAiIIgF0EUuYM7mQBBQDLhkrhzkg0JgQAjBn7nj1pNKk33vnT1zk6y3s/z7GdXV61aa1VV99vVq6u7FRGYmVk+NhrqDpiZ2drl4Dczy4yD38wsMw5+M7PMOPjNzDLj4Dczy4yDfx0naUdJKyVtPMjt3CHps2n6E5JubWPdj0qamKbPkfSjNtZ9pqRL21Vfqd627oP1naTLJf3rUPfD2sPB3waSnpb0qqSXJL0g6V5JJ0uqvH8j4tmI2DIiXm9HX/vZ5pURcWhf5fobBhGxR0TcUbVfkiZKWlRX9zci4rNV667X332wvklPvOekfXlHab4knSppnqSXJS2S9BNJew1hd9e6ZvtnQ+Pgb58jImI4MAY4D/gacNnQdmloSRo21H1Y161D++j/AF8CTgW2Bt4DXAd8tN0NrUPbvIZ1tV+DwcHfZhGxIiKuB44Djpe0J4Ckt0n6tqRnJS2RdLGkt6dlj0maXKtD0jBJz0kaL2mspKjdKSVtLen/SuqWtFzSdaX1JkuaW3rV8b5m/ZR0iKTHJa2QNB1QadkJku5O05J0gaSlqezDkvaUNA34BPDVNBR1Qyr/tKSvSXoYeDlty9OSPlxqfjNJV6dXSHMk7V1qOyTtUrp9uaR/lbQFcDMwKrW3UtKo+qEjSUemoaUX0vDVe0vLnpb05bQNK1IfNmuyf97cB6V+nSzpybTfvydJTdZ9u6QrUrnHJH21/EqlyT56b+rvC6n/R5bKvzkM10vfTpW0IN1vvqUBvNqUtCtwCjA1In4dEX+KiFfSq57zSkW3knRjOm4PSNo5rb/GfbS+z6m/96T70TLgnNo2pMfEcklPSTq8lz42vF+k6ZGSfpH23TJJd9W2P91HfiqpJ7VxaqmOcyRdI+lHkl4ETujvPlvfOfgHSUQ8CCwCDkyzzqc4ixoH7AJsD5yVls0CppZW/wjwXETMaVD1D4HNgT2AdwEXAEgaD8wETgK2Af4duF7S2+orkDQS+CnwT8BI4L+AA5psyqHAQanvIyie0J6PiBnAlcA301DUEaV1plKcKY6IiFUN6jwK+AnFmeWPgeskbdKkfQAi4mXgcKA7tbdlRHTXbdd7KPblaUAHcBNwg6RNS8U+BhwG7AS8j4E92CcD+wJ7p3o+0qTc2cBY4C+AQ4C/a1DmzX1E8aR7A3ArxTH9InClpN0G0LejgU5gPMX+/UyjQhFxTvq7IyImptkHA4vSfbY3U4F/BrYC5gNfH0D/9gMWUGzf10vznqC4D34TuKzZk2kfTqd4rHUA2wJnApHC/wbgIYrH28HAaZLKx+0o4BqK43Blk/2zwXHwD65uYOt0Z/4c8PcRsSwiXgK+AUxJ5X4MHClp83T742neGiRtRxF+J0fE8oj4c0T8Ni3+HPDvEfFARLweEVcAfwLe36Bfk4DfR8Q1EfFn4H8D/6/JNvwZGA78JaCIeCwiFvex3RdGxMKIeLXJ8tmltr8LbNaknwN1HHBjRNyW6v428HbgA3V9646IZRShMG4A9Z8XES9ExLPAb3pZ92PAN9IxWgRc2KBMeR+9H9gy1f9aRPwa+AVrngz05fx033qW4ngOZN1tgL6OKcDPIuLB9GR+JQPbd90RcVFErCrdL56JiEvS+1dXANtRBPdA/TmtOyY9Ju6K4kvI9gU6IuLctF8XAJew+nEHcF9EXBcRb/Ryf93gOPgH1/bAMoozkc2B2enl6AvALWk+ETEfeAw4IoX/kTQIfmA0sCwiljdYNgY4vVZ/amM0MKpB2VHAwtqN9CBZ2KAcKYSmA98DlkiaIekdfWx3w7oaLY+INyjO1hr1c6BGAc/U1b2Q4jjUlJ/gXqEI3P7q77pr7F8a74/yvFHAwtTfmmdYs999Kdf3DAPbn89TBGdfquy7Rvvgzfoi4pU0OZA6a75F8Qrk1jTcdUaaP4ZiaLD8mDiTNZ9c+rqvbpAc/INE0r4UD9y7geeAV4E9ImJE+ntnRJTv5LXhnqMozsbnN6h2IcUriBFNln29VP+IiNg8ImY1KLuY4kmh1leVb9eLiAsjYh+K4aX3AF+pLWq2SrO6knLbGwE7ULw6giJQNi+VffcA6u2meLDX6q5t1x/7WK/dFlNsU02jfVvelm5gdN24/I6s7vfLNN8njdrYkdX7sz9uB3aQ1DmAdcpeTv9762PVrwFuer+IiJci4vSI+AvgCOB/SjqY4jHxVN1jYnhETGpjv9ZLDv42k/QOFW/UXgX8KCIeSWdylwAXSHpXKrd93VjjVRTj6Z+n8dk+aYjlZuD7kraStImkg9LiS4CTJe2nwhaSPippeIOqbgT2kHRMekPuVBqHCZL2TXVuQvEA/2+gdmnpEopx7IHap9T2aRRDUvenZXOBj0vaWNJhwAdL6y0BtpH0zib1/gfwUUkHp/6enuq+t4U+VvEfwD+kY7Q98IU+yj9AsW+/mo7pRIoAuyotnwscI2nz9AbniQ3q+EpqbzTF1TlX97ezEfEk8H1glorLGDeVtJmkKaWz597W76F4kvq7dNw+A+zc3/b7qen9QsVFDbukJ/oXKe6frwMPAi+qeCP97WndPdNJWdYc/O1zg6SXKM4y/pFi7PrTpeVfo3g5en+6guBXwJtv3qVQv49iPLq3B+0nKcY0HweWUgQnEdFFMc4/HVie2jqhUQUR8RzwtxSXnT4P7Arc06S9d1A8qSynGEJ4nmLsHIrLVXdPL6Ova7J+Iz+nGI9fnrbnmDQmD0VoHQG8QHHV0Jv1RsTjFK+MFqQ21xjOiIgnKN5IvYjiVdYRFJfZvjaAvrXDuRTDV09RHOdrKJ6AGkr9O5Li/ZvnKEL4U2l7oXgD/zWKJ74rKMbX6/0cmE0RkDcy8EuJT2X1kN4LFG/4H03xPkh/fI7ileDzFK8M2/1k2/R+QXH//RWwkuIx9P305uzraZ1xFMfiOeBSoNmJQzYU/iEWs0El6fPAlIj4YJ+FW6s/gF2bDA+avYXP+M3aTNJ2kg6QtFG6JPN04Nqh7pdZTTafVDNbizal+BzFThRDE1dRDN+YrRM81GNmlhkP9ZiZZWadHOoZOXJkjB07dqi7YWa23pg9e/ZzEdHRn7LrZPCPHTuWrq6uoe6Gmdl6Q9IzfZcqeKjHzCwzDn4zs8w4+M3MMuPgNzPLjIPfzCwzDn4zs8w4+M3MMuPgNzPLjIPfzCwz6+Qndwdin6/8YKi7sMGb/a1PDXUXzKyNfMZvZpaZPs/4Jc0EJgNLI2LPNO9qVv9s4AjghYgY12Ddp4GXKH7/clVEtPpjzmZm1ib9Geq5nOK3ON8cU4mI42rTkr4DrOhl/Q+l33g1M7N1QJ/BHxF3ShrbaFn6VfuPAX/d3m6ZmdlgqTrGfyCwJCKebLI8gFslzZY0rbeKJE2T1CWpq6enp2K3zMysmarBPxWY1cvyAyJiPHA4cIqkg5oVjIgZEdEZEZ0dHf36LQEzM2tBy8EvaRhwDHB1szIR0Z3+LwWuBSa02p6ZmbVHlTP+DwOPR8SiRgslbSFpeG0aOBSYV6E9MzNrgz6DX9Is4D5gN0mLJJ2YFk2hbphH0ihJN6Wb2wJ3S3oIeBC4MSJuaV/XzcysFf25qmdqk/knNJjXDUxK0wuAvSv2z8zM2syf3DUzy4yD38wsMw5+M7PMOPjNzDLj4Dczy4yD38wsMw5+M7PMOPjNzDLj4Dczy4yD38wsMw5+M7PMOPjNzDLj4Dczy4yD38wsMw5+M7PMOPjNzDLj4Dczy4yD38wsMw5+M7PMOPjNzDLTZ/BLmilpqaR5pXnnSPqjpLnpb1KTdQ+T9ISk+ZLOaGfHzcysNf05478cOKzB/AsiYlz6u6l+oaSNge8BhwO7A1Ml7V6ls2ZmVl2fwR8RdwLLWqh7AjA/IhZExGvAVcBRLdRjZmZtVGWM/wuSHk5DQVs1WL49sLB0e1Ga15CkaZK6JHX19PRU6JaZmfWm1eD/N2BnYBywGPhOgzJqMC+aVRgRMyKiMyI6Ozo6WuyWmZn1paXgj4glEfF6RLwBXEIxrFNvETC6dHsHoLuV9szMrH1aCn5J25VuHg3Ma1Dsd8CuknaStCkwBbi+lfbMzKx9hvVVQNIsYCIwUtIi4GxgoqRxFEM3TwMnpbKjgEsjYlJErJL0BeCXwMbAzIh4dFC2wszM+q3P4I+IqQ1mX9akbDcwqXT7JuAtl3qamdnQ8Sd3zcwy4+A3M8uMg9/MLDMOfjOzzDj4zcwy4+A3M8uMg9/MLDMOfjOzzDj4zcwy4+A3M8uMg9/MLDMOfjOzzDj4zcwy4+A3M8uMg9/MLDMOfjOzzDj4zcwy4+A3M8uMg9/MLDMOfjOzzPQZ/JJmSloqaV5p3rckPS7pYUnXShrRZN2nJT0iaa6krnZ23MzMWtOfM/7LgcPq5t0G7BkR7wP+APxDL+t/KCLGRURna100M7N26jP4I+JOYFndvFsjYlW6eT+wwyD0zczMBkE7xvg/A9zcZFkAt0qaLWlab5VImiapS1JXT09PG7plZmaNVAp+Sf8IrAKubFLkgIgYDxwOnCLpoGZ1RcSMiOiMiM6Ojo4q3TIzs160HPySjgcmA5+IiGhUJiK60/+lwLXAhFbbMzOz9mgp+CUdBnwNODIiXmlSZgtJw2vTwKHAvEZlzcxs7enP5ZyzgPuA3SQtknQiMB0YDtyWLtW8OJUdJemmtOq2wN2SHgIeBG6MiFsGZSvMzKzfhvVVICKmNph9WZOy3cCkNL0A2LtS78zMrO38yV0zs8w4+M3MMuPgNzPLjIPfzCwzDn4zs8w4+M3MMuPgNzPLjIPfzCwzDn4zs8w4+M3MMuPgNzPLjIPfzCwzDn4zs8w4+M3MMuPgNzPLjIPfzCwzDn4zs8w4+M3MMuPgNzPLjIPfzCwz/Qp+STMlLZU0rzRva0m3SXoy/d+qybrHpzJPSjq+XR03M7PW9PeM/3LgsLp5ZwC3R8SuwO3p9hokbQ2cDewHTADObvYEYWZma0e/gj8i7gSW1c0+CrgiTV8B/E2DVT8C3BYRyyJiOXAbb30CMTOztajKGP+2EbEYIP1/V4My2wMLS7cXpXlvIWmapC5JXT09PRW6ZWZmvRnsN3fVYF40KhgRMyKiMyI6Ozo6BrlbZmb5qhL8SyRtB5D+L21QZhEwunR7B6C7QptmZlZRleC/HqhdpXM88PMGZX4JHCppq/Sm7qFpnpmZDZH+Xs45C7gP2E3SIkknAucBh0h6Ejgk3UZSp6RLASJiGfAvwO/S37lpnpmZDZFh/SkUEVObLDq4Qdku4LOl2zOBmS31zszM2s6f3DUzy4yD38wsMw5+M7PMOPjNzDLj4Dczy0y/ruoxGwzPnrvXUHchCzue9chQd8HWMT7jNzPLjIPfzCwzDn4zs8w4+M3MMuPgNzPLjIPfzCwzDn4zs8w4+M3MMuPgNzPLjIPfzCwzDn4zs8w4+M3MMuPgNzPLTMvBL2k3SXNLfy9KOq2uzERJK0plzqreZTMzq6Llr2WOiCeAcQCSNgb+CFzboOhdETG51XbMzKy92jXUczDwXxHxTJvqMzOzQdKu4J8CzGqybH9JD0m6WdIebWrPzMxaVDn4JW0KHAn8pMHiOcCYiNgbuAi4rpd6pknqktTV09NTtVtmZtZEO874DwfmRMSS+gUR8WJErEzTNwGbSBrZqJKImBERnRHR2dHR0YZumZlZI+0I/qk0GeaR9G5JStMTUnvPt6FNMzNrUaUfW5e0OXAIcFJp3skAEXExcCzweUmrgFeBKRERVdo0M7NqKgV/RLwCbFM37+LS9HRgepU2zMysvfzJXTOzzDj4zcwy4+A3M8uMg9/MLDMOfjOzzDj4zcwy4+A3M8uMg9/MLDMOfjOzzDj4zcwy4+A3M8uMg9/MLDMOfjOzzDj4zcwy4+A3M8uMg9/MLDMOfjOzzDj4zcwy4+A3M8uMg9/MLDOVg1/S05IekTRXUleD5ZJ0oaT5kh6WNL5qm2Zm1rphbarnQxHxXJNlhwO7pr/9gH9L/83MbAisjaGeo4AfROF+YISk7dZCu2Zm1kA7gj+AWyXNljStwfLtgYWl24vSvDVImiapS1JXT09PG7plZmaNtCP4D4iI8RRDOqdIOqhuuRqsE2+ZETEjIjojorOjo6MN3TIzs0YqB39EdKf/S4FrgQl1RRYBo0u3dwC6q7ZrZmatqRT8kraQNLw2DRwKzKsrdj3wqXR1z/uBFRGxuEq7ZmbWuqpX9WwLXCupVtePI+IWSScDRMTFwE3AJGA+8Arw6YptmplZBZWCPyIWAHs3mH9xaTqAU6q0Y2Zm7eNP7pqZZcbBb2aWGQe/mVlmHPxmZplx8JuZZcbBb2aWmXZ9O6eZZeaAiw4Y6i5s8O754j2DUq/P+M3MMuPgNzPLjIPfzCwzDn4zs8w4+M3MMuPgNzPLjIPfzCwzDn4zs8w4+M3MMuPgNzPLjIPfzCwzDn4zs8w4+M3MMtNy8EsaLek3kh6T9KikLzUoM1HSCklz099Z1bprZmZVVfla5lXA6RExR9JwYLak2yLi93Xl7oqIyRXaMTOzNmr5jD8iFkfEnDT9EvAYsH27OmZmZoOjLWP8ksYCfwU80GDx/pIeknSzpD16qWOapC5JXT09Pe3olpmZNVA5+CVtCfwUOC0iXqxbPAcYExF7AxcB1zWrJyJmRERnRHR2dHRU7ZaZmTVRKfglbUIR+ldGxM/ql0fEixGxMk3fBGwiaWSVNs3MrJoqV/UIuAx4LCK+26TMu1M5JE1I7T3faptmZlZdlat6DgA+CTwiaW6adyawI0BEXAwcC3xe0irgVWBKRESFNs3MrKKWgz8i7gbUR5npwPRW2zAzs/bzJ3fNzDLj4Dczy4yD38wsMw5+M7PMOPjNzDLj4Dczy4yD38wsMw5+M7PMOPjNzDLj4Dczy4yD38wsMw5+M7PMOPjNzDLj4Dczy4yD38wsMw5+M7PMOPjNzDLj4Dczy4yD38wsMw5+M7PMVAp+SYdJekLSfElnNFj+NklXp+UPSBpbpT0zM6uu5eCXtDHwPeBwYHdgqqTd64qdCCyPiF2AC4DzW23PzMzao8oZ/wRgfkQsiIjXgKuAo+rKHAVckaavAQ6WpAptmplZRcMqrLs9sLB0exGwX7MyEbFK0gpgG+C5+sokTQOmpZsrJT1RoW/rspE02P51mb59/FB3YV2y3h0/zva5Vsl6dfx06oCO3Zj+FqwS/I16FC2UKWZGzABmVOjPekFSV0R0DnU/rDU+fus3H79ClaGeRcDo0u0dgO5mZSQNA94JLKvQppmZVVQl+H8H7CppJ0mbAlOA6+vKXA/UxgmOBX4dEQ3P+M3MbO1oeagnjdl/AfglsDEwMyIelXQu0BUR1wOXAT+UNJ/iTH9KOzq9ntvgh7M2cD5+6zcfP0A+ATczy4s/uWtmlhkHv5lZZhz8TUi6QNJppdu/lHRp6fZ3JJ0p6ZoB1nuCpOnt7Ku1TtLRkkLSX1ao43JJx7azX9Y/klam/6MG+ljMmYO/uXuBDwBI2ojigx97lJZ/ALg9IvyAX79NBe7GFx6s1yKi24/F/nPwN3cPKfgpAn8e8JKkrSS9DXgvsFzSPHjzTP5nkm6R9KSkb9YqkvRpSX+Q9FvggNL8MZJul/Rw+r+jpI0lLVBhhKQ3JB2Uyt8laRdJH5Q0N/39p6Tha2mfbFAkbUlxPE4kBb+kiZLulHStpN9Lujg98SNpZXqlNycdr44Gde4j6beSZqdXidut1Y3KlKSx/XwsrixNHyvp8jT9c0mfStMnSbpyLW/CWuXgbyIiuoFVknakeAK4D3gA2B/oBB4GXqtbbRxwHLAXcJyk0emB/88UAXMIxRfa1UwHfhAR7wOuBC6MiNeBP6Ry/wOYDRyYnmx2iIj5wJeBUyJiHHAg8Gq7tz8TfwPcEhF/AJZJGp/mTwBOpziOOwPHpPlbAHMiYjzwW+DscmWSNgEuAo6NiH2AmcDXB30rrJG3PBb7KD8NOEvSgRTH/ouD3L8h5eDvXe2svxb895Vu39ug/O0RsSIi/hv4PcV3Z+wH3BERPenL7K4uld8f+HGa/iFF0APcBRyU/v5Xmr8vxYfmav36rqRTgRERsaoN25qjqRRfLkj6PzVNP5i+fPB1YBarj8sbrD5+PyrNr9kN2BO4TdJc4J8oPtFua1+jx2JTEbEEOAv4DXB6RGzQ3zBQ5bt6clAb59+LYqhnIcXZwIsUZ3P1/lSafp3V+7e/H5aolbsLOBkYRXFn/AowEbgTICLOk3QjMAm4X9KHI+LxfrZhgKRtgL8G9pQUFB9CDOAm3nq8mh2/Rt9N9WhE7N/OvlpL+vNY3Kxunb2A5ykedxs0n/H37h5gMrAsIl5PZwEjKM7U7+tnHQ8AEyVtk4YC/ra07F5Wv6n4CYo3GWvrfAB4I52xzAVOonhCQNLOEfFIRJwPdAEtX5GSsWMphtnGRMTYiBgNPEVxFj8hfRXJRhTDBbXjslFaD+Djpfk1TwAdkvaHYuhH0h7YumSJpPemY3t0baakCRS/LfJXwJcl7TRUHVwbHPy9e4Tiap776+atiIh+fbVrRCwGzqF4ovgVMKe0+FTg05IeBj4JfCmt8yeKVxe1du8Chqe2AU6TNE/SQxTj+zcPeMtsKnBt3byfUgT6fcB5FK/yniqVexnYQ9JsilcL55ZXTkN5xwLnp2Mzl9UXCNi64QzgF8CvgcVQ/FIgcAnwmfTe3unATGnD/e0Qf2WDWYmkicCXI2Jyg2UrI2LLtd8rs/byGb+ZWWZ8xm9mlhmf8ZuZZcbBb2aWGQe/mVlmHPxmZplx8JuZZeb/A02I5zls4kFsAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f2e946bcfd0>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.barplot(['Windows','Apple','linux'], [df_ubc.select('Device').where((df_ubc.Device =='w') & (df_ubc.y ==0)).count(), \\\n",
    "                                         df_ubc.select('Device').where((df_ubc.Device =='m') & (df_ubc.y ==0)).count(),\\\n",
    "                                         df_ubc.select('Device').where((df_ubc.Device =='l') & (df_ubc.y ==0)).count()])\n",
    "plt.title('Device distribution in group \"Churn user\"')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.3.3 `PlayedTime`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 142,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_ubc = df_ubc.fillna(0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 143,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x7f2e8c6093c8>"
      ]
     },
     "execution_count": 143,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD8CAYAAABn919SAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAGANJREFUeJzt3X9wVfWd//Hni0ATqqyipB0QMWitFdZuging0O4gtlRsS8WvTv2NLjOxdbVl3H6rbv/QnWk7dksr42x/iFVhvkqtrXXqMLqFuinUjoMkEik0sIClGqEQ6LLKfKst+t4/7kn2irnJ/Zmbe3w9ZjI593PPOZ83h5tXTj7nlyICMzOrfaOqXYCZmZWHA93MLCUc6GZmKeFANzNLCQe6mVlKONDNzFLCgW5mlhIOdDOzlHCgm5mlxOjh7GzChAnR1NQ0nF2amdW8zs7OgxHRONR8wxroTU1NdHR0DGeXZmY1T9If8pnPQy5mZinhQDczSwkHuplZSgzrGLqZ1aa//vWv9PT08Prrr1e7lFRraGhg8uTJjBkzpqjlHehmNqSenh7GjRtHU1MTkqpdTipFBIcOHaKnp4epU6cWtQ4PuZjZkF5//XVOPvlkh3kFSeLkk08u6a8gB7qZ5cVhXnmlbmMHuplZSngM3cwKtnrjS2Vd35Wzpgw5z9e//nVWr15NXV0do0aN4t5772XWrFksX76ctrY23vve95a1plpUM4Fe7g9Qn3w+SGZWXc8++yxr1qzh+eefp76+noMHD/KXv/wFgOXLl3P11VePyECPCCKCUaOGZzAk714k1UnaLGlN8nqqpI2Sdkr6saT3VK5MM3s327dvHxMmTKC+vh6ACRMmMGnSJO655x727t3L+eefz/nnnw/AF77wBVpbW5k+fTp33HEHAE8//TSLFi3qX9+6deu45JJL3tFPU1MTBw8eBKCjo4O5c+cCsH79epqbm2lubqalpYXXXnsNgG9961t85CMf4cMf/nB/X3v27OHss8/mxhtvZMaMGbz88suV2SgDKOTXxpeA7qzX3wTujogzgf8ClpSzMDOzPvPnz+fll1/mgx/8IDfeeCPr168H4Itf/CKTJk2ivb2d9vZ2IDM009HRwZYtW1i/fj1btmxh3rx5dHd309vbC8CDDz7I9ddfn3f/y5Yt47vf/S5dXV38+te/ZuzYsaxdu5adO3fy3HPP0dXVRWdnJxs2bABgx44dXHvttWzevJnTTjutzFsjt7wCXdJk4FPAD5PXAuYBP01mWQVcXIkCzcyOP/54Ojs7WbFiBY2NjXzuc59j5cqVA8776KOPMmPGDFpaWti2bRu/+93vkMQ111zDQw89xOHDh3n22WdZsGBB3v3PmTOHW265hXvuuYfDhw8zevRo1q5dy9q1a2lpaWHGjBls376dnTt3AnDaaacxe/bscvzTC5LvGPpy4CvAuOT1ycDhiDiavO4BTilzbWZm/erq6pg7dy5z587lnHPOYdWqVVx33XVvm+f3v/89y5YtY9OmTYwfP57rrruu/7zu66+/ns985jM0NDRw2WWXMXr0O+Nv9OjRvPXWWwBvOx/8tttu41Of+hRPPvkks2fP5pe//CURwe23384NN9zwtnXs2bOH4447rsz/+vwMuYcu6dPAgYjozG4eYNbIsXybpA5JHX1/7piZFWLHjh39e78AXV1d/UMZ48aN6x/TfvXVVznuuOM44YQT2L9/P0899VT/MpMmTWLSpEl87Wtfe8cvgj5NTU10dmai7rHHHutv3717N+eccw633norra2tbN++nU9+8pM88MADHDlyBIBXXnmFAwcOlPXfXah89tDnAAslXQQ0AH9DZo/9REmjk730ycDegRaOiBXACoDW1tYBQ9/Mastwnx125MgRbr755v7hjg984AOsWLECgLa2NhYsWMDEiRNpb2+npaWF6dOnc/rppzNnzpy3reeqq66it7eXadOmDdjPHXfcwZIlS/jGN77BrFmz+tuXL19Oe3s7dXV1TJs2jQULFlBfX093dzfnnXcekBkWeuihh6irq6vQVhiaIvLPWElzgS9HxKcl/QR4LCIekfQDYEtEfG+w5VtbW6PYB1z4tEWz6unu7ubss8+udhklu+mmm2hpaWHJkpF7DsdA21pSZ0S0DrVsKSdH3grcImkXmTH1+0tYl5lZRZ177rls2bKFq6++utqlVExBFxZFxK+AXyXTLwIzy1+SmVn59Y2Np5nv5WJmlhIOdDOzlHCgm5mlhAPdzCwlauZui2Y2gnQ8WN71tQ59X5U//vGPLF26lE2bNlFfX09TUxPLly9n7969LFu2jDVr1pS3phrkQDezES8iWLRoEYsXL+aRRx4BMleL7t+/v+R1Hz16dMDbAAyHct9e10MuZjbitbe3M2bMGD7/+c/3tzU3N/Oxj30MyFxJeumll/KhD32Iq666ir4LJnPdDvfOO++kra2N+fPnc+2117Jy5UouueQSLrzwQs4880y+8pWvDFjHSL+9rvfQzWzE27p1K+eee27O9zdv3sy2bduYNGkSc+bM4Te/+Q0f/ehHB11nZ2cnzzzzDGPHjmXlypV0dXWxefNm6uvrOeuss7j55ps59dRT86qv7/a6c+bM4ciRIzQ0NLzt9roRwcKFC9mwYQNTpkxhx44dPPjgg3zve4NeXF8w76GbWc2bOXMmkydPZtSoUTQ3N7Nnz54hl1m4cCFjx47tf33BBRdwwgkn0NDQwLRp0/jDH/6Qd/8j5fa6DnQzG/GmT58+6JWefU8ygsxtdo8ezdzZO9ftcIF33OI21zqyDXZ73R/+8If8+c9/Zvbs2Wzfvr3/9rpdXV10dXWxa9eu/nvIVOr2ug50Mxvx5s2bxxtvvMF9993X37Zp06b+Jxflkut2uMUa6bfX9Ri6mRUuj9MMy0kSjz/+OEuXLuWuu+6ioaGh/7TFV155JedyuW6HW6yRfnvdgm6fWyrfPtesNqXl9rm1oFq3zzUzsxHEgW5mlhIOdDPLy3AOz75blbqN83lIdIOk5yS9IGmbpH9J2ldK+r2kruSruaRKzGzEamho4NChQw71CooIDh06RENDQ9HryOcslzeAeRFxRNIY4BlJfY/S/r8R8dOiezezmjB58mR6enro7e2tdimp1tDQwOTJk4tefshAj8yv5CPJyzHJl39Nm72LjBkzhqlTp1a7DBtCXmPokuokdQEHgHURsTF56+uStki6W1L9IKswM7MKyyvQI+LNiGgGJgMzJf0tcDvwIeAjwEnArQMtK6lNUoekDv+5ZmZWOQWd5RIRh4FfARdGxL7IeAN4EJiZY5kVEdEaEa2NjY0lF2xmZgPL5yyXRkknJtNjgY8D2yVNTNoEXAxsrWShZmY2uHzOcpkIrJJUR+YXwKMRsUbSf0hqBAR0AZ8fbCVmZlZZ+ZzlsgVoGaB9XkUqMjOzovhKUTOzlHCgm5mlhAPdzCwlHOhmZinhQDczSwkHuplZSjjQzcxSwoFuZpYSDnQzs5RwoJuZpYQD3cwsJRzoZmYp4UA3M0sJB7qZWUo40M3MUsKBbmaWEg50M7OUyOeZog2SnpP0gqRtkv4laZ8qaaOknZJ+LOk9lS/XzMxyyWcP/Q1gXkT8HdAMXChpNvBN4O6IOBP4L2BJ5co0M7OhDBnokXEkeTkm+QpgHvDTpH0VcHFFKjQzs7zkNYYuqU5SF3AAWAfsBg5HxNFklh7glBzLtknqkNTR29tbjprNzGwAeQV6RLwZEc3AZGAmcPZAs+VYdkVEtEZEa2NjY/GVmpnZoAo6yyUiDgO/AmYDJ0oanbw1Gdhb3tLMzKwQ+Zzl0ijpxGR6LPBxoBtoBy5NZlsM/LxSRZqZ2dBGDz0LE4FVkurI/AJ4NCLWSPod8IikrwGbgfsrWKeZmQ1hyECPiC1AywDtL5IZT69pqze+VJH1XjlrSkXWa2aWi68UNTNLCQe6mVlKONDNzFLCgW5mlhIOdDOzlHCgm5mlhAPdzCwlHOhmZinhQDczSwkHuplZSjjQzcxSwoFuZpYSDnQzs5RwoJuZpYQD3cwsJRzoZmYpkc8j6E6V1C6pW9I2SV9K2u+U9IqkruTrosqXa2ZmueTzCLqjwD9FxPOSxgGdktYl790dEcsqV56ZmeUrn0fQ7QP2JdOvSeoGTql0YWZmVpiCxtAlNZF5vujGpOkmSVskPSBpfJlrMzOzAuQd6JKOBx4DlkbEq8D3gTOAZjJ78N/OsVybpA5JHb29vWUo2czMBpJXoEsaQybMH46InwFExP6IeDMi3gLuA2YOtGxErIiI1ohobWxsLFfdZmZ2jHzOchFwP9AdEd/Jap+YNdsiYGv5yzMzs3zlc5bLHOAa4LeSupK2fwaukNQMBLAHuKEiFZqZWV7yOcvlGUADvPVk+csxM7Ni+UpRM7OUcKCbmaWEA93MLCUc6GZmKeFANzNLiXxOW7QRYvXGlyqy3itnTanIes1seHkP3cwsJRzoZmYp4UA3M0sJB7qZWUo40M3MUsKBbmaWEg50M7OUcKCbmaWEA93MLCUc6GZmKZHPI+hOldQuqVvSNklfStpPkrRO0s7k+/jKl2tmZrnks4d+FPiniDgbmA38o6RpwG3A0xFxJvB08trMzKpkyECPiH0R8Xwy/RrQDZwCfBZYlcy2Cri4UkWamdnQChpDl9QEtAAbgfdHxD7IhD7wvnIXZ2Zm+cs70CUdDzwGLI2IVwtYrk1Sh6SO3t7eYmo0M7M85BXoksaQCfOHI+JnSfN+SROT9ycCBwZaNiJWRERrRLQ2NjaWo2YzMxtAPme5CLgf6I6I72S99QSwOJleDPy8/OWZmVm+8nli0RzgGuC3krqStn8G7gIelbQEeAm4rDIlmplZPoYM9Ih4BlCOty8obzlmZlYsXylqZpYSDnQzs5RwoJuZpYQD3cwsJRzoZmYp4UA3M0sJB7qZWUo40M3MUsKBbmaWEg50M7OUcKCbmaWEA93MLCUc6GZmKeFANzNLCQe6mVlK5POAixHjjJd+UvE+dk8pz3M6Vm98qSzrMTPLVz6PoHtA0gFJW7Pa7pT0iqSu5OuiypZpZmZDyWfIZSVw4QDtd0dEc/L1ZHnLMjOzQg0Z6BGxAfjTMNRiZmYlKOWg6E2StiRDMuPLVpGZmRWl2ED/PnAG0AzsA76da0ZJbZI6JHX09vYW2Z2ZmQ2lqECPiP0R8WZEvAXcB8wcZN4VEdEaEa2NjY3F1mlmZkMoKtAlTcx6uQjYmmteMzMbHkOehy7pR8BcYIKkHuAOYK6kZiCAPcANFazRzMzyMGSgR8QVAzTfX4FazMysBL7038wsJRzoZmYp4UA3M0sJB7qZWUo40M3MUsKBbmaWEg50M7OUcKCbmaWEA93MLCUc6GZmKeFANzNLCQe6mVlKONDNzFLCgW5mlhIOdDOzlBjyfuiWfqs3vlSR9V45a0pF1mtmA/MeuplZSgwZ6JIekHRA0tastpMkrZO0M/k+vrJlmpnZUPLZQ18JXHhM223A0xFxJvB08trMzKpoyECPiA3An45p/iywKpleBVxc5rrMzKxAxY6hvz8i9gEk39+Xa0ZJbZI6JHX09vYW2Z2ZmQ2l4gdFI2JFRLRGRGtjY2OluzMze9cqNtD3S5oIkHw/UL6SzMysGMUG+hPA4mR6MfDz8pRjZmbFyue0xR8BzwJnSeqRtAS4C/iEpJ3AJ5LXZmZWRUNeKRoRV+R464Iy12JmZiXwlaJmZinhQDczSwkHuplZSjjQzcxSwoFuZpYSDnQzs5RwoJuZpYQD3cwsJRzoZmYp4WeKHuOMl35S8T52T7ms4n2Y2buP99DNzFLCgW5mlhIOdDOzlHCgm5mlhAPdzCwlHOhmZilR0mmLkvYArwFvAkcjorUcRZmZWeHKcR76+RFxsAzrMTOzEnjIxcwsJUoN9ADWSuqU1FaOgszMrDilDrnMiYi9kt4HrJO0PSI2ZM+QBH0bwJQpU0rszszMcilpDz0i9ibfDwCPAzMHmGdFRLRGRGtjY2Mp3ZmZ2SCKDnRJx0ka1zcNzAe2lqswMzMrTClDLu8HHpfUt57VEfHvZanKzMwKVnSgR8SLwN+VsRYzMyuB74duFbN640sVWe+Vs3xw3WwgPg/dzCwlHOhmZinhQDczSwkHuplZSvigaBUMx4Ooh4Mfdm02sngP3cwsJRzoZmYp4UA3M0sJB7qZWUr4oKgVbTgO7g504LVSV6DWkkpdLVuJbesre4eP99DNzFLCgW5mlhIOdDOzlHCgm5mlhA+KmtUgHxiunFq+7XNJe+iSLpS0Q9IuSbeVqygzMytcKc8UrQO+CywApgFXSJpWrsLMzKwwpeyhzwR2RcSLEfEX4BHgs+Upy8zMClVKoJ8CvJz1uidpMzOzKijloKgGaIt3zCS1AW3JyyOSdhTZ3wTgYJHLVlut1j4C6v5yMQuNgLqLVqu156z7qmEupAjDss1L3A6n5TNTKYHeA5ya9XoysPfYmSJiBbCihH4AkNQREa2lrqcaarV21z38arX2Wq0barv2Y5Uy5LIJOFPSVEnvAS4HnihPWWZmVqii99Aj4qikm4BfAHXAAxGxrWyVmZlZQUq6sCgingSeLFMtQyl52KaKarV21z38arX2Wq0barv2t1HEO45jmplZDfK9XMzMUqImAr0atxiQdKqkdkndkrZJ+lLSfqekVyR1JV8XZS1ze1LjDkmfHKr+5IDyRkk7Jf04ObiMpPrk9a7k/aYi6t8j6bdJjR1J20mS1iX9rZM0PmmXpHuS/rZImpG1nsXJ/DslLc5qPzdZ/65kWQ3WR541n5W1XbskvSpp6Ujd5pIekHRA0tastqpt48H6yKPub0naniz3uKQTk/YmSX/O2vY/qER9ubZBnrVX9fORq4+qiIgR/UXmgOtu4HTgPcALwLRh6HciMCOZHgf8J5lbHNwJfHmA+acltdUDU5Oa6warH3gUuDyZ/gHwhWT6RuAHyfTlwI+LqH8PMOGYtn8FbkumbwO+mUxfBDxF5tqC2cDGpP0k4MXk+/hkenzy3nPAeckyTwELBuujyP/3P5I5/3ZEbnPg74EZwNaRsI1z9ZFn3fOB0cn0N7PW2ZQ93zHrKUt9g22DPGuv2ucjVx+Vzqecn8lqdZx3gZkPzC+yXt8O3F6FOn4OfGKQD8/b6iJz9s95uepPPtQHs36I+ufrWzaZHp3MpwLr3cM7A30HMDGZngjsSKbvBa44dj7gCuDerPZ7k7aJwPas9v75cvVRxPaeD/wmmR6x25xjAq+a2zhXH/nUfcx7i4CHB5uvnPXl2gYFbPOqfT5y9VHMZ74cX7Uw5FL1Wwwkf161ABuTppuSPxkf0P8OKeSqM1f7ycDhiDh6TPvb1pW8/9/J/IUIYK2kTmWu1gV4f0TsS9a7D3hfkbWfkkwf2z5YH4W6HPhR1uta2OZQ3W1crp+VfyCzJ91nqqTNktZL+lhWX+Wqrxx1V+vzUfV8ylYLgZ7XLQYq1rl0PPAYsDQiXgW+D5wBNAP7gG/3zTrA4lFE+2DrKsSciJhB5m6Y/yjp7weZt5y1lywZt1wI9D2Fula2+WCGYxuX/G+Q9FXgKPBw0rQPmBIRLcAtwGpJf1Pm+kqtu5qfj6rm07FqIdDzusVAJUgaQybMH46InwFExP6IeDMi3gLuI3PXycHqzNV+EDhR0uhj2t+2ruT9E4A/FVJ7ROxNvh8AHk/q3C9pYrLeicCBImvvSaaPbWeQPgqxAHg+IvYn/4aa2OaJam7jkn5WkoORnwauimT8ICLeiIhDyXQnmTHiD5a5vpLqrvLno2r5NJBaCPSq3GIgOWJ/P9AdEd/Jap+YNdsioO9o+xPA5cnR8KnAmWQOGg1Yf/ID0w5cmiy/mMw4fd+6FifTlwL/0fcDlmftx0ka1zdNZjx66zHrPba/a5OzEGYD/538qfwLYL6k8cmfsfPJjCnuA16TNDvZTtfmqD27j0JcQdZwSy1s8yzV3Ma5+hiSpAuBW4GFEfH/s9oblXn2AZJOJ7ONXyxzfQNug3zqTuqq5ucjVx/VUa3B+0K+yBwd/08yewdfHaY+P0rmT6ctQFfydRHw/4DfJu1PkHXQCfhqUuMOkiP+g9VP5ij7c8AuMsML9Ul7Q/J6V/L+6QXWfjqZI+8vANv6+iQz5vc0sDP5flLSLjIPK9md/Ntas9b1D0kdu4Drs9pbyfzg7Ab+jf+9SG3APgqo/b3AIeCErLYRuc3J/NLZB/yVzJ7akmpu48H6yKPuXWTGgvs+631ndPyf5DP0AvA88JlK1JdrG+RZe1U/H7n6qMaXrxQ1M0uJWhhyMTOzPDjQzcxSwoFuZpYSDnQzs5RwoJuZpYQD3cwsJRzoZmYp4UA3M0uJ/wFCr8IQ3/gueAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f2e8c6bca20>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# there is a None in the list, replace it with 0\n",
    "\n",
    "sns.distplot( [pt.PlayedTime for pt in df_ubc.select('PlayedTime').where(df_ubc.y == 1).collect()], label=\"Stay user\",kde = False)\n",
    "sns.distplot( [pt.PlayedTime for pt in df_ubc.select('PlayedTime').where(df_ubc.y == 0).collect()], label=\"Churn user\",kde = False)\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.3.4 `PaidTime`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x7f2e945d4978>"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD8CAYAAABn919SAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAFzhJREFUeJzt3X9wVfWd//Hny4AJRb6KknbAiMHWWmHtJpgCDu1+EVsrdmvV0anWKrrsxNbVlnG/q7I789WdaTt2i5Vxpu2KP4AZa62rdXUYu4XaCLXjIImkFAx8AaUaoRDYpcpOtaLv7x/3kI2acG9y703u/fh6zNy553zuuee8P+HmlcO553yOIgIzM6t+R410AWZmVhoOdDOzRDjQzcwS4UA3M0uEA93MLBEOdDOzRDjQzcwS4UA3M0uEA93MLBGjhnNjEyZMiMbGxuHcpJlZ1evo6NgXEfX5lhvWQG9sbKS9vX04N2lmVvUk/b6Q5XzIxcwsEQ50M7NEONDNzBIxrMfQzaw6vfXWW3R3d/PGG2+MdClJq6uro6GhgdGjRw/p/Q50M8uru7ubcePG0djYiKSRLidJEcH+/fvp7u5mypQpQ1pHwYdcJNVI2iBpZTY/RdI6Sdsk/VTS0UOqwMwq3htvvMEJJ5zgMC8jSZxwwglF/S9oMMfQvwl09Zn/LnBnRJwK/BewYMhVmFnFc5iXX7E/44ICXVID8AXg3mxewFzgkWyRFcCFRVViZmZFKfQY+hLgJmBcNn8CcCAiDmXz3cCJJa7NzCrUg+teLun6vjJzct5lvv3tb/Pggw9SU1PDUUcdxd13383MmTNZsmQJra2tfOhDHyppTdUob6BL+mtgb0R0SJpzuLmfRfu927SkVqAVYPLk/P9oNnil/uWCwn7BzIbLs88+y8qVK3n++eepra1l3759/PnPfwZgyZIlfPWrX63IQI8IIoKjjhqeM8QL2cps4AJJO4GHyB1qWQIcJ+nwH4QGYFd/b46IpRHREhEt9fV5hyIwM3uf3bt3M2HCBGprawGYMGECkyZN4q677mLXrl2cffbZnH322QB8/etfp6WlhWnTpnHrrbcC8NRTT3HRRRf1rm/16tVcfPHF79tOY2Mj+/btA6C9vZ05c+YAsGbNGpqammhqaqK5uZnXX38dgO9973t86lOf4pOf/GTvtnbu3Mnpp5/Oddddx/Tp03nllVfK80PpR95Aj4hFEdEQEY3AZcCvIuIKoA24JFtsPvB42ao0sw+0c889l1deeYWPf/zjXHfddaxZswaAb3zjG0yaNIm2tjba2tqA3KGZ9vZ2Nm7cyJo1a9i4cSNz586lq6uLnp4eAJYtW8Y111xT8PYXL17MD37wAzo7O/n1r3/NmDFjWLVqFdu2beO5556js7OTjo4O1q5dC8DWrVu56qqr2LBhAyeffHKJfxoDK+b/ATcDN0raTu6Y+n2lKcnM7N2OOeYYOjo6WLp0KfX19Xz5y19m+fLl/S778MMPM336dJqbm9m8eTMvvPACkrjyyit54IEHOHDgAM8++yzz5s0rePuzZ8/mxhtv5K677uLAgQOMGjWKVatWsWrVKpqbm5k+fTpbtmxh27ZtAJx88snMmjWrFF0flEFdWBQRTwNPZ9MvAjNKX5KZ2fvV1NQwZ84c5syZwxlnnMGKFSu4+uqr37XMSy+9xOLFi1m/fj3jx4/n6quv7j2v+5prruGLX/widXV1XHrppYwa9f74GzVqFO+88w7Au84Hv+WWW/jCF77Ak08+yaxZs/jlL39JRLBo0SKuvfbad61j586djB07tsS9L4zHcjGzird169bevV+Azs7O3kMZ48aN6z2m/dprrzF27FiOPfZY9uzZw89//vPe90yaNIlJkybxrW99631/CA5rbGyko6MDgEcffbS3fceOHZxxxhncfPPNtLS0sGXLFj7/+c9z//33c/DgQQBeffVV9u7dW9J+D5Yv/TezQRvus6AOHjzIDTfc0Hu442Mf+xhLly4FoLW1lXnz5jFx4kTa2tpobm5m2rRpnHLKKcyePftd67niiivo6elh6tSp/W7n1ltvZcGCBXznO99h5syZve1Lliyhra2Nmpoapk6dyrx586itraWrq4uzzjoLyB0WeuCBB6ipqSnTTyE/RfR7tmFZtLS0hG9wUXo+bdHKrauri9NPP32kyyja9ddfT3NzMwsWVO6F7f39rCV1RERLvvd6D93MPhDOPPNMxo4dyx133DHSpZSNA93MPhAOHxtPmb8UNTNLhAPdzCwRDnQzs0Q40M3MEuEvRc1s8NqXlXZ9LfnHVfnDH/7AwoULWb9+PbW1tTQ2NrJkyRJ27drF4sWLWblyZWlrqkIOdDOreBHBRRddxPz583nooYeA3NWie/bsKXrdhw4d6ncYgOFQ6uF1fcjFzCpeW1sbo0eP5mtf+1pvW1NTE5/5zGeA3JWkl1xyCZ/4xCe44oorOHzB5EDD4d522220trZy7rnnctVVV7F8+XIuvvhizjvvPE499VRuuummfuuo9OF1vYduZhVv06ZNnHnmmQO+vmHDBjZv3sykSZOYPXs2v/nNb/j0pz99xHV2dHTwzDPPMGbMGJYvX05nZycbNmygtraW0047jRtuuIGTTjqpoPoOD687e/ZsDh48SF1d3buG140ILrjgAtauXcvkyZPZunUry5Yt44c//OGgfg75eA/dzKrejBkzaGho4KijjqKpqYmdO3fmfc8FF1zAmDFjeufPOeccjj32WOrq6pg6dSq///3vC95+pQyv60A3s4o3bdq0I17pefhORpAbZvfQodztjgcaDhd43xC3A62jryMNr3vvvffypz/9iVmzZrFly5be4XU7Ozvp7Oxk+/btvWPIlGt4XQe6mVW8uXPn8uabb3LPPff0tq1fv773zkUDGWg43KGq9OF1fQzdzAavgNMMS0kSjz32GAsXLuT222+nrq6u97TFV199dcD3DTQc7lBV+vC6eYfPlVQHrAVqyf0BeCQibpW0HPjfwB+zRa+OiM4jrcvD55aHh8+1cktl+NxqUO7hc98E5kbEQUmjgWckHb4NyD9ExCODrtjMzEoub6BHbhf+YDY7OnsM310xzMysIAV9KSqpRlInsBdYHRHrspe+LWmjpDsl1R5hFWZW5Ybz7mYfVMX+jAsK9Ih4OyKagAZghqS/ABYBnwA+BRwP3NzfeyW1SmqX1N7T01NUsWY2Murq6ti/f79DvYwigv3791NXVzfkdQzqLJeIOCDpaeC8iFicNb8paRnwfwZ4z1JgKeS+FB1ypWY2YhoaGuju7sY7ZeVVV1dHQ0PDkN+fN9Al1QNvZWE+Bvgs8F1JEyNityQBFwKbhlyFmVW00aNHM2XKlJEuw/IoZA99IrBCUg25QzQPR8RKSb/Kwl5AJ/C1I63EzMzKq5CzXDYCzf20zy1LRWZmNiS+9N/MLBEOdDOzRDjQzcwS4UA3M0uEA93MLBEOdDOzRDjQzcwS4UA3M0uEA93MLBEOdDOzRDjQzcwS4UA3M0uEA93MLBEOdDOzRDjQzcwS4UA3M0uEA93MLBF5A11SnaTnJP1W0mZJ/5y1T5G0TtI2ST+VdHT5yzUzs4EUsof+JjA3Iv4SaALOkzQL+C5wZ0ScCvwXsKB8ZZqZWT55Az1yDmazo7NHAHOBR7L2FcCFZanQzMwKUtAxdEk1kjqBvcBqYAdwICIOZYt0AycO8N5WSe2S2nt6ekpRs5mZ9aOgQI+ItyOiCWgAZgCn97fYAO9dGhEtEdFSX18/9ErNzOyIBnWWS0QcAJ4GZgHHSRqVvdQA7CptaWZmNhiFnOVSL+m4bHoM8FmgC2gDLskWmw88Xq4izcwsv1H5F2EisEJSDbk/AA9HxEpJLwAPSfoWsAG4r4x1mplZHnkDPSI2As39tL9I7ni6mZlVAF8pamaWCAe6mVkiHOhmZolwoJuZJcKBbmaWCAe6mVkiHOhmZolwoJuZJcKBbmaWCAe6mVkiHOhmZolwoJuZJcKBbmaWCAe6mVkiHOhmZoko5AYXViIPrnt5pEsws4R5D93MLBGF3FP0JEltkrokbZb0zaz9NkmvSurMHueXv1wzMxtIIYdcDgF/HxHPSxoHdEhanb12Z0QsLl95ZmZWqELuKbob2J1Nvy6pCzix3IWZmdngDOoYuqRGcjeMXpc1XS9po6T7JY0f4D2tktoltff09BRVrJmZDazgQJd0DPAosDAiXgN+BHwUaCK3B39Hf++LiKUR0RIRLfX19SUo2czM+lNQoEsaTS7MfxwRPwOIiD0R8XZEvAPcA8woX5lmZpZPIWe5CLgP6IqI7/dpn9hnsYuATaUvz8zMClXIWS6zgSuB30nqzNr+EbhcUhMQwE7g2rJUaGZmBSnkLJdnAPXz0pOlL8fMzIbKV4qamSXCgW5mlggHuplZIhzoZmaJcKCbmSXCgW5mlggHuplZIhzoZmaJcKCbmSXCgW5mlggHuplZIhzoZmaJcKCbmSXCgW5mlggHuplZIhzoZmaJcKCbmSWikHuKniSpTVKXpM2Svpm1Hy9ptaRt2fP48pdrZmYDKWQP/RDw9xFxOjAL+DtJU4FbgKci4lTgqWzezMxGSN5Aj4jdEfF8Nv060AWcCHwJWJEttgK4sFxFmplZfoM6hi6pEWgG1gEfiYjdkAt94MMDvKdVUruk9p6enuKqNTOzARUc6JKOAR4FFkbEa4W+LyKWRkRLRLTU19cPpUYzMytAQYEuaTS5MP9xRPwsa94jaWL2+kRgb3lKNDOzQhRylouA+4CuiPh+n5eeAOZn0/OBx0tfnpmZFWpUAcvMBq4EfiepM2v7R+B24GFJC4CXgUvLU6KZmRUib6BHxDOABnj5nNKWY2ZmQ+UrRc3MEuFANzNLhAPdzCwRDnQzs0Q40M3MEuFANzNLhAPdzCwRDnQzs0Q40M3MEuFANzNLhAPdzCwRDnQzs0Q40M3MEuFANzNLhAPdzCwRhdzgwj6AHlz3csnX+ZWZk0u+znLUCeWp1azcCrkF3f2S9kra1KftNkmvSurMHueXt0wzM8unkEMuy4Hz+mm/MyKasseTpS3LzMwGK2+gR8Ra4D+HoRYzMytCMV+KXi9pY3ZIZnzJKjIzsyEZaqD/CPgo0ATsBu4YaEFJrZLaJbX39PQMcXNmZpbPkAI9IvZExNsR8Q5wDzDjCMsujYiWiGipr68fap1mZpbHkAJd0sQ+sxcBmwZa1szMhkfe89Al/QSYA0yQ1A3cCsyR1AQEsBO4tow1mplZAfIGekRc3k/zfWWoxczMiuBL/83MEuFANzNLhAPdzCwRDnQzs0Q40M3MEuFANzNLhAPdzCwRDnQzs0Q40M3MEuFANzNLhAPdzCwRvkl0odqXFb2Kj748tBs/7Zh8adHbrgTluqGzmeV4D93MLBEOdDOzRDjQzcwS4UA3M0uEA93MLBF5A13S/ZL2StrUp+14Saslbcuex5e3TDMzy6eQPfTlwHnvabsFeCoiTgWeyubNzGwE5Q30iFgLvPcE6i8BK7LpFcCFJa7LzMwGaajH0D8SEbsBsucPl64kMzMbirJ/KSqpVVK7pPaenp5yb87M7ANrqIG+R9JEgOx570ALRsTSiGiJiJb6+vohbs7MzPIZaqA/AczPpucDj5emHDMzG6pCTlv8CfAscJqkbkkLgNuBz0naBnwumzczsxGUd7TFiLh8gJfOKXEtZmZWhOoZPvcIw9eue2low9Ieycwpx5d8nWZm5eRL/83MEuFANzNLhAPdzCwRDnQzs0Q40M3MEuFANzNLhAPdzCwRDnQzs0Q40M3MEuFANzNLhAPdzCwRDnQzs0Q40M3MEuFANzNLhAPdzCwRDnQzs0QUdYMLSTuB14G3gUMR0VKKoszMbPBKcceisyNiXwnWY2ZmRfAhFzOzRBQb6AGsktQhqbUUBZmZ2dAUe8hldkTskvRhYLWkLRGxtu8CWdC3AkyePLnIzQ2fctx4eqg++vK/jdi2d0y+dMS2bWaDU9QeekTsyp73Ao8BM/pZZmlEtERES319fTGbMzOzIxhyoEsaK2nc4WngXGBTqQozM7PBKeaQy0eAxyQdXs+DEfEfJanKzMwGbciBHhEvAn9ZwlrMzKwIpTgP3cwK8OC6l0u+zq/MrJ4TDaz8fB66mVkiHOhmZolwoJuZJcKBbmaWCH8pataPcnyBaVZu3kM3M0uEA93MLBEOdDOzRDjQzcwS4S9FraJ56OAj89Wn1pf30M3MEuFANzNLhAPdzCwRDnQzs0Q40M3MEuGzXOyIRvIsk5H2ge17zfHQcs1IVzFiyjXsw3CcPVTUHrqk8yRtlbRd0i2lKsrMzAavmJtE1wA/AOYBU4HLJU0tVWFmZjY4xeyhzwC2R8SLEfFn4CHgS6Upy8zMBquYQD8ReKXPfHfWZmZmI6CYL0XVT1u8byGpFWjNZg9K2g/sK2K7lWYC6fQnpb6A+1OEvxmOjXyg/n2uKG7dJxeyUDGB3g2c1Ge+Adj13oUiYimw9PC8pPaIaCliuxUlpf6k1Bdwfyqd+1N6xRxyWQ+cKmmKpKOBy4AnSlOWmZkN1pD30CPikKTrgV8ANcD9EbG5ZJWZmdmgFHVhUUQ8CTw5yLctzb9IVUmpPyn1BdyfSuf+lJgi3vc9ppmZVSGP5WJmlohhC/RqGSZA0v2S9kra1KfteEmrJW3Lnsdn7ZJ0V9anjZKm93nP/Gz5bZLmj0RfsjpOktQmqUvSZknfrNY+SaqT9Jyk32Z9+eesfYqkdVldP82+pEdSbTa/PXu9sc+6FmXtWyV9frj70pekGkkbJK3M5qu2P5J2SvqdpE5J7Vlb1X3W+tRxnKRHJG3JfofOquj+RETZH+S+NN0BnAIcDfwWmDoc2x5CrX8FTAc29Wn7F+CWbPoW4LvZ9PnAz8mdkz8LWJe1Hw+8mD2Pz6bHj1B/JgLTs+lxwP8jN1RD1fUpq+mYbHo0sC6r8WHgsqz9X4GvZ9PXAf+aTV8G/DSbnpp9BmuBKdlns2YEP3M3Ag8CK7P5qu0PsBOY8J62qvus9al9BfC32fTRwHGV3J/h+qGcBfyiz/wiYNFI/AMVWG8j7w70rcDEbHoisDWbvhu4/L3LAZcDd/dpf9dyI9y3x4HPVXufgA8BzwMzyV3MMeq9nzVyZ2CdlU2PypbTez9/fZcbgX40AE8Bc4GVWX3V3J+dvD/Qq/KzBvwv4CWy7xqroT/Ddcil2ocJ+EhE7AbInj+ctQ/Ur4rsb/Zf9GZye7ZV2afs8EQnsBdYTW5v9EBEHOqnrt6as9f/CJxAhfQlswS4CXgnmz+B6u5PAKskdSh3lThU6WeN3BGFHmBZdkjsXkljqeD+DFegFzRMQBUaqF8V119JxwCPAgsj4rUjLdpPW8X0KSLejogmcnu2M4DT+1sse67ovkj6a2BvRHT0be5n0aroT2Z2REwnNwrr30n6qyMsW+n9GUXu8OuPIqIZ+G9yh1gGMuL9Ga5AL2iYgAq2R9JEgOx5b9Y+UL8qqr+SRpML8x9HxM+y5qruU0QcAJ4md6zyOEmHr6noW1dvzdnrxwL/SeX0ZTZwgaSd5EYrnUtuj71a+0NE7Mqe9wKPkfujW62ftW6gOyLWZfOPkAv4iu3PcAV6tQ8T8ARw+Jvp+eSOQx9uvyr7dnsW8Mfsv2C/AM6VND77BvzcrG3YSRJwH9AVEd/v81LV9UlSvaTjsukxwGeBLqANuCRb7L19OdzHS4BfRe4g5hPAZdlZI1OAU4HnhqcX/yMiFkVEQ0Q0kvud+FVEXEGV9kfSWEnjDk+T+4xsogo/awAR8QfgFUmnZU3nAC9Qyf0Zxi8Yzid3hsUO4J+G+wuOQdT5E2A38Ba5v6wLyB2nfArYlj0fny0rcjf52AH8Dmjps56/AbZnj2tGsD+fJvffu41AZ/Y4vxr7BHwS2JD1ZRPwf7P2U8gF2Hbg34DarL0um9+evX5Kn3X9U9bHrcC8CvjczeF/znKpyv5kdf82e2w+/HtejZ+1PnU0Ae3ZZ+7fyZ2lUrH98ZWiZmaJ8JWiZmaJcKCbmSXCgW5mlggHuplZIhzoZmaJcKCbmSXCgW5mlggHuplZIv4/yQfJ7khOaqUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f2e94617588>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.distplot( [pt.PaidTime for pt in df_ubc.select('PaidTime').where(df_ubc.y == 1).collect()], label=\"Stay user\",kde = False)\n",
    "sns.distplot( [pt.PaidTime for pt in df_ubc.select('PaidTime').where(df_ubc.y == 0).collect()], label=\"Churn user\",kde = False)\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.3.5 `PlayedNumber`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x7f2e945787f0>"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD8CAYAAABn919SAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAFx9JREFUeJzt3X9wVfWd//Hni4AJRVYR0g4YMdhSC6zdBFPAod1BbKnY1qqj0/oTXXZi62rruPv1x+7Mamfajv2WrYwz/SH+AGYstVbrtw5jv4XSCLXjIIlECgYWVKoRCpEuq3yn2qLv/eMe8o2YkJvce5N7P74eM3fuOZ97frxvuLxy8rnnfI4iAjMzq3wjhrsAMzMrDge6mVkiHOhmZolwoJuZJcKBbmaWCAe6mVkiHOhmZolwoJuZJcKBbmaWiJFDubMJEyZEfX39UO7SzKzitbW1vRYRtf0tN6SBXl9fT2tr61Du0sys4kn6Qz7LucvFzCwRDnQzs0Q40M3MEjGkfehmVpn++te/0tnZyZtvvjncpSStpqaGuro6Ro0aNaj1Hehm1q/Ozk7Gjh1LfX09koa7nCRFBAcOHKCzs5MpU6YMahvucjGzfr355puMHz/eYV5Ckhg/fnxBfwU50M0sLw7z0iv0Z+xANzNLhPvQzWzAVm18uajbu2z25H6X+da3vsWqVauoqqpixIgR3HPPPcyePZulS5fS3NzMBz7wgaLWVIkqJtCL/QE6Ip8PkpkNr6effprVq1fz7LPPUl1dzWuvvcZf/vIXAJYuXcoVV1xRloEeEUQEI0YMTWdI3nuRVCVps6TV2fwUSRsl7ZT0U0nHla5MM3s/27t3LxMmTKC6uhqACRMmMGnSJO6++2727NnD2Wefzdlnnw3AV7/6VZqampgxYwa33347AOvWrePCCy/s3t7atWu56KKL3rOf+vp6XnvtNQBaW1uZN28eAOvXr6ehoYGGhgYaGxt54403APjud7/LJz7xCT7+8Y9372v37t1MmzaN6667jpkzZ/LKK6+U5ofSi4H82vg60NFj/jvAXRExFfgvYHExCzMzO2LBggW88sorfPSjH+W6665j/fr1AHzta19j0qRJtLS00NLSAuS6ZlpbW9myZQvr169ny5YtzJ8/n46ODrq6ugBYvnw511xzTd77X7JkCd///vdpb2/nt7/9LaNHj2bNmjXs3LmTZ555hvb2dtra2tiwYQMAO3bs4KqrrmLz5s2ceuqpRf5p9C2vQJdUB3wOuC+bFzAfeCRbZCVwQSkKNDM7/vjjaWtrY9myZdTW1vKlL32JFStW9Lrsww8/zMyZM2lsbGTbtm08//zzSOLKK6/kwQcf5ODBgzz99NMsXLgw7/3PnTuXm266ibvvvpuDBw8ycuRI1qxZw5o1a2hsbGTmzJls376dnTt3AnDqqacyZ86cYrz1Acm3D30pcDMwNpsfDxyMiMPZfCdwcpFrMzPrVlVVxbx585g3bx5nnHEGK1eu5Oqrr37XMi+99BJLlixh06ZNjBs3jquvvrr7vO5rrrmGL3zhC9TU1HDJJZcwcuR742/kyJG88847AO86H/zWW2/lc5/7HE888QRz5szh17/+NRHBbbfdxrXXXvuubezevZsxY8YU+d3np98jdEmfB/ZHRFvP5l4WjT7Wb5bUKqn1yJ87ZmYDsWPHju6jX4D29vburoyxY8d292m//vrrjBkzhhNOOIF9+/bxy1/+snudSZMmMWnSJL75zW++5xfBEfX19bS15aLu0Ucf7W5/4YUXOOOMM7jllltoampi+/btfPazn+WBBx7g0KFDALz66qvs37+/qO97oPI5Qp8LnC/pPKAG+BtyR+wnShqZHaXXAXt6WzkilgHLAJqamnoNfTOrLEN9dtihQ4e44YYburs7PvKRj7Bs2TIAmpubWbhwIRMnTqSlpYXGxkZmzJjBaaedxty5c9+1ncsvv5yuri6mT5/e635uv/12Fi9ezLe//W1mz57d3b506VJaWlqoqqpi+vTpLFy4kOrqajo6OjjrrLOAXLfQgw8+SFVVVYl+Cv1TRP4ZK2ke8C8R8XlJPwMejYiHJP0I2BIRPzjW+k1NTTHYG1z4tEWz4dPR0cG0adOGu4yCXX/99TQ2NrJ4cfmew9Hbz1pSW0Q09bduISdH3gLcJGkXuT71+wvYlplZSZ155pls2bKFK664YrhLKZkBXVgUEU8CT2bTLwKzil+SmVnxHekbT5nHcjEzS4QD3cwsEQ50M7NEONDNzBJRMaMtmlkZaV1e3O019T+uyh//+EduvPFGNm3aRHV1NfX19SxdupQ9e/awZMkSVq9eXdyaKpAD3czKXkRw4YUXsmjRIh566CEgd7Xovn37Ct724cOHex0GYCgUe3hdd7mYWdlraWlh1KhRfOUrX+lua2ho4FOf+hSQu5L04osv5mMf+xiXX345Ry6Y7Gs43DvuuIPm5mYWLFjAVVddxYoVK7jooos499xzmTp1KjfffHOvdZT78Lo+Qjezsrd161bOPPPMPl/fvHkz27ZtY9KkScydO5ff/e53fPKTnzzmNtva2njqqacYPXo0K1asoL29nc2bN1NdXc3pp5/ODTfcwCmnnJJXfUeG1507dy6HDh2ipqbmXcPrRgTnn38+GzZsYPLkyezYsYPly5fzgx8c8+L6AfMRuplVvFmzZlFXV8eIESNoaGhg9+7d/a5z/vnnM3r06O75c845hxNOOIGamhqmT5/OH/7wh7z3Xy7D6zrQzazszZgx45hXeh65kxHkhtk9fDg3sndfw+EC7xnitq9t9HSs4XXvu+8+/vznPzNnzhy2b9/ePbxue3s77e3t7Nq1q3sMmVINr+tAN7OyN3/+fN566y3uvffe7rZNmzZ137moL30NhztY5T68rvvQzWzg8jjNsJgk8dhjj3HjjTdy5513UlNT033a4quvvtrnen0NhztY5T687oCGzy2Uh881q0ypDJ9bCYZr+FwzMysjDnQzs0Q40M0sL0PZPft+VejPOJ+bRNdIekbSc5K2SfpG1r5C0kuS2rNHQ0GVmFnZqqmp4cCBAw71EooIDhw4QE1NzaC3kc9ZLm8B8yPikKRRwFOSjtxK+39FxCOD3ruZVYS6ujo6Ozvp6uoa7lKSVlNTQ11d3aDX7zfQI/cr+VA2Oyp7+Ne02fvIqFGjmDJlynCXYf3Iqw9dUpWkdmA/sDYiNmYvfUvSFkl3Sao+xibMzKzE8gr0iHg7IhqAOmCWpL8FbgM+BnwCOAm4pbd1JTVLapXU6j/XzMxKZ0BnuUTEQeBJ4NyI2Bs5bwHLgVl9rLMsIpoioqm2trbggs3MrHf5nOVSK+nEbHo08Glgu6SJWZuAC4CtpSzUzMyOLZ+zXCYCKyVVkfsF8HBErJb0G0m1gIB24CvH2oiZmZVWPme5bAEae2mfX5KKzMxsUHylqJlZIhzoZmaJcKCbmSXCgW5mlggHuplZIhzoZmaJcKCbmSXCgW5mlggHuplZIhzoZmaJcKCbmSXCgW5mlggHuplZIhzoZmaJcKCbmSXCgW5mlggHuplZIvK5p2iNpGckPSdpm6RvZO1TJG2UtFPSTyUdV/pyzcysL/kcob8FzI+IvwMagHMlzQG+A9wVEVOB/wIWl65MMzPrT7+BHjmHstlR2SOA+cAjWftK4IKSVGhmZnnJqw9dUpWkdmA/sBZ4ATgYEYezRTqBk/tYt1lSq6TWrq6uYtRsZma9yCvQI+LtiGgA6oBZwLTeFutj3WUR0RQRTbW1tYOv1MzMjmlAZ7lExEHgSWAOcKKkkdlLdcCe4pZmZmYDkc9ZLrWSTsymRwOfBjqAFuDibLFFwC9KVaSZmfVvZP+LMBFYKamK3C+AhyNitaTngYckfRPYDNxfwjrNzKwf/QZ6RGwBGntpf5Fcf3pFW7Xx5ZJs97LZk0uyXTOzvvhKUTOzRDjQzcwS4UA3M0uEA93MLBEOdDOzRDjQzcwS4UA3M0uEA93MLBEOdDOzRDjQzcwS4UA3M0uEA93MLBEOdDOzRDjQzcwS4UA3M0uEA93MLBH53ILuFEktkjokbZP09az9DkmvSmrPHueVvlwzM+tLPregOwz8c0Q8K2ks0CZpbfbaXRGxpHTlmZlZvvK5Bd1eYG82/YakDuDkUhdmZmYDM6A+dEn15O4vujFrul7SFkkPSBpX5NrMzGwA8g50SccDjwI3RsTrwA+BDwMN5I7g/6OP9ZoltUpq7erqKkLJZmbWm7wCXdIocmH+44j4OUBE7IuItyPiHeBeYFZv60bEsohoioim2traYtVtZmZHyecsFwH3Ax0R8b0e7RN7LHYhsLX45ZmZWb7yOctlLnAl8HtJ7VnbvwKXSmoAAtgNXFuSCs3MLC/5nOXyFKBeXnqi+OWYmdlg+UpRM7NEONDNzBLhQDczS4QD3cwsEQ50M7NE5HPaopWJVRtfLsl2L5s9uSTbNbOh5SN0M7NEONDNzBLhQDczS4QD3cwsEQ50M7NEONDNzBLhQDczS4QD3cwsEQ50M7NEONDNzBKRzy3oTpHUIqlD0jZJX8/aT5K0VtLO7Hlc6cs1M7O+5HOEfhj454iYBswB/knSdOBWYF1ETAXWZfNmZjZM+g30iNgbEc9m028AHcDJwBeBldliK4ELSlWkmZn1b0B96JLqgUZgI/ChiNgLudAHPljs4szMLH95B7qk44FHgRsj4vUBrNcsqVVSa1dX12BqNDOzPOQV6JJGkQvzH0fEz7PmfZImZq9PBPb3tm5ELIuIpohoqq2tLUbNZmbWi3zOchFwP9AREd/r8dLjwKJsehHwi+KXZ2Zm+crnjkVzgSuB30tqz9r+FbgTeFjSYuBl4JLSlGhmZvnoN9Aj4ilAfbx8TnHLMTOzwfKVomZmiXCgm5klwoFuZpYIB7qZWSIc6GZmiXCgm5klwoFuZpYIB7qZWSIc6GZmiXCgm5klwoFuZpYIB7qZWSIc6GZmiXCgm5klwoFuZpaIfG5wUTY+/PLPSr6PFyYX5z4dqza+XJTtmJnlK59b0D0gab+krT3a7pD0qqT27HFeacs0M7P+5NPlsgI4t5f2uyKiIXs8UdyyzMxsoPoN9IjYAPxpCGoxM7MCFPKl6PWStmRdMuOKVpGZmQ3KYAP9h8CHgQZgL/AffS0oqVlSq6TWrq6uQe7OzMz6M6hAj4h9EfF2RLwD3AvMOsayyyKiKSKaamtrB1unmZn1Y1CBLmlij9kLga19LWtmZkOj3/PQJf0EmAdMkNQJ3A7Mk9QABLAbuLaENZqZWR76DfSIuLSX5vtLUIuZmRXAl/6bmSXCgW5mlggHuplZIhzoZmaJcKCbmSXCgW5mlggHuplZIhzoZmaJcKCbmSXCgW5mlggHuplZIhzoZmaJcKCbmSXCgW5mlggHuplZIvodD93St2rjyyXZ7mWzJ5dku2bWOx+hm5klot9Al/SApP2StvZoO0nSWkk7s+dxpS3TzMz6k88R+grg3KPabgXWRcRUYF02b2Zmw6jfQI+IDcCfjmr+IrAym14JXFDkuszMbIAG24f+oYjYC5A9f7CvBSU1S2qV1NrV1TXI3ZmZWX9K/qVoRCyLiKaIaKqtrS317szM3rcGG+j7JE0EyJ73F68kMzMbjMEG+uPAomx6EfCL4pRjZmaDlc9piz8BngZOl9QpaTFwJ/AZSTuBz2TzZmY2jPq9UjQiLu3jpXOKXIuZmRXAV4qamSXCgW5mlggHuplZIhzoZmaJcKCbmSXCgW5mlggHuplZIhzoZmaJcKCbmSXC9xQ9yodf/lnJ9/HC5EtKvg8ze//xEbqZWSIc6GZmiXCgm5klwoFuZpYIB7qZWSIc6GZmiSjotEVJu4E3gLeBwxHRVIyizMxs4IpxHvrZEfFaEbZjZmYFcJeLmVkiCg30ANZIapPUXIyCzMxscArtcpkbEXskfRBYK2l7RGzouUAW9M0AkydPLnB3ZmbWl4KO0CNiT/a8H3gMmNXLMssioikimmprawvZnZmZHcOgA13SGEljj0wDC4CtxSrMzMwGppAulw8Bj0k6sp1VEfF/i1KVmZkN2KADPSJeBP6uiLWYmVkBPB66lcyqjS+XZLuXzfaX62a98XnoZmaJcKCbmSXCgW5mlggHuplZIvyl6DAYihtRDwXf7NqsvPgI3cwsEQ50M7NEONDNzBLhQDczS4S/FLVBG4ovd3v74rVUV6BWklJdLVuKn62v7B06PkI3M0uEA93MLBEOdDOzRDjQzcwS4S9FzSqQvxgunUoe9rmgI3RJ50raIWmXpFuLVZSZmQ1cIfcUrQK+DywEpgOXSpperMLMzGxgCjlCnwXsiogXI+IvwEPAF4tTlpmZDVQhgX4y8EqP+c6szczMhkEhX4qql7Z4z0JSM9CczR6StGOQ+5sAvDbIdYdTpdYNZVH7vwxmpTKoe1CSrPvyISxkgIb0513gz+HUfBYqJNA7gVN6zNcBe45eKCKWAcsK2A8AklojoqnQ7Qy1Sq0bKrd21z20XHf5KKTLZRMwVdIUSccBXwYeL05ZZmY2UIM+Qo+Iw5KuB34FVAEPRMS2olVmZmYDUtCFRRHxBPBEkWrpT8HdNsOkUuuGyq3ddQ8t110mFPGe7zHNzKwCeSwXM7NEVESgl9sQA5IekLRf0tYebSdJWitpZ/Y8LmuXpLuz2rdImtljnUXZ8jslLRqCuk+R1CKpQ9I2SV+vhNol1Uh6RtJzWd3fyNqnSNqY1fDT7Mt5JFVn87uy1+t7bOu2rH2HpM+Wsu4e+6yStFnS6kqpW9JuSb+X1C6pNWsr689Jj32eKOkRSduzz/pZlVJ7wSKirB/kvnB9ATgNOA54Dpg+zDX9PTAT2Nqj7X8Dt2bTtwLfyabPA35J7rz9OcDGrP0k4MXseVw2Pa7EdU8EZmbTY4H/JDdsQ1nXnu3/+Gx6FLAxq+dh4MtZ+4+Ar2bT1wE/yqa/DPw0m56efX6qgSnZ56pqCD4vNwGrgNXZfNnXDewGJhzVVtafkx51rgT+MZs+DjixUmov+L0PdwF5/OOcBfyqx/xtwG1lUFc97w70HcDEbHoisCObvge49OjlgEuBe3q0v2u5IXoPvwA+U0m1Ax8AngVmk7soZOTRnxNyZ16dlU2PzJbT0Z+dnsuVsN46YB0wH1id1VEJde/mvYFe9p8T4G+Al8i+H6yk2ovxqIQul0oZYuBDEbEXIHv+YNbeV/3D+r6yP+cbyR3tln3tWbdFO7AfWEvuKPVgRBzupYbu+rLX/xsYPxx1A0uBm4F3svnxVEbdAayR1Kbc1d5QAZ8Tcn/JdwHLs26u+ySNqZDaC1YJgZ7XEANlrK/6h+19SToeeBS4MSJeP9aivbQNS+0R8XZENJA74p0FTDtGDWVRt6TPA/sjoq1n8zFqKIu6M3MjYia50VT/SdLfH2PZcqp7JLnu0B9GRCPw/8h1sfSlnGovWCUEel5DDJSBfZImAmTP+7P2vuoflvclaRS5MP9xRPw8a66I2gEi4iDwJLn+zhMlHbmWomcN3fVlr58A/Imhr3sucL6k3eRGI51P7oi93OsmIvZkz/uBx8j9Eq2Ez0kn0BkRG7P5R8gFfCXUXrBKCPRKGWLgceDIN+GLyPVPH2m/Kvs2fQ7w39mffL8CFkgal33jviBrKxlJAu4HOiLie5VSu6RaSSdm06OBTwMdQAtwcR91H3k/FwO/iVxH6OPAl7OzSaYAU4FnSlV3RNwWEXURUU/uc/ubiLi83OuWNEbS2CPT5P59t1LmnxOAiPgj8Iqk07Omc4DnK6H2ohjuTvw8v+g4j9wZGS8A/1YG9fwE2Av8ldxv8sXk+jrXATuz55OyZUXuRiAvAL8Hmnps5x+AXdnjmiGo+5Pk/mzcArRnj/PKvXbg48DmrO6twL9n7aeRC7ZdwM+A6qy9Jpvflb1+Wo9t/Vv2fnYAC4fwMzOP/3+WS1nXndX3XPbYduT/XLl/TnrsswFozT4v/4fcWSoVUXuhD18pamaWiErocjEzszw40M3MEuFANzNLhAPdzCwRDnQzs0Q40M3MEuFANzNLhAPdzCwR/wPOVcPHJucgYAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f2e945efb38>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.distplot( [pt.PlayNumber for pt in df_ubc.select('PlayNumber').where(df_ubc.y == 1).collect()], label=\"Stay user\",kde = False)\n",
    "sns.distplot( [pt.PlayNumber for pt in df_ubc.select('PlayNumber').where(df_ubc.y == 0).collect()], label=\"Churn user\",kde = False)\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.3.6 PovEva"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x7f2e94487f60>"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD8CAYAAABn919SAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAE3dJREFUeJzt3X+MVGe9x/H3lwV3kfb2B6ABt3SpogKp7tK10KAJpVqLP6pt2vgDLW1IVq1WG72p7b3JrSZqakQlTeqNqC0ktdZqNTWkRhBXqqah7JYViwsXUGy3VNiiWElsFfvcP+bshra7zOyP2d15eL+Sycx55sw532cYPnv2mXOejZQSkqTaN2m8C5AkjQ4DXZIyYaBLUiYMdEnKhIEuSZkw0CUpEwa6JGXCQJekTBjokpSJyWO5sxkzZqSmpqax3KUk1bzOzs6nU0ozy603poHe1NRER0fHWO5SkmpeRPypkvUccpGkTBjokpQJA12SMjGmY+iSatO//vUvenp6ePbZZ8e7lKw1NDTQ2NjIlClThvV6A11SWT09PZx++uk0NTUREeNdTpZSShw5coSenh7mzp07rG045CKprGeffZbp06cb5lUUEUyfPn1EvwUZ6JIqYphX30jfYwNdkjLhGLqkIbtn2+Ojur0PLp5Tdp0vfvGL3HPPPdTV1TFp0iS++c1vsnjxYtauXUtbWxsvf/nLR7WmWlQzgT7aH6A+lXyQJI2vhx9+mI0bN/Loo49SX1/P008/zT//+U8A1q5dy4c+9KEJGegpJVJKTJo0NoMhDrlImvCeeuopZsyYQX19PQAzZsxg9uzZ3H777Rw8eJCLL76Yiy++GICPfexjtLa2snDhQm699VYAtmzZwhVXXNG/vc2bN3PllVe+ZD9NTU08/fTTAHR0dLBs2TIAtm7dSnNzM83NzbS0tPD3v/8dgK985Su86U1v4g1veEP/vg4cOMD8+fO5/vrrWbRoEU888UR13pQBGOiSJrxLL72UJ554gte+9rVcf/31bN26FYBPfvKTzJ49m/b2dtrb24HS0ExHRwc7d+5k69at7Ny5k+XLl9Pd3U1vby8Ad911F9ddd13F+1+zZg133HEHXV1d/OpXv2Lq1Kls2rSJvXv38sgjj9DV1UVnZycPPfQQAHv27OGaa65hx44dnHvuuaP8bgzOQJc04Z122ml0dnaybt06Zs6cyfve9z7Wr18/4Lr33XcfixYtoqWlhV27dvH73/+eiODDH/4wd999N0ePHuXhhx9mxYoVFe9/6dKlfPrTn+b222/n6NGjTJ48mU2bNrFp0yZaWlpYtGgRu3fvZu/evQCce+65LFmyZDS6PiQ1M4Yu6dRWV1fHsmXLWLZsGeeffz4bNmzg2muvfcE6f/zjH1mzZg3bt2/nrLPO4tprr+0/r/u6667j3e9+Nw0NDVx99dVMnvzS+Js8eTLPP/88wAvOB7/55pt55zvfyYMPPsiSJUv4+c9/TkqJW265hY985CMv2MaBAweYNm3aKPe+Mh6hS5rw9uzZ03/0C9DV1dU/lHH66af3j2k/88wzTJs2jTPOOINDhw7x05/+tP81s2fPZvbs2XzhC194yQ+CPk1NTXR2dgJw//3397fv37+f888/n89+9rO0traye/du3v72t3PnnXdy7NgxAJ588kkOHz48qv0eKo/QJQ3ZWJ8dduzYMW644Yb+4Y7XvOY1rFu3DoC2tjZWrFjBrFmzaG9vp6WlhYULF3LeeeexdOnSF2xn5cqV9Pb2smDBggH3c+utt7J69Wq+9KUvsXjx4v72tWvX0t7eTl1dHQsWLGDFihXU19fT3d3NRRddBJSGhe6++27q6uqq9C6UFymlMdtZa2trGu4fuPC0RWn8dHd3M3/+/PEuY8Q+8YlP0NLSwurVq8e7lEEN9F5HRGdKqbXcaz1Cl3RKuOCCC5g2bRpf/epXx7uUqjHQJZ0S+sbGc+aXopKUCQNdkjJhoEtSJgx0ScqEX4pKGrqOu0Z3e63l51X585//zI033sj27dupr6+nqamJtWvXcvDgQdasWcPGjRtHt6YaZKBLmvBSSlxxxRWsWrWKe++9FyhdLXro0KERb/v48eMDTgMwFkZ7el2HXCRNeO3t7UyZMoWPfvSj/W3Nzc285S1vAUpXkl511VW8/vWvZ+XKlfRdMDnYdLif+9znaGtr49JLL+Waa65h/fr1XHnllVx22WXMmzePm266acA6Jvr0uh6hS5rwHnvsMS644IJBn9+xYwe7du1i9uzZLF26lN/85je8+c1vPuk2Ozs7+fWvf83UqVNZv349XV1d7Nixg/r6el73utdxww03cM4551RUX9/0ukuXLuXYsWM0NDS8YHrdlBKXX345Dz30EHPmzGHPnj3cddddfOMb3xjS+1COR+iSat6FF15IY2MjkyZNorm5mQMHDpR9zeWXX87UqVP7ly+55BLOOOMMGhoaWLBgAX/6058q3v9EmV7XQJc04S1cuPCkV3r2/SUjKE2ze/z4cWDw6XCBl0xxO9g2TnSy6XW//e1v849//IMlS5awe/fu/ul1u7q66OrqYt++ff1zyFRrel0DXdKEt3z5cp577jm+9a1v9bdt3769/y8XDWaw6XCHa6JPr+sYuqShq+A0w9EUEfz4xz/mxhtv5LbbbqOhoaH/tMUnn3xy0NcNNh3ucE306XWdPtfpc6Wycpk+txaMZPrciodcIqIuInZExMZieW5EbIuIvRHx/Yh42ZArlySNmqGMoX8K6D5h+cvA11NK84C/AhN3xnhJOgVUFOgR0Qi8E/h2sRzAcuCHxSobgPdWo0BJE8NYDs+eqkb6Hld6hL4WuAl4vlieDhxNKfWd19MDvGpElUiasBoaGjhy5IihXkUpJY4cOUJDQ8Owt1H2LJeIeBdwOKXUGRHL+poHqmeQ17cBbQBz5vgFpFSLGhsb6enpobe3d7xLyVpDQwONjY3Dfn0lpy0uBS6PiHcADcB/UDpiPzMiJhdH6Y3AwYFenFJaB6yD0lkuw65U0riZMmUKc+fOHe8yVEbZIZeU0i0ppcaUUhPwfuAXKaWVQDtwVbHaKuCBqlUpSSprJFeKfhb4dETsozSm/p3RKUmSNBxDulI0pfRL4JfF4z8AF45+SZKk4XAuF0nKhIEuSZkw0CUpEwa6JGXCQJekTBjokpQJA12SMmGgS1ImDHRJyoSBLkmZMNAlKRMGuiRlwkCXpEwY6JKUCQNdkjJhoEtSJgx0ScqEgS5JmTDQJSkTBrokZcJAl6RMGOiSlAkDXZIyYaBLUiYMdEnKhIEuSZkw0CUpEwa6JGXCQJekTBjokpQJA12SMmGgS1ImDHRJyoSBLkmZMNAlKRMGuiRlomygR0RDRDwSEb+NiF0R8fmifW5EbIuIvRHx/Yh4WfXLlSQNppIj9OeA5SmlNwLNwGURsQT4MvD1lNI84K/A6uqVKUkqp2ygp5JjxeKU4paA5cAPi/YNwHurUqEkqSIVjaFHRF1EdAGHgc3AfuBoSul4sUoP8KrqlChJqkRFgZ5S+ndKqRloBC4E5g+02kCvjYi2iOiIiI7e3t7hVypJOqkhneWSUjoK/BJYApwZEZOLpxqBg4O8Zl1KqTWl1Dpz5syR1CpJOolKznKZGRFnFo+nAm8FuoF24KpitVXAA9UqUpJU3uTyqzAL2BARdZR+ANyXUtoYEb8H7o2ILwA7gO9UsU5JUhllAz2ltBNoGaD9D5TG0yVJE4BXikpSJgx0ScqEgS5JmTDQJSkTBrokZcJAl6RMGOiSlAkDXZIyYaBLUiYMdEnKhIEuSZkw0CUpEwa6JGXCQJekTBjokpQJA12SMmGgS1ImDHRJyoSBLkmZMNAlKRMGuiRlwkCXpEwY6JKUCQNdkjJhoEtSJgx0ScqEgS5JmTDQJSkTBrokZcJAl6RMGOiSlInJ413AeLtn2+NV2e4HF8+pynYlaTAeoUtSJgx0ScqEgS5JmTjlx9CrpRpj847LSzqZskfoEXFORLRHRHdE7IqITxXtZ0fE5ojYW9yfVf1yJUmDqWTI5TjwmZTSfGAJ8PGIWADcDGxJKc0DthTLkqRxUjbQU0pPpZQeLR7/HegGXgW8B9hQrLYBeG+1ipQklTekL0UjogloAbYBr0wpPQWl0AdeMdrFSZIqV3GgR8RpwP3AjSmlZ4bwuraI6IiIjt7e3uHUKEmqQEWBHhFTKIX5d1NKPyqaD0XErOL5WcDhgV6bUlqXUmpNKbXOnDlzNGqWJA2gkrNcAvgO0J1S+toJT/0EWFU8XgU8MPrlSZIqVcl56EuBDwO/i4iuou2/gNuA+yJiNfA4cHV1SpQkVaJsoKeUfg3EIE9fMrrl6GScSEzSyXjpvyRlwkCXpEwY6JKUCQNdkjJhoEtSJgx0ScqEgS5JmTDQJSkTBrokZcJAl6RMGOiSlAkDXZIyYaBLUiYqmT53wnj14z+o2rb3z3H2X0m1zSN0ScqEgS5JmTDQJSkTBrokZcJAl6RMGOiSlAkDXZIyYaBLUiZq6sIiVcc92x6vynY/uHhOVbYraWAeoUtSJgx0ScqEgS5JmTDQJSkTBrokZcJAl6RMGOiSlAkDXZIyYaBLUiYMdEnKhIEuSZkw0CUpEwa6JGWibKBHxJ0RcTgiHjuh7eyI2BwRe4v7s6pbpiSpnEqO0NcDl72o7WZgS0ppHrClWJYkjaOygZ5Segj4y4ua3wNsKB5vAN47ynVJkoZouGPor0wpPQVQ3L9i9EqSJA1H1b8UjYi2iOiIiI7e3t5q706STlnDDfRDETELoLg/PNiKKaV1KaXWlFLrzJkzh7k7SVI5ww30nwCrisergAdGpxxJ0nBVctri94CHgddFRE9ErAZuA94WEXuBtxXLkqRxNLncCimlDwzy1CWjXIskaQS8UlSSMmGgS1ImDHRJyoSBLkmZMNAlKRMGuiRlwkCXpEwY6JKUCQNdkjJhoEtSJgx0ScqEgS5JmTDQJSkTZWdbPFW8+vEfVG3b++dcXbVtS1Ifj9AlKRMGuiRlwkCXpEwY6JKUCQNdkjJhoEtSJgx0ScqEgS5JmfDCItWUe7Y9XpXtfnDxnKpsVxpLHqFLUiYMdEnKhIEuSZlwDH0MVHPirwmt7uyB21uvG9s6pFOER+iSlAkDXZIyYaBLUiYMdEnKhF+KSgK8aKtPNd6HsXoPPEKXpEwY6JKUCQNdkjLhGLqqZtsf/zJg+/5/V2esdiSqNX5cLbU0Lu3Y/NgZ0RF6RFwWEXsiYl9E3DxaRUmShm7YgR4RdcAdwApgAfCBiFgwWoVJkoZmJEfoFwL7Ukp/SCn9E7gXeM/olCVJGqqRBPqrgCdOWO4p2iRJ42AkX4rGAG3pJStFtAFtxeKxiNgzzP3NAJ4e5msnslz7BYP27T/HvJBRNu7/Ziurt+lx71ulhvgejGu/RuHf69xKVhpJoPcA55yw3AgcfPFKKaV1wLoR7AeAiOhIKbWOdDsTTa79gnz7lmu/IN++5dqvFxvJkMt2YF5EzI2IlwHvB34yOmVJkoZq2EfoKaXjEfEJ4GdAHXBnSmnXqFUmSRqSEV1YlFJ6EHhwlGopZ8TDNhNUrv2CfPuWa78g377l2q8XiJRe8j2mJKkGOZeLJGWiJgK9lqcYiIg7I+JwRDx2QtvZEbE5IvYW92cV7RERtxf93BkRi8av8pOLiHMioj0iuiNiV0R8qmjPoW8NEfFIRPy26Nvni/a5EbGt6Nv3i5MBiIj6Ynlf8XzTeNZfTkTURcSOiNhYLOfSrwMR8buI6IqIjqKt5j+PQzHhAz2DKQbWA5e9qO1mYEtKaR6wpViGUh/nFbc24H/HqMbhOA58JqU0H1gCfLz4d8mhb88By1NKbwSagcsiYgnwZeDrRd/+Cqwu1l8N/DWl9Brg68V6E9mngO4TlnPpF8DFKaXmE05RzOHzWLmU0oS+ARcBPzth+RbglvGua4h9aAIeO2F5DzCreDwL2FM8/ibwgYHWm+g34AHgbbn1DXg58CiwmNKFKZOL9v7PJaUzvS4qHk8u1ovxrn2Q/jRSCrblwEZKFwjWfL+KGg8AM17UltXnsdxtwh+hk+cUA69MKT0FUNy/omivyb4Wv4q3ANvIpG/FsEQXcBjYDOwHjqaUjhernFh/f9+K5/8GTB/biiu2FrgJeL5Ynk4e/YLSleqbIqKzuEIdMvk8VqoW5kOvaIqBTNRcXyPiNOB+4MaU0jMRA3WhtOoAbRO2bymlfwPNEXEm8GNg/kCrFfc10beIeBdwOKXUGRHL+poHWLWm+nWCpSmlgxHxCmBzROw+ybq11reK1MIRekVTDNSYQxExC6C4P1y011RfI2IKpTD/bkrpR0VzFn3rk1I6CvyS0vcEZ0ZE30HQifX39614/gxg4L/uMb6WApdHxAFKs6Mup3TEXuv9AiCldLC4P0zph/CFZPZ5LKcWAj3HKQZ+AqwqHq+iNP7c135N8Q38EuBvfb8uTjRROhT/DtCdUvraCU/l0LeZxZE5ETEVeCulLxHbgauK1V7ct74+XwX8IhUDsxNJSumWlFJjSqmJ0v+jX6SUVlLj/QKIiGkRcXrfY+BS4DEy+DwOyXgP4lf4Zcc7gP+jNI753+NdzxBr/x7wFPAvSkcFqymNQ24B9hb3ZxfrBqUzevYDvwNax7v+k/TrzZR+Rd0JdBW3d2TStzcAO4q+PQb8T9F+HvAIsA/4AVBftDcUy/uK588b7z5U0MdlwMZc+lX04bfFbVdfTuTweRzKzStFJSkTtTDkIkmqgIEuSZkw0CUpEwa6JGXCQJekTBjokpQJA12SMmGgS1Im/h9ymPXQOBEYBgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f2e9451ec88>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.distplot( [pt.PosEva for pt in df_ubc.select('PosEva').where(df_ubc.y == 1).collect()], label=\"Stay user\",kde = False)\n",
    "sns.distplot( [pt.PosEva for pt in df_ubc.select('PosEva').where(df_ubc.y == 0).collect()], label=\"Churn user\",kde = False)\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.3.7 NegEva"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x7f2e94415e80>"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD8CAYAAABn919SAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAFn1JREFUeJzt3X+Q3HWd5/HnmyRmYuD4ldFLCGEiohLEnQkjhItehaBIdEWhpBRQApeqURGU0z2BtWrBK7XcMmqOKnSNCqEOkEWRk0rBLpibBbGokAkZQ2KSC0iEQEgG9ljInSCR9/3R3+SGMEP3zPRM93zzfFR1Tfenv/39vueT6dd88+nP9zORmUiSxr+DGl2AJKk+DHRJKgkDXZJKwkCXpJIw0CWpJAx0SSoJA12SSsJAl6SSMNAlqSQmjuXBpk2blm1tbWN5SEka99auXftsZrZW225MA72trY2enp6xPKQkjXsR8cdatnPIRZJKwkCXpJIw0CWpJMZ0DF3S+PTKK6+wfft2XnrppUaXUmotLS3MnDmTSZMmDev1BrqkqrZv384hhxxCW1sbEdHockopM3nuuefYvn07s2fPHtY+qg65RERLRDwUEb+LiI0R8fWifUVEPB4RvcWtfVgVSGp6L730EkceeaRhPooigiOPPHJE/wuq5Qz9ZWBhZu6OiEnAAxFxd/Hcf8nMXwz76JLGDcN89I20j6sGelb+Rt3u4uGk4ubfrZOkJlPTGHpETADWAm8HrsvM1RHxeeCbEfF3wCrgysx8efRKldQsbln9RF33d/4ps6pu881vfpNbbrmFCRMmcNBBB/GjH/2IU045hWXLltHV1cWb3/zmutY0HtUU6Jn5F6A9Ig4D7oiIdwNXAc8AbwKWA1cA/3X/10ZEF9AFMGtW9X+0wTTiB0hSc3jwwQdZuXIlDz/8MJMnT+bZZ5/lz3/+MwDLli3j05/+dFMGemaSmRx00NjMEB/SUTLzeeBfgDMzc0dWvAzcAJw8yGuWZ2ZnZna2tlZdikCSXmfHjh1MmzaNyZMnAzBt2jRmzJjBtddey9NPP81pp53GaaedBsDnP/95Ojs7OeGEE7j66qsBWLVqFWefffa+/d17772cc845rztOW1sbzz77LAA9PT0sWLAAgPvuu4/29nba29vp6OjgxRdfBOA73/kO733ve3nPe96z71jbtm3j+OOP55JLLmHu3Lk8+eSTo9MpA6hllktrcWZOREwBPgBsjojpRVsAHwc2jGahkg5cZ5xxBk8++STveMc7uOSSS7jvvvsA+OIXv8iMGTPo7u6mu7sbqAzN9PT0sH79eu677z7Wr1/PwoUL2bRpE319fQDccMMNXHzxxTUff+nSpVx33XX09vbym9/8hilTpnDPPfewdetWHnroIXp7e1m7di33338/AFu2bOHCCy9k3bp1HHPMMXXujcHVcoY+HeiOiPXAGuDezFwJ3BwRjwCPANOAb4xemZIOZAcffDBr165l+fLltLa28slPfpIVK1YMuO1tt93G3Llz6ejoYOPGjfz+978nIvjMZz7DTTfdxPPPP8+DDz7IokWLaj7+/Pnz+fKXv8y1117L888/z8SJE7nnnnu455576OjoYO7cuWzevJmtW7cCcMwxxzBv3rx6fOtDUsssl/VAxwDtC0elIkkawIQJE1iwYAELFizgxBNP5MYbb+Siiy56zTaPP/44S5cuZc2aNRx++OFcdNFF++Z1X3zxxXz0ox+lpaWFc889l4kTXx9/EydO5NVXXwV4zXzwK6+8ko985CPcddddzJs3j1//+tdkJldddRWf/exnX7OPbdu2MXXq1Dp/97VxLRdJTW/Lli37zn4Bent79w1lHHLIIfvGtF944QWmTp3KoYceys6dO7n77rv3vWbGjBnMmDGDb3zjG6/7RbBXW1sba9euBeD222/f1/7YY49x4okncsUVV9DZ2cnmzZv50Ic+xPXXX8/u3ZVZ3U899RS7du2q6/c9VF76L2nIxnqW2O7du7nsssv2DXe8/e1vZ/ny5QB0dXWxaNEipk+fTnd3Nx0dHZxwwgm87W1vY/78+a/ZzwUXXEBfXx9z5swZ8DhXX301S5Ys4Vvf+hannHLKvvZly5bR3d3NhAkTmDNnDosWLWLy5Mls2rSJU089FagMC910001MmDBhlHqhuqhcNzQ2Ojs7c7h/4MJpi1LjbNq0ieOPP77RZYzYpZdeSkdHB0uWLGl0KYMaqK8jYm1mdlZ7rWfokg4IJ510ElOnTuW73/1uo0sZNQa6pAPC3rHxMvNDUUkqCQNdkkrigB1yqfeHrOAHrZIayzN0SSqJA/YMXdII9NxQ3/11Vl9X5ZlnnuHyyy9nzZo1TJ48mba2NpYtW8bTTz/N0qVLWblyZX1rGocMdElNLzM5++yzWbx4MbfeeitQuVp0586dI973nj17BlwGYCzUe3ldh1wkNb3u7m4mTZrE5z73uX1t7e3tvP/97wcqV5J+4hOf4F3vehcXXHABey+YHGw53GuuuYauri7OOOMMLrzwQlasWME555zDmWeeyXHHHcdXv/rVAeto9uV1PUOX1PQ2bNjASSedNOjz69atY+PGjcyYMYP58+fz29/+lve9731vuM+1a9fywAMPMGXKFFasWEFvby/r1q1j8uTJvPOd7+Syyy7j6KOPrqm+vcvrzp8/n927d9PS0vKa5XUzk7POOov777+fWbNmsWXLFm644QZ+8IMfDKkfqvEMXdK4d/LJJzNz5kwOOugg2tvb2bZtW9XXnHXWWUyZMmXf49NPP51DDz2UlpYW5syZwx//+Meaj98sy+sa6JKa3gknnPCGV3ru/UtGUFlmd8+ePcDgy+ECr1vidrB99PdGy+v+5Cc/4U9/+hPz5s1j8+bN+5bX7e3tpbe3l0cffXTfGjKjtbyugS6p6S1cuJCXX36ZH//4x/va1qxZs+8vFw1msOVwh6vZl9d1DF3S0NUwzbCeIoI77riDyy+/nG9/+9u0tLTsm7b41FNPDfq6wZbDHa5mX173gF0+dzR4pajKqizL544HI1k+1yEXSSoJA12SSsJAl1STsRyePVCNtI+rBnpEtETEQxHxu4jYGBFfL9pnR8TqiNgaEf8YEW8aUSWSmlZLSwvPPfecoT6KMpPnnnuOlpaWYe+jllkuLwMLM3N3REwCHoiIu4EvA9/PzFsj4h+AJcAPh12JpKY1c+ZMtm/fTl9fX6NLKbWWlhZmzpw57NdXDfSs/EreXTycVNwSWAicX7TfCFyDgS6V0qRJk5g9e3ajy1AVNY2hR8SEiOgFdgH3Ao8Bz2fm3kuptgNHDfLarojoiYgef7tL0uipKdAz8y+Z2Q7MBE4GBpqQOuDgWmYuz8zOzOxsbW0dfqWSpDc0pFkumfk88C/APOCwiNg7ZDMTeLq+pUmShqKWWS6tEXFYcX8K8AFgE9ANfKLYbDHwq9EqUpJUXS2zXKYDN0bEBCq/AG7LzJUR8Xvg1oj4BrAO+Oko1ilJqqKWWS7rgY4B2v9AZTxdktQEvFJUkkrCQJekkjDQJakkDHRJKgkDXZJKwkCXpJIw0CWpJAx0SSoJA12SSsJAl6SSMNAlqSQMdEkqCQNdkkrCQJekkjDQJakkDHRJKgkDXZJKwkCXpJIw0CWpJAx0SSqJqoEeEUdHRHdEbIqIjRHxpaL9moh4KiJ6i9uHR79cSdJgJtawzR7gK5n5cEQcAqyNiHuL576fmUtHrzxJUq2qBnpm7gB2FPdfjIhNwFGjXZgkaWiGNIYeEW1AB7C6aLo0ItZHxPURcXida5MkDUHNgR4RBwO3A5dn5gvAD4FjgXYqZ/DfHeR1XRHRExE9fX19dShZkjSQmgI9IiZRCfObM/OXAJm5MzP/kpmvAj8GTh7otZm5PDM7M7OztbW1XnVLkvZTyyyXAH4KbMrM7/Vrn95vs7OBDfUvT5JUq1pmucwHPgM8EhG9RdvfAudFRDuQwDbgs6NSoSSpJrXMcnkAiAGeuqv+5UiShssrRSWpJAx0SSoJA12SSsJAl6SSMNAlqSQMdEkqCQNdkkrCQJekkjDQJakkDHRJKgkDXZJKwkCXpJIw0CWpJAx0SSoJA12SSsJAl6SSMNAlqSQMdEkqCQNdkkrCQJekkjDQJakkqgZ6RBwdEd0RsSkiNkbEl4r2IyLi3ojYWnw9fPTLlSQNppYz9D3AVzLzeGAe8IWImANcCazKzOOAVcVjSVKDVA30zNyRmQ8X918ENgFHAR8Dbiw2uxH4+GgVKUmqbkhj6BHRBnQAq4G3ZuYOqIQ+8JZBXtMVET0R0dPX1zeyaiVJg6o50CPiYOB24PLMfKHW12Xm8szszMzO1tbW4dQoSapBTYEeEZOohPnNmfnLonlnREwvnp8O7BqdEiVJtahllksAPwU2Zeb3+j11J7C4uL8Y+FX9y5Mk1WpiDdvMBz4DPBIRvUXb3wLfBm6LiCXAE8C5o1OiJKkWVQM9Mx8AYpCnT69vOZKk4arlDF01umX1E3Xd3/mnzKrr/iSVm5f+S1JJGOiSVBIGuiSVhIEuSSVhoEtSSRjoklQSBroklYSBLkklYaBLUkkY6JJUEga6JJWEa7k0MdeGkTQUnqFLUkkY6JJUEga6JJWEgS5JJWGgS1JJGOiSVBIGuiSVRNVAj4jrI2JXRGzo13ZNRDwVEb3F7cOjW6YkqZpaztBXAGcO0P79zGwvbnfVtyxJ0lBVDfTMvB/41zGoRZI0AiMZQ780ItYXQzKH160iSdKwDDfQfwgcC7QDO4DvDrZhRHRFRE9E9PT19Q3zcJKkaoYV6Jm5MzP/kpmvAj8GTn6DbZdnZmdmdra2tg63TklSFcMK9IiY3u/h2cCGwbaVJI2NqsvnRsTPgAXAtIjYDlwNLIiIdiCBbcBnR7FGSVINqgZ6Zp43QPNPR6EWSdIIeKWoJJWEgS5JJWGgS1JJGOiSVBIGuiSVhIEuSSVhoEtSSVSdh96sjn3i52N2rMdmnTtmx5Kk4fIMXZJKwkCXpJIw0CWpJAx0SSoJA12SSmLcznLR0N2y+om67/P8U2bVfZ+ShsczdEkqCQNdkkrCQJekkjDQJakkDHRJKgkDXZJKwkCXpJKoGugRcX1E7IqIDf3ajoiIeyNia/H18NEtU5JUTS1n6CuAM/druxJYlZnHAauKx5KkBqoa6Jl5P/Cv+zV/DLixuH8j8PE61yVJGqLhjqG/NTN3ABRf31K/kiRJwzHqH4pGRFdE9ERET19f32gfTpIOWMMN9J0RMR2g+LprsA0zc3lmdmZmZ2tr6zAPJ0mqZriBfiewuLi/GPhVfcqRJA1XLdMWfwY8CLwzIrZHxBLg28AHI2Ir8MHisSSpgaquh56Z5w3y1Ol1rkWSNAJeKSpJJWGgS1JJGOiSVBIGuiSVhIEuSSVhoEtSSVSdtig49omfj8lxHpt17pgcR1I5eYYuSSVhoEtSSRjoklQSBroklYSBLkklYaBLUkkY6JJUEga6JJWEgS5JJWGgS1JJGOiSVBKu5dJExmrNGHDdGKmMPEOXpJIw0CWpJEY05BIR24AXgb8AezKzsx5FSZKGrh5j6Kdl5rN12I8kaQQccpGkkhjpGXoC90REAj/KzOX7bxARXUAXwKxZs0Z4OJXdLaufqOv+zj/FnzkdOEZ6hj4/M+cCi4AvRMR/3H+DzFyemZ2Z2dna2jrCw0mSBjOiQM/Mp4uvu4A7gJPrUZQkaeiGHegRMTUiDtl7HzgD2FCvwiRJQzOSMfS3AndExN793JKZ/1SXqiRJQzbsQM/MPwB/VcdaJEkj4LRFSSoJA12SSsJAl6SSMNAlqSQMdEkqCQNdkkrCv1ikEan32iv11uz1gevNqH48Q5ekkjDQJakkDHRJKgkDXZJKwkCXpJJwlssB6tgnft7oEurqsVnnNroElchozI4ai9lMnqFLUkkY6JJUEga6JJWEgS5JJWGgS1JJOMtFarB6z6gYD2vDHIjf81jwDF2SSmJEgR4RZ0bEloh4NCKurFdRkqShG3agR8QE4DpgETAHOC8i5tSrMEnS0IzkDP1k4NHM/ENm/hm4FfhYfcqSJA3VSAL9KODJfo+3F22SpAYYySyXGKAtX7dRRBfQVTzcHRFbhnm8acCzw3ztWBsvtY6XOqFqrX8zZoVU0fA+vaD2TRte6xC8Ya1D+J5H26B1jrDGY2rZaCSBvh04ut/jmcDT+2+UmcuB5SM4DgAR0ZOZnSPdz1gYL7WOlzph/NQ6XuoEax0Nja5zJEMua4DjImJ2RLwJ+BRwZ33KkiQN1bDP0DNzT0RcCvwzMAG4PjM31q0ySdKQjOhK0cy8C7irTrVUM+JhmzE0XmodL3XC+Kl1vNQJ1joaGlpnZL7uc0xJ0jjkpf+SVBLjItCbdYmBiDg6IrojYlNEbIyILxXtR0TEvRGxtfh6eKNrhcrVvRGxLiJWFo9nR8Tqos5/LD7cbriIOCwifhERm4u+PbWJ+/Q/F//2GyLiZxHR0iz9GhHXR8SuiNjQr23AfoyKa4v32PqImNvgOr9T/Puvj4g7IuKwfs9dVdS5JSI+NFZ1DlZrv+f+JiIyIqYVj8e8T5s+0Jt8iYE9wFcy83hgHvCForYrgVWZeRywqnjcDL4EbOr3+O+B7xd1/m9gSUOqer3/BvxTZr4L+CsqNTddn0bEUcAXgc7MfDeVyQGfonn6dQVw5n5tg/XjIuC44tYF/HCMaoSB67wXeHdmvgf4X8BVAMX761PACcVrflBkxFhZwetrJSKOBj4I9F9Gcuz7NDOb+gacCvxzv8dXAVc1uq5Bav1V8Y+6BZhetE0HtjRBbTOpvIEXAiupXBj2LDBxoH5uYJ3/Dnic4vOdfu3N2Kd7r5Y+gsoEg5XAh5qpX4E2YEO1fgR+BJw30HaNqHO/584Gbi7uv+b9T2WW3amN7NOi7RdUTj62AdMa1adNf4bOOFliICLagA5gNfDWzNwBUHx9S+Mq22cZ8FXg1eLxkcDzmbmneNws/fo2oA+4oRge+klETKUJ+zQznwKWUjkr2wH8G7CW5uzXvQbrx2Z+n/0n4O7iftPVGRFnAU9l5u/2e2rMax0PgV7TEgONFBEHA7cDl2fmC42uZ38R8dfArsxc2795gE2boV8nAnOBH2ZmB/B/aILhlYEU488fA2YDM4CpVP6bvb9m6NdqmvLnISK+RmVo8+a9TQNs1rA6I+LNwNeAvxvo6QHaRrXW8RDoNS0x0CgRMYlKmN+cmb8smndGxPTi+enArkbVV5gPnBUR26isirmQyhn7YRGx91qEZunX7cD2zFxdPP4FlYBvtj4F+ADweGb2ZeYrwC+B/0Bz9uteg/Vj073PImIx8NfABVmMWdB8dR5L5Rf674r310zg4Yj49zSg1vEQ6E27xEBEBPBTYFNmfq/fU3cCi4v7i6mMrTdMZl6VmTMzs41K//3PzLwA6AY+UWzW8DoBMvMZ4MmIeGfRdDrwe5qsTwtPAPMi4s3Fz8LeWpuuX/sZrB/vBC4sZmbMA/5t79BMI0TEmcAVwFmZ+X/7PXUn8KmImBwRs6l84PhQI2oEyMxHMvMtmdlWvL+2A3OLn+Ox79Ox/DBhBB9CfJjKJ92PAV9rdD396noflf9CrQd6i9uHqYxPrwK2Fl+PaHSt/WpeAKws7r+NypvhUeDnwORG11fU1Q70FP36P4DDm7VPga8Dm4ENwH8HJjdLvwI/ozK2/wqVoFkyWD9SGR64rniPPUJl5k4j63yUyvjz3vfVP/Tb/mtFnVuARY3u0/2e38b//1B0zPvUK0UlqSTGw5CLJKkGBroklYSBLkklYaBLUkkY6JJUEga6JJWEgS5JJWGgS1JJ/D/jT9ydt0JbhAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f2e94558470>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.distplot( [pt.NegEva for pt in df_ubc.select('NegEva').where(df_ubc.y == 1).collect()], label=\"Stay user\",kde = False)\n",
    "sns.distplot( [pt.NegEva for pt in df_ubc.select('NegEva').where(df_ubc.y == 0).collect()], label=\"Churn user\",kde = False)\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 4. Modeling"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### manuel one hot encoder"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {},
   "outputs": [],
   "source": [
    "# man column\n",
    "functionf = udf(lambda x: 1 if x == 'M' else 0)\n",
    "df_ubc = df_ubc.withColumn('Gender_m', functionf(F.col('Gender')))\n",
    "\n",
    "# woman column\n",
    "functionf = udf(lambda x: 1 if x == 'F' else 0)\n",
    "df_ubc = df_ubc.withColumn('Gender_f', functionf(F.col('Gender')))\n",
    "\n",
    "# remove column gender\n",
    "df_ubc = df_ubc.drop('Gender')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "metadata": {},
   "outputs": [],
   "source": [
    "# w column\n",
    "functionf = udf(lambda x: 1 if x == 'w' else 0)\n",
    "df_ubc = df_ubc.withColumn('Device_w', functionf(F.col('Device')))\n",
    "\n",
    "# a column\n",
    "functionf = udf(lambda x: 1 if x == 'm' else 0)\n",
    "df_ubc = df_ubc.withColumn('Device_m', functionf(F.col('Device')))\n",
    "\n",
    "# l column\n",
    "functionf = udf(lambda x: 1 if x == 'l' else 0)\n",
    "df_ubc = df_ubc.withColumn('Device_l', functionf(F.col('Device')))\n",
    "\n",
    "# remove column gender\n",
    "df_ubc = df_ubc.drop('Device')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 118,
   "metadata": {},
   "outputs": [],
   "source": [
    "# convert column type\n",
    "\n",
    "df_ubc = df_ubc.withColumn(\"userId\", df_ubc[\"userId\"].cast(IntegerType()))\n",
    "df_ubc = df_ubc.withColumn(\"Gender_m\", df_ubc[\"Gender_m\"].cast(IntegerType()))\n",
    "df_ubc = df_ubc.withColumn(\"Gender_f\", df_ubc[\"Gender_f\"].cast(IntegerType()))\n",
    "df_ubc = df_ubc.withColumn(\"Device_w\", df_ubc[\"Device_w\"].cast(IntegerType()))\n",
    "df_ubc = df_ubc.withColumn(\"Device_m\", df_ubc[\"Device_m\"].cast(IntegerType()))\n",
    "df_ubc = df_ubc.withColumn(\"Device_l\", df_ubc[\"Device_l\"].cast(IntegerType()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 119,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+------+----------+--------+----------+------+------+---+--------+--------+--------+--------+--------+\n",
      "|userId|PlayedTime|PaidTime|PlayNumber|PosEva|NegEva|  y|Gender_m|Gender_f|Device_w|Device_m|Device_l|\n",
      "+------+----------+--------+----------+------+------+---+--------+--------+--------+--------+--------+\n",
      "|200002|     72934|    1681|       300|    20|    10|  1|       1|       0|       0|       1|       0|\n",
      "|   124|   1012312|    3157|      4079|   289|    86|  1|       0|       1|       0|       1|       0|\n",
      "|    51|    523275|     466|      2111|   152|    45|  0|       1|       0|       1|       0|       0|\n",
      "|    15|    477307|    1356|      1914|   140|    43|  1|       1|       0|       1|       0|       0|\n",
      "|    54|    600432|    2658|      2402|   201|    64|  0|       0|       1|       1|       0|       0|\n",
      "|   155|    174342|     565|       719|    72|    14|  1|       0|       1|       1|       0|       0|\n",
      "|100014|     67703|    2042|       257|    24|     9|  0|       1|       0|       1|       0|       0|\n",
      "|   132|    475673|    1605|      1900|   134|    38|  1|       0|       1|       0|       1|       0|\n",
      "|   101|    433239|    1295|      1743|   144|    39|  0|       1|       0|       1|       0|       0|\n",
      "|    11|     47660|    2987|       192|    16|     8|  1|       0|       1|       1|       0|       0|\n",
      "|   138|    463125|    1599|      1868|   142|    47|  1|       1|       0|       0|       1|       0|\n",
      "|300017|    897406|    1784|      3632|   416|    64|  1|       0|       1|       0|       1|       0|\n",
      "|    29|    676516|    1442|      2715|   212|    47|  0|       1|       0|       0|       1|       0|\n",
      "|    69|    279030|    1714|      1095|   103|    19|  1|       0|       1|       1|       0|       0|\n",
      "|    42|    857297|    1610|      3474|   264|    62|  1|       0|       1|       1|       0|       0|\n",
      "|    73|     90961|    1194|       365|    25|     9|  0|       0|       1|       1|       0|       0|\n",
      "|    87|     34184|    1419|       136|     2|     3|  0|       1|       0|       1|       0|       0|\n",
      "|     3|     54424|    1941|       214|    18|     7|  0|       1|       0|       1|       0|       0|\n",
      "|   113|    380538|    2964|      1510|   127|    26|  1|       0|       1|       0|       1|       0|\n",
      "|    30|    145731|    1513|       575|    43|    15|  1|       1|       0|       1|       0|       0|\n",
      "+------+----------+--------+----------+------+------+---+--------+--------+--------+--------+--------+\n",
      "only showing top 20 rows\n",
      "\n"
     ]
    }
   ],
   "source": [
    "df_ubc.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 144,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pyspark.ml import Pipeline\n",
    "from pyspark.ml.feature import StringIndexer, OneHotEncoder, StandardScaler,VectorAssembler\n",
    "from pyspark.ml.classification import LogisticRegression\n",
    "from pyspark.ml.tuning import CrossValidator, ParamGridBuilder\n",
    "from pyspark.ml.evaluation import BinaryClassificationEvaluator\n",
    "\n",
    "rest, validation = df_ubc.randomSplit([0.7, 0.3], seed=42)\n",
    "\n",
    "\n",
    "input_col = ['userId','PlayedTime','PaidTime','PlayNumber','PosEva','NegEva','Gender_m','Gender_f','Device_w','Device_m','Device_l']\n",
    "vecAssembler = VectorAssembler(inputCols=input_col, outputCol= \"features\")\n",
    "stringIndexer = StringIndexer(inputCol=\"y\", outputCol=\"label\")\n",
    "\n",
    "lr =  LogisticRegression(maxIter=10, regParam=0.0, elasticNetParam=0)\n",
    "pipeline = Pipeline(stages=[vecAssembler,stringIndexer,lr])\n",
    "\n",
    "paramGrid = ParamGridBuilder().addGrid(lr.regParam,[0.0, 0.1]).build()\n",
    "\n",
    "crossval = CrossValidator(estimator=pipeline,\n",
    "                          estimatorParamMaps=paramGrid,\n",
    "                          evaluator=BinaryClassificationEvaluator(),\n",
    "                          numFolds=3)\n",
    "\n",
    "cvModel_q1 = crossval.fit(rest)\n",
    "predict_df = cvModel_q1.transform(validation)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 145,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0.6552706552706553, 0.6966096866096867]"
      ]
     },
     "execution_count": 145,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cvModel_q1.avgMetrics"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 148,
   "metadata": {},
   "outputs": [],
   "source": [
    "results = cvModel_q1.transform(validation)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 151,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.8253968253968254\n"
     ]
    }
   ],
   "source": [
    "print(results.filter(results.label == results.prediction).count() / results.count())"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}