1-3. 스토킹 잠재_가해자 분류 모델 학습.ipynb

{"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"provenance":[],"authorship_tag":"ABX9TyOTzT3n1CL1j6dbolLE1q24"},"kernelspec":{"name":"python3","display_name":"Python 3"},"language_info":{"name":"python"}},"cells":[{"cell_type":"markdown","source":["# **1-3. 스토킹 잠재/가해자 분류 모델 학습**"],"metadata":{"id":"Kl03e16-7Ma3"}},{"cell_type":"markdown","source":["---\n","> 1-2에서 학습한 설문조사 임베딩 모델을 기반으로 스토킹 잠재/가해자 분류 모델을 학습합니다.\n","\n","> 해석을 최우선 목표로 하고 있기 때문에, 스토킹 잠재/가해 여부 판별에 있어 어떤 문항이 어느 정도 기여했는지, 어떤 문항의 어떤 단어들이 얼마나 기여했는지 Attention을 통해 학습니다.\n","---"],"metadata":{"id":"f07iiTK77Paj"}},{"cell_type":"code","source":["# 드라이브 내 Custom Module 및 .ipynb 파일, 그리고 학습 데이터를 저장한 Directory를 입력하세요.\n","# 설문조사는 총 11개 문항으로 이루어져 있습니다. 학습을 원하는 문항의 번호를 입력하세요.\n","\n","DIRECTORY = \"AI경진대회\" # 여기를 변경하세요."],"metadata":{"id":"5R8UTVRL8ktj"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["## **(1) 라이브러리 준비**"],"metadata":{"id":"S9EOC7M18YtI"}},{"cell_type":"code","execution_count":null,"metadata":{"id":"aEqv1Lip7I7G"},"outputs":[],"source":["# Google Colab을 기반으로 학습을 진행할 경우, BERT 계열의 모델을 사용하기 위해 필요한 라이브러리를 설치합니다.\n","\n","!pip install mxnet\n","!pip install gluonnlp pandas tqdm\n","!pip install sentencepiece\n","!pip install transformers==3.0.2\n","!pip install torch\n","!pip install git+https://git@github.com/SKTBrain/KoBERT.git@master\n","!pip install 'git+https://github.com/SKTBrain/KoBERT.git#egg=kobert_tokenizer&subdirectory=kobert_hf'\n","!pip install torchmetrics"]},{"cell_type":"code","source":["import torch\n","import time\n","import os\n","import pickle\n","import random\n","import pandas as pd\n","import numpy as np\n","import torch.nn as nn\n","import torch.optim as optim\n","import torch.nn.functional as F\n","import gluonnlp as nlp\n","import matplotlib.pyplot as plt\n","from tqdm import tqdm\n","from kobert_tokenizer import KoBERTTokenizer\n","from transformers import BertTokenizer, BertModel, AutoModel, AutoTokenizer\n","from kobert import get_pytorch_kobert_model"],"metadata":{"id":"44vYSNBG8WoV"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["## **(2) Drive Mount 및 Custom Module 불러오기**"],"metadata":{"id":"wvlCqmlV8c4F"}},{"cell_type":"code","source":["# Drive Mount\n","from google.colab import drive\n","drive.mount('/content/gdrive')\n","\n","# Directory 변경\n","path = \"/content/gdrive/My Drive/\" + DIRECTORY\n","os.chdir(path)"],"metadata":{"id":"H7RgEP728iUr"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":["device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')"],"metadata":{"id":"sTaWtY99_vYU"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":["from dataset import STALK\n","from model import NewSE, Stalking\n","from train import train_siamese\n","from utils import make_weights"],"metadata":{"id":"rukyNJXM9IqS"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["## **(3) 데이터셋 불러오기**"],"metadata":{"id":"nliQb69b9jBc"}},{"cell_type":"code","source":["# 데이터셋 준비\n","\n","with open('./DATA/3. 잠재 및 가해 분류/train_aug.pickle', 'rb') as f:\n","    train = pickle.load(f)\n","\n","with open('./DATA/3. 잠재 및 가해 분류/valid_aug.pickle', 'rb') as f:\n","    valid = pickle.load(f)"],"metadata":{"id":"GEbAZ7e09jUi"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":["# Train/Valid 데이터의 warning에 대해 Label Encoding\n","\n","temp = []\n","\n","for i in range(train.shape[0]):\n","    if (train.loc[i, 'warning'] == 1) or (train.loc[i, 'warning'] == 2):\n","        temp.append(1)\n","    else:\n","        temp.append(0)\n","\n","train['warning'] = temp\n","\n","temp = []\n","\n","for i in range(valid.shape[0]):\n","    if (valid.loc[i, 'warning'] == 1) | (valid.loc[i, 'warning'] == 2):\n","        temp.append(1)\n","    else:\n","        temp.append(0)\n","\n","valid['warning'] = temp"],"metadata":{"id":"tyaz4JFl94Z9"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":["# Train/Valid 데이터의 status에 대해 Label Encoding\n","\n","temp = []\n","\n","for i in range(train.shape[0]):\n","    if train.loc[i, 'status'] == '상':\n","        temp.append(2)\n","    elif train.loc[i, 'status'] == '하':\n","        temp.append(1)\n","    else:\n","        temp.append(0)\n","\n","train['status'] = temp\n","\n","temp = []\n","\n","for i in range(valid.shape[0]):\n","    if valid.loc[i, 'status'] == '상':\n","        temp.append(2)\n","    elif valid.loc[i, 'status'] == '하':\n","        temp.append(1)\n","    else:\n","        temp.append(0)\n","\n","valid['status'] = temp"],"metadata":{"id":"iP_B_N7GFBvS"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["## **(4) KcELECTRA, Tokenizer 불러오기**"],"metadata":{"id":"2o0j0y-J_cou"}},{"cell_type":"code","source":["# KcELECTRA, Tokenizer\n","\n","tokenizer = AutoTokenizer.from_pretrained(\"beomi/KcELECTRA-base-v2022\")\n","kc_model = AutoModel.from_pretrained(\"beomi/KcELECTRA-base-v2022\")"],"metadata":{"id":"FGuJ29Cw_bbw"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["## **(5) DataLoader 준비**"],"metadata":{"id":"4zccCzIS_mLc"}},{"cell_type":"code","source":["# 매우 불균형한 데이터의 형태를 고려하여 Train DataLoader에 Weighted Sampler 적용\n","\n","weights = make_weights(train['warning'], 2)\n","weights = torch.DoubleTensor(weights).to(dtype=torch.float32)\n","sampler = torch.utils.data.sampler.WeightedRandomSampler(weights, len(weights))"],"metadata":{"id":"DhNr7cVTFb0v"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":["# Dataset\n","\n","train_set = STALK(tokenizer=tokenizer, embed_model=kc_model, data=train, shuffle=False, device=device)\n","valid_set = STALK(tokenizer=tokenizer, embed_model=kc_model, data=valid, shuffle=False, device=device)"],"metadata":{"id":"nukv_FMY_tTA"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":["# DataLoader\n","\n","BATCH_SIZE=64\n","\n","train_loader = torch.utils.data.DataLoader(train_set, batch_size=BATCH_SIZE, shuffle=False, drop_last=True, sampler=sampler)\n","valid_loader = torch.utils.data.DataLoader(valid_set, batch_size=BATCH_SIZE, shuffle=False, drop_last=True)"],"metadata":{"id":"djLrlmXz__Pe"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["## **(6) 모델 생성**"],"metadata":{"id":"4vwsGGt2AD8o"}},{"cell_type":"code","source":["# Model (Classifier)\n","\n","model = Stalking(device=device, model_path='./TRAINED_MODEL/SE', batch_size=64) "],"metadata":{"id":"zOYzJJu1AGFz"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["## **(7) 모델 학습**"],"metadata":{"id":"vft965Q3AL21"}},{"cell_type":"code","source":["# Train Model\n","\n","model_aft, train_loss, valid_loss = train_stalking(train_dataloader=train_loader,\n","                                                   valid_dataloader=valid_loader,\n","                                                   valid=True,\n","                                                   model=model,\n","                                                   epochs=20,\n","                                                   optimizer=optim.Adam,\n","                                                   criterion=nn.BCELoss,\n","                                                   scheduler=optim.lr_scheduler.StepLR,\n","                                                   gamma=0.8,\n","                                                   step_size=2,\n","                                                   lr=8e-03,\n","                                                   coef=0.1,\n","                                                   device=device)"],"metadata":{"id":"Wqx2zjIkAKn-"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["## **(8) 학습 결과 시각화**"],"metadata":{"id":"ZhsksCRrAdBS"}},{"cell_type":"code","source":["# Accuracy 시각화\n","\n","plt.figure(figsize=(12, 6))\n","plt.plot(train_loss)\n","plt.plot(valid_loss)\n","plt.title('Loss History')\n","plt.xlabel(\"Epochs\")\n","plt.ylabel(\"Loss\")\n","plt.grid(True)\n","\n","plt.tight_layout()\n","plt.show()"],"metadata":{"id":"4sLd70joAQi_"},"execution_count":null,"outputs":[]}]}