examples/advanced/sklearn-linear/sklearn_linear_higgs.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "cada310b-e776-4b9a-aabe-f111c31efcc2",
   "metadata": {
    "tags": []
   },
   "source": [
    "# Federated Linear Model with Scikit-learn on HIGGS Dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0653cbf2-92f2-4a22-8317-69cfb0266e92",
   "metadata": {},
   "source": [
    "## Introduction to Scikit-learn, tabular data, and federated k-Means\n",
    "### Scikit-learn\n",
    "This example shows how to use [NVIDIA FLARE](https://nvflare.readthedocs.io/en/main/index.html) on tabular data.\n",
    "It uses [Scikit-learn](https://scikit-learn.org/),\n",
    "a widely used open-source machine learning library that supports supervised \n",
    "and unsupervised learning.\n",
    "### Tabular data\n",
    "The data used in this example is tabular in a format that can be handled by [pandas](https://pandas.pydata.org/), such that:\n",
    "- rows correspond to data samples\n",
    "- the first column represents the label \n",
    "- the other columns cover the features.    \n",
    "\n",
    "Each client is expected to have one local data file containing both training \n",
    "and validation samples. To load the data for each client, the following \n",
    "parameters are expected by the local learner:\n",
    "- data_file_path: string, the full path to the client's data file \n",
    "- train_start: int, start row index for the training set\n",
    "- train_end: int, end row index for the training set\n",
    "- valid_start: int, start row index for the validation set\n",
    "- valid_end: int, end row index for the validation set\n",
    "\n",
    "### Federated Linear Model\n",
    "This example shows the use of [linear classifiers with SGD training](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.SGDClassifier.html) in a federated scenario.\n",
    "Under this setting, federated learning can be formulated as a [FedAvg](https://arxiv.org/abs/1602.05629) process with local training that each client optimizes the local model starting from global parameters with SGD. \n",
    "This can be achieved by setting the `warm_start` flag of SGDClassifier to \n",
    "`True` in order to allow repeated fitting of the classifiers to the local data.\n",
    "\n",
    "Below we listed steps to run this example."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a5a0292c-78b6-4bde-96d6-699dae996173",
   "metadata": {},
   "source": [
    "## 1. Setup NVFLARE\n",
    "\n",
    "Follow the [Getting_Started](https://nvflare.readthedocs.io/en/main/getting_started.html) to setup virtual environment and install NVFLARE\n",
    "\n",
    "We also provide a [Notebook](../../nvflare_setup.ipynb) for this setup process. \n",
    "\n",
    "Assume you have already setup the venv, lets first install required packages."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f4130b15-09e6-456f-a3c7-87c8ee9e07f0",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "%pip install -r requirements.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1d872d8a-9e44-49dd-94b1-7862b3815ffe",
   "metadata": {},
   "source": [
    "## 2. Data preparation \n",
    "The examples illustrate a binary classification task based on [HIGGS dataset](https://mlphysics.ics.uci.edu/data/higgs/).\n",
    "This dataset contains 11 million instances, each with 28 attributes.\n",
    "By default, we assume the dataset is downloaded, uncompressed, and stored in `DATASET_ROOT/HIGGS.csv`.\n",
    "Please note that the UCI's website may experience occasional downtime.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "af257e69-2bb7-49b6-ac6c-f007b0e6618e",
   "metadata": {},
   "source": [
    "## 3. Prepare clients' configs with proper data split information \n",
    "We are using NVFlare's FL simulator to run the following experiments. Here we simulate 5 clients with uniform data split. Since HIGGS dataset is already randomly recorded, the split is done sequentially. Please change the DATASET_ROOT to the correct local path"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fdb7290a-48ff-4e80-be58-5e6b0e0f9379",
   "metadata": {
    "scrolled": true,
    "tags": []
   },
   "outputs": [],
   "source": [
    "%env DATASET_ROOT=/data\n",
    "!python3 ./utils/prepare_job_config.py \\\n",
    "        --task_name \"sklearn_linear\" \\\n",
    "        --data_path \"${DATASET_ROOT}/HIGGS.csv\" \\\n",
    "        --site_num 5 \\\n",
    "        --valid_frac 0.1 \\\n",
    "        --split_method \"uniform\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bd0713e2-e393-41c0-9da0-392535cf8a54",
   "metadata": {},
   "source": [
    "## 4. Run simulated experiment\n",
    "Now that we have the job configs ready, we run the experiment using Simulator."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bde19b89-3aef-49ac-99d9-800d65829830",
   "metadata": {},
   "source": [
    "Simulator can either be used with CLI command: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9bb6cab4-9c24-400a-bc3c-f1e4a6d5a346",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "! nvflare simulator ./jobs/sklearn_linear_5_uniform -w /tmp/nvflare/sklearn_linear_higgs -n 5 -t 5"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3915369e-62eb-4903-a120-69b238791075",
   "metadata": {},
   "source": [
    "or via Simulator API:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "33c75dcb-014d-40c4-8a4a-7a53847c486b",
   "metadata": {
    "scrolled": true,
    "tags": []
   },
   "outputs": [],
   "source": [
    "import os\n",
    "from nvflare.private.fed.app.simulator.simulator_runner import SimulatorRunner  \n",
    "\n",
    "simulator = SimulatorRunner(\n",
    "    job_folder=f\"./jobs/sklearn_linear_5_uniform\",\n",
    "    workspace=\"/tmp/nvflare/sklearn_linear_higgs\",\n",
    "    n_clients=5,\n",
    "    threads=5\n",
    ")\n",
    "run_status = simulator.run()\n",
    "print(\"Simulator finished with run_status\", run_status)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "913e9ee2-e993-442d-a525-d2baf92af539",
   "metadata": {},
   "source": [
    "## 5. Result visualization\n",
    "Model accuracy is computed as the homogeneity score between the cluster formed and the ground truth label, which can be visualized in tensorboard."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a6814434-4e6d-4460-b480-709cb3e77cc8",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "%load_ext tensorboard\n",
    "%tensorboard --logdir /tmp/nvflare/sklearn_linear_higgs"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "nvflare_example",
   "language": "python",
   "name": "nvflare_example"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}