Correct until 'Simulate the exporting'. Need to check the correct wri…

…ting of cell_types.pk in create_test_content, and then the nbm_update_cli
JaumeAmoresDS · Sep 6, 2024 · 0a84589 · 0a84589
1 parent 7a11908
commit 0a84589
Show file tree

Hide file tree

Showing 16 changed files with 804 additions and 96 deletions.
diff --git a/nbs/dev_tutorials/ast_examples.ipynb → nbm/dev_tutorials/ast_examples.ipynb b/nbs/dev_tutorials/ast_examples.ipynb → nbm/dev_tutorials/ast_examples.ipynb
diff --git a/nbm/dev_tutorials/writing_tests_for_nbmodular.ipynb b/nbm/dev_tutorials/writing_tests_for_nbmodular.ipynb
@@ -0,0 +1,319 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Writing tests for nbmodular"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Writing tests for export / import functionality"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Update functionality (import)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "#### 1. Simulate the exporting\n",
+    "\n",
+    "```python\n",
+    "new_root = \"test_nbm_update\"\n",
+    "nb_folder = \"nbm\"\n",
+    "nb_path = \"mixed/mixed_cells.ipynb\"\n",
+    "# Create notebook in \"new repo\", and cd to it\n",
+    "current_root, nb_paths = tst.create_test_content(\n",
+    "    nbs=tst.mixed_nb1,\n",
+    "    nb_paths=nb_path,\n",
+    "    nb_folder=nb_folder,\n",
+    "    new_root=new_root,\n",
+    ")\n",
+    "\n",
+    "nbm_export(path=f\"{nb_folder}/{nb_path}\")\n",
+    "\n",
+    "exported_nbs, updated_py_modules = tst.read_content_in_repo(\n",
+    "    [nb_path], \"./\", print_as_list=True\n",
+    ")\n",
+    "joblib.load(\"test_nbm_update/.nbmodular/cell_types.pk\")\n",
+    "```\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "vscode": {
+     "languageId": "plaintext"
+    }
+   },
+   "source": [
+    "#####  2. Manually updated the py modules\n",
+    "\n",
+    "```python\n",
+    "updated_py_modules = [...]\n",
+    "```\n",
+    "\n",
+    "Copy and paste into test_utils module:\n",
+    "\n",
+    "- content of `exported_nbs` and `paths`\n",
+    "- content of `updated_py_modules` and `paths`\n",
+    "- content of joblib.load(\"test_nbm_update/.nbmodular/cell_types.pk\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "vscode": {
+     "languageId": "plaintext"
+    }
+   },
+   "source": [
+    "#### 3. Implement *set up before running example*\n",
+    "\n",
+    "```python\n",
+    "updated_py_modules = [x.replace(\"@%%\", \"%%\") for x in tst.updated_py_modules]\n",
+    "new_root = \"test_nbm_update\"\n",
+    "nb_folder = \"nbm\"\n",
+    "lib_folder = \"nbmodular\"\n",
+    "# Create notebook in \"new repo\", and cd to it\n",
+    "current_root, nb_paths = tst.create_test_content(\n",
+    "    nbs=tst.exported_nbs,\n",
+    "    nb_paths=tst.exported_nb_paths,\n",
+    "    py_modules=updated_py_modules,\n",
+    "    py_paths=tst.updated_py_paths,\n",
+    "    nb_folder=\"\",\n",
+    "    lib_folder=\"\",\n",
+    "    new_root=new_root,\n",
+    ")\n",
+    "```\n",
+    "\n",
+    "The following should be a variable in `tst`:\n",
+    "\n",
+    "```python\n",
+    "tst.updated_cell_types = [\"code\", \"original\", \"test\"]\n",
+    "```\n",
+    "\n",
+    "The following should be part of `tst.create_test_content`\n",
+    "\n",
+    "```python\n",
+    "os.makedirs(\".nbmodular\", exist_ok=True)\n",
+    "joblib.dump(cell_types, \".nbmodular/cell_types.pk\")\n",
+    "```"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "vscode": {
+     "languageId": "plaintext"
+    }
+   },
+   "source": [
+    "#### 4. Write example usage\n",
+    "\n",
+    "```python\n",
+    "nb_path = \"mixed/mixed_cells.ipynb\"\n",
+    "nbm_update(path=f\"{nb_folder}/{nb_path}\")\n",
+    "```"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "vscode": {
+     "languageId": "plaintext"
+    }
+   },
+   "source": [
+    "#### 5. Write checks\n",
+    "\n",
+    "##### 5.1 Read result and manually check if it's correct\n",
+    "\n",
+    "```python\n",
+    "expected_nbs, expected_py_modules = tst.read_content_in_repo(\n",
+    "    [nb_path], \"./\", print_as_list=True\n",
+    ")\n",
+    "```\n",
+    "\n",
+    "##### 5.2 copy and paste the result\n",
+    "\n",
+    "```python\n",
+    "expected_nbs = [...]\n",
+    "expected_py_modules = [...]\n",
+    "```\n",
+    "\n",
+    "##### 5.3 call `check_test_repo_content` with the expected result\n",
+    "\n",
+    "```python\n",
+    "tst.check_test_repo_content(\n",
+    "    nb_paths,\n",
+    "    expected_nbs=expected_nbs,\n",
+    "    expected_py_modules=expected_py_modules,\n",
+    "    current_root=current_root,\n",
+    "    new_root=new_root,\n",
+    "    clean=False,\n",
+    "    keep_cwd=True,\n",
+    ")\n",
+    "```"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "#####\n",
+    "    # manually updated the py modules\n",
+    "    # updated_py_modules = [...]\n",
+    "\n",
+    "    # copy and paste into test_utils module:\n",
+    "    # - content of exported_nbs and paths\n",
+    "    # - content of updated_py_modules and paths\n",
+    "\n",
+    "# %% [markdown]\n",
+    "# #### Set up before running example\n",
+    "\n",
+    "\n",
+    "# %%\n",
+    "updated_py_modules = [x.replace(\"@%%\", \"%%\") for x in tst.updated_py_modules]\n",
+    "new_root = \"test_nbm_update\"\n",
+    "nb_folder = \"nbm\"\n",
+    "lib_folder = \"nbmodular\"\n",
+    "# Create notebook in \"new repo\", and cd to it\n",
+    "current_root, nb_paths = tst.create_test_content(\n",
+    "    nbs=tst.exported_nbs,\n",
+    "    nb_paths=tst.exported_nb_paths,\n",
+    "    py_modules=updated_py_modules,\n",
+    "    py_paths=tst.updated_py_paths,\n",
+    "    nb_folder=\"\",\n",
+    "    lib_folder=\"\",\n",
+    "    new_root=new_root,\n",
+    ")\n",
+    "cell_types = [\"code\", \"original\", \"test\"]\n",
+    "os.makedirs(\".nbmodular\", exist_ok=True)\n",
+    "joblib.dump(cell_types, \".nbmodular/cell_types.pk\")\n",
+    "\n",
+    "# %% [markdown]\n",
+    "# #### Example usage\n",
+    "\n",
+    "# %%\n",
+    "\n",
+    "nb_path = \"mixed/mixed_cells.ipynb\"\n",
+    "nbm_update(path=f\"{nb_folder}/{nb_path}\")\n",
+    "\n",
+    "# %% [markdown]\n",
+    "# #### Checks\n",
+    "\n",
+    "# %%\n",
+    "expected_nbs = [\n",
+    "    # nbm/mixed/mixed_cells.ipynb\n",
+    "    \"\"\"\n",
+    "[code]\n",
+    "%%function\n",
+    "def first():\n",
+    "    x = 3 + 1\n",
+    "\n",
+    "[markdown]\n",
+    "comment\n",
+    "\n",
+    "[code]\n",
+    "%%function --test\n",
+    "def second():\n",
+    "    print(\"hello\")\n",
+    "\"\"\",\n",
+    "    # nbs/mixed/mixed_cells.ipynb\n",
+    "    \"\"\"\n",
+    "[code]\n",
+    "#|default_exp mixed.mixed_cells\n",
+    "\n",
+    "[code]\n",
+    "#|export\n",
+    "#@@function\n",
+    "def first():\n",
+    "    x = 3 + 1\n",
+    "\"\"\",\n",
+    "    # nbs/mixed/test_mixed_cells.ipynb\n",
+    "    \"\"\"\n",
+    "[code]\n",
+    "#|default_exp tests.mixed.test_mixed_cells\n",
+    "\n",
+    "[code]\n",
+    "#|export\n",
+    "#@@function --test\n",
+    "def second():\n",
+    "    print(\"hello\")\n",
+    "\"\"\",\n",
+    "]\n",
+    "expected_py_modules = [\n",
+    "    # nbmodular/mixed/mixed_cells.py\n",
+    "    \"\"\"\n",
+    "\n",
+    "# @%% auto 0\n",
+    "__all__ = ['first']\n",
+    "\n",
+    "# @%% ../../nbs/mixed/mixed_cells.ipynb 1\n",
+    "#@@function\n",
+    "def first():\n",
+    "    x = 3 + 1\n",
+    "\n",
+    "\"\"\",\n",
+    "    # nbmodular/tests/mixed/test_mixed_cells.py\n",
+    "    \"\"\"\n",
+    "\n",
+    "# @%% auto 0\n",
+    "__all__ = ['second']\n",
+    "\n",
+    "# @%% ../../../nbs/mixed/test_mixed_cells.ipynb 1\n",
+    "#@@function --test\n",
+    "def second():\n",
+    "    print(\"hello\")\n",
+    "\n",
+    "\"\"\",\n",
+    "]\n",
+    "\n",
+    "\n",
+    "# %%\n",
+    "tst.check_test_repo_content(\n",
+    "    nb_paths,\n",
+    "    expected_nbs=expected_nbs,\n",
+    "    expected_py_modules=expected_py_modules,\n",
+    "    current_root=current_root,\n",
+    "    new_root=new_root,\n",
+    "    clean=False,\n",
+    "    keep_cwd=True,\n",
+    ")\n",
+    "\n",
+    "\n",
+    "# %%\n",
+    "\n",
+    "if False:\n",
+    "    # 3) read result and manually check if it's correct\n",
+    "    expected_nbs, expected_py_modules = tst.read_content_in_repo(\n",
+    "        [nb_path], \"./\", print_as_list=True\n",
+    "    )\n",
+    "\n",
+    "    # copy and paste the result:\n",
+    "    # expected_nbs = [...]\n",
+    "    # expected_py_modules = [...]\n",
+    "\n",
+    "    # Finally we check that the result is the same as the expected"
+   ]
+  }
+ ],
+ "metadata": {
+  "language_info": {
+   "name": "python"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}
diff --git a/nbm/test_nbs/nb.ipynb b/nbm/test_nbs/nb.ipynb
diff --git a/nbs/tutorials/.nodoc → nbm/tutorials/.nodoc b/nbs/tutorials/.nodoc → nbm/tutorials/.nodoc
diff --git a/nbs/tutorials/.notest → nbm/tutorials/.notest b/nbs/tutorials/.notest → nbm/tutorials/.notest
diff --git a/nbs/tutorials/debugging.ipynb → nbm/tutorials/debugging.ipynb b/nbs/tutorials/debugging.ipynb → nbm/tutorials/debugging.ipynb
diff --git a/nbs/tutorials/index.qmd → nbm/tutorials/index.qmd b/nbs/tutorials/index.qmd → nbm/tutorials/index.qmd
diff --git a/nbs/tutorials/test_tutorial.ipynb → nbm/tutorials/test_tutorial.ipynb b/nbs/tutorials/test_tutorial.ipynb → nbm/tutorials/test_tutorial.ipynb
diff --git a/nbmodular.yml b/nbmodular.yml
@@ -0,0 +1,4 @@
+name: nbmodular 
+dependencies:
+  - python=3.10
+  - pip