input_output.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Input / Output"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Table of Contents\n",
    "  - [Input / Output](#Input-/-Output)\n",
    "  - [Table of Contents](#Table-of-Contents)\n",
    "    - [References](#References)\n",
    "    - [Introduction](#Introduction)\n",
    "    - [String input and output ](#String-input-and-output)\n",
    "      - [String output](#String-output)\n",
    "        - [Quiz on string output:](#Quiz-on-string-output:)\n",
    "      - [String input](#String-input)\n",
    "        - [Exercises on string input and output](#Exercises-on-string-input-and-output)\n",
    "    - [File I/O](#File-I/O)\n",
    "      - [Paths](#Paths)\n",
    "        - [Quiz on Paths](#Quiz-on-Paths)\n",
    "        - [Exercises on paths](#Exercises-on-paths)\n",
    "      - [Reading from a file](#Reading-from-a-file)\n",
    "        - [Quiz on file reading](#Quiz-on-file-reading)\n",
    "        - [Exercises on file reading](#Exercises-on-file-reading)\n",
    "      - [Writing to a file](#Writing-to-a-file)\n",
    "        - [Quiz on file writing](#Quiz-on-file-writing)\n",
    "        - [Exercises on file writing](#Exercises-on-file-writing)\n",
    "      - [Context managers](#Context-managers)\n",
    "        - [Quiz on context managers](#Quiz-on-context-managers)\n",
    "      - [Binary I/O](#Binary-I/O)\n",
    "        - [Bytes and strings](#Bytes-and-strings)\n",
    "        - [Converting bytes to text ](#Converting-bytes-to-text)\n",
    "      - [Reading/Writing CSV files](#Reading/Writing-CSV-files)\n",
    "        - [Quiz on CSV](#Quiz-on-CSV)\n",
    "    - [Exercises](#Exercises)\n",
    "      - [Exercise 1: CSV to dictionary 🌶️](#Exercise-1:-CSV-to-dictionary-🌶️)\n",
    "      - [Exercise 2: Counting words 🌶️](#Exercise-2:-Counting-words-🌶️)\n",
    "      - [Exercise 3: Letter statistics 🌶️🌶️](#Exercise-3:-Letter-statistics-🌶️🌶️)\n",
    "      - [Exercise 4: Translating words 🌶️🌶️](#Exercise-4:-Translating-words-🌶️🌶️)\n",
    "      - [Exercise 5: Binary format 🌶️🌶️🌶️](#Exercise-5:-Binary-format-🌶️🌶️🌶️)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "\n",
    "## References\n",
    "- [Input and output tutorial](https://docs.python.org/3/tutorial/inputoutput.html) from the official Python tutorial.\n",
    "- [Is explicitly closing files important?](https://stackoverflow.com/questions/7395542/is-explicitly-closing-files-important)\n",
    "- [Context managers](https://docs.python.org/3/library/contextlib.html) from the Python standard library.\n",
    "\n",
    "---"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introduction\n",
    "An important aspect to learn in a new programming language is input/output (or I/O). For our program to be useful, we need to be able to interact with the outside world. This means performing operations like:\n",
    "- reading and writing data to a file\n",
    "- printing text output\n",
    "- asking for user input\n",
    "- connecting to databases or other network services\n",
    "\n",
    "\n",
    "The majority of these operations are covered by the Python standard library. We are going to see how to use them in this chapter.\n",
    "\n",
    "<div class=\"alert alert-block alert-info\">\n",
    "    <h4><b>Note</b></h4>\n",
    "In reference to the chapter on functional programming, it is interesting to note that these functions perform <b>side-effects</b>. Therefore, any code containing these operations is no longer <b>pure</b> and is not referentially transparent. The same function can return different values for the same argument if called multiple times, and the function can have <b>long-distance</b> effects. That means they can modify the program state elsewhere, leading to unexpected results.<br><br>\n",
    "Therefore, we suggest separating input and output from the other computations in your program. For example, if you have a complex calculation requiring several user inputs at several stages of the process, consider writing a function that only performs the calculation given all inputs and then requires all inputs separately, for example, through a single file. This makes your code easier to debug, test and understand.\n",
    "</div>"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## String input and output \n",
    "\n",
    "### String output\n",
    "The most basic I/O operation in Python is displaying text on the Python console. This is done using the `print` function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"I am some text\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is also possible to print any other python object using `print`. \n",
    "In that case, the `__str__` **magic method** on that object's class is [called](https://docs.python.org/3/reference/datamodel.html#object.__str__)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print([])\n",
    "print([1,2,3])\n",
    "print({\"key\": \"value\"})\n",
    "print(lambda x: x)\n",
    "print((\"some\", \"tuple\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we want to display the value of a variable in a string, we can do it most conveniently using **string interpolation**. \n",
    "To do so, we prepend `f` to a string, which can contain reference to the variables we want to print enclosed in \"{}\":"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "my_var = 3\n",
    "my_string = f\"my_var is {my_var}\"\n",
    "print(my_string)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Quiz on string output:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import tutorial.quiz.input_output as op\n",
    "op.StringOutput()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### String input\n",
    "\n",
    "The complementary operation, prompting for *user input*, is done through `input`.\n",
    "The function takes one string as an argument, which gives the prompt displayed on the console.\n",
    "The return value of the function contains the input the user typed:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "user_input = input(\"Write some text\")\n",
    "print(f\"The user wrote {user_input}\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "<div class=\"alert alert-block alert-danger\">\n",
    "    <h4><b>Important</b></h4>\n",
    "    ⚠️ Note that <code>input</code> is a <b>blocking operation</b>. The program execution  stops and waits until the user provides an input. Therefore, use <code>input</code> <b>carefully</b> and only when truly necessary. A common issue is that someone first writes an interactive program expecting user input through <code>input</code> and later integrates it into a larger application that is supposed to run automatically without any user interaction. Suddenly, the application stops somewhere and does not run further because of a well hidden <code>input</code> call...<br><br>\n",
    "\n",
    "It is more common to have an input file where we read from if we need a user input (reading from a file will come later). If you have to use <code>input</code>, do not do so in the middle of long-running computation or your model code, but ask for all user inputs upfront. For an interactive application, there are more robust solutions.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Quiz on string input"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import tutorial.quiz.input_output as op\n",
    "op.StringInput()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercises on string input and output"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%reload_ext tutorial.tests.testsuite"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. Complete the function `solution_print_odd` to **print** all the *odd* numbers between 0 and `n`, **without `n`**. The value of `n` is provided as the parameter of the function `solution_print_odd`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%ipytest debug\n",
    "\n",
    "def solution_print_odd(n: int) -> None: \n",
    "    \"\"\"Prints all odd numbers from 1 to n\n",
    "    \n",
    "    Args:\n",
    "        n : The maximum number to print (exclusive)\n",
    "\n",
    "    Returns:\n",
    "        - None (prints to console)\n",
    "    \"\"\"\n",
    "    pass"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "2. Complete the function `solution_print_salutation`. This function prompts the user for their name using `input` and should **print** the text \"Hello, `<name>`\", where `<name>` should be replaced with value read from input\n",
    "\n",
    "<div class=\"alert alert-block alert-warning\">\n",
    "    <h4><b>Note</b></h4>\n",
    "    There is no test for this exercise because we cannot easily interact with <code>input</code> in our testing framework. You can check visually if the input does what is expected.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def solution_print_salutation() -> None:\n",
    "    \"\"\"Prints a salutation to the console\n",
    "\n",
    "    Takes no arguments but takes input with the input() function\n",
    "    The salutation is Hello, {name} where {name} is the user input\n",
    "\n",
    "    Returns:\n",
    "        - None (prints to console)\n",
    "    \"\"\"\n",
    "    pass\n",
    "\n",
    "solution_print_salutation()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## File I/O\n",
    "A second common I/O operation in most programming languages is reading from and writing to files. This is more complex than just writing on the console and consists of several steps\n",
    "\n",
    "1. You need to find *where* in the operating system the file is located. This gives you the so-called *path* to the file.\n",
    "2. You need to *open* the file for reading or writing. In some operating systems, this *locks* the file, so that other users or processes cannot write simultaneously with you. This operation gives you a *file handle* that you can use to read or write.\n",
    "3. Now, you can read (or write) the contents of the file. \n",
    "4. Finally, you need to *close* the file to make it accessible to other processes again and to finalize the writing. In some implementations, `write` only writes the text to a temporary location in memory (a so-called *buffer*) and only writes the content to the file when you close it. This is done to increase response times, as writing to memory is faster than writing to a file on the disk.\n",
    "\n",
    "\n",
    "As a trick to remember closing files: *a file handle is like a door handle*. \n",
    "\n",
    "### Paths\n",
    "To handle the *where* a file is located, Python offers the `pathlib` module, which is used to represent *paths* to files in a portable manner.\n",
    "This means that you can use one Python object to represent the location of a file regardless of the operating system, and Python will convert it into a representation specific to the operating system where you are executing the program.\n",
    "For example, Windows uses `\\` as a separator for path elements, while Unix/Linux use `/`.\n",
    "If you use pathlib, you can write code using only `/` and `pathlib`  will automatically select the correct path separator where the program runs.\n",
    "\n",
    "Let's see how to generate a `path` object pointing to the text file [`hello.txt`](./tutorial/tests/data/hello.txt) in the main directory of this repository.\n",
    "To do so, we use the [`Path`](https://docs.python.org/3/library/pathlib.html#basic-use) class of `pathlib` module, whose constructor takes the location of the file as a string:\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import pathlib as pl\n",
    "\n",
    "path = pl.Path(\"./tutorial/tests/data/hello.txt\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `./` at the beginning of the path indicates that this path is a *relative path*. The `.` represents the directory we are working in, so where we are executing the program. This is just a path that points to a file *relative* to the directory we are working in. \n",
    "\n",
    "The `path` object has a method `absolute` that returns the *absolute* full path of the object in the OS:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "print(path.absolute())"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`path` objects also have additional methods like:\n",
    "1. `name` to get only the name of the file\n",
    "2. `parent` to get the parent folder of the current file\n",
    "3. `suffix` to get the suffix of a file\n",
    "4. `is_dir()` to check whether the path is a directory\n",
    "5. `is_file()` to check whether the path is a file\n",
    "6. `iterdir()` to get a list of all files (and directories) in the current path, if the path is a directory.\n",
    "7. Many more methods are listed [here](https://docs.python.org/3/library/pathlib.html#general-properties)\n",
    "\n",
    "Below, we see some of these methods at work:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pathlib as pl\n",
    "p = pl.Path(\"tutorial/tests/data/hello.txt\")\n",
    "print(p.name)\n",
    "print(p.parent)\n",
    "print(p.suffix) \n",
    "print(p.is_dir())\n",
    "print(list(p.parent.iterdir()))\n",
    "print(p.is_file())"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we prefer, we can also construct paths by combining their components with the `/` operator. For example, we can do:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import pathlib as pl\n",
    "\n",
    "path = pl.Path(\"./tutorial/tests\") / pl.Path(\"data\")\n",
    "print(path.absolute())\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This also works to combine string and paths, and even just strings, provided that as the first or second object in the chain of `/` we have a `Path` object:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "#This works\n",
    "p1 = pl.Path(\".\") / \"tutorial/tests/data/hello.txt\"\n",
    "#This too\n",
    "p2 = pl.Path(\".\") / \"tutorial\" / \"tests\" / \"data\" / \"hello.txt\"\n",
    "#This as well\n",
    "p3 = \".\" / pl.Path(\"tutorial\") / \"tests\" / \"data\" / \"hello.txt\"\n",
    "\n",
    "print(p1, p2, p3)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Sometimes one must refer to the parent folder of a file. A `..` in a path refers to the parent folder of the current location."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "#Although different paths, these lead to the same file:\n",
    "p1 = pl.Path(\"./tutorial/tests/data/hello.txt\")\n",
    "p2 = pl.Path(\"./tutorial/tests/../../tutorial/tests/data/hello.txt\")\n",
    "\n",
    "print(f\"The paths {p1} and {p2} lead to the same file: {p1.samefile(p2)}\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also list all files matching a certain pattern in a given directory using the `glob` method. \n",
    "The method returns an `iterable` of paths that can be turned into a list like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "[f for f in path.glob(\"*\")]"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `glob` method takes a `pattern` argument that expresses the form of the file names to look for.\n",
    "The `*` star means *match everything*. For more information on *glob patterns*, see the documentation of [fnmatch](https://docs.python.org/3/library/fnmatch.html#module-fnmatch)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Quiz on Paths\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import tutorial.quiz.input_output as op\n",
    "op.Paths()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercises on paths"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%reload_ext tutorial.tests.testsuite"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "1. Modify the function `solution_find_all_files` to find all files and directories in the [data](./tutorial/tests/data/) (./tutorial/tests/data/) directory and return them as a list of `Path` objects\n",
    "\n",
    "<div class=\"alert alert-block alert-info\">\n",
    "    <b>Hint:</b> The path to the data directory is available as the argument <code>current_path</code> of the function <code>solution_find_all_files</code>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%ipytest\n",
    "\n",
    "from pathlib import Path\n",
    "\n",
    "def solution_find_all_files(current_path: Path) -> list[Path]: \n",
    "    \"\"\"Finds all files in a directory\n",
    "\n",
    "    Args:\n",
    "        current_path : The directory to search for files\n",
    "\n",
    "    Returns:\n",
    "        - A list of all files in the directory\n",
    "    \"\"\"\n",
    "    return"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "2. Modify the function `solution_check_for_file` to check if there is a file named `file_name` in the directory `current_path` and return true if there is one and false if there isn't."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%ipytest\n",
    "\n",
    "from pathlib import Path\n",
    "\n",
    "def solution_check_for_file(current_path: Path, file_name: str) -> bool: \n",
    "    \"\"\"Checks if a file exists in a directory\n",
    "\n",
    "    Args:\n",
    "        current_path : The directory to search for the file\n",
    "        file_name : The name of the file to search for\n",
    "\n",
    "    Returns:\n",
    "        - True if the file exists, False otherwise    \n",
    "    \"\"\"\n",
    "    \n",
    "    return\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "3. Modify the function `solution_count_dirs` to count **all the directories** (including hidden ones) in the `input_path`. Return total number of directories found as an integer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%ipytest\n",
    "\n",
    "from pathlib import Path\n",
    "\n",
    "def solution_count_dirs(input_path: Path) -> int: \n",
    "    \"\"\"Counts the number of directories in a directory\n",
    "\n",
    "    Args:\n",
    "        input_path : The directory to search for directories\n",
    "\n",
    "    Returns:\n",
    "        - The number of directories in the directory\n",
    "    \"\"\"\n",
    "    pass\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "\n",
    "### Reading from a file\n",
    "\n",
    "We now want to learn how to read text from a file. \n",
    "Let's see how to do this with an example: we want to open the file [hello.txt](./data/hello.txt) and read its contents.\n",
    "\n",
    "1. The path is already identified, we know the file is in `./data/hello.txt`. We save this in a variable `path`.\n",
    "2. We now can use the built-in function [`open`](https://docs.python.org/3/library/functions.html#open) to open the file. This function returns a [file object](https://docs.python.org/3/glossary.html#term-file-object) that we can use to further manipulate the file. To ensure we only open the file for reading, we pass the string \"r\" to the second argument of `open`.\n",
    "3. Now we can read the contents using `read`, `readline` or `readlines`. `read` reads the whole file content into a single string, `readline` reads one line, while `readlines` reads the whole file content as a list of strings, one item per line in the file. This knowledge is useful when we only want to read part of a file or when the file is too big to fit in memory and we can only read parts.\n",
    "4. Finally, we close the file using the `close` method on the file object.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import pathlib as pl\n",
    "\n",
    "path = pl.Path(\"./tutorial/tests/data/hello.txt\")\n",
    "file_ob = open(path, 'r')\n",
    "contents = file_ob.readlines()\n",
    "file_ob.close()\n",
    "print(contents)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice that calling `read`, `readline` or `readlines` *consumes* the file, either fully or to the corresponding location. This means that if we call `readlines` twice, we will get an empty list the second time:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import pathlib as pl\n",
    "\n",
    "path = pl.Path(\"./tutorial/tests/data/hello.txt\")\n",
    "file_ob = open(path, 'r')\n",
    "contents = file_ob.readlines()\n",
    "print(contents)\n",
    "other_contents = file_ob.readlines()\n",
    "print(other_contents)\n",
    "file_ob.close()\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can use this to read a file line-by-line by just iterating over the file using a `for` loop or a list compherension. The `file` object implements the [iterator](https://docs.python.org/3/glossary.html#term-iterator) protocol:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import pathlib as pl\n",
    "\n",
    "path = pl.Path(\"./tutorial/tests/data/lines.txt\")\n",
    "file_ob = open(path, 'r')\n",
    "for line in file_ob:\n",
    "    print(line)\n",
    "file_ob.close()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is the most *pythonic* way to read a file line-by-line instead of reading the full contents at once."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Quiz on file reading"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import tutorial.quiz.input_output as op\n",
    "op.ReadFiles()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercises on file reading"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%reload_ext tutorial.tests.testsuite"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. Modify the function `solution_read_file` to return the content of the file passed as the `input_file` argument. Return the content as a **list of strings**, one string per line.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%ipytest\n",
    "\n",
    "from pathlib import Path\n",
    "\n",
    "def solution_read_file(input_file: Path) -> list[str]: \n",
    "    \"\"\"Reads the contents of a file\n",
    "\n",
    "    Args:\n",
    "        input_file : The file to read\n",
    "\n",
    "    Returns:\n",
    "        - A list of strings, each representing a line in the file\n",
    "    \"\"\"\n",
    "    pass"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Writing to a file\n",
    "The process to write data to a file is very similar, the main difference being that:\n",
    "- We use `w` as a second argument of `open` to specify that we want to write to the file.If the file already exists, it will be erased before we write something else to it. If you want to append to the file, you should use `a` instead.\n",
    "- We use `write` to write a *string* to the file. Other types of object should be converted to string before being written.\n",
    "\n",
    "\n",
    "Let's see this in action by writing your name in a file called `me.txt` in [data](./data/)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import pathlib as pl\n",
    "\n",
    "path = pl.Path(\"./tutorial/tests/data/me.txt\")\n",
    "file_ob = open(path, \"w\")\n",
    "file_ob.write(\"Simone\")\n",
    "file_ob.close()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Congratulations! Your name is now written in stone. \n",
    "\n",
    "If we want to write the contents of an *iterable* to a file, we can use the `writelines` method:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import pathlib as pl\n",
    "\n",
    "path = pl.Path(\"./tutorial/tests/data/numbers.txt\")\n",
    "file_ob = open(path, \"w\")\n",
    "file_ob.writelines([str(i) + \"\\n\" for i in range(10)])\n",
    "file_ob.close()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice that for each line, we concatenate the `newline` `\\n` symbol to the string to be written to write the text to a new line.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Quiz on file writing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import tutorial.quiz.input_output as op\n",
    "op.WriteFiles()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercises on file writing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%reload_ext tutorial.tests.testsuite"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. Modify the function `solution_write_file` to write the sentence \"python tutorial 2023\" (**without quotes**) to the file `output_file`, which is available as a `Path` object as argument to the function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%ipytest\n",
    "\n",
    "from pathlib import Path\n",
    "\n",
    "def solution_write_file(output_file: Path) -> None:\n",
    "    \"\"\"Writes a string to a file\n",
    "\n",
    "    String to be written is \"python tutorial 2023\"\n",
    "\n",
    "    Args:\n",
    "        output_file : The file to write to\n",
    "\n",
    "    Returns:\n",
    "        - None (writes to file)\n",
    "    \"\"\"\n",
    "    pass"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "2. Modify the function `solution_read_write_file` to read the lines from the file `input_file` and write them in the form `line, length`, to the file `output_file`. Here `line` is the line of text in `input_file` **without the line ending**, `length` is **number of characters** in that line **without the line separator**.\n",
    "If `input_file` contains these lines:\n",
    "    ```\n",
    "    first\n",
    "    second\n",
    "    ```\n",
    "    we expect the output file to contain these lines:\n",
    "    ```\n",
    "    first, 5\n",
    "    second, 6\n",
    "    ```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%ipytest\n",
    "\n",
    "from pathlib import Path\n",
    "\n",
    "def solution_read_write_file(input_file: Path, output_file: Path) -> None:\n",
    "    \"\"\"Reads the contents of a file and writes it to another file adding a length count\n",
    "\n",
    "    The length of each line is added to the end of the line with a comma and a space separating the line and the length\n",
    "\n",
    "    Args:\n",
    "        input_file : The file to read\n",
    "        output_file : The file to write to\n",
    "\n",
    "    Returns:\n",
    "        - None (writes to file)\n",
    "    \"\"\"\n",
    "    pass"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Context managers\n",
    "As you can see, after *opening* and performing operations on a file, we always have to remember to *close* it.\n",
    "If we forget to close it, unexpected behaviour can happen.\n",
    "If the program crashes later on, for example, we might have a situation where the text is not written to the file.\n",
    "If you open many files and you don't close them, the Python interpreter can run out of memory.\n",
    "On some operating systems, the file contents are only updated after closing, etc ...\n",
    "\n",
    "This pattern is common when using  many different types of *resources*: files, connections, threads, servers, etc...\n",
    "You acquire access to the resource, do some work on it, and, finally, clean up after yourself by closing it again.\n",
    "Because of this, Python offers a construct called [*context manager*](https://docs.python.org/3/reference/datamodel.html#context-managers) which implements exactly this beahvior:\n",
    "- Get access to a resource, e.g. open a file.\n",
    "- Do some work with the resource.\n",
    "- Release this resource, e.g close the file or the connection.\n",
    "\n",
    "In the case of files, we can replace the open-read-close or open-write-close sequence  with a context manager.\n",
    "Context managers are used in a `with` statement:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import pathlib as pl\n",
    "\n",
    "hello_file = pl.Path(\"./tutorial/tests/data/hello.txt\")\n",
    "with open(hello_file, \"r\") as file_ob: \n",
    "    contents = file_ob.readlines()\n",
    "    print(contents)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`with open(path) as name` opens the file in `path` and assigns it to the `name` file object.\n",
    "This object is only valid in the *scope* of the context manager, that is the indented block of code that follows the `:`.\n",
    "Once the Python interpreter leaves the context manager, `file_ob.close()` is automatically called, ensuring the file is properly closed no matter what happens. \n",
    "This means that you don't have to manually call `close` after `open` anymore, avoiding many potential bugs.\n",
    "\n",
    "This pattern can be extended to any other resource that should be managed in a similar way, for example database connections. \n",
    "Any object that implements `__enter__` and `__exit__` can be used with the context manager syntax.\n",
    "\n",
    "If you want to learn how to implement context managers for other types of objects, please refer to the `contextlib` [documentation](https://docs.python.org/3/library/contextlib.html) in the Python standard library."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Quiz on context managers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import tutorial.quiz.input_output as op\n",
    "op.ContextManagers()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Binary I/O\n",
    "\n",
    "Another aspect of file I/O is accessing files in [*binary mode*](https://docs.python.org/3/library/io.html#binary-i-o); that means that instead of writing and reading *text*, we manipulate `bytes` in order to represent non-textual data. This is useful for interacting with measurement data and other non-textual information like images, machine learning model parameters and other complex strcuture, although in most cases you won't need the low-level control of binary I/O and will use libraries instead. \n",
    "\n",
    "To look at an example, let's write a sequence of `int` to `output.dat`  as a  sequence of bytes. Because one byte corresponds to 8 bits, using one byte per integer means we can unambigously store `2^8=256` values.\n",
    "\n",
    "<div class=\"alert alert-block alert-warning\">\n",
    "    <h4><b>Note</b></h4>\n",
    "    <code>.dat</code> is a typical generic extension to indicate that the file contains some sort of data. Filename extensions does not have any binding meaning by themselves, they are simply a convention for users to quickly see what content to expect.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import pathlib as pl\n",
    "\n",
    "bin_file = pl.Path(\"./tutorial/tests/data/output.txt\")\n",
    "with open(bin_file, \"wb\") as out_file:\n",
    "    bs = b\"\".join([i.to_bytes(1, 'little') for i in  range(10)])\n",
    "    print(bs)\n",
    "    out_file.write(bs)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that we used the mode `wb` for `write, binary`. The `write` function expects a [`bytes`](https://docs.python.org/3/library/stdtypes.html#bytes) object. Because of the historical connection between bytes and strings, we can produce a bytes object by prepending `b` to a string literal. Therefore, we generate an array of bytes using [`to_bytes`](https://docs.python.org/3/library/stdtypes.html#int.to_bytes) and combine them with the [`join`](https://docs.python.org/3/library/stdtypes.html#str.join) method on the empty byte literal `b\"\"`.\n",
    "\n",
    "Now that we wrote out our sequence, we can try to read it back from the file:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import pathlib as pl\n",
    "\n",
    "bin_file = pl.Path(\"./tutorial/tests/data/output.txt\")\n",
    "with open(bin_file, \"rb\") as in_file:\n",
    "    data = in_file.read()\n",
    "    seq = [b for b in data]\n",
    "    print(seq)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And, surprise! We obtain our original sequence.\n",
    "Note that because of a quirk of Python, the `read` method returns a `bytes` object, but when we try to access a single element (as we would do with a string), the entry is already an `int` which corresponds to the Unicode codepoint.\n",
    "\n",
    "\n",
    "If instead we want to store our numbers using two bytes per digit, we do:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import pathlib as pl\n",
    "import struct\n",
    "\n",
    "bin_file = pl.Path(\"./tutorial/tests/data/output1.txt\")\n",
    "with open(bin_file, \"wb\") as out_file:\n",
    "    bs = b\"\".join([struct.pack(\">H\", i) for i in range(1024)])\n",
    "    out_file.write(bs)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We do this by using the [`struct`](https://docs.python.org/3/library/struct.html) module of the Python standard library, which offers methods to represent built-in data as different bytes formats.\n",
    "\n",
    "The first argument to `struct.pack` is the format the data should be interpreted into, \">\" means little-endian, or putting the byte representing the smallest digit of each number first. \"H\" means an *unsigned short*, which corresponds to two bytes.\n",
    "\n",
    "If we want to read the data, we can do:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import pathlib as pl\n",
    "import struct\n",
    "\n",
    "bin_file = pl.Path(\"./tutorial/tests/data/output1.txt\")\n",
    "with open(bin_file, \"rb\") as in_file:\n",
    "    data = in_file.read()\n",
    "    seq = [i for i, *rest in struct.iter_unpack(\">H\", data)]\n",
    "    print(seq)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Bytes and strings\n",
    "Earlier, we mentioned the connection between bytes and strings in Python. To learn more about this, let's briefly review the Unicode UTF-8 standard.\n",
    "\n",
    "The Unicode UTF-8  standard is a system to represent texts; in Unicode  every character is assigned a number (the *codepoint*) that gives the location of this symbol in the table of all symbols. Particularly, the Unicode UTF-8 standard stores each character as 1 to four bytes (each byte is a group of 8 bits). \n",
    "\n",
    "Let's see this more clearly with an example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "for i in range(0, 6):\n",
    "    bt = i.to_bytes(4, 'little')\n",
    "    print(f\"bytes: {[hex(b) for b in bt]}, integer: {i}, string: {chr(i)}\")\n",
    "for i in range(60, 74):\n",
    "    bt = i.to_bytes(4, 'little')\n",
    "    print(f\"bytes: {[hex(b) for b in bt]}, integer: {i}, string: {chr(i)}\")\n",
    "for i in range(8**3, 8**3 + 3):\n",
    "    bt = i.to_bytes(4, 'little')\n",
    "    print(f\"bytes: {[hex(b) for b in bt]}, integer: {i}, string: {chr(i)}\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `chr` function takes an integer and returns the corresponding Unicode character."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Converting bytes to text \n",
    "\n",
    "If we receive a `bytes` object and we want to transform it to text, we can use the `int.from_bytes` method and the `chr` function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "message = \"Ciao\"\n",
    "message_secret = bytes(message, \"utf-8\")\n",
    "[print(f\"The `uft8` codepoint  is = {enc}, the bytes representation = {enc.to_bytes(4, 'little')}, the representation is {chr(enc)}\") for plain, enc in zip(message, message_secret)]"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Reading/Writing CSV files\n",
    "If you ever worked with tabular data, you surely encountered CSV (comma separated values) files.\n",
    "These files are used to store table in text format row by row: each row is separated by a new line and the columns inside a row are separated by commas `,` or by semicolumns `;`.\n",
    "The first line in the file usually contains the header giving the names of the columns:\n",
    "\n",
    "```\n",
    "first_column,second_column\n",
    "1,2\n",
    "2,3\n",
    "```\n",
    "\n",
    "As CSV is very common to exchange tabular data such as statistics and time series, the Python standard library offers facilities to read and write CSV files through the [csv](https://docs.python.org/3/library/csv.html) module.\n",
    "Despite this, today most people prefer using [pandas](https://pandas.pydata.org/) or [polars](https://www.pola.rs/) to manipulate tabular data because they offer more convenience and faster handling of large datasets.\n",
    "These packages are outside of the scope of this tutorial and will not be covered here.\n",
    "\n",
    "\n",
    "Let's see how to read csv files using `csv` with an example by reading [example.csv](./data/example.csv):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import csv\n",
    "import pathlib as pl\n",
    "\n",
    "csv_file = pl.Path(\"./tutorial/tests/data/example.csv\")\n",
    "with open(csv_file) as input_file:\n",
    "    reader = csv.reader(input_file)\n",
    "    #Get the header\n",
    "    header = next(reader)\n",
    "    #Iterate over lines\n",
    "    for line in reader:\n",
    "        print(line)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `next` function called on any iterable returns the next value and advances the iterable counter. In the case of `csv.reader`, we use this to read the header.\n",
    "\n",
    "The `csv` module does not interpet the data; as a default everything is read as `str`.\n",
    "\n",
    "Similarly, you can use the CSV module to write a CSV using the [`writer`](https://docs.python.org/3/library/csv.html#csv.writer) class. The `writer` object has `writerow` method which takes an `iterable` of values to write as the current row"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import csv\n",
    "import pathlib as pl\n",
    "\n",
    "csv_file = pl.Path(\"./tutorial/tests/data/example1.csv\")\n",
    "\n",
    "with open(csv_file, \"w\") as output_file:\n",
    "    writer = csv.writer(output_file)\n",
    "    \n",
    "    # Get the header\n",
    "    writer.writerow([\"this\", \"is\", \"data\"])\n",
    "    \n",
    "    # Iterate over lines\n",
    "    [writer.writerow((i, i+1, i+2)) for i in range(10)]\n",
    "        "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Quiz on CSV"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import tutorial.quiz.input_output as op\n",
    "op.CSV()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Exercises"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "%reload_ext tutorial.tests.testsuite"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Exercise 1: CSV to dictionary 🌶️"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "Write a function that reads a CSV file and returns a dictionary.\n",
    "\n",
    "- The dictionary keys are in the first **column**\n",
    "- The dictionary values are in the second **column**\n",
    "\n",
    "For example, if the file contains the following lines:\n",
    "\n",
    "```\n",
    "key,value\n",
    "a,2\n",
    "b,4\n",
    "c,6\n",
    "```\n",
    "\n",
    "this function should return the dictionary:\n",
    "\n",
    "```python\n",
    "{\"a\":\"2\", \"b\": \"4\", \"c\": \"6\"}\n",
    "```\n",
    "\n",
    "<div class=\"alert alert-block alert-info\">\n",
    "    <h4><b>Hint</b></h4>\n",
    "    <ul>\n",
    "        <li>\n",
    "            The <code>csv.reader</code> reads each column entry as a string by default.\n",
    "        </li>\n",
    "        <li>\n",
    "            Calling <code>next(csv_reader)</code> immediately after you have created the <code>csv.reader</code> object returns the header to the file and advances the reader to the first data row.\n",
    "        </li>\n",
    "        <li>\n",
    "            The parameter <code>input_file</code> of the function skeleton <code>solution_exercise1</code> is the path of the input file. Read the file from that location.\n",
    "        </li>\n",
    "    </ul>\n",
    "</div>\n",
    "\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "%%ipytest\n",
    "\n",
    "import csv\n",
    "import pathlib as pl\n",
    "\n",
    "def solution_exercise1(input_file: pl.Path) -> dict[str, list[str]]:\n",
    "    \"\"\"Reads a CSV file and returns a dictionary with each row representing a key, value pair\n",
    "\n",
    "    The key is the first element of the row and the value is the second value of the row\n",
    "\n",
    "    Args:\n",
    "        input_file : The csv file to read\n",
    "\n",
    "    Returns:\n",
    "        - A dictionary with each row represented as a key, value pair\n",
    "    \"\"\"\n",
    "    pass"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Exercise 2: Counting words 🌶️"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "Write a function  to read all the lines from `input_file` and count the number of words in the file. The solution should be a single number.\n",
    "\n",
    "For example, for the file\n",
    "```\n",
    "this \n",
    "file \n",
    "has \n",
    "five\n",
    "lines\n",
    "```\n",
    "the result should be `5`. \n",
    "\n",
    "<div class=\"alert alert-block alert-info\">\n",
    "    <h4><b>Hint</b></h4>\n",
    "    <ul>\n",
    "        <li>\n",
    "            The file is available as the parameter <code>input_file</code> of <code>solution_exercise2</code> function\n",
    "        </li>\n",
    "        <li>\n",
    "           A word consists of <b>printable</b> characters without whitespaces, line breaks etc. Have a look at the basic_datatypes notebook if you forgot how to split a text into it's words.\n",
    "        </li>\n",
    "    </ul>\n",
    "</div>\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "%%ipytest\n",
    "\n",
    "import pathlib as pl\n",
    "\n",
    "def solution_exercise2(input_file: pl.Path) -> int:\n",
    "    \"\"\"Reads a file and returns the number of words in the file\n",
    "\n",
    "    Args:\n",
    "        input_file : The file to read\n",
    "\n",
    "    Returns:\n",
    "        - The number of words in the file\n",
    "    \"\"\"\n",
    "    pass"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Exercise 3: Letter statistics 🌶️🌶️"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "Write a function that reads all the lines from `lines.txt` and counts the occurences of every letter present in all the words.\n",
    "\n",
    "The result should be a dictionary, where each key-value pair is like `{letter: count}`. For example, `{a: 5}` means that the letter `a` appeared five times in this file.\n",
    "\n",
    "<div class=\"alert alert-block alert-info\">\n",
    "    <h4><b>Hints</b></h4>\n",
    "    <ul>\n",
    "        <li>\n",
    "          The file is available as the <code>input_file</code> parameter of <code>solution_exercise3</code> function.\n",
    "        </li>\n",
    "        <li>\n",
    "            You may assume all letters are lower case and contained in the collection <code>string.ascii_lowercase</code>.\n",
    "        </li>\n",
    "        <li>\n",
    "            To verify if a character is a letter, you can use the <code>isalpha()</code> string method. Example: \n",
    "    <code>'a'.isalpha()</code> will return <code>True</code>.\n",
    "        </li>\n",
    "        <li>\n",
    "            As a sidenote, historically the order of a dictionary behaved as randomly as the order of a set. Comparing dictionaries ignores the order of the entries. You may also ignore any ordering on the dictionary for this exercise. For some applications it may be helpful to have the dictionary sorted such that when extracting key value pairs you get a nice list. A dictionary is always sorted by order of insertion.\n",
    "        </li>\n",
    "    </ul>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%ipytest input_output\n",
    "\n",
    "import pathlib as pl\n",
    "import string\n",
    "\n",
    "def solution_exercise3(input_file: pl.Path) -> dict[str, int]:\n",
    "    \"\"\"Reads a file and returns a dictionary with a count of each letter\n",
    "\n",
    "    We consider only the letters a-z\n",
    "\n",
    "    Args:\n",
    "        input_file : The file to read\n",
    "\n",
    "    Returns:\n",
    "        - A dictionary with a count of each letter\n",
    "    \"\"\"\n",
    "    pass"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Exercise 4: Translating words 🌶️🌶️"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "Write a function which takes the words from the file `english.txt` and translates them to Italian using the dictionary file `dict.csv`. The output should be a **list of tuples** with the pair `italian, english` if the word is found and nothing otherwise.\n",
    "\n",
    "For example, given the `english.txt` file:\n",
    "\n",
    "```\n",
    "bread\n",
    "cat\n",
    "```\n",
    "\n",
    "and the `dict.csv` file:\n",
    "\n",
    "```\n",
    "120, pane, bread\n",
    "121, sole, sun\n",
    "```\n",
    "\n",
    "the result should be:\n",
    "\n",
    "`[(bread, pane), ]`\n",
    "\n",
    "\n",
    "<div class=\"alert alert-block alert-info\">\n",
    "    <h4><b>Hints</b></h4>\n",
    "    <ul>\n",
    "        <li>\n",
    "            Try to avoid loading the dictionary more than once. <b>\"Dictionary file\"</b> should suggest you the correct Python data structure to use to store the translations.\n",
    "        </li>\n",
    "        <li>\n",
    "            The path to the input file <code>english.txt</code> is available as the argument <code>english</code> of the function <code>solution_exercise4</code>, the file <code>dict.csv</code> as the argument <code>dictionary</code>.\n",
    "        </li>\n",
    "    </ul>\n",
    "<div>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%ipytest\n",
    "\n",
    "import pathlib as pl\n",
    "\n",
    "def solution_exercise4(english: pl.Path, dictionary: pl.Path) -> list[(str, str)]:\n",
    "    \"\"\"Reads two files and returns a outputs a list of translation tuples\n",
    "\n",
    "    Each word in the english file is translated using the dictionary file to an italian word\n",
    "    and a tuple (english, italian) is added to the list (or not if the word is not in the dictionary)\n",
    "\n",
    "    Args:\n",
    "        english : The file with the english words\n",
    "        dictionary : The file with the dictionary\n",
    "\n",
    "    Returns:\n",
    "        - A list of tuples with the english / italian words\n",
    "    \"\"\"\n",
    "\n",
    "    pass"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Exercise 5: Binary format 🌶️🌶️🌶️"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "\n",
    "The file `super_secret.dat` contains a secret message. We know that the message is stored in binary format as a sequence of bytes. The message starts with the byte sequence `b'\\xff\\xee\\xdd\\xcc\\xbb\\xaa'` and finishes with `b'\\xaa\\xbb\\xcc\\xdd\\xee\\xff'`. \n",
    "Write a function that reads the file and returns **only** the secret message as a string.\n",
    "\n",
    "\n",
    "<div class=\"alert alert-block alert-info\">\n",
    "    <h4><b>Hints</b></h4>\n",
    "    <ul>\n",
    "        <li>\n",
    "        The path to the input file is available as the argument <code>secret_file</code> of the function <code>solution_exercise5</code>\n",
    "        </li>\n",
    "        <li>\n",
    "            Every <code>bytes</code> object has the <code>decode()</code> method. For example, <code>b\"\\xbb\\xcc\\xdd\\xee\\xff\".decode()</code>\n",
    "        </li>\n",
    "    </ul>\n",
    "\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%ipytest\n",
    "\n",
    "import pathlib as pl\n",
    "\n",
    "def solution_exercise5(secret_file: pl.Path) -> str:\n",
    "    \"\"\"Reads a file and returns the secret message\n",
    "\n",
    "    The file contains a secret message encoded in binary\n",
    "    start and end of the message are marked by the bytesequences \n",
    "    b'\\xff\\xee\\xdd\\xcc\\xbb\\xaa' and b'\\xaa\\xbb\\xcc\\xdd\\xee\\xff'\n",
    "    \n",
    "    Args:\n",
    "        secret_file : The file with the secret message\n",
    "\n",
    "    Returns:\n",
    "        - The secret message\n",
    "    \"\"\"\n",
    "    pass"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}