doc: add settings file for TOML

Added deprecation warning to `doc/settings.md`.
Added `doc/toml_settings.md` to add instructions for
TOML settings file.
Also, renamed `lab` to `runtime` in the documentation.

Signed-off-by: Jeny Sadadia <[email protected]>

doc/settings.md

@@ -1,11 +1,15 @@
 ---
-title: "User settings"
+title: "[Deprecated] User settings"
 date: 2021-08-05
 draft: false
 description: "User-defined local settings"
 weight: 2
 ---
 
+> **Warning**
+> This document describes user settings file format `.conf` which will soon be deprecated. Please use the latest settings file format i.e. `.toml`. More information on it can be found [here](../toml_settings).
+
+
 The user settings file is intended to be created by end-users with options
 specific to their local setup.  It can also be used by automated CI systems, in
 particular to hide secret API tokens.  The format is similar to

doc/toml_settings.md

@@ -0,0 +1,167 @@
+---
+title: "User settings with TOML"
+date: 2023-08-10
+draft: false
+description: "User-defined local settings"
+weight: 2
+---
+
+The user settings file is intended to be created by end-users with options
+specific to their local setup.  It can also be used by automated CI systems, in
+particular to hide secret API tokens.
+
+The default name for the file is `kernelci.toml` and it has several standard
+potential locations:
+
+* `kernelci.toml` in the current working directory,
+* `~/.config/kernelci/kernelci.toml` for per-user settings,
+* `/etc/kernelci/kernelci.toml` for system-wide settings.
+
+Unless directly specified on the command line with the `--settings` argument,
+each of the default locations above will be visited in that order until one is
+found to exist.  Only one settings file will be loaded, the first one found.
+
+Settings are defined in separate sections within the file, as described in
+detail below.  Regardless of what is set in the file, each option can always be
+overridden on the command line.  So if you set `kdir = "linux"` in the settings
+file as the default path to the Linux kernel source directory, you can always
+override it with `--kdir=/some/other/path` on the command line (say, if you
+want to occasionally use a different source tree).  This applies to all the
+options as they are derived from the list of arguments supported by each
+command.
+
+There is a convention to convert the command line argument names to settings
+names, with the `--` being dropped and single dashes `-` replaced with
+underscores `_`.  This is identical to what the Python 3
+[`argparse`](https://docs.python.org/3/howto/argparse.html) module does to
+convert command line argument names to Python object attributes.  A few
+examples:
+
+* `--kdir` becomes `kdir`
+* `--runtime-config` becomes `runtime_config`
+
+To get started quickly, see the [kernelci.toml.sample](../kernelci.toml.sample)
+file.  You can copy it as `kernelci.toml` into a suitable location as described
+above and edit it to suit your particular needs.
+
+## Command sections
+
+Each command line tool will be looking for a section with a matching name in
+the settings file, such as `[kci_build]`, `[kci_test]` or `[kci_data]`.  These
+are not required to be in the file, but can be used to provide default values
+for command line arguments.  For example:
+
+```ini
+[kci_build]
+
+mirror = "linux-mirror.git"
+kdir = "linux"
+build_env = "gcc-10"
+j = 3
+```
+
+With the values set above, instead of running this:
+```
+kci_build update_mirror --build-config=mainline --mirror=linux-mirror.git
+```
+you can now omit the `--mirror` argument:
+```
+kci_build update_mirror --build-config=mainline
+```
+
+## Component sections
+
+Other sections are specific to a component, such as a runtime or a database
+backend.  This is to allow different values to be set for a same option
+depending on the component being used, and also to allow these values to be
+used by all the command line tools.
+
+The component names are derived from entries defined in the YAML configuration
+files, such as `runtime-configs.yaml` or `db-configs.yaml`.  However, the
+settings file can be used to keep values that don't belong in the YAML
+configuration such as secret API tokens or arbitrary user-specific choices.
+
+Each component section name in the settings file will be composed of two parts,
+separated by a dot `.` character:
+
+* a prefix with the type of component such as `runtime` or `db`
+* the name of the component
+
+For example, if you define a runtime called `my-lava-lab` in
+`runtime-configs.yaml`, you can create a section called
+`[runtime.my-lava-lab]` in the settings file.  You can refer to it with `--runtime-config=my-lava-lab` on the command line, or even set `runtime_config = "my-lava-lab"` in the `[DEFAULT]` section to always pick this one by default.
+
+Here's an example:
+
+```ini
+[DEFAULT]
+
+runtime_config = "my-lava-lab"
+
+[runtime.my-lava-lab]
+
+user = "user-name"
+runtime_token = "1234-5678"
+```
+
+> **Note**:
+> The component name needs to be surrounded by double-quotes `""` if it contains dot `"."` in the name e.g. [db."staging.kernelci.org"].
+
+
+## `[DEFAULT]` section
+
+As per the INI file standard specifications, the `[DEFAULT]` section is where
+catch-all default values can be set regardless of the settings section being
+looked up.  The example above shows how this can be used, for a default
+`runtime_config` value across all the command line tools.  But values in the
+`[DEFAULT]` section also apply to component sections.  Say, if you have set up several runtimes but they all need the same user name to access their API, you can set `user` in the `[DEFAULT]` section and not have to repeat it in each `[runtime.lab-name]` section.
+
+### A more complete example
+
+It's worth noting that for a same command, some settings values will be found
+in a tool section such as `[kci_test]` while others will be found in a
+component section such as `[runtime.lab-name]`.  Others may be found in the
+`[DEFAULT]` section, and finally any option can always be provided or
+overridden on the command line.
+
+For example, if you have these values in your `kernelci.toml` settings file:
+
+```ini
+
+[DEFAULT]
+
+db_config = "localhost"
+
+[db.localhost]
+
+db_token = "XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX"
+callback_id = "kernelci-callback-local"
+callback_url = "http://localhost:5001"
+
+[kci_build]
+
+kdir = "linux"
+build_env = "gcc-10"
+```
+
+you can then run these commands:
+
+```
+kci_build build_kernel --arch=arm64 --defconfig=defconfig
+kci_build install_kernel --build-config=mainline
+```
+
+The `kci_build build_kernel` command normally requires `--kdir` and
+`--build-env`, they are both defined in the `[kci_build]` section so they can
+be dropped on the command line.  The `--arch` and `--defconfig` options are
+still passed on the command line here, because they're more likely to change
+between kernel builds.  If you build the same architecture and defconfig most
+of the time, you can still set some default values for them under the
+`[kci_build]` section and drop them from the command line too until you need to build a different one.  Say if you mostly build `arm64`, you can set it in the settings file, but if you need to build `x86_64` one day you can pass
+`--arch=x86_64` on the command line without having to edit the settings file.
+
+Then the `kci_build install_kernel` command normally requires `--kdir`,
+`--db-token` and `--db-config`.  These can all be found in the settings file,
+so they don't need to be provided on the command line: `db_config` is in the
+`[DEFAULT]` section, `db_token` in the `[db:localhost]` section and `kdir` in
+the `[kci_build]` section.