RFC - Clerk TOML configuration & Expected usage

[vincent-botbol]

RFC - Clerk TOML configuration & Expected usage

Following discussions with Denis regarding a proposal for a project's
TOML configuration file, I drafted this proposal (based on the catala
examples repo).

Informal Spec.

A Catala project should have a single Clerk.toml file at the root of
the directory.

The configuration file may contain the project's global options, e.g., :

[project]
build_dir = "_build"
include_dirs = [
  "smic",
  "prestations_familiales",
  "allocations_familiales"
]

A project is then divided into several parts which may describe two
kind of build targets: modules and targets.

Modules

A configuration file might contain several Catala's modules which are
meant to be used in other modules and to be tested using the existing
clerk's command: clerk test <module_name>.

TOML modules could be also used as entrypoints for generating scope
tests: every immediate scope exposed by this module could be generated
as a test in the vscode extension.

[[module]]
name = "Allocations_familiales"
module_uses = [
  "France",
  "Bmaf",
  "Smic",
  "Prestations_familiales"
]
includes = [
  "allocations_familiales/prologue.catala_fr",
  "allocations_familiales/autres_codes.catala_fr",
  "allocations_familiales/securite_sociale_L.catala_fr",
  "allocations_familiales/securite_sociale_R.catala_fr",
  "allocations_familiales/securite_sociale_D.catala_fr",
  "allocations_familiales/decrets_divers.catala_fr",
  "allocations_familiales/epilogue.catala_fr",
]

[[module]]
name = "Prestations_familiales"
module_uses = [
  "France",
  "Smic",
]
includes = [
  "prestations_familiales/prologue.catala_fr",
  "prestations_familiales/sécurité_sociale_L.catala_fr",
  "prestations_familiales/securite_sociale_R.catala_fr",
]

Question: modules are already declared in catala files using the > Module <name> directive. Should we force every module to have its own
entry in the TOML or should we leave these implicit declarations and
let clerk work its magic?

Targets

Targets are a project's building endgoal. These are used to combine
modules, create an abstraction barrier and provide Catala ready-to-use
components.

For example, one might want to create a specific component to
distribute it to a mainframe that computes the "allocation
familiales". Hence, one would declare:

[[target]]
name = "mainframe"
entrypoints = [ "Allocations_familiales" ]
backends = [ "c", "ocaml" ]
backends_options = [
  { "c" = [ "--check_invariants", "--optimize" ]},
  { "ocaml" = [ "--closure-conversion" ]}
]
docs = [ "latex", "html" ]
docs_options = [
    { "latex" = [ "--wrap" ] }
]

It would yield transpiled versions of the module in OCaml and C that
exposes in its API the immediate scopes (through .mli or .h files) and
the necessary dependencies (e.g., France.Collectivité which is part of
a dependency but is an exposed scope input). It would also be
responsible of building the doc.

Regarding the TOML structure, it seems to be not very
idiomatic. Usually, one declares a dictionnary with very little depth.
I'm not quite sure on how to describe it in a more usual way. Thoughts?

Usage

With such rules present in the TOML configuration files, users would
be able to invoke clerk on these aliases. E.g.,

# Build the allocation familiales module
$ clerk build Allocation_familiale
# Build the mainframe target
$ clerk build mainframe
# No argument builds everything (i.e., all modules + all targets)
$ clerk build
# Tests also accepts TOML aliases
$ clerk test Allocation_familiale

[AltGr]

That looks pretty good to me!
For the targets maybe something more atomic would be more convenient: repeating a [[target]] section for each of C, OCaml, LaTeX, HTML.
For either C or OCaml, we might want to specify more than the catala flags: backend compiler flags, separate backend compiler linking flags, compiler commands, etc.
Another option would be to have these flags settable globally instead of by-target ?

With the approach above, it would be convenient to define meta-targets that regroup sub-targets

[rprimet]

Looks good! I don't have much feedback owing to my limited knowledge of the toolchain, the only thing that struck me is

Question: modules are already declared in catala files using the > Module directive. Should we force every module to have its own
entry in the TOML or should we leave these implicit declarations and
let clerk work its magic?

I like the python convention of packages and subpackages being defined by the directory name that the files are in and modules by their file names -- that's one thing less to think about and one thing less to rename when refactoring?

[denismerigoux]

Decision of the meeting of 2024-09-24 : the [[target]] section can only have one unique backend. If you want to target multiple backends, you should have different [[target]] sections. In the future we can add [[target]] aliases special sections that trigger multiple single [[target]] sections.