diff --git a/source/index.rst b/source/index.rst index af7bb20..aaa5dd7 100644 --- a/source/index.rst +++ b/source/index.rst @@ -41,6 +41,9 @@ inclusion of 3rd-party elements and complex dependency structures. getting_started advanced updating + + reference + faq .. toctree:: diff --git a/source/reference.rst b/source/reference.rst new file mode 100644 index 0000000..38ac793 --- /dev/null +++ b/source/reference.rst @@ -0,0 +1,8 @@ +Reference +========= + +.. toctree:: + :maxdepth: 1 + + reference/recipes + reference/language diff --git a/source/reference/recipes.rst b/source/reference/recipes.rst new file mode 100644 index 0000000..aca8a28 --- /dev/null +++ b/source/reference/recipes.rst @@ -0,0 +1,228 @@ +.. _recipes: + +========= + Recipes +========= + +.. + TODO turn this into an overview page with four children: + recipes-basics, recipes-project recipes-distribution + recipes-template + +.. toctree:: + :hidden: + + recipes/project + recipes/distribution + recipes/template + +Kinds of recipes: + +:ref:`Project Recipes ` + + Descriptions of :term:`projects ` + +:ref:`Distribution Recipes ` + + Descriptions of :term:`distributions ` which are + groups of :term:`project` versions that can be built and used + together. + +:ref:`Template Recipes ` + + Descriptions of common :term:`project` kinds including how to build + and install the respective kinds of :term:`projects ` + depending on the overall development mode. + +On the syntactic level, all descriptions are currently based on +`YAML`_. + +Variables and the Substitution Sub-Language +=========================================== + +Objects such as :term:`projects `, :term:`distributions +` are basically named entities with a dictionary of +variable names and their values. In `JSON`_, this translates into + +.. parsed-literal:: + + … + + variables: + :samp:`"{NAME₁}"`: :samp:`{VALUE₁}` + :samp:`"{NAME₂}"`: :samp:`{VALUE₂}` + … + +The value part of a variable definition is either a string or a list +of values. A small substitution sub-language, syntactically similar to +"parameter expansion" in UNIX shells, can be used to refer to values +of other variables. To this end we distinguish ``${}`` and ``@{}`` +substitution, which act differently on strings and lists. + +Some examples of variable definitions + +.. code-block:: yaml + + list.a: [ "abc", "def" ] + list.b: [ "123", "456" ] + list.c: [ "@{list.a}", "@{list.b}", "uvw", "${text}" ] + text: "xyz" + + dashed: "${list.c}-" + product: "(${list.a} * ${list.b})" + +The basic syntax is as follows: + +* Scalar substitution + + :samp:`$\{{NAME}\}` + + Expands to the value of the variable named :samp:`{NAME}`. + + Lists stay lists. However, ultimately a list is flattened into a + string by concatenating all quoted items with a space used as + delimiter. + + :samp:`$\{{NAME}|\}` or :samp:`$\{{NAME}|{DEFAULT}\}` + + Expands to the value of the variable named :samp:`{NAME}` if there + is one and :samp:`{DEFAULT}` otherwise. + + :samp:`$\{next-value|{DEFAULT}\}` + + Expands to the value of the variable being defined in the parent + scope if there is such a value and :samp:`{DEFAULT}` otherwise. As + before, :samp:`{DEFAULT}` can be empty or omitted entirely. + +* List substitution + + Concatenates the elements of a list into a single string if there is + surrounding text. Otherwise, i.e. if there is no surrounding text, + it evaluates to the first element of the list. Obviously, this only + applies to list variables. + + :samp:`@\{{NAME}\}` + + In string context, expands to the first item of list variable + :samp:`{NAME}`. + + In list context, expands to the list of items of list variable + :samp:`{NAME}`, which allows to extend lists this: + + .. code-block:: json + + [ "@{list.a}", "@{list.b|[]}", "new item" ] + + + :samp:`{TEXT₁}@{{NAME}}{TEXT₂}` + + Expands to the string concatenated from :samp:`{TEXT₁}`, elements + of list variable :samp:`{NAME}` and :samp:`{TEXT₂}`. + + :samp:`@\{{NAME}|[]\}` + + Expands to the value of the variable named :samp:`{NAME}` if there + is one and the empty list of values (as indicated by ``[]``) + otherwise. + + ``@{next-value|[]}`` + + Expands to the value of the variable being defined in the parent + scope if there is such a value and the empty list (``[]``) + otherwise. As before, ``[]`` can be omitted. + +* Products + + If the evaluation of a variable results in a list, this list is + "multiplied" with the rest of the value string. + +* Examples + + Assuming the following variables are defined (repeated from above): + + .. code-block:: yaml + + list.a: [ "abc", "def" ] + list.b: [ "123", "456" ] + list.c: [ "@{list.a}", "@{list.b}", "uvw", "${text}" ] + text: "xyz" + dashed: "${list.c}-" + product: "(${list.a} * ${list.b})" + + , the following results would be produced:: + + "${list.a}" => [ "abc", "def" ] + "@{list.a}" => "abc" + "@{list.a} foo" => "abcdef foo" + + "@{list.a} @{list.b}" => "abcdef 123456" + "@{list.c|[]} " => "abcdef123456uvwxyz" + + "${dashed}" => [ "abc-", "def-", "123-", "456-", "uvw-", "xyz-" ] + "@{dashed} " => "abc-def-123-456-uvw-xyz-" + + "${product}" => [ "(abc * 123)", "(abc * 456)", "(def * 123)", "(def * 456)" ] + "@{product} " => "(abc * 123)(abc * 456)(def * 123)(def * 456)" + +Like in UNIX shells, substitutions are performed recursively, that is +:samp:`${{${NAME}}}` expands to the value of the variable named by the +value of the variable :samp:`{NAME}`. The same is true for +:samp:`{DEFAULT}`. + +Variable Scopes +--------------- + +TODO Different objects in the description language like projects or +distributions have separate variable sections. However, all these +sections are effectively considered when generating a build job and +thereby resolving to the values of required variables. This resolution +process is determined by the parent-child relation of the different +objects. This relation is as follow (``->`` indicates that the left +side is a parent of the right side):: + + DISTRIBUTION -> PROJECT -> VERSION -> JOB[including template] + +TODO When a variable needs to be resolved the resolution process iterates +from the most specific object (e.g. a ``JOB``) to the most general +object until the first definition is found. This definition sets the +variable value. In case this definition contains a ``next-value`` (see +above), the iteration continues on the next level to resolve the +``next-value`` part. + +Access and Credentials +====================== + +:ref:`Project Descriptions ` and :ref:`Distribution +Descriptions ` can specify whether the described +:term:`project` or :term:`distribution` is publicly accessible. For a +publicly accessible :term:`project` or :term:`distribution`, the +entire associated source code can be obtained without any +authentication. + +Conversely, if project source code has to be obtained from a +repository that requires authentication, the necessary authentication +can be configured. + +For access and credentials specification, the following variables are +important: + +``variables`` » ``access`` + + Specifies access restrictions of the :term:`project` or + :term:`distribution`. If present, has to have one of the values + ``private`` and ``public``. Omitting the variable is equivalent to + ``public``. + +``variables`` » ``scm.credentials`` + + The value of this variable names an entry in Jenkins' global + credentials store that should be associated to the repository of + this project in generated Jenkins jobs. + +The following rules apply: + +* :term:`Projects ` with ``scm.credentials`` entries must + specify ``private`` ``access``. + +* :term:`Distributions ` with one or more ``private`` + projects must specify ``private`` ``access``. diff --git a/source/reference/recipes/distribution.rst b/source/reference/recipes/distribution.rst new file mode 100644 index 0000000..32886f1 --- /dev/null +++ b/source/reference/recipes/distribution.rst @@ -0,0 +1,93 @@ +.. _recipes-distribution: + +====================== + Distribution Recipes +====================== + +TODO introduction see project-recipe + +A basic :term:`distribution` description, which in this case would be +stored in a file named :file:`toolkit.distribution`, looks like this: + +.. code-block:: yaml + + variables: + … + + include: + … + + versions: + - project₁@version₁ + - project₂@version₂ + … + +The following fields are most important: + +:samp:`{FILENAME}` + + This is not an actual field but the name of the :term:`recipe file + ` with its extension removed (e.g. ``rsb`` for a + :term:`recipe file ` named :file:`rsb.distribution`). + + TODO + +``variables`` + + TODO + +``include`` + + TODO + +``versions`` + + A list of :term:`project` names and versions with elements of the + form + + .. parsed-literal:: + + :samp:`{NAME}@{VERSION}` + :samp:`name: "{NAME}" versions: - {VERSION₁} - {VERSION₂}` + + which should be part the :term:`distribution`. Note that multiple + versions of a :term:`project` can be specified in one entry but + multiple entry for the same :term:`project` are not allowed. + +``versions`` » ``name`` + + TODO + +``versions`` » ``match`` + + TODO + + Example: + + .. code-block:: yaml + + versions: + - pattern: "^(([^-]+)-stable)$" + variables: + name: "${match:1}" + numeric: "${match:2}" + +``versions`` » ``variables`` + + TODO + +``versions`` » ``include`` » ``distributions`` + + Like the global ``include`` » ``distributions`` field, but specific + to a versions. The value can refer to the value of the global + variable via ``${next-value}``. + + TODO overwriting entries? + +``versions`` » ``include`` » ``projects`` + + Like the global ``include`` » ``projects`` field, but specific to a + versions. The value can refer to the value of the global variable + via ``${next-value}``. + + TODO overwriting entries? diff --git a/source/reference/recipes/project.rst b/source/reference/recipes/project.rst new file mode 100644 index 0000000..846598b --- /dev/null +++ b/source/reference/recipes/project.rst @@ -0,0 +1,143 @@ +.. _recipes-project: + +================= + Project Recipes +================= + +Project :term:`recipes ` include some aspects of software +projects: + +* Name of the :term:`project` + +* :term:`Nature` of the :term:`project` (e.g. `CMake`_, Python + setuptools, …) + +* Repository URL + +* Specific build instructions if different from default build + instructions for the :term:`project's ` + :term:`nature`. Moreover, customizations are possible. + +A basic :term:`project` :term:`recipe`, which in this case would be +stored in a file named :file:`rsb-cpp.project`, looks like this: + +.. code-block:: yaml + + templates: + - code-cor-lab + - cmake-cpp + - base + + variables: + description: >- + C++ implementation of the robotics service bus (RSB) + keywords: + - middleware + - integration + - robotics + + repository: https://code.cor-lab.org/git/rsb.git.cpp + + redmine-project: rsb + + cmake.options: + - BUILD_TESTS=OFF + - BUILD_EXAMPLES=OFF + - "@{next-value|[]}" + +The following fields are most important: + +:samp:`{FILENAME}` + + This is not an actual field but the name of the :term:`recipe file + ` with its extension removed (e.g. ``rsb-cpp`` for a + :term:`recipe file ` named :file:`rsb-cpp.project`). + + Must be unique w.r.t. to all other :term:`projects ` in the + repository. Should in general not contain any version information + (unless particular versions of a project genuinely constitute their + own :term:`projects `). + +``templates`` + + TODO A list of :term:`template` names characterizing the technical + and organizational nature of the project. For most basic purposes, + it is sufficient to choose a suitable :term:`template` for the build + system used by the project and the ``base`` :term:`template` to + inherit specifications for the general nature of build jobs to + create. + + TODO It is important to note that :term:`templates