Demonstration 4

Purpose

Demonstration 3 brought ETCE into the EMANE domain. We focused on the structure of an ETCE Test Directory and the way that file naming and placement interacts with test execution steps.

While the directory layout of Demonstration 3 is easy to digest, the large number of files and, especially, files with only minor text differences makes such a layout impractical. Small test changes may require manual edits to many files, which is error prone. Replicate this over many tests and things quickly become unmaintainable.

Demonstration 4 repeats demo 3 but reduces the number of Test Directory files to a more manageable set by exploiting the configuration redundancy. The new files are standalone template files or are organized into entire template directories and use mako in a very similar way to what we've seen previously in the LXC Plan File. The new format requires an expanded test.xml file that indicates templates to ETCE and provides the template variable values for publishing the Test Directory to its final form.

Activity 1 - A Smaller Test Directory

Here is our Demonstration 4 Test Directory:

[etceuser@host]$ tree 04.templates
04.templates
|__ doc
|   |__ hostfile
|   |__ lxcplan.xml
|__ helper
|   |__ eelgenerator.xml
|   |__ eventservice.xml
|   |__ otestpoint-broker.xml
|   |__ scenario.eel
|__ node.tpl
|   |__ emaneshsnapshot.flag
|   |__ eventdaemon.xml
|   |__ gpsd.flag
|   |__ gpsdlocationagent.xml
|   |__ ieee80211abgmac.xml
|   |__ ieee80211abgnem.xml
|   |__ ieee80211pcr.xml
|   |__ mgen.script
|   |__ olsrd.conf
|   |__ otestpointd.xml
|   |__ otestpoint-recorder.xml
|   |__ platform.xml
|   |__ probe-emane-ieee80211abg.xml
|   |__ probe-emane-physicallayer.xml
|   |__ probe-emane-virtualtransport.xml
|   |__ transvirtual.xml
|__ steps.xml
|__ test.xml

The main change we see from the previous demo is that the ten node directories in 03.emane are replaced by one directory node.tpl. The tpl suffix indicates that node.tpl is a Template Directory. ETCE will treat every file in node.tpl as a file containing mako template strings that require values for publishing.

Let's also the list the test summary to look for changes:

[etceuser@host]$ etce-test list -v 04.templates
------------
04.templates
------------
location:
        04.templates
description:

    emane-tutorial demonstration 3 in ETCE Test Directory format
    and using templates.

overlays:
        etce_hostname
        etce_index
        etce_log_path
        eventservicedevice
        mgen_flow
        rtsthreshold

The important difference from the last demo is the expanded overlays section. In addition to the etce_log_path variable we saw there, we have five new ones: etce_hostname, etce_index, eventservicedevice, mgen_flow and rtsthreshold.

ETCE generates the overlays summary listing by searching through the Test Directory files to find mako template strings. Recall these are Python code snippets enclosed by a leading dollar sign and squiggly braces - ${some Python code here}. ETCE examines each template for variable names that will require values to convert the file into usable form - to change the mako templates into text. These are the names listed in the summary.

Where do we specify variable values? In test.xml. Here is 04.templates/test.xml:

<test>
  <name>04.templates</name>

  <description>
    emane-tutorial demonstration 3 in ETCE Test Directory format
    and using templates.
  </description>

  <overlays>
    <overlay name="eventservicedevice" value="backchan0"/>
  </overlays>

  <templates indices="1-10">
    <directory name="node">
      <overlay name="rtsthreshold" value="0"/>
      <overlaylist name="mgen_flow" values="1,2,3,4,5,6,7,8,9,10"/>
    </directory>
  </templates>
</test>

The new file adds an overlays and templates section to what we've seen previously. And we see three of our variables listed there.

The overlays and templates sections are independent - one can exist without the other. overlays strictly contains one or more overlay elements, each of which assigns a value to the named variable. Template variables defined here are visible to every file in the Test Directory. In our current example, an eventservicedevice mako template can be found in both the helper and node.tpl directories:

[etceuser@host]$ grep -R eventservicedevice 04.templates/*
04.templates/helper/eventservice.xml:  <param name="eventservicedevice" value="${eventservicedevice}"/>
04.templates/node.tpl/platform.xml:  <param name="eventservicedevice" value="${eventservicedevice}"/>
04.templates/node.tpl/eventdaemon.xml:  <param name="eventservicedevice" value="${eventservicedevice}"/>
04.templates/test.xml:    <overlay name="eventservicedevice" value="backchan0"/>

When the test is run, these will be replaced with the overlay value backchan0.

templates needs more explanation. The templates element requires a value for its indices attribute. indices is a list of one or more numbers separated by ',' or '-'. indices defines the list of values that the reserved variable etce_index will take when publishing the test. Some examples:

indices	etce_index values
1	1
1-10	1,2,3,4,5,6,7,8,9,10
1-4,7	1,2,3,4,7
1,4,7	1,4,7
1-4,5,6-10	1,2,3,4,5,6,7,8,9,10

The templates element may contain one or more file or directory children. In this case we have one directory element which points to our template directory node.tpl - the name attribute value does not include the tpl suffix, it is implied.

This is the important part - for each value of etce_index, ETCE will replicate the node.tpl directory and its contents filling in the mako fields based on the set values. Here is the nitty gritty for template directories. For each etce_index:

• ETCE builds a second variable, etce_hostname, by taking the root of the directory name (node) and appending the etce_index as a three character field. In the present case, node-001, node-002 and so on.
• In the destination directory, ETCE creates a subdirectory with the etce_hostname value. This becomes the target directory for the following steps.
• For each file in node.tpl, ETCE copies the file to the target directory replacing the mako template strings with text based on the available variable values. These are the reserved variables etce_index, etce_hostname and etce_log_path, the values specified in the overlays section of test.xml and the values specified as child overlay and overlaylist values to the directory element.

Here is the directory snippet of test.xml once more:

    <directory name="node">
      <overlay name="rtsthreshold" value="0"/>
      <overlaylist name="mgen_flow" values="1,2,3,4,5,6,7,8,9,10"/>
    </directory>

rtsthreshold and mgen_flow variables are only visible to the node.tpl directory - they would not be available for substitution for a second template directory unless repeated as children of that directory element. Other than this restricted visibility, rtsthreshold behaves the same as for values from the overlays section - it has a fixed value of 0 for all the directories generated from node.tpl. mgen_flow is an overlaylist which behaves much like an array in programming. The multiple values, separated by ',', align one-to-one with the values taken by etce_index. The number of values for both must agree.

Taken together - when etce-test executes 04.templates, it first publishes the test to /tmp/etce/current_test generating all of the files as described here. The output looks very much like the 03.emane demonstration directory with eleven host subdirectories, helper and node-001 through node-010. Based on our description of overlay variable values specified in test.xml and the built-in etce variables, here are the collection of values used to generate each target host subdirectory and its contents:

Target Subdir	etce_index	etce_hostname	etce_log_path	eventservicedevice	rtsthreshold	mgen_flow
helper	n/a	n/a	OUTDIR/helper	backchan0	n/a	n/a
node-001	1	node-001	OUTDIR/node-001	backchan0	0	1
node-002	2	node-002	OUTDIR/node-002	backchan0	0	2
node-003	3	node-003	OUTDIR/node-003	backchan0	0	3
node-004	4	node-004	OUTDIR/node-004	backchan0	0	4
node-005	5	node-005	OUTDIR/node-005	backchan0	0	5
node-006	6	node-006	OUTDIR/node-006	backchan0	0	6
node-007	7	node-007	OUTDIR/node-007	backchan0	0	7
node-008	8	node-008	OUTDIR/node-008	backchan0	0	8
node-009	9	node-009	OUTDIR/node-009	backchan0	0	9
node-010	10	node-010	OUTDIR/node-010	backchan0	0	10

A couple comments:

• etce_index and etce_hostname are not available when publishing the helper host directory because helper is not included within templates and is not associated with any index.
• Likewise, rtsthreshold and mgen_flow are not available to helper because these are defined with visibility restricted to node.tpl.
• OUTDIR is a placeholder for the output directory name generated at runtime (by default, under /tmp/etce/data) and discussed in 01.hello_etce.
• etce_index and mgen_flow take the same values for each row. The mgen_flow variable was added here to illustrate the use of an overlaylist. In practice etce_index would probably be used in its place. Keep in mind that mako template fields enclose Python code. Its typical to run into configuration situations where a text field can be related back to etce_index by a formula. In that case, the mako template can enclose the formula directly. As a simple example, in our present case, a mako template ${(2 * etce_index) - 1} appearing in one of the node.tpl files would generate the sequence of odd numbers from 1 to 19 within the ten generated files.

Activity 2 - Publishing a Test

So far we've used the etce-test commands list and run. There is also publish which performs the ETCE Test Directory translation we've been describing here, apart from running the Test. It can be helpful as an intermediate step during test development or as a check when reducing an existing test to use templates. It's also sometimes useful for transferring test configuration in a more suitable format to someone not using ETCE.

The basic form of publish takes an input Test Directory and an output directory to place the translated results. publish also has an optional logdirectory argument to use for the node indenpendent part of the etce_log_path variable during translation. The auto-generated path containing the timestamp doesn't usually make sense when publishing by hand. Let's publish the current demo directory and peak at the result:

[etceuser@host]$ etce-test publish --logdirectory /tmp 04.templates /tmp/04.templates
Publishing 04.templates to /tmp/04.templates
Processing template directory "node.tpl" for etce_index=1 and destination=/tmp/04.templates/node-001
Processing template directory "node.tpl" for etce_index=2 and destination=/tmp/04.templates/node-002
Processing template directory "node.tpl" for etce_index=3 and destination=/tmp/04.templates/node-003
Processing template directory "node.tpl" for etce_index=4 and destination=/tmp/04.templates/node-004
Processing template directory "node.tpl" for etce_index=5 and destination=/tmp/04.templates/node-005
Processing template directory "node.tpl" for etce_index=6 and destination=/tmp/04.templates/node-006
Processing template directory "node.tpl" for etce_index=7 and destination=/tmp/04.templates/node-007
Processing template directory "node.tpl" for etce_index=8 and destination=/tmp/04.templates/node-008
Processing template directory "node.tpl" for etce_index=9 and destination=/tmp/04.templates/node-009
Processing template directory "node.tpl" for etce_index=10 and destination=/tmp/04.templates/node-010

[etceuser@host]$ tree -L 1 /tmp/04.templates
/tmp/04.templates
|__ helper
|__ node-001
|__ node-002
|__ node-003
|__ node-004
|__ node-005
|__ node-006
|__ node-007
|__ node-008
|__ node-009
|__ node-010
|__ steps.xml
|__ test.xml

11 directories, 2 files

The output has the generated host subdirectories we expect. Let's also look at test.xml in the output:

[etceuser@host]$ cat /tmp/04.templates/test.xml
<test><name>04.templates</name>

  <description>
    emane-tutorial demonstration 3 in ETCE Test Directory format
    and using templates.
  </description>

  </test>

It's a minor point, but, the overlays and templates section doesn't appear in the published test.xml - they've been consumed. This insures the result directory is still a valid ETCE Test Directory:

[etceuser@host]$ etce-test list -v /tmp/04.templates
------------
04.templates
------------
location:
        /tmp/04.templates
description:

    emane-tutorial demonstration 3 in ETCE Test Directory format
    and using templates.

overlays:

Recall that our current demo is the same as 03.emane but rewritten to take advantage of templates. To be certain we've pared down the original correctly, let's publish it too and compare the results:

[etceuser@host]$ etce-test publish --logdirectory /tmp 03.emane /tmp/03.emane
Publishing 03.emane to /tmp/03.emane

[etceuser@host]$ diff -rq /tmp/03.emane /tmp/04.templates
Files /tmp/03.emane/test.xml and /tmp/04.templates/test.xml differ

[etceuser@host]$ diff /tmp/03.emane/test.xml /tmp/04.templates/test.xml
1c1
< <test><name>03.emane</name>
---
> <test><name>04.templates</name>
4c4,5
<     emane-tutorial demonstration 3 in ETCE Test Directory format.
---
>     emane-tutorial demonstration 3 in ETCE Test Directory format
>     and using templates.
7c8
< </test>
---
>   </test>

The first diff is recursive and only lists the files that differ - test.xml is the only one. Diffing those two files shows that the name and description are the only differences. The published tests are identical otherwise.

Activity 3 - test.xml Miscellany

Although our example test.xml file has grown from previous demonstrations, it still does not contain every available attribute or capture every possible element ordering. Here we collect a few items worth knowing:

• overlay and overlaylist elements may be placed as direct descendants of the templates element where they are visible to all of the directories and files generated from indices. This makes sense when the values are shared by more than one directory or file element.
• The directory and file elements beneath templates both have an optional indices attribute whose value must be a subset of the templates indices attribute value. The associated directory or file is only generated using the restricted subset. As an example - if the directory element above contained the attribute indices="1-4" then only the output subdirectories node-001 through node-004 are generated. The overlaylist child for mgen_flow would need to be adjusted accordingly to have only four values.
• directory and file elements have an optional hostname_format attribute. This attribute overrides the default etce_hostname format we saw in our example, where the index value is appended to the template directory name as a three digit field. hostname_format is usually a mako string that contains etce_index and formats it into an alternate name. hostname_format="node-${'%02d' % etce_index}" would generate the same names we had here, but with the index field changed to two characters: node-01, 'node-02', etc.
• overlay ane overlaylist elements have an optional type attribute that must be one of string, bool, int or float. When type is not specified, ETCE tries to infer the value type by converting the string to a more restrictive type. It attempts this in order int, float, bool and finally string. bool will be chosen when the value is true or false regardless of capitalization. The type field is useful when the inference doesn't work as expected. For example, an overlay value of 000346 that represents a fixed width integer string will be converted to an integer and lose the desired formatting. type="string" will prevent this.
• overlaylist has an optional separator attribute that sets the character used to delineate the multiple items in values. The default is ,. Use this to separate values that contain a comma.

Activity 4 - Run The Demo

As shown above, our current demo publishes to the identical form as the previous one (save for comments). Feel free to run it and check that the results are the same.