Skip to content

Commit

Permalink
NIAC suggestions: start with simple example at the top, histogram axi…
Browse files Browse the repository at this point in the history 
…s +1, indices can be omitted, indices cannot contradict @axes
  • Loading branch information
@woutdenolf
woutdenolf committed Aug 15, 2024
1 parent 37306ef commit 5dc447d
Show file tree
Hide file tree
Showing 3 changed files with 2,216 additions and 170 deletions.
300 changes: 130 additions & 170 deletions base_classes/NXdata.nxdl.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -60,222 +60,180 @@
does not describe how the data is to be plotted or even the dimensionality of the plot.
https://www.nexusformat.org/NIAC2018Minutes.html#nxdata-plottype--attribute

**Signals:**

.. index:: plotting

The :ref:`DATA </NXdata/DATA-field>` fields contain the signal values to be plotted. The name of the field
to be used as the *default plot signal* is provided by the :ref:`signal </NXdata@signal-attribute>` attribute.
The names of the fields to be used as *secondary plot signals* are provided by the
:ref:`auxiliary_signals</NXdata@auxiliary_signals-attribute>` attribute.

An example with three signals, one of which being the default

.. code-block::

data:NXdata
@signal = "data1"
@auxiliary_signals = ["data2", "data3"]
data1: float[10,20,30] --> the default signal
data2: float[10,20,30]
data3: float[10,20,30]

**Axes:**

.. index:: axes (attribute)
.. index:: coordinates
**Usage:**

The :ref:`AXISNAME </NXdata/AXISNAME-field>` fields contain the axis coordinates associated with the signal values.
.. admonition:: Plot data signal versus x axis

``Default axes``
.. code-block::

The :ref:`axes </NXdata@axes-attribute>` attribute provides the names of the :ref:`AXISNAME </NXdata/AXISNAME-field>`
fields to be used as the default axis for each dimension of the :ref:`DATA </NXdata/DATA-field>` fields.
As a result the length of the :ref:`axes </NXdata@axes-attribute>` attribute must be equal to the rank of the :ref:`DATA </NXdata/DATA-field>`
fields. When a particular dimension has no default axis, the string “.” is used in that position.
data:NXdata
@signal = "data"
@axes = ["x"]
data: float[100]
x: float[100]

``Spanned dimensions``
More complex cases are supported

The :ref:`AXISNAME_indices </NXdata@AXISNAME_indices-attribute>` attributes describe the
:ref:`DATA </NXdata/DATA-field>` dimensions spanned by the corresponding :ref:`AXISNAME </NXdata/AXISNAME-field>` fields.
* histogram data: ``x`` has one more value than ``data``.
* alternative axes: instead of a single ``x`` axis you can have several axes, one of which being the default.
* signals with more than one dimension: ``data`` could be 2D with axes ``x`` and ``y`` along each dimension.
* axes with more than one dimension: ``data`` could be 2D with axes ``x`` and ``y`` also being 2D, providing a
unique ``[x, y]`` coordinate for each ``data`` point.

When :ref:`AXISNAME_indices </NXdata@AXISNAME_indices-attribute>` is missing for a given :ref:`AXISNAME </NXdata/AXISNAME-field>` field,
the position(s) of the :ref:`AXISNAME </NXdata/AXISNAME-field>` field name in the :ref:`axes </NXdata@axes-attribute>` attribute
is(are) used. The shape of an :ref:`AXISNAME </NXdata/AXISNAME-field>` field must be equal to the shape of the
:ref:`DATA </NXdata/DATA-field>` dimensions it spans. This means that ``AXISNAME.shape[i] == DATA.shape[AXISNAME_indices[i]]``
for each ``i`` in ``[0, AXISNAME.ndim)``.

Note that an :ref:`AXISNAME </NXdata/AXISNAME-field>` field can have more than one dimension and can therefore span
more than one :ref:`DATA </NXdata/DATA-field>` dimension. Conversely, one :ref:`DATA </NXdata/DATA-field>` dimension
can be spanned by more than one :ref:`AXISNAME </NXdata/AXISNAME-field>` field. As stated before, the default
axis name (if any) of each dimension can be found in the :ref:`axes </NXdata@axes-attribute>` attribute.
**Signals:**

``Available axes``
.. index:: plotting

With this definition of the :ref:`axes </NXdata@axes-attribute>` and :ref:`AXISNAME_indices </NXdata@AXISNAME_indices-attribute>` attributes
a list of all available axes is not provided directly. All strings in the :ref:`axes </NXdata@axes-attribute>` attribute
(excluding the “.” string) are axis field names. In addition the prefix of an attribute ending with the string "_indices" is also
an axis field name.
.. admonition:: Defined by

**Axes examples:**
* :ref:`DATA </NXdata/DATA-field>` fields
* the :ref:`signal </NXdata@signal-attribute>` attribute
* the :ref:`auxiliary_signals</NXdata@auxiliary_signals-attribute>` attribute

``1. Single-dimensional axes``
The :ref:`DATA </NXdata/DATA-field>` fields contain the signal values to be plotted. The name of the field
to be used as the *default plot signal* is provided by the :ref:`signal </NXdata@signal-attribute>` attribute.
The names of the fields to be used as *secondary plot signals* are provided by the
:ref:`auxiliary_signals</NXdata@auxiliary_signals-attribute>` attribute.

:ref:`AXISNAME </NXdata/AXISNAME-field>` fields are typically one-dimensional arrays that span a single :ref:`DATA </NXdata/DATA-field>` dimension.
.. admonition:: An example with three signals, one of which being the default

An example of this would be 2D data on a regular coordinate grid
.. code-block::

.. code-block::
data:NXdata
@signal = "data1"
@auxiliary_signals = ["data2", "data3"]
data1: float[10,20,30] --> the default signal
data2: float[10,20,30]
data3: float[10,20,30]

data:NXdata
@signal = "data"
@axes = ["x", "y"]
data: float[10,20]
x: float[10] --> coordinates along the first dimension
y: float[20] --> coordinates along the second dimension
**Axes:**

In this example each data point ``data[i,j]`` has axis coordinates ``[x[i], y[j]]``.
.. index:: axes (attribute)
.. index:: coordinates

Note that ``@x_indices`` and ``@y_indices`` attributes can be omitted in this case. However it is strongly encouraged to provide them.
.. admonition:: Defined by

``2. Data dimensions spanned by more than one axis``
* :ref:`AXISNAME </NXdata/AXISNAME-field>` fields
* the :ref:`axes </NXdata@axes-attribute>` attribute
* :ref:`AXISNAME_indices </NXdata@AXISNAME_indices-attribute>` attributes

A common case is the need to specify alternative axes for the same dimension
The fields and attributes are defined as follows

.. code-block::
1. The :ref:`AXISNAME </NXdata/AXISNAME-field>` fields contain the axis coordinates associated with the signal values.

data:NXdata
@signal = "data"
@axes = ["eta", "q"]
data: float[10,20]
@eta_indices = [0]
@tth_indices = [1]
@q_indices = [1]
eta: float[10] --> coordinates along the first dimension
tth: float[20] --> coordinates along the second dimension (alternative)
q: float[20] --> coordinates along the second dimension (default)
2. The :ref:`axes </NXdata@axes-attribute>` attribute provides the names of the :ref:`AXISNAME </NXdata/AXISNAME-field>`
fields to be used as the `default axis` for each dimension of the :ref:`DATA </NXdata/DATA-field>` fields.

Note that to recognize ``tth`` as an axis, ``@tth_indices`` must be present. Readers that do
not make use of the :ref:`AXISNAME_indices </NXdata@AXISNAME_indices-attribute>` attributes will
not understand that ``tth`` is an axis and cannot take this field into account.
3. The :ref:`AXISNAME_indices </NXdata@AXISNAME_indices-attribute>` attributes describe the :ref:`DATA </NXdata/DATA-field>`
dimensions spanned by the corresponding :ref:`AXISNAME </NXdata/AXISNAME-field>` fields.

``3. Multi-dimensional axes``
Additional requirements for a complete definition

When coordinates do not form a regular grid, multi-dimensional :ref:`AXISNAME </NXdata/AXISNAME-field>` fields
can be used. Just remember that the shape of an :ref:`AXISNAME </NXdata/AXISNAME-field>` field must be equal to the shape
of the :ref:`DATA </NXdata/DATA-field>` dimensions it spans.
1. The length of the :ref:`axes </NXdata@axes-attribute>` attribute must be equal to the rank of the :ref:`DATA </NXdata/DATA-field>`
fields. When a particular dimension has no default axis, the string “.” is used in that position.

An example of this would be 2D scatter data where each data point has unique coordinates
2. When :ref:`AXISNAME_indices </NXdata@AXISNAME_indices-attribute>` is missing for a given
:ref:`AXISNAME </NXdata/AXISNAME-field>` field, the position(s) of the :ref:`AXISNAME </NXdata/AXISNAME-field>`
field name in the :ref:`axes </NXdata@axes-attribute>` attribute is(are) used.

.. code-block::
3. The shape of an :ref:`AXISNAME </NXdata/AXISNAME-field>` field must `correspond` to the shape of the
:ref:`DATA </NXdata/DATA-field>` dimensions it spans. This means that for each dimension ``i`` in ``[0, AXISNAME.ndim)``
the number of data points along this dimension ``DATA.shape[AXISNAME_indices[i]]`` must be equal to the number
of axis values along this dimension ``AXISNAME.shape[i]`` or the number of bin edges ``AXISNAME.shape[i]+1``
in case of histogram data.

data:NXdata
@signal = "data"
@axes = ["x", "y"]
@x_indices = [0, 1]
@y_indices = [0, 1]
data: float[10,20]
x: float[10,20] --> coordinates along both dimensions
y: float[10,20] --> coordinates along both dimensions
4. When :ref:`AXISNAME_indices </NXdata@AXISNAME_indices-attribute>` is the same as the indices of "AXISNAME" in the
:ref:`axes </NXdata@axes-attribute>` attribute, there is no need to provide
:ref:`AXISNAME_indices </NXdata@AXISNAME_indices-attribute>`.

In this example each data point ``data[i,j]`` has axis coordinates ``[x[i,j], y[i,j]]`` and when
plotting, ``x`` is used along the first data dimension by default and `y` along the second data dimension.
Since ``x`` and ``y`` span both dimensions, a reader could choose to use ``y`` for the first dimension
and ``x`` for the second as an alternative to the default. A writer could also choose to not specify
any default by defining ``@axes = [".", "."]`` and leave the decision up the the reader.
5. The indices of "AXISNAME" in the :ref:`axes </NXdata@axes-attribute>` attribute must be a subset of
:ref:`AXISNAME_indices </NXdata@AXISNAME_indices-attribute>`.

Note that omitting ``@x_indices`` would result in ``@x_indices = [0]`` as derived from the position of the string ``"x"`` in ``@axes``.
This would be invalid since the shape ``[10,20]`` of ``x`` is not equal to the shape ``[10]`` of the spanned data dimensions.
Highlight concequences that follow the definition

Omitting indices for multi-dimensional axes can only be done by repeating the :ref:`AXISNAME </NXdata/AXISNAME-field>`
name in all positions of the :ref:`axes </NXdata@axes-attribute>` attribute which they span. For example
1. An :ref:`AXISNAME </NXdata/AXISNAME-field>` field can have more than one dimension and can therefore span
more than one :ref:`DATA </NXdata/DATA-field>` dimension. Conversely, one :ref:`DATA </NXdata/DATA-field>` dimension
can be spanned by more than one :ref:`AXISNAME </NXdata/AXISNAME-field>` field. The default axis name (if any)
of each dimension can be found in the :ref:`axes </NXdata@axes-attribute>` attribute.

.. code-block::
2. A list of all available axes is not provided directly. All strings in the :ref:`axes </NXdata@axes-attribute>` attribute
(excluding the “.” string) are axis field names. In addition the prefix of an attribute ending with the string "_indices" is also
an axis field name.

data:NXdata
@signal = "data"
@axes = ["x", "x"]
data: float[10,20]
x: float[10,20] --> coordinates along both dimensions
.. admonition:: The following example covers all axes features supported

Note that since ``NXdata`` does not describe how the data is to be plotted or even the dimensionality of the plot,
a reader would probably treat this particular example as a 1D signal and plot it as such.
.. code-block::

In the case of multi-dimensional axes, single-dimensional axes are often introduced as default axes
to support readers that do not make use of the :ref:`AXISNAME_indices </NXdata@AXISNAME_indices-attribute>` attributes
or cannot handle multi-dimensional axes.
data:NXdata
@signal = "data"
@axes = ["x_set", "y_set", "."] --> default axes for all three dimensions
@x_encoder_indices = [0, 1]
@y_encoder_indices = 1 --> or [1]
data: float[10,7,1024]
x_encoder: float[11,7] --> coordinates along the first and second dimensions
y_encoder: float[7] --> coordinates along the second dimension
x_set: float[10] --> coordinates along the first dimension
y_set: float[7] --> coordinates along the second dimension

The 2D scatter example can be enriched as follows
One scientific application that provided such data is 2D scanning X-ray Fluorescence
where the sample is scanned through a focussed X-ray beam in two dimensions and an XRF spectrum
with 1024 channels is recorded at each position. In addition the x motor is scanned
continuously and both motors have an encoder which records the actual motor position as
opposed to the requested or `set` position.

.. code-block::
.. image:: data/example2D_hist.png
:width: 100%

data:NXdata
@signal = "data"
@axes = ["x_set", "y_set"]
@x_indices = [0, 1]
@y_indices = [0, 1]
@x_set_indices = 0 --> or [0]
@y_set_indices = 1 --> or [1]
data: float[10,20]
x: float[10,20] --> coordinates along both dimensions
y: float[10,20] --> coordinates along both dimensions
x_set: float[10] --> coordinates along the first dimension
y_set: float[20] --> coordinates along the second dimension
1. ``@axes`` has three values which corresponds to the ``data`` rank of three

The first dimension is spanned by three axes: ``x``, ``y`` and ``x_set``. The second dimension is also
spanned by three axes: ``x``, ``y`` and ``y_set``.
2. ``x_set`` and ``y_set`` are the default axes for the first two dimensions while the third
dimension does not have a default axis or any axis for that matter.

Technically ``@x_set_indices`` and ``@y_set_indices`` can be omitted. However it is strongly encouraged to provide them.
3. ``x_set_indices`` and ``y_set_indices`` are omitted because they would be equal to
the position of ``"x_set"`` and ``"y_set"`` in ``@axes``.

``4. Axes without defaults``
4. The first dimension is spanned by two axes ``x_set`` and ``x_encoder``. Since
the x motor is scanned continuously, the encoder records the edge of every bin
on which an XRF spectrum is recorded yielding 11 values instead of 10 along the
first dimension. The ``x_encoder`` spans two dimensions because the actual x edges
are slightly different for every x motion at each y position.

Expanding on the previous example, a stack of 2D scatter data where the stack dimension
does not have an axis can be described as follows
5. The second dimension is spanned by two axes ``y_set`` and ``y_encoder``. The axes
have as many values are there are data points along the second dimension. This is
because the y motor moves one step after each scan of the x motor. Since the y
motor does not move while scanning x, there is no need for ``y_encoder`` to span
the first dimension because the value along this dimension remains constant.

.. code-block::
**Uncertainties:**

data:NXdata
@signal = "data"
@axes = [".", "x_set", "y_set"]
@x_indices = [1, 2]
@y_indices = [1, 2]
@x_set_indices = 1
@y_set_indices = 2
data: float[5,10,20]
x: float[10,20] --> coordinates along the last two dimensions
y: float[10,20] --> coordinates along the last two dimensions
x_set: float[10] --> coordinates along the second dimension
y_set: float[20] --> coordinates along the third dimension

Note that the first dimension does not have a default axis but this doesn't necessarily mean
the first dimension is not spanned by any axis, although in this example that is the case.
.. admonition:: Defined by

**Uncertainties:**
* :ref:`FIELDNAME_errors </NXdata/FIELDNAME_errors-field>` fields

Standard deviations on data values as well as coordinates can be provided by
:ref:`FIELDNAME_errors </NXdata/FIELDNAME_errors-field>` fields where ``FIELDNAME`` is the name of a
:ref:`DATA </NXdata/DATA-field>` field or an :ref:`AXISNAME </NXdata/AXISNAME-field>` field.

An example of uncertainties on the signal, auxiliary signals and axis coordinates

.. code-block::

data:NXdata
@signal = "data1"
@auxiliary_signals = ["data2", "data3"]
@axes = ["x", "z"]
@x_indices = 0
@z_indices = 2
data1: float[10,20,30]
data2: float[10,20,30]
data3: float[10,20,30]
x: float[10]
z: float[30]
data1_errors: float[10,20,30]
data2_errors: float[10,20,30]
data3_errors: float[10,20,30]
x_errors: float[10]
z_errors: float[30]
.. admonition:: An example of uncertainties on the signal, auxiliary signals and axis coordinates

.. code-block::

data:NXdata
@signal = "data1"
@auxiliary_signals = ["data2", "data3"]
@axes = ["x", ".", "z"]
@x_indices = 0
@z_indices = 2
data1: float[10,20,30]
data2: float[10,20,30]
data3: float[10,20,30]
x: float[10]
z: float[30]
data1_errors: float[10,20,30]
data2_errors: float[10,20,30]
data3_errors: float[10,20,30]
x_errors: float[10]
z_errors: float[30]

</doc>

Expand Down Expand Up @@ -473,10 +431,12 @@
<doc>
"Errors" (meaning *uncertainties* or *standard deviations*)
associated with any field named ``FIELDNAME`` in this ``NXdata``
group (e.g. an axis, signal or auxiliary signal).
group. This can be a :ref:`DATA &lt;/NXdata/DATA-field&gt;` field
(signal or auxiliary signal) or a :ref:`AXISNAME &lt;/NXdata/AXISNAME-field&gt;`
field (axie).

The dimensions of the ``FIELDNAME_errors`` field must match
the dimensions of the ``FIELDNAME`` field.
the dimensions of the corresponding ``FIELDNAME`` field.
</doc>
</field>
<field name="errors" type="NX_NUMBER" deprecated="Use ``DATA_errors`` instead (NIAC2018)">
Expand Down
Loading

0 comments on commit 5dc447d

Please sign in to comment.