Better rewrite of NXdata scaling_factor and offset fields (#1343)

Adds FIELDNAME_scaling_factor and FIELDNAME_offset as fields to NXdata. These fields should be used instead of scaling_factor and offset, as it is ambiguous which fields to apply them to in the case of multiple signals. _scaling_factor and _offset are added as reserved suffixes.

Additionally adds a formula for how to apply these fields:

corrected values = (FIELDNAME + offset) * scaling_factor

For NXmx, adds clarification these fields can be used as background and gain correction fields, and defines these terms. Also the possible rank options are elaborated.

Closes #1332.

Also:
- Change NXdata scaling_factor to refer to "plotted" data
- Change NXdata to refer to "corrected" data, in addition to "physical" data, since it describes units of photons
- Use FIELDNAME_scaling_factor and FIELDNAME_offset.  This resolves ambiguity if there is more than one signal
- For NXmx specify data_scaling_factor and data_offset since the field data is named in the NXdata group
- NXmx: fully define pedestal, bias, and gain and how they relate to offset and scaling_factor
- Add reserved suffixes _scaling_factor and _offset
- NXdata: define better how the stored values in FIELDNAME are converted to physical values
- NXdata: more clarification on how scaling_factor and offset without FIELDNAME are ambiguous
- Add implied defaults for sclaing_factor and offset

applications/NXmx.nxdl.xml

@@ -142,6 +142,50 @@
                         <dim index="4" value="k" required="false"/>
                     </dimensions>
                 </field>
+
+                <field name="data_scaling_factor" type="NX_NUMBER" optional="true">
+                    <doc>
+                        An optional scaling factor to apply to the values in ``data``.
+
+                        The elements in ``data`` are often stored as integers for efficiency reasons and need
+                        further correction, generating floats.  The two fields data_scaling_factor and
+                        data_offset allow linear corrections using the following convention:
+
+                        .. code-block::
+
+                                corrected_data = (data + offset) * scaling_factor
+
+                        This formula will derive the corrected value, when necessary.
+
+                        data_scaling_factor is sometimes known as gain and data_offset is sometimes
+                        known as pedestal or background, depending on the community.
+
+                        Use these fields to specify constants that need to be applied
+                        to the data to correct it to physical values.  For example, if the detector gain
+                        is 10 counts per photon and a constant background of 400 needs to be subtracted
+                        off the pixels, specify data_scaling_factor as 0.1 and data_offset as -400 to
+                        specify the required conversion from raw counts to corrected photons. It
+                        is implied processing software will apply these corrections on-the-fly during processing.
+
+                        The rank of these fields should either be a single value for the full dataset, a
+                        single per-pixel array applied to every image (dimensions (i, j) or (i, j, k)),
+                        or a per-image correction specified with an array whose slowest rank is nP (dimensions
+                        (np, 1), (np, i, j) or (np, i, j, k)).
+
+                        When omitted, the scaling factor is assumed to be 1.
+                </doc>
+                </field>
+                <field name="data_offset" type="NX_NUMBER" optional="true">
+                <doc>
+                        An optional offset to apply to the values in data.
+
+                        When omitted, the offset is assumed to be 0.
+
+                        See :ref:`data_scaling_factor &lt;/NXmx/ENTRY/DATA/data_scaling_factor-field&gt;` for more information.
+                </doc>
+                </field>
+
+
             </group>
 
             <group type="NXsample">

base_classes/NXdata.nxdl.xml

@@ -446,21 +446,56 @@
 	</field>
 
 	<!-- Data vs. plot coordinates -->
-	<field name="scaling_factor" type="NX_FLOAT">
+	<field name="FIELDNAME_scaling_factor" type="NX_NUMBER">
 		<doc>
-			The elements in data are usually float values really. For
-			efficiency reasons these are usually stored as integers
-			after scaling with a scale factor. This value is the scale
-			factor. It is required to get the actual physical value,
-			when necessary.
+			An optional scaling factor to apply to the values in any field named ``FIELDNAME``
+			in this ``NXdata`` group. This can be a :ref:`DATA &lt;/NXdata/DATA-field&gt;` field
+			(signal or auxiliary signal) or a :ref:`AXISNAME &lt;/NXdata/AXISNAME-field&gt;`
+			field (axis).
+
+			The elements stored in NXdata datasets are often stored as integers for efficiency
+			reasons and need further correction or conversion, generating floats.  For example,
+			raw values could be stored from a device that need to be converted to values that
+			represent the physical values. The two fields FIELDNAME_scaling_factor and
+			FIELDNAME_offset allow linear corrections using the following convention:
+
+			.. code-block::
+
+				corrected values = (FIELDNAME + offset) * scaling_factor
+
+			This formula will derive the values to use in downstream applications, when necessary.
+
+			When omitted, the scaling factor is assumed to be 1.
 		</doc>
 	</field>
-	<field name="offset" type="NX_FLOAT">
+	<field name="FIELDNAME_offset" type="NX_NUMBER">
 		<doc>
-			An optional offset to apply to the values in data.
+			An optional offset to apply to the values in FIELDNAME (usually the signal).
+
+			When omitted, the offset is assumed to be 0.
+
+			See :ref:`FIELDNAME_scaling_factor &lt;/NXdata/FIELDNAME_scaling_factor-field&gt;` for more information.
 		</doc>
 	</field>
 
+	<field name="scaling_factor" type="NX_FLOAT" deprecated="Use FIELDNAME_scaling_factor instead">
+		<doc>
+			The scaling_factor and FIELDNAME_scaling_factor fields have similar semantics.
+			However, scaling_factor is ambiguous in the case of multiple signals.  Therefore
+			scaling_factor is deprecated.  Use FIELDNAME_scaling_factor instead, even when
+			only a single signal is present.
+		</doc>
+	</field>
+	<field name="offset" type="NX_FLOAT" deprecated="Use FIELDNAME_offset instead">
+		<doc>
+			The offset and FIELDNAME_offset fields have similar semantics.
+			However, offset is ambiguous in the case of multiple signals.  Therefore
+			offset is deprecated.  Use FIELDNAME_offset instead, even when
+			only a single signal is present.
+		</doc>
+	</field>
+
+
 	<!-- Other fields -->
 	<field name="title">
 		<doc>

manual/source/datarules.rst

@@ -231,21 +231,26 @@ the following table lists the suffixes reserved by NeXus.
     reserved suffixes; mask
     reserved suffixes; set
     reserved suffixes; weights
-
-==================  =========================================  =================================
-suffix              reference                                  meaning
-==================  =========================================  =================================
-``_end``            :ref:`NXtransformations`                   end points of the motions that start with ``DATASET``
-``_errors``         :ref:`NXdata`                              uncertainties (a.k.a., errors)
-``_increment_set``  :ref:`NXtransformations`                   intended average range through which the corresponding axis moves during the exposure of a frame
-``_indices``        :ref:`NXdata`                              Integer array that defines the indices of the signal field which need to be used in the ``DATASET`` in order to reference the corresponding axis value
-``_mask``           ..                                         Field containing a signal mask, where 0 means the pixel is not masked. If required, bit masks are defined in :ref:`NXdetector` ``pixel_mask``.
-``_set``            :ref:`target values <target_value>`        Target value of ``DATASET``
-``_weights``        ..                                         divide ``DATASET`` by these weights [#]_
-==================  =========================================  =================================
-
-.. [#] If ``DATASET_weights`` exists and has the same shape as the field,
-   you are supposed to divide ``DATASET`` by the weights.
+    reserved suffixes; scaling_factor
+    reserved suffixes; offset
+
+===================  =========================================  =================================
+suffix               reference                                  meaning
+===================  =========================================  =================================
+``_end``             :ref:`NXtransformations`                   end points of the motions that start with ``DATASET``
+``_errors``          :ref:`NXdata`                              uncertainties (a.k.a., errors)
+``_increment_set``   :ref:`NXtransformations`                   intended average range through which the corresponding axis moves during the exposure of a frame
+``_indices``         :ref:`NXdata`                              Integer array that defines the indices of the signal field which need to be used in the ``DATASET`` in order to reference the corresponding axis value
+``_mask``            ..                                         Field containing a signal mask, where 0 means the pixel is not masked. If required, bit masks are defined in :ref:`NXdetector` ``pixel_mask``.
+``_set``             :ref:`target values <target_value>`        Target value of ``DATASET``
+``_weights``         ..                                         divide ``DATASET`` by these weights [#suffix_note]_
+``_scaling_factor``  :ref:`NXdata`                              Multiply ``DATASET`` by this factor [#suffix_note]_
+``_offset``          :ref:`NXdata`                              Add this factor to ``DATASET`` [#suffix_note]_
+===================  =========================================  =================================
+
+.. [#suffix_note] If ``DATASET_weights`` exists and has the same shape as the field,
+   you are supposed to divide ``DATASET`` by the weights. Similarly, if `DATASET_scaling_factor` and/or
+   `DATASET_offset` exist, apply this equation: (``DATASET`` + ``DATASET_offset``) * ``DATASET_scaling_factor``
 
 .. Note that the following line might be added to the above table pending discussion: