update 1.2.8

Documentation/EGADS Algorithm Handbook - LATEX/cover/alg_cover.tex

@@ -24,7 +24,7 @@
 \setlength{\voffset}{.0cm}
 
 \logoeufar
-\title{EUFAR, Olivier Henry}{}{EGADS Lineage Algorithm Handbook}{Version 1.2.7}
+\title{EUFAR, Olivier Henry}{}{EGADS Lineage Algorithm Handbook}{Version 1.2.8}
 %\author{}{}
 
 

Documentation/changelog.txt

@@ -7,14 +7,24 @@ To do list:
   - check feasability of an online algorithm database.
   - add function to create algorithm template.
   - add function to add a folder, somewhere on the hard drive, to the list of algorithms.
-  - add new tests to test each function of each class
+  - add new tests to test each function of each class.
+  - add the possibility to add multiple attributes and groups for NetCdf and HDF classes.
 
 To fix list:
 ------------
 
 
+January 6 2021, Release version 1.2.8
+=====================================
+  FIXED:
+    - in EgadsNetCdf class, when writing data to file, attributes were added to the variable after the creation of the variable and addition of data. This leads to an issue with add_offset and scale_factor attributes. Now attributes are added after the creation of the variable and before the addition of data, fixing the issue.
+
+  ADDED:
+    - two parameters have been added to NetCdf class / _write_variable method, scale_factor and add_offset, to offer the user the possibility to use them even if the class EgadsNetCdf is not used.
+
+
 July 15 2020, Release version 1.2.7
-==================================
+===================================
 The results of the following functions get_variable_list(), get_group_list() and get_dimension_list() for Hdf and NetCdf classes is the same type whatever the option used in each function. Now if details = True, get_variable_list() and get_group_list() return a list of string, each string is composed of the path to the group/variable + group/variable name. get_dimension_list() returns a dictionary of keys, composed of the path to the dimension + the dimension name, and values composed of the size of the dimension. It is the same schema than the one before 1.2.0.
 
 

PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 1.2
 Name: egads-lineage
-Version: 1.2.7
+Version: 1.2.8
 Summary: EGADS: EUFAR General Airborne Data-processing Software
 Home-page: https://github.com/EUFAR/egads/tree/Lineage
 Author: EUFAR, Olivier Henry
@@ -22,6 +22,15 @@ Keywords: airbornescience,netcdf,nasa-ames,eufar,science,microphysics,thermodyna
 Platform: Windows
 Platform: Linux
 Platform: MacOS
+Classifier: Development Status :: 5 - Production/Stable
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Education
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: GNU General Public License v3 (GPLv3)
+Classifier: Natural Language :: English
+Classifier: Programming Language :: Python
+Classifier: Topic :: Scientific/Engineering :: Atmospheric Science
 Requires: numpy (>=1.14)
 Requires: netCDF4 (>=1.3.0)
 Requires: python_dateutil (>=2.6.1)

README.md

@@ -1,7 +1,7 @@
 Version:
 -------
 
-EGADS Lineage 1.2.7
+EGADS Lineage 1.2.8
 
 
 Developments:

doc/source/conf.py

@@ -48,9 +48,9 @@
 # built documents.
 #
 # The short X.Y version.
-version = '1.2.7'
+version = '1.2.8'
 # The full version, including alpha/beta/rc tags.
-release = '1.2.7'
+release = '1.2.8'
 
 # The language for content autogenerated by Sphinx. Refer to documentation
 # for a list of supported languages.

doc/source/install.rst

@@ -98,6 +98,7 @@ Uninstallation
 Just run the following command from your terminal:
 
    >>> pip uninstall egads-lineage
+
 or
   remove manually all folders in your Python site-packages folder containing egads name.
 

doc/source/tutorial-netcdf.inc

@@ -162,13 +162,13 @@ The following describe how to add dimensions or attributes to a file.
    >>> f.add_attribute('project', 'my project', 'test1/data/temperature')
    >>> f.add_attribute('project', 'my project', 'test1/data')
 
-Data can be output to variables using the ``write_variable()`` function as follows:
+If using :class:`~.EgadsNetCdf`, data can be output to variables using the ``write_variable()`` function as follows:
 
     >>> f.write_variable(data, varname, dims, ftype, fillvalue)
 
 .. function:: f.write_variable(data, varname[, dims=None, ftype='double', fillvalue=None])
 
-   Write the values contained in *data* in the variable *varname* in a NetCDF file. If *varname* contains a path to a group, the variable will be created in the specified group, but in that case the group has to be created before. If using :class:`~.NetCdf`, values for *data* passed into ``write_variable`` must be scalar or array. Otherwise, if using :class:`~.EgadsNetCdf`, an instance of :class:`~.EgadsData` must be passed into ``write_variable``. In this case, any attributes that are contained within the :class:`~.EgadsData` instance are applied to the NetCDF variable as well. If an attribute with a name equal to ``_FillValue`` or ``missing_value`` is found, NaN in data will be automatically replaced by the missing value.
+   Write the values contained in *data* in the variable *varname* in a NetCDF file. If *varname* contains a path to a group, the variable will be created in the specified group, but in that case the group has to be created before. An instance of :class:`~.EgadsData` must be passed into ``write_variable``. Any attributes that are contained within the :class:`~.EgadsData` instance are applied to the NetCDF variable as well. If an attribute with a name equal to ``_FillValue`` or ``missing_value`` is found, NaN in data will be automatically replaced by the missing value. If attributes with the name ``scale_factor`` and/or ``add_offset`` are found, those attributes are automatically applied to the data.
 
    :param data: values to be stored in the NetCDF file
    :type data: EgadsData|array|vector|scalar
@@ -185,6 +185,33 @@ Data can be output to variables using the ``write_variable()`` function as follo
    >>> f.write_variable(data, 'test1/data/particle_size', ('time', ))
 
 
+If using :class:`~.NetCdf`, data can be output to variables using the ``write_variable()`` function as follows:
+
+    >>> f.write_variable(data, varname, dims, ftype, fillvalue, scale_factor, add_offset)
+
+.. function:: f.write_variable(data, varname[, dims=None, ftype='double', fillvalue=None, scale_factor=None, add_offset=None])
+   
+   Write the values contained in *data* in the variable *varname* in a NetCDF file. If *varname* contains a path to a group, the variable will be created in the specified group, but in that case the group has to be created before. Values for *data* passed into ``write_variable`` must be scalar or array.
+
+   :param data: values to be stored in the NetCDF file
+   :type data: EgadsData|array|vector|scalar
+   :param varname: the name of the variable, or the path of group + the name of the variable, in the NetCDF file
+   :type varname: string
+   :param dims: a tuple of dimension names for data (not needed if the variable already exists) ; optional
+   :type dims: tuple
+   :param ftype: the data type of the variable, the default value is *double*, other valid options are *float*, *int*, *short*, *char* and *byte* ; optional
+   :type ftype: string
+   :param fillvalue: if it is provided, it overrides the default NetCDF _FillValue ; optional, it doesn't exist if using :class:`~.EgadsNetCdf`
+   :type fillvalue: float|int
+   :param scale_factor: if data must be scaled, use this parameter ; optional
+   :type scale_factor: float|int
+   :param add_offset: if an offset must be added to data, use this parameter ; optional
+   :type add_offset: float|int
+
+   >>> f.write_variable(data, 'particle_size', ('time', ))
+   >>> f.write_variable(data, 'test1/data/particle_size', ('time', ), scale_factor=1.256, add_offset=5.56)
+
+
 Conversion from NetCDF to NASA Ames file format
 ------------------------------------------------
 

egads/_version.py

@@ -1,2 +1,2 @@
-__version__ = '1.2.7b'
+__version__ = '1.2.8'
 __branch__ = 'Lineage'

egads/input/netcdf_io.py

@@ -1,6 +1,6 @@
 __author__ = "mfreer, ohenry"
 __date__ = "2016-12-6 15:47"
-__version__ = "1.25"
+__version__ = "1.26"
 __all__ = ["NetCdf", "EgadsNetCdf"]
 
 import logging
@@ -177,7 +177,8 @@ def change_variable_name(self, varname, newname):
                       + ', newname ' + str(newname))
         self._change_variable_name(varname, newname)
 
-    def write_variable(self, data, varname, dims=None, ftype='double', fillvalue=None):
+    def write_variable(self, data, varname, dims=None, ftype='double', fillvalue=None, scale_factor=None,
+                       add_offset=None):
         """
         Writes/creates variable in currently opened NetCDF file.
 
@@ -195,11 +196,15 @@ def write_variable(self, data, varname, dims=None, ftype='double', fillvalue=Non
             ``short``, ``char``, and ``byte``
         :param float fillvalue:
             Optional - Overrides default NetCDF _FillValue, if provided.
+        :param float scale_factor:
+            Optional - If data must be scaled, use this parameter.
+        :param float add_offset:
+            Optional - If an offset must be added to data, use this parameter.
         """
 
         logging.debug('egads - netcdf_io.py - NetCdf - write_variable - varname ' + str(varname) +
                       ', dims ' + str(dims) + ', ftype ' + str(ftype) + ', fillvalue ' + str(fillvalue))
-        self._write_variable(data, varname, dims, ftype, fillvalue)
+        self._write_variable(data, varname, dims, ftype, fillvalue, scale_factor, add_offset)
 
     def add_dim(self, name, size):
         """
@@ -674,7 +679,7 @@ def _read_variable(self, varname, input_range, read_as_float, replace_fill_value
         logging.debug('egads - netcdf_io.py - NetCdf - read_variable - varname ' + str(varname) + ' -> data read OK')
         return value
 
-    def _write_variable(self, data, varname, dims, ftype, fillvalue):
+    def _write_variable(self, data, varname, dims, ftype, fillvalue, scale_factor, add_offset):
         """
         Private method to write/create variable in currently opened NetCDF file.
         """
@@ -694,6 +699,12 @@ def _write_variable(self, data, varname, dims, ftype, fillvalue):
                 varout = orig_group.createVariable(varname, self.TYPE_DICT[ftype], dims, fill_value=fillvalue)
             except KeyError:
                 varout = orig_group.createVariable(varname, ftype, dims, fill_value=fillvalue)
+
+            if scale_factor is not None:
+                setattr(varout, 'scale_factor', scale_factor)
+            if add_offset is not None:
+                setattr(varout, 'add_offset', add_offset)
+
             varout[:] = data
         else:
             logging.error('egads - netcdf_io.py - NetCdf - _write_variable - AttributeError, no file open')
@@ -1362,10 +1373,7 @@ def _write_variable(self, data, varname, dims, ftype):
                     except KeyError:
                         pass
                 varout = orig_group.createVariable(varname, self.TYPE_DICT[ftype.lower()], dims, fill_value=fillvalue)
-            if fillvalue is not None:
-                varout[:] = numpy.where(numpy.isnan(data.value), fillvalue, data.value)
-            else:
-                varout[:] = data.value
+
             for key, val in data.metadata.items():
                 if key != '_FillValue':
                     if isinstance(val, list):
@@ -1375,6 +1383,12 @@ def _write_variable(self, data, varname, dims, ftype):
                         setattr(varout, str(key), tmp[:-2])
                     else:
                         setattr(varout, str(key), val)
+
+            if fillvalue is not None:
+                varout[:] = numpy.where(numpy.isnan(data.value), fillvalue, data.value)
+            else:
+                varout[:] = data.value
+
         logging.debug('egads - netcdf_io.py - EgadsNetCdf - _write_variable - varname ' + str(varname)
                       + ' -> data write OK')
 

egads_lineage.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 1.2
 Name: egads-lineage
-Version: 1.2.7
+Version: 1.2.8
 Summary: EGADS: EUFAR General Airborne Data-processing Software
 Home-page: https://github.com/EUFAR/egads/tree/Lineage
 Author: EUFAR, Olivier Henry

egads_lineage.egg-info/SOURCES.txt

@@ -3,8 +3,8 @@ MANIFEST.in
 README.md
 setup.cfg
 setup.py
-Documentation/EGADS Lineage Algorithm Handbook - v1.2.7.pdf
-Documentation/EGADS Lineage Documentation - v1.2.7.pdf
+Documentation/EGADS Lineage Algorithm Handbook - v1.2.8.pdf
+Documentation/EGADS Lineage Documentation - v1.2.8.pdf
 Documentation/changelog.txt
 Documentation/EGADS Algorithm Handbook - LATEX/EGADS_alg_doc.tex
 Documentation/EGADS Algorithm Handbook - LATEX/example.py

setup.py

@@ -30,7 +30,7 @@
 doclines = __doc__.split('\n')
 
 setup(name='egads-lineage',
-      version='1.2.7',
+      version='1.2.8',
       description=doclines[0],
       long_description='\n'.join(doclines[2:]),
       author='EUFAR, Olivier Henry',