Improve unit tests

[barbuz]

The unit tests included in the project are good, but could be improved. We should have at least one test function targeting each function implemented in the code. If this is tricky to manage for some functions we should consider refactoring them so that each part is easier to test. This is particularly important when we want to do changes to the code, so we can be sure that the new version will work as well as the old one (integration tests can reveal many issues but unit tests make it easier to check that every part of the code has been tested and to find out the source of errors when they arise).

Coverage (generated by coverage.py):

Name                                                                     Stmts   Miss  Cover
--------------------------------------------------------------------------------------------
aodn_cloud_optimised/__init__.py                                             3      0   100%
aodn_cloud_optimised/bin/__init__.py                                         0      0   100%
aodn_cloud_optimised/bin/create_aws_registry_dataset.py                    144     60    58%
aodn_cloud_optimised/bin/generic_cloud_optimised_creation.py                31      2    94%
aodn_cloud_optimised/lib/ArgoHandler.py                                     38      2    95%
aodn_cloud_optimised/lib/CommonHandler.py                                  175     51    71%
aodn_cloud_optimised/lib/GenericParquetHandler.py                          347     64    82%
aodn_cloud_optimised/lib/GenericZarrHandler.py                             193     70    64%
aodn_cloud_optimised/lib/ParquetDataQuery.py                               243    105    57%
aodn_cloud_optimised/lib/__init__.py                                         0      0   100%
aodn_cloud_optimised/lib/config.py                                          47      9    81%
aodn_cloud_optimised/lib/logging.py                                         41      3    93%
aodn_cloud_optimised/lib/s3Tools.py                                         64      9    86%
aodn_cloud_optimised/lib/schema.py                                         144     22    85%
integration_testing/__init__.py                                              0      0   100%
integration_testing/test_ardc_wave_nrt.py                                   32     13    59%
test_aodn_cloud_optimised/__init__.py                                        0      0   100%
test_aodn_cloud_optimised/test_bin_generic.py                               65      1    98%
test_aodn_cloud_optimised/test_config.py                                    43      1    98%
test_aodn_cloud_optimised/test_create_aws_registry.py                       34      1    97%
test_aodn_cloud_optimised/test_generic_parquet_handler.py                  173      1    99%
test_aodn_cloud_optimised/test_generic_zarr_handler.py                      88      3    97%
test_aodn_cloud_optimised/test_generic_zarr_handler_netcdf3_netcdf4.py      74      3    96%
test_aodn_cloud_optimised/test_logging.py                                   25      1    96%
test_aodn_cloud_optimised/test_parquet_argo_handler.py                      77      1    99%
test_aodn_cloud_optimised/test_parquet_queries.py                           73      1    99%
test_aodn_cloud_optimised/test_s3tools.py                                   54      4    93%
test_aodn_cloud_optimised/test_schema.py                                    65      1    98%
--------------------------------------------------------------------------------------------
TOTAL                                                                     2273    428    81%

The coverage is good in several files, but we should try to reach at least 80% in each of them (100% would be ideal, but better start with realistic targets).

Files to look at:

• create_aws_registry_dataset
• CommonHandler.py
• GenericZarrHandler.py
• ParquetDataQuery.py
• test_ardc_wave_nrt.py (test fails)

Some will need more tests to be implemented, others have unused code that should be removed or put to use.