qcodes v0.40.0 breaks pytest

[grothe6]

With qcodes v0.40.0, our CI (pytest) does not work any more.
PR #54 fixes this temporarily by restricting the qcodes version.

GH actions gives this error message:

Run coverage run -m pytest --junit-xml=test-results.xml src/tests
============================= test session starts ==============================
platform linux -- Python 3.11.5, pytest-7.4.2, pluggy-1.3.0
PyQt5 5.15.9 -- Qt runtime 5.15.2 -- Qt compiled 5.15.2
rootdir: /home/runner/work/QuMADA/QuMADA
configfile: pyproject.toml
plugins: cases-3.6.14, mock-3.11.1, xvfb-3.0.0, qt-4.2.0
collected 6 items / 1 error

==================================== ERRORS ====================================
__________________ ERROR collecting src/tests/mapping_test.py __________________
ImportError while importing test module '/home/runner/work/QuMADA/QuMADA/src/tests/mapping_test.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
src/tests/mapping_test.py:44: in <module>
    from qumada.measurement.scripts.generic_measurement import Generic_1D_Sweep
src/qumada/measurement/scripts/__init__.py:21: in <module>
    from .generic_measurement import (
src/qumada/measurement/scripts/generic_measurement.py:33: in <module>
    from qumada.measurement.doNd_enhanced.doNd_enhanced import (
src/qumada/measurement/doNd_enhanced/doNd_enhanced.py:44: in <module>
    from qcodes.dataset.dond.do_nd_utils import (
E   ImportError: cannot import name '_catch_interrupts' from 'qcodes.dataset.dond.do_nd_utils' (/opt/hostedtoolcache/Python/3.11.5/x64/lib/python3.11/site-packages/qcodes/dataset/dond/do_nd_utils.py)
----- generated xml file: /home/runner/work/QuMADA/QuMADA/test-results.xml -----
=========================== short test summary info ============================
ERROR src/tests/mapping_test.py
!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
=============================== 1 error in 2.14s ===============================
Error: Process completed with exit code 2.

[grothe6]

For qcodes 0.40.0 in qcodes.dataset.dond.do_nd_utils, _catch_interrupts gets renamed to catch_interrupts. This is imported and used in doNd_enhanced.py.

We could check the used qcodes version when importing or just require qcodes 0.40.0.

[grothe6]

Changelog from QCoDeS for 0.38.0 - 0.40.0:

QCoDeS 0.38.0 (2023-04-25)

Changelog

Breaking Changes:

• The modules qcodes_loop.data, qcodes_loop.plots, qcodes_loop.actions, qcodes_loop.loops,
  qcodes_loop.measure, qcodes_loop.extensions.slack and qcodes_loop.utils.magic,
  qcodes_loop.utils.qt_helpers have been split out into a separate package qcodes_loop.
  The respective modules in QCoDeS do still exist as deprecated aliases to the new modules in qcodes_loop
  but will be removed in a future release. To use the aliases QCoDeS must be installed with the loop extra e.g.
  you should install pip install qcodes[loop]. If you make use of these modules we recommend updating imports
  to use qcodes_loop as soon as possible. See the readme of
  qcodes_loop for more information.

  The functions qcodes.utils.helpers.tprint qcodes.utils.helpers.wait_secs and
  qcodes.utils.helpers.foreground_qt_window have been removed.
  These helper functions are only used in qcodes_loop and were moved there. (Deprecate qcodes.data plot loops actions and measure microsoft/Qcodes#4971)

• The default configuration of QCoDeS dataset GUID_components.GUID_type has changed from explicit_sample to random_sample.
  This means that QCoDeS GUIDs will no longer start with aaaaaaaa but with a random string. This significantly reduces the risk
  of duplicate GUIDs.
  This also means that that the ability to set a sample_id as part of the dataset GUID is disabled by default. With
  the default config its therefor an error to set GUID_components.sample to anything but the default value. The original
  behavior can be restored by setting GUID_components.GUID_type to explicit_sample in the config file but is not recommended.
  Users should make use of the sample_name attached to an experiment as an alternative. Note that if you have already created
  a local config you will need to update this for the changes to take effect. (Dataset: change default guid type to use a random str microsoft/Qcodes#5031)

Improved:

• The helper functions guids_from_dbs, guids_from_dir, and guids_from_list_str have been added
  to the documented dataset api at qcodes.dataset. (Add add guid helpers to the official api microsoft/Qcodes#4989)

Improved Drivers:

• Fixed a bug in version parsing for Keysight 344XXA and Keysight E4980A which
  would result in incompatibility with packaging>21 under certain circumstances. (Parsing version string error with Keysight_34465A microsoft/Qcodes#5007)
• Fixed a bug in interlock status querying for Keithley 3706A. Originally, not all
  potential responses from the system were accounted for when querying for interlock
  status. A dictionary is used to map the response from the system to a string describing
  the interlock status. When the system returns a response that was not accounted for, this
  resulted in a KeyError being raised. Now, this dictionary accounts for all potential responses
  from the system. (fix(Keithley3706A): Fix interlock status dictionary microsoft/Qcodes#5013)
• Added support for AimTTi QL355TP power supply.
  Moved _numOutputChannels lookup table to the class body. (Add support for AimTTi QL355TP microsoft/Qcodes#5021)
• Increased range on setpoint property of Lakeshore Model 335,336,372 to support degree Celsius (add driver for Lakeshore Model 335 microsoft/Qcodes#5043)
• Fix auto_sweep in Keysight PNA N52xx driver
  Fix units of polar sweeps in Keysight PNA N52xx driver (Fix auto_sweep in Keysight N52xx Driver microsoft/Qcodes#5090)

New:

• The QCoDeS Station Instrument and InstrumentModule classes gained a method get_component
  which allows the user to get a component by name. Allowing the user to go from
  the full name of a component to the component itself. (Add ability to lookup component in station by full name microsoft/Qcodes#5028)

New Drivers:

• Add driver for Lakeshore Model 335 (add driver for Lakeshore Model 335 microsoft/Qcodes#5043)

QCoDeS 0.39.0 (2023-06-20)

Changelog

Note that this will be the last version to support Python 3.8. Please upgrade to at least
python 3.9.

Improved:

• The in memory cache of a dataset can now be configured both from the qcodesrc.json config file and when using dond to perform measurements.
  The Exception raised when a dond, do1d etc. measurement is interrupted (BreakConditionInterrupt) has been made public as part of the
  qcodes.dataset module so it can be intercepted and handled by the user. (Make in memory cache configurable microsoft/Qcodes#5200)
• When writing a DataSetInMem back to a database the exp_id, counter and run_id are correctly updated
  to reflect the sate of the new database. write_metadata_to_db has also been fixed to use
  the database passed to init of the DataSetInMem class if no path is provided. (Update run_id when adding ds in mem to db microsoft/Qcodes#5209)

Improved Drivers:

• Fix Keithley 3706A driver to use the updated interlock status strings
  (following up on Parsing version string error with Keysight_34465A microsoft/Qcodes#5007) (Fix Keithley 3706A driver to refer to the new interlock status string microsoft/Qcodes#5147)
• Increase default timeout on R&S ZNB and allow the instrument to overwrite default timeout (Change R&S ZNB timeout calculation microsoft/Qcodes#5201)

QCoDeS 0.40.0 (2023-09-14)

Changelog

Improved:

• Very noisy log messages from azure.monitor.opentelemetry.exporter are now
  by default filtered and not shown in the console logger. This matches the
  behavior of the OpenCensus exporter. (Silence log messages related to opentelemetry upload to azure microsoft/Qcodes#5278)
• The InstrumentLoggerAdapter has been updated to store the instrument_name and instrument_type
  as fields on log records rather than the instrument it self. This enables opentelemetry to attache
  the fields to a transmitted LogRecord for better filtering. (InstrumentLoggerAdaptor store instrument_name and type in record microsoft/Qcodes#5297)
• In the QCoDeS DataSetProtocol newly created dataset now always have a captured_run_id / captured_counter
  that matches the run_id / result_counter that they are assigned on creation. Previously these could be
  out of sync if datasets measured elsewhere had been inserted into the database. (Calculate new run_id/counter from run_id/counter microsoft/Qcodes#5329)
• The pyvisa.ResourceManager of a VISA instrument is now exposed
  as instr.resource_manager. All VISA instruments will now use weakref.finalize
  to close the visa resource on shutdown of the instrument. This should reduce the
  chance that an instrument resource is not cleanup correctly on exit. (Use weakref finalize to close visa connections microsoft/Qcodes#5341)

Improved Drivers:

• Parameters controlling the pid-values, pumps and state of the temperature channels have been added to OxfordTriton.
  This allows temperature sweeps from base temperature to several Kelvin to be fully automatic. Code to create such sweeps can be found at https://github.com/qdev-dk/MeasFunc/blob/main/measfunc/temperature_sweep.py. (Oxford Triton: Add control of pumps and temperature channels microsoft/Qcodes#5316)

New:

• The QCoDeS Measurement Context manager, DataSaverBuilder and DoND functions have been instrumented as OpenTelemetry traces.
  This enables users to correlate log messages with the the measurement that they were performed within.
  See the OpenTelemetry documentation for examples of how this can be used.

  The log exporting using OpenCensus within QCoDeS is expected to be deprecated and eventually removed in line
  with OpenCensus being discontinued. Users that are interested in gathering telemetry of their setups are encouraged
  to provide their own solution based on OpenTelemetry. (Instrument Measurement and dond with opentelemetry microsoft/Qcodes#5289)

• New features: datasaver_builder and dond_into are intermediate measurement extensions designed to fill a gap between
  the low-level Measurement object and the high-level doNd functions. They allow convenient specification of parameter
  dependencies for registration and doNd-like syntax while also allowing direct access to the underlying datasaver objects. (Add datasaver_builder and dond_core microsoft/Qcodes#5294)

[grothe6]

@THuckemann
I don't see anything immediately, that could break qumada in the new QCoDeS versions, do you?

There are some new features, that could benefit the metadata acquisition (namely guid helpers from 0.38.0)
Also, also the datasaver_builder and/or dond_into from 0.40.0 could be interesting to look at.

The in memory cache of a dataset can now be configured both from the qcodesrc.json config file and when using dond to perform measurements.
The Exception raised when a dond, do1d etc. measurement is interrupted (BreakConditionInterrupt) has been made public as part of the
qcodes.dataset module so it can be intercepted and handled by the user.

Could this be beneficial in any way?