[Bug]: No module named 'hdmf.array' in ConverterPipe

[laurelrr]

What happened?

I am attempting to convert raw files collected as spikeglx and sorted with phy using the tutorial outlined here: https://neuroconv.readthedocs.io/en/main/conversion_examples_gallery/combinations/spikeglx_and_phy.html
Looks like it is failing to import ConverterPipe although I did include pip install "neuroconv[converterpipe]" in my environment.

Steps to Reproduce

from zoneinfo import ZoneInfo
from pathlib import Path
from neuroconv import ConverterPipe
from neuroconv.datainterfaces import SpikeGLXRecordingInterface, PhySortingInterface
import os

# to run this script first create a conda environment for pynwb
#   conda create -n pynwb python=3.12
#   conda activate pynwb
#   pip install -U pynwb
#
# then to run this script run:
#   pip install "neuroconv[spikeglx]"
#   pip install "neuroconv[phy]"
#   pip install "neuroconv[converterpipe]"

# 
# then update the neuropixels_data_path to point to your imec0.ap file
# then run this line at the terminal:
#   python convert_spikeglx_phy.py


# For this interface we need to pass the location of the ``.bin`` file. Change the file_path to the location in your system

# 🔹 Step 1: Define Paths

nwbfile_path = "/snlkt/laurel/for_others/Anousheh/nwb/20220906_Rew4_Pix_None_4470R_raw_processed.nwb"

raw_folder_path = f"/snlkt/tryptateam/Psilicon/_Raw/neuropix_raw/20220906_Rew4_Pix_None_4470R_4472L_M_RL_g0/20220906_Rew4_Pix_None_4470R_4472L_M_RL_g0_imec0"
if os.path.exists(raw_data_path):
    print("file exists!")

interface_spikeglx = SpikeGLXRecordingInterface(folder_path=raw_folder_path, stream_id="imec0.ap", verbose=False)
print(interface_spikeglx)

phy_folder_path = f"/snlkt/tryptateam/Psilicon/_Processed/neuropix_ecephys_physorted/20220906_Rew4_Pix_None_4470R_M_RL/imec0_ks2"
if os.path.exists(phy_data_path):
    print("file exists!")
interface_phy = PhySortingInterface(folder_path=phy_folder_path, verbose=False)
print(interface_phy)

 # Now that we have defined the two interfaces we pass them to the ConverterPipe which will coordinate the
 # concurrent conversion of the data
converter = ConverterPipe(data_interfaces=[interface_spikeglx, interface_phy], verbose=False)

# Extract what metadata we can from the source files
metadata = converter.get_metadata()
# For data provenance we add the time zone information to the conversion
session_start_time = metadata["NWBFile"]["session_start_time"].replace(tzinfo=ZoneInfo("US/Pacific"))
metadata["NWBFile"].update(session_start_time=session_start_time)

# Choose a path for saving the nwb file and run the conversion
converter.run_conversion(nwbfile_path=nwbfile_path, metadata=metadata)

Traceback

Traceback (most recent call last):
  File "/snlkt/laurel/for_others/Anousheh/nwb/convert_spikeglx_phy.py", line 4, in <module>
    from neuroconv import ConverterPipe
  File "/home/lkeyes/anaconda3/envs/pynwb/lib/python3.10/site-packages/neuroconv/__init__.py", line 1, in <module>
    from .basedatainterface import BaseDataInterface
  File "/home/lkeyes/anaconda3/envs/pynwb/lib/python3.10/site-packages/neuroconv/basedatainterface.py", line 11, in <module>
    from .tools.nwb_helpers import make_nwbfile_from_metadata, make_or_load_nwbfile
  File "/home/lkeyes/anaconda3/envs/pynwb/lib/python3.10/site-packages/neuroconv/tools/__init__.py", line 9, in <module>
    from .nwb_helpers import get_module
  File "/home/lkeyes/anaconda3/envs/pynwb/lib/python3.10/site-packages/neuroconv/tools/nwb_helpers/__init__.py", line 4, in <module>
    from hdmf_zarr import NWBZarrIO as _NWBZarrIO
  File "/home/lkeyes/anaconda3/envs/pynwb/lib/python3.10/site-packages/hdmf_zarr/__init__.py", line 1, in <module>
    from .backend import ZarrIO
  File "/home/lkeyes/anaconda3/envs/pynwb/lib/python3.10/site-packages/hdmf_zarr/backend.py", line 24, in <module>
    from .zarr_utils import BuilderZarrReferenceDataset, BuilderZarrTableDataset
  File "/home/lkeyes/anaconda3/envs/pynwb/lib/python3.10/site-packages/hdmf_zarr/zarr_utils.py", line 13, in <module>
    from hdmf.array import Array
ModuleNotFoundError: No module named 'hdmf.array'

Operating System

Windows

Python Executable

Conda

Python Version

3.10

Package Versions

environment_issue.txt

Code of Conduct

• I agree to follow this project's Code of Conduct
• Have you ensured this bug was not already reported?

[bendichter]

I found this too the other day, but was able to fix it by updating the packages pynwb, hdmf, and hdmf_zarr. Could you try that?

[laurelrr]

I was able to update those packages, although hdmf-zarr needed the actual version number before updating (pip install hdmf-zarr==0.11.0).

But now I am getting a new error message at this line:
interface_spikeglx = SpikeGLXRecordingInterface(folder_path=raw_folder_path, stream_id="imec0.ap", verbose=False)
where it does not recognize folder_path or stream_id:
TypeError: SpikeGLXRecordingInterface.__init__() got an unexpected keyword argument 'folder_path'.

If I remove the extra arguments, so the line is just SpikeGLXRecordingInterface(raw_folder_path), then the error is

Traceback (most recent call last):
  File "/snlkt/laurel/for_others/Anousheh/nwb/convert_spikeglx_phy.py", line 36, in <module>
    interface_spikeglx = SpikeGLXRecordingInterface(raw_folder_path)
  File "/home/lkeyes/anaconda3/envs/pynwb/lib/python3.10/site-packages/neuroconv/datainterfaces/ecephys/spikeglx/spikeglxdatainterface.py", line 50, in __init__
    self.stream_id = fetch_stream_id_for_spikelgx_file(file_path)
  File "/home/lkeyes/anaconda3/envs/pynwb/lib/python3.10/site-packages/neuroconv/datainterfaces/ecephys/spikeglx/spikeglx_utils.py", line 78, in fetch_stream_id_for_spikelgx_file
    device = next(suffix for suffix in suffixes if "imec" in suffix or "nidq" in suffix)
StopIteration

There is a file in that folder *.imec0.ap.bin, so I'm not sure how to specify to look for that if stream_id isn't included?

[bendichter]

This interface wants a "file_path" arg and it wants you to point to *.imec0.ap.bin