Skip to content

Commit

Permalink
update README
Browse files Browse the repository at this point in the history
  • Loading branch information
magland committed Apr 22, 2024
1 parent f8e69a5 commit 36a5e4f
Showing 1 changed file with 30 additions and 13 deletions.
43 changes: 30 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,39 +35,56 @@ pip install -e .

## Use cases

* Lazy-load a remote NWB/HDF5 file for efficient access to metadata and data.
* Represent a remote NWB/HDF5 file as a .nwb.lindi.json file.
* Read a local or remote .nwb.lindi.json file using pynwb or other tools.
* Edit a .nwb.lindi.json file using pynwb or other tools.
* Add datasets to a .nwb.lindi.json file using a local staging area.
* Upload a .nwb.lindi.json file to a cloud storage service such as DANDI.
* Upload a .nwb.lindi.json file with staged datasets to a cloud storage service such as DANDI.

### Lazy-load a remote NWB/HDF5 file for efficient access to metadata and data

```python
import pynwb
import lindi

# URL of the remote NWB file
h5_url = "https://api.dandiarchive.org/api/assets/11f512ba-5bcf-4230-a8cb-dc8d36db38cb/download/"

# Set up a local cache
local_cache = lindi.LocalCache(cache_dir='lindi_cache')

# Create the h5py-like client
client = lindi.LindiH5pyFile.from_hdf5_file(h5_url, local_cache=local_cache)

# Open using pynwb
with pynwb.NWBHDF5IO(file=client, mode="r") as io:
nwbfile = io.read()
print(nwbfile)

# The downloaded data will be cached locally, so subsequent reads will be faster
```

### Represent a remote NWB/HDF5 file as a .nwb.lindi.json file

```python
import json
import pynwb
import lindi

# URL of the remote NWB file
h5_url = "https://api.dandiarchive.org/api/assets/11f512ba-5bcf-4230-a8cb-dc8d36db38cb/download/"

# Create a read-only Zarr store as a wrapper for the h5 file
store = lindi.LindiH5ZarrStore.from_file(h5_url)
# Create the h5py-like client
client = lindi.LindiH5pyFile.from_hdf5_file(h5_url)

# Generate a reference file system
rfs = store.to_reference_file_system()
rfs = client.to_reference_file_system()

# Save it to a file for later use
with open("example.lindi.json", "w") as f:
json.dump(rfs, f, indent=2)

# Create an h5py-like client from the reference file system
client = lindi.LindiH5pyFile.from_reference_file_system(rfs)

# Open using pynwb
with pynwb.NWBHDF5IO(file=client, mode="r") as io:
nwbfile = io.read()
print(nwbfile)
# See the next example for how to read this file
```

### Read a local or remote .nwb.lindi.json file using pynwb or other tools
Expand Down Expand Up @@ -130,7 +147,7 @@ with lindi.StagingArea.create(base_dir='lindi_staging') as staging_area:
# upload the changes to the remote .nwb.lindi.json file
```

### Upload a .nwb.lindi.json file to a cloud storage service such as DANDI
### Upload a .nwb.lindi.json file with staged datasets to a cloud storage service such as DANDI

See [this example](https://github.com/magland/lindi-dandi/blob/main/devel/lindi_test_2.py).

Expand Down

0 comments on commit 36a5e4f

Please sign in to comment.