Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add aeon_api #102

Draft
wants to merge 5 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion .gitmodules
Original file line number Diff line number Diff line change
@@ -1,7 +1,11 @@
[submodule "aeon_mecha"]
path = aeon_mecha
url = https://github.com/SainsburyWellcomeCentre/aeon_mecha
branch = data-preview
branch = sfn2024
[submodule "aeon_api"]
path = aeon_api
url = https://github.com/SainsburyWellcomeCentre/aeon_api.git
branch = main
[submodule "aeon_experiments"]
path = aeon_experiments
url = https://github.com/SainsburyWellcomeCentre/aeon_experiments
Expand Down
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ help:

$(SOURCEDIR)/reference/api.rst:
@echo "Generating API documentation..."
@python make_mecha_doctree.py
@python make_api_doctree.py
@python make_acquisition_doctree.py

copy-examples:
Expand Down
1 change: 1 addition & 0 deletions aeon_api
Submodule aeon_api added at 2c5955
2 changes: 1 addition & 1 deletion make.bat
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ if "%1" == "" goto help

if not "%1" == "clean" (
@echo Building API docs...
python make_mecha_doctree.py
python make_api_doctree.py
python make_acquisition_doctree.py

@echo Copying example notebooks into 'src'...
Expand Down
12 changes: 4 additions & 8 deletions make_mecha_doctree.py → make_api_doctree.py
Original file line number Diff line number Diff line change
@@ -1,20 +1,16 @@
import os
from pathlib import Path


# modules to ignore in the api doc
ignore_modules = [
"aeon.dj_pipeline",
"aeon.qc.video",
]
ignore_modules = []


def make_mecha_doctree():
def make_api_doctree():
"""
Create a doctree of all modules in aeon_mecha/aeon.
"""
doctree = ""
api_path = Path("aeon_mecha") / "aeon"
api_path = Path("aeon_api") / "swc" / "aeon"
for path in sorted(api_path.rglob("*.py")):
if path.name.startswith("_"):
continue
Expand All @@ -36,4 +32,4 @@ def make_mecha_doctree():


if __name__ == "__main__":
make_mecha_doctree()
make_api_doctree()
10 changes: 9 additions & 1 deletion readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,12 +28,20 @@ Below are the required sets of credentials for Aeon members:
To be granted these credentials, please send a single email to [all contact parties](mailto:[email protected],[email protected],[email protected],[email protected],[email protected]?subject=Request%20for%20Aeon%20credentials) requesting this access.

## Repositories

### [aeon_api](https://github.com/SainsburyWellcomeCentre/aeon_api)

![aeon_api_env_build_and_tests](https://github.com/SainsburyWellcomeCentre/aeon_api/actions/workflows/build_env_run_tests.yml/badge.svg?branch=main)
[![aeon_api_tests_code_coverage](https://codecov.io/gh/SainsburyWellcomeCentre/aeon_api/branch/main/graph/badge.svg?token=973EC1CG03)](https://codecov.io/gh/SainsburyWellcomeCentre/aeon_api)

Aeon's low-level library for interfacing with acquired data. Contains Python modules for raw data file io, data querying, and data processing.

### [aeon_mecha](https://github.com/SainsburyWellcomeCentre/aeon_mecha)

![aeon_mecha_env_build_and_tests](https://github.com/SainsburyWellcomeCentre/aeon_mecha/actions/workflows/build_env_run_tests.yml/badge.svg?branch=main)
[![aeon_mecha_tests_code_coverage](https://codecov.io/gh/SainsburyWellcomeCentre/aeon_mecha/branch/main/graph/badge.svg?token=973EC1CG03)](https://codecov.io/gh/SainsburyWellcomeCentre/aeon_mecha)

Aeon's main library for interfacing with acquired data. Contains Python modules for raw data file io, data querying, data processing, data qc, database ingestion, and building computational data pipelines. This is the main user repository.
Aeon's library for the Aeon DataJoint pipeline. Contains Python modules for Aeon experiment schema definitions, data quality control, data processing, and data ingestion into a DataJoint MySQL database.

### [aeon_experiments](https://github.com/SainsburyWellcomeCentre/aeon_experiments)

Expand Down
8 changes: 4 additions & 4 deletions src/_templates/api_mecha_head.rst
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
..
This file is auto-generated.

.. _target-mecha-reference:
.. _target-aeon-api-reference:

``aeon_mecha``
``aeon_api``
==============

.. toctree::
:caption: aeon_mecha API
:caption: aeon API
:maxdepth: 1

.. autosummary::
:toctree: mecha
:toctree: api

6 changes: 4 additions & 2 deletions src/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
[
os.path.abspath("."),
os.path.abspath("../aeon_mecha/"),
os.path.abspath("../aeon_api/"),
os.path.abspath("../aeon_acquisition"),
os.path.abspath("../aeon_experiments"),
]
Expand Down Expand Up @@ -68,7 +69,7 @@ def get_current_release_tag():
"myst_nb",
"sphinx_design",
"sphinx_copybutton",
"convertworkflow"
"convertworkflow",
]

# Configure the myst parser to enable cool markdown features
Expand Down Expand Up @@ -146,7 +147,7 @@ def get_current_release_tag():
"css/custom.css",
]
html_js_files = [
'js/workflow.js', # javascript for embedded workflows
"js/workflow.js", # javascript for embedded workflows
]

# linkcheck will skip checking these URLs entirely
Expand Down Expand Up @@ -175,6 +176,7 @@ def get_current_release_tag():
"aeon-experiments-github": "https://github.com/SainsburyWellcomeCentre/aeon_experiments/{{path}}",
"aeon-lineardrive-github": "https://github.com/SainsburyWellcomeCentre/aeon_lineardrive/{{path}}",
"aeon-feeder-github": "https://github.com/SainsburyWellcomeCentre/aeon_feeder/{{path}}",
"aeon-api-github": "https://github.com/SainsburyWellcomeCentre/aeon_api/{{path}}",
"myst-parser": "https://myst-parser.readthedocs.io/en/latest/{{path}}#{{fragment}}",
"semver": "https://semver.org/",
"harp-tech": "https://harp-tech.org/{{path}}#{{fragment}}",
Expand Down
6 changes: 3 additions & 3 deletions src/getting_started/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,8 +41,8 @@ As each of these different components is built using different languages, spanni
::::{grid} 1 2 2 2
:gutter: 3

:::{grid-item-card} {fas}`database;sd-text-primary` aeon_mecha
:link: https://github.com/SainsburyWellcomeCentre/aeon_mecha
:::{grid-item-card} {fas}`database;sd-text-primary` aeon_api
:link: https://github.com/SainsburyWellcomeCentre/aeon_api
:link-type: url
Aeon's main library for interfacing with acquired data
:::
Expand Down Expand Up @@ -76,7 +76,7 @@ Source code for pellet delivery via feeders used in Aeon experiments
:::{note}
All experiment data is acquired and/or triggered and/or synced by
[Harp devices](https://www.cf-hw.org/harp).
Code in the `aeon_acquisition` and `aeon_mecha` repositories makes use of
Code in the `aeon_acquisition` and `aeon_api` repositories makes use of
the [Harp protocol](harp-tech:articles/about) during data acquisition,
raw data file writing, and raw data file reading.
See also the documentation on
Expand Down
16 changes: 8 additions & 8 deletions src/getting_started/installation.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
# Installation

This guide will walk you through the installation of the `aeon_mecha` package, the main package for interacting with raw Aeon data. It also includes instructions for setting up data access to the Aeon data hosted on SWC's Ceph storage (currently only applicable to SWC members).
This guide will walk you through the installation of the `aeon_api` package, the main package for interacting with raw Aeon data. It also includes instructions for setting up data access to the Aeon data hosted on SWC's Ceph storage (currently only applicable to SWC members).

## `aeon_mecha`
## `aeon_api`

:::{note}
We always recommend installing `aeon_mecha` inside a
We always recommend installing `aeon_api` inside a
[conda](https://docs.conda.io/en/latest/)
or [mamba](https://mamba.readthedocs.io/en/latest/) environment,
to avoid dependency conflicts with other packages.
Expand Down Expand Up @@ -40,12 +40,12 @@ export PATH=$PATH:/ceph/aeon/aeon/code/dotnet
```
:::

Clone the `aeon_mecha` repository into the `~/ProjectAeon/aeon_mecha` directory.
Clone the `aeon_api` repository into the `~/ProjectAeon/aeon_api` directory.
```sh
mkdir ~/ProjectAeon
cd ~/ProjectAeon
git clone https://github.com/SainsburyWellcomeCentre/aeon_mecha
cd aeon_mecha
git clone https://github.com/SainsburyWellcomeCentre/aeon_api
cd aeon_api
```

Create the `aeon` conda environment and activate it.
Expand All @@ -54,7 +54,7 @@ conda create -n aeon -c conda-forge python>=3.11
conda activate aeon
```

Install the `aeon_mecha` package in editable mode.
Install the `aeon_api` package in editable mode.
::::{tab-set}
:::{tab-item} Users
```sh
Expand Down Expand Up @@ -108,4 +108,4 @@ The next step is to configure your IDE to connect to the `swc-gateway` node via
::::
:::::

Within the `aeon` conda environment, Aeon data can be accessed directly from Ceph or the DataJoint database using the [`aeon_mecha` API](target-mecha-reference). Examples for retrieving and visualizing the data can be found in the [User Guide](target-user-guide).
Within the `aeon` conda environment, Aeon data can be accessed directly from Ceph or the DataJoint database using the [`aeon_api` API](target-aeon-api-reference). Examples for retrieving and visualizing the data can be found in the [User Guide](target-user-guide).
4 changes: 2 additions & 2 deletions src/reference/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,13 @@
:gutter: 3

:::{grid-item-card} Data API Reference
:link: target-mecha-reference
:link: target-aeon-api-reference
:link-type: ref
:img-bottom: ../images/api-python-logo.png
:class-img-bottom: sd-px-2
:class-card: sd-height-25

This reference guide contains a detailed description of the `aeon_mecha` low-level API.
This reference guide contains a detailed description of `aeon_api`, Aeon's low-level API.
:::

:::{grid-item-card} Acquisition API Reference
Expand Down