Update dependencies + Sub-class EntryCollection (#252)

Update dependencies:

* Update pre-commit requirement from ~=2.12 to ~=2.13
* Update optimade[mongo] requirement from ~=0.14.1 to ~=0.16.0
* Update uvicorn requirement from ~=0.13.4 to ~=0.14.0
* Update invoke requirement from ~=1.5 to ~=1.6
* Update pylint requirement from ~=2.8 to ~=2.9
* Update aiida-core requirement from ~=1.6.3 to ~=1.6.4

Update GH Actions:

* Bump codecov/codecov-action from 1.5.0 to 1.5.2

Update pre-commit hooks.

Add general --dev option to CLI.
It sets the AiiDA profile to `aiida-optimade_test` and forces the
`--debug` option for `aiida-optimade run`.

Update filtertransformer to v0.15 of `optimade`.
It now takes a mapper - so the tests related to this have been
un-skipped and the collection initialization has been fixed.

Move filter property resolution to mapper.
Add top-level "meta" to config file as a non-filterable property.

Re-set filter fields and handle special properties.
Provider-specific properties coming from unknown providers are now
handled "properly", meaning they are converted to
`"attributes.something.non.existing"`, which will similar to AiiDA
ignoring it, or rather returning `None`, which may result in some weird
results depending on how the filter is written.

Patch TOP_LEVEL_NON_ATTRIBUTES_FIELDS in mapper.

Make AiidaCollection sub-class EntryCollection:
Utilize the base `EntryCollection` from `optimade` and its methods.
Use `handle_response_fields()` from `optimade` - should be updated to
set `by_alias=True`.

Add AiidaCollections tests.

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

.ci/aiida-version.json

@@ -1,6 +1,6 @@
 {
   "schemaVersion": 1,
   "label": "AiiDA",
-  "message": "v1.6.3",
+  "message": "v1.6.4",
   "color": "brightgreen"
 }

.github/workflows/ci.yml

@@ -106,14 +106,14 @@ jobs:
 
     - name: Upload coverage to Codecov
       if: matrix.python-version == 3.8
-      uses: codecov/[email protected].0
+      uses: codecov/[email protected].2
       with:
         flags: aiida
         file: ./coverage.xml
 
     - name: Upload coverage to Codecov
       if: matrix.python-version == 3.8
-      uses: codecov/[email protected].0
+      uses: codecov/[email protected].2
       with:
         flags: mongo
         file: ./mongo_cov.xml

.pre-commit-config.yaml

@@ -1,14 +1,14 @@
 repos:
 - repo: https://github.com/pre-commit/pre-commit-hooks
-  rev: v3.4.0
+  rev: v4.0.1
   hooks:
   - id: trailing-whitespace
     exclude: README.md
   - id: check-yaml
   - id: check-json
 
 - repo: https://github.com/ambv/black
-  rev: 21.4b2
+  rev: 21.5b2
   hooks:
   - id: black
     name: Blacken
@@ -26,6 +26,6 @@ repos:
     language: system
 
 - repo: https://gitlab.com/pycqa/flake8
-  rev: '3.9.1'
+  rev: '3.9.2'
   hooks:
   - id: flake8

Dockerfile

@@ -3,8 +3,8 @@ FROM python:3.8
 WORKDIR /app
 
 # Install specific optimade and aiida-core versions
-ARG OPTIMADE_TOOLS_VERSION=0.14.1
-ARG AIIDA_VERSION=1.6.3
+ARG OPTIMADE_TOOLS_VERSION=0.16.0
+ARG AIIDA_VERSION=1.6.4
 
 # Copy repo contents
 COPY setup.py setup.json README.md requirements*.txt ./

README.md

@@ -2,7 +2,7 @@
 
 | Latest release | Build status | Activity |
 |:--------------:|:------------:|:--------:|
-| [![AiiDA](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/aiidateam/aiida-optimade/develop/.ci/aiida-version.json)](https://github.com/aiidateam/aiida-core/)<br>[![PyPI](https://img.shields.io/pypi/v/aiida-optimade)](https://pypi.org/project/aiida-optimade/)<br>[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/aiida-optimade)](https://pypi.org/project/aiida-optimade/)<br>[![OPTIMADE](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/Materials-Consortia/optimade-python-tools/v0.14.1/optimade-version.json)](https://github.com/Materials-Consortia/OPTIMADE/) | [![GitHub Workflow Status](https://img.shields.io/github/workflow/status/aiidateam/aiida-optimade/aiida-optimade)](https://github.com/aiidateam/aiida-optimade/actions/)<br>[![Codecov](https://img.shields.io/codecov/c/gh/aiidateam/aiida-optimade)](https://codecov.io/gh/aiidateam/aiida-optimade) | [![GitHub last commit](https://img.shields.io/github/last-commit/aiidateam/aiida-optimade)](https://github.com/aiidateam/aiida-optimade) |
+| [![AiiDA](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/aiidateam/aiida-optimade/develop/.ci/aiida-version.json)](https://github.com/aiidateam/aiida-core/)<br>[![PyPI](https://img.shields.io/pypi/v/aiida-optimade)](https://pypi.org/project/aiida-optimade/)<br>[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/aiida-optimade)](https://pypi.org/project/aiida-optimade/)<br>[![OPTIMADE](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/Materials-Consortia/optimade-python-tools/v0.16.0/optimade-version.json)](https://github.com/Materials-Consortia/OPTIMADE/) | [![GitHub Workflow Status](https://img.shields.io/github/workflow/status/aiidateam/aiida-optimade/aiida-optimade)](https://github.com/aiidateam/aiida-optimade/actions/)<br>[![Codecov](https://img.shields.io/codecov/c/gh/aiidateam/aiida-optimade)](https://codecov.io/gh/aiidateam/aiida-optimade) | [![GitHub last commit](https://img.shields.io/github/last-commit/aiidateam/aiida-optimade)](https://github.com/aiidateam/aiida-optimade) |
 
 This is a RESTful API server created with [FastAPI](https://fastapi.tiangolo.com/) that exposes an AiiDA database according to the [OPTIMADE specification](https://github.com/Materials-Consortia/OPTIMADE/blob/develop/optimade.rst).
 
@@ -59,7 +59,7 @@ For testing run `pytest`, which will run with an AiiDA backend as standard.
 The tests can also be run with the MongoDB backend by setting the environment variable `PYTEST_OPTIMADE_CONFIG_FILE`, the value being a path to the config file to be used:
 
 ```shell
-$ PYTEST_OPTIMADE_CONFIG_FILE=/path/to/aiida-optimade/tests/static/test_structures_mongo.json pytest
+$ PYTEST_OPTIMADE_CONFIG_FILE=/path/to/aiida-optimade/tests/static/test_mongo_config.json pytest
 ```
 
 ## Initialization
@@ -74,7 +74,7 @@ $ aiida-optimade -p <PROFILE> init
 
 Where `<PROFILE>` is the AiiDA profile.
 
-> **Note**: Currently, the default is `optimade_sqla`, if the `-p / --profile` option is now specified.
+> **Note**: Currently, the default is `optimade`, if the `-p / --profile` option is now specified.
 > This will be changed in the future to use the default AiiDA profile.
 
 Initialization goes through your profile's `StructureData` nodes, adding an `optimade` extra, wherein all OPTIMADE-specific fields that do not have an equivalent AiiDA property are stored.
@@ -94,7 +94,7 @@ $ aiida-optimade -p <PROFILE> run
 
 Where `<PROFILE>` is the AiiDA profile you wish to serve.
 
-> **Note**: Currently, the default is `optimade_sqla`, if the `-p / --profile` option is now specified.
+> **Note**: Currently, the default is `optimade`, if the `-p / --profile` option is now specified.
 > This will be changed in the future to use the default AiiDA profile.
 
 You also have the opportunity to specify the AiiDA profile via the environment variable `AIIDA_PROFILE`.

aiida_optimade/cli/cmd_aiida_optimade.py

@@ -5,9 +5,10 @@
 
 from aiida.cmdline.params.options import PROFILE as VERDI_PROFILE
 from aiida.cmdline.params.types import ProfileParamType as VerdiProfileParamType
-from aiida.manage.configuration import Profile
+from aiida.manage.configuration import get_config, Profile
 
 from aiida_optimade.cli.options import AIIDA_PROFILES
+from aiida_optimade.cli.utils import AIIDA_OPTIMADE_TEST_PROFILE
 
 
 @click.group(context_settings={"help_option_names": ["-h", "--help"]})
@@ -16,19 +17,31 @@
 )
 @VERDI_PROFILE(
     type=VerdiProfileParamType(),
-    default="optimade_sqla",
+    default="optimade",
     show_default=True,
     help="AiiDA profile to use and serve. Configured profiles: "
     f"{', '.join([repr(name) for name in AIIDA_PROFILES])}.",
 )
+@click.option(
+    "--dev",
+    is_flag=True,
+    default=False,
+    show_default=True,
+    help=f"Run in development mode (use the {AIIDA_OPTIMADE_TEST_PROFILE!r} AiiDA "
+    "profile and `--debug` options).",
+)
 @click.pass_context
-def cli(ctx, profile: Profile):  # pragma: no cover
+def cli(ctx, profile: Profile, dev: bool):  # pragma: no cover
     """AiiDA-OPTIMADE command line interface (CLI)."""
 
     if ctx.obj is None:
         ctx.obj = {}
 
+    if dev:
+        profile = get_config(create=True).get_profile(AIIDA_OPTIMADE_TEST_PROFILE)
+
     ctx.obj["profile"] = profile
+    ctx.obj["dev"] = dev
 
     # Set config
     if (

aiida_optimade/cli/cmd_run.py

@@ -48,6 +48,9 @@ def run(obj: dict, log_level: str, debug: bool, host: str, port: int, reload: bo
     import os
     import uvicorn
 
+    if obj.get("dev", False):
+        debug = True
+
     log_level = log_level.lower()
     if debug and log_level == "info":
         log_level = "debug"

aiida_optimade/cli/utils.py

@@ -4,6 +4,9 @@
 from aiida.manage.configuration import get_config
 
 
+AIIDA_OPTIMADE_TEST_PROFILE = "aiida-optimade_test"
+
+
 def get_aiida_profiles() -> List[str]:
     """Retrieve list of configured AiiDA profiles"""
 

aiida_optimade/config.json

@@ -26,7 +26,8 @@
       "last_modified": "mtime",
       "type": "node_type",
       "relationships": "attributes.something.non.existing",
-      "links": "attributes.something.non.existing"
+      "links": "attributes.something.non.existing",
+      "meta": "attributes.someting.non.existing"
     }
   }
 }