Skip to content

Commit

Permalink
Merge pull request #88 from FAIRmat-NFDI/bring-in-examples
Browse files Browse the repository at this point in the history
Bring in NOMAD examples
  • Loading branch information
lukaspie authored Nov 29, 2024
2 parents 1d8c392 + 8175be8 commit be2f10d
Show file tree
Hide file tree
Showing 15 changed files with 2,305 additions and 16 deletions.
4 changes: 4 additions & 0 deletions .github/workflows/pytest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,10 @@ jobs:
run: |
curl -LsSf https://astral.sh/uv/install.sh | sh
uv pip install coverage coveralls
- name: Install nomad
if: "${{ matrix.python_version != '3.8'}}"
run: |
uv pip install nomad-lab[infrastructure]@git+https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR.git
- name: Install package
run: |
uv pip install ".[dev]"
Expand Down
5 changes: 5 additions & 0 deletions MANIFEST.in
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
prune *
exclude *
recursive-include src/pynxtools_xps *.py *.json
include pyproject.toml README.md dev-requirements.txt
graft src/pynxtools_xps/nomad/examples
24 changes: 19 additions & 5 deletions docs/tutorial/standalone.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,11 +25,25 @@ An example script to run the XPS reader in `pynxtools`:
user@box:~$ dataconverter $<xps-file path> $<xps-file path> $<eln-file path> --reader xps --nxdl NXxps --output <output-file path>.nxs
```

Note that none of the supported file format have data/values for all required and recommended fields and attributes in NXxps. In order for the validation step of the XPS reader to pass, you need to provide an ELN file that contains the missing values. Example raw and converted data can be found in [*pynxtools_xps/examples*](https://github.com/FAIRmat-NFDI/pynxtools-xps/tree/main/examples).
Note that none of the supported file format have data/values for all required and recommended fields and attributes in ``NXxps``. In order for the validation step of the XPS reader to pass, you need to provide an ELN file that contains the missing values.

TODO: add more steps! <!--[The Jupyter notebook is available here](https://github.com/FAIRmat-NFDI/pynxtools-em/blob/main/examples/HowToUseTutorial.ipynb) TODO!-->
### Examples

You can find examples how to use `pynxtools-xps` for your XPS research data pipeline in [`src/pynxtools-xps/nomad/examples`](../../src/pynxtools_xps/nomad/examples/). These are designed for working with [`NOMAD`](https://nomad-lab.eu/) and its [`NOMAD Remote Tools Hub (NORTH)`](https://nomad-lab.eu/prod/v1/gui/analyze/north). Feel invited to try out the respective tutorial [here](tutorial/nomad.md).

There are also small example files with raw and converted data for using the `pynxtools` dataconverter with the `mpes` reader and the `NXmpes` application definition in the [`examples`](../../examples/) folder.

**Congrats! You now have a FAIR NeXus file!**
For this tutorial, we will work with the example data for the VAMAS reader (see here [](../../examples/vms/)). You can run the conversion as
```shell
dataconverter \\
--reader xps \\
--nxdl NXmpes \\
regular.vms \\
eln_data_vms.yaml \\
-c config_file.json \\
--output regular.vms.nxs
```

TODO: add more steps! <!--[The Jupyter notebook is available here](https://github.com/FAIRmat-NFDI/pynxtools-em/blob/main/examples/HowToUseTutorial.ipynb) TODO!-->

The above-mentioned parsing is also integrated into the NOMAD research data management system.
Feel invited to try out the respective tutorial [here]((tutorial/nomad.md)
**Congrats! You now have a FAIR NeXus file!**
24 changes: 13 additions & 11 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -25,18 +25,10 @@ dependencies = [
"h5py>=3.6.0",
"igor2",
"xarray>=0.20.2",
"numpy>=1.21.2",
'numpy>=1.22.4,<2.0.0',
"pint>=0.17",
"pynxtools>=0.7.0",
"pynxtools>=0.9.0",
]

[project.entry-points."pynxtools.reader"]
xps = "pynxtools_xps.reader:XPSReader"

[project.urls]
"Homepage" = "https://github.com/FAIRmat-NFDI/pynxtools-xps"
"Bug Tracker" = "https://github.com/FAIRmat-NFDI/pynxtools-xps/issues"

[project.optional-dependencies]
dev = [
"mypy",
Expand All @@ -60,6 +52,16 @@ docs = [
"mkdocs-click"
]

[project.urls]
"Homepage" = "https://github.com/FAIRmat-NFDI/pynxtools-xps"
"Bug Tracker" = "https://github.com/FAIRmat-NFDI/pynxtools-xps/issues"

[project.entry-points."pynxtools.reader"]
xps = "pynxtools_xps.reader:XPSReader"

[project.entry-points.'nomad.plugin']
xps_example = "pynxtools_xps.nomad.entrypoints:xps_example"

[tool.setuptools.packages.find]
where = [
"src",
Expand All @@ -82,7 +84,7 @@ select = [
"E", # pycodestyle
"W", # pycodestyle
"PL", # pylint
"NPY201",
# "NPY201",
]
ignore = [
"E501", # Line too long ({width} > {limit} characters)
Expand Down
38 changes: 38 additions & 0 deletions src/pynxtools_xps/nomad/entrypoints.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
#
# Copyright The NOMAD Authors.
#
# This file is part of NOMAD. See https://nomad-lab.eu for further info.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
"""Entry points for XPS examples."""

try:
from nomad.config.models.plugins import ExampleUploadEntryPoint
except ImportError as exc:
raise ImportError(
"Could not import nomad package. Please install the package 'nomad-lab'."
) from exc

xps_example = ExampleUploadEntryPoint(
title="X-ray Photoelectron Spectroscopy (XPS)",
category="FAIRmat examples",
description="""
This example presents the capabilities of the NOMAD platform to store and standardize X-ray Photoelectron Spectroscopy XPS data.
It shows the generation of a NeXus file according to the
[NXmpes](https://manual.nexusformat.org/classes/contributed_definitions/NXmpes.html#nxmpes)
application definition from an example measurement file and a subseqeuent analysis of that data set.
""",
plugin_package="pynxtools_xps",
resources=["nomad/examples/*"],
)
137 changes: 137 additions & 0 deletions src/pynxtools_xps/nomad/examples/E1 XPS data conversion to NeXus.ipynb
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"id": "38f1916e-2d0f-4f81-9b00-fedc615a96aa",
"metadata": {},
"source": [
"# XPS conversion example\n",
"\n",
"In this notebook a XPS measurement file from a SPECS detector (using the native SPECS .sle format) is read and converted into the [NXmpes](https://manual.nexusformat.org/classes/contributed_definitions/NXmpes.html#nxmpes) NeXus standard."
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "dc099289-78f5-4357-b7d3-715dd20179da",
"metadata": {},
"source": [
"## Create a NeXus file from measurement data\n",
"\n",
"To convert the available files to the NeXus format we use the convert function readily supplied by [`pynxtools`](https://github.com/FAIRmat-NFDI/pynxtools)."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7d2c1df4-1d96-4255-a18e-e323c69d32b4",
"metadata": {},
"outputs": [],
"source": [
"from pynxtools.dataconverter.convert import convert, logger\n",
"import logging\n",
"logger.setLevel(logging.ERROR)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "bf441135-d4c5-4310-ae6e-39c87da1b726",
"metadata": {},
"source": [
"The input parameters are defined as follows:\n",
"\n",
"**input_file**: The input files for the reader. This is a sle file from [SpecsLabProdigy](https://www.specs-group.com/nc/specs/products/detail/prodigy/) (v.63), which is the propietary format of SPECS GmbH,and a YAML ELN file containing additional information not contained in the measurement file (e.g. user name).\n",
"\n",
"**reader**: The specific reader which gets called inside `pynxtools`. This is supplied in the [`pynxtools-xps`](https://github.com/FAIRmat-NFDI/pynxtools-xps) reader plugin. For XPS data, the reader is called `xps`.\n",
"\n",
"**nxdl**: The specific NXDL application definition to which the converted file should conform. For XPS this should be `NXmpes`, the subdefinition `NXxps`, or any further subdefinitions of the form `NXxps_<name>`.\n",
" \n",
"**output**: The output filename of the NeXus file."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8e8a76ff-918e-483d-9ed1-5417613710e1",
"metadata": {},
"outputs": [],
"source": [
"convert(\n",
" input_file=[\"EX439_S718_Au_in_25_mbar_O2.sle\", \"eln_data.yaml\"],\n",
" reader='xps',\n",
" nxdl='NXmpes',\n",
" remove_align=True,\n",
" output='Au_25_mbar_O2_no_align.nxs'\n",
")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "fa5fe119-0a72-4744-a84d-4d1258f55031",
"metadata": {},
"source": [
"## View the data with H5Web\n",
"\n",
"H5Web is a tool for visualizing any data in the h5 data format. Since the NeXus format builds opon h5 it can be used to view this data as well. We just import the package and call H5Web with the output filename from the convert command above.\n",
"\n",
"You can also view this data with the H5Viewer or other tools from your local filesystem."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d4ff822a-4552-49b8-ba17-9b86fd8c2ac1",
"metadata": {},
"outputs": [],
"source": [
"from jupyterlab_h5web import H5Web"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f62097ae-eb0f-4572-b8d9-bebc7266b43a",
"metadata": {},
"outputs": [],
"source": [
"H5Web(\"Au_25_mbar_O2_no_align.nxs\")"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.10"
},
"vscode": {
"interpreter": {
"hash": "0cbdf5d5ef28617c8bf3753ff15cd1b7b5539de5aaa68a35c3d38ca27e1ab0fa"
}
},
"widgets": {
"application/vnd.jupyter.widget-state+json": {
"state": {},
"version_major": 2,
"version_minor": 0
}
}
},
"nbformat": 4,
"nbformat_minor": 5
}
Loading

0 comments on commit be2f10d

Please sign in to comment.