Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

restructure scienta-PEAK-ibw update #111

Draft
wants to merge 6 commits into
base: scienta-PEAK-ibw
Choose a base branch
from

Conversation

lukaspie
Copy link
Collaborator

@lukaspie lukaspie commented Feb 11, 2025

Summary by Sourcery

Update the Scienta parser to support newer .ibw files and improve data handling. Flatten nested dictionaries within the metadata for easier access. Standardize energy scales and units. Add support for exporting axes data for plotting. Update example data and identifiers.

Copy link

sourcery-ai bot commented Feb 11, 2025

Reviewer's Guide by Sourcery

This pull request restructures the Scienta PEAK IBW update by refactoring parser logic, updating data mapping and unit handling, and introducing JSON schema validation for PEAK metadata. Major changes include reorganizing parser classes into abstract and version-specific implementations, refactoring how raw spectrum data is processed and flattened, and streamlining unit mappings across the project.

Updated class diagram for Scienta Parser refactoring

classDiagram
    %% Base mapping class
    class XPSMapper {
      <<abstract>>
    }

    %% MapperScienta inherits from XPSMapper and uses parser logic
    class MapperScienta {
      - _xps_dict : dict
      + construct_data()
      + _select_parser()
      + _update_xps_dict_with_spectrum(spectrum: Dict)
    }
    XPSMapper <|-- MapperScienta

    %% Abstract parser class
    class ScientaIgorParser {
      <<abstract>>
      + parse_file(file: Union[str, Path])
      + _parse_unit(bunit: bytes) : List[Tuple[str, Optional[str]]]
      + axis_for_dim(wave_header: Dict, dim: int) : np.ndarray
      + axis_units_for_dim(wave_header: Dict, dim: int) : str
      # _parse_note(bnote: bytes) : Dict
      # _parse_region_metadata(region_id: int, notes: Dict) : Dict
    }

    %% Parser implementation for old data export
    class ScientaIgorParserOld {
      + _parse_note(bnote: bytes) : Dict
      + _parse_region_metadata(region_id: int, notes: Dict) : Dict
    }
    ScientaIgorParser <|-- ScientaIgorParserOld

    %% Parser implementation for PEAK software export
    class ScientaIgorParserPEAK {
      + _parse_note(bnote: bytes) : Dict
      + _parse_region_metadata(region_id: int, notes: Dict) : Dict
    }
    ScientaIgorParser <|-- ScientaIgorParserPEAK

    %% Utility function as a helper (not a class but represented for clarity)
    class flatten_dict {
      <<utility>>
      + flatten_dict(d: Dict, parent_key: str, sep: str) : Dict
    }

    %% Relationships
    MapperScienta ..> ScientaIgorParser : uses
    MapperScienta ..> flatten_dict : calls
Loading

File-Level Changes

Change Details Files
Refactoring and improvements in the scienta_reader module
  • Introduced a new utility function to flatten nested dictionaries for easier key management.
  • Added logging configuration via a dedicated logger.
  • Reworked the _select_parser method to use a try-except block that distinguishes between the PEAK and legacy IBW formats.
  • Updated the construct_data and _update_xps_dict_with_spectrum methods to remove hard-coded mapping templates, instead iterating over spectrum data dynamically.
  • Modified handling of energy axis and intensity arrays: replaced direct use of 'energy' with generalized 'axes' and improved data array assignment.
src/pynxtools_xps/scienta/scienta_reader.py
Enhancements to parser abstractions and implementations
  • Changed the base ScientaIgorParser class to an abstract class (using ABC), requiring implementation of parsing methods.
  • Introduced two distinct parser implementations: one for PEAK exports (ScientaIgorParserPEAK) and one legacy version (ScientaIgorParserOld).
  • Rewrote the _parse_note methods so that the PEAK parser validates against a JSON schema, while the legacy parser handles JSON decoding flexibly.
  • Updated the _parse_unit method to return a list of label-unit tuples using regex, supporting multiple unit formats.
src/pynxtools_xps/scienta/scienta_reader.py
src/pynxtools_xps/scienta/scienta_data_model.py
Addition of JSON schema for Scienta PEAK metadata
  • Defined a comprehensive JSON schema (scienta_igor_peak_schema) for validating PEAK export metadata.
  • Removed obsolete dataclass definitions unnecessary for the new PEAK format.
src/pynxtools_xps/scienta/scienta_data_model.py
Streamlining unit mappings
  • Simplified the UNIT mapping keys in the scienta_mappings module by removing grouped prefixes.
  • Ensured consistency across unit assignments in the data conversion and mapping functions.
src/pynxtools_xps/scienta/scienta_mappings.py
Enhancements in data extraction in the reader module
  • Extended the get_signals function to include 'axes' alongside energy channels.
  • Modified the get_data function to handle extraction of axes data correctly from xarray coordinates.
  • Added fallback handling for cases where a KeyError might occur when retrieving axis information.
src/pynxtools_xps/reader.py
Minor configuration and example updates
  • Updated instrument and sample identifiers in the example YAML file for clarity and consistency.
  • Adjusted the instrument identifier in the Scienta configuration JSON file.
examples/scienta/ibw/eln_data_scienta_ibw.yaml
src/pynxtools_xps/config/config_scienta.json

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!
  • Generate a plan of action for an issue: Comment @sourcery-ai plan on
    an issue to generate a plan of action for it.

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant