[REVIEW]: ratesb_python: Rate Law Analysis for SBML and Antimony

[editorialbot]

Submitting author: @longxf21 (Longxuan Fan)
Repository: https://github.com/sys-bio/ratesb_python
Branch with paper.md (empty if default branch):
Version: v0.2.6
Editor: @RMeli
Reviewers: @ayush9pandey, @dilawar
Archive: Pending

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/f12fdb3eb5db0349d08be12b9a337147"><img src="https://joss.theoj.org/papers/f12fdb3eb5db0349d08be12b9a337147/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/f12fdb3eb5db0349d08be12b9a337147/status.svg)](https://joss.theoj.org/papers/f12fdb3eb5db0349d08be12b9a337147)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@ayush9pandey & @dilawar, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review.
First of all you need to run this command in a separate comment to create the checklist:

@editorialbot generate my checklist

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @RMeli know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Checklists

📝 Checklist for @dilawar

📝 Checklist for @ayush9pandey

[editorialbot]

Hello humans, I'm @editorialbot, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@editorialbot generate pdf

[editorialbot]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

✅ OK DOIs

- 10.1186/s12859-023-05380-3 is OK
- 10.1093/bioinformatics/btg015 is OK
- 10.1093/bioinformatics/btp401 is OK

🟡 SKIP DOIs

- None

❌ MISSING DOIs

- None

❌ INVALID DOIs

- None

[editorialbot]

Software report:

github.com/AlDanial/cloc v 1.98  T=0.04 s (2413.7 files/s, 109668.9 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Python                          11            351            500           1669
XML                             62              8             10            686
reStructuredText                 8            122             98            278
JSON                             8              1              0            274
Markdown                         2             81              0            249
YAML                             4              8              4             88
TeX                              1              3              0             50
DOS Batch                        1              8              1             26
TOML                             1              0              0             21
Dockerfile                       1              5              7             20
make                             1              4              7              9
Text                             1              0              0              1
-------------------------------------------------------------------------------
SUM:                           101            591            627           3371
-------------------------------------------------------------------------------

Commit count by author:

    47	Aldrich Fan
     2	longxf
     1	Joseph Hellerstein

[editorialbot]

Paper file info:

📄 Wordcount for paper.md is 988

✅ The paper includes a Statement of need section

[editorialbot]

License info:

✅ License found: MIT License (Valid open source OSI approved license)

[editorialbot]

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

[RMeli]

ratesb_python relies heavily on approaches employed in SBMLKinetics [@Xu2023], which uses the sympy package to do symbolic analysis of rate laws. ratesb_python refines and extends these approaches to increase the accuracy of classification and to improve performance.

Can you please describe in a detailed manner, in the paper, how ratesb_python refines and extends the approaches used by SBMLKinetics? This should also be partially included in the "Statement of need" section: what ratesb_python brings to the table that other existing software do not?

Another thing that pops to my eyes when reading the paper is the use of error codes: could you expand why the codes are necessary, and where to find a detailed explanation of the grouped error codes?

[dilawar]

Review checklist for @dilawar

Conflict of interest

• I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

• I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• Repository: Is the source code for this software available at the https://github.com/sys-bio/ratesb_python?
• License: Does the repository contain a plain-text LICENSE or COPYING file with the contents of an OSI approved software license?
• Contribution and authorship: Has the submitting author (@longxf21) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
• Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines
• Data sharing: If the paper contains original data, data are accessible to the reviewers. If the paper contains no original data, please check this item.
• Reproducibility: If the paper contains original results, results are entirely reproducible by reviewers. If the paper contains no original results, please check this item.
• Human and animal research: If the paper contains original data research on humans subjects or animals, does it comply with JOSS's human participants research policy and/or animal research policy? If the paper contains no such data, please check this item.

Functionality

• Installation: Does installation proceed as outlined in the documentation?
• Functionality: Have the functional claims of the software been confirmed?
• Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
• Community guidelines: Are there clear guidelines for third parties wishing to 1. Contribute to the software 2. Report issues or problems with the software 3. Seek support

Software paper

• Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
• A statement of need: Does the paper have a section titled 'Statement of need' that clearly states what problems the software is designed to solve, who the target audience is, and its relation to other work?
• State of the field: Do the authors describe how this software compares to other commonly-used packages?
• Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
• References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

[longxf21]

@RMeli Hi, Sorry for the delayed response, I have just finished revising. But docker mac has an incident and it's not functioning so please it might take a few more days.

[RMeli]

No problem @longxf21, no rush. But thank you for letting me know.

[dilawar]

Hi,

I've started the review. All scripts/tests that I am going to write for this review will be available here https://github.com/dilawar/joss-review-7618-ratesb-python . I am going to open issues on the project repository and link them here. It will a bit noisy but everyone involved in the review will get an update.

English is not my native tounge so my apologies in advance for all sorts of silly embarassing typos and grammatical errors.

• SyntaxWarnings when running a snippet from the README.md sys-bio/ratesb_python#8
• Easy way to get result.Result as JSON/XML? sys-bio/ratesb_python#15
• Broken example in docs: Complex Example at https://longxf-ratesb-python.readthedocs-hosted.com/en/latest/quickstart.html#complex-example sys-bio/ratesb_python#16

[dilawar]

@RMeli @longxf21 I want to run the "complex example" mentioned in the README.md. Please advice which 'xml' and json file I should link to? Feel free to upload them here if they are not in repository. A note on expected output for the given xml/json combination will be most helpful.

[RMeli]

@dilawar, I'm the editor, not the author. Questions should be addressed to @longxf21. If they are because of unclear documentation, installation instructions, etc. please open an issue on the repository and link it to this PR.

[longxf21]

@RMeli @longxf21 I want to run the "complex example" mentioned in the README.md. Please advice which 'xml' and json file I should link to? Feel free to upload them here if they are not in repository. A note on expected output for the given xml/json combination will be most helpful.

I will compile and push files in examples folder into the repository soon, will keep you updated!

[longxf21]

@editorialbot generate pdf

[longxf21]

@RMeli Hi, I've pushed the revised pdf into the repo, thanks!

[editorialbot]

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

[ayush9pandey]

Review checklist for @ayush9pandey

Conflict of interest

• I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

• I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• Repository: Is the source code for this software available at the https://github.com/sys-bio/ratesb_python?
• License: Does the repository contain a plain-text LICENSE or COPYING file with the contents of an OSI approved software license?
• Contribution and authorship: Has the submitting author (@longxf21) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
• Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines
• Data sharing: If the paper contains original data, data are accessible to the reviewers. If the paper contains no original data, please check this item.
• Reproducibility: If the paper contains original results, results are entirely reproducible by reviewers. If the paper contains no original results, please check this item.
• Human and animal research: If the paper contains original data research on humans subjects or animals, does it comply with JOSS's human participants research policy and/or animal research policy? If the paper contains no such data, please check this item.

Functionality

• Installation: Does installation proceed as outlined in the documentation?
• Functionality: Have the functional claims of the software been confirmed?
• Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
• Community guidelines: Are there clear guidelines for third parties wishing to 1. Contribute to the software 2. Report issues or problems with the software 3. Seek support

Software paper

• Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
• A statement of need: Does the paper have a section titled 'Statement of need' that clearly states what problems the software is designed to solve, who the target audience is, and its relation to other work?
• State of the field: Do the authors describe how this software compares to other commonly-used packages?
• Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
• References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

[ayush9pandey]

ratesb_python is a handy tool for systems and synthetic biology modeling community of researchers. It provides users an API that can be integrated into other modeling pipeline or used standalone to verify the correctness of rate laws. Many possible applications will benefit from this package. Great work from the team!

For the JOSS checklist on "performance claims": the paper talks about polynomial time complexity, as compared to purely symbolic approaches that have been used for rate law validation before. Please provide a comparison of the time complexity for an example system (that is large enough to show the differences in the time complexity) with ratesb_python and other tools.

For software-related issues, I have raised my concerns on the original repository:
sys-bio/ratesb_python#14
sys-bio/ratesb_python#13
sys-bio/ratesb_python#12
sys-bio/ratesb_python#11
sys-bio/ratesb_python#10
sys-bio/ratesb_python#9

Will test the functionality of the software one more time and wait for the authors to resolve these issues before I finish my checklist.
I faced some issues with using SBML files generated from a different tool and raised that as an issue on the repository.

[RMeli]

Thank you @ayush9pandey for the update and for linking all the different issues you opened!

@dilawar how is you review progressing?

[longxf21]

Hi all,

I am having a tight schedule due to job interviews. I will try my best to catch up after those :)

Longxuan

[dilawar]

@RMeli I am sorry for being so slow. Yesterday, I spent a little bit of time and ran into SBML import issues which @ayush9pandey has already reported. I'll try to finish the review by this weekend. Hopefully by them author(s) would respond to other opened issues as well.

[dilawar]

I've opened a couple of issues on the package repository and have linked them in #7618 (comment).

Overall, the package is in good shape. There are decent number of tests which can always be extended to cover complex cases. Documentation is decent but formatting can be improved a bit. Especially, instead of copy-pasting code from examples, prefer refer the example/test file directly.

Following are a few nitpicks on the code/projects that I think will help improving codebase. These are optional for this review.

• Add typehints along with adding mypy or ruff to development tooling would greatly help in finding bugs and make codebase more robust.
• Prefer pathlib over os.path.
• In many internal files, authors have manually added script's parent directory path to system paths e.g.

current_dir = os.path.abspath(os.path.dirname(__file__))
parent_dir = os.path.dirname(current_dir)
sys.path.append(current_dir)
sys.path.append(parent_dir)

from common import util

Python import system is not without its problem. Ideally, one would expect a single line from ratesb_python.common import util here. Perhaps this https://setuptools.pypa.io/en/latest/userguide/development_mode.html during development will help.

I can happily recommend uv to manage this project.

• Perhaps black or ruff can be use to automatically format the code and docstrings.

[longxf21]

I've fixed the issues regarding the docs. I will take some time look into issue 15, 14, and 8.

[RMeli]

Hi, I hope you are all doing well. Would you mind please summarize in a couple of lines how this review is progressing? Many thanks in advance!

[RMeli]

Hi all, I haven't heard back from any of you in a while. Can you please let me know in a couple of sentences how this review is going?

I see @dilawar completed the checklist, but there are still some outstanding items in #7618 (comment).

In @ayush9pandey there are still some missing items.

@longxf21 can you please have a look at the open issues from the reviewers?

Thank you everyone!

[ayush9pandey]

Thanks for checking in @RMeli , I am still waiting for a couple of issues to be resolved, primarily sys-bio/ratesb_python#14
A few of the other issues I raised were closed without comment. I am looking for if they were addressed some place or whether the authors decided on better approaches that made the issue obsolete.

[dilawar]

@RMeli Sorry for late response. This fell through cracks at my end.

Except for open issues that authors have not yet responded to, rest looks fine to me. Also, I didn't create some issues to avoid duplicates since they have been reported by @ayush9pandey .

[longxf21]

@RMeli I will be giving a more detailed update and address each open issue accordingly in 2 days. But for now, most of the issues that are still open are either a problem of another dependency, or will be included in future updates!

[longxf21]

Hi all. I have replied to all issues and made a new release (0.2.6) on pypi. Apologies for the delay in resolving these matters – I appreciate the patience while we worked through the feedback.