Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* Development (#47) * added PubMed (refactored to OOP) module and scaffolding for IEEE module * initialized poetry project * added (blank) __init__.py to package and test directories * added testing, logging to PubMed.py (WIP) * further testing * added python venv dirs to .gitignore * .gitignore additions, formatting/linter changes * Added arXiv python file * Added python file for Springer API * Added python file for Elsevier API * Added python file for Wiley API * Added test for arXvip and updated code for arXvip to standardize name variations * Git added test for Spinger API * Updated code for test_Springer and Spinger.py for name Normalizion * Updated code to be in class format * deleted redundant comments * Updated code for Elsevier.py * Fixed Commenting * Updated code for Wiley.py normalize_author_name * added docstrings, better testing coverage * better error logging in PubMed.py * Fixed code format * Fixed standardize author name for all API * Hendrix additions (#1) * added PubMed (refactored to OOP) module and scaffolding for IEEE module * initialized poetry project * added (blank) __init__.py to package and test directories * added testing, logging to PubMed.py (WIP) * further testing * added python venv dirs to .gitignore * .gitignore additions, formatting/linter changes * deleted redundant comments * added docstrings, better testing coverage * better error logging in PubMed.py --------- Co-authored-by: Joseph Hendrix <[email protected]> Co-authored-by: Joseph Hendrix <[email protected]> * Fixed .gitignore file * Updated test_PubMed.py yo mach with recent changes * Updated to PubMed.py for merging with dev * Fixed returning authors for arXiv * Fixed standardize_author_name for Elsevier * Update Elsevier.py * Added try/catch for Elsevier and arXiv * Added try/catch for PubMed * Added try/catch to Springer * add try/catch for Wiley api * fixed imports and API keys * Added PLOS API along with try/catch * pyproject changes ensure extending from a Base class doesn't break pytest * removed old pubmed script at root of project * API modules extend base class, added secrets template * moved URLs to config.py * added functionality to main.py (WIP) * added bumpversion, poetry script entry point * main.py can now accept command line arguments (WIP) * fixed erroneous method call in MDPI class, added root logger object to main.py * Updated arXive test and code * Added more test for arXiv * Updated formate * Base class, main CLI script (#5) * pyproject changes ensure extending from a Base class doesn't break pytest * removed old pubmed script at root of project * API modules extend base class, added secrets template * moved URLs to config.py * added functionality to main.py (WIP) * added bumpversion, poetry script entry point * main.py can now accept command line arguments (WIP) * fixed erroneous method call in MDPI class, added root logger object to main.py * Updated with baseclass * Fxied main * Added Wiley Api and fixed nameing convention of DOI's * Updated config with urls also connecting new api files with Base * Cli additions (#9) * todo comments * added example input file, author names gathered from file * main script writes API results to (user configurable) file * added api-keyed endpoints to main script * updated .gitignore * crossref into cli_additions (#8) * added CrossRef module and tests (WIP) * crossref data extraction (WIP) * barebones crossref functionality * added full CrossRef functionality, began testing * CrossRef tests and accompanying module changes --------- Co-authored-by: Joseph Hendrix <[email protected]> * added functionality to specify targeted APIs through command line * removed obsolete requirements.txt * Updated README.md updated README.md to reflect command line changes * minor updates to secret.sample for clarity * updated TODO comments in main --------- Co-authored-by: Joseph Hendrix <[email protected]> Co-authored-by: Magret Adekunle <[email protected]> * Better testing (#11) * created dev dependency group * removed extraneous config from project root * added main.py tests * added description to pyproject.toml * better pubmd tests (WIP) * mock test for PubMed module * pubmed tests no longer query PubMed API * Springer tests no longer query Springer API * removed bad test from test_CrossRef.py (testing API rather than pubscraper functionality) * cleaned some comments * List apis (#12) * added --list flag to list APIs available to query * added help message for --apis and --list * prettier --list output * Updated Main to have PLOS/Weily and testing for PLOS/arXiv * Added test for Elsevier * Child class changes (#14) * privatized crossref methods, added DOI * linter changes, privatized Elsevier methods, changed result format * privatized MDPI methods, added DOI * PLOS linter changes, added DOI * added DOI to PubMed, privatized methods * privatized Springer methods, changed result formatting * privatized Wiley methods, changed result formatting * privatized arXiv methods, changed output formatting * added tests for --list CLI flag * minor typo fixes in comments * better logging on startup * Containerize (#15) * added Dockerfile (builds, no run) * changed some import statements to fix ModuleNotFoundErrors when running from wheel build * Added tablib to export data * Crossref fixes (#16) * privatized crossref methods, added DOI * linter changes, privatized Elsevier methods, changed result format * privatized MDPI methods, added DOI * PLOS linter changes, added DOI * added DOI to PubMed, privatized methods * privatized Springer methods, changed result formatting * privatized Wiley methods, changed result formatting * privatized arXiv methods, changed output formatting * added tests for --list CLI flag * minor typo fixes in comments * aggregating results is broken, all other tests pass * fixed a bad check in CrossRef.py where 0 valid publications would mistakenly break out of query loop * Aded changes to README.md * Update README.md * Update README.md * Updated MDPI and main.py * Updated MDPI.py and main.py * Update README.md * Main fixes (#20) * minor Dockerfile fixes in COPY statements * updated CLI options and their tests * changed input file and input parsing to better reflect expected input * Update README.md * Updated main * Updted main and sub class for logger * Updated ArXive date * Standardize CrossRef publication date * Standardize Elsevier publication date * Standardize MDPI publication date * Standardize PLOS publication date * Standardize PubMed publication date * Standardize Springer publication date * Standardize Wiley publication date * Standardize the publication date to "YYYY-MM-DD" * Updated main.py to allows user to pull publication from cutoff date and test for main.py * Added Documentations for xlsx * updated README for flag `--cutoff_date` or `-cd` * Update README.md * Update README.md * Added mock responses for test_Elsevier.py * Added Test for test_CrossRef.py * Update README.md * Docker io (#35) * script now reads a (default) xlsx file instead of a csv * added run.sh script (WIP) * added .env support and example for storing secrets * Update README.md * refactored API list to a const in main.py * changed --input to click.Path object * removed old files from Dockerfile * moved input and default input/output files value to /data directory, mount /data directory to docker container instead of the whole project * quick poetry.lock fix * Updated MDPI for pull date * Added logging.debug for MDPI.py * Updated code for pull date (CrossRef.py) * Updated code for Elsevire.py (date and added logging.debug) * Added code for loggin.debug * Added logging.debug for PLOS.py * Updated pull date from PubMed.py * Added logging.debug to Spinger.py * Added loggin.debug to Wiley.py * Updated authors naming * Bumpversion fixes (#37) * added .env version spec to pyproject; bumpversion now updates .env * Update README.md * got rid of /data folder (too confusing) * Update README.md * Added testing for test_MDPI.py * Updated test_Elsevier.py to use .env * Added testting for test_Elsevire.py * Wiley tests (#42) * added Wiley tests (WIP) * added Wiley tests * Development (#41) * Added testing for test_MDPI.py * Updated test_Elsevier.py to use .env * Added testting for test_Elsevire.py --------- Co-authored-by: Magret <[email protected]> Co-authored-by: Magret Adekunle <[email protected]> * fixed main.py tests (all tests pass!) --------- Co-authored-by: Magret <[email protected]> Co-authored-by: Magret Adekunle <[email protected]> * Runner versioning (#43) * run.sh now properly reads IMAGEVERSION environment variable from .env for building the container * removed old secrets sample, edited IMAGEVERSION in .env.sample to match our versioning scheme * changed help text for --list option for clarity * updated help text for test_main * Cleanup (#45) * added callback for --list option * Update README.md * Updated hard-coded vars to config.py * bumped version --------- Co-authored-by: Joseph Hendrix <[email protected]> Co-authored-by: Joseph Hendrix <[email protected]> Co-authored-by: Magret <[email protected]> Co-authored-by: Magret Adekunle <[email protected]> * Development (#49) * added PubMed (refactored to OOP) module and scaffolding for IEEE module * initialized poetry project * added (blank) __init__.py to package and test directories * added testing, logging to PubMed.py (WIP) * further testing * added python venv dirs to .gitignore * .gitignore additions, formatting/linter changes * Added arXiv python file * Added python file for Springer API * Added python file for Elsevier API * Added python file for Wiley API * Added test for arXvip and updated code for arXvip to standardize name variations * Git added test for Spinger API * Updated code for test_Springer and Spinger.py for name Normalizion * Updated code to be in class format * deleted redundant comments * Updated code for Elsevier.py * Fixed Commenting * Updated code for Wiley.py normalize_author_name * added docstrings, better testing coverage * better error logging in PubMed.py * Fixed code format * Fixed standardize author name for all API * Hendrix additions (#1) * added PubMed (refactored to OOP) module and scaffolding for IEEE module * initialized poetry project * added (blank) __init__.py to package and test directories * added testing, logging to PubMed.py (WIP) * further testing * added python venv dirs to .gitignore * .gitignore additions, formatting/linter changes * deleted redundant comments * added docstrings, better testing coverage * better error logging in PubMed.py --------- Co-authored-by: Joseph Hendrix <[email protected]> Co-authored-by: Joseph Hendrix <[email protected]> * Fixed .gitignore file * Updated test_PubMed.py yo mach with recent changes * Updated to PubMed.py for merging with dev * Fixed returning authors for arXiv * Fixed standardize_author_name for Elsevier * Update Elsevier.py * Added try/catch for Elsevier and arXiv * Added try/catch for PubMed * Added try/catch to Springer * add try/catch for Wiley api * fixed imports and API keys * Added PLOS API along with try/catch * pyproject changes ensure extending from a Base class doesn't break pytest * removed old pubmed script at root of project * API modules extend base class, added secrets template * moved URLs to config.py * added functionality to main.py (WIP) * added bumpversion, poetry script entry point * main.py can now accept command line arguments (WIP) * fixed erroneous method call in MDPI class, added root logger object to main.py * Updated arXive test and code * Added more test for arXiv * Updated formate * Base class, main CLI script (#5) * pyproject changes ensure extending from a Base class doesn't break pytest * removed old pubmed script at root of project * API modules extend base class, added secrets template * moved URLs to config.py * added functionality to main.py (WIP) * added bumpversion, poetry script entry point * main.py can now accept command line arguments (WIP) * fixed erroneous method call in MDPI class, added root logger object to main.py * Updated with baseclass * Fxied main * Added Wiley Api and fixed nameing convention of DOI's * Updated config with urls also connecting new api files with Base * Cli additions (#9) * todo comments * added example input file, author names gathered from file * main script writes API results to (user configurable) file * added api-keyed endpoints to main script * updated .gitignore * crossref into cli_additions (#8) * added CrossRef module and tests (WIP) * crossref data extraction (WIP) * barebones crossref functionality * added full CrossRef functionality, began testing * CrossRef tests and accompanying module changes --------- Co-authored-by: Joseph Hendrix <[email protected]> * added functionality to specify targeted APIs through command line * removed obsolete requirements.txt * Updated README.md updated README.md to reflect command line changes * minor updates to secret.sample for clarity * updated TODO comments in main --------- Co-authored-by: Joseph Hendrix <[email protected]> Co-authored-by: Magret Adekunle <[email protected]> * Better testing (#11) * created dev dependency group * removed extraneous config from project root * added main.py tests * added description to pyproject.toml * better pubmd tests (WIP) * mock test for PubMed module * pubmed tests no longer query PubMed API * Springer tests no longer query Springer API * removed bad test from test_CrossRef.py (testing API rather than pubscraper functionality) * cleaned some comments * List apis (#12) * added --list flag to list APIs available to query * added help message for --apis and --list * prettier --list output * Updated Main to have PLOS/Weily and testing for PLOS/arXiv * Added test for Elsevier * Child class changes (#14) * privatized crossref methods, added DOI * linter changes, privatized Elsevier methods, changed result format * privatized MDPI methods, added DOI * PLOS linter changes, added DOI * added DOI to PubMed, privatized methods * privatized Springer methods, changed result formatting * privatized Wiley methods, changed result formatting * privatized arXiv methods, changed output formatting * added tests for --list CLI flag * minor typo fixes in comments * better logging on startup * Containerize (#15) * added Dockerfile (builds, no run) * changed some import statements to fix ModuleNotFoundErrors when running from wheel build * Added tablib to export data * Crossref fixes (#16) * privatized crossref methods, added DOI * linter changes, privatized Elsevier methods, changed result format * privatized MDPI methods, added DOI * PLOS linter changes, added DOI * added DOI to PubMed, privatized methods * privatized Springer methods, changed result formatting * privatized Wiley methods, changed result formatting * privatized arXiv methods, changed output formatting * added tests for --list CLI flag * minor typo fixes in comments * aggregating results is broken, all other tests pass * fixed a bad check in CrossRef.py where 0 valid publications would mistakenly break out of query loop * Aded changes to README.md * Update README.md * Update README.md * Updated MDPI and main.py * Updated MDPI.py and main.py * Update README.md * Main fixes (#20) * minor Dockerfile fixes in COPY statements * updated CLI options and their tests * changed input file and input parsing to better reflect expected input * Update README.md * Updated main * Updted main and sub class for logger * Updated ArXive date * Standardize CrossRef publication date * Standardize Elsevier publication date * Standardize MDPI publication date * Standardize PLOS publication date * Standardize PubMed publication date * Standardize Springer publication date * Standardize Wiley publication date * Standardize the publication date to "YYYY-MM-DD" * Updated main.py to allows user to pull publication from cutoff date and test for main.py * Added Documentations for xlsx * updated README for flag `--cutoff_date` or `-cd` * Update README.md * Update README.md * Added mock responses for test_Elsevier.py * Added Test for test_CrossRef.py * Update README.md * Docker io (#35) * script now reads a (default) xlsx file instead of a csv * added run.sh script (WIP) * added .env support and example for storing secrets * Update README.md * refactored API list to a const in main.py * changed --input to click.Path object * removed old files from Dockerfile * moved input and default input/output files value to /data directory, mount /data directory to docker container instead of the whole project * quick poetry.lock fix * Updated MDPI for pull date * Added logging.debug for MDPI.py * Updated code for pull date (CrossRef.py) * Updated code for Elsevire.py (date and added logging.debug) * Added code for loggin.debug * Added logging.debug for PLOS.py * Updated pull date from PubMed.py * Added logging.debug to Spinger.py * Added loggin.debug to Wiley.py * Updated authors naming * Bumpversion fixes (#37) * added .env version spec to pyproject; bumpversion now updates .env * Update README.md * got rid of /data folder (too confusing) * Update README.md * Added testing for test_MDPI.py * Updated test_Elsevier.py to use .env * Added testting for test_Elsevire.py * Wiley tests (#42) * added Wiley tests (WIP) * added Wiley tests * Development (#41) * Added testing for test_MDPI.py * Updated test_Elsevier.py to use .env * Added testting for test_Elsevire.py --------- Co-authored-by: Magret <[email protected]> Co-authored-by: Magret Adekunle <[email protected]> * fixed main.py tests (all tests pass!) --------- Co-authored-by: Magret <[email protected]> Co-authored-by: Magret Adekunle <[email protected]> * Runner versioning (#43) * run.sh now properly reads IMAGEVERSION environment variable from .env for building the container * removed old secrets sample, edited IMAGEVERSION in .env.sample to match our versioning scheme * changed help text for --list option for clarity * updated help text for test_main * Cleanup (#45) * added callback for --list option * Update README.md * Updated hard-coded vars to config.py * bumped version * added actions yaml files (#48) * Actions (#50) * added actions yaml files * corrected job name in docker-image.yml --------- Co-authored-by: Joseph Hendrix <[email protected]> Co-authored-by: Joseph Hendrix <[email protected]> Co-authored-by: Magret <[email protected]> Co-authored-by: Magret Adekunle <[email protected]> * env fixes * env directory fixes --------- Co-authored-by: Joseph Hendrix <[email protected]> Co-authored-by: Joseph Hendrix <[email protected]> Co-authored-by: Magret <[email protected]> Co-authored-by: Magret Adekunle <[email protected]>
- Loading branch information
5 people
authored
Dec 17, 2024
1 parent
7e2c4d6
commit 953fb46