New Module "github_org" + various GitHub improvements#902

Merged

TheTechromancer merged 17 commits intoblacklanternsecurity:devfrom domwhewell-sage:github-improvements

Dec 21, 2023

+801-139

Commits on Dec 14, 2023

Added a function to validate a github organization is in-scope

domwhewell-sage
authored and
TheTechromancer
committed
Updated github to query an orgs repos and then pillage that repo for all raw urls. Made a test to fake responses for blacklanternsecurity/test_keys repo

domwhewell-sage
authored and
TheTechromancer
committed
Corrected function name and call in github.py

domwhewell-sage
authored and
TheTechromancer
committed
Allow recursive searching of the repo for files

domwhewell-sage
authored and
TheTechromancer
committed
Changed requests that were paginated that did not require it

domwhewell-sage
authored and
TheTechromancer
committed
Changed testcase to account for pagination

domwhewell-sage
authored and
TheTechromancer
committed
Added headers to repo contents request

domwhewell-sage
authored and
TheTechromancer
committed
Uncommented query code

domwhewell-sage
authored and
TheTechromancer
committed
Modified github test, to test the recursive find inside repos and to check the correct number of repos and files are discovered

domwhewell-sage
authored and
TheTechromancer
committed
Split github module into two and a template

domwhewell-sage
authored and
TheTechromancer
committed
Changed github codesearch to emit both CODE_REPOSITORY and URL_UNVERIFIED events

domwhewell-sage
authored and
TheTechromancer
committed
re-formatted github_org and the github template

domwhewell-sage
authored and
TheTechromancer
committed
Corrected missing page counter on test module

domwhewell-sage
authored and
TheTechromancer
committed
Created an optional_api_key async function to check if an API key has been supplied for any github module

domwhewell-sage
authored and
TheTechromancer
committed
added ORG_STUB event type
TheTechromancer
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New Module "github_org" + various GitHub improvements#902

Commits on Dec 14, 2023

Added a function to validate a github organization is in-scope

Updated github to query an orgs repos and then pillage that repo for all raw urls. Made a test to fake responses for blacklanternsecurity/test_keys repo

Corrected function name and call in github.py

Allow recursive searching of the repo for files

Changed requests that were paginated that did not require it

Changed testcase to account for pagination

Added headers to repo contents request

Uncommented query code

Modified github test, to test the recursive find inside repos and to check the correct number of repos and files are discovered

Split github module into two and a template

Changed github codesearch to emit both CODE_REPOSITORY and URL_UNVERIFIED events

re-formatted github_org and the github template

Corrected missing page counter on test module

Created an optional_api_key async function to check if an API key has been supplied for any github module

added ORG_STUB event type

Commits on Dec 15, 2023

only emit ORG_STUB for registered domains

fix social test