Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Presets #1058

Merged
merged 294 commits into from
Apr 26, 2024
Merged

Presets #1058

Changes from 1 commit
Commits
Show all changes
294 commits
Select commit Hold shift + click to select a range
edd2418
steady work on presets - module loader tests
TheTechromancer Mar 15, 2024
d3a9431
Merge branch 'dev' into presets
TheTechromancer Mar 15, 2024
ed4083d
Added launch.json for Live Debugging
stryker2k2 Mar 15, 2024
67b84a9
fix deps error
TheTechromancer Mar 15, 2024
1ad6a58
don't cache module python code
TheTechromancer Mar 15, 2024
9417025
fixed preload find-and-replace
TheTechromancer Mar 17, 2024
4377dda
fix scope bug
TheTechromancer Mar 17, 2024
e2f1707
create preload cache parent dir
TheTechromancer Mar 17, 2024
2c0590f
fix module test tests
TheTechromancer Mar 18, 2024
e619631
tolerant preset loading
TheTechromancer Mar 18, 2024
f65be4a
prep scan before enter prompt
TheTechromancer Mar 18, 2024
d357851
remove agent functionality
TheTechromancer Mar 18, 2024
fd82211
recursive preset inclusion feature
TheTechromancer Mar 18, 2024
1ab4277
social dependency
TheTechromancer Mar 19, 2024
c3d3f1b
merge dev
TheTechromancer Mar 19, 2024
ec2b35c
recursive inclusion, circular inclusion tests
TheTechromancer Mar 20, 2024
40ff3ba
cli arg tests passing
TheTechromancer Mar 20, 2024
7acc625
more cli tests
TheTechromancer Mar 20, 2024
0f50fd0
Merge branch 'dev' into presets
TheTechromancer Mar 20, 2024
1f2600e
better inclusion tests
TheTechromancer Mar 21, 2024
cf23bba
better cli tests, preset+cli tests passing
TheTechromancer Mar 22, 2024
9549091
preset conditions + tests
TheTechromancer Mar 22, 2024
0951a21
Merge branch 'dev' into presets
TheTechromancer Mar 25, 2024
7254273
merge dev
TheTechromancer Mar 25, 2024
7817365
starting work on documentation
TheTechromancer Mar 25, 2024
08e68c5
steady work on presets
TheTechromancer Mar 25, 2024
60aa0da
steady work on docs
TheTechromancer Mar 26, 2024
fd467c6
preset symlinking
TheTechromancer Mar 26, 2024
91da2cf
ensure presets have descriptions
TheTechromancer Mar 26, 2024
0766a24
Merge branch 'dev' into presets
TheTechromancer Mar 26, 2024
b6e6c9f
steady work on tests
TheTechromancer Mar 27, 2024
9de5048
tests stuff
TheTechromancer Mar 27, 2024
72e55b0
and more tests
TheTechromancer Mar 27, 2024
d1f7fa6
more tests
TheTechromancer Mar 27, 2024
f15f2be
more tests
TheTechromancer Mar 27, 2024
9885816
fix cloud bucket tests
TheTechromancer Mar 27, 2024
4730fce
fix excavate tests
TheTechromancer Mar 27, 2024
983ccd6
fix ffuf shortnames tests
TheTechromancer Mar 27, 2024
3b6192a
fix nmap, github_org tests
TheTechromancer Mar 27, 2024
bdb3f61
merge dev
TheTechromancer Mar 27, 2024
91c621c
more preset tests
TheTechromancer Mar 27, 2024
f8392b7
fix tests
TheTechromancer Mar 28, 2024
79039de
fix scan tests
TheTechromancer Mar 28, 2024
73b0602
steady work on tests
TheTechromancer Mar 28, 2024
251656a
fix tests
TheTechromancer Mar 28, 2024
84ff604
more tests
TheTechromancer Mar 28, 2024
43b3967
more presets
TheTechromancer Mar 28, 2024
7aad074
WIP developer docs
TheTechromancer Mar 28, 2024
a732efa
steady work on docs
TheTechromancer Mar 28, 2024
9031e26
fix tests
TheTechromancer Mar 29, 2024
b1c0200
better preset baking, more docs
TheTechromancer Mar 29, 2024
86b798e
blacked
TheTechromancer Mar 29, 2024
b7cacdb
more docs
TheTechromancer Mar 29, 2024
8ccd439
add core.md
TheTechromancer Mar 29, 2024
fd258ba
fix docs
TheTechromancer Mar 29, 2024
781bcc4
more docstrings
TheTechromancer Mar 31, 2024
2a735af
rename output.ndjson --> output.json
TheTechromancer Mar 31, 2024
da477e1
fix tests
TheTechromancer Mar 31, 2024
a00d8fc
Merge pull request #1227 from blacklanternsecurity/rename-ndjson
TheTechromancer Apr 1, 2024
24acdb7
let there be dns engine
TheTechromancer Apr 2, 2024
2649425
blacked
TheTechromancer Apr 2, 2024
ec2f89a
add pyzmq dependency
TheTechromancer Apr 2, 2024
42232d0
fix dns mocker
TheTechromancer Apr 2, 2024
2a19417
fix test_event
TheTechromancer Apr 2, 2024
cd015a6
working on tests
TheTechromancer Apr 2, 2024
45f8f1a
more work on tests
TheTechromancer Apr 2, 2024
98bbb73
still working on tests
TheTechromancer Apr 2, 2024
7bc49b0
logging tweaks
TheTechromancer Apr 2, 2024
7a7224f
more tests
TheTechromancer Apr 2, 2024
9d2c093
reintroduce dns parallelism
TheTechromancer Apr 3, 2024
f257fe6
performance improvements
TheTechromancer Apr 3, 2024
9a29253
fix massdns tests
TheTechromancer Apr 3, 2024
f6f1223
remove duplicate preset list
TheTechromancer Apr 3, 2024
d8deedc
validate http_response url
TheTechromancer Apr 3, 2024
d82eaff
blacked
TheTechromancer Apr 3, 2024
6c684a0
fix tests
TheTechromancer Apr 3, 2024
3fd47c2
restore verbosity toggling
TheTechromancer Apr 3, 2024
1108736
ascii art
TheTechromancer Apr 3, 2024
39949bd
move errors.py
TheTechromancer Apr 3, 2024
8ce6e5d
merge dev
TheTechromancer Apr 3, 2024
fe618b9
commit errors.py
TheTechromancer Apr 3, 2024
795a76e
Merge branch 'dev' into validate-httpresponse-url
stryker2k2 Apr 4, 2024
35ecd6c
faster event dns caching
TheTechromancer Apr 4, 2024
ba5b4ef
merge presets
TheTechromancer Apr 4, 2024
19329cb
remove obsolete massdns logic
TheTechromancer Apr 4, 2024
b54b567
add logger.py
TheTechromancer Apr 4, 2024
8c1ba88
remove vscode artifacts
TheTechromancer Apr 4, 2024
678d4ff
Merge branch 'presets' into dns-engine
TheTechromancer Apr 4, 2024
465e68d
remove redundant v in version
TheTechromancer Apr 4, 2024
16fe38a
version begins with v
TheTechromancer Apr 4, 2024
e24dbca
Merge pull request #1233 from blacklanternsecurity/validate-httprespo…
stryker2k2 Apr 4, 2024
74aa6fa
fix output module slowness
TheTechromancer Apr 4, 2024
4c35402
again
TheTechromancer Apr 4, 2024
efab083
merge presets
TheTechromancer Apr 5, 2024
eac3f12
Merge branch 'dev' into presets
TheTechromancer Apr 5, 2024
3fa7c66
Merge branch 'presets' into dns-engine
TheTechromancer Apr 5, 2024
fdd4c6e
remove custom stdout settings until better solution
TheTechromancer Apr 5, 2024
8e3c940
Merge branch 'presets' into dns-engine
TheTechromancer Apr 5, 2024
8223aa3
better cli error handling
TheTechromancer Apr 5, 2024
0f235bc
merge presets
TheTechromancer Apr 5, 2024
04bc206
prune unnecessary asyncio tasks
TheTechromancer Apr 7, 2024
9c431b8
Merge branch 'presets' into dns-engine
TheTechromancer Apr 7, 2024
f899bb9
remove unnecessary checks in deduplication tests
TheTechromancer Apr 7, 2024
c503557
Merge branch 'presets' into dns-engine
TheTechromancer Apr 7, 2024
60049ac
make sure zmq sockets are always cleaned up at the end of a scan
TheTechromancer Apr 7, 2024
d3e6247
Bump lxml from 5.2.0 to 5.2.1
dependabot[bot] Apr 8, 2024
c339568
Bump werkzeug from 3.0.1 to 3.0.2
dependabot[bot] Apr 8, 2024
4749bfc
Bump mkdocstrings-python from 1.9.0 to 1.9.2
dependabot[bot] Apr 8, 2024
b8ac41d
Bump mkdocstrings from 0.24.1 to 0.24.3
dependabot[bot] Apr 8, 2024
88bc77c
Bump mkdocs-material from 9.5.16 to 9.5.17
dependabot[bot] Apr 8, 2024
35ab8f8
Merge pull request #1237 from blacklanternsecurity/dependabot/pip/dev…
TheTechromancer Apr 8, 2024
9d0a667
Merge pull request #1238 from blacklanternsecurity/dependabot/pip/dev…
TheTechromancer Apr 8, 2024
6998f59
commit discord bot example
TheTechromancer Apr 8, 2024
b2121a2
WIP module hooks
TheTechromancer Apr 8, 2024
58f989a
Merge branch 'presets' into dns-engine
TheTechromancer Apr 8, 2024
95c677b
Merge branch 'dns-engine' into module-hooks
TheTechromancer Apr 8, 2024
84074c1
WIP hook modules
TheTechromancer Apr 8, 2024
07a4cbd
continued work on hooks
TheTechromancer Apr 9, 2024
b1a8c23
more work on hooks
TheTechromancer Apr 9, 2024
9b2ab0d
Merge pull request #1239 from blacklanternsecurity/dependabot/pip/dev…
TheTechromancer Apr 9, 2024
b26fd26
Merge pull request #1240 from blacklanternsecurity/dependabot/pip/dev…
TheTechromancer Apr 9, 2024
490bd36
more wip hooks
TheTechromancer Apr 9, 2024
ff194d0
Merge pull request #1241 from blacklanternsecurity/dependabot/pip/dev…
TheTechromancer Apr 10, 2024
b8d204f
fix tabulate bug
TheTechromancer Apr 10, 2024
7f58f31
merge dev
TheTechromancer Apr 10, 2024
35baf56
Merge branch 'presets' into dns-engine
TheTechromancer Apr 10, 2024
1ee6dc0
flaked
TheTechromancer Apr 10, 2024
ef52bc7
Merge branch 'presets' into dns-engine
TheTechromancer Apr 10, 2024
7d73633
nuclei - fix tests
TheTechromancer Apr 10, 2024
67c2c4a
Merge pull request #1249 from blacklanternsecurity/nuclei-budget-fix
TheTechromancer Apr 10, 2024
c631cfe
better error handling for download-related tasks
TheTechromancer Apr 10, 2024
5a427e7
Merge branch 'dev' into tabulate-fix
TheTechromancer Apr 10, 2024
8c24ea8
Merge branch 'dev' into presets
TheTechromancer Apr 10, 2024
7f9ba9d
Merge branch 'presets' into dns-engine
TheTechromancer Apr 10, 2024
16701c6
Merge branch 'dns-engine' into module-hooks
TheTechromancer Apr 10, 2024
34b5d3e
more work on hooks
TheTechromancer Apr 11, 2024
eae2dad
remove debugging statement
TheTechromancer Apr 11, 2024
60ffeff
Merge pull request #1253 from blacklanternsecurity/better-download-er…
TheTechromancer Apr 11, 2024
1165cfd
remove return statement in scope accuracy tests
TheTechromancer Apr 11, 2024
91ced12
Merge pull request #1247 from blacklanternsecurity/tabulate-fix
TheTechromancer Apr 11, 2024
bed5e54
Merge branch 'dev' into presets
TheTechromancer Apr 11, 2024
5ea0e61
Merge branch 'presets' into dns-engine
TheTechromancer Apr 11, 2024
ee9483d
Merge branch 'dns-engine' into module-hooks
TheTechromancer Apr 11, 2024
db2733c
adjusting paramminer error message
liquidsec Apr 14, 2024
9f7fd14
black
liquidsec Apr 14, 2024
cb73745
Merge pull request #1259 from blacklanternsecurity/paramminer-group-w…
liquidsec Apr 14, 2024
17598f7
dns tests passing
TheTechromancer Apr 14, 2024
dfd7b88
cloudcheck defragmentation
TheTechromancer Apr 15, 2024
315f251
Bump black from 24.3.0 to 24.4.0
dependabot[bot] Apr 15, 2024
33e416b
Bump deepdiff from 6.7.1 to 7.0.1
dependabot[bot] Apr 15, 2024
f4b963e
Bump pydantic from 2.6.4 to 2.7.0
dependabot[bot] Apr 15, 2024
bdfc567
Bump idna from 3.6 to 3.7
dependabot[bot] Apr 15, 2024
fe32041
Merge pull request #1260 from blacklanternsecurity/dependabot/pip/dev…
TheTechromancer Apr 15, 2024
badafcf
Merge pull request #1262 from blacklanternsecurity/dependabot/pip/dev…
TheTechromancer Apr 15, 2024
06a4d38
dns module tests
TheTechromancer Apr 15, 2024
ebb7295
fix dns tests
TheTechromancer Apr 15, 2024
c4e1a19
blacked
TheTechromancer Apr 15, 2024
f349526
Merge pull request #1263 from blacklanternsecurity/dependabot/pip/dev…
TheTechromancer Apr 15, 2024
50ba47f
Merge branch 'dev' into presets
TheTechromancer Apr 15, 2024
4f2e2e7
Merge branch 'presets' into dns-engine
TheTechromancer Apr 15, 2024
6b262a6
Merge branch 'dns-engine' into module-hooks
TheTechromancer Apr 15, 2024
db3961c
fix module tests
TheTechromancer Apr 15, 2024
b2679d2
quick emit revisit
TheTechromancer Apr 15, 2024
5746f6e
blacked
TheTechromancer Apr 15, 2024
3b0efcf
fix preset tests
TheTechromancer Apr 15, 2024
30f325f
fix affiliate tests
TheTechromancer Apr 15, 2024
b29356b
fix tests
TheTechromancer Apr 15, 2024
899e9e8
Merge pull request #1261 from blacklanternsecurity/dependabot/pip/dev…
TheTechromancer Apr 15, 2024
37d1a6e
fix bucket tests
TheTechromancer Apr 15, 2024
56a1557
fix csv tests
TheTechromancer Apr 15, 2024
ffada9e
rename HookModule --> InterceptModule
TheTechromancer Apr 16, 2024
d90b2b1
let there be shared module dependencies
TheTechromancer Apr 16, 2024
307aa11
add docker shared dependency
TheTechromancer Apr 16, 2024
22f877e
hook --> intercept
TheTechromancer Apr 16, 2024
fe8d2ef
Merge branch 'module-hooks' into shared-module-deps
TheTechromancer Apr 16, 2024
e92d92b
handle punycode bucket mutations
TheTechromancer Apr 17, 2024
56289e8
merge dev
TheTechromancer Apr 17, 2024
1f5db13
flaked
TheTechromancer Apr 17, 2024
f52bd3e
better storage bucket sanitization
TheTechromancer Apr 17, 2024
f56f0e9
merge dev
TheTechromancer Apr 17, 2024
06d278e
merge presets
TheTechromancer Apr 17, 2024
e884797
Merge branch 'dns-engine' into module-hooks
TheTechromancer Apr 17, 2024
3df1d63
fix inconsistency with dns host speculation
TheTechromancer Apr 17, 2024
cceb6ba
Merge branch 'module-hooks' into shared-module-deps
TheTechromancer Apr 17, 2024
378ee8a
fix tests
TheTechromancer Apr 17, 2024
d6511de
update poetry.lock
TheTechromancer Apr 17, 2024
e7021fa
Merge branch 'dns-engine' into module-hooks
TheTechromancer Apr 17, 2024
2e0e4ac
Merge branch 'module-hooks' into shared-module-deps
TheTechromancer Apr 17, 2024
3334efb
Merge pull request #1273 from blacklanternsecurity/punycode-bucket-mu…
TheTechromancer Apr 17, 2024
32145c5
fix attribute error
TheTechromancer Apr 17, 2024
36e3975
WIP regex optimization
TheTechromancer Apr 17, 2024
71e0c23
remove debug statement
TheTechromancer Apr 17, 2024
8b4d4ed
Merge branch 'module-hooks' into shared-module-deps
TheTechromancer Apr 17, 2024
f922cbb
Merge branch 'shared-module-deps' into faster-regexes
TheTechromancer Apr 17, 2024
697a188
Add an option to specify event types
domwhewell-sage Apr 18, 2024
d0ab202
Made changes to target to regex for more robust matching
domwhewell-sage Apr 18, 2024
4fd95d4
Linting
domwhewell-sage Apr 18, 2024
8717ea4
more WIP regex optimizations
TheTechromancer Apr 19, 2024
152bda0
moved regex into `add_target()`
domwhewell-sage Apr 19, 2024
7b332d7
Handle `FileNotFoundError`
domwhewell-sage Apr 19, 2024
e7617cb
Added if statment to override scope check if org is defined in target
domwhewell-sage Apr 19, 2024
9810305
steady work on regexes
TheTechromancer Apr 19, 2024
1a9db31
Merge pull request #1283 from domwhewell-sage/manual-target-types
TheTechromancer Apr 19, 2024
ed15066
Merge branch 'dev' into dev
domwhewell-sage Apr 19, 2024
da62adf
Merge pull request #1287 from domwhewell-sage/fix-dastardly-error
TheTechromancer Apr 19, 2024
7899408
don't start engine until necessary
TheTechromancer Apr 19, 2024
7c22371
Merge pull request #1288 from domwhewell-sage/dev
TheTechromancer Apr 19, 2024
0b074a2
merge dev
TheTechromancer Apr 19, 2024
46a8d1c
re-add poetry.lock
TheTechromancer Apr 19, 2024
829441b
merge presets
TheTechromancer Apr 19, 2024
7c75522
merge dns-engine
TheTechromancer Apr 19, 2024
145c2a2
Merge branch 'module-hooks' into shared-module-deps
TheTechromancer Apr 19, 2024
225b963
merge shared-module-deps
TheTechromancer Apr 19, 2024
3b460fd
fix merge
TheTechromancer Apr 19, 2024
58a2bc7
Merge branch 'presets' into dns-engine
TheTechromancer Apr 19, 2024
1eaadff
Merge branch 'dns-engine' into module-hooks
TheTechromancer Apr 19, 2024
9d4b972
Merge branch 'module-hooks' into shared-module-deps
TheTechromancer Apr 19, 2024
c0076ae
Merge branch 'shared-module-deps' into faster-regexes
TheTechromancer Apr 19, 2024
0f6240c
update cloudcheck
TheTechromancer Apr 19, 2024
e6c507f
tests for custom target types
TheTechromancer Apr 19, 2024
a6a7ade
fix tests
TheTechromancer Apr 19, 2024
eaf2cdf
implement radixtarget
TheTechromancer Apr 22, 2024
af110c9
better scope tests
TheTechromancer Apr 22, 2024
8f72db7
blacked
TheTechromancer Apr 22, 2024
4f07312
update cloudcheck
TheTechromancer Apr 22, 2024
53f71e9
fix cloudcheck
TheTechromancer Apr 22, 2024
a3c8e61
better target tests
TheTechromancer Apr 22, 2024
d42c189
fix typo
TheTechromancer Apr 22, 2024
8c07684
better dns name sanitization
TheTechromancer Apr 22, 2024
e4fd60a
fix ffuf tests
TheTechromancer Apr 22, 2024
1c1ab26
Merge pull request #1277 from blacklanternsecurity/proc-error
TheTechromancer Apr 23, 2024
faf61ee
small scope tweak
TheTechromancer Apr 23, 2024
e79d255
Better debugging during scan cancellation
TheTechromancer Apr 23, 2024
d158fa9
fix small cli bug and add tests for it
TheTechromancer Apr 23, 2024
630c87e
remove resolved/unresolved tags as they are redundant
TheTechromancer Apr 23, 2024
64a0c18
Merge pull request #1296 from blacklanternsecurity/better-cancellatio…
TheTechromancer Apr 23, 2024
1dd8ab2
Merge branch 'dev' into presets
TheTechromancer Apr 23, 2024
f53a0ac
Merge branch 'presets' into dns-engine
TheTechromancer Apr 23, 2024
2f0339c
Merge branch 'dns-engine' into module-hooks
TheTechromancer Apr 23, 2024
40a066d
Merge branch 'module-hooks' into shared-module-deps
TheTechromancer Apr 23, 2024
cc5de28
merge shared-module-deps
TheTechromancer Apr 23, 2024
c770d83
better engine error handling during scan cancellation
TheTechromancer Apr 23, 2024
1a45c9c
Merge branch 'dns-engine' into module-hooks
TheTechromancer Apr 23, 2024
ecad649
Merge branch 'module-hooks' into shared-module-deps
TheTechromancer Apr 23, 2024
7baf219
Merge branch 'shared-module-deps' into faster-regexes
TheTechromancer Apr 23, 2024
6feb345
Merge pull request #1295 from blacklanternsecurity/radixtarget
TheTechromancer Apr 26, 2024
fad1afe
Merge pull request #1278 from blacklanternsecurity/faster-regexes
TheTechromancer Apr 26, 2024
92e2fd3
Merge pull request #1270 from blacklanternsecurity/shared-module-deps
TheTechromancer Apr 26, 2024
18694c1
Merge pull request #1242 from blacklanternsecurity/module-hooks
TheTechromancer Apr 26, 2024
81ba613
Merge pull request #1231 from blacklanternsecurity/dns-engine
TheTechromancer Apr 26, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
social dependency
  • Loading branch information
TheTechromancer committed Mar 19, 2024
commit 1ab42770765632e660505ec5c328cfdbd1cca76d
2 changes: 1 addition & 1 deletion bbot/core/modules.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ class ModuleLoader:
module_dir_regex = re.compile(r"^[a-z][a-z0-9_]*$")

# if a module consumes these event types, automatically assume these dependencies
default_module_deps = {"HTTP_RESPONSE": "httpx", "URL": "httpx"}
default_module_deps = {"HTTP_RESPONSE": "httpx", "URL": "httpx", "SOCIAL": "social"}

def __init__(self):
self.core = CORE
Expand Down