Skip to content

Commit

Permalink
Merge branch 'dev' into service-record
Browse files Browse the repository at this point in the history
  • Loading branch information
TheTechromancer authored Aug 12, 2024
2 parents 46cac6c + cdfc484 commit b5d51b6
Show file tree
Hide file tree
Showing 121 changed files with 2,907 additions and 1,496 deletions.
5 changes: 5 additions & 0 deletions .flake8
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
[flake8]
select = F,E722
ignore = F403,F405,F541
per-file-ignores =
*/__init__.py:F401,F403
29 changes: 26 additions & 3 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,8 @@ jobs:
needs: lint
runs-on: ubuntu-latest
strategy:
# if one python version fails, let the others finish
fail-fast: false
matrix:
python-version: ["3.9", "3.10", "3.11", "3.12"]
steps:
Expand Down Expand Up @@ -76,14 +78,14 @@ jobs:
uses: EndBug/add-and-commit@v9
continue-on-error: true
with:
add: "*.md"
add: '["*.md", "docs/data/chord_graph/*.json"]'
author_name: "BBOT Docs Autopublish"
author_email: [email protected]
message: "Refresh module docs"
publish_docs:
needs: update_docs
runs-on: ubuntu-latest
if: github.event_name == 'push' && (github.ref == 'refs/heads/stable')
if: github.event_name == 'push' && (github.ref == 'refs/heads/stable' || github.ref == 'refs/heads/dev')
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
Expand All @@ -100,7 +102,28 @@ jobs:
run: |
pip install poetry
poetry install --only=docs
- run: poetry run mkdocs gh-deploy --force
- name: Configure Git
run: |
git config user.name github-actions
git config user.email [email protected]
git fetch origin gh-pages:refs/remotes/origin/gh-pages
if git show-ref --verify --quiet refs/heads/gh-pages; then
git branch -f gh-pages origin/gh-pages
else
git branch --track gh-pages origin/gh-pages
fi
- name: Generate docs (stable branch)
if: github.ref == 'refs/heads/stable'
run: |
poetry run mike deploy Stable
- name: Generate docs (dev branch)
if: github.ref == 'refs/heads/dev'
run: |
poetry run mike deploy Dev
- name: Publish docs
run: |
git switch gh-pages
git push
publish_code:
needs: update_docs
runs-on: ubuntu-latest
Expand Down
115 changes: 59 additions & 56 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[![bbot_banner](https://user-images.githubusercontent.com/20261699/158000235-6c1ace81-a267-4f8e-90a1-f4c16884ebac.png)](https://github.com/blacklanternsecurity/bbot)
[![bbot_banner](https://github.com/user-attachments/assets/f02804ce-9478-4f1e-ac4d-9cf5620a3214)](https://github.com/blacklanternsecurity/bbot)

[![Python Version](https://img.shields.io/badge/python-3.9+-FF8400)](https://www.python.org) [![License](https://img.shields.io/badge/license-GPLv3-FF8400.svg)](https://github.com/blacklanternsecurity/bbot/blob/dev/LICENSE) [![DEF CON Demo Labs 2023](https://img.shields.io/badge/DEF%20CON%20Demo%20Labs-2023-FF8400.svg)](https://forum.defcon.org/node/246338) [![PyPi Downloads](https://static.pepy.tech/personalized-badge/bbot?right_color=orange&left_color=grey)](https://pepy.tech/project/bbot) [![Black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black) [![Tests](https://github.com/blacklanternsecurity/bbot/actions/workflows/tests.yml/badge.svg?branch=stable)](https://github.com/blacklanternsecurity/bbot/actions?query=workflow%3A"tests") [![Codecov](https://codecov.io/gh/blacklanternsecurity/bbot/branch/dev/graph/badge.svg?token=IR5AZBDM5K)](https://codecov.io/gh/blacklanternsecurity/bbot) [![Discord](https://img.shields.io/discord/859164869970362439)](https://discord.com/invite/PZqkgxu5SA)
[![Python Version](https://img.shields.io/badge/python-3.9+-FF8400)](https://www.python.org) [![License](https://img.shields.io/badge/license-GPLv3-FF8400.svg)](https://github.com/blacklanternsecurity/bbot/blob/dev/LICENSE) [![DEF CON Recon Village 2024](https://img.shields.io/badge/DEF%20CON%20Demo%20Labs-2023-FF8400.svg)](https://www.reconvillage.org/talks) [![PyPi Downloads](https://static.pepy.tech/personalized-badge/bbot?right_color=orange&left_color=grey)](https://pepy.tech/project/bbot) [![Black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black) [![Tests](https://github.com/blacklanternsecurity/bbot/actions/workflows/tests.yml/badge.svg?branch=stable)](https://github.com/blacklanternsecurity/bbot/actions?query=workflow%3A"tests") [![Codecov](https://codecov.io/gh/blacklanternsecurity/bbot/branch/dev/graph/badge.svg?token=IR5AZBDM5K)](https://codecov.io/gh/blacklanternsecurity/bbot) [![Discord](https://img.shields.io/discord/859164869970362439)](https://discord.com/invite/PZqkgxu5SA)

### **BEE·bot** is a multipurpose scanner inspired by [Spiderfoot](https://github.com/smicallef/spiderfoot), built to automate your **Recon**, **Bug Bounties**, and **ASM**!

Expand All @@ -18,7 +18,7 @@ pipx install bbot
pipx install --pip-args '\--pre' bbot
```

_For more installation methods, including [Docker](https://hub.docker.com/r/blacklanternsecurity/bbot), see [Getting Started](https://www.blacklanternsecurity.com/bbot/)_
_For more installation methods, including [Docker](https://hub.docker.com/r/blacklanternsecurity/bbot), see [Getting Started](https://www.blacklanternsecurity.com/bbot/Stable/)_

## Example Commands

Expand Down Expand Up @@ -66,7 +66,7 @@ config:

<!-- END BBOT SUBDOMAIN-ENUM PRESET EXPANDABLE -->

BBOT consistently finds 20-50% more subdomains than other tools. The bigger the domain, the bigger the difference. To learn how this is possible, see [How It Works](https://www.blacklanternsecurity.com/bbot/how_it_works/).
BBOT consistently finds 20-50% more subdomains than other tools. The bigger the domain, the bigger the difference. To learn how this is possible, see [How It Works](https://www.blacklanternsecurity.com/bbot/Dev/how_it_works/).

![subdomain-stats-ebay](https://github.com/blacklanternsecurity/bbot/assets/20261699/de3e7f21-6f52-4ac4-8eab-367296cd385f)

Expand Down Expand Up @@ -228,19 +228,20 @@ config:

## How it Works

Click the graph below to explore the [inner workings](https://www.blacklanternsecurity.com/bbot/how_it_works/) of BBOT.
Click the graph below to explore the [inner workings](https://www.blacklanternsecurity.com/bbot/Stable/how_it_works/) of BBOT.

[![image](https://github.com/blacklanternsecurity/bbot/assets/20261699/e55ba6bd-6d97-48a6-96f0-e122acc23513)](https://www.blacklanternsecurity.com/bbot/how_it_works/)
[![image](https://github.com/blacklanternsecurity/bbot/assets/20261699/e55ba6bd-6d97-48a6-96f0-e122acc23513)](https://www.blacklanternsecurity.com/bbot/Stable/how_it_works/)

## BBOT as a Python Library

#### Synchronous
```python
from bbot.scanner import Scanner

scan = Scanner("evilcorp.com", presets=["subdomain-enum"])
for event in scan.start():
print(event)
if __name__ == "__main__":
scan = Scanner("evilcorp.com", presets=["subdomain-enum"])
for event in scan.start():
print(event)
```

#### Asynchronous
Expand All @@ -252,14 +253,15 @@ async def main():
async for event in scan.async_start():
print(event.json())

import asyncio
asyncio.run(main())
if __name__ == "__main__":
import asyncio
asyncio.run(main())
```

<details>
<summary><b>SEE: This Nefarious Discord Bot</b></summary>

A [BBOT Discord Bot](https://www.blacklanternsecurity.com/bbot/dev/discord_bot/) that responds to the `/scan` command. Scan the internet from the comfort of your discord server!
A [BBOT Discord Bot](https://www.blacklanternsecurity.com/bbot/Stable/dev/#discord-bot-example) that responds to the `/scan` command. Scan the internet from the comfort of your discord server!

![bbot-discord](https://github.com/blacklanternsecurity/bbot/assets/20261699/22b268a2-0dfd-4c2a-b7c5-548c0f2cc6f9)

Expand Down Expand Up @@ -292,7 +294,7 @@ Targets can be any of the following:
- `OPEN_TCP_PORT` (`192.168.0.1:80`)
- `URL` (`https://www.evilcorp.com`)

For more information, see [Targets](https://www.blacklanternsecurity.com/bbot/scanning/#targets-t). To learn how BBOT handles scope, see [Scope](https://www.blacklanternsecurity.com/bbot/scanning/#scope).
For more information, see [Targets](https://www.blacklanternsecurity.com/bbot/Stable/scanning/#targets-t). To learn how BBOT handles scope, see [Scope](https://www.blacklanternsecurity.com/bbot/Stable/scanning/#scope).

## API Keys

Expand All @@ -316,70 +318,71 @@ If you like, you can also specify them on the command line:
bbot -c modules.virustotal.api_key=dd5f0eee2e4a99b71a939bded450b246
```

For details, see [Configuration](https://www.blacklanternsecurity.com/bbot/scanning/configuration/).
For details, see [Configuration](https://www.blacklanternsecurity.com/bbot/Stable/scanning/configuration/).

## Complete Lists of Modules, Flags, etc.

- Complete list of [Modules](https://www.blacklanternsecurity.com/bbot/modules/list_of_modules/).
- Complete list of [Flags](https://www.blacklanternsecurity.com/bbot/scanning/#list-of-flags).
- Complete list of [Presets](https://www.blacklanternsecurity.com/bbot/scanning/presets_list/).
- Complete list of [Global Config Options](https://www.blacklanternsecurity.com/bbot/scanning/configuration/#global-config-options).
- Complete list of [Module Config Options](https://www.blacklanternsecurity.com/bbot/scanning/configuration/#module-config-options).
- Complete list of [Modules](https://www.blacklanternsecurity.com/bbot/Stable/modules/list_of_modules/).
- Complete list of [Flags](https://www.blacklanternsecurity.com/bbot/Stable/scanning/#list-of-flags).
- Complete list of [Presets](https://www.blacklanternsecurity.com/bbot/Stable/scanning/presets_list/).
- Complete list of [Global Config Options](https://www.blacklanternsecurity.com/bbot/Stable/scanning/configuration/#global-config-options).
- Complete list of [Module Config Options](https://www.blacklanternsecurity.com/bbot/Stable/scanning/configuration/#module-config-options).

## Documentation

<!-- BBOT DOCS TOC -->
- **User Manual**
- **Basics**
- [Getting Started](https://www.blacklanternsecurity.com/bbot/)
- [How it Works](https://www.blacklanternsecurity.com/bbot/how_it_works)
- [Comparison to Other Tools](https://www.blacklanternsecurity.com/bbot/comparison)
- [Getting Started](https://www.blacklanternsecurity.com/bbot/Stable/)
- [How it Works](https://www.blacklanternsecurity.com/bbot/Stable/how_it_works)
- [Comparison to Other Tools](https://www.blacklanternsecurity.com/bbot/Stable/comparison)
- **Scanning**
- [Scanning Overview](https://www.blacklanternsecurity.com/bbot/scanning/)
- [Scanning Overview](https://www.blacklanternsecurity.com/bbot/Stable/scanning/)
- **Presets**
- [Overview](https://www.blacklanternsecurity.com/bbot/scanning/presets)
- [List of Presets](https://www.blacklanternsecurity.com/bbot/scanning/presets_list)
- [Events](https://www.blacklanternsecurity.com/bbot/scanning/events)
- [Output](https://www.blacklanternsecurity.com/bbot/scanning/output)
- [Tips and Tricks](https://www.blacklanternsecurity.com/bbot/scanning/tips_and_tricks)
- [Advanced Usage](https://www.blacklanternsecurity.com/bbot/scanning/advanced)
- [Configuration](https://www.blacklanternsecurity.com/bbot/scanning/configuration)
- [Overview](https://www.blacklanternsecurity.com/bbot/Stable/scanning/presets)
- [List of Presets](https://www.blacklanternsecurity.com/bbot/Stable/scanning/presets_list)
- [Events](https://www.blacklanternsecurity.com/bbot/Stable/scanning/events)
- [Output](https://www.blacklanternsecurity.com/bbot/Stable/scanning/output)
- [Tips and Tricks](https://www.blacklanternsecurity.com/bbot/Stable/scanning/tips_and_tricks)
- [Advanced Usage](https://www.blacklanternsecurity.com/bbot/Stable/scanning/advanced)
- [Configuration](https://www.blacklanternsecurity.com/bbot/Stable/scanning/configuration)
- **Modules**
- [List of Modules](https://www.blacklanternsecurity.com/bbot/modules/list_of_modules)
- [Nuclei](https://www.blacklanternsecurity.com/bbot/modules/nuclei)
- [List of Modules](https://www.blacklanternsecurity.com/bbot/Stable/modules/list_of_modules)
- [Nuclei](https://www.blacklanternsecurity.com/bbot/Stable/modules/nuclei)
- **Misc**
- [Contribution](https://www.blacklanternsecurity.com/bbot/contribution)
- [Release History](https://www.blacklanternsecurity.com/bbot/release_history)
- [Troubleshooting](https://www.blacklanternsecurity.com/bbot/troubleshooting)
- [Contribution](https://www.blacklanternsecurity.com/bbot/Stable/contribution)
- [Release History](https://www.blacklanternsecurity.com/bbot/Stable/release_history)
- [Troubleshooting](https://www.blacklanternsecurity.com/bbot/Stable/troubleshooting)
- **Developer Manual**
- [Development Overview](https://www.blacklanternsecurity.com/bbot/dev/)
- [BBOT Internal Architecture](https://www.blacklanternsecurity.com/bbot/dev/architecture)
- [How to Write a BBOT Module](https://www.blacklanternsecurity.com/bbot/dev/module_howto)
- [Unit Tests](https://www.blacklanternsecurity.com/bbot/dev/tests)
- [Discord Bot Example](https://www.blacklanternsecurity.com/bbot/dev/discord_bot)
- [Development Overview](https://www.blacklanternsecurity.com/bbot/Stable/dev/)
- [Setting Up a Dev Environment](https://www.blacklanternsecurity.com/bbot/Stable/dev/dev_environment)
- [BBOT Internal Architecture](https://www.blacklanternsecurity.com/bbot/Stable/dev/architecture)
- [How to Write a BBOT Module](https://www.blacklanternsecurity.com/bbot/Stable/dev/module_howto)
- [Unit Tests](https://www.blacklanternsecurity.com/bbot/Stable/dev/tests)
- [Discord Bot Example](https://www.blacklanternsecurity.com/bbot/Stable/dev/discord_bot)
- **Code Reference**
- [Scanner](https://www.blacklanternsecurity.com/bbot/dev/scanner)
- [Presets](https://www.blacklanternsecurity.com/bbot/dev/presets)
- [Event](https://www.blacklanternsecurity.com/bbot/dev/event)
- [Target](https://www.blacklanternsecurity.com/bbot/dev/target)
- [BaseModule](https://www.blacklanternsecurity.com/bbot/dev/basemodule)
- [BBOTCore](https://www.blacklanternsecurity.com/bbot/dev/core)
- [Engine](https://www.blacklanternsecurity.com/bbot/dev/engine)
- [Scanner](https://www.blacklanternsecurity.com/bbot/Stable/dev/scanner)
- [Presets](https://www.blacklanternsecurity.com/bbot/Stable/dev/presets)
- [Event](https://www.blacklanternsecurity.com/bbot/Stable/dev/event)
- [Target](https://www.blacklanternsecurity.com/bbot/Stable/dev/target)
- [BaseModule](https://www.blacklanternsecurity.com/bbot/Stable/dev/basemodule)
- [BBOTCore](https://www.blacklanternsecurity.com/bbot/Stable/dev/core)
- [Engine](https://www.blacklanternsecurity.com/bbot/Stable/dev/engine)
- **Helpers**
- [Overview](https://www.blacklanternsecurity.com/bbot/dev/helpers/)
- [Command](https://www.blacklanternsecurity.com/bbot/dev/helpers/command)
- [DNS](https://www.blacklanternsecurity.com/bbot/dev/helpers/dns)
- [Interactsh](https://www.blacklanternsecurity.com/bbot/dev/helpers/interactsh)
- [Miscellaneous](https://www.blacklanternsecurity.com/bbot/dev/helpers/misc)
- [Web](https://www.blacklanternsecurity.com/bbot/dev/helpers/web)
- [Word Cloud](https://www.blacklanternsecurity.com/bbot/dev/helpers/wordcloud)
- [Overview](https://www.blacklanternsecurity.com/bbot/Stable/dev/helpers/)
- [Command](https://www.blacklanternsecurity.com/bbot/Stable/dev/helpers/command)
- [DNS](https://www.blacklanternsecurity.com/bbot/Stable/dev/helpers/dns)
- [Interactsh](https://www.blacklanternsecurity.com/bbot/Stable/dev/helpers/interactsh)
- [Miscellaneous](https://www.blacklanternsecurity.com/bbot/Stable/dev/helpers/misc)
- [Web](https://www.blacklanternsecurity.com/bbot/Stable/dev/helpers/web)
- [Word Cloud](https://www.blacklanternsecurity.com/bbot/Stable/dev/helpers/wordcloud)
<!-- END BBOT DOCS TOC -->

## Contribution

Some of the best BBOT modules were written by the community. BBOT is being constantly improved; every day it grows more powerful!

We welcome contributions. Not just code, but ideas too! If you have an idea for a new feature, please let us know in [Discussions](https://github.com/blacklanternsecurity/bbot/discussions). If you want to get your hands dirty, see [Contribution](https://www.blacklanternsecurity.com/bbot/contribution/). There you can find setup instructions and a simple tutorial on how to write a BBOT module. We also have extensive [Developer Documentation](https://www.blacklanternsecurity.com/bbot/dev/).
We welcome contributions. Not just code, but ideas too! If you have an idea for a new feature, please let us know in [Discussions](https://github.com/blacklanternsecurity/bbot/discussions). If you want to get your hands dirty, see [Contribution](https://www.blacklanternsecurity.com/bbot/Stable/contribution/). There you can find setup instructions and a simple tutorial on how to write a BBOT module. We also have extensive [Developer Documentation](https://www.blacklanternsecurity.com/bbot/Stable/dev/).

Thanks to these amazing people for contributing to BBOT! :heart:

Expand All @@ -391,7 +394,7 @@ Thanks to these amazing people for contributing to BBOT! :heart:

Special thanks to:

- @TheTechromancer for creating [BBOT](https://github.com/blacklanternsecurity/bbot)
- @TheTechromancer for creating BBOT
- @liquidsec for his extensive work on BBOT's web hacking features, including [badsecrets](https://github.com/blacklanternsecurity/badsecrets) and [baddns](https://github.com/blacklanternsecurity/baddns)
- Steve Micallef (@smicallef) for creating Spiderfoot
- @kerrymilan for his Neo4j and Ansible expertise
Expand Down
11 changes: 2 additions & 9 deletions bbot/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,13 +24,6 @@
www.blacklanternsecurity.com/bbot
"""
print(ascii_art, file=sys.stderr)
log_to_stderr(
"This is a pre-release of BBOT 2.0. If you upgraded from version 1, we recommend cleaning your old configs etc. before running this version!",
level="WARNING",
)
log_to_stderr(
"For details, see https://github.com/blacklanternsecurity/bbot/discussions/1540", level="WARNING"
)

scan_name = ""

Expand Down Expand Up @@ -219,7 +212,7 @@ def handle_keyboard_input(keyboard_input):

reader = asyncio.StreamReader()
protocol = asyncio.StreamReaderProtocol(reader)
await asyncio.get_event_loop().connect_read_pipe(lambda: protocol, sys.stdin)
await asyncio.get_running_loop().connect_read_pipe(lambda: protocol, sys.stdin)

# set stdout and stderr to blocking mode
# this is needed to prevent BlockingIOErrors in logging etc.
Expand Down Expand Up @@ -250,7 +243,7 @@ async def akeyboard_listen():

asyncio.create_task(akeyboard_listen())

await scan.async_start_without_generator()
await scan.async_start_without_generator()

return True

Expand Down
Loading

0 comments on commit b5d51b6

Please sign in to comment.