Excavate enhancement handle RAW_TEXT events

[domwhewell-sage]

Description
Now that RAW_TEXT events are being raised from parsed FILESYSTEM events (And corrections have been made to scope distance of such events/yara rules have been implemented) the next step should be to make changes to the internal excavate.py module to consume RAW_TEXT and extract useful tidbits (URLS/DNS_NAMES etc.)

As the data of RAW_TEXT events is a string instead of an object (as in HTTP_RESPONSE) the handle_event() of excavate might require changes.

This is the error im getting currently

2024-08-05 15:35:15,958 [ERROR] bbot.scanner scanner.py:1195 Error in excavate.handle_event(RAW_TEXT("Example http://schemas.example.com/Example/2", module=unstructured, tags={'distance-1'})): /root/lib/python3.12/site-packages/bbot/modules/internal/excavate.py:931:handle_event(): 'str' object has no attribute 'get'
2024-08-05 15:35:15,958 [TRACE] bbot.scanner logger.py:132 Traceback (most recent call last):
  File "/root/lib/python3.12/site-packages/bbot/scanner/scanner.py", line 1172, in _acatch
    yield
  File "/root/lib/python3.12/site-packages/bbot/modules/base.py", line 637, in _worker
    await self.handle_event(event)
  File "/root/lib/python3.12/site-packages/bbot/modules/internal/excavate.py", line 931, in handle_event
    body = event.data.get("body", "")
           ^^^^^^^^^^^^^^
AttributeError: 'str' object has no attribute 'get'

I have some time so will look into this this week

[TheTechromancer]

This is a super exciting feature. Feeding the contents of binary files like PDFs and word docs into excavate is a pretty unique capability, that will only get more insane as we keep adding filetypes. I'm imagining being able to download an app from the app store, decompile it, and feed its entire contents into excavate and trufflehog 🙌

[domwhewell-sage]

Some work being done on this #1636

[domwhewell-sage]

Closing as merged into dev