Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wappalyzer boefje detects less software instances than before #3037

Open
stephanie0x00 opened this issue Jun 6, 2024 · 2 comments · May be fixed by #3800
Open

Wappalyzer boefje detects less software instances than before #3037

stephanie0x00 opened this issue Jun 6, 2024 · 2 comments · May be fixed by #3800
Assignees
Labels
boefjes Issues related to boefjes regression Something is broken that explicitly worked before 😸 Review/QA feedback Review/QA feedback provided

Comments

@stephanie0x00
Copy link
Contributor

stephanie0x00 commented Jun 6, 2024

Describe the bug
With the replacement to the Wappalyzer boefje (#2727) some of the findings from the original Wappalyzer boefje disappeared. This is due to changes in the 'technologies.json' files. A result from this is less Software oois identified and thus less findings relating to outdated software components.

In the old version various jQuery objects were identified. With the new version these objects are not identified.

Expected behavior
At minimum, the wappalyzer boefje should be able to identify all common software instances identified on websites, including the software versions.

OpenKAT version
commit 233dc34 (HEAD -> main, origin/main, origin/HEAD)

Current solution idea
As discussed with @underdarknl - the current wappalyzer boefje is installed using the requirements.txt. The idea is to fork the wappalyzer branch and add more/newer technologies.json files from other projects. This should hopefully solve the reduced number of findings observed by wappalyzer.

The python implementation of our Wappalyzer uses a different format in the technologies file than the 'original' Wappalyzer version. There is a fork of the original Wappalyzer that seems to be active by HTTArchive. This would require adjusting the parsing of the Wappalyzer boefje to match these formats. See: https://github.com/HTTPArchive/wappalyzer

@stephanie0x00 stephanie0x00 added boefjes Issues related to boefjes regression Something is broken that explicitly worked before labels Jun 6, 2024
@stephanie0x00 stephanie0x00 self-assigned this Jun 6, 2024
@stephanie0x00 stephanie0x00 added this to KAT Jun 6, 2024
@github-project-automation github-project-automation bot moved this to Incoming features / Need assessment in KAT Jun 6, 2024
@stephanie0x00 stephanie0x00 moved this from Incoming features / Need assessment to In Progress in KAT Jun 6, 2024
@stephanie0x00 stephanie0x00 moved this from In Progress to To be discussed in KAT Jun 6, 2024
@stephanie0x00 stephanie0x00 removed their assignment Jun 6, 2024
@madelondohmen
Copy link
Contributor

Discussion meeting 18-06-2024:

  • technologies.json from community
  • Continue to use python-wappalyzer
  • Most efficient for current situation
  • Don't fork, only use pieces we deem necessary.
  • Keep regexing against our files

@underdarknl underdarknl moved this from To be discussed to Backlog / To do in KAT Jul 2, 2024
@underdarknl underdarknl self-assigned this Jul 2, 2024
@stephanie0x00 stephanie0x00 moved this from Backlog / To do to To be discussed in KAT Sep 19, 2024
@stephanie0x00 stephanie0x00 moved this from To be discussed to Backlog / To do in KAT Sep 24, 2024
@stephanie0x00 stephanie0x00 moved this from Backlog / To do to To be discussed in KAT Oct 28, 2024
@stephanie0x00
Copy link
Contributor Author

Discussion meet notes:

  • We provide technology files on a regular interval we update and package these.
  • We will setup some testing scenario's to test if software is still detected on some test cases.

@stephanie0x00 stephanie0x00 moved this from To be discussed to Backlog / To do in KAT Nov 5, 2024
@ammar92 ammar92 linked a pull request Nov 6, 2024 that will close this issue
9 tasks
@madelondohmen madelondohmen moved this from Backlog / To do to In Progress in KAT Nov 13, 2024
@stephanie0x00 stephanie0x00 added the 😸 Review/QA feedback Review/QA feedback provided label Dec 16, 2024
@ammar92 ammar92 moved this from Review to QA review / functional testing in KAT Dec 17, 2024
@stephanie0x00 stephanie0x00 moved this from QA review / functional testing to In Progress in KAT Dec 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
boefjes Issues related to boefjes regression Something is broken that explicitly worked before 😸 Review/QA feedback Review/QA feedback provided
Projects
Status: In Progress
Development

Successfully merging a pull request may close this issue.

4 participants