It should be possible to add GET parameters to targets

[liquidsec]

Sometimes you need to send a GET request along with a request to access some content. Imagine some vulnerable content appears with debug=1. You can DETECT things like this with paramminer of course, but you can't run a larger scan with such a parameter set.

This will probably be best accomplished with a command line option of some kind

[TheTechromancer]

@liquidsec I agree with this.

I propose we allow GET parameters in URLs, but not consider them when hashing/deduping events. In theory this would both prevent garbage data and allow us to make use of GET parameters. (This would be useful for other modules such as OAUTH/ADFS).

This would require us to:

• Update all the URL-related regexes
• Update URL sanitization functions to no longer strip out GET parameters
• Update the _data_id method on the URL_UNVERIFIED event to strip out GET parameters before hashing