Renamed User Agent and Added default maxthread to Go

[tombui99]

#886

As requested from @wicksipedia

• Renamed user agent to easier identify CodeAuditor
  • Following best naming practice for Crawler (https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/User-Agent)
• Added default maxthread to Go for throttling
• Added 3ms pause after each thread finishes

[wicksipedia]

dealing down the maximum number of threads is good, but is there any pauses after a thread completes to let the server catch it's breath?

[tombui99]

Hey @wicksipedia

dealing down the maximum number of threads is good, but is there any pauses after a thread completes to let the server catch it's breath?

I have added an extra 3ms pause after every thread completes (after each link scan finishes) if this is what you are after.

Hey @william-liebenberg

This change increases the scan duration greatly - let me know if you are happy.