Add ability to set headers per host

[lahabana]

Some websites (.e.g github) have aggressive throttling if we crawl unauthenticated.
It would be nice to be able to set some headers on specific hosts.

That would make it possible to add a Github token auth to check links to github.

Maybe this would require to introduce a "per host" config where we could have a set of host specific config.