Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option to support timeouts passed to RestClient #12

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions Gemfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
source 'https://rubygems.org'

gemspec
48 changes: 48 additions & 0 deletions Gemfile.lock
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
PATH
remote: .
specs:
url_resolver (0.2.2)
rest-client (~> 2.0)

GEM
remote: https://rubygems.org/
specs:
diff-lcs (1.3)
domain_name (0.5.20170404)
unf (>= 0.0.5, < 1.0.0)
http-cookie (1.0.3)
domain_name (~> 0.5)
mime-types (3.1)
mime-types-data (~> 3.2015)
mime-types-data (3.2016.0521)
netrc (0.11.0)
rest-client (2.0.2)
http-cookie (>= 1.0.2, < 2.0)
mime-types (>= 1.16, < 4.0)
netrc (~> 0.8)
rspec (3.6.0)
rspec-core (~> 3.6.0)
rspec-expectations (~> 3.6.0)
rspec-mocks (~> 3.6.0)
rspec-core (3.6.0)
rspec-support (~> 3.6.0)
rspec-expectations (3.6.0)
diff-lcs (>= 1.2.0, < 2.0)
rspec-support (~> 3.6.0)
rspec-mocks (3.6.0)
diff-lcs (>= 1.2.0, < 2.0)
rspec-support (~> 3.6.0)
rspec-support (3.6.0)
unf (0.1.4)
unf_ext
unf_ext (0.0.7.4)

PLATFORMS
ruby

DEPENDENCIES
rspec
url_resolver!

BUNDLED WITH
1.15.0
26 changes: 26 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ UrlResolver.configure do |config|
config.cache_failures = true # default: true
config.user_agent = 'Custom User-Agent' # default: 'Ruby'
config.errors_to_ignore << MyModule::MyError
config.timeout = 15
end
```

Expand All @@ -44,10 +45,35 @@ UrlResolver.configuration.cache = Redis.new
UrlResolver.configuration.cache_failures = true
UrlResolver.configuration.user_agent = 'Custom User-Agent'
UrlResolver.configuration.errors_to_ignore << MyModule::MyError
UrlResolver.configuration.timeout = 15
```

If you want to specify different options on a per-call basis (or you
prefer to not leverage the global configuration from above), you can
pass an optional hash to the `#resolve` method

```ruby
options = { :user_agent => 'Some Other User-Agent', :timeout => 60 }
UrlResolver.resolve('http://analytics.google.com', options)
# => "http://www.google.com/analytics/"
```

These options are passed to `RestClient`, so you can send any options
that `RestClient` acceptsm, such as `:max_redirects`


```ruby
options = { :max_redirects => 20 }
UrlResolver.resolve('http://analytics.google.com', options)
```

### Changelog

##### 0.2.1
+ Added timeout configuration option
+ Can pass a hash of options into the `#resolve` method to specify
configuration parameters on a per-call basis

##### 0.1.1
+ Handle redirects that resolve to nonstandard protocols

Expand Down
4 changes: 2 additions & 2 deletions lib/url_resolver.rb
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@
require_relative 'url_resolver/resolver.rb'

module UrlResolver
def self.resolve(url)
def self.resolve(url, options={})
@@resolver ||= UrlResolver::Resolver.new
@@resolver.resolve(url)
@@resolver.resolve(url, options)
end
end
3 changes: 2 additions & 1 deletion lib/url_resolver/configuration.rb
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
module UrlResolver
class Configuration
attr_accessor :cache_failures, :user_agent, :errors_to_ignore
attr_accessor :cache_failures, :user_agent, :timeout, :errors_to_ignore
attr_reader :cache, :url_cache

DEFAULT_ERRORS_TO_IGNORE = [SocketError,
Expand All @@ -25,6 +25,7 @@ def initialize
@cache = nil
@url_cache = Cache.new(@cache)
@user_agent = 'Ruby'
@timeout = 60
@errors_to_ignore = DEFAULT_ERRORS_TO_IGNORE
end

Expand Down
17 changes: 12 additions & 5 deletions lib/url_resolver/resolver.rb
Original file line number Diff line number Diff line change
Expand Up @@ -8,13 +8,20 @@ def user_agent
UrlResolver.configuration.user_agent
end

def resolve(url)
def timeout
UrlResolver.configuration.timeout
end

def resolve(url, options={})
url_to_check = URI.escape(url)
cached_url = cache.get_url(url_to_check)
return cached_url if cached_url

response = RestClient.head(url_to_check, user_agent: user_agent)
response.args[:url].tap do |final_url|

default_options = { :user_agent => user_agent, :timeout => timeout }
options = default_options.merge(options)

response = RestClient.head(url_to_check, options)
response.request.url.tap do |final_url|
cache.set_url(url_to_check, final_url)
end
rescue *UrlResolver.configuration.errors_to_ignore => e
Expand All @@ -23,7 +30,7 @@ def resolve(url)
response = RestClient.head(url_to_check) { |response, request, result, &block| response }
url = response.headers[:location] if response.code == 302 && response.headers[:location]
end

cache.set_url(url_to_check, url) if UrlResolver.configuration.cache_failures
url
rescue Exception => e
Expand Down
8 changes: 4 additions & 4 deletions url_resolver.gemspec
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
Gem::Specification.new do |s|
s.name = 'url_resolver'
s.version = '0.2.0'
s.date = '2016-05-18'
s.version = '0.2.3'
s.date = '2016-09-10'
s.summary = "Url Resolver!"
s.description = "Simple gem to follow redirects to resolve the destination of a URL. Caches results sometimes."
s.authors = ["Amir Manji"]
Expand All @@ -10,6 +10,6 @@ Gem::Specification.new do |s|
s.homepage =
'http://www.github.com/amirmanji/url_resolver'
s.license = 'MIT'
s.add_runtime_dependency "rest-client", '~> 1.8'
s.add_development_dependency "rspec", [ "2.14.7" ]
s.add_runtime_dependency "rest-client", '~> 2.0'
s.add_development_dependency "rspec", ['3.6.0']
end