Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

405 errors for all requests with Ruby versions > 2.3 #8

Open
xavriley opened this issue May 10, 2019 · 3 comments
Open

405 errors for all requests with Ruby versions > 2.3 #8

xavriley opened this issue May 10, 2019 · 3 comments

Comments

@xavriley
Copy link

I tried setting this up with Ruby 2.5.1 and all of my requests were being rejected with 405 errors. For example:

# using Ruby 2.5.1
$ https_proxy=http://localhost:8000 curl -vI https://google.com --insecure
* Rebuilt URL to: https://google.com/
*   Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 8000 (#0)
* Establish HTTP proxy tunnel to google.com:443
> CONNECT google.com:443 HTTP/1.1
> Host: google.com:443
> User-Agent: curl/7.54.0
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 200 OK
HTTP/1.1 200 OK
< Server: WEBrick/1.4.2 (Ruby/2.5.1/2018-03-29) OpenSSL/1.1.1b
Server: WEBrick/1.4.2 (Ruby/2.5.1/2018-03-29) OpenSSL/1.1.1b
< Date: Fri, 10 May 2019 10:47:37 GMT
Date: Fri, 10 May 2019 10:47:37 GMT
< Content-Length: 0
Content-Length: 0
* Ignoring Content-Length in CONNECT 200 response
< Connection: close
Connection: close
<

* Proxy replied OK to CONNECT request
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/cert.pem
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: C=US; O=mitm.proxy; OU=ca; OU=CA; CN=google.com
*  start date: May  9 20:46:12 2019 GMT
*  expire date: May  8 20:46:12 2020 GMT
*  issuer: C=US; O=mitm.proxy; OU=ca; CN=CA
*  SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
> HEAD / HTTP/1.1
> Host: google.com
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 405 Method Not Allowed
HTTP/1.1 405 Method Not Allowed
< Content-Type: text/html; charset=ISO-8859-1
Content-Type: text/html; charset=ISO-8859-1
< Server: WEBrick/1.4.2 (Ruby/2.5.1/2018-03-29) OpenSSL/1.1.1b
Server: WEBrick/1.4.2 (Ruby/2.5.1/2018-03-29) OpenSSL/1.1.1b
< Date: Fri, 10 May 2019 11:22:36 GMT
Date: Fri, 10 May 2019 11:22:36 GMT
< Content-Length: 315
Content-Length: 315
< Connection: close
Connection: close

<
* Closing connection 0
* TLSv1.2 (OUT), TLS alert, Client hello (1):

When I downgraded to 2.4 the issue was still present, but downgrading to Ruby 2.3 seems to have fixed it:

$ ruby -v
ruby 2.3.0p0 (2015-12-25 revision 53290) [x86_64-darwin14]
$ https_proxy=http://localhost:8000 curl -vI https://google.com --insecure
* Rebuilt URL to: https://google.com/
*   Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 8000 (#0)
* Establish HTTP proxy tunnel to google.com:443
> CONNECT google.com:443 HTTP/1.1
> Host: google.com:443
> User-Agent: curl/7.54.0
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 200 OK
HTTP/1.1 200 OK
< Server: WEBrick/1.3.1 (Ruby/2.3.0/2015-12-25) OpenSSL/1.0.2f
Server: WEBrick/1.3.1 (Ruby/2.3.0/2015-12-25) OpenSSL/1.0.2f
< Date: Fri, 10 May 2019 11:29:20 GMT
Date: Fri, 10 May 2019 11:29:20 GMT
< Content-Length: 0
Content-Length: 0
* Ignoring Content-Length in CONNECT 200 response
< Connection: close
Connection: close
<

* Proxy replied OK to CONNECT request
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/cert.pem
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / DHE-RSA-AES256-GCM-SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: C=US; O=mitm.proxy; OU=ca; OU=CA; CN=google.com
*  start date: May  9 20:46:12 2019 GMT
*  expire date: May  8 20:46:12 2020 GMT
*  issuer: C=US; O=mitm.proxy; OU=ca; CN=CA
*  SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
> HEAD / HTTP/1.1
> Host: google.com
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 301 Moved Permanently
HTTP/1.1 301 Moved Permanently
* HTTP/1.1 proxy connection set close!
< Proxy-Connection: close
Proxy-Connection: close
< Connection: close
Connection: close
< Location: https://www.google.com/
Location: https://www.google.com/
< Content-Type: text/html; charset=UTF-8
Content-Type: text/html; charset=UTF-8
< Date: Fri, 10 May 2019 11:29:18 GMT
Date: Fri, 10 May 2019 11:29:18 GMT
< Expires: Sun, 09 Jun 2019 11:29:18 GMT
Expires: Sun, 09 Jun 2019 11:29:18 GMT
< Cache-Control: public, max-age=2592000
Cache-Control: public, max-age=2592000
< Server: gws
Server: gws
< Content-Length: 220
Content-Length: 220
< X-Xss-Protection: 0
X-Xss-Protection: 0
< X-Frame-Options: SAMEORIGIN
X-Frame-Options: SAMEORIGIN
< Alt-Svc: quic=":443"; ma=2592000; v="46,44,43,39"
Alt-Svc: quic=":443"; ma=2592000; v="46,44,43,39"

<
* Closing connection 0
* TLSv1.2 (OUT), TLS alert, Client hello (1):

My server proxy code is as per the example:

require 'evil-proxy'
proxy = EvilProxy::MITMProxyServer.new Port: 8000

trap "INT"  do proxy.shutdown end
trap "TERM" do proxy.shutdown end

proxy.start

I haven't had much time to dig into this but I managed to narrow it down to the following error:

Exception: NoMethodError: undefined method `<<' for {:read_timeout=>60, :continue_timeout=>nil, :debug_output=>nil}:Hash
--
 0: /Users/xriley/.rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/net/protocol.rb:223:in `writing'
 1: /Users/xriley/.rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/net/protocol.rb:206:in `write'
 2: /Users/xriley/.rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/net/http/generic_request.rb:334:in `write_header'
 3: /Users/xriley/.rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/net/http/generic_request.rb:127:in `exec'
 4: /Users/xriley/.rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/net/http.rb:1492:in `block in transport_request'
 5: /Users/xriley/.rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/net/http.rb:1491:in `catch'
 6: /Users/xriley/.rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/net/http.rb:1491:in `transport_request'
 7: /Users/xriley/.rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/net/http.rb:1464:in `request'
 8: /Users/xriley/.rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/net/http.rb:1234:in `head'
 9: /Users/xriley/.rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/webrick/httpproxy.rb:222:in `block in do_HEAD'
10: /Users/xriley/.rvm/gems/ruby-2.5.1/gems/evil-proxy-0.2.0/lib/evil-proxy/agentproxy.rb:35:in `block in perform_proxy_request'
11: /Users/xriley/.rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/net/http.rb:910:in `start'
12: /Users/xriley/.rvm/gems/ruby-2.5.1/gems/evil-proxy-0.2.0/lib/evil-proxy/agentproxy.rb:27:in `perform_proxy_request'
13: /Users/xriley/.rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/webrick/httpproxy.rb:221:in `do_HEAD'
14: /Users/xriley/.rvm/gems/ruby-2.5.1/gems/evil-proxy-0.2.0/lib/evil-proxy/httpproxy.rb:111:in `block (2 levels) in <class:HTTPProxyServer>'
15: (pry):1:in `rescue in proxy_service'
16: /Users/xriley/.rvm/gems/ruby-2.5.1/gems/pry-0.12.2/lib/pry/pry_instance.rb:387:in `eval'
17: /Users/xriley/.rvm/gems/ruby-2.5.1/gems/pry-0.12.2/lib/pry/pry_instance.rb:387:in `evaluate_ruby'
18: /Users/xriley/.rvm/gems/ruby-2.5.1/gems/pry-0.12.2/lib/pry/pry_instance.rb:355:in `handle_line'
19: /Users/xriley/.rvm/gems/ruby-2.5.1/gems/pry-0.12.2/lib/pry/pry_instance.rb:274:in `block (2 levels) in eval'
20: /Users/xriley/.rvm/gems/ruby-2.5.1/gems/pry-0.12.2/lib/pry/pry_instance.rb:273:in `catch'
21: /Users/xriley/.rvm/gems/ruby-2.5.1/gems/pry-0.12.2/lib/pry/pry_instance.rb:273:in `block in eval'
22: /Users/xriley/.rvm/gems/ruby-2.5.1/gems/pry-0.12.2/lib/pry/pry_instance.rb:272:in `catch'
23: /Users/xriley/.rvm/gems/ruby-2.5.1/gems/pry-0.12.2/lib/pry/pry_instance.rb:272:in `eval'
24: /Users/xriley/.rvm/gems/ruby-2.5.1/gems/pry-0.12.2/lib/pry/repl.rb:77:in `block in repl'
25: /Users/xriley/.rvm/gems/ruby-2.5.1/gems/pry-0.12.2/lib/pry/repl.rb:67:in `loop'
26: /Users/xriley/.rvm/gems/ruby-2.5.1/gems/pry-0.12.2/lib/pry/repl.rb:67:in `repl'
27: /Users/xriley/.rvm/gems/ruby-2.5.1/gems/pry-0.12.2/lib/pry/repl.rb:38:in `block in start'
28: /Users/xriley/.rvm/gems/ruby-2.5.1/gems/pry-0.12.2/lib/pry/input_lock.rb:59:in `__with_ownership'
29: /Users/xriley/.rvm/gems/ruby-2.5.1/gems/pry-0.12.2/lib/pry/input_lock.rb:77:in `with_ownership'
30: /Users/xriley/.rvm/gems/ruby-2.5.1/gems/pry-0.12.2/lib/pry/repl.rb:38:in `start'
31: /Users/xriley/.rvm/gems/ruby-2.5.1/gems/pry-0.12.2/lib/pry/repl.rb:13:in `start'
32: /Users/xriley/.rvm/gems/ruby-2.5.1/gems/pry-0.12.2/lib/pry/pry_class.rb:200:in `start'
33: /Users/xriley/.rvm/gems/ruby-2.5.1/gems/pry-byebug-3.7.0/lib/byebug/processors/pry_processor.rb:117:in `block in resume_pry'
34: /Users/xriley/.rvm/gems/ruby-2.5.1/gems/pry-byebug-3.7.0/lib/byebug/processors/pry_processor.rb:30:in `block (2 levels) in run'
35: /Users/xriley/.rvm/gems/ruby-2.5.1/gems/byebug-11.0.1/lib/byebug/helpers/eval.rb:94:in `allowing_other_threads'
36: /Users/xriley/.rvm/gems/ruby-2.5.1/gems/pry-byebug-3.7.0/lib/byebug/processors/pry_processor.rb:30:in `block in run'
37: /Users/xriley/.rvm/gems/ruby-2.5.1/gems/pry-byebug-3.7.0/lib/byebug/processors/pry_processor.rb:29:in `catch'
38: /Users/xriley/.rvm/gems/ruby-2.5.1/gems/pry-byebug-3.7.0/lib/byebug/processors/pry_processor.rb:29:in `run'
39: /Users/xriley/.rvm/gems/ruby-2.5.1/gems/pry-byebug-3.7.0/lib/byebug/processors/pry_processor.rb:113:in `resume_pry'
40: /Users/xriley/.rvm/gems/ruby-2.5.1/gems/pry-byebug-3.7.0/lib/byebug/processors/pry_processor.rb:65:in `at_line'
41: /Users/xriley/.rvm/gems/ruby-2.5.1/gems/byebug-11.0.1/lib/byebug/context.rb:98:in `at_line'
42: /Users/xriley/.rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/webrick/httpproxy.rb:121:in `rescue in proxy_service'
43: /Users/xriley/.rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/webrick/httpproxy.rb:117:in `proxy_service'
44: /Users/xriley/.rvm/gems/ruby-2.5.1/gems/evil-proxy-0.2.0/lib/evil-proxy/agentproxy.rb:52:in `service'
45: /Users/xriley/.rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/webrick/httpserver.rb:96:in `run'
46: /Users/xriley/.rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/webrick/server.rb:307:in `block in start_thread'
Caused by: NoMethodError: undefined method `<<' for {:read_timeout=>60, :continue_timeout=>nil, :debug_output=>nil}:Hash
Did you mean?  <

To me, it looks like a change to Net::HTTP or `` after Ruby 2.3.0 where the config hash is getting passed to something that expects a String for logging. If anyone has any tips on how to narrow this down I'd appreciate it!
@xavriley
Copy link
Author

I should have mentioned, the reason this returns a 405 response is because of these lines in Webrick https://github.com/ruby/ruby/blob/trunk/lib/webrick/httpproxy.rb#L117-L125 It catches NoMethodError to handle metaprogramming for do_HTTPMETHODNAME however the NoMethodError here is coming from somewhere in the call chain below do_HEAD

@matti
Copy link

matti commented May 18, 2019

"like a change to Net::HTTP or ``"

what do you mean by `` change?

@xavriley
Copy link
Author

Sorry, that was meant to say

like a change to Net::HTTP or WEBrick::HTTPProxyServer

I appreciate that the issue might be upstream from this project, but until I can figure out how this is happening I thought I'd start here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@matti @xavriley