Connection hangs for very generic selectors

[ricogallo]

Hi, I'm building a forward proxy using node-http-proxy and I'm employing harmon to do some transformations.

I'm experiencing some issues with the connection kept hanging for selectors like div or *, eventually leading to an net::ERR_INCOMPLETE_CHUNKED_ENCODING in Chrome. Probably the issue lies in the number of elements to scan.

A website good for reproducing this is https://weather.com, or http://www.aliexpress.com for plain http.

Any ideas on why this is happening? Thanks

[No9]

Hi @ricogallo
I don't have the bandwidth to dive into this right now but have you tried the example/simple.js with line 26 commented out? and the target changed to the URL that your are trying to get to.
Just want to make sure it's a harmon issue and not a proxy one.

[ricogallo]

Hi @No9 , yep, I tested this several times, with the line 26 commented out it doesn't seem to have any issues. I tested it on aliexpress.com and if you open the chrome's console you can eventually see the before mentioned error. Regarding the selector, I'm using *, while the transformation function is just a noop.

[No9]

OK @ricogallo the main HTML from aliexpress parses fine just using trumpet so I would say there are some additional assets being proxied that are causing it to choke.
If you look at these tests you will see that you can set a param on harmon to make sure it only applies to HTML mimetypes you might want to give that a go https://github.com/No9/harmon/blob/master/test/host.js#L316
We should probably have that better documented that just in the tests.

[ricogallo]

Hi, I already tried that, because I've looked at your source thoroughly :) Yet, the problem still persists. I did further investigations and the streams that seem choked are the ones of html-select

[cmrigney]

I'm also having this issue

[cmrigney]

Could it relate to this? https://github.com/substack/node-trumpet/issues/52