Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Socket timeouts are not set while connecting to graphite #26

Open
varsha-abhinandan opened this issue Feb 18, 2016 · 1 comment
Open

Comments

@varsha-abhinandan
Copy link

Hi,

I am facing this problem where my entire storm topology gets stuck when the socket connection open to graphite gets stuck because of the timeout not being set. So I feel the tuples which enter the GraphiteMetricsConsumer don't get acked, my queue size keeps growing and eventually new tuples stop getting emitted.

Following is the jstack corresponding to it:

"Thread-205-__metricscom.verisign.storm.metrics.GraphiteMetricsConsumer" #244 prio=5 os_prio=0 tid=0x00007f10591f1800 nid=0x6a1e runnable [0x00007f0fd3bf9000]
java.lang.Thread.State: RUNNABLE
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
- locked <0x000000065e5d0068> (a java.net.SocksSocketImpl)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at java.net.Socket.connect(Socket.java:538)
at java.net.Socket.(Socket.java:434)
at java.net.Socket.(Socket.java:244)
at javax.net.DefaultSocketFactory.createSocket(SocketFactory.java:277)
at com.verisign.storm-graphite.com.codahale.metrics.graphite.Graphite.connect(Graphite.java:118)
at com.verisign.storm.metrics.reporters.graphite.GraphiteReporter.connect(GraphiteReporter.java:104)
at com.verisign.storm.metrics.reporters.graphite.GraphiteReporter.handleFailedSend(GraphiteReporter.java:174)
at com.verisign.storm.metrics.reporters.graphite.GraphiteReporter.appendToBuffer(GraphiteReporter.java:143)
at com.verisign.storm.metrics.GraphiteMetricsConsumer.appendToReporterBuffer(GraphiteMetricsConsumer.java:311)
at com.verisign.storm.metrics.GraphiteMetricsConsumer.handleDataPoints(GraphiteMetricsConsumer.java:227)
at backtype.storm.metric.MetricsConsumerBolt.execute(MetricsConsumerBolt.java:55)
at backtype.storm.daemon.executor$fn__6647$tuple_action_fn__6649.invoke(executor.clj:633)
at backtype.storm.daemon.executor$mk_task_receiver$fn__6570.invoke(executor.clj:401)

Thanks,
Varsha

kevinconaway pushed a commit to walmart-technology-cps/storm-graphite that referenced this issue Apr 19, 2016
kevinconaway pushed a commit to walmart-technology-cps/storm-graphite that referenced this issue Apr 19, 2016
kevinconaway pushed a commit to walmart-technology-cps/storm-graphite that referenced this issue Apr 19, 2016
KevinJMao added a commit that referenced this issue Apr 19, 2016
#26 Add tcp connect / read timeouts when interacting with Graphite
@kevinconaway
Copy link

Thanks @KevinJMao for merging #28.

Is it possible for you to deploy a release to maven central?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants