You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I am facing this problem where my entire storm topology gets stuck when the socket connection open to graphite gets stuck because of the timeout not being set. So I feel the tuples which enter the GraphiteMetricsConsumer don't get acked, my queue size keeps growing and eventually new tuples stop getting emitted.
Following is the jstack corresponding to it:
"Thread-205-__metricscom.verisign.storm.metrics.GraphiteMetricsConsumer" #244 prio=5 os_prio=0 tid=0x00007f10591f1800 nid=0x6a1e runnable [0x00007f0fd3bf9000]
java.lang.Thread.State: RUNNABLE
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
- locked <0x000000065e5d0068> (a java.net.SocksSocketImpl)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at java.net.Socket.connect(Socket.java:538)
at java.net.Socket.(Socket.java:434)
at java.net.Socket.(Socket.java:244)
at javax.net.DefaultSocketFactory.createSocket(SocketFactory.java:277)
at com.verisign.storm-graphite.com.codahale.metrics.graphite.Graphite.connect(Graphite.java:118)
at com.verisign.storm.metrics.reporters.graphite.GraphiteReporter.connect(GraphiteReporter.java:104)
at com.verisign.storm.metrics.reporters.graphite.GraphiteReporter.handleFailedSend(GraphiteReporter.java:174)
at com.verisign.storm.metrics.reporters.graphite.GraphiteReporter.appendToBuffer(GraphiteReporter.java:143)
at com.verisign.storm.metrics.GraphiteMetricsConsumer.appendToReporterBuffer(GraphiteMetricsConsumer.java:311)
at com.verisign.storm.metrics.GraphiteMetricsConsumer.handleDataPoints(GraphiteMetricsConsumer.java:227)
at backtype.storm.metric.MetricsConsumerBolt.execute(MetricsConsumerBolt.java:55)
at backtype.storm.daemon.executor$fn__6647$tuple_action_fn__6649.invoke(executor.clj:633)
at backtype.storm.daemon.executor$mk_task_receiver$fn__6570.invoke(executor.clj:401)
Thanks,
Varsha
The text was updated successfully, but these errors were encountered:
kevinconaway
pushed a commit
to walmart-technology-cps/storm-graphite
that referenced
this issue
Apr 19, 2016
Hi,
I am facing this problem where my entire storm topology gets stuck when the socket connection open to graphite gets stuck because of the timeout not being set. So I feel the tuples which enter the GraphiteMetricsConsumer don't get acked, my queue size keeps growing and eventually new tuples stop getting emitted.
Following is the jstack corresponding to it:
"Thread-205-__metricscom.verisign.storm.metrics.GraphiteMetricsConsumer" #244 prio=5 os_prio=0 tid=0x00007f10591f1800 nid=0x6a1e runnable [0x00007f0fd3bf9000]
java.lang.Thread.State: RUNNABLE
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
- locked <0x000000065e5d0068> (a java.net.SocksSocketImpl)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at java.net.Socket.connect(Socket.java:538)
at java.net.Socket.(Socket.java:434)
at java.net.Socket.(Socket.java:244)
at javax.net.DefaultSocketFactory.createSocket(SocketFactory.java:277)
at com.verisign.storm-graphite.com.codahale.metrics.graphite.Graphite.connect(Graphite.java:118)
at com.verisign.storm.metrics.reporters.graphite.GraphiteReporter.connect(GraphiteReporter.java:104)
at com.verisign.storm.metrics.reporters.graphite.GraphiteReporter.handleFailedSend(GraphiteReporter.java:174)
at com.verisign.storm.metrics.reporters.graphite.GraphiteReporter.appendToBuffer(GraphiteReporter.java:143)
at com.verisign.storm.metrics.GraphiteMetricsConsumer.appendToReporterBuffer(GraphiteMetricsConsumer.java:311)
at com.verisign.storm.metrics.GraphiteMetricsConsumer.handleDataPoints(GraphiteMetricsConsumer.java:227)
at backtype.storm.metric.MetricsConsumerBolt.execute(MetricsConsumerBolt.java:55)
at backtype.storm.daemon.executor$fn__6647$tuple_action_fn__6649.invoke(executor.clj:633)
at backtype.storm.daemon.executor$mk_task_receiver$fn__6570.invoke(executor.clj:401)
Thanks,
Varsha
The text was updated successfully, but these errors were encountered: