Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

remove connection and query deadlocks #187

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

lethalwp
Copy link
Contributor

By removing these locks, i fixed one of my performance issue:
When i uses prometheus with junos_exporter on a poll rate of 1 minute with 20 devices, it worked perfectly.

But when i tried to use it with 250 devices, i had a quick rampup of non reply, devices looking unresponsive, the http://junosurl/metrics took minutes to reply with a duration metric of some seconds when the answer came.

At each prometheus/junos exporter restart, I had to make a "rampup" to bypass: adjust the polling rate&timeout at 5 minutes, waiting that every device got connected, and then change the configuration to lower the polling rate at 1 minute.

Removing those deadlocks fixed this issue for me. Especially the newconnection one.
I'm not sure why they were needed.

@lethalwp
Copy link
Contributor Author

  • also when the timeout happened, i quickly had a "no free socket" error (1024 opened sockets).

Whith this lock removal, the opened sockets remain at ~250

@lethalwp
Copy link
Contributor Author

i've tested running a long-running-query on a device without the lock, and while running, ran a second query.
Both came out correctly. I don't think the locks are needed? What do you think?

@czerwonk czerwonk self-requested a review June 3, 2022 11:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant