Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Abort QueueItem.block_until_building after 100 HTTPErrors #803

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

Lucas-C
Copy link

@Lucas-C Lucas-C commented Dec 9, 2020

This aim to avoid this kind of situation:

2020-12-08 02:27:39,955 [ERROR] jenkinsbase.py:81 - Failed request at https://jenkins.example.com/queue/item/20799/api/python with params: None 
2020-12-08 02:27:39,956 [DEBUG] queue.py:153 - 503 Server Error: Service Unavailable for url: https://jenkins.example.com/queue/item/20799/api/python
2020-12-08 02:27:41,429 [ERROR] jenkinsbase.py:81 - Failed request at https://jenkins.example.com/queue/item/20798/api/python with params: None 
2020-12-08 02:27:41,429 [DEBUG] queue.py:153 - 503 Server Error: Service Unavailable for url: https://jenkins.example.com/queue/item/20798/api/python
2020-12-08 02:27:48,277 [ERROR] jenkinsbase.py:81 - Failed request at https://jenkins.example.com/queue/item/20798/api/python with params: None 
2020-12-08 02:27:48,277 [DEBUG] queue.py:153 - 404 Client Error: Not Found for url: https://jenkins.example.com/queue/item/20798/api/python
2020-12-08 02:27:48,278 [ERROR] jenkinsbase.py:81 - Failed request at https://jenkins.example.com/queue/item/20799/api/python with params: None 
2020-12-08 02:27:48,278 [DEBUG] queue.py:153 - 404 Client Error: Not Found for url: https://jenkins.example.com/queue/item/20799/api/python
2020-12-08 02:27:55,125 [ERROR] jenkinsbase.py:81 - Failed request at https://jenkins.example.com/queue/item/20798/api/python with params: None 
2020-12-08 02:27:55,125 [DEBUG] queue.py:153 - 404 Client Error: Not Found for url: https://jenkins.example.com/queue/item/20798/api/python
2020-12-08 02:27:55,127 [ERROR] jenkinsbase.py:81 - Failed request at https://jenkins.example.com/queue/item/20799/api/python with params: None 
2020-12-08 02:27:55,127 [DEBUG] queue.py:153 - 404 Client Error: Not Found for url: https://jenkins.example.com/queue/item/20799/api/python

With MANY HTTP errors occuring, and nothing ever noticing the issue :

$ grep -cF '503 Server Error' execution.log
384
$ grep -cF '404 Client Error' execution.log
1261

This PR aims to detect a problem after Jenkins returned 100 HTTP errors while polling the same job (the value is configurable)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant