[Bug]: SWE-Bench Evaluation fails with connection refused error constantly

[Randolph-zeng]

Is there an existing issue for the same bug?

• I have checked the existing issues.

Describe the bug and reproduction steps

When using the latest openhands.evaluation.swebench script, it always bumped into connection issues and exit the process after running a few issues.

Following is the step that I take to setup the openhands today:

1. Download OpenHands
2. Install the minimum required python environments
   1. conda create —name openhands python=3.12
   2. conda install -c conda-forge nodejs
   3. conda install -c conda-forge poetry
   4. make build
3. Configure the LLM Agent config
   1. make setup-config
   2. manually modify the [llm] in config.toml into [llm.qwenCoder]
4. Run the shell scripts to start the evaluation docker images (do not increase the worker num as there are rate limit of api-keys)
   1. tmux new -s openhands
   2. conda activate openhands
   3. make docker-run (start the runtime before you can interact with it)
   4. tmux new -s swe_eval
   5. conda activate openhands
   6. ./evaluation/swe_bench/scripts/run_infer.sh llm.qwenCoder HEAD CodeActAgent 300 30 5 princeton-nlp/SWE-bench_Lite test

OpenHands Installation

Development workflow

OpenHands Version

main

Operating System

Linux

Logs, Errors, Screenshots, and Additional Context

Instance django__django-11583: 2%| | 7/289 [42:21<31:00:00, 395.75s/it, Test Result: {'git_patch': 'diff --git a/testproject/manage.py b/t18:14:45 - openhands:INFO: shared.py:284 - Finished evaluation for instance django__django-11583: {'git_patch': 'diff --git a/testproject/manage.py b/testproject/manage.py\r\nnew file mode 100755\r\nindex 0000000000..fb52456bdf\r\n--- /dev/null\r\n+++ b/testproject/manage.py\r\n@@ -0,0 +1,21 @@\r\n+#!/usr/bin/env python\r\n+"""Django's command-line utility for administrative tasks."""\r\n+impor...

Instance django__django-11620 - 2024-11-19 18:14:46,665 - ERROR - [runtime a2a1fde5-89f0-4419-b7f3-d7e4c108e98c-84aac80784dce6b7] Unexpected error while running action HTTPConnectionPool(host='localhost', port=36662): Max retries exceeded with url: /execute_action (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6088f1b710>: Failed to establish a new connection: [Errno 111] Connection refused'))
Instance django__django-11620 - 2024-11-19 18:14:46,665 - ERROR - [runtime a2a1fde5-89f0-4419-b7f3-d7e4c108e98c-84aac80784dce6b7] Problematic action: CmdRunAction (source=EventSource.AGENT)
THOUGHT: Great, the server is running correctly on port 8000. Let's test the custom path converter by accessing a URL that should trigger the Http404 exception. We'll use curl to request a URL that does not match the condition in the to_python method of TestConverter.
COMMAND:
curl http://127.0.0.1:8000/456/
Instance django__django-11620 - 2024-11-19 18:14:46,665 - ERROR - HTTPConnectionPool(host='localhost', port=36662): Max retries exceeded with url: /execute_action (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6088f1b710>: Failed to establish a new connection: [Errno 111] Connection refused'))
================ DOCKER BUILD STARTED ================
Instance django__django-11620 - 2024-11-19 18:14:47,683 - ERROR - ----------
Error in instance [django__django-11620]: HTTPConnectionPool(host='localhost', port=36662): Max retries exceeded with url: /execute_action (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6088e14590>: Failed to establish a new connection: [Errno 111] Connection refused')). Stacktrace:
Traceback (most recent call last):
File "/root/.cache/pypoetry/virtualenvs/openhands-ai-5P0i1HOD-py3.12/lib/python3.12/site-packages/urllib3/connection.py", line 199, in _new_conn
sock = connection.create_connection(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/.cache/pypoetry/virtualenvs/openhands-ai-5P0i1HOD-py3.12/lib/python3.12/site-packages/urllib3/util/connection.py", line 85, in create_connection
raise err
File "/root/.cache/pypoetry/virtualenvs/openhands-ai-5P0i1HOD-py3.12/lib/python3.12/site-packages/urllib3/util/connection.py", line 73, in create_connection
sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/root/.cache/pypoetry/virtualenvs/openhands-ai-5P0i1HOD-py3.12/lib/python3.12/site-packages/urllib3/connectionpool.py", line 789, in urlopen
response = self._make_request(
^^^^^^^^^^^^^^^^^^^
File "/root/.cache/pypoetry/virtualenvs/openhands-ai-5P0i1HOD-py3.12/lib/python3.12/site-packages/urllib3/connectionpool.py", line 495, in _make_request
conn.request(
File "/root/.cache/pypoetry/virtualenvs/openhands-ai-5P0i1HOD-py3.12/lib/python3.12/site-packages/urllib3/connection.py", line 441, in request
self.endheaders()
File "/root/miniconda3/envs/openhands/lib/python3.12/http/client.py", line 1326, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/root/miniconda3/envs/openhands/lib/python3.12/http/client.py", line 1085, in _send_output
self.send(msg)
File "/root/miniconda3/envs/openhands/lib/python3.12/http/client.py", line 1029, in send
self.connect()
File "/root/.cache/pypoetry/virtualenvs/openhands-ai-5P0i1HOD-py3.12/lib/python3.12/site-packages/urllib3/connection.py", line 279, in connect
self.sock = self._new_conn()
^^^^^^^^^^^^^^^^
File "/root/.cache/pypoetry/virtualenvs/openhands-ai-5P0i1HOD-py3.12/lib/python3.12/site-packages/urllib3/connection.py", line 214, in _new_conn
raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f6088e14590>: Failed to establish a new connection: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/root/.cache/pypoetry/virtualenvs/openhands-ai-5P0i1HOD-py3.12/lib/python3.12/site-packages/requests/adapters.py", line 667, in send
resp = conn.urlopen(
^^^^^^^^^^^^^
File "/root/.cache/pypoetry/virtualenvs/openhands-ai-5P0i1HOD-py3.12/lib/python3.12/site-packages/urllib3/connectionpool.py", line 843, in urlopen
retries = retries.increment(
^^^^^^^^^^^^^^^^^^
File "/root/.cache/pypoetry/virtualenvs/openhands-ai-5P0i1HOD-py3.12/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment
raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=36662): Max retries exceeded with url: /execute_action (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6088e14590>: Failed to establish a new connection: [Errno 111] Connection refused'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/root/zhongshen/OpenHands/evaluation/utils/shared.py", line 313, in _process_instance_wrapper
result = process_instance_func(instance, metadata, use_mp)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/zhongshen/OpenHands/evaluation/swe_bench/run_infer.py", line 426, in process_instance
return_val = complete_runtime(runtime, instance)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/zhongshen/OpenHands/evaluation/swe_bench/run_infer.py", line 322, in complete_runtime
obs = runtime.run_action(action)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/zhongshen/OpenHands/openhands/runtime/impl/eventstream/eventstream_runtime.py", line 500, in run_action
with send_request(
^^^^^^^^^^^^^
File "/root/zhongshen/OpenHands/openhands/runtime/utils/request.py", line 60, in send_request
response = session.request(method, url, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/.cache/pypoetry/virtualenvs/openhands-ai-5P0i1HOD-py3.12/lib/python3.12/site-packages/requests/sessions.py", line 589, in request
resp = self.send(prep, **send_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/.cache/pypoetry/virtualenvs/openhands-ai-5P0i1HOD-py3.12/lib/python3.12/site-packages/requests/sessions.py", line 703, in send
r = adapter.send(request, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/.cache/pypoetry/virtualenvs/openhands-ai-5P0i1HOD-py3.12/lib/python3.12/site-packages/requests/adapters.py", line 700, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=36662): Max retries exceeded with url: /execute_action (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6088e14590>: Failed to establish a new connection: [Errno 111] Connection refused'))

----------[The above error occurred. Retrying... (attempt 1 of 5)]----------

---

Error in instance [django__django-11620]: HTTPConnectionPool(host='localhost', port=36662): Max retries exceeded with url: /execute_action (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6088e14590>: Failed to establish a new connection: [Errno 111] Connection refused')). Stacktrace:
Traceback (most recent call last):
File "/root/.cache/pypoetry/virtualenvs/openhands-ai-5P0i1HOD-py3.12/lib/python3.12/site-packages/urllib3/connection.py", line 199, in _new_conn
sock = connection.create_connection(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/.cache/pypoetry/virtualenvs/openhands-ai-5P0i1HOD-py3.12/lib/python3.12/site-packages/urllib3/util/connection.py", line 85, in create_connection
raise err
File "/root/.cache/pypoetry/virtualenvs/openhands-ai-5P0i1HOD-py3.12/lib/python3.12/site-packages/urllib3/util/connection.py", line 73, in create_connection
sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/root/.cache/pypoetry/virtualenvs/openhands-ai-5P0i1HOD-py3.12/lib/python3.12/site-packages/urllib3/connectionpool.py", line 789, in urlopen
response = self._make_request(
^^^^^^^^^^^^^^^^^^^
File "/root/.cache/pypoetry/virtualenvs/openhands-ai-5P0i1HOD-py3.12/lib/python3.12/site-packages/urllib3/connectionpool.py", line 495, in _make_request
conn.request(
File "/root/.cache/pypoetry/virtualenvs/openhands-ai-5P0i1HOD-py3.12/lib/python3.12/site-packages/urllib3/connection.py", line 441, in request
self.endheaders()
File "/root/miniconda3/envs/openhands/lib/python3.12/http/client.py", line 1326, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/root/miniconda3/envs/openhands/lib/python3.12/http/client.py", line 1085, in _send_output
self.send(msg)
File "/root/miniconda3/envs/openhands/lib/python3.12/http/client.py", line 1029, in send
self.connect()
File "/root/.cache/pypoetry/virtualenvs/openhands-ai-5P0i1HOD-py3.12/lib/python3.12/site-packages/urllib3/connection.py", line 279, in connect
self.sock = self._new_conn()
^^^^^^^^^^^^^^^^
File "/root/.cache/pypoetry/virtualenvs/openhands-ai-5P0i1HOD-py3.12/lib/python3.12/site-packages/urllib3/connection.py", line 214, in _new_conn
raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f6088e14590>: Failed to establish a new connection: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/root/.cache/pypoetry/virtualenvs/openhands-ai-5P0i1HOD-py3.12/lib/python3.12/site-packages/requests/adapters.py", line 667, in send
resp = conn.urlopen(
^^^^^^^^^^^^^
File "/root/.cache/pypoetry/virtualenvs/openhands-ai-5P0i1HOD-py3.12/lib/python3.12/site-packages/urllib3/connectionpool.py", line 843, in urlopen
retries = retries.increment(
^^^^^^^^^^^^^^^^^^
File "/root/.cache/pypoetry/virtualenvs/openhands-ai-5P0i1HOD-py3.12/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment
raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=36662): Max retries exceeded with url: /execute_action (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6088e14590>: Failed to establish a new connection: [Errno 111] Connection refused'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/root/zhongshen/OpenHands/evaluation/utils/shared.py", line 313, in _process_instance_wrapper
result = process_instance_func(instance, metadata, use_mp)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/zhongshen/OpenHands/evaluation/swe_bench/run_infer.py", line 426, in process_instance
return_val = complete_runtime(runtime, instance)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/zhongshen/OpenHands/evaluation/swe_bench/run_infer.py", line 322, in complete_runtime
obs = runtime.run_action(action)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/zhongshen/OpenHands/openhands/runtime/impl/eventstream/eventstream_runtime.py", line 500, in run_action
with send_request(
^^^^^^^^^^^^^
File "/root/zhongshen/OpenHands/openhands/runtime/utils/request.py", line 60, in send_request
response = session.request(method, url, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/.cache/pypoetry/virtualenvs/openhands-ai-5P0i1HOD-py3.12/lib/python3.12/site-packages/requests/sessions.py", line 589, in request
resp = self.send(prep, **send_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/.cache/pypoetry/virtualenvs/openhands-ai-5P0i1HOD-py3.12/lib/python3.12/site-packages/requests/sessions.py", line 703, in send
r = adapter.send(request, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/.cache/pypoetry/virtualenvs/openhands-ai-5P0i1HOD-py3.12/lib/python3.12/site-packages/requests/adapters.py", line 700, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=36662): Max retries exceeded with url: /execute_action (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6088e14590>: Failed to establish a new connection: [Errno 111] Connection refused'))

----------[The above error occurred. Retrying... (attempt 1 of 5)]----------

Instance django__django-11620 - 2024-11-19 18:14:52,684 - INFO - Starting evaluation for instance django__django-11620.
Hint: run "tail -f evaluation/evaluation_outputs/outputs/princeton-nlp__SWE-bench_Lite-test/CodeActAgent/qwen2.5-coder-32b-instruct_maxiter_30_N_v2.2-no-hint-run_1/infer_logs/instance_django__django-11620.log" to see live logs in a separate shell

Provider List: https://docs.litellm.ai/docs/providers

Provider List: https://docs.litellm.ai/docs/providers

Provider List: https://docs.litellm.ai/docs/providers

Provider List: https://docs.litellm.ai/docs/providers

Instance django__django-11564 - 2024-11-19 18:14:56,765 - ERROR - [runtime 06bf0682-97b9-4489-9e79-c5c0c226f1e9-54b4d423ee0909db] Unexpected error while running action HTTPConnectionPool(host='localhost', port=30754): Max retries exceeded with url: /execute_action (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6088cdcb30>: Failed to establish a new connection: [Errno 111] Connection refused'))
Instance django__django-11564 - 2024-11-19 18:14:56,765 - ERROR - [runtime 06bf0682-97b9-4489-9e79-c5c0c226f1e9-54b4d423ee0909db] Problematic action: IPythonRunCellAction
THOUGHT: Let's explicitly specify the new_str parameter with the modified url method. Here is the modified url method:

def url(self, name, request=None):
    if self.base_url is None:
        raise ValueError("This file is not accessible via a URL.")
    url = filepath_to_uri(name)
    if url is not None:
        url = url.lstrip('/')
    script_name = request.META.get('SCRIPT_NAME', '') if request else ''
    return urljoin(self.base_url, script_name + url)

Let's insert this method manually at line 328:
CODE:
print(file_editor(**{'command': 'insert', 'path': '/workspace/django__django__3.1/django/core/files/storage.py', 'insert_line': 328}))
Instance django__django-11564 - 2024-11-19 18:14:56,765 - ERROR - HTTPConnectionPool(host='localhost', port=30754): Max retries exceeded with url: /execute_action (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6088cdcb30>: Failed to establish a new connection: [Errno 111] Connection refused'))
Instance django__django-11564 - 2024-11-19 18:14:57,783 - ERROR - HTTPConnectionPool(host='localhost', port=30754): Max retries exceeded with url: /execute_action (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6088cdd9d0>: Failed to establish a new connection: [Errno 111] Connection refused'))
Traceback (most recent call last):
File "/root/.cache/pypoetry/virtualenvs/openhands-ai-5P0i1HOD-py3.12/lib/python3.12/site-packages/urllib3/connection.py", line 199, in _new_conn
sock = connection.create_connection(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/.cache/pypoetry/virtualenvs/openhands-ai-5P0i1HOD-py3.12/lib/python3.12/site-packages/urllib3/util/connection.py", line 85, in create_connection
raise err
File "/root/.cache/pypoetry/virtualenvs/openhands-ai-5P0i1HOD-py3.12/lib/python3.12/site-packages/urllib3/util/connection.py", line 73, in create_connection
sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/root/.cache/pypoetry/virtualenvs/openhands-ai-5P0i1HOD-py3.12/lib/python3.12/site-packages/urllib3/connectionpool.py", line 789, in urlopen
response = self._make_request(
^^^^^^^^^^^^^^^^^^^
File "/root/.cache/pypoetry/virtualenvs/openhands-ai-5P0i1HOD-py3.12/lib/python3.12/site-packages/urllib3/connectionpool.py", line 495, in _make_request
conn.request(
File "/root/.cache/pypoetry/virtualenvs/openhands-ai-5P0i1HOD-py3.12/lib/python3.12/site-packages/urllib3/connection.py", line 441, in request
self.endheaders()
File "/root/miniconda3/envs/openhands/lib/python3.12/http/client.py", line 1326, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/root/miniconda3/envs/openhands/lib/python3.12/http/client.py", line 1085, in _send_output
self.send(msg)
File "/root/miniconda3/envs/openhands/lib/python3.12/http/client.py", line 1029, in send
self.connect()
File "/root/.cache/pypoetry/virtualenvs/openhands-ai-5P0i1HOD-py3.12/lib/python3.12/site-packages/urllib3/connection.py", line 279, in connect
self.sock = self._new_conn()
^^^^^^^^^^^^^^^^
File "/root/.cache/pypoetry/virtualenvs/openhands-ai-5P0i1HOD-py3.12/lib/python3.12/site-packages/urllib3/connection.py", line 214, in _new_conn
raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f6088cdd9d0>: Failed to establish a new connection: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/root/.cache/pypoetry/virtualenvs/openhands-ai-5P0i1HOD-py3.12/lib/python3.12/site-packages/requests/adapters.py", line 667, in send
resp = conn.urlopen(
^^^^^^^^^^^^^
File "/root/.cache/pypoetry/virtualenvs/openhands-ai-5P0i1HOD-py3.12/lib/python3.12/site-packages/urllib3/connectionpool.py", line 843, in urlopen
retries = retries.increment(
^^^^^^^^^^^^^^^^^^
File "/root/.cache/pypoetry/virtualenvs/openhands-ai-5P0i1HOD-py3.12/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment
raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=30754): Max retries exceeded with url: /execute_action (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6088cdd9d0>: Failed to establish a new connection: [Errno 111] Connection refused'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/root/zhongshen/OpenHands/evaluation/utils/shared.py", line 313, in _process_instance_wrapper
result = process_instance_func(instance, metadata, use_mp)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/zhongshen/OpenHands/evaluation/swe_bench/run_infer.py", line 426, in process_instance
return_val = complete_runtime(runtime, instance)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/zhongshen/OpenHands/evaluation/swe_bench/run_infer.py", line 322, in complete_runtime
obs = runtime.run_action(action)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/zhongshen/OpenHands/openhands/runtime/impl/eventstream/eventstream_runtime.py", line 500, in run_action
with send_request(
^^^^^^^^^^^^^
File "/root/zhongshen/OpenHands/openhands/runtime/utils/request.py", line 60, in send_request
response = session.request(method, url, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/.cache/pypoetry/virtualenvs/openhands-ai-5P0i1HOD-py3.12/lib/python3.12/site-packages/requests/sessions.py", line 589, in request
resp = self.send(prep, **send_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/.cache/pypoetry/virtualenvs/openhands-ai-5P0i1HOD-py3.12/lib/python3.12/site-packages/requests/sessions.py", line 703, in send
r = adapter.send(request, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/.cache/pypoetry/virtualenvs/openhands-ai-5P0i1HOD-py3.12/lib/python3.12/site-packages/requests/adapters.py", line 700, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=30754): Max retries exceeded with url: /execute_action (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6088cdd9d0>: Failed to establish a new connection: [Errno 111] Connection refused'))
Instance django__django-11564 - 2024-11-19 18:14:57,784 - ERROR - HTTPConnectionPool(host='localhost', port=30754): Max retries exceeded with url: /execute_action (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6088cdd9d0>: Failed to establish a new connection: [Errno 111] Connection refused'))
Traceback (most recent call last):
File "/root/.cache/pypoetry/virtualenvs/openhands-ai-5P0i1HOD-py3.12/lib/python3.12/site-packages/urllib3/connection.py", line 199, in _new_conn
sock = connection.create_connection(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/.cache/pypoetry/virtualenvs/openhands-ai-5P0i1HOD-py3.12/lib/python3.12/site-packages/urllib3/util/connection.py", line 85, in create_connection
raise err
File "/root/.cache/pypoetry/virtualenvs/openhands-ai-5P0i1HOD-py3.12/lib/python3.12/site-packages/urllib3/util/connection.py", line 73, in create_connection
sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/root/.cache/pypoetry/virtualenvs/openhands-ai-5P0i1HOD-py3.12/lib/python3.12/site-packages/urllib3/connectionpool.py", line 789, in urlopen
response = self._make_request(
^^^^^^^^^^^^^^^^^^^
File "/root/.cache/pypoetry/virtualenvs/openhands-ai-5P0i1HOD-py3.12/lib/python3.12/site-packages/urllib3/connectionpool.py", line 495, in _make_request
conn.request(
File "/root/.cache/pypoetry/virtualenvs/openhands-ai-5P0i1HOD-py3.12/lib/python3.12/site-packages/urllib3/connection.py", line 441, in request
self.endheaders()
File "/root/miniconda3/envs/openhands/lib/python3.12/http/client.py", line 1326, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/root/miniconda3/envs/openhands/lib/python3.12/http/client.py", line 1085, in _send_output
self.send(msg)
File "/root/miniconda3/envs/openhands/lib/python3.12/http/client.py", line 1029, in send
self.connect()
File "/root/.cache/pypoetry/virtualenvs/openhands-ai-5P0i1HOD-py3.12/lib/python3.12/site-packages/urllib3/connection.py", line 279, in connect
self.sock = self._new_conn()
^^^^^^^^^^^^^^^^
File "/root/.cache/pypoetry/virtualenvs/openhands-ai-5P0i1HOD-py3.12/lib/python3.12/site-packages/urllib3/connection.py", line 214, in _new_conn
raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f6088cdd9d0>: Failed to establish a new connection: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/root/.cache/pypoetry/virtualenvs/openhands-ai-5P0i1HOD-py3.12/lib/python3.12/site-packages/requests/adapters.py", line 667, in send
resp = conn.urlopen(
^^^^^^^^^^^^^
File "/root/.cache/pypoetry/virtualenvs/openhands-ai-5P0i1HOD-py3.12/lib/python3.12/site-packages/urllib3/connectionpool.py", line 843, in urlopen
retries = retries.increment(
^^^^^^^^^^^^^^^^^^
File "/root/.cache/pypoetry/virtualenvs/openhands-ai-5P0i1HOD-py3.12/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment
raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=30754): Max retries exceeded with url: /execute_action (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6088cdd9d0>: Failed to establish a new connection: [Errno 111] Connection refused'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/root/zhongshen/OpenHands/evaluation/utils/shared.py", line 313, in _process_instance_wrapper
result = process_instance_func(instance, metadata, use_mp)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/zhongshen/OpenHands/evaluation/swe_bench/run_infer.py", line 426, in process_instance
return_val = complete_runtime(runtime, instance)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/zhongshen/OpenHands/evaluation/swe_bench/run_infer.py", line 322, in complete_runtime
obs = runtime.run_action(action)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/zhongshen/OpenHands/openhands/runtime/impl/eventstream/eventstream_runtime.py", line 500, in run_action
with send_request(
^^^^^^^^^^^^^
File "/root/zhongshen/OpenHands/openhands/runtime/utils/request.py", line 60, in send_request
response = session.request(method, url, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/.cache/pypoetry/virtualenvs/openhands-ai-5P0i1HOD-py3.12/lib/python3.12/site-packages/requests/sessions.py", line 589, in request
resp = self.send(prep, **send_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/.cache/pypoetry/virtualenvs/openhands-ai-5P0i1HOD-py3.12/lib/python3.12/site-packages/requests/sessions.py", line 703, in send
r = adapter.send(request, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/.cache/pypoetry/virtualenvs/openhands-ai-5P0i1HOD-py3.12/lib/python3.12/site-packages/requests/adapters.py", line 700, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=30754): Max retries exceeded with url: /execute_action (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6088cdd9d0>: Failed to establish a new connection: [Errno 111] Connection refused'))
Instance django__django-11742 - 2024-11-19 18:14:57,788 - INFO - Starting evaluation for instance django__django-11742.
Hint: run "tail -f evaluation/evaluation_outputs/outputs/princeton-nlp__SWE-bench_Lite-test/CodeActAgent/qwen2.5-coder-32b-instruct_maxiter_30_N_v2.2-no-hint-run_1/infer_logs/instance_django__django-11742.log" to see live logs in a separate shell
ERROR:root: File "/root/zhongshen/OpenHands/evaluation/swe_bench/run_infer.py", line 536, in
run_evaluation(
File "/root/zhongshen/OpenHands/evaluation/utils/shared.py", line 413, in run_evaluation
for result in results:
File "/root/miniconda3/envs/openhands/lib/python3.12/multiprocessing/pool.py", line 873, in next
raise value

ERROR:root:<class 'RuntimeError'>: Maximum error retries reached for instance django__django-11564
Instance django__django-11583: 2%| | 7/289 [42:34<28:34:53, 364.87s/it, Test Result: {'git_patch': 'diff --git a/testproject/manage.py b/t
(openhands) [root@iZrj9hd2z42esdxzd51ghoZ OpenHands]#

[Randolph-zeng]

Hi, before I dive in deeper into the bug, is there any insight or pointer that I should look into first ? Thanks!