Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feature] Support reconnect host when connectionpool raise error #243

Open
DachuanZhao opened this issue Jul 29, 2020 · 2 comments
Open

Comments

@DachuanZhao
Copy link

DachuanZhao commented Jul 29, 2020

Here is the code

class ConnectionPool(object):

    def __init__(self, size, hosts, **kwargs):
        if not isinstance(size, int):
            raise TypeError("Pool 'size' arg must be an integer")

        if not size > 0:
            raise ValueError("Pool 'size' arg must be greater than zero")

        logger.debug(
            "Initializing connection pool with %d connections", size)

        self._lock = threading.Lock()
        self._queue = queue.LifoQueue(maxsize=size)
        self._thread_connections = threading.local()
        self._hosts = hosts

        self.connection_kwargs = kwargs
        self.connection_kwargs = kwargs
        self.connection_kwargs['autoconnect'] = False

        # add connection
        host_size = len(hosts)
        for i in range(size):
            self.connection_kwargs['host'] = hosts[i % host_size]
            connection = Connection(**self.connection_kwargs)
            self._queue.put(connection)

    def _acquire_connection(self, timeout=None):
        try:
            return self._queue.get(True, timeout)
        except queue.Empty:
            raise NoConnectionsAvailable(
                "No connection available from pool within specified "
                "timeout")

    def _return_connection(self, connection):
        self._queue.put(connection)

    @contextlib.contextmanager
    def connection(self, timeout=None):
        connection = getattr(self._thread_connections, 'current', None)
        return_after_use = False
        if connection is None:
            return_after_use = True
            connection = self._acquire_connection(timeout)
            with self._lock:
                self._thread_connections.current = connection
        try:
            yield connection
        except (TException, socket.error) as e:
            logger.info("Replacing tainted pool connection")
            # add new connection
            host = connection.host
            _connection_kwargs = self.connection_kwargs
            for _host in self._hosts:
                if host != _host:
                    _connection_kwargs['host'] = _host
                    break
            connection = Connection(**_connection_kwargs)
            with self._lock:
                self._thread_connections.current = connection
            raise
        finally:
            if return_after_use:
                connection = self._thread_connections.current
                del self._thread_connections.current
                connection.close()
                self._return_connection(connection)
@wbolster
Copy link
Member

i think you forgot to submit any actual content. this is not productive nor does it show a cooperative attitude.

@DachuanZhao DachuanZhao changed the title [feature] Support multi thrift-server in happybase.ConnectionPool [feature] Support reconnect host when connectionpool raise error Jul 31, 2020
@DachuanZhao
Copy link
Author

i think you forgot to submit any actual content. this is not productive nor does it show a cooperative attitude.

Sorry , and I change the title and add the code now .

@wbolster wbolster reopened this Jul 31, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants