Tech Note 1

ZooKeeper watches are single threaded.

Details

When your watcher is called, it should return as quickly as possible. All ZooKeeper watchers are serialized – processed by a single thread. Thus, no other watchers can be processed while your watcher is running. For example, a Curator user had a watcher handler something like this:

...
InterProcessMutex   lock = ...

public void process(WatchedEvent event)
{
    lock.acquire();
       ...
}

This cannot work. Curator’s InterProcessMutex relies on ZooKeeper watchers getting notified. The code above, however, is holding on to the ZooKeeper watcher processing thread. The way to fix this is to run the code that needs a lock in a separate thread. e.g.

...
InterProcessMutex   lock = ...
ExecutorService    service = ...

public void process(WatchedEvent event)
{
    service.submit(new Callable<Void>(){
        Void call() {
            lock.acquire();
              ...
        }
    });
}