Fix for Playground setup possible error (#3061)

# Description
Updated handling of setting up log filter override Unix socket in
`observe` crate. In some specific situation the socet cannot be created
due to `Address already in use` error in `playground-autopilot` and
`playground-orderbook` containers when `autopilot` or `orderbook`
service previously crashed. This crash can be caused by misconfigruation
of environment (RPC node url) or just by stopping manually
`playground-chain` container.

# Changes
In case of `UnixListener::bind()` fails on socket file with OS error
`AddrInUse` afromentioned file is removed and a new attempt is made to
bind the socket.
If there is any error during the file remove it will be only logged as
it could happen that file will be removed in mean time by other process.

## How to test
Follow `playground/README.md` instruction, but in environment setup
provide wrong address to the node (`ETH_RPC_URL` variable).
Alternativelly when playground containers are working fine, stop
`playground-chain` container and restart `playground-autopilot` or
`playground-orderbook` containers. They will enter a loop of endless
restarts.

---------

Co-authored-by: Martin Magnus <[email protected]>

crates/observe/Cargo.toml

@@ -14,7 +14,7 @@ pin-project-lite = "0.2.14"
 prometheus = { workspace = true }
 prometheus-metric-storage = { workspace = true }
 time = { workspace = true }
-tokio = { workspace = true, features = [] }
+tokio = { workspace = true, features = [ "fs" ] }
 tracing = { workspace = true }
 tracing-subscriber = { workspace = true, features = ["env-filter", "fmt", "time"] }
 

crates/observe/src/tracing_reload_handler.rs

@@ -8,7 +8,7 @@ use {
 
 /// Spawns a new thread that listens for connections to a UNIX socket
 /// at "/tmp/log_filter_override_<process_name>_<pid>".
-/// Whenever a line gets writtedn to that socket the reload handler
+/// Whenever a line gets written to that socket the reload handler
 /// uses it as the new log filter.
 /// To reset to the original log filter send the message "reset".
 pub(crate) fn spawn_reload_handler<T: 'static>(
@@ -21,6 +21,7 @@ pub(crate) fn spawn_reload_handler<T: 'static>(
 
         let socket_path = format!("/tmp/log_filter_override_{name}_{id}.sock");
         tracing::warn!(file = socket_path, "open log filter reload socket");
+        let _ = tokio::fs::remove_file(&socket_path).await;
         let handle = SocketHandle {
             listener: UnixListener::bind(&socket_path).expect("socket handle is unique"),
             socket_path,