Tracing Notifications on the Backend

[gnunicorn]

and ideally all the way to the client to make sure we are not loosing them somewhere there before they even make it to the server.

Synapse and Sygnal both have support for OpenTelemetry/OpenTracing, so does the matrix-rust-sdk, and Sentry. If I am not mistaken we should technically be able to trace the notifications throughout the entire stack and thus figure out where they go lost....

[gnunicorn]

so, it appears that though we should see exceptions and warning in the logs, for some reason they do not show up. On better news, there is a promotheus metric to tell us how often submitting to the push-server failed in synapse...