Attempt to decode JSON with unexpected mimetype: text/html

[xxwhirlpool]

Hi, I was running my xmpp instance just fine up until a server apt upgrade and restart yesterday. Now, I am unable to log into my snikket accounts, with errors on the web portal and on the Gajim client.

The web portal error always reads as an internal error with the following error codes:

GURU MEDITATION
aiohttp.client_exceptions.ContentTypeError
error_id=ULBBUNMJK365S

When I check the logs upon portal login attempt, I see the following (my subdomain changed to 'chat.example.com'):

snikket-proxy   | 2024/09/09 21:00:23 [error] 22#22: *13 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.1.151, server: chat.example.com, request: "POST /login HTTP/1.1", upstream: "http://[::1]:5765/login", host: "chat.example.com", referrer: "https://chat.example.com/login"
snikket-portal  | ERROR:snikket_web:error_id=ULBBUNMJK365S returning 500 status page for exception
snikket-portal  | Traceback (most recent call last):
snikket-portal  |   File "/usr/lib/python3/dist-packages/quart/app.py", line 1673, in full_dispatch_request
snikket-portal  |     result = await self.dispatch_request(request_context)
snikket-portal  |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
snikket-portal  |   File "/usr/lib/python3/dist-packages/quart/app.py", line 1718, in dispatch_request
snikket-portal  |     return await self.ensure_async(handler)(**request_.view_args)
snikket-portal  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
snikket-portal  |   File "/opt/snikket-web-portal/snikket_web/main.py", line 80, in login
snikket-portal  |     await client.login(jid, password)
snikket-portal  |   File "/opt/snikket-web-portal/snikket_web/prosodyclient.py", line 445, in login
snikket-portal  |     token_info = await self._oauth2_bearer_token(
snikket-portal  |                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
snikket-portal  |   File "/opt/snikket-web-portal/snikket_web/prosodyclient.py", line 409, in _oauth2_bearer_token
snikket-portal  |     auth_info: typing.Mapping[str, str] = (await resp.json())
snikket-portal  |                                            ^^^^^^^^^^^^^^^^^
snikket-portal  |   File "/usr/lib/python3/dist-packages/aiohttp/client_reqrep.py", line 1104, in json
snikket-portal  |     raise ContentTypeError(
snikket-portal  | aiohttp.client_exceptions.ContentTypeError: 0, message='Attempt to decode JSON with unexpected mimetype: text/html; charset=utf-8', url=URL('http://127.0.0.1:5280//oauth2/token')
snikket-proxy   | 192.168.1.151 - - [09/Sep/2024:21:00:23 +0000] "POST /login HTTP/1.1" 500 1633 "https://chat.example.com/login" "Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/115.0"

And when I try to log in via Gajim, I see the following error in the logs from snikket service:

info       Client connected
warn       No stream features to offer on insecure session. Check encryption and security settings.
info       Client disconnected: connection closed

I've updated Snikket with docker compose pull so I was on the latest version. I did this pull after the apt upgrade on the machine. I'm not sure if this is a Snikket problem or a misconfiguration on my part. My ports are forwarded.

snikket.conf contents (domains and email changed for privacy):

SNIKKET_DOMAIN=chat.example.com
[email protected]
SNIKKET_TWEAK_HTTP_PORT=5080
SNIKKET_TWEAK_HTTPS_PORT=5443
SNIKKET_LETSENCRYPT_TOS_AGREE=Y
SNIKKET_TWEAK_HTTP_TLS_VERSIONS=TLSv1.2
SNIKKET_TWEAK_HTTP_TLS_CIPHERS=

I am new to opening Github issues and using Github in general so please excuse any mistakes. Thank you.

[mwild1]

Hi! Sorry to hear about the trouble with your setup. I'm not sure what the cause is, but I have a hunch...

I suspect you may have another XMPP server (such as prosody or ejabberd) running on the host machine. This would also make sense, if the server was stopped, an apt upgrade may have updated the XMPP server package which would cause it to start up again and interfere with Snikket.

In your logs, the Snikket web portal is trying to communicate with the XMPP server and it is getting an unexpected response. I suspect it's communicating with the XMPP server on the host, instead of the one that is bundled with Snikket.

The solution is to remove any XMPP server on the host, and then restart Snikket (docker compose down followed by docker compose up -d). After that (if my guess is correct) then it should begin working again!

[xxwhirlpool]

So I did some thorough checking, because this does make sense, I had tried other XMPP servers before landing on Snikket, but I can't seem to find anything else running that could be blocking Snikket. I don't have prosody or ejabberd installed anymore, and I checked with netstat for every port that Snikket uses with my instance down, and nothing came up.

The only things I can think of are that I did try to deploy XMPP-web, and ended up not using it, so I looked for traces of that, but couldn't find files anywhere for it on my systems. Additionally, it's possible my reverse proxy may be misconfigured, but I can't see how it could be, as it looks near identical to the base example on the docs with localhost swapped out for my system's IPv4 address (my Caddy instance runs on a different machine), and the domains swapped out from the examples.

[xxwhirlpool]

hello i know this is months old but just popping in to say i looked at this with a fresh mind like 7 months later, did some debugging, and somehow it started working again? it persisted after a docker pull but i just did some messing around and suddenly it's working again.

to anyone that has this issue i can't really offer any fixes because i don't know what i did to fix this necessarily or what was causing it but it might be the internal prosody configuration. i saw some conflicts with port 5280 which is used by the internal HTTP API. so try looking into that.

marking this as closed as it is now fixed for me.