While listening to music in my car, I suddenly found my connection failing. Once I got home, I noticed that my phone’s IP address had been blocked on my firewall and digging through the logs, I found these entries in my HAProxy:
These show Symfonium accessing the server normally, but then intermittently sending HTTP requests without the hosts header, which causes the external proxy to respond with a 403 denied error, and ultimately results in the remote IP getting blacklisted as my log scanning processes assume it is attacking my webserver.
Logs:
Upload description: steltek
Additional information:
Is this potentially caused by some issue/mis-configuration in my reverse proxy? (Issuing redirects to the default host/IP?)
Reproduction steps:
Host Navidrome behind HAProxy with a specific hostname and ACL rules like these:
use_backend subsonic_http if { hdr(host) -i external.hostname.com }
(Include a default backend that returns an error.)
Use Symfonium to play music. → Some requests will come in without the host header and fall through to the default backend.
Unfortunately, the entire communication between Symfonium and the proxy is encrypted with TLS1.3 making it very difficult to decrypt and read. I was hoping the debug logs would show something obvious.
On the Navidrome side, I don’t see any redirects as responses to the requests to /rest/stream at least:
Thanks for the info on the logs. I had forgotten that it’s possible go export them. (There’s no obvious “Logs” menu and the “Manage generated files” didn’t jump out at me.)
Just FYI, I think I’ve got it worked out. Managed to MitM Symfonium and here’s the request I see being sent that falls through to the wrong reverse proxy backend:
GET /rest/stream.view?id=ICCPUohlJNGojJ3BZ2Jyv1&u=admin&t=tredacted&s=sredacted&v=1.13.0&c=Symfonium&f=json HTTP/1.1
User-Agent: Lavf/60.16.100
Accept: */*
Range: bytes=0-
Connection: close
Host: external.hostname.com:443
Icy-MetaData: 1
My proxy config did not redirect to the correct backend when the header states “external.hostname.com:443” with the port included. (Not sure why some requests have only the hostname and others include the port, but that’s the difference that was causing me trouble.)
For anyone using HAProxy, my workaround was:
use_backend subsonic_http if { hdr(host) -i external.hostname.com external.hostname.com:443 }
… which will now match both variants, or use this at the top of your http frontend to strip the port from the header:
