Drop special-casing of syslog format

[msakrejda]

This was a bug introduced in #494. The code here was intended to pass
syslog-format messages to the old parsing code, but the old parsing
code did not actually use this code path for syslog. All the
syslog-specific processing is actually in
input/system/selfhosted/syslog_handler.go, and then the messages are
handed off to the parser. They will be matched against log_line_prefix
like any other log line.

Before #494 this worked fine, because #145 introduced a couple of
log_line_prefix options that could be used with this format.

The RsyslogRegexp was actually introduced in
73712dd (before our workflow used
pull requests consistently), and appears to have been intended to tail
log files in an older syslog format (the obsolete RFC3164, versus the
RFC5424 we currently support).

Drop mistaken delegating of syslog message handling to obsolete
parsing mechanism. Also drop this mechanism: it was broken by #494
anyway, and given it's a legacy format (RFC5424 came out in 2009),
there's no reason to believe it's still in use.

TODO:

• add CI test coverage (see below)
  • one of the reasons this bug occurred in the first place is our
    lack of tests around this; I'm looking into adding an integration
    test coverage

[lfittl]

The RsyslogRegexp was actually introduced in
73712dd (before our workflow used
pull requests consistently), and appears to have been intended to tail
log files in an older syslog format (the obsolete RFC3164, versus the
RFC5424 we currently support).

I think the use case for this might have been syslog written to a file, vs syslog being received via our Syslog server.

My memory is a bit hazy here, but I believe we might have added this since a common setup on one of the Linux distributions (either built-in packages or PGDG) caused this to happen. I want to say this occurred with Debian or a Debian derivative, where the logs are in /var/log/postgresql (or something like it) and written by syslog vs by Postgres itself.

Maybe worth double checking if we can find any evidence of this in a common distro (I'd probably check Debian and Ubuntu first) before dropping support for syslog-written-to-a-file (which I think as you note already broke when we refactored this recently, but I believe may have existed before then for a good reason, despite it not being used for the syslog server use case).

[msakrejda]

I run Ubuntu locally; the output there is not (or no longer) in that format. I tried Rocky Linux 9 to try to cover RHEL derivatives, and it looks like on startup, the postmaster logs to journalctl in that format, but immediately starts sending log output to (by default) a log directory in $PGDATA. That output is just the Postgres log_line_prefix-prefixed lines, not anything else.

[msakrejda]

So, I've tested this manually (for logs coming through syslog), but I have not figured out how we can add a CI test for this. Most of the current integration tests all use --dry-run to analyze the snapshot, but neither that nor --dry-run-logs support actually processing syslog. The reload test does not use --dry-run, but it also doesn't actually care about submitting data. Without --dry-run, in order to test that we can ingest logs over syslog, we need a server to get grants from and to send snapshots to, or to refactor the code to tease out the separate concerns.

Unless someone has other ideas, I think we should skip adding CI coverage for now.

[msakrejda]

I've now also confirmed that Bullseye, the oldest supported Debian release does not use the syslog format for Postgres logs (neither the default packages nor PGDG packages).

[lfittl]

I've now also confirmed that Bullseye, the oldest supported Debian release does not use the syslog format for Postgres logs (neither the default packages nor PGDG packages).

Great, thank you for checking! I think we're good to make this change and come back to it if we get reports of this causing issues.