---
title: "Reading the logs: levels, files, the Logs page, following a connection, and fail2ban"
url: https://www.hmailserver.co.uk/topic/28220/reading-the-logs-levels-files-the-logs-page-following-a-connection-and-fail2ban
author: "Progressiverobot"
published: 2026-09-27T19:02:13.731Z
updated: 2026-09-27T19:02:13.731Z
category: "General support"
tags: ["logs","diagnostics","linux","windows","systemd"]
site: hMailServer Community Forum
---

# Reading the logs: levels, files, the Logs page, following a connection, and fail2ban

*Written for 6.3.4. hMailServer 6.3.7 (17 October 2026) fixes the SMTP log writing some credentials in the clear with SMTP logging on: an `AUTH PLAIN` sent before `EHLO` or not in its one exact form, the bearer token of `AUTH XOAUTH2` and `AUTH OAUTHBEARER`, and the lines of SCRAM-SHA-256, OAUTHBEARER and GSSAPI went into the log as the client sent them, base64 and readable, in every version from 6.0.0 to 6.3.6. An SMTP log written by 6.3.6 or earlier may hold them: redact every line of a SASL exchange before posting one. A database write that failed is now reported (`HM5032`) where it was taken as done, so an error log that was quiet may name failures that were hidden; and the IMAP and POP3 listeners can take the client's address from a PROXY protocol header, as the SMTP listener does, both off by default. See [the 6.3.7 announcement](/topic/28277). hMailServer 6.3.8 (2 November 2026) no longer reports a stop of the servers as errors of their own: work that outlasted a stop, a reinitialise among them, writes a line to the debug log where it wrote `HM5136` or another code to the error log and ran an installed OnError script, so the error log of a server that reinitialises may be quieter; and an event script's call that meets a stop fails with "The server is stopping. The request was not completed.", its script error going to the debug log. A failed account sign-in at the REST listener from loopback, as through a reverse proxy on the same machine, is counted against the name alone and writes no `Auto-ban: logon failure` line for fail2ban to read. See [the 6.3.8 announcement](/topic/28278).*

This topic covers what hMailServer writes to disk and which file to open, the settings that decide how much is written and kept, the live views, following one connection (since 6.3.4), a copy to syslog or the journal, and the fail2ban filter the Linux package installs. The one thing to know first: open `ERROR_hmailserver_<date>.log` before anything else. It holds only the entries the server itself classed as an error, and its mere existence is a diagnostic, because the Control Panel's built-in self-test (**Monitoring & troubleshooting → Diagnostics**, so Windows) fails when any `ERROR*` file is present in the log directory.

### Where the logs are

| Platform | Log directory | Set by |
|---|---|---|
| Windows | `C:\Program Files\hMailServer\Logs` by default | `[Directories] LogFolder` in `hMailServer.INI` |
| Linux | `/var/log/hmailserver`, created by the package as `0750 hmailserver:hmailserver` and left in place when the package is removed | `[Directories]` in `/etc/hmailserver/hMailServer.ini` |

The format is the same on both platforms. On Linux the journal (`journalctl -u hmailserver`) carries the startup line, anything written to standard error and the operational events; a memory-safety fault is recorded in `/var/log/hmailserver/crash-oracle.log` as one line (signal, address, thread, event number) before the process dies by the signal it caught. `[Directories]` is per machine and is never mirrored into the database, so do not copy another server's file over this one's.

### The files

| File | Holds | Rotates |
|---|---|---|
| `hmailserver_<date>.log` | Everything: application lines, and the protocol conversations unless they are split out | Daily, by name |
| `ERROR_hmailserver_<date>.log` | Only entries the server classed as an error | Daily, by name |
| `hmailserver_SMTP_<date>.log`, `hmailserver_IMAP_<date>.log`, `hmailserver_POP3_<date>.log` | The per-protocol conversations, when `SepSvcLogs=1` | Daily, by name |
| `hmailserver_backup.log` | Every backup and restore run, line by line | Never: one growing file. `LogDeleteDays` matches it by name but deletes only a file nothing has written to for that many days |
| `hmailserver_events.log` | Event-script output | Never, as above |
| `hmailserver_awstats.log` | The AWStats-format feed, when AWStats logging is on | Never, as above |
| `hMailServer_messagestore_consistency.report` | The most recent message-store consistency scan: one line per message whose file is missing, as `messageid<TAB>account<TAB>expected-path` | Overwritten each run |

The backup and event logs are among the files the server marks with a byte-order mark; they are UTF-16LE on both platforms, so a file written on one reads on the other.

A log line looks like `"IMAPD"  5220  41  "2026-09-17 09:00:00.120"  "192.0.2.7"  "SENT: A1 OK LOGIN completed"`, where `IMAPD` is the category and `41` is the session id that every line of that conversation carries. The conversation categories are `SMTPD`, `SMTPC`, `POP3D`, `POP3C` and `IMAPD`; `APPLICATION`, `TCPIP`, `DEBUG` and `ERROR` are the rest.

### The settings

| Setting | Default | What it does | What it looks like when it is wrong |
|---|---|---|---|
| `LogLevel` | `9` | Not the choice of log types, which is the Logging page's tick-boxes (or `PUT /api/v1/settings/logging`). The Settings Reference records two effects: long lines are cut only when it is 2 or lower with debug off, and IMAP `SENT:` lines for FETCH and STATUS are logged only when it is above 2 or debug is on | — |
| Debug messages (the Logging page) | off | Adds the `LOG_DEBUG` lines, including the accept-pipeline stage timings | Left on, a busy server fills the disk in days, and a full disk stops mail |
| `SepSvcLogs` | `0` | Splits SMTP, IMAP and POP3 into their own files | Off, so a search for one protocol wades through all three |
| `LogDeleteDays` | `0` | Deletes files named `hmailserver_*.log` or `ERROR_hmailserver_*.log` whose last write is older than this many days, and nothing else in the directory. `0` keeps everything for ever | Left at `0` on a server with debug logging on: the usual cause of a full log volume |
| `JsonLogging` | `0` | Writes JSON lines, for Elasticsearch, Loki or Splunk | Overridden while `Logging.LogFormat` is NCSA: the pipeline gets NCSA lines and no error |
| `SlowQueryLogMilliseconds` | `0` (off) | Logs, and counts in `hmailserver_db_slow_queries_total`, any database statement slower than this | Off, so a database that has become slow leaves no trace but the latency histogram |
| `WindowsEventLogLevel` | `2` | How much also goes to the Windows event log | — |

**Where they are set.** On Windows, **Monitoring & troubleshooting → Logging** in the Control Panel. On either platform the table's `[Settings]` keys (all but Debug messages) are set with `PUT /api/v1/settings/ini/{name}` or `hmctl ini set`. On Linux there is no Control Panel, so the log categories, debug messages among them, are set with `PUT /api/v1/settings/logging`, `hmctl settings set logging` or the Deck's Settings page. There, `WindowsEventLogEnabled=1` is a Windows inheritance in name only, because on Linux it means syslog, and the operational events reach the journal through it. Since 6.3.4 these keys are stored in the database, whose value is the one used: where the service can write `hMailServer.INI`, a line edited there is put back and reported as HM5804. [How the settings store works](/topic/28222) has the rest, `[SettingsOverride]` (HM5807) among it.

**Rotation.** On Windows, `LogDeleteDays` is the rotation. On Linux the package installs `/etc/logrotate.d/hmailserver`; the rule uses `copytruncate` because the server does not reopen its log on a signal, and the rule says so at length.

**Debug logging: on to diagnose, off afterwards.** The stage-timing lines that diagnosing stalled mail depends on are debug lines; the slow ones (ten seconds or more) are written at application level as well. Leaving debug on matters because a full disk stops mail: the application log warns below `DiskSpaceWarningThresholdMB` (1024 MB), and below `MinimumFreeDiskSpaceMB` (100 MB) on the message store's volume the server answers `452 4.3.1 Insufficient system storage.` and senders retry.

### The live views

**Windows: the Control Panel.** **Monitoring & troubleshooting → Live logs** is a real-time, colour-coded tail of what the server is writing; every page in that group but Discovery reads the running service over COM (Discovery has no COM interface, so it reads the REST listener and needs `RestApiPort` set). **Diagnosing stalled mail**, in the same group, has a one-click debug-logging toggle.

**Both platforms: the Control Deck and `hmctl`.** The browser Control Deck at `/` on the REST listener has a **Logs** page that lists the log files and shows any file's last 200 or 2,000 lines, without a share to the log directory. Since 6.3.4, `hmctl log list` and `hmctl log tail <name> --lines N` read the same over `GET /api/v1/logs` and `GET /api/v1/logs/{name}`, and work against a Windows or a Linux server because they go over HTTP rather than DCOM. The Linux packages install `hmctl` as `/usr/bin/hmctl`; the `HmailServer` PowerShell module is the same vocabulary for PowerShell.

**The files themselves, from a shell.** On Windows:

```powershell
$log = "C:\Program Files\hMailServer\Logs\hmailserver_" + (Get-Date -Format "yyyy-MM-dd") + ".log"
Get-Content $log -Tail 0 -Wait | Select-String "SMTPD|SMTPConnection"
```

On Linux:

```sh
tail -f /var/log/hmailserver/hmailserver_$(date +%F).log
journalctl -u hmailserver -f
```

### Following one connection, since 6.3.4

Every SMTP, POP3 and IMAP conversation is a session with a number, and every line the log writes for it carries that number. Until 6.3.4 that meant searching the day's log for the number afterwards. Since 6.3.4 the **Connections** section at the top of the Deck's **Logs** page lists every connection the server has open now, newest first, and the last two hundred that ended: the id, the protocol, the address it came from, the port it came in on, the account once it has signed in, and when it started or ended. **Follow** beside any of them opens a pane holding every line the log has for that session and keeps adding lines as the server writes them; **Copy** puts the whole pane on the clipboard; **Stop following** ends it. A session that ended a second ago is still listed, so an SMTP conversation that lasted a quarter of a second can be picked and read.

**How it works, and what that means for you.**

- The server keeps the last 4,096 log entries in memory, across every session; the ring is always on, and nothing survives a restart. A debug-logged IMAP client can fill it in minutes, so a follow whose cursor points before the oldest entry still held answers `dropped: true` and starts from what is left; the page says so. A line older than that is in the log files and nowhere else.
- The conversation lines (`SMTPD`, `SMTPC`, `POP3D`, `POP3C`, `IMAPD`) carry the session id in their own column. `DEBUG` and `TCPIP` lines carry none and are filed by the first number after the word *session* in their text (`TCP connection started for session 41`, `Ending session 41`); that is an inference, and the text is not always the server's own. A line that does neither is nobody's and is not shown.
- It shows what the log mask lets through, and no more. With IMAP logging off, an IMAP session is listed and its follow is empty; the answer's `logged` field says whether the log mask lets that protocol's lines through, and the page names the switch to turn on. Debug logging is what makes the TLS handshake, the timeouts and the disconnect appear beside the conversation.
- The connections the server opens itself are listed too, marked `client`: `SMTP client`, `POP3 client`, `IMAP client`, and the short-lived `SpamAssassin` and `Diagnostics` connections. Not listed: anything that is not a `TCPConnection`, so the REST API, the webmail, the DAV services, ManageSieve and the anti-virus scanner's socket to `clamd` are not sessions here. A delivery to a host that refuses the connection is not listed either, though with debug logging on its lines can still be followed by id, with `known: false`.
- A follow is a long poll, not a stream: `GET /api/v1/logs/sessions/{id}?after=0&wait=10` answers the session's lines after sequence 0 and a `cursor` to ask after next time, waiting up to `wait` seconds for the next line. The wait is at most twenty seconds, and one waiting follow at a time is allowed, because a waiting follow holds one of the HTTP listener's four worker threads, which also serve the webmail; a second follow is answered at once with `waited: false`. An answer carries at most 500 lines, and the pane keeps the last 5,000.
- `GET /api/v1/logs/sessions` lists the sessions (the newest thousand when more are open, with `truncated: true`), with `held_lines` and `logging_enabled`. Both routes are under the `view_logs` permission, which the built-in read-only auditor role holds, and both are server-wide: a credential confined to some domains is refused them, as it is refused the log files.
- There is no Control Panel page and no `hmctl` verb for this; the Control Panel's Live logs is the older, whole-log form over COM. There is no filtering inside a session.

### A copy to syslog, or to the journal, since 6.3.4

Since 6.3.4 every entry the server logs can also be sent as RFC 5424 to a syslog collector or, on Linux, where the server runs as the packaged systemd unit with no collector host named, to the journal. It is a copy: the files above are written exactly as before, and nothing it does can stop a line reaching them, or delay or lose a message. Its settings are `hm_settings` rows, not `[Settings]` keys, read when the sink starts (the next service start or reinitialise), not when they are saved. With the journal on, `journalctl -u hmailserver HMAILSERVER_SESSION=41` follows one session from the journal, across a restart. [Alerts, the audit trail, syslog and the journal](/topic/28273) has the twelve settings, the transports, the rsyslog configurations and the journal's fields.

### Codes in the error log worth knowing since 6.3.4

| Code | What it says, and what to do |
|---|---|
| HM5804 | A value typed into `hMailServer.INI` `[Settings]` for a key the database holds was put back at the last start. Change it in the Control Panel, with `PUT /api/v1/settings/ini/{name}`, `hmctl ini set` or COM `Settings.SetIniSetting` |
| HM7460 | `AutoBanFirewall` is on and the Windows service account cannot write firewall rules (below) |
| HM7495 | The database records a stored-secrets key this server does not hold, and every account's second factor refuses every code: import the exported key with `hMailServer.exe --import-stored-secrets-key FILE` (elevated) or `sudo hmailserver --import-stored-secrets-key FILE`, then restart |
| HM7540 | A restore was refused because the archive holds an entry with an absolute path, a `..` segment or a link with somewhere to point; nothing was extracted. Find out where the archive came from |

### fail2ban on Linux, and the Windows contrast

An auto-ban (repeated failed logons from one address) is enforced by the server's own listeners: the kernel accepts the connection and the server refuses it. Shipped in 6.3.2, the Linux package offers two ways to push the ban below the server, and one is enough.

**fail2ban** needs no capability at all, because it runs as root and reads the application log. The package installs `/etc/fail2ban/filter.d/hmailserver.conf` and a jail at `/etc/fail2ban/jail.d/hmailserver.conf`, disabled as Debian asks of a shipped jail. The filter matches the two fixed-shape lines the server writes for every counted failure and every ban, whether or not protocol logging is on; auto-ban on logon failure (`AutoBanOnLogonFailureEnabled`, on as shipped; `auto_ban_on_logon_failure` over REST) is what writes them. Turn it on:

```
printf '[hmailserver]\nenabled = true\n' > /etc/fail2ban/jail.d/hmailserver.local
systemctl reload fail2ban
fail2ban-client status hmailserver
```

**The server itself**, with `AutoBanFirewall=1`, makes the firewall match its auto-ban ranges at start, on every ban, on every range deletion and once a minute. On Linux it runs `/usr/lib/hmailserver/autoban-hook`, which keeps an nftables set with a timeout (iptables where there is no nft) and drops TCP from the banned address to the ports the server listens on. Changing the firewall needs `CAP_NET_ADMIN`, which the unit deliberately does not grant a mail server by default. Grant it once with a drop-in, and the hook's `check` verb says whether it can act:

```
systemctl edit hmailserver
   [Service]
   AmbientCapabilities=CAP_NET_BIND_SERVICE CAP_NET_ADMIN
   CapabilityBoundingSet=CAP_NET_BIND_SERVICE CAP_NET_ADMIN
systemctl restart hmailserver
sudo -u hmailserver /usr/lib/hmailserver/autoban-hook check
```

`AutoBanCommand`, on either platform, runs a program of yours instead of, or as well as, `AutoBanFirewall` (`<command> ban <address> <minutes> <ports>` and `<command> unban <address>`) for a cloud firewall or a router. Before turning either on, list the addresses you administer from in `AutoBanNeverBan`: a ban that reaches the firewall locks that address out of every mail port on the machine. Since 6.3.4 `AutoBanFirewall`, `AutoBanCommand` and `AutoBanNeverBan` are stored settings like the other `[Settings]` keys: set them over the API or with `hmctl ini set`, not in the file.

The fail2ban filter and the hook are part of the Linux packages. On Windows, `AutoBanFirewall` writes one inbound block rule per banned address in Windows Defender Firewall, named `hMailServer auto-ban <address>` and scoped to the ports the server listens on, and since 6.3.4 a new installation's service account, `NT SERVICE\hMailServer`, cannot write them: HM7460 is logged once a start and no rules appear, while the listeners still refuse a banned address. Turn `AutoBanFirewall` off there, or use `AutoBanCommand` towards a router or cloud firewall; HM7461 names rules a LocalSystem run left behind, with the PowerShell that removes them.

### What to include in a report

[SUPPORT.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/SUPPORT.md) asks for, in order of value: the relevant log, not a description of it, with debug logging on and the window around the problem; the `ERROR_hmailserver_<date>.log` for the same window; the exact version, the platform and the database backend; and what you expected instead, and whether it is reproducible. Remove passwords, keys and real email addresses first, because a posted log is public. [Read this before you post](/topic/28185) asks for the same log lines and adds the build number, the package, whether the server was upgraded and the Diagnostics output; **Copy** on the Diagnostics page puts the whole result on the clipboard. Without a GitLab account, email the report to the project's Service Desk address, contact-project+hmailserver-hmailserver-86730559-issue-@incoming.gitlab.com, which GitLab files as a confidential issue; the maintainers answer by email.

### Where to read more

- Wiki: [Monitoring and Health](https://gitlab.com/hmailserver/hmailserver/-/wikis/Monitoring-and-Health), sections 16.2 and 16.2a; [Troubleshooting](https://gitlab.com/hmailserver/hmailserver/-/wikis/Troubleshooting), sections 19.10 and 19.11; [Installing on Linux](https://gitlab.com/hmailserver/hmailserver/-/wikis/Installing-on-Linux), for the journal and fail2ban.
- Repository: [FollowConnection.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/FollowConnection.md), [Syslog.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Syslog.md), [SUPPORT.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/SUPPORT.md).
- The public documentation: [Logs on disk](https://www.progressiverobot.com/hmailserver-documentation/#162-logs-on-disk), [Logging and log rotation on Linux](https://www.progressiverobot.com/hmailserver-documentation/#3511-logging-and-log-rotation), [Follow one message end to end in the logs](https://www.progressiverobot.com/hmailserver-documentation/#follow-one-message-end-to-end-in-the-logs), [Codes in the error log](https://www.progressiverobot.com/hmailserver-documentation/#328-codes-in-the-error-log).
- On this forum: [hMailServer on Linux: what you get and what you do not](/topic/28189), [The Control Deck](/topic/28192), [Diagnosing stalled mail](/topic/28219), and [IP ranges, auto-ban and lockout](/topic/28239) for when a ban is made and how to lift one.