Reading the logs: levels, files, the Logs page, following a connection, and fail2ban

Progressiverobot

Written for 6.3.4. hMailServer 6.3.7 (17 October 2026) fixes the SMTP log writing some credentials in the clear with SMTP logging on: an AUTH PLAIN sent before EHLO or not in its one exact form, the bearer token of AUTH XOAUTH2 and AUTH OAUTHBEARER, and the lines of SCRAM-SHA-256, OAUTHBEARER and GSSAPI went into the log as the client sent them, base64 and readable, in every version from 6.0.0 to 6.3.6. An SMTP log written by 6.3.6 or earlier may hold them: redact every line of a SASL exchange before posting one. A database write that failed is now reported (HM5032) where it was taken as done, so an error log that was quiet may name failures that were hidden; and the IMAP and POP3 listeners can take the client's address from a PROXY protocol header, as the SMTP listener does, both off by default. See the 6.3.7 announcement. hMailServer 6.3.8 (2 November 2026) no longer reports a stop of the servers as errors of their own: work that outlasted a stop, a reinitialise among them, writes a line to the debug log where it wrote HM5136 or another code to the error log and ran an installed OnError script, so the error log of a server that reinitialises may be quieter; and an event script's call that meets a stop fails with "The server is stopping. The request was not completed.", its script error going to the debug log. A failed account sign-in at the REST listener from loopback, as through a reverse proxy on the same machine, is counted against the name alone and writes no Auto-ban: logon failure line for fail2ban to read. See the 6.3.8 announcement.

This topic covers what hMailServer writes to disk and which file to open, the settings that decide how much is written and kept, the live views, following one connection (since 6.3.4), a copy to syslog or the journal, and the fail2ban filter the Linux package installs. The one thing to know first: open ERROR_hmailserver_<date>.log before anything else. It holds only the entries the server itself classed as an error, and its mere existence is a diagnostic, because the Control Panel's built-in self-test (Monitoring & troubleshooting → Diagnostics, so Windows) fails when any ERROR* file is present in the log directory.

Where the logs are

Platform	Log directory	Set by
Windows	`C:\Program Files\hMailServer\Logs` by default	`[Directories] LogFolder` in `hMailServer.INI`
Linux	`/var/log/hmailserver`, created by the package as `0750 hmailserver:hmailserver` and left in place when the package is removed	`[Directories]` in `/etc/hmailserver/hMailServer.ini`

The format is the same on both platforms. On Linux the journal (journalctl -u hmailserver) carries the startup line, anything written to standard error and the operational events; a memory-safety fault is recorded in /var/log/hmailserver/crash-oracle.log as one line (signal, address, thread, event number) before the process dies by the signal it caught. [Directories] is per machine and is never mirrored into the database, so do not copy another server's file over this one's.

The files

File	Holds	Rotates
`hmailserver_<date>.log`	Everything: application lines, and the protocol conversations unless they are split out	Daily, by name
`ERROR_hmailserver_<date>.log`	Only entries the server classed as an error	Daily, by name
`hmailserver_SMTP_<date>.log`, `hmailserver_IMAP_<date>.log`, `hmailserver_POP3_<date>.log`	The per-protocol conversations, when `SepSvcLogs=1`	Daily, by name
`hmailserver_backup.log`	Every backup and restore run, line by line	Never: one growing file. `LogDeleteDays` matches it by name but deletes only a file nothing has written to for that many days
`hmailserver_events.log`	Event-script output	Never, as above
`hmailserver_awstats.log`	The AWStats-format feed, when AWStats logging is on	Never, as above
`hMailServer_messagestore_consistency.report`	The most recent message-store consistency scan: one line per message whose file is missing, as `messageid<TAB>account<TAB>expected-path`	Overwritten each run

The backup and event logs are among the files the server marks with a byte-order mark; they are UTF-16LE on both platforms, so a file written on one reads on the other.

A log line looks like "IMAPD" 5220 41 "2026-09-17 09:00:00.120" "192.0.2.7" "SENT: A1 OK LOGIN completed", where IMAPD is the category and 41 is the session id that every line of that conversation carries. The conversation categories are SMTPD, SMTPC, POP3D, POP3C and IMAPD; APPLICATION, TCPIP, DEBUG and ERROR are the rest.

The settings

Setting	Default	What it does	What it looks like when it is wrong
`LogLevel`	`9`	Not the choice of log types, which is the Logging page's tick-boxes (or `PUT /api/v1/settings/logging`). The Settings Reference records two effects: long lines are cut only when it is 2 or lower with debug off, and IMAP `SENT:` lines for FETCH and STATUS are logged only when it is above 2 or debug is on	—
Debug messages (the Logging page)	off	Adds the `LOG_DEBUG` lines, including the accept-pipeline stage timings	Left on, a busy server fills the disk in days, and a full disk stops mail
`SepSvcLogs`	`0`	Splits SMTP, IMAP and POP3 into their own files	Off, so a search for one protocol wades through all three
`LogDeleteDays`	`0`	Deletes files named `hmailserver_.log` or `ERROR_hmailserver_.log` whose last write is older than this many days, and nothing else in the directory. `0` keeps everything for ever	Left at `0` on a server with debug logging on: the usual cause of a full log volume
`JsonLogging`	`0`	Writes JSON lines, for Elasticsearch, Loki or Splunk	Overridden while `Logging.LogFormat` is NCSA: the pipeline gets NCSA lines and no error
`SlowQueryLogMilliseconds`	`0` (off)	Logs, and counts in `hmailserver_db_slow_queries_total`, any database statement slower than this	Off, so a database that has become slow leaves no trace but the latency histogram
`WindowsEventLogLevel`	`2`	How much also goes to the Windows event log	—

Where they are set. On Windows, Monitoring & troubleshooting → Logging in the Control Panel. On either platform the table's [Settings] keys (all but Debug messages) are set with PUT /api/v1/settings/ini/{name} or hmctl ini set. On Linux there is no Control Panel, so the log categories, debug messages among them, are set with PUT /api/v1/settings/logging, hmctl settings set logging or the Deck's Settings page. There, WindowsEventLogEnabled=1 is a Windows inheritance in name only, because on Linux it means syslog, and the operational events reach the journal through it. Since 6.3.4 these keys are stored in the database, whose value is the one used: where the service can write hMailServer.INI, a line edited there is put back and reported as HM5804. How the settings store works has the rest, [SettingsOverride] (HM5807) among it.

Rotation. On Windows, LogDeleteDays is the rotation. On Linux the package installs /etc/logrotate.d/hmailserver; the rule uses copytruncate because the server does not reopen its log on a signal, and the rule says so at length.

Debug logging: on to diagnose, off afterwards. The stage-timing lines that diagnosing stalled mail depends on are debug lines; the slow ones (ten seconds or more) are written at application level as well. Leaving debug on matters because a full disk stops mail: the application log warns below DiskSpaceWarningThresholdMB (1024 MB), and below MinimumFreeDiskSpaceMB (100 MB) on the message store's volume the server answers 452 4.3.1 Insufficient system storage. and senders retry.

The live views

Windows: the Control Panel. Monitoring & troubleshooting → Live logs is a real-time, colour-coded tail of what the server is writing; every page in that group but Discovery reads the running service over COM (Discovery has no COM interface, so it reads the REST listener and needs RestApiPort set). Diagnosing stalled mail, in the same group, has a one-click debug-logging toggle.

Both platforms: the Control Deck and hmctl. The browser Control Deck at / on the REST listener has a Logs page that lists the log files and shows any file's last 200 or 2,000 lines, without a share to the log directory. Since 6.3.4, hmctl log list and hmctl log tail <name> --lines N read the same over GET /api/v1/logs and GET /api/v1/logs/{name}, and work against a Windows or a Linux server because they go over HTTP rather than DCOM. The Linux packages install hmctl as /usr/bin/hmctl; the HmailServer PowerShell module is the same vocabulary for PowerShell.

The files themselves, from a shell. On Windows:

$log = "C:\Program Files\hMailServer\Logs\hmailserver_" + (Get-Date -Format "yyyy-MM-dd") + ".log"
Get-Content $log -Tail 0 -Wait | Select-String "SMTPD|SMTPConnection"

On Linux:

tail -f /var/log/hmailserver/hmailserver_$(date +%F).log
journalctl -u hmailserver -f

Following one connection, since 6.3.4

Every SMTP, POP3 and IMAP conversation is a session with a number, and every line the log writes for it carries that number. Until 6.3.4 that meant searching the day's log for the number afterwards. Since 6.3.4 the Connections section at the top of the Deck's Logs page lists every connection the server has open now, newest first, and the last two hundred that ended: the id, the protocol, the address it came from, the port it came in on, the account once it has signed in, and when it started or ended. Follow beside any of them opens a pane holding every line the log has for that session and keeps adding lines as the server writes them; Copy puts the whole pane on the clipboard; Stop following ends it. A session that ended a second ago is still listed, so an SMTP conversation that lasted a quarter of a second can be picked and read.

How it works, and what that means for you.

The server keeps the last 4,096 log entries in memory, across every session; the ring is always on, and nothing survives a restart. A debug-logged IMAP client can fill it in minutes, so a follow whose cursor points before the oldest entry still held answers dropped: true and starts from what is left; the page says so. A line older than that is in the log files and nowhere else.
The conversation lines (SMTPD, SMTPC, POP3D, POP3C, IMAPD) carry the session id in their own column. DEBUG and TCPIP lines carry none and are filed by the first number after the word session in their text (TCP connection started for session 41, Ending session 41); that is an inference, and the text is not always the server's own. A line that does neither is nobody's and is not shown.
It shows what the log mask lets through, and no more. With IMAP logging off, an IMAP session is listed and its follow is empty; the answer's logged field says whether the log mask lets that protocol's lines through, and the page names the switch to turn on. Debug logging is what makes the TLS handshake, the timeouts and the disconnect appear beside the conversation.
The connections the server opens itself are listed too, marked client: SMTP client, POP3 client, IMAP client, and the short-lived SpamAssassin and Diagnostics connections. Not listed: anything that is not a TCPConnection, so the REST API, the webmail, the DAV services, ManageSieve and the anti-virus scanner's socket to clamd are not sessions here. A delivery to a host that refuses the connection is not listed either, though with debug logging on its lines can still be followed by id, with known: false.
A follow is a long poll, not a stream: GET /api/v1/logs/sessions/{id}?after=0&wait=10 answers the session's lines after sequence 0 and a cursor to ask after next time, waiting up to wait seconds for the next line. The wait is at most twenty seconds, and one waiting follow at a time is allowed, because a waiting follow holds one of the HTTP listener's four worker threads, which also serve the webmail; a second follow is answered at once with waited: false. An answer carries at most 500 lines, and the pane keeps the last 5,000.
GET /api/v1/logs/sessions lists the sessions (the newest thousand when more are open, with truncated: true), with held_lines and logging_enabled. Both routes are under the view_logs permission, which the built-in read-only auditor role holds, and both are server-wide: a credential confined to some domains is refused them, as it is refused the log files.
There is no Control Panel page and no hmctl verb for this; the Control Panel's Live logs is the older, whole-log form over COM. There is no filtering inside a session.

A copy to syslog, or to the journal, since 6.3.4

Since 6.3.4 every entry the server logs can also be sent as RFC 5424 to a syslog collector or, on Linux, where the server runs as the packaged systemd unit with no collector host named, to the journal. It is a copy: the files above are written exactly as before, and nothing it does can stop a line reaching them, or delay or lose a message. Its settings are hm_settings rows, not [Settings] keys, read when the sink starts (the next service start or reinitialise), not when they are saved. With the journal on, journalctl -u hmailserver HMAILSERVER_SESSION=41 follows one session from the journal, across a restart. Alerts, the audit trail, syslog and the journal has the twelve settings, the transports, the rsyslog configurations and the journal's fields.

Codes in the error log worth knowing since 6.3.4

Code	What it says, and what to do
HM5804	A value typed into `hMailServer.INI` `[Settings]` for a key the database holds was put back at the last start. Change it in the Control Panel, with `PUT /api/v1/settings/ini/{name}`, `hmctl ini set` or COM `Settings.SetIniSetting`
HM7460	`AutoBanFirewall` is on and the Windows service account cannot write firewall rules (below)
HM7495	The database records a stored-secrets key this server does not hold, and every account's second factor refuses every code: import the exported key with `hMailServer.exe --import-stored-secrets-key FILE` (elevated) or `sudo hmailserver --import-stored-secrets-key FILE`, then restart
HM7540	A restore was refused because the archive holds an entry with an absolute path, a `..` segment or a link with somewhere to point; nothing was extracted. Find out where the archive came from

fail2ban on Linux, and the Windows contrast

An auto-ban (repeated failed logons from one address) is enforced by the server's own listeners: the kernel accepts the connection and the server refuses it. Shipped in 6.3.2, the Linux package offers two ways to push the ban below the server, and one is enough.

fail2ban needs no capability at all, because it runs as root and reads the application log. The package installs /etc/fail2ban/filter.d/hmailserver.conf and a jail at /etc/fail2ban/jail.d/hmailserver.conf, disabled as Debian asks of a shipped jail. The filter matches the two fixed-shape lines the server writes for every counted failure and every ban, whether or not protocol logging is on; auto-ban on logon failure (AutoBanOnLogonFailureEnabled, on as shipped; auto_ban_on_logon_failure over REST) is what writes them. Turn it on:

printf '[hmailserver]\nenabled = true\n' > /etc/fail2ban/jail.d/hmailserver.local
systemctl reload fail2ban
fail2ban-client status hmailserver

The server itself, with AutoBanFirewall=1, makes the firewall match its auto-ban ranges at start, on every ban, on every range deletion and once a minute. On Linux it runs /usr/lib/hmailserver/autoban-hook, which keeps an nftables set with a timeout (iptables where there is no nft) and drops TCP from the banned address to the ports the server listens on. Changing the firewall needs CAP_NET_ADMIN, which the unit deliberately does not grant a mail server by default. Grant it once with a drop-in, and the hook's check verb says whether it can act:

systemctl edit hmailserver
   [Service]
   AmbientCapabilities=CAP_NET_BIND_SERVICE CAP_NET_ADMIN
   CapabilityBoundingSet=CAP_NET_BIND_SERVICE CAP_NET_ADMIN
systemctl restart hmailserver
sudo -u hmailserver /usr/lib/hmailserver/autoban-hook check

AutoBanCommand, on either platform, runs a program of yours instead of, or as well as, AutoBanFirewall (<command> ban <address> <minutes> <ports> and <command> unban <address>) for a cloud firewall or a router. Before turning either on, list the addresses you administer from in AutoBanNeverBan: a ban that reaches the firewall locks that address out of every mail port on the machine. Since 6.3.4 AutoBanFirewall, AutoBanCommand and AutoBanNeverBan are stored settings like the other [Settings] keys: set them over the API or with hmctl ini set, not in the file.

The fail2ban filter and the hook are part of the Linux packages. On Windows, AutoBanFirewall writes one inbound block rule per banned address in Windows Defender Firewall, named hMailServer auto-ban <address> and scoped to the ports the server listens on, and since 6.3.4 a new installation's service account, NT SERVICE\hMailServer, cannot write them: HM7460 is logged once a start and no rules appear, while the listeners still refuse a banned address. Turn AutoBanFirewall off there, or use AutoBanCommand towards a router or cloud firewall; HM7461 names rules a LocalSystem run left behind, with the PowerShell that removes them.

What to include in a report

SUPPORT.md asks for, in order of value: the relevant log, not a description of it, with debug logging on and the window around the problem; the ERROR_hmailserver_<date>.log for the same window; the exact version, the platform and the database backend; and what you expected instead, and whether it is reproducible. Remove passwords, keys and real email addresses first, because a posted log is public. Read this before you post asks for the same log lines and adds the build number, the package, whether the server was upgraded and the Diagnostics output; Copy on the Diagnostics page puts the whole result on the clipboard. Without a GitLab account, email the report to the project's Service Desk address, contact-project+hmailserver-hmailserver-86730559-issue-@incoming.gitlab.com, which GitLab files as a confidential issue; the maintainers answer by email.

Where to read more

Wiki: Monitoring and Health, sections 16.2 and 16.2a; Troubleshooting, sections 19.10 and 19.11; Installing on Linux, for the journal and fail2ban.
Repository: FollowConnection.md, Syslog.md, SUPPORT.md.
The public documentation: Logs on disk, Logging and log rotation on Linux, Follow one message end to end in the logs, Codes in the error log.
On this forum: hMailServer on Linux: what you get and what you do not, The Control Deck, Diagnosing stalled mail, and IP ranges, auto-ban and lockout for when a ban is made and how to lift one.