---
title: "Hardening a public mail server: secure defaults, checklist, sealed secrets and roles"
url: https://www.hmailserver.co.uk/topic/28241/hardening-a-public-mail-server-secure-defaults-checklist-sealed-secrets-and-roles
author: "Progressiverobot"
published: 2026-09-27T19:23:13.731Z
updated: 2026-09-27T19:23:13.731Z
category: "Guides & tutorials"
tags: ["security","tls","certificates","administration","dpapi"]
site: hMailServer Community Forum
---

# Hardening a public mail server: secure defaults, checklist, sealed secrets and roles

*Written for 6.3.4. hMailServer 6.3.7 (17 October 2026) is a security release that bears on this guide. On Windows, `hMailServer.exe /ProtectFolders`, which setup runs at every installation and upgrade, now hardens the program folder, `Bin` and `ControlPanel`: where hMailServer was installed in a folder that took its permissions from the root of a drive, any account that could sign in to the computer could replace the programs the service runs (CVE-2026-107571); the default folder under Program Files was never affected, and an installation at a drive's root is hardened by the upgrade itself. The `serveradmin` role given with a list of domains is confined to those domains over COM as over REST, a credential confined to named domains is refused the changes that reached beyond its domains over REST and COM, and a state-changing REST request that a browser reports as cross-site is refused. SMTP, POP3 and ManageSieve refuse a SASL PLAIN sign-in that asks to act as another account, and the stored-secrets key is no longer left in memory freed without being written over. An upgraded Windows server keeps the .NET 10 Desktop Runtime it had, 10.0.12 where 6.3.4 to 6.3.6 installed it, against which Microsoft published advisories on 13 October 2026: bring it to 10.0.13 through Microsoft Update or Microsoft's installer. See [the 6.3.7 announcement](/topic/28277). hMailServer 6.3.8 (2 November 2026) is a security release too: upgrade every server running 6.3.7 or earlier. On Windows setup now makes or hardens the program folder, `Bin` and `ControlPanel` before it copies a file or runs a program of the installation (the four narrower cases of work item 78 that 6.3.7's notes left open), and a program folder, `Bin` or `ControlPanel`, a folder or a link on the way to them, or a program, library, script or settings file in them, that another account than SYSTEM, the Administrators and TrustedInstaller controls ends the installation with exit code 7, naming the `icacls` that deals with it and, where it says so, a restart of Windows before setup runs again. The auto-ban saves the IP range it adds under a name of its own, so that it bans the address; the web services listener (CardDAV, CalDAV, Autodiscover) refuses an address an auto-ban or an IP range bans; and a sign-in's name is matched to its account as the per-name lockout and the compromised-account hold count it, so a mail client that signs in with another spelling of the account's address than the stored one, in more than the case of the letters A to Z, or with white space at either end, may no longer sign in: correct the name in the client. See [the 6.3.8 announcement](/topic/28278).*

This guide covers what stands between a mail server on the public internet and the people who guess passwords at it. All of it is new in 6.3.4 except auto-ban, account lockout, the administrator's second factor and the IP range option *Require TLS for auth*, which are older; 6.3.4 sets that option on a new installation and judges it in the checklist. The one thing to know first: **the secure defaults are applied only to a database the installation itself created, and an upgraded server is never changed.** After an upgrade your listeners, IP ranges and service account are as they were and nothing is sealed until you make a key; the checklist says what is left.

### What a new installation starts with, and what an upgrade keeps

In 6.3.3 and earlier a new installation listened on ports 25, 587, 110 and 143 with no TLS, and neither shipped IP range required TLS for authentication. The create scripts still make that database; the Windows installer, and the Linux packages' first start, now change it before the server first listens.

| | A new installation since 6.3.4 | An upgraded server |
|---|---|---|
| Ports 25 and 587 (SMTP), 110 (POP3), 143 (IMAP) | STARTTLS offered, optional | as it was |
| IP range *Internet* (every IPv4 address) | **TLS required for authentication** | as it was; the checklist flags it |
| IP range *My computer* (127.0.0.1) | unchanged | as it was |
| Certificates | *Self-signed (made at installation)* | as they were |
| Windows service account | `NT SERVICE\hMailServer` | the account it had (LocalSystem, if it began before 6.3.4) |

**What a client sees.** From another machine, `AUTH` on SMTP is not offered before STARTTLS and is refused with `530 A SSL/TLS-connection is required for authentication.` if it is sent; IMAP `LOGIN` and `AUTHENTICATE` answer `BAD A SSL/TLS-connection is required for authentication.`; POP3 `USER` and `AUTH` answer `-ERR`. After STARTTLS each works as before. From the server itself, on 127.0.0.1, nothing has changed, and mail from other servers on port 25 is taken with or without TLS.

**Why STARTTLS is optional.** A port's requirement applies to every address: on port 25 it would refuse every server that cannot start TLS, on the others this computer's own clients. The range's rule protects the password.

**The certificate.** `self-signed-certificate.pem` and its key `self-signed-key.pem` are in `<data directory>/Certificates`: `C:\Program Files\hMailServer\Data\Certificates` on a default Windows installation, `/var/lib/hmailserver/Certificates` from the Linux packages, where the key is mode 0600 and owned by the service's user. It is RSA 2048, runs 825 days, names the host and gives the organisation *hMailServer, self-signed at installation*. It encrypts, but nothing vouches for it, so mail clients warn. Replace it with ACME (set `AcmeEnabled`, `AcmeContactEmail` and `AcmeDomains`; the first certificate goes to every listener that holds the self-signed one) or with your own, set on each port. See [Let's Encrypt certificates are automatic](/topic/28187).

### Where the defaults are applied

The create step marks its database with the `hm_settings` row `NewInstallDefaultsPending`; the verb `--apply-new-install-defaults` requires the mark and removes it once it has decided.

| Platform | When it runs | What to know |
|---|---|---|
| Windows | The installer, after the database step of a new installation, with the service stopped | A database DBSetup attached, rather than created, carries no mark: exit 5, logged, nothing changed |
| Linux (`.deb`, `.rpm`) | The unit's first start after `sudo -u hmailserver hmailserver --create-database`, which marks the database and leaves a one-line file, `.new-installation`, in the `DataFolder` | The unit runs `ExecStartPre=-/usr/bin/hmailserver --apply-new-install-defaults --if-marked`: nothing unless that file is there, then one try at the database, given room by `TimeoutStartSec=300` (the leading `-` forgives a failure, not a timeout). It ignores a mark made more than a day from its clock; a run by hand has no such limit |
| A container (since 6.3.8) | The image's entrypoint, before whatever command the container runs; the Compose file and the Helm chart run the same step before the server | `hmailserver --apply-new-install-defaults --if-marked` secures a database the entrypoint's `--create-database` made a moment before and opens nothing at any other start; a failure (run as root, a database not reached) is said and forgiven, and the server starts all the same |
| By hand | A container from an image before 6.3.8, a server started by hand rather than by the unit, or defaults left for later | With the service stopped; on Linux as the service's user, because the verb refuses to run as root |

By hand, on Windows and then on Linux:

```
Stop-Service hMailServer
& "C:\Program Files\hMailServer\Bin\hMailServer.exe" --apply-new-install-defaults
Start-Service hMailServer
```

```
sudo systemctl stop hmailserver
sudo -u hmailserver hmailserver --apply-new-install-defaults
sudo systemctl start hmailserver
```

Exit codes: 0 applied (with `--if-marked`, also nothing to do); 1 something could not be made or written (`HM7510`, `HM7511` in the error log); 2 no database named, or not readable enough to judge, or on Linux run as root; 4 the Windows service is running; 5 not a database this installation created, or not as the create script left it: nothing changed.

**On Linux, mind the data directory.** A `DataFolder` outside the unit's `ReadWritePaths` (`/var/lib/hmailserver` and `/var/log/hmailserver`) is read-only to the service, so the first start can neither write the certificate nor store mail there. Add the directory with a drop-in (`sudo systemctl edit hmailserver`, then `ReadWritePaths=/srv/mail` under `[Service]`) before that first start.

**Not yet proved on a clean machine.** [SecureDefaults.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SecureDefaults.md)'s checks that the installer applies the defaults are not yet written, and those for the `.deb` ran only on GitHub Actions, so nothing runs them today.

### The setup checklist

The checklist judges every server, new or upgraded, in twelve items.

| Key | Done when |
|---|---|
| `profile` | a setting profile has been applied; asked for on a server with fewer than ten accounts |
| `domain` | an active domain exists, and none has a name the server reserves for its own directories |
| `account` | at least one account exists |
| `submission` | every SMTP listener other than port 25 has STARTTLS or implicit TLS, and a certificate other than the self-signed one |
| `sign_in_tls` | every IP range that lets a client connect from another address requires TLS for authentication, or reaches only listeners that start TLS before a password |
| `spf`, `dkim`, `dmarc` | every active domain has a TXT record beginning `v=spf1`, signs with a key whose record is published at `<selector>._domainkey.<domain>`, and has a TXT record beginning `v=DMARC1` at `_dmarc.<domain>` |
| `backup` | the destination is set and the folder exists |
| `second_factor` | the administrator password is enrolled with a one-time code |
| `stored_secrets_key` | this server holds the key the database records, and an export of it has been recorded |
| `service_account` | it is not LocalSystem on Windows, not root on Linux |

Each item is `done`, `action_needed`, `cannot_tell` or `not_needed`. Port 25 is never the submission port, and a plain submission port beside a secured one still needs action.

**Where it is.** `GET /api/v1/setup/checklist` (reading needs `view_configuration`; `?dns=0` skips the lookups), the Deck's *Checklist* view and, on Windows, COM's `Utilities.GetSetupChecklist(LookUpDns)` and the Control Panel's *External setup* page and dashboard.

### Passwords only over TLS, on a server you upgraded

"Require TLS for auth" is an option of an IP range, not of a port, and covers SMTP, POP3, IMAP and ManageSieve alike.

1. Read the `sign_in_tls` item: it names each range, and the listeners where a password from there would cross the network in the clear.
2. If the finding is `sign_in_no_tls`, a listener has no TLS to start, and requiring TLS would refuse every client on it: give those listeners a certificate and STARTTLS first. This finding carries no fix.
3. **Set every mail client that signs in from another machine to STARTTLS first.** Once the range requires TLS, a client that signs in without TLS is refused; that is why an upgrade is never changed by itself.
4. When the finding is `sign_in_plain`, apply its fix: the Deck's button *Require TLS for sign-in from this range*, or `PUT /api/v1/ipranges/{id}` with `{"require_tls_for_auth": true}`, which needs `manage_settings`. On Windows the Control Panel opens its IP ranges page, where the setting is *Require TLS for auth*.
5. Replace the self-signed certificate if you have one, so that `submission_self_signed` clears.

### Secrets at rest and the stored-secrets key

Until 6.3.4 the administrator password (unsalted MD5, from the Windows installer), each account's second factor and the `[Settings]` secrets were kept as written.

**The administrator password** is now written as PBKDF2. An older MD5 or SHA-256 value is rewritten as PBKDF2 at the first successful administrator sign-in where the service can write the file. Where it cannot (`NT SERVICE\hMailServer`, or the Linux package's root-owned file) the old value stays, `HM7494` says so once, and the verb rewrites it: on Windows `hMailServer.exe --set-admin-password` from an elevated prompt, with the password in `HM_ADMIN_PASSWORD`, or on one line of standard input from a pipe or a file for a password in plain ASCII (Windows PowerShell 5.1 turns every other character into `?`), never as an argument; on Linux `sudo hmailserver --set-admin-password`, which reads standard input.

**Second factors and ten `[Settings]` secrets** (`SRSSecret`, `BATVSecret`, `OAuth2HmacSecret` and the rest) are sealed with AES-256-GCM under the installation's stored-secrets key, in the form `HMK1:<key id>:<base64>`, **once the installation has a key**. The key is kept in each server's `hMailServer.ini` as `[Security] StoredSecretsKey`, and never in the database or a backup archive.

**Who gets a key without asking.** The Windows installer makes one on a new installation. On Linux, `hmailserver --create-database` makes one when it creates the schema and can write the configuration: in a container, whose `/etc/hmailserver` the service owns, it does (no image of 6.3.4 is published); run by the package as the service's user, it cannot. An upgraded installation, and a package installation on Linux, seal nothing until an administrator makes the key:

1. Make the key on one node. Windows, from an elevated prompt beside `hMailServer.exe`: `.\hMailServer.exe --create-stored-secrets-key`. Linux: `sudo hmailserver --create-stored-secrets-key`.
2. Export it at once: `.\hMailServer.exe --export-stored-secrets-key D:\Escrow\hmailserver-stored-secrets.key`, or `sudo hmailserver --export-stored-secrets-key FILE`. The export writes a new file, never over one that exists, readable only by SYSTEM and the administrators on Windows and mode 0600 on Linux, and prints the key id, never the key.
3. Restart the service on that node (`Restart-Service hMailServer`, or `sudo systemctl restart hmailserver`). Its next start seals what is stored as written; until then nothing is sealed.
4. On every other node that shares the database, `--import-stored-secrets-key FILE`, then restart each. An import is refused while the node holds another key that opens: a standby or new node whose installer made a key of its own removes its `[Security] StoredSecretsKey` line first, and until then the shared database refuses it (`HM7495`).
5. Keep the exported file with the database backups, stored apart from them.

**Why the export matters.** The product's own backup archive carries these secrets opened, and a restore seals them under the restoring server's key, so set a backup passphrase. A database-level backup (a dump, a copy of the `.sdf`) restored without the key brings back every second factor and `[Settings]` secret unopenable. A copy of `hMailServer.ini` is no copy of the key on Windows, since its line is sealed under that machine's DPAPI; never copy that line from one machine to another.

**When a node lacks the key** it says `HM7495` at every start and makes no key of its own; until the key is imported, a `[Settings]` secret sealed under it runs as though unset (`HM7492`) and a second factor refuses every code (`HM7493`). A key lost for good is recovered from with `--forget-sealed-secrets <key id>`, with the service stopped (exit 4 otherwise), which clears only what that key sealed: every second factor is enrolled again, every `[Settings]` secret entered again.

**`ProtectStoredSecretsWithDPAPI=0` (default 1) has a cost.** The next start opens what was sealed and stores it as written, readable by anyone who can read the file or the database, and the machine-bound secrets are written with the legacy Blowfish scheme, whose key is in the source code: the way back, not a setting for a new server.

**Two things to do after an upgrade.** Since 6.3.4 `PasswordPepper` stays in `hMailServer.ini` only and is left out of every backup, so keep a copy of that value with the backups. And a password hash stored before 6.3.4 from a password with a character above U+00FF is refused, to its own password too: an administrator sets such passwords again, and the application log names each account as its user tries to sign in.

### Checking new passwords against published breaches

Off by default, because it sends something off the server. When on, a new password found in the published corpus of breached passwords is refused.

| Setting (REST key) | Default | What it means |
|---|---|---|
| `breach_check_enabled` | off | The whole feature |
| `breach_check_url` | `https://api.pwnedpasswords.com/range/` | Where the five characters are sent |
| `breach_check_minimum_count` | `1` | Refuse a password seen this many times or more |
| `breach_check_unreachable_action` | `allow` | `allow` lets a change through unchecked; `refuse` refuses it |

**What leaves the server** is the first five characters of the password's SHA-1 and nothing else. The default endpoint is a third-party service, Have I Been Pwned's Pwned Passwords, and an organisation that cannot send it those five characters, its address and the time runs its own mirror.

**When it runs.** When a password is chosen, by the person or by an administrator, and never at logon. An endpoint that gives no complete answer within eight seconds counts as unusable, and the error log records `HM6600` (let through) or `HM6601` (refused). Under `refuse`, an endpoint that is down refuses every password change on the server. Bulk imports make one request per account, so run them against a mirror or with the check off.

**Where it is set.** The four settings are `hm_settings` rows, not `hMailServer.ini` lines, and a change applies from the next password chosen. They are on the Control Panel's *Administrative access → Password* page (the *Passwords found in published breaches* card), in the Deck's Settings view, over REST and in `hmctl settings set`:

```
hmctl settings set --set breach_check_enabled=true
```

### Compromised-account detection

This judges every authenticated submission (SMTP, the webmail's send, a scheduled draft) against the account's own history of the last 30 days, without reading the message. **It starts in `watch` on every installation, new or upgraded**: the baseline is built, the `account.compromised` alert says what enforcing would have done, and nothing else happens.

Four signals score: **volume** (2 points), the last hour's messages reaching `compromise_volume_floor` (30) and exceeding `compromise_volume_factor` (4) times the busiest baseline hour; **recipients** (2), the last hour's distinct recipients reaching `compromise_recipient_floor` (20), at least 80 per cent of them new to the account; **hour** (1), an hour of the day (UTC) the account has not submitted at in the baseline; **address** (1), a client network (the /24 of an IPv4 address, the /64 of an IPv6 one) not seen for this account. `compromise_score_to_lock` (3) is a takeover. In `enforce`, the message is held, the account is locked until an administrator unlocks it, and the alert and an `account.locked` webhook are raised. The client is still answered `250 Queued`. Held mail is reviewed on the Deck's **Held for review** page and under `/api/v1/review`; a held message nobody has looked at in `compromise_held_days` days (7) is discarded. Watch it before enforcing: the false positive is a person locked out of their own mail. There is no Control Panel page for it.

### Roles instead of the administrator password

Rather than share the administrator password, give each administrator a role in the domains they look after, on the Deck's *Roles* view, over REST, or on Windows in the Control Panel or over COM. Nothing changes on the day you upgrade: an account with no assignment holds the authority its admin level has always given it, and a narrow role cannot take over a wider one, so a help desk is refused a domain administrator's password reset. [Administrator roles end to end](/topic/28237) has the permissions, how to give a role, and the two stores only the administrator password opens.

### The rest of the list

- **Auto-ban** is on by default: 3 failures within 30 minutes, a 60-minute ban. `MaxInvalidLogonAttempts` at `0` disables it, and `AutoBanMinutes` at `0` disconnects the client but bans nothing. **Account lockout** (`AccountLockoutThreshold`) is off until set, and counts by the name being guessed at. [IP ranges, auto-ban, lockout and the rest](/topic/28239) has both in full.
- **The server-enforced administrator second factor** refuses the password alone everywhere, the REST API included.
- **On Windows**, move a service still on LocalSystem with the installer, the Control Panel's button, or from an elevated Command Prompt in `Bin`: `hMailServer.exe --confirm /GrantServiceAccess`, then, only if that ended in 0, `hMailServer.exe --service-account virtual /Register`, then `net stop hMailServer && net start hMailServer`. Check first what the virtual account cannot do, listed in [Installing on Windows](/topic/28227).
- **The event-script flaw fixed in 6.3.4** (CVE-2026-100741; its record was not yet public when this was written) is in every release from 6.0.0 to 6.3.3: upgrade, and until you can, switch event scripting off. See [Event scripts and the 6.3.4 security fix](/topic/28261) and [the 6.3.4 security announcement](/topic/28218).

### What it does not do

- Passkeys, single sign-on and the session policies do not reach IMAP, POP3 or SMTP, which still take the password or an app password.
- The breach check does not cache, and does not expire a password that enters the corpus later.
- Compromised-account detection does not drop a session that is already up.
- The stored-secrets key protects against a copy of the database that leaves without the file, not against a local account on the server.

### Where to read more

- Wiki: [Security Hardening](https://gitlab.com/hmailserver/hmailserver/-/wikis/Security-Hardening).
- Documents: [SecureDefaults.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SecureDefaults.md), [SetupChecklist.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SetupChecklist.md), [SecretsAtRest.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SecretsAtRest.md), [PasswordBreachCheck.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/PasswordBreachCheck.md), [CompromisedAccounts.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/CompromisedAccounts.md) and [Roles.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Roles.md).
- The public documentation: [chapter 17, Security hardening](https://www.progressiverobot.com/hmailserver-documentation/#17-security-hardening).
- On this forum: [Upgrading to 6.3.4](/topic/28193), [The Control Deck](/topic/28192) and [hMailServer on Linux](/topic/28189).

A suspected vulnerability is reported privately, never in a forum post: by a confidential GitLab issue, or by email to the project's Service Desk address, contact-project+hmailserver-hmailserver-86730559-issue-@incoming.gitlab.com, which GitLab files as a confidential issue ([SECURITY.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/SECURITY.md)).