# hMailServer Community Forum: full text > The community support forum for hMailServer, the free, open-source (AGPL-3.0) mail server for Windows and Linux. hMailServer 6 is maintained by Progressive Robot Ltd. The current release is hMailServer 6.3.7, released 17 October 2026. Source: https://www.hmailserver.co.uk. Generated 2026-10-25T11:38:19.271Z. Each section gives its canonical URL; cite that rather than this file. --- Canonical: https://www.hmailserver.co.uk/topic/28217/start-here-hmailserver-6-is-maintained-and-this-is-its-community-forum # Start here: hMailServer 6 is maintained, and this is its community forum hMailServer is a free, open-source (AGPL-3.0) mail server for Windows and Linux, and this is its community support forum, run by Progressive Robot Ltd, which maintains the hMailServer 6 line. The current release is hMailServer 6.3.7, released on 17 October 2026. 6.3.6 was the last release planned by date: the releases since are security releases, each cut from `master`, and the 6.3 line receives security fixes until 30 September 2027. 6.3.7 is a security release. Five of its fixes were public in the repository before it, from 6 and 7 October 2026 (on `master` from 7 and 9 October), until it was released on 17 October 2026; that was an exception to the project's rule of about a day between a fix becoming public and its release, which holds again after 6.3.7 (SECURITY.md, "What gets a release"). Upgrade to 6.3.7. ## What changed in hMailServer 6 hMailServer was created by Martin Knafve. The original project went quiet for years, and in 2026 Progressive Robot Ltd forked it and released hMailServer 6.0.0 on 12 June 2026. The original project has since resumed development separately. - **Platforms:** Windows 10 version 1607 or later, Windows 11 and Windows Server 2016 or newer, on x64. hMailServer 6.3.6 is the first release with an ARM64 installer, for Windows on ARM, and it is a preview: it has never run on ARM64 hardware, and a new ARM64 installation is set up against SQL Server or PostgreSQL, since SQL Server Compact has no ARM64 build. Linux on x86-64 and AArch64 since hMailServer 6.3.0, released on 10 September 2026. The Linux packages of hMailServer 6.3.7 are for both: a .deb for each of Debian 12 and 13 and Ubuntu 24.04 and an .rpm for each of Fedora 43 and 44, Enterprise Linux 9 and 10 and openSUSE Leap 16.0 and Tumbleweed, each built for that release, beside the generic .deb (Ubuntu 26.04's) and .rpm that an installed server's update check asks for, and an AppImage for each architecture. The AArch64 files are cross-compiled and tested under emulation: no AArch64 machine has run them. The container image, `registry.gitlab.com/hmailserver/hmailserver:6.3.7` (also tagged `6.3` and `latest`), is for linux/amd64 and linux/arm64. - **Upgrades:** the hMailServer 6.3.7 x64 Windows installer upgrades 5.3, 5.4, 5.5, 5.6, 6.0, 6.1, 6.2 and 6.3 in place, keeping mail, accounts, domains and settings. It does not upgrade the original hMailServer's 5.7 builds of August and September 2026 (5.7.0, 5.7.1 and 5.7.2), whose database went on from schema 5708 by steps this fork's chain does not have: move such a server's mail over IMAP instead ([Migration.md](https://gitlab.com/hmailserver/hmailserver/-/blob/master/hmailserver/docs/Migration.md)). 6.3.7 is signed by the project's GitLab pipeline, a signer compiled into every server from 6.3.4 on, so a server on 6.3.6 installs it by its live update, with 6.3.6's file as its rollback image and with the exceptions the 6.3.7 announcement lists; a Windows service that runs as `NT SERVICE\hMailServer` has an administrator press Install update in the Control Panel on the server, and on 6.3.4 or 6.3.5 such a service still leaves the install to an administrator, by hand, once. On Linux, from 6.3.6 the live update applies it to the generic packages and the AppImages; from 6.3.4 or 6.3.5, or from one of 6.3.6's per-distribution packages, install it with `apt`, `dnf` or `zypper`. A server from 6.2.28 to 6.3.3 trusts only the GitHub signer, and is upgraded by hand, once, as is a Linux server installed from 6.3.3's AArch64 packages or AppImages. Read "Before you upgrade" in [the 6.3.7 announcement](/topic/28277), since some changes alter what an upgraded server does without anything being switched on, and back up the database and the data directory first. - **Security:** TLS 1.2 and 1.3 only, with MTA-STS, DANE and DNSSEC validation enforced on outbound mail by default, and a built-in ACME v2 client for Let's Encrypt. Since hMailServer 6.3.4 a new installation offers STARTTLS on 25, 587, 110 and 143 with a self-signed certificate and requires TLS for sign-in from the Internet, and on Windows runs the service as `NT SERVICE\hMailServer`; an upgrade keeps what it had. - **Off by default:** the REST API, the browser Control Deck and the webmail at `/portal` stay off until `RestApiPort` is set. Since hMailServer 6.3.4 the database is the settings store, so it is set in the Control Panel on Windows or with `sudo hmailserver --set-setting RestApiPort=` on Linux, not by editing hMailServer.ini. Windows is the more complete platform: Linux has no Control Panel and no COM API. Since hMailServer 6.3.4 Linux runs event scripts too, in JavaScript rather than VBScript or JScript, once scripting is switched on, and per-domain DKIM is configured there over the REST API or on the Control Deck. Moving a Windows installation to Linux is not a supported migration: treat it as a new installation plus a mailbox-level migration. ## Where things are - **Downloads and release notes:** the [releases on GitLab](https://gitlab.com/hmailserver/hmailserver/-/releases) and [hMailServer downloads](https://www.progressiverobot.com/hmailserver-downloads/). https://updates.progressiverobot.com/download/ always leads to the newest stable release. Releases from 6.2.19 to 6.3.3, and from 6.3.5, carry Sigstore signatures, and the Windows installer is Authenticode-signed since 6.3.1, as is the ARM64 installer 6.3.6 adds. 6.3.4 carries no Sigstore bundle: its SHA-256 checksums are in a file signed with the release tags' SSH key. From 6.3.4 the installer and each package have an SBOM of their own; in 6.3.6 so do the AppImages and the source tarball, and SLSA build provenance names every file of the release by its SHA-256. The container image is signed with cosign, each platform's image with its SBOM attested to it. "Verifying a release" in the [security policy](https://gitlab.com/hmailserver/hmailserver/-/blob/master/SECURITY.md#verifying-a-release) has the commands that check a download. The pinned release history topic in Announcements summarises every version. - **Documentation:** [hMailServer documentation](https://www.progressiverobot.com/hmailserver-documentation/), 37 chapters. - **Source code and issues:** [GitLab](https://gitlab.com/hmailserver/hmailserver); since 18 October 2026 the source and every release are mirrored on [GitHub](https://github.com/RainbowWizard-coder/hMailServer), where issues are off: they and merge requests stay on GitLab. Without a GitLab account, email a bug report or a question for the maintainers to the project's Service Desk address, [contact-project+hmailserver-hmailserver-86730559-issue-@incoming.gitlab.com](mailto:contact-project+hmailserver-hmailserver-86730559-issue-@incoming.gitlab.com): GitLab files the message as a confidential issue, which only the maintainers see, and their replies reach you by email. - **Forum pages:** [about](/about), [FAQ](/faq), [rules](/rules) and [how to ask](/how-to-ask). - **Categories:** [Announcements & releases](/category/1), [General support](/category/8), [Installation, migration & upgrades](/category/15), [Linux](/category/16), [Webmail, portal & Control Deck](/category/17), [TLS, certificates & deliverability](/category/18), [REST API & integrations](/category/20) and [Bug reports](/category/23). - **Commercial support:** [Managed hMailServer](/category/24), the [managed hMailServer page](https://www.progressiverobot.com/hmailserver-services/), [support plans](https://www.progressiverobot.com/support-plans) and [contact](https://www.progressiverobot.com/contact/). hMailServer has no per-mailbox fees and no paid tiers, and no answer on this forum depends on buying anything. ## Asking for help Post in [General support](/category/8) or the matching category, using the template in [how to ask](/how-to-ask): the exact version, the platform, the database and a log excerpt with credentials redacted, because the forum is public and indexed. Give the full build stamp on Windows (6.3.7.46 for hMailServer 6.3.7) or the exact package version on Linux. Never post a suspected security vulnerability in a thread or public issue. Report it privately, as the project's [security policy](https://gitlab.com/hmailserver/hmailserver/-/blob/master/SECURITY.md#reporting-a-vulnerability) says: open a [confidential issue on GitLab](https://gitlab.com/hmailserver/hmailserver/-/issues/new?issue[confidential]=true), check that it is still marked confidential before you submit it, and choose the Security template; or, without a GitLab account, email contact-project+hmailserver-hmailserver-86730559-issue-@incoming.gitlab.com, which GitLab files as a confidential issue. Give the version, the platform and enough detail to reproduce it. --- Canonical: https://www.hmailserver.co.uk/faq # hMailServer FAQ ## The project ### Is hMailServer still maintained? Yes, hMailServer is still maintained. Progressive Robot Ltd forked it in 2026 and maintains the hMailServer 6 line. The current release is hMailServer 6.3.7, a security release published on 17 October 2026 for Windows and for Linux on x86-64 and AArch64. 6.3.6, of 6 October 2026, was the last release planned by date; the releases since, 6.3.7 the first, are security releases cut from master, and the 6.3 line receives security fixes until 30 September 2027. Martin Knafve created hMailServer. The original project went quiet for years. In 2026 Progressive Robot forked it, rebuilt it on a current toolchain (Visual Studio 2026 and OpenSSL 4, 64-bit only) and released hMailServer 6.0.0 on 12 June 2026. Linux support arrived in hMailServer 6.3.0 on 10 September 2026; 6.3.7 is packaged for x86-64 and AArch64, with a package for each of nine distribution releases beside the generic .deb and .rpm, and a container image. The original project has since resumed development as well. Its changes are reviewed against the hMailServer 6 code, and several have been ported. Release notes, SHA-256 checksums and packages for every version are on the [downloads page](https://www.progressiverobot.com/hmailserver-downloads/). New releases are announced in [Announcements & releases](/category/1). ### Who maintains hMailServer now? Progressive Robot Ltd, based at Chester Business Park in Chester, United Kingdom, forked hMailServer in 2026 and maintains the hMailServer 6 line. hMailServer was originally created by Martin Knafve, and the original project has since resumed development independently of this fork. Progressive Robot has published hMailServer 6 releases since 6.0.0 on 12 June 2026; the current release is hMailServer 6.3.7, released 17 October 2026. Since 6.3.1 the Windows installer has carried an Authenticode signature naming Progressive Robot Ltd, and the 6.3.2 release notes state that the project's GOVERNANCE.md names two maintainers. Progressive Robot also runs this forum, where its staff post as **chris** and **Progressiverobot**. hMailServer remains free and open source under AGPL-3.0, with no per-mailbox fees and no paid tiers. See [About this forum](/about), the official [hMailServer page](https://www.progressiverobot.com/hmailserver-services/) and the [GitLab repository](https://gitlab.com/hmailserver/hmailserver). ### Who originally created hMailServer? hMailServer was originally created by Martin Knafve. The readme of hMailServer 6.0.0 Beta 1, dated 12 June 2026, credits the software as "Based on hMailServer by Martin Knafve". Under the original project it was a free, Windows-only mail server from 2002. Progressive Robot did not create hMailServer. It forked the original project in 2026, after that project had gone quiet for years, and maintains the hMailServer 6 line. The first production release, hMailServer 6.0.0, is dated 12 June 2026, and hMailServer 6.3.0 added Linux support on 10 September 2026. The original project has since resumed development. The notes for the hMailServer 6.2.23-alpha2 pre-release of 4 September 2026 record that seven fixes were taken from it. See [About this forum](/about). ### Is hMailServer free? hMailServer is free and open source under the AGPL-3.0 licence, including for commercial use, with no per-mailbox fees and no paid tiers. The current release is hMailServer 6.3.7, released on 17 October 2026. It is a free download for Windows (x64, and a preview for ARM64) and for Linux on x86-64 and AArch64, and as a container image. Simply using hMailServer requires nothing of you under the licence. If you modify it and offer it to others over a network, you must publish your changes. The Windows installer and the Linux `.deb` and `.rpm` packages are on the [official downloads page](https://www.progressiverobot.com/hmailserver-downloads/), and the source code is on [GitLab](https://gitlab.com/hmailserver/hmailserver), with the source and every release mirrored on [GitHub](https://github.com/RainbowWizard-coder/hMailServer) since 18 October 2026. Progressive Robot Ltd, which maintains the hMailServer 6 line, sells optional [managed hMailServer](https://www.progressiverobot.com/hmailserver-services/) services and [support plans](https://www.progressiverobot.com/support-plans). No answer on this forum is gated behind a support plan. ## Versions and upgrades ### What is the latest version of hMailServer? hMailServer 6.3.7 is the latest release of hMailServer, published on 17 October 2026 for Windows and Linux under the AGPL-3.0 licence. It is a security release: 6.3.6, of 6 October 2026, was the last release planned by date, and the releases since, 6.3.7 the first, are security releases cut from master; the 6.3 line receives security fixes until 30 September 2027. It ships as an x64 Windows installer, `hMailServer-6.3.7-x64.exe`, a preview installer for Windows on ARM64, a .deb and an .rpm for x86-64 and AArch64 Linux with a package for each of nine distribution releases beside the generic .deb and .rpm, two AppImages and a container image, `registry.gitlab.com/hmailserver/hmailserver:6.3.7`. Every file is signed with Sigstore by the project's GitLab pipeline. Every server running 6.3.6 or earlier should upgrade. hMailServer 6.3.7 fixes security flaws in every earlier version, among them CVE-2026-107571 (on Windows, any account that could sign in to the computer could replace the programs of an hMailServer installed at the root of a drive), CVE-2026-107567 (mail from outside was DKIM-signed as a hosted domain) and CVE-2026-107569 (any sender could lift the rule loop limit). The schema stays at 6108, so from 6.3.6 the upgrade adds no one-way step. From 6.3.5 or older it still runs 6.3.6's three one-way steps from 6105 to 6108, so back up the database and the data directory first. On 6.3.4, 6.3.5 or 6.3.6 the live update installs it, with the exceptions the 6.3.7 announcement lists under "Upgrading"; a server from 6.2.28 to 6.3.3, or one from 6.3.3's AArch64 packages or AppImages, upgrades by hand, once. See the [release notes](https://gitlab.com/hmailserver/hmailserver/-/releases/v6.3.7), the [mirror on GitHub](https://github.com/RainbowWizard-coder/hMailServer/releases/tag/v6.3.7) and [Announcements & releases](/category/1). ### Can I upgrade from hMailServer 5.x to 6? Yes, hMailServer 5.3, 5.4, 5.5, 5.6 and 5.7 installations on Windows upgrade in place directly to hMailServer 6.3.7, released on 17 October 2026, with no intermediate versions. Run `hMailServer-6.3.7-x64.exe` as administrator and accept the upgrade. The installer upgrades the database schema to 6108 and keeps mail, accounts, domains and settings. The schema upgrade is one-way, so back up the database, the data directory and `hMailServer.INI` first, and on 5.3 to 5.5 also test the restore. On a large database, plan a maintenance window for four of the schema steps. On MySQL or MariaDB the upgrade needs MySQL 5.7.9 or MariaDB 10.2.2 or later and the ALTER privilege, or it stops at step 6041 having changed nothing. Change `hMailServer.INI` after the upgrade, not before: since 6.3.4 the database holds the settings. Since 6.3.6 the installer makes the data, log, database and event folders private to the administrators and the service, so a backup program or a script that read them as an ordinary account needs an entry of its own. Since 6.3.7 it also hardens the program folder; for an installation at the root of a drive, read "Before you upgrade" in the 6.3.7 announcement first. The classic Administrator (`hMailAdmin.exe`) is replaced by the Control Panel (`hMailCP.exe`), and the PHP WebAdmin is gone. If the installer cannot add the .NET 10 Desktop Runtime, the schema is not upgraded and the service will not start: install the runtime by hand, then run `DBUpdater.exe`. See chapter 18 of the [hMailServer documentation](https://www.progressiverobot.com/hmailserver-documentation/) or ask in [Installation, migration & upgrades](/category/15). ### How do I verify an hMailServer download? To verify an hMailServer 6.3.7 download (the current release, 17 October 2026), check the Sigstore bundle that sits beside every file of the release as `.cosign.bundle`. With cosign, run `cosign verify-blob --bundle hMailServer-6.3.7-x64.exe.cosign.bundle --certificate-identity 'https://gitlab.com/hmailserver/hmailserver//.gitlab-ci.yml@refs/tags/v6.3.7' --certificate-oidc-issuer https://gitlab.com hMailServer-6.3.7-x64.exe`, which must say `Verified OK`. The identity is the project's GitLab pipeline for the release's tag, and the certificate carries the project's id, 86730559; 6.3.5's files name `Progressiverobot/hmailserver`, the path the project had before 29 September 2026. Then `sha256sum --ignore-missing -c hmailserver-linux-6.3.7-SHA256SUMS.txt` must say OK for each Linux file, and on Windows `Get-AuthenticodeSignature .\hMailServer-6.3.7-x64.exe` must answer `Valid`, signed by Progressive Robot Ltd; the x64 installer's published SHA-256 is `dcae96bd75d6cae8e5a910d15940567067539ed99db5aeed40a815added4d85f`. On Linux a server from 6.3.4 on makes the same check as its live update with `hmailserver --verify-update `. The files on the [GitHub mirror](https://github.com/RainbowWizard-coder/hMailServer/releases/tag/v6.3.7) are the same files, each checked against GitLab's SHA-256 before it was uploaded, and verify the same way, except that GitHub names the Debian and Ubuntu per-release packages with a dot where the checksum list has a tilde (`hmailserver_6.3.7-1.debian12_amd64.deb` for `hmailserver_6.3.7-1~debian12_amd64.deb`): rename such a file before `sha256sum -c`, or compare its hash by hand. The releases from 6.2.19 to 6.3.3 carry Sigstore bundles signed by the project's former GitHub workflow, and 6.3.4 alone has none: its checksum list is signed with the SSH key that signs the release tags. See "Verifying a release" in [SECURITY.md](https://gitlab.com/hmailserver/hmailserver/-/blob/master/SECURITY.md), the [6.3.7 release](https://gitlab.com/hmailserver/hmailserver/-/releases/v6.3.7) and the [downloads page](https://www.progressiverobot.com/hmailserver-downloads/), which shows each file's SHA-256. ## Platforms and installation ### Does hMailServer run on Linux? hMailServer has run on 64-bit Linux since hMailServer 6.3.0, released on 10 September 2026. The current release, hMailServer 6.3.7 of 17 October 2026, ships for x86-64 and AArch64: a generic .deb and .rpm, and a package of its own for each of Debian 12 and 13, Ubuntu 24.04, Fedora 43 and 44, Enterprise Linux 9 and 10, and openSUSE Leap 16.0 and Tumbleweed, each built and tested in that release's own container (the AArch64 ones under emulation). Each installs a systemd service configured under `/etc/hmailserver`. The generic .deb asks for glibc 2.38 or later, so Debian 12 takes its own package, and the generic .rpm does not install on Enterprise Linux 9, which takes its own. There are also two AppImages, a container image, `registry.gitlab.com/hmailserver/hmailserver:6.3.7`, for linux/amd64 and linux/arm64, and a Helm chart; Arch uses the PKGBUILD in the source tree. The database is PostgreSQL or MySQL/MariaDB. Linux has no Control Panel, COM API or Microsoft SQL Server support. A Linux server is administered through the `hmailserver` command line, the REST API and the browser Control Deck, which is also where per-domain DKIM is configured. Since 6.3.4, event scripts run on Linux in JavaScript. Moving an existing Windows installation to Linux is not a supported migration. See [Running hMailServer on Linux](https://www.progressiverobot.com/hmailserver-documentation/#35-running-hmailserver-on-linux) and the [Linux](/category/16) category. ### Which Windows versions does hMailServer support? hMailServer 6.3.7, released on 17 October 2026, supports Windows 10 version 1607 or later, Windows 11, and Windows Server 2016 or newer, on 64-bit x64 systems. The Windows download is a single installer, `hMailServer-6.3.7-x64.exe`, and there has been no 32-bit build since hMailServer 6.0. A second installer, `hMailServer-6.3.7-arm64.exe`, is a preview for Windows on ARM: it is cross-compiled and has never run on ARM64 hardware, and a new ARM64 installation needs SQL Server or PostgreSQL, since SQL Server Compact has no ARM64 build. The documentation asks for at least 2 GB of RAM and 20 GB of free disk space. Run the installer as an administrator. It upgrades an existing hMailServer 5.3 to 5.7, 6.0, 6.1, 6.2 or 6.3 installation in place; a server on 6.3.4, 6.3.5 or 6.3.6 can install it by the live update instead. The Control Panel and the database tools are .NET 10 applications, and the installer adds the .NET 10 Desktop Runtime if it is missing, but does not replace one already installed: keep that up to date through Microsoft Update. Windows is the more complete platform: the Control Panel, the COM API and VBScript or JScript event scripts are Windows-only, while Linux runs event scripts in JavaScript and configures per-domain DKIM over the REST API. See chapter 3, "Before you install: the checklist", in the [hMailServer documentation](https://www.progressiverobot.com/hmailserver-documentation/), and [Installation, migration & upgrades](/category/15). ### Which databases does hMailServer support? hMailServer supports four databases on Windows: the built-in SQL Server Compact database (the default), MySQL or MariaDB, Microsoft SQL Server (including the Express edition) and PostgreSQL. On Linux, supported since hMailServer 6.3.0 (10 September 2026) and in the current release, 6.3.7 (17 October 2026), it supports two: PostgreSQL and MySQL/MariaDB. Upgrading to hMailServer 6.3.4 or later on MySQL or MariaDB needs MySQL 5.7.9 or MariaDB 10.2.2 or later and the ALTER privilege, and since 6.3.4 the server connects to them over TLS whenever the database server offers it. The x64 Windows installer of 6.3.7 carries MariaDB Connector/C 3.4.11, with seven authentication plugins where 6.3.6's had nine: a database account that signs in with MySQL Enterprise's Windows authentication can no longer do so. The Linux build refuses Microsoft SQL Server and SQL Server Compact by name, with an explanation in the error log, because both are reached through Windows-only interfaces. It was tested end to end against PostgreSQL 18 and MariaDB 11.8. PostgreSQL is reached through libpq, which the package depends on. The MySQL/MariaDB client is loaded at run time and must be installed separately: `libmariadb3` on Debian and Ubuntu, `mariadb-connector-c` on Fedora and RHEL, `mariadb-libs` on Arch. The documentation calls the built-in database adequate on Windows for up to a few dozen mailboxes, and lists MySQL/MariaDB for most production use and PostgreSQL for larger installations. See [choosing a database](https://www.progressiverobot.com/hmailserver-documentation/#35-a-database) and [creating the database on Linux](https://www.progressiverobot.com/hmailserver-documentation/#354-creating-the-database). ### Can I move hMailServer from Windows to Linux? Moving an hMailServer installation from Windows to Linux is not a supported migration in hMailServer 6.3.7, released 17 October 2026. Both platforms use database schema 6108 and the same message store format, but there is no tested path, so the documentation says to treat the move as a new installation plus a mailbox-level migration, not as a restore. Known breaks: - A configuration file with `Type=MSSQL` or `Type=MSSQLCE` is refused on Linux; migrate that database to PostgreSQL or MySQL on Windows first. - DPAPI-protected secrets, including route, fetch-account and relay passwords, private-key passphrases and the administrator's TOTP secret, cannot be read on Linux and must be re-entered. - COM integrations do not work on Linux, and event scripts run there in JavaScript only, so a VBScript or JScript file has to be ported. - Per-domain DKIM key files hold Windows paths: copy the keys and point each signing domain at them again. - Directory names derived from addresses are lower-cased, and the message store must be owned by `hmailserver`. If you try anyway, keep the Windows server running until the copy is verified. See section 35.14 of the [hMailServer documentation](https://www.progressiverobot.com/hmailserver-documentation/), or ask in [Installation, migration & upgrades](/category/15) or [Linux](/category/16). ## Features ### Does hMailServer have webmail? hMailServer has a built-in webmail at `/portal`, served by its REST API listener and compiled into the server binary on both Windows and Linux. A browser mailbox page arrived in hMailServer 6.2.28, it became a full webmail in 6.3.0, and hMailServer 6.3.3 rebuilt it in the shape of Gmail and Outlook.com; hMailServer 6.3.4 added calendars, contact groups, the organisation's directory and passkey sign-in; the current release is 6.3.7 (17 October 2026). It is off by default. Setting `RestApiPort` turns on the REST API, the Control Deck and the webmail together: in the Control Panel on Windows, or with `sudo hmailserver --set-setting RestApiPort=` on Linux. Since 6.3.4 the database holds the settings, so a line edited in `hMailServer.ini` after the first start is put back. The listener will not start while the administrator password is empty, binds to 127.0.0.1 by default, and needs a TLS certificate and key before it can bind to a non-loopback address. Users sign in with their own mailbox address and password; the administrator password and API keys are refused. IMAP clients, or Roundcube pointed at the IMAP service, remain a reasonable alternative. See [chapter 36](https://www.progressiverobot.com/hmailserver-documentation/#36-the-control-deck-and-the-webmail-portal), [the version history](https://www.progressiverobot.com/hmailserver-documentation/#25-version-history) and [Webmail, portal & Control Deck](/category/17). ### Does hMailServer support Let's Encrypt? hMailServer supports Let's Encrypt through a built-in ACME v2 client that obtains a certificate, assigns it to TLS ports that have none of their own and renews it before expiry, with no scheduled task or manual restart. Use hMailServer 6.2.25 or later on Windows, or 6.3.0 or later on Linux; the current release is 6.3.7 (17 October 2026). Since 6.3.7 the HTTP-01 challenge responder serves up to 64 connections at once; earlier versions served one at a time, so idle clients could make issuance or renewal fail. Releases up to 6.2.21 never obtained a certificate from Let's Encrypt, and 6.2.24 stopped the service whenever a certificate was issued or renewed. The client is off by default. Set `AcmeEnabled=1`, `AcmeContactEmail` and `AcmeDomains` in the Control Panel under Settings, Security, Certificates (ACME), over the REST API, or on Linux with `sudo hmailserver --set-setting`. Since 6.3.4 these are database settings: a line edited in `hMailServer.INI` after the first start is put back. Validation uses HTTP-01, so port 80 must be reachable from the internet and every name in `AcmeDomains` must already resolve to the server. Keep `AcmeReuseKey=1` (the default) so published DANE TLSA records stay valid across renewals. See the [hMailServer documentation](https://www.progressiverobot.com/hmailserver-documentation/) (sections 9.1 and 35.9) and [TLS, certificates & deliverability](/category/18). ### Does hMailServer support DKIM, DMARC, SPF, MTA-STS and DANE? hMailServer supports SPF, DKIM, DMARC, MTA-STS and DANE, plus ARC sealing and TLS-RPT. MTA-STS, DANE, DMARC, ARC, TLS-RPT and Ed25519 DKIM arrived in hMailServer 6.0.0. For outbound mail, `MtaStsEnabled`, `DaneEnforcementEnabled` and `DnssecValidationEnabled` are on by default. If a recipient domain has published a policy, it must present verified TLS or the message is deferred rather than sent unencrypted. To host your own MTA-STS policy, turn on the web-services listener with `WebServicesHttpsPort=443`. hMailServer signs outgoing mail with RSA or Ed25519 DKIM keys, and scores inbound SPF, DKIM and DMARC failures as spam tests. You still publish your own SPF, DKIM and DMARC records in DNS. Caveats: ARC sealing is off by default and seals only mail from hosted, DKIM-enabled domains. TLS-RPT reports are sent only once `TlsRptFromAddress` is set. On Linux, per-domain DKIM signing is configured over the REST API or on the Control Deck. Since 6.3.7 an authenticated submission is signed as a hosted domain only when its account may send as the From address, and mail from outside whose From names a hosted domain is no longer signed with that domain's key (CVE-2026-107567). Since 6.3.4 a domain's DKIM keys can also rotate on a policy, off by default, and SPF follows RFC 7208, so mail from 127.0.0.1 or `::1` no longer passes it unconditionally. See the [hMailServer documentation](https://www.progressiverobot.com/hmailserver-documentation/) (chapters 8 and 9) and [TLS, certificates & deliverability](/category/18). ### Does hMailServer have a REST API? hMailServer has had a REST API since version 6.0.0, served under `/api/v1/` and described by an OpenAPI 3.0.3 document at `GET /api/v1/openapi.json`. It gained a large write surface in 6.3.0 (10 September 2026). At hMailServer 6.3.4 the project's parity check counted 458 of the 466 properties the Windows Control Panel writes as writable over REST; the eight it lists as missing, an account's directory fields, are written on a route of their own. The API is off by default on Windows and Linux. Setting `RestApiPort` turns it on, bound to 127.0.0.1 by default: in the Control Panel, over COM, or on Linux with `sudo hmailserver --set-setting RestApiPort=`. Since 6.3.4 the database holds these settings, not `hMailServer.ini`. It will not start with an empty administrator password, and on any address other than 127.0.0.1, localhost or ::1 it needs a TLS certificate and key. Credentials are the administrator password, scoped API keys, session cookies and a mailbox's own login, which reaches only `/api/v1/me` and sign-in. Linux has no Control Panel or COM, so there the REST API and the Control Deck it serves at `/` do that job. See [the REST API chapter](https://www.progressiverobot.com/hmailserver-documentation/#24-the-rest-api) and [REST API & integrations](/category/20). ### Is hMailServer a replacement for Microsoft Exchange? hMailServer is not a replacement for Microsoft Exchange: it is a mail server for SMTP, IMAP and POP3, with no Teams integration and no ActiveSync. The current release, hMailServer 6.3.7 (17 October 2026), is free and open source under the AGPL-3.0 licence, with no per-mailbox fees, and runs on Windows and Linux. It hosts mail for many domains, with spam filtering, virus scanning through ClamAV, a rules engine and Sieve filtering. It serves each account's address book over CardDAV (since 6.3.3) and, since 6.3.4, calendars over CalDAV, with invitations, free-busy, delegation and bookable rooms, once the web services HTTPS listener is turned on. Users read mail in Outlook, Thunderbird, Apple Mail, on a phone or in the built-in webmail at `/portal` once the REST listener is turned on. The documentation suggests it for small businesses, developers needing a real SMTP and IMAP test endpoint and hosting providers. If nobody on staff is comfortable editing DNS records and watching a log file, it says a hosted mailbox provider will serve them better. See chapter 1 of the [hMailServer documentation](https://www.progressiverobot.com/hmailserver-documentation/). ## Getting help ### Where can I get help with hMailServer? Free help with hMailServer is available on the hMailServer community forum, run by Progressive Robot Ltd, which forked hMailServer in 2026 and maintains the hMailServer 6 line. hMailServer is free under AGPL-3.0, and no answer on the forum is gated behind a support plan. Post general problems in [General support](/category/8); [Linux](/category/16), [TLS, certificates & deliverability](/category/18) and [REST API & integrations](/category/20) have their own categories. Follow the [how to ask](/how-to-ask) template: the full version with its build stamp, the platform, the database, how the install got there, and a log excerpt with timestamps intact and credentials redacted. The [hMailServer documentation](https://www.progressiverobot.com/hmailserver-documentation/) runs to 37 chapters. Reproducible defects go on the [GitLab](https://gitlab.com/hmailserver/hmailserver) issue tracker, linked from the forum thread. Without a GitLab account, email the defect, or a question for the maintainers, to the project's Service Desk address, [contact-project+hmailserver-hmailserver-86730559-issue-@incoming.gitlab.com](mailto:contact-project+hmailserver-hmailserver-86730559-issue-@incoming.gitlab.com): GitLab files the message as a confidential issue, which only the maintainers see, and their replies reach you by email. Never post security vulnerabilities publicly: report them privately by one of the two routes in [SECURITY.md](https://gitlab.com/hmailserver/hmailserver/-/blob/master/SECURITY.md), a confidential GitLab issue or an email to the project's Service Desk address. For paid help, Progressive Robot offers deployment, migration, hardening and fully managed hMailServer on Windows and Linux. See [support plans](https://www.progressiverobot.com/support-plans), [Managed hMailServer](/category/24) or [contact Progressive Robot](https://www.progressiverobot.com/contact/). ### How do I report a security vulnerability in hMailServer? Report a security vulnerability in hMailServer privately, never in a forum thread or a public GitLab issue. There are two routes, and both reach the maintainers as a confidential issue: open a [confidential issue on GitLab](https://gitlab.com/hmailserver/hmailserver/-/issues/new?issue[confidential]=true) with the Security template, checking that *This issue is confidential* is still ticked, or email the project's Service Desk address, contact-project+hmailserver-hmailserver-86730559-issue-@incoming.gitlab.com, which needs no GitLab account. Include the hMailServer version, the platform, the database and enough detail to reproduce the problem. The maintainers aim to acknowledge a report within 5 working days and to ship a fix for a confirmed vulnerability within 90 days, and the release notes credit you unless you would rather not be named. Security advisories are posted in [Announcements & releases](/category/1). The full policy is in [SECURITY.md](https://gitlab.com/hmailserver/hmailserver/-/blob/master/SECURITY.md), and this is rule 6 of the [forum rules](/rules). ### Where is the hMailServer source code? The hMailServer 6 source code is on GitLab at [gitlab.com/hmailserver/hmailserver](https://gitlab.com/hmailserver/hmailserver), under the AGPL-3.0 licence. Progressive Robot Ltd forked hMailServer in 2026 and maintains the hMailServer 6 line in that repository. The original project, created by Martin Knafve, is separate and has resumed development. Since 18 October 2026 the source and every release are also mirrored on GitHub at [github.com/RainbowWizard-coder/hMailServer](https://github.com/RainbowWizard-coder/hMailServer); issues are off there, and issues and merge requests stay on GitLab. One source tree builds the 64-bit Windows server with Visual Studio 2026 and MSVC. Since hMailServer 6.3.0 it also builds the Linux server with clang or GCC. Every release from 6.0.0 to the current 6.3.7 has a tag. Since 6.2.24, two clean builds of each release have produced a byte-identical `hMailServer.exe`, so anyone with the same toolchain can rebuild a tag and compare the result. The Linux build steps are in [supported platforms and packages](https://www.progressiverobot.com/hmailserver-documentation/#352-supported-platforms-and-packages). Reproducible defects go on the [GitLab issue tracker](https://gitlab.com/hmailserver/hmailserver/-/work_items), or, without a GitLab account, by email to the project's Service Desk address, [contact-project+hmailserver-hmailserver-86730559-issue-@incoming.gitlab.com](mailto:contact-project+hmailserver-hmailserver-86730559-issue-@incoming.gitlab.com), which GitLab files as a confidential issue that only the maintainers see. ### Is this the same forum as the hMailServer forum at hmailserver.com? No. The hMailServer community forum at www.hmailserver.co.uk is separate from the forum at hmailserver.com, which belongs to the original hMailServer project created by Martin Knafve. This forum is run by Progressive Robot Ltd, which forked hMailServer in 2026 and maintains the hMailServer 6 line. The current release is hMailServer 6.3.7, released on 17 October 2026 for Windows and Linux under the AGPL-3.0 licence. The original project went quiet for years and has since resumed development separately. Accounts and posts from hmailserver.com were not carried over, so register here to post. Staff post under their own accounts, chris and Progressiverobot. Release notes, security advisories and forum notices are posted in [Announcements & releases](/category/1). Support questions go in categories such as [General support](/category/8), [Installation, migration & upgrades](/category/15), [Linux](/category/16) and [TLS, certificates & deliverability](/category/18). Before posting, read the [forum rules](/rules) and the template on [posting a support request](/how-to-ask). --- Canonical: https://www.hmailserver.co.uk/topic/28216/hmailserver-release-history-every-6.x-version-with-dates # hMailServer release history: every 6.x version, with dates Every hMailServer 6 release, newest first, with the date it was signed or built and a link to its announcement on this forum. Each is on [GitLab](https://gitlab.com/hmailserver/hmailserver/-/releases) with its notes and its files, and since 18 October 2026 on the [GitHub mirror](https://github.com/RainbowWizard-coder/hMailServer/releases) as well - but for 6.2.0 and 6.0.0-B4, whose files were not preserved - with Sigstore bundles from 6.2.19 to 6.3.3 and from 6.3.5 on, and SBOMs from 6.2.3 on and for the five earlier releases that carry an installer. 6.3.4 carries no Sigstore bundle: its checksums are in a file signed with the release tags' SSH key. From 6.3.4 the installer and each package have an SBOM of their own; from 6.3.5 the bundles are made by the project's GitLab pipeline for the release's tag. https://updates.progressiverobot.com/download/ always leads to the newest stable release. The current release is **hMailServer 6.3.7**, released 17 October 2026. ## Releases (37) | Version | Released | What changed | |---|---|---| | [6.3.7](/topic/28277/hmailserver-6.3.7-security-fixes-for-every-earlier-version-read-before-upgrading-and-the-first-security-release-of-the-pause) | 17 October 2026 | Security fixes for every earlier version, read before upgrading, and the first security release of the pause | | [6.3.6](/topic/28276/hmailserver-6.3.6-security-fixes-for-every-earlier-version-read-before-upgrading-and-the-last-release-for-the-time-being) | 6 October 2026 | Security fixes for every earlier version, read before upgrading, and the last release for the time being | | [6.3.5](/topic/28275/hmailserver-6.3.5-the-first-release-built-and-signed-by-its-gitlab-pipeline-and-the-quarantine-grouped-by-domain-and-helo) | 29 September 2026 | The first release built and signed by its GitLab pipeline, and the quarantine grouped by domain and HELO | | [6.3.4](/topic/28218/hmailserver-6.3.4-a-password-could-run-code-on-windows-servers-with-jscript-scripting) | 27 September 2026 | A password could run code on Windows servers with JScript scripting | | [6.3.3](/topic/28183/hmailserver-6.3.3-outbound-bdat-stalled-every-message-over-60-000-bytes) | 15 September 2026 | Outbound BDAT stalled every message over 60,000 bytes | | [6.3.2](/topic/28180/hmailserver-6.3.2-a-public-resolver-made-surbl-tag-every-message-with-a-link-as-spam) | 13 September 2026 | A public resolver made SURBL tag every message with a link as spam | | [6.3.1](/topic/28179/hmailserver-6.3.1-the-installer-is-signed-and-updaterequireauthenticode-works) | 11 September 2026 | The installer is signed, and UpdateRequireAuthenticode works | | [6.3.0](/topic/28184/hmailserver-6.3.0-the-server-runs-on-linux-but-not-with-per-domain-dkim-signing) | 10 September 2026 | The server runs on Linux, but not with per-domain DKIM signing | | [6.2.28](/topic/28178/hmailserver-6.2.28-the-server-can-verify-and-apply-its-own-updates) | 8 September 2026 | The server can verify and apply its own updates | | [6.2.27](/topic/28181/hmailserver-6.2.27-a-stripped-ds-record-turned-a-signed-zone-unsigned) | 7 September 2026 | A stripped DS record turned a signed zone unsigned | | [6.2.26](/topic/28167/hmailserver-6.2.26-sql-server-compact-upgrades-to-schema-6030-reported-as-failed) | 6 September 2026 | SQL Server Compact upgrades to schema 6030 reported as failed | | [6.2.25](/topic/28173/hmailserver-6.2.25-acme-issuance-and-renewal-terminated-the-server-process) | 6 September 2026 | ACME issuance and renewal terminated the server process | | [6.2.24](/topic/28170/hmailserver-6.2.24-delegated-imap-move-could-destroy-the-only-copy-of-a-message) | 4 September 2026 | Delegated IMAP MOVE could destroy the only copy of a message | | [6.2.21](/topic/28165/hmailserver-6.2.21-inbound-mail-from-a-postfix-relay-could-hang-forever) | 16 August 2026 | Inbound mail from a Postfix relay could hang forever | | [6.2.20](/topic/28169/hmailserver-6.2.20-fresh-installs-failed-and-silent-installs-hung-on-a-dialog) | 15 August 2026 | Fresh installs failed and silent installs hung on a dialog | | [6.2.19](/topic/28177/hmailserver-6.2.19-custom-dns-server-broke-every-lookup-since-6.2.16) | 15 August 2026 | Custom DNS server broke every lookup since 6.2.16 | | [6.2.18](/topic/28166/hmailserver-6.2.18-a-stalled-database-bounced-valid-mail-instead-of-retrying-it) | 12 August 2026 | A stalled database bounced valid mail instead of retrying it | | [6.2.17](/topic/28171/hmailserver-6.2.17-relayed-mail-could-stall-after-end-of-data-with-no-250) | 12 August 2026 | Relayed mail could stall after end-of-data with no 250 | | [6.2.16](/topic/28200/hmailserver-6.2.16-an-error-dialog-on-closing-the-ctrl-k-settings-palette) | 11 August 2026 | An error dialog on closing the Ctrl+K settings palette | | [6.2.15](/topic/28207/hmailserver-6.2.15-imap-sequence-sets-and-a-restore-that-could-empty-the-data-directory) | 11 August 2026 | IMAP sequence sets, and a restore that could empty the data directory | | [6.2.14](/topic/28205/hmailserver-6.2.14-imap-append-reported-success-for-mail-it-never-wrote) | 11 August 2026 | IMAP APPEND reported success for mail it never wrote | | [6.2.13](/topic/28204/hmailserver-6.2.13-a-data-stall-on-relayed-mail) | 11 August 2026 | A DATA stall on relayed mail | | [6.2.12](/topic/28206/hmailserver-6.2.12-every-c-component-now-targets-.net-8) | 10 August 2026 | Every C# component now targets .NET 8 | | [6.2.11](/topic/28208/hmailserver-6.2.11-two-control-panel-lists-read-their-class-name-to-screen-readers) | 7 August 2026 | Two Control Panel lists read their class name to screen readers | | [6.2.10](/topic/28201/hmailserver-6.2.10-the-com-api-reported-success-on-calls-it-refused) | 7 August 2026 | The COM API reported success on calls it refused | | [6.2.9](/topic/28198/hmailserver-6.2.9-the-dashboard-charts-that-rendered-white) | 7 August 2026 | The dashboard charts that rendered white | | [6.2.8](/topic/28202/hmailserver-6.2.8-blank-list-editors-and-a-control-panel-that-died-on-restart) | 7 August 2026 | Blank list editors and a Control Panel that died on restart | | [6.2.7](/topic/28195/hmailserver-6.2.7-file-pickers-and-one-click-dkim-in-the-control-panel) | 15 June 2026 | File pickers and one-click DKIM in the Control Panel | | [6.2.6](/topic/28199/hmailserver-6.2.6-imap4rev2-as-an-opt-in-mode-and-a-control-panel-redesign) | 15 June 2026 | IMAP4rev2 as an opt-in mode, and a Control Panel redesign | | [6.2.5](/topic/28197/hmailserver-6.2.5-default-installs-could-not-connect-to-their-own-database) | 15 June 2026 | Default installs could not connect to their own database | | [6.2.4](/topic/28203/hmailserver-6.2.4-active-directory-account-pickers-in-the-control-panel) | 15 June 2026 | Active Directory account pickers in the Control Panel | | [6.2.3](/topic/28196/hmailserver-6.2.3-sieve-and-managesieve-filtering) | 15 June 2026 | Sieve and ManageSieve filtering | | [6.2.2](/topic/28215/hmailserver-6.2.2-rfc-4013-saslprep-and-a-substantially-upgraded-control-panel) | 14 June 2026 | RFC 4013 SASLprep and a substantially upgraded Control Panel | | [6.2.1](/topic/28214/hmailserver-6.2.1-pipelining-smtputf8-dsn-and-srs) | 14 June 2026 | PIPELINING, SMTPUTF8, DSN and SRS | | [6.2.0](/topic/28209/hmailserver-6.2.0-the-new-control-panel-desktop-administration-app) | 14 June 2026 | The new Control Panel desktop administration app | | [6.1.0](/topic/28210/hmailserver-6.1.0-the-web-control-deck-and-a-light-dark-administrator) | 12 June 2026 | The Web Control Deck and a light/dark Administrator | | [6.0.0](/topic/28213/hmailserver-6.0.0-the-modernised-baseline-and-the-build-4-installer-fix) | 12 June 2026 | The modernised baseline, and the build 4 installer fix | ## Pre-releases (8) These were test builds. Everything in them shipped in a later release, and none of them should be installed today. | Version | Released | What changed | |---|---|---| | [6.2.23-alpha2](/topic/28172/hmailserver-6.2.23-alpha2-thunderbird-could-not-save-a-single-sent-copy) | 4 September 2026 | Thunderbird could not save a single Sent copy | | [6.2.23-alpha1](/topic/28175/hmailserver-6.2.23-alpha1-local-delivery-could-lose-a-message-after-answering-250) | 21 August 2026 | Local delivery could lose a message after answering 250 | | [6.2.22-pre6](/topic/28182/hmailserver-6.2.22-pre6-pre4-was-withdrawn-a-fresh-install-listened-on-nothing) | 20 August 2026 | Pre4 was withdrawn, a fresh install listened on nothing | | [6.2.22-pre3](/topic/28176/hmailserver-6.2.22-pre3-the-server-now-enforces-per-account-2fa-and-app-passwords) | 19 August 2026 | The server now enforces per-account 2FA and app passwords | | [6.2.22-pre2](/topic/28174/hmailserver-6.2.22-pre2-dmarc-aggregate-reports-rua-are-now-sent-not-just-consumed) | 18 August 2026 | DMARC aggregate reports (rua) are now sent, not just consumed | | [6.2.22-pre1](/topic/28168/hmailserver-6.2.22-pre1-the-built-in-acme-client-never-worked-with-let-s-encrypt) | 16 August 2026 | The built-in ACME client never worked with Let's Encrypt | | [6.0.0-B4](/topic/28212/hmailserver-6.0.0-b4-fixes-the-startup-crash-from-the-bundled-vc-runtime) | 12 June 2026 | Fixes the startup crash from the bundled VC++ runtime | | [6.0.0-B1](/topic/28211/hmailserver-6.0.0-b1-first-regression-validated-build-of-the-modernised-6.0-line) | 12 June 2026 | First regression-validated build of the modernised 6.0 line | --- Canonical: https://www.hmailserver.co.uk/topic/28185/read-this-before-you-post-what-a-support-request-needs # Read this before you post: what a support request needs Most threads here stall for the same reason. The report describes the symptom but not the system, and the first three replies get spent asking for things that should have been in the first post. Include all of the following. **Version, with the build stamp.** Not "6.3" and not "the latest". The full four-part number as the downloads page gives it. 6.3.4 ships as build 6.3.4.43 on Windows, and 6.3.7 as 6.3.7.46. Point releases days apart change behaviour, so the build matters. **Platform and package.** Windows 10 (1607 or later), Windows 11, or Windows Server 2016 or newer. 64-bit only. On Linux, x86-64 or AArch64 with systemd, supported since 6.3.0; 6.3.4's packages are x86-64 only, and the AArch64 packages and the AppImage were last published with [6.3.3](https://gitlab.com/hmailserver/hmailserver/-/releases/v6.3.3) before 6.3.6, and 6.3.6 and [6.3.7](https://gitlab.com/hmailserver/hmailserver/-/releases/v6.3.7) carry them again for both architectures, the AArch64 files cross-compiled and tested under emulation. Say which package you installed: `hMailServer-6.3.4-x64.exe`, a `.deb`, an `.rpm`, or the AppImage; with 6.3.7, also whether it was the ARM64 Windows installer (a preview), the generic `.deb` or `.rpm` or your release's own, or the container image. Behaviour differs between them. **Database backend.** Built-in (SQL Server Compact), MySQL or MariaDB, Microsoft SQL Server, or PostgreSQL. On Linux only PostgreSQL and MySQL/MariaDB are accepted. The built-in database and SQL Server are refused by name there. Give the database server version too, not just the product. **Upgrade or fresh install.** If it was an upgrade, say what it was upgraded from. Upgrades are in place and do not require stepping through intermediate versions, so if yours behaved otherwise that is itself the report. If you upgraded to 6.3.4 and a schema step stopped, give the step and the database's error number (for example 1071 or 1062 on MySQL or MariaDB). If you upgraded from 6.3.1 or 6.3.2, say whether the Control Panel was closed first. If the install is fresh, say so. It rules out half the causes immediately. **Diagnostics output.** Run Monitoring & troubleshooting → Diagnostics in the Control Panel before posting and paste the result. It self-tests DNS resolution, database connectivity, port bindings and disk access, and it often answers the question on its own. See §16.4 of the [documentation](https://www.progressiverobot.com/hmailserver-documentation/). **The relevant log lines.** Logs live in the `Logs` directory on Windows and `/var/log/hmailserver` on Linux. Monitoring & troubleshooting → Logging controls what is written. Turn debug logging on only long enough to reproduce the fault, then turn it off. It is verbose and it will fill the disk. Paste the lines around the failure in a fenced code block. Not a screenshot. Ten lines of context beats a 40 MB attachment. **What you already tried,** and what changed before it broke. **Redact before you paste.** SMTP AUTH exchanges and database connection strings both land in the logs verbatim. AUTH PLAIN and AUTH LOGIN arguments are base64, which is an encoding, not encryption, and anyone reading the thread can decode them. Replace passwords, API keys and connection strings with placeholders. Keep hostnames and IP addresses where you can, because they are usually the useful part. This forum is public and indexed. Downloads and release notes: --- Canonical: https://www.hmailserver.co.uk/topic/28186/where-to-post-what-a-map-of-the-forum-categories # Where to post what: a map of the forum categories The category tree is small on purpose, but a few boundaries are not obvious. This is where each thing goes. If you get it wrong, a moderator moves the topic rather than closing it. ## Running hMailServer **General support.** The catch-all. If none of the categories below is clearly right, post here. **Installation, migration and upgrades.** Installer failures, moving an install to new hardware, changing database backend, and anything that went wrong during an upgrade. Read §18 of the [documentation](https://www.progressiverobot.com/hmailserver-documentation/) first. **Linux.** Packaging, systemd, file permissions, and anything specific to the `.deb`, `.rpm` or AppImage builds. Linux support arrived in 6.3.0. A protocol question that would be identical on Windows goes to General support. **Webmail, portal and Control Deck.** The web interfaces themselves. Mail that never arrived is a delivery problem, not a webmail problem. **TLS certificates and deliverability.** Certificate chains and renewal, TLS on the mail ports, SPF, DKIM, DMARC and reverse DNS. Anything ending in "the recipient rejected it" starts here. Documentation §8 and §9. **Anti-spam and message filtering.** Scoring, blocklists, false positives, virus scanning. Documentation §10 and §11. **Guides and tutorials.** Finished, tested write-ups. Not questions. If you are still asking, ask elsewhere and write the guide afterwards. ## Automation and integration **Scripting and the COM API.** Server-side scripts and anything driving hMailServer through COM. **REST API and integrations.** The HTTP API, and third-party systems talking to it. **Rules, Sieve and routing.** Server and account rules, Sieve scripts, routes and relays. Documentation §12 and §14. **Monitoring, metrics and logs.** Reading the logs, `JsonLogging=1` output going into Elasticsearch, Loki or Splunk, health checks and alerting. Documentation §16. A log line you cannot interpret that turns out to be a delivery fault still belongs in the relevant support category. ## Development and roadmap **Bug reports.** Reproducible defects only. Version including build stamp, platform, database backend, and the steps that reproduce it. Confirmed bugs are tracked at . **Pre-release builds.** Anything that is not a signed release. Keep pre-release findings out of the support categories, because the answers there assume a released build. **Feature requests.** One request per topic, so each can be discussed and closed on its own. ## Community **Off-topic.** Everything else. **Forum feedback and help.** Problems with this forum: accounts, formatting, moderation, categories. ## Managed hMailServer For customers of the [managed service](https://www.progressiverobot.com/hmailserver-services/). Do not post account or billing details in public. Whichever category you pick, the checklist in the pinned posting guide still applies. Version with build stamp, platform, database backend, upgrade or fresh install, logs with credentials redacted. --- Canonical: https://www.hmailserver.co.uk/topic/28193/upgrading-to-6.3.4-any-5.3-or-later-goes-straight-there # Upgrading to 6.3.4: any 5.3 or later goes straight there *Written for 6.3.4. hMailServer 6.3.7 (17 October 2026) is the newest release, a security release: upgrade every server running 6.3.6 or earlier. It requires schema 6108 and adds no step of its own (from 6.3.5 or older the upgrade runs 6.3.6's three one-way steps, from 6.3.3 or older 6.3.4's steps first, so this topic still applies on the way); a server on 6.3.4, 6.3.5 or 6.3.6 installs it by the live update, with the exceptions the announcement lists, one from 6.2.28 to 6.3.3 by hand, once, and an older one by the installer, as always. Read "Before you upgrade" there first: some changes alter what an upgraded server does without anything being switched on. See [the 6.3.7 announcement](/topic/28277).* hMailServer upgrades in place. The installer stops the service, replaces the program files in `Bin`, upgrades the schema if needed, and restarts. Your data directory is untouched. The `[Settings]` section of `hMailServer.INI` is taken into the database at 6.3.4's first start (below). **Every server installs 6.3.4 by hand.** It is Authenticode-signed and carries no Sigstore bundle, so no server's live update installs it: run `hMailServer-6.3.4-x64.exe` on Windows, or install the `.deb` or `.rpm` on Linux ([release page](https://gitlab.com/hmailserver/hmailserver/-/releases/v6.3.4)). **Any 5.3 through 6.3.3 goes straight to 6.3.4.** There are no intermediate steps. The upgrade chain is continuous on MySQL/MariaDB, MS SQL, PostgreSQL and SQL Server Compact, and DBUpdater walks whichever steps your database still needs. 6.3.4 needs schema 6104. A 6.3.3 database sits at 6040 and is walked through 64 steps. A 6.2.27 or 6.2.28 database sits at 6031 and is walked through 6.3.2's seven steps, 6.3.3's two and 6.3.4's 64. Older ones are walked the whole way. Path table: [§18.2](https://www.progressiverobot.com/hmailserver-documentation/#182-the-upgrade-path). Most of what 6.3.0 to 6.3.4 added is off by default. The REST API, the Control Deck and the portal all need `RestApiPort` set. What 6.3.4 changes on upgrade without a switch is listed first in its [release notes](https://gitlab.com/hmailserver/hmailserver/-/releases/v6.3.4). The ones you will notice: the database becomes the settings store, so after the first start a line edited in `hMailServer.INI` is put back (`HM5804`) and a setting is changed in the Control Panel, over COM or REST, or with `hmailserver --set-setting` on Linux; SPF follows RFC 7208, so mail from 127.0.0.1 or `::1` no longer passes it unconditionally; the audit trail and the alerts are on, though nothing is mailed until `AlertRecipient` and `AlertSenderAddress` are set; a COM script that changes a route, a relay, a DNS black list, an SSL certificate or a group must now call `Save()` before the change takes effect; some IMAP answers change from `BAD` to `NO`; and a MySQL or MariaDB connection uses TLS whenever the database server offers it, without checking the certificate, and a SQL Server session through MSOLEDBSQL 18 or SQL Native Client is encrypted whole, where through SQLOLEDB only the login is unless `[Database] TlsMode` is `require` or `verify-full`, which checks the certificate ([DatabaseTls.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/DatabaseTls.md)). An upgrade keeps its listeners, its service account and its secrets as they were; only a new installation gets STARTTLS with a self-signed certificate, the Windows service as `NT SERVICE\hMailServer` and a stored-secrets key. **Edit `hMailServer.INI` after the upgrade, not to prepare it.** At 6.3.4's first start a `[Settings]` line changed since the old server last started is taken for an edit and put back (`HM5804`). **Set again every password with a character above U+00FF.** 6.3.4 refuses a hash stored before 6.3.4 of such a password (Cyrillic, Greek or CJK letters, say) at every password sign-in, and the application log names the account. Set those passwords again soon after the upgrade: in the Control Panel, over COM or REST, and the administrator's own with `hMailServer.exe --set-admin-password` (`hmailserver --set-admin-password` on Linux). ASCII passwords are unchanged. ### Before you start Back up the database, the data directory and `hMailServer.INI` ([chapter 15](https://www.progressiverobot.com/hmailserver-documentation/#15-backup-and-restore)). Record your current version. Pick a quiet window; the service is down for a minute or two and senders retry. On a large store allow longer for the steps below. That backup is the rollback. The upgrade is one way: an older server refuses a newer database rather than misreading it. Rolling back means restoring the pre-upgrade database backup and then reinstalling your previous version, with the pre-upgrade `hMailServer.ini` put back beside the database if 6.3.4 had sealed the stored secrets (the older version's installer would otherwise hand Windows the sealed service-account password, and the service would not start) ([§18.6](https://www.progressiverobot.com/hmailserver-documentation/#186-rolling-back)). ### The steps that can take time, and the two that can stop Four schema steps want a maintenance window on a large database. 6024 to 6025 rewrites `hm_messages.messageflags` from tinyint to smallint: a locking table rewrite on your largest table, on every backend except PostgreSQL. 6029 to 6030 adds seventeen FOREIGN KEY constraints with ON DELETE CASCADE and reads every child table once. 6100 to 6101 adds two columns to `hm_messages`, which is not instant on every backend. 6101 to 6102, on MySQL and MariaDB, rebuilds every table as utf8mb4. 6029 to 6030 is one place an upgrade can stop. It sweeps orphaned rows before adding those constraints, and that sweep ran children before parents, so pruning an orphaned account, fetch account or distribution list could re-orphan rows nothing revisits, and the constraint that followed was refused. The reordering fix shipped in 6.3.2 and is in 6.3.3 and 6.3.4. It fires only from a schema below 6030, only against a database that already holds orphaned rows, on all four backends. No installation has reported hitting it. When that step fails it fails loudly, in the database engine's own words, and rolls back rather than doing anything quietly. The database is where it was, the error names the constraint it could not add, and those orphaned rows have to go before the upgrade will pass. 6101 to 6102 is the other. It needs MySQL 5.7.9 or MariaDB 10.2.2 or later, as 6040 to 6041 and 6100 to 6101 do, so on an older server the upgrade stops at 6041 with error 1071 and changes nothing; and it needs the ALTER privilege on the database. Two rows the old collation told apart and utf8mb4 does not, such as `straße` and `strase` in one address book on MariaDB 11.8, stop it with error 1062 until one of them is removed; a step stopped part of the way is finished by running it again. SQL Server, SQL Server Compact and PostgreSQL only move the version number at that step ([Upgrading.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Upgrading.md)). ### On 6.3.1 or 6.3.2 6.3.2 could not upgrade a database older than its own schema; those attempts stopped with `The server has not loaded its configuration` and changed nothing. The Control Panel's live update to 6.3.2 failed with installer exit code 5, because the Control Panel that started it held its files open. Both are fixed in 6.3.3. No live update installs 6.3.4 on any version, so close the Control Panel and run the installer by hand. On Linux the package replaces the binary and its post-install step runs `hmailserver --upgrade-database`. 6.3.4's packages are x86-64 only; the AArch64 packages were last published with [6.3.3](https://gitlab.com/hmailserver/hmailserver/-/releases/v6.3.3) before 6.3.6, and 6.3.7 carries `hmailserver_6.3.7_arm64.deb` and `hmailserver-6.3.7-1.aarch64.rpm`, which a server from 6.3.3's AArch64 packages installs by hand, once. ### Servers you are not upgrading yet A server from 6.2.28 to 6.3.3 still looks for updates at GitHub, which answers 404. Under `[Settings]` in its `hMailServer.INI`, add `UpdateFeedUrl=https://updates.progressiverobot.com/releases.json`, then restart the service, and its update check works again. Keep `UpdateAutoDownload` off on such a server until it runs 6.3.4 ([UpdateFeed.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/UpdateFeed.md)). [Downloads and release notes](https://www.progressiverobot.com/hmailserver-downloads/) · [Chapter 18, upgrading](https://www.progressiverobot.com/hmailserver-documentation/#18-upgrading-hmailserver) --- Canonical: https://www.hmailserver.co.uk/topic/28194/moving-to-new-hardware-and-why-windows-to-linux-is-a-rebuild # Moving to new hardware, and why Windows to Linux is a rebuild Moving to new hardware is a restore onto fresh metal. [Chapter 15](https://www.progressiverobot.com/hmailserver-documentation/#15-backup-and-restore) is the prerequisite, and the first time, rehearse onto a test machine. ### The plan ([§33.2](https://www.progressiverobot.com/hmailserver-documentation/#332-moving-hmailserver-to-a-new-machine)) 1. Check the new machine against the chapter 3 checklist. Size the disk for the mail store you have now, not the one you had at installation. 2. Install the same version, in the same path: a different version or directory is where these moves go wrong. Since hMailServer 5.4 a message row holds a bare file name and the path is rebuilt from the data directory; a database from before 5.4 can still hold full paths. 3. Back up the old server. The built-in backup does not include messages unless you tick the option. For a large store, stop the service, copy `Data\`, dump the database natively, and take `hMailServer.INI`, `Events\`, your DKIM keys and, if one was made, an export of the stored-secrets key (`--export-stored-secrets-key`). 4. Stop the old service for good and set it to Disabled. Anything it accepts after the backup is stranded. 5. Restore the data directory, the database and `hMailServer.INI`, in chapter 15's order. 6. Re-enter the protected secrets. 7. Keep the host name if you can. Update the A record with a lowered TTL, get a PTR on the new IP, re-run the firewall rules, repoint your port forwards, and run Monitoring & troubleshooting, Diagnostics last. Leave the old machine powered off but intact for a week or two. It is the rollback. **Budget for step 6.** With `ProtectStoredSecretsWithDPAPI=1`, the default, stored passwords are encrypted with machine-scoped Windows DPAPI and cannot be decrypted on the new machine. Re-enter the database connection password with `Bin\DBSetup.exe`, then route, smart-host relayer and external-account passwords in the Control Panel. Forget this and you get a server that cannot reach its own database. Since 6.3.4 the stored-secrets key that seals second factors and the `[Settings]` secrets is kept in `hMailServer.INI` under the same DPAPI, so a copied INI does not carry it: import the exported file on the new machine with `hMailServer.exe --import-stored-secrets-key ` and restart the service. An upgraded server has no such key until an administrator makes one. Linux to Linux is easier: secrets sit under `/.hmailserver-secret-key`, so copy that file too, preserving mode 0600 and its ownership. ### Windows to Linux is not a migration It is not supported and the project does not claim it is. The schema is the same on both, 6.3.4 needs 6104 either way, and the message store's on-disk format is the same. What is missing is a tested path: nobody has run a Windows installation onto Linux and verified the result ([§35.14](https://www.progressiverobot.com/hmailserver-documentation/#3514-moving-from-windows)). What is known to break: - **SQL Server.** An INI with `Type=MSSQL` or `Type=MSSQLCE` is refused by name. Move to PostgreSQL or MySQL on Windows first. - **Every DPAPI secret.** Route, fetch-account and per-domain relay passwords, private-key passphrases and the administrator's TOTP secret. Each is reported once as HM6414 and answers empty. - **Event scripts and COM integrations.** COM does not exist on Linux, and since 6.3.4 event scripts run there in JavaScript only: a VBScript file is rewritten and a JScript one ported, without COM objects. - **DKIM key paths.** The per-domain key-file setting holds a Windows path such as `C:\keys\mail.pem`, which resolves to nothing on Linux. Copy the keys and re-point every signing domain. - **Directory case.** The Linux build lower-cases directory names derived from an address; Windows leaves them as found. A store carrying `Test.com` is one this server looks for under `test.com`, and everything under it must end up owned by `hmailserver`. So treat it as a rebuild and a cutover. A new installation on Linux, accounts and mail brought over at mailbox level by mirroring folders over IMAP from the running Windows server. The Import Tool is not an option here: it is a Windows .NET program that drives the COM API, and Linux has neither ([§33.8](https://www.progressiverobot.com/hmailserver-documentation/#338-importing-accounts-and-mail-the-import-tool)), verified against a still-running Windows server, DNS moved last. --- Canonical: https://www.hmailserver.co.uk/topic/28189/hmailserver-on-linux-what-you-get-and-what-you-do-not # hMailServer on Linux: what you get and what you do not *Written for 6.3.4. hMailServer 6.3.7 (17 October 2026) is the current release, a security release: its Linux packages are the generic `.deb` and `.rpm` for x86-64 and AArch64, a package for each of Debian 12 and 13, Ubuntu 24.04, Fedora 43 and 44, Enterprise Linux 9 and 10 and openSUSE Leap 16.0 and Tumbleweed, and both AppImages, and its container image is `registry.gitlab.com/hmailserver/hmailserver:6.3.7`; the AArch64 files are cross-compiled and tested under emulation, and no apt or dnf repository publishes it yet. From 6.3.6 the live update installs it through the root helper, for the generic packages and the AppImages; a server from one of 6.3.6's per-distribution packages upgrades with its package manager and its release's own package, and from 6.3.4 or 6.3.5 install it with `apt`, `dnf` or `zypper`. See [the 6.3.7 announcement](/topic/28277).* Since 6.3.0 the same source tree builds, installs and delivers mail on Linux, as a systemd service, from a `.deb`, an `.rpm` or an Arch PKGBUILD: on x86-64 and AArch64 up to 6.3.3, and on x86-64 only in 6.3.4's packages. That is why the release is numbered 6.3 and not 6.2.29. Read [§35.1](https://www.progressiverobot.com/hmailserver-documentation/#351-what-runs-on-linux-and-what-does-not) before you install anything. Finding out about a missing capability after the mailboxes are populated is the wrong order. ### What crossed The SMTP, POP3 and IMAP engines, delivery, routing, rules, anti-spam, anti-virus, ACME, the backup manager, the REST API, the Control Deck and the self-service portal. The on-disk message format is unchanged: one directory per domain and per mailbox, one file per message. The database schema is the same version the Windows build of the same release needs. ### The packages 6.3.4's `.deb` installs with `apt` on Ubuntu 26.04, where it is built, and on Ubuntu 24.04. It needs glibc 2.38 or later, so it does not install on Debian 12. Its `.rpm` installs with `dnf` on Fedora 43 and 44 and Enterprise Linux 10, not on Enterprise Linux 9, whose glibc and libstdc++ are older than it asks for, and nothing has been tried with openSUSE's zypper. There is no apt or dnf repository yet. Arch is a PKGBUILD in the tree, built with `makepkg -si` from the git tag, which it clones from `https://gitlab.com/hmailserver/hmailserver`. 6.3.4's packages are x86-64 only: the AArch64 packages were last published with [6.3.3](https://gitlab.com/hmailserver/hmailserver/-/releases/v6.3.3), built natively on the CI the project lost with its GitHub account. Nothing else is packaged. The AppImage is not for running mail. It runs as whoever started it, with no `hmailserver` user, no unit and none of the systemd hardening, it puts its store under the invoking user's home directory, and it cannot bind port 25. Use it to see the thing work on a laptop. 6.3.4 has none; the last AppImage is 6.3.3's. [§35.2](https://www.progressiverobot.com/hmailserver-documentation/#352-supported-platforms-and-packages). ### Where things live Configuration is `/etc/hmailserver/hMailServer.ini` at `0640 root:hmailserver`. The store is `/var/lib/hmailserver`, the logs `/var/log/hmailserver`, the unit `/usr/lib/systemd/system/hmailserver.service`. The service runs as `hmailserver:hmailserver` and never as root. Full table: [§35.6](https://www.progressiverobot.com/hmailserver-documentation/#356-the-filesystem-layout). The backends are PostgreSQL through `libpq`, a link-time dependency, and MySQL or MariaDB through a client loaded with `dlopen` at run time, which nothing pulls in for you. SQL Server and SQL Server Compact are refused by name, so an INI carried across from Windows is repointed or it does not start. ### The gaps, stated plainly - **No Control Panel and no COM API.** Both are Windows-only and are not compiled here. Every third-party COM script stops at that boundary. - **Event scripts in JavaScript only, since 6.3.4.** `EventHandlers.js` runs in an embedded QuickJS engine, for the same events under the same function names; the scripting language must be set to `JavaScript` (`hmctl settings set scripting --set-string language=JavaScript`), and VBScript and JScript are refused. Scripting is off until you turn it on. A script has no file, process or network access and cannot create COM objects, so a Windows script is ported, not copied ([EventScripts.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/EventScripts.md)). - **Domain properties were not writable when Linux arrived, and are now.** At 6.3.0 per-domain DKIM, per-domain size limits and the domain signature were readable over REST and not writable. 6.3.3 made `PUT /api/v1/domains/{domain}` take the whole domain - its DKIM signing, its limits, its signature and its relay host among the rest - and gave the Control Deck full domain editing. At 6.3.4 the project's census finds 458 of the 466 properties the Windows Control Panel writes writable over REST and reachable from a Deck view, and the eight it lists as missing, an account's directory fields, are written on a route of their own, `PUT /api/v1/accounts/{address}/directory`, which the Deck's account editor uses ([DeckParity.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/DeckParity.md)). - **Self-update from 6.3.4, but not to it.** The packages now install `hmailserver-update`, a root helper outside the server's unit, and the live update, when an administrator asks it to install, applies a `.deb`, an `.rpm` or an AppImage through it, rolling back when the server does not come back. The update check is off until you turn it on. 6.3.4 itself carries no Sigstore bundle, so no live update installs it: every server reaches 6.3.4 with apt or dnf, and the helper has nothing to apply before a release signed on the project's GitLab CI. Arch upgrades with pacman. - **No tested path from Windows.** The schema and the store format are shared, so the pieces are there. Nobody has run a Windows installation onto Linux and verified the result. [§35.14](https://www.progressiverobot.com/hmailserver-documentation/#3514-moving-from-windows) says what is known. Administration is therefore the Control Deck at `/` on the REST listener, and the REST API behind it. [§35.10](https://www.progressiverobot.com/hmailserver-documentation/#3510-administering-a-linux-server) covers the daily jobs. --- Canonical: https://www.hmailserver.co.uk/topic/28190/first-install-on-linux-the-order-of-operations-ubuntu-to-enterprise-linux # First install on Linux: the order of operations, Ubuntu to Enterprise Linux *Written for 6.3.4. hMailServer 6.3.7 (17 October 2026), a security release, is the release to install: beside the generic `.deb` and `.rpm` for x86-64 and AArch64 it carries a package for each of Debian 12 and 13, Ubuntu 24.04, Fedora 43 and 44, Enterprise Linux 9 and 10 and openSUSE Leap 16.0 and Tumbleweed, each installed in a clean container of its release by that release's own package manager before publication, so step 1 is `apt`, `dnf` or `zypper` with your release's own package; the AArch64 files are cross-compiled and tested under emulation. Check a download with `sha256sum -c --ignore-missing hmailserver-linux-6.3.7-SHA256SUMS.txt` first. See [the 6.3.7 announcement](/topic/28277).* The order matters more than the commands. The package installs the server **enabled and stopped**, on purpose: it has no database yet, and nothing in a package can know which backend you have or its credentials. The full sequence, with the output each step prints, is [§35.3](https://www.progressiverobot.com/hmailserver-documentation/#353-installing). Take the exact file names from the [downloads page](https://www.progressiverobot.com/hmailserver-downloads/), and note the RPM carries a `-1` release field in its name. **1. Install the package.** `sudo apt install ./.deb` on Ubuntu 26.04 or 24.04, `sudo dnf install ./.rpm` on Fedora 43 or 44 or Enterprise Linux 10. 6.3.4's `.deb` needs glibc 2.38 or later, so it does not install on Debian 12; its `.rpm` does not install on Enterprise Linux 9, and nothing has been tried with zypper on openSUSE. The maintainer script creates the `hmailserver` system user and group, creates the directories, and enables the unit without starting it. It deliberately does not walk or chown an existing store. **2. If your backend is MySQL or MariaDB, install the client library now.** The server opens it at run time and no package manager will pull it in: `libmariadb3` on Debian and Ubuntu, `mariadb-connector-c` on Fedora and RHEL, `mariadb-libs` on Arch. PostgreSQL needs nothing extra. **3. Create the database role.** Either let the role create its own database, or create an empty one yourself and leave the role without CREATEDB. [§35.4](https://www.progressiverobot.com/hmailserver-documentation/#354-creating-the-database) has both, and the SSL keys for a database on another host. **4. Edit `/etc/hmailserver/hMailServer.ini`.** Fill in `[Database]`. Two rules catch people. Only `;` starts a comment, and only at the start of a line. Text after a value on the same line is part of the value, so `Type=PostgreSQL ; the backend` matches no backend name. Write the port out: PostgreSQL wants 5432, MySQL and MariaDB want 3306. Leave `AdministratorPassword` alone. **5. Set the administrator password.** `sudo hmailserver --set-admin-password` reads it from standard input with echo off and writes the hash into the file. This is the one step that runs as root, because the file is `0640 root:hmailserver`. **6. Create the schema.** `sudo -u hmailserver hmailserver --create-database`. Then, as root, make the stored-secrets key that second factors and the `[Settings]` secrets are sealed under, and export it at once: `sudo hmailserver --create-stored-secrets-key`, then `sudo hmailserver --export-stored-secrets-key /root/hmailserver-stored-secrets.key`. Keep that file with the database backups, stored apart from them. Without a key those secrets are stored as written, as before 6.3.4. **7. Check it before starting anything.** `sudo -u hmailserver hmailserver --check-config`, as the service user and without `--config`, so the server has to find the configuration on its own. `Database type: 0` means the `Type` key did not take. It is a report and not a validator: it returns 0 unconditionally, so read the output rather than its exit status. **8. Start it.** `sudo systemctl start hmailserver`, then `journalctl -u hmailserver -f`. A freshly created database already holds four listeners: SMTP on 25 and 587, POP3 on 110 and IMAP on 143, bound to every address. Since 6.3.4 the unit's first start, if it comes within a day of `--create-database`, gives them a self-signed certificate and optional STARTTLS, and makes the Internet IP range require TLS for authentication ([SecureDefaults.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SecureDefaults.md)). Mail clients warn about a self-signed certificate, so replace it with ACME or your own before this faces the internet. The listeners and the certificate are database rows, not INI keys, so you change them over the API or the Control Deck's Ports and Certificates views. **9. Turn on administration.** `sudo hmailserver --set-setting RestApiPort=8080` (or another free port), leave `RestApiBindAddress` on loopback, and `sudo systemctl restart hmailserver`. Not by editing the file: since 6.3.4 the database is the settings store, and a `[Settings]` line edited after the first start is reported as `HM5804` and not applied: the service cannot write the packaged file, so the line stays as you left it and the stored value is used. TLS is required unless the bind address is exactly `127.0.0.1`, `localhost` or `::1`, so tunnel over SSH rather than exposing it. [§35.9](https://www.progressiverobot.com/hmailserver-documentation/#359-certificates-on-linux) and [§35.10](https://www.progressiverobot.com/hmailserver-documentation/#3510-administering-a-linux-server) are the rest. Post what `--check-config` printed and the last lines of `journalctl -u hmailserver` if a step stops. [§35.16](https://www.progressiverobot.com/hmailserver-documentation/#3516-troubleshooting-a-linux-installation) lists the Linux-specific failures by symptom. --- Canonical: https://www.hmailserver.co.uk/topic/28191/the-webmail-portal-at-portal # The webmail portal at /portal `/portal` is the self-service webmail hMailServer serves out of its REST listener. A mailbox owner signs in with their own address and password and gets folders, search, compose, drafts, flags and inline images. It has contacts too, and since 6.3.4 calendars and tasks and the organisation's directory. CardDAV (added in 6.3.3) and CalDAV (6.3.4), for other clients, are served on the web services HTTPS listener, not this one. **It is compiled into the binary.** There is no web root to deploy and no file to lose. At 6.3.4 the page is `Portal.html` and `Portal.js`, with its manifest, service worker, language catalogues and OpenPGP and S/MIME modules, all embedded at build time. So if the Control Deck answers and the portal does not, an absent file is not the cause. Check you are asking for `/portal` exactly, because the server answers the page at that path alone (what it serves below `/portal/` since 6.3.4 are other pages: single sign-on, password reset, password-protected messages), and check no proxy is rewriting it. [36.14](https://www.progressiverobot.com/hmailserver-documentation/#3614-troubleshooting) has the rest of the symptom table. **It fetches nothing externally.** No font, no image, no stylesheet, no script from a CDN. Its Content-Security-Policy is `default-src 'none'; script-src 'self'; style-src 'unsafe-inline'; img-src 'self' data: blob:; connect-src 'self'; frame-src 'self' blob:; manifest-src 'self'; worker-src 'self'; form-action 'none'; frame-ancestors 'none'; base-uri 'none'`. Note `img-src 'self' data: blob:`, which permits no image from any other host; `'self'` is there for images the server answers from its own routes, such as a colleague's directory photo. A sender's HTML is never merged into the page. It goes into an iframe sandboxed without `allow-scripts` and without `allow-same-origin`, under its own policy of `img-src data:`, so a remote tracking pixel stays in the markup and is blocked. Opening a message tells its sender nothing unless the reader chooses otherwise: showing remote content, for that message or always for that sender, serves the message under `img-src data: https: http:`, and the pixel then loads. [36.10](https://www.progressiverobot.com/hmailserver-documentation/#3610-how-inline-images-work-and-why) explains why the obvious design cannot work. **It is not an administration interface.** Everything the portal does with a mailbox is under `/api/v1/me/`, and those routes answer to an account's own credentials and to nothing else. The administrator password is refused on every one of them, and so is an API key. Administration is the other page, the Control Deck at `/`. Both sit on one listener behind one switch, `RestApiPort`, so exposing one exposes the other. If mailbox owners reach `/portal` from the internet, split the surfaces at a reverse proxy: the webmail public, `/api/v1/session` public because both pages sign in through it, and `/` plus the administration routes under `/api/v1/` on the internal network. At 6.3.4 the webmail is more than `/portal`, `/portal.js` and `/api/v1/me/*`: it loads `/portal-*.js`, `/portal.webmanifest`, `/portal-sw.js` and `/portal-lang/`, and, for the features you switch on, people who are not signed in reach pages and routes such as `/portal/*`, `/files/`, `/confidential/`, `/lists/`, `/held/`, `/recovery/verify`, `/api/v1/password-reset`, `/api/v1/passkeys/challenge` and `/api/v1/protected/`. Check the proxy's list against what you use. [36.13](https://www.progressiverobot.com/hmailserver-documentation/#3613-deploying-them-safely) **IMAP clients are unaffected.** The portal is another client against the same store. A flag change requires the rights `STORE` requires, and every IMAP session on the folder is told. A move is IMAP `MOVE`: a copy with a new UID, then the original expunged. Folder create, rename and delete run IMAP's checks in IMAP's order and refuse with IMAP's own sentences. The new-mail probe reads the cached per-folder collection that IMAP, POP3 and delivery already share, so polling opens no message file. Outlook, Thunderbird, Apple Mail and phones carry on unchanged, and Roundcube pointed at the IMAP port remains a reasonable choice. Full reference: [36.6 The portal: what a user gets](https://www.progressiverobot.com/hmailserver-documentation/#366-the-portal-what-a-user-gets). --- Canonical: https://www.hmailserver.co.uk/topic/28192/the-control-deck-browser-administration-at # The Control Deck: browser administration at / *Written for 6.3.4. hMailServer 6.3.7 (17 October 2026) refuses a state-changing REST request that carries a credential - HTTP Basic included - when the browser reports it as cross-site (`Sec-Fetch-Site`, or an `Origin` that is not this server's); the Deck and the webmail act on their own origin, and scripts, `hmctl` and the PowerShell module send no such header, so none of them is affected. A credential confined to named domains, a domain administrator among them, is refused a domain's rename, a domain alias, a custom role's creation, change or delete, a domain's or an account's rule's `bind_to_address`, `send_using_route` or `script_function` action, its domain's relay host changed to an address that is not public and the server-wide metric history, over REST as over COM, and so from the Deck. 6.3.7 also carries the first step of the Control Deck's and the webmail's rebuild planned for 6.4 - the framework decision (ControlDeckFramework.md), the threat model of the web surface and the authorisation matrix with its check - with no product code. See [the 6.3.7 announcement](/topic/28277).* The Control Deck is the administration page hMailServer serves at `/`, on the same listener as the API and the portal. On Linux it is the only administration front end there is, because that build has no Control Panel and no COM. On Windows it does not replace the Control Panel and it does less. **Turning it on.** One setting, `RestApiPort`, turns on the API, the Deck and the portal together. The default is 0, which leaves all three off, so a server upgraded without touching its settings serves none of them. The listener refuses to start rather than come up without a credential or without TLS, and logs which. Four messages begin `RestApi: Refusing to start`. The two you will meet are an unset administrator password and a non-loopback bind with no certificate. The TLS exemption applies only to a bind address that is exactly `127.0.0.1`, `localhost` or `::1`. [36.2](https://www.progressiverobot.com/hmailserver-documentation/#362-turning-them-on) **Signing in.** The password goes once to `POST /api/v1/session` and is exchanged for an `hmailsession` cookie that is `HttpOnly` and `SameSite=Strict`. Two ceilings apply, 30 minutes idle and 12 hours absolute. Since 6.3.4 those are the defaults: `SessionIdleSeconds` and `SessionLifetimeSeconds`, server-wide or per domain, change them, and neither is set as shipped, after an upgrade or in a new database ([DevicesAndSessions.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/DevicesAndSessions.md)). Any cookie-authenticated request whose method is not `GET` or `HEAD` must carry `X-Requested-With: hMailServer` or it answers 403; HTTP Basic and bearer keys are exempt. Since 6.3.7 a state-changing request that carries any credential is refused when the browser reports it as cross-site (`Sec-Fetch-Site`, or an `Origin` that is not this server's); scripts, `hmctl` and the PowerShell module send no such header and are unaffected, as are the Deck and the webmail, which act on their own origin. An API key cannot mint a session. Since 6.3.4 an account that holds an administrative role signs in with its own address and password, and a passkey can start the same session, as can the organisation's single sign-on for the addresses named in `oidc_administrators` or `saml_administrators` (both empty as shipped). Sessions are process-local, so a service restart ends every one. [36.3](https://www.progressiverobot.com/hmailserver-documentation/#363-signing-in-to-the-control-deck) **What it does.** Dashboard, Domains, Delivery queue, DANE/TLSA, Settings, Rules, Routes, Certificates, Ports and Logs. 6.3.3 added full domain editing, the account editor in full, distribution lists and aliases, an IP-ranges view, fetch-account and backup views, the scripting, cache and indexing groups, and a CI harness of 292 checks. At 6.3.4 it has 40 views, in twenty languages besides English. The settings forms are worth understanding. The Deck fetches `GET /api/v1/openapi.json` once per session and draws each form from the schema of that group's `PUT`. Nothing in the page lists a settings key by name, so a key added to the API appears on its own, with its type, its permitted words, and a badge when it is read-only, write-only, required, or takes effect only on restart; since 6.3.4 a key kept in the settings store says instead whether a save applies it at once, after a reinitialise or after a restart. The Ports view carries a Restart the services now button, which posts to `/api/v1/server/reinitialize`. It drops connections in progress and ends your session with everyone else's. [36.4](https://www.progressiverobot.com/hmailserver-documentation/#364-what-the-control-deck-can-do) **What the Deck cannot write is measured, not asserted.** `build/check-deck-parity.py` counts every field the desktop Control Panel writes against the REST API and the Deck, and reports in `hmailserver/docs/DeckParity.md`. Of 330 properties, 240 were writable over REST and 153 reachable from a Deck view when that work started; 328 and 322 at 6.3.3. At 6.3.4 the census counts 466, finds 458 of them writable over REST and every one of those reachable from a Deck view, and lists eight as missing: an account's directory fields (title, department, phone, mobile, office, note, photo and the opt-out). Those are written too, by `PUT /api/v1/accounts/{address}/directory` and the directory card of the Deck's account editor, under names the census does not match. [36.5](https://www.progressiverobot.com/hmailserver-documentation/#365-what-the-control-deck-cannot-do-yet) now opens with the 6.3.4 census and keeps its description of the 6.3.1 gaps, which were far wider, below it for the record; the 6.3.4 section of [CHANGELOG.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/CHANGELOG.md) is the current statement. Check `DeckParity.md` in the [source tree](https://gitlab.com/hmailserver/hmailserver) before assuming a route does not exist. --- Canonical: https://www.hmailserver.co.uk/topic/28187/let-s-encrypt-certificates-are-automatic-the-built-in-acme-v2-client # Let's Encrypt certificates are automatic: the built-in ACME v2 client *Written for 6.3.4. hMailServer 6.3.7 (17 October 2026) fixes the ACME HTTP-01 challenge responder, which served one connection at a time, so that idle clients could make a certificate's issuance or renewal fail in every earlier version (work item 97): it now serves up to 64 connections at once and closes one past that, though a client that holds 64 idle connections open through a whole validation can still make it fail. In 6.3.7's Control Panel the SSL certificates page is TLS certificates, and Ctrl+K takes the old title to it. See [the 6.3.7 announcement](/topic/28277).* hMailServer has an ACME v2 client built in (RFC 8555). It obtains a free certificate, installs it, assigns it to your TLS ports, renews it before expiry and reloads it. No restart, no scheduled task, no external client. Configure it under **TLS & certificates → Certificates (ACME)**, over REST with `PUT /api/v1/settings/ini/{name}`, or on Linux with `sudo hmailserver --set-setting NAME=VALUE`. Since 6.3.4 these are database settings, not lines to edit in `hMailServer.INI`: a line edited there is named in the error log (`HM5804`) and not used, and is put back where the service may write the file. The values: ``` AcmeEnabled=1 AcmeContactEmail=you@yourcompany.com AcmeDomains=mail.yourcompany.com,mta-sts.yourcompany.com,autoconfig.yourcompany.com AcmeHttpPort=80 AcmeReuseKey=1 ``` `AcmeEnabled` ships as `0`, and a change to it takes effect at a reinitialise (**Apply now**), not at the save. `AcmeDirectoryUrl` defaults to `https://acme-v02.api.letsencrypt.org/directory`. **Two requirements.** Port 80 must be reachable from the internet, because that is how the http-01 challenge proves you control the name. Every name in `AcmeDomains` must already resolve to this server before you switch it on. If you also host MTA-STS or client autoconfiguration, put `mta-sts.`, `autoconfig.` and `autodiscover.` in `AcmeDomains` so one certificate covers them. **After issuance.** The server creates or updates a certificate record named `ACME (automatic)` and assigns it to every TLS-enabled listener that has none of its own, and since 6.3.4 to every listener still holding the self-signed certificate a new installation is given, logging each assignment. A listener the assignment could not be saved for is reported as a High 6100 rather than logged as a success, because a port left without a certificate accepts no TLS at all. That is the error to alert on. **Renewal.** The renewal window is two thirds of the certificate lifetime, plus ARI. That is why there is nothing left to schedule. Files live in `Data\ACME` on Windows and `/var/lib/hmailserver/ACME/` on Linux: the account key, `fullchain.pem` and `privkey.pem`. Back that directory up with the rest of `Data\`. If a restore misses it the client re-issues, but the new key invalidates any TLSA records you have published. Leave `AcmeReuseKey=1`. Keeping the same private key across renewals is what keeps published TLSA records valid. **Linux.** Port 80 works without root. `CAP_NET_BIND_SERVICE` on the unit is what allows the bind. The default certificate directory is `/ACME`, which is inside `ReadWritePaths`. Point `AcmeCertificateDirectory` elsewhere and you must add that path to `ReadWritePaths` with a drop-in, or issuance fails on a read-only filesystem. **If it never worked for you.** The built-in client had never once succeeded against real Let's Encrypt until 6.2.24 (GitHub issue 34). Boulder pretty-prints its JSON and the challenge locator searched for the compact spelling. 6.2.25 then fixed issuance and renewal ending the process (GitHub issue 93). If you gave up on this on an older build, it is worth another look. **Another CA.** Leave `AcmeEnabled=0`, add the PEM certificate and key under TLS & certificates → SSL certificates, then assign it to ports under Connections & protocols → TCP/IP ports. On Linux, an external client that replaces the PEM files in place needs `POST /api/v1/server/reinitialize` or a restart before the new files are served. Chapter 9, encryption and certificates: https://www.progressiverobot.com/hmailserver-documentation/#9-encryption-and-certificates Report problems: https://gitlab.com/hmailserver/hmailserver, or without a GitLab account by email to the project's Service Desk address, [contact-project+hmailserver-hmailserver-86730559-issue-@incoming.gitlab.com](mailto:contact-project+hmailserver-hmailserver-86730559-issue-@incoming.gitlab.com) --- Canonical: https://www.hmailserver.co.uk/topic/28188/being-trusted-by-the-servers-you-send-to # Being trusted by the servers you send to *Written for 6.3.4. hMailServer 6.3.7 (17 October 2026) signs a message as a hosted domain only when the server is entitled to: mail from outside whose From names a domain hosted here is no longer signed with that domain's DKIM key, and an authenticated submission is signed as another hosted domain only when its account may send as the address its From names (CVE-2026-107567); the upgrade stops the signing, and only rotating the domain's key ends the signatures already made. A forward of outside mail whose From names the forwarding domain's own organisation is given an SRS or rewritten envelope sender only when the message passed DMARC for it on arrival; with SRS on, mail to an SRS address this server handed out is held to the relay rules unless it is a bounce; and an ARC seal this server makes attests its own SPF result and names the domains its results are for. See [the 6.3.7 announcement](/topic/28277).* Getting mail accepted is mostly DNS. hMailServer does the cryptography and the protocol work. It cannot publish records for you. This is the split. ## What the server does on its own **DKIM signing.** Open the domain's DKIM tab in the Control Panel and click Generate key. It creates an RSA-2048 key, saves the private key, fills in the path and shows the exact TXT record with a Copy button. Once the record is published it signs the domain's outgoing mail - a signed-in account's submission and every copy of it, a message the server makes itself, and mail an IP range lets in as one of this server's domains - but since 6.3.7 not mail from outside whose From names the domain: such mail leaves with the signature it came with. Since 6.3.7 an authenticated submission is signed as another hosted domain only when its account may send as the address its From names (CVE-2026-107567). Ed25519 DKIM (RFC 8463) is supported alongside RSA, for shorter keys. Since 6.3.4 a domain's DKIM keys can also rotate on a policy, off by default. `DkimOversignHeaders` over-signs the header fields you name. `DkimAcceptSha1=0` refuses rsa-sha1 signatures on inbound mail. **SPF and DMARC inbound.** SPF is checked as part of anti-spam scoring, with `SpfVoidLookupLimit=2`. Since 6.3.4 it follows RFC 7208: mail from 127.0.0.0/8 and `::1` no longer passes unconditionally, so a content filter that hands mail back should authenticate, or connect from an IP range with anti-spam off. DMARC policy is evaluated on inbound mail and the DMARCbis tree walk is on by default (`DmarcTreeWalkEnabled=1`). Aggregate reporting stays off until you set `DmarcRptFromAddress`. **Outbound transport security.** `MtaStsEnabled=1`, `DaneEnforcementEnabled=1` and `DnssecValidationEnabled=1` are all on by default, under TLS & certificates → Transport security. A recipient whose DNSSEC chain is bogus does not receive your mail rather than receiving it unencrypted. That is the point of enforcement, so read the logs before you reach for the switch. **TLS-RPT.** Off until you set `TlsRptFromAddress`. After that the server sends daily reports to recipient domains about TLS failures it hit delivering to them. **SRS and BATV.** Off by default. If you forward mail, set `SRSEnabled=1` and a `SRSSecret` to keep forwarded mail SPF-aligned. The secret must be stable. Outstanding SRS addresses remain valid for 21 days, so changing it loses bounces for mail you forwarded recently. BATV (prvs) signs your envelope sender so you can recognise and drop backscatter. Neither ever became an RFC. Both follow the widely deployed drafts, as does everything that interoperates with them. ## What you must publish yourself PTR, and your ISP sets that one, not you. Missing PTR is the single biggest cause of spam-foldering. Then SPF, the DKIM TXT record, and DMARC. For MTA-STS on the inbound side, point `mta-sts.` at the server, set `MtaStsHostingEnabled=1` and `WebServicesHttpsPort=443`, and publish the `_mta-sts.` TXT record. If you publish TLSA records for inbound DANE, keep `AcmeReuseKey=1` so renewals do not invalidate them. Start DMARC at `p=none`. Read the reports for a few weeks and confirm all your legitimate mail passes SPF and DKIM before moving to `p=quarantine`, then `p=reject`. Starting at `p=reject` silently loses mail from the sender you had forgotten about. ## Still incomplete ARC sealing (RFC 8617) covers mail from hosted DKIM-enabled domains. Relayed third-party mail is not sealed yet. Chapter 8, the DNS records with example syntax: https://www.progressiverobot.com/hmailserver-documentation/#8-making-the-internet-trust-you-dns Chapter 9.4, outbound transport security: https://www.progressiverobot.com/hmailserver-documentation/#94-outbound-transport-security Chapter 17.6, SRS and BATV: https://www.progressiverobot.com/hmailserver-documentation/#176-srs-and-batv --- Canonical: https://www.hmailserver.co.uk/topic/28219/diagnosing-stalled-mail-a-sender-that-times-out-and-a-message-that-never-leaves # Diagnosing stalled mail: a sender that times out, and a message that never leaves *Written for 6.3.4. hMailServer 6.3.7 (17 October 2026) adds reasons a message stays where it is. A forward, a redirect or a rule's forward of mail from outside whose From names the forwarding domain's own organisation, and that did not pass DMARC for it when it arrived, keeps the envelope sender it came with, with no SRS address or rewritten sender; where the copy would still carry an identity of that organisation (an empty envelope sender under this server's host name, or an envelope sender at this server's host name or at a hosted domain in that organisation) it is not sent to an outside address at all: the message stays in the mailbox, and the application log names each recipient. `RuleLoopLimit` 0 now stops out-of-office replies, as it stopped every other generated message. A delivery whose mail servers' DNS lookup failed tells the sender that the DNS query failed (4.4.4) and retries, where a failed lookup was read as a name with no addresses. On MySQL and MariaDB a lost database connection is reconnected and the statement run again, and an error log that was quiet may now name database failures that were hidden (`HM5032`). See [the 6.3.7 announcement](/topic/28277).* This topic covers the case where mail is not moving and the server looks healthy: no crash, no error, the service running, and a log that seems to stop mid-transaction. The server used to be silent about it; it is no longer, and the answer is usually in one line of the log. The one thing to establish first is **which half is stuck**: the server *accepting* a message from a sender, or *delivering* one it has already accepted. The two fail differently, for different causes, under different settings. ### Which half is stuck? **Accepting.** The sending server connects, sends the message, and then waits. From its side there is a timeout *after* the message body was transmitted; Postfix reports it as `timed out while sending end of data`. The message never appears in your queue. **Delivering.** The message was accepted (the sender got a `250`), is visible in the delivery queue, and never leaves. **Monitoring & troubleshooting → Delivery queue** in the Control Panel tells the two apart in one look, and so does `hmailserver_delivery_queue_messages` on the metrics endpoint once it has a port (see Commands). The Control Panel is Windows-only; on Linux the queue is `GET /api/v1/queue`, `hmctl queue list` (since 6.3.4) and the Control Deck's queue page. ### Turn on debug logging first Control Panel → **Logging** → tick **Debug messages**, and make sure logging is enabled; then reproduce the problem once. The stage lines below are written at debug level. The *slow* ones (ten seconds or more, for a whole stage or for any single spam test) are also written at application level, so a server with only application logging still records the important half. Turn debug logging off again afterwards on a busy server. [Reading the logs](/topic/28220) covers where the logging keys live on each platform. ### Accepting: the sender times out after sending the message Between the `354` and the `250` the server runs the accept pipeline: spam tests, message modifications, archiving, the `OnAcceptMessage` script and the database save. All of it runs on a bounded pool of threads, and the reply is sent only when it finishes. The log times each stage: ``` SMTPConnection - accept: start spam-protection. Spam test: SpamTestDNSBlackLists, Score: 0, Time: 12 ms Spam test: SpamTestSpamAssassin, Score: 0, Time: 120000 ms SMTPConnection - accept: done spam-protection in 120047 ms (session 5). SMTPConnection - accept: done message-modifications in 0 ms (session 5). SMTPConnection - accept: start script/save. SMTPConnection - accept: done script/save in 3 ms (session 5). ``` **How to read it.** Read it as a sequence and look for the last line written; the stage that is stuck is the one after it. Two stages announce themselves with a `start` line (`spam-protection` and `script/save`), so for those a `start` with no matching `done` is the whole diagnosis. The middle stage, message modifications, has no `start` line, so a stall there shows as `done spam-protection` followed by silence. If every stage completes but one took a long time, the `Time:` or `done ... in` value names the culprit. | What the log shows | Cause | |---|---| | `SpamTestSpamAssassin` with a large `Time:` | spamd is unreachable, overloaded, or accepting connections without answering | | `SpamTestDNSBlackLists`, `SpamTestSURBL` or `SpamTestSPF` slow | The resolver is not answering. Look in the TCP/IP log for `DNS - Query timed out` and `DNS - Query failure`. Do not reach for `DNSServer` as the fix (below) | | `done spam-protection`, then silence | Message modifications: the spam headers, the signature, the List-* headers, or the write of the modified message back to disk | | `script/save` slow | An `OnAcceptMessage` event script, or the database | | Nothing between `354` and silence | A version older than 6.2.17, which added these lines. Upgrade | **The `DNSServer` setting.** It was broken in 6.2.17 and 6.2.18 (CHANGELOG's 6.2.19 section names 6.2.16 as well) and fixed in 6.2.19: on the affected versions every lookup through a configured custom server failed, and DNSBL, SURBL, SPF, DKIM and MX lookups failed in silence, so such a server had quietly stopped most of its spam filtering while still accepting mail. One caveat survives the fix: a custom server list is only honoured together with `DNS_QUERY_BYPASS_CACHE`, so every lookup pays full price. Leave `DNSServer` empty unless you need a specific resolver. **SPF, and SPF on Linux.** In 6.3.3 and earlier the SPF evaluator resolved names through the Windows DNS client rather than through the server's resolver. On Windows an SPF check therefore went to the system resolvers whatever `DNSServer` said; on Linux every SPF check returned a temporary error, reported once per process as `HM6406`, and because `SpamTestDMARC` evaluates SPF, DMARC there lost its SPF half. The RFC 7208 evaluator that replaced it in 6.3.4 resolves through the same resolver as every other lookup. If you run 6.3.3 or earlier on Linux, an SPF or DMARC verdict from it is not a verdict. **The deadline.** Since 6.2.17 acceptance is bounded: if the pipeline exceeds `FinalizationTimeout` (240 seconds by default) the server answers `451 4.3.1 Server temporarily overloaded while accepting the message; please retry.` and the sender retries rather than waiting for a reply that never comes. The accompanying error entry names the deadline that was hit, and the stage timings above it name what consumed the time. ### If every message stalls, not just one Look for these two lines: ``` Task SMTP-accept session=42 ip=203.0.113.9 waited 8 seconds for a thread in work queue Asynchronous task queue. 14 task(s) are still queued. ``` ``` All 15 threads in work queue Asynchronous task queue have been busy for at least 120 seconds, so no further task on this queue can start. 14 task(s) are queued. Running: SMTP-accept session=42 ip=203.0.113.9 (thread 4120, 168s), ... ``` Every worker is occupied, and one slow dependency does this to the whole server. The task names say which sessions are stuck; the stage timings say what they are stuck on. `AsyncQueueStallThreshold` (120 seconds by default) is how long every worker may be busy before the second report is written; `0` turns the report off. ### Delivering: how the queue works There is no in-memory queue to inspect. The delivery queue *is* the set of rows in `hm_messages` whose type is delivering, and the delivery manager is a loop that polls for them. Three properties of that loop explain most of what a stuck queue looks like: - **The selection is `where messagetype = 1 and messagelocked = 0 and messagenexttrytime <= now`, ordered by size, then try count, then id.** Small and new goes first. - **The delivery pool is separate and smaller than the accept pool**: `maxdelivertythreads`, 10 by default, on the work queue named `SMTP delivery queue`. It is not the `Asynchronous task queue` that finalises inbound mail, so a saturated delivery pool does not stop mail being accepted; it stops it leaving. - **The manager wakes on demand, or once a minute otherwise.** A message whose next-try time has just passed is picked up within a minute, not instantly. At startup every message in the queue is unlocked (`update hm_messages set messagelocked = 0`), because locks held by a server that crashed would otherwise strand those messages for ever. That is also why exactly one hMailServer service may run against one database. ### The retry schedule, and when a message gives up | Setting | Where | Default | Effect | |---|---|---|---| | Number of retries | **Mail flow & delivery → Delivery of e-mail** (`smtpnoofretries`) | `4` | How many *further* attempts after the first | | Minutes between every retry | The same page (`smtpminutesbetweenretries`) | `60` | The gap between them | | `QuickRetries` | `[Settings]` | `0` (off) | The first N retries use the short gap below instead, for greylisting | | `QuickRetriesMinutes` | `[Settings]` | `6` | That short gap | | `QueueRandomnessMinutes` | `[Settings]` | `0` (off) | Adds a random 1..N minutes to every gap, so a burst does not retry in lockstep | | Route *number of tries* and *minutes between tries* | **Mail flow & delivery → Routes** | — | Overrides both, per matching route; where several routes match, the largest of each wins | With the shipped defaults that is five attempts an hour apart: a destination that is refusing temporarily holds a message for roughly four hours and then bounces it. The try count in the queue is the same `messagecurnooftries` the arithmetic uses. Two things do not advance that counter: *Deliver now* resets the next-try time without touching the count, and a message held because the virus scanner could not examine it is held on its own budget (below). ### Reading a stuck queue | What the queue looks like | What it means | Where to look next | |---|---|---| | Depth climbing, oldest message seconds old | A busy server draining normally. Not a fault | — | | Depth flat, oldest message hours old, try counts climbing | The far end is deferring you | The SMTP log for that destination; your IP's reputation and PTR | | Every stuck message shares one recipient domain | That destination, not your server | The route settings for it, and its MX | | Try counts stuck at `0`, next-try times in the past, nothing moving | Delivery threads are not running the tasks | `hmailserver_workqueue_depth{queue="SMTP delivery queue"}`, and the saturation report above | | Messages appear and vanish without arriving | Delivered to the wrong place, or discarded by a rule | **Mail flow & delivery → Rules**; enable `MessageTraceEnabled` and read the trace | | Try count `1` and a next-try time of `1901-01-01` | An ETRN hold: the one route these recipients match is an ETRN route | **Mail flow & delivery → Routes**; the log line is `SMTPDeliverer - Route Message: HOLD for later delivery..` | | Queue empty, but the sender got a `250` and nobody received it | Delivered locally into a folder you are not looking at, or the row was removed | The account's folders; the message trace | | `Message N could not be delivered since the data file does not exist` | The row and the store disagree: a restore that missed files, or a store on a share that vanished | Turn on `MessageStoreConsistencyCheck` and read the report | **When the virus scanner cannot examine a message.** With `AVFailAction=1` such a message is not delivered. It is held on its own schedule rather than on the SMTP retry budget: `AVFailRetryMinutes` apart (15 by default), at most `AVFailMaxHolds` times (16), and then returned to the sender. The log says where in that budget each hold is (`... this was hold 3 of 16 before it is returned to the sender.`). Each hold counts as a deferral in `hmailserver_messages_deferred_total`, so a scanner that has stopped answering shows as a deferral rate with no matching delivery rate. ### The usual causes of a genuinely stuck delivery - **A virus scanner that stops responding.** ClamAV is contacted after the message is accepted, so a wedged clamd shows up as accepted but never delivered. Each socket operation is bounded by `ClamMinTimeout` / `ClamMaxTimeout`, but that is a deadline on *one* read or write, armed fresh each time, and the message is streamed in chunks, so one large message can take considerably longer than `ClamMaxTimeout` in total. - **A remote server that answers extremely slowly.** The idle timeout is re-armed on every byte received, so a host that sends one byte occasionally used to hold a delivery thread indefinitely. Since 6.2.18 there is an absolute ceiling, `ClientSessionCeiling` (30 minutes by default), armed once and never re-armed. - **A custom virus scanner or external tool that hangs**, bounded by `ExternalProcessTimeout`. - **A message held for a sandbox's verdict** (since 6.3.4). With `SandboxEnabled` on, a message none of your own signed-in users submitted, whose attachment is of a submitted type, waits for the sandbox you named to open the file, up to `SandboxTimeoutMinutes` (15 by default). While it waits it is in no mailbox, not in the quarantine and not the queue's to retry: its Retry and Delete answer 409. It is listed by `GET /api/v1/sandbox/held`, by `GET /api/v1/queue` marked `held: sandbox` (the `held` column of `hmctl queue list`), and in the Deck's queue page under *Held for a sandbox*, where it can be delivered without the verdict or discarded. - **The database.** Check for errors mentioning the connection pool. Since 6.3.4 a message can also wait in other places, each on purpose: held for review by compromised-account detection in `enforce` (`/api/v1/review`); in a destination's back-off window, with the last error `Delivery deferred by this server's own shaping, not by a remote server.` (`GET /api/v1/queue/shaping`; such a deferral does not spend the message's retries); under an IP warm-up ceiling; or behind a drain, when `/readyz` says `draining`. Section 19.11a of the Troubleshooting page lists each with what to do. ### Settings that bound each stage All are in seconds. On 6.3.3 and earlier they are `[Settings]` keys in `hMailServer.ini`. Since 6.3.4 they are stored in the database and changed in the Control Panel, over the REST API at `/api/v1/settings/ini/` or with `hmctl`, and the save applies them. They still appear in the file's `[Settings]` section, but the database's value is the one used: where the service can write the file, an edit made there is reported (HM5804) and put back at the next start. | Setting | Default | Bounds | `0` means | |---|---|---|---| | `FinalizationTimeout` | 240 | The whole accept pipeline, after which the sender gets a `451` | no bound | | `SAMaxTimeout` | 90 | SpamAssassin: idle timeout, and a session ceiling of this plus 30s | **not** "no bound" | | `ClamMaxTimeout` | 90 | ClamAV: idle timeout per socket operation | **not** "no bound" | | `DNSQueryTimeout` | 10 | A single DNS query | no bound | | `ScriptTimeout` | 60 | One event script invocation | no bound | | `ExternalProcessTimeout` | 300 | An external scanner process | no bound | | `ClientSessionCeiling` | 1800 | An entire outbound delivery session | no bound | | `AsyncQueueStallThreshold` | 120 | How long every worker may be busy before the saturation report | reporting off | | `DBConnectionAcquireTimeout` | 60 | Waiting for a pooled database connection | no bound | **The two exceptions matter, because `0` tightens them instead of removing them.** `SAMaxTimeout` and `ClamMaxTimeout` are not passed straight through: they go to a calculator that returns the *minimum* whenever the maximum is lower than it. So `SAMaxTimeout=0` gives a 30-second idle timeout (`SAMinTimeout`) and a 30-second session ceiling, and `ClamMaxTimeout=0` gives 15 seconds (`ClamMinTimeout`). To lengthen either, raise the value; to lengthen it a long way, raise the matching `...MinTimeout` too. There is no way to make either unbounded. When the pool deadline expires, a recipient lookup answers `451 4.3.2` rather than `550`: the server can tell "the database did not answer" from "no such address", so a database locked by a backup defers the mail instead of bouncing it. ### Commands The COM examples below are Windows-only. The metrics endpoint and the REST routes work on both platforms. **Is it accepting, or is it delivering?** The metrics listener is off until `MetricsServerPort` is set (9100 here; since 6.3.4 the port, the bind address and the listener's other settings, its credentials included, wait for a reinitialise). Once it is on, the endpoint needs no credential on a loopback bind: ```powershell (Invoke-RestMethod http://127.0.0.1:9100/metrics) -split "`n" | Select-String '^hmailserver_delivery_queue_(messages|oldest_message_age_seconds) |^hmailserver_workqueue_depth|^hmailserver_messages_(delivered|deferred|bounced)_total ' ``` A rising `oldest_message_age_seconds` with a flat `delivered_total` is the delivering half. A rising `hmailserver_workqueue_depth{queue="Asynchronous task queue"}` is the accepting half. The COM examples start from a signed-in `$app`: ```powershell $app = New-Object -ComObject hMailServer.Application $app.Authenticate("Administrator", "") | Out-Null ``` **Push one message to the front of the queue** (resets its next-try time, leaves the try count alone, and releases an ETRN hold): ```powershell $q = $app.GlobalObjects.DeliveryQueue $q.ResetDeliveryTime(4711) $q.StartDelivery() ``` **Wake the delivery manager without touching any message:** ```powershell $app.GlobalObjects.DeliveryQueue.StartDelivery() ``` **Remove one message from the queue** (it is locked first, so a message a thread already has is not pulled from under it): ```powershell $app.GlobalObjects.DeliveryQueue.Remove(4711) ``` **Follow the log while you reproduce it** by tailing the day's file and filtering on `accept: |Spam test:|waited .* for a thread|have been busy|SMTPDeliverer`; [Reading the logs](/topic/28220) has the tail command for each platform. **Turn the message trace on** (`MessageTraceEnabled`, off by default) to see what was done with one message. It has no acceptance, spam or virus event, so an empty trace does not show that the server never had the message; [Troubleshooting by symptom](/topic/28225) says how to turn it on and what a row says. ### What to include in a report If none of the above identifies it, post in [General support](/category/8) or open an issue with: the log from the `354` (or from the delivery attempt) onwards, including the stage timing lines; the `ERROR_hmailserver_.log` for the same window; which scanners and event scripts are enabled; and whether the message eventually arrives, arrives twice, or never arrives. Say which half you have. [Read this before you post](/topic/28185) covers the rest. Without a GitLab account, email the report to the project's Service Desk address, contact-project+hmailserver-hmailserver-86730559-issue-@incoming.gitlab.com, which GitLab files as a confidential issue; the maintainers answer by email. ### Where to read more - Wiki: [Diagnosing Stalled Mail](https://gitlab.com/hmailserver/hmailserver/-/wikis/Diagnosing-Stalled-Mail) and [Troubleshooting](https://gitlab.com/hmailserver/hmailserver/-/wikis/Troubleshooting), sections 19.7 and 19.11a. - Repository: [DiagnosingStalledMail.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/DiagnosingStalledMail.md); [SettingsApply.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SettingsApply.md); [Sandbox.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Sandbox.md); [SUPPORT.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/SUPPORT.md). - The public documentation: [The delivery queue and the retry schedule](https://www.progressiverobot.com/hmailserver-documentation/#267-the-delivery-queue-and-the-retry-schedule) and [Troubleshooting](https://www.progressiverobot.com/hmailserver-documentation/#19-troubleshooting). - On this forum: [Being trusted by the servers you send to](/topic/28188), for when the far end is deferring you; [Outbound shaping and IP warm-up](/topic/28259), for when your own server is holding mail back; [Monitoring and health](/topic/28272), for turning the metrics listener on. --- Canonical: https://www.hmailserver.co.uk/topic/28219/diagnosing-stalled-mail-a-sender-that-times-out-and-a-message-that-never-leaves # Diagnosing stalled mail: a sender that times out, and a message that never leaves *Written for 6.3.4. hMailServer 6.3.7 (17 October 2026) adds reasons a message stays where it is. A forward, a redirect or a rule's forward of mail from outside whose From names the forwarding domain's own organisation, and that did not pass DMARC for it when it arrived, keeps the envelope sender it came with, with no SRS address or rewritten sender; where the copy would still carry an identity of that organisation (an empty envelope sender under this server's host name, or an envelope sender at this server's host name or at a hosted domain in that organisation) it is not sent to an outside address at all: the message stays in the mailbox, and the application log names each recipient. `RuleLoopLimit` 0 now stops out-of-office replies, as it stopped every other generated message. A delivery whose mail servers' DNS lookup failed tells the sender that the DNS query failed (4.4.4) and retries, where a failed lookup was read as a name with no addresses. On MySQL and MariaDB a lost database connection is reconnected and the statement run again, and an error log that was quiet may now name database failures that were hidden (`HM5032`). See [the 6.3.7 announcement](/topic/28277).* This topic covers the case where mail is not moving and the server looks healthy: no crash, no error, the service running, and a log that seems to stop mid-transaction. The server used to be silent about it; it is no longer, and the answer is usually in one line of the log. The one thing to establish first is **which half is stuck**: the server *accepting* a message from a sender, or *delivering* one it has already accepted. The two fail differently, for different causes, under different settings. ### Which half is stuck? **Accepting.** The sending server connects, sends the message, and then waits. From its side there is a timeout *after* the message body was transmitted; Postfix reports it as `timed out while sending end of data`. The message never appears in your queue. **Delivering.** The message was accepted (the sender got a `250`), is visible in the delivery queue, and never leaves. **Monitoring & troubleshooting → Delivery queue** in the Control Panel tells the two apart in one look, and so does `hmailserver_delivery_queue_messages` on the metrics endpoint once it has a port (see Commands). The Control Panel is Windows-only; on Linux the queue is `GET /api/v1/queue`, `hmctl queue list` (since 6.3.4) and the Control Deck's queue page. ### Turn on debug logging first Control Panel → **Logging** → tick **Debug messages**, and make sure logging is enabled; then reproduce the problem once. The stage lines below are written at debug level. The *slow* ones (ten seconds or more, for a whole stage or for any single spam test) are also written at application level, so a server with only application logging still records the important half. Turn debug logging off again afterwards on a busy server. [Reading the logs](/topic/28220) covers where the logging keys live on each platform. ### Accepting: the sender times out after sending the message Between the `354` and the `250` the server runs the accept pipeline: spam tests, message modifications, archiving, the `OnAcceptMessage` script and the database save. All of it runs on a bounded pool of threads, and the reply is sent only when it finishes. The log times each stage: ``` SMTPConnection - accept: start spam-protection. Spam test: SpamTestDNSBlackLists, Score: 0, Time: 12 ms Spam test: SpamTestSpamAssassin, Score: 0, Time: 120000 ms SMTPConnection - accept: done spam-protection in 120047 ms (session 5). SMTPConnection - accept: done message-modifications in 0 ms (session 5). SMTPConnection - accept: start script/save. SMTPConnection - accept: done script/save in 3 ms (session 5). ``` **How to read it.** Read it as a sequence and look for the last line written; the stage that is stuck is the one after it. Two stages announce themselves with a `start` line (`spam-protection` and `script/save`), so for those a `start` with no matching `done` is the whole diagnosis. The middle stage, message modifications, has no `start` line, so a stall there shows as `done spam-protection` followed by silence. If every stage completes but one took a long time, the `Time:` or `done ... in` value names the culprit. | What the log shows | Cause | |---|---| | `SpamTestSpamAssassin` with a large `Time:` | spamd is unreachable, overloaded, or accepting connections without answering | | `SpamTestDNSBlackLists`, `SpamTestSURBL` or `SpamTestSPF` slow | The resolver is not answering. Look in the TCP/IP log for `DNS - Query timed out` and `DNS - Query failure`. Do not reach for `DNSServer` as the fix (below) | | `done spam-protection`, then silence | Message modifications: the spam headers, the signature, the List-* headers, or the write of the modified message back to disk | | `script/save` slow | An `OnAcceptMessage` event script, or the database | | Nothing between `354` and silence | A version older than 6.2.17, which added these lines. Upgrade | **The `DNSServer` setting.** It was broken in 6.2.17 and 6.2.18 (CHANGELOG's 6.2.19 section names 6.2.16 as well) and fixed in 6.2.19: on the affected versions every lookup through a configured custom server failed, and DNSBL, SURBL, SPF, DKIM and MX lookups failed in silence, so such a server had quietly stopped most of its spam filtering while still accepting mail. One caveat survives the fix: a custom server list is only honoured together with `DNS_QUERY_BYPASS_CACHE`, so every lookup pays full price. Leave `DNSServer` empty unless you need a specific resolver. **SPF, and SPF on Linux.** In 6.3.3 and earlier the SPF evaluator resolved names through the Windows DNS client rather than through the server's resolver. On Windows an SPF check therefore went to the system resolvers whatever `DNSServer` said; on Linux every SPF check returned a temporary error, reported once per process as `HM6406`, and because `SpamTestDMARC` evaluates SPF, DMARC there lost its SPF half. The RFC 7208 evaluator that replaced it in 6.3.4 resolves through the same resolver as every other lookup. If you run 6.3.3 or earlier on Linux, an SPF or DMARC verdict from it is not a verdict. **The deadline.** Since 6.2.17 acceptance is bounded: if the pipeline exceeds `FinalizationTimeout` (240 seconds by default) the server answers `451 4.3.1 Server temporarily overloaded while accepting the message; please retry.` and the sender retries rather than waiting for a reply that never comes. The accompanying error entry names the deadline that was hit, and the stage timings above it name what consumed the time. ### If every message stalls, not just one Look for these two lines: ``` Task SMTP-accept session=42 ip=203.0.113.9 waited 8 seconds for a thread in work queue Asynchronous task queue. 14 task(s) are still queued. ``` ``` All 15 threads in work queue Asynchronous task queue have been busy for at least 120 seconds, so no further task on this queue can start. 14 task(s) are queued. Running: SMTP-accept session=42 ip=203.0.113.9 (thread 4120, 168s), ... ``` Every worker is occupied, and one slow dependency does this to the whole server. The task names say which sessions are stuck; the stage timings say what they are stuck on. `AsyncQueueStallThreshold` (120 seconds by default) is how long every worker may be busy before the second report is written; `0` turns the report off. ### Delivering: how the queue works There is no in-memory queue to inspect. The delivery queue *is* the set of rows in `hm_messages` whose type is delivering, and the delivery manager is a loop that polls for them. Three properties of that loop explain most of what a stuck queue looks like: - **The selection is `where messagetype = 1 and messagelocked = 0 and messagenexttrytime <= now`, ordered by size, then try count, then id.** Small and new goes first. - **The delivery pool is separate and smaller than the accept pool**: `maxdelivertythreads`, 10 by default, on the work queue named `SMTP delivery queue`. It is not the `Asynchronous task queue` that finalises inbound mail, so a saturated delivery pool does not stop mail being accepted; it stops it leaving. - **The manager wakes on demand, or once a minute otherwise.** A message whose next-try time has just passed is picked up within a minute, not instantly. At startup every message in the queue is unlocked (`update hm_messages set messagelocked = 0`), because locks held by a server that crashed would otherwise strand those messages for ever. That is also why exactly one hMailServer service may run against one database. ### The retry schedule, and when a message gives up | Setting | Where | Default | Effect | |---|---|---|---| | Number of retries | **Mail flow & delivery → Delivery of e-mail** (`smtpnoofretries`) | `4` | How many *further* attempts after the first | | Minutes between every retry | The same page (`smtpminutesbetweenretries`) | `60` | The gap between them | | `QuickRetries` | `[Settings]` | `0` (off) | The first N retries use the short gap below instead, for greylisting | | `QuickRetriesMinutes` | `[Settings]` | `6` | That short gap | | `QueueRandomnessMinutes` | `[Settings]` | `0` (off) | Adds a random 1..N minutes to every gap, so a burst does not retry in lockstep | | Route *number of tries* and *minutes between tries* | **Mail flow & delivery → Routes** | — | Overrides both, per matching route; where several routes match, the largest of each wins | With the shipped defaults that is five attempts an hour apart: a destination that is refusing temporarily holds a message for roughly four hours and then bounces it. The try count in the queue is the same `messagecurnooftries` the arithmetic uses. Two things do not advance that counter: *Deliver now* resets the next-try time without touching the count, and a message held because the virus scanner could not examine it is held on its own budget (below). ### Reading a stuck queue | What the queue looks like | What it means | Where to look next | |---|---|---| | Depth climbing, oldest message seconds old | A busy server draining normally. Not a fault | — | | Depth flat, oldest message hours old, try counts climbing | The far end is deferring you | The SMTP log for that destination; your IP's reputation and PTR | | Every stuck message shares one recipient domain | That destination, not your server | The route settings for it, and its MX | | Try counts stuck at `0`, next-try times in the past, nothing moving | Delivery threads are not running the tasks | `hmailserver_workqueue_depth{queue="SMTP delivery queue"}`, and the saturation report above | | Messages appear and vanish without arriving | Delivered to the wrong place, or discarded by a rule | **Mail flow & delivery → Rules**; enable `MessageTraceEnabled` and read the trace | | Try count `1` and a next-try time of `1901-01-01` | An ETRN hold: the one route these recipients match is an ETRN route | **Mail flow & delivery → Routes**; the log line is `SMTPDeliverer - Route Message: HOLD for later delivery..` | | Queue empty, but the sender got a `250` and nobody received it | Delivered locally into a folder you are not looking at, or the row was removed | The account's folders; the message trace | | `Message N could not be delivered since the data file does not exist` | The row and the store disagree: a restore that missed files, or a store on a share that vanished | Turn on `MessageStoreConsistencyCheck` and read the report | **When the virus scanner cannot examine a message.** With `AVFailAction=1` such a message is not delivered. It is held on its own schedule rather than on the SMTP retry budget: `AVFailRetryMinutes` apart (15 by default), at most `AVFailMaxHolds` times (16), and then returned to the sender. The log says where in that budget each hold is (`... this was hold 3 of 16 before it is returned to the sender.`). Each hold counts as a deferral in `hmailserver_messages_deferred_total`, so a scanner that has stopped answering shows as a deferral rate with no matching delivery rate. ### The usual causes of a genuinely stuck delivery - **A virus scanner that stops responding.** ClamAV is contacted after the message is accepted, so a wedged clamd shows up as accepted but never delivered. Each socket operation is bounded by `ClamMinTimeout` / `ClamMaxTimeout`, but that is a deadline on *one* read or write, armed fresh each time, and the message is streamed in chunks, so one large message can take considerably longer than `ClamMaxTimeout` in total. - **A remote server that answers extremely slowly.** The idle timeout is re-armed on every byte received, so a host that sends one byte occasionally used to hold a delivery thread indefinitely. Since 6.2.18 there is an absolute ceiling, `ClientSessionCeiling` (30 minutes by default), armed once and never re-armed. - **A custom virus scanner or external tool that hangs**, bounded by `ExternalProcessTimeout`. - **A message held for a sandbox's verdict** (since 6.3.4). With `SandboxEnabled` on, a message none of your own signed-in users submitted, whose attachment is of a submitted type, waits for the sandbox you named to open the file, up to `SandboxTimeoutMinutes` (15 by default). While it waits it is in no mailbox, not in the quarantine and not the queue's to retry: its Retry and Delete answer 409. It is listed by `GET /api/v1/sandbox/held`, by `GET /api/v1/queue` marked `held: sandbox` (the `held` column of `hmctl queue list`), and in the Deck's queue page under *Held for a sandbox*, where it can be delivered without the verdict or discarded. - **The database.** Check for errors mentioning the connection pool. Since 6.3.4 a message can also wait in other places, each on purpose: held for review by compromised-account detection in `enforce` (`/api/v1/review`); in a destination's back-off window, with the last error `Delivery deferred by this server's own shaping, not by a remote server.` (`GET /api/v1/queue/shaping`; such a deferral does not spend the message's retries); under an IP warm-up ceiling; or behind a drain, when `/readyz` says `draining`. Section 19.11a of the Troubleshooting page lists each with what to do. ### Settings that bound each stage All are in seconds. On 6.3.3 and earlier they are `[Settings]` keys in `hMailServer.ini`. Since 6.3.4 they are stored in the database and changed in the Control Panel, over the REST API at `/api/v1/settings/ini/` or with `hmctl`, and the save applies them. They still appear in the file's `[Settings]` section, but the database's value is the one used: where the service can write the file, an edit made there is reported (HM5804) and put back at the next start. | Setting | Default | Bounds | `0` means | |---|---|---|---| | `FinalizationTimeout` | 240 | The whole accept pipeline, after which the sender gets a `451` | no bound | | `SAMaxTimeout` | 90 | SpamAssassin: idle timeout, and a session ceiling of this plus 30s | **not** "no bound" | | `ClamMaxTimeout` | 90 | ClamAV: idle timeout per socket operation | **not** "no bound" | | `DNSQueryTimeout` | 10 | A single DNS query | no bound | | `ScriptTimeout` | 60 | One event script invocation | no bound | | `ExternalProcessTimeout` | 300 | An external scanner process | no bound | | `ClientSessionCeiling` | 1800 | An entire outbound delivery session | no bound | | `AsyncQueueStallThreshold` | 120 | How long every worker may be busy before the saturation report | reporting off | | `DBConnectionAcquireTimeout` | 60 | Waiting for a pooled database connection | no bound | **The two exceptions matter, because `0` tightens them instead of removing them.** `SAMaxTimeout` and `ClamMaxTimeout` are not passed straight through: they go to a calculator that returns the *minimum* whenever the maximum is lower than it. So `SAMaxTimeout=0` gives a 30-second idle timeout (`SAMinTimeout`) and a 30-second session ceiling, and `ClamMaxTimeout=0` gives 15 seconds (`ClamMinTimeout`). To lengthen either, raise the value; to lengthen it a long way, raise the matching `...MinTimeout` too. There is no way to make either unbounded. When the pool deadline expires, a recipient lookup answers `451 4.3.2` rather than `550`: the server can tell "the database did not answer" from "no such address", so a database locked by a backup defers the mail instead of bouncing it. ### Commands The COM examples below are Windows-only. The metrics endpoint and the REST routes work on both platforms. **Is it accepting, or is it delivering?** The metrics listener is off until `MetricsServerPort` is set (9100 here; since 6.3.4 the port, the bind address and the listener's other settings, its credentials included, wait for a reinitialise). Once it is on, the endpoint needs no credential on a loopback bind: ```powershell (Invoke-RestMethod http://127.0.0.1:9100/metrics) -split "`n" | Select-String '^hmailserver_delivery_queue_(messages|oldest_message_age_seconds) |^hmailserver_workqueue_depth|^hmailserver_messages_(delivered|deferred|bounced)_total ' ``` A rising `oldest_message_age_seconds` with a flat `delivered_total` is the delivering half. A rising `hmailserver_workqueue_depth{queue="Asynchronous task queue"}` is the accepting half. The COM examples start from a signed-in `$app`: ```powershell $app = New-Object -ComObject hMailServer.Application $app.Authenticate("Administrator", "") | Out-Null ``` **Push one message to the front of the queue** (resets its next-try time, leaves the try count alone, and releases an ETRN hold): ```powershell $q = $app.GlobalObjects.DeliveryQueue $q.ResetDeliveryTime(4711) $q.StartDelivery() ``` **Wake the delivery manager without touching any message:** ```powershell $app.GlobalObjects.DeliveryQueue.StartDelivery() ``` **Remove one message from the queue** (it is locked first, so a message a thread already has is not pulled from under it): ```powershell $app.GlobalObjects.DeliveryQueue.Remove(4711) ``` **Follow the log while you reproduce it** by tailing the day's file and filtering on `accept: |Spam test:|waited .* for a thread|have been busy|SMTPDeliverer`; [Reading the logs](/topic/28220) has the tail command for each platform. **Turn the message trace on** (`MessageTraceEnabled`, off by default) to see what was done with one message. It has no acceptance, spam or virus event, so an empty trace does not show that the server never had the message; [Troubleshooting by symptom](/topic/28225) says how to turn it on and what a row says. ### What to include in a report If none of the above identifies it, post in [General support](/category/8) or open an issue with: the log from the `354` (or from the delivery attempt) onwards, including the stage timing lines; the `ERROR_hmailserver_.log` for the same window; which scanners and event scripts are enabled; and whether the message eventually arrives, arrives twice, or never arrives. Say which half you have. [Read this before you post](/topic/28185) covers the rest. Without a GitLab account, email the report to the project's Service Desk address, contact-project+hmailserver-hmailserver-86730559-issue-@incoming.gitlab.com, which GitLab files as a confidential issue; the maintainers answer by email. ### Where to read more - Wiki: [Diagnosing Stalled Mail](https://gitlab.com/hmailserver/hmailserver/-/wikis/Diagnosing-Stalled-Mail) and [Troubleshooting](https://gitlab.com/hmailserver/hmailserver/-/wikis/Troubleshooting), sections 19.7 and 19.11a. - Repository: [DiagnosingStalledMail.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/DiagnosingStalledMail.md); [SettingsApply.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SettingsApply.md); [Sandbox.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Sandbox.md); [SUPPORT.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/SUPPORT.md). - The public documentation: [The delivery queue and the retry schedule](https://www.progressiverobot.com/hmailserver-documentation/#267-the-delivery-queue-and-the-retry-schedule) and [Troubleshooting](https://www.progressiverobot.com/hmailserver-documentation/#19-troubleshooting). - On this forum: [Being trusted by the servers you send to](/topic/28188), for when the far end is deferring you; [Outbound shaping and IP warm-up](/topic/28259), for when your own server is holding mail back; [Monitoring and health](/topic/28272), for turning the metrics listener on. --- Canonical: https://www.hmailserver.co.uk/topic/28220/reading-the-logs-levels-files-the-logs-page-following-a-connection-and-fail2ban # Reading the logs: levels, files, the Logs page, following a connection, and fail2ban *Written for 6.3.4. hMailServer 6.3.7 (17 October 2026) fixes the SMTP log writing some credentials in the clear with SMTP logging on: an `AUTH PLAIN` sent before `EHLO` or not in its one exact form, the bearer token of `AUTH XOAUTH2` and `AUTH OAUTHBEARER`, and the lines of SCRAM-SHA-256, OAUTHBEARER and GSSAPI went into the log as the client sent them, base64 and readable, in every version from 6.0.0 to 6.3.6. An SMTP log written by 6.3.6 or earlier may hold them: redact every line of a SASL exchange before posting one. A database write that failed is now reported (`HM5032`) where it was taken as done, so an error log that was quiet may name failures that were hidden; and the IMAP and POP3 listeners can take the client's address from a PROXY protocol header, as the SMTP listener does, both off by default. See [the 6.3.7 announcement](/topic/28277).* This topic covers what hMailServer writes to disk and which file to open, the settings that decide how much is written and kept, the live views, following one connection (since 6.3.4), a copy to syslog or the journal, and the fail2ban filter the Linux package installs. The one thing to know first: open `ERROR_hmailserver_.log` before anything else. It holds only the entries the server itself classed as an error, and its mere existence is a diagnostic, because the Control Panel's built-in self-test (**Monitoring & troubleshooting → Diagnostics**, so Windows) fails when any `ERROR*` file is present in the log directory. ### Where the logs are | Platform | Log directory | Set by | |---|---|---| | Windows | `C:\Program Files\hMailServer\Logs` by default | `[Directories] LogFolder` in `hMailServer.INI` | | Linux | `/var/log/hmailserver`, created by the package as `0750 hmailserver:hmailserver` and left in place when the package is removed | `[Directories]` in `/etc/hmailserver/hMailServer.ini` | The format is the same on both platforms. On Linux the journal (`journalctl -u hmailserver`) carries the startup line, anything written to standard error and the operational events; a memory-safety fault is recorded in `/var/log/hmailserver/crash-oracle.log` as one line (signal, address, thread, event number) before the process dies by the signal it caught. `[Directories]` is per machine and is never mirrored into the database, so do not copy another server's file over this one's. ### The files | File | Holds | Rotates | |---|---|---| | `hmailserver_.log` | Everything: application lines, and the protocol conversations unless they are split out | Daily, by name | | `ERROR_hmailserver_.log` | Only entries the server classed as an error | Daily, by name | | `hmailserver_SMTP_.log`, `hmailserver_IMAP_.log`, `hmailserver_POP3_.log` | The per-protocol conversations, when `SepSvcLogs=1` | Daily, by name | | `hmailserver_backup.log` | Every backup and restore run, line by line | Never: one growing file. `LogDeleteDays` matches it by name but deletes only a file nothing has written to for that many days | | `hmailserver_events.log` | Event-script output | Never, as above | | `hmailserver_awstats.log` | The AWStats-format feed, when AWStats logging is on | Never, as above | | `hMailServer_messagestore_consistency.report` | The most recent message-store consistency scan: one line per message whose file is missing, as `messageidaccountexpected-path` | Overwritten each run | The backup and event logs are among the files the server marks with a byte-order mark; they are UTF-16LE on both platforms, so a file written on one reads on the other. A log line looks like `"IMAPD" 5220 41 "2026-09-17 09:00:00.120" "192.0.2.7" "SENT: A1 OK LOGIN completed"`, where `IMAPD` is the category and `41` is the session id that every line of that conversation carries. The conversation categories are `SMTPD`, `SMTPC`, `POP3D`, `POP3C` and `IMAPD`; `APPLICATION`, `TCPIP`, `DEBUG` and `ERROR` are the rest. ### The settings | Setting | Default | What it does | What it looks like when it is wrong | |---|---|---|---| | `LogLevel` | `9` | Not the choice of log types, which is the Logging page's tick-boxes (or `PUT /api/v1/settings/logging`). The Settings Reference records two effects: long lines are cut only when it is 2 or lower with debug off, and IMAP `SENT:` lines for FETCH and STATUS are logged only when it is above 2 or debug is on | — | | Debug messages (the Logging page) | off | Adds the `LOG_DEBUG` lines, including the accept-pipeline stage timings | Left on, a busy server fills the disk in days, and a full disk stops mail | | `SepSvcLogs` | `0` | Splits SMTP, IMAP and POP3 into their own files | Off, so a search for one protocol wades through all three | | `LogDeleteDays` | `0` | Deletes files named `hmailserver_*.log` or `ERROR_hmailserver_*.log` whose last write is older than this many days, and nothing else in the directory. `0` keeps everything for ever | Left at `0` on a server with debug logging on: the usual cause of a full log volume | | `JsonLogging` | `0` | Writes JSON lines, for Elasticsearch, Loki or Splunk | Overridden while `Logging.LogFormat` is NCSA: the pipeline gets NCSA lines and no error | | `SlowQueryLogMilliseconds` | `0` (off) | Logs, and counts in `hmailserver_db_slow_queries_total`, any database statement slower than this | Off, so a database that has become slow leaves no trace but the latency histogram | | `WindowsEventLogLevel` | `2` | How much also goes to the Windows event log | — | **Where they are set.** On Windows, **Monitoring & troubleshooting → Logging** in the Control Panel. On either platform the table's `[Settings]` keys (all but Debug messages) are set with `PUT /api/v1/settings/ini/{name}` or `hmctl ini set`. On Linux there is no Control Panel, so the log categories, debug messages among them, are set with `PUT /api/v1/settings/logging`, `hmctl settings set logging` or the Deck's Settings page. There, `WindowsEventLogEnabled=1` is a Windows inheritance in name only, because on Linux it means syslog, and the operational events reach the journal through it. Since 6.3.4 these keys are stored in the database, whose value is the one used: where the service can write `hMailServer.INI`, a line edited there is put back and reported as HM5804. [How the settings store works](/topic/28222) has the rest, `[SettingsOverride]` (HM5807) among it. **Rotation.** On Windows, `LogDeleteDays` is the rotation. On Linux the package installs `/etc/logrotate.d/hmailserver`; the rule uses `copytruncate` because the server does not reopen its log on a signal, and the rule says so at length. **Debug logging: on to diagnose, off afterwards.** The stage-timing lines that diagnosing stalled mail depends on are debug lines; the slow ones (ten seconds or more) are written at application level as well. Leaving debug on matters because a full disk stops mail: the application log warns below `DiskSpaceWarningThresholdMB` (1024 MB), and below `MinimumFreeDiskSpaceMB` (100 MB) on the message store's volume the server answers `452 4.3.1 Insufficient system storage.` and senders retry. ### The live views **Windows: the Control Panel.** **Monitoring & troubleshooting → Live logs** is a real-time, colour-coded tail of what the server is writing; every page in that group but Discovery reads the running service over COM (Discovery has no COM interface, so it reads the REST listener and needs `RestApiPort` set). **Diagnosing stalled mail**, in the same group, has a one-click debug-logging toggle. **Both platforms: the Control Deck and `hmctl`.** The browser Control Deck at `/` on the REST listener has a **Logs** page that lists the log files and shows any file's last 200 or 2,000 lines, without a share to the log directory. Since 6.3.4, `hmctl log list` and `hmctl log tail --lines N` read the same over `GET /api/v1/logs` and `GET /api/v1/logs/{name}`, and work against a Windows or a Linux server because they go over HTTP rather than DCOM. The Linux packages install `hmctl` as `/usr/bin/hmctl`; the `HmailServer` PowerShell module is the same vocabulary for PowerShell. **The files themselves, from a shell.** On Windows: ```powershell $log = "C:\Program Files\hMailServer\Logs\hmailserver_" + (Get-Date -Format "yyyy-MM-dd") + ".log" Get-Content $log -Tail 0 -Wait | Select-String "SMTPD|SMTPConnection" ``` On Linux: ```sh tail -f /var/log/hmailserver/hmailserver_$(date +%F).log journalctl -u hmailserver -f ``` ### Following one connection, since 6.3.4 Every SMTP, POP3 and IMAP conversation is a session with a number, and every line the log writes for it carries that number. Until 6.3.4 that meant searching the day's log for the number afterwards. Since 6.3.4 the **Connections** section at the top of the Deck's **Logs** page lists every connection the server has open now, newest first, and the last two hundred that ended: the id, the protocol, the address it came from, the port it came in on, the account once it has signed in, and when it started or ended. **Follow** beside any of them opens a pane holding every line the log has for that session and keeps adding lines as the server writes them; **Copy** puts the whole pane on the clipboard; **Stop following** ends it. A session that ended a second ago is still listed, so an SMTP conversation that lasted a quarter of a second can be picked and read. **How it works, and what that means for you.** - The server keeps the last 4,096 log entries in memory, across every session; the ring is always on, and nothing survives a restart. A debug-logged IMAP client can fill it in minutes, so a follow whose cursor points before the oldest entry still held answers `dropped: true` and starts from what is left; the page says so. A line older than that is in the log files and nowhere else. - The conversation lines (`SMTPD`, `SMTPC`, `POP3D`, `POP3C`, `IMAPD`) carry the session id in their own column. `DEBUG` and `TCPIP` lines carry none and are filed by the first number after the word *session* in their text (`TCP connection started for session 41`, `Ending session 41`); that is an inference, and the text is not always the server's own. A line that does neither is nobody's and is not shown. - It shows what the log mask lets through, and no more. With IMAP logging off, an IMAP session is listed and its follow is empty; the answer's `logged` field says whether the log mask lets that protocol's lines through, and the page names the switch to turn on. Debug logging is what makes the TLS handshake, the timeouts and the disconnect appear beside the conversation. - The connections the server opens itself are listed too, marked `client`: `SMTP client`, `POP3 client`, `IMAP client`, and the short-lived `SpamAssassin` and `Diagnostics` connections. Not listed: anything that is not a `TCPConnection`, so the REST API, the webmail, the DAV services, ManageSieve and the anti-virus scanner's socket to `clamd` are not sessions here. A delivery to a host that refuses the connection is not listed either, though with debug logging on its lines can still be followed by id, with `known: false`. - A follow is a long poll, not a stream: `GET /api/v1/logs/sessions/{id}?after=0&wait=10` answers the session's lines after sequence 0 and a `cursor` to ask after next time, waiting up to `wait` seconds for the next line. The wait is at most twenty seconds, and one waiting follow at a time is allowed, because a waiting follow holds one of the HTTP listener's four worker threads, which also serve the webmail; a second follow is answered at once with `waited: false`. An answer carries at most 500 lines, and the pane keeps the last 5,000. - `GET /api/v1/logs/sessions` lists the sessions (the newest thousand when more are open, with `truncated: true`), with `held_lines` and `logging_enabled`. Both routes are under the `view_logs` permission, which the built-in read-only auditor role holds, and both are server-wide: a credential confined to some domains is refused them, as it is refused the log files. - There is no Control Panel page and no `hmctl` verb for this; the Control Panel's Live logs is the older, whole-log form over COM. There is no filtering inside a session. ### A copy to syslog, or to the journal, since 6.3.4 Since 6.3.4 every entry the server logs can also be sent as RFC 5424 to a syslog collector or, on Linux, where the server runs as the packaged systemd unit with no collector host named, to the journal. It is a copy: the files above are written exactly as before, and nothing it does can stop a line reaching them, or delay or lose a message. Its settings are `hm_settings` rows, not `[Settings]` keys, read when the sink starts (the next service start or reinitialise), not when they are saved. With the journal on, `journalctl -u hmailserver HMAILSERVER_SESSION=41` follows one session from the journal, across a restart. [Alerts, the audit trail, syslog and the journal](/topic/28273) has the twelve settings, the transports, the rsyslog configurations and the journal's fields. ### Codes in the error log worth knowing since 6.3.4 | Code | What it says, and what to do | |---|---| | HM5804 | A value typed into `hMailServer.INI` `[Settings]` for a key the database holds was put back at the last start. Change it in the Control Panel, with `PUT /api/v1/settings/ini/{name}`, `hmctl ini set` or COM `Settings.SetIniSetting` | | HM7460 | `AutoBanFirewall` is on and the Windows service account cannot write firewall rules (below) | | HM7495 | The database records a stored-secrets key this server does not hold, and every account's second factor refuses every code: import the exported key with `hMailServer.exe --import-stored-secrets-key FILE` (elevated) or `sudo hmailserver --import-stored-secrets-key FILE`, then restart | | HM7540 | A restore was refused because the archive holds an entry with an absolute path, a `..` segment or a link with somewhere to point; nothing was extracted. Find out where the archive came from | ### fail2ban on Linux, and the Windows contrast An auto-ban (repeated failed logons from one address) is enforced by the server's own listeners: the kernel accepts the connection and the server refuses it. Shipped in 6.3.2, the Linux package offers two ways to push the ban below the server, and one is enough. **fail2ban** needs no capability at all, because it runs as root and reads the application log. The package installs `/etc/fail2ban/filter.d/hmailserver.conf` and a jail at `/etc/fail2ban/jail.d/hmailserver.conf`, disabled as Debian asks of a shipped jail. The filter matches the two fixed-shape lines the server writes for every counted failure and every ban, whether or not protocol logging is on; auto-ban on logon failure (`AutoBanOnLogonFailureEnabled`, on as shipped; `auto_ban_on_logon_failure` over REST) is what writes them. Turn it on: ``` printf '[hmailserver]\nenabled = true\n' > /etc/fail2ban/jail.d/hmailserver.local systemctl reload fail2ban fail2ban-client status hmailserver ``` **The server itself**, with `AutoBanFirewall=1`, makes the firewall match its auto-ban ranges at start, on every ban, on every range deletion and once a minute. On Linux it runs `/usr/lib/hmailserver/autoban-hook`, which keeps an nftables set with a timeout (iptables where there is no nft) and drops TCP from the banned address to the ports the server listens on. Changing the firewall needs `CAP_NET_ADMIN`, which the unit deliberately does not grant a mail server by default. Grant it once with a drop-in, and the hook's `check` verb says whether it can act: ``` systemctl edit hmailserver [Service] AmbientCapabilities=CAP_NET_BIND_SERVICE CAP_NET_ADMIN CapabilityBoundingSet=CAP_NET_BIND_SERVICE CAP_NET_ADMIN systemctl restart hmailserver sudo -u hmailserver /usr/lib/hmailserver/autoban-hook check ``` `AutoBanCommand`, on either platform, runs a program of yours instead of, or as well as, `AutoBanFirewall` (` ban
` and ` unban
`) for a cloud firewall or a router. Before turning either on, list the addresses you administer from in `AutoBanNeverBan`: a ban that reaches the firewall locks that address out of every mail port on the machine. Since 6.3.4 `AutoBanFirewall`, `AutoBanCommand` and `AutoBanNeverBan` are stored settings like the other `[Settings]` keys: set them over the API or with `hmctl ini set`, not in the file. The fail2ban filter and the hook are part of the Linux packages. On Windows, `AutoBanFirewall` writes one inbound block rule per banned address in Windows Defender Firewall, named `hMailServer auto-ban
` and scoped to the ports the server listens on, and since 6.3.4 a new installation's service account, `NT SERVICE\hMailServer`, cannot write them: HM7460 is logged once a start and no rules appear, while the listeners still refuse a banned address. Turn `AutoBanFirewall` off there, or use `AutoBanCommand` towards a router or cloud firewall; HM7461 names rules a LocalSystem run left behind, with the PowerShell that removes them. ### What to include in a report [SUPPORT.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/SUPPORT.md) asks for, in order of value: the relevant log, not a description of it, with debug logging on and the window around the problem; the `ERROR_hmailserver_.log` for the same window; the exact version, the platform and the database backend; and what you expected instead, and whether it is reproducible. Remove passwords, keys and real email addresses first, because a posted log is public. [Read this before you post](/topic/28185) asks for the same log lines and adds the build number, the package, whether the server was upgraded and the Diagnostics output; **Copy** on the Diagnostics page puts the whole result on the clipboard. Without a GitLab account, email the report to the project's Service Desk address, contact-project+hmailserver-hmailserver-86730559-issue-@incoming.gitlab.com, which GitLab files as a confidential issue; the maintainers answer by email. ### Where to read more - Wiki: [Monitoring and Health](https://gitlab.com/hmailserver/hmailserver/-/wikis/Monitoring-and-Health), sections 16.2 and 16.2a; [Troubleshooting](https://gitlab.com/hmailserver/hmailserver/-/wikis/Troubleshooting), sections 19.10 and 19.11; [Installing on Linux](https://gitlab.com/hmailserver/hmailserver/-/wikis/Installing-on-Linux), for the journal and fail2ban. - Repository: [FollowConnection.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/FollowConnection.md), [Syslog.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Syslog.md), [SUPPORT.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/SUPPORT.md). - The public documentation: [Logs on disk](https://www.progressiverobot.com/hmailserver-documentation/#162-logs-on-disk), [Logging and log rotation on Linux](https://www.progressiverobot.com/hmailserver-documentation/#3511-logging-and-log-rotation), [Follow one message end to end in the logs](https://www.progressiverobot.com/hmailserver-documentation/#follow-one-message-end-to-end-in-the-logs), [Codes in the error log](https://www.progressiverobot.com/hmailserver-documentation/#328-codes-in-the-error-log). - On this forum: [hMailServer on Linux: what you get and what you do not](/topic/28189), [The Control Deck](/topic/28192), [Diagnosing stalled mail](/topic/28219), and [IP ranges, auto-ban and lockout](/topic/28239) for when a ban is made and how to lift one. --- Canonical: https://www.hmailserver.co.uk/topic/28220/reading-the-logs-levels-files-the-logs-page-following-a-connection-and-fail2ban # Reading the logs: levels, files, the Logs page, following a connection, and fail2ban *Written for 6.3.4. hMailServer 6.3.7 (17 October 2026) fixes the SMTP log writing some credentials in the clear with SMTP logging on: an `AUTH PLAIN` sent before `EHLO` or not in its one exact form, the bearer token of `AUTH XOAUTH2` and `AUTH OAUTHBEARER`, and the lines of SCRAM-SHA-256, OAUTHBEARER and GSSAPI went into the log as the client sent them, base64 and readable, in every version from 6.0.0 to 6.3.6. An SMTP log written by 6.3.6 or earlier may hold them: redact every line of a SASL exchange before posting one. A database write that failed is now reported (`HM5032`) where it was taken as done, so an error log that was quiet may name failures that were hidden; and the IMAP and POP3 listeners can take the client's address from a PROXY protocol header, as the SMTP listener does, both off by default. See [the 6.3.7 announcement](/topic/28277).* This topic covers what hMailServer writes to disk and which file to open, the settings that decide how much is written and kept, the live views, following one connection (since 6.3.4), a copy to syslog or the journal, and the fail2ban filter the Linux package installs. The one thing to know first: open `ERROR_hmailserver_.log` before anything else. It holds only the entries the server itself classed as an error, and its mere existence is a diagnostic, because the Control Panel's built-in self-test (**Monitoring & troubleshooting → Diagnostics**, so Windows) fails when any `ERROR*` file is present in the log directory. ### Where the logs are | Platform | Log directory | Set by | |---|---|---| | Windows | `C:\Program Files\hMailServer\Logs` by default | `[Directories] LogFolder` in `hMailServer.INI` | | Linux | `/var/log/hmailserver`, created by the package as `0750 hmailserver:hmailserver` and left in place when the package is removed | `[Directories]` in `/etc/hmailserver/hMailServer.ini` | The format is the same on both platforms. On Linux the journal (`journalctl -u hmailserver`) carries the startup line, anything written to standard error and the operational events; a memory-safety fault is recorded in `/var/log/hmailserver/crash-oracle.log` as one line (signal, address, thread, event number) before the process dies by the signal it caught. `[Directories]` is per machine and is never mirrored into the database, so do not copy another server's file over this one's. ### The files | File | Holds | Rotates | |---|---|---| | `hmailserver_.log` | Everything: application lines, and the protocol conversations unless they are split out | Daily, by name | | `ERROR_hmailserver_.log` | Only entries the server classed as an error | Daily, by name | | `hmailserver_SMTP_.log`, `hmailserver_IMAP_.log`, `hmailserver_POP3_.log` | The per-protocol conversations, when `SepSvcLogs=1` | Daily, by name | | `hmailserver_backup.log` | Every backup and restore run, line by line | Never: one growing file. `LogDeleteDays` matches it by name but deletes only a file nothing has written to for that many days | | `hmailserver_events.log` | Event-script output | Never, as above | | `hmailserver_awstats.log` | The AWStats-format feed, when AWStats logging is on | Never, as above | | `hMailServer_messagestore_consistency.report` | The most recent message-store consistency scan: one line per message whose file is missing, as `messageidaccountexpected-path` | Overwritten each run | The backup and event logs are among the files the server marks with a byte-order mark; they are UTF-16LE on both platforms, so a file written on one reads on the other. A log line looks like `"IMAPD" 5220 41 "2026-09-17 09:00:00.120" "192.0.2.7" "SENT: A1 OK LOGIN completed"`, where `IMAPD` is the category and `41` is the session id that every line of that conversation carries. The conversation categories are `SMTPD`, `SMTPC`, `POP3D`, `POP3C` and `IMAPD`; `APPLICATION`, `TCPIP`, `DEBUG` and `ERROR` are the rest. ### The settings | Setting | Default | What it does | What it looks like when it is wrong | |---|---|---|---| | `LogLevel` | `9` | Not the choice of log types, which is the Logging page's tick-boxes (or `PUT /api/v1/settings/logging`). The Settings Reference records two effects: long lines are cut only when it is 2 or lower with debug off, and IMAP `SENT:` lines for FETCH and STATUS are logged only when it is above 2 or debug is on | — | | Debug messages (the Logging page) | off | Adds the `LOG_DEBUG` lines, including the accept-pipeline stage timings | Left on, a busy server fills the disk in days, and a full disk stops mail | | `SepSvcLogs` | `0` | Splits SMTP, IMAP and POP3 into their own files | Off, so a search for one protocol wades through all three | | `LogDeleteDays` | `0` | Deletes files named `hmailserver_*.log` or `ERROR_hmailserver_*.log` whose last write is older than this many days, and nothing else in the directory. `0` keeps everything for ever | Left at `0` on a server with debug logging on: the usual cause of a full log volume | | `JsonLogging` | `0` | Writes JSON lines, for Elasticsearch, Loki or Splunk | Overridden while `Logging.LogFormat` is NCSA: the pipeline gets NCSA lines and no error | | `SlowQueryLogMilliseconds` | `0` (off) | Logs, and counts in `hmailserver_db_slow_queries_total`, any database statement slower than this | Off, so a database that has become slow leaves no trace but the latency histogram | | `WindowsEventLogLevel` | `2` | How much also goes to the Windows event log | — | **Where they are set.** On Windows, **Monitoring & troubleshooting → Logging** in the Control Panel. On either platform the table's `[Settings]` keys (all but Debug messages) are set with `PUT /api/v1/settings/ini/{name}` or `hmctl ini set`. On Linux there is no Control Panel, so the log categories, debug messages among them, are set with `PUT /api/v1/settings/logging`, `hmctl settings set logging` or the Deck's Settings page. There, `WindowsEventLogEnabled=1` is a Windows inheritance in name only, because on Linux it means syslog, and the operational events reach the journal through it. Since 6.3.4 these keys are stored in the database, whose value is the one used: where the service can write `hMailServer.INI`, a line edited there is put back and reported as HM5804. [How the settings store works](/topic/28222) has the rest, `[SettingsOverride]` (HM5807) among it. **Rotation.** On Windows, `LogDeleteDays` is the rotation. On Linux the package installs `/etc/logrotate.d/hmailserver`; the rule uses `copytruncate` because the server does not reopen its log on a signal, and the rule says so at length. **Debug logging: on to diagnose, off afterwards.** The stage-timing lines that diagnosing stalled mail depends on are debug lines; the slow ones (ten seconds or more) are written at application level as well. Leaving debug on matters because a full disk stops mail: the application log warns below `DiskSpaceWarningThresholdMB` (1024 MB), and below `MinimumFreeDiskSpaceMB` (100 MB) on the message store's volume the server answers `452 4.3.1 Insufficient system storage.` and senders retry. ### The live views **Windows: the Control Panel.** **Monitoring & troubleshooting → Live logs** is a real-time, colour-coded tail of what the server is writing; every page in that group but Discovery reads the running service over COM (Discovery has no COM interface, so it reads the REST listener and needs `RestApiPort` set). **Diagnosing stalled mail**, in the same group, has a one-click debug-logging toggle. **Both platforms: the Control Deck and `hmctl`.** The browser Control Deck at `/` on the REST listener has a **Logs** page that lists the log files and shows any file's last 200 or 2,000 lines, without a share to the log directory. Since 6.3.4, `hmctl log list` and `hmctl log tail --lines N` read the same over `GET /api/v1/logs` and `GET /api/v1/logs/{name}`, and work against a Windows or a Linux server because they go over HTTP rather than DCOM. The Linux packages install `hmctl` as `/usr/bin/hmctl`; the `HmailServer` PowerShell module is the same vocabulary for PowerShell. **The files themselves, from a shell.** On Windows: ```powershell $log = "C:\Program Files\hMailServer\Logs\hmailserver_" + (Get-Date -Format "yyyy-MM-dd") + ".log" Get-Content $log -Tail 0 -Wait | Select-String "SMTPD|SMTPConnection" ``` On Linux: ```sh tail -f /var/log/hmailserver/hmailserver_$(date +%F).log journalctl -u hmailserver -f ``` ### Following one connection, since 6.3.4 Every SMTP, POP3 and IMAP conversation is a session with a number, and every line the log writes for it carries that number. Until 6.3.4 that meant searching the day's log for the number afterwards. Since 6.3.4 the **Connections** section at the top of the Deck's **Logs** page lists every connection the server has open now, newest first, and the last two hundred that ended: the id, the protocol, the address it came from, the port it came in on, the account once it has signed in, and when it started or ended. **Follow** beside any of them opens a pane holding every line the log has for that session and keeps adding lines as the server writes them; **Copy** puts the whole pane on the clipboard; **Stop following** ends it. A session that ended a second ago is still listed, so an SMTP conversation that lasted a quarter of a second can be picked and read. **How it works, and what that means for you.** - The server keeps the last 4,096 log entries in memory, across every session; the ring is always on, and nothing survives a restart. A debug-logged IMAP client can fill it in minutes, so a follow whose cursor points before the oldest entry still held answers `dropped: true` and starts from what is left; the page says so. A line older than that is in the log files and nowhere else. - The conversation lines (`SMTPD`, `SMTPC`, `POP3D`, `POP3C`, `IMAPD`) carry the session id in their own column. `DEBUG` and `TCPIP` lines carry none and are filed by the first number after the word *session* in their text (`TCP connection started for session 41`, `Ending session 41`); that is an inference, and the text is not always the server's own. A line that does neither is nobody's and is not shown. - It shows what the log mask lets through, and no more. With IMAP logging off, an IMAP session is listed and its follow is empty; the answer's `logged` field says whether the log mask lets that protocol's lines through, and the page names the switch to turn on. Debug logging is what makes the TLS handshake, the timeouts and the disconnect appear beside the conversation. - The connections the server opens itself are listed too, marked `client`: `SMTP client`, `POP3 client`, `IMAP client`, and the short-lived `SpamAssassin` and `Diagnostics` connections. Not listed: anything that is not a `TCPConnection`, so the REST API, the webmail, the DAV services, ManageSieve and the anti-virus scanner's socket to `clamd` are not sessions here. A delivery to a host that refuses the connection is not listed either, though with debug logging on its lines can still be followed by id, with `known: false`. - A follow is a long poll, not a stream: `GET /api/v1/logs/sessions/{id}?after=0&wait=10` answers the session's lines after sequence 0 and a `cursor` to ask after next time, waiting up to `wait` seconds for the next line. The wait is at most twenty seconds, and one waiting follow at a time is allowed, because a waiting follow holds one of the HTTP listener's four worker threads, which also serve the webmail; a second follow is answered at once with `waited: false`. An answer carries at most 500 lines, and the pane keeps the last 5,000. - `GET /api/v1/logs/sessions` lists the sessions (the newest thousand when more are open, with `truncated: true`), with `held_lines` and `logging_enabled`. Both routes are under the `view_logs` permission, which the built-in read-only auditor role holds, and both are server-wide: a credential confined to some domains is refused them, as it is refused the log files. - There is no Control Panel page and no `hmctl` verb for this; the Control Panel's Live logs is the older, whole-log form over COM. There is no filtering inside a session. ### A copy to syslog, or to the journal, since 6.3.4 Since 6.3.4 every entry the server logs can also be sent as RFC 5424 to a syslog collector or, on Linux, where the server runs as the packaged systemd unit with no collector host named, to the journal. It is a copy: the files above are written exactly as before, and nothing it does can stop a line reaching them, or delay or lose a message. Its settings are `hm_settings` rows, not `[Settings]` keys, read when the sink starts (the next service start or reinitialise), not when they are saved. With the journal on, `journalctl -u hmailserver HMAILSERVER_SESSION=41` follows one session from the journal, across a restart. [Alerts, the audit trail, syslog and the journal](/topic/28273) has the twelve settings, the transports, the rsyslog configurations and the journal's fields. ### Codes in the error log worth knowing since 6.3.4 | Code | What it says, and what to do | |---|---| | HM5804 | A value typed into `hMailServer.INI` `[Settings]` for a key the database holds was put back at the last start. Change it in the Control Panel, with `PUT /api/v1/settings/ini/{name}`, `hmctl ini set` or COM `Settings.SetIniSetting` | | HM7460 | `AutoBanFirewall` is on and the Windows service account cannot write firewall rules (below) | | HM7495 | The database records a stored-secrets key this server does not hold, and every account's second factor refuses every code: import the exported key with `hMailServer.exe --import-stored-secrets-key FILE` (elevated) or `sudo hmailserver --import-stored-secrets-key FILE`, then restart | | HM7540 | A restore was refused because the archive holds an entry with an absolute path, a `..` segment or a link with somewhere to point; nothing was extracted. Find out where the archive came from | ### fail2ban on Linux, and the Windows contrast An auto-ban (repeated failed logons from one address) is enforced by the server's own listeners: the kernel accepts the connection and the server refuses it. Shipped in 6.3.2, the Linux package offers two ways to push the ban below the server, and one is enough. **fail2ban** needs no capability at all, because it runs as root and reads the application log. The package installs `/etc/fail2ban/filter.d/hmailserver.conf` and a jail at `/etc/fail2ban/jail.d/hmailserver.conf`, disabled as Debian asks of a shipped jail. The filter matches the two fixed-shape lines the server writes for every counted failure and every ban, whether or not protocol logging is on; auto-ban on logon failure (`AutoBanOnLogonFailureEnabled`, on as shipped; `auto_ban_on_logon_failure` over REST) is what writes them. Turn it on: ``` printf '[hmailserver]\nenabled = true\n' > /etc/fail2ban/jail.d/hmailserver.local systemctl reload fail2ban fail2ban-client status hmailserver ``` **The server itself**, with `AutoBanFirewall=1`, makes the firewall match its auto-ban ranges at start, on every ban, on every range deletion and once a minute. On Linux it runs `/usr/lib/hmailserver/autoban-hook`, which keeps an nftables set with a timeout (iptables where there is no nft) and drops TCP from the banned address to the ports the server listens on. Changing the firewall needs `CAP_NET_ADMIN`, which the unit deliberately does not grant a mail server by default. Grant it once with a drop-in, and the hook's `check` verb says whether it can act: ``` systemctl edit hmailserver [Service] AmbientCapabilities=CAP_NET_BIND_SERVICE CAP_NET_ADMIN CapabilityBoundingSet=CAP_NET_BIND_SERVICE CAP_NET_ADMIN systemctl restart hmailserver sudo -u hmailserver /usr/lib/hmailserver/autoban-hook check ``` `AutoBanCommand`, on either platform, runs a program of yours instead of, or as well as, `AutoBanFirewall` (` ban
` and ` unban
`) for a cloud firewall or a router. Before turning either on, list the addresses you administer from in `AutoBanNeverBan`: a ban that reaches the firewall locks that address out of every mail port on the machine. Since 6.3.4 `AutoBanFirewall`, `AutoBanCommand` and `AutoBanNeverBan` are stored settings like the other `[Settings]` keys: set them over the API or with `hmctl ini set`, not in the file. The fail2ban filter and the hook are part of the Linux packages. On Windows, `AutoBanFirewall` writes one inbound block rule per banned address in Windows Defender Firewall, named `hMailServer auto-ban
` and scoped to the ports the server listens on, and since 6.3.4 a new installation's service account, `NT SERVICE\hMailServer`, cannot write them: HM7460 is logged once a start and no rules appear, while the listeners still refuse a banned address. Turn `AutoBanFirewall` off there, or use `AutoBanCommand` towards a router or cloud firewall; HM7461 names rules a LocalSystem run left behind, with the PowerShell that removes them. ### What to include in a report [SUPPORT.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/SUPPORT.md) asks for, in order of value: the relevant log, not a description of it, with debug logging on and the window around the problem; the `ERROR_hmailserver_.log` for the same window; the exact version, the platform and the database backend; and what you expected instead, and whether it is reproducible. Remove passwords, keys and real email addresses first, because a posted log is public. [Read this before you post](/topic/28185) asks for the same log lines and adds the build number, the package, whether the server was upgraded and the Diagnostics output; **Copy** on the Diagnostics page puts the whole result on the clipboard. Without a GitLab account, email the report to the project's Service Desk address, contact-project+hmailserver-hmailserver-86730559-issue-@incoming.gitlab.com, which GitLab files as a confidential issue; the maintainers answer by email. ### Where to read more - Wiki: [Monitoring and Health](https://gitlab.com/hmailserver/hmailserver/-/wikis/Monitoring-and-Health), sections 16.2 and 16.2a; [Troubleshooting](https://gitlab.com/hmailserver/hmailserver/-/wikis/Troubleshooting), sections 19.10 and 19.11; [Installing on Linux](https://gitlab.com/hmailserver/hmailserver/-/wikis/Installing-on-Linux), for the journal and fail2ban. - Repository: [FollowConnection.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/FollowConnection.md), [Syslog.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Syslog.md), [SUPPORT.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/SUPPORT.md). - The public documentation: [Logs on disk](https://www.progressiverobot.com/hmailserver-documentation/#162-logs-on-disk), [Logging and log rotation on Linux](https://www.progressiverobot.com/hmailserver-documentation/#3511-logging-and-log-rotation), [Follow one message end to end in the logs](https://www.progressiverobot.com/hmailserver-documentation/#follow-one-message-end-to-end-in-the-logs), [Codes in the error log](https://www.progressiverobot.com/hmailserver-documentation/#328-codes-in-the-error-log). - On this forum: [hMailServer on Linux: what you get and what you do not](/topic/28189), [The Control Deck](/topic/28192), [Diagnosing stalled mail](/topic/28219), and [IP ranges, auto-ban and lockout](/topic/28239) for when a ban is made and how to lift one. --- Canonical: https://www.hmailserver.co.uk/topic/28221/backup-and-restore-what-is-in-the-archive-what-is-not-and-restoring-one-mailbox # Backup and restore: what is in the archive, what is not, and restoring one mailbox This topic covers the built-in backup, what its archive holds and leaves out, and restoring a server, one mailbox or one message. Know this first: **a restore is destructive and has no transaction**. It deletes every domain, every account and every public folder before it puts the archive's back, so whatever an account has that is not in the archive is deleted and not replaced. And the archive **does not include your messages unless you tick the box**. ### What one run writes One 7-Zip archive per run, named `HMBackup YYYY-MM-DD HHMMSS.7z`, in the configured destination: an XML index and, with **Back up messages** on, the whole data directory beside it. The data directory is `C:\Program Files\hMailServer\Data` by default on Windows and `/var/lib/hmailserver` on Linux, and what lives inside it travels with the messages: Sieve scripts and their state (`\Sieve`), and ACME certificates and keys (`\ACME`, unless `AcmeCertificateDirectory` moved them). Since 6.3.4, messages held in an object store (`MessageStoreBackend`) are read into the archive too: a store that cannot be read fails the backup, a restore puts the files back on the local disk, and the store's credentials are not carried. **What it cannot hold: files the database only points at.** Manually placed TLS certificates, private keys and DKIM keys are stored as *paths*. Restored onto a machine where those paths hold nothing, the server starts, serves no TLS and signs nothing. Keep them inside the data directory, or copy them yourself. ### What is in the archive | Area | Carried | |---|---| | Domains | Aliases, distribution lists and members, signatures, DKIM selectors and key *paths*; since 6.3.4 the domain's allow and block lists, its spam traps, the domain transforms, the S/MIME authority and the recovery key of a domain encrypted at rest | | Accounts | Address, password hash and its algorithm, forwarding, vacation, signature, quota, admin level, anti-spam thresholds, TOTP secret | | Per account, every release | Rules, fetch accounts with their passwords, application passwords as hashes and, with **Back up messages**, folders and messages with UID and flags | | Per account, since 6.3.4 | Message keywords (the webmail's labels, second and third stars, follow-up flags, mute and pin), the address book, webmail preferences, scheduled sends and snoozes, file-link records, S/MIME keys, the password history, calendars with their events and tasks, the first-contact memory, safe and blocked senders, the mailbox key of an account encrypted at rest, the role assignment | | Public folders | With the permissions on them. [Backup.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Backup.md) states no condition; section 15.2 of the wiki lists them under **Back up settings**. A restore deletes every public folder first | | Settings, with **Back up settings** | The `hm_settings` property set and the `[Settings]` rows of `hm_inisettings`; routes, IP ranges, global rules, TCP/IP ports, SSL certificate *records*, incoming relays, groups, blocked attachments, the DNSBL, SURBL and white lists; since 6.3.4 the remote domain policies and IP warm-up schedules | | Custom administrator roles | With domains or settings; restored first, so the assignments accounts carry find their role | **Nothing identity-shaped is written.** A restore gives every row a new id, which makes the archive portable between database backends; a reference that cannot be resolved drops that one row and names the account in `hmailserver_backup.log`. **Secrets travel opened.** Route, fetch-account and SSL private-key passwords are carried in a portable encoding and protected again on the restoring machine; since 6.3.4 each TOTP secret and the `[Settings]` secrets (`SRSSecret`, `OAuth2HmacSecret` and the rest) are carried opened and sealed again under the restoring server's stored-secrets key, so a restore onto new hardware keeps every second factor. An unencrypted archive therefore hands those secrets to whoever can read the destination folder. **In 6.3.3 and earlier** the per-account rows marked *since 6.3.4* are in no archive, and a restore deletes them. Until you upgrade, back up the database itself with its own tool (`pg_dump`, `mysqldump`, SQL Server's backup, or a copy of `hMailServer.sdf` with the service stopped), restore from that when recovering a whole server, and tell users before they find their address book and labels gone. An archive written by 6.3.3 or earlier restores on a newer server as it always did. ### What is not in the archive, deliberately | Not carried | Consequence of a restore | |---|---| | `[Database]` and `[Directories]` in `hMailServer.INI`; the administrator password and its second factor | Per machine. Keep a copy of the file, and never copy one machine's file over another's | | `PasswordPepper` (since 6.3.4) | Kept in the file only. A restore onto new hardware needs the same pepper in that machine's file before an Argon2id or scrypt password can sign in; PBKDF2, SHA-256 and MD5 hashes, and the administrator password, do not use it | | The stored-secrets key (since 6.3.4) | The archive does not need it. A *database-level* backup does: restored without the exported key, every second factor and `[Settings]` secret in it is unopenable. Export it once (`hMailServer.exe --export-stored-secrets-key FILE`, elevated, or `sudo hmailserver --export-stored-secrets-key FILE`) and keep it with the database backups, stored apart from them | | On Linux, the stored-secret key file `/.hmailserver-secret-key` | The archive does not carry it. Its passwords travel in a portable encoding and do not need it, but a domain's S/MIME authority key, and the mailbox key of an account encrypted at rest that has not yet signed in, are carried as envelopes that open only where this file is: restored without it, the authority must be rotated before it signs. A database-level backup needs it as well: back it up with the database, because a secret moved to a machine without it cannot be opened (HM6412) | | The delivery queue | Queued files are copied with the data directory, but nothing re-queues them. Let the queue drain before a migration | | Per-domain relay settings, and per-domain message retention days | Silently lost, and silently reset to `0` (keep for ever). Re-enter them | | Greylisting triplets; auto-ban and login-failure state | Senders are greylisted afresh; bans lapse. Permanent IP ranges *are* carried | | Quarantine, message trace, metric history; since 6.3.4 the audit trail, alert rules and events | Operational records. Back the database up with its own tool if you need them | | Deleted items kept for recovery, eDiscovery cases and holds, messages that expire, password-protected messages | The `Recoverable`, `Discovery`, `Confidential` and `Protected` folders are left out of the copy and left in place by a restore. A restore with domains is refused while an account or a domain is under a legal hold. Archives taken by earlier 6.3.4 builds, before `Confidential` and `Protected` were left out, *do* hold both: a message that has expired or been withdrawn since is still in them, so apply the mail's own retention to those archives, or delete them | | The full-text index, IMAP annotations, the QRESYNC expunge log | Rebuilt, or keyed on ids a restore reassigns. IMAP clients resynchronise | | Passkeys; calendar and address-book shares; the backup passphrase, the SNMP passwords, the object store's secret, the sandbox token | Enrol, share or enter them again | A domain named `recoverable`, `discovery`, `confidential` or `protected` keeps its mail in a folder the backup leaves out. Such names are refused for a new domain since 6.3.4, and an existing one is reported as HM7597; rename it before relying on the backup. ### The options, and the combinations that fail **Maintenance → Backup & restore** in the Control Panel (Windows-only), the Control Deck's Backup view, `PUT /api/v1/settings/backup` and `hmctl settings set backup` set the destination and what goes in. A fresh install seeds `backupoptions` as `0`: nothing is ticked. On a packaged Linux server the destination has to be inside the unit's `ReadWritePaths`, or the write fails; add it with a drop-in. | Option | What it changes | |---|---| | **Back up domains** | The domain tree goes into the index. Required for messages to mean anything | | **Back up messages** | Copies the whole data directory. Only with *Back up domains* as well: messages without domains stages nothing and then fails | | **Message metadata only** (`BackupMessagesDBOnly=1`) | Records which messages exist, not their contents; a restore expects the files still to be in the data directory. It governs the restore as well as the backup, and must be the same for both | | **Back up settings** | The settings row above and, per the wiki, public folders | | **Compress destination files** | Puts the message files inside the archive. Off, every run overwrites one `DataBackup` folder beside the archives: exactly one generation of mail, whatever retention says | | **Verify each backup** (`BackupVerifyRestore=1`, the default) | The next section | Tick domains and messages together, always, and **Back up settings** as well if the installation has public folders. The index is written to `hMailServerBackup.xml`, added to the archive and deleted, so what you restore from is the `.7z`; the Control Panel's file picker defaults to `*.xml`, so choose *All files*. ### How a run is verified Every refusal happens before anything in the destination is written, or deletes only the archive this run created. A run refuses when a message file lies outside the data directory, when the destination is unreachable, and when the database is down or any read did not run, so an outage cannot produce a hollow archive. Then the archive is read back: 1. **The index opens**, through the same code a restore uses, and describes this run. If not, the backup fails and the archive is deleted. 2. **The per-account rows are all there** (since 6.3.4), counted per table. A difference fails the backup. 3. **The message store extracts** to exactly the files and bytes that were staged. Fewer is a truncated archive, whatever 7-Zip's exit code said. 4. **Every message row has a file.** Reported, never failed: mail flow does not stop for a backup. The log names the first ten. Steps 3 and 4 run only when domains and messages with their files are included and `BackupVerifyRestore` is 1. The wiki puts step 2 under the same conditions; [Backup.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Backup.md) says only that `BackupVerifyRestore=0` turns off steps 3 and 4. Step 3 costs disk: the store exists twice in the temp directory during the check, and a temp volume that cannot hold it skips the step with an explanation rather than failing a good backup. A successful run ends `Backup completed successfully.`; a failed one has a `BACKUP ERROR` line. ### Schedule and retention Since 6.2.25. Nothing is configured on a fresh install. | Setting | Default | Meaning | |---|---|---| | `ScheduledBackupTime` | empty (off) | `HH:MM`, local time; at most one backup per calendar day | | `ScheduledBackupIntervalMinutes` | `0` (off) | Used only when no daily time is set; measured against the newest archive in the destination | | `ScheduledBackupKeepCount` | `0` (keep all) | How many of this server's own archives to keep | | `ScheduledBackupMaxAgeDays` | `0` (keep all) | Delete archives older than this | | `UpdateBackupBeforeApply` | `1` | An unattended live-update apply (in `UpdateWindow`) backs up to the destination first and is cancelled without a successful backup; an apply you start yourself takes none | Since 6.3.4 these are stored in the database: set them on the Backup & restore page, with `PUT /api/v1/settings/ini/{name}` or `hmctl ini set`, not in the file ([how the settings store works](/topic/28222)). **Retention has four rules:** nothing is deleted until the replacement has completed successfully; the newest archive is never deleted; the age rule always leaves the two newest alone; and only files whose names the server generated are considered, the timestamp read from the name. A scheduled run certain to fail (no destination, nothing selected, database unavailable, destination unreachable) is refused before it starts and backs off 5, 10, 20, 40 and then 60 minutes. *Test backup directory* on the Control Panel's Diagnostics page catches a schedule that has been failing silently. ### Encrypted archives, since 6.3.4 Off until a passphrase is set; then every archive is 7-Zip AES-256 with its headers encrypted (`-mhe=on`), so it cannot even be listed without the passphrase. Set it in the Control Panel (**Backup & restore → Encryption**), the Deck's Backup view, `encryption_passphrase` in `PUT /api/v1/settings/backup`, or `hmctl backup passphrase set`. It is 12 to 256 printable ASCII characters, kept in the database under the machine's secret store (DPAPI on Windows, the key file on Linux), never written into the archive, a command line or a log, and never read back. **If it is lost, everything in every archive written with it is lost.** Changing it does not touch archives already written, so write the passphrase and its date down together, away from the backups. **What it changes about a run.** Compression must be on when messages are backed up, or the run is refused. Staging moves into a directory under `TempDirectory`, so the temp volume needs room for the message store plus a little, and the run is refused with both numbers if it is not there. A stored passphrase this machine cannot open fails the backup; an unencrypted copy is never written in its place. `7zr`, the reduced p7zip build, cannot encrypt: a Linux server needs 7-Zip (`7zz`) or p7zip-full. The archive's file name, its size and the fact that a backup ran are not hidden. ### Restoring the whole server **Phase one decides and changes nothing.** Since 6.3.4 the archive is listed first (`7za l -slt`, the headers only): an entry with an absolute path, a `..` segment, or a link with somewhere to point refuses the whole archive as **HM7540**, because no backup the server writes has such an entry. Then the archive must be one of ours, written by an older or equal version (a newer one is refused, with no override), and hold what you asked to restore; then the store is extracted to a scratch directory. **Phase two acts:** delete every domain, delete every public folder, empty the data directory and copy the staged store in, load domains and accounts, load settings, reinitialise. A failure here leaves a half-restored server. The listing check works whichever 7-Zip is installed; the Windows installer's is 26.03 in 6.3.4, replacing 19.00. **Where.** The Control Panel's Restore card and COM (`BackupManager.LoadBackup`, `LoadEncryptedBackup`) on Windows; `POST /api/v1/backup/restore` with `file`, `passphrase` and the three parts, which is the only restore a Linux server has; the Deck's Backup view; `hmctl backup restore FILE --domains --messages --passphrase-file FILE`. The REST route's check is bounded at two minutes; past that the route answers 504 and starts nothing. 1. Install the same version, at the same drive letter and directory layout. 2. Restore or recreate `[Database]` in `hMailServer.INI` and confirm you can connect. 3. Choose the `.7z`, tick the categories, start, and watch `hmailserver_backup.log`. 4. Copy back what the archive could not hold: certificate, key and DKIM files, and `PasswordPepper`. 5. Re-enter the per-domain relay settings. 6. Check **Server status**, then **Diagnostics**. 7. Send a message in from outside, read it over IMAP, send one out. Rehearse this yearly on a scratch machine against a new, empty database, never production, and keep the list of what you fixed by hand with the backup. ### Restoring one mailbox, or one message, since 6.3.4 An archive in the backup destination can give back one account's mailbox, every folder and message it held when the backup was taken, or one message of it, and touch nothing else: not the account's settings, rules, contacts or calendars, and no other account. | Where | How | |---|---| | REST | `GET /api/v1/backup/archives`, `GET /api/v1/backup/{archive}/mailboxes/{address}`, `POST .../restore` with `mode`, or `message` and `folder`, `GET /api/v1/backup/restore-mailbox/{job}` | | Control Deck, and Control Panel (Windows) | The **Restore a mailbox** and **Restore one message** cards | | COM (Windows) | `BackupManager.RestoreMailbox`, `RestoreMessage`, `DescribeMailbox`, `MailboxRestoreStatus` | | hmctl | `hmctl backup archives`, `hmctl backup mailbox ARCHIVE ADDRESS`, `hmctl backup restore-mailbox ARCHIVE ADDRESS --merge\|--replace [--message ID [--folder F]]`, `hmctl backup restore-status JOB` | **Which archives.** Only one inside the destination, named as the listing names it and never by an arbitrary path. It must have been taken with domains and messages; an archive taken with `BackupMessagesDBOnly` holds rows and no files, and the job fails. An encrypted one needs its passphrase. **Merge** creates each archived folder that is missing and adds each archived message whose file is not in the live mailbox; nothing is removed, and merging twice adds nothing. **Replace** empties every live folder first, so the mailbox is the archive's and mail that arrived since the backup is gone; what it removes is kept only for `RecoverDeletedDays`, which is off by default. Replace is refused with 400 when the archive holds no messages for the account. **The identity rule.** Every message file is named once, with a GUID, and the same name anywhere in the live mailbox is the same message: a message the person moved is not brought back to its old folder, and one they deleted is. A restored message keeps its flags, keywords and dates, gets a fresh UID, and is charged to the account as a delivery is. **Refused, and counted.** The account must exist on this server and the archive must hold it (404); restoring into a different account is not offered, and neither is a public folder. A second restore while a backup or restore runs is 409. The outcome (`folders`, `added`, `skipped`, `failed`) is in the job's status and the backup log. Restoring needs `manage_server`; listing an account's messages out of an archive needs `view_mailbox`. ### Where to read more - Wiki: [Backup and Restore](https://gitlab.com/hmailserver/hmailserver/-/wikis/Backup-and-Restore), with COM commands in section 15.9; [Installing on Linux](https://gitlab.com/hmailserver/hmailserver/-/wikis/Installing-on-Linux) for the Linux paths and key file. - Repository: [Backup.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Backup.md); [SecretsAtRest.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SecretsAtRest.md). - The public documentation: [Backup and restore](https://www.progressiverobot.com/hmailserver-documentation/#15-backup-and-restore), [Backups on Linux](https://www.progressiverobot.com/hmailserver-documentation/#3512-backups-on-linux). - On this forum: [Moving to new hardware](/topic/28194), [Upgrading to 6.3.4](/topic/28193), and [Hardening a public mail server](/topic/28241) for exporting the stored-secrets key. --- Canonical: https://www.hmailserver.co.uk/topic/28221/backup-and-restore-what-is-in-the-archive-what-is-not-and-restoring-one-mailbox # Backup and restore: what is in the archive, what is not, and restoring one mailbox This topic covers the built-in backup, what its archive holds and leaves out, and restoring a server, one mailbox or one message. Know this first: **a restore is destructive and has no transaction**. It deletes every domain, every account and every public folder before it puts the archive's back, so whatever an account has that is not in the archive is deleted and not replaced. And the archive **does not include your messages unless you tick the box**. ### What one run writes One 7-Zip archive per run, named `HMBackup YYYY-MM-DD HHMMSS.7z`, in the configured destination: an XML index and, with **Back up messages** on, the whole data directory beside it. The data directory is `C:\Program Files\hMailServer\Data` by default on Windows and `/var/lib/hmailserver` on Linux, and what lives inside it travels with the messages: Sieve scripts and their state (`\Sieve`), and ACME certificates and keys (`\ACME`, unless `AcmeCertificateDirectory` moved them). Since 6.3.4, messages held in an object store (`MessageStoreBackend`) are read into the archive too: a store that cannot be read fails the backup, a restore puts the files back on the local disk, and the store's credentials are not carried. **What it cannot hold: files the database only points at.** Manually placed TLS certificates, private keys and DKIM keys are stored as *paths*. Restored onto a machine where those paths hold nothing, the server starts, serves no TLS and signs nothing. Keep them inside the data directory, or copy them yourself. ### What is in the archive | Area | Carried | |---|---| | Domains | Aliases, distribution lists and members, signatures, DKIM selectors and key *paths*; since 6.3.4 the domain's allow and block lists, its spam traps, the domain transforms, the S/MIME authority and the recovery key of a domain encrypted at rest | | Accounts | Address, password hash and its algorithm, forwarding, vacation, signature, quota, admin level, anti-spam thresholds, TOTP secret | | Per account, every release | Rules, fetch accounts with their passwords, application passwords as hashes and, with **Back up messages**, folders and messages with UID and flags | | Per account, since 6.3.4 | Message keywords (the webmail's labels, second and third stars, follow-up flags, mute and pin), the address book, webmail preferences, scheduled sends and snoozes, file-link records, S/MIME keys, the password history, calendars with their events and tasks, the first-contact memory, safe and blocked senders, the mailbox key of an account encrypted at rest, the role assignment | | Public folders | With the permissions on them. [Backup.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Backup.md) states no condition; section 15.2 of the wiki lists them under **Back up settings**. A restore deletes every public folder first | | Settings, with **Back up settings** | The `hm_settings` property set and the `[Settings]` rows of `hm_inisettings`; routes, IP ranges, global rules, TCP/IP ports, SSL certificate *records*, incoming relays, groups, blocked attachments, the DNSBL, SURBL and white lists; since 6.3.4 the remote domain policies and IP warm-up schedules | | Custom administrator roles | With domains or settings; restored first, so the assignments accounts carry find their role | **Nothing identity-shaped is written.** A restore gives every row a new id, which makes the archive portable between database backends; a reference that cannot be resolved drops that one row and names the account in `hmailserver_backup.log`. **Secrets travel opened.** Route, fetch-account and SSL private-key passwords are carried in a portable encoding and protected again on the restoring machine; since 6.3.4 each TOTP secret and the `[Settings]` secrets (`SRSSecret`, `OAuth2HmacSecret` and the rest) are carried opened and sealed again under the restoring server's stored-secrets key, so a restore onto new hardware keeps every second factor. An unencrypted archive therefore hands those secrets to whoever can read the destination folder. **In 6.3.3 and earlier** the per-account rows marked *since 6.3.4* are in no archive, and a restore deletes them. Until you upgrade, back up the database itself with its own tool (`pg_dump`, `mysqldump`, SQL Server's backup, or a copy of `hMailServer.sdf` with the service stopped), restore from that when recovering a whole server, and tell users before they find their address book and labels gone. An archive written by 6.3.3 or earlier restores on a newer server as it always did. ### What is not in the archive, deliberately | Not carried | Consequence of a restore | |---|---| | `[Database]` and `[Directories]` in `hMailServer.INI`; the administrator password and its second factor | Per machine. Keep a copy of the file, and never copy one machine's file over another's | | `PasswordPepper` (since 6.3.4) | Kept in the file only. A restore onto new hardware needs the same pepper in that machine's file before an Argon2id or scrypt password can sign in; PBKDF2, SHA-256 and MD5 hashes, and the administrator password, do not use it | | The stored-secrets key (since 6.3.4) | The archive does not need it. A *database-level* backup does: restored without the exported key, every second factor and `[Settings]` secret in it is unopenable. Export it once (`hMailServer.exe --export-stored-secrets-key FILE`, elevated, or `sudo hmailserver --export-stored-secrets-key FILE`) and keep it with the database backups, stored apart from them | | On Linux, the stored-secret key file `/.hmailserver-secret-key` | The archive does not carry it. Its passwords travel in a portable encoding and do not need it, but a domain's S/MIME authority key, and the mailbox key of an account encrypted at rest that has not yet signed in, are carried as envelopes that open only where this file is: restored without it, the authority must be rotated before it signs. A database-level backup needs it as well: back it up with the database, because a secret moved to a machine without it cannot be opened (HM6412) | | The delivery queue | Queued files are copied with the data directory, but nothing re-queues them. Let the queue drain before a migration | | Per-domain relay settings, and per-domain message retention days | Silently lost, and silently reset to `0` (keep for ever). Re-enter them | | Greylisting triplets; auto-ban and login-failure state | Senders are greylisted afresh; bans lapse. Permanent IP ranges *are* carried | | Quarantine, message trace, metric history; since 6.3.4 the audit trail, alert rules and events | Operational records. Back the database up with its own tool if you need them | | Deleted items kept for recovery, eDiscovery cases and holds, messages that expire, password-protected messages | The `Recoverable`, `Discovery`, `Confidential` and `Protected` folders are left out of the copy and left in place by a restore. A restore with domains is refused while an account or a domain is under a legal hold. Archives taken by earlier 6.3.4 builds, before `Confidential` and `Protected` were left out, *do* hold both: a message that has expired or been withdrawn since is still in them, so apply the mail's own retention to those archives, or delete them | | The full-text index, IMAP annotations, the QRESYNC expunge log | Rebuilt, or keyed on ids a restore reassigns. IMAP clients resynchronise | | Passkeys; calendar and address-book shares; the backup passphrase, the SNMP passwords, the object store's secret, the sandbox token | Enrol, share or enter them again | A domain named `recoverable`, `discovery`, `confidential` or `protected` keeps its mail in a folder the backup leaves out. Such names are refused for a new domain since 6.3.4, and an existing one is reported as HM7597; rename it before relying on the backup. ### The options, and the combinations that fail **Maintenance → Backup & restore** in the Control Panel (Windows-only), the Control Deck's Backup view, `PUT /api/v1/settings/backup` and `hmctl settings set backup` set the destination and what goes in. A fresh install seeds `backupoptions` as `0`: nothing is ticked. On a packaged Linux server the destination has to be inside the unit's `ReadWritePaths`, or the write fails; add it with a drop-in. | Option | What it changes | |---|---| | **Back up domains** | The domain tree goes into the index. Required for messages to mean anything | | **Back up messages** | Copies the whole data directory. Only with *Back up domains* as well: messages without domains stages nothing and then fails | | **Message metadata only** (`BackupMessagesDBOnly=1`) | Records which messages exist, not their contents; a restore expects the files still to be in the data directory. It governs the restore as well as the backup, and must be the same for both | | **Back up settings** | The settings row above and, per the wiki, public folders | | **Compress destination files** | Puts the message files inside the archive. Off, every run overwrites one `DataBackup` folder beside the archives: exactly one generation of mail, whatever retention says | | **Verify each backup** (`BackupVerifyRestore=1`, the default) | The next section | Tick domains and messages together, always, and **Back up settings** as well if the installation has public folders. The index is written to `hMailServerBackup.xml`, added to the archive and deleted, so what you restore from is the `.7z`; the Control Panel's file picker defaults to `*.xml`, so choose *All files*. ### How a run is verified Every refusal happens before anything in the destination is written, or deletes only the archive this run created. A run refuses when a message file lies outside the data directory, when the destination is unreachable, and when the database is down or any read did not run, so an outage cannot produce a hollow archive. Then the archive is read back: 1. **The index opens**, through the same code a restore uses, and describes this run. If not, the backup fails and the archive is deleted. 2. **The per-account rows are all there** (since 6.3.4), counted per table. A difference fails the backup. 3. **The message store extracts** to exactly the files and bytes that were staged. Fewer is a truncated archive, whatever 7-Zip's exit code said. 4. **Every message row has a file.** Reported, never failed: mail flow does not stop for a backup. The log names the first ten. Steps 3 and 4 run only when domains and messages with their files are included and `BackupVerifyRestore` is 1. The wiki puts step 2 under the same conditions; [Backup.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Backup.md) says only that `BackupVerifyRestore=0` turns off steps 3 and 4. Step 3 costs disk: the store exists twice in the temp directory during the check, and a temp volume that cannot hold it skips the step with an explanation rather than failing a good backup. A successful run ends `Backup completed successfully.`; a failed one has a `BACKUP ERROR` line. ### Schedule and retention Since 6.2.25. Nothing is configured on a fresh install. | Setting | Default | Meaning | |---|---|---| | `ScheduledBackupTime` | empty (off) | `HH:MM`, local time; at most one backup per calendar day | | `ScheduledBackupIntervalMinutes` | `0` (off) | Used only when no daily time is set; measured against the newest archive in the destination | | `ScheduledBackupKeepCount` | `0` (keep all) | How many of this server's own archives to keep | | `ScheduledBackupMaxAgeDays` | `0` (keep all) | Delete archives older than this | | `UpdateBackupBeforeApply` | `1` | An unattended live-update apply (in `UpdateWindow`) backs up to the destination first and is cancelled without a successful backup; an apply you start yourself takes none | Since 6.3.4 these are stored in the database: set them on the Backup & restore page, with `PUT /api/v1/settings/ini/{name}` or `hmctl ini set`, not in the file ([how the settings store works](/topic/28222)). **Retention has four rules:** nothing is deleted until the replacement has completed successfully; the newest archive is never deleted; the age rule always leaves the two newest alone; and only files whose names the server generated are considered, the timestamp read from the name. A scheduled run certain to fail (no destination, nothing selected, database unavailable, destination unreachable) is refused before it starts and backs off 5, 10, 20, 40 and then 60 minutes. *Test backup directory* on the Control Panel's Diagnostics page catches a schedule that has been failing silently. ### Encrypted archives, since 6.3.4 Off until a passphrase is set; then every archive is 7-Zip AES-256 with its headers encrypted (`-mhe=on`), so it cannot even be listed without the passphrase. Set it in the Control Panel (**Backup & restore → Encryption**), the Deck's Backup view, `encryption_passphrase` in `PUT /api/v1/settings/backup`, or `hmctl backup passphrase set`. It is 12 to 256 printable ASCII characters, kept in the database under the machine's secret store (DPAPI on Windows, the key file on Linux), never written into the archive, a command line or a log, and never read back. **If it is lost, everything in every archive written with it is lost.** Changing it does not touch archives already written, so write the passphrase and its date down together, away from the backups. **What it changes about a run.** Compression must be on when messages are backed up, or the run is refused. Staging moves into a directory under `TempDirectory`, so the temp volume needs room for the message store plus a little, and the run is refused with both numbers if it is not there. A stored passphrase this machine cannot open fails the backup; an unencrypted copy is never written in its place. `7zr`, the reduced p7zip build, cannot encrypt: a Linux server needs 7-Zip (`7zz`) or p7zip-full. The archive's file name, its size and the fact that a backup ran are not hidden. ### Restoring the whole server **Phase one decides and changes nothing.** Since 6.3.4 the archive is listed first (`7za l -slt`, the headers only): an entry with an absolute path, a `..` segment, or a link with somewhere to point refuses the whole archive as **HM7540**, because no backup the server writes has such an entry. Then the archive must be one of ours, written by an older or equal version (a newer one is refused, with no override), and hold what you asked to restore; then the store is extracted to a scratch directory. **Phase two acts:** delete every domain, delete every public folder, empty the data directory and copy the staged store in, load domains and accounts, load settings, reinitialise. A failure here leaves a half-restored server. The listing check works whichever 7-Zip is installed; the Windows installer's is 26.03 in 6.3.4, replacing 19.00. **Where.** The Control Panel's Restore card and COM (`BackupManager.LoadBackup`, `LoadEncryptedBackup`) on Windows; `POST /api/v1/backup/restore` with `file`, `passphrase` and the three parts, which is the only restore a Linux server has; the Deck's Backup view; `hmctl backup restore FILE --domains --messages --passphrase-file FILE`. The REST route's check is bounded at two minutes; past that the route answers 504 and starts nothing. 1. Install the same version, at the same drive letter and directory layout. 2. Restore or recreate `[Database]` in `hMailServer.INI` and confirm you can connect. 3. Choose the `.7z`, tick the categories, start, and watch `hmailserver_backup.log`. 4. Copy back what the archive could not hold: certificate, key and DKIM files, and `PasswordPepper`. 5. Re-enter the per-domain relay settings. 6. Check **Server status**, then **Diagnostics**. 7. Send a message in from outside, read it over IMAP, send one out. Rehearse this yearly on a scratch machine against a new, empty database, never production, and keep the list of what you fixed by hand with the backup. ### Restoring one mailbox, or one message, since 6.3.4 An archive in the backup destination can give back one account's mailbox, every folder and message it held when the backup was taken, or one message of it, and touch nothing else: not the account's settings, rules, contacts or calendars, and no other account. | Where | How | |---|---| | REST | `GET /api/v1/backup/archives`, `GET /api/v1/backup/{archive}/mailboxes/{address}`, `POST .../restore` with `mode`, or `message` and `folder`, `GET /api/v1/backup/restore-mailbox/{job}` | | Control Deck, and Control Panel (Windows) | The **Restore a mailbox** and **Restore one message** cards | | COM (Windows) | `BackupManager.RestoreMailbox`, `RestoreMessage`, `DescribeMailbox`, `MailboxRestoreStatus` | | hmctl | `hmctl backup archives`, `hmctl backup mailbox ARCHIVE ADDRESS`, `hmctl backup restore-mailbox ARCHIVE ADDRESS --merge\|--replace [--message ID [--folder F]]`, `hmctl backup restore-status JOB` | **Which archives.** Only one inside the destination, named as the listing names it and never by an arbitrary path. It must have been taken with domains and messages; an archive taken with `BackupMessagesDBOnly` holds rows and no files, and the job fails. An encrypted one needs its passphrase. **Merge** creates each archived folder that is missing and adds each archived message whose file is not in the live mailbox; nothing is removed, and merging twice adds nothing. **Replace** empties every live folder first, so the mailbox is the archive's and mail that arrived since the backup is gone; what it removes is kept only for `RecoverDeletedDays`, which is off by default. Replace is refused with 400 when the archive holds no messages for the account. **The identity rule.** Every message file is named once, with a GUID, and the same name anywhere in the live mailbox is the same message: a message the person moved is not brought back to its old folder, and one they deleted is. A restored message keeps its flags, keywords and dates, gets a fresh UID, and is charged to the account as a delivery is. **Refused, and counted.** The account must exist on this server and the archive must hold it (404); restoring into a different account is not offered, and neither is a public folder. A second restore while a backup or restore runs is 409. The outcome (`folders`, `added`, `skipped`, `failed`) is in the job's status and the backup log. Restoring needs `manage_server`; listing an account's messages out of an archive needs `view_mailbox`. ### Where to read more - Wiki: [Backup and Restore](https://gitlab.com/hmailserver/hmailserver/-/wikis/Backup-and-Restore), with COM commands in section 15.9; [Installing on Linux](https://gitlab.com/hmailserver/hmailserver/-/wikis/Installing-on-Linux) for the Linux paths and key file. - Repository: [Backup.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Backup.md); [SecretsAtRest.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SecretsAtRest.md). - The public documentation: [Backup and restore](https://www.progressiverobot.com/hmailserver-documentation/#15-backup-and-restore), [Backups on Linux](https://www.progressiverobot.com/hmailserver-documentation/#3512-backups-on-linux). - On this forum: [Moving to new hardware](/topic/28194), [Upgrading to 6.3.4](/topic/28193), and [Hardening a public mail server](/topic/28241) for exporting the stored-secrets key. --- Canonical: https://www.hmailserver.co.uk/topic/28222/settings-what-is-not-stock-when-a-save-takes-effect-and-the-three-levels # Settings: what is not stock, when a save takes effect, and the three levels This topic is about the settings themselves, whichever interface you use: where a setting is stored, which of yours are not what 6.3.4 ships with, when a saved value is in effect, and how the twenty-seven settings a domain or an account may hold its own value of combine with the server's. The one thing to know first: since 6.3.4 the database is the settings store. A `[Settings]` line edited by hand in `hMailServer.ini` changes nothing; at the next start the server uses the stored value, names the key in the error log as `HM5804`, and, where the service may write the file, puts the line back. Everything below describes 6.3.4 unless it says otherwise; a server on 6.3.3 still reads the file and applies a change at the next start. ### Two stores, and how a setting is changed **The table `hm_settings`.** Everything the COM `Settings` object has always carried, such as the anti-spam thresholds and the delivery options. Since 6.3.4 every new setting goes here, and no key is added to `[Settings]`. 6.3.4 added 193 rows for its features. **The `[Settings]` keys.** 241 keys read by the settings loader, plus `UseLanguage`, read per call. On 6.3.3 they live in the file, mirrored into `hm_inisettings`. Since 6.3.4 (schema 6042) `hm_inisettings` is the store and the file's section is a copy, which the server keeps current where the service may write the file. **Where the service cannot write the file, the copy goes stale.** That is a new Windows installation, whose service runs as `NT SERVICE\hMailServer`, and a Linux package installation, where `/etc/hmailserver/hMailServer.ini` is `0640 root:hmailserver`. There the copy is not refreshed and an edited line is not put back; the database's values are still the ones used. The server reports this once a start in the application log, not as an error, and `ServiceAccount.md` says the same of the Linux packages; the wiki's Installing on Linux page names `HM5806` at a save and `HM5803` at every start for them instead. On such a server read a setting from the store, not from the file. **The migration is the first start.** A key the table has never seen is adopted with the file's value, and there is nothing to run by hand. A line edited after the old server last started, including one edited to prepare the upgrade, is taken for an edit and put back, so make such a change after upgrading. A 6.3.3 server refuses a database at schema 6042 or later with `HM5011`; see [Upgrading to 6.3.4](/topic/28193). **What stays in the file.** `[Database]`, `[Directories]`, `[Security]` and `[GUILanguages]` are the bootstrap, read before the database is open and never in the store. `[LDAP]`, `[SendingLimits]` and `[SendingLimitsOverrides]` stay in the file. `PasswordPepper` is kept in the file only, never in the database and never in a backup. `[SettingsOverride]` is the one way to make the file win, for a database that is unreachable or holds a value the server will not start on: a key written there is applied over whatever is stored and named in the error log as `HM5807` at every start, because no page shows it and no backup carries it. Storing a value for a key an override shadows is accepted and reported as `HM5808`. | Surface | Set a `[Settings]` key | Return it to the default | |---|---|---| | Control Panel (Windows only) | the key's editor, written over COM even on the server itself | No. Clearing a box stores an empty value, which a number reads as `0` | | REST | `PUT /api/v1/settings/ini/{name}` with `{"value": "..."}` | `DELETE /api/v1/settings/ini/{name}` | | `hmctl`, over REST | `hmctl ini set ` | `hmctl ini delete ` | | PowerShell module | `Set-HmIniSetting` | none | | COM (Windows only) | `Settings.SetIniSetting(name, value)` | `Settings.DeleteIniSetting(name)` | | Linux command line, as root | `hmailserver --set-setting NAME=VALUE`, with no listener running. A secret's value is refused on the command line: give the name alone and type the value at the prompt | none | | Control Deck | only `SmtpTarpitCount`, `SmtpTarpitDelaySeconds`, `UseLanguage` and `RewriteEnvelopeFromWhenForwarding` | none | The REST ini routes take the administrator password only; an API key of any scope is refused. Since 6.3.4 a setting the environment names, as `HM_SETTING_` or `HM_PROPERTY_` ([configuration from the environment](/topic/28231)), refuses an administrator's change, 409 over REST, until a start without the variable. ### What this server has changed from stock **Since 6.3.4.** One list of every setting whose value is not the one this version ships with, with the shipped value beside it. The example in `ChangedFromStock.md`: ```text STORE NAME VALUE DEFAULT ini LogLevel 5 9 ini MaxLogLineLen 2000 500 ini SMTPDMaxTimeout 600 1800 ini OAuth2HmacSecret (set) ini DnsServer 10.0.0.1 (forced from [SettingsOverride]) database smtpnoofretries 6 4 database maxsmtpconnections 200 0 7 of 391 settings are not what this version ships with. ``` | Where | What to ask | |---|---| | Control Deck | **Changed from stock**, in the navigation beside Settings | | REST | `GET /api/v1/settings/changed` | | Command line | `hmctl config changed` | | PowerShell | `Get-HmChangedSetting` | | COM | `Settings.GetChangedFromDefault` | The REST route and the COM call answer the same document. `store` is `ini` (`hm_inisettings`) or `database` (`hm_settings`). **What "default" means.** The value the server behaves as though the setting has when nothing has been stored for it, not what a fresh installation's files contain. `hMailServer.ini` ships with lines the packaging wrote, such as `RestApiBindAddress=127.0.0.1`, and a line holding the shipped value is not a change; one holding another value is listed. **An upgraded installation lists more than a new one.** `database` defaults are what a database created by this version holds. An installation upgraded from an old schema got its values from the upgrade scripts of the time, and where those differ (`maxmessagesize`, `usecache`, `protocolimap`, `smtpallowincorrectlineendings` and half a dozen more) the older installation is genuinely different and is listed. A row nothing reads any more (`useordb`, `usespamhaus`, `tarpitcount`, `filters`, `imapport`, `relaymode`) is not reported at all. **The marks on an entry.** - `secret`: a setting whose name says its value is a secret is listed by name, with `(set)` or empty, and never shown. A key file, such as `RestApiPrivateKeyFile`, is a path and is shown. - `known` false: the server holds a value and ships no default for it, such as a key stored for a script of your own. `hmctl` prints `(none: this server does not read it)`. - `overridden`: `[SettingsOverride]` is forcing the value. Such a key is listed even when neither store holds anything for it; `hmctl` prints `(forced from [SettingsOverride])`. - `environment` is `variable` or `file`, and `hmctl` prints `(set by the environment)`. `profile` names the setting profile whose apply wrote the value, while the setting still holds it. **Which values these are.** The value stored, not the value the server settled on where the two differ: a `PasswordHashIterations` outside the range the server accepts is reported as `HM5561` and the built-in work factor used; an unusable `PreferredHashAlgorithm` is reported as `HM5528` (or `HM5607`, for Argon2id on a build whose OpenSSL has none). **Who may see it.** The administrator password, and nothing else. An API key of any scope is refused with a 401, and an account holding an administrative role, an `auditor` included, with a 403, because the list includes commands the server runs: `AutoBanCommand` runs as the service account. The feature changes nothing. It does not compare two servers (two `hmctl config changed --json` answers and `diff` do), and it does not say when or by whom a setting was changed; [the audit trail](/topic/28273) does (`GET /api/v1/audit?object_type=setting`). ### When a saved setting takes effect **Since 6.3.4.** Until then every `[Settings]` key was latched when the service started. Now a save publishes: the store takes the value, the server rebuilds its settings from the rows, and every key whose readers can take a new value is in effect when the save returns, whichever door saved it. Every key carries one of three marks: | Mark | Keys | What it means | |---|---|---| | `now` | 211 | A save applies it | | `reinitialize` | 28 | Bound into something the servers build when they start, so it waits for a reinitialise: **Apply now** in the Control Panel or the Deck, `POST /api/v1/server/reinitialize`, or `Application.Reinitialize` over COM | | `restart` | 31 | Read once per process: 2 `[Settings]` keys and the 29 bootstrap keys (20 `[Database]`, 5 `[Directories]`, 3 `[Security]`, 1 `[GUILanguages]`) | **The 28 that wait for a reinitialise.** | What is rebuilt | Keys | |---|---| | The REST listener | `RestApiBindAddress`, `RestApiPort`, `RestApiCertificateFile`, `RestApiPrivateKeyFile` | | The metrics listener | `MetricsServerBindAddress`, `MetricsServerPort`, `MetricsServerCertificateFile`, `MetricsServerPrivateKeyFile`, `MetricsServerAuthPassword`, `MetricsServerAuthToken`, `MetricsServerAuthUsername` | | The web-services listener | `WebServicesBindAddress`, `WebServicesHttpPort`, `WebServicesHttpsPort`, `WebServicesCertificateFile`, `WebServicesPrivateKeyFile` | | The ManageSieve listener | `ManageSieveServerBindAddress`, `ManageSieveServerPort` | | The SSL contexts of the SMTP, IMAP, POP3 and ManageSieve listeners and the outbound client | `TlsCipherSuites13`, `TlsKeyExchangeGroups`, `TlsSessionCacheSize`, `TlsSessionTicketsEnabled`, `TlsSessionTimeoutSeconds`, `TlsTicketKeyRotationSeconds` | | ACME, the database pool, the fetch queue | `AcmeEnabled`, `AcmeCertificateDirectory`, `DatabaseStatementTimeout`, `MaxNumberOfExternalFetchThreads` | **The 2 that wait for a restart** are `ServiceAccountName` and `ServiceAccountPassword`, which are retired: since 6.3.4 `hMailServer.exe /Register` takes the account from its command line alone, and a stored value is shown as pending until the next start. The bootstrap keys are never in the store, so none is ever pending. **Some keys are `now` because a subscriber re-latches them.** `MaxSubmissionsPerIPPerMinute` and `MaxOutboundPerDestinationPerMinute` start each limit's one-minute window again, so a saved limit counts from when it was saved. A cache of data is not a setting: a changed `DNSServer` is used for the next lookup, and an answer already cached stays until its TTL runs out. **Pending.** A `reinitialize` or `restart` key whose stored value is not the value the server is running on is pending. The list is recomputed at every request and never remembered, so a value put back before anything applied it is not pending. | Where | What | |---|---| | REST | `GET /api/v1/server/settings/pending` answers `[{name, stored, running, applies, secret}]`, and `applies` and `pending` come on every `/api/v1/settings/ini/{name}` answer. The administrator password's route | | COM | `Settings.IniSettingApplies(name)`, `Settings.IniSettingAppliesAll`, `Settings.PendingIniSettings` | | Control Panel | **Applies on save**, **After reinitialize** or **After a restart** under each field kept in the store, and a pending bar with **Apply now** | | Control Deck | The same words as a badge, and **Apply now** above its Settings view | A name the loader does not read (`UseLanguage`, a key a script of your own reads, a misspelling) has no mark: COM's `IniSettingApplies` refuses it and REST answers `applies: null`. **Apply now is a reinitialise.** Every service stops and starts again in place, which takes every `reinitialize` key and not a `restart` key. It asks first. A reinitialise ends maintenance mode, and `POST /api/v1/server/reinitialize` answers `{"reinitializing": true, "drained": true}` when it ended a drain. The Deck's session ends with the REST listener it was using; the Control Panel's COM session survives. **What a save does not do.** It does not re-read the bootstrap sections or `[SettingsOverride]`; a reinitialise or a start re-reads the override section and announces it as `HM5807`. It does not reconcile the file against the table; that is a start's and a reinitialise's only. And a value stored by another process (another node on the same database, or `hmailserver --set-setting` while the service runs) reaches this server at the next save here, a reinitialise or a restart, because nothing watches the table. `hmctl` has no pending verb. **On Linux** the regression fixture alone proves the marks, with the three COM members answered over REST; the internal tester that swaps settings under load does not run there. ### One value, three levels **Since 6.3.4** (schema 6088). Twenty-seven settings may be set for the server, for a domain and for an account, and every place that shows one shows what is in effect and where it came from. For an account: its own value, else its domain's, else the server's. Every setting on the list is exactly one kind: | Kind | How many | The rule | Who sets a level below the server | |---|---|---|---| | Limit | 15 | A level below may tighten the level above and never loosen it. A looser value is refused when it is written; a value the level above has since tightened past is capped when it is read | Anybody who administers that level: a domain administrator, a key restricted to the domain | | Gate | 4 | The deepest level that holds a value wins, either way | The server administrator alone; a domain-restricted key is refused with 403 | | Preference | 8 | The deepest level that holds a value wins | The domain's administrators, and a domain-restricted key | The last column is the rule for the levels held in `hm_settingoverrides`. A field the domain already had keeps the authority it had, whatever its kind: the domain's maximum message size and the domain's retention are the server administrator's. | Kind | Settings | |---|---| | Limits | `max_message_size_kb`, `session_idle_seconds`, `session_lifetime_seconds`, `session_one_per_device`, `session_bind_to_address`, `quota_grace_percent`, `quota_send_stop_percent`, `recall_days`, `recover_deleted_days`, `sending_max_messages`, `sending_max_recipients`, `reader_fetch_accounts_max`, `reader_sender_addresses_max`, `reader_masked_addresses_max`, `reader_offline_days_max` | | Gates | `plus_addressing_enabled`, `openpgp_portal_enabled`, `zero_access_allowed`, `directory_user_editable` | | Preferences | `quarantine_digest`, `quarantine_digest_hour`, `external_tag_subject`, `external_tag_header`, `impersonation_action`, `quota_warning_percent`, `country_policy`, `message_retention_days` | A server setting not on the list is the server's alone: its name is refused on the domain and account routes (400, `unknown field`) and by COM's `SettingOverride`. Of the `[Settings]` keys only `QuotaWarningPercent` has levels. **Where each level is held.** A field the domain or the account already had, such as the domain's `MaxMessageSize` or `MessageRetentionDays`, stays where it was, with the meaning it always had; its 0 or -1 is the level holding nothing. The rest are rows of `hm_settingoverrides`, where no row means "take the level above". An owner's rows are cached for sixty seconds; every write through the interfaces drops the cache at once, and so does `POST /api/v1/settings/cache/clear` or COM `Cache.Clear`. A row put in by hand that the interfaces would refuse is treated as holding nothing and reported as `HM7311`. Nothing changes for a domain or an account that holds no value of its own. | Where | What | |---|---| | REST | `GET /api/v1/settings/levels` lists the twenty-seven. `GET /api/v1/domains/{domain}` and `GET /api/v1/accounts/{address}` answer `_effective` and `_source`, and `` where the level is held in the overlay. For a level held in the overlay a `PUT` takes a value of the setting's own JSON type, and `null` makes it inherit; for a level held in a field of its own `null` leaves it unchanged, and the field's own 0 or -1 is "take the level above" | | COM (Windows only) | `Domain.SettingOverride(name)` and `Account.SettingOverride(name)`, an empty string for "take the level above", written at once and not by `Save`; `EffectiveSetting(name)` and `EffectiveSettingSource(name)` | | `hmctl` | `hmctl domain set ` and `hmctl account set
` | | Control Panel (Windows only) | A **Levels** tab on the domain and account dialogs, with "In effect: ..., from the ..." beside each setting and **Save levels now** | | Control Deck | The same card in its domain and account editors | **A saved level applies at once**, at every level and from every door, because the resolver is asked when the value is used (at `MAIL FROM`, when a session is checked) and never latched. **Deleting, backing up and moving.** Deleting an account or a domain deletes its rows; a delete that cannot remove them is reported as `HM7310` for a domain and `HM6109` for an account. The XML backup carries the rows, and on restore a row the build refuses is left out and reported as `HM7312`. A mailbox moved to another domain keeps its own values. **What is thinner than it sounds.** - Attachment blocking, web push, passkeys and account recovery have no levels. - The domain's session seconds and switches and its quota grace and send-stop shares could always be set looser than the server's and still can; the ceiling binds the account below them. - The maximum message size is resolved at `MAIL FROM` and at IMAP `APPEND`; the EHLO `SIZE` advertisement, the webmail's own send and the send API stay the server's. - The session policies are the browser sessions' (the webmail, the portal, the Deck); IMAP, POP3 and SMTP sessions have none. - `country_policy` cannot be set to empty over COM, where an empty string means "take the level above"; REST's `""` does it. - Twelve of the twenty-seven are proved by what the account and its domain answer rather than by driving the behaviour per account, and the levels' own regression fixture is Windows-only. ### When a setting appears to do nothing 1. Is it pending? A `reinitialize` key waits for **Apply now** and a `restart` key for a service restart. 2. Was it typed into `hMailServer.ini`? The stored value wins and `HM5804` names the key. Store it instead. 3. Is the key spelled exactly as the code reads it, and is a boolean literally `1`? An unknown key is the default, and only `1` is true: `2`, `yes` and `true` all read as false. 4. Does it need a companion? `SRSEnabled` needs `SRSSecret`, ACME needs `AcmeDomains`, autoconfig needs `WebServicesHttpPort` or `WebServicesHttpsPort`, and MTA-STS hosting needs the HTTPS one. 5. Does the error log show `HM5807` or `HM5808`? A `[SettingsOverride]` section is forcing a value. ### Where to read more - The tour: https://gitlab.com/hmailserver/hmailserver/-/wikis/Settings-Overview. Every key with its default, validation and mark: https://gitlab.com/hmailserver/hmailserver/-/wikis/Settings-Reference. Where the two disagree, the reference is generated from the code and is right. - The three documents behind this topic: https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/ChangedFromStock.md, https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SettingsApply.md and https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SettingLevels.md - The public documentation's settings chapter: https://www.progressiverobot.com/hmailserver-documentation/#21-settings-reference-hmailserverini - On this forum: [the Control Panel on Windows](/topic/28223), [`hmctl` and the PowerShell module](/topic/28265), [the `hmailserver` command line](/topic/28245), [the Control Deck](/topic/28192), [Upgrading to 6.3.4](/topic/28193) and [hMailServer on Linux](/topic/28189). --- Canonical: https://www.hmailserver.co.uk/topic/28222/settings-what-is-not-stock-when-a-save-takes-effect-and-the-three-levels # Settings: what is not stock, when a save takes effect, and the three levels This topic is about the settings themselves, whichever interface you use: where a setting is stored, which of yours are not what 6.3.4 ships with, when a saved value is in effect, and how the twenty-seven settings a domain or an account may hold its own value of combine with the server's. The one thing to know first: since 6.3.4 the database is the settings store. A `[Settings]` line edited by hand in `hMailServer.ini` changes nothing; at the next start the server uses the stored value, names the key in the error log as `HM5804`, and, where the service may write the file, puts the line back. Everything below describes 6.3.4 unless it says otherwise; a server on 6.3.3 still reads the file and applies a change at the next start. ### Two stores, and how a setting is changed **The table `hm_settings`.** Everything the COM `Settings` object has always carried, such as the anti-spam thresholds and the delivery options. Since 6.3.4 every new setting goes here, and no key is added to `[Settings]`. 6.3.4 added 193 rows for its features. **The `[Settings]` keys.** 241 keys read by the settings loader, plus `UseLanguage`, read per call. On 6.3.3 they live in the file, mirrored into `hm_inisettings`. Since 6.3.4 (schema 6042) `hm_inisettings` is the store and the file's section is a copy, which the server keeps current where the service may write the file. **Where the service cannot write the file, the copy goes stale.** That is a new Windows installation, whose service runs as `NT SERVICE\hMailServer`, and a Linux package installation, where `/etc/hmailserver/hMailServer.ini` is `0640 root:hmailserver`. There the copy is not refreshed and an edited line is not put back; the database's values are still the ones used. The server reports this once a start in the application log, not as an error, and `ServiceAccount.md` says the same of the Linux packages; the wiki's Installing on Linux page names `HM5806` at a save and `HM5803` at every start for them instead. On such a server read a setting from the store, not from the file. **The migration is the first start.** A key the table has never seen is adopted with the file's value, and there is nothing to run by hand. A line edited after the old server last started, including one edited to prepare the upgrade, is taken for an edit and put back, so make such a change after upgrading. A 6.3.3 server refuses a database at schema 6042 or later with `HM5011`; see [Upgrading to 6.3.4](/topic/28193). **What stays in the file.** `[Database]`, `[Directories]`, `[Security]` and `[GUILanguages]` are the bootstrap, read before the database is open and never in the store. `[LDAP]`, `[SendingLimits]` and `[SendingLimitsOverrides]` stay in the file. `PasswordPepper` is kept in the file only, never in the database and never in a backup. `[SettingsOverride]` is the one way to make the file win, for a database that is unreachable or holds a value the server will not start on: a key written there is applied over whatever is stored and named in the error log as `HM5807` at every start, because no page shows it and no backup carries it. Storing a value for a key an override shadows is accepted and reported as `HM5808`. | Surface | Set a `[Settings]` key | Return it to the default | |---|---|---| | Control Panel (Windows only) | the key's editor, written over COM even on the server itself | No. Clearing a box stores an empty value, which a number reads as `0` | | REST | `PUT /api/v1/settings/ini/{name}` with `{"value": "..."}` | `DELETE /api/v1/settings/ini/{name}` | | `hmctl`, over REST | `hmctl ini set ` | `hmctl ini delete ` | | PowerShell module | `Set-HmIniSetting` | none | | COM (Windows only) | `Settings.SetIniSetting(name, value)` | `Settings.DeleteIniSetting(name)` | | Linux command line, as root | `hmailserver --set-setting NAME=VALUE`, with no listener running. A secret's value is refused on the command line: give the name alone and type the value at the prompt | none | | Control Deck | only `SmtpTarpitCount`, `SmtpTarpitDelaySeconds`, `UseLanguage` and `RewriteEnvelopeFromWhenForwarding` | none | The REST ini routes take the administrator password only; an API key of any scope is refused. Since 6.3.4 a setting the environment names, as `HM_SETTING_` or `HM_PROPERTY_` ([configuration from the environment](/topic/28231)), refuses an administrator's change, 409 over REST, until a start without the variable. ### What this server has changed from stock **Since 6.3.4.** One list of every setting whose value is not the one this version ships with, with the shipped value beside it. The example in `ChangedFromStock.md`: ```text STORE NAME VALUE DEFAULT ini LogLevel 5 9 ini MaxLogLineLen 2000 500 ini SMTPDMaxTimeout 600 1800 ini OAuth2HmacSecret (set) ini DnsServer 10.0.0.1 (forced from [SettingsOverride]) database smtpnoofretries 6 4 database maxsmtpconnections 200 0 7 of 391 settings are not what this version ships with. ``` | Where | What to ask | |---|---| | Control Deck | **Changed from stock**, in the navigation beside Settings | | REST | `GET /api/v1/settings/changed` | | Command line | `hmctl config changed` | | PowerShell | `Get-HmChangedSetting` | | COM | `Settings.GetChangedFromDefault` | The REST route and the COM call answer the same document. `store` is `ini` (`hm_inisettings`) or `database` (`hm_settings`). **What "default" means.** The value the server behaves as though the setting has when nothing has been stored for it, not what a fresh installation's files contain. `hMailServer.ini` ships with lines the packaging wrote, such as `RestApiBindAddress=127.0.0.1`, and a line holding the shipped value is not a change; one holding another value is listed. **An upgraded installation lists more than a new one.** `database` defaults are what a database created by this version holds. An installation upgraded from an old schema got its values from the upgrade scripts of the time, and where those differ (`maxmessagesize`, `usecache`, `protocolimap`, `smtpallowincorrectlineendings` and half a dozen more) the older installation is genuinely different and is listed. A row nothing reads any more (`useordb`, `usespamhaus`, `tarpitcount`, `filters`, `imapport`, `relaymode`) is not reported at all. **The marks on an entry.** - `secret`: a setting whose name says its value is a secret is listed by name, with `(set)` or empty, and never shown. A key file, such as `RestApiPrivateKeyFile`, is a path and is shown. - `known` false: the server holds a value and ships no default for it, such as a key stored for a script of your own. `hmctl` prints `(none: this server does not read it)`. - `overridden`: `[SettingsOverride]` is forcing the value. Such a key is listed even when neither store holds anything for it; `hmctl` prints `(forced from [SettingsOverride])`. - `environment` is `variable` or `file`, and `hmctl` prints `(set by the environment)`. `profile` names the setting profile whose apply wrote the value, while the setting still holds it. **Which values these are.** The value stored, not the value the server settled on where the two differ: a `PasswordHashIterations` outside the range the server accepts is reported as `HM5561` and the built-in work factor used; an unusable `PreferredHashAlgorithm` is reported as `HM5528` (or `HM5607`, for Argon2id on a build whose OpenSSL has none). **Who may see it.** The administrator password, and nothing else. An API key of any scope is refused with a 401, and an account holding an administrative role, an `auditor` included, with a 403, because the list includes commands the server runs: `AutoBanCommand` runs as the service account. The feature changes nothing. It does not compare two servers (two `hmctl config changed --json` answers and `diff` do), and it does not say when or by whom a setting was changed; [the audit trail](/topic/28273) does (`GET /api/v1/audit?object_type=setting`). ### When a saved setting takes effect **Since 6.3.4.** Until then every `[Settings]` key was latched when the service started. Now a save publishes: the store takes the value, the server rebuilds its settings from the rows, and every key whose readers can take a new value is in effect when the save returns, whichever door saved it. Every key carries one of three marks: | Mark | Keys | What it means | |---|---|---| | `now` | 211 | A save applies it | | `reinitialize` | 28 | Bound into something the servers build when they start, so it waits for a reinitialise: **Apply now** in the Control Panel or the Deck, `POST /api/v1/server/reinitialize`, or `Application.Reinitialize` over COM | | `restart` | 31 | Read once per process: 2 `[Settings]` keys and the 29 bootstrap keys (20 `[Database]`, 5 `[Directories]`, 3 `[Security]`, 1 `[GUILanguages]`) | **The 28 that wait for a reinitialise.** | What is rebuilt | Keys | |---|---| | The REST listener | `RestApiBindAddress`, `RestApiPort`, `RestApiCertificateFile`, `RestApiPrivateKeyFile` | | The metrics listener | `MetricsServerBindAddress`, `MetricsServerPort`, `MetricsServerCertificateFile`, `MetricsServerPrivateKeyFile`, `MetricsServerAuthPassword`, `MetricsServerAuthToken`, `MetricsServerAuthUsername` | | The web-services listener | `WebServicesBindAddress`, `WebServicesHttpPort`, `WebServicesHttpsPort`, `WebServicesCertificateFile`, `WebServicesPrivateKeyFile` | | The ManageSieve listener | `ManageSieveServerBindAddress`, `ManageSieveServerPort` | | The SSL contexts of the SMTP, IMAP, POP3 and ManageSieve listeners and the outbound client | `TlsCipherSuites13`, `TlsKeyExchangeGroups`, `TlsSessionCacheSize`, `TlsSessionTicketsEnabled`, `TlsSessionTimeoutSeconds`, `TlsTicketKeyRotationSeconds` | | ACME, the database pool, the fetch queue | `AcmeEnabled`, `AcmeCertificateDirectory`, `DatabaseStatementTimeout`, `MaxNumberOfExternalFetchThreads` | **The 2 that wait for a restart** are `ServiceAccountName` and `ServiceAccountPassword`, which are retired: since 6.3.4 `hMailServer.exe /Register` takes the account from its command line alone, and a stored value is shown as pending until the next start. The bootstrap keys are never in the store, so none is ever pending. **Some keys are `now` because a subscriber re-latches them.** `MaxSubmissionsPerIPPerMinute` and `MaxOutboundPerDestinationPerMinute` start each limit's one-minute window again, so a saved limit counts from when it was saved. A cache of data is not a setting: a changed `DNSServer` is used for the next lookup, and an answer already cached stays until its TTL runs out. **Pending.** A `reinitialize` or `restart` key whose stored value is not the value the server is running on is pending. The list is recomputed at every request and never remembered, so a value put back before anything applied it is not pending. | Where | What | |---|---| | REST | `GET /api/v1/server/settings/pending` answers `[{name, stored, running, applies, secret}]`, and `applies` and `pending` come on every `/api/v1/settings/ini/{name}` answer. The administrator password's route | | COM | `Settings.IniSettingApplies(name)`, `Settings.IniSettingAppliesAll`, `Settings.PendingIniSettings` | | Control Panel | **Applies on save**, **After reinitialize** or **After a restart** under each field kept in the store, and a pending bar with **Apply now** | | Control Deck | The same words as a badge, and **Apply now** above its Settings view | A name the loader does not read (`UseLanguage`, a key a script of your own reads, a misspelling) has no mark: COM's `IniSettingApplies` refuses it and REST answers `applies: null`. **Apply now is a reinitialise.** Every service stops and starts again in place, which takes every `reinitialize` key and not a `restart` key. It asks first. A reinitialise ends maintenance mode, and `POST /api/v1/server/reinitialize` answers `{"reinitializing": true, "drained": true}` when it ended a drain. The Deck's session ends with the REST listener it was using; the Control Panel's COM session survives. **What a save does not do.** It does not re-read the bootstrap sections or `[SettingsOverride]`; a reinitialise or a start re-reads the override section and announces it as `HM5807`. It does not reconcile the file against the table; that is a start's and a reinitialise's only. And a value stored by another process (another node on the same database, or `hmailserver --set-setting` while the service runs) reaches this server at the next save here, a reinitialise or a restart, because nothing watches the table. `hmctl` has no pending verb. **On Linux** the regression fixture alone proves the marks, with the three COM members answered over REST; the internal tester that swaps settings under load does not run there. ### One value, three levels **Since 6.3.4** (schema 6088). Twenty-seven settings may be set for the server, for a domain and for an account, and every place that shows one shows what is in effect and where it came from. For an account: its own value, else its domain's, else the server's. Every setting on the list is exactly one kind: | Kind | How many | The rule | Who sets a level below the server | |---|---|---|---| | Limit | 15 | A level below may tighten the level above and never loosen it. A looser value is refused when it is written; a value the level above has since tightened past is capped when it is read | Anybody who administers that level: a domain administrator, a key restricted to the domain | | Gate | 4 | The deepest level that holds a value wins, either way | The server administrator alone; a domain-restricted key is refused with 403 | | Preference | 8 | The deepest level that holds a value wins | The domain's administrators, and a domain-restricted key | The last column is the rule for the levels held in `hm_settingoverrides`. A field the domain already had keeps the authority it had, whatever its kind: the domain's maximum message size and the domain's retention are the server administrator's. | Kind | Settings | |---|---| | Limits | `max_message_size_kb`, `session_idle_seconds`, `session_lifetime_seconds`, `session_one_per_device`, `session_bind_to_address`, `quota_grace_percent`, `quota_send_stop_percent`, `recall_days`, `recover_deleted_days`, `sending_max_messages`, `sending_max_recipients`, `reader_fetch_accounts_max`, `reader_sender_addresses_max`, `reader_masked_addresses_max`, `reader_offline_days_max` | | Gates | `plus_addressing_enabled`, `openpgp_portal_enabled`, `zero_access_allowed`, `directory_user_editable` | | Preferences | `quarantine_digest`, `quarantine_digest_hour`, `external_tag_subject`, `external_tag_header`, `impersonation_action`, `quota_warning_percent`, `country_policy`, `message_retention_days` | A server setting not on the list is the server's alone: its name is refused on the domain and account routes (400, `unknown field`) and by COM's `SettingOverride`. Of the `[Settings]` keys only `QuotaWarningPercent` has levels. **Where each level is held.** A field the domain or the account already had, such as the domain's `MaxMessageSize` or `MessageRetentionDays`, stays where it was, with the meaning it always had; its 0 or -1 is the level holding nothing. The rest are rows of `hm_settingoverrides`, where no row means "take the level above". An owner's rows are cached for sixty seconds; every write through the interfaces drops the cache at once, and so does `POST /api/v1/settings/cache/clear` or COM `Cache.Clear`. A row put in by hand that the interfaces would refuse is treated as holding nothing and reported as `HM7311`. Nothing changes for a domain or an account that holds no value of its own. | Where | What | |---|---| | REST | `GET /api/v1/settings/levels` lists the twenty-seven. `GET /api/v1/domains/{domain}` and `GET /api/v1/accounts/{address}` answer `_effective` and `_source`, and `` where the level is held in the overlay. For a level held in the overlay a `PUT` takes a value of the setting's own JSON type, and `null` makes it inherit; for a level held in a field of its own `null` leaves it unchanged, and the field's own 0 or -1 is "take the level above" | | COM (Windows only) | `Domain.SettingOverride(name)` and `Account.SettingOverride(name)`, an empty string for "take the level above", written at once and not by `Save`; `EffectiveSetting(name)` and `EffectiveSettingSource(name)` | | `hmctl` | `hmctl domain set ` and `hmctl account set
` | | Control Panel (Windows only) | A **Levels** tab on the domain and account dialogs, with "In effect: ..., from the ..." beside each setting and **Save levels now** | | Control Deck | The same card in its domain and account editors | **A saved level applies at once**, at every level and from every door, because the resolver is asked when the value is used (at `MAIL FROM`, when a session is checked) and never latched. **Deleting, backing up and moving.** Deleting an account or a domain deletes its rows; a delete that cannot remove them is reported as `HM7310` for a domain and `HM6109` for an account. The XML backup carries the rows, and on restore a row the build refuses is left out and reported as `HM7312`. A mailbox moved to another domain keeps its own values. **What is thinner than it sounds.** - Attachment blocking, web push, passkeys and account recovery have no levels. - The domain's session seconds and switches and its quota grace and send-stop shares could always be set looser than the server's and still can; the ceiling binds the account below them. - The maximum message size is resolved at `MAIL FROM` and at IMAP `APPEND`; the EHLO `SIZE` advertisement, the webmail's own send and the send API stay the server's. - The session policies are the browser sessions' (the webmail, the portal, the Deck); IMAP, POP3 and SMTP sessions have none. - `country_policy` cannot be set to empty over COM, where an empty string means "take the level above"; REST's `""` does it. - Twelve of the twenty-seven are proved by what the account and its domain answer rather than by driving the behaviour per account, and the levels' own regression fixture is Windows-only. ### When a setting appears to do nothing 1. Is it pending? A `reinitialize` key waits for **Apply now** and a `restart` key for a service restart. 2. Was it typed into `hMailServer.ini`? The stored value wins and `HM5804` names the key. Store it instead. 3. Is the key spelled exactly as the code reads it, and is a boolean literally `1`? An unknown key is the default, and only `1` is true: `2`, `yes` and `true` all read as false. 4. Does it need a companion? `SRSEnabled` needs `SRSSecret`, ACME needs `AcmeDomains`, autoconfig needs `WebServicesHttpPort` or `WebServicesHttpsPort`, and MTA-STS hosting needs the HTTPS one. 5. Does the error log show `HM5807` or `HM5808`? A `[SettingsOverride]` section is forcing a value. ### Where to read more - The tour: https://gitlab.com/hmailserver/hmailserver/-/wikis/Settings-Overview. Every key with its default, validation and mark: https://gitlab.com/hmailserver/hmailserver/-/wikis/Settings-Reference. Where the two disagree, the reference is generated from the code and is right. - The three documents behind this topic: https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/ChangedFromStock.md, https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SettingsApply.md and https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SettingLevels.md - The public documentation's settings chapter: https://www.progressiverobot.com/hmailserver-documentation/#21-settings-reference-hmailserverini - On this forum: [the Control Panel on Windows](/topic/28223), [`hmctl` and the PowerShell module](/topic/28265), [the `hmailserver` command line](/topic/28245), [the Control Deck](/topic/28192), [Upgrading to 6.3.4](/topic/28193) and [hMailServer on Linux](/topic/28189). --- Canonical: https://www.hmailserver.co.uk/topic/28223/the-control-panel-on-windows-every-page-what-it-writes-and-what-applies-when # The Control Panel on Windows: every page, what it writes, and what applies when *Written for 6.3.4. hMailServer 6.3.7 (17 October 2026) moves the Control Panel's pages: Updates is a page of its own under Maintenance, the sidebar has Compliance and Virus scanning groups, API & monitoring is Monitoring & alerts, the PROXY protocol and XCLIENT are a card on Incoming relays, and Ctrl+K takes every old title and route to its page. A Control Panel signed in to a server on another computer now shows and saves the server's `hMailServer.ini` `[Settings]` values. The anti-virus action named Delete infected attachments only in 6.3.6's Control Panel is Remove all attachments and deliver the text in 6.3.7's. The installers carry the .NET Desktop Runtime 10.0.13, Microsoft's security release of 13 October 2026, on which the Control Panel runs, but install it only where no .NET 10 Desktop Runtime is installed at all: an upgraded server keeps the 10.0.12 that 6.3.4, 6.3.5 or 6.3.6 installed, so bring it to 10.0.13 or later through Microsoft Update or Microsoft's installer. See [the 6.3.7 announcement](/topic/28277).* The hMailServer Control Panel (`hMailCP.exe`) is the Windows desktop administration application: a .NET 10 WPF program, x64 only, that talks to the server over the COM API. That is why it can administer a server on another machine over DCOM, and why there is none on Linux, where the Control Deck, the REST API and `hmctl` are the administration surfaces ([hMailServer on Linux: what you get and what you do not](/topic/28189)). This topic covers what it writes and where, connecting, the 65 pages with the store behind each, and, since 6.3.4, the mark under each field that says when a saved value applies, the pending bar, the Levels tab and the help pane. The one thing to know first: since 6.3.4 the Control Panel writes every `[Settings]` key over COM into the database, even when it runs on the server itself, and a Control Panel with no connection says it cannot change such a setting rather than writing to the file, where a stored key's line is ignored in favour of the database's value. ### What it holds, and what it writes where **The Control Panel holds no configuration of its own.** Its per-user state (window bounds, theme, language) is under `HKCU\Software\hMailServer\ControlPanel`. The server is reached over COM. Files on the server (`hMailServer.INI`, `hMailServerApiKeys.ini`, the log folder, the event script, certificate and key files) are reachable only when the Control Panel runs on the server itself. **Since 6.3.4 the database is the store of the `[Settings]` keys**, and the file holds a copy ([where each setting is kept, and what stays in the file](/topic/28222)). The Control Panel reads those keys over COM, falls back to the file only when it has no session, and writes them over COM only; a `[Settings]` line edited by hand in the file changes nothing, and the error log names it as `HM5804`. `[Database]`, `[Directories]` and `[LDAP]` are still read from the file only, and `[Database]` and `[Directories]` are deliberately not reachable over COM. **Since 6.3.4 a new installation runs the service as `NT SERVICE\hMailServer`**; an upgrade keeps the account it had ([the service account, and moving to or from it](/topic/28227)). `hMailServer.INI` is read-only to that account on purpose, so its copy of the settings goes stale there. For the Control Panel that changes four things, each refused with a sentence rather than failing quietly: | What | Under the virtual account | |---|---| | Settings saved on any page | Unchanged, since they are kept in the database; only `PasswordPepper`, kept in the file only, is refused | | The **Administrator password** card, and enrolling or removing the administrator's second factor | Refused. For the password the sentence names `hMailServer.exe /SetAdministratorPassword` from an elevated prompt | | The data, log and temporary directories | Refused: an administrator changes the file and approves the new folder for the service | | Creating or switching the database | Refused before anything is made | **Maintenance → Server limits & expert settings** carries a **Windows service account** card: what the service actually runs as, and three buttons, each run after the Windows elevation prompt: **Grant access** (`hMailServer.exe --confirm /GrantServiceAccess`), **Move to NT SERVICE\hMailServer** and **Move to LocalSystem**. The live update's apply and the auto-ban's firewall rules need LocalSystem, and are refused under the virtual account with a sentence. From 6.3.6 a service running as `NT SERVICE\hMailServer` downloads and verifies an update, and an administrator installs it with **Install update** on the Server status page of the Control Panel on the server, with one Windows prompt; on 6.3.4 and 6.3.5 Install update cannot do it, and the administrator runs the downloaded installer by hand. ### Connecting | Field | What to enter | When it is wrong | |---|---|---| | **Server** | `localhost`, or the remote server's name or IP; an empty box means `localhost` | A name that does not resolve, or does not answer on TCP 135, gets a sentence saying so. The check has a four-second budget, so a mistyped host is a pause, not a hang | | **User name** | `Administrator`. A mailbox address also authenticates, but only a server administrator can read the settings | *"Authentication failed. Check the user name and password."* The server never says which of the two was wrong | | **Password** | The administration password set during installation | The same message | *"hMailServer COM API is not registered on the target machine."* means you reached the machine but hMailServer is not installed there. *"Unable to reach the server (RPC unavailable)."* is HRESULT `0x800706BA`: firewall, DCOM permissions, or the service is not running. **Two second factors, and they are not the same thing.** The server-enforced one, whose secret is `[Security] AdministratorTotpSecret` in `hMailServer.ini`, protects every client of the administrator credential: this Control Panel, COM scripts, and the REST API through its `X-hMailServer-OTP` header. The older one, `AdminTotpSecret` under `HKLM\SOFTWARE\hMailServer`, protects this installation of the Control Panel on this machine and nothing else. The code is asked for once, never in a loop, because a wrong password and a missing code look identical from outside. `hMailCP.exe /connect ` signs in as the window loads; the password is on the command line, where anything that can list processes can read it. Connecting opens the **Dashboard**, not Welcome. COM sign-in is deliberately outside the auto-ban and the per-name lockout, so guessing an administrator's name over IMAP cannot lock that administrator out. ### The window and the tree **Since 6.3.4** the Control Panel is drawn on one design system. The window has three rows: the title bar; a command bar holding the sidebar toggle, a search field that opens the Ctrl+K palette, a server status pill (*Not connected*, *Connected*, *Reconnecting…*, *Connection lost*), the theme toggle and the administrator's menu; then the sidebar beside the content. Below 1,000 pixels of window width the sidebar collapses to a rail of one glyph per group. What did not change is the part you may have scripted: the navigation map, every page title, tab header, setting caption and automation id (`nav-`) are what they were. The sidebar holds **65 pages in eight groups** in 6.3.4 (55 in 6.3.3), never more than two levels deep: Monitoring & troubleshooting, Accounts & domains, Mail flow & delivery, Spam & virus filtering, Connections & protocols, TLS & certificates, Access & abuse protection and Maintenance, with Welcome, Dashboard and About outside them. The ten pages new in 6.3.4 are Suppression list, Journaling and dual delivery, Audit trail, Discovery, Administrator roles, Remote domains, Allow and block list, Greylisting bypass list, Domain white and black list and Setting profiles. **Search (Ctrl+K)** answers three questions in one ranked list: a page name or any alias; what you are trying to do, from 97 outcome phrases such as "stop spam"; and a setting name or key, from an index of 448 settings in 6.3.4 (393 in 6.3.3). It also resolves 54 old navigation paths, so `Settings > Security > Auto-ban & SSL/TLS` opens Auto-ban. **Welcome.** Since 6.3.4 it offers a throwaway sample domain, `example.test`, and a first-run tour of seven stops that never acts and is never modal. ### Every page, by what it writes The wiki lists each page with its navigation key, which never changes when a page is renamed. Grouped by the store behind them, the 65 are: | What the page writes | Pages | Applies | |---|---|---| | Nothing: no settings behind the page (6) | Welcome, Dashboard, Live logs, Diagnostics, MX query, About | not applicable | | Read-only, or actions only (12) | Server status, Delivery queue, Message trace, Audit trail, Discovery, External setup, DNS records, Server sendout, Spam filtering overview, Quarantine, Virus scanning overview, Transport encryption overview | Actions at once | | The COM settings tree, a COM collection, or the database through COM (25) | Diagnosing stalled mail, Suppression list, Journaling and dual delivery, Domains, Public folders, Groups, Administrator roles, Routes, Remote domains, Incoming relays, Rules, Server messages, SURBL servers, DNS blacklists, White list, Blocked senders, Allow and block list, Greylisting white list, Greylisting bypass list, Domain white and black list, Blocked attachments, TCP/IP ports, SSL certificates, IP ranges, Setting profiles | At once; a profile's entries as each setting's own mark says | | `[Settings]` keys, in the database since 6.3.4 (7) | API & monitoring, Web services & autoconfiguration, DNS resolver, Certificates (ACME), Transport security, Authentication, Server limits & expert settings | As each field's mark says | | COM and `[Settings]` keys together (11) | Logging, Delivery of e-mail, Anti-spam settings, Anti-virus settings, Protocols, SSL/TLS, Administrative access, Auto-ban, Backup & restore, Performance, Advanced | COM at once, except the syslog card on Logging (since 6.3.4), whose values the sink takes when it starts, at a reinitialise or a service start; the keys as each field's mark says | | A file on the server (4) | REST API keys (`hMailServerApiKeys.ini`, re-read on every request), Directory authentication (LDAP) and Directory synchronisation (`[LDAP]`, re-read within two seconds), Event scripts (the script file; **Save & reload** reloads the engine) | Without a restart | What the table cannot say: - **Three overview pages are deliberately read-only**: Spam filtering overview, Virus scanning overview and Transport encryption overview. They show what no single editor can: a virus scanner that is switched on but cannot run is treated as no virus found, so infected mail is delivered as though examined; a TLS port with no certificate fails to start rather than falling back to plaintext. - **Discovery** (since 6.3.4) changes nothing and is the one page that reads the REST listener, because discovery has no COM interface. It needs `RestApiPort` set, a certificate it can check for an address beyond loopback, and an administrator without a second factor. A case is opened, searched, held and exported on the Control Deck ([eDiscovery and legal holds](/topic/28242)). - **Journaling and dual delivery** and **Audit trail** are the server administrator's only. - **Message trace** and **Quarantine** are inert unless `MessageTraceEnabled` and `QuarantineEnabled` are set, and **Blocked attachments** unless attachment blocking is on in Anti-virus settings. - **Server limits & expert settings** also edits `[SendingLimits]` and `[SendingLimitsOverrides]`, which stay in the file and are re-read within seconds. **Performance** also edits the local `[Database]` rows, and since 6.3.4 has a **Message store** tab: `MessageStoreCompression`, off by default, and `MessageStoreDedupMinKB`, 256. - **The dialogs.** Since 6.3.4 the domain dialog has fourteen tabs (seven in 6.3.3) and the account dialog fifteen (twelve), each ending in **Levels**. ### The mark under each field, and the pending bar **Since 6.3.4.** On 6.3.3 a save of a page whose settings live in `hMailServer.INI` offers to restart the service. Against a 6.3.4 server a save publishes, and under every field kept in the settings store a small line says when the value takes effect ([which keys carry each mark, and what a reinitialise takes](/topic/28222)): | The line | Meaning | |---|---| | **Applies on save** | In effect when the save returns | | **After reinitialize** | A listener's port or bind address, a TLS context, a work queue's size: waits for **Apply now** | | **After a restart** | `ServiceAccountName` and `ServiceAccountPassword`, which are retired. The `[Database]` rows on Performance carry the same words | Every key's mark comes in one COM call when such a page is opened, `Settings.IniSettingAppliesAll`. A page's save writes only the fields whose value changed, because every write publishes, and it says how many of its keys still wait. **The pending bar** sits at the foot of the window: "N saved setting(s) wait for a reinitialize", with **Apply now**. The list behind it is `Settings.PendingIniSettings`, recomputed by the server at every request, so a value put back before anything applied it is not pending. **Apply now** is `Application.Reinitialize` over COM: every service stops and starts again in place, with no service restart and no UAC prompt. 1. It asks first. While the server is draining for maintenance the question says that the listeners open again and delivery resumes, because a reinitialise ends the drain. 2. It takes every `reinitialize` key and not a `restart` key. 3. When a `restart` key waits too, what it says afterwards is how many still wait for a service restart, not that the saved settings are in effect. 4. The Control Panel's COM session survives it. The Control Deck's session does not. **Three things the pages will not do.** - **Return a key to its default.** Clearing a box stores an empty value, which a number reads as `0`. The default comes back through `DELETE /api/v1/settings/ini/{name}`, `hmctl ini delete ` or COM `Settings.DeleteIniSetting(name)`. - **Change a setting the environment names.** A setting set by `HM_SETTING_` or `HM_PROPERTY_` ([configuration from the environment](/topic/28231)) is shown disabled, with *Set by the environment* in place of the mark. - **Claim more than an older server does.** Against a 6.3.3 server, which lacks `IniSettingAppliesAll` and `PendingIniSettings`, every field kept in the store is marked *After a restart*, a save says the service must be restarted, and no pending bar is shown. **The Levels tab.** Since 6.3.4 the domain and account dialogs list [the twenty-seven settings a domain or an account may hold its own value of](/topic/28222), ticked where this level holds one, with "In effect: ..., from the ..." beside each. **Save levels now** writes what changed over COM at once, reads every line back, and shows the server's sentence when it refuses one. Against a server older than schema 6088 the tab says the server has no levels. ### The help pane **Since 6.3.4.** Every page has a `?` button in the command bar, and `F1` does the same: a right-hand pane opens beside the page with what the operator documentation says about it, scrolled to the right heading. It has the document's headings as a list, a search box over the open document, and two links at the foot: *Open on the wiki* and *All documentation*. **The text is built in.** It is the repository's `hmailserver/docs/*.md`, embedded in the Control Panel as resources (each named `Help..md`, with `README.md` and `DeckParity.md` excluded). The Control Panel must work with no REST listener, so it carries its own copy and reads it over no route at all. **What is thinner than it sounds.** - A page whose subject has no document of its own (the TCP/IP ports, the certificates, the groups) opens the nearest one. - The search is within the open document only, not across all of them. - *Open on the wiki* is `HelpWikiBaseUrl` followed by the document's name. The wiki's pages are not named after the documents, so for most documents that link finds no page yet; the pane's own text is the complete one. `HelpWikiBaseUrl` is a row of `hm_settings`, shipped as `https://gitlab.com/hmailserver/hmailserver/-/wikis/` and edited on the **Advanced** page under General. A value that is not empty and not an absolute `http://` or `https://` URL is refused, and empty hides the link. The Control Deck and the webmail have the same pane, served by the server from `GET /api/v1/help/{page}`. ### Administering a server on another machine Type the server's name into the Server box and the Control Panel connects over DCOM. That covers everything the COM API can reach. What degrades is anything that means reading a file on the server: | On a remote session | What you see | |---|---| | Live logs | "Log folder not found on this machine (live logs need a local server)." | | REST API keys | The key store is a file beside `hMailServer.INI`; the page needs it | | The "Listening" badge on TCP/IP ports | Every port reads "Unknown": the TCP table probed is this machine's | | Certificate and private-key inspection | "cannot be checked from here" | | Event scripts | The script is loaded from and saved to local disk | | The `[LDAP]` and `[Database]` editors | Those sections are read from the file only | **Editing the `[Settings]` pages from another machine is not confirmed for 6.3.4.** The wiki's Control Panel page says, in section 5.5 on remote administration, that such a page answers "hMailServer.INI was not found on this machine. These settings can only be edited on the server itself." Its 6.3.4 note in section 5.0 says that these keys are now read and written over COM, and that section 5.5 describes 6.3.3 and was not re-checked against the change. Test it on your own installation before relying on it; if a remote page refuses, use Remote Desktop or the REST API. Since 6.3.7 a Control Panel signed in to a server on another computer shows and saves the server's `hMailServer.ini` `[Settings]` values, and no longer reads or writes this computer's file for the `[Database]`, `[SendingLimits]` and `[LDAP]` values, which it cannot read from there. ### When the Control Panel itself is the problem | Symptom | What it means | What to do | |---|---|---| | It does not reconnect after you stop the service | Deliberate: it rebuilds only a link that broke while the service was up, because activating the COM class would start a service you stopped on purpose | Start the service, then use any page | | An unexpected-error dialog | An unhandled UI fault | See `%LOCALAPPDATA%\hMailServer\ControlPanel\control-panel-errors.log` | | Signing in freezes the window | The host accepted TCP 135 and then stalled inside DCOM | Check DCOM permissions and the firewall on the server | With a server-enforced second factor, a silent reconnect cannot produce a fresh code, so after a service restart you sign in again through the prompt. ### What it does not do, and where to read more There is no Control Panel on Linux and no COM there; [the Control Deck](/topic/28192), [the REST API](/topic/28264) and [the `hmctl` client](/topic/28265) are the surfaces. The Control Panel never returns a key to its default, and it has no page for an account's kind (a person, a meeting room or equipment), encryption at rest, object storage or working an eDiscovery case: those are the Control Deck's and REST's. The public documentation's [chapter 20](https://www.progressiverobot.com/hmailserver-documentation/#20-every-control-panel-page) lists all 65 pages of 6.3.4 by group, and the wiki's Control Panel Pages is the source it follows. - Connecting, the shell, the tour and the help pane: https://gitlab.com/hmailserver/hmailserver/-/wikis/The-Control-Panel - Every page with its key and its store: https://gitlab.com/hmailserver/hmailserver/-/wikis/Control-Panel-Pages - Every page, dialog and field with the setting behind it: https://gitlab.com/hmailserver/hmailserver/-/wikis/Control-Panel-Reference - When a saved setting takes effect: https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SettingsApply.md - The help pane: https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Help.md - The public documentation: https://www.progressiverobot.com/hmailserver-documentation/#5-the-control-panel-your-first-look and https://www.progressiverobot.com/hmailserver-documentation/#20-every-control-panel-page --- Canonical: https://www.hmailserver.co.uk/topic/28223/the-control-panel-on-windows-every-page-what-it-writes-and-what-applies-when # The Control Panel on Windows: every page, what it writes, and what applies when *Written for 6.3.4. hMailServer 6.3.7 (17 October 2026) moves the Control Panel's pages: Updates is a page of its own under Maintenance, the sidebar has Compliance and Virus scanning groups, API & monitoring is Monitoring & alerts, the PROXY protocol and XCLIENT are a card on Incoming relays, and Ctrl+K takes every old title and route to its page. A Control Panel signed in to a server on another computer now shows and saves the server's `hMailServer.ini` `[Settings]` values. The anti-virus action named Delete infected attachments only in 6.3.6's Control Panel is Remove all attachments and deliver the text in 6.3.7's. The installers carry the .NET Desktop Runtime 10.0.13, Microsoft's security release of 13 October 2026, on which the Control Panel runs, but install it only where no .NET 10 Desktop Runtime is installed at all: an upgraded server keeps the 10.0.12 that 6.3.4, 6.3.5 or 6.3.6 installed, so bring it to 10.0.13 or later through Microsoft Update or Microsoft's installer. See [the 6.3.7 announcement](/topic/28277).* The hMailServer Control Panel (`hMailCP.exe`) is the Windows desktop administration application: a .NET 10 WPF program, x64 only, that talks to the server over the COM API. That is why it can administer a server on another machine over DCOM, and why there is none on Linux, where the Control Deck, the REST API and `hmctl` are the administration surfaces ([hMailServer on Linux: what you get and what you do not](/topic/28189)). This topic covers what it writes and where, connecting, the 65 pages with the store behind each, and, since 6.3.4, the mark under each field that says when a saved value applies, the pending bar, the Levels tab and the help pane. The one thing to know first: since 6.3.4 the Control Panel writes every `[Settings]` key over COM into the database, even when it runs on the server itself, and a Control Panel with no connection says it cannot change such a setting rather than writing to the file, where a stored key's line is ignored in favour of the database's value. ### What it holds, and what it writes where **The Control Panel holds no configuration of its own.** Its per-user state (window bounds, theme, language) is under `HKCU\Software\hMailServer\ControlPanel`. The server is reached over COM. Files on the server (`hMailServer.INI`, `hMailServerApiKeys.ini`, the log folder, the event script, certificate and key files) are reachable only when the Control Panel runs on the server itself. **Since 6.3.4 the database is the store of the `[Settings]` keys**, and the file holds a copy ([where each setting is kept, and what stays in the file](/topic/28222)). The Control Panel reads those keys over COM, falls back to the file only when it has no session, and writes them over COM only; a `[Settings]` line edited by hand in the file changes nothing, and the error log names it as `HM5804`. `[Database]`, `[Directories]` and `[LDAP]` are still read from the file only, and `[Database]` and `[Directories]` are deliberately not reachable over COM. **Since 6.3.4 a new installation runs the service as `NT SERVICE\hMailServer`**; an upgrade keeps the account it had ([the service account, and moving to or from it](/topic/28227)). `hMailServer.INI` is read-only to that account on purpose, so its copy of the settings goes stale there. For the Control Panel that changes four things, each refused with a sentence rather than failing quietly: | What | Under the virtual account | |---|---| | Settings saved on any page | Unchanged, since they are kept in the database; only `PasswordPepper`, kept in the file only, is refused | | The **Administrator password** card, and enrolling or removing the administrator's second factor | Refused. For the password the sentence names `hMailServer.exe /SetAdministratorPassword` from an elevated prompt | | The data, log and temporary directories | Refused: an administrator changes the file and approves the new folder for the service | | Creating or switching the database | Refused before anything is made | **Maintenance → Server limits & expert settings** carries a **Windows service account** card: what the service actually runs as, and three buttons, each run after the Windows elevation prompt: **Grant access** (`hMailServer.exe --confirm /GrantServiceAccess`), **Move to NT SERVICE\hMailServer** and **Move to LocalSystem**. The live update's apply and the auto-ban's firewall rules need LocalSystem, and are refused under the virtual account with a sentence. From 6.3.6 a service running as `NT SERVICE\hMailServer` downloads and verifies an update, and an administrator installs it with **Install update** on the Server status page of the Control Panel on the server, with one Windows prompt; on 6.3.4 and 6.3.5 Install update cannot do it, and the administrator runs the downloaded installer by hand. ### Connecting | Field | What to enter | When it is wrong | |---|---|---| | **Server** | `localhost`, or the remote server's name or IP; an empty box means `localhost` | A name that does not resolve, or does not answer on TCP 135, gets a sentence saying so. The check has a four-second budget, so a mistyped host is a pause, not a hang | | **User name** | `Administrator`. A mailbox address also authenticates, but only a server administrator can read the settings | *"Authentication failed. Check the user name and password."* The server never says which of the two was wrong | | **Password** | The administration password set during installation | The same message | *"hMailServer COM API is not registered on the target machine."* means you reached the machine but hMailServer is not installed there. *"Unable to reach the server (RPC unavailable)."* is HRESULT `0x800706BA`: firewall, DCOM permissions, or the service is not running. **Two second factors, and they are not the same thing.** The server-enforced one, whose secret is `[Security] AdministratorTotpSecret` in `hMailServer.ini`, protects every client of the administrator credential: this Control Panel, COM scripts, and the REST API through its `X-hMailServer-OTP` header. The older one, `AdminTotpSecret` under `HKLM\SOFTWARE\hMailServer`, protects this installation of the Control Panel on this machine and nothing else. The code is asked for once, never in a loop, because a wrong password and a missing code look identical from outside. `hMailCP.exe /connect ` signs in as the window loads; the password is on the command line, where anything that can list processes can read it. Connecting opens the **Dashboard**, not Welcome. COM sign-in is deliberately outside the auto-ban and the per-name lockout, so guessing an administrator's name over IMAP cannot lock that administrator out. ### The window and the tree **Since 6.3.4** the Control Panel is drawn on one design system. The window has three rows: the title bar; a command bar holding the sidebar toggle, a search field that opens the Ctrl+K palette, a server status pill (*Not connected*, *Connected*, *Reconnecting…*, *Connection lost*), the theme toggle and the administrator's menu; then the sidebar beside the content. Below 1,000 pixels of window width the sidebar collapses to a rail of one glyph per group. What did not change is the part you may have scripted: the navigation map, every page title, tab header, setting caption and automation id (`nav-`) are what they were. The sidebar holds **65 pages in eight groups** in 6.3.4 (55 in 6.3.3), never more than two levels deep: Monitoring & troubleshooting, Accounts & domains, Mail flow & delivery, Spam & virus filtering, Connections & protocols, TLS & certificates, Access & abuse protection and Maintenance, with Welcome, Dashboard and About outside them. The ten pages new in 6.3.4 are Suppression list, Journaling and dual delivery, Audit trail, Discovery, Administrator roles, Remote domains, Allow and block list, Greylisting bypass list, Domain white and black list and Setting profiles. **Search (Ctrl+K)** answers three questions in one ranked list: a page name or any alias; what you are trying to do, from 97 outcome phrases such as "stop spam"; and a setting name or key, from an index of 448 settings in 6.3.4 (393 in 6.3.3). It also resolves 54 old navigation paths, so `Settings > Security > Auto-ban & SSL/TLS` opens Auto-ban. **Welcome.** Since 6.3.4 it offers a throwaway sample domain, `example.test`, and a first-run tour of seven stops that never acts and is never modal. ### Every page, by what it writes The wiki lists each page with its navigation key, which never changes when a page is renamed. Grouped by the store behind them, the 65 are: | What the page writes | Pages | Applies | |---|---|---| | Nothing: no settings behind the page (6) | Welcome, Dashboard, Live logs, Diagnostics, MX query, About | not applicable | | Read-only, or actions only (12) | Server status, Delivery queue, Message trace, Audit trail, Discovery, External setup, DNS records, Server sendout, Spam filtering overview, Quarantine, Virus scanning overview, Transport encryption overview | Actions at once | | The COM settings tree, a COM collection, or the database through COM (25) | Diagnosing stalled mail, Suppression list, Journaling and dual delivery, Domains, Public folders, Groups, Administrator roles, Routes, Remote domains, Incoming relays, Rules, Server messages, SURBL servers, DNS blacklists, White list, Blocked senders, Allow and block list, Greylisting white list, Greylisting bypass list, Domain white and black list, Blocked attachments, TCP/IP ports, SSL certificates, IP ranges, Setting profiles | At once; a profile's entries as each setting's own mark says | | `[Settings]` keys, in the database since 6.3.4 (7) | API & monitoring, Web services & autoconfiguration, DNS resolver, Certificates (ACME), Transport security, Authentication, Server limits & expert settings | As each field's mark says | | COM and `[Settings]` keys together (11) | Logging, Delivery of e-mail, Anti-spam settings, Anti-virus settings, Protocols, SSL/TLS, Administrative access, Auto-ban, Backup & restore, Performance, Advanced | COM at once, except the syslog card on Logging (since 6.3.4), whose values the sink takes when it starts, at a reinitialise or a service start; the keys as each field's mark says | | A file on the server (4) | REST API keys (`hMailServerApiKeys.ini`, re-read on every request), Directory authentication (LDAP) and Directory synchronisation (`[LDAP]`, re-read within two seconds), Event scripts (the script file; **Save & reload** reloads the engine) | Without a restart | What the table cannot say: - **Three overview pages are deliberately read-only**: Spam filtering overview, Virus scanning overview and Transport encryption overview. They show what no single editor can: a virus scanner that is switched on but cannot run is treated as no virus found, so infected mail is delivered as though examined; a TLS port with no certificate fails to start rather than falling back to plaintext. - **Discovery** (since 6.3.4) changes nothing and is the one page that reads the REST listener, because discovery has no COM interface. It needs `RestApiPort` set, a certificate it can check for an address beyond loopback, and an administrator without a second factor. A case is opened, searched, held and exported on the Control Deck ([eDiscovery and legal holds](/topic/28242)). - **Journaling and dual delivery** and **Audit trail** are the server administrator's only. - **Message trace** and **Quarantine** are inert unless `MessageTraceEnabled` and `QuarantineEnabled` are set, and **Blocked attachments** unless attachment blocking is on in Anti-virus settings. - **Server limits & expert settings** also edits `[SendingLimits]` and `[SendingLimitsOverrides]`, which stay in the file and are re-read within seconds. **Performance** also edits the local `[Database]` rows, and since 6.3.4 has a **Message store** tab: `MessageStoreCompression`, off by default, and `MessageStoreDedupMinKB`, 256. - **The dialogs.** Since 6.3.4 the domain dialog has fourteen tabs (seven in 6.3.3) and the account dialog fifteen (twelve), each ending in **Levels**. ### The mark under each field, and the pending bar **Since 6.3.4.** On 6.3.3 a save of a page whose settings live in `hMailServer.INI` offers to restart the service. Against a 6.3.4 server a save publishes, and under every field kept in the settings store a small line says when the value takes effect ([which keys carry each mark, and what a reinitialise takes](/topic/28222)): | The line | Meaning | |---|---| | **Applies on save** | In effect when the save returns | | **After reinitialize** | A listener's port or bind address, a TLS context, a work queue's size: waits for **Apply now** | | **After a restart** | `ServiceAccountName` and `ServiceAccountPassword`, which are retired. The `[Database]` rows on Performance carry the same words | Every key's mark comes in one COM call when such a page is opened, `Settings.IniSettingAppliesAll`. A page's save writes only the fields whose value changed, because every write publishes, and it says how many of its keys still wait. **The pending bar** sits at the foot of the window: "N saved setting(s) wait for a reinitialize", with **Apply now**. The list behind it is `Settings.PendingIniSettings`, recomputed by the server at every request, so a value put back before anything applied it is not pending. **Apply now** is `Application.Reinitialize` over COM: every service stops and starts again in place, with no service restart and no UAC prompt. 1. It asks first. While the server is draining for maintenance the question says that the listeners open again and delivery resumes, because a reinitialise ends the drain. 2. It takes every `reinitialize` key and not a `restart` key. 3. When a `restart` key waits too, what it says afterwards is how many still wait for a service restart, not that the saved settings are in effect. 4. The Control Panel's COM session survives it. The Control Deck's session does not. **Three things the pages will not do.** - **Return a key to its default.** Clearing a box stores an empty value, which a number reads as `0`. The default comes back through `DELETE /api/v1/settings/ini/{name}`, `hmctl ini delete ` or COM `Settings.DeleteIniSetting(name)`. - **Change a setting the environment names.** A setting set by `HM_SETTING_` or `HM_PROPERTY_` ([configuration from the environment](/topic/28231)) is shown disabled, with *Set by the environment* in place of the mark. - **Claim more than an older server does.** Against a 6.3.3 server, which lacks `IniSettingAppliesAll` and `PendingIniSettings`, every field kept in the store is marked *After a restart*, a save says the service must be restarted, and no pending bar is shown. **The Levels tab.** Since 6.3.4 the domain and account dialogs list [the twenty-seven settings a domain or an account may hold its own value of](/topic/28222), ticked where this level holds one, with "In effect: ..., from the ..." beside each. **Save levels now** writes what changed over COM at once, reads every line back, and shows the server's sentence when it refuses one. Against a server older than schema 6088 the tab says the server has no levels. ### The help pane **Since 6.3.4.** Every page has a `?` button in the command bar, and `F1` does the same: a right-hand pane opens beside the page with what the operator documentation says about it, scrolled to the right heading. It has the document's headings as a list, a search box over the open document, and two links at the foot: *Open on the wiki* and *All documentation*. **The text is built in.** It is the repository's `hmailserver/docs/*.md`, embedded in the Control Panel as resources (each named `Help..md`, with `README.md` and `DeckParity.md` excluded). The Control Panel must work with no REST listener, so it carries its own copy and reads it over no route at all. **What is thinner than it sounds.** - A page whose subject has no document of its own (the TCP/IP ports, the certificates, the groups) opens the nearest one. - The search is within the open document only, not across all of them. - *Open on the wiki* is `HelpWikiBaseUrl` followed by the document's name. The wiki's pages are not named after the documents, so for most documents that link finds no page yet; the pane's own text is the complete one. `HelpWikiBaseUrl` is a row of `hm_settings`, shipped as `https://gitlab.com/hmailserver/hmailserver/-/wikis/` and edited on the **Advanced** page under General. A value that is not empty and not an absolute `http://` or `https://` URL is refused, and empty hides the link. The Control Deck and the webmail have the same pane, served by the server from `GET /api/v1/help/{page}`. ### Administering a server on another machine Type the server's name into the Server box and the Control Panel connects over DCOM. That covers everything the COM API can reach. What degrades is anything that means reading a file on the server: | On a remote session | What you see | |---|---| | Live logs | "Log folder not found on this machine (live logs need a local server)." | | REST API keys | The key store is a file beside `hMailServer.INI`; the page needs it | | The "Listening" badge on TCP/IP ports | Every port reads "Unknown": the TCP table probed is this machine's | | Certificate and private-key inspection | "cannot be checked from here" | | Event scripts | The script is loaded from and saved to local disk | | The `[LDAP]` and `[Database]` editors | Those sections are read from the file only | **Editing the `[Settings]` pages from another machine is not confirmed for 6.3.4.** The wiki's Control Panel page says, in section 5.5 on remote administration, that such a page answers "hMailServer.INI was not found on this machine. These settings can only be edited on the server itself." Its 6.3.4 note in section 5.0 says that these keys are now read and written over COM, and that section 5.5 describes 6.3.3 and was not re-checked against the change. Test it on your own installation before relying on it; if a remote page refuses, use Remote Desktop or the REST API. Since 6.3.7 a Control Panel signed in to a server on another computer shows and saves the server's `hMailServer.ini` `[Settings]` values, and no longer reads or writes this computer's file for the `[Database]`, `[SendingLimits]` and `[LDAP]` values, which it cannot read from there. ### When the Control Panel itself is the problem | Symptom | What it means | What to do | |---|---|---| | It does not reconnect after you stop the service | Deliberate: it rebuilds only a link that broke while the service was up, because activating the COM class would start a service you stopped on purpose | Start the service, then use any page | | An unexpected-error dialog | An unhandled UI fault | See `%LOCALAPPDATA%\hMailServer\ControlPanel\control-panel-errors.log` | | Signing in freezes the window | The host accepted TCP 135 and then stalled inside DCOM | Check DCOM permissions and the firewall on the server | With a server-enforced second factor, a silent reconnect cannot produce a fresh code, so after a service restart you sign in again through the prompt. ### What it does not do, and where to read more There is no Control Panel on Linux and no COM there; [the Control Deck](/topic/28192), [the REST API](/topic/28264) and [the `hmctl` client](/topic/28265) are the surfaces. The Control Panel never returns a key to its default, and it has no page for an account's kind (a person, a meeting room or equipment), encryption at rest, object storage or working an eDiscovery case: those are the Control Deck's and REST's. The public documentation's [chapter 20](https://www.progressiverobot.com/hmailserver-documentation/#20-every-control-panel-page) lists all 65 pages of 6.3.4 by group, and the wiki's Control Panel Pages is the source it follows. - Connecting, the shell, the tour and the help pane: https://gitlab.com/hmailserver/hmailserver/-/wikis/The-Control-Panel - Every page with its key and its store: https://gitlab.com/hmailserver/hmailserver/-/wikis/Control-Panel-Pages - Every page, dialog and field with the setting behind it: https://gitlab.com/hmailserver/hmailserver/-/wikis/Control-Panel-Reference - When a saved setting takes effect: https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SettingsApply.md - The help pane: https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Help.md - The public documentation: https://www.progressiverobot.com/hmailserver-documentation/#5-the-control-panel-your-first-look and https://www.progressiverobot.com/hmailserver-documentation/#20-every-control-panel-page --- Canonical: https://www.hmailserver.co.uk/topic/28224/imap-and-pop3-on-the-server-capabilities-switches-shared-mailboxes-search-limits # IMAP and POP3 on the server: capabilities, switches, shared mailboxes, search limits *Written for 6.3.4. hMailServer 6.3.7 (17 October 2026) changes some of what a client gets. IMAP `COPY` and `MOVE` by message number read their set in the client's numbering, and a set that names a message another session has just expunged is refused `NO [EXPUNGEISSUED]` for the client to send again; a `COPY` or `MOVE` that fails part way leaves the destination as it was; `SEARCH`, `SORT` and `THREAD` ask a criterion named again beside itself once; and `FETCH BODY[N.MIME]`, and `BODY[N]` of a multipart, are the stored octets. POP3 accepts the `UTF8` classic Outlook for Windows (2016 to 2024) sends after the login, until the session's first `STAT`, `LIST`, `UIDL`, `RETR`, `TOP`, `DELE` or `RSET`, so its downloads work again. SMTP, POP3 and ManageSieve refuse a SASL PLAIN sign-in that asks to act as another account. The IMAP and POP3 listeners can take the client's address from a PROXY protocol header, as the SMTP listener does, each from the peers on a list of its own, both off by default. The send-as grant now bears on DKIM too: an authenticated submission is signed as another hosted domain only when its account may send as the address its From names - its own, an alias pointing at it, a mailbox that granted it the post right, an address it proved, or a masked address of its own - and is otherwise accepted and sent unsigned as that domain. See [the 6.3.7 announcement](/topic/28277).* This topic is the server side of IMAP, POP3 and SMTP in hMailServer 6.3.4: what each listener advertises and which switch gates it, search limits and the full-text index, QRESYNC's expunge history, connection limits, shared mailboxes, and what 6.3.4 fixed that a client may notice. **Know this first: a capability missing from an answer is nearly always a switch or the session's state, not a fault.** Two IMAP switches ship off, and since 6.3.4 IMAP offers `STARTTLS` only before TLS and sign-in, and `COMPRESS=DEFLATE` only after sign-in. What a mail client needs is in [Connecting a mail client](/topic/28255); the ACL rights table is in [Aliases, distribution lists, mailing lists and public folders](/topic/28270). ### Seeing what the server advertises Ask the server, from any machine with OpenSSL. For SMTP type `EHLO test.example.org`; for IMAP, `CAPABILITY`: ``` openssl s_client -connect mail.yourcompany.com:25 -starttls smtp -quiet openssl s_client -connect mail.yourcompany.com:993 -quiet ``` On 6.3.4 ask IMAP before `LOGIN` and after: the answers differ by design. For POP3, connect to 995 the same way and send `CAPA`. Ports 993 and 995 exist only once you have created them (**Connections & protocols → TCP/IP ports** in the Windows Control Panel, or the Control Deck). ### IMAP: what is always there On every connection, as in 6.3.3: ``` IMAP4 IMAP4rev1 IMAP4rev2 CHILDREN THREAD=ORDEREDSUBJECT THREAD=REFERENCES NAMESPACE RIGHTS=texk MOVE ID SPECIAL-USE CREATE-SPECIAL-USE UNSELECT UIDPLUS ENABLE STATUS=SIZE ESEARCH CONDSTORE QRESYNC LIST-EXTENDED SEARCHRES WITHIN UTF8=ACCEPT LIST-STATUS PREVIEW MULTIAPPEND REPLACE SAVEDATE METADATA OBJECTID BINARY LITERAL- APPENDLIMIT ``` `APPENDLIMIT` is bare before authentication and `APPENDLIMIT=` after. `LITERAL-` caps non-synchronising literals at 4096 bytes, so an oversized `APPEND` can still be refused with `TOOBIG`. **A client opts in to IMAP4rev2 per session** with `ENABLE IMAP4rev2`, which implies `UTF8=ACCEPT`. `ENABLE` recognises exactly four names: `QRESYNC` (implies `CONDSTORE`), `CONDSTORE`, `UTF8=ACCEPT` and `IMAP4rev2`. After `ENABLE IMAP4rev2` the session drops the `RECENT` response and the `[UNSEEN]` code on `SELECT` and `EXAMINE`, drops `RECENT` as a `STATUS` item, and answers a `SEARCH` with no `RETURN` options as `ESEARCH`. ### IMAP: what is gated, and by what | Capability | Advertised when | Shipped | |---|---|---| | `IDLE` | `enableimapidle` is on | On | | `QUOTA QUOTA=RES-STORAGE` | `enableimapquota` is on | On | | `SORT` | `enableimapsort` is on | On | | `ACL` | `enableimapacl` is on; it also switches the `#Users` namespace | On | | `AUTH=PLAIN`, `AUTH=SCRAM-SHA-256` | `EnableImapSASLPlain` is on and authentication is available | **Off** | | `SASL-IR` | `EnableImapSASLInitialResponse` is on | **Off** | | `COMPRESS=DEFLATE` | `IMAPCompressionEnabled=1`, the client has signed in (since 6.3.4), and the session is not yet compressed | On | | `STARTTLS` | The port is STARTTLS-optional or -required, TLS is not active, and (since 6.3.4) the client has not signed in | Per port | | `LOGINDISABLED` | Authentication is refused on this cleartext connection (STARTTLS-required port, or `RequireTLSForAuth` on the IP range) | Per range | The rest depend on the connection: `AUTH=SCRAM-SHA-256-PLUS` only on TLS (and behind the same SASL switch), `AUTH=XOAUTH2 AUTH=OAUTHBEARER` with `OAuth2Enabled` and, unless `OAuth2RequireTLS=0`, only over TLS, `AUTH=EXTERNAL` on a verified client certificate that names an address, and `UNAUTHENTICATE` once signed in. - **`IDLE`** reports changes in the selected folder (`EXISTS`, `RECENT`, `EXPUNGE`, `FLAGS`) as they happen. A folder another client has just created is seen at the next folder-list refresh, as there is no IMAP `NOTIFY`; that is IMAP, not a fault. - **`QUOTA`** answers with one unnamed root, `""`, and a `STORAGE` resource in kilobytes from the account's maximum size; an account with no limit gets an empty list. `SETQUOTA` is refused, because the quota is the administrator's. Refusals of `APPEND`, `COPY` and `MOVE` for a full mailbox carry `OVERQUOTA`. - **`SORT`**: with message indexing off, as it ships, `SORT` reads each message's header from disk. - **`EnableImapSASLPlain`** gates SCRAM as well as PLAIN, despite its name. [Roadmap.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/Roadmap.md) says it gates the whole `AUTHENTICATE` command, while the Capabilities wiki gives OAuth2 and `EXTERNAL` conditions of their own: with it off, test `XOAUTH2`, `OAUTHBEARER` and `EXTERNAL` over IMAP before relying on them. **`COMPRESS DEFLATE`** (RFC 4978) compresses both directions of the rest of the session; a client that does not ask is unaffected. A second one is `NO [COMPRESSIONACTIVE]`, `STARTTLS` under compression is `BAD`, and `IMAPCompressionEnabled=0` withdraws it, the command then answering `BAD` like an unknown one. ### POP3 and SMTP: the same question **POP3 `CAPA`** always carries `UIDL`, `TOP`, `RESP-CODES`, `AUTH-RESP-CODE`, `IMPLEMENTATION hMailServer`, `PIPELINING`, `UTF8` and `EXPIRE NEVER`. The `SASL` mechanisms depend on TLS, `STLS` on the port, and `LOGIN-DELAY ` appears only when `Pop3LoginDelaySeconds` is above 0 (0 ships). That key advertises and enforces a minimum interval between sign-ins, refusing an early one with `-ERR [LOGIN-DELAY] …`; on Windows it is the POP3 tab's card *How often a client may check for mail*. **SMTP `EHLO`** always carries `SIZE ` (the maximum message size in KB times 1024; bare `SIZE` when 0), `8BITMIME`, `PIPELINING`, `CHUNKING`, `BINARYMIME`, `SMTPUTF8`, `ENHANCEDSTATUSCODES`, `DSN` and `HELP`. `BINARYMIME` is for local delivery: a recipient needing onward relay is refused at `RCPT TO` with 554 5.6.3. Conditionally: - `XCLIENT` only when `SMTPXClientEnabled=1` and the real peer is in `SMTPXClientTrustedIPs`; an untrusted peer that tries it gets 550. - `STARTTLS` on a cleartext connection to a STARTTLS-optional or -required port. - `AUTH` only when authentication is enabled, withheld on cleartext when the port requires STARTTLS or the range sets `RequireTLSForAuth`, and never on the ports in `DisableAUTHList`. - Not a keyword, but decided the same way: the HAProxy PROXY protocol is accepted only with `SMTPProxyProtocolEnabled=1` and from a peer in `SMTPProxyProtocolTrustedIPs`, and nothing in `EHLO` announces it. On SMTP and POP3 the `AUTH` line and its SASL responses take up to 12,288 octets; every other line keeps 510 (SMTP) or 500 (POP3). What 6.3.4 added to SMTP is refusals, not capabilities. ### Where the switches are set, and when a change applies | What | Windows Control Panel | Elsewhere | Applies | |---|---|---|---| | `IDLE`, `QUOTA`, `SORT`, `ACL`, the two SASL switches, the connection limits | **Connections & protocols → Protocols**, IMAP, SMTP and POP3 tabs | COM `Settings` (`IMAPIdleEnabled` and its neighbours); REST `PUT /api/v1/settings` (`imap_idle_enabled`, `imap_quota_enabled`, `imap_sort_enabled`, `imap_acl_enabled`, `imap_sasl_plain_enabled`, `imap_sasl_initial_response_enabled`); the Deck's Settings view; `hmctl settings set` | At once | | Message indexing | **Maintenance → Performance**, Indexing tab | COM `Settings.MessageIndexing`; `PUT /api/v1/settings/indexing` (`enabled`); `hmctl settings`, group `indexing` | At once | | `IMAPCompressionEnabled`, `IMAPSearchTimeout`, `IMAPSearchMaxMegabytes`, `IMAPExpungeRetentionRecords`, `Pop3LoginDelaySeconds`, `IndexerFullText` and its tuning keys | Cards on the IMAP and POP3 tabs; the Indexing tab | COM `Settings.SetIniSetting`; `PUT /api/v1/settings/ini/{name}` (administrator password only); `hmctl ini`; on Linux `hmailserver --set-setting NAME=VALUE`. No Deck page | When saved | The Control Panel and COM are Windows-only. The last row's keys were the `[Settings]` section of `hMailServer.ini`; **since 6.3.4 the database holds them**, none of them waits for a reinitialise or a restart ([docs/SettingsApply.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SettingsApply.md)), and the server does not take a line edited in the file, so set them through an interface. [Settings: when a save takes effect](/topic/28222) covers the store, `HM5804`, and a value stored by `--set-setting` while the service runs. The REST lines assume the listener on `127.0.0.1:8443`, as the REST API wiki's examples do; `RestApiPort` ships at 0 (off) and waits for a reinitialise. ```bash # Switch IDLE off; in effect when the answer comes back curl -u Administrator:your-admin-password -X PUT -H "Content-Type: application/json" \ -d '{"imap_idle_enabled":false}' http://127.0.0.1:8443/api/v1/settings # A [Settings] key: the administrator password only, never an API key curl -u Administrator:your-admin-password -X PUT -H "Content-Type: application/json" \ -d '{"value":"120"}' http://127.0.0.1:8443/api/v1/settings/ini/IMAPSearchTimeout # On Linux, then a restart so the running server takes it sudo hmailserver --set-setting IMAPSearchTimeout=120 sudo systemctl restart hmailserver ``` ### Search: two ceilings and the full-text index **Without the full-text index, `SEARCH BODY` and `TEXT` read every message in the mailbox**, attachments whose media type is text included (decoded, bounded at 1 MB). Two ceilings, measured from the moment a command starts and never re-armed by progress, bound `SEARCH`, `SORT` and `THREAD`: `IMAPSearchTimeout` (60 seconds) and `IMAPSearchMaxMegabytes` (2048 MB of message content read); 0 turns either off. They are the IMAP tab's card *IMAP search limits*, and [README.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/README.md) says to raise them for mailboxes of several hundred thousand messages. **What a client gets at a ceiling is stated two ways.** [Roadmap.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/Roadmap.md) records `NO` with RFC 9051 `LIMIT` and no untagged result, so a cut-short list cannot pass for a complete one; [README.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/README.md)'s comment beside `IMAPSearchTimeout` says the search returns the matches found so far. Until that is settled, treat a search that hit a ceiling as incomplete, and let nothing act on its result in bulk. **The full-text index is off, and needs two switches**: `IndexerFullText=1` indexes nothing unless message indexing (`Settings.MessageIndexing`) is on too. It costs a term table of a few kilobytes per message and a backfill pass over every message already delivered, which is why no upgrade turns it on. **The index can only prove that a message cannot match**; everything else is read and scanned as before, so results are identical on or off. The webmail's search and eDiscovery use the same filter. | Key | Default | What it does | |---|---|---| | `IndexerFullTextBatchSize` | 250 (1 to 100000) | Messages one backfill pass reads before the cursor is saved and the thread pauses | | `IndexerFullTextMinTokenLength` | 3 (3 to 64) | The shortest search string the index answers for | | `IndexerFullTextMaxTokensPerMessage` | 2048 (64 to 1000000) | Distinct terms before a message is marked always-scanned instead of indexed | The Indexing tab's *Index now, and show how far behind it is* and *Discard the index and rebuild it* are `POST /api/v1/settings/indexing/index` and `/clear` over REST, and `Index()` and `Clear()` over COM, where `TotalIndexedCount` against `TotalMessageCount` shows a rebuild's progress. With mailboxes encrypted at rest, the indexed count stays short by their messages for good: those are never indexed. ### QRESYNC: how much expunge history is kept `IMAPExpungeRetentionRecords` keeps the newest 5000 expunge records per mailbox (0 keeps all), pruned every twelve hours and once at service start; on Windows it is the card *Change history for synchronising clients*. The newest is never removed. **A client whose last sync is older than what is kept gets a full resync, never a wrong one**: every UID in its range that the mailbox does not hold. An installation upgraded from a release older than 6.2.23-alpha1 prunes the table in one pass at its first start. [README.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/README.md) says 0 defers that prune, and an `IMAPExpungeRetentionRecords` line of 0 added to `[Settings]` before the upgrade is taken: 6.3.4's first start adopts from the file a key the database has never held, and before 6.2.23-alpha1 the database held this one only if the file had the line. Restart the old service once after adding or changing it (Upgrading hMailServer wiki, 18.3, step 6): [CHANGELOG.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/CHANGELOG.md)'s note that a line edited to prepare an upgrade is put back concerns a key the database already holds, and that restart settles it. Set it back to 5000 afterwards through an interface; [Upgrading to 6.3.4](/topic/28193) has the rest of the order. ### Connection limits and what guards them `MaxSMTPConnections`, `MaxIMAPConnections` and `MaxPOP3Connections` are on the Protocols page (REST `max_smtp_connections`, `max_imap_connections`, `max_pop3_connections`). None has a limit by default; SMTP and IMAP ship at 0, no cap. The Services tab turns a protocol off altogether (`ServiceSMTP`, `ServiceIMAP`, `ServicePOP3`). Watch `hmailserver_sessions{protocol}` on `/metrics` (off until `MetricsServerPort` is set, which waits for a reinitialise) or `sessions` in `GET /api/v1/status`. Around the limits: auto-ban counts failed sign-ins per IP across SMTP, IMAP, POP3, ManageSieve and REST (on: 3 in 30 minutes, a 60-minute ban); `LogonTarpitSeconds` (off) delays each failed sign-in's refusal; a second POP3 session on one mailbox gets `-ERR [IN-USE]`; and since 6.3.4 an IMAP `LIST` or `LSUB` pattern over 8,192 characters, or more than 32 patterns in one `LIST`, is answered `BAD`. The submission rate limit and the tarpits are in [IP ranges, auto-ban, lockout and tarpits](/topic/28239); refusing a connection by country is in [allow and block lists and country blocking](/topic/28249). ### Opening a mailbox to a colleague Shared and delegated mailboxes are IMAP ACL (RFC 4314). The owner opens a folder to another account, and it appears to that account under the Other Users namespace as `#Users.owner@domain.folder`. **No administrator step is needed**: any account whose client speaks `SETACL` can do it, and the audit trail records no row for it. 1. Leave ACL on. With it off, `#Users` stops existing (`NAMESPACE` advertises `NIL`), public folders are open to everyone, and the send-as grant is not honoured. 2. The owner grants rights from an IMAP client with ACL support. An administrator can do it over REST: `GET` on `/api/v1/accounts/{address}/folders` gives the folder's id, and `/api/v1/accounts/{address}/folders/{id}/permissions` lists, adds, changes and removes entries (since 6.3.3; writes need a full key). On Windows, COM has `IMAPFolderPermissions.Add`. The grantee is an account, a group (`/api/v1/groups`, since 6.3.4), or Anyone, which is every authenticated account on the server. 3. Grant `l`, or the folder does not exist for the colleague. `r` reads, `s` changes read state, `i`, `t` and `e` file and delete, and `p` on the INBOX is the send-as grant. 4. Grant on each top-level folder needed. A right is decided at the nearest folder up the owner's tree that has any entries, and entries are not merged: a grant on INBOX reaches its sub-folders that have none of their own, not the Sent folder, and one entry on a sub-folder cuts it off from the parent's. 5. Over SMTP the `p` grant counts only while `SmtpAuthenticatedSenderCheck` is on, and it ships off: until then nothing constrains an authenticated `MAIL FROM`, so any account can send as the mailbox, or as anyone else. Turned on, it applies to every authenticated session on the server; the addresses it then allows are in [who may send as whom](/topic/28239). 6. In the colleague's client, a pane of subscribed folders shows only the ones the owner subscribed to; the webmail shows them under `Shared by `. **What follows from sharing:** - `\Seen` is stored on the message, not per person: with `s`, a delegate marks a message read for everyone; with `lr` alone they leave no trace, and their client cannot track what they have read. - Since 6.3.4 an `APPEND` into a `#Users` folder is charged to the owner's quota, as `COPY` and `MOVE` already were. - A delegate reads a folder of a mailbox encrypted at rest only while the owner's key is open; otherwise a `FETCH` that reads the message answers `NO [UNAVAILABLE]`, and copying out is refused. - In the webmail a read-only share refuses a flag or a delete with 403, a message cannot move between one's own mailbox and a shared one, and the owner cannot grant access from the page. - Since 6.3.4 a restore puts each folder's access grants back. Every `LIST` runs one more query while ACL is on. ### What 6.3.4 fixed that a client may notice Where Dovecot's `imaptest` found IMAP answering other than the RFCs say (`\Seen` without `.PEEK`, `APPEND` byte for byte, `SORT` and `THREAD` order, among others), [Connecting a mail client](/topic/28255) has the list. The rest: | What a client did | What it gets since 6.3.4 | |---|---| | Sent `STARTTLS` after `LOGIN`, or inside TLS | `BAD`, and no `STARTTLS` in `CAPABILITY` | | Sent `COMPRESS DEFLATE` before signing in | `NO` | | Appended over a compressed connection, data inflating past the 250,000-octet receive buffer | The whole message, where it was stored cut short or never finished | | Read `ENVELOPE` with a backslash in `Date`, or `BODYSTRUCTURE` with a quote or backslash in a part's parameters | A structure that parses: the date's backslashes left out; quotes and backslashes escaped, a control character sent as a space | | Fetched a message the server cannot read | `NO [UNAVAILABLE]` for a closed key; a damaged one (`HM7350`) left out, the rest fetched and the command ending `NO` - not `OK` with an empty `ENVELOPE` | | Changed one message's flags from two sessions at once | Both changes kept; `STORE ... (UNCHANGEDSINCE n)` tested in the same step | | Went on sending a `MULTIAPPEND`'s later literals after a refusal, one over the maximum message size (and 1 MB) | The connection closed with `BAD [TOOBIG]`, where it was read whatever its size | | Fetched a message only an unreachable object store holds | IMAP `NO [UNAVAILABLE]`, POP3 `-ERR [SYS/TEMP]` (object storage is new in 6.3.4, and off) | Two heap-corruption aborts are gone as well (two sessions on one message's keywords; `LIST` beside folder changes). Most of these were in 6.3.3 too. ### What it does not do - **No APOP or CRAM-MD5** on POP3, and `THREADID` is answered `NIL`. - **Delegation is not measured in Thunderbird or Outlook**, and an Outlook IMAP profile never acts on the Autodiscover answer's delegated mailboxes; the `#Users` namespace is what works. ### Where to read more - Wiki: [Capabilities and Configuration](https://gitlab.com/hmailserver/hmailserver/-/wikis/Capabilities-and-Configuration), [Settings Reference](https://gitlab.com/hmailserver/hmailserver/-/wikis/Settings-Reference) (9.12), [Connecting a Mail Client](https://gitlab.com/hmailserver/hmailserver/-/wikis/Connecting-a-Mail-Client), [Aliases, Lists and Public Folders](https://gitlab.com/hmailserver/hmailserver/-/wikis/Aliases-Lists-and-Public-Folders) (13.3), [Control Panel Reference](https://gitlab.com/hmailserver/hmailserver/-/wikis/Control-Panel-Reference) (7.5), [Security Hardening](https://gitlab.com/hmailserver/hmailserver/-/wikis/Security-Hardening) (17.5, 17.7), [Changes Since 6.2.10](https://gitlab.com/hmailserver/hmailserver/-/wikis/Changes-Since-6210) (4.3, 4.4). - At the 6.3.4 tag: [SettingsApply.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SettingsApply.md), [SendAs.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SendAs.md), [EncryptionAtRest.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/EncryptionAtRest.md), [DeckParity.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/DeckParity.md), [README.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/README.md), [CHANGELOG.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/CHANGELOG.md) (6.3.4, *IMAP, POP3 and the webmail*), [Roadmap.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/Roadmap.md). - The public documentation: [34.3 IMAP details worth knowing](https://www.progressiverobot.com/hmailserver-documentation/#343-imap-details-worth-knowing), [30.3 IMAP folders and permissions](https://www.progressiverobot.com/hmailserver-documentation/#303-imap-folders-and-permissions). - On this forum: [Upgrading to 6.3.4](/topic/28193), [The Control Deck](/topic/28192), [The REST API: turning it on](/topic/28264), [Message storage: object storage and encryption at rest](/topic/28233), [What runs out first: pools, connections, large folders](/topic/28238) and [Monitoring and health](/topic/28272). - A client that does not get on with these answers is a bug: a GitLab issue or, without a GitLab account, an email to contact-project+hmailserver-hmailserver-86730559-issue-@incoming.gitlab.com, filed as a confidential issue ([SUPPORT.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/SUPPORT.md)). A vulnerability goes only by [SECURITY.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/SECURITY.md)'s routes. --- Canonical: https://www.hmailserver.co.uk/topic/28224/imap-and-pop3-on-the-server-capabilities-switches-shared-mailboxes-search-limits # IMAP and POP3 on the server: capabilities, switches, shared mailboxes, search limits *Written for 6.3.4. hMailServer 6.3.7 (17 October 2026) changes some of what a client gets. IMAP `COPY` and `MOVE` by message number read their set in the client's numbering, and a set that names a message another session has just expunged is refused `NO [EXPUNGEISSUED]` for the client to send again; a `COPY` or `MOVE` that fails part way leaves the destination as it was; `SEARCH`, `SORT` and `THREAD` ask a criterion named again beside itself once; and `FETCH BODY[N.MIME]`, and `BODY[N]` of a multipart, are the stored octets. POP3 accepts the `UTF8` classic Outlook for Windows (2016 to 2024) sends after the login, until the session's first `STAT`, `LIST`, `UIDL`, `RETR`, `TOP`, `DELE` or `RSET`, so its downloads work again. SMTP, POP3 and ManageSieve refuse a SASL PLAIN sign-in that asks to act as another account. The IMAP and POP3 listeners can take the client's address from a PROXY protocol header, as the SMTP listener does, each from the peers on a list of its own, both off by default. The send-as grant now bears on DKIM too: an authenticated submission is signed as another hosted domain only when its account may send as the address its From names - its own, an alias pointing at it, a mailbox that granted it the post right, an address it proved, or a masked address of its own - and is otherwise accepted and sent unsigned as that domain. See [the 6.3.7 announcement](/topic/28277).* This topic is the server side of IMAP, POP3 and SMTP in hMailServer 6.3.4: what each listener advertises and which switch gates it, search limits and the full-text index, QRESYNC's expunge history, connection limits, shared mailboxes, and what 6.3.4 fixed that a client may notice. **Know this first: a capability missing from an answer is nearly always a switch or the session's state, not a fault.** Two IMAP switches ship off, and since 6.3.4 IMAP offers `STARTTLS` only before TLS and sign-in, and `COMPRESS=DEFLATE` only after sign-in. What a mail client needs is in [Connecting a mail client](/topic/28255); the ACL rights table is in [Aliases, distribution lists, mailing lists and public folders](/topic/28270). ### Seeing what the server advertises Ask the server, from any machine with OpenSSL. For SMTP type `EHLO test.example.org`; for IMAP, `CAPABILITY`: ``` openssl s_client -connect mail.yourcompany.com:25 -starttls smtp -quiet openssl s_client -connect mail.yourcompany.com:993 -quiet ``` On 6.3.4 ask IMAP before `LOGIN` and after: the answers differ by design. For POP3, connect to 995 the same way and send `CAPA`. Ports 993 and 995 exist only once you have created them (**Connections & protocols → TCP/IP ports** in the Windows Control Panel, or the Control Deck). ### IMAP: what is always there On every connection, as in 6.3.3: ``` IMAP4 IMAP4rev1 IMAP4rev2 CHILDREN THREAD=ORDEREDSUBJECT THREAD=REFERENCES NAMESPACE RIGHTS=texk MOVE ID SPECIAL-USE CREATE-SPECIAL-USE UNSELECT UIDPLUS ENABLE STATUS=SIZE ESEARCH CONDSTORE QRESYNC LIST-EXTENDED SEARCHRES WITHIN UTF8=ACCEPT LIST-STATUS PREVIEW MULTIAPPEND REPLACE SAVEDATE METADATA OBJECTID BINARY LITERAL- APPENDLIMIT ``` `APPENDLIMIT` is bare before authentication and `APPENDLIMIT=` after. `LITERAL-` caps non-synchronising literals at 4096 bytes, so an oversized `APPEND` can still be refused with `TOOBIG`. **A client opts in to IMAP4rev2 per session** with `ENABLE IMAP4rev2`, which implies `UTF8=ACCEPT`. `ENABLE` recognises exactly four names: `QRESYNC` (implies `CONDSTORE`), `CONDSTORE`, `UTF8=ACCEPT` and `IMAP4rev2`. After `ENABLE IMAP4rev2` the session drops the `RECENT` response and the `[UNSEEN]` code on `SELECT` and `EXAMINE`, drops `RECENT` as a `STATUS` item, and answers a `SEARCH` with no `RETURN` options as `ESEARCH`. ### IMAP: what is gated, and by what | Capability | Advertised when | Shipped | |---|---|---| | `IDLE` | `enableimapidle` is on | On | | `QUOTA QUOTA=RES-STORAGE` | `enableimapquota` is on | On | | `SORT` | `enableimapsort` is on | On | | `ACL` | `enableimapacl` is on; it also switches the `#Users` namespace | On | | `AUTH=PLAIN`, `AUTH=SCRAM-SHA-256` | `EnableImapSASLPlain` is on and authentication is available | **Off** | | `SASL-IR` | `EnableImapSASLInitialResponse` is on | **Off** | | `COMPRESS=DEFLATE` | `IMAPCompressionEnabled=1`, the client has signed in (since 6.3.4), and the session is not yet compressed | On | | `STARTTLS` | The port is STARTTLS-optional or -required, TLS is not active, and (since 6.3.4) the client has not signed in | Per port | | `LOGINDISABLED` | Authentication is refused on this cleartext connection (STARTTLS-required port, or `RequireTLSForAuth` on the IP range) | Per range | The rest depend on the connection: `AUTH=SCRAM-SHA-256-PLUS` only on TLS (and behind the same SASL switch), `AUTH=XOAUTH2 AUTH=OAUTHBEARER` with `OAuth2Enabled` and, unless `OAuth2RequireTLS=0`, only over TLS, `AUTH=EXTERNAL` on a verified client certificate that names an address, and `UNAUTHENTICATE` once signed in. - **`IDLE`** reports changes in the selected folder (`EXISTS`, `RECENT`, `EXPUNGE`, `FLAGS`) as they happen. A folder another client has just created is seen at the next folder-list refresh, as there is no IMAP `NOTIFY`; that is IMAP, not a fault. - **`QUOTA`** answers with one unnamed root, `""`, and a `STORAGE` resource in kilobytes from the account's maximum size; an account with no limit gets an empty list. `SETQUOTA` is refused, because the quota is the administrator's. Refusals of `APPEND`, `COPY` and `MOVE` for a full mailbox carry `OVERQUOTA`. - **`SORT`**: with message indexing off, as it ships, `SORT` reads each message's header from disk. - **`EnableImapSASLPlain`** gates SCRAM as well as PLAIN, despite its name. [Roadmap.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/Roadmap.md) says it gates the whole `AUTHENTICATE` command, while the Capabilities wiki gives OAuth2 and `EXTERNAL` conditions of their own: with it off, test `XOAUTH2`, `OAUTHBEARER` and `EXTERNAL` over IMAP before relying on them. **`COMPRESS DEFLATE`** (RFC 4978) compresses both directions of the rest of the session; a client that does not ask is unaffected. A second one is `NO [COMPRESSIONACTIVE]`, `STARTTLS` under compression is `BAD`, and `IMAPCompressionEnabled=0` withdraws it, the command then answering `BAD` like an unknown one. ### POP3 and SMTP: the same question **POP3 `CAPA`** always carries `UIDL`, `TOP`, `RESP-CODES`, `AUTH-RESP-CODE`, `IMPLEMENTATION hMailServer`, `PIPELINING`, `UTF8` and `EXPIRE NEVER`. The `SASL` mechanisms depend on TLS, `STLS` on the port, and `LOGIN-DELAY ` appears only when `Pop3LoginDelaySeconds` is above 0 (0 ships). That key advertises and enforces a minimum interval between sign-ins, refusing an early one with `-ERR [LOGIN-DELAY] …`; on Windows it is the POP3 tab's card *How often a client may check for mail*. **SMTP `EHLO`** always carries `SIZE ` (the maximum message size in KB times 1024; bare `SIZE` when 0), `8BITMIME`, `PIPELINING`, `CHUNKING`, `BINARYMIME`, `SMTPUTF8`, `ENHANCEDSTATUSCODES`, `DSN` and `HELP`. `BINARYMIME` is for local delivery: a recipient needing onward relay is refused at `RCPT TO` with 554 5.6.3. Conditionally: - `XCLIENT` only when `SMTPXClientEnabled=1` and the real peer is in `SMTPXClientTrustedIPs`; an untrusted peer that tries it gets 550. - `STARTTLS` on a cleartext connection to a STARTTLS-optional or -required port. - `AUTH` only when authentication is enabled, withheld on cleartext when the port requires STARTTLS or the range sets `RequireTLSForAuth`, and never on the ports in `DisableAUTHList`. - Not a keyword, but decided the same way: the HAProxy PROXY protocol is accepted only with `SMTPProxyProtocolEnabled=1` and from a peer in `SMTPProxyProtocolTrustedIPs`, and nothing in `EHLO` announces it. On SMTP and POP3 the `AUTH` line and its SASL responses take up to 12,288 octets; every other line keeps 510 (SMTP) or 500 (POP3). What 6.3.4 added to SMTP is refusals, not capabilities. ### Where the switches are set, and when a change applies | What | Windows Control Panel | Elsewhere | Applies | |---|---|---|---| | `IDLE`, `QUOTA`, `SORT`, `ACL`, the two SASL switches, the connection limits | **Connections & protocols → Protocols**, IMAP, SMTP and POP3 tabs | COM `Settings` (`IMAPIdleEnabled` and its neighbours); REST `PUT /api/v1/settings` (`imap_idle_enabled`, `imap_quota_enabled`, `imap_sort_enabled`, `imap_acl_enabled`, `imap_sasl_plain_enabled`, `imap_sasl_initial_response_enabled`); the Deck's Settings view; `hmctl settings set` | At once | | Message indexing | **Maintenance → Performance**, Indexing tab | COM `Settings.MessageIndexing`; `PUT /api/v1/settings/indexing` (`enabled`); `hmctl settings`, group `indexing` | At once | | `IMAPCompressionEnabled`, `IMAPSearchTimeout`, `IMAPSearchMaxMegabytes`, `IMAPExpungeRetentionRecords`, `Pop3LoginDelaySeconds`, `IndexerFullText` and its tuning keys | Cards on the IMAP and POP3 tabs; the Indexing tab | COM `Settings.SetIniSetting`; `PUT /api/v1/settings/ini/{name}` (administrator password only); `hmctl ini`; on Linux `hmailserver --set-setting NAME=VALUE`. No Deck page | When saved | The Control Panel and COM are Windows-only. The last row's keys were the `[Settings]` section of `hMailServer.ini`; **since 6.3.4 the database holds them**, none of them waits for a reinitialise or a restart ([docs/SettingsApply.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SettingsApply.md)), and the server does not take a line edited in the file, so set them through an interface. [Settings: when a save takes effect](/topic/28222) covers the store, `HM5804`, and a value stored by `--set-setting` while the service runs. The REST lines assume the listener on `127.0.0.1:8443`, as the REST API wiki's examples do; `RestApiPort` ships at 0 (off) and waits for a reinitialise. ```bash # Switch IDLE off; in effect when the answer comes back curl -u Administrator:your-admin-password -X PUT -H "Content-Type: application/json" \ -d '{"imap_idle_enabled":false}' http://127.0.0.1:8443/api/v1/settings # A [Settings] key: the administrator password only, never an API key curl -u Administrator:your-admin-password -X PUT -H "Content-Type: application/json" \ -d '{"value":"120"}' http://127.0.0.1:8443/api/v1/settings/ini/IMAPSearchTimeout # On Linux, then a restart so the running server takes it sudo hmailserver --set-setting IMAPSearchTimeout=120 sudo systemctl restart hmailserver ``` ### Search: two ceilings and the full-text index **Without the full-text index, `SEARCH BODY` and `TEXT` read every message in the mailbox**, attachments whose media type is text included (decoded, bounded at 1 MB). Two ceilings, measured from the moment a command starts and never re-armed by progress, bound `SEARCH`, `SORT` and `THREAD`: `IMAPSearchTimeout` (60 seconds) and `IMAPSearchMaxMegabytes` (2048 MB of message content read); 0 turns either off. They are the IMAP tab's card *IMAP search limits*, and [README.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/README.md) says to raise them for mailboxes of several hundred thousand messages. **What a client gets at a ceiling is stated two ways.** [Roadmap.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/Roadmap.md) records `NO` with RFC 9051 `LIMIT` and no untagged result, so a cut-short list cannot pass for a complete one; [README.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/README.md)'s comment beside `IMAPSearchTimeout` says the search returns the matches found so far. Until that is settled, treat a search that hit a ceiling as incomplete, and let nothing act on its result in bulk. **The full-text index is off, and needs two switches**: `IndexerFullText=1` indexes nothing unless message indexing (`Settings.MessageIndexing`) is on too. It costs a term table of a few kilobytes per message and a backfill pass over every message already delivered, which is why no upgrade turns it on. **The index can only prove that a message cannot match**; everything else is read and scanned as before, so results are identical on or off. The webmail's search and eDiscovery use the same filter. | Key | Default | What it does | |---|---|---| | `IndexerFullTextBatchSize` | 250 (1 to 100000) | Messages one backfill pass reads before the cursor is saved and the thread pauses | | `IndexerFullTextMinTokenLength` | 3 (3 to 64) | The shortest search string the index answers for | | `IndexerFullTextMaxTokensPerMessage` | 2048 (64 to 1000000) | Distinct terms before a message is marked always-scanned instead of indexed | The Indexing tab's *Index now, and show how far behind it is* and *Discard the index and rebuild it* are `POST /api/v1/settings/indexing/index` and `/clear` over REST, and `Index()` and `Clear()` over COM, where `TotalIndexedCount` against `TotalMessageCount` shows a rebuild's progress. With mailboxes encrypted at rest, the indexed count stays short by their messages for good: those are never indexed. ### QRESYNC: how much expunge history is kept `IMAPExpungeRetentionRecords` keeps the newest 5000 expunge records per mailbox (0 keeps all), pruned every twelve hours and once at service start; on Windows it is the card *Change history for synchronising clients*. The newest is never removed. **A client whose last sync is older than what is kept gets a full resync, never a wrong one**: every UID in its range that the mailbox does not hold. An installation upgraded from a release older than 6.2.23-alpha1 prunes the table in one pass at its first start. [README.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/README.md) says 0 defers that prune, and an `IMAPExpungeRetentionRecords` line of 0 added to `[Settings]` before the upgrade is taken: 6.3.4's first start adopts from the file a key the database has never held, and before 6.2.23-alpha1 the database held this one only if the file had the line. Restart the old service once after adding or changing it (Upgrading hMailServer wiki, 18.3, step 6): [CHANGELOG.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/CHANGELOG.md)'s note that a line edited to prepare an upgrade is put back concerns a key the database already holds, and that restart settles it. Set it back to 5000 afterwards through an interface; [Upgrading to 6.3.4](/topic/28193) has the rest of the order. ### Connection limits and what guards them `MaxSMTPConnections`, `MaxIMAPConnections` and `MaxPOP3Connections` are on the Protocols page (REST `max_smtp_connections`, `max_imap_connections`, `max_pop3_connections`). None has a limit by default; SMTP and IMAP ship at 0, no cap. The Services tab turns a protocol off altogether (`ServiceSMTP`, `ServiceIMAP`, `ServicePOP3`). Watch `hmailserver_sessions{protocol}` on `/metrics` (off until `MetricsServerPort` is set, which waits for a reinitialise) or `sessions` in `GET /api/v1/status`. Around the limits: auto-ban counts failed sign-ins per IP across SMTP, IMAP, POP3, ManageSieve and REST (on: 3 in 30 minutes, a 60-minute ban); `LogonTarpitSeconds` (off) delays each failed sign-in's refusal; a second POP3 session on one mailbox gets `-ERR [IN-USE]`; and since 6.3.4 an IMAP `LIST` or `LSUB` pattern over 8,192 characters, or more than 32 patterns in one `LIST`, is answered `BAD`. The submission rate limit and the tarpits are in [IP ranges, auto-ban, lockout and tarpits](/topic/28239); refusing a connection by country is in [allow and block lists and country blocking](/topic/28249). ### Opening a mailbox to a colleague Shared and delegated mailboxes are IMAP ACL (RFC 4314). The owner opens a folder to another account, and it appears to that account under the Other Users namespace as `#Users.owner@domain.folder`. **No administrator step is needed**: any account whose client speaks `SETACL` can do it, and the audit trail records no row for it. 1. Leave ACL on. With it off, `#Users` stops existing (`NAMESPACE` advertises `NIL`), public folders are open to everyone, and the send-as grant is not honoured. 2. The owner grants rights from an IMAP client with ACL support. An administrator can do it over REST: `GET` on `/api/v1/accounts/{address}/folders` gives the folder's id, and `/api/v1/accounts/{address}/folders/{id}/permissions` lists, adds, changes and removes entries (since 6.3.3; writes need a full key). On Windows, COM has `IMAPFolderPermissions.Add`. The grantee is an account, a group (`/api/v1/groups`, since 6.3.4), or Anyone, which is every authenticated account on the server. 3. Grant `l`, or the folder does not exist for the colleague. `r` reads, `s` changes read state, `i`, `t` and `e` file and delete, and `p` on the INBOX is the send-as grant. 4. Grant on each top-level folder needed. A right is decided at the nearest folder up the owner's tree that has any entries, and entries are not merged: a grant on INBOX reaches its sub-folders that have none of their own, not the Sent folder, and one entry on a sub-folder cuts it off from the parent's. 5. Over SMTP the `p` grant counts only while `SmtpAuthenticatedSenderCheck` is on, and it ships off: until then nothing constrains an authenticated `MAIL FROM`, so any account can send as the mailbox, or as anyone else. Turned on, it applies to every authenticated session on the server; the addresses it then allows are in [who may send as whom](/topic/28239). 6. In the colleague's client, a pane of subscribed folders shows only the ones the owner subscribed to; the webmail shows them under `Shared by `. **What follows from sharing:** - `\Seen` is stored on the message, not per person: with `s`, a delegate marks a message read for everyone; with `lr` alone they leave no trace, and their client cannot track what they have read. - Since 6.3.4 an `APPEND` into a `#Users` folder is charged to the owner's quota, as `COPY` and `MOVE` already were. - A delegate reads a folder of a mailbox encrypted at rest only while the owner's key is open; otherwise a `FETCH` that reads the message answers `NO [UNAVAILABLE]`, and copying out is refused. - In the webmail a read-only share refuses a flag or a delete with 403, a message cannot move between one's own mailbox and a shared one, and the owner cannot grant access from the page. - Since 6.3.4 a restore puts each folder's access grants back. Every `LIST` runs one more query while ACL is on. ### What 6.3.4 fixed that a client may notice Where Dovecot's `imaptest` found IMAP answering other than the RFCs say (`\Seen` without `.PEEK`, `APPEND` byte for byte, `SORT` and `THREAD` order, among others), [Connecting a mail client](/topic/28255) has the list. The rest: | What a client did | What it gets since 6.3.4 | |---|---| | Sent `STARTTLS` after `LOGIN`, or inside TLS | `BAD`, and no `STARTTLS` in `CAPABILITY` | | Sent `COMPRESS DEFLATE` before signing in | `NO` | | Appended over a compressed connection, data inflating past the 250,000-octet receive buffer | The whole message, where it was stored cut short or never finished | | Read `ENVELOPE` with a backslash in `Date`, or `BODYSTRUCTURE` with a quote or backslash in a part's parameters | A structure that parses: the date's backslashes left out; quotes and backslashes escaped, a control character sent as a space | | Fetched a message the server cannot read | `NO [UNAVAILABLE]` for a closed key; a damaged one (`HM7350`) left out, the rest fetched and the command ending `NO` - not `OK` with an empty `ENVELOPE` | | Changed one message's flags from two sessions at once | Both changes kept; `STORE ... (UNCHANGEDSINCE n)` tested in the same step | | Went on sending a `MULTIAPPEND`'s later literals after a refusal, one over the maximum message size (and 1 MB) | The connection closed with `BAD [TOOBIG]`, where it was read whatever its size | | Fetched a message only an unreachable object store holds | IMAP `NO [UNAVAILABLE]`, POP3 `-ERR [SYS/TEMP]` (object storage is new in 6.3.4, and off) | Two heap-corruption aborts are gone as well (two sessions on one message's keywords; `LIST` beside folder changes). Most of these were in 6.3.3 too. ### What it does not do - **No APOP or CRAM-MD5** on POP3, and `THREADID` is answered `NIL`. - **Delegation is not measured in Thunderbird or Outlook**, and an Outlook IMAP profile never acts on the Autodiscover answer's delegated mailboxes; the `#Users` namespace is what works. ### Where to read more - Wiki: [Capabilities and Configuration](https://gitlab.com/hmailserver/hmailserver/-/wikis/Capabilities-and-Configuration), [Settings Reference](https://gitlab.com/hmailserver/hmailserver/-/wikis/Settings-Reference) (9.12), [Connecting a Mail Client](https://gitlab.com/hmailserver/hmailserver/-/wikis/Connecting-a-Mail-Client), [Aliases, Lists and Public Folders](https://gitlab.com/hmailserver/hmailserver/-/wikis/Aliases-Lists-and-Public-Folders) (13.3), [Control Panel Reference](https://gitlab.com/hmailserver/hmailserver/-/wikis/Control-Panel-Reference) (7.5), [Security Hardening](https://gitlab.com/hmailserver/hmailserver/-/wikis/Security-Hardening) (17.5, 17.7), [Changes Since 6.2.10](https://gitlab.com/hmailserver/hmailserver/-/wikis/Changes-Since-6210) (4.3, 4.4). - At the 6.3.4 tag: [SettingsApply.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SettingsApply.md), [SendAs.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SendAs.md), [EncryptionAtRest.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/EncryptionAtRest.md), [DeckParity.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/DeckParity.md), [README.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/README.md), [CHANGELOG.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/CHANGELOG.md) (6.3.4, *IMAP, POP3 and the webmail*), [Roadmap.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/Roadmap.md). - The public documentation: [34.3 IMAP details worth knowing](https://www.progressiverobot.com/hmailserver-documentation/#343-imap-details-worth-knowing), [30.3 IMAP folders and permissions](https://www.progressiverobot.com/hmailserver-documentation/#303-imap-folders-and-permissions). - On this forum: [Upgrading to 6.3.4](/topic/28193), [The Control Deck](/topic/28192), [The REST API: turning it on](/topic/28264), [Message storage: object storage and encryption at rest](/topic/28233), [What runs out first: pools, connections, large folders](/topic/28238) and [Monitoring and health](/topic/28272). - A client that does not get on with these answers is a bug: a GitLab issue or, without a GitLab account, an email to contact-project+hmailserver-hmailserver-86730559-issue-@incoming.gitlab.com, filed as a confidential issue ([SUPPORT.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/SUPPORT.md)). A vulnerability goes only by [SECURITY.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/SECURITY.md)'s routes. --- Canonical: https://www.hmailserver.co.uk/topic/28225/troubleshooting-by-symptom-no-mail-in-no-mail-out-a-service-that-will-not-start # Troubleshooting by symptom: no mail in, no mail out, a service that will not start *Written for 6.3.4. hMailServer 6.3.7 (17 October 2026) changes what an upgraded server does without anything being switched on, and some of it looks like a fault. A forward, a redirect or a rule's forward of mail from outside that did not pass DMARC for the forwarding domain's organisation keeps the envelope sender it came with, with no SRS address or rewritten sender; where the copy would still carry an identity of that organisation (an empty envelope sender under this server's host name, or an envelope sender at this server's host name or at a hosted domain in that organisation) it is not sent to an outside address at all: the message stays in the mailbox, and the application log names each recipient. An authenticated submission is signed as another hosted domain only when the account may send as the address its From names, and is otherwise sent unsigned as that domain. A domain's or an account's rule may no longer set From, Sender, Resent-From, Resent-Sender, DKIM-Signature or the loop count, and a Sieve script may no longer add or delete them: one stored before the upgrade still runs without that action, and the application log names it. `RuleLoopLimit` 0 stops out-of-office replies. A delivery whose mail servers' DNS lookup failed tells the sender that the DNS query failed (4.4.4) and retries. On Windows the x64 installer's MariaDB Connector/C 3.4.11 has seven authentication plugins where 6.3.6's had nine, so a database account that signs in with MySQL Enterprise's Windows authentication can no longer sign in and the server cannot open its database. An error log that was quiet may now name database failures that were hidden (`HM5032`), and the Control Panel's API & monitoring page is Monitoring & alerts. Fixed from reports: the Control Panel's IP ranges page that could show no list at all, DANE enforcement deferring mail to a DNSSEC-signed domain whose MX answer came compressed (live.com among them), and classic Outlook's POP3 downloads stopping at the `UTF8` it sends after the login. See [the 6.3.7 announcement](/topic/28277).* This topic is the wiki's Troubleshooting chapter arranged by the sentence support threads open with: mail from outside does not arrive, a client cannot send, the Control Panel or the Deck will not connect, the service will not start. It also covers the server's replies, the Diagnostics self-test, the message trace and what only looks like a fault. The one thing to know first: on Windows the service tells the Service Control Manager it is **running** before it has opened its database or a single port, so "the service is running" proves nothing. Today's `ERROR_hmailserver_.log` does: it almost always holds the reason a start failed. ### Start here Three pages, in this order, under **Monitoring & troubleshooting** in the Control Panel (Windows): 1. **Diagnostics**, the built-in self-test (below). It names most problems outright. 2. **Live logs**, while you reproduce the problem, with **debug messages** on if the answer is not obvious, and off afterwards. 3. **Delivery queue**: is mail stuck, and what do **NextTry** and **Tries** say? On Linux the Deck's Diagnostics, Logs and queue views read the same, and three commands start every investigation: ```sh systemctl status hmailserver journalctl -u hmailserver -n 100 --no-pager sudo tail -n 100 /var/log/hmailserver/ERROR_hmailserver_$(date +%F).log ``` **Which problem is it?** Not running, or nothing listening: *The service will not start*. A sender got an SMTP error: the reply names the cause. A sender timed out, or outbound mail sits in the queue: [Diagnosing stalled mail](/topic/28219). Inbound with nothing at all: *I can't receive*. Outbound that never reaches the queue: *I can't send*. ### "I can't receive mail from outside" Work down the table in order, cheapest check first. | Check | How (Windows) | A failure means | |---|---|---| | Is the service running? | `Get-Service hMailServer` | Start it, then read the error log for why it stopped | | Is anything listening on 25? | `Get-NetTCPConnection -LocalPort 25 -State Listen` | The port is not configured, is bound to one address only, or another program has it: **Connections & protocols → TCP/IP ports** | | Is 25 open in Windows Firewall? | `Get-NetFirewallRule -Enabled True \| Where-Object DisplayName -like "*hMail*"` | Open it: see the firewall paragraph below | | Does the machine itself answer on 25? | `Test-NetConnection 127.0.0.1 -Port 25` | Then it is the server, not the network | | Is 25 forwarded by your router or cloud firewall? | Its configuration | The far end sees a refusal or a timeout | | Does your ISP block 25? | Ask them | Common on consumer lines: a relay host or a business line | | Is the MX record right? | The **MX query** page, or `Resolve-DnsName example.com -Type MX` | Fix DNS | | Can the world reach you? | From **outside**: `Test-NetConnection mail.example.com -Port 25` | It is the path in | | Is the domain in hMailServer and **Active**? | **Accounts & domains → Domains** | An inactive domain is not accepted for | | Is the mailbox **Active**, and full? | The account dialog | A full mailbox gets a temporary failure, and the sender retries | A port bound to one internal address answers only on the LAN. **Server status** lists the listeners the service actually brought up. **The firewall.** The installer opens no port, so nothing from outside reaches the server until you open them, on the machine and on the router or cloud firewall in front of it. Which ports, and the rules for Windows and Linux, are in [Every port hMailServer listens on or connects to](/topic/28226). ### "I can't send mail" The reply names most sending faults. In an ESMTP session an enhanced status code follows the number: `530 Must issue STARTTLS first.` arrives as `530 5.7.0 Must issue STARTTLS first.` | Reply | Likely cause | What to do | |---|---|---| | `550 Delivery is not allowed to this address.` ("relay denied") | The client is not authenticating, or uses 25 instead of 587 | Port 587 with authentication; a printer that cannot authenticate gets its own range under **Access & abuse protection → IP ranges** | | `530 SMTP authentication is required.` | The same, said earlier | As above | | `530 Must issue STARTTLS first.` | The port requires TLS first | STARTTLS or SSL/TLS in the client | | `535 Authentication failed.` | Wrong password, not the full address as user name, or an inactive account | Also check the auto-ban | | `535 Authentication failed. Too many invalid logon attempts.` | The auto-ban: **3** failures, **60** minutes, by default | Below | | `550 Too many recipients.` | Too many recipients in one transaction | Below | | `552 5.3.4 Message size exceeds fixed maximum message size. Size: N KB, Max size: M KB` | Over the server's maximum message size (20480 KB), or the domain's own | **Connections & protocols → Protocols**, SMTP tab, or the domain dialog | | `452 4.3.1 Insufficient system storage.` | Free space on the store's volume below `MinimumFreeDiskSpaceMB` (100 MB) | Free space; senders retry | **A banned client.** The ban is an expiring IP range: **Access & abuse protection → IP ranges** lists it, and deleting it releases the client (`DELETE /api/v1/ipranges/`, or the Deck's IP ranges view). **`550 Too many recipients.`** Two wiki pages disagree. Troubleshooting §19.3 ties it to `maxsmtprecipientsinbatch` (100) and says split the batch or raise the limit; the Architecture page's `RCPT TO` chart refuses at 50,000 recipients and uses `MaxSMTPRecipientsInBatch` to batch outbound deliveries. Splitting the message is right either way; raising the setting also changes the outbound batch size. ### What the server's replies mean A `4xx` is temporary and the sender retries; a `5xx` is permanent and the sender bounces. | Reply | Meaning | |---|---| | `421 4.7.0 Too many invalid commands. Bye!` | Disconnected for misbehaving: a scanner, or a broken client | | `451 4.3.1 Server temporarily overloaded while accepting the message; please retry.` | The accept pipeline exceeded `FinalizationTimeout`: [Diagnosing stalled mail](/topic/28219) | | `451 4.3.2 Unable to verify the recipient at the moment. Please retry later.` | The database did not answer, so the mail is deferred, not bounced. Check `/readyz` | | `451 4.3.2 The server is shutting down and cannot accept the message.` | A graceful stop with `ShutdownDrainSeconds` set | | `452 4.2.2 Mailbox is full.` | At or over quota, refused during the conversation; the holder had a `QUOTA_WARNING` | | `550 Unknown user` | No such mailbox, alias or list | | `553` or `554` from a *remote* server | The far end refused you; its text is the diagnosis, usually SPF, DKIM, PTR or a block list | ### "My mail goes to spam" Almost always DNS, in this order: the PTR record (nothing compensates for a missing or generic one), SPF, DKIM, DMARC even at `p=none`, then the block lists. With split-horizon DNS, the outside view is the one that counts. [Being trusted by the servers you send to](/topic/28188) covers it. ### The Control Panel, the Deck or the REST listener will not connect **The Control Panel** (Windows) talks to the running service over COM: no service, no connection. Use the **administration** password, not a mailbox's. After a restart it reconnects by itself; after you stop the service it deliberately does not. Remotely it needs port 135 and DCOM. Every page saying "Server unavailable" means the engine is paused: **Server status → Resume**. *Since 6.3.4*, under `NT SERVICE\hMailServer`, a change of the administrator password is refused because the service may not write the file: run `hMailServer.exe /SetAdministratorPassword` from an elevated prompt, then restart the service. More in [The Control Panel on Windows](/topic/28223). **The Deck and the REST API** share one listener, off until `RestApiPort` is set. It refuses to start without an administrator password, beyond loopback without a certificate, or with a certificate it cannot load (`HM5113` names the file), each logged once as `RestApi: Refusing to start`; the three lines and their fixes are in [The REST API: turning it on](/topic/28264). Only a `RestApi: Listening on` line proves it is up. *Since 6.3.4* the port waits for a reinitialise (**Apply now**, or `POST /api/v1/server/reinitialize`), and a line edited into `hMailServer.ini` after the first start loses to the stored value (HM5804). With the listener off, on Linux: ```sh sudo hmailserver --set-setting RestApiPort=8045 sudo systemctl restart hmailserver ``` On Windows, the Control Panel's **API & monitoring** page (**Monitoring & alerts** since 6.3.7). "Web administration page not installed" means the `WebAdmin` files are missing, or `ProgramFolder` does not point at the directory holding them. Repeated bad passwords get the address refused for five minutes; loopback is exempt. The Deck itself is [The Control Deck](/topic/28192). ### The service will not start The start, in order; where it stops names the problem: 1. **Windows is told the service is running**, before any of what follows. 2. **`hMailServer.INI` is read, alone**, which is how the database is found. A `[Database] Type` it does not recognise abandons the start, with nothing logged about a database. On Linux, `sudo -u hmailserver hmailserver --check-config` then prints `Database type: 0`. 3. **`HM5005`**: `Server` or `Database` is missing for the configured `Type`. 4. **The database is connected**, retried `ConnectionAttempts` times (6), `ConnectionAttemptsDelay` seconds apart (5). `HM4354`: the database server is down or moved, or the credentials changed. On Windows the `[Database]` password is DPAPI-protected per machine and cannot be copied from another server. *Since 6.3.4*, `HM7530` (an unknown `TlsMode`) and `HM7531` (`verify-full` with a missing `TlsCaFile`) stop it too; `--check-database` makes one connection and says what it got. 5. **The schema version**, compared both ways. `HM5010`: `Database version could not be detected.` `HM5011` too old: `DBUpdater.exe` from `Bin` on Windows, `sudo -u hmailserver hmailserver --upgrade-database` on Linux. `HM5011` too new: install the matching or a newer server; never downgrade the database. *Since 6.3.4*, `[Database] UpgradeOnStart=1` makes the service upgrade the schema as it starts: off by default, no backup first, one node at a time. 6. **Settings are read again** from the database, since 6.3.4 with any `[SettingsOverride]` keys over them. A configuration that will not load abandons the start. 7. **One listener per TCP/IP ports row.** A row whose protocol is off on **Protocols → Services** is skipped in silence, the commonest reason for nothing on 143. `HM4316`: something else holds the port (IIS on 25 or 443, another mail server, a leftover instance); that port alone stays down. `HM5113` with `Error initializing SSL. Certificate not set.`: that listener does not start. 8. **The optional listeners** (metrics, REST, web services, ManageSieve), each only with a non-zero port. `MetricsServer: Invalid bind address` means `MetricsServerBindAddress` is not an IP literal (`localhost` is refused too); that listener stays down and the server runs. A failure in steps 2 to 6 stops the whole server, and one with a code writes it to the ERROR log; steps 7 and 8 cost one listener and go to the application log. **Windows.** A service that stops at once with nothing logged is the account's rights on `Bin`, `Data`, `Logs` and `Database`. *Since 6.3.4* a new installation runs as `NT SERVICE\hMailServer` (an upgrade keeps its account), and a folder outside the program folder needs approving: run `hMailServer.exe --confirm /GrantServiceAccess` elevated in `Bin`, and approve only a folder you recognise, since the service may then write in it. Local SQL Server with integrated authentication needs a login for `NT SERVICE\hMailServer`. What the account may not do, and moving between it and LocalSystem, are in [Installing on Windows](/topic/28227). The installer's recovery settings restart a failed service after 1, 2 and 5 minutes (`sc.exe qfailure hMailServer`). **Linux, under systemd.** A start that fails before any listener opens puts `hMailServer could not start:` in the journal, with the reason on standard error in case the log directory is the fault. The unit runs as `hmailserver`, and `ProtectSystem=strict` makes anything outside its `ReadWritePaths` (`/var/lib/hmailserver` and `/var/log/hmailserver`) read-only to the service: a data, backup, archive or ACME directory elsewhere needs a drop-in (`sudo systemctl edit hmailserver`). `HM5094`: the MySQL or MariaDB client library is not installed. `HM5603` on every TLS listener: `/usr/bin/dh2048.pem` is missing; reinstall the package. ### The Diagnostics self-test **Monitoring & troubleshooting → Diagnostics** in the Control Panel; *since 6.3.4* also the Deck's Diagnostics view and `POST /api/v1/diagnostics/run`, on either platform. Ten tests: | Test | Fails when | |---|---| | Server details | Never; since 6.3.4 it also says whether the database connection is encrypted | | Installation paths *(6.3.0)* | A configured path does not exist | | Test IPv6 | Never | | Test outbound port | Outbound 25 is blocked, or the smart host's name, port or TLS mode is wrong | | Test backup directory | No destination, or the service cannot write there | | Test MX records | Public DNS is wrong, or the resolver is not answering | | Test local connect | Hairpin NAT fails, 25 is not forwarded, or the MX points elsewhere | | Test message file locations | Files outside the data directory, or full paths stored, which stops the built-in backup | | Test IP range configuration | `127.0.0.1` has been auto-banned | | Error logs | Any `ERROR*` file exists | Fill both domain boxes first: the MX and local-connect tests, and the outbound test without a smart host, are skipped rather than failed when they are empty. **Copy** puts the result on the clipboard for a post. ### "Did we ever get it?": the message trace The message trace records what the server did with each message, keyed on its queue id. **It is off by default** (`MessageTraceEnabled`), because it records who corresponds with whom, and nothing is recorded while it is off: turn it on before you need it. **Turning it on.** *Since 6.3.4* it applies when saved: the Control Panel's **Message trace** card on **Monitoring & troubleshooting → Logging**, `PUT /api/v1/settings/ini/{name}`, or `hmctl`, both with the administrator password: ```sh hmctl ini set MessageTraceEnabled 1 ``` On 6.3.3 and earlier it is `MessageTraceEnabled=1` under `[Settings]` and a restart. Rows are kept `MessageTraceRetentionDays` (30): since 6.3.4 an hourly task deletes older ones; on 6.3.3 and earlier only **Remove expired** on the Message trace page, or `MessageTrace.DeleteExpired` over COM, did. **Reading it.** The Control Panel's **Message trace** page searches an address as sender or recipient, newest first, and **Follow this message** shows one queue id. Over REST *since 6.3.4*: `GET /api/v1/message-trace`, `/api/v1/message-trace/address/{address}` and `/api/v1/message-trace/queue/{id}`, which need the `view_message_trace` permission. ```sh hmctl api GET /api/v1/message-trace/address/user@example.com ``` **What a row says.** One row per recipient per delivery attempt: when, the event, sender, recipient, source address and status code. Events include `delivered` and `failed`; on 6.3.4 also `deferred` (an attempt that will be retried) and `suppressed` (stopped by the suppression list). A refusal at `RCPT` carries queue id **0**. A forward, a list or a rule that re-sends makes a new queue entry. **Answering the question.** A `delivered` row: you had it; look at the account's folders and rules. A row with queue id 0: refused at `RCPT`, and the status says why. No row: either it never reached you, or it is still queued or held, came while the trace was off, or is past the retention. The trace records delivery attempts only, with no acceptance, spam or virus event, so an empty trace does not prove the server never had the message. The per-domain reports' traffic, failures, senders and recipients are counted from it, and say so while it is off: [Alerts, the audit trail and per-domain reports](/topic/28273). ### Things that look like faults and are not | What you see | Why | |---|---| | A new sender's first message is minutes late | Greylisting doing its job | | *(6.3.4)* A value typed into `[Settings]` is back after a restart, HM5804 in the ERROR log | The database is the settings store: set it in the Control Panel, over REST (`PUT /api/v1/settings/ini/{name}`) or with `hmctl ini set` | | *(6.3.4)* HM5807 at every start | A `[SettingsOverride]` section is forcing keys; remove it once the stored value is right | | *(6.3.4, Linux package)* HM5803 at every start | The service cannot write its copy into the root-owned file; the stored value is in effect | | 6.2.28 to 6.3.3: "The update feed answered HTTP 404." | Set `UpdateFeedUrl=https://updates.progressiverobot.com/releases.json` and restart | | "Release 6.3.4 carries no Sigstore bundle for its installer, so it cannot be verified." | Install 6.3.4 by hand: [Upgrading to 6.3.4](/topic/28193) | **Where a message waits, since 6.3.4.** Neither delivered nor queued may mean held on purpose: by the sandbox, by compromised-account detection, in a back-off window, under an IP warm-up ceiling or behind a drain; [Diagnosing stalled mail](/topic/28219) says where to look. And `550 5.7.1 Too many delivery attempts before the greylisting delay had passed` is greylisting's abuse lock, lifted after `GreyListingAbuseLockMinutes` (60); a `5.7.10` bounce to a partner domain is gateway encryption refusing to send in the clear, the partner's key having expired. ### What it does not do It does not diagnose a mail client ([Connecting a mail client](/topic/28255)), read logs line by line ([Reading the logs](/topic/28220)), or work a stalled queue. The Control Panel and COM are Windows-only. When nothing here names the fault, [Read this before you post](/topic/28185) says what a request needs. Without a GitLab account, email the Service Desk, contact-project+hmailserver-hmailserver-86730559-issue-@incoming.gitlab.com, which GitLab files as a confidential issue. ### Where to read more - Wiki: [Troubleshooting](https://gitlab.com/hmailserver/hmailserver/-/wikis/Troubleshooting), [Ports Reference](https://gitlab.com/hmailserver/hmailserver/-/wikis/Ports-Reference), [Monitoring and Health](https://gitlab.com/hmailserver/hmailserver/-/wikis/Monitoring-and-Health), [Installing hMailServer](https://gitlab.com/hmailserver/hmailserver/-/wikis/Installing-hMailServer), [Installing on Linux](https://gitlab.com/hmailserver/hmailserver/-/wikis/Installing-on-Linux), [The Control Panel](https://gitlab.com/hmailserver/hmailserver/-/wikis/The-Control-Panel), [The REST API](https://gitlab.com/hmailserver/hmailserver/-/wikis/The-REST-API), [Upgrading hMailServer](https://gitlab.com/hmailserver/hmailserver/-/wikis/Upgrading-hMailServer), [Architecture](https://gitlab.com/hmailserver/hmailserver/-/wikis/Architecture). - Documents: [ServiceAccount.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/ServiceAccount.md), [DatabaseTls.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/DatabaseTls.md), [Reports.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Reports.md), [SettingsApply.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SettingsApply.md). - On the forum: [Settings: when a save takes effect](/topic/28222) (HM5804 and the reinitialise), [IP ranges, auto-ban and lockout](/topic/28239), [Choosing a database and encrypting the link](/topic/28228) (`TlsMode`, HM7530 and HM7531) and [The hmailserver command line on Linux](/topic/28245) (the unit and its drop-ins). - Public documentation: [Troubleshooting](https://www.progressiverobot.com/hmailserver-documentation/#19-troubleshooting) and [Troubleshooting a Linux installation](https://www.progressiverobot.com/hmailserver-documentation/#3516-troubleshooting-a-linux-installation). --- Canonical: https://www.hmailserver.co.uk/topic/28225/troubleshooting-by-symptom-no-mail-in-no-mail-out-a-service-that-will-not-start # Troubleshooting by symptom: no mail in, no mail out, a service that will not start *Written for 6.3.4. hMailServer 6.3.7 (17 October 2026) changes what an upgraded server does without anything being switched on, and some of it looks like a fault. A forward, a redirect or a rule's forward of mail from outside that did not pass DMARC for the forwarding domain's organisation keeps the envelope sender it came with, with no SRS address or rewritten sender; where the copy would still carry an identity of that organisation (an empty envelope sender under this server's host name, or an envelope sender at this server's host name or at a hosted domain in that organisation) it is not sent to an outside address at all: the message stays in the mailbox, and the application log names each recipient. An authenticated submission is signed as another hosted domain only when the account may send as the address its From names, and is otherwise sent unsigned as that domain. A domain's or an account's rule may no longer set From, Sender, Resent-From, Resent-Sender, DKIM-Signature or the loop count, and a Sieve script may no longer add or delete them: one stored before the upgrade still runs without that action, and the application log names it. `RuleLoopLimit` 0 stops out-of-office replies. A delivery whose mail servers' DNS lookup failed tells the sender that the DNS query failed (4.4.4) and retries. On Windows the x64 installer's MariaDB Connector/C 3.4.11 has seven authentication plugins where 6.3.6's had nine, so a database account that signs in with MySQL Enterprise's Windows authentication can no longer sign in and the server cannot open its database. An error log that was quiet may now name database failures that were hidden (`HM5032`), and the Control Panel's API & monitoring page is Monitoring & alerts. Fixed from reports: the Control Panel's IP ranges page that could show no list at all, DANE enforcement deferring mail to a DNSSEC-signed domain whose MX answer came compressed (live.com among them), and classic Outlook's POP3 downloads stopping at the `UTF8` it sends after the login. See [the 6.3.7 announcement](/topic/28277).* This topic is the wiki's Troubleshooting chapter arranged by the sentence support threads open with: mail from outside does not arrive, a client cannot send, the Control Panel or the Deck will not connect, the service will not start. It also covers the server's replies, the Diagnostics self-test, the message trace and what only looks like a fault. The one thing to know first: on Windows the service tells the Service Control Manager it is **running** before it has opened its database or a single port, so "the service is running" proves nothing. Today's `ERROR_hmailserver_.log` does: it almost always holds the reason a start failed. ### Start here Three pages, in this order, under **Monitoring & troubleshooting** in the Control Panel (Windows): 1. **Diagnostics**, the built-in self-test (below). It names most problems outright. 2. **Live logs**, while you reproduce the problem, with **debug messages** on if the answer is not obvious, and off afterwards. 3. **Delivery queue**: is mail stuck, and what do **NextTry** and **Tries** say? On Linux the Deck's Diagnostics, Logs and queue views read the same, and three commands start every investigation: ```sh systemctl status hmailserver journalctl -u hmailserver -n 100 --no-pager sudo tail -n 100 /var/log/hmailserver/ERROR_hmailserver_$(date +%F).log ``` **Which problem is it?** Not running, or nothing listening: *The service will not start*. A sender got an SMTP error: the reply names the cause. A sender timed out, or outbound mail sits in the queue: [Diagnosing stalled mail](/topic/28219). Inbound with nothing at all: *I can't receive*. Outbound that never reaches the queue: *I can't send*. ### "I can't receive mail from outside" Work down the table in order, cheapest check first. | Check | How (Windows) | A failure means | |---|---|---| | Is the service running? | `Get-Service hMailServer` | Start it, then read the error log for why it stopped | | Is anything listening on 25? | `Get-NetTCPConnection -LocalPort 25 -State Listen` | The port is not configured, is bound to one address only, or another program has it: **Connections & protocols → TCP/IP ports** | | Is 25 open in Windows Firewall? | `Get-NetFirewallRule -Enabled True \| Where-Object DisplayName -like "*hMail*"` | Open it: see the firewall paragraph below | | Does the machine itself answer on 25? | `Test-NetConnection 127.0.0.1 -Port 25` | Then it is the server, not the network | | Is 25 forwarded by your router or cloud firewall? | Its configuration | The far end sees a refusal or a timeout | | Does your ISP block 25? | Ask them | Common on consumer lines: a relay host or a business line | | Is the MX record right? | The **MX query** page, or `Resolve-DnsName example.com -Type MX` | Fix DNS | | Can the world reach you? | From **outside**: `Test-NetConnection mail.example.com -Port 25` | It is the path in | | Is the domain in hMailServer and **Active**? | **Accounts & domains → Domains** | An inactive domain is not accepted for | | Is the mailbox **Active**, and full? | The account dialog | A full mailbox gets a temporary failure, and the sender retries | A port bound to one internal address answers only on the LAN. **Server status** lists the listeners the service actually brought up. **The firewall.** The installer opens no port, so nothing from outside reaches the server until you open them, on the machine and on the router or cloud firewall in front of it. Which ports, and the rules for Windows and Linux, are in [Every port hMailServer listens on or connects to](/topic/28226). ### "I can't send mail" The reply names most sending faults. In an ESMTP session an enhanced status code follows the number: `530 Must issue STARTTLS first.` arrives as `530 5.7.0 Must issue STARTTLS first.` | Reply | Likely cause | What to do | |---|---|---| | `550 Delivery is not allowed to this address.` ("relay denied") | The client is not authenticating, or uses 25 instead of 587 | Port 587 with authentication; a printer that cannot authenticate gets its own range under **Access & abuse protection → IP ranges** | | `530 SMTP authentication is required.` | The same, said earlier | As above | | `530 Must issue STARTTLS first.` | The port requires TLS first | STARTTLS or SSL/TLS in the client | | `535 Authentication failed.` | Wrong password, not the full address as user name, or an inactive account | Also check the auto-ban | | `535 Authentication failed. Too many invalid logon attempts.` | The auto-ban: **3** failures, **60** minutes, by default | Below | | `550 Too many recipients.` | Too many recipients in one transaction | Below | | `552 5.3.4 Message size exceeds fixed maximum message size. Size: N KB, Max size: M KB` | Over the server's maximum message size (20480 KB), or the domain's own | **Connections & protocols → Protocols**, SMTP tab, or the domain dialog | | `452 4.3.1 Insufficient system storage.` | Free space on the store's volume below `MinimumFreeDiskSpaceMB` (100 MB) | Free space; senders retry | **A banned client.** The ban is an expiring IP range: **Access & abuse protection → IP ranges** lists it, and deleting it releases the client (`DELETE /api/v1/ipranges/`, or the Deck's IP ranges view). **`550 Too many recipients.`** Two wiki pages disagree. Troubleshooting §19.3 ties it to `maxsmtprecipientsinbatch` (100) and says split the batch or raise the limit; the Architecture page's `RCPT TO` chart refuses at 50,000 recipients and uses `MaxSMTPRecipientsInBatch` to batch outbound deliveries. Splitting the message is right either way; raising the setting also changes the outbound batch size. ### What the server's replies mean A `4xx` is temporary and the sender retries; a `5xx` is permanent and the sender bounces. | Reply | Meaning | |---|---| | `421 4.7.0 Too many invalid commands. Bye!` | Disconnected for misbehaving: a scanner, or a broken client | | `451 4.3.1 Server temporarily overloaded while accepting the message; please retry.` | The accept pipeline exceeded `FinalizationTimeout`: [Diagnosing stalled mail](/topic/28219) | | `451 4.3.2 Unable to verify the recipient at the moment. Please retry later.` | The database did not answer, so the mail is deferred, not bounced. Check `/readyz` | | `451 4.3.2 The server is shutting down and cannot accept the message.` | A graceful stop with `ShutdownDrainSeconds` set | | `452 4.2.2 Mailbox is full.` | At or over quota, refused during the conversation; the holder had a `QUOTA_WARNING` | | `550 Unknown user` | No such mailbox, alias or list | | `553` or `554` from a *remote* server | The far end refused you; its text is the diagnosis, usually SPF, DKIM, PTR or a block list | ### "My mail goes to spam" Almost always DNS, in this order: the PTR record (nothing compensates for a missing or generic one), SPF, DKIM, DMARC even at `p=none`, then the block lists. With split-horizon DNS, the outside view is the one that counts. [Being trusted by the servers you send to](/topic/28188) covers it. ### The Control Panel, the Deck or the REST listener will not connect **The Control Panel** (Windows) talks to the running service over COM: no service, no connection. Use the **administration** password, not a mailbox's. After a restart it reconnects by itself; after you stop the service it deliberately does not. Remotely it needs port 135 and DCOM. Every page saying "Server unavailable" means the engine is paused: **Server status → Resume**. *Since 6.3.4*, under `NT SERVICE\hMailServer`, a change of the administrator password is refused because the service may not write the file: run `hMailServer.exe /SetAdministratorPassword` from an elevated prompt, then restart the service. More in [The Control Panel on Windows](/topic/28223). **The Deck and the REST API** share one listener, off until `RestApiPort` is set. It refuses to start without an administrator password, beyond loopback without a certificate, or with a certificate it cannot load (`HM5113` names the file), each logged once as `RestApi: Refusing to start`; the three lines and their fixes are in [The REST API: turning it on](/topic/28264). Only a `RestApi: Listening on` line proves it is up. *Since 6.3.4* the port waits for a reinitialise (**Apply now**, or `POST /api/v1/server/reinitialize`), and a line edited into `hMailServer.ini` after the first start loses to the stored value (HM5804). With the listener off, on Linux: ```sh sudo hmailserver --set-setting RestApiPort=8045 sudo systemctl restart hmailserver ``` On Windows, the Control Panel's **API & monitoring** page (**Monitoring & alerts** since 6.3.7). "Web administration page not installed" means the `WebAdmin` files are missing, or `ProgramFolder` does not point at the directory holding them. Repeated bad passwords get the address refused for five minutes; loopback is exempt. The Deck itself is [The Control Deck](/topic/28192). ### The service will not start The start, in order; where it stops names the problem: 1. **Windows is told the service is running**, before any of what follows. 2. **`hMailServer.INI` is read, alone**, which is how the database is found. A `[Database] Type` it does not recognise abandons the start, with nothing logged about a database. On Linux, `sudo -u hmailserver hmailserver --check-config` then prints `Database type: 0`. 3. **`HM5005`**: `Server` or `Database` is missing for the configured `Type`. 4. **The database is connected**, retried `ConnectionAttempts` times (6), `ConnectionAttemptsDelay` seconds apart (5). `HM4354`: the database server is down or moved, or the credentials changed. On Windows the `[Database]` password is DPAPI-protected per machine and cannot be copied from another server. *Since 6.3.4*, `HM7530` (an unknown `TlsMode`) and `HM7531` (`verify-full` with a missing `TlsCaFile`) stop it too; `--check-database` makes one connection and says what it got. 5. **The schema version**, compared both ways. `HM5010`: `Database version could not be detected.` `HM5011` too old: `DBUpdater.exe` from `Bin` on Windows, `sudo -u hmailserver hmailserver --upgrade-database` on Linux. `HM5011` too new: install the matching or a newer server; never downgrade the database. *Since 6.3.4*, `[Database] UpgradeOnStart=1` makes the service upgrade the schema as it starts: off by default, no backup first, one node at a time. 6. **Settings are read again** from the database, since 6.3.4 with any `[SettingsOverride]` keys over them. A configuration that will not load abandons the start. 7. **One listener per TCP/IP ports row.** A row whose protocol is off on **Protocols → Services** is skipped in silence, the commonest reason for nothing on 143. `HM4316`: something else holds the port (IIS on 25 or 443, another mail server, a leftover instance); that port alone stays down. `HM5113` with `Error initializing SSL. Certificate not set.`: that listener does not start. 8. **The optional listeners** (metrics, REST, web services, ManageSieve), each only with a non-zero port. `MetricsServer: Invalid bind address` means `MetricsServerBindAddress` is not an IP literal (`localhost` is refused too); that listener stays down and the server runs. A failure in steps 2 to 6 stops the whole server, and one with a code writes it to the ERROR log; steps 7 and 8 cost one listener and go to the application log. **Windows.** A service that stops at once with nothing logged is the account's rights on `Bin`, `Data`, `Logs` and `Database`. *Since 6.3.4* a new installation runs as `NT SERVICE\hMailServer` (an upgrade keeps its account), and a folder outside the program folder needs approving: run `hMailServer.exe --confirm /GrantServiceAccess` elevated in `Bin`, and approve only a folder you recognise, since the service may then write in it. Local SQL Server with integrated authentication needs a login for `NT SERVICE\hMailServer`. What the account may not do, and moving between it and LocalSystem, are in [Installing on Windows](/topic/28227). The installer's recovery settings restart a failed service after 1, 2 and 5 minutes (`sc.exe qfailure hMailServer`). **Linux, under systemd.** A start that fails before any listener opens puts `hMailServer could not start:` in the journal, with the reason on standard error in case the log directory is the fault. The unit runs as `hmailserver`, and `ProtectSystem=strict` makes anything outside its `ReadWritePaths` (`/var/lib/hmailserver` and `/var/log/hmailserver`) read-only to the service: a data, backup, archive or ACME directory elsewhere needs a drop-in (`sudo systemctl edit hmailserver`). `HM5094`: the MySQL or MariaDB client library is not installed. `HM5603` on every TLS listener: `/usr/bin/dh2048.pem` is missing; reinstall the package. ### The Diagnostics self-test **Monitoring & troubleshooting → Diagnostics** in the Control Panel; *since 6.3.4* also the Deck's Diagnostics view and `POST /api/v1/diagnostics/run`, on either platform. Ten tests: | Test | Fails when | |---|---| | Server details | Never; since 6.3.4 it also says whether the database connection is encrypted | | Installation paths *(6.3.0)* | A configured path does not exist | | Test IPv6 | Never | | Test outbound port | Outbound 25 is blocked, or the smart host's name, port or TLS mode is wrong | | Test backup directory | No destination, or the service cannot write there | | Test MX records | Public DNS is wrong, or the resolver is not answering | | Test local connect | Hairpin NAT fails, 25 is not forwarded, or the MX points elsewhere | | Test message file locations | Files outside the data directory, or full paths stored, which stops the built-in backup | | Test IP range configuration | `127.0.0.1` has been auto-banned | | Error logs | Any `ERROR*` file exists | Fill both domain boxes first: the MX and local-connect tests, and the outbound test without a smart host, are skipped rather than failed when they are empty. **Copy** puts the result on the clipboard for a post. ### "Did we ever get it?": the message trace The message trace records what the server did with each message, keyed on its queue id. **It is off by default** (`MessageTraceEnabled`), because it records who corresponds with whom, and nothing is recorded while it is off: turn it on before you need it. **Turning it on.** *Since 6.3.4* it applies when saved: the Control Panel's **Message trace** card on **Monitoring & troubleshooting → Logging**, `PUT /api/v1/settings/ini/{name}`, or `hmctl`, both with the administrator password: ```sh hmctl ini set MessageTraceEnabled 1 ``` On 6.3.3 and earlier it is `MessageTraceEnabled=1` under `[Settings]` and a restart. Rows are kept `MessageTraceRetentionDays` (30): since 6.3.4 an hourly task deletes older ones; on 6.3.3 and earlier only **Remove expired** on the Message trace page, or `MessageTrace.DeleteExpired` over COM, did. **Reading it.** The Control Panel's **Message trace** page searches an address as sender or recipient, newest first, and **Follow this message** shows one queue id. Over REST *since 6.3.4*: `GET /api/v1/message-trace`, `/api/v1/message-trace/address/{address}` and `/api/v1/message-trace/queue/{id}`, which need the `view_message_trace` permission. ```sh hmctl api GET /api/v1/message-trace/address/user@example.com ``` **What a row says.** One row per recipient per delivery attempt: when, the event, sender, recipient, source address and status code. Events include `delivered` and `failed`; on 6.3.4 also `deferred` (an attempt that will be retried) and `suppressed` (stopped by the suppression list). A refusal at `RCPT` carries queue id **0**. A forward, a list or a rule that re-sends makes a new queue entry. **Answering the question.** A `delivered` row: you had it; look at the account's folders and rules. A row with queue id 0: refused at `RCPT`, and the status says why. No row: either it never reached you, or it is still queued or held, came while the trace was off, or is past the retention. The trace records delivery attempts only, with no acceptance, spam or virus event, so an empty trace does not prove the server never had the message. The per-domain reports' traffic, failures, senders and recipients are counted from it, and say so while it is off: [Alerts, the audit trail and per-domain reports](/topic/28273). ### Things that look like faults and are not | What you see | Why | |---|---| | A new sender's first message is minutes late | Greylisting doing its job | | *(6.3.4)* A value typed into `[Settings]` is back after a restart, HM5804 in the ERROR log | The database is the settings store: set it in the Control Panel, over REST (`PUT /api/v1/settings/ini/{name}`) or with `hmctl ini set` | | *(6.3.4)* HM5807 at every start | A `[SettingsOverride]` section is forcing keys; remove it once the stored value is right | | *(6.3.4, Linux package)* HM5803 at every start | The service cannot write its copy into the root-owned file; the stored value is in effect | | 6.2.28 to 6.3.3: "The update feed answered HTTP 404." | Set `UpdateFeedUrl=https://updates.progressiverobot.com/releases.json` and restart | | "Release 6.3.4 carries no Sigstore bundle for its installer, so it cannot be verified." | Install 6.3.4 by hand: [Upgrading to 6.3.4](/topic/28193) | **Where a message waits, since 6.3.4.** Neither delivered nor queued may mean held on purpose: by the sandbox, by compromised-account detection, in a back-off window, under an IP warm-up ceiling or behind a drain; [Diagnosing stalled mail](/topic/28219) says where to look. And `550 5.7.1 Too many delivery attempts before the greylisting delay had passed` is greylisting's abuse lock, lifted after `GreyListingAbuseLockMinutes` (60); a `5.7.10` bounce to a partner domain is gateway encryption refusing to send in the clear, the partner's key having expired. ### What it does not do It does not diagnose a mail client ([Connecting a mail client](/topic/28255)), read logs line by line ([Reading the logs](/topic/28220)), or work a stalled queue. The Control Panel and COM are Windows-only. When nothing here names the fault, [Read this before you post](/topic/28185) says what a request needs. Without a GitLab account, email the Service Desk, contact-project+hmailserver-hmailserver-86730559-issue-@incoming.gitlab.com, which GitLab files as a confidential issue. ### Where to read more - Wiki: [Troubleshooting](https://gitlab.com/hmailserver/hmailserver/-/wikis/Troubleshooting), [Ports Reference](https://gitlab.com/hmailserver/hmailserver/-/wikis/Ports-Reference), [Monitoring and Health](https://gitlab.com/hmailserver/hmailserver/-/wikis/Monitoring-and-Health), [Installing hMailServer](https://gitlab.com/hmailserver/hmailserver/-/wikis/Installing-hMailServer), [Installing on Linux](https://gitlab.com/hmailserver/hmailserver/-/wikis/Installing-on-Linux), [The Control Panel](https://gitlab.com/hmailserver/hmailserver/-/wikis/The-Control-Panel), [The REST API](https://gitlab.com/hmailserver/hmailserver/-/wikis/The-REST-API), [Upgrading hMailServer](https://gitlab.com/hmailserver/hmailserver/-/wikis/Upgrading-hMailServer), [Architecture](https://gitlab.com/hmailserver/hmailserver/-/wikis/Architecture). - Documents: [ServiceAccount.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/ServiceAccount.md), [DatabaseTls.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/DatabaseTls.md), [Reports.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Reports.md), [SettingsApply.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SettingsApply.md). - On the forum: [Settings: when a save takes effect](/topic/28222) (HM5804 and the reinitialise), [IP ranges, auto-ban and lockout](/topic/28239), [Choosing a database and encrypting the link](/topic/28228) (`TlsMode`, HM7530 and HM7531) and [The hmailserver command line on Linux](/topic/28245) (the unit and its drop-ins). - Public documentation: [Troubleshooting](https://www.progressiverobot.com/hmailserver-documentation/#19-troubleshooting) and [Troubleshooting a Linux installation](https://www.progressiverobot.com/hmailserver-documentation/#3516-troubleshooting-a-linux-installation). --- Canonical: https://www.hmailserver.co.uk/topic/28226/every-port-hmailserver-listens-on-or-connects-to-and-the-firewall-rules-to-match # Every port hMailServer listens on or connects to, and the firewall rules to match *Written for 6.3.4. hMailServer 6.3.7 (17 October 2026) lets the IMAP and POP3 listeners take the client's address from a PROXY protocol header, as the SMTP listener does, each from the peers on a list of its own; both are off by default, and in the Control Panel the PROXY protocol and XCLIENT are a card on Incoming relays, while API & monitoring is Monitoring & alerts. The ACME http-01 challenge responder serves up to 64 connections at once and closes one past that, where it served one at a time. The REST listener refuses a state-changing request that carries a credential when the browser reports it as cross-site (`Sec-Fetch-Site`, or an `Origin` that is not this server's); scripts, `hmctl` and the PowerShell module send no such header and are unaffected. The container image is `registry.gitlab.com/hmailserver/hmailserver:6.3.7`, also tagged `6.3` and `latest`. See [the 6.3.7 announcement](/topic/28277).* This topic lists every socket hMailServer 6.3.4 can open, inbound and outbound, and the firewall rules that follow. The one thing to know first: **a new server listens on four mail ports and on nothing else.** Every other listener is off until a setting gives it a port, three of them bind only to `127.0.0.1` when you do, and almost every outbound connection beyond DNS, SMTP delivery and the recipient's MTA-STS policy exists because an administrator turned a feature on. Write the firewall for what you turned on. Connection security per mail port, and what a new 6.3.4 installation sets on each, is in [Connecting a mail client: IMAP, POP3, SMTP submission and the ports](/topic/28255). ### Two kinds of listener, set in two places **Mail-protocol listeners** (SMTP, POP3, IMAP) are rows of the table `hm_tcpipports`, edited on **Connections & protocols → TCP/IP ports** in the Control Panel (Windows), on the Control Deck, over COM (`Settings.TCPIPPorts`, Windows) and over REST at `/api/v1/ports`, where a new row's `address` defaults to `0.0.0.0`. A port you add, change or delete takes effect at a reinitialise (`POST /api/v1/server/reinitialize`, or the Deck's button that restarts the services in place) or a service restart. **Service listeners** are one key each, all off by default. Set their `[Settings]` keys in the Control Panel (Windows), over REST (`PUT /api/v1/settings/ini/{name}`), with `hmctl ini set` or `hmailserver --set-setting NAME=VALUE`; the Deck has no page for them. Since 6.3.4 the database holds these keys, so a line edited in `hMailServer.ini` turns no listener on (`HM5804`); [Settings: what is not stock, when a save takes effect](/topic/28222) explains the store. Each listener's port, bind address and certificate files are among the 28 keys that wait for a reinitialise: **Apply now** in the Control Panel or the Deck, `POST /api/v1/server/reinitialize`, `Application.Reinitialize` over COM, or a restart. `AcmeHttpPort` applies when it is saved. The SNMP agent's settings are `hm_settings` rows instead, set on the Deck's **Settings** page under **Server**, over REST (`/api/v1/settings`) or with `hmctl settings set`, with no Control Panel page, and read when the agent starts, at a service start or a Reinitialize. ### The mail listeners | Port | Protocol | Created by | Default bind | What it says on connect | |---|---|---|---|---| | 25 | SMTP | Seeded in every new database | The row's IP address (`0.0.0.0` or `::` is every address) | `220 MAIL01 ESMTP`: the *Welcome banner* from **Connections & protocols → Protocols → SMTP**, or the computer name, with ` ESMTP` appended | | 587 | SMTP submission | Seeded | The row's IP address | The same banner | | 110 | POP3 | Seeded | The row's IP address | `+OK POP3`, or your welcome message | | 143 | IMAP | Seeded | The row's IP address | `* OK IMAPrev1`, or your welcome message | | 465, 993, 995 | SMTP, IMAP, POP3 over implicit TLS | You, once you have a certificate | What you choose | Nothing until the TLS handshake completes | A listener bound to one internal address is the usual reason mail works from the LAN and not from outside. Two failures leave one port dead and the rest running: a TLS port with no certificate selected does not start (`HM5113`), and a port something else holds does not bind (`HM4316`). On Windows with WSL2 in mirrored networking mode, a socket bound inside WSL takes the port from Windows too, shows in neither `Get-NetTCPConnection` nor `netstat -ano`, and on the project's test bench only `wsl --shutdown` gave it back. ### The service listeners | Listener | Port key, default | Bind key, default | TLS | How to tell it is up | |---|---|---|---|---| | REST API, the Control Deck at `/`, the webmail at `/portal`, SCIM at `/scim/v2` (since 6.3.4) | `RestApiPort`, `0`; 8045 is the conventional port | `RestApiBindAddress`, `127.0.0.1` | Required unless bound to `127.0.0.1`, `localhost` or `::1` | `RestApi: Listening on (https)` or `(http, loopback only)` in the application log | | Public web services | `WebServicesHttpPort`, `WebServicesHttpsPort`, both `0`; 80 and 443 to enable | `WebServicesBindAddress`, `0.0.0.0` | `WebServicesCertificateFile` and `WebServicesPrivateKeyFile` on the HTTPS port, or the ACME certificate on disk | A URL it serves answers | | Prometheus metrics and health probes | `MetricsServerPort`, `0` | `MetricsServerBindAddress`, `127.0.0.1`, an IP literal only | `MetricsServerCertificateFile` + `MetricsServerPrivateKeyFile`; plain HTTP otherwise | `/livez` answers `200 alive` | | ManageSieve (RFC 5804) | `ManageSieveServerPort`, `0`; 4190 is the standard port | `ManageSieveServerBindAddress`, `127.0.0.1` | STARTTLS, only when an IMAP, POP3 or SMTP port has a certificate | Its capability response | | ACME http-01 challenge | `AcmeHttpPort`, `80` | Every address | Never | Open only during an issuance | | SNMP agent, UDP, since 6.3.4 | `SnmpPort`, `161`, with `SnmpEnabled` off | `SnmpBindAddress`, empty = every address | SNMPv3 at authPriv instead of TLS | A stranger gets no answer at all | **One port carries the REST API, the Deck and the webmail**; there is no second port to open. It will not start without an administrator password, nor beyond loopback without a certificate, and with `RestApiCertificateFile` empty it takes the ACME certificate when one exists. For users to reach `/portal`, or an identity provider SCIM, it needs a certificate they trust, a reachable bind address and a firewall rule, or a TLS-terminating proxy in front ([Behind a gateway, load balancer or reverse proxy](/topic/28240)). Turning it on is in [The REST API: turning it on, credentials, roles, one-time codes](/topic/28264). **The web services listener binds every address by default.** It serves the MTA-STS policy at `/.well-known/mta-sts.txt` (HTTPS only), ACME challenges, Thunderbird autoconfig, Outlook autodiscover and the Apple profile (HTTPS only), `security.txt`, CardDAV and CalDAV under `/dav/` (HTTPS only; CalDAV since 6.3.4), and, since 6.3.4, the OpenPGP Web Key Directory (HTTPS only) and an S/MIME authority's root and CRL (plain HTTP). `MtaStsHostingEnabled` and `AutoconfigEnabled` default to `1` while both ports stay `0`, which the server reports once at startup; the ports stay `0` on purpose, because port 80 may belong to IIS. The HTTPS port waits for a certificate, its own or ACME's, and says so in the log. **Metrics.** A host name or `localhost` as the bind address is refused (`MetricsServer: Invalid bind address`) and the probes go with the listener. Beyond loopback with no credential, `/metrics` answers 503 while the probes still answer. More, including the SNMP agent, is in [Monitoring and health: Prometheus metrics, health probes, Grafana, OTLP and SNMPv3](/topic/28272). **ManageSieve** has no certificate of its own and offers no STARTTLS, saying so in the log, until a mail port has one. **The ACME challenge listener** is not permanent. When `WebServicesHttpPort` equals `AcmeHttpPort`, the web services listener answers `/.well-known/acme-challenge/`; otherwise a temporary listener binds `AcmeHttpPort` for the issuance, and a failed bind logs `Failed to start the http-01 challenge listener. Is the port in use?` The client uses the `http-01` challenge only, so port 80 must be reachable from the internet ([Let's Encrypt certificates are automatic](/topic/28187)). **The SNMP agent** (since 6.3.4) answers nobody until `SnmpAllowedAddresses` names a manager; a datagram from any other address is dropped before it is read. Bind it to `127.0.0.1` or a management address, and name managers one by one: UDP does not prove a source address. Error `7420` is a port it could not bind, such as one the operating system's own SNMP service holds. [Snmp.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Snmp.md) lists the Linux bind to 161 under the packaged unit as not proved by any test. **A Control Panel on another machine** (Windows) reaches the COM API over DCOM, which must be permitted, after probing TCP port 135 (the RPC endpoint mapper). `hmctl` and the PowerShell module use the REST listener instead. ### Where Windows and Linux differ - **Where you set a listener.** On Windows, the TCP/IP ports page, **Connections & protocols → Web services & autoconfiguration**, **TLS & certificates → Certificates (ACME)** and **Monitoring & troubleshooting → API & monitoring** (**Monitoring & alerts** since 6.3.7). Linux has no Control Panel and no COM: the Deck for the mail listeners and the SNMP agent, and REST, `hmctl` or `hmailserver --set-setting` for every listener's `[Settings]` key. The packaged `RestApiPort=0` is stored at the first start, and a line edited afterwards turns nothing on; [First install on Linux](/topic/28190) has the order. - **Low ports.** The packaged `hmailserver.service` runs as its own user with `CAP_NET_BIND_SERVICE`, which it has for port 25 and which also lets the agent bind UDP 161. - **The database port.** On Windows `[Database] Port` of `0` means the provider's own default. On Linux write it out (`5432`, `3306`): it goes into the connection string as it stands. - **DNS.** On Windows the custom servers under **Connections & protocols → DNS resolver** are given port `0`, because the Windows DNS client supplies 53 itself and rejects an entry that names one; `DNSServer` is one IPv4 address with no port syntax. - **Containers.** The image exposes 25, 110, 143, 465, 587, 993, 995, 4190 and 8045, and SNMP is published as UDP. No image of 6.3.4 is published; 6.3.7's is `registry.gitlab.com/hmailserver/hmailserver:6.3.7`, also tagged `6.3` and `latest`, and [In a container or on Kubernetes](/topic/28246) has the rest. ### Proving what is listening On Windows, on the server and then from outside: ```powershell Get-NetTCPConnection -State Listen -LocalPort 587 | Select-Object LocalAddress, LocalPort, OwningProcess Test-NetConnection 127.0.0.1 -Port 25 Get-NetFirewallRule -Enabled True | Where-Object DisplayName -like "*hMail*" Test-NetConnection mail.example.com -Port 25 openssl s_client -connect mail.example.com:465 -quiet ``` On Linux, `ss -ltnp` lists listening sockets with their process, and the REST listener answers `curl -u Administrator http://127.0.0.1:8045/api/v1/status`. On Windows, **Monitoring & troubleshooting → Server status** lists the listeners the service brought up; senders timing out with nothing in the SMTP log means nothing reached a listener. ### What the server connects to **On every server, with nothing turned on:** | Destination | Port | Setting | If it is blocked | |---|---|---|---| | DNS resolvers | 53, UDP and TCP | The machine's resolvers, or `DNSServer` | Nothing is delivered externally, and DNSBL, SPF, DKIM, DMARC, MTA-STS and DANE stop at once | | Every recipient domain's MX host | 25 | None | Messages stay queued and are retried, by default 4 times at 60-minute intervals, then bounced | | The recipient domain's MTA-STS policy, from `mta-sts.` | 443 | `MtaStsEnabled`, `1` | Delivery to a domain with an enforcing policy falls back to what DANE and your TLS settings allow | DANE and TLS-RPT need no port of their own: TLSA records come over DNS, and TLS-RPT reports are sent as ordinary mail. Delivery leaves from `SMTPDeliveryBindToIP`, or a rule's *bind to address* action; since 6.3.4 `SMTPDeliveryBindToIP` may name one address of each family, separated by a comma. **Only when you configure it:** | Destination | Port | Setting, as shipped | |---|---|---| | Smart host, or a route's host | `SMTPRelayerPort` (25; `0` is read as 25), or the route's own | Empty | | Database server | The provider's port | `[Database]`; since 6.3.4 MySQL and MariaDB use TLS on it when offered | | SpamAssassin `spamd`, ClamAV `clamd` | `127.0.0.1:783`, `localhost:3310` | Both off; no TLS | | An HTTP filter engine | The URL's | `FilterHookUrl`, empty | | The ACME directory | 443 | `AcmeEnabled`, `0`; `AcmeDirectoryUrl` is Let's Encrypt | | The update feed, `https://updates.progressiverobot.com`, and the installer from the project's package registry on `gitlab.com` | 443 | `UpdateCheckEnabled`, `0` | | The password breach check, `https://api.pwnedpasswords.com/range/` | 443 | `BreachCheckEnabled`, `0` (since 6.3.4) | | An object store: `https://s3..amazonaws.com`, `https://.blob.core.windows.net` or `MessageStoreObjectEndpoint` | 443, or the endpoint's | `MessageStoreObjectService`, empty (since 6.3.4) | | OAuth2: a key set, an introspection endpoint, the outbound relay's token endpoint | The URL's | `OAuth2JwksUrl`, `OAuth2IntrospectionUrl`, `OutboundOAuth2TokenUrl`, empty | | An OpenID Connect issuer: `/.well-known/openid-configuration`, then its token endpoint | The issuer's | `oidc_issuer`, empty (since 6.3.4) | | Outbound webhooks, alert webhooks, a sandbox service, a route of mail in to a webhook | The URL's | No subscription, rule or route; `SandboxEnabled`, `0` (all since 6.3.4) | | A syslog collector | `SyslogPort`, 514 (UDP, TCP); 6514 is the registered TLS port | `SyslogEnabled`, off (since 6.3.4) | | An OpenTelemetry collector | The endpoint's | `OtelEndpoint`, `OtelMetricsEndpoint`, `OtelLogsEndpoint`, empty | | The browser vendor's push service | The service's | `WebPushEnabled`, `0` (since 6.3.4) | | External accounts and migrations: another server's POP3 or IMAP; `imap.gmail.com` or `outlook.office365.com` on 993; `graph.microsoft.com`; CardDAV and CalDAV over https | The account's; a person's own account 110, 995, 143 or 993 at public addresses unless `reader_fetch_accounts_private_addresses` is on | Nothing until an account or a migration is made | **At a person's request in the webmail**: since 6.3.4, a correspondent's OpenPGP key from their domain's Web Key Directory (`lookup_wkd`, on; never an address inside your network) or from `https://keys.openpgp.org` (`lookup_keyserver`, off), and a sender's BIMI logo (public addresses only, and only with `AuthenticationResultsEnabled`, which is `0`); and, since 6.3.2, a one-click unsubscribe, posted to a public address only. **Where an outbound request may go.** Every outbound web request is https, or plain http to this machine only; https is verified against the machine's root store, host name checked, TLS 1.2 at least, so a private CA must be in that store (or in `MessageStoreObjectCaFile` for an object store). Outbound webhooks may reach a private address, on purpose; a route of mail in to a webhook never reaches a link-local or cloud metadata address, and one not made by the server administrator reaches public addresses only. **Through a proxy.** `HttpProxy` (`host:port` or `[ipv6]:port`, since 6.2.28) carries the update feed and its downloads, the JWKS and introspection fetches, the breach check and the object store. It sends no proxy credentials, and a value with no port fails with `HttpProxy must be host:port (or [ipv6]:port).` rather than going direct. ACME issuance, the outbound relay's token request, a mail-in-to-webhook route and the MTA-STS policy fetch never use it; the MTA-STS fetch is on by default, so a proxy-only egress stops it. Whether the other requests above use the proxy is not documented: prove a proxy-only egress before relying on it. [UpdateFeed.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/UpdateFeed.md), written before the feed site's first deployment, says the address answers nothing usable until then. ### The firewall matrix | Direction | Port | From or to | Only when | |---|---|---|---| | In | TCP 25 | Anywhere | Always | | In | TCP 587, and 465, 993, 995 | Anywhere, or your users' networks | 587 always; the others once you have created them | | In | TCP 143, 110 | Your users | A client still uses them; deny them afterwards | | In | TCP 80 | Anywhere | ACME, or web services over HTTP | | In | TCP 443 | Anywhere | `WebServicesHttpsPort` is 443 | | In | TCP REST port (8045) | Your management network; users for `/portal`; the identity provider for SCIM | The listener is bound beyond loopback, with TLS | | In | TCP metrics port, ManageSieve 4190 | Prometheus; your management network | Bound beyond loopback (or keep `127.0.0.1` and a tunnel) | | In | UDP 161 | The SNMP managers only, on the bound address | The agent is on | | In | TCP 135 and DCOM | The administrator's machine | A remote Control Panel (Windows) | | Out | TCP 25 | Anywhere, or only the smart host on its port | Always | | Out | UDP and TCP 53 | Your resolvers | Always | | Out | TCP 443 | Anywhere, or the hosts in the tables above | MTA-STS is on (the default) or a feature needs it | | Out | Database, `spamd`, `clamd`, syslog, OTLP ports | Those hosts | They are not on this machine | | Out | TCP 110, 995, 143, 993, or the account's port | The servers mail is collected or migrated from | An external account or a migration exists | **Windows.** The installer opens no firewall port, so nothing reaches the server until you add the rules. In an elevated PowerShell: ```powershell New-NetFirewallRule -DisplayName "hMailServer SMTP" -Direction Inbound -Protocol TCP -LocalPort 25 -Action Allow New-NetFirewallRule -DisplayName "hMailServer Submission" -Direction Inbound -Protocol TCP -LocalPort 587 -Action Allow New-NetFirewallRule -DisplayName "hMailServer IMAPS" -Direction Inbound -Protocol TCP -LocalPort 993 -Action Allow New-NetFirewallRule -DisplayName "hMailServer POP3S" -Direction Inbound -Protocol TCP -LocalPort 995 -Action Allow ``` **Linux.** With firewalld, `firewall-cmd --permanent --add-port={25,587,993,995}/tcp` and `firewall-cmd --reload`; with ufw, `ufw allow 25,587,993,995/tcp`. The documentation page's chapter 4.4 also opens 110, 143 and 465; leave them out. On either system, add 143, 110, 465, 80 or 443 only when the matrix says so, and forward the same ports on the router or cloud firewall. The Linux package's own firewall hook (`AutoBanFirewall=1`) only blocks, and needs `CAP_NET_ADMIN`, which the unit does not grant by default; on Windows the same switch is refused under `NT SERVICE\hMailServer` (`HM7460`). Put the addresses you administer from in `AutoBanNeverBan` first; [Reading the logs, and fail2ban](/topic/28220) has auto-ban in the firewall. ### What it does not do - The SNMP agent sends no trap or inform and does not answer SNMPv1; nothing leaves it unasked. - Country blocking downloads no geographic database; the `.mmdb` file is yours to fetch and keep fresh. - No AI or machine-learning service is called by the server. ### Where to read more - Wiki: [Ports Reference](https://gitlab.com/hmailserver/hmailserver/-/wikis/Ports-Reference), [Before You Install](https://gitlab.com/hmailserver/hmailserver/-/wikis/Before-You-Install), [Installing hMailServer](https://gitlab.com/hmailserver/hmailserver/-/wikis/Installing-hMailServer), [Installing on Linux](https://gitlab.com/hmailserver/hmailserver/-/wikis/Installing-on-Linux), [Settings Reference](https://gitlab.com/hmailserver/hmailserver/-/wikis/Settings-Reference), [Troubleshooting](https://gitlab.com/hmailserver/hmailserver/-/wikis/Troubleshooting), [Encryption and Certificates](https://gitlab.com/hmailserver/hmailserver/-/wikis/Encryption-and-Certificates). - Documents at v6.3.4: [EvidencePack.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/EvidencePack.md) (what leaves the server), [SettingsApply.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SettingsApply.md), [Snmp.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Snmp.md), [UpdateFeed.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/UpdateFeed.md), [InboundWebhooks.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/InboundWebhooks.md), [ObjectStorage.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/ObjectStorage.md), [PasswordBreachCheck.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/PasswordBreachCheck.md), [Migration.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Migration.md). - The documentation page's [ports reference](https://www.progressiverobot.com/hmailserver-documentation/#22-ports-reference) and [Open the firewall](https://www.progressiverobot.com/hmailserver-documentation/#44-open-the-windows-firewall) chapters. - On the forum: [Let's Encrypt certificates are automatic](/topic/28187), [First install on Linux](/topic/28190), [The Control Deck](/topic/28192), [Installing on Windows: wizard pages, service account, profile and the first hour](/topic/28227), [Your first domain and mailbox, and the DNS records your domain needs](/topic/28229), [TLS on the listeners: your own certificate, versions, ciphers](/topic/28256) and [Troubleshooting by symptom: no mail in, no mail out](/topic/28225). --- Canonical: https://www.hmailserver.co.uk/topic/28226/every-port-hmailserver-listens-on-or-connects-to-and-the-firewall-rules-to-match # Every port hMailServer listens on or connects to, and the firewall rules to match *Written for 6.3.4. hMailServer 6.3.7 (17 October 2026) lets the IMAP and POP3 listeners take the client's address from a PROXY protocol header, as the SMTP listener does, each from the peers on a list of its own; both are off by default, and in the Control Panel the PROXY protocol and XCLIENT are a card on Incoming relays, while API & monitoring is Monitoring & alerts. The ACME http-01 challenge responder serves up to 64 connections at once and closes one past that, where it served one at a time. The REST listener refuses a state-changing request that carries a credential when the browser reports it as cross-site (`Sec-Fetch-Site`, or an `Origin` that is not this server's); scripts, `hmctl` and the PowerShell module send no such header and are unaffected. The container image is `registry.gitlab.com/hmailserver/hmailserver:6.3.7`, also tagged `6.3` and `latest`. See [the 6.3.7 announcement](/topic/28277).* This topic lists every socket hMailServer 6.3.4 can open, inbound and outbound, and the firewall rules that follow. The one thing to know first: **a new server listens on four mail ports and on nothing else.** Every other listener is off until a setting gives it a port, three of them bind only to `127.0.0.1` when you do, and almost every outbound connection beyond DNS, SMTP delivery and the recipient's MTA-STS policy exists because an administrator turned a feature on. Write the firewall for what you turned on. Connection security per mail port, and what a new 6.3.4 installation sets on each, is in [Connecting a mail client: IMAP, POP3, SMTP submission and the ports](/topic/28255). ### Two kinds of listener, set in two places **Mail-protocol listeners** (SMTP, POP3, IMAP) are rows of the table `hm_tcpipports`, edited on **Connections & protocols → TCP/IP ports** in the Control Panel (Windows), on the Control Deck, over COM (`Settings.TCPIPPorts`, Windows) and over REST at `/api/v1/ports`, where a new row's `address` defaults to `0.0.0.0`. A port you add, change or delete takes effect at a reinitialise (`POST /api/v1/server/reinitialize`, or the Deck's button that restarts the services in place) or a service restart. **Service listeners** are one key each, all off by default. Set their `[Settings]` keys in the Control Panel (Windows), over REST (`PUT /api/v1/settings/ini/{name}`), with `hmctl ini set` or `hmailserver --set-setting NAME=VALUE`; the Deck has no page for them. Since 6.3.4 the database holds these keys, so a line edited in `hMailServer.ini` turns no listener on (`HM5804`); [Settings: what is not stock, when a save takes effect](/topic/28222) explains the store. Each listener's port, bind address and certificate files are among the 28 keys that wait for a reinitialise: **Apply now** in the Control Panel or the Deck, `POST /api/v1/server/reinitialize`, `Application.Reinitialize` over COM, or a restart. `AcmeHttpPort` applies when it is saved. The SNMP agent's settings are `hm_settings` rows instead, set on the Deck's **Settings** page under **Server**, over REST (`/api/v1/settings`) or with `hmctl settings set`, with no Control Panel page, and read when the agent starts, at a service start or a Reinitialize. ### The mail listeners | Port | Protocol | Created by | Default bind | What it says on connect | |---|---|---|---|---| | 25 | SMTP | Seeded in every new database | The row's IP address (`0.0.0.0` or `::` is every address) | `220 MAIL01 ESMTP`: the *Welcome banner* from **Connections & protocols → Protocols → SMTP**, or the computer name, with ` ESMTP` appended | | 587 | SMTP submission | Seeded | The row's IP address | The same banner | | 110 | POP3 | Seeded | The row's IP address | `+OK POP3`, or your welcome message | | 143 | IMAP | Seeded | The row's IP address | `* OK IMAPrev1`, or your welcome message | | 465, 993, 995 | SMTP, IMAP, POP3 over implicit TLS | You, once you have a certificate | What you choose | Nothing until the TLS handshake completes | A listener bound to one internal address is the usual reason mail works from the LAN and not from outside. Two failures leave one port dead and the rest running: a TLS port with no certificate selected does not start (`HM5113`), and a port something else holds does not bind (`HM4316`). On Windows with WSL2 in mirrored networking mode, a socket bound inside WSL takes the port from Windows too, shows in neither `Get-NetTCPConnection` nor `netstat -ano`, and on the project's test bench only `wsl --shutdown` gave it back. ### The service listeners | Listener | Port key, default | Bind key, default | TLS | How to tell it is up | |---|---|---|---|---| | REST API, the Control Deck at `/`, the webmail at `/portal`, SCIM at `/scim/v2` (since 6.3.4) | `RestApiPort`, `0`; 8045 is the conventional port | `RestApiBindAddress`, `127.0.0.1` | Required unless bound to `127.0.0.1`, `localhost` or `::1` | `RestApi: Listening on (https)` or `(http, loopback only)` in the application log | | Public web services | `WebServicesHttpPort`, `WebServicesHttpsPort`, both `0`; 80 and 443 to enable | `WebServicesBindAddress`, `0.0.0.0` | `WebServicesCertificateFile` and `WebServicesPrivateKeyFile` on the HTTPS port, or the ACME certificate on disk | A URL it serves answers | | Prometheus metrics and health probes | `MetricsServerPort`, `0` | `MetricsServerBindAddress`, `127.0.0.1`, an IP literal only | `MetricsServerCertificateFile` + `MetricsServerPrivateKeyFile`; plain HTTP otherwise | `/livez` answers `200 alive` | | ManageSieve (RFC 5804) | `ManageSieveServerPort`, `0`; 4190 is the standard port | `ManageSieveServerBindAddress`, `127.0.0.1` | STARTTLS, only when an IMAP, POP3 or SMTP port has a certificate | Its capability response | | ACME http-01 challenge | `AcmeHttpPort`, `80` | Every address | Never | Open only during an issuance | | SNMP agent, UDP, since 6.3.4 | `SnmpPort`, `161`, with `SnmpEnabled` off | `SnmpBindAddress`, empty = every address | SNMPv3 at authPriv instead of TLS | A stranger gets no answer at all | **One port carries the REST API, the Deck and the webmail**; there is no second port to open. It will not start without an administrator password, nor beyond loopback without a certificate, and with `RestApiCertificateFile` empty it takes the ACME certificate when one exists. For users to reach `/portal`, or an identity provider SCIM, it needs a certificate they trust, a reachable bind address and a firewall rule, or a TLS-terminating proxy in front ([Behind a gateway, load balancer or reverse proxy](/topic/28240)). Turning it on is in [The REST API: turning it on, credentials, roles, one-time codes](/topic/28264). **The web services listener binds every address by default.** It serves the MTA-STS policy at `/.well-known/mta-sts.txt` (HTTPS only), ACME challenges, Thunderbird autoconfig, Outlook autodiscover and the Apple profile (HTTPS only), `security.txt`, CardDAV and CalDAV under `/dav/` (HTTPS only; CalDAV since 6.3.4), and, since 6.3.4, the OpenPGP Web Key Directory (HTTPS only) and an S/MIME authority's root and CRL (plain HTTP). `MtaStsHostingEnabled` and `AutoconfigEnabled` default to `1` while both ports stay `0`, which the server reports once at startup; the ports stay `0` on purpose, because port 80 may belong to IIS. The HTTPS port waits for a certificate, its own or ACME's, and says so in the log. **Metrics.** A host name or `localhost` as the bind address is refused (`MetricsServer: Invalid bind address`) and the probes go with the listener. Beyond loopback with no credential, `/metrics` answers 503 while the probes still answer. More, including the SNMP agent, is in [Monitoring and health: Prometheus metrics, health probes, Grafana, OTLP and SNMPv3](/topic/28272). **ManageSieve** has no certificate of its own and offers no STARTTLS, saying so in the log, until a mail port has one. **The ACME challenge listener** is not permanent. When `WebServicesHttpPort` equals `AcmeHttpPort`, the web services listener answers `/.well-known/acme-challenge/`; otherwise a temporary listener binds `AcmeHttpPort` for the issuance, and a failed bind logs `Failed to start the http-01 challenge listener. Is the port in use?` The client uses the `http-01` challenge only, so port 80 must be reachable from the internet ([Let's Encrypt certificates are automatic](/topic/28187)). **The SNMP agent** (since 6.3.4) answers nobody until `SnmpAllowedAddresses` names a manager; a datagram from any other address is dropped before it is read. Bind it to `127.0.0.1` or a management address, and name managers one by one: UDP does not prove a source address. Error `7420` is a port it could not bind, such as one the operating system's own SNMP service holds. [Snmp.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Snmp.md) lists the Linux bind to 161 under the packaged unit as not proved by any test. **A Control Panel on another machine** (Windows) reaches the COM API over DCOM, which must be permitted, after probing TCP port 135 (the RPC endpoint mapper). `hmctl` and the PowerShell module use the REST listener instead. ### Where Windows and Linux differ - **Where you set a listener.** On Windows, the TCP/IP ports page, **Connections & protocols → Web services & autoconfiguration**, **TLS & certificates → Certificates (ACME)** and **Monitoring & troubleshooting → API & monitoring** (**Monitoring & alerts** since 6.3.7). Linux has no Control Panel and no COM: the Deck for the mail listeners and the SNMP agent, and REST, `hmctl` or `hmailserver --set-setting` for every listener's `[Settings]` key. The packaged `RestApiPort=0` is stored at the first start, and a line edited afterwards turns nothing on; [First install on Linux](/topic/28190) has the order. - **Low ports.** The packaged `hmailserver.service` runs as its own user with `CAP_NET_BIND_SERVICE`, which it has for port 25 and which also lets the agent bind UDP 161. - **The database port.** On Windows `[Database] Port` of `0` means the provider's own default. On Linux write it out (`5432`, `3306`): it goes into the connection string as it stands. - **DNS.** On Windows the custom servers under **Connections & protocols → DNS resolver** are given port `0`, because the Windows DNS client supplies 53 itself and rejects an entry that names one; `DNSServer` is one IPv4 address with no port syntax. - **Containers.** The image exposes 25, 110, 143, 465, 587, 993, 995, 4190 and 8045, and SNMP is published as UDP. No image of 6.3.4 is published; 6.3.7's is `registry.gitlab.com/hmailserver/hmailserver:6.3.7`, also tagged `6.3` and `latest`, and [In a container or on Kubernetes](/topic/28246) has the rest. ### Proving what is listening On Windows, on the server and then from outside: ```powershell Get-NetTCPConnection -State Listen -LocalPort 587 | Select-Object LocalAddress, LocalPort, OwningProcess Test-NetConnection 127.0.0.1 -Port 25 Get-NetFirewallRule -Enabled True | Where-Object DisplayName -like "*hMail*" Test-NetConnection mail.example.com -Port 25 openssl s_client -connect mail.example.com:465 -quiet ``` On Linux, `ss -ltnp` lists listening sockets with their process, and the REST listener answers `curl -u Administrator http://127.0.0.1:8045/api/v1/status`. On Windows, **Monitoring & troubleshooting → Server status** lists the listeners the service brought up; senders timing out with nothing in the SMTP log means nothing reached a listener. ### What the server connects to **On every server, with nothing turned on:** | Destination | Port | Setting | If it is blocked | |---|---|---|---| | DNS resolvers | 53, UDP and TCP | The machine's resolvers, or `DNSServer` | Nothing is delivered externally, and DNSBL, SPF, DKIM, DMARC, MTA-STS and DANE stop at once | | Every recipient domain's MX host | 25 | None | Messages stay queued and are retried, by default 4 times at 60-minute intervals, then bounced | | The recipient domain's MTA-STS policy, from `mta-sts.` | 443 | `MtaStsEnabled`, `1` | Delivery to a domain with an enforcing policy falls back to what DANE and your TLS settings allow | DANE and TLS-RPT need no port of their own: TLSA records come over DNS, and TLS-RPT reports are sent as ordinary mail. Delivery leaves from `SMTPDeliveryBindToIP`, or a rule's *bind to address* action; since 6.3.4 `SMTPDeliveryBindToIP` may name one address of each family, separated by a comma. **Only when you configure it:** | Destination | Port | Setting, as shipped | |---|---|---| | Smart host, or a route's host | `SMTPRelayerPort` (25; `0` is read as 25), or the route's own | Empty | | Database server | The provider's port | `[Database]`; since 6.3.4 MySQL and MariaDB use TLS on it when offered | | SpamAssassin `spamd`, ClamAV `clamd` | `127.0.0.1:783`, `localhost:3310` | Both off; no TLS | | An HTTP filter engine | The URL's | `FilterHookUrl`, empty | | The ACME directory | 443 | `AcmeEnabled`, `0`; `AcmeDirectoryUrl` is Let's Encrypt | | The update feed, `https://updates.progressiverobot.com`, and the installer from the project's package registry on `gitlab.com` | 443 | `UpdateCheckEnabled`, `0` | | The password breach check, `https://api.pwnedpasswords.com/range/` | 443 | `BreachCheckEnabled`, `0` (since 6.3.4) | | An object store: `https://s3..amazonaws.com`, `https://.blob.core.windows.net` or `MessageStoreObjectEndpoint` | 443, or the endpoint's | `MessageStoreObjectService`, empty (since 6.3.4) | | OAuth2: a key set, an introspection endpoint, the outbound relay's token endpoint | The URL's | `OAuth2JwksUrl`, `OAuth2IntrospectionUrl`, `OutboundOAuth2TokenUrl`, empty | | An OpenID Connect issuer: `/.well-known/openid-configuration`, then its token endpoint | The issuer's | `oidc_issuer`, empty (since 6.3.4) | | Outbound webhooks, alert webhooks, a sandbox service, a route of mail in to a webhook | The URL's | No subscription, rule or route; `SandboxEnabled`, `0` (all since 6.3.4) | | A syslog collector | `SyslogPort`, 514 (UDP, TCP); 6514 is the registered TLS port | `SyslogEnabled`, off (since 6.3.4) | | An OpenTelemetry collector | The endpoint's | `OtelEndpoint`, `OtelMetricsEndpoint`, `OtelLogsEndpoint`, empty | | The browser vendor's push service | The service's | `WebPushEnabled`, `0` (since 6.3.4) | | External accounts and migrations: another server's POP3 or IMAP; `imap.gmail.com` or `outlook.office365.com` on 993; `graph.microsoft.com`; CardDAV and CalDAV over https | The account's; a person's own account 110, 995, 143 or 993 at public addresses unless `reader_fetch_accounts_private_addresses` is on | Nothing until an account or a migration is made | **At a person's request in the webmail**: since 6.3.4, a correspondent's OpenPGP key from their domain's Web Key Directory (`lookup_wkd`, on; never an address inside your network) or from `https://keys.openpgp.org` (`lookup_keyserver`, off), and a sender's BIMI logo (public addresses only, and only with `AuthenticationResultsEnabled`, which is `0`); and, since 6.3.2, a one-click unsubscribe, posted to a public address only. **Where an outbound request may go.** Every outbound web request is https, or plain http to this machine only; https is verified against the machine's root store, host name checked, TLS 1.2 at least, so a private CA must be in that store (or in `MessageStoreObjectCaFile` for an object store). Outbound webhooks may reach a private address, on purpose; a route of mail in to a webhook never reaches a link-local or cloud metadata address, and one not made by the server administrator reaches public addresses only. **Through a proxy.** `HttpProxy` (`host:port` or `[ipv6]:port`, since 6.2.28) carries the update feed and its downloads, the JWKS and introspection fetches, the breach check and the object store. It sends no proxy credentials, and a value with no port fails with `HttpProxy must be host:port (or [ipv6]:port).` rather than going direct. ACME issuance, the outbound relay's token request, a mail-in-to-webhook route and the MTA-STS policy fetch never use it; the MTA-STS fetch is on by default, so a proxy-only egress stops it. Whether the other requests above use the proxy is not documented: prove a proxy-only egress before relying on it. [UpdateFeed.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/UpdateFeed.md), written before the feed site's first deployment, says the address answers nothing usable until then. ### The firewall matrix | Direction | Port | From or to | Only when | |---|---|---|---| | In | TCP 25 | Anywhere | Always | | In | TCP 587, and 465, 993, 995 | Anywhere, or your users' networks | 587 always; the others once you have created them | | In | TCP 143, 110 | Your users | A client still uses them; deny them afterwards | | In | TCP 80 | Anywhere | ACME, or web services over HTTP | | In | TCP 443 | Anywhere | `WebServicesHttpsPort` is 443 | | In | TCP REST port (8045) | Your management network; users for `/portal`; the identity provider for SCIM | The listener is bound beyond loopback, with TLS | | In | TCP metrics port, ManageSieve 4190 | Prometheus; your management network | Bound beyond loopback (or keep `127.0.0.1` and a tunnel) | | In | UDP 161 | The SNMP managers only, on the bound address | The agent is on | | In | TCP 135 and DCOM | The administrator's machine | A remote Control Panel (Windows) | | Out | TCP 25 | Anywhere, or only the smart host on its port | Always | | Out | UDP and TCP 53 | Your resolvers | Always | | Out | TCP 443 | Anywhere, or the hosts in the tables above | MTA-STS is on (the default) or a feature needs it | | Out | Database, `spamd`, `clamd`, syslog, OTLP ports | Those hosts | They are not on this machine | | Out | TCP 110, 995, 143, 993, or the account's port | The servers mail is collected or migrated from | An external account or a migration exists | **Windows.** The installer opens no firewall port, so nothing reaches the server until you add the rules. In an elevated PowerShell: ```powershell New-NetFirewallRule -DisplayName "hMailServer SMTP" -Direction Inbound -Protocol TCP -LocalPort 25 -Action Allow New-NetFirewallRule -DisplayName "hMailServer Submission" -Direction Inbound -Protocol TCP -LocalPort 587 -Action Allow New-NetFirewallRule -DisplayName "hMailServer IMAPS" -Direction Inbound -Protocol TCP -LocalPort 993 -Action Allow New-NetFirewallRule -DisplayName "hMailServer POP3S" -Direction Inbound -Protocol TCP -LocalPort 995 -Action Allow ``` **Linux.** With firewalld, `firewall-cmd --permanent --add-port={25,587,993,995}/tcp` and `firewall-cmd --reload`; with ufw, `ufw allow 25,587,993,995/tcp`. The documentation page's chapter 4.4 also opens 110, 143 and 465; leave them out. On either system, add 143, 110, 465, 80 or 443 only when the matrix says so, and forward the same ports on the router or cloud firewall. The Linux package's own firewall hook (`AutoBanFirewall=1`) only blocks, and needs `CAP_NET_ADMIN`, which the unit does not grant by default; on Windows the same switch is refused under `NT SERVICE\hMailServer` (`HM7460`). Put the addresses you administer from in `AutoBanNeverBan` first; [Reading the logs, and fail2ban](/topic/28220) has auto-ban in the firewall. ### What it does not do - The SNMP agent sends no trap or inform and does not answer SNMPv1; nothing leaves it unasked. - Country blocking downloads no geographic database; the `.mmdb` file is yours to fetch and keep fresh. - No AI or machine-learning service is called by the server. ### Where to read more - Wiki: [Ports Reference](https://gitlab.com/hmailserver/hmailserver/-/wikis/Ports-Reference), [Before You Install](https://gitlab.com/hmailserver/hmailserver/-/wikis/Before-You-Install), [Installing hMailServer](https://gitlab.com/hmailserver/hmailserver/-/wikis/Installing-hMailServer), [Installing on Linux](https://gitlab.com/hmailserver/hmailserver/-/wikis/Installing-on-Linux), [Settings Reference](https://gitlab.com/hmailserver/hmailserver/-/wikis/Settings-Reference), [Troubleshooting](https://gitlab.com/hmailserver/hmailserver/-/wikis/Troubleshooting), [Encryption and Certificates](https://gitlab.com/hmailserver/hmailserver/-/wikis/Encryption-and-Certificates). - Documents at v6.3.4: [EvidencePack.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/EvidencePack.md) (what leaves the server), [SettingsApply.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SettingsApply.md), [Snmp.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Snmp.md), [UpdateFeed.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/UpdateFeed.md), [InboundWebhooks.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/InboundWebhooks.md), [ObjectStorage.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/ObjectStorage.md), [PasswordBreachCheck.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/PasswordBreachCheck.md), [Migration.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Migration.md). - The documentation page's [ports reference](https://www.progressiverobot.com/hmailserver-documentation/#22-ports-reference) and [Open the firewall](https://www.progressiverobot.com/hmailserver-documentation/#44-open-the-windows-firewall) chapters. - On the forum: [Let's Encrypt certificates are automatic](/topic/28187), [First install on Linux](/topic/28190), [The Control Deck](/topic/28192), [Installing on Windows: wizard pages, service account, profile and the first hour](/topic/28227), [Your first domain and mailbox, and the DNS records your domain needs](/topic/28229), [TLS on the listeners: your own certificate, versions, ciphers](/topic/28256) and [Troubleshooting by symptom: no mail in, no mail out](/topic/28225). --- Canonical: https://www.hmailserver.co.uk/topic/28227/installing-on-windows-wizard-pages-service-account-profile-and-the-first-hour # Installing on Windows: wizard pages, service account, profile and the first hour *Written for 6.3.4. hMailServer 6.3.7 (17 October 2026) is the release to install today, a security release: a new server runs `hMailServer-6.3.7-x64.exe`, Authenticode-signed and with its Sigstore bundle beside it, and a server on 6.3.4, 6.3.5 or 6.3.6 installs it by the live update, with the exceptions the announcement lists. Its installers carry the .NET Desktop Runtime 10.0.13 and install it only where no .NET 10 Desktop Runtime is installed at all; at every installation and upgrade `/ProtectFolders` hardens the program folder, `Bin` and `ControlPanel`, because in a folder that took its permissions from the root of a drive any account that could sign in to the computer could replace the programs the service runs (CVE-2026-107571; the default under Program Files was never affected); setup waits `ShutdownDrainSeconds` plus 90 seconds for the service's stop, where it waited plus 30; and the x64 installer's MySQL and MariaDB client is MariaDB Connector/C 3.4.11. See [the 6.3.7 announcement](/topic/28277).* This topic walks through a new installation of hMailServer 6.3.4 on Windows: what the installer checks before it shows a page, each page of the wizard, the two pages 6.3.4 adds (the setting profile and the service account), what runs after the files are copied, and the first hour after it. The one thing to know first: **6.3.4 is installed by hand on every server.** It carries an Authenticode signature and no Sigstore bundle, so no server's live update installs it; you download the installer, check it and run it yourself. If you already run hMailServer, read [Upgrading to 6.3.4](/topic/28193) instead. ### Before you download The installer enforces a floor rather than advising one: `MinVersion=10.0.14393` and `ArchitecturesAllowed=x64`. On anything older, or on 32-bit Windows, setup stops with *"hMailServer requires Windows 10 version 1607 (build 14393) / Windows Server 2016 or later, 64-bit."* It needs an administrator, because it creates a service and writes to `HKLM`. The .NET 10 Desktop Runtime, which the database tools need, is installed automatically when missing; 6.3.7's installers carry 10.0.13, Microsoft's security release of 13 October 2026, and install it only where no .NET 10 Desktop Runtime is installed at all, so an upgraded server keeps the one it had. RAM is guidance only: 2 GB minimum, and nothing enforces it. Decide four things first, because the wizard asks for them: - **The administrator password.** At least five characters. It is the master password for the whole server, not an email password. - **The database.** The wizard asks only "built-in or external". If in doubt take the built-in one; [Choosing a database](/topic/28228) covers the rest. - **The kind of server it is** (since 6.3.4): the setting profile, below. - **The account the service runs as** (since 6.3.4): `NT SERVICE\hMailServer`, unless something it would need, such as a data or backup folder outside the program folder or SQL Server with Windows authentication on the same machine, is planned for first. Plan antivirus exclusions for `Data\`, `Database\` and `Temp\`, and somewhere to keep the exported stored-secrets key, apart from the database backups. ### Getting the installer and checking it Get `hMailServer-6.3.4-x64.exe` from the [latest release](https://gitlab.com/hmailserver/hmailserver/-/releases/permalink/latest) on GitLab, or from `https://updates.progressiverobot.com/download/hMailServer-x64.exe`. Both now give 6.3.7, the newest stable release: `hMailServer-6.3.7-x64.exe`, SHA-256 `dcae96bd75d6cae8e5a910d15940567067539ed99db5aeed40a815added4d85f`, with its `.cosign.bundle` beside it; 6.3.4's own files are on [its release page](https://gitlab.com/hmailserver/hmailserver/-/releases/v6.3.4). **6.3.4 carries no `.cosign.bundle`.** It was built and Authenticode-signed by hand, after GitHub suspended the project's account on 18 September 2026 and before the project's GitLab CI signing job existed. Its SHA-256 sums are in a file signed with the release tags' SSH key: [the 6.3.4 announcement](/topic/28218) has the two commands that check a download against it. Then check the installer's own signature: ```powershell Get-AuthenticodeSignature .\hMailServer-6.3.4-x64.exe | Format-List Status, SignerCertificate ``` `Status` must be `Valid`, and the signer's subject names *Progressive Robot Ltd*. SmartScreen still warns: expect *Windows protected your PC* with the publisher's real name on it. `winget install hMailServer` and `choco install hmailserver` do not install this fork ([Unattended installs, winget and Chocolatey](/topic/28231)). ### What the installer checks before it shows a page Choose **Run as administrator**. `InitializeSetup()` runs first: | Check | Result | |---|---| | `/adminpassword`, or `HM_ADMIN_PASSWORD` (since 6.3.4), shorter than 5 characters | Setup stops | | A window titled **hMailServer Control Panel**, **hMailServer Administrator**, **hMailServer Database Setup**, **hMailServer Database Upgrader** or **DBSetup** is open | Setup stops: *"…You must close down this application before starting the installation."* | | Ports 25, 110 or 143 in use while the hMailServer service is not running | Warning only: *"…This indicates that there already is an email server running on this computer."* | Ignore the port warning and two servers will fight over port 25. ### The wizard, page by page 1. **Licence agreement.** hMailServer is AGPL-3.0. 2. **Destination folder.** `C:\Program Files\hMailServer` is fine. Since 6.3.7 setup's `/ProtectFolders` hardens the program folder, `Bin` and `ControlPanel` at every installation and upgrade: in a folder that took its permissions from the root of a drive, any account that could sign in to the computer could replace the programs the service runs (CVE-2026-107571, every version from 6.0.0 to 6.3.6); the default folder under Program Files was never affected. 3. **Components.** `server` is the mail server, the database tools and the DB scripts. `controlpanel` is the administration application; without it you have no GUI. `admintools`, *Remote administration support*, registers the COM type library on a machine that is *not* the server, for your own scripts; the Control Panel does not need it. 4. **Database type.** *Use the built-in database engine* unless you decided otherwise. If you choose external, `DBSetup.exe` runs after the file copy and asks for the engine, host, database name and credentials. Skipped when `[Database] Type` is already set: an upgrade never re-asks. 5. **Start menu folder.** 6. **Administrator password.** Minimum five characters; the two boxes must match. Since 6.3.4 a space, a double quote or a backslash reaches the database step as typed: the installer has the server store the password as PBKDF2 and hands it to the database tools in the environment variable `HM_ADMIN_PASSWORD`, never on a command line. Set one. While the value is empty, `COMAuthentication::AttempAnonymousAuthentication` grants server-administrator rights to any COM caller and `RestApiServer::Start` refuses to start the REST API. 7. **Server profile** (since 6.3.4), on a new installation only. 8. **Service account** (since 6.3.4). 9. **Ready to install.** Pressing **Install** stops an existing hMailServer service and waits `ShutdownDrainSeconds + 30` seconds for it (since 6.3.7, `ShutdownDrainSeconds` plus 90). If it will not stop, setup refuses to go on: *"The hMailServer service did not stop within 30 seconds."* Stop it in `services.msc` and run setup again; do not work around this. 10. **Install.** ### The setting profile page (since 6.3.4) A setting profile is the thirty or so settings that make a server a particular kind, named. The wizard offers four, with **None** (the shipped settings) chosen until you pick another: | Profile | For | |---|---| | `office` | One server that receives an office's mail, holds its mailboxes and sends what its people write: SPF, DKIM, DMARC and greylisting on, spam held in the quarantine, auto-ban and account lockout, a password policy and the breach check | | `gateway` | A server behind a filtering gateway that has already judged the mail: greylisting off, and XCLIENT and an ARC seal trusted only from the addresses and the sealer you name; putting the gateway's addresses in Incoming relays is yours to do ([Behind a gateway](/topic/28240)) | | `relay` | An outbound relay that holds no mailbox anybody reads: IMAP and POP3 closed, hard bounces suppressed, deliveries throttled per destination, every message traced | | `sender` | The relay's shape for a high-volume sender judged on its bounce and complaint rates | After the database step the installer runs `hMailServer.exe --apply-profile NAME --once --if-new --confirm`; `--if-new` leaves alone a database that already holds ten accounts or more. Every failure is said, naming the profile as not applied, and the installation succeeds with the shipped settings. In the wizard no difference is shown: each profile's sentence names the listener, the filter or the trust it changes, and choosing it is the confirmation, so `relay` and `sender` close IMAP and POP3 with no further question. Chosen later, from the Control Panel's **Setting profiles** page or the Control Deck's checklist, a profile shows its difference from this server before it changes anything. Either way it saves what it replaces and can be put back. A profile never sets a port, a directory, a `[Database]` value, a password or an IP range's *Require SSL/TLS when authenticating*. One caution from [Profiles.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Profiles.md): the installer page is among what it lists as "written and read, and not run", because "the installer was not built for this change". ### The service account page (since 6.3.4) Until 6.3.4 the Windows service ran as LocalSystem, so a flaw reachable through SMTP, IMAP or POP3 was a flaw with full control of the computer. A new installation now runs it as `NT SERVICE\hMailServer`, the service's own virtual account: no password, no account to create, and it reaches only the folders it is granted. | The machine | The page offers | The default | |---|---|---| | No hMailServer service yet | `NT SERVICE\hMailServer`, or LocalSystem | `NT SERVICE\hMailServer` | | A service running as LocalSystem | keep LocalSystem, or move it | keep LocalSystem | | A service running as any other account | keep that account, or move it back to LocalSystem | keep the account | An upgrade never changes the account unless this page is answered to. **What the account is granted**, by `hMailServer.exe /GrantServiceAccess`: Modify on the program folder's own `Data`, `Logs`, `Temp` and `Database`; Read on its other folders; Modify on `Bin\hMailServerApiKeys.ini`; nothing on `hMailServer.INI`; and a folder the configuration names elsewhere only once an administrator approves it. An attended installation asks (`--confirm`). A silent one approves nothing: when the configuration names a folder or key file outside those folders, it leaves the service on LocalSystem, working, and says why. The folders and the certificate and key files the configuration names are checked against those grants and reported: `NOT APPROVED`, a folder nobody has approved; `OUTSIDE`, a certificate or key file, which is never given Read where it is (move it into `Data\Certificates`, or approve its folder for Read); `REFUSED`, a folder that may never be granted. A folder is approved with `--approve-modify ""` or `--approve-read ""`, or by answering Yes to `--confirm`; approvals are kept in `HKEY_LOCAL_MACHINE\SOFTWARE\hMailServer\ServiceAccessPaths`, a key only Administrators and SYSTEM may change. Approve only a folder you recognise: the service may then write in it. Never granted, approved or not: anything not a full path on a fixed local drive; a drive's root; the program folder, a folder above it, or anything in it but its `Data`, `Logs`, `Temp` and `Database`; the Windows folder, a Program Files folder, the user profiles folder or `ProgramData\Microsoft`, or any folder in or above one of them; `ProgramData` or a folder above it (a folder of your own inside it may be approved); and any path through a junction, a symbolic link or a mount point. `--report-only` says what would be done and changes nothing. Exit codes: `0` every grant made and everything named reached, or confirmed, `1` a grant or an approval failed, `2` not in an installation's `Bin`, `3` not elevated, `4` something outside not confirmed, `5` a folder that is never granted is named, `6` answered No. **What only LocalSystem could do** is refused with a sentence: the live update's apply (the update is still checked; on 6.3.4 and 6.3.5 an administrator downloads the installer from the release page, checks its signature and runs it, and from 6.3.6 installs the verified update with **Install update** on the Control Panel's *Server status* page on the server, with one Windows prompt), the auto-ban's Windows Firewall rules (`AutoBanFirewall`, HM7460; banned addresses are still refused by the server) and `Utilities.MakeDependent`. SQL Server with integrated authentication on the same machine needs a login first: `CREATE LOGIN [NT SERVICE\hMailServer] FROM WINDOWS`, then a user with `db_owner`. MySQL, MariaDB and PostgreSQL use a user and password, and nothing changes. **`hMailServer.INI` is read-only to the service.** A change of the administrator password over COM or the Control Panel's *Administrator password* card is refused; make it from an elevated prompt in `Bin` with `hMailServer.exe --set-admin-password`, which takes the password from `HM_ADMIN_PASSWORD` or, failing that, from standard input, and restart the service ([ServiceAccount.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/ServiceAccount.md) writes it `/SetAdministratorPassword`). Enrolling the administrator's TOTP second factor needs an account that may write the file, so move the service to LocalSystem for the enrolment and back. To move a service that runs as LocalSystem, run the installer again and choose the move, use the Control Panel's *Windows service account* card under *Server limits & expert settings*, or, from an elevated prompt in `Bin`: ```bat hMailServer.exe --confirm /GrantServiceAccess hMailServer.exe --service-account virtual /Register net stop hMailServer && net start hMailServer ``` running the second line only when the first ended in `0`. To go back, run the installer again and choose *Move it back to LocalSystem*, use the same card, or run `hMailServer.exe --service-account localsystem /Register` and restart the service. [ServiceAccount.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/ServiceAccount.md) records that the live registration "is exercised by hand on a throwaway machine; nothing here registers a service on the bench". ### What runs after the files are copied | # | Step | What runs | If it fails | |---|---|---|---| | 1 | .NET 10 Desktop Runtime | Only when missing; since 6.3.4 pinned to 10.0.12 by version and SHA-512; 6.3.7 carries 10.0.13 and installs it only where no .NET 10 Desktop Runtime is installed at all | Error dialog; exit codes `0`, `3010` and `1638` count as success | | 2 | Database backend | `SSCERuntime_x64-ENU.msi`, built-in engine only | Step 4 is skipped | | 3 | Create the service | `hMailServer.exe --service-account /Register`; a service to be moved is registered as LocalSystem here | Step 5 is skipped | | 3a | Administrator password (since 6.3.4) | `hMailServer.exe --set-admin-password` | The install **fails** | | 4 | Create the database | `DBSetupQuick.exe` for the built-in engine, `DBSetup.exe` for an external one chosen for the first time | The install **fails**, naming the exit code | | 4a | Secure a new installation (since 6.3.4) | `hMailServer.exe --apply-new-install-defaults` | Said and logged; the installation succeeds | | 4b | Apply the profile (since 6.3.4) | `--apply-profile`, above | Said; the shipped settings stay | | 4c | Stored-secrets key (since 6.3.4) | `hMailServer.exe --create-stored-secrets-key` | Said, with the command to run by hand | | 4d | Move the service (since 6.3.4) | `/GrantServiceAccess`, and only when that ends in 0, `--service-account virtual /Register` | The service stays on LocalSystem, working | | 5 | Start the service | `net START hMailServer`, the verdict taken from the SCM | *"The hMailServer service was installed but did not start."* | The database step runs while the service is still LocalSystem, because it writes `[Database]` into `hMailServer.INI` through the service, which the virtual account may not do. `ServiceMain` reports `SERVICE_RUNNING` before it connects to the database, so a running service proves the process started, not that the schema is usable; step 4's exit code covers the schema. ### What a new installation starts with (since 6.3.4) Step 4a runs `hMailServer.exe --apply-new-install-defaults` after the database step, before the server first listens, and only on a database **this installation created**: on one `DBSetup.exe` attached the verb exits 5, which the installer only logs, and the installation succeeds. On a created database, ports 25, 587, 110 and 143 offer STARTTLS with a self-signed certificate that mail clients warn about, and the *Internet* IP range requires TLS for authentication, so a client on another machine must start TLS before it signs in; an upgraded server is left as it was. What each default is, how to replace the certificate ([Let's Encrypt certificates are automatic](/topic/28187)) and how to apply the defaults by hand are in [Hardening a public mail server](/topic/28241). ### The stored-secrets key (since 6.3.4) On a new installation step 4c makes the key that seals each account's TOTP second factor and the `[Settings]` secrets in the database. It is kept in this machine's `hMailServer.INI` alone, never in the database or a backup, and an upgrade makes none. Export it and keep the file with the database backups, stored apart from them: ```powershell # elevated, in the Bin folder .\hMailServer.exe --export-stored-secrets-key D:\keys\hmailserver-stored-secrets.key ``` A database restored without it opens none of those secrets. [Hardening a public mail server](/topic/28241) covers the key in full: what it seals, other nodes, and a key that is lost. ### Confirm it is running, and open the firewall `Get-Service hMailServer` should say `Running`. On a new installation that kept the page's default, `services.msc` shows it logging on as `NT SERVICE\hMailServer`. `sc.exe qfailure hMailServer` shows the recovery the installer set: restarts after 60, 120 and 300 seconds, the counter reset after 86400. **The installer does not open firewall ports.** Until you do, nothing reaches the server. One rule per port, in an elevated PowerShell; for SMTP: ```powershell New-NetFirewallRule -DisplayName "hMailServer SMTP" -Direction Inbound -Protocol TCP -LocalPort 25 -Action Allow ``` Which ports to open, and which to leave closed, are in [Every port hMailServer listens on](/topic/28226). ### The first hour: the checklist and the sample domain Since 6.3.4 the server judges its own first hour. Twelve items, each `done`, `action_needed`, `cannot_tell` or `not_needed`: a profile, a domain, an account, TLS on the submission port, passwords only over TLS, an SPF record, DKIM signing, a DMARC record, a backup destination, a second factor for the administrator, the stored-secrets key and the account the server runs as. It is the Control Deck's *Checklist* view, the head of the Control Panel's *External setup* page, `GET /api/v1/setup/checklist` and COM's `Utilities.GetSetupChecklist`. A button on the Control Panel's Welcome page and on the Deck's checklist makes a throwaway sample domain, `example.test`, with three mailboxes, a list, a rule and eight delivered messages; a second removes it. The server mints one password for the three mailboxes and shows it once, in the answer that makes the sample; [Your first domain and mailbox](/topic/28229) has the detail. Remove the sample before the server takes real mail. ### Silent installation A silent install answers the two 6.3.4 pages with `/profile=` and `/serviceaccount=`, and leaves the administrator password unset unless `/adminpassword=` or `HM_ADMIN_PASSWORD` gives it. **Always check the exit code**: a failed database create fails the install. The switches, the profiles and the password are in [Unattended installs](/topic/28231). ### What the installer wrote `{app}\Bin\hMailServer.INI` is the only configuration file. Since 6.3.4 the database is the settings store: the file keeps `[Directories]`, `[Database]`, `[Security]` and `[GUILanguages]`, its `[Settings]` section is the server's copy, and a hand edit to a key the database holds is named in the error log (HM5804) and does not take effect. A service that may write the file puts the line back at the next start; under `NT SERVICE\hMailServer` the copy is not refreshed and the edited line stays as you left it, so a line that survives a restart is still not the value in use. `[Security] AdministratorPassword` is PBKDF2, `$h1$$$`. `{app}\Data`, `{app}\Events`, `{app}\Addons` and the ini are left behind by an uninstall, on purpose. ### What it does not do, and where to read more The installer does not open the firewall, does not create ports 465, 993 and 995, publishes no DNS record, and never applies the secure defaults, a profile or a stored-secrets key to an upgraded database. All of this is Windows only: the Linux packages have no wizard, no Control Panel and no COM ([First install on Linux](/topic/28190)). Sources: the wiki's [Installing hMailServer](https://gitlab.com/hmailserver/hmailserver/-/wikis/Installing-hMailServer) and [Before You Install](https://gitlab.com/hmailserver/hmailserver/-/wikis/Before-You-Install); [SecureDefaults.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SecureDefaults.md), [ServiceAccount.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/ServiceAccount.md), [Profiles.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Profiles.md), [SetupChecklist.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SetupChecklist.md), [SampleDomain.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SampleDomain.md) and [Upgrading.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Upgrading.md); and chapters [3](https://www.progressiverobot.com/hmailserver-documentation/#3-before-you-install-the-checklist) and [4](https://www.progressiverobot.com/hmailserver-documentation/#4-installing-hmailserver) of the public documentation. The release announcement is [hMailServer 6.3.4: a password could run code on Windows servers with JScript scripting](/topic/28218). --- Canonical: https://www.hmailserver.co.uk/topic/28227/installing-on-windows-wizard-pages-service-account-profile-and-the-first-hour # Installing on Windows: wizard pages, service account, profile and the first hour *Written for 6.3.4. hMailServer 6.3.7 (17 October 2026) is the release to install today, a security release: a new server runs `hMailServer-6.3.7-x64.exe`, Authenticode-signed and with its Sigstore bundle beside it, and a server on 6.3.4, 6.3.5 or 6.3.6 installs it by the live update, with the exceptions the announcement lists. Its installers carry the .NET Desktop Runtime 10.0.13 and install it only where no .NET 10 Desktop Runtime is installed at all; at every installation and upgrade `/ProtectFolders` hardens the program folder, `Bin` and `ControlPanel`, because in a folder that took its permissions from the root of a drive any account that could sign in to the computer could replace the programs the service runs (CVE-2026-107571; the default under Program Files was never affected); setup waits `ShutdownDrainSeconds` plus 90 seconds for the service's stop, where it waited plus 30; and the x64 installer's MySQL and MariaDB client is MariaDB Connector/C 3.4.11. See [the 6.3.7 announcement](/topic/28277).* This topic walks through a new installation of hMailServer 6.3.4 on Windows: what the installer checks before it shows a page, each page of the wizard, the two pages 6.3.4 adds (the setting profile and the service account), what runs after the files are copied, and the first hour after it. The one thing to know first: **6.3.4 is installed by hand on every server.** It carries an Authenticode signature and no Sigstore bundle, so no server's live update installs it; you download the installer, check it and run it yourself. If you already run hMailServer, read [Upgrading to 6.3.4](/topic/28193) instead. ### Before you download The installer enforces a floor rather than advising one: `MinVersion=10.0.14393` and `ArchitecturesAllowed=x64`. On anything older, or on 32-bit Windows, setup stops with *"hMailServer requires Windows 10 version 1607 (build 14393) / Windows Server 2016 or later, 64-bit."* It needs an administrator, because it creates a service and writes to `HKLM`. The .NET 10 Desktop Runtime, which the database tools need, is installed automatically when missing; 6.3.7's installers carry 10.0.13, Microsoft's security release of 13 October 2026, and install it only where no .NET 10 Desktop Runtime is installed at all, so an upgraded server keeps the one it had. RAM is guidance only: 2 GB minimum, and nothing enforces it. Decide four things first, because the wizard asks for them: - **The administrator password.** At least five characters. It is the master password for the whole server, not an email password. - **The database.** The wizard asks only "built-in or external". If in doubt take the built-in one; [Choosing a database](/topic/28228) covers the rest. - **The kind of server it is** (since 6.3.4): the setting profile, below. - **The account the service runs as** (since 6.3.4): `NT SERVICE\hMailServer`, unless something it would need, such as a data or backup folder outside the program folder or SQL Server with Windows authentication on the same machine, is planned for first. Plan antivirus exclusions for `Data\`, `Database\` and `Temp\`, and somewhere to keep the exported stored-secrets key, apart from the database backups. ### Getting the installer and checking it Get `hMailServer-6.3.4-x64.exe` from the [latest release](https://gitlab.com/hmailserver/hmailserver/-/releases/permalink/latest) on GitLab, or from `https://updates.progressiverobot.com/download/hMailServer-x64.exe`. Both now give 6.3.7, the newest stable release: `hMailServer-6.3.7-x64.exe`, SHA-256 `dcae96bd75d6cae8e5a910d15940567067539ed99db5aeed40a815added4d85f`, with its `.cosign.bundle` beside it; 6.3.4's own files are on [its release page](https://gitlab.com/hmailserver/hmailserver/-/releases/v6.3.4). **6.3.4 carries no `.cosign.bundle`.** It was built and Authenticode-signed by hand, after GitHub suspended the project's account on 18 September 2026 and before the project's GitLab CI signing job existed. Its SHA-256 sums are in a file signed with the release tags' SSH key: [the 6.3.4 announcement](/topic/28218) has the two commands that check a download against it. Then check the installer's own signature: ```powershell Get-AuthenticodeSignature .\hMailServer-6.3.4-x64.exe | Format-List Status, SignerCertificate ``` `Status` must be `Valid`, and the signer's subject names *Progressive Robot Ltd*. SmartScreen still warns: expect *Windows protected your PC* with the publisher's real name on it. `winget install hMailServer` and `choco install hmailserver` do not install this fork ([Unattended installs, winget and Chocolatey](/topic/28231)). ### What the installer checks before it shows a page Choose **Run as administrator**. `InitializeSetup()` runs first: | Check | Result | |---|---| | `/adminpassword`, or `HM_ADMIN_PASSWORD` (since 6.3.4), shorter than 5 characters | Setup stops | | A window titled **hMailServer Control Panel**, **hMailServer Administrator**, **hMailServer Database Setup**, **hMailServer Database Upgrader** or **DBSetup** is open | Setup stops: *"…You must close down this application before starting the installation."* | | Ports 25, 110 or 143 in use while the hMailServer service is not running | Warning only: *"…This indicates that there already is an email server running on this computer."* | Ignore the port warning and two servers will fight over port 25. ### The wizard, page by page 1. **Licence agreement.** hMailServer is AGPL-3.0. 2. **Destination folder.** `C:\Program Files\hMailServer` is fine. Since 6.3.7 setup's `/ProtectFolders` hardens the program folder, `Bin` and `ControlPanel` at every installation and upgrade: in a folder that took its permissions from the root of a drive, any account that could sign in to the computer could replace the programs the service runs (CVE-2026-107571, every version from 6.0.0 to 6.3.6); the default folder under Program Files was never affected. 3. **Components.** `server` is the mail server, the database tools and the DB scripts. `controlpanel` is the administration application; without it you have no GUI. `admintools`, *Remote administration support*, registers the COM type library on a machine that is *not* the server, for your own scripts; the Control Panel does not need it. 4. **Database type.** *Use the built-in database engine* unless you decided otherwise. If you choose external, `DBSetup.exe` runs after the file copy and asks for the engine, host, database name and credentials. Skipped when `[Database] Type` is already set: an upgrade never re-asks. 5. **Start menu folder.** 6. **Administrator password.** Minimum five characters; the two boxes must match. Since 6.3.4 a space, a double quote or a backslash reaches the database step as typed: the installer has the server store the password as PBKDF2 and hands it to the database tools in the environment variable `HM_ADMIN_PASSWORD`, never on a command line. Set one. While the value is empty, `COMAuthentication::AttempAnonymousAuthentication` grants server-administrator rights to any COM caller and `RestApiServer::Start` refuses to start the REST API. 7. **Server profile** (since 6.3.4), on a new installation only. 8. **Service account** (since 6.3.4). 9. **Ready to install.** Pressing **Install** stops an existing hMailServer service and waits `ShutdownDrainSeconds + 30` seconds for it (since 6.3.7, `ShutdownDrainSeconds` plus 90). If it will not stop, setup refuses to go on: *"The hMailServer service did not stop within 30 seconds."* Stop it in `services.msc` and run setup again; do not work around this. 10. **Install.** ### The setting profile page (since 6.3.4) A setting profile is the thirty or so settings that make a server a particular kind, named. The wizard offers four, with **None** (the shipped settings) chosen until you pick another: | Profile | For | |---|---| | `office` | One server that receives an office's mail, holds its mailboxes and sends what its people write: SPF, DKIM, DMARC and greylisting on, spam held in the quarantine, auto-ban and account lockout, a password policy and the breach check | | `gateway` | A server behind a filtering gateway that has already judged the mail: greylisting off, and XCLIENT and an ARC seal trusted only from the addresses and the sealer you name; putting the gateway's addresses in Incoming relays is yours to do ([Behind a gateway](/topic/28240)) | | `relay` | An outbound relay that holds no mailbox anybody reads: IMAP and POP3 closed, hard bounces suppressed, deliveries throttled per destination, every message traced | | `sender` | The relay's shape for a high-volume sender judged on its bounce and complaint rates | After the database step the installer runs `hMailServer.exe --apply-profile NAME --once --if-new --confirm`; `--if-new` leaves alone a database that already holds ten accounts or more. Every failure is said, naming the profile as not applied, and the installation succeeds with the shipped settings. In the wizard no difference is shown: each profile's sentence names the listener, the filter or the trust it changes, and choosing it is the confirmation, so `relay` and `sender` close IMAP and POP3 with no further question. Chosen later, from the Control Panel's **Setting profiles** page or the Control Deck's checklist, a profile shows its difference from this server before it changes anything. Either way it saves what it replaces and can be put back. A profile never sets a port, a directory, a `[Database]` value, a password or an IP range's *Require SSL/TLS when authenticating*. One caution from [Profiles.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Profiles.md): the installer page is among what it lists as "written and read, and not run", because "the installer was not built for this change". ### The service account page (since 6.3.4) Until 6.3.4 the Windows service ran as LocalSystem, so a flaw reachable through SMTP, IMAP or POP3 was a flaw with full control of the computer. A new installation now runs it as `NT SERVICE\hMailServer`, the service's own virtual account: no password, no account to create, and it reaches only the folders it is granted. | The machine | The page offers | The default | |---|---|---| | No hMailServer service yet | `NT SERVICE\hMailServer`, or LocalSystem | `NT SERVICE\hMailServer` | | A service running as LocalSystem | keep LocalSystem, or move it | keep LocalSystem | | A service running as any other account | keep that account, or move it back to LocalSystem | keep the account | An upgrade never changes the account unless this page is answered to. **What the account is granted**, by `hMailServer.exe /GrantServiceAccess`: Modify on the program folder's own `Data`, `Logs`, `Temp` and `Database`; Read on its other folders; Modify on `Bin\hMailServerApiKeys.ini`; nothing on `hMailServer.INI`; and a folder the configuration names elsewhere only once an administrator approves it. An attended installation asks (`--confirm`). A silent one approves nothing: when the configuration names a folder or key file outside those folders, it leaves the service on LocalSystem, working, and says why. The folders and the certificate and key files the configuration names are checked against those grants and reported: `NOT APPROVED`, a folder nobody has approved; `OUTSIDE`, a certificate or key file, which is never given Read where it is (move it into `Data\Certificates`, or approve its folder for Read); `REFUSED`, a folder that may never be granted. A folder is approved with `--approve-modify ""` or `--approve-read ""`, or by answering Yes to `--confirm`; approvals are kept in `HKEY_LOCAL_MACHINE\SOFTWARE\hMailServer\ServiceAccessPaths`, a key only Administrators and SYSTEM may change. Approve only a folder you recognise: the service may then write in it. Never granted, approved or not: anything not a full path on a fixed local drive; a drive's root; the program folder, a folder above it, or anything in it but its `Data`, `Logs`, `Temp` and `Database`; the Windows folder, a Program Files folder, the user profiles folder or `ProgramData\Microsoft`, or any folder in or above one of them; `ProgramData` or a folder above it (a folder of your own inside it may be approved); and any path through a junction, a symbolic link or a mount point. `--report-only` says what would be done and changes nothing. Exit codes: `0` every grant made and everything named reached, or confirmed, `1` a grant or an approval failed, `2` not in an installation's `Bin`, `3` not elevated, `4` something outside not confirmed, `5` a folder that is never granted is named, `6` answered No. **What only LocalSystem could do** is refused with a sentence: the live update's apply (the update is still checked; on 6.3.4 and 6.3.5 an administrator downloads the installer from the release page, checks its signature and runs it, and from 6.3.6 installs the verified update with **Install update** on the Control Panel's *Server status* page on the server, with one Windows prompt), the auto-ban's Windows Firewall rules (`AutoBanFirewall`, HM7460; banned addresses are still refused by the server) and `Utilities.MakeDependent`. SQL Server with integrated authentication on the same machine needs a login first: `CREATE LOGIN [NT SERVICE\hMailServer] FROM WINDOWS`, then a user with `db_owner`. MySQL, MariaDB and PostgreSQL use a user and password, and nothing changes. **`hMailServer.INI` is read-only to the service.** A change of the administrator password over COM or the Control Panel's *Administrator password* card is refused; make it from an elevated prompt in `Bin` with `hMailServer.exe --set-admin-password`, which takes the password from `HM_ADMIN_PASSWORD` or, failing that, from standard input, and restart the service ([ServiceAccount.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/ServiceAccount.md) writes it `/SetAdministratorPassword`). Enrolling the administrator's TOTP second factor needs an account that may write the file, so move the service to LocalSystem for the enrolment and back. To move a service that runs as LocalSystem, run the installer again and choose the move, use the Control Panel's *Windows service account* card under *Server limits & expert settings*, or, from an elevated prompt in `Bin`: ```bat hMailServer.exe --confirm /GrantServiceAccess hMailServer.exe --service-account virtual /Register net stop hMailServer && net start hMailServer ``` running the second line only when the first ended in `0`. To go back, run the installer again and choose *Move it back to LocalSystem*, use the same card, or run `hMailServer.exe --service-account localsystem /Register` and restart the service. [ServiceAccount.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/ServiceAccount.md) records that the live registration "is exercised by hand on a throwaway machine; nothing here registers a service on the bench". ### What runs after the files are copied | # | Step | What runs | If it fails | |---|---|---|---| | 1 | .NET 10 Desktop Runtime | Only when missing; since 6.3.4 pinned to 10.0.12 by version and SHA-512; 6.3.7 carries 10.0.13 and installs it only where no .NET 10 Desktop Runtime is installed at all | Error dialog; exit codes `0`, `3010` and `1638` count as success | | 2 | Database backend | `SSCERuntime_x64-ENU.msi`, built-in engine only | Step 4 is skipped | | 3 | Create the service | `hMailServer.exe --service-account /Register`; a service to be moved is registered as LocalSystem here | Step 5 is skipped | | 3a | Administrator password (since 6.3.4) | `hMailServer.exe --set-admin-password` | The install **fails** | | 4 | Create the database | `DBSetupQuick.exe` for the built-in engine, `DBSetup.exe` for an external one chosen for the first time | The install **fails**, naming the exit code | | 4a | Secure a new installation (since 6.3.4) | `hMailServer.exe --apply-new-install-defaults` | Said and logged; the installation succeeds | | 4b | Apply the profile (since 6.3.4) | `--apply-profile`, above | Said; the shipped settings stay | | 4c | Stored-secrets key (since 6.3.4) | `hMailServer.exe --create-stored-secrets-key` | Said, with the command to run by hand | | 4d | Move the service (since 6.3.4) | `/GrantServiceAccess`, and only when that ends in 0, `--service-account virtual /Register` | The service stays on LocalSystem, working | | 5 | Start the service | `net START hMailServer`, the verdict taken from the SCM | *"The hMailServer service was installed but did not start."* | The database step runs while the service is still LocalSystem, because it writes `[Database]` into `hMailServer.INI` through the service, which the virtual account may not do. `ServiceMain` reports `SERVICE_RUNNING` before it connects to the database, so a running service proves the process started, not that the schema is usable; step 4's exit code covers the schema. ### What a new installation starts with (since 6.3.4) Step 4a runs `hMailServer.exe --apply-new-install-defaults` after the database step, before the server first listens, and only on a database **this installation created**: on one `DBSetup.exe` attached the verb exits 5, which the installer only logs, and the installation succeeds. On a created database, ports 25, 587, 110 and 143 offer STARTTLS with a self-signed certificate that mail clients warn about, and the *Internet* IP range requires TLS for authentication, so a client on another machine must start TLS before it signs in; an upgraded server is left as it was. What each default is, how to replace the certificate ([Let's Encrypt certificates are automatic](/topic/28187)) and how to apply the defaults by hand are in [Hardening a public mail server](/topic/28241). ### The stored-secrets key (since 6.3.4) On a new installation step 4c makes the key that seals each account's TOTP second factor and the `[Settings]` secrets in the database. It is kept in this machine's `hMailServer.INI` alone, never in the database or a backup, and an upgrade makes none. Export it and keep the file with the database backups, stored apart from them: ```powershell # elevated, in the Bin folder .\hMailServer.exe --export-stored-secrets-key D:\keys\hmailserver-stored-secrets.key ``` A database restored without it opens none of those secrets. [Hardening a public mail server](/topic/28241) covers the key in full: what it seals, other nodes, and a key that is lost. ### Confirm it is running, and open the firewall `Get-Service hMailServer` should say `Running`. On a new installation that kept the page's default, `services.msc` shows it logging on as `NT SERVICE\hMailServer`. `sc.exe qfailure hMailServer` shows the recovery the installer set: restarts after 60, 120 and 300 seconds, the counter reset after 86400. **The installer does not open firewall ports.** Until you do, nothing reaches the server. One rule per port, in an elevated PowerShell; for SMTP: ```powershell New-NetFirewallRule -DisplayName "hMailServer SMTP" -Direction Inbound -Protocol TCP -LocalPort 25 -Action Allow ``` Which ports to open, and which to leave closed, are in [Every port hMailServer listens on](/topic/28226). ### The first hour: the checklist and the sample domain Since 6.3.4 the server judges its own first hour. Twelve items, each `done`, `action_needed`, `cannot_tell` or `not_needed`: a profile, a domain, an account, TLS on the submission port, passwords only over TLS, an SPF record, DKIM signing, a DMARC record, a backup destination, a second factor for the administrator, the stored-secrets key and the account the server runs as. It is the Control Deck's *Checklist* view, the head of the Control Panel's *External setup* page, `GET /api/v1/setup/checklist` and COM's `Utilities.GetSetupChecklist`. A button on the Control Panel's Welcome page and on the Deck's checklist makes a throwaway sample domain, `example.test`, with three mailboxes, a list, a rule and eight delivered messages; a second removes it. The server mints one password for the three mailboxes and shows it once, in the answer that makes the sample; [Your first domain and mailbox](/topic/28229) has the detail. Remove the sample before the server takes real mail. ### Silent installation A silent install answers the two 6.3.4 pages with `/profile=` and `/serviceaccount=`, and leaves the administrator password unset unless `/adminpassword=` or `HM_ADMIN_PASSWORD` gives it. **Always check the exit code**: a failed database create fails the install. The switches, the profiles and the password are in [Unattended installs](/topic/28231). ### What the installer wrote `{app}\Bin\hMailServer.INI` is the only configuration file. Since 6.3.4 the database is the settings store: the file keeps `[Directories]`, `[Database]`, `[Security]` and `[GUILanguages]`, its `[Settings]` section is the server's copy, and a hand edit to a key the database holds is named in the error log (HM5804) and does not take effect. A service that may write the file puts the line back at the next start; under `NT SERVICE\hMailServer` the copy is not refreshed and the edited line stays as you left it, so a line that survives a restart is still not the value in use. `[Security] AdministratorPassword` is PBKDF2, `$h1$$$`. `{app}\Data`, `{app}\Events`, `{app}\Addons` and the ini are left behind by an uninstall, on purpose. ### What it does not do, and where to read more The installer does not open the firewall, does not create ports 465, 993 and 995, publishes no DNS record, and never applies the secure defaults, a profile or a stored-secrets key to an upgraded database. All of this is Windows only: the Linux packages have no wizard, no Control Panel and no COM ([First install on Linux](/topic/28190)). Sources: the wiki's [Installing hMailServer](https://gitlab.com/hmailserver/hmailserver/-/wikis/Installing-hMailServer) and [Before You Install](https://gitlab.com/hmailserver/hmailserver/-/wikis/Before-You-Install); [SecureDefaults.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SecureDefaults.md), [ServiceAccount.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/ServiceAccount.md), [Profiles.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Profiles.md), [SetupChecklist.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SetupChecklist.md), [SampleDomain.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SampleDomain.md) and [Upgrading.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Upgrading.md); and chapters [3](https://www.progressiverobot.com/hmailserver-documentation/#3-before-you-install-the-checklist) and [4](https://www.progressiverobot.com/hmailserver-documentation/#4-installing-hmailserver) of the public documentation. The release announcement is [hMailServer 6.3.4: a password could run code on Windows servers with JScript scripting](/topic/28218). --- Canonical: https://www.hmailserver.co.uk/topic/28228/choosing-a-database-creating-it-encrypting-the-link-and-leaving-the-built-in-one # Choosing a database, creating it, encrypting the link and leaving the built-in one *Written for 6.3.4. hMailServer 6.3.7 (17 October 2026) adds no schema step (6108, as 6.3.6). Its x64 installer's MySQL and MariaDB client is MariaDB Connector/C 3.4.11, built from MariaDB's source, in place of the 3.4.9 that 6.2.3 to 6.3.6 carried (CVE-2026-107819), with seven authentication plugins where 6.3.6's had nine: a database account that signs in with MySQL Enterprise's Windows authentication can no longer sign in, and the server then cannot open its database; the ARM64 installer carries no MySQL or MariaDB client. On MySQL, MariaDB, SQL Server and SQL Server Compact an insert that forgives a duplicate key now forgives that alone and any other failure is reported (HM5032), and on MySQL and MariaDB a lost connection is reconnected and the statement run again, so an error log that was quiet may now name failures that were hidden. See [the 6.3.7 announcement](/topic/28277).* This topic covers the four databases hMailServer 6.3.4 can keep its data in, what each needs before the schema can be made, how the connection to the database server is encrypted since 6.3.4, and how to move an installation off the built-in engine. The one thing to know first: **on MySQL and MariaDB, 6.3.4 needs MySQL 5.7.9 or MariaDB 10.2.2 or later**, because since schema 6102 every table is `utf8mb4`. On an older server the create script stops at its first table, and an upgrade from 6.3.3 stops at its first step, 6040 to 6041, with error 1071 and nothing changed. ### What the database holds, and what it does not hMailServer stores accounts, domains, message *metadata* and settings in the database. The message bodies are files on disk, one per message, at `\\\\{guid}.eml`, so the database stays small relative to `Data\`. Since 6.3.4 the database is also the settings store: the `[Settings]` keys of `hMailServer.ini` are rows from schema 6042, the file's section is the server's copy, and a line edited there for a key the database holds is named in the error log as `HM5804` and does not take effect. Where the service may write the file the line is put back; where it may not (a new Windows installation under `NT SERVICE\hMailServer`, a Linux package installation) the copy is not refreshed, the edited line stays, and the database's value is used. The `[Database]` section itself stays in the file, because it says how the store is reached. ### The four choices | Database | `[Database] Type` | What is bundled | What you must have first | |---|---|---|---| | Built-in (SQL Server Compact 4.0 SP1) | `MSSQLCE` | `SSCERuntime_x64-ENU.msi`, installed by setup | Nothing. This is the default | | MySQL / MariaDB | `MYSQL` | MariaDB Connector/C 3.4.9 as `Bin\libmysql.dll` (3.4.11 since 6.3.7, with seven authentication plugins where 6.3.6's had nine) | A reachable server, MySQL 5.7.9 or MariaDB 10.2.2 or later, an empty database, and a user that may create tables | | Microsoft SQL Server | `MSSQL` | Nothing; the OS's own data stack | The same, including Express edition | | PostgreSQL | `PostgreSQL` | `libpq` from PostgreSQL 18.3 | The same | **On Linux** there is no built-in database: a Linux server keeps its data in PostgreSQL or MySQL/MariaDB, and `hmailserver --create-database` refuses SQL Server and SQL Server Compact by name. For MySQL or MariaDB install the client library yourself, because the server opens `libmariadb.so.3` at run time: `libmariadb3` on Debian and Ubuntu, `mariadb-connector-c` on Fedora and RHEL. [Databases on Linux](/topic/28247) has the steps for each backend, and [First install on Linux](/topic/28190) the order of the whole first install. **Two things about the built-in engine.** It runs on a single database connection: `NumberOfConnections` is forced to 1 for `MSSQLCE`. And Microsoft has withdrawn the SQL Server Compact download; the copy in `installation/SQLCE/` is the archive of record, and it still installs. [MigratingDatabaseBackend.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/MigratingDatabaseBackend.md) calls it "the one real dependency liability in this tree". For anything up to a few dozen mailboxes it is adequate, and the last section of this topic is the way off it. ### How the four behave differently | | MS SQL Server | PostgreSQL | MySQL / MariaDB | SQL Server Compact | |---|---|---|---|---| | DDL in a transaction | Mostly | Yes, the only one of the four | No, commits as it goes | No, commits as it goes | | A failed schema step leaves | Normally nothing applied | Nothing applied | A partly built database | A partly built database | | Isolation the server uses | READ UNCOMMITTED | its default | its default | its default | | Transactions used at all | yes | yes | only if every table reports InnoDB at connect time | asked for, but a no-op: `SQLCEConnection::BeginTransaction` returns true without starting one ([Roadmap.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/Roadmap.md)) | | Choose it when | You already run SQL Server | You want a failed schema step to roll back cleanly | You already run it | As a new installation's default, up to a few dozen mailboxes; as the target of a migration, only for a test rig | ### MySQL and MariaDB since 6.3.4 **MySQL 5.7.9 or MariaDB 10.2.2, or later.** Since schema 6102 every table is `utf8mb4` under `utf8mb4_general_ci`, InnoDB, with `ROW_FORMAT=DYNAMIC`, and a key over a `varchar(255)` is then 1020 bytes: more than the 767 bytes an InnoDB key part may be in the older COMPACT rows, and within the 3072 bytes these releases, as they ship, take. **Why.** MySQL's `utf8` is three bytes a character, so an emoji in a contact, a folder or a signature was refused with error 1366 under strict mode and made a question mark without it. `utf8mb4_general_ci` is the twin of `utf8_general_ci`, so an address or a folder compares as it did, but it does not tell characters outside the Basic Multilingual Plane apart: two address books called `Family 👪` and `Family 🎉` are one name to its unique index. **Creating the database yourself:** ```sql CREATE DATABASE hmailserver CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci; GRANT ALL PRIVILEGES ON hmailserver.* TO 'hmailserver'@'%'; ``` The 6102 step also changes the database's own default character set, which needs the ALTER privilege on the database, not only on its tables. Every table must end up InnoDB: the server issues transactions only if all of them report it at connect time, and silently does not otherwise. A database the server creates itself is made `utf8mb4` already. **The 6.3.4 upgrade on these two** rebuilds every table, and on a server older than 5.7.9 or 10.2.2 it stops at 6041 with error 1071, having changed nothing. [Databases on Linux](/topic/28247) covers that step in full, its time, disk and duplicates (error 1062) included, and [Upgrading to 6.3.4](/topic/28193) the rest of the upgrade. ### Creating the database on Windows The wizard asks only "built-in or external". For the built-in engine, `DBSetupQuick.exe` creates the database at the current schema version, 6104. For an external engine, `DBSetup.exe` runs after the file copy, asks for the connection, and either creates the database or attaches one that exists; the secure defaults of a new 6.3.4 installation are applied only to one it created. Since 6.3.4 the administrator password reaches both tools in the environment variable `HM_ADMIN_PASSWORD`, never on a command line. The tools write `[Database]` in `Bin\hMailServer.INI`: | Key | Meaning | |---|---| | `Type` | `MSSQL`, `MYSQL`, `PostgreSQL` or `MSSQLCE`, matched case-insensitively | | `Server`, `Database`, `Username`, `Password` | The connection. `Server` is empty for the internal engine | | `PasswordEncryption` | Written by the server; says how `Password` is protected | | `Port` | `0` uses the backend's default | | `Internal` | `1` only for the embedded engine | | `NumberOfConnections` | Pool size; default 5 | **The password in this file may be machine-bound.** With `ProtectStoredSecretsWithDPAPI=1`, the default, the server writes `Password` through machine-scoped DPAPI (`PasswordEncryption=6`), and it cannot be decrypted on another machine. If you are also moving hosts, set the password on the new machine through `DBSetup.exe` rather than copying the encrypted one across. **Under `NT SERVICE\hMailServer`**, the account a new 6.3.4 installation runs as, `hMailServer.INI` is read-only to the service, and the COM methods the two tools call to write `[Database]` are refused before anything is made. The installer therefore runs its database step while the service is still LocalSystem. To change the database later, move the service to LocalSystem for the change and back ([Installing on Windows](/topic/28227) has the commands), or edit `[Database]` by hand and restart. ### Encrypting the connection (since 6.3.4) The test lab measured on 25 September 2026 that the connection to MySQL and MariaDB through MariaDB Connector/C, the client bundled on Windows and the one Ubuntu packages, was never encrypted: every account was reached in plaintext, password exchange, queries and rows alike, and one made with `REQUIRE SSL` refused the server. Since 6.3.4 two keys under `[Database]` choose the behaviour: ``` [Database] TlsMode=verify-full TlsCaFile=C:\hMailServer\certs\database-ca.pem ``` | `TlsMode` | What the connection does | |---|---| | *(empty, the default)* | **prefer**: TLS whenever the server offers it, and the connection without it when the server offers none, or when the TLS it offers cannot be agreed on (one attempt with TLS and one without, for that failure only). The certificate is not checked | | `prefer` | As the default, said explicitly | | `disable` | No TLS. SQL Server's providers still encrypt the login itself | | `require` | TLS or no connection; the certificate is not checked | | `verify-full` | TLS or no connection, the certificate checked: its chain to a CA in `TlsCaFile` (or the system's trust store when that is empty), and its name or address against `Server=` | `TlsCaFile` is a PEM file of CA certificates, read by `verify-full` and by nothing else, so that a CA file left beside `require` cannot quietly make it stronger or weaker than it says. A `TlsCaFile` that `verify-full` would read and that is not there refuses the connection (HM7531) rather than falling back to the system's store. Any other word in `TlsMode` is refused without an attempt (HM7530); `verify-ca` is refused by name, and PostgreSQL's own `PostgreSQLSslMode=verify-ca` remains for PostgreSQL. The keys are read at the service's start. | Backend | Before 6.3.4 | Since 6.3.4, with nothing set | |---|---|---| | MySQL / MariaDB, Windows and Linux | Plaintext, always | TLS whenever offered, unchecked; plaintext when not | | PostgreSQL, both platforms | libpq's default, prefer | Unchanged: libpq's prefer, or a `PGSSLMODE` in the service's environment, stands. `TlsMode` and `TlsCaFile` become `sslmode` and `sslrootcert` unless `PostgreSQLSslMode` and `PostgreSQLSslRootCert` are set, which win. `verify-full` with no `TlsCaFile` is `sslrootcert=system` on Linux (libpq 16 and later), and libpq's own default file on Windows, `%APPDATA%\postgresql\root.crt` of the service's account | | SQL Server, Windows | The login encrypted and nothing after it, unless the server forced encryption | The whole session encrypted through MSOLEDBSQL, when version 18 or later is installed, and through SQLNCLI for a failover partner; through SQLOLEDB only the login, unless `TlsMode` is `require` or `verify-full` | | SQL Server, Linux | Refused: ADO is Windows (HM6390) | Unchanged | | SQL Server Compact | A file on this machine | Unchanged | On MySQL and MariaDB the old `AllowUnencryptedConnection` is still read and changes nothing: what it described, TLS preferred and plaintext allowed, is the default now. On SQL Server `verify-full` checks against the Windows certificate store; `TlsCaFile` is not read (HM7532), and `ConnectionStringOptions` naming encryption beside a `TlsMode` is refused (HM7533). **Seeing it.** The application log says how the connection is protected, once the service has started and again whenever it changes, for example `Database connection: PostgreSQL: NOT encrypted: the server offered no TLS (TlsMode prefer, the default)`; the diagnostics run on the Diagnostics pages carries the same line under *Server details*. At the service's start a connection refused for TLS is the connect failure it always was, HM4354, with the database client's own reason and a sentence naming the setting to change. `--check-database` makes one connection with the TLS settings in the configuration, says what it got, and exits 0 connected, 1 not, 2 not configured. It changes nothing, so it runs beside a running service: ``` hmailserver --config /etc/hmailserver/hMailServer.ini --check-database hMailServer.exe --check-database --config C:\hMailServer\try.ini ``` **What the server does not do for you.** It cannot switch TLS on at the database server or give it a certificate. Under `verify-full` a MariaDB 11.4 server's own self-signed certificate is accepted without a CA file when the account's password proves it (MariaDB's zero-configuration TLS, for `mysql_native_password`, `ed25519` and `parsec` accounts), and not once a CA file is named. A database that must refuse a plaintext connection should say so itself: `REQUIRE SSL` on the MySQL account, `hostssl` only in `pg_hba.conf`, Force Encryption on SQL Server. **What is thinner than it could be.** There is no editor for these keys in the Control Panel or the Deck. `require` on the MariaDB client is checked after the login, and an unencrypted connection is closed before a statement crosses it: by then a challenge and a response have crossed in the clear for `mysql_native_password`, `caching_sha2_password` and `ed25519` accounts, and the password itself for an account that signs in through PAM or the clear-text plugin. `verify-full` refuses before the login. `require` protects against someone who can listen and not against someone who can alter the traffic, and `prefer` connects without TLS to a server that seems to offer none; only `verify-full` protects against someone on the path. And only a login with VIEW SERVER STATE lets SQL Server say whether its session is encrypted. ### Moving off the built-in database There is no migration tool. [Upgrading.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Upgrading.md) records that as a gap; [MigratingDatabaseBackend.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/MigratingDatabaseBackend.md) says it is not an omission, because backup and restore are already backend-agnostic: back up, repoint, restore. It works because the archive's XML carries no identity values, and the message store is addressed by domain name, local part and the file's own guid, never by id. Your mail files never move. **Before you start.** Write down how many domains, accounts, aliases and distribution lists there are, to verify against afterwards. Take a filesystem-level backup of the whole installation. Backup refuses to run if any message row points outside `DataFolder` ("All messages are not located in the data folder"). The restore refuses an archive taken by a newer hMailServer, so migrate on one version, then upgrade. Keep a copy of `hMailServer.ini`; the administrator password lives in it, so it carries over. And any mail delivered between the backup and the cutover is lost to the new database: hold inbound traffic, or accept the gap knowingly. 1. **Stop the service** (`Stop-Service hMailServer`). Restart it for the backup itself, which runs through the running server, but stop accepting new mail first if you can. 2. **Take the backup** on **Maintenance → Backup & restore** with all three of **Back up domains**, **Back up messages** and **Back up settings**. The archive lands as `HMBackup .7z`. All three are required: the restore refuses a combination that would delete something and put nothing back, and says so before it changes anything. **Set `BackupMessagesDBOnly` to `1` before both the backup and the restore.** Since 6.3.4 it is a stored setting that applies when it is saved, with no restart: set it in the Control Panel, over REST (`PUT /api/v1/settings/ini/BackupMessagesDBOnly` with `1`) or with `hmctl ini set BackupMessagesDBOnly 1`. On 6.3.3 it was a `[Settings]` line in the file, read at start, so restart there. Without it, **a normal restore empties the live data directory and copies the staged message store into it**; with it that step is skipped and your `.eml` files are never touched. Remove it afterwards. 3. **Stop the service again** (`Stop-Service hMailServer`) and wait for it to stop. 4. **Create the new database**: empty, with a login that may create tables; on MySQL or MariaDB, `utf8mb4`. Edit `[Database]` to point at it, with `Internal=0`. If the service runs as `NT SERVICE\hMailServer`, move it to LocalSystem first (`hMailServer.exe --service-account localsystem /Register`) and back once the migration is verified: under the virtual account the COM methods `DBSetupQuick.exe` calls are refused before anything is made. Then: ```powershell & "C:\Program Files\hMailServer\Bin\DBSetupQuick.exe" if ($LASTEXITCODE -ne 0) { throw "Database setup failed with exit code $LASTEXITCODE" } ``` On MySQL and SQL Server Compact a creation that fails halfway leaves a partly built database: drop it and start this step again. 5. **Restore.** Start the service (`Start-Service hMailServer`). At its first start against the new database the server adopts the file's `[Settings]` section into the empty store. **Before you restore, check `BackupMessagesDBOnly` on the new database** (`hmctl ini get BackupMessagesDBOnly`, or the Control Panel). The setting has to be identical for the backup and the restore, and the restore runs against the new database, which has a settings store of its own. If it is not `1` there, set it again, which applies when it is saved. That is the case when the old server could not write its line into the file, which it reported as `HM5806` or `HM5803`, and under `NT SERVICE\hMailServer`, which does not write the file's copy. A restore without it is a normal one, which empties the data directory. Then: ```powershell $app = New-Object -ComObject hMailServer.Application $app.Authenticate("Administrator", "") $backup = $app.BackupManager.LoadBackup("D:\hmail-migration\HMBackup 2026-08-13 220511.7z") $backup.RestoreDomains = $true $backup.RestoreMessages = $true $backup.RestoreSettings = $true $backup.StartRestore() ``` The server restarts itself at the end; that is expected. **Verify all of these**, because a partial restore looks like a working server: the domain, account, alias and list counts, against what you wrote down before you started; the folder list over IMAP as a real account; one whole message body; certificates, ports, IP ranges, routes and rules; one message in and one out; and an empty `ERROR_hmailserver_.log`. **Rolling back** is exact until you delete the old database, which was only ever read: stop the service, put the original `[Database]` section back, start the service. Since 6.3.4 the settings come back with it, and a setting changed since the cutover is named as `HM5804` on that start, which is the rollback working, not a fault. Ids change, so a script that recorded `hm_accounts.accountid` will break; key on the address instead. ### What it does not cover, and where to read more Moving the data folder to different storage is a separate operation, [Relocating an installation to another drive](/topic/28244); the path is `[Directories] DataFolder`. Moving to a new machine is [Moving to new hardware, and why Windows to Linux is a rebuild](/topic/28194). What the backup archive holds, and what it does not, is in [Backup and restore](/topic/28221). Sources: the wiki's [Before You Install](https://gitlab.com/hmailserver/hmailserver/-/wikis/Before-You-Install), [Migrating the Database Backend](https://gitlab.com/hmailserver/hmailserver/-/wikis/Migrating-the-Database-Backend) and [Installing on Linux](https://gitlab.com/hmailserver/hmailserver/-/wikis/Installing-on-Linux); [MigratingDatabaseBackend.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/MigratingDatabaseBackend.md), [DatabaseTls.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/DatabaseTls.md) and [Upgrading.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Upgrading.md); and sections [3.5](https://www.progressiverobot.com/hmailserver-documentation/#35-a-database) and [33.3](https://www.progressiverobot.com/hmailserver-documentation/#333-switching-database-engines) of the public documentation. --- Canonical: https://www.hmailserver.co.uk/topic/28228/choosing-a-database-creating-it-encrypting-the-link-and-leaving-the-built-in-one # Choosing a database, creating it, encrypting the link and leaving the built-in one *Written for 6.3.4. hMailServer 6.3.7 (17 October 2026) adds no schema step (6108, as 6.3.6). Its x64 installer's MySQL and MariaDB client is MariaDB Connector/C 3.4.11, built from MariaDB's source, in place of the 3.4.9 that 6.2.3 to 6.3.6 carried (CVE-2026-107819), with seven authentication plugins where 6.3.6's had nine: a database account that signs in with MySQL Enterprise's Windows authentication can no longer sign in, and the server then cannot open its database; the ARM64 installer carries no MySQL or MariaDB client. On MySQL, MariaDB, SQL Server and SQL Server Compact an insert that forgives a duplicate key now forgives that alone and any other failure is reported (HM5032), and on MySQL and MariaDB a lost connection is reconnected and the statement run again, so an error log that was quiet may now name failures that were hidden. See [the 6.3.7 announcement](/topic/28277).* This topic covers the four databases hMailServer 6.3.4 can keep its data in, what each needs before the schema can be made, how the connection to the database server is encrypted since 6.3.4, and how to move an installation off the built-in engine. The one thing to know first: **on MySQL and MariaDB, 6.3.4 needs MySQL 5.7.9 or MariaDB 10.2.2 or later**, because since schema 6102 every table is `utf8mb4`. On an older server the create script stops at its first table, and an upgrade from 6.3.3 stops at its first step, 6040 to 6041, with error 1071 and nothing changed. ### What the database holds, and what it does not hMailServer stores accounts, domains, message *metadata* and settings in the database. The message bodies are files on disk, one per message, at `\\\\{guid}.eml`, so the database stays small relative to `Data\`. Since 6.3.4 the database is also the settings store: the `[Settings]` keys of `hMailServer.ini` are rows from schema 6042, the file's section is the server's copy, and a line edited there for a key the database holds is named in the error log as `HM5804` and does not take effect. Where the service may write the file the line is put back; where it may not (a new Windows installation under `NT SERVICE\hMailServer`, a Linux package installation) the copy is not refreshed, the edited line stays, and the database's value is used. The `[Database]` section itself stays in the file, because it says how the store is reached. ### The four choices | Database | `[Database] Type` | What is bundled | What you must have first | |---|---|---|---| | Built-in (SQL Server Compact 4.0 SP1) | `MSSQLCE` | `SSCERuntime_x64-ENU.msi`, installed by setup | Nothing. This is the default | | MySQL / MariaDB | `MYSQL` | MariaDB Connector/C 3.4.9 as `Bin\libmysql.dll` (3.4.11 since 6.3.7, with seven authentication plugins where 6.3.6's had nine) | A reachable server, MySQL 5.7.9 or MariaDB 10.2.2 or later, an empty database, and a user that may create tables | | Microsoft SQL Server | `MSSQL` | Nothing; the OS's own data stack | The same, including Express edition | | PostgreSQL | `PostgreSQL` | `libpq` from PostgreSQL 18.3 | The same | **On Linux** there is no built-in database: a Linux server keeps its data in PostgreSQL or MySQL/MariaDB, and `hmailserver --create-database` refuses SQL Server and SQL Server Compact by name. For MySQL or MariaDB install the client library yourself, because the server opens `libmariadb.so.3` at run time: `libmariadb3` on Debian and Ubuntu, `mariadb-connector-c` on Fedora and RHEL. [Databases on Linux](/topic/28247) has the steps for each backend, and [First install on Linux](/topic/28190) the order of the whole first install. **Two things about the built-in engine.** It runs on a single database connection: `NumberOfConnections` is forced to 1 for `MSSQLCE`. And Microsoft has withdrawn the SQL Server Compact download; the copy in `installation/SQLCE/` is the archive of record, and it still installs. [MigratingDatabaseBackend.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/MigratingDatabaseBackend.md) calls it "the one real dependency liability in this tree". For anything up to a few dozen mailboxes it is adequate, and the last section of this topic is the way off it. ### How the four behave differently | | MS SQL Server | PostgreSQL | MySQL / MariaDB | SQL Server Compact | |---|---|---|---|---| | DDL in a transaction | Mostly | Yes, the only one of the four | No, commits as it goes | No, commits as it goes | | A failed schema step leaves | Normally nothing applied | Nothing applied | A partly built database | A partly built database | | Isolation the server uses | READ UNCOMMITTED | its default | its default | its default | | Transactions used at all | yes | yes | only if every table reports InnoDB at connect time | asked for, but a no-op: `SQLCEConnection::BeginTransaction` returns true without starting one ([Roadmap.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/Roadmap.md)) | | Choose it when | You already run SQL Server | You want a failed schema step to roll back cleanly | You already run it | As a new installation's default, up to a few dozen mailboxes; as the target of a migration, only for a test rig | ### MySQL and MariaDB since 6.3.4 **MySQL 5.7.9 or MariaDB 10.2.2, or later.** Since schema 6102 every table is `utf8mb4` under `utf8mb4_general_ci`, InnoDB, with `ROW_FORMAT=DYNAMIC`, and a key over a `varchar(255)` is then 1020 bytes: more than the 767 bytes an InnoDB key part may be in the older COMPACT rows, and within the 3072 bytes these releases, as they ship, take. **Why.** MySQL's `utf8` is three bytes a character, so an emoji in a contact, a folder or a signature was refused with error 1366 under strict mode and made a question mark without it. `utf8mb4_general_ci` is the twin of `utf8_general_ci`, so an address or a folder compares as it did, but it does not tell characters outside the Basic Multilingual Plane apart: two address books called `Family 👪` and `Family 🎉` are one name to its unique index. **Creating the database yourself:** ```sql CREATE DATABASE hmailserver CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci; GRANT ALL PRIVILEGES ON hmailserver.* TO 'hmailserver'@'%'; ``` The 6102 step also changes the database's own default character set, which needs the ALTER privilege on the database, not only on its tables. Every table must end up InnoDB: the server issues transactions only if all of them report it at connect time, and silently does not otherwise. A database the server creates itself is made `utf8mb4` already. **The 6.3.4 upgrade on these two** rebuilds every table, and on a server older than 5.7.9 or 10.2.2 it stops at 6041 with error 1071, having changed nothing. [Databases on Linux](/topic/28247) covers that step in full, its time, disk and duplicates (error 1062) included, and [Upgrading to 6.3.4](/topic/28193) the rest of the upgrade. ### Creating the database on Windows The wizard asks only "built-in or external". For the built-in engine, `DBSetupQuick.exe` creates the database at the current schema version, 6104. For an external engine, `DBSetup.exe` runs after the file copy, asks for the connection, and either creates the database or attaches one that exists; the secure defaults of a new 6.3.4 installation are applied only to one it created. Since 6.3.4 the administrator password reaches both tools in the environment variable `HM_ADMIN_PASSWORD`, never on a command line. The tools write `[Database]` in `Bin\hMailServer.INI`: | Key | Meaning | |---|---| | `Type` | `MSSQL`, `MYSQL`, `PostgreSQL` or `MSSQLCE`, matched case-insensitively | | `Server`, `Database`, `Username`, `Password` | The connection. `Server` is empty for the internal engine | | `PasswordEncryption` | Written by the server; says how `Password` is protected | | `Port` | `0` uses the backend's default | | `Internal` | `1` only for the embedded engine | | `NumberOfConnections` | Pool size; default 5 | **The password in this file may be machine-bound.** With `ProtectStoredSecretsWithDPAPI=1`, the default, the server writes `Password` through machine-scoped DPAPI (`PasswordEncryption=6`), and it cannot be decrypted on another machine. If you are also moving hosts, set the password on the new machine through `DBSetup.exe` rather than copying the encrypted one across. **Under `NT SERVICE\hMailServer`**, the account a new 6.3.4 installation runs as, `hMailServer.INI` is read-only to the service, and the COM methods the two tools call to write `[Database]` are refused before anything is made. The installer therefore runs its database step while the service is still LocalSystem. To change the database later, move the service to LocalSystem for the change and back ([Installing on Windows](/topic/28227) has the commands), or edit `[Database]` by hand and restart. ### Encrypting the connection (since 6.3.4) The test lab measured on 25 September 2026 that the connection to MySQL and MariaDB through MariaDB Connector/C, the client bundled on Windows and the one Ubuntu packages, was never encrypted: every account was reached in plaintext, password exchange, queries and rows alike, and one made with `REQUIRE SSL` refused the server. Since 6.3.4 two keys under `[Database]` choose the behaviour: ``` [Database] TlsMode=verify-full TlsCaFile=C:\hMailServer\certs\database-ca.pem ``` | `TlsMode` | What the connection does | |---|---| | *(empty, the default)* | **prefer**: TLS whenever the server offers it, and the connection without it when the server offers none, or when the TLS it offers cannot be agreed on (one attempt with TLS and one without, for that failure only). The certificate is not checked | | `prefer` | As the default, said explicitly | | `disable` | No TLS. SQL Server's providers still encrypt the login itself | | `require` | TLS or no connection; the certificate is not checked | | `verify-full` | TLS or no connection, the certificate checked: its chain to a CA in `TlsCaFile` (or the system's trust store when that is empty), and its name or address against `Server=` | `TlsCaFile` is a PEM file of CA certificates, read by `verify-full` and by nothing else, so that a CA file left beside `require` cannot quietly make it stronger or weaker than it says. A `TlsCaFile` that `verify-full` would read and that is not there refuses the connection (HM7531) rather than falling back to the system's store. Any other word in `TlsMode` is refused without an attempt (HM7530); `verify-ca` is refused by name, and PostgreSQL's own `PostgreSQLSslMode=verify-ca` remains for PostgreSQL. The keys are read at the service's start. | Backend | Before 6.3.4 | Since 6.3.4, with nothing set | |---|---|---| | MySQL / MariaDB, Windows and Linux | Plaintext, always | TLS whenever offered, unchecked; plaintext when not | | PostgreSQL, both platforms | libpq's default, prefer | Unchanged: libpq's prefer, or a `PGSSLMODE` in the service's environment, stands. `TlsMode` and `TlsCaFile` become `sslmode` and `sslrootcert` unless `PostgreSQLSslMode` and `PostgreSQLSslRootCert` are set, which win. `verify-full` with no `TlsCaFile` is `sslrootcert=system` on Linux (libpq 16 and later), and libpq's own default file on Windows, `%APPDATA%\postgresql\root.crt` of the service's account | | SQL Server, Windows | The login encrypted and nothing after it, unless the server forced encryption | The whole session encrypted through MSOLEDBSQL, when version 18 or later is installed, and through SQLNCLI for a failover partner; through SQLOLEDB only the login, unless `TlsMode` is `require` or `verify-full` | | SQL Server, Linux | Refused: ADO is Windows (HM6390) | Unchanged | | SQL Server Compact | A file on this machine | Unchanged | On MySQL and MariaDB the old `AllowUnencryptedConnection` is still read and changes nothing: what it described, TLS preferred and plaintext allowed, is the default now. On SQL Server `verify-full` checks against the Windows certificate store; `TlsCaFile` is not read (HM7532), and `ConnectionStringOptions` naming encryption beside a `TlsMode` is refused (HM7533). **Seeing it.** The application log says how the connection is protected, once the service has started and again whenever it changes, for example `Database connection: PostgreSQL: NOT encrypted: the server offered no TLS (TlsMode prefer, the default)`; the diagnostics run on the Diagnostics pages carries the same line under *Server details*. At the service's start a connection refused for TLS is the connect failure it always was, HM4354, with the database client's own reason and a sentence naming the setting to change. `--check-database` makes one connection with the TLS settings in the configuration, says what it got, and exits 0 connected, 1 not, 2 not configured. It changes nothing, so it runs beside a running service: ``` hmailserver --config /etc/hmailserver/hMailServer.ini --check-database hMailServer.exe --check-database --config C:\hMailServer\try.ini ``` **What the server does not do for you.** It cannot switch TLS on at the database server or give it a certificate. Under `verify-full` a MariaDB 11.4 server's own self-signed certificate is accepted without a CA file when the account's password proves it (MariaDB's zero-configuration TLS, for `mysql_native_password`, `ed25519` and `parsec` accounts), and not once a CA file is named. A database that must refuse a plaintext connection should say so itself: `REQUIRE SSL` on the MySQL account, `hostssl` only in `pg_hba.conf`, Force Encryption on SQL Server. **What is thinner than it could be.** There is no editor for these keys in the Control Panel or the Deck. `require` on the MariaDB client is checked after the login, and an unencrypted connection is closed before a statement crosses it: by then a challenge and a response have crossed in the clear for `mysql_native_password`, `caching_sha2_password` and `ed25519` accounts, and the password itself for an account that signs in through PAM or the clear-text plugin. `verify-full` refuses before the login. `require` protects against someone who can listen and not against someone who can alter the traffic, and `prefer` connects without TLS to a server that seems to offer none; only `verify-full` protects against someone on the path. And only a login with VIEW SERVER STATE lets SQL Server say whether its session is encrypted. ### Moving off the built-in database There is no migration tool. [Upgrading.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Upgrading.md) records that as a gap; [MigratingDatabaseBackend.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/MigratingDatabaseBackend.md) says it is not an omission, because backup and restore are already backend-agnostic: back up, repoint, restore. It works because the archive's XML carries no identity values, and the message store is addressed by domain name, local part and the file's own guid, never by id. Your mail files never move. **Before you start.** Write down how many domains, accounts, aliases and distribution lists there are, to verify against afterwards. Take a filesystem-level backup of the whole installation. Backup refuses to run if any message row points outside `DataFolder` ("All messages are not located in the data folder"). The restore refuses an archive taken by a newer hMailServer, so migrate on one version, then upgrade. Keep a copy of `hMailServer.ini`; the administrator password lives in it, so it carries over. And any mail delivered between the backup and the cutover is lost to the new database: hold inbound traffic, or accept the gap knowingly. 1. **Stop the service** (`Stop-Service hMailServer`). Restart it for the backup itself, which runs through the running server, but stop accepting new mail first if you can. 2. **Take the backup** on **Maintenance → Backup & restore** with all three of **Back up domains**, **Back up messages** and **Back up settings**. The archive lands as `HMBackup .7z`. All three are required: the restore refuses a combination that would delete something and put nothing back, and says so before it changes anything. **Set `BackupMessagesDBOnly` to `1` before both the backup and the restore.** Since 6.3.4 it is a stored setting that applies when it is saved, with no restart: set it in the Control Panel, over REST (`PUT /api/v1/settings/ini/BackupMessagesDBOnly` with `1`) or with `hmctl ini set BackupMessagesDBOnly 1`. On 6.3.3 it was a `[Settings]` line in the file, read at start, so restart there. Without it, **a normal restore empties the live data directory and copies the staged message store into it**; with it that step is skipped and your `.eml` files are never touched. Remove it afterwards. 3. **Stop the service again** (`Stop-Service hMailServer`) and wait for it to stop. 4. **Create the new database**: empty, with a login that may create tables; on MySQL or MariaDB, `utf8mb4`. Edit `[Database]` to point at it, with `Internal=0`. If the service runs as `NT SERVICE\hMailServer`, move it to LocalSystem first (`hMailServer.exe --service-account localsystem /Register`) and back once the migration is verified: under the virtual account the COM methods `DBSetupQuick.exe` calls are refused before anything is made. Then: ```powershell & "C:\Program Files\hMailServer\Bin\DBSetupQuick.exe" if ($LASTEXITCODE -ne 0) { throw "Database setup failed with exit code $LASTEXITCODE" } ``` On MySQL and SQL Server Compact a creation that fails halfway leaves a partly built database: drop it and start this step again. 5. **Restore.** Start the service (`Start-Service hMailServer`). At its first start against the new database the server adopts the file's `[Settings]` section into the empty store. **Before you restore, check `BackupMessagesDBOnly` on the new database** (`hmctl ini get BackupMessagesDBOnly`, or the Control Panel). The setting has to be identical for the backup and the restore, and the restore runs against the new database, which has a settings store of its own. If it is not `1` there, set it again, which applies when it is saved. That is the case when the old server could not write its line into the file, which it reported as `HM5806` or `HM5803`, and under `NT SERVICE\hMailServer`, which does not write the file's copy. A restore without it is a normal one, which empties the data directory. Then: ```powershell $app = New-Object -ComObject hMailServer.Application $app.Authenticate("Administrator", "") $backup = $app.BackupManager.LoadBackup("D:\hmail-migration\HMBackup 2026-08-13 220511.7z") $backup.RestoreDomains = $true $backup.RestoreMessages = $true $backup.RestoreSettings = $true $backup.StartRestore() ``` The server restarts itself at the end; that is expected. **Verify all of these**, because a partial restore looks like a working server: the domain, account, alias and list counts, against what you wrote down before you started; the folder list over IMAP as a real account; one whole message body; certificates, ports, IP ranges, routes and rules; one message in and one out; and an empty `ERROR_hmailserver_.log`. **Rolling back** is exact until you delete the old database, which was only ever read: stop the service, put the original `[Database]` section back, start the service. Since 6.3.4 the settings come back with it, and a setting changed since the cutover is named as `HM5804` on that start, which is the rollback working, not a fault. Ids change, so a script that recorded `hm_accounts.accountid` will break; key on the address instead. ### What it does not cover, and where to read more Moving the data folder to different storage is a separate operation, [Relocating an installation to another drive](/topic/28244); the path is `[Directories] DataFolder`. Moving to a new machine is [Moving to new hardware, and why Windows to Linux is a rebuild](/topic/28194). What the backup archive holds, and what it does not, is in [Backup and restore](/topic/28221). Sources: the wiki's [Before You Install](https://gitlab.com/hmailserver/hmailserver/-/wikis/Before-You-Install), [Migrating the Database Backend](https://gitlab.com/hmailserver/hmailserver/-/wikis/Migrating-the-Database-Backend) and [Installing on Linux](https://gitlab.com/hmailserver/hmailserver/-/wikis/Installing-on-Linux); [MigratingDatabaseBackend.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/MigratingDatabaseBackend.md), [DatabaseTls.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/DatabaseTls.md) and [Upgrading.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Upgrading.md); and sections [3.5](https://www.progressiverobot.com/hmailserver-documentation/#35-a-database) and [33.3](https://www.progressiverobot.com/hmailserver-documentation/#333-switching-database-engines) of the public documentation. --- Canonical: https://www.hmailserver.co.uk/topic/28229/your-first-domain-and-mailbox-and-the-dns-records-your-domain-needs # Your first domain and mailbox, and the DNS records your domain needs This topic takes a freshly installed hMailServer 6.3.4 to its first working mailbox, proves it locally, and then lists the DNS records without which its mail either does not arrive or is filed as spam. The one thing to know first: the domain's **Postmaster address** box is the catch-all, not the `postmaster@` mailbox the email standards ask for. Create `postmaster@` as an account, and leave that box empty unless you want every misaddressed message accepted and stored. The paths below are the Windows Control Panel's; on Linux, which has no Control Panel, the same is done in the Control Deck or over the REST API. ### If you are learning, make the sample domain first (since 6.3.4) One button, on the Control Panel's Welcome page and on the Control Deck's Checklist view, or one command (`hmctl sample-domain create`, `hmailserver --sample-domain create` on Linux, `POST /api/v1/setup/sample-domain`) makes a throwaway `example.test`: three mailboxes, a distribution list of the three, one rule and eight delivered messages. The same button, or `... remove`, takes it away and says what it removed; a domain called `example.test` that it did not make is never touched. All three mailboxes sign in with one password, which the server mints when it makes the sample: twenty characters of Crockford base32, grouped in fives. It is shown once, in the answer to the call that made the sample, and is stored nowhere, so write it down; if you lose it, remove the sample and make it again. Older wiki text that speaks of "one documented password" describes an earlier version of the feature and is out of date. **Remove the sample before the server takes real mail.** Its mailboxes are ordinary accounts that can sign in and relay. ### Add your domain Go to **Accounts & domains → Domains**, type the name and press **Add domain**, then select it and press **Properties**. On Linux, use the Deck's *Domains* view, or: ```sh curl -u Administrator -H "Content-Type: application/json" \ -d '{"name":"example.com","active":true}' \ http://127.0.0.1:8045/api/v1/domains ``` `postmaster` is optional in that body and is left out here, because it is the same catch-all field. The call answers `400` if the name is not a valid domain name and `409` if it already exists. Since 6.3.4 the Control Panel's domain editor has fourteen tabs; the seven that 6.3.4 added are each off, or empty, until set. The fields that matter on the first day: | Tab | Field | Default | What happens when it is wrong | |---|---|---|---| | General | Domain enabled | ticked | Unticked: every `RCPT TO` is answered `550 Domain has been disabled.` | | General | Domain name | | The part after the `@`. Setting it to the host name (`mail.example.com`) is the classic first-day mistake | | General | Postmaster address | empty | **The catch-all.** Set: no address in the domain can be refused. Empty: unknown recipients get `550 Unknown user` | | Limits | Maximum message size (KB) | `0` | `0` falls back to the server-wide `maxmessagesize`, seeded at 20480 KB | | Limits | Enable plus addressing | off, `+` | Off: `alice+invoices@` is an unknown recipient | | Relay | Relay host | empty | Where mail **from** this domain leaves through. This is not a route | | DKIM | Enable signing | off | Enabling before the DNS record is live makes every message fail DKIM at the receiver | In `RecipientParser::CheckDeliveryPossibility`, an address that matches no account, alias, distribution list or route falls through to the postmaster address and the resolution restarts with it. That is why the box is a catch-all. ### Add a mailbox Select the domain, type the address and a password under the account list, and press **Create account**; then **Edit** for the full editor. | Field | Value to use | Notes | |---|---|---| | Password | Press **Generate strong password** | 16 characters from a cryptographic RNG, shown once and copied to the clipboard | | Administration level | **Normal user** | **Server administrator** can manage everything. Give it only to yourself | | Quota (MB) | `0` for unlimited | | | Account enabled | ticked | Unticked: `550 Account is not active.` at `RCPT TO` | On Linux, make the account on the Control Deck or with `POST /api/v1/domains/{domain}/accounts`. Create at least `postmaster@yourcompany.com` and your own mailbox. A full mailbox is refused during the SMTP conversation, not bounced: with `RejectFullMailboxAtRcpt` on, as it is by default, `RCPT TO` is answered `452 4.2.2 Mailbox is full.`, a temporary refusal, so the sending server holds the message and no backscatter is sent. Since 6.3.4, accounts can be imported from a CSV file over REST, with `hmctl` and with the PowerShell module, with a dry run that changes nothing; and an identity provider can make and remove them over SCIM. Three things about an account are set only on the Control Deck and over REST, not in the Control Panel: its kind (a person, a room or equipment), encryption at rest, and a webhook route. ### How an address is resolved Every `RCPT TO` is answered by one function, in a fixed order that ends at the domain's postmaster address; the full order, with domain aliases, plus addressing and the webhook route, is in [Aliases, distribution lists and public folders](/topic/28270). What it means on the first day: | Question | Answer | |---|---| | Why did mail to a mistyped address get delivered? | The domain has a postmaster address, so it fell through to the catch-all | | Why does `alice+anything@` not work? | Plus addressing is off by default and is per domain | | Why does an alias to an external address still count as local? | Only at the first hop, so that it is treated as inbound mail and not as relaying | ### Test it locally, before touching DNS The test below works because a fresh database's *My computer* IP range lets the server itself, on `127.0.0.1`, send as one of its own domains without a password, while anyone may send you mail and nobody may relay through you without logging in. A refused direction is `550 Delivery is not allowed to this address.`; one that needs authentication is `530 SMTP authentication is required.` The two seeded ranges and what each allows are in [IP ranges, auto-ban, lockout and sending limits](/topic/28239). ```powershell Send-MailMessage -SmtpServer localhost -Port 25 ` -From "you@yourcompany.com" -To "you@yourcompany.com" ` -Subject "First test" -Body "If you can read this, hMailServer is working." ``` Or by hand with `telnet localhost 25`. The replies worth recognising: | Line | What it tells you | |---|---| | `250-SIZE 20971520` | The server-wide `maxmessagesize`, 20480 KB, in bytes | | `250-STARTTLS` | Present only when the port's connection security is STARTTLS optional or required | | `250 AUTH …` | Absent when the port requires TLS, or the connecting range requires TLS when authenticating | | `250 2.0.0 Queued (0.016 seconds)` | The message is on disk and in the database | | `250 2.0.0 Queued for delivery` | **Not** the same reply: the message was quarantined | Then check **Monitoring & troubleshooting → Delivery queue**, which should be empty, and look on disk: a delivered message is one file, `C:\Program Files\hMailServer\Data\yourcompany.com\you\a3\{a3f01e6c-....eml}`. Its first `Received` header reads `with ESMTP`, and `with ESMTPSA` for an authenticated submission over TLS. ### The DNS records your domain needs Do them in this order. Replace `yourcompany.com` and `203.0.113.10` with your own. | Record | Example | Without it | |---|---|---| | A / AAAA | `mail.yourcompany.com. IN A 203.0.113.10` | Nothing resolves; the MX points at nothing | | MX | `yourcompany.com. IN MX 10 mail.yourcompany.com.` | Inbound mail bounces as "no mail server for this domain" | | PTR | `10.113.0.203.in-addr.arpa. IN PTR mail.yourcompany.com.` | Google and Microsoft refuse or spam-folder outbound mail | | SPF | `yourcompany.com. IN TXT "v=spf1 mx a:mail.yourcompany.com ~all"` | Anybody can forge your domain; DMARC cannot pass on the SPF side | | DKIM | `dkim._domainkey.yourcompany.com. IN TXT "v=DKIM1; k=rsa; p=MIIBIjANBgkq..."` | Forwarded mail fails everything, because forwarding breaks SPF | | DMARC | `_dmarc.yourcompany.com. IN TXT "v=DMARC1; p=none; rua=mailto:dmarc@yourcompany.com"` | No policy and no reports | **A / AAAA and MX.** Add an `AAAA` only if the machine has a public IPv6 address that answers. An MX must point at a host name, never an IP address, and that name must not be a CNAME. The MX name is what MTA-STS matches, what TLSA records hang off, and what the certificate on port 25 should be issued for. **PTR.** Only your ISP or hosting provider can set it. It should name the same host as your `HELO`/`EHLO`, which hMailServer takes from **Connections & protocols → Protocols → SMTP**, *Host name (HELO/EHLO greeting)*, falling back to the Windows computer name when blank. The Control Panel's DNS records page does not check it, and says so. **SPF.** Start with `~all`; move to `-all` once the DMARC reports have shown you every system that sends as you. Publish exactly one SPF record per name: two beginning `v=spf1` is a permanent error at every receiver. RFC 7208 caps the DNS-querying mechanisms at ten. In the other direction, checking SPF on incoming mail as a scored test is off by default (`usespf` is seeded 0), but DMARC checking is on by default and evaluates SPF and DKIM itself. **DKIM.** On the domain's **DKIM** tab click **Generate key pair…**: it creates an RSA-2048 key, saves the private key and shows the exact record, with a **Copy DNS value** button. `dkim` is the default selector. Switching signing on is gated on the published record matching. Ed25519 is supported too, with a key you supply. Since 6.3.4 a rotation policy per domain generates the new key under a dated selector (`s20260915`), checks the record every fifteen minutes and switches signing only once it has carried that key for a propagation period. It never edits DNS, and it is off until switched on. **DMARC.** Start with `p=none`, read the reports for a few weeks, then tighten to `p=quarantine` and `p=reject`. Do not start at `p=reject`. The DNS records page pre-fills `rua=` with the domain's postmaster address and omits the tag when there is none. Since 6.3.4, name that mailbox as the domain's **report mailbox** and the server parses the DMARC and TLS-RPT reports that arrive there and charts them on the Control Deck's DMARC page. Nothing read there acts on any mail. **Optional hardening**, at any point afterwards: | Record | Name | Notes | |---|---|---| | MTA-STS | `_mta-sts.yourcompany.com. IN TXT "v=STSv1; id=20260908120000"` | hMailServer serves the policy itself once `WebServicesHttpsPort` is set and the listener has a certificate for `mta-sts.yourcompany.com`; `MtaStsPolicyMode` defaults to `enforce`, so set it to `testing` to start | | TLS-RPT | `_smtp._tls.yourcompany.com. IN TXT "v=TLSRPTv1; rua=mailto:tlsrpt@yourcompany.com"` | Asks other servers to tell you when TLS to you failed | | TLSA | `_25._tcp.mail.yourcompany.com. IN TLSA 3 1 1 ` | Only meaningful if your zone is DNSSEC-signed. Re-publish before you rotate a certificate, never after | | CNAME | `autoconfig`, `autodiscover`, `mta-sts` → `mail.yourcompany.com.` | For client autoconfiguration and the policy host | | A / AAAA / CNAME | `openpgpkey.yourcompany.com` | Since 6.3.4, only if you publish OpenPGP keys | BIMI is read, not published: since 6.3.4 the webmail shows a sender's logo, but this server publishes no BIMI record for your domains. ### Checking your work Give DNS an hour to propagate. **Monitoring & troubleshooting → DNS records** lists, per domain, the exact SPF, DKIM, DMARC, MTA-STS and TLS-RPT records to publish, with a **Check** button per record: | Verdict | What to do | |---|---| | **Not published** | Publish it | | **Published but different** | Compare the two; the page shows what is there | | **Published and correct** | Nothing | | **Lookup failed** | A resolver or network problem, not a DNS-content one. Retry | On Linux the Control Deck has a DNS records view. On either platform the first-hour checklist (since 6.3.4) judges the `spf`, `dkim` and `dmarc` items itself, for the first twenty active domains. From a command line: ```powershell Resolve-DnsName yourcompany.com -Type MX Resolve-DnsName yourcompany.com -Type TXT Resolve-DnsName dkim._domainkey.yourcompany.com -Type TXT Resolve-DnsName _dmarc.yourcompany.com -Type TXT Resolve-DnsName 10.113.0.203.in-addr.arpa -Type PTR ``` [Being trusted by the servers you send to](/topic/28188) takes deliverability further. ### The ports your users will need The create scripts seed four listeners, SMTP 25 and 587, POP3 110 and IMAP 143, with no encryption. Since 6.3.4 a new installation sets all four to STARTTLS optional with the self-signed certificate and requires TLS for authentication from the *Internet* IP range; an upgraded server is left as it was ([Hardening a public mail server](/topic/28241)). Ports 465, 993 and 995 are not created for you: add them on **Connections & protocols → TCP/IP ports** once you have a certificate. A TLS port with no certificate selected does not start (HM5113, `Error initializing SSL. Certificate not set.`); a port already in use is HM4316. Which connection security each port should have, and why STARTTLS optional belongs on port 25 alone, is in [Connecting a mail client](/topic/28255); which ports to open in the firewall is in [Every port hMailServer listens on](/topic/28226). ### Before you tell anyone the address - `postmaster@` exists as a real account. - MX, SPF, DKIM and DMARC are published and verified. - A certificate is installed and the client ports created: [Let's Encrypt certificates are automatic](/topic/28187). - The administrator password is a PBKDF2 one. 6.3.4's installer writes it so; the wiki's checklist item about changing it from "the installer's MD5" describes 6.3.3 and earlier. - A backup that you have restored once. ### What it does not do, and where to read more Nothing here publishes a DNS record for you: the Control Panel builds the text and checks it, and the DKIM rotation never edits DNS. The PTR record is your provider's to set. More on the records above: [DKIM rotation, DMARC and TLS reports](/topic/28258), and [Requiring TLS](/topic/28257) for MTA-STS, TLS-RPT and DANE. The webmail and browser administration are separate topics: [The webmail portal at /portal](/topic/28191) and [The Control Deck](/topic/28192). Sources: the wiki's [Your First Domain and Mailbox](https://gitlab.com/hmailserver/hmailserver/-/wikis/Your-First-Domain-and-Mailbox), [DNS for Your Domain](https://gitlab.com/hmailserver/hmailserver/-/wikis/DNS-for-Your-Domain) and [Ports Reference](https://gitlab.com/hmailserver/hmailserver/-/wikis/Ports-Reference); [SampleDomain.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SampleDomain.md) and [SetupChecklist.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SetupChecklist.md); and chapters [6](https://www.progressiverobot.com/hmailserver-documentation/#6-your-first-domain-and-mailbox), [8](https://www.progressiverobot.com/hmailserver-documentation/#8-making-the-internet-trust-you-dns) and [22](https://www.progressiverobot.com/hmailserver-documentation/#22-ports-reference) of the public documentation. --- Canonical: https://www.hmailserver.co.uk/topic/28229/your-first-domain-and-mailbox-and-the-dns-records-your-domain-needs # Your first domain and mailbox, and the DNS records your domain needs This topic takes a freshly installed hMailServer 6.3.4 to its first working mailbox, proves it locally, and then lists the DNS records without which its mail either does not arrive or is filed as spam. The one thing to know first: the domain's **Postmaster address** box is the catch-all, not the `postmaster@` mailbox the email standards ask for. Create `postmaster@` as an account, and leave that box empty unless you want every misaddressed message accepted and stored. The paths below are the Windows Control Panel's; on Linux, which has no Control Panel, the same is done in the Control Deck or over the REST API. ### If you are learning, make the sample domain first (since 6.3.4) One button, on the Control Panel's Welcome page and on the Control Deck's Checklist view, or one command (`hmctl sample-domain create`, `hmailserver --sample-domain create` on Linux, `POST /api/v1/setup/sample-domain`) makes a throwaway `example.test`: three mailboxes, a distribution list of the three, one rule and eight delivered messages. The same button, or `... remove`, takes it away and says what it removed; a domain called `example.test` that it did not make is never touched. All three mailboxes sign in with one password, which the server mints when it makes the sample: twenty characters of Crockford base32, grouped in fives. It is shown once, in the answer to the call that made the sample, and is stored nowhere, so write it down; if you lose it, remove the sample and make it again. Older wiki text that speaks of "one documented password" describes an earlier version of the feature and is out of date. **Remove the sample before the server takes real mail.** Its mailboxes are ordinary accounts that can sign in and relay. ### Add your domain Go to **Accounts & domains → Domains**, type the name and press **Add domain**, then select it and press **Properties**. On Linux, use the Deck's *Domains* view, or: ```sh curl -u Administrator -H "Content-Type: application/json" \ -d '{"name":"example.com","active":true}' \ http://127.0.0.1:8045/api/v1/domains ``` `postmaster` is optional in that body and is left out here, because it is the same catch-all field. The call answers `400` if the name is not a valid domain name and `409` if it already exists. Since 6.3.4 the Control Panel's domain editor has fourteen tabs; the seven that 6.3.4 added are each off, or empty, until set. The fields that matter on the first day: | Tab | Field | Default | What happens when it is wrong | |---|---|---|---| | General | Domain enabled | ticked | Unticked: every `RCPT TO` is answered `550 Domain has been disabled.` | | General | Domain name | | The part after the `@`. Setting it to the host name (`mail.example.com`) is the classic first-day mistake | | General | Postmaster address | empty | **The catch-all.** Set: no address in the domain can be refused. Empty: unknown recipients get `550 Unknown user` | | Limits | Maximum message size (KB) | `0` | `0` falls back to the server-wide `maxmessagesize`, seeded at 20480 KB | | Limits | Enable plus addressing | off, `+` | Off: `alice+invoices@` is an unknown recipient | | Relay | Relay host | empty | Where mail **from** this domain leaves through. This is not a route | | DKIM | Enable signing | off | Enabling before the DNS record is live makes every message fail DKIM at the receiver | In `RecipientParser::CheckDeliveryPossibility`, an address that matches no account, alias, distribution list or route falls through to the postmaster address and the resolution restarts with it. That is why the box is a catch-all. ### Add a mailbox Select the domain, type the address and a password under the account list, and press **Create account**; then **Edit** for the full editor. | Field | Value to use | Notes | |---|---|---| | Password | Press **Generate strong password** | 16 characters from a cryptographic RNG, shown once and copied to the clipboard | | Administration level | **Normal user** | **Server administrator** can manage everything. Give it only to yourself | | Quota (MB) | `0` for unlimited | | | Account enabled | ticked | Unticked: `550 Account is not active.` at `RCPT TO` | On Linux, make the account on the Control Deck or with `POST /api/v1/domains/{domain}/accounts`. Create at least `postmaster@yourcompany.com` and your own mailbox. A full mailbox is refused during the SMTP conversation, not bounced: with `RejectFullMailboxAtRcpt` on, as it is by default, `RCPT TO` is answered `452 4.2.2 Mailbox is full.`, a temporary refusal, so the sending server holds the message and no backscatter is sent. Since 6.3.4, accounts can be imported from a CSV file over REST, with `hmctl` and with the PowerShell module, with a dry run that changes nothing; and an identity provider can make and remove them over SCIM. Three things about an account are set only on the Control Deck and over REST, not in the Control Panel: its kind (a person, a room or equipment), encryption at rest, and a webhook route. ### How an address is resolved Every `RCPT TO` is answered by one function, in a fixed order that ends at the domain's postmaster address; the full order, with domain aliases, plus addressing and the webhook route, is in [Aliases, distribution lists and public folders](/topic/28270). What it means on the first day: | Question | Answer | |---|---| | Why did mail to a mistyped address get delivered? | The domain has a postmaster address, so it fell through to the catch-all | | Why does `alice+anything@` not work? | Plus addressing is off by default and is per domain | | Why does an alias to an external address still count as local? | Only at the first hop, so that it is treated as inbound mail and not as relaying | ### Test it locally, before touching DNS The test below works because a fresh database's *My computer* IP range lets the server itself, on `127.0.0.1`, send as one of its own domains without a password, while anyone may send you mail and nobody may relay through you without logging in. A refused direction is `550 Delivery is not allowed to this address.`; one that needs authentication is `530 SMTP authentication is required.` The two seeded ranges and what each allows are in [IP ranges, auto-ban, lockout and sending limits](/topic/28239). ```powershell Send-MailMessage -SmtpServer localhost -Port 25 ` -From "you@yourcompany.com" -To "you@yourcompany.com" ` -Subject "First test" -Body "If you can read this, hMailServer is working." ``` Or by hand with `telnet localhost 25`. The replies worth recognising: | Line | What it tells you | |---|---| | `250-SIZE 20971520` | The server-wide `maxmessagesize`, 20480 KB, in bytes | | `250-STARTTLS` | Present only when the port's connection security is STARTTLS optional or required | | `250 AUTH …` | Absent when the port requires TLS, or the connecting range requires TLS when authenticating | | `250 2.0.0 Queued (0.016 seconds)` | The message is on disk and in the database | | `250 2.0.0 Queued for delivery` | **Not** the same reply: the message was quarantined | Then check **Monitoring & troubleshooting → Delivery queue**, which should be empty, and look on disk: a delivered message is one file, `C:\Program Files\hMailServer\Data\yourcompany.com\you\a3\{a3f01e6c-....eml}`. Its first `Received` header reads `with ESMTP`, and `with ESMTPSA` for an authenticated submission over TLS. ### The DNS records your domain needs Do them in this order. Replace `yourcompany.com` and `203.0.113.10` with your own. | Record | Example | Without it | |---|---|---| | A / AAAA | `mail.yourcompany.com. IN A 203.0.113.10` | Nothing resolves; the MX points at nothing | | MX | `yourcompany.com. IN MX 10 mail.yourcompany.com.` | Inbound mail bounces as "no mail server for this domain" | | PTR | `10.113.0.203.in-addr.arpa. IN PTR mail.yourcompany.com.` | Google and Microsoft refuse or spam-folder outbound mail | | SPF | `yourcompany.com. IN TXT "v=spf1 mx a:mail.yourcompany.com ~all"` | Anybody can forge your domain; DMARC cannot pass on the SPF side | | DKIM | `dkim._domainkey.yourcompany.com. IN TXT "v=DKIM1; k=rsa; p=MIIBIjANBgkq..."` | Forwarded mail fails everything, because forwarding breaks SPF | | DMARC | `_dmarc.yourcompany.com. IN TXT "v=DMARC1; p=none; rua=mailto:dmarc@yourcompany.com"` | No policy and no reports | **A / AAAA and MX.** Add an `AAAA` only if the machine has a public IPv6 address that answers. An MX must point at a host name, never an IP address, and that name must not be a CNAME. The MX name is what MTA-STS matches, what TLSA records hang off, and what the certificate on port 25 should be issued for. **PTR.** Only your ISP or hosting provider can set it. It should name the same host as your `HELO`/`EHLO`, which hMailServer takes from **Connections & protocols → Protocols → SMTP**, *Host name (HELO/EHLO greeting)*, falling back to the Windows computer name when blank. The Control Panel's DNS records page does not check it, and says so. **SPF.** Start with `~all`; move to `-all` once the DMARC reports have shown you every system that sends as you. Publish exactly one SPF record per name: two beginning `v=spf1` is a permanent error at every receiver. RFC 7208 caps the DNS-querying mechanisms at ten. In the other direction, checking SPF on incoming mail as a scored test is off by default (`usespf` is seeded 0), but DMARC checking is on by default and evaluates SPF and DKIM itself. **DKIM.** On the domain's **DKIM** tab click **Generate key pair…**: it creates an RSA-2048 key, saves the private key and shows the exact record, with a **Copy DNS value** button. `dkim` is the default selector. Switching signing on is gated on the published record matching. Ed25519 is supported too, with a key you supply. Since 6.3.4 a rotation policy per domain generates the new key under a dated selector (`s20260915`), checks the record every fifteen minutes and switches signing only once it has carried that key for a propagation period. It never edits DNS, and it is off until switched on. **DMARC.** Start with `p=none`, read the reports for a few weeks, then tighten to `p=quarantine` and `p=reject`. Do not start at `p=reject`. The DNS records page pre-fills `rua=` with the domain's postmaster address and omits the tag when there is none. Since 6.3.4, name that mailbox as the domain's **report mailbox** and the server parses the DMARC and TLS-RPT reports that arrive there and charts them on the Control Deck's DMARC page. Nothing read there acts on any mail. **Optional hardening**, at any point afterwards: | Record | Name | Notes | |---|---|---| | MTA-STS | `_mta-sts.yourcompany.com. IN TXT "v=STSv1; id=20260908120000"` | hMailServer serves the policy itself once `WebServicesHttpsPort` is set and the listener has a certificate for `mta-sts.yourcompany.com`; `MtaStsPolicyMode` defaults to `enforce`, so set it to `testing` to start | | TLS-RPT | `_smtp._tls.yourcompany.com. IN TXT "v=TLSRPTv1; rua=mailto:tlsrpt@yourcompany.com"` | Asks other servers to tell you when TLS to you failed | | TLSA | `_25._tcp.mail.yourcompany.com. IN TLSA 3 1 1 ` | Only meaningful if your zone is DNSSEC-signed. Re-publish before you rotate a certificate, never after | | CNAME | `autoconfig`, `autodiscover`, `mta-sts` → `mail.yourcompany.com.` | For client autoconfiguration and the policy host | | A / AAAA / CNAME | `openpgpkey.yourcompany.com` | Since 6.3.4, only if you publish OpenPGP keys | BIMI is read, not published: since 6.3.4 the webmail shows a sender's logo, but this server publishes no BIMI record for your domains. ### Checking your work Give DNS an hour to propagate. **Monitoring & troubleshooting → DNS records** lists, per domain, the exact SPF, DKIM, DMARC, MTA-STS and TLS-RPT records to publish, with a **Check** button per record: | Verdict | What to do | |---|---| | **Not published** | Publish it | | **Published but different** | Compare the two; the page shows what is there | | **Published and correct** | Nothing | | **Lookup failed** | A resolver or network problem, not a DNS-content one. Retry | On Linux the Control Deck has a DNS records view. On either platform the first-hour checklist (since 6.3.4) judges the `spf`, `dkim` and `dmarc` items itself, for the first twenty active domains. From a command line: ```powershell Resolve-DnsName yourcompany.com -Type MX Resolve-DnsName yourcompany.com -Type TXT Resolve-DnsName dkim._domainkey.yourcompany.com -Type TXT Resolve-DnsName _dmarc.yourcompany.com -Type TXT Resolve-DnsName 10.113.0.203.in-addr.arpa -Type PTR ``` [Being trusted by the servers you send to](/topic/28188) takes deliverability further. ### The ports your users will need The create scripts seed four listeners, SMTP 25 and 587, POP3 110 and IMAP 143, with no encryption. Since 6.3.4 a new installation sets all four to STARTTLS optional with the self-signed certificate and requires TLS for authentication from the *Internet* IP range; an upgraded server is left as it was ([Hardening a public mail server](/topic/28241)). Ports 465, 993 and 995 are not created for you: add them on **Connections & protocols → TCP/IP ports** once you have a certificate. A TLS port with no certificate selected does not start (HM5113, `Error initializing SSL. Certificate not set.`); a port already in use is HM4316. Which connection security each port should have, and why STARTTLS optional belongs on port 25 alone, is in [Connecting a mail client](/topic/28255); which ports to open in the firewall is in [Every port hMailServer listens on](/topic/28226). ### Before you tell anyone the address - `postmaster@` exists as a real account. - MX, SPF, DKIM and DMARC are published and verified. - A certificate is installed and the client ports created: [Let's Encrypt certificates are automatic](/topic/28187). - The administrator password is a PBKDF2 one. 6.3.4's installer writes it so; the wiki's checklist item about changing it from "the installer's MD5" describes 6.3.3 and earlier. - A backup that you have restored once. ### What it does not do, and where to read more Nothing here publishes a DNS record for you: the Control Panel builds the text and checks it, and the DKIM rotation never edits DNS. The PTR record is your provider's to set. More on the records above: [DKIM rotation, DMARC and TLS reports](/topic/28258), and [Requiring TLS](/topic/28257) for MTA-STS, TLS-RPT and DANE. The webmail and browser administration are separate topics: [The webmail portal at /portal](/topic/28191) and [The Control Deck](/topic/28192). Sources: the wiki's [Your First Domain and Mailbox](https://gitlab.com/hmailserver/hmailserver/-/wikis/Your-First-Domain-and-Mailbox), [DNS for Your Domain](https://gitlab.com/hmailserver/hmailserver/-/wikis/DNS-for-Your-Domain) and [Ports Reference](https://gitlab.com/hmailserver/hmailserver/-/wikis/Ports-Reference); [SampleDomain.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SampleDomain.md) and [SetupChecklist.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SetupChecklist.md); and chapters [6](https://www.progressiverobot.com/hmailserver-documentation/#6-your-first-domain-and-mailbox), [8](https://www.progressiverobot.com/hmailserver-documentation/#8-making-the-internet-trust-you-dns) and [22](https://www.progressiverobot.com/hmailserver-documentation/#22-ports-reference) of the public documentation. --- Canonical: https://www.hmailserver.co.uk/topic/28230/bringing-mailboxes-in-imap-mirror-google-microsoft-365-mbox-maildir-and-pst # Bringing mailboxes in: IMAP mirror, Google, Microsoft 365, mbox, Maildir and PST *Written for 6.3.4. hMailServer 6.3.7 (17 October 2026) has the migrations as a card on the Control Panel's Backup, restore & migration page, where this topic says there is no Control Panel page; and a message collected from an external account is sent to SpamAssassin with the Return-Path the other server recorded, where it was the guess from the From field, so its SPF and envelope rules may score such mail differently. See [the 6.3.7 announcement](/topic/28277).* This topic covers every route 6.3.4 offers for bringing existing mail into this server: the webmail's *Other accounts* page for one person, an external account with the mirror on for a few mailboxes that must stay in step, migration jobs for hundreds of mailboxes from an IMAP server, Google Workspace or Microsoft 365, the Import Tool for mbox files and a Maildir, and the PST import for Outlook data files. It ends with the mailbox move, for a mailbox that is already here and needs a new address or a new domain. The one thing to know first: **every route needs the local account to exist**. Create the accounts with the Import Tool's text-file import, `POST /api/v1/domains/{domain}/accounts`, [a spreadsheet over REST and hmctl](/topic/28265), or [SCIM](/topic/28267), and only then bring the mail. Moving an *installation* is a different subject: [Upgrading to 6.3.4](/topic/28193) for an upgrade and [Moving to new hardware](/topic/28194) for a new machine. ### Which route | You are moving | Use | Keeps | Does not keep | |---|---|---|---| | One person's own mailbox, by that person | The webmail's **Other accounts** (since 6.3.4) | Folders as copies, then new mail collected | Rules, signatures | | A few mailboxes that must stay in step until the cut-over | An external account with **Mirror every folder** on (6.3.3 and later) | Folders, every message byte for byte, the five standard flags, the internal date | Subscriptions, custom keywords, ACLs | | Many mailboxes from an IMAP server, Google Workspace or Microsoft 365 | A **migration**, a job per mailbox (since 6.3.4, schema 6090) | The above, with keywords, special-use folders, contacts and calendars | Rules, signatures, shared mailboxes, ACLs, Gmail's labels as labels | | mbox files | The **Import Tool**, one folder per file (Windows only) | Every message, the date from the headers | Flags | | A Maildir | The Import Tool's **Select Maildir** (Windows only) | The Maildir++ hierarchy, the flags in the file names | Custom keywords | | Outlook PST files | **PST import**, a migration of kind `pst` (since 6.3.4) | Folders, messages rebuilt as Internet mail, contacts, calendar items, tasks | Attachments by reference, OLE objects, distribution lists | | What PST import refuses | Outlook itself, over IMAP | Whatever Outlook uploads | Anything Outlook does not put on the wire | ### One person, from the webmail: Other accounts *Since 6.3.4.* **Settings, Other accounts** lets a person do the IMAP route for their own mailbox without an administrator. They pick the old provider (Gmail, Outlook.com, Yahoo Mail and iCloud Mail fill in the server, port and encryption), give the user name and password, choose whether to leave the mail there, and tick **Bring the folders over too**. It is an external account made through `/api/v1/me/fetch-accounts`, with these differences from one an administrator makes: - The spam and virus filters are on for new mail; folders brought over arrive as copies, without the filters. - The host must resolve to public addresses only, on port 110 or 995 (POP3) or 143 or 993 (IMAP), unless the administrator turns on `reader_fetch_accounts_private_addresses`. Schema 6104 marks such an account `reader_created`, and the host is judged again at every collection: a name that answers a private address later is not collected, and the error log says why (`HM5909`). - The interval is at least 15 minutes, and a person has at most `reader_fetch_accounts_max` of them: 3 by default, and 0 takes the page away. - A host on `FetchOAuth2Hosts` (`outlook.office365.com` by default) is refused from the page; only a server administrator adds one. **Signing in rather than giving a password.** Outlook.com no longer accepts a password from another mail server, and Gmail takes only an app password. Where an administrator has registered the server with Google and Microsoft, the page offers **Sign in to Gmail** and **Sign in to Outlook.com**: the person grants IMAP access there and the account logs in with XOAUTH2. No password is stored, and the permission can be taken back at the provider. The first collection then lists the folders to tick, with Gmail's All Mail, Starred and Important unticked so that nothing arrives twice. The registration is set on `PUT /api/v1/settings`: `import_oauth_redirect_base_url` (the redirect URI to register is that plus `/portal/oauth/callback`), `import_oauth_google_client_id` and `import_oauth_google_client_secret`, `import_oauth_microsoft_client_id` and `import_oauth_microsoft_client_secret`. The Google client needs the Gmail API and the `https://mail.google.com/` scope; the Microsoft Entra registration needs the delegated `IMAP.AccessAsUser.All` and `offline_access` permissions. Until Google verifies the app, only the test users named on its consent screen can sign in. ### A few mailboxes that must stay in step: the mirror With **Mirror every folder** on, the server's own IMAP client does what `imapsync` does. The switch is on the account dialog's **External** tab in the Control Panel (Windows) and on the Deck's fetch-accounts view; over COM (Windows) it is `FetchAccount.ServerType = 1` and `FetchAccount.MirrorFolders = true`. 1. Create the local account. 2. Add an external account on it: server type IMAP, the remote host, port and security (implicit TLS on 993, STARTTLS on 143), the remote credentials, and the mirror on. 3. Decide what happens on the far side. **Days to keep messages** at 0 deletes each message from the remote folder once it is filed here; any other value leaves it there, so the two servers stay in step for as long as the account polls. A migration usually leaves the messages in place until the cut-over, then one last poll, and the account is disabled. 4. Poll. **Download now** starts a collection at once. A large mailbox is collected over several polls, and a message is recorded as collected only after it has been filed. A mirrored message is stored byte for byte: no `X-hMailServer-ExternalAccount` header, no delivery, no rules, no anti-spam or anti-virus, because it is a copy of mail the other server had already accepted. The remote flags and internal date become the local ones, and `Archive/2025` becomes `Archive.2025`. **Use a new external account for the mirror.** One that has been collecting the INBOX without the mirror collects it once more when the mirror is switched on. A `\Noselect` mailbox is skipped with a line in the application log, `OnExternalAccountDownload` does not fire for a mirrored message, and subscriptions, ACLs, quotas, Sieve and vacation settings on the far side are not read. ### Many mailboxes: migration jobs *Since 6.3.4, schema 6090.* A **migration** is a batch the server runs itself: a job per mailbox, throttled, resumed after a failure or a restart, with a report per mailbox and the contacts and calendars beside the mail. It is administered under `/api/v1/migrations` and in the Control Deck's **Migrations** view; there is no COM interface, and in 6.3.4 no Control Panel page (in 6.3.7's Control Panel the migrations are a card on **Backup, restore & migration**). A migration names its source: `kind` `imap` (a password per mailbox), `google` or `microsoft365` (each mailbox signed in to by its owner), or `pst`; `concurrency` (1 to 32, 4 by default); and `messages_per_minute` (0 for no limit). A Google or Microsoft 365 migration reads `imap.gmail.com` or `outlook.office365.com` on port 993 over TLS, and any other host is refused. A job is `queued`, `running`, `paused`, `done`, `failed` or `cancelled`, through the phases `mail`, `contacts`, `calendars` and `report`. ```text POST /api/v1/migrations {"name": "Old server", "host": "imap.old.example"} POST /api/v1/migrations/{id}/jobs the CSV below, or one job as JSON POST /api/v1/migrations/{id}/start GET /api/v1/migrations/{id}/jobs/{job}/report GET /api/v1/migrations/{id}/summary.csv ``` ```text sourceuser,secret,target anna@old.example,Anna's old password,anna@example.com carla@example.com,,carla@example.com ``` One request takes up to 5,000 lines. Every line is checked before any job is made, and a bad line refuses the whole request and names itself, so nothing is half added. A job into an account another migration is copying into now is refused (`409`). **Google Workspace and Microsoft 365.** A job added with an empty secret waits for its owner: the answer carries `signin_url`, a one-use link for that person to open, sign in as that mailbox and grant access. A sign-in as anybody else attaches nothing, and `POST .../jobs/{job}/signin` makes a new link. The scopes asked for are Google's `https://mail.google.com/`, `https://www.googleapis.com/auth/carddav` and `https://www.googleapis.com/auth/calendar`, and Microsoft's `https://outlook.office.com/IMAP.AccessAsUser.All` and `offline_access` with Microsoft Graph `Contacts.Read` and `Calendars.Read`. A work or school tenant may need an administrator's consent. **What a job does.** It maps the INBOX to the INBOX, a special-use folder (`\Sent`, `\Drafts`, `\Junk`, `\Trash`, `\Archive`) to the account's own folder of that use whatever it is called, and everything else to the same path. It opens each folder with `EXAMINE`, so nothing on the source is marked read, and stores each message byte for byte with its flags, keywords and internal date. A message too large is skipped as `too_large`; an account over its quota pauses the job. No rule, filter or scan runs over a copied message. Contacts and calendars follow: from an IMAP server by CardDAV and CalDAV found by RFC 6764 discovery (or `dav_url`), from Google by its CardDAV and CalDAV, from Microsoft 365 by Microsoft Graph. A phase that cannot run is skipped with its reason and never fails a job whose mail is copied. **A job is one copy, not a synchronisation**: move the MX records first and migrate after, or keep a mirror for the mailboxes that must stay in step. **It survives being interrupted.** After every 50 messages and at the end of each folder the job writes its resume marks, and a job that has run before reads back what it wrote, so each message arrives once. A failure time may mend is retried after 1, 5 and 15 minutes and then hourly; after 24 attempts in a row without a message copied the job is `failed`. Wrong credentials fail it at once, and `POST .../jobs/{job}/retry` with `{"secret": "..."}` takes a mended password. **Throttling** is threefold: `concurrency` per migration; `MigrationConnectionsPerHost` (4 by default, 1 to 64) across every migration from one source host, because a provider counts sessions per client address; and at most 64 jobs on the server at once. **Who may.** Reading needs `view_accounts` and writing `manage_accounts` in the target's domain. The server administrator creates a migration, and a domain administrator then sees and adds to it. The audit trail records migrations and jobs, never a secret's value. **Jobs are not carried by a backup.** `HM7240` (a resume mark could not be written) and `HM7241` (a message could not be filed) are the two codes that reach the error log for a job from an IMAP server, Google or Microsoft 365. A PST job says the same two as `HM7370` and `HM7371`, and its contacts and calendars phases add `HM7410` and `HM7411`. **The report** says, folder by folder, how many messages the source held, how many were copied and why any were skipped: `duplicate`, `another_folder`, `too_large`, `refused`, `gone`, `empty`. ### mbox files, a Maildir and accounts from a text file: the Import Tool The Import Tool is a Windows program (`Start menu > hMailServer > Import Tool`, or `Addons\ImportTool\ImportTool.exe`) that works through COM, each message going to the server through `Utilities.ImportMessageFromFileToIMAPFolder`; **there is no Import Tool on Linux**. It imports a folder of mbox files into one account, one IMAP folder per file; `Inbox` in any case is the INBOX, and a subfolder's files are a second run. Each message is dated from its `Received` or `Date` header. mbox carries no flags the tool could read, so every imported message is unread. **Select Maildir** points at the directory holding `cur`, `new` and `tmp`, and lists the INBOX and every Maildir++ folder beside it. The flags after `:2,` in a file name are set on the copy: `S` seen, `F` flagged, `R` answered, `D` draft, `T` deleted. A colon is not allowed in a Windows file name, so the tool also reads `;2,` and the same marker written with an exclamation mark in place of the colon; a copy whose names lost the info part imports unread. **Accounts from a text file.** The tool's other button creates or updates the accounts of one domain from a comma-separated text file, three fields a line: the account's name without the domain, its password and its size limit in MB (0 for none). Where [the breach check](/topic/28241) is on, it runs for each password the tool sets through COM's `Account.Password`, one request per account: run a large import against your own mirror of the endpoint, or with the check off. ### Outlook data files: PST import *Since 6.3.4.* The server reads a PST itself, with a reader of this project's own written from Microsoft's published specifications and no third-party PST code. It is a migration of kind `pst`: one job per file, each into one account. **Off until a directory is named.** A job names a file, never a path, and the file is looked for only in the subdirectory named for the target account's domain inside the directory `pst_import_directory` names (`PstImportDirectory` in `hm_settings`, set with `PUT /api/v1/settings` or on the Deck's Settings page). It is empty as shipped, and while it is empty every PST job is refused with *No PST import directory is set on this server*. ```text D:\PstImport\ <- pst_import_directory D:\PstImport\example.com\archive.pst <- a job for anna@example.com names archive.pst ``` 1. Copy the file into the domain's subdirectory, where the service's account can read it. A share on another machine is refused, and so is a name that climbs out of the domain's directory. 2. Make the migration, `POST /api/v1/migrations` with `{"name": "Archives", "kind": "pst"}`, then jobs as CSV lines of `file,password,target` over REST or in the Deck, or one job as JSON with `hmctl api POST /api/v1/migrations/{id}/jobs`. There is no Import Tool button for it and no COM route. 3. Start it. It needs `manage_accounts` and `manage_mailbox` in the target's domain. **What comes in.** The folders a person sees in Outlook, with Sent Items, Deleted Items, Drafts and Junk E-mail found by what the store says they are, never by their names. Every message is rebuilt as Internet mail, deterministically, so a resumed job knows what it filed. Read, unsent, flagged and replied state become `\Seen`, `\Draft`, `\Flagged` and `\Answered`, forwarded the keyword `$Forwarded`, categories keywords. Contacts go into the address book, calendar items and tasks into the calendars, notes and journal entries into mail folders as messages. ANSI files (Outlook 97 to 2002) are read. **What it refuses, with a sentence saying what to do**: the 4K-page layout Outlook 2013 and later write for an OST, an OST caching an Exchange mailbox, a file under Windows Information Protection, and a missing or wrong password. A file Outlook still has open pauses the job. A message over 128 MB of text and data together is left out and named. [Migration.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Migration.md) is candid about its evidence: no password-protected PST and no OST from Outlook were at hand. **The reader runs in a process of its own** (`hMailServer.exe --pst-read` on Windows, `hmailserver --pst-read` on Linux, not a command for an administrator), killed on a pause, a cancel or a stop, or when it has not answered in ten minutes. A file that crashes it costs one message, not the service. **This is crash containment, not a sandbox**: the reader runs as the service's own account, bounded in memory (2 GB committed on Windows, 4 GB of address space on Linux). The fixture `API.PstImport` runs on Windows; nothing runs it on Linux yet. ### What is not migrated - **Rules and filters, signatures, out-of-office replies and forwarding**: make them again here. - **Shared mailboxes and permissions**; a Microsoft 365 shared mailbox, which nobody signs in as, cannot be a job. - **Subscriptions**: every folder is created subscribed. - **Gmail's labels as labels.** A `google` migration copies each message once, and the star arrives as `\Flagged`. - **Contact groups**; from Microsoft 365, the default contacts folder and calendar only. ### A mailbox already here: renaming it, or moving it to another domain Do not copy it into a second account: that loses everything hanging off the account id, which is its rules, folders, application passwords, second factor, contacts, calendars and role. **Move or rename** (since 6.3.4) keeps the account's row, and leaves the old address as an alias unless you say otherwise. [Relocating an installation, and moving or renaming a mailbox](/topic/28244) has where it is done, what moves and what is refused. ### Where to read more - The wiki page: [Migration](https://gitlab.com/hmailserver/hmailserver/-/wikis/Migration) - The document, with every field, limit and refusal: [Migration.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Migration.md) - Renaming and moving a mailbox: [MailboxMove.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/MailboxMove.md) - The documentation page: [Importing accounts and mail, the Import Tool](https://www.progressiverobot.com/hmailserver-documentation/#338-importing-accounts-and-mail-the-import-tool) - On this forum: [The Control Deck](/topic/28192), where the Migrations view is, and [The webmail portal at /portal](/topic/28191), where Other accounts is. --- Canonical: https://www.hmailserver.co.uk/topic/28230/bringing-mailboxes-in-imap-mirror-google-microsoft-365-mbox-maildir-and-pst # Bringing mailboxes in: IMAP mirror, Google, Microsoft 365, mbox, Maildir and PST *Written for 6.3.4. hMailServer 6.3.7 (17 October 2026) has the migrations as a card on the Control Panel's Backup, restore & migration page, where this topic says there is no Control Panel page; and a message collected from an external account is sent to SpamAssassin with the Return-Path the other server recorded, where it was the guess from the From field, so its SPF and envelope rules may score such mail differently. See [the 6.3.7 announcement](/topic/28277).* This topic covers every route 6.3.4 offers for bringing existing mail into this server: the webmail's *Other accounts* page for one person, an external account with the mirror on for a few mailboxes that must stay in step, migration jobs for hundreds of mailboxes from an IMAP server, Google Workspace or Microsoft 365, the Import Tool for mbox files and a Maildir, and the PST import for Outlook data files. It ends with the mailbox move, for a mailbox that is already here and needs a new address or a new domain. The one thing to know first: **every route needs the local account to exist**. Create the accounts with the Import Tool's text-file import, `POST /api/v1/domains/{domain}/accounts`, [a spreadsheet over REST and hmctl](/topic/28265), or [SCIM](/topic/28267), and only then bring the mail. Moving an *installation* is a different subject: [Upgrading to 6.3.4](/topic/28193) for an upgrade and [Moving to new hardware](/topic/28194) for a new machine. ### Which route | You are moving | Use | Keeps | Does not keep | |---|---|---|---| | One person's own mailbox, by that person | The webmail's **Other accounts** (since 6.3.4) | Folders as copies, then new mail collected | Rules, signatures | | A few mailboxes that must stay in step until the cut-over | An external account with **Mirror every folder** on (6.3.3 and later) | Folders, every message byte for byte, the five standard flags, the internal date | Subscriptions, custom keywords, ACLs | | Many mailboxes from an IMAP server, Google Workspace or Microsoft 365 | A **migration**, a job per mailbox (since 6.3.4, schema 6090) | The above, with keywords, special-use folders, contacts and calendars | Rules, signatures, shared mailboxes, ACLs, Gmail's labels as labels | | mbox files | The **Import Tool**, one folder per file (Windows only) | Every message, the date from the headers | Flags | | A Maildir | The Import Tool's **Select Maildir** (Windows only) | The Maildir++ hierarchy, the flags in the file names | Custom keywords | | Outlook PST files | **PST import**, a migration of kind `pst` (since 6.3.4) | Folders, messages rebuilt as Internet mail, contacts, calendar items, tasks | Attachments by reference, OLE objects, distribution lists | | What PST import refuses | Outlook itself, over IMAP | Whatever Outlook uploads | Anything Outlook does not put on the wire | ### One person, from the webmail: Other accounts *Since 6.3.4.* **Settings, Other accounts** lets a person do the IMAP route for their own mailbox without an administrator. They pick the old provider (Gmail, Outlook.com, Yahoo Mail and iCloud Mail fill in the server, port and encryption), give the user name and password, choose whether to leave the mail there, and tick **Bring the folders over too**. It is an external account made through `/api/v1/me/fetch-accounts`, with these differences from one an administrator makes: - The spam and virus filters are on for new mail; folders brought over arrive as copies, without the filters. - The host must resolve to public addresses only, on port 110 or 995 (POP3) or 143 or 993 (IMAP), unless the administrator turns on `reader_fetch_accounts_private_addresses`. Schema 6104 marks such an account `reader_created`, and the host is judged again at every collection: a name that answers a private address later is not collected, and the error log says why (`HM5909`). - The interval is at least 15 minutes, and a person has at most `reader_fetch_accounts_max` of them: 3 by default, and 0 takes the page away. - A host on `FetchOAuth2Hosts` (`outlook.office365.com` by default) is refused from the page; only a server administrator adds one. **Signing in rather than giving a password.** Outlook.com no longer accepts a password from another mail server, and Gmail takes only an app password. Where an administrator has registered the server with Google and Microsoft, the page offers **Sign in to Gmail** and **Sign in to Outlook.com**: the person grants IMAP access there and the account logs in with XOAUTH2. No password is stored, and the permission can be taken back at the provider. The first collection then lists the folders to tick, with Gmail's All Mail, Starred and Important unticked so that nothing arrives twice. The registration is set on `PUT /api/v1/settings`: `import_oauth_redirect_base_url` (the redirect URI to register is that plus `/portal/oauth/callback`), `import_oauth_google_client_id` and `import_oauth_google_client_secret`, `import_oauth_microsoft_client_id` and `import_oauth_microsoft_client_secret`. The Google client needs the Gmail API and the `https://mail.google.com/` scope; the Microsoft Entra registration needs the delegated `IMAP.AccessAsUser.All` and `offline_access` permissions. Until Google verifies the app, only the test users named on its consent screen can sign in. ### A few mailboxes that must stay in step: the mirror With **Mirror every folder** on, the server's own IMAP client does what `imapsync` does. The switch is on the account dialog's **External** tab in the Control Panel (Windows) and on the Deck's fetch-accounts view; over COM (Windows) it is `FetchAccount.ServerType = 1` and `FetchAccount.MirrorFolders = true`. 1. Create the local account. 2. Add an external account on it: server type IMAP, the remote host, port and security (implicit TLS on 993, STARTTLS on 143), the remote credentials, and the mirror on. 3. Decide what happens on the far side. **Days to keep messages** at 0 deletes each message from the remote folder once it is filed here; any other value leaves it there, so the two servers stay in step for as long as the account polls. A migration usually leaves the messages in place until the cut-over, then one last poll, and the account is disabled. 4. Poll. **Download now** starts a collection at once. A large mailbox is collected over several polls, and a message is recorded as collected only after it has been filed. A mirrored message is stored byte for byte: no `X-hMailServer-ExternalAccount` header, no delivery, no rules, no anti-spam or anti-virus, because it is a copy of mail the other server had already accepted. The remote flags and internal date become the local ones, and `Archive/2025` becomes `Archive.2025`. **Use a new external account for the mirror.** One that has been collecting the INBOX without the mirror collects it once more when the mirror is switched on. A `\Noselect` mailbox is skipped with a line in the application log, `OnExternalAccountDownload` does not fire for a mirrored message, and subscriptions, ACLs, quotas, Sieve and vacation settings on the far side are not read. ### Many mailboxes: migration jobs *Since 6.3.4, schema 6090.* A **migration** is a batch the server runs itself: a job per mailbox, throttled, resumed after a failure or a restart, with a report per mailbox and the contacts and calendars beside the mail. It is administered under `/api/v1/migrations` and in the Control Deck's **Migrations** view; there is no COM interface, and in 6.3.4 no Control Panel page (in 6.3.7's Control Panel the migrations are a card on **Backup, restore & migration**). A migration names its source: `kind` `imap` (a password per mailbox), `google` or `microsoft365` (each mailbox signed in to by its owner), or `pst`; `concurrency` (1 to 32, 4 by default); and `messages_per_minute` (0 for no limit). A Google or Microsoft 365 migration reads `imap.gmail.com` or `outlook.office365.com` on port 993 over TLS, and any other host is refused. A job is `queued`, `running`, `paused`, `done`, `failed` or `cancelled`, through the phases `mail`, `contacts`, `calendars` and `report`. ```text POST /api/v1/migrations {"name": "Old server", "host": "imap.old.example"} POST /api/v1/migrations/{id}/jobs the CSV below, or one job as JSON POST /api/v1/migrations/{id}/start GET /api/v1/migrations/{id}/jobs/{job}/report GET /api/v1/migrations/{id}/summary.csv ``` ```text sourceuser,secret,target anna@old.example,Anna's old password,anna@example.com carla@example.com,,carla@example.com ``` One request takes up to 5,000 lines. Every line is checked before any job is made, and a bad line refuses the whole request and names itself, so nothing is half added. A job into an account another migration is copying into now is refused (`409`). **Google Workspace and Microsoft 365.** A job added with an empty secret waits for its owner: the answer carries `signin_url`, a one-use link for that person to open, sign in as that mailbox and grant access. A sign-in as anybody else attaches nothing, and `POST .../jobs/{job}/signin` makes a new link. The scopes asked for are Google's `https://mail.google.com/`, `https://www.googleapis.com/auth/carddav` and `https://www.googleapis.com/auth/calendar`, and Microsoft's `https://outlook.office.com/IMAP.AccessAsUser.All` and `offline_access` with Microsoft Graph `Contacts.Read` and `Calendars.Read`. A work or school tenant may need an administrator's consent. **What a job does.** It maps the INBOX to the INBOX, a special-use folder (`\Sent`, `\Drafts`, `\Junk`, `\Trash`, `\Archive`) to the account's own folder of that use whatever it is called, and everything else to the same path. It opens each folder with `EXAMINE`, so nothing on the source is marked read, and stores each message byte for byte with its flags, keywords and internal date. A message too large is skipped as `too_large`; an account over its quota pauses the job. No rule, filter or scan runs over a copied message. Contacts and calendars follow: from an IMAP server by CardDAV and CalDAV found by RFC 6764 discovery (or `dav_url`), from Google by its CardDAV and CalDAV, from Microsoft 365 by Microsoft Graph. A phase that cannot run is skipped with its reason and never fails a job whose mail is copied. **A job is one copy, not a synchronisation**: move the MX records first and migrate after, or keep a mirror for the mailboxes that must stay in step. **It survives being interrupted.** After every 50 messages and at the end of each folder the job writes its resume marks, and a job that has run before reads back what it wrote, so each message arrives once. A failure time may mend is retried after 1, 5 and 15 minutes and then hourly; after 24 attempts in a row without a message copied the job is `failed`. Wrong credentials fail it at once, and `POST .../jobs/{job}/retry` with `{"secret": "..."}` takes a mended password. **Throttling** is threefold: `concurrency` per migration; `MigrationConnectionsPerHost` (4 by default, 1 to 64) across every migration from one source host, because a provider counts sessions per client address; and at most 64 jobs on the server at once. **Who may.** Reading needs `view_accounts` and writing `manage_accounts` in the target's domain. The server administrator creates a migration, and a domain administrator then sees and adds to it. The audit trail records migrations and jobs, never a secret's value. **Jobs are not carried by a backup.** `HM7240` (a resume mark could not be written) and `HM7241` (a message could not be filed) are the two codes that reach the error log for a job from an IMAP server, Google or Microsoft 365. A PST job says the same two as `HM7370` and `HM7371`, and its contacts and calendars phases add `HM7410` and `HM7411`. **The report** says, folder by folder, how many messages the source held, how many were copied and why any were skipped: `duplicate`, `another_folder`, `too_large`, `refused`, `gone`, `empty`. ### mbox files, a Maildir and accounts from a text file: the Import Tool The Import Tool is a Windows program (`Start menu > hMailServer > Import Tool`, or `Addons\ImportTool\ImportTool.exe`) that works through COM, each message going to the server through `Utilities.ImportMessageFromFileToIMAPFolder`; **there is no Import Tool on Linux**. It imports a folder of mbox files into one account, one IMAP folder per file; `Inbox` in any case is the INBOX, and a subfolder's files are a second run. Each message is dated from its `Received` or `Date` header. mbox carries no flags the tool could read, so every imported message is unread. **Select Maildir** points at the directory holding `cur`, `new` and `tmp`, and lists the INBOX and every Maildir++ folder beside it. The flags after `:2,` in a file name are set on the copy: `S` seen, `F` flagged, `R` answered, `D` draft, `T` deleted. A colon is not allowed in a Windows file name, so the tool also reads `;2,` and the same marker written with an exclamation mark in place of the colon; a copy whose names lost the info part imports unread. **Accounts from a text file.** The tool's other button creates or updates the accounts of one domain from a comma-separated text file, three fields a line: the account's name without the domain, its password and its size limit in MB (0 for none). Where [the breach check](/topic/28241) is on, it runs for each password the tool sets through COM's `Account.Password`, one request per account: run a large import against your own mirror of the endpoint, or with the check off. ### Outlook data files: PST import *Since 6.3.4.* The server reads a PST itself, with a reader of this project's own written from Microsoft's published specifications and no third-party PST code. It is a migration of kind `pst`: one job per file, each into one account. **Off until a directory is named.** A job names a file, never a path, and the file is looked for only in the subdirectory named for the target account's domain inside the directory `pst_import_directory` names (`PstImportDirectory` in `hm_settings`, set with `PUT /api/v1/settings` or on the Deck's Settings page). It is empty as shipped, and while it is empty every PST job is refused with *No PST import directory is set on this server*. ```text D:\PstImport\ <- pst_import_directory D:\PstImport\example.com\archive.pst <- a job for anna@example.com names archive.pst ``` 1. Copy the file into the domain's subdirectory, where the service's account can read it. A share on another machine is refused, and so is a name that climbs out of the domain's directory. 2. Make the migration, `POST /api/v1/migrations` with `{"name": "Archives", "kind": "pst"}`, then jobs as CSV lines of `file,password,target` over REST or in the Deck, or one job as JSON with `hmctl api POST /api/v1/migrations/{id}/jobs`. There is no Import Tool button for it and no COM route. 3. Start it. It needs `manage_accounts` and `manage_mailbox` in the target's domain. **What comes in.** The folders a person sees in Outlook, with Sent Items, Deleted Items, Drafts and Junk E-mail found by what the store says they are, never by their names. Every message is rebuilt as Internet mail, deterministically, so a resumed job knows what it filed. Read, unsent, flagged and replied state become `\Seen`, `\Draft`, `\Flagged` and `\Answered`, forwarded the keyword `$Forwarded`, categories keywords. Contacts go into the address book, calendar items and tasks into the calendars, notes and journal entries into mail folders as messages. ANSI files (Outlook 97 to 2002) are read. **What it refuses, with a sentence saying what to do**: the 4K-page layout Outlook 2013 and later write for an OST, an OST caching an Exchange mailbox, a file under Windows Information Protection, and a missing or wrong password. A file Outlook still has open pauses the job. A message over 128 MB of text and data together is left out and named. [Migration.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Migration.md) is candid about its evidence: no password-protected PST and no OST from Outlook were at hand. **The reader runs in a process of its own** (`hMailServer.exe --pst-read` on Windows, `hmailserver --pst-read` on Linux, not a command for an administrator), killed on a pause, a cancel or a stop, or when it has not answered in ten minutes. A file that crashes it costs one message, not the service. **This is crash containment, not a sandbox**: the reader runs as the service's own account, bounded in memory (2 GB committed on Windows, 4 GB of address space on Linux). The fixture `API.PstImport` runs on Windows; nothing runs it on Linux yet. ### What is not migrated - **Rules and filters, signatures, out-of-office replies and forwarding**: make them again here. - **Shared mailboxes and permissions**; a Microsoft 365 shared mailbox, which nobody signs in as, cannot be a job. - **Subscriptions**: every folder is created subscribed. - **Gmail's labels as labels.** A `google` migration copies each message once, and the star arrives as `\Flagged`. - **Contact groups**; from Microsoft 365, the default contacts folder and calendar only. ### A mailbox already here: renaming it, or moving it to another domain Do not copy it into a second account: that loses everything hanging off the account id, which is its rules, folders, application passwords, second factor, contacts, calendars and role. **Move or rename** (since 6.3.4) keeps the account's row, and leaves the old address as an alias unless you say otherwise. [Relocating an installation, and moving or renaming a mailbox](/topic/28244) has where it is done, what moves and what is refused. ### Where to read more - The wiki page: [Migration](https://gitlab.com/hmailserver/hmailserver/-/wikis/Migration) - The document, with every field, limit and refusal: [Migration.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Migration.md) - Renaming and moving a mailbox: [MailboxMove.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/MailboxMove.md) - The documentation page: [Importing accounts and mail, the Import Tool](https://www.progressiverobot.com/hmailserver-documentation/#338-importing-accounts-and-mail-the-import-tool) - On this forum: [The Control Deck](/topic/28192), where the Migrations view is, and [The webmail portal at /portal](/topic/28191), where Other accounts is. --- Canonical: https://www.hmailserver.co.uk/topic/28231/unattended-installs-winget-and-chocolatey-and-configuration-from-the-environment # Unattended installs, winget and Chocolatey, and configuration from the environment *Written for 6.3.4. hMailServer 6.3.6 (6 October 2026) publishes the first container image, `registry.gitlab.com/hmailserver/hmailserver:6.3.6`; its installer exits 10 or 11 when a step after its files did not complete (it exited 0), and it makes the data, log, temp, database and event folders private to the administrators and the service. See [the 6.3.6 announcement](/topic/28276). hMailServer 6.3.7 (17 October 2026), a security release, publishes `registry.gitlab.com/hmailserver/hmailserver:6.3.7` (also tagged `6.3` and `latest`); its Windows installers carry the .NET Desktop Runtime 10.0.13, installed only where no .NET 10 Desktop Runtime is installed at all, so an upgraded server keeps the one it had; at every installation and upgrade `/ProtectFolders` hardens the program folder, `Bin` and `ControlPanel`, since an installation in a folder that took its permissions from the root of a drive let any account that could sign in to the computer replace the programs (CVE-2026-107571): before upgrading one, follow `ServiceAccount.md`'s steps, and where other accounts could sign in while it was so installed, reinstall into a new folder only the administrators may change, given to setup with `/DIR=`, since over an installation setup installs into the folder it finds; and setup waits `ShutdownDrainSeconds` plus 90 seconds for the service's stop, where it waited plus 30. Chocolatey and the apt and dnf repositories still publish nothing: no channel is live as 6.3.7 is made, and 6.3.7 is not on the Snap Store. See [the 6.3.7 announcement](/topic/28277).* This topic covers installing hMailServer 6.3.4 without a wizard: the Windows installer's silent switches and what a silent install does with the administrator password; where the `winget` and Chocolatey packages stand; the signed apt and dnf repositories and why they are not live; and, since 6.3.4, every setting from the environment and the four setting profiles, the two ways a deployment shapes a server before it starts. The one thing to know first: **an unattended install leaves the administrator password unset unless you pass it**, and 6.3.4 is installed by hand from the release page, because neither package manager carries it and no [live update](/topic/28234) installs it ([the 6.3.4 announcement](/topic/28218)). ### The Windows installer's switches The installer is Inno Setup, so it takes the standard switches and a few of its own. | Switch | Effect | |---|---| | `/SILENT` | No wizard; progress window only | | `/VERYSILENT` | No wizard and no progress window | | `/SUPPRESSMSGBOXES` | Suppress message boxes; some failure paths still raise one without it | | `/LOG=""` | Write an install log; it is the only record of what happened | | `/DIR=""` | Installation directory | | `/COMPONENTS="server,admintools,controlpanel"` | `server` is the service; `admintools` registers the COM type library; `controlpanel` installs the Control Panel, which does not need `admintools` | | `/adminpassword=` | Custom: sets the administrator password (at least 5 characters) and hands it to the database tools | | `/upgradetoken=` | Custom: the single-use token of a live update; not for people to type | | `/useinternaldbms=true\|false` | Custom: `true` (the default) uses the bundled SQL Server Compact database | | `/serviceaccount=virtual\|localsystem\|keep` | Custom: `virtual` is `NT SERVICE\hMailServer`, the default for a new installation; `keep` is the default for an upgrade | | `/profile=NAME` | Custom, since 6.3.4: the setting profile a new installation applies, or `none`. Written and read, and not run (below) | ```bat hMailServer-x.y.z-x64.exe /VERYSILENT /SUPPRESSMSGBOXES /LOG="C:\Temp\hmail-install.log" ^ /COMPONENTS="server,controlpanel" /useinternaldbms=true ``` **Treat a non-zero exit code as a failed install and read the log.** The installer checks both that the database tool launched and its exit code, so a failed or cancelled schema step fails the install rather than passing silently. The log has the tool's code: from `DBSetupQuick.exe`, 0 is a database created or already up to date, 1 a failed create or a `DBUpdater.exe` that would not start, 3 a wrong or missing administrator password, and any other `DBUpdater.exe`'s own code. A `hm_dbversion` still on the old value afterwards is the same failure. A silent installation does not approve folders or key files the configuration names outside the program folder, and leaves such a service on LocalSystem rather than moving it to the virtual account. ### The administrator password, silently - **A fresh silent install leaves the administrator password unset** unless you pass `/adminpassword=` or set `HM_ADMIN_PASSWORD`: without either, the `AdministratorPassword` key is left out of `hMailServer.ini` entirely. - **Set it afterwards with `hMailServer.exe --set-admin-password`**, new in 6.3.4, from an elevated prompt, then restart the service. It takes the password from `HM_ADMIN_PASSWORD` or, when that is not set, from one line of standard input when that is a pipe or a file, and never from an argument. Prefer the variable: a pipe from Windows PowerShell 5.1 turns every non-ASCII character into `?`, storing a password nobody typed. On Linux the command is `hmailserver --set-admin-password`. [ServiceAccount.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/ServiceAccount.md) and [Upgrading.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Upgrading.md) write the Windows step as `hMailServer.exe /SetAdministratorPassword`. - **The Control Panel and COM are refused under the virtual account.** A new 6.3.4 installation runs the service as `NT SERVICE\hMailServer` by default, and under that account `hMailServer.ini` is read-only to the service: the Control Panel's *Administrator password* card and COM's `Settings.SetAdministratorPassword` are each refused with a sentence. They set the password only where the service may write the file, as it may under LocalSystem, which an upgrade keeps unless it is answered otherwise. - **A silent upgrade of an installation that has a password still needs it** for the database upgrade, and cannot prompt. Without it the database upgrade fails and the installer reports a non-zero exit code rather than waiting. With the password empty, a silent upgrade works. - **Since 6.3.4, `HM_ADMIN_PASSWORD` is the safer of the two.** Inno Setup drops every double quote from its command line, so `/adminpassword=ab"cd` sets `abcd`, and anybody listing processes while setup runs can read the switch. The variable takes any character. Setup reads it only when neither `/adminpassword=` nor `/upgradetoken=` is given. - **Set the variable in an elevated prompt.** Setup started without administrator rights elevates itself through the UAC prompt, and the variable does not cross it. Until 6.3.4 the password went unquoted on the database tools' command lines, so one with a space, a double quote or backslashes before one failed the database step with exit code 3. It now goes in `HM_ADMIN_PASSWORD`. ### winget and Chocolatey: not in the public repositories yet The manifests exist and are validated for each release, from the release's own Authenticode-signed installer. **They are not in the public repositories**: neither `winget install ProgressiveRobot.hMailServer` nor `choco install hmailserver` installs this fork until a first submission has been reviewed and merged. **Do not run `choco install hmailserver` expecting this fork.** [PackageManagers.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/PackageManagers.md) records none on 15 September 2026, but on 27 September 2026 the community feed held an unlisted `hmailserver` package: the original project's 5.6.8, not this fork's. What the packages are built to do, once accepted: - Both run the signed installer the release page offers with `/VERYSILENT /SUPPRESSMSGBOXES /NORESTART`. `/SUPPRESSMSGBOXES` is not optional: without it a database-setup failure shows a message box, silent mode answers it with the default button, and a broken install reports success. - `/TYPE=full` is deliberately not passed, so an upgrade keeps the components already installed and does not put the Control Panel back on a server where it was left out. - `MinimumOSVersion: 10.0.14393.0` is Windows 10 1607, the floor of the .NET 10 Desktop Runtime the Control Panel needs. - **The administrator password is left unset**, and neither package invents one. Set one before the server is reachable from anywhere but the machine itself. - Uninstalling runs the installer's own uninstaller, which does not delete the message store, the database or `hMailServer.ini`. - A pre-release is never submitted. ### apt and dnf on Linux: written, tested in a lab, not live The project keeps signed apt and dnf repositories on the update feed's site, `https://updates.progressiverobot.com`. **They are not live.** The job that makes them, `package-repos`, was tested on the lab machine on 26 September 2026 with a throwaway key and dummy packages, and publishes nothing until the owner has made the signing key. This is what a Debian or Ubuntu user will type once it is live; on a release with no suite, `apt update` answers 404 for its `InRelease`. ```sh sudo install -d -m 0755 /etc/apt/keyrings sudo curl -fsSLo /etc/apt/keyrings/hmailserver.asc https://updates.progressiverobot.com/hmailserver-repository-key.asc gpg --show-keys /etc/apt/keyrings/hmailserver.asc # compare the fingerprint with the one below echo "deb [signed-by=/etc/apt/keyrings/hmailserver.asc] https://updates.progressiverobot.com/apt $(. /etc/os-release && echo "$ID$VERSION_ID") main" \ | sudo tee /etc/apt/sources.list.d/hmailserver.list sudo apt update sudo apt install hmailserver ``` The fingerprint does not exist yet: [PackageManagers.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/PackageManagers.md) records it as `(not made yet)`. **There is no dnf repository at all** until a stable release carries `.rpm` files built for each distribution, and 6.3.4 carries the generic one alone. Once one does, on Fedora and Enterprise Linux it is `sudo curl -fsSLo /etc/yum.repos.d/hmailserver.repo https://updates.progressiverobot.com/rpm/fc44/hmailserver.repo` and then `sudo dnf install hmailserver`, with `fc44` replaced by the release's directory. **What 6.3.4 carries limits what the repositories could hold.** Its Linux packages are one generic x86-64 `.deb` and one generic `.rpm`, neither built for one distribution; [hMailServer on Linux: what you get and what you do not](/topic/28189) says where each installs. In the repositories as [PackageManagers.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/PackageManagers.md) describes them, the `.deb` belongs in the `ubuntu26.04` suite, and the `.rpm`, built on Ubuntu 26.04, goes in no repository. On 26 September 2026 `dnf install` of the `.rpm` built from `master` that day resolved its dependencies on Fedora 43 and 44 and AlmaLinux 10, and was refused on AlmaLinux 9, whose glibc and libstdc++ are older than it asks for. Packages built for each distribution come from the release after 6.3.4. The packages carry no RPM signature of their own: the signed repository metadata carries every package's SHA-256. Not tried: the job on GitLab's runners, HTTPS end to end, arm64 and aarch64. Until it is live, [First install on Linux](/topic/28190) is the walk. ### Every setting from the environment *Since 6.3.4.* Every `[Settings]` key and every `hm_settings` row the server reads has a name in the environment. | Variable | Sets | |---|---| | `HM_SETTING_` | The `[Settings]` key, one of 241: `HM_SETTING_MaxSubmissionsPerIPPerMinute=30` | | `HM_PROPERTY_` | The `hm_settings` row: `HM_PROPERTY_maxmessagesize=20480` | | `HM_SETTING__FILE`, `HM_PROPERTY__FILE` | The same, read from the file the variable names, such as a Kubernetes or Compose Secret; UTF-8, at most 64 KB, trailing line breaks stripped | The prefixes and `_FILE` are written in capitals; the name after the prefix is matched without regard to case, and a key is stored in its own spelling. **What is refused, before anything is stored**: a name that is not a setting of the store its prefix names (`MaxMessageSize` after `HM_SETTING_` is refused naming `HM_PROPERTY_maxmessagesize`); a misspelt prefix such as `HM_SETTINGS_`; a read-only property such as `WebPushVapidPublicKey`; the plain and the `_FILE` form of one setting together; a `_FILE` naming a directory, a missing path or a file over 64 KB, refused naming the path, never the content; the `_FILE` form of a setting a domain or an account may hold its own value of; and a value the setting's own REST check would refuse, in that check's words. `maxmessagesize=off` is refused rather than stored as 0, which is "no limit". **The verb.** ```sh hmailserver --config /etc/hmailserver/hMailServer.ini --apply-environment ``` `hMailServer.exe --apply-environment` on Windows is the same verb from the same code. It opens the configuration and the database as the service does, so run it as the user the service runs as. It **checks every value first and exits 1 having changed nothing** when any is refused, then writes each through the setter `PUT /api/v1/settings/...` uses and prints a line per variable: `set`, `unchanged` or `refused`. A secret's value is never printed. Exit 2 is a configuration file that names no database. The verb's writes are not rows in the audit trail: its own output, in a container the container's log, says what it changed. **Who runs it.** No container image of 6.3.4 is published: the image is one you build yourself, from the Dockerfile in the tree. Its entrypoint runs the verb after `--create-database` and `--upgrade-database` and before the server. **On Windows nothing runs it for you**: a Windows service has no environment of its own that anything sets, so the verb is there for a script or a deployment tool. Run beside a running server, its values reach that server at the next [reinitialise](/topic/28222) (**Apply now**), or the next start for a key marked `restart`. **The environment wins, every start.** The server records the managed names in the `hm_settings` row `EnvironmentManagedSettings` and refuses an administrator's change to one of them. | Where | What happens | |---|---| | REST | A `PUT` that would change a managed key is `409`, with a sentence naming the variable; `GET /api/v1/settings/environment` lists every managed setting with no value | | COM (Windows) | Every setter refuses with the same sentence | | Control Deck | **Set by the environment** beside the key, with its value and no control | | Control Panel (Windows) | The same mark, with the field disabled | | `hmctl` | `hmctl config changed` prints `(set by the environment)`; `hmctl config export` leaves managed settings out | Unset the variable and restart to manage the setting in the server again. A value a `_FILE` supplied stays withheld after the variable is gone: REST answers `(set)`, the Control Panel leaves the field empty, and it is never on a command line or in a log. The environment sets the **server** level only, and the two `[SendingLimits]` keys have no environment name. **The container's own variables are not settings**: the database's `HM_DB_*`, TLS included; `HM_ADMIN_PASSWORD` and `HM_ADMIN_PASSWORD_FILE`; `HM_INI`; `HM_CREATE_DATABASE` and `HM_UPGRADE_DATABASE`; `HM_PROFILE` and `HM_PROFILE_CONFIRM`. The four REST ones are aliases: `HM_REST_PORT` stands for `HM_SETTING_RestApiPort`. The Helm chart takes `settings`, `properties` and `settingsFrom`. [In a container or on Kubernetes](/topic/28246) has the table, the chart, and the stored-secrets key a first start makes, to export at once. **What has not been run.** The verb has been run as `hMailServer.exe --apply-environment` on Windows only. Its POSIX half compiles; the image's entrypoint as a whole, the POSIX verb against a database, the chart and the Compose file have not been executed. There is no live reload from a ConfigMap change: a restart applies it. ### Profiles: the shape a new server starts in *Since 6.3.4.* A profile is the thirty or so settings that make a server the kind it is, each with a reason written from what the setting's reader does. Four ship as reviewed documents; there is no route that makes one of your own. | Profile | For | What you would notice | |---|---|---| | `office` | One server that receives an office's mail and holds its mailboxes | SPF, the MX check, DKIM and DMARC scoring; greylisting; the quarantine; auto-ban and lockout; a password policy and the breach check (31 entries) | | `gateway` | Behind a filtering gateway that has already judged the mail | Greylisting off; XCLIENT and an ARC seal trusted only from the gateway and sealer you name (17 entries) | | `relay` | An outbound relay that holds no mailbox anybody reads | IMAP and POP3 closed at the next restart; hard bounces suppressed; the trace kept three months (24 entries) | | `sender` | A high-volume sender judged on its bounce and complaint rates | The relay's shape, more deliveries at once, the feedback loops, the trace kept six months (32 entries) | Every surface shows **a diff first**, then saves what it replaces, then writes, all or nothing. An entry that closes a listener, trusts an address range, turns a filter off or weakens a check, or shortens a window of days asks for a confirmation. **A profile never touches** a port, a path, a bind address, the database connection, a password or any secret, an IP range, Incoming relays, a DNS block list, a listener's TLS or a domain's own settings. What only you know, such as the gateway's addresses, is a **step** listed under the diff. **Revert** puts back each value the profile replaced and leaves alone one an administrator changed since. It puts back settings, not data: where a profile shortened a window of days, what the sweep deleted under the shorter window stays deleted, and a listener a profile closed opens again only at the next restart. There is one snapshot: a second apply waits for the first to be reverted or discarded. Where a profile is chosen at install time: - **The Windows installer** offers the four on a page of their own, after the administrator password and before the service-account page, for a new installation only, with "None" chosen until another is. No diff is shown there: each profile's sentence names the listener, filter or trust it changes, and choosing it is the confirmation, so `relay` and `sender` close IMAP and POP3 with no further question. After the database step it runs `hMailServer.exe --apply-profile NAME --once --if-new --confirm` before it starts the service. Silently, that is `/profile=NAME`. Every failure is said, and the installation still succeeds with the shipped settings. - **The container** applies `HM_PROFILE` once, before the server starts; `HM_PROFILE_CONFIRM=yes` confirms an entry that asks, and `HM_PROFILE=none` records that no profile is wanted. - **The verb itself**: ```text hmailserver --apply-profile NAME [--confirm] [--once] [--if-new] hmailserver --revert-profile ``` | Exit | Meaning | |---|---| | 0 | Done, or nothing to do | | 1 | Refused or failed | | 2 | No such profile, or no database | | 3 | An entry needs `--confirm` | | 4 | The service is running | | 5 | `--if-new`, and the server is in service | **On Windows the verbs refuse while the service runs (exit 4).** On Linux nothing says whether a server is running, so run them with the server stopped, as the container's entrypoint does. `--if-new` does nothing on a server of ten accounts or more. A setting the environment manages is skipped, with its variable named. Afterwards a profile is chosen in the Control Panel's **Maintenance, Setting profiles** (Windows), the Deck's **Checklist**, COM's `Settings.ApplyProfile` (Windows), REST or `hmctl`. Read the diff at `GET /api/v1/settings/profiles/{name}`, or with `hmctl settings profile NAME`, which prints a `diff_token`. Apply with `{"confirm": true, "diff_token": ...}` on `POST /api/v1/settings/profiles/{name}/apply`, or `hmctl settings apply-profile NAME` with `--confirm TOKEN`: a diff that has moved since it was read is refused (`409`) and nothing is written. The way back is `POST /api/v1/settings/profiles/revert`; `POST /api/v1/settings/profiles/discard` drops the snapshot. **The installer page with its `/profile=NAME` switch, the container's `HM_PROFILE` and the Linux verbs have been written and read, and not run**: the installer was not built and the Linux server not compiled for this change, and no container was started. ### What it does not do - No package configures the server: no password, no domain, no firewall port. - No export of the configuration as an environment. - Four profiles, no parameters, no profile per domain. ### Where to read more - The wiki pages: [Profiles and Environment](https://gitlab.com/hmailserver/hmailserver/-/wikis/Profiles-and-Environment) and [Running in a Container](https://gitlab.com/hmailserver/hmailserver/-/wikis/Running-in-a-Container) - winget, Chocolatey, apt and dnf: [PackageManagers.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/PackageManagers.md) - The naming rule and every refusal: [EnvironmentConfiguration.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/EnvironmentConfiguration.md) - Every entry of the four profiles: [Profiles.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Profiles.md) - The silent upgrade: [Upgrading.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Upgrading.md) - The documentation page: [Run the installer](https://www.progressiverobot.com/hmailserver-documentation/#42-run-the-installer) - On this forum: [Upgrading to 6.3.4](/topic/28193), [Installing on Windows](/topic/28227) for the wizard's pages, and [Behind a gateway](/topic/28240) for the gateway profile's steps. --- Canonical: https://www.hmailserver.co.uk/topic/28231/unattended-installs-winget-and-chocolatey-and-configuration-from-the-environment # Unattended installs, winget and Chocolatey, and configuration from the environment *Written for 6.3.4. hMailServer 6.3.6 (6 October 2026) publishes the first container image, `registry.gitlab.com/hmailserver/hmailserver:6.3.6`; its installer exits 10 or 11 when a step after its files did not complete (it exited 0), and it makes the data, log, temp, database and event folders private to the administrators and the service. See [the 6.3.6 announcement](/topic/28276). hMailServer 6.3.7 (17 October 2026), a security release, publishes `registry.gitlab.com/hmailserver/hmailserver:6.3.7` (also tagged `6.3` and `latest`); its Windows installers carry the .NET Desktop Runtime 10.0.13, installed only where no .NET 10 Desktop Runtime is installed at all, so an upgraded server keeps the one it had; at every installation and upgrade `/ProtectFolders` hardens the program folder, `Bin` and `ControlPanel`, since an installation in a folder that took its permissions from the root of a drive let any account that could sign in to the computer replace the programs (CVE-2026-107571): before upgrading one, follow `ServiceAccount.md`'s steps, and where other accounts could sign in while it was so installed, reinstall into a new folder only the administrators may change, given to setup with `/DIR=`, since over an installation setup installs into the folder it finds; and setup waits `ShutdownDrainSeconds` plus 90 seconds for the service's stop, where it waited plus 30. Chocolatey and the apt and dnf repositories still publish nothing: no channel is live as 6.3.7 is made, and 6.3.7 is not on the Snap Store. See [the 6.3.7 announcement](/topic/28277).* This topic covers installing hMailServer 6.3.4 without a wizard: the Windows installer's silent switches and what a silent install does with the administrator password; where the `winget` and Chocolatey packages stand; the signed apt and dnf repositories and why they are not live; and, since 6.3.4, every setting from the environment and the four setting profiles, the two ways a deployment shapes a server before it starts. The one thing to know first: **an unattended install leaves the administrator password unset unless you pass it**, and 6.3.4 is installed by hand from the release page, because neither package manager carries it and no [live update](/topic/28234) installs it ([the 6.3.4 announcement](/topic/28218)). ### The Windows installer's switches The installer is Inno Setup, so it takes the standard switches and a few of its own. | Switch | Effect | |---|---| | `/SILENT` | No wizard; progress window only | | `/VERYSILENT` | No wizard and no progress window | | `/SUPPRESSMSGBOXES` | Suppress message boxes; some failure paths still raise one without it | | `/LOG=""` | Write an install log; it is the only record of what happened | | `/DIR=""` | Installation directory | | `/COMPONENTS="server,admintools,controlpanel"` | `server` is the service; `admintools` registers the COM type library; `controlpanel` installs the Control Panel, which does not need `admintools` | | `/adminpassword=` | Custom: sets the administrator password (at least 5 characters) and hands it to the database tools | | `/upgradetoken=` | Custom: the single-use token of a live update; not for people to type | | `/useinternaldbms=true\|false` | Custom: `true` (the default) uses the bundled SQL Server Compact database | | `/serviceaccount=virtual\|localsystem\|keep` | Custom: `virtual` is `NT SERVICE\hMailServer`, the default for a new installation; `keep` is the default for an upgrade | | `/profile=NAME` | Custom, since 6.3.4: the setting profile a new installation applies, or `none`. Written and read, and not run (below) | ```bat hMailServer-x.y.z-x64.exe /VERYSILENT /SUPPRESSMSGBOXES /LOG="C:\Temp\hmail-install.log" ^ /COMPONENTS="server,controlpanel" /useinternaldbms=true ``` **Treat a non-zero exit code as a failed install and read the log.** The installer checks both that the database tool launched and its exit code, so a failed or cancelled schema step fails the install rather than passing silently. The log has the tool's code: from `DBSetupQuick.exe`, 0 is a database created or already up to date, 1 a failed create or a `DBUpdater.exe` that would not start, 3 a wrong or missing administrator password, and any other `DBUpdater.exe`'s own code. A `hm_dbversion` still on the old value afterwards is the same failure. A silent installation does not approve folders or key files the configuration names outside the program folder, and leaves such a service on LocalSystem rather than moving it to the virtual account. ### The administrator password, silently - **A fresh silent install leaves the administrator password unset** unless you pass `/adminpassword=` or set `HM_ADMIN_PASSWORD`: without either, the `AdministratorPassword` key is left out of `hMailServer.ini` entirely. - **Set it afterwards with `hMailServer.exe --set-admin-password`**, new in 6.3.4, from an elevated prompt, then restart the service. It takes the password from `HM_ADMIN_PASSWORD` or, when that is not set, from one line of standard input when that is a pipe or a file, and never from an argument. Prefer the variable: a pipe from Windows PowerShell 5.1 turns every non-ASCII character into `?`, storing a password nobody typed. On Linux the command is `hmailserver --set-admin-password`. [ServiceAccount.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/ServiceAccount.md) and [Upgrading.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Upgrading.md) write the Windows step as `hMailServer.exe /SetAdministratorPassword`. - **The Control Panel and COM are refused under the virtual account.** A new 6.3.4 installation runs the service as `NT SERVICE\hMailServer` by default, and under that account `hMailServer.ini` is read-only to the service: the Control Panel's *Administrator password* card and COM's `Settings.SetAdministratorPassword` are each refused with a sentence. They set the password only where the service may write the file, as it may under LocalSystem, which an upgrade keeps unless it is answered otherwise. - **A silent upgrade of an installation that has a password still needs it** for the database upgrade, and cannot prompt. Without it the database upgrade fails and the installer reports a non-zero exit code rather than waiting. With the password empty, a silent upgrade works. - **Since 6.3.4, `HM_ADMIN_PASSWORD` is the safer of the two.** Inno Setup drops every double quote from its command line, so `/adminpassword=ab"cd` sets `abcd`, and anybody listing processes while setup runs can read the switch. The variable takes any character. Setup reads it only when neither `/adminpassword=` nor `/upgradetoken=` is given. - **Set the variable in an elevated prompt.** Setup started without administrator rights elevates itself through the UAC prompt, and the variable does not cross it. Until 6.3.4 the password went unquoted on the database tools' command lines, so one with a space, a double quote or backslashes before one failed the database step with exit code 3. It now goes in `HM_ADMIN_PASSWORD`. ### winget and Chocolatey: not in the public repositories yet The manifests exist and are validated for each release, from the release's own Authenticode-signed installer. **They are not in the public repositories**: neither `winget install ProgressiveRobot.hMailServer` nor `choco install hmailserver` installs this fork until a first submission has been reviewed and merged. **Do not run `choco install hmailserver` expecting this fork.** [PackageManagers.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/PackageManagers.md) records none on 15 September 2026, but on 27 September 2026 the community feed held an unlisted `hmailserver` package: the original project's 5.6.8, not this fork's. What the packages are built to do, once accepted: - Both run the signed installer the release page offers with `/VERYSILENT /SUPPRESSMSGBOXES /NORESTART`. `/SUPPRESSMSGBOXES` is not optional: without it a database-setup failure shows a message box, silent mode answers it with the default button, and a broken install reports success. - `/TYPE=full` is deliberately not passed, so an upgrade keeps the components already installed and does not put the Control Panel back on a server where it was left out. - `MinimumOSVersion: 10.0.14393.0` is Windows 10 1607, the floor of the .NET 10 Desktop Runtime the Control Panel needs. - **The administrator password is left unset**, and neither package invents one. Set one before the server is reachable from anywhere but the machine itself. - Uninstalling runs the installer's own uninstaller, which does not delete the message store, the database or `hMailServer.ini`. - A pre-release is never submitted. ### apt and dnf on Linux: written, tested in a lab, not live The project keeps signed apt and dnf repositories on the update feed's site, `https://updates.progressiverobot.com`. **They are not live.** The job that makes them, `package-repos`, was tested on the lab machine on 26 September 2026 with a throwaway key and dummy packages, and publishes nothing until the owner has made the signing key. This is what a Debian or Ubuntu user will type once it is live; on a release with no suite, `apt update` answers 404 for its `InRelease`. ```sh sudo install -d -m 0755 /etc/apt/keyrings sudo curl -fsSLo /etc/apt/keyrings/hmailserver.asc https://updates.progressiverobot.com/hmailserver-repository-key.asc gpg --show-keys /etc/apt/keyrings/hmailserver.asc # compare the fingerprint with the one below echo "deb [signed-by=/etc/apt/keyrings/hmailserver.asc] https://updates.progressiverobot.com/apt $(. /etc/os-release && echo "$ID$VERSION_ID") main" \ | sudo tee /etc/apt/sources.list.d/hmailserver.list sudo apt update sudo apt install hmailserver ``` The fingerprint does not exist yet: [PackageManagers.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/PackageManagers.md) records it as `(not made yet)`. **There is no dnf repository at all** until a stable release carries `.rpm` files built for each distribution, and 6.3.4 carries the generic one alone. Once one does, on Fedora and Enterprise Linux it is `sudo curl -fsSLo /etc/yum.repos.d/hmailserver.repo https://updates.progressiverobot.com/rpm/fc44/hmailserver.repo` and then `sudo dnf install hmailserver`, with `fc44` replaced by the release's directory. **What 6.3.4 carries limits what the repositories could hold.** Its Linux packages are one generic x86-64 `.deb` and one generic `.rpm`, neither built for one distribution; [hMailServer on Linux: what you get and what you do not](/topic/28189) says where each installs. In the repositories as [PackageManagers.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/PackageManagers.md) describes them, the `.deb` belongs in the `ubuntu26.04` suite, and the `.rpm`, built on Ubuntu 26.04, goes in no repository. On 26 September 2026 `dnf install` of the `.rpm` built from `master` that day resolved its dependencies on Fedora 43 and 44 and AlmaLinux 10, and was refused on AlmaLinux 9, whose glibc and libstdc++ are older than it asks for. Packages built for each distribution come from the release after 6.3.4. The packages carry no RPM signature of their own: the signed repository metadata carries every package's SHA-256. Not tried: the job on GitLab's runners, HTTPS end to end, arm64 and aarch64. Until it is live, [First install on Linux](/topic/28190) is the walk. ### Every setting from the environment *Since 6.3.4.* Every `[Settings]` key and every `hm_settings` row the server reads has a name in the environment. | Variable | Sets | |---|---| | `HM_SETTING_` | The `[Settings]` key, one of 241: `HM_SETTING_MaxSubmissionsPerIPPerMinute=30` | | `HM_PROPERTY_` | The `hm_settings` row: `HM_PROPERTY_maxmessagesize=20480` | | `HM_SETTING__FILE`, `HM_PROPERTY__FILE` | The same, read from the file the variable names, such as a Kubernetes or Compose Secret; UTF-8, at most 64 KB, trailing line breaks stripped | The prefixes and `_FILE` are written in capitals; the name after the prefix is matched without regard to case, and a key is stored in its own spelling. **What is refused, before anything is stored**: a name that is not a setting of the store its prefix names (`MaxMessageSize` after `HM_SETTING_` is refused naming `HM_PROPERTY_maxmessagesize`); a misspelt prefix such as `HM_SETTINGS_`; a read-only property such as `WebPushVapidPublicKey`; the plain and the `_FILE` form of one setting together; a `_FILE` naming a directory, a missing path or a file over 64 KB, refused naming the path, never the content; the `_FILE` form of a setting a domain or an account may hold its own value of; and a value the setting's own REST check would refuse, in that check's words. `maxmessagesize=off` is refused rather than stored as 0, which is "no limit". **The verb.** ```sh hmailserver --config /etc/hmailserver/hMailServer.ini --apply-environment ``` `hMailServer.exe --apply-environment` on Windows is the same verb from the same code. It opens the configuration and the database as the service does, so run it as the user the service runs as. It **checks every value first and exits 1 having changed nothing** when any is refused, then writes each through the setter `PUT /api/v1/settings/...` uses and prints a line per variable: `set`, `unchanged` or `refused`. A secret's value is never printed. Exit 2 is a configuration file that names no database. The verb's writes are not rows in the audit trail: its own output, in a container the container's log, says what it changed. **Who runs it.** No container image of 6.3.4 is published: the image is one you build yourself, from the Dockerfile in the tree. Its entrypoint runs the verb after `--create-database` and `--upgrade-database` and before the server. **On Windows nothing runs it for you**: a Windows service has no environment of its own that anything sets, so the verb is there for a script or a deployment tool. Run beside a running server, its values reach that server at the next [reinitialise](/topic/28222) (**Apply now**), or the next start for a key marked `restart`. **The environment wins, every start.** The server records the managed names in the `hm_settings` row `EnvironmentManagedSettings` and refuses an administrator's change to one of them. | Where | What happens | |---|---| | REST | A `PUT` that would change a managed key is `409`, with a sentence naming the variable; `GET /api/v1/settings/environment` lists every managed setting with no value | | COM (Windows) | Every setter refuses with the same sentence | | Control Deck | **Set by the environment** beside the key, with its value and no control | | Control Panel (Windows) | The same mark, with the field disabled | | `hmctl` | `hmctl config changed` prints `(set by the environment)`; `hmctl config export` leaves managed settings out | Unset the variable and restart to manage the setting in the server again. A value a `_FILE` supplied stays withheld after the variable is gone: REST answers `(set)`, the Control Panel leaves the field empty, and it is never on a command line or in a log. The environment sets the **server** level only, and the two `[SendingLimits]` keys have no environment name. **The container's own variables are not settings**: the database's `HM_DB_*`, TLS included; `HM_ADMIN_PASSWORD` and `HM_ADMIN_PASSWORD_FILE`; `HM_INI`; `HM_CREATE_DATABASE` and `HM_UPGRADE_DATABASE`; `HM_PROFILE` and `HM_PROFILE_CONFIRM`. The four REST ones are aliases: `HM_REST_PORT` stands for `HM_SETTING_RestApiPort`. The Helm chart takes `settings`, `properties` and `settingsFrom`. [In a container or on Kubernetes](/topic/28246) has the table, the chart, and the stored-secrets key a first start makes, to export at once. **What has not been run.** The verb has been run as `hMailServer.exe --apply-environment` on Windows only. Its POSIX half compiles; the image's entrypoint as a whole, the POSIX verb against a database, the chart and the Compose file have not been executed. There is no live reload from a ConfigMap change: a restart applies it. ### Profiles: the shape a new server starts in *Since 6.3.4.* A profile is the thirty or so settings that make a server the kind it is, each with a reason written from what the setting's reader does. Four ship as reviewed documents; there is no route that makes one of your own. | Profile | For | What you would notice | |---|---|---| | `office` | One server that receives an office's mail and holds its mailboxes | SPF, the MX check, DKIM and DMARC scoring; greylisting; the quarantine; auto-ban and lockout; a password policy and the breach check (31 entries) | | `gateway` | Behind a filtering gateway that has already judged the mail | Greylisting off; XCLIENT and an ARC seal trusted only from the gateway and sealer you name (17 entries) | | `relay` | An outbound relay that holds no mailbox anybody reads | IMAP and POP3 closed at the next restart; hard bounces suppressed; the trace kept three months (24 entries) | | `sender` | A high-volume sender judged on its bounce and complaint rates | The relay's shape, more deliveries at once, the feedback loops, the trace kept six months (32 entries) | Every surface shows **a diff first**, then saves what it replaces, then writes, all or nothing. An entry that closes a listener, trusts an address range, turns a filter off or weakens a check, or shortens a window of days asks for a confirmation. **A profile never touches** a port, a path, a bind address, the database connection, a password or any secret, an IP range, Incoming relays, a DNS block list, a listener's TLS or a domain's own settings. What only you know, such as the gateway's addresses, is a **step** listed under the diff. **Revert** puts back each value the profile replaced and leaves alone one an administrator changed since. It puts back settings, not data: where a profile shortened a window of days, what the sweep deleted under the shorter window stays deleted, and a listener a profile closed opens again only at the next restart. There is one snapshot: a second apply waits for the first to be reverted or discarded. Where a profile is chosen at install time: - **The Windows installer** offers the four on a page of their own, after the administrator password and before the service-account page, for a new installation only, with "None" chosen until another is. No diff is shown there: each profile's sentence names the listener, filter or trust it changes, and choosing it is the confirmation, so `relay` and `sender` close IMAP and POP3 with no further question. After the database step it runs `hMailServer.exe --apply-profile NAME --once --if-new --confirm` before it starts the service. Silently, that is `/profile=NAME`. Every failure is said, and the installation still succeeds with the shipped settings. - **The container** applies `HM_PROFILE` once, before the server starts; `HM_PROFILE_CONFIRM=yes` confirms an entry that asks, and `HM_PROFILE=none` records that no profile is wanted. - **The verb itself**: ```text hmailserver --apply-profile NAME [--confirm] [--once] [--if-new] hmailserver --revert-profile ``` | Exit | Meaning | |---|---| | 0 | Done, or nothing to do | | 1 | Refused or failed | | 2 | No such profile, or no database | | 3 | An entry needs `--confirm` | | 4 | The service is running | | 5 | `--if-new`, and the server is in service | **On Windows the verbs refuse while the service runs (exit 4).** On Linux nothing says whether a server is running, so run them with the server stopped, as the container's entrypoint does. `--if-new` does nothing on a server of ten accounts or more. A setting the environment manages is skipped, with its variable named. Afterwards a profile is chosen in the Control Panel's **Maintenance, Setting profiles** (Windows), the Deck's **Checklist**, COM's `Settings.ApplyProfile` (Windows), REST or `hmctl`. Read the diff at `GET /api/v1/settings/profiles/{name}`, or with `hmctl settings profile NAME`, which prints a `diff_token`. Apply with `{"confirm": true, "diff_token": ...}` on `POST /api/v1/settings/profiles/{name}/apply`, or `hmctl settings apply-profile NAME` with `--confirm TOKEN`: a diff that has moved since it was read is refused (`409`) and nothing is written. The way back is `POST /api/v1/settings/profiles/revert`; `POST /api/v1/settings/profiles/discard` drops the snapshot. **The installer page with its `/profile=NAME` switch, the container's `HM_PROFILE` and the Linux verbs have been written and read, and not run**: the installer was not built and the Linux server not compiled for this change, and no container was started. ### What it does not do - No package configures the server: no password, no domain, no firewall port. - No export of the configuration as an environment. - Four profiles, no parameters, no profile per domain. ### Where to read more - The wiki pages: [Profiles and Environment](https://gitlab.com/hmailserver/hmailserver/-/wikis/Profiles-and-Environment) and [Running in a Container](https://gitlab.com/hmailserver/hmailserver/-/wikis/Running-in-a-Container) - winget, Chocolatey, apt and dnf: [PackageManagers.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/PackageManagers.md) - The naming rule and every refusal: [EnvironmentConfiguration.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/EnvironmentConfiguration.md) - Every entry of the four profiles: [Profiles.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Profiles.md) - The silent upgrade: [Upgrading.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Upgrading.md) - The documentation page: [Run the installer](https://www.progressiverobot.com/hmailserver-documentation/#42-run-the-installer) - On this forum: [Upgrading to 6.3.4](/topic/28193), [Installing on Windows](/topic/28227) for the wizard's pages, and [Behind a gateway](/topic/28240) for the gateway profile's steps. --- Canonical: https://www.hmailserver.co.uk/topic/28232/warm-standby-and-high-availability-what-a-second-machine-can-and-cannot-do # Warm standby and high availability: what a second machine can and cannot do *Written for 6.3.4. hMailServer 6.3.7 (17 October 2026) adds no schema step, so a 6.3.6 node and a 6.3.7 node both require schema 6108 and the version check at start does not tell them apart: upgrade both nodes all the same. On MySQL and MariaDB a lost database connection is reconnected and the statement run again, so the statement after a database restart or a failover no longer fails; and the first units of the active-active work are in the tree, and it is still not built. See [the 6.3.7 announcement](/topic/28277).* This topic covers what hMailServer 6.3.4 offers for a second machine. Two operator documents describe one topology from two angles: [WarmStandby.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/WarmStandby.md), on what has to reach the standby and the constraints that decide the design, and [HighAvailabilityRunbook.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/HighAvailabilityRunbook.md), the active/passive procedure with a virtual IP and a health check. The one thing to know before anything else: **exactly one hMailServer service may run against the database at a time**. There is no clustering code and no active/active mode. Failover is starting a service on the other machine, which takes minutes, and a second service started against the same database while the first still runs delivers mail twice, with no error anywhere. ### Why exactly one service may run The code makes two servers destructive to each other in three ways: - **Startup unlocks the whole queue.** `PersistentMessage::UnlockAll` runs `update hm_messages set messagelocked = 0` for every queued message when the service starts, because locks held by a crashed server would otherwise stall those messages for ever. A second server started against the same database releases every message the first is delivering, and both then deliver them. - **Delivery selection is not coordinated.** The queue is driven by `select ... where messagelocked = 0 and messagenexttrytime <= now`, and the lock is taken after selection, so two servers race that window even after both have started cleanly. - **External fetch has the same shape.** `PersistentFetchAccount::UnlockAll` runs at fetch-manager start, and a POP3 account fetched twice with delete-after-download is mail split between two stores. The rehearsal of 9 September 2026 measured the first. With the primary holding a queued message locked, the standby was started on a second machine: it bound the same ports, because the listeners are rows in `hm_tcpipports` and both servers read them, and the lock went to 0. Nothing warned, on either side. So the standby is *warm*: installed, configured, patched along with the primary, and its service set to **Manual** start and left stopped. [WarmStandby.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/WarmStandby.md) says Manual rather than Disabled, because a disabled service cannot be started by the runbook without an extra step. The HA runbook accepts either in its section 4 and its check-list, though the wiki's copy of it lists Disabled among the ways the pair is got wrong, for the same reason. Use Manual. ### What lives where | State | Lives in | Reaches the standby by | |---|---|---| | Domains, accounts, aliases, lists, rules, routes, IP ranges | The database | Sharing the database | | Message metadata, folders, IMAP state | The database | Sharing the database | | The delivery queue | `hm_messages` rows and spool files on disk | The database and the data directory | | Greylisting triplets, auto-ban ranges, login-failure state | The database | Sharing the database | | Message files (`.eml`) | The data directory | Replication or shared storage, below | | The `[Settings]` section | Since 6.3.4 the database (`hm_inisettings`, schema 6042); the INI section is a cache | Sharing the database | | `[Directories]`, `[Database]`, `[Security]`, `[GUILanguages]` | Each machine's own `hMailServer.ini` | Configured per machine, deliberately | | `[SettingsOverride]` | Each machine's own INI | Not shared, on purpose | | `PasswordPepper` | Each machine's own INI; no database and no backup holds it | Copied by hand | | TLS certificates and keys, DKIM keys | Files on disk; the database stores their paths | The same files at the same paths | | Route, external-fetch and per-domain-relay passwords | The database, DPAPI-protected | They do not travel | | An account's second factor and the `[Settings]` secrets | The database, sealed under the stored-secrets key (schema 6103) | Sharing the database, once the standby holds the key | **The settings store, since 6.3.4.** A standby that shares the database inherits the primary's server settings at its next start, without anything copying INI files. Each setting the primary changed since the standby's file was written is named in the standby's error log as `HM5804` and its line rewritten: that is the copy catching up, not a fault, and it appears once. A stale file can still do two things. A setting **deleted** on the primary whose line the standby's file still carries is adopted back into the shared store when the standby starts, so delete it from the standby's file as well. And a key the standby's file holds that the primary never had is adopted the same way. **Do not copy the primary's INI over the standby's.** On Windows `[Database] Password` is sealed with the machine's own DPAPI key, and the copy breaks the standby's database connection outright. Treat "no `[SettingsOverride]` section on either machine" as part of the check-list: a key in that section is applied over whatever the database holds. **On 6.3.3 it was different.** The wiki records that the `[Settings]` section was not carried by the shared database there, and had to be copied into the other machine's INI after every settings change. The wiki also notes that `[LDAP]`, `[SendingLimits]` and `[SendingLimitsOverrides]` stay in each node's file in 6.3.4, to be kept the same by hand. **Paths must resolve on both machines.** If the primary's row says `C:\certs\mail.pem`, the standby needs that file at that path. Rotated DKIM keys are written into `\DKIM`; a standby promoted with an out-of-date copy of that folder signs with a key file it does not have. ### What does not travel **Passwords the server presents to somebody else.** Route smart-host credentials, external fetch accounts, the per-domain relay password and SSL private-key passphrases are stored protected by Windows DPAPI with `CRYPTPROTECT_LOCAL_MACHINE`, and a blob written by one machine cannot be decrypted by another. After a failover the server keeps running, with local delivery, IMAP, POP3 and unauthenticated relaying untouched, but each such credential fails until it is re-entered on the standby. Failback breaks them the other way. - **Keep the list** of credentialed routes, fetch accounts and relay-configured domains, with the passwords in a password manager. It is between zero and five entries on a typical installation. The wiki has a PowerShell script over COM that lists them; none of its getters returns a password. - **On Linux** the server seals these with a key file, `/.hmailserver-secret-key`. In the rehearsal the route arrived without its password, the standby reported `HM6412`, and the delivery attempt went out unauthenticated. A replication that copies the data directory wholesale copies that file too, and the standby then reads every secret the primary could. Decide which you want; the rehearsal excluded it. **The administrator password.** It lives in `hMailServer.ini`, not in the database. Until it is set on the standby, the standby has no REST API and no COM authentication. On Linux the command is `sudo hmailserver --set-admin-password`. On Windows [WarmStandby.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/WarmStandby.md) names the Control Panel, but a standby installed with 6.3.4 is a new installation, whose service runs as `NT SERVICE\hMailServer` by default, and under that account the Control Panel's password card is refused: use `hMailServer.exe --set-admin-password` from an elevated prompt, which is new in 6.3.4. The rehearsal found this when the standby came up mute. **`PasswordPepper`.** With a different value on the standby, or none, no account whose password is an Argon2id or scrypt hash signs in there. **The stored-secrets key, since 6.3.4.** It is kept in each server's INI as `[Security] StoredSecretsKey`, never in the database, so the standby needs it carried: export it on the primary, elevated, with `hMailServer.exe --export-stored-secrets-key FILE` (`sudo hmailserver ...` on Linux), and import it on the standby with `--import-stored-secrets-key FILE`. If the standby's installer made a key of its own, remove its `[Security] StoredSecretsKey` line first, or the import is refused. Never copy the line itself; it is sealed under the primary's own DPAPI. Without the key every second factor refuses every code on the standby, and SRS, BATV, HS256 tokens and `/metrics` stop working there; the standby says so as `HM7495` at each start. [Hardening a public mail server](/topic/28241) has what the key seals, and what to do on an upgraded server, which has none until you make one. ### The message store: the two documents differ **[WarmStandby.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/WarmStandby.md) says replicate, and do not share.** It argues against putting the data directory on an SMB share or a Cluster Shared Volume: - The accept path flushes the spool file to disk before answering 250. Over SMB that guarantee depends on the file server and on the network between the machines, which is the failure the topology exists for. - Local delivery copies the spool file once per local recipient, and the delivery loop rewrites message files; each becomes network I/O. - Handles and oplocks held by a crashed primary take time to expire, so the standby can find its files locked by a machine that no longer exists. - Cluster Shared Volumes are built for large block-oriented I/O, not millions of small files. It recommends a one-way scheduled `robocopy /MIR` of the data directory every few minutes, or DFS-R or storage-level replication, stopped as part of failover, so that a stale mirror pass from a half-dead primary cannot delete newly accepted mail. Messages accepted after the last sync are rows whose files are missing on the standby. The server treats a missing file as an error on access, not a crash, so a five-minute interval bounds the loss to five minutes of message bodies; metadata, folders and flags lose nothing. The document calls the SMB and CSV reasoning operational judgement built on code facts, not itself a code fact. **[HighAvailabilityRunbook.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/HighAvailabilityRunbook.md) says share.** Both nodes must see the same `[Directories]` `DataFolder` on shared storage (SAN, NAS, a clustered file system or a cloud file share), with identical read and write access for both service accounts, and the database and the store on the same failover boundary. Its validation check-list begins with that. Neither document withdraws the other. Choose knowing what each claims. ### The database Use a server backend, SQL Server, MySQL or MariaDB, or PostgreSQL, with its own availability. **SQL Server Compact is excluded**: it is a single local file. Both machines must run the same build: `REQUIRED_DB_VERSION` is checked at every start and refuses a mismatch in both directions, so a half-upgraded pair fails safe at startup where the two builds require different schemas; 6.3.6 and 6.3.7 both require 6108, so that check does not catch a pair split between them. ### Health probes and the load balancer The probes are on the metrics listener, enabled on both nodes. Since 6.3.4 these are stored settings: set them in the Control Panel (Windows), over `/api/v1/settings/ini`, with `hmctl`, or on a Linux node whose REST listener is not yet on with `hmailserver --set-setting NAME=VALUE`. Then reinitialise (**Apply now**, or `POST /api/v1/server/reinitialize`) or restart. ```text [Settings] MetricsServerPort=8080 MetricsServerBindAddress=0.0.0.0 MetricsServerAuthToken=<32+ random characters> ShutdownDrainSeconds=30 ``` - **Point the health check at `/readyz`.** It answers 200 only when the server is `Running` and the database is connected, and 503 while stopping, while draining, or when the database is lost. The passive node's service is stopped, so its connection is refused and it is never routed to. - `/metrics` answers 503 on a non-loopback bind until `MetricsServerAuthToken` is set; the probes need no credential. - `ShutdownDrainSeconds` defaults to 0, which waits for nothing. Set it. - The drain is `/api/v1/server/drain`, on the REST listener: `GET` its state, `POST` to enter, `DELETE` to leave. [Monitoring and health](/topic/28272) has the three probes and the listener's rules. Two matter here: `MetricsServerBindAddress` takes an IP literal and nothing else, since a host name keeps the listener, and so the probes, from starting; and the listener serves one connection at a time, so no sub-second health check. ### Building the standby 1. Install the same build as the primary, with the same drive letter and directory layout. 2. Point its `[Database]` section at the same database server, verify that it connects, then **stop the service and set it to Manual**. 3. Set up the store: replication from primary to standby, or the shared directory. 4. Record the credential list somewhere that survives the primary. 5. Set the administrator password, copy `PasswordPepper`, and import the stored-secrets key. 6. Rehearse the failover at least once before it is real. ### Planned failover *The drain as a state of its own and `UpgradeOnStart` are new in 6.3.4. On 6.3.3, step 1 is a plain stop with `ShutdownDrainSeconds` set.* 1. **Drain A**: `POST /api/v1/server/drain` (`manage_server`), or the **Drain** button in the Deck's update view. A closes its SMTP, IMAP, POP3 and ManageSieve listeners, pauses its delivery queue, and reports `/readyz` 503 `draining`. A new connection is refused at the socket, which every MTA retries. REST, metrics and web services stay up. Watch `GET /api/v1/server/drain` until `drained` is true. 2. **Stop A.** 3. **Verify A is down**: `/readyz` on A is refused. 4. **Stop the replication job**, in whichever direction it runs, where the store is replicated rather than shared. Do it before B is started. The runbook has no such step, because it assumes a shared store; [WarmStandby.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/WarmStandby.md)'s failover has it. In a planned failover A's machine stays up, so a scheduled mirror pass from it would delete the mail B has just accepted. 5. **Move the VIP** to B. 6. **Start B** and wait for `/readyz` 200. If B's binaries are newer than the shared schema, B refuses to start (`HM5011`) unless its INI says `[Database] UpgradeOnStart=1`. Take the database backup before this step. One node at a time. 7. **Check the key on B**: no `HM7495`, `HM7492` or `HM7493` in its error log. `/readyz` is 200 either way, so this is the check that finds a missing key. 8. **Confirm**: a test message through the VIP, and `/healthz` reporting `database: up` and `state: running`. Then re-enter the route, fetch and relay passwords on B, as after an unplanned failover. Reverse the steps to fail back, the check of the key included; with a replicated store, follow **Failback** below, which resynchronises the store first. To take A off the air without a failover, do step 1 and then `DELETE /api/v1/server/drain`, which reopens the same ports. ### Unplanned failover 1. The health check on `/readyz` fails; stop routing to A. 2. **Fence A**: power it off, or isolate it from the storage and database network. Unreachable is not the same as dead. 3. Stop the replication job, where the store is replicated. 4. Move the VIP, or the DNS record or port forward, to B. 5. Start the service on B. Its startup unlock now works for you: messages the dead primary held mid-delivery are released and retried. A message the primary had delivered but not yet deleted from the queue may be delivered again, which is the known, bounded cost. 6. Re-enter the route, fetch and relay passwords on B. 7. When A is repaired, bring it back stopped, as the new passive node. The two documents order the middle differently: the runbook moves the VIP and then starts B, and [WarmStandby.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/WarmStandby.md) starts the standby and then moves the endpoint. Both put the fence first. Keep MX TTLs modest, 300 to 3600 s, as a standing decision. Sending servers retry for days, so mail queued remotely during the gap is delayed, not lost. **Failback** is the same with the roles swapped, plus one step first where the store is replicated: resynchronise the data directory backwards, because the standby has been accepting mail and the primary's store is now the stale one. Then stop the standby's service, replicate from standby to primary once more for the final delta, start the primary, move the endpoint back, re-enter the route, fetch and relay passwords on the primary, and set up the replication from primary to standby again. ### Verify, after either direction - `GET /readyz` answers 200: the service is running and the database has answered a real round trip within 20 s. - A message sent in from outside, read back over IMAP or POP3. - A message sent out through each credentialed route. - No `ERROR` entries in the application log since the failover. - TLS on 25, 143, 110, 587, 993 and 995 presents the right certificate. ### What it does not do, and what has not been rehearsed - **No active/active, no sharding, no clustering code.** The runbook puts them out of scope. - **Not seconds.** Failover is minutes, and the timings on the wiki pages are the shape of a rehearsed failover, not a promise. - **No rolling restart on one node.** A single node refuses connections for the length of an install. - **The drain's state is per process**, so a pair drains one node at a time. - **The rehearsal was a Linux pair**: an Ubuntu 26.04 standby, one PostgreSQL 18 database, an SSH tunnel standing in for a LAN route. Not rehearsed: a Windows-to-Windows pair, DPAPI itself, a file-replication product rather than a copy at the moment of failure, DNS or IP takeover, and failback. ### Where to read more - The wiki pages: [Warm Standby](https://gitlab.com/hmailserver/hmailserver/-/wikis/Warm-Standby), with the worked commands and the recovery-window table, and [High Availability Runbook](https://gitlab.com/hmailserver/hmailserver/-/wikis/High-Availability-Runbook), with the probe-lifecycle table and load-balancer configurations - The documents: [WarmStandby.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/WarmStandby.md) and [HighAvailabilityRunbook.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/HighAvailabilityRunbook.md) - The documentation page: [A warm standby, active/passive high availability](https://www.progressiverobot.com/hmailserver-documentation/#337-a-warm-standby-activepassive-high-availability) and [Prometheus metrics and health probes](https://www.progressiverobot.com/hmailserver-documentation/#163-prometheus-metrics-and-health-probes) - On this forum: [Upgrading to 6.3.4](/topic/28193), [Moving to new hardware](/topic/28194), and [Settings: what is not stock and when a save takes effect](/topic/28222), for the settings store and `[SettingsOverride]` --- Canonical: https://www.hmailserver.co.uk/topic/28232/warm-standby-and-high-availability-what-a-second-machine-can-and-cannot-do # Warm standby and high availability: what a second machine can and cannot do *Written for 6.3.4. hMailServer 6.3.7 (17 October 2026) adds no schema step, so a 6.3.6 node and a 6.3.7 node both require schema 6108 and the version check at start does not tell them apart: upgrade both nodes all the same. On MySQL and MariaDB a lost database connection is reconnected and the statement run again, so the statement after a database restart or a failover no longer fails; and the first units of the active-active work are in the tree, and it is still not built. See [the 6.3.7 announcement](/topic/28277).* This topic covers what hMailServer 6.3.4 offers for a second machine. Two operator documents describe one topology from two angles: [WarmStandby.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/WarmStandby.md), on what has to reach the standby and the constraints that decide the design, and [HighAvailabilityRunbook.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/HighAvailabilityRunbook.md), the active/passive procedure with a virtual IP and a health check. The one thing to know before anything else: **exactly one hMailServer service may run against the database at a time**. There is no clustering code and no active/active mode. Failover is starting a service on the other machine, which takes minutes, and a second service started against the same database while the first still runs delivers mail twice, with no error anywhere. ### Why exactly one service may run The code makes two servers destructive to each other in three ways: - **Startup unlocks the whole queue.** `PersistentMessage::UnlockAll` runs `update hm_messages set messagelocked = 0` for every queued message when the service starts, because locks held by a crashed server would otherwise stall those messages for ever. A second server started against the same database releases every message the first is delivering, and both then deliver them. - **Delivery selection is not coordinated.** The queue is driven by `select ... where messagelocked = 0 and messagenexttrytime <= now`, and the lock is taken after selection, so two servers race that window even after both have started cleanly. - **External fetch has the same shape.** `PersistentFetchAccount::UnlockAll` runs at fetch-manager start, and a POP3 account fetched twice with delete-after-download is mail split between two stores. The rehearsal of 9 September 2026 measured the first. With the primary holding a queued message locked, the standby was started on a second machine: it bound the same ports, because the listeners are rows in `hm_tcpipports` and both servers read them, and the lock went to 0. Nothing warned, on either side. So the standby is *warm*: installed, configured, patched along with the primary, and its service set to **Manual** start and left stopped. [WarmStandby.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/WarmStandby.md) says Manual rather than Disabled, because a disabled service cannot be started by the runbook without an extra step. The HA runbook accepts either in its section 4 and its check-list, though the wiki's copy of it lists Disabled among the ways the pair is got wrong, for the same reason. Use Manual. ### What lives where | State | Lives in | Reaches the standby by | |---|---|---| | Domains, accounts, aliases, lists, rules, routes, IP ranges | The database | Sharing the database | | Message metadata, folders, IMAP state | The database | Sharing the database | | The delivery queue | `hm_messages` rows and spool files on disk | The database and the data directory | | Greylisting triplets, auto-ban ranges, login-failure state | The database | Sharing the database | | Message files (`.eml`) | The data directory | Replication or shared storage, below | | The `[Settings]` section | Since 6.3.4 the database (`hm_inisettings`, schema 6042); the INI section is a cache | Sharing the database | | `[Directories]`, `[Database]`, `[Security]`, `[GUILanguages]` | Each machine's own `hMailServer.ini` | Configured per machine, deliberately | | `[SettingsOverride]` | Each machine's own INI | Not shared, on purpose | | `PasswordPepper` | Each machine's own INI; no database and no backup holds it | Copied by hand | | TLS certificates and keys, DKIM keys | Files on disk; the database stores their paths | The same files at the same paths | | Route, external-fetch and per-domain-relay passwords | The database, DPAPI-protected | They do not travel | | An account's second factor and the `[Settings]` secrets | The database, sealed under the stored-secrets key (schema 6103) | Sharing the database, once the standby holds the key | **The settings store, since 6.3.4.** A standby that shares the database inherits the primary's server settings at its next start, without anything copying INI files. Each setting the primary changed since the standby's file was written is named in the standby's error log as `HM5804` and its line rewritten: that is the copy catching up, not a fault, and it appears once. A stale file can still do two things. A setting **deleted** on the primary whose line the standby's file still carries is adopted back into the shared store when the standby starts, so delete it from the standby's file as well. And a key the standby's file holds that the primary never had is adopted the same way. **Do not copy the primary's INI over the standby's.** On Windows `[Database] Password` is sealed with the machine's own DPAPI key, and the copy breaks the standby's database connection outright. Treat "no `[SettingsOverride]` section on either machine" as part of the check-list: a key in that section is applied over whatever the database holds. **On 6.3.3 it was different.** The wiki records that the `[Settings]` section was not carried by the shared database there, and had to be copied into the other machine's INI after every settings change. The wiki also notes that `[LDAP]`, `[SendingLimits]` and `[SendingLimitsOverrides]` stay in each node's file in 6.3.4, to be kept the same by hand. **Paths must resolve on both machines.** If the primary's row says `C:\certs\mail.pem`, the standby needs that file at that path. Rotated DKIM keys are written into `\DKIM`; a standby promoted with an out-of-date copy of that folder signs with a key file it does not have. ### What does not travel **Passwords the server presents to somebody else.** Route smart-host credentials, external fetch accounts, the per-domain relay password and SSL private-key passphrases are stored protected by Windows DPAPI with `CRYPTPROTECT_LOCAL_MACHINE`, and a blob written by one machine cannot be decrypted by another. After a failover the server keeps running, with local delivery, IMAP, POP3 and unauthenticated relaying untouched, but each such credential fails until it is re-entered on the standby. Failback breaks them the other way. - **Keep the list** of credentialed routes, fetch accounts and relay-configured domains, with the passwords in a password manager. It is between zero and five entries on a typical installation. The wiki has a PowerShell script over COM that lists them; none of its getters returns a password. - **On Linux** the server seals these with a key file, `/.hmailserver-secret-key`. In the rehearsal the route arrived without its password, the standby reported `HM6412`, and the delivery attempt went out unauthenticated. A replication that copies the data directory wholesale copies that file too, and the standby then reads every secret the primary could. Decide which you want; the rehearsal excluded it. **The administrator password.** It lives in `hMailServer.ini`, not in the database. Until it is set on the standby, the standby has no REST API and no COM authentication. On Linux the command is `sudo hmailserver --set-admin-password`. On Windows [WarmStandby.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/WarmStandby.md) names the Control Panel, but a standby installed with 6.3.4 is a new installation, whose service runs as `NT SERVICE\hMailServer` by default, and under that account the Control Panel's password card is refused: use `hMailServer.exe --set-admin-password` from an elevated prompt, which is new in 6.3.4. The rehearsal found this when the standby came up mute. **`PasswordPepper`.** With a different value on the standby, or none, no account whose password is an Argon2id or scrypt hash signs in there. **The stored-secrets key, since 6.3.4.** It is kept in each server's INI as `[Security] StoredSecretsKey`, never in the database, so the standby needs it carried: export it on the primary, elevated, with `hMailServer.exe --export-stored-secrets-key FILE` (`sudo hmailserver ...` on Linux), and import it on the standby with `--import-stored-secrets-key FILE`. If the standby's installer made a key of its own, remove its `[Security] StoredSecretsKey` line first, or the import is refused. Never copy the line itself; it is sealed under the primary's own DPAPI. Without the key every second factor refuses every code on the standby, and SRS, BATV, HS256 tokens and `/metrics` stop working there; the standby says so as `HM7495` at each start. [Hardening a public mail server](/topic/28241) has what the key seals, and what to do on an upgraded server, which has none until you make one. ### The message store: the two documents differ **[WarmStandby.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/WarmStandby.md) says replicate, and do not share.** It argues against putting the data directory on an SMB share or a Cluster Shared Volume: - The accept path flushes the spool file to disk before answering 250. Over SMB that guarantee depends on the file server and on the network between the machines, which is the failure the topology exists for. - Local delivery copies the spool file once per local recipient, and the delivery loop rewrites message files; each becomes network I/O. - Handles and oplocks held by a crashed primary take time to expire, so the standby can find its files locked by a machine that no longer exists. - Cluster Shared Volumes are built for large block-oriented I/O, not millions of small files. It recommends a one-way scheduled `robocopy /MIR` of the data directory every few minutes, or DFS-R or storage-level replication, stopped as part of failover, so that a stale mirror pass from a half-dead primary cannot delete newly accepted mail. Messages accepted after the last sync are rows whose files are missing on the standby. The server treats a missing file as an error on access, not a crash, so a five-minute interval bounds the loss to five minutes of message bodies; metadata, folders and flags lose nothing. The document calls the SMB and CSV reasoning operational judgement built on code facts, not itself a code fact. **[HighAvailabilityRunbook.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/HighAvailabilityRunbook.md) says share.** Both nodes must see the same `[Directories]` `DataFolder` on shared storage (SAN, NAS, a clustered file system or a cloud file share), with identical read and write access for both service accounts, and the database and the store on the same failover boundary. Its validation check-list begins with that. Neither document withdraws the other. Choose knowing what each claims. ### The database Use a server backend, SQL Server, MySQL or MariaDB, or PostgreSQL, with its own availability. **SQL Server Compact is excluded**: it is a single local file. Both machines must run the same build: `REQUIRED_DB_VERSION` is checked at every start and refuses a mismatch in both directions, so a half-upgraded pair fails safe at startup where the two builds require different schemas; 6.3.6 and 6.3.7 both require 6108, so that check does not catch a pair split between them. ### Health probes and the load balancer The probes are on the metrics listener, enabled on both nodes. Since 6.3.4 these are stored settings: set them in the Control Panel (Windows), over `/api/v1/settings/ini`, with `hmctl`, or on a Linux node whose REST listener is not yet on with `hmailserver --set-setting NAME=VALUE`. Then reinitialise (**Apply now**, or `POST /api/v1/server/reinitialize`) or restart. ```text [Settings] MetricsServerPort=8080 MetricsServerBindAddress=0.0.0.0 MetricsServerAuthToken=<32+ random characters> ShutdownDrainSeconds=30 ``` - **Point the health check at `/readyz`.** It answers 200 only when the server is `Running` and the database is connected, and 503 while stopping, while draining, or when the database is lost. The passive node's service is stopped, so its connection is refused and it is never routed to. - `/metrics` answers 503 on a non-loopback bind until `MetricsServerAuthToken` is set; the probes need no credential. - `ShutdownDrainSeconds` defaults to 0, which waits for nothing. Set it. - The drain is `/api/v1/server/drain`, on the REST listener: `GET` its state, `POST` to enter, `DELETE` to leave. [Monitoring and health](/topic/28272) has the three probes and the listener's rules. Two matter here: `MetricsServerBindAddress` takes an IP literal and nothing else, since a host name keeps the listener, and so the probes, from starting; and the listener serves one connection at a time, so no sub-second health check. ### Building the standby 1. Install the same build as the primary, with the same drive letter and directory layout. 2. Point its `[Database]` section at the same database server, verify that it connects, then **stop the service and set it to Manual**. 3. Set up the store: replication from primary to standby, or the shared directory. 4. Record the credential list somewhere that survives the primary. 5. Set the administrator password, copy `PasswordPepper`, and import the stored-secrets key. 6. Rehearse the failover at least once before it is real. ### Planned failover *The drain as a state of its own and `UpgradeOnStart` are new in 6.3.4. On 6.3.3, step 1 is a plain stop with `ShutdownDrainSeconds` set.* 1. **Drain A**: `POST /api/v1/server/drain` (`manage_server`), or the **Drain** button in the Deck's update view. A closes its SMTP, IMAP, POP3 and ManageSieve listeners, pauses its delivery queue, and reports `/readyz` 503 `draining`. A new connection is refused at the socket, which every MTA retries. REST, metrics and web services stay up. Watch `GET /api/v1/server/drain` until `drained` is true. 2. **Stop A.** 3. **Verify A is down**: `/readyz` on A is refused. 4. **Stop the replication job**, in whichever direction it runs, where the store is replicated rather than shared. Do it before B is started. The runbook has no such step, because it assumes a shared store; [WarmStandby.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/WarmStandby.md)'s failover has it. In a planned failover A's machine stays up, so a scheduled mirror pass from it would delete the mail B has just accepted. 5. **Move the VIP** to B. 6. **Start B** and wait for `/readyz` 200. If B's binaries are newer than the shared schema, B refuses to start (`HM5011`) unless its INI says `[Database] UpgradeOnStart=1`. Take the database backup before this step. One node at a time. 7. **Check the key on B**: no `HM7495`, `HM7492` or `HM7493` in its error log. `/readyz` is 200 either way, so this is the check that finds a missing key. 8. **Confirm**: a test message through the VIP, and `/healthz` reporting `database: up` and `state: running`. Then re-enter the route, fetch and relay passwords on B, as after an unplanned failover. Reverse the steps to fail back, the check of the key included; with a replicated store, follow **Failback** below, which resynchronises the store first. To take A off the air without a failover, do step 1 and then `DELETE /api/v1/server/drain`, which reopens the same ports. ### Unplanned failover 1. The health check on `/readyz` fails; stop routing to A. 2. **Fence A**: power it off, or isolate it from the storage and database network. Unreachable is not the same as dead. 3. Stop the replication job, where the store is replicated. 4. Move the VIP, or the DNS record or port forward, to B. 5. Start the service on B. Its startup unlock now works for you: messages the dead primary held mid-delivery are released and retried. A message the primary had delivered but not yet deleted from the queue may be delivered again, which is the known, bounded cost. 6. Re-enter the route, fetch and relay passwords on B. 7. When A is repaired, bring it back stopped, as the new passive node. The two documents order the middle differently: the runbook moves the VIP and then starts B, and [WarmStandby.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/WarmStandby.md) starts the standby and then moves the endpoint. Both put the fence first. Keep MX TTLs modest, 300 to 3600 s, as a standing decision. Sending servers retry for days, so mail queued remotely during the gap is delayed, not lost. **Failback** is the same with the roles swapped, plus one step first where the store is replicated: resynchronise the data directory backwards, because the standby has been accepting mail and the primary's store is now the stale one. Then stop the standby's service, replicate from standby to primary once more for the final delta, start the primary, move the endpoint back, re-enter the route, fetch and relay passwords on the primary, and set up the replication from primary to standby again. ### Verify, after either direction - `GET /readyz` answers 200: the service is running and the database has answered a real round trip within 20 s. - A message sent in from outside, read back over IMAP or POP3. - A message sent out through each credentialed route. - No `ERROR` entries in the application log since the failover. - TLS on 25, 143, 110, 587, 993 and 995 presents the right certificate. ### What it does not do, and what has not been rehearsed - **No active/active, no sharding, no clustering code.** The runbook puts them out of scope. - **Not seconds.** Failover is minutes, and the timings on the wiki pages are the shape of a rehearsed failover, not a promise. - **No rolling restart on one node.** A single node refuses connections for the length of an install. - **The drain's state is per process**, so a pair drains one node at a time. - **The rehearsal was a Linux pair**: an Ubuntu 26.04 standby, one PostgreSQL 18 database, an SSH tunnel standing in for a LAN route. Not rehearsed: a Windows-to-Windows pair, DPAPI itself, a file-replication product rather than a copy at the moment of failure, DNS or IP takeover, and failback. ### Where to read more - The wiki pages: [Warm Standby](https://gitlab.com/hmailserver/hmailserver/-/wikis/Warm-Standby), with the worked commands and the recovery-window table, and [High Availability Runbook](https://gitlab.com/hmailserver/hmailserver/-/wikis/High-Availability-Runbook), with the probe-lifecycle table and load-balancer configurations - The documents: [WarmStandby.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/WarmStandby.md) and [HighAvailabilityRunbook.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/HighAvailabilityRunbook.md) - The documentation page: [A warm standby, active/passive high availability](https://www.progressiverobot.com/hmailserver-documentation/#337-a-warm-standby-activepassive-high-availability) and [Prometheus metrics and health probes](https://www.progressiverobot.com/hmailserver-documentation/#163-prometheus-metrics-and-health-probes) - On this forum: [Upgrading to 6.3.4](/topic/28193), [Moving to new hardware](/topic/28194), and [Settings: what is not stock and when a save takes effect](/topic/28222), for the settings store and `[SettingsOverride]` --- Canonical: https://www.hmailserver.co.uk/topic/28233/message-storage-compression-de-duplication-object-storage-and-encryption-at-rest # Message storage: compression, de-duplication, object storage and encryption at rest This topic covers the three things 6.3.4 can do to a stored message that 6.3.3 could not: store it compressed with its large attachments held once, keep it in an S3-compatible or Azure Blob store instead of under the data directory, and store it encrypted to a key of the mailbox's own. All three are new in 6.3.4, and **all three are off until an administrator turns them on**: a server that turns none of them on stores every message exactly as 6.3.3 did. The one thing to know first: **the delivery queue is never compressed, encrypted or put in the store**. A queued message is plain RFC 822 on the local disk, so the event scripts, the virus scanners, SpamAssassin, the sandbox and the filter hook read what they always read. ### One door, and what a client sees Every place that reads, sizes, copies, moves or deletes a stored message now goes through one seam, `MessageStore`. Everything that hands a message out reads the original bytes back through it, so a client sees nothing different: IMAP `FETCH BODY[]`, `RFC822.SIZE`, `BODYSTRUCTURE`, `SEARCH`, `SORT` and `THREAD`; POP3 `RETR`, `TOP`, `LIST` and `STAT`; the webmail and the REST API; the rules, Sieve, the quota (which counts the original sizes), and a DKIM signature checked against the stored copy. **What changes is the file itself.** `Message.Filename` over COM still answers the stored file's path. With compression on, a COM client or a script that opens that path reads a container, not RFC 822; with an object store, once the message is committed there is no file there. Read the message through the API instead. ### Compression and attachments held once *Schema 6098.* With `MessageStoreCompression` on, a message delivered to a mailbox, or put there by IMAP `APPEND`, is stored as a *container*: the message compressed with deflate, and each large MIME part, an attachment of at least `MessageStoreDedupMinKB` once decoded, cut out and held once as a *blob* that every message carrying the same bytes shares. A mail sent to forty people on this server with a 10 MB presentation keeps the presentation once. | Where | What | |---|---| | Control Panel (Windows) | *Settings > Maintenance > Performance*, the **Message store** tab | | Control Deck | The two settings on the server settings page, and a **Message store** section on the dashboard | | REST | `message_store_compression` and `message_store_dedup_min_kb` in `/api/v1/settings`; `GET /api/v1/store`; `POST /api/v1/store/measure` | | COM (Windows) | `Settings.MessageStoreCompression`, `Settings.MessageStoreDedupMinKB`, `Settings.MessageStoreStatus` | Both are rows of `hm_settings` and take effect for the next message stored. `MessageStoreDedupMinKB` is 256 unless changed, 0 to 65536, and 0 holds nothing once and compresses only. - **The switch rewrites nothing.** The reader never asks the setting what a file is: it reads either form, per file, by the file's first eight bytes. Turning it off leaves every stored message readable. - **Only delivery and IMAP `APPEND` write the new form.** A webmail draft, a message saved through COM, the importers and the IMAP mirror write plain. - **A message larger than 64 MB is stored as it came**, because a container is decoded whole in memory. - **Every container is checked before a byte is served**: its structure, each blob against its hash, and the SHA-256 of the whole. A damaged one is refused and named in the error log (`HM7350`), never served as something else. - **Two messages share a blob when the part's encoded bytes are identical.** The same file attached twice by a client that wraps its base64 differently is two blobs. **The command line**, run as the user the server runs as (`hMailServer.exe` takes the same switches on Windows): ```text hmailserver --measure-store [--account
| --domain ] hmailserver --compress-store [--account
| --domain ] [--decompress] hmailserver --recount-store ``` | Verb | What it does | Server | |---|---|---| | `--measure-store` | Changes nothing; prints what the store takes now, what it would take, and how much more it would save | May be running | | `--compress-store` | Rewrites every stored file in place, each written beside itself, read back and compared byte for byte, then renamed over its name | Stopped | | `--compress-store --decompress` | Puts the files back as plain RFC 822 | Stopped | | `--recount-store` | Sets every blob's count to what the stored files reference | Stopped | Exit 0 is a finished walk, 1 a file that could not be read or written, 2 no database or no such account or domain, and 4 a running server. **How much it saves has not been measured on a real mailbox.** [MessageStore.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/MessageStore.md) holds a placeholder where the figures are to go, and the wiki claims none. Run `--measure-store` on your own store and read the answer. The document also records that its regression fixture had not yet run against a bench when it was written. ### Object storage: S3 and Azure Blob The messages in the mailboxes, the blobs and the recover-deleted store's kept copies can be kept in an **S3-compatible** store (AWS S3, MinIO, Wasabi, Backblaze B2's S3 API, Ceph RGW), signed with AWS Signature Version 4, or in **Azure Blob Storage**, with Shared Key or a SAS. The database stays the index, and an object is named by the path its file had. There is no schema step. **Nothing changes until a store is named.** **Every write is a local one.** A message is placed on the local disk exactly as without a store, and a **committer** puts it into the store afterwards, and only then removes the local file. It runs every minute; what it has been handed is journalled in `\_ObjectStore\pending.txt` and taken up again after a restart. So **a delivery never waits for the store and never fails for it**, and every read looks at the local disk first. | Property | REST key | Default | |---|---|---| | `MessageStoreBackend` | `message_store_backend` | `file`; the other value is `object` | | `MessageStoreObjectService` | `message_store_object_service` | empty; `s3` or `azure` | | `MessageStoreObjectEndpoint` | `message_store_object_endpoint` | empty, which is the service's own | | `MessageStoreObjectRegion` | `message_store_object_region` | `us-east-1` | | `MessageStoreObjectBucket` | `message_store_object_bucket` | empty | | `MessageStoreObjectAddressing` | `message_store_object_addressing` | `auto`; also `path`, `virtual` | | `MessageStoreObjectPrefix` | `message_store_object_prefix` | empty | | `MessageStoreObjectAccessKey` | `message_store_object_access_key` | empty | | `MessageStoreObjectSecret` | `message_store_object_secret` | empty | | `MessageStoreObjectSasToken` | `message_store_object_sas_token` | empty | | `MessageStoreObjectCaFile` | `message_store_object_ca_file` | empty | | `MessageStoreObjectCacheMB` | `message_store_object_cache_mb` | `32` | These are rows of `hm_settings`, set over `/api/v1/settings` or on the Control Deck's *Settings > Server* form. **There is no COM member and no Control Panel page for them.** The secret and the SAS are protected, never emitted, and left out of a backup, so after a restore onto another server they are entered again. **TLS verification is always on, and no setting turns it off.** With `MessageStoreObjectCaFile` set, the certificate is verified against that file: beside the system's bundle on Linux, and alone on Windows. Turning it on: 1. **Make the bucket or container.** This server makes none. For S3, an access key allowed `s3:GetObject`, `s3:PutObject`, `s3:DeleteObject`, `s3:ListBucket` and the multipart upload calls, and a lifecycle rule that aborts incomplete multipart uploads after a day. 2. **Name it**, with `message_store_backend` still `file`. 3. **Copy the store into it**: `hmailserver --copy-store-to-object-storage`. It runs beside the server, verifies every object, and goes on where it stopped when run again. 4. **Switch new mail to it**: `message_store_backend` to `object`, taken within five seconds. 5. **Remove the local copies** when you are ready: stop the server and run the copy again with `--remove-source`. **Going back** is the same the other way: `message_store_backend` to `file`, then `hmailserver --copy-store-from-object-storage`, then, once it says nothing failed, `message_store_object_service` to empty. **When the store cannot be reached, mail is not lost.** | Door | Answer for a message only the store holds | |---|---| | A delivery | Placed on the local disk and done; committed when the store answers again | | IMAP `FETCH` | `NO [UNAVAILABLE]` | | POP3 `RETR` and `TOP` | `-ERR [SYS/TEMP]` | | The webmail and REST | 503 | `HM7473` says, once an hour at most, how many files wait on the local disk. `HM7471` is the store refusing this server's credentials (`401`, `403`), which is never taken for an absent message. `HM7476` names a message that is neither on the disk nor in the store; no placeholder is written for it while a store is named. **What is thinner than it could be**: one connection per request; one committer, a pass at a time; credentials are the two the settings hold, with no instance role, no workload identity and no Azure AD token; the Import Tool and the data directory synchronizer walk the local disk only; a changed prefix is not a move. The requests were proved against MinIO and Azurite on the project's lab on 25 September 2026. The server's fixture needs that lab, is skipped on a bench without one, and has not run in the Linux suite. ### Mailbox encryption at rest *Schema 6099.* A mailbox whose messages are stored encrypted to an X25519 key of the account's own, with AES-256-GCM, as the container's last stage, after compression. The key opens when the person signs in with their password and closes `MailboxKeyIdleMinutes` after the owner's last use (15 by default, 1 to 1440). Mail that arrives while nobody is signed in is encrypted all the same, because encrypting needs only the key's public half. When a password is lost, the domain's recovery key opens the key again. It is off for every account until an administrator switches it on for that account. **What it protects**: a disk taken out of the machine, a snapshot of the volume, a backup archive, the data directory copied somewhere, the database dumped, once the account has signed in with its password after the switch. **What it does not protect.** It does not protect the mail from the running server, which reads it while the key is open, nor from whoever runs the server, who can capture a password at sign-in. A person who needs that wants a zero-access mailbox, and the two are never on one mailbox. **Until the first sign-in the key is *pending***, held under this machine's own secret: the machine's DPAPI key on Windows, and the key file `.hmailserver-secret-key` inside the data directory on Linux. On Linux a copy of the data directory together with the database therefore opens every pending key. An account that only receives mail and never signs in with its password stays pending. Switching it on, on the Control Deck or over REST: 1. **The domain's recovery key first**: the domain page's *Mailbox recovery key* card, or `PUT /api/v1/domains/{domain}/mailbox-recovery` with `{"passphrase": ..., "hint": ...}`. The passphrase is at least 12 characters, and this server never stores it. Keep it where this server is not. 2. **Then the account**: the *Encryption at rest* card, or `PUT /api/v1/accounts/{address}/mailbox-encryption` with `{"enabled": true}`. **The switch-on ends every session the account had.** A mail client sees its connection drop; the person signs in again with the password. 3. **Then what is already there**: `hmailserver --encrypt-store --account
`, with the server stopped. **Refused, with a sentence**: an account whose domain has no recovery key; a zero-access mailbox; an account that signs in against a Windows directory; and an account with a TOTP second factor enrolled. The last runs both ways: a second factor cannot be enrolled on an account that has a mailbox key. **Sign-in.** IMAP and POP3 sign a keyed mailbox in only with its own password. An app password, an OAuth2 token, a client certificate or a Kerberos ticket is answered `NO [UNAVAILABLE]` (POP3: `-ERR [SYS/PERM]`), and the IMAP master user is refused whether or not the owner's key is open. The webmail refuses a passkey, single sign-on or an app password (403, `mailbox_key`). SMTP, ManageSieve and DAV are not refused, since they do not read the mailbox. **Passwords.** The person's own change wraps the key by the new password. An administrator's reset does so while the key is open or pending, and **is refused while it is closed**, with a sentence naming the recovery: `POST /api/v1/accounts/{address}/mailbox-encryption/recover` with the domain's passphrase and the new password. **A lost password and a lost recovery passphrase together are the mail lost.** Nobody can open it. `DELETE /api/v1/accounts/{address}/mailbox-encryption?confirm=forfeit` removes the key, and with it every message still encrypted to it, for ever. It is refused while a legal hold covers the account. **Switching it off** stores new mail as it comes. What was stored encrypted stays encrypted until `hmailserver --decrypt-store`, run with the server stopped and the domain's passphrase in the environment variable `HMAILSERVER_RECOVERY_PASSPHRASE`, never on the command line. **A keyed mailbox stays in its domain.** A [move to another domain](/topic/28244) is refused (`409`); a rename inside the domain goes ahead. **What stays in the clear**: the queue and the copy a delivery works on, which is overwritten with zeros once placed, though a copy-on-write filesystem, a snapshot or an SSD may not honour that; the calendar and the address book; the quarantine, the archive and journal copies, and a forward; each message's row in the database, with its size, flags, folder and dates; logs and the trace. **An encrypted mailbox holds no part once.** A blob shared between accounts cannot be under one account's key, so every part stays inside the message's own container. The cost is the de-duplication, for these mailboxes only. ### Backups - **A backup with messages** copies the data directory as always, with the containers, the `Blobs` directory and encrypted files as they are. It copies file by file while the server runs, so an archive taken while mail is delivered and deleted can hold a container whose part was written after the `Blobs` directory was copied: take the backup when the server is quiet. With an object store it also reads every message and blob that is only in the store into the archive; a store that cannot be read fails the backup. - **A full restore** rebuilds the blobs' counts from the stored files. `hm_storeblobs` is not in the archive. - **An encrypted mailbox's key** is carried with its account, and the domain's recovery key with its domain. A pending key on Windows opens only on the machine that wrote it; elsewhere the mailbox is opened with the recovery. On Linux it opens wherever the key file goes, and the archive does not carry that file. - **A restore onto a server that never turned compression on** reads the containers the archive brought. ### Known limits in 6.3.4 | Limit | Where it is stated | |---|---| | Encryption at rest is on REST and the Control Deck, **with no webmail card and no Control Panel page**. The person's own view is `GET /api/v1/me/mailbox-encryption` | CHANGELOG, [EncryptionAtRest.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/EncryptionAtRest.md) | | No COM for the switch, the recovery or `MailboxKeyIdleMinutes` | [EncryptionAtRest.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/EncryptionAtRest.md) | | **A search over an encrypted mailbox reads every message**, and works only while its key is open. The full-text index holds none of its words | [EncryptionAtRest.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/EncryptionAtRest.md) | | With compression on, **`Message.Filename` over COM names the stored form**; with an object store it names a path that holds no file once committed | CHANGELOG, [MessageStore.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/MessageStore.md), [ObjectStorage.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/ObjectStorage.md) | | An account's key is never rotated; the domain's recovery key can be replaced | [EncryptionAtRest.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/EncryptionAtRest.md) | | The encryption fixture runs on Windows and is not in the Linux suite | [EncryptionAtRest.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/EncryptionAtRest.md) | | No zstd: deflate is the one stage | [MessageStore.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/MessageStore.md) | ### Where to read more - The wiki page: [Message Storage](https://gitlab.com/hmailserver/hmailserver/-/wikis/Message-Storage) - The documents: [MessageStore.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/MessageStore.md), [ObjectStorage.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/ObjectStorage.md) and [EncryptionAtRest.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/EncryptionAtRest.md), each with its error codes and what proves each claim - Zero-access mailboxes: the wiki's [Encryption for People](https://gitlab.com/hmailserver/hmailserver/-/wikis/Encryption-for-People), and on this forum [End-to-end encryption](/topic/28260) - The documentation page: [Backup and restore](https://www.progressiverobot.com/hmailserver-documentation/#15-backup-and-restore) - On this forum: [The Control Deck](/topic/28192), where the cards and the settings form are, and [Backup and restore](/topic/28221), for what the archive holds and what it leaves out --- Canonical: https://www.hmailserver.co.uk/topic/28233/message-storage-compression-de-duplication-object-storage-and-encryption-at-rest # Message storage: compression, de-duplication, object storage and encryption at rest This topic covers the three things 6.3.4 can do to a stored message that 6.3.3 could not: store it compressed with its large attachments held once, keep it in an S3-compatible or Azure Blob store instead of under the data directory, and store it encrypted to a key of the mailbox's own. All three are new in 6.3.4, and **all three are off until an administrator turns them on**: a server that turns none of them on stores every message exactly as 6.3.3 did. The one thing to know first: **the delivery queue is never compressed, encrypted or put in the store**. A queued message is plain RFC 822 on the local disk, so the event scripts, the virus scanners, SpamAssassin, the sandbox and the filter hook read what they always read. ### One door, and what a client sees Every place that reads, sizes, copies, moves or deletes a stored message now goes through one seam, `MessageStore`. Everything that hands a message out reads the original bytes back through it, so a client sees nothing different: IMAP `FETCH BODY[]`, `RFC822.SIZE`, `BODYSTRUCTURE`, `SEARCH`, `SORT` and `THREAD`; POP3 `RETR`, `TOP`, `LIST` and `STAT`; the webmail and the REST API; the rules, Sieve, the quota (which counts the original sizes), and a DKIM signature checked against the stored copy. **What changes is the file itself.** `Message.Filename` over COM still answers the stored file's path. With compression on, a COM client or a script that opens that path reads a container, not RFC 822; with an object store, once the message is committed there is no file there. Read the message through the API instead. ### Compression and attachments held once *Schema 6098.* With `MessageStoreCompression` on, a message delivered to a mailbox, or put there by IMAP `APPEND`, is stored as a *container*: the message compressed with deflate, and each large MIME part, an attachment of at least `MessageStoreDedupMinKB` once decoded, cut out and held once as a *blob* that every message carrying the same bytes shares. A mail sent to forty people on this server with a 10 MB presentation keeps the presentation once. | Where | What | |---|---| | Control Panel (Windows) | *Settings > Maintenance > Performance*, the **Message store** tab | | Control Deck | The two settings on the server settings page, and a **Message store** section on the dashboard | | REST | `message_store_compression` and `message_store_dedup_min_kb` in `/api/v1/settings`; `GET /api/v1/store`; `POST /api/v1/store/measure` | | COM (Windows) | `Settings.MessageStoreCompression`, `Settings.MessageStoreDedupMinKB`, `Settings.MessageStoreStatus` | Both are rows of `hm_settings` and take effect for the next message stored. `MessageStoreDedupMinKB` is 256 unless changed, 0 to 65536, and 0 holds nothing once and compresses only. - **The switch rewrites nothing.** The reader never asks the setting what a file is: it reads either form, per file, by the file's first eight bytes. Turning it off leaves every stored message readable. - **Only delivery and IMAP `APPEND` write the new form.** A webmail draft, a message saved through COM, the importers and the IMAP mirror write plain. - **A message larger than 64 MB is stored as it came**, because a container is decoded whole in memory. - **Every container is checked before a byte is served**: its structure, each blob against its hash, and the SHA-256 of the whole. A damaged one is refused and named in the error log (`HM7350`), never served as something else. - **Two messages share a blob when the part's encoded bytes are identical.** The same file attached twice by a client that wraps its base64 differently is two blobs. **The command line**, run as the user the server runs as (`hMailServer.exe` takes the same switches on Windows): ```text hmailserver --measure-store [--account
| --domain ] hmailserver --compress-store [--account
| --domain ] [--decompress] hmailserver --recount-store ``` | Verb | What it does | Server | |---|---|---| | `--measure-store` | Changes nothing; prints what the store takes now, what it would take, and how much more it would save | May be running | | `--compress-store` | Rewrites every stored file in place, each written beside itself, read back and compared byte for byte, then renamed over its name | Stopped | | `--compress-store --decompress` | Puts the files back as plain RFC 822 | Stopped | | `--recount-store` | Sets every blob's count to what the stored files reference | Stopped | Exit 0 is a finished walk, 1 a file that could not be read or written, 2 no database or no such account or domain, and 4 a running server. **How much it saves has not been measured on a real mailbox.** [MessageStore.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/MessageStore.md) holds a placeholder where the figures are to go, and the wiki claims none. Run `--measure-store` on your own store and read the answer. The document also records that its regression fixture had not yet run against a bench when it was written. ### Object storage: S3 and Azure Blob The messages in the mailboxes, the blobs and the recover-deleted store's kept copies can be kept in an **S3-compatible** store (AWS S3, MinIO, Wasabi, Backblaze B2's S3 API, Ceph RGW), signed with AWS Signature Version 4, or in **Azure Blob Storage**, with Shared Key or a SAS. The database stays the index, and an object is named by the path its file had. There is no schema step. **Nothing changes until a store is named.** **Every write is a local one.** A message is placed on the local disk exactly as without a store, and a **committer** puts it into the store afterwards, and only then removes the local file. It runs every minute; what it has been handed is journalled in `\_ObjectStore\pending.txt` and taken up again after a restart. So **a delivery never waits for the store and never fails for it**, and every read looks at the local disk first. | Property | REST key | Default | |---|---|---| | `MessageStoreBackend` | `message_store_backend` | `file`; the other value is `object` | | `MessageStoreObjectService` | `message_store_object_service` | empty; `s3` or `azure` | | `MessageStoreObjectEndpoint` | `message_store_object_endpoint` | empty, which is the service's own | | `MessageStoreObjectRegion` | `message_store_object_region` | `us-east-1` | | `MessageStoreObjectBucket` | `message_store_object_bucket` | empty | | `MessageStoreObjectAddressing` | `message_store_object_addressing` | `auto`; also `path`, `virtual` | | `MessageStoreObjectPrefix` | `message_store_object_prefix` | empty | | `MessageStoreObjectAccessKey` | `message_store_object_access_key` | empty | | `MessageStoreObjectSecret` | `message_store_object_secret` | empty | | `MessageStoreObjectSasToken` | `message_store_object_sas_token` | empty | | `MessageStoreObjectCaFile` | `message_store_object_ca_file` | empty | | `MessageStoreObjectCacheMB` | `message_store_object_cache_mb` | `32` | These are rows of `hm_settings`, set over `/api/v1/settings` or on the Control Deck's *Settings > Server* form. **There is no COM member and no Control Panel page for them.** The secret and the SAS are protected, never emitted, and left out of a backup, so after a restore onto another server they are entered again. **TLS verification is always on, and no setting turns it off.** With `MessageStoreObjectCaFile` set, the certificate is verified against that file: beside the system's bundle on Linux, and alone on Windows. Turning it on: 1. **Make the bucket or container.** This server makes none. For S3, an access key allowed `s3:GetObject`, `s3:PutObject`, `s3:DeleteObject`, `s3:ListBucket` and the multipart upload calls, and a lifecycle rule that aborts incomplete multipart uploads after a day. 2. **Name it**, with `message_store_backend` still `file`. 3. **Copy the store into it**: `hmailserver --copy-store-to-object-storage`. It runs beside the server, verifies every object, and goes on where it stopped when run again. 4. **Switch new mail to it**: `message_store_backend` to `object`, taken within five seconds. 5. **Remove the local copies** when you are ready: stop the server and run the copy again with `--remove-source`. **Going back** is the same the other way: `message_store_backend` to `file`, then `hmailserver --copy-store-from-object-storage`, then, once it says nothing failed, `message_store_object_service` to empty. **When the store cannot be reached, mail is not lost.** | Door | Answer for a message only the store holds | |---|---| | A delivery | Placed on the local disk and done; committed when the store answers again | | IMAP `FETCH` | `NO [UNAVAILABLE]` | | POP3 `RETR` and `TOP` | `-ERR [SYS/TEMP]` | | The webmail and REST | 503 | `HM7473` says, once an hour at most, how many files wait on the local disk. `HM7471` is the store refusing this server's credentials (`401`, `403`), which is never taken for an absent message. `HM7476` names a message that is neither on the disk nor in the store; no placeholder is written for it while a store is named. **What is thinner than it could be**: one connection per request; one committer, a pass at a time; credentials are the two the settings hold, with no instance role, no workload identity and no Azure AD token; the Import Tool and the data directory synchronizer walk the local disk only; a changed prefix is not a move. The requests were proved against MinIO and Azurite on the project's lab on 25 September 2026. The server's fixture needs that lab, is skipped on a bench without one, and has not run in the Linux suite. ### Mailbox encryption at rest *Schema 6099.* A mailbox whose messages are stored encrypted to an X25519 key of the account's own, with AES-256-GCM, as the container's last stage, after compression. The key opens when the person signs in with their password and closes `MailboxKeyIdleMinutes` after the owner's last use (15 by default, 1 to 1440). Mail that arrives while nobody is signed in is encrypted all the same, because encrypting needs only the key's public half. When a password is lost, the domain's recovery key opens the key again. It is off for every account until an administrator switches it on for that account. **What it protects**: a disk taken out of the machine, a snapshot of the volume, a backup archive, the data directory copied somewhere, the database dumped, once the account has signed in with its password after the switch. **What it does not protect.** It does not protect the mail from the running server, which reads it while the key is open, nor from whoever runs the server, who can capture a password at sign-in. A person who needs that wants a zero-access mailbox, and the two are never on one mailbox. **Until the first sign-in the key is *pending***, held under this machine's own secret: the machine's DPAPI key on Windows, and the key file `.hmailserver-secret-key` inside the data directory on Linux. On Linux a copy of the data directory together with the database therefore opens every pending key. An account that only receives mail and never signs in with its password stays pending. Switching it on, on the Control Deck or over REST: 1. **The domain's recovery key first**: the domain page's *Mailbox recovery key* card, or `PUT /api/v1/domains/{domain}/mailbox-recovery` with `{"passphrase": ..., "hint": ...}`. The passphrase is at least 12 characters, and this server never stores it. Keep it where this server is not. 2. **Then the account**: the *Encryption at rest* card, or `PUT /api/v1/accounts/{address}/mailbox-encryption` with `{"enabled": true}`. **The switch-on ends every session the account had.** A mail client sees its connection drop; the person signs in again with the password. 3. **Then what is already there**: `hmailserver --encrypt-store --account
`, with the server stopped. **Refused, with a sentence**: an account whose domain has no recovery key; a zero-access mailbox; an account that signs in against a Windows directory; and an account with a TOTP second factor enrolled. The last runs both ways: a second factor cannot be enrolled on an account that has a mailbox key. **Sign-in.** IMAP and POP3 sign a keyed mailbox in only with its own password. An app password, an OAuth2 token, a client certificate or a Kerberos ticket is answered `NO [UNAVAILABLE]` (POP3: `-ERR [SYS/PERM]`), and the IMAP master user is refused whether or not the owner's key is open. The webmail refuses a passkey, single sign-on or an app password (403, `mailbox_key`). SMTP, ManageSieve and DAV are not refused, since they do not read the mailbox. **Passwords.** The person's own change wraps the key by the new password. An administrator's reset does so while the key is open or pending, and **is refused while it is closed**, with a sentence naming the recovery: `POST /api/v1/accounts/{address}/mailbox-encryption/recover` with the domain's passphrase and the new password. **A lost password and a lost recovery passphrase together are the mail lost.** Nobody can open it. `DELETE /api/v1/accounts/{address}/mailbox-encryption?confirm=forfeit` removes the key, and with it every message still encrypted to it, for ever. It is refused while a legal hold covers the account. **Switching it off** stores new mail as it comes. What was stored encrypted stays encrypted until `hmailserver --decrypt-store`, run with the server stopped and the domain's passphrase in the environment variable `HMAILSERVER_RECOVERY_PASSPHRASE`, never on the command line. **A keyed mailbox stays in its domain.** A [move to another domain](/topic/28244) is refused (`409`); a rename inside the domain goes ahead. **What stays in the clear**: the queue and the copy a delivery works on, which is overwritten with zeros once placed, though a copy-on-write filesystem, a snapshot or an SSD may not honour that; the calendar and the address book; the quarantine, the archive and journal copies, and a forward; each message's row in the database, with its size, flags, folder and dates; logs and the trace. **An encrypted mailbox holds no part once.** A blob shared between accounts cannot be under one account's key, so every part stays inside the message's own container. The cost is the de-duplication, for these mailboxes only. ### Backups - **A backup with messages** copies the data directory as always, with the containers, the `Blobs` directory and encrypted files as they are. It copies file by file while the server runs, so an archive taken while mail is delivered and deleted can hold a container whose part was written after the `Blobs` directory was copied: take the backup when the server is quiet. With an object store it also reads every message and blob that is only in the store into the archive; a store that cannot be read fails the backup. - **A full restore** rebuilds the blobs' counts from the stored files. `hm_storeblobs` is not in the archive. - **An encrypted mailbox's key** is carried with its account, and the domain's recovery key with its domain. A pending key on Windows opens only on the machine that wrote it; elsewhere the mailbox is opened with the recovery. On Linux it opens wherever the key file goes, and the archive does not carry that file. - **A restore onto a server that never turned compression on** reads the containers the archive brought. ### Known limits in 6.3.4 | Limit | Where it is stated | |---|---| | Encryption at rest is on REST and the Control Deck, **with no webmail card and no Control Panel page**. The person's own view is `GET /api/v1/me/mailbox-encryption` | CHANGELOG, [EncryptionAtRest.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/EncryptionAtRest.md) | | No COM for the switch, the recovery or `MailboxKeyIdleMinutes` | [EncryptionAtRest.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/EncryptionAtRest.md) | | **A search over an encrypted mailbox reads every message**, and works only while its key is open. The full-text index holds none of its words | [EncryptionAtRest.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/EncryptionAtRest.md) | | With compression on, **`Message.Filename` over COM names the stored form**; with an object store it names a path that holds no file once committed | CHANGELOG, [MessageStore.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/MessageStore.md), [ObjectStorage.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/ObjectStorage.md) | | An account's key is never rotated; the domain's recovery key can be replaced | [EncryptionAtRest.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/EncryptionAtRest.md) | | The encryption fixture runs on Windows and is not in the Linux suite | [EncryptionAtRest.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/EncryptionAtRest.md) | | No zstd: deflate is the one stage | [MessageStore.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/MessageStore.md) | ### Where to read more - The wiki page: [Message Storage](https://gitlab.com/hmailserver/hmailserver/-/wikis/Message-Storage) - The documents: [MessageStore.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/MessageStore.md), [ObjectStorage.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/ObjectStorage.md) and [EncryptionAtRest.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/EncryptionAtRest.md), each with its error codes and what proves each claim - Zero-access mailboxes: the wiki's [Encryption for People](https://gitlab.com/hmailserver/hmailserver/-/wikis/Encryption-for-People), and on this forum [End-to-end encryption](/topic/28260) - The documentation page: [Backup and restore](https://www.progressiverobot.com/hmailserver-documentation/#15-backup-and-restore) - On this forum: [The Control Deck](/topic/28192), where the cards and the settings form are, and [Backup and restore](/topic/28221), for what the archive holds and what it leaves out --- Canonical: https://www.hmailserver.co.uk/topic/28234/the-update-check-and-the-live-update-feed-verification-window-and-rollback # The update check and the live update: feed, verification, window and rollback *Written for 6.3.4. hMailServer 6.3.6 (6 October 2026) is signed by the GitLab pipeline at the group path and installs by the live update from 6.3.4 or 6.3.5; from 6.3.6 the Control Panel installs an update with one click where the service is not an administrator, and on Linux a 6.3.4 or 6.3.5 server installs 6.3.6 with its package manager rather than by the live update, whose apply step on those versions is the one 6.3.6 fixes. See [the 6.3.6 announcement](/topic/28276). hMailServer 6.3.7 (17 October 2026), a security release signed at the same group path, is what the feed now offers: from 6.3.6 the live update installs it, with 6.3.6's file as its rollback image and no schema step between them; on Windows a service running as LocalSystem installs it itself, and under `NT SERVICE\hMailServer` an administrator presses **Install update** on the Control Panel's *Server status* page on the server, one Windows prompt; on Linux 6.3.6's own helper applies it to the generic packages and both AppImages, while a server from a per-distribution package upgrades with its package manager. From 6.3.4 or 6.3.5 what 6.3.6's notes said still holds, and a server whose `UpdateSigningIdentity`, `UpdateSigningIssuer` and `UpdateSourceRepository` still name the project's first path refuses 6.3.7 until the three are cleared, on 6.3.6 by deleting their lines from `[Settings]` in `hMailServer.ini` with an editor run as an administrator (on Linux `sudo hmailserver --set-setting =`) and restarting the service. In 6.3.7's Control Panel, Updates is a page of its own under Maintenance. See [the 6.3.7 announcement](/topic/28277).* This topic covers the update check and the live update as 6.3.4 has them. **The one thing to know first: no live update installs 6.3.4.** It carries an Authenticode signature and no Sigstore bundle, so a server of any version that is offered it says it cannot be verified and downloads nothing. Every server installs 6.3.4 by hand, as [Upgrading to 6.3.4](/topic/28193) describes. A server from 6.2.28 to 6.3.3 has a second reason to do it soon: its live update will refuse 6.3.5, the first release to be signed on GitLab. ### Four steps, and where each is done Each step is separate and opt-in, and only applying runs anything. | Step | Control Panel (Windows) | REST | COM (Windows) | |---|---|---|---| | Check | **Check for updates** on *Server status* | `POST /api/v1/update/check` | `Status.CheckForUpdate` | | Download and verify | **Download update** | `POST /api/v1/update/download` | `Status.DownloadUpdate` | | Apply | **Install update** | `POST /api/v1/update/install` | `Status.InstallUpdate` | | Read the verdict | the Server card of *Server status* | `GET /api/v1/update` | `Status.UpdateState`, `Status.AvailableVersion`, `Status.UpdateLastError`, `Status.UpdateApplyOutcome` | The Control Deck's **Update** view does all four on either platform; on Linux there is no Control Panel and no COM. The state is 0 not checked, 1 up to date, 2 a newer release exists, 3 downloaded and verified, 4 installing, or 5 the last step failed (`lastError` says why). An on-demand check reads the feed whether or not the scheduled check is on. Starting an install from the Control Panel closes it, because the installer replaces its files. ### Turning the check on **`UpdateCheckEnabled` is 0 by default**, because a server must not call out to anyone until its administrator says it may. At 1 the server reads the feed at startup and then every `UpdateCheckHours` (24; below 1 is taken as 1, and the server sets no ceiling although the Control Panel's label says "1 to 168"). `UpdateChannel` is `stable` by default; `prerelease` makes pre-releases eligible. **What it sends.** A plain `GET` of the feed with nothing but the `User-Agent` the server always sends: no identifier, no configuration, no counts. Since 6.3.4 it no longer sends GitHub's `X-GitHub-Api-Version` header. **Where the result shows.** The application log, the Control Panel's *Server status* page, the Deck's **Update** view, COM, and `GET /api/v1/update`, which adds the release's page and date, the installer's address, size and digest, and the last apply's outcome. **Setting the keys since 6.3.4.** Use the **Updates** card of the Control Panel's *API & monitoring* page (in 6.3.7 Updates is a page of its own under *Maintenance*, and Ctrl+K takes the old title to it), `PUT /api/v1/settings/ini/{name}` with `{"value": "..."}` (administrator password only), `hmctl ini set`, or on Linux `hmailserver --set-setting`; the Deck has no editor for these keys. Every key here applies when it is saved, but a value stored with `--set-setting` while the service runs reaches it only at its next save, reinitialise or restart. Do not edit the `[Settings]` lines in `hMailServer.ini`: the database is the settings store, and [Settings: when a save takes effect](/topic/28222) explains what happens to a line edited in the file. ```sh curl -u Administrator:pw -X PUT -H "Content-Type: application/json" -d '{"value":"1"}' http://127.0.0.1:8443/api/v1/settings/ini/UpdateCheckEnabled curl -u Administrator:pw -X POST http://127.0.0.1:8443/api/v1/update/check ``` ### The feed Since 6.3.4 a server reads `https://updates.progressiverobot.com` without being told: `/releases/latest`, the newest stable release, on the stable channel, and `/releases.json`, every release, on the pre-release channel. GitLab Pages serves it from the project's releases. For the rollback the server looks one release up at `/releases/tags/v`. Every address a server builds is a file answered 200, and **the server follows no redirect**. A release has the shape of GitHub's Releases API, and each installer's Sigstore bundle is an asset of its own, named after its file with `.cosign.bundle` added. A download is held to the size and the `digest` the feed gives. **Since 6.3.4, a version with a suffix** (`6.3.5-rc1`, `6.3.5rc1`) is a pre-release whatever the feed's `prerelease` flag says, so the stable channel passes over it. **A mirror.** `UpdateFeedUrl` points the server at another feed, for a network without Internet access; empty is the built-in feed. It must be https unless it is on this machine. It may serve one release or a list, and the rollback's lookup keeps everything up to `/releases` of the value and adds `/tags/v`, so the mirror answers that path too. **A mirror changes where the server looks, not whom it trusts**: every installer is still verified against the signers compiled in. **Through a proxy.** `HttpProxy=host:port` (or `[ipv6]:port`) sends every web request the server makes as a client through a forward proxy: the feed, the installer and its bundle, and the JWKS and token-introspection fetches of OAuth 2. An https address is tunnelled with `CONNECT`, and the certificate is verified against the real host. There is no place for credentials; a proxy that demands them refuses. A refusal reads `The proxy refused CONNECT to : `, and a value without a port fails with `HttpProxy must be host:port (or [ipv6]:port).`, never quietly going direct. ACME and the outbound-relay OAuth2 token request do not use it. ### Download and verification **Download update**, `POST /api/v1/update/download`, `Status.DownloadUpdate` or `UpdateAutoDownload=1` fetches the installer and its bundle into the data directory's `Updates` folder (`Data\Updates` on Windows; on Linux the file is the `.deb` or `.rpm` this server was installed from) and checks them without cosign and without the network: - the file's SHA-256 is the one signed and the one the transparency log recorded; - the certificate chains to Sigstore's Fulcio authority at the moment the log recorded the signature; - the certificate is one of this project's signers, whole; - the signature verifies under the certificate's key, and the log vouches for the entry under its own key. A file that fails is deleted, and the reason is in `Status.UpdateLastError` and the log. Nothing is run. **The two signers, since 6.3.4.** A certificate must match one entry whole. | | The GitHub workflow | The GitLab CI pipeline | |---|---|---| | Signs | Every release from 6.2.19 to 6.3.3 | The releases from 6.3.5 on: 6.3.5 at the path the project began at, 6.3.6 and later at the group path `hmailserver/hmailserver` | | Identity begins | `https://github.com/Progressiverobot/hmailserver/.github/workflows/sign-release.yml@` | `https://gitlab.com/Progressiverobot/hmailserver//.gitlab-ci.yml@refs/tags/v` for 6.3.5; from 6.3.6, `https://gitlab.com/hmailserver/hmailserver//.gitlab-ci.yml@refs/tags/v`, the path the project moved to on 29 September 2026 | | Issuer | `https://token.actions.githubusercontent.com` | `https://gitlab.com` | | Repository id | `1267388815`, required | `86730559`, checked when the certificate carries one | | Held to | Certificates issued before 18 September 2026, and releases up to 6.3.3 | The tag the certificate names | The pipeline is also trusted at the path of the group `hmailserver` the project is to move to, where the id `86730559` is required whatever any setting says. **A server from 6.2.28 to 6.3.3 trusts the GitHub workflow alone.** **Further checks since 6.3.4.** A file signed for one tag is refused as any other release. On Windows the installer's version resource must name the release it is installed as; on Linux the update helper installs nothing that is not an upgrade. On Windows only SYSTEM, the Administrators and the service's account may change the `Updates` folder; one that is a link, owned by anyone else or beyond repair is refused with `HM6894`, and nothing is downloaded or applied: delete it, and the server makes it again. **Authenticode.** `UpdateRequireAuthenticode=1` adds, on Windows only, a check that the installer carries an Authenticode signature Windows trusts (no revocation check, no publisher pin). The installers have been signed since 6.3.1; 1 refuses earlier ones. It is on top of the Sigstore check, which cannot be turned off. **The trust settings** `UpdateTrustRootsFile`, `UpdateLogPublicKeyFile`, `UpdateSigningIdentity`, `UpdateSigningIssuer` and `UpdateSourceRepository` are for a site that signs its own releases. Leave them empty; set, the last three replace their part of every signer. ### Applying an update On Windows, **Install update**, `POST /api/v1/update/install`, `Status.InstallUpdate` or the unattended window does this, in order: 1. Verifies the downloaded installer once more. 2. Fetches and verifies the running version's installer as a rollback image, where the feed still offers it. 3. Issues a single-use token for the database upgrade: 32 random bytes in a file only the machine's administrators can read, good for one hour and one use, given to the installer as `/upgradetoken=` in place of the administrator password. 4. Drains, since 6.3.4: the SMTP, IMAP, POP3 and ManageSieve listeners close at the socket, so senders retry, and the queue pauses. It refuses with `already draining` while somebody else's drain is in progress: leave it, or wait, and apply again. 5. Hands the installer to `hMailServer.Updater.exe`, copied from `Bin` into `Data\Updates` because the installer replaces `Bin`. The helper runs it silently, and only with the SHA-256 the server read; waits `UpdateServiceWaitSeconds` (180) for the service; and if it does not come back, reinstalls the previous version. 6. Reports the outcome: one line, in the log, `Status.UpdateApplyOutcome`, the status page and `GET /api/v1/update`; since 6.3.4, as soon as the service is back. If the helper finishes without stopping the service, the server leaves the drain itself after ten minutes at most (`HM7151` when the helper wrote nothing); the helper's log is `apply-.log` in the `Updates` folder. Set `ShutdownDrainSeconds` (30 is a reasonable figure) so the stop waits for what is in flight. **What a rollback cannot do.** It reinstalls the previous binaries; it cannot take the database back. When the new version had moved the schema and then failed to start, the previous version refuses the newer schema, and the outcome says so. Every release's notes say whether it moves the schema; before one that does, take a backup ([Backup and restore](/topic/28221)). The rollback image is verified like any installer, and 6.3.4's has no bundle. [UpdateFeed.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/UpdateFeed.md) does not say what a 6.3.4 server's first live update does about that, so plan it as one without a rollback image, and take a backup first. In the one case [UpdateFeed.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/UpdateFeed.md) describes, an old server set to the GitLab signer (below), the install runs without a rollback image, the log says `no rollback image ... could be had`, and a new version that does not start has to be reinstalled by hand. **Under `NT SERVICE\hMailServer`**, the account a new 6.3.4 installation on Windows runs as, the apply is refused before anything happens, before the rollback image and the drain: the installer needs an administrator. The update is still checked, downloaded and verified. REST, COM and the Deck say so, and tell an administrator to download the installer from the release page, check its Authenticode signature (`Get-AuthenticodeSignature` reports it `Valid`) and run it; never the copy in `Data\Updates`, which the service can write. An open unattended window logs `HM7463` once per process instead of taking a backup for nothing, and the setup checklist carries `update_apply_by_administrator` while the check is on. An upgraded installation that kept LocalSystem applies as before. [Installing on Windows](/topic/28227) covers the account. **On Linux**, since 6.3.4, the service writes a request to `/run/hmailserver` and the root path unit `hmailserver-update.path` runs `/usr/lib/hmailserver/hmailserver-update`, which verifies the package and the rollback image again as root, installs, and rolls back when the service does not come back. No token is involved: the package's post-install step runs the schema upgrade. A server not under systemd is told so and left with the verified file to install by hand. [The hmailserver command line on Linux](/topic/28245) has the helper in full. Linux builds from 6.3.0 to 6.3.3 answer `This build cannot apply an update`. ### Unattended updates | Key | Default | What it does | |---|---|---| | `UpdateAutoDownload` | 0 | 1 downloads and verifies a newer release's installer as soon as a check finds it | | `UpdateWindow` | empty, never | When a verified update may be applied without a click | | `UpdateBackupBeforeApply` | 1 | An unattended apply backs up to the configured backup destination first, and does not go ahead without a destination or when the backup fails | `UpdateWindow` takes `03:00` (every day from 03:00, for an hour), `Sun 03:00` or `Sat,Sun 02:00-05:00`: English three-letter days, local 24-hour times, and a range that crosses midnight belongs to the day it starts on. An unparsable value is logged once per process and nothing is applied. The backup covers the unattended path only, so take one yourself before an apply you start. Under `NT SERVICE\hMailServer` a window applies nothing (`HM7463`). On Linux: ```sh sudo hmailserver --set-setting UpdateCheckEnabled=1 sudo hmailserver --set-setting UpdateAutoDownload=1 sudo hmailserver --set-setting "UpdateWindow=Sun 03:00" sudo systemctl restart hmailserver ``` ### A server from 6.2.28 to 6.3.3 These versions read GitHub's Releases API, which has answered 404 since GitHub suspended the project's account on 18 September 2026, so their check says `The update feed answered HTTP 404.` Nothing else breaks. Pointed at the new feed, such a server trusts the re-hosted releases up to 6.3.3, which carry GitHub's bundles, but its check now offers 6.3.7, the newest stable release, signed by the GitLab pipeline, which it cannot verify either: its live update installs nothing until 6.3.4 or later is installed by hand. It will also refuse 6.3.5 (`The certificate names https://gitlab.com/Progressiverobot/hmailserver//.gitlab-ci.yml@refs/tags/v..., not this repository's release workflow.`). These versions read `[Settings]` from the file, and a change needs a service restart. 1. **Turn automatic download off.** With `UpdateAutoDownload=1` such a server would download a GitLab-signed installer, about 80 MB, at every check and refuse it every time, because the verification comes after the download. Set it to 0. 2. **Point it at the feed**, with exactly this line under `[Settings]` in `hMailServer.ini`, then restart the service: ```ini [Settings] UpdateFeedUrl=https://updates.progressiverobot.com/releases.json ``` Without `.json` the address is a directory, answered with a redirect, and the check says `The update feed answered HTTP 302.` On Windows set it on the **Updates** card and let the Control Panel restart the service; or add it to `Bin\hMailServer.INI` as an administrator and run `net stop hMailServer` and `net start hMailServer` from an elevated prompt. On Linux: ```sh sudoedit /etc/hmailserver/hMailServer.ini # add the line under [Settings] sudo systemctl restart hmailserver ``` An AppImage keeps it in `$HMAILSERVER_HOME/hMailServer.ini`, or else `$XDG_DATA_HOME/hmailserver/hMailServer.ini`, normally `~/.local/share/hmailserver/hMailServer.ini`; start it again after the edit. 3. **Check.** The answer should be that 6.3.4 is available. Still 404 means the service has not read the line: check the section, the spelling and the restart. `The update feed could not be read: ...` names what stopped it: no route out, a proxy (`HttpProxy`), or a certificate the machine does not trust. 4. **Install 6.3.4 by hand**, following [Upgrading to 6.3.4](/topic/28193), with the backup [Upgrading.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Upgrading.md) describes. The service must have been restarted after your last `[Settings]` change, or 6.3.4's first start treats that change as an edit (`HM5804`) and undoes it. 6.3.4 and 6.3.5 had no AppImage and no AArch64 package; 6.3.6 and 6.3.7 have both again, and such an installation takes the newest release's, 6.3.7's, by hand, once. 5. **Afterwards**, 6.3.4 reads the new feed without being told and trusts both signers. The `UpdateFeedUrl` line can go, although leaving it changes nothing: clear it in the Control Panel, with `DELETE /api/v1/settings/ini/UpdateFeedUrl`, or on Linux with `hmailserver --set-setting UpdateFeedUrl=`, not in the file. Turn `UpdateAutoDownload` back on if you want it. **The other way, Windows only.** [UpdateFeed.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/UpdateFeed.md) gives values for `UpdateSigningIdentity`, `UpdateSigningIssuer` and `UpdateSourceRepository` that let such a server install a GitLab-signed release. While set, they make it refuse every GitHub-signed release, its own rollback image included; they are proved only once a GitLab-signed release exists; and they must stay set until the new version runs, then be cleared as step 5 clears `UpdateFeedUrl`, or the new version refuses what is signed after the project's move. **Never point them at the group's path**: an old server checks no repository id. Installing 6.3.4 by hand avoids all of this. ### The download links, and checking a file by hand `https://updates.progressiverobot.com/download/` always leads to a file of the newest stable release, never a pre-release, and `/download/` to that release's page. GitLab's own `/-/releases/permalink/latest` follows the release published last, a pre-release included. A server does not read these links. `` is a file's name without its version: `hMailServer-x64.exe`, `hMailServer-arm64.exe`, `hmailserver_amd64.deb`, `hmailserver_arm64.deb`, `hmailserver.x86_64.rpm`, `hmailserver.aarch64.rpm`, `hMailServer-x86_64.AppImage` and `hMailServer-aarch64.AppImage`, each with its bundle at the same name plus `.cosign.bundle`. A name the release does not carry answers 404, never another release's file. 6.3.4 carries the x64 installer, the amd64 `.deb` and the x86-64 `.rpm`, and no bundle. **Checking 6.3.4.** With no bundle to check, 6.3.4 is checked by `hmailserver-6.3.4-SHA256SUMS.txt`, whose signature `hmailserver-6.3.4-SHA256SUMS.txt.sig` is made with the key that signs the release tags, and on Windows by the installer's Authenticode signature, which `Get-AuthenticodeSignature` should answer `Valid`, signed by Progressive Robot Ltd. [The 6.3.4 announcement](/topic/28218) has the commands and what each must answer. **Other releases.** Up to 6.3.3, `cosign verify-blob` checks a file against its bundle and the GitHub identity (*Verifying a release* in the Security Policy). From 6.3.5 a release has no `.sig`; its files are signed with Sigstore, and on Linux `hmailserver --verify-update ` checks a download the way the server does. ### What it does not do - It does not install 6.3.4, and does not apply under `NT SERVICE\hMailServer`. - A rollback does not take the database back; a backup does. - It follows no redirect and sends no proxy credentials. - It does not drain a warm-standby pair: the drain is per process, so a pair drains one node at a time ([Warm standby and high availability](/topic/28232)). ### Where to read more - [UpdateFeed.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/UpdateFeed.md), [ServiceAccount.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/ServiceAccount.md), [Upgrading.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Upgrading.md), [SettingsApply.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SettingsApply.md), and the *Update check* section of the [README](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/README.md). - On the wiki: [Upgrading hMailServer](https://gitlab.com/hmailserver/hmailserver/-/wikis/Upgrading-hMailServer), [Settings Reference](https://gitlab.com/hmailserver/hmailserver/-/wikis/Settings-Reference), [The REST API](https://gitlab.com/hmailserver/hmailserver/-/wikis/The-REST-API), [Security Policy](https://gitlab.com/hmailserver/hmailserver/-/wikis/Security-Policy) and [Installing on Linux](https://gitlab.com/hmailserver/hmailserver/-/wikis/Installing-on-Linux). - The documentation page: [Staying up to date](https://www.progressiverobot.com/hmailserver-documentation/#187-staying-up-to-date) and [Self-update](https://www.progressiverobot.com/hmailserver-documentation/#self-update). - On the forum: [Upgrading to 6.3.4](/topic/28193); [Upgrading from 5.x, 6.2 or 6.3](/topic/28235), for what 6.3.4 changes and rolling back by hand; and [The Control Deck](/topic/28192). - Without a GitLab account, a question or a bug can go to the project's Service Desk, `contact-project+hmailserver-hmailserver-86730559-issue-@incoming.gitlab.com`. A suspected vulnerability goes only by the routes of [SECURITY.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/SECURITY.md): a [confidential GitLab issue](https://gitlab.com/hmailserver/hmailserver/-/issues/new?issue[confidential]=true) or that same address, which GitLab files as a confidential issue. --- Canonical: https://www.hmailserver.co.uk/topic/28234/the-update-check-and-the-live-update-feed-verification-window-and-rollback # The update check and the live update: feed, verification, window and rollback *Written for 6.3.4. hMailServer 6.3.6 (6 October 2026) is signed by the GitLab pipeline at the group path and installs by the live update from 6.3.4 or 6.3.5; from 6.3.6 the Control Panel installs an update with one click where the service is not an administrator, and on Linux a 6.3.4 or 6.3.5 server installs 6.3.6 with its package manager rather than by the live update, whose apply step on those versions is the one 6.3.6 fixes. See [the 6.3.6 announcement](/topic/28276). hMailServer 6.3.7 (17 October 2026), a security release signed at the same group path, is what the feed now offers: from 6.3.6 the live update installs it, with 6.3.6's file as its rollback image and no schema step between them; on Windows a service running as LocalSystem installs it itself, and under `NT SERVICE\hMailServer` an administrator presses **Install update** on the Control Panel's *Server status* page on the server, one Windows prompt; on Linux 6.3.6's own helper applies it to the generic packages and both AppImages, while a server from a per-distribution package upgrades with its package manager. From 6.3.4 or 6.3.5 what 6.3.6's notes said still holds, and a server whose `UpdateSigningIdentity`, `UpdateSigningIssuer` and `UpdateSourceRepository` still name the project's first path refuses 6.3.7 until the three are cleared, on 6.3.6 by deleting their lines from `[Settings]` in `hMailServer.ini` with an editor run as an administrator (on Linux `sudo hmailserver --set-setting =`) and restarting the service. In 6.3.7's Control Panel, Updates is a page of its own under Maintenance. See [the 6.3.7 announcement](/topic/28277).* This topic covers the update check and the live update as 6.3.4 has them. **The one thing to know first: no live update installs 6.3.4.** It carries an Authenticode signature and no Sigstore bundle, so a server of any version that is offered it says it cannot be verified and downloads nothing. Every server installs 6.3.4 by hand, as [Upgrading to 6.3.4](/topic/28193) describes. A server from 6.2.28 to 6.3.3 has a second reason to do it soon: its live update will refuse 6.3.5, the first release to be signed on GitLab. ### Four steps, and where each is done Each step is separate and opt-in, and only applying runs anything. | Step | Control Panel (Windows) | REST | COM (Windows) | |---|---|---|---| | Check | **Check for updates** on *Server status* | `POST /api/v1/update/check` | `Status.CheckForUpdate` | | Download and verify | **Download update** | `POST /api/v1/update/download` | `Status.DownloadUpdate` | | Apply | **Install update** | `POST /api/v1/update/install` | `Status.InstallUpdate` | | Read the verdict | the Server card of *Server status* | `GET /api/v1/update` | `Status.UpdateState`, `Status.AvailableVersion`, `Status.UpdateLastError`, `Status.UpdateApplyOutcome` | The Control Deck's **Update** view does all four on either platform; on Linux there is no Control Panel and no COM. The state is 0 not checked, 1 up to date, 2 a newer release exists, 3 downloaded and verified, 4 installing, or 5 the last step failed (`lastError` says why). An on-demand check reads the feed whether or not the scheduled check is on. Starting an install from the Control Panel closes it, because the installer replaces its files. ### Turning the check on **`UpdateCheckEnabled` is 0 by default**, because a server must not call out to anyone until its administrator says it may. At 1 the server reads the feed at startup and then every `UpdateCheckHours` (24; below 1 is taken as 1, and the server sets no ceiling although the Control Panel's label says "1 to 168"). `UpdateChannel` is `stable` by default; `prerelease` makes pre-releases eligible. **What it sends.** A plain `GET` of the feed with nothing but the `User-Agent` the server always sends: no identifier, no configuration, no counts. Since 6.3.4 it no longer sends GitHub's `X-GitHub-Api-Version` header. **Where the result shows.** The application log, the Control Panel's *Server status* page, the Deck's **Update** view, COM, and `GET /api/v1/update`, which adds the release's page and date, the installer's address, size and digest, and the last apply's outcome. **Setting the keys since 6.3.4.** Use the **Updates** card of the Control Panel's *API & monitoring* page (in 6.3.7 Updates is a page of its own under *Maintenance*, and Ctrl+K takes the old title to it), `PUT /api/v1/settings/ini/{name}` with `{"value": "..."}` (administrator password only), `hmctl ini set`, or on Linux `hmailserver --set-setting`; the Deck has no editor for these keys. Every key here applies when it is saved, but a value stored with `--set-setting` while the service runs reaches it only at its next save, reinitialise or restart. Do not edit the `[Settings]` lines in `hMailServer.ini`: the database is the settings store, and [Settings: when a save takes effect](/topic/28222) explains what happens to a line edited in the file. ```sh curl -u Administrator:pw -X PUT -H "Content-Type: application/json" -d '{"value":"1"}' http://127.0.0.1:8443/api/v1/settings/ini/UpdateCheckEnabled curl -u Administrator:pw -X POST http://127.0.0.1:8443/api/v1/update/check ``` ### The feed Since 6.3.4 a server reads `https://updates.progressiverobot.com` without being told: `/releases/latest`, the newest stable release, on the stable channel, and `/releases.json`, every release, on the pre-release channel. GitLab Pages serves it from the project's releases. For the rollback the server looks one release up at `/releases/tags/v`. Every address a server builds is a file answered 200, and **the server follows no redirect**. A release has the shape of GitHub's Releases API, and each installer's Sigstore bundle is an asset of its own, named after its file with `.cosign.bundle` added. A download is held to the size and the `digest` the feed gives. **Since 6.3.4, a version with a suffix** (`6.3.5-rc1`, `6.3.5rc1`) is a pre-release whatever the feed's `prerelease` flag says, so the stable channel passes over it. **A mirror.** `UpdateFeedUrl` points the server at another feed, for a network without Internet access; empty is the built-in feed. It must be https unless it is on this machine. It may serve one release or a list, and the rollback's lookup keeps everything up to `/releases` of the value and adds `/tags/v`, so the mirror answers that path too. **A mirror changes where the server looks, not whom it trusts**: every installer is still verified against the signers compiled in. **Through a proxy.** `HttpProxy=host:port` (or `[ipv6]:port`) sends every web request the server makes as a client through a forward proxy: the feed, the installer and its bundle, and the JWKS and token-introspection fetches of OAuth 2. An https address is tunnelled with `CONNECT`, and the certificate is verified against the real host. There is no place for credentials; a proxy that demands them refuses. A refusal reads `The proxy refused CONNECT to : `, and a value without a port fails with `HttpProxy must be host:port (or [ipv6]:port).`, never quietly going direct. ACME and the outbound-relay OAuth2 token request do not use it. ### Download and verification **Download update**, `POST /api/v1/update/download`, `Status.DownloadUpdate` or `UpdateAutoDownload=1` fetches the installer and its bundle into the data directory's `Updates` folder (`Data\Updates` on Windows; on Linux the file is the `.deb` or `.rpm` this server was installed from) and checks them without cosign and without the network: - the file's SHA-256 is the one signed and the one the transparency log recorded; - the certificate chains to Sigstore's Fulcio authority at the moment the log recorded the signature; - the certificate is one of this project's signers, whole; - the signature verifies under the certificate's key, and the log vouches for the entry under its own key. A file that fails is deleted, and the reason is in `Status.UpdateLastError` and the log. Nothing is run. **The two signers, since 6.3.4.** A certificate must match one entry whole. | | The GitHub workflow | The GitLab CI pipeline | |---|---|---| | Signs | Every release from 6.2.19 to 6.3.3 | The releases from 6.3.5 on: 6.3.5 at the path the project began at, 6.3.6 and later at the group path `hmailserver/hmailserver` | | Identity begins | `https://github.com/Progressiverobot/hmailserver/.github/workflows/sign-release.yml@` | `https://gitlab.com/Progressiverobot/hmailserver//.gitlab-ci.yml@refs/tags/v` for 6.3.5; from 6.3.6, `https://gitlab.com/hmailserver/hmailserver//.gitlab-ci.yml@refs/tags/v`, the path the project moved to on 29 September 2026 | | Issuer | `https://token.actions.githubusercontent.com` | `https://gitlab.com` | | Repository id | `1267388815`, required | `86730559`, checked when the certificate carries one | | Held to | Certificates issued before 18 September 2026, and releases up to 6.3.3 | The tag the certificate names | The pipeline is also trusted at the path of the group `hmailserver` the project is to move to, where the id `86730559` is required whatever any setting says. **A server from 6.2.28 to 6.3.3 trusts the GitHub workflow alone.** **Further checks since 6.3.4.** A file signed for one tag is refused as any other release. On Windows the installer's version resource must name the release it is installed as; on Linux the update helper installs nothing that is not an upgrade. On Windows only SYSTEM, the Administrators and the service's account may change the `Updates` folder; one that is a link, owned by anyone else or beyond repair is refused with `HM6894`, and nothing is downloaded or applied: delete it, and the server makes it again. **Authenticode.** `UpdateRequireAuthenticode=1` adds, on Windows only, a check that the installer carries an Authenticode signature Windows trusts (no revocation check, no publisher pin). The installers have been signed since 6.3.1; 1 refuses earlier ones. It is on top of the Sigstore check, which cannot be turned off. **The trust settings** `UpdateTrustRootsFile`, `UpdateLogPublicKeyFile`, `UpdateSigningIdentity`, `UpdateSigningIssuer` and `UpdateSourceRepository` are for a site that signs its own releases. Leave them empty; set, the last three replace their part of every signer. ### Applying an update On Windows, **Install update**, `POST /api/v1/update/install`, `Status.InstallUpdate` or the unattended window does this, in order: 1. Verifies the downloaded installer once more. 2. Fetches and verifies the running version's installer as a rollback image, where the feed still offers it. 3. Issues a single-use token for the database upgrade: 32 random bytes in a file only the machine's administrators can read, good for one hour and one use, given to the installer as `/upgradetoken=` in place of the administrator password. 4. Drains, since 6.3.4: the SMTP, IMAP, POP3 and ManageSieve listeners close at the socket, so senders retry, and the queue pauses. It refuses with `already draining` while somebody else's drain is in progress: leave it, or wait, and apply again. 5. Hands the installer to `hMailServer.Updater.exe`, copied from `Bin` into `Data\Updates` because the installer replaces `Bin`. The helper runs it silently, and only with the SHA-256 the server read; waits `UpdateServiceWaitSeconds` (180) for the service; and if it does not come back, reinstalls the previous version. 6. Reports the outcome: one line, in the log, `Status.UpdateApplyOutcome`, the status page and `GET /api/v1/update`; since 6.3.4, as soon as the service is back. If the helper finishes without stopping the service, the server leaves the drain itself after ten minutes at most (`HM7151` when the helper wrote nothing); the helper's log is `apply-.log` in the `Updates` folder. Set `ShutdownDrainSeconds` (30 is a reasonable figure) so the stop waits for what is in flight. **What a rollback cannot do.** It reinstalls the previous binaries; it cannot take the database back. When the new version had moved the schema and then failed to start, the previous version refuses the newer schema, and the outcome says so. Every release's notes say whether it moves the schema; before one that does, take a backup ([Backup and restore](/topic/28221)). The rollback image is verified like any installer, and 6.3.4's has no bundle. [UpdateFeed.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/UpdateFeed.md) does not say what a 6.3.4 server's first live update does about that, so plan it as one without a rollback image, and take a backup first. In the one case [UpdateFeed.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/UpdateFeed.md) describes, an old server set to the GitLab signer (below), the install runs without a rollback image, the log says `no rollback image ... could be had`, and a new version that does not start has to be reinstalled by hand. **Under `NT SERVICE\hMailServer`**, the account a new 6.3.4 installation on Windows runs as, the apply is refused before anything happens, before the rollback image and the drain: the installer needs an administrator. The update is still checked, downloaded and verified. REST, COM and the Deck say so, and tell an administrator to download the installer from the release page, check its Authenticode signature (`Get-AuthenticodeSignature` reports it `Valid`) and run it; never the copy in `Data\Updates`, which the service can write. An open unattended window logs `HM7463` once per process instead of taking a backup for nothing, and the setup checklist carries `update_apply_by_administrator` while the check is on. An upgraded installation that kept LocalSystem applies as before. [Installing on Windows](/topic/28227) covers the account. **On Linux**, since 6.3.4, the service writes a request to `/run/hmailserver` and the root path unit `hmailserver-update.path` runs `/usr/lib/hmailserver/hmailserver-update`, which verifies the package and the rollback image again as root, installs, and rolls back when the service does not come back. No token is involved: the package's post-install step runs the schema upgrade. A server not under systemd is told so and left with the verified file to install by hand. [The hmailserver command line on Linux](/topic/28245) has the helper in full. Linux builds from 6.3.0 to 6.3.3 answer `This build cannot apply an update`. ### Unattended updates | Key | Default | What it does | |---|---|---| | `UpdateAutoDownload` | 0 | 1 downloads and verifies a newer release's installer as soon as a check finds it | | `UpdateWindow` | empty, never | When a verified update may be applied without a click | | `UpdateBackupBeforeApply` | 1 | An unattended apply backs up to the configured backup destination first, and does not go ahead without a destination or when the backup fails | `UpdateWindow` takes `03:00` (every day from 03:00, for an hour), `Sun 03:00` or `Sat,Sun 02:00-05:00`: English three-letter days, local 24-hour times, and a range that crosses midnight belongs to the day it starts on. An unparsable value is logged once per process and nothing is applied. The backup covers the unattended path only, so take one yourself before an apply you start. Under `NT SERVICE\hMailServer` a window applies nothing (`HM7463`). On Linux: ```sh sudo hmailserver --set-setting UpdateCheckEnabled=1 sudo hmailserver --set-setting UpdateAutoDownload=1 sudo hmailserver --set-setting "UpdateWindow=Sun 03:00" sudo systemctl restart hmailserver ``` ### A server from 6.2.28 to 6.3.3 These versions read GitHub's Releases API, which has answered 404 since GitHub suspended the project's account on 18 September 2026, so their check says `The update feed answered HTTP 404.` Nothing else breaks. Pointed at the new feed, such a server trusts the re-hosted releases up to 6.3.3, which carry GitHub's bundles, but its check now offers 6.3.7, the newest stable release, signed by the GitLab pipeline, which it cannot verify either: its live update installs nothing until 6.3.4 or later is installed by hand. It will also refuse 6.3.5 (`The certificate names https://gitlab.com/Progressiverobot/hmailserver//.gitlab-ci.yml@refs/tags/v..., not this repository's release workflow.`). These versions read `[Settings]` from the file, and a change needs a service restart. 1. **Turn automatic download off.** With `UpdateAutoDownload=1` such a server would download a GitLab-signed installer, about 80 MB, at every check and refuse it every time, because the verification comes after the download. Set it to 0. 2. **Point it at the feed**, with exactly this line under `[Settings]` in `hMailServer.ini`, then restart the service: ```ini [Settings] UpdateFeedUrl=https://updates.progressiverobot.com/releases.json ``` Without `.json` the address is a directory, answered with a redirect, and the check says `The update feed answered HTTP 302.` On Windows set it on the **Updates** card and let the Control Panel restart the service; or add it to `Bin\hMailServer.INI` as an administrator and run `net stop hMailServer` and `net start hMailServer` from an elevated prompt. On Linux: ```sh sudoedit /etc/hmailserver/hMailServer.ini # add the line under [Settings] sudo systemctl restart hmailserver ``` An AppImage keeps it in `$HMAILSERVER_HOME/hMailServer.ini`, or else `$XDG_DATA_HOME/hmailserver/hMailServer.ini`, normally `~/.local/share/hmailserver/hMailServer.ini`; start it again after the edit. 3. **Check.** The answer should be that 6.3.4 is available. Still 404 means the service has not read the line: check the section, the spelling and the restart. `The update feed could not be read: ...` names what stopped it: no route out, a proxy (`HttpProxy`), or a certificate the machine does not trust. 4. **Install 6.3.4 by hand**, following [Upgrading to 6.3.4](/topic/28193), with the backup [Upgrading.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Upgrading.md) describes. The service must have been restarted after your last `[Settings]` change, or 6.3.4's first start treats that change as an edit (`HM5804`) and undoes it. 6.3.4 and 6.3.5 had no AppImage and no AArch64 package; 6.3.6 and 6.3.7 have both again, and such an installation takes the newest release's, 6.3.7's, by hand, once. 5. **Afterwards**, 6.3.4 reads the new feed without being told and trusts both signers. The `UpdateFeedUrl` line can go, although leaving it changes nothing: clear it in the Control Panel, with `DELETE /api/v1/settings/ini/UpdateFeedUrl`, or on Linux with `hmailserver --set-setting UpdateFeedUrl=`, not in the file. Turn `UpdateAutoDownload` back on if you want it. **The other way, Windows only.** [UpdateFeed.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/UpdateFeed.md) gives values for `UpdateSigningIdentity`, `UpdateSigningIssuer` and `UpdateSourceRepository` that let such a server install a GitLab-signed release. While set, they make it refuse every GitHub-signed release, its own rollback image included; they are proved only once a GitLab-signed release exists; and they must stay set until the new version runs, then be cleared as step 5 clears `UpdateFeedUrl`, or the new version refuses what is signed after the project's move. **Never point them at the group's path**: an old server checks no repository id. Installing 6.3.4 by hand avoids all of this. ### The download links, and checking a file by hand `https://updates.progressiverobot.com/download/` always leads to a file of the newest stable release, never a pre-release, and `/download/` to that release's page. GitLab's own `/-/releases/permalink/latest` follows the release published last, a pre-release included. A server does not read these links. `` is a file's name without its version: `hMailServer-x64.exe`, `hMailServer-arm64.exe`, `hmailserver_amd64.deb`, `hmailserver_arm64.deb`, `hmailserver.x86_64.rpm`, `hmailserver.aarch64.rpm`, `hMailServer-x86_64.AppImage` and `hMailServer-aarch64.AppImage`, each with its bundle at the same name plus `.cosign.bundle`. A name the release does not carry answers 404, never another release's file. 6.3.4 carries the x64 installer, the amd64 `.deb` and the x86-64 `.rpm`, and no bundle. **Checking 6.3.4.** With no bundle to check, 6.3.4 is checked by `hmailserver-6.3.4-SHA256SUMS.txt`, whose signature `hmailserver-6.3.4-SHA256SUMS.txt.sig` is made with the key that signs the release tags, and on Windows by the installer's Authenticode signature, which `Get-AuthenticodeSignature` should answer `Valid`, signed by Progressive Robot Ltd. [The 6.3.4 announcement](/topic/28218) has the commands and what each must answer. **Other releases.** Up to 6.3.3, `cosign verify-blob` checks a file against its bundle and the GitHub identity (*Verifying a release* in the Security Policy). From 6.3.5 a release has no `.sig`; its files are signed with Sigstore, and on Linux `hmailserver --verify-update ` checks a download the way the server does. ### What it does not do - It does not install 6.3.4, and does not apply under `NT SERVICE\hMailServer`. - A rollback does not take the database back; a backup does. - It follows no redirect and sends no proxy credentials. - It does not drain a warm-standby pair: the drain is per process, so a pair drains one node at a time ([Warm standby and high availability](/topic/28232)). ### Where to read more - [UpdateFeed.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/UpdateFeed.md), [ServiceAccount.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/ServiceAccount.md), [Upgrading.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Upgrading.md), [SettingsApply.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SettingsApply.md), and the *Update check* section of the [README](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/README.md). - On the wiki: [Upgrading hMailServer](https://gitlab.com/hmailserver/hmailserver/-/wikis/Upgrading-hMailServer), [Settings Reference](https://gitlab.com/hmailserver/hmailserver/-/wikis/Settings-Reference), [The REST API](https://gitlab.com/hmailserver/hmailserver/-/wikis/The-REST-API), [Security Policy](https://gitlab.com/hmailserver/hmailserver/-/wikis/Security-Policy) and [Installing on Linux](https://gitlab.com/hmailserver/hmailserver/-/wikis/Installing-on-Linux). - The documentation page: [Staying up to date](https://www.progressiverobot.com/hmailserver-documentation/#187-staying-up-to-date) and [Self-update](https://www.progressiverobot.com/hmailserver-documentation/#self-update). - On the forum: [Upgrading to 6.3.4](/topic/28193); [Upgrading from 5.x, 6.2 or 6.3](/topic/28235), for what 6.3.4 changes and rolling back by hand; and [The Control Deck](/topic/28192). - Without a GitLab account, a question or a bug can go to the project's Service Desk, `contact-project+hmailserver-hmailserver-86730559-issue-@incoming.gitlab.com`. A suspected vulnerability goes only by the routes of [SECURITY.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/SECURITY.md): a [confidential GitLab issue](https://gitlab.com/hmailserver/hmailserver/-/issues/new?issue[confidential]=true) or that same address, which GitLab files as a confidential issue. --- Canonical: https://www.hmailserver.co.uk/topic/28235/upgrading-from-5.x-6.2-or-6.3-the-24-changes-without-a-switch-and-rolling-back # Upgrading from 5.x, 6.2 or 6.3: the 24 changes without a switch, and rolling back *Written for 6.3.4. hMailServer 6.3.6 (6 October 2026) adds three one-way schema steps (6106 to 6108), changes more without a switch (DANE lookups that do not come back whole, external accounts' days to keep messages, the COM helpers, Windows folder permissions, the REST listener's Host) and installs by the live update from 6.3.4 or 6.3.5: read [the 6.3.6 announcement](/topic/28276) before upgrading. hMailServer 6.3.7 (17 October 2026), a security release, adds no schema step (6108 stays, so the live update's rollback from it to 6.3.6 meets no newer schema) and installs by the live update from 6.3.4, 6.3.5 or 6.3.6, with the exceptions its announcement lists; it too changes what an upgraded server does without a switch: an authenticated submission is DKIM-signed as another hosted domain only when the account may send as the From address; a domain's or an account's rule may no longer set From, Sender, Resent-From, Resent-Sender, DKIM-Signature or the loop count, nor a Sieve script add or delete them or send notify from an arbitrary :from; RuleLoopLimit 0 stops out-of-office replies; the serveradmin role with a list of domains is confined to those domains over COM; forwards of outside mail take an envelope sender in the hosted domain only on a DMARC pass; the anti-virus action that removes attachments replaces a single-part attachment message with its note and deletes unreadable mail; SASL PLAIN acting as another account is refused; on Windows `/ProtectFolders` hardens the program folder, `Bin` and `ControlPanel`, the x64 installer's MariaDB Connector/C 3.4.11 drops MySQL Enterprise's Windows authentication, and an upgraded server keeps the .NET 10 Desktop Runtime it had, where the installers carry 10.0.13. See [the 6.3.7 announcement](/topic/28277).* This topic covers what comes after the installer in an upgrade to 6.3.4 (27 September 2026, schema 6104): the 24 changes [CHANGELOG.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/CHANGELOG.md) lists under "What changes on upgrade without a switch", with what to do about each; what else differs coming from 5.x or early 6.x; what the server says when the schema and the server do not match; and how to go back. The path, the slow steps and the backup are in [Upgrading to 6.3.4](/topic/28193). **Know this first: an upgrade cannot undo itself.** An older server refuses a newer schema rather than misread it, so the database backup taken before the installer ran is the only way back. Going back from 6.3.4 can also need the `hMailServer.ini` you had before the upgrade, as *Rolling back from 6.3.4* below explains. Keep a copy of both. ### The 24 changes, and what to do about each Numbered as [CHANGELOG.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/CHANGELOG.md) numbers them; all are since 6.3.4. | # | What changes | What to do | Detail | |---|---|---|---| | 1 | The database is the settings store (schema 6042); `[Settings]` in `hMailServer.ini` is its copy. A line edited in the file is put back and named as `HM5804` | Do not edit the file to prepare the upgrade; restart the old service once after your last change. Afterwards use the Control Panel, COM (`Settings.SetIniSetting`), REST (`PUT /api/v1/settings/ini/{name}`) or, on Linux, `hmailserver --set-setting NAME=VALUE` | [Upgrading to 6.3.4](/topic/28193); [Settings](/topic/28222) | | 2 | 211 keys apply when saved, 28 at a reinitialise, 2 at a restart | A script that counted on nothing happening until the next start now sees its change at once | [Settings: when a save takes effect](/topic/28222) | | 3 | SPF follows RFC 7208: 127.0.0.0/8 and `::1` no longer pass before any policy is read. SPF now works on Linux, where every check had been TempError | A content filter that re-injects mail authenticates, or connects from an IP range with anti-spam off | [The anti-spam stack](/topic/28248) | | 4 | The audit trail is on (schema 6043), kept until a retention is set; alerts are on, but nothing is mailed until `AlertRecipient` and `AlertSenderAddress` are set | Set `AuditRetentionDays`, and the two addresses | [Alerts and the audit trail](/topic/28273) | | 5 | Compromised-account detection starts in watch mode (schema 6071): it raises `account.compromised` and holds nothing | Set `compromise_detection_mode` to `enforce` when you want a takeover held | [Hardening: compromised-account detection](/topic/28241) | | 6 | A 421, or a provider's rate-limit 4xx, pauses that destination for `OutboundBackoffMinutes` (2, doubling to 60) | Nothing; 0 turns it off | [Outbound shaping](/topic/28259) | | 7 | A backup carries what an account keeps beyond its mail: address book, preferences, calendars, S/MIME keys, keywords and more | Expect larger archives. One from 6.3.3 or earlier restores as it did, without them | [Backup and restore](/topic/28221) | | 8 | The out-of-office memory is a table (schema 6060) | Nothing: a restart no longer sends everyone the reply again | [Rules and Sieve: vacation replies](/topic/28268) | | 9 | Collected addresses are suggestions under Other contacts; CardDAV serves only kept contacts | Tell users a phone no longer receives collected addresses; **Keep** makes one a contact | [Calendars and contacts](/topic/28252) | | 10 | In the container image, `HM_SETTING_` and `HM_PROPERTY_` are stored at every start, and the setting is refused to the other interfaces | Remove the variable and start once without it. No container image of 6.3.4 is published | [In a container](/topic/28246) | | 11 | `GET /api/v1/me/search` goes on from where it stopped | Below | Below | | 12 | IMAP answers as the RFCs say: `NO` rather than `BAD` for a missing mailbox, `\Seen` set by every `BODY[
]` without `.PEEK`, SORT and THREAD ordering, APPEND byte for byte | Test clients and scripts that relied on the old answers | [Connecting a mail client](/topic/28255) | | 13 | MySQL and MariaDB tables become `utf8mb4` (schema 6102), every table rebuilt | MySQL 5.7.9 or MariaDB 10.2.2 or later, the ALTER privilege on the database, free disk of the order of the largest table | [Upgrading to 6.3.4](/topic/28193); [Databases on Linux: the upgrade floor](/topic/28247) | | 14 | Schema 6101 adds two columns to `hm_messages` | On a large store, time it on a copy first | [Upgrading to 6.3.4](/topic/28193) | | 15 | The database connection is encrypted when the database server offers TLS, for MySQL and MariaDB without checking the certificate; PostgreSQL is unchanged | Read the application log at the first start, or run `--check-database`; `TlsMode` and `TlsCaFile` in `[Database]` choose otherwise | [Choosing a database: encrypting the link](/topic/28228) | | 16 | The administrator password is hashed again at the first successful administrator sign-in; a browser session made under the old hash ends once | Sign in once. A file the service cannot write keeps the old value (`HM7494`) | [Hardening a public mail server](/topic/28241) | | 17 | A COM change to a route, an incoming relay, a DNS black list, an SSL certificate or five more kinds of shared object takes effect at `Save()` | Add `Save()` where a script relied on setting a property; the Control Panel is not affected | [The COM API on Windows: Save()](/topic/28262) | | 18 | Mail labelled ISO-8859-1 is decoded as windows-1252 | Below | Below | | 19 | A restore refuses an unsafe archive whole (`HM7540`) | Nothing: no backup this server writes has such an entry | [Backup and restore](/topic/28221) | | 20 | A rule's Run function action must name a plain function: otherwise `HM5018` on Windows, and on Linux nothing runs and the ERROR log has a Script Error line | Rename such functions to ASCII letters, digits, `_` and `$`, not starting with a digit | [Event scripts: the 6.3.4 security fix](/topic/28261); [the advisory](/topic/28218) | | 21 | A domain can no longer be named as one of the server's own directories | Below | Below | | 22 | A one-time code is taken once, over COM and REST: a TOTP code the server has accepted is refused if it comes again within its window, but not counted as a wrong code, so it locks nobody out | A script that sent the administrator's password and a code with every REST request now has only its first request in each half minute accepted, the others refused with 401: sign it in once with `POST /api/v1/session` and use the `hmailsession` cookie, or give it an API key | [The REST API: one-time codes](/topic/28264) | | 23 | The outbound relay sends its OAuth2 token over TLS only: a route or the SMTP relay whose host is on `OutboundOAuth2Hosts` and whose connection security is none, or STARTTLS optional against a server that does not offer it, fails with a temporary `450 4.7.0` (`HM5912`), and the message waits for the route's retries | Set that connection security to TLS or STARTTLS required. A route to Exchange Online on 587, set up as Microsoft documents it, is not affected | [Routes, relays and smart hosts: OAuth2](/topic/28269) | | 24 | A hash stored before 6.3.4 of a password with a character above U+00FF is refused | Set such passwords again, and soon | [Upgrading to 6.3.4](/topic/28193); [Hardening a public mail server](/topic/28241) | ### Three that no table row can hold **ISO-8859 (18).** ISO-8859-1 is decoded as windows-1252, as every mail client reads it, and ISO-8859-2 to -9, -13 and -15 by their own code pages. There was no ISO-8859 name in the server's table at all, so a Western Windows server decoded Latin-1 right by accident and decodes it as before, one with another ANSI code page now decodes it right too, and on Linux Latin-1 is no longer turned into replacement characters. There is nothing to set. **Reserved domain names (21).** A domain's mail is kept in the directory named after it under the data directory, beside the server's own stores. So a single-label name such as `protected`, `confidential`, `recoverable`, `discovery`, `quarantine`, `files`, `lists` or `blobs`, in any case, or `temp`, `events` and `database`, which the Linux packages put there, is refused for a new domain and for a rename over COM (`Domain.Save`) and REST, and so in the Deck. The Control Panel already refused every name without a dot. A domain that already has such a name is still saved, so that it can be managed, and is named as `HM7597` in the error log at every start and as `domain_name_reserved` on the first-hour checklist. **Rename it**: the rename moves each of its accounts' mailboxes and leaves the store's files where they are. Until then, a domain named `recoverable`, `discovery`, `confidential` or `protected` has its mail in a folder the backup leaves out. **The search route (11).** Only a client of `GET /api/v1/me/search` other than the webmail needs to change. A search with more hits than `limit` answered `complete: true, more: true` and could not reach the rest; it now answers `complete: false, more: true` with `next_before_id`, which `before_id` continues from. Hits come newest first across every folder by message id, where they came folder by folder ([Performance.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Performance.md)). ### Coming from 5.x or early 6.x The 24 apply to you too, and so do the changes of every release in between: section 5 of [Changes Since 6.2.10](https://gitlab.com/hmailserver/hmailserver/-/wikis/Changes-Since-6210) lists what came without a switch from 6.2.10 to 6.2.28. On 5.3 to 5.5, back up first and test the restore. Four things are particular to an older installation, all on Windows: - **The Administrator is replaced by the Control Panel.** `hMailAdmin.exe` was retired in 6.2. The Control Panel, `hMailCP.exe`, does everything it did, and its navigation tree mirrors the old layout. Your TOTP secret carries over. - **The PHP WebAdmin is gone.** Administer a remote server with the Control Panel, which connects to it directly, or from a browser with [the Control Deck](/topic/28192). The "Administrative tools" component now only registers the COM type library for scripts on another machine. - **A pre-5 `hMailServer.ini` moves into `Bin`.** The installer copies it from the Windows directory to `Bin\hMailServer.ini`, leaves a `.old` copy where it was and deletes the original; the server reads only the one in `Bin`. - **The .NET 10 runtime.** Since the first release after 6.2.18 the Control Panel and the setup tools (`DBSetup`, `DBSetupQuick`, `DBUpdater`) need the .NET 10 Desktop Runtime. The installer bundles `windowsdesktop-runtime-10.0-win-x64.exe` and installs it `/quiet /norestart` when it is missing (6.3.7's installers carry 10.0.13 and install it only where no .NET 10 Desktop Runtime is installed at all: an upgraded server keeps the one it had, 10.0.12 where 6.3.4 to 6.3.6 installed it, and brings it to 10.0.13 or later through Microsoft Update or Microsoft's installer). **If the runtime does not install.** [Upgrading.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Upgrading.md) says the server, native code with no .NET dependency, is left working without the Control Panel. The documentation's [18.5](https://www.progressiverobot.com/hmailserver-documentation/#185-things-that-changed-and-what-to-do-about-them) says otherwise, and for an upgrade to 6.3.4, which always moves the schema, it is right: [Upgrading.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Upgrading.md) itself says the setup tools need the runtime too, and the server refuses to start against an old schema, so there is no mail server until the schema is moved. Install the runtime (x64) by hand from `https://dotnet.microsoft.com/download/dotnet/10.0`, run `DBUpdater.exe` from the installation's `Bin` folder, then start the service. ### When it goes wrong At every start the server compares the version in `hm_dbversion` with the one it requires and refuses to run if they differ, in either direction. The ERROR log has one of these: | Code and message | What it means | What to do | |---|---|---| | `HM5011`: `The database is too old for this version of hMailServer. Please run hMailServer Database updater (DBUpdater.exe) to upgrade it.` | The schema did not get all the way up | Run `DBUpdater.exe` from the installation's `Bin` folder | | `HM5011`: `The database is too new for this version of hMailServer. Please upgrade hMailServer.` | An older server against a newer schema: an old binary left in place, or a previous release reinstalled | Install the matching or a newer server. Do not try to downgrade the database | | `HM5010`: `Database version could not be detected.` | `hm_dbversion` could not be read at all | Check the connection details in `[Database]` and that the login can read the table | Both `HM5011` messages end with `Database version: , Required database version: `. On Windows the service reports itself running before this check, so a refused start leaves a service Windows calls Running with nothing listening: read the ERROR log. A 6.3.3 server against a 6.3.4 database is the too-new case on purpose: step 6041 to 6042 changes only the version, the fence that keeps a 6.3.3 server, which lets `hMailServer.INI` win, away from a database that is now the settings store. **Resuming.** DBUpdater picks up from whichever version the database actually reports, so a retry after a partial upgrade resumes rather than restarting. A failed step leaves PostgreSQL exactly as it was and SQL Server normally so; MySQL, MariaDB and SQL Server Compact commit DDL as it executes and keep the steps that ran. The same chain also runs: - on Windows, since 6.3.4, as `hMailServer.exe --upgrade-database` with the service stopped, without DBUpdater's window or the administrator password; it exits 0 (done, or nothing to do), 1 (failed; the output names the step) or 2 (no `[Database]` configured); - on Linux as `hmailserver --upgrade-database`, which the `.deb` and `.rpm` run after installing; - at the service's own start with `UpgradeOnStart=1` in `[Database]` (since 6.3.4, off by default): a failing step refuses the start with `HM5011` naming the step, and the next start resumes there. It backs nothing up, and belongs on one node at a time ([Warm standby](/topic/28232)). DBUpdater exits 0 on success, 1 when the upgrade failed, 2 on a configuration error, and 3 when authentication was cancelled or, under `/silent`, could not be satisfied: see `HM_ADMIN_PASSWORD` in [Unattended installs](/topic/28231). The installer fails with a non-zero exit code of its own; treat that, or a schema version still on the old value, as the failure: ```sql select value from hm_dbversion; ``` **Asking for help.** Give the versions you started from, the step DBUpdater stopped on and the database's own error number; [Read this before you post](/topic/28185) has the rest. Without a GitLab account, email the Service Desk address, [contact-project+hmailserver-hmailserver-86730559-issue-@incoming.gitlab.com](mailto:contact-project+hmailserver-hmailserver-86730559-issue-@incoming.gitlab.com): GitLab files it as a confidential issue, and replies come by email. ### Rolling back from 6.3.4 1. **Uninstall 6.3.4.** 2. **Restore the pre-upgrade database backup.** This is the essential step: the schema is at 6104, and the release you go back to refuses it. 3. **Put the right `hMailServer.ini` in place before the older installer runs**, because its `/Register` reads the file. Which file depends on whether 6.3.4 sealed the stored secrets, that is, whether a stored-secrets key was made: - **Sealed: restore the pre-upgrade `hMailServer.ini`.** 6.3.4 writes the `[Settings]` secrets into the file as `HMK1:` envelopes, and an older server reads them as they are: its `/Register` hands `ServiceAccountPassword`'s envelope to Windows as the service account's password, and the service cannot log on. Its first moments also run SRS, BATV and OAuth2 on the envelopes. - **Not sealed: the 6.3.4 file can stay.** Its `[Settings]` section is a complete copy of the stored settings, and an older server reads the file as the truth, so a setting changed after the upgrade survives, wherever the server could write the file, which a Linux package installation's unit does not allow. Delete the `[SettingsStore]` section, which older builds ignore. They ignore `[SettingsOverride]` too, so move any value only it supplied into `[Settings]`. 4. **Check `PasswordPepper`.** Since 6.3.4 it is kept in the file only, never sealed and in no backup 6.3.4 writes. The file the older server starts with must carry the value the hashes were made with: no pepper, or a different one, refuses every account hashed with Argon2id or scrypt. PBKDF2, the default, is not peppered. 5. **Install the release you came from.** 6. **Sealed, with no pre-upgrade file?** Set `ServiceAccountPassword`, and any other of those secrets you use, again with the older version's Control Panel, then run `hMailServer.exe /Register` again. The wiki's and the documentation's 18.6 reinstall before they restore the database; [Upgrading.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Upgrading.md) restores first. Follow [Upgrading.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Upgrading.md): the older installer runs DBUpdater, which refuses a database newer than its own build, and its `/Register` reads the file. **The data directory normally stays.** [Upgrading.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Upgrading.md) and the wiki's 18.6 say it does not normally need restoring, because no upgrade step rewrites message files; the documentation's 18.6 says to restore it with `hMailServer.INI`. Left in place, mail that arrived after the backup exists on disk with no metadata row in the restored database, and is invisible. That is what a rollback costs. ### What it does not do - The rollback above is the Windows one. Chapter 18 and [Upgrading.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Upgrading.md) give no package command for going back on Linux; the database rule is the same. - No backup carries a `[SettingsOverride]` value; it lives only in the file of the machine that has it. - Moving to another machine is not an upgrade: [Moving to new hardware](/topic/28194). ### Where to read more - [Upgrading hMailServer](https://gitlab.com/hmailserver/hmailserver/-/wikis/Upgrading-hMailServer), 18.5 and 18.6; the documentation's [18.6](https://www.progressiverobot.com/hmailserver-documentation/#186-rolling-back) - [Upgrading.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Upgrading.md) and its wiki copy, [Upgrading Guide](https://gitlab.com/hmailserver/hmailserver/-/wikis/Upgrading-Guide) - [CHANGELOG.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/CHANGELOG.md), 6.3.4, and [Changes Since 6.2.10](https://gitlab.com/hmailserver/hmailserver/-/wikis/Changes-Since-6210), section 5 - [SecretsAtRest.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SecretsAtRest.md), "Rolling back", and [Settings Overview](https://gitlab.com/hmailserver/hmailserver/-/wikis/Settings-Overview) - [SetupChecklist.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SetupChecklist.md) and [AccountRecovery.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/AccountRecovery.md), for items 21 and 22 - [Troubleshooting](https://gitlab.com/hmailserver/hmailserver/-/wikis/Troubleshooting), for a start that fails --- Canonical: https://www.hmailserver.co.uk/topic/28235/upgrading-from-5.x-6.2-or-6.3-the-24-changes-without-a-switch-and-rolling-back # Upgrading from 5.x, 6.2 or 6.3: the 24 changes without a switch, and rolling back *Written for 6.3.4. hMailServer 6.3.6 (6 October 2026) adds three one-way schema steps (6106 to 6108), changes more without a switch (DANE lookups that do not come back whole, external accounts' days to keep messages, the COM helpers, Windows folder permissions, the REST listener's Host) and installs by the live update from 6.3.4 or 6.3.5: read [the 6.3.6 announcement](/topic/28276) before upgrading. hMailServer 6.3.7 (17 October 2026), a security release, adds no schema step (6108 stays, so the live update's rollback from it to 6.3.6 meets no newer schema) and installs by the live update from 6.3.4, 6.3.5 or 6.3.6, with the exceptions its announcement lists; it too changes what an upgraded server does without a switch: an authenticated submission is DKIM-signed as another hosted domain only when the account may send as the From address; a domain's or an account's rule may no longer set From, Sender, Resent-From, Resent-Sender, DKIM-Signature or the loop count, nor a Sieve script add or delete them or send notify from an arbitrary :from; RuleLoopLimit 0 stops out-of-office replies; the serveradmin role with a list of domains is confined to those domains over COM; forwards of outside mail take an envelope sender in the hosted domain only on a DMARC pass; the anti-virus action that removes attachments replaces a single-part attachment message with its note and deletes unreadable mail; SASL PLAIN acting as another account is refused; on Windows `/ProtectFolders` hardens the program folder, `Bin` and `ControlPanel`, the x64 installer's MariaDB Connector/C 3.4.11 drops MySQL Enterprise's Windows authentication, and an upgraded server keeps the .NET 10 Desktop Runtime it had, where the installers carry 10.0.13. See [the 6.3.7 announcement](/topic/28277).* This topic covers what comes after the installer in an upgrade to 6.3.4 (27 September 2026, schema 6104): the 24 changes [CHANGELOG.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/CHANGELOG.md) lists under "What changes on upgrade without a switch", with what to do about each; what else differs coming from 5.x or early 6.x; what the server says when the schema and the server do not match; and how to go back. The path, the slow steps and the backup are in [Upgrading to 6.3.4](/topic/28193). **Know this first: an upgrade cannot undo itself.** An older server refuses a newer schema rather than misread it, so the database backup taken before the installer ran is the only way back. Going back from 6.3.4 can also need the `hMailServer.ini` you had before the upgrade, as *Rolling back from 6.3.4* below explains. Keep a copy of both. ### The 24 changes, and what to do about each Numbered as [CHANGELOG.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/CHANGELOG.md) numbers them; all are since 6.3.4. | # | What changes | What to do | Detail | |---|---|---|---| | 1 | The database is the settings store (schema 6042); `[Settings]` in `hMailServer.ini` is its copy. A line edited in the file is put back and named as `HM5804` | Do not edit the file to prepare the upgrade; restart the old service once after your last change. Afterwards use the Control Panel, COM (`Settings.SetIniSetting`), REST (`PUT /api/v1/settings/ini/{name}`) or, on Linux, `hmailserver --set-setting NAME=VALUE` | [Upgrading to 6.3.4](/topic/28193); [Settings](/topic/28222) | | 2 | 211 keys apply when saved, 28 at a reinitialise, 2 at a restart | A script that counted on nothing happening until the next start now sees its change at once | [Settings: when a save takes effect](/topic/28222) | | 3 | SPF follows RFC 7208: 127.0.0.0/8 and `::1` no longer pass before any policy is read. SPF now works on Linux, where every check had been TempError | A content filter that re-injects mail authenticates, or connects from an IP range with anti-spam off | [The anti-spam stack](/topic/28248) | | 4 | The audit trail is on (schema 6043), kept until a retention is set; alerts are on, but nothing is mailed until `AlertRecipient` and `AlertSenderAddress` are set | Set `AuditRetentionDays`, and the two addresses | [Alerts and the audit trail](/topic/28273) | | 5 | Compromised-account detection starts in watch mode (schema 6071): it raises `account.compromised` and holds nothing | Set `compromise_detection_mode` to `enforce` when you want a takeover held | [Hardening: compromised-account detection](/topic/28241) | | 6 | A 421, or a provider's rate-limit 4xx, pauses that destination for `OutboundBackoffMinutes` (2, doubling to 60) | Nothing; 0 turns it off | [Outbound shaping](/topic/28259) | | 7 | A backup carries what an account keeps beyond its mail: address book, preferences, calendars, S/MIME keys, keywords and more | Expect larger archives. One from 6.3.3 or earlier restores as it did, without them | [Backup and restore](/topic/28221) | | 8 | The out-of-office memory is a table (schema 6060) | Nothing: a restart no longer sends everyone the reply again | [Rules and Sieve: vacation replies](/topic/28268) | | 9 | Collected addresses are suggestions under Other contacts; CardDAV serves only kept contacts | Tell users a phone no longer receives collected addresses; **Keep** makes one a contact | [Calendars and contacts](/topic/28252) | | 10 | In the container image, `HM_SETTING_` and `HM_PROPERTY_` are stored at every start, and the setting is refused to the other interfaces | Remove the variable and start once without it. No container image of 6.3.4 is published | [In a container](/topic/28246) | | 11 | `GET /api/v1/me/search` goes on from where it stopped | Below | Below | | 12 | IMAP answers as the RFCs say: `NO` rather than `BAD` for a missing mailbox, `\Seen` set by every `BODY[
]` without `.PEEK`, SORT and THREAD ordering, APPEND byte for byte | Test clients and scripts that relied on the old answers | [Connecting a mail client](/topic/28255) | | 13 | MySQL and MariaDB tables become `utf8mb4` (schema 6102), every table rebuilt | MySQL 5.7.9 or MariaDB 10.2.2 or later, the ALTER privilege on the database, free disk of the order of the largest table | [Upgrading to 6.3.4](/topic/28193); [Databases on Linux: the upgrade floor](/topic/28247) | | 14 | Schema 6101 adds two columns to `hm_messages` | On a large store, time it on a copy first | [Upgrading to 6.3.4](/topic/28193) | | 15 | The database connection is encrypted when the database server offers TLS, for MySQL and MariaDB without checking the certificate; PostgreSQL is unchanged | Read the application log at the first start, or run `--check-database`; `TlsMode` and `TlsCaFile` in `[Database]` choose otherwise | [Choosing a database: encrypting the link](/topic/28228) | | 16 | The administrator password is hashed again at the first successful administrator sign-in; a browser session made under the old hash ends once | Sign in once. A file the service cannot write keeps the old value (`HM7494`) | [Hardening a public mail server](/topic/28241) | | 17 | A COM change to a route, an incoming relay, a DNS black list, an SSL certificate or five more kinds of shared object takes effect at `Save()` | Add `Save()` where a script relied on setting a property; the Control Panel is not affected | [The COM API on Windows: Save()](/topic/28262) | | 18 | Mail labelled ISO-8859-1 is decoded as windows-1252 | Below | Below | | 19 | A restore refuses an unsafe archive whole (`HM7540`) | Nothing: no backup this server writes has such an entry | [Backup and restore](/topic/28221) | | 20 | A rule's Run function action must name a plain function: otherwise `HM5018` on Windows, and on Linux nothing runs and the ERROR log has a Script Error line | Rename such functions to ASCII letters, digits, `_` and `$`, not starting with a digit | [Event scripts: the 6.3.4 security fix](/topic/28261); [the advisory](/topic/28218) | | 21 | A domain can no longer be named as one of the server's own directories | Below | Below | | 22 | A one-time code is taken once, over COM and REST: a TOTP code the server has accepted is refused if it comes again within its window, but not counted as a wrong code, so it locks nobody out | A script that sent the administrator's password and a code with every REST request now has only its first request in each half minute accepted, the others refused with 401: sign it in once with `POST /api/v1/session` and use the `hmailsession` cookie, or give it an API key | [The REST API: one-time codes](/topic/28264) | | 23 | The outbound relay sends its OAuth2 token over TLS only: a route or the SMTP relay whose host is on `OutboundOAuth2Hosts` and whose connection security is none, or STARTTLS optional against a server that does not offer it, fails with a temporary `450 4.7.0` (`HM5912`), and the message waits for the route's retries | Set that connection security to TLS or STARTTLS required. A route to Exchange Online on 587, set up as Microsoft documents it, is not affected | [Routes, relays and smart hosts: OAuth2](/topic/28269) | | 24 | A hash stored before 6.3.4 of a password with a character above U+00FF is refused | Set such passwords again, and soon | [Upgrading to 6.3.4](/topic/28193); [Hardening a public mail server](/topic/28241) | ### Three that no table row can hold **ISO-8859 (18).** ISO-8859-1 is decoded as windows-1252, as every mail client reads it, and ISO-8859-2 to -9, -13 and -15 by their own code pages. There was no ISO-8859 name in the server's table at all, so a Western Windows server decoded Latin-1 right by accident and decodes it as before, one with another ANSI code page now decodes it right too, and on Linux Latin-1 is no longer turned into replacement characters. There is nothing to set. **Reserved domain names (21).** A domain's mail is kept in the directory named after it under the data directory, beside the server's own stores. So a single-label name such as `protected`, `confidential`, `recoverable`, `discovery`, `quarantine`, `files`, `lists` or `blobs`, in any case, or `temp`, `events` and `database`, which the Linux packages put there, is refused for a new domain and for a rename over COM (`Domain.Save`) and REST, and so in the Deck. The Control Panel already refused every name without a dot. A domain that already has such a name is still saved, so that it can be managed, and is named as `HM7597` in the error log at every start and as `domain_name_reserved` on the first-hour checklist. **Rename it**: the rename moves each of its accounts' mailboxes and leaves the store's files where they are. Until then, a domain named `recoverable`, `discovery`, `confidential` or `protected` has its mail in a folder the backup leaves out. **The search route (11).** Only a client of `GET /api/v1/me/search` other than the webmail needs to change. A search with more hits than `limit` answered `complete: true, more: true` and could not reach the rest; it now answers `complete: false, more: true` with `next_before_id`, which `before_id` continues from. Hits come newest first across every folder by message id, where they came folder by folder ([Performance.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Performance.md)). ### Coming from 5.x or early 6.x The 24 apply to you too, and so do the changes of every release in between: section 5 of [Changes Since 6.2.10](https://gitlab.com/hmailserver/hmailserver/-/wikis/Changes-Since-6210) lists what came without a switch from 6.2.10 to 6.2.28. On 5.3 to 5.5, back up first and test the restore. Four things are particular to an older installation, all on Windows: - **The Administrator is replaced by the Control Panel.** `hMailAdmin.exe` was retired in 6.2. The Control Panel, `hMailCP.exe`, does everything it did, and its navigation tree mirrors the old layout. Your TOTP secret carries over. - **The PHP WebAdmin is gone.** Administer a remote server with the Control Panel, which connects to it directly, or from a browser with [the Control Deck](/topic/28192). The "Administrative tools" component now only registers the COM type library for scripts on another machine. - **A pre-5 `hMailServer.ini` moves into `Bin`.** The installer copies it from the Windows directory to `Bin\hMailServer.ini`, leaves a `.old` copy where it was and deletes the original; the server reads only the one in `Bin`. - **The .NET 10 runtime.** Since the first release after 6.2.18 the Control Panel and the setup tools (`DBSetup`, `DBSetupQuick`, `DBUpdater`) need the .NET 10 Desktop Runtime. The installer bundles `windowsdesktop-runtime-10.0-win-x64.exe` and installs it `/quiet /norestart` when it is missing (6.3.7's installers carry 10.0.13 and install it only where no .NET 10 Desktop Runtime is installed at all: an upgraded server keeps the one it had, 10.0.12 where 6.3.4 to 6.3.6 installed it, and brings it to 10.0.13 or later through Microsoft Update or Microsoft's installer). **If the runtime does not install.** [Upgrading.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Upgrading.md) says the server, native code with no .NET dependency, is left working without the Control Panel. The documentation's [18.5](https://www.progressiverobot.com/hmailserver-documentation/#185-things-that-changed-and-what-to-do-about-them) says otherwise, and for an upgrade to 6.3.4, which always moves the schema, it is right: [Upgrading.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Upgrading.md) itself says the setup tools need the runtime too, and the server refuses to start against an old schema, so there is no mail server until the schema is moved. Install the runtime (x64) by hand from `https://dotnet.microsoft.com/download/dotnet/10.0`, run `DBUpdater.exe` from the installation's `Bin` folder, then start the service. ### When it goes wrong At every start the server compares the version in `hm_dbversion` with the one it requires and refuses to run if they differ, in either direction. The ERROR log has one of these: | Code and message | What it means | What to do | |---|---|---| | `HM5011`: `The database is too old for this version of hMailServer. Please run hMailServer Database updater (DBUpdater.exe) to upgrade it.` | The schema did not get all the way up | Run `DBUpdater.exe` from the installation's `Bin` folder | | `HM5011`: `The database is too new for this version of hMailServer. Please upgrade hMailServer.` | An older server against a newer schema: an old binary left in place, or a previous release reinstalled | Install the matching or a newer server. Do not try to downgrade the database | | `HM5010`: `Database version could not be detected.` | `hm_dbversion` could not be read at all | Check the connection details in `[Database]` and that the login can read the table | Both `HM5011` messages end with `Database version: , Required database version: `. On Windows the service reports itself running before this check, so a refused start leaves a service Windows calls Running with nothing listening: read the ERROR log. A 6.3.3 server against a 6.3.4 database is the too-new case on purpose: step 6041 to 6042 changes only the version, the fence that keeps a 6.3.3 server, which lets `hMailServer.INI` win, away from a database that is now the settings store. **Resuming.** DBUpdater picks up from whichever version the database actually reports, so a retry after a partial upgrade resumes rather than restarting. A failed step leaves PostgreSQL exactly as it was and SQL Server normally so; MySQL, MariaDB and SQL Server Compact commit DDL as it executes and keep the steps that ran. The same chain also runs: - on Windows, since 6.3.4, as `hMailServer.exe --upgrade-database` with the service stopped, without DBUpdater's window or the administrator password; it exits 0 (done, or nothing to do), 1 (failed; the output names the step) or 2 (no `[Database]` configured); - on Linux as `hmailserver --upgrade-database`, which the `.deb` and `.rpm` run after installing; - at the service's own start with `UpgradeOnStart=1` in `[Database]` (since 6.3.4, off by default): a failing step refuses the start with `HM5011` naming the step, and the next start resumes there. It backs nothing up, and belongs on one node at a time ([Warm standby](/topic/28232)). DBUpdater exits 0 on success, 1 when the upgrade failed, 2 on a configuration error, and 3 when authentication was cancelled or, under `/silent`, could not be satisfied: see `HM_ADMIN_PASSWORD` in [Unattended installs](/topic/28231). The installer fails with a non-zero exit code of its own; treat that, or a schema version still on the old value, as the failure: ```sql select value from hm_dbversion; ``` **Asking for help.** Give the versions you started from, the step DBUpdater stopped on and the database's own error number; [Read this before you post](/topic/28185) has the rest. Without a GitLab account, email the Service Desk address, [contact-project+hmailserver-hmailserver-86730559-issue-@incoming.gitlab.com](mailto:contact-project+hmailserver-hmailserver-86730559-issue-@incoming.gitlab.com): GitLab files it as a confidential issue, and replies come by email. ### Rolling back from 6.3.4 1. **Uninstall 6.3.4.** 2. **Restore the pre-upgrade database backup.** This is the essential step: the schema is at 6104, and the release you go back to refuses it. 3. **Put the right `hMailServer.ini` in place before the older installer runs**, because its `/Register` reads the file. Which file depends on whether 6.3.4 sealed the stored secrets, that is, whether a stored-secrets key was made: - **Sealed: restore the pre-upgrade `hMailServer.ini`.** 6.3.4 writes the `[Settings]` secrets into the file as `HMK1:` envelopes, and an older server reads them as they are: its `/Register` hands `ServiceAccountPassword`'s envelope to Windows as the service account's password, and the service cannot log on. Its first moments also run SRS, BATV and OAuth2 on the envelopes. - **Not sealed: the 6.3.4 file can stay.** Its `[Settings]` section is a complete copy of the stored settings, and an older server reads the file as the truth, so a setting changed after the upgrade survives, wherever the server could write the file, which a Linux package installation's unit does not allow. Delete the `[SettingsStore]` section, which older builds ignore. They ignore `[SettingsOverride]` too, so move any value only it supplied into `[Settings]`. 4. **Check `PasswordPepper`.** Since 6.3.4 it is kept in the file only, never sealed and in no backup 6.3.4 writes. The file the older server starts with must carry the value the hashes were made with: no pepper, or a different one, refuses every account hashed with Argon2id or scrypt. PBKDF2, the default, is not peppered. 5. **Install the release you came from.** 6. **Sealed, with no pre-upgrade file?** Set `ServiceAccountPassword`, and any other of those secrets you use, again with the older version's Control Panel, then run `hMailServer.exe /Register` again. The wiki's and the documentation's 18.6 reinstall before they restore the database; [Upgrading.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Upgrading.md) restores first. Follow [Upgrading.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Upgrading.md): the older installer runs DBUpdater, which refuses a database newer than its own build, and its `/Register` reads the file. **The data directory normally stays.** [Upgrading.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Upgrading.md) and the wiki's 18.6 say it does not normally need restoring, because no upgrade step rewrites message files; the documentation's 18.6 says to restore it with `hMailServer.INI`. Left in place, mail that arrived after the backup exists on disk with no metadata row in the restored database, and is invisible. That is what a rollback costs. ### What it does not do - The rollback above is the Windows one. Chapter 18 and [Upgrading.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Upgrading.md) give no package command for going back on Linux; the database rule is the same. - No backup carries a `[SettingsOverride]` value; it lives only in the file of the machine that has it. - Moving to another machine is not an upgrade: [Moving to new hardware](/topic/28194). ### Where to read more - [Upgrading hMailServer](https://gitlab.com/hmailserver/hmailserver/-/wikis/Upgrading-hMailServer), 18.5 and 18.6; the documentation's [18.6](https://www.progressiverobot.com/hmailserver-documentation/#186-rolling-back) - [Upgrading.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Upgrading.md) and its wiki copy, [Upgrading Guide](https://gitlab.com/hmailserver/hmailserver/-/wikis/Upgrading-Guide) - [CHANGELOG.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/CHANGELOG.md), 6.3.4, and [Changes Since 6.2.10](https://gitlab.com/hmailserver/hmailserver/-/wikis/Changes-Since-6210), section 5 - [SecretsAtRest.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SecretsAtRest.md), "Rolling back", and [Settings Overview](https://gitlab.com/hmailserver/hmailserver/-/wikis/Settings-Overview) - [SetupChecklist.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SetupChecklist.md) and [AccountRecovery.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/AccountRecovery.md), for items 21 and 22 - [Troubleshooting](https://gitlab.com/hmailserver/hmailserver/-/wikis/Troubleshooting), for a start that fails --- Canonical: https://www.hmailserver.co.uk/topic/28236/quotas-retention-and-the-archive-how-much-mail-the-server-keeps-and-for-how-long # Quotas, retention and the archive: how much mail the server keeps, and for how long This topic covers how much mail an hMailServer 6.3.4 server keeps, and for how long: each mailbox's **limit** and the quota policy 6.3.4 adds around it; **retention**, the administrator's and each person's, with the recover-deleted window; and the **server archive** under `ArchiveDir`. The one thing to know first: **out of the box nothing is deleted because of its age.** A new account has no limit (`0`), no domain has a retention policy, the archive is off, and once it is on, `ArchiveRetentionDays` is `0`, keep for ever. ### The limit, and what a full mailbox does **Where it is set.** `Account.MaxSize` over COM, `max_size_mb` over REST, in megabytes; `0`, the default, is no limit. Set it on the Control Panel's account dialog (**Quota (MB)**, Windows only), in the Control Deck's account editor, with `PUT /api/v1/accounts/{address}`, in the `max_size_mb` column of an accounts CSV file (since 6.3.4), or with `Set-HmAccount`. A domain's **Limits** tab caps the whole domain (**Maximum domain size (MB)**, `0` for none; once reached, delivery to its mailboxes fails) and sets the quota new accounts get. **What counts** is each message's original size: compression does not lower it, and a mailbox encrypted at rest keeps its sizes in the clear. The kept copy of a message the person sent to expire counts against their quota (413, `quota_exceeded`). | Where | At the limit | |---|---| | SMTP `RCPT TO`, with `RejectFullMailboxAtRcpt` on (`1`, the default) | A mailbox already at or over its limit, and past any grace (below), is answered `452 4.2.2 Mailbox is full.`, a temporary refusal: the sending server keeps the message and retries | | A message that arrives while there is room and would overflow it | Caught at delivery, with a non-delivery report to the envelope sender | | `RejectFullMailboxAtRcpt` at `0` | Accepted, then bounced to an envelope sender that spam has usually forged. Leave it on | | IMAP `APPEND`, `COPY`, `MOVE` | Refused with `[OVERQUOTA]`, so a client can tell a full mailbox from any other refusal | | `APPEND` into a folder shared from another mailbox | Since 6.3.4 charged to the folder's owner, as `COPY` and `MOVE` were | | A migration or PST import job | Pauses with *the account has no room left under its quota* and tries again; raise the limit or make room | **A full disk is a different refusal.** Below `MinimumFreeDiskSpaceMB` (100) free on the store's volume, `MAIL FROM` gets `452 4.3.1 Insufficient system storage.` and IMAP `APPEND` `NO [UNAVAILABLE]`, for every mailbox. The log warns first, below `DiskSpaceWarningThresholdMB` (1024). **What the person sees.** - The webmail's **Storage** page (since 6.3.2, `GET /api/v1/me/storage`): the share used, every folder by size, the twenty largest messages, and a clean-up of Trash and Junk older than 7, 30 or 90 days, or everything. See [The webmail portal at /portal](/topic/28191). - `GET /api/v1/me` answers `quota` with `limit_mb` and `used_bytes`. - **IMAP QUOTA**: `QUOTA=RES-STORAGE` is advertised; `GETQUOTA` and `GETQUOTAROOT` answer one unnamed root for the whole account with `STORAGE` in kilobytes, and an empty list for an account with no limit. `SETQUOTA` is refused: the limit is the administrator's. It is on as seeded; the switch is `imap_quota_enabled` on `PUT /api/v1/settings` (`IMAPQuotaEnabled` over COM). ### The quota policy (since 6.3.4) The server-wide warning at `QuotaWarningPercent` (90) has been there since 6.2.22-pre6. Schema 6061 adds a domain's own warning share and words, a grace and a send-stop, each a share of the limit; schema 6088 lets an account hold its own shares too. | Point | Server | Domain (`-1` takes the server's) | What it does | |---|---|---|---| | Warning | `QuotaWarningPercent`, 90; `0` warns nobody | `quota_warning_percent` | The delivery that takes the mailbox across this share sends its owner one notice, from the domain's postmaster | | Grace | `quota_grace_percent`, 0 | `quota_grace_percent` | How far past the limit mail from other people is still accepted | | Send-stop | `quota_send_stop_percent`, 0 (never) | `quota_send_stop_percent` | The share at which the account can no longer send | Over COM (Windows): `Settings.QuotaGracePercent`, `Settings.QuotaSendStopPercent`, and `Domain` properties of the same names. **The warning is sent once**, by the delivery that crosses the threshold; nothing records who was warned, so a mailbox emptied and filled again is warned again. **The grace is for mail other people send**, the mail otherwise lost. A 1 GB mailbox with a 50% grace receives until it holds 1.5 GB; the owner's own saves and copies over IMAP and the webmail stop at the limit. **The send-stop refuses the account, not the address.** An authenticated `MAIL FROM` is refused with `552 5.2.2` and the reason, keyed on the account that signed in, even under a domain alias, not the address it named. Receiving goes on. [QuotaPolicy.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/QuotaPolicy.md) describes the refusal at SMTP only; test the webmail's send and the send API before relying on it there. **The words** are the `QUOTA_WARNING_SUBJECT` and `QUOTA_WARNING` server messages, with `%MACRO_MAILBOX%`, `%MACRO_QUOTA%` (MB), `%MACRO_USED%` (MB) and `%MACRO_PERCENT%`. Server messages are server-wide (the Control Panel's **Mail flow & delivery → Server messages**, the Deck's Server messages page, `PUT /api/v1/settings/messages/{name}`). A domain sets its own subject and text, `quota_warning_subject` and `quota_warning_text` (`Domain.QuotaWarningSubject`, `Domain.QuotaWarningText`), so its people can be warned in their own language. They are chosen apart: a domain that sets only the text keeps the server's subject. **Levels.** The warning is a *preference*: the deepest level holding a value wins. The grace and the send-stop are *limits*: an account's own value may tighten its domain's, never loosen it, while a domain's may sit either side of the server's. The domain's shares are the domain administrator's, and a domain-restricted key's. `_effective` and `_source` on the domain and account routes say why a mailbox was not warned. An account's own quota shares are tested only through what the routes answer, and only on Windows. The mechanism is in [Settings: what is not stock, when a save takes effect, and the three levels](/topic/28222). **Where it is set.** - **Control Deck:** the domain editor's *Quota policy* group; the server's grace and send-stop on the Settings page. - **Control Panel** (Windows): the domain dialog's **Limits** tab; the server's grace and send-stop under SMTP, beside `QuotaWarningPercent`. - **REST:** the five domain keys on `PUT /api/v1/domains/{name}`; the server's two on `PUT /api/v1/settings`. - **An account's own value:** the **Levels** tab of the Control Panel's account dialog, the Deck account editor's levels card, `hmctl account set`, or `PUT /api/v1/accounts/{address}`. `QuotaWarningPercent` is a key of the settings store: set it in the Control Panel (Windows) on **Connections & protocols → Protocols**, SMTP tab, with `PUT /api/v1/settings/ini/{name}` or `hmctl ini set`, which answer to the administrator password alone (never an API key, nor an account holding a role), or, on the Linux server itself, `hmailserver --set-setting`, not by editing `hMailServer.ini`. Every value here applies when it is saved through the server, at every level; [when a save takes effect](/topic/28222) explains why a line edited in the file does not hold and when a value stored by `--set-setting` arrives. A backup taken before the policy restores with the server's values. One domain warned at 85%, stopped from sending at the limit and given a 20% grace, with a German subject; then one account put back on its domain's grace: ```bash hmctl domain set example.com quota_warning_percent 85 hmctl domain set example.com quota_send_stop_percent 100 hmctl domain set example.com quota_grace_percent 20 hmctl domain update example.com --set "quota_warning_subject=Ihr Postfach ist fast voll" hmctl account set ann@example.com quota_grace_percent inherit ``` ### Who is nearest their limit `GET /api/v1/reports/quota` lists every mailbox with a limit and at least one message, nearest first: `megabytes`, `quota_megabytes`, `percent`, the `warns_at` and `sending_stops_at` points that apply to it, and a `state` of `ok`, `warned`, `stopped` or `over`. It takes `domain`, `top` (1 to 200, default 10) and `format=csv`, and ignores the date window. Its test has run on the Windows bench only. The Deck's **Reports** page shows it as *Near their quota*. Other than the administrator password, a credential needs `view_configuration` and `view_message_trace`. The sections `hmctl report` documents do not include it: ```bash curl -u Administrator:your-admin-password \ "http://127.0.0.1:8443/api/v1/reports/quota?domain=example.com&top=200&format=csv" ``` Nothing records what a mailbox held yesterday; per-domain storage by day is the `storage` section, in [Alerts, the audit trail, syslog and the journal, and per-domain reports](/topic/28273). ### Retention: the administrator's policy `MessageRetentionDays` (since 6.2.25) is set per domain and per account; there is no server value. On a domain, `0` (the default) is no policy and a number deletes mail older than that many days. On an account, `0` (the default) takes the domain's, a number is its own, and `-1` keeps for ever whatever the domain says. **The sweep.** `MailboxRetentionTask` runs at start-up and every six hours. It removes delivered messages whose stored creation time is older than the window, in **every folder** of the mailbox, through the same deletion an IMAP `EXPUNGE` performs, so the quota, the QRESYNC record and an open IMAP session all see it. `Utilities.RunMessageRetention` (COM, Windows only) runs it now and returns the count; no document names a REST route or `hmctl` verb for it, so on Linux wait for the next pass. **Where it is set.** On Windows the Control Panel's domain dialog (**Limits** tab) and account dialog (General tab); the Deck's domain and account editors; `message_retention_days` on the domain and account routes; `hmctl domain set` and `hmctl account set`. **A domain's retention is the server administrator's** (COM refuses a domain administrator, the domain route a restricted key); an account's own value is the domain administrator's, so an account's `-1` can switch its domain's policy off for that account. **Before switching a domain's policy on:** 1. Give `-1` to each account that must keep everything. 2. Decide whether what the sweep removes should be recoverable for a while (`recover_deleted_days`, below). Off, a removed message is gone. 3. Set the domain's days. It applies when saved; the next pass, at most six hours away, deletes. **Easy to miss.** A **restore** of the built-in backup does not carry a domain's retention days: the wiki's Backup and Restore page lists them as silently reset to `0`, so re-enter them. And a **legal hold** outranks retention: a held account is left alone by the policy and by the person's own choices, and the log says so. ### Each person's own retention (since 6.3.4) Three choices on the webmail's preferences page: *Empty Trash after*, *Empty Junk after* and *Archive Inbox mail after*, each never, 7, 30 or 90 days, or a year, kept as `retention_trash_days`, `retention_junk_days` and `retention_archive_days`. The same six-hourly sweep runs them after the administrator's policy. Trash and Junk (the folders designated `\Trash` and `\Junk`, or named as clients name them) are emptied of what is older; Inbox mail older than the choice is copied to the `\Archive` folder, made as a top-level `Archive` if there is none, then removed from the Inbox. That is a folder of the person's mailbox, not the server archive. A value that is not a whole number of days from 1 to 3650 is off. ### The recover-deleted window (since 6.3.4) `recover_deleted_days` on `PUT /api/v1/settings` or the Deck's Settings view (0 to 365; `0`, off, by default) keeps a copy of what a person deletes, and of what either retention removes, for the person to put back from the webmail's *Recover deleted items*. A domain or an account may keep copies for fewer days than the level above, never more. The copies, under `/Recoverable//`, **count against no quota** and are **not in the backup**, so size the window with the disk in mind; the six-hourly task ends those past the window, and all of them at `0`. The person's side is in [The webmail in depth: search, offline mail, expiring messages, send-as and recovery](/topic/28254). ### The server archive With `ArchiveDir` set, the server writes a copy of every message received by SMTP into a tree of its own, as received and in the clear: encryption at rest does not cover it, and the mailbox retention policy does not reach it. The layout, as the Settings Reference writes it (Windows separators, names lower-cased): | Message | Where the copy goes | |---|---| | From a local sender | `ArchiveDir\\\Sent-` | | From anybody else | `ArchiveDir\Inbound\` | | For each local recipient | `ArchiveDir\\\`, made from the first copy when there is one | | A bounce or report with a null sender | The archive's `Error` folder, with no index row | The archive's settings: | Key | Default | What it does | |---|---|---| | `ArchiveDir` | empty (off) | The folder the copies go to | | `ArchiveDomains` | empty (every message) | Comma-separated: a message is archived only when its local sender or a recipient is in a listed domain, and only the listed domains' copies are made | | `ArchiveHardLinks` | `0` | Hard-link instead of copy: a promise that nothing outside the server edits a message file in place | | `ArchiveRetentionDays` | `0` (keep for ever) | Deletes archived copies older than this many days | **Its sweep** runs at start-up and every twelve hours. It removes only `.eml` files, follows no symbolic link or junction, is depth-capped, and asks the index first: a **held** copy stays however old. **The index** (since 6.2.25, schema 6029): a row per copy in `hm_archiveindex`, with who, when, what, where and whether it is held; copies made before schema 6029 have none. Search it in the Deck's **Archive** view (since 6.3.4), with `GET /api/v1/archive` (`domain`, `mailbox`, `sender`, `recipient`, `subject`, `since`, `until`, `hold=1`, `limit` 1 to 1000, default 200, newest first), or with `Utilities.SearchArchive` over COM (Windows). A role needs `view_archive`; a key restricted to named domains must name one of its own, and the Inbound copies are the administrator's alone. Searching the words of archived mail is an eDiscovery case's work. **A hold on one copy**, `POST /api/v1/archive/{id}/hold` or `Utilities.SetArchiveHold`, keeps its file and row from the sweep and from an address's erasure; `DELETE` lifts it. Holds on whole accounts and domains are eDiscovery's: see [Compliance: eDiscovery, legal holds, personal data, erasure and the audit trail](/topic/28242). ```bash curl -u Administrator:your-admin-password \ "http://127.0.0.1:8443/api/v1/archive?domain=example.com&sender=alice@example.com&since=2026-09-01&limit=50" curl -u Administrator:your-admin-password -X POST http://127.0.0.1:8443/api/v1/archive/4471/hold ``` **Where it is set.** On Windows, the Control Panel's **Maintenance → Advanced**, *Copies of mail* tab (`ArchiveDir`, `ArchiveHardLinks`, `ArchiveDomains`), and `ArchiveRetentionDays` on the SMTP tab of **Connections & protocols → Protocols**. The Deck has no page for these keys: use `PUT /api/v1/settings/ini/{name}` or `hmctl ini set ` (the administrator password only), which take effect when saved, or `hmailserver --set-setting NAME=VALUE` (with `sudo`) on the Linux server itself, which, while the service runs, waits for the next save, a reinitialise (`POST /api/v1/server/reinitialize`) or a restart. **Setting one up:** 1. Choose a folder **outside the data directory**: a full restore with the messages option replaces that directory, and an archive inside it too, held copies included. 2. On Windows, a new installation runs the service as `NT SERVICE\hMailServer`, which writes only where it is granted: approve the folder by running `hMailServer.exe /GrantServiceAccess` with `--approve-modify ""` from an elevated Command Prompt in the installation's `Bin` folder. It must be a full path on a fixed local drive, outside the program folder, Program Files, the Windows folder and the profiles folder: never a share, a drive's root, or a path through a junction or link. 3. On Linux the service runs as `User=hmailserver`, and the packaged unit lets it write only under `/var/lib/hmailserver`, the data directory itself, and `/var/log/hmailserver`. Every other folder is read-only to it (`/home` is not even visible), and an `ArchiveDir` it cannot write stops archiving. Make the folder writable by the `hmailserver` user, add it to the unit with a drop-in (`sudo systemctl edit hmailserver`, then under `[Service]` a `ReadWritePaths=` line naming the folder) and restart the service. [SecureDefaults.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SecureDefaults.md) gives this drop-in for a data folder; no document gives it for the archive. 4. Set `ArchiveDomains` first if only some domains are archived, then `ArchiveDir`. 5. Send a test message and find its row in the Deck's Archive view. **Before relying on it.** - **The backup does not carry it.** The backup copies the data directory, and `hm_archiveindex` is not carried: back up the archive folder, and the database with its own tool for the index. - **Moving it strands the index.** `hm_archiveindex.archivepath` is matched as an exact string by the sweep, holds and erasure; do not rewrite it in bulk. If the folder moves, move it back or accept that older rows point at the old place. - **A mailbox move leaves the archive under the old address**, and an address's erasure with the archive included leaves the copies in other people's trees and in `Inbound` until `ArchiveRetentionDays` ages them out. - **It is not journaling**, which sends a copy elsewhere and keeps nothing here: [Journaling and dual delivery](/topic/28271). - **SMTP, as documented.** No 6.3.4 document says whether mail the server queues itself, such as the webmail's send, or mail a fetch account collects, is archived. ### What it does not do - **No message-count quota**: storage is the only quota enforced. - **No server-wide retention policy**, and no documented way to run the retention sweep on demand on Linux. - **No write-once archive**: a hold is a flag an administrator can lift. - **The archive only grows** while `ArchiveRetentionDays` is `0`, and the recover-deleted store is bounded by its window alone: watch the disk they are on. ### Where to read more - Documents at v6.3.4: [QuotaPolicy.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/QuotaPolicy.md), [SettingLevels.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SettingLevels.md), [Reports.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Reports.md), [RecoverDeleted.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/RecoverDeleted.md), [Discovery.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Discovery.md), [Backup.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Backup.md), [ServiceAccount.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/ServiceAccount.md), [RelocatingAnInstallation.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/RelocatingAnInstallation.md), [PersonalData.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/PersonalData.md), [Journaling.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Journaling.md), [SecureDefaults.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SecureDefaults.md), [SettingsApply.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SettingsApply.md). - Wiki: [Settings Reference](https://gitlab.com/hmailserver/hmailserver/-/wikis/Settings-Reference) (section 9.13), [Your First Domain and Mailbox](https://gitlab.com/hmailserver/hmailserver/-/wikis/Your-First-Domain-and-Mailbox), [Compliance and eDiscovery](https://gitlab.com/hmailserver/hmailserver/-/wikis/Compliance-and-eDiscovery), [Backup and Restore](https://gitlab.com/hmailserver/hmailserver/-/wikis/Backup-and-Restore), [APIs Reference](https://gitlab.com/hmailserver/hmailserver/-/wikis/APIs-Reference). - The documentation page: [Storage, retention and archive](https://www.progressiverobot.com/hmailserver-documentation/#storage-retention-and-archive), [Queue, quarantine and archive](https://www.progressiverobot.com/hmailserver-documentation/#2411-queue-quarantine-and-archive). - On this forum: [Backup and restore](/topic/28221), for what a restore resets; [Relocating an installation, and moving or renaming a mailbox](/topic/28244), for the archive index; [the service account on Windows](/topic/28227) and [the Linux unit](/topic/28245), for the folders the service may write; [The Control Deck](/topic/28192). --- Canonical: https://www.hmailserver.co.uk/topic/28236/quotas-retention-and-the-archive-how-much-mail-the-server-keeps-and-for-how-long # Quotas, retention and the archive: how much mail the server keeps, and for how long This topic covers how much mail an hMailServer 6.3.4 server keeps, and for how long: each mailbox's **limit** and the quota policy 6.3.4 adds around it; **retention**, the administrator's and each person's, with the recover-deleted window; and the **server archive** under `ArchiveDir`. The one thing to know first: **out of the box nothing is deleted because of its age.** A new account has no limit (`0`), no domain has a retention policy, the archive is off, and once it is on, `ArchiveRetentionDays` is `0`, keep for ever. ### The limit, and what a full mailbox does **Where it is set.** `Account.MaxSize` over COM, `max_size_mb` over REST, in megabytes; `0`, the default, is no limit. Set it on the Control Panel's account dialog (**Quota (MB)**, Windows only), in the Control Deck's account editor, with `PUT /api/v1/accounts/{address}`, in the `max_size_mb` column of an accounts CSV file (since 6.3.4), or with `Set-HmAccount`. A domain's **Limits** tab caps the whole domain (**Maximum domain size (MB)**, `0` for none; once reached, delivery to its mailboxes fails) and sets the quota new accounts get. **What counts** is each message's original size: compression does not lower it, and a mailbox encrypted at rest keeps its sizes in the clear. The kept copy of a message the person sent to expire counts against their quota (413, `quota_exceeded`). | Where | At the limit | |---|---| | SMTP `RCPT TO`, with `RejectFullMailboxAtRcpt` on (`1`, the default) | A mailbox already at or over its limit, and past any grace (below), is answered `452 4.2.2 Mailbox is full.`, a temporary refusal: the sending server keeps the message and retries | | A message that arrives while there is room and would overflow it | Caught at delivery, with a non-delivery report to the envelope sender | | `RejectFullMailboxAtRcpt` at `0` | Accepted, then bounced to an envelope sender that spam has usually forged. Leave it on | | IMAP `APPEND`, `COPY`, `MOVE` | Refused with `[OVERQUOTA]`, so a client can tell a full mailbox from any other refusal | | `APPEND` into a folder shared from another mailbox | Since 6.3.4 charged to the folder's owner, as `COPY` and `MOVE` were | | A migration or PST import job | Pauses with *the account has no room left under its quota* and tries again; raise the limit or make room | **A full disk is a different refusal.** Below `MinimumFreeDiskSpaceMB` (100) free on the store's volume, `MAIL FROM` gets `452 4.3.1 Insufficient system storage.` and IMAP `APPEND` `NO [UNAVAILABLE]`, for every mailbox. The log warns first, below `DiskSpaceWarningThresholdMB` (1024). **What the person sees.** - The webmail's **Storage** page (since 6.3.2, `GET /api/v1/me/storage`): the share used, every folder by size, the twenty largest messages, and a clean-up of Trash and Junk older than 7, 30 or 90 days, or everything. See [The webmail portal at /portal](/topic/28191). - `GET /api/v1/me` answers `quota` with `limit_mb` and `used_bytes`. - **IMAP QUOTA**: `QUOTA=RES-STORAGE` is advertised; `GETQUOTA` and `GETQUOTAROOT` answer one unnamed root for the whole account with `STORAGE` in kilobytes, and an empty list for an account with no limit. `SETQUOTA` is refused: the limit is the administrator's. It is on as seeded; the switch is `imap_quota_enabled` on `PUT /api/v1/settings` (`IMAPQuotaEnabled` over COM). ### The quota policy (since 6.3.4) The server-wide warning at `QuotaWarningPercent` (90) has been there since 6.2.22-pre6. Schema 6061 adds a domain's own warning share and words, a grace and a send-stop, each a share of the limit; schema 6088 lets an account hold its own shares too. | Point | Server | Domain (`-1` takes the server's) | What it does | |---|---|---|---| | Warning | `QuotaWarningPercent`, 90; `0` warns nobody | `quota_warning_percent` | The delivery that takes the mailbox across this share sends its owner one notice, from the domain's postmaster | | Grace | `quota_grace_percent`, 0 | `quota_grace_percent` | How far past the limit mail from other people is still accepted | | Send-stop | `quota_send_stop_percent`, 0 (never) | `quota_send_stop_percent` | The share at which the account can no longer send | Over COM (Windows): `Settings.QuotaGracePercent`, `Settings.QuotaSendStopPercent`, and `Domain` properties of the same names. **The warning is sent once**, by the delivery that crosses the threshold; nothing records who was warned, so a mailbox emptied and filled again is warned again. **The grace is for mail other people send**, the mail otherwise lost. A 1 GB mailbox with a 50% grace receives until it holds 1.5 GB; the owner's own saves and copies over IMAP and the webmail stop at the limit. **The send-stop refuses the account, not the address.** An authenticated `MAIL FROM` is refused with `552 5.2.2` and the reason, keyed on the account that signed in, even under a domain alias, not the address it named. Receiving goes on. [QuotaPolicy.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/QuotaPolicy.md) describes the refusal at SMTP only; test the webmail's send and the send API before relying on it there. **The words** are the `QUOTA_WARNING_SUBJECT` and `QUOTA_WARNING` server messages, with `%MACRO_MAILBOX%`, `%MACRO_QUOTA%` (MB), `%MACRO_USED%` (MB) and `%MACRO_PERCENT%`. Server messages are server-wide (the Control Panel's **Mail flow & delivery → Server messages**, the Deck's Server messages page, `PUT /api/v1/settings/messages/{name}`). A domain sets its own subject and text, `quota_warning_subject` and `quota_warning_text` (`Domain.QuotaWarningSubject`, `Domain.QuotaWarningText`), so its people can be warned in their own language. They are chosen apart: a domain that sets only the text keeps the server's subject. **Levels.** The warning is a *preference*: the deepest level holding a value wins. The grace and the send-stop are *limits*: an account's own value may tighten its domain's, never loosen it, while a domain's may sit either side of the server's. The domain's shares are the domain administrator's, and a domain-restricted key's. `_effective` and `_source` on the domain and account routes say why a mailbox was not warned. An account's own quota shares are tested only through what the routes answer, and only on Windows. The mechanism is in [Settings: what is not stock, when a save takes effect, and the three levels](/topic/28222). **Where it is set.** - **Control Deck:** the domain editor's *Quota policy* group; the server's grace and send-stop on the Settings page. - **Control Panel** (Windows): the domain dialog's **Limits** tab; the server's grace and send-stop under SMTP, beside `QuotaWarningPercent`. - **REST:** the five domain keys on `PUT /api/v1/domains/{name}`; the server's two on `PUT /api/v1/settings`. - **An account's own value:** the **Levels** tab of the Control Panel's account dialog, the Deck account editor's levels card, `hmctl account set`, or `PUT /api/v1/accounts/{address}`. `QuotaWarningPercent` is a key of the settings store: set it in the Control Panel (Windows) on **Connections & protocols → Protocols**, SMTP tab, with `PUT /api/v1/settings/ini/{name}` or `hmctl ini set`, which answer to the administrator password alone (never an API key, nor an account holding a role), or, on the Linux server itself, `hmailserver --set-setting`, not by editing `hMailServer.ini`. Every value here applies when it is saved through the server, at every level; [when a save takes effect](/topic/28222) explains why a line edited in the file does not hold and when a value stored by `--set-setting` arrives. A backup taken before the policy restores with the server's values. One domain warned at 85%, stopped from sending at the limit and given a 20% grace, with a German subject; then one account put back on its domain's grace: ```bash hmctl domain set example.com quota_warning_percent 85 hmctl domain set example.com quota_send_stop_percent 100 hmctl domain set example.com quota_grace_percent 20 hmctl domain update example.com --set "quota_warning_subject=Ihr Postfach ist fast voll" hmctl account set ann@example.com quota_grace_percent inherit ``` ### Who is nearest their limit `GET /api/v1/reports/quota` lists every mailbox with a limit and at least one message, nearest first: `megabytes`, `quota_megabytes`, `percent`, the `warns_at` and `sending_stops_at` points that apply to it, and a `state` of `ok`, `warned`, `stopped` or `over`. It takes `domain`, `top` (1 to 200, default 10) and `format=csv`, and ignores the date window. Its test has run on the Windows bench only. The Deck's **Reports** page shows it as *Near their quota*. Other than the administrator password, a credential needs `view_configuration` and `view_message_trace`. The sections `hmctl report` documents do not include it: ```bash curl -u Administrator:your-admin-password \ "http://127.0.0.1:8443/api/v1/reports/quota?domain=example.com&top=200&format=csv" ``` Nothing records what a mailbox held yesterday; per-domain storage by day is the `storage` section, in [Alerts, the audit trail, syslog and the journal, and per-domain reports](/topic/28273). ### Retention: the administrator's policy `MessageRetentionDays` (since 6.2.25) is set per domain and per account; there is no server value. On a domain, `0` (the default) is no policy and a number deletes mail older than that many days. On an account, `0` (the default) takes the domain's, a number is its own, and `-1` keeps for ever whatever the domain says. **The sweep.** `MailboxRetentionTask` runs at start-up and every six hours. It removes delivered messages whose stored creation time is older than the window, in **every folder** of the mailbox, through the same deletion an IMAP `EXPUNGE` performs, so the quota, the QRESYNC record and an open IMAP session all see it. `Utilities.RunMessageRetention` (COM, Windows only) runs it now and returns the count; no document names a REST route or `hmctl` verb for it, so on Linux wait for the next pass. **Where it is set.** On Windows the Control Panel's domain dialog (**Limits** tab) and account dialog (General tab); the Deck's domain and account editors; `message_retention_days` on the domain and account routes; `hmctl domain set` and `hmctl account set`. **A domain's retention is the server administrator's** (COM refuses a domain administrator, the domain route a restricted key); an account's own value is the domain administrator's, so an account's `-1` can switch its domain's policy off for that account. **Before switching a domain's policy on:** 1. Give `-1` to each account that must keep everything. 2. Decide whether what the sweep removes should be recoverable for a while (`recover_deleted_days`, below). Off, a removed message is gone. 3. Set the domain's days. It applies when saved; the next pass, at most six hours away, deletes. **Easy to miss.** A **restore** of the built-in backup does not carry a domain's retention days: the wiki's Backup and Restore page lists them as silently reset to `0`, so re-enter them. And a **legal hold** outranks retention: a held account is left alone by the policy and by the person's own choices, and the log says so. ### Each person's own retention (since 6.3.4) Three choices on the webmail's preferences page: *Empty Trash after*, *Empty Junk after* and *Archive Inbox mail after*, each never, 7, 30 or 90 days, or a year, kept as `retention_trash_days`, `retention_junk_days` and `retention_archive_days`. The same six-hourly sweep runs them after the administrator's policy. Trash and Junk (the folders designated `\Trash` and `\Junk`, or named as clients name them) are emptied of what is older; Inbox mail older than the choice is copied to the `\Archive` folder, made as a top-level `Archive` if there is none, then removed from the Inbox. That is a folder of the person's mailbox, not the server archive. A value that is not a whole number of days from 1 to 3650 is off. ### The recover-deleted window (since 6.3.4) `recover_deleted_days` on `PUT /api/v1/settings` or the Deck's Settings view (0 to 365; `0`, off, by default) keeps a copy of what a person deletes, and of what either retention removes, for the person to put back from the webmail's *Recover deleted items*. A domain or an account may keep copies for fewer days than the level above, never more. The copies, under `/Recoverable//`, **count against no quota** and are **not in the backup**, so size the window with the disk in mind; the six-hourly task ends those past the window, and all of them at `0`. The person's side is in [The webmail in depth: search, offline mail, expiring messages, send-as and recovery](/topic/28254). ### The server archive With `ArchiveDir` set, the server writes a copy of every message received by SMTP into a tree of its own, as received and in the clear: encryption at rest does not cover it, and the mailbox retention policy does not reach it. The layout, as the Settings Reference writes it (Windows separators, names lower-cased): | Message | Where the copy goes | |---|---| | From a local sender | `ArchiveDir\\\Sent-` | | From anybody else | `ArchiveDir\Inbound\` | | For each local recipient | `ArchiveDir\\\`, made from the first copy when there is one | | A bounce or report with a null sender | The archive's `Error` folder, with no index row | The archive's settings: | Key | Default | What it does | |---|---|---| | `ArchiveDir` | empty (off) | The folder the copies go to | | `ArchiveDomains` | empty (every message) | Comma-separated: a message is archived only when its local sender or a recipient is in a listed domain, and only the listed domains' copies are made | | `ArchiveHardLinks` | `0` | Hard-link instead of copy: a promise that nothing outside the server edits a message file in place | | `ArchiveRetentionDays` | `0` (keep for ever) | Deletes archived copies older than this many days | **Its sweep** runs at start-up and every twelve hours. It removes only `.eml` files, follows no symbolic link or junction, is depth-capped, and asks the index first: a **held** copy stays however old. **The index** (since 6.2.25, schema 6029): a row per copy in `hm_archiveindex`, with who, when, what, where and whether it is held; copies made before schema 6029 have none. Search it in the Deck's **Archive** view (since 6.3.4), with `GET /api/v1/archive` (`domain`, `mailbox`, `sender`, `recipient`, `subject`, `since`, `until`, `hold=1`, `limit` 1 to 1000, default 200, newest first), or with `Utilities.SearchArchive` over COM (Windows). A role needs `view_archive`; a key restricted to named domains must name one of its own, and the Inbound copies are the administrator's alone. Searching the words of archived mail is an eDiscovery case's work. **A hold on one copy**, `POST /api/v1/archive/{id}/hold` or `Utilities.SetArchiveHold`, keeps its file and row from the sweep and from an address's erasure; `DELETE` lifts it. Holds on whole accounts and domains are eDiscovery's: see [Compliance: eDiscovery, legal holds, personal data, erasure and the audit trail](/topic/28242). ```bash curl -u Administrator:your-admin-password \ "http://127.0.0.1:8443/api/v1/archive?domain=example.com&sender=alice@example.com&since=2026-09-01&limit=50" curl -u Administrator:your-admin-password -X POST http://127.0.0.1:8443/api/v1/archive/4471/hold ``` **Where it is set.** On Windows, the Control Panel's **Maintenance → Advanced**, *Copies of mail* tab (`ArchiveDir`, `ArchiveHardLinks`, `ArchiveDomains`), and `ArchiveRetentionDays` on the SMTP tab of **Connections & protocols → Protocols**. The Deck has no page for these keys: use `PUT /api/v1/settings/ini/{name}` or `hmctl ini set ` (the administrator password only), which take effect when saved, or `hmailserver --set-setting NAME=VALUE` (with `sudo`) on the Linux server itself, which, while the service runs, waits for the next save, a reinitialise (`POST /api/v1/server/reinitialize`) or a restart. **Setting one up:** 1. Choose a folder **outside the data directory**: a full restore with the messages option replaces that directory, and an archive inside it too, held copies included. 2. On Windows, a new installation runs the service as `NT SERVICE\hMailServer`, which writes only where it is granted: approve the folder by running `hMailServer.exe /GrantServiceAccess` with `--approve-modify ""` from an elevated Command Prompt in the installation's `Bin` folder. It must be a full path on a fixed local drive, outside the program folder, Program Files, the Windows folder and the profiles folder: never a share, a drive's root, or a path through a junction or link. 3. On Linux the service runs as `User=hmailserver`, and the packaged unit lets it write only under `/var/lib/hmailserver`, the data directory itself, and `/var/log/hmailserver`. Every other folder is read-only to it (`/home` is not even visible), and an `ArchiveDir` it cannot write stops archiving. Make the folder writable by the `hmailserver` user, add it to the unit with a drop-in (`sudo systemctl edit hmailserver`, then under `[Service]` a `ReadWritePaths=` line naming the folder) and restart the service. [SecureDefaults.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SecureDefaults.md) gives this drop-in for a data folder; no document gives it for the archive. 4. Set `ArchiveDomains` first if only some domains are archived, then `ArchiveDir`. 5. Send a test message and find its row in the Deck's Archive view. **Before relying on it.** - **The backup does not carry it.** The backup copies the data directory, and `hm_archiveindex` is not carried: back up the archive folder, and the database with its own tool for the index. - **Moving it strands the index.** `hm_archiveindex.archivepath` is matched as an exact string by the sweep, holds and erasure; do not rewrite it in bulk. If the folder moves, move it back or accept that older rows point at the old place. - **A mailbox move leaves the archive under the old address**, and an address's erasure with the archive included leaves the copies in other people's trees and in `Inbound` until `ArchiveRetentionDays` ages them out. - **It is not journaling**, which sends a copy elsewhere and keeps nothing here: [Journaling and dual delivery](/topic/28271). - **SMTP, as documented.** No 6.3.4 document says whether mail the server queues itself, such as the webmail's send, or mail a fetch account collects, is archived. ### What it does not do - **No message-count quota**: storage is the only quota enforced. - **No server-wide retention policy**, and no documented way to run the retention sweep on demand on Linux. - **No write-once archive**: a hold is a flag an administrator can lift. - **The archive only grows** while `ArchiveRetentionDays` is `0`, and the recover-deleted store is bounded by its window alone: watch the disk they are on. ### Where to read more - Documents at v6.3.4: [QuotaPolicy.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/QuotaPolicy.md), [SettingLevels.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SettingLevels.md), [Reports.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Reports.md), [RecoverDeleted.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/RecoverDeleted.md), [Discovery.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Discovery.md), [Backup.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Backup.md), [ServiceAccount.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/ServiceAccount.md), [RelocatingAnInstallation.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/RelocatingAnInstallation.md), [PersonalData.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/PersonalData.md), [Journaling.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Journaling.md), [SecureDefaults.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SecureDefaults.md), [SettingsApply.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SettingsApply.md). - Wiki: [Settings Reference](https://gitlab.com/hmailserver/hmailserver/-/wikis/Settings-Reference) (section 9.13), [Your First Domain and Mailbox](https://gitlab.com/hmailserver/hmailserver/-/wikis/Your-First-Domain-and-Mailbox), [Compliance and eDiscovery](https://gitlab.com/hmailserver/hmailserver/-/wikis/Compliance-and-eDiscovery), [Backup and Restore](https://gitlab.com/hmailserver/hmailserver/-/wikis/Backup-and-Restore), [APIs Reference](https://gitlab.com/hmailserver/hmailserver/-/wikis/APIs-Reference). - The documentation page: [Storage, retention and archive](https://www.progressiverobot.com/hmailserver-documentation/#storage-retention-and-archive), [Queue, quarantine and archive](https://www.progressiverobot.com/hmailserver-documentation/#2411-queue-quarantine-and-archive). - On this forum: [Backup and restore](/topic/28221), for what a restore resets; [Relocating an installation, and moving or renaming a mailbox](/topic/28244), for the archive index; [the service account on Windows](/topic/28227) and [the Linux unit](/topic/28245), for the folders the service may write; [The Control Deck](/topic/28192). --- Canonical: https://www.hmailserver.co.uk/topic/28237/administrator-roles-end-to-end-the-27-permissions-giving-a-role-where-each-works # Administrator roles end to end: the 27 permissions, giving a role, where each works *Written for 6.3.4. hMailServer 6.3.7 (17 October 2026) is a security release that closes gaps in what this guide describes. Over COM an account given `serveradmin` with a list of domains was the server's administrator, its list aside, and one given `domainadmin` with a list that left out its own domain administered that domain's ordinary accounts; both are now confined as over REST, so before upgrading give `serveradmin` with no list to any account that is to administer the whole server over COM, a script's or a Control Panel's account among them. A credential confined to named domains is refused a domain's rename, a domain alias, a custom role's creation, change or deletion, a rule action that reaches the server's own addresses, a route's stored credentials or the event script, a relay host that is not public, and the server-wide metric history, over REST and COM alike; and a role is given or taken away only by a credential that takes in what the account holds now and afterwards, so `manage_roles` in named domains can no longer take a role from an account whose admin level is the server administrator's. A state-changing REST request that a browser reports as cross-site is refused. In the Control Panel, *API & monitoring* is *Monitoring & alerts* and *REST API keys* is under *Sign-in & access*; Ctrl+K takes every old title to its page. See [the 6.3.7 announcement](/topic/28277).* This guide puts administrator roles in one place. Roles are new since 6.3.4 (schema 6047). **The one thing to know first: an upgrade changes nobody's authority.** The upgrade writes no assignment, and an account with no assignment holds exactly what its admin level has always given it. Nothing narrows until you give somebody a role. ### Permissions, roles and assignments A **permission** is one thing a credential may do, a **role** is a named set of permissions, and an **assignment** gives one account one role in named domains. An account holds at most one role. | Role | For | Reaches when no domain is named | The same as | |---|---|---|---| | `serveradmin` | Everything, including the secrets no other role reads | Every domain | The administrator password, admin level *Server*, an API key with `Scope=full` and no role | | `domainadmin` | The accounts, aliases, lists and mailboxes of a domain | The account's own domain | Admin level *Domain* | | `helpdesk` | Finding an account, resetting its password, unlocking it, and reading where its mail went, and nothing else | The account's own domain | Nothing before 6.3.4 | | `auditor` | Reading everything the administration exposes, the audit trail included, and changing nothing | Every domain | An API key with `Scope=readonly` and no role | **The built-in roles cannot be edited or deleted.** Their permissions are in the server's code, not the `hm_roles` table, so a change to the table cannot hand anybody a permission. ### The 27 permissions and who holds them A permission marked *server-wide* is reached only by a credential with no list of domains. The queue, the quarantine, the archive, the logs and the settings carry every domain's mail, so a role confined to named domains is refused them whatever it holds. | Permission | What it reaches | `serveradmin` | `domainadmin` | `helpdesk` | `auditor` | |---|---|---|---|---|---| | `administer_server` | Everything, secrets such as password hashes and private keys included; COM's "is this the server administrator?" | yes | | | | | `administer_domain` | COM's "is this a domain administrator?" | yes | yes | | | | `view_configuration` | Status, settings, listeners, certificates, rules, routes, anti-spam lists, the roles and who holds them, alert rules and events, bounce and complaint readings; never a secret | yes | yes | | yes | | `view_accounts` | Domains, accounts, aliases, lists, groups *(server-wide)*, fetch accounts | yes | yes | yes | yes | | `view_mailbox` | Another account's folders and messages | yes | yes | | yes | | `view_queue` | The delivery queue *(server-wide)* | yes | | | yes | | `view_quarantine` | The quarantine *(server-wide)* | yes | yes | | yes | | `view_archive` | The archive *(server-wide)* | yes | | | yes | | `view_logs` | The logs *(server-wide)* | yes | | | yes | | `view_message_trace` | The message trace; with `view_configuration`, the reports and a domain's suppression list | yes | yes | yes | yes | | `view_audit` | The audit trail *(server-wide)* | yes | | | yes | | `manage_accounts` | Creating, changing and deleting accounts, aliases, lists, groups, fetch accounts and app passwords; a domain's spam traps, DKIM rotation and suppression list; moving a mailbox (needs both domains) | yes | yes | | | | `reset_password` | `POST /api/v1/accounts/{address}/password`, `Application.ResetPassword` | yes | yes | yes | | | `unlock_account` | `POST /api/v1/accounts/{address}/unlock`, `Application.UnlockAccount`, `Account.UnlockMailbox`; clearing the logon-failure list *(server-wide)* | yes | yes | yes | | | `manage_mailbox` | Another account's folders, messages and folder rights, a support session, a PST migration job (with `manage_accounts`) | yes | yes | | | | `manage_queue` | Retrying or deleting a queued message *(server-wide)* | yes | | | | | `manage_quarantine` | Releasing or deleting held mail *(server-wide)* | yes | yes | | | | `manage_archive` | Holding and releasing archived messages *(server-wide)* | yes | | | | | `manage_settings` | Every setting, listener, certificate, route, rule and anti-spam list, the alert rules *(server-wide)*; the webmail's branding and file-link policy, a confined credential only its own domains' | yes | | | | | `manage_domains` | Which domains exist *(server-wide)* | yes | | | | | `manage_server` | Reinitialise, back up, reload scripts, clear caches, install an update, run a pass now *(server-wide)* | yes | | | | | `manage_roles` | Roles and who holds them *(server-wide)* | yes | | | | | `send_through_api` | Sending through `POST /api/v1/send`; only an API key reaches it | yes | | | | | `provision` | SCIM 2.0 at `/scim/v2`, reads and writes; only an API key reaches it | yes | | | | | `view_discovery` | eDiscovery's cases, searches, holds and exports | yes | | | yes | | `manage_discovery` | Opening and closing a case, searching, legal holds, exports | yes | | | | | `manage_inbound_webhooks` | Routing an address or a domain to a URL, as much as reading every message the route takes | yes | | | | Four rules sit beside the table: - **The auditor reads mail.** It holds `view_mailbox`; the help desk does not. - **The reports need two permissions.** `GET /api/v1/reports` needs `view_configuration` and `view_message_trace`, because it says who mails whom. `domainadmin` and `auditor` read them; `helpdesk` is refused, and the Deck leaves the *Reports* view out for any role without both. - **The trace follows the role's reach.** A confined help desk traces one whole address in a domain it reaches and sees only events naming that address exactly: `user@example.test` does not also show `otheruser@example.test`. - **Clearing the logon-failure list is server-wide.** `hm_logon_failures` records only the address a failure came from, with no account or domain, so a confined help desk is refused `POST /api/v1/settings/logon-failures/clear`, and `Settings.ClearLogonFailureList` answers the server administrator alone. The help desk unlocks one account with `POST /api/v1/accounts/{address}/unlock` instead; a password reset clears the lockout too. A ban already placed is an IP range under `/api/v1/ipranges`, which needs `manage_settings`. ### What only serveradmin has **Reading a secret.** No `view_` permission covers a password hash or a private key: an auditor that could read one would be an offline attack away from being every role. **Handing out authority.** No custom role may hold `administer_server`, `administer_domain` or `manage_roles`, and a credential may never give a role holding a permission, or reaching a domain, that it does not hold itself. **Taking over a wider account.** A credential may not reset the password of, or change, an account whose whole authority it does not take in. A help desk is refused a domain administrator's or an auditor's password, even in its own domain, because otherwise it could sign in as them; a domain administrator is refused an auditor's. Where the server administrator's own mailbox is in a domain somebody else administers, that domain's administrator is refused all of these on it, over REST and COM. Over COM this is asked where the write is made, not where the object was reached: `Account.Save`, `Account.Delete`, `Account.MoveTo`, `Application.ResetPassword`, the app passwords and every write to what is in an account (`FetchAccount.Save`, `Rule.Save`, `IMAPFolder.Save`, `Message.Save` and the rest) ask that the credential reach the account and take in its authority. An event script's objects carry no credential and are left as they were. [Roles.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Roles.md) lists every route and COM member that asks this. ### Which domains a role reaches | Credential | No domains named | `*` | A list | |---|---|---|---| | An assignment of `serveradmin` or `auditor` | Every domain | Every domain | Only those | | An assignment of any other role, custom ones included | The account's own domain | Every domain | Only those | | An API key's `Domains=` line, whatever its role | Every domain | Matches no domain | Only those | **A custom role holding a server-wide permission needs `*`** before an account can use it at all. The listings answer `every_domain` over REST, and a fourth field over COM. A credential confined to named domains sees only the assignments of accounts in those domains. ### Giving a role, surface by surface **Control Panel (Windows only):** *Accounts & domains → Administrator roles* lists the roles and who holds them. Type an address, choose a role, optionally name the domains, and press *Give the role*; select a row and press *Take the role away*. It applies at once. **Control Deck (Windows and Linux):** the *Roles* view does the same. **REST:** giving or taking away a role needs `manage_roles`, which only `serveradmin` holds and which is server-wide: the administrator password, or a `serveradmin` credential reaching every domain. ``` PUT /api/v1/role-assignments/helen@example.test {"role":"helpdesk","domains":"example.test,example.org"} DELETE /api/v1/role-assignments/helen@example.test GET /api/v1/role-assignments GET /api/v1/roles ``` **COM (Windows only):** an empty role name takes the role away, and `Account.RoleName`, `RoleDomains` and `RolePermissions` read it back: ```powershell $app = New-Object -ComObject 'hMailServer.Application' $account = $app.Authenticate('Administrator', 'your-admin-password') if (-not $account) { throw 'Authentication failed.' } $app.AssignRole('helen@example.test', 'helpdesk', 'example.test') $helen = $app.Domains.ItemByName('example.test').Accounts.ItemByAddress('helen@example.test') $helen.RoleName; $helen.RoleDomains; $helen.RolePermissions ``` With a second factor on the administrator, `Authenticate` refuses and the script uses `AuthenticateWithCode`. `Application.CurrentRole` and `CurrentPermissions` say what a session holds. **An API key:** `POST /api/v1/apikeys` with `"role":"helpdesk"` writes a `Role=` line to `hMailServerApiKeys.ini`. The key store answers to the administrator password alone. The Deck's *API keys* view, signed in with that password, also makes a key with a role and its domains, showing the token once. ```bash curl -u Administrator:your-admin-password \ -H "Content-Type: application/json" \ -d '{"label":"help desk tool","role":"helpdesk","domains":"example.test"}' \ https://:8045/api/v1/apikeys ``` Use the address and port your listener answers on; bound to loopback with no certificate, it is `http://127.0.0.1:8045` (`https://` once a certificate, or an ACME one, is in place). A key's role decides what it reaches. The Control Panel's *REST API keys* page, as its reference describes it, names no role; give a key its role over REST or in the Deck. **No API key signs in to the Deck.** **Custom roles** are made over REST from a name, a description and a comma-separated permission list; both editors list them and give them out. A role that releases held mail holds server-wide permissions, so it is given with `*` and reaches every domain's held mail: ``` POST /api/v1/roles {"name":"quarantine-desk","description":"releases held mail","permissions":"view_quarantine,manage_quarantine"} PUT /api/v1/role-assignments/helen@example.test {"role":"quarantine-desk","domains":"*"} ``` A permission the server does not know is dropped, not refused, so a custom role ends with less authority than it asked for, never more. **A custom role somebody holds cannot be deleted:** `DELETE /api/v1/roles/{name}` answers `409` saying how many accounts hold it, and the Deck's *Delete* shows that sentence. Give each holder another role or take this one away first; assignments are never removed for you, because an account with none falls back to its admin level. A built-in role answers `400`, and COM cannot delete a role. **An assignment decides even when its role holds nothing:** holders of a custom role whose permissions name nothing this version knows are ordinary accounts, not their admin level. ### Where each role works | Role | Control Deck | REST API | Control Panel (COM) | |---|---|---|---| | `serveradmin` | Every view | Every route | Every page | | `domainadmin` | Dashboard, DANE / TLSA, domains, fetch accounts, lists, roles and reports for its domains | Its domains' accounts, aliases, lists and mailboxes, the status, who holds a role there | Domains and Administrator roles | | `helpdesk` | Domains, fetch accounts and lists, with *Reset password* and *Unlock* on every account | The reset, the unlock, the account listings, the trace of an address in its domains | Welcome and About only; `Application.ResetPassword`, `UnlockAccount` and `GlobalObjects.MessageTrace` answer a script | | `auditor` | Every view that reads, the audit trail and reports included | Every read, the audit trail and alerts included; every write refused | Welcome, About and Administrator roles | **The help desk and the auditor work in the Deck.** COM's settings objects answer only the server administrator, so an auditor reads the configuration through the Deck or REST. A domain administrator is not shown the settings, rules, routes, remote domains, certificates, ports, IP ranges, groups, backup, server messages, queue, logs or audit trail, which are server-wide. The quarantine is server-wide as well, so a domain administrator confined to its domain is refused it. **Hiding a page is a courtesy, not the boundary.** The server refuses the request whatever a page draws, and a Control Panel connected to a server that predates roles shows every page. **A role holder signs in with its own address and password**, and its second factor where it has one. **An app password never carries the role:** over REST it is an ordinary account's request, and COM's `Application.Authenticate` and `AuthenticateWithCode` treat it as a wrong password. A support session into a mailbox does not carry the person's role either. A role holder's own mail and settings under `/api/v1/me` stay its own. ### Setting up a help desk 1. **Turn the REST listener on** if it is off. It needs `RestApiPort` set, `RestApiBindAddress` set to an address staff can reach (the default, `127.0.0.1`, serves this machine alone), a certificate unless it is bound to loopback, and a name staff can reach. On Windows set them on the Control Panel's *API & monitoring* page (*Monitoring & alerts* since 6.3.7) and press *Apply now*, since they wait for a reinitialise. On Linux run `sudo hmailserver --set-setting RestApiPort=8045` and the same for `RestApiBindAddress` and, for any address but loopback, `RestApiCertificateFile` and `RestApiPrivateKeyFile` (or let an ACME certificate stand in), then `sudo systemctl restart hmailserver`. [The Control Deck](/topic/28192) topic has the details. 2. **Give each person a mailbox of their own**; the role goes to that account. 3. **Decide the domains.** With none named, `helpdesk` reaches only the account's own domain; name several, comma separated, for a desk serving more. 4. **Give the role** on any surface above and check it with `GET /api/v1/role-assignments` or the Roles view. 5. **The person signs in** to the Deck at `https://:8045/` as themselves, then uses *Reset password* or *Unlock* on an account. 6. **Tell them the limits:** no mail reading, no reports, no domain administrator's or auditor's password, no removing a ban, and, when confined, no clearing of the logon-failure list. Each change the help desk makes is on the audit trail as its own, over COM and REST, with actor type `account` and a detail ending `[role: helpdesk]`. ### What a refusal says - **A permission the role lacks:** `403` naming both, for example `the role 'helpdesk' does not hold the permission 'manage_settings'`. - **A domain it does not reach:** `the role 'helpdesk' does not reach that domain`, **whether or not the account exists**, so nobody can list another domain's mailboxes one request at a time. Over COM, `Application.ResetPassword` and `UnlockAccount` answer `No such account.` for both. - **An account wider than the credential:** *that account holds authority this credential does not*. - **Over COM,** since 6.3.4 a refusal carries the server's sentence rather than a bare `Exception from HRESULT: 0x800403E9`. - **An API key on the key store, the `[Settings]` routes, the audit trail or the alerts:** `401` with `{"error":"authentication failed"}`, the same as "not a key". ### The two stores only the administrator password opens The **key store** (`/api/v1/apikeys`) and the **`[Settings]` section of `hMailServer.ini`** (`/api/v1/settings/ini`) answer to the administrator password and nothing else: not an account holding `serveradmin`, not a key of any role or scope. A key that could mint keys could issue itself an unrestricted replacement, and the file holds the server's secrets. An account is refused `403` with the reason, a key `401`. The **audit trail** (`/api/v1/audit`) and the **alerts** (`/api/v1/alerts/...`) are refused to every API key with the same `401`, so a stolen key cannot read what it did or silence the alerting. An account whose role reaches every domain reaches them through `view_audit`, `view_configuration`, `manage_settings` or `manage_server`; a confined role is refused them. ### Upgrades, the audit trail and backups **The upgrade.** Schema step 6047 adds `hm_roles`, seeded with the four built-in rows, and `hm_roleassignments`, and writes no assignment. Roles are data, not settings: nothing goes into `hm_settings` or `hMailServer.ini`. **The audit trail.** Every role created, changed or deleted, and every role given or taken away, is a row naming the credential that did it; an assignment's row reads like `role: helpdesk -> auditor; domains: the account's own domain -> every domain`. The actor type follows the role: the administrator password and holders of `serveradmin` or `domainadmin` are `administrator`, every other account is `account`. **Backup and restore.** An account's assignment travels with it, by role name, and the custom roles travel whenever the domains or the settings do, restored first. A role the restoring server already has is kept as it is, with any difference named in the backup log. The admin level waits until the assignment is written, so a restore that stops part-way leaves no account wider than its role. An assignment naming a role neither side has is dropped, the level stays withheld, and `HM6129` names the account. No archive carries the API key store. ### What it does not do - **No mapping from an identity provider.** There is no claims mapping; an identity provider saying somebody is an administrator is not obeyed. The server's roles and `oidc_administrators` or `saml_administrators` decide. - **No role on an app password or a support session.** - **No Control Panel or COM on Linux:** roles are given there in the Deck and over REST. ### Where to read more - [Roles.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Roles.md), the authority on all of this. - [Sign-in and Identity](https://gitlab.com/hmailserver/hmailserver/-/wikis/Sign-in-and-Identity) §7, [The Control Deck](https://gitlab.com/hmailserver/hmailserver/-/wikis/The-Control-Deck), [The REST API](https://gitlab.com/hmailserver/hmailserver/-/wikis/The-REST-API) §2 and [The COM API and Scripting](https://gitlab.com/hmailserver/hmailserver/-/wikis/The-COM-API-and-Scripting) §23.2. - [AlertsAndAuditTrail.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/AlertsAndAuditTrail.md), [Backup.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Backup.md) and [Reports.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Reports.md). - On this forum: [The Control Deck: browser administration at /](/topic/28192), [The REST API: turning it on, credentials and roles](/topic/28264), [The COM API on Windows](/topic/28262), [Hardening a public mail server](/topic/28241), [Alerts and the audit trail](/topic/28273) and [IP ranges, auto-ban and lockout](/topic/28239). --- Canonical: https://www.hmailserver.co.uk/topic/28237/administrator-roles-end-to-end-the-27-permissions-giving-a-role-where-each-works # Administrator roles end to end: the 27 permissions, giving a role, where each works *Written for 6.3.4. hMailServer 6.3.7 (17 October 2026) is a security release that closes gaps in what this guide describes. Over COM an account given `serveradmin` with a list of domains was the server's administrator, its list aside, and one given `domainadmin` with a list that left out its own domain administered that domain's ordinary accounts; both are now confined as over REST, so before upgrading give `serveradmin` with no list to any account that is to administer the whole server over COM, a script's or a Control Panel's account among them. A credential confined to named domains is refused a domain's rename, a domain alias, a custom role's creation, change or deletion, a rule action that reaches the server's own addresses, a route's stored credentials or the event script, a relay host that is not public, and the server-wide metric history, over REST and COM alike; and a role is given or taken away only by a credential that takes in what the account holds now and afterwards, so `manage_roles` in named domains can no longer take a role from an account whose admin level is the server administrator's. A state-changing REST request that a browser reports as cross-site is refused. In the Control Panel, *API & monitoring* is *Monitoring & alerts* and *REST API keys* is under *Sign-in & access*; Ctrl+K takes every old title to its page. See [the 6.3.7 announcement](/topic/28277).* This guide puts administrator roles in one place. Roles are new since 6.3.4 (schema 6047). **The one thing to know first: an upgrade changes nobody's authority.** The upgrade writes no assignment, and an account with no assignment holds exactly what its admin level has always given it. Nothing narrows until you give somebody a role. ### Permissions, roles and assignments A **permission** is one thing a credential may do, a **role** is a named set of permissions, and an **assignment** gives one account one role in named domains. An account holds at most one role. | Role | For | Reaches when no domain is named | The same as | |---|---|---|---| | `serveradmin` | Everything, including the secrets no other role reads | Every domain | The administrator password, admin level *Server*, an API key with `Scope=full` and no role | | `domainadmin` | The accounts, aliases, lists and mailboxes of a domain | The account's own domain | Admin level *Domain* | | `helpdesk` | Finding an account, resetting its password, unlocking it, and reading where its mail went, and nothing else | The account's own domain | Nothing before 6.3.4 | | `auditor` | Reading everything the administration exposes, the audit trail included, and changing nothing | Every domain | An API key with `Scope=readonly` and no role | **The built-in roles cannot be edited or deleted.** Their permissions are in the server's code, not the `hm_roles` table, so a change to the table cannot hand anybody a permission. ### The 27 permissions and who holds them A permission marked *server-wide* is reached only by a credential with no list of domains. The queue, the quarantine, the archive, the logs and the settings carry every domain's mail, so a role confined to named domains is refused them whatever it holds. | Permission | What it reaches | `serveradmin` | `domainadmin` | `helpdesk` | `auditor` | |---|---|---|---|---|---| | `administer_server` | Everything, secrets such as password hashes and private keys included; COM's "is this the server administrator?" | yes | | | | | `administer_domain` | COM's "is this a domain administrator?" | yes | yes | | | | `view_configuration` | Status, settings, listeners, certificates, rules, routes, anti-spam lists, the roles and who holds them, alert rules and events, bounce and complaint readings; never a secret | yes | yes | | yes | | `view_accounts` | Domains, accounts, aliases, lists, groups *(server-wide)*, fetch accounts | yes | yes | yes | yes | | `view_mailbox` | Another account's folders and messages | yes | yes | | yes | | `view_queue` | The delivery queue *(server-wide)* | yes | | | yes | | `view_quarantine` | The quarantine *(server-wide)* | yes | yes | | yes | | `view_archive` | The archive *(server-wide)* | yes | | | yes | | `view_logs` | The logs *(server-wide)* | yes | | | yes | | `view_message_trace` | The message trace; with `view_configuration`, the reports and a domain's suppression list | yes | yes | yes | yes | | `view_audit` | The audit trail *(server-wide)* | yes | | | yes | | `manage_accounts` | Creating, changing and deleting accounts, aliases, lists, groups, fetch accounts and app passwords; a domain's spam traps, DKIM rotation and suppression list; moving a mailbox (needs both domains) | yes | yes | | | | `reset_password` | `POST /api/v1/accounts/{address}/password`, `Application.ResetPassword` | yes | yes | yes | | | `unlock_account` | `POST /api/v1/accounts/{address}/unlock`, `Application.UnlockAccount`, `Account.UnlockMailbox`; clearing the logon-failure list *(server-wide)* | yes | yes | yes | | | `manage_mailbox` | Another account's folders, messages and folder rights, a support session, a PST migration job (with `manage_accounts`) | yes | yes | | | | `manage_queue` | Retrying or deleting a queued message *(server-wide)* | yes | | | | | `manage_quarantine` | Releasing or deleting held mail *(server-wide)* | yes | yes | | | | `manage_archive` | Holding and releasing archived messages *(server-wide)* | yes | | | | | `manage_settings` | Every setting, listener, certificate, route, rule and anti-spam list, the alert rules *(server-wide)*; the webmail's branding and file-link policy, a confined credential only its own domains' | yes | | | | | `manage_domains` | Which domains exist *(server-wide)* | yes | | | | | `manage_server` | Reinitialise, back up, reload scripts, clear caches, install an update, run a pass now *(server-wide)* | yes | | | | | `manage_roles` | Roles and who holds them *(server-wide)* | yes | | | | | `send_through_api` | Sending through `POST /api/v1/send`; only an API key reaches it | yes | | | | | `provision` | SCIM 2.0 at `/scim/v2`, reads and writes; only an API key reaches it | yes | | | | | `view_discovery` | eDiscovery's cases, searches, holds and exports | yes | | | yes | | `manage_discovery` | Opening and closing a case, searching, legal holds, exports | yes | | | | | `manage_inbound_webhooks` | Routing an address or a domain to a URL, as much as reading every message the route takes | yes | | | | Four rules sit beside the table: - **The auditor reads mail.** It holds `view_mailbox`; the help desk does not. - **The reports need two permissions.** `GET /api/v1/reports` needs `view_configuration` and `view_message_trace`, because it says who mails whom. `domainadmin` and `auditor` read them; `helpdesk` is refused, and the Deck leaves the *Reports* view out for any role without both. - **The trace follows the role's reach.** A confined help desk traces one whole address in a domain it reaches and sees only events naming that address exactly: `user@example.test` does not also show `otheruser@example.test`. - **Clearing the logon-failure list is server-wide.** `hm_logon_failures` records only the address a failure came from, with no account or domain, so a confined help desk is refused `POST /api/v1/settings/logon-failures/clear`, and `Settings.ClearLogonFailureList` answers the server administrator alone. The help desk unlocks one account with `POST /api/v1/accounts/{address}/unlock` instead; a password reset clears the lockout too. A ban already placed is an IP range under `/api/v1/ipranges`, which needs `manage_settings`. ### What only serveradmin has **Reading a secret.** No `view_` permission covers a password hash or a private key: an auditor that could read one would be an offline attack away from being every role. **Handing out authority.** No custom role may hold `administer_server`, `administer_domain` or `manage_roles`, and a credential may never give a role holding a permission, or reaching a domain, that it does not hold itself. **Taking over a wider account.** A credential may not reset the password of, or change, an account whose whole authority it does not take in. A help desk is refused a domain administrator's or an auditor's password, even in its own domain, because otherwise it could sign in as them; a domain administrator is refused an auditor's. Where the server administrator's own mailbox is in a domain somebody else administers, that domain's administrator is refused all of these on it, over REST and COM. Over COM this is asked where the write is made, not where the object was reached: `Account.Save`, `Account.Delete`, `Account.MoveTo`, `Application.ResetPassword`, the app passwords and every write to what is in an account (`FetchAccount.Save`, `Rule.Save`, `IMAPFolder.Save`, `Message.Save` and the rest) ask that the credential reach the account and take in its authority. An event script's objects carry no credential and are left as they were. [Roles.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Roles.md) lists every route and COM member that asks this. ### Which domains a role reaches | Credential | No domains named | `*` | A list | |---|---|---|---| | An assignment of `serveradmin` or `auditor` | Every domain | Every domain | Only those | | An assignment of any other role, custom ones included | The account's own domain | Every domain | Only those | | An API key's `Domains=` line, whatever its role | Every domain | Matches no domain | Only those | **A custom role holding a server-wide permission needs `*`** before an account can use it at all. The listings answer `every_domain` over REST, and a fourth field over COM. A credential confined to named domains sees only the assignments of accounts in those domains. ### Giving a role, surface by surface **Control Panel (Windows only):** *Accounts & domains → Administrator roles* lists the roles and who holds them. Type an address, choose a role, optionally name the domains, and press *Give the role*; select a row and press *Take the role away*. It applies at once. **Control Deck (Windows and Linux):** the *Roles* view does the same. **REST:** giving or taking away a role needs `manage_roles`, which only `serveradmin` holds and which is server-wide: the administrator password, or a `serveradmin` credential reaching every domain. ``` PUT /api/v1/role-assignments/helen@example.test {"role":"helpdesk","domains":"example.test,example.org"} DELETE /api/v1/role-assignments/helen@example.test GET /api/v1/role-assignments GET /api/v1/roles ``` **COM (Windows only):** an empty role name takes the role away, and `Account.RoleName`, `RoleDomains` and `RolePermissions` read it back: ```powershell $app = New-Object -ComObject 'hMailServer.Application' $account = $app.Authenticate('Administrator', 'your-admin-password') if (-not $account) { throw 'Authentication failed.' } $app.AssignRole('helen@example.test', 'helpdesk', 'example.test') $helen = $app.Domains.ItemByName('example.test').Accounts.ItemByAddress('helen@example.test') $helen.RoleName; $helen.RoleDomains; $helen.RolePermissions ``` With a second factor on the administrator, `Authenticate` refuses and the script uses `AuthenticateWithCode`. `Application.CurrentRole` and `CurrentPermissions` say what a session holds. **An API key:** `POST /api/v1/apikeys` with `"role":"helpdesk"` writes a `Role=` line to `hMailServerApiKeys.ini`. The key store answers to the administrator password alone. The Deck's *API keys* view, signed in with that password, also makes a key with a role and its domains, showing the token once. ```bash curl -u Administrator:your-admin-password \ -H "Content-Type: application/json" \ -d '{"label":"help desk tool","role":"helpdesk","domains":"example.test"}' \ https://:8045/api/v1/apikeys ``` Use the address and port your listener answers on; bound to loopback with no certificate, it is `http://127.0.0.1:8045` (`https://` once a certificate, or an ACME one, is in place). A key's role decides what it reaches. The Control Panel's *REST API keys* page, as its reference describes it, names no role; give a key its role over REST or in the Deck. **No API key signs in to the Deck.** **Custom roles** are made over REST from a name, a description and a comma-separated permission list; both editors list them and give them out. A role that releases held mail holds server-wide permissions, so it is given with `*` and reaches every domain's held mail: ``` POST /api/v1/roles {"name":"quarantine-desk","description":"releases held mail","permissions":"view_quarantine,manage_quarantine"} PUT /api/v1/role-assignments/helen@example.test {"role":"quarantine-desk","domains":"*"} ``` A permission the server does not know is dropped, not refused, so a custom role ends with less authority than it asked for, never more. **A custom role somebody holds cannot be deleted:** `DELETE /api/v1/roles/{name}` answers `409` saying how many accounts hold it, and the Deck's *Delete* shows that sentence. Give each holder another role or take this one away first; assignments are never removed for you, because an account with none falls back to its admin level. A built-in role answers `400`, and COM cannot delete a role. **An assignment decides even when its role holds nothing:** holders of a custom role whose permissions name nothing this version knows are ordinary accounts, not their admin level. ### Where each role works | Role | Control Deck | REST API | Control Panel (COM) | |---|---|---|---| | `serveradmin` | Every view | Every route | Every page | | `domainadmin` | Dashboard, DANE / TLSA, domains, fetch accounts, lists, roles and reports for its domains | Its domains' accounts, aliases, lists and mailboxes, the status, who holds a role there | Domains and Administrator roles | | `helpdesk` | Domains, fetch accounts and lists, with *Reset password* and *Unlock* on every account | The reset, the unlock, the account listings, the trace of an address in its domains | Welcome and About only; `Application.ResetPassword`, `UnlockAccount` and `GlobalObjects.MessageTrace` answer a script | | `auditor` | Every view that reads, the audit trail and reports included | Every read, the audit trail and alerts included; every write refused | Welcome, About and Administrator roles | **The help desk and the auditor work in the Deck.** COM's settings objects answer only the server administrator, so an auditor reads the configuration through the Deck or REST. A domain administrator is not shown the settings, rules, routes, remote domains, certificates, ports, IP ranges, groups, backup, server messages, queue, logs or audit trail, which are server-wide. The quarantine is server-wide as well, so a domain administrator confined to its domain is refused it. **Hiding a page is a courtesy, not the boundary.** The server refuses the request whatever a page draws, and a Control Panel connected to a server that predates roles shows every page. **A role holder signs in with its own address and password**, and its second factor where it has one. **An app password never carries the role:** over REST it is an ordinary account's request, and COM's `Application.Authenticate` and `AuthenticateWithCode` treat it as a wrong password. A support session into a mailbox does not carry the person's role either. A role holder's own mail and settings under `/api/v1/me` stay its own. ### Setting up a help desk 1. **Turn the REST listener on** if it is off. It needs `RestApiPort` set, `RestApiBindAddress` set to an address staff can reach (the default, `127.0.0.1`, serves this machine alone), a certificate unless it is bound to loopback, and a name staff can reach. On Windows set them on the Control Panel's *API & monitoring* page (*Monitoring & alerts* since 6.3.7) and press *Apply now*, since they wait for a reinitialise. On Linux run `sudo hmailserver --set-setting RestApiPort=8045` and the same for `RestApiBindAddress` and, for any address but loopback, `RestApiCertificateFile` and `RestApiPrivateKeyFile` (or let an ACME certificate stand in), then `sudo systemctl restart hmailserver`. [The Control Deck](/topic/28192) topic has the details. 2. **Give each person a mailbox of their own**; the role goes to that account. 3. **Decide the domains.** With none named, `helpdesk` reaches only the account's own domain; name several, comma separated, for a desk serving more. 4. **Give the role** on any surface above and check it with `GET /api/v1/role-assignments` or the Roles view. 5. **The person signs in** to the Deck at `https://:8045/` as themselves, then uses *Reset password* or *Unlock* on an account. 6. **Tell them the limits:** no mail reading, no reports, no domain administrator's or auditor's password, no removing a ban, and, when confined, no clearing of the logon-failure list. Each change the help desk makes is on the audit trail as its own, over COM and REST, with actor type `account` and a detail ending `[role: helpdesk]`. ### What a refusal says - **A permission the role lacks:** `403` naming both, for example `the role 'helpdesk' does not hold the permission 'manage_settings'`. - **A domain it does not reach:** `the role 'helpdesk' does not reach that domain`, **whether or not the account exists**, so nobody can list another domain's mailboxes one request at a time. Over COM, `Application.ResetPassword` and `UnlockAccount` answer `No such account.` for both. - **An account wider than the credential:** *that account holds authority this credential does not*. - **Over COM,** since 6.3.4 a refusal carries the server's sentence rather than a bare `Exception from HRESULT: 0x800403E9`. - **An API key on the key store, the `[Settings]` routes, the audit trail or the alerts:** `401` with `{"error":"authentication failed"}`, the same as "not a key". ### The two stores only the administrator password opens The **key store** (`/api/v1/apikeys`) and the **`[Settings]` section of `hMailServer.ini`** (`/api/v1/settings/ini`) answer to the administrator password and nothing else: not an account holding `serveradmin`, not a key of any role or scope. A key that could mint keys could issue itself an unrestricted replacement, and the file holds the server's secrets. An account is refused `403` with the reason, a key `401`. The **audit trail** (`/api/v1/audit`) and the **alerts** (`/api/v1/alerts/...`) are refused to every API key with the same `401`, so a stolen key cannot read what it did or silence the alerting. An account whose role reaches every domain reaches them through `view_audit`, `view_configuration`, `manage_settings` or `manage_server`; a confined role is refused them. ### Upgrades, the audit trail and backups **The upgrade.** Schema step 6047 adds `hm_roles`, seeded with the four built-in rows, and `hm_roleassignments`, and writes no assignment. Roles are data, not settings: nothing goes into `hm_settings` or `hMailServer.ini`. **The audit trail.** Every role created, changed or deleted, and every role given or taken away, is a row naming the credential that did it; an assignment's row reads like `role: helpdesk -> auditor; domains: the account's own domain -> every domain`. The actor type follows the role: the administrator password and holders of `serveradmin` or `domainadmin` are `administrator`, every other account is `account`. **Backup and restore.** An account's assignment travels with it, by role name, and the custom roles travel whenever the domains or the settings do, restored first. A role the restoring server already has is kept as it is, with any difference named in the backup log. The admin level waits until the assignment is written, so a restore that stops part-way leaves no account wider than its role. An assignment naming a role neither side has is dropped, the level stays withheld, and `HM6129` names the account. No archive carries the API key store. ### What it does not do - **No mapping from an identity provider.** There is no claims mapping; an identity provider saying somebody is an administrator is not obeyed. The server's roles and `oidc_administrators` or `saml_administrators` decide. - **No role on an app password or a support session.** - **No Control Panel or COM on Linux:** roles are given there in the Deck and over REST. ### Where to read more - [Roles.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Roles.md), the authority on all of this. - [Sign-in and Identity](https://gitlab.com/hmailserver/hmailserver/-/wikis/Sign-in-and-Identity) §7, [The Control Deck](https://gitlab.com/hmailserver/hmailserver/-/wikis/The-Control-Deck), [The REST API](https://gitlab.com/hmailserver/hmailserver/-/wikis/The-REST-API) §2 and [The COM API and Scripting](https://gitlab.com/hmailserver/hmailserver/-/wikis/The-COM-API-and-Scripting) §23.2. - [AlertsAndAuditTrail.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/AlertsAndAuditTrail.md), [Backup.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Backup.md) and [Reports.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Reports.md). - On this forum: [The Control Deck: browser administration at /](/topic/28192), [The REST API: turning it on, credentials and roles](/topic/28264), [The COM API on Windows](/topic/28262), [Hardening a public mail server](/topic/28241), [Alerts and the audit trail](/topic/28273) and [IP ranges, auto-ban and lockout](/topic/28239). --- Canonical: https://www.hmailserver.co.uk/topic/28238/how-the-service-is-built-and-what-runs-out-first-pools-connections-large-folders # How the service is built and what runs out first: pools, connections, large folders *Written for 6.3.4. hMailServer 6.3.7 (17 October 2026) changes three things this guide touches. On MySQL and MariaDB a lost database connection is reconnected and the statement run again, so the statement after a database restart, a failover or `wait_timeout` no longer fails, and an insert that forgives a duplicate key forgives that alone, so an error log that was quiet may now name failures that were hidden. On Windows a stop or a reinitialise no longer waits past a work queue's ten seconds for a thread held by a host or a database that has stopped answering, and setup, the live update's silent install and the uninstaller wait `ShutdownDrainSeconds` plus 90 seconds for the stop, where they waited plus 30. The ACME HTTP-01 challenge responder, which served one connection at a time, serves up to 64 at once. See [the 6.3.7 announcement](/topic/28277).* This topic describes a running hMailServer 6.3.4 service: its parts and pools, the connection ceilings in front of them, the scheduled work behind them, what the machine needs, and what a folder of a hundred thousand messages costs the webmail. The one thing to know first: **nothing in the server is unbounded.** There is no thread per connection and no pool that grows. Every piece of work runs on a pool of a set size, and a dependency that stops answering uses up a pool long before it uses up the processor, the memory or the disk. What it cannot give you is a throughput figure: none has been measured yet. ### The parts of the running service Everything runs in one process: the Windows service `hMailServer`, or the systemd unit `hmailserver` on Linux. | Part | What it is | Notes | |---|---|---| | Mail listeners | SMTP, POP3 and IMAP on `IOService` + `TCPServer` | Boost.Asio | | `HttpServer` | The engine under two listeners: REST (`/api/v1`, the webmail at `/portal`, the Control Deck) and web services (MTA-STS policy, autoconfig, ACME http-01) | Its own `io_context`, so a request storm cannot starve SMTP accept. Since 6.2.28 | | Metrics listener | `/metrics` and the health probes | Raw sockets, one worker thread | | ManageSieve | Sieve script management | Raw sockets, a thread of its own | | SNMP agent | Read-only, UDP 161 | Since 6.3.4; off, and answering nobody until `SnmpAllowedAddresses` names a manager | | Background | `Scheduler`, `SMTPDeliveryManager`, `ExternalFetchManager` | Long-lived tasks on the main server queue | | State | The database (accounts, domains, message metadata and settings, since 6.3.4 including the `[Settings]` keys), one file per message, and `hMailServer.ini` as the bootstrap | The database stays small next to the store | **The handover that matters for capacity:** an SMTP session never delivers anything. The final `250` is sent from a worker on the asynchronous task queue once the message is written, and delivery happens later on a delivery thread, so the accepting side and the delivering side run out separately. [Diagnosing stalled mail](/topic/28219) takes each half in turn; [How email works](/topic/28243) follows one message through both. ### The pools, their sizes, and what a full one looks like Seven work queues are created at startup, and the two HTTP listeners have workers of their own. The first column is the `queue` label of `hmailserver_workqueue_depth{queue}` on the metrics endpoint, which counts tasks accepted and not yet started; the last row is not a work queue and has no label. | Queue | Threads | Sized by | What runs on it | When it is full | |---|---|---|---|---| | `IOCPQueue` | 15 | `tcpipthreads` | Every socket read, write and TLS handshake for SMTP, POP3 and IMAP | New connections are accepted but nothing progresses; clients time out mid-command | | `Asynchronous task queue` | 15 | `MaxNumberOfAsynchronousTasks` | Message finalization, which sends the final `250` | Senders sit after `CRLF.CRLF` with no reply | | `Name lookup queue` | 15 | the same setting | Blocking reverse-DNS lookups for live sessions | Reverse names become "Unknown", so a dead reverse zone cannot stall acknowledgements | | `SMTP delivery queue` | 10 | `maxdelivertythreads` | The delivery of one queued message | The queue grows and mail is late; nothing is lost, since the sender already has its `250` | | `External fetch queue` | 15 | `MaxNumberOfExternalFetchThreads` | One run per fetch account | Fetch accounts fall behind their schedule | | `Maintenance queue` | 5, fixed | nothing | One-shot scheduled tasks; a backup started over REST | A slow sweep delays the other one-shot sweeps | | `Server queue` | 4, fixed | nothing | `Scheduler`, `IOService`, `SMTPDeliveryManager`, `ExternalFetchManager` | Cannot fill in normal operation: each task holds one slot for the life of the server | | `HttpServer` workers | 4 per listener, fixed | nothing | One REST or web-services request each | Requests queue behind the four; SMTP is unaffected | **Where the four sizes are set.** On Windows, the Control Panel's **Maintenance → Performance** page, *Threads* tab, holds the COM `Settings` properties `TCPIPThreads`, `MaxAsynchronousThreads` and `MaxDeliveryThreads`, and the `[Settings]` key `MaxNumberOfExternalFetchThreads`. On either platform the first three are in the server settings group: the Control Deck's Settings view, `PUT /api/v1/settings` with `tcpip_threads`, `max_asynchronous_threads` and `max_delivery_threads`, or `hmctl settings set`. The fourth goes through `/api/v1/settings/ini/{name}` (administrator password only), `hmctl ini set`, or on Linux `hmailserver --set-setting` as root. Since 6.3.4 the database is the settings store, so do not edit the `[Settings]` line in `hMailServer.ini`: a line edited there is ignored in favour of the stored value. [Settings: when a save takes effect](/topic/28222) covers the store, what becomes of the file's line (HM5804) and when each key applies. The tab also shows `WorkerThreadPriority`, read-only: the server stores that value and never reads it. **When a new size takes effect.** Since 6.3.4 a save applies most settings at once, but `MaxNumberOfExternalFetchThreads` waits for a reinitialise, because the queue's size is set when the fetch manager is built; `GET /api/v1/server/settings/pending` lists it until then. For the other three the documents leave a gap: the Control Panel reports a save of its COM settings as "applied immediately", while the Architecture page has the pools created when the servers start, and no document says when a changed count reaches a running pool. Until one does, treat all four alike: after saving, reinitialise (the Control Panel's Reinitialize or **Apply now**, `POST /api/v1/server/reinitialize`, or `hmctl api POST /api/v1/server/reinitialize`) or restart the service. A reinitialise stops and starts every service in place, ends the Control Deck's session, and ends a drain. ```bash # The REST listener on this machine: read the server settings group, # then give delivery 25 threads (or: hmctl settings set --set max_delivery_threads=25) curl -u Administrator:pw http://127.0.0.1:8443/api/v1/settings curl -u Administrator:pw -X PUT -H "Content-Type: application/json" \ -d '{"max_delivery_threads":25}' http://127.0.0.1:8443/api/v1/settings # The fetch pool is a [Settings] key; it waits for a reinitialise hmctl ini set MaxNumberOfExternalFetchThreads 20 hmctl api POST /api/v1/server/reinitialize ``` **Two asynchronous threads are kept back.** Finalization runs the spam tests and event scripts, which wait on things outside the process, so `AsyncQueueReservedThreads` (2) keeps that many threads free of it: with the defaults at most 13 finalizations run at once. A task waiting for a slot holds no thread and is counted by `hmailserver_workqueue_blocking_tasks_waiting{queue}`; a count that stays above zero means something those tasks depend on is slow. On Windows the key is on **Server limits & expert settings**, card *Timeouts and queue bounds*; anywhere, `hmctl ini set`. **Only two queues report a stall**: the asynchronous and name lookup queues write the "every thread busy" report that [Diagnosing stalled mail](/topic/28219) reads. The others are left out deliberately, since their tasks are long-lived by design, so a full `IOCPQueue`, `SMTP delivery queue` or `External fetch queue` shows only in `hmailserver_workqueue_depth{queue}` and in its symptoms. ### The database connections every pool shares The deliveries, the sign-ins and the rest of the server share one pool of database connections, `NumberOfConnections` in the `[Database]` section of `hMailServer.ini`, 5 as shipped. A wait for one longer than `DBConnectionAcquireTimeout` (60 seconds) fails. In `hmailserver_db_connections{state}`, `available` pinned at zero with `busy` at the pool size is saturation, and every mail path is then waiting out that timeout. - **Raise it with the delivery threads.** The relay profile sets `maxdelivertythreads` to 25 and the sender profile to 50, and each tells you to raise `NumberOfConnections` to about the delivery thread count. - **The built-in database has one connection.** On Windows, `NumberOfConnections` is forced to 1 for `MSSQLCE`. Before You Install calls the built-in engine adequate for anything up to a few dozen mailboxes, and its single connection the first thing to change if the server feels slow under concurrency. Linux has no built-in database; it uses PostgreSQL or MariaDB. [Choosing a database](/topic/28228) covers the choice. - **It is a bootstrap key.** `[Database]` stays in the file, never in the database, and its keys wait for a restart. On Windows the Performance page's *Database* tab edits it, on the server machine only. On Linux: ```bash sudoedit /etc/hmailserver/hMailServer.ini # [Database] NumberOfConnections=25 sudo systemctl restart hmailserver ``` ### Connections and requests: the ceilings in front of the pools | Where | Limit | As shipped | Set on | |---|---|---|---| | SMTP sessions | `MaxSMTPConnections` | 0, no cap | **Connections & protocols → Protocols**, SMTP tab; `max_smtp_connections` | | IMAP sessions | `MaxIMAPConnections` | 0, no cap | The IMAP tab; `max_imap_connections` | | POP3 sessions | `MaxPOP3Connections` | not given in the documents; `GET /api/v1/settings` shows it | The POP3 tab; `max_pop3_connections` | | Recipients in one message | 50,000 | 50,000 | No setting is named; beyond it, `550 Too many recipients` | | REST listener, with the webmail and the Deck | 64 connections, closed at accept before any TLS handshake; 4 workers; 64 KB a request (16 MB, with 300 seconds, on the two attachment routes); 30 seconds a request; 300 seconds a connection | fixed | no key | | Web-services listener | 128 connections; 4 workers; 64 KB a request; 15 seconds a request; 120 seconds a connection | fixed | no key | **The webmail shares the REST listener**: its users, the Control Deck and every script that calls the API meet the same 64 connections and four workers. Its request rate is in [the topic on the REST API](/topic/28264). Outbound, since 6.3.4, `MaxOutboundConnsPerDestination` and `MaxOutboundConnsPerTenant` (both 0, off) cap sessions per destination and per sending domain; a tenant share larger than the delivery thread count is no share at all ([Outbound shaping and IP warm-up](/topic/28259)). ### Scheduled work The `Scheduler` polls its recurring tasks once a minute; one-shot tasks go through the `Maintenance queue` at once. External fetch is not one of them: the `ExternalFetchManager` runs its own one-minute loop. `Roadmap.md` recorded in August 2026 that a due task runs on the scheduler's own thread, so one long task delays the others, and that the real period is the interval plus the run time. The tasks, grouped by cadence: | Task | Cadence | Registered | |---|---|---| | `GreyListCleanerTask` | every `GreylistingRecordExpirationInterval` minutes (240 as shipped) | when SMTP is on | | `RemoveExpiredRecords`, `WorkQueueHealthTask` | every minute | always; the second writes the stall report | | `TlsRptReporterTask`, `DmarcRptReporterTask` | hourly | always; each sends only when its from-address is set | | `LogRetentionTask`, `MailboxRetentionTask` | start + 6 hours | always | | `ArchiveRetentionTask`, `IMAPExpungeRetentionTask` | start + 12 hours | always | | `MetricsHistoryTask` | start + every minute | always | | `UpdateCheckTask` | start + every 15 minutes | always; does nothing until `UpdateCheckEnabled=1` | | `MessageStoreConsistencyTask`, `DiskSpaceMonitorTask` | start + hourly | always | | `BackupScheduleTask` | a one-minute tick against the wall clock | only when a schedule is set | | `DirectorySyncScheduleTask` | start + every `[LDAP] SyncScheduleMinutes` | only when that is set | | `AcmeRenewalTask` | start + hourly | only when `AcmeEnabled=1` | ### Sizing the machine **Memory.** Before You Install asks for 2 GB as a minimum on Windows, more for a busy server. It is guidance; nothing enforces it, and memory per session has not been measured. **Disk.** Size it for how much mail you will keep, not how much you have today. The store is `Data\` under the program folder on Windows and `/var/lib/hmailserver` on Linux, one file per message. For scale, the webmail bench's hundred thousand messages, 5 % of them with an attachment of 20 to 512 KB, are 1,323,987,827 bytes. Below `MinimumFreeDiskSpaceMB` (100) new mail is refused temporarily, `452 4.3.1` over SMTP and `NO [UNAVAILABLE]` to an IMAP `APPEND`, so senders retry; below `DiskSpaceWarningThresholdMB` (1024) the server logs a warning and keeps accepting. **Antivirus.** On Windows, exclude `Data\`, `Database\` and `Temp\` from real-time scanning. A scanner holding a message file open is a documented cause of delivery that silently does nothing; the server detects one shape of it and logs error 5760. Before You Install names no exclusions for Linux, where the store is `/var/lib/hmailserver` with its `temp` and `database` subdirectories. **Durability.** `MessageStoreFsync` (0 as shipped) flushes the spool file to disk before the `250`, and a failed flush refuses the message. The published-numbers method runs with it on and labels the shipped default "not durable". Set it on Windows on **Server limits & expert settings** (card *Message store durability*), or anywhere with `hmctl ini set`. ### A folder of a hundred thousand messages in the webmail Every webmail path reads the folder's shared message collection, the one IMAP, POP3 and delivery use: one small object per message. Until 6.3.4 opening a folder copied the whole of it, under the lock IMAP sessions and delivery wait on, for every page of 200. Since 6.3.4: | Operation | What it does now | What still grows with the folder | |---|---|---| | Open, `GET /api/v1/me/folders/{id}/messages` | One pass under the lock counts the folder and copies only the page of 200; each row's file head is read twice, not three times | That pass; and the first open after a start, or after thirty idle minutes in the cache, reads every row of the folder from the database, as an IMAP `SELECT` does | | Scroll | Older and Newer by `before_uid`; from three pages up, Oldest in one request (`oldest_before_uid`) | Every page costs what the first costs, wherever it is in the folder: the one pass that counts the folder | | Search | At most 2,000 message files and about two and a half seconds a request, then answers where to go on from; across every folder it walks the mailbox newest first and answers `next_before_id`; with `IndexerFullText` and message indexing on (off as shipped), a message the index rules out is passed without opening its file | `Performance.md` does not say | | Select-all and one action | The selection goes to `POST /api/v1/me/folders/{id}/bulk`: at most 1,000 messages a call for a flag, 500 for a move or a delete, nothing new taken on after about two and a half seconds | A changed flag is a row written, a move is a file copied, and a selection narrowed to an inbox tab reads the head of every message | **Junk over a whole folder is a hundred thousand lessons.** With SpamAssassin on, each message moved into or out of Junk queues one lesson, as the same move over IMAP does. In the Settings Reference the lesson on a move is `SpamAssassinLearnOnMove` (off as shipped), run as a task on the asynchronous work queue. **A move holds the mail twice, briefly**: original and copy, both counted against the quota, for no longer than one slice of the work. **Not changed.** The Storage, Subscriptions and Scheduled views still copy each folder they look at; emptying Junk or Trash copies the folder once per 500 messages removed; a quick step over a whole-folder selection still reads at most five thousand messages. **For other callers of `GET /api/v1/me/search`**, it now answers `complete: false, more: true` with `next_before_id` whenever it is not the end, and its hits come newest first across every folder, where they came folder by folder. **Over IMAP**, one `SEARCH` or `SORT` is bounded by `IMAPSearchTimeout` (60 seconds) and `IMAPSearchMaxMegabytes` (2048 MB examined). [The webmail portal at /portal](/topic/28191) covers the rest of the client. ### What has not been measured **The webmail at a hundred thousand.** `Performance.md` has the method and a results table for the Windows bench (SQL Server Compact, 100,000 messages in the inbox, the full-text index off and on). Every cell reads *to be measured on the bench*; nothing was estimated. Its tools, `build/perf/seed-folder.py` and `build/perf/webmail-timing.py`, are in the repository and also run against a Linux server. Record the machine, database, message store, index state and build with any numbers. **How much the server carries.** `PublishedNumbers.md` defines messages in and out per second, latency percentiles, IMAP sessions per core with every operation's 95th percentile under 200 ms, memory per session and Dovecot's imaptest, measured against Postfix and Dovecot on the same machine under ten fairness rules. Its results say **Not measured yet**, and until they are filled the first line of `Roadmap2.md` is a goal, not a claim. The load generator, in the repository since 6.3.4, is `build/perf/load/loadgen.py`; a run in which the generator was busy for more than 85 per cent of it gives only a floor and is not published. The Windows bench gives hMailServer's own numbers only, not to be compared with the Linux table. ```bash python build/perf/load/loadgen.py --help python build/perf/load/loadgen.py self-test # proves the generator, says nothing about any server ``` Size from the ceilings above, watch the queue depths and the database pool, and treat any throughput figure quoted elsewhere as unmeasured by the project. ### What it does not do - No document gives a rule for sizing the pools to a machine; the only sized examples are the relay and sender profiles' 25 and 50 delivery threads. - The main server queue, the maintenance queue and the HTTP listeners' workers, connection caps and deadlines are fixed; no key changes them. ### Where to read more - Wiki: [Architecture](https://gitlab.com/hmailserver/hmailserver/-/wikis/Architecture), [Before You Install](https://gitlab.com/hmailserver/hmailserver/-/wikis/Before-You-Install), [Control Panel Reference](https://gitlab.com/hmailserver/hmailserver/-/wikis/Control-Panel-Reference) section 7.8, [Settings Reference](https://gitlab.com/hmailserver/hmailserver/-/wikis/Settings-Reference), [The REST API](https://gitlab.com/hmailserver/hmailserver/-/wikis/The-REST-API) section 8, [APIs Reference](https://gitlab.com/hmailserver/hmailserver/-/wikis/APIs-Reference) section 1 and [Monitoring and Health](https://gitlab.com/hmailserver/hmailserver/-/wikis/Monitoring-and-Health). - Documents: [Performance.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Performance.md), [PublishedNumbers.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/PublishedNumbers.md), [SettingsApply.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SettingsApply.md), [Profiles.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Profiles.md) and [OutboundShaping.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/OutboundShaping.md). - The public documentation: [Before you install](https://www.progressiverobot.com/hmailserver-documentation/#3-before-you-install-the-checklist), [A database](https://www.progressiverobot.com/hmailserver-documentation/#35-a-database) and [Prometheus metrics and health probes](https://www.progressiverobot.com/hmailserver-documentation/#163-prometheus-metrics-and-health-probes). - On this forum: [Diagnosing stalled mail](/topic/28219), [Reading the logs](/topic/28220), [Monitoring and health](/topic/28272), [The webmail in depth](/topic/28254), [The Control Deck](/topic/28192) and [hMailServer on Linux](/topic/28189). Without a GitLab account, a bug or a question can go to the Service Desk address contact-project+hmailserver-hmailserver-86730559-issue-@incoming.gitlab.com, which GitLab files as a confidential issue; the replies arrive by email. --- Canonical: https://www.hmailserver.co.uk/topic/28238/how-the-service-is-built-and-what-runs-out-first-pools-connections-large-folders # How the service is built and what runs out first: pools, connections, large folders *Written for 6.3.4. hMailServer 6.3.7 (17 October 2026) changes three things this guide touches. On MySQL and MariaDB a lost database connection is reconnected and the statement run again, so the statement after a database restart, a failover or `wait_timeout` no longer fails, and an insert that forgives a duplicate key forgives that alone, so an error log that was quiet may now name failures that were hidden. On Windows a stop or a reinitialise no longer waits past a work queue's ten seconds for a thread held by a host or a database that has stopped answering, and setup, the live update's silent install and the uninstaller wait `ShutdownDrainSeconds` plus 90 seconds for the stop, where they waited plus 30. The ACME HTTP-01 challenge responder, which served one connection at a time, serves up to 64 at once. See [the 6.3.7 announcement](/topic/28277).* This topic describes a running hMailServer 6.3.4 service: its parts and pools, the connection ceilings in front of them, the scheduled work behind them, what the machine needs, and what a folder of a hundred thousand messages costs the webmail. The one thing to know first: **nothing in the server is unbounded.** There is no thread per connection and no pool that grows. Every piece of work runs on a pool of a set size, and a dependency that stops answering uses up a pool long before it uses up the processor, the memory or the disk. What it cannot give you is a throughput figure: none has been measured yet. ### The parts of the running service Everything runs in one process: the Windows service `hMailServer`, or the systemd unit `hmailserver` on Linux. | Part | What it is | Notes | |---|---|---| | Mail listeners | SMTP, POP3 and IMAP on `IOService` + `TCPServer` | Boost.Asio | | `HttpServer` | The engine under two listeners: REST (`/api/v1`, the webmail at `/portal`, the Control Deck) and web services (MTA-STS policy, autoconfig, ACME http-01) | Its own `io_context`, so a request storm cannot starve SMTP accept. Since 6.2.28 | | Metrics listener | `/metrics` and the health probes | Raw sockets, one worker thread | | ManageSieve | Sieve script management | Raw sockets, a thread of its own | | SNMP agent | Read-only, UDP 161 | Since 6.3.4; off, and answering nobody until `SnmpAllowedAddresses` names a manager | | Background | `Scheduler`, `SMTPDeliveryManager`, `ExternalFetchManager` | Long-lived tasks on the main server queue | | State | The database (accounts, domains, message metadata and settings, since 6.3.4 including the `[Settings]` keys), one file per message, and `hMailServer.ini` as the bootstrap | The database stays small next to the store | **The handover that matters for capacity:** an SMTP session never delivers anything. The final `250` is sent from a worker on the asynchronous task queue once the message is written, and delivery happens later on a delivery thread, so the accepting side and the delivering side run out separately. [Diagnosing stalled mail](/topic/28219) takes each half in turn; [How email works](/topic/28243) follows one message through both. ### The pools, their sizes, and what a full one looks like Seven work queues are created at startup, and the two HTTP listeners have workers of their own. The first column is the `queue` label of `hmailserver_workqueue_depth{queue}` on the metrics endpoint, which counts tasks accepted and not yet started; the last row is not a work queue and has no label. | Queue | Threads | Sized by | What runs on it | When it is full | |---|---|---|---|---| | `IOCPQueue` | 15 | `tcpipthreads` | Every socket read, write and TLS handshake for SMTP, POP3 and IMAP | New connections are accepted but nothing progresses; clients time out mid-command | | `Asynchronous task queue` | 15 | `MaxNumberOfAsynchronousTasks` | Message finalization, which sends the final `250` | Senders sit after `CRLF.CRLF` with no reply | | `Name lookup queue` | 15 | the same setting | Blocking reverse-DNS lookups for live sessions | Reverse names become "Unknown", so a dead reverse zone cannot stall acknowledgements | | `SMTP delivery queue` | 10 | `maxdelivertythreads` | The delivery of one queued message | The queue grows and mail is late; nothing is lost, since the sender already has its `250` | | `External fetch queue` | 15 | `MaxNumberOfExternalFetchThreads` | One run per fetch account | Fetch accounts fall behind their schedule | | `Maintenance queue` | 5, fixed | nothing | One-shot scheduled tasks; a backup started over REST | A slow sweep delays the other one-shot sweeps | | `Server queue` | 4, fixed | nothing | `Scheduler`, `IOService`, `SMTPDeliveryManager`, `ExternalFetchManager` | Cannot fill in normal operation: each task holds one slot for the life of the server | | `HttpServer` workers | 4 per listener, fixed | nothing | One REST or web-services request each | Requests queue behind the four; SMTP is unaffected | **Where the four sizes are set.** On Windows, the Control Panel's **Maintenance → Performance** page, *Threads* tab, holds the COM `Settings` properties `TCPIPThreads`, `MaxAsynchronousThreads` and `MaxDeliveryThreads`, and the `[Settings]` key `MaxNumberOfExternalFetchThreads`. On either platform the first three are in the server settings group: the Control Deck's Settings view, `PUT /api/v1/settings` with `tcpip_threads`, `max_asynchronous_threads` and `max_delivery_threads`, or `hmctl settings set`. The fourth goes through `/api/v1/settings/ini/{name}` (administrator password only), `hmctl ini set`, or on Linux `hmailserver --set-setting` as root. Since 6.3.4 the database is the settings store, so do not edit the `[Settings]` line in `hMailServer.ini`: a line edited there is ignored in favour of the stored value. [Settings: when a save takes effect](/topic/28222) covers the store, what becomes of the file's line (HM5804) and when each key applies. The tab also shows `WorkerThreadPriority`, read-only: the server stores that value and never reads it. **When a new size takes effect.** Since 6.3.4 a save applies most settings at once, but `MaxNumberOfExternalFetchThreads` waits for a reinitialise, because the queue's size is set when the fetch manager is built; `GET /api/v1/server/settings/pending` lists it until then. For the other three the documents leave a gap: the Control Panel reports a save of its COM settings as "applied immediately", while the Architecture page has the pools created when the servers start, and no document says when a changed count reaches a running pool. Until one does, treat all four alike: after saving, reinitialise (the Control Panel's Reinitialize or **Apply now**, `POST /api/v1/server/reinitialize`, or `hmctl api POST /api/v1/server/reinitialize`) or restart the service. A reinitialise stops and starts every service in place, ends the Control Deck's session, and ends a drain. ```bash # The REST listener on this machine: read the server settings group, # then give delivery 25 threads (or: hmctl settings set --set max_delivery_threads=25) curl -u Administrator:pw http://127.0.0.1:8443/api/v1/settings curl -u Administrator:pw -X PUT -H "Content-Type: application/json" \ -d '{"max_delivery_threads":25}' http://127.0.0.1:8443/api/v1/settings # The fetch pool is a [Settings] key; it waits for a reinitialise hmctl ini set MaxNumberOfExternalFetchThreads 20 hmctl api POST /api/v1/server/reinitialize ``` **Two asynchronous threads are kept back.** Finalization runs the spam tests and event scripts, which wait on things outside the process, so `AsyncQueueReservedThreads` (2) keeps that many threads free of it: with the defaults at most 13 finalizations run at once. A task waiting for a slot holds no thread and is counted by `hmailserver_workqueue_blocking_tasks_waiting{queue}`; a count that stays above zero means something those tasks depend on is slow. On Windows the key is on **Server limits & expert settings**, card *Timeouts and queue bounds*; anywhere, `hmctl ini set`. **Only two queues report a stall**: the asynchronous and name lookup queues write the "every thread busy" report that [Diagnosing stalled mail](/topic/28219) reads. The others are left out deliberately, since their tasks are long-lived by design, so a full `IOCPQueue`, `SMTP delivery queue` or `External fetch queue` shows only in `hmailserver_workqueue_depth{queue}` and in its symptoms. ### The database connections every pool shares The deliveries, the sign-ins and the rest of the server share one pool of database connections, `NumberOfConnections` in the `[Database]` section of `hMailServer.ini`, 5 as shipped. A wait for one longer than `DBConnectionAcquireTimeout` (60 seconds) fails. In `hmailserver_db_connections{state}`, `available` pinned at zero with `busy` at the pool size is saturation, and every mail path is then waiting out that timeout. - **Raise it with the delivery threads.** The relay profile sets `maxdelivertythreads` to 25 and the sender profile to 50, and each tells you to raise `NumberOfConnections` to about the delivery thread count. - **The built-in database has one connection.** On Windows, `NumberOfConnections` is forced to 1 for `MSSQLCE`. Before You Install calls the built-in engine adequate for anything up to a few dozen mailboxes, and its single connection the first thing to change if the server feels slow under concurrency. Linux has no built-in database; it uses PostgreSQL or MariaDB. [Choosing a database](/topic/28228) covers the choice. - **It is a bootstrap key.** `[Database]` stays in the file, never in the database, and its keys wait for a restart. On Windows the Performance page's *Database* tab edits it, on the server machine only. On Linux: ```bash sudoedit /etc/hmailserver/hMailServer.ini # [Database] NumberOfConnections=25 sudo systemctl restart hmailserver ``` ### Connections and requests: the ceilings in front of the pools | Where | Limit | As shipped | Set on | |---|---|---|---| | SMTP sessions | `MaxSMTPConnections` | 0, no cap | **Connections & protocols → Protocols**, SMTP tab; `max_smtp_connections` | | IMAP sessions | `MaxIMAPConnections` | 0, no cap | The IMAP tab; `max_imap_connections` | | POP3 sessions | `MaxPOP3Connections` | not given in the documents; `GET /api/v1/settings` shows it | The POP3 tab; `max_pop3_connections` | | Recipients in one message | 50,000 | 50,000 | No setting is named; beyond it, `550 Too many recipients` | | REST listener, with the webmail and the Deck | 64 connections, closed at accept before any TLS handshake; 4 workers; 64 KB a request (16 MB, with 300 seconds, on the two attachment routes); 30 seconds a request; 300 seconds a connection | fixed | no key | | Web-services listener | 128 connections; 4 workers; 64 KB a request; 15 seconds a request; 120 seconds a connection | fixed | no key | **The webmail shares the REST listener**: its users, the Control Deck and every script that calls the API meet the same 64 connections and four workers. Its request rate is in [the topic on the REST API](/topic/28264). Outbound, since 6.3.4, `MaxOutboundConnsPerDestination` and `MaxOutboundConnsPerTenant` (both 0, off) cap sessions per destination and per sending domain; a tenant share larger than the delivery thread count is no share at all ([Outbound shaping and IP warm-up](/topic/28259)). ### Scheduled work The `Scheduler` polls its recurring tasks once a minute; one-shot tasks go through the `Maintenance queue` at once. External fetch is not one of them: the `ExternalFetchManager` runs its own one-minute loop. `Roadmap.md` recorded in August 2026 that a due task runs on the scheduler's own thread, so one long task delays the others, and that the real period is the interval plus the run time. The tasks, grouped by cadence: | Task | Cadence | Registered | |---|---|---| | `GreyListCleanerTask` | every `GreylistingRecordExpirationInterval` minutes (240 as shipped) | when SMTP is on | | `RemoveExpiredRecords`, `WorkQueueHealthTask` | every minute | always; the second writes the stall report | | `TlsRptReporterTask`, `DmarcRptReporterTask` | hourly | always; each sends only when its from-address is set | | `LogRetentionTask`, `MailboxRetentionTask` | start + 6 hours | always | | `ArchiveRetentionTask`, `IMAPExpungeRetentionTask` | start + 12 hours | always | | `MetricsHistoryTask` | start + every minute | always | | `UpdateCheckTask` | start + every 15 minutes | always; does nothing until `UpdateCheckEnabled=1` | | `MessageStoreConsistencyTask`, `DiskSpaceMonitorTask` | start + hourly | always | | `BackupScheduleTask` | a one-minute tick against the wall clock | only when a schedule is set | | `DirectorySyncScheduleTask` | start + every `[LDAP] SyncScheduleMinutes` | only when that is set | | `AcmeRenewalTask` | start + hourly | only when `AcmeEnabled=1` | ### Sizing the machine **Memory.** Before You Install asks for 2 GB as a minimum on Windows, more for a busy server. It is guidance; nothing enforces it, and memory per session has not been measured. **Disk.** Size it for how much mail you will keep, not how much you have today. The store is `Data\` under the program folder on Windows and `/var/lib/hmailserver` on Linux, one file per message. For scale, the webmail bench's hundred thousand messages, 5 % of them with an attachment of 20 to 512 KB, are 1,323,987,827 bytes. Below `MinimumFreeDiskSpaceMB` (100) new mail is refused temporarily, `452 4.3.1` over SMTP and `NO [UNAVAILABLE]` to an IMAP `APPEND`, so senders retry; below `DiskSpaceWarningThresholdMB` (1024) the server logs a warning and keeps accepting. **Antivirus.** On Windows, exclude `Data\`, `Database\` and `Temp\` from real-time scanning. A scanner holding a message file open is a documented cause of delivery that silently does nothing; the server detects one shape of it and logs error 5760. Before You Install names no exclusions for Linux, where the store is `/var/lib/hmailserver` with its `temp` and `database` subdirectories. **Durability.** `MessageStoreFsync` (0 as shipped) flushes the spool file to disk before the `250`, and a failed flush refuses the message. The published-numbers method runs with it on and labels the shipped default "not durable". Set it on Windows on **Server limits & expert settings** (card *Message store durability*), or anywhere with `hmctl ini set`. ### A folder of a hundred thousand messages in the webmail Every webmail path reads the folder's shared message collection, the one IMAP, POP3 and delivery use: one small object per message. Until 6.3.4 opening a folder copied the whole of it, under the lock IMAP sessions and delivery wait on, for every page of 200. Since 6.3.4: | Operation | What it does now | What still grows with the folder | |---|---|---| | Open, `GET /api/v1/me/folders/{id}/messages` | One pass under the lock counts the folder and copies only the page of 200; each row's file head is read twice, not three times | That pass; and the first open after a start, or after thirty idle minutes in the cache, reads every row of the folder from the database, as an IMAP `SELECT` does | | Scroll | Older and Newer by `before_uid`; from three pages up, Oldest in one request (`oldest_before_uid`) | Every page costs what the first costs, wherever it is in the folder: the one pass that counts the folder | | Search | At most 2,000 message files and about two and a half seconds a request, then answers where to go on from; across every folder it walks the mailbox newest first and answers `next_before_id`; with `IndexerFullText` and message indexing on (off as shipped), a message the index rules out is passed without opening its file | `Performance.md` does not say | | Select-all and one action | The selection goes to `POST /api/v1/me/folders/{id}/bulk`: at most 1,000 messages a call for a flag, 500 for a move or a delete, nothing new taken on after about two and a half seconds | A changed flag is a row written, a move is a file copied, and a selection narrowed to an inbox tab reads the head of every message | **Junk over a whole folder is a hundred thousand lessons.** With SpamAssassin on, each message moved into or out of Junk queues one lesson, as the same move over IMAP does. In the Settings Reference the lesson on a move is `SpamAssassinLearnOnMove` (off as shipped), run as a task on the asynchronous work queue. **A move holds the mail twice, briefly**: original and copy, both counted against the quota, for no longer than one slice of the work. **Not changed.** The Storage, Subscriptions and Scheduled views still copy each folder they look at; emptying Junk or Trash copies the folder once per 500 messages removed; a quick step over a whole-folder selection still reads at most five thousand messages. **For other callers of `GET /api/v1/me/search`**, it now answers `complete: false, more: true` with `next_before_id` whenever it is not the end, and its hits come newest first across every folder, where they came folder by folder. **Over IMAP**, one `SEARCH` or `SORT` is bounded by `IMAPSearchTimeout` (60 seconds) and `IMAPSearchMaxMegabytes` (2048 MB examined). [The webmail portal at /portal](/topic/28191) covers the rest of the client. ### What has not been measured **The webmail at a hundred thousand.** `Performance.md` has the method and a results table for the Windows bench (SQL Server Compact, 100,000 messages in the inbox, the full-text index off and on). Every cell reads *to be measured on the bench*; nothing was estimated. Its tools, `build/perf/seed-folder.py` and `build/perf/webmail-timing.py`, are in the repository and also run against a Linux server. Record the machine, database, message store, index state and build with any numbers. **How much the server carries.** `PublishedNumbers.md` defines messages in and out per second, latency percentiles, IMAP sessions per core with every operation's 95th percentile under 200 ms, memory per session and Dovecot's imaptest, measured against Postfix and Dovecot on the same machine under ten fairness rules. Its results say **Not measured yet**, and until they are filled the first line of `Roadmap2.md` is a goal, not a claim. The load generator, in the repository since 6.3.4, is `build/perf/load/loadgen.py`; a run in which the generator was busy for more than 85 per cent of it gives only a floor and is not published. The Windows bench gives hMailServer's own numbers only, not to be compared with the Linux table. ```bash python build/perf/load/loadgen.py --help python build/perf/load/loadgen.py self-test # proves the generator, says nothing about any server ``` Size from the ceilings above, watch the queue depths and the database pool, and treat any throughput figure quoted elsewhere as unmeasured by the project. ### What it does not do - No document gives a rule for sizing the pools to a machine; the only sized examples are the relay and sender profiles' 25 and 50 delivery threads. - The main server queue, the maintenance queue and the HTTP listeners' workers, connection caps and deadlines are fixed; no key changes them. ### Where to read more - Wiki: [Architecture](https://gitlab.com/hmailserver/hmailserver/-/wikis/Architecture), [Before You Install](https://gitlab.com/hmailserver/hmailserver/-/wikis/Before-You-Install), [Control Panel Reference](https://gitlab.com/hmailserver/hmailserver/-/wikis/Control-Panel-Reference) section 7.8, [Settings Reference](https://gitlab.com/hmailserver/hmailserver/-/wikis/Settings-Reference), [The REST API](https://gitlab.com/hmailserver/hmailserver/-/wikis/The-REST-API) section 8, [APIs Reference](https://gitlab.com/hmailserver/hmailserver/-/wikis/APIs-Reference) section 1 and [Monitoring and Health](https://gitlab.com/hmailserver/hmailserver/-/wikis/Monitoring-and-Health). - Documents: [Performance.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Performance.md), [PublishedNumbers.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/PublishedNumbers.md), [SettingsApply.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SettingsApply.md), [Profiles.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Profiles.md) and [OutboundShaping.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/OutboundShaping.md). - The public documentation: [Before you install](https://www.progressiverobot.com/hmailserver-documentation/#3-before-you-install-the-checklist), [A database](https://www.progressiverobot.com/hmailserver-documentation/#35-a-database) and [Prometheus metrics and health probes](https://www.progressiverobot.com/hmailserver-documentation/#163-prometheus-metrics-and-health-probes). - On this forum: [Diagnosing stalled mail](/topic/28219), [Reading the logs](/topic/28220), [Monitoring and health](/topic/28272), [The webmail in depth](/topic/28254), [The Control Deck](/topic/28192) and [hMailServer on Linux](/topic/28189). Without a GitLab account, a bug or a question can go to the Service Desk address contact-project+hmailserver-hmailserver-86730559-issue-@incoming.gitlab.com, which GitLab files as a confidential issue; the replies arrive by email. --- Canonical: https://www.hmailserver.co.uk/topic/28239/ip-ranges-auto-ban-lockout-tarpits-sending-limits-send-as-and-password-storage # IP ranges, auto-ban, lockout, tarpits, sending limits, send-as and password storage *Written for 6.3.4. hMailServer 6.3.7 (17 October 2026) is a security release that bears on the send-as and sign-in sections here. An authenticated submission is now DKIM-signed as another domain hosted here only when its account may send as the address its From names (its own address, an alias pointing at it, a mailbox that granted it the post right, an address it proved, or a masked address of its own, the addresses the send-as table below lists); otherwise the mail is still accepted and sent, unsigned as that domain, so give such an account the alias or the grant before upgrading. A domain's or an account's rule replies only from an address its owner may send as. SMTP, POP3 and ManageSieve refuse a SASL PLAIN sign-in that asks to act as another account; with SRS on, mail to an SRS address this server handed out is held to the relay rules unless it is a bounce; the IMAP and POP3 listeners can take the client's address from a PROXY protocol header as the SMTP listener does, each from a list of peers of its own, both off by default; and the Control Panel's IP ranges page, which could show no list of ranges at all, is fixed. See [the 6.3.7 announcement](/topic/28277).* This topic covers who may use this server and how hard it is to abuse it. Know this first: **only the IP ranges and auto-ban work out of the box; every other brake here is off until you set it.** Until you do, nothing else stops a botnet spending one guess per address, or a stolen password sending thousands of messages. The secure defaults of a new installation, TLS for sign-in and compromised-account detection are in [Hardening a public mail server](/topic/28241). ### Where each one is set Since 6.3.4 the database is the settings store, and for a key it holds, a `[Settings]` line edited in `hMailServer.ini` changes nothing; on 6.3.3 these keys were lines of the file, applied at the next start. The store, and when a save takes effect, are in [the settings topic](/topic/28222). | What | Kept in | Set with | Takes effect | |---|---|---|---| | IP ranges | the database | Control Panel **Access & abuse protection → IP ranges**; the Control Deck's IP ranges view; `/api/v1/ipranges`; `hmctl iprange`; COM `Settings.SecurityRanges` | at once | | Auto-ban's four settings | the database | Control Panel **Access & abuse protection → Auto-ban**; the Deck's settings; `PUT /api/v1/settings`; COM | at once | | Lockout, tarpits, rate limit, send-as check, hashing, policy | the settings store, under their `[Settings]` names | Control Panel; `PUT /api/v1/settings/ini/{name}`; `hmctl ini set `; COM `Settings.SetIniSetting`; on Linux `hmailserver --set-setting NAME=VALUE` | when saved | | `[SendingLimits]`, `[SendingLimitsOverrides]`, `PasswordPepper` | `hMailServer.ini` only | the file, by hand or through the Control Panel (below) | sending limits within seconds; the pepper after a restart | The Control Panel and COM are Windows-only. Of these `[Settings]` keys the Deck edits only `SmtpTarpitCount` and `SmtpTarpitDelaySeconds`. A value set with `hmailserver --set-setting` while the service runs waits for the next save, reinitialise or restart. ### IP ranges: priority and the relay grid When a client connects, every range containing its address is found and **the highest priority wins**. The shipped ranges are *Internet* (priority 10, `0.0.0.0`–`255.255.255.255`) and *My computer* (priority 30, `127.0.0.1`); a range added on the Control Panel's page starts at 15, so a narrow range sits in front of *Internet* without editing it. At every `RCPT TO` two decisions are made, in order: may this combination be delivered at all (if not, `550 Delivery is not allowed to this address.`), and must the client authenticate first (if so, `530 SMTP authentication is required.`). "Local" and "remote" describe the **addresses**, not the client: an address is local when its domain is a hosted active domain or a route says so. | Combination | REST fields: allowed / auth required | *Internet* | *My computer* | |---|---|---|---| | Local → local | `deliver_local_to_local` / `require_auth_local_to_local` | allowed, **auth required** | allowed, no auth | | Local → remote | `deliver_local_to_remote` / `require_auth_local_to_remote` | allowed, **auth required** | allowed, no auth | | Remote → local | `deliver_remote_to_local` / `require_auth_remote_to_local` | allowed, no auth | allowed, no auth | | Remote → remote (relay) | `deliver_remote_to_remote` / `require_auth_remote_to_remote` | allowed, **auth required** | allowed, **auth required** | | Anti-spam, anti-virus | `spam_protection`, `virus_protection` | on | on | | Require TLS for auth | `require_tls_for_auth` | on in a new 6.3.4 installation; otherwise as the server had it | off | **Remote → remote is the open-relay box.** Both shipped ranges allow it with authentication required, even from 127.0.0.1, so a client that has not signed in gets `530`; a range that does not allow it at all gets `550`. The Control Panel's Server status page raises a Critical warning for any range that allows it without authentication. **Remote → local** is inbound mail and must need no authentication. **Local → local** needs it on *Internet* so that nobody outside can forge mail from one of your addresses to another. **The protection switches reach further than their names.** Clearing *Enable anti-spam* turns off every spam test, greylisting and the recipient tarpit for the range; clearing *Enable anti-virus* means its mail is never flagged for scanning. Since 6.3.4 SPF gives loopback no pass, so a content filter that re-injects mail should authenticate or use a range with anti-spam off. **Require TLS for auth belongs to the range, not the port.** With it set, `AUTH` is not advertised on a cleartext connection from that range, on SMTP, POP3, IMAP and ManageSieve. **A device that cannot sign in** gets a range of its own above *Internet*. For a scanner at 192.0.2.50 that sends from your domains to your own mailboxes only: ```bash hmctl iprange create --set name=scanner --set lower=192.0.2.50 --set upper=192.0.2.50 \ --set priority=20 --set allow_smtp=true --set allow_imap=false --set allow_pop3=false \ --set deliver_local_to_local=true --set require_auth_local_to_local=false \ --set deliver_local_to_remote=false --set deliver_remote_to_local=false \ --set deliver_remote_to_remote=false --set spam_protection=true --set virus_protection=true ``` `hmctl iprange update ` and `PUT /api/v1/ipranges/{id}` (since 6.3.3) change any subset of the fields. The routes refuse a domain-restricted API key; a role needs `manage_settings`. **The Control Panel's Restore defaults** (`SecurityRanges.SetDefault()`) discards every range, your own and the auto-ban ones, and writes back the two shipped ones as the create scripts make them, without *Require TLS for auth*. ### Auto-ban, and why its ranges sit at priority 100 Auto-ban counts failed logons per **connecting address**; SMTP `AUTH`, POP3, IMAP, ManageSieve and the REST API feed one counter, but REST does not count a client on loopback (`127.0.0.1`, `::1`). **Behind a reverse proxy on the same machine** (the Linux package binds REST to `127.0.0.1`), the webmail's failed sign-ins are never auto-banned; account lockout, below, is what limits them there, and it is off until you set it. At the limit auto-ban clears that address's counted failures, creates an expiring range named `Auto-ban: ` at **priority 100**, and disconnects; SMTP says `535 Authentication failed. Too many invalid logon attempts.` Priority 100, above both shipped ranges, is what makes a ban effective; `AutoBanNeverBan`, below, exempts an address. | Setting (COM name) | Default | Trap | |---|---|---| | `AutoBanOnLogonFailure` | on | — | | `MaxInvalidLogonAttempts` | 3 | **`0` disables auto-ban entirely**, whatever the switch says | | `MaxInvalidLogonAttemptsWithin` | 30 minutes | Not a sliding window: how long a failure record survives, deleted by a pass every minute that runs only while auto-ban is on | | `AutoBanMinutes` | 60 | **`0` disconnects the client but creates no range**, so nothing is banned | Their REST keys include `auto_ban_on_logon_failure` and `max_invalid_logon_attempts`. The `office` profile (since 6.3.4) raises the limit to 5. - **`AutoBanNeverBan`** (since 6.3.2): addresses and CIDR blocks whose failures are never counted; the per-connection disconnect still applies, and a bad entry is reported once as `HM6524`. Server status warns "Localhost is currently banned in the IP ranges." when a ban covers 127.0.0.1. - **Control Panel sign-in over COM** is outside auto-ban and lockout, so guessing an administrator's mailbox name cannot lock them out of the tool. - **Spam traps and the multi-recipient null-sender check** ban through the same ranges (`Auto-ban: spam trap
`, `Auto-ban: multi-recipient null sender
`): see [Allow and block lists](/topic/28249) and [the anti-spam stack](/topic/28248). Firewall bans (`AutoBanFirewall`, `AutoBanCommand`, fail2ban) are in [Reading the logs](/topic/28220). **Releasing a banned address.** **Clear logon-failure list** (`POST /api/v1/settings/logon-failures/clear`, COM `Settings.ClearLogonFailureList()`) clears counted failures only, for every address at once; a ban stays until its range expires. To release one now: 1. Have the person correct the password in their client first: the counted failures were cleared when the ban was made, so a client still retrying the old one reaches the limit again as soon as the range is gone. 2. Find the range: the IP ranges page, the Deck's IP ranges view, `hmctl iprange list` or `GET /api/v1/ipranges`. Its name begins `Auto-ban:` and its priority is 100. 3. Delete it: on the page, `hmctl iprange delete `, or `DELETE /api/v1/ipranges/`. With `AutoBanFirewall` on, the firewall is reconciled on every range deletion, so its rule goes too. Five error-log codes mean the control has quietly stopped working: | Code | Meaning | |---|---| | `HM6114` | A failed logon could not be recorded, so no ban will trigger while this persists | | `HM6099` | The auto-ban range could not be saved: the address has **not** been blocked | | `HM6113` | Spent failures could not be cleared, so the address may be banned again as soon as its ban expires | | `HM6112` | Expired IP ranges could not be removed, so an expired ban keeps blocking | | `HM6115` | Expired failures could not be cleared, so occasional failures may add up to a ban | ### Account lockout, by name A distributed attack never crosses any one address's threshold. Lockout counts the **name** being guessed at, across SMTP `AUTH`, POP3, IMAP, ManageSieve, REST and the webmail's sign-in. It is off until set; the `office` profile sets 10. ```ini AccountLockoutThreshold=10 ; failed logons per username before a lockout. 0 = off AccountLockoutWindowMinutes=30 ; the window they must fall inside AccountLockoutMinutes=30 ; how long the lock lasts ``` - **It counts names, not accounts**, so a nonexistent name locks as a real one would, and reveals nothing. - **A locked name is refused before the password is checked**, with the ordinary invalid-credentials reply; only its timing differs, deliberately. - **A lockout refusal is not charged to the connecting address**, so an owner retrying from an office NAT does not get everyone behind it auto-banned. - **Passkeys and single sign-on are not counted**, being no guesses. - **To lift one early**: `POST /api/v1/accounts/{address}/unlock`, which a help desk scoped to the account's domain may call, COM `Application.UnlockAccount`, or **Unlock** on the Deck's Accounts row. A password reset clears it too. ### Tarpits and the submission rate limit All **0, off, until set**; these values are a starting point. ```ini LogonTarpitSeconds=2 ; per failure on this connection, capped at 30 s. 0 = off SmtpTarpitCount=10 ; recipients an unauthenticated session may name before... SmtpTarpitDelaySeconds=5 ; ...each further RCPT TO waits this long, capped at 30 s MaxSubmissionsPerIPPerMinute=30 ; MAIL FROM commands per source IP per minute. 0 = off ``` - **The logon tarpit** (Auto-ban page) makes a failed logon's refusal wait `LogonTarpitSeconds` times the failures so far on that connection, on SMTP, POP3 and IMAP. **A correct password never waits.** - **The recipient tarpit** (Anti-spam settings page; COM `AntiSpam.TarpitCount` and `AntiSpam.TarpitDelay`; REST `tarpit_count` and `tarpit_delay` on `/api/v1/settings/antispam`) spares authenticated sessions and ranges with anti-spam off. - **`MaxSubmissionsPerIPPerMinute` counts every `MAIL FROM`, authenticated or not**, and refuses the excess with `421 Too many messages from your IP address`, which contains a compromised account. The `office` profile sets 60; `gateway` keeps 0, since all inbound mail comes from the gateway's few addresses. Since 6.3.4 a save restarts its one-minute window. Behind a load balancer, give the SMTP listener `SMTPProxyProtocolEnabled` with `SMTPProxyProtocolTrustedIPs` (or `XCLIENT`), or auto-ban, the rate limit and the ranges all reason about the proxy's address: see [Behind a gateway or reverse proxy](/topic/28240). ### Per-account sending limits The brake on a compromised account, counted against the **authenticated user name** (never `MAIL FROM`) over a rolling period. The server level stays in the file: ```ini [SendingLimits] MaxMessagesPerAccountPerPeriod=500 MaxRecipientsPerAccountPerPeriod=2000 PeriodHours=24 ; clamped to 1-168 StateSaveIntervalSeconds=10 ; clamped to 1-3600 [SendingLimitsOverrides] newsletter@yourcompany.com=5000:50000 ceo@yourcompany.com=0:0 ``` Both limits default to 0, no limit; recipients are the stricter measure. An override replaces the global pair for that address, `0:0` exempts it, and an optional third field sets its period in hours; a malformed line is ignored with a log entry and the global limit applies. On Windows edit `C:\Program Files\hMailServer\Bin\hMailServer.INI` with an elevated editor, or use the Control Panel's **Maintenance → Server limits & expert settings** on the server itself; on Linux, `sudoedit /etc/hmailserver/hMailServer.ini`. They are not in the settings store, so no environment variable or setting profile writes them. **What the sender sees**: `452 4.7.0 Sending limit for this account has been reached. Please try again later.` at `MAIL FROM`, or `452 4.5.3 Recipient sending limit for this account has been reached. Please try again later.` at `RCPT TO`. The session stays open, the refusal spends no budget, and the application log, not the error log, says `SendingLimit refused submission`. Counters survive a restart in `hmailserver_sendinglimits.dat` in the data directory. **It fails open**: an unreadable file, a limiter exception or a full table of tracked accounts (error 5813) lets the message through. **Since 6.3.4 there are three levels.** A domain and an account may hold `sending_max_messages` and `sending_max_recipients`, which **tighten** the level above and never loosen it; an address's override line is its server level. Set them on the **Levels** tab of the domain and account dialogs, in the Deck's editors, with `PUT /api/v1/accounts/{address}`, or `hmctl account set
sending_max_messages 100` (`inherit` returns to the level above); they apply at once. Also since 6.3.4 the webmail's send and the send API consume the same quota, answering `429`; a scheduled draft is not counted when it goes. ### Who may send as whom `SmtpAuthenticatedSenderCheck` (Control Panel **Connections & protocols → Protocols**, SMTP tab) came in 6.2.25 and is off by default, since an installation may depend on what off allows: any account sending as anyone. The `office` and `gateway` profiles set it to 1. Turned on, it applies to every authenticated session, whose `MAIL FROM` must then be: | Kind | Given by | |---|---| | The account's own address | always | | An alias resolving to it, up to five hops | the administrator | | A mailbox whose owner granted the **post (`p`) right on their INBOX** | that owner, with any IMAP client's `SETACL` | | An address elsewhere proved by a code mailed to it (since 6.3.4) | the person, in the webmail's *Settings, Other accounts, Send mail as* | | A masked address the account made, while it is on (since 6.3.4) | the person, in the webmail's *Settings, Masked addresses* | Anything else is refused `550 5.7.1` before the message is accepted. The webmail's From picker lists exactly these, so what it offers is what the server takes. The null sender is always allowed; unauthenticated sessions are unaffected. **The post right counts only while IMAP ACL enforcement is on** (`enableimapacl`, on by default). Before turning the check on, grant `p` to every delegate who sends as a shared mailbox. Mail sent as a proved outside address is not aligned with that address's domain, so a receiver honouring its DMARC policy may quarantine or refuse it. ### How passwords are stored In the Control Panel, **Access & abuse protection → Authentication**, card **Password storage**. Schemes: 0 none, 1 Blowfish, 2 MD5, 3 SHA-256, 4 PBKDF2-HMAC-SHA256 (the default), 5 Argon2id, 6 DPAPI (stored secrets only), 7 scrypt. ```ini PreferredHashAlgorithm=4 ; 3, 4, 5 or 7. Anything else is refused (HM5528) and read as 4 MinimumAcceptedHashAlgorithm=0 ; refuse a logon whose stored hash is weaker. 0 = off PasswordPepper= ; a server-wide secret mixed into Argon2id and scrypt hashes PasswordHashIterations=0 ; PBKDF2. 0 = 210,000. Range 10,000 to 10,000,000 PasswordHashMemoryKB=0 ; Argon2id. 0 = 19,456 KiB. Range 4,096 to 1,048,576 PasswordHashTimeCost=0 ; Argon2id passes. 0 = 2. Range 1 to 20 ``` - **Hashes upgrade at a successful logon, only upward.** Raising the scheme or a work factor re-derives each hash when its owner next signs in; lowering either affects new hashes only. - **`MinimumAcceptedHashAlgorithm` set too high is a lockout.** The refusal follows a correct comparison and the upgrade needs a successful logon, so an account below the floor never climbs out: its password must be reset. Raise the preference first, let accounts sign in, then set the floor. Argon2id and scrypt are peers, so 5 accepts either. - **A work factor out of range** is reported as `HM5561` and read as 0; `HM5607` means Argon2id was asked for on a build whose OpenSSL has none, and another scheme is used. Argon2id allocates `PasswordHashMemoryKB` on a connection thread at every logon. - **`PasswordPepper` must be set before accounts exist**: changing it invalidates every Argon2id and scrypt password. PBKDF2 is not peppered, so it stays usable for SCRAM. Since 6.3.4 it lives in `hMailServer.ini` only and no backup carries it, so **keep a copy of the value with the backups**: a database or archive restored onto a machine whose file has no pepper, or a different one, refuses every Argon2id and scrypt password, and a warm standby needs the same value in its own file. Where the service cannot write the file (a new Windows installation running as `NT SERVICE\hMailServer`, or a Linux package installation), COM and REST refuse it: set it with an elevated editor (`sudoedit` on Linux) and restart the service. ### The password policy All off until set, applied **where a password is chosen**, never at logon; on Windows under **Access & abuse protection → Administrative access**, Password tab. ```ini PasswordPolicyMinimumLength=12 PasswordPolicyRequireMixedCase=1 PasswordPolicyRequireDigit=1 PasswordPolicyRequireNonAlphanumeric=1 PasswordPolicyRejectCommon=1 ; refuse passwords from a built-in common list PasswordPolicyHistoryCount=5 ; refuse the last N passwords PasswordPolicyMaximumAgeDays=0 ; 0 = passwords never expire ``` The `office` profile sets a minimum of 10 and the common list. **Expiry needs care**: IMAP, POP3 and SMTP cannot change a password and `/portal` refuses an expired one, so an administrator renews it, or its owner through an app password, which still signs in to `/portal`. Active Directory accounts are exempt. The breach check is in [Hardening a public mail server](/topic/28241); the rest of sign-in is in [Signing in](/topic/28253). ### What it does not do - None of it spam-tests your own users: an authenticated session never is. - A setting profile never writes an IP range. ### Where to read more - Wiki: [Security Hardening](https://gitlab.com/hmailserver/hmailserver/-/wikis/Security-Hardening) sections 17.1, 17.2, 17.5, 17.7 and 17.9; [Settings Reference](https://gitlab.com/hmailserver/hmailserver/-/wikis/Settings-Reference) sections 9.1, 9.9 and 10; [Stopping Spam](https://gitlab.com/hmailserver/hmailserver/-/wikis/Stopping-Spam) section 10.1; [Troubleshooting](https://gitlab.com/hmailserver/hmailserver/-/wikis/Troubleshooting) sections 19.3 and 19.7. - Documents: [SettingsApply.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SettingsApply.md), [SettingLevels.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SettingLevels.md), [SendAs.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SendAs.md), [Profiles.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Profiles.md), [Roles.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Roles.md), [ServiceAccount.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/ServiceAccount.md). - Public documentation: [SecurityRange and SecurityRanges](https://www.progressiverobot.com/hmailserver-documentation/#securityrange-and-securityranges), for the COM names of the range flags. - On this forum: [Being trusted by the servers you send to](/topic/28188), [The Control Deck](/topic/28192), [Upgrading to 6.3.4](/topic/28193), [hMailServer on Linux](/topic/28189). --- Canonical: https://www.hmailserver.co.uk/topic/28239/ip-ranges-auto-ban-lockout-tarpits-sending-limits-send-as-and-password-storage # IP ranges, auto-ban, lockout, tarpits, sending limits, send-as and password storage *Written for 6.3.4. hMailServer 6.3.7 (17 October 2026) is a security release that bears on the send-as and sign-in sections here. An authenticated submission is now DKIM-signed as another domain hosted here only when its account may send as the address its From names (its own address, an alias pointing at it, a mailbox that granted it the post right, an address it proved, or a masked address of its own, the addresses the send-as table below lists); otherwise the mail is still accepted and sent, unsigned as that domain, so give such an account the alias or the grant before upgrading. A domain's or an account's rule replies only from an address its owner may send as. SMTP, POP3 and ManageSieve refuse a SASL PLAIN sign-in that asks to act as another account; with SRS on, mail to an SRS address this server handed out is held to the relay rules unless it is a bounce; the IMAP and POP3 listeners can take the client's address from a PROXY protocol header as the SMTP listener does, each from a list of peers of its own, both off by default; and the Control Panel's IP ranges page, which could show no list of ranges at all, is fixed. See [the 6.3.7 announcement](/topic/28277).* This topic covers who may use this server and how hard it is to abuse it. Know this first: **only the IP ranges and auto-ban work out of the box; every other brake here is off until you set it.** Until you do, nothing else stops a botnet spending one guess per address, or a stolen password sending thousands of messages. The secure defaults of a new installation, TLS for sign-in and compromised-account detection are in [Hardening a public mail server](/topic/28241). ### Where each one is set Since 6.3.4 the database is the settings store, and for a key it holds, a `[Settings]` line edited in `hMailServer.ini` changes nothing; on 6.3.3 these keys were lines of the file, applied at the next start. The store, and when a save takes effect, are in [the settings topic](/topic/28222). | What | Kept in | Set with | Takes effect | |---|---|---|---| | IP ranges | the database | Control Panel **Access & abuse protection → IP ranges**; the Control Deck's IP ranges view; `/api/v1/ipranges`; `hmctl iprange`; COM `Settings.SecurityRanges` | at once | | Auto-ban's four settings | the database | Control Panel **Access & abuse protection → Auto-ban**; the Deck's settings; `PUT /api/v1/settings`; COM | at once | | Lockout, tarpits, rate limit, send-as check, hashing, policy | the settings store, under their `[Settings]` names | Control Panel; `PUT /api/v1/settings/ini/{name}`; `hmctl ini set `; COM `Settings.SetIniSetting`; on Linux `hmailserver --set-setting NAME=VALUE` | when saved | | `[SendingLimits]`, `[SendingLimitsOverrides]`, `PasswordPepper` | `hMailServer.ini` only | the file, by hand or through the Control Panel (below) | sending limits within seconds; the pepper after a restart | The Control Panel and COM are Windows-only. Of these `[Settings]` keys the Deck edits only `SmtpTarpitCount` and `SmtpTarpitDelaySeconds`. A value set with `hmailserver --set-setting` while the service runs waits for the next save, reinitialise or restart. ### IP ranges: priority and the relay grid When a client connects, every range containing its address is found and **the highest priority wins**. The shipped ranges are *Internet* (priority 10, `0.0.0.0`–`255.255.255.255`) and *My computer* (priority 30, `127.0.0.1`); a range added on the Control Panel's page starts at 15, so a narrow range sits in front of *Internet* without editing it. At every `RCPT TO` two decisions are made, in order: may this combination be delivered at all (if not, `550 Delivery is not allowed to this address.`), and must the client authenticate first (if so, `530 SMTP authentication is required.`). "Local" and "remote" describe the **addresses**, not the client: an address is local when its domain is a hosted active domain or a route says so. | Combination | REST fields: allowed / auth required | *Internet* | *My computer* | |---|---|---|---| | Local → local | `deliver_local_to_local` / `require_auth_local_to_local` | allowed, **auth required** | allowed, no auth | | Local → remote | `deliver_local_to_remote` / `require_auth_local_to_remote` | allowed, **auth required** | allowed, no auth | | Remote → local | `deliver_remote_to_local` / `require_auth_remote_to_local` | allowed, no auth | allowed, no auth | | Remote → remote (relay) | `deliver_remote_to_remote` / `require_auth_remote_to_remote` | allowed, **auth required** | allowed, **auth required** | | Anti-spam, anti-virus | `spam_protection`, `virus_protection` | on | on | | Require TLS for auth | `require_tls_for_auth` | on in a new 6.3.4 installation; otherwise as the server had it | off | **Remote → remote is the open-relay box.** Both shipped ranges allow it with authentication required, even from 127.0.0.1, so a client that has not signed in gets `530`; a range that does not allow it at all gets `550`. The Control Panel's Server status page raises a Critical warning for any range that allows it without authentication. **Remote → local** is inbound mail and must need no authentication. **Local → local** needs it on *Internet* so that nobody outside can forge mail from one of your addresses to another. **The protection switches reach further than their names.** Clearing *Enable anti-spam* turns off every spam test, greylisting and the recipient tarpit for the range; clearing *Enable anti-virus* means its mail is never flagged for scanning. Since 6.3.4 SPF gives loopback no pass, so a content filter that re-injects mail should authenticate or use a range with anti-spam off. **Require TLS for auth belongs to the range, not the port.** With it set, `AUTH` is not advertised on a cleartext connection from that range, on SMTP, POP3, IMAP and ManageSieve. **A device that cannot sign in** gets a range of its own above *Internet*. For a scanner at 192.0.2.50 that sends from your domains to your own mailboxes only: ```bash hmctl iprange create --set name=scanner --set lower=192.0.2.50 --set upper=192.0.2.50 \ --set priority=20 --set allow_smtp=true --set allow_imap=false --set allow_pop3=false \ --set deliver_local_to_local=true --set require_auth_local_to_local=false \ --set deliver_local_to_remote=false --set deliver_remote_to_local=false \ --set deliver_remote_to_remote=false --set spam_protection=true --set virus_protection=true ``` `hmctl iprange update ` and `PUT /api/v1/ipranges/{id}` (since 6.3.3) change any subset of the fields. The routes refuse a domain-restricted API key; a role needs `manage_settings`. **The Control Panel's Restore defaults** (`SecurityRanges.SetDefault()`) discards every range, your own and the auto-ban ones, and writes back the two shipped ones as the create scripts make them, without *Require TLS for auth*. ### Auto-ban, and why its ranges sit at priority 100 Auto-ban counts failed logons per **connecting address**; SMTP `AUTH`, POP3, IMAP, ManageSieve and the REST API feed one counter, but REST does not count a client on loopback (`127.0.0.1`, `::1`). **Behind a reverse proxy on the same machine** (the Linux package binds REST to `127.0.0.1`), the webmail's failed sign-ins are never auto-banned; account lockout, below, is what limits them there, and it is off until you set it. At the limit auto-ban clears that address's counted failures, creates an expiring range named `Auto-ban: ` at **priority 100**, and disconnects; SMTP says `535 Authentication failed. Too many invalid logon attempts.` Priority 100, above both shipped ranges, is what makes a ban effective; `AutoBanNeverBan`, below, exempts an address. | Setting (COM name) | Default | Trap | |---|---|---| | `AutoBanOnLogonFailure` | on | — | | `MaxInvalidLogonAttempts` | 3 | **`0` disables auto-ban entirely**, whatever the switch says | | `MaxInvalidLogonAttemptsWithin` | 30 minutes | Not a sliding window: how long a failure record survives, deleted by a pass every minute that runs only while auto-ban is on | | `AutoBanMinutes` | 60 | **`0` disconnects the client but creates no range**, so nothing is banned | Their REST keys include `auto_ban_on_logon_failure` and `max_invalid_logon_attempts`. The `office` profile (since 6.3.4) raises the limit to 5. - **`AutoBanNeverBan`** (since 6.3.2): addresses and CIDR blocks whose failures are never counted; the per-connection disconnect still applies, and a bad entry is reported once as `HM6524`. Server status warns "Localhost is currently banned in the IP ranges." when a ban covers 127.0.0.1. - **Control Panel sign-in over COM** is outside auto-ban and lockout, so guessing an administrator's mailbox name cannot lock them out of the tool. - **Spam traps and the multi-recipient null-sender check** ban through the same ranges (`Auto-ban: spam trap
`, `Auto-ban: multi-recipient null sender
`): see [Allow and block lists](/topic/28249) and [the anti-spam stack](/topic/28248). Firewall bans (`AutoBanFirewall`, `AutoBanCommand`, fail2ban) are in [Reading the logs](/topic/28220). **Releasing a banned address.** **Clear logon-failure list** (`POST /api/v1/settings/logon-failures/clear`, COM `Settings.ClearLogonFailureList()`) clears counted failures only, for every address at once; a ban stays until its range expires. To release one now: 1. Have the person correct the password in their client first: the counted failures were cleared when the ban was made, so a client still retrying the old one reaches the limit again as soon as the range is gone. 2. Find the range: the IP ranges page, the Deck's IP ranges view, `hmctl iprange list` or `GET /api/v1/ipranges`. Its name begins `Auto-ban:` and its priority is 100. 3. Delete it: on the page, `hmctl iprange delete `, or `DELETE /api/v1/ipranges/`. With `AutoBanFirewall` on, the firewall is reconciled on every range deletion, so its rule goes too. Five error-log codes mean the control has quietly stopped working: | Code | Meaning | |---|---| | `HM6114` | A failed logon could not be recorded, so no ban will trigger while this persists | | `HM6099` | The auto-ban range could not be saved: the address has **not** been blocked | | `HM6113` | Spent failures could not be cleared, so the address may be banned again as soon as its ban expires | | `HM6112` | Expired IP ranges could not be removed, so an expired ban keeps blocking | | `HM6115` | Expired failures could not be cleared, so occasional failures may add up to a ban | ### Account lockout, by name A distributed attack never crosses any one address's threshold. Lockout counts the **name** being guessed at, across SMTP `AUTH`, POP3, IMAP, ManageSieve, REST and the webmail's sign-in. It is off until set; the `office` profile sets 10. ```ini AccountLockoutThreshold=10 ; failed logons per username before a lockout. 0 = off AccountLockoutWindowMinutes=30 ; the window they must fall inside AccountLockoutMinutes=30 ; how long the lock lasts ``` - **It counts names, not accounts**, so a nonexistent name locks as a real one would, and reveals nothing. - **A locked name is refused before the password is checked**, with the ordinary invalid-credentials reply; only its timing differs, deliberately. - **A lockout refusal is not charged to the connecting address**, so an owner retrying from an office NAT does not get everyone behind it auto-banned. - **Passkeys and single sign-on are not counted**, being no guesses. - **To lift one early**: `POST /api/v1/accounts/{address}/unlock`, which a help desk scoped to the account's domain may call, COM `Application.UnlockAccount`, or **Unlock** on the Deck's Accounts row. A password reset clears it too. ### Tarpits and the submission rate limit All **0, off, until set**; these values are a starting point. ```ini LogonTarpitSeconds=2 ; per failure on this connection, capped at 30 s. 0 = off SmtpTarpitCount=10 ; recipients an unauthenticated session may name before... SmtpTarpitDelaySeconds=5 ; ...each further RCPT TO waits this long, capped at 30 s MaxSubmissionsPerIPPerMinute=30 ; MAIL FROM commands per source IP per minute. 0 = off ``` - **The logon tarpit** (Auto-ban page) makes a failed logon's refusal wait `LogonTarpitSeconds` times the failures so far on that connection, on SMTP, POP3 and IMAP. **A correct password never waits.** - **The recipient tarpit** (Anti-spam settings page; COM `AntiSpam.TarpitCount` and `AntiSpam.TarpitDelay`; REST `tarpit_count` and `tarpit_delay` on `/api/v1/settings/antispam`) spares authenticated sessions and ranges with anti-spam off. - **`MaxSubmissionsPerIPPerMinute` counts every `MAIL FROM`, authenticated or not**, and refuses the excess with `421 Too many messages from your IP address`, which contains a compromised account. The `office` profile sets 60; `gateway` keeps 0, since all inbound mail comes from the gateway's few addresses. Since 6.3.4 a save restarts its one-minute window. Behind a load balancer, give the SMTP listener `SMTPProxyProtocolEnabled` with `SMTPProxyProtocolTrustedIPs` (or `XCLIENT`), or auto-ban, the rate limit and the ranges all reason about the proxy's address: see [Behind a gateway or reverse proxy](/topic/28240). ### Per-account sending limits The brake on a compromised account, counted against the **authenticated user name** (never `MAIL FROM`) over a rolling period. The server level stays in the file: ```ini [SendingLimits] MaxMessagesPerAccountPerPeriod=500 MaxRecipientsPerAccountPerPeriod=2000 PeriodHours=24 ; clamped to 1-168 StateSaveIntervalSeconds=10 ; clamped to 1-3600 [SendingLimitsOverrides] newsletter@yourcompany.com=5000:50000 ceo@yourcompany.com=0:0 ``` Both limits default to 0, no limit; recipients are the stricter measure. An override replaces the global pair for that address, `0:0` exempts it, and an optional third field sets its period in hours; a malformed line is ignored with a log entry and the global limit applies. On Windows edit `C:\Program Files\hMailServer\Bin\hMailServer.INI` with an elevated editor, or use the Control Panel's **Maintenance → Server limits & expert settings** on the server itself; on Linux, `sudoedit /etc/hmailserver/hMailServer.ini`. They are not in the settings store, so no environment variable or setting profile writes them. **What the sender sees**: `452 4.7.0 Sending limit for this account has been reached. Please try again later.` at `MAIL FROM`, or `452 4.5.3 Recipient sending limit for this account has been reached. Please try again later.` at `RCPT TO`. The session stays open, the refusal spends no budget, and the application log, not the error log, says `SendingLimit refused submission`. Counters survive a restart in `hmailserver_sendinglimits.dat` in the data directory. **It fails open**: an unreadable file, a limiter exception or a full table of tracked accounts (error 5813) lets the message through. **Since 6.3.4 there are three levels.** A domain and an account may hold `sending_max_messages` and `sending_max_recipients`, which **tighten** the level above and never loosen it; an address's override line is its server level. Set them on the **Levels** tab of the domain and account dialogs, in the Deck's editors, with `PUT /api/v1/accounts/{address}`, or `hmctl account set
sending_max_messages 100` (`inherit` returns to the level above); they apply at once. Also since 6.3.4 the webmail's send and the send API consume the same quota, answering `429`; a scheduled draft is not counted when it goes. ### Who may send as whom `SmtpAuthenticatedSenderCheck` (Control Panel **Connections & protocols → Protocols**, SMTP tab) came in 6.2.25 and is off by default, since an installation may depend on what off allows: any account sending as anyone. The `office` and `gateway` profiles set it to 1. Turned on, it applies to every authenticated session, whose `MAIL FROM` must then be: | Kind | Given by | |---|---| | The account's own address | always | | An alias resolving to it, up to five hops | the administrator | | A mailbox whose owner granted the **post (`p`) right on their INBOX** | that owner, with any IMAP client's `SETACL` | | An address elsewhere proved by a code mailed to it (since 6.3.4) | the person, in the webmail's *Settings, Other accounts, Send mail as* | | A masked address the account made, while it is on (since 6.3.4) | the person, in the webmail's *Settings, Masked addresses* | Anything else is refused `550 5.7.1` before the message is accepted. The webmail's From picker lists exactly these, so what it offers is what the server takes. The null sender is always allowed; unauthenticated sessions are unaffected. **The post right counts only while IMAP ACL enforcement is on** (`enableimapacl`, on by default). Before turning the check on, grant `p` to every delegate who sends as a shared mailbox. Mail sent as a proved outside address is not aligned with that address's domain, so a receiver honouring its DMARC policy may quarantine or refuse it. ### How passwords are stored In the Control Panel, **Access & abuse protection → Authentication**, card **Password storage**. Schemes: 0 none, 1 Blowfish, 2 MD5, 3 SHA-256, 4 PBKDF2-HMAC-SHA256 (the default), 5 Argon2id, 6 DPAPI (stored secrets only), 7 scrypt. ```ini PreferredHashAlgorithm=4 ; 3, 4, 5 or 7. Anything else is refused (HM5528) and read as 4 MinimumAcceptedHashAlgorithm=0 ; refuse a logon whose stored hash is weaker. 0 = off PasswordPepper= ; a server-wide secret mixed into Argon2id and scrypt hashes PasswordHashIterations=0 ; PBKDF2. 0 = 210,000. Range 10,000 to 10,000,000 PasswordHashMemoryKB=0 ; Argon2id. 0 = 19,456 KiB. Range 4,096 to 1,048,576 PasswordHashTimeCost=0 ; Argon2id passes. 0 = 2. Range 1 to 20 ``` - **Hashes upgrade at a successful logon, only upward.** Raising the scheme or a work factor re-derives each hash when its owner next signs in; lowering either affects new hashes only. - **`MinimumAcceptedHashAlgorithm` set too high is a lockout.** The refusal follows a correct comparison and the upgrade needs a successful logon, so an account below the floor never climbs out: its password must be reset. Raise the preference first, let accounts sign in, then set the floor. Argon2id and scrypt are peers, so 5 accepts either. - **A work factor out of range** is reported as `HM5561` and read as 0; `HM5607` means Argon2id was asked for on a build whose OpenSSL has none, and another scheme is used. Argon2id allocates `PasswordHashMemoryKB` on a connection thread at every logon. - **`PasswordPepper` must be set before accounts exist**: changing it invalidates every Argon2id and scrypt password. PBKDF2 is not peppered, so it stays usable for SCRAM. Since 6.3.4 it lives in `hMailServer.ini` only and no backup carries it, so **keep a copy of the value with the backups**: a database or archive restored onto a machine whose file has no pepper, or a different one, refuses every Argon2id and scrypt password, and a warm standby needs the same value in its own file. Where the service cannot write the file (a new Windows installation running as `NT SERVICE\hMailServer`, or a Linux package installation), COM and REST refuse it: set it with an elevated editor (`sudoedit` on Linux) and restart the service. ### The password policy All off until set, applied **where a password is chosen**, never at logon; on Windows under **Access & abuse protection → Administrative access**, Password tab. ```ini PasswordPolicyMinimumLength=12 PasswordPolicyRequireMixedCase=1 PasswordPolicyRequireDigit=1 PasswordPolicyRequireNonAlphanumeric=1 PasswordPolicyRejectCommon=1 ; refuse passwords from a built-in common list PasswordPolicyHistoryCount=5 ; refuse the last N passwords PasswordPolicyMaximumAgeDays=0 ; 0 = passwords never expire ``` The `office` profile sets a minimum of 10 and the common list. **Expiry needs care**: IMAP, POP3 and SMTP cannot change a password and `/portal` refuses an expired one, so an administrator renews it, or its owner through an app password, which still signs in to `/portal`. Active Directory accounts are exempt. The breach check is in [Hardening a public mail server](/topic/28241); the rest of sign-in is in [Signing in](/topic/28253). ### What it does not do - None of it spam-tests your own users: an authenticated session never is. - A setting profile never writes an IP range. ### Where to read more - Wiki: [Security Hardening](https://gitlab.com/hmailserver/hmailserver/-/wikis/Security-Hardening) sections 17.1, 17.2, 17.5, 17.7 and 17.9; [Settings Reference](https://gitlab.com/hmailserver/hmailserver/-/wikis/Settings-Reference) sections 9.1, 9.9 and 10; [Stopping Spam](https://gitlab.com/hmailserver/hmailserver/-/wikis/Stopping-Spam) section 10.1; [Troubleshooting](https://gitlab.com/hmailserver/hmailserver/-/wikis/Troubleshooting) sections 19.3 and 19.7. - Documents: [SettingsApply.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SettingsApply.md), [SettingLevels.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SettingLevels.md), [SendAs.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SendAs.md), [Profiles.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Profiles.md), [Roles.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Roles.md), [ServiceAccount.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/ServiceAccount.md). - Public documentation: [SecurityRange and SecurityRanges](https://www.progressiverobot.com/hmailserver-documentation/#securityrange-and-securityranges), for the COM names of the range flags. - On this forum: [Being trusted by the servers you send to](/topic/28188), [The Control Deck](/topic/28192), [Upgrading to 6.3.4](/topic/28193), [hMailServer on Linux](/topic/28189). --- Canonical: https://www.hmailserver.co.uk/topic/28240/behind-a-gateway-load-balancer-or-reverse-proxy-relays-proxy-xclient-and-https # Behind a gateway, load balancer or reverse proxy: relays, PROXY, XCLIENT and HTTPS *Written for 6.3.4. hMailServer 6.3.7 (17 October 2026) adds the PROXY protocol to the IMAP and POP3 listeners: each takes the client's address from a PROXY header as the SMTP listener does, from the peers on a list of its own, and both are off by default; their switches and peer lists are on the IMAP and POP3 cards of the Control Panel's Protocols page. The SMTP listener's PROXY protocol and XCLIENT are a card on Incoming relays, not on Server limits & expert settings, and Ctrl+K takes every old title to its page. A state-changing REST request that carries a credential is now refused when the browser reports it as cross-site (`Sec-Fetch-Site`, or an `Origin` that is not this server's); scripts, `hmctl` and the PowerShell module send no such header and are unaffected. The ACME HTTP-01 challenge responder, which served one connection at a time, serves up to 64 at once. See [the 6.3.7 announcement](/topic/28277).* This topic is for a server that does not face the internet alone: behind a hosted spam filter or a filtering appliance, behind a load balancer, or with a reverse proxy terminating HTTPS in front of the webmail, the Control Deck, CalDAV and CardDAV. **The one thing to know first: hMailServer judges the address that connected to it, and behind a front end that address is the front end's until you tell the server otherwise.** Auto-ban, the rate limits, the IP ranges and every spam test then reason about the wrong machine: SPF fails on every message, and a block list that lists your gateway blocks all your mail. Each kind of front end is handled in its own way, and behind an HTTP reverse proxy the address stays the proxy's. ### Which mechanism fits which front end | Front end | What reaches hMailServer | What tells it the truth | |---|---|---| | A filtering gateway, hosted or an appliance, that accepts the mail and sends it on | A new SMTP session from the gateway, the sender's address in its `Received` header | **Incoming relays**, `XCLIENT` if the gateway can send it, and the `gateway` setting profile | | A load balancer or TLS terminator in front of the SMTP listener | The client's session, from the balancer's address | HAProxy `PROXY` v1 or v2 | | A reverse proxy terminating HTTPS for `/portal`, `/` and `/api/v1` | HTTP from the proxy | `X-Forwarded-Proto: https` from the proxy on the same machine, for the cookies; the address stays the proxy's | | A reverse proxy in front of CalDAV, CardDAV and the other web services | HTTP from the proxy | `X-Forwarded-Proto: https`, believed from loopback and from `WebServicesTrustedProxies` | ### Incoming relays: what listing one changes **Mail flow & delivery → Incoming relays** in the Control Panel (Windows) lists servers that legitimately forward mail to you, each a name and an IP range (a lower and an upper address); a save applies at once. The same list is in the Control Deck, at `/api/v1/incoming-relays` over REST since 6.3.3 (a key confined to named domains is refused), and in COM `Settings.IncomingRelays` on Windows, where since 6.3.4 a change takes effect only at `Save()`. Listing a relay changes two things: - **When spam protection runs.** A connection from a listed relay is switched from pre-transmission to post-transmission testing. DNSBL, PTR and HELO checks judge the connecting IP, so they wait until the body has arrived and the real origin can be read from it. Spam is then no longer refused before the body: with quarantine on, a message reaching the delete threshold is held for review; without it, it is refused with `554 5.7.1` after the body. - **Which IP counts as the origin.** The `Received` chain is walked from the top, and the first address that is not a listed relay becomes the originating IP. SPF, DMARC and ARC are evaluated against it. ```text Received: from gw2.example.net [198.51.100.8] listed relay: keep walking Received: from gw1.example.net [198.51.100.7] listed relay: keep walking Received: from mail.sender.example [203.0.113.44] not listed: the originating IP ``` Since 6.2.24 the address taken is the one the relay observed, not the client's own HELO literal; thresholds tuned before then need checking again. **Since 6.3.4 a listed relay changes more:** - A remote domain policy's inbound rule, *Require TLS on mail from this domain* (`530 5.7.0` at `MAIL FROM`), is not applied to it. The hop that rule is about happened before the gateway, so require TLS on the gateway. [Requiring TLS and remote domain policies](/topic/28257) covers the policies. - Mail it hands over counts as coming from inside. One rule, `MessageOrigin::IsExternalSender`, decides that for the external-sender tag, the first-contact note and impersonation protection, so none of the three applies behind a listed gateway. [DomainTransforms.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/DomainTransforms.md): the choice is between the tag and the relay declaration, and there is no third answer today. [Mail flow rules and what a domain does to a message](/topic/28251) covers the tag. - A spam trap never bans it; the sender is still judged. - The domain white and black list sees the relay's own HELO name, so key rows for mail behind a relay on the envelope domain. **Keep the range to the exact gateway addresses.** An attacker inside a range listed too wide can insert a forged `Received` header and choose which IP you evaluate. **And leave spam protection on for the IP range the gateway connects from**: clearing *Enable anti-spam* on a range turns off every spam test, greylisting and the recipient tarpit for it, relay or not. | Symptom | Cause | |---|---| | Every message is scored as though it came from your own filter; SPF always fails | The front end is not listed as an incoming relay | | A DNSBL hit on the gateway's address blocks all mail | The same | | Adding a relay stopped spam being rejected before the body | Expected: testing moved to after the body. Quarantine and scoring still apply | ### PROXY protocol and XCLIENT If the front end can pass the real client address itself, that is better than reconstructing it from headers. Two mechanisms have been accepted since 6.2.23-alpha1, both off by default and both honoured only from the addresses you list: | Setting | Default | What it does | |---|---|---| | `SMTPProxyProtocolEnabled` | `0` | HAProxy `PROXY` v1 and v2 on the SMTP listener, read before TLS and before the banner | | `SMTPProxyProtocolTrustedIPs` | empty | The real TCP peers allowed to send a `PROXY` header, comma-separated addresses or CIDR ranges. Empty means nobody | | `SMTPXClientEnabled` | `0` | The Postfix `XCLIENT` verb | | `SMTPXClientTrustedIPs` | empty | The peers allowed to use it. Empty means nobody | - **Trust is decided against the real TCP peer**, never an address a header supplied, so chained proxies cannot bootstrap trust. An entry that does not parse matches nothing and is logged once per run (`HM6281`). - **Once an address is listed for PROXY, the header is required from it.** Configure the proxy to send it (in HAProxy, `send-proxy` or `send-proxy-v2`) or its connections are dropped; and a `PROXY` line from a peer that is not listed closes the connection without a reply. Change the proxy and the settings together. - **`XCLIENT` is advertised in `EHLO` only to a trusted peer**, as `XCLIENT ADDR NAME PORT PROTO HELO LOGIN`; an untrusted peer never learns the verb exists and gets `550` if it tries. With `LOGIN` the upstream vouches for a signed-in account: [RoomsAndEquipment.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/RoomsAndEquipment.md) counts such a submission as a signed-in session, like SMTP `AUTH`. Listing an address for `XCLIENT` trusts it with that too. **What then sees the real client:** auto-ban, the rate limits, the IP ranges, the spam tests, and since 6.3.4 the compromised-account detection's address signal (which never scores behind a proxy that speaks neither) and, with PROXY, the live view of one connection. **What does not: country blocking** (since 6.3.4). It judges the address the TCP connection came from, the proxy, and exempts loopback and private ranges; put the geographic policy on the proxy. Since 6.3.7 the IMAP and POP3 listeners take the client's address from a PROXY protocol header too, each from the peers on a list of its own (`IMAPProxyProtocolEnabled` and `IMAPProxyProtocolTrustedIPs`, `POP3ProxyProtocolEnabled` and `POP3ProxyProtocolTrustedIPs`), both off by default, the header read before an IMAPS or POP3S port's TLS handshake and before the greeting. No document describes either mechanism for ManageSieve. **Where they are set.** On Windows, the Control Panel's **Maintenance → Server limits & expert settings** (card *Front-end proxies (PROXY protocol and XCLIENT)*; in 6.3.7's Control Panel the PROXY protocol and XCLIENT are a card on **Incoming relays**) or COM `Settings.SetIniSetting`. On either system, `PUT /api/v1/settings/ini/{name}` (the administrator password only; an API key is refused) or `hmctl ini set `; on Linux also `hmailserver --set-setting`. The Control Deck has no editor for these keys. All four apply when saved. Since 6.3.4 the database holds them, so a line edited in `hMailServer.ini` changes nothing the server uses; the store, the file's copy and `HM5804` are in [Settings: what is not stock and when a save takes effect](/topic/28222). ```sh # Over REST with the administrator password (an API key is refused), from any machine that reaches the listener; applies when saved hmctl ini set SMTPProxyProtocolTrustedIPs 10.0.0.5,10.0.0.6 hmctl ini set SMTPProxyProtocolEnabled 1 # On a Linux server; a running service takes it at its next save, reinitialise or restart sudo hmailserver --set-setting SMTPProxyProtocolTrustedIPs=10.0.0.5,10.0.0.6 sudo hmailserver --set-setting SMTPProxyProtocolEnabled=1 ``` ### The gateway profile (since 6.3.4) A setting profile is a named, reviewed set of settings, shown as a diff first. `gateway` is for a server whose inbound mail a filtering gateway has already judged. Its 17 entries include: | Entry | Value | Why ([Profiles.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Profiles.md), shortened) | |---|---|---| | `usegreylisting` | `0` | It judges the connecting address, which is the gateway's even through Incoming relays | | `SMTPXClientEnabled` | `1` | Honoured only from `SMTPXClientTrustedIPs`, and from nobody while that is empty | | `ASArcFilteringEnabled` | `1` | With DMARC on, the DMARC penalty is offset when a sealer named in `arc_trusted_sealers` sealed a pass | | `MaxSubmissionsPerIPPerMinute`, `SmtpTarpitCount` | `0` | All inbound mail comes from the gateway's few addresses | | `QuarantineEnabled`, `RejectFullMailboxAtRcpt` | `1` | A message at the delete threshold is held, not refused back to the gateway; a full mailbox is refused at RCPT so the gateway returns the message | | `usespf`, `ascheckhostinhelo`, `ascheckptr`, `ASDMARCEnabled` | `1` | Through Incoming relays they judge the real sender | | `AutoBanOnLogonFailureEnabled`, `SmtpAuthenticatedSenderCheck` | `1` | For people who sign in here directly | The rest switch on DKIM verification, the MX check and attachment blocking, all three off as shipped, and keep the reason headers on. The first four rows turn a check off or trust an address range, so applying asks for an explicit confirmation wherever one of them would change something. A profile never writes an incoming relay, an IP range, a port's TLS or the ARC sealer. The steps are yours, in this order: 1. **Add the gateway's addresses to Incoming relays before applying the profile**, and leave spam protection on for their IP range. Until they are there, the tests judge the gateway's own address. 2. **If the gateway sends `XCLIENT`**, put its addresses in `SMTPXClientTrustedIPs`. 3. **Name the ARC sealer the gateway signs as** in `arc_trusted_sealers` (`PUT /api/v1/settings/antispam`, or COM `AntiSpam.ArcTrustedSealers`). Until then no seal is trusted. 4. **Read the diff and apply it**, on the Control Panel's **Setting profiles** page under Maintenance, in the Deck's checklist, over REST or with `hmctl`, confirming the entries that ask. The apply is all or nothing and saves the values it replaces first; while an earlier profile is neither reverted nor discarded, another apply is refused with 409. The routes, the diff's token, the way back and the command-line verbs are in [the profiles section of the unattended installs topic](/topic/28231). 5. **Check "Require SSL/TLS when authenticating" on the Internet IP range**, or make the submission port STARTTLS-required, for people who sign in here directly. A new installation turns it on; an upgraded one keeps what it had. 6. **On the listener the gateway delivers to, leave TLS optional unless the gateway offers it**: a port's security is the port's, not a setting. ### A filter on the same machine Since 6.3.4 SPF is RFC 7208, and `127.0.0.0/8` and `::1` no longer pass unconditionally. A content filter that re-injects mail over loopback should authenticate, or connect from an IP range with anti-spam off. ### The HTTP listeners behind a TLS-terminating proxy | | REST listener | Web-services listener | |---|---|---| | Serves | The webmail at `/portal`, the Control Deck at `/`, the API under `/api/v1` | ACME challenges, MTA-STS, `security.txt`, autoconfiguration, CalDAV, CardDAV, the OpenPGP key directory | | Plain HTTP behind the proxy | Only on loopback: TLS is required unless bound to 127.0.0.1, localhost or ::1 | `WebServicesHttpPort` | | Whose `X-Forwarded-Proto: https` counts | The proxy on this machine | Loopback, and `WebServicesTrustedProxies` (since 6.3.3) | | What it decides | `Secure` on the session and sign-in cookies | Whether CalDAV and CardDAV treat the request as HTTPS; the Apple `.mobileconfig` profile; the key directory | A changed port or bind address (`RestApiPort`, `RestApiBindAddress`, `WebServicesHttpPort`, `WebServicesHttpsPort`) waits for a reinitialise (**Apply now**). `WebServicesTrustedProxies` applies when saved, and no document names a Control Panel field for it: set it with `PUT /api/v1/settings/ini/{name}`, `hmctl ini set`, COM `Settings.SetIniSetting` on Windows or `hmailserver --set-setting` on Linux (which a running service takes at its next save, reinitialise or restart). **The cookies, since 6.3.4.** `hmailsession` and the sign-in cookies `hmoidc`, `hmsso` and `hmpasskey` are marked `Secure` over TLS, or when the proxy on this machine says `X-Forwarded-Proto: https`. Before 6.3.4 only a TLS listener counted, so behind the usual proxy a browser would also send the session cookie over plain HTTP. A proxy that does not set the header still leaves them without `Secure`; a proxy on another machine can only reach a REST listener that speaks TLS itself. **CalDAV (since 6.3.4) and CardDAV** take HTTP Basic over HTTPS only: over plain HTTP the answer is 403 with the reason, unless a trusted proxy says `X-Forwarded-Proto: https`. From any other peer the header is ignored; an entry that does not parse matches nothing (`HM6281`). The Apple profile honours the header on the same terms, and the OpenPGP key directory (since 6.3.4) answers a plain request with the same 404 as an unknown address; point `openpgpkey.` at the proxy and forward `/.well-known/openpgpkey/` with the header. [The CalDAV and CardDAV topic](/topic/28252) covers the rest. **The client address stays the proxy's.** Nothing reads `X-Forwarded-For`, so: - The auto-ban leaves `127.0.0.1` and `::1` alone, and behind a proxy on this machine every webmail and Deck request comes from there. What brakes guessing is the per-name lockout, when `AccountLockoutThreshold` is set, and since 6.3.4 each owner's second-factor budget: five wrong codes in fifteen minutes refuse that owner's codes for the next fifteen (429, `second_factor_locked`). - The forgotten-password route (since 6.3.4) holds loopback to thirty requests in any sixty seconds, all together. A proxy on another host shares one per-client bucket of twenty an hour. Neither has a setting: put the proxy on the mail server or accept the cap. - For CalDAV and CardDAV the per-name lockout and the per-IP auto-ban see the proxy's address; [README.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/README.md) says to weigh the auto-ban accordingly. **Passkeys (since 6.3.4).** By default the relying party id is the `Host` header, which behind a TLS-terminating proxy is still what the browser sent, so the check holds. `X-Forwarded-Proto` is not consulted, which is why the origin check accepts `https` at the page's host and plain `http` only on a loopback name. A server reached at several names needs `passkey_relying_party` and `passkey_allowed_origins` together, the origins written as the browser writes them, port included; [the topic on signing in](/topic/28253) has the rest. **The address people use.** The server cannot learn it from the proxy. The links in a message that expires, a password-protected message and a password reset use `confidential_base_url` (then `quarantine_digest_base_url`, then `import_oauth_redirect_base_url`); the quarantine digest's links use `quarantine_digest_base_url` alone, a mailing list's use `list_base_url`, and single sign-on (since 6.3.4) uses `oidc_redirect_base_url`. Which paths to publish is in [The webmail portal at /portal](/topic/28191). ### Compression: let one side do it (since 6.3.4) Both listeners compress when the client asks, and there is no setting. - **Negotiation.** `Accept-Encoding` is the whole of it: `gzip` or `deflate`, `gzip` where both are acceptable, never `br`. `deflate` is the zlib format of RFC 1950, which a raw inflater fails on; ask for `gzip`. - **What.** Text, JSON, XML, a manifest, `message/rfc822` and an icon, between 1,024 bytes and 4 MB. An image, a PDF or a zip goes as it is, and a download (`/api/v1/me/messages/{id}/attachments/{index}`, `/files/{token}`) is never compressed. - **Headers.** `Content-Length` is the bytes sent; no chunked transfer encoding, no byte ranges. `Vary: Accept-Encoding` is on every answer of a type it would compress, which matters for the web-services listener, whose answers may be cached; the webmail and the API are `Cache-Control: no-store`. **Behind a reverse proxy**, either leave `Accept-Encoding` alone and pass the answer through (the server will not encode twice, and the proxy must not either), or strip `Accept-Encoding` from the request and compress at the proxy. What does not is forwarding `Accept-Encoding: gzip`, receiving `Content-Encoding: gzip`, and handing the bytes to a client that did not ask. ### What it does not do - **It reads no forwarded-for header.** XCLIENT is an SMTP setting only; since 6.3.7 the PROXY protocol is taken on the IMAP and POP3 listeners too, each from its own list of peers. - **A listed gateway turns off the external-sender tag**, the first-contact note and impersonation protection for what it hands over. - **Compression has no switch.** A client that wants none sends no `Accept-Encoding`, or `Accept-Encoding: identity`. - **A second machine behind a balancer** is [the warm standby and high availability topic](/topic/28232). ### Where to read more - Wiki: [Routes and Relays](https://gitlab.com/hmailserver/hmailserver/-/wikis/Routes-and-Relays) §14.3, [Security Hardening](https://gitlab.com/hmailserver/hmailserver/-/wikis/Security-Hardening) §17.8, [Stopping Spam](https://gitlab.com/hmailserver/hmailserver/-/wikis/Stopping-Spam) §10.1, [Settings Overview](https://gitlab.com/hmailserver/hmailserver/-/wikis/Settings-Overview), and [chapter 14.3](https://www.progressiverobot.com/hmailserver-documentation/#143-incoming-relays) of the documentation page. - Documents at the 6.3.4 tag: [Profiles.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Profiles.md), [ResponseCompression.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/ResponseCompression.md), [DomainTransforms.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/DomainTransforms.md), [PasskeysAndSingleSignOn.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/PasskeysAndSingleSignOn.md), [DevicesAndSessions.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/DevicesAndSessions.md), [AccountRecovery.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/AccountRecovery.md), [CountryBlocking.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/CountryBlocking.md), [SettingsApply.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SettingsApply.md), [RoomsAndEquipment.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/RoomsAndEquipment.md), and [README.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/README.md) for the `[Settings]` notes on PROXY, XCLIENT and the web services. - On this forum: [The webmail portal at /portal](/topic/28191), [The Control Deck](/topic/28192), [the anti-spam stack end to end](/topic/28248), [IP ranges, auto-ban, lockout and tarpits](/topic/28239) and [TLS on the listeners](/topic/28256). A way past one of these trust lists is a vulnerability: report it by a [confidential GitLab issue](https://gitlab.com/hmailserver/hmailserver/-/issues/new?issue[confidential]=true) or by email to the project's Service Desk address, contact-project+hmailserver-hmailserver-86730559-issue-@incoming.gitlab.com, which GitLab files as a confidential issue, and not in a forum post. --- Canonical: https://www.hmailserver.co.uk/topic/28240/behind-a-gateway-load-balancer-or-reverse-proxy-relays-proxy-xclient-and-https # Behind a gateway, load balancer or reverse proxy: relays, PROXY, XCLIENT and HTTPS *Written for 6.3.4. hMailServer 6.3.7 (17 October 2026) adds the PROXY protocol to the IMAP and POP3 listeners: each takes the client's address from a PROXY header as the SMTP listener does, from the peers on a list of its own, and both are off by default; their switches and peer lists are on the IMAP and POP3 cards of the Control Panel's Protocols page. The SMTP listener's PROXY protocol and XCLIENT are a card on Incoming relays, not on Server limits & expert settings, and Ctrl+K takes every old title to its page. A state-changing REST request that carries a credential is now refused when the browser reports it as cross-site (`Sec-Fetch-Site`, or an `Origin` that is not this server's); scripts, `hmctl` and the PowerShell module send no such header and are unaffected. The ACME HTTP-01 challenge responder, which served one connection at a time, serves up to 64 at once. See [the 6.3.7 announcement](/topic/28277).* This topic is for a server that does not face the internet alone: behind a hosted spam filter or a filtering appliance, behind a load balancer, or with a reverse proxy terminating HTTPS in front of the webmail, the Control Deck, CalDAV and CardDAV. **The one thing to know first: hMailServer judges the address that connected to it, and behind a front end that address is the front end's until you tell the server otherwise.** Auto-ban, the rate limits, the IP ranges and every spam test then reason about the wrong machine: SPF fails on every message, and a block list that lists your gateway blocks all your mail. Each kind of front end is handled in its own way, and behind an HTTP reverse proxy the address stays the proxy's. ### Which mechanism fits which front end | Front end | What reaches hMailServer | What tells it the truth | |---|---|---| | A filtering gateway, hosted or an appliance, that accepts the mail and sends it on | A new SMTP session from the gateway, the sender's address in its `Received` header | **Incoming relays**, `XCLIENT` if the gateway can send it, and the `gateway` setting profile | | A load balancer or TLS terminator in front of the SMTP listener | The client's session, from the balancer's address | HAProxy `PROXY` v1 or v2 | | A reverse proxy terminating HTTPS for `/portal`, `/` and `/api/v1` | HTTP from the proxy | `X-Forwarded-Proto: https` from the proxy on the same machine, for the cookies; the address stays the proxy's | | A reverse proxy in front of CalDAV, CardDAV and the other web services | HTTP from the proxy | `X-Forwarded-Proto: https`, believed from loopback and from `WebServicesTrustedProxies` | ### Incoming relays: what listing one changes **Mail flow & delivery → Incoming relays** in the Control Panel (Windows) lists servers that legitimately forward mail to you, each a name and an IP range (a lower and an upper address); a save applies at once. The same list is in the Control Deck, at `/api/v1/incoming-relays` over REST since 6.3.3 (a key confined to named domains is refused), and in COM `Settings.IncomingRelays` on Windows, where since 6.3.4 a change takes effect only at `Save()`. Listing a relay changes two things: - **When spam protection runs.** A connection from a listed relay is switched from pre-transmission to post-transmission testing. DNSBL, PTR and HELO checks judge the connecting IP, so they wait until the body has arrived and the real origin can be read from it. Spam is then no longer refused before the body: with quarantine on, a message reaching the delete threshold is held for review; without it, it is refused with `554 5.7.1` after the body. - **Which IP counts as the origin.** The `Received` chain is walked from the top, and the first address that is not a listed relay becomes the originating IP. SPF, DMARC and ARC are evaluated against it. ```text Received: from gw2.example.net [198.51.100.8] listed relay: keep walking Received: from gw1.example.net [198.51.100.7] listed relay: keep walking Received: from mail.sender.example [203.0.113.44] not listed: the originating IP ``` Since 6.2.24 the address taken is the one the relay observed, not the client's own HELO literal; thresholds tuned before then need checking again. **Since 6.3.4 a listed relay changes more:** - A remote domain policy's inbound rule, *Require TLS on mail from this domain* (`530 5.7.0` at `MAIL FROM`), is not applied to it. The hop that rule is about happened before the gateway, so require TLS on the gateway. [Requiring TLS and remote domain policies](/topic/28257) covers the policies. - Mail it hands over counts as coming from inside. One rule, `MessageOrigin::IsExternalSender`, decides that for the external-sender tag, the first-contact note and impersonation protection, so none of the three applies behind a listed gateway. [DomainTransforms.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/DomainTransforms.md): the choice is between the tag and the relay declaration, and there is no third answer today. [Mail flow rules and what a domain does to a message](/topic/28251) covers the tag. - A spam trap never bans it; the sender is still judged. - The domain white and black list sees the relay's own HELO name, so key rows for mail behind a relay on the envelope domain. **Keep the range to the exact gateway addresses.** An attacker inside a range listed too wide can insert a forged `Received` header and choose which IP you evaluate. **And leave spam protection on for the IP range the gateway connects from**: clearing *Enable anti-spam* on a range turns off every spam test, greylisting and the recipient tarpit for it, relay or not. | Symptom | Cause | |---|---| | Every message is scored as though it came from your own filter; SPF always fails | The front end is not listed as an incoming relay | | A DNSBL hit on the gateway's address blocks all mail | The same | | Adding a relay stopped spam being rejected before the body | Expected: testing moved to after the body. Quarantine and scoring still apply | ### PROXY protocol and XCLIENT If the front end can pass the real client address itself, that is better than reconstructing it from headers. Two mechanisms have been accepted since 6.2.23-alpha1, both off by default and both honoured only from the addresses you list: | Setting | Default | What it does | |---|---|---| | `SMTPProxyProtocolEnabled` | `0` | HAProxy `PROXY` v1 and v2 on the SMTP listener, read before TLS and before the banner | | `SMTPProxyProtocolTrustedIPs` | empty | The real TCP peers allowed to send a `PROXY` header, comma-separated addresses or CIDR ranges. Empty means nobody | | `SMTPXClientEnabled` | `0` | The Postfix `XCLIENT` verb | | `SMTPXClientTrustedIPs` | empty | The peers allowed to use it. Empty means nobody | - **Trust is decided against the real TCP peer**, never an address a header supplied, so chained proxies cannot bootstrap trust. An entry that does not parse matches nothing and is logged once per run (`HM6281`). - **Once an address is listed for PROXY, the header is required from it.** Configure the proxy to send it (in HAProxy, `send-proxy` or `send-proxy-v2`) or its connections are dropped; and a `PROXY` line from a peer that is not listed closes the connection without a reply. Change the proxy and the settings together. - **`XCLIENT` is advertised in `EHLO` only to a trusted peer**, as `XCLIENT ADDR NAME PORT PROTO HELO LOGIN`; an untrusted peer never learns the verb exists and gets `550` if it tries. With `LOGIN` the upstream vouches for a signed-in account: [RoomsAndEquipment.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/RoomsAndEquipment.md) counts such a submission as a signed-in session, like SMTP `AUTH`. Listing an address for `XCLIENT` trusts it with that too. **What then sees the real client:** auto-ban, the rate limits, the IP ranges, the spam tests, and since 6.3.4 the compromised-account detection's address signal (which never scores behind a proxy that speaks neither) and, with PROXY, the live view of one connection. **What does not: country blocking** (since 6.3.4). It judges the address the TCP connection came from, the proxy, and exempts loopback and private ranges; put the geographic policy on the proxy. Since 6.3.7 the IMAP and POP3 listeners take the client's address from a PROXY protocol header too, each from the peers on a list of its own (`IMAPProxyProtocolEnabled` and `IMAPProxyProtocolTrustedIPs`, `POP3ProxyProtocolEnabled` and `POP3ProxyProtocolTrustedIPs`), both off by default, the header read before an IMAPS or POP3S port's TLS handshake and before the greeting. No document describes either mechanism for ManageSieve. **Where they are set.** On Windows, the Control Panel's **Maintenance → Server limits & expert settings** (card *Front-end proxies (PROXY protocol and XCLIENT)*; in 6.3.7's Control Panel the PROXY protocol and XCLIENT are a card on **Incoming relays**) or COM `Settings.SetIniSetting`. On either system, `PUT /api/v1/settings/ini/{name}` (the administrator password only; an API key is refused) or `hmctl ini set `; on Linux also `hmailserver --set-setting`. The Control Deck has no editor for these keys. All four apply when saved. Since 6.3.4 the database holds them, so a line edited in `hMailServer.ini` changes nothing the server uses; the store, the file's copy and `HM5804` are in [Settings: what is not stock and when a save takes effect](/topic/28222). ```sh # Over REST with the administrator password (an API key is refused), from any machine that reaches the listener; applies when saved hmctl ini set SMTPProxyProtocolTrustedIPs 10.0.0.5,10.0.0.6 hmctl ini set SMTPProxyProtocolEnabled 1 # On a Linux server; a running service takes it at its next save, reinitialise or restart sudo hmailserver --set-setting SMTPProxyProtocolTrustedIPs=10.0.0.5,10.0.0.6 sudo hmailserver --set-setting SMTPProxyProtocolEnabled=1 ``` ### The gateway profile (since 6.3.4) A setting profile is a named, reviewed set of settings, shown as a diff first. `gateway` is for a server whose inbound mail a filtering gateway has already judged. Its 17 entries include: | Entry | Value | Why ([Profiles.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Profiles.md), shortened) | |---|---|---| | `usegreylisting` | `0` | It judges the connecting address, which is the gateway's even through Incoming relays | | `SMTPXClientEnabled` | `1` | Honoured only from `SMTPXClientTrustedIPs`, and from nobody while that is empty | | `ASArcFilteringEnabled` | `1` | With DMARC on, the DMARC penalty is offset when a sealer named in `arc_trusted_sealers` sealed a pass | | `MaxSubmissionsPerIPPerMinute`, `SmtpTarpitCount` | `0` | All inbound mail comes from the gateway's few addresses | | `QuarantineEnabled`, `RejectFullMailboxAtRcpt` | `1` | A message at the delete threshold is held, not refused back to the gateway; a full mailbox is refused at RCPT so the gateway returns the message | | `usespf`, `ascheckhostinhelo`, `ascheckptr`, `ASDMARCEnabled` | `1` | Through Incoming relays they judge the real sender | | `AutoBanOnLogonFailureEnabled`, `SmtpAuthenticatedSenderCheck` | `1` | For people who sign in here directly | The rest switch on DKIM verification, the MX check and attachment blocking, all three off as shipped, and keep the reason headers on. The first four rows turn a check off or trust an address range, so applying asks for an explicit confirmation wherever one of them would change something. A profile never writes an incoming relay, an IP range, a port's TLS or the ARC sealer. The steps are yours, in this order: 1. **Add the gateway's addresses to Incoming relays before applying the profile**, and leave spam protection on for their IP range. Until they are there, the tests judge the gateway's own address. 2. **If the gateway sends `XCLIENT`**, put its addresses in `SMTPXClientTrustedIPs`. 3. **Name the ARC sealer the gateway signs as** in `arc_trusted_sealers` (`PUT /api/v1/settings/antispam`, or COM `AntiSpam.ArcTrustedSealers`). Until then no seal is trusted. 4. **Read the diff and apply it**, on the Control Panel's **Setting profiles** page under Maintenance, in the Deck's checklist, over REST or with `hmctl`, confirming the entries that ask. The apply is all or nothing and saves the values it replaces first; while an earlier profile is neither reverted nor discarded, another apply is refused with 409. The routes, the diff's token, the way back and the command-line verbs are in [the profiles section of the unattended installs topic](/topic/28231). 5. **Check "Require SSL/TLS when authenticating" on the Internet IP range**, or make the submission port STARTTLS-required, for people who sign in here directly. A new installation turns it on; an upgraded one keeps what it had. 6. **On the listener the gateway delivers to, leave TLS optional unless the gateway offers it**: a port's security is the port's, not a setting. ### A filter on the same machine Since 6.3.4 SPF is RFC 7208, and `127.0.0.0/8` and `::1` no longer pass unconditionally. A content filter that re-injects mail over loopback should authenticate, or connect from an IP range with anti-spam off. ### The HTTP listeners behind a TLS-terminating proxy | | REST listener | Web-services listener | |---|---|---| | Serves | The webmail at `/portal`, the Control Deck at `/`, the API under `/api/v1` | ACME challenges, MTA-STS, `security.txt`, autoconfiguration, CalDAV, CardDAV, the OpenPGP key directory | | Plain HTTP behind the proxy | Only on loopback: TLS is required unless bound to 127.0.0.1, localhost or ::1 | `WebServicesHttpPort` | | Whose `X-Forwarded-Proto: https` counts | The proxy on this machine | Loopback, and `WebServicesTrustedProxies` (since 6.3.3) | | What it decides | `Secure` on the session and sign-in cookies | Whether CalDAV and CardDAV treat the request as HTTPS; the Apple `.mobileconfig` profile; the key directory | A changed port or bind address (`RestApiPort`, `RestApiBindAddress`, `WebServicesHttpPort`, `WebServicesHttpsPort`) waits for a reinitialise (**Apply now**). `WebServicesTrustedProxies` applies when saved, and no document names a Control Panel field for it: set it with `PUT /api/v1/settings/ini/{name}`, `hmctl ini set`, COM `Settings.SetIniSetting` on Windows or `hmailserver --set-setting` on Linux (which a running service takes at its next save, reinitialise or restart). **The cookies, since 6.3.4.** `hmailsession` and the sign-in cookies `hmoidc`, `hmsso` and `hmpasskey` are marked `Secure` over TLS, or when the proxy on this machine says `X-Forwarded-Proto: https`. Before 6.3.4 only a TLS listener counted, so behind the usual proxy a browser would also send the session cookie over plain HTTP. A proxy that does not set the header still leaves them without `Secure`; a proxy on another machine can only reach a REST listener that speaks TLS itself. **CalDAV (since 6.3.4) and CardDAV** take HTTP Basic over HTTPS only: over plain HTTP the answer is 403 with the reason, unless a trusted proxy says `X-Forwarded-Proto: https`. From any other peer the header is ignored; an entry that does not parse matches nothing (`HM6281`). The Apple profile honours the header on the same terms, and the OpenPGP key directory (since 6.3.4) answers a plain request with the same 404 as an unknown address; point `openpgpkey.` at the proxy and forward `/.well-known/openpgpkey/` with the header. [The CalDAV and CardDAV topic](/topic/28252) covers the rest. **The client address stays the proxy's.** Nothing reads `X-Forwarded-For`, so: - The auto-ban leaves `127.0.0.1` and `::1` alone, and behind a proxy on this machine every webmail and Deck request comes from there. What brakes guessing is the per-name lockout, when `AccountLockoutThreshold` is set, and since 6.3.4 each owner's second-factor budget: five wrong codes in fifteen minutes refuse that owner's codes for the next fifteen (429, `second_factor_locked`). - The forgotten-password route (since 6.3.4) holds loopback to thirty requests in any sixty seconds, all together. A proxy on another host shares one per-client bucket of twenty an hour. Neither has a setting: put the proxy on the mail server or accept the cap. - For CalDAV and CardDAV the per-name lockout and the per-IP auto-ban see the proxy's address; [README.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/README.md) says to weigh the auto-ban accordingly. **Passkeys (since 6.3.4).** By default the relying party id is the `Host` header, which behind a TLS-terminating proxy is still what the browser sent, so the check holds. `X-Forwarded-Proto` is not consulted, which is why the origin check accepts `https` at the page's host and plain `http` only on a loopback name. A server reached at several names needs `passkey_relying_party` and `passkey_allowed_origins` together, the origins written as the browser writes them, port included; [the topic on signing in](/topic/28253) has the rest. **The address people use.** The server cannot learn it from the proxy. The links in a message that expires, a password-protected message and a password reset use `confidential_base_url` (then `quarantine_digest_base_url`, then `import_oauth_redirect_base_url`); the quarantine digest's links use `quarantine_digest_base_url` alone, a mailing list's use `list_base_url`, and single sign-on (since 6.3.4) uses `oidc_redirect_base_url`. Which paths to publish is in [The webmail portal at /portal](/topic/28191). ### Compression: let one side do it (since 6.3.4) Both listeners compress when the client asks, and there is no setting. - **Negotiation.** `Accept-Encoding` is the whole of it: `gzip` or `deflate`, `gzip` where both are acceptable, never `br`. `deflate` is the zlib format of RFC 1950, which a raw inflater fails on; ask for `gzip`. - **What.** Text, JSON, XML, a manifest, `message/rfc822` and an icon, between 1,024 bytes and 4 MB. An image, a PDF or a zip goes as it is, and a download (`/api/v1/me/messages/{id}/attachments/{index}`, `/files/{token}`) is never compressed. - **Headers.** `Content-Length` is the bytes sent; no chunked transfer encoding, no byte ranges. `Vary: Accept-Encoding` is on every answer of a type it would compress, which matters for the web-services listener, whose answers may be cached; the webmail and the API are `Cache-Control: no-store`. **Behind a reverse proxy**, either leave `Accept-Encoding` alone and pass the answer through (the server will not encode twice, and the proxy must not either), or strip `Accept-Encoding` from the request and compress at the proxy. What does not is forwarding `Accept-Encoding: gzip`, receiving `Content-Encoding: gzip`, and handing the bytes to a client that did not ask. ### What it does not do - **It reads no forwarded-for header.** XCLIENT is an SMTP setting only; since 6.3.7 the PROXY protocol is taken on the IMAP and POP3 listeners too, each from its own list of peers. - **A listed gateway turns off the external-sender tag**, the first-contact note and impersonation protection for what it hands over. - **Compression has no switch.** A client that wants none sends no `Accept-Encoding`, or `Accept-Encoding: identity`. - **A second machine behind a balancer** is [the warm standby and high availability topic](/topic/28232). ### Where to read more - Wiki: [Routes and Relays](https://gitlab.com/hmailserver/hmailserver/-/wikis/Routes-and-Relays) §14.3, [Security Hardening](https://gitlab.com/hmailserver/hmailserver/-/wikis/Security-Hardening) §17.8, [Stopping Spam](https://gitlab.com/hmailserver/hmailserver/-/wikis/Stopping-Spam) §10.1, [Settings Overview](https://gitlab.com/hmailserver/hmailserver/-/wikis/Settings-Overview), and [chapter 14.3](https://www.progressiverobot.com/hmailserver-documentation/#143-incoming-relays) of the documentation page. - Documents at the 6.3.4 tag: [Profiles.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Profiles.md), [ResponseCompression.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/ResponseCompression.md), [DomainTransforms.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/DomainTransforms.md), [PasskeysAndSingleSignOn.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/PasskeysAndSingleSignOn.md), [DevicesAndSessions.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/DevicesAndSessions.md), [AccountRecovery.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/AccountRecovery.md), [CountryBlocking.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/CountryBlocking.md), [SettingsApply.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SettingsApply.md), [RoomsAndEquipment.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/RoomsAndEquipment.md), and [README.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/README.md) for the `[Settings]` notes on PROXY, XCLIENT and the web services. - On this forum: [The webmail portal at /portal](/topic/28191), [The Control Deck](/topic/28192), [the anti-spam stack end to end](/topic/28248), [IP ranges, auto-ban, lockout and tarpits](/topic/28239) and [TLS on the listeners](/topic/28256). A way past one of these trust lists is a vulnerability: report it by a [confidential GitLab issue](https://gitlab.com/hmailserver/hmailserver/-/issues/new?issue[confidential]=true) or by email to the project's Service Desk address, contact-project+hmailserver-hmailserver-86730559-issue-@incoming.gitlab.com, which GitLab files as a confidential issue, and not in a forum post. --- Canonical: https://www.hmailserver.co.uk/topic/28241/hardening-a-public-mail-server-secure-defaults-checklist-sealed-secrets-and-roles # Hardening a public mail server: secure defaults, checklist, sealed secrets and roles *Written for 6.3.4. hMailServer 6.3.7 (17 October 2026) is a security release that bears on this guide. On Windows, `hMailServer.exe /ProtectFolders`, which setup runs at every installation and upgrade, now hardens the program folder, `Bin` and `ControlPanel`: where hMailServer was installed in a folder that took its permissions from the root of a drive, any account that could sign in to the computer could replace the programs the service runs (CVE-2026-107571); the default folder under Program Files was never affected, and an installation at a drive's root is hardened by the upgrade itself. The `serveradmin` role given with a list of domains is confined to those domains over COM as over REST, a credential confined to named domains is refused the changes that reached beyond its domains over REST and COM, and a state-changing REST request that a browser reports as cross-site is refused. SMTP, POP3 and ManageSieve refuse a SASL PLAIN sign-in that asks to act as another account, and the stored-secrets key is no longer left in memory freed without being written over. An upgraded Windows server keeps the .NET 10 Desktop Runtime it had, 10.0.12 where 6.3.4 to 6.3.6 installed it, against which Microsoft published advisories on 13 October 2026: bring it to 10.0.13 through Microsoft Update or Microsoft's installer. Upgrade every server running 6.3.6 or earlier. See [the 6.3.7 announcement](/topic/28277).* This guide covers what stands between a mail server on the public internet and the people who guess passwords at it. All of it is new in 6.3.4 except auto-ban, account lockout, the administrator's second factor and the IP range option *Require TLS for auth*, which are older; 6.3.4 sets that option on a new installation and judges it in the checklist. The one thing to know first: **the secure defaults are applied only to a database the installation itself created, and an upgraded server is never changed.** After an upgrade your listeners, IP ranges and service account are as they were and nothing is sealed until you make a key; the checklist says what is left. ### What a new installation starts with, and what an upgrade keeps In 6.3.3 and earlier a new installation listened on ports 25, 587, 110 and 143 with no TLS, and neither shipped IP range required TLS for authentication. The create scripts still make that database; the Windows installer, and the Linux packages' first start, now change it before the server first listens. | | A new installation since 6.3.4 | An upgraded server | |---|---|---| | Ports 25 and 587 (SMTP), 110 (POP3), 143 (IMAP) | STARTTLS offered, optional | as it was | | IP range *Internet* (every IPv4 address) | **TLS required for authentication** | as it was; the checklist flags it | | IP range *My computer* (127.0.0.1) | unchanged | as it was | | Certificates | *Self-signed (made at installation)* | as they were | | Windows service account | `NT SERVICE\hMailServer` | the account it had (LocalSystem, if it began before 6.3.4) | **What a client sees.** From another machine, `AUTH` on SMTP is not offered before STARTTLS and is refused with `530 A SSL/TLS-connection is required for authentication.` if it is sent; IMAP `LOGIN` and `AUTHENTICATE` answer `BAD A SSL/TLS-connection is required for authentication.`; POP3 `USER` and `AUTH` answer `-ERR`. After STARTTLS each works as before. From the server itself, on 127.0.0.1, nothing has changed, and mail from other servers on port 25 is taken with or without TLS. **Why STARTTLS is optional.** A port's requirement applies to every address: on port 25 it would refuse every server that cannot start TLS, on the others this computer's own clients. The range's rule protects the password. **The certificate.** `self-signed-certificate.pem` and its key `self-signed-key.pem` are in `/Certificates`: `C:\Program Files\hMailServer\Data\Certificates` on a default Windows installation, `/var/lib/hmailserver/Certificates` from the Linux packages, where the key is mode 0600 and owned by the service's user. It is RSA 2048, runs 825 days, names the host and gives the organisation *hMailServer, self-signed at installation*. It encrypts, but nothing vouches for it, so mail clients warn. Replace it with ACME (set `AcmeEnabled`, `AcmeContactEmail` and `AcmeDomains`; the first certificate goes to every listener that holds the self-signed one) or with your own, set on each port. See [Let's Encrypt certificates are automatic](/topic/28187). ### Where the defaults are applied The create step marks its database with the `hm_settings` row `NewInstallDefaultsPending`; the verb `--apply-new-install-defaults` requires the mark and removes it once it has decided. | Platform | When it runs | What to know | |---|---|---| | Windows | The installer, after the database step of a new installation, with the service stopped | A database DBSetup attached, rather than created, carries no mark: exit 5, logged, nothing changed | | Linux (`.deb`, `.rpm`) | The unit's first start after `sudo -u hmailserver hmailserver --create-database`, which marks the database and leaves a one-line file, `.new-installation`, in the `DataFolder` | The unit runs `ExecStartPre=-/usr/bin/hmailserver --apply-new-install-defaults --if-marked`: nothing unless that file is there, then one try at the database, given room by `TimeoutStartSec=300` (the leading `-` forgives a failure, not a timeout). It ignores a mark made more than a day from its clock; a run by hand has no such limit | | By hand | A container, a server started by hand rather than by the unit, or defaults left for later | With the service stopped; on Linux as the service's user, because the verb refuses to run as root | By hand, on Windows and then on Linux: ``` Stop-Service hMailServer & "C:\Program Files\hMailServer\Bin\hMailServer.exe" --apply-new-install-defaults Start-Service hMailServer ``` ``` sudo systemctl stop hmailserver sudo -u hmailserver hmailserver --apply-new-install-defaults sudo systemctl start hmailserver ``` Exit codes: 0 applied (with `--if-marked`, also nothing to do); 1 something could not be made or written (`HM7510`, `HM7511` in the error log); 2 no database named, or not readable enough to judge, or on Linux run as root; 4 the Windows service is running; 5 not a database this installation created, or not as the create script left it: nothing changed. **On Linux, mind the data directory.** A `DataFolder` outside the unit's `ReadWritePaths` (`/var/lib/hmailserver` and `/var/log/hmailserver`) is read-only to the service, so the first start can neither write the certificate nor store mail there. Add the directory with a drop-in (`sudo systemctl edit hmailserver`, then `ReadWritePaths=/srv/mail` under `[Service]`) before that first start. **Not yet proved on a clean machine.** [SecureDefaults.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SecureDefaults.md)'s checks that the installer applies the defaults are not yet written, and those for the `.deb` ran only on GitHub Actions, so nothing runs them today. ### The setup checklist The checklist judges every server, new or upgraded, in twelve items. | Key | Done when | |---|---| | `profile` | a setting profile has been applied; asked for on a server with fewer than ten accounts | | `domain` | an active domain exists, and none has a name the server reserves for its own directories | | `account` | at least one account exists | | `submission` | every SMTP listener other than port 25 has STARTTLS or implicit TLS, and a certificate other than the self-signed one | | `sign_in_tls` | every IP range that lets a client connect from another address requires TLS for authentication, or reaches only listeners that start TLS before a password | | `spf`, `dkim`, `dmarc` | every active domain has a TXT record beginning `v=spf1`, signs with a key whose record is published at `._domainkey.`, and has a TXT record beginning `v=DMARC1` at `_dmarc.` | | `backup` | the destination is set and the folder exists | | `second_factor` | the administrator password is enrolled with a one-time code | | `stored_secrets_key` | this server holds the key the database records, and an export of it has been recorded | | `service_account` | it is not LocalSystem on Windows, not root on Linux | Each item is `done`, `action_needed`, `cannot_tell` or `not_needed`. Port 25 is never the submission port, and a plain submission port beside a secured one still needs action. **Where it is.** `GET /api/v1/setup/checklist` (reading needs `view_configuration`; `?dns=0` skips the lookups), the Deck's *Checklist* view and, on Windows, COM's `Utilities.GetSetupChecklist(LookUpDns)` and the Control Panel's *External setup* page and dashboard. ### Passwords only over TLS, on a server you upgraded "Require TLS for auth" is an option of an IP range, not of a port, and covers SMTP, POP3, IMAP and ManageSieve alike. 1. Read the `sign_in_tls` item: it names each range, and the listeners where a password from there would cross the network in the clear. 2. If the finding is `sign_in_no_tls`, a listener has no TLS to start, and requiring TLS would refuse every client on it: give those listeners a certificate and STARTTLS first. This finding carries no fix. 3. **Set every mail client that signs in from another machine to STARTTLS first.** Once the range requires TLS, a client that signs in without TLS is refused; that is why an upgrade is never changed by itself. 4. When the finding is `sign_in_plain`, apply its fix: the Deck's button *Require TLS for sign-in from this range*, or `PUT /api/v1/ipranges/{id}` with `{"require_tls_for_auth": true}`, which needs `manage_settings`. On Windows the Control Panel opens its IP ranges page, where the setting is *Require TLS for auth*. 5. Replace the self-signed certificate if you have one, so that `submission_self_signed` clears. ### Secrets at rest and the stored-secrets key Until 6.3.4 the administrator password (unsalted MD5, from the Windows installer), each account's second factor and the `[Settings]` secrets were kept as written. **The administrator password** is now written as PBKDF2. An older MD5 or SHA-256 value is rewritten as PBKDF2 at the first successful administrator sign-in where the service can write the file. Where it cannot (`NT SERVICE\hMailServer`, or the Linux package's root-owned file) the old value stays, `HM7494` says so once, and the verb rewrites it: on Windows `hMailServer.exe --set-admin-password` from an elevated prompt, with the password in `HM_ADMIN_PASSWORD`, or on one line of standard input from a pipe or a file for a password in plain ASCII (Windows PowerShell 5.1 turns every other character into `?`), never as an argument; on Linux `sudo hmailserver --set-admin-password`, which reads standard input. **Second factors and ten `[Settings]` secrets** (`SRSSecret`, `BATVSecret`, `OAuth2HmacSecret` and the rest) are sealed with AES-256-GCM under the installation's stored-secrets key, in the form `HMK1::`, **once the installation has a key**. The key is kept in each server's `hMailServer.ini` as `[Security] StoredSecretsKey`, and never in the database or a backup archive. **Who gets a key without asking.** The Windows installer makes one on a new installation. On Linux, `hmailserver --create-database` makes one when it creates the schema and can write the configuration: in a container, whose `/etc/hmailserver` the service owns, it does (no image of 6.3.4 is published); run by the package as the service's user, it cannot. An upgraded installation, and a package installation on Linux, seal nothing until an administrator makes the key: 1. Make the key on one node. Windows, from an elevated prompt beside `hMailServer.exe`: `.\hMailServer.exe --create-stored-secrets-key`. Linux: `sudo hmailserver --create-stored-secrets-key`. 2. Export it at once: `.\hMailServer.exe --export-stored-secrets-key D:\Escrow\hmailserver-stored-secrets.key`, or `sudo hmailserver --export-stored-secrets-key FILE`. The export writes a new file, never over one that exists, readable only by SYSTEM and the administrators on Windows and mode 0600 on Linux, and prints the key id, never the key. 3. Restart the service on that node (`Restart-Service hMailServer`, or `sudo systemctl restart hmailserver`). Its next start seals what is stored as written; until then nothing is sealed. 4. On every other node that shares the database, `--import-stored-secrets-key FILE`, then restart each. An import is refused while the node holds another key that opens: a standby or new node whose installer made a key of its own removes its `[Security] StoredSecretsKey` line first, and until then the shared database refuses it (`HM7495`). 5. Keep the exported file with the database backups, stored apart from them. **Why the export matters.** The product's own backup archive carries these secrets opened, and a restore seals them under the restoring server's key, so set a backup passphrase. A database-level backup (a dump, a copy of the `.sdf`) restored without the key brings back every second factor and `[Settings]` secret unopenable. A copy of `hMailServer.ini` is no copy of the key on Windows, since its line is sealed under that machine's DPAPI; never copy that line from one machine to another. **When a node lacks the key** it says `HM7495` at every start and makes no key of its own; until the key is imported, a `[Settings]` secret sealed under it runs as though unset (`HM7492`) and a second factor refuses every code (`HM7493`). A key lost for good is recovered from with `--forget-sealed-secrets `, with the service stopped (exit 4 otherwise), which clears only what that key sealed: every second factor is enrolled again, every `[Settings]` secret entered again. **`ProtectStoredSecretsWithDPAPI=0` (default 1) has a cost.** The next start opens what was sealed and stores it as written, readable by anyone who can read the file or the database, and the machine-bound secrets are written with the legacy Blowfish scheme, whose key is in the source code: the way back, not a setting for a new server. **Two things to do after an upgrade.** Since 6.3.4 `PasswordPepper` stays in `hMailServer.ini` only and is left out of every backup, so keep a copy of that value with the backups. And a password hash stored before 6.3.4 from a password with a character above U+00FF is refused, to its own password too: an administrator sets such passwords again, and the application log names each account as its user tries to sign in. ### Checking new passwords against published breaches Off by default, because it sends something off the server. When on, a new password found in the published corpus of breached passwords is refused. | Setting (REST key) | Default | What it means | |---|---|---| | `breach_check_enabled` | off | The whole feature | | `breach_check_url` | `https://api.pwnedpasswords.com/range/` | Where the five characters are sent | | `breach_check_minimum_count` | `1` | Refuse a password seen this many times or more | | `breach_check_unreachable_action` | `allow` | `allow` lets a change through unchecked; `refuse` refuses it | **What leaves the server** is the first five characters of the password's SHA-1 and nothing else. The default endpoint is a third-party service, Have I Been Pwned's Pwned Passwords, and an organisation that cannot send it those five characters, its address and the time runs its own mirror. **When it runs.** When a password is chosen, by the person or by an administrator, and never at logon. An endpoint that gives no complete answer within eight seconds counts as unusable, and the error log records `HM6600` (let through) or `HM6601` (refused). Under `refuse`, an endpoint that is down refuses every password change on the server. Bulk imports make one request per account, so run them against a mirror or with the check off. **Where it is set.** The four settings are `hm_settings` rows, not `hMailServer.ini` lines, and a change applies from the next password chosen. They are on the Control Panel's *Administrative access → Password* page (the *Passwords found in published breaches* card), in the Deck's Settings view, over REST and in `hmctl settings set`: ``` hmctl settings set --set breach_check_enabled=true ``` ### Compromised-account detection This judges every authenticated submission (SMTP, the webmail's send, a scheduled draft) against the account's own history of the last 30 days, without reading the message. **It starts in `watch` on every installation, new or upgraded**: the baseline is built, the `account.compromised` alert says what enforcing would have done, and nothing else happens. Four signals score: **volume** (2 points), the last hour's messages reaching `compromise_volume_floor` (30) and exceeding `compromise_volume_factor` (4) times the busiest baseline hour; **recipients** (2), the last hour's distinct recipients reaching `compromise_recipient_floor` (20), at least 80 per cent of them new to the account; **hour** (1), an hour of the day (UTC) the account has not submitted at in the baseline; **address** (1), a client network (the /24 of an IPv4 address, the /64 of an IPv6 one) not seen for this account. `compromise_score_to_lock` (3) is a takeover. In `enforce`, the message is held, the account is locked until an administrator unlocks it, and the alert and an `account.locked` webhook are raised. The client is still answered `250 Queued`. Held mail is reviewed on the Deck's **Held for review** page and under `/api/v1/review`; a held message nobody has looked at in `compromise_held_days` days (7) is discarded. Watch it before enforcing: the false positive is a person locked out of their own mail. There is no Control Panel page for it. ### Roles instead of the administrator password Rather than share the administrator password, give each administrator a role in the domains they look after, on the Deck's *Roles* view, over REST, or on Windows in the Control Panel or over COM. Nothing changes on the day you upgrade: an account with no assignment holds the authority its admin level has always given it, and a narrow role cannot take over a wider one, so a help desk is refused a domain administrator's password reset. [Administrator roles end to end](/topic/28237) has the permissions, how to give a role, and the two stores only the administrator password opens. ### The rest of the list - **Auto-ban** is on by default: 3 failures within 30 minutes, a 60-minute ban. `MaxInvalidLogonAttempts` at `0` disables it, and `AutoBanMinutes` at `0` disconnects the client but bans nothing. **Account lockout** (`AccountLockoutThreshold`) is off until set, and counts by the name being guessed at. [IP ranges, auto-ban, lockout and the rest](/topic/28239) has both in full. - **The server-enforced administrator second factor** refuses the password alone everywhere, the REST API included. - **On Windows**, move a service still on LocalSystem with the installer, the Control Panel's button, or from an elevated Command Prompt in `Bin`: `hMailServer.exe --confirm /GrantServiceAccess`, then, only if that ended in 0, `hMailServer.exe --service-account virtual /Register`, then `net stop hMailServer && net start hMailServer`. Check first what the virtual account cannot do, listed in [Installing on Windows](/topic/28227). - **The event-script flaw fixed in 6.3.4** (CVE-2026-100741; its record was not yet public when this was written) is in every release from 6.0.0 to 6.3.3: upgrade, and until you can, switch event scripting off. See [Event scripts and the 6.3.4 security fix](/topic/28261) and [the 6.3.4 security announcement](/topic/28218). ### What it does not do - Passkeys, single sign-on and the session policies do not reach IMAP, POP3 or SMTP, which still take the password or an app password. - The breach check does not cache, and does not expire a password that enters the corpus later. - Compromised-account detection does not drop a session that is already up. - The stored-secrets key protects against a copy of the database that leaves without the file, not against a local account on the server. ### Where to read more - Wiki: [Security Hardening](https://gitlab.com/hmailserver/hmailserver/-/wikis/Security-Hardening). - Documents: [SecureDefaults.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SecureDefaults.md), [SetupChecklist.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SetupChecklist.md), [SecretsAtRest.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SecretsAtRest.md), [PasswordBreachCheck.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/PasswordBreachCheck.md), [CompromisedAccounts.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/CompromisedAccounts.md) and [Roles.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Roles.md). - The public documentation: [chapter 17, Security hardening](https://www.progressiverobot.com/hmailserver-documentation/#17-security-hardening). - On this forum: [Upgrading to 6.3.4](/topic/28193), [The Control Deck](/topic/28192) and [hMailServer on Linux](/topic/28189). A suspected vulnerability is reported privately, never in a forum post: by a confidential GitLab issue, or by email to the project's Service Desk address, contact-project+hmailserver-hmailserver-86730559-issue-@incoming.gitlab.com, which GitLab files as a confidential issue ([SECURITY.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/SECURITY.md)). --- Canonical: https://www.hmailserver.co.uk/topic/28241/hardening-a-public-mail-server-secure-defaults-checklist-sealed-secrets-and-roles # Hardening a public mail server: secure defaults, checklist, sealed secrets and roles *Written for 6.3.4. hMailServer 6.3.7 (17 October 2026) is a security release that bears on this guide. On Windows, `hMailServer.exe /ProtectFolders`, which setup runs at every installation and upgrade, now hardens the program folder, `Bin` and `ControlPanel`: where hMailServer was installed in a folder that took its permissions from the root of a drive, any account that could sign in to the computer could replace the programs the service runs (CVE-2026-107571); the default folder under Program Files was never affected, and an installation at a drive's root is hardened by the upgrade itself. The `serveradmin` role given with a list of domains is confined to those domains over COM as over REST, a credential confined to named domains is refused the changes that reached beyond its domains over REST and COM, and a state-changing REST request that a browser reports as cross-site is refused. SMTP, POP3 and ManageSieve refuse a SASL PLAIN sign-in that asks to act as another account, and the stored-secrets key is no longer left in memory freed without being written over. An upgraded Windows server keeps the .NET 10 Desktop Runtime it had, 10.0.12 where 6.3.4 to 6.3.6 installed it, against which Microsoft published advisories on 13 October 2026: bring it to 10.0.13 through Microsoft Update or Microsoft's installer. Upgrade every server running 6.3.6 or earlier. See [the 6.3.7 announcement](/topic/28277).* This guide covers what stands between a mail server on the public internet and the people who guess passwords at it. All of it is new in 6.3.4 except auto-ban, account lockout, the administrator's second factor and the IP range option *Require TLS for auth*, which are older; 6.3.4 sets that option on a new installation and judges it in the checklist. The one thing to know first: **the secure defaults are applied only to a database the installation itself created, and an upgraded server is never changed.** After an upgrade your listeners, IP ranges and service account are as they were and nothing is sealed until you make a key; the checklist says what is left. ### What a new installation starts with, and what an upgrade keeps In 6.3.3 and earlier a new installation listened on ports 25, 587, 110 and 143 with no TLS, and neither shipped IP range required TLS for authentication. The create scripts still make that database; the Windows installer, and the Linux packages' first start, now change it before the server first listens. | | A new installation since 6.3.4 | An upgraded server | |---|---|---| | Ports 25 and 587 (SMTP), 110 (POP3), 143 (IMAP) | STARTTLS offered, optional | as it was | | IP range *Internet* (every IPv4 address) | **TLS required for authentication** | as it was; the checklist flags it | | IP range *My computer* (127.0.0.1) | unchanged | as it was | | Certificates | *Self-signed (made at installation)* | as they were | | Windows service account | `NT SERVICE\hMailServer` | the account it had (LocalSystem, if it began before 6.3.4) | **What a client sees.** From another machine, `AUTH` on SMTP is not offered before STARTTLS and is refused with `530 A SSL/TLS-connection is required for authentication.` if it is sent; IMAP `LOGIN` and `AUTHENTICATE` answer `BAD A SSL/TLS-connection is required for authentication.`; POP3 `USER` and `AUTH` answer `-ERR`. After STARTTLS each works as before. From the server itself, on 127.0.0.1, nothing has changed, and mail from other servers on port 25 is taken with or without TLS. **Why STARTTLS is optional.** A port's requirement applies to every address: on port 25 it would refuse every server that cannot start TLS, on the others this computer's own clients. The range's rule protects the password. **The certificate.** `self-signed-certificate.pem` and its key `self-signed-key.pem` are in `/Certificates`: `C:\Program Files\hMailServer\Data\Certificates` on a default Windows installation, `/var/lib/hmailserver/Certificates` from the Linux packages, where the key is mode 0600 and owned by the service's user. It is RSA 2048, runs 825 days, names the host and gives the organisation *hMailServer, self-signed at installation*. It encrypts, but nothing vouches for it, so mail clients warn. Replace it with ACME (set `AcmeEnabled`, `AcmeContactEmail` and `AcmeDomains`; the first certificate goes to every listener that holds the self-signed one) or with your own, set on each port. See [Let's Encrypt certificates are automatic](/topic/28187). ### Where the defaults are applied The create step marks its database with the `hm_settings` row `NewInstallDefaultsPending`; the verb `--apply-new-install-defaults` requires the mark and removes it once it has decided. | Platform | When it runs | What to know | |---|---|---| | Windows | The installer, after the database step of a new installation, with the service stopped | A database DBSetup attached, rather than created, carries no mark: exit 5, logged, nothing changed | | Linux (`.deb`, `.rpm`) | The unit's first start after `sudo -u hmailserver hmailserver --create-database`, which marks the database and leaves a one-line file, `.new-installation`, in the `DataFolder` | The unit runs `ExecStartPre=-/usr/bin/hmailserver --apply-new-install-defaults --if-marked`: nothing unless that file is there, then one try at the database, given room by `TimeoutStartSec=300` (the leading `-` forgives a failure, not a timeout). It ignores a mark made more than a day from its clock; a run by hand has no such limit | | By hand | A container, a server started by hand rather than by the unit, or defaults left for later | With the service stopped; on Linux as the service's user, because the verb refuses to run as root | By hand, on Windows and then on Linux: ``` Stop-Service hMailServer & "C:\Program Files\hMailServer\Bin\hMailServer.exe" --apply-new-install-defaults Start-Service hMailServer ``` ``` sudo systemctl stop hmailserver sudo -u hmailserver hmailserver --apply-new-install-defaults sudo systemctl start hmailserver ``` Exit codes: 0 applied (with `--if-marked`, also nothing to do); 1 something could not be made or written (`HM7510`, `HM7511` in the error log); 2 no database named, or not readable enough to judge, or on Linux run as root; 4 the Windows service is running; 5 not a database this installation created, or not as the create script left it: nothing changed. **On Linux, mind the data directory.** A `DataFolder` outside the unit's `ReadWritePaths` (`/var/lib/hmailserver` and `/var/log/hmailserver`) is read-only to the service, so the first start can neither write the certificate nor store mail there. Add the directory with a drop-in (`sudo systemctl edit hmailserver`, then `ReadWritePaths=/srv/mail` under `[Service]`) before that first start. **Not yet proved on a clean machine.** [SecureDefaults.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SecureDefaults.md)'s checks that the installer applies the defaults are not yet written, and those for the `.deb` ran only on GitHub Actions, so nothing runs them today. ### The setup checklist The checklist judges every server, new or upgraded, in twelve items. | Key | Done when | |---|---| | `profile` | a setting profile has been applied; asked for on a server with fewer than ten accounts | | `domain` | an active domain exists, and none has a name the server reserves for its own directories | | `account` | at least one account exists | | `submission` | every SMTP listener other than port 25 has STARTTLS or implicit TLS, and a certificate other than the self-signed one | | `sign_in_tls` | every IP range that lets a client connect from another address requires TLS for authentication, or reaches only listeners that start TLS before a password | | `spf`, `dkim`, `dmarc` | every active domain has a TXT record beginning `v=spf1`, signs with a key whose record is published at `._domainkey.`, and has a TXT record beginning `v=DMARC1` at `_dmarc.` | | `backup` | the destination is set and the folder exists | | `second_factor` | the administrator password is enrolled with a one-time code | | `stored_secrets_key` | this server holds the key the database records, and an export of it has been recorded | | `service_account` | it is not LocalSystem on Windows, not root on Linux | Each item is `done`, `action_needed`, `cannot_tell` or `not_needed`. Port 25 is never the submission port, and a plain submission port beside a secured one still needs action. **Where it is.** `GET /api/v1/setup/checklist` (reading needs `view_configuration`; `?dns=0` skips the lookups), the Deck's *Checklist* view and, on Windows, COM's `Utilities.GetSetupChecklist(LookUpDns)` and the Control Panel's *External setup* page and dashboard. ### Passwords only over TLS, on a server you upgraded "Require TLS for auth" is an option of an IP range, not of a port, and covers SMTP, POP3, IMAP and ManageSieve alike. 1. Read the `sign_in_tls` item: it names each range, and the listeners where a password from there would cross the network in the clear. 2. If the finding is `sign_in_no_tls`, a listener has no TLS to start, and requiring TLS would refuse every client on it: give those listeners a certificate and STARTTLS first. This finding carries no fix. 3. **Set every mail client that signs in from another machine to STARTTLS first.** Once the range requires TLS, a client that signs in without TLS is refused; that is why an upgrade is never changed by itself. 4. When the finding is `sign_in_plain`, apply its fix: the Deck's button *Require TLS for sign-in from this range*, or `PUT /api/v1/ipranges/{id}` with `{"require_tls_for_auth": true}`, which needs `manage_settings`. On Windows the Control Panel opens its IP ranges page, where the setting is *Require TLS for auth*. 5. Replace the self-signed certificate if you have one, so that `submission_self_signed` clears. ### Secrets at rest and the stored-secrets key Until 6.3.4 the administrator password (unsalted MD5, from the Windows installer), each account's second factor and the `[Settings]` secrets were kept as written. **The administrator password** is now written as PBKDF2. An older MD5 or SHA-256 value is rewritten as PBKDF2 at the first successful administrator sign-in where the service can write the file. Where it cannot (`NT SERVICE\hMailServer`, or the Linux package's root-owned file) the old value stays, `HM7494` says so once, and the verb rewrites it: on Windows `hMailServer.exe --set-admin-password` from an elevated prompt, with the password in `HM_ADMIN_PASSWORD`, or on one line of standard input from a pipe or a file for a password in plain ASCII (Windows PowerShell 5.1 turns every other character into `?`), never as an argument; on Linux `sudo hmailserver --set-admin-password`, which reads standard input. **Second factors and ten `[Settings]` secrets** (`SRSSecret`, `BATVSecret`, `OAuth2HmacSecret` and the rest) are sealed with AES-256-GCM under the installation's stored-secrets key, in the form `HMK1::`, **once the installation has a key**. The key is kept in each server's `hMailServer.ini` as `[Security] StoredSecretsKey`, and never in the database or a backup archive. **Who gets a key without asking.** The Windows installer makes one on a new installation. On Linux, `hmailserver --create-database` makes one when it creates the schema and can write the configuration: in a container, whose `/etc/hmailserver` the service owns, it does (no image of 6.3.4 is published); run by the package as the service's user, it cannot. An upgraded installation, and a package installation on Linux, seal nothing until an administrator makes the key: 1. Make the key on one node. Windows, from an elevated prompt beside `hMailServer.exe`: `.\hMailServer.exe --create-stored-secrets-key`. Linux: `sudo hmailserver --create-stored-secrets-key`. 2. Export it at once: `.\hMailServer.exe --export-stored-secrets-key D:\Escrow\hmailserver-stored-secrets.key`, or `sudo hmailserver --export-stored-secrets-key FILE`. The export writes a new file, never over one that exists, readable only by SYSTEM and the administrators on Windows and mode 0600 on Linux, and prints the key id, never the key. 3. Restart the service on that node (`Restart-Service hMailServer`, or `sudo systemctl restart hmailserver`). Its next start seals what is stored as written; until then nothing is sealed. 4. On every other node that shares the database, `--import-stored-secrets-key FILE`, then restart each. An import is refused while the node holds another key that opens: a standby or new node whose installer made a key of its own removes its `[Security] StoredSecretsKey` line first, and until then the shared database refuses it (`HM7495`). 5. Keep the exported file with the database backups, stored apart from them. **Why the export matters.** The product's own backup archive carries these secrets opened, and a restore seals them under the restoring server's key, so set a backup passphrase. A database-level backup (a dump, a copy of the `.sdf`) restored without the key brings back every second factor and `[Settings]` secret unopenable. A copy of `hMailServer.ini` is no copy of the key on Windows, since its line is sealed under that machine's DPAPI; never copy that line from one machine to another. **When a node lacks the key** it says `HM7495` at every start and makes no key of its own; until the key is imported, a `[Settings]` secret sealed under it runs as though unset (`HM7492`) and a second factor refuses every code (`HM7493`). A key lost for good is recovered from with `--forget-sealed-secrets `, with the service stopped (exit 4 otherwise), which clears only what that key sealed: every second factor is enrolled again, every `[Settings]` secret entered again. **`ProtectStoredSecretsWithDPAPI=0` (default 1) has a cost.** The next start opens what was sealed and stores it as written, readable by anyone who can read the file or the database, and the machine-bound secrets are written with the legacy Blowfish scheme, whose key is in the source code: the way back, not a setting for a new server. **Two things to do after an upgrade.** Since 6.3.4 `PasswordPepper` stays in `hMailServer.ini` only and is left out of every backup, so keep a copy of that value with the backups. And a password hash stored before 6.3.4 from a password with a character above U+00FF is refused, to its own password too: an administrator sets such passwords again, and the application log names each account as its user tries to sign in. ### Checking new passwords against published breaches Off by default, because it sends something off the server. When on, a new password found in the published corpus of breached passwords is refused. | Setting (REST key) | Default | What it means | |---|---|---| | `breach_check_enabled` | off | The whole feature | | `breach_check_url` | `https://api.pwnedpasswords.com/range/` | Where the five characters are sent | | `breach_check_minimum_count` | `1` | Refuse a password seen this many times or more | | `breach_check_unreachable_action` | `allow` | `allow` lets a change through unchecked; `refuse` refuses it | **What leaves the server** is the first five characters of the password's SHA-1 and nothing else. The default endpoint is a third-party service, Have I Been Pwned's Pwned Passwords, and an organisation that cannot send it those five characters, its address and the time runs its own mirror. **When it runs.** When a password is chosen, by the person or by an administrator, and never at logon. An endpoint that gives no complete answer within eight seconds counts as unusable, and the error log records `HM6600` (let through) or `HM6601` (refused). Under `refuse`, an endpoint that is down refuses every password change on the server. Bulk imports make one request per account, so run them against a mirror or with the check off. **Where it is set.** The four settings are `hm_settings` rows, not `hMailServer.ini` lines, and a change applies from the next password chosen. They are on the Control Panel's *Administrative access → Password* page (the *Passwords found in published breaches* card), in the Deck's Settings view, over REST and in `hmctl settings set`: ``` hmctl settings set --set breach_check_enabled=true ``` ### Compromised-account detection This judges every authenticated submission (SMTP, the webmail's send, a scheduled draft) against the account's own history of the last 30 days, without reading the message. **It starts in `watch` on every installation, new or upgraded**: the baseline is built, the `account.compromised` alert says what enforcing would have done, and nothing else happens. Four signals score: **volume** (2 points), the last hour's messages reaching `compromise_volume_floor` (30) and exceeding `compromise_volume_factor` (4) times the busiest baseline hour; **recipients** (2), the last hour's distinct recipients reaching `compromise_recipient_floor` (20), at least 80 per cent of them new to the account; **hour** (1), an hour of the day (UTC) the account has not submitted at in the baseline; **address** (1), a client network (the /24 of an IPv4 address, the /64 of an IPv6 one) not seen for this account. `compromise_score_to_lock` (3) is a takeover. In `enforce`, the message is held, the account is locked until an administrator unlocks it, and the alert and an `account.locked` webhook are raised. The client is still answered `250 Queued`. Held mail is reviewed on the Deck's **Held for review** page and under `/api/v1/review`; a held message nobody has looked at in `compromise_held_days` days (7) is discarded. Watch it before enforcing: the false positive is a person locked out of their own mail. There is no Control Panel page for it. ### Roles instead of the administrator password Rather than share the administrator password, give each administrator a role in the domains they look after, on the Deck's *Roles* view, over REST, or on Windows in the Control Panel or over COM. Nothing changes on the day you upgrade: an account with no assignment holds the authority its admin level has always given it, and a narrow role cannot take over a wider one, so a help desk is refused a domain administrator's password reset. [Administrator roles end to end](/topic/28237) has the permissions, how to give a role, and the two stores only the administrator password opens. ### The rest of the list - **Auto-ban** is on by default: 3 failures within 30 minutes, a 60-minute ban. `MaxInvalidLogonAttempts` at `0` disables it, and `AutoBanMinutes` at `0` disconnects the client but bans nothing. **Account lockout** (`AccountLockoutThreshold`) is off until set, and counts by the name being guessed at. [IP ranges, auto-ban, lockout and the rest](/topic/28239) has both in full. - **The server-enforced administrator second factor** refuses the password alone everywhere, the REST API included. - **On Windows**, move a service still on LocalSystem with the installer, the Control Panel's button, or from an elevated Command Prompt in `Bin`: `hMailServer.exe --confirm /GrantServiceAccess`, then, only if that ended in 0, `hMailServer.exe --service-account virtual /Register`, then `net stop hMailServer && net start hMailServer`. Check first what the virtual account cannot do, listed in [Installing on Windows](/topic/28227). - **The event-script flaw fixed in 6.3.4** (CVE-2026-100741; its record was not yet public when this was written) is in every release from 6.0.0 to 6.3.3: upgrade, and until you can, switch event scripting off. See [Event scripts and the 6.3.4 security fix](/topic/28261) and [the 6.3.4 security announcement](/topic/28218). ### What it does not do - Passkeys, single sign-on and the session policies do not reach IMAP, POP3 or SMTP, which still take the password or an app password. - The breach check does not cache, and does not expire a password that enters the corpus later. - Compromised-account detection does not drop a session that is already up. - The stored-secrets key protects against a copy of the database that leaves without the file, not against a local account on the server. ### Where to read more - Wiki: [Security Hardening](https://gitlab.com/hmailserver/hmailserver/-/wikis/Security-Hardening). - Documents: [SecureDefaults.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SecureDefaults.md), [SetupChecklist.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SetupChecklist.md), [SecretsAtRest.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SecretsAtRest.md), [PasswordBreachCheck.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/PasswordBreachCheck.md), [CompromisedAccounts.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/CompromisedAccounts.md) and [Roles.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Roles.md). - The public documentation: [chapter 17, Security hardening](https://www.progressiverobot.com/hmailserver-documentation/#17-security-hardening). - On this forum: [Upgrading to 6.3.4](/topic/28193), [The Control Deck](/topic/28192) and [hMailServer on Linux](/topic/28189). A suspected vulnerability is reported privately, never in a forum post: by a confidential GitLab issue, or by email to the project's Service Desk address, contact-project+hmailserver-hmailserver-86730559-issue-@incoming.gitlab.com, which GitLab files as a confidential issue ([SECURITY.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/SECURITY.md)). --- Canonical: https://www.hmailserver.co.uk/topic/28242/compliance-ediscovery-legal-holds-personal-data-erasure-and-the-audit-trail # Compliance: eDiscovery, legal holds, personal data, erasure and the audit trail *Written for 6.3.4. hMailServer 6.3.7 (17 October 2026) regroups the Control Panel's sidebar: the audit trail, journaling and eDiscovery are a Compliance group of their own rather than the last rows of *Monitoring & troubleshooting*, as the table below has it, and Ctrl+K takes every old title and route to its page. See [the 6.3.7 announcement](/topic/28277).* This guide covers what an operator is asked for when a lawyer, a regulator or a person with a data request arrives: eDiscovery (cases, searches, legal holds and exports), a person's own data (the export and the erasure), and the audit trail that records who did what. eDiscovery, the person's data as one archive, the erasure of log lines, the erasure's reach into the stores beside the mailbox, and the audit trail are new since 6.3.4. The one thing to know first: **cases, their files and their holds are not in the backup.** A server rebuilt from a backup has no cases, no exports and no holds, so place the holds again before its mail can be deleted. ### Where it is | Where | What | |---|---| | The Control Deck, *Discovery* view | The whole of it: cases, the search form, holds with **Lift the hold**, exports with **Download**, **Open** and **Close** | | REST, `/api/v1/discovery/cases` | The same ten routes the Deck calls, in the OpenAPI document | | The Control Panel (Windows only), *Monitoring & troubleshooting → Discovery* (*eDiscovery*, in the *Compliance* group, since 6.3.7) | Read-only: the cases and their lines of the audit trail, read over the REST listener, which must be on (`RestApiPort`). It changes nothing | | COM, `hmctl` | Neither | On Linux, then, discovery is worked from the Deck or over REST. Discovery and the erasure do nothing until somebody opens a case or asks for an erasure; the audit trail is on from the first start. Two neighbouring tools are not this one. The archive has had a legal hold of its own since before 6.3.4 (`/api/v1/archive/{id}/hold`), which keeps one archived copy from the archive's retention sweep ([Quotas, retention and the archive](/topic/28236)). Journaling, a copy of a domain's or an account's mail sent elsewhere as it passes, has its own topic: [Journaling and dual delivery](/topic/28271). ### Cases A case has a name (up to 255 characters), a description (up to 1,000), who opened it and when, and a state: open or closed. Its searches, holds and exports belong to it. **Closing a case releases every hold of it that is still live**, each release on the audit trail; a closed case runs no search, places no hold and makes no export, and what it made stays readable and downloadable. A case is never deleted, and an address's erasure does not reach it. The files a case makes are under the data directory, in `Discovery\case-\`: `search-.json` for each search and `export-.zip` for each export. Nothing removes an export; delete its zip by hand when it is no longer needed. ### Searching `POST /api/v1/discovery/cases/{id}/searches` takes: | Part | What it means | |---|---| | `custodians` (required, at most 100) | An address, a domain, or `*` for every account on the server and the whole archive | | `from`, `to` (optional, `YYYY-MM-DD`) | The day a message is dated by its `Date` field, read as UTC; else the day the server stored it | | `words` (optional, at most 50) | Every word must be found, a quoted phrase as one: a case-insensitive substring of what IMAP `SEARCH TEXT` reads | | `attachment` (optional) | `name`, `type` and `min_kb`; `{}` is any attachment at all | | `scope` | `mailboxes`, `archive` or `both`, the default | There is no sender or recipient criterion: a word is looked for in the whole decoded header, and a person is named as a custodian. A query the server cannot take is refused with `400` and the reason; `202` answers with the search's id, and the search is queued. **Where it looks.** For each custodian account: its folders, its deleted items (what a hold kept, and what `RecoverDeletedDays` keeps), and the archive. A search copies nothing; it lists each hit, and lists separately what it could not read, with the reason. **What it cannot see.** - **100,000 hits.** A search stops there and says `truncated`. Narrow it and run it again. - **Text inside a PDF, a Word document or a spreadsheet.** Only `text/*` attachments are read, up to 1 MB of characters a message. An attachment criterion still finds the PDF by name, type and size. - **More than 80 MB a message.** It is listed as not searched when the query asks about its body, and never dropped as a non-match. - **Zero-access mailboxes and messages encrypted end to end** are matched on their headers alone and marked `headers_only`. - **The stores beside the mailbox**, `\Confidential` and `\Protected`, and archived copies with no index row (a bounce, a delivery report). **The full-text index is a filter, never the answer.** It is off by default (`IndexerFullText`). When it is on, a message the index proves cannot contain a word is ruled out; every other message is read. **One at a time.** Searches and exports are queued and run by one runner, the server's own. Only one hMailServer service may run against a database at a time ([Warm standby and high availability](/topic/28232)), so a standby started after a failover takes over the queue rather than sharing it. ### Holds `POST /api/v1/discovery/cases/{id}/holds` takes one of: | Body | Holds | |---|---| | `{"scope":"account","target":"alice@example.com"}` | Every message of the account, and its archived copies | | `{"scope":"domain","target":"example.com"}` | Every message of every account in the domain, and their archived copies | | `{"scope":"archive","search":12}` | The archived copies search 12 of this case matched, at most 10,000 at once | A hold is live until it is released or its case is closed, and what it keeps is kept by any live hold of any case. | Path | Under a hold | |---|---| | The person's delete (IMAP `EXPUNGE`, POP3 `DELE`, the webmail's delete, a folder deleted) and an administrator's delete of a message | The message leaves the folder, and a copy is kept among the account's deleted items whatever `RecoverDeletedDays` says (0 by default). If the copy cannot be made, the delete is refused (HM7333) | | An IMAP `MOVE` out of the account | The original is kept among the held account's deleted items | | Every message of the account deleted at once | Refused, `409` over REST | | The account's own filters at delivery: a rule's Delete or Reject, a Sieve `discard`, `reject` or `redirect`, a forward that keeps no copy | The copy is kept among the deleted items instead of dropped; if it cannot be kept, the message is delivered | | Retention, the recover-deleted sweep, deleting a deleted item for good | Nothing of the held account is deleted | | A mailbox restore with `mode` replace, a full restore with the domains option | Refused (the full restore as HM5014) | | Moving the account to another domain, removing its mailbox key, deleting the account or its domain | Refused, over REST, SCIM and COM alike | | The archive's retention sweep and an address's erasure | A held copy keeps its file and its index row | **What a hold does not reach:** the server's and a domain's rules, which decide before a message is split between its recipients; spam and viruses refused or deleted before a message is the account's, and what the quarantine discards; the delivery queue; the confidential and password-protected stores; archived copies with no index row. If `ArchiveDir` was put inside the data directory, a full restore with the messages option replaces held archived copies too, so keep the archive outside it. **The hold is not hidden from the custodian.** The account's own `GET /api/v1/me/recoverable` answers `held: true` and lists what the hold kept, and the webmail's *Recover deleted items* page offers no delete for good. **It fails closed.** When the holds cannot be read, every question is answered held, and the error log says so once a minute (HM7332). A hold is a row in the database. The server reads the holds again as soon as one is placed or released, and no more than five seconds after a change made elsewhere. ### Exports `POST .../searches/{search}/export` queues an export of a finished search's hits. The zip holds: - **Each hit as EML**, the message's own bytes as the server stores them, under `//-.eml`, and an mbox per custodian folder beside them, in parts of at most 256 MB. - **`SHA256SUMS`**, so that `sha256sum -c SHA256SUMS` checks the unpacked export. - **`manifest.json`**: the case, the search as it was run, who ran it, who asked for the export, every item with its path, SHA-256 and size, and every item withheld, with the reason. The zip's own SHA-256 is kept with the export, and the case answers it (`exports[].sha256`). **Withheld:** a zero-access mailbox's messages, and anything deleted, moved or swept since the search ran. Search again to find where a message is now. **Limits.** The zip written is the 32-bit format, so no export passes 4 GB: narrow the search and export each part. A download past 500 MB answers `413` with where the file is on the server's disk and its SHA-256. An export that would leave the data directory's volume with less than a gigabyte free fails. A discovery export is never a PST; an examiner who needs one converts the EML files with a third-party converter. ### Who may | Permission | Allows | Held by | |---|---|---| | `view_discovery` | Reading the cases, their searches and hits, holds and exports | The server administrator and the `auditor` role | | `manage_discovery` | Opening and closing a case, running a search, placing and releasing a hold, making an export | The server administrator; a custom role may be given it | Downloading an export needs `view_mailbox` as well, since it hands over the mail itself, the custodians' deleted items among it. A search that reads the archive needs `view_archive` as well. A credential confined to named domains names no custodian outside them, never `*`, and is answered `404` for a case that reaches outside them. ### A person's data: the export One .zip of everything the server holds for an account. The person downloads it from the webmail's *Your data* card (`GET /api/v1/me/export.zip`), with no administrator involved; that download is refused (403) to a request authenticated with an app password. The administrator has the Deck's "Everything (.zip)" on the account row, `GET /api/v1/accounts/{address}/export.zip` (the administrator password's alone), `POST /api/v1/accounts/{address}/export` with `archive`, `hmctl privacy export
--archive ` and, on Windows, COM `Account.ExportArchive(file)`. | Entry | What it is | |---|---| | `manifest.json` | The address, the moment, and the counts of what follows | | `mail/.mbox` | Every folder with mail, one mbox each | | `account.xml` | What the backup carries for the account, less the credentials | | `contacts.vcf` | The address book as vCard 3.0 | | `calendars//` | Every calendar object as the client sent it | | `preferences.json`, `filters.sieve` | The webmail's preferences, and the active Sieve script | | `logs.txt` | The lines of the server's own log files that name the address, up to 50 MB | A download past 500 MB is refused with 413, and the operator writes the archive to disk instead; the disk form has no ceiling but the format's 4 GB. ### A person's data: the erasure `POST /api/v1/addresses/erase`, `hmctl privacy erase`, the Deck's "Erase traces" and, on Windows, the Control Panel's "Erase traces..." remove what deleting the account deliberately leaves: quarantine entries, message-trace rows, greylisting triplets, distribution-list memberships, aliases on either side, the bounce and complaint records, and the rows naming the address in other people's address books and first-contact memory. With the archive included, the person's archive tree is swept and every index row naming the address goes. The log lines are asked for by name (`include_logs` on the REST body, `--logs`): every occurrence becomes `[erased]`, and a log past 256 MB is reported instead of rewritten. **Since 6.3.4 it also reaches the stores beside the mailbox.** A message held for review that the address sent or would receive is discarded. The address is taken out of the recipients of a message that expires, and one it sent is ended. A password-protected message to it is ended. The recipients an account remembers writing to, the out-of-office memory, booking policies and correspondents' S/MIME certificates and OpenPGP keys lose it. **Every list of addresses is matched whole.** Erasing `ann@example.com` leaves `joann@example.com` and `ann@example.com.au` as they are. **A legal hold outranks it.** An erasure never deletes a message from a mailbox, so a hold meets it only in the stores it deletes from, and there the hold wins. The REST answer's `held` counts what a hold kept; run the erasure again once the case is closed. When the holds themselves cannot be read, the erasure fails (HM6250; `500` over REST) and says nothing about a legal hold. **What it leaves, by design,** among others: mail in mailboxes, the person's own and everybody else's, and a mailbox's deleted items; a list's archive of its posts; other people's calendars; configuration somebody made that names the address (a forward, a rule, a list's moderator); the audit trail, and the alert events, which nothing prunes yet; the spam trap record; the delivery queue and the webhook queue; the contacts in DMARC and TLS reports; the archive's Inbound folder; and backups, which are outside the running server's reach. [PersonalData.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/PersonalData.md) has the full list, with each reason. ### The audit trail, for a compliance reader The audit trail is on by default. It keeps one row in `hm_audit` for each administrative change made over COM, REST or the Deck, a mailbox's move included, with a few exceptions such as an API key made or revoked or a quarantined message released or deleted; it records changes, never reads. A password, key, token or other secret is recorded as `(changed)`, never as a value. Each row carries the SHA-256 of the row before it and its own, so `GET /api/v1/audit/verify` tells an edited row from a removed one: tampering is detectable, not impossible, since anyone who can write the table can recompute the whole chain. `audit_retention_days` is 0 by default, which keeps everything. Three things on it matter here. Every discovery search is recorded with its whole query; a download of a discovery export is recorded on the trail alone, so with the trail off a download is recorded nowhere; and an erasure is recorded with the address it erased wherever it deletes from recorded tables (aliases, list recipients, the archive index): those rows name the person until `audit_retention_days` ages them out. [Alerts, the audit trail, syslog and the journal](/topic/28273) has the columns, reading and putting a change back, who may read it, and what it does not record. ### Regulatory scope, in one paragraph The project's own determination, reviewed 12 August 2026, is that the EU Cyber Resilience Act and the revised Product Liability Directive do not place a manufacturer's obligations on it, because it is supplied outside a commercial activity. The document calls itself a self-assessment and not legal advice, and lists the triggers that would re-open it. Under GDPR and UK GDPR the project is neither controller nor processor: the operator of an installation is. The export, the erasure and the log retention limits exist so that operators can meet their own obligations. ### What it does not do - No PST export, and no text read from attachments but `text/*`. - No silent hold: the custodian can see what a hold kept. - No export past 4 GB and no download past 500 MB. - No saved queries, review tags, redaction or de-duplication across custodians: a message sent to two custodians is two hits. - No search of backups, and no case carried by a backup. ### Where to read more - Wiki: [Compliance and eDiscovery](https://gitlab.com/hmailserver/hmailserver/-/wikis/Compliance-and-eDiscovery). - Documents: [Discovery.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Discovery.md), [PersonalData.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/PersonalData.md), [AlertsAndAuditTrail.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/AlertsAndAuditTrail.md) and [RegulatoryScope.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/RegulatoryScope.md). - On this forum: [The Control Deck](/topic/28192), [Journaling and dual delivery](/topic/28271), and what a backup carries and what it leaves in [Backup and restore](/topic/28221). --- Canonical: https://www.hmailserver.co.uk/topic/28242/compliance-ediscovery-legal-holds-personal-data-erasure-and-the-audit-trail # Compliance: eDiscovery, legal holds, personal data, erasure and the audit trail *Written for 6.3.4. hMailServer 6.3.7 (17 October 2026) regroups the Control Panel's sidebar: the audit trail, journaling and eDiscovery are a Compliance group of their own rather than the last rows of *Monitoring & troubleshooting*, as the table below has it, and Ctrl+K takes every old title and route to its page. See [the 6.3.7 announcement](/topic/28277).* This guide covers what an operator is asked for when a lawyer, a regulator or a person with a data request arrives: eDiscovery (cases, searches, legal holds and exports), a person's own data (the export and the erasure), and the audit trail that records who did what. eDiscovery, the person's data as one archive, the erasure of log lines, the erasure's reach into the stores beside the mailbox, and the audit trail are new since 6.3.4. The one thing to know first: **cases, their files and their holds are not in the backup.** A server rebuilt from a backup has no cases, no exports and no holds, so place the holds again before its mail can be deleted. ### Where it is | Where | What | |---|---| | The Control Deck, *Discovery* view | The whole of it: cases, the search form, holds with **Lift the hold**, exports with **Download**, **Open** and **Close** | | REST, `/api/v1/discovery/cases` | The same ten routes the Deck calls, in the OpenAPI document | | The Control Panel (Windows only), *Monitoring & troubleshooting → Discovery* (*eDiscovery*, in the *Compliance* group, since 6.3.7) | Read-only: the cases and their lines of the audit trail, read over the REST listener, which must be on (`RestApiPort`). It changes nothing | | COM, `hmctl` | Neither | On Linux, then, discovery is worked from the Deck or over REST. Discovery and the erasure do nothing until somebody opens a case or asks for an erasure; the audit trail is on from the first start. Two neighbouring tools are not this one. The archive has had a legal hold of its own since before 6.3.4 (`/api/v1/archive/{id}/hold`), which keeps one archived copy from the archive's retention sweep ([Quotas, retention and the archive](/topic/28236)). Journaling, a copy of a domain's or an account's mail sent elsewhere as it passes, has its own topic: [Journaling and dual delivery](/topic/28271). ### Cases A case has a name (up to 255 characters), a description (up to 1,000), who opened it and when, and a state: open or closed. Its searches, holds and exports belong to it. **Closing a case releases every hold of it that is still live**, each release on the audit trail; a closed case runs no search, places no hold and makes no export, and what it made stays readable and downloadable. A case is never deleted, and an address's erasure does not reach it. The files a case makes are under the data directory, in `Discovery\case-\`: `search-.json` for each search and `export-.zip` for each export. Nothing removes an export; delete its zip by hand when it is no longer needed. ### Searching `POST /api/v1/discovery/cases/{id}/searches` takes: | Part | What it means | |---|---| | `custodians` (required, at most 100) | An address, a domain, or `*` for every account on the server and the whole archive | | `from`, `to` (optional, `YYYY-MM-DD`) | The day a message is dated by its `Date` field, read as UTC; else the day the server stored it | | `words` (optional, at most 50) | Every word must be found, a quoted phrase as one: a case-insensitive substring of what IMAP `SEARCH TEXT` reads | | `attachment` (optional) | `name`, `type` and `min_kb`; `{}` is any attachment at all | | `scope` | `mailboxes`, `archive` or `both`, the default | There is no sender or recipient criterion: a word is looked for in the whole decoded header, and a person is named as a custodian. A query the server cannot take is refused with `400` and the reason; `202` answers with the search's id, and the search is queued. **Where it looks.** For each custodian account: its folders, its deleted items (what a hold kept, and what `RecoverDeletedDays` keeps), and the archive. A search copies nothing; it lists each hit, and lists separately what it could not read, with the reason. **What it cannot see.** - **100,000 hits.** A search stops there and says `truncated`. Narrow it and run it again. - **Text inside a PDF, a Word document or a spreadsheet.** Only `text/*` attachments are read, up to 1 MB of characters a message. An attachment criterion still finds the PDF by name, type and size. - **More than 80 MB a message.** It is listed as not searched when the query asks about its body, and never dropped as a non-match. - **Zero-access mailboxes and messages encrypted end to end** are matched on their headers alone and marked `headers_only`. - **The stores beside the mailbox**, `\Confidential` and `\Protected`, and archived copies with no index row (a bounce, a delivery report). **The full-text index is a filter, never the answer.** It is off by default (`IndexerFullText`). When it is on, a message the index proves cannot contain a word is ruled out; every other message is read. **One at a time.** Searches and exports are queued and run by one runner, the server's own. Only one hMailServer service may run against a database at a time ([Warm standby and high availability](/topic/28232)), so a standby started after a failover takes over the queue rather than sharing it. ### Holds `POST /api/v1/discovery/cases/{id}/holds` takes one of: | Body | Holds | |---|---| | `{"scope":"account","target":"alice@example.com"}` | Every message of the account, and its archived copies | | `{"scope":"domain","target":"example.com"}` | Every message of every account in the domain, and their archived copies | | `{"scope":"archive","search":12}` | The archived copies search 12 of this case matched, at most 10,000 at once | A hold is live until it is released or its case is closed, and what it keeps is kept by any live hold of any case. | Path | Under a hold | |---|---| | The person's delete (IMAP `EXPUNGE`, POP3 `DELE`, the webmail's delete, a folder deleted) and an administrator's delete of a message | The message leaves the folder, and a copy is kept among the account's deleted items whatever `RecoverDeletedDays` says (0 by default). If the copy cannot be made, the delete is refused (HM7333) | | An IMAP `MOVE` out of the account | The original is kept among the held account's deleted items | | Every message of the account deleted at once | Refused, `409` over REST | | The account's own filters at delivery: a rule's Delete or Reject, a Sieve `discard`, `reject` or `redirect`, a forward that keeps no copy | The copy is kept among the deleted items instead of dropped; if it cannot be kept, the message is delivered | | Retention, the recover-deleted sweep, deleting a deleted item for good | Nothing of the held account is deleted | | A mailbox restore with `mode` replace, a full restore with the domains option | Refused (the full restore as HM5014) | | Moving the account to another domain, removing its mailbox key, deleting the account or its domain | Refused, over REST, SCIM and COM alike | | The archive's retention sweep and an address's erasure | A held copy keeps its file and its index row | **What a hold does not reach:** the server's and a domain's rules, which decide before a message is split between its recipients; spam and viruses refused or deleted before a message is the account's, and what the quarantine discards; the delivery queue; the confidential and password-protected stores; archived copies with no index row. If `ArchiveDir` was put inside the data directory, a full restore with the messages option replaces held archived copies too, so keep the archive outside it. **The hold is not hidden from the custodian.** The account's own `GET /api/v1/me/recoverable` answers `held: true` and lists what the hold kept, and the webmail's *Recover deleted items* page offers no delete for good. **It fails closed.** When the holds cannot be read, every question is answered held, and the error log says so once a minute (HM7332). A hold is a row in the database. The server reads the holds again as soon as one is placed or released, and no more than five seconds after a change made elsewhere. ### Exports `POST .../searches/{search}/export` queues an export of a finished search's hits. The zip holds: - **Each hit as EML**, the message's own bytes as the server stores them, under `//-.eml`, and an mbox per custodian folder beside them, in parts of at most 256 MB. - **`SHA256SUMS`**, so that `sha256sum -c SHA256SUMS` checks the unpacked export. - **`manifest.json`**: the case, the search as it was run, who ran it, who asked for the export, every item with its path, SHA-256 and size, and every item withheld, with the reason. The zip's own SHA-256 is kept with the export, and the case answers it (`exports[].sha256`). **Withheld:** a zero-access mailbox's messages, and anything deleted, moved or swept since the search ran. Search again to find where a message is now. **Limits.** The zip written is the 32-bit format, so no export passes 4 GB: narrow the search and export each part. A download past 500 MB answers `413` with where the file is on the server's disk and its SHA-256. An export that would leave the data directory's volume with less than a gigabyte free fails. A discovery export is never a PST; an examiner who needs one converts the EML files with a third-party converter. ### Who may | Permission | Allows | Held by | |---|---|---| | `view_discovery` | Reading the cases, their searches and hits, holds and exports | The server administrator and the `auditor` role | | `manage_discovery` | Opening and closing a case, running a search, placing and releasing a hold, making an export | The server administrator; a custom role may be given it | Downloading an export needs `view_mailbox` as well, since it hands over the mail itself, the custodians' deleted items among it. A search that reads the archive needs `view_archive` as well. A credential confined to named domains names no custodian outside them, never `*`, and is answered `404` for a case that reaches outside them. ### A person's data: the export One .zip of everything the server holds for an account. The person downloads it from the webmail's *Your data* card (`GET /api/v1/me/export.zip`), with no administrator involved; that download is refused (403) to a request authenticated with an app password. The administrator has the Deck's "Everything (.zip)" on the account row, `GET /api/v1/accounts/{address}/export.zip` (the administrator password's alone), `POST /api/v1/accounts/{address}/export` with `archive`, `hmctl privacy export
--archive ` and, on Windows, COM `Account.ExportArchive(file)`. | Entry | What it is | |---|---| | `manifest.json` | The address, the moment, and the counts of what follows | | `mail/.mbox` | Every folder with mail, one mbox each | | `account.xml` | What the backup carries for the account, less the credentials | | `contacts.vcf` | The address book as vCard 3.0 | | `calendars//` | Every calendar object as the client sent it | | `preferences.json`, `filters.sieve` | The webmail's preferences, and the active Sieve script | | `logs.txt` | The lines of the server's own log files that name the address, up to 50 MB | A download past 500 MB is refused with 413, and the operator writes the archive to disk instead; the disk form has no ceiling but the format's 4 GB. ### A person's data: the erasure `POST /api/v1/addresses/erase`, `hmctl privacy erase`, the Deck's "Erase traces" and, on Windows, the Control Panel's "Erase traces..." remove what deleting the account deliberately leaves: quarantine entries, message-trace rows, greylisting triplets, distribution-list memberships, aliases on either side, the bounce and complaint records, and the rows naming the address in other people's address books and first-contact memory. With the archive included, the person's archive tree is swept and every index row naming the address goes. The log lines are asked for by name (`include_logs` on the REST body, `--logs`): every occurrence becomes `[erased]`, and a log past 256 MB is reported instead of rewritten. **Since 6.3.4 it also reaches the stores beside the mailbox.** A message held for review that the address sent or would receive is discarded. The address is taken out of the recipients of a message that expires, and one it sent is ended. A password-protected message to it is ended. The recipients an account remembers writing to, the out-of-office memory, booking policies and correspondents' S/MIME certificates and OpenPGP keys lose it. **Every list of addresses is matched whole.** Erasing `ann@example.com` leaves `joann@example.com` and `ann@example.com.au` as they are. **A legal hold outranks it.** An erasure never deletes a message from a mailbox, so a hold meets it only in the stores it deletes from, and there the hold wins. The REST answer's `held` counts what a hold kept; run the erasure again once the case is closed. When the holds themselves cannot be read, the erasure fails (HM6250; `500` over REST) and says nothing about a legal hold. **What it leaves, by design,** among others: mail in mailboxes, the person's own and everybody else's, and a mailbox's deleted items; a list's archive of its posts; other people's calendars; configuration somebody made that names the address (a forward, a rule, a list's moderator); the audit trail, and the alert events, which nothing prunes yet; the spam trap record; the delivery queue and the webhook queue; the contacts in DMARC and TLS reports; the archive's Inbound folder; and backups, which are outside the running server's reach. [PersonalData.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/PersonalData.md) has the full list, with each reason. ### The audit trail, for a compliance reader The audit trail is on by default. It keeps one row in `hm_audit` for each administrative change made over COM, REST or the Deck, a mailbox's move included, with a few exceptions such as an API key made or revoked or a quarantined message released or deleted; it records changes, never reads. A password, key, token or other secret is recorded as `(changed)`, never as a value. Each row carries the SHA-256 of the row before it and its own, so `GET /api/v1/audit/verify` tells an edited row from a removed one: tampering is detectable, not impossible, since anyone who can write the table can recompute the whole chain. `audit_retention_days` is 0 by default, which keeps everything. Three things on it matter here. Every discovery search is recorded with its whole query; a download of a discovery export is recorded on the trail alone, so with the trail off a download is recorded nowhere; and an erasure is recorded with the address it erased wherever it deletes from recorded tables (aliases, list recipients, the archive index): those rows name the person until `audit_retention_days` ages them out. [Alerts, the audit trail, syslog and the journal](/topic/28273) has the columns, reading and putting a change back, who may read it, and what it does not record. ### Regulatory scope, in one paragraph The project's own determination, reviewed 12 August 2026, is that the EU Cyber Resilience Act and the revised Product Liability Directive do not place a manufacturer's obligations on it, because it is supplied outside a commercial activity. The document calls itself a self-assessment and not legal advice, and lists the triggers that would re-open it. Under GDPR and UK GDPR the project is neither controller nor processor: the operator of an installation is. The export, the erasure and the log retention limits exist so that operators can meet their own obligations. ### What it does not do - No PST export, and no text read from attachments but `text/*`. - No silent hold: the custodian can see what a hold kept. - No export past 4 GB and no download past 500 MB. - No saved queries, review tags, redaction or de-duplication across custodians: a message sent to two custodians is two hits. - No search of backups, and no case carried by a backup. ### Where to read more - Wiki: [Compliance and eDiscovery](https://gitlab.com/hmailserver/hmailserver/-/wikis/Compliance-and-eDiscovery). - Documents: [Discovery.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Discovery.md), [PersonalData.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/PersonalData.md), [AlertsAndAuditTrail.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/AlertsAndAuditTrail.md) and [RegulatoryScope.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/RegulatoryScope.md). - On this forum: [The Control Deck](/topic/28192), [Journaling and dual delivery](/topic/28271), and what a backup carries and what it leaves in [Backup and restore](/topic/28221). --- Canonical: https://www.hmailserver.co.uk/topic/28243/how-email-works-and-what-hmailserver-is-and-is-not # How email works, and what hMailServer is and is not *Written for 6.3.4. hMailServer 6.3.6 (6 October 2026) carries Linux packages for x86-64 and AArch64 and one for each of ten distribution releases, both AppImages, an ARM64 Windows installer (a preview) and the first container image. See [the 6.3.6 announcement](/topic/28276). hMailServer 6.3.7 (17 October 2026) is a security release: mail from outside whose From names a domain hosted here is no longer signed with that domain's DKIM key, an authenticated submission is signed as another hosted domain only when its account may send as the address its From names, and a forward of outside mail is given an envelope sender in the forwarding domain's organisation only when the message passed DMARC for it on arrival. The current release is 6.3.7. See [the 6.3.7 announcement](/topic/28277).* This tutorial is for someone who has not run a mail server before. It follows a message from one person to another, shows where hMailServer sits on that journey, and says what the software does and does not do. The one thing to know first: **port 25 has no password, and it has to be that way.** Any server on the internet may connect to your port 25 and offer you mail for your own domains, or nobody could write to you. Your own users send through a different door, port 587, which requires a login. Most of what follows is about keeping those two doors apart. ### What hMailServer is hMailServer is a mail server: the software that owns an email domain. When someone sends a message to `you@yourcompany.com`, their mail server looks up where `yourcompany.com` receives mail and delivers it to whatever is listening there. If you run hMailServer, that is your machine. Progressive Robot Ltd forked hMailServer in 2026 and maintains the hMailServer 6 line; the current release is 6.3.7, of 17 October 2026. | Job | Protocol | What it means | |---|---|---| | Accept mail from the world | **SMTP** | Other mail servers connect and hand you messages for your users | | Let your users read mail | **IMAP** and **POP3** | Outlook, Thunderbird, Apple Mail and phones connect to fetch messages | | Let your users send mail | **SMTP submission** | Your users hand you messages and you deliver them onward | ### Where it sits, and what is not yours Running your own mail server means owning the middle of three boxes. DNS is your registrar's, and the mail clients are your users'. The wiki puts it plainly: almost every problem people describe as "hMailServer is broken" turns out to be in one of the two boxes it does not own. | You provide | hMailServer provides | |---|---| | A machine reachable from the internet | The SMTP, IMAP and POP3 servers, and the whole delivery pipeline | | A host name with an A record, and an MX record pointing at it | The MTA-STS policy derived from that MX, and a live check of every record | | A **PTR** record, which only your ISP can set | Nothing can substitute for it | | SPF, DKIM and DMARC records in your zone | The DKIM key generation with a copyable record, the signing, and the inbound evaluation of everyone else's | | Port 25 reachable inbound, and outbound (many consumer ISPs block outbound 25) | Nothing can work around a blocked port; a smart host is the usual answer | | A certificate, or nothing, and let ACME get one | Built-in Let's Encrypt issuance and hourly renewal | | Somebody who will read a log occasionally | The logs, the diagnostics, the message trace and the metrics to read | ### A message's journey You send mail to `alice@example.com`. This is what happens: 1. Your mail client submits the message to your hMailServer on port 587. You log in first. 2. hMailServer answers `250 Queued`. 3. It asks DNS who receives mail for `example.com`. The MX record answers: `mail.example.com`. 4. It looks up the A and AAAA records for that host, plus TLSA and the MTA-STS policy. 5. It delivers the message by SMTP on port 25: no login, TLS if offered. 6. The recipient's server answers `250 OK` and stores the message in Alice's mailbox. 7. Alice's mail client reads it over IMAP. **An open relay is the thing to avoid.** Never let an unauthenticated stranger send mail through you to the outside world. The wiki's warning is that it will get your server blacklisted within hours. hMailServer refuses this by default. **A note on ports.** A fresh install seeds ports 25, 587, 110 and 143. The TLS ports 465, 993 and 995 exist only after you create them on the TCP/IP ports page. ### A submission, and the checks in the order the server runs them The order matters, because it decides when a sender is told no: before the message body is transferred, or after. | Stage | What is checked | Reply when it fails | |---|---|---| | Any command | On a `STARTTLS required` port, everything but `NOOP`, `EHLO`, `STARTTLS` and `QUIT` | `530 Must issue STARTTLS first.` | | `MAIL FROM` | Free disk space: `MinimumFreeDiskSpaceMB`, default 100 | `452 4.3.1 Insufficient system storage` | | `MAIL FROM` | Send-As, only when `SmtpAuthenticatedSenderCheck=1` (default 0) | `550 5.7.1 Sender address rejected` | | `MAIL FROM` | Per-IP rate: `MaxSubmissionsPerIPPerMinute` (default 0, unlimited) | `421 Too many messages from your IP address` | | `MAIL FROM` | Declared `SIZE` against the domain's maximum message size (default 20480 KB) | `552 5.3.4 Message size exceeds fixed maximum message size` | | `RCPT TO` | Is the address deliverable at all? | `550 Unknown user`, or `451 4.3.2 Unable to verify the recipient at the moment` | | `RCPT TO` | Mailbox full | `452 4.2.2` | | `RCPT TO` | Relay permission for this IP range | `550 Delivery is not allowed to this address.` | | `RCPT TO` | Authentication required and not supplied | `530 SMTP authentication is required.` | | `RCPT TO` | Greylisting, when it is on (default off) | `451 Please try again later.` | | end of data | Actual size, bare LF check, post-transmission spam tests | `552 5.3.4`, `554 5.6.0 Rejected - Message containing bare LF's`, `554 5.7.1` | **4xx and 5xx.** A 4xx reply always means "come back later", while a 5xx means "never". The difference decides whether a legitimate sender's message waits in their queue or is destroyed, which is why a full mailbox is `452` and not `550`. **Refusing early.** Every check at `MAIL FROM` and `RCPT TO` happens before a single byte of the message crosses the wire. Refusing early costs the sender nothing, and it generates no bounce to an address a spammer only claimed to be. **The timing in the last reply is real.** A submission ends with a line such as `250 2.0.0 Queued (0.412 seconds)`: the server times the accept-and-save stage and reports it. A number that climbs into whole seconds is the first sign of a slow spam test, a stalled virus scanner or a saturated work queue. **Since 6.3.4** the same session has more places to say no, each of them off until an administrator sets it: country blocking, a domain white and black list, restricted delivery, mail flow rules and others. One thing differs without a switch, and only on a new installation, not an upgrade: the *Internet* IP range requires TLS for authentication, so from another machine `AUTH` is not offered before `STARTTLS`. ### Who may send through you This is the decision that separates a mail server from an open relay, and hMailServer makes it per IP range. Mail from other domains to yours is taken on port 25 without a password, or nobody could write to you. From the *Internet* range, mail that gives one of your own domains as its sender needs a login, whoever it is for, and relaying from one remote address to another is allowed on both shipped ranges only with authentication: a stranger who has not logged in is answered `530 SMTP authentication is required.` A range that allows relaying without authentication must never exist (the Control Panel's status page warns of one as an open relay). Ranges are matched by priority, highest first, so a narrow range for your own network can sit in front of the broad *Internet* range; [IP ranges, auto-ban, lockout and the rest](/topic/28239) has the whole grid and the shipped values. ### After "250 Queued": the delivery pipeline Acceptance and delivery are separate. The `250` means the message is on disk and in the database; delivery happens on the delivery threads afterwards, in this order: 1. The `OnDeliveryStart` script event. 2. Virus scanning. 3. Global rules. 4. The `OnDeliverMessage` script event. 5. DKIM signing, then mirroring, each on the first attempt only. 6. Local delivery: account rules, Sieve, forwarding, distribution lists, public folders. 7. External delivery for any recipient outside this server: routes or an MX lookup, MTA-STS, DANE, TLS, then SMTP. A temporary failure is rescheduled: 4 tries, 60 minutes apart by default. A permanent failure, or retries exhausted, produces a delivery status notification to the sender. ### Where the filters sit Anti-spam runs in two phases, before and after the message is transmitted, and a test's phase is fixed by the test. Two thresholds decide what a score means: **mark** (default 5) adds spam headers, and **delete** (default 20) refuses the message, or quarantines it when the quarantine is on. On a fresh install almost every test is off. `zen.spamhaus.org`, `bl.spamcop.net` and `multi.surbl.org` are seeded but inactive; the HELO, PTR, MX and SPF tests are off; SpamAssassin is off; greylisting is off. DMARC is on (`ASDMARCEnabled` 1), and it evaluates SPF and DKIM itself for its own purposes. Anti-spam does not run at all for an authenticated session, for a whitelisted sender, or for an IP range with spam protection switched off. ### Reading mail back: IMAP or POP3 | | IMAP | POP3 | |---|---|---| | Mail lives | on the server | on one device | | Folders | yes, and shared/public folders too | INBOX only | | Read/unread, flags | shared across devices | per device | | New-mail notification | `IDLE` pushes it | polling only | | Server-side search | yes | no | | Concurrent sessions | many | one at a time; a second gets `-ERR [IN-USE]` | | Default ports here | 143, and 993 once you create it | 110, and 995 once you create it | The wiki's advice is short: use IMAP. ### The three records that make mail arrive - **SPF**: a list of the servers allowed to send mail for your domain. - **DKIM**: a cryptographic signature proving the message really came from you and was not altered. - **DMARC**: a policy saying what to do when SPF and DKIM disagree, and where to send reports. Without them your mail will go to spam. The wiki calls this the most common reason a new mail server "doesn't work". Note the asymmetry: hMailServer signs your outgoing mail with DKIM as soon as you configure a key, but checking SPF on incoming mail is off by default. [Being trusted by the servers you send to](/topic/28188) has the detail. ### What hMailServer is not - **It was not a webmail client; since 6.3.2 it is one.** The page at `/portal` on the REST listener became a mail client in 6.3.2, and a separate webmail such as Roundcube is no longer needed, though it still works over IMAP. Of 6.3.4's paging of a folder of a hundred thousand messages, the wiki says: built, and not yet measured. See [The webmail portal at /portal](/topic/28191). - **It is not a spam filter you can point at someone else's mail.** It filters mail it receives, for domains it hosts. - **It is not Exchange.** There is no ActiveSync, no MAPI and no Teams integration, so Outlook connects over IMAP and SMTP like any other client. Since 6.3.4 it carries the pieces of groupware that ride on standards, such as calendars over CalDAV. **Who should run it:** a small business that wants its own mail on its own server; a developer who needs a real SMTP/IMAP endpoint for testing; a hosting provider running mail for many domains. **Who should not:** if nobody in your organisation is comfortable editing DNS records and watching a log file occasionally, a hosted mailbox provider will serve you better. ### What you run it on | | Windows | Linux | |---|---|---| | Platform | 64-bit Windows 10 1607 / Server 2016 or later | Since 6.3.0. 6.3.4's packages are x86-64 only: a `.deb` for Ubuntu 26.04 and 24.04, an `.rpm` for Fedora 43 and 44 and Enterprise Linux 10 | | Administration | The Control Panel, the COM API, the Control Deck and the REST API | The Control Deck and the REST API. The Control Panel and the COM API are Windows-only | | Event scripts | VBScript or JScript | JavaScript, since 6.3.4 | The AArch64 packages and the AppImage were last published with 6.3.3. 6.3.4 is installed by hand; no live update installs it. [hMailServer on Linux: what you get and what you do not](/topic/28189) and [First install on Linux](/topic/28190) go further. ### "Additive and off by default" The design rule is that an existing installation upgrades without behaving differently until you turn something on. | Category | State on a fresh install | |---|---| | Optional listeners: the REST API and `/portal`, web services, metrics, ManageSieve | Off. Each stays inert until its port setting is non-zero | | Optional features: quarantine, message trace, full-text index, archive | Off | | Outbound transport security: MTA-STS, DANE, DNSSEC validation | On (`MtaStsEnabled`, `DaneEnforcementEnabled`, `DnssecValidationEnabled`). Each can be switched off | | A new installation of 6.3.4, not an upgrade | A self-signed certificate, STARTTLS offered on 25, 587, 110 and 143, TLS required for authentication from the Internet range: see [Hardening a public mail server](/topic/28241) | Since 6.3.4 the database is the settings store: a `[Settings]` line edited in `hMailServer.ini` is ignored in favour of the stored value, and put back where the service can write the file. ### Words you will meet | Term | Meaning | |---|---| | **MTA** | Mail Transfer Agent: a mail server that accepts and forwards messages. hMailServer is one | | **MSA** | Message Submission Agent: the server your own users hand mail to, normally on port 587, normally with authentication | | **MUA** | Mail User Agent: the mail client | | **Relay** | Accepting a message for a domain you do not host and passing it on | | **Smart host** | Another server you send all your outbound mail through, often your ISP's | | **STARTTLS** | Upgrading a plaintext connection to an encrypted one, in-band | | **Implicit TLS** | TLS from the first byte, no upgrade step: ports 465, 993, 995 | | **Envelope vs header** | The `MAIL FROM`/`RCPT TO` addresses the servers use, versus the `From:`/`To:` a person reads. They need not match | | **Deferral** | Holding a message for a later attempt after a temporary failure | | **Backscatter** | Bounces sent to a forged sender address | | **MX record** | The DNS record saying which server receives a domain's mail | | **PTR record** | Reverse DNS: maps an IP back to a host name | ### Where to read more - Wiki: [How Email Works](https://gitlab.com/hmailserver/hmailserver/-/wikis/How-Email-Works), [What hMailServer Is and Is Not](https://gitlab.com/hmailserver/hmailserver/-/wikis/What-hMailServer-Is-and-Is-Not) and the [Glossary](https://gitlab.com/hmailserver/hmailserver/-/wikis/Glossary). - The public documentation: [chapter 1](https://www.progressiverobot.com/hmailserver-documentation/#1-what-hmailserver-is-and-what-it-is-not), [chapter 2](https://www.progressiverobot.com/hmailserver-documentation/#2-how-email-actually-works) and the [glossary](https://www.progressiverobot.com/hmailserver-documentation/#37-glossary). - On this forum: [Read this before you post](/topic/28185) and [Where to post what](/topic/28186); for the next steps, [Your first domain and mailbox, and the DNS records your domain needs](/topic/28229), [Connecting a mail client](/topic/28255), [The anti-spam stack end to end](/topic/28248) and [Every port hMailServer listens on](/topic/28226). --- Canonical: https://www.hmailserver.co.uk/topic/28243/how-email-works-and-what-hmailserver-is-and-is-not # How email works, and what hMailServer is and is not *Written for 6.3.4. hMailServer 6.3.6 (6 October 2026) carries Linux packages for x86-64 and AArch64 and one for each of ten distribution releases, both AppImages, an ARM64 Windows installer (a preview) and the first container image. See [the 6.3.6 announcement](/topic/28276). hMailServer 6.3.7 (17 October 2026) is a security release: mail from outside whose From names a domain hosted here is no longer signed with that domain's DKIM key, an authenticated submission is signed as another hosted domain only when its account may send as the address its From names, and a forward of outside mail is given an envelope sender in the forwarding domain's organisation only when the message passed DMARC for it on arrival. The current release is 6.3.7. See [the 6.3.7 announcement](/topic/28277).* This tutorial is for someone who has not run a mail server before. It follows a message from one person to another, shows where hMailServer sits on that journey, and says what the software does and does not do. The one thing to know first: **port 25 has no password, and it has to be that way.** Any server on the internet may connect to your port 25 and offer you mail for your own domains, or nobody could write to you. Your own users send through a different door, port 587, which requires a login. Most of what follows is about keeping those two doors apart. ### What hMailServer is hMailServer is a mail server: the software that owns an email domain. When someone sends a message to `you@yourcompany.com`, their mail server looks up where `yourcompany.com` receives mail and delivers it to whatever is listening there. If you run hMailServer, that is your machine. Progressive Robot Ltd forked hMailServer in 2026 and maintains the hMailServer 6 line; the current release is 6.3.7, of 17 October 2026. | Job | Protocol | What it means | |---|---|---| | Accept mail from the world | **SMTP** | Other mail servers connect and hand you messages for your users | | Let your users read mail | **IMAP** and **POP3** | Outlook, Thunderbird, Apple Mail and phones connect to fetch messages | | Let your users send mail | **SMTP submission** | Your users hand you messages and you deliver them onward | ### Where it sits, and what is not yours Running your own mail server means owning the middle of three boxes. DNS is your registrar's, and the mail clients are your users'. The wiki puts it plainly: almost every problem people describe as "hMailServer is broken" turns out to be in one of the two boxes it does not own. | You provide | hMailServer provides | |---|---| | A machine reachable from the internet | The SMTP, IMAP and POP3 servers, and the whole delivery pipeline | | A host name with an A record, and an MX record pointing at it | The MTA-STS policy derived from that MX, and a live check of every record | | A **PTR** record, which only your ISP can set | Nothing can substitute for it | | SPF, DKIM and DMARC records in your zone | The DKIM key generation with a copyable record, the signing, and the inbound evaluation of everyone else's | | Port 25 reachable inbound, and outbound (many consumer ISPs block outbound 25) | Nothing can work around a blocked port; a smart host is the usual answer | | A certificate, or nothing, and let ACME get one | Built-in Let's Encrypt issuance and hourly renewal | | Somebody who will read a log occasionally | The logs, the diagnostics, the message trace and the metrics to read | ### A message's journey You send mail to `alice@example.com`. This is what happens: 1. Your mail client submits the message to your hMailServer on port 587. You log in first. 2. hMailServer answers `250 Queued`. 3. It asks DNS who receives mail for `example.com`. The MX record answers: `mail.example.com`. 4. It looks up the A and AAAA records for that host, plus TLSA and the MTA-STS policy. 5. It delivers the message by SMTP on port 25: no login, TLS if offered. 6. The recipient's server answers `250 OK` and stores the message in Alice's mailbox. 7. Alice's mail client reads it over IMAP. **An open relay is the thing to avoid.** Never let an unauthenticated stranger send mail through you to the outside world. The wiki's warning is that it will get your server blacklisted within hours. hMailServer refuses this by default. **A note on ports.** A fresh install seeds ports 25, 587, 110 and 143. The TLS ports 465, 993 and 995 exist only after you create them on the TCP/IP ports page. ### A submission, and the checks in the order the server runs them The order matters, because it decides when a sender is told no: before the message body is transferred, or after. | Stage | What is checked | Reply when it fails | |---|---|---| | Any command | On a `STARTTLS required` port, everything but `NOOP`, `EHLO`, `STARTTLS` and `QUIT` | `530 Must issue STARTTLS first.` | | `MAIL FROM` | Free disk space: `MinimumFreeDiskSpaceMB`, default 100 | `452 4.3.1 Insufficient system storage` | | `MAIL FROM` | Send-As, only when `SmtpAuthenticatedSenderCheck=1` (default 0) | `550 5.7.1 Sender address rejected` | | `MAIL FROM` | Per-IP rate: `MaxSubmissionsPerIPPerMinute` (default 0, unlimited) | `421 Too many messages from your IP address` | | `MAIL FROM` | Declared `SIZE` against the domain's maximum message size (default 20480 KB) | `552 5.3.4 Message size exceeds fixed maximum message size` | | `RCPT TO` | Is the address deliverable at all? | `550 Unknown user`, or `451 4.3.2 Unable to verify the recipient at the moment` | | `RCPT TO` | Mailbox full | `452 4.2.2` | | `RCPT TO` | Relay permission for this IP range | `550 Delivery is not allowed to this address.` | | `RCPT TO` | Authentication required and not supplied | `530 SMTP authentication is required.` | | `RCPT TO` | Greylisting, when it is on (default off) | `451 Please try again later.` | | end of data | Actual size, bare LF check, post-transmission spam tests | `552 5.3.4`, `554 5.6.0 Rejected - Message containing bare LF's`, `554 5.7.1` | **4xx and 5xx.** A 4xx reply always means "come back later", while a 5xx means "never". The difference decides whether a legitimate sender's message waits in their queue or is destroyed, which is why a full mailbox is `452` and not `550`. **Refusing early.** Every check at `MAIL FROM` and `RCPT TO` happens before a single byte of the message crosses the wire. Refusing early costs the sender nothing, and it generates no bounce to an address a spammer only claimed to be. **The timing in the last reply is real.** A submission ends with a line such as `250 2.0.0 Queued (0.412 seconds)`: the server times the accept-and-save stage and reports it. A number that climbs into whole seconds is the first sign of a slow spam test, a stalled virus scanner or a saturated work queue. **Since 6.3.4** the same session has more places to say no, each of them off until an administrator sets it: country blocking, a domain white and black list, restricted delivery, mail flow rules and others. One thing differs without a switch, and only on a new installation, not an upgrade: the *Internet* IP range requires TLS for authentication, so from another machine `AUTH` is not offered before `STARTTLS`. ### Who may send through you This is the decision that separates a mail server from an open relay, and hMailServer makes it per IP range. Mail from other domains to yours is taken on port 25 without a password, or nobody could write to you. From the *Internet* range, mail that gives one of your own domains as its sender needs a login, whoever it is for, and relaying from one remote address to another is allowed on both shipped ranges only with authentication: a stranger who has not logged in is answered `530 SMTP authentication is required.` A range that allows relaying without authentication must never exist (the Control Panel's status page warns of one as an open relay). Ranges are matched by priority, highest first, so a narrow range for your own network can sit in front of the broad *Internet* range; [IP ranges, auto-ban, lockout and the rest](/topic/28239) has the whole grid and the shipped values. ### After "250 Queued": the delivery pipeline Acceptance and delivery are separate. The `250` means the message is on disk and in the database; delivery happens on the delivery threads afterwards, in this order: 1. The `OnDeliveryStart` script event. 2. Virus scanning. 3. Global rules. 4. The `OnDeliverMessage` script event. 5. DKIM signing, then mirroring, each on the first attempt only. 6. Local delivery: account rules, Sieve, forwarding, distribution lists, public folders. 7. External delivery for any recipient outside this server: routes or an MX lookup, MTA-STS, DANE, TLS, then SMTP. A temporary failure is rescheduled: 4 tries, 60 minutes apart by default. A permanent failure, or retries exhausted, produces a delivery status notification to the sender. ### Where the filters sit Anti-spam runs in two phases, before and after the message is transmitted, and a test's phase is fixed by the test. Two thresholds decide what a score means: **mark** (default 5) adds spam headers, and **delete** (default 20) refuses the message, or quarantines it when the quarantine is on. On a fresh install almost every test is off. `zen.spamhaus.org`, `bl.spamcop.net` and `multi.surbl.org` are seeded but inactive; the HELO, PTR, MX and SPF tests are off; SpamAssassin is off; greylisting is off. DMARC is on (`ASDMARCEnabled` 1), and it evaluates SPF and DKIM itself for its own purposes. Anti-spam does not run at all for an authenticated session, for a whitelisted sender, or for an IP range with spam protection switched off. ### Reading mail back: IMAP or POP3 | | IMAP | POP3 | |---|---|---| | Mail lives | on the server | on one device | | Folders | yes, and shared/public folders too | INBOX only | | Read/unread, flags | shared across devices | per device | | New-mail notification | `IDLE` pushes it | polling only | | Server-side search | yes | no | | Concurrent sessions | many | one at a time; a second gets `-ERR [IN-USE]` | | Default ports here | 143, and 993 once you create it | 110, and 995 once you create it | The wiki's advice is short: use IMAP. ### The three records that make mail arrive - **SPF**: a list of the servers allowed to send mail for your domain. - **DKIM**: a cryptographic signature proving the message really came from you and was not altered. - **DMARC**: a policy saying what to do when SPF and DKIM disagree, and where to send reports. Without them your mail will go to spam. The wiki calls this the most common reason a new mail server "doesn't work". Note the asymmetry: hMailServer signs your outgoing mail with DKIM as soon as you configure a key, but checking SPF on incoming mail is off by default. [Being trusted by the servers you send to](/topic/28188) has the detail. ### What hMailServer is not - **It was not a webmail client; since 6.3.2 it is one.** The page at `/portal` on the REST listener became a mail client in 6.3.2, and a separate webmail such as Roundcube is no longer needed, though it still works over IMAP. Of 6.3.4's paging of a folder of a hundred thousand messages, the wiki says: built, and not yet measured. See [The webmail portal at /portal](/topic/28191). - **It is not a spam filter you can point at someone else's mail.** It filters mail it receives, for domains it hosts. - **It is not Exchange.** There is no ActiveSync, no MAPI and no Teams integration, so Outlook connects over IMAP and SMTP like any other client. Since 6.3.4 it carries the pieces of groupware that ride on standards, such as calendars over CalDAV. **Who should run it:** a small business that wants its own mail on its own server; a developer who needs a real SMTP/IMAP endpoint for testing; a hosting provider running mail for many domains. **Who should not:** if nobody in your organisation is comfortable editing DNS records and watching a log file occasionally, a hosted mailbox provider will serve you better. ### What you run it on | | Windows | Linux | |---|---|---| | Platform | 64-bit Windows 10 1607 / Server 2016 or later | Since 6.3.0. 6.3.4's packages are x86-64 only: a `.deb` for Ubuntu 26.04 and 24.04, an `.rpm` for Fedora 43 and 44 and Enterprise Linux 10 | | Administration | The Control Panel, the COM API, the Control Deck and the REST API | The Control Deck and the REST API. The Control Panel and the COM API are Windows-only | | Event scripts | VBScript or JScript | JavaScript, since 6.3.4 | The AArch64 packages and the AppImage were last published with 6.3.3. 6.3.4 is installed by hand; no live update installs it. [hMailServer on Linux: what you get and what you do not](/topic/28189) and [First install on Linux](/topic/28190) go further. ### "Additive and off by default" The design rule is that an existing installation upgrades without behaving differently until you turn something on. | Category | State on a fresh install | |---|---| | Optional listeners: the REST API and `/portal`, web services, metrics, ManageSieve | Off. Each stays inert until its port setting is non-zero | | Optional features: quarantine, message trace, full-text index, archive | Off | | Outbound transport security: MTA-STS, DANE, DNSSEC validation | On (`MtaStsEnabled`, `DaneEnforcementEnabled`, `DnssecValidationEnabled`). Each can be switched off | | A new installation of 6.3.4, not an upgrade | A self-signed certificate, STARTTLS offered on 25, 587, 110 and 143, TLS required for authentication from the Internet range: see [Hardening a public mail server](/topic/28241) | Since 6.3.4 the database is the settings store: a `[Settings]` line edited in `hMailServer.ini` is ignored in favour of the stored value, and put back where the service can write the file. ### Words you will meet | Term | Meaning | |---|---| | **MTA** | Mail Transfer Agent: a mail server that accepts and forwards messages. hMailServer is one | | **MSA** | Message Submission Agent: the server your own users hand mail to, normally on port 587, normally with authentication | | **MUA** | Mail User Agent: the mail client | | **Relay** | Accepting a message for a domain you do not host and passing it on | | **Smart host** | Another server you send all your outbound mail through, often your ISP's | | **STARTTLS** | Upgrading a plaintext connection to an encrypted one, in-band | | **Implicit TLS** | TLS from the first byte, no upgrade step: ports 465, 993, 995 | | **Envelope vs header** | The `MAIL FROM`/`RCPT TO` addresses the servers use, versus the `From:`/`To:` a person reads. They need not match | | **Deferral** | Holding a message for a later attempt after a temporary failure | | **Backscatter** | Bounces sent to a forged sender address | | **MX record** | The DNS record saying which server receives a domain's mail | | **PTR record** | Reverse DNS: maps an IP back to a host name | ### Where to read more - Wiki: [How Email Works](https://gitlab.com/hmailserver/hmailserver/-/wikis/How-Email-Works), [What hMailServer Is and Is Not](https://gitlab.com/hmailserver/hmailserver/-/wikis/What-hMailServer-Is-and-Is-Not) and the [Glossary](https://gitlab.com/hmailserver/hmailserver/-/wikis/Glossary). - The public documentation: [chapter 1](https://www.progressiverobot.com/hmailserver-documentation/#1-what-hmailserver-is-and-what-it-is-not), [chapter 2](https://www.progressiverobot.com/hmailserver-documentation/#2-how-email-actually-works) and the [glossary](https://www.progressiverobot.com/hmailserver-documentation/#37-glossary). - On this forum: [Read this before you post](/topic/28185) and [Where to post what](/topic/28186); for the next steps, [Your first domain and mailbox, and the DNS records your domain needs](/topic/28229), [Connecting a mail client](/topic/28255), [The anti-spam stack end to end](/topic/28248) and [Every port hMailServer listens on](/topic/28226). --- Canonical: https://www.hmailserver.co.uk/topic/28244/relocating-an-installation-to-another-drive-and-moving-or-renaming-a-mailbox # Relocating an installation to another drive, and moving or renaming a mailbox *Written for 6.3.4. hMailServer 6.3.7 (17 October 2026) bears on relocating a Windows installation to another drive. `hMailServer.exe /ProtectFolders`, which setup runs at every installation and upgrade, now hardens the program folder, `Bin` and `ControlPanel`: where hMailServer was installed in a folder that took its permissions from the root of a drive, any account that could sign in to the computer could replace the programs the service runs (CVE-2026-107571); the default folder under Program Files was never affected. The procedure below copies the tree and re-registers it without running setup, and a folder below a drive's root takes the permissions the root hands down, so an installation relocated there is hardened by the upgrade to 6.3.7, or on 6.3.7 by running `hMailServer.exe /ProtectFolders` as an administrator, not by the move (on 6.3.6 or earlier follow the icacls steps in `ServiceAccount.md`, since those versions' /ProtectFolders leaves the three program folders as they are); before upgrading an installation at a drive's root, follow the steps "Before upgrading an installation at a drive's root" in `ServiceAccount.md`. Where accounts that are not administrators could sign in to the computer while the installation was in such a folder, the release says to reinstall the programs into a new folder that only the administrators may change, not over the installation, with setup given the new folder by `/DIR=`: over an installation setup installs into the folder it finds, and a 32-bit installation's uninstall entry overrides `/DIR=` until it is taken away, which the "re-run the installer" repair below meets. See [the 6.3.7 announcement](/topic/28277).* This guide covers two different moves. The first is relocating a configured installation to another directory or drive on the same machine, for example from `C:\Program Files (x86)\hMailServer` to `C:\Program Files\hMailServer`. The second is renaming a mailbox or moving it to another domain, which is an operation of its own since 6.3.4. Moving to new hardware is a third thing, and [Moving to new hardware, and why Windows to Linux is a rebuild](/topic/28194) covers it. The one thing to know first: **relocating is not one change.** The installation path is recorded in the registry, in the service, in the COM registration, in `hMailServer.ini` and in a handful of database columns, and since 6.3.4 a `[Settings]` path edited in the file is ignored in favour of the value stored in the database. The relocation runbook is written in Windows terms: the registry, the service and the COM registration. The documents used here give no separate procedure for a Linux installation. Installing somewhere new is a different, solved problem: `/DIR=""` on the installer puts a fresh installation wherever you want. ### The good news first The mail store itself is already relocatable. Message rows hold a bare `{guid}.eml` with no directory, and the path is rebuilt at every read against the data directory. Moving the data directory and pointing `DataFolder` at its new home moves every message with it. The exception is a database that predates hMailServer 5.4, where the column can hold a full path. ### Before you touch anything 1. **Take a backup and verify it restores** ([Backup and restore](/topic/28221)). Everything below is reversible only while the old tree and a good backup both exist. 2. **Ask the server whether its message rows are relative.** In the Control Panel, *Monitoring & troubleshooting → Diagnostics → Test message file locations*. If it reports `ERROR: Full paths are stored in the database`, the move is still possible, but converting the paths becomes mandatory rather than optional. 3. **Write down what you are about to strand.** The same Diagnostics page has an **Installation paths** section that lists the registry root, every `[Directories]` key and every path-shaped setting the running server holds, with the ones that do not exist marked. It changes nothing. ### The inventory | Where | What | Repaired by | |---|---|---| | `Bin\hMailServer.ini`, `[Directories]` | `ProgramFolder`, `DataFolder`, `DatabaseFolder`, `LogFolder`, `TempFolder`, `EventFolder` | Editing the file | | `Bin\hMailServer.ini`, `[Database]` | `PostgreSQLSslRootCert` | Editing the file. Leave the connection keys alone: they name a server, not a path | | The settings store (`hm_inisettings`) | `ArchiveDir`, `AcmeCertificateDirectory`, `OAuth2PublicKeyFile`, `UpdateTrustRootsFile`, `UpdateLogPublicKeyFile`, and the certificate and private key files of the REST API, metrics and web services listeners | Storing the new value, not editing the file (since 6.3.4) | | Registry, `HKLM\SOFTWARE\hMailServer` in the 32-bit view | `InstallLocation` | By hand | | Registry: the service's `ImagePath`, `LocalServer32` (79 keys), `InprocServer32` (15), the TypeLib values | | `hMailServer.exe /Register` | | Registry, `HKLM\...\Uninstall\hMailServer_is1` | `InstallLocation`, `UninstallString`, `DisplayIcon` | Nothing; re-run the installer | | The database | Certificate and key files, DKIM key files, the client-certificate CA bundle, `backupdestination`, the ClamWin and custom scanner paths, rule actions | Each on its own page | **What breaks when a key is left behind.** With a stale `DataFolder` the store appears empty. With a stale `LogFolder` logs are written to the old tree, silently. With a stale `EventFolder` event scripts are not found and events stop running. **The 32-bit view is not optional.** The server reads `InstallLocation` through `KEY_WOW64_32KEY`, and the installer writes it there. Writing the 64-bit view changes nothing and looks like it worked. ### The procedure 1. **Stop the service**, and confirm it is stopped. Nothing here may run against a live server. 2. **Copy the tree. Copy, do not move**: the old tree is the rollback. Include `Bin\hMailServerApiKeys.ini` if it exists, since the REST API key store lives there and no installer manifest mentions it. Grant the service account access to the new tree. A server that cannot write its log directory starts and then says nothing. 3. **Rewrite the file keys** in `\Bin\hMailServer.ini` whose value was under the old root: the `[Directories]` folders and `[Database]` `PostgreSQLSslRootCert`. Nothing else, and not a folder that was outside the old tree and has not moved: a `DataFolder` pointed at an empty directory makes the store appear empty. 4. **Repoint the registry root**, from an elevated prompt: `reg add "HKLM\SOFTWARE\hMailServer" /v InstallLocation /t REG_SZ /d "" /reg:32 /f`. `/reg:32` is the whole of the point. 5. **Re-register the server**, from an elevated prompt, from the **new** `Bin`: `hMailServer.exe /Register`. It registers the path of the executable that runs it. 6. **Start it, and check what it read.** Read the application log, and since 6.3.4 the ERROR log too. Confirm that *Settings → Directories* shows the new paths, that your domains are listed, and that the log file you are reading is under the new log directory. 7. **Convert full message paths**, only if the diagnostic found any: run the **Data Directory Synchronizer** against the new data directory. A hand-written `UPDATE hm_messages SET ...` is not a supported converter. 8. **Store the `[Settings]` paths and fix the database-borne paths** (both below). 9. **Verify before deleting anything**: send and receive a message; complete a TLS handshake on every port that has a certificate; run a backup to completion; run an event script, if you use them; check the log directory is being written. 10. **Keep the old tree**, unreachable by the service, for at least one full backup cycle. Do not make it unreachable until the `[Settings]` paths are stored and a reinitialise has picked up the new certificate and ACME paths. **The failure this prevents.** A server whose anchors are stale reports `Running` and serves an empty configuration. That failure is silent. ### The `[Settings]` paths: 6.3.3 and earlier, and since 6.3.4 On 6.3.3 and earlier these are `[Settings]` keys of `hMailServer.ini` like any other, and editing the file moves them. **Since 6.3.4 it does not**: the stored value is used, the edit is named in the error log as `HM5804`, and, where the service can write the file, the line is put back to the old path. [RelocatingAnInstallation.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/RelocatingAnInstallation.md) and the wiki page between them give three ways to store the new values. The first is the document's own, and the one to use: | When | How | Where it is described | |---|---|---| | After the moved server is running | In the Control Panel under **Settings**, with `Settings.SetIniSetting` over COM, `PUT /api/v1/settings/ini/{name}`, or `hmctl ini set ` | [RelocatingAnInstallation.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/RelocatingAnInstallation.md), step 9, and the wiki | | Before stopping the old server | The same four ways, as the last thing before the service is stopped. A save applies `ArchiveDir`, `OAuth2PublicKeyFile`, `UpdateTrustRootsFile` and `UpdateLogPublicKeyFile` at once, not at the next start as the wiki page has it, so the running old server turns to a tree not yet copied: archiving stops, and token validation and update verification refuse. The database then holds the new paths, so going back to the old tree means storing the old values again | The wiki, corrected by [SettingsApply.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SettingsApply.md) | | At the first start, if the service is already stopped | A `[SettingsOverride]` section in the new `hMailServer.ini`, named in the error log as HM5807; then store the same values and remove the section before the next restart | The wiki | A save applies every one of these paths at once except the listeners' certificate paths and the ACME certificate directory, which are bound when the servers start and wait for a reinitialise (**Apply now**, or `POST /api/v1/server/reinitialize`). Until then the server runs on the old paths, which still resolve because the tree was copied rather than moved. `PasswordPepper` is the exception to all of this: it lives in the file only, is not a path, and comes across with the copied file. ### The database-borne paths - **SSL certificates.** The Control Panel's certificate grid is read-only except for the passphrase, so each certificate is deleted and re-added with its new file names, or set over COM. - **DKIM keys.** *Domain → DKIM*, the primary and, if used, the secondary. A domain whose key rotation manages its keys refuses that edit. - **Client-certificate CA bundle**, per port. A port set to require a client certificate refuses every client when its bundle cannot be loaded. - **Archive index.** `hm_archiveindex.archivepath` is matched on the exact string by retention, legal hold and the address eraser. Do not bulk-rewrite it. If the archive directory moved, move the archive back, or accept that the rows written before the move point at the old location; there is no supported converter for this column. ### What stays broken, and why - **Add/Remove Programs** still points at the old `unins000.exe`. Do not edit `unins000.dat`. The supported repair is to re-run the installer at the new root. - **Start-menu shortcuts** hold absolute targets and nothing repairs them. - **Message content.** The old path appears inside messages the server itself wrote, such as a delivery report. Never rewrite message content. ### Renaming a mailbox, or moving it to another domain New in 6.3.4 (schema 6056, whose table `hm_mailboxmoves` is the move's journal). `new_address` carries the domain, so renaming and moving are the same operation: `ann@example.com` to `anne@example.com` is a rename, `ann@example.com` to `ann@example.net` is a move. **A move keeps the account's row**, so everything keyed on the account id follows without being touched: rules, folders with their subscriptions and access-control entries, application passwords, the second factor, contacts, calendars and the role assignment. Copying the mail into a second account loses all of it. | Interface | How | |---|---| | Control Deck | The **Move or rename** card in the account editor | | `hmctl` | `hmctl mailbox status ann@example.com`, then `hmctl mailbox move ann@example.com ann@example.net` | | REST | `POST /api/v1/accounts/ann%40example.com/move` with `{"new_address": "ann@example.net", "keep_alias": true}`; `GET` on the same path says whether anything holds the mailbox | | PowerShell | `Move-HmMailbox -Address ann@example.com -NewAddress ann@example.net` | | Control Panel (Windows) | **Move or rename…** on the account's General tab | | COM (Windows) | `Account.MoveTo(NewAddress, KeepOldAddressAsAlias)` or `Utilities.MoveMailbox(Address, NewAddress, KeepOldAddressAsAlias)` | **The old address is kept as an alias by default**, pointing at the new address. Turn it off (`keep_alias` false, `--drop-alias`) only when the address is to stop working: mail to it is then refused at `RCPT TO`. **What moves:** the message store directory `//`; the Sieve tree; aliases in any domain that point at the account; distribution list memberships; recipients of mail still in the delivery queue; and quarantined mail, up to a ceiling of 200,000 rows. **What does not move, on purpose.** The archive and the message trace stay under the old address, as a record of the past. Erasure works by address, so erasing the new address will not reach the archive written under the old one: erase both ([Compliance: personal data and erasure](/topic/28242)). The domains a role reaches are not rewritten either, so check a role after a cross-domain move ([Administrator roles end to end](/topic/28237)). **What is refused, before the first byte is moved:** - a domain this server does not hold, or one that is not active; - a destination domain at its maximum number of accounts, or that would exceed its maximum total size; - an account, an alias or a distribution list already at the new address; - a mailbox encrypted at rest moving to another domain (`409`); a rename inside the domain is not refused; - an installation whose message file names are still full paths; - a credential that may not administer accounts in both domains; - a mailbox in use. **In use** means a POP3 session, or an IMAP session with one of the account's folders selected. What is not counted: an IMAP session that has selected nothing, a webmail request in flight, a ManageSieve session and an SMTP delivery to the mailbox in flight. A message that arrives during the copy is written under the old directory, and the old directory is swept once afterwards; anything that could not be carried is `HM6948`. That is why a move of a large busy mailbox belongs out of hours. ### When a move fails or is interrupted There is no transaction across a filesystem and four database backends, so the order is chosen to make each step undoable: the store, then the account row, then the references, then the alias. A journal row is opened before the first step and closed after the last, and an interrupted move is said once per start as `HM6947` in the application log, naming both addresses and the stage it stopped at. | Stage | What it means | |---|---| | `started` | Nothing has been changed | | `moving the store` | The copy was under way; look at both directories. The database has not been touched | | `store moved` | The directories are at the new address. While the account row says the old one, the account authenticates and its mail is unreadable | | `account written` | The store and the account row agree; the references have not all been rewritten | | `references rewritten` | Everything but the alias | | `complete` | The move finished | | `rolled back` | A step failed and everything before it was put back | | `rollback failed` | Something could not be put back; `HM6940` is in the application log | To finish a move interrupted at `moving the store` or `store moved`, ask for the move again, naming the address the directories now have. Read the journal with `hmctl mailbox status` or `GET /api/v1/accounts/
/move`. The journal is not in the backup. | Code | Meaning | |---|---| | `HM6941` | The move succeeded but the alias for the old address could not be created | | `HM6942` | Aliases pointing at the account could not be re-pointed | | `HM6943` | Distribution list memberships could not be moved | | `HM6944` | Messages in the delivery queue still name the old address | | `HM6945` | Quarantined messages still name the old address | | `HM6946` | The move could not be journalled. It still happened, and is on the audit trail | **Who may.** `manage_accounts`, in both domains. The server administrator and a domain administrator of both domains have it; the help-desk role and the auditor do not. `view_accounts` is enough to read the journal. A move is one entry on the audit trail: object type `account`, action `moved`. ### What it does not do - The relocation runbook does not repair the uninstall entry or the shortcuts, and it has no converter for the archive index. - A mailbox move does not wait for a delivery in flight; nothing in the server records one. - Setting `Account.Address` and saving goes through the same implementation, but leaves no alias, does not refuse while the mailbox is in use, and cannot cross domains. - A domain rename (`Domain.Name` and `Save()`) is a different thing again, and keeps the path it has always taken. ### Where to read more - Wiki: [Relocating an Installation](https://gitlab.com/hmailserver/hmailserver/-/wikis/Relocating-an-Installation). - Documents: [RelocatingAnInstallation.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/RelocatingAnInstallation.md) and [MailboxMove.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/MailboxMove.md). - The public documentation: [33.1, the installation on disk](https://www.progressiverobot.com/hmailserver-documentation/#331-the-installation-on-disk). - On this forum: [Moving to new hardware, and why Windows to Linux is a rebuild](/topic/28194), [Upgrading to 6.3.4](/topic/28193), and how the settings store works and when a save takes effect in [Settings: what is not stock, when a save takes effect, and the three levels](/topic/28222). --- Canonical: https://www.hmailserver.co.uk/topic/28244/relocating-an-installation-to-another-drive-and-moving-or-renaming-a-mailbox # Relocating an installation to another drive, and moving or renaming a mailbox *Written for 6.3.4. hMailServer 6.3.7 (17 October 2026) bears on relocating a Windows installation to another drive. `hMailServer.exe /ProtectFolders`, which setup runs at every installation and upgrade, now hardens the program folder, `Bin` and `ControlPanel`: where hMailServer was installed in a folder that took its permissions from the root of a drive, any account that could sign in to the computer could replace the programs the service runs (CVE-2026-107571); the default folder under Program Files was never affected. The procedure below copies the tree and re-registers it without running setup, and a folder below a drive's root takes the permissions the root hands down, so an installation relocated there is hardened by the upgrade to 6.3.7, or on 6.3.7 by running `hMailServer.exe /ProtectFolders` as an administrator, not by the move (on 6.3.6 or earlier follow the icacls steps in `ServiceAccount.md`, since those versions' /ProtectFolders leaves the three program folders as they are); before upgrading an installation at a drive's root, follow the steps "Before upgrading an installation at a drive's root" in `ServiceAccount.md`. Where accounts that are not administrators could sign in to the computer while the installation was in such a folder, the release says to reinstall the programs into a new folder that only the administrators may change, not over the installation, with setup given the new folder by `/DIR=`: over an installation setup installs into the folder it finds, and a 32-bit installation's uninstall entry overrides `/DIR=` until it is taken away, which the "re-run the installer" repair below meets. See [the 6.3.7 announcement](/topic/28277).* This guide covers two different moves. The first is relocating a configured installation to another directory or drive on the same machine, for example from `C:\Program Files (x86)\hMailServer` to `C:\Program Files\hMailServer`. The second is renaming a mailbox or moving it to another domain, which is an operation of its own since 6.3.4. Moving to new hardware is a third thing, and [Moving to new hardware, and why Windows to Linux is a rebuild](/topic/28194) covers it. The one thing to know first: **relocating is not one change.** The installation path is recorded in the registry, in the service, in the COM registration, in `hMailServer.ini` and in a handful of database columns, and since 6.3.4 a `[Settings]` path edited in the file is ignored in favour of the value stored in the database. The relocation runbook is written in Windows terms: the registry, the service and the COM registration. The documents used here give no separate procedure for a Linux installation. Installing somewhere new is a different, solved problem: `/DIR=""` on the installer puts a fresh installation wherever you want. ### The good news first The mail store itself is already relocatable. Message rows hold a bare `{guid}.eml` with no directory, and the path is rebuilt at every read against the data directory. Moving the data directory and pointing `DataFolder` at its new home moves every message with it. The exception is a database that predates hMailServer 5.4, where the column can hold a full path. ### Before you touch anything 1. **Take a backup and verify it restores** ([Backup and restore](/topic/28221)). Everything below is reversible only while the old tree and a good backup both exist. 2. **Ask the server whether its message rows are relative.** In the Control Panel, *Monitoring & troubleshooting → Diagnostics → Test message file locations*. If it reports `ERROR: Full paths are stored in the database`, the move is still possible, but converting the paths becomes mandatory rather than optional. 3. **Write down what you are about to strand.** The same Diagnostics page has an **Installation paths** section that lists the registry root, every `[Directories]` key and every path-shaped setting the running server holds, with the ones that do not exist marked. It changes nothing. ### The inventory | Where | What | Repaired by | |---|---|---| | `Bin\hMailServer.ini`, `[Directories]` | `ProgramFolder`, `DataFolder`, `DatabaseFolder`, `LogFolder`, `TempFolder`, `EventFolder` | Editing the file | | `Bin\hMailServer.ini`, `[Database]` | `PostgreSQLSslRootCert` | Editing the file. Leave the connection keys alone: they name a server, not a path | | The settings store (`hm_inisettings`) | `ArchiveDir`, `AcmeCertificateDirectory`, `OAuth2PublicKeyFile`, `UpdateTrustRootsFile`, `UpdateLogPublicKeyFile`, and the certificate and private key files of the REST API, metrics and web services listeners | Storing the new value, not editing the file (since 6.3.4) | | Registry, `HKLM\SOFTWARE\hMailServer` in the 32-bit view | `InstallLocation` | By hand | | Registry: the service's `ImagePath`, `LocalServer32` (79 keys), `InprocServer32` (15), the TypeLib values | | `hMailServer.exe /Register` | | Registry, `HKLM\...\Uninstall\hMailServer_is1` | `InstallLocation`, `UninstallString`, `DisplayIcon` | Nothing; re-run the installer | | The database | Certificate and key files, DKIM key files, the client-certificate CA bundle, `backupdestination`, the ClamWin and custom scanner paths, rule actions | Each on its own page | **What breaks when a key is left behind.** With a stale `DataFolder` the store appears empty. With a stale `LogFolder` logs are written to the old tree, silently. With a stale `EventFolder` event scripts are not found and events stop running. **The 32-bit view is not optional.** The server reads `InstallLocation` through `KEY_WOW64_32KEY`, and the installer writes it there. Writing the 64-bit view changes nothing and looks like it worked. ### The procedure 1. **Stop the service**, and confirm it is stopped. Nothing here may run against a live server. 2. **Copy the tree. Copy, do not move**: the old tree is the rollback. Include `Bin\hMailServerApiKeys.ini` if it exists, since the REST API key store lives there and no installer manifest mentions it. Grant the service account access to the new tree. A server that cannot write its log directory starts and then says nothing. 3. **Rewrite the file keys** in `\Bin\hMailServer.ini` whose value was under the old root: the `[Directories]` folders and `[Database]` `PostgreSQLSslRootCert`. Nothing else, and not a folder that was outside the old tree and has not moved: a `DataFolder` pointed at an empty directory makes the store appear empty. 4. **Repoint the registry root**, from an elevated prompt: `reg add "HKLM\SOFTWARE\hMailServer" /v InstallLocation /t REG_SZ /d "" /reg:32 /f`. `/reg:32` is the whole of the point. 5. **Re-register the server**, from an elevated prompt, from the **new** `Bin`: `hMailServer.exe /Register`. It registers the path of the executable that runs it. 6. **Start it, and check what it read.** Read the application log, and since 6.3.4 the ERROR log too. Confirm that *Settings → Directories* shows the new paths, that your domains are listed, and that the log file you are reading is under the new log directory. 7. **Convert full message paths**, only if the diagnostic found any: run the **Data Directory Synchronizer** against the new data directory. A hand-written `UPDATE hm_messages SET ...` is not a supported converter. 8. **Store the `[Settings]` paths and fix the database-borne paths** (both below). 9. **Verify before deleting anything**: send and receive a message; complete a TLS handshake on every port that has a certificate; run a backup to completion; run an event script, if you use them; check the log directory is being written. 10. **Keep the old tree**, unreachable by the service, for at least one full backup cycle. Do not make it unreachable until the `[Settings]` paths are stored and a reinitialise has picked up the new certificate and ACME paths. **The failure this prevents.** A server whose anchors are stale reports `Running` and serves an empty configuration. That failure is silent. ### The `[Settings]` paths: 6.3.3 and earlier, and since 6.3.4 On 6.3.3 and earlier these are `[Settings]` keys of `hMailServer.ini` like any other, and editing the file moves them. **Since 6.3.4 it does not**: the stored value is used, the edit is named in the error log as `HM5804`, and, where the service can write the file, the line is put back to the old path. [RelocatingAnInstallation.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/RelocatingAnInstallation.md) and the wiki page between them give three ways to store the new values. The first is the document's own, and the one to use: | When | How | Where it is described | |---|---|---| | After the moved server is running | In the Control Panel under **Settings**, with `Settings.SetIniSetting` over COM, `PUT /api/v1/settings/ini/{name}`, or `hmctl ini set ` | [RelocatingAnInstallation.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/RelocatingAnInstallation.md), step 9, and the wiki | | Before stopping the old server | The same four ways, as the last thing before the service is stopped. A save applies `ArchiveDir`, `OAuth2PublicKeyFile`, `UpdateTrustRootsFile` and `UpdateLogPublicKeyFile` at once, not at the next start as the wiki page has it, so the running old server turns to a tree not yet copied: archiving stops, and token validation and update verification refuse. The database then holds the new paths, so going back to the old tree means storing the old values again | The wiki, corrected by [SettingsApply.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SettingsApply.md) | | At the first start, if the service is already stopped | A `[SettingsOverride]` section in the new `hMailServer.ini`, named in the error log as HM5807; then store the same values and remove the section before the next restart | The wiki | A save applies every one of these paths at once except the listeners' certificate paths and the ACME certificate directory, which are bound when the servers start and wait for a reinitialise (**Apply now**, or `POST /api/v1/server/reinitialize`). Until then the server runs on the old paths, which still resolve because the tree was copied rather than moved. `PasswordPepper` is the exception to all of this: it lives in the file only, is not a path, and comes across with the copied file. ### The database-borne paths - **SSL certificates.** The Control Panel's certificate grid is read-only except for the passphrase, so each certificate is deleted and re-added with its new file names, or set over COM. - **DKIM keys.** *Domain → DKIM*, the primary and, if used, the secondary. A domain whose key rotation manages its keys refuses that edit. - **Client-certificate CA bundle**, per port. A port set to require a client certificate refuses every client when its bundle cannot be loaded. - **Archive index.** `hm_archiveindex.archivepath` is matched on the exact string by retention, legal hold and the address eraser. Do not bulk-rewrite it. If the archive directory moved, move the archive back, or accept that the rows written before the move point at the old location; there is no supported converter for this column. ### What stays broken, and why - **Add/Remove Programs** still points at the old `unins000.exe`. Do not edit `unins000.dat`. The supported repair is to re-run the installer at the new root. - **Start-menu shortcuts** hold absolute targets and nothing repairs them. - **Message content.** The old path appears inside messages the server itself wrote, such as a delivery report. Never rewrite message content. ### Renaming a mailbox, or moving it to another domain New in 6.3.4 (schema 6056, whose table `hm_mailboxmoves` is the move's journal). `new_address` carries the domain, so renaming and moving are the same operation: `ann@example.com` to `anne@example.com` is a rename, `ann@example.com` to `ann@example.net` is a move. **A move keeps the account's row**, so everything keyed on the account id follows without being touched: rules, folders with their subscriptions and access-control entries, application passwords, the second factor, contacts, calendars and the role assignment. Copying the mail into a second account loses all of it. | Interface | How | |---|---| | Control Deck | The **Move or rename** card in the account editor | | `hmctl` | `hmctl mailbox status ann@example.com`, then `hmctl mailbox move ann@example.com ann@example.net` | | REST | `POST /api/v1/accounts/ann%40example.com/move` with `{"new_address": "ann@example.net", "keep_alias": true}`; `GET` on the same path says whether anything holds the mailbox | | PowerShell | `Move-HmMailbox -Address ann@example.com -NewAddress ann@example.net` | | Control Panel (Windows) | **Move or rename…** on the account's General tab | | COM (Windows) | `Account.MoveTo(NewAddress, KeepOldAddressAsAlias)` or `Utilities.MoveMailbox(Address, NewAddress, KeepOldAddressAsAlias)` | **The old address is kept as an alias by default**, pointing at the new address. Turn it off (`keep_alias` false, `--drop-alias`) only when the address is to stop working: mail to it is then refused at `RCPT TO`. **What moves:** the message store directory `//`; the Sieve tree; aliases in any domain that point at the account; distribution list memberships; recipients of mail still in the delivery queue; and quarantined mail, up to a ceiling of 200,000 rows. **What does not move, on purpose.** The archive and the message trace stay under the old address, as a record of the past. Erasure works by address, so erasing the new address will not reach the archive written under the old one: erase both ([Compliance: personal data and erasure](/topic/28242)). The domains a role reaches are not rewritten either, so check a role after a cross-domain move ([Administrator roles end to end](/topic/28237)). **What is refused, before the first byte is moved:** - a domain this server does not hold, or one that is not active; - a destination domain at its maximum number of accounts, or that would exceed its maximum total size; - an account, an alias or a distribution list already at the new address; - a mailbox encrypted at rest moving to another domain (`409`); a rename inside the domain is not refused; - an installation whose message file names are still full paths; - a credential that may not administer accounts in both domains; - a mailbox in use. **In use** means a POP3 session, or an IMAP session with one of the account's folders selected. What is not counted: an IMAP session that has selected nothing, a webmail request in flight, a ManageSieve session and an SMTP delivery to the mailbox in flight. A message that arrives during the copy is written under the old directory, and the old directory is swept once afterwards; anything that could not be carried is `HM6948`. That is why a move of a large busy mailbox belongs out of hours. ### When a move fails or is interrupted There is no transaction across a filesystem and four database backends, so the order is chosen to make each step undoable: the store, then the account row, then the references, then the alias. A journal row is opened before the first step and closed after the last, and an interrupted move is said once per start as `HM6947` in the application log, naming both addresses and the stage it stopped at. | Stage | What it means | |---|---| | `started` | Nothing has been changed | | `moving the store` | The copy was under way; look at both directories. The database has not been touched | | `store moved` | The directories are at the new address. While the account row says the old one, the account authenticates and its mail is unreadable | | `account written` | The store and the account row agree; the references have not all been rewritten | | `references rewritten` | Everything but the alias | | `complete` | The move finished | | `rolled back` | A step failed and everything before it was put back | | `rollback failed` | Something could not be put back; `HM6940` is in the application log | To finish a move interrupted at `moving the store` or `store moved`, ask for the move again, naming the address the directories now have. Read the journal with `hmctl mailbox status` or `GET /api/v1/accounts/
/move`. The journal is not in the backup. | Code | Meaning | |---|---| | `HM6941` | The move succeeded but the alias for the old address could not be created | | `HM6942` | Aliases pointing at the account could not be re-pointed | | `HM6943` | Distribution list memberships could not be moved | | `HM6944` | Messages in the delivery queue still name the old address | | `HM6945` | Quarantined messages still name the old address | | `HM6946` | The move could not be journalled. It still happened, and is on the audit trail | **Who may.** `manage_accounts`, in both domains. The server administrator and a domain administrator of both domains have it; the help-desk role and the auditor do not. `view_accounts` is enough to read the journal. A move is one entry on the audit trail: object type `account`, action `moved`. ### What it does not do - The relocation runbook does not repair the uninstall entry or the shortcuts, and it has no converter for the archive index. - A mailbox move does not wait for a delivery in flight; nothing in the server records one. - Setting `Account.Address` and saving goes through the same implementation, but leaves no alias, does not refuse while the mailbox is in use, and cannot cross domains. - A domain rename (`Domain.Name` and `Save()`) is a different thing again, and keeps the path it has always taken. ### Where to read more - Wiki: [Relocating an Installation](https://gitlab.com/hmailserver/hmailserver/-/wikis/Relocating-an-Installation). - Documents: [RelocatingAnInstallation.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/RelocatingAnInstallation.md) and [MailboxMove.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/MailboxMove.md). - The public documentation: [33.1, the installation on disk](https://www.progressiverobot.com/hmailserver-documentation/#331-the-installation-on-disk). - On this forum: [Moving to new hardware, and why Windows to Linux is a rebuild](/topic/28194), [Upgrading to 6.3.4](/topic/28193), and how the settings store works and when a save takes effect in [Settings: what is not stock, when a save takes effect, and the three levels](/topic/28222). --- Canonical: https://www.hmailserver.co.uk/topic/28245/the-hmailserver-command-line-on-linux-every-verb-who-runs-it-the-files-the-unit # The hmailserver command line on Linux: every verb, who runs it, the files, the unit *Written for 6.3.4. hMailServer 6.3.6 (6 October 2026) carries .deb and .rpm packages for x86-64 and AArch64, one for each of Debian 12 and 13, Ubuntu 24.04, Fedora 43 and 44, Enterprise Linux 9 and 10, and openSUSE Leap 16.0 and Tumbleweed, both AppImages and the first container image. Its commands say what they did on their own output rather than in the error log, and the store commands run as root refuse a data directory that another account could have linked into. See [the 6.3.6 announcement](/topic/28276). From 6.3.6 the root helper applies hMailServer 6.3.7 (17 October 2026), a security release, by the live update, for the generic `.deb` and `.rpm` of either architecture and both AppImages; a server from one of 6.3.6's per-distribution packages is offered the generic package, which `apt` refuses on Debian 12 and `dnf` on Enterprise Linux 9, and is upgraded with its package manager and its release's own package, and from 6.3.4 or 6.3.5 install 6.3.7 with `apt`, `dnf` or `zypper`, not by the live update. A server whose `UpdateSigningIdentity`, `UpdateSigningIssuer` and `UpdateSourceRepository` still name the path the project began at refuses 6.3.7 until `sudo hmailserver --set-setting =` clears each of the three and the service is restarted. See [the 6.3.7 announcement](/topic/28277).* On Linux the server is one executable, `/usr/bin/hmailserver`, and what cannot be done over the REST API, from creating the schema to rewriting the message store, is done with its own options. This topic lists every verb 6.3.4's documents name and who runs it, where the package puts its files, what the systemd unit lets the service touch, and how the root helper applies a live update. What a Linux server has and lacks is in [hMailServer on Linux](/topic/28189), the first install in [First install on Linux](/topic/28190). The one thing to know first: on a package installation `/etc/hmailserver/hMailServer.ini` is `0640 root:hmailserver`, and the service cannot write it. ### Where the package puts things | Path | What it is | |---|---| | `/usr/bin/hmailserver` | the server, one file | | `/usr/bin/tlds.txt`, `/usr/bin/dh2048.pem` | read from beside the binary; the second is required by every TLS context | | `/usr/share/hmailserver/DBScripts/*.sql` | the schema creation and upgrade scripts | | `/usr/lib/systemd/system/hmailserver.service` | the unit | | `/etc/hmailserver/hMailServer.ini` | the configuration, a conffile an upgrade never overwrites; since 6.3.4 its `[Settings]` section is a copy of the database | | `/usr/bin/hmctl` | since 6.3.4, the command-line client over the REST API | | `/etc/logrotate.d/hmailserver` | the log rotation rule | | `/etc/fail2ban/filter.d/hmailserver.conf`, `/etc/fail2ban/jail.d/hmailserver.conf` | the fail2ban filter and jail, the jail disabled as shipped | | `/usr/lib/hmailserver/hmailserver-update`, `/usr/lib/hmailserver/autoban-hook` | the root update helper (since 6.3.4) and the firewall hook | | `/var/lib/hmailserver`, with `temp`, `events` and `database` under it | the mail store, one directory per domain and one per mailbox; scratch space; event scripts | | `/var/log/hmailserver` | the logs | **What the maintainer scripts do.** They create the `hmailserver` system user and group, and the last two rows above, all `0750 hmailserver:hmailserver`; install the configuration; and enable `hmailserver.service` without starting it, on purpose, because there is no database yet. On an upgrade they run `hmailserver --upgrade-database` as the service user, only if a database is configured, then `try-restart` the service if it was running. On removal they stop and disable, and leave the mail store and the logs alone; a `.deb` purge removes the configuration as well, and `/etc/hmailserver` with it when nothing else is in it, and still leaves the store. **Which packages exist.** 6.3.4 carries two x86-64 packages, `hmailserver_6.3.4_amd64.deb` and `hmailserver-6.3.4-1.x86_64.rpm`, and no container image of 6.3.4 is published. The `.deb` installs on Ubuntu 26.04 and 24.04, not on Debian 12; the `.rpm` on Fedora 43 and 44 and Enterprise Linux 10, not on Enterprise Linux 9; the rest of the platform picture is in [hMailServer on Linux](/topic/28189). The packages carry no Sigstore bundle, so no live update installs them: 6.3.4 is installed by hand ([Upgrading to 6.3.4](/topic/28193)). ### Who runs what **As root (`sudo`), because the file is root's:** `--set-admin-password`, `--set-setting`, and the four stored-secrets verbs. All but the export write `/etc/hmailserver/hMailServer.ini`, and the export a new file of its own (below); `--set-setting` keeps the file's owner and mode. **As the service user (`sudo -u hmailserver`), because the verb opens the database or writes the data directory:** the rest, from `--create-database` to the store walks. `--apply-new-install-defaults` refuses to run as root (exit 2), whose certificate key the service could not read, and `--sample-domain` run as root leaves today's log owned by root. Two things follow from the file being root's. The running server cannot write its copy of a stored value back into `[Settings]`: a value stored over the API is in effect and the file's line goes stale, which the server says once a start in the application log, not as an error. The stale line is never taken for an edit, so nothing needs changing by hand; only what reads the file itself, such as a server that cannot read its table, sees the older value ([how the settings store works](/topic/28222)). The exception is a `[Settings]` secret sealed once the stored-secrets key exists (below): its line cannot be rewritten, and `HM5803` names it at each start until `sudo hmailserver --set-setting ` writes it. And a server that verifies an older MD5 or SHA-256 administrator password hash cannot write it again as PBKDF2: it says so once a process (`HM7494`), and `sudo hmailserver --set-admin-password` does it. ### Every verb Every row from `--check-database` to the stored-secrets verbs is new since 6.3.4. | Verb | Runs as | What it does | |---|---|---| | `--config ` | | The configuration to read. Default: `hMailServer.ini` beside the executable, then `/etc/hmailserver/hMailServer.ini` | | `--foreground` | the unit | Run in the foreground and log to standard error as well as the log directory. It is the default | | `--check-config` | service user | Print the configuration path, the directories and the database type, and exit without opening a listener or the database. `Database type: 0` means the `Type` key did not take | | `--set-admin-password` | root | Read a new administrator password from standard input with echo off, hash it with PBKDF2, and write it to the file | | `--create-database` | service user | Create the database `[Database]` names, or use an existing empty one, and run the create script for its type. Refuses SQL Server and SQL Server Compact by name. Since 6.3.4, says the wiki's Installing on Linux, a database that holds this build's schema or an older one is left as it is with exit 0, and a newer one exits 1; 6.3.3 refused any database that held a schema. The documentation page (35.4, 35.7) still says a schema is refused, exit 1: test it on a scratch database first | | `--upgrade-database` | service user | Run every upgrade script from the database's version to this build's, in order. Refuses a database from a newer build; stops rather than loops if a script runs and the version does not move. Safe to run by hand | | `--check-database` | service user | Connect once with `TlsMode` and `TlsCaFile`, say whether the connection is encrypted and the certificate checked: exit 0 connected, 1 not, 2 not configured. Changes nothing, and runs beside a running service | | `--set-setting NAME=VALUE`, `--set-setting NAME` | root | Store one `[Settings]` value in the database and write the server's copy into the file (below) | | `--apply-environment` | service user | Store every setting the environment names (`HM_SETTING_`, `HM_PROPERTY_`, their `_FILE` forms, the container's `HM_REST_*`), each checked first: exit 1, with nothing stored, if one is refused. It writes the file's copy too, so the file must be writable, as the container's is; where it is not, as for the service user on a package installation, the verb refuses before it stores anything. [EnvironmentConfiguration.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/EnvironmentConfiguration.md) says it has been run on Windows only, and on Linux compiled but not run: try it on a scratch database first | | `--apply-profile NAME [--confirm] [--once] [--if-new]`, `--revert-profile` | service user, server stopped | Apply one of the four setting profiles (`office`, `gateway`, `relay`, `sender`; `none` records that none is wanted), or put back what the last one changed. Exit 3 means an entry needs `--confirm`, 5 that `--if-new` found ten accounts or more. On Windows the verb refuses while the service runs (exit 4); [Profiles.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Profiles.md) says that on Linux nothing tells it whether a server runs, and that the POSIX verbs have been written and read, and not run: try them on a scratch database first | | `--apply-new-install-defaults [--if-marked []]` | service user, server stopped | The secure defaults of a new installation (below). Exit 0 applied, 1 something could not be written (`HM7510`, `HM7511`), 2 no database or run as root, 5 not a database this installation created, or not as the create script left it | | `--sample-domain create`, `--sample-domain remove` | service user | Make or remove the throwaway `example.test`. Exit 0 done, 2 refused, 1 failed. Remove it before the server takes real mail | | `--measure-store` | service user | What compressing the store and holding its large parts once would save. Changes nothing, and runs beside a running server | | `--compress-store [--decompress]`, `--recount-store` | service user, server stopped | Rewrite the store in place, or back to plain RFC 822; recount the parts held once. Refused while the server runs (exit 4): on Linux the server holds a lock in the log directory for as long as it runs | | `--encrypt-store`, `--decrypt-store` | service user, server stopped | Encrypt what a mailbox encrypted at rest stored before it was switched on, or put it back. `--decrypt-store` reads the domain's recovery passphrase from `HMAILSERVER_RECOVERY_PASSPHRASE`, never from the command line | | `--copy-store-to-object-storage`, `--copy-store-from-object-storage` | service user | Copy the message store into the object store the settings name, or back, each file verified and a stopped run resumed. They run beside the server; `--remove-source` removes what was copied from once its copy is verified, and only with the server stopped (exit 4 beside it) | | `--verify-update ` | | Check a downloaded package or AppImage against the Sigstore bundle beside it; exit 0 when a signer the server trusts made it | | `--create-stored-secrets-key`, `--export-stored-secrets-key `, `--import-stored-secrets-key `, `--forget-sealed-secrets ` | root | Make the stored-secrets key, export it, import it on another node, or, with the server stopped (exit 4 beside it), clear what was sealed under a key lost for good (below) | | `--version`, `--help` | | | `--measure-store`, `--compress-store`, `--encrypt-store`, `--decrypt-store` and the two copies take `--account
` for one mailbox or `--domain ` for one domain's, and neither for the whole store; `--decrypt-store` needs one of the two. **Signals.** `SIGTERM` and `SIGINT` stop the server. `SIGHUP`, which `systemctl reload hmailserver` sends, re-reads the configuration; it stops and restarts the listeners inside the running process, so do it when nothing is mid-delivery. Since 6.3.4 the server drains before it stops: the mail listeners close at the socket, so senders retry, and `ShutdownDrainSeconds` lets what is in flight finish. At its default of 0 nothing is waited for, and a session mid-DATA at the moment of the stop is cut: set it on a production server, where [Upgrading.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Upgrading.md) calls 30 a reasonable figure. ### `--set-setting`: the way in before the API exists (since 6.3.4) From schema 6042 the database is the settings store. Every `[Settings]` key the file holds at the server's first start is taken into the database then, the packaged `RestApiPort=0` among them. Afterwards a line edited in the file is reported in the error log as `HM5804` and the stored value is used, so an edit to the file brings no listener up, and with the listener off nothing that speaks the API can change it either. ```sh sudo hmailserver --set-setting RestApiPort=8045 sudo hmailserver --set-setting RestApiBindAddress=127.0.0.1 sudo systemctl restart hmailserver ``` The verb stores the value as the REST API does and prints what it stored. It refuses a name the server's settings loader does not read, a whole number that is not one, a port that is not 0 to 65535, and a `RestApiBindAddress` that is not an IP address or `localhost`: the checks `--apply-environment` makes, in the same words. A secret's value is refused on the command line, where the shell's history, the process list and sudo's log would keep it. Give the name alone, `sudo hmailserver --set-setting SRSSecret`, and type the value at a prompt that does not echo it, or pipe it in. Which sections and keys stay in the file rather than the database, and the file's `[SettingsOverride]` section, which forces a value over the stored one (`HM5807`, `HM5808`), are in [Settings: what is not stock, when a save takes effect](/topic/28222). ### The stored-secrets key on Linux (since 6.3.4) A Linux server has two secret stores. The **machine's** is the key file `/.hmailserver-secret-key`, mode 0600, made by the server on first use: the database password in the configuration and the route and fetch-account passwords are kept under it as `LINUX1:` envelopes. It travels with the data directory. A secret moved to a machine without it cannot be opened (`HM6414`), and there is no re-keying command: replacing the file means re-entering every stored password. The **installation's** is the stored-secrets key, `[Security] StoredSecretsKey`, under which each account's TOTP second factor and the `[Settings]` secrets are sealed in the database as `HMK1:` envelopes, so that every node sharing the database can open them. **The service never makes it.** On a package installation `--create-database` runs as the service user, cannot write the root-owned file, makes no key and says what to run instead. An upgrade makes none. Until a key exists those secrets are kept as written, and the setup checklist's `stored_secrets_key` item offers the step. Make the key, export it and restart the service; nothing is sealed until that restart: ```sh sudo hmailserver --create-stored-secrets-key sudo hmailserver --export-stored-secrets-key /root/hmailserver-stored-secrets.key sudo systemctl restart hmailserver ``` Keep the exported file with the database backups, apart from them: a database dump restored without it brings back every second factor and `[Settings]` secret unopenable. Carrying the key to a standby with `--import-stored-secrets-key`, what a node without it says (`HM7495`) and `--forget-sealed-secrets` for a key lost for good are in [Hardening a public mail server](/topic/28241). ### The unit, and what it lets the service touch The packaged unit runs the server with `--foreground` as `hmailserver`, and before every start: ```ini ExecStartPre=-/usr/bin/hmailserver --apply-new-install-defaults --if-marked TimeoutStartSec=300 ``` The unit's hardening: `ProtectSystem=strict`, which leaves `/etc` read-only to the service; `/usr` read-only, which with the single capability below is why the service can neither run the package manager nor stop itself; `ProtectHome=yes`, which hides `/home`; `ReadWritePaths` of `/var/lib/hmailserver` and `/var/log/hmailserver`; `RuntimeDirectory=hmailserver`, which is `/run/hmailserver`; and one capability, `CAP_NET_BIND_SERVICE`, which binds the mail ports. `CAP_NET_ADMIN` is deliberately not granted, so the firewall hook behind `AutoBanFirewall` cannot act until a drop-in (`systemctl edit hmailserver`) adds it to `AmbientCapabilities` and `CapabilityBoundingSet`; `sudo -u hmailserver /usr/lib/hmailserver/autoban-hook check` says whether it can. A `DataFolder` outside `ReadWritePaths` needs a drop-in too, before the first start: ```sh sudo systemctl edit hmailserver # [Service] # ReadWritePaths=/srv/mail ``` A memory-safety fault is recorded in one line of `/var/log/hmailserver/crash-oracle.log`; core dumps are off by default. **The first start secures a new installation.** On a database `--create-database` made, the `ExecStartPre` line gives the server a self-signed certificate, STARTTLS on 25, 587, 110 and 143, and TLS required for authentication on the *Internet* IP range; mail clients warn about the certificate until ACME ([Let's Encrypt certificates](/topic/28187)) or your own replaces it. It acts only on a database marked within a day of its clock and still as the create script left it; an older mark is kept for a run by hand, `sudo -u hmailserver hmailserver --apply-new-install-defaults` with the server stopped, and an upgraded or attached database is never changed. The mark, the certificate and the checklist item that follows up are in [Hardening a public mail server](/topic/28241). ### The root helper `hmailserver-update` (since 6.3.4) A live update is applied outside the server's unit. The server writes a one-file request into `/run/hmailserver`; `hmailserver-update.path`, a root path unit the package enables, takes it and runs `/usr/lib/hmailserver/hmailserver-update`, a shell script readable end to end. The script: - checks every line of the request against root's own configuration; - verifies the package, and the rollback image, again as root against their Sigstore bundles, with `hmailserver --verify-update`: the check the server made, made by a process the server could not have influenced; - refuses anything that is not a genuine upgrade: the package's own version is compared with what `dpkg` or `rpm` says is installed, so a signed but superseded release cannot be put back; - stops the unit, installs with `apt-get`/`dpkg` or `dnf`/`rpm`, starts the unit, and waits for the server to stay up, and for `/readyz` when the metrics listener is on; - reinstalls the previous version if it does not, and writes one line to `/Updates/last-apply.txt`, which the server reports at its next check and the Control Deck shows. A refused request is explained in `journalctl -u hmailserver-update` and nowhere else, on purpose. ```sh sudo /usr/lib/hmailserver/hmailserver-update check ``` names the installer it would use, whether the path unit is active and whether the running unit has the runtime directory the request needs. The Debian and RPM packages enable and start the path unit; Arch does not enable units from a package, so there `sudo systemctl enable --now hmailserver-update.path` is yours to run. Nothing happens unless the update check is on and the install is asked for, from the Control Deck or over the API, and only for a release signed by a signer the server trusts, which 6.3.4 is not. ### `hmctl` and the PowerShell module The other command line is `hmctl` (`/usr/bin/hmctl`, one Python 3 file), with the `HmailServer` PowerShell module beside it in the repository. Both speak the REST API, so both administer a Windows or a Linux server, here or on another machine. They cannot turn the REST listener on, set the administrator password, or make or carry the stored-secrets key. They have a topic of their own: [hmctl and the PowerShell module](/topic/28265). ### Logs and the journal The server's own logs are in `/var/log/hmailserver`. The journal (`journalctl -u hmailserver -f`) carries the startup line, standard error and the operational events: `WindowsEventLogEnabled=1` is a Windows inheritance in name only, and on Linux means syslog. The logrotate rule uses `copytruncate`, because the server does not reopen its log on a signal. ### Where to read more - On the wiki: [Installing on Linux](https://gitlab.com/hmailserver/hmailserver/-/wikis/Installing-on-Linux), and [Command Line](https://gitlab.com/hmailserver/hmailserver/-/wikis/Command-Line) for `hmctl`. - [CommandLine.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/CommandLine.md), [SecretsAtRest.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SecretsAtRest.md), [SecureDefaults.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SecureDefaults.md), [Profiles.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Profiles.md) and [MessageStore.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/MessageStore.md). - The documentation page: [35.2 Supported platforms and packages](https://www.progressiverobot.com/hmailserver-documentation/#352-supported-platforms-and-packages), [35.7 Command-line reference](https://www.progressiverobot.com/hmailserver-documentation/#357-command-line-reference), which lists none of the verbs new in 6.3.4, and [35.8 The systemd service](https://www.progressiverobot.com/hmailserver-documentation/#358-the-systemd-service). - On this forum: [hMailServer on Linux](/topic/28189), [First install on Linux](/topic/28190), [Upgrading to 6.3.4](/topic/28193) and [The Control Deck](/topic/28192); [Databases on Linux](/topic/28247) for `--create-database` and the schema upgrade; [the update check and the live update](/topic/28234) for what the root helper is asked to apply; and [message storage](/topic/28233) for what the store verbs change. --- Canonical: https://www.hmailserver.co.uk/topic/28245/the-hmailserver-command-line-on-linux-every-verb-who-runs-it-the-files-the-unit # The hmailserver command line on Linux: every verb, who runs it, the files, the unit *Written for 6.3.4. hMailServer 6.3.6 (6 October 2026) carries .deb and .rpm packages for x86-64 and AArch64, one for each of Debian 12 and 13, Ubuntu 24.04, Fedora 43 and 44, Enterprise Linux 9 and 10, and openSUSE Leap 16.0 and Tumbleweed, both AppImages and the first container image. Its commands say what they did on their own output rather than in the error log, and the store commands run as root refuse a data directory that another account could have linked into. See [the 6.3.6 announcement](/topic/28276). From 6.3.6 the root helper applies hMailServer 6.3.7 (17 October 2026), a security release, by the live update, for the generic `.deb` and `.rpm` of either architecture and both AppImages; a server from one of 6.3.6's per-distribution packages is offered the generic package, which `apt` refuses on Debian 12 and `dnf` on Enterprise Linux 9, and is upgraded with its package manager and its release's own package, and from 6.3.4 or 6.3.5 install 6.3.7 with `apt`, `dnf` or `zypper`, not by the live update. A server whose `UpdateSigningIdentity`, `UpdateSigningIssuer` and `UpdateSourceRepository` still name the path the project began at refuses 6.3.7 until `sudo hmailserver --set-setting =` clears each of the three and the service is restarted. See [the 6.3.7 announcement](/topic/28277).* On Linux the server is one executable, `/usr/bin/hmailserver`, and what cannot be done over the REST API, from creating the schema to rewriting the message store, is done with its own options. This topic lists every verb 6.3.4's documents name and who runs it, where the package puts its files, what the systemd unit lets the service touch, and how the root helper applies a live update. What a Linux server has and lacks is in [hMailServer on Linux](/topic/28189), the first install in [First install on Linux](/topic/28190). The one thing to know first: on a package installation `/etc/hmailserver/hMailServer.ini` is `0640 root:hmailserver`, and the service cannot write it. ### Where the package puts things | Path | What it is | |---|---| | `/usr/bin/hmailserver` | the server, one file | | `/usr/bin/tlds.txt`, `/usr/bin/dh2048.pem` | read from beside the binary; the second is required by every TLS context | | `/usr/share/hmailserver/DBScripts/*.sql` | the schema creation and upgrade scripts | | `/usr/lib/systemd/system/hmailserver.service` | the unit | | `/etc/hmailserver/hMailServer.ini` | the configuration, a conffile an upgrade never overwrites; since 6.3.4 its `[Settings]` section is a copy of the database | | `/usr/bin/hmctl` | since 6.3.4, the command-line client over the REST API | | `/etc/logrotate.d/hmailserver` | the log rotation rule | | `/etc/fail2ban/filter.d/hmailserver.conf`, `/etc/fail2ban/jail.d/hmailserver.conf` | the fail2ban filter and jail, the jail disabled as shipped | | `/usr/lib/hmailserver/hmailserver-update`, `/usr/lib/hmailserver/autoban-hook` | the root update helper (since 6.3.4) and the firewall hook | | `/var/lib/hmailserver`, with `temp`, `events` and `database` under it | the mail store, one directory per domain and one per mailbox; scratch space; event scripts | | `/var/log/hmailserver` | the logs | **What the maintainer scripts do.** They create the `hmailserver` system user and group, and the last two rows above, all `0750 hmailserver:hmailserver`; install the configuration; and enable `hmailserver.service` without starting it, on purpose, because there is no database yet. On an upgrade they run `hmailserver --upgrade-database` as the service user, only if a database is configured, then `try-restart` the service if it was running. On removal they stop and disable, and leave the mail store and the logs alone; a `.deb` purge removes the configuration as well, and `/etc/hmailserver` with it when nothing else is in it, and still leaves the store. **Which packages exist.** 6.3.4 carries two x86-64 packages, `hmailserver_6.3.4_amd64.deb` and `hmailserver-6.3.4-1.x86_64.rpm`, and no container image of 6.3.4 is published. The `.deb` installs on Ubuntu 26.04 and 24.04, not on Debian 12; the `.rpm` on Fedora 43 and 44 and Enterprise Linux 10, not on Enterprise Linux 9; the rest of the platform picture is in [hMailServer on Linux](/topic/28189). The packages carry no Sigstore bundle, so no live update installs them: 6.3.4 is installed by hand ([Upgrading to 6.3.4](/topic/28193)). ### Who runs what **As root (`sudo`), because the file is root's:** `--set-admin-password`, `--set-setting`, and the four stored-secrets verbs. All but the export write `/etc/hmailserver/hMailServer.ini`, and the export a new file of its own (below); `--set-setting` keeps the file's owner and mode. **As the service user (`sudo -u hmailserver`), because the verb opens the database or writes the data directory:** the rest, from `--create-database` to the store walks. `--apply-new-install-defaults` refuses to run as root (exit 2), whose certificate key the service could not read, and `--sample-domain` run as root leaves today's log owned by root. Two things follow from the file being root's. The running server cannot write its copy of a stored value back into `[Settings]`: a value stored over the API is in effect and the file's line goes stale, which the server says once a start in the application log, not as an error. The stale line is never taken for an edit, so nothing needs changing by hand; only what reads the file itself, such as a server that cannot read its table, sees the older value ([how the settings store works](/topic/28222)). The exception is a `[Settings]` secret sealed once the stored-secrets key exists (below): its line cannot be rewritten, and `HM5803` names it at each start until `sudo hmailserver --set-setting ` writes it. And a server that verifies an older MD5 or SHA-256 administrator password hash cannot write it again as PBKDF2: it says so once a process (`HM7494`), and `sudo hmailserver --set-admin-password` does it. ### Every verb Every row from `--check-database` to the stored-secrets verbs is new since 6.3.4. | Verb | Runs as | What it does | |---|---|---| | `--config ` | | The configuration to read. Default: `hMailServer.ini` beside the executable, then `/etc/hmailserver/hMailServer.ini` | | `--foreground` | the unit | Run in the foreground and log to standard error as well as the log directory. It is the default | | `--check-config` | service user | Print the configuration path, the directories and the database type, and exit without opening a listener or the database. `Database type: 0` means the `Type` key did not take | | `--set-admin-password` | root | Read a new administrator password from standard input with echo off, hash it with PBKDF2, and write it to the file | | `--create-database` | service user | Create the database `[Database]` names, or use an existing empty one, and run the create script for its type. Refuses SQL Server and SQL Server Compact by name. Since 6.3.4, says the wiki's Installing on Linux, a database that holds this build's schema or an older one is left as it is with exit 0, and a newer one exits 1; 6.3.3 refused any database that held a schema. The documentation page (35.4, 35.7) still says a schema is refused, exit 1: test it on a scratch database first | | `--upgrade-database` | service user | Run every upgrade script from the database's version to this build's, in order. Refuses a database from a newer build; stops rather than loops if a script runs and the version does not move. Safe to run by hand | | `--check-database` | service user | Connect once with `TlsMode` and `TlsCaFile`, say whether the connection is encrypted and the certificate checked: exit 0 connected, 1 not, 2 not configured. Changes nothing, and runs beside a running service | | `--set-setting NAME=VALUE`, `--set-setting NAME` | root | Store one `[Settings]` value in the database and write the server's copy into the file (below) | | `--apply-environment` | service user | Store every setting the environment names (`HM_SETTING_`, `HM_PROPERTY_`, their `_FILE` forms, the container's `HM_REST_*`), each checked first: exit 1, with nothing stored, if one is refused. It writes the file's copy too, so the file must be writable, as the container's is; where it is not, as for the service user on a package installation, the verb refuses before it stores anything. [EnvironmentConfiguration.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/EnvironmentConfiguration.md) says it has been run on Windows only, and on Linux compiled but not run: try it on a scratch database first | | `--apply-profile NAME [--confirm] [--once] [--if-new]`, `--revert-profile` | service user, server stopped | Apply one of the four setting profiles (`office`, `gateway`, `relay`, `sender`; `none` records that none is wanted), or put back what the last one changed. Exit 3 means an entry needs `--confirm`, 5 that `--if-new` found ten accounts or more. On Windows the verb refuses while the service runs (exit 4); [Profiles.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Profiles.md) says that on Linux nothing tells it whether a server runs, and that the POSIX verbs have been written and read, and not run: try them on a scratch database first | | `--apply-new-install-defaults [--if-marked []]` | service user, server stopped | The secure defaults of a new installation (below). Exit 0 applied, 1 something could not be written (`HM7510`, `HM7511`), 2 no database or run as root, 5 not a database this installation created, or not as the create script left it | | `--sample-domain create`, `--sample-domain remove` | service user | Make or remove the throwaway `example.test`. Exit 0 done, 2 refused, 1 failed. Remove it before the server takes real mail | | `--measure-store` | service user | What compressing the store and holding its large parts once would save. Changes nothing, and runs beside a running server | | `--compress-store [--decompress]`, `--recount-store` | service user, server stopped | Rewrite the store in place, or back to plain RFC 822; recount the parts held once. Refused while the server runs (exit 4): on Linux the server holds a lock in the log directory for as long as it runs | | `--encrypt-store`, `--decrypt-store` | service user, server stopped | Encrypt what a mailbox encrypted at rest stored before it was switched on, or put it back. `--decrypt-store` reads the domain's recovery passphrase from `HMAILSERVER_RECOVERY_PASSPHRASE`, never from the command line | | `--copy-store-to-object-storage`, `--copy-store-from-object-storage` | service user | Copy the message store into the object store the settings name, or back, each file verified and a stopped run resumed. They run beside the server; `--remove-source` removes what was copied from once its copy is verified, and only with the server stopped (exit 4 beside it) | | `--verify-update ` | | Check a downloaded package or AppImage against the Sigstore bundle beside it; exit 0 when a signer the server trusts made it | | `--create-stored-secrets-key`, `--export-stored-secrets-key `, `--import-stored-secrets-key `, `--forget-sealed-secrets ` | root | Make the stored-secrets key, export it, import it on another node, or, with the server stopped (exit 4 beside it), clear what was sealed under a key lost for good (below) | | `--version`, `--help` | | | `--measure-store`, `--compress-store`, `--encrypt-store`, `--decrypt-store` and the two copies take `--account
` for one mailbox or `--domain ` for one domain's, and neither for the whole store; `--decrypt-store` needs one of the two. **Signals.** `SIGTERM` and `SIGINT` stop the server. `SIGHUP`, which `systemctl reload hmailserver` sends, re-reads the configuration; it stops and restarts the listeners inside the running process, so do it when nothing is mid-delivery. Since 6.3.4 the server drains before it stops: the mail listeners close at the socket, so senders retry, and `ShutdownDrainSeconds` lets what is in flight finish. At its default of 0 nothing is waited for, and a session mid-DATA at the moment of the stop is cut: set it on a production server, where [Upgrading.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Upgrading.md) calls 30 a reasonable figure. ### `--set-setting`: the way in before the API exists (since 6.3.4) From schema 6042 the database is the settings store. Every `[Settings]` key the file holds at the server's first start is taken into the database then, the packaged `RestApiPort=0` among them. Afterwards a line edited in the file is reported in the error log as `HM5804` and the stored value is used, so an edit to the file brings no listener up, and with the listener off nothing that speaks the API can change it either. ```sh sudo hmailserver --set-setting RestApiPort=8045 sudo hmailserver --set-setting RestApiBindAddress=127.0.0.1 sudo systemctl restart hmailserver ``` The verb stores the value as the REST API does and prints what it stored. It refuses a name the server's settings loader does not read, a whole number that is not one, a port that is not 0 to 65535, and a `RestApiBindAddress` that is not an IP address or `localhost`: the checks `--apply-environment` makes, in the same words. A secret's value is refused on the command line, where the shell's history, the process list and sudo's log would keep it. Give the name alone, `sudo hmailserver --set-setting SRSSecret`, and type the value at a prompt that does not echo it, or pipe it in. Which sections and keys stay in the file rather than the database, and the file's `[SettingsOverride]` section, which forces a value over the stored one (`HM5807`, `HM5808`), are in [Settings: what is not stock, when a save takes effect](/topic/28222). ### The stored-secrets key on Linux (since 6.3.4) A Linux server has two secret stores. The **machine's** is the key file `/.hmailserver-secret-key`, mode 0600, made by the server on first use: the database password in the configuration and the route and fetch-account passwords are kept under it as `LINUX1:` envelopes. It travels with the data directory. A secret moved to a machine without it cannot be opened (`HM6414`), and there is no re-keying command: replacing the file means re-entering every stored password. The **installation's** is the stored-secrets key, `[Security] StoredSecretsKey`, under which each account's TOTP second factor and the `[Settings]` secrets are sealed in the database as `HMK1:` envelopes, so that every node sharing the database can open them. **The service never makes it.** On a package installation `--create-database` runs as the service user, cannot write the root-owned file, makes no key and says what to run instead. An upgrade makes none. Until a key exists those secrets are kept as written, and the setup checklist's `stored_secrets_key` item offers the step. Make the key, export it and restart the service; nothing is sealed until that restart: ```sh sudo hmailserver --create-stored-secrets-key sudo hmailserver --export-stored-secrets-key /root/hmailserver-stored-secrets.key sudo systemctl restart hmailserver ``` Keep the exported file with the database backups, apart from them: a database dump restored without it brings back every second factor and `[Settings]` secret unopenable. Carrying the key to a standby with `--import-stored-secrets-key`, what a node without it says (`HM7495`) and `--forget-sealed-secrets` for a key lost for good are in [Hardening a public mail server](/topic/28241). ### The unit, and what it lets the service touch The packaged unit runs the server with `--foreground` as `hmailserver`, and before every start: ```ini ExecStartPre=-/usr/bin/hmailserver --apply-new-install-defaults --if-marked TimeoutStartSec=300 ``` The unit's hardening: `ProtectSystem=strict`, which leaves `/etc` read-only to the service; `/usr` read-only, which with the single capability below is why the service can neither run the package manager nor stop itself; `ProtectHome=yes`, which hides `/home`; `ReadWritePaths` of `/var/lib/hmailserver` and `/var/log/hmailserver`; `RuntimeDirectory=hmailserver`, which is `/run/hmailserver`; and one capability, `CAP_NET_BIND_SERVICE`, which binds the mail ports. `CAP_NET_ADMIN` is deliberately not granted, so the firewall hook behind `AutoBanFirewall` cannot act until a drop-in (`systemctl edit hmailserver`) adds it to `AmbientCapabilities` and `CapabilityBoundingSet`; `sudo -u hmailserver /usr/lib/hmailserver/autoban-hook check` says whether it can. A `DataFolder` outside `ReadWritePaths` needs a drop-in too, before the first start: ```sh sudo systemctl edit hmailserver # [Service] # ReadWritePaths=/srv/mail ``` A memory-safety fault is recorded in one line of `/var/log/hmailserver/crash-oracle.log`; core dumps are off by default. **The first start secures a new installation.** On a database `--create-database` made, the `ExecStartPre` line gives the server a self-signed certificate, STARTTLS on 25, 587, 110 and 143, and TLS required for authentication on the *Internet* IP range; mail clients warn about the certificate until ACME ([Let's Encrypt certificates](/topic/28187)) or your own replaces it. It acts only on a database marked within a day of its clock and still as the create script left it; an older mark is kept for a run by hand, `sudo -u hmailserver hmailserver --apply-new-install-defaults` with the server stopped, and an upgraded or attached database is never changed. The mark, the certificate and the checklist item that follows up are in [Hardening a public mail server](/topic/28241). ### The root helper `hmailserver-update` (since 6.3.4) A live update is applied outside the server's unit. The server writes a one-file request into `/run/hmailserver`; `hmailserver-update.path`, a root path unit the package enables, takes it and runs `/usr/lib/hmailserver/hmailserver-update`, a shell script readable end to end. The script: - checks every line of the request against root's own configuration; - verifies the package, and the rollback image, again as root against their Sigstore bundles, with `hmailserver --verify-update`: the check the server made, made by a process the server could not have influenced; - refuses anything that is not a genuine upgrade: the package's own version is compared with what `dpkg` or `rpm` says is installed, so a signed but superseded release cannot be put back; - stops the unit, installs with `apt-get`/`dpkg` or `dnf`/`rpm`, starts the unit, and waits for the server to stay up, and for `/readyz` when the metrics listener is on; - reinstalls the previous version if it does not, and writes one line to `/Updates/last-apply.txt`, which the server reports at its next check and the Control Deck shows. A refused request is explained in `journalctl -u hmailserver-update` and nowhere else, on purpose. ```sh sudo /usr/lib/hmailserver/hmailserver-update check ``` names the installer it would use, whether the path unit is active and whether the running unit has the runtime directory the request needs. The Debian and RPM packages enable and start the path unit; Arch does not enable units from a package, so there `sudo systemctl enable --now hmailserver-update.path` is yours to run. Nothing happens unless the update check is on and the install is asked for, from the Control Deck or over the API, and only for a release signed by a signer the server trusts, which 6.3.4 is not. ### `hmctl` and the PowerShell module The other command line is `hmctl` (`/usr/bin/hmctl`, one Python 3 file), with the `HmailServer` PowerShell module beside it in the repository. Both speak the REST API, so both administer a Windows or a Linux server, here or on another machine. They cannot turn the REST listener on, set the administrator password, or make or carry the stored-secrets key. They have a topic of their own: [hmctl and the PowerShell module](/topic/28265). ### Logs and the journal The server's own logs are in `/var/log/hmailserver`. The journal (`journalctl -u hmailserver -f`) carries the startup line, standard error and the operational events: `WindowsEventLogEnabled=1` is a Windows inheritance in name only, and on Linux means syslog. The logrotate rule uses `copytruncate`, because the server does not reopen its log on a signal. ### Where to read more - On the wiki: [Installing on Linux](https://gitlab.com/hmailserver/hmailserver/-/wikis/Installing-on-Linux), and [Command Line](https://gitlab.com/hmailserver/hmailserver/-/wikis/Command-Line) for `hmctl`. - [CommandLine.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/CommandLine.md), [SecretsAtRest.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SecretsAtRest.md), [SecureDefaults.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SecureDefaults.md), [Profiles.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Profiles.md) and [MessageStore.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/MessageStore.md). - The documentation page: [35.2 Supported platforms and packages](https://www.progressiverobot.com/hmailserver-documentation/#352-supported-platforms-and-packages), [35.7 Command-line reference](https://www.progressiverobot.com/hmailserver-documentation/#357-command-line-reference), which lists none of the verbs new in 6.3.4, and [35.8 The systemd service](https://www.progressiverobot.com/hmailserver-documentation/#358-the-systemd-service). - On this forum: [hMailServer on Linux](/topic/28189), [First install on Linux](/topic/28190), [Upgrading to 6.3.4](/topic/28193) and [The Control Deck](/topic/28192); [Databases on Linux](/topic/28247) for `--create-database` and the schema upgrade; [the update check and the live update](/topic/28234) for what the root helper is asked to apply; and [message storage](/topic/28233) for what the store verbs change. --- Canonical: https://www.hmailserver.co.uk/topic/28246/in-a-container-or-on-kubernetes-the-image-compose-helm-and-the-environment # In a container or on Kubernetes: the image, Compose, Helm and the environment *Written at 6.3.4, when no image was published. hMailServer 6.3.6 (6 October 2026) publishes the first: `registry.gitlab.com/hmailserver/hmailserver:6.3.6`, also tagged `6.3` and `latest`, one index over linux/amd64 and linux/arm64, signed with cosign as the release's pipeline; the Helm chart is a file of the release, and the packaging Compose file now runs PostgreSQL 18. See [the 6.3.6 announcement](/topic/28276) and `ContainerImage.md` at the v6.3.6 tag. hMailServer 6.3.7 (17 October 2026), a security release, is `registry.gitlab.com/hmailserver/hmailserver:6.3.7`, for linux/amd64 and linux/arm64, and `6.3` and `latest` now name it; in the container the update check names the image to pull and installs nothing, and the schema stays at 6108, so a container recreated on it from 6.3.6 changes no table. Its IMAP and POP3 listeners take the client's address from a PROXY protocol header, as the SMTP listener does, each from the peers on a list of its own, both off by default. See [the 6.3.7 announcement](/topic/28277).* The Linux server is also a container image: the x86-64 `.deb` installed on Ubuntu 24.04 and run as the `hmailserver` user, with the database connection, the REST listener, the administrator password and, since 6.3.4, every other setting taken from the environment at start. This topic describes the image, the Compose file and the Helm chart as they are in the tree at the v6.3.4 tag, and the `HM_*` variables the entrypoint reads. The one thing to know first, from the 6.3.4 changelog: **no container image of 6.3.4 is published.** The images published on `ghcr.io/progressiverobot/hmailserver` before GitHub's suspension answer 404, and `registry.gitlab.com/progressiverobot/hmailserver`, which the Compose file and the chart now name, holds nothing. Meanwhile, build the image from the release's `.deb` and use your own tag wherever the files say `registry.gitlab.com/...`. What a Linux server is and is not is in [hMailServer on Linux](/topic/28189). ### Building the image yourself Use the `docker` directory of the same tag as the package: 6.3.4's entrypoint runs verbs a 6.3.3 binary does not have. ```sh git clone https://gitlab.com/hmailserver/hmailserver.git cd hmailserver git checkout v6.3.4 cd hmailserver/source/Server/platform/packaging/docker mkdir packages curl -L -o packages/hmailserver_6.3.4_amd64.deb \ https://gitlab.com/hmailserver/hmailserver/-/releases/v6.3.4/downloads/hmailserver_amd64.deb docker build -t hmailserver:6.3.4 . ``` The Dockerfile copies `packages/hmailserver_*_amd64.deb` (the `HM_DEB` build argument names another path) and pins its Ubuntu base by digest. 6.3.4's package carries no Sigstore bundle to check it against, as 6.3.3's did. The image is x86-64 only: the Dockerfile takes the amd64 package. ### What is in the image, and what is not **In:** `/usr/bin/hmailserver`, the packaged `hMailServer.ini` at `/etc/hmailserver/`, the database scripts, the Control Deck, the fail2ban filter and jail the package carries, `ca-certificates` for the TLS the server does as a client, and an entrypoint. About 40 MB. The process runs as `hmailserver`; ports below 1024 are bound through `cap_net_bind_service` on the binary, not through root. Since 6.3.4 the Dockerfile installs the package's two Recommends, `7zip` and `python3`: without them a backup failed with "Could not add the backup settings to the archive", and `hmctl` had no Python 3. **Not in: the database**, and it never will be. Point the image at a PostgreSQL or MariaDB server, in another container or anywhere else; on first start the entrypoint creates the schema there, and on every later start upgrades it if it is behind. Also not in: `curl`, an editor, or a shell of any consequence beyond `bash`. There is no Control Panel and no COM; the REST API is the administration. ### The variables the entrypoint reads itself Sixteen of these are older than 6.3.4 and keep their names and meanings. | Variable | Writes | Notes | |---|---|---| | `HM_DB_TYPE` | `[Database] Type` | `PostgreSQL` or `MySQL` (MariaDB is `MySQL`). Setting this is what makes the section be written at all | | `HM_DB_HOST`, `HM_DB_PORT` | `Server`, `Port` | Defaults `db`, and the type's port (5432 or 3306) | | `HM_DB_NAME`, `HM_DB_USER`, `HM_DB_PASSWORD` | `Database`, `Username`, `Password` | Defaults `hmailserver`, `hmailserver`, empty. The user needs the right to create the database on first start | | `HM_DB_TLS_MODE`, `HM_DB_TLS_CA` | `TlsMode`, `TlsCaFile` | Since 6.3.4. `disable`, `prefer` (empty, the default), `require` or `verify-full`, and the path of a PEM file of CA certificates mounted into the container | | `HM_ADMIN_PASSWORD` | `[Security] AdministratorPassword` | Through the server's own `--set-admin-password`: the file holds a PBKDF2 hash, never the password. Without one the REST API refuses to start | | `HM_REST_PORT` | `RestApiPort` | The packaged file has 0, which is off; 8045 is the conventional port | | `HM_REST_BIND_ADDRESS` | `RestApiBindAddress` | The packaged file has `127.0.0.1`, which inside a container is the container's own loopback | | `HM_REST_CERTIFICATE_FILE`, `HM_REST_PRIVATE_KEY_FILE` | `RestApiCertificateFile`, `RestApiPrivateKeyFile` | Paths inside the container, readable by the `hmailserver` user. Paths the server reads, not `_FILE` forms | | `HM_DB_PASSWORD_FILE`, `HM_ADMIN_PASSWORD_FILE` | the same two secrets | Read from a file: a Docker or Compose secret. Setting both a variable and its `_FILE` is refused | | `HM_CREATE_DATABASE`, `HM_UPGRADE_DATABASE` | nothing | `0` skips `--create-database` or `--upgrade-database` at start. Both are no-ops when there is nothing to do | | `HM_INI` | which file is written | Default `/etc/hmailserver/hMailServer.ini` | | `HM_PROFILE`, `HM_PROFILE_CONFIRM` | nothing | Since 6.3.4: the setting profile applied once, and its confirmation | **On 6.3.3 and earlier** the entrypoint writes these into `hMailServer.ini` on every start, and every other setting is the packaged file in the `/etc/hmailserver` volume. **Since 6.3.4** the database is the settings store, so the four `HM_REST_*` variables are aliases of `HM_SETTING_RestApiPort` and its three neighbours, stored like any other setting; `[Database]` is still written into the file at every start, because it says where the database is. The image's `/etc/hmailserver` is owned by the `hmailserver` user, so there, unlike on a package installation, the server keeps its copy of each stored value in the file, and a `--create-database` that creates the schema also makes the stored-secrets key. ### Every setting from the environment (since 6.3.4) After `--create-database` and `--upgrade-database`, and before the server, the entrypoint runs the server's own verb, `hmailserver --apply-environment`, whenever there is anything for it to do. `HM_SETTING_` sets a `[Settings]` key and `HM_PROPERTY_` a database setting (`HM_SETTING_MaxSubmissionsPerIPPerMinute=30`, `HM_PROPERTY_maxmessagesize=20480`), and either with `_FILE` reads the value from a file, a Compose secret or a Kubernetes Secret mounted as one. - **The environment wins, at every start.** An administrator cannot change a setting the environment names until the variable is unset and the container restarted, and a ConfigMap change, too, is applied by a restart and not live. - **Everything is checked before anything is stored.** When anything is refused, nothing is stored and **the container does not start**; `docker compose logs` or `kubectl logs` says which variable and why. The naming rule, every refusal, what the REST API and the Control Deck show for a setting the environment manages, and how a value from a file stays secret are in [configuration from the environment](/topic/28231). **`HM_PROFILE`** names one of the four setting profiles, `office`, `gateway`, `relay` or `sender`, which the entrypoint applies once, with `--apply-profile NAME --once`, after the environment's settings and before the server starts. `HM_PROFILE=none` records that none is wanted. A profile that closes a listener, trusts an address range, turns a filter or a check off or weakens it, or shortens a window of days needs `HM_PROFILE_CONFIRM=yes`, and without it the container does not start. A profile skips a setting the environment manages, naming its variable. **Not yet run in a container.** The verb has been run as `hMailServer.exe --apply-environment` on Windows, and its POSIX half compiles; the entrypoint as a whole, the POSIX verb against a database, the chart and the Compose file have been read and not executed. `HM_PROFILE` is in the same state. Try it on a scratch database first. ### Reaching the API The server requires TLS on a REST listener that is not bound to loopback. A listener left at the packaged bind address answers only processes inside the container, which is none, since the image carries no client. Two ways out; either will do: - **Bind it and give it a certificate.** `HM_REST_BIND_ADDRESS=0.0.0.0`, with `HM_REST_CERTIFICATE_FILE` and `HM_REST_PRIVATE_KEY_FILE` pointing at a certificate and key mounted into the container, and the port published. Publish it on the host's loopback (`127.0.0.1:8045:8045`) unless the certificate is one clients will trust. - **Host networking** (`--network host`). The container's `127.0.0.1` is the host's, the packaged bind address is right as it is, no certificate is needed, and the API is at `http://127.0.0.1:8045` from the host and through an SSH tunnel from elsewhere. The mail ports are then the host's ports directly. ### First start, with Compose The file to use is in the repository, at `hmailserver/source/Server/platform/packaging/docker/docker-compose.yml`. It reads the two passwords as Compose secrets from files, mounts the REST certificate and key in, and carries the `HM_SETTING_` and `HM_PROPERTY_` examples commented out. A workflow resolved it with `docker compose config` on every change until 18 September 2026, so it is known to parse. Below is the `mail` service of the wiki's shorter example, which puts it under `services:` beside a `postgres:16` service named `db` and declares the named volumes. It names the image built above, not the empty registry: ```yaml mail: image: hmailserver:6.3.4 depends_on: - db environment: HM_DB_TYPE: PostgreSQL HM_DB_HOST: db HM_DB_NAME: hmailserver HM_DB_USER: hmailserver HM_DB_PASSWORD: change-me HM_ADMIN_PASSWORD: change-me-too HM_REST_PORT: 8045 HM_REST_BIND_ADDRESS: 0.0.0.0 HM_REST_CERTIFICATE_FILE: /certs/rest.pem HM_REST_PRIVATE_KEY_FILE: /certs/rest.key volumes: - ./certs:/certs:ro - hm-data:/var/lib/hmailserver - hm-logs:/var/log/hmailserver - hm-etc:/etc/hmailserver ports: - "25:25" - "587:587" - "993:993" - "127.0.0.1:8045:8045" restart: unless-stopped ``` `restart: unless-stopped` matters on the first start: `depends_on` starts the database container first but does not wait for it to accept connections, so the entrypoint's `--create-database` fails and the container exits if the database is not there yet; the restart tries again a moment later. The first start logs the `[Database]` section written, the administrator password hashed and the database created at schema 6104, and also what `--create-database` did about the stored-secrets key and one line per `HM_SETTING_` or `HM_PROPERTY_` variable stored. The first domain is then one request away ([First install on Linux](/topic/28190)). Passwords in a Compose file are passwords in a file: for anything beyond a first look, use `HM_DB_PASSWORD_FILE` and `HM_ADMIN_PASSWORD_FILE` with Compose secrets. **The secure defaults of a new installation are not applied in a container by themselves.** On a package installation the systemd unit's first start gives a database `--create-database` made a self-signed certificate, STARTTLS and TLS required for authentication on the *Internet* IP range. The image has no unit, so that step is run by hand, as the service's user, with the server stopped. [SecureDefaults.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SecureDefaults.md) gives the verb and no container command for it, and lists its refusal of a running service (exit 4) for Windows only: ```sh hmailserver --apply-new-install-defaults ``` A run by hand has no one-day deadline, so an image first booted days after its `--create-database` is secured this way. What it changes, its exit codes and the checklist item that says what is left are in [Hardening a public mail server](/topic/28241). ### Kubernetes, with the chart Since 6.3.4 the chart is at `hmailserver/source/Server/platform/packaging/helm/hmailserver`. It is not published to a chart repository; install it from a checkout. It renders a **StatefulSet of one**, because a message store is not something two pods may share: the three volumes the image declares become claim templates, and the database is pointed at, never bundled. The chart refuses to render without `database.host`. Its `image.repository` names the empty registry, so set it to the image you built, pushed to a registry the cluster's nodes can pull from: ``` helm install mail ./hmailserver \ --set image.repository=/hmailserver \ --set database.host=postgres.databases.svc.cluster.local \ --set database.password=... \ --set administrator.password=... \ --set rest.tlsSecretName=mail-tls ``` | Value | What it is | |---|---| | `image.repository` | `registry.gitlab.com/progressiverobot/hmailserver` as shipped, which holds no image: set it to where you pushed the image you built | | `database.host`, `.type`, `.name`, `.user` | The PostgreSQL or MariaDB the server uses; required. `database.create` and `database.upgrade` make the schema on first start and move it forward on an upgrade | | `database.password`, `administrator.password` | Go into a Secret the chart makes. Point `existingSecret` at a Secret you made and the chart writes none; its two keys are named by `existingSecretKeys` | | `rest.enabled`, `.port`, `.tlsSecretName` | The one listener that serves the REST API, the Control Deck and the webmail. Off loopback the server requires TLS, so without a `kubernetes.io/tls` Secret named here the listener will not start; the chart's notes say so on install | | `settings`, `properties`, `settingsFrom` | Since 6.3.4: any `[Settings]` key (rendered as `HM_SETTING_`), any database setting (`HM_PROPERTY_`), and a key of a Secret you made, projected into `/run/secrets/hmailserver-settings/` and named through `_FILE`, so a secret value is in no environment variable, rendered manifest or log line | | `service.type`, `.ports`, `.externalTrafficPolicy` | A LoadBalancer with the mail ports, and the REST port when the listener is on. `Local` keeps the client's address, which auto-ban, greylisting and the `Received` header all need | | `persistence.*` | The message store, the logs and the configuration, one claim each | | Probes | Port 25, as the image's own health check is, or the metrics listener's `/livez` and `/readyz` when that listener is on | `settings`, `properties` and `settingsFrom` look like this in a values file; a `settingsFrom` entry says `setting:` in place of `property:` for a `[Settings]` key: ```yaml settings: MaxSubmissionsPerIPPerMinute: "30" properties: maxmessagesize: "20480" settingsFrom: - secretName: mail-relay key: password property: smtprelayerpassword ``` A switch is `1` or `0` for a `[Settings]` key; `true`, `false`, yes, no, on and off are taken as well for a setting whose REST settings row is a switch, and only there: `maxmessagesize=off` is refused rather than stored as 0, which is "no limit". ### Volumes, ports and the health check | Path | What lives there | |---|---| | `/var/lib/hmailserver` | The mail store, and `.hmailserver-secret-key`, the key under which this machine's stored secrets are protected. The key travels with this directory: a database restored beside a fresh data volume cannot open the secrets it holds | | `/var/log/hmailserver` | The logs. Bind-mount it to a host path if fail2ban on the host is to read it | | `/etc/hmailserver` | `hMailServer.ini`; since 6.3.4 the only home of `PasswordPepper` and `[Security] StoredSecretsKey`, which no database and no backup holds | The image exposes 25, 110, 143, 465, 587, 993, 995, 4190 (ManageSieve) and 8045; publish the ones you serve. The health check opens a connection to the container's own port 25 every 30 seconds after a 60-second start period, and three failures in a row mark the container unhealthy. ### Upgrades, backups and auto-ban **Upgrading** is building the image of the new tag, as no 6.3.4 image is published to pull, and recreating the container: the entrypoint runs `--upgrade-database` on every start, so a schema that is behind is brought forward before the server opens a port. On MySQL and MariaDB the 6102 step rebuilds every table as `utf8mb4`, which takes time on a large database and needs the database server version that [Databases on Linux](/topic/28247) gives. An upgraded database gets no stored-secrets key until one is made with `hmailserver --create-stored-secrets-key`, and never the new installation's defaults. Take the backup first ([Upgrading to 6.3.4](/topic/28193)). **Backups** are three things together: the database, the `/var/lib/hmailserver` volume, and `/etc/hmailserver`. Two things are only in `/etc/hmailserver/hMailServer.ini`: a `PasswordPepper`, if one is set, without which a restored database refuses the password of every account hashed with Argon2id or scrypt, and the stored-secrets key, without which a restored database opens no account's second factor and no `[Settings]` secret. Export the key once, and keep the file with the database backups, apart from them: ```sh docker exec hmailserver --export-stored-secrets-key /var/lib/hmailserver/stored-secrets.key docker cp :/var/lib/hmailserver/stored-secrets.key ./stored-secrets.key ``` A replacement container takes it back with `--import-stored-secrets-key FILE`. **Auto-ban** works as everywhere: an address that fails to log on repeatedly is refused by the listeners. The hook that adds the address to the host's firewall cannot run from inside a container, so run fail2ban on the host instead: bind-mount `/var/log/hmailserver`, copy the filter and jail out of the image, point the jail's `logpath` at the mounted directory, and enable it. ### What was proven, and what has not been Until 18 September 2026 the `image` job of the GitHub workflow built the image on every release tag, pushed it, and smoke-tested the pushed image as the example above runs it: the container healthy, the SMTP banner and the API answering, and the administrator password in no file in clear. Nothing has run it since. GitLab CI's `linux-image` job is written to build the image from the `.deb` with kaniko and inspect it; it neither pushes nor runs the image, so the smoke test has no replacement yet. The `chart-and-compose` job is written to lint and render the chart and resolve the Compose file. Neither had run when [ContinuousIntegration.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/ContinuousIntegration.md) was written, on 25 September 2026. The changelog's 6.3.4 section records the first full pipeline the day after: it finished failed, the image job among those that passed, and no pipeline has run on the fixed tree since. 6.3.4 was made by hand. ### Where to read more - [Running in a Container](https://gitlab.com/hmailserver/hmailserver/-/wikis/Running-in-a-Container) and [Profiles and Environment](https://gitlab.com/hmailserver/hmailserver/-/wikis/Profiles-and-Environment) on the wiki. - [EnvironmentConfiguration.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/EnvironmentConfiguration.md) for the naming rule and every refusal, [Profiles.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Profiles.md) for every entry of the four profiles, [SecureDefaults.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SecureDefaults.md) for the verb run by hand, and [DatabaseTls.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/DatabaseTls.md) for `HM_DB_TLS_MODE`. - The 6.3.4 section of [CHANGELOG.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/CHANGELOG.md) for the state of the image. - On this forum: [the 6.3.4 announcement](/topic/28218), [hMailServer on Linux](/topic/28189), [First install on Linux](/topic/28190) and [Upgrading to 6.3.4](/topic/28193); [configuration from the environment](/topic/28231) for `HM_SETTING_` and `HM_PROPERTY_` in full, and [the hmailserver command line on Linux](/topic/28245) for the verbs the entrypoint runs. --- Canonical: https://www.hmailserver.co.uk/topic/28246/in-a-container-or-on-kubernetes-the-image-compose-helm-and-the-environment # In a container or on Kubernetes: the image, Compose, Helm and the environment *Written at 6.3.4, when no image was published. hMailServer 6.3.6 (6 October 2026) publishes the first: `registry.gitlab.com/hmailserver/hmailserver:6.3.6`, also tagged `6.3` and `latest`, one index over linux/amd64 and linux/arm64, signed with cosign as the release's pipeline; the Helm chart is a file of the release, and the packaging Compose file now runs PostgreSQL 18. See [the 6.3.6 announcement](/topic/28276) and `ContainerImage.md` at the v6.3.6 tag. hMailServer 6.3.7 (17 October 2026), a security release, is `registry.gitlab.com/hmailserver/hmailserver:6.3.7`, for linux/amd64 and linux/arm64, and `6.3` and `latest` now name it; in the container the update check names the image to pull and installs nothing, and the schema stays at 6108, so a container recreated on it from 6.3.6 changes no table. Its IMAP and POP3 listeners take the client's address from a PROXY protocol header, as the SMTP listener does, each from the peers on a list of its own, both off by default. See [the 6.3.7 announcement](/topic/28277).* The Linux server is also a container image: the x86-64 `.deb` installed on Ubuntu 24.04 and run as the `hmailserver` user, with the database connection, the REST listener, the administrator password and, since 6.3.4, every other setting taken from the environment at start. This topic describes the image, the Compose file and the Helm chart as they are in the tree at the v6.3.4 tag, and the `HM_*` variables the entrypoint reads. The one thing to know first, from the 6.3.4 changelog: **no container image of 6.3.4 is published.** The images published on `ghcr.io/progressiverobot/hmailserver` before GitHub's suspension answer 404, and `registry.gitlab.com/progressiverobot/hmailserver`, which the Compose file and the chart now name, holds nothing. Meanwhile, build the image from the release's `.deb` and use your own tag wherever the files say `registry.gitlab.com/...`. What a Linux server is and is not is in [hMailServer on Linux](/topic/28189). ### Building the image yourself Use the `docker` directory of the same tag as the package: 6.3.4's entrypoint runs verbs a 6.3.3 binary does not have. ```sh git clone https://gitlab.com/hmailserver/hmailserver.git cd hmailserver git checkout v6.3.4 cd hmailserver/source/Server/platform/packaging/docker mkdir packages curl -L -o packages/hmailserver_6.3.4_amd64.deb \ https://gitlab.com/hmailserver/hmailserver/-/releases/v6.3.4/downloads/hmailserver_amd64.deb docker build -t hmailserver:6.3.4 . ``` The Dockerfile copies `packages/hmailserver_*_amd64.deb` (the `HM_DEB` build argument names another path) and pins its Ubuntu base by digest. 6.3.4's package carries no Sigstore bundle to check it against, as 6.3.3's did. The image is x86-64 only: the Dockerfile takes the amd64 package. ### What is in the image, and what is not **In:** `/usr/bin/hmailserver`, the packaged `hMailServer.ini` at `/etc/hmailserver/`, the database scripts, the Control Deck, the fail2ban filter and jail the package carries, `ca-certificates` for the TLS the server does as a client, and an entrypoint. About 40 MB. The process runs as `hmailserver`; ports below 1024 are bound through `cap_net_bind_service` on the binary, not through root. Since 6.3.4 the Dockerfile installs the package's two Recommends, `7zip` and `python3`: without them a backup failed with "Could not add the backup settings to the archive", and `hmctl` had no Python 3. **Not in: the database**, and it never will be. Point the image at a PostgreSQL or MariaDB server, in another container or anywhere else; on first start the entrypoint creates the schema there, and on every later start upgrades it if it is behind. Also not in: `curl`, an editor, or a shell of any consequence beyond `bash`. There is no Control Panel and no COM; the REST API is the administration. ### The variables the entrypoint reads itself Sixteen of these are older than 6.3.4 and keep their names and meanings. | Variable | Writes | Notes | |---|---|---| | `HM_DB_TYPE` | `[Database] Type` | `PostgreSQL` or `MySQL` (MariaDB is `MySQL`). Setting this is what makes the section be written at all | | `HM_DB_HOST`, `HM_DB_PORT` | `Server`, `Port` | Defaults `db`, and the type's port (5432 or 3306) | | `HM_DB_NAME`, `HM_DB_USER`, `HM_DB_PASSWORD` | `Database`, `Username`, `Password` | Defaults `hmailserver`, `hmailserver`, empty. The user needs the right to create the database on first start | | `HM_DB_TLS_MODE`, `HM_DB_TLS_CA` | `TlsMode`, `TlsCaFile` | Since 6.3.4. `disable`, `prefer` (empty, the default), `require` or `verify-full`, and the path of a PEM file of CA certificates mounted into the container | | `HM_ADMIN_PASSWORD` | `[Security] AdministratorPassword` | Through the server's own `--set-admin-password`: the file holds a PBKDF2 hash, never the password. Without one the REST API refuses to start | | `HM_REST_PORT` | `RestApiPort` | The packaged file has 0, which is off; 8045 is the conventional port | | `HM_REST_BIND_ADDRESS` | `RestApiBindAddress` | The packaged file has `127.0.0.1`, which inside a container is the container's own loopback | | `HM_REST_CERTIFICATE_FILE`, `HM_REST_PRIVATE_KEY_FILE` | `RestApiCertificateFile`, `RestApiPrivateKeyFile` | Paths inside the container, readable by the `hmailserver` user. Paths the server reads, not `_FILE` forms | | `HM_DB_PASSWORD_FILE`, `HM_ADMIN_PASSWORD_FILE` | the same two secrets | Read from a file: a Docker or Compose secret. Setting both a variable and its `_FILE` is refused | | `HM_CREATE_DATABASE`, `HM_UPGRADE_DATABASE` | nothing | `0` skips `--create-database` or `--upgrade-database` at start. Both are no-ops when there is nothing to do | | `HM_INI` | which file is written | Default `/etc/hmailserver/hMailServer.ini` | | `HM_PROFILE`, `HM_PROFILE_CONFIRM` | nothing | Since 6.3.4: the setting profile applied once, and its confirmation | **On 6.3.3 and earlier** the entrypoint writes these into `hMailServer.ini` on every start, and every other setting is the packaged file in the `/etc/hmailserver` volume. **Since 6.3.4** the database is the settings store, so the four `HM_REST_*` variables are aliases of `HM_SETTING_RestApiPort` and its three neighbours, stored like any other setting; `[Database]` is still written into the file at every start, because it says where the database is. The image's `/etc/hmailserver` is owned by the `hmailserver` user, so there, unlike on a package installation, the server keeps its copy of each stored value in the file, and a `--create-database` that creates the schema also makes the stored-secrets key. ### Every setting from the environment (since 6.3.4) After `--create-database` and `--upgrade-database`, and before the server, the entrypoint runs the server's own verb, `hmailserver --apply-environment`, whenever there is anything for it to do. `HM_SETTING_` sets a `[Settings]` key and `HM_PROPERTY_` a database setting (`HM_SETTING_MaxSubmissionsPerIPPerMinute=30`, `HM_PROPERTY_maxmessagesize=20480`), and either with `_FILE` reads the value from a file, a Compose secret or a Kubernetes Secret mounted as one. - **The environment wins, at every start.** An administrator cannot change a setting the environment names until the variable is unset and the container restarted, and a ConfigMap change, too, is applied by a restart and not live. - **Everything is checked before anything is stored.** When anything is refused, nothing is stored and **the container does not start**; `docker compose logs` or `kubectl logs` says which variable and why. The naming rule, every refusal, what the REST API and the Control Deck show for a setting the environment manages, and how a value from a file stays secret are in [configuration from the environment](/topic/28231). **`HM_PROFILE`** names one of the four setting profiles, `office`, `gateway`, `relay` or `sender`, which the entrypoint applies once, with `--apply-profile NAME --once`, after the environment's settings and before the server starts. `HM_PROFILE=none` records that none is wanted. A profile that closes a listener, trusts an address range, turns a filter or a check off or weakens it, or shortens a window of days needs `HM_PROFILE_CONFIRM=yes`, and without it the container does not start. A profile skips a setting the environment manages, naming its variable. **Not yet run in a container.** The verb has been run as `hMailServer.exe --apply-environment` on Windows, and its POSIX half compiles; the entrypoint as a whole, the POSIX verb against a database, the chart and the Compose file have been read and not executed. `HM_PROFILE` is in the same state. Try it on a scratch database first. ### Reaching the API The server requires TLS on a REST listener that is not bound to loopback. A listener left at the packaged bind address answers only processes inside the container, which is none, since the image carries no client. Two ways out; either will do: - **Bind it and give it a certificate.** `HM_REST_BIND_ADDRESS=0.0.0.0`, with `HM_REST_CERTIFICATE_FILE` and `HM_REST_PRIVATE_KEY_FILE` pointing at a certificate and key mounted into the container, and the port published. Publish it on the host's loopback (`127.0.0.1:8045:8045`) unless the certificate is one clients will trust. - **Host networking** (`--network host`). The container's `127.0.0.1` is the host's, the packaged bind address is right as it is, no certificate is needed, and the API is at `http://127.0.0.1:8045` from the host and through an SSH tunnel from elsewhere. The mail ports are then the host's ports directly. ### First start, with Compose The file to use is in the repository, at `hmailserver/source/Server/platform/packaging/docker/docker-compose.yml`. It reads the two passwords as Compose secrets from files, mounts the REST certificate and key in, and carries the `HM_SETTING_` and `HM_PROPERTY_` examples commented out. A workflow resolved it with `docker compose config` on every change until 18 September 2026, so it is known to parse. Below is the `mail` service of the wiki's shorter example, which puts it under `services:` beside a `postgres:16` service named `db` and declares the named volumes. It names the image built above, not the empty registry: ```yaml mail: image: hmailserver:6.3.4 depends_on: - db environment: HM_DB_TYPE: PostgreSQL HM_DB_HOST: db HM_DB_NAME: hmailserver HM_DB_USER: hmailserver HM_DB_PASSWORD: change-me HM_ADMIN_PASSWORD: change-me-too HM_REST_PORT: 8045 HM_REST_BIND_ADDRESS: 0.0.0.0 HM_REST_CERTIFICATE_FILE: /certs/rest.pem HM_REST_PRIVATE_KEY_FILE: /certs/rest.key volumes: - ./certs:/certs:ro - hm-data:/var/lib/hmailserver - hm-logs:/var/log/hmailserver - hm-etc:/etc/hmailserver ports: - "25:25" - "587:587" - "993:993" - "127.0.0.1:8045:8045" restart: unless-stopped ``` `restart: unless-stopped` matters on the first start: `depends_on` starts the database container first but does not wait for it to accept connections, so the entrypoint's `--create-database` fails and the container exits if the database is not there yet; the restart tries again a moment later. The first start logs the `[Database]` section written, the administrator password hashed and the database created at schema 6104, and also what `--create-database` did about the stored-secrets key and one line per `HM_SETTING_` or `HM_PROPERTY_` variable stored. The first domain is then one request away ([First install on Linux](/topic/28190)). Passwords in a Compose file are passwords in a file: for anything beyond a first look, use `HM_DB_PASSWORD_FILE` and `HM_ADMIN_PASSWORD_FILE` with Compose secrets. **The secure defaults of a new installation are not applied in a container by themselves.** On a package installation the systemd unit's first start gives a database `--create-database` made a self-signed certificate, STARTTLS and TLS required for authentication on the *Internet* IP range. The image has no unit, so that step is run by hand, as the service's user, with the server stopped. [SecureDefaults.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SecureDefaults.md) gives the verb and no container command for it, and lists its refusal of a running service (exit 4) for Windows only: ```sh hmailserver --apply-new-install-defaults ``` A run by hand has no one-day deadline, so an image first booted days after its `--create-database` is secured this way. What it changes, its exit codes and the checklist item that says what is left are in [Hardening a public mail server](/topic/28241). ### Kubernetes, with the chart Since 6.3.4 the chart is at `hmailserver/source/Server/platform/packaging/helm/hmailserver`. It is not published to a chart repository; install it from a checkout. It renders a **StatefulSet of one**, because a message store is not something two pods may share: the three volumes the image declares become claim templates, and the database is pointed at, never bundled. The chart refuses to render without `database.host`. Its `image.repository` names the empty registry, so set it to the image you built, pushed to a registry the cluster's nodes can pull from: ``` helm install mail ./hmailserver \ --set image.repository=/hmailserver \ --set database.host=postgres.databases.svc.cluster.local \ --set database.password=... \ --set administrator.password=... \ --set rest.tlsSecretName=mail-tls ``` | Value | What it is | |---|---| | `image.repository` | `registry.gitlab.com/progressiverobot/hmailserver` as shipped, which holds no image: set it to where you pushed the image you built | | `database.host`, `.type`, `.name`, `.user` | The PostgreSQL or MariaDB the server uses; required. `database.create` and `database.upgrade` make the schema on first start and move it forward on an upgrade | | `database.password`, `administrator.password` | Go into a Secret the chart makes. Point `existingSecret` at a Secret you made and the chart writes none; its two keys are named by `existingSecretKeys` | | `rest.enabled`, `.port`, `.tlsSecretName` | The one listener that serves the REST API, the Control Deck and the webmail. Off loopback the server requires TLS, so without a `kubernetes.io/tls` Secret named here the listener will not start; the chart's notes say so on install | | `settings`, `properties`, `settingsFrom` | Since 6.3.4: any `[Settings]` key (rendered as `HM_SETTING_`), any database setting (`HM_PROPERTY_`), and a key of a Secret you made, projected into `/run/secrets/hmailserver-settings/` and named through `_FILE`, so a secret value is in no environment variable, rendered manifest or log line | | `service.type`, `.ports`, `.externalTrafficPolicy` | A LoadBalancer with the mail ports, and the REST port when the listener is on. `Local` keeps the client's address, which auto-ban, greylisting and the `Received` header all need | | `persistence.*` | The message store, the logs and the configuration, one claim each | | Probes | Port 25, as the image's own health check is, or the metrics listener's `/livez` and `/readyz` when that listener is on | `settings`, `properties` and `settingsFrom` look like this in a values file; a `settingsFrom` entry says `setting:` in place of `property:` for a `[Settings]` key: ```yaml settings: MaxSubmissionsPerIPPerMinute: "30" properties: maxmessagesize: "20480" settingsFrom: - secretName: mail-relay key: password property: smtprelayerpassword ``` A switch is `1` or `0` for a `[Settings]` key; `true`, `false`, yes, no, on and off are taken as well for a setting whose REST settings row is a switch, and only there: `maxmessagesize=off` is refused rather than stored as 0, which is "no limit". ### Volumes, ports and the health check | Path | What lives there | |---|---| | `/var/lib/hmailserver` | The mail store, and `.hmailserver-secret-key`, the key under which this machine's stored secrets are protected. The key travels with this directory: a database restored beside a fresh data volume cannot open the secrets it holds | | `/var/log/hmailserver` | The logs. Bind-mount it to a host path if fail2ban on the host is to read it | | `/etc/hmailserver` | `hMailServer.ini`; since 6.3.4 the only home of `PasswordPepper` and `[Security] StoredSecretsKey`, which no database and no backup holds | The image exposes 25, 110, 143, 465, 587, 993, 995, 4190 (ManageSieve) and 8045; publish the ones you serve. The health check opens a connection to the container's own port 25 every 30 seconds after a 60-second start period, and three failures in a row mark the container unhealthy. ### Upgrades, backups and auto-ban **Upgrading** is building the image of the new tag, as no 6.3.4 image is published to pull, and recreating the container: the entrypoint runs `--upgrade-database` on every start, so a schema that is behind is brought forward before the server opens a port. On MySQL and MariaDB the 6102 step rebuilds every table as `utf8mb4`, which takes time on a large database and needs the database server version that [Databases on Linux](/topic/28247) gives. An upgraded database gets no stored-secrets key until one is made with `hmailserver --create-stored-secrets-key`, and never the new installation's defaults. Take the backup first ([Upgrading to 6.3.4](/topic/28193)). **Backups** are three things together: the database, the `/var/lib/hmailserver` volume, and `/etc/hmailserver`. Two things are only in `/etc/hmailserver/hMailServer.ini`: a `PasswordPepper`, if one is set, without which a restored database refuses the password of every account hashed with Argon2id or scrypt, and the stored-secrets key, without which a restored database opens no account's second factor and no `[Settings]` secret. Export the key once, and keep the file with the database backups, apart from them: ```sh docker exec hmailserver --export-stored-secrets-key /var/lib/hmailserver/stored-secrets.key docker cp :/var/lib/hmailserver/stored-secrets.key ./stored-secrets.key ``` A replacement container takes it back with `--import-stored-secrets-key FILE`. **Auto-ban** works as everywhere: an address that fails to log on repeatedly is refused by the listeners. The hook that adds the address to the host's firewall cannot run from inside a container, so run fail2ban on the host instead: bind-mount `/var/log/hmailserver`, copy the filter and jail out of the image, point the jail's `logpath` at the mounted directory, and enable it. ### What was proven, and what has not been Until 18 September 2026 the `image` job of the GitHub workflow built the image on every release tag, pushed it, and smoke-tested the pushed image as the example above runs it: the container healthy, the SMTP banner and the API answering, and the administrator password in no file in clear. Nothing has run it since. GitLab CI's `linux-image` job is written to build the image from the `.deb` with kaniko and inspect it; it neither pushes nor runs the image, so the smoke test has no replacement yet. The `chart-and-compose` job is written to lint and render the chart and resolve the Compose file. Neither had run when [ContinuousIntegration.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/ContinuousIntegration.md) was written, on 25 September 2026. The changelog's 6.3.4 section records the first full pipeline the day after: it finished failed, the image job among those that passed, and no pipeline has run on the fixed tree since. 6.3.4 was made by hand. ### Where to read more - [Running in a Container](https://gitlab.com/hmailserver/hmailserver/-/wikis/Running-in-a-Container) and [Profiles and Environment](https://gitlab.com/hmailserver/hmailserver/-/wikis/Profiles-and-Environment) on the wiki. - [EnvironmentConfiguration.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/EnvironmentConfiguration.md) for the naming rule and every refusal, [Profiles.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Profiles.md) for every entry of the four profiles, [SecureDefaults.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SecureDefaults.md) for the verb run by hand, and [DatabaseTls.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/DatabaseTls.md) for `HM_DB_TLS_MODE`. - The 6.3.4 section of [CHANGELOG.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/CHANGELOG.md) for the state of the image. - On this forum: [the 6.3.4 announcement](/topic/28218), [hMailServer on Linux](/topic/28189), [First install on Linux](/topic/28190) and [Upgrading to 6.3.4](/topic/28193); [configuration from the environment](/topic/28231) for `HM_SETTING_` and `HM_PROPERTY_` in full, and [the hmailserver command line on Linux](/topic/28245) for the verbs the entrypoint runs. --- Canonical: https://www.hmailserver.co.uk/topic/28247/databases-on-linux-postgresql-or-mariadb-the-client-tls-and-the-upgrade-floor # Databases on Linux: PostgreSQL or MariaDB, the client, TLS and the upgrade floor *Written for 6.3.4. hMailServer 6.3.7 (17 October 2026), a security release, keeps the schema at 6108: an upgrade from 6.3.6 changes no table, from 6.3.5 or older the upgrade runs 6.3.6's three one-way steps to 6108, and from 6.3.3 or older 6.3.4's steps come first. On MySQL and MariaDB an insert that forgives a duplicate key now forgives that alone and any other failure is reported (`HM5032`), where it was taken as done with no row written, and a lost connection is reconnected and the statement run again, so the statement after a database restart, a failover or `wait_timeout` no longer fails; an error log that was quiet may now name failures that were hidden. See [the 6.3.7 announcement](/topic/28277).* A Linux server keeps its accounts, domains, settings and message metadata in PostgreSQL or in MySQL/MariaDB, and in nothing else. There is no built-in database on Linux, `hmailserver --create-database` refuses SQL Server and SQL Server Compact by name, and SQL Server is refused on Linux as `HM6390`, because ADO is Windows. This topic covers the choice between the two, the client library each needs, the `[Database]` section, each backend's set-up step by step, TLS to the database, and what the schema upgrade asks of MySQL and MariaDB. The one thing to know first: since 6.3.4 every table on MySQL and MariaDB is `utf8mb4`, which needs **MySQL 5.7.9 or MariaDB 10.2.2, or later**. On an older server the create script stops at its first table, and an upgrade from 6.3.3 stops at its first step, 6040 to 6041, with error 1071, having changed nothing. Check the version before anything else. The first install as a whole is [First install on Linux](/topic/28190). ### Choosing between the two | | PostgreSQL | MySQL / MariaDB | |---|---|---| | `[Database] Type` | `PostgreSQL` | `MySQL` (MariaDB is `MySQL`) | | `Port` | `5432` | `3306` | | Proven on | PostgreSQL 18 for the first Linux proof; PostgreSQL 17 in the TLS lab | MySQL 5.7.44 and 8.4.11, MariaDB 10.6.28, 11.4.13 and 11.8.9, on 25 September 2026 | | Client library | `libpq`, pulled in by the package's own dependencies | `libmariadb.so.3`, opened at run time, installed by you | | Upgrade steps that ship | every step from `5001` onwards | every step, back to schema `0` | | Transactions | DDL is transactional, so a failed schema step rolls back cleanly | used only if every table reports InnoDB at connect time | | TLS with nothing set | libpq's own `prefer` | `prefer` since 6.3.4; never encrypted through MariaDB Connector/C before | [MigratingDatabaseBackend.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/MigratingDatabaseBackend.md) calls PostgreSQL the best-behaved backend in this codebase, and gives "you already run it" as the reason for MySQL or MariaDB. Both are proven live on Linux: schema created, account made, message delivered and read back. That the PostgreSQL steps begin at `5001` is not a gap, because PostgreSQL was not a backend before hMailServer 5. Moving between the two later is not an upgrade, and the chain does not do it: it is a new installation against the new backend and a migration. ### The client library to install The `.deb` and the `.rpm` declare their library dependencies automatically (`dpkg-shlibdeps` for the `.deb`, rpm's own auto-requires), so `libpq` is pulled in with the package. MySQL and MariaDB are different: the server opens `libmariadb.so.3` at run time rather than linking it, so the wiki tells you to install the client library yourself. | Distribution | Package | |---|---| | Debian, Ubuntu | `libmariadb3` | | Fedora, RHEL | `mariadb-connector-c` | | Arch | `mariadb-libs` | Where there is no `libmariadb`, the server loads Oracle's `libmysqlclient`. Only Oracle's 8.x client is given TLS options: with any other, `require` and `verify-full` are refused with the version it reports, rather than claimed. Oracle's `libmysqlclient.so.24`, MySQL 8.4's client as Ubuntu 26.04 packages it, is not among the names the Linux server loads. On Windows the client is bundled, as `libmysql.dll`; it is the same MariaDB Connector/C: 3.4.9 in the x64 installers of 6.2.3 to 6.3.6, and 3.4.11 in 6.3.7's, built from MariaDB's source, with seven authentication plugins where 6.3.6's had nine. The Linux server loads the distribution's `libmariadb.so.3`, which the distribution patches. ### The `[Database]` section The file is root's, so edit it with `sudoedit /etc/hmailserver/hMailServer.ini`. | Key | What to write | |---|---| | `Type` | `PostgreSQL` or `MySQL` | | `Server`, `Database`, `Username`, `Password` | Where the database is and who the server connects as. The Linux package's template reads a password typed with `Passwordencryption=0` as plain text | | `Port` | Written out, `5432` or `3306`: it goes into the connection string as it stands, and there is no useful default | | `TlsMode`, `TlsCaFile` | Since 6.3.4, optional (below) | | `UpgradeOnStart` | Since 6.3.4: `1` lets the service bring its own schema forward as it starts. Off by default, and not needed on a package installation, whose post-install step runs the upgrade | | `NumberOfConnections` | The pool that deliveries, sign-ins and the rest share; five as shipped | These keys are read at the service's start, and are in the file rather than the settings store because they say how the store is reached. `sudo -u hmailserver hmailserver --check-config` prints the database type it made of what you wrote, without opening the database; `Database type: 0` means the `Type` key did not take. ### PostgreSQL, step by step 1. **Create the role.** With `--createdb` the server creates the database itself in step 3: ```sh sudo -u postgres createuser --pwprompt --createdb hmailserver ``` If you would rather the service role could not create databases, which the wiki calls the tidier policy, leave the flag off and create the database yourself: ```sh sudo -u postgres createuser --pwprompt hmailserver sudo -u postgres createdb --owner hmailserver hmailserver ``` 2. **Write `[Database]`**: `Type=PostgreSQL`, the server, the database, the role, its password, and `Port=5432`. 3. **Create the schema**, as the service user: ```sh sudo -u hmailserver hmailserver --create-database ``` 4. **If TLS is to be required**, that is the database server's to say: `ssl=on` and a certificate in `postgresql.conf`, and `hostssl` lines in `pg_hba.conf`. ### MySQL or MariaDB, step by step 1. **Check the version** against the floor. The reason is key size: a key over a `varchar(255)`, an account's address or a folder's name, is 1020 bytes at four bytes a character, more than an InnoDB key part may be in the older COMPACT rows (767 bytes). MySQL 5.7.9 and MariaDB 10.2.2 are the first releases whose InnoDB, as it ships, takes a key part of up to 3072 bytes and makes its tables DYNAMIC. On a MySQL 5.7 set up as 5.6 shipped, the error is 1071, "Specified key was too long; max key length is 767 bytes". 2. **Install the client library** from the table above. 3. **Create the database**, or let the verb create it, and give the account the server connects with every privilege on it: ```sql CREATE DATABASE hmailserver CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci; GRANT ALL PRIVILEGES ON hmailserver.* TO 'hmailserver'@'%'; ``` Every table names its own character set, so a database made with another default still works. But the upgrade steps create, alter and index tables, and the 6102 step also changes the database's own default character set, which needs the ALTER privilege on the database itself; `ALL PRIVILEGES ON .*` has it. A database the server creates itself is made `utf8mb4` already. Make sure every table ends up InnoDB: the server issues transactions only if they all report it at connect time, and silently does not if any does not. 4. **Write `[Database]`**: `Type=MySQL`, the server, `Port=3306`, the database, the account and its password. 5. **Create the schema**: `sudo -u hmailserver hmailserver --create-database`. **Why `utf8mb4`.** MySQL's `utf8` is `utf8mb3`, three bytes a character, so nothing outside the Basic Multilingual Plane fits in it: no emoji. Before schema 6102 such a character in a contact, a folder or a signature was refused with error 1366 under strict mode, and made a question mark where strict mode is off. `utf8mb4_general_ci` is the twin of `utf8_general_ci`, the collation the tables had, so an address or a name compares as it did. What it does not do is tell characters outside the Basic Multilingual Plane apart: two address books of one account whose names differ only in an emoji are one name to its unique index. ### What `--create-database` does The package installs the server enabled and stopped, on purpose: there is no database yet, and the schema is made afterwards with this verb. If the database is not there, the verb connects to the server `[Database]` names with no database selected, creates it in the backend's own dialect and reconnects; if it is there and empty, it uses it. Either way it then runs the create script for the backend's type, which is what `DBSetupQuick` does over COM on Windows, and a database 6.3.4 creates is stamped 6104. Since 6.3.4, says the wiki's Installing on Linux, a database that already holds this build's schema, or an older one, is left as it is with exit 0, and a newer one exits 1; 6.3.3 refused any database that already held a schema. The documentation page (35.4, 35.7) still says a schema is refused, exit 1: test it on a scratch database first. Two things follow a create, both since 6.3.4. The database, an empty one you created first included, is marked for a new installation's secure defaults, which the systemd unit's first start applies if it comes within a day; a database that already held a schema is not marked. And run as the service user on a package installation, the verb cannot write the root-owned configuration, so it makes no stored-secrets key and says what to run instead, `sudo hmailserver --create-stored-secrets-key`. Export that key and keep the file with the database dumps, apart from them: a dump restored without it brings back every second factor and `[Settings]` secret unopenable. Both are covered in full in [Hardening a public mail server](/topic/28241). ### TLS to the database (since 6.3.4) Before 6.3.4 the connection to MySQL and MariaDB through MariaDB Connector/C, the client Ubuntu packages, was never encrypted, and a database account made with `REQUIRE SSL` refused the server. Since 6.3.4 two `[Database]` keys choose the behaviour on every backend (on PostgreSQL the older `PostgreSQLSslMode` and `PostgreSQLSslRootCert` win when they are set), and are read at the service's start: `TlsMode`, one of `disable`, `prefer`, `require` or `verify-full` (empty is `prefer`: TLS when the server offers it, the certificate unchecked), and `TlsCaFile`, a PEM file of CA certificates that `verify-full` checks against: ``` [Database] TlsMode=verify-full TlsCaFile=/etc/hmailserver/db-ca.pem ``` Only `verify-full` checks the certificate, and only it protects against someone on the path who can alter the traffic. On Linux, **MySQL and MariaDB** take the CA file or the system's store, `/etc/ssl/certs`, and which client library is loaded matters: of Oracle's clients only 8.x is given TLS options (above). On **PostgreSQL**, `verify-full` with no `TlsCaFile` is `sslrootcert=system` on Linux, with libpq 16 and later. What each mode does, the refusals (`HM7530`, `HM7531`), what each backend did before 6.3.4 and does now, and what is thinner than it could be are in [Choosing a database, creating it, encrypting the link](/topic/28228). The server cannot make the database server refuse plaintext: that is `REQUIRE SSL` on the MySQL account, or `hostssl` lines only in `pg_hba.conf`. **Try it before the service is given it.** `--check-database` makes one connection with the TLS settings in the configuration, prints the result, and exits 0 connected, 1 not, 2 not configured. It changes nothing, so it runs beside a running service: ``` $ sudo -u hmailserver hmailserver --check-database Configuration: /etc/hmailserver/hMailServer.ini Database: MySQL/MariaDB on 192.0.2.20:3306, database hmailserver, as hmailserver TLS: TlsMode verify-full, the certificate checked against /etc/hmailserver/db-ca.pem Connected. MySQL/MariaDB: encrypted with TLS (TLS_AES_256_GCM_SHA384), the server's certificate checked (TlsMode verify-full) ``` The application log says the same of the running service's connection, in a line that begins `Database connection:`. In [a container](/topic/28246) (no image of 6.3.4 is published, so one you build from the `.deb`) the entrypoint writes `[Database]` from the environment at every start, so there `HM_DB_TLS_MODE` and `HM_DB_TLS_CA` set these, not a line in the file. ### Upgrading the schema, and the floor On a package upgrade the maintainer scripts run `hmailserver --upgrade-database` as the service user, only when a database is configured, then `try-restart` the service if it was running. A failure of the schema upgrade is reported and does not fail the package, so read what it printed. The verb is safe to run by hand, and says "nothing to do" when done: ```sh sudo -u hmailserver hmailserver --upgrade-database ``` The chain is applied one step at a time from whatever version the database reports, which `select value from hm_dbversion` answers. A database from a newer build is refused, and a server refuses to start on a database that is too old or too new for it. Take the dump first, with `pg_dump` or `mysqldump`, and the data directory with it: the upgrade is one-way, and rolling back is restoring that dump and reinstalling the older package ([Upgrading from 5.x, 6.2 or 6.3: rolling back](/topic/28235)). **From 6.3.3 to 6.3.4** the schema moves from 6040 to 6104 in 64 steps. On PostgreSQL step 6102 only moves the version. On MySQL and MariaDB it converts every table to `utf8mb4` under `utf8mb4_general_ci`, InnoDB and `ROW_FORMAT=DYNAMIC`: | What | Detail | |---|---| | Time and disk | It takes as long as copying the database. Each table is rebuilt in turn, and `hm_messages`, `hm_message_metadata`, `hm_messageindexterms`, `hm_messagerecipients` and `hm_audit` take the time: one to five seconds on an empty schema, minutes or more on a large installation. Have free disk space of the order of the largest table | | The floor, error 1071 | Steps 6041, 6101 and 6102 each open with a statement that needs a long key; 6041 and 6101 make and drop a probe table, `hm_upgradeprobe`. On a server older than MySQL 5.7.9 or MariaDB 10.2.2 the upgrade stops at 6040 to 6041 with error 1071, and nothing has been changed. Upgrade the database server, then the schema | | The privilege, error 1044 | The step changes the database's default character set, which needs ALTER on the database itself, not only on its tables | | A duplicate, error 1062 | Two rows that the old collation told apart and the new one does not, under one unique index, stop the step; the error names the index and the value. The wiki's examples are `straße` and `strase` in one address book on MariaDB 11.8, and on MySQL 8.4 two names that differ only in an emoji. Rename or remove one and run the upgrade again: every statement of the step can run twice | | Afterwards | A `text` column may read `mediumtext`, and a `mediumtext` one `longtext`. The server reads both alike | Until 26 September 2026 steps 6041 and 6101 made their first tables before the index that failed, so an unreleased build from before that day could leave a database half done at 6040 or 6100. No upgrade path repairs that; [Upgrading.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Upgrading.md) gives the `DROP TABLE` statements that do. **Two nodes on one database** are serialised step by step if they start together: on PostgreSQL by the version row, which each step locks for the length of its transaction, and on MySQL and MariaDB by a named lock, `GET_LOCK('hmailserver_schema_upgrade')`, held until the step is over and waited on for up to four hours. A server that refuses `GET_LOCK` altogether, MariaDB Galera with wsrep on, still upgrades with the row lock alone, and the output says so for each step: there, upgrade from one node at a time. ### Where to read more - [Installing on Linux](https://gitlab.com/hmailserver/hmailserver/-/wikis/Installing-on-Linux) and [Before You Install](https://gitlab.com/hmailserver/hmailserver/-/wikis/Before-You-Install), section 3.5, on the wiki. - [DatabaseTls.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/DatabaseTls.md), with the Linux lab run of 25 September 2026 line by line; [Upgrading.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Upgrading.md) for the 6102 step and the repair of a half-done one; [MigratingDatabaseBackend.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/MigratingDatabaseBackend.md) for why `utf8mb4`. - The documentation page: [3.5 A database](https://www.progressiverobot.com/hmailserver-documentation/#35-a-database), [35.4 Creating the database](https://www.progressiverobot.com/hmailserver-documentation/#354-creating-the-database) and [35.13 Upgrading and rollback](https://www.progressiverobot.com/hmailserver-documentation/#3513-upgrading-and-rollback). - On this forum: [First install on Linux](/topic/28190), [Upgrading to 6.3.4](/topic/28193) and [Moving to new hardware](/topic/28194); [choosing a database and encrypting the link](/topic/28228) for TLS on every backend, and [the hmailserver command line on Linux](/topic/28245) for the other verbs. --- Canonical: https://www.hmailserver.co.uk/topic/28247/databases-on-linux-postgresql-or-mariadb-the-client-tls-and-the-upgrade-floor # Databases on Linux: PostgreSQL or MariaDB, the client, TLS and the upgrade floor *Written for 6.3.4. hMailServer 6.3.7 (17 October 2026), a security release, keeps the schema at 6108: an upgrade from 6.3.6 changes no table, from 6.3.5 or older the upgrade runs 6.3.6's three one-way steps to 6108, and from 6.3.3 or older 6.3.4's steps come first. On MySQL and MariaDB an insert that forgives a duplicate key now forgives that alone and any other failure is reported (`HM5032`), where it was taken as done with no row written, and a lost connection is reconnected and the statement run again, so the statement after a database restart, a failover or `wait_timeout` no longer fails; an error log that was quiet may now name failures that were hidden. See [the 6.3.7 announcement](/topic/28277).* A Linux server keeps its accounts, domains, settings and message metadata in PostgreSQL or in MySQL/MariaDB, and in nothing else. There is no built-in database on Linux, `hmailserver --create-database` refuses SQL Server and SQL Server Compact by name, and SQL Server is refused on Linux as `HM6390`, because ADO is Windows. This topic covers the choice between the two, the client library each needs, the `[Database]` section, each backend's set-up step by step, TLS to the database, and what the schema upgrade asks of MySQL and MariaDB. The one thing to know first: since 6.3.4 every table on MySQL and MariaDB is `utf8mb4`, which needs **MySQL 5.7.9 or MariaDB 10.2.2, or later**. On an older server the create script stops at its first table, and an upgrade from 6.3.3 stops at its first step, 6040 to 6041, with error 1071, having changed nothing. Check the version before anything else. The first install as a whole is [First install on Linux](/topic/28190). ### Choosing between the two | | PostgreSQL | MySQL / MariaDB | |---|---|---| | `[Database] Type` | `PostgreSQL` | `MySQL` (MariaDB is `MySQL`) | | `Port` | `5432` | `3306` | | Proven on | PostgreSQL 18 for the first Linux proof; PostgreSQL 17 in the TLS lab | MySQL 5.7.44 and 8.4.11, MariaDB 10.6.28, 11.4.13 and 11.8.9, on 25 September 2026 | | Client library | `libpq`, pulled in by the package's own dependencies | `libmariadb.so.3`, opened at run time, installed by you | | Upgrade steps that ship | every step from `5001` onwards | every step, back to schema `0` | | Transactions | DDL is transactional, so a failed schema step rolls back cleanly | used only if every table reports InnoDB at connect time | | TLS with nothing set | libpq's own `prefer` | `prefer` since 6.3.4; never encrypted through MariaDB Connector/C before | [MigratingDatabaseBackend.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/MigratingDatabaseBackend.md) calls PostgreSQL the best-behaved backend in this codebase, and gives "you already run it" as the reason for MySQL or MariaDB. Both are proven live on Linux: schema created, account made, message delivered and read back. That the PostgreSQL steps begin at `5001` is not a gap, because PostgreSQL was not a backend before hMailServer 5. Moving between the two later is not an upgrade, and the chain does not do it: it is a new installation against the new backend and a migration. ### The client library to install The `.deb` and the `.rpm` declare their library dependencies automatically (`dpkg-shlibdeps` for the `.deb`, rpm's own auto-requires), so `libpq` is pulled in with the package. MySQL and MariaDB are different: the server opens `libmariadb.so.3` at run time rather than linking it, so the wiki tells you to install the client library yourself. | Distribution | Package | |---|---| | Debian, Ubuntu | `libmariadb3` | | Fedora, RHEL | `mariadb-connector-c` | | Arch | `mariadb-libs` | Where there is no `libmariadb`, the server loads Oracle's `libmysqlclient`. Only Oracle's 8.x client is given TLS options: with any other, `require` and `verify-full` are refused with the version it reports, rather than claimed. Oracle's `libmysqlclient.so.24`, MySQL 8.4's client as Ubuntu 26.04 packages it, is not among the names the Linux server loads. On Windows the client is bundled, as `libmysql.dll`; it is the same MariaDB Connector/C: 3.4.9 in the x64 installers of 6.2.3 to 6.3.6, and 3.4.11 in 6.3.7's, built from MariaDB's source, with seven authentication plugins where 6.3.6's had nine. The Linux server loads the distribution's `libmariadb.so.3`, which the distribution patches. ### The `[Database]` section The file is root's, so edit it with `sudoedit /etc/hmailserver/hMailServer.ini`. | Key | What to write | |---|---| | `Type` | `PostgreSQL` or `MySQL` | | `Server`, `Database`, `Username`, `Password` | Where the database is and who the server connects as. The Linux package's template reads a password typed with `Passwordencryption=0` as plain text | | `Port` | Written out, `5432` or `3306`: it goes into the connection string as it stands, and there is no useful default | | `TlsMode`, `TlsCaFile` | Since 6.3.4, optional (below) | | `UpgradeOnStart` | Since 6.3.4: `1` lets the service bring its own schema forward as it starts. Off by default, and not needed on a package installation, whose post-install step runs the upgrade | | `NumberOfConnections` | The pool that deliveries, sign-ins and the rest share; five as shipped | These keys are read at the service's start, and are in the file rather than the settings store because they say how the store is reached. `sudo -u hmailserver hmailserver --check-config` prints the database type it made of what you wrote, without opening the database; `Database type: 0` means the `Type` key did not take. ### PostgreSQL, step by step 1. **Create the role.** With `--createdb` the server creates the database itself in step 3: ```sh sudo -u postgres createuser --pwprompt --createdb hmailserver ``` If you would rather the service role could not create databases, which the wiki calls the tidier policy, leave the flag off and create the database yourself: ```sh sudo -u postgres createuser --pwprompt hmailserver sudo -u postgres createdb --owner hmailserver hmailserver ``` 2. **Write `[Database]`**: `Type=PostgreSQL`, the server, the database, the role, its password, and `Port=5432`. 3. **Create the schema**, as the service user: ```sh sudo -u hmailserver hmailserver --create-database ``` 4. **If TLS is to be required**, that is the database server's to say: `ssl=on` and a certificate in `postgresql.conf`, and `hostssl` lines in `pg_hba.conf`. ### MySQL or MariaDB, step by step 1. **Check the version** against the floor. The reason is key size: a key over a `varchar(255)`, an account's address or a folder's name, is 1020 bytes at four bytes a character, more than an InnoDB key part may be in the older COMPACT rows (767 bytes). MySQL 5.7.9 and MariaDB 10.2.2 are the first releases whose InnoDB, as it ships, takes a key part of up to 3072 bytes and makes its tables DYNAMIC. On a MySQL 5.7 set up as 5.6 shipped, the error is 1071, "Specified key was too long; max key length is 767 bytes". 2. **Install the client library** from the table above. 3. **Create the database**, or let the verb create it, and give the account the server connects with every privilege on it: ```sql CREATE DATABASE hmailserver CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci; GRANT ALL PRIVILEGES ON hmailserver.* TO 'hmailserver'@'%'; ``` Every table names its own character set, so a database made with another default still works. But the upgrade steps create, alter and index tables, and the 6102 step also changes the database's own default character set, which needs the ALTER privilege on the database itself; `ALL PRIVILEGES ON .*` has it. A database the server creates itself is made `utf8mb4` already. Make sure every table ends up InnoDB: the server issues transactions only if they all report it at connect time, and silently does not if any does not. 4. **Write `[Database]`**: `Type=MySQL`, the server, `Port=3306`, the database, the account and its password. 5. **Create the schema**: `sudo -u hmailserver hmailserver --create-database`. **Why `utf8mb4`.** MySQL's `utf8` is `utf8mb3`, three bytes a character, so nothing outside the Basic Multilingual Plane fits in it: no emoji. Before schema 6102 such a character in a contact, a folder or a signature was refused with error 1366 under strict mode, and made a question mark where strict mode is off. `utf8mb4_general_ci` is the twin of `utf8_general_ci`, the collation the tables had, so an address or a name compares as it did. What it does not do is tell characters outside the Basic Multilingual Plane apart: two address books of one account whose names differ only in an emoji are one name to its unique index. ### What `--create-database` does The package installs the server enabled and stopped, on purpose: there is no database yet, and the schema is made afterwards with this verb. If the database is not there, the verb connects to the server `[Database]` names with no database selected, creates it in the backend's own dialect and reconnects; if it is there and empty, it uses it. Either way it then runs the create script for the backend's type, which is what `DBSetupQuick` does over COM on Windows, and a database 6.3.4 creates is stamped 6104. Since 6.3.4, says the wiki's Installing on Linux, a database that already holds this build's schema, or an older one, is left as it is with exit 0, and a newer one exits 1; 6.3.3 refused any database that already held a schema. The documentation page (35.4, 35.7) still says a schema is refused, exit 1: test it on a scratch database first. Two things follow a create, both since 6.3.4. The database, an empty one you created first included, is marked for a new installation's secure defaults, which the systemd unit's first start applies if it comes within a day; a database that already held a schema is not marked. And run as the service user on a package installation, the verb cannot write the root-owned configuration, so it makes no stored-secrets key and says what to run instead, `sudo hmailserver --create-stored-secrets-key`. Export that key and keep the file with the database dumps, apart from them: a dump restored without it brings back every second factor and `[Settings]` secret unopenable. Both are covered in full in [Hardening a public mail server](/topic/28241). ### TLS to the database (since 6.3.4) Before 6.3.4 the connection to MySQL and MariaDB through MariaDB Connector/C, the client Ubuntu packages, was never encrypted, and a database account made with `REQUIRE SSL` refused the server. Since 6.3.4 two `[Database]` keys choose the behaviour on every backend (on PostgreSQL the older `PostgreSQLSslMode` and `PostgreSQLSslRootCert` win when they are set), and are read at the service's start: `TlsMode`, one of `disable`, `prefer`, `require` or `verify-full` (empty is `prefer`: TLS when the server offers it, the certificate unchecked), and `TlsCaFile`, a PEM file of CA certificates that `verify-full` checks against: ``` [Database] TlsMode=verify-full TlsCaFile=/etc/hmailserver/db-ca.pem ``` Only `verify-full` checks the certificate, and only it protects against someone on the path who can alter the traffic. On Linux, **MySQL and MariaDB** take the CA file or the system's store, `/etc/ssl/certs`, and which client library is loaded matters: of Oracle's clients only 8.x is given TLS options (above). On **PostgreSQL**, `verify-full` with no `TlsCaFile` is `sslrootcert=system` on Linux, with libpq 16 and later. What each mode does, the refusals (`HM7530`, `HM7531`), what each backend did before 6.3.4 and does now, and what is thinner than it could be are in [Choosing a database, creating it, encrypting the link](/topic/28228). The server cannot make the database server refuse plaintext: that is `REQUIRE SSL` on the MySQL account, or `hostssl` lines only in `pg_hba.conf`. **Try it before the service is given it.** `--check-database` makes one connection with the TLS settings in the configuration, prints the result, and exits 0 connected, 1 not, 2 not configured. It changes nothing, so it runs beside a running service: ``` $ sudo -u hmailserver hmailserver --check-database Configuration: /etc/hmailserver/hMailServer.ini Database: MySQL/MariaDB on 192.0.2.20:3306, database hmailserver, as hmailserver TLS: TlsMode verify-full, the certificate checked against /etc/hmailserver/db-ca.pem Connected. MySQL/MariaDB: encrypted with TLS (TLS_AES_256_GCM_SHA384), the server's certificate checked (TlsMode verify-full) ``` The application log says the same of the running service's connection, in a line that begins `Database connection:`. In [a container](/topic/28246) (no image of 6.3.4 is published, so one you build from the `.deb`) the entrypoint writes `[Database]` from the environment at every start, so there `HM_DB_TLS_MODE` and `HM_DB_TLS_CA` set these, not a line in the file. ### Upgrading the schema, and the floor On a package upgrade the maintainer scripts run `hmailserver --upgrade-database` as the service user, only when a database is configured, then `try-restart` the service if it was running. A failure of the schema upgrade is reported and does not fail the package, so read what it printed. The verb is safe to run by hand, and says "nothing to do" when done: ```sh sudo -u hmailserver hmailserver --upgrade-database ``` The chain is applied one step at a time from whatever version the database reports, which `select value from hm_dbversion` answers. A database from a newer build is refused, and a server refuses to start on a database that is too old or too new for it. Take the dump first, with `pg_dump` or `mysqldump`, and the data directory with it: the upgrade is one-way, and rolling back is restoring that dump and reinstalling the older package ([Upgrading from 5.x, 6.2 or 6.3: rolling back](/topic/28235)). **From 6.3.3 to 6.3.4** the schema moves from 6040 to 6104 in 64 steps. On PostgreSQL step 6102 only moves the version. On MySQL and MariaDB it converts every table to `utf8mb4` under `utf8mb4_general_ci`, InnoDB and `ROW_FORMAT=DYNAMIC`: | What | Detail | |---|---| | Time and disk | It takes as long as copying the database. Each table is rebuilt in turn, and `hm_messages`, `hm_message_metadata`, `hm_messageindexterms`, `hm_messagerecipients` and `hm_audit` take the time: one to five seconds on an empty schema, minutes or more on a large installation. Have free disk space of the order of the largest table | | The floor, error 1071 | Steps 6041, 6101 and 6102 each open with a statement that needs a long key; 6041 and 6101 make and drop a probe table, `hm_upgradeprobe`. On a server older than MySQL 5.7.9 or MariaDB 10.2.2 the upgrade stops at 6040 to 6041 with error 1071, and nothing has been changed. Upgrade the database server, then the schema | | The privilege, error 1044 | The step changes the database's default character set, which needs ALTER on the database itself, not only on its tables | | A duplicate, error 1062 | Two rows that the old collation told apart and the new one does not, under one unique index, stop the step; the error names the index and the value. The wiki's examples are `straße` and `strase` in one address book on MariaDB 11.8, and on MySQL 8.4 two names that differ only in an emoji. Rename or remove one and run the upgrade again: every statement of the step can run twice | | Afterwards | A `text` column may read `mediumtext`, and a `mediumtext` one `longtext`. The server reads both alike | Until 26 September 2026 steps 6041 and 6101 made their first tables before the index that failed, so an unreleased build from before that day could leave a database half done at 6040 or 6100. No upgrade path repairs that; [Upgrading.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Upgrading.md) gives the `DROP TABLE` statements that do. **Two nodes on one database** are serialised step by step if they start together: on PostgreSQL by the version row, which each step locks for the length of its transaction, and on MySQL and MariaDB by a named lock, `GET_LOCK('hmailserver_schema_upgrade')`, held until the step is over and waited on for up to four hours. A server that refuses `GET_LOCK` altogether, MariaDB Galera with wsrep on, still upgrades with the row lock alone, and the output says so for each step: there, upgrade from one node at a time. ### Where to read more - [Installing on Linux](https://gitlab.com/hmailserver/hmailserver/-/wikis/Installing-on-Linux) and [Before You Install](https://gitlab.com/hmailserver/hmailserver/-/wikis/Before-You-Install), section 3.5, on the wiki. - [DatabaseTls.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/DatabaseTls.md), with the Linux lab run of 25 September 2026 line by line; [Upgrading.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Upgrading.md) for the 6102 step and the repair of a half-done one; [MigratingDatabaseBackend.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/MigratingDatabaseBackend.md) for why `utf8mb4`. - The documentation page: [3.5 A database](https://www.progressiverobot.com/hmailserver-documentation/#35-a-database), [35.4 Creating the database](https://www.progressiverobot.com/hmailserver-documentation/#354-creating-the-database) and [35.13 Upgrading and rollback](https://www.progressiverobot.com/hmailserver-documentation/#3513-upgrading-and-rollback). - On this forum: [First install on Linux](/topic/28190), [Upgrading to 6.3.4](/topic/28193) and [Moving to new hardware](/topic/28194); [choosing a database and encrypting the link](/topic/28228) for TLS on every backend, and [the hmailserver command line on Linux](/topic/28245) for the other verbs. --- Canonical: https://www.hmailserver.co.uk/topic/28248/the-anti-spam-stack-end-to-end-score-greylisting-spf-dkim-dmarc-dnsbl-surbl # The anti-spam stack end to end: score, greylisting, SPF, DKIM, DMARC, DNSBL, SURBL *Written for 6.3.4. hMailServer 6.3.7 (17 October 2026) reads a DNS lookup that fails as a failure, not as a name with no addresses: the HELO and PTR tests no longer score a name they could not look up, a DNS blacklist and SURBL no longer read a failed lookup as "not listed", and a DKIM selector that is a CNAME whose query failed is `TempFail`, not `PermFail`. SURBL reads a link's host that is not ASCII by UTS 46 and an HTML part's character references as a browser does, and SpamAssassin is sent a collected message's own `Return-Path`, so its SPF and envelope rules may score such mail differently. The DMARC check's pass now also decides whether a forward of mail from outside whose `From` names a hosted domain's organisation is given an envelope sender in that organisation, and no pass is recorded for a message the check did not judge: DMARC off, a sender on the white list or an IP range with spam protection off, a message over `AntiSpam.MaximumMessageSize`, or one the tests before it had already scored to the higher threshold. In the Control Panel the sidebar has a Spam filtering group, with its six allow and block lists under Lists, and a Virus scanning group, and Ctrl+K takes every old title to its page. See [the 6.3.7 announcement](/topic/28277).* This topic walks through every layer hMailServer 6.3.4 puts between an SMTP session and a mailbox: the two thresholds, the twelve tests in the order they run, greylisting and its 6.3.4 hardening, the DNS and URL blocklists, SPF, DKIM, DMARC and ARC, SpamAssassin, and the quarantine. The one thing to know first: none of these tests is a verdict on its own. Each failing test adds a score, two thresholds decide what the total means, and three kinds of session are never tested at all: an authenticated session, a sender or IP on the anti-spam white list, and any session from an IP range with spam protection switched off. ### Where the decisions are made Everything happens in one of two moments: before the body (at `MAIL FROM` or `RCPT TO`) and after the body (after `DATA` or the last `BDAT`). The pre-transmission tests run at `MAIL FROM` by default: `DNSBLChecksAfterMailFrom` defaults to `1`, and `0` moves them to the first `RCPT TO`, which leaks less to an address-probing client but accepts more of the conversation first. A session from [a configured incoming relay](/topic/28240) has both phases deferred to after `DATA`. **Who is never tested.** `SMTPConnection::GetDoSpamProtection_` returns false the moment the session is authenticated, so your own users' outbound mail is never scored; [the sending limits and rate limits](/topic/28239) are the controls for that. A sender or IP on the anti-spam **White list** skips the tests entirely, not merely the scoring: SPF is not evaluated, no `Authentication-Results` verdict is recorded and no reason header is written. **The read-only overview.** **Spam & virus filtering → Spam filtering overview** in the Control Panel (Windows only) shows on one screen which checks are on, what each scores, what the thresholds then do, and which checks are switched on but cannot run. ### The score and the two thresholds Both thresholds are on **Spam & virus filtering → Anti-spam settings → General**. | Setting | Default | At or above it | `0` means | |---|---|---|---| | `AntiSpam.SpamMarkThreshold` | 5 | Delivered but tagged: `X-hMailServer-Spam: YES`, `X-hMailServer-Reason-*` headers and, optionally, a subject prefix | Never mark | | `AntiSpam.SpamDeleteThreshold` | 20 | Refused during the conversation, or quarantined if quarantine is on and the verdict came after the body | Never refuse | | `AntiSpam.MaximumMessageSize` | 1024 KB | Post-transmission tests are skipped for a larger message | No limit of its own; the MIME parser's 80 MB ceiling still applies | The higher of the two thresholds is also the early-out: `SpamTestRunner::RunSpamTest` stops calling tests once the running total reaches it, which is why a message condemned by the blocked-senders list never pays for a DNS lookup. With both thresholds at 0 the early-out is skipped, because greylisting's SPF bypass needs the SPF result and SPF is the sixth test. With `AddHeaderSpam` and `AddHeaderReason` both on (the defaults), a marked message carries: ```text X-hMailServer-Spam: YES X-hMailServer-Reason-1: Blocked by SPF. (example.com does not designate 203.0.113.9) - (Score: 3) X-hMailServer-Reason-2: Rejected by Spamhaus. - (Score: 3) X-hMailServer-Reason-Score: 6 Subject: [SPAM] Your invoice ``` Only failing tests get a `Reason-N` line. `X-hMailServer-Reason-Score` is the sum of the failing tests, and with `AddHeaderReason` off it is not written at all. The subject prefix is off by default; its text is `[SPAM]` (`antispamprependsubjecttext`). **Start by tagging, not deleting.** The wiki's advice is a fortnight with the delete threshold at `0`, reading what got tagged, before turning deletion on well above anything legitimate has scored. ### Every test, in the order it runs This is the registration order in `SpamTestRunner::LoadSpamTests`; nothing reorders it at runtime. Tests 1 to 6 run before the body, 7 to 12 after it. | # | Test | Switched on by | Default | Default score | |---|---|---|---|---| | 1 | Blocked senders | a non-empty **Blocked senders** list | list empty | 100 on a new entry | | 2 | DNS blacklists | an active row on **DNS blacklists** | two rows, both inactive | 3 | | 3 | HELO host | `AntiSpam.CheckHostInHelo` | off | 2 | | 4 | PTR record | `AntiSpam.CheckPTR` | off | 1 | | 5 | Sender MX | `AntiSpam.UseMXChecks` | off | 2 | | 6 | SPF | `AntiSpam.UseSPF` | off | 3 | | 7 | SURBL | an active row on **SURBL servers** | one row, inactive | 3 | | 8 | DKIM | `AntiSpam.DKIMVerificationEnabled` | off | 5 | | 9 | ARC | `AntiSpam.ArcFilteringEnabled`, a trusted-sealer list and DMARC on | off | minus the DMARC failure score | | 10 | DMARC | `AntiSpam.DMARCEnabled` | **on** | 5 | | 11 | SpamAssassin | `AntiSpam.SpamAssassinEnabled` | off | 5, or the merged score | | 12 | Filter hook | a non-empty `FilterHookUrl` | empty | the engine's score; 100 for a reject | **DMARC is the only test on by default**, and at 5 a DMARC failure marks a message without refusing it. That is the shipped posture: tell the recipient, do not destroy the mail. What makes a test fail matters as much as its score. SPF scores only a hard `Fail`. DKIM scores only `PermFail`; a message with no signature is `None`, not a failure. DMARC scores a `p=reject` or `p=quarantine` policy the message failed and, separately, a message with more than one `From` header field; `p=none` failures are logged and score nothing. The blocked-senders list never matches the null sender, and its `550` says only "Sender address is blocked." without naming the entry. **Every DNS-backed check fails open**: a lookup that does not complete is read as "not spam". Since 6.3.7 a lookup that fails is a failure, not a name with no addresses: the HELO and PTR tests no longer score a name they could not look up, a DNS blacklist and SURBL no longer read a failed lookup as "not listed" (the blacklist says so in the TCP/IP log and, at most every 15 minutes, the application log), and a DKIM selector that is a CNAME whose query failed is `TempFail`, not `PermFail`, so it no longer scores. ### Greylisting Greylisting is not a score. It is a temporary refusal of a triplet (sender address, recipient address, connecting IP) this server has not seen before. The settings are on **Anti-spam settings → Greylisting**. | Setting | Default | Meaning | |---|---|---| | `AntiSpam.GreyListingEnabled` | off | Master switch | | `AntiSpam.GreyListingInitialDelay` | 30 minutes | How long a new triplet is refused for | | `AntiSpam.GreyListingInitialDelete` | 24 hours | An unconfirmed triplet is forgotten after this | | `AntiSpam.GreyListingFinalDelete` | 864 hours (36 days) | A confirmed triplet's expiry, pushed forward on every accepted message | | `AntiSpam.BypassGreylistingOnSPFSuccess` | on | Needs the SPF test enabled and to have run | | `AntiSpam.BypassGreylistingOnMailFromMX` | off | Costs an A and an MX lookup per message | A greylisted recipient gets `451 Please try again later` at `RCPT TO`; when the envelope sender is empty the refusal is postponed to the end of `DATA` and answered `451` ([docs/Greylisting.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Greylisting.md); the wiki prints `450`), because an empty sender at `RCPT TO` is very often another server's callout. Greylisting is skipped for a recipient domain with **Enable greylisting for this domain** unticked, for an IP on the greylisting white list, and for a sender or IP on the anti-spam white list. The wiki records that `GreylistingEnabledDuringRecordExpiration=0` did nothing at 6.2.28, so greylisting stays in force during the expiry sweep whatever it says. **Hardened since 6.3.4** (schema 6072; [docs/Greylisting.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Greylisting.md)): - **The triplet key no longer truncates.** Addresses used to be stored cut to 200 characters, so two long addresses sharing their first 200 were one triplet. Each is now keyed by an MD5 digest of the whole address with its domain lower-cased. This is the one change at the defaults: on PostgreSQL, and on MySQL with a binary collation, `A@Example.com` and `a@example.com` were two triplets and are now one, so a sender waits once less, never once more. Old rows are not rewritten at upgrade. - **A sliding wait and an abuse lock**, both off. With `GreyListingSlideOnEarlyRetry` on, each retry before the window opens moves the window to now plus the initial delay. `GreyListingAbuseThreshold` (0; the operator who asked for it uses 20) is the attempt answered `550 5.7.1 Too many delivery attempts before the greylisting delay had passed. ...`, with the triplet locked for `GreyListingAbuseLockMinutes` (60). It counts attempts, not retries: a server delivering twenty queued messages to one recipient down one connection makes twenty attempts in seconds. A triplet that has passed once is never refused. **Clear greylisting triplets** lifts every lock, at the cost of greylisting every sender once more. - **The greylisting bypass list** (`/api/v1/greylisting-bypass`, table `hm_greylisting_bypass`): a domain such as `example.com`, or a suffix such as `.mail.example.com`, matched on the envelope domain, the HELO host or either, passes greylisting only when SPF, DKIM or DMARC passes **for that name**. At least one requirement is needed. An entry requiring DKIM or DMARC cannot be judged at `RCPT TO`, so its greylisting runs after `DATA`, where a `451` or `550` ends the transaction for every recipient. **The test an entry requires must be switched on**: an entry requiring DKIM while DKIM verification is off (the default), or DMARC while DMARC is off, can never be satisfied, so the message is greylisted at `RCPT TO` as if the entry were not there; for an entry matched on the HELO, the SPF check of the HELO name is made only while SPF is switched on. It skips greylisting only; every spam test still runs. `BypassGreylistingOnSPFSuccess`, by contrast, covers every SPF pass in the world, a hijacked mailbox at a large provider included. ### DNS blacklists and SURBL **Spam & virus filtering → DNS blacklists** seeds two rows, both inactive until you tick them: | DNS host | Expected result | Reject message | Score | |---|---|---|---| | `zen.spamhaus.org` | `127.0.0.2-8\|127.0.0.10-11` | Rejected by Spamhaus. | 3 | | `bl.spamcop.net` | `127.0.0.2` | Rejected by SpamCop. | 3 | The expected-result field takes `|` for alternatives, a range on the last octet and wildcards; a malformed range is error `HM5342`. Every lookup, match or not, is written to the TCP/IP log, which is how to prove a listing to a sender who disputes it: ```text DNS lookup: 9.113.0.203.zen.spamhaus.org, 1 addresses found: 127.0.0.4, Match: 1 ``` The wiki warns that public mirrors of the large lists rate-limit or refuse resolvers that use a public DNS service; if a list stops matching, check its operator's terms first. **SURBL** checks the links in the body, so it runs after `DATA`. Collection stops once more than 15 distinct addresses are found, the run for one list aborts after 10 seconds, and, to 6.3.6, the first failed lookup abandoned the rest of the list; since 6.3.7 a failed lookup is a failure rather than "not listed", and SURBL goes on to ask the message's other names. One row is seeded inactive: `multi.surbl.org`, "Rejected by SURBL.", score 3. ### SPF, DKIM, DMARC and ARC **SPF** (`AntiSpam.UseSPF`, off, score 3). Evaluation is bounded at 10 DNS-querying terms and at `SpfVoidLookupLimit` void lookups (default 2; `0` switches that limit off). Since 6.3.4 the evaluator follows RFC 7208, ported from upstream hMailServer, in place of the RFC 4408 library the server carried: | | 6.3.3 and earlier | Since 6.3.4 | |---|---|---| | DNS | Its own lookups; `DNSServer`, the cache and the timeout were ignored on Windows | The server's own resolver settings apply | | Linux | Every check was `TempError` (`HM6406`), so DMARC lost its SPF half | Works | | Loopback clients | `127.0.0.0/8` and `::1` passed before any policy was read | No exemption | | `Authentication-Results` | Everything but `pass` and `fail` written as `neutral` | Each of the seven results under its own keyword | Nothing is scored differently: only a fail scores, as before. The loopback row is the one to check when upgrading. CHANGELOG says that where the SPF test or DMARC runs for a loopback client, mail from a domain whose policy does not name the address is judged like any other, so **a content filter that re-injects mail should authenticate, or connect from an IP range with anti-spam off**. **DKIM** (`AntiSpam.DKIMVerificationEnabled`, off, score 5). RSA keys shorter than 1024 bits are refused outright, and `rsa-sha1` signatures are invalid unless `DkimAcceptSha1=1`. **DMARC** (`AntiSpam.DMARCEnabled`, on, score 5). More than one `From` field is scored before any alignment is computed, because DKIM signs the last `From` while the header reader answers with the first. **ARC** (`AntiSpam.ArcFilteringEnabled`, off; `AntiSpam.ArcTrustedSealers`, empty). Its only output is an offset of exactly minus the DMARC failure score, applied when a chain whose every sealer is trusted validates and DMARC is about to fail. With the list empty it does nothing, deliberately: anyone can fabricate a chain and seal it with their own keys. Matching is exact. **Since 6.3.4: policy the score cannot express** (schema 6073; all off by default; [docs/SmtpPolicy.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SmtpPolicy.md)): | Setting | What it does | |---|---| | `UseSPFSoftFailScore`, `UseSPFTempErrorScore`, `UseSPFPermErrorScore` | Scores of their own for SoftFail (`~all`), TempError and PermError. `0` is "not scored", as before | | `SPFTempErrorDefer` | Answers a TempError with `451 4.4.3`, so a resolver blip defers the message. Never a `451` on PermError | | `RejectMultiRcptNullSender` | A null sender's second and every later distinct recipient, and its `DATA`, are refused `550 5.7.1`. A single-recipient null sender is never touched | | `BanMultiRcptNullSender`, `BanMultiRcptNullSenderHours` | Also bans the address through the auto-ban, once per session, for 24 hours by default (1 to 8760); never an address on `AutoBanNeverBan` or an incoming relay. Does nothing while `RejectMultiRcptNullSender` is off | The **domain white and black list** (**Spam & virus filtering → Domain white and black list**; `/api/v1/domain-policy-list`) is keyed by the envelope domain or the HELO suffix rather than by IP, for the correspondent whose sending hosts change every week. It is consulted at `MAIL FROM` for unauthenticated sessions of a range that runs the spam tests. **Black** is `550 5.7.1 Mail from is refused by this server's policy.` before anything else runs. **White** skips greylisting, is not deferred for a TempError and is not refused or quarantined on the score, while every test still runs and the mark threshold still marks. It never overrides authentication. The list is ASCII, so a U-label such as `bücher.example` is refused, and `hmctl` has no verb for it. ### SpamAssassin and the filter hook hMailServer speaks the spamd protocol over TCP to any SpamAssassin daemon, on this machine or on a Linux host. Set `AntiSpam.SpamAssassinHost` (`127.0.0.1`) and `AntiSpam.SpamAssassinPort` (`783`) and press **Test SpamAssassin connection**. `AntiSpam.SpamAssassinMergeScore` (off) uses the integer part of SpamAssassin's own `score=` in place of the fixed `AntiSpam.SpamAssassinScore` (5). The timeout moves between `SAMinTimeout` and `SAMaxTimeout` (30 and 90 seconds) with load. `SpamAssassinLearnOnMove` (off) tells spamd that a message moved or copied into Junk is spam, and one moved out of it ham, with spamc's TELL, which spamd refuses unless it was started with `--allow-tell`. An unresolvable host is `HM5507`; a scan that does not finish in time is `HM5508`, and the message is accepted with no verdict; a message that cannot be reloaded after the scan is `HM5509` and is left as SpamAssassin wrote it. `FilterHookUrl` puts rspamd, or anything answering JSON over HTTP, in the path of every accepted message; `http://127.0.0.1:11333/checkv2` needs no adapter. `FilterHookTimeoutSeconds` is 10, and with `FilterHookFailClosed` (off) turned on an unanswered check scores `FilterHookRejectScore` (100) and reports `HM5541`. ### Quarantine, and what a refused sender sees **Quarantine** (`QuarantineEnabled`, off; `QuarantineRetentionDays`, 30) changes what the delete threshold does after the body: the message is answered `250 Queued for delivery` and held on **Spam & virus filtering → Quarantine**. A verdict reached before the body has no message to hold, so it stays a refusal, and a quarantine that fails to store falls through to refusing, so a spam refusal never becomes silent deletion. Since 6.3.4 the quarantine also holds what [a mail flow rule](/topic/28251) sends there (`Rule: `), what impersonation protection holds and what a recipient's blocked-senders list holds. **Per-account overrides**, on the account dialog's **Spam** tab, re-judge that account's own copy at delivery. They only read `X-hMailServer-Reason-Score`, and only while **Add X-hMailServer-Reason header** is on; with it off the value could be the sender's own, so the overrides decline to act. | Moment | Client sees | |---|---| | Score over delete threshold, before the body | `550 5.7.1 ` | | Score over delete threshold, after the body | `554 5.7.1 ` | | Same, but quarantined | `250 2.0.0 Queued for delivery` | | Greylisted, real sender | `451 Please try again later.` | | Greylisted, null sender | `451` at end of `DATA` (the wiki prints `450`) | Per-test timings go to the debug log (`Spam test: SpamTestSPF, Score: 3, Time: 41 ms`); a test taking 10 seconds or more is promoted to the application log. Because every DNS-backed check fails open, a resolver that has stopped answering looks like a run of clean mail, so DKIM, DMARC and SURBL, and since 6.3.7 a DNS blacklist, each write one application-log line at most every 15 minutes when their lookups stop completing. ### A configuration that works The wiki's order for a small server: 1. Leave the thresholds at 5 and 20 and leave DMARC on. Watch for a fortnight. 2. Activate `zen.spamhaus.org` (score 3). 3. Turn on SPF (score 3) and DKIM verification (score 5). With a DNSBL hit they reach 11, still under the delete threshold. 4. Turn on `AuthenticationResultsEnabled`, so every delivered message records what the server concluded. 5. Add greylisting if your users can tolerate a first-message delay, leaving bypass on SPF success on. 6. Only then add SpamAssassin or an external engine, and consider raising the delete threshold rather than lowering it. ### What this topic does not cover, and where to read more The per-recipient allow and block lists, the quarantine digest, spam traps, country blocking and impersonation protection are in [allow and block lists, digests and spam traps](/topic/28249), and viruses in [viruses and dangerous attachments](/topic/28250). The other direction is [Being trusted by the servers you send to](/topic/28188). The Control Panel and COM are Windows-only; the 6.3.4 switches are also on the Settings view of [the Control Deck](/topic/28192) and `PUT /api/v1/settings/antispam`. - Wiki: https://gitlab.com/hmailserver/hmailserver/-/wikis/Stopping-Spam - https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Greylisting.md - https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SmtpPolicy.md - Public documentation: https://www.progressiverobot.com/hmailserver-documentation/#10-stopping-spam and https://www.progressiverobot.com/hmailserver-documentation/#265-the-anti-spam-pipeline - [The 6.3.4 security announcement](/topic/28218) and [Upgrading to 6.3.4](/topic/28193) --- Canonical: https://www.hmailserver.co.uk/topic/28248/the-anti-spam-stack-end-to-end-score-greylisting-spf-dkim-dmarc-dnsbl-surbl # The anti-spam stack end to end: score, greylisting, SPF, DKIM, DMARC, DNSBL, SURBL *Written for 6.3.4. hMailServer 6.3.7 (17 October 2026) reads a DNS lookup that fails as a failure, not as a name with no addresses: the HELO and PTR tests no longer score a name they could not look up, a DNS blacklist and SURBL no longer read a failed lookup as "not listed", and a DKIM selector that is a CNAME whose query failed is `TempFail`, not `PermFail`. SURBL reads a link's host that is not ASCII by UTS 46 and an HTML part's character references as a browser does, and SpamAssassin is sent a collected message's own `Return-Path`, so its SPF and envelope rules may score such mail differently. The DMARC check's pass now also decides whether a forward of mail from outside whose `From` names a hosted domain's organisation is given an envelope sender in that organisation, and no pass is recorded for a message the check did not judge: DMARC off, a sender on the white list or an IP range with spam protection off, a message over `AntiSpam.MaximumMessageSize`, or one the tests before it had already scored to the higher threshold. In the Control Panel the sidebar has a Spam filtering group, with its six allow and block lists under Lists, and a Virus scanning group, and Ctrl+K takes every old title to its page. See [the 6.3.7 announcement](/topic/28277).* This topic walks through every layer hMailServer 6.3.4 puts between an SMTP session and a mailbox: the two thresholds, the twelve tests in the order they run, greylisting and its 6.3.4 hardening, the DNS and URL blocklists, SPF, DKIM, DMARC and ARC, SpamAssassin, and the quarantine. The one thing to know first: none of these tests is a verdict on its own. Each failing test adds a score, two thresholds decide what the total means, and three kinds of session are never tested at all: an authenticated session, a sender or IP on the anti-spam white list, and any session from an IP range with spam protection switched off. ### Where the decisions are made Everything happens in one of two moments: before the body (at `MAIL FROM` or `RCPT TO`) and after the body (after `DATA` or the last `BDAT`). The pre-transmission tests run at `MAIL FROM` by default: `DNSBLChecksAfterMailFrom` defaults to `1`, and `0` moves them to the first `RCPT TO`, which leaks less to an address-probing client but accepts more of the conversation first. A session from [a configured incoming relay](/topic/28240) has both phases deferred to after `DATA`. **Who is never tested.** `SMTPConnection::GetDoSpamProtection_` returns false the moment the session is authenticated, so your own users' outbound mail is never scored; [the sending limits and rate limits](/topic/28239) are the controls for that. A sender or IP on the anti-spam **White list** skips the tests entirely, not merely the scoring: SPF is not evaluated, no `Authentication-Results` verdict is recorded and no reason header is written. **The read-only overview.** **Spam & virus filtering → Spam filtering overview** in the Control Panel (Windows only) shows on one screen which checks are on, what each scores, what the thresholds then do, and which checks are switched on but cannot run. ### The score and the two thresholds Both thresholds are on **Spam & virus filtering → Anti-spam settings → General**. | Setting | Default | At or above it | `0` means | |---|---|---|---| | `AntiSpam.SpamMarkThreshold` | 5 | Delivered but tagged: `X-hMailServer-Spam: YES`, `X-hMailServer-Reason-*` headers and, optionally, a subject prefix | Never mark | | `AntiSpam.SpamDeleteThreshold` | 20 | Refused during the conversation, or quarantined if quarantine is on and the verdict came after the body | Never refuse | | `AntiSpam.MaximumMessageSize` | 1024 KB | Post-transmission tests are skipped for a larger message | No limit of its own; the MIME parser's 80 MB ceiling still applies | The higher of the two thresholds is also the early-out: `SpamTestRunner::RunSpamTest` stops calling tests once the running total reaches it, which is why a message condemned by the blocked-senders list never pays for a DNS lookup. With both thresholds at 0 the early-out is skipped, because greylisting's SPF bypass needs the SPF result and SPF is the sixth test. With `AddHeaderSpam` and `AddHeaderReason` both on (the defaults), a marked message carries: ```text X-hMailServer-Spam: YES X-hMailServer-Reason-1: Blocked by SPF. (example.com does not designate 203.0.113.9) - (Score: 3) X-hMailServer-Reason-2: Rejected by Spamhaus. - (Score: 3) X-hMailServer-Reason-Score: 6 Subject: [SPAM] Your invoice ``` Only failing tests get a `Reason-N` line. `X-hMailServer-Reason-Score` is the sum of the failing tests, and with `AddHeaderReason` off it is not written at all. The subject prefix is off by default; its text is `[SPAM]` (`antispamprependsubjecttext`). **Start by tagging, not deleting.** The wiki's advice is a fortnight with the delete threshold at `0`, reading what got tagged, before turning deletion on well above anything legitimate has scored. ### Every test, in the order it runs This is the registration order in `SpamTestRunner::LoadSpamTests`; nothing reorders it at runtime. Tests 1 to 6 run before the body, 7 to 12 after it. | # | Test | Switched on by | Default | Default score | |---|---|---|---|---| | 1 | Blocked senders | a non-empty **Blocked senders** list | list empty | 100 on a new entry | | 2 | DNS blacklists | an active row on **DNS blacklists** | two rows, both inactive | 3 | | 3 | HELO host | `AntiSpam.CheckHostInHelo` | off | 2 | | 4 | PTR record | `AntiSpam.CheckPTR` | off | 1 | | 5 | Sender MX | `AntiSpam.UseMXChecks` | off | 2 | | 6 | SPF | `AntiSpam.UseSPF` | off | 3 | | 7 | SURBL | an active row on **SURBL servers** | one row, inactive | 3 | | 8 | DKIM | `AntiSpam.DKIMVerificationEnabled` | off | 5 | | 9 | ARC | `AntiSpam.ArcFilteringEnabled`, a trusted-sealer list and DMARC on | off | minus the DMARC failure score | | 10 | DMARC | `AntiSpam.DMARCEnabled` | **on** | 5 | | 11 | SpamAssassin | `AntiSpam.SpamAssassinEnabled` | off | 5, or the merged score | | 12 | Filter hook | a non-empty `FilterHookUrl` | empty | the engine's score; 100 for a reject | **DMARC is the only test on by default**, and at 5 a DMARC failure marks a message without refusing it. That is the shipped posture: tell the recipient, do not destroy the mail. What makes a test fail matters as much as its score. SPF scores only a hard `Fail`. DKIM scores only `PermFail`; a message with no signature is `None`, not a failure. DMARC scores a `p=reject` or `p=quarantine` policy the message failed and, separately, a message with more than one `From` header field; `p=none` failures are logged and score nothing. The blocked-senders list never matches the null sender, and its `550` says only "Sender address is blocked." without naming the entry. **Every DNS-backed check fails open**: a lookup that does not complete is read as "not spam". Since 6.3.7 a lookup that fails is a failure, not a name with no addresses: the HELO and PTR tests no longer score a name they could not look up, a DNS blacklist and SURBL no longer read a failed lookup as "not listed" (the blacklist says so in the TCP/IP log and, at most every 15 minutes, the application log), and a DKIM selector that is a CNAME whose query failed is `TempFail`, not `PermFail`, so it no longer scores. ### Greylisting Greylisting is not a score. It is a temporary refusal of a triplet (sender address, recipient address, connecting IP) this server has not seen before. The settings are on **Anti-spam settings → Greylisting**. | Setting | Default | Meaning | |---|---|---| | `AntiSpam.GreyListingEnabled` | off | Master switch | | `AntiSpam.GreyListingInitialDelay` | 30 minutes | How long a new triplet is refused for | | `AntiSpam.GreyListingInitialDelete` | 24 hours | An unconfirmed triplet is forgotten after this | | `AntiSpam.GreyListingFinalDelete` | 864 hours (36 days) | A confirmed triplet's expiry, pushed forward on every accepted message | | `AntiSpam.BypassGreylistingOnSPFSuccess` | on | Needs the SPF test enabled and to have run | | `AntiSpam.BypassGreylistingOnMailFromMX` | off | Costs an A and an MX lookup per message | A greylisted recipient gets `451 Please try again later` at `RCPT TO`; when the envelope sender is empty the refusal is postponed to the end of `DATA` and answered `451` ([docs/Greylisting.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Greylisting.md); the wiki prints `450`), because an empty sender at `RCPT TO` is very often another server's callout. Greylisting is skipped for a recipient domain with **Enable greylisting for this domain** unticked, for an IP on the greylisting white list, and for a sender or IP on the anti-spam white list. The wiki records that `GreylistingEnabledDuringRecordExpiration=0` did nothing at 6.2.28, so greylisting stays in force during the expiry sweep whatever it says. **Hardened since 6.3.4** (schema 6072; [docs/Greylisting.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Greylisting.md)): - **The triplet key no longer truncates.** Addresses used to be stored cut to 200 characters, so two long addresses sharing their first 200 were one triplet. Each is now keyed by an MD5 digest of the whole address with its domain lower-cased. This is the one change at the defaults: on PostgreSQL, and on MySQL with a binary collation, `A@Example.com` and `a@example.com` were two triplets and are now one, so a sender waits once less, never once more. Old rows are not rewritten at upgrade. - **A sliding wait and an abuse lock**, both off. With `GreyListingSlideOnEarlyRetry` on, each retry before the window opens moves the window to now plus the initial delay. `GreyListingAbuseThreshold` (0; the operator who asked for it uses 20) is the attempt answered `550 5.7.1 Too many delivery attempts before the greylisting delay had passed. ...`, with the triplet locked for `GreyListingAbuseLockMinutes` (60). It counts attempts, not retries: a server delivering twenty queued messages to one recipient down one connection makes twenty attempts in seconds. A triplet that has passed once is never refused. **Clear greylisting triplets** lifts every lock, at the cost of greylisting every sender once more. - **The greylisting bypass list** (`/api/v1/greylisting-bypass`, table `hm_greylisting_bypass`): a domain such as `example.com`, or a suffix such as `.mail.example.com`, matched on the envelope domain, the HELO host or either, passes greylisting only when SPF, DKIM or DMARC passes **for that name**. At least one requirement is needed. An entry requiring DKIM or DMARC cannot be judged at `RCPT TO`, so its greylisting runs after `DATA`, where a `451` or `550` ends the transaction for every recipient. **The test an entry requires must be switched on**: an entry requiring DKIM while DKIM verification is off (the default), or DMARC while DMARC is off, can never be satisfied, so the message is greylisted at `RCPT TO` as if the entry were not there; for an entry matched on the HELO, the SPF check of the HELO name is made only while SPF is switched on. It skips greylisting only; every spam test still runs. `BypassGreylistingOnSPFSuccess`, by contrast, covers every SPF pass in the world, a hijacked mailbox at a large provider included. ### DNS blacklists and SURBL **Spam & virus filtering → DNS blacklists** seeds two rows, both inactive until you tick them: | DNS host | Expected result | Reject message | Score | |---|---|---|---| | `zen.spamhaus.org` | `127.0.0.2-8\|127.0.0.10-11` | Rejected by Spamhaus. | 3 | | `bl.spamcop.net` | `127.0.0.2` | Rejected by SpamCop. | 3 | The expected-result field takes `|` for alternatives, a range on the last octet and wildcards; a malformed range is error `HM5342`. Every lookup, match or not, is written to the TCP/IP log, which is how to prove a listing to a sender who disputes it: ```text DNS lookup: 9.113.0.203.zen.spamhaus.org, 1 addresses found: 127.0.0.4, Match: 1 ``` The wiki warns that public mirrors of the large lists rate-limit or refuse resolvers that use a public DNS service; if a list stops matching, check its operator's terms first. **SURBL** checks the links in the body, so it runs after `DATA`. Collection stops once more than 15 distinct addresses are found, the run for one list aborts after 10 seconds, and, to 6.3.6, the first failed lookup abandoned the rest of the list; since 6.3.7 a failed lookup is a failure rather than "not listed", and SURBL goes on to ask the message's other names. One row is seeded inactive: `multi.surbl.org`, "Rejected by SURBL.", score 3. ### SPF, DKIM, DMARC and ARC **SPF** (`AntiSpam.UseSPF`, off, score 3). Evaluation is bounded at 10 DNS-querying terms and at `SpfVoidLookupLimit` void lookups (default 2; `0` switches that limit off). Since 6.3.4 the evaluator follows RFC 7208, ported from upstream hMailServer, in place of the RFC 4408 library the server carried: | | 6.3.3 and earlier | Since 6.3.4 | |---|---|---| | DNS | Its own lookups; `DNSServer`, the cache and the timeout were ignored on Windows | The server's own resolver settings apply | | Linux | Every check was `TempError` (`HM6406`), so DMARC lost its SPF half | Works | | Loopback clients | `127.0.0.0/8` and `::1` passed before any policy was read | No exemption | | `Authentication-Results` | Everything but `pass` and `fail` written as `neutral` | Each of the seven results under its own keyword | Nothing is scored differently: only a fail scores, as before. The loopback row is the one to check when upgrading. CHANGELOG says that where the SPF test or DMARC runs for a loopback client, mail from a domain whose policy does not name the address is judged like any other, so **a content filter that re-injects mail should authenticate, or connect from an IP range with anti-spam off**. **DKIM** (`AntiSpam.DKIMVerificationEnabled`, off, score 5). RSA keys shorter than 1024 bits are refused outright, and `rsa-sha1` signatures are invalid unless `DkimAcceptSha1=1`. **DMARC** (`AntiSpam.DMARCEnabled`, on, score 5). More than one `From` field is scored before any alignment is computed, because DKIM signs the last `From` while the header reader answers with the first. **ARC** (`AntiSpam.ArcFilteringEnabled`, off; `AntiSpam.ArcTrustedSealers`, empty). Its only output is an offset of exactly minus the DMARC failure score, applied when a chain whose every sealer is trusted validates and DMARC is about to fail. With the list empty it does nothing, deliberately: anyone can fabricate a chain and seal it with their own keys. Matching is exact. **Since 6.3.4: policy the score cannot express** (schema 6073; all off by default; [docs/SmtpPolicy.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SmtpPolicy.md)): | Setting | What it does | |---|---| | `UseSPFSoftFailScore`, `UseSPFTempErrorScore`, `UseSPFPermErrorScore` | Scores of their own for SoftFail (`~all`), TempError and PermError. `0` is "not scored", as before | | `SPFTempErrorDefer` | Answers a TempError with `451 4.4.3`, so a resolver blip defers the message. Never a `451` on PermError | | `RejectMultiRcptNullSender` | A null sender's second and every later distinct recipient, and its `DATA`, are refused `550 5.7.1`. A single-recipient null sender is never touched | | `BanMultiRcptNullSender`, `BanMultiRcptNullSenderHours` | Also bans the address through the auto-ban, once per session, for 24 hours by default (1 to 8760); never an address on `AutoBanNeverBan` or an incoming relay. Does nothing while `RejectMultiRcptNullSender` is off | The **domain white and black list** (**Spam & virus filtering → Domain white and black list**; `/api/v1/domain-policy-list`) is keyed by the envelope domain or the HELO suffix rather than by IP, for the correspondent whose sending hosts change every week. It is consulted at `MAIL FROM` for unauthenticated sessions of a range that runs the spam tests. **Black** is `550 5.7.1 Mail from is refused by this server's policy.` before anything else runs. **White** skips greylisting, is not deferred for a TempError and is not refused or quarantined on the score, while every test still runs and the mark threshold still marks. It never overrides authentication. The list is ASCII, so a U-label such as `bücher.example` is refused, and `hmctl` has no verb for it. ### SpamAssassin and the filter hook hMailServer speaks the spamd protocol over TCP to any SpamAssassin daemon, on this machine or on a Linux host. Set `AntiSpam.SpamAssassinHost` (`127.0.0.1`) and `AntiSpam.SpamAssassinPort` (`783`) and press **Test SpamAssassin connection**. `AntiSpam.SpamAssassinMergeScore` (off) uses the integer part of SpamAssassin's own `score=` in place of the fixed `AntiSpam.SpamAssassinScore` (5). The timeout moves between `SAMinTimeout` and `SAMaxTimeout` (30 and 90 seconds) with load. `SpamAssassinLearnOnMove` (off) tells spamd that a message moved or copied into Junk is spam, and one moved out of it ham, with spamc's TELL, which spamd refuses unless it was started with `--allow-tell`. An unresolvable host is `HM5507`; a scan that does not finish in time is `HM5508`, and the message is accepted with no verdict; a message that cannot be reloaded after the scan is `HM5509` and is left as SpamAssassin wrote it. `FilterHookUrl` puts rspamd, or anything answering JSON over HTTP, in the path of every accepted message; `http://127.0.0.1:11333/checkv2` needs no adapter. `FilterHookTimeoutSeconds` is 10, and with `FilterHookFailClosed` (off) turned on an unanswered check scores `FilterHookRejectScore` (100) and reports `HM5541`. ### Quarantine, and what a refused sender sees **Quarantine** (`QuarantineEnabled`, off; `QuarantineRetentionDays`, 30) changes what the delete threshold does after the body: the message is answered `250 Queued for delivery` and held on **Spam & virus filtering → Quarantine**. A verdict reached before the body has no message to hold, so it stays a refusal, and a quarantine that fails to store falls through to refusing, so a spam refusal never becomes silent deletion. Since 6.3.4 the quarantine also holds what [a mail flow rule](/topic/28251) sends there (`Rule: `), what impersonation protection holds and what a recipient's blocked-senders list holds. **Per-account overrides**, on the account dialog's **Spam** tab, re-judge that account's own copy at delivery. They only read `X-hMailServer-Reason-Score`, and only while **Add X-hMailServer-Reason header** is on; with it off the value could be the sender's own, so the overrides decline to act. | Moment | Client sees | |---|---| | Score over delete threshold, before the body | `550 5.7.1 ` | | Score over delete threshold, after the body | `554 5.7.1 ` | | Same, but quarantined | `250 2.0.0 Queued for delivery` | | Greylisted, real sender | `451 Please try again later.` | | Greylisted, null sender | `451` at end of `DATA` (the wiki prints `450`) | Per-test timings go to the debug log (`Spam test: SpamTestSPF, Score: 3, Time: 41 ms`); a test taking 10 seconds or more is promoted to the application log. Because every DNS-backed check fails open, a resolver that has stopped answering looks like a run of clean mail, so DKIM, DMARC and SURBL, and since 6.3.7 a DNS blacklist, each write one application-log line at most every 15 minutes when their lookups stop completing. ### A configuration that works The wiki's order for a small server: 1. Leave the thresholds at 5 and 20 and leave DMARC on. Watch for a fortnight. 2. Activate `zen.spamhaus.org` (score 3). 3. Turn on SPF (score 3) and DKIM verification (score 5). With a DNSBL hit they reach 11, still under the delete threshold. 4. Turn on `AuthenticationResultsEnabled`, so every delivered message records what the server concluded. 5. Add greylisting if your users can tolerate a first-message delay, leaving bypass on SPF success on. 6. Only then add SpamAssassin or an external engine, and consider raising the delete threshold rather than lowering it. ### What this topic does not cover, and where to read more The per-recipient allow and block lists, the quarantine digest, spam traps, country blocking and impersonation protection are in [allow and block lists, digests and spam traps](/topic/28249), and viruses in [viruses and dangerous attachments](/topic/28250). The other direction is [Being trusted by the servers you send to](/topic/28188). The Control Panel and COM are Windows-only; the 6.3.4 switches are also on the Settings view of [the Control Deck](/topic/28192) and `PUT /api/v1/settings/antispam`. - Wiki: https://gitlab.com/hmailserver/hmailserver/-/wikis/Stopping-Spam - https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Greylisting.md - https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SmtpPolicy.md - Public documentation: https://www.progressiverobot.com/hmailserver-documentation/#10-stopping-spam and https://www.progressiverobot.com/hmailserver-documentation/#265-the-anti-spam-pipeline - [The 6.3.4 security announcement](/topic/28218) and [Upgrading to 6.3.4](/topic/28193) --- Canonical: https://www.hmailserver.co.uk/topic/28249/allow-and-block-lists-digests-spam-traps-country-blocking-and-impersonation # Allow and block lists, digests, spam traps, country blocking and impersonation *Written for 6.3.4. hMailServer 6.3.7 (17 October 2026) reads a link's host that is not ASCII by UTS 46, and an HTML part's character references as a browser does, for the URL entries of the allow and block lists, so a message whose links escaped a list that way can now be tagged or held; a URL entry written in Unicode is stored in its `xn--` form and covers a link written either way, and one with a label UTS 46 refuses is refused when it is saved. In the Control Panel the Spam filtering group's six allow and block lists are under Lists, and Ctrl+K takes every old title to its page. See [the 6.3.7 announcement](/topic/28277).* This topic covers five layers that sit beside the score-based spam tests rather than inside them: the allow and block lists at three levels, the quarantine digest, spam traps, country blocking and impersonation protection. All five are new since 6.3.4, and the one thing to know first is that every one of them is off, or inert, until an administrator sets it: a server upgraded to 6.3.4 and left alone has no list entries, sends no digest, has no traps, judges nothing by country and checks no display name. ### The allow and block list: one table, three levels *Since 6.3.4 (schema 6046).* The older lists (the anti-spam white list, **Blocked senders**, the greylisting white list) act during the SMTP conversation, for every recipient at once. The allow and block list is per recipient. It is one table, `hm_senderlist`, with a scope on each row: each account's own safe and blocked senders, a domain's list, and the server's. | Entry | Covers | |---|---| | `someone@example.com` | That one address, in any case. No wildcards | | `example.com` | `example.com` and every subdomain; not `notexample.com`, not `example.com.attacker.net` | | `@example.com` | The same as `example.com`, and stored as `example.com` | | `https://bad.example/anything` | A URL: stored as `bad.example/`, covering a link in the message to that host or a subdomain, whoever sent it | An entry covers a message when it matches **either** the envelope sender **or** the address in the `From` header, because mail sent through a mailing service carries the service's address in the envelope and the brand's in the header. A message with the null sender is never decided by any list. **URL entries are in 6.3.4.** CHANGELOG's 6.3.4 section lists them, and they are consulted only when neither address decided anything. The closing section of [docs/SenderListsAndDigests.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SenderListsAndDigests.md), which says "URLs are not list entries", is older text left in the file. **What allow and block do.** The lists are read at local delivery, once per recipient, after the spam tests have run. | | The message was classified as spam | It was not | |---|---|---| | **Allowed** | Delivered unmarked for this recipient: the spam flag, the `X-hMailServer-Spam` and reason headers and the subject tag are removed from this copy | Delivered as it is | | **Blocked** | Held in the quarantine for this recipient when the quarantine is on; marked as spam and delivered when it is not | The same | A block **never deletes**, and no score is recorded on a blocked message, because nothing measured it. A message released from the quarantine is never held again by a list. The other recipients of the same message are not affected. **Precedence is nearest first**: the account, then its domain, then the server. The first of the three that has anything to say about the sender is the only one consulted, so an account's allow beats a domain's block. That is the reverse of Microsoft 365's order, and deliberate: the person the message is addressed to can tell whether they wanted it. An administrator who needs a sender stopped for everyone still has **Blocked senders**, which nobody's own list overrules. Inside one list the more specific entry wins (an address beats a domain, a longer domain beats a shorter one), and on an exact tie the block wins. **Neither address is proof.** Both are whatever the sender wrote, so an allow entry is as forgeable as the address it names. The document's advice: keep allow entries to correspondents whose domains publish DMARC, and prefer a single address to a whole domain. | List | Webmail | Control Panel (Windows) | Control Deck | REST | |---|---|---|---|---| | The account's own | Settings → **Safe senders** | none | none | `/api/v1/me/sender-list` | | A domain's | none | Domain → **Allow and block list** tab | Domain editor → **Allow and block list** | `/api/v1/domains/{domain}/sender-list` | | The server's | none | Spam & virus filtering → **Allow and block list** | Lists → **Allow and block list** | `/api/v1/sender-list` | Over COM (Windows only) the domain's list is `Domain.SenderList` and the server's `Settings.AntiSpam.SenderList`. There is no administrator path to an account's own list. **Block this sender** and **Report phishing** in the webmail now add the sender to the account's blocked senders; they used to write a rule into the Sieve script, and rules written that way are left where they are. A list holds at most a thousand entries; adding a value already there rewrites the entry (`200` rather than `201`). Expiry can be set over REST, in the Deck and in the Control Panel's two list editors (a switch and a date); the webmail does not offer it. [docs/SenderListsAndDigests.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SenderListsAndDigests.md) still says the Control Panel has no expiry; that passage predates the change, which the repository's roadmap records as made after 6.3.3. ### The quarantine digest Once a day, one plain-text message goes to each account of a domain that has something **newly** held for it, listing each message's subject, sender, when it was held and why, with a link that delivers each entry, a link that delivers everything listed, and a footer link that stops the digest (also the `List-Unsubscribe` header). An empty digest is never sent, and an account that has had one is not sent another for twenty hours. The message comes from the domain's postmaster with the null envelope sender and `Auto-Submitted: auto-generated`, and never asks for a password. | Setting | Default | |---|---| | `QuarantineDigestBaseUrl`, the address the links point at, such as `https://mail.example.com` | empty: **no digest is sent** | | `QuarantineDigestLinkHours`, 1 to 720 | 72 | | The domain's switch and hour (`quarantine_digest`, `quarantine_digest_hour`; 0 to 23, local time), which an account may override | off; 7 | All three are in the database, none in `hMailServer.ini`. The quarantine itself (`QuarantineEnabled`) has to be on, and the address must be an absolute `http://` or `https://` URL of the listener the webmail is served from. **When it is sent.** The task wakes every five minutes and, within the digest hour, sends only its share, so a domain of a thousand accounts is spread across the hour. At most five hundred digests go in one run. Over COM (Windows only), `AntiSpam.SendQuarantineDigests()` sends now, ignoring the hour and the twenty-hour rule and nothing else. **The links.** Each is 32 random bytes recorded against one account and one entry, usable once, and no longer valid after `QuarantineDigestLinkHours`. Following a link does not act on its own: it opens a page with one button, because mail security products follow every link in a message, and a link that released mail on its own would have one of them releasing every held message. Stopping the digest does not stop mail being held. ### Spam traps *Since 6.3.4 (schema 6058).* A trap is an address that was never real and never given to anybody, so a message to one comes from a harvested, bought or guessed list. At `RCPT TO` a trap is refused **exactly as any unknown address is**, `550 Unknown user`, made by the same code, so the sender cannot tell. The check runs before the delivery lookups, so a catch-all, greylisting or a backup-MX callout cannot answer first. | | What happens | Setting | |---|---|---| | The connecting address | Banned through the server's own auto-ban, as an expiring range named `Auto-ban: spam trap
` | `SpamTrapBanMinutes`, 1440. 0 bans nobody | | The envelope sender | Added to the server's block list with an expiry, **only when its domain's SPF passes for the connecting address** | `SpamTrapBlockDays`, 30. 0 blocks nobody | | The hit | Recorded in `hm_spamtraphits` and counted in `hmailserver_spam_trap_hits_total` | `SpamTrapHitRetentionDays`, 90 | The application log gets one line per hit and one per ban: ``` "Auto-ban: 203.0.113.9 is banned for 1440 minutes after addressing the spam trap never-used@example.com" ``` [The packaged fail2ban filter](/topic/28220) matches that line, so a Linux installation that runs fail2ban with the packaged jail turned on bans the address at its own firewall as well. The jail ships disabled; the wiki's Installing on Linux page turns it on with: ``` printf '[hmailserver]\nenabled = true\n' > /etc/fail2ban/jail.d/hmailserver.local systemctl reload fail2ban ``` **The SPF condition.** As first written, in the source tree, a trap put the envelope sender of every message sent to it on the block list, whatever that sender was; spam traps are new in 6.3.4, which carries the fix, so no release did. CHANGELOG records why that changed: the envelope is whatever the client types, so one connection naming a bank's address could have had the bank's real mail held or marked for every mailbox. Now only an SPF **pass** blocks; the address is banned either way. The wiki's summary in Stopping Spam §10.15 does not mention the condition; [docs/SpamTraps.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SpamTraps.md) and CHANGELOG do. **Who is never banned:** an authenticated session, an address whose IP range does not run the spam tests, an address or sender the anti-spam white list covers, an incoming relay, and an address on `AutoBanNeverBan`. The sender is not blocked for the null sender, for a sender claiming a domain this server hosts, or where the server's allow list already names it. **Training.** A trap hit trains nothing by itself, because no content arrives. With `SpamTrapTeachSpamAssassin` on (off by default; it needs SpamAssassin enabled and a spamd started with `--allow-tell`), a message sent to a real mailbox in the same transaction is taught to spamd as spam. **Declaring a trap.** It must be an address in the domain it is declared on, and one nobody uses: an account, alias or list with that address is refused, and none can be given a trap's address while the trap exists. | Where | How | |---|---| | Control Deck | Domain editor → **Spam traps**; the **Spam traps** view for the record across every domain | | REST | `GET`/`POST /api/v1/domains/{domain}/spam-traps`, `DELETE /api/v1/spam-traps/{address}`, `GET`/`DELETE /api/v1/spam-traps/hits` | | Command line | `hmctl spamtrap create --domain example.com --set address=never-used@example.com`, `hmctl traphits list` | | PowerShell | `Add-HmSpamTrap`, `Get-HmSpamTrapHit` | | COM (Windows only) | `Domain.AddSpamTrap`, `Utilities.GetSpamTrapHits` | A good trap is an address never published anywhere, or one published once, long ago, and unused for years (give it a year of refusals first). A bad trap is a typo of a real address such as `saels@`, or any address in a domain with a catch-all: there every other unknown address is accepted and the trap is not, which is the difference a sender can measure. Clearing the record of hits undoes nothing; the bans and blocks stay until they expire or are removed. The traps are in the backup with their domain; the record of hits is not. ### Country blocking *Since 6.3.4 (schema 6052).* Two kinds of policy, in one grammar. A policy **per listener family** (SMTP, IMAP, POP3, and the HTTP listeners) is applied when a connection is accepted: before the greeting, before the TLS handshake and before anything touches the database. A policy **per domain** is applied when somebody signs in to an account of that domain, on any protocol, and covers OAuth2 bearer tokens, client certificates, Kerberos tickets, passkeys and single sign-on as well as passwords. | Value | What it means | |---|---| | *(empty)* | Nothing is judged by country. The default everywhere | | `allow:GB,IE,FR` | Only these countries may connect, or sign in | | `block:CN,RU` | These countries may not connect, or sign in | **The database is yours to supply.** Point `GeoIpDatabaseFile` at a MaxMind `.mmdb` file, such as `/var/lib/GeoIP/GeoLite2-Country.mmdb` or `C:\GeoIP\GeoLite2-Country.mmdb`. The server ships none and never fetches one. Replace the file in place and the server reloads it within five minutes. `GeoIpDatabaseMaxAgeDays` (45) is how old the file's build date may be before the log says so; a stale database is still used. **What is exempt.** `CountryBlockingExemptRanges` defaults to the loopback addresses and every private network: ``` 127.0.0.0/8,::1/128,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,169.254.0.0/16,100.64.0.0/10,fe80::/10,fc00::/7 ``` There is no second list compiled into the server. If you empty it, do so with `country_policy_http` off, or the Control Deck is the next thing you lose. An open session is never disturbed; a policy takes effect on the next connection. **Which way it fails.** With no database at all (none configured, or the file missing or unreadable) **everything is allowed**, whatever the policies say, and the reason is logged once. With a database loaded that has no country for an address, an allow list refuses it and a block list allows it. **What a refused client is told: nothing.** A refused connection is closed with no banner. A refused sign-in gets the ordinary invalid-credentials reply, byte-identical to a wrong password, and is counted against neither the account's lockout nor the address's auto-ban, since the person refused is very often the account's real owner, travelling. Your copy is in the application log: ``` Country blocking: refused SMTP from 203.0.113.9 (CN) for connection. Country blocking: refused sign-in from 203.0.113.9 (CN) for example.com. ``` The counters are `hmailserver_country_blocked_total` and `hmailserver_country_blocked_by_country_total{country="CN"}`, and the report is `GET /api/v1/reports/countries`. **Setting it up.** In the Control Deck, **Settings → Server** holds the server's settings and **Domains → (a domain) → Country blocking** the domain's policy. From a shell: ``` hmctl settings set --set geoip_database_file=/var/lib/GeoIP/GeoLite2-Country.mmdb hmctl settings set --set country_policy_smtp=block:CN,RU hmctl domain update example.com --set active=true --set country_policy=allow:GB,IE ``` The REST keys are `country_policy_smtp`, `country_policy_imap`, `country_policy_pop3`, `country_policy_http` and, on a domain, `country_policy`. Over COM (Windows only) it is `Settings.CountryPolicy("smtp")` and `Domain.CountryPolicy`. **What it does not do.** It is not a per-message control. It says nothing about a proxy: [behind HAProxy with the PROXY protocol](/topic/28240), the address judged is the proxy's, so put the policy on the proxy. And it is only as true as the database; it is not an authentication control. ### Impersonation protection *Since 6.3.4 (schema 6087).* Per domain, off on every existing and new domain. It runs at local delivery on the recipient's own copy, only on mail from **outside**, and makes two checks. **A borrowed name.** The `From` display name, folded, equals the name of an active account of the recipient's domain or of a **protected person**, while the address is not theirs. **A lookalike domain.** The `From` domain, and the envelope sender's when it differs, is equal to a hosted domain, an alias domain or a **protected partner** after the confusable folding while not equal before it (`examp1e.test`, `exarnple.test`), or is one Damerau-Levenshtein edit away inside one label (`examplee.test`, `exampel.test`). Distance 1 is the rule and 2 is not. A DMARC pass changes nothing here: the attacker publishes the lookalike's records exactly as a legitimate domain would. ``` X-hMailServer-Impersonation: name "Ada L0velace" matches Ada Lovelace (ada@example.test); from ada.lovelace@evil.test X-hMailServer-Impersonation: domain examp1e.test looks like example.test; from invoices@examp1e.test ``` **Windows and Linux differ in one step of the folding.** Unicode NFKC normalisation, which turns a full-width or ligature spelling into its letters, runs on Windows. It does **not** run on Linux, which has no Unicode tables in this build; there the step is skipped and the confusable table does the rest. The other steps run on both. **What slips past.** Word order and decorations are not folded: `Holloway, Chris` and `Chris Holloway (CEO)` are not `Chris Holloway`. A name carried only in the local part of the address is not compared. And a domain with an account named **Support** or **Info** will have every vendor's "Support" flagged. | `impersonation_action` | What happens | |---|---| | `off` (0) | Nothing is read | | `flag` (1) | The copy gets the `X-hMailServer-Impersonation` header. The subject is left alone and the sender's DKIM signature is untouched. The webmail shows a warning bar and a **Suspicious sender** mark | | `quarantine` (2) | The copy goes to the quarantine with the reason and score 0. A release does not re-check. With `QuarantineEnabled` off, the domain gets flag instead | **What is exempt:** a sender the recipient's own safe-sender list allows (or the domain's or server's allow list, where nothing nearer decides), and a `From` domain that is itself hosted here or a protected partner. Nothing else: a person's own name from their personal address at another provider is flagged until that address is on the safe-sender list. The setting is on the **Impersonation protection** tab of the Control Panel's domain dialog (Windows only), in the Deck's domain editor, and on `PUT /api/v1/domains/{domain}` as `impersonation_action`, `protected_persons` and `protected_partners`. Each list holds at most 2,000 characters. ### Where to read more The score-based pipeline these layers sit beside is in [the anti-spam stack end to end](/topic/28248), the outside-sender tag in [the topic on mail flow rules](/topic/28251), and the auto-ban a trap uses in [IP ranges, auto-ban, lockout and tarpits](/topic/28239). For the Deck itself, see [The Control Deck](/topic/28192). - Wiki, §10.10a and §10.15: https://gitlab.com/hmailserver/hmailserver/-/wikis/Stopping-Spam - https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SenderListsAndDigests.md - https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SpamTraps.md - https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/CountryBlocking.md - https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/ImpersonationProtection.md - Public documentation: https://www.progressiverobot.com/hmailserver-documentation/#104-white-lists --- Canonical: https://www.hmailserver.co.uk/topic/28249/allow-and-block-lists-digests-spam-traps-country-blocking-and-impersonation # Allow and block lists, digests, spam traps, country blocking and impersonation *Written for 6.3.4. hMailServer 6.3.7 (17 October 2026) reads a link's host that is not ASCII by UTS 46, and an HTML part's character references as a browser does, for the URL entries of the allow and block lists, so a message whose links escaped a list that way can now be tagged or held; a URL entry written in Unicode is stored in its `xn--` form and covers a link written either way, and one with a label UTS 46 refuses is refused when it is saved. In the Control Panel the Spam filtering group's six allow and block lists are under Lists, and Ctrl+K takes every old title to its page. See [the 6.3.7 announcement](/topic/28277).* This topic covers five layers that sit beside the score-based spam tests rather than inside them: the allow and block lists at three levels, the quarantine digest, spam traps, country blocking and impersonation protection. All five are new since 6.3.4, and the one thing to know first is that every one of them is off, or inert, until an administrator sets it: a server upgraded to 6.3.4 and left alone has no list entries, sends no digest, has no traps, judges nothing by country and checks no display name. ### The allow and block list: one table, three levels *Since 6.3.4 (schema 6046).* The older lists (the anti-spam white list, **Blocked senders**, the greylisting white list) act during the SMTP conversation, for every recipient at once. The allow and block list is per recipient. It is one table, `hm_senderlist`, with a scope on each row: each account's own safe and blocked senders, a domain's list, and the server's. | Entry | Covers | |---|---| | `someone@example.com` | That one address, in any case. No wildcards | | `example.com` | `example.com` and every subdomain; not `notexample.com`, not `example.com.attacker.net` | | `@example.com` | The same as `example.com`, and stored as `example.com` | | `https://bad.example/anything` | A URL: stored as `bad.example/`, covering a link in the message to that host or a subdomain, whoever sent it | An entry covers a message when it matches **either** the envelope sender **or** the address in the `From` header, because mail sent through a mailing service carries the service's address in the envelope and the brand's in the header. A message with the null sender is never decided by any list. **URL entries are in 6.3.4.** CHANGELOG's 6.3.4 section lists them, and they are consulted only when neither address decided anything. The closing section of [docs/SenderListsAndDigests.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SenderListsAndDigests.md), which says "URLs are not list entries", is older text left in the file. **What allow and block do.** The lists are read at local delivery, once per recipient, after the spam tests have run. | | The message was classified as spam | It was not | |---|---|---| | **Allowed** | Delivered unmarked for this recipient: the spam flag, the `X-hMailServer-Spam` and reason headers and the subject tag are removed from this copy | Delivered as it is | | **Blocked** | Held in the quarantine for this recipient when the quarantine is on; marked as spam and delivered when it is not | The same | A block **never deletes**, and no score is recorded on a blocked message, because nothing measured it. A message released from the quarantine is never held again by a list. The other recipients of the same message are not affected. **Precedence is nearest first**: the account, then its domain, then the server. The first of the three that has anything to say about the sender is the only one consulted, so an account's allow beats a domain's block. That is the reverse of Microsoft 365's order, and deliberate: the person the message is addressed to can tell whether they wanted it. An administrator who needs a sender stopped for everyone still has **Blocked senders**, which nobody's own list overrules. Inside one list the more specific entry wins (an address beats a domain, a longer domain beats a shorter one), and on an exact tie the block wins. **Neither address is proof.** Both are whatever the sender wrote, so an allow entry is as forgeable as the address it names. The document's advice: keep allow entries to correspondents whose domains publish DMARC, and prefer a single address to a whole domain. | List | Webmail | Control Panel (Windows) | Control Deck | REST | |---|---|---|---|---| | The account's own | Settings → **Safe senders** | none | none | `/api/v1/me/sender-list` | | A domain's | none | Domain → **Allow and block list** tab | Domain editor → **Allow and block list** | `/api/v1/domains/{domain}/sender-list` | | The server's | none | Spam & virus filtering → **Allow and block list** | Lists → **Allow and block list** | `/api/v1/sender-list` | Over COM (Windows only) the domain's list is `Domain.SenderList` and the server's `Settings.AntiSpam.SenderList`. There is no administrator path to an account's own list. **Block this sender** and **Report phishing** in the webmail now add the sender to the account's blocked senders; they used to write a rule into the Sieve script, and rules written that way are left where they are. A list holds at most a thousand entries; adding a value already there rewrites the entry (`200` rather than `201`). Expiry can be set over REST, in the Deck and in the Control Panel's two list editors (a switch and a date); the webmail does not offer it. [docs/SenderListsAndDigests.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SenderListsAndDigests.md) still says the Control Panel has no expiry; that passage predates the change, which the repository's roadmap records as made after 6.3.3. ### The quarantine digest Once a day, one plain-text message goes to each account of a domain that has something **newly** held for it, listing each message's subject, sender, when it was held and why, with a link that delivers each entry, a link that delivers everything listed, and a footer link that stops the digest (also the `List-Unsubscribe` header). An empty digest is never sent, and an account that has had one is not sent another for twenty hours. The message comes from the domain's postmaster with the null envelope sender and `Auto-Submitted: auto-generated`, and never asks for a password. | Setting | Default | |---|---| | `QuarantineDigestBaseUrl`, the address the links point at, such as `https://mail.example.com` | empty: **no digest is sent** | | `QuarantineDigestLinkHours`, 1 to 720 | 72 | | The domain's switch and hour (`quarantine_digest`, `quarantine_digest_hour`; 0 to 23, local time), which an account may override | off; 7 | All three are in the database, none in `hMailServer.ini`. The quarantine itself (`QuarantineEnabled`) has to be on, and the address must be an absolute `http://` or `https://` URL of the listener the webmail is served from. **When it is sent.** The task wakes every five minutes and, within the digest hour, sends only its share, so a domain of a thousand accounts is spread across the hour. At most five hundred digests go in one run. Over COM (Windows only), `AntiSpam.SendQuarantineDigests()` sends now, ignoring the hour and the twenty-hour rule and nothing else. **The links.** Each is 32 random bytes recorded against one account and one entry, usable once, and no longer valid after `QuarantineDigestLinkHours`. Following a link does not act on its own: it opens a page with one button, because mail security products follow every link in a message, and a link that released mail on its own would have one of them releasing every held message. Stopping the digest does not stop mail being held. ### Spam traps *Since 6.3.4 (schema 6058).* A trap is an address that was never real and never given to anybody, so a message to one comes from a harvested, bought or guessed list. At `RCPT TO` a trap is refused **exactly as any unknown address is**, `550 Unknown user`, made by the same code, so the sender cannot tell. The check runs before the delivery lookups, so a catch-all, greylisting or a backup-MX callout cannot answer first. | | What happens | Setting | |---|---|---| | The connecting address | Banned through the server's own auto-ban, as an expiring range named `Auto-ban: spam trap
` | `SpamTrapBanMinutes`, 1440. 0 bans nobody | | The envelope sender | Added to the server's block list with an expiry, **only when its domain's SPF passes for the connecting address** | `SpamTrapBlockDays`, 30. 0 blocks nobody | | The hit | Recorded in `hm_spamtraphits` and counted in `hmailserver_spam_trap_hits_total` | `SpamTrapHitRetentionDays`, 90 | The application log gets one line per hit and one per ban: ``` "Auto-ban: 203.0.113.9 is banned for 1440 minutes after addressing the spam trap never-used@example.com" ``` [The packaged fail2ban filter](/topic/28220) matches that line, so a Linux installation that runs fail2ban with the packaged jail turned on bans the address at its own firewall as well. The jail ships disabled; the wiki's Installing on Linux page turns it on with: ``` printf '[hmailserver]\nenabled = true\n' > /etc/fail2ban/jail.d/hmailserver.local systemctl reload fail2ban ``` **The SPF condition.** As first written, in the source tree, a trap put the envelope sender of every message sent to it on the block list, whatever that sender was; spam traps are new in 6.3.4, which carries the fix, so no release did. CHANGELOG records why that changed: the envelope is whatever the client types, so one connection naming a bank's address could have had the bank's real mail held or marked for every mailbox. Now only an SPF **pass** blocks; the address is banned either way. The wiki's summary in Stopping Spam §10.15 does not mention the condition; [docs/SpamTraps.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SpamTraps.md) and CHANGELOG do. **Who is never banned:** an authenticated session, an address whose IP range does not run the spam tests, an address or sender the anti-spam white list covers, an incoming relay, and an address on `AutoBanNeverBan`. The sender is not blocked for the null sender, for a sender claiming a domain this server hosts, or where the server's allow list already names it. **Training.** A trap hit trains nothing by itself, because no content arrives. With `SpamTrapTeachSpamAssassin` on (off by default; it needs SpamAssassin enabled and a spamd started with `--allow-tell`), a message sent to a real mailbox in the same transaction is taught to spamd as spam. **Declaring a trap.** It must be an address in the domain it is declared on, and one nobody uses: an account, alias or list with that address is refused, and none can be given a trap's address while the trap exists. | Where | How | |---|---| | Control Deck | Domain editor → **Spam traps**; the **Spam traps** view for the record across every domain | | REST | `GET`/`POST /api/v1/domains/{domain}/spam-traps`, `DELETE /api/v1/spam-traps/{address}`, `GET`/`DELETE /api/v1/spam-traps/hits` | | Command line | `hmctl spamtrap create --domain example.com --set address=never-used@example.com`, `hmctl traphits list` | | PowerShell | `Add-HmSpamTrap`, `Get-HmSpamTrapHit` | | COM (Windows only) | `Domain.AddSpamTrap`, `Utilities.GetSpamTrapHits` | A good trap is an address never published anywhere, or one published once, long ago, and unused for years (give it a year of refusals first). A bad trap is a typo of a real address such as `saels@`, or any address in a domain with a catch-all: there every other unknown address is accepted and the trap is not, which is the difference a sender can measure. Clearing the record of hits undoes nothing; the bans and blocks stay until they expire or are removed. The traps are in the backup with their domain; the record of hits is not. ### Country blocking *Since 6.3.4 (schema 6052).* Two kinds of policy, in one grammar. A policy **per listener family** (SMTP, IMAP, POP3, and the HTTP listeners) is applied when a connection is accepted: before the greeting, before the TLS handshake and before anything touches the database. A policy **per domain** is applied when somebody signs in to an account of that domain, on any protocol, and covers OAuth2 bearer tokens, client certificates, Kerberos tickets, passkeys and single sign-on as well as passwords. | Value | What it means | |---|---| | *(empty)* | Nothing is judged by country. The default everywhere | | `allow:GB,IE,FR` | Only these countries may connect, or sign in | | `block:CN,RU` | These countries may not connect, or sign in | **The database is yours to supply.** Point `GeoIpDatabaseFile` at a MaxMind `.mmdb` file, such as `/var/lib/GeoIP/GeoLite2-Country.mmdb` or `C:\GeoIP\GeoLite2-Country.mmdb`. The server ships none and never fetches one. Replace the file in place and the server reloads it within five minutes. `GeoIpDatabaseMaxAgeDays` (45) is how old the file's build date may be before the log says so; a stale database is still used. **What is exempt.** `CountryBlockingExemptRanges` defaults to the loopback addresses and every private network: ``` 127.0.0.0/8,::1/128,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,169.254.0.0/16,100.64.0.0/10,fe80::/10,fc00::/7 ``` There is no second list compiled into the server. If you empty it, do so with `country_policy_http` off, or the Control Deck is the next thing you lose. An open session is never disturbed; a policy takes effect on the next connection. **Which way it fails.** With no database at all (none configured, or the file missing or unreadable) **everything is allowed**, whatever the policies say, and the reason is logged once. With a database loaded that has no country for an address, an allow list refuses it and a block list allows it. **What a refused client is told: nothing.** A refused connection is closed with no banner. A refused sign-in gets the ordinary invalid-credentials reply, byte-identical to a wrong password, and is counted against neither the account's lockout nor the address's auto-ban, since the person refused is very often the account's real owner, travelling. Your copy is in the application log: ``` Country blocking: refused SMTP from 203.0.113.9 (CN) for connection. Country blocking: refused sign-in from 203.0.113.9 (CN) for example.com. ``` The counters are `hmailserver_country_blocked_total` and `hmailserver_country_blocked_by_country_total{country="CN"}`, and the report is `GET /api/v1/reports/countries`. **Setting it up.** In the Control Deck, **Settings → Server** holds the server's settings and **Domains → (a domain) → Country blocking** the domain's policy. From a shell: ``` hmctl settings set --set geoip_database_file=/var/lib/GeoIP/GeoLite2-Country.mmdb hmctl settings set --set country_policy_smtp=block:CN,RU hmctl domain update example.com --set active=true --set country_policy=allow:GB,IE ``` The REST keys are `country_policy_smtp`, `country_policy_imap`, `country_policy_pop3`, `country_policy_http` and, on a domain, `country_policy`. Over COM (Windows only) it is `Settings.CountryPolicy("smtp")` and `Domain.CountryPolicy`. **What it does not do.** It is not a per-message control. It says nothing about a proxy: [behind HAProxy with the PROXY protocol](/topic/28240), the address judged is the proxy's, so put the policy on the proxy. And it is only as true as the database; it is not an authentication control. ### Impersonation protection *Since 6.3.4 (schema 6087).* Per domain, off on every existing and new domain. It runs at local delivery on the recipient's own copy, only on mail from **outside**, and makes two checks. **A borrowed name.** The `From` display name, folded, equals the name of an active account of the recipient's domain or of a **protected person**, while the address is not theirs. **A lookalike domain.** The `From` domain, and the envelope sender's when it differs, is equal to a hosted domain, an alias domain or a **protected partner** after the confusable folding while not equal before it (`examp1e.test`, `exarnple.test`), or is one Damerau-Levenshtein edit away inside one label (`examplee.test`, `exampel.test`). Distance 1 is the rule and 2 is not. A DMARC pass changes nothing here: the attacker publishes the lookalike's records exactly as a legitimate domain would. ``` X-hMailServer-Impersonation: name "Ada L0velace" matches Ada Lovelace (ada@example.test); from ada.lovelace@evil.test X-hMailServer-Impersonation: domain examp1e.test looks like example.test; from invoices@examp1e.test ``` **Windows and Linux differ in one step of the folding.** Unicode NFKC normalisation, which turns a full-width or ligature spelling into its letters, runs on Windows. It does **not** run on Linux, which has no Unicode tables in this build; there the step is skipped and the confusable table does the rest. The other steps run on both. **What slips past.** Word order and decorations are not folded: `Holloway, Chris` and `Chris Holloway (CEO)` are not `Chris Holloway`. A name carried only in the local part of the address is not compared. And a domain with an account named **Support** or **Info** will have every vendor's "Support" flagged. | `impersonation_action` | What happens | |---|---| | `off` (0) | Nothing is read | | `flag` (1) | The copy gets the `X-hMailServer-Impersonation` header. The subject is left alone and the sender's DKIM signature is untouched. The webmail shows a warning bar and a **Suspicious sender** mark | | `quarantine` (2) | The copy goes to the quarantine with the reason and score 0. A release does not re-check. With `QuarantineEnabled` off, the domain gets flag instead | **What is exempt:** a sender the recipient's own safe-sender list allows (or the domain's or server's allow list, where nothing nearer decides), and a `From` domain that is itself hosted here or a protected partner. Nothing else: a person's own name from their personal address at another provider is flagged until that address is on the safe-sender list. The setting is on the **Impersonation protection** tab of the Control Panel's domain dialog (Windows only), in the Deck's domain editor, and on `PUT /api/v1/domains/{domain}` as `impersonation_action`, `protected_persons` and `protected_partners`. Each list holds at most 2,000 characters. ### Where to read more The score-based pipeline these layers sit beside is in [the anti-spam stack end to end](/topic/28248), the outside-sender tag in [the topic on mail flow rules](/topic/28251), and the auto-ban a trap uses in [IP ranges, auto-ban, lockout and tarpits](/topic/28239). For the Deck itself, see [The Control Deck](/topic/28192). - Wiki, §10.10a and §10.15: https://gitlab.com/hmailserver/hmailserver/-/wikis/Stopping-Spam - https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SenderListsAndDigests.md - https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SpamTraps.md - https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/CountryBlocking.md - https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/ImpersonationProtection.md - Public documentation: https://www.progressiverobot.com/hmailserver-documentation/#104-white-lists --- Canonical: https://www.hmailserver.co.uk/topic/28250/viruses-and-dangerous-attachments-clamav-custom-scanners-the-sandbox-sealed-mail # Viruses and dangerous attachments: ClamAV, custom scanners, the sandbox, sealed mail *Written for 6.3.4. hMailServer 6.3.7 (17 October 2026) calls the anti-virus action that strips attachments Remove all attachments and deliver the text (Delete infected attachments only in 6.3.6's Control Panel), and under it a message whose one part is an attachment, and an S/MIME message signed as one part, now arrive as the action's note alone, where every version to 6.3.6 delivered them as they arrived with what the scanner had found in them; a message the server cannot read is deleted and each of its recipients sent the "message deleted" notice whatever the two notify settings say, while an encrypted message, and one that is one text part, are delivered as they are. In the Control Panel the sidebar has a Virus scanning group, and Ctrl+K takes every old title to its page. See [the 6.3.7 announcement](/topic/28277).* hMailServer does not scan for viruses itself. It hands each message to an engine you install (ClamAV, ClamWin or any command-line scanner) and acts on the answer, and nothing described here is enabled on a fresh install. This topic covers the three scanners, what happens when one finds something or cannot run, the blocked-attachment list, and what 6.3.4 added: the sandbox, and the handling of mail sealed end to end. The one thing to know first is the failure mode: a configuration that **looks enabled and is inert**. A scanner switched on with a wrong host or a missing executable does not refuse mail and does not disable itself; when every scanner has errored the verdict is `NoVirusFound`, which from the message's point of view is identical to having been examined and found clean. 6.3.4 fixed one such case in the custom scanner, described below. ### What runs, and when Scanning happens on the delivery side, not during the SMTP conversation: the message has already been accepted and spooled by the time any scanner sees it. **Which messages are scanned.** For SMTP, a message is flagged when the connecting IP's security range has the virus-protection option; both default ranges ("Internet" and "My computer") have it. For mail this server fetches over POP3 or IMAP, the flag is the external account's own "use anti-virus" setting. A flagged message larger than `AntiVirus.MaximumMessageSize` is not scanned at all, silently; the default is `0`, no limit. **Two passes.** The whole spool file is scanned first; then every MIME attachment is written to a temporary file and scanned separately, which is what matters for a scanner that does not decode MIME itself. If the message cannot be parsed for the second pass, error `HM6001` is reported and only the whole file was scanned; if one attachment cannot be written to the temp directory, `HM6002` is reported and that attachment was not examined. **Concurrency.** At most 10 scans run at once (`MaxRunningScanners`). A thread that cannot get a slot waits up to 60 seconds and then scans anyway, writing `Scanning message without an available virus scanner slot.` to the debug log. **The read-only overview.** **Spam & virus filtering → Virus scanning overview** in the Control Panel (Windows only) answers the question the settings page cannot: will an infected message be caught here. ### The three scanners All three can be enabled at once. They run in the order ClamWin, custom, ClamAV, stopping at the first that reports a virus. A scanner that errors does not stop the others. | | ClamAV (clamd) | ClamWin (`clamscan.exe`) | Custom scanner | |---|---|---|---| | Switch | `AntiVirus.ClamAVEnabled` | `AntiVirus.ClamWinEnabled` | `AntiVirus.CustomScannerEnabled` | | Default | off | off | off | | Reached by | TCP socket, `INSTREAM` | one process per file | one process per file | | Settings | `AntiVirus.ClamAVHost` (`localhost`), `AntiVirus.ClamAVPort` (`3310`) | `AntiVirus.ClamWinExecutable`, `AntiVirus.ClamWinDBFolder` | `AntiVirus.CustomScannerExecutable` | | "Infected" is | a reply matching `stream…: FOUND` | exit code 1 | exit code equal to `AntiVirus.CustomScannerReturnValue` | | Timeout | `ClamMinTimeout` 15 s to `ClamMaxTimeout` 90 s | `ExternalProcessTimeout`, 300 s | `ExternalProcessTimeout`, 300 s | | Virus name reported | the name clamd returned | always `Unknown` | always `Unknown` | **ClamAV.** Install ClamAV, run `freshclam` to download signatures, run `clamd` listening on TCP 3310, set the host and port, and press **Test ClamAV connection**. The button does three things in order: 1. It sends `nPING` and requires `PONG`, so "nothing is listening" is told apart from "something else is listening". 2. It sends `nVERSION` and reports what answered. 3. It streams a harmless sample and then the EICAR test string, both from memory, and requires the first to come back clean and the second `FOUND`. A success looks like `ClamAV 1.4.2/27700/Mon Sep 8 09:12:11 2026. Result: stream: Eicar-Signature FOUND`; a failure names the step, for example `Unable to connect to ClamAV server at localhost:3310.` **The custom scanner.** Any program that exits with a distinguishable code will do. Put `%FILE%` where the file to scan belongs; without it, the file name is appended as the single quoted argument. The Control Panel ships presets, all of them Windows programs: | Preset | Command line | "Infected" exit code | |---|---|---| | Microsoft Defender | `"C:\Program Files\Windows Defender\MpCmdRun.exe" -Scan -ScanType 3 -File "%FILE%" -DisableRemediation` | 2 | | Sophos | `"C:\Program Files\Sophos\Sophos Anti-Virus\savscan.exe" -ss -archive "%FILE%"` | 3 | | ESET | `"C:\Program Files\ESET\ESET Security\ecls.exe" "%FILE%"` | 50 | | Bitdefender | `"C:\Program Files\Bitdefender\Endpoint Security\bdscan.exe" "%FILE%"` | 1 | | Kaspersky | `"C:\Program Files (x86)\Kaspersky Lab\Kaspersky Endpoint Security\avp.com" SCAN "%FILE%"` | 2 | **The 6.3.4 fix: a bare `%FILE%`.** CHANGELOG records that a custom scanner whose command line holds a bare, unquoted `%FILE%` was handed the message file's path cut at its first space, and the default data directory, under `C:\Program Files`, has one. The scanner therefore scanned no file, answered with an exit code other than the virus one, and the message was delivered as clean. Since 6.3.4 a `%FILE%` outside double quotes is replaced by the path in quotes. One already inside them, `"%FILE%"` as the documented examples above write it, was not affected and is replaced as before. If you run an earlier release with a command line of your own, check it for an unquoted `%FILE%`. The same fix removes a trailing backslash from ClamWin's database and temporary folders, which escaped the closing quote of `--database="..."` and took the file to scan with it. **The service account must be able to run the scanner.** A new 6.3.4 installation on Windows runs the service as `NT SERVICE\hMailServer` by default; an upgrade keeps the account it had, LocalSystem unless the installer's page is answered otherwise ([docs/ServiceAccount.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/ServiceAccount.md); the wiki's "`LocalSystem` by default" describes earlier releases and upgraded servers). Under the virtual account a custom scanner or ClamWin runs as that account: it reads the message in the temp folder, which is granted, and its own program and signatures, which the Users group can normally read; a scanner that needs more has to be granted it ([Installing on Windows](/topic/28227) covers the account and its grants). Whichever account runs yours, the executable, its database directory and the temp directory must all be reachable by it. The symptom of getting it wrong is `Unable to launch the virus scanner…` reported as `HM5406`, once per message. For an engine with no command-line interface, the wiki points to an `OnAcceptMessage` event script or the HTTP filter hook. If you edit `hm_settings` by hand, note that the custom scanner's return value is stored under the misspelled name `customviursscannerreturnvalue`. ### Testing it end to end The test buttons check the scanner. To check the pipeline, send yourself a message with the EICAR test file attached and watch the application log: ```text Virus found in message from someone@example.com. Taking actions SMTPDeliverer - Message 4711: Message will be deleted (contained virus Eicar-Signature). ``` Two things spoil that test on Windows. The host antivirus eats the EICAR file before hMailServer sees it, so exclude the mail store and the temp directory from real-time scanning. And your own outbound mail is not scanned by this server if it never comes back in through SMTP from an unauthenticated range, so send the test from an external address. ### What happens when a virus is found | `AntiVirus.Action` | What happens | |---|---| | **Delete entire e-mail** (0, the default) | The message is delivered to nobody. Logged as `Message will be deleted (contained virus X).` | | **Delete infected attachments only** (1; **Remove all attachments and deliver the text** in 6.3.7's Control Panel) | The message is rebuilt, not edited: the first `text` part is kept, everything else is discarded, the `VIRUS_ATTACHMENT_REMOVED` server message is prepended to the body and the subject is prefixed with the `VIRUS_FOUND` message. Logged as `Message attachments stripped (contained virus X).` Since 6.3.7 a message whose one part is an attachment arrives as the note alone, where every version to 6.3.6 delivered it as it arrived, and one the server cannot read is deleted, each recipient sent the "message deleted" notice whatever the two notify settings say | Understand the rebuild before you choose it: an HTML alternative, inline images and every clean attachment go with the infected one. `AntiVirus.NotifySender` and `AntiVirus.NotifyReceiver` are both off by default, and the wiki's advice is to leave them off: the sender address on a virus is virtually always forged, so a notification is backscatter. ### When a scanner cannot run | Setting | Default | Meaning | |---|---|---| | `AVFailAction` | 0 | `0` delivers the message anyway. `1` never delivers unscanned mail | | `AVFailRetryMinutes` | 15 | Minutes between scan attempts while a message is held | | `AVFailMaxHolds` | 16 | Holds before the message is returned to its sender. `0` tells the sender immediately | They are on **Anti-virus settings → General**, card **When a scanner cannot run**. The wiki describes them as stored in `hMailServer.INI`; since 6.3.4 the database is the settings store and a `[Settings]` line edited in the file is put back, so change them in the Control Panel, over REST (`PUT /api/v1/settings/ini/{name}`) or, on Linux, with `hmailserver --set-setting NAME=VALUE`. At the defaults, `AVFailAction` 1 gives a message about four hours to be scanned before its sender is told. The hold counter is the policy's own, not the message's try counter, because incrementing that would make the pass that finally scans the message skip DKIM signing. It lives in memory, so a restart gives every held message a fresh budget, and it is bounded at 100,000 tracked messages; beyond that messages are still held, never delivered unscanned, and `HM6195` is reported. Every scanner error is reported as `HM5406` with the failing scanner's own text. ### Blocked attachments **Spam & virus filtering → Blocked attachments** refuses files by name pattern. The master switch is `AntiVirus.EnableAttachmentBlocking`, default off. The wiki's starting list: ```text *.exe *.scr *.pif *.bat *.cmd *.com *.vbs *.js *.jse *.wsf *.wsh *.hta *.jar *.msi *.msp *.cpl *.lnk *.reg *.ps1 *.iso *.img ``` **The attachment is not deleted.** Its file name has `.txt` appended and its content is replaced by the `ATTACHMENT_REMOVED` server message, which names the original file through the `%MACRO_FILE%` macro, so the recipient receives `invoice.exe.txt` containing an explanation. Matching is a case-insensitive wildcard over the declared file name. It does not look inside archives and cannot see the real type of a file whose extension lies: it is a policy control, not a scanner. **Since 6.3.4 the rewrite is logged.** A rewritten message produces `VirusScanner - Message 1234 from a@example.com: the attachment(s) invoice.exe matched the blocked-attachment list. Their content has been replaced with a notice and the message rewritten.` Before this the rewrite left no record anywhere but inside the message itself. ### Sealed mail: what the list and the scanner do with it *Since 6.3.4.* A message encrypted end to end, S/MIME or PGP/MIME, is **opaque** to the server, which decides that from the body, never from the `Content-Type` label, so a message only labelled encrypted is blocked, stripped and scanned like any other. What opaque means exactly, and what the rest of the server does with such a message, is in [the topic on end-to-end encryption and sealed mail](/topic/28260). For the list and the scanner, none of it is a setting: - **The blocked-attachment list never rewrites an opaque message.** In 6.3.3 and earlier blocking `*.asc`, `*.p7m`, `*.p7s` or `*.*` replaced the content of every sealed message with the notice, with no log line, and the plaintext existed nowhere else. It is now delivered as it arrived, and the application log says it was not checked against the list. A signed message is checked like any other, with its detached signature left alone. - **The scanner still scans it, and what it examines is ciphertext**, so a clean result is not a verdict on the message; the log says its content `COULD NOT BE EXAMINED for viruses`. It is deliberately not treated as a scanner failure, so a server with `AVFailAction` 1 does not hold every encrypted message it receives; a policy that unscannable mail must not be delivered belongs in a rule or a Sieve script, not the scanner. - **The "delete attachments" action never strips an encrypted message.** When the scanner objects to one, the message is left as it arrived and the log ends *Set the anti-virus action to delete the message if such a message must not be delivered*. That is not a quiet pass: the message still carries what the scanner objected to. To 6.3.6 this held for every opaque message; since 6.3.7 an S/MIME message signed as one part (`application/pkcs7-mime` holding signed-data, which is opaque but not encrypted) is replaced by the action's note, as any other single-part attachment is. ### The sandbox *Since 6.3.4 (schema 6091).* ClamAV says whether a file is known to be bad. What an unknown file does when it is opened takes a sandbox, a service outside this server which you run or subscribe to. **It does nothing until you name one**: `SandboxEnabled` is off, and cannot be saved on while `SandboxSubmitUrl` or `SandboxVerdictUrl` is empty. Both addresses must be https, or plain http to this machine. After the anti-virus scan and before the rules, a message that no signed-in user of this server submitted, with an attachment whose extension or media type is in `SandboxTypes`, is **held**: in no mailbox, not in the quarantine, and in nobody's digest. The default types are: ``` exe,dll,scr,js,jse,vbs,wsf,hta,msi,bat,cmd,ps1,jar,lnk,iso,img,docm,xlsm,pptm, doc,xls,ppt,rtf,pdf,zip,7z,rar ``` | The sandbox says | What happens | |---|---| | every file clean | delivered, with an `X-hMailServer-Sandbox` header reading `clean ( file(s), s)` | | any file malicious, or suspicious at or above `SandboxScoreThreshold` (70) | quarantined with the verdict as the reason; returned to its sender with 5.7.1 while `QuarantineEnabled` is off | | an error, once every file has answered, or no answer within `SandboxTimeoutMinutes` (15) | `SandboxTimeoutAction`: `deliver` (0, the default) delivers with a header saying so; `bounce` (1) returns it to its sender | **The contract.** One POST per file to `SandboxSubmitUrl` as `multipart/form-data` with four fields (`file`, `filename`, `sha256`, `message_id`), then a GET to `SandboxVerdictUrl` every `SandboxPollSeconds` (30), answered with JSON such as `{"state":"done","verdict":"clean","score":3,"detail":"opened, nothing happened"}`. With `SandboxToken` set, every request carries `Authorization: Bearer `; the token is not in the backup archive. **No vendor adapter is shipped.** The document notes that a service which shares what it is sent, as VirusTotal shares uploads with its subscribers, is a place the attachments of the mail you receive go. **What is exempt.** What your own signed-in users submit (authenticated SMTP, the webmail, the send API) is not uploaded unless `SandboxInternalMail` is on. The document states the threat model of that default: a stolen password used to send attachments to colleagues is not examined by the sandbox, though ClamAV, the blocked-attachment list and compromised-account detection still apply. **Limits.** An attachment larger than `SandboxMaxSizeKB` (20480) is not submitted. A message with more files to submit than `SandboxMaxFiles` (10) is not held, and `SandboxTimeoutAction` decides it at once. Archives are submitted whole; links are not opened anywhere. A clean or malicious verdict is remembered by the file's SHA-256 for `SandboxCacheHours` (24), server-wide. **What the administrator sees.** `GET /api/v1/sandbox/held` lists what is held; `POST /api/v1/sandbox/held/{queue_id}/release` delivers without the verdict and `DELETE /api/v1/sandbox/held/{queue_id}` discards, both on the audit trail. The Control Deck's queue page has a *Held for a sandbox* box, and `hmctl queue list` has a `held` column. The settings are on the Control Panel's Anti-virus page, *Sandbox* tab (Windows only), and on `PUT /api/v1/settings`. ### Rules, and the message store Since 6.3.4 the mail flow rules can match the attachment's name and, in the outbound content policy, its type read from its bytes, so a rule can hold or refuse what the blocked-attachment list would only rename; that is in [the topic on mail flow rules](/topic/28251). [The message store options added in 6.3.4](/topic/28233) change nothing for a scanner: a message is scanned as plain RFC 822 in the queue and only placed in the mailbox in its stored form afterwards. A scanner pointed at the data directory from outside the server reads a compressed or encrypted container, not a message. ### Error codes | Code | Meaning | |---|---| | `HM5406` | An enabled scanner errored; the detail names the scanner and the failure | | `HM6001` | The message could not be parsed, so only the whole file was scanned | | `HM6002` | An attachment could not be written to the temp directory for scanning | | `HM6195` | 100,000 messages are already held for want of a scan; further holds are not counted | | `HM7272` | The sandbox called a message malicious and the quarantine could not store it; it is held again, never delivered | ### Where to read more - Wiki: https://gitlab.com/hmailserver/hmailserver/-/wikis/Stopping-Viruses - https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Sandbox.md - https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SealedMail.md - Public documentation: https://www.progressiverobot.com/hmailserver-documentation/#11-stopping-viruses, https://www.progressiverobot.com/hmailserver-documentation/#111-clamav and https://www.progressiverobot.com/hmailserver-documentation/#112-blocked-attachments - [Upgrading to 6.3.4](/topic/28193) and [the 6.3.4 security announcement](/topic/28218) --- Canonical: https://www.hmailserver.co.uk/topic/28250/viruses-and-dangerous-attachments-clamav-custom-scanners-the-sandbox-sealed-mail # Viruses and dangerous attachments: ClamAV, custom scanners, the sandbox, sealed mail *Written for 6.3.4. hMailServer 6.3.7 (17 October 2026) calls the anti-virus action that strips attachments Remove all attachments and deliver the text (Delete infected attachments only in 6.3.6's Control Panel), and under it a message whose one part is an attachment, and an S/MIME message signed as one part, now arrive as the action's note alone, where every version to 6.3.6 delivered them as they arrived with what the scanner had found in them; a message the server cannot read is deleted and each of its recipients sent the "message deleted" notice whatever the two notify settings say, while an encrypted message, and one that is one text part, are delivered as they are. In the Control Panel the sidebar has a Virus scanning group, and Ctrl+K takes every old title to its page. See [the 6.3.7 announcement](/topic/28277).* hMailServer does not scan for viruses itself. It hands each message to an engine you install (ClamAV, ClamWin or any command-line scanner) and acts on the answer, and nothing described here is enabled on a fresh install. This topic covers the three scanners, what happens when one finds something or cannot run, the blocked-attachment list, and what 6.3.4 added: the sandbox, and the handling of mail sealed end to end. The one thing to know first is the failure mode: a configuration that **looks enabled and is inert**. A scanner switched on with a wrong host or a missing executable does not refuse mail and does not disable itself; when every scanner has errored the verdict is `NoVirusFound`, which from the message's point of view is identical to having been examined and found clean. 6.3.4 fixed one such case in the custom scanner, described below. ### What runs, and when Scanning happens on the delivery side, not during the SMTP conversation: the message has already been accepted and spooled by the time any scanner sees it. **Which messages are scanned.** For SMTP, a message is flagged when the connecting IP's security range has the virus-protection option; both default ranges ("Internet" and "My computer") have it. For mail this server fetches over POP3 or IMAP, the flag is the external account's own "use anti-virus" setting. A flagged message larger than `AntiVirus.MaximumMessageSize` is not scanned at all, silently; the default is `0`, no limit. **Two passes.** The whole spool file is scanned first; then every MIME attachment is written to a temporary file and scanned separately, which is what matters for a scanner that does not decode MIME itself. If the message cannot be parsed for the second pass, error `HM6001` is reported and only the whole file was scanned; if one attachment cannot be written to the temp directory, `HM6002` is reported and that attachment was not examined. **Concurrency.** At most 10 scans run at once (`MaxRunningScanners`). A thread that cannot get a slot waits up to 60 seconds and then scans anyway, writing `Scanning message without an available virus scanner slot.` to the debug log. **The read-only overview.** **Spam & virus filtering → Virus scanning overview** in the Control Panel (Windows only) answers the question the settings page cannot: will an infected message be caught here. ### The three scanners All three can be enabled at once. They run in the order ClamWin, custom, ClamAV, stopping at the first that reports a virus. A scanner that errors does not stop the others. | | ClamAV (clamd) | ClamWin (`clamscan.exe`) | Custom scanner | |---|---|---|---| | Switch | `AntiVirus.ClamAVEnabled` | `AntiVirus.ClamWinEnabled` | `AntiVirus.CustomScannerEnabled` | | Default | off | off | off | | Reached by | TCP socket, `INSTREAM` | one process per file | one process per file | | Settings | `AntiVirus.ClamAVHost` (`localhost`), `AntiVirus.ClamAVPort` (`3310`) | `AntiVirus.ClamWinExecutable`, `AntiVirus.ClamWinDBFolder` | `AntiVirus.CustomScannerExecutable` | | "Infected" is | a reply matching `stream…: FOUND` | exit code 1 | exit code equal to `AntiVirus.CustomScannerReturnValue` | | Timeout | `ClamMinTimeout` 15 s to `ClamMaxTimeout` 90 s | `ExternalProcessTimeout`, 300 s | `ExternalProcessTimeout`, 300 s | | Virus name reported | the name clamd returned | always `Unknown` | always `Unknown` | **ClamAV.** Install ClamAV, run `freshclam` to download signatures, run `clamd` listening on TCP 3310, set the host and port, and press **Test ClamAV connection**. The button does three things in order: 1. It sends `nPING` and requires `PONG`, so "nothing is listening" is told apart from "something else is listening". 2. It sends `nVERSION` and reports what answered. 3. It streams a harmless sample and then the EICAR test string, both from memory, and requires the first to come back clean and the second `FOUND`. A success looks like `ClamAV 1.4.2/27700/Mon Sep 8 09:12:11 2026. Result: stream: Eicar-Signature FOUND`; a failure names the step, for example `Unable to connect to ClamAV server at localhost:3310.` **The custom scanner.** Any program that exits with a distinguishable code will do. Put `%FILE%` where the file to scan belongs; without it, the file name is appended as the single quoted argument. The Control Panel ships presets, all of them Windows programs: | Preset | Command line | "Infected" exit code | |---|---|---| | Microsoft Defender | `"C:\Program Files\Windows Defender\MpCmdRun.exe" -Scan -ScanType 3 -File "%FILE%" -DisableRemediation` | 2 | | Sophos | `"C:\Program Files\Sophos\Sophos Anti-Virus\savscan.exe" -ss -archive "%FILE%"` | 3 | | ESET | `"C:\Program Files\ESET\ESET Security\ecls.exe" "%FILE%"` | 50 | | Bitdefender | `"C:\Program Files\Bitdefender\Endpoint Security\bdscan.exe" "%FILE%"` | 1 | | Kaspersky | `"C:\Program Files (x86)\Kaspersky Lab\Kaspersky Endpoint Security\avp.com" SCAN "%FILE%"` | 2 | **The 6.3.4 fix: a bare `%FILE%`.** CHANGELOG records that a custom scanner whose command line holds a bare, unquoted `%FILE%` was handed the message file's path cut at its first space, and the default data directory, under `C:\Program Files`, has one. The scanner therefore scanned no file, answered with an exit code other than the virus one, and the message was delivered as clean. Since 6.3.4 a `%FILE%` outside double quotes is replaced by the path in quotes. One already inside them, `"%FILE%"` as the documented examples above write it, was not affected and is replaced as before. If you run an earlier release with a command line of your own, check it for an unquoted `%FILE%`. The same fix removes a trailing backslash from ClamWin's database and temporary folders, which escaped the closing quote of `--database="..."` and took the file to scan with it. **The service account must be able to run the scanner.** A new 6.3.4 installation on Windows runs the service as `NT SERVICE\hMailServer` by default; an upgrade keeps the account it had, LocalSystem unless the installer's page is answered otherwise ([docs/ServiceAccount.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/ServiceAccount.md); the wiki's "`LocalSystem` by default" describes earlier releases and upgraded servers). Under the virtual account a custom scanner or ClamWin runs as that account: it reads the message in the temp folder, which is granted, and its own program and signatures, which the Users group can normally read; a scanner that needs more has to be granted it ([Installing on Windows](/topic/28227) covers the account and its grants). Whichever account runs yours, the executable, its database directory and the temp directory must all be reachable by it. The symptom of getting it wrong is `Unable to launch the virus scanner…` reported as `HM5406`, once per message. For an engine with no command-line interface, the wiki points to an `OnAcceptMessage` event script or the HTTP filter hook. If you edit `hm_settings` by hand, note that the custom scanner's return value is stored under the misspelled name `customviursscannerreturnvalue`. ### Testing it end to end The test buttons check the scanner. To check the pipeline, send yourself a message with the EICAR test file attached and watch the application log: ```text Virus found in message from someone@example.com. Taking actions SMTPDeliverer - Message 4711: Message will be deleted (contained virus Eicar-Signature). ``` Two things spoil that test on Windows. The host antivirus eats the EICAR file before hMailServer sees it, so exclude the mail store and the temp directory from real-time scanning. And your own outbound mail is not scanned by this server if it never comes back in through SMTP from an unauthenticated range, so send the test from an external address. ### What happens when a virus is found | `AntiVirus.Action` | What happens | |---|---| | **Delete entire e-mail** (0, the default) | The message is delivered to nobody. Logged as `Message will be deleted (contained virus X).` | | **Delete infected attachments only** (1; **Remove all attachments and deliver the text** in 6.3.7's Control Panel) | The message is rebuilt, not edited: the first `text` part is kept, everything else is discarded, the `VIRUS_ATTACHMENT_REMOVED` server message is prepended to the body and the subject is prefixed with the `VIRUS_FOUND` message. Logged as `Message attachments stripped (contained virus X).` Since 6.3.7 a message whose one part is an attachment arrives as the note alone, where every version to 6.3.6 delivered it as it arrived, and one the server cannot read is deleted, each recipient sent the "message deleted" notice whatever the two notify settings say | Understand the rebuild before you choose it: an HTML alternative, inline images and every clean attachment go with the infected one. `AntiVirus.NotifySender` and `AntiVirus.NotifyReceiver` are both off by default, and the wiki's advice is to leave them off: the sender address on a virus is virtually always forged, so a notification is backscatter. ### When a scanner cannot run | Setting | Default | Meaning | |---|---|---| | `AVFailAction` | 0 | `0` delivers the message anyway. `1` never delivers unscanned mail | | `AVFailRetryMinutes` | 15 | Minutes between scan attempts while a message is held | | `AVFailMaxHolds` | 16 | Holds before the message is returned to its sender. `0` tells the sender immediately | They are on **Anti-virus settings → General**, card **When a scanner cannot run**. The wiki describes them as stored in `hMailServer.INI`; since 6.3.4 the database is the settings store and a `[Settings]` line edited in the file is put back, so change them in the Control Panel, over REST (`PUT /api/v1/settings/ini/{name}`) or, on Linux, with `hmailserver --set-setting NAME=VALUE`. At the defaults, `AVFailAction` 1 gives a message about four hours to be scanned before its sender is told. The hold counter is the policy's own, not the message's try counter, because incrementing that would make the pass that finally scans the message skip DKIM signing. It lives in memory, so a restart gives every held message a fresh budget, and it is bounded at 100,000 tracked messages; beyond that messages are still held, never delivered unscanned, and `HM6195` is reported. Every scanner error is reported as `HM5406` with the failing scanner's own text. ### Blocked attachments **Spam & virus filtering → Blocked attachments** refuses files by name pattern. The master switch is `AntiVirus.EnableAttachmentBlocking`, default off. The wiki's starting list: ```text *.exe *.scr *.pif *.bat *.cmd *.com *.vbs *.js *.jse *.wsf *.wsh *.hta *.jar *.msi *.msp *.cpl *.lnk *.reg *.ps1 *.iso *.img ``` **The attachment is not deleted.** Its file name has `.txt` appended and its content is replaced by the `ATTACHMENT_REMOVED` server message, which names the original file through the `%MACRO_FILE%` macro, so the recipient receives `invoice.exe.txt` containing an explanation. Matching is a case-insensitive wildcard over the declared file name. It does not look inside archives and cannot see the real type of a file whose extension lies: it is a policy control, not a scanner. **Since 6.3.4 the rewrite is logged.** A rewritten message produces `VirusScanner - Message 1234 from a@example.com: the attachment(s) invoice.exe matched the blocked-attachment list. Their content has been replaced with a notice and the message rewritten.` Before this the rewrite left no record anywhere but inside the message itself. ### Sealed mail: what the list and the scanner do with it *Since 6.3.4.* A message encrypted end to end, S/MIME or PGP/MIME, is **opaque** to the server, which decides that from the body, never from the `Content-Type` label, so a message only labelled encrypted is blocked, stripped and scanned like any other. What opaque means exactly, and what the rest of the server does with such a message, is in [the topic on end-to-end encryption and sealed mail](/topic/28260). For the list and the scanner, none of it is a setting: - **The blocked-attachment list never rewrites an opaque message.** In 6.3.3 and earlier blocking `*.asc`, `*.p7m`, `*.p7s` or `*.*` replaced the content of every sealed message with the notice, with no log line, and the plaintext existed nowhere else. It is now delivered as it arrived, and the application log says it was not checked against the list. A signed message is checked like any other, with its detached signature left alone. - **The scanner still scans it, and what it examines is ciphertext**, so a clean result is not a verdict on the message; the log says its content `COULD NOT BE EXAMINED for viruses`. It is deliberately not treated as a scanner failure, so a server with `AVFailAction` 1 does not hold every encrypted message it receives; a policy that unscannable mail must not be delivered belongs in a rule or a Sieve script, not the scanner. - **The "delete attachments" action never strips an encrypted message.** When the scanner objects to one, the message is left as it arrived and the log ends *Set the anti-virus action to delete the message if such a message must not be delivered*. That is not a quiet pass: the message still carries what the scanner objected to. To 6.3.6 this held for every opaque message; since 6.3.7 an S/MIME message signed as one part (`application/pkcs7-mime` holding signed-data, which is opaque but not encrypted) is replaced by the action's note, as any other single-part attachment is. ### The sandbox *Since 6.3.4 (schema 6091).* ClamAV says whether a file is known to be bad. What an unknown file does when it is opened takes a sandbox, a service outside this server which you run or subscribe to. **It does nothing until you name one**: `SandboxEnabled` is off, and cannot be saved on while `SandboxSubmitUrl` or `SandboxVerdictUrl` is empty. Both addresses must be https, or plain http to this machine. After the anti-virus scan and before the rules, a message that no signed-in user of this server submitted, with an attachment whose extension or media type is in `SandboxTypes`, is **held**: in no mailbox, not in the quarantine, and in nobody's digest. The default types are: ``` exe,dll,scr,js,jse,vbs,wsf,hta,msi,bat,cmd,ps1,jar,lnk,iso,img,docm,xlsm,pptm, doc,xls,ppt,rtf,pdf,zip,7z,rar ``` | The sandbox says | What happens | |---|---| | every file clean | delivered, with an `X-hMailServer-Sandbox` header reading `clean ( file(s), s)` | | any file malicious, or suspicious at or above `SandboxScoreThreshold` (70) | quarantined with the verdict as the reason; returned to its sender with 5.7.1 while `QuarantineEnabled` is off | | an error, once every file has answered, or no answer within `SandboxTimeoutMinutes` (15) | `SandboxTimeoutAction`: `deliver` (0, the default) delivers with a header saying so; `bounce` (1) returns it to its sender | **The contract.** One POST per file to `SandboxSubmitUrl` as `multipart/form-data` with four fields (`file`, `filename`, `sha256`, `message_id`), then a GET to `SandboxVerdictUrl` every `SandboxPollSeconds` (30), answered with JSON such as `{"state":"done","verdict":"clean","score":3,"detail":"opened, nothing happened"}`. With `SandboxToken` set, every request carries `Authorization: Bearer `; the token is not in the backup archive. **No vendor adapter is shipped.** The document notes that a service which shares what it is sent, as VirusTotal shares uploads with its subscribers, is a place the attachments of the mail you receive go. **What is exempt.** What your own signed-in users submit (authenticated SMTP, the webmail, the send API) is not uploaded unless `SandboxInternalMail` is on. The document states the threat model of that default: a stolen password used to send attachments to colleagues is not examined by the sandbox, though ClamAV, the blocked-attachment list and compromised-account detection still apply. **Limits.** An attachment larger than `SandboxMaxSizeKB` (20480) is not submitted. A message with more files to submit than `SandboxMaxFiles` (10) is not held, and `SandboxTimeoutAction` decides it at once. Archives are submitted whole; links are not opened anywhere. A clean or malicious verdict is remembered by the file's SHA-256 for `SandboxCacheHours` (24), server-wide. **What the administrator sees.** `GET /api/v1/sandbox/held` lists what is held; `POST /api/v1/sandbox/held/{queue_id}/release` delivers without the verdict and `DELETE /api/v1/sandbox/held/{queue_id}` discards, both on the audit trail. The Control Deck's queue page has a *Held for a sandbox* box, and `hmctl queue list` has a `held` column. The settings are on the Control Panel's Anti-virus page, *Sandbox* tab (Windows only), and on `PUT /api/v1/settings`. ### Rules, and the message store Since 6.3.4 the mail flow rules can match the attachment's name and, in the outbound content policy, its type read from its bytes, so a rule can hold or refuse what the blocked-attachment list would only rename; that is in [the topic on mail flow rules](/topic/28251). [The message store options added in 6.3.4](/topic/28233) change nothing for a scanner: a message is scanned as plain RFC 822 in the queue and only placed in the mailbox in its stored form afterwards. A scanner pointed at the data directory from outside the server reads a compressed or encrypted container, not a message. ### Error codes | Code | Meaning | |---|---| | `HM5406` | An enabled scanner errored; the detail names the scanner and the failure | | `HM6001` | The message could not be parsed, so only the whole file was scanned | | `HM6002` | An attachment could not be written to the temp directory for scanning | | `HM6195` | 100,000 messages are already held for want of a scan; further holds are not counted | | `HM7272` | The sandbox called a message malicious and the quarantine could not store it; it is held again, never delivered | ### Where to read more - Wiki: https://gitlab.com/hmailserver/hmailserver/-/wikis/Stopping-Viruses - https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Sandbox.md - https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SealedMail.md - Public documentation: https://www.progressiverobot.com/hmailserver-documentation/#11-stopping-viruses, https://www.progressiverobot.com/hmailserver-documentation/#111-clamav and https://www.progressiverobot.com/hmailserver-documentation/#112-blocked-attachments - [Upgrading to 6.3.4](/topic/28193) and [the 6.3.4 security announcement](/topic/28218) --- Canonical: https://www.hmailserver.co.uk/topic/28251/mail-flow-rules-the-outbound-content-policy-and-what-a-domain-does-to-a-message # Mail flow rules, the outbound content policy and what a domain does to a message *Written for 6.3.4. hMailServer 6.3.7 (17 October 2026) holds a domain's or an account's rule to its owner: it may no longer set `From`, `Sender`, `Resent-From`, `Resent-Sender`, `DKIM-Signature` or the loop count, and its Reply goes only from an address the owner may send as; such a rule is refused when it is saved, one stored before the upgrade runs without that action, and the application log names it. Its Warn sender notice, and the notice of a Reject met after acceptance, come from the postmaster of the rule owner's domain. A credential confined to named domains may no longer give a domain's or an account's rule a route, a bound address or a script function, and a Run function that is not a plain function name is refused when the rule is saved. A rule's forward of mail from outside whose `From` names a hosted domain's organisation is given an envelope sender in that organisation only when the message passed DMARC for it on arrival; otherwise it keeps the envelope sender it came with or, where that is itself at this server's host name or at a hosted domain in the organisation its `From` names, is not sent to an outside address at all, and the application log names each recipient. A message over 80 MB goes through the rules by its header alone: the conditions that need the body do not match, a script function is not run, and the disclaimer and the signature are not appended. See [the 6.3.7 announcement](/topic/28277).* This topic covers the rules the server runs on every message, the conditions and actions 6.3.4 added to them to make an outbound content policy, and the three things a domain can be told to do to the mail passing through it: mark what comes from outside, say when a sender has never written before, and put a legal footer on what leaves. The one thing to know first is that these run at three different moments, and the moment decides what each can see. The server's and the domains' rules run in the deliverer on every delivery attempt, with `reject` also asked at the end of `DATA`, and an account's own rules once for their recipient; the external tag and the first-contact note are applied to the recipient's own copy at local delivery; and the disclaimer goes on at submission, before this server's DKIM signature is made. ### Three collections of rules | Collection | Where it is edited | Runs for | |---|---|---| | The server's | `Application.Rules`, `/api/v1/rules`, the Control Panel's Rules page, the Deck's | every message, first | | A domain's (since 6.3.4, schema 6063) | `Domain.Rules`, `/api/v1/domains/{name}/rules`, the same pages with the domain chosen at the top | every message with a recipient in the domain or a sender it hosts, after the server's | | An account's | `Account.Rules`, `/api/v1/accounts/{address}/rules` | that account's mail, at local delivery | The server's and the domains' rules run in `SMTPDeliverer` on every delivery attempt, after virus scanning and before DKIM signing and the split between local and external delivery. An account's own rules run after the split, at local delivery, on that recipient's own copy: once for their recipient, when its copy is stored. All three have the same conditions and actions, except that the wiki's Rules and Sieve page lists two actions, send using a route and bind to an address, as absent from an account's rules. Since 6.3.7 a domain's or an account's rule may not set `From`, `Sender`, `Resent-From`, `Resent-Sender` or `DKIM-Signature`, which are for the server's rules alone, nor the loop count, which no rule sets, and its *Reply* goes only from an address the owner may send as: one the account may send as for an account's rule, an address of the domain or of one of its domain aliases for a domain's; a credential confined to named domains may not give such a rule a route, a bound address or a script function. A rule stored before the upgrade runs without the refused action, and the application log names it; to set `From` on a domain's mail, make the rule a server rule with a condition on the domain. A domain's rules are how a domain administrator has rules of their own: a domain-restricted key reaches its domain's rules and no other's. The Control Panel and the COM objects are Windows-only; the Deck and REST serve both platforms. **A repeated pass does not repeat the side effects.** A pass that repeats an attempt because this server's own [outbound shaping](/topic/28259) held the message, without counting a try, runs the rules again but does not forward, copy, reply or call a script function again. CHANGELOG records the fix: before it, a held message sent one copy per pass. **What a rule queues goes at once.** Since 6.3.4 a rule's forward, copy, reply or notice to the sender, and an account's own forwarding, no longer wait up to a minute in the queue: each is queued inside the delivery that caused it, and the delivery manager is now told of it, where before it sat until something else woke the manager. **A domain's rules act only on what is that domain's.** Domain rules are new in 6.3.4 and shipped with this limit. - **The sending domain's rules act on the whole message.** The sending domain is that of the account that submitted the message (SMTP `AUTH`, the webmail, a scheduled send), and the envelope sender's domain when that is vouched for: its account submitted the message, DMARC passed for the domain on mail from outside, or the message reaches no other domain's mailboxes. An envelope sender nobody vouched for does not choose a domain's rules for mail to another domain's mailboxes. - **A recipient domain's rules act on the whole message only when every recipient is its own.** Otherwise they act on the domain's own recipients: a delete or a reject takes those recipients off the message, a quarantine keeps a copy for them under the rule's name, and a copy goes to them alone. Such a reject is met at delivery, not in the SMTP session, whose one answer is for every recipient; a sender this server hosts is mailed a notice naming the recipients refused. A header, a disclaimer, a script function, a route, a bound address and required TLS are not applied, since they would change the message for everybody. As domain rules were first written in the source tree, without this limit, one tenant's delete, reject, header or route reached another tenant's recipients; no release had that. ### Conditions and actions The long-standing conditions (From, To, CC, Subject, body, size, the recipient list, delivery attempts, any named header), the match types, the long-standing actions and how a rule set is evaluated are the rule engine's, and are set out in [Rules and Sieve: what runs when](/topic/28268). Since 6.3.4 there is one more condition, **the attachment name** (`attachment_name`): every attachment's file name, with `;` between them, so `contains .exe` or a wildcard `*.zip` reads as "any attachment is". **Run a script function takes a plain function name since 6.3.4**, without a switch: ASCII letters, digits, `_` and `$`, not starting with a digit. On Windows any other name is refused as `HM5018` each time the rule fires; on Linux it finds no function, nothing is run, and the error log has a Script Error line. Since 6.3.7 such a rule is also refused when it is saved, over COM and REST, for a rule of every owner. Give every such rule a plain function name before you upgrade, and look for `HM5018` after; why it changed is in [event scripts and the 6.3.4 security fix](/topic/28261) and [the 6.3.4 security announcement](/topic/28218). Since 6.3.4 there are five more actions: | Action | REST word | What it does | |---|---|---| | Add a disclaimer | `add_disclaimer` | Appends the text to the message's plain and HTML parts when the rule matches, with `X-hMailServer-Disclaimer` naming the rule. The domain disclaimer stays the unconditional footer it was | | Reject with a text | `reject` | The SMTP session answers `550 5.7.1` with the text before the message is accepted. A reject with no text of its own names the rule | | Require TLS | `require_tls` | Every external delivery of the message needs STARTTLS. A destination that offers none is deferred with a reason naming the rule; the message is not sent in the clear | | Quarantine | `quarantine` | Held in the quarantine under `Rule: ` for an administrator to release or delete. A quarantine that could not store it delivers the message rather than lose it, and says so in the error log | | Warn the sender | `warn_sender` | The message goes where the rule's other actions send it, and a sender this server hosts is mailed a notice | **Where `reject` is asked.** The server's rules, and those of every domain whose rules act on the whole message, are asked at the end of `DATA`, and the message is parsed for the match only when one of them has a reject action at all. The webmail's send route asks the same rules before it queues, and answers `403`, code `policy_rejected`, with the text. A message that reached delivery another way is dropped, as a delete is, with the rule named in the log. ### The outbound content policy *Since 6.3.4.* What Exchange calls data loss prevention is here four more conditions and one more action in the same rules, with no second engine. Each condition answers as a list with `;` between the entries, so `contains x` reads as "any of them is x" and `equals ""` as "none". | Condition | REST word | Answers | |---|---|---| | Sensitive content | `sensitive_content` | The detectors that found something in the readable text: `card_number`, `iban`, `account_number` | | Attachment type | `attachment_type` | Every attachment's type from its first bytes, not its name: `executable`, `elf`, `pdf`, `zip`, `office_xml`, `jar`, `office`, `rar`, `7z`, `gzip`, `png`, `jpeg`, `gif`, `rtf`, `script`, `text`, `empty`, `binary` | | External recipients | `external_recipient` | Every envelope recipient this server does not host; empty when every recipient is local | | Readable text | `content` | The text itself, for a pattern of the rule's own | **The readable text** is the plain part, the HTML part with its tags removed, and every attachment whose bytes are text, a `.csv` sent as `application/octet-stream` included. It is bounded at a megabyte. **The detectors.** - `card_number`: a run of 13 to 19 digits, with at most one space or hyphen between any two, that passes the Luhn check and is not one digit repeated. A run longer than 19 digits is not searched for a card inside it. - `iban`: two letters of a country in the IBAN registry and two digits, then exactly that country's number of characters, passing the ISO 13616 check. - `account_number`: the regular expression in the `dlp_account_number_pattern` setting, such as `\b\d{2}-\d{2}-\d{2} \d{8}\b` for a sort code and account. Empty, the default, is a detector that never fires. A pattern that does not compile is reported as `HM6821`, and a search given up as too complex, or one that takes over the 250 ms budget, as `HM6822`; either way the pattern is rested for five minutes, finding nothing meanwhile. That is fail-open, on purpose and said out loud. **Marked internal** needs no new condition: `header` `Sensitivity` `contains` `Company-Confidential`, combined with `external_recipient` `contains` `@`. **A sealed message**, encrypted end to end and judged from its body rather than its label, has no text the server can read, so `sensitive_content` and `content` do not match it, whichever way they are phrased, and the application log says why once per rule. `attachment_type` and `external_recipient` still answer. **A worked policy.** From the document: ```json POST /api/v1/rules { "name": "No card numbers leave", "all_criteria": true, "criteria": [ { "field": "external_recipient", "match": "contains", "value": "@" }, { "field": "sensitive_content", "match": "contains", "value": "card_number" } ], "actions": [ { "type": "reject", "value": "Card numbers may not be sent by mail. Use the payment portal." } ] } ``` A rule with `external_recipient contains @` fires only on a message with a recipient this server does not host; a rule without that condition applies to inbound mail too. A policy is a rule of the server or of the sender's domain. It cannot be an account's rule, because those are the recipient's own filters and never see the account's own outgoing mail. **The notice `warn_sender` sends** comes from `postmaster@` (to 6.3.6; since 6.3.7 from the postmaster of the rule owner's domain, and for a server rule of the default domain when one is set and hosted here, else of the hosted domain the message was addressed to, else of the sender's own, signed as that domain) with the subject `Policy notice: `, the headers `X-hMailServer-Policy: ` and `X-hMailServer-Policy-Outcome: delivered`, and `Auto-Submitted: auto-generated`. It is sent only when this server hosts the sender's domain, because a notice to an outside sender is backscatter, and it goes once per message, on the first delivery attempt. A rule with only that action delivers the message. **A released message is not held again** by the rule that quarantined it; the rule's other actions still run. **What the policy does not do.** It is pattern matching, and patterns have false positives: the document says one 16-digit reference number in ten passes Luhn, and recommends `warn_sender` or `quarantine` as the first choice for a new pattern rather than `reject`. It reads text, not documents: a card number inside a PDF, a Word file or a spreadsheet is not found, an image of a card is an image, and an encrypted zip is `zip`. There are no fingerprints, classifiers or dictionaries, no per-user exceptions, and no policy tip in the sender's mail client before they press Send. ### What a domain does to a message *Since 6.3.4 (schema 6044).* Three per-domain features, **off on every existing and every new domain**, costing nothing while off. They are set in the Control Panel's domain dialog, tab **Tagging and disclaimer** (Windows only); in the Control Deck's domain editor, **What this domain does to a message**; or with `PUT /api/v1/domains/{name}` and the keys `external_tag_subject`, `external_tag_header`, `external_tag_text`, `first_contact_tip`, `disclaimer_enabled`, `disclaimer_plain_text` and `disclaimer_html`. **Who counts as outside** is one rule, `MessageOrigin::IsExternalSender`. A sender is outside when none of these is true: 1. The session authenticated. This is read from the topmost `Received:` header this server wrote; `X-AuthUser` is deliberately not consulted. 2. A trusted incoming relay handed the message over. 3. Every address the message names, the envelope sender and the `From:` address, is at a domain this server hosts. Either being outside is enough. **The sharp edge: an inbound gateway.** If you run a filtering gateway in front of this server and list it as [an incoming relay](/topic/28240), nothing arriving through it is ever tagged. The choice is between the tag and the relay declaration; the document says there is no third answer today. **It is not anti-spoofing.** A message that forges a hosted address in both its envelope and its `From` is judged inside and is not tagged. The control for that is DMARC with a policy on your own domain. | Switch | What the reader gets | Cost | |---|---|---| | `external_tag_subject` | The tag text at the front of the Subject: `external_tag_text`, at most 100 characters, `[EXTERNAL]` when empty. A subject that already begins with it gets no second copy | **Breaks any DKIM signature the sender made over the Subject** for anyone who verifies downstream, as the spam subject prefix does. The verdict this server reached is unaffected | | `external_tag_header` | `X-hMailServer-External: YES`, which the webmail shows as *This message came from outside your organisation.* | None to the signature. Prefer this wherever the reader's client can show it | | `first_contact_tip` | `X-hMailServer-First-Contact: YES` when an outside sender writes to an account that has had mail from other people but never from them. The webmail shows *You do not usually get mail from this sender.* | One indexed `SELECT` per delivered copy, then an `UPDATE`, or a further `SELECT` and an `INSERT` the first time | The tag is applied to the recipient account's own copy, above the account's rules and its Sieve script, so both can match on the header, and below the per-account spam settings. A domain with tagging on does not share one delivered file between several recipients (`DeliveryHardLinks`). Where a domain has turned the feature on, a forged copy of the header on a message judged inside is removed. **The first-contact memory** is the table `hm_knownsenders`, one row per account and sender. When an account that signed in sends a message, each recipient is recorded as somebody that account knows, so their reply is not announced; an unauthenticated session records nothing. The wiki adds that mail sent from the webmail or by send-later does not record its recipients, so a reply to a message first sent that way can be announced once. Two limits are by design. After a **cold start**, a new installation or the day the switch is turned on, most senders look new; the document says to expect a week. After a **restore**, the memory is as of the backup. ### The domain disclaimer `disclaimer_enabled` appends the domain's legal footer to mail leaving the organisation. Filling in only `disclaimer_plain_text` uses it for both forms. It runs where the domain signature runs, before `DKIMSigner::Sign`, so this server's own signature covers the footer. The footer is appended to the `text/plain` part and the `text/html` part in place; a message with neither is left alone. **It is never added to:** - a signed or encrypted message (`multipart/signed`, `multipart/encrypted`, `application/pkcs7-mime`, `application/x-pkcs7-mime`, `application/pgp-encrypted` or `application/pgp-signature`), which is sent exactly as written, with a line in the application log; - an automatic reply, a vacation message or a delivery report; - a bounce; - a list posting (`List-Id`, `List-Unsubscribe`, or a `Precedence:` of bulk, list or junk); - a message with no recipient outside this server; - a message this server already footed, which carries `X-hMailServer-Disclaimer`; - a message relayed in from outside that merely claims a hosted sender domain. A reply quoting a message that already carried the footer does not collect a second copy. A footer shorter than eight characters is never treated as already present. **Webmail and send-later get the footers too, since 6.3.4.** The domain's signature and disclaimer go on mail sent from the webmail (`POST /api/v1/me/messages`, which is also its reply, forward and phishing report) and from its send-later, through the same function as SMTP ([docs/DomainTransforms.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/DomainTransforms.md); CHANGELOG). In 6.3.3 and earlier only the SMTP path had either footer, which is what the wiki's Stopping Spam §10.7a still describes. **Two sharp edges on the webmail paths.** Send-later sends with the account as the envelope sender, so a draft scheduled with a chosen From at another hosted domain gets the account's domain's footers. And with the domain's signature method set to overwrite the account's, the webmail has already written the account's signature into the body, which the server does not take out. **Signed and encrypted messages get no signature either**, since 6.3.4. Before, a signature was appended inside a `multipart/signed`, breaking the S/MIME signature. This is a change on the SMTP path too: an installation that relied on a signature inside signed mail will now see none there. ### Where to read more Sieve, the account's own filters and the rule engine itself are in [Rules and Sieve](/topic/28268); the blocked-attachment list and the sandbox are in [the topic on viruses and dangerous attachments](/topic/28250). The two browser surfaces have topics of their own: [The Control Deck](/topic/28192) and [The webmail portal at /portal](/topic/28191). - https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/MailFlowRules.md - https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/OutboundContentPolicy.md - https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/DomainTransforms.md - Wiki: https://gitlab.com/hmailserver/hmailserver/-/wikis/Rules-and-Sieve and, for the tag and the note, https://gitlab.com/hmailserver/hmailserver/-/wikis/Stopping-Spam - Public documentation: https://www.progressiverobot.com/hmailserver-documentation/#121-hmailserver-rules and https://www.progressiverobot.com/hmailserver-documentation/#27-rules-in-practice-a-cookbook --- Canonical: https://www.hmailserver.co.uk/topic/28251/mail-flow-rules-the-outbound-content-policy-and-what-a-domain-does-to-a-message # Mail flow rules, the outbound content policy and what a domain does to a message *Written for 6.3.4. hMailServer 6.3.7 (17 October 2026) holds a domain's or an account's rule to its owner: it may no longer set `From`, `Sender`, `Resent-From`, `Resent-Sender`, `DKIM-Signature` or the loop count, and its Reply goes only from an address the owner may send as; such a rule is refused when it is saved, one stored before the upgrade runs without that action, and the application log names it. Its Warn sender notice, and the notice of a Reject met after acceptance, come from the postmaster of the rule owner's domain. A credential confined to named domains may no longer give a domain's or an account's rule a route, a bound address or a script function, and a Run function that is not a plain function name is refused when the rule is saved. A rule's forward of mail from outside whose `From` names a hosted domain's organisation is given an envelope sender in that organisation only when the message passed DMARC for it on arrival; otherwise it keeps the envelope sender it came with or, where that is itself at this server's host name or at a hosted domain in the organisation its `From` names, is not sent to an outside address at all, and the application log names each recipient. A message over 80 MB goes through the rules by its header alone: the conditions that need the body do not match, a script function is not run, and the disclaimer and the signature are not appended. See [the 6.3.7 announcement](/topic/28277).* This topic covers the rules the server runs on every message, the conditions and actions 6.3.4 added to them to make an outbound content policy, and the three things a domain can be told to do to the mail passing through it: mark what comes from outside, say when a sender has never written before, and put a legal footer on what leaves. The one thing to know first is that these run at three different moments, and the moment decides what each can see. The server's and the domains' rules run in the deliverer on every delivery attempt, with `reject` also asked at the end of `DATA`, and an account's own rules once for their recipient; the external tag and the first-contact note are applied to the recipient's own copy at local delivery; and the disclaimer goes on at submission, before this server's DKIM signature is made. ### Three collections of rules | Collection | Where it is edited | Runs for | |---|---|---| | The server's | `Application.Rules`, `/api/v1/rules`, the Control Panel's Rules page, the Deck's | every message, first | | A domain's (since 6.3.4, schema 6063) | `Domain.Rules`, `/api/v1/domains/{name}/rules`, the same pages with the domain chosen at the top | every message with a recipient in the domain or a sender it hosts, after the server's | | An account's | `Account.Rules`, `/api/v1/accounts/{address}/rules` | that account's mail, at local delivery | The server's and the domains' rules run in `SMTPDeliverer` on every delivery attempt, after virus scanning and before DKIM signing and the split between local and external delivery. An account's own rules run after the split, at local delivery, on that recipient's own copy: once for their recipient, when its copy is stored. All three have the same conditions and actions, except that the wiki's Rules and Sieve page lists two actions, send using a route and bind to an address, as absent from an account's rules. Since 6.3.7 a domain's or an account's rule may not set `From`, `Sender`, `Resent-From`, `Resent-Sender` or `DKIM-Signature`, which are for the server's rules alone, nor the loop count, which no rule sets, and its *Reply* goes only from an address the owner may send as: one the account may send as for an account's rule, an address of the domain or of one of its domain aliases for a domain's; a credential confined to named domains may not give such a rule a route, a bound address or a script function. A rule stored before the upgrade runs without the refused action, and the application log names it; to set `From` on a domain's mail, make the rule a server rule with a condition on the domain. A domain's rules are how a domain administrator has rules of their own: a domain-restricted key reaches its domain's rules and no other's. The Control Panel and the COM objects are Windows-only; the Deck and REST serve both platforms. **A repeated pass does not repeat the side effects.** A pass that repeats an attempt because this server's own [outbound shaping](/topic/28259) held the message, without counting a try, runs the rules again but does not forward, copy, reply or call a script function again. CHANGELOG records the fix: before it, a held message sent one copy per pass. **What a rule queues goes at once.** Since 6.3.4 a rule's forward, copy, reply or notice to the sender, and an account's own forwarding, no longer wait up to a minute in the queue: each is queued inside the delivery that caused it, and the delivery manager is now told of it, where before it sat until something else woke the manager. **A domain's rules act only on what is that domain's.** Domain rules are new in 6.3.4 and shipped with this limit. - **The sending domain's rules act on the whole message.** The sending domain is that of the account that submitted the message (SMTP `AUTH`, the webmail, a scheduled send), and the envelope sender's domain when that is vouched for: its account submitted the message, DMARC passed for the domain on mail from outside, or the message reaches no other domain's mailboxes. An envelope sender nobody vouched for does not choose a domain's rules for mail to another domain's mailboxes. - **A recipient domain's rules act on the whole message only when every recipient is its own.** Otherwise they act on the domain's own recipients: a delete or a reject takes those recipients off the message, a quarantine keeps a copy for them under the rule's name, and a copy goes to them alone. Such a reject is met at delivery, not in the SMTP session, whose one answer is for every recipient; a sender this server hosts is mailed a notice naming the recipients refused. A header, a disclaimer, a script function, a route, a bound address and required TLS are not applied, since they would change the message for everybody. As domain rules were first written in the source tree, without this limit, one tenant's delete, reject, header or route reached another tenant's recipients; no release had that. ### Conditions and actions The long-standing conditions (From, To, CC, Subject, body, size, the recipient list, delivery attempts, any named header), the match types, the long-standing actions and how a rule set is evaluated are the rule engine's, and are set out in [Rules and Sieve: what runs when](/topic/28268). Since 6.3.4 there is one more condition, **the attachment name** (`attachment_name`): every attachment's file name, with `;` between them, so `contains .exe` or a wildcard `*.zip` reads as "any attachment is". **Run a script function takes a plain function name since 6.3.4**, without a switch: ASCII letters, digits, `_` and `$`, not starting with a digit. On Windows any other name is refused as `HM5018` each time the rule fires; on Linux it finds no function, nothing is run, and the error log has a Script Error line. Since 6.3.7 such a rule is also refused when it is saved, over COM and REST, for a rule of every owner. Give every such rule a plain function name before you upgrade, and look for `HM5018` after; why it changed is in [event scripts and the 6.3.4 security fix](/topic/28261) and [the 6.3.4 security announcement](/topic/28218). Since 6.3.4 there are five more actions: | Action | REST word | What it does | |---|---|---| | Add a disclaimer | `add_disclaimer` | Appends the text to the message's plain and HTML parts when the rule matches, with `X-hMailServer-Disclaimer` naming the rule. The domain disclaimer stays the unconditional footer it was | | Reject with a text | `reject` | The SMTP session answers `550 5.7.1` with the text before the message is accepted. A reject with no text of its own names the rule | | Require TLS | `require_tls` | Every external delivery of the message needs STARTTLS. A destination that offers none is deferred with a reason naming the rule; the message is not sent in the clear | | Quarantine | `quarantine` | Held in the quarantine under `Rule: ` for an administrator to release or delete. A quarantine that could not store it delivers the message rather than lose it, and says so in the error log | | Warn the sender | `warn_sender` | The message goes where the rule's other actions send it, and a sender this server hosts is mailed a notice | **Where `reject` is asked.** The server's rules, and those of every domain whose rules act on the whole message, are asked at the end of `DATA`, and the message is parsed for the match only when one of them has a reject action at all. The webmail's send route asks the same rules before it queues, and answers `403`, code `policy_rejected`, with the text. A message that reached delivery another way is dropped, as a delete is, with the rule named in the log. ### The outbound content policy *Since 6.3.4.* What Exchange calls data loss prevention is here four more conditions and one more action in the same rules, with no second engine. Each condition answers as a list with `;` between the entries, so `contains x` reads as "any of them is x" and `equals ""` as "none". | Condition | REST word | Answers | |---|---|---| | Sensitive content | `sensitive_content` | The detectors that found something in the readable text: `card_number`, `iban`, `account_number` | | Attachment type | `attachment_type` | Every attachment's type from its first bytes, not its name: `executable`, `elf`, `pdf`, `zip`, `office_xml`, `jar`, `office`, `rar`, `7z`, `gzip`, `png`, `jpeg`, `gif`, `rtf`, `script`, `text`, `empty`, `binary` | | External recipients | `external_recipient` | Every envelope recipient this server does not host; empty when every recipient is local | | Readable text | `content` | The text itself, for a pattern of the rule's own | **The readable text** is the plain part, the HTML part with its tags removed, and every attachment whose bytes are text, a `.csv` sent as `application/octet-stream` included. It is bounded at a megabyte. **The detectors.** - `card_number`: a run of 13 to 19 digits, with at most one space or hyphen between any two, that passes the Luhn check and is not one digit repeated. A run longer than 19 digits is not searched for a card inside it. - `iban`: two letters of a country in the IBAN registry and two digits, then exactly that country's number of characters, passing the ISO 13616 check. - `account_number`: the regular expression in the `dlp_account_number_pattern` setting, such as `\b\d{2}-\d{2}-\d{2} \d{8}\b` for a sort code and account. Empty, the default, is a detector that never fires. A pattern that does not compile is reported as `HM6821`, and a search given up as too complex, or one that takes over the 250 ms budget, as `HM6822`; either way the pattern is rested for five minutes, finding nothing meanwhile. That is fail-open, on purpose and said out loud. **Marked internal** needs no new condition: `header` `Sensitivity` `contains` `Company-Confidential`, combined with `external_recipient` `contains` `@`. **A sealed message**, encrypted end to end and judged from its body rather than its label, has no text the server can read, so `sensitive_content` and `content` do not match it, whichever way they are phrased, and the application log says why once per rule. `attachment_type` and `external_recipient` still answer. **A worked policy.** From the document: ```json POST /api/v1/rules { "name": "No card numbers leave", "all_criteria": true, "criteria": [ { "field": "external_recipient", "match": "contains", "value": "@" }, { "field": "sensitive_content", "match": "contains", "value": "card_number" } ], "actions": [ { "type": "reject", "value": "Card numbers may not be sent by mail. Use the payment portal." } ] } ``` A rule with `external_recipient contains @` fires only on a message with a recipient this server does not host; a rule without that condition applies to inbound mail too. A policy is a rule of the server or of the sender's domain. It cannot be an account's rule, because those are the recipient's own filters and never see the account's own outgoing mail. **The notice `warn_sender` sends** comes from `postmaster@` (to 6.3.6; since 6.3.7 from the postmaster of the rule owner's domain, and for a server rule of the default domain when one is set and hosted here, else of the hosted domain the message was addressed to, else of the sender's own, signed as that domain) with the subject `Policy notice: `, the headers `X-hMailServer-Policy: ` and `X-hMailServer-Policy-Outcome: delivered`, and `Auto-Submitted: auto-generated`. It is sent only when this server hosts the sender's domain, because a notice to an outside sender is backscatter, and it goes once per message, on the first delivery attempt. A rule with only that action delivers the message. **A released message is not held again** by the rule that quarantined it; the rule's other actions still run. **What the policy does not do.** It is pattern matching, and patterns have false positives: the document says one 16-digit reference number in ten passes Luhn, and recommends `warn_sender` or `quarantine` as the first choice for a new pattern rather than `reject`. It reads text, not documents: a card number inside a PDF, a Word file or a spreadsheet is not found, an image of a card is an image, and an encrypted zip is `zip`. There are no fingerprints, classifiers or dictionaries, no per-user exceptions, and no policy tip in the sender's mail client before they press Send. ### What a domain does to a message *Since 6.3.4 (schema 6044).* Three per-domain features, **off on every existing and every new domain**, costing nothing while off. They are set in the Control Panel's domain dialog, tab **Tagging and disclaimer** (Windows only); in the Control Deck's domain editor, **What this domain does to a message**; or with `PUT /api/v1/domains/{name}` and the keys `external_tag_subject`, `external_tag_header`, `external_tag_text`, `first_contact_tip`, `disclaimer_enabled`, `disclaimer_plain_text` and `disclaimer_html`. **Who counts as outside** is one rule, `MessageOrigin::IsExternalSender`. A sender is outside when none of these is true: 1. The session authenticated. This is read from the topmost `Received:` header this server wrote; `X-AuthUser` is deliberately not consulted. 2. A trusted incoming relay handed the message over. 3. Every address the message names, the envelope sender and the `From:` address, is at a domain this server hosts. Either being outside is enough. **The sharp edge: an inbound gateway.** If you run a filtering gateway in front of this server and list it as [an incoming relay](/topic/28240), nothing arriving through it is ever tagged. The choice is between the tag and the relay declaration; the document says there is no third answer today. **It is not anti-spoofing.** A message that forges a hosted address in both its envelope and its `From` is judged inside and is not tagged. The control for that is DMARC with a policy on your own domain. | Switch | What the reader gets | Cost | |---|---|---| | `external_tag_subject` | The tag text at the front of the Subject: `external_tag_text`, at most 100 characters, `[EXTERNAL]` when empty. A subject that already begins with it gets no second copy | **Breaks any DKIM signature the sender made over the Subject** for anyone who verifies downstream, as the spam subject prefix does. The verdict this server reached is unaffected | | `external_tag_header` | `X-hMailServer-External: YES`, which the webmail shows as *This message came from outside your organisation.* | None to the signature. Prefer this wherever the reader's client can show it | | `first_contact_tip` | `X-hMailServer-First-Contact: YES` when an outside sender writes to an account that has had mail from other people but never from them. The webmail shows *You do not usually get mail from this sender.* | One indexed `SELECT` per delivered copy, then an `UPDATE`, or a further `SELECT` and an `INSERT` the first time | The tag is applied to the recipient account's own copy, above the account's rules and its Sieve script, so both can match on the header, and below the per-account spam settings. A domain with tagging on does not share one delivered file between several recipients (`DeliveryHardLinks`). Where a domain has turned the feature on, a forged copy of the header on a message judged inside is removed. **The first-contact memory** is the table `hm_knownsenders`, one row per account and sender. When an account that signed in sends a message, each recipient is recorded as somebody that account knows, so their reply is not announced; an unauthenticated session records nothing. The wiki adds that mail sent from the webmail or by send-later does not record its recipients, so a reply to a message first sent that way can be announced once. Two limits are by design. After a **cold start**, a new installation or the day the switch is turned on, most senders look new; the document says to expect a week. After a **restore**, the memory is as of the backup. ### The domain disclaimer `disclaimer_enabled` appends the domain's legal footer to mail leaving the organisation. Filling in only `disclaimer_plain_text` uses it for both forms. It runs where the domain signature runs, before `DKIMSigner::Sign`, so this server's own signature covers the footer. The footer is appended to the `text/plain` part and the `text/html` part in place; a message with neither is left alone. **It is never added to:** - a signed or encrypted message (`multipart/signed`, `multipart/encrypted`, `application/pkcs7-mime`, `application/x-pkcs7-mime`, `application/pgp-encrypted` or `application/pgp-signature`), which is sent exactly as written, with a line in the application log; - an automatic reply, a vacation message or a delivery report; - a bounce; - a list posting (`List-Id`, `List-Unsubscribe`, or a `Precedence:` of bulk, list or junk); - a message with no recipient outside this server; - a message this server already footed, which carries `X-hMailServer-Disclaimer`; - a message relayed in from outside that merely claims a hosted sender domain. A reply quoting a message that already carried the footer does not collect a second copy. A footer shorter than eight characters is never treated as already present. **Webmail and send-later get the footers too, since 6.3.4.** The domain's signature and disclaimer go on mail sent from the webmail (`POST /api/v1/me/messages`, which is also its reply, forward and phishing report) and from its send-later, through the same function as SMTP ([docs/DomainTransforms.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/DomainTransforms.md); CHANGELOG). In 6.3.3 and earlier only the SMTP path had either footer, which is what the wiki's Stopping Spam §10.7a still describes. **Two sharp edges on the webmail paths.** Send-later sends with the account as the envelope sender, so a draft scheduled with a chosen From at another hosted domain gets the account's domain's footers. And with the domain's signature method set to overwrite the account's, the webmail has already written the account's signature into the body, which the server does not take out. **Signed and encrypted messages get no signature either**, since 6.3.4. Before, a signature was appended inside a `multipart/signed`, breaking the S/MIME signature. This is a change on the SMTP path too: an installation that relied on a signature inside signed mail will now see none there. ### Where to read more Sieve, the account's own filters and the rule engine itself are in [Rules and Sieve](/topic/28268); the blocked-attachment list and the sandbox are in [the topic on viruses and dangerous attachments](/topic/28250). The two browser surfaces have topics of their own: [The Control Deck](/topic/28192) and [The webmail portal at /portal](/topic/28191). - https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/MailFlowRules.md - https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/OutboundContentPolicy.md - https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/DomainTransforms.md - Wiki: https://gitlab.com/hmailserver/hmailserver/-/wikis/Rules-and-Sieve and, for the tag and the note, https://gitlab.com/hmailserver/hmailserver/-/wikis/Stopping-Spam - Public documentation: https://www.progressiverobot.com/hmailserver-documentation/#121-hmailserver-rules and https://www.progressiverobot.com/hmailserver-documentation/#27-rules-in-practice-a-cookbook --- Canonical: https://www.hmailserver.co.uk/topic/28252/calendars-contacts-the-directory-and-shared-address-books-over-caldav-and-carddav # Calendars, contacts, the directory and shared address books over CalDAV and CardDAV This topic covers what hMailServer 6.3.4 serves to a phone or a desktop client over CalDAV and CardDAV, and what the webmail's Calendar, Tasks, Contacts and Directory pages do with the same data. The one thing to know first: CalDAV and CardDAV are served on the **web-services listener**, they take HTTP Basic as the account - its password or an app password - over HTTPS only, and a passkey or single sign-on does not reach them. CardDAV has been there since 6.3.3 with one address book; calendars, several and shared address books, the directory and rooms are all since 6.3.4. ### Setting up a phone or a desktop client A client given the mail address finds the server by discovery: `/.well-known/carddav` and `/.well-known/caldav` redirect to `/dav/`, a PROPFIND there answers the principal, and the principal answers the address-book home and the calendar home. iOS, macOS Contacts and Calendar, DAVx5 on Android and Thunderbird all sync what is there. The IMAP logon's [lockout and auto-ban](/topic/28239) apply to the sign-in. **Which password.** An account with a TOTP second factor enrolled uses an app password, because once a secret exists the account password stops being a mailbox credential. Minting an app password on the webmail's Security page asks for the account's own password again, so an account that single sign-on or SCIM made, whose random password nobody was told, needs a password set by an administrator before its person can make an app password for a phone. **The listener.** The web services are off as shipped: `WebServicesHttpPort` and `WebServicesHttpsPort` default to `0`, and the HTTPS port takes `WebServicesCertificateFile` and `WebServicesPrivateKeyFile`. Since 6.3.4 these keys are [kept in the database](/topic/28222): change them in the Control Panel (Windows), over REST, or on Linux with `hmailserver --set-setting NAME=VALUE`, not by editing `hMailServer.ini`. A saved port or certificate waits for a reinitialise or a restart, since the listener binds when it starts. **On the Devices list** a DAV client appears with the protocol `dav` and its user agent. It has no connection to drop - each request authenticates afresh - so revoking it means revoking the app password behind it. | Collection | What it is | |---|---| | `/dav/calendars/
/calendar/` | The default calendar, `Calendar`, made the first time it is asked for | | `/dav/calendars/
//` | Another calendar of the account's own, made in the webmail or with `MKCALENDAR` | | `/dav/calendars/
/shared-/` | A calendar another account, or a group, shares with this one | | `/dav/addressbooks/
/contacts/` | The default address book, where a client that has synced since 6.3.3 still finds it | | `/dav/addressbooks/
/book-/` | Another book of the account's own (schema 6079) | | `/dav/addressbooks/
/shared-/` | A book shared with this account, or with the whole domain | | `/dav/addressbooks/
/directory/` | The organisation's directory, read-only (schema 6078) | Each calendar collection advertises `VEVENT` and `VTODO`, so a phone's calendar and its reminders list are the same collection. ### Calendars, events and tasks An account has one calendar until it makes more. The Calendars box beside the grid adds one with a name and a colour (`POST /api/v1/me/calendars`), renames or recolours it and deletes it with its events; the default cannot be deleted. Over CalDAV the colour travels as Apple's `calendar-color`, so a phone shows the same one, and `PROPPATCH` sets the name and colour from the phone's side. **Events.** An event takes a title, `start` and `end` as wall-clock times in `tz` (an IANA zone the server knows), `all_day`, a location and notes, `rrule` (daily, weekly on chosen days, monthly, yearly, with an interval and an end by count or by date), `alarm_minutes` and guests. The server writes `DTSTART;TZID=` so a weekly 09:00 stays 09:00 across a clock change, and a `VALARM` for the reminder, which is what CalDAV clients ring for. A repeating event is expanded across daylight saving in any of the 592 zone names the server knows; `calendar-query` with a time range, `calendar-multiget` and `sync-collection` are all answered, with ETags and `If-Match`. Changing one occurrence alone writes an override; deleting one writes an `EXDATE` on the master. **Tasks.** The Tasks page lists every `VTODO` of every calendar the account can see, soonest due first. Ticking one off writes `STATUS:COMPLETED`, `COMPLETED` and `PERCENT-COMPLETE:100` as a client does. A second calendar is a task list of its own. ### Sharing a calendar, and delegates The owner names another account of this server and whether it may write (`PUT /api/v1/me/calendars/{id}/shares/{address}` with `write`). The other account sees the calendar in its own Calendars box and in its CalDAV home as `shared-{id}/`, with `owner` pointing at the sharer's principal and a `current-user-privilege-set` of read, or read and write. A read-only share takes no PUT, no event from the page and no PROPPATCH. The sharee leaves with a CalDAV DELETE of `shared-{id}/` or from the page. **Shared with a group.** Since schema 6086 the name may be one of the server's groups (`hm_groups`) instead of an address; a name that is both is the account. The membership is read at the moment a member asks, so somebody added later sees the calendar and somebody removed no longer does. A member cannot leave a group share (403): they leave the group, or the owner takes the share back by the group's name. **A delegate.** A share may be a delegation (`delegate: true`, which implies `write`): the sharee may answer the owner's invitations on the owner's behalf. The answer is `POST /api/v1/me/calendar/events/{id}/reply` with `partstat` `ACCEPTED`, `TENTATIVE` or `DECLINED`, allowed to the owner and to a delegate and refused (403) to a plain writer or a reader. The organiser receives a `METHOD:REPLY` whose `ATTENDEE` carries `SENT-BY="mailto:"`, `From:` the owner with `Sender:` the delegate. ### Free-busy An account of the domain can see when a colleague is busy, and nothing else about them. `GET /api/v1/me/freebusy?address=&from=&to=` answers the busy periods of that account's own calendars - not the ones shared with it - over a window of at most 62 days (400 beyond). An event that is `TRANSP:TRANSPARENT` or `STATUS:CANCELLED`, or that the account declined, is left out; a period is `tentative` when the account's `PARTSTAT` is `TENTATIVE` or `NEEDS-ACTION`, else `busy`. No title, location or description is answered. Another hosted domain's account, an address nobody has, an inactive account, and an account whose `freebusy` preference is `nobody` are one and the same 404, so the route enumerates nothing. The preference is the person's own, in Settings: "Who may see when you are busy". Over CalDAV the same computation answers the `free-busy-query` REPORT on a calendar the caller may read, as one bare `VFREEBUSY`, over a `time-range` that needs both bounds and spans at most 62 days. The webmail's event editor draws an availability strip for the guests of a meeting from the same computation. ### Invitations by mail (iMIP) An event with guests, saved with "Send invitations by e-mail", mails each guest the event as a `text/calendar` part with `method=REQUEST` (RFC 6047) - what Outlook, Gmail and Apple Mail show as an invitation card. Saving a change mails an update with the sequence stepped; deleting the event mails `METHOD:CANCEL`. A message that carries a `text/calendar` part or an `.ics` attachment shows a card with Accept, Maybe, Decline and "Propose a new time" (`METHOD:COUNTER`). **Filed on delivery.** A `METHOD:REQUEST` delivered to an account is filed into its default calendar as it arrived (`NEEDS-ACTION` as a rule) when, and only when, it names the account as an `ATTENDEE` and not as the organiser. A message flagged as spam files nothing; a `REQUEST` naming somebody else is not kept; and an account that has never opened a calendar gets nothing filed. A stored copy under the same UID is replaced only by the same or a later `SEQUENCE`, and never when its `ORGANIZER` is not the invitation's. A `REPLY` delivered to the organiser sets that attendee's `PARTSTAT` in the stored event, so every phone syncing the calendar shows who accepted without anyone opening the mail; a `CANCEL` delivered to a guest marks the stored copy cancelled. ### Rooms and equipment (since 6.3.4) A room or a projector is an **account whose kind says so**: `person` (the default), `room` or `equipment`, a column of `hm_accounts` (schema 6101). It is invited like any guest and answers by itself, from its own address, by its booking policy; what it does not decide it leaves to its **delegates**. | Where | What | |---|---| | The Control Deck | The account editor's **General** group has a select of the three kinds; a room or equipment gets a **Booking policy** card | | REST | `kind` on `GET` and `PUT /api/v1/accounts/{address}`; the policy at `GET` and `PUT /api/v1/accounts/{address}/booking-policy` | | COM (Windows) | `Account.Kind` (0 person, 1 room, 2 equipment), saved by `Save()`; the booking policy is not on COM | | The Windows Control Panel | Does not show it: a room is an ordinary account to it | From the moment its kind is saved every sign-in as the room is refused, whatever the credential, with the ordinary wrong-password answer. A room with no policy row answers by the defaults: | Field | Default | What it decides | |---|---|---| | `auto_accept` | `true` | Answer invitations by this policy; `false` leaves every one for the delegates | | `window_days` | 180 | How many days ahead a booking may start, 1 to 1080 | | `max_duration_minutes` | 1440 | The longest booking; 0 is no limit | | `working_hours_only` | `false` | Only inside the working hours, on the working days | | `work_days`, `work_start`, `work_end` | Monday to Friday, 08:00 to 17:00 | The working days and hours | | `time_zone` | UTC | The zone the working hours are in, such as `Europe/London` | | `allow_conflicts` | `false` | Accept a booking over another one | | `allow_recurring` | `true` | Accept a recurring meeting | | `bookers` | empty | Empty is everyone in this server's own domains; a list is an address, `@domain`, a distribution list's address or a group's name. An organiser from outside must be named here | | `capacity` | 0 | How many people the room seats, shown in the directory; it decides nothing | | `delegates` | none | Addresses and group names, each given a delegate share of the room's calendar | **Who is asking.** The room first decides whether the invitation's `ORGANIZER` is who sent it, from what the server recorded when the message arrived and never from a `From:` header or a `MAIL FROM`. An organiser at this server's own domains is believed only when the message was submitted by a signed-in session as that account, or as one allowed to send as it; a message sent through an API key is not believed. An organiser from outside is believed only when `bookers` names the address or its `@domain` **and** the message passed DMARC for that domain. SPF on its own is not enough, and with DMARC off no outside organiser is ever believed. Anything else is neither filed nor answered, and the delegates are told, within limits. **The answer** is decided just after delivery, on a work queue of the rooms' own; no delivery and no SMTP answer waits for it. In order: who is asking; may they book; sent automatically (an empty reverse path or `Auto-Submitted`, never answered); `auto_accept`; at most 20 answers an hour to one organiser; recurrence; the window; the longest booking; working hours; conflicts, counting every booking the room holds that is not cancelled as busy. Accepted, the booking is filed and a `METHOD:REPLY` goes from the room's address, marked `Auto-Submitted: auto-replied`. Declined, the reply carries the reason alone; a conflict says when, never whose. A request left for the delegates is never put into the room's calendar on the sender's word. **A room with no delegates keeps what it leaves and tells nobody**: name at least one. Two servers sharing one database are not serialised for rooms: run rooms on one node. **Finding a room.** The directory lists a room with its kind and `capacity`, and its CardDAV card says `KIND:location` (`KIND:device` for equipment). Every CalDAV principal says its `calendar-user-type` - `INDIVIDUAL`, `ROOM` or `RESOURCE` - and the `principal-property-search` REPORT on `/dav/principals/`, which is what a client's room finder sends, answers the domain's rooms and equipment. Apple Calendar searches for locations this way; Thunderbird has no room finder. Every request stays in the room's mailbox, and a full mailbox answers nothing more, so give a room a quota and a `MessageRetentionDays` that suit it. ### Contacts, groups and several address books **Kept, and other contacts.** A contact has a `source`: `manual` for what the person added, imported or kept, and `collected` for every address the person has written to - from the webmail's send and, since 6.3.4, from a mail client over an authenticated SMTP submission. A collected address is offered under **Other contacts** and by the To, Cc and Bcc completion, and nowhere else: CardDAV serves only what the person kept, so a phone's address book is not filled with everyone ever written to. **Keep** makes it a contact. **What a card carries.** A card a CardDAV client sends is kept as sent and served back byte for byte. The page reads `birthday` (`BDAY`), `note`, `phones` (every `TEL`) and the embedded `PHOTO`; a photo that is only a URL is not the server's to fetch. **Groups.** A group is a card of kind group - `KIND:group` in vCard 4.0, `X-ADDRESSBOOKSERVER-KIND:group` in 3.0 - so a group made on the phone appears in the webmail and the reverse. A group's name in To becomes that group's recipients, so the reader sees the people and not the group. The ceilings hold on both surfaces: 500 members in one group (a card past it is refused `403 CARDDAV:valid-address-data`) and 200 groups in one address book (400 over REST, `507 Insufficient Storage` over CardDAV). A group is not a distribution list: nothing about it reaches SMTP. **Several address books, and shared ones.** Since schema 6079 the account starts with one book, `Contacts`, which cannot be deleted, and may have at most 50. The owner shares one from the Contacts page with an address - one account of this server - or with its own domain's name; *Can edit* makes the share writable, and a book takes at most 200 shares. A domain-wide share aimed at another hosted domain is 403, and a person cannot leave a domain share: only the owner or an administrator removes it. One account holds one address once across all its books, so a contact is moved between books, never copied. Over CardDAV a read-only collection grants `read` and `read-current-user-privilege-set` and nothing else, a PUT into it is 403, and DELETE on the collection gives the share back and deletes nobody's contacts. `MKCOL` is refused: a book is made from the Contacts page or `/api/v1/me/address-books`. No group is shared, in either direction. The backup carries the books and their contacts; `hm_addressbookshares` is not carried, so the sharing is set again. The Control Panel and the Control Deck have no page for this. An administrator has `GET /api/v1/accounts/{address}/address-books`, which shows the shape of the sharing and never the contacts, and a route that removes one share, recorded as `address-book-share-removed`. **Import and export.** The Contacts page imports a vCard file or a CSV, including Google's and Outlook's exports. *Export as vCard* downloads `contacts.vcf`; *Export as CSV* downloads `contacts.csv` and leaves groups out. ### The organisation's directory It is **one domain at a time, and always the reader's own**: every active account of the domain whose owner has not opted out, and every enabled distribution list, with what is set on top in `hm_directory` (schema 6078) - job title, department, work telephone, mobile, office, a photo (JPEG, PNG or GIF, at most 100 KB decoded, judged by its bytes), and a note that is the administrator's alone. **Two switches**, both rows of `hm_settings` and both on by default. `DirectoryEnabled` is whether the directory is answered at all; turn it off on a server whose one domain holds people who are not one organisation. With it off the webmail hides its Directory page and the CardDAV collection stays, empty, so a subscribed phone syncs to nothing rather than failing every poll. `DirectoryUserEditable` is whether a person may set their own fields. **The opt-out is never what a switch takes away**, and it is not invisibility: mail still arrives and the address still works. An administrator changing somebody's opt-out is on the audit trail as `directory-opt-out-changed`. **The surfaces.** The webmail's Directory page (`#/directory`); the To completion, which offers the reader's own contacts first and the directory hits after; `GET /api/v1/me/directory?q=&limit=` for the person and `GET` and `PUT /api/v1/accounts/{address}/directory` for the administrator; a **Directory entry** tab in the Windows Control Panel's account dialog, and the same card in the Control Deck; and the read-only CardDAV collection, where PUT, DELETE and PROPPATCH are refused 403. Nothing is learned from anywhere else: there is no import from Active Directory or LDAP, even on a server that authenticates against one, and no LDAP server for a mail client to search. ### What it does not do - No RFC 6638 scheduling (no `schedule-inbox` or `schedule-outbox`), no free-busy of a calendar on another server, no per-calendar time zone (`calendar-timezone` is not kept), no partial `calendar-data`, and no time-range filter on a `VALARM`. - No ActiveSync and no Exchange Web Services: a phone uses IMAP and SMTP for mail, and CalDAV and CardDAV for the rest. - Calendar shares and address-book shares are not in the backup, so set them again after a restore; the booking policy, the books, their contacts and the directory rows are. ### Where to read more - Wiki: [Connecting a Mail Client](https://gitlab.com/hmailserver/hmailserver/-/wikis/Connecting-a-Mail-Client), [Rooms and Equipment](https://gitlab.com/hmailserver/hmailserver/-/wikis/Rooms-and-Equipment), [The Webmail](https://gitlab.com/hmailserver/hmailserver/-/wikis/The-Webmail), [Ports Reference](https://gitlab.com/hmailserver/hmailserver/-/wikis/Ports-Reference). - Documents: [Calendar.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Calendar.md), [Contacts.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Contacts.md), [SharedAddressBooks.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SharedAddressBooks.md), [Directory.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Directory.md), [RoomsAndEquipment.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/RoomsAndEquipment.md). - On this forum: [The webmail portal at /portal](/topic/28191), [The Control Deck](/topic/28192), [Signing in](/topic/28253), [Connecting a mail client](/topic/28255), and the [6.3.4 announcement](/topic/28218). --- Canonical: https://www.hmailserver.co.uk/topic/28252/calendars-contacts-the-directory-and-shared-address-books-over-caldav-and-carddav # Calendars, contacts, the directory and shared address books over CalDAV and CardDAV This topic covers what hMailServer 6.3.4 serves to a phone or a desktop client over CalDAV and CardDAV, and what the webmail's Calendar, Tasks, Contacts and Directory pages do with the same data. The one thing to know first: CalDAV and CardDAV are served on the **web-services listener**, they take HTTP Basic as the account - its password or an app password - over HTTPS only, and a passkey or single sign-on does not reach them. CardDAV has been there since 6.3.3 with one address book; calendars, several and shared address books, the directory and rooms are all since 6.3.4. ### Setting up a phone or a desktop client A client given the mail address finds the server by discovery: `/.well-known/carddav` and `/.well-known/caldav` redirect to `/dav/`, a PROPFIND there answers the principal, and the principal answers the address-book home and the calendar home. iOS, macOS Contacts and Calendar, DAVx5 on Android and Thunderbird all sync what is there. The IMAP logon's [lockout and auto-ban](/topic/28239) apply to the sign-in. **Which password.** An account with a TOTP second factor enrolled uses an app password, because once a secret exists the account password stops being a mailbox credential. Minting an app password on the webmail's Security page asks for the account's own password again, so an account that single sign-on or SCIM made, whose random password nobody was told, needs a password set by an administrator before its person can make an app password for a phone. **The listener.** The web services are off as shipped: `WebServicesHttpPort` and `WebServicesHttpsPort` default to `0`, and the HTTPS port takes `WebServicesCertificateFile` and `WebServicesPrivateKeyFile`. Since 6.3.4 these keys are [kept in the database](/topic/28222): change them in the Control Panel (Windows), over REST, or on Linux with `hmailserver --set-setting NAME=VALUE`, not by editing `hMailServer.ini`. A saved port or certificate waits for a reinitialise or a restart, since the listener binds when it starts. **On the Devices list** a DAV client appears with the protocol `dav` and its user agent. It has no connection to drop - each request authenticates afresh - so revoking it means revoking the app password behind it. | Collection | What it is | |---|---| | `/dav/calendars/
/calendar/` | The default calendar, `Calendar`, made the first time it is asked for | | `/dav/calendars/
//` | Another calendar of the account's own, made in the webmail or with `MKCALENDAR` | | `/dav/calendars/
/shared-/` | A calendar another account, or a group, shares with this one | | `/dav/addressbooks/
/contacts/` | The default address book, where a client that has synced since 6.3.3 still finds it | | `/dav/addressbooks/
/book-/` | Another book of the account's own (schema 6079) | | `/dav/addressbooks/
/shared-/` | A book shared with this account, or with the whole domain | | `/dav/addressbooks/
/directory/` | The organisation's directory, read-only (schema 6078) | Each calendar collection advertises `VEVENT` and `VTODO`, so a phone's calendar and its reminders list are the same collection. ### Calendars, events and tasks An account has one calendar until it makes more. The Calendars box beside the grid adds one with a name and a colour (`POST /api/v1/me/calendars`), renames or recolours it and deletes it with its events; the default cannot be deleted. Over CalDAV the colour travels as Apple's `calendar-color`, so a phone shows the same one, and `PROPPATCH` sets the name and colour from the phone's side. **Events.** An event takes a title, `start` and `end` as wall-clock times in `tz` (an IANA zone the server knows), `all_day`, a location and notes, `rrule` (daily, weekly on chosen days, monthly, yearly, with an interval and an end by count or by date), `alarm_minutes` and guests. The server writes `DTSTART;TZID=` so a weekly 09:00 stays 09:00 across a clock change, and a `VALARM` for the reminder, which is what CalDAV clients ring for. A repeating event is expanded across daylight saving in any of the 592 zone names the server knows; `calendar-query` with a time range, `calendar-multiget` and `sync-collection` are all answered, with ETags and `If-Match`. Changing one occurrence alone writes an override; deleting one writes an `EXDATE` on the master. **Tasks.** The Tasks page lists every `VTODO` of every calendar the account can see, soonest due first. Ticking one off writes `STATUS:COMPLETED`, `COMPLETED` and `PERCENT-COMPLETE:100` as a client does. A second calendar is a task list of its own. ### Sharing a calendar, and delegates The owner names another account of this server and whether it may write (`PUT /api/v1/me/calendars/{id}/shares/{address}` with `write`). The other account sees the calendar in its own Calendars box and in its CalDAV home as `shared-{id}/`, with `owner` pointing at the sharer's principal and a `current-user-privilege-set` of read, or read and write. A read-only share takes no PUT, no event from the page and no PROPPATCH. The sharee leaves with a CalDAV DELETE of `shared-{id}/` or from the page. **Shared with a group.** Since schema 6086 the name may be one of the server's groups (`hm_groups`) instead of an address; a name that is both is the account. The membership is read at the moment a member asks, so somebody added later sees the calendar and somebody removed no longer does. A member cannot leave a group share (403): they leave the group, or the owner takes the share back by the group's name. **A delegate.** A share may be a delegation (`delegate: true`, which implies `write`): the sharee may answer the owner's invitations on the owner's behalf. The answer is `POST /api/v1/me/calendar/events/{id}/reply` with `partstat` `ACCEPTED`, `TENTATIVE` or `DECLINED`, allowed to the owner and to a delegate and refused (403) to a plain writer or a reader. The organiser receives a `METHOD:REPLY` whose `ATTENDEE` carries `SENT-BY="mailto:"`, `From:` the owner with `Sender:` the delegate. ### Free-busy An account of the domain can see when a colleague is busy, and nothing else about them. `GET /api/v1/me/freebusy?address=&from=&to=` answers the busy periods of that account's own calendars - not the ones shared with it - over a window of at most 62 days (400 beyond). An event that is `TRANSP:TRANSPARENT` or `STATUS:CANCELLED`, or that the account declined, is left out; a period is `tentative` when the account's `PARTSTAT` is `TENTATIVE` or `NEEDS-ACTION`, else `busy`. No title, location or description is answered. Another hosted domain's account, an address nobody has, an inactive account, and an account whose `freebusy` preference is `nobody` are one and the same 404, so the route enumerates nothing. The preference is the person's own, in Settings: "Who may see when you are busy". Over CalDAV the same computation answers the `free-busy-query` REPORT on a calendar the caller may read, as one bare `VFREEBUSY`, over a `time-range` that needs both bounds and spans at most 62 days. The webmail's event editor draws an availability strip for the guests of a meeting from the same computation. ### Invitations by mail (iMIP) An event with guests, saved with "Send invitations by e-mail", mails each guest the event as a `text/calendar` part with `method=REQUEST` (RFC 6047) - what Outlook, Gmail and Apple Mail show as an invitation card. Saving a change mails an update with the sequence stepped; deleting the event mails `METHOD:CANCEL`. A message that carries a `text/calendar` part or an `.ics` attachment shows a card with Accept, Maybe, Decline and "Propose a new time" (`METHOD:COUNTER`). **Filed on delivery.** A `METHOD:REQUEST` delivered to an account is filed into its default calendar as it arrived (`NEEDS-ACTION` as a rule) when, and only when, it names the account as an `ATTENDEE` and not as the organiser. A message flagged as spam files nothing; a `REQUEST` naming somebody else is not kept; and an account that has never opened a calendar gets nothing filed. A stored copy under the same UID is replaced only by the same or a later `SEQUENCE`, and never when its `ORGANIZER` is not the invitation's. A `REPLY` delivered to the organiser sets that attendee's `PARTSTAT` in the stored event, so every phone syncing the calendar shows who accepted without anyone opening the mail; a `CANCEL` delivered to a guest marks the stored copy cancelled. ### Rooms and equipment (since 6.3.4) A room or a projector is an **account whose kind says so**: `person` (the default), `room` or `equipment`, a column of `hm_accounts` (schema 6101). It is invited like any guest and answers by itself, from its own address, by its booking policy; what it does not decide it leaves to its **delegates**. | Where | What | |---|---| | The Control Deck | The account editor's **General** group has a select of the three kinds; a room or equipment gets a **Booking policy** card | | REST | `kind` on `GET` and `PUT /api/v1/accounts/{address}`; the policy at `GET` and `PUT /api/v1/accounts/{address}/booking-policy` | | COM (Windows) | `Account.Kind` (0 person, 1 room, 2 equipment), saved by `Save()`; the booking policy is not on COM | | The Windows Control Panel | Does not show it: a room is an ordinary account to it | From the moment its kind is saved every sign-in as the room is refused, whatever the credential, with the ordinary wrong-password answer. A room with no policy row answers by the defaults: | Field | Default | What it decides | |---|---|---| | `auto_accept` | `true` | Answer invitations by this policy; `false` leaves every one for the delegates | | `window_days` | 180 | How many days ahead a booking may start, 1 to 1080 | | `max_duration_minutes` | 1440 | The longest booking; 0 is no limit | | `working_hours_only` | `false` | Only inside the working hours, on the working days | | `work_days`, `work_start`, `work_end` | Monday to Friday, 08:00 to 17:00 | The working days and hours | | `time_zone` | UTC | The zone the working hours are in, such as `Europe/London` | | `allow_conflicts` | `false` | Accept a booking over another one | | `allow_recurring` | `true` | Accept a recurring meeting | | `bookers` | empty | Empty is everyone in this server's own domains; a list is an address, `@domain`, a distribution list's address or a group's name. An organiser from outside must be named here | | `capacity` | 0 | How many people the room seats, shown in the directory; it decides nothing | | `delegates` | none | Addresses and group names, each given a delegate share of the room's calendar | **Who is asking.** The room first decides whether the invitation's `ORGANIZER` is who sent it, from what the server recorded when the message arrived and never from a `From:` header or a `MAIL FROM`. An organiser at this server's own domains is believed only when the message was submitted by a signed-in session as that account, or as one allowed to send as it; a message sent through an API key is not believed. An organiser from outside is believed only when `bookers` names the address or its `@domain` **and** the message passed DMARC for that domain. SPF on its own is not enough, and with DMARC off no outside organiser is ever believed. Anything else is neither filed nor answered, and the delegates are told, within limits. **The answer** is decided just after delivery, on a work queue of the rooms' own; no delivery and no SMTP answer waits for it. In order: who is asking; may they book; sent automatically (an empty reverse path or `Auto-Submitted`, never answered); `auto_accept`; at most 20 answers an hour to one organiser; recurrence; the window; the longest booking; working hours; conflicts, counting every booking the room holds that is not cancelled as busy. Accepted, the booking is filed and a `METHOD:REPLY` goes from the room's address, marked `Auto-Submitted: auto-replied`. Declined, the reply carries the reason alone; a conflict says when, never whose. A request left for the delegates is never put into the room's calendar on the sender's word. **A room with no delegates keeps what it leaves and tells nobody**: name at least one. Two servers sharing one database are not serialised for rooms: run rooms on one node. **Finding a room.** The directory lists a room with its kind and `capacity`, and its CardDAV card says `KIND:location` (`KIND:device` for equipment). Every CalDAV principal says its `calendar-user-type` - `INDIVIDUAL`, `ROOM` or `RESOURCE` - and the `principal-property-search` REPORT on `/dav/principals/`, which is what a client's room finder sends, answers the domain's rooms and equipment. Apple Calendar searches for locations this way; Thunderbird has no room finder. Every request stays in the room's mailbox, and a full mailbox answers nothing more, so give a room a quota and a `MessageRetentionDays` that suit it. ### Contacts, groups and several address books **Kept, and other contacts.** A contact has a `source`: `manual` for what the person added, imported or kept, and `collected` for every address the person has written to - from the webmail's send and, since 6.3.4, from a mail client over an authenticated SMTP submission. A collected address is offered under **Other contacts** and by the To, Cc and Bcc completion, and nowhere else: CardDAV serves only what the person kept, so a phone's address book is not filled with everyone ever written to. **Keep** makes it a contact. **What a card carries.** A card a CardDAV client sends is kept as sent and served back byte for byte. The page reads `birthday` (`BDAY`), `note`, `phones` (every `TEL`) and the embedded `PHOTO`; a photo that is only a URL is not the server's to fetch. **Groups.** A group is a card of kind group - `KIND:group` in vCard 4.0, `X-ADDRESSBOOKSERVER-KIND:group` in 3.0 - so a group made on the phone appears in the webmail and the reverse. A group's name in To becomes that group's recipients, so the reader sees the people and not the group. The ceilings hold on both surfaces: 500 members in one group (a card past it is refused `403 CARDDAV:valid-address-data`) and 200 groups in one address book (400 over REST, `507 Insufficient Storage` over CardDAV). A group is not a distribution list: nothing about it reaches SMTP. **Several address books, and shared ones.** Since schema 6079 the account starts with one book, `Contacts`, which cannot be deleted, and may have at most 50. The owner shares one from the Contacts page with an address - one account of this server - or with its own domain's name; *Can edit* makes the share writable, and a book takes at most 200 shares. A domain-wide share aimed at another hosted domain is 403, and a person cannot leave a domain share: only the owner or an administrator removes it. One account holds one address once across all its books, so a contact is moved between books, never copied. Over CardDAV a read-only collection grants `read` and `read-current-user-privilege-set` and nothing else, a PUT into it is 403, and DELETE on the collection gives the share back and deletes nobody's contacts. `MKCOL` is refused: a book is made from the Contacts page or `/api/v1/me/address-books`. No group is shared, in either direction. The backup carries the books and their contacts; `hm_addressbookshares` is not carried, so the sharing is set again. The Control Panel and the Control Deck have no page for this. An administrator has `GET /api/v1/accounts/{address}/address-books`, which shows the shape of the sharing and never the contacts, and a route that removes one share, recorded as `address-book-share-removed`. **Import and export.** The Contacts page imports a vCard file or a CSV, including Google's and Outlook's exports. *Export as vCard* downloads `contacts.vcf`; *Export as CSV* downloads `contacts.csv` and leaves groups out. ### The organisation's directory It is **one domain at a time, and always the reader's own**: every active account of the domain whose owner has not opted out, and every enabled distribution list, with what is set on top in `hm_directory` (schema 6078) - job title, department, work telephone, mobile, office, a photo (JPEG, PNG or GIF, at most 100 KB decoded, judged by its bytes), and a note that is the administrator's alone. **Two switches**, both rows of `hm_settings` and both on by default. `DirectoryEnabled` is whether the directory is answered at all; turn it off on a server whose one domain holds people who are not one organisation. With it off the webmail hides its Directory page and the CardDAV collection stays, empty, so a subscribed phone syncs to nothing rather than failing every poll. `DirectoryUserEditable` is whether a person may set their own fields. **The opt-out is never what a switch takes away**, and it is not invisibility: mail still arrives and the address still works. An administrator changing somebody's opt-out is on the audit trail as `directory-opt-out-changed`. **The surfaces.** The webmail's Directory page (`#/directory`); the To completion, which offers the reader's own contacts first and the directory hits after; `GET /api/v1/me/directory?q=&limit=` for the person and `GET` and `PUT /api/v1/accounts/{address}/directory` for the administrator; a **Directory entry** tab in the Windows Control Panel's account dialog, and the same card in the Control Deck; and the read-only CardDAV collection, where PUT, DELETE and PROPPATCH are refused 403. Nothing is learned from anywhere else: there is no import from Active Directory or LDAP, even on a server that authenticates against one, and no LDAP server for a mail client to search. ### What it does not do - No RFC 6638 scheduling (no `schedule-inbox` or `schedule-outbox`), no free-busy of a calendar on another server, no per-calendar time zone (`calendar-timezone` is not kept), no partial `calendar-data`, and no time-range filter on a `VALARM`. - No ActiveSync and no Exchange Web Services: a phone uses IMAP and SMTP for mail, and CalDAV and CardDAV for the rest. - Calendar shares and address-book shares are not in the backup, so set them again after a restore; the booking policy, the books, their contacts and the directory rows are. ### Where to read more - Wiki: [Connecting a Mail Client](https://gitlab.com/hmailserver/hmailserver/-/wikis/Connecting-a-Mail-Client), [Rooms and Equipment](https://gitlab.com/hmailserver/hmailserver/-/wikis/Rooms-and-Equipment), [The Webmail](https://gitlab.com/hmailserver/hmailserver/-/wikis/The-Webmail), [Ports Reference](https://gitlab.com/hmailserver/hmailserver/-/wikis/Ports-Reference). - Documents: [Calendar.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Calendar.md), [Contacts.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Contacts.md), [SharedAddressBooks.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/SharedAddressBooks.md), [Directory.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/Directory.md), [RoomsAndEquipment.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/RoomsAndEquipment.md). - On this forum: [The webmail portal at /portal](/topic/28191), [The Control Deck](/topic/28192), [Signing in](/topic/28253), [Connecting a mail client](/topic/28255), and the [6.3.4 announcement](/topic/28218). --- Canonical: https://www.hmailserver.co.uk/topic/28253/signing-in-passwords-second-factor-passkeys-single-sign-on-recovery-devices # Signing in: passwords, second factor, passkeys, single sign-on, recovery, devices *Written for 6.3.4. hMailServer 6.3.7 (17 October 2026) refuses a state-changing REST request that carries a credential - HTTP Basic included, which a browser keeps once it has answered the sign-in box - when the browser reports it as cross-site (`Sec-Fetch-Site`, or an `Origin` that is not this server's), and holds the password-reset request to the same rule, so another site cannot have a current browser drive it; the webmail and the Control Deck act on their own origin, and scripts, `hmctl` and the PowerShell module send no such header, so none of them is affected. SMTP, POP3 and ManageSieve now refuse a SASL PLAIN sign-in that asks to act as another account. See [the 6.3.7 announcement](/topic/28277).* This topic covers how people prove who they are to hMailServer 6.3.4: what 6.3.3 already had, and what 6.3.4 adds. The one thing to know first is that there are two front doors. The **mail protocols** - IMAP, POP3, SMTP submission, CalDAV and CardDAV - take a password: the account's own, an app password, or an OAuth2 bearer token where that is switched on. The **two pages** - the webmail at `/portal` and the Control Deck at `/` - take the password as well and, since 6.3.4, a passkey or a sign-in at the organisation's identity provider. Operators upgrading should read the note on passwords with characters above U+00FF first. ### What 6.3.3 already had | Feature | What it is | |---|---| | A second factor for an account | A TOTP secret on the mailbox. Once one exists, the account's password stops being a credential wherever a code cannot be presented alongside it - every mail client, and the webmail's sign-in - and app passwords are what those use | | App passwords | One per device, shown once, revocable one at a time; they work for directory-linked accounts too and are untouched by password expiry | | A second factor for the administrator | Enforced by the server on every surface - COM, REST, the Control Panel | | The password policy | Length, mixed case, digits, symbols, a common-password list, history and maximum age | | LDAP and Active Directory | Passwords checked against the directory; provisioning from it on a preview-then-apply | | OAuth2 bearer tokens on the protocols | SASL XOAUTH2 and OAUTHBEARER validated against an identity provider's published keys | There is no self-service enrolment of an authenticator app in the webmail: an administrator sets and clears the second factor, over COM and the Windows Control Panel. Create the app passwords first and enrol the secret second, or every configured client stops working with no way back in. **The breach check (since 6.3.4).** A password being chosen, never at logon, can be refused because it is in the published corpus of breached passwords; the check is **off by default**, because it sends five characters of the password's SHA-1 hash off the server, and its settings are in [Hardening a public mail server](/topic/28241). **Four more changes in 6.3.4.** The administrator password, which the Windows installer wrote as an unsalted MD5 digest, is written again as PBKDF2 at the first successful administrator sign-in (a file the service cannot write keeps the old value and says so once, `HM7494`). An account's TOTP secret is sealed at rest once the installation has a stored-secrets key. A mailbox encrypted at rest takes its own password only: no TOTP secret can be enrolled on it, and its app passwords are refused. And a room or a piece of equipment cannot be signed into by any credential. ### Upgrading to 6.3.4: passwords with characters above U+00FF Before 6.3.4 every password hash was made over a conversion that gave NUL bytes on Windows from the first character above U+00FF on, and `?` for each such character on Linux, so a password of Cyrillic, Greek or CJK letters could be opened by any other password of its length. Hashes are now made over UTF-8, which leaves every ASCII password's hash as it was. The consequence, item 24 of the CHANGELOG's upgrade list: **a password hash stored before 6.3.4 of a password with a character above U+00FF is refused**, to its own password too, and the application log names the account. An administrator sets such a password again, and soon: until then `SCRAM-SHA-256`, whose client derives its key itself, still opens such a PBKDF2 hash to the NUL bytes (Windows) or question marks (Linux) it was made over, which the server cannot tell from the password; and on Linux such a hash also still opens to the password written with `?` for those characters. Every ASCII and Latin-1 password signs in as before. The same release fixed SCRAM-SHA-256 having accepted the account's own password for an account with a second factor enrolled, and a password past `PasswordPolicyMaximumAgeDays`; each is now a forced failure of the exchange. SCRAM is always the account's own password, since an app password's hash is not one SCRAM can derive from, so such an account cannot sign in over SCRAM at all. ### Passkeys A passkey (schema 6077) is a WebAuthn credential held by the person's own device, such as a phone or a security key. The private half never leaves it; the server keeps the public half. Passkeys are offered on a stock installation, and nobody has one until they enrol it. **Enrolling.** The webmail's **Security** page has a Passkeys card, and the Control Deck has a Passkeys view for the administrator's own. Both ask for the owner's password again, with the one-time code in `X-hMailServer-OTP` where a second factor is enrolled, or for an account one of its recovery codes in `X-hMailServer-Recovery-Code`; a request that was itself authenticated with an app password is refused outright. At most ten per owner. ``` POST /api/v1/passkeys/options what the browser needs to make one POST /api/v1/passkeys the credential it made GET /api/v1/passkeys the ones this credential holds DELETE /api/v1/passkeys/{id} remove one ``` Listing and removing are answered while `passkeys_enabled` is off as well. **Signing in.** `POST /api/v1/passkeys/challenge` hands out thirty-two random bytes, kept in memory for five minutes and spent on first use; the assertion goes to `POST /api/v1/session`, the one place this server mints a browser session, so the sign-in gets the same ceilings and a device-list entry marked `passkey`. Nobody types an address first. **A passkey stands in for the one-time code**: user verification - a PIN, a fingerprint or a face - is required at registration and on every assertion. It carries nobody past an inactive account or domain, the compromised-account hold or the domain's country policy. **Behind several names.** By default the relying party id is the `Host` header the page was served on. `passkey_relying_party` set to a registrable parent (`example.com`) lets a passkey made at `mail.example.com` work at `webmail.example.com`. With it set, **`passkey_allowed_origins` must be set too**, comma separated as the browser writes them (`https://mail.example.com, https://webmail.example.com`); until it is, no passkey is registered or taken. **Not verified:** the attestation statement, so an installation cannot say "only hardware keys of this make". **Not in the backup:** enrol the device again after a restore elsewhere. Registering and removing a passkey is on the audit trail. ### Single sign-on Since 6.3.4 a person can sign in at the organisation's identity provider instead, by OpenID Connect or by SAML 2.0 (for a provider such as ADFS that speaks only SAML). Once it is set up, both sign-in pages draw a button: **Sign in with** and the issuer's host name, or **Sign in with SAML**. It opens the webmail and the Control Deck and nothing else: IMAP, POP3 and SMTP still take the password or an app password. **An identity provider that says somebody is an administrator is not obeyed**; the Deck opens as the administrator only for an address listed in `oidc_administrators` or `saml_administrators`. There is no logout in either direction, so a person disabled at the provider keeps a session already started here until its own ceiling ends it, which is what the session policies below are for. The settings, the Entra ID, Okta and ADFS entries and the flows are in [SCIM provisioning and single sign-on](/topic/28267). ### Account recovery Three independent parts (schema 6076): a recovery address and ten one-time codes, which the person sets on the webmail's Security page, and the forgotten-password flow on the sign-in card. Setting either asks for the account's own password again - never an app password - and the second factor where there is one. **What it needs.** `account_recovery_enabled` is on by default and inert on a stock installation: links are written against the server's public base URL - `confidential_base_url`, falling back to `quarantine_digest_base_url` and then the import sign-in's `import_oauth_redirect_base_url` - and with none set `POST /api/v1/password-reset` answers 503. `recovery_reset_link_minutes` (30; 5 to 1440) is how long a reset link lives, and `recovery_verify_link_hours` (24; 1 to 168) how long the confirmation link lives. **The recovery address** counts for nothing until the link mailed to it is followed, on a page with a button rather than on the GET, which a mail scanner would follow first. It is never answered in full again, to anybody: `other@elsewhere.example` reads as `o....@e.........example`. The confirmation is metered at one a minute and six an hour for the address, and again for the asking account, answering 429 with `retry_after`. **The ten one-time codes** are made together, shown once and stored hashed. A code goes in `X-hMailServer-Recovery-Code` wherever an authenticator's code is asked for: `POST /api/v1/session`, `POST /api/v1/me/password`, `POST /api/v1/me/app-passwords`, `PUT /api/v1/me/recovery` and `POST /api/v1/me/recovery/codes`. The password is verified before the code is looked at, and the code is spent only once the whole attempt has been accepted. Every owner has a budget that needs no setting: **five wrong codes in fifteen minutes, and for the next fifteen every code for that owner is refused** - the right one too - with 429 and `code` `second_factor_locked`. And a code is taken once (RFC 6238, section 5.2): an accepted TOTP code is refused if it comes again within its window, without being counted as wrong. A script that sent the administrator's password and a code with every request signs in once instead, or uses an API key. **Forgotten your password?** on the sign-in card posts the address to `POST /api/v1/password-reset`, which takes no credentials and answers **always the same** - 202 and one sentence, in the same time - whether or not the address is a mailbox with a confirmed recovery address. Since 6.3.7 a request a browser reports as cross-site (`Sec-Fetch-Site`, or an `Origin` that is not this server's) is refused, so another site cannot have a current browser drive it. The limits are one request a minute and six an hour per address, and twenty an hour per client address; a loopback peer, such as a reverse proxy on the same machine, is held to thirty in any sixty seconds instead. The link goes to the confirmed recovery address and nowhere else, is good once, and dies after `recovery_reset_link_minutes`. Completing it applies the password policy, the reuse history and the breach check, **ends every browser session of the account**, tells the mailbox and the recovery address, and raises the `account.password-reset` alert. A used, expired or unknown token answers 410. **The administrator** can see whether an address is set and confirmed, its masked spelling and how many codes are unused - over REST, on the Deck's **Recovery** card and the Windows Control Panel's **Recovery** tab - and can clear either: `DELETE /api/v1/accounts/{address}/recovery` and `DELETE /api/v1/accounts/{address}/recovery/codes`. The administrator cannot read the address in full, read a code, set an address on somebody's behalf or generate codes for them. **A recovery address is exactly as strong as the mailbox that receives it**: a convenience, not a second factor, and an installation that cannot accept that sets `account_recovery_enabled` to 0. Recovery does not enrol a new authenticator either. Nor does a reset link open encrypted mail: for a mailbox encrypted at rest the link is refused before it is spent and the person is sent to their administrator, and for a zero-access mailbox the password it sets does not open the key, so at the next sign-in the page asks for the previous password or the recovery key. ### Devices and sessions Every client that has authenticated to an account is one entry of a list: the protocol (`imap`, `pop3`, `smtp`, `web`, `dav`, `autodiscover` or `managesieve`), the address it came from, the user agent where the protocol carries one, the credential it used, the app password's name where one was used, and when it was first and last seen. IMAP's `ID` command is deliberately not used, so two mail programs on one machine over IMAP are one entry. At most fifty entries are kept per account. | Surface | Read | Revoke | |---|---|---| | Webmail | *Settings → Security*, the **Devices** card | *Revoke* beside the entry; the box under the table adds the app password | | REST, the account | `GET /api/v1/me/devices` | `DELETE /api/v1/me/devices/{id}`, with `?credential=revoke` and `?forget=true` | | REST, an administrator | `GET /api/v1/accounts/{address}/devices` | `DELETE /api/v1/accounts/{address}/devices/{id}` | | COM (Windows) | `Account.GetDevices()` | `Account.RevokeDevice(id, revokeCredential)` | | Control Deck | The **Devices** view | *Revoke*, with the app password box | | `hmctl` | `hmctl device ann@example.com list` | `hmctl device ann@example.com revoke ID [--credential] [--forget]` | | PowerShell | `Get-HmDevice -Address ann@example.com` | `Revoke-HmDevice -Address ann@example.com -DeviceId ID [-Credential] [-Forget]` | **What revoking does, exactly.** A browser session's token is invalidated, so the next request on that cookie is 401; a live IMAP, POP3 or SMTP connection is dropped where it stands; and with `credential=revoke` the app password is deactivated - not deleted, so the person can put it back if the phone turns up. CalDAV, CardDAV and ManageSieve clients have no connection to drop, so revoking one means revoking its app password. **A revoke is not a ban**: a client with a credential that still works signs in again and clears the mark. The account's own revoke is refused (403) to a request that came in on an app password, and an administrator's revoke is recorded in the audit trail. The list, `hm_signins`, is not in the backup. **The session policies** govern every signed-in browser - the webmail, the portal and the Deck - server-wide and per domain, and are re-read at every request. A mail program's connection is not a session in this sense. | Policy | Setting | Unset means | |---|---|---| | Idle timeout | `session_idle_seconds` | 1800 seconds; at most 2592000 (thirty days) | | Absolute lifetime | `session_lifetime_seconds` | 43200 seconds, twelve hours; at most 2592000 | | One session per device | `session_one_per_device` (a domain's: 0 the server's answer, 1 on, 2 off) | Off | | Bound to the address it began on | `session_bind_to_address` (0, 1 or 2 as above) | Off - it ends a phone's session when it moves from wi-fi to mobile | ```bash hmctl settings set --set session_idle_seconds=900 --set session_bind_to_address=true hmctl domain update example.com --set session_lifetime_seconds=28800 --set session_one_per_device=1 ``` A domain's value wins over the server's where it is set. **Nothing changes on the day you upgrade**: every policy reads 0, which means the ceilings the server has always had. The cookie, `hmailsession`, is HttpOnly and `SameSite=Strict`, and `Secure` whenever the browser came over HTTPS; a proxy that terminates HTTPS in front of the loopback listener should set `X-Forwarded-Proto: https`. Browser sessions live in the server's memory, so a restart ends every one, and a password change ends every session and drops every connection of the account. ### What none of this does - It does not reach the mail protocols. A phone's IMAP and SMTP still take the password or an app password, and an account that only ever signs in through its identity provider needs a password set before its person can mint an app password. - No password-less accounts: a passkey sits beside the password, and no policy forbids the password. ### Where to read more - Wiki: [Sign-in and Identity](https://gitlab.com/hmailserver/hmailserver/-/wikis/Sign-in-and-Identity), [Connecting a Mail Client](https://gitlab.com/hmailserver/hmailserver/-/wikis/Connecting-a-Mail-Client), [Security Hardening](https://gitlab.com/hmailserver/hmailserver/-/wikis/Security-Hardening). - Documents: [PasskeysAndSingleSignOn.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/PasskeysAndSingleSignOn.md), [AccountRecovery.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/AccountRecovery.md), [DevicesAndSessions.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/DevicesAndSessions.md), [PasswordBreachCheck.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/PasswordBreachCheck.md), and item 24 of the 6.3.4 section of the [CHANGELOG](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/CHANGELOG.md). - The public documentation: [17.2 Modern authentication](https://www.progressiverobot.com/hmailserver-documentation/#172-modern-authentication). - On this forum: [Connecting a mail client](/topic/28255) for app passwords on a phone, [Active Directory and LDAP sign-in](/topic/28263), [encryption at rest](/topic/28233), [Upgrading to 6.3.4](/topic/28193), [The Control Deck](/topic/28192), [The webmail portal at /portal](/topic/28191) and the [6.3.4 announcement](/topic/28218). --- Canonical: https://www.hmailserver.co.uk/topic/28253/signing-in-passwords-second-factor-passkeys-single-sign-on-recovery-devices # Signing in: passwords, second factor, passkeys, single sign-on, recovery, devices *Written for 6.3.4. hMailServer 6.3.7 (17 October 2026) refuses a state-changing REST request that carries a credential - HTTP Basic included, which a browser keeps once it has answered the sign-in box - when the browser reports it as cross-site (`Sec-Fetch-Site`, or an `Origin` that is not this server's), and holds the password-reset request to the same rule, so another site cannot have a current browser drive it; the webmail and the Control Deck act on their own origin, and scripts, `hmctl` and the PowerShell module send no such header, so none of them is affected. SMTP, POP3 and ManageSieve now refuse a SASL PLAIN sign-in that asks to act as another account. See [the 6.3.7 announcement](/topic/28277).* This topic covers how people prove who they are to hMailServer 6.3.4: what 6.3.3 already had, and what 6.3.4 adds. The one thing to know first is that there are two front doors. The **mail protocols** - IMAP, POP3, SMTP submission, CalDAV and CardDAV - take a password: the account's own, an app password, or an OAuth2 bearer token where that is switched on. The **two pages** - the webmail at `/portal` and the Control Deck at `/` - take the password as well and, since 6.3.4, a passkey or a sign-in at the organisation's identity provider. Operators upgrading should read the note on passwords with characters above U+00FF first. ### What 6.3.3 already had | Feature | What it is | |---|---| | A second factor for an account | A TOTP secret on the mailbox. Once one exists, the account's password stops being a credential wherever a code cannot be presented alongside it - every mail client, and the webmail's sign-in - and app passwords are what those use | | App passwords | One per device, shown once, revocable one at a time; they work for directory-linked accounts too and are untouched by password expiry | | A second factor for the administrator | Enforced by the server on every surface - COM, REST, the Control Panel | | The password policy | Length, mixed case, digits, symbols, a common-password list, history and maximum age | | LDAP and Active Directory | Passwords checked against the directory; provisioning from it on a preview-then-apply | | OAuth2 bearer tokens on the protocols | SASL XOAUTH2 and OAUTHBEARER validated against an identity provider's published keys | There is no self-service enrolment of an authenticator app in the webmail: an administrator sets and clears the second factor, over COM and the Windows Control Panel. Create the app passwords first and enrol the secret second, or every configured client stops working with no way back in. **The breach check (since 6.3.4).** A password being chosen, never at logon, can be refused because it is in the published corpus of breached passwords; the check is **off by default**, because it sends five characters of the password's SHA-1 hash off the server, and its settings are in [Hardening a public mail server](/topic/28241). **Four more changes in 6.3.4.** The administrator password, which the Windows installer wrote as an unsalted MD5 digest, is written again as PBKDF2 at the first successful administrator sign-in (a file the service cannot write keeps the old value and says so once, `HM7494`). An account's TOTP secret is sealed at rest once the installation has a stored-secrets key. A mailbox encrypted at rest takes its own password only: no TOTP secret can be enrolled on it, and its app passwords are refused. And a room or a piece of equipment cannot be signed into by any credential. ### Upgrading to 6.3.4: passwords with characters above U+00FF Before 6.3.4 every password hash was made over a conversion that gave NUL bytes on Windows from the first character above U+00FF on, and `?` for each such character on Linux, so a password of Cyrillic, Greek or CJK letters could be opened by any other password of its length. Hashes are now made over UTF-8, which leaves every ASCII password's hash as it was. The consequence, item 24 of the CHANGELOG's upgrade list: **a password hash stored before 6.3.4 of a password with a character above U+00FF is refused**, to its own password too, and the application log names the account. An administrator sets such a password again, and soon: until then `SCRAM-SHA-256`, whose client derives its key itself, still opens such a PBKDF2 hash to the NUL bytes (Windows) or question marks (Linux) it was made over, which the server cannot tell from the password; and on Linux such a hash also still opens to the password written with `?` for those characters. Every ASCII and Latin-1 password signs in as before. The same release fixed SCRAM-SHA-256 having accepted the account's own password for an account with a second factor enrolled, and a password past `PasswordPolicyMaximumAgeDays`; each is now a forced failure of the exchange. SCRAM is always the account's own password, since an app password's hash is not one SCRAM can derive from, so such an account cannot sign in over SCRAM at all. ### Passkeys A passkey (schema 6077) is a WebAuthn credential held by the person's own device, such as a phone or a security key. The private half never leaves it; the server keeps the public half. Passkeys are offered on a stock installation, and nobody has one until they enrol it. **Enrolling.** The webmail's **Security** page has a Passkeys card, and the Control Deck has a Passkeys view for the administrator's own. Both ask for the owner's password again, with the one-time code in `X-hMailServer-OTP` where a second factor is enrolled, or for an account one of its recovery codes in `X-hMailServer-Recovery-Code`; a request that was itself authenticated with an app password is refused outright. At most ten per owner. ``` POST /api/v1/passkeys/options what the browser needs to make one POST /api/v1/passkeys the credential it made GET /api/v1/passkeys the ones this credential holds DELETE /api/v1/passkeys/{id} remove one ``` Listing and removing are answered while `passkeys_enabled` is off as well. **Signing in.** `POST /api/v1/passkeys/challenge` hands out thirty-two random bytes, kept in memory for five minutes and spent on first use; the assertion goes to `POST /api/v1/session`, the one place this server mints a browser session, so the sign-in gets the same ceilings and a device-list entry marked `passkey`. Nobody types an address first. **A passkey stands in for the one-time code**: user verification - a PIN, a fingerprint or a face - is required at registration and on every assertion. It carries nobody past an inactive account or domain, the compromised-account hold or the domain's country policy. **Behind several names.** By default the relying party id is the `Host` header the page was served on. `passkey_relying_party` set to a registrable parent (`example.com`) lets a passkey made at `mail.example.com` work at `webmail.example.com`. With it set, **`passkey_allowed_origins` must be set too**, comma separated as the browser writes them (`https://mail.example.com, https://webmail.example.com`); until it is, no passkey is registered or taken. **Not verified:** the attestation statement, so an installation cannot say "only hardware keys of this make". **Not in the backup:** enrol the device again after a restore elsewhere. Registering and removing a passkey is on the audit trail. ### Single sign-on Since 6.3.4 a person can sign in at the organisation's identity provider instead, by OpenID Connect or by SAML 2.0 (for a provider such as ADFS that speaks only SAML). Once it is set up, both sign-in pages draw a button: **Sign in with** and the issuer's host name, or **Sign in with SAML**. It opens the webmail and the Control Deck and nothing else: IMAP, POP3 and SMTP still take the password or an app password. **An identity provider that says somebody is an administrator is not obeyed**; the Deck opens as the administrator only for an address listed in `oidc_administrators` or `saml_administrators`. There is no logout in either direction, so a person disabled at the provider keeps a session already started here until its own ceiling ends it, which is what the session policies below are for. The settings, the Entra ID, Okta and ADFS entries and the flows are in [SCIM provisioning and single sign-on](/topic/28267). ### Account recovery Three independent parts (schema 6076): a recovery address and ten one-time codes, which the person sets on the webmail's Security page, and the forgotten-password flow on the sign-in card. Setting either asks for the account's own password again - never an app password - and the second factor where there is one. **What it needs.** `account_recovery_enabled` is on by default and inert on a stock installation: links are written against the server's public base URL - `confidential_base_url`, falling back to `quarantine_digest_base_url` and then the import sign-in's `import_oauth_redirect_base_url` - and with none set `POST /api/v1/password-reset` answers 503. `recovery_reset_link_minutes` (30; 5 to 1440) is how long a reset link lives, and `recovery_verify_link_hours` (24; 1 to 168) how long the confirmation link lives. **The recovery address** counts for nothing until the link mailed to it is followed, on a page with a button rather than on the GET, which a mail scanner would follow first. It is never answered in full again, to anybody: `other@elsewhere.example` reads as `o....@e.........example`. The confirmation is metered at one a minute and six an hour for the address, and again for the asking account, answering 429 with `retry_after`. **The ten one-time codes** are made together, shown once and stored hashed. A code goes in `X-hMailServer-Recovery-Code` wherever an authenticator's code is asked for: `POST /api/v1/session`, `POST /api/v1/me/password`, `POST /api/v1/me/app-passwords`, `PUT /api/v1/me/recovery` and `POST /api/v1/me/recovery/codes`. The password is verified before the code is looked at, and the code is spent only once the whole attempt has been accepted. Every owner has a budget that needs no setting: **five wrong codes in fifteen minutes, and for the next fifteen every code for that owner is refused** - the right one too - with 429 and `code` `second_factor_locked`. And a code is taken once (RFC 6238, section 5.2): an accepted TOTP code is refused if it comes again within its window, without being counted as wrong. A script that sent the administrator's password and a code with every request signs in once instead, or uses an API key. **Forgotten your password?** on the sign-in card posts the address to `POST /api/v1/password-reset`, which takes no credentials and answers **always the same** - 202 and one sentence, in the same time - whether or not the address is a mailbox with a confirmed recovery address. Since 6.3.7 a request a browser reports as cross-site (`Sec-Fetch-Site`, or an `Origin` that is not this server's) is refused, so another site cannot have a current browser drive it. The limits are one request a minute and six an hour per address, and twenty an hour per client address; a loopback peer, such as a reverse proxy on the same machine, is held to thirty in any sixty seconds instead. The link goes to the confirmed recovery address and nowhere else, is good once, and dies after `recovery_reset_link_minutes`. Completing it applies the password policy, the reuse history and the breach check, **ends every browser session of the account**, tells the mailbox and the recovery address, and raises the `account.password-reset` alert. A used, expired or unknown token answers 410. **The administrator** can see whether an address is set and confirmed, its masked spelling and how many codes are unused - over REST, on the Deck's **Recovery** card and the Windows Control Panel's **Recovery** tab - and can clear either: `DELETE /api/v1/accounts/{address}/recovery` and `DELETE /api/v1/accounts/{address}/recovery/codes`. The administrator cannot read the address in full, read a code, set an address on somebody's behalf or generate codes for them. **A recovery address is exactly as strong as the mailbox that receives it**: a convenience, not a second factor, and an installation that cannot accept that sets `account_recovery_enabled` to 0. Recovery does not enrol a new authenticator either. Nor does a reset link open encrypted mail: for a mailbox encrypted at rest the link is refused before it is spent and the person is sent to their administrator, and for a zero-access mailbox the password it sets does not open the key, so at the next sign-in the page asks for the previous password or the recovery key. ### Devices and sessions Every client that has authenticated to an account is one entry of a list: the protocol (`imap`, `pop3`, `smtp`, `web`, `dav`, `autodiscover` or `managesieve`), the address it came from, the user agent where the protocol carries one, the credential it used, the app password's name where one was used, and when it was first and last seen. IMAP's `ID` command is deliberately not used, so two mail programs on one machine over IMAP are one entry. At most fifty entries are kept per account. | Surface | Read | Revoke | |---|---|---| | Webmail | *Settings → Security*, the **Devices** card | *Revoke* beside the entry; the box under the table adds the app password | | REST, the account | `GET /api/v1/me/devices` | `DELETE /api/v1/me/devices/{id}`, with `?credential=revoke` and `?forget=true` | | REST, an administrator | `GET /api/v1/accounts/{address}/devices` | `DELETE /api/v1/accounts/{address}/devices/{id}` | | COM (Windows) | `Account.GetDevices()` | `Account.RevokeDevice(id, revokeCredential)` | | Control Deck | The **Devices** view | *Revoke*, with the app password box | | `hmctl` | `hmctl device ann@example.com list` | `hmctl device ann@example.com revoke ID [--credential] [--forget]` | | PowerShell | `Get-HmDevice -Address ann@example.com` | `Revoke-HmDevice -Address ann@example.com -DeviceId ID [-Credential] [-Forget]` | **What revoking does, exactly.** A browser session's token is invalidated, so the next request on that cookie is 401; a live IMAP, POP3 or SMTP connection is dropped where it stands; and with `credential=revoke` the app password is deactivated - not deleted, so the person can put it back if the phone turns up. CalDAV, CardDAV and ManageSieve clients have no connection to drop, so revoking one means revoking its app password. **A revoke is not a ban**: a client with a credential that still works signs in again and clears the mark. The account's own revoke is refused (403) to a request that came in on an app password, and an administrator's revoke is recorded in the audit trail. The list, `hm_signins`, is not in the backup. **The session policies** govern every signed-in browser - the webmail, the portal and the Deck - server-wide and per domain, and are re-read at every request. A mail program's connection is not a session in this sense. | Policy | Setting | Unset means | |---|---|---| | Idle timeout | `session_idle_seconds` | 1800 seconds; at most 2592000 (thirty days) | | Absolute lifetime | `session_lifetime_seconds` | 43200 seconds, twelve hours; at most 2592000 | | One session per device | `session_one_per_device` (a domain's: 0 the server's answer, 1 on, 2 off) | Off | | Bound to the address it began on | `session_bind_to_address` (0, 1 or 2 as above) | Off - it ends a phone's session when it moves from wi-fi to mobile | ```bash hmctl settings set --set session_idle_seconds=900 --set session_bind_to_address=true hmctl domain update example.com --set session_lifetime_seconds=28800 --set session_one_per_device=1 ``` A domain's value wins over the server's where it is set. **Nothing changes on the day you upgrade**: every policy reads 0, which means the ceilings the server has always had. The cookie, `hmailsession`, is HttpOnly and `SameSite=Strict`, and `Secure` whenever the browser came over HTTPS; a proxy that terminates HTTPS in front of the loopback listener should set `X-Forwarded-Proto: https`. Browser sessions live in the server's memory, so a restart ends every one, and a password change ends every session and drops every connection of the account. ### What none of this does - It does not reach the mail protocols. A phone's IMAP and SMTP still take the password or an app password, and an account that only ever signs in through its identity provider needs a password set before its person can mint an app password. - No password-less accounts: a passkey sits beside the password, and no policy forbids the password. ### Where to read more - Wiki: [Sign-in and Identity](https://gitlab.com/hmailserver/hmailserver/-/wikis/Sign-in-and-Identity), [Connecting a Mail Client](https://gitlab.com/hmailserver/hmailserver/-/wikis/Connecting-a-Mail-Client), [Security Hardening](https://gitlab.com/hmailserver/hmailserver/-/wikis/Security-Hardening). - Documents: [PasskeysAndSingleSignOn.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/PasskeysAndSingleSignOn.md), [AccountRecovery.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/AccountRecovery.md), [DevicesAndSessions.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/DevicesAndSessions.md), [PasswordBreachCheck.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/PasswordBreachCheck.md), and item 24 of the 6.3.4 section of the [CHANGELOG](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/CHANGELOG.md). - The public documentation: [17.2 Modern authentication](https://www.progressiverobot.com/hmailserver-documentation/#172-modern-authentication). - On this forum: [Connecting a mail client](/topic/28255) for app passwords on a phone, [Active Directory and LDAP sign-in](/topic/28263), [encryption at rest](/topic/28233), [Upgrading to 6.3.4](/topic/28193), [The Control Deck](/topic/28192), [The webmail portal at /portal](/topic/28191) and the [6.3.4 announcement](/topic/28218). --- Canonical: https://www.hmailserver.co.uk/topic/28254/the-webmail-in-depth-search-offline-mail-expiring-messages-send-as-and-recovery # The webmail in depth: search, offline mail, expiring messages, send-as and recovery *Written for 6.3.4. hMailServer 6.3.7 (17 October 2026) signs an authenticated SMTP submission as another domain hosted here only when its account may send as the address its From names - the list the From picker below draws on: its own address, an alias pointing at it, a mailbox that granted it the post right, an address it proved or a masked address of its own; otherwise the mail is still accepted and sent, unsigned as that domain. The webmail's own send already holds From to the account, so mail sent from the page is signed as it was. 6.3.7 also carries the first step of the Control Deck's and the webmail's rebuild planned for 6.4 - the framework decision, the threat model of the web surface and the authorisation matrix with its check - with no product code. See [the 6.3.7 announcement](/topic/28277).* This topic goes deeper into the webmail that hMailServer serves at `/portal` on the REST listener; what the page is and how it is turned on is in [The webmail portal at /portal](/topic/28191). The one thing to know first: most of what follows is the reader's own choice, bounded by a setting the administrator holds. Search with operators, the keyboard map, the rich-text editor, the From picker over aliases and granted mailboxes, plain signatures and the offline inbox listing were in 6.3.3; the three stars, the importance marker and its keys, and the rest of what follows are since 6.3.4. ### What the administrator bounds Each is a setting on `PUT /api/v1/settings` and the Control Deck's Settings view; since 6.3.4 many can also be set per domain or per account. | Setting | What it bounds | Default | |---|---|---| | `recover_deleted_days` | How long deleted mail can be put back | 0, off; at most 365 | | `reader_offline_days_max` | The most days a reader may keep offline | 30; at most 365; 0 offers none | | `reader_masked_addresses_max` | Masked addresses per person | 20; at most 200; 0 takes the page away | | `reader_sender_addresses_max` | Send-as addresses per person, proved or not | 5; at most 20; 0 takes the section away | | `brand_logos_enabled` | Brand logos at all | On; needs `AuthenticationResultsEnabled` and DMARC | | `confidential_base_url` | The address the webmail is reached at from outside, which links are written against; falls back to `quarantine_digest_base_url`, then `import_oauth_redirect_base_url` | Empty | ### Search operators and keyboard shortcuts The search box searches the folder shown; the options panel beside it builds the same syntax from fields and can search every folder the account may read. | Operator | What it matches | |---|---| | `word`, `"quoted phrase"` | The word in the subject or the sender, failing those anywhere in the message; the words together, in that order | | `-term`, `a OR b` | What follows the minus must not match; one of the two must match. `OR` in capitals - `or` is a word | | `from:`, `to:`, `cc:`, `bcc:`, `subject:` | The sender's address or name; a recipient's, in To or Cc; an address or name in Cc, or in Bcc (only in the sender's own copy); the subject | | `has:attachment`, `has:link`, `filename:` | An attachment; an `http` or `https` link; an attachment whose file name contains the text | | `larger:`, `smaller:` | By size, as in `larger:10M` | | `before:YYYY-MM-DD`, `after:YYYY-MM-DD` | By date | | `older_than:`, `newer_than:` | By age: `7d`, `2w`, `3m`, `1y` | | `in:folder` | Messages in that folder; a whole path may be given | | `is:unread`, `is:read`, `is:flagged`, `is:unflagged`, `is:answered`, `is:muted`, `is:pinned` | By the flag or keyword | | `label:name` | An IMAP keyword; `label:$Important` is the sidebar's **Important** view | | `category:` | The inbox tab: `primary`, `social`, `promotions`, `updates` or `forums` | | `in_reply_to:` | Messages whose `In-Reply-To` or `References` names the message-id | Since 6.3.4 a search request opens at most 2,000 message files and stops after about two and a half seconds, saying where to go on from; **Search older messages** carries on. The keyboard map (`?` lists it) acts on the message list and an open message, and nothing is captured while the cursor is in a text field. `j` and `k` move, `Enter` or `o` opens, `u` or `Esc` goes back, `x` ticks a row, `e` archives, `!` files as junk (or, in Junk, as not junk), `#` deletes to Trash, `l` labels, `m` mutes, `c` writes, `r` `a` `f` reply, reply to all and forward, `1` to `9` run a quick step, `/` searches and `Ctrl`+`K` opens the palette. `s` gives the next of three stars; the second and third are the `$Star2` and `$Star3` keywords on top of the IMAP flag, so another client sees a flagged message. ### Offline mail The webmail has always kept the inbox listing and the last opened messages - thirty-one of them according to [OfflineMail.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/OfflineMail.md) and [EvidencePack.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/EvidencePack.md), thirty according to [README.md](https://gitlab.com/hmailserver/hmailserver/-/blob/v6.3.4/hmailserver/docs/README.md) and the wiki - and an outbox of sends that failed for want of a connection. Since 6.3.4 it can also keep the text, formatting and attachments of the last days in the browser's own storage, and **nothing is kept until the reader chooses**. Settings, *Offline in this browser*, offers *Nothing beyond the inbox listing* and then the last 7, 14, 30, 60, 90, 180 or 365 days, up to the server's ceiling. The choice is the account's `offline_days` preference, so it follows the person between browsers; what is kept does not. **How it fills.** The store is IndexedDB, database `hmPortalOffline`. A message goes in when it is opened and the window covers it, and in the background, newest first. An attachment above 8 MB is not kept, and a message's parts stop at 32 MB together. When the window is bigger than the browser allows the site, the card says so and the newest are kept. **Without a connection** a message opens from the store, a folder lists what the store holds, and a search runs over the kept words with the operators the store can answer. `has:link` and `in_reply_to:` let a message through rather than hide it, `OR` is ignored, and a date operator is measured on the browser's clock. **What it is not.** It is not a backup and it is not synchronised back: a flag set offline is not sent when the connection returns. It is not encrypted beyond what the browser does with its own storage, so anyone who can open this browser profile can read what is kept. It does not survive sign-out: the database is deleted. **Clear what is kept** deletes everything now and sets the choice to nothing. ### Important, and sections of the inbox Gmail's importance marker, decided **in the browser**: no model is trained anywhere, and nothing about whom this person reads or writes to is sent to the server to decide it. The page counts in `localStorage` and forgets the whole of it at sign-out. What it decides is one IMAP keyword: `$Important` on a message judged important, `$NotImportant` on one the reader says is not, so Thunderbird, a phone and `hmctl` all see them. Each inbox message is judged once, the first time this session lists it; **two points** make it important. | Reason | Points | |---|---| | You marked mail from this sender important, or not important | +4, or -4 | | You write to this sender | +3 | | You answered this sender | +2 | | You open mail from this sender (two or more opened here) | +1 | | You starred mail from this sender | +1 | | It was sent to you by name (a To of five or fewer) | +1 | | It is in one of the other tabs: Social, Promotions, Updates or Forums | -2 | A message that already carries either keyword is never judged again. The marker is changed from the arrow on the row, with an Undo, or with the keys `+` or `=` and `-`. Sent and Drafts have no marker. In Settings, **Importance marker** offers the page deciding (the default), marking by hand only, or no marker; **Inbox sections** can draw *Important, Starred, Everything else* (off by default). It will be wrong, which is why every mark can be undone in one press. It does not sort, hide or delay anything, and what it has learned stays in this browser; the keywords on the mail travel. ### A message that expires Gmail's *confidential mode*: the message is kept on this server, and what crosses the internet is a short notice with a link. **It is not end-to-end encryption.** A message this server can delete on a timer is a message this server can read, and anyone who administers the server, or a backup of it, can read a kept message while it exists. Nor does it stop a recipient photographing the screen. **Sending.** On the compose form, **Expires** (never, in an hour, a day, a week, a month) and an optional **Passcode**. Over the API, `expires_in` (seconds, 1 to 7,776,000 - ninety days) and `passcode` (4 to 64 characters) on `POST /api/v1/me/messages`. The send is refused when the server has no public address to write the link against (403, `confidential_unavailable`), and for a message the page signed or encrypted. The kept copy is a file under `/Confidential//` with a row in `hm_confidential` (schema 6069); it counts against the mailbox's quota (413, `quota_exceeded`), and an account has at most 1,000 outstanding at once (400, `confidential_ceiling`). Each recipient is sent the notice; the sender's Sent folder gets the full message. **Reading.** The page at `/confidential/` takes no credentials. The reader proves they are a recipient in one of three ways: signed in to this server's webmail as a recipient address; a six-digit code mailed to a recipient address, good for ten minutes and five tries; or the passcode, where ten wrong answers pause it for a quarter of an hour. After thirty wrong answers in all, codes stop opening the message. The page answers the same way whether or not the address typed is a recipient's. It shows the text, names other attachments without offering them, and runs no script. After the expiry or a withdrawal it answers 410. **The sender's side.** **Messages that expire**, on the Sent folder's menu, lists what was sent and whether and by whom it was opened, with a **Withdraw** button. The kept file is deleted when the expiry passes. The copies are not in the backup, but an archive taken by a 6.3.4 build made before the release holds them, in the clear: treat it as the mail itself, or delete it. ### Send-as and masked addresses The webmail's **From** picker offers four kinds of address: | Kind | Given by | |---|---| | The account's own address | Always | | An alias that points at it | The administrator | | Another mailbox on this server | That mailbox's owner, with the post (`p`) right on their INBOX | | An address at another provider | The person, proved with a code mailed to it | With `SmtpAuthenticatedSenderCheck` on, an authenticated SMTP session's `MAIL FROM` is held to the same list: see [who may send as whom](/topic/28239). Since 6.3.7 the DKIM signature follows the list too: an authenticated submission whose From names another domain hosted here is signed as that domain only when the account may send as that address, and is otherwise accepted and sent unsigned as that domain; the webmail's own send already holds From to the account. **Proving an address elsewhere (since 6.3.4).** Under **Settings, Other accounts, Send mail as** the person types the address and presses **Send the code**. The server mails a six-digit code to it from the postmaster of the account's domain, and the person types it back. The code lives in the server's memory for an hour and is never stored; five wrong tries end it. An address of a domain on this server is refused. **What this cannot change:** mail this server sends as `someone@gmail.com` is not signed by Gmail's DKIM key and not sent from an address Gmail's SPF record names, so a receiver that honours the domain's DMARC policy will quarantine or refuse it when that policy says so. The page says so beside an address whose domain publishes `quarantine` or `reject`. For such an address, send from its own provider. **Masked addresses.** Under **Settings, Masked addresses** the person says what the address is for and presses **Make an address**; the server makes one in the account's own domain, such as `bookshop.k3v9qa@example.com`. While it is on, mail to it is delivered to the account, it is in the From picker (`kind: masked`), and a reply to mail that came through it goes out from it. Switched off, mail to it is refused at `RCPT` exactly as mail to an address that never existed is - not handed to the domain's catch-all. An account, alias or list made later with the same address takes it over. Both kinds of address are carried by the backup with the account. ### Recovering deleted mail It is **off by default**: `recover_deleted_days` (0 to 365) turns it on for the whole server, and a domain or an account may keep its copies for fewer days than the level above, never for more. **What is kept** is a message a person deletes: IMAP `EXPUNGE`, `UID EXPUNGE` and `CLOSE` on a folder with messages marked `\Deleted`; POP3 `DELE`; the webmail's delete and its emptying of a folder; and the retention the person chose, or the administrator's retention policy. What is removed for any other reason is not kept: an IMAP `MOVE`, a scheduled draft that has been sent, an account being deleted. **Putting it back.** **Recover deleted items**, on the Trash folder's menu and in the command palette, lists what was deleted within the window. **Put back** returns a message to its folder with its flags - or to a folder of the same path, or otherwise the INBOX - as a new message with a new UID. **Delete for good** removes it now. The copies are files under `/Recoverable//`; they count against no quota and are not in the backup. The retention task, six-hourly, ends the copies older than the window. ### Signatures and formatted messages **A logo in the signature.** **Add image** in the signature's toolbar (**Settings > Name and signature**) takes a PNG, JPEG, GIF or WebP file, judged by its first bytes. The whole formatted signature is at most **64 KB - 64,000 bytes**, its images included, which leaves room for a logo of up to about 45 KB. A file that does not fit is drawn smaller, starting at 600 by 200 pixels. The limit is the account's signature through the webmail: the Control Panel, COM and the administrator's REST route do not check it. **What the recipient receives.** Gmail and Outlook do not display a `data:` image, so the server moves each one into a part of its own with a `Content-ID` and points the HTML at it with a `cid:` URL - `multipart/related` inside the alternative. At most 64 different images in one message are moved. Not every mail client displays WebP; a logo that has to show everywhere is safer as a PNG. **The formatted editor** rebuilds what is written from an allowed subset of HTML and drops everything else while keeping the words: a style, a `colspan`, a `