Written for 6.3.4. hMailServer 6.3.7 (17 October 2026) regroups the Control Panel's sidebar: the audit trail, journaling and eDiscovery are a Compliance group of their own rather than the last rows of Monitoring & troubleshooting, as the table below has it, and Ctrl+K takes every old title and route to its page. See the 6.3.7 announcement. This guide covers what an operator is asked for when a lawyer, a regulator or a person with a data request arrives: eDiscovery (cases, searches, legal holds and exports), a person's own data (the export and the erasure), and the audit trail that records who did what. eDiscovery, the person's data as one archive, the erasure of log lines, the erasure's reach into the stores beside the mailbox, and the audit trail are new since 6.3.4. The one thing to know first: cases, their files and their holds are not in the backup. A server rebuilt from a backup has no cases, no exports and no holds, so place the holds again before its mail can be deleted. Where it is Where What The Control Deck, Discovery view The whole of it: cases, the search form, holds with Lift the hold, exports with Download, Open and Close REST, /api/v1/discovery/cases The same ten routes the Deck calls, in the OpenAPI document The Control Panel (Windows only), Monitoring & troubleshooting → Discovery (eDiscovery, in the Compliance group, since 6.3.7) Read-only: the cases and their lines of the audit trail, read over the REST listener, which must be on (RestApiPort). It changes nothing COM, hmctl Neither On Linux, then, discovery is worked from the Deck or over REST. Discovery and the erasure do nothing until somebody opens a case or asks for an erasure; the audit trail is on from the first start. Two neighbouring tools are not this one. The archive has had a legal hold of its own since before 6.3.4 (/api/v1/archive/{id}/hold), which keeps one archived copy from the archive's retention sweep (Quotas, retention and the archive). Journaling, a copy of a domain's or an account's mail sent elsewhere as it passes, has its own topic: Journaling and dual delivery. Cases A case has a name (up to 255 characters), a description (up to 1,000), who opened it and when, and a state: open or closed. Its searches, holds and exports belong to it. Closing a case releases every hold of it that is still live, each release on the audit trail; a closed case runs no search, places no hold and makes no export, and what it made stays readable and downloadable. A case is never deleted, and an address's erasure does not reach it. The files a case makes are under the data directory, in Discovery\case-<id>\: search-<id>.json for each search and export-<id>.zip for each export. Nothing removes an export; delete its zip by hand when it is no longer needed. Searching POST /api/v1/discovery/cases/{id}/searches takes: Part What it means custodians (required, at most 100) An address, a domain, or * for every account on the server and the whole archive from, to (optional, YYYY-MM-DD) The day a message is dated by its Date field, read as UTC; else the day the server stored it words (optional, at most 50) Every word must be found, a quoted phrase as one: a case-insensitive substring of what IMAP SEARCH TEXT reads attachment (optional) name, type and min_kb; {} is any attachment at all scope mailboxes, archive or both, the default There is no sender or recipient criterion: a word is looked for in the whole decoded header, and a person is named as a custodian. A query the server cannot take is refused with 400 and the reason; 202 answers with the search's id, and the search is queued. Where it looks. For each custodian account: its folders, its deleted items (what a hold kept, and what RecoverDeletedDays keeps), and the archive. A search copies nothing; it lists each hit, and lists separately what it could not read, with the reason. What it cannot see. 100,000 hits. A search stops there and says truncated. Narrow it and run it again. Text inside a PDF, a Word document or a spreadsheet. Only text/* attachments are read, up to 1 MB of characters a message. An attachment criterion still finds the PDF by name, type and size. More than 80 MB a message. It is listed as not searched when the query asks about its body, and never dropped as a non-match. Zero-access mailboxes and messages encrypted end to end are matched on their headers alone and marked headers_only. The stores beside the mailbox, <data directory>\Confidential and \Protected, and archived copies with no index row (a bounce, a delivery report). The full-text index is a filter, never the answer. It is off by default (IndexerFullText). When it is on, a message the index proves cannot contain a word is ruled out; every other message is read. One at a time. Searches and exports are queued and run by one runner, the server's own. Only one hMailServer service may run against a database at a time (Warm standby and high availability), so a standby started after a failover takes over the queue rather than sharing it. Holds POST /api/v1/discovery/cases/{id}/holds takes one of: Body Holds {"scope":"account","target":"[email protected]"} Every message of the account, and its archived copies {"scope":"domain","target":"example.com"} Every message of every account in the domain, and their archived copies {"scope":"archive","search":12} The archived copies search 12 of this case matched, at most 10,000 at once A hold is live until it is released or its case is closed, and what it keeps is kept by any live hold of any case. Path Under a hold The person's delete (IMAP EXPUNGE, POP3 DELE, the webmail's delete, a folder deleted) and an administrator's delete of a message The message leaves the folder, and a copy is kept among the account's deleted items whatever RecoverDeletedDays says (0 by default). If the copy cannot be made, the delete is refused (HM7333) An IMAP MOVE out of the account The original is kept among the held account's deleted items Every message of the account deleted at once Refused, 409 over REST The account's own filters at delivery: a rule's Delete or Reject, a Sieve discard, reject or redirect, a forward that keeps no copy The copy is kept among the deleted items instead of dropped; if it cannot be kept, the message is delivered Retention, the recover-deleted sweep, deleting a deleted item for good Nothing of the held account is deleted A mailbox restore with mode replace, a full restore with the domains option Refused (the full restore as HM5014) Moving the account to another domain, removing its mailbox key, deleting the account or its domain Refused, over REST, SCIM and COM alike The archive's retention sweep and an address's erasure A held copy keeps its file and its index row What a hold does not reach: the server's and a domain's rules, which decide before a message is split between its recipients; spam and viruses refused or deleted before a message is the account's, and what the quarantine discards; the delivery queue; the confidential and password-protected stores; archived copies with no index row. If ArchiveDir was put inside the data directory, a full restore with the messages option replaces held archived copies too, so keep the archive outside it. The hold is not hidden from the custodian. The account's own GET /api/v1/me/recoverable answers held: true and lists what the hold kept, and the webmail's Recover deleted items page offers no delete for good. It fails closed. When the holds cannot be read, every question is answered held, and the error log says so once a minute (HM7332). A hold is a row in the database. The server reads the holds again as soon as one is placed or released, and no more than five seconds after a change made elsewhere. Exports POST .../searches/{search}/export queues an export of a finished search's hits. The zip holds: Each hit as EML, the message's own bytes as the server stores them, under <custodian>/<folder>/<n>-<message id>.eml, and an mbox per custodian folder beside them, in parts of at most 256 MB. SHA256SUMS, so that sha256sum -c SHA256SUMS checks the unpacked export. manifest.json: the case, the search as it was run, who ran it, who asked for the export, every item with its path, SHA-256 and size, and every item withheld, with the reason. The zip's own SHA-256 is kept with the export, and the case answers it (exports[].sha256). Withheld: a zero-access mailbox's messages, and anything deleted, moved or swept since the search ran. Search again to find where a message is now. Limits. The zip written is the 32-bit format, so no export passes 4 GB: narrow the search and export each part. A download past 500 MB answers 413 with where the file is on the server's disk and its SHA-256. An export that would leave the data directory's volume with less than a gigabyte free fails. A discovery export is never a PST; an examiner who needs one converts the EML files with a third-party converter. Who may Permission Allows Held by view_discovery Reading the cases, their searches and hits, holds and exports The server administrator and the auditor role manage_discovery Opening and closing a case, running a search, placing and releasing a hold, making an export The server administrator; a custom role may be given it Downloading an export needs view_mailbox as well, since it hands over the mail itself, the custodians' deleted items among it. A search that reads the archive needs view_archive as well. A credential confined to named domains names no custodian outside them, never *, and is answered 404 for a case that reaches outside them. A person's data: the export One .zip of everything the server holds for an account. The person downloads it from the webmail's Your data card (GET /api/v1/me/export.zip), with no administrator involved; that download is refused (403) to a request authenticated with an app password. The administrator has the Deck's "Everything (.zip)" on the account row, GET /api/v1/accounts/{address}/export.zip (the administrator password's alone), POST /api/v1/accounts/{address}/export with archive, hmctl privacy export <address> --archive <file> and, on Windows, COM Account.ExportArchive(file). Entry What it is manifest.json The address, the moment, and the counts of what follows mail/<folder path>.mbox Every folder with mail, one mbox each account.xml What the backup carries for the account, less the credentials contacts.vcf The address book as vCard 3.0 calendars/<calendar>/<object> Every calendar object as the client sent it preferences.json, filters.sieve The webmail's preferences, and the active Sieve script logs.txt The lines of the server's own log files that name the address, up to 50 MB A download past 500 MB is refused with 413, and the operator writes the archive to disk instead; the disk form has no ceiling but the format's 4 GB. A person's data: the erasure POST /api/v1/addresses/erase, hmctl privacy erase, the Deck's "Erase traces" and, on Windows, the Control Panel's "Erase traces..." remove what deleting the account deliberately leaves: quarantine entries, message-trace rows, greylisting triplets, distribution-list memberships, aliases on either side, the bounce and complaint records, and the rows naming the address in other people's address books and first-contact memory. With the archive included, the person's archive tree is swept and every index row naming the address goes. The log lines are asked for by name (include_logs on the REST body, --logs every occurrence becomes [erased], and a log past 256 MB is reported instead of rewritten. Since 6.3.4 it also reaches the stores beside the mailbox. A message held for review that the address sent or would receive is discarded. The address is taken out of the recipients of a message that expires, and one it sent is ended. A password-protected message to it is ended. The recipients an account remembers writing to, the out-of-office memory, booking policies and correspondents' S/MIME certificates and OpenPGP keys lose it. Every list of addresses is matched whole. Erasing [email protected] leaves [email protected] and [email protected] as they are. A legal hold outranks it. An erasure never deletes a message from a mailbox, so a hold meets it only in the stores it deletes from, and there the hold wins. The REST answer's held counts what a hold kept; run the erasure again once the case is closed. When the holds themselves cannot be read, the erasure fails (HM6250; 500 over REST) and says nothing about a legal hold. What it leaves, by design, among others: mail in mailboxes, the person's own and everybody else's, and a mailbox's deleted items; a list's archive of its posts; other people's calendars; configuration somebody made that names the address (a forward, a rule, a list's moderator); the audit trail, and the alert events, which nothing prunes yet; the spam trap record; the delivery queue and the webhook queue; the contacts in DMARC and TLS reports; the archive's Inbound folder; and backups, which are outside the running server's reach. PersonalData.md has the full list, with each reason. The audit trail, for a compliance reader The audit trail is on by default. It keeps one row in hm_audit for each administrative change made over COM, REST or the Deck, a mailbox's move included, with a few exceptions such as an API key made or revoked or a quarantined message released or deleted; it records changes, never reads. A password, key, token or other secret is recorded as (changed), never as a value. Each row carries the SHA-256 of the row before it and its own, so GET /api/v1/audit/verify tells an edited row from a removed one: tampering is detectable, not impossible, since anyone who can write the table can recompute the whole chain. audit_retention_days is 0 by default, which keeps everything. Three things on it matter here. Every discovery search is recorded with its whole query; a download of a discovery export is recorded on the trail alone, so with the trail off a download is recorded nowhere; and an erasure is recorded with the address it erased wherever it deletes from recorded tables (aliases, list recipients, the archive index): those rows name the person until audit_retention_days ages them out. Alerts, the audit trail, syslog and the journal has the columns, reading and putting a change back, who may read it, and what it does not record. Regulatory scope, in one paragraph The project's own determination, reviewed 12 August 2026, is that the EU Cyber Resilience Act and the revised Product Liability Directive do not place a manufacturer's obligations on it, because it is supplied outside a commercial activity. The document calls itself a self-assessment and not legal advice, and lists the triggers that would re-open it. Under GDPR and UK GDPR the project is neither controller nor processor: the operator of an installation is. The export, the erasure and the log retention limits exist so that operators can meet their own obligations. What it does not do No PST export, and no text read from attachments but text/*. No silent hold: the custodian can see what a hold kept. No export past 4 GB and no download past 500 MB. No saved queries, review tags, redaction or de-duplication across custodians: a message sent to two custodians is two hits. No search of backups, and no case carried by a backup. Where to read more Wiki: Compliance and eDiscovery. Documents: Discovery.md, PersonalData.md, AlertsAndAuditTrail.md and RegulatoryScope.md. On this forum: The Control Deck, Journaling and dual delivery, and what a backup carries and what it leaves in Backup and restore.

Written for 6.3.4. hMailServer 6.3.8 (2 November 2026), a security release, reads on Windows the CA file MessageStoreObjectCaFile names from the file configured, or refuses its name: a name it cannot hand to OpenSSL as it is named, as a path holding characters outside ASCII may be, is refused with HM7536 in the error log, and then no request to the object store is allowed, so give the file, and the folders above it, an ASCII name before upgrading. See the 6.3.8 announcement. This topic covers the three things 6.3.4 can do to a stored message that 6.3.3 could not: store it compressed with its large attachments held once, keep it in an S3-compatible or Azure Blob store instead of under the data directory, and store it encrypted to a key of the mailbox's own. All three are new in 6.3.4, and all three are off until an administrator turns them on: a server that turns none of them on stores every message exactly as 6.3.3 did. The one thing to know first: the delivery queue is never compressed, encrypted or put in the store. A queued message is plain RFC 822 on the local disk, so the event scripts, the virus scanners, SpamAssassin, the sandbox and the filter hook read what they always read. One door, and what a client sees Every place that reads, sizes, copies, moves or deletes a stored message now goes through one seam, MessageStore. Everything that hands a message out reads the original bytes back through it, so a client sees nothing different: IMAP FETCH BODY[], RFC822.SIZE, BODYSTRUCTURE, SEARCH, SORT and THREAD; POP3 RETR, TOP, LIST and STAT; the webmail and the REST API; the rules, Sieve, the quota (which counts the original sizes), and a DKIM signature checked against the stored copy. What changes is the file itself. Message.Filename over COM still answers the stored file's path. With compression on, a COM client or a script that opens that path reads a container, not RFC 822; with an object store, once the message is committed there is no file there. Read the message through the API instead. Compression and attachments held once Schema 6098. With MessageStoreCompression on, a message delivered to a mailbox, or put there by IMAP APPEND, is stored as a container: the message compressed with deflate, and each large MIME part, an attachment of at least MessageStoreDedupMinKB once decoded, cut out and held once as a blob that every message carrying the same bytes shares. A mail sent to forty people on this server with a 10 MB presentation keeps the presentation once. Where What Control Panel (Windows) Settings > Maintenance > Performance, the Message store tab Control Deck The two settings on the server settings page, and a Message store section on the dashboard REST message_store_compression and message_store_dedup_min_kb in /api/v1/settings; GET /api/v1/store; POST /api/v1/store/measure COM (Windows) Settings.MessageStoreCompression, Settings.MessageStoreDedupMinKB, Settings.MessageStoreStatus Both are rows of hm_settings and take effect for the next message stored. MessageStoreDedupMinKB is 256 unless changed, 0 to 65536, and 0 holds nothing once and compresses only. The switch rewrites nothing. The reader never asks the setting what a file is: it reads either form, per file, by the file's first eight bytes. Turning it off leaves every stored message readable. Only delivery and IMAP APPEND write the new form. A webmail draft, a message saved through COM, the importers and the IMAP mirror write plain. A message larger than 64 MB is stored as it came, because a container is decoded whole in memory. Every container is checked before a byte is served: its structure, each blob against its hash, and the SHA-256 of the whole. A damaged one is refused and named in the error log (HM7350), never served as something else. Two messages share a blob when the part's encoded bytes are identical. The same file attached twice by a client that wraps its base64 differently is two blobs. The command line, run as the user the server runs as (hMailServer.exe takes the same switches on Windows): hmailserver --measure-store [--account <address> | --domain <name>] hmailserver --compress-store [--account <address> | --domain <name>] [--decompress] hmailserver --recount-store Verb What it does Server --measure-store Changes nothing; prints what the store takes now, what it would take, and how much more it would save May be running --compress-store Rewrites every stored file in place, each written beside itself, read back and compared byte for byte, then renamed over its name Stopped --compress-store --decompress Puts the files back as plain RFC 822 Stopped --recount-store Sets every blob's count to what the stored files reference Stopped Exit 0 is a finished walk, 1 a file that could not be read or written, 2 no database or no such account or domain, and 4 a running server. How much it saves has not been measured on a real mailbox. MessageStore.md holds a placeholder where the figures are to go, and the wiki claims none. Run --measure-store on your own store and read the answer. The document also records that its regression fixture had not yet run against a bench when it was written. Object storage: S3 and Azure Blob The messages in the mailboxes, the blobs and the recover-deleted store's kept copies can be kept in an S3-compatible store (AWS S3, MinIO, Wasabi, Backblaze B2's S3 API, Ceph RGW), signed with AWS Signature Version 4, or in Azure Blob Storage, with Shared Key or a SAS. The database stays the index, and an object is named by the path its file had. There is no schema step. Nothing changes until a store is named. Every write is a local one. A message is placed on the local disk exactly as without a store, and a committer puts it into the store afterwards, and only then removes the local file. It runs every minute; what it has been handed is journalled in <data directory>\_ObjectStore\pending.txt and taken up again after a restart. So a delivery never waits for the store and never fails for it, and every read looks at the local disk first. Property REST key Default MessageStoreBackend message_store_backend file; the other value is object MessageStoreObjectService message_store_object_service empty; s3 or azure MessageStoreObjectEndpoint message_store_object_endpoint empty, which is the service's own MessageStoreObjectRegion message_store_object_region us-east-1 MessageStoreObjectBucket message_store_object_bucket empty MessageStoreObjectAddressing message_store_object_addressing auto; also path, virtual MessageStoreObjectPrefix message_store_object_prefix empty MessageStoreObjectAccessKey message_store_object_access_key empty MessageStoreObjectSecret message_store_object_secret empty MessageStoreObjectSasToken message_store_object_sas_token empty MessageStoreObjectCaFile message_store_object_ca_file empty MessageStoreObjectCacheMB message_store_object_cache_mb 32 These are rows of hm_settings, set over /api/v1/settings or on the Control Deck's Settings > Server form. There is no COM member and no Control Panel page for them. The secret and the SAS are protected, never emitted, and left out of a backup, so after a restore onto another server they are entered again. TLS verification is always on, and no setting turns it off. With MessageStoreObjectCaFile set, the certificate is verified against that file: beside the system's bundle on Linux, and alone on Windows. Turning it on: Make the bucket or container. This server makes none. For S3, an access key allowed s3:GetObject, s3:PutObject, s3:DeleteObject, s3:ListBucket and the multipart upload calls, and a lifecycle rule that aborts incomplete multipart uploads after a day. Name it, with message_store_backend still file. Copy the store into it: hmailserver --copy-store-to-object-storage. It runs beside the server, verifies every object, and goes on where it stopped when run again. Switch new mail to it: message_store_backend to object, taken within five seconds. Remove the local copies when you are ready: stop the server and run the copy again with --remove-source. Going back is the same the other way: message_store_backend to file, then hmailserver --copy-store-from-object-storage, then, once it says nothing failed, message_store_object_service to empty. When the store cannot be reached, mail is not lost. Door Answer for a message only the store holds A delivery Placed on the local disk and done; committed when the store answers again IMAP FETCH NO [UNAVAILABLE] POP3 RETR and TOP -ERR [SYS/TEMP] The webmail and REST 503 HM7473 says, once an hour at most, how many files wait on the local disk. HM7471 is the store refusing this server's credentials (401, 403), which is never taken for an absent message. HM7476 names a message that is neither on the disk nor in the store; no placeholder is written for it while a store is named. What is thinner than it could be: one connection per request; one committer, a pass at a time; credentials are the two the settings hold, with no instance role, no workload identity and no Azure AD token; the Import Tool and the data directory synchronizer walk the local disk only; a changed prefix is not a move. The requests were proved against MinIO and Azurite on the project's lab on 25 September 2026. The server's fixture needs that lab, is skipped on a bench without one, and has not run in the Linux suite. Mailbox encryption at rest Schema 6099. A mailbox whose messages are stored encrypted to an X25519 key of the account's own, with AES-256-GCM, as the container's last stage, after compression. The key opens when the person signs in with their password and closes MailboxKeyIdleMinutes after the owner's last use (15 by default, 1 to 1440). Mail that arrives while nobody is signed in is encrypted all the same, because encrypting needs only the key's public half. When a password is lost, the domain's recovery key opens the key again. It is off for every account until an administrator switches it on for that account. What it protects: a disk taken out of the machine, a snapshot of the volume, a backup archive, the data directory copied somewhere, the database dumped, once the account has signed in with its password after the switch. What it does not protect. It does not protect the mail from the running server, which reads it while the key is open, nor from whoever runs the server, who can capture a password at sign-in. A person who needs that wants a zero-access mailbox, and the two are never on one mailbox. Until the first sign-in the key is pending, held under this machine's own secret: the machine's DPAPI key on Windows, and the key file .hmailserver-secret-key inside the data directory on Linux. On Linux a copy of the data directory together with the database therefore opens every pending key. An account that only receives mail and never signs in with its password stays pending. Switching it on, on the Control Deck or over REST: The domain's recovery key first: the domain page's Mailbox recovery key card, or PUT /api/v1/domains/{domain}/mailbox-recovery with {"passphrase": ..., "hint": ...}. The passphrase is at least 12 characters, and this server never stores it. Keep it where this server is not. Then the account: the Encryption at rest card, or PUT /api/v1/accounts/{address}/mailbox-encryption with {"enabled": true}. The switch-on ends every session the account had. A mail client sees its connection drop; the person signs in again with the password. Then what is already there: hmailserver --encrypt-store --account <address>, with the server stopped. Refused, with a sentence: an account whose domain has no recovery key; a zero-access mailbox; an account that signs in against a Windows directory; and an account with a TOTP second factor enrolled. The last runs both ways: a second factor cannot be enrolled on an account that has a mailbox key. Sign-in. IMAP and POP3 sign a keyed mailbox in only with its own password. An app password, an OAuth2 token, a client certificate or a Kerberos ticket is answered NO [UNAVAILABLE] (POP3: -ERR [SYS/PERM]), and the IMAP master user is refused whether or not the owner's key is open. The webmail refuses a passkey, single sign-on or an app password (403, mailbox_key). SMTP, ManageSieve and DAV are not refused, since they do not read the mailbox. Passwords. The person's own change wraps the key by the new password. An administrator's reset does so while the key is open or pending, and is refused while it is closed, with a sentence naming the recovery: POST /api/v1/accounts/{address}/mailbox-encryption/recover with the domain's passphrase and the new password. A lost password and a lost recovery passphrase together are the mail lost. Nobody can open it. DELETE /api/v1/accounts/{address}/mailbox-encryption?confirm=forfeit removes the key, and with it every message still encrypted to it, for ever. It is refused while a legal hold covers the account. Switching it off stores new mail as it comes. What was stored encrypted stays encrypted until hmailserver --decrypt-store, run with the server stopped and the domain's passphrase in the environment variable HMAILSERVER_RECOVERY_PASSPHRASE, never on the command line. A keyed mailbox stays in its domain. A move to another domain is refused (409); a rename inside the domain goes ahead. What stays in the clear: the queue and the copy a delivery works on, which is overwritten with zeros once placed, though a copy-on-write filesystem, a snapshot or an SSD may not honour that; the calendar and the address book; the quarantine, the archive and journal copies, and a forward; each message's row in the database, with its size, flags, folder and dates; logs and the trace. An encrypted mailbox holds no part once. A blob shared between accounts cannot be under one account's key, so every part stays inside the message's own container. The cost is the de-duplication, for these mailboxes only. Backups A backup with messages copies the data directory as always, with the containers, the Blobs directory and encrypted files as they are. It copies file by file while the server runs, so an archive taken while mail is delivered and deleted can hold a container whose part was written after the Blobs directory was copied: take the backup when the server is quiet. With an object store it also reads every message and blob that is only in the store into the archive; a store that cannot be read fails the backup. A full restore rebuilds the blobs' counts from the stored files. hm_storeblobs is not in the archive. An encrypted mailbox's key is carried with its account, and the domain's recovery key with its domain. A pending key on Windows opens only on the machine that wrote it; elsewhere the mailbox is opened with the recovery. On Linux it opens wherever the key file goes, and the archive does not carry that file. A restore onto a server that never turned compression on reads the containers the archive brought. Known limits in 6.3.4 Limit Where it is stated Encryption at rest is on REST and the Control Deck, with no webmail card and no Control Panel page. The person's own view is GET /api/v1/me/mailbox-encryption CHANGELOG, EncryptionAtRest.md No COM for the switch, the recovery or MailboxKeyIdleMinutes EncryptionAtRest.md A search over an encrypted mailbox reads every message, and works only while its key is open. The full-text index holds none of its words EncryptionAtRest.md With compression on, Message.Filename over COM names the stored form; with an object store it names a path that holds no file once committed CHANGELOG, MessageStore.md, ObjectStorage.md An account's key is never rotated; the domain's recovery key can be replaced EncryptionAtRest.md The encryption fixture runs on Windows and is not in the Linux suite EncryptionAtRest.md No zstd: deflate is the one stage MessageStore.md Where to read more The wiki page: Message Storage The documents: MessageStore.md, ObjectStorage.md and EncryptionAtRest.md, each with its error codes and what proves each claim Zero-access mailboxes: the wiki's Encryption for People, and on this forum End-to-end encryption The documentation page: Backup and restore On this forum: The Control Deck, where the cards and the settings form are, and Backup and restore, for what the archive holds and what it leaves out

This topic covers the built-in backup, what its archive holds and leaves out, and restoring a server, one mailbox or one message. Know this first: a restore is destructive and has no transaction. It deletes every domain, every account and every public folder before it puts the archive's back, so whatever an account has that is not in the archive is deleted and not replaced. And the archive does not include your messages unless you tick the box. What one run writes One 7-Zip archive per run, named HMBackup YYYY-MM-DD HHMMSS.7z, in the configured destination: an XML index and, with Back up messages on, the whole data directory beside it. The data directory is C:\Program Files\hMailServer\Data by default on Windows and /var/lib/hmailserver on Linux, and what lives inside it travels with the messages: Sieve scripts and their state (<data directory>\Sieve), and ACME certificates and keys (<data directory>\ACME, unless AcmeCertificateDirectory moved them). Since 6.3.4, messages held in an object store (MessageStoreBackend) are read into the archive too: a store that cannot be read fails the backup, a restore puts the files back on the local disk, and the store's credentials are not carried. What it cannot hold: files the database only points at. Manually placed TLS certificates, private keys and DKIM keys are stored as paths. Restored onto a machine where those paths hold nothing, the server starts, serves no TLS and signs nothing. Keep them inside the data directory, or copy them yourself. What is in the archive Area Carried Domains Aliases, distribution lists and members, signatures, DKIM selectors and key paths; since 6.3.4 the domain's allow and block lists, its spam traps, the domain transforms, the S/MIME authority and the recovery key of a domain encrypted at rest Accounts Address, password hash and its algorithm, forwarding, vacation, signature, quota, admin level, anti-spam thresholds, TOTP secret Per account, every release Rules, fetch accounts with their passwords, application passwords as hashes and, with Back up messages, folders and messages with UID and flags Per account, since 6.3.4 Message keywords (the webmail's labels, second and third stars, follow-up flags, mute and pin), the address book, webmail preferences, scheduled sends and snoozes, file-link records, S/MIME keys, the password history, calendars with their events and tasks, the first-contact memory, safe and blocked senders, the mailbox key of an account encrypted at rest, the role assignment Public folders With the permissions on them. Backup.md states no condition; section 15.2 of the wiki lists them under Back up settings. A restore deletes every public folder first Settings, with Back up settings The hm_settings property set and the [Settings] rows of hm_inisettings; routes, IP ranges, global rules, TCP/IP ports, SSL certificate records, incoming relays, groups, blocked attachments, the DNSBL, SURBL and white lists; since 6.3.4 the remote domain policies and IP warm-up schedules Custom administrator roles With domains or settings; restored first, so the assignments accounts carry find their role Nothing identity-shaped is written. A restore gives every row a new id, which makes the archive portable between database backends; a reference that cannot be resolved drops that one row and names the account in hmailserver_backup.log. Secrets travel opened. Route, fetch-account and SSL private-key passwords are carried in a portable encoding and protected again on the restoring machine (since 6.3.8 one the encoding cannot carry, holding a character above U+00FF or longer than 248 characters, as an external account's OAuth token often is, is left out of the archive and named in the error log and the backup log as HM7280, to be entered again after a restore, where 6.3.7 and earlier wrote it empty); since 6.3.4 each TOTP secret and the [Settings] secrets (SRSSecret, OAuth2HmacSecret and the rest) are carried opened and sealed again under the restoring server's stored-secrets key, so a restore onto new hardware keeps every second factor. An unencrypted archive therefore hands those secrets to whoever can read the destination folder. In 6.3.3 and earlier the per-account rows marked since 6.3.4 are in no archive, and a restore deletes them. Until you upgrade, back up the database itself with its own tool (pg_dump, mysqldump, SQL Server's backup, or a copy of hMailServer.sdf with the service stopped), restore from that when recovering a whole server, and tell users before they find their address book and labels gone. An archive written by 6.3.3 or earlier restores on a newer server as it always did. What is not in the archive, deliberately Not carried Consequence of a restore [Database] and [Directories] in hMailServer.INI; the administrator password and its second factor Per machine. Keep a copy of the file, and never copy one machine's file over another's PasswordPepper (since 6.3.4) Kept in the file only. A restore onto new hardware needs the same pepper in that machine's file before an Argon2id or scrypt password can sign in; PBKDF2, SHA-256 and MD5 hashes, and the administrator password, do not use it The stored-secrets key (since 6.3.4) The archive does not need it. A database-level backup does: restored without the exported key, every second factor and [Settings] secret in it is unopenable. Export it once (hMailServer.exe --export-stored-secrets-key FILE, elevated, or sudo hmailserver --export-stored-secrets-key FILE) and keep it with the database backups, stored apart from them On Linux, the stored-secret key file <DataFolder>/.hmailserver-secret-key The archive does not carry it. Its passwords travel in a portable encoding and do not need it, but a domain's S/MIME authority key, and the mailbox key of an account encrypted at rest that has not yet signed in, are carried as envelopes that open only where this file is: restored without it, the authority must be rotated before it signs. A database-level backup needs it as well: back it up with the database, because a secret moved to a machine without it cannot be opened (HM6412) The delivery queue Queued files are copied with the data directory, but nothing re-queues them. Let the queue drain before a migration Per-domain relay settings, and per-domain message retention days Silently lost, and silently reset to 0 (keep for ever). Re-enter them Greylisting triplets; auto-ban and login-failure state Senders are greylisted afresh; bans lapse. Permanent IP ranges are carried Quarantine, message trace, metric history; since 6.3.4 the audit trail, alert rules and events Operational records. Back the database up with its own tool if you need them Deleted items kept for recovery, eDiscovery cases and holds, messages that expire, password-protected messages The Recoverable, Discovery, Confidential and Protected folders are left out of the copy and left in place by a restore. A restore with domains is refused while an account or a domain is under a legal hold. Archives taken by earlier 6.3.4 builds, before Confidential and Protected were left out, do hold both: a message that has expired or been withdrawn since is still in them, so apply the mail's own retention to those archives, or delete them The full-text index, IMAP annotations, the QRESYNC expunge log Rebuilt, or keyed on ids a restore reassigns. IMAP clients resynchronise Passkeys; calendar and address-book shares; the backup passphrase, the SNMP passwords, the object store's secret, the sandbox token Enrol, share or enter them again A domain named recoverable, discovery, confidential or protected keeps its mail in a folder the backup leaves out. Such names are refused for a new domain since 6.3.4, and an existing one is reported as HM7597; rename it before relying on the backup. The options, and the combinations that fail Maintenance → Backup & restore in the Control Panel (Windows-only), the Control Deck's Backup view, PUT /api/v1/settings/backup and hmctl settings set backup set the destination and what goes in. A fresh install seeds backupoptions as 0: nothing is ticked. On a packaged Linux server the destination has to be inside the unit's ReadWritePaths, or the write fails; add it with a drop-in. Option What it changes Back up domains The domain tree goes into the index. Required for messages to mean anything Back up messages Copies the whole data directory. Only with Back up domains as well: messages without domains stages nothing and then fails Message metadata only (BackupMessagesDBOnly=1) Records which messages exist, not their contents; a restore expects the files still to be in the data directory. It governs the restore as well as the backup, and must be the same for both Back up settings The settings row above and, per the wiki, public folders Compress destination files Puts the message files inside the archive. Off, every run overwrites one DataBackup folder beside the archives: exactly one generation of mail, whatever retention says Verify each backup (BackupVerifyRestore=1, the default) The next section Tick domains and messages together, always, and Back up settings as well if the installation has public folders. The index is written to hMailServerBackup.xml, added to the archive and deleted, so what you restore from is the .7z; the Control Panel's file picker defaults to *.xml, so choose All files. How a run is verified Every refusal happens before anything in the destination is written, or deletes only the archive this run created. A run refuses when a message file lies outside the data directory, when the destination is unreachable, and when the database is down or any read did not run, so an outage cannot produce a hollow archive. Then the archive is read back: The index opens, through the same code a restore uses, and describes this run. If not, the backup fails and the archive is deleted. The per-account rows are all there (since 6.3.4), counted per table. A difference fails the backup. The message store extracts to exactly the files and bytes that were staged. Fewer is a truncated archive, whatever 7-Zip's exit code said. Every message row has a file. Reported, never failed: mail flow does not stop for a backup. The log names the first ten. Steps 3 and 4 run only when domains and messages with their files are included and BackupVerifyRestore is 1. The wiki puts step 2 under the same conditions; Backup.md says only that BackupVerifyRestore=0 turns off steps 3 and 4. Step 3 costs disk: the store exists twice in the temp directory during the check, and a temp volume that cannot hold it skips the step with an explanation rather than failing a good backup. A successful run ends Backup completed successfully.; a failed one has a BACKUP ERROR line. Schedule and retention Since 6.2.25. Nothing is configured on a fresh install. Setting Default Meaning ScheduledBackupTime empty (off) HH:MM, local time; at most one backup per calendar day ScheduledBackupIntervalMinutes 0 (off) Used only when no daily time is set; measured against the newest archive in the destination ScheduledBackupKeepCount 0 (keep all) How many of this server's own archives to keep ScheduledBackupMaxAgeDays 0 (keep all) Delete archives older than this UpdateBackupBeforeApply 1 An unattended live-update apply (in UpdateWindow) backs up to the destination first and is cancelled without a successful backup; an apply you start yourself takes none Since 6.3.4 these are stored in the database: set them on the Backup & restore page, with PUT /api/v1/settings/ini/{name} or hmctl ini set, not in the file (how the settings store works). Retention has four rules: nothing is deleted until the replacement has completed successfully; the newest archive is never deleted; the age rule always leaves the two newest alone; and only files whose names the server generated are considered, the timestamp read from the name. A scheduled run certain to fail (no destination, nothing selected, database unavailable, destination unreachable) is refused before it starts and backs off 5, 10, 20, 40 and then 60 minutes. Test backup directory on the Control Panel's Diagnostics page catches a schedule that has been failing silently. Encrypted archives, since 6.3.4 Off until a passphrase is set; then every archive is 7-Zip AES-256 with its headers encrypted (-mhe=on), so it cannot even be listed without the passphrase. Set it in the Control Panel (Backup & restore → Encryption), the Deck's Backup view, encryption_passphrase in PUT /api/v1/settings/backup, or hmctl backup passphrase set. It is 12 to 256 printable ASCII characters, kept in the database under the machine's secret store (DPAPI on Windows, the key file on Linux), never written into the archive, a command line or a log, and never read back. If it is lost, everything in every archive written with it is lost. Changing it does not touch archives already written, so write the passphrase and its date down together, away from the backups. What it changes about a run. Compression must be on when messages are backed up, or the run is refused. Staging moves into a directory under TempDirectory, so the temp volume needs room for the message store plus a little, and the run is refused with both numbers if it is not there. A stored passphrase this machine cannot open fails the backup; an unencrypted copy is never written in its place. 7zr, the reduced p7zip build, cannot encrypt: a Linux server needs 7-Zip (7zz) or p7zip-full. The archive's file name, its size and the fact that a backup ran are not hidden. Restoring the whole server Phase one decides and changes nothing. Since 6.3.4 the archive is listed first (7za l -slt, the headers only): an entry with an absolute path, a .. segment, or a link with somewhere to point refuses the whole archive as HM7540, because no backup the server writes has such an entry. Then the archive must be one of ours, written by an older or equal version (a newer one is refused, with no override), and hold what you asked to restore; then the store is extracted to a scratch directory. Phase two acts: delete every domain, delete every public folder, empty the data directory and copy the staged store in, load domains and accounts, load settings, reinitialise. A failure here leaves a half-restored server. The listing check works whichever 7-Zip is installed; the Windows installer's is 26.03 in 6.3.4, replacing 19.00. Where. The Control Panel's Restore card and COM (BackupManager.LoadBackup, LoadEncryptedBackup) on Windows; POST /api/v1/backup/restore with file, passphrase and the three parts, which is the only restore a Linux server has; the Deck's Backup view; hmctl backup restore FILE --domains --messages --passphrase-file FILE. The REST route's check is bounded at two minutes; past that the route answers 504 and starts nothing. Install the same version, at the same drive letter and directory layout. Restore or recreate [Database] in hMailServer.INI and confirm you can connect. Choose the .7z, tick the categories, start, and watch hmailserver_backup.log. Copy back what the archive could not hold: certificate, key and DKIM files, and PasswordPepper. Re-enter the per-domain relay settings. Check Server status, then Diagnostics. Send a message in from outside, read it over IMAP, send one out. Rehearse this yearly on a scratch machine against a new, empty database, never production, and keep the list of what you fixed by hand with the backup. Restoring one mailbox, or one message, since 6.3.4 An archive in the backup destination can give back one account's mailbox, every folder and message it held when the backup was taken, or one message of it, and touch nothing else: not the account's settings, rules, contacts or calendars, and no other account. Where How REST GET /api/v1/backup/archives, GET /api/v1/backup/{archive}/mailboxes/{address}, POST .../restore with mode, or message and folder, GET /api/v1/backup/restore-mailbox/{job} Control Deck, and Control Panel (Windows) The Restore a mailbox and Restore one message cards COM (Windows) BackupManager.RestoreMailbox, RestoreMessage, DescribeMailbox, MailboxRestoreStatus hmctl hmctl backup archives, hmctl backup mailbox ARCHIVE ADDRESS, hmctl backup restore-mailbox ARCHIVE ADDRESS --merge\|--replace [--message ID [--folder F]], hmctl backup restore-status JOB Which archives. Only one inside the destination, named as the listing names it and never by an arbitrary path. It must have been taken with domains and messages; an archive taken with BackupMessagesDBOnly holds rows and no files, and the job fails. An encrypted one needs its passphrase. Merge creates each archived folder that is missing and adds each archived message whose file is not in the live mailbox; nothing is removed, and merging twice adds nothing. Replace empties every live folder first, so the mailbox is the archive's and mail that arrived since the backup is gone; what it removes is kept only for RecoverDeletedDays, which is off by default. Replace is refused with 400 when the archive holds no messages for the account. The identity rule. Every message file is named once, with a GUID, and the same name anywhere in the live mailbox is the same message: a message the person moved is not brought back to its old folder, and one they deleted is. A restored message keeps its flags, keywords and dates, gets a fresh UID, and is charged to the account as a delivery is. Refused, and counted. The account must exist on this server and the archive must hold it (404); restoring into a different account is not offered, and neither is a public folder. A second restore while a backup or restore runs is 409. The outcome (folders, added, skipped, failed) is in the job's status and the backup log. Restoring needs manage_server; listing an account's messages out of an archive needs view_mailbox. Where to read more Wiki: Backup and Restore, with COM commands in section 15.9; Installing on Linux for the Linux paths and key file. Repository: Backup.md; SecretsAtRest.md. The public documentation: Backup and restore, Backups on Linux. On this forum: Moving to new hardware, Upgrading to 6.3.4, and Hardening a public mail server for exporting the stored-secrets key.