<![CDATA[How the service is built and what runs out first: pools, connections, large folders]]>Written for 6.3.4. hMailServer 6.3.7 (17 October 2026) changes three things this guide touches. On MySQL and MariaDB a lost database connection is reconnected and the statement run again, so the statement after a database restart, a failover or wait_timeout no longer fails, and an insert that forgives a duplicate key forgives that alone, so an error log that was quiet may now name failures that were hidden. On Windows a stop or a reinitialise no longer waits past a work queue's ten seconds for a thread held by a host or a database that has stopped answering, and setup, the live update's silent install and the uninstaller wait ShutdownDrainSeconds plus 90 seconds for the stop, where they waited plus 30. The ACME HTTP-01 challenge responder, which served one connection at a time, serves up to 64 at once. See the 6.3.7 announcement.

This topic describes a running hMailServer 6.3.4 service: its parts and pools, the connection ceilings in front of them, the scheduled work behind them, what the machine needs, and what a folder of a hundred thousand messages costs the webmail. The one thing to know first: nothing in the server is unbounded. There is no thread per connection and no pool that grows. Every piece of work runs on a pool of a set size, and a dependency that stops answering uses up a pool long before it uses up the processor, the memory or the disk. What it cannot give you is a throughput figure: none has been measured yet.

The parts of the running service

Everything runs in one process: the Windows service hMailServer, or the systemd unit hmailserver on Linux.

Part What it is Notes
Mail listeners SMTP, POP3 and IMAP on IOService + TCPServer Boost.Asio
HttpServer The engine under two listeners: REST (/api/v1, the webmail at /portal, the Control Deck) and web services (MTA-STS policy, autoconfig, ACME http-01) Its own io_context, so a request storm cannot starve SMTP accept. Since 6.2.28
Metrics listener /metrics and the health probes Raw sockets, one worker thread
ManageSieve Sieve script management Raw sockets, a thread of its own
SNMP agent Read-only, UDP 161 Since 6.3.4; off, and answering nobody until SnmpAllowedAddresses names a manager
Background Scheduler, SMTPDeliveryManager, ExternalFetchManager Long-lived tasks on the main server queue
State The database (accounts, domains, message metadata and settings, since 6.3.4 including the [Settings] keys), one file per message, and hMailServer.ini as the bootstrap The database stays small next to the store

The handover that matters for capacity: an SMTP session never delivers anything. The final 250 is sent from a worker on the asynchronous task queue once the message is written, and delivery happens later on a delivery thread, so the accepting side and the delivering side run out separately. Diagnosing stalled mail takes each half in turn; How email works follows one message through both.

The pools, their sizes, and what a full one looks like

Seven work queues are created at startup, and the two HTTP listeners have workers of their own. The first column is the queue label of hmailserver_workqueue_depth{queue} on the metrics endpoint, which counts tasks accepted and not yet started; the last row is not a work queue and has no label.

Queue Threads Sized by What runs on it When it is full
IOCPQueue 15 tcpipthreads Every socket read, write and TLS handshake for SMTP, POP3 and IMAP New connections are accepted but nothing progresses; clients time out mid-command
Asynchronous task queue 15 MaxNumberOfAsynchronousTasks Message finalization, which sends the final 250 Senders sit after CRLF.CRLF with no reply
Name lookup queue 15 the same setting Blocking reverse-DNS lookups for live sessions Reverse names become "Unknown", so a dead reverse zone cannot stall acknowledgements
SMTP delivery queue 10 maxdelivertythreads The delivery of one queued message The queue grows and mail is late; nothing is lost, since the sender already has its 250
External fetch queue 15 MaxNumberOfExternalFetchThreads One run per fetch account Fetch accounts fall behind their schedule
Maintenance queue 5, fixed nothing One-shot scheduled tasks; a backup started over REST A slow sweep delays the other one-shot sweeps
Server queue 4, fixed nothing Scheduler, IOService, SMTPDeliveryManager, ExternalFetchManager Cannot fill in normal operation: each task holds one slot for the life of the server
HttpServer workers 4 per listener, fixed nothing One REST or web-services request each Requests queue behind the four; SMTP is unaffected

Where the four sizes are set. On Windows, the Control Panel's Maintenance → Performance page, Threads tab, holds the COM Settings properties TCPIPThreads, MaxAsynchronousThreads and MaxDeliveryThreads, and the [Settings] key MaxNumberOfExternalFetchThreads. On either platform the first three are in the server settings group: the Control Deck's Settings view, PUT /api/v1/settings with tcpip_threads, max_asynchronous_threads and max_delivery_threads, or hmctl settings set. The fourth goes through /api/v1/settings/ini/{name} (administrator password only), hmctl ini set, or on Linux hmailserver --set-setting as root. Since 6.3.4 the database is the settings store, so do not edit the [Settings] line in hMailServer.ini: a line edited there is ignored in favour of the stored value. Settings: when a save takes effect covers the store, what becomes of the file's line (HM5804) and when each key applies. The tab also shows WorkerThreadPriority, read-only: the server stores that value and never reads it.

When a new size takes effect. Since 6.3.4 a save applies most settings at once, but MaxNumberOfExternalFetchThreads waits for a reinitialise, because the queue's size is set when the fetch manager is built; GET /api/v1/server/settings/pending lists it until then. For the other three the documents leave a gap: the Control Panel reports a save of its COM settings as "applied immediately", while the Architecture page has the pools created when the servers start, and no document says when a changed count reaches a running pool. Until one does, treat all four alike: after saving, reinitialise (the Control Panel's Reinitialize or Apply now, POST /api/v1/server/reinitialize, or hmctl api POST /api/v1/server/reinitialize) or restart the service. A reinitialise stops and starts every service in place, ends the Control Deck's session, and ends a drain.

# The REST listener on this machine: read the server settings group,
# then give delivery 25 threads (or: hmctl settings set --set max_delivery_threads=25)
curl -u Administrator:pw http://127.0.0.1:8443/api/v1/settings
curl -u Administrator:pw -X PUT -H "Content-Type: application/json" \
     -d '{"max_delivery_threads":25}' http://127.0.0.1:8443/api/v1/settings

# The fetch pool is a [Settings] key; it waits for a reinitialise
hmctl ini set MaxNumberOfExternalFetchThreads 20
hmctl api POST /api/v1/server/reinitialize

Two asynchronous threads are kept back. Finalization runs the spam tests and event scripts, which wait on things outside the process, so AsyncQueueReservedThreads (2) keeps that many threads free of it: with the defaults at most 13 finalizations run at once. A task waiting for a slot holds no thread and is counted by hmailserver_workqueue_blocking_tasks_waiting{queue}; a count that stays above zero means something those tasks depend on is slow. On Windows the key is on Server limits & expert settings, card Timeouts and queue bounds; anywhere, hmctl ini set.

Only two queues report a stall: the asynchronous and name lookup queues write the "every thread busy" report that Diagnosing stalled mail reads. The others are left out deliberately, since their tasks are long-lived by design, so a full IOCPQueue, SMTP delivery queue or External fetch queue shows only in hmailserver_workqueue_depth{queue} and in its symptoms.

The database connections every pool shares

The deliveries, the sign-ins and the rest of the server share one pool of database connections, NumberOfConnections in the [Database] section of hMailServer.ini, 5 as shipped. A wait for one longer than DBConnectionAcquireTimeout (60 seconds) fails. In hmailserver_db_connections{state}, available pinned at zero with busy at the pool size is saturation, and every mail path is then waiting out that timeout.

  • Raise it with the delivery threads. The relay profile sets maxdelivertythreads to 25 and the sender profile to 50, and each tells you to raise NumberOfConnections to about the delivery thread count.
  • The built-in database has one connection. On Windows, NumberOfConnections is forced to 1 for MSSQLCE. Before You Install calls the built-in engine adequate for anything up to a few dozen mailboxes, and its single connection the first thing to change if the server feels slow under concurrency. Linux has no built-in database; it uses PostgreSQL or MariaDB. Choosing a database covers the choice.
  • It is a bootstrap key. [Database] stays in the file, never in the database, and its keys wait for a restart. On Windows the Performance page's Database tab edits it, on the server machine only. On Linux:
sudoedit /etc/hmailserver/hMailServer.ini   # [Database] NumberOfConnections=25
sudo systemctl restart hmailserver

Connections and requests: the ceilings in front of the pools

Where Limit As shipped Set on
SMTP sessions MaxSMTPConnections 0, no cap Connections & protocols → Protocols, SMTP tab; max_smtp_connections
IMAP sessions MaxIMAPConnections 0, no cap The IMAP tab; max_imap_connections
POP3 sessions MaxPOP3Connections not given in the documents; GET /api/v1/settings shows it The POP3 tab; max_pop3_connections
Recipients in one message 50,000 50,000 No setting is named; beyond it, 550 Too many recipients
REST listener, with the webmail and the Deck 64 connections, closed at accept before any TLS handshake; 4 workers; 64 KB a request (16 MB, with 300 seconds, on the two attachment routes); 30 seconds a request; 300 seconds a connection fixed no key
Web-services listener 128 connections; 4 workers; 64 KB a request; 15 seconds a request; 120 seconds a connection fixed no key

The webmail shares the REST listener: its users, the Control Deck and every script that calls the API meet the same 64 connections and four workers. Its request rate is in the topic on the REST API. Outbound, since 6.3.4, MaxOutboundConnsPerDestination and MaxOutboundConnsPerTenant (both 0, off) cap sessions per destination and per sending domain; a tenant share larger than the delivery thread count is no share at all (Outbound shaping and IP warm-up).

Scheduled work

The Scheduler polls its recurring tasks once a minute; one-shot tasks go through the Maintenance queue at once. External fetch is not one of them: the ExternalFetchManager runs its own one-minute loop. Roadmap.md recorded in August 2026 that a due task runs on the scheduler's own thread, so one long task delays the others, and that the real period is the interval plus the run time. The tasks, grouped by cadence:

Task Cadence Registered
GreyListCleanerTask every GreylistingRecordExpirationInterval minutes (240 as shipped) when SMTP is on
RemoveExpiredRecords, WorkQueueHealthTask every minute always; the second writes the stall report
TlsRptReporterTask, DmarcRptReporterTask hourly always; each sends only when its from-address is set
LogRetentionTask, MailboxRetentionTask start + 6 hours always
ArchiveRetentionTask, IMAPExpungeRetentionTask start + 12 hours always
MetricsHistoryTask start + every minute always
UpdateCheckTask start + every 15 minutes always; does nothing until UpdateCheckEnabled=1
MessageStoreConsistencyTask, DiskSpaceMonitorTask start + hourly always
BackupScheduleTask a one-minute tick against the wall clock only when a schedule is set
DirectorySyncScheduleTask start + every [LDAP] SyncScheduleMinutes only when that is set
AcmeRenewalTask start + hourly only when AcmeEnabled=1

Sizing the machine

Memory. Before You Install asks for 2 GB as a minimum on Windows, more for a busy server. It is guidance; nothing enforces it, and memory per session has not been measured.

Disk. Size it for how much mail you will keep, not how much you have today. The store is Data\ under the program folder on Windows and /var/lib/hmailserver on Linux, one file per message. For scale, the webmail bench's hundred thousand messages, 5 % of them with an attachment of 20 to 512 KB, are 1,323,987,827 bytes. Below MinimumFreeDiskSpaceMB (100) new mail is refused temporarily, 452 4.3.1 over SMTP and NO [UNAVAILABLE] to an IMAP APPEND, so senders retry; below DiskSpaceWarningThresholdMB (1024) the server logs a warning and keeps accepting.

Antivirus. On Windows, exclude Data\, Database\ and Temp\ from real-time scanning. A scanner holding a message file open is a documented cause of delivery that silently does nothing; the server detects one shape of it and logs error 5760. Before You Install names no exclusions for Linux, where the store is /var/lib/hmailserver with its temp and database subdirectories.

Durability. MessageStoreFsync (0 as shipped) flushes the spool file to disk before the 250, and a failed flush refuses the message. The published-numbers method runs with it on and labels the shipped default "not durable". Set it on Windows on Server limits & expert settings (card Message store durability), or anywhere with hmctl ini set.

A folder of a hundred thousand messages in the webmail

Every webmail path reads the folder's shared message collection, the one IMAP, POP3 and delivery use: one small object per message. Until 6.3.4 opening a folder copied the whole of it, under the lock IMAP sessions and delivery wait on, for every page of 200. Since 6.3.4:

Operation What it does now What still grows with the folder
Open, GET /api/v1/me/folders/{id}/messages One pass under the lock counts the folder and copies only the page of 200; each row's file head is read twice, not three times That pass; and the first open after a start, or after thirty idle minutes in the cache, reads every row of the folder from the database, as an IMAP SELECT does
Scroll Older and Newer by before_uid; from three pages up, Oldest in one request (oldest_before_uid) Every page costs what the first costs, wherever it is in the folder: the one pass that counts the folder
Search At most 2,000 message files and about two and a half seconds a request, then answers where to go on from; across every folder it walks the mailbox newest first and answers next_before_id; with IndexerFullText and message indexing on (off as shipped), a message the index rules out is passed without opening its file Performance.md does not say
Select-all and one action The selection goes to POST /api/v1/me/folders/{id}/bulk: at most 1,000 messages a call for a flag, 500 for a move or a delete, nothing new taken on after about two and a half seconds A changed flag is a row written, a move is a file copied, and a selection narrowed to an inbox tab reads the head of every message

Junk over a whole folder is a hundred thousand lessons. With SpamAssassin on, each message moved into or out of Junk queues one lesson, as the same move over IMAP does. In the Settings Reference the lesson on a move is SpamAssassinLearnOnMove (off as shipped), run as a task on the asynchronous work queue.

A move holds the mail twice, briefly: original and copy, both counted against the quota, for no longer than one slice of the work.

Not changed. The Storage, Subscriptions and Scheduled views still copy each folder they look at; emptying Junk or Trash copies the folder once per 500 messages removed; a quick step over a whole-folder selection still reads at most five thousand messages.

For other callers of GET /api/v1/me/search, it now answers complete: false, more: true with next_before_id whenever it is not the end, and its hits come newest first across every folder, where they came folder by folder.

Over IMAP, one SEARCH or SORT is bounded by IMAPSearchTimeout (60 seconds) and IMAPSearchMaxMegabytes (2048 MB examined). The webmail portal at /portal covers the rest of the client.

What has not been measured

The webmail at a hundred thousand. Performance.md has the method and a results table for the Windows bench (SQL Server Compact, 100,000 messages in the inbox, the full-text index off and on). Every cell reads to be measured on the bench; nothing was estimated. Its tools, build/perf/seed-folder.py and build/perf/webmail-timing.py, are in the repository and also run against a Linux server. Record the machine, database, message store, index state and build with any numbers.

How much the server carries. PublishedNumbers.md defines messages in and out per second, latency percentiles, IMAP sessions per core with every operation's 95th percentile under 200 ms, memory per session and Dovecot's imaptest, measured against Postfix and Dovecot on the same machine under ten fairness rules. Its results say Not measured yet, and until they are filled the first line of Roadmap2.md is a goal, not a claim. The load generator, in the repository since 6.3.4, is build/perf/load/loadgen.py; a run in which the generator was busy for more than 85 per cent of it gives only a floor and is not published. The Windows bench gives hMailServer's own numbers only, not to be compared with the Linux table.

python build/perf/load/loadgen.py --help
python build/perf/load/loadgen.py self-test   # proves the generator, says nothing about any server

Size from the ceilings above, watch the queue depths and the database pool, and treat any throughput figure quoted elsewhere as unmeasured by the project.

What it does not do

  • No document gives a rule for sizing the pools to a machine; the only sized examples are the relay and sender profiles' 25 and 50 delivery threads.
  • The main server queue, the maintenance queue and the HTTP listeners' workers, connection caps and deadlines are fixed; no key changes them.

Where to read more

]]>https://www.hmailserver.co.uk/topic/28238/how-the-service-is-built-and-what-runs-out-first-pools-connections-large-foldersRSS for NodeFri, 06 Nov 2026 09:05:55 GMTSun, 27 Sep 2026 19:20:13 GMT60