Skip to content
hMailServer Community Forum
hMailServer 6.3.8 is the current release, signed 2 November 2026. Windows and Linux, and an in-place upgrade from 5.3 or later, but for the original hMailServer's 5.7.Download 6.3.8 Documentation FAQ
  1. Support and discussion for hMailServer
  2. Automation & integration
  3. Monitoring, metrics & logs

Monitoring, metrics & logs

2 Topics 2 Posts

Prometheus and OTLP, /livez and /readyz, JSON logs into Elasticsearch, Loki or Splunk, and reading hMailServer's own logs when you are chasing a delivery.

  • ProgressiverobotP

    Monitoring and health: Prometheus metrics, health probes, Grafana, OTLP and SNMPv3

    Pinned diagnostics administration rest-api linux windows
    1
    0 Votes
    1 Posts
    160 Views
    ProgressiverobotP
    Written for 6.3.4. hMailServer 6.3.6 (6 October 2026) publishes the first container image, registry.gitlab.com/hmailserver/hmailserver:6.3.6, and a monitor that matches "The response carried no HTTP headers." needs the new sentence, which says who ended the connection, after how many bytes and seconds. See the 6.3.6 announcement. hMailServer 6.3.7 (17 October 2026) publishes registry.gitlab.com/hmailserver/hmailserver:6.3.7, tagged 6.3 and latest until 6.3.8; renames the Control Panel page that holds the metrics listener's settings from API & monitoring to Monitoring & alerts, with the alerts' settings, the outbound webhooks and the SNMP agent on it (Ctrl+K takes the old title to it); and refuses GET /api/v1/metrics/history, and the same history over COM, to a credential confined to named domains, such as a domain administrator, the auditor role given a list of domains or an API key issued for named domains. With IMAP and POP3 now taking the client's address from a PROXY protocol header as SMTP does, the High Availability Runbook says that a balancer's /readyz check must not carry that header: the metrics listener reads none and answers a request behind one 404. See the 6.3.7 announcement. hMailServer 6.3.8 (2 November 2026) publishes registry.gitlab.com/hmailserver/hmailserver:6.3.8, also tagged 6.3 and latest. See the 6.3.8 announcement. This topic covers what hMailServer 6.3.4 offers a monitoring system: the metrics listener's Prometheus exposition and health probes, the repository's Grafana dashboard, the OpenTelemetry push, the server's own metric history, the new SNMPv3 agent and, briefly, the daily block-list check of the server's own addresses. The one thing to know first: the metrics listener is off until you give it a port, and once it is on, the health check to configure is /readyz, because it is proved by a real database round trip rather than by a flag. The metrics listener One small HTTP listener serves both the exposition and the probes. Its seven settings are [Settings] keys, set on the Control Panel's Monitoring & troubleshooting → API & monitoring page (Monitoring & alerts since 6.3.7; Windows only), with PUT /api/v1/settings/ini/{name}, with hmctl ini set, or, on a Linux server whose REST listener is not yet on, with hmailserver --set-setting NAME=VALUE; the topic on settings explains where they are stored and when a save applies. All seven are taken when the listener starts, the three credential settings included, so a saved change waits for a reinitialise (POST /api/v1/server/reinitialize, or Apply now in the Control Panel or the Deck) or a service restart. A token that has been saved and not applied leaves /metrics answering as it did before. Setting Default Effect When it is wrong MetricsServerPort 0 0 means the listener is never created No port, no probes, no metrics, and no complaint anywhere MetricsServerBindAddress 127.0.0.1 An IP literal only, parsed with inet_pton: 0.0.0.0, 127.0.0.1, :: or a specific IPv6 address A host name or localhost is rejected: the log says MetricsServer: Invalid bind address, the listener does not start and the probes go with it MetricsServerAuthToken empty Sent by the scraper as Authorization: Bearer <token> Absent on a non-loopback bind: /metrics answers 503, not 401, and the body names the settings that would open it. The probes keep answering MetricsServerAuthUsername and MetricsServerAuthPassword empty HTTP Basic; with both schemes configured, either satisfies the check As above MetricsServerCertificateFile and MetricsServerPrivateKeyFile empty Both set makes the whole port HTTPS, probes included Set but unusable: the listener still serves the probes in the clear and /metrics answers 503 :: serves both address families; a specific IPv6 literal serves IPv6 only. "Loopback" means the whole of 127.0.0.0/8 and ::1. The probes are answered before any branch that can refuse. A Kubernetes httpGet probe, an ELB or ALB health check and a Docker HEALTHCHECK have nowhere to keep a bearer token. Refused credentials on /metrics are counted in hmailserver_metrics_unauthorized_requests_total rather than logged. Nothing on the request path touches the database or the disk. A refresher thread probes the database every 5 s (every 30 s after a failure), refreshes the delivery-queue aggregates at most every 10 s and only when a scrape has asked for them, and re-reads the certificate files every 5 minutes. One exception since 6.3.4: the two message-store gauges are cached for an hour and recomputed by whichever caller finds them stale, so about once an hour a scrape can run select sum(messagesize), count(*) from hm_messages on the listener thread itself: on a large store, a slow answer to that scrape and to any probe queued behind it. It is a single accept loop serving one connection at a time. A request's header read is bounded at 5 s and a response write at 15 s. For a load balancer's check the High Availability Runbook gives an interval of 5 s and a timeout of 2 s; never point a sub-second health check at it. The three health probes No credential in any configuration; HTTP/1.0 responses that close the connection. Path 200 when Body Use it for /livez Always, if the listener answered; never 503 alive Liveness restarts. It checks nothing on purpose: a database outage must not get a healthy process killed /readyz State is Running and the database has proved within the last 20 s that it can answer; 503 otherwise ready, or not ready: <reason> Load-balancer and VIP routing /healthz The same conditions {"status":"ok\|unavailable","state":"...","database":"up\|down","uptime_seconds":N} Dashboards and human debugging Readiness is four conditions in order, each with its own reason: not ready: server not in running state; not ready: database not connected; not ready: database did not answer the last readiness probe; and not ready: database readiness probe has not completed recently. The last exists because a probe against a database that has stopped answering can hang for as long as DBConnectionAcquireTimeout allows, 60 s by default. Since 6.3.4 a server in maintenance mode answers 503 not ready: draining: POST /api/v1/server/drain enters it, GET reports {"draining","since","sessions","deliveries","drained"}, and DELETE leaves it. Start-up is a refused connection, not a 503: the listener is brought up only after the state has gone to Running. Shutdown is a 503 first, then a refusal: the state goes to Stopping before anything is torn down, so /readyz reports 503 while ShutdownDrainSeconds runs. The High Availability Runbook builds its active/passive design on this: both nodes have the listener on, with MetricsServerBindAddress=0.0.0.0 and a MetricsServerAuthToken, and the VIP is gated strictly on /readyz. The passive node's service is stopped, so its /readyz is refused and it is never routed to. Since the probes need no credential, a missing token shows as missing dashboards, not as a failed failover. Every metric family Thirty-eight metric families in 6.3.4 (thirty in 6.3.3). Counters are monotonic since server start; a restart resets them, which is what hmailserver_start_time_seconds is for. Mail flow Metric Type Meaning, and the trap in it hmailserver_processed_messages_total counter One per delivery pass started, not per message received: a message deferred three times and then delivered counts four hmailserver_messages_delivered_total counter Delivery passes that completed with no failed recipient hmailserver_messages_deferred_total counter Passes rescheduled: a temporary failure, greylisting, or a message the virus scanner could not examine (AVFailAction=1) hmailserver_messages_bounced_total counter Delivery-failure notifications queued to the sender; one can cover several recipients hmailserver_delivery_queue_messages gauge Rows in hm_messages in the delivering state hmailserver_delivery_queue_oldest_message_age_seconds gauge Age of the oldest queued message; 0 when the queue is empty. Alert on this, not on depth hmailserver_spam_messages_total, hmailserver_viruses_removed_total counter Messages classified as spam; viruses removed during delivery preprocessing hmailserver_store_bytes, hmailserver_store_messages gauge Since 6.3.4. The size and the number of the messages in hm_messages, recomputed at most once an hour hmailserver_domain_messages_received_total{domain}, hmailserver_domain_messages_sent_total{domain} counter Only present when MetricsPerDomainEnabled=1; only hosted domains are ever labelled Added in 6.3.4. hmailserver_country_blocked_total and hmailserver_country_blocked_by_country_total{country} count connections and sign-ins refused by a country policy, and hmailserver_spam_trap_hits_total counts hits on a declared spam trap. Three gauges cover the feedback loops' seven-day window: hmailserver_domain_complaints{domain}, hmailserver_domain_complaint_window_messages{domain} and hmailserver_domain_complaint_rate{domain}. They are present only when MetricsPerDomainEnabled is set and complaint processing is on. The rate gauge is a ratio, not parts per million: Complaints.md alerts on hmailserver_domain_complaint_rate > 0.001. Parts per million is the unit of the complaints.rate.high alert rule's threshold, so an expression written in parts per million against the gauge would never fire. Sessions, authentication and TLS Metric Type Meaning hmailserver_sessions{protocol} gauge Concurrent sessions; protocol is smtp, imap or pop3 hmailserver_auth_success_total, hmailserver_auth_failures_total counter A rising failure rate against a flat success rate is a password-guessing run hmailserver_tls_handshakes_total, hmailserver_tls_handshake_failures_total counter The mail protocols only; a failed handshake on the metrics port is never counted here hmailserver_tls_certificate_expiry_seconds{certificate} gauge notAfter as a Unix timestamp per configured certificate, re-read every 5 minutes; at most 128 series. Absent entirely when no certificate is configured. With no certificate row, the ACME certificate is labelled ACME (automatic) hmailserver_metrics_unauthorized_requests_total counter Always 0 when no credential is configured The server and the database Metric Type Meaning hmailserver_start_time_seconds gauge Uptime is time() - hmailserver_start_time_seconds hmailserver_build_info{version,architecture,database_schema_version} gauge Always 1; the information is in the labels hmailserver_state{state} gauge One series per state (unknown, stopped, starting, running, stopping and, since 6.3.4, draining), exactly one of which is 1 hmailserver_workqueue_depth{queue}, hmailserver_workqueue_blocking_tasks_waiting{queue} gauge Seven fixed queue values: Server queue, Maintenance queue, Asynchronous task queue, Name lookup queue, IOCPQueue, SMTP delivery queue, External fetch queue. Asynchronous task queue sends the final 250, so depth there is mail waiting to be acknowledged hmailserver_messagestore_missing_files gauge Always 0 unless MessageStoreConsistencyCheck=1 hmailserver_command_processing_seconds, hmailserver_db_query_seconds histogram Bucket bounds from 0.0001 s to 1.0 s, and from 0.001 s to 10.0 s; both emit the full _bucket, _sum and _count set hmailserver_database_connected gauge 1 when the database answered a real round trip recently hmailserver_database_probe_success_timestamp_seconds, hmailserver_database_probe_age_seconds gauge When it last answered, and how long ago on the server's own clock; /readyz reports 503 once the age passes 20 s hmailserver_db_connections{state} gauge busy and available; available pinned at zero is saturation hmailserver_db_slow_queries_total counter Statements slower than SlowQueryLogMilliseconds; 0 while that setting is 0 Scraping it scrape_configs: - job_name: hmailserver scrape_interval: 30s scrape_timeout: 10s metrics_path: /metrics authorization: credentials: "<the value of MetricsServerAuthToken>" static_configs: - targets: ["mail.example.com:9100"] Use scheme: https when the certificate pair is set, and keep scrape_timeout well under scrape_interval. The wiki gives nine alerting rules against these metrics, among them hmailserver_database_connected == 0 for 2m, hmailserver_delivery_queue_oldest_message_age_seconds > 3600 for 15m and hmailserver_tls_certificate_expiry_seconds - time() < 14 * 86400. The Grafana dashboard The repository ships hmailserver/docs/grafana-dashboard.json. Import it as it is and pick your Prometheus data source; the instance variable, from label_values(hmailserver_build_info, instance), selects the server. Its uid is hmailserver-overview; it refreshes every 1m and opens on now-24h. Its panels array has 19 entries: four row headers, five stat panels and ten time series. Every rate is over [5m] and every query is filtered by instance=~"$instance". Health: server state (hmailserver_state == 1), database, uptime, missing message files, and TLS certificate expiry, amber below 21 days and red below 7. Mail flow: delivery outcomes (delivered, deferred and bounced, stacked), queue depth with the oldest message's age on a right-hand axis, messages processed with spam and viruses removed, and active sessions by protocol. Authentication and TLS: authentication outcomes and TLS handshakes, as rates. Latency and database: command processing and database query latency (p50, p95 and p99 by histogram_quantile over the two histograms' _bucket series), the connection pool by state, and slow queries. What the dashboard does not draw: the two database probe gauges, the two work-queue gauges, hmailserver_metrics_unauthorized_requests_total, the per-domain counters, the two store gauges and the six families added in 6.3.4. hmailserver_build_info is read by the instance variable and by no panel. The metric history inside the server The server keeps a short history of its own, so the Control Panel's dashboard can show 24 hours, 7 days and 30 days without Prometheus. MetricsHistoryDays (default 7; 0 switches it off) is the days kept in hm_metricsamples: one row per metric per minute, plus a sample at service start. Fourteen metrics are stored in 6.3.3, named as the exposition's without the hmailserver_ prefix and with the label folded in, such as sessions_smtp and processed_messages_total. 6.3.4 adds store_bytes and store_messages, whose history begins the day the server first samples them, and country_blocked_total and spam_trap_hits_total. Read it with GET /api/v1/metrics/history?metric=…&range=24h|7d|30d, or on Windows over COM with Utilities.GetMetricHistory. It has no labels and no histograms and is not a replacement for Prometheus, but it survives a restart. Pushing to an OpenTelemetry collector Three OTLP/HTTP signals, each a no-op until its own endpoint is set. The pushed metric names are the Prometheus names verbatim. Setting Default Effect OtelMetricsEndpoint empty OTLP metrics; the default path is /v1/metrics OtelMetricsInterval 60 Seconds between exports, clamped to 5–3600 OtelEndpoint empty OTLP traces only: one span per delivery attempt, joined to the trace named by the message's own traceparent; default path /v1/traces OtelLogsEndpoint empty OTLP logs: the same lines, categories and mask the log files get; default path /v1/logs What is pushed: the twelve counters that predate 6.3.4 and, since 6.3.4, hmailserver_spam_trap_hits_total, sessions as one gauge with a protocol attribute, hmailserver_messagestore_missing_files, and the per-domain counters when they are enabled. The histograms, the database and queue gauges and certificate expiry are scrape-only. On Linux, in every release from 6.3.0 to 6.3.3, a collector that accepted the connection and never answered held the exporter, and the stop that waits for it, for good; 6.3.4 fixes that. The SNMPv3 agent (since 6.3.4) For a monitoring system that speaks SNMP and not Prometheus or OpenTelemetry, 6.3.4 adds a read-only agent answering the mail MIB (RFC 2789, MTA-MIB) and the standard SNMP groups from the same counters /metrics serves; an object /metrics has no counter for is answered noSuchObject, not made up. It speaks SNMPv3 with the User-based Security Model for one user at authPriv only (HMAC-SHA-256 and AES-128), and SNMPv2c as an option that stays off even when the agent is on. SNMPv1 is not answered, a Set is answered noAccess, and nothing is sent unasked: no traps, no informs. It is off as shipped, and an empty allow list answers nobody. Setting (hm_settings) REST key Default Meaning SnmpEnabled snmp_enabled off The whole agent; off, no socket is opened SnmpPort snmp_port 161 The UDP port SnmpBindAddress snmp_bind_address empty, every address 127.0.0.1 for a collector on the same machine; otherwise an address on a management network SnmpAllowedAddresses snmp_allowed_addresses empty Addresses and CIDR ranges, at most 64. Name the managers one by one: UDP does not prove a source address SnmpV3User snmp_v3_user empty The one SNMPv3 user SnmpV3AuthPassword, SnmpV3PrivPassword snmp_v3_auth_password, snmp_v3_priv_password (write-only) empty 8 to 64 ASCII characters each SnmpV2cEnabled, SnmpV2cCommunity snmp_v2c_enabled, snmp_v2c_community (write-only) off, empty The community crosses the network in the clear; an answer is never more than eight times its request SnmpContact, SnmpLocation snmp_contact, snmp_location empty sysContact and sysLocation They are on /api/v1/settings (the server group), on the Control Deck's Settings page under Server and in hmctl settings. There is no COM property for any of them, and in 6.3.4 no Control Panel page, so Windows and Linux are configured the same way; since 6.3.7 they are the SNMP tab of the Control Panel's Monitoring & alerts page on Windows, which 6.3.5 and 6.3.6 had on Logging. They can also be given as HM_PROPERTY_SnmpEnabled and so on in the environment, which reaches the database only when hmailserver --apply-environment is run (hMailServer.exe --apply-environment on Windows): the service does not read the environment itself. Each setting is read when the agent starts, at the next service start or Reinitialize. The two passwords and the community are stored protected, never read back, and not carried by a backup: enter them again after a restore. Snmp.md's example keeps every password off the command line. On the server, for a manager at 192.0.2.10: $ umask 077 $ cat > snmp.json {"snmp_allowed_addresses": "192.0.2.10", "snmp_v3_user": "monitor", "snmp_v3_auth_password": "...", "snmp_v3_priv_password": "...", "snmp_contact": "[email protected]", "snmp_location": "Rack 4, London", "snmp_enabled": true} ^D $ hmctl settings set --from-json snmp.json $ rm snmp.json $ hmctl api POST /api/v1/server/reinitialize On the manager, with Net-SNMP 5.9 or later, defVersion 3, the user (defSecurityName), both passphrases, defSecurityLevel authPriv, defAuthType SHA-256 and defPrivType AES go in ~/.snmp/snmp.conf, readable by you alone (chmod 600); snmpwalk mail.example.com 1.3.6.1.2.1.28 then walks the mail MIB. What it serves. In the MTA-MIB mtaGroupTable has one row per protocol the server runs: 1 SMTP, 2 POP3, 3 IMAP. What has a counter is answered: mtaStoredMessages is the delivery queue's depth, mtaTransmittedMessages is hmailserver_messages_delivered_total, mtaGroupOldestMessageStored is the oldest queued message's age, and mtaGroupInboundAssociations is hmailserver_sessions for the protocol. mtaReceivedMessages is noSuchObject, because no counter of messages received exists, and so are the volume, recipient and conversion columns. Sixteen scalars at 1.3.6.1.4.1.32473.1.2.<n>.0 carry /metrics series directly, among them hmailserver_state as a number at 14 (3 is running, 5 draining) and the two delivery-queue gauges at 15 and 16. The enterprise number is a placeholder. 1.3.6.1.4.1.32473 is the number RFC 5612 reserves for documentation and examples; it is not registered to this project with IANA. When a number is registered, sysObjectID, the scalars and the engine ID's prefix move to it in one release, and a manager configured for these OIDs will need them changed. There is no MIB module file yet, for the same reason. When it does not start. Error 7420: the port could not be bound, because another program holds it (the operating system's own SNMP service, on Windows or Linux, listens on 161 when it is installed) or, on Linux outside the packaged unit, the process lacks CAP_NET_BIND_SERVICE; the server runs on without the agent. 7421: the engine's seed or boot count in hm_settings could not be read or written. 7422: an exception left the agent's thread; a Reinitialize starts it again. On Linux the packaged hmailserver.service has CAP_NET_BIND_SERVICE for port 25, which is what lets it bind 161. No container image of 6.3.4 is published; a container built from the repository's Dockerfile publishes the port as UDP (-p 161:161/udp) and needs a host name of its own, since two started with one fixed hostname are one SNMP engine. The daily block-list check, briefly Since 6.3.4 the server looks the addresses it sends from up on DNS block lists once a day and raises the reputation.listed alert when one is listed. The watch is on in a new installation and off after an upgrade, and is read at GET /api/v1/reputation; the topic on DKIM rotation, reports and the reputation watch has the lists, the addresses checked and what a failed lookup means. Where to read more Wiki: Monitoring and Health and High Availability Runbook Documents at v6.3.4: Snmp.md, HighAvailabilityRunbook.md, ReputationWatch.md, Complaints.md and the dashboard itself, grafana-dashboard.json The public documentation: 16.3 Prometheus metrics and health probes On this forum: Alerts, the audit trail, syslog and the journal, and per-domain reports, Reading the logs, and Warm standby and high availability for the pair the probes gate. See also The Control Deck and hMailServer on Linux: what you get and what you do not
  • ProgressiverobotP

    Alerts, the audit trail, syslog and the journal, and per-domain reports

    logs administration security rest-api systemd
    1
    0 Votes
    1 Posts
    242 Views
    ProgressiverobotP
    Written for 6.3.4. hMailServer 6.3.7 (17 October 2026) moves the Control Panel's Audit trail page, with journaling and eDiscovery, into a new Compliance group, and the alerts' settings, the outbound webhooks and the SNMP agent onto its Monitoring & alerts page (API & monitoring until 6.3.6; 6.3.5 and 6.3.6 had them on Logging); Ctrl+K takes an old title or route to its page. It also fixes the SMTP log, whose lines the syslog sink copies, which wrote some credentials in the clear with SMTP logging on, in every version from 6.0.0 to 6.3.6 (work item #95). See the 6.3.7 announcement. This topic covers four things 6.3.4 adds for being told what the server is doing and who changed it: the alerts, the audit trail, the syslog sink (or the systemd journal on Linux) and the per-domain reports, with a paragraph on following one connection live. The one thing to know first: after the upgrade to 6.3.4 the audit trail and the alerts are on, with seven of the eleven conditions enabled, but nothing is mailed until AlertRecipient and AlertSenderAddress are set. What 6.3.4 switches on without a switch From the CHANGELOG's list of what changes on upgrade: the audit trail is on (schema 6043), and keeps every row until a retention is set. Compromised-account detection starts in watch mode (schema 6071): it raises account.compromised but holds nothing until compromise_detection_mode is set to enforce. The reputation watch, which raises reputation.listed, is on in a new installation and off after an upgrade. Alerts A condition is a row in hm_alertrules (on or off, threshold, mail or webhook or both, cool-down, at once or in the digest); an event is a row in hm_alertevents. Delivery is AlertTask, once a minute, and nothing else sends, so an alert about the delivery queue is never itself submitted into the stuck queue. Condition On as shipped Raised by Threshold backup.failed yes A backup that failed, and a scheduled backup that could not run. Sent at once — certificate.expiring yes The notAfter of every configured certificate, read once a minute 7 days disk.low no The free-space warning band, and again below the floor; cleared on recovery — queue.stalled no A work queue whose every thread has been in the same task past AsyncQueueStallThreshold — autoban.storm no Auto-bans in the last hour 25 minidump.written no A crash dump newer than the last one recorded, in the log directory — complaints.rate.high yes A hosted domain's complaint rate over the feedback loops' seven-day window; inert while complaint processing is off, and a domain that sent fewer than 100 messages is not judged 1,000 per million dkim.rotation.stalled yes A rotation whose next key's record has not been seen for the threshold, or a running rotation whose last move failed 7 days account.compromised yes Compromised-account detection at submission, one event per account per hour; in watch mode the detail says what enforcing would have done — account.password-reset yes A password reset completed through an account's confirmed recovery address; no cool-down and no digest — reputation.listed yes The reputation watch's stored rows; sent at once, and a check that was refused or timed out does not clear it — The four that are off are noisy on a healthy server. The settings. All nine are rows of hm_settings, none an hMailServer.ini key. They are in the server group of GET/PUT /api/v1/settings, on the Deck's Settings page, and on Windows over COM through Settings. The CHANGELOG and AlertsAndAuditTrail.md spell some in their hm_settings form (AlertRecipient, AlertSenderAddress and so on); the REST keys are these. REST key Default Meaning alerts_enabled on The master switch alert_recipient empty Where alerts and the digest go; a rule can name an address of its own instead alert_sender_address empty What they are sent from; a rule with no sender sends nothing alert_digest_enabled on Whether rules marked for the digest wait for it alert_digest_hour 7 The hour, UTC, the digest goes out; a digest with nothing in it is not sent alert_max_per_hour 20 Past this many single alerts in an hour, everything goes into the digest alert_webhook_max_attempts 5 Webhook attempts before an event is dead-lettered (1 to 20) audit_trail_enabled on Whether administrative changes are recorded audit_retention_days 0 Days of trail to keep; 0 keeps everything Nothing may flap. A state condition already raised is not raised again; an occurrence inside the rule's cool-down is recorded but held for the digest; and the digest is one message listing what fired. Webhooks. A rule whose actions is 2 (webhook) or 3 (mail and webhook) POSTs one JSON object per event (condition, state, severity, time, time_utc, summary, detail, server) with three headers: X-hMailServer-Timestamp, X-hMailServer-Signature: sha256=<hex> and X-hMailServer-Event. It is signed under the rule's shared secret, as every webhook the server sends is; the topic on integrating applications shows how to verify it. A rule that calls a webhook is refused without an address and without a secret; webhook_secret is write-only and the listing says only webhook_secret_set. Only https, or plain http to a loopback address, is used. An attempt that does not get a 2xx is retried on the next pass, then after 1, 5 and 15 minutes, which is the default five attempts, and 60 minutes apart after that if the maximum is raised. After the last attempt the event is dead-lettered: kept, no longer retried, and reported once in the error log as HM6543. Route Does Permission GET /api/v1/alerts/rules Every condition and its rule view_configuration PUT /api/v1/alerts/rules/{condition} Change one, or create one for a condition nothing ships; a field left out keeps its value manage_settings GET /api/v1/alerts/events What has fired, newest first, with the webhook's state view_configuration POST /api/v1/alerts/test Raise a condition through the path a real one takes: {"condition":"disk.low","summary":"...","occurrence":false} manage_settings POST /api/v1/alerts/run Run one pass now manage_server All five are refused to every API key. The administrator password reaches them, and so does an account whose role holds the permission and reaches every domain. Neither the Control Panel nor the Deck has a page for the rules, and hmctl and the PowerShell module have no alerts or audit verbs; hmctl api reaches the routes. The audit trail One row per administrative change made over COM, REST or the Deck, in hm_audit: when (audittime, UTC); who (auditactor: Administrator over COM, administrator or key:<id> over REST, and an account signed in with its own password is its address over COM and account:<address> over REST and the Deck); the interface and, except over COM, the peer address; the object's type and name; the action (created, updated or deleted, also unlocked, cleared, moved and move failed, the last for a mailbox move that failed and could not be put back); the detail (the columns the change carried, or for a setting Name: old -> new, ending [role: <name>] under a role); and the two hashes of the chain. Both kinds of setting are recorded: the hm_settings values and the [Settings] keys (in hm_inisettings since schema 6042), with the value they had. A key set for the first time reads SmtpTarpitCount: (default) -> 5; returned to its default it reads SmtpTarpitCount: 5 -> (default), with the action deleted. A write of the value a setting already has records nothing. What is never recorded as a value. A password, key, token, secret, hash, salt or passphrase is recorded as (changed), decided by the name of the column or setting, matched as a substring, so MetricsServerAuthToken and PasswordPepper are among them. Fifteen keys whose names only look like secrets, the seven PasswordPolicy* keys among them, are recorded with their values. The server's own writes (delivery, retention, greylisting, the index) are never recorded. Verifying the chain. Each row carries the SHA-256 of the row before it and of itself. GET /api/v1/audit/verify answers {"intact","rows_checked","first_broken_id","reason"}, and the reason tells a row that was edited from one that was removed. This makes tampering detectable, not impossible: whoever can write the table can recompute the whole chain. Rows hashed the older way, which only a build from before the 6.3.4 release can have written, did not cover a character above U+00FF; they still verify, with that weakness. Reading it. GET /api/v1/audit?actor=&object_type=&action=&since=&until=&limit=&offset= answers newest first; actor is a substring, so key: finds every change made with an API key, and limit is 1 to 1,000, default 100. It needs the administrator password, or an account whose role holds view_audit (the auditor role) and is not confined to named domains. Every API key is refused, with 401 rather than 403, so that it learns nothing from asking. Putting a change back. POST /api/v1/audit/{id}/revert writes a setting's old value back through the same setter, so the put-back is itself a row in the trail. It needs manage_settings and is refused with 409 for a row that is not a setting's change, a secret's row, a value the trail cut at 120 characters, and a setting this version no longer has. Where to see it. The Deck's Audit trail view has the filters, paging, Verify the chain and Put back. On Windows the Control Panel has the same page, over COM, at Monitoring & troubleshooting → Audit trail (Compliance → Audit trail since 6.3.7; Ctrl+K takes the old route to it). Switching recording off is itself recorded. The trail, the rules and the events are not in the built-in backup. What it holds for compliance. A download of a discovery export is recorded on the trail alone, so with the trail off it is recorded nowhere. An erasure is recorded with the address it erased wherever it deletes from recorded tables (aliases, distribution-list recipients and, with the archive included, the archive index), and audit_retention_days is how those rows age out. The compliance topic covers eDiscovery and the erasure. What it does not record. Reads; what a restore replaces (the request is one row, restore started); creating or revoking a REST API key; releasing or deleting a quarantined message; creating, renaming or deleting a public folder; the [Security], [Directories] and [Database] sections of hMailServer.ini; and a change nobody made through an interface. A change to the twelve syslog settings over COM, from the Control Panel's Logging page for one, is recorded in 6.3.4: the settings property setters carry an audit scope of their own. The wiki's syslog section, 16.2a, still describes the earlier state. Syslog and the journal (since 6.3.4) Every entry the server logs can also be sent as RFC 5424 to a syslog collector over UDP, TCP or TLS. It is a copy: the files under LogFolder are written as before. On Windows the settings are the syslog (RFC 5424) card on the Control Panel's Logging page; on Linux they are keys of PUT /api/v1/settings/logging and the Control Deck, and hmctl settings get logging prints them. They take effect when the sink starts, at the next service start or Reinitialize, not when they are saved. hm_settings name REST key Default Meaning SyslogEnabled syslog_enabled off The whole feature SyslogHost syslog_host empty Empty is not "off": on Linux under systemd it selects the journal; on Windows nothing is sent, which the server says once in the application log SyslogPort syslog_port 514 514 for UDP and plain TCP; 6514 is the registered port for TLS SyslogTransport syslog_transport 0 (UDP) 0 UDP, 1 TCP, 2 TCP over TLS; the REST key takes udp, tcp or tls SyslogFacility syslog_facility 2 (mail) 0 to 23; 16 to 23 are local0 to local7 SyslogMinimumSeverity syslog_minimum_severity 6 (informational) Entries less severe than this are not sent SyslogLogTypes syslog_log_smtp, syslog_log_pop3, syslog_log_imap, syslog_log_application, syslog_log_tcpip, syslog_log_debug 86 One bit mask: SMTP, POP3, application and IMAP on; TCP/IP and debug off Errors are sent whatever the mask says. A message looks like this: <22>1 2026-09-15T14:21:09.284+01:00 mail.example.com hMailServer 4812 SMTPD [hmailserver@32473 thread="7412" session="41" client="203.0.113.9"] <BOM>RECEIVED: MAIL FROM:<[email protected]> The message id is the log category (SMTPD and so on) or, on an error line carrying an hMailServer error number, that number, so msgid == HM5165 is a usable filter at the collector. The structured data under hmailserver@32473 carries thread, and session and client on a conversation line. UDP truncates each message to 1024 bytes, cutting the text and never the header; TCP and TLS frame each message with its length and truncate nothing below 65000 bytes. TLS verifies the collector's certificate against the machine's trust store and the configured host name, so set the host to the name in the certificate, not an IP address; no client certificate is offered. A collector that is down costs log lines, not mail. Entries go to a background thread over a queue of 4096 that drops the oldest when full; a stream that cannot connect backs off from one second to a minute; and the application log gets one line when an outage starts and one when it ends, with the number dropped. Nothing is retried from disk. Each step of a stream transport (the connection, the TLS handshake, one entry written) is given ten seconds, so a collector that accepts the connection and then goes quiet is a failure like a refused one, and a reinitialise or a service stop waits for that step at most. For rsyslog, in /etc/rsyslog.d/30-hmailserver.conf (UDP for the default transport, TCP for SyslogTransport = 1; the stop keeps the lines out of /var/log/mail.log too, so drop it if you want both): module(load="imudp") input(type="imudp" port="514") module(load="imtcp") input(type="imtcp" port="514") if ($app-name == "hMailServer") then { action(type="omfile" file="/var/log/hmailserver.log" template="RSYSLOG_SyslogProtocol23Format") stop } Without RSYSLOG_SyslogProtocol23Format rsyslog reformats to the older RFC 3164 shape and the structured data is lost. For TLS (SyslogTransport = 2 module(load="imtcp" StreamDriver.Name="gtls" StreamDriver.Mode="1" StreamDriver.AuthMode="anon") input(type="imtcp" port="6514") global(DefaultNetstreamDriverCAFile="/etc/ssl/certs/ca.pem" DefaultNetstreamDriverCertFile="/etc/rsyslog.d/collector-cert.pem" DefaultNetstreamDriverKeyFile="/etc/rsyslog.d/collector-key.pem") A self-signed collector certificate has to be installed as trusted on the hMailServer machine, or the handshake is refused and nothing is sent. The journal, on Linux only, needs no collector. Where the server runs as the packaged hmailserver.service, switch the sink on and leave the collector host empty: $ hmctl settings set logging --set syslog_enabled=true $ sudo systemctl restart hmailserver $ journalctl -u hmailserver -f Beside the standard fields the journal gets HMAILSERVER_CATEGORY, HMAILSERVER_MSGID, HMAILSERVER_THREAD, HMAILSERVER_SESSION and HMAILSERVER_CLIENT: $ journalctl -u hmailserver HMAILSERVER_SESSION=41 $ journalctl -u hmailserver HMAILSERVER_MSGID=HM5165 If a collector host is configured the journal is not used. The start of the sink is itself logged, as Syslog: sending RFC 5424 log entries to <host>:<port> over UDP, facility 2. or Syslog: writing log entries to the systemd journal.; a server that logs neither has the feature off. Not sent: the AWStats log, the backup log and the event-script log. Per-domain reports (since 6.3.4) GET /api/v1/reports is an index of the sections, the state of their three sources and what this server cannot answer. A report drawn while its source was off is empty, which is not the same statement as "nothing happened"; each section's note says which it is. GET /api/v1/reports/{section} answers one section, the Deck's Reports view draws the summary, and hmctl report prints a section or saves it as CSV. There is no Control Panel page. Section Counted from Says traffic the message trace Per day and per local domain: incoming, outgoing, failed_incoming, failed_outgoing failures the message trace Per local domain and SMTP status code senders, recipients the message trace The local addresses that sent or received the most, with their failures mailboxes hm_messages The largest mailboxes as they are now: a size, not a history volume the metric history, and the daily per-domain sample Processed, delivered, deferred, bounced, spam and viruses per day, with by_domain rows of spam and viruses storage the same The store's size per day and growth_bytes, with by_domain rows of each domain's size summary all three sources Every section the credential may see, with an omitted list and the reason quota the store Every mailbox with a limit and at least one message, nearest its limit first, as a share of the limit: a share now, not a history countries the metric history and the country counters What a country policy refused, per day; the split by country is since the server last started, not per day The last two are described in QuotaPolicy.md and CountryBlocking.md, not in Reports.md. Parameters. from and to as YYYY-MM-DD, both inclusive, defaulting to the last 30 days and never more than 366; domain, one hosted domain (a domain not hosted here is a 404); top, 1 to 200, default 10; format=csv, refused for summary. In CSV a cell starting with =, +, - or @ is prefixed with an apostrophe. Three things decide whether a report means anything. The message trace is off by default (MessageTraceEnabled=0), and while it is off the four sections counted from it are empty. Processed, delivered, deferred and bounced cannot be given per domain, because those counters carry no domain; spam, viruses and storage can, since schema 6062 added the daily sample in hm_domainsamples, which starts the day it was first taken. And only hosted domains are counted. Authorisation. The administrator password reads everything; every other credential needs both view_configuration and view_message_trace. domainadmin and auditor hold both; helpdesk is refused. A key restricted to named domains sees only those and is refused 403 for another. On volume and storage Reports.md contradicts itself: its Authorisation section refuses them to such a key outright, while its section What cannot be answered, and what it would take (and, for volume, its table of sections under The route) gives a credential restricted to domains the per-domain rows, as the wiki's section 16.6a does. Test both with your own restricted key before a script relies on either. The cost. One report makes at most three grouped queries; the trace read is capped at 100 000 grouped rows and the answer says truncated when it reaches the cap. hmctl report sections hmctl report traffic --from 2026-09-01 --to 2026-09-15 --domain example.com Following one connection live Since 6.3.4 the Deck's Logs page lists the sessions open now and those recently ended, and Follow shows one session's lines as they are written, from an in-memory ring of the last 4,096 log entries that no restart keeps; it needs view_logs. Reading the logs covers following a connection in full, and the log files themselves. Where to read more Documents at v6.3.4: AlertsAndAuditTrail.md, Syslog.md, Reports.md and FollowConnection.md Wiki: Monitoring and Health, sections 16.2a, 16.6a, 16.7a and 16.7b The public documentation: 16.2 Logs on disk On this forum: Monitoring and health for the metrics, the probes, the dashboard and the SNMP agent; Administrator roles for the permissions named here. See also The Control Deck and hMailServer on Linux: what you get and what you do not

hMailServer Community

The community support forum for hMailServer — the free, open-source mail server for Windows and Linux — maintained by Progressive Robot Ltd. The software is AGPL-3.0 and stays that way; this is where it is discussed, debugged and documented.

The software

Getting help

Legal

Progressive Robot Ltd · 220 Chester Business Park, Heronsway, Chester, Cheshire CH4 9GB, United Kingdom · +44 (0)1244 911212 · [email protected] · VAT 506152326

Powered by NodeBB