Vote

Watchdogs

MailCleaner Support
Added 6 months ago

Work in progress¶

MailCleaner Enterprise Edition has a number of Watchdogs which report common errors to MailCleaner Staff. An effort to make these errors more visible for clients as well as for Community Edition users so that they can correct the issues for themselves is in progress.

Below are a list of the current Watchdogs as well as recommendations on the possible resolutions. Note that many of the watchdogs that are not specifically checking a daemon's status operate by monitoring the current day's logs, so corrections will not be noticed until the next day.

detect_bad_git.sh¶

This indicates that the last time the automatic update script (/root/Updater4MC/updater4mc.sh) ran, it exited with an error showing that the Git repository couldn't be synced due to local corruption. You can try to diagnose the issue by moving to the MailCleaner repository and checking the status and/or attempting to pull changes:

cd /usr/mailcleaner
git status
git pull

detech_Community¶

This test simply reports whether the version of the host is seen to be Community. This watchdog is only downloaded when a machine is registered for Enterprise and will report if the machine that had been registered now appears to be using Community. Some events, such as running a 'reset' on the Git tree can cause a registered appliance to revert to community.

If you believe the appliance should still be registered, log in to the web interface of the affected host and use the Configuration->Base System->Registration form to register the host again. If this does not resolve the issue, you can contact support.

If you intentionally unregistered the appliance, it is possible that this script failed to be removed. You can simply delete the script and its configuration file:

rm /usr/mailcleaner/bin/watchdog/MC_mod_detect_Community.sh /usr/mailcleaner/etc/watchdog/MC_mod_detect_Community.conf

detect_F2B¶

Checks whether the Fail2Ban server is running. This feature was in development for quite some time and this watchdog was used to alert us to any hosts using the beta feature. It has since been released. It can appear for two reasons. The first being that you have an older version of the script which is erroneously reporting that fail2ban is not running. Otherwise you have an updated script that reports that Fail2Ban is not running.

Some admins will choose to disable Fail2Ban since they may have other firewall protections already in place. If this is the case, you can simply disable this script by adding 'disable' to the TAGS in /usr/mailcleaner/etc/watchdog/MC_mod_detect_F2B.conf :

TAGS=all dix disable

detect_license_Kaspersky¶

Check to see if the Kaspersky license is expired, if installed. Contact your sales representative if your license has expired. If you trialed Kaspersky or used it for a time but no longer wish to use it you can uninstall the package with:

apt-get remove kaspersky-64-2.0

detect_msg_sniffer_db¶

Searches for MessageSniffer logs without any database information, indicating that it has not been successfully downloaded.

Contact support to let us know that you need the DB to be re-downloaded.

detect_PrefTDaemon¶

Checks for redundant PrefTDaemon processes

Restart the Preferences daemon to kill any orphaned processes. This can be done from Monitoring->Status (you will need to expand the Status column with "Show more..."), or with the command:

/usr/mailcleaner/etc/init.d/preftdaemon restart

NOTE: An update to the init scripts should have removed the chance of seeing this error during normal operation. If you are regularly seeing this watchdog it is likely that your system has not fully updated.

detect_pyenv_install.sh¶

Detects the unsuccessful installation of the new Python libraries being used to develop new tools. These libraries are not yet used for any component of the system other than Fail2Ban, so it is not critical to fix at this time. We will monitor the reports that we receive as it nears broader usage.

If you would like to try installing these libraries manually to remove the error, you can do so with:

pip install mailcleaner-library -U --trusted-host repository.mailcleaner.net --index https://repository.mailcleaner.net/python/ --extra-index https://pypi.org/simple/

detect_Spam_Handler¶

Checks for redundant Spam_Handler processes

Restart the SpamHandler daemon to kill any orphaned processes. This can be done from Monitoring->Status (SpamHandler is identified as "Filtering Engine"), or with the command:

/usr/mailcleaner/etc/init.d/spamhandler restart

detect_StatsDaemon¶

Checks for redundant StatsDaemon processes

Restart the StatsDaemon to kill any orphaned processes. This can be done from Monitoring->Status, or with the command:

/usr/mailcleaner/etc/init.d/statsdaemon restart

detect_Summaries¶

Reports an inability for the summaries to be sent due to an inability to connect to the database. This normally indicates that the Summaries have conflicted with a MailCleaner restart. This will happen if the summaries are sent too soon after the nightly updates at 10:30pm, or if they are still running prior to the updates. Consider changing the daily task timing from Configuration->General Settings->Periodic Tasks.

disk_full¶

Reports if the / or /var partitions are over 85% used.

You may choose to clear space, reduce the retention time for quarantined items, or increase the size of your disk.gg

disk_full_inodes¶

Same as previous except that it is based on the inode count (available unique files) instead of the actual bytes used.

exim_4.XX¶

Checks to make sure that you are using the latest supported version of Exim.

Attempt to upgrade with:

apt-get update && apt-get upgrade mc-exim

git_conflicts.sh¶

The last automatic update was unable to pull changes to the git tree because there are conflicting changes that could not be stashed and restored. This is probably because you've modified files that have sinced been changed upstream. Manually resolving the merge conflict is generally the only solution. You can find which files cause the issue from the last update log (/root/Updater4MC/updater_.log) which should contain:

error: The following untracked working tree files would be overwritten by merge:

followed by the list of files. Once these files are known, you can use the 'git diff' command to find out what changes you made:

cd /usr/mailcleaner
git diff path/to/file

You can then reset this file(s):

git reset -- path/to/file

then try manualy running the updater again:

/root/Updater4MC/updater4mc.sh

and reapply your modifications to the updated file, if possible/necessary.

Internal keys¶

Reports missing or excessive internal keys in /root/.ssh/authorized_keys

modified_exim1¶

Checks for modifications to: /usr/mailcleaner/etc/exim/exim_stage1.conf_template

Historically many clients have customized this template in order to change Exim settings not available in the UI. Unfortunately, this file is tracked by Git, so changes would be overwitten any time that an upstream change to the file was published. This alert allows us to know who will be impacted by an upstream change.

Currently, this customization is rarely necessary as there are dedicated files to hold customizations to all of the most frequently modified settings in the /usr/mailcleaner/etc/exim/stage1 directory. These will not be overwritten since the upstream copy should never change.

nb_mails_in_queue¶

Reports when there are a greater number of messages in each of the Exim queues than are expected during normal operation. Reasons for queuing vary. Open the queued item list from Monitoring->Status by clicking on the queue count. If the issue is resolved, you can try to flush the queue directly:

https://support.mailcleaner.net/boards/3/topics/49

If you have several machines and only one with a backlog, you may wish to temporarily stop the Inbound MTA on that machine until it clears.

slave_status¶

Checks if the Slave DB is in sync. If instructed, correct with:

/usr/mailcleaner/bin/resync_db.sh

TesseractOcr¶

Not yet implemented:

Checks to ensure that TesseractOcr dependencies have been installed. These will be necessary when the TesseractOcr module is added to SpamAssassin. The dependencies require about 250MB, so the upgrade is scheduled to avoid installing them if there won't be room to spare.

Check do see available disk space:

df -h /

If you have at least 400MB, you can install the dependencies manually:

apt-get install tesseract-ocr libopencv-dev libswitch-perl

Updater_Status¶

Reports troubles with last automatic update. Will report

if the update exited unsuccessfully
if the main /etc/mailcleaner.conf file is invalid and thus cannot provide the necessary details to do an update
if it started running over 1 hour ago and never finished.

You can try manually running the updater (/root/Updater4MC/updater4mc.sh) or inspect the last log (/root/Updater4MC/updater_.log).

May appear alongside other watchdogs with a better description of the issue.

Updater_TLS¶

Checks for TLS issues during the update process. This is likely to be an issue on our end. Contact us if you see this error.

Others¶

Watchogs are added and removed regularly. Check to make sure you are on the latest version because you could be seeing a watchdog that is no longer relevant. If you do believe that you are on the latest version and see an error that isn't listed, report it to us as we may be required to update the documentatino. Thanks!

MailCleaner Support

Message

Related messages