2 Remote commands

Overview

With remote commands you can define that a certain pre-defined command is automatically executed on the monitored host upon some condition.

Thus remote commands are a powerful mechanism for smart pro-active monitoring.

In the most obvious uses of the feature you can try to:

• Automatically restart some application (web server, middleware, CRM) if it does not respond
• Use IPMI 'reboot' command to reboot some remote server if it does not answer requests
• Automatically free disk space (removing older files, cleaning /tmp) if running out of disk space
• Migrate a VM from one physical box to another depending on the CPU load
• Add new nodes to a cloud environment upon insufficient CPU (disk, memory, whatever) resources

Configuring an action for remote commands is similar to that for sending a message, the only difference being that Zabbix will execute a command instead of sending a message.

Remote commands can be executed by Zabbix server, proxy or agent. Remote commands on Zabbix agent can be executed directly by Zabbix server or through Zabbix proxy. Both on Zabbix agent and Zabbix proxy remote commands are disabled by default. They can be enabled by:

• adding an AllowKey=system.run[*] parameter in agent configuration;
• setting the EnableRemoteCommands parameter to '1' in proxy configuration.

Remote commands executed by Zabbix server are run as described in Command execution including exit code checking.

Remote commands are executed even if the target host is in maintenance.

Remote command limit

Remote command limit after resolving all macros depends on the type of database and character set (non-ASCII characters require more than one byte to be stored):

Remote command execution output (return value) is limited to 16MB (including trailing whitespace that is truncated). IPMI remote command limit is based on the installed IPMI library. Note that database limits apply to all remote commands.

Configuration

Those remote commands that are executed on Zabbix agent (custom scripts) must be first enabled in the agent configuration.

Make sure that the AllowKey=system.run[<command>,*] parameter is added for each allowed command in agent configuration to allow specific command with nowait mode. Restart agent daemon if changing this parameter.

Then, when configuring a new action in Alerts → Actions → Trigger actions:

1. Define the appropriate conditions, for example, set that the action is activated upon any disaster problems with one of Apache applications.

2. In the Operations tab, click on Add in the Operations, Recovery operations, or Update operations block.

3. Select one of the predefined scripts from the Operation dropdown list and set the Target list for the script.

Predefined scripts

Scripts that are available for action operations (webhook, script, SSH, Telnet, IPMI) are defined in global scripts.

For example:

sudo /etc/init.d/apache restart 

In this case, Zabbix will try to restart an Apache process. With this command, make sure that the command is executed on Zabbix agent (click the Zabbix agent button against Execute on).

Remote commands on Zabbix agent are executed without timeout by the system.run[,nowait] key and are not checked for execution results. On Zabbix server and Zabbix proxy, remote commands are executed with timeout as set in the TrapperTimeout parameter of zabbix_server.conf or zabbix_proxy.conf file and are checked for execution results. For additional information, see Script timeout.

Access permissions

Make sure that the 'zabbix' user has execute permissions for configured commands. One may be interested in using sudo to give access to privileged commands. To configure access, execute as root:

visudo

Example lines that could be used in sudoers file:

# allows 'zabbix' user to run all commands without password.
       zabbix ALL=NOPASSWD: ALL
       
       # allows 'zabbix' user to restart apache without password.
       zabbix ALL=NOPASSWD: /etc/init.d/apache restart

Remote commands with multiple interfaces

If the target system has multiple interfaces of the selected type (Zabbix agent or IPMI), remote commands will be executed on the default interface.

It is possible to execute remote commands via SSH and Telnet using another interface than the Zabbix agent one. The available interface to use is selected in the following order:

• Zabbix agent default interface
• SNMP default interface
• JMX default interface
• IPMI default interface

IPMI remote commands

For IPMI remote commands the following syntax should be used:

<command> [<value>]

where

• <command> - one of IPMI commands without spaces
• <value> - 'on', 'off' or any unsigned integer. <value> is an optional parameter.

Examples

Examples of global scripts that may be used as remote commands in action operations.

Example 1

Restart of Windows on certain condition.

In order to automatically restart Windows upon a problem detected by Zabbix, define the following script:

Example 2

Restart the host by using IPMI control.

Example 3

Power off the host by using IPMI control.