Warning threshold for GPU overall utilization, in %.

Critical threshold for GPU overall utilization, in %.

Warning threshold for encoder utilization, in %.

Critical threshold for encoder utilization, in %.

Warning threshold for decoder utilization, in %.

Critical threshold for decoder utilization, in %.

Warning threshold for memory utilization, in %.

Critical threshold for memory utilization, in %.

Warning threshold for fan speed, in %.

Critical threshold for fan speed, in %.

Warning threshold for temperature, in %.

Critical threshold for temperature, in %.

Warning threshold for power usage, in %.

Critical threshold for power usage, in %.

Filter to include GPUs by name in discovery.

Filter to exclude GPUs by name in discovery.

Filter to include GPUs by UUID in discovery.

Filter to exclude GPUs by UUID in discovery.

Retrieves the version of the system's graphics driver.

For all Nvidia products.

Preprocessing

• Discard unchanged with heartbeat: 1d

Retrieves the version of the NVML library.

For all Nvidia products.

Preprocessing

• Discard unchanged with heartbeat: 1d

Retrieves the number of compute devices in the system. A compute device is a single GPU.

For all Nvidia products.

Preprocessing

• Discard unchanged with heartbeat: 1d

Retrieves list of Nvidia devices in the system.

Driver version has changed.
Check out changelog for specific driver version at Nvidia website: https://www.nvidia.com/en-us/drivers/

NVML library version has changed.
Changelog can be found here: https://docs.nvidia.com/deploy/nvml-api/change-log.html

Number of devices has changed. Check out if it was intentional.

Nvidia GPU discovery in the system.

Preprocessing

• Discard unchanged with heartbeat: 1d

Retrieves the globally unique board serial number associated with this device's board.

For all products with an inforom.

This number matches the serial number tag that is physically attached to the board.

Preprocessing

• Check for not supported value: The text is too long. Please see the template.

  ⛔️Custom on fail: Set error to: The device does not support operation to retrieve serial number.

Retrieves the current utilization for the Encoder.

For Nvidia Kepler or newer fully supported devices.

Retrieves the current utilization for the Decoder.

For Nvidia Kepler or newer fully supported devices.

Retrieves the intended operating speed of the device's specified fan.

Note: The reported speed is the intended fan speed. If the fan is physically blocked and unable to spin, the output will not match the actual fan speed.

For all Nvidia discrete products with dedicated fans.

The fan speed is expressed as a percentage of the product's maximum noise tolerance fan speed. This value may exceed 100% in certain cases.

Retrieves power usage for this GPU in watts and its associated circuitry (e.g. memory).

For Nvidia Fermi or newer fully supported devices.

On Fermi and Kepler GPUs the reading is accurate to within +/- 5% of current power draw. On Ampere (except GA100) or newer GPUs, the API returns power averaged over 1 sec interval. On GA100 and older architectures, instantaneous power is returned.

Preprocessing

• Custom multiplier: 0.001

Retrieves the power management limit associated with this device.

For Nvidia Fermi or newer fully supported devices.

The power limit defines the upper boundary for the card's power draw. If the card's total power draw reaches this limit the power management algorithm kicks in.

This reading is only available if power management mode is supported.

Preprocessing

• Custom multiplier: 0.001

Retrieves total energy consumption for this GPU in joules (J) since the driver was last reloaded.

For Nvidia Volta or newer fully supported devices.

Preprocessing

• Custom multiplier: 0.001

Retrieves the current temperature readings for the device, in degrees C.

For Nvidia all products.

Retrieves the current memory clock speed for the device.

For Nvidia Fermi or newer fully supported devices.

Preprocessing

• Custom multiplier: 1000000

Retrieves the current SM clock speed for the device.

For Nvidia Fermi or newer fully supported devices.

Preprocessing

• Custom multiplier: 1000000

Retrieves the current graphics clock speed for the device.

For Nvidia Fermi or newer fully supported devices.

Preprocessing

• Custom multiplier: 1000000

Retrieves the current video encoder/decoder clock speed for the device.

For Nvidia Fermi or newer fully supported devices.

Preprocessing

• Custom multiplier: 1000000

Retrieves the current performance state for the device.

For Nvidia Fermi or newer fully supported devices.

Retrieves the current utilization rates for the device's major subsystems.

For Nvidia Fermi or newer fully supported devices.

Percent of time over the past sample period during which one or more kernels was executing on the GPU.

For Nvidia Fermi or newer fully supported devices.

Preprocessing

• JSON Path: $.device

Percent of time over the past sample period during which global (device) memory was being read or written.

For Nvidia Fermi or newer fully supported devices.

Preprocessing

• JSON Path: $.memory

Retrieves the current encoder statistics for a given device.

For Nvidia Maxwell or newer fully supported devices.

Retrieves the current count of active encoder sessions for a given device.

For Nvidia Maxwell or newer fully supported devices.

Preprocessing

• JSON Path: $.session_count

Retrieves the trailing average FPS of all active encoder sessions for a given device.

For Nvidia Maxwell or newer fully supported devices.

Preprocessing

• JSON Path: $.average_fps

Retrieves the current encode latency for a given device.

For Nvidia Maxwell or newer fully supported devices.

Preprocessing

• JSON Path: $.average_latency_ms

• Custom multiplier: 0.001

Retrieves the amount of used, free, reserved and total memory available on the device.

For all Nvidia products.

Enabling ECC reduces the amount of total available memory, due to the extra required parity bits. Under WDDM most device memory is allocated and managed on startup by Windows.

Under Linux and Windows TCC, the reported amount of used memory is equal to the sum of memory allocated by all active channels on the device.

Total physical memory on the device.

For all Nvidia products.

Preprocessing

• JSON Path: $.total_memory_bytes

Memory reserved for system use (driver or firmware) on the device.

For all Nvidia products.

Preprocessing

• JSON Path: $.reserved_memory_bytes

  ⛔️Custom on fail: Set error to: NVML library too old to support this metric.

Unallocated memory on the device.

For all Nvidia products.

Preprocessing

• JSON Path: $.free_memory_bytes

Allocated memory on the device.

For all Nvidia products.

Preprocessing

• JSON Path: $.used_memory_bytes

Gets Total, Available and Used size of BAR1 memory.

BAR1 is used to map the FB (device memory) so that it can be directly accessed by the CPU or by 3rd party devices (peer-to-peer on the PCIE bus).

For Nvidia Kepler or newer fully supported devices

Total BAR1 memory on the device.

For Nvidia Kepler or newer fully supported devices

Preprocessing

• JSON Path: $.total_memory_bytes

Unallocated BAR1 memory on the device.

For Nvidia Kepler or newer fully supported devices

Preprocessing

• JSON Path: $.free_memory_bytes

Allocated used BAR1 memory on the device.

For Nvidia Kepler or newer fully supported devices

Preprocessing

• JSON Path: $.used_memory_bytes

Retrieves the GPU device memory error counters for the device.

For Nvidia Fermi or newer fully supported devices.

Requires NVML_INFOROM_ECC version 2.0 or higher to report aggregate location-based memory error counts. Requires NVML_INFOROM_ECC version 1.0 or higher to report all other memory error counts.

Only applicable to devices with ECC.

Requires ECC Mode to be enabled.

Preprocessing

• Check for not supported value: The text is too long. Please see the template.

  ⛔️Custom on fail: Set error to: No ECC on the device or ECC mode is turned off.

Retrieves the count of GPU device memory errors that were corrected. For ECC errors, these are single bit errors, for Texture memory, these are errors fixed by resend.

For Nvidia Fermi or newer fully supported devices.

Preprocessing

• JSON Path: $.corrected

Retrieves the count of GPU device memory errors that were not corrected. For ECC errors, these are double bit errors, for Texture memory, these are errors where the resend fails.

For Nvidia Fermi or newer fully supported devices.

Preprocessing

• JSON Path: $.uncorrected

Retrieves the GPU register file error counters for the device.

For Nvidia Fermi or newer fully supported devices.

Requires NVML_INFOROM_ECC version 2.0 or higher to report aggregate location-based memory error counts. Requires NVML_INFOROM_ECC version 1.0 or higher to report all other memory error counts.

Only applicable to devices with ECC.

Requires ECC Mode to be enabled.

Preprocessing

• Check for not supported value: The text is too long. Please see the template.

  ⛔️Custom on fail: Set error to: No ECC on the device or ECC mode is turned off.

Retrieves the count of GPU register file errors that were corrected. For ECC errors, these are single bit errors, for Texture memory, these are errors fixed by resend.

For Nvidia Fermi or newer fully supported devices.

Preprocessing

• JSON Path: $.corrected

Retrieves the count of GPU register file errors that were not corrected. For ECC errors, these are double bit errors, for Texture memory, these are errors where the resend fails.

For Nvidia Fermi or newer fully supported devices.

Preprocessing

• JSON Path: $.uncorrected

Retrieve PCIe utilization information.

For Maxwell or newer fully supported devices.

The PCIe Rx (receive) throughput over 20ms interval on the device.

For Maxwell or newer fully supported devices.

Preprocessing

• JSON Path: $.rx_rate_kb_s

• Custom multiplier: 1024

The PCIe Tx (transmit) throughput over 20ms interval on the device.

For Maxwell or newer fully supported devices.

Preprocessing

• JSON Path: $.tx_rate_kb_s

• Custom multiplier: 1024

[{#UUID}]: Encoder utilization is very high. It may indicate abnormal behavior/activity. Change corresponding macro in case of false-positive.

[{#UUID}]: Encoder utilization is high. It may indicate abnormal behavior/activity. Change corresponding macro in case of false-positive.

• Nvidia: [{#UUID}]: Encoder utilization exceeded critical threshold

[{#UUID}]: Decoder utilization is very high. It may indicate abnormal behavior/activity. Change corresponding macro in case of false-positive.

[{#UUID}]: Decoder utilization is high. It may indicate abnormal behavior/activity. Change corresponding macro in case of false-positive.

• Nvidia: [{#UUID}]: Decoder utilization exceeded critical threshold

[{#UUID}]: Fan speed is very high. It may indicate abnormal behavior/activity. Change corresponding macro in case of false-positive.

[{#UUID}]: Fan speed is high. It may indicate abnormal behavior/activity. Change corresponding macro in case of false-positive.

• Nvidia: [{#UUID}]: Fan speed exceeded critical threshold

[{#UUID}]: Power usage is very high. It may indicate abnormal behavior/activity. Change corresponding macro in case of false-positive.

[{#UUID}]: Power usage is high. It may indicate abnormal behavior/activity. Change corresponding macro in case of false-positive.

• Nvidia: [{#UUID}]: Power usage exceeded critical threshold

Power limit for the device has changed. Checkout out if it was intentional.

[{#UUID}]: Temperature is very high. It may indicate abnormal behavior/activity. Change corresponding macro in case of false-positive.

[{#UUID}]: Temperature is high. It may indicate abnormal behavior/activity. Change corresponding macro in case of false-positive.

• Nvidia: [{#UUID}]: Temperature exceeded critical threshold

[{#UUID}]: GPU utilization is very high. It may indicate abnormal behavior/activity. Change corresponding macro in case of false-positive.

[{#UUID}]: GPU utilization is high. It may indicate abnormal behavior/activity. Change corresponding macro in case of false-positive.

• Nvidia: [{#UUID}]: GPU utilization exceeded critical threshold

[{#UUID}]: Memory utilization is very high. It may indicate abnormal behavior/activity. Change corresponding macro in case of false-positive.

[{#UUID}]: Memory utilization is high. It may indicate abnormal behavior/activity. Change corresponding macro in case of false-positive.

• Nvidia: [{#UUID}]: Memory utilization exceeded critical threshold

Total FB memory has changed. That could mean possible memory degradation, hardware configuration changes or memory reservation by system or software.

Total BAR1 memory has changed. That could mean possible memory degradation, hardware configuration changes or memory reservation by system or software.

Increasing number of corrected ECC errors can indicate (but not necessary mean) aging or degrading memory.

Increasing number of uncorrected ECC errors can indicate potential issues such as: data corruption, system instability, hardware issues

Increasing number of corrected register file errors can indicate (but not necessary mean) wearing, aging or degrading memory.

Increasing number of uncorrected register file errors can indicate potential issues such as: data corruption, system instability, hardware degradation