maybe not a good idea

adventures in scripting.

installing aws-cli on MacOS 10.11

If you are experiencing troubles installing aws-cli via pip on MacOS El Capitan - here’s the key:

1
sudo -H pip install awscli --upgrade --ignore-installed six

I found this solution on github

Monitoring GPU Performance on Windows via PRTG and nvidia-smi

Last time we talked about Monitoring GPU Performance on Windows via PRTG and OpenHardwareMonitor. Today I would like to introduce the possibility to do this with NVIDIA’s smi.

The NIVIDIA System Management Interface is a small command line tool. You can find it here: C:\Program files\NVIDIA Corporation\NVSMI\nvidia-smi.exe. As PRTG needs a very specific output, smi needs to generate a very specific output. In fact: just the unit of the utilisation.
This can be done with these parameters:

1
C:\Program files\NVIDIA Corporation\NVSMI\nvidia-smi.exe' --query-gpu=utilization.gpu --format="csv,nounits,noheader" --id=$ID

So now smi ouputs the current gpu utilisazation of the GPU with the ID $ID. The rest is pretty simple: we provide our credentials to the script and execute the command remotely via Invoke-Command. In addition I built a loop for the command, because most of the render server provide more than one GPU. So with this loop we’ll get the correct amout and value of all provided GPUs. We’ll also provide the result in the specific scheme for a PRTG custom EXE/XML sensor. So we’re able to use only one sensor and every GPU gets its own sensor. Here we go:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
param(
[string]$cn, #computername
[string]$un, #username
[string]$pw, #password
[string]$dn) #domain
$Password = $pw | ConvertTo-SecureString -AsPlainText -Force
$Username = $dn+'\'+$un
$cred = New-Object System.Management.Automation.PSCredential($Username,$Password)
$number = Invoke-Command -ComputerName $cn -Credential $cred -ScriptBlock { & 'C:\Program files\NVIDIA Corporation\NVSMI\nvidia-smi.exe' -q }
$numberfilter = $number | find "Attached GPUs"
$amount = ($numberfilter) -replace '\D+(\d+)','$1'
"<prtg>"
For ($i = 0; $i -lt $amount; $i++) {
"<result>
<channel>"
"GPU $i"
"</channel>
<value>"
Invoke-Command -ComputerName $cn -Credential $cred -ArgumentList $i -ScriptBlock {param($i) & 'C:\Program files\NVIDIA Corporation\NVSMI\nvidia-smi.exe' --query-gpu=utilization.gpu --format="csv,nounits,noheader" --id=$i}
"</value>
</result>"}
"</prtg>"

The parameters for the custom sensor are the same than last time: '%host' '%windowsuser' '%windowspassword' '%windowsdomain'

Monitoring GPU Performance on Windows via PRTG and OpenHardwareMonitor

Monitoring the performance of a GPU is not pretty easy and it depends on the GPU. In this case we’re monitoring a NVIDIA Titan X on a Windows Machine, mainly for statistics. However there are mostly two ways to get this done.

OpenHardwareMonitor is a small tool to monitor various hardware sensors like fans, temperature, load and stuff like this. The really special about this tool is: It outputs the values via WMI.

First we would have to be sure that WMI and OpendHardwareMonitor is up and running on our target. Then we would see what the WMI Object actually returns:

1
Get-WmiObject -Namespace root\openhardwaremonitor -ComputerName $computername -Credential $domain\user

It will return a lot of stuff, but we’re looking for the GPU load or other sensors we would like to monitor. Now we can send a specific query to get the value:

1
Get-WmiObject -Namespace root\openhardwaremonitor -ComputerName $computername -Credential $domain\user -Query "SELECT * FROM Sensor WHERE Name like '%GPU Core%' AND SensorType = 'Load'"

As we would like to use this as Custom Sensor in PRTG, we well have to provide the credentials in the script. In my case the monitored computer provides 4 GPUs in total and we want to monitor all of them. To make it easier to handle I decided to use only one sensor with 4 channels in PRTG using the custom XML sensor, which needs to return the values formated in a special scheme:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
param(
[string]$cn, #Computername
[string]$un, #Username
[string]$pw, #Password
[string]$dn) #Domain
$Password = $pw | ConvertTo-SecureString -AsPlainText -Force
$Username = $dn+'\'+$un
$cred = New-Object System.Management.Automation.PSCredential($Username,$Password)
$vid = Get-WmiObject -Namespace root\openhardwaremonitor -ComputerName $cn -Credential $cred -Query "SELECT * FROM Sensor WHERE Name like '%GPU Core%' AND SensorType = 'Load'"
$amount = $vid.count
"<prtg>"
For ($i = 0; $i -lt $amount; $i++) {
"<result>
<channel>"
"GPU $i"
"</channel>
<value>"
Get-WmiObject -Namespace root\openhardwaremonitor -ComputerName $cn -Credential $cred -Query "SELECT Value FROM Sensor WHERE Identifier like '%/nvidiagpu/$i/load/0%'" | Select-Object -ExpandProperty Value
"</value>
</result>"}
"</prtg>"

Next we copy the ps script to C:\Program Files (x86)\PRTG Network Monitor\Custom Sensors\EXEXML on the probe.

Finally we have to configure the Sensor: As parameters we will use the placeholders provided by PRTG: '%host' '%windowsuser' '%windowspassword' '%windowsdomain'


© 2017 Martin Groiss