SCOM - Check for and alert on low space on mount points - Part 1

I have a number of sites still running Exchange 2010 and because of the buggy management pack there is often not a way to alert on the drive space of an exchange server mount point because sites have removed the management pack.

In searching for a solution, I came across a nice article that helped get me started here:

http://www.powershellneedfulthings.com/?p=36

I took that script and added some additional information so that I could generate both an error condition on low drive space as well as a recovery alert so SCOM knew when to close the alert itself.

In order for this script to run cleanly, you first need to run the following powershell command on any server you want to run mout-point checks on:

new-eventlog -LogName System -Source OpsHealthScript

Next, schedule the following script to run periodically on your server via task scheduler or whatever program you might use to run scripts on a recurring basis.

$TotalGB = @{Name="Capacity(GB)";expression={[math]::round(($_.Capacity/ 1073741824),2)}}
$FreeGB = @{Name="FreeSpace(GB)";expression={[math]::round(($_.FreeSpace / 1073741824),2)}}
$FreePerc =    @{Name="Free(%)";expression={[math]::round(((($_.FreeSpace / 1073741824)/($_.Capacity / 1073741824)) * 100),0)}}
$Check = @{Name="Failed";expression={[math]::round(((($_.FreeSpace / 1073741824)/($_.Capacity / 1073741824)) * 100),0) -lt 10}}

 function get-mountpoints {
 $global:volumes = Get-WmiObject -computer $server win32_volume | Where-object {$_.DriveLetter -eq $null}
 $global:volumes | Select SystemName, Label, Capacity, FreeSpace, $Check | Format-Table -AutoSize
        }

$servers = [system.environment]::MachineName
foreach ($server in $servers){
get-mountpoints
}
$Flag = $global:volumes | Select $Check
If ($Flag -match "True") {
Write-EventLog -logname System -source OpsHealthScript -eventID 500 -Entrytype Error -message 'One or more Exchange mount points are below 10% free space.'
}
Else {
Write-EventLog -logname System -source OpsHealthScript -eventID 800 -Entrytype Information -message 'Exchange mount points are healthy.'
}

You can access the file here: mount-point-space.ps1

Once this is scheduled, you can setup alerting in a couple of ways, one is a simple alert detection within SCOM to detect Error 500 in the system event log, the other is a correlated alert detection so that each alert generated by the script does not create a new alert within SCOM. I'll go over that in additional detail.

Using Monitors for Automated Disk Space Recovery

Thought I would do a new post today. It's the day after Thanksgiving and guess who drew the after-hours phone a monitoring ticket this week? So, bunch of disk space monitors came in and I decided I didn't want to have to respond to them, especially since I will have after-hours duties during Christmas as well, lucky me.

So the concept I'll illustrate here is to have SCOM take automated actions to clean-up drive space on its own, hopefully averting an impending disaster and the need for you to get up from your peaceful slumber, camping trip or whatever it is you do when you try to have a life outside of daily IT tasks.

As with any Microsoft product, there are about a dozen ways to do this. I give you probably the simplest way, but long term would probably be difficult to maintain in a large server environment. I'll improve on this and post any updates when I do.

First, let us start by creating a folder on the root drive of a target server. I called the folder "C:\Scripts". Create a batch file in the folder. I called mine "cleanup.bat"

I have included the contents of the sample script. It is pretty basic, essentially clearing out temporary files in a variety of locations. This could certainly be expanded to remove old IIS or Blackberry log files, remove SQL backups, etc. Any action that can be scripted can essentially be run here.

del C:\Temp\*.* /s /q

del %Windir%\Temp /s /q

IF EXIST "C:\Users\" (

    for /D %%x in ("C:\Users\*") do (

        del /f /s /q "%%x\AppData\Local\Temp\"

        del /f /s /q "%%x\AppData\Local\Microsoft\Windows\Temporary Internet Files\"

    )

)

IF EXIST "C:\Documents and Settings\" (

    for /D %%x in ("C:\Documents and Settings\*") do (

        del /f /s /q "%%x\Local Settings\Temp\"

        del /f /s /q "%%x\Local Settings\Temporary Internet Files\"

    )

)

With the script loaded on the target server, go to your SCOM management console and navigate to the authoring tab and then select the "Monitors" option.

After the monitors all load, in th search field, type in "Logical Disk" to narrow the monitors. Once the search finishes, you can then select one of the Logical Disk Free Space monitors, such as Windows Server 2000, Windows Server 2003 or Windows Server 2008. I chose to alter the settings for the Windows Server 2008 Logical Disk Free Space monitor. Right-click on the Logical Disk Free Space monitor and select "Properties".

Navigate to the "Diagnostic and Recovery" tab. Here, we will want to add a recovery task. You can put in the same task for both warning and critical recovery tasks if you like.

In the "Recovery Task Type", select "Run Command" and change the destination management pack to either the default client overrides or an alternate management pack you might have for such customizations.

On the next section, give the recovery task a name that makes sense. You can choose to have the system recalculate the monitor. If disk space falls back into the norm, the alert will clear itself automatically.

On the last screen, enter the path on the target server where the script was located. As I outlined in the beginning, I put the scripts in the "C:\Scripts\" folder with a file called cleanup.bat.

You are basically done at this point. When the alert or warning condition occurs (which ever you setup to respond to), the script will kick off and delete the files.

The nice thing about running a recovery task in this manner is that if the alert persists, you know there is a larger problem as the basic steps of clearing up misc., temporary files has already been accomplished. Now you know there is something out of the ordinary going on.

You can now test this script by running a program called Philip 2.10. This will create a large file in one of the temporary directories cleared by the sample script. You can download the software from the following location:

http://www.softpedia.com/get/System/File-Management/Philip.shtml