LSI Megaraid Control and Monitoring Tools by Jefferson Ogata
------------------------------------------------------------


Disclaimer
----------

WARNING: Use this software at your own risk. The author accepts no
responsibility for the consequences of your use of this software.

WARNING: Use this software at your own risk. The author accepts no
responsibility for the consequences of your use of this software.

WARNING: Use this software at your own risk. The author accepts no
responsibility for the consequences of your use of this software.

These programs directly query megaraid adapters via the ioctl(2) driver
interface and do a number of undocumented things. I and my colleagues use this
software regularly and have had no problems, but your mileage may vary. If
something goes terribly wrong and your RAID configs all get blown away, the
author accepts no responsibility for the consequences.

Please read this document carefully as it contains a warning or two. If you
have built the programs but are having any issues running them, please see the
Building, Device Nodes, and Limitations notes further down in this document.


Introduction
------------

I've spent a fair amount of time working out the low-level interface to
megaraid adapters. This stems from the fact that I use a lot of these beasts
and have had failures at one time or another. The adapters are fast and
extremely useful, but they aren't bulletproof. For example, disks showing a
certain number of media errors are not failed immediately by the adapters; the
adapters seem to want to pass some threshold of failure before they decide that
a disk really needs to be dropped from a RAID, and by that time it's possible
there could be consistency problems. I wrote these tools so I could more
effectively monitor media errors (dellmgr makes this very tedious) and also
take advantage of the device self-test functions provided with drives.
Self-tests are in my opinion a suitable way to detect imminent drive failures
without tying up the adapter and SCSI bandwidth doing patrol reads. It is also
very useful to be able to conduct a self-test on a spare disk before using it
for a rebuild.

Another issue I've had with megaraids is keeping current documentation on RAID
configuration. You may choose a completely logical RAID layout when first
configuration a system, but that doesn't mean you'll remember it if you have to
reconstruct it in an emergency--did I leave out a disk for a hot spare? Which
one? Furthermore, as disks fail and hot spares are rotated in place, the
configuration changes over time. If you don't update your documentation every
time a disk fails, you lose track of it. While LSI's MegaCli program provides
methods for dumping the configuration (although verbosely) and saving and
restoring it from files, Dell's dellmgr doesn't give you any sensible way to
track down what spans comprise a logical disk. I wrote these programs in part
to solve this problem.


Programs
--------

This distribution contains three programs and a script:


*megactl*

megactl queries PERC2, PERC3, and PERC4 adapters and reports adapter
configuration, physical and logical drive condition, drive log sense pages, and
various other useful information. This allows you to document the actual
configuration of the adapter, since, for one thing, Dell's dellmgr does not
tell you which specific logical drive a given disk belongs to.

megactl has several features to query SCSI drive log pages, where each drive
controller saves accumulated error counts, drive temperature, self-test
results, et al. To get a list of supported log pages for a given drive, use
"megactl -vv -l 0 <target>" where target is the name of the drive, e.g. a0c1t2
for SCSI target 2 on channel 1 of adapter 0. In addition "-s" is shorthand for
"-l 0x10", "-t" is shorthand for "-l 0x0d", and "-e" is shorthand for "-l 0x02
-l 0x03 -l 0x05". megactl knows how to parse several useful log pages, but
there are others where you'll have to interpret the results yourself. Feel free
to write more parsing code.

megactl output is governed by a verbosity flag. At lower verbosity levels, the
program tends to minimize log page output unless it represents an actual
problem. So to see the full self-test log, you need to add "-vv". I usually run
the program with a single "-v".

Self-test and most status operations allow you to designate either an entire
adapter (a0), a specific channel (a0c1), or a specific drive (a0c1t2). When
performing drive self-test operations (q.v.), be sure to specify the actual
drive you wish to test, or you will end up starting a test on every drive on
the system. You may designate as many objects as you please, e.g.
"a0c0t{1,2,3,4,5,8} a0c1 a1".

megactl provides a health-check function, which inspects health check operation
allows only entire adapters to be designated. If no target is designated, the
program operates on all possible objects.

megactl with the -H option performs a health check of all (or specified)
adapters. By default, the health check checks the state of the adapter battery,
the state of all logical drives, and for each physical drive, the media error
count logged by the adapter, the read, write,and verify error log pages, and
the temperature log page. You can tune the log pages the health check will
inspect by specifying them with "-e", "-s", "-t", or "-l"; note that if you do
this, there is no default and you must specify every log page you wish to
inspect ("-et" for the default behaviour). If a problem is found, the program
prints the adapter and relevant drive info. If everything is okay, the program
is completely silent. So "megactl -vH" can be a useful cron job.

When using the health check you may specify which adapters you want to check,
but you may not designate specific channels or drives.

megactl also allows you to instruct the drive to perform a long ("-T long") or
short ("-T short") background self-test procedure, which does not take the
drive offline. I have performed self-tests on drives that are part of an
operational RAID many times with no problems. I recommend that you self-test
only one drive in a given span at a time; if the self-test causes the drive to
log errors, the adapter may fail the drive, and you don't want that to happen
to two drives in a span simultaneously or you may lose data.

You can get full usage info for megactl by executing it with the -? flag.

I use megactl on a number of PERC models, especially PERC3/QCs and PERC4/DCs.
In the past, megactl was known to work well with PERC2/DC adapters but I no
longer operate any systems with these adapters, so this may have broken. Please
let me know if you have success or problems.

megactl generally tries not to do anything harmful, so it's pretty safe.
Primarily it queries disks; the only instructions it issues in its current form
are to execute self-test operations.


*megasasctl*

The second program, megasasctl, is just like megactl, but intended for PERC5
adapters. The only syntactic difference is that instead of naming targets with
channels and ids, they are named with enclosures and slots, e.g. a0, a1e0,
a2e1s9.

The SAS support is brand new, and I'm sure I've got some things wrong. I
haven't been able to fully test SAS support yet because I don't have any bad
SAS disks.


*megatrace*

megatrace is a debugging program which can be used to trace PERC-related
ioctl() system calls that another program makes. You won't need that unless
you're trying to add features to megactl, or are exceptionally curious. This
program uses ptrace(2) and can inspect and modify data structures to help you
suss out what's going on in dellmgr and MegaCli.


*megarpt*

megarpt is a script I run in a cron job each night. It performs a health check
on all adapters, and emails any problems, along with the adapter configuration,
to root. It is handy to have the adapter configuration logged in case you need
to reconstruct adapter state in a catastrophic failure. There are various
scenarios involving hot spares and multiple drive failures where the adapter
configuration may not be quite what you thought it was. To use megarpt, copy
megarpt and megactl into /root/ and add a cron entry for /root/megarpt. Or
tweak as you see fit; there isn't much to it.

megarpt needs to be tweaked for megasasctl applications; it currently works
only with pre-SAS models.


*megasasrpt*

megasasrpt is just like megarpt, but for PERC5 adapters.


Building
--------

This software has been built successfully on RHEL 3, 4, and 5, and Debian Etch,
using the default gcc compiler, and Red Hat Linux 7.3 or RHEL 2.1 should work
as well. Simply run make in the src directory.


Device Nodes
------------

megactl and megasasctl require the existence of an appropriate device node in
order to communicate with adapters. For megactl, this device node should be
/dev/megadev0, and is created automatically by Dell's dellmgr program. You may
create it yourself by finding the major device number for megadev in
/proc/devices and creating a character device with that major number and minor
number 0. See the megarpt shell script if this is not clear. For megasasctl,
the device node is /dev/megaraid_sas_ioctl_node, and is created automatically
by LSI's MegaCli program. It should have the major number of megaraid_sas_ioctl
from /proc/devices and minor number 0. See the megasasrpt shell script for an
example of how to create this.


Limitations
-----------

Currently these programs only operate if built as 32-bit targets. On 64-bit
architectures, you therefore will need 32-bit compatibility libraries. This
should not require any special action on Red Hat, but on Debian you may need to
install a few things.