iotop, your best friend when LED of HD stays on

The server is slow to hell, badly accessing remotely, a simple ls it’s enougth to take a lot of seconds running… openning a program, no way… lot of minutes, time to take a cup of coffee.

You, don’t understanding what’s goin on, go get a look at the server, sometimes wishing to reset it, then, surprisely see that the HD LED look’s like the POWER one, but, Red…

In situations like that where are a lot of disk activities, come the question “What now?”, how to verify what is doing such a lot of disk access?

I will show you how to install iotop on Linux Red Hat / CentOS servers.

For that, there’s a very handy tool, iotop, it have a very nice CLI interface that show the I/O data. Utilize the very own values that kernel pass throught and show us in a very fancy interface :)

• http://freshmeat.net/projects/iotop

Needs to have Python >= 2.5 and kernel Linux >= 2.6.20 with those options enabled

• CONFIG_TASKSTATS
• CONFIG_TASK_DELAY_ACCT
• CONFIG_TASK_IO_ACCOUNTING

Pre-requisites

• CentOS / RedHat 5.3 minimal installation (Portuguese only)
• Compiling a new kernel (2.6.30.5) in Linux CentOS 5.3 / Red Hat 5.3

wget http://guichaz.free.fr/iotop/files/iotop-0.3.1.tar.bz2
tar jxvf iotop-0.3.1.tar.bz2 -C /usr/src
cd /usr/src/iotop-0.3.1/

To execute the program without installing it, just call via CLI:

./iotop.py

In case that the following error comes:

Traceback (most recent call last):
  File "./iotop.py", line 8, in ?
    from iotop.ui import main
  File "/usr/src/iotop-0.3.1/iotop/ui.py", line 12, in ?
    from iotop.data import find_uids, TaskStatsNetlink, ProcessList
  File "/usr/src/iotop-0.3.1/iotop/data.py", line 12, in ?
    from iotop import ioprio, vmstat
  File "/usr/src/iotop-0.3.1/iotop/ioprio.py", line 1, in ?
    import ctypes

It’s because you haven’t the python installed os it isn’t the 2.5 version. It’s necessary to install it.

Installing Python 2.5 on Red Hat 5.3 / CentOS 5.3

Red Hat 5 / CentOS 5 comes with python 2.4 by default. It isn’t cool to simply install the new version over the default one, instead, I’m disponibilizing here the RPMs of python 2.5 for Red Hat 5 / CentOS 5 that will be installed in his own directory /usr/bin/python25, so, the scripts that Yum and others scripts made use of python 2.4 won’t get hurt.

• python25-2.5.1-fogonacaixadagua2.i386.rpm wget http://www.virtualxp.org/downloads/python25/RPMS/i386/python25-2.5.1-fogonacaixadagua2.i386.rpm
• python25-devel-2.5.1-fogonacaixadagua2.i386.rpm wget http://www.virtualxp.org/downloads/python25/RPMS/i386/python25-devel-2.5.1-fogonacaixadagua2.i386.rpm
• python25-libs-2.5.1-fogonacaixadagua2.i386.rpm wget http://www.virtualxp.org/downloads/python25/RPMS/i386/python25-libs-2.5.1-fogonacaixadagua2.i386.rpm
• python25-test-2.5.1-fogonacaixadagua2.i386.rpm wget http://www.virtualxp.org/downloads/python25/RPMS/i386/python25-test-2.5.1-fogonacaixadagua2.i386.rpm

The news is, always that you need to use the new version of python 2.5 you’ll have to change the interpretor PATH that call python25. It’s better than the whole system going unstable, isn’t it? ;)

Executing iotop.py

With python25 installed, it’s time to modify the script iotop.py and change the interpreter on the first line, from:

#!/usr/bin/python

to

#!/usr/bin/python25

Ok, time to run the script

./iotop.py

If any of the pre-requisites won’t be safisfied, a message like this one will show informing what is missing

Could not run iotop as some of the requirements are not met:
- Python >= 2.5 for AF_NETLINK support: Found
- Linux >= 2.6.20 with I/O accounting support (CONFIG_TASKSTATS, CONFIG_TASK_DELAY_ACCT, CONFIG_TASK_IO_ACCOUNTING): Not found

Python 2.5 it’s OK, but kernel isn’t.

Simple, compiling kernel with those options:

To compile the new kernel, there’s a post here that explains it, here I’ll show you just which options to be considered, compile and install the kernel it’s with you now ;)

• Compiling a new kernel (2.6.30.5) in Linux CentOS 5.3 / Red Hat 5.3 (Portuguese only)

General setup  --->
...
[*] Export task/process statistics through netlink (EXPERIMENTAL)
[*]   Enable per-task delay accounting (EXPERIMENTAL)
[*]   Enable extended accounting over taskstats (EXPERIMENTAL)
[*]     Enable per-task storage I/O accounting (EXPERIMENTAL)
...

Kernel 2.6.30.5 (rpm) with process statistics ready to be used with iotop

• kernel-2.6.30.5fogonacaixadagua-1.i386.rpm wget http://www.virtualxp.org/downloads/kernel-2.6.30.5/RPMS/i386/IOTASKS/kernel-2.6.30.5fogonacaixadagua-1.i386.rpm

To install

rpm -ivh kernel-2.6.30.5fogonacaixadagua-1.i386.rpm

in case you already installed this kernel before, will need to generate again an image initrd and use the flag --force with rpm -ivh --force command.

iotop running

Installing iotop

Into the directory you extracted iotop-0.3.1 run the command
./setup.py install

cd /usr/src/iotop-0.3.1
./setup.py install

How it was installed python 2.5, if you run /usr/bin/iotop the following error will be gracefuly show to you

Traceback (most recent call last):
File "/usr/bin/iotop", line 4, in 

import pkg_resources
ImportError: No module named pkg_resources

To resolve that one, it’s quite simple, into directory where you extracted iotop, in this case /usr/src/iotop-0.3.1 copy the entire directory iotop to /usr/lib/python-2.5

cd /usr/src/iotop-0.3.1
cp -Rp iotop /usr/lib/python2.5/

And after that, edit the file /usr/bin/iptop and modify the line of interpreter

from

#!/usr/bin/python

to

#!/usr/bin/python25

Ok, iotop running!