debugfs :: handy utility to debug an ext2 or ext3 file system

Today I am learning a useful utility program named 'debugfs'. It is part of e2fsprogs package, an essential package containing axillary programs for ext2 and ext3 file system under Linux.

For a regular file, 'debugfs' can help you find an inode by any data block the file or dir entry is using. Then you can turn around and ask for the name of the inode. This could be handy when some mysterious files causing df and du to disagree whether the filie system is full, or the file system is corrupted or can't mounted to be accessed as usual. More advanced file system features are available too.

# to find what inode is claiming a given data block
# debugfs -R "icheck 12345" /dev/hda1
debugfs 1.35 (28-Feb-2004) Block Inode number 12345 340

# to find the file name given the inode number
# debugfs -R "ncheck 49153" /dev/hda1 debugfs 1.35 (28-Feb-2004) Inode Pathname
49153 /usr/share/locale/ar/LC_MESSAGES/libbonobo-2.0.mo

# Print the location of the inode data structure
# debugfs -R "imap /boot/vmlinuz-2.6.9-42.0.2.EL" /dev/hda1 debugfs 1.35 (28-Feb-2004)
Inode 557516 is part of block group 34
located at block 1114128, offset 0x0580

# to dump the direntry (filespec, per man page)
debugfs -R "dump -p /boot/vmlinuz- 2.6.9-42.0.2.EL /tmp/vmlinuz_dumped" /dev/hda1
# md5sum /boot/vmlinuz-2.6.9-42.0.2.EL /tmp/vmlinuz_dumped e5c536b539b5ffcaa03b22bd7fcc164a /boot/vmlinuz-2.6.9-42.0.2.EL e5c536b539b5ffcaa03b22bd7fcc164a /tmp/vmlinuz_dumped

# to get the contents of a file, assume the fs can't be mounted and accessed the usually way.
# debugfs -R "cat /etc/redhat-release" /dev/hda1
debugfs 1.35 (28-Feb-2004) CentOS release 4.4 (Final)

Noteworthy is, for files under /selinux ( a pseudo fs), it can find inode number associated with a data block. However, it couldn't find the file name for the very inode number.
# debugfs -R "ncheck 8" /dev/hda1 debugfs 1.35 (28-Feb-2004)
Inode Pathname 8
# find / -inum 8
/selinux/relabel
# ls -id /selinux/relabel
8 /selinux/relabel
# debugfs -R "icheck 4567" /dev/hda1 debugfs 1.35 (28-Feb-2004) Block Inode number
4567 8
# / is on /dev/hda1
/dev/hda1 8127400 6738524 1306308 84% /

There are a lot of powerful (and dangerous) features such as

• feature you can set or clear various file system features in the superblock
• freeb to mark data blocks as unallocated vs. setb
  
• freei to free the inode specified
• clri to clear the contents of the inode
• chroot to chroot to the directory
• find_free_block
• find_free_inode
• init_filesys to create an ext2 file system
• kill_file deallocate the file and its blocks. It doesn't remove any direntry to this inode. not ' rm' or 'unlink'.
• logdump to dump the ext3 journal
• modify_inode modify the contents of the inode structure
• ls/mkdir/mknod/rm/rmdir
  

'debugfs' starts interactively by default, unless you have '-R' to request one-time use only. A session would be like below:
# debugfs
debugfs 1.35 (28-Feb-2004)
debugfs: open /dev/hda1
debugfs: icheck 12345
Block Inode number
12345 340
debugfs: ncheck 340
Inode Pathname
340 /usr/X11R6/lib/xscreensaver/mountain
debugfs: close
debugfs: quit

Maxtor Personal Storage 3200 320 GB external USB 2.0 drive to stage backups under Linux

Today I bought a Maxtor Personal Storage 3200 320 GB (Model U01H320) from Circuit City. 320G with 8M cache for merely $159. They matched their online price by taking $20 off the $179 in-store price. Amazon.com is selling the same item on behalf of CompUSA for $169.
The drive will be used to stage backups at our co-lo. At work, we've waited too long for a budget to come through for a real NAS or a tape library/magzine/autoloader. It is one of those day-to-day challenges of small businesses which face the same challenges with much more limited resources. The daily full backup set now amounts to 12G, all compressed by gzip, my favorite tool.

From a RHEL 4.4 AS (CentOS 4.4, to be more exact) guest inside VMWare, the drive was detected as USB 1.1 (full speed) instead of USB 2.0. I assume it is a limitation of the VMWare emulation.
usb 1-1: new full speed USB device using address 2
Initializing USB Mass Storage driver...
scsi1 : SCSI emulation for USB Mass Storage devices
Vendor: Maxtor Model: 3200 Rev: 0341
Type: Direct-Access ANSI SCSI revision: 02 USB Mass Storage device found at 2
usbcore: registered new driver usb-storage
USB Mass Storage support registered.
SCSI device sda: 625142448 512-byte hdwr sectors (320073 MB) sda: assuming drive cache: write through
SCSI device sda: 625142448 512-byte hdwr sectors (320073 MB) sda: assuming drive cache: write through
sda: sda1 Attached scsi disk sda at scsi1, channel 0, id 0, lun 0

Pretty interesting to see it actually has NTFS as partition type. wonder how it would fly with MacOS out of the box?
Disk /dev/sda: 320.0 GB, 320072933376 bytes 255 heads, 63 sectors/track, 38913 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System /dev/sda1 * 1 38913 312568641 7 HPFS/NTFS
Since I am gonna use it under RHEL 4, I go ahead relabel the partition as 'Linux' and formatted it as EXT3.
/root# mke2fs -L E320G -O sparse_super,dir_index,filetype -T largefile4 -j /dev/sda1
mke2fs 1.35 (28-Feb-2004)
Filesystem label=E320G
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
76320 inodes, 78142160 blocks
3907108 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=79691776 2385 block groups
32768 blocks per group, 32768 fragments per group
32 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 4096000, 7962624, 11239424, 20480000, 23887872, 71663616 Writing inode tables: done
Creating journal (8192 blocks): done Writing superblocks and filesystem accounting information: done This filesystem will be automatically checked every 33 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
Ahh, I always forgot to reduce the percentage reserved for super user from the default 5% to 1%.
/dev/sda1 312538740 98368 296811940 1% /mnt
/root# tune2fs -m 1 /dev/sda1
tune2fs 1.35 (28-Feb-2004)
Setting reserved blocks percentage to 1 (781421 blocks)
/root# df -kv /mnt
Filesystem 1K-blocks Used Available Use% Mounted on /dev/sda1 312538740 98368 309314688 1% /mnt

step-by-step: how to set up a large ramdisk on CentOS 4.4/i386

On our new database server running CentOS 4.4 on a Dell PE 6850, a 2G ramdisk is needed. The option to use 'tmpfs' was considered and forfeited due to its potential of using disk swap.

• To make the ramdisk 2G apiece, a kernel parameter (ramdisk_size) needs to be passed ( grub.conf or lilo.conf) and a quick reboot is required to make it effective. The ramdisk by default is 16M in size.
  

Here is my grub.conf entry:

kernel /vmlinuz-2.6.9-42.ELsmp ro root=LABEL=/ console=ttyS0,57600n8 console=tty1 ramdisk_size=2097152

• To make the ramdisk accessible to the database after reboot, I cooked a chkconfig-compliant init script

---cut----x------/etc/init.d/ramdisk4syb-----x-----cut---------
#!/bin/bash
#
# $Id: rc.ramdisk4syb,v 1.00 2006/10/23 13:57:14 jer Exp $
#
# rc file for set up ramdisk fs for sybase to use for tempdb
#
#

# For Redhat-ish systems
#
# chkconfig: 2345 30 70
# processname: ramdisk4syb
# config: none
# description: set up ramdisk for sybase to use for tempdb
case $1 in
start)
test -d /mnt/ram1 || mkdir /mnt/ram1
mke2fs -m 1 -b 1024 /dev/ram1 >/tmp/ramdisk.mke2fs.log &&
mount -t ext2 /dev/ram1 /mnt/ram1 &&
chmod 770 /mnt/ram1 && chown sybase:dba /mnt/ram1
;;
stop)
umount /mnt/ram1
;;
*)
echo "Usage: $0 start|stop"
;;
esac

• to pin the init script to run levels

chkconfig --add ramdisk4syb

• to test the init script before reboot

/etc/rc3.d/S30ramdisk4syb start && df -k /mnt/ram1 && ls -ld /man/ram1

• to make all this effective by reboot.
  

init 6

• to verify the resultant ramdisk file system
  

df -k /mnt/ram1 && ls -ld /man/ram1

Much to my surprise, the ramdisk is not mounted after the reboot! The mke2fs log showed the mke2fs was successful. When I attempted to mount it manually, I got
# mount /dev/ram1 /mnt/ram1 mount: wrong fs type, bad option, bad superblock on /dev/ram1, or too many mounted file systems

In the log, I did notice the block size was 4K was instead of the 1K block size I observed with a 512M ramdisk.
Block size=4096 (log=2) Fragment size=4096 (log=2) 262144 inodes, 524288 blocks

Since the Sybase ASE uses 2K page size, I tried mke2fs using a 2K block. No joy! Mounting attempts were met with the same errors.

1K block size is known to have worked for 512M ramdisk, so I gave it a try. Bingo!
#mke2fs -b 1024 /dev/ram1 Block size=1024 (log=0) Fragment size=1024 (log=0) 262144 inodes, 2097152 blocks
Now we have a working ramdisk as below:
/dev/ram1 2063763 3086 1955820 1% /mnt/ram1
I don't see much of a point to waste space on 5% reserved for superuser. So, I tuned it down to 1%
# tune2fs -m 1 /dev/ram1
/dev/ram1 2063763 3086 2039706 1% /mnt/ram1
From the 'free' output, I am a bit surprised to see the 2G is not counted as 'used'.
total used free shared buffers cached Mem: 16627288 231236 16396052 0 48300 89240 -/+ buffers/cache: 93696 16533592 Swap: 16779884 0 16779884

bootparam man[7] page actually says " These days ram disks use the buffer cache, and grow dynamically." If so, ramdisk truly behaves similar to tmpfs at this point, except

• ramdisk's maximum size (the reserved?) needs a reboot to adjust, while tmpfs can be adjusted on the fly.
  
• ramdisk won't have swap in the picture at all. [good thing]
  

However, by growing dynamically, FWIW, it may not able to grow to its full reserved size when it needs it [bad bad thing], if physical RAM is used up by some real processes already. Questions yet to be answered:

• Does the 2.6 kernel or other series know or set ramdisk has higher priority on its claim to thus reserved memory?!
• why ext2 file system made with block sizes other than 1K failed to be mounted ? ramdisk is the only case here?