24 Oct 2007

Nontrivial Hard Drive Partitioning for Linux

When looking to build an PC to replace my old 1.3 GHz AMD Athalon XP, I created a list of requirements (really just desires) and a price limit of $600. A few of those requirements were applied to storage including:

  1. The machine must support at least RAID1
  2. There must be plenty of storage space for multimedia with room for expansion.

I spent endless hours on the web researching hardware, looking for that perfect balance between features and cost. I finally settled on a motherboard that supports 4 SATA 3.0GB/s connectors (supports four higher performance drives), 2 PATA (4 older drives, ideal for reusing my existing DVD and CD drives), and built in RAID support.

Only after I started trying to assemble my machine did I learn that one of my requirements was misguided. Linux does not need special hardware for RAID support. If I had known this when choosing a motherboard, I could have saved tens of dollars, or at leased used that additional money to buy a better part elsewhere in the machine. Interestingly, the software based RAID is a better solution for my needs, as it is not restricted to the hard drive level. Instead, RAID arrays can be constructed with hard drive partitions, allowing for more flexible configurations even when there are fewer hard drives to work with. I bet RAID arrays could be built with media other than hard drives, such as flash drives.

After discovering software based RAID and the mdadm command, I configured my two 250GB hard drives (surprisingly affordable from http://newegg.com) to have identically sized (to the block) partitions as follows:

/dev/hda1 ext3 140MB - /boot
/dev/hda5 swap   3GB - with option pri=1
/dev/hda6 jfs   40GB - / (root directory)
/dev/hda7 jfs  200GB - /home in RAID1 array /dev/md0

/dev/hdb1 ext3 140MB - Not Used
/dev/hdb5 swap   3GB - with option pri=1 
/dev/hdb6 jfs   40GB - Not Used
/dev/hdb7 jfs  200GB - /home in RAID1 array /dev/md0

I was keeping /dev/hda6 open in case I wanted to play with a different linux distribution or a whole other operating system (e.g. http://JNode.org).

I am not sure what happened to the 2,3 and 4 partitions when I created these, as this was a fresh install. I know that one of them (I think it is the 4 partition) is used to create the extended partition for which the later numbered partitions are contained. Maybe the Gparted partition editor that comes with Ubuntu is smart enough to reserve the smaller numbers for the maximum number of partitions outside the extended partition.

I made sure that the /etc/fstab had the priority option set to 1 for both swap partitions, giving the swap space a performance boost due to twice the IO bandwidth of two hard drives. Since the swap partitions have the swap load distributed equally, this is similar to a RAID0 configuration. Keep in mind the performance gain exists only because the swap space is split between multiple hard drives that can perform input and output operations in parallel. If multiple swap spaces were configured on the same hard drive or on multiple hard drives attached to the same old-style IDE (PATA) connector, this would not have a positive effect.

Then, I used the mdadm command line tool to build a RAID1 array (/dev/md0) for the home directory. Since RAID1 mirrors the same data between hard drives, this added degree of assurance that I will not loose my important data in the event of a hard drive crash.

After a few months, I decided to order a digital TV tuner (the hdhomerun) for my PC hoping to use it like a TIVO on steroids. After recording one TV show, I realized my seemingly large 200GB of storage space really wasn’t that large. HTDV records at roughly 2MB per second. At this rate, I could record at most 28 hours of video before exhausting the space in my home directory. Considering this machine is also my personal computer for doing software development, image editing and just about everything else I do outside of work, I was afraid that the system could start to feel a bit crowded. I bought a half terabyte of hard drive space to get away from that constraint, and yet I was faced with it again.

Most of the data I will be storing on my hard drive will not be sensitive in nature. Software can be downloaded or reinstalled, and TV shows will have reruns. Yes, obtaining all that information will be time consuming and painful, but loosing custom code I’ve spent countless hours on, documentation, family photos and the like would be unrecoverable and a shame. Certainly, redundancy beyond a single machine is important, and for the most valuable data I do use removable media, but I haven’t gotten to the point of a disaster recovery plan for my personal files yet. Keeping this in mind, I reconfigured my hard drive partitioning to reduce my home directory to a 40GB RAID1 array, allowing me to combine most of my hard drive space into a single RAID0 partition for large file storage. This is what I am presently working with:

/dev/hda1 ext3 140MB - /boot
/dev/hda5 swap   3GB - with option pri=1
/dev/hda6 jfs   40GB - / (root directory)
/dev/hda7 jfs   40GB - /home in RAID1 array /dev/md0
/dev/hda8 jfs  160GB - /storage in RAID0 array /dev/md1

/dev/hdb1 ext3 140MB - Not Used
/dev/hdb5 swap   3GB - with option pri=1 
/dev/hdb6 jfs   40GB - /home in RAID1 array /dev/md0
/dev/hdb7 jfs  200GB - /storage in RAID0 array /dev/md1

The RAID1 array requires that the partitions match in size exactly since one partition mirrors the other exactly, otherwise excess space in the larger partition is simply never used. The RAID0 array combines the two partitions into one larger partition. With RAID0, the partitions would ideally be the same size, allowing the workload to be evenly distributed across both drives (read performance), but this is not a firm requirement. I would be surprised if I tax my system to the point where I notice a performance difference. After all, this machine is not a high performance network server. I have unfortunately lost the flexibility of the unused 40GB partition, as it is now used in conjunction with part of the larger /dev/hda7 partition for the home drive. What this does do, however, is give me the redundant storage I want for my sensitive data while allowing a large 360GB partition for less valuable data. Using the df command now gives me something that looks like the following.

matlikj@hydra:~$ df
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/sda6             40923460  14805828  26117632  37% /
varrun                 1037780       112   1037668   1% /var/run
varlock                1037780         0   1037780   0% /var/lock
udev                   1037780       116   1037664   1% /dev
devshm                 1037780         0   1037780   0% /dev/shm
lrm                    1037780     34696   1003084   4% /lib/modules/2.6.22-14-generic/volatile
/dev/sda1               194442     40590    143813  23% /boot
/dev/sdb1               194442      5664    178739   4% /media/sdb1
/dev/md0              40923460  15309264  25614196  38% /home
/dev/md1             358920168     47940 358872228   1% /storage

Getting to this point not at all easy, at least not with the automatic assembly feature of mdadm. After several hours of creating, destroying and reformatting the RAID arrays, I eventually gave up on the automatic assembly feature. I’m sure I’m missing something about how the auto assemble feature works, but I finally decided to take the easy way out and maintain the mdadm.conf file. I just have one more file that needs to be carefully maintained when a software update comes through. Since this post has gotten quite long, I will write the details on how the partitions were created later.