GNU Linux/LVM

From WhyAskWhy.org Wiki
Jump to: navigation, search




The following content is a Work In Progress and may contain broken links, incomplete directions or other errors. Once the initial work is complete this notice will be removed. Please contact me via Twitter with any questions and I'll try to help you out.


Adding a new disk (LVM) to a VMware Workstation Ubuntu VM

For all steps listed, I’m working with an Ubuntu 10.04 LTS virtual machine. It consists of a single disk (/dev/sda) that I didn't size properly when I originally created the VM.

Later I added a second disk (/dev/sdb, independent of snapshots) to hold audio files from ripping cds prior to transferring to them to a player. I did not add this disk to the existing logical volume.

Now we’re going add a third disk (/dev/sdc) to the VM and place it in the same volume group as the original disk to help alleviate the space problem as shown here:

df -h

Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/ubuntu-root
                      7.3G  6.5G  487M  94% /
none                  245M  264K  244M   1% /dev
none                  249M  1.1M  248M   1% /dev/shm
none                  249M   92K  249M   1% /var/run
none                  249M     0  249M   0% /var/lock
none                  249M     0  249M   0% /lib/init/rw
/dev/sda1             236M   54M  170M  24% /boot
/dev/sdb1              30G  497M   30G   2% /media/bucket
.host:/                79G   68G   12G  86% /mnt/hgfs


Adding the new disk

Go about adding the virtual disk through the method you're familiar with, keeping in mind that it will be combined with the existing LVM group already in place.


Creating the physical volume

I've found that you don't have to partition the disk prior to running this command.

pvcreate /dev/sdc

  Physical volume "/dev/sdc" successfully created


Listing the existing volume groups

vgdisplay

  --- Volume group ---
  VG Name               ubuntu
  System ID             
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  3
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                2
  Open LV               2
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               7.76 GiB
  PE Size               4.00 MiB
  Total PE              1986
  Alloc PE / Size       1986 / 7.76 GiB
  Free  PE / Size       0 / 0   
  VG UUID               t8CdgQ-IyfB-SqI5-C9Jc-M6w2-oNhe-1UuNZy


Extending the existing volume group

The ubuntu volume group was found, so let's add to that one.

vgextend ubuntu /dev/sdc

  Volume group "ubuntu" successfully extended


Using the new disk

From Unix and Linux System Administration Handbook, 4th Edition:

The logical volume manager doesn’t know anything about the contents of its volumes, so you must do your resizing at both the volume and filesystem levels. The order depends on the specific operation. Reductions must be filesystem-first, and enlargements must be volume-first. Don’t memorize these rules: just think about what’s actually happening and use common sense.


Resizing the logical volume

We added a 20 GB virtual disk and added the space to the volume group with the last command, but we haven't resized the logical volume (think of it as a container for a partition) to use any of the new space.

I've tried specifying the space I wanted to extend the logical volume in megabtyes, but I prefer specifying it via logical extents (the units of space allocation within a volume group) instead. I find this easier for some reason.

To do that, we need to get the number of logical extents available. We'll use vgdisplay for this.

  --- Volume group ---
  VG Name               ubuntu
  System ID             
  Format                lvm2
  Metadata Areas        2
  Metadata Sequence No  4
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                2
  Open LV               2
  Max PV                0
  Cur PV                2
  Act PV                2
  VG Size               27.75 GiB
  PE Size               4.00 MiB
  Total PE              7105
  Alloc PE / Size       1986 / 7.76 GiB
  Free  PE / Size       5119 / 20.00 GiB
  VG UUID               t8CdgQ-IyfB-SqI5-C9Jc-M6w2-oNhe-1UuNZy

What we're looking for is prefaced with Free PE / Size and and in this case it's 5119. So, that's one piece of information we need. Now, we need to know which logical volume we're resizing and we can get that information by using lvdisplay to show the logical volume names:

lvdisplay | grep 'LV Name'

  LV Name                /dev/ubuntu/root
  LV Name                /dev/ubuntu/swap_1

Well, we don't want to resize the swap_1 logical volume, so it's safe to say we want to resize ubuntu/root.

lvresize -l +5119 ubuntu/root

  Extending logical volume root to 27.37 GiB
  Logical volume root successfully resized


Resizing the file system

Now that the container has grown large enough to hold a bigger file system, we resize the file system to fill it.

resize2fs /dev/ubuntu/root

resize2fs 1.41.11 (14-Mar-2010)
Filesystem at /dev/ubuntu/root is mounted on /; on-line resizing required
old desc_blocks = 1, new_desc_blocks = 2
Performing an on-line resize of /dev/ubuntu/root to 7175168 (4k) blocks.
The filesystem on /dev/ubuntu/root is now 7175168 blocks long.


The result

Here's our file system/disk space usage:

df -h

root@ubuntu:~# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/ubuntu-root
                       27G  6.5G   20G  26% /
none                  245M  264K  244M   1% /dev
none                  249M  156K  249M   1% /dev/shm
none                  249M   88K  249M   1% /var/run
none                  249M     0  249M   0% /var/lock
none                  249M     0  249M   0% /lib/init/rw
/dev/sda1             236M   54M  170M  24% /boot
/dev/sdb1              30G  497M   30G   2% /media/bucket
.host:/                79G   68G   12G  86% /mnt/hgfs


Removing a disk from LVM on an Ubuntu system

This section pulls heavily from The Linux Documentation Project's "LVM HOWTO" [1]. It also assumes that the disk you want to remove is /dev/sdc. Substitute the device you wish to remove accordingly.


You are encouraged to backup your data before proceeding. Improperly resizing the root file system can cause your system to not boot and may make your data difficult to recover.


Distributing Old Extents to Existing Disks in Volume Group

To do this, you'll need to have enough free extents on the other disks in the volume group.


You do not have enough free extents on the other disks in the volume group

Like myself, you probably assigned 100% of the free extents on this disk and all other disks in the volume group, so you'll need to free some up to hold any data displaced by removing /dev/sdc from the system. [2]


Shrink file system and logical volume via a Rescue Disc

Before you can shrink the filesystem, you need to make sure that there is enough free space to do this. As mentioned previously, either offload content to another system, an external drive or simply uninstall/delete old content you no longer care about. Since in this example we're reverting back to the single disk in the volume group that we started out with, we know that we need to shrink the file system small enough to where we can then shrink the logical volume that surrounds it by 5119 extents.


File system

In my case I'm using Ubuntu 10.04.x Server LTS, so I used that install disc. If you have a desktop environment, use the desktop install cd. From the boot menu, choose Try Ubuntu without any change to your computer .

  1. I inserted the installation iso in the drive (or virtual cdrom) and booted from it.
  2. If you used the desktop install disc, you'll need to run sudo -s to gain root privileges and then apt-get install lvm2 to gain access to the lvm2 package and related utilities.
  3. If you use the server install disc, chose not to mount a file system (since we're going to be resizing the root file system).
  4. Open a shell. If you've booted from the server disc, you're already there. Otherwise, open a terminal window and enter vgchange -a y to activate volume groups.
    2 logical volume(s) in volume group "ubuntu" now active

We're now ready to attempt resizing the file system, [3] but before we do, let's view how much disk space is in use on /dev/ubuntu/root:

  1. root@ubuntu:~# mkdir /mnt/tmp
  2. root@ubuntu:~# mount /dev/ubuntu/root /mnt/tmp
  3. root@ubuntu:~# df -h
Filesystem            Size  Used Avail Use% Mounted on
aufs                  502M   44M  459M   9% /
none                  497M  260K  497M   1% /dev
/dev/sr0              686M  686M     0 100% /cdrom
/dev/loop0            658M  658M     0 100% /rofs
none                  502M  140K  501M   1% /dev/shm
tmpfs                 502M   20K  502M   1% /tmp
none                  502M   88K  502M   1% /var/run
none                  502M     0  502M   0% /var/lock
none                  502M     0  502M   0% /lib/init/rw
/dev/mapper/ubuntu-root
                       27G  5.6G   21G  22% /mnt/tmp

The important part is 5.6G. That means we're using less disk space in the logical volume than the original disk provided on its own; that is promising.

Before we try to resize the file system, we'll need to run e2fsck on it. If we don't do it now we'll be forced to later by resize2fs.

  1. umount /dev/ubuntu/root
    • Note that we unmounted the filesystem before running the filesystem check. Running a filesystem check against a mounted filesystem is a good for way "bad things" to happen
  2. e2fsck -f /dev/ubuntu/root
e2fsck 1.41.11 (14-Mar-2010)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/ubuntu/root: 236011/3588096 files (3.8% non-contiguous), 1565040/7175168 blocks

We saw that we're only using 5.6 GB of space on the logical volume /dev/ubuntu/root, but we need to know the size of the remaining disk once we remove /dev/sdc.

pvdisplay

  --- Physical volume ---
  PV Name               /dev/sda5
  VG Name               ubuntu
  PV Size               7.76 GiB / not usable 2.18 MiB
  Allocatable           yes (but full)
  PE Size               4.00 MiB
  Total PE              1986
  Free PE               0
  Allocated PE          1986
  PV UUID               A6eYdH-alVm-S2l3-vUI1-V1Ri-YWaV-TcP651
   
  --- Physical volume ---
  PV Name               /dev/sdc
  VG Name               ubuntu
  PV Size               20.00 GiB / not usable 4.00 MiB
  Allocatable           yes (but full)
  PE Size               4.00 MiB
  Total PE              5119
  Free PE               0
  Allocated PE          5119
  PV UUID               ba2TFV-zfoP-BNIh-7hCc-mc82-Vwie-Z7SfyO

As shown above, it is 7.76 GB. We've seen previously that it was displayed as 7.3 GB when running df -h, so let's play this safe and resize the file system to 6 GB.

resize2fs /dev/ubuntu/root 6G.

resize2fs 1.41.11 (14-Mar-2010)
Resizing the filesystem on /dev/ubuntu/root to 1572864 (4k) blocks.
The filesystem on /dev/ubuntu/root is now 1572864 blocks long.

Note that you can't use 6GB, but instead need to specify it as 6G.

Logical volume

Now that the file system has been shrunk, let's resize the logical volume using the lvresize. Before doing that, we need to decide how much we're going to shrink it.

Earlier we saw that /dev/sdc is 20 GB and has 5119 total extents. We know we want to shrink it by at least 20 GB/5119 extents, but we also need to shrink it a little further so we can move the contents of any used extents on dev/sdc to other disks in the volume (which for us is just /dev/sda5.

We resized the file system on /dev/ubuntu/root to 6 GB earlier and we know that the total size for /dev/sda5 is at least 7.3 GB, so it should be safe to resize the logical volume dev/ubuntu/root to say, 6.3 GB.

Let's recap:

Total logical volume size:

root@ubuntu:~# lvdisplay /dev/ubuntu/root | grep Size

  LV Size                27.37 GiB

Physical volume size for /dev/sda5:

root@ubuntu:~# pvdisplay /dev/sda5 | grep 'PV Size'

  PV Size               7.76 GiB / not usable 2.18 MiB

I don't know a way to show it, but earlier we resized the file system to 6 GB without any errors, so we'll use that number.

  27.37 GB (logical volume size)
-  6.00 GB (file system size)
-------------------------------
= 21.37 GB (shrink by number)

Now, to be on the safe side, I'm going to shrink by an even 21 GB.

lvresize -L -21G ubuntu/root

  WARNING: Reducing active logical volume to 6.37 GiB
  THIS MAY DESTROY YOUR DATA (filesystem etc.)
Do you really want to reduce root? [y/n]: y
  Reducing logical volume root to 6.37 GiB
  Logical volume root successfully resized


Let's see how many extents we freed up.

root@ubuntu:~# vgdisplay | grep 'Free PE'

  Free  PE / Size       5376 / 21.00 GiB

So, 257 extents free (5376 total - 5119 that we're about to remove) for any remaining content on /dev/sdc. Let's hope it's enough.


Now let's try to move any remaining content on /dev/sdc to other disks in the ubuntu volume group, which in our case is just /dev/sda5.

root@ubuntu:~# pvmove /dev/sdc

  No data to move for ubuntu

That's fantastic. Now we are clear to remove the disk from the volume.

root@ubuntu:~# vgreduce ubuntu /dev/sdc

  Removed "/dev/sdc" from volume group "ubuntu"

Super. Now if you'll remember, we were pretty conservative earlier with resizing the file system and logical volume. Now that /dev/sdc has been removed, you can stretch the logical volume /dev/ubuntu/root back out followed by the file system on that volume.


To resize the logical volume, let's see how many extents are free in the volume group.


root@ubuntu:~# vgdisplay | grep 'Free PE'

  Free  PE / Size       257 / 1.00 GiB

We can expand the logical volume /dev/ubuntu/root by 257 extents.

root@ubuntu:~# lvresize -l +257 /dev/ubuntu/root

  Extending logical volume root to 7.38 GiB
  Logical volume root successfully resized

Now that we've resized the logical volume, let's expand the file system to fill it. We do this by running resize2fs without a size option.

root@ubuntu:~# resize2fs /dev/ubuntu/root

resize2fs 1.41.11 (14-Mar-2010)
Resizing the filesystem on /dev/ubuntu/root to 1670144 (4k) blocks.
The filesystem on /dev/ubuntu/root is now 1670144 blocks long.

Let's double-check and make sure everything is good so far.

e2fsck -f /dev/ubuntu/root

e2fsck 1.41.11 (14-Mar-2010)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/ubuntu/root: 236011/835584 files (4.7% non-contiguous), 1477742/1670144 blocks

That's it, now cross your fingers and reboot.


You have enough free extents on the other disks in the volume group

If you choose to have your root file system in one logical volume and the extra disk placed in another logical volume, you can follow the directions below to resize the logical volume and filesystem so you can safely remove the disk.

pvmove /dev/sdc

pvmove -- moving physical extents in active volume group "ubuntu"
pvmove -- WARNING: moving of active logical volumes may cause data loss!
pvmove -- do you want to continue? [y/n] y
pvmove -- 5119 extents of physical volume "/dev/sdc" successfully moved


Remove the unused disk

vgreduce ubuntu /dev/sdc

vgreduce -- doing automatic backup of volume group "ubuntu"
vgreduce -- volume group "ubuntu" successfully reduced by physical volume:
vgreduce -- /dev/sdc


References

  1. LVM-HOWTO - Removing an Old Disk
  2. Shrinking and growing a logical volume in Fedora 15
  3. Did you remember to backup?


Related

I did not use these to assist with writing this guide, but believe the information is worth reviewing.