From Wiki
< GNU Linux
Revision as of 20:01, 15 July 2012 by Deoren (talk | contribs) (Added numbered bullets.)
Jump to: navigation, search

Adding a new disk (LVM) to a VMware Workstation Ubuntu VM

For all steps listed, I’m working with an Ubuntu 10.04 LTS virtual machine. It consists of a single disk (/dev/sda) that I didn't size properly when I originally created the VM.

Later I added a second disk (/dev/sdb, independent of snapshots) to hold audio files from ripping cds prior to transferring to them to a player. I did not add this disk to the existing logical volume.

Now we’re going add a third disk (/dev/sdc) to the VM and place it in the same volume group as the original disk to help alleviate the space problem as shown here:

df -h

Filesystem            Size  Used Avail Use% Mounted on
                      7.3G  6.5G  487M  94% /
none                  245M  264K  244M   1% /dev
none                  249M  1.1M  248M   1% /dev/shm
none                  249M   92K  249M   1% /var/run
none                  249M     0  249M   0% /var/lock
none                  249M     0  249M   0% /lib/init/rw
/dev/sda1             236M   54M  170M  24% /boot
/dev/sdb1              30G  497M   30G   2% /media/bucket
.host:/                79G   68G   12G  86% /mnt/hgfs

Adding the new disk

Go about adding the virtual disk through the method you're familiar with, keeping in mind that it will be combined with the existing LVM group already in place.

Creating the physical volume

I've found that you don't have to partition the disk prior to running this command.

pvcreate /dev/sdc

  Physical volume "/dev/sdc" successfully created

Listing the existing volume groups


  --- Volume group ---
  VG Name               ubuntu
  System ID             
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  3
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                2
  Open LV               2
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               7.76 GiB
  PE Size               4.00 MiB
  Total PE              1986
  Alloc PE / Size       1986 / 7.76 GiB
  Free  PE / Size       0 / 0   
  VG UUID               t8CdgQ-IyfB-SqI5-C9Jc-M6w2-oNhe-1UuNZy

Extending the existing volume group

The ubuntu volume group was found, so let's add to that one.

vgextend ubuntu /dev/sdc

  Volume group "ubuntu" successfully extended

Using the new disk

From Unix and Linux System Administration Handbook, 4th Edition:

The logical volume manager doesn’t know anything about the contents of its volumes, so you must do your resizing at both the volume and filesystem levels. The order depends on the specific operation. Reductions must be filesystem-first, and enlargements must be volume-first. Don’t memorize these rules: just think about what’s actually happening and use common sense.

Resizing the logical volume

We added a 20 GB virtual disk and added the space to the volume group with the last command, but we haven't resized the logical volume (think of it as a container for a partition) to use any of the new space.

I've tried specifying the space I wanted to extend the logical volume in megabtyes, but I prefer specifying it via logical extents (the units of space allocation within a volume group) instead. I find this easier for some reason.

To do that, we need to get the number of logical extents available. We'll use vgdisplay for this.

  --- Volume group ---
  VG Name               ubuntu
  System ID             
  Format                lvm2
  Metadata Areas        2
  Metadata Sequence No  4
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                2
  Open LV               2
  Max PV                0
  Cur PV                2
  Act PV                2
  VG Size               27.75 GiB
  PE Size               4.00 MiB
  Total PE              7105
  Alloc PE / Size       1986 / 7.76 GiB
  Free  PE / Size       5119 / 20.00 GiB
  VG UUID               t8CdgQ-IyfB-SqI5-C9Jc-M6w2-oNhe-1UuNZy

What we're looking for is prefaced with Free PE / Size and and in this case it's 5119. So, that's one piece of information we need. Now, we need to know which logical volume we're resizing and we can get that information by using lvdisplay to show the logical volume names:

lvdisplay | grep 'LV Name'

  LV Name                /dev/ubuntu/root
  LV Name                /dev/ubuntu/swap_1

Well, we don't want to resize the swap_1 logical volume, so it's safe to say we want to resize ubuntu/root.

lvresize -l +5119 ubuntu/root

  Extending logical volume root to 27.37 GiB
  Logical volume root successfully resized

Resizing the file system

Now that the container has grown large enough to hold a bigger file system, we resize the file system to fill it.

resize2fs /dev/ubuntu/root

resize2fs 1.41.11 (14-Mar-2010)
Filesystem at /dev/ubuntu/root is mounted on /; on-line resizing required
old desc_blocks = 1, new_desc_blocks = 2
Performing an on-line resize of /dev/ubuntu/root to 7175168 (4k) blocks.
The filesystem on /dev/ubuntu/root is now 7175168 blocks long.

The result

Here's our file system/disk space usage:

df -h

root@ubuntu:~# df -h
Filesystem            Size  Used Avail Use% Mounted on
                       27G  6.5G   20G  26% /
none                  245M  264K  244M   1% /dev
none                  249M  156K  249M   1% /dev/shm
none                  249M   88K  249M   1% /var/run
none                  249M     0  249M   0% /var/lock
none                  249M     0  249M   0% /lib/init/rw
/dev/sda1             236M   54M  170M  24% /boot
/dev/sdb1              30G  497M   30G   2% /media/bucket
.host:/                79G   68G   12G  86% /mnt/hgfs

References (in the order I used them)

Removing a disk from LVM on an Ubuntu system

This section pulls heavily from The Linux Documentation Project's "LVM HOWTO" [1]. It also assumes that the disk you want to remove is /dev/sdc. Substitute the device you wish to remove accordingly.

You are encouraged to backup your data before proceeding. Improperly resizing the root file system can cause your system to not boot and may make your data difficult to recover.

Distributing Old Extents to Existing Disks in Volume Group

To do this, you'll need to have enough free extents on the other disks in the volume group.

You do not have enough free extents on the other disks in the volume group

Like myself, you probably assigned 100% of the free extents on this disk and all other disks in the volume group, so you'll need to free some up to hold any data displaced by removing /dev/sdc from the system. [2]

Shrink file system and logical volume via a Rescue Disc

Before you can shrink the filesystem, you need to make sure that there is enough free space to do this. As mentioned previously, either offload content to another system, an external drive or simply uninstall/delete old content you no longer care about. Since in this example we're reverting back to the single disk in the volume group that we started out with, we know that we need to shrink the file system small enough to where we can then shrink the logical volume that surrounds it by 5119 extents.

File system

In my case I'm using Ubuntu 10.04.x Server LTS, so I used that install disc. If you have a desktop environment, use the desktop install cd. From the boot menu, choose Try Ubuntu without any change to your computer .

  1. I inserted the installation iso in the drive (or virtual cdrom) and booted from it.
  2. If you used the desktop install disc, you'll need to run sudo -s to gain root privileges and then apt-get install lvm2 to gain access to the lvm2 package and related utilities.
  3. If you use the server install disc, chose not to mount a file system (since we're going to be resizing the root file system).
  4. Open a shell. If you've booted from the server disc, you're already there. Otherwise, open a terminal window and enter vgchange -a y to activate volume groups.
    2 logical volume(s) in volume group "ubuntu" now active

We're now ready to attempt resizing the file system, [3] but before we do, let's view how much disk space is in use on /dev/ubuntu/root:

  1. root@ubuntu:~# mkdir /mnt/tmp
  2. root@ubuntu:~# mount /dev/ubuntu/root /mnt/tmp
  3. root@ubuntu:~# df -h
Filesystem            Size  Used Avail Use% Mounted on
aufs                  502M   44M  459M   9% /
none                  497M  260K  497M   1% /dev
/dev/sr0              686M  686M     0 100% /cdrom
/dev/loop0            658M  658M     0 100% /rofs
none                  502M  140K  501M   1% /dev/shm
tmpfs                 502M   20K  502M   1% /tmp
none                  502M   88K  502M   1% /var/run
none                  502M     0  502M   0% /var/lock
none                  502M     0  502M   0% /lib/init/rw
                       27G  5.6G   21G  22% /mnt/tmp

The important part is 5.6G. That means we're using less disk space in the logical volume than the original disk provided on its own; that is promising.

Let's try to resize the file system:

Resume work here

  1. resize2fs /dev/ubuntu/root -M
    Please run 'e2fsck -f /dev/ubuntu/root' first
  2. e2fsck -f /dev/ubuntu/root
  3. Once the disk passes the e2fsck, run resize2fs /dev/ubuntu/root -M. This may take a while. [4]
Logical volume

Now that the file system has been shrunk, let's resize the logical volume using the lvresize. Before doing that, we'll want to verify the extents used by the disk we want to remove and then shrink the logical volume by that many extents.

  1. pvdislay /dev/sdc | grep 'Total PE'
    Total PE            5119
  2. lvresize -l -5119 ubuntu/root
  3. Accept any warnings and continue, because after all, you did do a backup first (right?).
  4. resize2fs /dev/ubuntu/root (run without a size option so it will expand to the size it was before adding /dev/sdc)
  5. e2fsck -f /dev/ubuntu/root
  6. pvmove ubuntu /dev/sdc
    No extents available for allocation
  7. That probably shouldn't be ignored, but I went ahead and ran vgreduce ubuntu /dev/sdc and got
    Removed "/dev/sdc" from volume group "ubuntu"
  8. Reboot back into the main OS to see if it implods.

I skipped a step somewhere and will need to go back and fix the directions here

Removing Physical Disk from LVM via pvmove

You have enough free extents on the other disks in the volume group

If you choose to have your root file system in one logical volume and the extra disk placed in another logical volume, you can follow the directions below to resize the logical volume and filesystem so you can safely remove the disk.

pvmove /dev/sdc

pvmove -- moving physical extents in active volume group "ubuntu"
pvmove -- WARNING: moving of active logical volumes may cause data loss!
pvmove -- do you want to continue? [y/n] y
pvmove -- 5119 extents of physical volume "/dev/sdc" successfully moved

Remove the unused disk

vgreduce ubuntu /dev/sdc

vgreduce -- doing automatic backup of volume group "ubuntu"
vgreduce -- volume group "ubuntu" successfully reduced by physical volume:
vgreduce -- /dev/sdc


  1. LVM-HOWTO - Removing an Old Disk
  2. Shrinking and growing a logical volume in Fedora 15
  3. Did you remember to backup?
  4. This is probably not the most efficient way to handle it, but it should work fine, even if it takes longer than it would if we used (NEW_SIZE = CURRENT_SIZE - DISK_TO_REMOVE_SIZE)