Resizing LVM Volumes in Linux under vmware or native

This is a great set of instructions by David Maphis on how to resize an LVM volume in Linux. I just used these instructions to expand my linux virtual machine hard drive under vmware fusion. OS was Fedora core 6 using lvm. Vmware I used was VMware fusion for the mac but this applies to any linux system using lvm.

Under my vm settings in VMware fusion I expanded the disk to be larger and then followed these instructions for the linux side of things. Obviously the vm has to be shutdown before you mess with the hd size in vmware but all of the linux steps I did on the live system within Xwindows. LV's are great!

Here are David's instructions. You can skip his intro if you are not using vmware server. I injected some notes which you will find in bold.

URL to original pdf: http://www.google.com/url?sa=t&source=web&ct=res&cd=1&url=http%3A%2F%2Fcommunities.vmware.co

Publish Post

m%2Fservlet%2FJiveServlet%2FpreviewBody%2F4900-102-2-3019%2FResizing%2520LVM%2520Volumes%2520in%2520Linux.pdf%3Bjsessionid%3D9DC6B8A9615CEBD28FE94412BA638CE1&ei=VJcVSsioAZio8gSaxKjHAg&usg=AFQjCNGSDpxwdH7ojtEpCwiGR_Yueod_Kg&sig2=oJ4vPtV31iEBovG4xMFuHw

By David Maphis 4/29/2008

In this example I will changed the size of my root partition without
affecting any data. Since this is a VM, I can extend my vmdk with the
“vmkfstools” command. To do this I will have to shutdown the VM to release
the lock on the file then resize it. Power up when complete.

Once this is done, make a partition that uses the available space you wish to
add to the LVM volume using “parted”. It has an interactive shell, so just
enter “parted” in a shell:

parted
(parted) print
Model: VMware Virtual disk (scsi)
Disk /dev/sda: 10.7GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number Start End Size Type File system Flags
1 32.3kB 206MB 206MB primary ext3 boot
2 206MB 8587MB 8382MB primary lvm


So I have from 8588MB to 10.7GB available for a new partition. For this
example I’m only using up to 9588MB. Your numbers will obviously vary.

So now let’s make a partition from the new space:

(parted) mkpart primary 8588 9588

(parted) print
Model: VMware Virtual disk (scsi)
Disk /dev/sda: 10.7GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number Start End Size Type File system Flags
1 32.3kB 206MB 206MB primary ext3 boot
2 206MB 8587MB 8382MB primary lvm
3 8587MB 9588MB 1001MB primary

You can now see that we have allocated a total of 1001MB of space to
partition 3 or /dev/sda3.

Next we need to create a Physical Volume (PV) with “pvcreate”. LVM also has
an interactive shell, so type “lvm” in a shell prompt:

lvm
lvm> pvcreate /dev/sda3
Physical volume "/dev/sda3" successfully created

lvm> pvs
PV VG Fmt Attr PSize PFree
/dev/sda2 VolGroup00 lvm2 a- 7.78G 32.00M
/dev/sda3 lvm2 -- 954.44M 954.44M

BFISK NOTE: I did not see sda3 when I ran the pvs command but I was able to proceed with the directions.

Now there is a partition that is about 1GB of space available on PV
/dev/sda3.

Now we need to extend the Volume Group (VG) to include the new PV with the
“vgextend” command:

lvm> vgextend VolGroup00 /dev/sda3
Volume group "VolGroup00" successfully extended
lvm> pvs
PV VG Fmt Attr PSize PFree
/dev/sda2 VolGroup00 lvm2 a- 7.78G 32.00M
/dev/sda3 VolGroup00 lvm2 a- 928.00M 928.00M

BFISK NOTE: I dont think I saw sda3 here either but continued with directions

By virtue of the fact that we used the same name as the existing VG, we have
now included this PV in the VG.

Extend the Logical Volume (LV) to include the new space with “lvextend”. To
determine that size of the extents we want to add, we first need to view the
VG:

lvm> vgdisplay VolGroup00
--- Volume group ---
VG Name VolGroup00
System ID
Format lvm2
Metadata Areas 2
Metadata Sequence No 4
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 2
Open LV 2
Max PV 0
Cur PV 2
Act PV 2
VG Size 8.69 GB
PE Size 32.00 MB
Total PE 278
Alloc PE / Size 248 / 7.75 GB
Free PE / Size 30 / 960.00 MB
VG UUID Ll4Bg4-jIFV-lfut-z4Ae-JTfH-0Hzp-Zf9RjM

Looking at this we can see that the total “Free PE” is 30. We will use this
in the following command:

lvm> lvextend -l+30 /dev/VolGroup00/LogVol00
Extending logical volume LogVol00 to 8.19 GB
Logical volume LogVol00 successfully resized

BFISK NOTE: I am not sure what the +30 is. It's not MB and it's not GB. Maybe %? I dunno, I just issued the command until it told me there was no space left. At that point I used -l+10 untill I ran out and then +1, etc until I had no empty space left.

lvm> lvs
LV VG Attr LSize Origin Snap% Move Log Copy%
LogVol00 VolGroup00 -wi-ao 8.19G
LogVol01 VolGroup00 -wi-ao 512.00M

So there you are, you have now extended your LV. Now we have to clue the OS
in what is happening.
We need to expand the file system to make the new space visible to the OS
with resize2fs (Fedora, CentOS) or ext2online (RedHat)

[root@labserver02 dev]# df -h /
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
7.1G 1.3G 5.4G 20% /

[root@labserver02 dev]# resize2fs
resize2fs 1.40.2 (12-Jul-2007)
Usage: resize2fs [-d debug_flags] [-f] [-F] [-p] device [new_size]

[root@labserver02 dev]# resize2fs /dev/VolGroup00/LogVol00
resize2fs 1.40.2 (12-Jul-2007)
Filesystem at /dev/VolGroup00/LogVol00 is mounted on /; on-line resizing
required
old desc_blocks = 1, new_desc_blocks = 1
Performing an on-line resize of /dev/VolGroup00/LogVol00 to 2146304 (4k)
blocks.
The filesystem on /dev/VolGroup00/LogVol00 is now 2146304 blocks long.

[root@labserver02 dev]# df -h /
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
8.0G 1.3G 6.3G 17% /

Xen: Move an LVM-based virtual machine to another host without shared storage between the two hosts

For those running Xen on servers with no back-end SAN, the following instructions detail the steps necessary to move an LVM-based virtual machine to a new physical host. There may be more elegant ways to achieve this, but this is what worked for me.

Environment
Ok, so let’s set the scene:

• vmhost is a RHEL 5 server running several virtual machines using the Xen virtualization technology. Each virtual machine is allocated a 60 GB logical volume within an LVM volume group. One of these virtual machines, “win2k3vm”, needs to be moved to a new server.
• new-vmhost is almost identical to vmhost, but has updated hardware and more memory. This is the server that “win2k3vm” will be moved to.

Migrate the Virtual Machine
1. Create a snapshot of virtual machine LVM volume. This can be run on a live virtual machine, but it is probably safer to perform a graceful shutdown of your virtual machine first.

[root@vmhost]# lvcreate -s -L 300m -n win2k3vm-snap VolGroup01

2. Export the snapshot of the guest VM to a file that can be moved between physical servers.

[root@vmhost]# dd if=/dev/VolGroup01/win2k3vm-snap of=/mnt/temp/win2k3vm.img bs=4096

3. Remove the snapshot LVM volume.

[root@vmhost]# lvremove /dev/VolGroup01/win2k3vm-snap

4. Copy the file to the new VM host. Make sure you have sufficient free drive space.

[root@vmhost]# scp /mnt/temp/win2k3vm.img user@new-vmhost:/mnt/temp/

5. On the new VM host, create an LVM volume that is at least as big as the guest VM file.

[root@new-vmhost]# lvcreate -n win2k3vm -L 60G VMGroup

6. Transfer the guest VM file to the new LVM volume.

[root@new-vmhost]# dd if=win2k3vm.img of=/dev/VMGroup/win2k3vm bs=4096

7. Copy the VM config file from the old VM host to the new VM host server.

[root@vmhost]# scp /etc/xen/win2k3vm user@new-vmhost:

8. Copy the file to the appropriate directory.

[usr@new-vmhost]$ sudo mv ~/win2k3vm /etc/xen/

Summary of Commands
[root@vmhost]# lvcreate -s -L 300m -n win2k3vm-snap VolGroup01
[root@vmhost]# dd if=/dev/VolGroup01/win2k3vm-snap of=/mnt/temp/win2k3vm.img bs=4096
[root@vmhost]# lvremove /dev/VolGroup01/win2k3vm-snap
[root@vmhost]# scp /mnt/temp/win2k3vm.img user@new-vmhost:/mnt/temp/
[root@new-vmhost]# lvcreate -n win2k3vm -L 60G VMGroup
[root@new-vmhost]# dd if=win2k3vm.img of=/dev/VMGroup/win2k3vm bs=4096
[root@vmhost]# scp /etc/xen/win2k3vm user@new-vmhost:
[usr@new-vmhost]$ sudo mv ~/win2k3vm /etc/xen/


Credit goes to: http://www.alethe.com/brad/2008/04/move-an-lvm-based-virtual-machine-to-another-host/ who these instructions were created by.

These instructions were perfect. Thanks Brad!

Xen: Converting a physical linux machine to xen virtual machine (fully virtualized) in Redhat Enterprise (64-bit)

I am running Redhat Enterprise Linux with xen virtualization. My server is a quad core 3ghz IBM x3550 server with 20gb of ram.

Today I had a need to move a standalone linux box into a xen vm. I had to keep the standalone machine completely intact as it was a build server which compiled code. The purpose of moving the build server to a xen VM was to test a build and see how long it took on the new VM machine compared to the older standalone server. The build server was fedora core 6 but I imagine this post applies to any machine, any software.


Preparing the standalone server

1. Shut down the standalone machine
2. Connect external USB disk to the system (use a disk that is larger then the drive in the system)
   
3. Boot the standalone system with a linux boot cd (I used a ubuntu live recovery disk)
4. Once the machine has booted the live cd, mount the external USB drive
5. Use dd to make an image of the drive in the standalone server and output the image to a file on the USB drive: dd if=/dev/sda of=/mnt/usbdrive/standalone-server-image.dd (This will take some time)
6. Once DD has completed, go ahead and shutdown the standalone machine, remove the USB drive, remove the Live CD and boot the machine back up to normal operation (if you will continue to use the standalone machine)

Preparing the Xen Server

Note: I am using LVM for my VM's. If you are not sure what this is, dont follow these instructions word for word

1. Create a LV a little bit bigger then the original drive you imaged (my original was 80gb and I created a LV 100GB): # lvcreate -L100G -n newserver_vm /dev/data
   
2. Copy the image of the standalone server to the new LV you just created:# dd if=/mnt/usb/standalone-server-image.dd of=/dev/data/newserver_vm
   
3. Create a Xen configuration file for the new virtual machine (usually located in /etc/xen/

# /etc/xen/newserver_vm

import os, re
arch = os.uname()[4]

#I had to change the line below to match the path of where hvmloader was
kernel = "/usr/lib/xen/boot/hvmloader"

builder='hvm'

memory = 512
shadow_memory = 8

#change the line below to reflect the name of your vm
name = "newserver_vm"

#change the line below to match your network bridge. I copied and pasted this line from
#another working VM in xen that I started from scratch. Xen networking is a whole another
#blog!
vif = [ "mac=00:16:3e:15:f5:a6,bridge=xenbr0" ]

# I had to change the line below to match the patch of qemu-dm
device_model = '/usr/lib64/xen/bin/qemu-dm'
# boot on floppy (a), hard disk (c) or CD-ROM (d)
# default: hard disk, cd-rom, floppy
boot="dc"

sdl=0
vnc=1
vnclisten="0.0.0.0"
vncconsole=1
stdvga=0
serial='pty'

on_poweroff = 'destroy'
on_reboot = 'restart'
on_crash = 'restart'

#make sure this matches the LV you created and copied the standalone image to
disk = [ "phy:/dev/data/buildserver_vm,xvda,w" ]

Start the new VM

# xm create /etc/xen/newserver_vm

Once I started the new VM I used the gui xen manager in redhat. I could not get to the console from command line by starting the vm with "-c"


I hope this helps. This is made more as a reference then a step by step guide. Feel free to comment back with questions or suggestions.

Sharepoint: Indexing pdf files in sharepoint moss 2007 windows 2008 64-bit

After following a couple different instructions I found on the web here is the real deal. Follow the instructions in this link:

http://www.adobe.com/special/acrobat/configuring_pdf_ifilter_for_ms_sharepoint_2007.pdf


Worked great for me. Let me know if you have problems.

Sharepoint: Moving library and list items within your library or list

In this post we will be talking about moving SharePoint library and list items within a library or list.

One great feature of ShaprePoint libraries and lists is that you can edit the library or list settings to allow you to create sub folders. I use this functionality for project management, general information management, etc.

Using the "Site Content and Structure" page found under "Site Settings" and "Site Administration", if you want to move an item within a library or list to a different folder within that library or list, SharePoint will give you the error "An error was encountered performing this operation". For some reason you cannot move items around within a list or library. There is no problem though moving items to different libraries or lists as long as they are the same type as the one you are working with.

Workaround: Since you can move items to different libraries and lists but not within, under your main site, create a new library or list called something like "Temp: Project Task" or "Temp: Document Library". When you have an item in a library or list that you want to move within to a different folder, first move the item to your temp library or list you created and then move the item from there to the desitnation you want.

Hope this helps.

Sharepoint: Making Sharepoint (MOSS 2007) easier to use in non-windows enviroments

In my opinion, Microsoft Sharepoint Server (MOSS 2007) is an amazing tool for enterprise document management, communications and collaboration. Sharepoint is not software, it is a platform for business. The tool is almost a "fix all" for corporate users if they are using a windows machine as a client. If you are not using windows but are using mac os x or linux, you will find that sharepoint is lacking some very basic features. This is due to all web browsers other then Internet explorer are labeled as class 2 browsers. Although I understand that Microsoft is going to fine tune MOSS for active x and Internet explorer, some of the features they left out of level 2 browsers are inexcusable. Out of the box, in order to use some of the most basic features of sharepoint, you will need to use Internet explorer. I have found a few workarounds that make sharepoint a product which can be used by cross platform clients. Here are some of those workarounds:

Problem: Some key feature of sharepoint are blogs, discussion groups and wikis. All of these features allow the user to create and edit formatted text using the built in "what you see is what you get" (WYSIWYG) editor... that is if you are using Internet explorer. If you are using any other web browser software, all you will see is raq html. This happens when replying to a discussion group post, editing a wiki page, creating a blog, posting a comment, etc. This makes it hard to believe that Microsoft thinks people will use sharepoint as an Internet web server.

Solution: RadEditor by Telerik. http://www.telerik.com/products/aspnet-ajax/sharepoint.aspx Telerik has a group of tools which come for a fee but you can download and use radeditor lite for free. It is a server add-on which will give you a wysiwyg editor for all browsers.

Problem: Accessing document libraries with non-IE browsers and non-win os. If you are not running windows you are forced to use a web browser to checkout, download, edit, upload and check in your documents and you can only upload one file at a time. If you are using Windows and Internet Explorer you can open a document library in an explorer window just like you would a network share. On any other OS you do not have this option from the actions menu in Sharepoint through the web interface.

Solution: Use a Webdav file client which is similar to a ftp client. In fact, some software provides both webdav and ftp connections. In mac os x I am using cyberduck. In linux I am still evaluating different ways to mount a webdav site (sharepoint document library). I am working on this using davfs but there seems to be a common problem that davfs sometimes does not support folders or files with spaces. I.E. "Shared Documents". I am still trying to figure out the exact details but I have recreated this problem in different linux distributions.

In order for you to use most webdav programs make sure your sharepoint site has basic authentication enabled as most programs wont use NTLM auth. This results in a login failed error when trying to authenticate. You can enable basic authentication under IIS admin, your sharepoint site and authentication. One other note that is helpful is to make sure your document library settings are set to have "Require documents to be checked out before they can be edited" set to "no". This way in your webdav client, you can easily drag and drop files into the document library. If you overwrite a file it will create a new version automatically as long as you have versioning enabled.

Problem: When using Microsoft Windows and Office 2007, you can work with sharepoint items directly from the Office programs. Some examples are editing sharepoint calendars within outlook or saving word documents directly to a sharepoint site without the use of 3rd party tools or a web browser. If you are on a Mac, Office programs have no interaction with sharepoint. This is very depressing especially that Entourage, Microsofts mac mail client comparable to Outlook, does not tie in at all to sharepoint. For linux users and open office there is no way working directly with sharepoint other then through the web interface or using 3rd party apps.

Solution: For mac os x, Microsoft Mac Business unit is working on a program call Document Collaboration Companion (DCC). Microsoft says DCC will provide:

• Integrated (offline/local) access to Office Live Workspaces and SharePoint Websites
• Document upload
• Document check-out/in
• SharePoint Workspace and Doc Library access & browsing
• An independent application with File Menu integration

Right now DCC is in private beta and I cannot figure out how to get on the list. Even MacBU cant give us a status update. The last update they gave was that the beta would be available in Feb of 2009 which has come and past. The website for DCC is http://www.microsoft.com/mac/itpros/dcc.mspx


In conclusion, I hope this helps with some the larger incompatibilities of a base install of Sharepoint when working with non-windows client machines. There are still other setbacks that I have not mentioned here but will post updates on in the future. Sharepoint is an AMAZING tool for enterprise content management and collaboration. It's just too bad it has taken me days and a lot of research to put these solutions together to make it a viable option for my business which is 50% pc, 25% mac and 25% linux.

Comments are appreciated!

Links to things mentioned in this blog:

• Cyberduck (note, only version 3.1.2 seems to work. 3.2 does not seem to work: http://update.cyberduck.ch/Cyberduck-3.1.2.dmg
• Davfs: http://dav.sourceforge.net/
• DCC: http://www.microsoft.com/mac/itpros/dcc.mspx
• RadEditor Lite: http://www.telerik.com/community/free-products.aspx

Current and Upcoming Projects

I apologize for not keeping my blog up to date. What I hope to provide with this blogs is help with problems which took me a lot of work and research to fix. Blogs have been a very good source of information trying to find that "missing piece" when trying to fix a problem, bug or deal with an incompatability.

Here are some projects that I am currently working on will be working on in the future:

• Sharepoint: Microsoft Office Sharepoint Server (Moss) 2007
• Sharepoint compatibility across different OS's (non windows users, mac, linux, iphone, entourage, ical, webdav)
• Wifi localization (RTLS)
• Consolidating multiple linux servers using mixed distributions to Redhat Enterprise with Xen virtualization
• Contention management/Warhouse Management Systems (WMS)
• Remote management and monitoring of devices at multiple locations, specifically dealing with nagios and zenoss
• Window Server 2008 and related technologies

That's it for now. I am going to start creating training for MOSS for my users so look forward to some posts on that.

F&*$ crappy web software

I just made a new blog that covered a bunch of current and new projects but blogger gave me an error, told me to go back and when I did, all of my content was gone. So since you waited this long for a new post please have patience waiting for a new post as mine has currently run out. I guess that's why they have the save button.