Wednesday, May 13, 2009

Xen: Move an LVM-based virtual machine to another host without shared storage between the two hosts

For those running Xen on servers with no back-end SAN, the following instructions detail the steps necessary to move an LVM-based virtual machine to a new physical host. There may be more elegant ways to achieve this, but this is what worked for me.

Environment
Ok, so let’s set the scene:

  • vmhost is a RHEL 5 server running several virtual machines using the Xen virtualization technology. Each virtual machine is allocated a 60 GB logical volume within an LVM volume group. One of these virtual machines, “win2k3vm”, needs to be moved to a new server.
  • new-vmhost is almost identical to vmhost, but has updated hardware and more memory. This is the server that “win2k3vm” will be moved to.


Migrate the Virtual Machine
1. Create a snapshot of virtual machine LVM volume. This can be run on a live virtual machine, but it is probably safer to perform a graceful shutdown of your virtual machine first.

[root@vmhost]# lvcreate -s -L 300m -n win2k3vm-snap VolGroup01

2. Export the snapshot of the guest VM to a file that can be moved between physical servers.

[root@vmhost]# dd if=/dev/VolGroup01/win2k3vm-snap of=/mnt/temp/win2k3vm.img bs=4096

3. Remove the snapshot LVM volume.

[root@vmhost]# lvremove /dev/VolGroup01/win2k3vm-snap

4. Copy the file to the new VM host. Make sure you have sufficient free drive space.

[root@vmhost]# scp /mnt/temp/win2k3vm.img user@new-vmhost:/mnt/temp/

5. On the new VM host, create an LVM volume that is at least as big as the guest VM file.

[root@new-vmhost]# lvcreate -n win2k3vm -L 60G VMGroup

6. Transfer the guest VM file to the new LVM volume.

[root@new-vmhost]# dd if=win2k3vm.img of=/dev/VMGroup/win2k3vm bs=4096

7. Copy the VM config file from the old VM host to the new VM host server.

[root@vmhost]# scp /etc/xen/win2k3vm user@new-vmhost:

8. Copy the file to the appropriate directory.

[usr@new-vmhost]$ sudo mv ~/win2k3vm /etc/xen/

Summary of Commands
[root@vmhost]# lvcreate -s -L 300m -n win2k3vm-snap VolGroup01
[root@vmhost]# dd if=/dev/VolGroup01/win2k3vm-snap of=/mnt/temp/win2k3vm.img bs=4096
[root@vmhost]# lvremove /dev/VolGroup01/win2k3vm-snap
[root@vmhost]# scp /mnt/temp/win2k3vm.img user@new-vmhost:/mnt/temp/
[root@new-vmhost]# lvcreate -n win2k3vm -L 60G VMGroup
[root@new-vmhost]# dd if=win2k3vm.img of=/dev/VMGroup/win2k3vm bs=4096
[root@vmhost]# scp /etc/xen/win2k3vm user@new-vmhost:
[usr@new-vmhost]$ sudo mv ~/win2k3vm /etc/xen/



Credit goes to: http://www.alethe.com/brad/2008/04/move-an-lvm-based-virtual-machine-to-another-host/ who these instructions were created by.

These instructions were perfect. Thanks Brad!

Tuesday, May 12, 2009

Xen: Converting a physical linux machine to xen virtual machine (fully virtualized) in Redhat Enterprise (64-bit)

I am running Redhat Enterprise Linux with xen virtualization. My server is a quad core 3ghz IBM x3550 server with 20gb of ram.

Today I had a need to move a standalone linux box into a xen vm. I had to keep the standalone machine completely intact as it was a build server which compiled code. The purpose of moving the build server to a xen VM was to test a build and see how long it took on the new VM machine compared to the older standalone server. The build server was fedora core 6 but I imagine this post applies to any machine, any software.


Preparing the standalone server
  1. Shut down the standalone machine
  2. Connect external USB disk to the system (use a disk that is larger then the drive in the system)
  3. Boot the standalone system with a linux boot cd (I used a ubuntu live recovery disk)
  4. Once the machine has booted the live cd, mount the external USB drive
  5. Use dd to make an image of the drive in the standalone server and output the image to a file on the USB drive: dd if=/dev/sda of=/mnt/usbdrive/standalone-server-image.dd (This will take some time)
  6. Once DD has completed, go ahead and shutdown the standalone machine, remove the USB drive, remove the Live CD and boot the machine back up to normal operation (if you will continue to use the standalone machine)
Preparing the Xen Server

Note: I am using LVM for my VM's. If you are not sure what this is, dont follow these instructions word for word

  1. Create a LV a little bit bigger then the original drive you imaged (my original was 80gb and I created a LV 100GB): # lvcreate -L100G -n newserver_vm /dev/data
  2. Copy the image of the standalone server to the new LV you just created:# dd if=/mnt/usb/standalone-server-image.dd of=/dev/data/newserver_vm
  3. Create a Xen configuration file for the new virtual machine (usually located in /etc/xen/
# /etc/xen/newserver_vm

import os, re
arch = os.uname()[4]

#I had to change the line below to match the path of where hvmloader was
kernel = "/usr/lib/xen/boot/hvmloader"

builder='hvm'

memory = 512
shadow_memory = 8

#change the line below to reflect the name of your vm
name = "newserver_vm"

#change the line below to match your network bridge. I copied and pasted this line from
#another working VM in xen that I started from scratch. Xen networking is a whole another
#blog!
vif = [ "mac=00:16:3e:15:f5:a6,bridge=xenbr0" ]

# I had to change the line below to match the patch of qemu-dm
device_model = '/usr/lib64/xen/bin/qemu-dm'
# boot on floppy (a), hard disk (c) or CD-ROM (d)
# default: hard disk, cd-rom, floppy
boot="dc"

sdl=0
vnc=1
vnclisten="0.0.0.0"
vncconsole=1
stdvga=0
serial='pty'

on_poweroff = 'destroy'
on_reboot = 'restart'
on_crash = 'restart'

#make sure this matches the LV you created and copied the standalone image to
disk = [ "phy:/dev/data/buildserver_vm,xvda,w" ]

Start the new VM

# xm create /etc/xen/newserver_vm

Once I started the new VM I used the gui xen manager in redhat. I could not get to the console from command line by starting the vm with "-c"


I hope this helps. This is made more as a reference then a step by step guide. Feel free to comment back with questions or suggestions.

Sunday, May 10, 2009

Sharepoint: Indexing pdf files in sharepoint moss 2007 windows 2008 64-bit

After following a couple different instructions I found on the web here is the real deal. Follow the instructions in this link:

http://www.adobe.com/special/acrobat/configuring_pdf_ifilter_for_ms_sharepoint_2007.pdf


Worked great for me. Let me know if you have problems.

Friday, May 8, 2009

Sharepoint: Moving library and list items within your library or list

In this post we will be talking about moving SharePoint library and list items within a library or list.

One great feature of ShaprePoint libraries and lists is that you can edit the library or list settings to allow you to create sub folders. I use this functionality for project management, general information management, etc.

Using the "Site Content and Structure" page found under "Site Settings" and "Site Administration", if you want to move an item within a library or list to a different folder within that library or list, SharePoint will give you the error "An error was encountered performing this operation". For some reason you cannot move items around within a list or library. There is no problem though moving items to different libraries or lists as long as they are the same type as the one you are working with.

Workaround: Since you can move items to different libraries and lists but not within, under your main site, create a new library or list called something like "Temp: Project Task" or "Temp: Document Library". When you have an item in a library or list that you want to move within to a different folder, first move the item to your temp library or list you created and then move the item from there to the desitnation you want.

Hope this helps.

Tuesday, May 5, 2009

Sharepoint: Making Sharepoint (MOSS 2007) easier to use in non-windows enviroments

In my opinion, Microsoft Sharepoint Server (MOSS 2007) is an amazing tool for enterprise document management, communications and collaboration. Sharepoint is not software, it is a platform for business. The tool is almost a "fix all" for corporate users if they are using a windows machine as a client. If you are not using windows but are using mac os x or linux, you will find that sharepoint is lacking some very basic features. This is due to all web browsers other then Internet explorer are labeled as class 2 browsers. Although I understand that Microsoft is going to fine tune MOSS for active x and Internet explorer, some of the features they left out of level 2 browsers are inexcusable. Out of the box, in order to use some of the most basic features of sharepoint, you will need to use Internet explorer. I have found a few workarounds that make sharepoint a product which can be used by cross platform clients. Here are some of those workarounds:

Problem: Some key feature of sharepoint are blogs, discussion groups and wikis. All of these features allow the user to create and edit formatted text using the built in "what you see is what you get" (WYSIWYG) editor... that is if you are using Internet explorer. If you are using any other web browser software, all you will see is raq html. This happens when replying to a discussion group post, editing a wiki page, creating a blog, posting a comment, etc. This makes it hard to believe that Microsoft thinks people will use sharepoint as an Internet web server.

Solution: RadEditor by Telerik. http://www.telerik.com/products/aspnet-ajax/sharepoint.aspx Telerik has a group of tools which come for a fee but you can download and use radeditor lite for free. It is a server add-on which will give you a wysiwyg editor for all browsers.

Problem: Accessing document libraries with non-IE browsers and non-win os. If you are not running windows you are forced to use a web browser to checkout, download, edit, upload and check in your documents and you can only upload one file at a time. If you are using Windows and Internet Explorer you can open a document library in an explorer window just like you would a network share. On any other OS you do not have this option from the actions menu in Sharepoint through the web interface.

Solution: Use a Webdav file client which is similar to a ftp client. In fact, some software provides both webdav and ftp connections. In mac os x I am using cyberduck. In linux I am still evaluating different ways to mount a webdav site (sharepoint document library). I am working on this using davfs but there seems to be a common problem that davfs sometimes does not support folders or files with spaces. I.E. "Shared Documents". I am still trying to figure out the exact details but I have recreated this problem in different linux distributions.

In order for you to use most webdav programs make sure your sharepoint site has basic authentication enabled as most programs wont use NTLM auth. This results in a login failed error when trying to authenticate. You can enable basic authentication under IIS admin, your sharepoint site and authentication. One other note that is helpful is to make sure your document library settings are set to have "Require documents to be checked out before they can be edited" set to "no". This way in your webdav client, you can easily drag and drop files into the document library. If you overwrite a file it will create a new version automatically as long as you have versioning enabled.

Problem: When using Microsoft Windows and Office 2007, you can work with sharepoint items directly from the Office programs. Some examples are editing sharepoint calendars within outlook or saving word documents directly to a sharepoint site without the use of 3rd party tools or a web browser. If you are on a Mac, Office programs have no interaction with sharepoint. This is very depressing especially that Entourage, Microsofts mac mail client comparable to Outlook, does not tie in at all to sharepoint. For linux users and open office there is no way working directly with sharepoint other then through the web interface or using 3rd party apps.

Solution: For mac os x, Microsoft Mac Business unit is working on a program call Document Collaboration Companion (DCC). Microsoft says DCC will provide:
  • Integrated (offline/local) access to Office Live Workspaces and SharePoint Websites
  • Document upload
  • Document check-out/in
  • SharePoint Workspace and Doc Library access & browsing
  • An independent application with File Menu integration
Right now DCC is in private beta and I cannot figure out how to get on the list. Even MacBU cant give us a status update. The last update they gave was that the beta would be available in Feb of 2009 which has come and past. The website for DCC is http://www.microsoft.com/mac/itpros/dcc.mspx


In conclusion, I hope this helps with some the larger incompatibilities of a base install of Sharepoint when working with non-windows client machines. There are still other setbacks that I have not mentioned here but will post updates on in the future. Sharepoint is an AMAZING tool for enterprise content management and collaboration. It's just too bad it has taken me days and a lot of research to put these solutions together to make it a viable option for my business which is 50% pc, 25% mac and 25% linux.

Comments are appreciated!

Links to things mentioned in this blog:

  • Cyberduck (note, only version 3.1.2 seems to work. 3.2 does not seem to work: http://update.cyberduck.ch/Cyberduck-3.1.2.dmg
  • Davfs: http://dav.sourceforge.net/
  • DCC: http://www.microsoft.com/mac/itpros/dcc.mspx
  • RadEditor Lite: http://www.telerik.com/community/free-products.aspx

Monday, May 4, 2009

Current and Upcoming Projects

I apologize for not keeping my blog up to date. What I hope to provide with this blogs is help with problems which took me a lot of work and research to fix. Blogs have been a very good source of information trying to find that "missing piece" when trying to fix a problem, bug or deal with an incompatability.

Here are some projects that I am currently working on will be working on in the future:
  • Sharepoint: Microsoft Office Sharepoint Server (Moss) 2007
  • Sharepoint compatibility across different OS's (non windows users, mac, linux, iphone, entourage, ical, webdav)
  • Wifi localization (RTLS)
  • Consolidating multiple linux servers using mixed distributions to Redhat Enterprise with Xen virtualization
  • Contention management/Warhouse Management Systems (WMS)
  • Remote management and monitoring of devices at multiple locations, specifically dealing with nagios and zenoss
  • Window Server 2008 and related technologies

That's it for now. I am going to start creating training for MOSS for my users so look forward to some posts on that.

F&*$ crappy web software

I just made a new blog that covered a bunch of current and new projects but blogger gave me an error, told me to go back and when I did, all of my content was gone. So since you waited this long for a new post please have patience waiting for a new post as mine has currently run out. I guess that's why they have the save button.