Sunday, October 3, 2010

Migrating CentOS servers from Linode to Amazon EC2, or what I did at the end

The guys at 619Cloud have built an interesting script to migrate Linux servers across cloud providers. Although it was created specifically for CentOS 5.X servers, the mechanisms it uses are simple enough that should work with most Linux distributions.

The script is very simple, as it mainly uses rsync to efficiently copy everything (excluding some special folders) over SSH to the target server. Because of this, the kernel version of the target server must be the same, as otherwise you could have unexpected issues.

Some bits related to the journey:

  • You can use rsync with the .pem key files you get when setting the new EC2 box. In fact you can use the following rsync option to set any other SSH parameters: --rsh "ssh –i pathToPemFile".
  • Make sure you do chmod 400 to the key file, otherwise SSH will reject using it.
  • You can use scp if you just need to copy a couple small files. From a Windows box you can use pscp from Putty.
  • If you do use the script / rsync to copy from root, make sure you don’t have any network locations mounted in the file system. Ignore this and you’ll get a lot more than you asked for. Alternatively you can explicitly exclude them, like the script does for some system dirs.

I tried using the script on a migration from Linode to Amazon Elastic Compute Cloud (EC2) and ran into issues where the newly created target boxes would lock down. It could have been because of not using the same kernel version (there wasn’t a matching ami), because of Amazon’s kernel modifications or maybe something as simple as network/firewall related configurations on the box that prevented it to work on the new environment.

After being faced with the above, I decided it was time to avoid the issue altogether and try a different approach. This is what I did at the end after I had an EC2 box (with ebs) that I was having trouble getting up, but already had all the data I needed (from the use of the script):

  • Stopped the EC2 instance
  • Detached the ebs volume
  • Launched a new EC2 instance
  • Attached the ebs volume
  • Moved the data out of the ebs volume, as I didn’t want all the rest of the system or application files it contained
  • Quickly installed the services we had in the original box and pointed them to/imported the original data

2 comments:

  1. Why are you moving off of Linode?

    ReplyDelete
  2. @michael the client involved had part of their services in EC2 and the other at Linode, with the earlier having larger + more important services.

    There was some $ advantage to running these particular services at Linode, but the introduction of Amazon's micro instances changed that.

    It was mostly about consolidating it in one provider, rather than either being better. It made sense for their scenario, but no full blown study on EC2 vs. Linode was done.

    ReplyDelete