A collection of computer systems and programming tips that you may find useful.
Brought to you by Craic Computing LLC, a bioinformatics consulting company.

Tuesday, August 11, 2009

Creating and Attaching AWS EBS Volumes to an EC2 node

AWS Elastic Block Store (EBS) Volumes are arbitrary blocks of storage that can be mounted on EC2 nodes and used like regular filesystems. Here are the basic steps needed to set them up (and the associated gotchas to avoid).

1. Create a new EBS Volume using the AWS Management Console.
Know what size in GiB you want in advance and make sure you create it in the same Availability Zone as the EC2 node you want to attach it to.

2. Attach it to your EC2 instance
Do this from the Management Console. If you have more than one EC2 node running make sure you know the Instance ID, PLUS know which device you want to attach the Volume to. Your options are typically /dev/sdf, /dev/sdg, etc. Not sure what happens if you try to attach to a device that is already in use. It probably won't let you but I would rather just not go there...

Wait until the Console tells you the Volume is 'attached'. You might want to refresh the console to see this.

Now go to the EC2 node...

3. Create a Filesystem on the Volume
If (and ONLY if) this is a NEW Volume then you need to create a filesystem on it before you can do anything. Skip this step if the Volume has previously been mounted.

You can create different types of filesystem on the Volume. On linux the current recommended type is ext3. Here I am using /dev/sdh. Note that it asks if I want to use the entire device - Yes I do!
# mkfs -t ext3 /dev/sdh
mke2fs 1.40.4 (31-Dec-2007)
/dev/sdh is entire device, not just one partition!
Proceed anyway? (y,n) y
4. Create a mount point for the filesystem
Again, you may not need to do this if the EC2 node has previously mounted this Volume, but for a new setup you want to create a mount point under /mnt for the volume. For example:
# mkdir /mnt/myvolume
And then proceed to mount the Volume:
# mount /dev/sdh /mnt/myvolume
There should be no output from this command. Check that it is mounted with 'df' and then just 'cd' to it and start using the space.

5: To detach your Volume
First you need to un-mount it from your EC2 node while the node is running.
# umount /mnt/myvolume
Then you can detach it using the Management Console (and remember to Detach, not Delete!)
Only then should you shutdown your EC2 node, if you choose to do that.

6: Creating Snapshots of Volumes
You can create Snapshots of your Volume as backups or as a copy from which to clone other Volumes to attach to other nodes. (A given Volume can only be attached to one node at a time).
You can do this from the Management Console. You should do this on Volumes that are not attached to EC2 nodes so as to avoid anything being written to the Volume during Snapshot creation. You should do that but you can create a snapshot from a mounted Volume if you want to. How risky this is depends on what you are doing - never do this if it the Volume carries a live database, but it is OK if the Volume has a read-only dataset on it. Caveat emptor.

In their web pages, AWS talks about an EBS volume having higher performance than local disk. The impression I get from other blogs is that this is very variable and depends on what you use the disk for. I have not formed an opinion for my applications as yet.



w3bsmith said...

I wanted to add one more step to make it easier. On my instances I always make a change in my fstab (/etc/fstab) to have it mounted automatically on a reboot:

/dev/sdh /mnt/my_EBS_volume ext3 defaults 0 0

Then you can just do a:

mount /mnt/my_EBS_volume

paolooo said...

What happen if you will not format it?

Archive of Tips