In this walkthrough, we are going to create an EC2 server, using the pre-existing public images that amazon has so kindly set up for us.

1. Make sure that you are set up and ready to preform ec2 commands and manage images/instances. If you need to set this up, the available documentation is avalable here
2. As soon as you are ready to rock out with your new ec2 toys, you are going to want to create a keypair. This keypair will be used to attach to a public image at runtime, ensuring your, and only your access to it. You can do this with the following command:
   ec2-add-keypair [name-of-keypair]
   it helps to name it something specific to the image that you are building. IE if you are building a database backup, you may name the keypair “db-backup”
   this command will output something very similar to:

   KEYPAIR db-backup 1f:51:ae:28:bf:89:e9:d8:1f:25:5d:37:2d:7d:b8:ca:9f:f5:f1:6f
   -----BEGIN RSA PRIVATE KEY-----
   MIICXAIBAAKBgQC2i/Sgs5BGGd4sunpYQfEkcprgzP9M/hnVJTc1j0nZBeIE2JBuLRSNoqkO7Gw8
   nBcdNptaLedzqN8t78jGkX1TPWVAKJTfxRSvU/oViGJaRqIBar0Mpc/wC27kyzHezUNS5+mvONb3
   4h/j2EZwDLY75Uxrpka0aN6OkvyIP5gYMQIDAQABAoGAOKH65tBOdjEYSHAh/LeYhGI5wnxWyCAd
   C49cLXWix32XvUEircu2kKpiIIsgmT0jvqBuWe/b2noNo0a81z3TzzRYyLvn5J8mUlL6a8nsssQ3
   xCHkGM+SE7ZzfBS5WUkbh5Exd3ZXKfCJvJW6auOzJ581JB5yUNbqixWzHuQGGAECQQDwq4LQoyb5
   OVSpZwSy+GW/p0yRsqRp89ECNQ+hySGBjkSXBcbt75C+5ebo88/V2V4QOGGa0T0tMsMgKTJ8oukh
   AkEAwiyoFM0Zwk0Os3rBZ8PyZoNW5e5SBwrEbLRv4JCaNiQme0ighsDr2bL/nGLI7p13g22+9REM
   i/WAmsln50H9EQJASMun6tGepT2pFQBbFIM7y4egCmXdg0rDSoagLtB2eQh+SKvvquKOhp9lg8rT
   b5yq7f8PztNBTN2Q1baAVeC04QJAGgN5kS/ZH5rLOWhcuNYbh3hZD/zZqG/c2ONjiaZVwqMdNK8K
   MoNuFYBRllX1rWITPNxbFOHv2GBPlm0dKnJAwQJBAOgwjgLY3UpXFX9ZvG4RGEYgfui49Ffz10CH
   5sSZpsFYn42E6a2NUJeL4hTzfbGTQ8iCIVjOXFH/9XLTDCNQEPM=
   -----END RSA PRIVATE KEY-----

3. Once you spit out the keypair, you have to save it somewhere. It doesnt really matter where, but use your favorite text editor to create a file (in you current directory or in /keypair is usually a nice spot) name the file: id_rsa-db-backup-keypair or something memmorable related to the keypair you created. It will all make sense soon I promise.
   IMPORTANT: Copy into that text file ONLY what appears between -----BEGIN RSA PRIVATE KEY----- and -----END RSA PRIVATE KEY-----
4. You are now ready to boot a public image. Use ec2-describe-images to list the available public images. It should output something like:

   IMAGE   ami-5bae4b32    ec2-public-images/getting-started.manifest      206029621532    available public
   IMAGE   ami-68ae4b01    ec2-public-images/fedora-core4-base.manifest    206029621532    available public
   IMAGE   ami-69ae4b00    ec2-public-images/fedora-core4-apache-mysql.manifest    206029621532    available public
   IMAGE   ami-6dae4b04    ec2-public-images/fedora-core4-apache.manifest  206029621532    available public
   IMAGE   ami-6fae4b06    ec2-public-images/fedora-core4-mysql.manifest   206029621532    available public
   IMAGE   ami-aca84dc5    ubuntu-base/image.manifest      554263365884    available       private

5. Choose your target, and boot the crap out of it
   ec2-run-instances ami-69ae4b00 -k gsg-db-backup
   notice the "-k gsg-db-backup" this is the name of the keypair we created, not the name of the file we stored the keypair in. That comes in later.
6. Your instance will now be pending for a minute or three, and then will be running. Use ec2-describe-instances to get an update on whats going on. It should output something like (after the instance boots of course):

   RESERVATION     r-fea54097  495219933132   EC2
   INSTANCE        i-10a64379  ami-69ae4b00   domU-12-34-31-00-00-05.usma1.compute.amazonaws.com EC2    running   gsg-db-backup

   If you do not see the name of your keypair after the instance, then you did something wrong, so you may as well shut down the instance and start over.

7. We now want to authorize port 22 for use with ssh (you can authorize other ports for webserving and whatever else later… this is the important one to ensure you can get into the thing.) use the following command:
   ec2-authorize default -P tcp -p 22
8. Lets go ahead and get into that puppy. To do this, we are going to ssh, but not use a password, so we are going to call upon the keypair file that we created. Your command should look something like:
   ssh -i id_rsa-db-backup root@domU-12-34-31-00-00-05.usma1.compute.amazonaws.com
   If you stored the keypair file elsewhere, you may have to give the full path:
   ssh -i /path_to_keypair/id_rsa-db-backup root@domU-12-34-31-00-00-05.usma1.compute.amazonaws.com
   If all goes smoothly, and everything was done right, you will now be logged in to a running instance as root. Yeah!
9. This step is easy, simply add all of the users you want, install whatever packages you want, basically make that server as personal as you want. View Jack’s post here for further details. Just make sure that you change passwords and add users so that you can ssh it later.
10. This part is important, make sure that you upload your private key to the running instance, so that when we bundle, you have the proper permissions. Trust me, realizing you messed up after transfering a few gigs for an hour is not fun.
    scp pk-XXXXXXXXXXXXXXXXXXXXXXXXXXXX.pem root@domU-12-34-31-00-00-05.usma1.compute.amazonaws.com:/tmp
    This is going in the /tmp directory so that it is not carried over at bundle.
11. Now we are going to bundle this server into a nice little package that you can boot anytime, anywhere. Thanks to amazon, the ec2 tools to do so are already installed. So use the following command:
    ec2-bundle-vol -k /path_to_your_key/pk-XXXXXXXXXXXXXXXXXXXXXXXXXXXX.pem -s 1000 -u [your_user_id]
    This command is going to take 5 or 10 minutes, and does not have any output while doing so, so just be patient. When it is done, it should output the image.manifest file, along with all of its part buddies.
12. These files should all be stored in the /tmp directory. We want to scp them over to the machine that you were working off of originally. This is easier to scp from the local machine, especially if it is behind some sort of firewall with no external ip. Like so:
    scp user@domu-12-31-33-00-03-d7.usma1.compute.amazonaws.com:/tmp/* /path_on_local_machine/
    Again, this is a lot of information, (a gig to be exact) it took me 20 minutes or so to transfer. This time you are awarded the luxury of progress however, so either watch the water boil, or get a cup of coffee.
13. With the image and image parts are on your local machine, uploading them to your s3 cloud should be as easy as it always is. Just make sure you use the proper keys on upload:
    ec2-upload-bundle -b my-bucket -m image.manifest -a [your_key_id] -s [secret_key_id]
    You can get your key id, and secret key id from your account section when logged into aws.amazon.com
    This is going to take another 20 minutes or so… so if you already got coffee… you should try a donut.
14. It would be a shame not to authorize this image for use in EC2. It’s easy:
    ec2-register your-bucket/image.manifest
    Told you.
15. Use ec2-describe-images and you should see your brand new image ready and waiting to be booted. I would suggest shutting down the public instance before proceeding. If you really dont want to, the worst that will happen is you will be double charged, and may get confused if you see two seemingly identicle instances running.
    All that is left to do is boot it. You already know how to do that, just ec2-run-instances ami-61a54008
    Use ec2-describe-instances and it should say pending
    When it is done booting, voila! You just created an instance from a public image!

1. Ensure that environment variables are correct, as per the previous post.
   If the command echo $JAVA_HOME outputs the path of a java binary, you should be all set.
2. ec2-describe-images will describe the various images on your S3 storage that are available:

   IMAGE   ami-5bae4b32    ec2-public-images/getting-started.manifest      206029621532    available       public
   IMAGE   ami-68ae4b01    ec2-public-images/fedora-core4-base.manifest    206029621532    available       public
   IMAGE   ami-69ae4b00    ec2-public-images/fedora-core4-apache-mysql.manifest    206029621532    available       public
   IMAGE   ami-6dae4b04    ec2-public-images/fedora-core4-apache.manifest  206029621532    available       public
   IMAGE   ami-6fae4b06    ec2-public-images/fedora-core4-mysql.manifest   206029621532    available       public
   IMAGE   ami-aca84dc5    ubuntu-base/image.manifest      554263365884    available       private

   In this case, “ami-aca84dc5” is a custome image that we made.

3. ec2-run-instances ami-aca84dc5 will create an instance of the “ami-aca84dc5” image.
   It will take a few minutes to get started up. You can use ec2-describe-instances to check the status:

   RESERVATION     r-58d53031      554263365884    default
   INSTANCE        i-4bb45022      ami-aca84dc5    domU-12-31-33-00-01-6E.usma1.compute.amazonaws.com      running
   RESERVATION     r-5cd53035      554263365884    default
   INSTANCE        i-4fb45026      ami-aca84dc5            pending

4. ec2-authorize default -P tcp -p 22 -s [your ip address]/32 will update the firewall rules to allow you to SSH into the server from your IP address only. The “/32″ is the IP mask.
5. To open up ports for the general internet public, you’ll need to use:
   ec2-authorize default -P tcp -p 80
   In this case, we opened up TCP port 80 for a webserver.
6. If you ever want to shut down an instance, use it’s instance id from the ec2-describe-instances command:
   ec2-terminate-instances i-4bb45022

REMEMBER! When you terminate an instance, you will lose ALL of its data. Even if you start a new instance from your image, you will be starting from scratch!

Amazon Web Services has recently made available a beta service called the Amazon Elastic Compute Cloud which allows for rapid deployment of virtual servers. As some of the documentation was wrong and some of the resource materials were incomplete, we thought that it might be useful to create a walkthrough for the entire process of developing server images, transferring them to Amazon’s Simple Storage Service, and subsequently deploying instances of those server images.

First, you’ll need to sign up for an Amazon Web Services account. Once you have an account, you will be provided with a link to your “AWS Access Identifiers” page. From here, you’ll need an X.509 Certificate which you can create and download at the bottom of the page. You will get a public certificate and a private key. I saved mine to:
~/ec2/auth/cert-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.pem
~/ec2/auth/pk-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.pem
I also saved my Access Key ID and Secret Access Key to
~/ec2/auth/access_key_id
~/ec2/auth/secret_access_key

You’ll also need your Account Number, which can be found at the top of your Account Summary. Of course, I saved mine to:
~/ec2/auth/account_number
Next, there are two sets of API tools that you’ll need to download from the Resource Center: The “Amazon EC2 AMI Tools” which will help you create, bundle, and upload your custom server images, and the “Amazon EC2 Command-Line Tools” which allow you to manage your uploaded AMI images as well as instances of those images. Note: The AMI Tools are currently only available as an RPM. I installed my Command Line Tools under
~/ec2/api
These tools tend to be a bit finicky, as they need environment variables set in order to run correctly. Customize the following according to your setup, and then you can either paste it into your shell, or paste it into the bottom of your ~/.bash_profile if you’d like them set every time you login:

export JAVA_HOME=/usr/lib/jvm/java-1.5.0-sun-1.5.0.06/
export EC2_HOME=~/ec2/api
export PATH=$PATH:$EC2_HOME/bin
export EC2_PRIVATE_KEY=~/ec2/auth/pk-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.pem
export EC2_CERT=~/ec2/auth/cert-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.pem

My java binary is in /usr/lib/jvm/java-1.5.0-sun-1.5.0.06/bin/java , yours may be somewhere else.

If everything has been set up correctly up to this point, run ~/ec2/api/bin/ec2-describe-images, and you should get the following output after a couple of seconds:

IMAGE   ami-5bae4b32    ec2-public-images/getting-started.manifest      206029621532    available       public
IMAGE   ami-68ae4b01    ec2-public-images/fedora-core4-base.manifest    206029621532    available       public
IMAGE   ami-69ae4b00    ec2-public-images/fedora-core4-apache-mysql.manifest    206029621532    available       public
IMAGE   ami-6dae4b04    ec2-public-images/fedora-core4-apache.manifest  206029621532    available       public
IMAGE   ami-6fae4b06    ec2-public-images/fedora-core4-mysql.manifest   206029621532    available       public

This is a listing of the publicly available images on the EC2 service. You COULD instantiate any one of them into a virtual server. Instead, we’re going to create our own custom image!

1. dd if=/dev/zero of=ubuntu.fs count=1024 bs=1M
   Creates an empty 1 gig loopback file called “ubuntu.fs”
2. mke2fs -F -j ubuntu.fs
   Creates a filesystem for the file
3. sudo mount -o loop ubuntu.fs /mnt
   Mounts the loopback file under /mnt
4. sudo debootstrap dapper /mnt
   Uses the debootstrap utility to install ubuntu’s core packages into /mnt. If you don’t have debootstrap, you can install it with apt-get install debootstrap.
5. sudo cp /etc/apt/sources.list /mnt/etc/apt/sources.list
   Copy your apt source list to the target filesystem
6. sudo chroot /mnt
   Effectively change the target filesystem (/mnt) to be your new root (/). This is where we’ll be doing most of the setup for the server image.
7. Use passwd to update the image’s root password. Don’t lose it.
8. apt-get update and then apt-get upgrade
   Updates apt’s package cache and then updates any of the base packages installed by debootstrap.
9. localedef -i en_US -c -f UTF-8 en_US.UTF-8
   This sets your locale variables.
10. apt-get install openssh-server nano subversion rsync man
    Install some base packages. Tweak this according to your needs.
11. If you want to install Ruby on Rails, do these as well:
    apt-get install ruby ri rdoc mysql-server libmysql-ruby lighttpd
sudo wget http://rubyforge.org/frs/download.php/11289/rubygems-0.9.0.tgz
tar -xvzf rubygems-0.9.0.tgz
cd rubygems-0.9.0
sudo ruby setup.rb
sudo gem install rails --include-dependencies
    And maybe install some of the awesome gems that go along with it:
    sudo gem install capistrano redcloth bluecloth --include-dependencies
12. Paste the following code into the file /etc/fstab

    /dev/sda2       /mnt    ext3    defaults        1 2
    /dev/sda3       swap    swap    defaults        0 0

13. Paste the following code into the file /etc/network/interfaces

    auto lo
    iface lo inet loopback
    
    auto eth0
    iface eth0 inet dhcp

14. Next, I go about setting up my non-root user accounts with adduser. If there are any remaining packages or configuration steps you want in your default image, go ahead and do them now.
15. Note: Make sure that you don’t have any processes running from within the /mnt filesystem. You might need to use /etc/init.d/lighttpd stop or /etc/init.d/mysql stop. Once you’ve confirmed this, use exit to back out of the mounted /mnt filesystem, and then sudo umount /mnt to unmount it.

Your base server is now fully contained in the “ubuntu.fs” loopback file! Now, it’s time to use the Amazon AMI tools to begin the bundling process.

1. ec2-bundle-image -i ubuntu.fs -u [user account number from ~/ec2/auth/account_number]
   This will take a while. The tool will break your ubuntu.fs file up into 10 meg pieces and encrypt them.
2. ec2-upload-bundle -b test-image -m image.manifest -a [key from ~/ec2/auth/access_key_id] -s [secret key from ~/ec2/auth/secret_access_key]
   This will upload the bundles from the previous step to Amazon’s S3 service and put them in the “test-image” bucket.
3. ec2-register test-image/image.manifest
   This will register your image with EC2 in order to make it available for instantiation.

Congratulations! You’ve created an image file, bundled it, uploaded it to S3, and registered it on the EC2 system. In the next entry, we’ll show you how to make virtual servers from your image.

---

Immense thanks to Doug Winter for getting us going on the basics, most of which have been replicated and expanded here.