Making your own EC2 server using public images

October 17th, 2006 by Forrest

In this walkthrough, we are going to create an EC2 server, using the pre-existing public images that amazon has so kindly set up for us.

  1. Make sure that you are set up and ready to preform ec2 commands and manage images/instances. If you need to set this up, the available documentation is avalable here
  2. As soon as you are ready to rock out with your new ec2 toys, you are going to want to create a keypair. This keypair will be used to attach to a public image at runtime, ensuring your, and only your access to it. You can do this with the following command:
    ec2-add-keypair [name-of-keypair]
    it helps to name it something specific to the image that you are building. IE if you are building a database backup, you may name the keypair “db-backup”
    this command will output something very similar to:

    KEYPAIR db-backup 1f:51:ae:28:bf:89:e9:d8:1f:25:5d:37:2d:7d:b8:ca:9f:f5:f1:6f
    -----BEGIN RSA PRIVATE KEY-----
    MIICXAIBAAKBgQC2i/Sgs5BGGd4sunpYQfEkcprgzP9M/hnVJTc1j0nZBeIE2JBuLRSNoqkO7Gw8
    nBcdNptaLedzqN8t78jGkX1TPWVAKJTfxRSvU/oViGJaRqIBar0Mpc/wC27kyzHezUNS5+mvONb3
    4h/j2EZwDLY75Uxrpka0aN6OkvyIP5gYMQIDAQABAoGAOKH65tBOdjEYSHAh/LeYhGI5wnxWyCAd
    C49cLXWix32XvUEircu2kKpiIIsgmT0jvqBuWe/b2noNo0a81z3TzzRYyLvn5J8mUlL6a8nsssQ3
    xCHkGM+SE7ZzfBS5WUkbh5Exd3ZXKfCJvJW6auOzJ581JB5yUNbqixWzHuQGGAECQQDwq4LQoyb5
    OVSpZwSy+GW/p0yRsqRp89ECNQ+hySGBjkSXBcbt75C+5ebo88/V2V4QOGGa0T0tMsMgKTJ8oukh
    AkEAwiyoFM0Zwk0Os3rBZ8PyZoNW5e5SBwrEbLRv4JCaNiQme0ighsDr2bL/nGLI7p13g22+9REM
    i/WAmsln50H9EQJASMun6tGepT2pFQBbFIM7y4egCmXdg0rDSoagLtB2eQh+SKvvquKOhp9lg8rT
    b5yq7f8PztNBTN2Q1baAVeC04QJAGgN5kS/ZH5rLOWhcuNYbh3hZD/zZqG/c2ONjiaZVwqMdNK8K
    MoNuFYBRllX1rWITPNxbFOHv2GBPlm0dKnJAwQJBAOgwjgLY3UpXFX9ZvG4RGEYgfui49Ffz10CH
    5sSZpsFYn42E6a2NUJeL4hTzfbGTQ8iCIVjOXFH/9XLTDCNQEPM=
    -----END RSA PRIVATE KEY-----
  3. Once you spit out the keypair, you have to save it somewhere. It doesnt really matter where, but use your favorite text editor to create a file (in you current directory or in /keypair is usually a nice spot) name the file: id_rsa-db-backup-keypair or something memmorable related to the keypair you created. It will all make sense soon I promise.
    IMPORTANT: Copy into that text file ONLY what appears between -----BEGIN RSA PRIVATE KEY----- and -----END RSA PRIVATE KEY-----
  4. You are now ready to boot a public image. Use ec2-describe-images to list the available public images. It should output something like:
    IMAGE   ami-5bae4b32    ec2-public-images/getting-started.manifest      206029621532    available public
    IMAGE   ami-68ae4b01    ec2-public-images/fedora-core4-base.manifest    206029621532    available public
    IMAGE   ami-69ae4b00    ec2-public-images/fedora-core4-apache-mysql.manifest    206029621532    available public
    IMAGE   ami-6dae4b04    ec2-public-images/fedora-core4-apache.manifest  206029621532    available public
    IMAGE   ami-6fae4b06    ec2-public-images/fedora-core4-mysql.manifest   206029621532    available public
    IMAGE   ami-aca84dc5    ubuntu-base/image.manifest      554263365884    available       private
  5. Choose your target, and boot the crap out of it
    ec2-run-instances ami-69ae4b00 -k gsg-db-backup
    notice the "-k gsg-db-backup" this is the name of the keypair we created, not the name of the file we stored the keypair in. That comes in later.
  6. Your instance will now be pending for a minute or three, and then will be running. Use ec2-describe-instances to get an update on whats going on. It should output something like (after the instance boots of course):
    RESERVATION     r-fea54097  495219933132   EC2
    INSTANCE        i-10a64379  ami-69ae4b00   domU-12-34-31-00-00-05.usma1.compute.amazonaws.com EC2    running   gsg-db-backup

    If you do not see the name of your keypair after the instance, then you did something wrong, so you may as well shut down the instance and start over.

  7. We now want to authorize port 22 for use with ssh (you can authorize other ports for webserving and whatever else later… this is the important one to ensure you can get into the thing.) use the following command:
    ec2-authorize default -P tcp -p 22
  8. Lets go ahead and get into that puppy. To do this, we are going to ssh, but not use a password, so we are going to call upon the keypair file that we created. Your command should look something like:
    ssh -i id_rsa-db-backup root@domU-12-34-31-00-00-05.usma1.compute.amazonaws.com
    If you stored the keypair file elsewhere, you may have to give the full path:
    ssh -i /path_to_keypair/id_rsa-db-backup root@domU-12-34-31-00-00-05.usma1.compute.amazonaws.com
    If all goes smoothly, and everything was done right, you will now be logged in to a running instance as root. Yeah!
  9. This step is easy, simply add all of the users you want, install whatever packages you want, basically make that server as personal as you want. View Jack’s post here for further details. Just make sure that you change passwords and add users so that you can ssh it later.
  10. This part is important, make sure that you upload your private key to the running instance, so that when we bundle, you have the proper permissions. Trust me, realizing you messed up after transfering a few gigs for an hour is not fun.
    scp pk-XXXXXXXXXXXXXXXXXXXXXXXXXXXX.pem root@domU-12-34-31-00-00-05.usma1.compute.amazonaws.com:/tmp
    This is going in the /tmp directory so that it is not carried over at bundle.
  11. Now we are going to bundle this server into a nice little package that you can boot anytime, anywhere. Thanks to amazon, the ec2 tools to do so are already installed. So use the following command:
    ec2-bundle-vol -k /path_to_your_key/pk-XXXXXXXXXXXXXXXXXXXXXXXXXXXX.pem -s 1000 -u [your_user_id]
    This command is going to take 5 or 10 minutes, and does not have any output while doing so, so just be patient. When it is done, it should output the image.manifest file, along with all of its part buddies.
  12. These files should all be stored in the /tmp directory. We want to scp them over to the machine that you were working off of originally. This is easier to scp from the local machine, especially if it is behind some sort of firewall with no external ip. Like so:
    scp user@domu-12-31-33-00-03-d7.usma1.compute.amazonaws.com:/tmp/* /path_on_local_machine/
    Again, this is a lot of information, (a gig to be exact) it took me 20 minutes or so to transfer. This time you are awarded the luxury of progress however, so either watch the water boil, or get a cup of coffee.
  13. With the image and image parts are on your local machine, uploading them to your s3 cloud should be as easy as it always is. Just make sure you use the proper keys on upload:
    ec2-upload-bundle -b my-bucket -m image.manifest -a [your_key_id] -s [secret_key_id]
    You can get your key id, and secret key id from your account section when logged into aws.amazon.com
    This is going to take another 20 minutes or so… so if you already got coffee… you should try a donut.
  14. It would be a shame not to authorize this image for use in EC2. It’s easy:
    ec2-register your-bucket/image.manifest
    Told you.
  15. Use ec2-describe-images and you should see your brand new image ready and waiting to be booted. I would suggest shutting down the public instance before proceeding. If you really dont want to, the worst that will happen is you will be double charged, and may get confused if you see two seemingly identicle instances running.
    All that is left to do is boot it. You already know how to do that, just ec2-run-instances ami-61a54008
    Use ec2-describe-instances and it should say pending
    When it is done booting, voila! You just created an instance from a public image!

Posted in ec2, walkthrough |

2 Responses

  1. Josh Says:

    Just curious in step 13, can you upload the bundle directly from the ec2 machine instance to s3 (saving the data from going to your local machine)? Or does ec2-upload-bundle not work from there?

    Thanks for the great breakout, very helpful for thinking about using ec2.

  2. Forrest Says:

    Excellent question! The answer is yes. The public instances come installed with ec2 tools, and work just like any other server. These tools are limited however and only have a few of the tools, including bundle, and upload.

    The reason I decided to scp it to a local machine is simply for backup purposes. Not much else…

    Good luck!

Leave a Comment

Please note: Comment moderation is enabled and may delay your comment. There is no need to resubmit your comment.