Temporarily upgrading the resources on your Amazon Web Services EC2 instance

authored by Frank Lynam at 12/02/2015 16:32:48

Any regular visitors to this blog will be aware that I have been working with the Apache Jena RDF triplestore and Fuseki SPARQL interface setup as part of my building of the linkedarc.net Linked Data archaeological web service. In general this combination works pretty well but it is constrained in one major way and that is the resources that are needed load data into Fuseki using the built-in s-put and s-post scripts. On my local test server this is not really an issue as I can run my Ubuntu image on top of about 8GB of RAM, which is fine. However, Fuseki often crashes dues to a lack of available memory when I run these data updates on my online testing server, which is an AWS EC2 micro instance, which has about 700MB of RAM available to it. This post shows how you can temporarily bump up the EC2 micro instance to run as a medium or large instance, do your heavy lifting and then revert back to the micro environment.

Here’s how you do it:

  1. Open up the AWS EC2 console and log in.
  2. Go to the Instances page and select to stop your current micro instance. Note that I originally tried the following steps with the source instance still running but the new instance failed to start correctly. Stopping the source instance obviously does result in its services being withdrawn, however.
  3. Once the instance is stopped, go up to Actions and select to create an image from your source instance.
  4. You will be asked for a name for the new image and you can also choose to alter the size of the new disk that it runs on if you want. Click <Create Image> to start the creation.
  5. In a couple of minutes you should find a new image listed in the Images/AMIs folder. Select this and click <Launch>.
  6. On the first page of the wizard, you get to choose the type of instance family that your new instance will run as.
  7. Once you have chosen one, make sure that you run through all of the steps in the wizard, particularly the final step, which asks you to specify a security group for the new instance as you will not be able to affect this after it is launched.
  8. The wizard finally asks you to confirm how you will SSH into the new instance and you should follow the same practice as you used with the original instance, which in my case is to use public and private keys so as to negate the requirement to constantly re-enter passwords.
  9. When the instance is launched it will appear in the Instances list. If you are using Elastic IPs, it makes sense to now associate the public IP address, which is currently associated with the now stopped source instance, with the new instance.
  10. Now you have a running instance that is identical to your original instance except for the fact that it’s system resources have been changed.
  11. If you redirected the old Elastic IP to point to the new instance, the next time you attempt to SSH to this IP address the SSH client will warn you that the server appears to have changed as follows (with my specific details redacted).

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
The fingerprint for the RSA key sent by the remote host is
############################################.
Please contact your system administrator.
Add correct host key in ################# to get rid of this message.
Offending RSA key in #################:#
RSA host key for ########## has changed and you have requested strict checking.
Host key verification failed.
  1. Remove the target IP address from the file that is noted in the message.
  2. Now try running SSH again and it should ask you if you want to add the IP address to the list of known hosts and after that you should be in.
  3. When you finish whatever it is that you have to do on the upgraded server, go back to the AWS EC2 console. Stop the new instance as before. Create an image of it and launch it specifying to run it using the original resources. Reassign the Elastic IP to the new instance. Repeat the SSH IP address known hosts removal and addition as required.

This is undoubtedly a convoluted series of steps for what is a relatively straightforward outcome but until Amazon decide that they will support the dynamic reallocation of RAM for an instance, it is a necessary evil.

Comments

What's wrong with the following?

1. Stop your instance

2. Use AWS Management Console to change the instance type to the higher instance type

3. Restart your instance

4. Do your work

5. Stop your instance

6. Revert the instance type to the original micro

7. Restart your instance

 

(Matt, 12/02/2015 16:59:04)

Are you able to change the type of a stopped AWS instance though?

(Frank, 27/05/2015 22:06:29)

submit