Managing AWS Instances

Please read Workshops Organisation if you haven’t done so.

The environments and configurations presented in the previous lesson, Setting Up Your Cloud and Terminal Environments, comprise a base development environment to create and manage AWS services with both the AWS Console and the AWS CLI (command line interface). If you are using an AWS personal account (that you created and configured as described in that lesson), you can create and manage any service in any AWS region, as your account was configured with the AdministratorAccess permissions policy which “Provides full access to AWS services and resources”.

Towards using the Scripts to create and manage AWS instances, we will first need to configure Internet access for the instances. The instances configuration and the Scripts were designed for each instance to be accessed with the program ssh using an encrypted login key and an optional domain name, as outlined below. If you decide on using domain names to access instances, you must then create a base domain name from which the Scripts will create a subdomain for each instance. Creating a base domain name in your AWS account will incur some cost, as low as US $5.00 per year or more depending on the suffix /top-level domain (TLD) that you choose for your base domain name. Examples of TLDs include: .com, .net, .org, .link, among many others (the cheapest in AWS is .link). If you already have a domain name in place, you can use that instead. Your account will also incur costs for any service you launch that is not included in the AWS Free Tier — check the lesson AWS Costs Explained from another Cloud-SPAN course, so you know your free limits.

If you are using an AWS institutional account (that was created and configured by the IT department in your institution), and you decide on using domain names, you will most likely not be directly responsible for the cost of using domain names or any other service with your account. However, you may need to ask your IT department to configure the base domain name for you, or to follow some guidelines to specify that domain name. Your account may also have restrictions as to the AWS region where you can create and manage AWS services. For instance, the AWS institutionl account of the Cloud-SPAN project can only make use of services in the eu-west-1 Ireland region, and the Cloud-SPAN base domain name was suggested and configured by our IT department. If your institutional account is restricted to use an AWS region other than Ireland, you may need to ask your IT department to make a copy, to your AWS account, of the Amazon Machine Image (AMI) template used by the Scripts to create AWS instances. Your IT department should be able to help you with this and other matters; just let them know what you need to configure in your AWS account to use the Scripts, and whenever you come accross Access Denied or similar messages when using the Scripts.

Overview

We use the Scripts to create and manage multiple AWS instances for training. When running a workshop, for each participant an instance is created as a copy of an AMI that is configured with ’omics data and software analysis tools that are relevant to the workshop.

Workshop participants login to the csuser account in their instance, using ssh with an encrypted login key file and either with a domain name (if a base domain name was configured and is specified to the Scripts as described later in the course) or with the IP address allocated to the instance by AWS.

Examples. Suppose two instances have been created with the names instance01 and instance02 (which must be specified in a file that is passed to the Scripts as described later in the course). Each instance would be accessed as shown below (depending on whether a base domain name was specified to create the instances):

Code
$ ssh -i login-key-instance01.pem  csuser@instance01.cloud-span.aws.york.ac.uk
$ ssh -i login-key-instance02.pem  csuser@instance02.cloud-span.aws.york.ac.uk

$ ssh -i login-key-instance01.pem  csuser@instance01.awsplaicloud.com
$ ssh -i login-key-instance02.pem  csuser@instance02.awsplaicloud.com

$ ssh -i login-key-instance01.pem  csuser@10.134.43.226
$ ssh -i login-key-instance02.pem  csuser@43.226.10.134

The ssh parameter -i stands for identify file.

The first two lines correspond to the case where the base domain name specified was that of the Cloud-SPAN project institutional account: cloud-span.aws.york.ac.uk.

The third and fourth lines correspond to the case where the base domain name specified was that of an AWS personal account, for example: awsplaicloud.com.

The last two lines correspond to the case where no base domain name was specified. Note that in this case the instance names are still used to identify the login key files but cannot be used (as part of a domain name) to identify the corresponding instances.

Episode 1: Configure Instances Internet Access

Episode 1 will guide you to configure internet access for the instances you will create with the Scripts. This involves:

  • creating a base domain name — optional
  • creating a few access rules for the communication ports used by the ssh program
  • selecting an AWS network to which the instances will be attached so that they can be reached from anywhere with ssh — optional but please read the overview of the episode

Episode 2: Instances Management Tasks Using the Scripts

Episode 2 is the guide to using the Scripts to create and manage multiple instances for a course /workshop. The episode shows:

  • how to configure the Scripts with the names of the instances to create and the AWS resources to use (base domain name, AMI template, etc.).
  • how to organise the Scripts configuration files for multiple courses.
  • how to use /run the Scripts
  • how to manage unforseen instances management requests such as cancellations or late registrations by workshop participants, and
  • some troubleshooting

Episode 3: AMIs Management

Episode 3 is about managing Amazon Machine Images (AMIs). As AWS instances are copies of an AMI, you need to create a new AMI if the software or data used in a course change, but there are other reasons that may require creating a new AMI. The episode presents the management of AMIs that we have done as part of managing AWS instances with the Scripts, including how to create and customise a new AMI.

Episode 4: The Scripts Design

Episode 4 presents the organisation and workings of the Scripts, how the Scripts were developed using the AWS CLI Command Reference, and a few ideas to improve the Scripts.

Back to top