Application Migration to AWS with Elastic Beanstalk

Prerequisites

Install the AWS Elastic Beanstalk CLI

Copy
eb --version

Setting Up AWS Resources

1. Creating an S3 Bucket:

1. Open the AWS Management Console and navigate to the S3 service. Please make sure you are in the correct AWS region. You can select the region by clicking on the region dropdown in the top right corner of the console. Refer to the image below, where the region selection is highlighted with a red box for guidance on changing the region

   
2. Click on Create bucket. Refer to the image below if you need guidance, where the "Create bucket" option is highlighted with a red box:

   
3. Provide a unique bucket name (e.g., ar-image-processing-bucket). It's recommended to include your initials or another unique identifier in the bucket name to avoid naming conflicts. Refer to the image below, where the field for entering the bucket name is highlighted with a red box:

   
4. Leave the other settings as default unless you have specific requirements. These settings have been covered in detail in Creating an S3 Bucket
5. Click Create bucket once you're comfortable with these settings. Refer to the image below for guidance, where the "Create bucket" button is highlighted with a red box:

   

2. Creating a DynamoDB Table:

1. Open the AWS Management Console and navigate to the DynamoDB service. Please make sure you are in the correct AWS region. You can select the region by clicking on the region dropdown in the top right corner of the console. Refer to the image below, where the region selection is highlighted with a red box for guidance on changing the region.

   
2. In the left panel, click on Tables. This will display the tables panel on the right-hand side, where you'll see a Create table button. Click on Create table to begin setting up your new table. Refer to the images below for guidance, with each relevant section highlighted in a red box:

   • 
   • 
3. Provide a table name (e.g., ar-ImageMetadata) and set the Partition key to id (String). It's recommended to include your initials or another unique identifier in the table name to avoid naming conflicts. Refer to the image below for guidance, where the table name and partition key fields are highlighted with a red box:

   
4. You can leave the other settings as default unless you need advanced configuration.
5. Click Create table to finalize the setup. Refer to the image below for guidance, where the "Create table" button is highlighted with a red box:

   

Cloning the Application from GitHub

1. Clone the repository from GitHub:

Copy
git clone https://github.com/Oxford-Research-Cloud-Competency-Centre/image-processing-cloud-app

2. Navigate to the project directory:

Copy
cd image-processing-cloud-app

Copy
REGION_NAME = 'YOUR_AWS_REGION'
S3_BUCKET = 'YOUR_AWS_S3_BUCKET_NAME'
IMAGE_METADATA_TABLE = 'YOUR_AWS_DYNAMODB_TABLE_NAME'

Deploying the Application to AWS Elastic Beanstalk

Step 1: Verify Configuration Files

1. flask.conf in .ebextensions/

Copy
option_settings:
  aws:elasticbeanstalk:application:environment:
    PYTHONPATH: "/var/app/current:$PYTHONPATH"
  aws:elasticbeanstalk:container:python:
    WSGIPath: "app.app:app"

• option_settings: Specifies configuration options for Elastic Beanstalk.
• aws:elasticbeanstalk:application:environment:: Sets environment variables for the application. Here, the PYTHONPATH is set to include /var/app/current, ensuring that the Python environment includes the application's directory, which is essential for importing modules correctly.
• aws:elasticbeanstalk:container:python:: Configures the Python container. The WSGIPath option defines the entry point for the Flask application. In this case, "app.app:app" points to a Python file and a Flask application instance named app inside the app.py file in the app directory. This setting tells the web server where to find the application's entry point, allowing it to serve HTTP requests to the Flask app.

2. config.yml in .elasticbeanstalk/

Copy
branch-defaults:
  default:
    environment: null
    group_suffix: null
environment-defaults:
  Image-processing-cloud-app-env:
    branch: null
    repository: null
global:
  application_name: image-processing-cloud-app
  default_ec2_keyname: ar-fsl-keypair
  default_platform: Python 3.11 running on 64bit Amazon Linux 2023
  default_region: eu-west-2
  include_git_submodules: true
  instance_profile: null
  platform_name: null
  platform_version: null
  profile: null
  sc: null
  workspace_type: Application

• branch-defaults: Defines default settings for any branches in version control systems (e.g., Git). Setting these to null here means there are no specific branch configurations.
• environment-defaults: Holds configurations for specific environment names. In this example, Image-processing-cloud-app-env is set, which is the name of the environment Elastic Beanstalk will deploy.
• global: Contains general settings for the application:

  • application_name: Specifies the name of the Elastic Beanstalk application (e.g., image-processing-cloud-app). This helps identify the app in the AWS Console.
  • default_ec2_keyname: The name of the EC2 key pair used for SSH access to application instances, here set to ar-fsl-keypair. You will learn to create this key pair in the next section.
  • default_platform: Defines the platform and runtime environment (Python 3.11 on Amazon Linux 2023), ensuring compatibility with the Flask app.
  • default_region: Sets the AWS region (e.g., eu-west-2 for London), determining the location of resources.
  • include_git_submodules: When set to true, Elastic Beanstalk includes any Git submodules, which is helpful if the application has dependencies managed as submodules.
  • workspace_type: Indicates the type of workspace, in this case, Application, meaning it's an Elastic Beanstalk application workspace.

Step 1.2: Create and use your own EC2 key

Copy
global:
  application_name: image-processing-cloud-app
  default_ec2_keyname: ar-fsl-keypair

Step 2: Initialize the Elastic Beanstalk Application

1. Open your terminal and navigate to your project directory.
2. Run the following command to initialize the Elastic Beanstalk application:

   Copy
   eb init

Step 3: Create the Environment and Deploy the Application

Copy
eb create image-processing-cloud-app-env 

• Set up the necessary AWS resources based on your configuration, including:

  • S3 bucket for storing application versions.
  • Security groups for managing access to resources.
  • CloudWatch logs for monitoring and logging application activity.
  • EC2 instance to host the application.
  • Load balancer to distribute incoming traffic across multiple instances (if applicable).

Copy
eb create image-processing-cloud-app-env --enable-spot

• Environment name: The name of your environment.
• Health status: Displayed in green with "OK," indicating that the environment is running smoothly.
• Application name: Associated with the environment.
• Platform: The platform on which the application runs.
• Platform state: Displays as "Supported" in green, meaning the platform version is actively maintained and will continue to receive updates and security patches from AWS.
• Domain: The unique domain endpoint for accessing the environment.
• Running version: The current deployed version of your application.
• Tier name: Typically set as "Web Server" for web applications.

Copy
eb deploy

Step 4: Monitor and Manage the Application

• To check the application's status:

Copy
eb status

• To view logs and troubleshoot:

Copy
eb logs

• To open the application in a browser:

1. Directly from the Elastic Beanstalk Console: Go to your application in the AWS Elastic Beanstalk Console and click on the Domain link, highlighted in red in the image below. This link takes you directly to the URL of your environment.

   
2. Using the EB CLI: Run the following command in your console:

Copy
eb open

Step 5: Terminate the Environment and Application

• To terminate the environment:

Copy
eb terminate

1. Go to the AWS Elastic Beanstalk Console.
2. In the left navigation panel, click on Applications to see a list of applications.
3. Select the application you wish to delete.
4. Click on Actions and choose Delete Application.
5. Confirm the deletion by typing the application name when prompted.

• To terminate the environment and delete the application:

Copy
eb terminate --all

Step 6: Resource Cleanup

1. Delete the S3 Bucket (Application Storage):

   • Go to the S3 console and locate the bucket you created specifically for storing images.
   • Select the bucket, then click on Empty to delete all objects within it. Refer to the image below, where the Empty button is highlighted in red for guidance:

     
   • After clicking Empty, a new page will open. To confirm, type "permanently delete" in the text input field, then click on Empty again to finalize. Refer to the image below for guidance, where the confirmation field and Empty button are highlighted in red:

     
   • After emptying the bucket, return to the bucket's main page and click on Delete to remove the bucket itself. Refer to the image below, where the Delete button is highlighted in red:

     
   • A new window will open. To confirm the bucket deletion, enter the bucket name in the text input field, then click on Delete Bucket. Refer to the image below, where the confirmation field and Delete Bucket button are highlighted in red:

     
   • Please Note: It may take a few moments for AWS to fully delete the bucket. Wait for a confirmation message indicating that the bucket has been deleted to ensure the resource is fully removed.
2. Delete the Elastic Beanstalk S3 Bucket (Application Versions): Elastic Beanstalk creates an additional S3 bucket (with a name starting with elasticbeanstalk-<region>-<account-id>) to store application versions, logs, and other deployment artifacts. The eb terminate command does not delete this bucket automatically.

   Why is this bucket not deleted automatically? Elastic Beanstalk keeps this bucket as a safeguard against accidental data loss. This bucket stores valuable information, such as application versions and logs, which can be essential for tasks like troubleshooting, re-deployment, or audit purposes. By retaining this bucket, AWS allows users to access and manage these artifacts independently, deciding if and when to delete them based on their needs.

   To delete this bucket manually:

   1. Open the Amazon S3 console.
   2. Locate the bucket that begins with elasticbeanstalk-<region>-<account-id>.
   3. Follow the same steps as above to Empty and Delete the bucket, ensuring all deployment versions and logs are permanently removed.

   This step completes the cleanup of all Elastic Beanstalk-related resources.
3. Delete the DynamoDB Table:

   • Navigate to the DynamoDB console.
   • In the Tables section, locate the table you created (e.g., ar-ImageMetadata).
   • Select the table, then click on Delete table to remove it. Refer to the image below, where the Delete table button is highlighted in red:

     
   • After clicking Delete table, a new window will open asking for confirmation. Type confirm in the text input field, then click on the Delete button to finalize the deletion. Refer to the image below, where the confirmation field and Delete button are highlighted in red:

     
   • Please Note: It may take a few moments for AWS to fully delete the table. Wait until you see a confirmation message indicating that the table has been deleted to ensure it is completely removed.

Important: Be sure to back up any necessary data from these resources before deleting them, as this action is irreversible.