Skip to content

DEPLOYMENT_GUIDE.md

Latest commit

 

History

History
265 lines (221 loc) · 10.7 KB

DEPLOYMENT_GUIDE.md

File metadata and controls

265 lines (221 loc) · 10.7 KB
Raw

DropNote AWS Architecture & Deployment Guide

This guide provides step-by-step instructions for deploying DropNote onto Amazon Web Services (AWS) using a high-availability, auto-scaling architecture.

1. AWS Architecture Overview

                          [ Internet Users ]
                                  │
                                  ▼
                   [ Application Load Balancer (ALB) ]
                     (Multi-AZ, Port 80, /health)
                                  │
                 ┌────────────────┴────────────────┐
                 ▼                                 ▼
   [ EC2 Instance (AZ-1a) ]          [ EC2 Instance (AZ-1b) ]
   (Gunicorn + Nginx + IMDSv2)       (Gunicorn + Nginx + IMDSv2)
   └──────────────────────┬────────────────────────┘
                          │ Auto Scaling Group (2 - 4 Instances)
        ┌─────────────────┴─────────────────┐
        ▼                                   ▼
[ Amazon RDS MySQL ]                [ Amazon S3 Bucket ]
 (Encrypted DB Data)                 (Encrypted File Store)

Key Components

  1. Application Load Balancer (ALB): Public entrypoint distributing user requests across multiple Availability Zones. Health checks target /health.
  2. Auto Scaling Group (ASG): Maintains 2 to 4 EC2 instances across 2 AZs, dynamically scaling out when CPU Utilization exceeds 50%.
  3. IAM Instance Profile (LabRole): Grants EC2 instances secure access to Amazon S3 without hardcoding API keys.
  4. Amazon RDS MySQL: Relational database storing note metadata, share codes, and expiration timestamps.
  5. Amazon S3: Object storage for uploaded attachments (<10 MB) with pre-signed download URLs.
  6. IMDSv2: Secure Instance Metadata Service utilized by the /status page to display target instance IDs and AZs.

2. Step-by-Step Deployment Instructions

Step 1: Create the Amazon S3 Bucket

  1. Open the Amazon S3 Console.
  2. Click Create bucket.
  3. Bucket name: e.g., dropnote-storage-<your-student-id>.
  4. AWS Region: us-east-1 (or your preferred region).
  5. Block Public Access: Keep "Block all public access" checked (S3 files are securely accessed via backend pre-signed URLs).
  6. Leave default encryption (SSE-S3) enabled and click Create bucket.

Step 2: Create the RDS MySQL Database

  1. Open the Amazon RDS Console → Databases → Create database.
  2. Engine: MySQL (Version 8.0.x).
  3. Templates: Free Tier / Dev/Test.
  4. DB instance identifier: dropnote-mysql-db.
  5. Master username: admin.
  6. Master password: Choose a strong password (e.g., YourSecurePassword123!).
  7. Instance configuration: db.t3.micro or db.t4g.micro.
  8. Connectivity:
    • VPC: Default VPC (or your custom lab VPC).
    • Public access: No.
    • VPC Security Group: Create new / select dropnote-db-sg (Allow inbound TCP port 3306 from the EC2 security group dropnote-ec2-sg).
  9. Initial database name (under Additional Configuration): dropnote_db.
  10. Click Create database and copy the RDS Endpoint once available (e.g., dropnote-mysql-db.xxxx.us-east-1.rds.amazonaws.com).

Step 3: Configure Security Groups

Security Group 1: dropnote-alb-sg (Application Load Balancer)

  • Inbound Rules:
    • Type: HTTP (Port 80), Source: 0.0.0.0/0 (Anywhere)
  • Outbound Rules:
    • All Traffic (0.0.0.0/0)

Security Group 2: dropnote-ec2-sg (EC2 Instances)

  • Inbound Rules:
    • Type: HTTP (Port 80), Source: dropnote-alb-sg (Only allow traffic from the ALB)
    • Type: SSH (Port 22), Source: My IP (Optional, for debugging)
  • Outbound Rules:
    • All Traffic (0.0.0.0/0)

Security Group 3: dropnote-db-sg (RDS MySQL)

  • Inbound Rules:
    • Type: MYSQL/Aurora (Port 3306), Source: dropnote-ec2-sg (Only allow traffic from EC2)
  • Outbound Rules:
    • All Traffic

Step 4: Create Target Group & Application Load Balancer

A. Target Group (dropnote-tg)

  1. Open EC2 Console → Target Groups → Create target group.
  2. Target type: Instances.
  3. Target group name: dropnote-tg.
  4. Protocol: HTTP, Port: 80, VPC: Default VPC.
  5. Health checks:
    • Health check protocol: HTTP
    • Health check path: /health
    • Advanced health check settings:
      • Healthy threshold: 2
      • Unhealthy threshold: 3
      • Timeout: 5 seconds
      • Interval: 15 seconds
      • Success codes: 200
  6. Click Next → Create target group (Instances will be registered automatically by the ASG).

B. Application Load Balancer (dropnote-alb)

  1. Open EC2 Console → Load Balancers → Create load balancer → Application Load Balancer.
  2. Name: dropnote-alb.
  3. Scheme: Internet-facing.
  4. IP address type: IPv4.
  5. Network mapping: Select VPC and check at least two Availability Zones (e.g., us-east-1a and us-east-1b).
  6. Security groups: Select dropnote-alb-sg.
  7. Listeners and routing:
    • Protocol: HTTP, Port: 80 → Forward to dropnote-tg.
  8. Click Create load balancer. Copy the DNS Name (e.g. dropnote-alb-123456.us-east-1.elb.amazonaws.com).

Step 5: Create EC2 Launch Template

  1. Open EC2 Console → Launch Templates → Create launch template.
  2. Launch template name: dropnote-launch-template.
  3. AMI: Amazon Linux 2023 AMI (x86_64).
  4. Instance type: t2.micro or t3.micro.
  5. IAM instance profile: Select LabRole (or a custom role with AmazonS3FullAccess).
  6. Security Groups: Select dropnote-ec2-sg.
  7. Metadata version (IMDS): Ensure IMDSv2 is enabled / optional.
  8. Advanced details → User data: Paste the following script (with your actual RDS and S3 details filled in):
#!/bin/bash
set -e
exec > >(tee /var/log/user-data.log|logger -t user-data -s 2>/dev/console) 2>&1

# 1. Environment Configuration
DB_HOST="your-rds-endpoint.xxxxxx.us-east-1.rds.amazonaws.com"
DB_PORT="3306"
DB_USER="admin"
DB_PASSWORD="YourSecurePassword123!"
DB_NAME="dropnote_db"
S3_BUCKET_NAME="your-dropnote-s3-bucket-name"
AWS_REGION="us-east-1"
ADMIN_SECRET_KEY="dropnote-admin-secret-2024"

# 2. Package Installation (Amazon Linux 2023)
dnf update -y
dnf install -y git python3 python3-pip python3-devel gcc nginx

# 3. Clone Repository
APP_DIR="/opt/dropnote"
APP_USER="ec2-user"
mkdir -p "$APP_DIR" /var/log/dropnote
git clone https://github.com/PRSXFENG/cloud-computing-final.git "$APP_DIR"

# 4. Setup Python Environment & Dependencies
cd "$APP_DIR"
python3 -m venv "$APP_DIR/venv"
"$APP_DIR/venv/bin/pip" install --upgrade pip
"$APP_DIR/venv/bin/pip" install -r "$APP_DIR/requirements.txt"

# 5. Generate .env File
cat <<EOF > "$APP_DIR/.env"
FLASK_ENV=production
FLASK_DEBUG=0
PORT=5000
SECRET_KEY=$(openssl rand -hex 24)
DB_HOST=$DB_HOST
DB_PORT=$DB_PORT
DB_USER=$DB_USER
DB_PASSWORD=$DB_PASSWORD
DB_NAME=$DB_NAME
S3_BUCKET_NAME=$S3_BUCKET_NAME
AWS_REGION=$AWS_REGION
ADMIN_SECRET_KEY=$ADMIN_SECRET_KEY
MAX_CONTENT_LENGTH=10485760
EOF

chmod 600 "$APP_DIR/.env"
chown -R "$APP_USER:$APP_USER" "$APP_DIR" /var/log/dropnote

# 6. Setup Systemd Service
cat <<EOF > /etc/systemd/system/dropnote.service
[Unit]
Description=DropNote Gunicorn Server
After=network.target

[Service]
Type=simple
User=$APP_USER
Group=$APP_USER
WorkingDirectory=$APP_DIR
EnvironmentFile=$APP_DIR/.env
ExecStart=$APP_DIR/venv/bin/gunicorn --workers 3 --threads 2 --bind 127.0.0.1:5000 --access-logfile /var/log/dropnote/access.log --error-logfile /var/log/dropnote/error.log --timeout 90 wsgi:application
Restart=always
RestartSec=3

[Install]
WantedBy=multi-user.target
EOF

systemctl daemon-reload
systemctl enable --now dropnote.service

# 7. Setup Nginx Reverse Proxy
cp "$APP_DIR/nginx.conf" /etc/nginx/conf.d/dropnote.conf
rm -f /etc/nginx/conf.d/default.conf
systemctl enable --now nginx

# 8. Setup 7-Day Expired Cleanup Cron
echo "0 3 * * * $APP_USER $APP_DIR/venv/bin/python $APP_DIR/scripts/cleanup_cron.py >> /var/log/dropnote/cleanup.log 2>&1" > /etc/cron.d/dropnote-cleanup
chmod 644 /etc/cron.d/dropnote-cleanup
  1. Click Create launch template.

Step 6: Create Auto Scaling Group (ASG)

  1. Open EC2 Console → Auto Scaling Groups → Create Auto Scaling group.
  2. Name: dropnote-asg.
  3. Launch template: Select dropnote-launch-template.
  4. Network: Select your VPC and select subnets across at least 2 Availability Zones (us-east-1a, us-east-1b).
  5. Load balancing:
    • Select Attach to an existing load balancer.
    • Choose Choose from your load balancer target groups → Select dropnote-tg.
    • Health checks: Check both EC2 and Elastic Load Balancing (ELB). Health check grace period: 180 seconds.
  6. Group size:
    • Desired capacity: 2
    • Minimum capacity: 2
    • Maximum capacity: 4
  7. Scaling policies:
    • Select Target tracking scaling policy.
    • Metric type: Average CPU utilization.
    • Target value: 50%.
    • Warmup time: 60 seconds.
  8. Click Next through notifications/tags → Create Auto Scaling group.

3. Verification & Coursework Demonstration

A. Verify ALB Health & Routing

  1. Open your browser and navigate to http://<ALB-DNS-NAME>/health.
    • Expect: {"status": "ok", "timestamp": "..."} with HTTP 200.
  2. Navigate to http://<ALB-DNS-NAME>/status.
    • Observe the EC2 Instance ID and Availability Zone.
    • Refresh the page several times to see requests handled alternatively by instances in us-east-1a and us-east-1b.

B. Test Note & File Sharing Flow

  1. Navigate to http://<ALB-DNS-NAME>/.
  2. Write a note, attach an image/file (< 10 MB), and click Drop Note.
  3. Copy the generated 6-character code (e.g. K9X2P7) or the direct link http://<ALB-DNS-NAME>/?code=K9X2P7.
  4. Open an incognito browser window or separate device, retrieve the note using the code, and click the Download Attachment button.

C. Demonstrate Dynamic Auto Scaling (Stress Test)

  1. Navigate to http://<ALB-DNS-NAME>/status.
  2. In the Internal Auto Scaling & Demo Tools card:
    • Enter your ADMIN_SECRET_KEY (e.g., dropnote-admin-secret-2024) and click Save Key.
    • Select a 45s or 60s duration and click Start CPU Stress Test.
  3. Open the AWS CloudWatch Console → Metrics → EC2 / ASG CPUUtilization.
  4. Observe the CPU utilization jump above 70%.
  5. Watch the Auto Scaling Group trigger a Scale-Out Alarm and spin up additional EC2 instances up to the maximum capacity of 4.