Tuesday, April 26, 2016

AWS Lambda: Setting a temporary maintenance page in an Elastic Load Balancer during a CodeDeploy deployment

When using CodeDeploy to push updates or perform maintenance on an Auto Scaling Group, the default behavior is for the ASG instances to be excluded from the ELB, leaving users with a blank page. This is awful UX, and is not acceptable for a production application. It seems to me that it should be basic functionality for ELB to allow developers to define a 503 page, but that's a rant for another day. Some people have toyed around with Route53 weighted routes, but that solution is very clunky as it takes time for the DNS entries to expire before they are refreshed by clients. I've come up with a reasonable solution that uses CodeDeploy triggers and an AWS Lambda function to temporarily display a maintenance page when a deployment is in progress.

First, create a micro sized instance to host our static maintenance page. It does not matter what type of instance this is, so long as it has a web server that serves your temporary page (you probably also want to configure it using rewrite rules to ensure any requested route will serve your maintenance page).

Now we need to create a SNS topic to be used for the CodeDeploy triggers. In the SNS console, create a topic with a unique name. Then in CodeDeploy, create a trigger for your deployment group for all deployment statuses, and select the SNS topic that you just created. Now CodeDeploy will send notifications to this topic for every step of the deployment process.

The next step is to create your AWS Lambda function. Name your function something unique and choose your language. I used Python, and my code is provided below. Define your handler name. In my code example, this value would be lambda_function.process_cd_trigger. Create a new Lambda role that has EC2 permissions in order to make changes to the ELB/ASG via Lambda. Define the VPC that your ELB/ASG exist in and save your function.

Go back to the SNS console, select your topic, and subscribe your Lambda function to your SNS topic. Your ELB will now contain only your static maintenance site when CodeDeploy is in progress!

Here is my Lambda function:
import json
import boto3

asg = boto3.client('autoscaling', region_name='us-east-1')
elb = boto3.client('elb', region_name='us-east-1')
cd = boto3.client('codedeploy', region_name='us-east-1')

def show_maint_page():
    try:
        asg.detach_load_balancers(
            AutoScalingGroupName='[ASG NAME]',
            LoadBalancerNames=['[ELB NAME]']
        )
    except:
        pass
    try:
        elb.register_instances_with_load_balancer(
            LoadBalancerName='[ELB NAME]',
            Instances=[{'InstanceId': '[INSTANCE ID]'}]
        )
    except:
        pass

def hide_maint_page():
    try:
        elb.deregister_instances_from_load_balancer(
            LoadBalancerName='[ELB NAME]',
            Instances=[{'InstanceId': '[INSTANCE ID]'}]
        )
    except:
        pass
    try:
        asg.attach_load_balancers(
            AutoScalingGroupName='[ASG NAME]',
            LoadBalancerNames=['[ELB NAME]']
        )
    except:
        pass
def process_cd_trigger(event, context):
    cdMsg = json.loads(event['Records'][0]['Sns']['Message'])
    depId = cdMsg['deploymentId']
    status = cdMsg['status'].upper()
    dgName = cdMsg['deploymentGroupName'].upper()
    
    #Parse environment name from Deployment Group name
    env = dgName[:dgName.find('-')]
    
    #Make sure this is not just a deployment to a new instance
    deployment = cd.get_deployment(deploymentId = depId)
    creator = deployment['deploymentInfo']['creator'].upper()
    
    if(creator != "AUTOSCALING" and status == "CREATED"):
        show_maint_page(env)
    elif(status == "SUCCEEDED" or status == "FAILED" or status == "ABORTED"):
        hide_maint_page(env)

3 comments:

  1. This comment has been removed by a blog administrator.

    ReplyDelete
  2. This comment has been removed by a blog administrator.

    ReplyDelete
  3. This comment has been removed by a blog administrator.

    ReplyDelete

Note: Only a member of this blog may post a comment.