Posted onFeb 12, 2023

EC2 Cost Saving With Lambda and EventBridge

Usually when we should manage a large amount of AWS accounts, saving costs at scale is not a simple task, we need to use third-parties tools or code scripts to do that. In this post I will show you how to manage EC2 instances running time efficiently usingLambda andEventBridge Scheduler based on invented requirements.

Disclaimer

I describe anhypothetical state to walk through an use case and solution proposed.

Current Status

I am a member of the IT team that manages AWS resources for a company around the world. We have some EC2 instances that run software to be used for our company’s employees during work days in office hours.

Requirements

We need to establish a mechanism tosave cost keeping in mind that employees are allocated indifferent time zones.

Solution proposal

We can purchase saving plans, or reserved instances, but that does not make sense because we don’t need 24x7 EC2 running, we only want these instances up from Monday to Friday in office hours. Sothe most efficient way, in this case, is to maintain instances running for the period of time that users need to access to the software installed and stop them the rest of the time.

How can we do that? I will show you an example with Lambda to start and stop instances and EventBridge Scheduler to manage Lambda invocations.

For this example I deploy some EC2 instances that are being used from Mexico and Spain users and all of them are deployed on eu-west-1 region (Ireland). To manage instances based on country I have tagged them with the“country” key and the value of this tag is based onISO 3166-1 alpha-2 country code. That means that I should manage two time zones to support users in two different countries.

Below you can find a diagram with resources involved in the solution:

Step by step

1.- Launch EC2 Instances

I use the smallest instance type for this example based on a Linux distribution, you can launch instance following any of methods explained inLaunch your instance topic from EC2 user guide. I have used two instances but you could launch more if you want.

The most important thing for this example is totag all of the instances withkey:countryvalue:es orvalue:mx

2.- Deploy lambda functions for starting and stopping instances

In this case I have code two different Lambdas coded in python, one for starting instances and other for stopping instances. I hope you can find code self-explained.

I have decided to deployone lambda per action and country, so there are 4 lambda functions.

To deploy lambda functions you can follow the steps described inAWS Lambda Developer Guide.

At the end of this section you can find code for each function and custom policies to be attached to Lambda's role.

Keep in mind that you need to create a Role to be attached to Lambda function and must contains a custom policy to grant permission for Lambda execution.

After deployment you should have 4 Lambda functions

Lambda code for starting instances

import loggingimport boto3logger = logging.getLogger()logger.setLevel(logging.INFO)ec2 = boto3.resource('ec2')#Change value based on ISO 3166-1 alpha-2 country codeCOUNTRY_CODE = 'es'STOPPED_INSTANCE_CODE = 80def is_stopped(instance):    if instance.state['Code'] == STOPPED_INSTANCE_CODE:        return True    return Falsedef has_country(country, instance):    tags = instance.tags    for tag in tags:        if (tag['Key'] == 'country') & (tag['Value'] == country):            return True    return Falsedef get_stopped_instances_id(country):    stop_instances_id = []    instances = list(ec2.instances.all())    logger.info("Number of instances found: {}".format(len(instances)))    for instance in instances:        if has_country(country,instance) & is_stopped(instance):            stop_instances_id.append(instance.instance_id)    return stop_instances_iddef start_instances(country):    stopped__instances_id = get_stopped_instances_id(country)    logger.info("{} number of instances that will be start".format(str(len(stopped__instances_id))))    for id in stopped__instances_id:        ec2.Instance(id).start()        logger.info('Instance with id {} has been started'.format(id)  )def lambda_handler(event, context):    country=COUNTRY_CODE    logger.info('Starting instances of: {}'.format(country))    start_instances(country)

Custom policy for Lambda execution's Role (start instances)

{    "Version": "2012-10-17",    "Statement": [        {            "Sid": "VisualEditor0",            "Effect": "Allow",            "Action": [                "ec2:DescribeInstances",                "ec2:StartInstances"            ],            "Resource": "*"        },        {            "Sid": "VisualEditor1",            "Effect": "Allow",            "Action": [                "logs:CreateLogStream",                "logs:CreateLogGroup",                "logs:PutLogEvents"            ],            "Resource": "arn:aws:logs:*:*:*"        }    ]}

Lambda code for stopping instances

import loggingimport boto3logger = logging.getLogger()logger.setLevel(logging.INFO)ec2 = boto3.resource('ec2')#Change value based on ISO 3166-1 alpha-2 country codeCOUNTRY_CODE = 'es'RUNNING_INSTANCE_CODE = 16def is_running(instance):    if instance.state['Code'] == RUNNING_INSTANCE_CODE:        return True    return Falsedef has_country(country, instance):    tags = instance.tags    for tag in tags:        if (tag['Key'] == 'country') & (tag['Value'] == country):            return True    return Falsedef get_running_instances_id(country):    running_instances_id = []    instances = list(ec2.instances.all())    logger.info("Number of instances found: {}".format(len(instances)))    for instance in instances:        if has_country(country,instance) & is_running(instance):            running_instances_id.append(instance.instance_id)    return running_instances_iddef stop_instances(country):    running_instances_id = get_running_instances_id(country)    logger.info("{} number of instances that will be stop".format(str(len(running_instances_id))))    for id in running_instances_id:        ec2.Instance(id).stop()        logger.info('Instance with id {} has been stopped'.format(id))def lambda_handler(event, context):    country=COUNTRY_CODE    logger.info('Stopping instances of: {}'.format(country))    stop_instances(country)

Custom policy for Lambda execution's Role (stop instances)

{    "Version": "2012-10-17",    "Statement": [        {            "Sid": "VisualEditor0",            "Effect": "Allow",            "Action": [                "ec2:DescribeInstances",                "ec2:StopInstances"            ],            "Resource": "*"        },        {            "Sid": "VisualEditor1",            "Effect": "Allow",            "Action": [                "logs:CreateLogStream",                "logs:CreateLogGroup",                "logs:PutLogEvents"            ],            "Resource": "arn:aws:logs:*:*:*"        }    ]}

3.- Configure EventBridge scheduler

We have code for starting and stopping instances, now we're going to configure EventBridge escheduler to run these lambdas. For this example I only mantain EC2 instances running 2 hours a day

I have configure four different schedulers:

Start instances for Spain from Monday to Friday at 9:00 AM (Europe Timezone)
Stop instances for Spain from Monday to Friday at 11:00 AM(Europe Timezone)
Start instances for Mexico from Monday to Friday at 9:00 AM (America/Mexico City Timezone)
Stop instances for Mexico from Monday to Friday at 11:00 AM(America/Mexico City Timezone)

Below you can find a screenshot of 4 schedulers