Tackling IP Whitelisting Challenges on SendGrid with Automation

Prerequisites

Active AWS Account
Active Twilio SendGrid Account

Introduction

As a DevOps engineer, I often encounter opportunities to streamline manual processes. In this article, I’ll share how I automated a repetitive task that was causing friction in production operations. Here’s some context:

In my current role, I manage a client’s application in production—an infrastructure I didn’t build or deploy. The application, which uses SendGrid for email services, runs on AWS EC2 instances behind a load balancer and an auto-scaling group. The challenge? Each time the auto-scaling group replaced instances during scaling events, the client had to manually whitelist the public IPs of the new instances on SendGrid.

This manual process was tedious and error-prone, so I set out to automate it using AWS Lambda. Here’s how I solved it.

Thought Process

To solve this problem, I explored several approaches before deciding on AWS Lambda. I used a Lambda function triggered by the Auto Scaling group whenever an instance was successfully launched or terminated. However, since the application was already in production and I hadn’t designed the infrastructure, I needed a solution that wouldn’t disrupt the existing setup.

One key challenge was ensuring the Lambda function could access the application’s Auto Scaling group. Since I didn’t want to modify the existing VPC, I created a new VPC for the Lambda function and established a VPC peering connection to the application’s VPC. This setup allowed the Lambda function to fetch the instance IPs during scaling events without interfering with the existing infrastructure.

Another hurdle was packaging the code for the Lambda function. AWS Lambda doesn’t natively support installing dependencies during runtime, so I had to install the required modules locally, bundle everything into a ZIP file, and upload it to the Lambda function. This ensured the code had all the dependencies it needed to execute properly.

Working Solution Steps;

Create a VPC for the Lambda Function
I created a dedicated VPC for the Lambda function to ensure separation from the existing application’s VPC. To enable communication between the two VPCs, I set up a VPC peering connection. This was necessary for the Lambda function to access the Auto Scaling group in the application’s VPC.
Set Up the Lambda Function
Using the AWS Console, I created a Lambda function with basic options. Then, I uploaded a ZIP file containing my code and the required dependencies.
- The code file must be named lambda_function.py before zipping and uploading.
- Dependencies were installed locally before being packaged.

Main Lambda Function Code
Below is the core Python code for the Lambda function:

 import json
 import boto3
 import os
 from sendgrid import SendGridAPIClient

 # AWS clients
 autoscaling_client = boto3.client('autoscaling')
 ec2_client = boto3.client('ec2')

 # SendGrid client
 sg = SendGridAPIClient(os.environ.get('SENDGRID_API_KEY'))

 def get_instance_ids_from_asg(asg_name):
     """
     Get instance IDs from the Auto Scaling group.
     """
     response = autoscaling_client.describe_auto_scaling_groups(
         AutoScalingGroupNames=[asg_name]
     )

     instances = response['AutoScalingGroups'][0]['Instances']
     instance_ids = [instance['InstanceId'] for instance in instances if instance['LifecycleState'] == 'InService']

     return instance_ids

 def get_public_ips(instance_ids):
     """
     Get public IPs of EC2 instances from instance IDs.
     """
     response = ec2_client.describe_instances(
         InstanceIds=instance_ids
     )

     public_ips = []

     for reservation in response['Reservations']:
         for instance in reservation['Instances']:
             if 'PublicIpAddress' in instance:
                 public_ips.append(instance['PublicIpAddress'])

     return public_ips

 def whitelist_ips(ips):
     """
     Whitelist IPs in SendGrid
     """
     data = {
         "ips": [{"ip": ip} for ip in ips]
     }

     response = sg.client.access_settings.whitelist.post(
         request_body=data
     )

     return response

 def lambda_handler(event, context):
     """
     Lambda function entry point.
     """
     # Get the Auto Scaling group name from environment variable
     asg_name = os.environ.get('ASG_NAME')

     if not asg_name:
         return {
             'statusCode': 400,
             'body': json.dumps('ASG_NAME environment variable is not set.')
         }

     # Get all instance IDs in the Auto Scaling group
     instance_ids = get_instance_ids_from_asg(asg_name)

     if not instance_ids:
         return {
             'statusCode': 404,
             'body': json.dumps('No instances found in the Auto Scaling group.')
         }

     # Get public IP addresses of instances
     public_ips = get_public_ips(instance_ids)

     if not public_ips:
         return {
             'statusCode': 404,
             'body': json.dumps('No public IPs found for the instances.')
         }

     # Whitelist the IPs in SendGrid
     sendgrid_response = whitelist_ips(public_ips)

     return {
         'statusCode': sendgrid_response.status_code,
         'body': json.dumps({
             'message': 'IPs whitelisted successfully',
             'whitelisted_ips': public_ips,
             'sendgrid_response': {
                 'body': sendgrid_response.body.decode('utf-8'),
                 'headers': dict(sendgrid_response.headers)
             }
         })
     }

Generate a SendGrid API Key
I generated a SendGrid API key from the SendGrid account, which was required for making API calls.
Configure Environment Variables
- ASG_NAME: The name of the Auto Scaling group the application runs in.
- SENDGRID_API_KEY: The API key generated in the previous step.

These variables were set in the Lambda function’s configuration tab.

Adjust Lambda Timeout
I increased the function timeout to 5 minutes (default is 3 seconds). Scaling events can take time to provision or terminate instances, so this adjustment was essential.
Attach the VPC to the Lambda Function
I attached the VPC created earlier to the Lambda function, including the necessary subnets and security groups.
Add a Trigger
I configured an AWS EventBridge trigger with the following settings:
- Event Source: Auto Scaling
- Event Types: EC2 Instance Launch Successful, EC2 Instance Terminate Successful.

Test and Deploy
After configuring everything, I tested the function. Upon verifying that the output contained the correct instance IPs, I deployed it.
Final Validation
- I logged into the SendGrid account and confirmed that the whitelisted IP list matched the instance IPs output by the Lambda function.
- With the trigger in place, any Auto Scaling events now automatically update the whitelisted IPs on SendGrid.

Conclusion

In this article, I’ve outlined how I successfully automated the process of updating SendGrid’s whitelisted IPs using AWS Lambda. By following the steps described, you can implement a similar solution to streamline IP management in dynamic environments.

I hope you found this guide helpful and insightful. Thank you for reading!

Automating AWS Instance IP Whitelisting on SendGrid with Lambda Functions

Prerequisites

Introduction

Thought Process

Working Solution Steps;

Conclusion