Security Group-Restricted Outbound Access to AWS Gateway VPC endpoints-S3 and DynamoDB
AWS gateway VPC endpoint allows services in the VPC to connect to S3 and DynamoDB privately. In order to access AWS gateway endpoint, security groups and NACLs in the VPC should allow outbound connection to gateway VPC endpoints. By default, security group allows all the outbound access but the best practice is to restrict outbound access and allow only required connection. Here we focus on how to add a outbound rule to security group that allows access to S3 and DynamoDB gateway endpoint using cloud formation and lambda function.
For example, Requirement is that AWS Fargate service needs access to S3 and DynamoDB via private AWS connection as shown in the diagram below and Fargate security group restricts outbound access. Existing AWS ECS cluster creation CF template needs to be updated with new security group outbound rules to allow access to S3 and DynamoDB gateway endpoint.
Solutions: To add outbound access rule to gateway endpoint, first we need to get the AWS prefix list ID(IP address ranges for an AWS service) and then use it in the security group outbound rules as destination.
This can be automated via cloud formation as below.
- Use the following CF template to create lambda function to get AWS prefix list ID for S3 and DynamoDB gateway endpoints.
AWSTemplateFormatVersion: '2010-09-09'
Description: 'Lambda: Function To Get AWS Managed Prefix List'
Resources:
LambdaRole:
Type: AWS::IAM::Role
Properties:
RoleName: 'get-aws-managed-prefix-list-lambda-role'
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Principal:
Service: ["lambda.amazonaws.com"]
Action: ["sts:AssumeRole"]
Path: /
Policies:
- PolicyName: get-aws-managed-prefix-list-lambda-policy
PolicyDocument:
Statement:
- Effect: Allow
Action:
- ec2:DescribeManagedPrefixLists
Resource: '*'
- Effect: Allow
Action:
- logs:CreateLogGroup
- logs:CreateLogStream
- logs:PutLogEvents
Resource: '*'
LambdaFunction:
Type: AWS::Lambda::Function
Properties:
Description: Get AWS Prefix List
FunctionName: 'get-aws-managed-prefix-list-lambda'
Role: !GetAtt LambdaRole.Arn
Timeout: 10
Handler: index.lambda_handler
Runtime: python3.7
Code:
ZipFile: |
import boto3
import json
import logging
import sys
import json
import urllib3
logger = logging.getLogger()
logger.setLevel(logging.INFO)
http = urllib3.PoolManager()
def lambda_handler(event, context):
response = {
'StackId': event['StackId'],
'RequestId': event['RequestId'],
'LogicalResourceId': event['LogicalResourceId'],
'PhysicalResourceId': 'customresource',
'Status': 'FAILED',
'Data': {}
}
cfn_response_url = event['ResponseURL']
# There is nothing to do for a delete request
if event['RequestType'] == 'Delete':
logger.info('Nothing to do. Request Type : {0}'.format(event['RequestType']))
response['Status'] = 'SUCCESS'
elif event['RequestType'] == 'Create' or event['RequestType'] == 'Update':
try:
client = boto3.client('ec2')
prefix_list_s3 = client.describe_managed_prefix_lists(
Filters=[
{
'Name': 'prefix-list-name',
'Values': [
'com.amazonaws.ap-southeast-2.s3',
]
},
]
)
prefix_list_dynamodb = client.describe_managed_prefix_lists(
Filters=[
{
'Name': 'prefix-list-name',
'Values': [
'com.amazonaws.ap-southeast-2.dynamodb',
]
},
]
)
prefix_list_id_s3 = prefix_list_s3['PrefixLists'][0]['PrefixListId']
prefix_list_id_dynamodb = prefix_list_dynamodb['PrefixLists'][0]['PrefixListId']
response['Data'] = {"prefix_list_id_s3" : prefix_list_id_s3,
"prefix_list_id_dynamodb" : prefix_list_id_dynamodb }
response['Status'] = 'SUCCESS'
logger.info(response)
except Exception as e:
logging.error('Error: {0}'.format(sys.exc_info() ))
response['Status'] = 'FAILED'
http.request('PUT', cfn_response_url, body=json.dumps(response).encode('utf-8'), headers={'Content-Type': 'application/json'})
return 'Done'
Outputs:
TemplateID:
Description: 'Template id.'
Value: 'lambda/get-aws-managed-prefix-list'
TemplateVersion:
Description: 'Template version.'
Value: '1.0'
StackName:
Description: 'Stack name.'
Value: !Sub '${AWS::StackName}'
LambdaFunctionArn:
Description: 'Lambda Function Arn'
Value: !GetAtt LambdaFunction.Arn
Export:
Name: !Sub '${AWS::StackName}-Arn'2. Add the above lambda function as custom resources in CF template and create security group outbound rules as follow.
AWSTemplateFormatVersion: '2010-09-09'
Resources:
AWSPrefixList:
Type: Custom::AWSPrefixList
Properties:
ServiceToken: !ImportValue LambdaFunctionArn
TestSecurityGroup:
Type: 'AWS::EC2::SecurityGroup'
Properties:
GroupName: !Sub 'test-sg'
GroupDescription: Test security group
VpcId: xxxxxxx
Tags:
- Key: Name
Value: 'test-sg'
OutboundS3GatewayEndpointRule:
Type: 'AWS::EC2::SecurityGroupEgress'
Properties:
GroupId: !Ref TestSecurityGroup
IpProtocol: tcp
FromPort: 443
ToPort: 443
DestinationPrefixListId: !GetAtt AWSPrefixList.prefix_list_id_s3
Description: 'Allowing access to s3 gateway VPC endpoint.'
OutboundDynamodbGatewayEndpointRule:
Type: 'AWS::EC2::SecurityGroupEgress'
Properties:
GroupId: !Ref TestSecurityGroup
IpProtocol: tcp
FromPort: 443
ToPort: 443
DestinationPrefixListId: !GetAtt AWSPrefixList.prefix_list_id_dynamodb
Description: 'Allowing access to dynamodb gateway VPC endpoint.'Above provided cloud formation templates will automate the creation of security group outbound rules to allow access to S3 and DynamoDB gateway endpoint. As a security best practice, it is highly recommended to restrict access to outbound connection for critical services in VPC at security group level.
Once the stack created successfully with above CF template, security group outbound rules will appear as below.
