Security Group-Restricted Outbound Access to AWS Gateway VPC endpoints-S3 and DynamoDB

AWS gateway VPC endpoint allows services in the VPC to connect to S3 and DynamoDB privately. In order to access AWS gateway endpoint, security groups and NACLs in the VPC should allow outbound connection to gateway VPC endpoints. By default, security group allows all the outbound access but the best practice is to restrict outbound access and allow only required connection. Here we focus on how to add a outbound rule to security group that allows access to S3 and DynamoDB gateway endpoint using cloud formation and lambda function.

For example, Requirement is that AWS Fargate service needs access to S3 and DynamoDB via private AWS connection as shown in the diagram below and Fargate security group restricts outbound access. Existing AWS ECS cluster creation CF template needs to be updated with new security group outbound rules to allow access to S3 and DynamoDB gateway endpoint.

Solutions: To add outbound access rule to gateway endpoint, first we need to get the AWS prefix list ID(IP address ranges for an AWS service) and then use it in the security group outbound rules as destination.

This can be automated via cloud formation as below.

1. Use the following CF template to create lambda function to get AWS prefix list ID for S3 and DynamoDB gateway endpoints.

AWSTemplateFormatVersion: '2010-09-09'
Description: 'Lambda: Function To Get AWS Managed Prefix List'

Resources:
  LambdaRole:
    Type: AWS::IAM::Role
    Properties:
      RoleName: 'get-aws-managed-prefix-list-lambda-role'
      AssumeRolePolicyDocument:
        Version: "2012-10-17"
        Statement:
        - Effect: Allow
          Principal:
            Service: ["lambda.amazonaws.com"]
          Action: ["sts:AssumeRole"]
      Path: /
      Policies:
      - PolicyName: get-aws-managed-prefix-list-lambda-policy
        PolicyDocument:
          Statement:
          - Effect: Allow
            Action: 
              - ec2:DescribeManagedPrefixLists
            Resource: '*'
          - Effect: Allow
            Action:
              - logs:CreateLogGroup
              - logs:CreateLogStream
              - logs:PutLogEvents
            Resource: '*'

  LambdaFunction:
    Type: AWS::Lambda::Function
    Properties:
      Description: Get AWS Prefix List
      FunctionName: 'get-aws-managed-prefix-list-lambda'
      Role: !GetAtt LambdaRole.Arn
      Timeout: 10
      Handler: index.lambda_handler
      Runtime: python3.7
      Code:
        ZipFile: |
          import boto3
          import json
          import logging
          import sys
          import json
          import urllib3

          logger = logging.getLogger()
          logger.setLevel(logging.INFO)
          http = urllib3.PoolManager()

          def lambda_handler(event, context):
            response = {
              'StackId': event['StackId'],
              'RequestId': event['RequestId'],
              'LogicalResourceId': event['LogicalResourceId'],
              'PhysicalResourceId': 'customresource',
              'Status': 'FAILED',
              'Data': {}
            }
            cfn_response_url = event['ResponseURL']

            # There is nothing to do for a delete request
            if event['RequestType'] == 'Delete':
              logger.info('Nothing to do. Request Type : {0}'.format(event['RequestType']))
              response['Status'] = 'SUCCESS'

            elif event['RequestType'] == 'Create' or event['RequestType'] == 'Update':
                try:
                  client = boto3.client('ec2')
                  prefix_list_s3 = client.describe_managed_prefix_lists(
                                Filters=[
                                        {
                                            'Name': 'prefix-list-name',
                                            'Values': [
                                                'com.amazonaws.ap-southeast-2.s3',
                                            ]
                                        },
                                    ]
                                )
                  prefix_list_dynamodb = client.describe_managed_prefix_lists(
                                Filters=[
                                        {
                                            'Name': 'prefix-list-name',
                                            'Values': [
                                                'com.amazonaws.ap-southeast-2.dynamodb',
                                            ]
                                        },
                                    ]
                                )
                  prefix_list_id_s3 = prefix_list_s3['PrefixLists'][0]['PrefixListId']
                  prefix_list_id_dynamodb = prefix_list_dynamodb['PrefixLists'][0]['PrefixListId']

                  response['Data'] = {"prefix_list_id_s3" :  prefix_list_id_s3,
                                      "prefix_list_id_dynamodb" : prefix_list_id_dynamodb }
                  response['Status'] = 'SUCCESS'
                  logger.info(response)
                except Exception as e:
                  logging.error('Error: {0}'.format(sys.exc_info() ))
                  response['Status'] = 'FAILED'

            http.request('PUT', cfn_response_url, body=json.dumps(response).encode('utf-8'), headers={'Content-Type': 'application/json'})
            return 'Done'


Outputs:
  TemplateID:
    Description: 'Template id.'
    Value: 'lambda/get-aws-managed-prefix-list'
  TemplateVersion:
    Description: 'Template version.'
    Value: '1.0'
  StackName:
    Description: 'Stack name.'
    Value: !Sub '${AWS::StackName}'
  LambdaFunctionArn:
    Description: 'Lambda Function Arn'
    Value: !GetAtt LambdaFunction.Arn
    Export:
      Name: !Sub '${AWS::StackName}-Arn'

2. Add the above lambda function as custom resources in CF template and create security group outbound rules as follow.

AWSTemplateFormatVersion: '2010-09-09'

Resources:

  AWSPrefixList:
    Type: Custom::AWSPrefixList
    Properties:
      ServiceToken: !ImportValue LambdaFunctionArn

  TestSecurityGroup:
    Type: 'AWS::EC2::SecurityGroup'
    Properties:
      GroupName: !Sub 'test-sg'
      GroupDescription: Test security group
      VpcId: xxxxxxx
      Tags:
        - Key: Name
          Value: 'test-sg'

  OutboundS3GatewayEndpointRule:
    Type: 'AWS::EC2::SecurityGroupEgress'
    Properties:
      GroupId: !Ref TestSecurityGroup
      IpProtocol: tcp
      FromPort: 443
      ToPort: 443
      DestinationPrefixListId: !GetAtt AWSPrefixList.prefix_list_id_s3
      Description: 'Allowing access to s3 gateway VPC endpoint.'

  OutboundDynamodbGatewayEndpointRule:
    Type: 'AWS::EC2::SecurityGroupEgress'
    Properties:
      GroupId: !Ref TestSecurityGroup
      IpProtocol: tcp
      FromPort: 443
      ToPort: 443
      DestinationPrefixListId: !GetAtt AWSPrefixList.prefix_list_id_dynamodb
      Description: 'Allowing access to dynamodb gateway VPC endpoint.'

Above provided cloud formation templates will automate the creation of security group outbound rules to allow access to S3 and DynamoDB gateway endpoint. As a security best practice, it is highly recommended to restrict access to outbound connection for critical services in VPC at security group level.

Once the stack created successfully with above CF template, security group outbound rules will appear as below.