How to configure lambda to access Amazon RDS with AWS CDK and Golang

Challenge

We have an Amazon RDS (PostgreSQL) in a private subnet within a Virtual Private Cloud (VPC).The database password is securely stored in AWS SecretsManager for heightened security, ensuring optimal protection. Our goal is to enable a Lambda function to securely connect to Amazon RDS by retrieving the password from AWS SecretsManager.

Solution

The key steps are as follows:

1. Place the Lambda function inside our VPC to enable secure communication with internal resources.
2. Configure the Lambda function’s role with appropriate permissions for accessing AWS services.
3. Create a VPC endpoint to allow the Lambda function to access SecretsManager without internet exposure securely.
4. Set up security groups for the Lambda function to access SecretsManager via the VPC endpoint.
5. Configure another security group to enable the Lambda function to access the RDS instance securely.
6. Apply a security group to the RDS instance that allows inbound connections from authorized sources.

We can easily test the Lambda function by directly invoking it using a Lambda function URL, which can be accessed using a simple curl command.

Our solution will be done with AWS CDK and Golang. The CDK code structure comprises three stacks:

• Network stack: This stack includes the configuration for VPC, VPC endpoint, and Lambda security groups.
• Storage stack: Here, we set up Amazon RDS PostgreSQL with credentials and security group settings.
• Application stack: This stack involves creating a Lambda function with its corresponding security groups and role, along with the Lambda function URL.

For the sake of simplicity, we’ll treat the code here as if it were consolidated into one stack. However, you can find the complete example with the CDK code divided into three stacks on our GitHub repository below.

Network

First, we create a new VPC with two public and two private subnets.

vpc := awsec2.NewVpc(stack, jsii.String("vpc"), &awsec2.VpcProps{
    IpAddresses:                  awsec2.IpAddresses_Cidr(jsii.String(CIDR)),
    MaxAzs:                       jsii.Number(2),
    NatGateways:                  jsii.Number(0),
    RestrictDefaultSecurityGroup: jsii.Bool(true),
    SubnetConfiguration: &[]*awsec2.SubnetConfiguration{
        {
            Name:       jsii.String("private-subnet-isolated"),
            SubnetType: awsec2.SubnetType_PRIVATE_ISOLATED,
            CidrMask:   jsii.Number(26),
        },
        {
            Name:       jsii.String("public-subnet"),
            SubnetType: awsec2.SubnetType_PUBLIC,
            CidrMask:   jsii.Number(26),
        },
    },
})

Then we create two security groups, one for the VPC endpoint and one for the Lambda. We need to configure the VPC endpoint security group to allow port 443 inbound traffic from the Lambda security group and the Lambda security group to allow port 443 outbound traffic to the VPC endpoint security group. We do this here because you can pass the security group to other lambdas, if needed.

func createSecurityGroup(stack awscdk.Stack, vpc awsec2.Vpc, name string) awsec2.SecurityGroup {
	return awsec2.NewSecurityGroup(stack, jsii.String(name+"sg"), &awsec2.SecurityGroupProps{
		Vpc:               vpc,
		AllowAllOutbound:  jsii.Bool(false),
		SecurityGroupName: jsii.String(name),
	})
}

secretsManagerVpcEndpointSg := createSecurityGroup(stack, vpc, "secrets-manager-vpc-endpoint")
lambdaSecretsManagerSg := createSecurityGroup(stack, vpc, "lambda-secrets-manager")

// We need to configure SecretsManagerInterfaceVPCEndpoint security group to allow port 443 inbound traffic
// from the Lambda security group and, the Lambda security group to allow port 443 outbound traffic
// to SecretsManagerInterfaceVPCEndpoint security group.
secretsManagerVpcEndpointSg.AddIngressRule(
    lambdaSecretsManagerSg,
    awsec2.Port_Tcp(jsii.Number(443)),
    jsii.String("Allow connections from lambda."),
    jsii.Bool(false))

lambdaSecretsManagerSg.AddEgressRule(
    secretsManagerVpcEndpointSg,
    awsec2.Port_Tcp(jsii.Number(443)),
    jsii.String("Allow connections to SecretsManager VPC endpoint."),
    jsii.Bool(false))

With VPC and security groups created above, we can create a VPC endpoint for the SecretsManager.

vpc.AddInterfaceEndpoint(jsii.String("secrets-manager-endpoint"), &awsec2.InterfaceVpcEndpointOptions{
    Service:           awsec2.InterfaceVpcEndpointAwsService_SECRETS_MANAGER(),
    PrivateDnsEnabled: jsii.Bool(true),
    Open:              jsii.Bool(false),
    SecurityGroups:    &[]awsec2.ISecurityGroup{secretsManagerVpcEndpointSg},
})

Storage

First, we create a security group that will allow connections to the database from the VPC.

dbSg := createSecurityGroup(stack, props.NetworkStackData.Vpc, "rds")
dbSg.AddIngressRule(
    awsec2.Peer_Ipv4(jsii.String(CIDR)),
    awsec2.Port_Tcp(jsii.Number(5432)),
    jsii.String("Allow connections to the database."),
    jsii.Bool(false),
)

Then we create a database instance with credentials. CDK will automatically create an AWS Secrets Manager secret and store it there.

cred := awsrds.Credentials_FromGeneratedSecret(jsii.String("postgres"), &awsrds.CredentialsBaseOptions{
    SecretName:        jsii.String("postgres-secret"),
    ExcludeCharacters: jsii.String("\"@/\\:"),
})

engine := awsrds.DatabaseInstanceEngine_Postgres(&awsrds.PostgresInstanceEngineProps{
    Version: awsrds.PostgresEngineVersion_VER_14_6(),
})

db := awsrds.NewDatabaseInstance(stack, jsii.String("rds"), &awsrds.DatabaseInstanceProps{
    Port:                    jsii.Number(5432),
    Engine:                  engine,
    StorageEncrypted:        jsii.Bool(true),
    MultiAz:                 jsii.Bool(false),
    AutoMinorVersionUpgrade: jsii.Bool(false),
    AllocatedStorage:        jsii.Number(25),
    StorageType:             awsrds.StorageType_GP2,
    BackupRetention:         awscdk.Duration_Days(jsii.Number(5)),
    DeletionProtection:      jsii.Bool(false),
    DatabaseName:            jsii.String("tutorial"),
    InstanceType:            awsec2.InstanceType_Of(awsec2.InstanceClass_BURSTABLE3, awsec2.InstanceSize_SMALL),
    Credentials:             cred,
    Vpc:                     props.NetworkStackData.Vpc,
    VpcSubnets: &awsec2.SubnetSelection{
        SubnetType: awsec2.SubnetType_PRIVATE_ISOLATED,
    },
    SecurityGroups: &[]awsec2.ISecurityGroup{dbSg},
    RemovalPolicy:  awscdk.RemovalPolicy_DESTROY,
})

Application

First, we create a Lambda role.

func createPingdbLambdaRole(stack awscdk.Stack) awsiam.Role {
	return awsiam.NewRole(stack, jsii.String("pingdb-lambda-role"), &awsiam.RoleProps{
		AssumedBy: awsiam.NewServicePrincipal(jsii.String("lambda.amazonaws.com"), &awsiam.ServicePrincipalOpts{}),
		ManagedPolicies: &[]awsiam.IManagedPolicy{
			awsiam.ManagedPolicy_FromManagedPolicyArn(stack, jsii.String("AWSLambdaBasicExecutionRole"), jsii.String("arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole")),
			awsiam.ManagedPolicy_FromManagedPolicyArn(stack, jsii.String("AWSLambdaVPCAccessExecutionRole"), jsii.String("arn:aws:iam::aws:policy/service-role/AWSLambdaVPCAccessExecutionRole")),
			awsiam.ManagedPolicy_FromManagedPolicyArn(stack, jsii.String("SecretsManagerReadWrite"), jsii.String("arn:aws:iam::aws:policy/SecretsManagerReadWrite")),
		},
	})
}

Then we create a security group that will allow Lambda to connect to the database.

lambdaToRdsSg := createSecurityGroup(stack, props.NetworkStackData.Vpc, "lambda-to-rds")
lambdaToRdsSg.AddEgressRule(
    awsec2.Peer_Ipv4(jsii.String(CIDR)),
    awsec2.Port_Tcp(jsii.Number(5432)),
    jsii.String("Allow connections to the database (RDS)."),
    jsii.Bool(false))

With this, we can now create our Lambda function, named pingdb, and also create the URL to trigger it.

lambda := awscdklambdagoalpha.NewGoFunction(stack, jsii.String("ping-db-lambda"), &awscdklambdagoalpha.GoFunctionProps{
    Runtime:      awslambda.Runtime_PROVIDED_AL2(),
    Architecture: awslambda.Architecture_ARM_64(),
    Entry:        jsii.String("./lambdas/pingdb"),
    Bundling: &awscdklambdagoalpha.BundlingOptions{
        GoBuildFlags: jsii.Strings(`-ldflags "-s -w"`),
    },
    Environment: &map[string]*string{
        "DB_HOST":     props.StorageStackData.TutorialDB.DbInstanceEndpointAddress(),
        "DB_PORT":     props.StorageStackData.TutorialDB.DbInstanceEndpointPort(),
        "DB_USERNAME": jsii.String("postgres"),
    },
    Role:    role,
    Timeout: awscdk.Duration_Seconds(aws.Float64(5)),
    Vpc:     props.NetworkStackData.Vpc,
    VpcSubnets: &awsec2.SubnetSelection{
        SubnetType: awsec2.SubnetType_PRIVATE_ISOLATED,
    },
    SecurityGroups: &[]awsec2.ISecurityGroup{props.NetworkStackData.LambdaSecretsManagerSg, lambdaToRdsSg},
})

lambdaUrl := awslambda.NewFunctionUrl(stack, jsii.String("pingdb-function-url"), &awslambda.FunctionUrlProps{
    Function: function,
    AuthType: awslambda.FunctionUrlAuthType_NONE,
})
lambdaUrl.GrantInvokeUrl(awsiam.NewAnyPrincipal())

awscdk.NewCfnOutput(stack, jsii.String("pingdb-function-url-output"), &awscdk.CfnOutputProps{
    ExportName:  jsii.String("pingdb-function-url"),
    Value:       lambdaUrl.Url(),
    Description: jsii.String("PingDB Function Url"),
})

You have noticed that we have environment variables for our function which we can use in our code. Speaking of code, here is the Lambda handler code to test the DB connectivity:

// HandleRequest will verify that the connection to the database is still alive,
// establishing a connection if necessary. Connection parameters are provided via environment variables and from
// SecretsManager. Credentials from SecretsManager are cached using github.com/aws/aws-secretsmanager-caching-go/secretcache
// package.
func HandleRequest() (string, error) {

	host := os.Getenv("DB_HOST")
	port := os.Getenv("DB_PORT")
	username := os.Getenv("DB_USERNAME")

	secret, err := secretCache.GetSecretString("postgres-secret")
	if err != nil {
		return "can't get secret", err
	}
	var c credentials
	err = json.Unmarshal([]byte(secret), &c)
	if err != nil {
		return "can't parse secret", err
	}
	psqlInfo := fmt.Sprintf(
		"host=%s port=%s user=%s password=%s dbname=%s sslmode=disable",
		host, port, username, c.Password, c.DbName,
	)
	db, err := sql.Open("postgres", psqlInfo)
	if err != nil {
		panic(err)
	}
	defer db.Close()

	err = db.Ping()
	if err != nil {
		return "can't ping db", err
	}

	fmt.Println("Successfully connected!")
	return "Successfully connected!", nil
}

After CDK deployment is successful, we can use the function URL, which will be in the Outputs, to trigger the Lambda function.

You can find the code here: https://github.com/ProductDock/lambda-vpc-rds-example

Other useful links and documentation:

• https://docs.aws.amazon.com/whitepapers/latest/aws-privatelink/what-are-vpc-endpoints.html
• https://docs.aws.amazon.com/aws-managed-policy/latest/reference/AWSLambdaVPCAccessExecutionRole.html
• https://docs.aws.amazon.com/aws-managed-policy/latest/reference/SecretsManagerReadWrite.html

Originally published in the ProductDock blog section