Amazon Simple Notification Service (Amazon SNS) is a fully managed messaging service for both application-to-application (A2A) and application-to-person (A2P) communication. We can either fan-out messages to a large number of subscribing services simultaneously using SNS topics leveraging A2A functionality or send messages to large audiences via SMS, push notifications, and emails using A2P functionality.
In this article, we will look into using go lang to publish messages to SNS topics and using message filtering to selectively trigger AWS lambda, reducing the overall infra cost.
I assume you have basic knowledge of the following prerequisites for this tutorial.
- AWS CLI and SAM CLI installed and configured.
- Knows basics of Go lang and is installed.
- And the basics of AWS CloudFormation templates.
How to leverage message filtering to reduce infrastructure costs in AWS?
In cloud-native applications, it is common to use messaging brokers to pass a message between services or to invoke the cloud functions (In AWS jargon lambda). Consider, we have a banking service that processes banking transactions. The bank's customers need to be notified about every transaction. Here customers can choose to be notified via SMS on their cell phones or via email.
The accounting service generates the alert message push it to the SNS topic. If we don’t have a filtering policy set up for the lambda invocation, for each of the messages published to the SNS, both lambda will be invoked. It is then the developer's responsibility to take care of alerting the customer. AWS uses lambda invocations, memory used, and lambda runtime among other different parameters for calculating the costs. If two lambdas are invoked for every alert message generated, then it would cost two times the actual cost needed. Now if we selectively invoke the lambdas based on the message attribute, we could reduce the cost to fifty percent.
Please refer to the video below for further clarification.
Implementation
For this tutorial, we will first create two Lambda functions using Go lang and deploy them using CloudFormation SAM templates and then an SNS topic using AWS CloudFormation. We will create Lambda triggers from the SNS CloudFormation template and attach filter policies to the SNS.
AWS Lambda
SAM template is used for deploying AWS Lambda. SAM templates are custom CloudFormation templates to deploy ServerLess Applications written in go lang. Outline of the deployment template is as below.
AWSTemplateFormatVersion: 2010-09-09
Transform: AWS::Serverless-2016-10-31
Resources:
LambdaEmailAlert:
Type: AWS::Serverless::Function
Properties:
CodeUri: build/
Handler: main
Role: !GetAtt LambdaExecutionRole.Arn
Runtime: go1.x
Timeout: 10
MemorySize: 128
EmailLambdaExecutionRole:
Type: AWS::IAM::Role
Properties:
FunctionName: !Sub 'test-lambda-email'
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: "Allow"
Principal:
Service:
- "lambda.amazonaws.com"
Action:
- "sts:AssumeRole"
Path: "/"
ManagedPolicyArns:
- "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
Outputs:
TestSNSMessageAttributes:
Value: !GetAtt LambdaEmailAlert.Arn
Export:
Name: !Sub 'test-lambda-email'
The Go code skeleton for the lambda function is as below:
package main
import (
"github.com/cloudfactory/service-core/lambda"
...
)
func handler (ctx context.Context, snsEvent events.SNSEvent) error {
}
func main(){
lambda.Start(handler)
}
Finally, use the command below to deploy the lambda function to the AWS cloud.
go build -o build/main src/main.go
sam package \
--template-file sam.yml \
--s3-bucket <some-bucket-name> \
--output-template-file package.yml
sam deploy --template-file package.yaml \
--stack-name <stack-name> \
--capabilities <iam-capabiolities> \
--profile <aws-profile> \
--region <aws-region>
AWS SNS
This CloudFormation template creates the SNS Topic that invokes lambda on the basis of the value of alert in the message attribute.
AWSTemplateFormatVersion: '2010-09-09'
Description: SNS topic for triggering notification lambdas
Resources:
AlertSNSTopic:
Type: AWS::SNS::Topic
Properties:
DisplayName: !Sub 'test-sns-message-sttributes'
SubscriptionEmail:
Type: AWS::SNS::Subscription
Properties:
Endpoint: !ImportValue
'Fn::Sub': 'test-lambda-email'
FilterPolicy:
alert:
- email
Protocol: lambda
TopicArn: !Ref AlertSNSTopic
SubscriptionSMS:
Type: AWS::SNS::Subscription
Properties:
Endpoint: !ImportValue
'Fn::Sub': 'test-lambda-sms'
FilterPolicy:
alert:
- sms
Protocol: lambda
TopicArn: !Ref AlertSNSTopic
AlertSNSMessageAttributesPolicy:
Type: AWS::SNS::TopicPolicy
Properties:
Topics:
- !Ref AlertSNSTopic
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
AWS: '*'
Action:
- "SNS:Publish"
Resource: '*'
LambdaTriggerPermissionEmail:
Type: AWS::Lambda::Permission
Properties:
Action: 'lambda:InvokeFunction'
FunctionName: !ImportValue
'Fn::Sub': 'test-lambda-email'
Principal: "sns.amazonaws.com"
SourceArn: !Ref AlertSNSTopic
LambdaTriggerPermissionSMS:
Type: AWS::Lambda::Permission
Properties:
Action: 'lambda:InvokeFunction'
FunctionName: !ImportValue
'Fn::Sub': 'test-lambda-sms'
Principal: "sns.amazonaws.com"
SourceArn: !Ref AlertSNSTopic
And use the following command to create an SNS topic.
aws cloudformation create-stack \
--stack-name test-sns-message-attributes \
--template-body file:
Sending messages using Go code
In go, aws-sdk-go package is used to send messages to the SNS topics. The message attributes are the key-value pairs. The message attribute, in aws-go-sdk, expect the instance of the map with string keys and value of type *sns.MessageAttributeValue. It expects the message attributes in the following format.
messageAttributes:{
"alert": {
dataType:"String.Array",
stringValue:"[\\\"sms\\\"]
},
}
The general code to create a message with associated message attributes is as below:
package main
import (
"encoding/json"
"fmt"
"os"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/sns"
)
func main() {
msg := "this message is for test email lambda"
topic := "arn:aws:sns:region:your-sns-topic-here"
messageAttributes := make(map[string]*sns.MessageAttributeValue)
alert := []string{"email"}
message, _ := json.Marshal(alert)
messageAttributes["alert"] = &sns.MessageAttributeValue{
DataType: aws.String("String.Array"),
StringValue: aws.String(string(message)),
}
sess := session.Must(session.NewSessionWithOptions(session.Options{
SharedConfigState: session.SharedConfigEnable,
}))
svc := sns.New(sess)
result, err := svc.Publish(&sns.PublishInput{
Message: &msg,
TopicArn: &topic,
MessageAttributes: messageAttributes,
})
if err != nil {
fmt.Println(err.Error())
os.Exit(1)
}
fmt.Println(*result.MessageId)
}
If we run this code with go run main.go the first lambda will be invoked. Now if we replace “alert := []string{“email”}” with “alert := []string{“sms”}” in the code and run again, then the second lambda gets executed.
