Amazon Simple Notification Service (Amazon SNS) is a fully managed messaging service for both application-to-application (A2A) and application-to-person (A2P) communication. We can either fan-out messages to a large number of subscribing services simultaneously using SNS topics leveraging A2A functionality or send messages to large audiences via SMS, push notifications, and emails using A2P functionality.

In this article, we will look into using go lang to publish messages to SNS topics and using message filtering to selectively trigger AWS lambda, reducing the overall infra cost.

I assume you have basic knowledge of the following prerequisites for this tutorial.

• AWS CLI and SAM CLI installed and configured.
• Knows basics of Go lang and is installed.
• And the basics of AWS CloudFormation templates.

How to leverage message filtering to reduce infrastructure costs in AWS?

In cloud-native applications, it is common to use messaging brokers to pass a message between services or to invoke the cloud functions (In AWS jargon lambda). Consider, we have a banking service that processes banking transactions. The bank's customers need to be notified about every transaction. Here customers can choose to be notified via SMS on their cell phones or via email.

The accounting service generates the alert message push it to the SNS topic. If we don’t have a filtering policy set up for the lambda invocation, for each of the messages published to the SNS, both lambda will be invoked. It is then the developer's responsibility to take care of alerting the customer. AWS uses lambda invocations, memory used, and lambda runtime among other different parameters for calculating the costs. If two lambdas are invoked for every alert message generated, then it would cost two times the actual cost needed. Now if we selectively invoke the lambdas based on the message attribute, we could reduce the cost to fifty percent.

Please refer to the video below for further clarification.

Implementation

For this tutorial, we will first create two Lambda functions using Go lang and deploy them using CloudFormation SAM templates and then an SNS topic using AWS CloudFormation. We will create Lambda triggers from the SNS CloudFormation template and attach filter policies to the SNS.

AWS Lambda

SAM template is used for deploying AWS Lambda. SAM templates are custom CloudFormation templates to deploy ServerLess Applications written in go lang. Outline of the deployment template is as below.

AWSTemplateFormatVersion: 2010-09-09
Transform: AWS::Serverless-2016-10-31
Resources:
  LambdaEmailAlert:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: build/
      Handler: main
      Role: !GetAtt LambdaExecutionRole.Arn
      Runtime: go1.x
      Timeout: 10
      MemorySize: 128
  EmailLambdaExecutionRole:
    Type: AWS::IAM::Role
    Properties:
      FunctionName: !Sub 'test-lambda-email'
      AssumeRolePolicyDocument:
        Version: "2012-10-17"
        Statement:
        - Effect: "Allow"
          Principal:
            Service:
            - "lambda.amazonaws.com"
          Action: 
          - "sts:AssumeRole"
      Path: "/"
      ManagedPolicyArns:
      - "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
Outputs:
  TestSNSMessageAttributes:
    Value: !GetAtt LambdaEmailAlert.Arn
    Export:
      Name: !Sub 'test-lambda-email'

The Go code skeleton for the lambda function is as below:

package main

import (
  "github.com/cloudfactory/service-core/lambda"
  ...
)

func handler (ctx context.Context, snsEvent events.SNSEvent) error {
  // process events here
}

func main(){
  lambda.Start(handler)
}

Finally, use the command below to deploy the lambda function to the AWS cloud.

go build -o build/main src/main.go
sam package \
  --template-file sam.yml \
  --s3-bucket <some-bucket-name> \
  --output-template-file package.yml

sam deploy --template-file package.yaml \
  --stack-name <stack-name> \
  --capabilities <iam-capabiolities> \
  --profile <aws-profile> \
  --region <aws-region>

AWS SNS

This CloudFormation template creates the SNS Topic that invokes lambda on the basis of the value of alert in the message attribute.

AWSTemplateFormatVersion: '2010-09-09'
Description: SNS topic for triggering notification lambdas
Resources:
  AlertSNSTopic:
    Type: AWS::SNS::Topic
    Properties: 
      DisplayName: !Sub 'test-sns-message-sttributes'

  SubscriptionEmail:
    Type: AWS::SNS::Subscription
    Properties:
      Endpoint: !ImportValue
        'Fn::Sub': 'test-lambda-email'
      FilterPolicy:
        alert: 
          - email
      Protocol: lambda
      TopicArn: !Ref AlertSNSTopic

  SubscriptionSMS:
    Type: AWS::SNS::Subscription
    Properties:
      Endpoint: !ImportValue
        'Fn::Sub': 'test-lambda-sms'
      FilterPolicy:
        alert: 
          - sms
      Protocol: lambda
      TopicArn: !Ref AlertSNSTopic

  AlertSNSMessageAttributesPolicy:
    Type: AWS::SNS::TopicPolicy
    Properties:
      Topics:
        - !Ref AlertSNSTopic
      PolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              AWS: '*'
            Action:
              - "SNS:Publish"
            Resource: '*'

  LambdaTriggerPermissionEmail:
    Type: AWS::Lambda::Permission
    Properties: 
      Action: 'lambda:InvokeFunction'
      FunctionName: !ImportValue
        'Fn::Sub': 'test-lambda-email'
      Principal: "sns.amazonaws.com"
      SourceArn: !Ref AlertSNSTopic

  LambdaTriggerPermissionSMS:
    Type: AWS::Lambda::Permission
    Properties: 
      Action: 'lambda:InvokeFunction'
      FunctionName: !ImportValue
        'Fn::Sub': 'test-lambda-sms'
      Principal: "sns.amazonaws.com"
      SourceArn: !Ref AlertSNSTopic

And use the following command to create an SNS topic.

aws cloudformation create-stack \
  --stack-name test-sns-message-attributes \
  --template-body file://sns.yml

Sending messages using Go code

In go, aws-sdk-go package is used to send messages to the SNS topics. The message attributes are the key-value pairs. The message attribute, in aws-go-sdk, expect the instance of the map with string keys and value of type *sns.MessageAttributeValue. It expects the message attributes in the following format.

messageAttributes:{
  "alert": {
    dataType:"String.Array",
   stringValue:"[\\\"sms\\\"]
 },
}

The general code to create a message with associated message attributes is as below:

package main
import (
 "encoding/json"
 "fmt"
 "os"
"github.com/aws/aws-sdk-go/aws"
 "github.com/aws/aws-sdk-go/aws/session"
 "github.com/aws/aws-sdk-go/service/sns"
)
func main() {
 msg := "this message is for test email lambda"
 topic := "arn:aws:sns:region:your-sns-topic-here"
 messageAttributes := make(map[string]*sns.MessageAttributeValue)
 alert := []string{"email"}
 message, _ := json.Marshal(alert)
 messageAttributes["alert"] = &sns.MessageAttributeValue{
  DataType:    aws.String("String.Array"),
  StringValue: aws.String(string(message)),
 }
// Initialize a session that the SDK will use to load
 // credentials from the shared credentials file
 //(~/.aws/credentials).
sess := session.Must(session.NewSessionWithOptions(session.Options{
  SharedConfigState: session.SharedConfigEnable,
 }))
svc := sns.New(sess)
result, err := svc.Publish(&sns.PublishInput{
  Message:           &msg,
  TopicArn:          &topic,
  MessageAttributes: messageAttributes,
 })
if err != nil {
  fmt.Println(err.Error())
  os.Exit(1)
 }
fmt.Println(*result.MessageId)
}

If we run this code with go run main.go the first lambda will be invoked. Now if we replace “alert := []string{“email”}” with “alert := []string{“sms”}” in the code and run again, then the second lambda gets executed.