Whats the difference between SNS and SQS service provided by AWS

AWS provide 2 services called SNS and SQS, Whats the difference ? When to use which service ?

Introduction

While building any application, we encounter a scenario where we might have to consider getting a huge amount of data and processing it synchronously.

What do I mean by that ? Some examples :

• Picture resize : User upload a picture to server, and server responds with success without doing any processing on the picture, but in background server sends this picture for Thumbnail creation, resizing, etc to a message queue.

• Sending Emails : User can upload 1000s of emails and we queue them on background without letting user wait for a response to process in a queue 1 by 1.

• Guaranteed delivery of webhooks : Lets say you get a webhook from twitter anytime a user tweets and your application than converts these webhooks into messages which sent to Slack (very hypothetical use case). lets say your app grew very popular and 10000s of webhooks are being received on your server, server goes down, which means you miss some tweets.

  To guarantee the delivery we can use a message queuing system like Kafka or SQS. where the webhooks are stored in a queue and removed 1 by 1 after proper processing.

Similarly to scale a server, message queuing, a very widely used technique used around the app development domain.

Today in this post we are going to talk about 2 services which are confusing in terms of the usage, SNS and SQS

What is Amazon SNS ?

SNS short for Simple Notification Service is the pub/sub service of AWS. it allows sending message to topics. consumer can subscribe to topics and decide if they want to receive messages or not. The message get fanned out to many subs at the same time.

SNS differentiates into two types of destinations. Application to Application and Application to Person

- Application to Application Services - Kinesis Firehose - Lambda - SQS

- Application to Person Services - Email - SMS - Chatbot

What is Amazon SQS

SQS short for Simple Queuing Service, is the managed queueing service by AWS. It is a poll-based service. producers send messages to a queue and consumers pick them up. compared to SNS which is a push based service where the receiver does not have to poll the queue in SQS the consumer has to make a polling request.

SQS is the oldest AWS Service 👴

As shown in the picture, the clients send a message to SQS which is stored, could be a request or a webhook from external service, later the Lambda need to poll the queue and process the messages.

What are the differences ?

Both SNS and SQS handle messages in same way. but they differ in their very nature, as we just explained SNS is a push based vs SQS being poll based

• SNS can be many to many vs SQS being many to one.
• SQS is always App to App, vs SNS being App to App or App to Person.
• The messages in SQS are always persistence, where as SNS only pushes the message to app or person and do not store it in any memory.
• Reliability on SQS is higher as it has persistence.
• SQS also provides Batching

𝗣𝘂𝘀𝗵 𝘃𝘀. 𝗣𝗼𝗹𝗹 SNS is a push based service. Producer send messages and SNS simply forwards the messages to all subscribers SQS is poll based. That means producer send messages to SQS. SQS saves the messages till a consumer picks them up and deletes it.

Use cases for SNS

• You have many subscribers to a topic and want up to date info about any event
• You need to send SNS, Email, or In-App Notifications
• You want to use a fanout pattern (https://go.aws/3tMxJl8)
• You want to send several Emails and SMS if a cloudwatch alarm is red, this is easily possible with SNS. No Persistence, batching, or retries are necessary.
• You got a new follower in a social media platform, this is easily possible with SNS since it natively supports In-app Notifications.

Use cases for SQS

• Your only need one subscriber (your server)
• Persistence and error handeling is really important
• You need to batch the requests
• You want to simply decouple your sever or application
• You run a quiz where people vote for best spaceship at a certain time.
  • You need to handle tons of votes
  • You need to persist this data
  • You need to reliably work with this data
  • Your service simple sends message to SQS and SQS takes care of the load.

SNS and SQS Together with a Fanout Pattern

A very common scenario is that you need both services in conjunction. This is called the Fanout pattern SNS takes care of sending all message to different SQS queues and SQS takes care of the reliability.

Examples:

- Image Uploads : - You could have several services that needs to start when a user uploads an image - Image Recognition - Creation of Thumbnail - Notify User

- Social Media Post Processing - You have a social media platform and want to have several actions for each post published like - Translate Post - Create an Audio Version - Update Statistics - Notify users via E-mail and In-App Notifications

SNS and SQS are both awesome services but there are a lot of things to learn, almost every application you will build would need both SNS and SQS at some point, and its better to know which is better at what.

I hope this post helped you in some way. If it did please follow me on Twitter for more such content and I am also planning to start my email newsletter so please feel free to subscribe to it.