SQS

fully managed message queuing service

AWS Document | Workshop | FAQs |

Overview

• fully managed message queuing service

• provide way for asynchronously communicate between applications, without loosing messages

  • How? It stores the copies of messages redundantly across multi-AZ.

• decouple and scale microservices, distributed system and serverless applications.

SQS queue

Concepts

• Dead-letter queue (DLQ): contains messages that could not be processed.

• Short polling:

  • default polling setting.

  • WaitTimeSeconds = 0. Messages are returned immediately.

• ReceiveMessageWaitTimeSeconds: when the wait time of ReceiveMessage API > 0, long polling are in effect. Long polling help reduce empty responses.

  • Long polling:

    • WaitTimeSeconds > 0

    • Maximum wait time: 20 secs

• VisibilityTimeout: after a message is received, it still remains in the queue. The length of time which a message unavailable after delivering. To stop other consumers from processing that message again -> set visibility timeout on the ReceiveMessage API call to beyond what the longest transaction takes to process.

  • Default: 30 seconds. Min is 0 sec, Max is 12 hours.

  • Increase VisibilityTimeout if a consumer received message more than one.

aws sqs set-queue-attributes --queue-url <queue-url> --attributes VisibilityTimeout=120

• MessageRetentionPeriod: seconds that SQS will keep the message. Default is 4 days.

• DelaySeconds: seconds for which the delivery of all messages is delayed. Default is 0 second.

Benefits

• Eliminate administrative overhead.

• Keep sensitive data secure. Refer encryption.

• Reliability delivery messages

• Scale elastically and cost-effectively.

Features

• Redundantly stores messages across multiple servers.

• Automatically scale

• Built-in encryption, access control features

• Message locking: when a message received, it is locked for processing.

  • When completed, it deletes the message from the queue.

  • When failed, the lock expire -> the message is available again.

• Queue sharing: queue can be shared anonymously or to specific AWS accounts.

Queue types

-	Standard	FIFO
Throughput	unlimited	3000 msg/sec
Delivery type	Best-effort ordering (out of order)	FIFO
Duplicate?	Yes	No (exactly-once send) built-in deduplication and ordering capabilities.
Ordering	No	Yes

Use cases

Buffer to database writes

Decouple between application tiers

Integrate 2 Lambda functions (producer & consumer)

integrating serverless application

Security

Encryption

• In-flight encryption: HTTPS API

• At rest encryption: KMS keys

• Client-side encryption

• Server-side encryption (SSE): encrypt each message body

SQS Access Policies

• Similar to S3 bucket policies

• Useful for cross-account access to SQS queues.

Best practices

• Use batching: sending messages in batch can improve performance and reduce costs.

• Setting appropriate visibility timeouts

• Monitor your queues.

Trivia

• Cannot change the queue type (Standard -> FIFO or FIFO -> Standard) after you create the queue.

• Message is persisted in SQS until a consumer deletes it.

• Message retention: default 4 days, up to 14 days.

• When need a solution with decoup or performance improvement by distribute load, use SQS.