AWS SQS & MQ

2021-09-25 本文已影响0人 Lyudmilalala

Amazon SQS

Amazon SQS offers two queue types for different application requirements:

SQS can be used to:

Decouple live user requests from intensive background work
Allocate tasks to multiple worker nodes
Batch messages for future processing

Message Body max = 256KB

Allow for encryption of queues and encryption and decryption of messages

SQS only support HTTP/HTTPS protocol

Long Polling - When requests are sending to a queue to retrieve messages, doesn't return a response until a message arrives in the message queue, or the long poll times out. It reduce extraneous polling to minimize cost while receiving new messages as quickly as possible. It will wait for the WaitTimeSeconds
Long polling expect to get as many message back as possible.

Short polling - When requests are sending to a queue to retrieve messages, returns immediately, even if the message queue being polled is empty,
When queue is short, short polling is more possibly to get empty result than long polling.

Dead Letter Queues (DLQ) - Handle messages that have not been successfully processed by a consumer with Dead Letter Queues. When the maximum receive count is exceeded for a message it will be moved to the DLQ associated with the original queue. DLQs must be of the same type as the source queue (standard or FIFO).

Immediately after a message is received, it remains in the queue, because there's no guarantee that the consumer actually receives the message. To prevent other consumers from processing the message again, Amazon SQS sets a visibility timeout, a period of time during which Amazon SQS prevents other consumers from receiving and processing the message. The default visibility timeout for a message is 30 seconds.

""MaxNumberOfMessages"" - The maximum number of messages to return. Default to 1.

Standard Queue

Unlimited Throughput: Standard queues support a nearly unlimited number of transactions per second (TPS) per API action.
At-Least-Once Delivery: A message is delivered at least once, but occasionally more than one copy of a message is delivered.
Best-Effort Ordering: Occasionally, messages might be delivered in an order different from which they were sent.

You can use standard queue in scenarios that:handling a message that arrive more than once or out of order is acceptable

FIFO Queue

High Throughput: By default, FIFO queues support up to 300 messages per second. When you batch 10 messages per operation (maximum), FIFO queues can support up to 3,000 messages per second.
Exactly-Once Processing: A message is delivered once and remains available until a consumer processes and deletes it.
First-In-First-Out Delivery: The order in which messages are sent and received is strictly preserved.

You can use FIFO queue in scenarios that:

The order of operations and events is critical
Duplicated operations are not acceptable

Lifecycle

Consumer need to handle and delete message in the visibility timeout
DeleteMessage needs parameters QueueUrl, ReceiptHandle

Amazon MQ

A managed message broker service for Apache MQ.
Support IMS, NMS, AMQP, STOMP, MQTT, and Websocket, suitable for IoT.
Hybrid distribution of MQ brokers on both AWS Cloud and on-premise.