2020/12/17 - Amazon Simple Queue Service - 2 updated api methods
Changes Amazon SQS adds queue attributes to enable high throughput FIFO.
{'AttributeNames': {'FifoThroughputLimit', 'DeduplicationScope'}}
Gets attributes for the specified queue.
Note
To determine whether a queue is FIFO , you can check whether QueueName ends with the .fifo suffix.
See also: AWS API Documentation
Request Syntax
client.get_queue_attributes(
QueueUrl='string',
AttributeNames=[
'All'|'Policy'|'VisibilityTimeout'|'MaximumMessageSize'|'MessageRetentionPeriod'|'ApproximateNumberOfMessages'|'ApproximateNumberOfMessagesNotVisible'|'CreatedTimestamp'|'LastModifiedTimestamp'|'QueueArn'|'ApproximateNumberOfMessagesDelayed'|'DelaySeconds'|'ReceiveMessageWaitTimeSeconds'|'RedrivePolicy'|'FifoQueue'|'ContentBasedDeduplication'|'KmsMasterKeyId'|'KmsDataKeyReusePeriodSeconds'|'DeduplicationScope'|'FifoThroughputLimit',
]
)
string
[REQUIRED]
The URL of the Amazon SQS queue whose attribute information is retrieved.
Queue URLs and names are case-sensitive.
list
A list of attributes for which to retrieve information.
Note
In the future, new attributes might be added. If you write code that calls this action, we recommend that you structure your code so that it can handle new attributes gracefully.
The following attributes are supported:
Warning
The ApproximateNumberOfMessagesDelayed , ApproximateNumberOfMessagesNotVisible , and ApproximateNumberOfMessagesVisible metrics may not achieve consistency until at least 1 minute after the producers stop sending messages. This period is required for the queue metadata to reach eventual consistency.
All – Returns all values.
ApproximateNumberOfMessages – Returns the approximate number of messages available for retrieval from the queue.
ApproximateNumberOfMessagesDelayed – Returns the approximate number of messages in the queue that are delayed and not available for reading immediately. This can happen when the queue is configured as a delay queue or when a message has been sent with a delay parameter.
ApproximateNumberOfMessagesNotVisible – Returns the approximate number of messages that are in flight. Messages are considered to be in flight if they have been sent to a client but have not yet been deleted or have not yet reached the end of their visibility window.
CreatedTimestamp – Returns the time when the queue was created in seconds (epoch time ).
DelaySeconds – Returns the default delay on the queue in seconds.
LastModifiedTimestamp – Returns the time when the queue was last changed in seconds (epoch time ).
MaximumMessageSize – Returns the limit of how many bytes a message can contain before Amazon SQS rejects it.
MessageRetentionPeriod – Returns the length of time, in seconds, for which Amazon SQS retains a message.
Policy – Returns the policy of the queue.
QueueArn – Returns the Amazon resource name (ARN) of the queue.
ReceiveMessageWaitTimeSeconds – Returns the length of time, in seconds, for which the ReceiveMessage action waits for a message to arrive.
RedrivePolicy – The string that includes the parameters for the dead-letter queue functionality of the source queue as a JSON object. For more information about the redrive policy and dead-letter queues, see Using Amazon SQS Dead-Letter Queues in the Amazon Simple Queue Service Developer Guide .
deadLetterTargetArn – The Amazon Resource Name (ARN) of the dead-letter queue to which Amazon SQS moves messages after the value of maxReceiveCount is exceeded.
maxReceiveCount – The number of times a message is delivered to the source queue before being moved to the dead-letter queue. When the ReceiveCount for a message exceeds the maxReceiveCount for a queue, Amazon SQS moves the message to the dead-letter-queue.
VisibilityTimeout – Returns the visibility timeout for the queue. For more information about the visibility timeout, see Visibility Timeout in the Amazon Simple Queue Service Developer Guide .
The following attributes apply only to server-side-encryption :
KmsMasterKeyId – Returns the ID of an AWS-managed customer master key (CMK) for Amazon SQS or a custom CMK. For more information, see Key Terms .
KmsDataKeyReusePeriodSeconds – Returns the length of time, in seconds, for which Amazon SQS can reuse a data key to encrypt or decrypt messages before calling AWS KMS again. For more information, see How Does the Data Key Reuse Period Work? .
The following attributes apply only to FIFO (first-in-first-out) queues :
FifoQueue – Returns information about whether the queue is FIFO. For more information, see FIFO Queue Logic in the Amazon Simple Queue Service Developer Guide .
Note
To determine whether a queue is FIFO , you can check whether QueueName ends with the .fifo suffix.
ContentBasedDeduplication – Returns whether content-based deduplication is enabled for the queue. For more information, see Exactly-Once Processing in the Amazon Simple Queue Service Developer Guide .
Preview: High throughput for FIFO queues
High throughput for Amazon SQS FIFO queues is in preview release and is subject to change. This feature provides a high number of transactions per second (TPS) for messages in FIFO queues. For information on throughput quotas, see Quotas related to messages in the Amazon Simple Queue Service Developer Guide .
This preview includes two new attributes:
DeduplicationScope – Specifies whether message deduplication occurs at the message group or queue level. Valid values are messageGroup and queue .
FifoThroughputLimit – Specifies whether the FIFO queue throughput quota applies to the entire queue or per message group. Valid values are perQueue and perMessageGroupId . The perMessageGroupId value is allowed only when the value for DeduplicationScope is messageGroup .
To enable high throughput for FIFO queues, do the following:
Set DeduplicationScope to messageGroup .
Set FifoThroughputLimit to perMessageGroupId .
If you set these attributes to anything other than the values shown for enabling high throughput, standard throughput is in effect and deduplication occurs as specified.
This preview is available in the following AWS Regions:
US East (Ohio); us-east-2
US East (N. Virginia); us-east-1
US West (Oregon); us-west-2
Europe (Ireland); eu-west-1
For more information about high throughput for FIFO queues, see Preview: High throughput for FIFO queues in the Amazon Simple Queue Service Developer Guide .
(string) --
dict
Response Syntax
{
'Attributes': {
'string': 'string'
}
}
Response Structure
(dict) --
A list of returned queue attributes.
Attributes (dict) --
A map of attributes to their respective values.
(string) --
(string) --
{'AttributeNames': {'FifoThroughputLimit', 'DeduplicationScope'}}
Retrieves one or more messages (up to 10), from the specified queue. Using the WaitTimeSeconds parameter enables long-poll support. For more information, see Amazon SQS Long Polling in the Amazon Simple Queue Service Developer Guide .
Short poll is the default behavior where a weighted random set of machines is sampled on a ReceiveMessage call. Thus, only the messages on the sampled machines are returned. If the number of messages in the queue is small (fewer than 1,000), you most likely get fewer messages than you requested per ReceiveMessage call. If the number of messages in the queue is extremely small, you might not receive any messages in a particular ReceiveMessage response. If this happens, repeat the request.
For each message returned, the response includes the following:
The message body.
An MD5 digest of the message body. For information about MD5, see RFC1321 .
The MessageId you received when you sent the message to the queue.
The receipt handle.
The message attributes.
An MD5 digest of the message attributes.
The receipt handle is the identifier you must provide when deleting the message. For more information, see Queue and Message Identifiers in the Amazon Simple Queue Service Developer Guide .
You can provide the VisibilityTimeout parameter in your request. The parameter is applied to the messages that Amazon SQS returns in the response. If you don't include the parameter, the overall visibility timeout for the queue is used for the returned messages. For more information, see Visibility Timeout in the Amazon Simple Queue Service Developer Guide .
A message that isn't deleted or a message whose visibility isn't extended before the visibility timeout expires counts as a failed receive. Depending on the configuration of the queue, the message might be sent to the dead-letter queue.
Note
In the future, new attributes might be added. If you write code that calls this action, we recommend that you structure your code so that it can handle new attributes gracefully.
See also: AWS API Documentation
Request Syntax
client.receive_message(
QueueUrl='string',
AttributeNames=[
'All'|'Policy'|'VisibilityTimeout'|'MaximumMessageSize'|'MessageRetentionPeriod'|'ApproximateNumberOfMessages'|'ApproximateNumberOfMessagesNotVisible'|'CreatedTimestamp'|'LastModifiedTimestamp'|'QueueArn'|'ApproximateNumberOfMessagesDelayed'|'DelaySeconds'|'ReceiveMessageWaitTimeSeconds'|'RedrivePolicy'|'FifoQueue'|'ContentBasedDeduplication'|'KmsMasterKeyId'|'KmsDataKeyReusePeriodSeconds'|'DeduplicationScope'|'FifoThroughputLimit',
],
MessageAttributeNames=[
'string',
],
MaxNumberOfMessages=123,
VisibilityTimeout=123,
WaitTimeSeconds=123,
ReceiveRequestAttemptId='string'
)
string
[REQUIRED]
The URL of the Amazon SQS queue from which messages are received.
Queue URLs and names are case-sensitive.
list
A list of attributes that need to be returned along with each message. These attributes include:
All – Returns all values.
ApproximateFirstReceiveTimestamp – Returns the time the message was first received from the queue (epoch time in milliseconds).
ApproximateReceiveCount – Returns the number of times a message has been received across all queues but not deleted.
AWSTraceHeader – Returns the AWS X-Ray trace header string.
SenderId
For an IAM user, returns the IAM user ID, for example ABCDEFGHI1JKLMNOPQ23R .
For an IAM role, returns the IAM role ID, for example ABCDE1F2GH3I4JK5LMNOP:i-a123b456 .
SentTimestamp – Returns the time the message was sent to the queue (epoch time in milliseconds).
MessageDeduplicationId – Returns the value provided by the producer that calls the `` SendMessage `` action.
MessageGroupId – Returns the value provided by the producer that calls the `` SendMessage `` action. Messages with the same MessageGroupId are returned in sequence.
SequenceNumber – Returns the value provided by Amazon SQS.
(string) --
list
The name of the message attribute, where N is the index.
The name can contain alphanumeric characters and the underscore (_ ), hyphen (- ), and period (. ).
The name is case-sensitive and must be unique among all attribute names for the message.
The name must not start with AWS-reserved prefixes such as AWS. or Amazon. (or any casing variants).
The name must not start or end with a period (. ), and it should not have periods in succession (.. ).
The name can be up to 256 characters long.
When using ReceiveMessage , you can send a list of attribute names to receive, or you can return all of the attributes by specifying All or .* in your request. You can also use all message attributes starting with a prefix, for example bar.* .
(string) --
integer
The maximum number of messages to return. Amazon SQS never returns more messages than this value (however, fewer messages might be returned). Valid values: 1 to 10. Default: 1.
integer
The duration (in seconds) that the received messages are hidden from subsequent retrieve requests after being retrieved by a ReceiveMessage request.
integer
The duration (in seconds) for which the call waits for a message to arrive in the queue before returning. If a message is available, the call returns sooner than WaitTimeSeconds . If no messages are available and the wait time expires, the call returns successfully with an empty list of messages.
Warning
To avoid HTTP errors, ensure that the HTTP response timeout for ReceiveMessage requests is longer than the WaitTimeSeconds parameter. For example, with the Java SDK, you can set HTTP transport settings using the NettyNioAsyncHttpClient for asynchronous clients, or the ApacheHttpClient for synchronous clients.
string
This parameter applies only to FIFO (first-in-first-out) queues.
The token used for deduplication of ReceiveMessage calls. If a networking issue occurs after a ReceiveMessage action, and instead of a response you receive a generic error, it is possible to retry the same action with an identical ReceiveRequestAttemptId to retrieve the same set of messages, even if their visibility timeout has not yet expired.
You can use ReceiveRequestAttemptId only for 5 minutes after a ReceiveMessage action.
When you set FifoQueue , a caller of the ReceiveMessage action can provide a ReceiveRequestAttemptId explicitly.
If a caller of the ReceiveMessage action doesn't provide a ReceiveRequestAttemptId , Amazon SQS generates a ReceiveRequestAttemptId .
It is possible to retry the ReceiveMessage action with the same ReceiveRequestAttemptId if none of the messages have been modified (deleted or had their visibility changes).
During a visibility timeout, subsequent calls with the same ReceiveRequestAttemptId return the same messages and receipt handles. If a retry occurs within the deduplication interval, it resets the visibility timeout. For more information, see Visibility Timeout in the Amazon Simple Queue Service Developer Guide .
Warning
If a caller of the ReceiveMessage action still processes messages when the visibility timeout expires and messages become visible, another worker consuming from the same queue can receive the same messages and therefore process duplicates. Also, if a consumer whose message processing time is longer than the visibility timeout tries to delete the processed messages, the action fails with an error. To mitigate this effect, ensure that your application observes a safe threshold before the visibility timeout expires and extend the visibility timeout as necessary.
While messages with a particular MessageGroupId are invisible, no more messages belonging to the same MessageGroupId are returned until the visibility timeout expires. You can still receive messages with another MessageGroupId as long as it is also visible.
If a caller of ReceiveMessage can't track the ReceiveRequestAttemptId , no retries work until the original visibility timeout expires. As a result, delays might occur but the messages in the queue remain in a strict order.
The maximum length of ReceiveRequestAttemptId is 128 characters. ReceiveRequestAttemptId can contain alphanumeric characters (a-z , A-Z , 0-9 ) and punctuation (!"#$%&'()*+,-./:;<=>?@[\]^_`{|}~ ).
For best practices of using ReceiveRequestAttemptId , see Using the ReceiveRequestAttemptId Request Parameter in the Amazon Simple Queue Service Developer Guide .
dict
Response Syntax
{
'Messages': [
{
'MessageId': 'string',
'ReceiptHandle': 'string',
'MD5OfBody': 'string',
'Body': 'string',
'Attributes': {
'string': 'string'
},
'MD5OfMessageAttributes': 'string',
'MessageAttributes': {
'string': {
'StringValue': 'string',
'BinaryValue': b'bytes',
'StringListValues': [
'string',
],
'BinaryListValues': [
b'bytes',
],
'DataType': 'string'
}
}
},
]
}
Response Structure
(dict) --
A list of received messages.
Messages (list) --
A list of messages.
(dict) --
An Amazon SQS message.
MessageId (string) --
A unique identifier for the message. A MessageId is considered unique across all AWS accounts for an extended period of time.
ReceiptHandle (string) --
An identifier associated with the act of receiving the message. A new receipt handle is returned every time you receive a message. When deleting a message, you provide the last received receipt handle to delete the message.
MD5OfBody (string) --
An MD5 digest of the non-URL-encoded message body string.
Body (string) --
The message's contents (not URL-encoded).
Attributes (dict) --
A map of the attributes requested in `` ReceiveMessage `` to their respective values. Supported attributes:
ApproximateReceiveCount
ApproximateFirstReceiveTimestamp
MessageDeduplicationId
MessageGroupId
SenderId
SentTimestamp
SequenceNumber
ApproximateFirstReceiveTimestamp and SentTimestamp are each returned as an integer representing the epoch time in milliseconds.
(string) --
(string) --
MD5OfMessageAttributes (string) --
An MD5 digest of the non-URL-encoded message attribute string. You can use this attribute to verify that Amazon SQS received the message correctly. Amazon SQS URL-decodes the message before creating the MD5 digest. For information about MD5, see RFC1321 .
MessageAttributes (dict) --
Each message attribute consists of a Name , Type , and Value . For more information, see Amazon SQS Message Attributes in the Amazon Simple Queue Service Developer Guide .
(string) --
(dict) --
The user-specified message attribute value. For string data types, the Value attribute has the same restrictions on the content as the message body. For more information, see `` SendMessage .``
Name , type , value and the message body must not be empty or null. All parts of the message attribute, including Name , Type , and Value , are part of the message size restriction (256 KB or 262,144 bytes).
StringValue (string) --
Strings are Unicode with UTF-8 binary encoding. For a list of code values, see ASCII Printable Characters .
BinaryValue (bytes) --
Binary type attributes can store any binary data, such as compressed data, encrypted data, or images.
StringListValues (list) --
Not implemented. Reserved for future use.
(string) --
BinaryListValues (list) --
Not implemented. Reserved for future use.
(bytes) --
DataType (string) --
Amazon SQS supports the following logical data types: String , Number , and Binary . For the Number data type, you must use StringValue .
You can also append custom labels. For more information, see Amazon SQS Message Attributes in the Amazon Simple Queue Service Developer Guide .