2018/08/02 - Amazon Kinesis - 4 new 2 updated api methods
Changes Updates API to latest version.
Registers a consumer with a Kinesis data stream. When you use this operation, the consumer you register can read data from the stream at a rate of up to 2 MiB per second. This rate is unaffected by the total number of consumers that read from the same stream.
You can register up to 5 consumers per stream. A given consumer can only be registered with one stream.
This operation has a limit of five transactions per second per account.
See also: AWS API Documentation
Request Syntax
client.register_stream_consumer(
StreamARN='string',
ConsumerName='string'
)
string
[REQUIRED]
The ARN of the Kinesis data stream that you want to register the consumer with. For more info, see Amazon Resource Names (ARNs) and AWS Service Namespaces .
string
[REQUIRED]
For a given Kinesis data stream, each consumer must have a unique name. However, consumer names don't have to be unique across data streams.
dict
Response Syntax
{
'Consumer': {
'ConsumerName': 'string',
'ConsumerARN': 'string',
'ConsumerStatus': 'CREATING'|'DELETING'|'ACTIVE',
'ConsumerCreationTimestamp': datetime(2015, 1, 1)
}
}
Response Structure
(dict) --
Consumer (dict) --
An object that represents the details of the consumer you registered. When you register a consumer, it gets an ARN that is generated by Kinesis Data Streams.
ConsumerName (string) --
The name of the consumer is something you choose when you register the consumer.
ConsumerARN (string) --
When you register a consumer, Kinesis Data Streams generates an ARN for it. You need this ARN to be able to call SubscribeToShard .
If you delete a consumer and then create a new one with the same name, it won't have the same ARN. That's because consumer ARNs contain the creation timestamp. This is important to keep in mind if you have IAM policies that reference consumer ARNs.
ConsumerStatus (string) --
A consumer can't read data while in the CREATING or DELETING states.
ConsumerCreationTimestamp (datetime) --
Lists the consumers registered to receive data from a stream using enhanced fan-out, and provides information about each consumer.
This operation has a limit of 10 transactions per second per account.
See also: AWS API Documentation
Request Syntax
client.list_stream_consumers(
StreamARN='string',
NextToken='string',
MaxResults=123,
StreamCreationTimestamp=datetime(2015, 1, 1)
)
string
[REQUIRED]
The ARN of the Kinesis data stream for which you want to list the registered consumers. For more information, see Amazon Resource Names (ARNs) and AWS Service Namespaces .
string
When the number of consumers that are registered with the data stream is greater than the default value for the MaxResults parameter, or if you explicitly specify a value for MaxResults that is less than the number of consumers that are registered with the data stream, the response includes a pagination token named NextToken . You can specify this NextToken value in a subsequent call to ListStreamConsumers to list the next set of registered consumers.
Don't specify StreamName or StreamCreationTimestamp if you specify NextToken because the latter unambiguously identifies the stream.
You can optionally specify a value for the MaxResults parameter when you specify NextToken . If you specify a MaxResults value that is less than the number of consumers that the operation returns if you don't specify MaxResults , the response will contain a new NextToken value. You can use the new NextToken value in a subsequent call to the ListStreamConsumers operation to list the next set of consumers.
Warning
Tokens expire after 300 seconds. When you obtain a value for NextToken in the response to a call to ListStreamConsumers , you have 300 seconds to use that value. If you specify an expired token in a call to ListStreamConsumers , you get ExpiredNextTokenException .
integer
The maximum number of consumers that you want a single call of ListStreamConsumers to return.
datetime
Specify this input parameter to distinguish data streams that have the same name. For example, if you create a data stream and then delete it, and you later create another data stream with the same name, you can use this input parameter to specify which of the two streams you want to list the consumers for.
You can't specify this parameter if you specify the NextToken parameter.
dict
Response Syntax
{
'Consumers': [
{
'ConsumerName': 'string',
'ConsumerARN': 'string',
'ConsumerStatus': 'CREATING'|'DELETING'|'ACTIVE',
'ConsumerCreationTimestamp': datetime(2015, 1, 1)
},
],
'NextToken': 'string'
}
Response Structure
(dict) --
Consumers (list) --
An array of JSON objects. Each object represents one registered consumer.
(dict) --
An object that represents the details of the consumer you registered.
ConsumerName (string) --
The name of the consumer is something you choose when you register the consumer.
ConsumerARN (string) --
When you register a consumer, Kinesis Data Streams generates an ARN for it. You need this ARN to be able to call SubscribeToShard .
If you delete a consumer and then create a new one with the same name, it won't have the same ARN. That's because consumer ARNs contain the creation timestamp. This is important to keep in mind if you have IAM policies that reference consumer ARNs.
ConsumerStatus (string) --
A consumer can't read data while in the CREATING or DELETING states.
ConsumerCreationTimestamp (datetime) --
NextToken (string) --
When the number of consumers that are registered with the data stream is greater than the default value for the MaxResults parameter, or if you explicitly specify a value for MaxResults that is less than the number of registered consumers, the response includes a pagination token named NextToken . You can specify this NextToken value in a subsequent call to ListStreamConsumers to list the next set of registered consumers. For more information about the use of this pagination token when calling the ListStreamConsumers operation, see ListStreamConsumersInput$NextToken .
Warning
Tokens expire after 300 seconds. When you obtain a value for NextToken in the response to a call to ListStreamConsumers , you have 300 seconds to use that value. If you specify an expired token in a call to ListStreamConsumers , you get ExpiredNextTokenException .
To deregister a consumer, provide its ARN. Alternatively, you can provide the ARN of the data stream and the name you gave the consumer when you registered it. You may also provide all three parameters, as long as they don't conflict with each other. If you don't know the name or ARN of the consumer that you want to deregister, you can use the ListStreamConsumers operation to get a list of the descriptions of all the consumers that are currently registered with a given data stream. The description of a consumer contains its name and ARN.
This operation has a limit of five transactions per second per account.
See also: AWS API Documentation
Request Syntax
client.deregister_stream_consumer(
StreamARN='string',
ConsumerName='string',
ConsumerARN='string'
)
string
The ARN of the Kinesis data stream that the consumer is registered with. For more information, see Amazon Resource Names (ARNs) and AWS Service Namespaces .
string
The name that you gave to the consumer.
string
The ARN returned by Kinesis Data Streams when you registered the consumer. If you don't know the ARN of the consumer that you want to deregister, you can use the ListStreamConsumers operation to get a list of the descriptions of all the consumers that are currently registered with a given data stream. The description of a consumer contains its ARN.
None
To get the description of a registered consumer, provide the ARN of the consumer. Alternatively, you can provide the ARN of the data stream and the name you gave the consumer when you registered it. You may also provide all three parameters, as long as they don't conflict with each other. If you don't know the name or ARN of the consumer that you want to describe, you can use the ListStreamConsumers operation to get a list of the descriptions of all the consumers that are currently registered with a given data stream.
This operation has a limit of 20 transactions per second per account.
See also: AWS API Documentation
Request Syntax
client.describe_stream_consumer(
StreamARN='string',
ConsumerName='string',
ConsumerARN='string'
)
string
The ARN of the Kinesis data stream that the consumer is registered with. For more information, see Amazon Resource Names (ARNs) and AWS Service Namespaces .
string
The name that you gave to the consumer.
string
The ARN returned by Kinesis Data Streams when you registered the consumer.
dict
Response Syntax
{
'ConsumerDescription': {
'ConsumerName': 'string',
'ConsumerARN': 'string',
'ConsumerStatus': 'CREATING'|'DELETING'|'ACTIVE',
'ConsumerCreationTimestamp': datetime(2015, 1, 1),
'StreamARN': 'string'
}
}
Response Structure
(dict) --
ConsumerDescription (dict) --
An object that represents the details of the consumer.
ConsumerName (string) --
The name of the consumer is something you choose when you register the consumer.
ConsumerARN (string) --
When you register a consumer, Kinesis Data Streams generates an ARN for it. You need this ARN to be able to call SubscribeToShard .
If you delete a consumer and then create a new one with the same name, it won't have the same ARN. That's because consumer ARNs contain the creation timestamp. This is important to keep in mind if you have IAM policies that reference consumer ARNs.
ConsumerStatus (string) --
A consumer can't read data while in the CREATING or DELETING states.
ConsumerCreationTimestamp (datetime) --
StreamARN (string) --
The ARN of the stream with which you registered the consumer.
{'EnforceConsumerDeletion': 'boolean'}
Deletes a Kinesis data stream and all its shards and data. You must shut down any applications that are operating on the stream before you delete the stream. If an application attempts to operate on a deleted stream, it receives the exception ResourceNotFoundException .
If the stream is in the ACTIVE state, you can delete it. After a DeleteStream request, the specified stream is in the DELETING state until Kinesis Data Streams completes the deletion.
Note: Kinesis Data Streams might continue to accept data read and write operations, such as PutRecord , PutRecords , and GetRecords , on a stream in the DELETING state until the stream deletion is complete.
When you delete a stream, any shards in that stream are also deleted, and any tags are dissociated from the stream.
You can use the DescribeStream operation to check the state of the stream, which is returned in StreamStatus .
DeleteStream has a limit of five transactions per second per account.
See also: AWS API Documentation
Request Syntax
client.delete_stream(
StreamName='string',
EnforceConsumerDeletion=True|False
)
string
[REQUIRED]
The name of the stream to delete.
boolean
If this parameter is unset (null ) or if you set it to false , and the stream has registered consumers, the call to DeleteStream fails with a ResourceInUseException .
None
{'StreamDescriptionSummary': {'ConsumerCount': 'integer'}}
Provides a summarized description of the specified Kinesis data stream without the shard list.
The information returned includes the stream name, Amazon Resource Name (ARN), status, record retention period, approximate creation time, monitoring, encryption details, and open shard count.
See also: AWS API Documentation
Request Syntax
client.describe_stream_summary(
StreamName='string'
)
string
[REQUIRED]
The name of the stream to describe.
dict
Response Syntax
{
'StreamDescriptionSummary': {
'StreamName': 'string',
'StreamARN': 'string',
'StreamStatus': 'CREATING'|'DELETING'|'ACTIVE'|'UPDATING',
'RetentionPeriodHours': 123,
'StreamCreationTimestamp': datetime(2015, 1, 1),
'EnhancedMonitoring': [
{
'ShardLevelMetrics': [
'IncomingBytes'|'IncomingRecords'|'OutgoingBytes'|'OutgoingRecords'|'WriteProvisionedThroughputExceeded'|'ReadProvisionedThroughputExceeded'|'IteratorAgeMilliseconds'|'ALL',
]
},
],
'EncryptionType': 'NONE'|'KMS',
'KeyId': 'string',
'OpenShardCount': 123,
'ConsumerCount': 123
}
}
Response Structure
(dict) --
StreamDescriptionSummary (dict) --
A StreamDescriptionSummary containing information about the stream.
StreamName (string) --
The name of the stream being described.
StreamARN (string) --
The Amazon Resource Name (ARN) for the stream being described.
StreamStatus (string) --
The current status of the stream being described. The stream status is one of the following states:
CREATING - The stream is being created. Kinesis Data Streams immediately returns and sets StreamStatus to CREATING .
DELETING - The stream is being deleted. The specified stream is in the DELETING state until Kinesis Data Streams completes the deletion.
ACTIVE - The stream exists and is ready for read and write operations or deletion. You should perform read and write operations only on an ACTIVE stream.
UPDATING - Shards in the stream are being merged or split. Read and write operations continue to work while the stream is in the UPDATING state.
RetentionPeriodHours (integer) --
The current retention period, in hours.
StreamCreationTimestamp (datetime) --
The approximate time that the stream was created.
EnhancedMonitoring (list) --
Represents the current enhanced monitoring settings of the stream.
(dict) --
Represents enhanced metrics types.
ShardLevelMetrics (list) --
List of shard-level metrics.
The following are the valid shard-level metrics. The value "ALL " enhances every metric.
IncomingBytes
IncomingRecords
OutgoingBytes
OutgoingRecords
WriteProvisionedThroughputExceeded
ReadProvisionedThroughputExceeded
IteratorAgeMilliseconds
ALL
For more information, see Monitoring the Amazon Kinesis Data Streams Service with Amazon CloudWatch in the Amazon Kinesis Data Streams Developer Guide .
(string) --
EncryptionType (string) --
The encryption type used. This value is one of the following:
KMS
NONE
KeyId (string) --
The GUID for the customer-managed AWS KMS key to use for encryption. This value can be a globally unique identifier, a fully specified ARN to either an alias or a key, or an alias name prefixed by "alias/".You can also use a master key owned by Kinesis Data Streams by specifying the alias aws/kinesis .
Key ARN example: arn:aws:kms:us-east-1:123456789012:key/12345678-1234-1234-1234-123456789012
Alias ARN example: arn:aws:kms:us-east-1:123456789012:alias/MyAliasName
Globally unique key ID example: 12345678-1234-1234-1234-123456789012
Alias name example: alias/MyAliasName
Master key owned by Kinesis Data Streams: alias/aws/kinesis
OpenShardCount (integer) --
The number of open shards in the stream.
ConsumerCount (integer) --
The number of enhanced fan-out consumers registered with the stream.