2016/04/18 - Amazon Kinesis - 2 new 2 updated api methods
Enables enhanced Amazon Kinesis stream monitoring for shard-level metrics.
Request Syntax
client.enable_enhanced_monitoring(
StreamName='string',
ShardLevelMetrics=[
'IncomingBytes'|'IncomingRecords'|'OutgoingBytes'|'OutgoingRecords'|'WriteProvisionedThroughputExceeded'|'ReadProvisionedThroughputExceeded'|'IteratorAgeMilliseconds'|'ALL',
]
)
string
[REQUIRED]
The name of the stream for which to enable enhanced monitoring.
list
[REQUIRED]
List of shard-level metrics to enable.
The following are the valid shard-level metrics. The value "ALL " enables every metric.
IncomingBytes
IncomingRecords
OutgoingBytes
OutgoingRecords
WriteProvisionedThroughputExceeded
ReadProvisionedThroughputExceeded
IteratorAgeMilliseconds
ALL
For more information, see Monitoring the Amazon Kinesis Streams Service with Amazon CloudWatch in the Amazon Kinesis Streams Developer Guide .
(string) --
dict
Response Syntax
{
'StreamName': 'string',
'CurrentShardLevelMetrics': [
'IncomingBytes'|'IncomingRecords'|'OutgoingBytes'|'OutgoingRecords'|'WriteProvisionedThroughputExceeded'|'ReadProvisionedThroughputExceeded'|'IteratorAgeMilliseconds'|'ALL',
],
'DesiredShardLevelMetrics': [
'IncomingBytes'|'IncomingRecords'|'OutgoingBytes'|'OutgoingRecords'|'WriteProvisionedThroughputExceeded'|'ReadProvisionedThroughputExceeded'|'IteratorAgeMilliseconds'|'ALL',
]
}
Response Structure
(dict) --
Represents the output for EnableEnhancedMonitoring and DisableEnhancedMonitoring .
StreamName (string) --
The name of the Amazon Kinesis stream.
CurrentShardLevelMetrics (list) --
Represents the current state of the metrics that are in the enhanced state before the operation.
(string) --
DesiredShardLevelMetrics (list) --
Represents the list of all the metrics that would be in the enhanced state after the operation.
(string) --
Disables enhanced monitoring.
Request Syntax
client.disable_enhanced_monitoring(
StreamName='string',
ShardLevelMetrics=[
'IncomingBytes'|'IncomingRecords'|'OutgoingBytes'|'OutgoingRecords'|'WriteProvisionedThroughputExceeded'|'ReadProvisionedThroughputExceeded'|'IteratorAgeMilliseconds'|'ALL',
]
)
string
[REQUIRED]
The name of the Amazon Kinesis stream for which to disable enhanced monitoring.
list
[REQUIRED]
List of shard-level metrics to disable.
The following are the valid shard-level metrics. The value "ALL " disables every metric.
IncomingBytes
IncomingRecords
OutgoingBytes
OutgoingRecords
WriteProvisionedThroughputExceeded
ReadProvisionedThroughputExceeded
IteratorAgeMilliseconds
ALL
For more information, see Monitoring the Amazon Kinesis Streams Service with Amazon CloudWatch in the Amazon Kinesis Streams Developer Guide .
(string) --
dict
Response Syntax
{
'StreamName': 'string',
'CurrentShardLevelMetrics': [
'IncomingBytes'|'IncomingRecords'|'OutgoingBytes'|'OutgoingRecords'|'WriteProvisionedThroughputExceeded'|'ReadProvisionedThroughputExceeded'|'IteratorAgeMilliseconds'|'ALL',
],
'DesiredShardLevelMetrics': [
'IncomingBytes'|'IncomingRecords'|'OutgoingBytes'|'OutgoingRecords'|'WriteProvisionedThroughputExceeded'|'ReadProvisionedThroughputExceeded'|'IteratorAgeMilliseconds'|'ALL',
]
}
Response Structure
(dict) --
Represents the output for EnableEnhancedMonitoring and DisableEnhancedMonitoring .
StreamName (string) --
The name of the Amazon Kinesis stream.
CurrentShardLevelMetrics (list) --
Represents the current state of the metrics that are in the enhanced state before the operation.
(string) --
DesiredShardLevelMetrics (list) --
Represents the list of all the metrics that would be in the enhanced state after the operation.
(string) --
{'StreamDescription': {'EnhancedMonitoring': [{'ShardLevelMetrics': ['IncomingBytes '
'| '
'IncomingRecords '
'| '
'OutgoingBytes '
'| '
'OutgoingRecords '
'| '
'WriteProvisionedThroughputExceeded '
'| '
'ReadProvisionedThroughputExceeded '
'| '
'IteratorAgeMilliseconds '
'| '
'ALL']}]}}
Describes the specified Amazon Kinesis stream.
The information about the stream includes its current status, its Amazon Resource Name (ARN), and an array of shard objects. For each shard object, there is information about the hash key and sequence number ranges that the shard spans, and the IDs of any earlier shards that played in a role in creating the shard. A sequence number is the identifier associated with every record ingested in the stream. The sequence number is assigned when a record is put into the stream.
You can limit the number of returned shards using the Limit parameter. The number of shards in a stream may be too large to return from a single call to DescribeStream . You can detect this by using the HasMoreShards flag in the returned output. HasMoreShards is set to true when there is more data available.
DescribeStream is a paginated operation. If there are more shards available, you can request them using the shard ID of the last shard returned. Specify this ID in the ExclusiveStartShardId parameter in a subsequent request to DescribeStream .
There are no guarantees about the chronological order shards returned in DescribeStream results. If you want to process shards in chronological order, use ParentShardId to track lineage to the oldest shard.
DescribeStream has a limit of 10 transactions per second per account.
Request Syntax
client.describe_stream(
StreamName='string',
Limit=123,
ExclusiveStartShardId='string'
)
string
[REQUIRED]
The name of the stream to describe.
integer
The maximum number of shards to return.
string
The shard ID of the shard to start with.
dict
Response Syntax
{
'StreamDescription': {
'StreamName': 'string',
'StreamARN': 'string',
'StreamStatus': 'CREATING'|'DELETING'|'ACTIVE'|'UPDATING',
'Shards': [
{
'ShardId': 'string',
'ParentShardId': 'string',
'AdjacentParentShardId': 'string',
'HashKeyRange': {
'StartingHashKey': 'string',
'EndingHashKey': 'string'
},
'SequenceNumberRange': {
'StartingSequenceNumber': 'string',
'EndingSequenceNumber': 'string'
}
},
],
'HasMoreShards': True|False,
'RetentionPeriodHours': 123,
'EnhancedMonitoring': [
{
'ShardLevelMetrics': [
'IncomingBytes'|'IncomingRecords'|'OutgoingBytes'|'OutgoingRecords'|'WriteProvisionedThroughputExceeded'|'ReadProvisionedThroughputExceeded'|'IteratorAgeMilliseconds'|'ALL',
]
},
]
}
}
Response Structure
(dict) --
Represents the output for DescribeStream .
StreamDescription (dict) --
The current status of the stream, the stream ARN, an array of shard objects that comprise the stream, and states whether there are more shards available.
StreamName (string) --
The name of the stream being described.
StreamARN (string) --
The Amazon Resource Name (ARN) for the stream being described.
StreamStatus (string) --
The current status of the stream being described. The stream status is one of the following states:
CREATING - The stream is being created. Amazon Kinesis immediately returns and sets StreamStatus to CREATING .
DELETING - The stream is being deleted. The specified stream is in the DELETING state until Amazon Kinesis completes the deletion.
ACTIVE - The stream exists and is ready for read and write operations or deletion. You should perform read and write operations only on an ACTIVE stream.
UPDATING - Shards in the stream are being merged or split. Read and write operations continue to work while the stream is in the UPDATING state.
Shards (list) --
The shards that comprise the stream.
(dict) --
A uniquely identified group of data records in an Amazon Kinesis stream.
ShardId (string) --
The unique identifier of the shard within the stream.
ParentShardId (string) --
The shard ID of the shard's parent.
AdjacentParentShardId (string) --
The shard ID of the shard adjacent to the shard's parent.
HashKeyRange (dict) --
The range of possible hash key values for the shard, which is a set of ordered contiguous positive integers.
StartingHashKey (string) --
The starting hash key of the hash key range.
EndingHashKey (string) --
The ending hash key of the hash key range.
SequenceNumberRange (dict) --
The range of possible sequence numbers for the shard.
StartingSequenceNumber (string) --
The starting sequence number for the range.
EndingSequenceNumber (string) --
The ending sequence number for the range. Shards that are in the OPEN state have an ending sequence number of null .
HasMoreShards (boolean) --
If set to true , more shards in the stream are available to describe.
RetentionPeriodHours (integer) --
The current retention period, in hours.
EnhancedMonitoring (list) --
Represents the current enhanced monitoring settings of the stream.
(dict) --
Represents enhanced metrics types.
ShardLevelMetrics (list) --
List of shard-level metrics.
The following are the valid shard-level metrics. The value "ALL " enhances every metric.
IncomingBytes
IncomingRecords
OutgoingBytes
OutgoingRecords
WriteProvisionedThroughputExceeded
ReadProvisionedThroughputExceeded
IteratorAgeMilliseconds
ALL
For more information, see Monitoring the Amazon Kinesis Streams Service with Amazon CloudWatch in the Amazon Kinesis Streams Developer Guide .
(string) --
{'ShardIteratorType': ['AT_TIMESTAMP'], 'Timestamp': 'timestamp'}
Gets an Amazon Kinesis shard iterator. A shard iterator expires five minutes after it is returned to the requester.
A shard iterator specifies the shard position from which to start reading data records sequentially. The position is specified using the sequence number of a data record in a shard. A sequence number is the identifier associated with every record ingested in the stream, and is assigned when a record is put into the stream. Each stream has one or more shards.
You must specify the shard iterator type. For example, you can set the ShardIteratorType parameter to read exactly from the position denoted by a specific sequence number by using the AT_SEQUENCE_NUMBER shard iterator type, or right after the sequence number by using the AFTER_SEQUENCE_NUMBER shard iterator type, using sequence numbers returned by earlier calls to PutRecord , PutRecords , GetRecords , or DescribeStream . In the request, you can specify the shard iterator type AT_TIMESTAMP to read records from an arbitrary point in time, TRIM_HORIZON to cause ShardIterator to point to the last untrimmed record in the shard in the system (the oldest data record in the shard), or LATEST so that you always read the most recent data in the shard.
When you read repeatedly from a stream, use a GetShardIterator request to get the first shard iterator for use in your first GetRecords request and for subsequent reads use the shard iterator returned by the GetRecords request in NextShardIterator . A new shard iterator is returned by every GetRecords request in NextShardIterator , which you use in the ShardIterator parameter of the next GetRecords request.
If a GetShardIterator request is made too often, you receive a ProvisionedThroughputExceededException . For more information about throughput limits, see GetRecords , and Streams Limits in the Amazon Kinesis Streams Developer Guide .
If the shard is closed, GetShardIterator returns a valid iterator for the last sequence number of the shard. Note that a shard can be closed as a result of using SplitShard or MergeShards .
GetShardIterator has a limit of 5 transactions per second per account per open shard.
Request Syntax
client.get_shard_iterator(
StreamName='string',
ShardId='string',
ShardIteratorType='AT_SEQUENCE_NUMBER'|'AFTER_SEQUENCE_NUMBER'|'TRIM_HORIZON'|'LATEST'|'AT_TIMESTAMP',
StartingSequenceNumber='string',
Timestamp=datetime(2015, 1, 1)
)
string
[REQUIRED]
The name of the Amazon Kinesis stream.
string
[REQUIRED]
The shard ID of the Amazon Kinesis shard to get the iterator for.
string
[REQUIRED]
Determines how the shard iterator is used to start reading data records from the shard.
The following are the valid Amazon Kinesis shard iterator types:
AT_SEQUENCE_NUMBER - Start reading from the position denoted by a specific sequence number, provided in the value StartingSequenceNumber .
AFTER_SEQUENCE_NUMBER - Start reading right after the position denoted by a specific sequence number, provided in the value StartingSequenceNumber .
AT_TIMESTAMP - Start reading from the position denoted by a specific timestamp, provided in the value Timestamp .
TRIM_HORIZON - Start reading at the last untrimmed record in the shard in the system, which is the oldest data record in the shard.
LATEST - Start reading just after the most recent record in the shard, so that you always read the most recent data in the shard.
string
The sequence number of the data record in the shard from which to start reading. Used with shard iterator type AT_SEQUENCE_NUMBER and AFTER_SEQUENCE_NUMBER.
datetime
The timestamp of the data record from which to start reading. Used with shard iterator type AT_TIMESTAMP. A timestamp is the Unix epoch date with precision in milliseconds. For example, 2016-04-04T19:58:46.480-00:00 or 1459799926.480 . If a record with this exact timestamp does not exist, the iterator returned is for the next (later) record. If the timestamp is older than the current trim horizon, the iterator returned is for the oldest untrimmed data record (TRIM_HORIZON).
dict
Response Syntax
{
'ShardIterator': 'string'
}
Response Structure
(dict) --
Represents the output for GetShardIterator .
ShardIterator (string) --
The position in the shard from which to start reading data records sequentially. A shard iterator specifies this position using the sequence number of a data record in a shard.