2020/05/28 - Managed Streaming for Kafka - 2 new 2 updated api methods
Changes New APIs for upgrading the Apache Kafka version of a cluster and to find out compatible upgrade paths
Updates the Apache Kafka version for the cluster.
See also: AWS API Documentation
Request Syntax
client.update_cluster_kafka_version(
ClusterArn='string',
ConfigurationInfo={
'Arn': 'string',
'Revision': 123
},
CurrentVersion='string',
TargetKafkaVersion='string'
)
string
[REQUIRED]
The Amazon Resource Name (ARN) of the cluster to be updated.
dict
The custom configuration that should be applied on the new version of cluster.
Arn (string) -- [REQUIRED]
ARN of the configuration to use.
Revision (integer) -- [REQUIRED]
The revision of the configuration to use.
string
[REQUIRED]
Current cluster version.
string
[REQUIRED]
Target Kafka version.
dict
Response Syntax
{
'ClusterArn': 'string',
'ClusterOperationArn': 'string'
}
Response Structure
(dict) --
Successful response.
ClusterArn (string) --
The Amazon Resource Name (ARN) of the cluster.
ClusterOperationArn (string) --
The Amazon Resource Name (ARN) of the cluster operation.
Gets the Apache Kafka versions to which you can update the MSK cluster.
See also: AWS API Documentation
Request Syntax
client.get_compatible_kafka_versions(
ClusterArn='string'
)
string
The Amazon Resource Name (ARN) of the cluster check.
dict
Response Syntax
{
'CompatibleKafkaVersions': [
{
'SourceVersion': 'string',
'TargetVersions': [
'string',
]
},
]
}
Response Structure
(dict) --
Successful response.
CompatibleKafkaVersions (list) --
A list of CompatibleKafkaVersion objects.
(dict) --
Contains source Kafka versions and compatible target Kafka versions.
SourceVersion (string) --
A Kafka version.
TargetVersions (list) --
A list of Kafka versions.
(string) --
{'ClusterOperationInfo': {'OperationSteps': [{'StepInfo': {'StepStatus': 'string'},
'StepName': 'string'}],
'SourceClusterInfo': {'KafkaVersion': 'string'},
'TargetClusterInfo': {'KafkaVersion': 'string'}}}
Returns a description of the cluster operation specified by the ARN.
See also: AWS API Documentation
Request Syntax
client.describe_cluster_operation(
ClusterOperationArn='string'
)
string
[REQUIRED]
The Amazon Resource Name (ARN) that uniquely identifies the MSK cluster operation.
dict
Response Syntax
{
'ClusterOperationInfo': {
'ClientRequestId': 'string',
'ClusterArn': 'string',
'CreationTime': datetime(2015, 1, 1),
'EndTime': datetime(2015, 1, 1),
'ErrorInfo': {
'ErrorCode': 'string',
'ErrorString': 'string'
},
'OperationArn': 'string',
'OperationState': 'string',
'OperationSteps': [
{
'StepInfo': {
'StepStatus': 'string'
},
'StepName': 'string'
},
],
'OperationType': 'string',
'SourceClusterInfo': {
'BrokerEBSVolumeInfo': [
{
'KafkaBrokerNodeId': 'string',
'VolumeSizeGB': 123
},
],
'ConfigurationInfo': {
'Arn': 'string',
'Revision': 123
},
'NumberOfBrokerNodes': 123,
'EnhancedMonitoring': 'DEFAULT'|'PER_BROKER'|'PER_TOPIC_PER_BROKER',
'OpenMonitoring': {
'Prometheus': {
'JmxExporter': {
'EnabledInBroker': True|False
},
'NodeExporter': {
'EnabledInBroker': True|False
}
}
},
'KafkaVersion': 'string',
'LoggingInfo': {
'BrokerLogs': {
'CloudWatchLogs': {
'Enabled': True|False,
'LogGroup': 'string'
},
'Firehose': {
'DeliveryStream': 'string',
'Enabled': True|False
},
'S3': {
'Bucket': 'string',
'Enabled': True|False,
'Prefix': 'string'
}
}
}
},
'TargetClusterInfo': {
'BrokerEBSVolumeInfo': [
{
'KafkaBrokerNodeId': 'string',
'VolumeSizeGB': 123
},
],
'ConfigurationInfo': {
'Arn': 'string',
'Revision': 123
},
'NumberOfBrokerNodes': 123,
'EnhancedMonitoring': 'DEFAULT'|'PER_BROKER'|'PER_TOPIC_PER_BROKER',
'OpenMonitoring': {
'Prometheus': {
'JmxExporter': {
'EnabledInBroker': True|False
},
'NodeExporter': {
'EnabledInBroker': True|False
}
}
},
'KafkaVersion': 'string',
'LoggingInfo': {
'BrokerLogs': {
'CloudWatchLogs': {
'Enabled': True|False,
'LogGroup': 'string'
},
'Firehose': {
'DeliveryStream': 'string',
'Enabled': True|False
},
'S3': {
'Bucket': 'string',
'Enabled': True|False,
'Prefix': 'string'
}
}
}
}
}
}
Response Structure
(dict) --
200 response
ClusterOperationInfo (dict) --
Cluster operation information
ClientRequestId (string) --
The ID of the API request that triggered this operation.
ClusterArn (string) --
ARN of the cluster.
CreationTime (datetime) --
The time that the operation was created.
EndTime (datetime) --
The time at which the operation finished.
ErrorInfo (dict) --
Describes the error if the operation fails.
ErrorCode (string) --
A number describing the error programmatically.
ErrorString (string) --
An optional field to provide more details about the error.
OperationArn (string) --
ARN of the cluster operation.
OperationState (string) --
State of the cluster operation.
OperationSteps (list) --
Steps completed during the operation.
(dict) --
Step taken during a cluster operation.
StepInfo (dict) --
Information about the step and its status.
StepStatus (string) --
The steps current status.
StepName (string) --
The name of the step.
OperationType (string) --
Type of the cluster operation.
SourceClusterInfo (dict) --
Information about cluster attributes before a cluster is updated.
BrokerEBSVolumeInfo (list) --
Specifies the size of the EBS volume and the ID of the associated broker.
(dict) --
Specifies the EBS volume upgrade information. The broker identifier must be set to the keyword ALL. This means the changes apply to all the brokers in the cluster.
KafkaBrokerNodeId (string) --
The ID of the broker to update.
VolumeSizeGB (integer) --
Size of the EBS volume to update.
ConfigurationInfo (dict) --
Information about the changes in the configuration of the brokers.
Arn (string) --
ARN of the configuration to use.
Revision (integer) --
The revision of the configuration to use.
NumberOfBrokerNodes (integer) --
The number of broker nodes in the cluster.
EnhancedMonitoring (string) --
Specifies which Apache Kafka metrics Amazon MSK gathers and sends to Amazon CloudWatch for this cluster.
OpenMonitoring (dict) --
The settings for open monitoring.
Prometheus (dict) --
Prometheus settings.
JmxExporter (dict) --
Indicates whether you want to enable or disable the JMX Exporter.
EnabledInBroker (boolean) --
Indicates whether you want to enable or disable the JMX Exporter.
NodeExporter (dict) --
Indicates whether you want to enable or disable the Node Exporter.
EnabledInBroker (boolean) --
Indicates whether you want to enable or disable the Node Exporter.
KafkaVersion (string) --
The Kafka version.
LoggingInfo (dict) --
BrokerLogs (dict) --
CloudWatchLogs (dict) --
Enabled (boolean) --
LogGroup (string) --
Firehose (dict) --
DeliveryStream (string) --
Enabled (boolean) --
S3 (dict) --
Bucket (string) --
Enabled (boolean) --
Prefix (string) --
TargetClusterInfo (dict) --
Information about cluster attributes after a cluster is updated.
BrokerEBSVolumeInfo (list) --
Specifies the size of the EBS volume and the ID of the associated broker.
(dict) --
Specifies the EBS volume upgrade information. The broker identifier must be set to the keyword ALL. This means the changes apply to all the brokers in the cluster.
KafkaBrokerNodeId (string) --
The ID of the broker to update.
VolumeSizeGB (integer) --
Size of the EBS volume to update.
ConfigurationInfo (dict) --
Information about the changes in the configuration of the brokers.
Arn (string) --
ARN of the configuration to use.
Revision (integer) --
The revision of the configuration to use.
NumberOfBrokerNodes (integer) --
The number of broker nodes in the cluster.
EnhancedMonitoring (string) --
Specifies which Apache Kafka metrics Amazon MSK gathers and sends to Amazon CloudWatch for this cluster.
OpenMonitoring (dict) --
The settings for open monitoring.
Prometheus (dict) --
Prometheus settings.
JmxExporter (dict) --
Indicates whether you want to enable or disable the JMX Exporter.
EnabledInBroker (boolean) --
Indicates whether you want to enable or disable the JMX Exporter.
NodeExporter (dict) --
Indicates whether you want to enable or disable the Node Exporter.
EnabledInBroker (boolean) --
Indicates whether you want to enable or disable the Node Exporter.
KafkaVersion (string) --
The Kafka version.
LoggingInfo (dict) --
BrokerLogs (dict) --
CloudWatchLogs (dict) --
Enabled (boolean) --
LogGroup (string) --
Firehose (dict) --
DeliveryStream (string) --
Enabled (boolean) --
S3 (dict) --
Bucket (string) --
Enabled (boolean) --
Prefix (string) --
{'ClusterOperationInfoList': {'OperationSteps': [{'StepInfo': {'StepStatus': 'string'},
'StepName': 'string'}],
'SourceClusterInfo': {'KafkaVersion': 'string'},
'TargetClusterInfo': {'KafkaVersion': 'string'}}}
Returns a list of all the operations that have been performed on the specified MSK cluster.
See also: AWS API Documentation
Request Syntax
client.list_cluster_operations(
ClusterArn='string',
MaxResults=123,
NextToken='string'
)
string
[REQUIRED]
The Amazon Resource Name (ARN) that uniquely identifies the cluster.
integer
The maximum number of results to return in the response. If there are more results, the response includes a NextToken parameter.
string
The paginated results marker. When the result of the operation is truncated, the call returns NextToken in the response. To get the next batch, provide this token in your next request.
dict
Response Syntax
{
'ClusterOperationInfoList': [
{
'ClientRequestId': 'string',
'ClusterArn': 'string',
'CreationTime': datetime(2015, 1, 1),
'EndTime': datetime(2015, 1, 1),
'ErrorInfo': {
'ErrorCode': 'string',
'ErrorString': 'string'
},
'OperationArn': 'string',
'OperationState': 'string',
'OperationSteps': [
{
'StepInfo': {
'StepStatus': 'string'
},
'StepName': 'string'
},
],
'OperationType': 'string',
'SourceClusterInfo': {
'BrokerEBSVolumeInfo': [
{
'KafkaBrokerNodeId': 'string',
'VolumeSizeGB': 123
},
],
'ConfigurationInfo': {
'Arn': 'string',
'Revision': 123
},
'NumberOfBrokerNodes': 123,
'EnhancedMonitoring': 'DEFAULT'|'PER_BROKER'|'PER_TOPIC_PER_BROKER',
'OpenMonitoring': {
'Prometheus': {
'JmxExporter': {
'EnabledInBroker': True|False
},
'NodeExporter': {
'EnabledInBroker': True|False
}
}
},
'KafkaVersion': 'string',
'LoggingInfo': {
'BrokerLogs': {
'CloudWatchLogs': {
'Enabled': True|False,
'LogGroup': 'string'
},
'Firehose': {
'DeliveryStream': 'string',
'Enabled': True|False
},
'S3': {
'Bucket': 'string',
'Enabled': True|False,
'Prefix': 'string'
}
}
}
},
'TargetClusterInfo': {
'BrokerEBSVolumeInfo': [
{
'KafkaBrokerNodeId': 'string',
'VolumeSizeGB': 123
},
],
'ConfigurationInfo': {
'Arn': 'string',
'Revision': 123
},
'NumberOfBrokerNodes': 123,
'EnhancedMonitoring': 'DEFAULT'|'PER_BROKER'|'PER_TOPIC_PER_BROKER',
'OpenMonitoring': {
'Prometheus': {
'JmxExporter': {
'EnabledInBroker': True|False
},
'NodeExporter': {
'EnabledInBroker': True|False
}
}
},
'KafkaVersion': 'string',
'LoggingInfo': {
'BrokerLogs': {
'CloudWatchLogs': {
'Enabled': True|False,
'LogGroup': 'string'
},
'Firehose': {
'DeliveryStream': 'string',
'Enabled': True|False
},
'S3': {
'Bucket': 'string',
'Enabled': True|False,
'Prefix': 'string'
}
}
}
}
},
],
'NextToken': 'string'
}
Response Structure
(dict) --
Successful response.
ClusterOperationInfoList (list) --
An array of cluster operation information objects.
(dict) --
Returns information about a cluster operation.
ClientRequestId (string) --
The ID of the API request that triggered this operation.
ClusterArn (string) --
ARN of the cluster.
CreationTime (datetime) --
The time that the operation was created.
EndTime (datetime) --
The time at which the operation finished.
ErrorInfo (dict) --
Describes the error if the operation fails.
ErrorCode (string) --
A number describing the error programmatically.
ErrorString (string) --
An optional field to provide more details about the error.
OperationArn (string) --
ARN of the cluster operation.
OperationState (string) --
State of the cluster operation.
OperationSteps (list) --
Steps completed during the operation.
(dict) --
Step taken during a cluster operation.
StepInfo (dict) --
Information about the step and its status.
StepStatus (string) --
The steps current status.
StepName (string) --
The name of the step.
OperationType (string) --
Type of the cluster operation.
SourceClusterInfo (dict) --
Information about cluster attributes before a cluster is updated.
BrokerEBSVolumeInfo (list) --
Specifies the size of the EBS volume and the ID of the associated broker.
(dict) --
Specifies the EBS volume upgrade information. The broker identifier must be set to the keyword ALL. This means the changes apply to all the brokers in the cluster.
KafkaBrokerNodeId (string) --
The ID of the broker to update.
VolumeSizeGB (integer) --
Size of the EBS volume to update.
ConfigurationInfo (dict) --
Information about the changes in the configuration of the brokers.
Arn (string) --
ARN of the configuration to use.
Revision (integer) --
The revision of the configuration to use.
NumberOfBrokerNodes (integer) --
The number of broker nodes in the cluster.
EnhancedMonitoring (string) --
Specifies which Apache Kafka metrics Amazon MSK gathers and sends to Amazon CloudWatch for this cluster.
OpenMonitoring (dict) --
The settings for open monitoring.
Prometheus (dict) --
Prometheus settings.
JmxExporter (dict) --
Indicates whether you want to enable or disable the JMX Exporter.
EnabledInBroker (boolean) --
Indicates whether you want to enable or disable the JMX Exporter.
NodeExporter (dict) --
Indicates whether you want to enable or disable the Node Exporter.
EnabledInBroker (boolean) --
Indicates whether you want to enable or disable the Node Exporter.
KafkaVersion (string) --
The Kafka version.
LoggingInfo (dict) --
BrokerLogs (dict) --
CloudWatchLogs (dict) --
Enabled (boolean) --
LogGroup (string) --
Firehose (dict) --
DeliveryStream (string) --
Enabled (boolean) --
S3 (dict) --
Bucket (string) --
Enabled (boolean) --
Prefix (string) --
TargetClusterInfo (dict) --
Information about cluster attributes after a cluster is updated.
BrokerEBSVolumeInfo (list) --
Specifies the size of the EBS volume and the ID of the associated broker.
(dict) --
Specifies the EBS volume upgrade information. The broker identifier must be set to the keyword ALL. This means the changes apply to all the brokers in the cluster.
KafkaBrokerNodeId (string) --
The ID of the broker to update.
VolumeSizeGB (integer) --
Size of the EBS volume to update.
ConfigurationInfo (dict) --
Information about the changes in the configuration of the brokers.
Arn (string) --
ARN of the configuration to use.
Revision (integer) --
The revision of the configuration to use.
NumberOfBrokerNodes (integer) --
The number of broker nodes in the cluster.
EnhancedMonitoring (string) --
Specifies which Apache Kafka metrics Amazon MSK gathers and sends to Amazon CloudWatch for this cluster.
OpenMonitoring (dict) --
The settings for open monitoring.
Prometheus (dict) --
Prometheus settings.
JmxExporter (dict) --
Indicates whether you want to enable or disable the JMX Exporter.
EnabledInBroker (boolean) --
Indicates whether you want to enable or disable the JMX Exporter.
NodeExporter (dict) --
Indicates whether you want to enable or disable the Node Exporter.
EnabledInBroker (boolean) --
Indicates whether you want to enable or disable the Node Exporter.
KafkaVersion (string) --
The Kafka version.
LoggingInfo (dict) --
BrokerLogs (dict) --
CloudWatchLogs (dict) --
Enabled (boolean) --
LogGroup (string) --
Firehose (dict) --
DeliveryStream (string) --
Enabled (boolean) --
S3 (dict) --
Bucket (string) --
Enabled (boolean) --
Prefix (string) --
NextToken (string) --
If the response of ListClusterOperations is truncated, it returns a NextToken in the response. This Nexttoken should be sent in the subsequent request to ListClusterOperations.