Amazon SageMaker Service

2026/11/20 - Amazon SageMaker Service - 9 new 3 updated api methods

Changes  Adds HyperPod Global Inference Gateway APIs to route inference across HyperPod clusters in multiple Regions. Adds the MoveClusterNodes API, which moves running nodes between instance groups, within a cluster or between clusters in the same region, account and subnet, without releasing the instances.

ListGlobalInferenceGateways (new) Link ¶

Lists summaries of the global inference gateways in your account. You can filter the results by status.

See also: AWS API Documentation

Request Syntax

client.list_global_inference_gateways(
    Filters=[
        {
            'Name': 'Status',
            'Value': 'string'
        },
    ],
    NextToken='string',
    MaxResults=123
)
type Filters:

list

param Filters:

The filters to apply to the results. You can filter by Status. Multiple filters are combined with a logical AND.

  • (dict) --

    A filter for ListGlobalInferenceGateways. The filter matches the global inference gateways whose attribute named by Name equals Value.

    • Name (string) -- [REQUIRED]

      The attribute to filter global inference gateways on. The only supported value is Status.

    • Value (string) -- [REQUIRED]

      The value to match. For the Status filter, specify a global inference gateway status.

type NextToken:

string

param NextToken:

If the result of the previous ListGlobalInferenceGateways request was truncated, the response includes a NextToken. To retrieve the next set of global inference gateways, use the token in the next request.

type MaxResults:

integer

param MaxResults:

The maximum number of global inference gateways to return in the response.

rtype:

dict

returns:

Response Syntax

{
    'GlobalInferenceGatewaySummaries': [
        {
            'Arn': 'string',
            'Id': 'string',
            'Name': 'string',
            'Status': 'CREATING'|'ACTIVE'|'UPDATING'|'DELETING'|'FAILED',
            'CreationTime': datetime(2015, 1, 1)
        },
    ],
    'NextToken': 'string'
}

Response Structure

  • (dict) --

    • GlobalInferenceGatewaySummaries (list) --

      The summaries of the global inference gateways.

      • (dict) --

        Summary information about a global inference gateway.

        • Arn (string) --

          The Amazon Resource Name (ARN) of the global inference gateway.

        • Id (string) --

          The ID of the global inference gateway.

        • Name (string) --

          The name of the global inference gateway.

        • Status (string) --

          The status of the global inference gateway.

        • CreationTime (datetime) --

          The time when the global inference gateway was created.

    • NextToken (string) --

      If the result of the previous ListGlobalInferenceGateways request was truncated, the response includes a NextToken. To retrieve the next set of global inference gateways, use the token in the next request.

ListInferenceGateways (new) Link ¶

Lists the in-cluster inference gateways in a global inference gateway, in the order that they were added. Each summary includes the status of the in-cluster inference gateway and, if the status is FAILED, the reason for the failure. DescribeGlobalInferenceGateway doesn't return the in-cluster inference gateways; use this operation to list them.

See also: AWS API Documentation

Request Syntax

client.list_inference_gateways(
    Identifier='string',
    Filters=[
        {
            'Name': 'Status'|'ClusterArn',
            'Value': 'string'
        },
    ],
    NextToken='string',
    MaxResults=123
)
type Identifier:

string

param Identifier:

[REQUIRED]

The name, Amazon Resource Name (ARN), or ID of the global inference gateway.

type Filters:

list

param Filters:

The filters to apply to the results. You can filter by Status and ClusterArn. Multiple filters are combined with a logical AND.

  • (dict) --

    A filter for ListInferenceGateways. The filter matches the in-cluster inference gateways whose attribute named by Name equals Value.

    • Name (string) -- [REQUIRED]

      The attribute to filter in-cluster inference gateways on. Supported values are Status and ClusterArn.

    • Value (string) -- [REQUIRED]

      The value to match. For the Status filter, specify a status of an in-cluster inference gateway. For the ClusterArn filter, specify the Amazon Resource Name (ARN) of a SageMaker HyperPod cluster.

type NextToken:

string

param NextToken:

If the result of the previous ListInferenceGateways request was truncated, the response includes a NextToken. To retrieve the next set of in-cluster inference gateways, use the token in the next request.

type MaxResults:

integer

param MaxResults:

The maximum number of in-cluster inference gateways to return in the response.

rtype:

dict

returns:

Response Syntax

{
    'InferenceGatewaySummaries': [
        {
            'Name': 'string',
            'Namespace': 'string',
            'ClusterArn': 'string',
            'Status': 'ADDING'|'ACTIVE'|'REMOVING'|'FAILED',
            'FailureReason': 'string'
        },
    ],
    'NextToken': 'string'
}

Response Structure

  • (dict) --

    • InferenceGatewaySummaries (list) --

      The summaries of the in-cluster inference gateways, in the order that they were added.

      • (dict) --

        Summary information about an in-cluster inference gateway in a global inference gateway, including its status.

        • Name (string) --

          The name of the InferenceGatewayConfig resource that defines the in-cluster inference gateway.

        • Namespace (string) --

          The Kubernetes namespace of the in-cluster inference gateway.

        • ClusterArn (string) --

          The Amazon Resource Name (ARN) of the SageMaker HyperPod cluster that runs the in-cluster inference gateway.

        • Status (string) --

          The status of the in-cluster inference gateway in the global inference gateway. This status describes the membership in the global inference gateway, not the health of the in-cluster inference gateway.

        • FailureReason (string) --

          The reason that adding or removing the in-cluster inference gateway failed. This value is set only when the status is FAILED.

    • NextToken (string) --

      If the result of the previous ListInferenceGateways request was truncated, the response includes a NextToken. To retrieve the next set of in-cluster inference gateways, use the token in the next request.

AddInferenceGateway (new) Link ¶

Adds an in-cluster inference gateway to a global inference gateway. This operation is asynchronous: the in-cluster inference gateway starts in the ADDING status. To check its status, call ListInferenceGateways. If the in-cluster inference gateway is already in the global inference gateway, the operation succeeds and returns its current status.

An in-cluster inference gateway can belong to only one global inference gateway. If it already belongs to another global inference gateway, the operation fails with ResourceInUse. The operation fails with ConflictException while another operation is in progress on the global inference gateway, or if the global inference gateway failed to create.

See also: AWS API Documentation

Request Syntax

client.add_inference_gateway(
    Identifier='string',
    ClusterArn='string',
    Namespace='string',
    Name='string'
)
type Identifier:

string

param Identifier:

[REQUIRED]

The name, Amazon Resource Name (ARN), or ID of the global inference gateway.

type ClusterArn:

string

param ClusterArn:

[REQUIRED]

The Amazon Resource Name (ARN) of the SageMaker HyperPod cluster that runs the in-cluster inference gateway. The cluster must be in the same account and partition as the global inference gateway. It can be in a different Region, but not in an opt-in Region.

type Namespace:

string

param Namespace:

[REQUIRED]

The Kubernetes namespace of the in-cluster inference gateway.

type Name:

string

param Name:

[REQUIRED]

The name of the InferenceGatewayConfig resource that defines the in-cluster inference gateway.

rtype:

dict

returns:

Response Syntax

{
    'Arn': 'string',
    'Id': 'string',
    'Name': 'string',
    'Gateway': {
        'Name': 'string',
        'Namespace': 'string',
        'ClusterArn': 'string'
    },
    'Status': 'ADDING'|'ACTIVE'|'REMOVING'|'FAILED'
}

Response Structure

  • (dict) --

    • Arn (string) --

      The Amazon Resource Name (ARN) of the global inference gateway.

    • Id (string) --

      The ID of the global inference gateway.

    • Name (string) --

      The name of the global inference gateway.

    • Gateway (dict) --

      The in-cluster inference gateway that you added.

      • Name (string) --

        The name of the InferenceGatewayConfig resource that defines the in-cluster inference gateway.

      • Namespace (string) --

        The Kubernetes namespace of the in-cluster inference gateway.

      • ClusterArn (string) --

        The Amazon Resource Name (ARN) of the SageMaker HyperPod cluster that runs the in-cluster inference gateway. The cluster must be in the same account and partition as the global inference gateway. It can be in a different Region, but not in an opt-in Region.

    • Status (string) --

      The status of the in-cluster inference gateway in the global inference gateway.

RemoveInferenceGateway (new) Link ¶

Removes an in-cluster inference gateway from a global inference gateway. This operation is asynchronous: the in-cluster inference gateway moves to the REMOVING status, and then no longer appears in ListInferenceGateways. If the in-cluster inference gateway isn't in the global inference gateway, the operation succeeds and returns no status. The operation returns ResourceNotFound only if the global inference gateway doesn't exist.

The operation fails with ConflictException while another operation is in progress on the global inference gateway. It fails with ValidationException if the in-cluster inference gateway is the last one, because a global inference gateway must have at least one.

See also: AWS API Documentation

Request Syntax

client.remove_inference_gateway(
    Identifier='string',
    ClusterArn='string',
    Namespace='string',
    Name='string'
)
type Identifier:

string

param Identifier:

[REQUIRED]

The name, Amazon Resource Name (ARN), or ID of the global inference gateway.

type ClusterArn:

string

param ClusterArn:

[REQUIRED]

The Amazon Resource Name (ARN) of the SageMaker HyperPod cluster that runs the in-cluster inference gateway.

type Namespace:

string

param Namespace:

[REQUIRED]

The Kubernetes namespace of the in-cluster inference gateway.

type Name:

string

param Name:

[REQUIRED]

The name of the InferenceGatewayConfig resource that defines the in-cluster inference gateway.

rtype:

dict

returns:

Response Syntax

{
    'Arn': 'string',
    'Id': 'string',
    'Name': 'string',
    'Gateway': {
        'Name': 'string',
        'Namespace': 'string',
        'ClusterArn': 'string'
    },
    'Status': 'ADDING'|'ACTIVE'|'REMOVING'|'FAILED'
}

Response Structure

  • (dict) --

    • Arn (string) --

      The Amazon Resource Name (ARN) of the global inference gateway.

    • Id (string) --

      The ID of the global inference gateway.

    • Name (string) --

      The name of the global inference gateway.

    • Gateway (dict) --

      The in-cluster inference gateway that you removed.

      • Name (string) --

        The name of the InferenceGatewayConfig resource that defines the in-cluster inference gateway.

      • Namespace (string) --

        The Kubernetes namespace of the in-cluster inference gateway.

      • ClusterArn (string) --

        The Amazon Resource Name (ARN) of the SageMaker HyperPod cluster that runs the in-cluster inference gateway. The cluster must be in the same account and partition as the global inference gateway. It can be in a different Region, but not in an opt-in Region.

    • Status (string) --

      The status of the in-cluster inference gateway in the global inference gateway. This value is absent if the in-cluster inference gateway wasn't in the global inference gateway.

CreateGlobalInferenceGateway (new) Link ¶

Creates a global inference gateway. This operation is asynchronous: the global inference gateway starts in the CREATING status. To check its status, call DescribeGlobalInferenceGateway until the status is ACTIVE or FAILED. The operation fails with ResourceInUse if the name is already in use, or if one of the in-cluster inference gateways in Gateways already belongs to another global inference gateway.

See also: AWS API Documentation

Request Syntax

client.create_global_inference_gateway(
    Name='string',
    Routing={
        'Weights': {
            'KVHeadroom': ...,
            'QueueDepth': ...,
            'RegionProximity': ...,
            'ErrorRate': ...
        }
    },
    ExecutionRoleArn='string',
    Gateways=[
        {
            'Name': 'string',
            'Namespace': 'string',
            'ClusterArn': 'string'
        },
    ],
    ClientToken='string',
    Tags=[
        {
            'Key': 'string',
            'Value': 'string'
        },
    ]
)
type Name:

string

param Name:

[REQUIRED]

The name of the global inference gateway. The name must be unique in your account in the current Region, and is case-sensitive. The name can't be exactly 12 lowercase letters or digits, because the global inference gateway ID uses that format.

type Routing:

dict

param Routing:

The routing configuration of the global inference gateway. If you omit it, the global inference gateway uses default routing weights.

  • Weights (dict) -- [REQUIRED]

    The weights that the global inference gateway applies to each routing signal when it scores the in-cluster inference gateways for a request.

    • KVHeadroom (float) --

      The weight of the free key-value (KV) cache capacity on the cluster.

    • QueueDepth (float) --

      The weight of the number of requests that are waiting on the cluster.

    • RegionProximity (float) --

      The weight of how close the cluster is to the caller.

    • ErrorRate (float) --

      The weight of the recent error rate of the cluster.

type ExecutionRoleArn:

string

param ExecutionRoleArn:

[REQUIRED]

The Amazon Resource Name (ARN) of the IAM role that the global inference gateway assumes to access your SageMaker HyperPod clusters. The role must be in your account and partition, and you must have iam:PassRole permission for it. You can't change the role after you create the global inference gateway.

type Gateways:

list

param Gateways:

[REQUIRED]

The 1 to 10 in-cluster inference gateways to add to the global inference gateway. A global inference gateway must always have at least one in-cluster inference gateway, so RemoveInferenceGateway can't remove the last one.

  • (dict) --

    An in-cluster inference gateway, identified by the SageMaker HyperPod cluster that runs it ( ClusterArn), its Kubernetes namespace ( Namespace), and the name of the InferenceGatewayConfig resource that defines it ( Name).

    • Name (string) -- [REQUIRED]

      The name of the InferenceGatewayConfig resource that defines the in-cluster inference gateway.

    • Namespace (string) -- [REQUIRED]

      The Kubernetes namespace of the in-cluster inference gateway.

    • ClusterArn (string) -- [REQUIRED]

      The Amazon Resource Name (ARN) of the SageMaker HyperPod cluster that runs the in-cluster inference gateway. The cluster must be in the same account and partition as the global inference gateway. It can be in a different Region, but not in an opt-in Region.

type ClientToken:

string

param ClientToken:

A unique, case-sensitive identifier that you provide to ensure the idempotency of the request. An idempotent operation completes no more than one time. If you don't specify a client token, the Amazon Web Services SDKs and the Amazon Web Services CLI generate one for you.

This field is autopopulated if not provided.

type Tags:

list

param Tags:

Custom tags for managing the global inference gateway as an Amazon Web Services resource. You can add tags to your global inference gateway in the same way you add them in other Amazon Web Services services that support tagging. For more information about tagging Amazon Web Services resources, see Tagging your Amazon Web Services resources in the Tagging Amazon Web Services Resources User Guide.

The global inference gateway doesn't return tags in its responses. To read the tags, use ListTags.

  • (dict) --

    A tag object that consists of a key and an optional value, used to manage metadata for SageMaker Amazon Web Services resources.

    You can add tags to notebook instances, training jobs, hyperparameter tuning jobs, batch transform jobs, models, labeling jobs, work teams, endpoint configurations, and endpoints. For more information on adding tags to SageMaker resources, see AddTags.

    For more information on adding metadata to your Amazon Web Services resources with tagging, see Tagging Amazon Web Services resources. For advice on best practices for managing Amazon Web Services resources with tagging, see Tagging Best Practices: Implement an Effective Amazon Web Services Resource Tagging Strategy.

    • Key (string) -- [REQUIRED]

      The tag key. Tag keys must be unique per resource.

    • Value (string) -- [REQUIRED]

      The tag value.

rtype:

dict

returns:

Response Syntax

{
    'Arn': 'string',
    'Id': 'string',
    'Name': 'string',
    'Status': 'CREATING'|'ACTIVE'|'UPDATING'|'DELETING'|'FAILED',
    'ExecutionRoleArn': 'string',
    'Routing': {
        'Weights': {
            'KVHeadroom': ...,
            'QueueDepth': ...,
            'RegionProximity': ...,
            'ErrorRate': ...
        }
    },
    'Gateways': [
        {
            'Name': 'string',
            'Namespace': 'string',
            'ClusterArn': 'string'
        },
    ]
}

Response Structure

  • (dict) --

    Contains the identifiers and status of the new global inference gateway, and the parameters from the request except Tags. To read the tags, use ListTags.

    • Arn (string) --

      The Amazon Resource Name (ARN) of the global inference gateway.

    • Id (string) --

      The ID of the global inference gateway.

    • Name (string) --

      The name of the global inference gateway.

    • Status (string) --

      The status of the global inference gateway. A new global inference gateway starts in the CREATING status.

    • ExecutionRoleArn (string) --

      The Amazon Resource Name (ARN) of the IAM role that the global inference gateway assumes to access your SageMaker HyperPod clusters.

    • Routing (dict) --

      The routing configuration of the global inference gateway.

      • Weights (dict) --

        The weights that the global inference gateway applies to each routing signal when it scores the in-cluster inference gateways for a request.

        • KVHeadroom (float) --

          The weight of the free key-value (KV) cache capacity on the cluster.

        • QueueDepth (float) --

          The weight of the number of requests that are waiting on the cluster.

        • RegionProximity (float) --

          The weight of how close the cluster is to the caller.

        • ErrorRate (float) --

          The weight of the recent error rate of the cluster.

    • Gateways (list) --

      The in-cluster inference gateways that you specified in the request.

      • (dict) --

        An in-cluster inference gateway, identified by the SageMaker HyperPod cluster that runs it ( ClusterArn), its Kubernetes namespace ( Namespace), and the name of the InferenceGatewayConfig resource that defines it ( Name).

        • Name (string) --

          The name of the InferenceGatewayConfig resource that defines the in-cluster inference gateway.

        • Namespace (string) --

          The Kubernetes namespace of the in-cluster inference gateway.

        • ClusterArn (string) --

          The Amazon Resource Name (ARN) of the SageMaker HyperPod cluster that runs the in-cluster inference gateway. The cluster must be in the same account and partition as the global inference gateway. It can be in a different Region, but not in an opt-in Region.

DeleteGlobalInferenceGateway (new) Link ¶

Delete a global inference gateway and its member inference gateways, which are then free to join another global inference gateway. Asynchronous: the global inference gateway is DELETING while its managed endpoint resources are torn down; poll DescribeGlobalInferenceGateway until it returns ResourceNotFound. Does not touch the customer's clusters. Idempotent: a Delete whose Identifier matches no global inference gateway succeeds and echoes what the Identifier says (Arn and Id for an ARN or Id, Name for a name).

See also: AWS API Documentation

Request Syntax

client.delete_global_inference_gateway(
    Identifier='string'
)
type Identifier:

string

param Identifier:

[REQUIRED]

The name, Amazon Resource Name (ARN), or ID of the global inference gateway to delete.

rtype:

dict

returns:

Response Syntax

{
    'Arn': 'string',
    'Id': 'string',
    'Name': 'string'
}

Response Structure

  • (dict) --

    • Arn (string) --

      Set when a global inference gateway was deleted, and for an already-gone one when the Identifier was an ARN or Id.

    • Id (string) --

      Set when a global inference gateway was deleted, and for an already-gone one when the Identifier was an ARN or Id.

    • Name (string) --

      Set when a global inference gateway was deleted, and for an already-gone one when the Identifier was a name.

UpdateGlobalInferenceGateway (new) Link ¶

Updates the routing configuration of a global inference gateway. To add or remove in-cluster inference gateways, use AddInferenceGateway and RemoveInferenceGateway. This operation is asynchronous: the global inference gateway moves to the UPDATING status. The operation fails with ConflictException while another operation is in progress on the global inference gateway.

See also: AWS API Documentation

Request Syntax

client.update_global_inference_gateway(
    Identifier='string',
    Routing={
        'Weights': {
            'KVHeadroom': ...,
            'QueueDepth': ...,
            'RegionProximity': ...,
            'ErrorRate': ...
        }
    }
)
type Identifier:

string

param Identifier:

[REQUIRED]

The name, Amazon Resource Name (ARN), or ID of the global inference gateway to update.

type Routing:

dict

param Routing:

[REQUIRED]

The new routing configuration of the global inference gateway.

  • Weights (dict) -- [REQUIRED]

    The weights that the global inference gateway applies to each routing signal when it scores the in-cluster inference gateways for a request.

    • KVHeadroom (float) --

      The weight of the free key-value (KV) cache capacity on the cluster.

    • QueueDepth (float) --

      The weight of the number of requests that are waiting on the cluster.

    • RegionProximity (float) --

      The weight of how close the cluster is to the caller.

    • ErrorRate (float) --

      The weight of the recent error rate of the cluster.

rtype:

dict

returns:

Response Syntax

{
    'Arn': 'string',
    'Id': 'string',
    'Name': 'string',
    'Status': 'CREATING'|'ACTIVE'|'UPDATING'|'DELETING'|'FAILED',
    'Routing': {
        'Weights': {
            'KVHeadroom': ...,
            'QueueDepth': ...,
            'RegionProximity': ...,
            'ErrorRate': ...
        }
    }
}

Response Structure

  • (dict) --

    • Arn (string) --

      The Amazon Resource Name (ARN) of the global inference gateway.

    • Id (string) --

      The ID of the global inference gateway.

    • Name (string) --

      The name of the global inference gateway.

    • Status (string) --

      The status of the global inference gateway.

    • Routing (dict) --

      The routing configuration that you specified in the request.

      • Weights (dict) --

        The weights that the global inference gateway applies to each routing signal when it scores the in-cluster inference gateways for a request.

        • KVHeadroom (float) --

          The weight of the free key-value (KV) cache capacity on the cluster.

        • QueueDepth (float) --

          The weight of the number of requests that are waiting on the cluster.

        • RegionProximity (float) --

          The weight of how close the cluster is to the caller.

        • ErrorRate (float) --

          The weight of the recent error rate of the cluster.

MoveClusterNodes (new) Link ¶

Moves running nodes from one instance group to another instance group. The target instance group can be in the same HyperPod cluster or in a different HyperPod cluster in the same account and Region. The underlying instances aren't terminated.

Each node keeps its instance ID, its NodeLogicalId, its network interfaces and IP addresses, and its capacity. The node then joins the target instance group with the target's configuration and lifecycle scripts.

Specify the nodes to move with exactly one of NodeIds, NodeLogicalIds, or InstanceCount. The operation is asynchronous and returns after it accepts the nodes for the move. While a node moves, its status is DetachInProgress and then AttachInProgress. The node appears in the source cluster while it detaches. After HyperPod hands the node off to the target instance group, the node appears in the target cluster. To track the node, call DescribeClusterNode or ListClusterNodes on the source or target cluster.

See also: AWS API Documentation

Request Syntax

client.move_cluster_nodes(
    SourceClusterArn='string',
    SourceInstanceGroupName='string',
    TargetClusterArn='string',
    TargetInstanceGroupName='string',
    InstanceCount=123,
    NodeIds=[
        'string',
    ],
    NodeLogicalIds=[
        'string',
    ],
    ClientToken='string'
)
type SourceClusterArn:

string

param SourceClusterArn:

[REQUIRED]

The Amazon Resource Name (ARN) of the HyperPod cluster that the nodes are moved from.

type SourceInstanceGroupName:

string

param SourceInstanceGroupName:

[REQUIRED]

The name of the instance group in the source cluster that the nodes are moved from.

type TargetClusterArn:

string

param TargetClusterArn:

[REQUIRED]

The Amazon Resource Name (ARN) of the HyperPod cluster that the nodes are moved to. The target cluster must be in the same account and Region as the source cluster, and can be the same cluster as the source.

type TargetInstanceGroupName:

string

param TargetInstanceGroupName:

[REQUIRED]

The name of the instance group in the target cluster that the nodes are moved to. If the target cluster is the same as the source cluster, the target instance group must be different from the source instance group.

type InstanceCount:

integer

param InstanceCount:

The number of nodes to move, from 1 to 50. HyperPod selects the nodes from the source instance group. If fewer than this number of nodes are eligible to move, the request fails and no nodes are moved. Specify exactly one of NodeIds, NodeLogicalIds, or InstanceCount.

type NodeIds:

list

param NodeIds:

A list of instance IDs of the nodes to move. You can specify up to 50 instance IDs. Each node is evaluated separately, so some nodes can be moved while others fail. The request fails if moving the eligible nodes would reduce the source instance group below its minimum instance count. Specify exactly one of NodeIds, NodeLogicalIds, or InstanceCount.

  • (string) --

type NodeLogicalIds:

list

param NodeLogicalIds:

A list of NodeLogicalIds of the nodes to move. You can specify up to 50 NodeLogicalIds. Each node is evaluated separately, so some nodes can be moved while others fail. The request fails if moving the eligible nodes would reduce the source instance group below its minimum instance count. Specify exactly one of NodeIds, NodeLogicalIds, or InstanceCount.

  • (string) --

type ClientToken:

string

param ClientToken:

A unique, case-sensitive identifier that you provide to ensure that the operation completes no more than one time. If you retry the request with the same client token and the same parameters within 8 hours, the service returns the original response. It doesn't start a new move. If you reuse a client token with different parameters, the request fails.

This field is autopopulated if not provided.

rtype:

dict

returns:

Response Syntax

{
    'Successful': [
        'string',
    ],
    'Failed': [
        {
            'Code': 'NodeIdNotFound'|'InvalidNodeStatus'|'NodeIdInUse'|'NodeInstanceTypeMismatch',
            'Message': 'string',
            'NodeId': 'string'
        },
    ],
    'SuccessfulNodeLogicalIds': [
        'string',
    ],
    'FailedNodeLogicalIds': [
        {
            'Code': 'NodeIdNotFound'|'InvalidNodeStatus'|'NodeIdInUse'|'NodeInstanceTypeMismatch',
            'Message': 'string',
            'NodeLogicalId': 'string'
        },
    ]
}

Response Structure

  • (dict) --

    • Successful (list) --

      This list contains the instance IDs of the nodes that were accepted for the move. The response includes this list when you specify NodeIds or InstanceCount.

      • (string) --

    • Failed (list) --

      The instance IDs that could not be moved, each with an error code and message. The response includes this list when you specify NodeIds. If none of the specified nodes can be moved, the request fails and no nodes are moved.

      • (dict) --

        Describes why a node, identified by its instance ID, could not be moved in a MoveClusterNodes request. Contains the instance ID, an error code, and an error message.

        • Code (string) --

          The error code associated with the error encountered when moving the node. The error code is one of the following values:

          • NodeIdNotFound: The node does not exist in the source instance group.

          • InvalidNodeStatus: The node is not in a state that allows the move. For example, the node is not Running, is not in a subnet of the target instance group, or has an attached Amazon EBS volume that you must detach first.

          • NodeIdInUse: Another operation is already in progress for this node. Wait for the operation to complete before retrying.

          • NodeInstanceTypeMismatch: The target instance group does not support the node's instance type.

        • Message (string) --

          A human-readable message describing the error encountered when moving the node.

        • NodeId (string) --

          The instance ID of the node that could not be moved.

    • SuccessfulNodeLogicalIds (list) --

      The NodeLogicalIds of the nodes that were accepted for the move. The response includes this list when you specify NodeLogicalIds.

      • (string) --

    • FailedNodeLogicalIds (list) --

      The NodeLogicalIds that could not be moved, each with an error code and message. The response includes this list when you specify NodeLogicalIds. If none of the specified nodes can be moved, the request fails and no nodes are moved.

      • (dict) --

        Describes why a node, identified by its NodeLogicalId, could not be moved in a MoveClusterNodes request. Contains the NodeLogicalId, an error code, and an error message.

        • Code (string) --

          The error code associated with the error encountered when moving the node. The error code is one of the following values:

          • NodeIdNotFound: The node does not exist in the source instance group.

          • InvalidNodeStatus: The node is not in a state that allows the move. For example, the node is not Running, is not in a subnet of the target instance group, or has an attached Amazon EBS volume that you must detach first.

          • NodeIdInUse: Another operation is already in progress for this node. Wait for the operation to complete before retrying.

          • NodeInstanceTypeMismatch: The target instance group does not support the node's instance type.

        • Message (string) --

          A human-readable message describing the error encountered when moving the node.

        • NodeLogicalId (string) --

          The NodeLogicalId of the node that could not be moved.

DescribeGlobalInferenceGateway (new) Link ¶

Retrieves information about a global inference gateway, including its status, endpoints, and routing configuration. To list the in-cluster inference gateways of the global inference gateway, use ListInferenceGateways. To read the tags of the global inference gateway, use ListTags.

See also: AWS API Documentation

Request Syntax

client.describe_global_inference_gateway(
    Identifier='string'
)
type Identifier:

string

param Identifier:

[REQUIRED]

The name, Amazon Resource Name (ARN), or ID of the global inference gateway.

rtype:

dict

returns:

Response Syntax

{
    'Arn': 'string',
    'Id': 'string',
    'Name': 'string',
    'ExecutionRoleArn': 'string',
    'Status': 'CREATING'|'ACTIVE'|'UPDATING'|'DELETING'|'FAILED',
    'EndpointUrl': 'string',
    'VpcEndpointServiceName': 'string',
    'Routing': {
        'Weights': {
            'KVHeadroom': ...,
            'QueueDepth': ...,
            'RegionProximity': ...,
            'ErrorRate': ...
        }
    },
    'CreationTime': datetime(2015, 1, 1),
    'FailureReason': 'string'
}

Response Structure

  • (dict) --

    • Arn (string) --

      The Amazon Resource Name (ARN) of the global inference gateway.

    • Id (string) --

      The ID of the global inference gateway.

    • Name (string) --

      The name of the global inference gateway.

    • ExecutionRoleArn (string) --

      The Amazon Resource Name (ARN) of the IAM role that the global inference gateway assumes to access your SageMaker HyperPod clusters. You can't change the role after you create the global inference gateway.

    • Status (string) --

      The status of the global inference gateway.

    • EndpointUrl (string) --

      The URL that you use to invoke the global inference gateway. This value is set after the global inference gateway becomes ACTIVE.

    • VpcEndpointServiceName (string) --

      The name of the VPC endpoint service to create an interface VPC endpoint (AWS PrivateLink) from, so that you can call EndpointUrl without going over the internet. This value is set after the global inference gateway becomes ACTIVE.

    • Routing (dict) --

      The routing configuration of the global inference gateway.

      • Weights (dict) --

        The weights that the global inference gateway applies to each routing signal when it scores the in-cluster inference gateways for a request.

        • KVHeadroom (float) --

          The weight of the free key-value (KV) cache capacity on the cluster.

        • QueueDepth (float) --

          The weight of the number of requests that are waiting on the cluster.

        • RegionProximity (float) --

          The weight of how close the cluster is to the caller.

        • ErrorRate (float) --

          The weight of the recent error rate of the cluster.

    • CreationTime (datetime) --

      The time when the global inference gateway was created.

    • FailureReason (string) --

      The reason that the global inference gateway failed. SageMaker returns this value only when the status is FAILED.

BatchAddClusterNodes (updated) Link ¶
Changes (response)
{'Successful': {'Status': {'AttachInProgress', 'DetachInProgress'}}}

Adds nodes to a HyperPod cluster by incrementing the target count for one or more instance groups. This operation returns a unique NodeLogicalId for each node being added, which can be used to track the provisioning status of the node. This API provides a safer alternative to UpdateCluster for scaling operations by avoiding unintended configuration changes.

See also: AWS API Documentation

Request Syntax

client.batch_add_cluster_nodes(
    ClusterName='string',
    ClientToken='string',
    NodesToAdd=[
        {
            'InstanceGroupName': 'string',
            'IncrementTargetCountBy': 123,
            'AvailabilityZones': [
                'string',
            ],
            'InstanceTypes': [
                'ml.p4d.24xlarge'|'ml.p4de.24xlarge'|'ml.p5.48xlarge'|'ml.p5.4xlarge'|'ml.p6e-gb200.36xlarge'|'ml.trn1.32xlarge'|'ml.trn1n.32xlarge'|'ml.g5.xlarge'|'ml.g5.2xlarge'|'ml.g5.4xlarge'|'ml.g5.8xlarge'|'ml.g5.12xlarge'|'ml.g5.16xlarge'|'ml.g5.24xlarge'|'ml.g5.48xlarge'|'ml.c5.large'|'ml.c5.xlarge'|'ml.c5.2xlarge'|'ml.c5.4xlarge'|'ml.c5.9xlarge'|'ml.c5.12xlarge'|'ml.c5.18xlarge'|'ml.c5.24xlarge'|'ml.c5n.large'|'ml.c5n.2xlarge'|'ml.c5n.4xlarge'|'ml.c5n.9xlarge'|'ml.c5n.18xlarge'|'ml.m5.large'|'ml.m5.xlarge'|'ml.m5.2xlarge'|'ml.m5.4xlarge'|'ml.m5.8xlarge'|'ml.m5.12xlarge'|'ml.m5.16xlarge'|'ml.m5.24xlarge'|'ml.t3.medium'|'ml.t3.large'|'ml.t3.xlarge'|'ml.t3.2xlarge'|'ml.g6.xlarge'|'ml.g6.2xlarge'|'ml.g6.4xlarge'|'ml.g6.8xlarge'|'ml.g6.16xlarge'|'ml.g6.12xlarge'|'ml.g6.24xlarge'|'ml.g6.48xlarge'|'ml.gr6.4xlarge'|'ml.gr6.8xlarge'|'ml.g6e.xlarge'|'ml.g6e.2xlarge'|'ml.g6e.4xlarge'|'ml.g6e.8xlarge'|'ml.g6e.16xlarge'|'ml.g6e.12xlarge'|'ml.g6e.24xlarge'|'ml.g6e.48xlarge'|'ml.p5e.48xlarge'|'ml.p5en.48xlarge'|'ml.p6-b200.48xlarge'|'ml.trn2.3xlarge'|'ml.trn2.48xlarge'|'ml.c6i.large'|'ml.c6i.xlarge'|'ml.c6i.2xlarge'|'ml.c6i.4xlarge'|'ml.c6i.8xlarge'|'ml.c6i.12xlarge'|'ml.c6i.16xlarge'|'ml.c6i.24xlarge'|'ml.c6i.32xlarge'|'ml.m6i.large'|'ml.m6i.xlarge'|'ml.m6i.2xlarge'|'ml.m6i.4xlarge'|'ml.m6i.8xlarge'|'ml.m6i.12xlarge'|'ml.m6i.16xlarge'|'ml.m6i.24xlarge'|'ml.m6i.32xlarge'|'ml.r6i.large'|'ml.r6i.xlarge'|'ml.r6i.2xlarge'|'ml.r6i.4xlarge'|'ml.r6i.8xlarge'|'ml.r6i.12xlarge'|'ml.r6i.16xlarge'|'ml.r6i.24xlarge'|'ml.r6i.32xlarge'|'ml.i3en.large'|'ml.i3en.xlarge'|'ml.i3en.2xlarge'|'ml.i3en.3xlarge'|'ml.i3en.6xlarge'|'ml.i3en.12xlarge'|'ml.i3en.24xlarge'|'ml.m7i.large'|'ml.m7i.xlarge'|'ml.m7i.2xlarge'|'ml.m7i.4xlarge'|'ml.m7i.8xlarge'|'ml.m7i.12xlarge'|'ml.m7i.16xlarge'|'ml.m7i.24xlarge'|'ml.m7i.48xlarge'|'ml.r7i.large'|'ml.r7i.xlarge'|'ml.r7i.2xlarge'|'ml.r7i.4xlarge'|'ml.r7i.8xlarge'|'ml.r7i.12xlarge'|'ml.r7i.16xlarge'|'ml.r7i.24xlarge'|'ml.r7i.48xlarge'|'ml.r5d.16xlarge'|'ml.g7e.2xlarge'|'ml.g7e.4xlarge'|'ml.g7e.8xlarge'|'ml.g7e.12xlarge'|'ml.g7e.24xlarge'|'ml.g7e.48xlarge'|'ml.p6-b300.48xlarge'|'ml.g4dn.xlarge'|'ml.g4dn.2xlarge'|'ml.g4dn.4xlarge'|'ml.g4dn.8xlarge'|'ml.g4dn.12xlarge'|'ml.g4dn.16xlarge'|'ml.c6g.medium'|'ml.c6g.large'|'ml.c6g.xlarge'|'ml.c6g.2xlarge'|'ml.c6g.4xlarge'|'ml.c6g.8xlarge'|'ml.c6g.12xlarge'|'ml.c6g.16xlarge'|'ml.c7g.medium'|'ml.c7g.large'|'ml.c7g.xlarge'|'ml.c7g.2xlarge'|'ml.c7g.4xlarge'|'ml.c7g.8xlarge'|'ml.c7g.12xlarge'|'ml.c7g.16xlarge'|'ml.c8g.medium'|'ml.c8g.large'|'ml.c8g.xlarge'|'ml.c8g.2xlarge'|'ml.c8g.4xlarge'|'ml.c8g.8xlarge'|'ml.c8g.12xlarge'|'ml.c8g.16xlarge'|'ml.c8g.24xlarge'|'ml.c8g.48xlarge'|'ml.c6a.large'|'ml.c6a.xlarge'|'ml.c6a.2xlarge'|'ml.c6a.4xlarge'|'ml.c6a.8xlarge'|'ml.c6a.12xlarge'|'ml.c6a.16xlarge'|'ml.c6a.24xlarge'|'ml.c6a.32xlarge'|'ml.c6a.48xlarge'|'ml.m6a.large'|'ml.m6a.xlarge'|'ml.m6a.2xlarge'|'ml.m6a.4xlarge'|'ml.m6a.8xlarge'|'ml.m6a.12xlarge'|'ml.m6a.16xlarge'|'ml.m6a.24xlarge'|'ml.m6a.32xlarge'|'ml.m6a.48xlarge'|'ml.m6g.medium'|'ml.m6g.large'|'ml.m6g.xlarge'|'ml.m6g.2xlarge'|'ml.m6g.4xlarge'|'ml.m6g.8xlarge'|'ml.m6g.12xlarge'|'ml.m6g.16xlarge'|'ml.m7g.medium'|'ml.m7g.large'|'ml.m7g.xlarge'|'ml.m7g.2xlarge'|'ml.m7g.4xlarge'|'ml.m7g.8xlarge'|'ml.m7g.12xlarge'|'ml.m7g.16xlarge'|'ml.m8g.medium'|'ml.m8g.large'|'ml.m8g.xlarge'|'ml.m8g.2xlarge'|'ml.m8g.4xlarge'|'ml.m8g.8xlarge'|'ml.m8g.12xlarge'|'ml.m8g.16xlarge'|'ml.m8g.24xlarge'|'ml.m8g.48xlarge'|'ml.g7.2xlarge'|'ml.g7.4xlarge'|'ml.g7.8xlarge'|'ml.g7.12xlarge'|'ml.g7.24xlarge'|'ml.g7.48xlarge'|'ml.c8a.16xlarge'|'ml.m8a.16xlarge',
            ]
        },
    ]
)
type ClusterName:

string

param ClusterName:

[REQUIRED]

The name of the HyperPod cluster to which you want to add nodes.

type ClientToken:

string

param ClientToken:

A unique, case-sensitive identifier that you provide to ensure the idempotency of the request. This token is valid for 8 hours. If you retry the request with the same client token within this timeframe and the same parameters, the API returns the same set of NodeLogicalIds with their latest status.

This field is autopopulated if not provided.

type NodesToAdd:

list

param NodesToAdd:

[REQUIRED]

A list of instance groups and the number of nodes to add to each. You can specify up to 5 instance groups in a single request, with a maximum of 50 nodes total across all instance groups.

  • (dict) --

    Specifies an instance group and the number of nodes to add to it.

    • InstanceGroupName (string) -- [REQUIRED]

      The name of the instance group to which you want to add nodes.

    • IncrementTargetCountBy (integer) -- [REQUIRED]

      The number of nodes to add to the specified instance group. The total number of nodes across all instance groups in a single request cannot exceed 50.

    • AvailabilityZones (list) --

      The availability zones in which to add nodes. Use this to target node placement in specific availability zones within a flexible instance group.

      • (string) --

    • InstanceTypes (list) --

      The instance types to use when adding nodes. Use this to target specific instance types within a flexible instance group.

      • (string) --

rtype:

dict

returns:

Response Syntax

{
    'Successful': [
        {
            'NodeLogicalId': 'string',
            'InstanceGroupName': 'string',
            'Status': 'Running'|'Failure'|'Pending'|'ShuttingDown'|'SystemUpdating'|'DeepHealthCheckInProgress'|'NotFound'|'DetachInProgress'|'AttachInProgress',
            'AvailabilityZones': [
                'string',
            ],
            'InstanceTypes': [
                'ml.p4d.24xlarge'|'ml.p4de.24xlarge'|'ml.p5.48xlarge'|'ml.p5.4xlarge'|'ml.p6e-gb200.36xlarge'|'ml.trn1.32xlarge'|'ml.trn1n.32xlarge'|'ml.g5.xlarge'|'ml.g5.2xlarge'|'ml.g5.4xlarge'|'ml.g5.8xlarge'|'ml.g5.12xlarge'|'ml.g5.16xlarge'|'ml.g5.24xlarge'|'ml.g5.48xlarge'|'ml.c5.large'|'ml.c5.xlarge'|'ml.c5.2xlarge'|'ml.c5.4xlarge'|'ml.c5.9xlarge'|'ml.c5.12xlarge'|'ml.c5.18xlarge'|'ml.c5.24xlarge'|'ml.c5n.large'|'ml.c5n.2xlarge'|'ml.c5n.4xlarge'|'ml.c5n.9xlarge'|'ml.c5n.18xlarge'|'ml.m5.large'|'ml.m5.xlarge'|'ml.m5.2xlarge'|'ml.m5.4xlarge'|'ml.m5.8xlarge'|'ml.m5.12xlarge'|'ml.m5.16xlarge'|'ml.m5.24xlarge'|'ml.t3.medium'|'ml.t3.large'|'ml.t3.xlarge'|'ml.t3.2xlarge'|'ml.g6.xlarge'|'ml.g6.2xlarge'|'ml.g6.4xlarge'|'ml.g6.8xlarge'|'ml.g6.16xlarge'|'ml.g6.12xlarge'|'ml.g6.24xlarge'|'ml.g6.48xlarge'|'ml.gr6.4xlarge'|'ml.gr6.8xlarge'|'ml.g6e.xlarge'|'ml.g6e.2xlarge'|'ml.g6e.4xlarge'|'ml.g6e.8xlarge'|'ml.g6e.16xlarge'|'ml.g6e.12xlarge'|'ml.g6e.24xlarge'|'ml.g6e.48xlarge'|'ml.p5e.48xlarge'|'ml.p5en.48xlarge'|'ml.p6-b200.48xlarge'|'ml.trn2.3xlarge'|'ml.trn2.48xlarge'|'ml.c6i.large'|'ml.c6i.xlarge'|'ml.c6i.2xlarge'|'ml.c6i.4xlarge'|'ml.c6i.8xlarge'|'ml.c6i.12xlarge'|'ml.c6i.16xlarge'|'ml.c6i.24xlarge'|'ml.c6i.32xlarge'|'ml.m6i.large'|'ml.m6i.xlarge'|'ml.m6i.2xlarge'|'ml.m6i.4xlarge'|'ml.m6i.8xlarge'|'ml.m6i.12xlarge'|'ml.m6i.16xlarge'|'ml.m6i.24xlarge'|'ml.m6i.32xlarge'|'ml.r6i.large'|'ml.r6i.xlarge'|'ml.r6i.2xlarge'|'ml.r6i.4xlarge'|'ml.r6i.8xlarge'|'ml.r6i.12xlarge'|'ml.r6i.16xlarge'|'ml.r6i.24xlarge'|'ml.r6i.32xlarge'|'ml.i3en.large'|'ml.i3en.xlarge'|'ml.i3en.2xlarge'|'ml.i3en.3xlarge'|'ml.i3en.6xlarge'|'ml.i3en.12xlarge'|'ml.i3en.24xlarge'|'ml.m7i.large'|'ml.m7i.xlarge'|'ml.m7i.2xlarge'|'ml.m7i.4xlarge'|'ml.m7i.8xlarge'|'ml.m7i.12xlarge'|'ml.m7i.16xlarge'|'ml.m7i.24xlarge'|'ml.m7i.48xlarge'|'ml.r7i.large'|'ml.r7i.xlarge'|'ml.r7i.2xlarge'|'ml.r7i.4xlarge'|'ml.r7i.8xlarge'|'ml.r7i.12xlarge'|'ml.r7i.16xlarge'|'ml.r7i.24xlarge'|'ml.r7i.48xlarge'|'ml.r5d.16xlarge'|'ml.g7e.2xlarge'|'ml.g7e.4xlarge'|'ml.g7e.8xlarge'|'ml.g7e.12xlarge'|'ml.g7e.24xlarge'|'ml.g7e.48xlarge'|'ml.p6-b300.48xlarge'|'ml.g4dn.xlarge'|'ml.g4dn.2xlarge'|'ml.g4dn.4xlarge'|'ml.g4dn.8xlarge'|'ml.g4dn.12xlarge'|'ml.g4dn.16xlarge'|'ml.c6g.medium'|'ml.c6g.large'|'ml.c6g.xlarge'|'ml.c6g.2xlarge'|'ml.c6g.4xlarge'|'ml.c6g.8xlarge'|'ml.c6g.12xlarge'|'ml.c6g.16xlarge'|'ml.c7g.medium'|'ml.c7g.large'|'ml.c7g.xlarge'|'ml.c7g.2xlarge'|'ml.c7g.4xlarge'|'ml.c7g.8xlarge'|'ml.c7g.12xlarge'|'ml.c7g.16xlarge'|'ml.c8g.medium'|'ml.c8g.large'|'ml.c8g.xlarge'|'ml.c8g.2xlarge'|'ml.c8g.4xlarge'|'ml.c8g.8xlarge'|'ml.c8g.12xlarge'|'ml.c8g.16xlarge'|'ml.c8g.24xlarge'|'ml.c8g.48xlarge'|'ml.c6a.large'|'ml.c6a.xlarge'|'ml.c6a.2xlarge'|'ml.c6a.4xlarge'|'ml.c6a.8xlarge'|'ml.c6a.12xlarge'|'ml.c6a.16xlarge'|'ml.c6a.24xlarge'|'ml.c6a.32xlarge'|'ml.c6a.48xlarge'|'ml.m6a.large'|'ml.m6a.xlarge'|'ml.m6a.2xlarge'|'ml.m6a.4xlarge'|'ml.m6a.8xlarge'|'ml.m6a.12xlarge'|'ml.m6a.16xlarge'|'ml.m6a.24xlarge'|'ml.m6a.32xlarge'|'ml.m6a.48xlarge'|'ml.m6g.medium'|'ml.m6g.large'|'ml.m6g.xlarge'|'ml.m6g.2xlarge'|'ml.m6g.4xlarge'|'ml.m6g.8xlarge'|'ml.m6g.12xlarge'|'ml.m6g.16xlarge'|'ml.m7g.medium'|'ml.m7g.large'|'ml.m7g.xlarge'|'ml.m7g.2xlarge'|'ml.m7g.4xlarge'|'ml.m7g.8xlarge'|'ml.m7g.12xlarge'|'ml.m7g.16xlarge'|'ml.m8g.medium'|'ml.m8g.large'|'ml.m8g.xlarge'|'ml.m8g.2xlarge'|'ml.m8g.4xlarge'|'ml.m8g.8xlarge'|'ml.m8g.12xlarge'|'ml.m8g.16xlarge'|'ml.m8g.24xlarge'|'ml.m8g.48xlarge'|'ml.g7.2xlarge'|'ml.g7.4xlarge'|'ml.g7.8xlarge'|'ml.g7.12xlarge'|'ml.g7.24xlarge'|'ml.g7.48xlarge'|'ml.c8a.16xlarge'|'ml.m8a.16xlarge',
            ]
        },
    ],
    'Failed': [
        {
            'InstanceGroupName': 'string',
            'ErrorCode': 'InstanceGroupNotFound'|'InvalidInstanceGroupStatus'|'IncompatibleAvailabilityZones'|'IncompatibleInstanceTypes',
            'FailedCount': 123,
            'AvailabilityZones': [
                'string',
            ],
            'InstanceTypes': [
                'ml.p4d.24xlarge'|'ml.p4de.24xlarge'|'ml.p5.48xlarge'|'ml.p5.4xlarge'|'ml.p6e-gb200.36xlarge'|'ml.trn1.32xlarge'|'ml.trn1n.32xlarge'|'ml.g5.xlarge'|'ml.g5.2xlarge'|'ml.g5.4xlarge'|'ml.g5.8xlarge'|'ml.g5.12xlarge'|'ml.g5.16xlarge'|'ml.g5.24xlarge'|'ml.g5.48xlarge'|'ml.c5.large'|'ml.c5.xlarge'|'ml.c5.2xlarge'|'ml.c5.4xlarge'|'ml.c5.9xlarge'|'ml.c5.12xlarge'|'ml.c5.18xlarge'|'ml.c5.24xlarge'|'ml.c5n.large'|'ml.c5n.2xlarge'|'ml.c5n.4xlarge'|'ml.c5n.9xlarge'|'ml.c5n.18xlarge'|'ml.m5.large'|'ml.m5.xlarge'|'ml.m5.2xlarge'|'ml.m5.4xlarge'|'ml.m5.8xlarge'|'ml.m5.12xlarge'|'ml.m5.16xlarge'|'ml.m5.24xlarge'|'ml.t3.medium'|'ml.t3.large'|'ml.t3.xlarge'|'ml.t3.2xlarge'|'ml.g6.xlarge'|'ml.g6.2xlarge'|'ml.g6.4xlarge'|'ml.g6.8xlarge'|'ml.g6.16xlarge'|'ml.g6.12xlarge'|'ml.g6.24xlarge'|'ml.g6.48xlarge'|'ml.gr6.4xlarge'|'ml.gr6.8xlarge'|'ml.g6e.xlarge'|'ml.g6e.2xlarge'|'ml.g6e.4xlarge'|'ml.g6e.8xlarge'|'ml.g6e.16xlarge'|'ml.g6e.12xlarge'|'ml.g6e.24xlarge'|'ml.g6e.48xlarge'|'ml.p5e.48xlarge'|'ml.p5en.48xlarge'|'ml.p6-b200.48xlarge'|'ml.trn2.3xlarge'|'ml.trn2.48xlarge'|'ml.c6i.large'|'ml.c6i.xlarge'|'ml.c6i.2xlarge'|'ml.c6i.4xlarge'|'ml.c6i.8xlarge'|'ml.c6i.12xlarge'|'ml.c6i.16xlarge'|'ml.c6i.24xlarge'|'ml.c6i.32xlarge'|'ml.m6i.large'|'ml.m6i.xlarge'|'ml.m6i.2xlarge'|'ml.m6i.4xlarge'|'ml.m6i.8xlarge'|'ml.m6i.12xlarge'|'ml.m6i.16xlarge'|'ml.m6i.24xlarge'|'ml.m6i.32xlarge'|'ml.r6i.large'|'ml.r6i.xlarge'|'ml.r6i.2xlarge'|'ml.r6i.4xlarge'|'ml.r6i.8xlarge'|'ml.r6i.12xlarge'|'ml.r6i.16xlarge'|'ml.r6i.24xlarge'|'ml.r6i.32xlarge'|'ml.i3en.large'|'ml.i3en.xlarge'|'ml.i3en.2xlarge'|'ml.i3en.3xlarge'|'ml.i3en.6xlarge'|'ml.i3en.12xlarge'|'ml.i3en.24xlarge'|'ml.m7i.large'|'ml.m7i.xlarge'|'ml.m7i.2xlarge'|'ml.m7i.4xlarge'|'ml.m7i.8xlarge'|'ml.m7i.12xlarge'|'ml.m7i.16xlarge'|'ml.m7i.24xlarge'|'ml.m7i.48xlarge'|'ml.r7i.large'|'ml.r7i.xlarge'|'ml.r7i.2xlarge'|'ml.r7i.4xlarge'|'ml.r7i.8xlarge'|'ml.r7i.12xlarge'|'ml.r7i.16xlarge'|'ml.r7i.24xlarge'|'ml.r7i.48xlarge'|'ml.r5d.16xlarge'|'ml.g7e.2xlarge'|'ml.g7e.4xlarge'|'ml.g7e.8xlarge'|'ml.g7e.12xlarge'|'ml.g7e.24xlarge'|'ml.g7e.48xlarge'|'ml.p6-b300.48xlarge'|'ml.g4dn.xlarge'|'ml.g4dn.2xlarge'|'ml.g4dn.4xlarge'|'ml.g4dn.8xlarge'|'ml.g4dn.12xlarge'|'ml.g4dn.16xlarge'|'ml.c6g.medium'|'ml.c6g.large'|'ml.c6g.xlarge'|'ml.c6g.2xlarge'|'ml.c6g.4xlarge'|'ml.c6g.8xlarge'|'ml.c6g.12xlarge'|'ml.c6g.16xlarge'|'ml.c7g.medium'|'ml.c7g.large'|'ml.c7g.xlarge'|'ml.c7g.2xlarge'|'ml.c7g.4xlarge'|'ml.c7g.8xlarge'|'ml.c7g.12xlarge'|'ml.c7g.16xlarge'|'ml.c8g.medium'|'ml.c8g.large'|'ml.c8g.xlarge'|'ml.c8g.2xlarge'|'ml.c8g.4xlarge'|'ml.c8g.8xlarge'|'ml.c8g.12xlarge'|'ml.c8g.16xlarge'|'ml.c8g.24xlarge'|'ml.c8g.48xlarge'|'ml.c6a.large'|'ml.c6a.xlarge'|'ml.c6a.2xlarge'|'ml.c6a.4xlarge'|'ml.c6a.8xlarge'|'ml.c6a.12xlarge'|'ml.c6a.16xlarge'|'ml.c6a.24xlarge'|'ml.c6a.32xlarge'|'ml.c6a.48xlarge'|'ml.m6a.large'|'ml.m6a.xlarge'|'ml.m6a.2xlarge'|'ml.m6a.4xlarge'|'ml.m6a.8xlarge'|'ml.m6a.12xlarge'|'ml.m6a.16xlarge'|'ml.m6a.24xlarge'|'ml.m6a.32xlarge'|'ml.m6a.48xlarge'|'ml.m6g.medium'|'ml.m6g.large'|'ml.m6g.xlarge'|'ml.m6g.2xlarge'|'ml.m6g.4xlarge'|'ml.m6g.8xlarge'|'ml.m6g.12xlarge'|'ml.m6g.16xlarge'|'ml.m7g.medium'|'ml.m7g.large'|'ml.m7g.xlarge'|'ml.m7g.2xlarge'|'ml.m7g.4xlarge'|'ml.m7g.8xlarge'|'ml.m7g.12xlarge'|'ml.m7g.16xlarge'|'ml.m8g.medium'|'ml.m8g.large'|'ml.m8g.xlarge'|'ml.m8g.2xlarge'|'ml.m8g.4xlarge'|'ml.m8g.8xlarge'|'ml.m8g.12xlarge'|'ml.m8g.16xlarge'|'ml.m8g.24xlarge'|'ml.m8g.48xlarge'|'ml.g7.2xlarge'|'ml.g7.4xlarge'|'ml.g7.8xlarge'|'ml.g7.12xlarge'|'ml.g7.24xlarge'|'ml.g7.48xlarge'|'ml.c8a.16xlarge'|'ml.m8a.16xlarge',
            ],
            'Message': 'string'
        },
    ]
}

Response Structure

  • (dict) --

    • Successful (list) --

      A list of NodeLogicalIDs that were successfully added to the cluster. The NodeLogicalID is unique per cluster and does not change between instance replacements. Each entry includes a NodeLogicalId that can be used to track the node's provisioning status (with DescribeClusterNode), the instance group name, and the current status of the node.

      • (dict) --

        Information about a node that was successfully added to the cluster.

        • NodeLogicalId (string) --

          A unique identifier assigned to the node that can be used to track its provisioning status through the DescribeClusterNode operation.

        • InstanceGroupName (string) --

          The name of the instance group to which the node was added.

        • Status (string) --

          The current status of the node. Possible values include Pending, Running, Failed, ShuttingDown, SystemUpdating, DeepHealthCheckInProgress, and NotFound.

        • AvailabilityZones (list) --

          The availability zones associated with the successfully added node.

          • (string) --

        • InstanceTypes (list) --

          The instance types associated with the successfully added node.

          • (string) --

    • Failed (list) --

      A list of errors that occurred during the node addition operation. Each entry includes the instance group name, error code, number of failed additions, and an error message.

      • (dict) --

        Information about an error that occurred during the node addition operation.

        • InstanceGroupName (string) --

          The name of the instance group for which the error occurred.

        • ErrorCode (string) --

          The error code associated with the failure. Possible values include InstanceGroupNotFound and InvalidInstanceGroupState.

        • FailedCount (integer) --

          The number of nodes that failed to be added to the specified instance group.

        • AvailabilityZones (list) --

          The availability zones associated with the failed node addition request.

          • (string) --

        • InstanceTypes (list) --

          The instance types associated with the failed node addition request.

          • (string) --

        • Message (string) --

          A descriptive message providing additional details about the error.

DescribeClusterNode (updated) Link ¶
Changes (response)
{'NodeDetails': {'InstanceStatus': {'Status': {'AttachInProgress',
                                               'DetachInProgress'}}}}

Retrieves information of a node (also called a instance interchangeably) of a SageMaker HyperPod cluster.

See also: AWS API Documentation

Request Syntax

client.describe_cluster_node(
    ClusterName='string',
    NodeId='string',
    NodeLogicalId='string'
)
type ClusterName:

string

param ClusterName:

[REQUIRED]

The string name or the Amazon Resource Name (ARN) of the SageMaker HyperPod cluster in which the node is.

type NodeId:

string

param NodeId:

The ID of the SageMaker HyperPod cluster node.

type NodeLogicalId:

string

param NodeLogicalId:

The logical identifier of the node to describe. You can specify either NodeLogicalId or InstanceId, but not both. NodeLogicalId can be used to describe nodes that are still being provisioned and don't yet have an InstanceId assigned.

rtype:

dict

returns:

Response Syntax

{
    'NodeDetails': {
        'InstanceGroupName': 'string',
        'InstanceId': 'string',
        'NodeLogicalId': 'string',
        'InstanceStatus': {
            'Status': 'Running'|'Failure'|'Pending'|'ShuttingDown'|'SystemUpdating'|'DeepHealthCheckInProgress'|'NotFound'|'DetachInProgress'|'AttachInProgress',
            'Message': 'string'
        },
        'InstanceType': 'ml.p4d.24xlarge'|'ml.p4de.24xlarge'|'ml.p5.48xlarge'|'ml.p5.4xlarge'|'ml.p6e-gb200.36xlarge'|'ml.trn1.32xlarge'|'ml.trn1n.32xlarge'|'ml.g5.xlarge'|'ml.g5.2xlarge'|'ml.g5.4xlarge'|'ml.g5.8xlarge'|'ml.g5.12xlarge'|'ml.g5.16xlarge'|'ml.g5.24xlarge'|'ml.g5.48xlarge'|'ml.c5.large'|'ml.c5.xlarge'|'ml.c5.2xlarge'|'ml.c5.4xlarge'|'ml.c5.9xlarge'|'ml.c5.12xlarge'|'ml.c5.18xlarge'|'ml.c5.24xlarge'|'ml.c5n.large'|'ml.c5n.2xlarge'|'ml.c5n.4xlarge'|'ml.c5n.9xlarge'|'ml.c5n.18xlarge'|'ml.m5.large'|'ml.m5.xlarge'|'ml.m5.2xlarge'|'ml.m5.4xlarge'|'ml.m5.8xlarge'|'ml.m5.12xlarge'|'ml.m5.16xlarge'|'ml.m5.24xlarge'|'ml.t3.medium'|'ml.t3.large'|'ml.t3.xlarge'|'ml.t3.2xlarge'|'ml.g6.xlarge'|'ml.g6.2xlarge'|'ml.g6.4xlarge'|'ml.g6.8xlarge'|'ml.g6.16xlarge'|'ml.g6.12xlarge'|'ml.g6.24xlarge'|'ml.g6.48xlarge'|'ml.gr6.4xlarge'|'ml.gr6.8xlarge'|'ml.g6e.xlarge'|'ml.g6e.2xlarge'|'ml.g6e.4xlarge'|'ml.g6e.8xlarge'|'ml.g6e.16xlarge'|'ml.g6e.12xlarge'|'ml.g6e.24xlarge'|'ml.g6e.48xlarge'|'ml.p5e.48xlarge'|'ml.p5en.48xlarge'|'ml.p6-b200.48xlarge'|'ml.trn2.3xlarge'|'ml.trn2.48xlarge'|'ml.c6i.large'|'ml.c6i.xlarge'|'ml.c6i.2xlarge'|'ml.c6i.4xlarge'|'ml.c6i.8xlarge'|'ml.c6i.12xlarge'|'ml.c6i.16xlarge'|'ml.c6i.24xlarge'|'ml.c6i.32xlarge'|'ml.m6i.large'|'ml.m6i.xlarge'|'ml.m6i.2xlarge'|'ml.m6i.4xlarge'|'ml.m6i.8xlarge'|'ml.m6i.12xlarge'|'ml.m6i.16xlarge'|'ml.m6i.24xlarge'|'ml.m6i.32xlarge'|'ml.r6i.large'|'ml.r6i.xlarge'|'ml.r6i.2xlarge'|'ml.r6i.4xlarge'|'ml.r6i.8xlarge'|'ml.r6i.12xlarge'|'ml.r6i.16xlarge'|'ml.r6i.24xlarge'|'ml.r6i.32xlarge'|'ml.i3en.large'|'ml.i3en.xlarge'|'ml.i3en.2xlarge'|'ml.i3en.3xlarge'|'ml.i3en.6xlarge'|'ml.i3en.12xlarge'|'ml.i3en.24xlarge'|'ml.m7i.large'|'ml.m7i.xlarge'|'ml.m7i.2xlarge'|'ml.m7i.4xlarge'|'ml.m7i.8xlarge'|'ml.m7i.12xlarge'|'ml.m7i.16xlarge'|'ml.m7i.24xlarge'|'ml.m7i.48xlarge'|'ml.r7i.large'|'ml.r7i.xlarge'|'ml.r7i.2xlarge'|'ml.r7i.4xlarge'|'ml.r7i.8xlarge'|'ml.r7i.12xlarge'|'ml.r7i.16xlarge'|'ml.r7i.24xlarge'|'ml.r7i.48xlarge'|'ml.r5d.16xlarge'|'ml.g7e.2xlarge'|'ml.g7e.4xlarge'|'ml.g7e.8xlarge'|'ml.g7e.12xlarge'|'ml.g7e.24xlarge'|'ml.g7e.48xlarge'|'ml.p6-b300.48xlarge'|'ml.g4dn.xlarge'|'ml.g4dn.2xlarge'|'ml.g4dn.4xlarge'|'ml.g4dn.8xlarge'|'ml.g4dn.12xlarge'|'ml.g4dn.16xlarge'|'ml.c6g.medium'|'ml.c6g.large'|'ml.c6g.xlarge'|'ml.c6g.2xlarge'|'ml.c6g.4xlarge'|'ml.c6g.8xlarge'|'ml.c6g.12xlarge'|'ml.c6g.16xlarge'|'ml.c7g.medium'|'ml.c7g.large'|'ml.c7g.xlarge'|'ml.c7g.2xlarge'|'ml.c7g.4xlarge'|'ml.c7g.8xlarge'|'ml.c7g.12xlarge'|'ml.c7g.16xlarge'|'ml.c8g.medium'|'ml.c8g.large'|'ml.c8g.xlarge'|'ml.c8g.2xlarge'|'ml.c8g.4xlarge'|'ml.c8g.8xlarge'|'ml.c8g.12xlarge'|'ml.c8g.16xlarge'|'ml.c8g.24xlarge'|'ml.c8g.48xlarge'|'ml.c6a.large'|'ml.c6a.xlarge'|'ml.c6a.2xlarge'|'ml.c6a.4xlarge'|'ml.c6a.8xlarge'|'ml.c6a.12xlarge'|'ml.c6a.16xlarge'|'ml.c6a.24xlarge'|'ml.c6a.32xlarge'|'ml.c6a.48xlarge'|'ml.m6a.large'|'ml.m6a.xlarge'|'ml.m6a.2xlarge'|'ml.m6a.4xlarge'|'ml.m6a.8xlarge'|'ml.m6a.12xlarge'|'ml.m6a.16xlarge'|'ml.m6a.24xlarge'|'ml.m6a.32xlarge'|'ml.m6a.48xlarge'|'ml.m6g.medium'|'ml.m6g.large'|'ml.m6g.xlarge'|'ml.m6g.2xlarge'|'ml.m6g.4xlarge'|'ml.m6g.8xlarge'|'ml.m6g.12xlarge'|'ml.m6g.16xlarge'|'ml.m7g.medium'|'ml.m7g.large'|'ml.m7g.xlarge'|'ml.m7g.2xlarge'|'ml.m7g.4xlarge'|'ml.m7g.8xlarge'|'ml.m7g.12xlarge'|'ml.m7g.16xlarge'|'ml.m8g.medium'|'ml.m8g.large'|'ml.m8g.xlarge'|'ml.m8g.2xlarge'|'ml.m8g.4xlarge'|'ml.m8g.8xlarge'|'ml.m8g.12xlarge'|'ml.m8g.16xlarge'|'ml.m8g.24xlarge'|'ml.m8g.48xlarge'|'ml.g7.2xlarge'|'ml.g7.4xlarge'|'ml.g7.8xlarge'|'ml.g7.12xlarge'|'ml.g7.24xlarge'|'ml.g7.48xlarge'|'ml.c8a.16xlarge'|'ml.m8a.16xlarge',
        'LaunchTime': datetime(2015, 1, 1),
        'LastSoftwareUpdateTime': datetime(2015, 1, 1),
        'LifeCycleConfig': {
            'SourceS3Uri': 'string',
            'OnCreate': 'string',
            'OnInitComplete': 'string'
        },
        'OverrideVpcConfig': {
            'SecurityGroupIds': [
                'string',
            ],
            'Subnets': [
                'string',
            ]
        },
        'ThreadsPerCore': 123,
        'InstanceStorageConfigs': [
            {
                'EbsVolumeConfig': {
                    'VolumeSizeInGB': 123,
                    'VolumeKmsKeyId': 'string',
                    'RootVolume': True|False
                },
                'FsxLustreConfig': {
                    'DnsName': 'string',
                    'MountName': 'string',
                    'MountPath': 'string'
                },
                'FsxOpenZfsConfig': {
                    'DnsName': 'string',
                    'MountPath': 'string'
                }
            },
        ],
        'PrivatePrimaryIp': 'string',
        'PrivatePrimaryIpv6': 'string',
        'PrivateDnsHostname': 'string',
        'Placement': {
            'AvailabilityZone': 'string',
            'AvailabilityZoneId': 'string'
        },
        'CurrentImageId': 'string',
        'DesiredImageId': 'string',
        'CurrentImageReleaseVersion': 'string',
        'DesiredImageReleaseVersion': 'string',
        'ImageVersionStatus': 'UpToDate'|'UpdateAvailable'|'SecurityUpdateRequired'|'EndOfLife',
        'UltraServerInfo': {
            'Id': 'string',
            'Type': 'string'
        },
        'KubernetesConfig': {
            'CurrentLabels': {
                'string': 'string'
            },
            'DesiredLabels': {
                'string': 'string'
            },
            'CurrentTaints': [
                {
                    'Key': 'string',
                    'Value': 'string',
                    'Effect': 'NoSchedule'|'PreferNoSchedule'|'NoExecute'
                },
            ],
            'DesiredTaints': [
                {
                    'Key': 'string',
                    'Value': 'string',
                    'Effect': 'NoSchedule'|'PreferNoSchedule'|'NoExecute'
                },
            ]
        },
        'CapacityType': 'Spot'|'OnDemand',
        'NetworkInterface': {
            'InterfaceType': 'efa'|'efa-only'
        }
    }
}

Response Structure

  • (dict) --

    • NodeDetails (dict) --

      The details of the SageMaker HyperPod cluster node.

      • InstanceGroupName (string) --

        The instance group name in which the instance is.

      • InstanceId (string) --

        The ID of the instance.

      • NodeLogicalId (string) --

        A unique identifier for the node that persists throughout its lifecycle, from provisioning request to termination. This identifier can be used to track the node even before it has an assigned InstanceId.

      • InstanceStatus (dict) --

        The status of the instance.

        • Status (string) --

          The status of an instance in a SageMaker HyperPod cluster.

        • Message (string) --

          The message from an instance in a SageMaker HyperPod cluster.

      • InstanceType (string) --

        The type of the instance.

      • LaunchTime (datetime) --

        The time when the instance is launched.

      • LastSoftwareUpdateTime (datetime) --

        The time when the cluster was last updated.

      • LifeCycleConfig (dict) --

        The LifeCycle configuration applied to the instance.

        • SourceS3Uri (string) --

          An Amazon S3 bucket path where your lifecycle scripts are stored.

        • OnCreate (string) --

          The file name of the entrypoint script of lifecycle scripts under SourceS3Uri. This entrypoint script runs during cluster creation.

        • OnInitComplete (string) --

          The file name of the entrypoint script of lifecycle scripts under SourceS3Uri. This script runs on the node after the AMI-based initialization is complete.

      • OverrideVpcConfig (dict) --

        The customized Amazon VPC configuration at the instance group level that overrides the default Amazon VPC configuration of the SageMaker HyperPod cluster.

        • SecurityGroupIds (list) --

          The VPC security group IDs, in the form sg-xxxxxxxx. Specify the security groups for the VPC that is specified in the Subnets field.

          • (string) --

        • Subnets (list) --

          The ID of the subnets in the VPC to which you want to connect your training job or model. For information about the availability of specific instance types, see Supported Instance Types and Availability Zones.

          • (string) --

      • ThreadsPerCore (integer) --

        The number of threads per CPU core you specified under CreateCluster.

      • InstanceStorageConfigs (list) --

        The configurations of additional storage specified to the instance group where the instance (node) is launched.

        • (dict) --

          Defines the configuration for attaching additional storage to the instances in the SageMaker HyperPod cluster instance group. To learn more, see SageMaker HyperPod release notes: June 20, 2024.

          • EbsVolumeConfig (dict) --

            Defines the configuration for attaching additional Amazon Elastic Block Store (EBS) volumes to the instances in the SageMaker HyperPod cluster instance group. The additional EBS volume is attached to each instance within the SageMaker HyperPod cluster instance group and mounted to /opt/sagemaker.

            • VolumeSizeInGB (integer) --

              The size in gigabytes (GB) of the additional EBS volume to be attached to the instances in the SageMaker HyperPod cluster instance group. The additional EBS volume is attached to each instance within the SageMaker HyperPod cluster instance group and mounted to /opt/sagemaker.

            • VolumeKmsKeyId (string) --

              The ID of a KMS key to encrypt the Amazon EBS volume.

            • RootVolume (boolean) --

              Specifies whether the configuration is for the cluster's root or secondary Amazon EBS volume. You can specify two ClusterEbsVolumeConfig fields to configure both the root and secondary volumes. Set the value to True if you'd like to provide your own customer managed Amazon Web Services KMS key to encrypt the root volume. When True:

              • The configuration is applied to the root volume.

              • You can't specify the VolumeSizeInGB field. The size of the root volume is determined for you.

              • You must specify a KMS key ID for VolumeKmsKeyId to encrypt the root volume with your own KMS key instead of an Amazon Web Services owned KMS key.

              Otherwise, by default, the value is False, and the following applies:

              • The configuration is applied to the secondary volume, while the root volume is encrypted with an Amazon Web Services owned key.

              • You must specify the VolumeSizeInGB field.

              • You can optionally specify the VolumeKmsKeyId to encrypt the secondary volume with your own KMS key instead of an Amazon Web Services owned KMS key.

          • FsxLustreConfig (dict) --

            Defines the configuration for attaching an Amazon FSx for Lustre file system to the instances in the SageMaker HyperPod cluster instance group.

            • DnsName (string) --

              The DNS name of the Amazon FSx for Lustre file system.

            • MountName (string) --

              The mount name of the Amazon FSx for Lustre file system.

            • MountPath (string) --

              The local path where the Amazon FSx for Lustre file system is mounted on instances.

          • FsxOpenZfsConfig (dict) --

            Defines the configuration for attaching an Amazon FSx for OpenZFS file system to the instances in the SageMaker HyperPod cluster instance group.

            • DnsName (string) --

              The DNS name of the Amazon FSx for OpenZFS file system.

            • MountPath (string) --

              The local path where the Amazon FSx for OpenZFS file system is mounted on instances.

      • PrivatePrimaryIp (string) --

        The private primary IP address of the SageMaker HyperPod cluster node.

      • PrivatePrimaryIpv6 (string) --

        The private primary IPv6 address of the SageMaker HyperPod cluster node when configured with an Amazon VPC that supports IPv6 and includes subnets with IPv6 addressing enabled in either the cluster Amazon VPC configuration or the instance group Amazon VPC configuration.

      • PrivateDnsHostname (string) --

        The private DNS hostname of the SageMaker HyperPod cluster node.

      • Placement (dict) --

        The placement details of the SageMaker HyperPod cluster node.

        • AvailabilityZone (string) --

          The Availability Zone where the node in the SageMaker HyperPod cluster is launched.

        • AvailabilityZoneId (string) --

          The unique identifier (ID) of the Availability Zone where the node in the SageMaker HyperPod cluster is launched.

      • CurrentImageId (string) --

        The ID of the Amazon Machine Image (AMI) currently in use by the node.

      • DesiredImageId (string) --

        The ID of the Amazon Machine Image (AMI) desired for the node.

      • CurrentImageReleaseVersion (string) --

        The version of the HyperPod-managed AMI currently running on the node.

      • DesiredImageReleaseVersion (string) --

        The desired version of the HyperPod-managed AMI for the node. This may differ from the current version when an update is pending.

      • ImageVersionStatus (string) --

        The status of the image version for the cluster node.

      • UltraServerInfo (dict) --

        Contains information about the UltraServer.

        • Id (string) --

          The unique identifier of the UltraServer.

        • Type (string) --

          The type of the UltraServer.

      • KubernetesConfig (dict) --

        The Kubernetes configuration applied to this node, showing both the current and desired state of labels and taints. The cluster works to reconcile the actual state with the declared state.

        • CurrentLabels (dict) --

          The current labels applied to the cluster node.

          • (string) --

            • (string) --

        • DesiredLabels (dict) --

          The desired labels to be applied to the cluster node.

          • (string) --

            • (string) --

        • CurrentTaints (list) --

          The current taints applied to the cluster node.

          • (dict) --

            A Kubernetes taint that can be applied to cluster nodes.

            • Key (string) --

              The key of the taint.

            • Value (string) --

              The value of the taint.

            • Effect (string) --

              The effect of the taint. Valid values are NoSchedule, PreferNoSchedule, and NoExecute.

        • DesiredTaints (list) --

          The desired taints to be applied to the cluster node.

          • (dict) --

            A Kubernetes taint that can be applied to cluster nodes.

            • Key (string) --

              The key of the taint.

            • Value (string) --

              The value of the taint.

            • Effect (string) --

              The effect of the taint. Valid values are NoSchedule, PreferNoSchedule, and NoExecute.

      • CapacityType (string) --

        The capacity type of the node. Valid values are OnDemand and Spot. When set to OnDemand, the node is launched as an On-Demand instance. When set to Spot, the node is launched as a Spot instance.

      • NetworkInterface (dict) --

        The network interface configuration for the cluster node.

        • InterfaceType (string) --

          The type of network interface for the instance group. Valid values are efa and efa-only.

ListClusterNodes (updated) Link ¶
Changes (response)
{'ClusterNodeSummaries': {'InstanceStatus': {'Status': {'AttachInProgress',
                                                        'DetachInProgress'}}}}

Retrieves the list of instances (also called nodes interchangeably) in a SageMaker HyperPod cluster.

See also: AWS API Documentation

Request Syntax

client.list_cluster_nodes(
    ClusterName='string',
    CreationTimeAfter=datetime(2015, 1, 1),
    CreationTimeBefore=datetime(2015, 1, 1),
    InstanceGroupNameContains='string',
    MaxResults=123,
    NextToken='string',
    SortBy='CREATION_TIME'|'NAME',
    SortOrder='Ascending'|'Descending',
    IncludeNodeLogicalIds=True|False
)
type ClusterName:

string

param ClusterName:

[REQUIRED]

The string name or the Amazon Resource Name (ARN) of the SageMaker HyperPod cluster in which you want to retrieve the list of nodes.

type CreationTimeAfter:

datetime

param CreationTimeAfter:

A filter that returns nodes in a SageMaker HyperPod cluster created after the specified time. Timestamps are formatted according to the ISO 8601 standard.

Acceptable formats include:

  • YYYY-MM-DDThh:mm:ss.sssTZD (UTC), for example, 2014-10-01T20:30:00.000Z

  • YYYY-MM-DDThh:mm:ss.sssTZD (with offset), for example, 2014-10-01T12:30:00.000-08:00

  • YYYY-MM-DD, for example, 2014-10-01

  • Unix time in seconds, for example, 1412195400. This is also referred to as Unix Epoch time and represents the number of seconds since midnight, January 1, 1970 UTC.

For more information about the timestamp format, see Timestamp in the Amazon Web Services Command Line Interface User Guide.

type CreationTimeBefore:

datetime

param CreationTimeBefore:

A filter that returns nodes in a SageMaker HyperPod cluster created before the specified time. The acceptable formats are the same as the timestamp formats for CreationTimeAfter. For more information about the timestamp format, see Timestamp in the Amazon Web Services Command Line Interface User Guide.

type InstanceGroupNameContains:

string

param InstanceGroupNameContains:

A filter that returns the instance groups whose name contain a specified string.

type MaxResults:

integer

param MaxResults:

The maximum number of nodes to return in the response.

type NextToken:

string

param NextToken:

If the result of the previous ListClusterNodes request was truncated, the response includes a NextToken. To retrieve the next set of cluster nodes, use the token in the next request.

type SortBy:

string

param SortBy:

The field by which to sort results. The default value is CREATION_TIME.

type SortOrder:

string

param SortOrder:

The sort order for results. The default value is Ascending.

type IncludeNodeLogicalIds:

boolean

param IncludeNodeLogicalIds:

Specifies whether to include nodes that are still being provisioned in the response. When set to true, the response includes all nodes regardless of their provisioning status. When set to False (default), only nodes with assigned InstanceIds are returned.

rtype:

dict

returns:

Response Syntax

{
    'NextToken': 'string',
    'ClusterNodeSummaries': [
        {
            'InstanceGroupName': 'string',
            'InstanceId': 'string',
            'NodeLogicalId': 'string',
            'InstanceType': 'ml.p4d.24xlarge'|'ml.p4de.24xlarge'|'ml.p5.48xlarge'|'ml.p5.4xlarge'|'ml.p6e-gb200.36xlarge'|'ml.trn1.32xlarge'|'ml.trn1n.32xlarge'|'ml.g5.xlarge'|'ml.g5.2xlarge'|'ml.g5.4xlarge'|'ml.g5.8xlarge'|'ml.g5.12xlarge'|'ml.g5.16xlarge'|'ml.g5.24xlarge'|'ml.g5.48xlarge'|'ml.c5.large'|'ml.c5.xlarge'|'ml.c5.2xlarge'|'ml.c5.4xlarge'|'ml.c5.9xlarge'|'ml.c5.12xlarge'|'ml.c5.18xlarge'|'ml.c5.24xlarge'|'ml.c5n.large'|'ml.c5n.2xlarge'|'ml.c5n.4xlarge'|'ml.c5n.9xlarge'|'ml.c5n.18xlarge'|'ml.m5.large'|'ml.m5.xlarge'|'ml.m5.2xlarge'|'ml.m5.4xlarge'|'ml.m5.8xlarge'|'ml.m5.12xlarge'|'ml.m5.16xlarge'|'ml.m5.24xlarge'|'ml.t3.medium'|'ml.t3.large'|'ml.t3.xlarge'|'ml.t3.2xlarge'|'ml.g6.xlarge'|'ml.g6.2xlarge'|'ml.g6.4xlarge'|'ml.g6.8xlarge'|'ml.g6.16xlarge'|'ml.g6.12xlarge'|'ml.g6.24xlarge'|'ml.g6.48xlarge'|'ml.gr6.4xlarge'|'ml.gr6.8xlarge'|'ml.g6e.xlarge'|'ml.g6e.2xlarge'|'ml.g6e.4xlarge'|'ml.g6e.8xlarge'|'ml.g6e.16xlarge'|'ml.g6e.12xlarge'|'ml.g6e.24xlarge'|'ml.g6e.48xlarge'|'ml.p5e.48xlarge'|'ml.p5en.48xlarge'|'ml.p6-b200.48xlarge'|'ml.trn2.3xlarge'|'ml.trn2.48xlarge'|'ml.c6i.large'|'ml.c6i.xlarge'|'ml.c6i.2xlarge'|'ml.c6i.4xlarge'|'ml.c6i.8xlarge'|'ml.c6i.12xlarge'|'ml.c6i.16xlarge'|'ml.c6i.24xlarge'|'ml.c6i.32xlarge'|'ml.m6i.large'|'ml.m6i.xlarge'|'ml.m6i.2xlarge'|'ml.m6i.4xlarge'|'ml.m6i.8xlarge'|'ml.m6i.12xlarge'|'ml.m6i.16xlarge'|'ml.m6i.24xlarge'|'ml.m6i.32xlarge'|'ml.r6i.large'|'ml.r6i.xlarge'|'ml.r6i.2xlarge'|'ml.r6i.4xlarge'|'ml.r6i.8xlarge'|'ml.r6i.12xlarge'|'ml.r6i.16xlarge'|'ml.r6i.24xlarge'|'ml.r6i.32xlarge'|'ml.i3en.large'|'ml.i3en.xlarge'|'ml.i3en.2xlarge'|'ml.i3en.3xlarge'|'ml.i3en.6xlarge'|'ml.i3en.12xlarge'|'ml.i3en.24xlarge'|'ml.m7i.large'|'ml.m7i.xlarge'|'ml.m7i.2xlarge'|'ml.m7i.4xlarge'|'ml.m7i.8xlarge'|'ml.m7i.12xlarge'|'ml.m7i.16xlarge'|'ml.m7i.24xlarge'|'ml.m7i.48xlarge'|'ml.r7i.large'|'ml.r7i.xlarge'|'ml.r7i.2xlarge'|'ml.r7i.4xlarge'|'ml.r7i.8xlarge'|'ml.r7i.12xlarge'|'ml.r7i.16xlarge'|'ml.r7i.24xlarge'|'ml.r7i.48xlarge'|'ml.r5d.16xlarge'|'ml.g7e.2xlarge'|'ml.g7e.4xlarge'|'ml.g7e.8xlarge'|'ml.g7e.12xlarge'|'ml.g7e.24xlarge'|'ml.g7e.48xlarge'|'ml.p6-b300.48xlarge'|'ml.g4dn.xlarge'|'ml.g4dn.2xlarge'|'ml.g4dn.4xlarge'|'ml.g4dn.8xlarge'|'ml.g4dn.12xlarge'|'ml.g4dn.16xlarge'|'ml.c6g.medium'|'ml.c6g.large'|'ml.c6g.xlarge'|'ml.c6g.2xlarge'|'ml.c6g.4xlarge'|'ml.c6g.8xlarge'|'ml.c6g.12xlarge'|'ml.c6g.16xlarge'|'ml.c7g.medium'|'ml.c7g.large'|'ml.c7g.xlarge'|'ml.c7g.2xlarge'|'ml.c7g.4xlarge'|'ml.c7g.8xlarge'|'ml.c7g.12xlarge'|'ml.c7g.16xlarge'|'ml.c8g.medium'|'ml.c8g.large'|'ml.c8g.xlarge'|'ml.c8g.2xlarge'|'ml.c8g.4xlarge'|'ml.c8g.8xlarge'|'ml.c8g.12xlarge'|'ml.c8g.16xlarge'|'ml.c8g.24xlarge'|'ml.c8g.48xlarge'|'ml.c6a.large'|'ml.c6a.xlarge'|'ml.c6a.2xlarge'|'ml.c6a.4xlarge'|'ml.c6a.8xlarge'|'ml.c6a.12xlarge'|'ml.c6a.16xlarge'|'ml.c6a.24xlarge'|'ml.c6a.32xlarge'|'ml.c6a.48xlarge'|'ml.m6a.large'|'ml.m6a.xlarge'|'ml.m6a.2xlarge'|'ml.m6a.4xlarge'|'ml.m6a.8xlarge'|'ml.m6a.12xlarge'|'ml.m6a.16xlarge'|'ml.m6a.24xlarge'|'ml.m6a.32xlarge'|'ml.m6a.48xlarge'|'ml.m6g.medium'|'ml.m6g.large'|'ml.m6g.xlarge'|'ml.m6g.2xlarge'|'ml.m6g.4xlarge'|'ml.m6g.8xlarge'|'ml.m6g.12xlarge'|'ml.m6g.16xlarge'|'ml.m7g.medium'|'ml.m7g.large'|'ml.m7g.xlarge'|'ml.m7g.2xlarge'|'ml.m7g.4xlarge'|'ml.m7g.8xlarge'|'ml.m7g.12xlarge'|'ml.m7g.16xlarge'|'ml.m8g.medium'|'ml.m8g.large'|'ml.m8g.xlarge'|'ml.m8g.2xlarge'|'ml.m8g.4xlarge'|'ml.m8g.8xlarge'|'ml.m8g.12xlarge'|'ml.m8g.16xlarge'|'ml.m8g.24xlarge'|'ml.m8g.48xlarge'|'ml.g7.2xlarge'|'ml.g7.4xlarge'|'ml.g7.8xlarge'|'ml.g7.12xlarge'|'ml.g7.24xlarge'|'ml.g7.48xlarge'|'ml.c8a.16xlarge'|'ml.m8a.16xlarge',
            'LaunchTime': datetime(2015, 1, 1),
            'LastSoftwareUpdateTime': datetime(2015, 1, 1),
            'InstanceStatus': {
                'Status': 'Running'|'Failure'|'Pending'|'ShuttingDown'|'SystemUpdating'|'DeepHealthCheckInProgress'|'NotFound'|'DetachInProgress'|'AttachInProgress',
                'Message': 'string'
            },
            'UltraServerInfo': {
                'Id': 'string',
                'Type': 'string'
            },
            'PrivateDnsHostname': 'string',
            'CurrentImageReleaseVersion': 'string',
            'ImageVersionStatus': 'UpToDate'|'UpdateAvailable'|'SecurityUpdateRequired'|'EndOfLife'
        },
    ]
}

Response Structure

  • (dict) --

    • NextToken (string) --

      The next token specified for listing instances in a SageMaker HyperPod cluster.

    • ClusterNodeSummaries (list) --

      The summaries of listed instances in a SageMaker HyperPod cluster

      • (dict) --

        Lists a summary of the properties of an instance (also called a node interchangeably) of a SageMaker HyperPod cluster.

        • InstanceGroupName (string) --

          The name of the instance group in which the instance is.

        • InstanceId (string) --

          The ID of the instance.

        • NodeLogicalId (string) --

          A unique identifier for the node that persists throughout its lifecycle, from provisioning request to termination. This identifier can be used to track the node even before it has an assigned InstanceId. This field is only included when IncludeNodeLogicalIds is set to True in the ListClusterNodes request.

        • InstanceType (string) --

          The type of the instance.

        • LaunchTime (datetime) --

          The time when the instance is launched.

        • LastSoftwareUpdateTime (datetime) --

          The time when SageMaker last updated the software of the instances in the cluster.

        • InstanceStatus (dict) --

          The status of the instance.

          • Status (string) --

            The status of an instance in a SageMaker HyperPod cluster.

          • Message (string) --

            The message from an instance in a SageMaker HyperPod cluster.

        • UltraServerInfo (dict) --

          Contains information about the UltraServer.

          • Id (string) --

            The unique identifier of the UltraServer.

          • Type (string) --

            The type of the UltraServer.

        • PrivateDnsHostname (string) --

          The private DNS hostname of the SageMaker HyperPod cluster node.

        • CurrentImageReleaseVersion (string) --

          The version of the HyperPod-managed AMI currently running on the node.

        • ImageVersionStatus (string) --

          The status of the image version for the cluster node.