2026/06/05 - Amazon SageMaker Service - 4 updated api methods
Changes This release adds support for MLflow experiment tracking in SageMaker inference optimization. CreateAIRecommendationJob and CreateAIBenchmarkJob now accept an optional OutputConfig.MlflowConfig (MLflow App ARN, experiment, run name) to stream benchmark metrics and artifacts to your own MLflow App.
{'OutputConfig': {'MlflowConfig': {'MlflowExperimentName': 'string',
'MlflowResourceArn': 'string',
'MlflowRunName': 'string'}}}
Creates a benchmark job that runs performance benchmarks against inference infrastructure using a predefined AI workload configuration. The benchmark job measures metrics such as latency, throughput, and cost for your generative AI inference endpoints.
See also: AWS API Documentation
Request Syntax
client.create_ai_benchmark_job(
AIBenchmarkJobName='string',
BenchmarkTarget={
'Endpoint': {
'Identifier': 'string',
'TargetContainerHostname': 'string',
'InferenceComponents': [
{
'Identifier': 'string'
},
]
}
},
OutputConfig={
'S3OutputLocation': 'string',
'MlflowConfig': {
'MlflowResourceArn': 'string',
'MlflowExperimentName': 'string',
'MlflowRunName': 'string'
}
},
AIWorkloadConfigIdentifier='string',
RoleArn='string',
NetworkConfig={
'VpcConfig': {
'SecurityGroupIds': [
'string',
],
'Subnets': [
'string',
]
}
},
Tags=[
{
'Key': 'string',
'Value': 'string'
},
]
)
string
[REQUIRED]
The name of the AI benchmark job. The name must be unique within your Amazon Web Services account in the current Amazon Web Services Region.
dict
[REQUIRED]
The target endpoint to benchmark. Specify a SageMaker endpoint by providing its name or Amazon Resource Name (ARN).
Endpoint (dict) --
The SageMaker endpoint to benchmark.
Identifier (string) -- [REQUIRED]
The name or Amazon Resource Name (ARN) of the SageMaker endpoint to benchmark.
TargetContainerHostname (string) --
The hostname of the specific container to target within a multi-container endpoint.
InferenceComponents (list) --
The list of inference components to benchmark on the endpoint.
(dict) --
An inference component to benchmark.
Identifier (string) -- [REQUIRED]
The name or Amazon Resource Name (ARN) of the inference component.
dict
[REQUIRED]
The output configuration for the benchmark job, including the Amazon S3 location where benchmark results are stored.
S3OutputLocation (string) -- [REQUIRED]
The Amazon S3 URI where benchmark results are stored.
MlflowConfig (dict) --
The MLflow tracking configuration for the job. If you don't specify this parameter, MLflow tracking is disabled.
MlflowResourceArn (string) -- [REQUIRED]
The Amazon Resource Name (ARN) of the SageMaker managed MLflow resource.
MlflowExperimentName (string) --
The MLflow experiment name used for tracking.
MlflowRunName (string) --
The MLflow run name used for tracking.
string
[REQUIRED]
The name or Amazon Resource Name (ARN) of the AI workload configuration to use for this benchmark job.
string
[REQUIRED]
The Amazon Resource Name (ARN) of an IAM role that enables Amazon SageMaker AI to perform tasks on your behalf.
dict
The network configuration for the benchmark job, including VPC settings.
VpcConfig (dict) --
The VPC configuration, including security group IDs and subnet IDs.
SecurityGroupIds (list) -- [REQUIRED]
The VPC security group IDs, in the form sg-xxxxxxxx. Specify the security groups for the VPC that is specified in the Subnets field.
(string) --
Subnets (list) -- [REQUIRED]
The ID of the subnets in the VPC to which you want to connect your training job or model. For information about the availability of specific instance types, see Supported Instance Types and Availability Zones.
(string) --
list
The metadata that you apply to Amazon Web Services resources to help you categorize and organize them. Each tag consists of a key and a value, both of which you define.
(dict) --
A tag object that consists of a key and an optional value, used to manage metadata for SageMaker Amazon Web Services resources.
You can add tags to notebook instances, training jobs, hyperparameter tuning jobs, batch transform jobs, models, labeling jobs, work teams, endpoint configurations, and endpoints. For more information on adding tags to SageMaker resources, see AddTags.
For more information on adding metadata to your Amazon Web Services resources with tagging, see Tagging Amazon Web Services resources. For advice on best practices for managing Amazon Web Services resources with tagging, see Tagging Best Practices: Implement an Effective Amazon Web Services Resource Tagging Strategy.
Key (string) -- [REQUIRED]
The tag key. Tag keys must be unique per resource.
Value (string) -- [REQUIRED]
The tag value.
dict
Response Syntax
{
'AIBenchmarkJobArn': 'string'
}
Response Structure
(dict) --
AIBenchmarkJobArn (string) --
The Amazon Resource Name (ARN) of the created benchmark job.
{'OutputConfig': {'MlflowConfig': {'MlflowExperimentName': 'string',
'MlflowResourceArn': 'string',
'MlflowRunName': 'string'}}}
Creates a recommendation job that generates intelligent optimization recommendations for generative AI inference deployments. The job analyzes your model, workload configuration, and performance targets to recommend optimal instance types, model optimization techniques (such as quantization and speculative decoding), and deployment configurations.
See also: AWS API Documentation
Request Syntax
client.create_ai_recommendation_job(
AIRecommendationJobName='string',
ModelSource={
'S3': {
'S3Uri': 'string'
}
},
OutputConfig={
'S3OutputLocation': 'string',
'ModelPackageGroupIdentifier': 'string',
'MlflowConfig': {
'MlflowResourceArn': 'string',
'MlflowExperimentName': 'string',
'MlflowRunName': 'string'
}
},
AIWorkloadConfigIdentifier='string',
PerformanceTarget={
'Constraints': [
{
'Metric': 'ttft-ms'|'throughput'|'cost'
},
]
},
RoleArn='string',
InferenceSpecification={
'Framework': 'LMI'|'VLLM'
},
OptimizeModel=True|False,
ComputeSpec={
'InstanceTypes': [
'ml.g5.xlarge'|'ml.g5.2xlarge'|'ml.g5.4xlarge'|'ml.g5.8xlarge'|'ml.g5.12xlarge'|'ml.g5.16xlarge'|'ml.g5.24xlarge'|'ml.g5.48xlarge'|'ml.g6.xlarge'|'ml.g6.2xlarge'|'ml.g6.4xlarge'|'ml.g6.8xlarge'|'ml.g6.12xlarge'|'ml.g6.16xlarge'|'ml.g6.24xlarge'|'ml.g6.48xlarge'|'ml.g6e.xlarge'|'ml.g6e.2xlarge'|'ml.g6e.4xlarge'|'ml.g6e.8xlarge'|'ml.g6e.12xlarge'|'ml.g6e.16xlarge'|'ml.g6e.24xlarge'|'ml.g6e.48xlarge'|'ml.g7e.2xlarge'|'ml.g7e.4xlarge'|'ml.g7e.8xlarge'|'ml.g7e.12xlarge'|'ml.g7e.24xlarge'|'ml.g7e.48xlarge'|'ml.p3.2xlarge'|'ml.p3.8xlarge'|'ml.p3.16xlarge'|'ml.p4d.24xlarge'|'ml.p4de.24xlarge'|'ml.p5.4xlarge'|'ml.p5.48xlarge'|'ml.p5e.48xlarge'|'ml.p5en.48xlarge'|'ml.p6-b200.48xlarge',
],
'CapacityReservationConfig': {
'CapacityReservationPreference': 'capacity-reservations-only',
'MlReservationArns': [
'string',
]
}
},
Tags=[
{
'Key': 'string',
'Value': 'string'
},
]
)
string
[REQUIRED]
The name of the AI recommendation job. The name must be unique within your Amazon Web Services account in the current Amazon Web Services Region.
dict
[REQUIRED]
The source of the model to optimize. Specify the Amazon S3 location of the model artifacts.
S3 (dict) --
The Amazon S3 location of the model artifacts.
S3Uri (string) --
The Amazon S3 URI of the model artifacts.
dict
[REQUIRED]
The output configuration for the recommendation job, including the Amazon S3 location for results and an optional model package group where the optimized model is registered.
S3OutputLocation (string) --
The Amazon S3 URI where recommendation results are stored.
ModelPackageGroupIdentifier (string) --
The name or Amazon Resource Name (ARN) of the model package group where the optimized model is registered as a new model package version.
MlflowConfig (dict) --
The MLflow tracking configuration for the job. If you don't specify this parameter, MLflow tracking is disabled.
MlflowResourceArn (string) -- [REQUIRED]
The Amazon Resource Name (ARN) of the SageMaker managed MLflow resource.
MlflowExperimentName (string) --
The MLflow experiment name used for tracking.
MlflowRunName (string) --
The MLflow run name used for tracking.
string
[REQUIRED]
The name or Amazon Resource Name (ARN) of the AI workload configuration to use for this recommendation job.
dict
[REQUIRED]
The performance targets for the recommendation job. Specify constraints on metrics such as time to first token ( ttft-ms), throughput, or cost.
Constraints (list) -- [REQUIRED]
An array of performance constraints that define the optimization objectives.
(dict) --
A performance constraint for an AI recommendation job.
Metric (string) -- [REQUIRED]
The performance metric. Valid values are ttft-ms (time to first token in milliseconds), throughput, and cost.
string
[REQUIRED]
The Amazon Resource Name (ARN) of an IAM role that enables Amazon SageMaker AI to perform tasks on your behalf.
dict
The inference framework configuration. Specify the framework (such as LMI or vLLM) for the recommendation job.
Framework (string) --
The inference framework. Valid values are LMI and VLLM.
boolean
Whether to allow model optimization techniques such as quantization, speculative decoding, and kernel tuning. The default is true.
dict
The compute resource specification for the recommendation job. You can specify up to 3 instance types to consider, and optionally provide capacity reservation configuration.
InstanceTypes (list) --
The list of instance types to consider for recommendations. You can specify up to 3 instance types.
(string) --
CapacityReservationConfig (dict) --
The capacity reservation configuration.
CapacityReservationPreference (string) --
The capacity reservation preference. The only valid value is capacity-reservations-only.
MlReservationArns (list) --
The list of ML reservation ARNs to use.
(string) --
list
The metadata that you apply to Amazon Web Services resources to help you categorize and organize them.
(dict) --
A tag object that consists of a key and an optional value, used to manage metadata for SageMaker Amazon Web Services resources.
You can add tags to notebook instances, training jobs, hyperparameter tuning jobs, batch transform jobs, models, labeling jobs, work teams, endpoint configurations, and endpoints. For more information on adding tags to SageMaker resources, see AddTags.
For more information on adding metadata to your Amazon Web Services resources with tagging, see Tagging Amazon Web Services resources. For advice on best practices for managing Amazon Web Services resources with tagging, see Tagging Best Practices: Implement an Effective Amazon Web Services Resource Tagging Strategy.
Key (string) -- [REQUIRED]
The tag key. Tag keys must be unique per resource.
Value (string) -- [REQUIRED]
The tag value.
dict
Response Syntax
{
'AIRecommendationJobArn': 'string'
}
Response Structure
(dict) --
AIRecommendationJobArn (string) --
The Amazon Resource Name (ARN) of the created recommendation job.
{'OutputConfig': {'MlflowConfig': {'MlflowExperimentName': 'string',
'MlflowResourceArn': 'string',
'MlflowRunName': 'string'}}}
Returns details of an AI benchmark job, including its status, configuration, target endpoint, and timing information.
See also: AWS API Documentation
Request Syntax
client.describe_ai_benchmark_job(
AIBenchmarkJobName='string'
)
string
[REQUIRED]
The name of the AI benchmark job to describe.
dict
Response Syntax
{
'AIBenchmarkJobName': 'string',
'AIBenchmarkJobArn': 'string',
'AIBenchmarkJobStatus': 'InProgress'|'Completed'|'Failed'|'Stopping'|'Stopped',
'FailureReason': 'string',
'BenchmarkTarget': {
'Endpoint': {
'Identifier': 'string',
'TargetContainerHostname': 'string',
'InferenceComponents': [
{
'Identifier': 'string'
},
]
}
},
'OutputConfig': {
'S3OutputLocation': 'string',
'CloudWatchLogs': [
{
'LogGroupArn': 'string',
'LogStreamName': 'string'
},
],
'MlflowConfig': {
'MlflowResourceArn': 'string',
'MlflowExperimentName': 'string',
'MlflowRunName': 'string'
}
},
'AIWorkloadConfigIdentifier': 'string',
'RoleArn': 'string',
'NetworkConfig': {
'VpcConfig': {
'SecurityGroupIds': [
'string',
],
'Subnets': [
'string',
]
}
},
'CreationTime': datetime(2015, 1, 1),
'StartTime': datetime(2015, 1, 1),
'EndTime': datetime(2015, 1, 1),
'Tags': [
{
'Key': 'string',
'Value': 'string'
},
]
}
Response Structure
(dict) --
AIBenchmarkJobName (string) --
The name of the AI benchmark job.
AIBenchmarkJobArn (string) --
The Amazon Resource Name (ARN) of the AI benchmark job.
AIBenchmarkJobStatus (string) --
The status of the AI benchmark job.
FailureReason (string) --
If the benchmark job failed, the reason it failed.
BenchmarkTarget (dict) --
The target endpoint that was benchmarked.
Endpoint (dict) --
The SageMaker endpoint to benchmark.
Identifier (string) --
The name or Amazon Resource Name (ARN) of the SageMaker endpoint to benchmark.
TargetContainerHostname (string) --
The hostname of the specific container to target within a multi-container endpoint.
InferenceComponents (list) --
The list of inference components to benchmark on the endpoint.
(dict) --
An inference component to benchmark.
Identifier (string) --
The name or Amazon Resource Name (ARN) of the inference component.
OutputConfig (dict) --
The output configuration for the benchmark job, including the Amazon S3 output location and CloudWatch log information.
S3OutputLocation (string) --
The Amazon S3 URI where benchmark results are stored.
CloudWatchLogs (list) --
The CloudWatch log information for the benchmark job.
(dict) --
CloudWatch log information for an AI benchmark or recommendation job.
LogGroupArn (string) --
The Amazon Resource Name (ARN) of the CloudWatch log group.
LogStreamName (string) --
The name of the CloudWatch log stream.
MlflowConfig (dict) --
The MLflow tracking configuration for the job.
MlflowResourceArn (string) --
The Amazon Resource Name (ARN) of the SageMaker managed MLflow resource.
MlflowExperimentName (string) --
The MLflow experiment name used for tracking.
MlflowRunName (string) --
The MLflow run name used for tracking.
AIWorkloadConfigIdentifier (string) --
The name or Amazon Resource Name (ARN) of the AI workload configuration used for this benchmark job.
RoleArn (string) --
The Amazon Resource Name (ARN) of the IAM role used by the benchmark job.
NetworkConfig (dict) --
The network configuration for the benchmark job.
VpcConfig (dict) --
The VPC configuration, including security group IDs and subnet IDs.
SecurityGroupIds (list) --
The VPC security group IDs, in the form sg-xxxxxxxx. Specify the security groups for the VPC that is specified in the Subnets field.
(string) --
Subnets (list) --
The ID of the subnets in the VPC to which you want to connect your training job or model. For information about the availability of specific instance types, see Supported Instance Types and Availability Zones.
(string) --
CreationTime (datetime) --
A timestamp that indicates when the benchmark job was created.
StartTime (datetime) --
A timestamp that indicates when the benchmark job started running.
EndTime (datetime) --
A timestamp that indicates when the benchmark job completed.
Tags (list) --
The tags associated with the benchmark job.
(dict) --
A tag object that consists of a key and an optional value, used to manage metadata for SageMaker Amazon Web Services resources.
You can add tags to notebook instances, training jobs, hyperparameter tuning jobs, batch transform jobs, models, labeling jobs, work teams, endpoint configurations, and endpoints. For more information on adding tags to SageMaker resources, see AddTags.
For more information on adding metadata to your Amazon Web Services resources with tagging, see Tagging Amazon Web Services resources. For advice on best practices for managing Amazon Web Services resources with tagging, see Tagging Best Practices: Implement an Effective Amazon Web Services Resource Tagging Strategy.
Key (string) --
The tag key. Tag keys must be unique per resource.
Value (string) --
The tag value.
{'OutputConfig': {'MlflowConfig': {'MlflowExperimentName': 'string',
'MlflowResourceArn': 'string',
'MlflowRunName': 'string'}}}
Returns details of an AI recommendation job, including its status, model source, performance targets, optimization recommendations, and deployment configurations.
See also: AWS API Documentation
Request Syntax
client.describe_ai_recommendation_job(
AIRecommendationJobName='string'
)
string
[REQUIRED]
The name of the AI recommendation job to describe.
dict
Response Syntax
{
'AIRecommendationJobName': 'string',
'AIRecommendationJobArn': 'string',
'AIRecommendationJobStatus': 'InProgress'|'Completed'|'Failed'|'Stopping'|'Stopped',
'FailureReason': 'string',
'ModelSource': {
'S3': {
'S3Uri': 'string'
}
},
'OutputConfig': {
'S3OutputLocation': 'string',
'ModelPackageGroupIdentifier': 'string',
'MlflowConfig': {
'MlflowResourceArn': 'string',
'MlflowExperimentName': 'string',
'MlflowRunName': 'string'
}
},
'InferenceSpecification': {
'Framework': 'LMI'|'VLLM'
},
'AIWorkloadConfigIdentifier': 'string',
'OptimizeModel': True|False,
'PerformanceTarget': {
'Constraints': [
{
'Metric': 'ttft-ms'|'throughput'|'cost'
},
]
},
'Recommendations': [
{
'RecommendationDescription': 'string',
'OptimizationDetails': [
{
'OptimizationType': 'SpeculativeDecoding'|'KernelTuning',
'OptimizationConfig': {
'string': 'string'
}
},
],
'ModelDetails': {
'ModelPackageArn': 'string',
'InferenceSpecificationName': 'string',
'InstanceDetails': [
{
'InstanceType': 'ml.g5.xlarge'|'ml.g5.2xlarge'|'ml.g5.4xlarge'|'ml.g5.8xlarge'|'ml.g5.12xlarge'|'ml.g5.16xlarge'|'ml.g5.24xlarge'|'ml.g5.48xlarge'|'ml.g6.xlarge'|'ml.g6.2xlarge'|'ml.g6.4xlarge'|'ml.g6.8xlarge'|'ml.g6.12xlarge'|'ml.g6.16xlarge'|'ml.g6.24xlarge'|'ml.g6.48xlarge'|'ml.g6e.xlarge'|'ml.g6e.2xlarge'|'ml.g6e.4xlarge'|'ml.g6e.8xlarge'|'ml.g6e.12xlarge'|'ml.g6e.16xlarge'|'ml.g6e.24xlarge'|'ml.g6e.48xlarge'|'ml.g7e.2xlarge'|'ml.g7e.4xlarge'|'ml.g7e.8xlarge'|'ml.g7e.12xlarge'|'ml.g7e.24xlarge'|'ml.g7e.48xlarge'|'ml.p3.2xlarge'|'ml.p3.8xlarge'|'ml.p3.16xlarge'|'ml.p4d.24xlarge'|'ml.p4de.24xlarge'|'ml.p5.4xlarge'|'ml.p5.48xlarge'|'ml.p5e.48xlarge'|'ml.p5en.48xlarge'|'ml.p6-b200.48xlarge',
'InstanceCount': 123,
'CopyCountPerInstance': 123
},
]
},
'DeploymentConfiguration': {
'S3': [
{
'ChannelName': 'string',
'Uri': 'string'
},
],
'ImageUri': 'string',
'InstanceType': 'ml.g5.xlarge'|'ml.g5.2xlarge'|'ml.g5.4xlarge'|'ml.g5.8xlarge'|'ml.g5.12xlarge'|'ml.g5.16xlarge'|'ml.g5.24xlarge'|'ml.g5.48xlarge'|'ml.g6.xlarge'|'ml.g6.2xlarge'|'ml.g6.4xlarge'|'ml.g6.8xlarge'|'ml.g6.12xlarge'|'ml.g6.16xlarge'|'ml.g6.24xlarge'|'ml.g6.48xlarge'|'ml.g6e.xlarge'|'ml.g6e.2xlarge'|'ml.g6e.4xlarge'|'ml.g6e.8xlarge'|'ml.g6e.12xlarge'|'ml.g6e.16xlarge'|'ml.g6e.24xlarge'|'ml.g6e.48xlarge'|'ml.g7e.2xlarge'|'ml.g7e.4xlarge'|'ml.g7e.8xlarge'|'ml.g7e.12xlarge'|'ml.g7e.24xlarge'|'ml.g7e.48xlarge'|'ml.p3.2xlarge'|'ml.p3.8xlarge'|'ml.p3.16xlarge'|'ml.p4d.24xlarge'|'ml.p4de.24xlarge'|'ml.p5.4xlarge'|'ml.p5.48xlarge'|'ml.p5e.48xlarge'|'ml.p5en.48xlarge'|'ml.p6-b200.48xlarge',
'InstanceCount': 123,
'CopyCountPerInstance': 123,
'EnvironmentVariables': {
'string': 'string'
}
},
'AIBenchmarkJobArn': 'string',
'ExpectedPerformance': [
{
'Metric': 'string',
'Stat': 'string',
'Value': 'string',
'Unit': 'string'
},
]
},
],
'RoleArn': 'string',
'ComputeSpec': {
'InstanceTypes': [
'ml.g5.xlarge'|'ml.g5.2xlarge'|'ml.g5.4xlarge'|'ml.g5.8xlarge'|'ml.g5.12xlarge'|'ml.g5.16xlarge'|'ml.g5.24xlarge'|'ml.g5.48xlarge'|'ml.g6.xlarge'|'ml.g6.2xlarge'|'ml.g6.4xlarge'|'ml.g6.8xlarge'|'ml.g6.12xlarge'|'ml.g6.16xlarge'|'ml.g6.24xlarge'|'ml.g6.48xlarge'|'ml.g6e.xlarge'|'ml.g6e.2xlarge'|'ml.g6e.4xlarge'|'ml.g6e.8xlarge'|'ml.g6e.12xlarge'|'ml.g6e.16xlarge'|'ml.g6e.24xlarge'|'ml.g6e.48xlarge'|'ml.g7e.2xlarge'|'ml.g7e.4xlarge'|'ml.g7e.8xlarge'|'ml.g7e.12xlarge'|'ml.g7e.24xlarge'|'ml.g7e.48xlarge'|'ml.p3.2xlarge'|'ml.p3.8xlarge'|'ml.p3.16xlarge'|'ml.p4d.24xlarge'|'ml.p4de.24xlarge'|'ml.p5.4xlarge'|'ml.p5.48xlarge'|'ml.p5e.48xlarge'|'ml.p5en.48xlarge'|'ml.p6-b200.48xlarge',
],
'CapacityReservationConfig': {
'CapacityReservationPreference': 'capacity-reservations-only',
'MlReservationArns': [
'string',
]
}
},
'CreationTime': datetime(2015, 1, 1),
'StartTime': datetime(2015, 1, 1),
'EndTime': datetime(2015, 1, 1),
'Tags': [
{
'Key': 'string',
'Value': 'string'
},
]
}
Response Structure
(dict) --
AIRecommendationJobName (string) --
The name of the AI recommendation job.
AIRecommendationJobArn (string) --
The Amazon Resource Name (ARN) of the AI recommendation job.
AIRecommendationJobStatus (string) --
The status of the AI recommendation job.
FailureReason (string) --
If the recommendation job failed, the reason it failed.
ModelSource (dict) --
The source of the model that was analyzed.
S3 (dict) --
The Amazon S3 location of the model artifacts.
S3Uri (string) --
The Amazon S3 URI of the model artifacts.
OutputConfig (dict) --
The output configuration for the recommendation job.
S3OutputLocation (string) --
The Amazon S3 URI where the recommendation job writes its output results.
ModelPackageGroupIdentifier (string) --
The name or Amazon Resource Name (ARN) of the model package group where deployment-ready model packages are registered.
MlflowConfig (dict) --
The MLflow tracking configuration for the job.
MlflowResourceArn (string) --
The Amazon Resource Name (ARN) of the SageMaker managed MLflow resource.
MlflowExperimentName (string) --
The MLflow experiment name used for tracking.
MlflowRunName (string) --
The MLflow run name used for tracking.
InferenceSpecification (dict) --
The inference framework configuration.
Framework (string) --
The inference framework. Valid values are LMI and VLLM.
AIWorkloadConfigIdentifier (string) --
The name or Amazon Resource Name (ARN) of the AI workload configuration used for this recommendation job.
OptimizeModel (boolean) --
Whether model optimization techniques were allowed.
PerformanceTarget (dict) --
The performance targets specified for the recommendation job.
Constraints (list) --
An array of performance constraints that define the optimization objectives.
(dict) --
A performance constraint for an AI recommendation job.
Metric (string) --
The performance metric. Valid values are ttft-ms (time to first token in milliseconds), throughput, and cost.
Recommendations (list) --
The list of optimization recommendations generated by the job. Each recommendation includes optimization details, deployment configuration, expected performance metrics, and the associated benchmark job ARN.
(dict) --
An optimization recommendation generated by an AI recommendation job.
RecommendationDescription (string) --
A description of the recommendation.
OptimizationDetails (list) --
The optimization techniques applied in this recommendation.
(dict) --
Details about an optimization technique applied in a recommendation.
OptimizationType (string) --
The type of optimization. Valid values are SpeculativeDecoding and KernelTuning.
OptimizationConfig (dict) --
A map of configuration parameters for the optimization technique.
(string) --
(string) --
ModelDetails (dict) --
Details about the model package associated with this recommendation.
ModelPackageArn (string) --
The Amazon Resource Name (ARN) of the model package.
InferenceSpecificationName (string) --
The name of the inference specification within the model package.
InstanceDetails (list) --
The instance details for this recommendation, including instance type, count, and model copies per instance.
(dict) --
Instance details for a recommendation.
InstanceType (string) --
The recommended instance type.
InstanceCount (integer) --
The recommended number of instances.
CopyCountPerInstance (integer) --
The number of model copies per instance.
DeploymentConfiguration (dict) --
The deployment configuration for this recommendation, including the container image, instance type, instance count, and environment variables.
S3 (list) --
The Amazon S3 data channels for the deployment.
(dict) --
An Amazon S3 data channel for a recommended deployment configuration, containing model artifacts or optimized model outputs.
ChannelName (string) --
A custom name for this Amazon S3 data channel.
Uri (string) --
The Amazon S3 URI of the data for this channel.
ImageUri (string) --
The URI of the container image for the deployment.
InstanceType (string) --
The recommended instance type for the deployment.
InstanceCount (integer) --
The recommended number of instances for the deployment.
CopyCountPerInstance (integer) --
The number of model copies per instance.
EnvironmentVariables (dict) --
The environment variables for the deployment.
(string) --
(string) --
AIBenchmarkJobArn (string) --
The Amazon Resource Name (ARN) of the benchmark job associated with this recommendation.
ExpectedPerformance (list) --
The expected performance metrics for this recommendation.
(dict) --
An expected performance metric for a recommendation.
Metric (string) --
The name of the performance metric.
Stat (string) --
The statistical measure for the metric.
Value (string) --
The value of the metric.
Unit (string) --
The unit of the metric value.
RoleArn (string) --
The Amazon Resource Name (ARN) of the IAM role used by the recommendation job.
ComputeSpec (dict) --
The compute resource specification for the recommendation job.
InstanceTypes (list) --
The list of instance types to consider for recommendations. You can specify up to 3 instance types.
(string) --
CapacityReservationConfig (dict) --
The capacity reservation configuration.
CapacityReservationPreference (string) --
The capacity reservation preference. The only valid value is capacity-reservations-only.
MlReservationArns (list) --
The list of ML reservation ARNs to use.
(string) --
CreationTime (datetime) --
A timestamp that indicates when the recommendation job was created.
StartTime (datetime) --
A timestamp that indicates when the recommendation job started running.
EndTime (datetime) --
A timestamp that indicates when the recommendation job completed.
Tags (list) --
The tags associated with the recommendation job.
(dict) --
A tag object that consists of a key and an optional value, used to manage metadata for SageMaker Amazon Web Services resources.
You can add tags to notebook instances, training jobs, hyperparameter tuning jobs, batch transform jobs, models, labeling jobs, work teams, endpoint configurations, and endpoints. For more information on adding tags to SageMaker resources, see AddTags.
For more information on adding metadata to your Amazon Web Services resources with tagging, see Tagging Amazon Web Services resources. For advice on best practices for managing Amazon Web Services resources with tagging, see Tagging Best Practices: Implement an Effective Amazon Web Services Resource Tagging Strategy.
Key (string) --
The tag key. Tag keys must be unique per resource.
Value (string) --
The tag value.