2020/06/26 - Amazon SageMaker Service - 2 updated api methods
Changes The new 'ModelClientConfig' parameter being added for CreateTransformJob and DescribeTransformJob api actions enable customers to configure model invocation related parameters such as timeout and retry.
{'ModelClientConfig': {'InvocationsMaxRetries': 'integer',
'InvocationsTimeoutInSeconds': 'integer'}}
Starts a transform job. A transform job uses a trained model to get inferences on a dataset and saves these results to an Amazon S3 location that you specify.
To perform batch transformations, you create a transform job and use the data that you have readily available.
In the request body, you provide the following:
TransformJobName - Identifies the transform job. The name must be unique within an AWS Region in an AWS account.
ModelName - Identifies the model to use. ModelName must be the name of an existing Amazon SageMaker model in the same AWS Region and AWS account. For information on creating a model, see CreateModel .
TransformInput - Describes the dataset to be transformed and the Amazon S3 location where it is stored.
TransformOutput - Identifies the Amazon S3 location where you want Amazon SageMaker to save the results from the transform job.
TransformResources - Identifies the ML compute instances for the transform job.
For more information about how batch transformation works, see Batch Transform .
See also: AWS API Documentation
Request Syntax
client.create_transform_job(
TransformJobName='string',
ModelName='string',
MaxConcurrentTransforms=123,
ModelClientConfig={
'InvocationsTimeoutInSeconds': 123,
'InvocationsMaxRetries': 123
},
MaxPayloadInMB=123,
BatchStrategy='MultiRecord'|'SingleRecord',
Environment={
'string': 'string'
},
TransformInput={
'DataSource': {
'S3DataSource': {
'S3DataType': 'ManifestFile'|'S3Prefix'|'AugmentedManifestFile',
'S3Uri': 'string'
}
},
'ContentType': 'string',
'CompressionType': 'None'|'Gzip',
'SplitType': 'None'|'Line'|'RecordIO'|'TFRecord'
},
TransformOutput={
'S3OutputPath': 'string',
'Accept': 'string',
'AssembleWith': 'None'|'Line',
'KmsKeyId': 'string'
},
TransformResources={
'InstanceType': 'ml.m4.xlarge'|'ml.m4.2xlarge'|'ml.m4.4xlarge'|'ml.m4.10xlarge'|'ml.m4.16xlarge'|'ml.c4.xlarge'|'ml.c4.2xlarge'|'ml.c4.4xlarge'|'ml.c4.8xlarge'|'ml.p2.xlarge'|'ml.p2.8xlarge'|'ml.p2.16xlarge'|'ml.p3.2xlarge'|'ml.p3.8xlarge'|'ml.p3.16xlarge'|'ml.c5.xlarge'|'ml.c5.2xlarge'|'ml.c5.4xlarge'|'ml.c5.9xlarge'|'ml.c5.18xlarge'|'ml.m5.large'|'ml.m5.xlarge'|'ml.m5.2xlarge'|'ml.m5.4xlarge'|'ml.m5.12xlarge'|'ml.m5.24xlarge',
'InstanceCount': 123,
'VolumeKmsKeyId': 'string'
},
DataProcessing={
'InputFilter': 'string',
'OutputFilter': 'string',
'JoinSource': 'Input'|'None'
},
Tags=[
{
'Key': 'string',
'Value': 'string'
},
],
ExperimentConfig={
'ExperimentName': 'string',
'TrialName': 'string',
'TrialComponentDisplayName': 'string'
}
)
string
[REQUIRED]
The name of the transform job. The name must be unique within an AWS Region in an AWS account.
string
[REQUIRED]
The name of the model that you want to use for the transform job. ModelName must be the name of an existing Amazon SageMaker model within an AWS Region in an AWS account.
integer
The maximum number of parallel requests that can be sent to each instance in a transform job. If MaxConcurrentTransforms is set to 0 or left unset, Amazon SageMaker checks the optional execution-parameters to determine the settings for your chosen algorithm. If the execution-parameters endpoint is not enabled, the default value is 1 . For more information on execution-parameters, see How Containers Serve Requests . For built-in algorithms, you don't need to set a value for MaxConcurrentTransforms .
dict
Configures the timeout and maximum number of retries for processing a transform job invocation.
InvocationsTimeoutInSeconds (integer) --
The timeout value in seconds for an invocation request.
InvocationsMaxRetries (integer) --
The maximum number of retries when invocation requests are failing.
integer
The maximum allowed size of the payload, in MB. A payload is the data portion of a record (without metadata). The value in MaxPayloadInMB must be greater than, or equal to, the size of a single record. To estimate the size of a record in MB, divide the size of your dataset by the number of records. To ensure that the records fit within the maximum payload size, we recommend using a slightly larger value. The default value is 6 MB.
For cases where the payload might be arbitrarily large and is transmitted using HTTP chunked encoding, set the value to 0 . This feature works only in supported algorithms. Currently, Amazon SageMaker built-in algorithms do not support HTTP chunked encoding.
string
Specifies the number of records to include in a mini-batch for an HTTP inference request. A record is a single unit of input data that inference can be made on. For example, a single line in a CSV file is a record.
To enable the batch strategy, you must set the SplitType property to Line , RecordIO , or TFRecord .
To use only one record when making an HTTP invocation request to a container, set BatchStrategy to SingleRecord and SplitType to Line .
To fit as many records in a mini-batch as can fit within the MaxPayloadInMB limit, set BatchStrategy to MultiRecord and SplitType to Line .
dict
The environment variables to set in the Docker container. We support up to 16 key and values entries in the map.
(string) --
(string) --
dict
[REQUIRED]
Describes the input source and the way the transform job consumes it.
DataSource (dict) -- [REQUIRED]
Describes the location of the channel data, which is, the S3 location of the input data that the model can consume.
S3DataSource (dict) -- [REQUIRED]
The S3 location of the data source that is associated with a channel.
S3DataType (string) -- [REQUIRED]
If you choose S3Prefix , S3Uri identifies a key name prefix. Amazon SageMaker uses all objects with the specified key name prefix for batch transform.
If you choose ManifestFile , S3Uri identifies an object that is a manifest file containing a list of object keys that you want Amazon SageMaker to use for batch transform.
The following values are compatible: ManifestFile , S3Prefix
The following value is not compatible: AugmentedManifestFile
S3Uri (string) -- [REQUIRED]
Depending on the value specified for the S3DataType , identifies either a key name prefix or a manifest. For example:
A key name prefix might look like this: s3://bucketname/exampleprefix .
A manifest might look like this: s3://bucketname/example.manifest The manifest is an S3 object which is a JSON file with the following format: [ {"prefix": "s3://customer_bucket/some/prefix/"}, "relative/path/to/custdata-1", "relative/path/custdata-2", ... "relative/path/custdata-N" ] The preceding JSON matches the following s3Uris : s3://customer_bucket/some/prefix/relative/path/to/custdata-1 s3://customer_bucket/some/prefix/relative/path/custdata-2 ... s3://customer_bucket/some/prefix/relative/path/custdata-N The complete set of S3Uris in this manifest constitutes the input data for the channel for this datasource. The object that each S3Uris points to must be readable by the IAM role that Amazon SageMaker uses to perform tasks on your behalf.
ContentType (string) --
The multipurpose internet mail extension (MIME) type of the data. Amazon SageMaker uses the MIME type with each http call to transfer data to the transform job.
CompressionType (string) --
If your transform data is compressed, specify the compression type. Amazon SageMaker automatically decompresses the data for the transform job accordingly. The default value is None .
SplitType (string) --
The method to use to split the transform job's data files into smaller batches. Splitting is necessary when the total size of each object is too large to fit in a single request. You can also use data splitting to improve performance by processing multiple concurrent mini-batches. The default value for SplitType is None , which indicates that input data files are not split, and request payloads contain the entire contents of an input object. Set the value of this parameter to Line to split records on a newline character boundary. SplitType also supports a number of record-oriented binary data formats.
When splitting is enabled, the size of a mini-batch depends on the values of the BatchStrategy and MaxPayloadInMB parameters. When the value of BatchStrategy is MultiRecord , Amazon SageMaker sends the maximum number of records in each request, up to the MaxPayloadInMB limit. If the value of BatchStrategy is SingleRecord , Amazon SageMaker sends individual records in each request.
Note
Some data formats represent a record as a binary payload wrapped with extra padding bytes. When splitting is applied to a binary data format, padding is removed if the value of BatchStrategy is set to SingleRecord . Padding is not removed if the value of BatchStrategy is set to MultiRecord .
For more information about RecordIO , see Create a Dataset Using RecordIO in the MXNet documentation. For more information about TFRecord , see Consuming TFRecord data in the TensorFlow documentation.
dict
[REQUIRED]
Describes the results of the transform job.
S3OutputPath (string) -- [REQUIRED]
The Amazon S3 path where you want Amazon SageMaker to store the results of the transform job. For example, s3://bucket-name/key-name-prefix .
For every S3 object used as input for the transform job, batch transform stores the transformed data with an .``out`` suffix in a corresponding subfolder in the location in the output prefix. For example, for the input data stored at s3://bucket-name/input-name-prefix/dataset01/data.csv , batch transform stores the transformed data at s3://bucket-name/output-name-prefix/input-name-prefix/data.csv.out . Batch transform doesn't upload partially processed objects. For an input S3 object that contains multiple records, it creates an .``out`` file only if the transform job succeeds on the entire file. When the input contains multiple S3 objects, the batch transform job processes the listed S3 objects and uploads only the output for successfully processed objects. If any object fails in the transform job batch transform marks the job as failed to prompt investigation.
Accept (string) --
The MIME type used to specify the output data. Amazon SageMaker uses the MIME type with each http call to transfer data from the transform job.
AssembleWith (string) --
Defines how to assemble the results of the transform job as a single S3 object. Choose a format that is most convenient to you. To concatenate the results in binary format, specify None . To add a newline character at the end of every transformed record, specify Line .
KmsKeyId (string) --
The AWS Key Management Service (AWS KMS) key that Amazon SageMaker uses to encrypt the model artifacts at rest using Amazon S3 server-side encryption. The KmsKeyId can be any of the following formats:
Key ID: 1234abcd-12ab-34cd-56ef-1234567890ab
Key ARN: arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab
Alias name: alias/ExampleAlias
Alias name ARN: arn:aws:kms:us-west-2:111122223333:alias/ExampleAlias
If you don't provide a KMS key ID, Amazon SageMaker uses the default KMS key for Amazon S3 for your role's account. For more information, see KMS-Managed Encryption Keys in the Amazon Simple Storage Service Developer Guide.
The KMS key policy must grant permission to the IAM role that you specify in your CreateModel request. For more information, see Using Key Policies in AWS KMS in the AWS Key Management Service Developer Guide .
dict
[REQUIRED]
Describes the resources, including ML instance types and ML instance count, to use for the transform job.
InstanceType (string) -- [REQUIRED]
The ML compute instance type for the transform job. If you are using built-in algorithms to transform moderately sized datasets, we recommend using ml.m4.xlarge or ml.m5.large instance types.
InstanceCount (integer) -- [REQUIRED]
The number of ML compute instances to use in the transform job. For distributed transform jobs, specify a value greater than 1. The default value is 1 .
VolumeKmsKeyId (string) --
The AWS Key Management Service (AWS KMS) key that Amazon SageMaker uses to encrypt model data on the storage volume attached to the ML compute instance(s) that run the batch transform job. The VolumeKmsKeyId can be any of the following formats:
Key ID: 1234abcd-12ab-34cd-56ef-1234567890ab
Key ARN: arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab
Alias name: alias/ExampleAlias
Alias name ARN: arn:aws:kms:us-west-2:111122223333:alias/ExampleAlias
dict
The data structure used to specify the data to be used for inference in a batch transform job and to associate the data that is relevant to the prediction results in the output. The input filter provided allows you to exclude input data that is not needed for inference in a batch transform job. The output filter provided allows you to include input data relevant to interpreting the predictions in the output from the job. For more information, see Associate Prediction Results with their Corresponding Input Records .
InputFilter (string) --
A JSONPath expression used to select a portion of the input data to pass to the algorithm. Use the InputFilter parameter to exclude fields, such as an ID column, from the input. If you want Amazon SageMaker to pass the entire input dataset to the algorithm, accept the default value $ .
Examples: "$" , "$[1:]" , "$.features"
OutputFilter (string) --
A JSONPath expression used to select a portion of the joined dataset to save in the output file for a batch transform job. If you want Amazon SageMaker to store the entire input dataset in the output file, leave the default value, $ . If you specify indexes that aren't within the dimension size of the joined dataset, you get an error.
Examples: "$" , "$[0,5:]" , "$['id','SageMakerOutput']"
JoinSource (string) --
Specifies the source of the data to join with the transformed data. The valid values are None and Input . The default value is None , which specifies not to join the input with the transformed data. If you want the batch transform job to join the original input data with the transformed data, set JoinSource to Input .
For JSON or JSONLines objects, such as a JSON array, Amazon SageMaker adds the transformed data to the input JSON object in an attribute called SageMakerOutput . The joined result for JSON must be a key-value pair object. If the input is not a key-value pair object, Amazon SageMaker creates a new JSON file. In the new JSON file, and the input data is stored under the SageMakerInput key and the results are stored in SageMakerOutput .
For CSV files, Amazon SageMaker combines the transformed data with the input data at the end of the input data and stores it in the output file. The joined data has the joined input data followed by the transformed data and the output is a CSV file.
list
(Optional) An array of key-value pairs. For more information, see Using Cost Allocation Tags in the AWS Billing and Cost Management User Guide .
(dict) --
Describes a tag.
Key (string) -- [REQUIRED]
The tag key.
Value (string) -- [REQUIRED]
The tag value.
dict
Configuration for the experiment.
ExperimentName (string) --
The name of the experiment.
TrialName (string) --
The name of the trial.
TrialComponentDisplayName (string) --
Display name for the trial component.
dict
Response Syntax
{
'TransformJobArn': 'string'
}
Response Structure
(dict) --
TransformJobArn (string) --
The Amazon Resource Name (ARN) of the transform job.
{'ModelClientConfig': {'InvocationsMaxRetries': 'integer',
'InvocationsTimeoutInSeconds': 'integer'}}
Returns information about a transform job.
See also: AWS API Documentation
Request Syntax
client.describe_transform_job(
TransformJobName='string'
)
string
[REQUIRED]
The name of the transform job that you want to view details of.
dict
Response Syntax
{
'TransformJobName': 'string',
'TransformJobArn': 'string',
'TransformJobStatus': 'InProgress'|'Completed'|'Failed'|'Stopping'|'Stopped',
'FailureReason': 'string',
'ModelName': 'string',
'MaxConcurrentTransforms': 123,
'ModelClientConfig': {
'InvocationsTimeoutInSeconds': 123,
'InvocationsMaxRetries': 123
},
'MaxPayloadInMB': 123,
'BatchStrategy': 'MultiRecord'|'SingleRecord',
'Environment': {
'string': 'string'
},
'TransformInput': {
'DataSource': {
'S3DataSource': {
'S3DataType': 'ManifestFile'|'S3Prefix'|'AugmentedManifestFile',
'S3Uri': 'string'
}
},
'ContentType': 'string',
'CompressionType': 'None'|'Gzip',
'SplitType': 'None'|'Line'|'RecordIO'|'TFRecord'
},
'TransformOutput': {
'S3OutputPath': 'string',
'Accept': 'string',
'AssembleWith': 'None'|'Line',
'KmsKeyId': 'string'
},
'TransformResources': {
'InstanceType': 'ml.m4.xlarge'|'ml.m4.2xlarge'|'ml.m4.4xlarge'|'ml.m4.10xlarge'|'ml.m4.16xlarge'|'ml.c4.xlarge'|'ml.c4.2xlarge'|'ml.c4.4xlarge'|'ml.c4.8xlarge'|'ml.p2.xlarge'|'ml.p2.8xlarge'|'ml.p2.16xlarge'|'ml.p3.2xlarge'|'ml.p3.8xlarge'|'ml.p3.16xlarge'|'ml.c5.xlarge'|'ml.c5.2xlarge'|'ml.c5.4xlarge'|'ml.c5.9xlarge'|'ml.c5.18xlarge'|'ml.m5.large'|'ml.m5.xlarge'|'ml.m5.2xlarge'|'ml.m5.4xlarge'|'ml.m5.12xlarge'|'ml.m5.24xlarge',
'InstanceCount': 123,
'VolumeKmsKeyId': 'string'
},
'CreationTime': datetime(2015, 1, 1),
'TransformStartTime': datetime(2015, 1, 1),
'TransformEndTime': datetime(2015, 1, 1),
'LabelingJobArn': 'string',
'AutoMLJobArn': 'string',
'DataProcessing': {
'InputFilter': 'string',
'OutputFilter': 'string',
'JoinSource': 'Input'|'None'
},
'ExperimentConfig': {
'ExperimentName': 'string',
'TrialName': 'string',
'TrialComponentDisplayName': 'string'
}
}
Response Structure
(dict) --
TransformJobName (string) --
The name of the transform job.
TransformJobArn (string) --
The Amazon Resource Name (ARN) of the transform job.
TransformJobStatus (string) --
The status of the transform job. If the transform job failed, the reason is returned in the FailureReason field.
FailureReason (string) --
If the transform job failed, FailureReason describes why it failed. A transform job creates a log file, which includes error messages, and stores it as an Amazon S3 object. For more information, see Log Amazon SageMaker Events with Amazon CloudWatch .
ModelName (string) --
The name of the model used in the transform job.
MaxConcurrentTransforms (integer) --
The maximum number of parallel requests on each instance node that can be launched in a transform job. The default value is 1.
ModelClientConfig (dict) --
The timeout and maximum number of retries for processing a transform job invocation.
InvocationsTimeoutInSeconds (integer) --
The timeout value in seconds for an invocation request.
InvocationsMaxRetries (integer) --
The maximum number of retries when invocation requests are failing.
MaxPayloadInMB (integer) --
The maximum payload size, in MB, used in the transform job.
BatchStrategy (string) --
Specifies the number of records to include in a mini-batch for an HTTP inference request. A record is a single unit of input data that inference can be made on. For example, a single line in a CSV file is a record.
To enable the batch strategy, you must set SplitType to Line , RecordIO , or TFRecord .
Environment (dict) --
The environment variables to set in the Docker container. We support up to 16 key and values entries in the map.
(string) --
(string) --
TransformInput (dict) --
Describes the dataset to be transformed and the Amazon S3 location where it is stored.
DataSource (dict) --
Describes the location of the channel data, which is, the S3 location of the input data that the model can consume.
S3DataSource (dict) --
The S3 location of the data source that is associated with a channel.
S3DataType (string) --
If you choose S3Prefix , S3Uri identifies a key name prefix. Amazon SageMaker uses all objects with the specified key name prefix for batch transform.
If you choose ManifestFile , S3Uri identifies an object that is a manifest file containing a list of object keys that you want Amazon SageMaker to use for batch transform.
The following values are compatible: ManifestFile , S3Prefix
The following value is not compatible: AugmentedManifestFile
S3Uri (string) --
Depending on the value specified for the S3DataType , identifies either a key name prefix or a manifest. For example:
A key name prefix might look like this: s3://bucketname/exampleprefix .
A manifest might look like this: s3://bucketname/example.manifest The manifest is an S3 object which is a JSON file with the following format: [ {"prefix": "s3://customer_bucket/some/prefix/"}, "relative/path/to/custdata-1", "relative/path/custdata-2", ... "relative/path/custdata-N" ] The preceding JSON matches the following s3Uris : s3://customer_bucket/some/prefix/relative/path/to/custdata-1 s3://customer_bucket/some/prefix/relative/path/custdata-2 ... s3://customer_bucket/some/prefix/relative/path/custdata-N The complete set of S3Uris in this manifest constitutes the input data for the channel for this datasource. The object that each S3Uris points to must be readable by the IAM role that Amazon SageMaker uses to perform tasks on your behalf.
ContentType (string) --
The multipurpose internet mail extension (MIME) type of the data. Amazon SageMaker uses the MIME type with each http call to transfer data to the transform job.
CompressionType (string) --
If your transform data is compressed, specify the compression type. Amazon SageMaker automatically decompresses the data for the transform job accordingly. The default value is None .
SplitType (string) --
The method to use to split the transform job's data files into smaller batches. Splitting is necessary when the total size of each object is too large to fit in a single request. You can also use data splitting to improve performance by processing multiple concurrent mini-batches. The default value for SplitType is None , which indicates that input data files are not split, and request payloads contain the entire contents of an input object. Set the value of this parameter to Line to split records on a newline character boundary. SplitType also supports a number of record-oriented binary data formats.
When splitting is enabled, the size of a mini-batch depends on the values of the BatchStrategy and MaxPayloadInMB parameters. When the value of BatchStrategy is MultiRecord , Amazon SageMaker sends the maximum number of records in each request, up to the MaxPayloadInMB limit. If the value of BatchStrategy is SingleRecord , Amazon SageMaker sends individual records in each request.
Note
Some data formats represent a record as a binary payload wrapped with extra padding bytes. When splitting is applied to a binary data format, padding is removed if the value of BatchStrategy is set to SingleRecord . Padding is not removed if the value of BatchStrategy is set to MultiRecord .
For more information about RecordIO , see Create a Dataset Using RecordIO in the MXNet documentation. For more information about TFRecord , see Consuming TFRecord data in the TensorFlow documentation.
TransformOutput (dict) --
Identifies the Amazon S3 location where you want Amazon SageMaker to save the results from the transform job.
S3OutputPath (string) --
The Amazon S3 path where you want Amazon SageMaker to store the results of the transform job. For example, s3://bucket-name/key-name-prefix .
For every S3 object used as input for the transform job, batch transform stores the transformed data with an .``out`` suffix in a corresponding subfolder in the location in the output prefix. For example, for the input data stored at s3://bucket-name/input-name-prefix/dataset01/data.csv , batch transform stores the transformed data at s3://bucket-name/output-name-prefix/input-name-prefix/data.csv.out . Batch transform doesn't upload partially processed objects. For an input S3 object that contains multiple records, it creates an .``out`` file only if the transform job succeeds on the entire file. When the input contains multiple S3 objects, the batch transform job processes the listed S3 objects and uploads only the output for successfully processed objects. If any object fails in the transform job batch transform marks the job as failed to prompt investigation.
Accept (string) --
The MIME type used to specify the output data. Amazon SageMaker uses the MIME type with each http call to transfer data from the transform job.
AssembleWith (string) --
Defines how to assemble the results of the transform job as a single S3 object. Choose a format that is most convenient to you. To concatenate the results in binary format, specify None . To add a newline character at the end of every transformed record, specify Line .
KmsKeyId (string) --
The AWS Key Management Service (AWS KMS) key that Amazon SageMaker uses to encrypt the model artifacts at rest using Amazon S3 server-side encryption. The KmsKeyId can be any of the following formats:
Key ID: 1234abcd-12ab-34cd-56ef-1234567890ab
Key ARN: arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab
Alias name: alias/ExampleAlias
Alias name ARN: arn:aws:kms:us-west-2:111122223333:alias/ExampleAlias
If you don't provide a KMS key ID, Amazon SageMaker uses the default KMS key for Amazon S3 for your role's account. For more information, see KMS-Managed Encryption Keys in the Amazon Simple Storage Service Developer Guide.
The KMS key policy must grant permission to the IAM role that you specify in your CreateModel request. For more information, see Using Key Policies in AWS KMS in the AWS Key Management Service Developer Guide .
TransformResources (dict) --
Describes the resources, including ML instance types and ML instance count, to use for the transform job.
InstanceType (string) --
The ML compute instance type for the transform job. If you are using built-in algorithms to transform moderately sized datasets, we recommend using ml.m4.xlarge or ml.m5.large instance types.
InstanceCount (integer) --
The number of ML compute instances to use in the transform job. For distributed transform jobs, specify a value greater than 1. The default value is 1 .
VolumeKmsKeyId (string) --
The AWS Key Management Service (AWS KMS) key that Amazon SageMaker uses to encrypt model data on the storage volume attached to the ML compute instance(s) that run the batch transform job. The VolumeKmsKeyId can be any of the following formats:
Key ID: 1234abcd-12ab-34cd-56ef-1234567890ab
Key ARN: arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab
Alias name: alias/ExampleAlias
Alias name ARN: arn:aws:kms:us-west-2:111122223333:alias/ExampleAlias
CreationTime (datetime) --
A timestamp that shows when the transform Job was created.
TransformStartTime (datetime) --
Indicates when the transform job starts on ML instances. You are billed for the time interval between this time and the value of TransformEndTime .
TransformEndTime (datetime) --
Indicates when the transform job has been completed, or has stopped or failed. You are billed for the time interval between this time and the value of TransformStartTime .
LabelingJobArn (string) --
The Amazon Resource Name (ARN) of the Amazon SageMaker Ground Truth labeling job that created the transform or training job.
AutoMLJobArn (string) --
The Amazon Resource Name (ARN) of the AutoML transform job.
DataProcessing (dict) --
The data structure used to specify the data to be used for inference in a batch transform job and to associate the data that is relevant to the prediction results in the output. The input filter provided allows you to exclude input data that is not needed for inference in a batch transform job. The output filter provided allows you to include input data relevant to interpreting the predictions in the output from the job. For more information, see Associate Prediction Results with their Corresponding Input Records .
InputFilter (string) --
A JSONPath expression used to select a portion of the input data to pass to the algorithm. Use the InputFilter parameter to exclude fields, such as an ID column, from the input. If you want Amazon SageMaker to pass the entire input dataset to the algorithm, accept the default value $ .
Examples: "$" , "$[1:]" , "$.features"
OutputFilter (string) --
A JSONPath expression used to select a portion of the joined dataset to save in the output file for a batch transform job. If you want Amazon SageMaker to store the entire input dataset in the output file, leave the default value, $ . If you specify indexes that aren't within the dimension size of the joined dataset, you get an error.
Examples: "$" , "$[0,5:]" , "$['id','SageMakerOutput']"
JoinSource (string) --
Specifies the source of the data to join with the transformed data. The valid values are None and Input . The default value is None , which specifies not to join the input with the transformed data. If you want the batch transform job to join the original input data with the transformed data, set JoinSource to Input .
For JSON or JSONLines objects, such as a JSON array, Amazon SageMaker adds the transformed data to the input JSON object in an attribute called SageMakerOutput . The joined result for JSON must be a key-value pair object. If the input is not a key-value pair object, Amazon SageMaker creates a new JSON file. In the new JSON file, and the input data is stored under the SageMakerInput key and the results are stored in SageMakerOutput .
For CSV files, Amazon SageMaker combines the transformed data with the input data at the end of the input data and stores it in the output file. The joined data has the joined input data followed by the transformed data and the output is a CSV file.
ExperimentConfig (dict) --
Configuration for the experiment.
ExperimentName (string) --
The name of the experiment.
TrialName (string) --
The name of the trial.
TrialComponentDisplayName (string) --
Display name for the trial component.