2023/02/28 - Amazon Comprehend - 11 new 13 updated api methods
Changes Amazon Comprehend now supports flywheels to help you train and manage new model versions for custom models.
Information about the history of a flywheel iteration. For more information about flywheels, see Flywheel overview in the Amazon Comprehend Developer Guide .
See also: AWS API Documentation
Request Syntax
client.list_flywheel_iteration_history(
FlywheelArn='string',
Filter={
'CreationTimeAfter': datetime(2015, 1, 1),
'CreationTimeBefore': datetime(2015, 1, 1)
},
NextToken='string',
MaxResults=123
)
string
[REQUIRED]
The ARN of the flywheel.
dict
Filter the flywheel iteration history based on creation time.
CreationTimeAfter (datetime) --
Filter the flywheel iterations to include iterations created after the specified time.
CreationTimeBefore (datetime) --
Filter the flywheel iterations to include iterations created before the specified time.
string
Next token
integer
Maximum number of iteration history results to return
dict
Response Syntax
{
'FlywheelIterationPropertiesList': [
{
'FlywheelArn': 'string',
'FlywheelIterationId': 'string',
'CreationTime': datetime(2015, 1, 1),
'EndTime': datetime(2015, 1, 1),
'Status': 'TRAINING'|'EVALUATING'|'COMPLETED'|'FAILED'|'STOP_REQUESTED'|'STOPPED',
'Message': 'string',
'EvaluatedModelArn': 'string',
'EvaluatedModelMetrics': {
'AverageF1Score': 123.0,
'AveragePrecision': 123.0,
'AverageRecall': 123.0,
'AverageAccuracy': 123.0
},
'TrainedModelArn': 'string',
'TrainedModelMetrics': {
'AverageF1Score': 123.0,
'AveragePrecision': 123.0,
'AverageRecall': 123.0,
'AverageAccuracy': 123.0
},
'EvaluationManifestS3Prefix': 'string'
},
],
'NextToken': 'string'
}
Response Structure
(dict) --
FlywheelIterationPropertiesList (list) --
List of flywheel iteration properties
(dict) --
The configuration properties of a flywheel iteration.
FlywheelArn (string) --
FlywheelIterationId (string) --
CreationTime (datetime) --
The creation start time of the flywheel iteration.
EndTime (datetime) --
The completion time of this flywheel iteration.
Status (string) --
The status of the flywheel iteration.
Message (string) --
A description of the status of the flywheel iteration.
EvaluatedModelArn (string) --
The ARN of the evaluated model associated with this flywheel iteration.
EvaluatedModelMetrics (dict) --
The evaluation metrics associated with the evaluated model.
AverageF1Score (float) --
The average F1 score from the evaluation metrics.
AveragePrecision (float) --
Average precision metric for the model.
AverageRecall (float) --
Average recall metric for the model.
AverageAccuracy (float) --
Average accuracy metric for the model.
TrainedModelArn (string) --
The ARN of the trained model associated with this flywheel iteration.
TrainedModelMetrics (dict) --
The metrics associated with the trained model.
AverageF1Score (float) --
The average F1 score from the evaluation metrics.
AveragePrecision (float) --
Average precision metric for the model.
AverageRecall (float) --
Average recall metric for the model.
AverageAccuracy (float) --
Average accuracy metric for the model.
EvaluationManifestS3Prefix (string) --
NextToken (string) --
Next token
Deletes a flywheel. When you delete the flywheel, Amazon Comprehend does not delete the data lake or the model associated with the flywheel.
For more information about flywheels, see Flywheel overview in the Amazon Comprehend Developer Guide .
See also: AWS API Documentation
Request Syntax
client.delete_flywheel(
FlywheelArn='string'
)
string
[REQUIRED]
The Amazon Resource Number (ARN) of the flywheel to delete.
dict
Response Syntax
{}
Response Structure
(dict) --
Start the flywheel iteration.This operation uses any new datasets to train a new model version. For more information about flywheels, see Flywheel overview in the Amazon Comprehend Developer Guide .
See also: AWS API Documentation
Request Syntax
client.start_flywheel_iteration(
FlywheelArn='string',
ClientRequestToken='string'
)
string
[REQUIRED]
The ARN of the flywheel.
string
A unique identifier for the request. If you don't set the client request token, Amazon Comprehend generates one.
dict
Response Syntax
{
'FlywheelArn': 'string',
'FlywheelIterationId': 'string'
}
Response Structure
(dict) --
FlywheelArn (string) --
FlywheelIterationId (string) --
Retrieve the configuration properties of a flywheel iteration. For more information about flywheels, see Flywheel overview in the Amazon Comprehend Developer Guide .
See also: AWS API Documentation
Request Syntax
client.describe_flywheel_iteration(
FlywheelArn='string',
FlywheelIterationId='string'
)
string
[REQUIRED]
string
[REQUIRED]
dict
Response Syntax
{
'FlywheelIterationProperties': {
'FlywheelArn': 'string',
'FlywheelIterationId': 'string',
'CreationTime': datetime(2015, 1, 1),
'EndTime': datetime(2015, 1, 1),
'Status': 'TRAINING'|'EVALUATING'|'COMPLETED'|'FAILED'|'STOP_REQUESTED'|'STOPPED',
'Message': 'string',
'EvaluatedModelArn': 'string',
'EvaluatedModelMetrics': {
'AverageF1Score': 123.0,
'AveragePrecision': 123.0,
'AverageRecall': 123.0,
'AverageAccuracy': 123.0
},
'TrainedModelArn': 'string',
'TrainedModelMetrics': {
'AverageF1Score': 123.0,
'AveragePrecision': 123.0,
'AverageRecall': 123.0,
'AverageAccuracy': 123.0
},
'EvaluationManifestS3Prefix': 'string'
}
}
Response Structure
(dict) --
FlywheelIterationProperties (dict) --
The configuration properties of a flywheel iteration.
FlywheelArn (string) --
FlywheelIterationId (string) --
CreationTime (datetime) --
The creation start time of the flywheel iteration.
EndTime (datetime) --
The completion time of this flywheel iteration.
Status (string) --
The status of the flywheel iteration.
Message (string) --
A description of the status of the flywheel iteration.
EvaluatedModelArn (string) --
The ARN of the evaluated model associated with this flywheel iteration.
EvaluatedModelMetrics (dict) --
The evaluation metrics associated with the evaluated model.
AverageF1Score (float) --
The average F1 score from the evaluation metrics.
AveragePrecision (float) --
Average precision metric for the model.
AverageRecall (float) --
Average recall metric for the model.
AverageAccuracy (float) --
Average accuracy metric for the model.
TrainedModelArn (string) --
The ARN of the trained model associated with this flywheel iteration.
TrainedModelMetrics (dict) --
The metrics associated with the trained model.
AverageF1Score (float) --
The average F1 score from the evaluation metrics.
AveragePrecision (float) --
Average precision metric for the model.
AverageRecall (float) --
Average recall metric for the model.
AverageAccuracy (float) --
Average accuracy metric for the model.
EvaluationManifestS3Prefix (string) --
Returns information about the dataset that you specify. For more information about datasets, see Flywheel overview in the Amazon Comprehend Developer Guide .
See also: AWS API Documentation
Request Syntax
client.describe_dataset(
DatasetArn='string'
)
string
[REQUIRED]
The ARN of the dataset.
dict
Response Syntax
{
'DatasetProperties': {
'DatasetArn': 'string',
'DatasetName': 'string',
'DatasetType': 'TRAIN'|'TEST',
'DatasetS3Uri': 'string',
'Description': 'string',
'Status': 'CREATING'|'COMPLETED'|'FAILED',
'Message': 'string',
'NumberOfDocuments': 123,
'CreationTime': datetime(2015, 1, 1),
'EndTime': datetime(2015, 1, 1)
}
}
Response Structure
(dict) --
DatasetProperties (dict) --
The dataset properties.
DatasetArn (string) --
The ARN of the dataset.
DatasetName (string) --
The name of the dataset.
DatasetType (string) --
The dataset type (training data or test data).
DatasetS3Uri (string) --
The S3 URI where the dataset is stored.
Description (string) --
Description of the dataset.
Status (string) --
The dataset status. While the system creates the dataset, the status is CREATING . When the dataset is ready to use, the status changes to COMPLETED .
Message (string) --
A description of the status of the dataset.
NumberOfDocuments (integer) --
The number of documents in the dataset.
CreationTime (datetime) --
Creation time of the dataset.
EndTime (datetime) --
Time when the data from the dataset becomes available in the data lake.
Update the configuration information for an existing flywheel.
See also: AWS API Documentation
Request Syntax
client.update_flywheel(
FlywheelArn='string',
ActiveModelArn='string',
DataAccessRoleArn='string',
DataSecurityConfig={
'ModelKmsKeyId': 'string',
'VolumeKmsKeyId': 'string',
'VpcConfig': {
'SecurityGroupIds': [
'string',
],
'Subnets': [
'string',
]
}
}
)
string
[REQUIRED]
The Amazon Resource Number (ARN) of the flywheel to update.
string
The Amazon Resource Number (ARN) of the active model version.
string
The Amazon Resource Name (ARN) of the AWS Identity and Access Management (IAM) role that grants Amazon Comprehend permission to access the flywheel data.
dict
Flywheel data security configuration.
ModelKmsKeyId (string) --
ID for the AWS Key Management Service (KMS) key that Amazon Comprehend uses to encrypt trained custom models. The ModelKmsKeyId can be either of the following formats:
KMS Key ID: "1234abcd-12ab-34cd-56ef-1234567890ab"
Amazon Resource Name (ARN) of a KMS Key: "arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab"
VolumeKmsKeyId (string) --
ID for the KMS key that Amazon Comprehend uses to encrypt the volume.
VpcConfig (dict) --
Configuration parameters for an optional private Virtual Private Cloud (VPC) containing the resources you are using for the job. For more information, see Amazon VPC .
SecurityGroupIds (list) -- [REQUIRED]
The ID number for a security group on an instance of your private VPC. Security groups on your VPC function serve as a virtual firewall to control inbound and outbound traffic and provides security for the resources that you’ll be accessing on the VPC. This ID number is preceded by "sg-", for instance: "sg-03b388029b0a285ea". For more information, see Security Groups for your VPC .
(string) --
Subnets (list) -- [REQUIRED]
The ID for each subnet being used in your private VPC. This subnet is a subset of the a range of IPv4 addresses used by the VPC and is specific to a given availability zone in the VPC’s region. This ID number is preceded by "subnet-", for instance: "subnet-04ccf456919e69055". For more information, see VPCs and Subnets .
(string) --
dict
Response Syntax
{
'FlywheelProperties': {
'FlywheelArn': 'string',
'ActiveModelArn': 'string',
'DataAccessRoleArn': 'string',
'TaskConfig': {
'LanguageCode': 'en'|'es'|'fr'|'de'|'it'|'pt'|'ar'|'hi'|'ja'|'ko'|'zh'|'zh-TW',
'DocumentClassificationConfig': {
'Mode': 'MULTI_CLASS'|'MULTI_LABEL',
'Labels': [
'string',
]
},
'EntityRecognitionConfig': {
'EntityTypes': [
{
'Type': 'string'
},
]
}
},
'DataLakeS3Uri': 'string',
'DataSecurityConfig': {
'ModelKmsKeyId': 'string',
'VolumeKmsKeyId': 'string',
'DataLakeKmsKeyId': 'string',
'VpcConfig': {
'SecurityGroupIds': [
'string',
],
'Subnets': [
'string',
]
}
},
'Status': 'CREATING'|'ACTIVE'|'UPDATING'|'DELETING'|'FAILED',
'ModelType': 'DOCUMENT_CLASSIFIER'|'ENTITY_RECOGNIZER',
'Message': 'string',
'CreationTime': datetime(2015, 1, 1),
'LastModifiedTime': datetime(2015, 1, 1),
'LatestFlywheelIteration': 'string'
}
}
Response Structure
(dict) --
FlywheelProperties (dict) --
The flywheel properties.
FlywheelArn (string) --
The Amazon Resource Number (ARN) of the flywheel.
ActiveModelArn (string) --
The Amazon Resource Number (ARN) of the active model version.
DataAccessRoleArn (string) --
The Amazon Resource Name (ARN) of the AWS Identity and Access Management (IAM) role that grants Amazon Comprehend permission to access the flywheel data.
TaskConfig (dict) --
Configuration about the custom classifier associated with the flywheel.
LanguageCode (string) --
Language code for the language that the model supports.
DocumentClassificationConfig (dict) --
Configuration required for a classification model.
Mode (string) --
Classification mode indicates whether the documents are MULTI_CLASS or MULTI_LABEL .
Labels (list) --
One or more labels to associate with the custom classifier.
(string) --
EntityRecognitionConfig (dict) --
Configuration required for an entity recognition model.
EntityTypes (list) --
Up to 25 entity types that the model is trained to recognize.
(dict) --
An entity type within a labeled training dataset that Amazon Comprehend uses to train a custom entity recognizer.
Type (string) --
An entity type within a labeled training dataset that Amazon Comprehend uses to train a custom entity recognizer.
Entity types must not contain the following invalid characters: n (line break), \n (escaped line break, r (carriage return), \r (escaped carriage return), t (tab), \t (escaped tab), space, and , (comma).
DataLakeS3Uri (string) --
Amazon S3 URI of the data lake location.
DataSecurityConfig (dict) --
Data security configuration.
ModelKmsKeyId (string) --
ID for the AWS Key Management Service (KMS) key that Amazon Comprehend uses to encrypt trained custom models. The ModelKmsKeyId can be either of the following formats:
KMS Key ID: "1234abcd-12ab-34cd-56ef-1234567890ab"
Amazon Resource Name (ARN) of a KMS Key: "arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab"
VolumeKmsKeyId (string) --
ID for the KMS key that Amazon Comprehend uses to encrypt the volume.
DataLakeKmsKeyId (string) --
ID for the KMS key that Amazon Comprehend uses to encrypt the data in the data lake.
VpcConfig (dict) --
Configuration parameters for an optional private Virtual Private Cloud (VPC) containing the resources you are using for the job. For more information, see Amazon VPC .
SecurityGroupIds (list) --
The ID number for a security group on an instance of your private VPC. Security groups on your VPC function serve as a virtual firewall to control inbound and outbound traffic and provides security for the resources that you’ll be accessing on the VPC. This ID number is preceded by "sg-", for instance: "sg-03b388029b0a285ea". For more information, see Security Groups for your VPC .
(string) --
Subnets (list) --
The ID for each subnet being used in your private VPC. This subnet is a subset of the a range of IPv4 addresses used by the VPC and is specific to a given availability zone in the VPC’s region. This ID number is preceded by "subnet-", for instance: "subnet-04ccf456919e69055". For more information, see VPCs and Subnets .
(string) --
Status (string) --
The status of the flywheel.
ModelType (string) --
Model type of the flywheel's model.
Message (string) --
A description of the status of the flywheel.
CreationTime (datetime) --
Creation time of the flywheel.
LastModifiedTime (datetime) --
Last modified time for the flywheel.
LatestFlywheelIteration (string) --
The most recent flywheel iteration.
A flywheel is an AWS resource that orchestrates the ongoing training of a model for custom classification or custom entity recognition. You can create a flywheel to start with an existing trained model, or Comprehend can create and train a new model.
When you create the flywheel, Comprehend creates a data lake in your account. The data lake holds the training data and test data for all versions of the model.
To use a flywheel with an existing trained model, you specify the active model version. Comprehend copies the model's training data and test data into the flywheel's data lake.
To use the flywheel with a new model, you need to provide a dataset for training data (and optional test data) when you create the flywheel.
For more information about flywheels, see Flywheel overview in the Amazon Comprehend Developer Guide .
See also: AWS API Documentation
Request Syntax
client.create_flywheel(
FlywheelName='string',
ActiveModelArn='string',
DataAccessRoleArn='string',
TaskConfig={
'LanguageCode': 'en'|'es'|'fr'|'de'|'it'|'pt'|'ar'|'hi'|'ja'|'ko'|'zh'|'zh-TW',
'DocumentClassificationConfig': {
'Mode': 'MULTI_CLASS'|'MULTI_LABEL',
'Labels': [
'string',
]
},
'EntityRecognitionConfig': {
'EntityTypes': [
{
'Type': 'string'
},
]
}
},
ModelType='DOCUMENT_CLASSIFIER'|'ENTITY_RECOGNIZER',
DataLakeS3Uri='string',
DataSecurityConfig={
'ModelKmsKeyId': 'string',
'VolumeKmsKeyId': 'string',
'DataLakeKmsKeyId': 'string',
'VpcConfig': {
'SecurityGroupIds': [
'string',
],
'Subnets': [
'string',
]
}
},
ClientRequestToken='string',
Tags=[
{
'Key': 'string',
'Value': 'string'
},
]
)
string
[REQUIRED]
Name for the flywheel.
string
To associate an existing model with the flywheel, specify the Amazon Resource Number (ARN) of the model version.
string
[REQUIRED]
The Amazon Resource Name (ARN) of the AWS Identity and Access Management (IAM) role that grants Amazon Comprehend the permissions required to access the flywheel data in the data lake.
dict
Configuration about the custom classifier associated with the flywheel.
LanguageCode (string) -- [REQUIRED]
Language code for the language that the model supports.
DocumentClassificationConfig (dict) --
Configuration required for a classification model.
Mode (string) -- [REQUIRED]
Classification mode indicates whether the documents are MULTI_CLASS or MULTI_LABEL .
Labels (list) --
One or more labels to associate with the custom classifier.
(string) --
EntityRecognitionConfig (dict) --
Configuration required for an entity recognition model.
EntityTypes (list) -- [REQUIRED]
Up to 25 entity types that the model is trained to recognize.
(dict) --
An entity type within a labeled training dataset that Amazon Comprehend uses to train a custom entity recognizer.
Type (string) -- [REQUIRED]
An entity type within a labeled training dataset that Amazon Comprehend uses to train a custom entity recognizer.
Entity types must not contain the following invalid characters: n (line break), \n (escaped line break, r (carriage return), \r (escaped carriage return), t (tab), \t (escaped tab), space, and , (comma).
string
The model type.
string
[REQUIRED]
Enter the S3 location for the data lake. You can specify a new S3 bucket or a new folder of an existing S3 bucket. The flywheel creates the data lake at this location.
dict
Data security configurations.
ModelKmsKeyId (string) --
ID for the AWS Key Management Service (KMS) key that Amazon Comprehend uses to encrypt trained custom models. The ModelKmsKeyId can be either of the following formats:
KMS Key ID: "1234abcd-12ab-34cd-56ef-1234567890ab"
Amazon Resource Name (ARN) of a KMS Key: "arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab"
VolumeKmsKeyId (string) --
ID for the KMS key that Amazon Comprehend uses to encrypt the volume.
DataLakeKmsKeyId (string) --
ID for the KMS key that Amazon Comprehend uses to encrypt the data in the data lake.
VpcConfig (dict) --
Configuration parameters for an optional private Virtual Private Cloud (VPC) containing the resources you are using for the job. For more information, see Amazon VPC .
SecurityGroupIds (list) -- [REQUIRED]
The ID number for a security group on an instance of your private VPC. Security groups on your VPC function serve as a virtual firewall to control inbound and outbound traffic and provides security for the resources that you’ll be accessing on the VPC. This ID number is preceded by "sg-", for instance: "sg-03b388029b0a285ea". For more information, see Security Groups for your VPC .
(string) --
Subnets (list) -- [REQUIRED]
The ID for each subnet being used in your private VPC. This subnet is a subset of the a range of IPv4 addresses used by the VPC and is specific to a given availability zone in the VPC’s region. This ID number is preceded by "subnet-", for instance: "subnet-04ccf456919e69055". For more information, see VPCs and Subnets .
(string) --
string
A unique identifier for the request. If you don't set the client request token, Amazon Comprehend generates one.
This field is autopopulated if not provided.
list
The tags to associate with this flywheel.
(dict) --
A key-value pair that adds as a metadata to a resource used by Amazon Comprehend. For example, a tag with the key-value pair ‘Department’:’Sales’ might be added to a resource to indicate its use by a particular department.
Key (string) -- [REQUIRED]
The initial part of a key-value pair that forms a tag associated with a given resource. For instance, if you want to show which resources are used by which departments, you might use “Department” as the key portion of the pair, with multiple possible values such as “sales,” “legal,” and “administration.”
Value (string) --
The second part of a key-value pair that forms a tag associated with a given resource. For instance, if you want to show which resources are used by which departments, you might use “Department” as the initial (key) portion of the pair, with a value of “sales” to indicate the sales department.
dict
Response Syntax
{
'FlywheelArn': 'string',
'ActiveModelArn': 'string'
}
Response Structure
(dict) --
FlywheelArn (string) --
The Amazon Resource Number (ARN) of the flywheel.
ActiveModelArn (string) --
The Amazon Resource Number (ARN) of the active model version.
Creates a dataset to upload training or test data for a model associated with a flywheel. For more information about datasets, see Flywheel overview in the Amazon Comprehend Developer Guide .
See also: AWS API Documentation
Request Syntax
client.create_dataset(
FlywheelArn='string',
DatasetName='string',
DatasetType='TRAIN'|'TEST',
Description='string',
InputDataConfig={
'AugmentedManifests': [
{
'AttributeNames': [
'string',
],
'S3Uri': 'string',
'AnnotationDataS3Uri': 'string',
'SourceDocumentsS3Uri': 'string',
'DocumentType': 'PLAIN_TEXT_DOCUMENT'|'SEMI_STRUCTURED_DOCUMENT'
},
],
'DataFormat': 'COMPREHEND_CSV'|'AUGMENTED_MANIFEST',
'DocumentClassifierInputDataConfig': {
'S3Uri': 'string',
'LabelDelimiter': 'string'
},
'EntityRecognizerInputDataConfig': {
'Annotations': {
'S3Uri': 'string'
},
'Documents': {
'S3Uri': 'string',
'InputFormat': 'ONE_DOC_PER_FILE'|'ONE_DOC_PER_LINE'
},
'EntityList': {
'S3Uri': 'string'
}
}
},
ClientRequestToken='string',
Tags=[
{
'Key': 'string',
'Value': 'string'
},
]
)
string
[REQUIRED]
The Amazon Resource Number (ARN) of the flywheel of the flywheel to receive the data.
string
[REQUIRED]
Name of the dataset.
string
The dataset type. You can specify that the data in a dataset is for training the model or for testing the model.
string
Description of the dataset.
dict
[REQUIRED]
Information about the input data configuration. The type of input data varies based on the format of the input and whether the data is for a classifier model or an entity recognition model.
AugmentedManifests (list) --
A list of augmented manifest files that provide training data for your custom model. An augmented manifest file is a labeled dataset that is produced by Amazon SageMaker Ground Truth.
(dict) --
An augmented manifest file that provides training data for your custom model. An augmented manifest file is a labeled dataset that is produced by Amazon SageMaker Ground Truth.
AttributeNames (list) -- [REQUIRED]
The JSON attribute that contains the annotations for your training documents. The number of attribute names that you specify depends on whether your augmented manifest file is the output of a single labeling job or a chained labeling job.
If your file is the output of a single labeling job, specify the LabelAttributeName key that was used when the job was created in Ground Truth.
If your file is the output of a chained labeling job, specify the LabelAttributeName key for one or more jobs in the chain. Each LabelAttributeName key provides the annotations from an individual job.
(string) --
S3Uri (string) -- [REQUIRED]
The Amazon S3 location of the augmented manifest file.
AnnotationDataS3Uri (string) --
The S3 prefix to the annotation files that are referred in the augmented manifest file.
SourceDocumentsS3Uri (string) --
The S3 prefix to the source files (PDFs) that are referred to in the augmented manifest file.
DocumentType (string) --
The type of augmented manifest. If you don't specify, the default is PlainTextDocument.
PLAIN_TEXT_DOCUMENT A document type that represents any unicode text that is encoded in UTF-8.
DataFormat (string) --
COMPREHEND_CSV : The data format is a two-column CSV file, where the first column contains labels and the second column contains documents.
AUGMENTED_MANIFEST : The data format
DocumentClassifierInputDataConfig (dict) --
The input properties for training a document classifier model.
For more information on how the input file is formatted, see Preparing training data in the Comprehend Developer Guide.
S3Uri (string) -- [REQUIRED]
The Amazon S3 URI for the input data. The S3 bucket must be in the same region as the API endpoint that you are calling. The URI can point to a single input file or it can provide the prefix for a collection of input files.
For example, if you use the URI S3://bucketName/prefix , if the prefix is a single file, Amazon Comprehend uses that file as input. If more than one file begins with the prefix, Amazon Comprehend uses all of them as input.
This parameter is required if you set DataFormat to COMPREHEND_CSV .
LabelDelimiter (string) --
Indicates the delimiter used to separate each label for training a multi-label classifier. The default delimiter between labels is a pipe (|). You can use a different character as a delimiter (if it's an allowed character) by specifying it under Delimiter for labels. If the training documents use a delimiter other than the default or the delimiter you specify, the labels on that line will be combined to make a single unique label, such as LABELLABELLABEL.
EntityRecognizerInputDataConfig (dict) --
The input properties for training an entity recognizer model.
Annotations (dict) --
The S3 location of the annotation documents for your custom entity recognizer.
S3Uri (string) -- [REQUIRED]
Specifies the Amazon S3 location where the training documents for an entity recognizer are located. The URI must be in the same region as the API endpoint that you are calling.
Documents (dict) -- [REQUIRED]
The format and location of the training documents for your custom entity recognizer.
S3Uri (string) -- [REQUIRED]
Specifies the Amazon S3 location where the documents for the dataset are located.
InputFormat (string) --
Specifies how the text in an input file should be processed. This is optional, and the default is ONE_DOC_PER_LINE. ONE_DOC_PER_FILE - Each file is considered a separate document. Use this option when you are processing large documents, such as newspaper articles or scientific papers. ONE_DOC_PER_LINE - Each line in a file is considered a separate document. Use this option when you are processing many short documents, such as text messages.
EntityList (dict) --
The S3 location of the entity list for your custom entity recognizer.
S3Uri (string) -- [REQUIRED]
Specifies the Amazon S3 location where the entity list is located.
string
A unique identifier for the request. If you don't set the client request token, Amazon Comprehend generates one.
This field is autopopulated if not provided.
list
Tags for the dataset.
(dict) --
A key-value pair that adds as a metadata to a resource used by Amazon Comprehend. For example, a tag with the key-value pair ‘Department’:’Sales’ might be added to a resource to indicate its use by a particular department.
Key (string) -- [REQUIRED]
The initial part of a key-value pair that forms a tag associated with a given resource. For instance, if you want to show which resources are used by which departments, you might use “Department” as the key portion of the pair, with multiple possible values such as “sales,” “legal,” and “administration.”
Value (string) --
The second part of a key-value pair that forms a tag associated with a given resource. For instance, if you want to show which resources are used by which departments, you might use “Department” as the initial (key) portion of the pair, with a value of “sales” to indicate the sales department.
dict
Response Syntax
{
'DatasetArn': 'string'
}
Response Structure
(dict) --
DatasetArn (string) --
The ARN of the dataset.
Gets a list of the flywheels that you have created.
See also: AWS API Documentation
Request Syntax
client.list_flywheels(
Filter={
'Status': 'CREATING'|'ACTIVE'|'UPDATING'|'DELETING'|'FAILED',
'CreationTimeAfter': datetime(2015, 1, 1),
'CreationTimeBefore': datetime(2015, 1, 1)
},
NextToken='string',
MaxResults=123
)
dict
Filters the flywheels that are returned. You can filter flywheels on their status, or the date and time that they were submitted. You can only set one filter at a time.
Status (string) --
Filter the flywheels based on the flywheel status.
CreationTimeAfter (datetime) --
Filter the flywheels to include flywheels created after the specified time.
CreationTimeBefore (datetime) --
Filter the flywheels to include flywheels created before the specified time.
string
Identifies the next page of results to return.
integer
Maximum number of results to return in a response. The default is 100.
dict
Response Syntax
{
'FlywheelSummaryList': [
{
'FlywheelArn': 'string',
'ActiveModelArn': 'string',
'DataLakeS3Uri': 'string',
'Status': 'CREATING'|'ACTIVE'|'UPDATING'|'DELETING'|'FAILED',
'ModelType': 'DOCUMENT_CLASSIFIER'|'ENTITY_RECOGNIZER',
'Message': 'string',
'CreationTime': datetime(2015, 1, 1),
'LastModifiedTime': datetime(2015, 1, 1),
'LatestFlywheelIteration': 'string'
},
],
'NextToken': 'string'
}
Response Structure
(dict) --
FlywheelSummaryList (list) --
A list of flywheel properties retrieved by the service in response to the request.
(dict) --
Flywheel summary information.
FlywheelArn (string) --
The Amazon Resource Number (ARN) of the flywheel
ActiveModelArn (string) --
ARN of the active model version for the flywheel.
DataLakeS3Uri (string) --
Amazon S3 URI of the data lake location.
Status (string) --
The status of the flywheel.
ModelType (string) --
Model type of the flywheel's model.
Message (string) --
A description of the status of the flywheel.
CreationTime (datetime) --
Creation time of the flywheel.
LastModifiedTime (datetime) --
Last modified time for the flywheel.
LatestFlywheelIteration (string) --
The most recent flywheel iteration.
NextToken (string) --
Identifies the next page of results to return.
List the datasets that you have configured in this region. For more information about datasets, see Flywheel overview in the Amazon Comprehend Developer Guide .
See also: AWS API Documentation
Request Syntax
client.list_datasets(
FlywheelArn='string',
Filter={
'Status': 'CREATING'|'COMPLETED'|'FAILED',
'DatasetType': 'TRAIN'|'TEST',
'CreationTimeAfter': datetime(2015, 1, 1),
'CreationTimeBefore': datetime(2015, 1, 1)
},
NextToken='string',
MaxResults=123
)
string
The Amazon Resource Number (ARN) of the flywheel.
dict
Filters the datasets to be returned in the response.
Status (string) --
Filter the datasets based on the dataset status.
DatasetType (string) --
Filter the datasets based on the dataset type.
CreationTimeAfter (datetime) --
Filter the datasets to include datasets created after the specified time.
CreationTimeBefore (datetime) --
Filter the datasets to include datasets created before the specified time.
string
Identifies the next page of results to return.
integer
Maximum number of results to return in a response. The default is 100.
dict
Response Syntax
{
'DatasetPropertiesList': [
{
'DatasetArn': 'string',
'DatasetName': 'string',
'DatasetType': 'TRAIN'|'TEST',
'DatasetS3Uri': 'string',
'Description': 'string',
'Status': 'CREATING'|'COMPLETED'|'FAILED',
'Message': 'string',
'NumberOfDocuments': 123,
'CreationTime': datetime(2015, 1, 1),
'EndTime': datetime(2015, 1, 1)
},
],
'NextToken': 'string'
}
Response Structure
(dict) --
DatasetPropertiesList (list) --
The dataset properties list.
(dict) --
Properties associated with the dataset.
DatasetArn (string) --
The ARN of the dataset.
DatasetName (string) --
The name of the dataset.
DatasetType (string) --
The dataset type (training data or test data).
DatasetS3Uri (string) --
The S3 URI where the dataset is stored.
Description (string) --
Description of the dataset.
Status (string) --
The dataset status. While the system creates the dataset, the status is CREATING . When the dataset is ready to use, the status changes to COMPLETED .
Message (string) --
A description of the status of the dataset.
NumberOfDocuments (integer) --
The number of documents in the dataset.
CreationTime (datetime) --
Creation time of the dataset.
EndTime (datetime) --
Time when the data from the dataset becomes available in the data lake.
NextToken (string) --
Identifies the next page of results to return.
Provides configuration information about the flywheel. For more information about flywheels, see Flywheel overview in the Amazon Comprehend Developer Guide .
See also: AWS API Documentation
Request Syntax
client.describe_flywheel(
FlywheelArn='string'
)
string
[REQUIRED]
The Amazon Resource Number (ARN) of the flywheel.
dict
Response Syntax
{
'FlywheelProperties': {
'FlywheelArn': 'string',
'ActiveModelArn': 'string',
'DataAccessRoleArn': 'string',
'TaskConfig': {
'LanguageCode': 'en'|'es'|'fr'|'de'|'it'|'pt'|'ar'|'hi'|'ja'|'ko'|'zh'|'zh-TW',
'DocumentClassificationConfig': {
'Mode': 'MULTI_CLASS'|'MULTI_LABEL',
'Labels': [
'string',
]
},
'EntityRecognitionConfig': {
'EntityTypes': [
{
'Type': 'string'
},
]
}
},
'DataLakeS3Uri': 'string',
'DataSecurityConfig': {
'ModelKmsKeyId': 'string',
'VolumeKmsKeyId': 'string',
'DataLakeKmsKeyId': 'string',
'VpcConfig': {
'SecurityGroupIds': [
'string',
],
'Subnets': [
'string',
]
}
},
'Status': 'CREATING'|'ACTIVE'|'UPDATING'|'DELETING'|'FAILED',
'ModelType': 'DOCUMENT_CLASSIFIER'|'ENTITY_RECOGNIZER',
'Message': 'string',
'CreationTime': datetime(2015, 1, 1),
'LastModifiedTime': datetime(2015, 1, 1),
'LatestFlywheelIteration': 'string'
}
}
Response Structure
(dict) --
FlywheelProperties (dict) --
The flywheel properties.
FlywheelArn (string) --
The Amazon Resource Number (ARN) of the flywheel.
ActiveModelArn (string) --
The Amazon Resource Number (ARN) of the active model version.
DataAccessRoleArn (string) --
The Amazon Resource Name (ARN) of the AWS Identity and Access Management (IAM) role that grants Amazon Comprehend permission to access the flywheel data.
TaskConfig (dict) --
Configuration about the custom classifier associated with the flywheel.
LanguageCode (string) --
Language code for the language that the model supports.
DocumentClassificationConfig (dict) --
Configuration required for a classification model.
Mode (string) --
Classification mode indicates whether the documents are MULTI_CLASS or MULTI_LABEL .
Labels (list) --
One or more labels to associate with the custom classifier.
(string) --
EntityRecognitionConfig (dict) --
Configuration required for an entity recognition model.
EntityTypes (list) --
Up to 25 entity types that the model is trained to recognize.
(dict) --
An entity type within a labeled training dataset that Amazon Comprehend uses to train a custom entity recognizer.
Type (string) --
An entity type within a labeled training dataset that Amazon Comprehend uses to train a custom entity recognizer.
Entity types must not contain the following invalid characters: n (line break), \n (escaped line break, r (carriage return), \r (escaped carriage return), t (tab), \t (escaped tab), space, and , (comma).
DataLakeS3Uri (string) --
Amazon S3 URI of the data lake location.
DataSecurityConfig (dict) --
Data security configuration.
ModelKmsKeyId (string) --
ID for the AWS Key Management Service (KMS) key that Amazon Comprehend uses to encrypt trained custom models. The ModelKmsKeyId can be either of the following formats:
KMS Key ID: "1234abcd-12ab-34cd-56ef-1234567890ab"
Amazon Resource Name (ARN) of a KMS Key: "arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab"
VolumeKmsKeyId (string) --
ID for the KMS key that Amazon Comprehend uses to encrypt the volume.
DataLakeKmsKeyId (string) --
ID for the KMS key that Amazon Comprehend uses to encrypt the data in the data lake.
VpcConfig (dict) --
Configuration parameters for an optional private Virtual Private Cloud (VPC) containing the resources you are using for the job. For more information, see Amazon VPC .
SecurityGroupIds (list) --
The ID number for a security group on an instance of your private VPC. Security groups on your VPC function serve as a virtual firewall to control inbound and outbound traffic and provides security for the resources that you’ll be accessing on the VPC. This ID number is preceded by "sg-", for instance: "sg-03b388029b0a285ea". For more information, see Security Groups for your VPC .
(string) --
Subnets (list) --
The ID for each subnet being used in your private VPC. This subnet is a subset of the a range of IPv4 addresses used by the VPC and is specific to a given availability zone in the VPC’s region. This ID number is preceded by "subnet-", for instance: "subnet-04ccf456919e69055". For more information, see VPCs and Subnets .
(string) --
Status (string) --
The status of the flywheel.
ModelType (string) --
Model type of the flywheel's model.
Message (string) --
A description of the status of the flywheel.
CreationTime (datetime) --
Creation time of the flywheel.
LastModifiedTime (datetime) --
Last modified time for the flywheel.
LatestFlywheelIteration (string) --
The most recent flywheel iteration.
{'OutputDataConfig': {'FlywheelStatsS3Prefix': 'string'}}
Creates a new document classifier that you can use to categorize documents. To create a classifier, you provide a set of training documents that labeled with the categories that you want to use. After the classifier is trained you can use it to categorize a set of labeled documents into the categories. For more information, see Document Classification in the Comprehend Developer Guide.
See also: AWS API Documentation
Request Syntax
client.create_document_classifier(
DocumentClassifierName='string',
VersionName='string',
DataAccessRoleArn='string',
Tags=[
{
'Key': 'string',
'Value': 'string'
},
],
InputDataConfig={
'DataFormat': 'COMPREHEND_CSV'|'AUGMENTED_MANIFEST',
'S3Uri': 'string',
'TestS3Uri': 'string',
'LabelDelimiter': 'string',
'AugmentedManifests': [
{
'S3Uri': 'string',
'Split': 'TRAIN'|'TEST',
'AttributeNames': [
'string',
],
'AnnotationDataS3Uri': 'string',
'SourceDocumentsS3Uri': 'string',
'DocumentType': 'PLAIN_TEXT_DOCUMENT'|'SEMI_STRUCTURED_DOCUMENT'
},
]
},
OutputDataConfig={
'S3Uri': 'string',
'KmsKeyId': 'string',
'FlywheelStatsS3Prefix': 'string'
},
ClientRequestToken='string',
LanguageCode='en'|'es'|'fr'|'de'|'it'|'pt'|'ar'|'hi'|'ja'|'ko'|'zh'|'zh-TW',
VolumeKmsKeyId='string',
VpcConfig={
'SecurityGroupIds': [
'string',
],
'Subnets': [
'string',
]
},
Mode='MULTI_CLASS'|'MULTI_LABEL',
ModelKmsKeyId='string',
ModelPolicy='string'
)
string
[REQUIRED]
The name of the document classifier.
string
The version name given to the newly created classifier. Version names can have a maximum of 256 characters. Alphanumeric characters, hyphens (-) and underscores (_) are allowed. The version name must be unique among all models with the same classifier name in the account/AWS Region.
string
[REQUIRED]
The Amazon Resource Name (ARN) of the AWS Identity and Access Management (IAM) role that grants Amazon Comprehend read access to your input data.
list
Tags to associate with the document classifier. A tag is a key-value pair that adds as a metadata to a resource used by Amazon Comprehend. For example, a tag with "Sales" as the key might be added to a resource to indicate its use by the sales department.
(dict) --
A key-value pair that adds as a metadata to a resource used by Amazon Comprehend. For example, a tag with the key-value pair ‘Department’:’Sales’ might be added to a resource to indicate its use by a particular department.
Key (string) -- [REQUIRED]
The initial part of a key-value pair that forms a tag associated with a given resource. For instance, if you want to show which resources are used by which departments, you might use “Department” as the key portion of the pair, with multiple possible values such as “sales,” “legal,” and “administration.”
Value (string) --
The second part of a key-value pair that forms a tag associated with a given resource. For instance, if you want to show which resources are used by which departments, you might use “Department” as the initial (key) portion of the pair, with a value of “sales” to indicate the sales department.
dict
[REQUIRED]
Specifies the format and location of the input data for the job.
DataFormat (string) --
The format of your training data:
COMPREHEND_CSV : A two-column CSV file, where labels are provided in the first column, and documents are provided in the second. If you use this value, you must provide the S3Uri parameter in your request.
AUGMENTED_MANIFEST : A labeled dataset that is produced by Amazon SageMaker Ground Truth. This file is in JSON lines format. Each line is a complete JSON object that contains a training document and its associated labels. If you use this value, you must provide the AugmentedManifests parameter in your request.
If you don't specify a value, Amazon Comprehend uses COMPREHEND_CSV as the default.
S3Uri (string) --
The Amazon S3 URI for the input data. The S3 bucket must be in the same region as the API endpoint that you are calling. The URI can point to a single input file or it can provide the prefix for a collection of input files.
For example, if you use the URI S3://bucketName/prefix , if the prefix is a single file, Amazon Comprehend uses that file as input. If more than one file begins with the prefix, Amazon Comprehend uses all of them as input.
This parameter is required if you set DataFormat to COMPREHEND_CSV .
TestS3Uri (string) --
This specifies the Amazon S3 location where the test annotations for an entity recognizer are located. The URI must be in the same AWS Region as the API endpoint that you are calling.
LabelDelimiter (string) --
Indicates the delimiter used to separate each label for training a multi-label classifier. The default delimiter between labels is a pipe (|). You can use a different character as a delimiter (if it's an allowed character) by specifying it under Delimiter for labels. If the training documents use a delimiter other than the default or the delimiter you specify, the labels on that line will be combined to make a single unique label, such as LABELLABELLABEL.
AugmentedManifests (list) --
A list of augmented manifest files that provide training data for your custom model. An augmented manifest file is a labeled dataset that is produced by Amazon SageMaker Ground Truth.
This parameter is required if you set DataFormat to AUGMENTED_MANIFEST .
(dict) --
An augmented manifest file that provides training data for your custom model. An augmented manifest file is a labeled dataset that is produced by Amazon SageMaker Ground Truth.
S3Uri (string) -- [REQUIRED]
The Amazon S3 location of the augmented manifest file.
Split (string) --
The purpose of the data you've provided in the augmented manifest. You can either train or test this data. If you don't specify, the default is train.
TRAIN - all of the documents in the manifest will be used for training. If no test documents are provided, Amazon Comprehend will automatically reserve a portion of the training documents for testing.
TEST - all of the documents in the manifest will be used for testing.
AttributeNames (list) -- [REQUIRED]
The JSON attribute that contains the annotations for your training documents. The number of attribute names that you specify depends on whether your augmented manifest file is the output of a single labeling job or a chained labeling job.
If your file is the output of a single labeling job, specify the LabelAttributeName key that was used when the job was created in Ground Truth.
If your file is the output of a chained labeling job, specify the LabelAttributeName key for one or more jobs in the chain. Each LabelAttributeName key provides the annotations from an individual job.
(string) --
AnnotationDataS3Uri (string) --
The S3 prefix to the annotation files that are referred in the augmented manifest file.
SourceDocumentsS3Uri (string) --
The S3 prefix to the source files (PDFs) that are referred to in the augmented manifest file.
DocumentType (string) --
The type of augmented manifest. PlainTextDocument or SemiStructuredDocument. If you don't specify, the default is PlainTextDocument.
PLAIN_TEXT_DOCUMENT A document type that represents any unicode text that is encoded in UTF-8.
SEMI_STRUCTURED_DOCUMENT A document type with positional and structural context, like a PDF. For training with Amazon Comprehend, only PDFs are supported. For inference, Amazon Comprehend support PDFs, DOCX and TXT.
dict
Enables the addition of output results configuration parameters for custom classifier jobs.
S3Uri (string) --
When you use the OutputDataConfig object while creating a custom classifier, you specify the Amazon S3 location where you want to write the confusion matrix. The URI must be in the same region as the API endpoint that you are calling. The location is used as the prefix for the actual location of this output file.
When the custom classifier job is finished, the service creates the output file in a directory specific to the job. The S3Uri field contains the location of the output file, called output.tar.gz . It is a compressed archive that contains the confusion matrix.
KmsKeyId (string) --
ID for the AWS Key Management Service (KMS) key that Amazon Comprehend uses to encrypt the output results from an analysis job. The KmsKeyId can be one of the following formats:
KMS Key ID: "1234abcd-12ab-34cd-56ef-1234567890ab"
Amazon Resource Name (ARN) of a KMS Key: "arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab"
KMS Key Alias: "alias/ExampleAlias"
ARN of a KMS Key Alias: "arn:aws:kms:us-west-2:111122223333:alias/ExampleAlias"
FlywheelStatsS3Prefix (string) --
The Amazon S3 prefix for the data lake location of the flywheel statistics.
string
A unique identifier for the request. If you don't set the client request token, Amazon Comprehend generates one.
This field is autopopulated if not provided.
string
[REQUIRED]
The language of the input documents. You can specify any of the languages supported by Amazon Comprehend. All documents must be in the same language.
string
ID for the AWS Key Management Service (KMS) key that Amazon Comprehend uses to encrypt data on the storage volume attached to the ML compute instance(s) that process the analysis job. The VolumeKmsKeyId can be either of the following formats:
KMS Key ID: "1234abcd-12ab-34cd-56ef-1234567890ab"
Amazon Resource Name (ARN) of a KMS Key: "arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab"
dict
Configuration parameters for an optional private Virtual Private Cloud (VPC) containing the resources you are using for your custom classifier. For more information, see Amazon VPC .
SecurityGroupIds (list) -- [REQUIRED]
The ID number for a security group on an instance of your private VPC. Security groups on your VPC function serve as a virtual firewall to control inbound and outbound traffic and provides security for the resources that you’ll be accessing on the VPC. This ID number is preceded by "sg-", for instance: "sg-03b388029b0a285ea". For more information, see Security Groups for your VPC .
(string) --
Subnets (list) -- [REQUIRED]
The ID for each subnet being used in your private VPC. This subnet is a subset of the a range of IPv4 addresses used by the VPC and is specific to a given availability zone in the VPC’s region. This ID number is preceded by "subnet-", for instance: "subnet-04ccf456919e69055". For more information, see VPCs and Subnets .
(string) --
string
Indicates the mode in which the classifier will be trained. The classifier can be trained in multi-class mode, which identifies one and only one class for each document, or multi-label mode, which identifies one or more labels for each document. In multi-label mode, multiple labels for an individual document are separated by a delimiter. The default delimiter between labels is a pipe (|).
string
ID for the AWS Key Management Service (KMS) key that Amazon Comprehend uses to encrypt trained custom models. The ModelKmsKeyId can be either of the following formats:
KMS Key ID: "1234abcd-12ab-34cd-56ef-1234567890ab"
Amazon Resource Name (ARN) of a KMS Key: "arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab"
string
The resource-based policy to attach to your custom document classifier model. You can use this policy to allow another AWS account to import your custom model.
Provide your policy as a JSON body that you enter as a UTF-8 encoded string without line breaks. To provide valid JSON, enclose the attribute names and values in double quotes. If the JSON body is also enclosed in double quotes, then you must escape the double quotes that are inside the policy:
"{\"attribute\": \"value\", \"attribute\": [\"value\"]}"
To avoid escaping quotes, you can use single quotes to enclose the policy and double quotes to enclose the JSON names and values:
'{"attribute": "value", "attribute": ["value"]}'
dict
Response Syntax
{
'DocumentClassifierArn': 'string'
}
Response Structure
(dict) --
DocumentClassifierArn (string) --
The Amazon Resource Name (ARN) that identifies the document classifier.
{'FlywheelArn': 'string'}
Response {'ModelArn': 'string'}
Creates a model-specific endpoint for synchronous inference for a previously trained custom model For information about endpoints, see Managing endpoints .
See also: AWS API Documentation
Request Syntax
client.create_endpoint(
EndpointName='string',
ModelArn='string',
DesiredInferenceUnits=123,
ClientRequestToken='string',
Tags=[
{
'Key': 'string',
'Value': 'string'
},
],
DataAccessRoleArn='string',
FlywheelArn='string'
)
string
[REQUIRED]
This is the descriptive suffix that becomes part of the EndpointArn used for all subsequent requests to this resource.
string
The Amazon Resource Number (ARN) of the model to which the endpoint will be attached.
integer
[REQUIRED]
The desired number of inference units to be used by the model using this endpoint. Each inference unit represents of a throughput of 100 characters per second.
string
An idempotency token provided by the customer. If this token matches a previous endpoint creation request, Amazon Comprehend will not return a ResourceInUseException .
This field is autopopulated if not provided.
list
Tags to associate with the endpoint. A tag is a key-value pair that adds metadata to the endpoint. For example, a tag with "Sales" as the key might be added to an endpoint to indicate its use by the sales department.
(dict) --
A key-value pair that adds as a metadata to a resource used by Amazon Comprehend. For example, a tag with the key-value pair ‘Department’:’Sales’ might be added to a resource to indicate its use by a particular department.
Key (string) -- [REQUIRED]
The initial part of a key-value pair that forms a tag associated with a given resource. For instance, if you want to show which resources are used by which departments, you might use “Department” as the key portion of the pair, with multiple possible values such as “sales,” “legal,” and “administration.”
Value (string) --
The second part of a key-value pair that forms a tag associated with a given resource. For instance, if you want to show which resources are used by which departments, you might use “Department” as the initial (key) portion of the pair, with a value of “sales” to indicate the sales department.
string
The Amazon Resource Name (ARN) of the AWS Identity and Access Management (IAM) role that grants Amazon Comprehend read access to trained custom models encrypted with a customer managed key (ModelKmsKeyId).
string
The Amazon Resource Number (ARN) of the flywheel to which the endpoint will be attached.
dict
Response Syntax
{
'EndpointArn': 'string',
'ModelArn': 'string'
}
Response Structure
(dict) --
EndpointArn (string) --
The Amazon Resource Number (ARN) of the endpoint being created.
ModelArn (string) --
The Amazon Resource Number (ARN) of the model to which the endpoint is attached.
{'DocumentClassificationJobProperties': {'FlywheelArn': 'string'}}
Gets the properties associated with a document classification job. Use this operation to get the status of a classification job.
See also: AWS API Documentation
Request Syntax
client.describe_document_classification_job(
JobId='string'
)
string
[REQUIRED]
The identifier that Amazon Comprehend generated for the job. The StartDocumentClassificationJob operation returns this identifier in its response.
dict
Response Syntax
{
'DocumentClassificationJobProperties': {
'JobId': 'string',
'JobArn': 'string',
'JobName': 'string',
'JobStatus': 'SUBMITTED'|'IN_PROGRESS'|'COMPLETED'|'FAILED'|'STOP_REQUESTED'|'STOPPED',
'Message': 'string',
'SubmitTime': datetime(2015, 1, 1),
'EndTime': datetime(2015, 1, 1),
'DocumentClassifierArn': 'string',
'InputDataConfig': {
'S3Uri': 'string',
'InputFormat': 'ONE_DOC_PER_FILE'|'ONE_DOC_PER_LINE',
'DocumentReaderConfig': {
'DocumentReadAction': 'TEXTRACT_DETECT_DOCUMENT_TEXT'|'TEXTRACT_ANALYZE_DOCUMENT',
'DocumentReadMode': 'SERVICE_DEFAULT'|'FORCE_DOCUMENT_READ_ACTION',
'FeatureTypes': [
'TABLES'|'FORMS',
]
}
},
'OutputDataConfig': {
'S3Uri': 'string',
'KmsKeyId': 'string'
},
'DataAccessRoleArn': 'string',
'VolumeKmsKeyId': 'string',
'VpcConfig': {
'SecurityGroupIds': [
'string',
],
'Subnets': [
'string',
]
},
'FlywheelArn': 'string'
}
}
Response Structure
(dict) --
DocumentClassificationJobProperties (dict) --
An object that describes the properties associated with the document classification job.
JobId (string) --
The identifier assigned to the document classification job.
JobArn (string) --
The Amazon Resource Name (ARN) of the document classification job. It is a unique, fully qualified identifier for the job. It includes the AWS account, Region, and the job ID. The format of the ARN is as follows:
arn:<partition>:comprehend:<region>:<account-id>:document-classification-job/<job-id>
The following is an example job ARN:
arn:aws:comprehend:us-west-2:111122223333:document-classification-job/1234abcd12ab34cd56ef1234567890ab
JobName (string) --
The name that you assigned to the document classification job.
JobStatus (string) --
The current status of the document classification job. If the status is FAILED , the Message field shows the reason for the failure.
Message (string) --
A description of the status of the job.
SubmitTime (datetime) --
The time that the document classification job was submitted for processing.
EndTime (datetime) --
The time that the document classification job completed.
DocumentClassifierArn (string) --
The Amazon Resource Name (ARN) that identifies the document classifier.
InputDataConfig (dict) --
The input data configuration that you supplied when you created the document classification job.
S3Uri (string) --
The Amazon S3 URI for the input data. The URI must be in same region as the API endpoint that you are calling. The URI can point to a single input file or it can provide the prefix for a collection of data files.
For example, if you use the URI S3://bucketName/prefix , if the prefix is a single file, Amazon Comprehend uses that file as input. If more than one file begins with the prefix, Amazon Comprehend uses all of them as input.
InputFormat (string) --
Specifies how the text in an input file should be processed:
ONE_DOC_PER_FILE - Each file is considered a separate document. Use this option when you are processing large documents, such as newspaper articles or scientific papers.
ONE_DOC_PER_LINE - Each line in a file is considered a separate document. Use this option when you are processing many short documents, such as text messages.
DocumentReaderConfig (dict) --
Provides configuration parameters to override the default actions for extracting text from PDF documents and image files.
DocumentReadAction (string) --
This field defines the Amazon Textract API operation that Amazon Comprehend uses to extract text from PDF files and image files. Enter one of the following values:
TEXTRACT_DETECT_DOCUMENT_TEXT - The Amazon Comprehend service uses the DetectDocumentText API operation.
TEXTRACT_ANALYZE_DOCUMENT - The Amazon Comprehend service uses the AnalyzeDocument API operation.
DocumentReadMode (string) --
Determines the text extraction actions for PDF files. Enter one of the following values:
SERVICE_DEFAULT - use the Amazon Comprehend service defaults for PDF files.
FORCE_DOCUMENT_READ_ACTION - Amazon Comprehend uses the Textract API specified by DocumentReadAction for all PDF files, including digital PDF files.
FeatureTypes (list) --
Specifies the type of Amazon Textract features to apply. If you chose TEXTRACT_ANALYZE_DOCUMENT as the read action, you must specify one or both of the following values:
TABLES - Returns information about any tables that are detected in the input document.
FORMS - Returns information and the data from any forms that are detected in the input document.
(string) --
Specifies the type of Amazon Textract features to apply. If you chose TEXTRACT_ANALYZE_DOCUMENT as the read action, you must specify one or both of the following values:
TABLES - Returns additional information about any tables that are detected in the input document.
FORMS - Returns additional information about any forms that are detected in the input document.
OutputDataConfig (dict) --
The output data configuration that you supplied when you created the document classification job.
S3Uri (string) --
When you use the OutputDataConfig object with asynchronous operations, you specify the Amazon S3 location where you want to write the output data. The URI must be in the same region as the API endpoint that you are calling. The location is used as the prefix for the actual location of the output file.
When the topic detection job is finished, the service creates an output file in a directory specific to the job. The S3Uri field contains the location of the output file, called output.tar.gz . It is a compressed archive that contains the ouput of the operation.
For a PII entity detection job, the output file is plain text, not a compressed archive. The output file name is the same as the input file, with .out appended at the end.
KmsKeyId (string) --
ID for the AWS Key Management Service (KMS) key that Amazon Comprehend uses to encrypt the output results from an analysis job. The KmsKeyId can be one of the following formats:
KMS Key ID: "1234abcd-12ab-34cd-56ef-1234567890ab"
Amazon Resource Name (ARN) of a KMS Key: "arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab"
KMS Key Alias: "alias/ExampleAlias"
ARN of a KMS Key Alias: "arn:aws:kms:us-west-2:111122223333:alias/ExampleAlias"
DataAccessRoleArn (string) --
The Amazon Resource Name (ARN) of the AWS Identity and Access Management (IAM) role that grants Amazon Comprehend read access to your input data.
VolumeKmsKeyId (string) --
ID for the AWS Key Management Service (KMS) key that Amazon Comprehend uses to encrypt data on the storage volume attached to the ML compute instance(s) that process the analysis job. The VolumeKmsKeyId can be either of the following formats:
KMS Key ID: "1234abcd-12ab-34cd-56ef-1234567890ab"
Amazon Resource Name (ARN) of a KMS Key: "arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab"
VpcConfig (dict) --
Configuration parameters for a private Virtual Private Cloud (VPC) containing the resources you are using for your document classification job. For more information, see Amazon VPC .
SecurityGroupIds (list) --
The ID number for a security group on an instance of your private VPC. Security groups on your VPC function serve as a virtual firewall to control inbound and outbound traffic and provides security for the resources that you’ll be accessing on the VPC. This ID number is preceded by "sg-", for instance: "sg-03b388029b0a285ea". For more information, see Security Groups for your VPC .
(string) --
Subnets (list) --
The ID for each subnet being used in your private VPC. This subnet is a subset of the a range of IPv4 addresses used by the VPC and is specific to a given availability zone in the VPC’s region. This ID number is preceded by "subnet-", for instance: "subnet-04ccf456919e69055". For more information, see VPCs and Subnets .
(string) --
FlywheelArn (string) --
The Amazon Resource Number (ARN) of the flywheel
{'DocumentClassifierProperties': {'FlywheelArn': 'string',
'OutputDataConfig': {'FlywheelStatsS3Prefix': 'string'}}}
Gets the properties associated with a document classifier.
See also: AWS API Documentation
Request Syntax
client.describe_document_classifier(
DocumentClassifierArn='string'
)
string
[REQUIRED]
The Amazon Resource Name (ARN) that identifies the document classifier. The CreateDocumentClassifier operation returns this identifier in its response.
dict
Response Syntax
{
'DocumentClassifierProperties': {
'DocumentClassifierArn': 'string',
'LanguageCode': 'en'|'es'|'fr'|'de'|'it'|'pt'|'ar'|'hi'|'ja'|'ko'|'zh'|'zh-TW',
'Status': 'SUBMITTED'|'TRAINING'|'DELETING'|'STOP_REQUESTED'|'STOPPED'|'IN_ERROR'|'TRAINED',
'Message': 'string',
'SubmitTime': datetime(2015, 1, 1),
'EndTime': datetime(2015, 1, 1),
'TrainingStartTime': datetime(2015, 1, 1),
'TrainingEndTime': datetime(2015, 1, 1),
'InputDataConfig': {
'DataFormat': 'COMPREHEND_CSV'|'AUGMENTED_MANIFEST',
'S3Uri': 'string',
'TestS3Uri': 'string',
'LabelDelimiter': 'string',
'AugmentedManifests': [
{
'S3Uri': 'string',
'Split': 'TRAIN'|'TEST',
'AttributeNames': [
'string',
],
'AnnotationDataS3Uri': 'string',
'SourceDocumentsS3Uri': 'string',
'DocumentType': 'PLAIN_TEXT_DOCUMENT'|'SEMI_STRUCTURED_DOCUMENT'
},
]
},
'OutputDataConfig': {
'S3Uri': 'string',
'KmsKeyId': 'string',
'FlywheelStatsS3Prefix': 'string'
},
'ClassifierMetadata': {
'NumberOfLabels': 123,
'NumberOfTrainedDocuments': 123,
'NumberOfTestDocuments': 123,
'EvaluationMetrics': {
'Accuracy': 123.0,
'Precision': 123.0,
'Recall': 123.0,
'F1Score': 123.0,
'MicroPrecision': 123.0,
'MicroRecall': 123.0,
'MicroF1Score': 123.0,
'HammingLoss': 123.0
}
},
'DataAccessRoleArn': 'string',
'VolumeKmsKeyId': 'string',
'VpcConfig': {
'SecurityGroupIds': [
'string',
],
'Subnets': [
'string',
]
},
'Mode': 'MULTI_CLASS'|'MULTI_LABEL',
'ModelKmsKeyId': 'string',
'VersionName': 'string',
'SourceModelArn': 'string',
'FlywheelArn': 'string'
}
}
Response Structure
(dict) --
DocumentClassifierProperties (dict) --
An object that contains the properties associated with a document classifier.
DocumentClassifierArn (string) --
The Amazon Resource Name (ARN) that identifies the document classifier.
LanguageCode (string) --
The language code for the language of the documents that the classifier was trained on.
Status (string) --
The status of the document classifier. If the status is TRAINED the classifier is ready to use. If the status is FAILED you can see additional information about why the classifier wasn't trained in the Message field.
Message (string) --
Additional information about the status of the classifier.
SubmitTime (datetime) --
The time that the document classifier was submitted for training.
EndTime (datetime) --
The time that training the document classifier completed.
TrainingStartTime (datetime) --
Indicates the time when the training starts on documentation classifiers. You are billed for the time interval between this time and the value of TrainingEndTime.
TrainingEndTime (datetime) --
The time that training of the document classifier was completed. Indicates the time when the training completes on documentation classifiers. You are billed for the time interval between this time and the value of TrainingStartTime.
InputDataConfig (dict) --
The input data configuration that you supplied when you created the document classifier for training.
DataFormat (string) --
The format of your training data:
COMPREHEND_CSV : A two-column CSV file, where labels are provided in the first column, and documents are provided in the second. If you use this value, you must provide the S3Uri parameter in your request.
AUGMENTED_MANIFEST : A labeled dataset that is produced by Amazon SageMaker Ground Truth. This file is in JSON lines format. Each line is a complete JSON object that contains a training document and its associated labels. If you use this value, you must provide the AugmentedManifests parameter in your request.
If you don't specify a value, Amazon Comprehend uses COMPREHEND_CSV as the default.
S3Uri (string) --
The Amazon S3 URI for the input data. The S3 bucket must be in the same region as the API endpoint that you are calling. The URI can point to a single input file or it can provide the prefix for a collection of input files.
For example, if you use the URI S3://bucketName/prefix , if the prefix is a single file, Amazon Comprehend uses that file as input. If more than one file begins with the prefix, Amazon Comprehend uses all of them as input.
This parameter is required if you set DataFormat to COMPREHEND_CSV .
TestS3Uri (string) --
This specifies the Amazon S3 location where the test annotations for an entity recognizer are located. The URI must be in the same AWS Region as the API endpoint that you are calling.
LabelDelimiter (string) --
Indicates the delimiter used to separate each label for training a multi-label classifier. The default delimiter between labels is a pipe (|). You can use a different character as a delimiter (if it's an allowed character) by specifying it under Delimiter for labels. If the training documents use a delimiter other than the default or the delimiter you specify, the labels on that line will be combined to make a single unique label, such as LABELLABELLABEL.
AugmentedManifests (list) --
A list of augmented manifest files that provide training data for your custom model. An augmented manifest file is a labeled dataset that is produced by Amazon SageMaker Ground Truth.
This parameter is required if you set DataFormat to AUGMENTED_MANIFEST .
(dict) --
An augmented manifest file that provides training data for your custom model. An augmented manifest file is a labeled dataset that is produced by Amazon SageMaker Ground Truth.
S3Uri (string) --
The Amazon S3 location of the augmented manifest file.
Split (string) --
The purpose of the data you've provided in the augmented manifest. You can either train or test this data. If you don't specify, the default is train.
TRAIN - all of the documents in the manifest will be used for training. If no test documents are provided, Amazon Comprehend will automatically reserve a portion of the training documents for testing.
TEST - all of the documents in the manifest will be used for testing.
AttributeNames (list) --
The JSON attribute that contains the annotations for your training documents. The number of attribute names that you specify depends on whether your augmented manifest file is the output of a single labeling job or a chained labeling job.
If your file is the output of a single labeling job, specify the LabelAttributeName key that was used when the job was created in Ground Truth.
If your file is the output of a chained labeling job, specify the LabelAttributeName key for one or more jobs in the chain. Each LabelAttributeName key provides the annotations from an individual job.
(string) --
AnnotationDataS3Uri (string) --
The S3 prefix to the annotation files that are referred in the augmented manifest file.
SourceDocumentsS3Uri (string) --
The S3 prefix to the source files (PDFs) that are referred to in the augmented manifest file.
DocumentType (string) --
The type of augmented manifest. PlainTextDocument or SemiStructuredDocument. If you don't specify, the default is PlainTextDocument.
PLAIN_TEXT_DOCUMENT A document type that represents any unicode text that is encoded in UTF-8.
SEMI_STRUCTURED_DOCUMENT A document type with positional and structural context, like a PDF. For training with Amazon Comprehend, only PDFs are supported. For inference, Amazon Comprehend support PDFs, DOCX and TXT.
OutputDataConfig (dict) --
Provides output results configuration parameters for custom classifier jobs.
S3Uri (string) --
When you use the OutputDataConfig object while creating a custom classifier, you specify the Amazon S3 location where you want to write the confusion matrix. The URI must be in the same region as the API endpoint that you are calling. The location is used as the prefix for the actual location of this output file.
When the custom classifier job is finished, the service creates the output file in a directory specific to the job. The S3Uri field contains the location of the output file, called output.tar.gz . It is a compressed archive that contains the confusion matrix.
KmsKeyId (string) --
ID for the AWS Key Management Service (KMS) key that Amazon Comprehend uses to encrypt the output results from an analysis job. The KmsKeyId can be one of the following formats:
KMS Key ID: "1234abcd-12ab-34cd-56ef-1234567890ab"
Amazon Resource Name (ARN) of a KMS Key: "arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab"
KMS Key Alias: "alias/ExampleAlias"
ARN of a KMS Key Alias: "arn:aws:kms:us-west-2:111122223333:alias/ExampleAlias"
FlywheelStatsS3Prefix (string) --
The Amazon S3 prefix for the data lake location of the flywheel statistics.
ClassifierMetadata (dict) --
Information about the document classifier, including the number of documents used for training the classifier, the number of documents used for test the classifier, and an accuracy rating.
NumberOfLabels (integer) --
The number of labels in the input data.
NumberOfTrainedDocuments (integer) --
The number of documents in the input data that were used to train the classifier. Typically this is 80 to 90 percent of the input documents.
NumberOfTestDocuments (integer) --
The number of documents in the input data that were used to test the classifier. Typically this is 10 to 20 percent of the input documents, up to 10,000 documents.
EvaluationMetrics (dict) --
Describes the result metrics for the test data associated with an documentation classifier.
Accuracy (float) --
The fraction of the labels that were correct recognized. It is computed by dividing the number of labels in the test documents that were correctly recognized by the total number of labels in the test documents.
Precision (float) --
A measure of the usefulness of the classifier results in the test data. High precision means that the classifier returned substantially more relevant results than irrelevant ones.
Recall (float) --
A measure of how complete the classifier results are for the test data. High recall means that the classifier returned most of the relevant results.
F1Score (float) --
A measure of how accurate the classifier results are for the test data. It is derived from the Precision and Recall values. The F1Score is the harmonic average of the two scores. The highest score is 1, and the worst score is 0.
MicroPrecision (float) --
A measure of the usefulness of the recognizer results in the test data. High precision means that the recognizer returned substantially more relevant results than irrelevant ones. Unlike the Precision metric which comes from averaging the precision of all available labels, this is based on the overall score of all precision scores added together.
MicroRecall (float) --
A measure of how complete the classifier results are for the test data. High recall means that the classifier returned most of the relevant results. Specifically, this indicates how many of the correct categories in the text that the model can predict. It is a percentage of correct categories in the text that can found. Instead of averaging the recall scores of all labels (as with Recall), micro Recall is based on the overall score of all recall scores added together.
MicroF1Score (float) --
A measure of how accurate the classifier results are for the test data. It is a combination of the Micro Precision and Micro Recall values. The Micro F1Score is the harmonic mean of the two scores. The highest score is 1, and the worst score is 0.
HammingLoss (float) --
Indicates the fraction of labels that are incorrectly predicted. Also seen as the fraction of wrong labels compared to the total number of labels. Scores closer to zero are better.
DataAccessRoleArn (string) --
The Amazon Resource Name (ARN) of the AWS Identity and Access Management (IAM) role that grants Amazon Comprehend read access to your input data.
VolumeKmsKeyId (string) --
ID for the AWS Key Management Service (KMS) key that Amazon Comprehend uses to encrypt data on the storage volume attached to the ML compute instance(s) that process the analysis job. The VolumeKmsKeyId can be either of the following formats:
KMS Key ID: "1234abcd-12ab-34cd-56ef-1234567890ab"
Amazon Resource Name (ARN) of a KMS Key: "arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab"
VpcConfig (dict) --
Configuration parameters for a private Virtual Private Cloud (VPC) containing the resources you are using for your custom classifier. For more information, see Amazon VPC .
SecurityGroupIds (list) --
The ID number for a security group on an instance of your private VPC. Security groups on your VPC function serve as a virtual firewall to control inbound and outbound traffic and provides security for the resources that you’ll be accessing on the VPC. This ID number is preceded by "sg-", for instance: "sg-03b388029b0a285ea". For more information, see Security Groups for your VPC .
(string) --
Subnets (list) --
The ID for each subnet being used in your private VPC. This subnet is a subset of the a range of IPv4 addresses used by the VPC and is specific to a given availability zone in the VPC’s region. This ID number is preceded by "subnet-", for instance: "subnet-04ccf456919e69055". For more information, see VPCs and Subnets .
(string) --
Mode (string) --
Indicates the mode in which the specific classifier was trained. This also indicates the format of input documents and the format of the confusion matrix. Each classifier can only be trained in one mode and this cannot be changed once the classifier is trained.
ModelKmsKeyId (string) --
ID for the AWS Key Management Service (KMS) key that Amazon Comprehend uses to encrypt trained custom models. The ModelKmsKeyId can be either of the following formats:
KMS Key ID: "1234abcd-12ab-34cd-56ef-1234567890ab"
Amazon Resource Name (ARN) of a KMS Key: "arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab"
VersionName (string) --
The version name that you assigned to the document classifier.
SourceModelArn (string) --
The Amazon Resource Name (ARN) of the source model. This model was imported from a different AWS account to create the document classifier model in your AWS account.
FlywheelArn (string) --
The Amazon Resource Number (ARN) of the flywheel
{'EndpointProperties': {'FlywheelArn': 'string'}}
Gets the properties associated with a specific endpoint. Use this operation to get the status of an endpoint. For information about endpoints, see Managing endpoints .
See also: AWS API Documentation
Request Syntax
client.describe_endpoint(
EndpointArn='string'
)
string
[REQUIRED]
The Amazon Resource Number (ARN) of the endpoint being described.
dict
Response Syntax
{
'EndpointProperties': {
'EndpointArn': 'string',
'Status': 'CREATING'|'DELETING'|'FAILED'|'IN_SERVICE'|'UPDATING',
'Message': 'string',
'ModelArn': 'string',
'DesiredModelArn': 'string',
'DesiredInferenceUnits': 123,
'CurrentInferenceUnits': 123,
'CreationTime': datetime(2015, 1, 1),
'LastModifiedTime': datetime(2015, 1, 1),
'DataAccessRoleArn': 'string',
'DesiredDataAccessRoleArn': 'string',
'FlywheelArn': 'string'
}
}
Response Structure
(dict) --
EndpointProperties (dict) --
Describes information associated with the specific endpoint.
EndpointArn (string) --
The Amazon Resource Number (ARN) of the endpoint.
Status (string) --
Specifies the status of the endpoint. Because the endpoint updates and creation are asynchronous, so customers will need to wait for the endpoint to be Ready status before making inference requests.
Message (string) --
Specifies a reason for failure in cases of Failed status.
ModelArn (string) --
The Amazon Resource Number (ARN) of the model to which the endpoint is attached.
DesiredModelArn (string) --
ARN of the new model to use for updating an existing endpoint. This ARN is going to be different from the model ARN when the update is in progress
DesiredInferenceUnits (integer) --
The desired number of inference units to be used by the model using this endpoint. Each inference unit represents of a throughput of 100 characters per second.
CurrentInferenceUnits (integer) --
The number of inference units currently used by the model using this endpoint.
CreationTime (datetime) --
The creation date and time of the endpoint.
LastModifiedTime (datetime) --
The date and time that the endpoint was last modified.
DataAccessRoleArn (string) --
The Amazon Resource Name (ARN) of the AWS Identity and Access Management (IAM) role that grants Amazon Comprehend read access to trained custom models encrypted with a customer managed key (ModelKmsKeyId).
DesiredDataAccessRoleArn (string) --
Data access role ARN to use in case the new model is encrypted with a customer KMS key.
FlywheelArn (string) --
The Amazon Resource Number (ARN) of the flywheel
{'EntityRecognizerProperties': {'FlywheelArn': 'string',
'OutputDataConfig': {'FlywheelStatsS3Prefix': 'string'}}}
Provides details about an entity recognizer including status, S3 buckets containing training data, recognizer metadata, metrics, and so on.
See also: AWS API Documentation
Request Syntax
client.describe_entity_recognizer(
EntityRecognizerArn='string'
)
string
[REQUIRED]
The Amazon Resource Name (ARN) that identifies the entity recognizer.
dict
Response Syntax
{
'EntityRecognizerProperties': {
'EntityRecognizerArn': 'string',
'LanguageCode': 'en'|'es'|'fr'|'de'|'it'|'pt'|'ar'|'hi'|'ja'|'ko'|'zh'|'zh-TW',
'Status': 'SUBMITTED'|'TRAINING'|'DELETING'|'STOP_REQUESTED'|'STOPPED'|'IN_ERROR'|'TRAINED',
'Message': 'string',
'SubmitTime': datetime(2015, 1, 1),
'EndTime': datetime(2015, 1, 1),
'TrainingStartTime': datetime(2015, 1, 1),
'TrainingEndTime': datetime(2015, 1, 1),
'InputDataConfig': {
'DataFormat': 'COMPREHEND_CSV'|'AUGMENTED_MANIFEST',
'EntityTypes': [
{
'Type': 'string'
},
],
'Documents': {
'S3Uri': 'string',
'TestS3Uri': 'string',
'InputFormat': 'ONE_DOC_PER_FILE'|'ONE_DOC_PER_LINE'
},
'Annotations': {
'S3Uri': 'string',
'TestS3Uri': 'string'
},
'EntityList': {
'S3Uri': 'string'
},
'AugmentedManifests': [
{
'S3Uri': 'string',
'Split': 'TRAIN'|'TEST',
'AttributeNames': [
'string',
],
'AnnotationDataS3Uri': 'string',
'SourceDocumentsS3Uri': 'string',
'DocumentType': 'PLAIN_TEXT_DOCUMENT'|'SEMI_STRUCTURED_DOCUMENT'
},
]
},
'RecognizerMetadata': {
'NumberOfTrainedDocuments': 123,
'NumberOfTestDocuments': 123,
'EvaluationMetrics': {
'Precision': 123.0,
'Recall': 123.0,
'F1Score': 123.0
},
'EntityTypes': [
{
'Type': 'string',
'EvaluationMetrics': {
'Precision': 123.0,
'Recall': 123.0,
'F1Score': 123.0
},
'NumberOfTrainMentions': 123
},
]
},
'DataAccessRoleArn': 'string',
'VolumeKmsKeyId': 'string',
'VpcConfig': {
'SecurityGroupIds': [
'string',
],
'Subnets': [
'string',
]
},
'ModelKmsKeyId': 'string',
'VersionName': 'string',
'SourceModelArn': 'string',
'FlywheelArn': 'string',
'OutputDataConfig': {
'FlywheelStatsS3Prefix': 'string'
}
}
}
Response Structure
(dict) --
EntityRecognizerProperties (dict) --
Describes information associated with an entity recognizer.
EntityRecognizerArn (string) --
The Amazon Resource Name (ARN) that identifies the entity recognizer.
LanguageCode (string) --
The language of the input documents. All documents must be in the same language. Only English ("en") is currently supported.
Status (string) --
Provides the status of the entity recognizer.
Message (string) --
A description of the status of the recognizer.
SubmitTime (datetime) --
The time that the recognizer was submitted for processing.
EndTime (datetime) --
The time that the recognizer creation completed.
TrainingStartTime (datetime) --
The time that training of the entity recognizer started.
TrainingEndTime (datetime) --
The time that training of the entity recognizer was completed.
InputDataConfig (dict) --
The input data properties of an entity recognizer.
DataFormat (string) --
The format of your training data:
COMPREHEND_CSV : A CSV file that supplements your training documents. The CSV file contains information about the custom entities that your trained model will detect. The required format of the file depends on whether you are providing annotations or an entity list. If you use this value, you must provide your CSV file by using either the Annotations or EntityList parameters. You must provide your training documents by using the Documents parameter.
AUGMENTED_MANIFEST : A labeled dataset that is produced by Amazon SageMaker Ground Truth. This file is in JSON lines format. Each line is a complete JSON object that contains a training document and its labels. Each label annotates a named entity in the training document. If you use this value, you must provide the AugmentedManifests parameter in your request.
If you don't specify a value, Amazon Comprehend uses COMPREHEND_CSV as the default.
EntityTypes (list) --
The entity types in the labeled training data that Amazon Comprehend uses to train the custom entity recognizer. Any entity types that you don't specify are ignored.
A maximum of 25 entity types can be used at one time to train an entity recognizer. Entity types must not contain the following invalid characters: n (line break), \n (escaped line break), r (carriage return), \r (escaped carriage return), t (tab), \t (escaped tab), space, and , (comma).
(dict) --
An entity type within a labeled training dataset that Amazon Comprehend uses to train a custom entity recognizer.
Type (string) --
An entity type within a labeled training dataset that Amazon Comprehend uses to train a custom entity recognizer.
Entity types must not contain the following invalid characters: n (line break), \n (escaped line break, r (carriage return), \r (escaped carriage return), t (tab), \t (escaped tab), space, and , (comma).
Documents (dict) --
The S3 location of the folder that contains the training documents for your custom entity recognizer.
This parameter is required if you set DataFormat to COMPREHEND_CSV .
S3Uri (string) --
Specifies the Amazon S3 location where the training documents for an entity recognizer are located. The URI must be in the same region as the API endpoint that you are calling.
TestS3Uri (string) --
Specifies the Amazon S3 location where the test documents for an entity recognizer are located. The URI must be in the same AWS Region as the API endpoint that you are calling.
InputFormat (string) --
Specifies how the text in an input file should be processed. This is optional, and the default is ONE_DOC_PER_LINE. ONE_DOC_PER_FILE - Each file is considered a separate document. Use this option when you are processing large documents, such as newspaper articles or scientific papers. ONE_DOC_PER_LINE - Each line in a file is considered a separate document. Use this option when you are processing many short documents, such as text messages.
Annotations (dict) --
The S3 location of the CSV file that annotates your training documents.
S3Uri (string) --
Specifies the Amazon S3 location where the annotations for an entity recognizer are located. The URI must be in the same region as the API endpoint that you are calling.
TestS3Uri (string) --
Specifies the Amazon S3 location where the test annotations for an entity recognizer are located. The URI must be in the same region as the API endpoint that you are calling.
EntityList (dict) --
The S3 location of the CSV file that has the entity list for your custom entity recognizer.
S3Uri (string) --
Specifies the Amazon S3 location where the entity list is located. The URI must be in the same region as the API endpoint that you are calling.
AugmentedManifests (list) --
A list of augmented manifest files that provide training data for your custom model. An augmented manifest file is a labeled dataset that is produced by Amazon SageMaker Ground Truth.
This parameter is required if you set DataFormat to AUGMENTED_MANIFEST .
(dict) --
An augmented manifest file that provides training data for your custom model. An augmented manifest file is a labeled dataset that is produced by Amazon SageMaker Ground Truth.
S3Uri (string) --
The Amazon S3 location of the augmented manifest file.
Split (string) --
The purpose of the data you've provided in the augmented manifest. You can either train or test this data. If you don't specify, the default is train.
TRAIN - all of the documents in the manifest will be used for training. If no test documents are provided, Amazon Comprehend will automatically reserve a portion of the training documents for testing.
TEST - all of the documents in the manifest will be used for testing.
AttributeNames (list) --
The JSON attribute that contains the annotations for your training documents. The number of attribute names that you specify depends on whether your augmented manifest file is the output of a single labeling job or a chained labeling job.
If your file is the output of a single labeling job, specify the LabelAttributeName key that was used when the job was created in Ground Truth.
If your file is the output of a chained labeling job, specify the LabelAttributeName key for one or more jobs in the chain. Each LabelAttributeName key provides the annotations from an individual job.
(string) --
AnnotationDataS3Uri (string) --
The S3 prefix to the annotation files that are referred in the augmented manifest file.
SourceDocumentsS3Uri (string) --
The S3 prefix to the source files (PDFs) that are referred to in the augmented manifest file.
DocumentType (string) --
The type of augmented manifest. PlainTextDocument or SemiStructuredDocument. If you don't specify, the default is PlainTextDocument.
PLAIN_TEXT_DOCUMENT A document type that represents any unicode text that is encoded in UTF-8.
SEMI_STRUCTURED_DOCUMENT A document type with positional and structural context, like a PDF. For training with Amazon Comprehend, only PDFs are supported. For inference, Amazon Comprehend support PDFs, DOCX and TXT.
RecognizerMetadata (dict) --
Provides information about an entity recognizer.
NumberOfTrainedDocuments (integer) --
The number of documents in the input data that were used to train the entity recognizer. Typically this is 80 to 90 percent of the input documents.
NumberOfTestDocuments (integer) --
The number of documents in the input data that were used to test the entity recognizer. Typically this is 10 to 20 percent of the input documents.
EvaluationMetrics (dict) --
Detailed information about the accuracy of an entity recognizer.
Precision (float) --
A measure of the usefulness of the recognizer results in the test data. High precision means that the recognizer returned substantially more relevant results than irrelevant ones.
Recall (float) --
A measure of how complete the recognizer results are for the test data. High recall means that the recognizer returned most of the relevant results.
F1Score (float) --
A measure of how accurate the recognizer results are for the test data. It is derived from the Precision and Recall values. The F1Score is the harmonic average of the two scores. For plain text entity recognizer models, the range is 0 to 100, where 100 is the best score. For PDF/Word entity recognizer models, the range is 0 to 1, where 1 is the best score.
EntityTypes (list) --
Entity types from the metadata of an entity recognizer.
(dict) --
Individual item from the list of entity types in the metadata of an entity recognizer.
Type (string) --
Type of entity from the list of entity types in the metadata of an entity recognizer.
EvaluationMetrics (dict) --
Detailed information about the accuracy of the entity recognizer for a specific item on the list of entity types.
Precision (float) --
A measure of the usefulness of the recognizer results for a specific entity type in the test data. High precision means that the recognizer returned substantially more relevant results than irrelevant ones.
Recall (float) --
A measure of how complete the recognizer results are for a specific entity type in the test data. High recall means that the recognizer returned most of the relevant results.
F1Score (float) --
A measure of how accurate the recognizer results are for a specific entity type in the test data. It is derived from the Precision and Recall values. The F1Score is the harmonic average of the two scores. The highest score is 1, and the worst score is 0.
NumberOfTrainMentions (integer) --
Indicates the number of times the given entity type was seen in the training data.
DataAccessRoleArn (string) --
The Amazon Resource Name (ARN) of the AWS Identity and Access Management (IAM) role that grants Amazon Comprehend read access to your input data.
VolumeKmsKeyId (string) --
ID for the AWS Key Management Service (KMS) key that Amazon Comprehend uses to encrypt data on the storage volume attached to the ML compute instance(s) that process the analysis job. The VolumeKmsKeyId can be either of the following formats:
KMS Key ID: "1234abcd-12ab-34cd-56ef-1234567890ab"
Amazon Resource Name (ARN) of a KMS Key: "arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab"
VpcConfig (dict) --
Configuration parameters for a private Virtual Private Cloud (VPC) containing the resources you are using for your custom entity recognizer. For more information, see Amazon VPC .
SecurityGroupIds (list) --
The ID number for a security group on an instance of your private VPC. Security groups on your VPC function serve as a virtual firewall to control inbound and outbound traffic and provides security for the resources that you’ll be accessing on the VPC. This ID number is preceded by "sg-", for instance: "sg-03b388029b0a285ea". For more information, see Security Groups for your VPC .
(string) --
Subnets (list) --
The ID for each subnet being used in your private VPC. This subnet is a subset of the a range of IPv4 addresses used by the VPC and is specific to a given availability zone in the VPC’s region. This ID number is preceded by "subnet-", for instance: "subnet-04ccf456919e69055". For more information, see VPCs and Subnets .
(string) --
ModelKmsKeyId (string) --
ID for the AWS Key Management Service (KMS) key that Amazon Comprehend uses to encrypt trained custom models. The ModelKmsKeyId can be either of the following formats:
KMS Key ID: "1234abcd-12ab-34cd-56ef-1234567890ab"
Amazon Resource Name (ARN) of a KMS Key: "arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab"
VersionName (string) --
The version name you assigned to the entity recognizer.
SourceModelArn (string) --
The Amazon Resource Name (ARN) of the source model. This model was imported from a different AWS account to create the entity recognizer model in your AWS account.
FlywheelArn (string) --
The Amazon Resource Number (ARN) of the flywheel
OutputDataConfig (dict) --
Output data configuration.
FlywheelStatsS3Prefix (string) --
The Amazon S3 prefix for the data lake location of the flywheel statistics.
{'DocumentClassificationJobPropertiesList': {'FlywheelArn': 'string'}}
Gets a list of the documentation classification jobs that you have submitted.
See also: AWS API Documentation
Request Syntax
client.list_document_classification_jobs(
Filter={
'JobName': 'string',
'JobStatus': 'SUBMITTED'|'IN_PROGRESS'|'COMPLETED'|'FAILED'|'STOP_REQUESTED'|'STOPPED',
'SubmitTimeBefore': datetime(2015, 1, 1),
'SubmitTimeAfter': datetime(2015, 1, 1)
},
NextToken='string',
MaxResults=123
)
dict
Filters the jobs that are returned. You can filter jobs on their names, status, or the date and time that they were submitted. You can only set one filter at a time.
JobName (string) --
Filters on the name of the job.
JobStatus (string) --
Filters the list based on job status. Returns only jobs with the specified status.
SubmitTimeBefore (datetime) --
Filters the list of jobs based on the time that the job was submitted for processing. Returns only jobs submitted before the specified time. Jobs are returned in ascending order, oldest to newest.
SubmitTimeAfter (datetime) --
Filters the list of jobs based on the time that the job was submitted for processing. Returns only jobs submitted after the specified time. Jobs are returned in descending order, newest to oldest.
string
Identifies the next page of results to return.
integer
The maximum number of results to return in each page. The default is 100.
dict
Response Syntax
{
'DocumentClassificationJobPropertiesList': [
{
'JobId': 'string',
'JobArn': 'string',
'JobName': 'string',
'JobStatus': 'SUBMITTED'|'IN_PROGRESS'|'COMPLETED'|'FAILED'|'STOP_REQUESTED'|'STOPPED',
'Message': 'string',
'SubmitTime': datetime(2015, 1, 1),
'EndTime': datetime(2015, 1, 1),
'DocumentClassifierArn': 'string',
'InputDataConfig': {
'S3Uri': 'string',
'InputFormat': 'ONE_DOC_PER_FILE'|'ONE_DOC_PER_LINE',
'DocumentReaderConfig': {
'DocumentReadAction': 'TEXTRACT_DETECT_DOCUMENT_TEXT'|'TEXTRACT_ANALYZE_DOCUMENT',
'DocumentReadMode': 'SERVICE_DEFAULT'|'FORCE_DOCUMENT_READ_ACTION',
'FeatureTypes': [
'TABLES'|'FORMS',
]
}
},
'OutputDataConfig': {
'S3Uri': 'string',
'KmsKeyId': 'string'
},
'DataAccessRoleArn': 'string',
'VolumeKmsKeyId': 'string',
'VpcConfig': {
'SecurityGroupIds': [
'string',
],
'Subnets': [
'string',
]
},
'FlywheelArn': 'string'
},
],
'NextToken': 'string'
}
Response Structure
(dict) --
DocumentClassificationJobPropertiesList (list) --
A list containing the properties of each job returned.
(dict) --
Provides information about a document classification job.
JobId (string) --
The identifier assigned to the document classification job.
JobArn (string) --
The Amazon Resource Name (ARN) of the document classification job. It is a unique, fully qualified identifier for the job. It includes the AWS account, Region, and the job ID. The format of the ARN is as follows:
arn:<partition>:comprehend:<region>:<account-id>:document-classification-job/<job-id>
The following is an example job ARN:
arn:aws:comprehend:us-west-2:111122223333:document-classification-job/1234abcd12ab34cd56ef1234567890ab
JobName (string) --
The name that you assigned to the document classification job.
JobStatus (string) --
The current status of the document classification job. If the status is FAILED , the Message field shows the reason for the failure.
Message (string) --
A description of the status of the job.
SubmitTime (datetime) --
The time that the document classification job was submitted for processing.
EndTime (datetime) --
The time that the document classification job completed.
DocumentClassifierArn (string) --
The Amazon Resource Name (ARN) that identifies the document classifier.
InputDataConfig (dict) --
The input data configuration that you supplied when you created the document classification job.
S3Uri (string) --
The Amazon S3 URI for the input data. The URI must be in same region as the API endpoint that you are calling. The URI can point to a single input file or it can provide the prefix for a collection of data files.
For example, if you use the URI S3://bucketName/prefix , if the prefix is a single file, Amazon Comprehend uses that file as input. If more than one file begins with the prefix, Amazon Comprehend uses all of them as input.
InputFormat (string) --
Specifies how the text in an input file should be processed:
ONE_DOC_PER_FILE - Each file is considered a separate document. Use this option when you are processing large documents, such as newspaper articles or scientific papers.
ONE_DOC_PER_LINE - Each line in a file is considered a separate document. Use this option when you are processing many short documents, such as text messages.
DocumentReaderConfig (dict) --
Provides configuration parameters to override the default actions for extracting text from PDF documents and image files.
DocumentReadAction (string) --
This field defines the Amazon Textract API operation that Amazon Comprehend uses to extract text from PDF files and image files. Enter one of the following values:
TEXTRACT_DETECT_DOCUMENT_TEXT - The Amazon Comprehend service uses the DetectDocumentText API operation.
TEXTRACT_ANALYZE_DOCUMENT - The Amazon Comprehend service uses the AnalyzeDocument API operation.
DocumentReadMode (string) --
Determines the text extraction actions for PDF files. Enter one of the following values:
SERVICE_DEFAULT - use the Amazon Comprehend service defaults for PDF files.
FORCE_DOCUMENT_READ_ACTION - Amazon Comprehend uses the Textract API specified by DocumentReadAction for all PDF files, including digital PDF files.
FeatureTypes (list) --
Specifies the type of Amazon Textract features to apply. If you chose TEXTRACT_ANALYZE_DOCUMENT as the read action, you must specify one or both of the following values:
TABLES - Returns information about any tables that are detected in the input document.
FORMS - Returns information and the data from any forms that are detected in the input document.
(string) --
Specifies the type of Amazon Textract features to apply. If you chose TEXTRACT_ANALYZE_DOCUMENT as the read action, you must specify one or both of the following values:
TABLES - Returns additional information about any tables that are detected in the input document.
FORMS - Returns additional information about any forms that are detected in the input document.
OutputDataConfig (dict) --
The output data configuration that you supplied when you created the document classification job.
S3Uri (string) --
When you use the OutputDataConfig object with asynchronous operations, you specify the Amazon S3 location where you want to write the output data. The URI must be in the same region as the API endpoint that you are calling. The location is used as the prefix for the actual location of the output file.
When the topic detection job is finished, the service creates an output file in a directory specific to the job. The S3Uri field contains the location of the output file, called output.tar.gz . It is a compressed archive that contains the ouput of the operation.
For a PII entity detection job, the output file is plain text, not a compressed archive. The output file name is the same as the input file, with .out appended at the end.
KmsKeyId (string) --
ID for the AWS Key Management Service (KMS) key that Amazon Comprehend uses to encrypt the output results from an analysis job. The KmsKeyId can be one of the following formats:
KMS Key ID: "1234abcd-12ab-34cd-56ef-1234567890ab"
Amazon Resource Name (ARN) of a KMS Key: "arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab"
KMS Key Alias: "alias/ExampleAlias"
ARN of a KMS Key Alias: "arn:aws:kms:us-west-2:111122223333:alias/ExampleAlias"
DataAccessRoleArn (string) --
The Amazon Resource Name (ARN) of the AWS Identity and Access Management (IAM) role that grants Amazon Comprehend read access to your input data.
VolumeKmsKeyId (string) --
ID for the AWS Key Management Service (KMS) key that Amazon Comprehend uses to encrypt data on the storage volume attached to the ML compute instance(s) that process the analysis job. The VolumeKmsKeyId can be either of the following formats:
KMS Key ID: "1234abcd-12ab-34cd-56ef-1234567890ab"
Amazon Resource Name (ARN) of a KMS Key: "arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab"
VpcConfig (dict) --
Configuration parameters for a private Virtual Private Cloud (VPC) containing the resources you are using for your document classification job. For more information, see Amazon VPC .
SecurityGroupIds (list) --
The ID number for a security group on an instance of your private VPC. Security groups on your VPC function serve as a virtual firewall to control inbound and outbound traffic and provides security for the resources that you’ll be accessing on the VPC. This ID number is preceded by "sg-", for instance: "sg-03b388029b0a285ea". For more information, see Security Groups for your VPC .
(string) --
Subnets (list) --
The ID for each subnet being used in your private VPC. This subnet is a subset of the a range of IPv4 addresses used by the VPC and is specific to a given availability zone in the VPC’s region. This ID number is preceded by "subnet-", for instance: "subnet-04ccf456919e69055". For more information, see VPCs and Subnets .
(string) --
FlywheelArn (string) --
The Amazon Resource Number (ARN) of the flywheel
NextToken (string) --
Identifies the next page of results to return.
{'DocumentClassifierPropertiesList': {'FlywheelArn': 'string',
'OutputDataConfig': {'FlywheelStatsS3Prefix': 'string'}}}
Gets a list of the document classifiers that you have created.
See also: AWS API Documentation
Request Syntax
client.list_document_classifiers(
Filter={
'Status': 'SUBMITTED'|'TRAINING'|'DELETING'|'STOP_REQUESTED'|'STOPPED'|'IN_ERROR'|'TRAINED',
'DocumentClassifierName': 'string',
'SubmitTimeBefore': datetime(2015, 1, 1),
'SubmitTimeAfter': datetime(2015, 1, 1)
},
NextToken='string',
MaxResults=123
)
dict
Filters the jobs that are returned. You can filter jobs on their name, status, or the date and time that they were submitted. You can only set one filter at a time.
Status (string) --
Filters the list of classifiers based on status.
DocumentClassifierName (string) --
The name that you assigned to the document classifier
SubmitTimeBefore (datetime) --
Filters the list of classifiers based on the time that the classifier was submitted for processing. Returns only classifiers submitted before the specified time. Classifiers are returned in ascending order, oldest to newest.
SubmitTimeAfter (datetime) --
Filters the list of classifiers based on the time that the classifier was submitted for processing. Returns only classifiers submitted after the specified time. Classifiers are returned in descending order, newest to oldest.
string
Identifies the next page of results to return.
integer
The maximum number of results to return in each page. The default is 100.
dict
Response Syntax
{
'DocumentClassifierPropertiesList': [
{
'DocumentClassifierArn': 'string',
'LanguageCode': 'en'|'es'|'fr'|'de'|'it'|'pt'|'ar'|'hi'|'ja'|'ko'|'zh'|'zh-TW',
'Status': 'SUBMITTED'|'TRAINING'|'DELETING'|'STOP_REQUESTED'|'STOPPED'|'IN_ERROR'|'TRAINED',
'Message': 'string',
'SubmitTime': datetime(2015, 1, 1),
'EndTime': datetime(2015, 1, 1),
'TrainingStartTime': datetime(2015, 1, 1),
'TrainingEndTime': datetime(2015, 1, 1),
'InputDataConfig': {
'DataFormat': 'COMPREHEND_CSV'|'AUGMENTED_MANIFEST',
'S3Uri': 'string',
'TestS3Uri': 'string',
'LabelDelimiter': 'string',
'AugmentedManifests': [
{
'S3Uri': 'string',
'Split': 'TRAIN'|'TEST',
'AttributeNames': [
'string',
],
'AnnotationDataS3Uri': 'string',
'SourceDocumentsS3Uri': 'string',
'DocumentType': 'PLAIN_TEXT_DOCUMENT'|'SEMI_STRUCTURED_DOCUMENT'
},
]
},
'OutputDataConfig': {
'S3Uri': 'string',
'KmsKeyId': 'string',
'FlywheelStatsS3Prefix': 'string'
},
'ClassifierMetadata': {
'NumberOfLabels': 123,
'NumberOfTrainedDocuments': 123,
'NumberOfTestDocuments': 123,
'EvaluationMetrics': {
'Accuracy': 123.0,
'Precision': 123.0,
'Recall': 123.0,
'F1Score': 123.0,
'MicroPrecision': 123.0,
'MicroRecall': 123.0,
'MicroF1Score': 123.0,
'HammingLoss': 123.0
}
},
'DataAccessRoleArn': 'string',
'VolumeKmsKeyId': 'string',
'VpcConfig': {
'SecurityGroupIds': [
'string',
],
'Subnets': [
'string',
]
},
'Mode': 'MULTI_CLASS'|'MULTI_LABEL',
'ModelKmsKeyId': 'string',
'VersionName': 'string',
'SourceModelArn': 'string',
'FlywheelArn': 'string'
},
],
'NextToken': 'string'
}
Response Structure
(dict) --
DocumentClassifierPropertiesList (list) --
A list containing the properties of each job returned.
(dict) --
Provides information about a document classifier.
DocumentClassifierArn (string) --
The Amazon Resource Name (ARN) that identifies the document classifier.
LanguageCode (string) --
The language code for the language of the documents that the classifier was trained on.
Status (string) --
The status of the document classifier. If the status is TRAINED the classifier is ready to use. If the status is FAILED you can see additional information about why the classifier wasn't trained in the Message field.
Message (string) --
Additional information about the status of the classifier.
SubmitTime (datetime) --
The time that the document classifier was submitted for training.
EndTime (datetime) --
The time that training the document classifier completed.
TrainingStartTime (datetime) --
Indicates the time when the training starts on documentation classifiers. You are billed for the time interval between this time and the value of TrainingEndTime.
TrainingEndTime (datetime) --
The time that training of the document classifier was completed. Indicates the time when the training completes on documentation classifiers. You are billed for the time interval between this time and the value of TrainingStartTime.
InputDataConfig (dict) --
The input data configuration that you supplied when you created the document classifier for training.
DataFormat (string) --
The format of your training data:
COMPREHEND_CSV : A two-column CSV file, where labels are provided in the first column, and documents are provided in the second. If you use this value, you must provide the S3Uri parameter in your request.
AUGMENTED_MANIFEST : A labeled dataset that is produced by Amazon SageMaker Ground Truth. This file is in JSON lines format. Each line is a complete JSON object that contains a training document and its associated labels. If you use this value, you must provide the AugmentedManifests parameter in your request.
If you don't specify a value, Amazon Comprehend uses COMPREHEND_CSV as the default.
S3Uri (string) --
The Amazon S3 URI for the input data. The S3 bucket must be in the same region as the API endpoint that you are calling. The URI can point to a single input file or it can provide the prefix for a collection of input files.
For example, if you use the URI S3://bucketName/prefix , if the prefix is a single file, Amazon Comprehend uses that file as input. If more than one file begins with the prefix, Amazon Comprehend uses all of them as input.
This parameter is required if you set DataFormat to COMPREHEND_CSV .
TestS3Uri (string) --
This specifies the Amazon S3 location where the test annotations for an entity recognizer are located. The URI must be in the same AWS Region as the API endpoint that you are calling.
LabelDelimiter (string) --
Indicates the delimiter used to separate each label for training a multi-label classifier. The default delimiter between labels is a pipe (|). You can use a different character as a delimiter (if it's an allowed character) by specifying it under Delimiter for labels. If the training documents use a delimiter other than the default or the delimiter you specify, the labels on that line will be combined to make a single unique label, such as LABELLABELLABEL.
AugmentedManifests (list) --
A list of augmented manifest files that provide training data for your custom model. An augmented manifest file is a labeled dataset that is produced by Amazon SageMaker Ground Truth.
This parameter is required if you set DataFormat to AUGMENTED_MANIFEST .
(dict) --
An augmented manifest file that provides training data for your custom model. An augmented manifest file is a labeled dataset that is produced by Amazon SageMaker Ground Truth.
S3Uri (string) --
The Amazon S3 location of the augmented manifest file.
Split (string) --
The purpose of the data you've provided in the augmented manifest. You can either train or test this data. If you don't specify, the default is train.
TRAIN - all of the documents in the manifest will be used for training. If no test documents are provided, Amazon Comprehend will automatically reserve a portion of the training documents for testing.
TEST - all of the documents in the manifest will be used for testing.
AttributeNames (list) --
The JSON attribute that contains the annotations for your training documents. The number of attribute names that you specify depends on whether your augmented manifest file is the output of a single labeling job or a chained labeling job.
If your file is the output of a single labeling job, specify the LabelAttributeName key that was used when the job was created in Ground Truth.
If your file is the output of a chained labeling job, specify the LabelAttributeName key for one or more jobs in the chain. Each LabelAttributeName key provides the annotations from an individual job.
(string) --
AnnotationDataS3Uri (string) --
The S3 prefix to the annotation files that are referred in the augmented manifest file.
SourceDocumentsS3Uri (string) --
The S3 prefix to the source files (PDFs) that are referred to in the augmented manifest file.
DocumentType (string) --
The type of augmented manifest. PlainTextDocument or SemiStructuredDocument. If you don't specify, the default is PlainTextDocument.
PLAIN_TEXT_DOCUMENT A document type that represents any unicode text that is encoded in UTF-8.
SEMI_STRUCTURED_DOCUMENT A document type with positional and structural context, like a PDF. For training with Amazon Comprehend, only PDFs are supported. For inference, Amazon Comprehend support PDFs, DOCX and TXT.
OutputDataConfig (dict) --
Provides output results configuration parameters for custom classifier jobs.
S3Uri (string) --
When you use the OutputDataConfig object while creating a custom classifier, you specify the Amazon S3 location where you want to write the confusion matrix. The URI must be in the same region as the API endpoint that you are calling. The location is used as the prefix for the actual location of this output file.
When the custom classifier job is finished, the service creates the output file in a directory specific to the job. The S3Uri field contains the location of the output file, called output.tar.gz . It is a compressed archive that contains the confusion matrix.
KmsKeyId (string) --
ID for the AWS Key Management Service (KMS) key that Amazon Comprehend uses to encrypt the output results from an analysis job. The KmsKeyId can be one of the following formats:
KMS Key ID: "1234abcd-12ab-34cd-56ef-1234567890ab"
Amazon Resource Name (ARN) of a KMS Key: "arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab"
KMS Key Alias: "alias/ExampleAlias"
ARN of a KMS Key Alias: "arn:aws:kms:us-west-2:111122223333:alias/ExampleAlias"
FlywheelStatsS3Prefix (string) --
The Amazon S3 prefix for the data lake location of the flywheel statistics.
ClassifierMetadata (dict) --
Information about the document classifier, including the number of documents used for training the classifier, the number of documents used for test the classifier, and an accuracy rating.
NumberOfLabels (integer) --
The number of labels in the input data.
NumberOfTrainedDocuments (integer) --
The number of documents in the input data that were used to train the classifier. Typically this is 80 to 90 percent of the input documents.
NumberOfTestDocuments (integer) --
The number of documents in the input data that were used to test the classifier. Typically this is 10 to 20 percent of the input documents, up to 10,000 documents.
EvaluationMetrics (dict) --
Describes the result metrics for the test data associated with an documentation classifier.
Accuracy (float) --
The fraction of the labels that were correct recognized. It is computed by dividing the number of labels in the test documents that were correctly recognized by the total number of labels in the test documents.
Precision (float) --
A measure of the usefulness of the classifier results in the test data. High precision means that the classifier returned substantially more relevant results than irrelevant ones.
Recall (float) --
A measure of how complete the classifier results are for the test data. High recall means that the classifier returned most of the relevant results.
F1Score (float) --
A measure of how accurate the classifier results are for the test data. It is derived from the Precision and Recall values. The F1Score is the harmonic average of the two scores. The highest score is 1, and the worst score is 0.
MicroPrecision (float) --
A measure of the usefulness of the recognizer results in the test data. High precision means that the recognizer returned substantially more relevant results than irrelevant ones. Unlike the Precision metric which comes from averaging the precision of all available labels, this is based on the overall score of all precision scores added together.
MicroRecall (float) --
A measure of how complete the classifier results are for the test data. High recall means that the classifier returned most of the relevant results. Specifically, this indicates how many of the correct categories in the text that the model can predict. It is a percentage of correct categories in the text that can found. Instead of averaging the recall scores of all labels (as with Recall), micro Recall is based on the overall score of all recall scores added together.
MicroF1Score (float) --
A measure of how accurate the classifier results are for the test data. It is a combination of the Micro Precision and Micro Recall values. The Micro F1Score is the harmonic mean of the two scores. The highest score is 1, and the worst score is 0.
HammingLoss (float) --
Indicates the fraction of labels that are incorrectly predicted. Also seen as the fraction of wrong labels compared to the total number of labels. Scores closer to zero are better.
DataAccessRoleArn (string) --
The Amazon Resource Name (ARN) of the AWS Identity and Access Management (IAM) role that grants Amazon Comprehend read access to your input data.
VolumeKmsKeyId (string) --
ID for the AWS Key Management Service (KMS) key that Amazon Comprehend uses to encrypt data on the storage volume attached to the ML compute instance(s) that process the analysis job. The VolumeKmsKeyId can be either of the following formats:
KMS Key ID: "1234abcd-12ab-34cd-56ef-1234567890ab"
Amazon Resource Name (ARN) of a KMS Key: "arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab"
VpcConfig (dict) --
Configuration parameters for a private Virtual Private Cloud (VPC) containing the resources you are using for your custom classifier. For more information, see Amazon VPC .
SecurityGroupIds (list) --
The ID number for a security group on an instance of your private VPC. Security groups on your VPC function serve as a virtual firewall to control inbound and outbound traffic and provides security for the resources that you’ll be accessing on the VPC. This ID number is preceded by "sg-", for instance: "sg-03b388029b0a285ea". For more information, see Security Groups for your VPC .
(string) --
Subnets (list) --
The ID for each subnet being used in your private VPC. This subnet is a subset of the a range of IPv4 addresses used by the VPC and is specific to a given availability zone in the VPC’s region. This ID number is preceded by "subnet-", for instance: "subnet-04ccf456919e69055". For more information, see VPCs and Subnets .
(string) --
Mode (string) --
Indicates the mode in which the specific classifier was trained. This also indicates the format of input documents and the format of the confusion matrix. Each classifier can only be trained in one mode and this cannot be changed once the classifier is trained.
ModelKmsKeyId (string) --
ID for the AWS Key Management Service (KMS) key that Amazon Comprehend uses to encrypt trained custom models. The ModelKmsKeyId can be either of the following formats:
KMS Key ID: "1234abcd-12ab-34cd-56ef-1234567890ab"
Amazon Resource Name (ARN) of a KMS Key: "arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab"
VersionName (string) --
The version name that you assigned to the document classifier.
SourceModelArn (string) --
The Amazon Resource Name (ARN) of the source model. This model was imported from a different AWS account to create the document classifier model in your AWS account.
FlywheelArn (string) --
The Amazon Resource Number (ARN) of the flywheel
NextToken (string) --
Identifies the next page of results to return.
{'EndpointPropertiesList': {'FlywheelArn': 'string'}}
Gets a list of all existing endpoints that you've created. For information about endpoints, see Managing endpoints .
See also: AWS API Documentation
Request Syntax
client.list_endpoints(
Filter={
'ModelArn': 'string',
'Status': 'CREATING'|'DELETING'|'FAILED'|'IN_SERVICE'|'UPDATING',
'CreationTimeBefore': datetime(2015, 1, 1),
'CreationTimeAfter': datetime(2015, 1, 1)
},
NextToken='string',
MaxResults=123
)
dict
Filters the endpoints that are returned. You can filter endpoints on their name, model, status, or the date and time that they were created. You can only set one filter at a time.
ModelArn (string) --
The Amazon Resource Number (ARN) of the model to which the endpoint is attached.
Status (string) --
Specifies the status of the endpoint being returned. Possible values are: Creating, Ready, Updating, Deleting, Failed.
CreationTimeBefore (datetime) --
Specifies a date before which the returned endpoint or endpoints were created.
CreationTimeAfter (datetime) --
Specifies a date after which the returned endpoint or endpoints were created.
string
Identifies the next page of results to return.
integer
The maximum number of results to return in each page. The default is 100.
dict
Response Syntax
{
'EndpointPropertiesList': [
{
'EndpointArn': 'string',
'Status': 'CREATING'|'DELETING'|'FAILED'|'IN_SERVICE'|'UPDATING',
'Message': 'string',
'ModelArn': 'string',
'DesiredModelArn': 'string',
'DesiredInferenceUnits': 123,
'CurrentInferenceUnits': 123,
'CreationTime': datetime(2015, 1, 1),
'LastModifiedTime': datetime(2015, 1, 1),
'DataAccessRoleArn': 'string',
'DesiredDataAccessRoleArn': 'string',
'FlywheelArn': 'string'
},
],
'NextToken': 'string'
}
Response Structure
(dict) --
EndpointPropertiesList (list) --
Displays a list of endpoint properties being retrieved by the service in response to the request.
(dict) --
Specifies information about the specified endpoint. For information about endpoints, see Managing endpoints .
EndpointArn (string) --
The Amazon Resource Number (ARN) of the endpoint.
Status (string) --
Specifies the status of the endpoint. Because the endpoint updates and creation are asynchronous, so customers will need to wait for the endpoint to be Ready status before making inference requests.
Message (string) --
Specifies a reason for failure in cases of Failed status.
ModelArn (string) --
The Amazon Resource Number (ARN) of the model to which the endpoint is attached.
DesiredModelArn (string) --
ARN of the new model to use for updating an existing endpoint. This ARN is going to be different from the model ARN when the update is in progress
DesiredInferenceUnits (integer) --
The desired number of inference units to be used by the model using this endpoint. Each inference unit represents of a throughput of 100 characters per second.
CurrentInferenceUnits (integer) --
The number of inference units currently used by the model using this endpoint.
CreationTime (datetime) --
The creation date and time of the endpoint.
LastModifiedTime (datetime) --
The date and time that the endpoint was last modified.
DataAccessRoleArn (string) --
The Amazon Resource Name (ARN) of the AWS Identity and Access Management (IAM) role that grants Amazon Comprehend read access to trained custom models encrypted with a customer managed key (ModelKmsKeyId).
DesiredDataAccessRoleArn (string) --
Data access role ARN to use in case the new model is encrypted with a customer KMS key.
FlywheelArn (string) --
The Amazon Resource Number (ARN) of the flywheel
NextToken (string) --
Identifies the next page of results to return.
{'EntityRecognizerPropertiesList': {'FlywheelArn': 'string',
'OutputDataConfig': {'FlywheelStatsS3Prefix': 'string'}}}
Gets a list of the properties of all entity recognizers that you created, including recognizers currently in training. Allows you to filter the list of recognizers based on criteria such as status and submission time. This call returns up to 500 entity recognizers in the list, with a default number of 100 recognizers in the list.
The results of this list are not in any particular order. Please get the list and sort locally if needed.
See also: AWS API Documentation
Request Syntax
client.list_entity_recognizers(
Filter={
'Status': 'SUBMITTED'|'TRAINING'|'DELETING'|'STOP_REQUESTED'|'STOPPED'|'IN_ERROR'|'TRAINED',
'RecognizerName': 'string',
'SubmitTimeBefore': datetime(2015, 1, 1),
'SubmitTimeAfter': datetime(2015, 1, 1)
},
NextToken='string',
MaxResults=123
)
dict
Filters the list of entities returned. You can filter on Status , SubmitTimeBefore , or SubmitTimeAfter . You can only set one filter at a time.
Status (string) --
The status of an entity recognizer.
RecognizerName (string) --
The name that you assigned the entity recognizer.
SubmitTimeBefore (datetime) --
Filters the list of entities based on the time that the list was submitted for processing. Returns only jobs submitted before the specified time. Jobs are returned in descending order, newest to oldest.
SubmitTimeAfter (datetime) --
Filters the list of entities based on the time that the list was submitted for processing. Returns only jobs submitted after the specified time. Jobs are returned in ascending order, oldest to newest.
string
Identifies the next page of results to return.
integer
The maximum number of results to return on each page. The default is 100.
dict
Response Syntax
{
'EntityRecognizerPropertiesList': [
{
'EntityRecognizerArn': 'string',
'LanguageCode': 'en'|'es'|'fr'|'de'|'it'|'pt'|'ar'|'hi'|'ja'|'ko'|'zh'|'zh-TW',
'Status': 'SUBMITTED'|'TRAINING'|'DELETING'|'STOP_REQUESTED'|'STOPPED'|'IN_ERROR'|'TRAINED',
'Message': 'string',
'SubmitTime': datetime(2015, 1, 1),
'EndTime': datetime(2015, 1, 1),
'TrainingStartTime': datetime(2015, 1, 1),
'TrainingEndTime': datetime(2015, 1, 1),
'InputDataConfig': {
'DataFormat': 'COMPREHEND_CSV'|'AUGMENTED_MANIFEST',
'EntityTypes': [
{
'Type': 'string'
},
],
'Documents': {
'S3Uri': 'string',
'TestS3Uri': 'string',
'InputFormat': 'ONE_DOC_PER_FILE'|'ONE_DOC_PER_LINE'
},
'Annotations': {
'S3Uri': 'string',
'TestS3Uri': 'string'
},
'EntityList': {
'S3Uri': 'string'
},
'AugmentedManifests': [
{
'S3Uri': 'string',
'Split': 'TRAIN'|'TEST',
'AttributeNames': [
'string',
],
'AnnotationDataS3Uri': 'string',
'SourceDocumentsS3Uri': 'string',
'DocumentType': 'PLAIN_TEXT_DOCUMENT'|'SEMI_STRUCTURED_DOCUMENT'
},
]
},
'RecognizerMetadata': {
'NumberOfTrainedDocuments': 123,
'NumberOfTestDocuments': 123,
'EvaluationMetrics': {
'Precision': 123.0,
'Recall': 123.0,
'F1Score': 123.0
},
'EntityTypes': [
{
'Type': 'string',
'EvaluationMetrics': {
'Precision': 123.0,
'Recall': 123.0,
'F1Score': 123.0
},
'NumberOfTrainMentions': 123
},
]
},
'DataAccessRoleArn': 'string',
'VolumeKmsKeyId': 'string',
'VpcConfig': {
'SecurityGroupIds': [
'string',
],
'Subnets': [
'string',
]
},
'ModelKmsKeyId': 'string',
'VersionName': 'string',
'SourceModelArn': 'string',
'FlywheelArn': 'string',
'OutputDataConfig': {
'FlywheelStatsS3Prefix': 'string'
}
},
],
'NextToken': 'string'
}
Response Structure
(dict) --
EntityRecognizerPropertiesList (list) --
The list of properties of an entity recognizer.
(dict) --
Describes information about an entity recognizer.
EntityRecognizerArn (string) --
The Amazon Resource Name (ARN) that identifies the entity recognizer.
LanguageCode (string) --
The language of the input documents. All documents must be in the same language. Only English ("en") is currently supported.
Status (string) --
Provides the status of the entity recognizer.
Message (string) --
A description of the status of the recognizer.
SubmitTime (datetime) --
The time that the recognizer was submitted for processing.
EndTime (datetime) --
The time that the recognizer creation completed.
TrainingStartTime (datetime) --
The time that training of the entity recognizer started.
TrainingEndTime (datetime) --
The time that training of the entity recognizer was completed.
InputDataConfig (dict) --
The input data properties of an entity recognizer.
DataFormat (string) --
The format of your training data:
COMPREHEND_CSV : A CSV file that supplements your training documents. The CSV file contains information about the custom entities that your trained model will detect. The required format of the file depends on whether you are providing annotations or an entity list. If you use this value, you must provide your CSV file by using either the Annotations or EntityList parameters. You must provide your training documents by using the Documents parameter.
AUGMENTED_MANIFEST : A labeled dataset that is produced by Amazon SageMaker Ground Truth. This file is in JSON lines format. Each line is a complete JSON object that contains a training document and its labels. Each label annotates a named entity in the training document. If you use this value, you must provide the AugmentedManifests parameter in your request.
If you don't specify a value, Amazon Comprehend uses COMPREHEND_CSV as the default.
EntityTypes (list) --
The entity types in the labeled training data that Amazon Comprehend uses to train the custom entity recognizer. Any entity types that you don't specify are ignored.
A maximum of 25 entity types can be used at one time to train an entity recognizer. Entity types must not contain the following invalid characters: n (line break), \n (escaped line break), r (carriage return), \r (escaped carriage return), t (tab), \t (escaped tab), space, and , (comma).
(dict) --
An entity type within a labeled training dataset that Amazon Comprehend uses to train a custom entity recognizer.
Type (string) --
An entity type within a labeled training dataset that Amazon Comprehend uses to train a custom entity recognizer.
Entity types must not contain the following invalid characters: n (line break), \n (escaped line break, r (carriage return), \r (escaped carriage return), t (tab), \t (escaped tab), space, and , (comma).
Documents (dict) --
The S3 location of the folder that contains the training documents for your custom entity recognizer.
This parameter is required if you set DataFormat to COMPREHEND_CSV .
S3Uri (string) --
Specifies the Amazon S3 location where the training documents for an entity recognizer are located. The URI must be in the same region as the API endpoint that you are calling.
TestS3Uri (string) --
Specifies the Amazon S3 location where the test documents for an entity recognizer are located. The URI must be in the same AWS Region as the API endpoint that you are calling.
InputFormat (string) --
Specifies how the text in an input file should be processed. This is optional, and the default is ONE_DOC_PER_LINE. ONE_DOC_PER_FILE - Each file is considered a separate document. Use this option when you are processing large documents, such as newspaper articles or scientific papers. ONE_DOC_PER_LINE - Each line in a file is considered a separate document. Use this option when you are processing many short documents, such as text messages.
Annotations (dict) --
The S3 location of the CSV file that annotates your training documents.
S3Uri (string) --
Specifies the Amazon S3 location where the annotations for an entity recognizer are located. The URI must be in the same region as the API endpoint that you are calling.
TestS3Uri (string) --
Specifies the Amazon S3 location where the test annotations for an entity recognizer are located. The URI must be in the same region as the API endpoint that you are calling.
EntityList (dict) --
The S3 location of the CSV file that has the entity list for your custom entity recognizer.
S3Uri (string) --
Specifies the Amazon S3 location where the entity list is located. The URI must be in the same region as the API endpoint that you are calling.
AugmentedManifests (list) --
A list of augmented manifest files that provide training data for your custom model. An augmented manifest file is a labeled dataset that is produced by Amazon SageMaker Ground Truth.
This parameter is required if you set DataFormat to AUGMENTED_MANIFEST .
(dict) --
An augmented manifest file that provides training data for your custom model. An augmented manifest file is a labeled dataset that is produced by Amazon SageMaker Ground Truth.
S3Uri (string) --
The Amazon S3 location of the augmented manifest file.
Split (string) --
The purpose of the data you've provided in the augmented manifest. You can either train or test this data. If you don't specify, the default is train.
TRAIN - all of the documents in the manifest will be used for training. If no test documents are provided, Amazon Comprehend will automatically reserve a portion of the training documents for testing.
TEST - all of the documents in the manifest will be used for testing.
AttributeNames (list) --
The JSON attribute that contains the annotations for your training documents. The number of attribute names that you specify depends on whether your augmented manifest file is the output of a single labeling job or a chained labeling job.
If your file is the output of a single labeling job, specify the LabelAttributeName key that was used when the job was created in Ground Truth.
If your file is the output of a chained labeling job, specify the LabelAttributeName key for one or more jobs in the chain. Each LabelAttributeName key provides the annotations from an individual job.
(string) --
AnnotationDataS3Uri (string) --
The S3 prefix to the annotation files that are referred in the augmented manifest file.
SourceDocumentsS3Uri (string) --
The S3 prefix to the source files (PDFs) that are referred to in the augmented manifest file.
DocumentType (string) --
The type of augmented manifest. PlainTextDocument or SemiStructuredDocument. If you don't specify, the default is PlainTextDocument.
PLAIN_TEXT_DOCUMENT A document type that represents any unicode text that is encoded in UTF-8.
SEMI_STRUCTURED_DOCUMENT A document type with positional and structural context, like a PDF. For training with Amazon Comprehend, only PDFs are supported. For inference, Amazon Comprehend support PDFs, DOCX and TXT.
RecognizerMetadata (dict) --
Provides information about an entity recognizer.
NumberOfTrainedDocuments (integer) --
The number of documents in the input data that were used to train the entity recognizer. Typically this is 80 to 90 percent of the input documents.
NumberOfTestDocuments (integer) --
The number of documents in the input data that were used to test the entity recognizer. Typically this is 10 to 20 percent of the input documents.
EvaluationMetrics (dict) --
Detailed information about the accuracy of an entity recognizer.
Precision (float) --
A measure of the usefulness of the recognizer results in the test data. High precision means that the recognizer returned substantially more relevant results than irrelevant ones.
Recall (float) --
A measure of how complete the recognizer results are for the test data. High recall means that the recognizer returned most of the relevant results.
F1Score (float) --
A measure of how accurate the recognizer results are for the test data. It is derived from the Precision and Recall values. The F1Score is the harmonic average of the two scores. For plain text entity recognizer models, the range is 0 to 100, where 100 is the best score. For PDF/Word entity recognizer models, the range is 0 to 1, where 1 is the best score.
EntityTypes (list) --
Entity types from the metadata of an entity recognizer.
(dict) --
Individual item from the list of entity types in the metadata of an entity recognizer.
Type (string) --
Type of entity from the list of entity types in the metadata of an entity recognizer.
EvaluationMetrics (dict) --
Detailed information about the accuracy of the entity recognizer for a specific item on the list of entity types.
Precision (float) --
A measure of the usefulness of the recognizer results for a specific entity type in the test data. High precision means that the recognizer returned substantially more relevant results than irrelevant ones.
Recall (float) --
A measure of how complete the recognizer results are for a specific entity type in the test data. High recall means that the recognizer returned most of the relevant results.
F1Score (float) --
A measure of how accurate the recognizer results are for a specific entity type in the test data. It is derived from the Precision and Recall values. The F1Score is the harmonic average of the two scores. The highest score is 1, and the worst score is 0.
NumberOfTrainMentions (integer) --
Indicates the number of times the given entity type was seen in the training data.
DataAccessRoleArn (string) --
The Amazon Resource Name (ARN) of the AWS Identity and Access Management (IAM) role that grants Amazon Comprehend read access to your input data.
VolumeKmsKeyId (string) --
ID for the AWS Key Management Service (KMS) key that Amazon Comprehend uses to encrypt data on the storage volume attached to the ML compute instance(s) that process the analysis job. The VolumeKmsKeyId can be either of the following formats:
KMS Key ID: "1234abcd-12ab-34cd-56ef-1234567890ab"
Amazon Resource Name (ARN) of a KMS Key: "arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab"
VpcConfig (dict) --
Configuration parameters for a private Virtual Private Cloud (VPC) containing the resources you are using for your custom entity recognizer. For more information, see Amazon VPC .
SecurityGroupIds (list) --
The ID number for a security group on an instance of your private VPC. Security groups on your VPC function serve as a virtual firewall to control inbound and outbound traffic and provides security for the resources that you’ll be accessing on the VPC. This ID number is preceded by "sg-", for instance: "sg-03b388029b0a285ea". For more information, see Security Groups for your VPC .
(string) --
Subnets (list) --
The ID for each subnet being used in your private VPC. This subnet is a subset of the a range of IPv4 addresses used by the VPC and is specific to a given availability zone in the VPC’s region. This ID number is preceded by "subnet-", for instance: "subnet-04ccf456919e69055". For more information, see VPCs and Subnets .
(string) --
ModelKmsKeyId (string) --
ID for the AWS Key Management Service (KMS) key that Amazon Comprehend uses to encrypt trained custom models. The ModelKmsKeyId can be either of the following formats:
KMS Key ID: "1234abcd-12ab-34cd-56ef-1234567890ab"
Amazon Resource Name (ARN) of a KMS Key: "arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab"
VersionName (string) --
The version name you assigned to the entity recognizer.
SourceModelArn (string) --
The Amazon Resource Name (ARN) of the source model. This model was imported from a different AWS account to create the entity recognizer model in your AWS account.
FlywheelArn (string) --
The Amazon Resource Number (ARN) of the flywheel
OutputDataConfig (dict) --
Output data configuration.
FlywheelStatsS3Prefix (string) --
The Amazon S3 prefix for the data lake location of the flywheel statistics.
NextToken (string) --
Identifies the next page of results to return.
{'FlywheelArn': 'string'}
Response {'DocumentClassifierArn': 'string'}
Starts an asynchronous document classification job. Use the DescribeDocumentClassificationJob operation to track the progress of the job.
See also: AWS API Documentation
Request Syntax
client.start_document_classification_job(
JobName='string',
DocumentClassifierArn='string',
InputDataConfig={
'S3Uri': 'string',
'InputFormat': 'ONE_DOC_PER_FILE'|'ONE_DOC_PER_LINE',
'DocumentReaderConfig': {
'DocumentReadAction': 'TEXTRACT_DETECT_DOCUMENT_TEXT'|'TEXTRACT_ANALYZE_DOCUMENT',
'DocumentReadMode': 'SERVICE_DEFAULT'|'FORCE_DOCUMENT_READ_ACTION',
'FeatureTypes': [
'TABLES'|'FORMS',
]
}
},
OutputDataConfig={
'S3Uri': 'string',
'KmsKeyId': 'string'
},
DataAccessRoleArn='string',
ClientRequestToken='string',
VolumeKmsKeyId='string',
VpcConfig={
'SecurityGroupIds': [
'string',
],
'Subnets': [
'string',
]
},
Tags=[
{
'Key': 'string',
'Value': 'string'
},
],
FlywheelArn='string'
)
string
The identifier of the job.
string
The Amazon Resource Name (ARN) of the document classifier to use to process the job.
dict
[REQUIRED]
Specifies the format and location of the input data for the job.
S3Uri (string) -- [REQUIRED]
The Amazon S3 URI for the input data. The URI must be in same region as the API endpoint that you are calling. The URI can point to a single input file or it can provide the prefix for a collection of data files.
For example, if you use the URI S3://bucketName/prefix , if the prefix is a single file, Amazon Comprehend uses that file as input. If more than one file begins with the prefix, Amazon Comprehend uses all of them as input.
InputFormat (string) --
Specifies how the text in an input file should be processed:
ONE_DOC_PER_FILE - Each file is considered a separate document. Use this option when you are processing large documents, such as newspaper articles or scientific papers.
ONE_DOC_PER_LINE - Each line in a file is considered a separate document. Use this option when you are processing many short documents, such as text messages.
DocumentReaderConfig (dict) --
Provides configuration parameters to override the default actions for extracting text from PDF documents and image files.
DocumentReadAction (string) -- [REQUIRED]
This field defines the Amazon Textract API operation that Amazon Comprehend uses to extract text from PDF files and image files. Enter one of the following values:
TEXTRACT_DETECT_DOCUMENT_TEXT - The Amazon Comprehend service uses the DetectDocumentText API operation.
TEXTRACT_ANALYZE_DOCUMENT - The Amazon Comprehend service uses the AnalyzeDocument API operation.
DocumentReadMode (string) --
Determines the text extraction actions for PDF files. Enter one of the following values:
SERVICE_DEFAULT - use the Amazon Comprehend service defaults for PDF files.
FORCE_DOCUMENT_READ_ACTION - Amazon Comprehend uses the Textract API specified by DocumentReadAction for all PDF files, including digital PDF files.
FeatureTypes (list) --
Specifies the type of Amazon Textract features to apply. If you chose TEXTRACT_ANALYZE_DOCUMENT as the read action, you must specify one or both of the following values:
TABLES - Returns information about any tables that are detected in the input document.
FORMS - Returns information and the data from any forms that are detected in the input document.
(string) --
Specifies the type of Amazon Textract features to apply. If you chose TEXTRACT_ANALYZE_DOCUMENT as the read action, you must specify one or both of the following values:
TABLES - Returns additional information about any tables that are detected in the input document.
FORMS - Returns additional information about any forms that are detected in the input document.
dict
[REQUIRED]
Specifies where to send the output files.
S3Uri (string) -- [REQUIRED]
When you use the OutputDataConfig object with asynchronous operations, you specify the Amazon S3 location where you want to write the output data. The URI must be in the same region as the API endpoint that you are calling. The location is used as the prefix for the actual location of the output file.
When the topic detection job is finished, the service creates an output file in a directory specific to the job. The S3Uri field contains the location of the output file, called output.tar.gz . It is a compressed archive that contains the ouput of the operation.
For a PII entity detection job, the output file is plain text, not a compressed archive. The output file name is the same as the input file, with .out appended at the end.
KmsKeyId (string) --
ID for the AWS Key Management Service (KMS) key that Amazon Comprehend uses to encrypt the output results from an analysis job. The KmsKeyId can be one of the following formats:
KMS Key ID: "1234abcd-12ab-34cd-56ef-1234567890ab"
Amazon Resource Name (ARN) of a KMS Key: "arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab"
KMS Key Alias: "alias/ExampleAlias"
ARN of a KMS Key Alias: "arn:aws:kms:us-west-2:111122223333:alias/ExampleAlias"
string
[REQUIRED]
The Amazon Resource Name (ARN) of the AWS Identity and Access Management (IAM) role that grants Amazon Comprehend read access to your input data.
string
A unique identifier for the request. If you do not set the client request token, Amazon Comprehend generates one.
This field is autopopulated if not provided.
string
ID for the AWS Key Management Service (KMS) key that Amazon Comprehend uses to encrypt data on the storage volume attached to the ML compute instance(s) that process the analysis job. The VolumeKmsKeyId can be either of the following formats:
KMS Key ID: "1234abcd-12ab-34cd-56ef-1234567890ab"
Amazon Resource Name (ARN) of a KMS Key: "arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab"
dict
Configuration parameters for an optional private Virtual Private Cloud (VPC) containing the resources you are using for your document classification job. For more information, see Amazon VPC .
SecurityGroupIds (list) -- [REQUIRED]
The ID number for a security group on an instance of your private VPC. Security groups on your VPC function serve as a virtual firewall to control inbound and outbound traffic and provides security for the resources that you’ll be accessing on the VPC. This ID number is preceded by "sg-", for instance: "sg-03b388029b0a285ea". For more information, see Security Groups for your VPC .
(string) --
Subnets (list) -- [REQUIRED]
The ID for each subnet being used in your private VPC. This subnet is a subset of the a range of IPv4 addresses used by the VPC and is specific to a given availability zone in the VPC’s region. This ID number is preceded by "subnet-", for instance: "subnet-04ccf456919e69055". For more information, see VPCs and Subnets .
(string) --
list
Tags to associate with the document classification job. A tag is a key-value pair that adds metadata to a resource used by Amazon Comprehend. For example, a tag with "Sales" as the key might be added to a resource to indicate its use by the sales department.
(dict) --
A key-value pair that adds as a metadata to a resource used by Amazon Comprehend. For example, a tag with the key-value pair ‘Department’:’Sales’ might be added to a resource to indicate its use by a particular department.
Key (string) -- [REQUIRED]
The initial part of a key-value pair that forms a tag associated with a given resource. For instance, if you want to show which resources are used by which departments, you might use “Department” as the key portion of the pair, with multiple possible values such as “sales,” “legal,” and “administration.”
Value (string) --
The second part of a key-value pair that forms a tag associated with a given resource. For instance, if you want to show which resources are used by which departments, you might use “Department” as the initial (key) portion of the pair, with a value of “sales” to indicate the sales department.
string
The Amazon Resource Number (ARN) of the flywheel associated with the model to use.
dict
Response Syntax
{
'JobId': 'string',
'JobArn': 'string',
'JobStatus': 'SUBMITTED'|'IN_PROGRESS'|'COMPLETED'|'FAILED'|'STOP_REQUESTED'|'STOPPED',
'DocumentClassifierArn': 'string'
}
Response Structure
(dict) --
JobId (string) --
The identifier generated for the job. To get the status of the job, use this identifier with the DescribeDocumentClassificationJob operation.
JobArn (string) --
The Amazon Resource Name (ARN) of the document classification job. It is a unique, fully qualified identifier for the job. It includes the AWS account, Region, and the job ID. The format of the ARN is as follows:
arn:<partition>:comprehend:<region>:<account-id>:document-classification-job/<job-id>
The following is an example job ARN:
arn:aws:comprehend:us-west-2:111122223333:document-classification-job/1234abcd12ab34cd56ef1234567890ab
JobStatus (string) --
The status of the job:
SUBMITTED - The job has been received and queued for processing.
IN_PROGRESS - Amazon Comprehend is processing the job.
COMPLETED - The job was successfully completed and the output is available.
FAILED - The job did not complete. For details, use the DescribeDocumentClassificationJob operation.
STOP_REQUESTED - Amazon Comprehend has received a stop request for the job and is processing the request.
STOPPED - The job was successfully stopped without completing.
DocumentClassifierArn (string) --
The ARN of the custom classification model.
{'FlywheelArn': 'string'}
Response {'EntityRecognizerArn': 'string'}
Starts an asynchronous entity detection job for a collection of documents. Use the operation to track the status of a job.
This API can be used for either standard entity detection or custom entity recognition. In order to be used for custom entity recognition, the optional EntityRecognizerArn must be used in order to provide access to the recognizer being used to detect the custom entity.
See also: AWS API Documentation
Request Syntax
client.start_entities_detection_job(
InputDataConfig={
'S3Uri': 'string',
'InputFormat': 'ONE_DOC_PER_FILE'|'ONE_DOC_PER_LINE',
'DocumentReaderConfig': {
'DocumentReadAction': 'TEXTRACT_DETECT_DOCUMENT_TEXT'|'TEXTRACT_ANALYZE_DOCUMENT',
'DocumentReadMode': 'SERVICE_DEFAULT'|'FORCE_DOCUMENT_READ_ACTION',
'FeatureTypes': [
'TABLES'|'FORMS',
]
}
},
OutputDataConfig={
'S3Uri': 'string',
'KmsKeyId': 'string'
},
DataAccessRoleArn='string',
JobName='string',
EntityRecognizerArn='string',
LanguageCode='en'|'es'|'fr'|'de'|'it'|'pt'|'ar'|'hi'|'ja'|'ko'|'zh'|'zh-TW',
ClientRequestToken='string',
VolumeKmsKeyId='string',
VpcConfig={
'SecurityGroupIds': [
'string',
],
'Subnets': [
'string',
]
},
Tags=[
{
'Key': 'string',
'Value': 'string'
},
],
FlywheelArn='string'
)
dict
[REQUIRED]
Specifies the format and location of the input data for the job.
S3Uri (string) -- [REQUIRED]
The Amazon S3 URI for the input data. The URI must be in same region as the API endpoint that you are calling. The URI can point to a single input file or it can provide the prefix for a collection of data files.
For example, if you use the URI S3://bucketName/prefix , if the prefix is a single file, Amazon Comprehend uses that file as input. If more than one file begins with the prefix, Amazon Comprehend uses all of them as input.
InputFormat (string) --
Specifies how the text in an input file should be processed:
ONE_DOC_PER_FILE - Each file is considered a separate document. Use this option when you are processing large documents, such as newspaper articles or scientific papers.
ONE_DOC_PER_LINE - Each line in a file is considered a separate document. Use this option when you are processing many short documents, such as text messages.
DocumentReaderConfig (dict) --
Provides configuration parameters to override the default actions for extracting text from PDF documents and image files.
DocumentReadAction (string) -- [REQUIRED]
This field defines the Amazon Textract API operation that Amazon Comprehend uses to extract text from PDF files and image files. Enter one of the following values:
TEXTRACT_DETECT_DOCUMENT_TEXT - The Amazon Comprehend service uses the DetectDocumentText API operation.
TEXTRACT_ANALYZE_DOCUMENT - The Amazon Comprehend service uses the AnalyzeDocument API operation.
DocumentReadMode (string) --
Determines the text extraction actions for PDF files. Enter one of the following values:
SERVICE_DEFAULT - use the Amazon Comprehend service defaults for PDF files.
FORCE_DOCUMENT_READ_ACTION - Amazon Comprehend uses the Textract API specified by DocumentReadAction for all PDF files, including digital PDF files.
FeatureTypes (list) --
Specifies the type of Amazon Textract features to apply. If you chose TEXTRACT_ANALYZE_DOCUMENT as the read action, you must specify one or both of the following values:
TABLES - Returns information about any tables that are detected in the input document.
FORMS - Returns information and the data from any forms that are detected in the input document.
(string) --
Specifies the type of Amazon Textract features to apply. If you chose TEXTRACT_ANALYZE_DOCUMENT as the read action, you must specify one or both of the following values:
TABLES - Returns additional information about any tables that are detected in the input document.
FORMS - Returns additional information about any forms that are detected in the input document.
dict
[REQUIRED]
Specifies where to send the output files.
S3Uri (string) -- [REQUIRED]
When you use the OutputDataConfig object with asynchronous operations, you specify the Amazon S3 location where you want to write the output data. The URI must be in the same region as the API endpoint that you are calling. The location is used as the prefix for the actual location of the output file.
When the topic detection job is finished, the service creates an output file in a directory specific to the job. The S3Uri field contains the location of the output file, called output.tar.gz . It is a compressed archive that contains the ouput of the operation.
For a PII entity detection job, the output file is plain text, not a compressed archive. The output file name is the same as the input file, with .out appended at the end.
KmsKeyId (string) --
ID for the AWS Key Management Service (KMS) key that Amazon Comprehend uses to encrypt the output results from an analysis job. The KmsKeyId can be one of the following formats:
KMS Key ID: "1234abcd-12ab-34cd-56ef-1234567890ab"
Amazon Resource Name (ARN) of a KMS Key: "arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab"
KMS Key Alias: "alias/ExampleAlias"
ARN of a KMS Key Alias: "arn:aws:kms:us-west-2:111122223333:alias/ExampleAlias"
string
[REQUIRED]
The Amazon Resource Name (ARN) of the AWS Identity and Access Management (IAM) role that grants Amazon Comprehend read access to your input data. For more information, see https://docs.aws.amazon.com/comprehend/latest/dg/access-control-managing-permissions.html#auth-role-permissions .
string
The identifier of the job.
string
The Amazon Resource Name (ARN) that identifies the specific entity recognizer to be used by the StartEntitiesDetectionJob . This ARN is optional and is only used for a custom entity recognition job.
string
[REQUIRED]
The language of the input documents. All documents must be in the same language. You can specify any of the languages supported by Amazon Comprehend. If custom entities recognition is used, this parameter is ignored and the language used for training the model is used instead.
string
A unique identifier for the request. If you don't set the client request token, Amazon Comprehend generates one.
This field is autopopulated if not provided.
string
ID for the AWS Key Management Service (KMS) key that Amazon Comprehend uses to encrypt data on the storage volume attached to the ML compute instance(s) that process the analysis job. The VolumeKmsKeyId can be either of the following formats:
KMS Key ID: "1234abcd-12ab-34cd-56ef-1234567890ab"
Amazon Resource Name (ARN) of a KMS Key: "arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab"
dict
Configuration parameters for an optional private Virtual Private Cloud (VPC) containing the resources you are using for your entity detection job. For more information, see Amazon VPC .
SecurityGroupIds (list) -- [REQUIRED]
The ID number for a security group on an instance of your private VPC. Security groups on your VPC function serve as a virtual firewall to control inbound and outbound traffic and provides security for the resources that you’ll be accessing on the VPC. This ID number is preceded by "sg-", for instance: "sg-03b388029b0a285ea". For more information, see Security Groups for your VPC .
(string) --
Subnets (list) -- [REQUIRED]
The ID for each subnet being used in your private VPC. This subnet is a subset of the a range of IPv4 addresses used by the VPC and is specific to a given availability zone in the VPC’s region. This ID number is preceded by "subnet-", for instance: "subnet-04ccf456919e69055". For more information, see VPCs and Subnets .
(string) --
list
Tags to associate with the entities detection job. A tag is a key-value pair that adds metadata to a resource used by Amazon Comprehend. For example, a tag with "Sales" as the key might be added to a resource to indicate its use by the sales department.
(dict) --
A key-value pair that adds as a metadata to a resource used by Amazon Comprehend. For example, a tag with the key-value pair ‘Department’:’Sales’ might be added to a resource to indicate its use by a particular department.
Key (string) -- [REQUIRED]
The initial part of a key-value pair that forms a tag associated with a given resource. For instance, if you want to show which resources are used by which departments, you might use “Department” as the key portion of the pair, with multiple possible values such as “sales,” “legal,” and “administration.”
Value (string) --
The second part of a key-value pair that forms a tag associated with a given resource. For instance, if you want to show which resources are used by which departments, you might use “Department” as the initial (key) portion of the pair, with a value of “sales” to indicate the sales department.
string
The Amazon Resource Number (ARN) of the flywheel associated with the model to use.
dict
Response Syntax
{
'JobId': 'string',
'JobArn': 'string',
'JobStatus': 'SUBMITTED'|'IN_PROGRESS'|'COMPLETED'|'FAILED'|'STOP_REQUESTED'|'STOPPED',
'EntityRecognizerArn': 'string'
}
Response Structure
(dict) --
JobId (string) --
The identifier generated for the job. To get the status of job, use this identifier with the operation.
JobArn (string) --
The Amazon Resource Name (ARN) of the entities detection job. It is a unique, fully qualified identifier for the job. It includes the AWS account, Region, and the job ID. The format of the ARN is as follows:
arn:<partition>:comprehend:<region>:<account-id>:entities-detection-job/<job-id>
The following is an example job ARN:
arn:aws:comprehend:us-west-2:111122223333:entities-detection-job/1234abcd12ab34cd56ef1234567890ab
JobStatus (string) --
The status of the job.
SUBMITTED - The job has been received and is queued for processing.
IN_PROGRESS - Amazon Comprehend is processing the job.
COMPLETED - The job was successfully completed and the output is available.
FAILED - The job did not complete. To get details, use the operation.
STOP_REQUESTED - Amazon Comprehend has received a stop request for the job and is processing the request.
STOPPED - The job was successfully stopped without completing.
EntityRecognizerArn (string) --
The ARN of the custom entity recognition model.
{'FlywheelArn': 'string'}
Response {'DesiredModelArn': 'string'}
Updates information about the specified endpoint. For information about endpoints, see Managing endpoints .
See also: AWS API Documentation
Request Syntax
client.update_endpoint(
EndpointArn='string',
DesiredModelArn='string',
DesiredInferenceUnits=123,
DesiredDataAccessRoleArn='string',
FlywheelArn='string'
)
string
[REQUIRED]
The Amazon Resource Number (ARN) of the endpoint being updated.
string
The ARN of the new model to use when updating an existing endpoint.
integer
The desired number of inference units to be used by the model using this endpoint. Each inference unit represents of a throughput of 100 characters per second.
string
Data access role ARN to use in case the new model is encrypted with a customer CMK.
string
The Amazon Resource Number (ARN) of the flywheel
dict
Response Syntax
{
'DesiredModelArn': 'string'
}
Response Structure
(dict) --
DesiredModelArn (string) --
The Amazon Resource Number (ARN) of the new model.