2018/11/16 - Amazon Comprehend - 4 new 3 updated api methods
Changes Amazon Comprehend Custom Entities automatically trains entity recognition models using your entities and noun-based phrases.
Creates an entity recognizer using submitted files. After your CreateEntityRecognizer request is submitted, you can check job status using the API.
See also: AWS API Documentation
Request Syntax
client.create_entity_recognizer(
RecognizerName='string',
DataAccessRoleArn='string',
InputDataConfig={
'EntityTypes': [
{
'Type': 'string'
},
],
'Documents': {
'S3Uri': 'string'
},
'Annotations': {
'S3Uri': 'string'
},
'EntityList': {
'S3Uri': 'string'
}
},
ClientRequestToken='string',
LanguageCode='en'|'es'|'fr'|'de'|'it'|'pt'
)
string
[REQUIRED]
The name given to the newly created recognizer. Recognizer names can be a maximum of 256 characters. Alphanumeric characters, hyphens (-) and underscores (_) are allowed. The name must be unique in the account/region.
string
[REQUIRED]
The Amazon Resource Name (ARN) of the AWS Identity and Management (IAM) role that grants Amazon Comprehend read access to your input data.
dict
[REQUIRED]
Specifies the format and location of the input data. The S3 bucket containing the input data must be located in the same region as the entity recognizer being created.
EntityTypes (list) -- [REQUIRED]
The entity types in the input data for an entity recognizer.
(dict) --
Information about an individual item on a list of entity types.
Type (string) -- [REQUIRED]
Entity type of an item on an entity type list.
Documents (dict) -- [REQUIRED]
S3 location of the documents folder for an entity recognizer
S3Uri (string) -- [REQUIRED]
Specifies the Amazon S3 location where the training documents for an entity recognizer are located. The URI must be in the same region as the API endpoint that you are calling.
Annotations (dict) --
S3 location of the annotations file for an entity recognizer.
S3Uri (string) -- [REQUIRED]
Specifies the Amazon S3 location where the annotations for an entity recognizer are located. The URI must be in the same region as the API endpoint that you are calling.
EntityList (dict) --
S3 location of the entity list for an entity recognizer.
S3Uri (string) -- [REQUIRED]
Specifies the Amazon S3 location where the entity list is located. The URI must be in the same region as the API endpoint that you are calling.
string
A unique identifier for the request. If you don't set the client request token, Amazon Comprehend generates one.
This field is autopopulated if not provided.
string
[REQUIRED]
The language of the input documents. All documents must be in the same language. Only English ("en") is currently supported.
dict
Response Syntax
{
'EntityRecognizerArn': 'string'
}
Response Structure
(dict) --
EntityRecognizerArn (string) --
The Amazon Resource Name (ARN) that identifies the entity recognizer.
Provides details about an entity recognizer including status, S3 buckets containing training data, recognizer metadata, metrics, and so on.
See also: AWS API Documentation
Request Syntax
client.describe_entity_recognizer(
EntityRecognizerArn='string'
)
string
[REQUIRED]
The Amazon Resource Name (ARN) that identifies the entity recognizer.
dict
Response Syntax
{
'EntityRecognizerProperties': {
'EntityRecognizerArn': 'string',
'LanguageCode': 'en'|'es'|'fr'|'de'|'it'|'pt',
'Status': 'SUBMITTED'|'TRAINING'|'DELETING'|'IN_ERROR'|'TRAINED',
'Message': 'string',
'SubmitTime': datetime(2015, 1, 1),
'EndTime': datetime(2015, 1, 1),
'TrainingStartTime': datetime(2015, 1, 1),
'TrainingEndTime': datetime(2015, 1, 1),
'InputDataConfig': {
'EntityTypes': [
{
'Type': 'string'
},
],
'Documents': {
'S3Uri': 'string'
},
'Annotations': {
'S3Uri': 'string'
},
'EntityList': {
'S3Uri': 'string'
}
},
'RecognizerMetadata': {
'NumberOfTrainedDocuments': 123,
'NumberOfTestDocuments': 123,
'EvaluationMetrics': {
'Precision': 123.0,
'Recall': 123.0,
'F1Score': 123.0
},
'EntityTypes': [
{
'Type': 'string'
},
]
},
'DataAccessRoleArn': 'string'
}
}
Response Structure
(dict) --
EntityRecognizerProperties (dict) --
Describes information associated with an entity recognizer.
EntityRecognizerArn (string) --
The Amazon Resource Name (ARN) that identifies the entity recognizer.
LanguageCode (string) --
The language of the input documents. All documents must be in the same language. Only English ("en") is currently supported.
Status (string) --
Provides the status of the entity recognizer.
Message (string) --
A description of the status of the recognizer.
SubmitTime (datetime) --
The time that the recognizer was submitted for processing.
EndTime (datetime) --
The time that the recognizer creation completed.
TrainingStartTime (datetime) --
The time that training of the entity recognizer started.
TrainingEndTime (datetime) --
The time that training of the entity recognizer was completed.
InputDataConfig (dict) --
The input data properties of an entity recognizer.
EntityTypes (list) --
The entity types in the input data for an entity recognizer.
(dict) --
Information about an individual item on a list of entity types.
Type (string) --
Entity type of an item on an entity type list.
Documents (dict) --
S3 location of the documents folder for an entity recognizer
S3Uri (string) --
Specifies the Amazon S3 location where the training documents for an entity recognizer are located. The URI must be in the same region as the API endpoint that you are calling.
Annotations (dict) --
S3 location of the annotations file for an entity recognizer.
S3Uri (string) --
Specifies the Amazon S3 location where the annotations for an entity recognizer are located. The URI must be in the same region as the API endpoint that you are calling.
EntityList (dict) --
S3 location of the entity list for an entity recognizer.
S3Uri (string) --
Specifies the Amazon S3 location where the entity list is located. The URI must be in the same region as the API endpoint that you are calling.
RecognizerMetadata (dict) --
Provides information about an entity recognizer.
NumberOfTrainedDocuments (integer) --
The number of documents in the input data that were used to train the entity recognizer. Typically this is 80 to 90 percent of the input documents.
NumberOfTestDocuments (integer) --
The number of documents in the input data that were used to test the entity recognizer. Typically this is 10 to 20 percent of the input documents.
EvaluationMetrics (dict) --
Detailed information about the accuracy of an entity recognizer.
Precision (float) --
A measure of the usefulness of the recognizer results in the test data. High precision means that the recognizer returned substantially more relevant results than irrelevant ones.
Recall (float) --
A measure of how complete the recognizer results are for the test data. High recall means that the recognizer returned most of the relevant results.
F1Score (float) --
A measure of how accurate the recognizer results are for the test data. It is derived from the Precision and Recall values. The F1Score is the harmonic average of the two scores. The highest score is 1, and the worst score is 0.
EntityTypes (list) --
Entity types from the metadata of an entity recognizer.
(dict) --
Individual item from the list of entity types in the metadata of an entity recognizer.
Type (string) --
Type of entity from the list of entity types in the metadata of an entity recognizer.
DataAccessRoleArn (string) --
The Amazon Resource Name (ARN) of the AWS Identity and Management (IAM) role that grants Amazon Comprehend read access to your input data.
Deletes an entity recognizer.
Only those recognizers that are in terminated states (IN_ERROR, TRAINED) will be deleted. If an active inference job is using the model, a ResourceInUseException will be returned.
This is an asynchronous action that puts the recognizer into a DELETING state, and it is then removed by a background job. Once removed, the recognizer disappears from your account and is no longer available for use.
See also: AWS API Documentation
Request Syntax
client.delete_entity_recognizer(
EntityRecognizerArn='string'
)
string
[REQUIRED]
The Amazon Resource Name (ARN) that identifies the entity recognizer.
dict
Response Syntax
{}
Response Structure
(dict) --
Gets a list of the properties of all entity recognizers that you created, including recognizers currently in training. Allows you to filter the list of recognizers based on criteria such as status and submission time. This call returns up to 500 entity recognizers in the list, with a default number of 100 recognizers in the list.
The results of this list are not in any particular order. Please get the list and sort locally if needed.
See also: AWS API Documentation
Request Syntax
client.list_entity_recognizers(
Filter={
'Status': 'SUBMITTED'|'TRAINING'|'DELETING'|'IN_ERROR'|'TRAINED',
'SubmitTimeBefore': datetime(2015, 1, 1),
'SubmitTimeAfter': datetime(2015, 1, 1)
},
NextToken='string',
MaxResults=123
)
dict
Filters the list of entities returned. You can filter on Status , SubmitTimeBefore , or SubmitTimeAfter . You can only set one filter at a time.
Status (string) --
The status of an entity recognizer.
SubmitTimeBefore (datetime) --
Filters the list of entities based on the time that the list was submitted for processing. Returns only jobs submitted before the specified time. Jobs are returned in descending order, newest to oldest.
SubmitTimeAfter (datetime) --
Filters the list of entities based on the time that the list was submitted for processing. Returns only jobs submitted after the specified time. Jobs are returned in ascending order, oldest to newest.
string
Identifies the next page of results to return.
integer
The maximum number of results to return on each page. The default is 100.
dict
Response Syntax
{
'EntityRecognizerPropertiesList': [
{
'EntityRecognizerArn': 'string',
'LanguageCode': 'en'|'es'|'fr'|'de'|'it'|'pt',
'Status': 'SUBMITTED'|'TRAINING'|'DELETING'|'IN_ERROR'|'TRAINED',
'Message': 'string',
'SubmitTime': datetime(2015, 1, 1),
'EndTime': datetime(2015, 1, 1),
'TrainingStartTime': datetime(2015, 1, 1),
'TrainingEndTime': datetime(2015, 1, 1),
'InputDataConfig': {
'EntityTypes': [
{
'Type': 'string'
},
],
'Documents': {
'S3Uri': 'string'
},
'Annotations': {
'S3Uri': 'string'
},
'EntityList': {
'S3Uri': 'string'
}
},
'RecognizerMetadata': {
'NumberOfTrainedDocuments': 123,
'NumberOfTestDocuments': 123,
'EvaluationMetrics': {
'Precision': 123.0,
'Recall': 123.0,
'F1Score': 123.0
},
'EntityTypes': [
{
'Type': 'string'
},
]
},
'DataAccessRoleArn': 'string'
},
],
'NextToken': 'string'
}
Response Structure
(dict) --
EntityRecognizerPropertiesList (list) --
The list of properties of an entity recognizer.
(dict) --
Describes information about an entity recognizer.
EntityRecognizerArn (string) --
The Amazon Resource Name (ARN) that identifies the entity recognizer.
LanguageCode (string) --
The language of the input documents. All documents must be in the same language. Only English ("en") is currently supported.
Status (string) --
Provides the status of the entity recognizer.
Message (string) --
A description of the status of the recognizer.
SubmitTime (datetime) --
The time that the recognizer was submitted for processing.
EndTime (datetime) --
The time that the recognizer creation completed.
TrainingStartTime (datetime) --
The time that training of the entity recognizer started.
TrainingEndTime (datetime) --
The time that training of the entity recognizer was completed.
InputDataConfig (dict) --
The input data properties of an entity recognizer.
EntityTypes (list) --
The entity types in the input data for an entity recognizer.
(dict) --
Information about an individual item on a list of entity types.
Type (string) --
Entity type of an item on an entity type list.
Documents (dict) --
S3 location of the documents folder for an entity recognizer
S3Uri (string) --
Specifies the Amazon S3 location where the training documents for an entity recognizer are located. The URI must be in the same region as the API endpoint that you are calling.
Annotations (dict) --
S3 location of the annotations file for an entity recognizer.
S3Uri (string) --
Specifies the Amazon S3 location where the annotations for an entity recognizer are located. The URI must be in the same region as the API endpoint that you are calling.
EntityList (dict) --
S3 location of the entity list for an entity recognizer.
S3Uri (string) --
Specifies the Amazon S3 location where the entity list is located. The URI must be in the same region as the API endpoint that you are calling.
RecognizerMetadata (dict) --
Provides information about an entity recognizer.
NumberOfTrainedDocuments (integer) --
The number of documents in the input data that were used to train the entity recognizer. Typically this is 80 to 90 percent of the input documents.
NumberOfTestDocuments (integer) --
The number of documents in the input data that were used to test the entity recognizer. Typically this is 10 to 20 percent of the input documents.
EvaluationMetrics (dict) --
Detailed information about the accuracy of an entity recognizer.
Precision (float) --
A measure of the usefulness of the recognizer results in the test data. High precision means that the recognizer returned substantially more relevant results than irrelevant ones.
Recall (float) --
A measure of how complete the recognizer results are for the test data. High recall means that the recognizer returned most of the relevant results.
F1Score (float) --
A measure of how accurate the recognizer results are for the test data. It is derived from the Precision and Recall values. The F1Score is the harmonic average of the two scores. The highest score is 1, and the worst score is 0.
EntityTypes (list) --
Entity types from the metadata of an entity recognizer.
(dict) --
Individual item from the list of entity types in the metadata of an entity recognizer.
Type (string) --
Type of entity from the list of entity types in the metadata of an entity recognizer.
DataAccessRoleArn (string) --
The Amazon Resource Name (ARN) of the AWS Identity and Management (IAM) role that grants Amazon Comprehend read access to your input data.
NextToken (string) --
Identifies the next page of results to return.
{'EntitiesDetectionJobProperties': {'EntityRecognizerArn': 'string'}}
Gets the properties associated with an entities detection job. Use this operation to get the status of a detection job.
See also: AWS API Documentation
Request Syntax
client.describe_entities_detection_job(
JobId='string'
)
string
[REQUIRED]
The identifier that Amazon Comprehend generated for the job. The operation returns this identifier in its response.
dict
Response Syntax
{
'EntitiesDetectionJobProperties': {
'JobId': 'string',
'JobName': 'string',
'JobStatus': 'SUBMITTED'|'IN_PROGRESS'|'COMPLETED'|'FAILED'|'STOP_REQUESTED'|'STOPPED',
'Message': 'string',
'SubmitTime': datetime(2015, 1, 1),
'EndTime': datetime(2015, 1, 1),
'EntityRecognizerArn': 'string',
'InputDataConfig': {
'S3Uri': 'string',
'InputFormat': 'ONE_DOC_PER_FILE'|'ONE_DOC_PER_LINE'
},
'OutputDataConfig': {
'S3Uri': 'string'
},
'LanguageCode': 'en'|'es'|'fr'|'de'|'it'|'pt',
'DataAccessRoleArn': 'string'
}
}
Response Structure
(dict) --
EntitiesDetectionJobProperties (dict) --
An object that contains the properties associated with an entities detection job.
JobId (string) --
The identifier assigned to the entities detection job.
JobName (string) --
The name that you assigned the entities detection job.
JobStatus (string) --
The current status of the entities detection job. If the status is FAILED , the Message field shows the reason for the failure.
Message (string) --
A description of the status of a job.
SubmitTime (datetime) --
The time that the entities detection job was submitted for processing.
EndTime (datetime) --
The time that the entities detection job completed
EntityRecognizerArn (string) --
The Amazon Resource Name (ARN) that identifies the entity recognizer.
InputDataConfig (dict) --
The input data configuration that you supplied when you created the entities detection job.
S3Uri (string) --
The Amazon S3 URI for the input data. The URI must be in same region as the API endpoint that you are calling. The URI can point to a single input file or it can provide the prefix for a collection of data files.
For example, if you use the URI S3://bucketName/prefix , if the prefix is a single file, Amazon Comprehend uses that file as input. If more than one file begins with the prefix, Amazon Comprehend uses all of them as input.
InputFormat (string) --
Specifies how the text in an input file should be processed:
ONE_DOC_PER_FILE - Each file is considered a separate document. Use this option when you are processing large documents, such as newspaper articles or scientific papers.
ONE_DOC_PER_LINE - Each line in a file is considered a separate document. Use this option when you are processing many short documents, such as text messages.
OutputDataConfig (dict) --
The output data configuration that you supplied when you created the entities detection job.
S3Uri (string) --
When you use the OutputDataConfig object with asynchronous operations, you specify the Amazon S3 location where you want to write the output data. The URI must be in the same region as the API endpoint that you are calling. The location is used as the prefix for the actual location of the output file.
When the topic detection job is finished, the service creates an output file in a directory specific to the job. The S3Uri field contains the location of the output file, called output.tar.gz . It is a compressed archive that contains the ouput of the operation.
LanguageCode (string) --
The language code of the input documents.
DataAccessRoleArn (string) --
The Amazon Resource Name (ARN) that gives Amazon Comprehend read access to your input data.
{'EntitiesDetectionJobPropertiesList': {'EntityRecognizerArn': 'string'}}
Gets a list of the entity detection jobs that you have submitted.
See also: AWS API Documentation
Request Syntax
client.list_entities_detection_jobs(
Filter={
'JobName': 'string',
'JobStatus': 'SUBMITTED'|'IN_PROGRESS'|'COMPLETED'|'FAILED'|'STOP_REQUESTED'|'STOPPED',
'SubmitTimeBefore': datetime(2015, 1, 1),
'SubmitTimeAfter': datetime(2015, 1, 1)
},
NextToken='string',
MaxResults=123
)
dict
Filters the jobs that are returned. You can filter jobs on their name, status, or the date and time that they were submitted. You can only set one filter at a time.
JobName (string) --
Filters on the name of the job.
JobStatus (string) --
Filters the list of jobs based on job status. Returns only jobs with the specified status.
SubmitTimeBefore (datetime) --
Filters the list of jobs based on the time that the job was submitted for processing. Returns only jobs submitted before the specified time. Jobs are returned in ascending order, oldest to newest.
SubmitTimeAfter (datetime) --
Filters the list of jobs based on the time that the job was submitted for processing. Returns only jobs submitted after the specified time. Jobs are returned in descending order, newest to oldest.
string
Identifies the next page of results to return.
integer
The maximum number of results to return in each page. The default is 100.
dict
Response Syntax
{
'EntitiesDetectionJobPropertiesList': [
{
'JobId': 'string',
'JobName': 'string',
'JobStatus': 'SUBMITTED'|'IN_PROGRESS'|'COMPLETED'|'FAILED'|'STOP_REQUESTED'|'STOPPED',
'Message': 'string',
'SubmitTime': datetime(2015, 1, 1),
'EndTime': datetime(2015, 1, 1),
'EntityRecognizerArn': 'string',
'InputDataConfig': {
'S3Uri': 'string',
'InputFormat': 'ONE_DOC_PER_FILE'|'ONE_DOC_PER_LINE'
},
'OutputDataConfig': {
'S3Uri': 'string'
},
'LanguageCode': 'en'|'es'|'fr'|'de'|'it'|'pt',
'DataAccessRoleArn': 'string'
},
],
'NextToken': 'string'
}
Response Structure
(dict) --
EntitiesDetectionJobPropertiesList (list) --
A list containing the properties of each job that is returned.
(dict) --
Provides information about an entities detection job.
JobId (string) --
The identifier assigned to the entities detection job.
JobName (string) --
The name that you assigned the entities detection job.
JobStatus (string) --
The current status of the entities detection job. If the status is FAILED , the Message field shows the reason for the failure.
Message (string) --
A description of the status of a job.
SubmitTime (datetime) --
The time that the entities detection job was submitted for processing.
EndTime (datetime) --
The time that the entities detection job completed
EntityRecognizerArn (string) --
The Amazon Resource Name (ARN) that identifies the entity recognizer.
InputDataConfig (dict) --
The input data configuration that you supplied when you created the entities detection job.
S3Uri (string) --
The Amazon S3 URI for the input data. The URI must be in same region as the API endpoint that you are calling. The URI can point to a single input file or it can provide the prefix for a collection of data files.
For example, if you use the URI S3://bucketName/prefix , if the prefix is a single file, Amazon Comprehend uses that file as input. If more than one file begins with the prefix, Amazon Comprehend uses all of them as input.
InputFormat (string) --
Specifies how the text in an input file should be processed:
ONE_DOC_PER_FILE - Each file is considered a separate document. Use this option when you are processing large documents, such as newspaper articles or scientific papers.
ONE_DOC_PER_LINE - Each line in a file is considered a separate document. Use this option when you are processing many short documents, such as text messages.
OutputDataConfig (dict) --
The output data configuration that you supplied when you created the entities detection job.
S3Uri (string) --
When you use the OutputDataConfig object with asynchronous operations, you specify the Amazon S3 location where you want to write the output data. The URI must be in the same region as the API endpoint that you are calling. The location is used as the prefix for the actual location of the output file.
When the topic detection job is finished, the service creates an output file in a directory specific to the job. The S3Uri field contains the location of the output file, called output.tar.gz . It is a compressed archive that contains the ouput of the operation.
LanguageCode (string) --
The language code of the input documents.
DataAccessRoleArn (string) --
The Amazon Resource Name (ARN) that gives Amazon Comprehend read access to your input data.
NextToken (string) --
Identifies the next page of results to return.
{'EntityRecognizerArn': 'string'}
Starts an asynchronous entity detection job for a collection of documents. Use the operation to track the status of a job.
This API can be used for either standard entity detection or custom entity recognition. In order to be used for custom entity recognition, the optional EntityRecognizerArn must be used in order to provide access to the recognizer being used to detect the custom entity.
See also: AWS API Documentation
Request Syntax
client.start_entities_detection_job(
InputDataConfig={
'S3Uri': 'string',
'InputFormat': 'ONE_DOC_PER_FILE'|'ONE_DOC_PER_LINE'
},
OutputDataConfig={
'S3Uri': 'string'
},
DataAccessRoleArn='string',
JobName='string',
EntityRecognizerArn='string',
LanguageCode='en'|'es'|'fr'|'de'|'it'|'pt',
ClientRequestToken='string'
)
dict
[REQUIRED]
Specifies the format and location of the input data for the job.
S3Uri (string) -- [REQUIRED]
The Amazon S3 URI for the input data. The URI must be in same region as the API endpoint that you are calling. The URI can point to a single input file or it can provide the prefix for a collection of data files.
For example, if you use the URI S3://bucketName/prefix , if the prefix is a single file, Amazon Comprehend uses that file as input. If more than one file begins with the prefix, Amazon Comprehend uses all of them as input.
InputFormat (string) --
Specifies how the text in an input file should be processed:
ONE_DOC_PER_FILE - Each file is considered a separate document. Use this option when you are processing large documents, such as newspaper articles or scientific papers.
ONE_DOC_PER_LINE - Each line in a file is considered a separate document. Use this option when you are processing many short documents, such as text messages.
dict
[REQUIRED]
Specifies where to send the output files.
S3Uri (string) -- [REQUIRED]
When you use the OutputDataConfig object with asynchronous operations, you specify the Amazon S3 location where you want to write the output data. The URI must be in the same region as the API endpoint that you are calling. The location is used as the prefix for the actual location of the output file.
When the topic detection job is finished, the service creates an output file in a directory specific to the job. The S3Uri field contains the location of the output file, called output.tar.gz . It is a compressed archive that contains the ouput of the operation.
string
[REQUIRED]
The Amazon Resource Name (ARN) of the AWS Identity and Access Management (IAM) role that grants Amazon Comprehend read access to your input data. For more information, see https://docs.aws.amazon.com/comprehend/latest/dg/access-control-managing-permissions.html#auth-role-permissions .
string
The identifier of the job.
string
The Amazon Resource Name (ARN) that identifies the specific entity recognizer to be used by the StartEntitiesDetectionJob . This ARN is optional and is only used for a custom entity recognition job.
string
[REQUIRED]
The language of the input documents. All documents must be in the same language. You can specify any of the languages supported by Amazon Comprehend: English ("en"), Spanish ("es"), French ("fr"), German ("de"), Italian ("it"), or Portuguese ("pt"). If custom entities recognition is used, this parameter is ignored and the language used for training the model is used instead.
string
A unique identifier for the request. If you don't set the client request token, Amazon Comprehend generates one.
This field is autopopulated if not provided.
dict
Response Syntax
{
'JobId': 'string',
'JobStatus': 'SUBMITTED'|'IN_PROGRESS'|'COMPLETED'|'FAILED'|'STOP_REQUESTED'|'STOPPED'
}
Response Structure
(dict) --
JobId (string) --
The identifier generated for the job. To get the status of job, use this identifier with the operation.
JobStatus (string) --
The status of the job.
SUBMITTED - The job has been received and is queued for processing.
IN_PROGRESS - Amazon Comprehend is processing the job.
COMPLETED - The job was successfully completed and the output is available.
FAILED - The job did not complete. To get details, use the operation.
STOP_REQUESTED - Amazon Comprehend has received a stop request for the job and is processing the request.
STOPPED - The job was successfully stopped without completing.