2018/09/18 - Amazon Rekognition - 1 updated api methods
Changes This release updates the Amazon Rekognition IndexFaces API operation. It introduces a QualityFilter parameter that allows you to automatically filter out detected faces that are deemed to be of low quality by Amazon Rekognition. The quality bar is based on a variety of common use cases. You can filter low-quality detected faces by setting QualityFilter to AUTO, which is also the default setting. To index all detected faces regardless of quality, you can specify NONE. This release also provides a MaxFaces parameter that is useful when you want to only index the most prominent and largest faces in an image and don't want to index other faces detected in the image, such as smaller faces belonging to people standing in the background.
{'MaxFaces': 'integer', 'QualityFilter': 'NONE | AUTO'}
Response {'UnindexedFaces': [{'FaceDetail': {'AgeRange': {'High': 'integer',
'Low': 'integer'},
'Beard': {'Confidence': 'float',
'Value': 'boolean'},
'BoundingBox': {'Height': 'float',
'Left': 'float',
'Top': 'float',
'Width': 'float'},
'Confidence': 'float',
'Emotions': [{'Confidence': 'float',
'Type': 'HAPPY | SAD | ANGRY '
'| CONFUSED | '
'DISGUSTED | '
'SURPRISED | CALM | '
'UNKNOWN'}],
'Eyeglasses': {'Confidence': 'float',
'Value': 'boolean'},
'EyesOpen': {'Confidence': 'float',
'Value': 'boolean'},
'Gender': {'Confidence': 'float',
'Value': 'Male | Female'},
'Landmarks': [{'Type': 'eyeLeft | eyeRight '
'| nose | mouthLeft '
'| mouthRight | '
'leftEyeBrowLeft | '
'leftEyeBrowRight | '
'leftEyeBrowUp | '
'rightEyeBrowLeft | '
'rightEyeBrowRight '
'| rightEyeBrowUp | '
'leftEyeLeft | '
'leftEyeRight | '
'leftEyeUp | '
'leftEyeDown | '
'rightEyeLeft | '
'rightEyeRight | '
'rightEyeUp | '
'rightEyeDown | '
'noseLeft | '
'noseRight | '
'mouthUp | '
'mouthDown | '
'leftPupil | '
'rightPupil',
'X': 'float',
'Y': 'float'}],
'MouthOpen': {'Confidence': 'float',
'Value': 'boolean'},
'Mustache': {'Confidence': 'float',
'Value': 'boolean'},
'Pose': {'Pitch': 'float',
'Roll': 'float',
'Yaw': 'float'},
'Quality': {'Brightness': 'float',
'Sharpness': 'float'},
'Smile': {'Confidence': 'float',
'Value': 'boolean'},
'Sunglasses': {'Confidence': 'float',
'Value': 'boolean'}},
'Reasons': ['EXCEEDS_MAX_FACES | EXTREME_POSE | '
'LOW_BRIGHTNESS | LOW_SHARPNESS | '
'LOW_CONFIDENCE | SMALL_BOUNDING_BOX']}]}
Detects faces in the input image and adds them to the specified collection.
Amazon Rekognition does not save the actual faces detected. Instead, the underlying detection algorithm first detects the faces in the input image, and for each face extracts facial features into a feature vector, and stores it in the back-end database. Amazon Rekognition uses feature vectors when performing face match and search operations using the and operations.
To get the number of faces in a collection, call .
If you are using version 1.0 of the face detection model, IndexFaces indexes the 15 largest faces in the input image. Later versions of the face detection model index the 100 largest faces in the input image. To determine which version of the model you are using, call and supply the collection ID. You also get the model version from the value of FaceModelVersion in the response from IndexFaces .
For more information, see Model Versioning in the Amazon Rekognition Developer Guide.
If you provide the optional ExternalImageID for the input image you provided, Amazon Rekognition associates this ID with all faces that it detects. When you call the operation, the response returns the external ID. You can use this external image ID to create a client-side index to associate the faces with each image. You can then use the index to find all faces in an image.
You can specify the maximum number of faces to index with the MaxFaces input parameter. This is useful when you want to index the largest faces in an image, and you don't want to index other faces detected in the image.
The QualityFilter input parameter allows you to filter out detected faces that don’t meet the required quality bar chosen by Amazon Rekognition. The quality bar is based on a variety of common use cases.
In response, the operation returns an array of metadata for all detected faces, FaceRecords . This includes:
The bounding box, BoundingBox , of the detected face.
A confidence value, Confidence , indicating the confidence that the bounding box contains a face.
A face ID, faceId , assigned by the service for each face that is detected and stored.
An image ID, ImageId , assigned by the service for the input image.
If you request all facial attributes (using the detectionAttributes parameter), Amazon Rekognition returns detailed facial attributes such as facial landmarks (for example, location of eye and mouth) and other facial attributes such gender. If you provide the same image, specify the same collection, and use the same external ID in the IndexFaces operation, Amazon Rekognition doesn't save duplicate face metadata.
Information about faces detected in an image, but not indexed, is returned in an array of objects, UnindexedFaces . Faces are not indexed for reasons such as:
The face is too blurry.
The image is too dark.
The face has an extreme pose.
The face is too small.
The number of faces detected exceeds the value of the MaxFaces request parameter.
For more information, see Adding Faces to a Collection in the Amazon Rekognition Developer Guide.
The input image is passed either as base64-encoded image bytes or as a reference to an image in an Amazon S3 bucket. If you use the Amazon CLI to call Amazon Rekognition operations, passing image bytes is not supported. The image must be either a PNG or JPEG formatted file.
This operation requires permissions to perform the rekognition:IndexFaces action.
See also: AWS API Documentation
Request Syntax
client.index_faces(
CollectionId='string',
Image={
'Bytes': b'bytes',
'S3Object': {
'Bucket': 'string',
'Name': 'string',
'Version': 'string'
}
},
ExternalImageId='string',
DetectionAttributes=[
'DEFAULT'|'ALL',
],
MaxFaces=123,
QualityFilter='NONE'|'AUTO'
)
string
[REQUIRED]
The ID of an existing collection to which you want to add the faces that are detected in the input images.
dict
[REQUIRED]
The input image as base64-encoded bytes or an S3 object. If you use the AWS CLI to call Amazon Rekognition operations, passing base64-encoded image bytes is not supported.
Bytes (bytes) --
Blob of image bytes up to 5 MBs.
S3Object (dict) --
Identifies an S3 object as the image source.
Bucket (string) --
Name of the S3 bucket.
Name (string) --
S3 object key name.
Version (string) --
If the bucket is versioning enabled, you can specify the object version.
string
ID you want to assign to all the faces detected in the image.
list
An array of facial attributes that you want to be returned. This can be the default list of attributes or all attributes. If you don't specify a value for Attributes or if you specify ["DEFAULT"] , the API returns the following subset of facial attributes: BoundingBox , Confidence , Pose , Quality and Landmarks . If you provide ["ALL"] , all facial attributes are returned but the operation will take longer to complete.
If you provide both, ["ALL", "DEFAULT"] , the service uses a logical AND operator to determine which attributes to return (in this case, all attributes).
(string) --
integer
The maximum number of faces to index. The value of MaxFaces must be greater than or equal to 1. IndexFaces returns no more that 100 detected faces in an image, even if you specify a larger value for MaxFaces .
If IndexFaces detects more faces than the value of MaxFaces , the faces with the lowest quality are filtered out first. If there are still more faces than the value of MaxFaces , the faces with the smallest bounding boxes are filtered out (up to the number needed to satisfy the value of MaxFaces ). Information about the unindexed faces is available in the UnindexedFaces array.
The faces returned by IndexFaces are sorted, in descending order, by the largest face bounding box size, to the smallest.
string
Specifies how much filtering is done to identify faces detected with low quality. Filtered faces are not indexed. If you specify AUTO , filtering prioritizes the identification of faces that don’t meet the required quality bar chosen by Amazon Rekognition. The quality bar is based on a variety of common use cases. Low quality detections can arise for a number of reasons. For example, an object misidentified as a face, a face that is too blurry, or a face with a pose that is too extreme to use. If you specify NONE , no filtering is performed. The default value is NONE.
dict
Response Syntax
{
'FaceRecords': [
{
'Face': {
'FaceId': 'string',
'BoundingBox': {
'Width': ...,
'Height': ...,
'Left': ...,
'Top': ...
},
'ImageId': 'string',
'ExternalImageId': 'string',
'Confidence': ...
},
'FaceDetail': {
'BoundingBox': {
'Width': ...,
'Height': ...,
'Left': ...,
'Top': ...
},
'AgeRange': {
'Low': 123,
'High': 123
},
'Smile': {
'Value': True|False,
'Confidence': ...
},
'Eyeglasses': {
'Value': True|False,
'Confidence': ...
},
'Sunglasses': {
'Value': True|False,
'Confidence': ...
},
'Gender': {
'Value': 'Male'|'Female',
'Confidence': ...
},
'Beard': {
'Value': True|False,
'Confidence': ...
},
'Mustache': {
'Value': True|False,
'Confidence': ...
},
'EyesOpen': {
'Value': True|False,
'Confidence': ...
},
'MouthOpen': {
'Value': True|False,
'Confidence': ...
},
'Emotions': [
{
'Type': 'HAPPY'|'SAD'|'ANGRY'|'CONFUSED'|'DISGUSTED'|'SURPRISED'|'CALM'|'UNKNOWN',
'Confidence': ...
},
],
'Landmarks': [
{
'Type': 'eyeLeft'|'eyeRight'|'nose'|'mouthLeft'|'mouthRight'|'leftEyeBrowLeft'|'leftEyeBrowRight'|'leftEyeBrowUp'|'rightEyeBrowLeft'|'rightEyeBrowRight'|'rightEyeBrowUp'|'leftEyeLeft'|'leftEyeRight'|'leftEyeUp'|'leftEyeDown'|'rightEyeLeft'|'rightEyeRight'|'rightEyeUp'|'rightEyeDown'|'noseLeft'|'noseRight'|'mouthUp'|'mouthDown'|'leftPupil'|'rightPupil',
'X': ...,
'Y': ...
},
],
'Pose': {
'Roll': ...,
'Yaw': ...,
'Pitch': ...
},
'Quality': {
'Brightness': ...,
'Sharpness': ...
},
'Confidence': ...
}
},
],
'OrientationCorrection': 'ROTATE_0'|'ROTATE_90'|'ROTATE_180'|'ROTATE_270',
'FaceModelVersion': 'string',
'UnindexedFaces': [
{
'Reasons': [
'EXCEEDS_MAX_FACES'|'EXTREME_POSE'|'LOW_BRIGHTNESS'|'LOW_SHARPNESS'|'LOW_CONFIDENCE'|'SMALL_BOUNDING_BOX',
],
'FaceDetail': {
'BoundingBox': {
'Width': ...,
'Height': ...,
'Left': ...,
'Top': ...
},
'AgeRange': {
'Low': 123,
'High': 123
},
'Smile': {
'Value': True|False,
'Confidence': ...
},
'Eyeglasses': {
'Value': True|False,
'Confidence': ...
},
'Sunglasses': {
'Value': True|False,
'Confidence': ...
},
'Gender': {
'Value': 'Male'|'Female',
'Confidence': ...
},
'Beard': {
'Value': True|False,
'Confidence': ...
},
'Mustache': {
'Value': True|False,
'Confidence': ...
},
'EyesOpen': {
'Value': True|False,
'Confidence': ...
},
'MouthOpen': {
'Value': True|False,
'Confidence': ...
},
'Emotions': [
{
'Type': 'HAPPY'|'SAD'|'ANGRY'|'CONFUSED'|'DISGUSTED'|'SURPRISED'|'CALM'|'UNKNOWN',
'Confidence': ...
},
],
'Landmarks': [
{
'Type': 'eyeLeft'|'eyeRight'|'nose'|'mouthLeft'|'mouthRight'|'leftEyeBrowLeft'|'leftEyeBrowRight'|'leftEyeBrowUp'|'rightEyeBrowLeft'|'rightEyeBrowRight'|'rightEyeBrowUp'|'leftEyeLeft'|'leftEyeRight'|'leftEyeUp'|'leftEyeDown'|'rightEyeLeft'|'rightEyeRight'|'rightEyeUp'|'rightEyeDown'|'noseLeft'|'noseRight'|'mouthUp'|'mouthDown'|'leftPupil'|'rightPupil',
'X': ...,
'Y': ...
},
],
'Pose': {
'Roll': ...,
'Yaw': ...,
'Pitch': ...
},
'Quality': {
'Brightness': ...,
'Sharpness': ...
},
'Confidence': ...
}
},
]
}
Response Structure
(dict) --
FaceRecords (list) --
An array of faces detected and added to the collection. For more information, see Searching Faces in a Collection in the Amazon Rekognition Developer Guide.
(dict) --
Object containing both the face metadata (stored in the back-end database) and facial attributes that are detected but aren't stored in the database.
Face (dict) --
Describes the face properties such as the bounding box, face ID, image ID of the input image, and external image ID that you assigned.
FaceId (string) --
Unique identifier that Amazon Rekognition assigns to the face.
BoundingBox (dict) --
Bounding box of the face.
Width (float) --
Width of the bounding box as a ratio of the overall image width.
Height (float) --
Height of the bounding box as a ratio of the overall image height.
Left (float) --
Left coordinate of the bounding box as a ratio of overall image width.
Top (float) --
Top coordinate of the bounding box as a ratio of overall image height.
ImageId (string) --
Unique identifier that Amazon Rekognition assigns to the input image.
ExternalImageId (string) --
Identifier that you assign to all the faces in the input image.
Confidence (float) --
Confidence level that the bounding box contains a face (and not a different object such as a tree).
FaceDetail (dict) --
Structure containing attributes of the face that the algorithm detected.
BoundingBox (dict) --
Bounding box of the face. Default attribute.
Width (float) --
Width of the bounding box as a ratio of the overall image width.
Height (float) --
Height of the bounding box as a ratio of the overall image height.
Left (float) --
Left coordinate of the bounding box as a ratio of overall image width.
Top (float) --
Top coordinate of the bounding box as a ratio of overall image height.
AgeRange (dict) --
The estimated age range, in years, for the face. Low represents the lowest estimated age and High represents the highest estimated age.
Low (integer) --
The lowest estimated age.
High (integer) --
The highest estimated age.
Smile (dict) --
Indicates whether or not the face is smiling, and the confidence level in the determination.
Value (boolean) --
Boolean value that indicates whether the face is smiling or not.
Confidence (float) --
Level of confidence in the determination.
Eyeglasses (dict) --
Indicates whether or not the face is wearing eye glasses, and the confidence level in the determination.
Value (boolean) --
Boolean value that indicates whether the face is wearing eye glasses or not.
Confidence (float) --
Level of confidence in the determination.
Sunglasses (dict) --
Indicates whether or not the face is wearing sunglasses, and the confidence level in the determination.
Value (boolean) --
Boolean value that indicates whether the face is wearing sunglasses or not.
Confidence (float) --
Level of confidence in the determination.
Gender (dict) --
Gender of the face and the confidence level in the determination.
Value (string) --
Gender of the face.
Confidence (float) --
Level of confidence in the determination.
Beard (dict) --
Indicates whether or not the face has a beard, and the confidence level in the determination.
Value (boolean) --
Boolean value that indicates whether the face has beard or not.
Confidence (float) --
Level of confidence in the determination.
Mustache (dict) --
Indicates whether or not the face has a mustache, and the confidence level in the determination.
Value (boolean) --
Boolean value that indicates whether the face has mustache or not.
Confidence (float) --
Level of confidence in the determination.
EyesOpen (dict) --
Indicates whether or not the eyes on the face are open, and the confidence level in the determination.
Value (boolean) --
Boolean value that indicates whether the eyes on the face are open.
Confidence (float) --
Level of confidence in the determination.
MouthOpen (dict) --
Indicates whether or not the mouth on the face is open, and the confidence level in the determination.
Value (boolean) --
Boolean value that indicates whether the mouth on the face is open or not.
Confidence (float) --
Level of confidence in the determination.
Emotions (list) --
The emotions detected on the face, and the confidence level in the determination. For example, HAPPY, SAD, and ANGRY.
(dict) --
The emotions detected on the face, and the confidence level in the determination. For example, HAPPY, SAD, and ANGRY.
Type (string) --
Type of emotion detected.
Confidence (float) --
Level of confidence in the determination.
Landmarks (list) --
Indicates the location of landmarks on the face. Default attribute.
(dict) --
Indicates the location of the landmark on the face.
Type (string) --
Type of the landmark.
X (float) --
x-coordinate from the top left of the landmark expressed as the ratio of the width of the image. For example, if the images is 700x200 and the x-coordinate of the landmark is at 350 pixels, this value is 0.5.
Y (float) --
y-coordinate from the top left of the landmark expressed as the ratio of the height of the image. For example, if the images is 700x200 and the y-coordinate of the landmark is at 100 pixels, this value is 0.5.
Pose (dict) --
Indicates the pose of the face as determined by its pitch, roll, and yaw. Default attribute.
Roll (float) --
Value representing the face rotation on the roll axis.
Yaw (float) --
Value representing the face rotation on the yaw axis.
Pitch (float) --
Value representing the face rotation on the pitch axis.
Quality (dict) --
Identifies image brightness and sharpness. Default attribute.
Brightness (float) --
Value representing brightness of the face. The service returns a value between 0 and 100 (inclusive). A higher value indicates a brighter face image.
Sharpness (float) --
Value representing sharpness of the face. The service returns a value between 0 and 100 (inclusive). A higher value indicates a sharper face image.
Confidence (float) --
Confidence level that the bounding box contains a face (and not a different object such as a tree). Default attribute.
OrientationCorrection (string) --
The orientation of the input image (counterclockwise direction). If your application displays the image, you can use this value to correct image orientation. The bounding box coordinates returned in FaceRecords represent face locations before the image orientation is corrected.
Note
If the input image is in jpeg format, it might contain exchangeable image (Exif) metadata. If so, and the Exif metadata populates the orientation field, the value of OrientationCorrection is null and the bounding box coordinates in FaceRecords represent face locations after Exif metadata is used to correct the image orientation. Images in .png format don't contain Exif metadata.
FaceModelVersion (string) --
Version number of the face detection model associated with the input collection (CollectionId ).
UnindexedFaces (list) --
An array of faces that detected in the image but not indexed either because the quality filter deemed them to be of low-quality or the MaxFaces request parameter filtered them out. To use the quality filter, you specify the QualityFilter request parameter.
(dict) --
A face detected by but not indexed. Use the Reasons response attribute to determine why a face is not indexed.
Reasons (list) --
An array of reasons specifying why a face was not indexed.
EXTREME_POSE - The face is at a pose that can't be detected. For example, the head is turned too far away from the camera.
EXCEEDS_MAX_FACES - The number of faces detected is already higher than that specified by the MaxFaces input parameter for IndexFaces .
LOW_BRIGHTNESS - The image is too dark.
LOW_SHARPNESS - The image is too blurry.
LOW_CONFIDENCE - The face was detected with a low confidence.
SMALL_BOUNDING_BOX - The bounding box around the face is too small.
(string) --
FaceDetail (dict) --
Structure containing attributes of a face that was detected, but not indexed, by IndexFaces .
BoundingBox (dict) --
Bounding box of the face. Default attribute.
Width (float) --
Width of the bounding box as a ratio of the overall image width.
Height (float) --
Height of the bounding box as a ratio of the overall image height.
Left (float) --
Left coordinate of the bounding box as a ratio of overall image width.
Top (float) --
Top coordinate of the bounding box as a ratio of overall image height.
AgeRange (dict) --
The estimated age range, in years, for the face. Low represents the lowest estimated age and High represents the highest estimated age.
Low (integer) --
The lowest estimated age.
High (integer) --
The highest estimated age.
Smile (dict) --
Indicates whether or not the face is smiling, and the confidence level in the determination.
Value (boolean) --
Boolean value that indicates whether the face is smiling or not.
Confidence (float) --
Level of confidence in the determination.
Eyeglasses (dict) --
Indicates whether or not the face is wearing eye glasses, and the confidence level in the determination.
Value (boolean) --
Boolean value that indicates whether the face is wearing eye glasses or not.
Confidence (float) --
Level of confidence in the determination.
Sunglasses (dict) --
Indicates whether or not the face is wearing sunglasses, and the confidence level in the determination.
Value (boolean) --
Boolean value that indicates whether the face is wearing sunglasses or not.
Confidence (float) --
Level of confidence in the determination.
Gender (dict) --
Gender of the face and the confidence level in the determination.
Value (string) --
Gender of the face.
Confidence (float) --
Level of confidence in the determination.
Beard (dict) --
Indicates whether or not the face has a beard, and the confidence level in the determination.
Value (boolean) --
Boolean value that indicates whether the face has beard or not.
Confidence (float) --
Level of confidence in the determination.
Mustache (dict) --
Indicates whether or not the face has a mustache, and the confidence level in the determination.
Value (boolean) --
Boolean value that indicates whether the face has mustache or not.
Confidence (float) --
Level of confidence in the determination.
EyesOpen (dict) --
Indicates whether or not the eyes on the face are open, and the confidence level in the determination.
Value (boolean) --
Boolean value that indicates whether the eyes on the face are open.
Confidence (float) --
Level of confidence in the determination.
MouthOpen (dict) --
Indicates whether or not the mouth on the face is open, and the confidence level in the determination.
Value (boolean) --
Boolean value that indicates whether the mouth on the face is open or not.
Confidence (float) --
Level of confidence in the determination.
Emotions (list) --
The emotions detected on the face, and the confidence level in the determination. For example, HAPPY, SAD, and ANGRY.
(dict) --
The emotions detected on the face, and the confidence level in the determination. For example, HAPPY, SAD, and ANGRY.
Type (string) --
Type of emotion detected.
Confidence (float) --
Level of confidence in the determination.
Landmarks (list) --
Indicates the location of landmarks on the face. Default attribute.
(dict) --
Indicates the location of the landmark on the face.
Type (string) --
Type of the landmark.
X (float) --
x-coordinate from the top left of the landmark expressed as the ratio of the width of the image. For example, if the images is 700x200 and the x-coordinate of the landmark is at 350 pixels, this value is 0.5.
Y (float) --
y-coordinate from the top left of the landmark expressed as the ratio of the height of the image. For example, if the images is 700x200 and the y-coordinate of the landmark is at 100 pixels, this value is 0.5.
Pose (dict) --
Indicates the pose of the face as determined by its pitch, roll, and yaw. Default attribute.
Roll (float) --
Value representing the face rotation on the roll axis.
Yaw (float) --
Value representing the face rotation on the yaw axis.
Pitch (float) --
Value representing the face rotation on the pitch axis.
Quality (dict) --
Identifies image brightness and sharpness. Default attribute.
Brightness (float) --
Value representing brightness of the face. The service returns a value between 0 and 100 (inclusive). A higher value indicates a brighter face image.
Sharpness (float) --
Value representing sharpness of the face. The service returns a value between 0 and 100 (inclusive). A higher value indicates a sharper face image.
Confidence (float) --
Confidence level that the bounding box contains a face (and not a different object such as a tree). Default attribute.