2022/01/05 - AWS Lake Formation - 2 new 2 updated api methods
Changes Add new APIs for 3rd Party Support for Lake Formation
Allows a caller in a secure environment to assume a role with permission to access Amazon S3. In order to vend such credentials, Lake Formation assumes the role associated with a registered location, for example an Amazon S3 bucket, with a scope down policy which restricts the access to a single prefix.
See also: AWS API Documentation
Request Syntax
client.get_temporary_glue_table_credentials(
TableArn='string',
Permissions=[
'ALL'|'SELECT'|'ALTER'|'DROP'|'DELETE'|'INSERT'|'DESCRIBE'|'CREATE_DATABASE'|'CREATE_TABLE'|'DATA_LOCATION_ACCESS'|'CREATE_TAG'|'ALTER_TAG'|'DELETE_TAG'|'DESCRIBE_TAG'|'ASSOCIATE_TAG',
],
DurationSeconds=123,
AuditContext={
'AdditionalAuditContext': 'string'
},
SupportedPermissionTypes=[
'COLUMN_PERMISSION'|'CELL_FILTER_PERMISSION',
]
)
string
[REQUIRED]
The ARN identifying a table in the Data Catalog for the temporary credentials request.
list
Filters the request based on the user having been granted a list of specified permissions on the requested resource(s).
(string) --
integer
The time period, between 900 and 21,600 seconds, for the timeout of the temporary credentials.
dict
A structure representing context to access a resource (column names, query ID, etc).
AdditionalAuditContext (string) --
The filter engine can populate the 'AdditionalAuditContext' information with the request ID for you to track. This information will be displayed in CloudTrail log in your account.
list
[REQUIRED]
A list of supported permission types for the table. Valid values are COLUMN_PERMISSION and CELL_FILTER_PERMISSION .
(string) --
dict
Response Syntax
{
'AccessKeyId': 'string',
'SecretAccessKey': 'string',
'SessionToken': 'string',
'Expiration': datetime(2015, 1, 1)
}
Response Structure
(dict) --
AccessKeyId (string) --
The access key ID for the temporary credentials.
SecretAccessKey (string) --
The secret key for the temporary credentials.
SessionToken (string) --
The session token for the temporary credentials.
Expiration (datetime) --
The date and time when the temporary credentials expire.
This API is identical to GetTemporaryTableCredentials except that this is used when the target Data Catalog resource is of type Partition. Lake Formation restricts the permission of the vended credentials with the same scope down policy which restricts access to a single Amazon S3 prefix.
See also: AWS API Documentation
Request Syntax
client.get_temporary_glue_partition_credentials(
TableArn='string',
Partition={
'Values': [
'string',
]
},
Permissions=[
'ALL'|'SELECT'|'ALTER'|'DROP'|'DELETE'|'INSERT'|'DESCRIBE'|'CREATE_DATABASE'|'CREATE_TABLE'|'DATA_LOCATION_ACCESS'|'CREATE_TAG'|'ALTER_TAG'|'DELETE_TAG'|'DESCRIBE_TAG'|'ASSOCIATE_TAG',
],
DurationSeconds=123,
AuditContext={
'AdditionalAuditContext': 'string'
},
SupportedPermissionTypes=[
'COLUMN_PERMISSION'|'CELL_FILTER_PERMISSION',
]
)
string
[REQUIRED]
The ARN of the partitions' table.
dict
[REQUIRED]
A list of partition values identifying a single partition.
Values (list) -- [REQUIRED]
The list of partition values.
(string) --
list
Filters the request based on the user having been granted a list of specified permissions on the requested resource(s).
(string) --
integer
The time period, between 900 and 21,600 seconds, for the timeout of the temporary credentials.
dict
A structure representing context to access a resource (column names, query ID, etc).
AdditionalAuditContext (string) --
The filter engine can populate the 'AdditionalAuditContext' information with the request ID for you to track. This information will be displayed in CloudTrail log in your account.
list
[REQUIRED]
A list of supported permission types for the partition. Valid values are COLUMN_PERMISSION and CELL_FILTER_PERMISSION .
(string) --
dict
Response Syntax
{
'AccessKeyId': 'string',
'SecretAccessKey': 'string',
'SessionToken': 'string',
'Expiration': datetime(2015, 1, 1)
}
Response Structure
(dict) --
AccessKeyId (string) --
The access key ID for the temporary credentials.
SecretAccessKey (string) --
The secret key for the temporary credentials.
SessionToken (string) --
The session token for the temporary credentials.
Expiration (datetime) --
The date and time when the temporary credentials expire.
{'DataLakeSettings': {'AllowExternalDataFiltering': 'boolean',
'AuthorizedSessionTagValueList': ['string'],
'ExternalDataFilteringAllowList': [{'DataLakePrincipalIdentifier': 'string'}]}}
Retrieves the list of the data lake administrators of a Lake Formation-managed data lake.
See also: AWS API Documentation
Request Syntax
client.get_data_lake_settings(
CatalogId='string'
)
string
The identifier for the Data Catalog. By default, the account ID. The Data Catalog is the persistent metadata store. It contains database definitions, table definitions, and other control information to manage your Lake Formation environment.
dict
Response Syntax
{
'DataLakeSettings': {
'DataLakeAdmins': [
{
'DataLakePrincipalIdentifier': 'string'
},
],
'CreateDatabaseDefaultPermissions': [
{
'Principal': {
'DataLakePrincipalIdentifier': 'string'
},
'Permissions': [
'ALL'|'SELECT'|'ALTER'|'DROP'|'DELETE'|'INSERT'|'DESCRIBE'|'CREATE_DATABASE'|'CREATE_TABLE'|'DATA_LOCATION_ACCESS'|'CREATE_TAG'|'ALTER_TAG'|'DELETE_TAG'|'DESCRIBE_TAG'|'ASSOCIATE_TAG',
]
},
],
'CreateTableDefaultPermissions': [
{
'Principal': {
'DataLakePrincipalIdentifier': 'string'
},
'Permissions': [
'ALL'|'SELECT'|'ALTER'|'DROP'|'DELETE'|'INSERT'|'DESCRIBE'|'CREATE_DATABASE'|'CREATE_TABLE'|'DATA_LOCATION_ACCESS'|'CREATE_TAG'|'ALTER_TAG'|'DELETE_TAG'|'DESCRIBE_TAG'|'ASSOCIATE_TAG',
]
},
],
'TrustedResourceOwners': [
'string',
],
'AllowExternalDataFiltering': True|False,
'ExternalDataFilteringAllowList': [
{
'DataLakePrincipalIdentifier': 'string'
},
],
'AuthorizedSessionTagValueList': [
'string',
]
}
}
Response Structure
(dict) --
DataLakeSettings (dict) --
A structure representing a list of Lake Formation principals designated as data lake administrators.
DataLakeAdmins (list) --
A list of Lake Formation principals. Supported principals are IAM users or IAM roles.
(dict) --
The Lake Formation principal. Supported principals are IAM users or IAM roles.
DataLakePrincipalIdentifier (string) --
An identifier for the Lake Formation principal.
CreateDatabaseDefaultPermissions (list) --
Specifies whether access control on newly created database is managed by Lake Formation permissions or exclusively by IAM permissions. You can override this default setting when you create a database.
A null value indicates access control by Lake Formation permissions. A value that assigns ALL to IAM_ALLOWED_PRINCIPALS indicates access control by IAM permissions. This is referred to as the setting "Use only IAM access control," and is for backward compatibility with the Glue permission model implemented by IAM permissions.
The only permitted values are an empty array or an array that contains a single JSON object that grants ALL to IAM_ALLOWED_PRINCIPALS.
For more information, see Changing the Default Security Settings for Your Data Lake .
(dict) --
Permissions granted to a principal.
Principal (dict) --
The principal who is granted permissions.
DataLakePrincipalIdentifier (string) --
An identifier for the Lake Formation principal.
Permissions (list) --
The permissions that are granted to the principal.
(string) --
CreateTableDefaultPermissions (list) --
Specifies whether access control on newly created table is managed by Lake Formation permissions or exclusively by IAM permissions.
A null value indicates access control by Lake Formation permissions. A value that assigns ALL to IAM_ALLOWED_PRINCIPALS indicates access control by IAM permissions. This is referred to as the setting "Use only IAM access control," and is for backward compatibility with the Glue permission model implemented by IAM permissions.
The only permitted values are an empty array or an array that contains a single JSON object that grants ALL to IAM_ALLOWED_PRINCIPALS.
For more information, see Changing the Default Security Settings for Your Data Lake .
(dict) --
Permissions granted to a principal.
Principal (dict) --
The principal who is granted permissions.
DataLakePrincipalIdentifier (string) --
An identifier for the Lake Formation principal.
Permissions (list) --
The permissions that are granted to the principal.
(string) --
TrustedResourceOwners (list) --
A list of the resource-owning account IDs that the caller's account can use to share their user access details (user ARNs). The user ARNs can be logged in the resource owner's CloudTrail log.
You may want to specify this property when you are in a high-trust boundary, such as the same team or company.
(string) --
AllowExternalDataFiltering (boolean) --
Whether to allow Amazon EMR clusters to access data managed by Lake Formation.
If true, you allow Amazon EMR clusters to access data in Amazon S3 locations that are registered with Lake Formation.
If false or null, no Amazon EMR clusters will be able to access data in Amazon S3 locations that are registered with Lake Formation.
For more information, see (Optional) Allow Data Filtering on Amazon EMR .
ExternalDataFilteringAllowList (list) --
A list of the account IDs of Amazon Web Services accounts with Amazon EMR clusters that are to perform data filtering.>
(dict) --
The Lake Formation principal. Supported principals are IAM users or IAM roles.
DataLakePrincipalIdentifier (string) --
An identifier for the Lake Formation principal.
AuthorizedSessionTagValueList (list) --
Lake Formation relies on a privileged process secured by Amazon EMR or the third party integrator to tag the user's role while assuming it. Lake Formation will publish the acceptable key-value pair, for example key = "LakeFormationTrustedCaller" and value = "TRUE" and the third party integrator must properly tag the temporary security credentials that will be used to call Lake Formation's administrative APIs.
(string) --
{'DataLakeSettings': {'AllowExternalDataFiltering': 'boolean',
'AuthorizedSessionTagValueList': ['string'],
'ExternalDataFilteringAllowList': [{'DataLakePrincipalIdentifier': 'string'}]}}
Sets the list of data lake administrators who have admin privileges on all resources managed by Lake Formation. For more information on admin privileges, see Granting Lake Formation Permissions .
This API replaces the current list of data lake admins with the new list being passed. To add an admin, fetch the current list and add the new admin to that list and pass that list in this API.
See also: AWS API Documentation
Request Syntax
client.put_data_lake_settings(
CatalogId='string',
DataLakeSettings={
'DataLakeAdmins': [
{
'DataLakePrincipalIdentifier': 'string'
},
],
'CreateDatabaseDefaultPermissions': [
{
'Principal': {
'DataLakePrincipalIdentifier': 'string'
},
'Permissions': [
'ALL'|'SELECT'|'ALTER'|'DROP'|'DELETE'|'INSERT'|'DESCRIBE'|'CREATE_DATABASE'|'CREATE_TABLE'|'DATA_LOCATION_ACCESS'|'CREATE_TAG'|'ALTER_TAG'|'DELETE_TAG'|'DESCRIBE_TAG'|'ASSOCIATE_TAG',
]
},
],
'CreateTableDefaultPermissions': [
{
'Principal': {
'DataLakePrincipalIdentifier': 'string'
},
'Permissions': [
'ALL'|'SELECT'|'ALTER'|'DROP'|'DELETE'|'INSERT'|'DESCRIBE'|'CREATE_DATABASE'|'CREATE_TABLE'|'DATA_LOCATION_ACCESS'|'CREATE_TAG'|'ALTER_TAG'|'DELETE_TAG'|'DESCRIBE_TAG'|'ASSOCIATE_TAG',
]
},
],
'TrustedResourceOwners': [
'string',
],
'AllowExternalDataFiltering': True|False,
'ExternalDataFilteringAllowList': [
{
'DataLakePrincipalIdentifier': 'string'
},
],
'AuthorizedSessionTagValueList': [
'string',
]
}
)
string
The identifier for the Data Catalog. By default, the account ID. The Data Catalog is the persistent metadata store. It contains database definitions, table definitions, and other control information to manage your Lake Formation environment.
dict
[REQUIRED]
A structure representing a list of Lake Formation principals designated as data lake administrators.
DataLakeAdmins (list) --
A list of Lake Formation principals. Supported principals are IAM users or IAM roles.
(dict) --
The Lake Formation principal. Supported principals are IAM users or IAM roles.
DataLakePrincipalIdentifier (string) --
An identifier for the Lake Formation principal.
CreateDatabaseDefaultPermissions (list) --
Specifies whether access control on newly created database is managed by Lake Formation permissions or exclusively by IAM permissions. You can override this default setting when you create a database.
A null value indicates access control by Lake Formation permissions. A value that assigns ALL to IAM_ALLOWED_PRINCIPALS indicates access control by IAM permissions. This is referred to as the setting "Use only IAM access control," and is for backward compatibility with the Glue permission model implemented by IAM permissions.
The only permitted values are an empty array or an array that contains a single JSON object that grants ALL to IAM_ALLOWED_PRINCIPALS.
For more information, see Changing the Default Security Settings for Your Data Lake .
(dict) --
Permissions granted to a principal.
Principal (dict) --
The principal who is granted permissions.
DataLakePrincipalIdentifier (string) --
An identifier for the Lake Formation principal.
Permissions (list) --
The permissions that are granted to the principal.
(string) --
CreateTableDefaultPermissions (list) --
Specifies whether access control on newly created table is managed by Lake Formation permissions or exclusively by IAM permissions.
A null value indicates access control by Lake Formation permissions. A value that assigns ALL to IAM_ALLOWED_PRINCIPALS indicates access control by IAM permissions. This is referred to as the setting "Use only IAM access control," and is for backward compatibility with the Glue permission model implemented by IAM permissions.
The only permitted values are an empty array or an array that contains a single JSON object that grants ALL to IAM_ALLOWED_PRINCIPALS.
For more information, see Changing the Default Security Settings for Your Data Lake .
(dict) --
Permissions granted to a principal.
Principal (dict) --
The principal who is granted permissions.
DataLakePrincipalIdentifier (string) --
An identifier for the Lake Formation principal.
Permissions (list) --
The permissions that are granted to the principal.
(string) --
TrustedResourceOwners (list) --
A list of the resource-owning account IDs that the caller's account can use to share their user access details (user ARNs). The user ARNs can be logged in the resource owner's CloudTrail log.
You may want to specify this property when you are in a high-trust boundary, such as the same team or company.
(string) --
AllowExternalDataFiltering (boolean) --
Whether to allow Amazon EMR clusters to access data managed by Lake Formation.
If true, you allow Amazon EMR clusters to access data in Amazon S3 locations that are registered with Lake Formation.
If false or null, no Amazon EMR clusters will be able to access data in Amazon S3 locations that are registered with Lake Formation.
For more information, see (Optional) Allow Data Filtering on Amazon EMR .
ExternalDataFilteringAllowList (list) --
A list of the account IDs of Amazon Web Services accounts with Amazon EMR clusters that are to perform data filtering.>
(dict) --
The Lake Formation principal. Supported principals are IAM users or IAM roles.
DataLakePrincipalIdentifier (string) --
An identifier for the Lake Formation principal.
AuthorizedSessionTagValueList (list) --
Lake Formation relies on a privileged process secured by Amazon EMR or the third party integrator to tag the user's role while assuming it. Lake Formation will publish the acceptable key-value pair, for example key = "LakeFormationTrustedCaller" and value = "TRUE" and the third party integrator must properly tag the temporary security credentials that will be used to call Lake Formation's administrative APIs.
(string) --
dict
Response Syntax
{}
Response Structure
(dict) --