2021/12/20 - FinSpace Public API - 5 updated api methods
Changes Make dataset description optional and allow s3 export for dataviews
{'destinationTypeParams': {'s3DestinationExportFileFormat': 'PARQUET | '
'DELIMITED_TEXT',
's3DestinationExportFileFormatOptions': {'string': 'string'}}}
Creates a Dataview for a Dataset.
See also: AWS API Documentation
Request Syntax
client.create_data_view(
clientToken='string',
datasetId='string',
autoUpdate=True|False,
sortColumns=[
'string',
],
partitionColumns=[
'string',
],
asOfTimestamp=123,
destinationTypeParams={
'destinationType': 'string',
's3DestinationExportFileFormat': 'PARQUET'|'DELIMITED_TEXT',
's3DestinationExportFileFormatOptions': {
'string': 'string'
}
}
)
string
A token used to ensure idempotency.
This field is autopopulated if not provided.
string
[REQUIRED]
The unique Dataset identifier that is used to create a Dataview.
boolean
Flag to indicate Dataview should be updated automatically.
list
Columns to be used for sorting the data.
(string) -- 1 - 255 character String
list
Ordered set of column names used to partition data.
(string) -- 1 - 255 character String
integer
Beginning time to use for the Dataview. The value is determined as Epoch time in milliseconds. For example, the value for Monday, November 1, 2021 12:00:00 PM UTC is specified as 1635768000000.
dict
[REQUIRED]
Options that define the destination type for the Dataview.
destinationType (string) -- [REQUIRED]
Destination type for a Dataview.
GLUE_TABLE - Glue table destination type.
s3DestinationExportFileFormat (string) -- Data View Export File Format
s3DestinationExportFileFormatOptions (dict) --
(string) --
(string) --
dict
Response Syntax
{
'datasetId': 'string',
'dataViewId': 'string'
}
Response Structure
(dict) -- Response for creating a data view.
datasetId (string) --
The unique identifier of the Dataset used for the Dataview.
dataViewId (string) --
The unique identifier for the created Dataview.
{'activeFromTimestamp': 'long'}
Get information about a Changeset.
See also: AWS API Documentation
Request Syntax
client.get_changeset(
datasetId='string',
changesetId='string'
)
string
[REQUIRED]
The unique identifier for the FinSpace Dataset where the Changeset is created.
string
[REQUIRED]
The unique identifier of the Changeset for which to get data.
dict
Response Syntax
{
'changesetId': 'string',
'changesetArn': 'string',
'datasetId': 'string',
'changeType': 'REPLACE'|'APPEND'|'MODIFY',
'sourceParams': {
'string': 'string'
},
'formatParams': {
'string': 'string'
},
'createTime': 123,
'status': 'PENDING'|'FAILED'|'SUCCESS'|'RUNNING'|'STOP_REQUESTED',
'errorInfo': {
'errorMessage': 'string',
'errorCategory': 'VALIDATION'|'SERVICE_QUOTA_EXCEEDED'|'ACCESS_DENIED'|'RESOURCE_NOT_FOUND'|'THROTTLING'|'INTERNAL_SERVICE_EXCEPTION'|'CANCELLED'|'USER_RECOVERABLE'
},
'activeUntilTimestamp': 123,
'activeFromTimestamp': 123,
'updatesChangesetId': 'string',
'updatedByChangesetId': 'string'
}
Response Structure
(dict) -- The response from a describe changeset operation
changesetId (string) --
The unique identifier for a Changeset.
changesetArn (string) --
The ARN identifier of the Changeset.
datasetId (string) --
The unique identifier for the FinSpace Dataset where the Changeset is created.
changeType (string) --
Type that indicates how a Changeset is applied to a Dataset.
REPLACE - Changeset is considered as a replacement to all prior loaded Changesets.
APPEND - Changeset is considered as an addition to the end of all prior loaded Changesets.
MODIFY - Changeset is considered as a replacement to a specific prior ingested Changeset.
sourceParams (dict) --
Options that define the location of the data being ingested.
(string) --
(string) --
formatParams (dict) --
Structure of the source file(s).
(string) --
(string) --
createTime (integer) --
The timestamp at which the Changeset was created in FinSpace. The value is determined as Epoch time in milliseconds. For example, the value for Monday, November 1, 2021 12:00:00 PM UTC is specified as 1635768000000.
status (string) --
The status of Changeset creation operation.
errorInfo (dict) --
The structure with error messages.
errorMessage (string) --
The text of the error message.
errorCategory (string) --
The category of the error.
VALIDATION -The inputs to this request are invalid.
SERVICE_QUOTA_EXCEEDED - Service quotas have been exceeded. Please contact AWS support to increase quotas.
ACCESS_DENIED - Missing required permission to perform this request.
RESOURCE_NOT_FOUND - One or more inputs to this request were not found.
THROTTLING - The system temporarily lacks sufficient resources to process the request.
INTERNAL_SERVICE_EXCEPTION - An internal service error has occurred.
CANCELLED - Cancelled.
USER_RECOVERABLE - A user recoverable error has occurred.
activeUntilTimestamp (integer) --
Time until which the Changeset is active. The value is determined as Epoch time in milliseconds. For example, the value for Monday, November 1, 2021 12:00:00 PM UTC is specified as 1635768000000.
activeFromTimestamp (integer) -- Milliseconds since UTC epoch
updatesChangesetId (string) --
The unique identifier of the Changeset that is being updated.
updatedByChangesetId (string) --
The unique identifier of the updated Changeset.
{'destinationTypeParams': {'s3DestinationExportFileFormat': 'PARQUET | '
'DELIMITED_TEXT',
's3DestinationExportFileFormatOptions': {'string': 'string'}}}
Gets information about a Dataview.
See also: AWS API Documentation
Request Syntax
client.get_data_view(
dataViewId='string',
datasetId='string'
)
string
[REQUIRED]
The unique identifier for the Dataview.
string
[REQUIRED]
The unique identifier for the Dataset used in the Dataview.
dict
Response Syntax
{
'autoUpdate': True|False,
'partitionColumns': [
'string',
],
'datasetId': 'string',
'asOfTimestamp': 123,
'errorInfo': {
'errorMessage': 'string',
'errorCategory': 'VALIDATION'|'SERVICE_QUOTA_EXCEEDED'|'ACCESS_DENIED'|'RESOURCE_NOT_FOUND'|'THROTTLING'|'INTERNAL_SERVICE_EXCEPTION'|'CANCELLED'|'USER_RECOVERABLE'
},
'lastModifiedTime': 123,
'createTime': 123,
'sortColumns': [
'string',
],
'dataViewId': 'string',
'dataViewArn': 'string',
'destinationTypeParams': {
'destinationType': 'string',
's3DestinationExportFileFormat': 'PARQUET'|'DELIMITED_TEXT',
's3DestinationExportFileFormatOptions': {
'string': 'string'
}
},
'status': 'RUNNING'|'STARTING'|'FAILED'|'CANCELLED'|'TIMEOUT'|'SUCCESS'|'PENDING'|'FAILED_CLEANUP_FAILED'
}
Response Structure
(dict) -- Response from retrieving a dataview, which includes details on the target database and table name
autoUpdate (boolean) --
Flag to indicate Dataview should be updated automatically.
partitionColumns (list) --
Ordered set of column names used to partition data.
(string) -- 1 - 255 character String
datasetId (string) --
The unique identifier for the Dataset used in the Dataview.
asOfTimestamp (integer) --
Time range to use for the Dataview. The value is determined as Epoch time in milliseconds. For example, the value for Monday, November 1, 2021 12:00:00 PM UTC is specified as 1635768000000.
errorInfo (dict) --
Information about an error that occurred for the Dataview.
errorMessage (string) --
The text of the error message.
errorCategory (string) --
The category of the error.
VALIDATION -The inputs to this request are invalid.
SERVICE_QUOTA_EXCEEDED - Service quotas have been exceeded. Please contact AWS support to increase quotas.
ACCESS_DENIED - Missing required permission to perform this request.
RESOURCE_NOT_FOUND - One or more inputs to this request were not found.
THROTTLING - The system temporarily lacks sufficient resources to process the request.
INTERNAL_SERVICE_EXCEPTION - An internal service error has occurred.
CANCELLED - Cancelled.
USER_RECOVERABLE - A user recoverable error has occurred.
lastModifiedTime (integer) --
The last time that a Dataview was modified. The value is determined as Epoch time in milliseconds. For example, the value for Monday, November 1, 2021 12:00:00 PM UTC is specified as 1635768000000.
createTime (integer) --
The timestamp at which the Dataview was created in FinSpace. The value is determined as Epoch time in milliseconds. For example, the value for Monday, November 1, 2021 12:00:00 PM UTC is specified as 1635768000000.
sortColumns (list) --
Columns to be used for sorting the data.
(string) -- 1 - 255 character String
dataViewId (string) --
The unique identifier for the Dataview.
dataViewArn (string) --
The ARN identifier of the Dataview.
destinationTypeParams (dict) --
Options that define the destination type for the Dataview.
destinationType (string) --
Destination type for a Dataview.
GLUE_TABLE - Glue table destination type.
s3DestinationExportFileFormat (string) -- Data View Export File Format
s3DestinationExportFileFormatOptions (dict) --
(string) --
(string) --
status (string) --
The status of a Dataview creation.
RUNNING - Dataview creation is running.
STARTING - Dataview creation is starting.
FAILED - Dataview creation has failed.
CANCELLED - Dataview creation has been cancelled.
TIMEOUT - Dataview creation has timed out.
SUCCESS - Dataview creation has succeeded.
PENDING - Dataview creation is pending.
FAILED_CLEANUP_FAILED - Dataview creation failed and resource cleanup failed.
{'changesets': {'activeFromTimestamp': 'long'}}
Lists the FinSpace Changesets for a Dataset.
See also: AWS API Documentation
Request Syntax
client.list_changesets(
datasetId='string',
maxResults=123,
nextToken='string'
)
string
[REQUIRED]
The unique identifier for the FinSpace Dataset to which the Changeset belongs.
integer
The maximum number of results per page.
string
A token indicating where a results page should begin.
dict
Response Syntax
{
'changesets': [
{
'changesetId': 'string',
'changesetArn': 'string',
'datasetId': 'string',
'changeType': 'REPLACE'|'APPEND'|'MODIFY',
'sourceParams': {
'string': 'string'
},
'formatParams': {
'string': 'string'
},
'createTime': 123,
'status': 'PENDING'|'FAILED'|'SUCCESS'|'RUNNING'|'STOP_REQUESTED',
'errorInfo': {
'errorMessage': 'string',
'errorCategory': 'VALIDATION'|'SERVICE_QUOTA_EXCEEDED'|'ACCESS_DENIED'|'RESOURCE_NOT_FOUND'|'THROTTLING'|'INTERNAL_SERVICE_EXCEPTION'|'CANCELLED'|'USER_RECOVERABLE'
},
'activeUntilTimestamp': 123,
'activeFromTimestamp': 123,
'updatesChangesetId': 'string',
'updatedByChangesetId': 'string'
},
],
'nextToken': 'string'
}
Response Structure
(dict) -- Response to ListChangesetsResponse. This returns a list of dataset changesets that match the query criteria.
changesets (list) --
List of Changesets found.
(dict) --
A Changeset is unit of data in a Dataset.
changesetId (string) --
The unique identifier for a Changeset.
changesetArn (string) --
The ARN identifier of the Changeset.
datasetId (string) --
The unique identifier for the FinSpace Dataset in which the Changeset is created.
changeType (string) --
Type that indicates how a Changeset is applied to a Dataset.
REPLACE - Changeset is considered as a replacement to all prior loaded Changesets.
APPEND - Changeset is considered as an addition to the end of all prior loaded Changesets.
MODIFY - Changeset is considered as a replacement to a specific prior ingested Changeset.
sourceParams (dict) --
Options that define the location of the data being ingested.
(string) --
(string) --
formatParams (dict) --
Options that define the structure of the source file(s).
(string) --
(string) --
createTime (integer) --
The timestamp at which the Changeset was created in FinSpace. The value is determined as Epoch time in milliseconds. For example, the value for Monday, November 1, 2021 12:00:00 PM UTC is specified as 1635768000000.
status (string) --
Status of the Changeset ingestion.
PENDING - Changeset is pending creation.
FAILED - Changeset creation has failed.
SUCCESS - Changeset creation has succeeded.
RUNNING - Changeset creation is running.
STOP_REQUESTED - User requested Changeset creation to stop.
errorInfo (dict) --
The structure with error messages.
errorMessage (string) --
The text of the error message.
errorCategory (string) --
The category of the error.
VALIDATION -The inputs to this request are invalid.
SERVICE_QUOTA_EXCEEDED - Service quotas have been exceeded. Please contact AWS support to increase quotas.
ACCESS_DENIED - Missing required permission to perform this request.
RESOURCE_NOT_FOUND - One or more inputs to this request were not found.
THROTTLING - The system temporarily lacks sufficient resources to process the request.
INTERNAL_SERVICE_EXCEPTION - An internal service error has occurred.
CANCELLED - Cancelled.
USER_RECOVERABLE - A user recoverable error has occurred.
activeUntilTimestamp (integer) --
Time until which the Changeset is active. The value is determined as Epoch time in milliseconds. For example, the value for Monday, November 1, 2021 12:00:00 PM UTC is specified as 1635768000000.
activeFromTimestamp (integer) -- Milliseconds since UTC epoch
updatesChangesetId (string) --
The unique identifier of the Changeset that is updated.
updatedByChangesetId (string) --
The unique identifier of the updated Changeset.
nextToken (string) --
A token indicating where a results page should begin.
{'dataViews': {'destinationTypeProperties': {'s3DestinationExportFileFormat': 'PARQUET '
'| '
'DELIMITED_TEXT',
's3DestinationExportFileFormatOptions': {'string': 'string'}}}}
Lists all available Dataviews for a Dataset.
See also: AWS API Documentation
Request Syntax
client.list_data_views(
datasetId='string',
nextToken='string',
maxResults=123
)
string
[REQUIRED]
The unique identifier of the Dataset for which to retrieve Dataviews.
string
A token indicating where a results page should begin.
integer
The maximum number of results per page.
dict
Response Syntax
{
'nextToken': 'string',
'dataViews': [
{
'dataViewId': 'string',
'dataViewArn': 'string',
'datasetId': 'string',
'asOfTimestamp': 123,
'partitionColumns': [
'string',
],
'sortColumns': [
'string',
],
'status': 'RUNNING'|'STARTING'|'FAILED'|'CANCELLED'|'TIMEOUT'|'SUCCESS'|'PENDING'|'FAILED_CLEANUP_FAILED',
'errorInfo': {
'errorMessage': 'string',
'errorCategory': 'VALIDATION'|'SERVICE_QUOTA_EXCEEDED'|'ACCESS_DENIED'|'RESOURCE_NOT_FOUND'|'THROTTLING'|'INTERNAL_SERVICE_EXCEPTION'|'CANCELLED'|'USER_RECOVERABLE'
},
'destinationTypeProperties': {
'destinationType': 'string',
's3DestinationExportFileFormat': 'PARQUET'|'DELIMITED_TEXT',
's3DestinationExportFileFormatOptions': {
'string': 'string'
}
},
'autoUpdate': True|False,
'createTime': 123,
'lastModifiedTime': 123
},
]
}
Response Structure
(dict) --
nextToken (string) --
A token indicating where a results page should begin.
dataViews (list) --
A list of Dataviews.
(dict) --
Structure for the summary of a Dataview.
dataViewId (string) --
The unique identifier for the Dataview.
dataViewArn (string) --
The ARN identifier of the Dataview.
datasetId (string) --
Th unique identifier for the Dataview Dataset.
asOfTimestamp (integer) --
Time range to use for the Dataview. The value is determined as Epoch time in milliseconds. For example, the value for Monday, November 1, 2021 12:00:00 PM UTC is specified as 1635768000000.
partitionColumns (list) --
Ordered set of column names used to partition data.
(string) -- 1 - 255 character String
sortColumns (list) --
Columns to be used for sorting the data.
(string) -- 1 - 255 character String
status (string) --
The status of a Dataview creation.
RUNNING - Dataview creation is running.
STARTING - Dataview creation is starting.
FAILED - Dataview creation has failed.
CANCELLED - Dataview creation has been cancelled.
TIMEOUT - Dataview creation has timed out.
SUCCESS - Dataview creation has succeeded.
PENDING - Dataview creation is pending.
FAILED_CLEANUP_FAILED - Dataview creation failed and resource cleanup failed.
errorInfo (dict) --
The structure with error messages.
errorMessage (string) --
The text of the error message.
errorCategory (string) --
The category of the error.
VALIDATION -The inputs to this request are invalid.
SERVICE_QUOTA_EXCEEDED - Service quotas have been exceeded. Please contact AWS support to increase quotas.
ACCESS_DENIED - Missing required permission to perform this request.
RESOURCE_NOT_FOUND - One or more inputs to this request were not found.
THROTTLING - The system temporarily lacks sufficient resources to process the request.
INTERNAL_SERVICE_EXCEPTION - An internal service error has occurred.
CANCELLED - Cancelled.
USER_RECOVERABLE - A user recoverable error has occurred.
destinationTypeProperties (dict) --
Information about the Dataview destination.
destinationType (string) --
Destination type for a Dataview.
GLUE_TABLE - Glue table destination type.
s3DestinationExportFileFormat (string) -- Data View Export File Format
s3DestinationExportFileFormatOptions (dict) --
(string) --
(string) --
autoUpdate (boolean) --
The flag to indicate Dataview should be updated automatically.
createTime (integer) --
The timestamp at which the Dataview was created in FinSpace. The value is determined as Epoch time in milliseconds. For example, the value for Monday, November 1, 2021 12:00:00 PM UTC is specified as 1635768000000.
lastModifiedTime (integer) --
The last time that a Dataview was modified. The value is determined as Epoch time in milliseconds. For example, the value for Monday, November 1, 2021 12:00:00 PM UTC is specified as 1635768000000.