AWS API Changes

2023/12/04 - FinSpace User Environment Management service - 4 updated api methods

Changes Release General Purpose type clusters

CreateKxCluster (updated)

Link ¶
Changes (both)

{'clusterType': {'GP'}}

Creates a new kdb cluster.

See also: AWS API Documentation

Request Syntax

client.create_kx_cluster(
    clientToken='string',
    environmentId='string',
    clusterName='string',
    clusterType='HDB'|'RDB'|'GATEWAY'|'GP',
    databases=[
        {
            'databaseName': 'string',
            'cacheConfigurations': [
                {
                    'cacheType': 'string',
                    'dbPaths': [
                        'string',
                    ]
                },
            ],
            'changesetId': 'string'
        },
    ],
    cacheStorageConfigurations=[
        {
            'type': 'string',
            'size': 123
        },
    ],
    autoScalingConfiguration={
        'minNodeCount': 123,
        'maxNodeCount': 123,
        'autoScalingMetric': 'CPU_UTILIZATION_PERCENTAGE',
        'metricTarget': 123.0,
        'scaleInCooldownSeconds': 123.0,
        'scaleOutCooldownSeconds': 123.0
    },
    clusterDescription='string',
    capacityConfiguration={
        'nodeType': 'string',
        'nodeCount': 123
    },
    releaseLabel='string',
    vpcConfiguration={
        'vpcId': 'string',
        'securityGroupIds': [
            'string',
        ],
        'subnetIds': [
            'string',
        ],
        'ipAddressType': 'IP_V4'
    },
    initializationScript='string',
    commandLineArguments=[
        {
            'key': 'string',
            'value': 'string'
        },
    ],
    code={
        's3Bucket': 'string',
        's3Key': 'string',
        's3ObjectVersion': 'string'
    },
    executionRole='string',
    savedownStorageConfiguration={
        'type': 'SDS01',
        'size': 123
    },
    azMode='SINGLE'|'MULTI',
    availabilityZoneId='string',
    tags={
        'string': 'string'
    }
)

type clientToken

string

param clientToken

A token that ensures idempotency. This token expires in 10 minutes.

This field is autopopulated if not provided.

type environmentId

string

param environmentId

[REQUIRED]

A unique identifier for the kdb environment.

type clusterName

string

param clusterName

[REQUIRED]

A unique name for the cluster that you want to create.

type clusterType

string

param clusterType

[REQUIRED]

Specifies the type of KDB database that is being created. The following types are available:

HDB – A Historical Database. The data is only accessible with read-only permissions from one of the FinSpace managed kdb databases mounted to the cluster.
RDB – A Realtime Database. This type of database captures all the data from a ticker plant and stores it in memory until the end of day, after which it writes all of its data to a disk and reloads the HDB. This cluster type requires local storage for temporary storage of data during the savedown process. If you specify this field in your request, you must provide the savedownStorageConfiguration parameter.
GATEWAY – A gateway cluster allows you to access data across processes in kdb systems. It allows you to create your own routing logic using the initialization scripts and custom code. This type of cluster does not require a writable local storage.
GP – A general purpose cluster allows you to quickly iterate on code during development by granting greater access to system commands and enabling a fast reload of custom code. This cluster type can optionally mount databases including cache and savedown storage. For this cluster type, the node count is fixed at 1. It does not support autoscaling and supports only SINGLE AZ mode.

type databases

list

param databases

A list of databases that will be available for querying.

(dict) --

The configuration of data that is available for querying from this database.
- databaseName (string) -- [REQUIRED]
  
  The name of the kdb database. When this parameter is specified in the structure, S3 with the whole database is included by default.
- cacheConfigurations (list) --
  
  Configuration details for the disk cache used to increase performance reading from a kdb database mounted to the cluster.
  - (dict) --
    
    The structure of database cache configuration that is used for mapping database paths to cache types in clusters.
    - cacheType (string) -- [REQUIRED]
      
      The type of disk cache. This parameter is used to map the database path to cache storage. The valid values are:
      - CACHE_1000 – This type provides at least 1000 MB/s disk access throughput.
    - dbPaths (list) -- [REQUIRED]
      
      Specifies the portions of database that will be loaded into the cache for access.
      - (string) --
- changesetId (string) --
  
  A unique identifier of the changeset that is associated with the cluster.

type cacheStorageConfigurations

list

param cacheStorageConfigurations

The configurations for a read only cache storage associated with a cluster. This cache will be stored as an FSx Lustre that reads from the S3 store.

(dict) --

The configuration for read only disk cache associated with a cluster.
- type (string) -- [REQUIRED]
  
  The type of cache storage. The valid values are:
  - CACHE_1000 – This type provides at least 1000 MB/s disk access throughput.
  - CACHE_250 – This type provides at least 250 MB/s disk access throughput.
  - CACHE_12 – This type provides at least 12 MB/s disk access throughput.
  For cache type CACHE_1000 and CACHE_250 you can select cache size as 1200 GB or increments of 2400 GB. For cache type CACHE_12 you can select the cache size in increments of 6000 GB.
- size (integer) -- [REQUIRED]
  
  The size of cache in Gigabytes.

type autoScalingConfiguration

dict

param autoScalingConfiguration

The configuration based on which FinSpace will scale in or scale out nodes in your cluster.

minNodeCount (integer) --

The lowest number of nodes to scale. This value must be at least 1 and less than the maxNodeCount . If the nodes in a cluster belong to multiple availability zones, then minNodeCount must be at least 3.
maxNodeCount (integer) --

The highest number of nodes to scale. This value cannot be greater than 5.
autoScalingMetric (string) --

The metric your cluster will track in order to scale in and out. For example, CPU_UTILIZATION_PERCENTAGE is the average CPU usage across all the nodes in a cluster.
metricTarget (float) --

The desired value of the chosen autoScalingMetric . When the metric drops below this value, the cluster will scale in. When the metric goes above this value, the cluster will scale out. You can set the target value between 1 and 100 percent.
scaleInCooldownSeconds (float) --

The duration in seconds that FinSpace will wait after a scale in event before initiating another scaling event.
scaleOutCooldownSeconds (float) --

The duration in seconds that FinSpace will wait after a scale out event before initiating another scaling event.

type clusterDescription

string

param clusterDescription

A description of the cluster.

type capacityConfiguration

dict

param capacityConfiguration

[REQUIRED]

A structure for the metadata of a cluster. It includes information like the CPUs needed, memory of instances, and number of instances.

nodeType (string) --

The type that determines the hardware of the host computer used for your cluster instance. Each node type offers different memory and storage capabilities. Choose a node type based on the requirements of the application or software that you plan to run on your instance.

You can only specify one of the following values:
- kx.s.large – The node type with a configuration of 12 GiB memory and 2 vCPUs.
- kx.s.xlarge – The node type with a configuration of 27 GiB memory and 4 vCPUs.
- kx.s.2xlarge – The node type with a configuration of 54 GiB memory and 8 vCPUs.
- kx.s.4xlarge – The node type with a configuration of 108 GiB memory and 16 vCPUs.
- kx.s.8xlarge – The node type with a configuration of 216 GiB memory and 32 vCPUs.
- kx.s.16xlarge – The node type with a configuration of 432 GiB memory and 64 vCPUs.
- kx.s.32xlarge – The node type with a configuration of 864 GiB memory and 128 vCPUs.
nodeCount (integer) --

The number of instances running in a cluster.

type releaseLabel

string

param releaseLabel

[REQUIRED]

The version of FinSpace managed kdb to run.

type vpcConfiguration

dict

param vpcConfiguration

Configuration details about the network where the Privatelink endpoint of the cluster resides.

vpcId (string) --

The identifier of the VPC endpoint.
securityGroupIds (list) --

The unique identifier of the VPC security group applied to the VPC endpoint ENI for the cluster.
- (string) --
subnetIds (list) --

The identifier of the subnet that the Privatelink VPC endpoint uses to connect to the cluster.
- (string) --
ipAddressType (string) --

The IP address type for cluster network configuration parameters. The following type is available:
- IP_V4 – IP address version 4

type initializationScript

string

param initializationScript

Specifies a Q program that will be run at launch of a cluster. It is a relative path within .zip file that contains the custom code, which will be loaded on the cluster. It must include the file name itself. For example, somedir/init.q .

type commandLineArguments

list

param commandLineArguments

Defines the key-value pairs to make them available inside the cluster.

(dict) --

Defines the key-value pairs to make them available inside the cluster.
- key (string) --
  
  The name of the key.
- value (string) --
  
  The value of the key.

type code

dict

param code

The details of the custom code that you want to use inside a cluster when analyzing a data. It consists of the S3 source bucket, location, S3 object version, and the relative path from where the custom code is loaded into the cluster.

s3Bucket (string) --

A unique name for the S3 bucket.
s3Key (string) --

The full S3 path (excluding bucket) to the .zip file. This file contains the code that is loaded onto the cluster when it's started.
s3ObjectVersion (string) --

The version of an S3 object.

type executionRole

string

param executionRole

An IAM role that defines a set of permissions associated with a cluster. These permissions are assumed when a cluster attempts to access another cluster.

type savedownStorageConfiguration

dict

param savedownStorageConfiguration

The size and type of the temporary storage that is used to hold data during the savedown process. This parameter is required when you choose clusterType as RDB. All the data written to this storage space is lost when the cluster node is restarted.

type (string) -- [REQUIRED]

The type of writeable storage space for temporarily storing your savedown data. The valid values are:
- SDS01 – This type represents 3000 IOPS and io2 ebs volume type.
size (integer) -- [REQUIRED]

The size of temporary storage in gibibytes.

type azMode

string

param azMode

[REQUIRED]

The number of availability zones you want to assign per cluster. This can be one of the following

SINGLE – Assigns one availability zone per cluster.
MULTI – Assigns all the availability zones per cluster.

type availabilityZoneId

string

param availabilityZoneId

The availability zone identifiers for the requested regions.

type tags

dict

param tags

A list of key-value pairs to label the cluster. You can add up to 50 tags to a cluster.

(string) --
- (string) --

rtype

dict

returns

Response Syntax

{
    'environmentId': 'string',
    'status': 'PENDING'|'CREATING'|'CREATE_FAILED'|'RUNNING'|'UPDATING'|'DELETING'|'DELETED'|'DELETE_FAILED',
    'statusReason': 'string',
    'clusterName': 'string',
    'clusterType': 'HDB'|'RDB'|'GATEWAY'|'GP',
    'databases': [
        {
            'databaseName': 'string',
            'cacheConfigurations': [
                {
                    'cacheType': 'string',
                    'dbPaths': [
                        'string',
                    ]
                },
            ],
            'changesetId': 'string'
        },
    ],
    'cacheStorageConfigurations': [
        {
            'type': 'string',
            'size': 123
        },
    ],
    'autoScalingConfiguration': {
        'minNodeCount': 123,
        'maxNodeCount': 123,
        'autoScalingMetric': 'CPU_UTILIZATION_PERCENTAGE',
        'metricTarget': 123.0,
        'scaleInCooldownSeconds': 123.0,
        'scaleOutCooldownSeconds': 123.0
    },
    'clusterDescription': 'string',
    'capacityConfiguration': {
        'nodeType': 'string',
        'nodeCount': 123
    },
    'releaseLabel': 'string',
    'vpcConfiguration': {
        'vpcId': 'string',
        'securityGroupIds': [
            'string',
        ],
        'subnetIds': [
            'string',
        ],
        'ipAddressType': 'IP_V4'
    },
    'initializationScript': 'string',
    'commandLineArguments': [
        {
            'key': 'string',
            'value': 'string'
        },
    ],
    'code': {
        's3Bucket': 'string',
        's3Key': 'string',
        's3ObjectVersion': 'string'
    },
    'executionRole': 'string',
    'lastModifiedTimestamp': datetime(2015, 1, 1),
    'savedownStorageConfiguration': {
        'type': 'SDS01',
        'size': 123
    },
    'azMode': 'SINGLE'|'MULTI',
    'availabilityZoneId': 'string',
    'createdTimestamp': datetime(2015, 1, 1)
}

Response Structure

(dict) --
- environmentId (string) --
  
  A unique identifier for the kdb environment.
- status (string) --
  
  The status of cluster creation.
  - PENDING – The cluster is pending creation.
  - CREATING – The cluster creation process is in progress.
  - CREATE_FAILED – The cluster creation process has failed.
  - RUNNING – The cluster creation process is running.
  - UPDATING – The cluster is in the process of being updated.
  - DELETING – The cluster is in the process of being deleted.
  - DELETED – The cluster has been deleted.
  - DELETE_FAILED – The cluster failed to delete.
- statusReason (string) --
  
  The error message when a failed state occurs.
- clusterName (string) --
  
  A unique name for the cluster.
- clusterType (string) --
  
  Specifies the type of KDB database that is being created. The following types are available:
  - HDB – A Historical Database. The data is only accessible with read-only permissions from one of the FinSpace managed kdb databases mounted to the cluster.
  - RDB – A Realtime Database. This type of database captures all the data from a ticker plant and stores it in memory until the end of day, after which it writes all of its data to a disk and reloads the HDB. This cluster type requires local storage for temporary storage of data during the savedown process. If you specify this field in your request, you must provide the savedownStorageConfiguration parameter.
  - GATEWAY – A gateway cluster allows you to access data across processes in kdb systems. It allows you to create your own routing logic using the initialization scripts and custom code. This type of cluster does not require a writable local storage.
  - GP – A general purpose cluster allows you to quickly iterate on code during development by granting greater access to system commands and enabling a fast reload of custom code. This cluster type can optionally mount databases including cache and savedown storage. For this cluster type, the node count is fixed at 1. It does not support autoscaling and supports only SINGLE AZ mode.
- databases (list) --
  
  A list of databases that will be available for querying.
  - (dict) --
    
    The configuration of data that is available for querying from this database.
    - databaseName (string) --
      
      The name of the kdb database. When this parameter is specified in the structure, S3 with the whole database is included by default.
    - cacheConfigurations (list) --
      
      Configuration details for the disk cache used to increase performance reading from a kdb database mounted to the cluster.
      - (dict) --
        
        The structure of database cache configuration that is used for mapping database paths to cache types in clusters.
        
        cacheType (string) --
        
        The type of disk cache. This parameter is used to map the database path to cache storage. The valid values are:
        
        CACHE_1000 – This type provides at least 1000 MB/s disk access throughput.
        
        dbPaths (list) --
        
        Specifies the portions of database that will be loaded into the cache for access.
        
        (string) --
    - changesetId (string) --
      
      A unique identifier of the changeset that is associated with the cluster.
- cacheStorageConfigurations (list) --
  
  The configurations for a read only cache storage associated with a cluster. This cache will be stored as an FSx Lustre that reads from the S3 store.
  - (dict) --
    
    The configuration for read only disk cache associated with a cluster.
    - type (string) --
      
      The type of cache storage. The valid values are:
      - CACHE_1000 – This type provides at least 1000 MB/s disk access throughput.
      - CACHE_250 – This type provides at least 250 MB/s disk access throughput.
      - CACHE_12 – This type provides at least 12 MB/s disk access throughput.
      For cache type CACHE_1000 and CACHE_250 you can select cache size as 1200 GB or increments of 2400 GB. For cache type CACHE_12 you can select the cache size in increments of 6000 GB.
    - size (integer) --
      
      The size of cache in Gigabytes.
- autoScalingConfiguration (dict) --
  
  The configuration based on which FinSpace will scale in or scale out nodes in your cluster.
  - minNodeCount (integer) --
    
    The lowest number of nodes to scale. This value must be at least 1 and less than the maxNodeCount . If the nodes in a cluster belong to multiple availability zones, then minNodeCount must be at least 3.
  - maxNodeCount (integer) --
    
    The highest number of nodes to scale. This value cannot be greater than 5.
  - autoScalingMetric (string) --
    
    The metric your cluster will track in order to scale in and out. For example, CPU_UTILIZATION_PERCENTAGE is the average CPU usage across all the nodes in a cluster.
  - metricTarget (float) --
    
    The desired value of the chosen autoScalingMetric . When the metric drops below this value, the cluster will scale in. When the metric goes above this value, the cluster will scale out. You can set the target value between 1 and 100 percent.
  - scaleInCooldownSeconds (float) --
    
    The duration in seconds that FinSpace will wait after a scale in event before initiating another scaling event.
  - scaleOutCooldownSeconds (float) --
    
    The duration in seconds that FinSpace will wait after a scale out event before initiating another scaling event.
- clusterDescription (string) --
  
  A description of the cluster.
- capacityConfiguration (dict) --
  
  A structure for the metadata of a cluster. It includes information like the CPUs needed, memory of instances, and number of instances.
  - nodeType (string) --
    
    The type that determines the hardware of the host computer used for your cluster instance. Each node type offers different memory and storage capabilities. Choose a node type based on the requirements of the application or software that you plan to run on your instance.
    
    You can only specify one of the following values:
    - kx.s.large – The node type with a configuration of 12 GiB memory and 2 vCPUs.
    - kx.s.xlarge – The node type with a configuration of 27 GiB memory and 4 vCPUs.
    - kx.s.2xlarge – The node type with a configuration of 54 GiB memory and 8 vCPUs.
    - kx.s.4xlarge – The node type with a configuration of 108 GiB memory and 16 vCPUs.
    - kx.s.8xlarge – The node type with a configuration of 216 GiB memory and 32 vCPUs.
    - kx.s.16xlarge – The node type with a configuration of 432 GiB memory and 64 vCPUs.
    - kx.s.32xlarge – The node type with a configuration of 864 GiB memory and 128 vCPUs.
  - nodeCount (integer) --
    
    The number of instances running in a cluster.
- releaseLabel (string) --
  
  A version of the FinSpace managed kdb to run.
- vpcConfiguration (dict) --
  
  Configuration details about the network where the Privatelink endpoint of the cluster resides.
  - vpcId (string) --
    
    The identifier of the VPC endpoint.
  - securityGroupIds (list) --
    
    The unique identifier of the VPC security group applied to the VPC endpoint ENI for the cluster.
    - (string) --
  - subnetIds (list) --
    
    The identifier of the subnet that the Privatelink VPC endpoint uses to connect to the cluster.
    - (string) --
  - ipAddressType (string) --
    
    The IP address type for cluster network configuration parameters. The following type is available:
    - IP_V4 – IP address version 4
- initializationScript (string) --
  
  Specifies a Q program that will be run at launch of a cluster. It is a relative path within .zip file that contains the custom code, which will be loaded on the cluster. It must include the file name itself. For example, somedir/init.q .
- commandLineArguments (list) --
  
  Defines the key-value pairs to make them available inside the cluster.
  - (dict) --
    
    Defines the key-value pairs to make them available inside the cluster.
    - key (string) --
      
      The name of the key.
    - value (string) --
      
      The value of the key.
- code (dict) --
  
  The details of the custom code that you want to use inside a cluster when analyzing a data. It consists of the S3 source bucket, location, S3 object version, and the relative path from where the custom code is loaded into the cluster.
  - s3Bucket (string) --
    
    A unique name for the S3 bucket.
  - s3Key (string) --
    
    The full S3 path (excluding bucket) to the .zip file. This file contains the code that is loaded onto the cluster when it's started.
  - s3ObjectVersion (string) --
    
    The version of an S3 object.
- executionRole (string) --
  
  An IAM role that defines a set of permissions associated with a cluster. These permissions are assumed when a cluster attempts to access another cluster.
- lastModifiedTimestamp (datetime) --
  
  The last time that the cluster was modified. The value is determined as epoch time in milliseconds. For example, the value for Monday, November 1, 2021 12:00:00 PM UTC is specified as 1635768000000.
- savedownStorageConfiguration (dict) --
  
  The size and type of the temporary storage that is used to hold data during the savedown process. This parameter is required when you choose clusterType as RDB. All the data written to this storage space is lost when the cluster node is restarted.
  - type (string) --
    
    The type of writeable storage space for temporarily storing your savedown data. The valid values are:
    - SDS01 – This type represents 3000 IOPS and io2 ebs volume type.
  - size (integer) --
    
    The size of temporary storage in gibibytes.
- azMode (string) --
  
  The number of availability zones you want to assign per cluster. This can be one of the following
  - SINGLE – Assigns one availability zone per cluster.
  - MULTI – Assigns all the availability zones per cluster.
- availabilityZoneId (string) --
  
  The availability zone identifiers for the requested regions.
- createdTimestamp (datetime) --
  
  The timestamp at which the cluster was created in FinSpace. The value is determined as epoch time in milliseconds. For example, the value for Monday, November 1, 2021 12:00:00 PM UTC is specified as 1635768000000.

GetKxCluster (updated)

Link ¶
Changes (response)

{'clusterType': {'GP'}}

Retrieves information about a kdb cluster.

See also: AWS API Documentation

Request Syntax

client.get_kx_cluster(
    environmentId='string',
    clusterName='string'
)

type environmentId

string

param environmentId

[REQUIRED]

A unique identifier for the kdb environment.

type clusterName

string

param clusterName

[REQUIRED]

The name of the cluster that you want to retrieve.

rtype

dict

returns

Response Syntax

{
    'status': 'PENDING'|'CREATING'|'CREATE_FAILED'|'RUNNING'|'UPDATING'|'DELETING'|'DELETED'|'DELETE_FAILED',
    'statusReason': 'string',
    'clusterName': 'string',
    'clusterType': 'HDB'|'RDB'|'GATEWAY'|'GP',
    'databases': [
        {
            'databaseName': 'string',
            'cacheConfigurations': [
                {
                    'cacheType': 'string',
                    'dbPaths': [
                        'string',
                    ]
                },
            ],
            'changesetId': 'string'
        },
    ],
    'cacheStorageConfigurations': [
        {
            'type': 'string',
            'size': 123
        },
    ],
    'autoScalingConfiguration': {
        'minNodeCount': 123,
        'maxNodeCount': 123,
        'autoScalingMetric': 'CPU_UTILIZATION_PERCENTAGE',
        'metricTarget': 123.0,
        'scaleInCooldownSeconds': 123.0,
        'scaleOutCooldownSeconds': 123.0
    },
    'clusterDescription': 'string',
    'capacityConfiguration': {
        'nodeType': 'string',
        'nodeCount': 123
    },
    'releaseLabel': 'string',
    'vpcConfiguration': {
        'vpcId': 'string',
        'securityGroupIds': [
            'string',
        ],
        'subnetIds': [
            'string',
        ],
        'ipAddressType': 'IP_V4'
    },
    'initializationScript': 'string',
    'commandLineArguments': [
        {
            'key': 'string',
            'value': 'string'
        },
    ],
    'code': {
        's3Bucket': 'string',
        's3Key': 'string',
        's3ObjectVersion': 'string'
    },
    'executionRole': 'string',
    'lastModifiedTimestamp': datetime(2015, 1, 1),
    'savedownStorageConfiguration': {
        'type': 'SDS01',
        'size': 123
    },
    'azMode': 'SINGLE'|'MULTI',
    'availabilityZoneId': 'string',
    'createdTimestamp': datetime(2015, 1, 1)
}

Response Structure

(dict) --
- status (string) --
  
  The status of cluster creation.
  - PENDING – The cluster is pending creation.
  - CREATING – The cluster creation process is in progress.
  - CREATE_FAILED – The cluster creation process has failed.
  - RUNNING – The cluster creation process is running.
  - UPDATING – The cluster is in the process of being updated.
  - DELETING – The cluster is in the process of being deleted.
  - DELETED – The cluster has been deleted.
  - DELETE_FAILED – The cluster failed to delete.
- statusReason (string) --
  
  The error message when a failed state occurs.
- clusterName (string) --
  
  A unique name for the cluster.
- clusterType (string) --
  
  Specifies the type of KDB database that is being created. The following types are available:
  - HDB – A Historical Database. The data is only accessible with read-only permissions from one of the FinSpace managed kdb databases mounted to the cluster.
  - RDB – A Realtime Database. This type of database captures all the data from a ticker plant and stores it in memory until the end of day, after which it writes all of its data to a disk and reloads the HDB. This cluster type requires local storage for temporary storage of data during the savedown process. If you specify this field in your request, you must provide the savedownStorageConfiguration parameter.
  - GATEWAY – A gateway cluster allows you to access data across processes in kdb systems. It allows you to create your own routing logic using the initialization scripts and custom code. This type of cluster does not require a writable local storage.
  - GP – A general purpose cluster allows you to quickly iterate on code during development by granting greater access to system commands and enabling a fast reload of custom code. This cluster type can optionally mount databases including cache and savedown storage. For this cluster type, the node count is fixed at 1. It does not support autoscaling and supports only SINGLE AZ mode.
- databases (list) --
  
  A list of databases mounted on the cluster.
  - (dict) --
    
    The configuration of data that is available for querying from this database.
    - databaseName (string) --
      
      The name of the kdb database. When this parameter is specified in the structure, S3 with the whole database is included by default.
    - cacheConfigurations (list) --
      
      Configuration details for the disk cache used to increase performance reading from a kdb database mounted to the cluster.
      - (dict) --
        
        The structure of database cache configuration that is used for mapping database paths to cache types in clusters.
        
        cacheType (string) --
        
        The type of disk cache. This parameter is used to map the database path to cache storage. The valid values are:
        
        CACHE_1000 – This type provides at least 1000 MB/s disk access throughput.
        
        dbPaths (list) --
        
        Specifies the portions of database that will be loaded into the cache for access.
        
        (string) --
    - changesetId (string) --
      
      A unique identifier of the changeset that is associated with the cluster.
- cacheStorageConfigurations (list) --
  
  The configurations for a read only cache storage associated with a cluster. This cache will be stored as an FSx Lustre that reads from the S3 store.
  - (dict) --
    
    The configuration for read only disk cache associated with a cluster.
    - type (string) --
      
      The type of cache storage. The valid values are:
      - CACHE_1000 – This type provides at least 1000 MB/s disk access throughput.
      - CACHE_250 – This type provides at least 250 MB/s disk access throughput.
      - CACHE_12 – This type provides at least 12 MB/s disk access throughput.
      For cache type CACHE_1000 and CACHE_250 you can select cache size as 1200 GB or increments of 2400 GB. For cache type CACHE_12 you can select the cache size in increments of 6000 GB.
    - size (integer) --
      
      The size of cache in Gigabytes.
- autoScalingConfiguration (dict) --
  
  The configuration based on which FinSpace will scale in or scale out nodes in your cluster.
  - minNodeCount (integer) --
    
    The lowest number of nodes to scale. This value must be at least 1 and less than the maxNodeCount . If the nodes in a cluster belong to multiple availability zones, then minNodeCount must be at least 3.
  - maxNodeCount (integer) --
    
    The highest number of nodes to scale. This value cannot be greater than 5.
  - autoScalingMetric (string) --
    
    The metric your cluster will track in order to scale in and out. For example, CPU_UTILIZATION_PERCENTAGE is the average CPU usage across all the nodes in a cluster.
  - metricTarget (float) --
    
    The desired value of the chosen autoScalingMetric . When the metric drops below this value, the cluster will scale in. When the metric goes above this value, the cluster will scale out. You can set the target value between 1 and 100 percent.
  - scaleInCooldownSeconds (float) --
    
    The duration in seconds that FinSpace will wait after a scale in event before initiating another scaling event.
  - scaleOutCooldownSeconds (float) --
    
    The duration in seconds that FinSpace will wait after a scale out event before initiating another scaling event.
- clusterDescription (string) --
  
  A description of the cluster.
- capacityConfiguration (dict) --
  
  A structure for the metadata of a cluster. It includes information like the CPUs needed, memory of instances, and number of instances.
  - nodeType (string) --
    
    The type that determines the hardware of the host computer used for your cluster instance. Each node type offers different memory and storage capabilities. Choose a node type based on the requirements of the application or software that you plan to run on your instance.
    
    You can only specify one of the following values:
    - kx.s.large – The node type with a configuration of 12 GiB memory and 2 vCPUs.
    - kx.s.xlarge – The node type with a configuration of 27 GiB memory and 4 vCPUs.
    - kx.s.2xlarge – The node type with a configuration of 54 GiB memory and 8 vCPUs.
    - kx.s.4xlarge – The node type with a configuration of 108 GiB memory and 16 vCPUs.
    - kx.s.8xlarge – The node type with a configuration of 216 GiB memory and 32 vCPUs.
    - kx.s.16xlarge – The node type with a configuration of 432 GiB memory and 64 vCPUs.
    - kx.s.32xlarge – The node type with a configuration of 864 GiB memory and 128 vCPUs.
  - nodeCount (integer) --
    
    The number of instances running in a cluster.
- releaseLabel (string) --
  
  The version of FinSpace managed kdb to run.
- vpcConfiguration (dict) --
  
  Configuration details about the network where the Privatelink endpoint of the cluster resides.
  - vpcId (string) --
    
    The identifier of the VPC endpoint.
  - securityGroupIds (list) --
    
    The unique identifier of the VPC security group applied to the VPC endpoint ENI for the cluster.
    - (string) --
  - subnetIds (list) --
    
    The identifier of the subnet that the Privatelink VPC endpoint uses to connect to the cluster.
    - (string) --
  - ipAddressType (string) --
    
    The IP address type for cluster network configuration parameters. The following type is available:
    - IP_V4 – IP address version 4
- initializationScript (string) --
  
  Specifies a Q program that will be run at launch of a cluster. It is a relative path within .zip file that contains the custom code, which will be loaded on the cluster. It must include the file name itself. For example, somedir/init.q .
- commandLineArguments (list) --
  
  Defines key-value pairs to make them available inside the cluster.
  - (dict) --
    
    Defines the key-value pairs to make them available inside the cluster.
    - key (string) --
      
      The name of the key.
    - value (string) --
      
      The value of the key.
- code (dict) --
  
  The details of the custom code that you want to use inside a cluster when analyzing a data. It consists of the S3 source bucket, location, S3 object version, and the relative path from where the custom code is loaded into the cluster.
  - s3Bucket (string) --
    
    A unique name for the S3 bucket.
  - s3Key (string) --
    
    The full S3 path (excluding bucket) to the .zip file. This file contains the code that is loaded onto the cluster when it's started.
  - s3ObjectVersion (string) --
    
    The version of an S3 object.
- executionRole (string) --
  
  An IAM role that defines a set of permissions associated with a cluster. These permissions are assumed when a cluster attempts to access another cluster.
- lastModifiedTimestamp (datetime) --
  
  The last time that the cluster was modified. The value is determined as epoch time in milliseconds. For example, the value for Monday, November 1, 2021 12:00:00 PM UTC is specified as 1635768000000.
- savedownStorageConfiguration (dict) --
  
  The size and type of the temporary storage that is used to hold data during the savedown process. This parameter is required when you choose clusterType as RDB. All the data written to this storage space is lost when the cluster node is restarted.
  - type (string) --
    
    The type of writeable storage space for temporarily storing your savedown data. The valid values are:
    - SDS01 – This type represents 3000 IOPS and io2 ebs volume type.
  - size (integer) --
    
    The size of temporary storage in gibibytes.
- azMode (string) --
  
  The number of availability zones you want to assign per cluster. This can be one of the following
  - SINGLE – Assigns one availability zone per cluster.
  - MULTI – Assigns all the availability zones per cluster.
- availabilityZoneId (string) --
  
  The availability zone identifiers for the requested regions.
- createdTimestamp (datetime) --
  
  The timestamp at which the cluster was created in FinSpace. The value is determined as epoch time in milliseconds. For example, the value for Monday, November 1, 2021 12:00:00 PM UTC is specified as 1635768000000.

ListKxClusters (updated)

Link ¶
Changes (request, response)
Request

{'clusterType': {'GP'}}

Response

{'kxClusterSummaries': {'clusterType': {'GP'}}}

Returns a list of clusters.

See also: AWS API Documentation

Request Syntax

client.list_kx_clusters(
    environmentId='string',
    clusterType='HDB'|'RDB'|'GATEWAY'|'GP',
    maxResults=123,
    nextToken='string'
)

type environmentId

string

param environmentId

[REQUIRED]

A unique identifier for the kdb environment.

type clusterType

string

param clusterType

Specifies the type of KDB database that is being created. The following types are available:

HDB – A Historical Database. The data is only accessible with read-only permissions from one of the FinSpace managed kdb databases mounted to the cluster.
RDB – A Realtime Database. This type of database captures all the data from a ticker plant and stores it in memory until the end of day, after which it writes all of its data to a disk and reloads the HDB. This cluster type requires local storage for temporary storage of data during the savedown process. If you specify this field in your request, you must provide the savedownStorageConfiguration parameter.
GATEWAY – A gateway cluster allows you to access data across processes in kdb systems. It allows you to create your own routing logic using the initialization scripts and custom code. This type of cluster does not require a writable local storage.
GP – A general purpose cluster allows you to quickly iterate on code during development by granting greater access to system commands and enabling a fast reload of custom code. This cluster type can optionally mount databases including cache and savedown storage. For this cluster type, the node count is fixed at 1. It does not support autoscaling and supports only SINGLE AZ mode.

type maxResults

integer

param maxResults

The maximum number of results to return in this request.

type nextToken

string

param nextToken

A token that indicates where a results page should begin.

rtype

dict

returns

Response Syntax

{
    'kxClusterSummaries': [
        {
            'status': 'PENDING'|'CREATING'|'CREATE_FAILED'|'RUNNING'|'UPDATING'|'DELETING'|'DELETED'|'DELETE_FAILED',
            'statusReason': 'string',
            'clusterName': 'string',
            'clusterType': 'HDB'|'RDB'|'GATEWAY'|'GP',
            'clusterDescription': 'string',
            'releaseLabel': 'string',
            'initializationScript': 'string',
            'executionRole': 'string',
            'azMode': 'SINGLE'|'MULTI',
            'availabilityZoneId': 'string',
            'lastModifiedTimestamp': datetime(2015, 1, 1),
            'createdTimestamp': datetime(2015, 1, 1)
        },
    ],
    'nextToken': 'string'
}

Response Structure

(dict) --
- kxClusterSummaries (list) --
  
  Lists the cluster details.
  - (dict) --
    
    The details of a kdb cluster.
    - status (string) --
      
      The status of a cluster.
      - PENDING – The cluster is pending creation.
      - CREATING –The cluster creation process is in progress.
      - CREATE_FAILED– The cluster creation process has failed.
      - RUNNING – The cluster creation process is running.
      - UPDATING – The cluster is in the process of being updated.
      - DELETING – The cluster is in the process of being deleted.
      - DELETED – The cluster has been deleted.
      - DELETE_FAILED – The cluster failed to delete.
    - statusReason (string) --
      
      The error message when a failed state occurs.
    - clusterName (string) --
      
      A unique name for the cluster.
    - clusterType (string) --
      
      Specifies the type of KDB database that is being created. The following types are available:
      - HDB – A Historical Database. The data is only accessible with read-only permissions from one of the FinSpace managed kdb databases mounted to the cluster.
      - RDB – A Realtime Database. This type of database captures all the data from a ticker plant and stores it in memory until the end of day, after which it writes all of its data to a disk and reloads the HDB. This cluster type requires local storage for temporary storage of data during the savedown process. If you specify this field in your request, you must provide the savedownStorageConfiguration parameter.
      - GATEWAY – A gateway cluster allows you to access data across processes in kdb systems. It allows you to create your own routing logic using the initialization scripts and custom code. This type of cluster does not require a writable local storage.
      - GP – A general purpose cluster allows you to quickly iterate on code during development by granting greater access to system commands and enabling a fast reload of custom code. This cluster type can optionally mount databases including cache and savedown storage. For this cluster type, the node count is fixed at 1. It does not support autoscaling and supports only SINGLE AZ mode.
    - clusterDescription (string) --
      
      A description of the cluster.
    - releaseLabel (string) --
      
      A version of the FinSpace managed kdb to run.
    - initializationScript (string) --
      
      Specifies a Q program that will be run at launch of a cluster. It is a relative path within .zip file that contains the custom code, which will be loaded on the cluster. It must include the file name itself. For example, somedir/init.q .
    - executionRole (string) --
      
      An IAM role that defines a set of permissions associated with a cluster. These permissions are assumed when a cluster attempts to access another cluster.
    - azMode (string) --
      
      The number of availability zones assigned per cluster. This can be one of the following:
      - SINGLE – Assigns one availability zone per cluster.
      - MULTI – Assigns all the availability zones per cluster.
    - availabilityZoneId (string) --
      
      The availability zone identifiers for the requested regions.
    - lastModifiedTimestamp (datetime) --
      
      The last time that the cluster was modified. The value is determined as epoch time in milliseconds. For example, the value for Monday, November 1, 2021 12:00:00 PM UTC is specified as 1635768000000.
    - createdTimestamp (datetime) --
      
      The timestamp at which the cluster was created in FinSpace. The value is determined as epoch time in milliseconds. For example, the value for Monday, November 1, 2021 12:00:00 PM UTC is specified as 1635768000000.
- nextToken (string) --
  
  A token that indicates where a results page should begin.

UpdateKxClusterCodeConfiguration (updated)

Link ¶
Changes (request)

{'deploymentConfiguration': {'deploymentStrategy': {'NO_RESTART'}}}

Allows you to update code configuration on a running cluster. By using this API you can update the code, the initialization script path, and the command line arguments for a specific cluster. The configuration that you want to update will override any existing configurations on the cluster.

See also: AWS API Documentation

Request Syntax

client.update_kx_cluster_code_configuration(
    environmentId='string',
    clusterName='string',
    clientToken='string',
    code={
        's3Bucket': 'string',
        's3Key': 'string',
        's3ObjectVersion': 'string'
    },
    initializationScript='string',
    commandLineArguments=[
        {
            'key': 'string',
            'value': 'string'
        },
    ],
    deploymentConfiguration={
        'deploymentStrategy': 'NO_RESTART'|'ROLLING'|'FORCE'
    }
)

type environmentId

string

param environmentId

[REQUIRED]

A unique identifier of the kdb environment.

type clusterName

string

param clusterName

[REQUIRED]

The name of the cluster.

type clientToken

string

param clientToken

A token that ensures idempotency. This token expires in 10 minutes.

This field is autopopulated if not provided.

type code

dict

param code

[REQUIRED]

The structure of the customer code available within the running cluster.

s3Bucket (string) --

A unique name for the S3 bucket.
s3Key (string) --

The full S3 path (excluding bucket) to the .zip file. This file contains the code that is loaded onto the cluster when it's started.
s3ObjectVersion (string) --

The version of an S3 object.

type initializationScript

string

param initializationScript

You cannot update this parameter for a NO_RESTART deployment.

type commandLineArguments

list

param commandLineArguments

Specifies the key-value pairs to make them available inside the cluster.

You cannot update this parameter for a NO_RESTART deployment.

(dict) --

Defines the key-value pairs to make them available inside the cluster.
- key (string) --
  
  The name of the key.
- value (string) --
  
  The value of the key.

type deploymentConfiguration

dict

param deploymentConfiguration

The configuration that allows you to choose how you want to update the code on a cluster.

deploymentStrategy (string) -- [REQUIRED]

The type of deployment that you want on a cluster.
- ROLLING – This options updates the cluster by stopping the exiting q process and starting a new q process with updated configuration.
- NO_RESTART – This option updates the cluster without stopping the running q process. It is only available for GP type cluster. This option is quicker as it reduces the turn around time to update configuration on a cluster. With this deployment mode, you cannot update the initializationScript and commandLineArguments parameters.
- FORCE – This option updates the cluster by immediately stopping all the running processes before starting up new ones with the updated configuration.

rtype

dict

returns

Response Syntax

{}

Response Structure

(dict) --