AWS API Changes

2017/11/16 - AWS Glue - 8 updated api methods

Changes API update for AWS Glue. New crawler configuration attribute enables customers to specify crawler behavior. New XML classifier enables classification of XML data.

CreateClassifier (updated)

Link ¶
Changes (request)

{'XMLClassifier': {'Classification': 'string',
                   'Name': 'string',
                   'RowTag': 'string'}}

Creates a classifier in the user's account. This may be either a GrokClassifier or an XMLClassifier .

See also: AWS API Documentation

Request Syntax

client.create_classifier(
    GrokClassifier={
        'Classification': 'string',
        'Name': 'string',
        'GrokPattern': 'string',
        'CustomPatterns': 'string'
    },
    XMLClassifier={
        'Classification': 'string',
        'Name': 'string',
        'RowTag': 'string'
    }
)

type GrokClassifier

dict

param GrokClassifier

A GrokClassifier object specifying the classifier to create.

Classification (string) -- [REQUIRED]

An identifier of the data format that the classifier matches, such as Twitter, JSON, Omniture logs, Amazon CloudWatch Logs, and so on.
Name (string) -- [REQUIRED]

The name of the new classifier.
GrokPattern (string) -- [REQUIRED]

The grok pattern used by this classifier.
CustomPatterns (string) --

Optional custom grok patterns used by this classifier.

type XMLClassifier

dict

param XMLClassifier

An XMLClassifier object specifying the classifier to create.

Classification (string) -- [REQUIRED]

An identifier of the data format that the classifier matches.
Name (string) -- [REQUIRED]

The name of the classifier.
RowTag (string) --

The XML tag designating the element that contains each record in an XML document being parsed. Note that this cannot be an empty element. It must contain child elements representing fields in the record.

rtype

dict

returns

Response Syntax

{}

Response Structure

(dict) --

CreateCrawler (updated)

Link ¶
Changes (request)

{'Configuration': 'string'}

Creates a new crawler with specified targets, role, configuration, and optional schedule. At least one crawl target must be specified, in either the s3Targets or the jdbcTargets field.

See also: AWS API Documentation

Request Syntax

client.create_crawler(
    Name='string',
    Role='string',
    DatabaseName='string',
    Description='string',
    Targets={
        'S3Targets': [
            {
                'Path': 'string',
                'Exclusions': [
                    'string',
                ]
            },
        ],
        'JdbcTargets': [
            {
                'ConnectionName': 'string',
                'Path': 'string',
                'Exclusions': [
                    'string',
                ]
            },
        ]
    },
    Schedule='string',
    Classifiers=[
        'string',
    ],
    TablePrefix='string',
    SchemaChangePolicy={
        'UpdateBehavior': 'LOG'|'UPDATE_IN_DATABASE',
        'DeleteBehavior': 'LOG'|'DELETE_FROM_DATABASE'|'DEPRECATE_IN_DATABASE'
    },
    Configuration='string'
)

type Name

string

param Name

[REQUIRED]

Name of the new crawler.

type Role

string

param Role

[REQUIRED]

The IAM role (or ARN of an IAM role) used by the new crawler to access customer resources.

type DatabaseName

string

param DatabaseName

[REQUIRED]

The AWS Glue database where results are written, such as: arn:aws:daylight:us-east-1::database/sometable/* .

type Description

string

param Description

A description of the new crawler.

type Targets

dict

param Targets

[REQUIRED]

A list of collection of targets to crawl.

S3Targets (list) --

Specifies Amazon S3 targets.
- (dict) --
  
  Specifies a data store in Amazon S3.
  - Path (string) --
    
    The path to the Amazon S3 target.
  - Exclusions (list) --
    
    A list of glob patterns used to exclude from the crawl. For more information, see Catalog Tables with a Crawler .
    - (string) --
JdbcTargets (list) --

Specifies JDBC targets.
- (dict) --
  
  Specifies a JDBC data store to crawl.
  - ConnectionName (string) --
    
    The name of the connection to use to connect to the JDBC target.
  - Path (string) --
    
    The path of the JDBC target.
  - Exclusions (list) --
    
    A list of glob patterns used to exclude from the crawl. For more information, see Catalog Tables with a Crawler .
    - (string) --

type Schedule

string

param Schedule

A cron expression used to specify the schedule (see Time-Based Schedules for Jobs and Crawlers . For example, to run something every day at 12:15 UTC, you would specify: cron(15 12 * * ? *) .

type Classifiers

list

param Classifiers

A list of custom classifiers that the user has registered. By default, all AWS classifiers are included in a crawl, but these custom classifiers always override the default classifiers for a given classification.

(string) --

type TablePrefix

string

param TablePrefix

The table prefix used for catalog tables that are created.

type SchemaChangePolicy

dict

param SchemaChangePolicy

Policy for the crawler's update and deletion behavior.

UpdateBehavior (string) --

The update behavior when the crawler finds a changed schema.
DeleteBehavior (string) --

The deletion behavior when the crawler finds a deleted object.

type Configuration

string

param Configuration

Crawler configuration information. This versioned JSON string allows users to specify aspects of a Crawler's behavior.

You can use this field to force partitions to inherit metadata such as classification, input format, output format, serde information, and schema from their parent table, rather than detect this information separately for each partition.

rtype

dict

returns

Response Syntax

{}

Response Structure

(dict) --

GetClassifier (updated)

Link ¶
Changes (response)

{'Classifier': {'XMLClassifier': {'Classification': 'string',
                                  'CreationTime': 'timestamp',
                                  'LastUpdated': 'timestamp',
                                  'Name': 'string',
                                  'RowTag': 'string',
                                  'Version': 'long'}}}

Retrieve a classifier by name.

See also: AWS API Documentation

Request Syntax

client.get_classifier(
    Name='string'
)

type Name

string

param Name

[REQUIRED]

Name of the classifier to retrieve.

rtype

dict

returns

Response Syntax

{
    'Classifier': {
        'GrokClassifier': {
            'Name': 'string',
            'Classification': 'string',
            'CreationTime': datetime(2015, 1, 1),
            'LastUpdated': datetime(2015, 1, 1),
            'Version': 123,
            'GrokPattern': 'string',
            'CustomPatterns': 'string'
        },
        'XMLClassifier': {
            'Name': 'string',
            'Classification': 'string',
            'CreationTime': datetime(2015, 1, 1),
            'LastUpdated': datetime(2015, 1, 1),
            'Version': 123,
            'RowTag': 'string'
        }
    }
}

Response Structure

(dict) --
- Classifier (dict) --
  
  The requested classifier.
  - GrokClassifier (dict) --
    
    A GrokClassifier object.
    - Name (string) --
      
      The name of the classifier.
    - Classification (string) --
      
      An identifier of the data format that the classifier matches, such as Twitter, JSON, Omniture logs, and so on.
    - CreationTime (datetime) --
      
      The time this classifier was registered.
    - LastUpdated (datetime) --
      
      The time this classifier was last updated.
    - Version (integer) --
      
      The version of this classifier.
    - GrokPattern (string) --
      
      The grok pattern applied to a data store by this classifier. For more information, see built-in patterns in Writing Custom Classifers .
    - CustomPatterns (string) --
      
      Optional custom grok patterns defined by this classifier. For more information, see custom patterns in Writing Custom Classifers .
  - XMLClassifier (dict) --
    
    An XMLClassifier object.
    - Name (string) --
      
      The name of the classifier.
    - Classification (string) --
      
      An identifier of the data format that the classifier matches.
    - CreationTime (datetime) --
      
      The time this classifier was registered.
    - LastUpdated (datetime) --
      
      The time this classifier was last updated.
    - Version (integer) --
      
      The version of this classifier.
    - RowTag (string) --
      
      The XML tag designating the element that contains each record in an XML document being parsed. Note that this cannot be an empty element. It must contain child elements representing fields in the record.

GetClassifiers (updated)

Link ¶
Changes (response)

{'Classifiers': {'XMLClassifier': {'Classification': 'string',
                                   'CreationTime': 'timestamp',
                                   'LastUpdated': 'timestamp',
                                   'Name': 'string',
                                   'RowTag': 'string',
                                   'Version': 'long'}}}

Lists all classifier objects in the Data Catalog.

See also: AWS API Documentation

Request Syntax

client.get_classifiers(
    MaxResults=123,
    NextToken='string'
)

type MaxResults

integer

param MaxResults

Size of the list to return (optional).

type NextToken

string

param NextToken

An optional continuation token.

rtype

dict

returns

Response Syntax

{
    'Classifiers': [
        {
            'GrokClassifier': {
                'Name': 'string',
                'Classification': 'string',
                'CreationTime': datetime(2015, 1, 1),
                'LastUpdated': datetime(2015, 1, 1),
                'Version': 123,
                'GrokPattern': 'string',
                'CustomPatterns': 'string'
            },
            'XMLClassifier': {
                'Name': 'string',
                'Classification': 'string',
                'CreationTime': datetime(2015, 1, 1),
                'LastUpdated': datetime(2015, 1, 1),
                'Version': 123,
                'RowTag': 'string'
            }
        },
    ],
    'NextToken': 'string'
}

Response Structure

(dict) --
- Classifiers (list) --
  
  The requested list of classifier objects.
  - (dict) --
    
    Classifiers are written in Python and triggered during a crawl task. You can write your own classifiers to best categorize your data sources and specify the appropriate schemas to use for them. A classifier checks whether a given file is in a format it can handle, and if it is, the classifier creates a schema in the form of a StructType object that matches that data format.
    
    A classifier can be either a grok classifier or an XML classifier, specified in one or the other field of the Classifier object.
    - GrokClassifier (dict) --
      
      A GrokClassifier object.
      - Name (string) --
        
        The name of the classifier.
      - Classification (string) --
        
        An identifier of the data format that the classifier matches, such as Twitter, JSON, Omniture logs, and so on.
      - CreationTime (datetime) --
        
        The time this classifier was registered.
      - LastUpdated (datetime) --
        
        The time this classifier was last updated.
      - Version (integer) --
        
        The version of this classifier.
      - GrokPattern (string) --
        
        The grok pattern applied to a data store by this classifier. For more information, see built-in patterns in Writing Custom Classifers .
      - CustomPatterns (string) --
        
        Optional custom grok patterns defined by this classifier. For more information, see custom patterns in Writing Custom Classifers .
    - XMLClassifier (dict) --
      
      An XMLClassifier object.
      - Name (string) --
        
        The name of the classifier.
      - Classification (string) --
        
        An identifier of the data format that the classifier matches.
      - CreationTime (datetime) --
        
        The time this classifier was registered.
      - LastUpdated (datetime) --
        
        The time this classifier was last updated.
      - Version (integer) --
        
        The version of this classifier.
      - RowTag (string) --
        
        The XML tag designating the element that contains each record in an XML document being parsed. Note that this cannot be an empty element. It must contain child elements representing fields in the record.
- NextToken (string) --
  
  A continuation token.

GetCrawler (updated)

Link ¶
Changes (response)

{'Crawler': {'Configuration': 'string'}}

Retrieves metadata for a specified crawler.

See also: AWS API Documentation

Request Syntax

client.get_crawler(
    Name='string'
)

type Name

string

param Name

[REQUIRED]

Name of the crawler to retrieve metadata for.

rtype

dict

returns

Response Syntax

{
    'Crawler': {
        'Name': 'string',
        'Role': 'string',
        'Targets': {
            'S3Targets': [
                {
                    'Path': 'string',
                    'Exclusions': [
                        'string',
                    ]
                },
            ],
            'JdbcTargets': [
                {
                    'ConnectionName': 'string',
                    'Path': 'string',
                    'Exclusions': [
                        'string',
                    ]
                },
            ]
        },
        'DatabaseName': 'string',
        'Description': 'string',
        'Classifiers': [
            'string',
        ],
        'SchemaChangePolicy': {
            'UpdateBehavior': 'LOG'|'UPDATE_IN_DATABASE',
            'DeleteBehavior': 'LOG'|'DELETE_FROM_DATABASE'|'DEPRECATE_IN_DATABASE'
        },
        'State': 'READY'|'RUNNING'|'STOPPING',
        'TablePrefix': 'string',
        'Schedule': {
            'ScheduleExpression': 'string',
            'State': 'SCHEDULED'|'NOT_SCHEDULED'|'TRANSITIONING'
        },
        'CrawlElapsedTime': 123,
        'CreationTime': datetime(2015, 1, 1),
        'LastUpdated': datetime(2015, 1, 1),
        'LastCrawl': {
            'Status': 'SUCCEEDED'|'CANCELLED'|'FAILED',
            'ErrorMessage': 'string',
            'LogGroup': 'string',
            'LogStream': 'string',
            'MessagePrefix': 'string',
            'StartTime': datetime(2015, 1, 1)
        },
        'Version': 123,
        'Configuration': 'string'
    }
}

Response Structure

(dict) --
- Crawler (dict) --
  
  The metadata for the specified crawler.
  - Name (string) --
    
    The crawler name.
  - Role (string) --
    
    The IAM role (or ARN of an IAM role) used to access customer resources, such as data in Amazon S3.
  - Targets (dict) --
    
    A collection of targets to crawl.
    - S3Targets (list) --
      
      Specifies Amazon S3 targets.
      - (dict) --
        
        Specifies a data store in Amazon S3.
        
        Path (string) --
        
        The path to the Amazon S3 target.
        
        Exclusions (list) --
        
        A list of glob patterns used to exclude from the crawl. For more information, see Catalog Tables with a Crawler .
        
        (string) --
    - JdbcTargets (list) --
      
      Specifies JDBC targets.
      - (dict) --
        
        Specifies a JDBC data store to crawl.
        
        ConnectionName (string) --
        
        The name of the connection to use to connect to the JDBC target.
        
        Path (string) --
        
        The path of the JDBC target.
        
        Exclusions (list) --
        
        A list of glob patterns used to exclude from the crawl. For more information, see Catalog Tables with a Crawler .
        
        (string) --
  - DatabaseName (string) --
    
    The database where metadata is written by this crawler.
  - Description (string) --
    
    A description of the crawler.
  - Classifiers (list) --
    
    A list of custom classifiers associated with the crawler.
    - (string) --
  - SchemaChangePolicy (dict) --
    
    Sets the behavior when the crawler finds a changed or deleted object.
    - UpdateBehavior (string) --
      
      The update behavior when the crawler finds a changed schema.
    - DeleteBehavior (string) --
      
      The deletion behavior when the crawler finds a deleted object.
  - State (string) --
    
    Indicates whether the crawler is running, or whether a run is pending.
  - TablePrefix (string) --
    
    The prefix added to the names of tables that are created.
  - Schedule (dict) --
    
    For scheduled crawlers, the schedule when the crawler runs.
    - ScheduleExpression (string) --
      
      A cron expression used to specify the schedule (see Time-Based Schedules for Jobs and Crawlers . For example, to run something every day at 12:15 UTC, you would specify: cron(15 12 * * ? *) .
    - State (string) --
      
      The state of the schedule.
  - CrawlElapsedTime (integer) --
    
    If the crawler is running, contains the total time elapsed since the last crawl began.
  - CreationTime (datetime) --
    
    The time when the crawler was created.
  - LastUpdated (datetime) --
    
    The time the crawler was last updated.
  - LastCrawl (dict) --
    
    The status of the last crawl, and potentially error information if an error occurred.
    - Status (string) --
      
      Status of the last crawl.
    - ErrorMessage (string) --
      
      If an error occurred, the error information about the last crawl.
    - LogGroup (string) --
      
      The log group for the last crawl.
    - LogStream (string) --
      
      The log stream for the last crawl.
    - MessagePrefix (string) --
      
      The prefix for a message about this crawl.
    - StartTime (datetime) --
      
      The time at which the crawl started.
  - Version (integer) --
    
    The version of the crawler.
  - Configuration (string) --
    
    Crawler configuration information. This versioned JSON string allows users to specify aspects of a Crawler's behavior.
    
    You can use this field to force partitions to inherit metadata such as classification, input format, output format, serde information, and schema from their parent table, rather than detect this information separately for each partition. Use the following JSON string to specify that behavior:

GetCrawlers (updated)

Link ¶
Changes (response)

{'Crawlers': {'Configuration': 'string'}}

Retrieves metadata for all crawlers defined in the customer account.

See also: AWS API Documentation

Request Syntax

client.get_crawlers(
    MaxResults=123,
    NextToken='string'
)

type MaxResults

integer

param MaxResults

The number of crawlers to return on each call.

type NextToken

string

param NextToken

A continuation token, if this is a continuation request.

rtype

dict

returns

Response Syntax

{
    'Crawlers': [
        {
            'Name': 'string',
            'Role': 'string',
            'Targets': {
                'S3Targets': [
                    {
                        'Path': 'string',
                        'Exclusions': [
                            'string',
                        ]
                    },
                ],
                'JdbcTargets': [
                    {
                        'ConnectionName': 'string',
                        'Path': 'string',
                        'Exclusions': [
                            'string',
                        ]
                    },
                ]
            },
            'DatabaseName': 'string',
            'Description': 'string',
            'Classifiers': [
                'string',
            ],
            'SchemaChangePolicy': {
                'UpdateBehavior': 'LOG'|'UPDATE_IN_DATABASE',
                'DeleteBehavior': 'LOG'|'DELETE_FROM_DATABASE'|'DEPRECATE_IN_DATABASE'
            },
            'State': 'READY'|'RUNNING'|'STOPPING',
            'TablePrefix': 'string',
            'Schedule': {
                'ScheduleExpression': 'string',
                'State': 'SCHEDULED'|'NOT_SCHEDULED'|'TRANSITIONING'
            },
            'CrawlElapsedTime': 123,
            'CreationTime': datetime(2015, 1, 1),
            'LastUpdated': datetime(2015, 1, 1),
            'LastCrawl': {
                'Status': 'SUCCEEDED'|'CANCELLED'|'FAILED',
                'ErrorMessage': 'string',
                'LogGroup': 'string',
                'LogStream': 'string',
                'MessagePrefix': 'string',
                'StartTime': datetime(2015, 1, 1)
            },
            'Version': 123,
            'Configuration': 'string'
        },
    ],
    'NextToken': 'string'
}

Response Structure

(dict) --
- Crawlers (list) --
  
  A list of crawler metadata.
  - (dict) --
    
    Specifies a crawler program that examines a data source and uses classifiers to try to determine its schema. If successful, the crawler records metadata concerning the data source in the AWS Glue Data Catalog.
    - Name (string) --
      
      The crawler name.
    - Role (string) --
      
      The IAM role (or ARN of an IAM role) used to access customer resources, such as data in Amazon S3.
    - Targets (dict) --
      
      A collection of targets to crawl.
      - S3Targets (list) --
        
        Specifies Amazon S3 targets.
        
        (dict) --
        
        Specifies a data store in Amazon S3.
        
        Path (string) --
        
        The path to the Amazon S3 target.
        
        Exclusions (list) --
        
        A list of glob patterns used to exclude from the crawl. For more information, see Catalog Tables with a Crawler .
        
        (string) --
      - JdbcTargets (list) --
        
        Specifies JDBC targets.
        
        (dict) --
        
        Specifies a JDBC data store to crawl.
        
        ConnectionName (string) --
        
        The name of the connection to use to connect to the JDBC target.
        
        Path (string) --
        
        The path of the JDBC target.
        
        Exclusions (list) --
        
        A list of glob patterns used to exclude from the crawl. For more information, see Catalog Tables with a Crawler .
        
        (string) --
    - DatabaseName (string) --
      
      The database where metadata is written by this crawler.
    - Description (string) --
      
      A description of the crawler.
    - Classifiers (list) --
      
      A list of custom classifiers associated with the crawler.
      - (string) --
    - SchemaChangePolicy (dict) --
      
      Sets the behavior when the crawler finds a changed or deleted object.
      - UpdateBehavior (string) --
        
        The update behavior when the crawler finds a changed schema.
      - DeleteBehavior (string) --
        
        The deletion behavior when the crawler finds a deleted object.
    - State (string) --
      
      Indicates whether the crawler is running, or whether a run is pending.
    - TablePrefix (string) --
      
      The prefix added to the names of tables that are created.
    - Schedule (dict) --
      
      For scheduled crawlers, the schedule when the crawler runs.
      - ScheduleExpression (string) --
        
        A cron expression used to specify the schedule (see Time-Based Schedules for Jobs and Crawlers . For example, to run something every day at 12:15 UTC, you would specify: cron(15 12 * * ? *) .
      - State (string) --
        
        The state of the schedule.
    - CrawlElapsedTime (integer) --
      
      If the crawler is running, contains the total time elapsed since the last crawl began.
    - CreationTime (datetime) --
      
      The time when the crawler was created.
    - LastUpdated (datetime) --
      
      The time the crawler was last updated.
    - LastCrawl (dict) --
      
      The status of the last crawl, and potentially error information if an error occurred.
      - Status (string) --
        
        Status of the last crawl.
      - ErrorMessage (string) --
        
        If an error occurred, the error information about the last crawl.
      - LogGroup (string) --
        
        The log group for the last crawl.
      - LogStream (string) --
        
        The log stream for the last crawl.
      - MessagePrefix (string) --
        
        The prefix for a message about this crawl.
      - StartTime (datetime) --
        
        The time at which the crawl started.
    - Version (integer) --
      
      The version of the crawler.
    - Configuration (string) --
      
      Crawler configuration information. This versioned JSON string allows users to specify aspects of a Crawler's behavior.
      
      You can use this field to force partitions to inherit metadata such as classification, input format, output format, serde information, and schema from their parent table, rather than detect this information separately for each partition. Use the following JSON string to specify that behavior:
- NextToken (string) --
  
  A continuation token, if the returned list has not reached the end of those defined in this customer account.

UpdateClassifier (updated)

Link ¶
Changes (request)

{'XMLClassifier': {'Classification': 'string',
                   'Name': 'string',
                   'RowTag': 'string'}}

Modifies an existing classifier (either a GrokClassifier or an XMLClassifier ).

See also: AWS API Documentation

Request Syntax

client.update_classifier(
    GrokClassifier={
        'Name': 'string',
        'Classification': 'string',
        'GrokPattern': 'string',
        'CustomPatterns': 'string'
    },
    XMLClassifier={
        'Name': 'string',
        'Classification': 'string',
        'RowTag': 'string'
    }
)

type GrokClassifier

dict

param GrokClassifier

A GrokClassifier object with updated fields.

Name (string) -- [REQUIRED]

The name of the GrokClassifier .
Classification (string) --

An identifier of the data format that the classifier matches, such as Twitter, JSON, Omniture logs, Amazon CloudWatch Logs, and so on.
GrokPattern (string) --

The grok pattern used by this classifier.
CustomPatterns (string) --

Optional custom grok patterns used by this classifier.

type XMLClassifier

dict

param XMLClassifier

An XMLClassifier object with updated fields.

Name (string) -- [REQUIRED]

The name of the classifier.
Classification (string) --

An identifier of the data format that the classifier matches.
RowTag (string) --

The XML tag designating the element that contains each record in an XML document being parsed. Note that this cannot be an empty element. It must contain child elements representing fields in the record.

rtype

dict

returns

Response Syntax

{}

Response Structure

(dict) --

UpdateCrawler (updated)

Link ¶
Changes (request)

{'Configuration': 'string'}

Updates a crawler. If a crawler is running, you must stop it using StopCrawler before updating it.

See also: AWS API Documentation

Request Syntax

client.update_crawler(
    Name='string',
    Role='string',
    DatabaseName='string',
    Description='string',
    Targets={
        'S3Targets': [
            {
                'Path': 'string',
                'Exclusions': [
                    'string',
                ]
            },
        ],
        'JdbcTargets': [
            {
                'ConnectionName': 'string',
                'Path': 'string',
                'Exclusions': [
                    'string',
                ]
            },
        ]
    },
    Schedule='string',
    Classifiers=[
        'string',
    ],
    TablePrefix='string',
    SchemaChangePolicy={
        'UpdateBehavior': 'LOG'|'UPDATE_IN_DATABASE',
        'DeleteBehavior': 'LOG'|'DELETE_FROM_DATABASE'|'DEPRECATE_IN_DATABASE'
    },
    Configuration='string'
)

type Name

string

param Name

[REQUIRED]

Name of the new crawler.

type Role

string

param Role

The IAM role (or ARN of an IAM role) used by the new crawler to access customer resources.

type DatabaseName

string

param DatabaseName

The AWS Glue database where results are stored, such as: arn:aws:daylight:us-east-1::database/sometable/* .

type Description

string

param Description

A description of the new crawler.

type Targets

dict

param Targets

A list of targets to crawl.

S3Targets (list) --

Specifies Amazon S3 targets.
- (dict) --
  
  Specifies a data store in Amazon S3.
  - Path (string) --
    
    The path to the Amazon S3 target.
  - Exclusions (list) --
    
    A list of glob patterns used to exclude from the crawl. For more information, see Catalog Tables with a Crawler .
    - (string) --
JdbcTargets (list) --

Specifies JDBC targets.
- (dict) --
  
  Specifies a JDBC data store to crawl.
  - ConnectionName (string) --
    
    The name of the connection to use to connect to the JDBC target.
  - Path (string) --
    
    The path of the JDBC target.
  - Exclusions (list) --
    
    A list of glob patterns used to exclude from the crawl. For more information, see Catalog Tables with a Crawler .
    - (string) --

type Schedule

string

param Schedule

A cron expression used to specify the schedule (see Time-Based Schedules for Jobs and Crawlers . For example, to run something every day at 12:15 UTC, you would specify: cron(15 12 * * ? *) .

type Classifiers

list

param Classifiers

A list of custom classifiers that the user has registered. By default, all classifiers are included in a crawl, but these custom classifiers always override the default classifiers for a given classification.

(string) --

type TablePrefix

string

param TablePrefix

The table prefix used for catalog tables that are created.

type SchemaChangePolicy

dict

param SchemaChangePolicy

Policy for the crawler's update and deletion behavior.

UpdateBehavior (string) --

The update behavior when the crawler finds a changed schema.
DeleteBehavior (string) --

The deletion behavior when the crawler finds a deleted object.

type Configuration

string

param Configuration

Crawler configuration information. This versioned JSON string allows users to specify aspects of a Crawler's behavior.

rtype

dict

returns

Response Syntax

{}

Response Structure

(dict) --