OpenSearch Service Serverless

2026/10/22 - OpenSearch Service Serverless - 8 new 1 updated api methods

Changes  Adds APIs to create, get, list, and cancel ingestion jobs, and to create, get, list, and delete ingestion streams, for ingesting data from Amazon S3 into OpenSearch Serverless collections.

GetIngestionStream (new) Link ¶

Returns information about an ingestion stream, including its configuration, status, and processing state.

See also: AWS API Documentation

Request Syntax

client.get_ingestion_stream(
    collectionId='string',
    ingestionStreamId='string'
)
type collectionId:

string

param collectionId:

[REQUIRED]

The unique identifier of the collection that the ingestion stream belongs to.

type ingestionStreamId:

string

param ingestionStreamId:

[REQUIRED]

The unique identifier of the ingestion stream to retrieve.

rtype:

dict

returns:

Response Syntax

{
    'ingestionStreamDetail': {
        'ingestionStreamId': 'string',
        'arn': 'string',
        'collectionId': 'string',
        'ingestionStreamName': 'string',
        'indexName': 'string',
        'failureIndexName': 'string',
        'source': {
            's3': {
                'sqsArn': 'string',
                'buckets': [
                    {
                        'bucket': 'string',
                        'bucketOwnerAccountId': 'string',
                        'initialScan': 'ENABLED'|'DISABLED'
                    },
                ]
            }
        },
        'roleArn': 'string',
        'status': 'CREATING'|'ACTIVE'|'DELETING'|'DELETED'|'CREATE_FAILED'|'DELETE_FAILED',
        'state': 'STARTING'|'SCANNING'|'INGESTION_IN_PROGRESS'|'INGESTION_COMPLETE'|'STOPPED'|'FAILED',
        'statistics': {
            's3': {
                'objectsDiscovered': 123,
                'objectsIngested': 123,
                'objectsFailed': 123
            }
        },
        'revision': 123,
        'createdAt': datetime(2015, 1, 1),
        'updatedAt': datetime(2015, 1, 1),
        'failureReasons': [
            {
                'code': 'SOURCE_ACCESS_DENIED'|'SOURCE_RESOURCE_NOT_FOUND'|'DESTINATION_ACCESS_DENIED'|'DESTINATION_RESOURCE_NOT_FOUND'|'INTERNAL_SERVER_ERROR',
                'message': 'string'
            },
        ],
        'activeAlerts': [
            {
                'code': 'SOURCE_ACCESS_DENIED'|'SOURCE_RESOURCE_NOT_FOUND'|'DESTINATION_ACCESS_DENIED'|'DESTINATION_RESOURCE_NOT_FOUND'|'INTERNAL_SERVER_ERROR',
                'message': 'string'
            },
        ]
    }
}

Response Structure

  • (dict) --

    • ingestionStreamDetail (dict) --

      Details about the requested ingestion stream.

      • ingestionStreamId (string) --

        The unique identifier of the ingestion stream.

      • arn (string) --

        The Amazon Resource Name (ARN) of the ingestion stream.

      • collectionId (string) --

        The unique identifier of the collection that the ingestion stream belongs to.

      • ingestionStreamName (string) --

        The name of the ingestion stream.

      • indexName (string) --

        The name of the index that documents are ingested into.

      • failureIndexName (string) --

        The name of the index where documents that fail to be ingested are recorded.

      • source (dict) --

        The Amazon S3 source that the ingestion stream ingests data from.

        • s3 (dict) --

          The Amazon S3 source configuration for the ingestion stream.

          • sqsArn (string) --

            The Amazon Resource Name (ARN) of the Amazon SQS queue that receives Amazon S3 event notifications for the source buckets.

          • buckets (list) --

            The configuration for each Amazon S3 bucket that the ingestion stream ingests data from.

            • (dict) --

              S3 bucket configuration for SQS-driven ingestion. The scan mode controls only the optional one-time delta scan of existing objects in this bucket.

              • bucket (string) --

                The name or Amazon Resource Name (ARN) of the Amazon S3 bucket to ingest data from.

              • bucketOwnerAccountId (string) --

                The AWS account ID that owns the S3 bucket. Optional. If omitted, the control plane derives it from the account segment of sqsArn.

              • initialScan (string) --

                Specifies whether to perform a one-time scan of the objects that already exist in the bucket before the stream begins processing Amazon S3 event notifications. If DISABLED, only new objects are ingested.

      • roleArn (string) --

        The Amazon Resource Name (ARN) of the IAM role that OpenSearch Serverless assumes to read data from the source.

      • status (string) --

        The current lifecycle status of the ingestion stream. For example, CREATING, ACTIVE, or DELETING.

      • state (string) --

        The current processing state of the ingestion stream. For example, STARTING or INGESTION_IN_PROGRESS.

      • statistics (dict) --

        Processing statistics for the ingestion stream, such as the number of Amazon S3 objects discovered, ingested, and failed.

        • s3 (dict) --

          Processing statistics for the Amazon S3 source.

          • objectsDiscovered (integer) --

            The number of Amazon S3 objects discovered for ingestion.

          • objectsIngested (integer) --

            The number of Amazon S3 objects successfully ingested.

          • objectsFailed (integer) --

            The number of Amazon S3 objects that failed to be ingested.

      • revision (integer) --

        The revision number of the ingestion stream.

      • createdAt (datetime) --

        The date and time when the ingestion stream was created.

      • updatedAt (datetime) --

        The date and time when the ingestion stream was last updated.

      • failureReasons (list) --

        The reasons that the ingestion stream reached a terminal failure state, if applicable.

        • (dict) --

          A terminal failure reason explaining why an ingestion stopped.

          • code (string) --

            The code that identifies the reason for the failure.

          • message (string) --

            A description of the reason for the failure.

      • activeAlerts (list) --

        Active alerts about operational issues that currently affect the ingestion stream. Alerts are cleared when the issue is resolved.

        • (dict) --

          An active alert about a running ingestion — operational issues or updates to processing state. Cleared when the issue resolves.

          • code (string) --

            The code that identifies the alert.

          • message (string) --

            A description of the alert.

DeleteIngestionStream (new) Link ¶

Deletes an ingestion stream. Documents that were already indexed by the stream remain in the index.

See also: AWS API Documentation

Request Syntax

client.delete_ingestion_stream(
    collectionId='string',
    ingestionStreamId='string',
    clientToken='string'
)
type collectionId:

string

param collectionId:

[REQUIRED]

The unique identifier of the collection that the ingestion stream belongs to.

type ingestionStreamId:

string

param ingestionStreamId:

[REQUIRED]

The unique identifier of the ingestion stream to delete.

type clientToken:

string

param clientToken:

Unique, case-sensitive identifier to ensure idempotency of the request.

This field is autopopulated if not provided.

rtype:

dict

returns:

Response Syntax

{
    'ingestionStreamId': 'string',
    'collectionId': 'string',
    'ingestionStreamName': 'string',
    'status': 'CREATING'|'ACTIVE'|'DELETING'|'DELETED'|'CREATE_FAILED'|'DELETE_FAILED'
}

Response Structure

  • (dict) --

    • ingestionStreamId (string) --

      The unique identifier of the deleted ingestion stream.

    • collectionId (string) --

      The unique identifier of the collection that the ingestion stream belonged to.

    • ingestionStreamName (string) --

      The name of the deleted ingestion stream.

    • status (string) --

      The current status of the ingestion stream.

ListIngestionJobs (new) Link ¶

Returns a list of ingestion jobs in a collection.

See also: AWS API Documentation

Request Syntax

client.list_ingestion_jobs(
    collectionId='string',
    nextToken='string',
    maxResults=123
)
type collectionId:

string

param collectionId:

[REQUIRED]

The unique identifier of the collection to list ingestion jobs for.

type nextToken:

string

param nextToken:

If your initial ListIngestionJobs operation returns a nextToken, you can include the returned nextToken in subsequent ListIngestionJobs operations, which returns results in the next page.

type maxResults:

integer

param maxResults:

The maximum number of results to return. Valid values are 1–100. If you don't specify a value, the default is 100. You can use nextToken to get the next page of results.

A response can contain fewer results than maxResults, or no results, and still include a nextToken. To retrieve all results, continue to make the call with the returned nextToken until no nextToken is returned.

rtype:

dict

returns:

Response Syntax

{
    'ingestionJobSummaries': [
        {
            'ingestionJobId': 'string',
            'arn': 'string',
            'ingestionJobName': 'string',
            'indexName': 'string',
            'status': 'CREATING'|'ACTIVE'|'COMPLETED'|'CANCELLING'|'CANCELLED'|'FAILED'|'CREATE_FAILED'|'CANCEL_FAILED',
            'createdAt': datetime(2015, 1, 1),
            'updatedAt': datetime(2015, 1, 1)
        },
    ],
    'nextToken': 'string'
}

Response Structure

  • (dict) --

    • ingestionJobSummaries (list) --

      Summary information about each ingestion job.

      • (dict) --

        Summary information about an ingestion job.

        • ingestionJobId (string) --

          The unique identifier of the ingestion job.

        • arn (string) --

          The Amazon Resource Name (ARN) of the ingestion job.

        • ingestionJobName (string) --

          The name of the ingestion job.

        • indexName (string) --

          The name of the index that documents are ingested into.

        • status (string) --

          The current lifecycle status of the ingestion job.

        • createdAt (datetime) --

          The date and time when the ingestion job was created.

        • updatedAt (datetime) --

          The date and time when the ingestion job was last updated.

    • nextToken (string) --

      When nextToken is returned, there are more results available. The value of nextToken is a unique pagination token for each page. Make the call again using the returned token to retrieve the next page.

ListIngestionStreams (new) Link ¶

Returns a list of ingestion streams in a collection.

See also: AWS API Documentation

Request Syntax

client.list_ingestion_streams(
    collectionId='string',
    nextToken='string',
    maxResults=123
)
type collectionId:

string

param collectionId:

[REQUIRED]

The unique identifier of the collection to list ingestion streams for.

type nextToken:

string

param nextToken:

If your initial ListIngestionStreams operation returns a nextToken, you can include the returned nextToken in subsequent ListIngestionStreams operations, which returns results in the next page.

type maxResults:

integer

param maxResults:

The maximum number of results to return. Valid values are 1–100. If you don't specify a value, the default is 100. You can use nextToken to get the next page of results.

A response can contain fewer results than maxResults, or no results, and still include a nextToken. To retrieve all results, continue to make the call with the returned nextToken until no nextToken is returned.

rtype:

dict

returns:

Response Syntax

{
    'ingestionStreamSummaries': [
        {
            'ingestionStreamId': 'string',
            'arn': 'string',
            'ingestionStreamName': 'string',
            'indexName': 'string',
            'status': 'CREATING'|'ACTIVE'|'DELETING'|'DELETED'|'CREATE_FAILED'|'DELETE_FAILED',
            'createdAt': datetime(2015, 1, 1),
            'updatedAt': datetime(2015, 1, 1)
        },
    ],
    'nextToken': 'string'
}

Response Structure

  • (dict) --

    • ingestionStreamSummaries (list) --

      Summary information about each ingestion stream.

      • (dict) --

        Summary information about an ingestion stream.

        • ingestionStreamId (string) --

          The unique identifier of the ingestion stream.

        • arn (string) --

          The Amazon Resource Name (ARN) of the ingestion stream.

        • ingestionStreamName (string) --

          The name of the ingestion stream.

        • indexName (string) --

          The name of the index that documents are ingested into.

        • status (string) --

          The current lifecycle status of the ingestion stream.

        • createdAt (datetime) --

          The date and time when the ingestion stream was created.

        • updatedAt (datetime) --

          The date and time when the ingestion stream was last updated.

    • nextToken (string) --

      When nextToken is returned, there are more results available. The value of nextToken is a unique pagination token for each page. Make the call again using the returned token to retrieve the next page.

CancelIngestionJob (new) Link ¶

Cancels an in-progress ingestion job. Documents that were already indexed before the cancellation remain in the index.

See also: AWS API Documentation

Request Syntax

client.cancel_ingestion_job(
    collectionId='string',
    ingestionJobId='string',
    clientToken='string'
)
type collectionId:

string

param collectionId:

[REQUIRED]

The unique identifier of the collection that the ingestion job belongs to.

type ingestionJobId:

string

param ingestionJobId:

[REQUIRED]

The unique identifier of the ingestion job to cancel.

type clientToken:

string

param clientToken:

Unique, case-sensitive identifier to ensure idempotency of the request.

This field is autopopulated if not provided.

rtype:

dict

returns:

Response Syntax

{
    'ingestionJobId': 'string',
    'collectionId': 'string',
    'ingestionJobName': 'string',
    'status': 'CREATING'|'ACTIVE'|'COMPLETED'|'CANCELLING'|'CANCELLED'|'FAILED'|'CREATE_FAILED'|'CANCEL_FAILED'
}

Response Structure

  • (dict) --

    • ingestionJobId (string) --

      The unique identifier of the cancelled ingestion job.

    • collectionId (string) --

      The unique identifier of the collection that the ingestion job belongs to.

    • ingestionJobName (string) --

      The name of the cancelled ingestion job.

    • status (string) --

      The current status of the ingestion job.

GetIngestionJob (new) Link ¶

Returns information about an ingestion job, including its configuration, status, and processing state.

See also: AWS API Documentation

Request Syntax

client.get_ingestion_job(
    collectionId='string',
    ingestionJobId='string'
)
type collectionId:

string

param collectionId:

[REQUIRED]

The unique identifier of the collection that the ingestion job belongs to.

type ingestionJobId:

string

param ingestionJobId:

[REQUIRED]

The unique identifier of the ingestion job to retrieve.

rtype:

dict

returns:

Response Syntax

{
    'ingestionJobDetail': {
        'ingestionJobId': 'string',
        'arn': 'string',
        'collectionId': 'string',
        'ingestionJobName': 'string',
        'indexName': 'string',
        'failureIndexName': 'string',
        'source': {
            's3': {
                'bucket': 'string',
                'bucketOwnerAccountId': 'string',
                'keyPrefix': 'string'
            }
        },
        'status': 'CREATING'|'ACTIVE'|'COMPLETED'|'CANCELLING'|'CANCELLED'|'FAILED'|'CREATE_FAILED'|'CANCEL_FAILED',
        'state': 'STARTING'|'SCANNING'|'INGESTION_IN_PROGRESS'|'INGESTION_COMPLETE'|'STOPPED'|'FAILED',
        'statistics': {
            's3': {
                'objectsDiscovered': 123,
                'objectsIngested': 123,
                'objectsFailed': 123
            }
        },
        'revision': 123,
        'createdAt': datetime(2015, 1, 1),
        'updatedAt': datetime(2015, 1, 1),
        'failureReasons': [
            {
                'code': 'SOURCE_ACCESS_DENIED'|'SOURCE_RESOURCE_NOT_FOUND'|'DESTINATION_ACCESS_DENIED'|'DESTINATION_RESOURCE_NOT_FOUND'|'INTERNAL_SERVER_ERROR',
                'message': 'string'
            },
        ],
        'activeAlerts': [
            {
                'code': 'SOURCE_ACCESS_DENIED'|'SOURCE_RESOURCE_NOT_FOUND'|'DESTINATION_ACCESS_DENIED'|'DESTINATION_RESOURCE_NOT_FOUND'|'INTERNAL_SERVER_ERROR',
                'message': 'string'
            },
        ]
    }
}

Response Structure

  • (dict) --

    • ingestionJobDetail (dict) --

      Details about the requested ingestion job.

      • ingestionJobId (string) --

        The unique identifier of the ingestion job.

      • arn (string) --

        The Amazon Resource Name (ARN) of the ingestion job.

      • collectionId (string) --

        The unique identifier of the collection that the ingestion job belongs to.

      • ingestionJobName (string) --

        The name of the ingestion job.

      • indexName (string) --

        The name of the index that documents are ingested into.

      • failureIndexName (string) --

        The name of the index where documents that fail to be ingested are recorded.

      • source (dict) --

        The Amazon S3 source that the ingestion job ingests data from.

        • s3 (dict) --

          The Amazon S3 source configuration for the ingestion job.

          • bucket (string) --

            The name or Amazon Resource Name (ARN) of the Amazon S3 bucket to ingest data from.

          • bucketOwnerAccountId (string) --

            The Amazon Web Services account ID that owns the Amazon S3 bucket.

          • keyPrefix (string) --

            A key prefix that limits ingestion to objects in the Amazon S3 bucket whose keys begin with the prefix.

      • status (string) --

        The current lifecycle status of the ingestion job. For example, CREATING, ACTIVE, COMPLETED, or CANCELLED.

      • state (string) --

        The current processing state of the ingestion job. For example, SCANNING, INGESTION_IN_PROGRESS, or INGESTION_COMPLETE.

      • statistics (dict) --

        Processing statistics for the ingestion job, such as the number of Amazon S3 objects discovered, ingested, and failed.

        • s3 (dict) --

          Processing statistics for the Amazon S3 source.

          • objectsDiscovered (integer) --

            The number of Amazon S3 objects discovered for ingestion.

          • objectsIngested (integer) --

            The number of Amazon S3 objects successfully ingested.

          • objectsFailed (integer) --

            The number of Amazon S3 objects that failed to be ingested.

      • revision (integer) --

        The revision number of the ingestion job.

      • createdAt (datetime) --

        The date and time when the ingestion job was created.

      • updatedAt (datetime) --

        The date and time when the ingestion job was last updated.

      • failureReasons (list) --

        The reasons that the ingestion job reached a terminal failure state, if applicable.

        • (dict) --

          A terminal failure reason explaining why an ingestion stopped.

          • code (string) --

            The code that identifies the reason for the failure.

          • message (string) --

            A description of the reason for the failure.

      • activeAlerts (list) --

        Active alerts about operational issues that currently affect the ingestion job. Alerts are cleared when the issue is resolved.

        • (dict) --

          An active alert about a running ingestion — operational issues or updates to processing state. Cleared when the issue resolves.

          • code (string) --

            The code that identifies the alert.

          • message (string) --

            A description of the alert.

CreateIngestionStream (new) Link ¶

Creates an ingestion stream that continuously ingests data from an Amazon S3 bucket into an index in an OpenSearch Serverless collection. An ingestion stream processes Amazon S3 event notifications delivered to an Amazon SQS queue and ingests new objects as they arrive. The stream continues to ingest data until you delete it. To perform a one-time bulk ingestion instead, use CreateIngestionJob.

See also: AWS API Documentation

Request Syntax

client.create_ingestion_stream(
    collectionId='string',
    ingestionStreamName='string',
    indexName='string',
    failureIndexName='string',
    source={
        's3': {
            'sqsArn': 'string',
            'buckets': [
                {
                    'bucket': 'string',
                    'bucketOwnerAccountId': 'string',
                    'initialScan': 'ENABLED'|'DISABLED'
                },
            ]
        }
    },
    roleArn='string',
    tags=[
        {
            'key': 'string',
            'value': 'string'
        },
    ],
    clientToken='string'
)
type collectionId:

string

param collectionId:

[REQUIRED]

The unique identifier of the collection to ingest data into.

type ingestionStreamName:

string

param ingestionStreamName:

[REQUIRED]

The name of the ingestion stream. The name must be unique within the collection.

type indexName:

string

param indexName:

[REQUIRED]

The name of the index to ingest documents into.

type failureIndexName:

string

param failureIndexName:

The name of the index where documents that fail to be ingested are recorded.

type source:

dict

param source:

[REQUIRED]

The Amazon S3 source to ingest data from, including the Amazon SQS queue that receives S3 event notifications.

  • s3 (dict) -- [REQUIRED]

    The Amazon S3 source configuration for the ingestion stream.

    • sqsArn (string) -- [REQUIRED]

      The Amazon Resource Name (ARN) of the Amazon SQS queue that receives Amazon S3 event notifications for the source buckets.

    • buckets (list) --

      The configuration for each Amazon S3 bucket that the ingestion stream ingests data from.

      • (dict) --

        S3 bucket configuration for SQS-driven ingestion. The scan mode controls only the optional one-time delta scan of existing objects in this bucket.

        • bucket (string) -- [REQUIRED]

          The name or Amazon Resource Name (ARN) of the Amazon S3 bucket to ingest data from.

        • bucketOwnerAccountId (string) --

          The AWS account ID that owns the S3 bucket. Optional. If omitted, the control plane derives it from the account segment of sqsArn.

        • initialScan (string) --

          Specifies whether to perform a one-time scan of the objects that already exist in the bucket before the stream begins processing Amazon S3 event notifications. If DISABLED, only new objects are ingested.

type roleArn:

string

param roleArn:

[REQUIRED]

The Amazon Resource Name (ARN) of the IAM role that OpenSearch Serverless assumes to read data from the source. The role must be in the same account as the collection.

type tags:

list

param tags:

An arbitrary set of tags (key–value pairs) to associate with the ingestion stream.

  • (dict) --

    A map of key-value pairs associated to an OpenSearch Serverless resource.

    • key (string) -- [REQUIRED]

      The key to use in the tag.

    • value (string) -- [REQUIRED]

      The value of the tag.

type clientToken:

string

param clientToken:

Unique, case-sensitive identifier to ensure idempotency of the request.

This field is autopopulated if not provided.

rtype:

dict

returns:

Response Syntax

{
    'ingestionStreamDetail': {
        'ingestionStreamId': 'string',
        'arn': 'string',
        'collectionId': 'string',
        'ingestionStreamName': 'string',
        'indexName': 'string',
        'failureIndexName': 'string',
        'source': {
            's3': {
                'sqsArn': 'string',
                'buckets': [
                    {
                        'bucket': 'string',
                        'bucketOwnerAccountId': 'string',
                        'initialScan': 'ENABLED'|'DISABLED'
                    },
                ]
            }
        },
        'roleArn': 'string',
        'status': 'CREATING'|'ACTIVE'|'DELETING'|'DELETED'|'CREATE_FAILED'|'DELETE_FAILED',
        'state': 'STARTING'|'SCANNING'|'INGESTION_IN_PROGRESS'|'INGESTION_COMPLETE'|'STOPPED'|'FAILED',
        'statistics': {
            's3': {
                'objectsDiscovered': 123,
                'objectsIngested': 123,
                'objectsFailed': 123
            }
        },
        'revision': 123,
        'createdAt': datetime(2015, 1, 1),
        'updatedAt': datetime(2015, 1, 1),
        'failureReasons': [
            {
                'code': 'SOURCE_ACCESS_DENIED'|'SOURCE_RESOURCE_NOT_FOUND'|'DESTINATION_ACCESS_DENIED'|'DESTINATION_RESOURCE_NOT_FOUND'|'INTERNAL_SERVER_ERROR',
                'message': 'string'
            },
        ],
        'activeAlerts': [
            {
                'code': 'SOURCE_ACCESS_DENIED'|'SOURCE_RESOURCE_NOT_FOUND'|'DESTINATION_ACCESS_DENIED'|'DESTINATION_RESOURCE_NOT_FOUND'|'INTERNAL_SERVER_ERROR',
                'message': 'string'
            },
        ]
    }
}

Response Structure

  • (dict) --

    • ingestionStreamDetail (dict) --

      Details about the created ingestion stream.

      • ingestionStreamId (string) --

        The unique identifier of the ingestion stream.

      • arn (string) --

        The Amazon Resource Name (ARN) of the ingestion stream.

      • collectionId (string) --

        The unique identifier of the collection that the ingestion stream belongs to.

      • ingestionStreamName (string) --

        The name of the ingestion stream.

      • indexName (string) --

        The name of the index that documents are ingested into.

      • failureIndexName (string) --

        The name of the index where documents that fail to be ingested are recorded.

      • source (dict) --

        The Amazon S3 source that the ingestion stream ingests data from.

        • s3 (dict) --

          The Amazon S3 source configuration for the ingestion stream.

          • sqsArn (string) --

            The Amazon Resource Name (ARN) of the Amazon SQS queue that receives Amazon S3 event notifications for the source buckets.

          • buckets (list) --

            The configuration for each Amazon S3 bucket that the ingestion stream ingests data from.

            • (dict) --

              S3 bucket configuration for SQS-driven ingestion. The scan mode controls only the optional one-time delta scan of existing objects in this bucket.

              • bucket (string) --

                The name or Amazon Resource Name (ARN) of the Amazon S3 bucket to ingest data from.

              • bucketOwnerAccountId (string) --

                The AWS account ID that owns the S3 bucket. Optional. If omitted, the control plane derives it from the account segment of sqsArn.

              • initialScan (string) --

                Specifies whether to perform a one-time scan of the objects that already exist in the bucket before the stream begins processing Amazon S3 event notifications. If DISABLED, only new objects are ingested.

      • roleArn (string) --

        The Amazon Resource Name (ARN) of the IAM role that OpenSearch Serverless assumes to read data from the source.

      • status (string) --

        The current lifecycle status of the ingestion stream. For example, CREATING, ACTIVE, or DELETING.

      • state (string) --

        The current processing state of the ingestion stream. For example, STARTING or INGESTION_IN_PROGRESS.

      • statistics (dict) --

        Processing statistics for the ingestion stream, such as the number of Amazon S3 objects discovered, ingested, and failed.

        • s3 (dict) --

          Processing statistics for the Amazon S3 source.

          • objectsDiscovered (integer) --

            The number of Amazon S3 objects discovered for ingestion.

          • objectsIngested (integer) --

            The number of Amazon S3 objects successfully ingested.

          • objectsFailed (integer) --

            The number of Amazon S3 objects that failed to be ingested.

      • revision (integer) --

        The revision number of the ingestion stream.

      • createdAt (datetime) --

        The date and time when the ingestion stream was created.

      • updatedAt (datetime) --

        The date and time when the ingestion stream was last updated.

      • failureReasons (list) --

        The reasons that the ingestion stream reached a terminal failure state, if applicable.

        • (dict) --

          A terminal failure reason explaining why an ingestion stopped.

          • code (string) --

            The code that identifies the reason for the failure.

          • message (string) --

            A description of the reason for the failure.

      • activeAlerts (list) --

        Active alerts about operational issues that currently affect the ingestion stream. Alerts are cleared when the issue is resolved.

        • (dict) --

          An active alert about a running ingestion — operational issues or updates to processing state. Cleared when the issue resolves.

          • code (string) --

            The code that identifies the alert.

          • message (string) --

            A description of the alert.

CreateIngestionJob (new) Link ¶

Creates an ingestion job that performs a one-time bulk ingestion of data from an Amazon S3 bucket into an index in an OpenSearch Serverless collection. An ingestion job scans the source bucket, indexes the documents it finds, and then completes. To continuously ingest new data as it arrives, use CreateIngestionStream instead.

See also: AWS API Documentation

Request Syntax

client.create_ingestion_job(
    collectionId='string',
    ingestionJobName='string',
    indexName='string',
    failureIndexName='string',
    source={
        's3': {
            'bucket': 'string',
            'bucketOwnerAccountId': 'string',
            'keyPrefix': 'string'
        }
    },
    tags=[
        {
            'key': 'string',
            'value': 'string'
        },
    ],
    clientToken='string'
)
type collectionId:

string

param collectionId:

[REQUIRED]

The unique identifier of the collection to ingest data into.

type ingestionJobName:

string

param ingestionJobName:

[REQUIRED]

The name of the ingestion job. The name must be unique within the collection.

type indexName:

string

param indexName:

[REQUIRED]

The name of the index to ingest documents into.

type failureIndexName:

string

param failureIndexName:

The name of the index where documents that fail to be ingested are recorded.

type source:

dict

param source:

[REQUIRED]

The Amazon S3 source to ingest data from.

  • s3 (dict) -- [REQUIRED]

    The Amazon S3 source configuration for the ingestion job.

    • bucket (string) --

      The name or Amazon Resource Name (ARN) of the Amazon S3 bucket to ingest data from.

    • bucketOwnerAccountId (string) --

      The Amazon Web Services account ID that owns the Amazon S3 bucket.

    • keyPrefix (string) --

      A key prefix that limits ingestion to objects in the Amazon S3 bucket whose keys begin with the prefix.

type tags:

list

param tags:

An arbitrary set of tags (key–value pairs) to associate with the ingestion job.

  • (dict) --

    A map of key-value pairs associated to an OpenSearch Serverless resource.

    • key (string) -- [REQUIRED]

      The key to use in the tag.

    • value (string) -- [REQUIRED]

      The value of the tag.

type clientToken:

string

param clientToken:

Unique, case-sensitive identifier to ensure idempotency of the request.

This field is autopopulated if not provided.

rtype:

dict

returns:

Response Syntax

{
    'ingestionJobDetail': {
        'ingestionJobId': 'string',
        'arn': 'string',
        'collectionId': 'string',
        'ingestionJobName': 'string',
        'indexName': 'string',
        'failureIndexName': 'string',
        'source': {
            's3': {
                'bucket': 'string',
                'bucketOwnerAccountId': 'string',
                'keyPrefix': 'string'
            }
        },
        'status': 'CREATING'|'ACTIVE'|'COMPLETED'|'CANCELLING'|'CANCELLED'|'FAILED'|'CREATE_FAILED'|'CANCEL_FAILED',
        'state': 'STARTING'|'SCANNING'|'INGESTION_IN_PROGRESS'|'INGESTION_COMPLETE'|'STOPPED'|'FAILED',
        'statistics': {
            's3': {
                'objectsDiscovered': 123,
                'objectsIngested': 123,
                'objectsFailed': 123
            }
        },
        'revision': 123,
        'createdAt': datetime(2015, 1, 1),
        'updatedAt': datetime(2015, 1, 1),
        'failureReasons': [
            {
                'code': 'SOURCE_ACCESS_DENIED'|'SOURCE_RESOURCE_NOT_FOUND'|'DESTINATION_ACCESS_DENIED'|'DESTINATION_RESOURCE_NOT_FOUND'|'INTERNAL_SERVER_ERROR',
                'message': 'string'
            },
        ],
        'activeAlerts': [
            {
                'code': 'SOURCE_ACCESS_DENIED'|'SOURCE_RESOURCE_NOT_FOUND'|'DESTINATION_ACCESS_DENIED'|'DESTINATION_RESOURCE_NOT_FOUND'|'INTERNAL_SERVER_ERROR',
                'message': 'string'
            },
        ]
    }
}

Response Structure

  • (dict) --

    • ingestionJobDetail (dict) --

      Details about the created ingestion job.

      • ingestionJobId (string) --

        The unique identifier of the ingestion job.

      • arn (string) --

        The Amazon Resource Name (ARN) of the ingestion job.

      • collectionId (string) --

        The unique identifier of the collection that the ingestion job belongs to.

      • ingestionJobName (string) --

        The name of the ingestion job.

      • indexName (string) --

        The name of the index that documents are ingested into.

      • failureIndexName (string) --

        The name of the index where documents that fail to be ingested are recorded.

      • source (dict) --

        The Amazon S3 source that the ingestion job ingests data from.

        • s3 (dict) --

          The Amazon S3 source configuration for the ingestion job.

          • bucket (string) --

            The name or Amazon Resource Name (ARN) of the Amazon S3 bucket to ingest data from.

          • bucketOwnerAccountId (string) --

            The Amazon Web Services account ID that owns the Amazon S3 bucket.

          • keyPrefix (string) --

            A key prefix that limits ingestion to objects in the Amazon S3 bucket whose keys begin with the prefix.

      • status (string) --

        The current lifecycle status of the ingestion job. For example, CREATING, ACTIVE, COMPLETED, or CANCELLED.

      • state (string) --

        The current processing state of the ingestion job. For example, SCANNING, INGESTION_IN_PROGRESS, or INGESTION_COMPLETE.

      • statistics (dict) --

        Processing statistics for the ingestion job, such as the number of Amazon S3 objects discovered, ingested, and failed.

        • s3 (dict) --

          Processing statistics for the Amazon S3 source.

          • objectsDiscovered (integer) --

            The number of Amazon S3 objects discovered for ingestion.

          • objectsIngested (integer) --

            The number of Amazon S3 objects successfully ingested.

          • objectsFailed (integer) --

            The number of Amazon S3 objects that failed to be ingested.

      • revision (integer) --

        The revision number of the ingestion job.

      • createdAt (datetime) --

        The date and time when the ingestion job was created.

      • updatedAt (datetime) --

        The date and time when the ingestion job was last updated.

      • failureReasons (list) --

        The reasons that the ingestion job reached a terminal failure state, if applicable.

        • (dict) --

          A terminal failure reason explaining why an ingestion stopped.

          • code (string) --

            The code that identifies the reason for the failure.

          • message (string) --

            A description of the reason for the failure.

      • activeAlerts (list) --

        Active alerts about operational issues that currently affect the ingestion job. Alerts are cleared when the issue is resolved.

        • (dict) --

          An active alert about a running ingestion — operational issues or updates to processing state. Cleared when the issue resolves.

          • code (string) --

            The code that identifies the alert.

          • message (string) --

            A description of the alert.

UpdateCollection (updated) Link ¶
Changes (request, response)
Request
{'encryptionConfig': {'aWSOwnedKey': 'boolean', 'kmsKeyArn': 'string'}}
Response
{'updateCollectionDetail': {'kmsKeyArn': 'string'}}

Updates an OpenSearch Serverless collection.

See also: AWS API Documentation

Request Syntax

client.update_collection(
    id='string',
    description='string',
    vectorOptions={
        'ServerlessVectorAcceleration': 'ENABLED'|'DISABLED'|'ALLOWED'
    },
    deletionProtection='ENABLED'|'DISABLED',
    encryptionConfig={
        'aWSOwnedKey': True|False,
        'kmsKeyArn': 'string'
    },
    clientToken='string'
)
type id:

string

param id:

[REQUIRED]

The unique identifier of the collection.

type description:

string

param description:

A description of the collection.

type vectorOptions:

dict

param vectorOptions:

Configuration options for vector search capabilities in the collection.

  • ServerlessVectorAcceleration (string) -- [REQUIRED]

    Specifies whether serverless vector acceleration is enabled for the collection.

type deletionProtection:

string

param deletionProtection:

Indicates whether to enable or disable deletion protection for the collection. When set to ENABLED, the collection cannot be deleted.

type encryptionConfig:

dict

param encryptionConfig:

Encryption settings for a collection.

  • aWSOwnedKey (boolean) --

    Indicates whether to use an Amazon Web Services-owned key for encryption.

  • kmsKeyArn (string) --

    The ARN of the Amazon Web Services Key Management Service key used to encrypt the collection.

type clientToken:

string

param clientToken:

Unique, case-sensitive identifier to ensure idempotency of the request.

This field is autopopulated if not provided.

rtype:

dict

returns:

Response Syntax

{
    'updateCollectionDetail': {
        'id': 'string',
        'name': 'string',
        'status': 'CREATING'|'UPDATING'|'DELETING'|'ACTIVE'|'FAILED'|'UPDATE_FAILED',
        'type': 'SEARCH'|'TIMESERIES'|'VECTORSEARCH',
        'description': 'string',
        'vectorOptions': {
            'ServerlessVectorAcceleration': 'ENABLED'|'DISABLED'|'ALLOWED'
        },
        'kmsKeyArn': 'string',
        'arn': 'string',
        'createdDate': 123,
        'lastModifiedDate': 123,
        'deletionProtection': 'ENABLED'|'DISABLED'
    }
}

Response Structure

  • (dict) --

    • updateCollectionDetail (dict) --

      Details about the updated collection.

      • id (string) --

        The unique identifier of the collection.

      • name (string) --

        The name of the collection.

      • status (string) --

        The current status of the collection.

      • type (string) --

        The collection type.

      • description (string) --

        The description of the collection.

      • vectorOptions (dict) --

        Configuration options for vector search capabilities in the collection.

        • ServerlessVectorAcceleration (string) --

          Specifies whether serverless vector acceleration is enabled for the collection.

      • kmsKeyArn (string) --

        The Amazon Resource Name (ARN) of the Amazon Web Services Key Management Service (Amazon Web Services KMS) key used to encrypt the collection.

      • arn (string) --

        The Amazon Resource Name (ARN) of the collection.

      • createdDate (integer) --

        The date and time when the collection was created.

      • lastModifiedDate (integer) --

        The date and time when the collection was last modified.

      • deletionProtection (string) --

        Indicates whether deletion protection is ENABLED or DISABLED for the collection.