Submodules

airbyte_cdk.sources.declarative.stream_slicers.cartesian_product_stream_slicer module

class airbyte_cdk.sources.declarative.stream_slicers.cartesian_product_stream_slicer.CartesianProductStreamSlicer(stream_slicers: List[airbyte_cdk.sources.declarative.stream_slicers.stream_slicer.StreamSlicer], parameters: dataclasses.InitVar[Mapping[str, Any]])

Bases: airbyte_cdk.sources.declarative.stream_slicers.stream_slicer.StreamSlicer

Stream slicers that iterates over the cartesian product of input stream slicers Given 2 stream slicers with the following slices: A: [{“i”: 0}, {“i”: 1}, {“i”: 2}] B: [{“s”: “hello”}, {“s”: “world”}] the resulting stream slices are [

{“i”: 0, “s”: “hello”}, {“i”: 0, “s”: “world”}, {“i”: 1, “s”: “hello”}, {“i”: 1, “s”: “world”}, {“i”: 2, “s”: “hello”}, {“i”: 2, “s”: “world”},

]

stream_slicers

Underlying stream slicers. The RequestOptions (e.g: Request headers, parameters, etc..) returned by this slicer are the combination of the RequestOptions of its input slicers. If there are conflicts e.g: two slicers define the same header or request param, the conflict is resolved by taking the value from the first slicer, where ordering is determined by the order in which slicers were input to this composite slicer.

Type: List[StreamSlicer]

get_request_body_data(*, stream_state: Optional[Mapping[str, Any]] = None, stream_slice: Optional[Mapping[str, Any]] = None, next_page_token: Optional[Mapping[str, Any]] = None) → Mapping[str, Any]

Specifies how to populate the body of the request with a non-JSON payload.

If returns a ready text that it will be sent as is. If returns a dict that it will be converted to a urlencoded form. E.g. {“key1”: “value1”, “key2”: “value2”} => “key1=value1&key2=value2”

At the same time only one of the ‘request_body_data’ and ‘request_body_json’ functions can be overridden.

get_request_body_json(*, stream_state: Optional[Mapping[str, Any]] = None, stream_slice: Optional[Mapping[str, Any]] = None, next_page_token: Optional[Mapping[str, Any]] = None) → Optional[Mapping]

Specifies how to populate the body of the request with a JSON payload.

At the same time only one of the ‘request_body_data’ and ‘request_body_json’ functions can be overridden.

get_request_headers(*, stream_state: Optional[Mapping[str, Any]] = None, stream_slice: Optional[Mapping[str, Any]] = None, next_page_token: Optional[Mapping[str, Any]] = None) → Mapping[str, Any]: Return any non-auth headers. Authentication headers will overwrite any overlapping headers returned from this method.

get_request_params(*, stream_state: Optional[Mapping[str, Any]] = None, stream_slice: Optional[Mapping[str, Any]] = None, next_page_token: Optional[Mapping[str, Any]] = None) → Mapping[str, Any]

Specifies the query parameters that should be set on an outgoing HTTP request given the inputs.

E.g: you might want to define query parameters for paging if next_page_token is not None.

parameters: dataclasses.InitVar[Mapping[str, Any]]

stream_slicers: List[airbyte_cdk.sources.declarative.stream_slicers.stream_slicer.StreamSlicer]

stream_slices() → Iterable[Mapping[str, Any]]

Defines stream slices

Returns: List of stream slices

airbyte_cdk.sources.declarative.stream_slicers.datetime_stream_slicer module

airbyte_cdk.sources.declarative.stream_slicers.list_stream_slicer module

airbyte_cdk.sources.declarative.stream_slicers.single_slice module

airbyte_cdk.sources.declarative.stream_slicers.stream_slicer module

class airbyte_cdk.sources.declarative.stream_slicers.stream_slicer.StreamSlicer

Bases: airbyte_cdk.sources.declarative.requesters.request_options.request_options_provider.RequestOptionsProvider

Slices the stream into a subset of records. Slices enable state checkpointing and data retrieval parallelization.

The stream slicer keeps track of the cursor state as a dict of cursor_field -> cursor_value

See the stream slicing section of the docs for more information.

abstract stream_slices() → Iterable[Mapping[str, Any]]

Defines stream slices

Returns: List of stream slices

airbyte_cdk.sources.declarative.stream_slicers.substream_slicer module

Module contents

class airbyte_cdk.sources.declarative.stream_slicers.CartesianProductStreamSlicer(stream_slicers: List[airbyte_cdk.sources.declarative.stream_slicers.stream_slicer.StreamSlicer], parameters: dataclasses.InitVar[Mapping[str, Any]])

Bases: airbyte_cdk.sources.declarative.stream_slicers.stream_slicer.StreamSlicer

Stream slicers that iterates over the cartesian product of input stream slicers Given 2 stream slicers with the following slices: A: [{“i”: 0}, {“i”: 1}, {“i”: 2}] B: [{“s”: “hello”}, {“s”: “world”}] the resulting stream slices are [

{“i”: 0, “s”: “hello”}, {“i”: 0, “s”: “world”}, {“i”: 1, “s”: “hello”}, {“i”: 1, “s”: “world”}, {“i”: 2, “s”: “hello”}, {“i”: 2, “s”: “world”},

]

stream_slicers

Underlying stream slicers. The RequestOptions (e.g: Request headers, parameters, etc..) returned by this slicer are the combination of the RequestOptions of its input slicers. If there are conflicts e.g: two slicers define the same header or request param, the conflict is resolved by taking the value from the first slicer, where ordering is determined by the order in which slicers were input to this composite slicer.

Type: List[StreamSlicer]

get_request_body_data(*, stream_state: Optional[Mapping[str, Any]] = None, stream_slice: Optional[Mapping[str, Any]] = None, next_page_token: Optional[Mapping[str, Any]] = None) → Mapping[str, Any]

Specifies how to populate the body of the request with a non-JSON payload.

If returns a ready text that it will be sent as is. If returns a dict that it will be converted to a urlencoded form. E.g. {“key1”: “value1”, “key2”: “value2”} => “key1=value1&key2=value2”

At the same time only one of the ‘request_body_data’ and ‘request_body_json’ functions can be overridden.

get_request_body_json(*, stream_state: Optional[Mapping[str, Any]] = None, stream_slice: Optional[Mapping[str, Any]] = None, next_page_token: Optional[Mapping[str, Any]] = None) → Optional[Mapping]

Specifies how to populate the body of the request with a JSON payload.

At the same time only one of the ‘request_body_data’ and ‘request_body_json’ functions can be overridden.

get_request_headers(*, stream_state: Optional[Mapping[str, Any]] = None, stream_slice: Optional[Mapping[str, Any]] = None, next_page_token: Optional[Mapping[str, Any]] = None) → Mapping[str, Any]: Return any non-auth headers. Authentication headers will overwrite any overlapping headers returned from this method.

get_request_params(*, stream_state: Optional[Mapping[str, Any]] = None, stream_slice: Optional[Mapping[str, Any]] = None, next_page_token: Optional[Mapping[str, Any]] = None) → Mapping[str, Any]

Specifies the query parameters that should be set on an outgoing HTTP request given the inputs.

E.g: you might want to define query parameters for paging if next_page_token is not None.

parameters: dataclasses.InitVar[Mapping[str, Any]]

stream_slicers: List[airbyte_cdk.sources.declarative.stream_slicers.stream_slicer.StreamSlicer]

stream_slices() → Iterable[Mapping[str, Any]]

Defines stream slices

Returns: List of stream slices

class airbyte_cdk.sources.declarative.stream_slicers.StreamSlicer

Bases: airbyte_cdk.sources.declarative.requesters.request_options.request_options_provider.RequestOptionsProvider

Slices the stream into a subset of records. Slices enable state checkpointing and data retrieval parallelization.

The stream slicer keeps track of the cursor state as a dict of cursor_field -> cursor_value

See the stream slicing section of the docs for more information.

abstract stream_slices() → Iterable[Mapping[str, Any]]

Defines stream slices

Returns: List of stream slices