Submodules

airbyte_cdk.utils.airbyte_secrets_utils module

airbyte_cdk.utils.airbyte_secrets_utils.filter_secrets(string: str) → str: Filter secrets from a string by replacing them with ****

airbyte_cdk.utils.airbyte_secrets_utils.get_secret_paths(spec: Mapping[str, Any]) → List[List[str]]

airbyte_cdk.utils.airbyte_secrets_utils.get_secrets(connection_specification: Mapping[str, Any], config: Mapping[str, Any]) → List[Any]: Get a list of secret values from the source config based on the source specification :type connection_specification: the connection_specification field of an AirbyteSpecification i.e the JSONSchema definition

airbyte_cdk.utils.airbyte_secrets_utils.update_secrets(secrets: List[str]): Update the list of secrets to be replaced

airbyte_cdk.utils.event_timing module

class airbyte_cdk.utils.event_timing.Event(name: str, start: float = <factory>, end: Optional[float] = None)

Bases: object

property duration: float: Returns the elapsed time in seconds or positive infinity if event was never finished

end: Optional[float] = None

finish()

name: str

start: float

class airbyte_cdk.utils.event_timing.EventTimer(name)

Bases: object

Simple nanosecond resolution event timer for debugging, initially intended to be used to record streams execution time for a source.

Event nesting follows a LIFO pattern, so finish will apply to the last started event.

finish_event(): Finish the current event and pop it from the stack.

report(order_by='name')

Parameters: order_by – ‘name’ or ‘duration’

start_event(name): Start a new event and push it to the stack.

airbyte_cdk.utils.event_timing.create_timer(name): Creates a new EventTimer as a context manager to improve code readability.

airbyte_cdk.utils.traced_exception module

exception airbyte_cdk.utils.traced_exception.AirbyteTracedException(internal_message: Optional[str] = None, message: Optional[str] = None, failure_type: airbyte_protocol.models.airbyte_protocol.FailureType = FailureType.system_error, exception: Optional[BaseException] = None)

Bases: Exception

An exception that should be emitted as an AirbyteTraceMessage

as_airbyte_message() → airbyte_protocol.models.airbyte_protocol.AirbyteMessage: Builds an AirbyteTraceMessage from the exception

as_connection_status_message() → airbyte_protocol.models.airbyte_protocol.AirbyteMessage

emit_message() → None: Prints the exception as an AirbyteTraceMessage. Note that this will be called automatically on uncaught exceptions when using the airbyte_cdk entrypoint.

classmethod from_exception(exc: BaseException, *args, **kwargs) → airbyte_cdk.utils.traced_exception.AirbyteTracedException: Helper to create an AirbyteTracedException from an existing exception :param exc: the exception that caused the error

Module contents

exception airbyte_cdk.utils.AirbyteTracedException(internal_message: Optional[str] = None, message: Optional[str] = None, failure_type: airbyte_protocol.models.airbyte_protocol.FailureType = FailureType.system_error, exception: Optional[BaseException] = None)

Bases: Exception

An exception that should be emitted as an AirbyteTraceMessage

as_airbyte_message() → airbyte_protocol.models.airbyte_protocol.AirbyteMessage: Builds an AirbyteTraceMessage from the exception

as_connection_status_message() → airbyte_protocol.models.airbyte_protocol.AirbyteMessage

emit_message() → None: Prints the exception as an AirbyteTraceMessage. Note that this will be called automatically on uncaught exceptions when using the airbyte_cdk entrypoint.

classmethod from_exception(exc: BaseException, *args, **kwargs) → airbyte_cdk.utils.traced_exception.AirbyteTracedException: Helper to create an AirbyteTracedException from an existing exception :param exc: the exception that caused the error

class airbyte_cdk.utils.SchemaInferrer

Bases: object

This class is used to infer a JSON schema which fits all the records passed into it throughout its lifecycle via the accumulate method.

Instances of this class are stateful, meaning they build their inferred schemas from every record passed into the accumulate method.

accumulate(record: airbyte_protocol.models.airbyte_protocol.AirbyteRecordMessage) → None: Uses the input record to add to the inferred schemas maintained by this object

get_inferred_schemas() → Dict[str, Dict[str, Any]]: Returns the JSON schemas for all encountered streams inferred by inspecting all records passed via the accumulate method

get_stream_schema(stream_name: str) → Optional[Dict[str, Any]]: Returns the inferred JSON schema for the specified stream. Might be None if there were no records for the given stream name.

stream_to_builder: Dict[str, genson.schema.builder.SchemaBuilder]

airbyte_cdk.utils.is_cloud_environment() → bool

Returns True if the connector is running in a cloud environment, False otherwise.

The function checks the value of the DEPLOYMENT_MODE environment variable which is set by the platform. This function can be used to determine whether stricter security measures should be applied.