Ingestion & CDC interview questions

1. A.A change event representing a delete, often a key with a null value in Kafka, signaling downstream systems to remove the corresponding row.
2. B.A reusable integration component that extracts from a specific source or loads to a destination, the building block of managed ELT platforms.
3. C.A side channel where records that repeatedly fail processing are diverted so the main pipeline keeps flowing and bad data can be inspected later.
4. D.A CDC approach that uses database triggers to write changes to shadow tables, capturing every change but adding write overhead on the source.

Answer + AI explanation with Pro

1. A.tombstone record
2. B.trigger-based CDC
3. C.exactly-once delivery
4. D.Fivetran

Answer + AI explanation with Pro

1. A.trigger-based CDC — A CDC approach that polls rows whose updated_at is newer than the last run; simple but can miss hard deletes and out-of-order updates.
2. B.trigger-based CDC — A connector mode that fetches only records changed since the last sync using a cursor field, minimizing data transfer.
3. C.trigger-based CDC — A CDC approach that uses database triggers to write changes to shadow tables, capturing every change but adding write overhead on the source.
4. D.trigger-based CDC — A Debezium snapshot setting that reads the full current state of tables once before streaming ongoing log changes, giving a complete starting baseline.

Answer + AI explanation with Pro

1. A.An open-source and cloud ELT platform with a large connector catalog and a CDK for building custom source and destination connectors.
2. B.A CDC approach that polls rows whose updated_at is newer than the last run; simple but can miss hard deletes and out-of-order updates.
3. C.Automatically detecting and propagating source schema changes (new or altered columns) into the destination without breaking the pipeline.
4. D.A Debezium snapshot setting that reads the full current state of tables once before streaming ongoing log changes, giving a complete starting baseline.

Answer + AI explanation with Pro

1. A.dead-letter queue
2. B.Debezium
3. C.incremental sync
4. D.timestamp-based CDC

Answer + AI explanation with Pro

1. A.timestamp-based CDC — A CDC approach that polls rows whose updated_at is newer than the last run; simple but can miss hard deletes and out-of-order updates.
2. B.timestamp-based CDC — An initial bulk snapshot of a table followed by continuous log-based change streaming, the standard pattern Debezium uses to bootstrap then keep current.
3. C.timestamp-based CDC — A guarantee that each event affects the destination exactly once despite retries, achieved via idempotent writes or transactional offset commits.
4. D.timestamp-based CDC — A CDC approach that uses database triggers to write changes to shadow tables, capturing every change but adding write overhead on the source.

Answer + AI explanation with Pro

1. A.A Kafka message with a non-null key and null value that signals deletion of that key, used by compacted topics and CDC delete events.
2. B.An initial bulk snapshot of a table followed by continuous log-based change streaming, the standard pattern Debezium uses to bootstrap then keep current.
3. C.A Debezium snapshot setting that reads the full current state of tables once before streaming ongoing log changes, giving a complete starting baseline.
4. D.An open-source CDC platform that reads database transaction logs and streams row-level change events, commonly through Kafka Connect.

Answer + AI explanation with Pro

1. A.Snowpipe Streaming
2. B.incremental snapshot
3. C.Debezium
4. D.schema registry

Answer + AI explanation with Pro

1. A.Debezium — An open-source CDC platform that reads database transaction logs and streams row-level change events, commonly through Kafka Connect.
2. B.Debezium — An initial bulk snapshot of a table followed by continuous log-based change streaming, the standard pattern Debezium uses to bootstrap then keep current.
3. C.Debezium — A guarantee that each event affects the destination exactly once despite retries, achieved via idempotent writes or transactional offset commits.
4. D.Debezium — A connector mode that fetches only records changed since the last sync using a cursor field, minimizing data transfer.

Answer + AI explanation with Pro

1. A.Automatically detecting and propagating source schema changes (new or altered columns) into the destination without breaking the pipeline.
2. B.A framework of reusable source and sink connectors that move data between Kafka and external systems without custom producer or consumer code.
3. C.A framework of reusable source and sink connectors for streaming data in and out of Kafka without writing custom producer or consumer code.
4. D.A monotonically increasing position in a database transaction log (LSN) used by log-based CDC to resume exactly where it left off.

Answer + AI explanation with Pro

1. A.timestamp-based CDC
2. B.Snowpipe Streaming
3. C.trigger-based CDC
4. D.Kafka Connect

Answer + AI explanation with Pro

1. A.Kafka Connect — A CDC approach that polls rows whose updated_at is newer than the last run; simple but can miss hard deletes and out-of-order updates.
2. B.Kafka Connect — A change event representing a delete, often a key with a null value in Kafka, signaling downstream systems to remove the corresponding row.
3. C.Kafka Connect — Snowflake's rowset streaming ingestion that writes rows directly into tables with low latency, avoiding the file-staging step of classic Snowpipe.
4. D.Kafka Connect — A framework of reusable source and sink connectors for streaming data in and out of Kafka without writing custom producer or consumer code.

Answer + AI explanation with Pro

1. A.A managed ELT connector service that automates extraction and loading from SaaS and database sources into a warehouse with schema handling.
2. B.A change event representing a delete, often a key with a null value in Kafka, signaling downstream systems to remove the corresponding row.
3. C.A Kafka message with a non-null key and null value that signals deletion of that key, used by compacted topics and CDC delete events.
4. D.A Debezium snapshot setting that reads the full current state of tables once before streaming ongoing log changes, giving a complete starting baseline.

Answer + AI explanation with Pro

1. A.Debezium
2. B.trigger-based CDC
3. C.schema registry
4. D.Fivetran

Answer + AI explanation with Pro

1. A.Fivetran — An open-source and cloud ELT platform with a large connector catalog and a CDK for building custom source and destination connectors.
2. B.Fivetran — A connector mode that fetches only records changed since the last sync using a cursor field, minimizing data transfer.
3. C.Fivetran — Snowflake's rowset streaming ingestion that writes rows directly into tables with low latency, avoiding the file-staging step of classic Snowpipe.
4. D.Fivetran — A managed ELT connector service that automates extraction and loading from SaaS and database sources into a warehouse with schema handling.

Answer + AI explanation with Pro

1. A.An open-source CDC platform that reads database transaction logs and emits row-level change events to Kafka.
2. B.A sink that can process the same record more than once without changing the result, typically by upserting on a key, enabling safe retries.
3. C.A managed ELT connector service that automates extraction and loading from SaaS and database sources into a warehouse with schema handling.
4. D.An open-source and cloud ELT platform with a large connector catalog and a CDK for building custom source and destination connectors.

Answer + AI explanation with Pro

1. A.timestamp-based CDC
2. B.Kafka Connect
3. C.Airbyte
4. D.Snowpipe Streaming

Answer + AI explanation with Pro

1. A.Airbyte — A Debezium technique that chunks a table snapshot and interleaves it with live change streaming, avoiding long locks and allowing resumption.
2. B.Airbyte — An open-source and cloud ELT platform with a large connector catalog and a CDK for building custom source and destination connectors.
3. C.Airbyte — A guarantee that each source change is reflected exactly once downstream despite retries, typically achieved with idempotent writes and offset tracking.
4. D.Airbyte — An initial bulk snapshot of a table followed by continuous log-based change streaming, the standard pattern Debezium uses to bootstrap then keep current.

Answer + AI explanation with Pro

1. A.An open-source CDC platform that reads database transaction logs and streams row-level change events, commonly through Kafka Connect.
2. B.A framework of reusable source and sink connectors that move data between Kafka and external systems without custom producer or consumer code.
3. C.A Debezium technique that chunks a table snapshot and interleaves it with live change streaming, avoiding long locks and allowing resumption.
4. D.An initial bulk snapshot of a table followed by continuous log-based change streaming, the standard pattern Debezium uses to bootstrap then keep current.

Answer + AI explanation with Pro

1. A.snapshot plus streaming CDC
2. B.schema registry
3. C.log sequence number
4. D.incremental snapshot

Answer + AI explanation with Pro

1. A.snapshot plus streaming CDC — A CDC approach that polls rows whose updated_at is newer than the last run; simple but can miss hard deletes and out-of-order updates.
2. B.snapshot plus streaming CDC — An initial bulk snapshot of a table followed by continuous log-based change streaming, the standard pattern Debezium uses to bootstrap then keep current.
3. C.snapshot plus streaming CDC — Snowflake's rowset streaming ingestion that writes rows directly into tables with low latency, avoiding the file-staging step of classic Snowpipe.
4. D.snapshot plus streaming CDC — A guarantee that each event affects the destination exactly once despite retries, achieved via idempotent writes or transactional offset commits.

Answer + AI explanation with Pro

1. A.A change event representing a delete, often a key with a null value in Kafka, signaling downstream systems to remove the corresponding row.
2. B.A framework of reusable source and sink connectors for streaming data in and out of Kafka without writing custom producer or consumer code.
3. C.A managed ELT connector service that automates extraction and loading from SaaS and database sources into a warehouse with schema handling.
4. D.Snowflake's rowset streaming ingestion that writes rows directly into tables with low latency, avoiding the file-staging step of classic Snowpipe.

Answer + AI explanation with Pro

1. A.Fivetran
2. B.tombstone record
3. C.initial snapshot mode
4. D.schema drift handling

Answer + AI explanation with Pro

1. A.tombstone record — A change event representing a delete, often a key with a null value in Kafka, signaling downstream systems to remove the corresponding row.
2. B.tombstone record — A Kafka message with a non-null key and null value that signals deletion of that key, used by compacted topics and CDC delete events.
3. C.tombstone record — A framework of reusable source and sink connectors that move data between Kafka and external systems without custom producer or consumer code.
4. D.tombstone record — Automatically detecting and propagating source schema changes (new or altered columns) into the destination without breaking the pipeline.

Answer + AI explanation with Pro

1. A.A framework of reusable source and sink connectors for streaming data in and out of Kafka without writing custom producer or consumer code.
2. B.A sink that can process the same record more than once without changing the result, typically by upserting on a key, enabling safe retries.
3. C.A guarantee that each source change is reflected exactly once downstream despite retries, typically achieved with idempotent writes and offset tracking.
4. D.A change event representing a delete, often a key with a null value in Kafka, signaling downstream systems to remove the corresponding row.

Answer + AI explanation with Pro

1. A.trigger-based CDC
2. B.incremental snapshot
3. C.exactly-once delivery
4. D.snapshot plus streaming CDC

Answer + AI explanation with Pro

1. A.exactly-once delivery — A guarantee that each source change is reflected exactly once downstream despite retries, typically achieved with idempotent writes and offset tracking.
2. B.exactly-once delivery — An open-source CDC platform that reads database transaction logs and streams row-level change events, commonly through Kafka Connect.
3. C.exactly-once delivery — A Debezium snapshot setting that reads the full current state of tables once before streaming ongoing log changes, giving a complete starting baseline.
4. D.exactly-once delivery — A reusable integration component that extracts from a specific source or loads to a destination, the building block of managed ELT platforms.

Answer + AI explanation with Pro

1. A.Snowflake's rowset streaming ingestion that writes rows directly into tables with low latency, avoiding the file-staging step of classic Snowpipe.
2. B.A managed ELT connector service that automates extraction and loading from SaaS and database sources into a warehouse with schema handling.
3. C.A monotonically increasing position in a database transaction log (LSN) used by log-based CDC to resume exactly where it left off.
4. D.A reusable integration component that extracts from a specific source or loads to a destination, the building block of managed ELT platforms.

Answer + AI explanation with Pro

1. A.exactly-once delivery
2. B.Debezium
3. C.Snowpipe Streaming
4. D.connector

Answer + AI explanation with Pro

1. A.connector — A reusable integration component that extracts from a specific source or loads to a destination, the building block of managed ELT platforms.
2. B.connector — A CDC approach that polls rows whose updated_at is newer than the last run; simple but can miss hard deletes and out-of-order updates.
3. C.connector — A Debezium technique that chunks a table snapshot and interleaves it with live change streaming, avoiding long locks and allowing resumption.
4. D.connector — A central service that stores and versions message schemas (Avro, Protobuf, JSON) and enforces compatibility rules between producers and consumers.

Answer + AI explanation with Pro