Streaming interview questions

1. A.Streaming computation that retains accumulated state (counts, aggregations, joins) across micro-batches, checkpointed for fault tolerance.
2. B.A Kafka topic subdivision that provides ordering and parallelism; records with the same key go to the same partition.
3. C.Two delivery guarantees: at-least-once may duplicate on retry but never loses data; at-most-once never duplicates but may drop data.
4. D.A heuristic timestamp in a stream processor marking how far event time has progressed, used to decide when a window can close despite late data.

Answer + AI explanation with Pro

1. A.tumbling window
2. B.allowed lateness
3. C.watermark
4. D.consumer offset

Answer + AI explanation with Pro

1. A.watermark — The position a consumer group has read to in a partition, committed so processing can resume after a restart without reprocessing.
2. B.watermark — A heuristic timestamp in a stream processor marking how far event time has progressed, used to decide when a window can close despite late data.
3. C.watermark — A configured grace period after a window's watermark passes during which late events are still accepted and update the window result.
4. D.watermark — Event time is when an event actually occurred; processing time is when the system handles it; correct streaming uses event time with watermarks for late data.

Answer + AI explanation with Pro

1. A.A series of fixed-size, non-overlapping time windows over a stream, so each event belongs to exactly one window.
2. B.A Spark Structured Streaming sink that hands each micro-batch to a function as a DataFrame, enabling upserts, multi-sink writes, and arbitrary logic.
3. C.A named, partitioned, append-only log in Kafka to which producers publish and from which consumer groups read independently.
4. D.A streaming guarantee that each record affects state and output once despite failures, achieved with checkpointing, idempotent sinks, or transactional writes.

Answer + AI explanation with Pro

1. A.consumer offset
2. B.Kinesis
3. C.event time vs processing time
4. D.tumbling window

Answer + AI explanation with Pro

1. A.tumbling window — A Spark Structured Streaming sink that hands each micro-batch to a function as a DataFrame, enabling upserts, multi-sink writes, and arbitrary logic.
2. B.tumbling window — A Spark Structured Streaming mechanism that persists offsets and state to durable storage so a failed query resumes exactly where it left off.
3. C.tumbling window — The position a consumer group has read to in a partition, committed so processing can resume after a restart without reprocessing.
4. D.tumbling window — A series of fixed-size, non-overlapping time windows over a stream, so each event belongs to exactly one window.

Answer + AI explanation with Pro

1. A.A windowing scheme of fixed-size intervals that advance by a smaller step, so windows overlap and an event can fall into several.
2. B.Overlapping fixed-size windows that advance by a step smaller than the window length, so each event can fall into multiple windows.
3. C.A Spark Structured Streaming sink that hands each micro-batch to a function as a DataFrame, enabling upserts, multi-sink writes, and arbitrary logic.
4. D.A Spark Structured Streaming setting controlling batch cadence, with modes like fixed processing-time interval, availableNow, and continuous.

Answer + AI explanation with Pro

1. A.session window
2. B.watermark
3. C.stateful processing
4. D.sliding window

Answer + AI explanation with Pro

1. A.sliding window — Google Cloud's managed, globally scalable message bus for decoupled, at-least-once asynchronous event delivery between services.
2. B.sliding window — A named, partitioned, append-only log in Kafka to which producers publish and from which consumer groups read independently.
3. C.sliding window — Overlapping fixed-size windows that advance by a step smaller than the window length, so each event can fall into multiple windows.
4. D.sliding window — A Spark Structured Streaming trigger that processes all currently available data in multiple batches and then stops, ideal for incremental scheduled jobs.

Answer + AI explanation with Pro

1. A.Google Cloud's managed, globally scalable message bus for decoupled, at-least-once asynchronous event delivery between services.
2. B.A streaming engine's moving estimate of event-time progress that declares when all events up to a timestamp are assumed to have arrived, triggering window output.
3. C.A Spark Structured Streaming mechanism that persists offsets and state to durable storage so a failed query resumes exactly where it left off.
4. D.A dynamic window that groups events separated by less than a configured inactivity gap, ideal for user-activity sessionization.

Answer + AI explanation with Pro

1. A.session window
2. B.foreachBatch
3. C.stateful processing
4. D.topic

Answer + AI explanation with Pro

1. A.session window — Two delivery guarantees: at-least-once may duplicate on retry but never loses data; at-most-once never duplicates but may drop data.
2. B.session window — Streaming computation that retains accumulated state (counts, aggregations, joins) across micro-batches, checkpointed for fault tolerance.
3. C.session window — A dynamic window that groups events separated by less than a configured inactivity gap, ideal for user-activity sessionization.
4. D.session window — A windowing scheme that groups events separated by gaps smaller than a timeout, producing variable-length windows that close after inactivity.

Answer + AI explanation with Pro

1. A.Streaming computation that retains accumulated state (counts, aggregations, joins) across micro-batches, checkpointed for fault tolerance.
2. B.Event time is when an event actually occurred; processing time is when the system handles it; correct streaming uses event time with watermarks for late data.
3. C.A Spark Structured Streaming sink that hands each micro-batch to a function as a DataFrame, enabling upserts, multi-sink writes, and arbitrary logic.
4. D.The position a consumer group has read to in a partition, committed so processing can resume after a restart without reprocessing.

Answer + AI explanation with Pro

1. A.tumbling window
2. B.event time vs processing time
3. C.exactly-once semantics
4. D.foreachBatch

Answer + AI explanation with Pro

1. A.event time vs processing time — A flow-control condition where a slow downstream operator forces upstream stages to slow down, preventing unbounded buffering and memory growth.
2. B.event time vs processing time — A dynamic window that groups events separated by less than a configured inactivity gap, ideal for user-activity sessionization.
3. C.event time vs processing time — A configured grace period after a window's watermark passes during which late events are still accepted and update the window result.
4. D.event time vs processing time — Event time is when an event actually occurred; processing time is when the system handles it; correct streaming uses event time with watermarks for late data.

Answer + AI explanation with Pro

1. A.Two delivery guarantees: at-least-once may duplicate on retry but never loses data; at-most-once never duplicates but may drop data.
2. B.A Spark Structured Streaming trigger that processes all currently available data in multiple batches and then stops, ideal for incremental scheduled jobs.
3. C.Event time is when an event actually occurred; processing time is when the system handles it; correct streaming uses event time with watermarks for late data.
4. D.A streaming guarantee that each record affects state and output once despite failures, achieved with checkpointing, idempotent sinks, or transactional writes.

Answer + AI explanation with Pro

1. A.allowed lateness
2. B.exactly-once semantics
3. C.topic
4. D.consumer offset

Answer + AI explanation with Pro

1. A.exactly-once semantics — Two delivery guarantees: at-least-once may duplicate on retry but never loses data; at-most-once never duplicates but may drop data.
2. B.exactly-once semantics — A streaming guarantee that each record affects state and output once despite failures, achieved with checkpointing, idempotent sinks, or transactional writes.
3. C.exactly-once semantics — A windowing scheme of fixed-size, non-overlapping time intervals so each event belongs to exactly one window.
4. D.exactly-once semantics — A dynamic window that groups events separated by less than a configured inactivity gap, ideal for user-activity sessionization.

Answer + AI explanation with Pro

1. A.AWS managed service for real-time streaming data ingestion and processing, with shards as the unit of throughput and ordering.
2. B.A windowing scheme that groups events separated by gaps smaller than a timeout, producing variable-length windows that close after inactivity.
3. C.Stream operations (aggregations, joins, dedup) that retain state across events, requiring checkpointing for fault tolerance and recovery.
4. D.A windowing scheme of fixed-size intervals that advance by a smaller step, so windows overlap and an event can fall into several.

Answer + AI explanation with Pro

1. A.watermark
2. B.Kinesis
3. C.session window
4. D.stateful processing

Answer + AI explanation with Pro

1. A.stateful processing — AWS managed service for real-time streaming data ingestion and processing, with shards as the unit of throughput and ordering.
2. B.stateful processing — Stream operations (aggregations, joins, dedup) that retain state across events, requiring checkpointing for fault tolerance and recovery.
3. C.stateful processing — A Spark Structured Streaming trigger that processes all currently available data in multiple batches and then stops, ideal for incremental scheduled jobs.
4. D.stateful processing — A streaming guarantee that each record affects state and output once despite failures, achieved with checkpointing, idempotent sinks, or transactional writes.

Answer + AI explanation with Pro

1. A.A flow-control condition where a slow downstream operator forces upstream stages to slow down, preventing unbounded buffering and memory growth.
2. B.AWS managed service for real-time streaming data ingestion and processing, with shards as the unit of throughput and ordering.
3. C.Google Cloud's managed, globally scalable message bus for decoupled, at-least-once asynchronous event delivery between services.
4. D.A Spark Structured Streaming sink that hands each micro-batch to a function as a DataFrame, enabling upserts, multi-sink writes, and arbitrary logic.

Answer + AI explanation with Pro

1. A.Pub/Sub
2. B.sliding window
3. C.backpressure
4. D.Kinesis

Answer + AI explanation with Pro

1. A.backpressure — AWS managed service for real-time streaming data ingestion and processing, with shards as the unit of throughput and ordering.
2. B.backpressure — A Spark Structured Streaming mechanism that persists offsets and state to durable storage so a failed query resumes exactly where it left off.
3. C.backpressure — A heuristic timestamp in a stream processor marking how far event time has progressed, used to decide when a window can close despite late data.
4. D.backpressure — A flow-control condition where a slow downstream operator forces upstream stages to slow down, preventing unbounded buffering and memory growth.

Answer + AI explanation with Pro

1. A.AWS managed service for real-time streaming data ingestion and processing, with shards as the unit of throughput and ordering.
2. B.A windowing scheme that groups events separated by gaps smaller than a timeout, producing variable-length windows that close after inactivity.
3. C.Google Cloud's managed, globally scalable message bus for decoupled, at-least-once asynchronous event delivery between services.
4. D.A windowing scheme of fixed-size, non-overlapping time intervals so each event belongs to exactly one window.

Answer + AI explanation with Pro

1. A.stateful processing
2. B.foreachBatch
3. C.trigger
4. D.Kinesis

Answer + AI explanation with Pro

1. A.Kinesis — The position a consumer group has read to in a partition, committed so processing can resume after a restart without reprocessing.
2. B.Kinesis — A dynamic window that groups events separated by less than a configured inactivity gap, ideal for user-activity sessionization.
3. C.Kinesis — AWS managed service for real-time streaming data ingestion and processing, with shards as the unit of throughput and ordering.
4. D.Kinesis — Google Cloud's managed, globally scalable message bus for decoupled, at-least-once asynchronous event delivery between services.

Answer + AI explanation with Pro

1. A.A Spark Structured Streaming trigger that processes all currently available data in multiple batches and then stops, ideal for incremental scheduled jobs.
2. B.Google Cloud's managed, globally scalable message bus for decoupled, at-least-once asynchronous event delivery between services.
3. C.A windowing scheme of fixed-size intervals that advance by a smaller step, so windows overlap and an event can fall into several.
4. D.A dynamic window that groups events separated by less than a configured inactivity gap, ideal for user-activity sessionization.

Answer + AI explanation with Pro

1. A.Kinesis
2. B.Pub/Sub
3. C.topic
4. D.allowed lateness

Answer + AI explanation with Pro

1. A.Pub/Sub — Google Cloud's managed, globally scalable message bus for decoupled, at-least-once asynchronous event delivery between services.
2. B.Pub/Sub — A Spark Structured Streaming trigger that processes all currently available data in multiple batches and then stops, ideal for incremental scheduled jobs.
3. C.Pub/Sub — Streaming computation that retains accumulated state (counts, aggregations, joins) across micro-batches, checkpointed for fault tolerance.
4. D.Pub/Sub — A dynamic window that groups events separated by less than a configured inactivity gap, ideal for user-activity sessionization.

Answer + AI explanation with Pro