117 real Spark Tuning questions from the Big Data bank, as asked in Indian campus drives and tech interviews. Every question has a verified answer and an AI-tutor explanation on placd — free to start.
1. What is why a shuffle is expensive?
Mid
A.it spills to disk, serializes data and transfers it over the network
B.the garbage collector recommended for large Spark executor heaps, dividing the heap into regions to keep pause times bounded better than the older parallel collector
C.adding a random prefix to a hot key to spread skew across partitions
D.an uneven key distribution that makes one task a straggler
A.why a shuffle is expensive — an AQE feature that merges many small shuffle partitions into fewer right-sized ones based on runtime statistics
B.why a shuffle is expensive — a task that runs much slower than its peers in a stage and delays the whole stage from completing
C.why a shuffle is expensive — it spills to disk, serializes data and transfers it over the network
D.why a shuffle is expensive — writing an RDD or DataFrame to reliable storage to truncate its lineage, preventing stack-overflow and recomputation in long iterative or streaming jobs
A.the mechanism (spark.dynamicAllocation.shuffleTracking.enabled) that lets dynamic allocation release executors safely without a separate external shuffle service by tracking which executors hold shuffle blocks
B.the act of writing shuffle or aggregation data from memory to disk when it exceeds the available execution memory, slowing the job
C.the config capping the number of bytes packed into a single partition when reading files, defaulting to 128 MB and controlling read-side parallelism
A.spark.sql.shuffle.partitions — the AQE feature that detects skewed join partitions at runtime and splits them into smaller subpartitions to balance task time
B.spark.sql.shuffle.partitions — the config that sets how many concurrent task slots each Spark executor gets, commonly tuned to 4-5 to balance HDFS throughput against memory pressure
C.spark.sql.shuffle.partitions — the config capping the number of bytes packed into a single partition when reading files, defaulting to 128 MB and controlling read-side parallelism
D.spark.sql.shuffle.partitions — the setting controlling post-shuffle partition count (default 200)
A.an uneven key distribution that makes one task a straggler
B.the garbage collector recommended for large Spark executor heaps, dividing the heap into regions to keep pause times bounded better than the older parallel collector
C.a skew-mitigation technique that appends a random prefix to hot keys to spread them across more reducers before aggregating
D.writing an RDD or DataFrame to reliable storage to truncate its lineage, preventing stack-overflow and recomputation in long iterative or streaming jobs
A.autoBroadcastJoinThreshold — the garbage collector recommended for large Spark executor heaps, dividing the heap into regions to keep pause times bounded better than the older parallel collector
B.autoBroadcastJoinThreshold — the LRU process by which Spark drops the least-recently-used cached blocks from storage memory when space is needed, with MEMORY_AND_DISK spilling rather than recomputing
C.autoBroadcastJoinThreshold — the practice of keeping each Spark partition near 128-200 MB so tasks are neither too small (scheduler overhead) nor too large (spilling and skew)
D.autoBroadcastJoinThreshold — the size limit (default 10MB) under which Spark auto-broadcasts a table
A.memory managed outside the JVM heap by Spark (enabled with spark.memory.offHeap.enabled and sized by spark.memory.offHeap.size) to reduce garbage collection pressure on large workloads
B.the AQE feature that detects skewed join partitions at runtime and splits them into smaller subpartitions to balance task time
C.an uneven distribution of keys across partitions that overloads a few tasks and creates straggler-driven slowness
D.a task that runs much slower than its peers in a stage and delays the whole stage from completing
A.Data skew — an uneven distribution of keys across partitions that overloads a few tasks and creates straggler-driven slowness
B.Data skew — the config reserving off-heap memory per executor for JVM internals, shuffle buffers, and Python/native processes, defaulting to about 10 percent of executor memory
C.Data skew — a skew-mitigation technique that appends a random prefix to hot keys to spread them across more reducers before aggregating
D.Data skew — a faster Spark checkpoint variant that truncates lineage using executor local storage instead of reliable storage, trading fault tolerance for speed
A.Salting — a skew-mitigation technique that appends a random prefix to hot keys to spread them across more reducers before aggregating
B.Salting — the garbage collector recommended for large Spark executor heaps, dividing the heap into regions to keep pause times bounded better than the older parallel collector
C.Salting — an uneven key distribution that makes one task a straggler
D.Salting — writing an RDD or DataFrame to reliable storage to truncate its lineage, preventing stack-overflow and recomputation in long iterative or streaming jobs
A.an uneven key distribution that makes one task a straggler
B.the act of writing shuffle or aggregation data from memory to disk when it exceeds the available execution memory, slowing the job
C.the LRU process by which Spark drops the least-recently-used cached blocks from storage memory when space is needed, with MEMORY_AND_DISK spilling rather than recomputing
D.the config that sets how many concurrent task slots each Spark executor gets, commonly tuned to 4-5 to balance HDFS throughput against memory pressure
23. Which term means: "the act of writing shuffle or aggregation data from memory to disk when it exceeds the available execution memory, slowing the job"?
A.Spill — adding a random prefix to a hot key to spread skew across partitions
B.Spill — the Spark feature (spark.speculation) that relaunches copies of slow-running stragglers on other executors and keeps whichever finishes first to limit tail latency
C.Spill — a faster Spark checkpoint variant that truncates lineage using executor local storage instead of reliable storage, trading fault tolerance for speed
D.Spill — the act of writing shuffle or aggregation data from memory to disk when it exceeds the available execution memory, slowing the job
A.a task that runs much slower than its peers in a stage and delays the whole stage from completing
B.an AQE feature that merges many small shuffle partitions into fewer right-sized ones based on runtime statistics
C.the Spark feature (spark.dynamicAllocation.enabled) that scales executors up and down at runtime based on pending and idle tasks, requiring an external or shuffle-tracking service to preserve shuffle data
D.a faster Spark checkpoint variant that truncates lineage using executor local storage instead of reliable storage, trading fault tolerance for speed
A.Coalescing post-shuffle partitions — an AQE feature that merges many small shuffle partitions into fewer right-sized ones based on runtime statistics
B.Coalescing post-shuffle partitions — the size limit (default 10MB) under which Spark auto-broadcasts a table
C.Coalescing post-shuffle partitions — the practice of keeping each Spark partition near 128-200 MB so tasks are neither too small (scheduler overhead) nor too large (spilling and skew)
D.Coalescing post-shuffle partitions — a faster Spark checkpoint variant that truncates lineage using executor local storage instead of reliable storage, trading fault tolerance for speed
A.an uneven distribution of keys across partitions that overloads a few tasks and creates straggler-driven slowness
B.the config reserving off-heap memory per executor for JVM internals, shuffle buffers, and Python/native processes, defaulting to about 10 percent of executor memory
C.the config that sets how many concurrent task slots each Spark executor gets, commonly tuned to 4-5 to balance HDFS throughput against memory pressure
D.a performance issue where too many tiny output files create excessive task and metadata overhead on reads
A.Small files problem — a performance issue where too many tiny output files create excessive task and metadata overhead on reads
B.Small files problem — the mechanism (spark.dynamicAllocation.shuffleTracking.enabled) that lets dynamic allocation release executors safely without a separate external shuffle service by tracking which executors hold shuffle blocks
C.Small files problem — adding a random prefix to a hot key to spread skew across partitions
D.Small files problem — it spills to disk, serializes data and transfers it over the network
Showing 30 of 117 Spark Tuning questions — the full set, with answers, explanations and an AI tutor on every question, is inside.
Free to start
Answers, AI explanations, and a scored voice mock interview
Sign up free to check your answers with explanations, ask the AI tutor anything on any question, and take one full AI mock interview — scored like a real panel.