File Formats interview questions

1. A.Pushing filter conditions down to the storage layer so only matching row groups or files are read, using column statistics to skip data.
2. B.Storing values of each column together so analytic scans read only needed columns and compress better, the basis of Parquet and ORC.
3. C.An optimized row-columnar file format from the Hadoop ecosystem with strong compression, lightweight indexes, and built-in statistics.
4. D.Hierarchical structures (structs, arrays, maps) that columnar formats like Parquet encode with repetition and definition levels.

Answer + AI explanation with Pro

1. A.Avro
2. B.columnar storage
3. C.run-length encoding
4. D.compaction

Answer + AI explanation with Pro

1. A.columnar storage — Pushing filter conditions down to the storage layer so only matching row groups or files are read, using column statistics to skip data.
2. B.columnar storage — A row-oriented binary format with a self-describing schema, well suited to streaming and write-heavy workloads and schema evolution.
3. C.columnar storage — An optimized row-columnar file format from the Hadoop ecosystem with strong compression, lightweight indexes, and built-in statistics.
4. D.columnar storage — Storing values of each column together so analytic scans read only needed columns and compress better, the basis of Parquet and ORC.

Answer + AI explanation with Pro

1. A.Performance degradation from many tiny files that bloat metadata and scheduling overhead, mitigated by compaction into larger files.
2. B.Hierarchical structures (structs, arrays, maps) that columnar formats like Parquet encode with repetition and definition levels.
3. C.A compression technique replacing repeated values with small integer codes referencing a dictionary, very effective for low-cardinality columns.
4. D.An optimized row-columnar file format from the Hadoop ecosystem with strong compression, lightweight indexes, and built-in statistics.

Answer + AI explanation with Pro

1. A.ORC
2. B.predicate pushdown
3. C.page
4. D.Avro

Answer + AI explanation with Pro

1. A.ORC — A horizontal chunk of a Parquet file holding column chunks plus min/max statistics, the unit at which readers prune and parallelize.
2. B.ORC — A compression technique replacing repeated values with small integer codes referencing a dictionary, very effective for low-cardinality columns.
3. C.ORC — Performance degradation from many tiny files that bloat metadata and scheduling overhead, mitigated by compaction into larger files.
4. D.ORC — An optimized row-columnar file format from the Hadoop ecosystem with strong compression, lightweight indexes, and built-in statistics.

Answer + AI explanation with Pro

1. A.A compression method storing repeated consecutive values as a value plus a count, paired with dictionary encoding in Parquet.
2. B.Applying structure to data at query time rather than at write time, the flexibility that lets lakes store raw semi-structured files.
3. C.Pushing filter conditions down to the storage layer so only matching row groups or files are read, using column statistics to skip data.
4. D.The smallest unit of storage and compression within a Parquet column chunk, holding encoded values, definition, and repetition levels.

Answer + AI explanation with Pro

1. A.predicate pushdown
2. B.schema-on-read
3. C.small files problem
4. D.columnar storage

Answer + AI explanation with Pro

1. A.predicate pushdown — A maintenance operation that rewrites many small data files into fewer large ones to improve scan efficiency in lakes and open table formats.
2. B.predicate pushdown — Pushing filter conditions down to the storage layer so only matching row groups or files are read, using column statistics to skip data.
3. C.predicate pushdown — A horizontal chunk of a Parquet file holding column chunks plus min/max statistics, the unit at which readers prune and parallelize.
4. D.predicate pushdown — A row-oriented binary format with a self-describing schema, well suited to streaming and write-heavy workloads and schema evolution.

Answer + AI explanation with Pro

1. A.Hierarchical structures (structs, arrays, maps) that columnar formats like Parquet encode with repetition and definition levels.
2. B.A horizontal chunk of a Parquet file holding column chunks plus min/max statistics, the unit at which readers prune and parallelize.
3. C.A row-oriented binary format with a self-describing schema, well suited to streaming and write-heavy workloads and schema evolution.
4. D.Pushing filter conditions down to the storage layer so only matching row groups or files are read, using column statistics to skip data.

Answer + AI explanation with Pro

1. A.columnar storage
2. B.compaction
3. C.row group
4. D.small files problem

Answer + AI explanation with Pro

1. A.row group — The smallest unit of storage and compression within a Parquet column chunk, holding encoded values, definition, and repetition levels.
2. B.row group — A horizontal chunk of a Parquet file holding column chunks plus min/max statistics, the unit at which readers prune and parallelize.
3. C.row group — A row-oriented binary format with a self-describing schema, well suited to streaming and write-heavy workloads and schema evolution.
4. D.row group — An optimized row-columnar file format from the Hadoop ecosystem with strong compression, lightweight indexes, and built-in statistics.

Answer + AI explanation with Pro

1. A.A maintenance operation that rewrites many small data files into fewer large ones to improve scan efficiency in lakes and open table formats.
2. B.A compression technique replacing repeated values with small integer codes referencing a dictionary, very effective for low-cardinality columns.
3. C.Hierarchical structures (structs, arrays, maps) that columnar formats like Parquet encode with repetition and definition levels.
4. D.An optimized row-columnar file format from the Hadoop ecosystem with strong compression, lightweight indexes, and built-in statistics.

Answer + AI explanation with Pro

1. A.compaction
2. B.dictionary encoding
3. C.columnar storage
4. D.run-length encoding

Answer + AI explanation with Pro

1. A.dictionary encoding — The smallest unit of storage and compression within a Parquet column chunk, holding encoded values, definition, and repetition levels.
2. B.dictionary encoding — A compression technique replacing repeated values with small integer codes referencing a dictionary, very effective for low-cardinality columns.
3. C.dictionary encoding — A compression method storing repeated consecutive values as a value plus a count, paired with dictionary encoding in Parquet.
4. D.dictionary encoding — A maintenance operation that rewrites many small data files into fewer large ones to improve scan efficiency in lakes and open table formats.

Answer + AI explanation with Pro

1. A.The smallest unit of storage and compression within a Parquet column chunk, holding encoded values, definition, and repetition levels.
2. B.A compression method storing repeated consecutive values as a value plus a count, paired with dictionary encoding in Parquet.
3. C.A maintenance operation that rewrites many small data files into fewer large ones to improve scan efficiency in lakes and open table formats.
4. D.Performance degradation from many tiny files that bloat metadata and scheduling overhead, mitigated by compaction into larger files.

Answer + AI explanation with Pro

1. A.nested data
2. B.schema-on-read
3. C.dictionary encoding
4. D.small files problem

Answer + AI explanation with Pro

1. A.small files problem — Pushing filter conditions down to the storage layer so only matching row groups or files are read, using column statistics to skip data.
2. B.small files problem — Storing values of each column together so analytic scans read only needed columns and compress better, the basis of Parquet and ORC.
3. C.small files problem — Performance degradation from many tiny files that bloat metadata and scheduling overhead, mitigated by compaction into larger files.
4. D.small files problem — Hierarchical structures (structs, arrays, maps) that columnar formats like Parquet encode with repetition and definition levels.

Answer + AI explanation with Pro

1. A.Hierarchical structures (structs, arrays, maps) that columnar formats like Parquet encode with repetition and definition levels.
2. B.Performance degradation from many tiny files that bloat metadata and scheduling overhead, mitigated by compaction into larger files.
3. C.Pushing filter conditions down to the storage layer so only matching row groups or files are read, using column statistics to skip data.
4. D.A maintenance operation that rewrites many small data files into fewer large ones to improve scan efficiency in lakes and open table formats.

Answer + AI explanation with Pro

1. A.page
2. B.ORC
3. C.compaction
4. D.dictionary encoding

Answer + AI explanation with Pro

1. A.compaction — A compression technique replacing repeated values with small integer codes referencing a dictionary, very effective for low-cardinality columns.
2. B.compaction — Pushing filter conditions down to the storage layer so only matching row groups or files are read, using column statistics to skip data.
3. C.compaction — A maintenance operation that rewrites many small data files into fewer large ones to improve scan efficiency in lakes and open table formats.
4. D.compaction — A row-oriented binary format with a self-describing schema, well suited to streaming and write-heavy workloads and schema evolution.

Answer + AI explanation with Pro

1. A.Applying structure to data at query time rather than at write time, the flexibility that lets lakes store raw semi-structured files.
2. B.A compression method storing repeated consecutive values as a value plus a count, paired with dictionary encoding in Parquet.
3. C.A maintenance operation that rewrites many small data files into fewer large ones to improve scan efficiency in lakes and open table formats.
4. D.A row-oriented binary format with a self-describing schema, well suited to streaming and write-heavy workloads and schema evolution.

Answer + AI explanation with Pro

1. A.run-length encoding
2. B.Avro
3. C.schema-on-read
4. D.ORC

Answer + AI explanation with Pro

1. A.schema-on-read — A horizontal chunk of a Parquet file holding column chunks plus min/max statistics, the unit at which readers prune and parallelize.
2. B.schema-on-read — Storing values of each column together so analytic scans read only needed columns and compress better, the basis of Parquet and ORC.
3. C.schema-on-read — The smallest unit of storage and compression within a Parquet column chunk, holding encoded values, definition, and repetition levels.
4. D.schema-on-read — Applying structure to data at query time rather than at write time, the flexibility that lets lakes store raw semi-structured files.

Answer + AI explanation with Pro

1. A.An optimized row-columnar file format from the Hadoop ecosystem with strong compression, lightweight indexes, and built-in statistics.
2. B.Applying structure to data at query time rather than at write time, the flexibility that lets lakes store raw semi-structured files.
3. C.A row-oriented binary format with a self-describing schema, well suited to streaming and write-heavy workloads and schema evolution.
4. D.Pushing filter conditions down to the storage layer so only matching row groups or files are read, using column statistics to skip data.

Answer + AI explanation with Pro

1. A.Avro
2. B.dictionary encoding
3. C.page
4. D.ORC

Answer + AI explanation with Pro

1. A.Avro — A maintenance operation that rewrites many small data files into fewer large ones to improve scan efficiency in lakes and open table formats.
2. B.Avro — A horizontal chunk of a Parquet file holding column chunks plus min/max statistics, the unit at which readers prune and parallelize.
3. C.Avro — Applying structure to data at query time rather than at write time, the flexibility that lets lakes store raw semi-structured files.
4. D.Avro — A row-oriented binary format with a self-describing schema, well suited to streaming and write-heavy workloads and schema evolution.

Answer + AI explanation with Pro

1. A.Applying structure to data at query time rather than at write time, the flexibility that lets lakes store raw semi-structured files.
2. B.A compression method storing repeated consecutive values as a value plus a count, paired with dictionary encoding in Parquet.
3. C.The smallest unit of storage and compression within a Parquet column chunk, holding encoded values, definition, and repetition levels.
4. D.A compression technique replacing repeated values with small integer codes referencing a dictionary, very effective for low-cardinality columns.

Answer + AI explanation with Pro

1. A.Avro
2. B.predicate pushdown
3. C.page
4. D.schema-on-read

Answer + AI explanation with Pro

1. A.page — Hierarchical structures (structs, arrays, maps) that columnar formats like Parquet encode with repetition and definition levels.
2. B.page — A compression method storing repeated consecutive values as a value plus a count, paired with dictionary encoding in Parquet.
3. C.page — The smallest unit of storage and compression within a Parquet column chunk, holding encoded values, definition, and repetition levels.
4. D.page — Performance degradation from many tiny files that bloat metadata and scheduling overhead, mitigated by compaction into larger files.

Answer + AI explanation with Pro