Posts - Page 25

All the articles I've posted, page 25.

• 24 MIN READ•May 22, 2026

  Maintaining Apache Iceberg Tables: Compaction, Snapshot Expiration, and Orphan File Cleanup

  An in-depth guide to orchestrating maintenance operations on Apache Iceberg tables, covering bin-packing, sort-based, Z-Order compaction, snapshot.

  Apache IcebergCompactionData Engineering
• 24 MIN READ•May 22, 2026

  Apache Iceberg with Spark: Create, MERGE, Upsert, and Evolve Tables End to End

  A comprehensive developer guide to configuring Apache Spark with Apache Iceberg, executing transactional writes, and managing schema evolution.

  apache sparkapache icebergdata engineering
• 21 MIN READ•May 22, 2026

  Building a Multicloud Agentic Lakehouse Reference Architecture

  A reference architecture for building an open, multicloud Data Lakehouse optimized for AI Agents using Apache Polaris, Apache Iceberg, and Dremio.

  data lakehouseagentic lakehouseapache polaris
• 21 MIN READ•May 22, 2026

  Common Misconceptions About Data Lakehouse and Apache Iceberg

  Addressing common search queries and reader confusion about Data Lakehouse architectures, Apache Iceberg catalogs, partitions, and lock-in.

  data lakehouseapache icebergdata engineering
• 7 MIN READ•Apr 29, 2026

  Migrating to Apache Iceberg: Strategies for Every Source System

  Migrate to Iceberg from Hive, data warehouses, or raw files using in-place migration, full rewrite, or the zero-downtime view swap pattern.

  migrating to Apache IcebergHive to Iceberg migrationIceberg migration strategy
• 7 MIN READ•Apr 29, 2026

  Hands-On with Apache Iceberg Using Dremio Cloud

  A practical walkthrough of creating, querying, and optimizing Iceberg tables on Dremio Cloud, from account setup to AI-powered analytics.

  Dremio Cloud Apache IcebergDremio Cloud tutorialIceberg hands-on
• 7 MIN READ•Apr 29, 2026

  Approaches to Streaming Data into Apache Iceberg Tables

  Stream data into Iceberg with Spark Structured Streaming, Flink, or Kafka Connect. Here is how each works and the trade-offs between latency and maintenance.

  streaming to Apache IcebergSpark Structured Streaming IcebergFlink Iceberg sink
• 7 MIN READ•Apr 29, 2026

  Using Apache Iceberg with Python and MPP Query Engines

  Access Iceberg tables from Python with PyIceberg, DuckDB, and Polars, or through MPP engines like Dremio, Spark, and Trino. Here is how each approach works.

  Python Apache IcebergPyIcebergDuckDB Iceberg
• 7 MIN READ•Apr 29, 2026

  Apache Iceberg Metadata Tables: Querying the Internals

  Iceberg metadata tables let you query snapshots, files, manifests, and partitions using SQL. Here is every metadata table and how to use them.

  Iceberg metadata tablestable snapshotstable files
• 7 MIN READ•Apr 29, 2026

  Maintaining Apache Iceberg Tables: Compaction, Expiry, and Cleanup

  Keep Iceberg tables fast with compaction, snapshot expiry, orphan cleanup, and manifest rewriting. Here is when and how to run each operation.

  Iceberg table maintenancecompactionsnapshot expiry
• 9 MIN READ•Apr 29, 2026

  Concurrency, Isolation, and MVCC: How Engines Handle Contention

  Databases handle concurrent access using locks, MVCC, or optimistic concurrency control. Here is how each approach works and what tradeoffs each creates.

  database concurrency controlMVCC databaseisolation levels
• 7 MIN READ•Apr 29, 2026

  How Data Lake Table Storage Degrades Over Time

  Iceberg tables degrade through small files, orphan files, metadata bloat, sort order decay, and partition skew. Here is how to diagnose each problem.

  Iceberg storage degradationsmall file problemorphan files