Apache DataFusion

Using External Indexes, Metadata Stores, Catalogs and Caches to Accelerate Queries on Apache Parquet by Andrew Lamb (InfluxData)

Embedding User-Defined Indexes in Apache Parquet Files by Qi Zhu (Cloudera), Jigao Luo (Systems Group at TU Darmstadt), and Andrew Lamb (InfluxData)

Apache DataFusion 49.0.0 Released

Highlights I found impressive

• Dynamic Filters and TopK pushdown
• Async User-Defined Functions (ask_llm?!)
• Better Cancellation for Certain Long-Running Queries