Ibis Project Blog

Python productivity framework for the Apache Hadoop ecosystem. Development updates, use cases, and internals.

Ibis 0.6: Expanded Impala support, better pandas integration

Ibis 0.6.0 has been released! This release brings substantial new functionality for Impala users; see the Using Ibis with Impala guide for a deep dive on these features. These enable Ibis to solve many data management problems beyond the analytical SQL capabilities exposed through Ibis expressions.

Some highlights of the new functionality include:

  • Create tables and insert into existing tables directly from pandas DataFrame objects
  • Inspect a variety of table and partition metadata in a user-friendly manner
  • Comprehensive partitioned table support
  • Modify table and partition metadata
  • Set session options such as Parquet compression codec

Several bugs were fixed as well.

The upcoming roadmap will feature Impala's nested types support and expanded SQL system coverage.

Thanks to all who contributed patches:

$ git log v0.5.0..v0.6.0 --pretty=format:%aN | sort | uniq -c | sort -rn
  46 Wes McKinney
   3 Uri Laserson
   1 Phillip Cloud
   1 mariusvniekerk
   1 Kristopher Overholt