Ibis Project Blog

Python productivity framework for the Apache Hadoop ecosystem. Development updates, use cases, and internals.

Ibis 0.4.0 release

The next Ibis release is out, check out the detailed release notes.

Notable features in this release (some of which will be the subject of upcoming blog posts):

  • Using Impala user-defined C++ functions with Ibis (docs here).
  • Better composability and chaining by passing functions as parameters
  • An ORM-like layer for interacting with database objects
  • Enhanced TopK expressions
  • Support for Kerberos-enabled secure HDFS clusters

Thanks to all who contributed patches:

$ git log v0.3.0..v0.4.0 --pretty=format:%aN | sort | uniq -c | sort -rn
     38 Wes McKinney
      9 Uri Laserson
      2 Meghana Vuyyuru
      2 Kristopher Overholt
      1 Marius van Niekerk

The next major release will offer

  • Python 3 support (this was blocked on upstream dependencies, sorry)
  • Impala C++ aggregate function (UDA) support
  • Async expression execution API (cancellation, progress, etc.)

I'm looking for people interested in helping build Ibis support for any of the following compute engines:

  • Hive
  • PostgreSQL flavors: Redshift, PostgreSQL, HP Vertica
  • MySQL
  • SQLite
  • Presto

If interested please contact me.