Breakout Session Presented by Sadayuki Furuhashi of Treasure Data at PGConf Silicon Valley 2015

• Report rights infringement
• published: 25 Nov 2015
• views: 286

Prestogres: Hacking PostgreSQL Internals to Solve Data Access Problems In data analytics, visualization is highly important because that's how 95% of the data is consumed. For data sources such as Excel and PostgreSQL, both open-source and commercial visualization software have mature data connectors. However, scaling traditional databases for PBs of big data is a serious challenge. On the other hand, newer distributed query engines like Presto is scalable because they're originally designed for distributed systems. But they don't have matured connectivity with these visualization software yet. A well-trodden path is to build ODBC/JDBC driver, but writing ODBC/JDBC drivers require tremendous efforts and often built as a proprietary add-on to make up for the cost of development and maintenance. Hence, we needed a novel, fully open-source solution. Prestogres's idea was to use the PostgreSQL protocol so that we can utilize any PostgreSQL tools. Prestogres hacks PostgreSQL and pgpool-II so that queries are run on Presto and not PostgreSQL. Query rewriting is done inside pgpool-II while the Presto API client and faked system catalogs are built in PL/Python functions. This talk is intended for those who always wished to tinker with PostgreSQL at low-level. I first introduce how Presto runs query plans across distributed servers, and describe about Prestogres as an example approach to highlight some core principles of PostgreSQL™s internals and show you how to get started on hacking PostgreSQL for fun and profits.