Monday, March 25

Author: Pavan Deolasee

Postgres-XL 9.5r1 has finally arrived!

2ndQuadrant, Pavan's PlanetPostgreSQL
After months of efforts, I'm pleased that Postgres-XL 9.5r1 is seeing the daylight. It has been tremendous collective efforts by many, both inside and outside 2ndQuadrant. Often it's not visible via commit history or mailing list communications, but I must admit that many folks have contributed in making this grand release. Contributors who wrote code and sent ideas, those who reviewed the code, QA team for testing and enhancing regression coverage, those who did benchmarks to show how we are doing, those who wrote automation to deploy the product and of course the user community for testing the product and providing invaluable feedback. So thank you folks! Over the year, I've periodically wrote about various enhancements we made to the product. But let me summarise some of them here (more…)

Postgres-XL 9.5R1Beta2 Released!

Pavan's PlanetPostgreSQL
The Postgres-XL 9.5R1Beta2 release went out yesterday. It's another step forward to have a stable 9.5 release sometime very soon. A few key enhancements from the last beta release are captured in this blog. For the full list, I would recommend to read the release notes. Support for binary data transfer for JDBC and libpq If you'd trouble receiving data in binary format via JDBC or libpq, this release should fix those issues for you. The coordinator now have intelligence to figure out if the client is requesting data in binary format and handle those cases correctly. Pushdown of Append and MergeAppend plans One of the problems reported with the beta1 release was that for inherited tables, coordinator fails to pushdown plans to the remote nodes even when its possible. A simple example (more…)

Sharding: Bringing back Postgres-XL technology into core PostgreSQL

Pavan's PlanetPostgreSQL
Sharding or horizontal scalability is a popular topic, discussed widely on PostgreSQL mailing lists these days. When we started the Postgres-XC project back in 2010, not everyone was convinced that we need a multi-node PostgreSQL cluster that scales with increasing demand. Even more likely, we, the PostgreSQL community, were skeptical about whether we have enough resources to design and implement a complex sharding architecture. Also, there was so much work that we could have done for vertical scalability and the obvious choice for many was to focus on the vertical scalability aspects of performance. With great work that Robert Haas, Andres Freund and others have done in the last few releases, law of marginal utility is now catching up with the vertical scalability enhancements.  Now, (more…)

Postgres-XL 9.5R1Beta1 is here!

2ndQuadrant, Pavan's PlanetPostgreSQL
I'm extremely pleased to see that after months of efforts and contributions by different people around the world, Postgres-XL 9.5 R1 Beta1 has finally arrived. This release is significantly better, in all respects such as performance, stability and high availability, as compared to the past release. Enormous amount of work has gone into PostgreSQL in the last few years and majority of that is now available in this release of Postgres-XL 9.5R1 which is based on the latest PostgreSQL release. If you're looking for a PostgreSQL based technology that can support terabytes of data, massively parallel processing, supports OLAP and OLTP workloads, offers a consistent view of the entire cluster and is compliant with PostgreSQL as far as client APIs are concerned, have a look at the latest (more…)

Looking forward to PGDay India 2016

Pavan's PlanetPostgreSQL
With just couple of weeks to go for PGDay India, I'm quite excited about the upcoming PostgreSQL conference at Bangalore on 26th February, 2016. This is going to be the biggest ever conglomeration of users, developers and companies interested in PostgreSQL in India. This is also by far the largest PostgreSQL conference in terms of number of speakers and attendees. Till last year, we'd to almost find speakers for the conference, but this year we'd a good problem of rejecting almost 1 in every 2 submitted proposals. It's almost certain that in coming years, we either have to do multiple-track or multiple-day event. Again, a nice to have problem. For the first time, we are looking to easily cross 100 mark in terms of attendees. We have secured 3 Platinum Sponsors, 2 Gold Sponsors, 1 (more…)

Postgres-XL: Dealing with read-only transactions

2ndQuadrant, Pavan's PlanetPostgreSQL
One of the problems with Postgres-XL 9.2 is that it assigns a global transaction identifier (GXID) for every transaction that is started in the cluster. Since its hard for the system to know if a transaction is read-only or not (remember, even SELECT queries can do write activities), Postgres-XL would always assign a GXID and send that to the datanodes. This is quite bad for system performance because every read-only transaction now must go to the GTM, start a new transaction, grab an GXID and also finish the transaction on the GTM when its committed or aborted. For short transactions, like read-only pgbench workload, this adds a lot of overhead and latency. The overhead is even more noticeable when the actual work done by the transaction is very fast, say because the data is fully (more…)

Performance of Sequences and Serials in Postgres-XL

2ndQuadrant, Pavan's PlanetPostgreSQL, PostgreSQL
In Postgres-XL, sequences are maintained at the Global Transaction Manager (GTM) to ensure that they are assigned non-conflicting values when they are incremented from multiple nodes. This adds significant overhead for a query doing thousands of INSERTs in a table with a serial column, incrementing sequence one at a time and making a network roundtrip to the GTM, for every INSERT. Shaun Thomas in a recent blog complained about INSERTs running a magnitude slower on Postgres-XL as compared to vanilla PostgreSQL. There is already a way to improve performance for sequences, but it’s clearly not well advertised. I thought this is a good opportunity to explain the facility. Postgres-XL provides a user-settable GUC called sequence_range. Every backend requests a block of sequence values as (more…)
Benchmarking Postgres-XL

Benchmarking Postgres-XL

2ndQuadrant, Featured, Pavan's PlanetPostgreSQL, PostgreSQL
As you may have noted from my previous blog, the last few months were busy in getting Postgres-XL up-to-date with the latest 9.5 release of PostgreSQL. Once we had a reasonably stable version of Postgres-XL 9.5, we shifted our attention to measure performance of this brand new version of Postgres-XL. Our choice of the benchmark is largely influenced by the ongoing work on the AXLE project, funded by the European Union under grant agreement 318633. Since we are using TPC BENCHMARK™ H to measure performance of all other work done under this project, we decided to use the same benchmark for evaluating Postgres-XL. It also suits Postgres-XL because TPC-H tries to measure OLAP workloads, something Postgres-XL should do well. 1. Postgres-XL Cluster Setup Once the benchmark was decided, (more…)

Working towards Postgres-XL 9.5

2ndQuadrant, Pavan's PlanetPostgreSQL, PostgreSQL
It’s been busy few months as we work towards merging Postgres-XL with the latest and greatest release of PostgreSQL. Postgres-XL is an open source fork of PostgreSQL that provides a scalable platform for OLTP and Business Intelligence. The current release of Postgres-XL is based on PostgreSQL 9.2, so it lacks all the improvements made to PostgreSQL over the last three years. 2ndQuadrant and other companies are working on bringing distributed scalability into PostgreSQL core as well as building tools and extensions outside the core. As part of that, Postgres-XL has a number of features that we’d like to bring back into core PostgreSQL, so 2ndQuadrant has picked up the task of updating the Postgres-XL code base to the latest PostgreSQL release as the first step. After more than 3 months (more…)