Tuesday, October 24

When autovacuum does not vacuum

2ndQuadrant, PostgreSQL, Tomas' PlanetPostgreSQL
A few weeks ago I explained basics of autovacuum tuning. At the end of that post I promised to look into problems with vacuuming soon. Well, it took a bit longer than I planned, but here we go. To quickly recap, autovacuum is a background process cleaning up dead rows, e.g. old deleted row versions. You can also perform the cleanup manually by running VACUUM, but autovacuum does that automatically depending on the amount of dead rows in the table, at the right moment - not too often but frequently enough to keep the amount of "garbage" under control. (more…)

Using the PostgreSQL TAP framework in extensions

Craig's PlanetPostgreSQL
Support for using the TAP protocol to run extended regression tests was added to PostgreSQL back in 9.4 with the adoption of Perl's prove tool and Test::More to test initdb, pg_basebackup, etc. Since then the TAP-based tests have been greatly expanded, particularly with the advent of the src/test/recovery tests and the PostgresNode module in PostgreSQL 9.6. PostgreSQL now comes with a built-in test harness for easily starting up postgres instances, creating and restoring backups for replication, setting up streaming, and lots more. You can now use this to test your extensions. pg_regress and its limitations Extensions have long supported pg_regress based tests. Just drop the test scripts in sql/. Put the expected results in expected/. List the test names (sans directory and file (more…)

2ndQuadrant – Not the Gartner Magic Quadrant

It’s still appropriate though. Because that is what we are - thought leaders in open source PostgreSQL. But that's not what I am here to talk about. The name "2ndQuadrant" comes from "The Seven Habits of Highly Effective People" by Stephen Covey, specifically Habit 3 "Put First Things First", p.151. It refers to the classification of tasks in terms of 2 axes: Importance and Urgency. The second quadrant is the Important, Not Urgent quadrant. According to Covey, if you concentrate on doing work in that space, it leads to "vision, perspective, balance, discipline, control and few crises" - i.e. you focus on your long-term vision and your ability to execute short to medium term goals is also impeccable. These are all qualities highly valued by every individual at 2ndQuadrant; so (more…)

PgDay Paris 2017

2ndQuadrant, Simon's PlanetPostgreSQL
Great conference! Paris is a great venue for travellers across Europe and worldwide. PgDay Paris 2017 was held in English and attracted a wide audience from many other countries: UK, NL, CH, BE, US, SE - and that was just the people I spoke to. Je suis desolee ne parler ou ecrit pas en francais. Je suis un developpeur seulement. I spoke in English about the new features in PostgreSQL 10 regarding Replication & Recovery. All very well received by a large technical audience. Logical replication, physical replication improvements, quorum commit, replication lag measurement and a ton of fine detailed improvements. No slides, sorry. Come to the conferences! Meet people, hear their stories and share yours. I travelled to Paris through London on a day of public murders that made news (more…)

PGConf India 2017 – An Event to Remember

2ndQuadrant, Pavan's PlanetPostgreSQL
Yet another edition of PGConf India came to conclusion in early March. You may have noticed the change from PGDay to PGConf, which signals a much larger gathering of PostgreSQL enthusiasts, now and in future. What started as a small meet-up of like minded people 4 years back, has now grown into a 2-day conference with a dedicated training day and a multi-track event at the main conference. The keynote this year was delivered by Simon Riggs, a major developer and committer at PostgreSQL. He presented his thoughts on why Persistence is key to PostgreSQL's success. Persistence is why users trust PostgreSQL to manage their critical data and persistence is why PostgreSQL community is able to deliver a solid product, release after release. This year's conference was attended by more (more…)

Re-import repository keys for BDR and pglogical apt repositories

Craig's PlanetPostgreSQL
The BDR and pglogical apt repository GnuPG signing keys have been renewed. Users should re-import the existing keys. You can verify that it's still the same key as referenced in the documentation, just with a later expiry date. Simply run: wget --quiet -O - http://packages.2ndquadrant.com/bdr/apt/AA7A6805.asc | sudo apt-key add - sudo apt-key finger AA7A6805 | grep -A2 -B3 BDR Now check the fingerprint printed by the second command to verify it's the same as this output: pub 2048R/AA7A6805 2015-03-24 [expires: 2019-03-23] Key fingerprint = 855A F5C7 B897 6564 17FA 73D6 5D94 1908 AA7A 6805 uid BDR Apt Signing Key for 2ndQuadrant sub 2048R/739C93DD 2015-03-24 [expires: 2019-03-23] and if it is, run: sudo apt-get update (more…)

Support for enums in btree_gin and btree_gist

Andrew's PlanetPostgreSQL
I have just committed a series of patches that provide support for enum types in the btree_gin and btree_gist standard extensions. This is something I first started work on about a year ago. It turned out to be more involved that I had first thought it would be, as it requires some additional core code due to the way that enum comparisons work, which is a bit more complex than for most data types, and involves use of PostgresSQL's internal caching mechanism. The practical upshot of this, however, is that starting with PostgreSQL 10 you will be able to use enum columns in exclusion constraints. That's something that could be very useful - I started this work when I found, somewhat to my surprise, that it wasn't possible.    

JSON version of XMLTABLE example

Andrew's PlanetPostgreSQL
My colleague Álvaro Herrera gave a terrific explanation of the new XMLTABLE feature from Pavel Stěhule that he's put a huge amount of effort into, and finally recently committed. I thought it would be fun to see how the example he gave might work today with JSON. First, I came up with a JSON equivalent version of his piece of XML. Notice that in JSON, unlike in XML, containers are nameless, so we have no "room"  tags, for example, a room is just an object in the relevant array. This is what I came up with: CREATE TABLE IF NOT EXISTS hoteldata AS SELECT json $$[ {"id": "mancha", "name": "La Mancha", "rooms": [ {"id": "201", "capacity": 3, "comment": "Great view of the Channel"}, { "id": "202", "capacity": 5 } ], "personnel": [ {"id": "1025"," (more…)