Monday, February 18

PostgreSQL with passphrase-protected SSL keys under systemd

Eisentraut's PlanetPostgreSQL
PostgreSQL supports SSL, and SSL private keys can be protected by a passphrase. Many people choose not to use passphrases with their SSL keys, and that's perhaps fine. This blog post is about what happens when you do have a passphrase. If you have SSL enabled and a key with a passphrase and you start the server, the server will stop to ask for the passphrase. This happens automatically from within the OpenSSL library. Stopping to ask for a passphrase obviously prevents automatic starts, restarts, and reboots, but we're assuming here that you have made that tradeoff consciously. When you run PostgreSQL under systemd, which is very common nowadays, there is an additional problem. Under systemd, the server process does not have terminal access, and so it cannot ask for any (more…)

Webinar: Banking on Postgres – Financial Application Considerations [Follow up]

Liaqat's PlanetPostgreSQL, PostgreSQL
The demand for PostgreSQL within the financial industry has been rapidly increasing in the recent years; mainly due to reduction in licensing costs, better choice of open source tools, and the robust enterprise features that PostgreSQL provides. 2ndQuadrant hosted the "Banking on Postgres" webinar to discuss attributes of financial databases based on Postgres, configuration processes, hardware needs, availability, backups, and more. The webinar was presented by Shaun Thomas, Principal Consultant at 2ndQuadrant. Those who weren’t able to attend the live event can now view the recording here. For any questions or comments regarding Postgres-BDR, please send an email to [email protected] (more…)

PG Phriday: Terrific Throughput Tracking

Shaun's PlanetPostgreSQL
Postgres has a lot of built-in information functions that most people don't know about. Some of these are critical components to identifying replication lag, but what if we could use them for something else, like throughput? This man's one simple trick can track actual database throughput; DBAs hate him! Everybody Knows Let's take a look at a common query we might use to track replication lag between a Postgres 11 Primary and one or more Replica nodes. SELECT client_addr, pg_wal_lsn_diff( pg_current_wal_lsn(), sent_lsn ) AS sent_lag, pg_wal_lsn_diff( pg_current_wal_lsn(), write_lsn ) AS write_lag, pg_wal_lsn_diff( pg_current_wal_lsn(), flush_lsn ) AS flush_lag (more…)

Maintaining feature branches and submitting patches with Git

Eisentraut's PlanetPostgreSQL
I have developed a particular Git workflow for maintaining PostgreSQL feature branches and submitting patches to the pgsql-hackers mailing list and commit fests. Perhaps it's also useful to others. This workflow is useful for features that take a long time to develop, will be submitted for review several times, and will require a significant amount of changes over time. In simpler cases, it's probably too much overhead. You start as usual with a new feature branch off master git checkout -b reindex-concurrently master and code away. Make as many commits as you like for every change you make. Never rebase this branch. Push it somewhere else regularly for backup. When it's time to submit your feature for the first time, first merge in the current master branch, fix any (more…)

PG Phriday: PgBouncer or Bust

Shaun's PlanetPostgreSQL
What is the role of PgBouncer in a Postgres High Availability stack? What even is PgBouncer at the end of the day? Is it a glorified traffic cop, or an integral component critical to the long-term survival of a Postgres deployment? When we talk about Postgres High Availability, a lot of terms might spring to mind. Replicas, streaming, disaster recovery, fail-over, automation; it's a ceaseless litany of architectural concepts and methodologies. The real question is: how do we get from Here to There? The Importance of Proxies It's no secret that the application stack must communicate with the database. Regardless of how many layers of decoupling, queues, and atomicity of our implementation, data must eventually be stored for reference. But where is that endpoint? Presuming that write (more…)

Sequential UUID Generators on SSD

2ndQuadrant, Tomas' PlanetPostgreSQL
After I shared the sequential UUID benchmarks a couple of weeks ago, one of the points raised in feedback was the choice of the storage space. I've intentionally used a fairly weak storage system (RAID10 on three 7.2k SATA drives) because I wanted to demonstrate the benefits. But a couple of readers suggested using SSDs might significantly reduce the difference between regular and sequential UUIDs due to SSDs handling random I/O much better than rotational storage. My hypothesis was that while using SSDs may reduce the gap, it certainly won't eliminate it entirely because the amplification (both in terms of number of I/O requests and WAL volume) is independent of the storage system. But the only way to verify this it is to repeat the tests, this time on SSDs. So here we go ... (more…)

Webinar: pglogical and Postgres-BDR Update [Follow Up]

2ndQuadrant, Liaqat's PlanetPostgreSQL
Since the release of v3, Postgres-BDR has evolved into the go-to clustering technology built specially for businesses that require geographically distributed databases with multiple masters. To get an update on Postgres-BDR’s development, new features, and future roadmap, 2ndQuadrant held the pglogical and Postgres-BDR Update webinar as part of its PostgreSQL webinar series. The webinar was presented by Simon Riggs, Founder & CEO of 2ndQuadrant, who is also a major contributor of the open source PostgreSQL project. Those who weren’t able to attend the live event can now view the recording here. For any questions or comments regarding Postgres-BDR, please send an email to [email protected] (more…)

PostgreSQL 9.3 EOL – Why is it Important to Upgrade?

2ndQuadrant, Liaqat's PlanetPostgreSQL, PostgreSQL
After the final release of patch 9.3.25 on November 8th 2018, PostgreSQL 9.3 is no longer supported. Therefore it’s time for all users of PG 9.3 to upgrade their databases to a newer supported version. The benefits of having a supported version are many and that’s what Craig Ringer talks about in the Q&A session below: Why is it important to upgrade your PostgreSQL database to the latest version? Craig: Always update to the latest minor version. 2ndQuadrant's 24/7 support services often help customers who could've avoided experiencing a production outage or fault simply by updating every minor version or two. The PostgreSQL community (including 2ndQuadrant) releases minor point releases conservatively, and for good reasons. Keep up to date on those patches. Note: A "minor (more…)

[Video] Ansible and PostgreSQL

PostgreSQL, Tom's PlanetPostgreSQL
I don’t often get to speak on technical topics, but the video of my presentation below covers various concepts around "Ansible and PostgreSQL" - something I am very enthusiastic about. This presentation covers the following topics: Overview of Ansible and PostgreSQL Best strategies for mixed cloud and on-premises deployments How to deploy AlwaysOn PostgreSQL clusters How to perform maintenance updates How to create a variety of cluster types How to backup servers for mixed on-premises and multi-cloud deployments

Databases vs. encryption

2ndQuadrant, Tomas' PlanetPostgreSQL
Let's assume you have some sensitive data, that you need to protect by encryption. It might be credit card numbers (the usual example), social security numbers, or pretty much anything you consider sensitive. It does not matter if the encryption is mandated by a standard like PCI DSS or if you just decided to encrypt the sensitive stuff. You need to do the encryption right and actually protecting the information in both cases. Unfortunately, full-disk-encrytion and pgcrypto are not a good fit for multiple reasons, and application-level encryption reduces the database to "dumb" storage. Let's look at an alternative approach - offloading the encryption to a separate trusted component, implemented as a custom data type. (more…)