Studying Stored Procs in Postgres 11
With Postgres 11 looming on the near horizon, it's only appropriate to check out a recent beta and kick the tires a few times. Whether it's improvements in parallelism, partitions, stored procedures, JIT functionality, or any number of elements in the release page, there's a lot to investigate.
It just so happens that I ran across a fortuitous event on Twitter when deciding on an appropriate topic. Behold!
Wait! No! That's not what stored procedures are for!
I felt so good like anything was possible
When confronted with such a blatant corruption of such a cool new feature, it's only natural to question the wisdom of doing so. It is, after all, not a great idea to programatically consume transaction IDs. I said as much and moved on with life
Postgres contains a moving event horizon, which is in effect about 2 billion transactions ahead of or behind the current transaction id. Transactions more than 2 billion ahead of or more than 2 billion behind the current transaction id are considered to be in the future, and will thus be invisible to current transactions.
Postgres avoids this catastrophic data loss by specially marking old rows so that no matter where they are in relation to the current transaction id they will be visible.
Freezing is this process of marking old live tuples (i.e. database rows) so that they don't get run over by the moving event horizon that would otherwise make them appear to be in the future. This is in contrast to vacuuming, which is the freeing up of space consumed by old dead tuples that are no
We seldom credit patch reviewers. I decided to pay a little homage to those silent heroes for a few of them: here's the list of people who were credited as having reviewed the patches mentioned in my previous article for PostgreSQL 11. The number in front is the number of times they were credited as reviewers.
Amit Langote, Robert Haas
Dilip Kumar, Jesper Pedersen, Rajkumar Raghuwanshi
Amul Sul, David Rowley, Rafia Sabih, Simon Riggs, Thomas Munro
Antonin Houska, Ashutosh Bapat, Kyotaro Horiguchi
Álvaro Herrera, Amit Kapila, Amit Khandekar, Etsuro Fujita, Jaime Casanova, Keith Fiske, Konstantin Knizhnik, Pascal Legrand, Pavan Deolasee, Rajkumar Raghuanshi, Rushabh Lathia, Sven Kunze, Thom Brown, Yugo Nagata
The video of my presentation below walks you through 'Indexing in PostgreSQL' - a key component to database performance.
This presentation covers the following topics:
Various ways to access data in PostgreSQL
Different types of indexes supported by PostgreSQL
Internals of BTree and BRIN indexes
Overview of GIN and GiST indexes
How to find missing indexes
How to find unused indexes
Database security is an increasingly critical topic for any business handling personal data. Data breach can have serious ramifications for an organization, especially if the proper security protocols are not in place.
There are many ways to harden your database. As an example PostgreSQL addresses security using firewalls, encryption and authentication levels among other ways.
2ndQuadrant hosted a webinar on Database Security in PostgreSQL to highlight security concepts, features and architecture. The webinar was presented by Kirk Roybal, Principal Consultant at 2ndQuadrant - the recording is now available here.
Some of the questions that Kirk responded to are listed below:
Q1: What are your thoughts on performance of row-level security vs. doing that filtering via WHERE at
About a year ago, we published PostgreSQL 10 with support for native logical replication. One of the uses of logical replication is to allow low- or no-downtime upgrading between PostgreSQL major versions. Until now, PostgreSQL 10 was the only PostgreSQL release with native logical replication, so there weren't many opportunities for upgrading in this way. (Logical replication can also be used for moving data between instances on different operating systems or CPU architectures or with different low-level configuration settings such as block size or locale -- sidegrading if you will.) Now that PostgreSQL 11 is near, there will be more reasons to make use of this functionality.
Let's first compare the three main ways to upgrade a PostgreSQL installation:
PostgreSQL installer or PGInstaller - is a user-friendly, graphical installation and configuration tool for PostgreSQL. With just a few clicks you can install PostgreSQL - version 9.5, 9.6, 10 and 11(beta) - on Windows, Linux and macOS.
The latest PGInstaller release includes support for OmniDB - an interactive and user-friendly database management tool to manage multiple databases in a unified workspace.
Using utilities bundled with PostgreSQL, the only means to connect to the database is via psql. Psql works via the command line, which can be fairly tricky for new users especially if they are migrating from another database and are not used to the interface. PGInstaller makes the connection process easier with OmniDB.
PGInstaller comes bundled with OmniDB, as an optional
Just in case you missed the live broadcast, the video of my presentation below covers various topics around integration of PostgreSQL with other data sources and database technologies.
This presentation covers the following topics:
What is a Foreign Data Wrapper?
How to query MySQL, a Flat file, a Python script, a REST interface and a different Postgres Node
Perform all of the above simultaneously
Take snapshots of this data for fast access
Tweak remote systems for better performance
Package as an API for distribution
Stream to Kafka
Write data to... MongoDB!?
What does all of this have in common?
It's an exciting topic, and I hope more developers and admins begin to see Postgres as the global integration system it really is
In my previous post we looked at various partitioning techniques in PostgreSQL for efficient IoT data management using IoT Solution. We do understand that the basic objective behind time based partitions is to achieve better performance, especially in IoT environments, where active data is usually the most recent data. New data is usually append only and it can grow pretty quickly depending on the frequency of the data points.
Some might argue on why to have multiple write nodes (as would be inherently needed in a BDR cluster) when a single node can effectively handle incoming IoT data utilizing features such as time based partitioning. Gartner estimated 8.4 billion connected devices in 2017, and it expects that this number will grow to over 20 billion by 2020. The scale at which
I spent a couple of days in São Paulo, Brazil last week, for the top-notch PGConf.Brazil 2018 experience. This year I gave a talk about improvements in the declarative partitioning area in the upcoming PostgreSQL 11 release — a huge step forward from what PostgreSQL 10 offers. We have some new features, some DDL handling enhancements, and some performance improvements, all worth checking out.
I'm told that the organization is going to publish video recordings at some point; for the time being, here's my talk slides.
I'm very happy that they invited me to talk once again in Brazil. I had a great time there, even if they won't allow me to give my talk in Spanish! Like every time I go there, I regret it once it's time to come home, because it's so easy to feel at home with the