Wednesday, January 16

Author: Simon Riggs

Oracle’s rising open source problem

Simon's PlanetPostgreSQL
A salesman from Oracle recently spoke to my sales colleague at a conference, teasing him that he should "come make some money". That was quite surprising, given Oracle's well documented problems in gaining new sales, most especially what they say in their own public filings. The reply was unsurprisingly: "No, thanks - I'd like a permanent job." Oracle's rising open source problem Oracle Sales Erode as Startups Embrace Souped-Up Free Software Of course, its well known that squeezing existing customers is one of the things they're doing to avoid a catastrophic slump in sales. The only thing I can (more…)

Finding your PostgreSQL version

2ndQuadrant, Simon's PlanetPostgreSQL
Questions How do you tell what version of PostgreSQL you are using? What is the version number of your PostgreSQL server? What’s the release of your Postgres server? What release is your Postgres database? How do I write a program that acts differently for different versions of the PostgreSQL server? Answer For a long time it’s been possible to write a query to find out what version your server is. SELECT version(); or you could look at that information by the preset parameter SHOW server_version; Both of those mechanisms refer to an internal string, accessible to C programs as PG_VERSION Since PostgreSQL 8.2 we’ve been able to access the version number in a numeric form, allowing us to write programs that depend upon version number for any server version (more…)

PgDay France 2016

Simon's PlanetPostgreSQL
31 May 2016 - PgDay France was a well attended conference with more than 140 attendees, with many presentations from PostgreSQL users further demonstrating just how popular PostgreSQL is now. It was good to see a large well organized conference in Lille, the “capital of Northern France”. Lille is an industrial hub and university town, with an airport to give access to rest of Europe. Fast rail links with London, as well as Brussels and Paris, allowed me to attend easily. I'd not been there before, but I'll be making sure to go back for a better visit. As we should expect, the conference was in French language, a challenge for me I admit, but I was able to follow most of the presentations. Parlée lentement, je peux comprendre parfois. I arrived just after (more…)

Planning to succeed

2ndQuadrant, Simon's PlanetPostgreSQL
PostgreSQL 9.6 has a lot of good features; many of the changes are in the SQL planner, aiming to improve performance by carefully selecting the right execution plan. The great thing here is that doing less work makes many queries much, much faster than they were before. First, we are now using Foreign Key data in the planner to improve estimates. Next, we are combining aggregates to avoid duplicating effort. We're also improving the way that GROUP BY estimation occurs. And we're using partial indexes for index-only scans in more cases. And we've improved estimates for distinct rows, leading to more accurate planning of hash joins and other plan types. Congratulations to my 2ndQuadrant colleagues Tomas Vondra and David Rowley for their insight, rigour and persistence in (more…)

The Physics of Multi-Master

Simon's PlanetPostgreSQL
If you try to update the same data at the same time in multiple locations, your application has a significant problem, period. That's what I call the physics of multi-master. How that problem manifests itself is really based upon your choice of technology. Choosing Postgres, Oracle or ProblemoDB won't change the problem, just gives you choices for handling the problem. If you choose single master, then you get an error because one of the nodes you try to update is read-only and so can't be updated at all. If you have multiple masters, then you get to choose between an early abort because of serialization errors, or a later difficulty when conflict resolution kicks in. Eager serialization causes massive delays on every transaction, even ones that have no conflict problems. A (more…)

Why Logical Replication?

pglogical, Simon's PlanetPostgreSQL
PostgreSQL has built-in streaming replication. Why do we need new replication? Well, in some cases, we do need more. Which is why we have pglogical. The existing replication is more properly known as Physical Streaming Replication since we are streaming a series of physical changes from one node to another. That means that when we insert a row into a table we generate change records for the insert plus all of the index entries. When we VACUUM a table we also generate change records. Also, Physical Streaming Replication records all changes at the byte/block level, making it very hard to do anything other than just replay everything. Logical Streaming Replication sends changes in a more flexible form, sending only the logical change. So when we do an insert we send only the insert record (more…)

Joins Don’t Scale!

Simon's PlanetPostgreSQL
"Joins Don't Scale". Well, that's what I heard MongoDB said anyway. My response was "Huh? Yeh, they do". So what gives? Who is right? Why the mixup? Well, first thing to realise is that the implicit topic we are talking about is massively parallel (MPP) databases, so what they are talking about is the scalability of a join between two tables that are spread across multiple nodes of an MPP database, such as Postgres-XL. All of the MPP databases I've worked with allow you to spread data across multiple nodes, typically using a simple hashing method, using one or more columns. That's often called the Distribution Key, Primary Index or similar. If you have two tables with the same DK, then the join happens evenly across all nodes. That scales very nicely, meaning that if you have twice as (more…)

The End of MongoDB

Simon's PlanetPostgreSQL
On hearing news that MongoDB had lost half its value recently, the phrase "Half sunk" came to mind from Shelley's Ozymandias. I met a traveller from an antique land Who said: "Two vast and trunkless legs of stone Stand in the desert. Near them, on the sand, Half sunk, a shattered visage lies, whose frown, And wrinkled lip, and sneer of cold command, Tell that its sculptor well those passions read Which yet survive, stamped on these lifeless things, The hand that mocked them and the heart that fed: And on the pedestal these words appear: 'My name is MongoDB, king of Databases: Look on my works, ye Mighty, and despair!' Nothing beside remains. Round the decay Of that colossal wreck, boundless and bare The lone and level sands stretch far away (more…)