2ndQuadrant, Professional PostgreSQL

External web tables in Greenplum

Tue, 10 Apr 2012 11:12:00 +0000

External web tables are one of the most useful features when you you have to load data into a Greenplum database from different sources.

Intel SSDs: Lifetime and the 320 vs. 710 Series

Fri, 23 Mar 2012 22:03:25 +0000

This week I've been digging deep into PostgreSQL storage hardware again. Since I'm giving a conference talk on database storage in Austin and in the DC area next week, it seems like a good time for me to actually know the material. One of the most common questions here is "what's the cheapest SSD I can put my database on?", with the implied hope "...without losing it all the time". Last year the first inexpensive answer to that appeared on the market, and I suggested people take a look at Intel's 320 series drives. With 217 days of runtime on my first 320 drive here, and Intel's 3rd generation storage line filled out with the more enterprise oriented 710 Series now, it's worth reviewing how that turned out.

It wasn't long after the 320 series drives were introduced that people started reporting a firmware problem with the drive, where it did things like report a capacity of 8MB after a restart along with "BAD_CTX 0000013x" errors. A firmware update to fix that was released. There's still some claims of continued problems floating around. You have to expect some percentage of any product are going to be bad, and the later production of this drive (after the big bug was fixed) don't seem above the usual risk level in hard drives to me. With the warranty here extended to 5 years (unless you're using it at 'enterprise usage levels'), I think that Intel would be getting killed if the reliability on these was as bad as some people claim.

The reason behind the usage level caveat is the main thing worth talking about here. The long version of the warranty suggests "The media wear-out indicator reports a normalized value of 100 (when the SSD is brand new out of the factory) and declines to a minimum value of 1. When the value reads 1, this indicates that the SSD is reaching the wear-out limit". Here's what my first 320 looks like so far:


[root@toy ~]# smartctl -a /dev/sdc
=== START OF INFORMATION SECTION ===
Device Model:     INTEL SSDSA2CW120G3
...
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  3 Spin_Up_Time            0x0020   100   100   000    Old_age   Offline      -       0
  4 Start_Stop_Count        0x0030   100   100   000    Old_age   Offline      -       0
  5 Reallocated_Sector_Ct   0x0032   100   100   000    Old_age   Always       -       0
  9 Power_On_Hours          0x0032   100   100   000    Old_age   Always       -       5225
 12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       58
170 Unknown_Attribute       0x0033   100   100   010    Pre-fail  Always       -       0
171 Unknown_Attribute       0x0032   100   100   000    Old_age   Always       -       0
172 Unknown_Attribute       0x0032   100   100   000    Old_age   Always       -       0
184 End-to-End_Error        0x0033   100   100   090    Pre-fail  Always       -       0
187 Reported_Uncorrect      0x0032   100   100   000    Old_age   Always       -       0
192 Power-Off_Retract_Count 0x0032   100   100   000    Old_age   Always       -       34
225 Load_Cycle_Count        0x0032   100   100   000    Old_age   Always       -       201389
226 Load-in_Time            0x0032   100   100   000    Old_age   Always       -       2687040
227 Torq-amp_Count          0x0032   100   100   000    Old_age   Always       -       0
228 Power-off_Retract_Count 0x0032   100   100   000    Old_age   Always       -       314526
232 Available_Reservd_Space 0x0033   100   100   010    Pre-fail  Always       -       0
233 Media_Wearout_Indicator 0x0032   099   099   000    Old_age   Always       -       0
241 Total_LBAs_Written      0x0032   100   100   000    Old_age   Always       -       201389
242 Total_LBAs_Read         0x0032   100   100   000    Old_age   Always       -       133021

Not all of these attributes are labeled correctly, and some are "Unknown"; all the gory details are in the product specifications. You can see the Media Wearout above; here's the raw values for other interesting ones, formatted so they're more blog-friendly:


ID# ATTRIBUTE_NAME          RAW_VALUE
  9 Power_On_Hours          5225
 12 Power_Cycle_Count       58
192 Power-Off_Retract_Count 34
225 Load_Cycle_Count        201389
241 Total_LBAs_Written      201389
242 Total_LBAs_Read         133021

192 (hex C0) is "Power-Off Retract Count". That's how many unsafe shutdowns the drive has been through, which are the situations where the battery backed cache in the drive has been triggered. With 34 of them here, you can see I've tried to get this drive to die that way.

The first interesting wear figure is 225 (hex code E1) which Intel's documentation describes as "Host Writes". The units for that are 32MB. If you look carefully, you'll see that's the same value given in 241 "Total LBAs Written". That suggests the LBA unit for the drive is also 32MB, which I double-checked last year. At 32MB each, my write value of 201389 means I've written 6.15TB to this drive.

Now, computing the true lifetime of an SSD depends on a couple of magic values, like the "write amplification" of your workload. That suggests how often your workload forces small bits of data out to flash, using up some of the NAND cell lifetime faster than it might otherwise last. These numbers are really hard to estimate. The most realistic way is figure this out is to run a workload simulation after resetting the drive's internal counters, then see just how much you burned through. The process is walked through with an example at "Measuring How Long Your Intel SSD Will Last", and it's not too hard to translate that example (which uses Intel's SSD Toolbox software) into a set of of smartctl commands if you're on Linux--the article even uses smartctl for the counter reset part.

The official documentation is this is Intel's Enterprise Server addendum, and here we finally find some hard numbers about the expected life of these drives. My 120GB drive is said to have a "write endurance" of 15TB. A pessimistic look at my sample drive here would check total writes and say that, having written over 6TB, I've gone through 40% of the drive lifetime. But write endurance doesn't work that way; the firmware is constantly doing tricks to extend the life of the drive. Intel's official number they sometimes tie the warranty to, the Media Wearout, is showing 99% left! If that's true--I've only used 1% of the drive's lifespan--then I might manage 600TB of writes before this one really dies on me.

So what's the story with the true Enterprise lifetime 710 Series drives? Those almost the same drives as the 320 series ones, with three significant changes. First, they're said to use higher quality flash, probably with the same sort of "put the best tested chips first in the expensive models" approach Intel is said to use on their CPU production--what's sometimes called binning. Second, the drives are overprovisioned with a lot more unused flash compared to the 320 series models, and unused flash really helps extend longevity. Finally, they don't claim the capabilities to be quite as good. Random write IOPS numbers on the 710 series drives are lower; my 120GB 320 series drive is specified at 14K write IOPS, while the 100GB 710 series only aims for 2700. The drive doesn't claim to support lots of tiny writes and still last for years, which means it's aimed at a different set of write amplification expectations. Similarly, the 710 series drives don't refresh the stored cells in the same way. The downside there is that 710 models are only specified to retain their data for 3 months. That's probably fine for data center use, but that wouldn't be very acceptable to the more consumer oriented market the 320 series is sold to.

The end result of that, and how the 710 compares to the 320 series drives, is nicely summarized in the "Write Endurance" table in the Tom's Hardware Review. Instead of the 15TB endurance number my 320 drive specifies, the similar 100GB 710 series model aims for 500TB. That's just over 30X as long. In the real world, there may not be that big of a difference, as shown by the projected 600TB figure I'm seeing out of my 320 drive so far. But Intel's aiming at conservative engineering lifetimes on the specification sheets, and by that measure the storage cells 710 will last longer; the 320 models only may last longer. And an expected lifetime 30X as long is something some people are surely willing to pay the 710's price premium for.

Using the PostgreSQL System Columns

Tue, 31 Jan 2012 21:53:43 +0000

There are a few parts of the PostgreSQL internals that poke out usefully if you look in the right place for them. One useful set to know about are the System Columns, which you can explicitly request but don't see by default. For example:

psql -x -c "SELECT oid,* FROM pg_class LIMIT 1"

There is no column named oid in the pg_class table, but it's there if you ask for it. The oid used to be relied on more heavily in PostgreSQL as a way to identify rows. That's not true for regular tables anymore, and you really don't want to start doing that for your own tables. OIDs are mainly useful now when joining parts of the System Catalog together. A good example is the Disk Usage query. If you want to find the namespace a table is in, you need to know you can ask for its OID. It's possible to get some of this data out of more portable views like information_schema.tables. But many of the useful things in this area are PostgreSQL specific. Sometimes I see people starting with the information_schema views and joining against other tables using its text name fields, such as the listed table_name. That approach has several edge cases that don't work out correctly; not handling TOAST columns is a common example. That makes them more prone to breaking on you later, probably after your system has gone into production, than an OID based join.

There is also a tableoid system column. As described in the documentation, its main use case is identifying which partition a row come from. That's not a great thing to be driving application logic from, but it can be useful for monitoring or troubleshooting purposes. For example, if you SELECT rows from the parent table in a partitioning inheritance scheme, it's normally expected that no rows will actually be stored there. Checking the tableoid is one way to confirm that. You might confirm that your INSERT/UPDATE trigger is moving rows to the right place using tableoid as well. It's possible to do that for each individual partition section, but running a query against the parent will make sure you hit every row in the table.

Another internal column related to uniquely identifying rows is the ctid. The ctid is a direct pointer to the physical block (using PostgreSQL's 8K page size) and position of a row. ctids are a pair of numbers, and the first row will be (0,1). While this is the fastest way to find a row more than once in the same transaction block, these numbers are not stable in the long term. Any UPDATE and some maintenance operations will change them. One thing you can use these for is finding duplicate data in a table. Let's say you're trying to add a unique constraint, but one row in the table is duplicated 3 times, which blocks the unique index from being created. When rows are identical in every column, you can't write any simple SELECT statement to uniquely identify them. That means deleting all of them but one copy requires some annoying and fragile SQL code, combining DELETE with LIMIT and/or OFFSET--which is always scary. If you use the ctid instead, the implementation will be PostgreSQL specific, but it will also be faster and cleaner. See Deleting Duplicate Records in a Table for an example of how that can be done.

The other system columns all relate to transaction visibility: xmin, cmin, xmax, cmax. When you delete a row in PostgreSQL, it isn't eliminated from disk immediately. It's possible that some other query that's executing at the same time will still need to see that row, and the transaction isolation in PostgreSQL worries about such things. If you ever want to learn how that isolation works, the way the Multiversion Concurrency Control (MVCC) implementation is handled, you can watch parts of it happen. Just open transactions in two different sessions, UPDATE/DELETE in one of them, and then look at those rows in the other. You can still see them in the session where they weren't touched, but they'll be marked to expire in the future via their xmax being set. To really pull that all together, you also need to know about some of the System Information Functions. txid_current() is the most useful for this sort of learning experience, it provides a reference point for the always increasing system transaction ID. You can find a more detailed exploration of using these functions and system columns in Bruce's MVCC Unmasked talk. The "Routine Maintenance" chapter of my book also shows examples how how MVCC works through the perspective of the system columns.

Setting JDBC with Greenplum

Thu, 19 Jan 2012 12:16:59 +0000

JDBC is the driver used to access a database with Java. Greenplum has a full working JDBC implementation. In this short article we'll see how to use it.

Greenplum 4.2 is out!

Mon, 02 Jan 2012 08:39:34 +0000

With an announce on the forum, Greenplum staff has spoke out about the new version of their Database Management System. I can't resist to blog about some of its new features.

How to initialize Greenplum on multiple nodes

Fri, 16 Dec 2011 12:00:33 +0000

In the previous article we have seen how to install Greenplum on multiple nodes. After installation steps, we must init the entire system. Let's see how.

A Greenplum 4.1 installation handbook

Mon, 05 Dec 2011 17:32:53 +0000

One of the main advantages using Greenplum is that it gains power when it uses multiple nodes. Horizontal scalability is a main feature of Greenplum.

Here is a compact handbook to install a multi-node Data Warehouse environment with Greenplum.

PGDay.IT 2011 was "bellissimo"!

Mon, 28 Nov 2011 12:26:39 +0000

The fifth edition of the Italian PGDay went well beyond our initial expectations. We had about 75 participants, a total of 95 people including staff and speakers.
As I said during the event, rather than PGDay Italy, this should be named PGDay for Italian speakers given the presence of staff from Switzerland (Canton Ticino). Participants came from 12 regions: all regions but Val d'Aosta in the north/centre area, but also from Southern Italy (Naples and Calabria).

Using Greenplum 4.1 in Ubuntu 11.10

Mon, 28 Nov 2011 08:36:08 +0000

Greenplum does not officially support Ubuntu Server 11.10 as underlying operating system. However, I needed to install it on the most recent Ubuntu server just to perform some tests and evaluate it.

Mapreduce in Greenplum 4.1 - 2nd part

Thu, 17 Nov 2011 14:25:09 +0000

Through this article, we are going to complete the MapReduce job started in the previous article.

Global trends in deploying PostgreSQL

Mon, 14 Nov 2011 18:08:39 +0000

This year's conference lineup led me all over the world, a giant rectangle triangle going from the west coast of the US, north to Canada, east to the UK and Amsterdam, then ending south in Brazil. I've now locked myself in to focus on the 3rd CommitFest for PostgreSQL 9.2, which began a few days ago. Check out the 2011-11 section of the CommitFest tracker to see what changes have been submitted, we're always looking for new volunteer patch reviewers.

Talking to people deploying PostgreSQL in several countries during a short span of time has given me some interesting perspective on where the project is at. I follow a lot of adoption trends in the US, and some of those I assume are quirks in how business is done in this country. But when I hear the same sort of feedback from people in all four of the countries I've been to this year, too, it's clear this is a larger issue.

The first thing I'm seeing a surprising amount of is satisfaction with the feature set in PostgreSQL. A few years ago, conversations about what you could and couldn't do with PostgreSQL usually stalled on one of a few common requests. There's a good survey of PostgreSQL feature feedback at User Voice. 13 important features originally on that list have been closed already, with Index Only scans as the next expected to fall in the upcoming 9.2. PostgreSQL now includes regular and synchronous replication as of 9.1. pg_upgrade has been getting an increasing amount of testing that proves it works for many in-place upgrade scenarios. Extensions are dramatically easier to use now.

It seems the total feature set has crossed the threshold where PostgreSQL is good enough for a whole lot more deployment situations than it used to be. What I'm hearing from people all over the world now is that the basic feature set and performance of PostgreSQL isn't failing the "checkbox test" so often anymore, where business people require certain things before they'll even consider a database. There are some major wants that are some distance off, such as materialized views and better OLAP support (cube/rollup/etc.) And using partitioning for bigger data sets is harder than people would like. But these are all things that are only needed for larger deployments, and some workarounds exist if you're willing to work at them.

If the feature set isn't holding back as many deployments now, what is? Well, the next thing I've been hearing everywhere is on that survey list too: better administration and monitoring tools. You really need a whole open-source stack to monitor PostgreSQL right now, from OS+database trending to query log analysis. It's fine for these tools to live outside the database core, but some changes are clearly needed to make such tools easier to write. For example, the one built-in tool that allows query monitoring is pg_stat_statements, and the limitations preventing it from being useful to most people are so obvious we've gotten two submissions to improve it in the last month.

There are a few projects that aim at the monitoring/administration problem. EnterpriseDB's PostgreSQL Enterprise Manager, Cybertec's pgwatch, OmniTI's Reconnoiter, the suite of smaller tools from End Point, and even the text UI of pg_statsinfo all hit the edges of this problem. What I hear when I have my advocacy hat on is that the community needs a major open-source project bigger than any of these to make database monitoring easier. That's now one of the major distinguishing features the commercial competition has. Getting enough of the people developing in this area all pointing in the same direction and working together is a big challenge though.

On a related note, now that the underlying features are there, it seem making replication easier to monitor and setup is a major issue too. There are so many choices in replication technologies available for PostgreSQL it's easy for new people to get overwhelmed by them all. And the documentation guides around this area are still filled with a lot of complications that aren't even really necessary to get started at this point. It's easy for newcomers get dragged into details like how old style archiving works as a precusor to setting up even basic replication, despite that they're using the easier features in the current PostgreSQL instead. This area still has some work in the core database happening in 9.2, and it will be important for the community to create replication guides that include current information covering both 9.1 and that release. What I'm hear from every country I visit now is "I need material to help me compete against the idea of using Oracle RAC".

The last of the global trends that have really jumped out at me is how companies everywhere are reinventing the development process around database applications. In some places, mostly bigger companies and government installations in particular, the expected staff "stack" is business as usual; it hasn't changed in a long time. New applications go from Developer to DBA to systems administrator. Management ideas like DevOps are catching on to improvement interface between these roles, but not really upset its basic structure. Everywhere I go now, I'm seeing everything but the developer role being squeezed out. ORM-driven development is eliminating the DBA's role in database design. Managed application hosting platforms are wiping out the systems administrator role. Startups with an idea for a web application go right from developer to deployment, and happily this is increasingly happening with a PostgreSQL backend in the database role. There isn't even the perception that DBA-like help might be needed until the application grows quite a bit. I'm seeing the need for better database specific optimization skills than a typical developer has being deferred until the application has tens of gigabytes of data to sling around.

Being able to deploy small PostgreSQL installs and grow them to a reasonable size without specialized DBA knowledge is a great thing as far as I'm concerned. The exact advances in things like ORMs that have allowed reaching this point across the world are a topic that deserves its own long discussion. I'm going to cut this off here and return to that later. In this country, there's some concrete work around the 9.2 release that needs to get done this month.

Performing ETL using Kettle with GPFDIST and GPLOAD

Mon, 07 Nov 2011 09:30:00 +0000

Scenario:

We have a remote datasource, served by a gpfdist server. We need to import the data in a Greenplum database, while performing some ETL manipulation during the import.
It is possible to accomplish this goal with a simple transformation in a few steps using Kettle.

Mapreduce in Greenplum 4.1

Mon, 31 Oct 2011 13:46:11 +0000

Mapreduce is a very trendy software framework. It has been introduced by Google (TM) in 2004. It is a large topic, and it is not possible to cover all of its aspetcs in a single blog article. This is a simple introduction to the mapreduce usage in Greenplum 4.1.

Apple's Lossless Audio Codec and Software Patents

Fri, 28 Oct 2011 13:07:45 +0000

Today my mailbox was crowded with some Apple news. The source code to the Apple Lossless Audio Codec, encoders and decoders, was released to the world. A few open-source projects such as FFmpeg and FAAC has already had support for the resulting .m4a file; this gives them an official release to validate against. As a well documented audio snob, I'm known to be a fan of lossless audio, with a few hundred CDs here ripped into FLAC. Unfortunately, this source code drop doesn't change anything for me, and that's all because of software patents. Right now I consider such patents to be the greatest risk to software I work on like PostgreSQL, so I spent some time looking into just what Apple has done here, as a data point on that topic.

If your baseline is MP3 files, Apple Lossless appears to be a step forward as far as licensing goes. All MP3 playback requires a patent license, while playback of Apple's format does not--only the encoding side need be licensed. That's not my baseline though. FLAC has specifically avoided using any known patented technology, so it's the clear winner in being clean of patent issues.

You can't even read the just released source code without being shown the door that leads to Apple's patents. ALACEncoder.cpp pushes you that way when you read it, saying "The relevance of the ALAC coefficients is explained in detail in patent documents." So you can't fully understand what this code does unless you dip into the patent description. That's a big sign of trouble. I'm not sure exactly which patent they are referring to; it may be Determining scale factor values in encoding audio data with AAC.

I'm not sure because I don't read patent descriptions if I can avoid it, due to how damages are tripled with willful infringement. That rule puts open-source developers in a weird place. The act of researching which patents your free implementation might infringe on can have a wildly negative return on investment. If it's impossible to implement the idea without infringing in the patents you find, which can easily be the case given the ridiculously low bar for grating such patents, if a lawsuit does happen you'd be better off not knowing about that risk when it starts. On the PostgreSQL mailing lists, mentioning how a patented implementation of something works is one of the few things that will get you a warning and potentially blocked from the lists. Knowingly implementing a patented idea can be a very expensive mistake for the project to make.

Now, you can claim this problem has gone away for this bit of software due to how Apple has released this code under the Apache 2.0 license. If you build something using the Apple Lossless code, and you distribute the result under the Apache 2.0 license, you get the protection of its Grant of Patent License. You'll only lose it if you sue someone in a way that involves this code+patent combination. That doesn't necessarily protect manufacturers of hardware who want to include this capability, as they may not want to license the result that way. I don't really care about them though. People who want to take advantage of open-source code but not contribute back to can worry about their own legal issues, they're not my concern.

My real issue with using this format is that it implicitly approves of business practices around software patents I find hostile to my own work, and that leads me back to the patent-less FLAC again. Apple used to more often be the victim of valid and frivolous patent claims, but increasingly they've become the originator of them instead. In mid 2006, Apple was sued over patent issues by Creative Technology; they settled paying Creative $100M dollars. Since then, patent issues have increasingly been part of their well exercised legal arm.

Starting last year, Apple has seriously turned around in how it handles patents--it's now aggressively enforcing its patents rather than just being sued for violations. Steve Jobs issued a warning shot about that, saying "competitors should create their own original technology, not steal ours". Considering it had only been four years before then that Apple was sued--and paid out in a big way--for stealing other people's patented ideas, that came off as rather hypocritical to me.

The company has done well staying ahead of its competitors by production innovation, rather than court fights. iPod competitors failed to gain traction because their products weren't as good, not because they couldn't steal the design. Apple's iPad should have the same property; the knock-offs are not selling because they're not as good. The only real threat to their product line right now are how Android phones are displacing the iPhone. Dumping so much money into offensive lawsuits is burning up money that could be used for real product advances there instead. It's a shame they've resorted to this tactic.

At this point, a cautious person would avoid using technology encumbered by Apple's patents, as they clearly have the means and intent to sue for violations. And someone who values open source projects should avoid patented approaches even when they are freely licensed. Whether or not individuals using Apple Lossless is particular are exposed to problems here is missing the big picture.

As an advocate for free software, I reflexively pick the less patent encumbered approach to any problem, using that as the tie breaker for decisions that are otherwise even. Encoding into and helping popularize Apple Lossless may be legal now. But I'll keep encoding into FLAC, copying the result onto my Sansa Fuse player, and avoiding their entire music ecosystem. Software patents are too dangerous to implicitly endorse them if it can be avoided, and here they easily can.

Sync rep scaling

Wed, 26 Oct 2011 17:31:04 +0000

I'm almost done with this year's crazy conference season schedule, just Brazil's PG.BR next week left. All of my recent presentations are now available at our talks page. You'll also find many of the talks from our CHAR(11) conference this summer there for the first time. Those were unavailable for a while due to an unfortunately timed web site change.

I like to do some original research for my talks, and this year that included a look into Synchronous Replication and Durability Tuning in PostgreSQL 9.1, specifically the performance side. At last week's PG.EU I gave an updated version of this talk (with Simon Riggs), including a bit more info than I had available during the Postgres Open presentation on the same topic.

The good news/bad news performance results are nicely summarized by the "Group commit magic" graph on slide 17. There I was replicating across the Atlantic Ocean, crossing from my home here in Baltimore over to the conference site in Amsterdam. When doing synchronous replication, the speed of light determines how fast any single client can commit. You can only reach about 50% of that round trip time given current (and expected future) technology here. The efficiency rate of the fiber-optic cables used is not perfect, and tasks like routing add some overhead too.

With that sort of distance, the round trip time is at least 100 milliseconds. What this means is that no one client can commit more than 10 times per second. This shows just how difficult sync rep is to run well, the maximum rate drops fast if you expect to leave your local data center to commit. Light turns out to be really slow compared with what many people expect.

The great thing I proved in that slide is that the efficiency when multiple clients are committing at the same time scales almost linearly. If you want 1000 transactions/second over this sort of distance, you can get that--but you'll need just over 125 clients going at once to do it. Each one of those clients will be seeing 10 commits/second and 100 millisecond latencies (and higher for a small percentage, peak latency in the test was closer to 600ms). But each commit reply will be acknowledging a pile of clients at once, just by sending a small packet with an updated "committed up to this point" response.

When the speed of light turns out to be your bottleneck, there's not much you can do about that attacking directly. Some people who want to reach higher rates might architect their systems with many smaller clients, as I've shown in this example. I was able to reach my goal of 2000 INSERT commits/second here, but it took 275 clients to get there.

The other great thing about how PostgreSQL implements sync rep is that it's controlled per transaction. Once you see how expensive the commits are, if that's too high for some of your data, you can always tweak that for higher performance just by disabling sync rep for some transactions. Having such fine-grained control over synchronous commits is a unique feature to PostgreSQL, allowing something like a Quality of Service suggestion to the database. The PostgreSQL code as of 9.1 really has an unprecedented range of trade-offs here. You can go for faster but not very durable at all (with unlogged tables), locally durable but not guaranteed to a remote data center at a medium speed, all the way up to fully synchronous and very expensive to commit. It's possible to argue that other database choices are better at one end of this range. You might use MongoDB for higher speeds at the low durability range, and Oracle for their better tested sync rep capabilities (I saw better tested simply because the PostgreSQL 9.1 code is very new relative to Oracle's implementation).

PostgreSQL started in the middle here, and with 9.1 it's expanded nicely toward both ends of the spectrum at once. It's now providing options for higher and lower durability at the same time, in one database, and with the speed/durability trade-off adjustable for every transaction. Building one size fits all software is really hard, and the new features in 9.1 nicely push out capabilities here for several popular use cases, all at the same time, and only when you want to pay for them.