-- this returns a different result each time it is ran
with recursive s as (
  select random()
union
  select random() from s
) select count(*) from s;

I like this example: a surprising result, which can be explained by (and indeed helps to explain) CTE behaviour.

Unexpected truths are denoted with the word “paradox”, and in fact this is a manifestation (an “instance”, in programmers’ jargon) of what is known as the Birthday Paradox.

Its simplest formulation is probably this: if you randomly choose 23 persons, the probability that two of them share the same birthday is greater than 50%.

The result is unexpected, because there are 366 different birthdays, and the number 23 seems very small compared to 366.

However it is correct, as it can be shown with a direct computation. In PostgreSQL we can run another recursive CTE:

WITH RECURSIVE r(i,acc) AS (
  SELECT 1, 1 :: double precision
UNION
  SELECT i + 1,
    acc * (((366 - i) :: double precision) / 366)
  FROM r WHERE acc > 0.5
) SELECT count(1) FROM r;

producing 23 as the result.

A recursive CTE stops when the recursive step does not add any new rows. In the last query, acc represents the probability that the first i birthdays are distinct, so recursion stops when that number is not above 50%.

In the query mentioned at the beginning, which we’ll call the “random query” for short, the recursive CTE terminates when random() does not add a new row. That is, when the randomly-computed value has already been computed in a previous iteration; that’s because the recursive CTE is using UNION instead of UNION ALL.

This is indeed the Birthday paradox, with 366 replaced by the maximum possible number of distinct values that random() can produce. What exactly is that number?

The random() function returns a double precision value, whose exact definition depends on the system. Not all the double precision values can be produced, though; the underlying C function can produce 2^31 different results, regardless of the bit size of a double precision value. This is good enough in practice, and at the same time compatibility with all the various architectures and library implementations is ensured.

So we can replace 366 with 2^31 in our query, and instead of 23 we get 54563 as the answer.

Does it come near to the actual output of the random query? Let us run it a few times, collect the result, and compute the average:

gianni=# create table t(n int);
CREATE TABLE

gianni=# with recursive s as (
  select random()
union
  select random() from s
) insert into t select count(1) from s;
INSERT 0 1

/* repeat the last query 49 times */

gianni=# select count(1), avg(n) from t;

 count | avg
-------+--------------------
    50 | 54712.060000000000
 (1 row)

The average of the actual results is quite close to the expected threshold of 54563; the difference is less than 0.3%, quite orthodoxically, we might say!

The topic of the first of my two talks has been a collection of PostgreSQL objects that play chess, either between themselves or against a human (see this nice photo, thanks steerio!).

My intention was to present the software, tidy up the code a bit (out of respect for the fellow PostgreSQL user :-) and then release it as free software, for the usual reasons, if there was any interest during the talk.
That being the case, I have named the program pgChess and published the code on GitHub so that the interested reader can verify my claim that the code weights just around 20KB.
I split the core of the code in two files: the pgchess library, which in turn invokes the 2podg library, which stands for “2-player open deterministic games”: open meaning that each player knows 100% of the status of the  game, and deterministic because the whole game is determined uniquely by the choices that the players make.
Two (counter)examples: most card games are not open; Backgammon is not deterministic.
The strategy is quite simple, and is tailored on PostgreSQL features. As short as I can:

• custom types are defined both for the game state, and for a single move
• the chessboard is defined as an 8×8 array
• a quite large function computes the valid moves available from a given game state
• a recursive algorithm populates the tree of possible moves, up to the specified level (to David Fetter: I am really looking forward to writable CTEs for this part!)
• a recursive SQL query selects the “best” move, according to the desired strategy which is encoded in the query itself
• a multi-transactional psql script displays the chessboard, thanks to the Unicode chess characters: ♟,♜,♞,♝,♛ and ♚ for the Black; ♙,♖,♘,♗,♕ and ♔ for the White.
• players can type their move like “b2Pb4″, or “g8Nh6″ (the piece letter always uppercase regardless of the colour).

The reason for the talk is to show that PostgreSQL is an excellent framework to develop application prototypes quickly (which is not a big secret; but providing one more example reinforces the claim). And speed prototyping is one key summand that I recall from Simon Phipps in his remarkable keynote speech, whose slides I recommend when they will become available.
If you attended the conference, please consider giving feedback: who knows, you could win a free entrance for the 2011 conference!
Please note finally that my talk slides will be shortly published on the conference website, and that this article will be updated accordingly.

Another common (albeit implicit) point has been the focus on “high-level” solutions, that is: connecting several database servers at a higher level to emulate a single server with larger resources.
An obvious advantage is that you don’t need to alter the well-scrutinised PostgreSQL code; a drawback is that using multiple database servers with their independent timelines you are losing some useful properties. Two examples: the partial loss of transactional semantics generates conflicts; pre-parsing each query outside the database introduces limitations on the accepted queries.
The discussion was quite interesting, and when Dimitri Fontaine mentioned remote tablespaces I started wondering around a related but distinct idea, namely: whether a lower-level approach to the problem of resource pooling would really be impractical. Before I could elaborate on the details the workshop ended, and I could only sketch the idea to some of the people that were around the whiteboard (among which Gabriele Bartolini, Nic Ferrier, Marko Kreen, Hannu Krosing, Greg Smith) together with the basic questions “does it look feasible?” and “does that resemble something you already know?”.
A brief sketch: an application stack can be represented in this way

(application) --> (connection) --> (db server) --> (resources)

where the resources used by the database include storage, RAM and CPUs. The purpose is to allow the application to command more resources in order to increase capacity and speed. “Clever” applications that manage several databases can be represented as

(application) --> (connection) --> (db server) --> (resources)
|
+---------> (connection) --> (db server) --> (resources)

while “connection pooling” solutions can be represented as

(application) --> (connection) --> (db server) --> (resources)
|
+---------> (db server) --> (resources)

by “lower-level” solutions I mean something like

(application) --> (connection) --> (db server) --> (resources)
|
+---------> (resources)

which might resemble something familiar, but it is not what I am proposing here. To explain the difference I can increase the detail and write

(resources) = (virtual resources) --> (physical resources)

to represent the fact that at the lowest level you can have a non-trivial mapping between physical objects and virtual ones. For instance, SAN storage or RAID striping can provide larger virtual disks by joining together smaller physical disks. Such cases could be pictured as

(application) --> (connection) --> (db server) --> (virt.res.) --> (ph.res.)
|
+--------> (ph.res.)

My proposal is to pool resources at the database server level, so that we can have a more efficient “virtualisation” by using the knowledge of the specific use cases for each resource (CPU, RAM, disk), and at the same time we can avoid may of the difficulties of the transactional paradigm. The picture would be:

(application) --> (connection) --> (db server) --> (virt.res.) --> (ph.res.)
|
+--------> (virt.res.) --> (ph.res.)

The advantage is that we don’t need to manage all the possible use cases for each virtual resource; we just have to manage (and optimise for) the use cases that are actually needed by PostgreSQL. For instance: WAL should still be written in local “unvirtualised” storage, the bgwriter will access local and remote resources (RAM and disk), etc.
Some final words about reliability. To operate properly the whole system needs each subsystem; partial failures are not managed, because this architecture is not redundant. It is a distributed system, but not shared. If this architecture could provide cheap and simple scalability via a virtual database server which is functionally equivalent to a physical server with larger resources, then high availability could be obtained in the standard way by setting up two identical virtual servers in a Hot Standby configuration.
Network quality has a large impact on the overall performance; this design might be useful only if you have an array of machines in the same LAN, not only for speed reasons but also because a network failure would actually be a system failure. Even with these restrictions, my opinion is that having this option would be quite useful.
This is still a sketch, to be used as a reference for further discussion. Next possible steps:

• to make a detailed list of the resource use cases
• to decide which technologies can help best in each use case
• to estimate the actual performance/development costs