2ndQuadrant, Professional PostgreSQL - Gabriele's PlanetPostgreSQL

PGDay.IT 2011 was "bellissimo"!

2011-11-28T12:26:39Z

The fifth edition of the Italian PGDay went well beyond our initial expectations. We had about 75 participants, a total of 95 people including staff and speakers.
As I said during the event, rather than PGDay Italy, this should be named PGDay for Italian speakers given the presence of staff from Switzerland (Canton Ticino). Participants came from 12 regions: all regions but Val d'Aosta in the north/centre area, but also from Southern Italy (Naples and Calabria).
]]> Even the quality of the talks was in my humble opinion very high.
The audience paid a tribute to Magnus Hagander for his nomination in the core team, then Magnus wonderfully covered the new features of PostgreSQL 9.1.

Simon Riggs talks were on the future of PostgreSQL and on NoSQL databases. Very interesting. Andreas Scherbaum covered data warehouse topics.
Other interesting talks were:

the experience on open source and Postgres for CSI Piemonte, one of the main organisations working for local governments
ORM and Perl by Ferruccio Zamuner
Node.js and Postgres by Lucio Granzi
Serialisable snapshot isolation, covered by Marco Nenciarini
repmgr by Carlo Ascani
foreign tables and data wrappers by Giulio Calacoci

Unfortunately I missed Gianni's talk on debugging with CTEs as I was giving my speech on the project we (as 2ndQuadrant Italia) have been working in the last months: BaRMan, backup and recovery manager for PostgreSQL. The feedback I received was excellent (especially by certified Oracle engineers). We hope we can release it by the end of 2011 (depending on sponsorships) or, at the latest, early 2012.

In any case, even though organising this kind of events as community is not an easy task, the success of this edition is a source of motivation for all of us. We hope we can start much earlier with the organisation and we hope we can find more sponsors/partners for next year.

I take the opportunity to thank all the speakers that took part to PGDay and all the staff members: Diego, Luca, Gianluca, Cosimo, Maurizio, Marco (Tofanari), Emanuele and the team from 2ndQuadrant Italia (Carlo, Giulio, Marco, Gianni and Simone).

Call for papers for PGDay.IT 2011 has been extended

2011-10-15T14:08:03Z

The Call for Papers for the Italian PGDay has been extended of a week. The new deadline for submitting a paper is October 23. http://2011.pgday.it/sites/default/files/PGDay2011-CFP-EN_0.pdf ).
Rooms for developers and training are available as well.

The Italian PGDay 2011 will take place in Prato, on Friday November 25th, at the Monash University Prato Centre. Exactly, where it all started.

Ci vediamo a Prato! (See you in Prato!)
]]>

Early bird registrations open for PGDay.IT 2011

2011-09-19T14:41:24Z

The Italian PGDay 2011 will take place in Prato, on Friday November 25th, at the Monash University Prato Centre. Exactly, where it all started.
The event, organised by the Italian PostgreSQL Users Group, will be a great chance for both Italian and European members of the PostgreSQL community to gather together and to promote PostgreSQL.

]]>
Even though the schedule is not out, early bird registrations are now open. With less than 50 euro you can buy an early bird ticket to the event, which will grant you access to the conference and - as usual - a lot of food and drinks throughout the day. If you intend to participate, please contact the organisers at pgday2011@itpug.org or fill out the registration form (in Italian)

I remind you that the Call for Papers is still open and that you can submit a paper in English here: http://2011.pgday.it/sites/default/files/PGDay2011-CFP-EN_0.pdf

Ci vediamo a Prato! (See you in Prato!)

Australia: Last seats for our PostgreSQL courses in Melbourne

2010-11-19T06:17:43Z

2ndQuadrant will be delivering two courses on PostgreSQL in Australia. The location will be the prestigious Rialto Towers, one of the greatest attractions in Melbourne, right in the heart of the Central Business District, and one of the tallest office buildings in the world.

]]>

Gabriele Bartolini will be teaching the most requested “PostgreSQL Database Administration” course on November 29 and 30, as a two-day workshop with practical exercises.

On December 1 and 2, Gabriele will be teaching “PostgreSQL Datawarehousing and partitioning” course. During the two-day workshop, students will learn the main concepts behind the data warehousing discipline as well as the main techniques and features that can turn PostgreSQL into a very efficient and cost-effective solution for data warehouse.

A few seats are still available for both courses and registration is available online.

During this year’s “tour” in Australia, 2ndQuadrant will be sponsoring the 2010 edition of the Open Source Developers’ Conference (OSDC, the main conference for open-source developers in Australia) which will be held in Melbourne from November 23 to November 25. On this occasion, copies of our recently released books from Simon Riggs, Hannu Krosing and Greg Smith will be given to some lucky participants of the conference.

How to install multiple PostgreSQL servers on RedHat Linux

2010-05-10T13:00:05Z

If you have a Linux server of the RedHat family (inclusing CentOS and Fedora), you might envy the way Debian/Ubuntu distributions handle PostgreSQL clusters management.

Although it is not easy to install different PostgreSQL versions on the same RedHat Linux server using RPMs, it is much simpler to install several instances of PostgreSQL (servers) and, at the same time, take advantage of the services infrastructure.

]]> Once you have setup the RPM installation, by following the instructions that you find at the PostgreSQL YUM Repository, you will notice that the process will create two files among the others:

/etc/init.d/postgresql: init script for the PostgreSQL server
/etc/sysconfig/pgsql/postgresql: system configuration for the postgresql service

By default, PostgreSQL data directory (PGDATA) points to the /var/lib/pgsql/data directory. It is possible to change it by modifying the /etc/sysconfig/pgsql/postgresql file.

Let's suppose we want to install two PostgreSQL servers on the same RedHat Linux, by adding a second server to the default one which will be used for development purposes. We will call this postgresql-devel. It will be installed in the /var/lib/pgsql/data-devel directory and will run on the 5433 port.

We create a symbolic link to the main postgresql init script, and call it postgresql-devel:


cd /etc/init.d/
ln -s postgresql postgresql-devel

Then we start filling the postgresql-devel configuration file in the /etc/sysconfig/pgsql directory. It is important to note that the init script and the system configuration file have the same name.


cat < /etc/sysconfig/pgsql/postgresql-devel
PGDATA=/var/lib/pgsql/data-devel
PGPORT=5433
PGLOG=/var/lib/pgsql/pgstartup.\${PGPORT}.log
EOF

Once this is done, you can initialise the data directory by running: /etc/init.d/postgresql-devel initdb or simply service postgresql-devel initdb.

Similarly you can control the startup and the shutdown of the service, by running - respectively:

service postgresql-devel start
service postgresql-devel stop

You can add/remove the script from the startup and the shutdown of the system by using chkconfig the same way you would with other services.

The PostgreSQL wiki contains a detailed page about this topic, and I suggest that you read it along with this one. However, this simple article shows you how to easiliy integrate multiple PostgreSQL instances on the same Linux server, and manage them using the standard RedHat services infrastructure (thanks to the great job done by Devrim Gunduz).

Installing Greenplum Single Node Edition on Amazon's EC2

2010-03-23T17:00:02Z

I have been thinking for a while now about adding Greenplum support to an open-source application for web analytics that I wrote a few years ago, which is called ht://Miner and uses PostgreSQL.

In order to do this, I need a multi-CPU environment. While still waiting to get our new servers installed here in our data centre in Italy, I decided to look at Amazon's Elastic Compute Cloud (EC2) infrastructure. My intention is to do some benchmarking and spot the main differences in terms of performances between Greenplum Single Node Edition and PostgreSQL 8.4, my favourite DBMS.

If you wish to follow this article, you need to have an Amazon AWS account with a valid credit card. Do not worry, this test will only cost you a couple of dollars!

]]> Greenplum SNE is a free version of the Greenplum database, one of the most advanced solutions for data warehousing and analytics, which is based on a shared nothing architecture and allows for data distribution and parallel processing on several nodes (servers).

The Single Node edition of Greenplum is a freely distributed version of Greenplum which can be installed on a single node. On a multi-processor architecture, Greenplum Single Node Edition allows to create multiple segments (usually one per core) and hence to take advantage of parallel processing. Greenplum Single Node Edition can be downloaded for free from the main website.

My intention is to install it on a Large Instance running CentOS Linux 5.4 on Amazon. EC2's large instance has the following characteristics:

7.5 GB of memory
4 EC2 Compute Units (2 virtual cores with 2 EC2 Compute Units each)
850 GB of local instance storage
64-bit platform

I also decided to get a 10GB volume of Elastic Block Store (1 dollar a month), which I will format using the XFS file system. This volume will contain Greenplum data directories (this time I will try with just one single volume - next time I will try with a volume per segment).

The first step is to log into your Amazon AWS management console. Get your 10GB EBS volume and then launch a large instance using the ami-ebe4cf9f AMI file (AMI stands for Amazon Machine Image), a CentOS 5.4 image file distributed by RightScale for a 64 bit architecture. You may have a different code, as I use a Europe based server.

I then attach the created volume to the instance I just started. The management console informs me that the volume has been attached on /dev/sdf. I grab the public DNS information and connect to the server via ssh as root, using my EC2 identity.

I install the YUM packages for XFS support, by running:

yum install kmod-xfs.x86_64 xfsprogs xfsdump

I create a primary partition on /dev/sdf using fdisk and format it:

mkfs -t xfs /dev/sdf1

I then add the entry to /etc/fstab:

/dev/sdf1 /greenplum xfs noatime 0 0

and mount the partition on the /greenplum mount point:

mkdir /greenplum
mount /greenplum

Download Greenplum's Quickstart guide from the download area. Grab the URL of the 64bit RedHat installation of Greenplum and download it from the EC2 server using wget (or upload it from your computer using scp).

Follow the instructions on the quickstart guide about preparing your system to Greenplum (in particular kernel settings and limits).

Unzip the Greenplum's zip file and execute the .bin file. Answer yes to all the questions and Greenplum at the end of the process is installed in the /usr/local/greenplum-db directory.

Create the gpadmin user and set the password:

useradd gpadmin
passwd gpadmin

Prepare the data directories for the master and the segments:

mkdir -p /greenplum/master
mkdir -p /greenplum/segment1
mkdir -p /greenplum/segment2
chown -R gpadmin:gpadmin /greenplum

Become gpadmin using the su command and include source /usr/local/greenplum-db/greenplum_path.sh into gpadmin's ~/.bashrc file. Load these settings. Edit the ~/single_host_file file, add localhost to its contents and launch:

gpssh-exkeys -f ~/single_host_file

Create the ~/gp_init_config file with the following content:

ARRAY_NAME="Greenplum" 
MACHINE_LIST_FILE=/home/gpadmin/single_host_file
SEG_PREFIX=gp
PORT_BASE=50000
declare -a DATA_DIRECTORY=(/greenplum/segment1 /greenplum/segment2)
MASTER_HOSTNAME=localhost
MASTER_DIRECTORY=/greenplum/master
MASTER_PORT=5432
ENCODING=UNICODE

Finally launch:

gpinitsystem -c ~/gp_init_config

At the end of the process, Greenplum SNE edition is installed on your Amazon's EC2 server running CentOS 5.4. On this server you can test the solution at quite a reasonable price (I was on the server for 7 hours today and I spent only 3 dollars).

I will post a few more articles on this topic in the next few days, and hopefully I will be able to post the first benchmarks too. Enjoy!