Friday, November 16

Barman is finally out

Barman, backup and recovery manager for PostgreSQLIt took longer than expected, but we have finally managed to release Barman as open-source under GNU GPL 3.
Barman stands for “Backup and Recovery Manager” and it is an administration tool for disaster recovery of PostgreSQL servers. Currently only Linux systems are officially supported, however Python allows to plan porting and maintenance on different platforms as well.

The idea of starting Barman began about a year ago. We were tired of managing disaster recovery solutions using custom scripts, hard to maintain and to monitor (you probably know what I mean).

Also, we needed a tool specifically designed for disaster recovery – and not high availability as well. Some of the features we were eager to see:

  • remote backup
  • management of multiple servers
  • catalogue of backups
  • management of retention policies
  • incremental backup
  • compression of WAL files
  • etc.

None of the tools available as open source was able to offer all of these features at once.

Remote backup allows database architects to organise a business continuity and/or high scalability cluster (made up of several Postgres servers) so that physical backup of the database server is not invasive.

What we can do now with Barman is to simply setup the archive_command on the master. The rest is performed from a remote server. A simple/typical scenario is the one where one backup server sits in the middle of your Postgres cluster and manages all the backup activity. In case, you can manage the recovery process from that same console – even though you always hope that recovery operations are limited to regular simulations only (remember, it is very important to simulate recovery, usually every 6 months at least).

I already mentioned the centralised architecture of the backup, where you can have a backup server regularly receiving WAL information from all the configured Postgres servers. I said ‘all’, because with Barman you can manage backup (and recovery) of multiple Postgres servers.

Remote backup and management of multiple servers are really handy features. However, one of the features I love the most of Barman is the ability to keep a catalogue of backups per each server. The “barman list-backup” command on a given server, allows you to see all the available periodical backups in your Barman installation for that server.
The dual management of periodical backups (base backups) and WAL archive, allow Barman to assign a specific WAL file to the closest previous backup available. This way, you can actually see the size of a backup, with or without the associated stream of WAL files. (Neat!)

WAL files can be compressed using standard compression algorithms such as gzip and bzip2. Compression is handled by the backup server (not the server).

The catalogue of backups has made much simpler the setup and the implementation of backup policies and retention policies for a Postgres database server. Unfortunately, it is not yet possible to configure retention policies directly in the configuration file, like “recovery window of 30 days” or “redundancy 5” (last 5 periodical backups). But we will get there soon (sponsors are welcome).

Last but not least. Incremental backup. This feature is not that important in small size environments, I know. But it is crucial for large and very database servers (1TB and over). It will definitely be available in the next versions of Barman (again we are looking for sponsors here).

I must admit that we could have released Barman months ago. However, we decided to release it when it was stable enough, being the software highly critical (we are talking about disaster recovery).
We had the chance to install it and test it on some business critical environments in Italy, and the feedback has been great.

So … sorry if it took so long (we started talking about it at PGConf.EU 2011 and PGDay.IT 2011), but I really hope that all of this waiting has been worthwile.

Finally, I want to thank all the other members of the Italian team that worked on the project and supported the development: Marco (lead programmer), Carlo, Gianni and Giulio.

2 Comments

  • Username*

    Sorry, it’s a hard wish, but someone using a backup tool, could be using hot-standby too…

    It could be nice if barman had an aditional configuration for hot-standby, serving as a gateway for the transaction logs.

    I love Python, I love PostgreSQL, but unfortunelly I had the idea, not the time to build a patch.

    • gabriele.bartolini

      Hi Daniel,

      I am not sure I understand completely what you mean here. The only very interesting use case I can see of hot standby is with PostgreSQL 9.1+ and replay/pause functions at recovery time (for more accurate Point-In-Time-Recovery operations).

      It is definitely something we have in our mind.

      However, I would be interested in hearing more from your idea.

      Finally: even having the idea is a very very good thing (in my opinion, much more than developing that in many cases). It is important as it allows us to understand and to get users’ feedback and desidered features.

      Thank you,
      Gabriele

Leave a Reply

Your email address will not be published. Required fields are marked *