Internet of Things tends to generate large volumes of data at a great velocity. Often times this data is collected from geographically distributed sources and aggregated at a central location for data scientists to perform their magic i.e. find patterns, trends and make predictions.
Let’s explore what the IoT Solution using Postgres-BDR has to offer for Data Analytics. Postgres-BDR is offered as an extension on top of PostgreSQL 10 and above. It is not a fork. Therefore, we get the power of complete set of analytic functions that PostgreSQL has to offer. In particular, I am going to play around with PostgreSQL's Window Functions here to analyze a sample of time series data from temperature sensors.
Let's take an example of IoT temperature sensor time series data spread over a (more…)
The announcement of Postgres-BDR 3.0 last month at PostgresConf US had been long-awaited in the PostgreSQL community. Complex use cases from customers have driven the development of BDR far beyond the original feature set, resulting in a more robust technology than ever imagined - so we're happy to say that it was worth the wait.
Postgres-BDR (Bi-Directional Replication) enhances PostgreSQL with advanced multi-master replication technology that can be used to implement very high availability applications.
For an introduction to Postgres-BDR covering an overview of its complex architecture and its common use cases - 2ndQuadrant held the "Introduction to Postgres-BDR" webinar as part of its PostgreSQL webinar series.
The webinar was presented by Simon Riggs, Founder and CEO of (more…)
With the addition of logical replication in Postgres 10, it's natural to ask "what's next"? Though not directly supported yet, would it be possible to subscribe two Postgres 10 nodes to each other? What kind of future would that be, and what kind of scenarios would be ideal for such an arrangement?
As it turns out, we already have a kind of answer thanks to the latency inherent to the speed of light: locality. If we can provide a local database node for every physical app location, we also reduce latency by multiple orders of magnitude.
Let's explore the niche BDR was designed to fill.
What is Postgres-BDR?
Postgres-BDR is simply short for Postgres Bi-Directional Replication. Believe it or not, that's all it needs to be. The implications of the name itself are numerous once fully (more…)
The world of PostgreSQL continues to grow stronger by the year!
Last week we saw it all come together at PostgresConf US 2018 in Jersey City. 2ndQuadrant was proud to participate again this year as a platinum sponsor. The conference was held to promote awareness and usage of PostgreSQL through tutorials and case-studies, as well as providing the opportunity to listen first hand to some of the best minds in the community.
The 5-day event allocated two days to hands-on trainings on various PostgreSQL development and management topics. The remaining days were filled with over 80 breakout sessions covering everything related to PostgreSQL you could imagine.
At 2ndQuadrant, we take pride in supporting the continued development of the world’s most advanced open source database. (more…)
Unless you were in Sydney for linux.conf.au 2018 last week you probably didn't see my talk Geographically distributed multi-master replication with PostgreSQL and BDR. Luckily for you it's on archive.org and YouTube.
If you're interested in multi-master replication of any sort, even non-PostgreSQL-based multi-master, it should be worth taking a look. The talk could've been better titled "Understand Multi-Master and if it's right for you" or "Physics is Mean".
Comments and questions are welcome - @craig2ndq.
You can also just read the slides (pdf), but they're really intended to support an explanation, not to stand alone.
Huge thanks to Next Day Video for the amazing work they did on the recordings, to the conference conveners, the organizing team and volunteers.
Postgres-BDR (or just BDR, for short) is an open source project from 2ndQuadrant that provides multi-master features for PostgreSQL.
Here we will show how to build a test environment to play with BDR and how to configure it using the OmniDB 2.1 web interface.
2. Building test environment
Let's build a 2-node test environment to illustrate how to configure BDR within OmniDB.
2.1. Pull OmniDB repo
The first thing you need to do is to download OmniDB repo from GitHub and make sure you are in the development branch. Run the following:
git clone https://github.com/OmniDB/OmniDB
git checkout dev
2.2. Create 2 virtual machines with BDR
On your host machine, you need to have installed:
Vagrant plugin (more…)
Postgres-BDR is an open source project from 2ndQuadrant that provides multi-master features for PostgreSQL. We have pursued a joint strategy of providing both working code available now and also submitting the features into core PostgreSQL.
Postgres-BDR 1.0 runs on a variant distro of PG9.4. This is in Production now and receives regular maintenance and security updates. 2ndQuadrant will support this until 9.4 End of Life in December 2019.
One of the greatest achievements to come out of our work on BDR is the logical replication technology. Our engineers spent a considerable amount of energy to contribute the tech to PostgreSQL core and I feel especially proud that this is a headline feature of the upcoming PG10 release.
And Now BDR 2.0 …
BDR 2.0 runs on community PG9.6 as (more…)
The BDR and pglogical apt repository GnuPG signing keys have been renewed.
Users should re-import the existing keys. You can verify that it's still the same key as referenced in the documentation, just with a later expiry date.
wget --quiet -O - http://packages.2ndquadrant.com/bdr/apt/AA7A6805.asc | sudo apt-key add -
sudo apt-key finger AA7A6805 | grep -A2 -B3 BDR
Now check the fingerprint printed by the second command to verify it's the same as this output:
pub 2048R/AA7A6805 2015-03-24 [expires: 2019-03-23]
Key fingerprint = 855A F5C7 B897 6564 17FA 73D6 5D94 1908 AA7A 6805
uid BDR Apt Signing Key for 2ndQuadrant <[email protected]>
sub 2048R/739C93DD 2015-03-24 [expires: 2019-03-23]
and if it is, run:
sudo apt- (more…)
BDR is both a patch to PostgreSQL core and an extension on top of PostgreSQL core. How did that come about, and what's it's future?
Development of BDR was initiated around the time PostgreSQL 9.2 was in development. Arguably earlier if you count things like the extension mechanism. The goal of BDR is, and has always been, to add necessary features to core PostgreSQL to perform asynchronous loosely-coupled multi-master logical replication.
BDR improvements to core PostgreSQL
Since it's such a large set of changes it was necessary to structure development as a series of discrete features. A natural dividing line was "things that require changes to the core PostgreSQL code" vs "things that can be done in an extension". So the code was structured accordingly, making BDR a set of patches (more…)
I'm pleased to say that Postgres-BDR is on its way to PostgreSQL 9.6, and even better, it works without a patched PostgreSQL.
BDR has always been an extension, but on 9.4 it required a heavily patched PostgreSQL, one that isn't fully on-disk-format compatible with stock community PostgreSQL 9.4. The goal all along has been to allow it to run as an extension on an unmodified PostgreSQL ... and now we're there.
The years of effort we at 2ndQuadrant have put into getting the series of patches from BDR into PostgreSQL core have paid off. As of PostgreSQL 9.6, the only major patch that Postgres-BDR on 9.4 has that PostgreSQL core doesn't, is the sequence access method patch that powers global sequences.
This means that Postgres-BDR on 9.6 will not support global sequences, at least not (more…)