Tuesday, October 24

Tag: aggregate

Parallel Aggregate – Getting the most out of your CPUs

David's PlanetPostgreSQL
A small peek into the future of what should be arriving for PostgreSQL 9.6. Today PostgreSQL took a big step ahead in the data warehouse world and we now are able to perform aggregation in parallel using multiple worker processes! This is great news for those of you who are running large aggregate queries over 10's of millions or even billions of records, as the workload can now be divided up and shared between worker processes seamlessly. We performed some tests on a 4 CPU 64 core server with 256GB of RAM using TPC-H @ 100 GB scale on query 1. This query performs some complex aggregation on just over 600 million records and produces 4 output rows. The base time for this query without parallel aggregates (max_parallel_degree = 0) is 1375 seconds. ┬áIf we add a single worker ( (more…)