Zero Bloat Postgres Queue | Scaling Postgres 416
Get the full intelligence
Search transcripts, export clips, track mentions, and explore all topics from “Zero Bloat Postgres Queue | Scaling Postgres 416” inside PodZeus.
PostgreSQL's traditional job queues, reliant on `FOR UPDATE SKIP LOCKED`, often spiral into performance decay due to dead tuples and index bloat under load—what the host calls the 'Q Spiral of Death.' But a new tool called PGQ, built by Nick from Postgres FM, offers a radical alternative: a zero-bloat queue that avoids updates and deletes entirely. Instead of modifying rows, PGQ uses snapshot-based batching and a rotating table system with three tables cycling through truncate operations. This design eliminates dead tuples, maintains consistent performance even at scale, and works on managed cloud Postgres. The tradeoff? Latency: jobs are processed at 50–150ms intervals by default (10 ticks per second), making it unsuitable for sub-millisecond needs but ideal for stable, high-throughput systems. The episode also covers broader ecosystem updates, including PG Backrest’s revival through community funding, PGX Backup as a potential fork, Postgres 19’s expanded multi-exact member support, and Figma’s new connection pooler PG-Keeper built to overcome PG Bouncer’s limitations. Finally, it shares practical disaster response rules and insights into how PostgreSQL committers are selected.
Use PGQ’s snapshot-based batching and truncate rotation to eliminate dead tuples and prevent performance decay in high-load queues.
PGQ trades sub-millisecond dispatch for stability—jobs are processed every 50–150ms by default due to 10 ticks per second.
Avoid the 'Q Spiral of Death' by replacing `FOR UPDATE SKIP LOCKED` with a design that never updates or deletes queue rows.
PGQ works on managed Postgres clouds and requires only a periodic ticker (e.g., via PG-Cron) to function.
Figma built PG-Keeper to overcome PG Bouncer’s single-threaded limitations and lack of observability and traffic prioritization.
…and 3 more takeaways available in PodZeus
The Problem with Traditional Postgres Queues
The episode opens by highlighting the performance decay caused by `FOR UPDATE SKIP LOCKED` in high-load scenarios, leading to the 'Q Spiral of Death' due to dead tuples and index bloat.
Introducing PGQ: Zero Bloat Queue Architecture
“PGQ essentially has no dead tuples because it never updates or deletes.”
Ecosystem Updates: PG Backrest, PGX, and PG-Keeper
“I understand that is a nuisance. They also couldn't prioritize any particular traffic.”
“PGQ essentially has no dead tuples because it never updates or deletes.”
“Jobs are going to have a 50 to 150 millisecond latency.”
“The only changes that happen at this table are the inserts of the new jobs coming in.”
Host
PGQ
product
Nick
person
pg-backrest
product
Figma
organization
PG-Keeper
product
David
person
PGX Backup
product
Postgres FM
organization
CyberTech
organization
Kristoff
person
Get the full intelligence
Search transcripts, export clips, track mentions, and explore all topics from “Zero Bloat Postgres Queue | Scaling Postgres 416” inside PodZeus.
Start discovering podcast insights today
Start with a 7-day trial and explore a growing catalog of popular podcasts. No credit card required.
No credit card required • 7-day trial • Cancel anytime
