Scaling PostgreSQL to power 800 million ChatGPT users

Scaling PostgreSQL to power 800 million ChatGPT users

OpenAI has successfully scaled PostgreSQL to support 800 million ChatGPT users, handling millions of queries per second with a single primary Azure PostgreSQL flexible server instance and nearly 50 read replicas. This achievement demonstrates PostgreSQL’s capacity to handle larger read-heavy workloads than previously assumed.

Why This Matters

Many architectural discussions assume sharding is essential at extreme scale, but OpenAI’s experience shows that a well-optimized, single-primary PostgreSQL instance can handle massive read traffic. The alternative – prematurely sharding – introduces significant complexity and operational overhead, and failures in distributed systems can be costly, potentially leading to service outages and user impact.

Key Insights

• 10x Load Growth: PostgreSQL load increased by over 10x in the past year at OpenAI.
• MVCC Challenges: PostgreSQL’s Multiversion Concurrency Control (MVCC) can lead to write amplification and performance issues under heavy write loads.
• PgBouncer Adoption: OpenAI deployed PgBouncer to pool database connections, reducing connection time from 50ms to 5ms.

Working Example

(No code provided in the context)

Practical Applications

• High-Traffic Applications: OpenAI uses PostgreSQL as a core data system for ChatGPT and its API, demonstrating its suitability for applications with massive user bases.
• Pitfall: Premature sharding can add unnecessary complexity; optimize existing infrastructure before resorting to distributed systems.

References:

• https://openai.com/index/scaling-postgresql/