According to OpenAI’s blog post from October 23, 2026 (source), the rapid growth of ChatGPT required scaling their Habitat online storage layer to reliably serve more than a billion users. This second part in the series covers key strategies: introducing robust multi-tenancy, deploying a global cache, and migrating data to Azure Cosmos DB.
Why multi-tenancy mattered
By leveraging multi-tenancy, OpenAI could isolate user data while efficiently sharing infrastructure among the massive user base. This was essential to keep Habitat scalable, reliable, and cost-effective for more than one billion global users.
Global caching for performance
OpenAI implemented a global caching layer to reduce latency and hiccups, ensuring users get fast responses regardless of their region. This architecture meant read-heavy workloads could hit the cache instead of always querying the database directly.
Moving to Azure Cosmos DB
A big part of the speed improvement came from Habitat’s transition to Azure Cosmos DB. OpenAI reports this drove a 12× improvement in read performance, supporting much greater amounts of traffic and user data with predictable low latency worldwide.
- Multi-tenancy secures and isolates user data.
- Global cache dramatically reduces perceived latency.
- Cosmos DB migration lifted read speeds 12× for Habitat data retrieval.
For anyone building planet-scale services, this post is a practical overview of making multi-tenant distributed storage work under real-world, enormous demand. Key practices here—like isolation, edge caching, and managed distributed data stores—are worth considering for web apps expecting to scale.