‘ChatGPT is Microsoft Cosmos DB’s Largest User’

Microsoft CEO Satya Nadella revealed at the Microsoft AI Tour in Bengaluru that OpenAI’s ChatGPT is the largest user of Cosmos DB, Microsoft’s database service, designed to build scalable and AI-ready apps. 

“If you look at ChatGPT, they are some of the biggest users of Cosmos because [it] serves as a stateful application,” Nadella said. This underscores Cosmos DB’s role in managing extensive real-time interactions and session data required by ChatGPT, a chatbot that handles millions of simultaneous conversations.

Nadella emphasised cloud-native databases such as Cosmos DB, SQL Hyperscale, and Fabric for modern AI workloads. He said that AI doesn’t sit on its own and, instead, relies on a comprehensive compute stack to function effectively. 

Cosmos DB’s scalable architecture supports high-throughput, low-latency demands during both AI training and inference, thereby ensuring responsive and stateful interactions for applications like ChatGPT.

Linking AI’s progress to Moore’s Law, Nadella noted that scaling laws remain strong for both training and inference and highlighted the significance of test-time compute”, where databases like Cosmos DB must handle high-throughput and low-latency demands post-training. 

He pointed out that merging operational and analytical data in the cloud enables AI-driven apps to access high-volume data reliably and in real time.

Nadella attributed ChatGPT’s success to the foundational role of databases in AI and said that Cosmos DB integrates operational and analytical data to provide reliable, high-volume access in real time. 

According to the company, this focus on robust data infrastructure aligns with its broader strategy, including a $3 billion investment in Azure data centres across India, where the aim is to expand global capacity to meet rising AI demands. 

These investments reinforce Cosmos DB’s position as a backbone for next-generation AI applications.

At last year’s Microsoft Build, the company introduced Microsoft Fabric, real-time intelligence to its AI analytics platform. This update integrated SQL Server into Fabric databases and merged operational and analytical data within a single unified platform.Moreover, the data API builder for Azure Cosmos DB was released to the public. This open-source tool requires no coding to set up secure GraphQL endpoints.