Amazon Web Services creates innovative cooling system for Nvidia GPU as AI grows

Amazon Web Services (AWS), the cloud division of the largest American retailer Amazon, announced the development of a unique hardware solution for cooling next-generation Nvidia graphics processors used in artificial intelligence (AI) computing. This initiative reflects the growing demands of the market for an efficient infrastructure to support large-scale AI work and highlights Amazon's commitment to maintaining leadership in cloud computing.

Nvidia GPUs are the foundation of the current generation of generative AI. However, high computing power is accompanied by significant power consumption and, as a result, increased heat generation. Conventional cooling systems are no longer able to meet the increased requirements, which forces market players to look for new technical solutions.

Initially, AWS considered building specialized data centers with a large-scale liquid cooling system that could provide optimal conditions for the operation of powerful Nvidia GPUs. However, this approach proved impractical: not only would the construction of new centers have taken considerable time, but the existing commercial equipment did not match the scale of AWS's operations. AWS Vice President of Computing and Machine Learning Dave Brown said in a statement that traditional solutions take up too much space and require excessive water consumption, which is unacceptable for a company with millions of servers.

In response to these challenges, Amazon engineers have developed an innovative In-Row Heat Exchanger (IRHX) system-a device that can be integrated into both existing and new data centers, significantly improving cooling efficiency. This solution is different from the traditional air cooling methods that were still sufficient for previous generations of Nvidia GPUs.

New AWS compute instances called P6e are already available to customers and are optimized for high-density computing tasks typical of AI. They are based on the Nvidia GB200 NVL72 architecture, a system that includes 72 Blackwell GPUs in a single server rack. This configuration allows you to effectively train and run large artificial intelligence models.

Previously, such computing clusters were available through Microsoft and CoreWeave, but AWS, as the world's largest cloud service provider, is significantly expanding its range of capabilities and infrastructure. Using its own innovative solutions allows the company to reduce its dependence on external suppliers, optimize costs and ensure a competitive advantage.

In addition to cooling, Amazon is actively developing its own hardware platforms. The company has developed specialized chips for general and AI computing, its own storage servers and network routers. This approach allows you to not only control quality and performance, but also reduce costs, which is reflected in the financial results of AWS, which in the first quarter showed a record operating margin in recent years.

Following AWS, similar efforts are being made by Microsoft, which in 2023 introduced its own cooling solutions for its Maia AI chips — the so-called Sidekicks systems. This confirms that cloud market leaders understand the strategic importance of developing hardware technologies to support AI and high-performance computing.

The development of innovations in the field of cooling is becoming a key factor that allows large cloud providers to cope with growing workloads and create new opportunities for large-scale implementation of artificial intelligence. Investment in technologies such as In-Row Heat Exchanger underscores Amazon's strategic vision for sustainable and efficient cloud infrastructure development.

Against the background of rapid growth in the use of AI and increasing demand for computing resources, we can expect further innovations in hardware and infrastructure. AWS and other major market players continue to expand their competencies, which is becoming an important driver of global digital transformation.