Is having many GPUs enough?…HPE: "AI Infrastructure Divided by Power and Cooling"

HPE emphasized that to enhance the competitiveness of AI (artificial intelligence) infrastructure, it is essential to design not only GPU (graphics processing unit) performance but also power supply, cooling systems, and networks together. The company explained that securing a large number of high-performance GPUs matters less than increasing their actual utilization rates.

HPE announced on the 3rd that it presented this strategy at the AI infrastructure technology symposium 'HPE Cray AI 2026: Build the Machine' held the previous day at Seoul Four Seasons Hotel. Over 510 participants, including corporate AI, HPC (high-performance computing), and data center executives as well as AI startup representatives, pre-registered for the event sponsored by NVIDIA.

Fumiki Negishi, HPE HPC & AI Asia-Pacific and Japan Managing Director and Executive Vice President, who delivered the keynote address, along with Kim Tae-yong, Head of HPE HPC & AI Business in Korea, identified power and cooling capacity, GPU utilization rates, and system availability as key evaluation criteria for AI infrastructure. They also stressed the importance of the time required from equipment delivery to actual service deployment.

Particularly, as AI servers become more powerful, the burden on power and cooling systems is increasing. While traditional enterprise server racks consumed around 10kW of power, a single rack of NVIDIA's 'GB200 NVL72' reaches 120–130kW. HPE explained that since air-cooling alone cannot handle the heat dissipation, cooling facilities must be considered from the initial data center design stage.

HPE stated that it has accumulated liquid cooling technology for approximately 50 years through Cray and SGI, holding over 300 related patents. The company can configure systems ranging from a single server to more than 200 racks using DLC (direct liquid cooling). HPE emphasized, "Cooling is no longer an auxiliary facility but part of the compute architecture."

Networks and data processing also determine GPU utilization rates. According to HPE, in a deployment of 10,000 GPUs, even a 1 percentage point drop in utilization due to network latency can result in losses amounting to millions of dollars. HPE's strategy involves leveraging its proprietary networking technology alongside NVIDIA's high-speed networks to accelerate data transfer between GPUs and reduce waiting times.