Analysts estimate that the memory semiconductor (hereinafter referred to as "memory") installed in a single AI server rack could cost up to 360 billion won. As competition intensifies for AI accelerator performance, the amount of HBM (high-bandwidth memory) and LPDDR (low-power DRAM) required increases, along with their performance, making memory a core cost factor in AI infrastructure.
According to the electronics industry on the 16th, investment bank Wells Fargo recently estimated the component cost of NVIDIA's next-generation AI server rack, "Vera Rubin VR200 NVL72," at $8.34 million (approximately 11.8 billion won). This is nearly double the previous generation's "Grace Blackwell GB300 NVL72" at $4.23 million.
The VR200 NVL72 is an AI platform that bundles 72 next-generation Rubin GPUs (graphics processing units) and 36 Vera CPUs (central processing units) into a single rack unit. In addition to GPUs and CPUs, HBM, LPDDR, storage devices, cooling systems, network equipment, and power supply units are all concentrated in one rack. NVIDIA plans to begin full-scale supply of the Vera Rubin platform in the second half of this year.
In reality, the selling price of server racks may vary depending on customer contract terms and configurations. While some industry forecasts suggest the VR200 NVL72 could sell for between $5 million and $7 million, Wells Fargo's estimated component cost is higher than that. In particular, the company estimates the cost of HBM and LPDDR installed in the VR200 NVL72 at $2.57 million (approximately 36 billion won).
Approximately 278 units of NVL72 are required to build an AI data center with a capacity of 20,000 GPUs. Applying Wells Fargo's estimate directly, the component cost alone for these server racks would reach approximately 3.28 trillion won, with 1 trillion won attributed solely to memory costs.
One of the components driving price increases is memory. The memory component cost for the VR200 NVL72 has increased by 128% compared to the previous generation's $1.13 million. Additionally, the share of memory in total product costs rises from 26.6% to 30.8%. This means that approximately one-third of the component cost for a single AI server rack is attributed to memory.
Morgan Stanley's analysis also showed a similar trend. Morgan Stanley estimated the product cost of a single VR200 NVL72 at approximately $7.8 million, with memory accounting for about $20 thousand won. Memory costs have surged by approximately 5.3 times compared to the previous generation.
The increasing share of memory is because as GPU computing performance improves, the required memory capacity and bandwidth must also increase accordingly. If only GPUs become faster while the speed at which memory supplies data lags behind, there will be limits to improving overall system performance.
Furthermore, with expanded AI investments leading to tight supply and demand for memory, prices across the board are rising. High-performance upgrades, increased installation volumes, and price hikes are occurring simultaneously for memory. This is why NVIDIA, along with major global AI data center operators, is signing long-term contracts with Samsung Electronics and SK Hynix.
In the case of HBM, each Rubin GPU will be equipped with 288GB (gigabytes) of HBM4. While the capacity remains the same as previous models, the shift from HBM3E (5th generation) to HBM4 (6th generation) has increased both performance and price. Since each NVL72 unit contains 72 GPUs, the total HBM4 capacity per rack reaches approximately 20.7TB (terabytes).
Large-capacity memory is also attached to CPUs. The Vera CPU in the Vera Rubin platform will be equipped with 1.5TB of LPDDR5X. A single NVL72 unit requires a total of 54TB of LPDDR5X, equivalent to the memory capacity found in approximately 4,600 latest smartphones (based on 12GB per device). Notably, the LPDDR capacity per rack has more than tripled compared to the previous generation.
There are also forecasts that rising memory prices and supply constraints could impact next-generation server designs. It is reported that NVIDIA is re-evaluating the HBM configuration for its upcoming "Rubin Ultra," scheduled for release next year. In addition to the originally considered 12-layer HBM4E (7th generation), options such as using 8-layer HBM4E or standard HBM4 are being discussed. Some configurations may even involve reducing the HBM capacity per GPU to 192GB.
This is believed to be an effort to simultaneously reduce cost burdens on customers purchasing AI servers and alleviate NVIDIA's HBM procurement challenges. Market research firm TrendForce forecasts that even if HBM shipments increase by 50% to 60% in 2027 compared to the previous year, supply shortages will persist. Reducing the HBM capacity required per GPU would allow more GPUs to be supplied with the same amount of HBM, thereby alleviating bottlenecks.
in INDUSTRY
