In Q2 of fiscal year 2024, Nvidia disclosed datacenter hardware sales totaling $10.3 billion. Detailed analysis by market research firm Omdia reveals that during Q2 of 2023, Nvidia dispatched 900 tons of its H100 processors. Omdia's research indicates that over 900 tons (equivalent to 1.8 million pounds) of H100 compute GPUs, tailored for artificial intelligence (AI) and high-performance computing (HPC) applications, were shipped by Nvidia in this period.
These findings are based on an assumption that an average Nvidia H100 compute GPU with a heatsink weighs more than 3 kilograms (around 6.6 pounds), suggesting Nvidia's distribution of approximately 300,000 H100 GPUs during the quarter.
The H100 product line from Nvidia comprises three unique form factors with distinct weights. An Nvidia H100 PCIe card weighs 1.2 kilograms, whereas the weight of the Nvidia H100 SXM module hasn't been disclosed. A comparative OAM module, similar in size and thermal design power (TDP) to the H100 SXM, weighs 2 kilograms. If we assume an 80% shipment share for modules and 20% for cards in Nvidia's H100 distribution, the average weight for an H100 device would be around 1.84 kilograms. Based on Omdia's estimates of H100 units shipped in Q2, the actual total weight might be marginally below 900 tons. However, it's clear that Nvidia dispatched a significant number of H100 GPUs in this period.
For the upcoming quarters, Omdia predicts Nvidia will maintain similar GPU shipment volumes. They project a potential total of roughly 3,600 tons (or 7.2 million pounds) of H100 GPUs shipped annually, which implies a production of about 1.2 million H100 GPUs if the trend remains consistent.
The aforementioned estimate points to over 300,000 H100 GPUs being shipped in just one quarter. It should be noted that the H100 figure might also include Nvidia's China-centric H800 processors. Moreover, Nvidia persistently provides substantial amounts of its prior-generation A100, A800, and A30 compute GPUs to enterprises that utilize them for AI inference and technical computing. Therefore, Nvidia's cumulative quarterly GPU shipments are likely higher than 300,000 units and may surpass 900 tons.
Notably, despite these substantial shipment figures, Nvidia's GPU supply has not met the global demand. AI GPUs are anticipated to be in shortage through the end of the year. This indicates a potential increase in shipment rates in the subsequent year, in line with the rising demand for generative AI.
For a clearer understanding, objects with a rough weight of 900 tons include:
Source: Toms Hardware / Omdia