GenAI hardware

By Falk Meissner, Thomas Kirschstein and Alexander Fey

As generative artificial intelligence – GenAI, for short – takes off, it will trigger a major expansion of the global market for the hardware and semiconductors the new technology relies on. We forecast that the market for graphics processing units (GPUs) for GenAI datacenters could grow from its current level of USD 100 billion to at least USD 3 trillion by 2040. Expansion on this scale will not be entirely free of problems, such as the massive energy requirements of the new GenAI datacenters and the extremely high cost of ownership for end users. But it will also bring opportunities for many in the GenAI space, from semiconductor equipment suppliers to the providers of AI solutions. A new study by Roland Berger examines the current state of the market, its predicted growth, the limitations on that growth and the opportunities for investment and innovation.

Sizing the market

To estimate the future size of the market for the hardware and semiconductors required by GenAI, we investigate two scenarios. Our base scenario draws on a financial market model and is calculated by looking at the implied GenAI hardware market and resulting semiconductor fab capacity needed, based the valuations of GenAI hardware players: It puts the size of the GenAI hardware market in 2040 at around USD 3 trillion. By contrast, our optimistic scenario looks at the hardware and semiconductor capacity needed to meet optimistic revenues for GenAI service scenarios, taking into account overall demand expectations for different AI use cases. In this scenario the market is worth USD 10 trillion in 2040. For the semiconductor capacity and equipment market, that translates into additional cumulative demand of between USD 300 billion and USD 1 trillion, depending on the scenario.

Hurdles to growth

While this is great news for the industry, two potential hurdles to growth exist. The first is the increase in energy demand that will accompany expansion of GenAI technology on this scale. The annual energy consumption of GenAI hardware in datacenters could reach up to 4,800 TWh by 2040 in the lower of our two scenarios – more than the current electricity consumption of the entire United States. This will have an enormous impact on the environment unless it is possible to rapidly expand large-scale renewable energy generation.

The second hurdle relates to cost of ownership. Prices for GenAI cloud infrastructure are driven by "margin stacking" across the value chain, and GenAI hardware companies currently enjoy much larger profits than are common for traditional computing hardware. Thus, one AI accelerator card can cost up to USD 40,000.

Recommendations for players in the GenAI space

Despite the challenges, the GenAI hardware market offers many attractive opportunities for investment and innovation. Semiconductor equipment and material suppliers stand to benefit from additional growth in demand for semiconductor capacity and equipment. New chip design companies should aim to customize their chips for different AI applications and workloads. Another strategy for these players would be to focus on increasing energy efficiency. Contract manufacturers and system integrators will benefit from growing demand for GenAI hardware and systems.

We recommend that they build strong relationships with chip design companies (while trying to avoid downstream integration by them). For their part, AI infrastructure, service and solution providers can enhance their AI offerings with GenAI, while helping make the industry more circular by implementing energy-efficient practices and optimizing their data and computing needs. While industrial and technology companies should consider moving into GenAI semiconductor design and hardware, especially if they themselves have high computing demands.

Register now to access the full publication and learn more about the development of the market for GPUs.