Surging demand for AI creating shortage of high-powered chips

Much of the talk about the semiconductor shortages that plagued the globe during the pandemic may have subsided, but now a new chip shortage has emerged.

As more companies scramble to develop advanced generative artificial intelligence models or integrate them into their systems amid the popularity of new tools like OpenAI’s ChatGPT, demand for the high-powered graphics processing units (GPUs) needed to deploy them is surging. 

The souped-up computer chips are critical for running the countless calculations involved in training and deploying AI algorithms, but very few companies make them.

The shortage is one that many businesses and investors saw coming as interest in AI grew.

WHAT IS ARTIFICIAL INTELLIGENCE (AI)?

Santa Clara, California-based company Nvidia is the dominant powerhouse in the AI chip space because of its GPUs used to train AI, and that recognition drove its market capitalization above $1 trillion in May 2023. Industry analysts estimate Nvidia controls somewhere between 80% to 95% of the market and is set for further growth given its hold on the industry.

Nvidia has vowed to ramp up production to meet the spike in demand from AI, but so far it is not enough to keep up in the ongoing boom.

Microsoft’s latest annual report pointed to the shortage of GPUs as a potential risk factor for investors, and even OpenAI CEO Sam Altman told Congress in testimony earlier this year that the dearth of GPUs was making it difficult for ChatGPT to handle its workload.

WHAT IS CHATGPT?

Meanwhile, deep pockets continue to enter the AI market, gobbling up chips and driving prices even higher. 

The Financial Times reported this week Saudi Arabia purchased at least 3,000 of NVIDIA’s high-powered H100 chips for $40,000 each, and last week the outlet reported four Chinese tech titans, including Alibaba and TikTok parent Bytedance, ordered some $5 billion in GPUs from the firm.

The soaring demand is creating significant concern from startups seeking to enter the AI space over fears they will not be able to access the chips they need when ready to launch.

“There’s a lot of worry from AI startups that there may not be enough GPUs available to serve inference [the process of generating answers from AI models] when they find commercial success,” CoreWeave co-founder and CTO Brian Venturo told Barron’s Tech in a recent interview.