# Volatility ahead: The compute cost conundrum
The increasing popularity of artificial intelligence (AI) has propelled the demand for graphics chips, or GPUs, which power large language models (LLMs) used in various AI applications. As the prices of GPUs are expected to fluctuate significantly in the coming years, many businesses will need to learn how to manage variable costs for this critical product.
## Compute cost volatility and its impact
Industries that have no experience with managing fluctuating costs, such as financial services and pharmaceutical companies, will need to quickly adapt to the challenges posed by compute cost volatility. These industries, which stand to benefit greatly from AI, must develop the necessary skills to navigate this new terrain.
Nvidia, the main provider of GPUs, has seen its valuation soar due to the high demand for its chips. GPUs are highly sought after because of their ability to process multiple calculations simultaneously, making them ideal for training and deploying LLMs. The scarcity of GPUs has become so severe that some companies have resorted to having them delivered by armored cars.
The costs associated with GPUs are expected to continue fluctuating significantly, driven by the fundamentals of supply and demand.
## Drivers of GPU cost volatility
Demand for GPUs is predicted to surge as companies rapidly adopt AI technology. Investment firm Mizuho forecasts that the total market for GPUs could grow tenfold over the next five years, reaching over $400 billion. However, supply remains uncertain and depends on factors such as manufacturing capacity and geopolitical considerations. The manufacturing of GPUs, for example, heavily relies on Taiwan, whose independence is threatened by China.
Already, companies have experienced lengthy delays in acquiring Nvidia’s powerful H100 chips, with wait times reportedly stretching up to six months. As businesses become increasingly reliant on GPUs for AI applications, it becomes crucial for them to learn how to manage variable costs effectively.
## Strategies for GPU cost management
To mitigate the impact of fluctuating costs, some companies may choose to manage their own GPU servers instead of renting them from cloud providers. While this approach incurs additional overhead, it provides greater control and potential cost savings in the long run. Another strategy is for companies to purchase GPUs defensively, ensuring future access to these chips and gaining a competitive advantage.
Companies should also optimize costs by selecting the right type of GPUs for their specific needs. For organizations that primarily perform higher volume inference work, a greater number of lower-performance GPUs would be more suitable. Additionally, choosing the right geographic location for GPU servers can significantly reduce electricity costs, as power consumption is a significant part of the unit economics of GPUs.
CIOs should consider the trade-offs between cost and quality when deploying AI applications. By using less computing power for applications that require less accuracy or are less strategic to their business, organizations can strike a balance between cost optimization and performance.
Adopting technologies that optimize the cost of operating LLM models for different use cases can also make GPU usage more efficient. Similar to how logistics companies use different transport modes and shipping routes to manage costs, organizations can switch between different cloud service providers and AI models to optimize costs.
## The challenge of demand forecasting
Forecasting GPU demand accurately poses a significant challenge for organizations. The rapid evolution of AI computing, coupled with the emergence of new applications and use cases, makes it difficult to predict future demand. Chip makers are continually developing more efficient architectures, and changes in AI applications can drive GPU demand up or down. Accurately forecasting GPU demand will require companies to navigate uncharted territory.
## Start planning for volatile GPU costs now
The AI industry shows no signs of slowing down, with global revenue associated with AI projected to grow 19% per year through 2026, reaching $900 billion. While this growth is positive for chip makers like Nvidia, it presents a new challenge for businesses: effectively managing the costs associated with AI infrastructure. By starting to plan and implement cost management strategies now, organizations can better navigate the volatility of GPU costs and ensure long-term success in the AI revolution.
*Florian Douetteau is the CEO and co-founder of Dataiku.*
—
Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. [Learn More](https://venturebeat.com/newsletters/?utm_source=VBsite&utm_medium=desktopNav)