Comprehensive software reviews to make better IT decisions
Review: NVIDIA GTC 2025 Keynote
NVIDIA’s GTC Conference kicked off on March 17, 2025, and this year’s keynote unveiled NVIDIA’s upcoming products and major collaboration initiatives. A large part of the keynote focused on how AI infrastructure would need to improve to keep up with future demands.
NVIDIA CEO Jensen Huang’s keynote focused on compute requirements of AI and highlighted that the demand for AI compute power continues to grow at a hyper-accelerated rate. With the introduction of agentic AI, Hensen said the demand is a hundred times greater today than it was last year.
To demonstrate the scale of token usage by different AI models, Hensen contrasted new reasoning models (such as OpenAI o1 or o3-mini) against traditional LLMs. Traditional LLMs learn by studying large data sets; any content produced would be derived only from those data sets. Reasoning models go a step further by employing a multistep approach that uses chain-of-thought technologies, contextual understanding, and inference. With these reasoning models, prior information and consideration of other approaches/interpretations are reviewed at each step; significantly, with each step, more tokens are produced as more data is processed. Unlike traditional LLMs that review hundreds of tokens when generating content, reasoning models require tens of thousands of tokens for each step of its process. As a result, tens of trillions of tokens will be needed to train these models.
As demand for AI continues to accelerate, the volume of tokens that must be produced and processed will increase exponentially. In response to the surging demand for advanced AI infrastructure, NVIDIA has unveiled a new offering: the AI factory. The AI factory is designed to revolutionize the deployment and development of AI, offering data processing power to support large-scale reasoning models. By harnessing NVIDIA's technologies, these AI factories will drive innovation and enable organizations to capitalize on the growing potential of AI.
Our Take
NVIDIA’s announcements reveal an aggressive growth plan to address the continued demand for AI infrastructure, targeting both cloud vendors and organizations using traditional on-premises servers. AI workloads will continue to grow thanks to agentic AI and the emerging physical AI market (i.e. robotics). NVIDIA continues to be the market leader for AI infrastructure, and competing companies will face significant challenges in offering solutions that will be as comprehensive as NVIDIA’s at scale.