NVIDIA’s Jensen Huang Says That We are in the Era of “Hyper Moore’s Law”

5 months ago 42

Last updated November 9, 2024
In AI News

The future, according to Huang, lies in “codesign,” a method where both hardware and software are developed together to ensure optimal performance.

NVIDIA’s chief Jensen Huang in a recent interview said that the era of traditional scaling might be coming to an end. He sees the emergence of what he calls “Hyper Moore’s Law” as the future.

“We’re going to be on some kind of a hyper Moore’s law curve, and I fully hope that we continue to do that,” Huang said, indicating a shift towards new paradigms in computing that extend beyond the conventional model of chip performance improvement.

Historically, Moore’s Law relied on Dennard scaling and Carver Mead’s VLSI (Very Large-Scale Integration) techniques to boost chip performance. However, Huang acknowledges that these methods have reached their limits. “The two fundamental technical pillars were Dennard scaling and Carver Mead’s VLSI scaling. Both of these techniques were rigorous. However, these techniques have really run out of steam,” he explained.

The future, according to Huang, lies in “codesign,” a method where both hardware and software are developed together to ensure optimal performance. “Unless you can modify or change the algorithm to reflect the architecture of the system or change and then change the system to reflect the architecture of the new software and go back and forth—unless you can control both sides of it, you have no hope,” he said.

This codesign approach, which integrates hardware and software, allows for innovations in precision, such as moving from floating point operations like FP64 to FP32, and even as low as FP4, according to Huang.

“The second part of it is the data centre. This is a big part of the equation. Full-stack approaches and innovative scaling strategies will reshape our data centre infrastructure,” he added.

Earlier this year, when NVIDIA released Blackwell, it was said that Moore’s Law is dead. Reflecting on the rapid advancement in computing power, Huang said that in just eight years, NVIDIA has increased computational capacity by a thousandfold, a progress that far exceeds the benchmarks set during the heyday of Moore’s Law.

However, he rued that even after this remarkable growth, the industry’s accelerating demands are far from being met. “In the past eight years, we’ve increased computation by 1000 times, and we have two more years to go. So that puts it into perspective [the fact that] the rate at which we’re advancing computing is insane. And it’s still not fast enough,” said Huang.

Siddharth Jindal

Siddharth is a media graduate who loves to explore tech through journalism and putting forward ideas worth pondering about in the era of artificial intelligence.

8th Nov 2024
Meet 30+ top CDOs, ITDMs & AI Leaders.

Upcoming Large format Conference

India's Biggest Developers Summit

February 5 – 7, 2025 | Nimhans Convention Center, Bangalore

Download the easiest way to
stay informed

Why India Needs a Quantum Mission

Vandana Nair

In April 2023, India inaugurated its ambitious National Quantum Mission (NQM), representing a critical step forward in the nation’s journey towards harnessing quantum technology. With a target budget of ₹6,000

Subscribe to The Belamy: Our Weekly Newsletter

Biggest AI stories, delivered to your inbox every week.

Rising 2025 | DE&I in Tech & AI Summit

Mar 20 and 21, 2025 | 📍 J N Tata Auditorium, Bengaluru

Data Engineering Summit 2024

May 30 and 31, 2024 | 📍 Bangalore, India

February 5 – 7, 2025 | Nimhans Convention Center, Bangalore

MachineCon GCC Summit 2024

June 28 2024 | 📍Bangalore, India

September 25-27, 2024 | 📍Bangalore, India

25 July 2025 | 583 Park Avenue, New York

Our Discord Community for AI Ecosystem, In collaboration with NVIDIA.

World's Biggest Media & Analyst firm specializing in AI

AIM publishes every day, and we believe in quality over quantity, honesty over spin. We offer a wide variety of branding and targeting options to make it easy for you to propagate your brand.

AIM Brand Solutions, a marketing division within AIM, specializes in creating diverse content such as documentaries, public artworks, podcasts, videos, articles, and more to effectively tell compelling stories.

ADaSci Corporate training program on Generative AI provides a unique opportunity to empower, retain and advance your talent

With MachineHack you can not only find qualified developers with hiring challenges but can also engage the developer community and your internal workforce by hosting hackathons.

Conduct Customized Online Assessments on our Powerful Cloud-based Platform, Secured with Best-in-class Proctoring

AIM Research produces a series of annual reports on AI & Data Science covering every aspect of the industry. Request Customised Reports & AIM Surveys for a study on topics of your interest.

Immerse yourself in AI and business conferences tailored to your role, designed to elevate your performance and empower you to accomplish your organization’s vital objectives.