Elon Musk : AGI to Surpass Human Intelligence by 2026, But Power Constraints Loom Large

Elon Musk, CEO of Tesla and founder of xAI, has made bold predictions about the development of artificial general intelligence (AGI) and the challenges facing the industry. He believes that AGI could soon surpass human intelligence by 2026, but training such technology will require a massive number of processors and, consequently, a huge amount of electricity.

According to Reuters, Musk’s company xAI is currently training the second version of its large language model called Grok and expects to complete the next phase of training by May 2024.

The second version of Grok is being trained on about 20,000 Nvidia H100 GPUs, and Musk predicts that future versions will require significantly more resources, with Grok 3 requiring about 100,000 Nvidia H100 processors for training.

According to Musk, two major obstacles are limiting the advancement of AI technology: the limited supply of advanced processors like the Nvidia H100 and the availability of electricity.

Each Nvidia H100 GPU consumes about 700 watts of power at full load, so 100,000 of these processors for AI training and high-performance computing (HPC) could consume up to 70 megawatts of power, which is a significant amount.

Considering the servers and cooling systems needed for the processors, it can be said that each data center with 100,000 H100 processors will consume about 100 megawatts of energy, which is equivalent to the energy consumption of a small city.

Despite the challenges facing AI development, advances in computing and memory architecture will enable the training of much larger language models in the coming years.

At GTC 2024, Nvidia unveiled the Blackwell architecture and the B200 GPU, which are designed to meet the processing needs of training large language models with trillions of parameters. This architecture could play a significant role in the development of AGI.

Musk believes that AI smarter than humans will emerge in the next year or two. He said in an interview, “If you define AGI as smarter than the smartest human, I think we’ll probably get there in the next one or two years.”


Additional Information:

  • AGI: Artificial General Intelligence
  • xAI: OpenAI’s competitor founded by Elon Musk
  • Grok: xAI’s large language model
  • Nvidia H100: Nvidia’s latest GPU
  • Blackwell: Nvidia’s next-generation GPU architecture
  • B200: Nvidia’s next-generation GPU based on Blackwell architecture
Back to top button