Dylan Patel

This talk examines the geopolitical landscape of AI infrastructure, focusing on the capabilities of China and the Middle East in contrast to US limitations. It highlights how China, despite sanctions, is advancing its AI chip manufacturing and deployment through innovative engineering and supply chain circumvention. Simultaneously, the Middle East is emerging as a significant hub for AI infrastructure development, attracting substantial investment and GPU resources, partly due to power constraints in the US.

This talk discusses the system design challenges and future directions for running and training next-generation large language models. It highlights the significant computational and memory bandwidth requirements for inference, particularly for models with trillions of parameters. The presentation also touches upon the massive scale of infrastructure being built for training these models and the associated engineering hurdles.