Europe 2026
Under 5 minutes to a deployed LLM endpoint — Audry Hsu, RunPod
Overview
This talk introduces RunPod as a cloud AI infrastructure platform designed to simplify GPU access and model deployment for developers. The core thesis is that managing complex infrastructure, especially GPU hardware, is a significant hurdle for builders, and RunPod aims to abstract this away. The platform allows users to bring their own code and models, whether private or open-source, and deploy them quickly, focusing on enabling developers to build applications rather than manage hardware.
Who should watch
- AI Engineers
- Product Managers
- Builders looking to deploy LLMs and AI models
- Developers struggling with GPU infrastructure management
- Teams needing flexible and scalable GPU compute for inference
Key takeaways
- RunPod provides cloud infrastructure with GPUs, simplifying model deployment for developers.
- The platform addresses challenges like slow and opaque GPU access due to global supply crunches.
- RunPod offers several deployment options: Pods (sandbox environments), Serverless (auto-scaling for inference), Clusters (for heavy training), and the Hub (a repository of pre-configured AI repos).
- The Serverless product is optimized for real-time inference, featuring auto-scaling, configurable worker limits, and spending caps.
- Deploying a model from the RunPod Hub to a Serverless endpoint can take under five minutes, including model download and container initialization.
- The platform charges only for active worker time, making it cost-effective for bursty or batch workloads.
- RunPod provides telemetry for observability into endpoint performance, including request queues, execution times, and delays.
- The company originated from founders sharing spare GPU capacity on Reddit, emphasizing a community-driven approach.
Notable quotes
*RunPod, we are a cloud AI infrastructure company. So, we have the hardware, we have the GPUs, and we make it easy for developers to deploy models.*
*We want to build we as software developers, we bring bring the value through the applications that we build, not from managing the infrastructure.*
*For a lot of teams, serverless is the fastest way if you want to start deploying a production-ready API.*
Unofficial community note. Prefer the recording for nuance.