Build enterprise generative AI apps using Llama 3 at 1,000 tokens/s on the SambaNova AI platform

This workshop provides a comprehensive guide to GitHub Copilot, positioning it as the world's most adopted AI developer tool. It covers Copilot's functionality as an AI pair programmer, its integration within Integrated Development Environments (IDEs), and best practices for maximizing its utility. The session emphasizes how Copilot synthesizes code based on context from open files, comments, and direct chat queries, aiming to streamline the software development lifecycle.

This workshop demonstrates how to integrate secure code interpretation capabilities into AI applications, similar to Anthropic's AI Artifacts feature. The process involves building a system that allows an AI model, specifically Claude's Sonnet, to generate and execute Python code within a secure sandbox environment. The workshop guides participants through setting up the necessary tools, defining code execution logic, and displaying the results, including visualizations, directly within the application's user interface.

This workshop demonstrates SambaNova's AI platform, highlighting its capability to achieve over 1,000 tokens per second inference speed with Llama 3. The platform offers a full-stack solution from chip design to software, aiming to simplify the process of fine-tuning, pre-training, and deploying AI models. It addresses enterprise needs for scalability, security, and cost-efficiency by integrating various expert models behind a single endpoint and providing orchestration and access control.

This talk explores how programmers can leverage Large Language Models (LLMs) to significantly boost their productivity, aiming for a "10x" improvement. The core thesis is that LLMs should be treated not as autonomous reasoning agents, but as powerful translation engines and world simulators. By understanding how to decompose problems into language translation steps and by creatively framing prompts, developers can unlock new levels of efficiency and innovation.

This talk introduces the Azure AI model catalog as a comprehensive platform for accelerating AI development. It highlights how the catalog provides access to a wide range of foundation models, including flagship LLMs and smaller models, alongside tools for prototyping, optimizing, and operationalizing generative AI applications. The platform emphasizes ease of model switching, enterprise-grade security, and data privacy to support production workloads.