World's Fair 2024

This talk explores the concept of personality-driven development in AI agents, where agents are given distinct forms and personalities to enhance user interaction and understanding. The speaker argues that anthropomorphizing AI, while not new, becomes particularly relevant with modern agents, influencing user perception and expectations. This approach aims to simplify complex AI functionalities by mapping them to familiar human roles and traits, thereby improving product development and user experience.

This talk focuses on the advantages and practicalities of using open-source models for production AI applications. It highlights how custom-tuned, smaller models can outperform larger, general-purpose proprietary models in terms of speed, cost, and domain-specific accuracy. The presentation also addresses the challenges of deploying open-source models, such as setup complexity and optimization, and introduces Fireworks AI's platform designed to streamline these processes.

This talk demonstrates an agentic workflow using Claude 3 Haiku to control a Minecraft bot named Rocky. The agent takes chat inputs and uses a set of defined tools to perform actions within the game, such as moving, locating players, digging, and even building structures. The system leverages Amazon Bedrock Agents for a managed agentic workflow, allowing for orchestration of multiple tasks and providing return of control to the user.

This talk introduces Ario, a personal assistant AI designed to give users back approximately one hour per day by automating mundane tasks. The core of Ario's system is adversarial ETL, which focuses on extracting user data from various online sources. This data is then used to build a dynamic, evolving profile of the user, enabling more personalized and context-aware assistance than generic AI tools.

This talk focuses on scaling Retrieval Augmented Generation (RAG) applications for production use, specifically highlighting the capabilities of Azure AI Search. It addresses the challenges that arise when moving from prototypes to production, such as increased data volume, higher data change rates, and more complex multi-step workflows. The presentation emphasizes how Azure AI Search provides integrated solutions for these scaling dimensions, including advanced vector search, hybrid search, and efficient data ingestion.

This talk focuses on optimizing the infrastructure for training large AI models, specifically addressing the bottlenecks in distributed training caused by network communication. Crusoe Cloud, an AI cloud platform powered by renewable energy, highlights its rail-optimized InfiniBand networking solution designed to accelerate Mixture of Experts (MoE) training by reducing the time gpus spend idle waiting for data exchange.

This talk discusses the system design challenges and future directions for running and training next-generation large language models. It highlights the significant computational and memory bandwidth requirements for inference, particularly for models with trillions of parameters. The presentation also touches upon the massive scale of infrastructure being built for training these models and the associated engineering hurdles.

This talk explores the rapidly evolving landscape of AI music generation, demystifying the technology and demonstrating practical applications. It covers various categories of AI music, including voice conversion, text-to-music, and audio-to-music generation, highlighting tools like Suno and Udio. The presentation aims to equip attendees with the knowledge to transform musical ideas into finished songs and discusses the emerging opportunities for artists in this new domain.

This talk explores the evolution of voice AI, moving beyond the limitations of earlier systems to a new era of conversational agents. The core thesis is that true human-like interaction in AI is not solely about speed or accuracy, but critically depends on the contextual understanding and generation capabilities within the entire voice AI pipeline. This contextual awareness is presented as the key innovation that will make AI agents feel genuinely conversational.

This talk explores the engineering challenges and architectural considerations for building extremely fast voice bots. The core thesis is that achieving human-like conversational latency requires a deep understanding of media processing, real-time data pipelines, and strategic model deployment, often necessitating self-hosting and colocation of components to minimize network delays.

This talk introduces the latest advancements and additions to the Gemma model family, emphasizing Google DeepMind's commitment to empowering the open-source community. The Gemma models are presented as lightweight, state-of-the-art, open-source models built from the same technology as the Gemini models, designed for responsible AI development, high performance, and extensibility across various platforms and frameworks.

This talk demonstrates how to fine-tune and deploy multiple AI models directly from Python code without requiring complex infrastructure like Kubernetes or Docker. The core idea is to use a Pythonic interface to abstract away the underlying compute resources, allowing users to specify hardware requirements and execution parameters with simple decorators. This approach simplifies the process of running computationally intensive tasks, such as model training and inference, on various backends, including local machines, on-premise systems, or cloud-based GPU clusters.

This talk argues that most AI engineers and builders likely do not need fine-tuning for their current projects. The core thesis is that while fine-tuning can significantly improve model performance and cost-effectiveness, it introduces upfront time investment and reduces flexibility. The presentation aims to help the audience determine if and when fine-tuning becomes a beneficial step in their development process.

This talk introduces Floyd, an AI assistant designed to make phone calls, demonstrating how to integrate various technologies to create a more capable personal assistant. The core idea is to leverage real-time speech-to-text, text-to-speech, and AI models to handle conversations with humans, managing tasks like booking appointments or ordering services. The system aims to provide a seamless experience by learning user context and proactively managing interactions.

This talk focuses on optimizing Large Language Model (LLM) quality through fine-tuning, addressing common pain points like high operational costs and the inability to meet production-ready quality standards. It positions fine-tuning within a broader "crawl, walk, run" strategy for LLM quality, emphasizing that it should be considered after prompt engineering and Retrieval Augmented Generation (RAG) have been explored. The presentation outlines a continuous deployment cycle for fine-tuned LLMs, including data collection, model fine-tuning, deployment, and evaluation, aiming to demystify the process for AI engineers.

This talk addresses the critical need for robust security measures in machine learning systems. It highlights the inherent fragility of ML models, making them susceptible to various attacks. The discussion covers data poisoning, model theft, adversarial examples, supply chain vulnerabilities, and software exploits, emphasizing proactive strategies and continuous vigilance to protect ML deployments.

This talk focuses on how GitHub is integrating AI into its Advanced Security platform to enhance developer productivity and security. The core thesis is that AI can significantly improve the identification and remediation of security vulnerabilities and secrets, making security a more integrated and less burdensome part of the daily development workflow. By embedding AI capabilities directly into the platform, GitHub aims to bridge the gap between security teams and developers, fostering a shared responsibility for application security.

This talk explores the concept of AI agents as the primary interface for human interaction with generative AI. It defines an agent as a system designed to achieve specific goals through environmental interaction, comprising a reasoning model, tools for action, and orchestration for memory and state management. The discussion emphasizes the responsibility that comes with building these agents, focusing on ethical considerations, safety, cybersecurity, and data privacy.

This talk focuses on Snorkel AI's approach to data development for enterprise AI models, emphasizing the challenges of achieving enterprise-grade quality, latency, and cost requirements with off-the-shelf large language models. Snorkel AI specializes in developing data to fine-tune these models, enabling them to meet specific business needs. The discussion also touches upon the infrastructure and best practices for running large-scale AI training and inference workloads on Azure.

This talk explores the integration of Retrieval Augmented Generation (RAG) with MongoDB's document model and Atlas Vector Search. It highlights how combining a flexible document database with vector search capabilities enables more sophisticated and context-aware AI applications. The presentation emphasizes that modern AI applications require more than just generic LLMs, necessitating the augmentation of prompts with relevant, up-to-date data.

This workshop demonstrates SambaNova's AI platform, highlighting its capability to achieve over 1,000 tokens per second inference speed with Llama 3. The platform offers a full-stack solution from chip design to software, aiming to simplify the process of fine-tuning, pre-training, and deploying AI models. It addresses enterprise needs for scalability, security, and cost-efficiency by integrating various expert models behind a single endpoint and providing orchestration and access control.

This workshop provides a comprehensive guide to GitHub Copilot, positioning it as the world's most adopted AI developer tool. It covers Copilot's functionality as an AI pair programmer, its integration within Integrated Development Environments (IDEs), and best practices for maximizing its utility. The session emphasizes how Copilot synthesizes code based on context from open files, comments, and direct chat queries, aiming to streamline the software development lifecycle.

This workshop demonstrates how to integrate secure code interpretation capabilities into AI applications, similar to Anthropic's AI Artifacts feature. The process involves building a system that allows an AI model, specifically Claude's Sonnet, to generate and execute Python code within a secure sandbox environment. The workshop guides participants through setting up the necessary tools, defining code execution logic, and displaying the results, including visualizations, directly within the application's user interface.

Substrate is presented as an API for building modular AI applications, moving away from monolithic models. The core thesis is that complex AI tasks are best accomplished by systems of multiple inference runs orchestrated logically, rather than relying on a single foundation model. This modular approach offers benefits in legibility, debuggability, extensibility, and easier evaluation due to explicit decision trees.

GitHub Copilot has evolved from an AI pair programmer generating code snippets to a comprehensive tool integrated across development workflows. It now offers chat functionalities within IDEs and on GitHub.com, enabling code explanation, debugging, test generation, and interaction with enterprise knowledge bases.

This talk demonstrates how to build, evaluate, and deploy a Retrieval-Augmented Generation (RAG) based copilot for a retail environment using Azure AI services. The core concept involves creating a chatbot that can answer customer questions by accessing product information from a vector database and customer data from a relational database, all orchestrated through Azure AI Studio's Prompt Flow.

This talk focuses on the practical challenges and solutions for building effective LLM evaluation systems in real-world applications. It distinguishes between model evals, which rank models against benchmarks, and task evals, which assess whether an LLM application is functioning correctly for its intended purpose. The core thesis is that robust task evals, especially those providing explanations for failures, are crucial for iterating and improving deployed LLM applications.

This talk introduces the Azure AI model catalog as a comprehensive platform for accelerating AI development. It highlights how the catalog provides access to a wide range of foundation models, including flagship LLMs and smaller models, alongside tools for prototyping, optimizing, and operationalizing generative AI applications. The platform emphasizes ease of model switching, enterprise-grade security, and data privacy to support production workloads.

This talk introduces AI templates as a rapid method for deploying AI applications, focusing on how Microsoft for Startups and its Founders Hub platform can support builders. The core thesis is that these templates, particularly for complex tasks like Retrieval Augmented Generation (RAG), significantly reduce the time and complexity of getting started with AI development, enabling faster prototyping and deployment.

This talk showcases advancements in Azure AI, focusing on multimodal and multi-agent capabilities. It highlights how new models and tools, such as GPT-4o and Phi-3 Vision, can process and reason across text, vision, and speech. The presentation emphasizes practical applications and the integration of these technologies within Azure AI Studio for building sophisticated AI solutions.

This talk explores the potential for AI agents to replace human tasks, moving beyond simple prompting to autonomous operation. The core thesis is that agents can handle low-leverage, reactive tasks, freeing humans to focus on high-leverage, proactive activities. The discussion outlines a path from basic prompting to more complex agentic workflows, emphasizing the eventual goal of agents operating independently within broader workflows.

This talk introduces Azure AI Studio and Prompt Flow, a suite of tools designed to streamline the development, evaluation, and deployment of AI applications. The core thesis is that while large language models are powerful, they require integration with domain-specific knowledge and tools, along with careful filtering, evaluation, and continuous monitoring to be effective in enterprise settings. These tools aim to make AI application building efficient and trustworthy.

This talk introduces Ionic's mission to enable AI agents to interact with the economy, starting with e-commerce. The core thesis is that the current digital world is built for advertisements, not for actions agents can take. Ionic aims to bridge this gap by providing agents with enriched, dynamic product data and facilitating direct transactions, thereby creating a new economic model where merchants pay for direct access to consumers matched with suitable products.

This talk addresses common accuracy issues in Retrieval Augmented Generation (RAG) applications, particularly with complex enterprise documents. The core thesis is that RAG failures are typically due to content ingestion problems rather than LLM or prompt issues. The presented solution focuses on a novel ingestion pipeline that preserves crucial context lost during traditional chunking and vectorization, leading to significantly improved accuracy.

This talk introduces BotDojo, an AI enablement company focused on helping businesses deploy AI applications to production. The demonstration highlights how to use synthetic data generation combined with evaluations to improve the performance of chatbots. It showcases a low-code editor for building AI flows, emphasizing the importance of detailed tracing for debugging and the integration of evaluations to identify issues like hallucinations and insufficient information retrieval.

This talk details Khan Academy's journey in scaling AI for educational purposes, focusing on their AI-powered tutor and teacher assistant, Khanmigo. The core thesis is that AI can democratize one-on-one tutoring and teaching assistance at scale, addressing learning gaps and transforming educational experiences. The presentation highlights how Khan Academy shifted to an AI-first organization by rapidly prototyping, iterating, and integrating AI deeply into their platform and content, while also addressing key technical challenges and future directions.

Cohere is an enterprise AI company focused on building trustworthy AI models for real-world business use cases. The company emphasizes practical application, efficiency, and scalability over simply having the largest models. Key offerings include generative models like Command R and R Plus, and advanced retrieval models such as embeddings and rerankers. Cohere prioritizes enterprise-specific performance, customization, data privacy, and deployment flexibility across various cloud and on-premise environments.

This talk focuses on optimizing Large Language Model (LLM) inference, emphasizing the practical challenges and cost-effectiveness of deploying these models at scale. It delves into the technical intricacies of the LLM inference workload, from tokenization and attention mechanisms to the crucial role of the KV cache in managing GPU memory and performance. The presentation aims to provide attendees with a deeper intuition about these processes and actionable strategies for measuring and improving deployment efficiency.

This talk addresses the practical challenges of deploying secure, accurate, and compliant AI systems, particularly focusing on open-access large language models (LLMs) in enterprise environments. It outlines common risks such as hallucinations, supply chain vulnerabilities, data breaches, and prompt injection, and proposes mitigation strategies. The core thesis is that by understanding and proactively addressing these risks with existing and emerging technologies, organizations can build more trustworthy AI applications.

This talk discusses how a team at Twilio navigated the rapidly evolving landscape of AI, focusing on disruptive innovation without compromising customer trust. The core thesis is that companies must embrace emerging AI technologies, particularly agents, by adopting principles of customer obsession, shipping early and often, fostering a curious and problem-solving team culture, and sharing learnings both internally and externally. This approach allows for rapid prototyping and iteration while managing customer expectations regarding quality and reliability.

This talk addresses the critical and evolving landscape of AI evaluations, particularly as systems become more agentic and automated. It argues that current evaluation methods are often simplistic and prone to "benchmark hacking," failing to capture the true performance and values embedded in AI systems. The discussion emphasizes the need for more robust, multifaceted evaluation strategies that consider real-world scenarios, user experience, and the underlying values driving AI development to ensure these systems align with human goals.

This talk focuses on the practicalities of building and hiring for AI engineering teams, emphasizing a shift from pure ML research to production-ready AI product development. It argues that AI engineering requires a blend of software engineering, product thinking, and data intuition, and that the hiring process should reflect these needs. The core thesis is that successful AI teams are built by understanding the evolving stages of AI product development and hiring individuals with specific, complementary skill sets.

This talk focuses on Retrieval Augmented Generation (RAG) for enterprise applications, emphasizing the challenges of moving from prototype to production. It highlights the critical role of data processing and quality in achieving accurate, production-ready AI applications. The speaker, Jerry Liu, co-founder and CEO of LlamaIndex, discusses how their platform aims to empower developers to build robust LLM applications over their data, addressing issues like data silos, accuracy, and scalability.

This talk discusses the engineering principles and platform considerations necessary for building and scaling Generative AI applications within an organization. It draws parallels with the evolution of DevOps, emphasizing the need for structured platforms, enablement, and governance to manage the complexities of AI development and deployment effectively. The core thesis is that successful AI adoption requires a dedicated platform engineering approach, similar to how DevOps matured, to bridge the gap between AI capabilities and traditional software development.

This talk focuses on how enterprises can achieve real return on investment (ROI) from large language models (LLMs) and generative AI products. It emphasizes that the initial hype phase is over, and companies are now generating tangible revenue and cost savings. The core message is that success hinges on a strategic approach to team composition, robust evaluation methods, and appropriate tooling, rather than solely on complex model architectures.

This talk explores the relationship between the stakes of an AI application and the obstacles encountered during production. Higher stakes, defined by potential impacts on finances, reputation, health, or safety, correlate with increased challenges. The discussion contrasts vector-based retrieval-augmented generation (RAG) with knowledge graphs, suggesting that structured data and reasoning from knowledge graphs can enhance reliability for high-stakes use cases.

This talk addresses the challenge of measuring the return on investment (ROI) of AI tools, particularly for developers. The core thesis is that while AI can significantly enhance developer productivity, demonstrating its business impact requires moving beyond anecdotal evidence or simple engagement metrics to more structured evaluation frameworks. The speaker emphasizes that the difficulty in measuring AI ROI is akin to the NP-hard problem of measuring developer productivity itself, suggesting that a precise, universally applicable solution is unlikely, but practical frameworks can be developed.

This talk introduces Moondream, a small, open-source vision language model with under two billion parameters, designed to be efficient and runnable on various devices. Despite its size, Moondream achieves performance comparable to much larger models on vision benchmarks. The development focused on creating a tool for developers, prioritizing accuracy and avoiding hallucinations, rather than general world knowledge.

This talk focuses on optimizing Retrieval Augmented Generation (RAG) systems by employing an evaluation-driven approach. It highlights the common challenges encountered in RAG implementations, from data processing and retrieval to response generation, and proposes systematic methods for improvement. The core thesis is that continuous, data-driven evaluation is essential for effectively refining RAG performance and achieving desired outcomes.

This talk details Zapier's approach to building AI-powered products, emphasizing an iterative development process heavily reliant on robust evaluation and observability. The speakers share their journey of integrating AI features, highlighting the challenges and successes encountered when developing and refining tools like the AI Zap Builder and Zapier Copilot. Their strategy involves close collaboration between product and engineering teams, continuous testing, and leveraging platforms like Braintrust to ensure product quality and performance.

This talk addresses the challenges of deploying Generative AI applications to enterprises, focusing on the critical need for accuracy and trust. It highlights how traditional methods of analyzing customer interactions through manual review or scripted analysis are insufficient at scale. Generative AI offers a solution by enabling 100% coverage of conversations, surfacing unknown insights, and transforming the process of understanding customer needs and business operations.

This talk explores building effective Generative AI applications using Knowledge Graphs and Graph Retrieval Augmented Generation (GraphRAG). It demonstrates how to combine vector search with graph traversal and graph embeddings to create more personalized and contextually relevant AI-driven responses. The session focuses on practical implementation using Neo4j, LangChain, and OpenAI, showcasing a fashion recommendation email generation use case.

This talk introduces State Space Models (SSMs) as a promising architecture for real-time multimodal intelligence, contrasting them with traditional batch-oriented AI systems. The core thesis is that SSMs, by efficiently modeling long contexts and compressing information, can enable faster, cheaper, and more ubiquitous AI applications, particularly in areas requiring instant responses like conversational interfaces and robotics.

This talk explores the unpredictable second-order effects of AI, moving beyond immediate consequences to consider broader societal and individual impacts. It suggests that as AI automates tasks, the focus may shift from efficiency to personal learning and skill development, leading to new forms of interaction and creative expression. The discussion also touches on how AI can widen information bandwidth and personalize user experiences, potentially transforming communication and interface design.

This talk introduces the fundamental concepts of AI agents, detailing their components and use cases. It emphasizes that agents are best suited for complex, multi-step tasks requiring the integration of various capabilities, such as data aggregation, visualization, and reasoning, or when personalization and adaptive responses are necessary. The session also provides a hands-on guide to building a research agent from scratch, incorporating tools, memory, and reasoning patterns.

This talk explores the transformative potential of multimodal AI in education, addressing critical needs for improved literacy, bridging learning gaps, and upskilling. It highlights how AI can serve as a powerful tool for students and educators alike, fostering deeper understanding and engagement. The presentation emphasizes the importance of cultivating AI literacy and critical thinking from a young age.

The core thesis is that while Generative AI models are easy to demo, productionizing them presents significant challenges. The talk emphasizes that the AI development process is fundamentally experimental and non-deterministic, unlike traditional software development. This necessitates a robust approach to tracking learnings, ensuring reproducibility, and building comprehensive evaluation frameworks to move AI applications from demo to production successfully.

This talk addresses the critical importance of training data development for large language models, emphasizing that the quality and format of data directly impact model performance. It proposes a hierarchy of needs for dataset development, starting with clean data and progressing through evaluations, dataset management, and advanced techniques like synthetic data generation and quality scoring. The discussion highlights the challenges posed by the increasing scale and multimodality of AI data and introduces the Lance format as a solution for efficient data handling.

This talk addresses the challenge of connecting Large Language Models (LLMs) to real-time data and business logic. The core thesis is that LLMs, while capable of complex tasks like coding, often struggle to interact intelligently with structured and unstructured data sources. The proposed solution involves making live data and business logic available to LLMs as tools, enabling them to perform data-driven actions.

This talk focuses on architecting and testing reliable AI agents, moving beyond traditional chains and open-ended agents that suffer from poor reliability. The presenter introduces Lang Graph as a solution for building controllable agents that balance flexibility with robustness by expressing control flows as graphs with nodes and edges, incorporating state for memory.

This talk draws a parallel between the historical achievement of microprocessors breaking the 1 GHz barrier and the current rapid advancements in Large Language Models (LLMs). It posits that LLMs are innovating at a pace exceeding Moore's Law, leading to transformative capabilities. The core thesis is that the increasing speed and efficiency of LLMs will fundamentally change how we build, run, and scale software, potentially making LLMs the core of future computing paradigms, akin to the industrial revolution's impact on manufacturing.

This talk focuses on the advantages of using open-source AI models for modern applications, emphasizing how they can achieve significantly better performance and speed compared to proprietary models. The presentation highlights the challenges in productionizing open models, such as complex setup, optimization, and achieving enterprise-scale reliability, and introduces Fireworks AI's solutions for these issues. The core argument is that by optimizing serving stacks and enabling fine-tuning, open models can be made 10x faster and more cost-effective for a wide range of applications.

This talk introduces Convex as a platform that accidentally became an AI development tool. Initially designed to replace traditional backend engineering with a high-level, functional interface similar to Firebase, Convex's pervasive data flow tracking and reactive paradigm proved exceptionally well-suited for generative AI applications. The platform's ability to seamlessly sync state between backend processes and the application frontend has led to over 90% of its projects being AI-related since the post-ChatGPT boom.

This talk introduces a new voice agent API that consolidates speech-to-text, large language model (LLM) processing, and text-to-speech into a single audio-in, audio-out interface. This approach simplifies development by abstracting away the complexities of integrating these components, enabling developers to build real-time voice applications more efficiently. The presentation demonstrates how to create a voice AI agent capable of function calling and discusses strategies for scaling these agents.

This talk explores the evolution of product design, moving from traditional screen-based interfaces to "unbounded products" that transcend typical input methods. It emphasizes the challenges of unpredictability and user confusion in these new paradigms and proposes design structures to create clarity and familiarity. The discussion covers historical precedents, current AI product patterns, and future interface directions.

This talk outlines a systematic approach to constructing domain-specific LLM evaluation systems. It emphasizes the importance of moving beyond initial "vibe checks" and prompt engineering to establish a robust framework for consistent AI improvement. The core thesis is that a well-defined evaluation system, built on foundational principles, is crucial for developing production-ready AI applications and unlocking advanced capabilities like fine-tuning.

This talk introduces TensorRT-LLM, an NVIDIA SDK designed for high-performance deep learning inference on NVIDIA GPUs. It focuses on optimizing large language models (LLMs) to achieve higher throughput and lower latency, crucial for production environments. The presentation covers building, configuring, benchmarking, and deploying TensorRT-LLM engines, emphasizing practical application through live coding and detailed explanations.

This talk introduces SambaNova's full-stack AI platform, highlighting its capability to achieve over 1,000 tokens per second inference speed for Llama 3. The platform integrates hardware, system software, and AI models to simplify the development and deployment of enterprise-grade generative AI applications. It aims to combine the broad capabilities of large monolithic models with the adaptability and control of smaller, open-source models.

This talk introduces Extended Mind Transformers (EMTs), a novel approach to enhance language model performance by integrating a retrieval mechanism directly into the Transformer's attention mechanism. Unlike traditional methods like long context windows or Retrieval-Augmented Generation (RAG), EMTs allow the model to dynamically retrieve and attend to relevant information from a memory store during generation, without requiring fine-tuning. This method aims to improve accuracy, enable more granular citations, and reduce hallucinations.

This talk uses a courtroom drama format to highlight common pitfalls in AI engineering, particularly concerning Large Language Model (LLM) development and deployment. The core thesis is that rigorous evaluation, logging, and prompt iteration are crucial for successful LLM projects, and neglecting these can lead to significant problems. The presentation emphasizes the importance of a human-in-the-loop approach for effective LLM judging and evaluation.

This talk argues that Pydantic remains the essential tool for building reliable applications with large language models (LLMs). The core thesis is that by leveraging Pydantic's schema definition and validation capabilities, developers can move beyond the unreliability of unstructured text outputs and program with data structures, similar to classical software development. This approach enhances composability, reliability, and developer experience when interacting with LLMs.

Hyperspace is developing a decentralized AI network that leverages community-contributed computing resources without relying on centralized GPUs. Their product, Hyperspace, aims to provide a superior AI experience by integrating diverse, fine-tuned models rather than relying on a single large model. This platform combines prompt engineering, visual react flow, Python execution, and retrieval-augmented generation (RAG) for advanced AI workflows.

This talk introduces Graph RAG, a method that combines knowledge graphs with Retrieval Augmented Generation (RAG) to enhance the accuracy and capabilities of LLM applications. It posits that by integrating structured knowledge graph data with unstructured text retrieval, developers can achieve more precise and comprehensive answers, moving beyond the limitations of traditional RAG.

This talk demonstrates how to build and iterate on AI features within applications by leveraging a platform that integrates models with traditional programming paradigms. It showcases practical examples, starting with a game called Hyper Categories to illustrate AI-powered scoring and uniqueness validation, and then progresses to applying these concepts to real-world problems like triaging GitHub issues and enabling natural language search for similar issues. The core thesis is that combining AI models with functions and data sources, managed through a cohesive development environment, simplifies the creation of powerful AI-driven applications.

This talk introduces Hypermode, a runtime and toolset designed to simplify AI integration into applications. The core thesis is that rapid iteration is key to success in software development, a principle that also applies to AI. Hypermode aims to reduce the friction and fear associated with getting AI wrong, enabling developers to experiment, integrate, and observe AI functions in production with ease.

This talk introduces Trun Tools, a generative AI provider focused on the construction industry. It highlights the immense complexity and data volume in construction projects, where millions of pages of documentation are common for a single skyscraper. The core problem addressed is data discrepancies and rework, which cost the industry an estimated $1.5 trillion annually. Trun Tools aims to solve this by creating a centralized knowledge base, referred to as the "brain behind construction," and deploying AI agents on top of it to assist human decision-making and resolve issues.

This talk explores how programmers can leverage Large Language Models (LLMs) to significantly boost their productivity, aiming for a "10x" improvement. The core thesis is that LLMs should be treated not as autonomous reasoning agents, but as powerful translation engines and world simulators. By understanding how to decompose problems into language translation steps and by creatively framing prompts, developers can unlock new levels of efficiency and innovation.

This talk explores practical lessons learned from building autonomous software engineering systems, referred to as droids. It defines agentic systems by three core characteristics: planning, decision-making, and environmental grounding. The discussion emphasizes strategies for enhancing reliability and effectiveness in these systems, drawing inspiration from control systems, robotics, and symbolic AI.

This talk, Building with Anthropic Claude: Prompt Workshop with Zack Witten, focuses on practical prompt engineering techniques for using Anthropic's Claude models. The session involved live testing and iteration of user-submitted prompts, demonstrating how to refine prompts for better accuracy, conciseness, and desired output formats. Key themes include structuring prompts, utilizing specific formatting like XML, managing response length, and employing advanced techniques like pre-fills and stop sequences.

This talk demonstrates how to quickly deploy AI applications using AI templates and Azure services. It highlights the benefits of Microsoft's startup programs, including Azure credits and access to developer tools, and introduces AI templates as a way to accelerate development with pre-built skeletons. The session focuses on practical, hands-on deployment of various AI applications, including chat and Retrieval Augmented Generation (RAG) models.

This talk provides a deep dive into the inner workings of Large Language Models (LLMs) by dissecting GPT-2 small and reconstructing its functionality within a Microsoft Excel spreadsheet. The presentation aims to demystify LLMs for individuals without formal machine learning degrees, illustrating how text generation is fundamentally a complex mathematical problem. It covers the model's anatomy, its thought process through a virtual MRI, and concludes with a demonstration of AI "brain surgery" to alter its behavior.

This talk explores the practical application of AI agents in building a company, emphasizing their ability to handle complex automations and adapt in real-time. The speaker argues that the software development paradigm is shifting from strictly defined inputs and outputs to a more fluid, AI-driven approach. The core thesis is that AI agents are not just a future concept but a present reality, rapidly transforming how businesses operate and build products.

This talk draws a parallel between the early adoption of electricity in factories and the current integration of AI and LLMs. It argues that true innovation comes not from simply replacing old technologies with new ones, but from redesigning systems from the ground up with the new technology at their core. The presentation highlights Anthropic's latest advancements, particularly Claude 3.5 Sonnet and the new Artifacts feature, as steps towards enabling this paradigm shift in AI product development.

Emergence is developing AI agents and infrastructural platforms to enable complex enterprise workflows. The core thesis is that AI agents, capable of acting, planning, and verifying, are poised to drive significant productivity benefits, particularly within enterprise environments. The company is focused on advancing the science of AI agents through self-improvement, planning, and reasoning, with an emphasis on agent-oriented programming for stitching agents together.

This talk delves into the low-level technical aspects of Large Language Models (LLMs), focusing on practical implementation details and common pitfalls. The speaker, Daniel Han, shares insights gained from analyzing and debugging open-source models like Gemma and NVIDIA's NeMo, highlighting issues in tokenization, model architecture, and training methodologies. The presentation aims to equip AI engineers with the knowledge to identify and fix bugs, optimize fine-tuning processes, and understand the underlying mathematical principles of LLMs.

This talk explores the evolution and integration of AI-powered coding assistants, focusing on GitHub Copilot. It posits that AI tools are shifting from simple autocompletion to comprehensive development partners, aiming to enhance developer productivity and democratize access to coding. The core thesis is that AI should augment human capabilities, not replace them, by handling tedious tasks and facilitating exploration within the codebase.

This talk introduces MAX, an AI framework designed to enhance developer productivity and performance across both CPU and GPU hardware. It addresses the fragmentation and complexity in the current AI development landscape, aiming to provide a unified, high-performance solution that allows developers to own and control their AI models and data. MAX focuses on inference and aims to simplify the deployment of PyTorch models and generative AI applications.

This talk outlines five practical steps for software developers transitioning to AI engineering. It emphasizes that while deep ML research is no longer a prerequisite, understanding AI fundamentals, leveraging AI developer tools for productivity, and actively prototyping with AI models are crucial. The presentation highlights the evolving role of AI engineers and the tools available to facilitate this transition, including AI-powered assistants and managed services for model experimentation.

This talk, delivered by a collective of six AI engineers, distills a year of practical experience building with Large Language Models (LLMs). The core thesis is that successful LLM application development hinges not on proprietary models, but on strategic product building, robust operational processes, and meticulous tactical execution. The speakers emphasize a continuous improvement loop, drawing parallels to established software engineering and machine learning practices, to navigate the complexities and uncertainties inherent in LLM development.

The talk addresses the evolving landscape of AI models, highlighting the democratization of advanced capabilities previously exclusive to GPT-4. It emphasizes that the barrier to high-quality AI models has been significantly lowered, with multiple organizations now offering comparable performance. However, the core challenges shift from model access to effective and responsible utilization, including navigating complex tool interfaces, building trust, and mitigating risks like prompt injection and the proliferation of unreviewed AI-generated content.

Llamafile is an open-source project from Mozilla aiming to democratize AI access by packaging AI models into single, executable files. These files run on various operating systems and hardware, including CPUs and GPUs, without requiring installation. The project emphasizes improving CPU inference speed and enabling local, private AI applications.

This talk explores the evolution of knowledge assistants, moving beyond basic retrieval-augmented generation (RAG) to sophisticated single-agent and multi-agent systems. The core thesis is that building production-grade knowledge assistants requires advancements in data processing, agentic query flows, and multi-agent orchestration to handle complex tasks and interactions effectively.

This talk introduces Devin, an autonomous AI software engineer developed by Cognition AI. It highlights Devin's capabilities through demonstrations, including building a mobile-friendly website and contributing to Cognition AI's own codebase by adding a search bar. The presentation emphasizes the shift from text completion AI to autonomous agents capable of decision-making and problem-solving within a software engineering context.

This talk explores the evolution and future of AI, focusing on OpenAI's advancements in multimodality, enabling more natural human-computer interactions. It highlights the journey from text-based models to integrating vision, audio, and video, culminating in the GPT-4o model. The core thesis is that by embracing these multimodal capabilities and focusing on developer experience, builders can create the next generation of AI-native products.