World's Fair 2024
Scaling AI in Education: A Khanmigo case study: Shawn Jansepar
Overview
This talk details Khan Academy's journey in scaling AI for educational purposes, focusing on their AI-powered tutor and teacher assistant, Khanmigo. The core thesis is that AI can democratize one-on-one tutoring and teaching assistance at scale, addressing learning gaps and transforming educational experiences. The presentation highlights how Khan Academy shifted to an AI-first organization by rapidly prototyping, iterating, and integrating AI deeply into their platform and content, while also addressing key technical challenges and future directions.
Who should watch
- AI Engineers
- Product Managers
- Educators and EdTech Builders
- Those interested in AI applications for learning
- Individuals facing challenges in personalized education at scale
Key takeaways
- Khanmigo is designed as a Socratic tutor that guides students to answers rather than providing them directly, ensuring deeper learning and mathematical accuracy.
- The platform integrates AI deeply into educational content, allowing for context-aware tutoring and assistance, such as with essay writing and complex math problems.
- Khan Academy transformed into an AI-first organization through rapid prototyping, iterative development, and a shift in core values to embrace intuition and resist perfectionism.
- A key strategy involved embedding domain experts, including learning scientists and educators, directly into the AI development process.
- Technical hurdles in tutor accuracy were addressed by introducing a math agent, employing Chain of Thought prompting, and providing additional context via Retrieval-Augmented Generation (RAG).
- The organization developed a new product development process that differentiates code quality based on confidence in a feature's success, enabling faster prototyping.
- Future plans include releasing a math accuracy benchmark dataset, exploring integrations with tools like Wolfram, and fine-tuning models for specific educational tasks.
- Scaling efforts involve leveraging multiple models, optimizing for performance and cost, and implementing perceived performance improvements to enhance user experience.
Notable quotes
"We believe AI can be used to successfully fill those unfinished learning gaps at scale by democratizing one-on-one tutoring for anyone, anywhere."
"We're really not just trying to use AI for AI's sake."
"We resisted perfectionism, which I would call an issue of Khan Academy of the past."
Unofficial community note. Prefer the recording for nuance.