Session brief

Using AI to Build an Infinite Game: Jeff Schomay

Overview

This talk details the process of building a game with entirely AI-generated content, focusing on creating unique, procedurally generated scenes for an infinite exploration experience. The core idea is to leverage AI for both text descriptions of game scenes and the accompanying visual assets, ensuring each playthrough is distinct.

Who should watch

• AI Engineers interested in generative content pipelines.
• Game Developers exploring novel content creation methods.
• Builders looking to create unique, replayable experiences.
• Product Managers evaluating AI for creative applications.

Key takeaways

• AI can be used to generate both descriptive text for game scenes and the visual assets for those scenes.
• Fine-tuning a language model on custom data can improve prompt reliability and reduce token usage for scene generation.
• Stylistically consistent images for a game can be achieved by fine-tuning image generation models, though this requires careful attention to training data consistency.
• A pipeline was built to request scene descriptions from a fine-tuned AI model, then use those descriptions to generate images with another AI tool.
• To address slow generation times, caching mechanisms were implemented to prefill scenes and ensure a smooth player experience.
• The generated scenes and images were unique for each playthrough, creating a game of infinite exploration.
• Potential improvements include using AI upscalers for higher resolution images and incorporating more dynamic elements like themes or weather into scene generation prompts.
• The described AI content generation process is applicable to various other projects beyond game development.

Notable quotes

Watch on YouTube →

Unofficial community note. Prefer the recording for nuance.