Phoebe Klett

This talk introduces Extended Mind Transformers (EMTs), a novel approach to enhance language model performance by integrating a retrieval mechanism directly into the Transformer's attention mechanism. Unlike traditional methods like long context windows or Retrieval-Augmented Generation (RAG), EMTs allow the model to dynamically retrieve and attend to relevant information from a memory store during generation, without requiring fine-tuning. This method aims to improve accuracy, enable more granular citations, and reduce hallucinations.