World's Fair 2024
Building with Anthropic Claude: Prompt Workshop with Zack Witten
Overview
This talk, Building with Anthropic Claude: Prompt Workshop with Zack Witten, focuses on practical prompt engineering techniques for using Anthropic's Claude models. The session involved live testing and iteration of user-submitted prompts, demonstrating how to refine prompts for better accuracy, conciseness, and desired output formats. Key themes include structuring prompts, utilizing specific formatting like XML, managing response length, and employing advanced techniques like pre-fills and stop sequences.
Who should watch
- AI engineers and developers looking to improve their prompt engineering skills.
- Product Managers and builders seeking to leverage LLMs more effectively.
- Anyone struggling with specific output formats or model behavior.
- Users interested in practical tips for working with Anthropic's Claude models.
Key takeaways
- **Prompt Structure:** Placing information before instructions generally leads to better adherence. For complex tasks, consider structuring prompts with XML tags for clarity, as Claude was trained on extensive XML data.
- **Response Length Control:** Explicitly defining desired response length (e.g., "one to two sentences" or "never more than three") is more effective than general instructions like "be concise."
- **Output Formatting:** Using pre-fills, especially for structured outputs like JSON, can significantly improve reliability. Assistant pre-fills allow you to partially complete a response, guiding the model.
- **Handling Specific Outputs:** For JSON outputs, using assistant pre-fills with the opening bracket or employing JSON tags with stop sequences can prevent unwanted preamble text.
- **Iterative Refinement:** Prompt engineering is an iterative process. Testing prompts with various inputs and using console features like the evaluate tab are crucial for improvement.
- **Examples are Powerful:** Providing high-quality examples, including negative examples (anti-examples), is one of the most effective ways to guide model behavior and modulate tone.
- **Code Over Prompting:** For deterministic tasks like formatting, offloading the work to code (e.g., using post-processing with regex or API parameters like stop sequences) is often more reliable and cost-effective than complex prompting.
- **Persona Prompting:** For multi-persona interactions, consider routing queries dynamically via code rather than embedding all persona logic within a single prompt.
Notable quotes
*Claude was trained with a lot of XML in its training data and so it's sort of seen more of that than it's seen of other formats.*
*Instructions are more tightly followed the closer they are to the bottom of the prompt.*
*It's cheaper and easier to do a little bit of work outside of a call to the LLM and not even worry about prompting.*
Unofficial community note. Prefer the recording for nuance.