온디맨드

Evals for AI Agents: How Product Builders Get the Most Out of Every New Model

Join Anthropic's Applied AI team for a session on evaluating AI agents.

Most teams shipping AI agents can't tell whether a new model actually improves their product.The hard part is that the products people are shipping now aren't single prompts. They're agents that call tools, retrieve context, and take several steps before a customer sees output, with a different way to fail at each step. We'll show how the teams we work with evaluate agents end to end, decide quickly whether a new model is worth switching to, and stay confident their product performs at the level customers expect.

You'll see real examples from startups building on Claude and leave with something you can put into practice the same week.

주요 연사

  • Preston Tuggle

    Applied AI @ Anthropic

  • Jimmy Chan

    Applied AI @ Anthropic

학습 내용

  • Why single-turn evals miss most of what goes wrong in an AI agent, and what to measure instead

  • How to build a first eval set from real production failures

  • How to decide quickly whether a new model release is worth adopting for your product

Claude로 조직의 운영 방식을 혁신하세요