Sitelet https://github.com/langchain-ai/langsmith-java/pull/212
Skip to content

fix(evaluation): retain runs after target errors - #212

Draft
Abhinav Kumar Singh (abhinavkr26104) wants to merge 3 commits into
langchain-ai:mainfrom
abhinavkr26104:fix/capture-traced-evaluation-errors
Draft

fix(evaluation): retain runs after target errors#212
Abhinav Kumar Singh (abhinavkr26104) wants to merge 3 commits into
langchain-ai:mainfrom
abhinavkr26104:fix/capture-traced-evaluation-errors

Conversation

@abhinavkr26104

@abhinavkr26104 Abhinav Kumar Singh (abhinavkr26104) commented Aug 9, 2026

Copy link
Copy Markdown

Fixes #211

Summary

  • Capture the traced run before calling the evaluation target.
  • Retain error-bearing runs when a target throws under EvaluateErrorHandling.LOG.
  • Add regression coverage for this path.

Testing

  • git diff --check
  • Focused EvaluateRunnerTest was attempted but exceeded the local execution timeout.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Logged target errors abort dataset evaluation

1 participant