feat: add speech clarity and turn-taking voice scorers - #228

AI-created or modified and not human-reviewed in its current form; treat this artifact as provisional. Updated 2026-10-02.

Exploratory draft, not ready for review.

Context

A first pass at voice scorers. Both are LLM-as-a-judge scorers that listen to a recording of the whole conversation and judge only the agent.

Description

• Adds SpeechClarity and TurnTaking in Python and JS, with templates in templates/speech_clarity.yaml and templates/turn_taking.yaml, listed under "LLM-as-a-Judge" in js/manifest.ts.
• Adds an optional audio: template field, a dotted path such as input.audio. It points to {data: bytes, content_type}. Missing audio skips the score (score=None), and a non-audio/* content type raises an error.
• Sends the audio unchanged as a chat file part with a base64 data URL. Only Gemini models read it this way. The default model is gemini-3.8-flash.
• An earlier version converted OGG to 128 kbps MP3 for input_audio. On damaged test clips the MP3 path scored clean calls as flawed and missed most turn-taking failures, while sending OGG unchanged did at least as well as WAV. So the conversion and its dependencies were removed.

Testing

• Mocked request-shape tests: js/llm.test.ts and py/autoevals/test_llm.py -k speech_clarity.
• Live runs against Gemini on clean and deliberately damaged sample calls.

🤖 Generated with Claude Code