Finding / eve
2026-10-13-eve-configured-datadog-eval-reporters-export-content-by-default
Configured Datadog eval reporters export content by default. The Datadog reporter changes from metadata/scores by default to recording eval prompts, outputs, authored expectations, errors and assertion details. It reuses a dataset across runs for comparison. Without Datadog credentials it skips reporting; the respective record options can disable content.
Channel: tagged-release. Half: both. Date: 2026-10-01 (eve@0.69.0).
Operator consequence: Before running sensitive fixtures, review recordInputs, recordOutputs, recordExpectedOutputs, recordErrors and recordAssertionDetails and set them false where necessary. Review this separately from runtime audience-based trace policy; its unchanged table does not establish unchanged eval export.
Receipt
Finding metadata
Run: 2026-10-13-profile-refresh-2026-09-21_2026-10-13-frontier-v0
Finding ID: 2026-10-13-eve-configured-datadog-eval-reporters-export-content-by-default
Source links
Primary links, including exact changelog lines when available.
- commit2026-10-13-eve-configured-datadog-eval-reporters-export-content-by-defaultgithub.com/vercel/eve/commit/f0a9ee199cc3ea0048d4d8c527888d888f8bade6github release2026-10-13-eve-configured-datadog-eval-reporters-export-content-by-defaultvercel/eve / eve%400.69.0tagged commit file2026-10-13-eve-configured-datadog-eval-reporters-export-content-by-defaultvercel/eve / docs/evals/reporters.mdx