Research / evaluation / Safety & wellbeing
RELATE tests how models direct simulated support conversations towards the AI or towards human connection. Automated judges found increasing AI-directed language, but human annotators found the opposite trend in the small validation sample. The disagreement is central to interpreting the result.
Why it matters
Relevant to companion evaluation and to checking whether automated measures reflect human interpretation.
Read with care
Two research-team annotators examined 881 sentences from 21 dialogues. In the 19 dialogues with usable first/final turns, humans found a decrease where both automated judges found an increase. The headline trend is judge-dependent. No real-user outcomes, long-term dependency or child-specific effects established. Abstract, selected methods, human-validation and limitations sections checked; no independent replication or peer-review confirmation.
What was checked
Source record and stated publication date checked; see reading cautions for the extent and limits of the review. Checked 2026-10-12.