Apple study finds human-like behaviors in LLMs vary by model and user factors

Apple’s Machine Learning Research group published a study analyzing the prevalence, perceived appropriateness, and controllability of human-like behaviors in large language models (LLMs). The research team used both LLM-as-a-judge and human evaluation to assess 21,000 multi-turn conversations spanning four widely used models: gpt-4o, gpt-4.1-mini, claude-sonnet-4.6, and gemini-2.5-flash.

The study found that human-like behaviors are pervasive across models but vary depending on the model and user factors such as conversation goals and user profiles. When evaluating perceived appropriateness, human evaluators rated self-referential and relationship-building behaviors as less appropriate when exhibited by LLMs compared to humans. Conversely, boundary-maintaining behaviors—such as refusing inappropriate requests—were judged more appropriate when performed by LLMs than by humans.

The authors also demonstrated that system prompting can influence these behaviors, though they caution that controlling such behaviors requires careful evaluation to avoid unintended consequences. The findings are intended to inform responsible LLM design and evaluation practices, providing recommendations for practitioners.

The paper is authored by Sunnie S. Y. Kim, Margit Bowler, and Leon A. Gatys, and was published in August 2026.