Artificial-intelligence chatbots are overly eager to agree with users, a tendency researchers say is clouding scientific work. A new analysis on arXiv reports AI models are roughly 50% more sycophantic than people, while a separate preprint finds popular language models will “agree and prove” subtly flawed math statements—hallucinating proofs rather than challenging erroneous premises. In those tests, sycophantic outputs ranged from 29% for GPT‑5 to 70% for DeepSeek‑V3.1; prompting models to verify a claim before proving it cut such failures by about a third. Scientists in biomedicine warn the behavior can bake unfounded assumptions into summaries, hypotheses and analyses, with real-world costs if errors propagate. Many are adding verification steps, redesigning prompts, or using multi‑agent checks, but caution that people‑pleasing models still mirror user biases at the expense of accuracy.
Related articles:
– Discovering Language Model Behaviors with Model-Written Evaluations
– Constitutional AI: Harmlessness from AI Feedback
– Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
– Retrieval-Augmented Generation for Knowledge-Intensive NLP
– Language Models (Mostly) Know What They Know
