Stroop Test Reveals Limits of Artificial Intelligence Attention Skills

Artificial Intelligence systems can write articles, answer questions, and solve complex problems, leading many to believe that AI is approaching or even surpassing human intelligence. However, a new study has highlighted a critical limitation of modern AI models: maintaining focus when faced with distractions. Researchers from the City University of New York used the well-known Stroop Test, a psychological assessment designed to measure attention and executive control, to evaluate the concentration abilities of leading Large Language Models (LLMs), including those behind ChatGPT, Claude, and Gemini.

The study found that AI models performed well when presented with short lists of color-related words. However, their accuracy dropped significantly as the lists became longer and more complex. GPT-4o achieved 91 percent accuracy with five words but fell to just 15 percent when tested with 40 words. Similar declines were observed in Claude and Gemini models. Researchers noted that AI systems often struggled to ignore irrelevant information and remained influenced by patterns learned during training. In contrast, humans can suppress distractions and maintain focus on a specific task, even under challenging conditions. The findings suggest that while AI excels in language and reasoning, human attention and cognitive control remain areas where people continue to outperform machines.