Do LLMs Understand Idioms? Evidence for a Theory of Simulated Artificial Phraseological Competence

Abstract

This study examines, by means of a quasi-controlled experiment, the performance of the Claude 4.5 Haiku large language model in processing Brazilian Portuguese idiomatic expressions. Grounded in the intersection of Phraseology and Cognitive Linguistics, the study proposes the concept of simulated artificial phraseological competence, defined as a functional performance based on statistical-distributional regularities, devoid of the embodied experience, sociocultural immersion, and pragmatic inference characteristic of human language users. The experimental corpus, consisting of 140 phraseological units (including opaque, somatic, and cultural idioms, alongside experimental control samples), was evaluated across three prompting conditions—zero-shot, contextualized few-shot, and Chain-of-Thought with role-playing. Four dimensions were assessed: idiomaticity detection, semantic precision, pragmatic appropriateness, and cultural sensitivity. The findings demonstrate high semantic accuracy for conventionalized units, yet reveal asymmetrical performance across the pragmatic and cultural dimensions, alongside a pronounced tendency toward figurative hallucination when encountering non-existent idioms. The Chain-of-Thought condition yielded consistent qualitative improvements, reducing false positives. These results reinforce the hypothesis that the analyzed LLM lacks mechanisms equivalent to human phraseological competence, although evidence suggests that prompt engineering strategies can partially mitigate these architectural limitations.