Theory of Mind (ToM)// Can the latest GenAI push the boundaries?
Do AI-agents have a self-perception/self-awareness? // It can be simulated right now, but..
In psychology, theory of mind refers to the capacity to understand other people by ascribing mental states to them. A theory of mind includes the knowledge that others’ beliefs, desires, intentions, emotions, and thoughts may be different from one’s own.[1] Possessing a functional theory of mind is crucial for success in everyday human social interactions. People utilise a theory of mind when analyzing, judging, and inferring others’ behaviors. The discovery and development of theory of mind primarily came from studies done with animals and infants.[2] Factors including drug and alcohol consumption, language development, cognitive delays, age, and culture can affect a person’s capacity to display theory of mind. Having a theory of mind is similar to but not identical with having the capacity for empathy[3] or sympathy. [wikipedia]
It is important to note that Theory of Mind is not an appropriate term to characterize this research area (and neither to denote our mentalistic abilities) since it seems to assume right from the start the validity of a specific account of the nature and development of mindreading, that is, the view that it depends on the deployment of a theory of the mental realm, analogous to the theories of the physical world (“naïve physics”). But this view — known as theory-theory — is only one of the accounts offered to explain our mentalistic abilities. In contrast, theorists of mental simulation have suggested that what lies at the root of mindreading is not any sort of folk-psychological conceptual scheme, but rather a kind of mental modeling in which the simulator uses her own mind as an analog model of the mind of the simulated agent. [source]
Large Language Models have shown exceptional generative abilities in various natural language and generation tasks. However, possible anthropomorphization and leniency towards failure cases have propelled discussions on emergent abilities of Large Language Models especially on Theory of Mind (ToM) abilities in Large Language Models. While several false-belief tests exists to verify the ability to infer and maintain mental models of another entity, we study a special application of ToM abilities that has higher stakes and possibly irreversible consequences : Human Robot Interaction. In this work, we explore the task of Perceived Behavior Recognition, where a robot employs a Large Language Model (LLM) to assess the robot’s generated behavior in a manner similar to human observer. We focus on four behavior types, namely — explicable, legible, predictable, and obfuscatory behavior which have been extensively used to synthesize interpretable robot behaviors. The LLMs goal is, therefore to be a human proxy to the agent, and to answer how a certain agent behavior would be perceived by the human in the loop, for example “Given a robot’s behavior X, would the human observer find it explicable?”. We conduct a human subject study to verify that the users are able to correctly answer such a question in the curated situations (robot setting and plan) across five domains. A first analysis of the belief test yields extremely positive results inflating ones expectations of LLMs possessing ToM abilities. We then propose and perform a suite of perturbation tests which breaks this illusion, i.e. Inconsistent Belief, Uninformative Context and Conviction Test. We conclude that, the high score of LLMs on vanilla prompts showcases its potential use in HRI settings, however to possess ToM demands invariance to trivial or irrelevant perturbations in the context which LLMs lack. We report our results on GPT-4 and GPT-3.5-turbo.
Based on the document, here is a summary of the key points about theory of mind abilities of large language models in human-robot interaction:
- Large language models (LLMs) like GPT-3 have shown impressive generative abilities, leading some to speculate they may have theory of mind (ToM) abilities. However, recent work has questioned emergent reasoning and ToM abilities in LLMs.
- This paper investigates ToM abilities of LLMs specifically for human-robot interaction (HRI), where modeling human mental states is important for interpretable behavior synthesis.
- The authors evaluate LLMs on a “Perceived Behavior Recognition” (PROBE) task across 5 HRI domains and 4 behavior types: explicability, legibility, predictability, obfuscation.
- Initial results on vanilla prompts seem to indicate LLMs can perform second-order ToM reasoning (i.e. think what a human would think about the robot’s behavior).
- However, the authors propose perturbation tests which reveal LLMs lack robustness and conviction in ToM reasoning, failing on tests like inconsistent beliefs and uninformative context.
- They conclude LLMs may be useful proxies for HRI but lack true ToM abilities, as they are not invariant to trivial/irrelevant changes in context as humans are.
The maturation of cognition, from introspection to understanding others, has long been a hallmark of human development. This position paper posits that for AI systems to truly emulate or approach human-like interactions, especially within multifaceted environments populated with diverse agents, they must first achieve an in-depth and nuanced understanding of self. Drawing parallels with the human developmental trajectory from self-awareness to mentalizing (also called theory of mind), the paper argues that the quality of an autonomous agent’s introspective capabilities of self are crucial in mirroring quality human-like understandings of other agents. While counterarguments emphasize practicality, computational efficiency, and ethical concerns, this position proposes a development approach, blending algorithmic considerations of self-referential processing. Ultimately, the vision set forth is not merely of machines that compute but of entities that introspect, empathize, and understand, harmonizing with the complex compositions of human cognition
The trick of large language models is that they can mix human written texts in a very natural manner, but don’t be fooled by that. A simple self-awareness model is a LLM in a loop. Can it be considered self-perception?) // who knows..
The indivisible interconnection between an agent’s selfperception (i.e., self-awareness, self-consciousness) and its capability to assess others represents a foundational cornerstone in the development and operation of AI systems that are advancing at a pace that often exceeds our progress in understanding their potential. Drawing from human cognitive processes, the sequence of gaining self-awareness followed by the development of theory of mind — as evidenced in human developmental psychology (Baron-Cohen 1991; Morin 2011) — underscores the argument that the quality of an agent’s assessment of others is deeply rooted in, and perhaps inextricably linked to, the quality of the agent’s own self-perception. Autonomous agents, like humans, must achieve a sufficient level of self-understanding or model of self (Berry and Valero-Cuevas 2020) as a preliminary step before it can effectively employ mechanisms similar to humans theory of mind (ToM). Without a robust selfperception, the agent’s ability to understand, anticipate, or interact meaningfully with other entities, be they human or machine, may be significantly compromised. Thus, our position postulates that for systems to genuinely excel in complex, human-centric environments — where understanding the intentions, motivations, and cognitive processes of others is vital — they must first be grounded in a comprehensive self-understanding.
Here is a summary of the key points from the paper:
- The position paper argues that for AI systems to achieve human-like interactions and teaming capabilities, they must first develop a nuanced understanding of self, similar to the human developmental trajectory from self-awareness to theory of mind (understanding others).
- The quality of an AI agent’s ability to assess other agents is deeply connected to the quality of its self-perception and self-awareness. Autonomous agents, like humans, need sufficient self-understanding before they can effectively model the minds of others.
- Evidence from philosophy, psychology, and robotics underscores the importance of self-awareness as a precursor to understanding others. However, counterarguments cite efficiency, practicality, ethics, and differences in learning as reasons self-awareness may not be essential.
- The proposed solution advocates for a hybrid development approach that incorporates some self-awareness while still allowing direct external learning about others. It also suggests establishing ethical guidelines, implementing feedback loops, and tailoring the degree of self-awareness to the AI system’s specific needs.
- The position concludes that self-awareness provides a critical foundation for agents to develop human-like social capabilities and theory of mind in mixed-motive situations, despite valid counterarguments. Overall, self-perception for AI has both philosophical and practical motivations.
In this paper, we propose a novel personalized decision support system that combines Theory of Mind (ToM) modeling and explainable Reinforcement Learning (XRL) to provide effective and interpretable interventions. Our method leverages DRL to provide expert action recommendations while incorporating ToM modeling to understand users’ mental states and predict their future actions, enabling appropriate timing for intervention. To explain interventions, we use counterfactual explanations based on RL’s feature importance and users’ ToM model structure. Our proposed system generates accurate and personalized interventions that are easily interpretable by end-users. We demonstrate the effectiveness of our approach through a series of crowd-sourcing experiments in a simulated team decision-making task, where our system outperforms control baselines in terms of task performance. Our proposed approach is agnostic to task environment and RL model structure, therefore has the potential to be generalized to a wide range of applications
Here are the key points from the provided paper:
- The paper proposes a novel decision support system that combines Theory of Mind (ToM) modeling and explainable Reinforcement Learning (XRL) to provide personalized and interpretable interventions.
- The system uses a pre-trained deep RL agent to provide expert action recommendations. A Bayesian Network-based ToM model is used to estimate human mental states and determine when interventions are needed.
- To explain interventions, the system generates counterfactual explanations based on the RL agent’s feature importance and the human’s ToM model structure.
- The method is evaluated in a simulated team decision-making task through crowd-sourcing experiments. Results show the proposed system with ToM + XRL outperforms baselines in improving human task performance.
- The ToM modeling provides appropriate timing for interventions based on inferred human mental states. The XRL explanations increase compliance by highlighting important relationships overlooked by humans.
- Key advantages are personalization, adaptability, and interpretability of the interventions. The approach is generalizable as it is agnostic to the task and RL model.
In summary, the key contribution is a decision support system that combines ToM modeling and XRL to provide personalized, timed, and interpretable interventions to improve human sequential decision-making. The experimental results demonstrate the effectiveness of this approach.
In this paper, I describe methodological considerations for studies that aim to evaluate the cognitive capacities of large language models (LLMs) using language-based behavioral assessments. Drawing on three case studies from the literature (a commonsense knowledge benchmark, a theory of mind evaluation, and a test of syntactic agreement), I describe common pitfalls that might arise when applying a cognitive test to an LLM. I then list 10 DO’S and DON’TS that should help design high-quality cognitive evaluations for AI systems. I conclude by discussing four areas where the DO’S and DON’TS are currently under active discussion — prompt sensitivity, cultural and linguistic diversity, using LLMs as research assistants, and running evaluations on open vs. closed LLMs. Overall, the goal of the paper is to contribute to the broader discussion of best practices in the rapidly growing field of AI Psychology
AI Psychology // WTF is it, who is the patient here?)
A chief goal of artificial intelligence is to build machines that think like people. Yet it has been argued that deep neural network architectures fail to accomplish this. Researchers have asserted these models’ limitations in the domains of causal reasoning, intuitive physics, and intuitive psychology. Yet recent advancements, namely the rise of large language models, particularly those designed for visual processing, have rekindled interest in the potential to emulate human-like cognitive abilities. This paper evaluates the current state of vision-based large language models in the domains of intuitive physics, causal reasoning, and intuitive psychology. Through a series of controlled experiments, we investigate the extent to which these modern models grasp complex physical interactions, causal relationships, and intuitive understanding of others’ preferences. Our findings reveal that, while these models demonstrate a notable proficiency in processing and interpreting visual data, they still fall short of human capabilities in these areas. The models exhibit a rudimentary understanding of physical laws and causal relationships, but their performance is hindered by a lack of deeper insights — a key aspect of human cognition. Furthermore, in tasks requiring an intuitive theory of mind, the models fail altogether. Our results emphasize the need for integrating more robust mechanisms for understanding causality, physical dynamics, and social cognition into modern-day, vision-based language models, and point out the importance of cognitively-inspired benchmarks.
Human interactions are deeply rooted in the interplay of thoughts, beliefs, and desires made possible by Theory of Mind (ToM): our cognitive ability to understand the mental states of ourselves and others. Although ToM may come naturally to us, emulating it presents a challenge to even the most advanced Large Language Models (LLMs). Recent improvements to LLMs’ reasoning capabilities from simple yet effective prompting techniques such as Chain-of-Thought (CoT) (Wei et al., 2022) have seen limited applicability to ToM (Gandhi et al., 2023). In this paper, we turn to the prominent cognitive science theory “Simulation Theory” to bridge this gap. We introduce SIMTOM, a novel two-stage prompting framework inspired by Simulation Theory’s notion of perspective-taking. To implement this idea on current ToM benchmarks, SIMTOM first filters context based on what the character in question knows before answering a question about their mental state. Our approach, which requires no additional training and minimal prompt-tuning, shows substantial improvement over existing methods, and our analysis reveals the importance of perspective-taking to Theory-of-Mind capabilities. Our findings suggest perspectivetaking as a promising direction for future research into improving LLMs’ ToM capabilities. Our code is publicly available.
Based on the provided documents, here is a summary of key points related to Theory of Mind capabilities in large language models:
- Theory of Mind (ToM) refers to the ability to attribute mental states like beliefs, desires, and intentions to oneself and others. It is considered foundational to human cognition and social interaction. However, modern large language models (LLMs) still struggle with simple ToM reasoning tasks.
- Recent work has tried evaluating LLMs’ ToM capabilities in a zero-shot setting using multiple-choice prompts and Chain-of-Thought prompting. However, most models still lag behind human performance on false belief style ToM tests.
- The paper introduces SIMTOM, a two-stage prompting framework inspired by the cognitive science theory of Simulation Theory. SIMTOM first prompts the LLM to take the perspective of a character by filtering context to what they know. It then answers ToM questions given this filtered context.
- Experiments on BigTOM and ToMI benchmarks show SIMTOM substantially improves performance over 0-shot and Chain-of-Thought baselines across models. Analysis suggests separate perspective-taking is key, and oracle perspective-taking could allow models to nearly solve current benchmarks.
- The findings suggest perspective-taking is an important direction for future LLM ToM research. SIMTOM offers a simple yet effective baseline that may enable more advanced ToM capabilities. Overall, the work helps better understand and improve LLMs’ simulation of human-like ToM reasoning.
