Canada

Limits and risks of GPT-3 applications
Recent GPT-3 applications have fed new optimism in AI research. To explore limits and risks, we investigated how far we can get in developing a so-called digital replica of Daniel Dennett, generating text outputs from a large language model trained on Dennett’s philosophical writing. In consultation with Dennett himself, we compare several fine-tuning strategies and evaluate outputs. Analyzing the failures and the successes will allow us to address technical and ethical issues such as:
1. How accurate can such models be with current technology?
2. To what extent (if at all) might it be acceptable to present a model’s outputs as representing an author’s view?
3. Should copyright holders have control over such models?
4. How one can address risks of making such models public (including possible over-reliance on their accuracy)?
5. How good are experts and non-experts in distinguishing humans from replicas?

CogSci 2022

Limits and risks of GPT-3 applications

### Welcome to ACL 2023, the 61st Annual Meeting of the Association for Computational Linguistics! 
 The conference will be held in Toronto, Canada, July 9-14, 2023. 
Following the succession of the recent conferences in our field, ACL 2023 will adopt a hybrid format.
While the impact of Covid has considerably diminished in terms of traveling, obtaining visas to Canada
entails a very long process. Moreover, the global economic conditions pose challenges for many individuals to travel to conferences. Recognizing these circumstances, we know many participants may not be
able to attend the conference in person. Therefore, we are committed to providing a great virtual platform
so everyone has the opportunity to interact with other participants and enjoy the conference. Based on the
current registered participants, approxiately 30% have chosen to attend the conference virtually. Whether
you join us in person or virtually, we sincerely hope everyone has a remarkable conference experience. 
This General Chair’s message is where I express my gratitude to the many individuals who have made
enormous contributions to the conference over the past year.

Read [**ACL 2023 General Chair's message**](https://docs.google.com/document/d/1WobYM7norbG4dI48s75HfJoD89qgX5a_F-6U8AteLSA/edit?usp=sharing/) in full.

##### **[Conference Handbook](https://2023.aclweb.org/downloads/acl2023-handbook.pdf)**

ACL 2023

The Association for Computational Linguistics (ACL) is the premier international scientific and professional society for people working on computational problems involving human language, a field often referred to as either computational linguistics or natural language processing.

poster

Join us for CogSci2022! Our 44th Annual Conference will be centered around the theme of diversity, with an accompanying range of keynotes and invited speakers who themselves highlight the diversity of our field. Our three symposia are Cognition across Diverse Populations, Cognition within Diverse Societies, and Global Challenges for Cognitive Science. The conference is being offered in a fully hybrid environment for onsite and online attendees.
 
 [**The Program at a Glance can be found here**](https://cognitivesciencesociety.org/wp-content/uploads/2022/07/Cogsci-2022-PAAG-Detailed-July-12final-1.pdf)
 [**The Detailed Program can be found here**](https://cognitivesciencesociety.org/wp-content/uploads/2022/08/DeArmond-CogSci2022-program-v113-final.pdf)

This conference requires a registration to attend. Please use the link below to secure your ticket. The website will be available to all registered attendees on July 25th. 

[Register to Attend](https://cognitivesciencesociety.org/registration/)

Registration Is Required

The 44th Annual Meeting of the Cognitive Science Society highlighted research on the theme Cognitive Diversity in addition to the full breadth and diversity of research topics offered by the society’s membership. 





The impressive recent performance of large language models such as GPT-3 has led many to wonder to what extent they can serve as models of general intelligence or are similar to human cognition. We address this issue by applying GPT-3 to a classic problem in human inductive reasoning known as property induction. Our results suggest that while GPT-3 can qualitatively mimic human performance for some inductive phenomena (especially those that depend primarily on similarity relationships), it reasons in a qualitatively distinct way on phenomena that require more theoretical understanding. We propose that this emerges due to the reasoning abilities of GPT-3 rather than its underlying representations, and suggest that increasing its scale is unlikely to change this pattern.

Human-like property induction is a challenge for large language models

In an agent-based simulation, we investigate the implications of social interaction and cognitive diversity on creative processes of divergent thinking. Agents performed a verbal association task individually and jointly in pairs. We created pairs of varying cognitive diversity by manipulating properties of the vector spaces defining their semantic memory. We find that cognitive diversity positively stimulates the flexibility of agents’ collective cognitive search, giving rise to higher fluency (more solutions) and originality (more ‘rare’ solutions). While cognitively similar agents tend to exploit local semantic neighborhoods, diversity promotes more explorative search, with longer distances traveled in semantic space. This helps diverse pairs reach more distant areas of semantic space and escape cognitive fixation. However, our model also suggests that too high levels of diversity can have detrimental effects, as overly exploratory behaviors make pairs leave solution saturated areas prematurely and increase the risk of reaching semantic “dead ends”.

Cognitive diversity promotes collective creativity: an agent-based simulation

Pursuing the mechanisms and methods to improve the creative potential in team activities becomes significant in modern society. One candidate of the creative method is LEGO Serious Play (LSP), where intuitive modelling with LEGO blocks, corresponding to Play in LSP, is considered effective in teams’ creative problem-solving. To understand the mechanism for Play to influence creativity in interpersonal interactions, we compared Play (intuitive modelling) and Non-Play modelling with LEGO blocks. Our experiments consisted of two phases, individual and collaborative modelling, and two groups conducted the former phase, intuitive modelling before thinking (Play) and thinking before modelling (Non-Play). We found that Play significantly promoted utterances about abstract concepts during the collaborative phase, while utterances about concrete models showed no difference between the two groups. This finding suggests that the interplay between mentioning abstract concepts and handling concrete objects may be a mechanism of Play as intuitive modelling to improve team creativity.

Play as intuitive modelling for fostering creativity in interacion

What factors induce empathy for virtual agents? To explore this question, we examine the relationship between humans and agents in the inverted cyberball task where a participant chose to help an ostracised agent. This study especially controls experience in the preparation task from self-involvement as the base of empathy. In the one condition, participants have induced the same experience as the ostracised agent, while in the other condition, participants observed ostracising relations between two agents. As a result, self-involvement in the preparation task does not influence the helping behavior of the ostracised agent. The factor relating to the helping behavior is whether the participants noticed the relation. This notice was also associated with empathy trait. From these results, we can conclude that empathy can acquire others' perspectives even when they have no experience with the pain of ostracism.

Effect of self-experience on empathic helping to virtual agents

Inferring others' preferences is one of the central cognitive tasks of social life. The inferred preferences are used to predict the behavior of others in novel situations; therefore, a certain level of abstraction is required. We considered the inference of preferences as a classification problem on an abstract latent feature space. Participants (n=96) were asked to show 89 images of automobiles to either an agent that had been programmed with a specific person's preference and which showed facial expressions according to that preference or an agent that did not show facial expressions. Next, participants were asked to predict the agent's likes and dislikes for novel 23 automobiles. The results showed that participants could predict the agent's likes and dislikes for the unseen automobiles with 74% accuracy, indicating that they learned the decision boundary of agents' preferences on the latent feature space using expressions of emotion as labels.

People learn other's preferences on a latent feature space using emotion expressions as labels

The ancient Greek hero Ulysses chose to bind himself to resist the temptation of Sirens, highlighting the fact that humans may voluntarily sacrifice their freedom of choice to achieve committed goals. In this work, we propose a computational model for such commitment under the framework of Bayesian Theory of Mind. The model is based on the idea that even when alone, humans act to better demonstrate their intentions to an imagined third-party observer (ITO) censoring their actions. Our model successfully captures the Ulysses-constraint of freedom, as the freedom confuses the ITO’s inference of their intention. We further show that, trajectories generated both by human actors and actors modeled with ITO censorship are easy to interpret both in the eyes of an actual human ob- server and an ITO. The results demonstrate that under conflict- ing desires, humans achieve commitment by spontaneously censoring their actions with an internalized theory of mind.

Intentional commitment through an internalized theory of mind: Acting in the eyes of an imagined observer

People collaborate daily to achieve joint goals, negotiating sidewalks, preparing meals, constructing furniture. Here we analyzed how strangers coordinate a complex novel task, assembling a TV cart. Before beginning and before major actions, partners used a photo of the completed cart, speech, and gesture to establish a joint model of the structure of the completed cart. Most coordination was implicit, a conversation of actions that could be understood in the context of the current state of the object and the shared mental representation. Actions served a dual purpose, both to communicate and to advance the task. The basic unit of coordination was the dyad, not the individual.

Explicit and Implicit Coordination of Joint Action

Although pointing is sparse, overloaded, and indirect, it allows humans to effectively decode shared information, (ex)change their minds, and plan accordingly. Pointing is an invitation to jointly attend to an object, which triggers the mutual inference between agents of each other’s mind. Relevance is a fundamental assumption underlying all human communication, including pointing. We define relevance as how much a signaler’s belief can make a positive difference to its receiver’s well being. We build a Theory of Mind (ToM) model to test our definition of relevance and use pointing as a case study. In two experiments, we test our relevance model in a classic artificial intelligence (AI) task, the Wumpus world, with the key difference that there is a guide that points to help a hunter. Agents with our relevance model gain significantly higher rewards than agents who ignore signals from the guide. Agents with our model also achieve better performance than agents who receive an additional observation of the environment. The results show that the power of pointing comes from the ToM inference of relevance, rather than providing more precise individual perception.

What Is the point? a Theory of Mind Model of Relevance

Testing on previously learned materials can enhance the learning of new materials studied after the test (the forward effect of testing). We examined how the timing of interim testing influenced subsequent learning. Participants studied trivia facts in two learning sections (A & B) and provided a mental effort rating for each section. Between the two sections, participants either restudied, took an immediate test, or a 15-min delayed test on Section A. They then proceeded to Section B and took a final test on both sections. The results demonstrated that both the immediate and delayed testing groups outperformed the restudy group on Section B, indicating that the forward effect of testing was independent of the test timing. Furthermore, the testing groups showed a bigger increase in their mental effort from Section A to B than the restudy group, suggesting that the forward testing effect occurred probably due to more mental effort.

Does delayed interim testing enhance subsequent learning?

Computational cognitive theories of Autism Spectrum Disorder have received renewed attention in recent years. Consistent with the predictive processing framework, ASD has
been re-conceptualized as a disorder of aberrant prediction and learning-rate estimation involving multiple levels of a putative cognitive computational hierarchy. Specifically, behavioral symptoms of individuals with ASD might manifest due to an aberrant overestimation of the volatility of environmental contingencies (i.e. tendency of change in cue-outcome probabilities) which in turn might induce a dysfunctional setting of learning rates. In this work, we attempted to conceptually replicate computational modeling analyses of an impactful study of the recent ASD modeling literature in an independent
sample of subjects. We were not able to replicate some prior reported effects likely due to differences in model architecture and cognitive task setup. We found statistical trends in similar directions.

Premium content

Downloads

Next from CogSci 2022

Human-like property induction is a challenge for large language models

Similar lecture

Reliability Check: An Analysis of GPT-3's Response to Sensitive Topics and Prompt Wording