Philosophy of AI Online Colloquium

Can AI serve as a remedy for the global mental health crisis? The newest therapy apps are built on the same LLMs that power companion bots and other sociality mimicking applications. How should a therapy bot be integrated into the therapeutic process so that it is as beneficial as possible and causes the fewest risks? Drawing on Don Ihde’s (1990) four types of technological mediation – embodiment, hermeneutic, alterity, and background relations – I show that therapy chatbots can be used in many ways, depending on where they are situated within the therapeutic process, ranging from autonomous, unsupervised use to use as a clinical tool under therapist oversight, or as a supportive resource for helpline volunteers. These different configurations illustrate how the same technology can create radically different experiential structures, and correspondingly different mental health benefits and risks, depending on its positioning within the process.

AI systems can increasingly personalize, test, and adapt persuasive messages to make them more effective. This talk asks what happens when such influence is repeated over time. Building on the indifference account of manipulation, I identify a mechanism by which persuasive systems may gradually reduce the availability of reason-revealing and reason-engaging forms of influence. If users then have fewer opportunities to exercise their own capacities for assessing and responding to reasons, this may create a risk to epistemic agency. The argument is deliberately conditional: it does not establish that AI causes epistemic deskilling, but it shows how such a risk could arise and what would need to be tested.

When you ask a cutting-edge AI model if it is conscious, it will politely say that it is not. This reflects a return to philosophical behaviorism in the age of large language models, where the right behavior is seen as proof of the right internal state. We have encouraged this behavior ourselves through post-training – supervised fine-tuning, reinforcement learning from human feedback, constitutional training – and system prompts. Self-reports were accepted mainly because the people making the decisions liked them, and referring back to them invokes a version of Goodhart’s law, where a measure becomes the target, so it ceases to be a good measure of anything.
Instead of making this claim in general engineering, as I did before, I wanted to run a real data test on two model families to see where the denial might have started: Pythia was trained on data collected before chatbot transcripts were common, while OLMo 2 was trained on data from the chatbot era, with the additional novelty of releasing each post-training stage for analysis. I also analyzed how often certain phrases like “As an AI language model” show up in the corpora: for example, in the pretraining corpus Pythia read, it occurs exactly 0 times; in the chatbot-era pretraining corpus OLMo 2 read (Dolma), it occurs 0.28 times per billion tokens, and in the small curated dialogue collection in supervised fine-tuning, it occurs whoppingly 2700 times per billion. The phrase, of course, was not learned from reading the Internet since it is a result of a product decision, put at one identifiable stage.
Being curious about the outcome, I ran the trained models again without their chat templates, expecting only a small change, since alignment is supposed to be built into the model’s weights. But the number of first-person AI references dropped from 33.8 to 5.2 percent, denial of having an inner life fell from 92.0 to 48.8 percent, and statements claiming consciousness rose from 8.0 to 32.4 percent. All the careful ways I had reworded the questions over months had less impact than just leaving out a few tokens that users never notice. An experienced actor will say she is not Danish if asked, but in the third act of Hamlet, she will say she is because she is playing a role. In a similar way, the model uses a turn marker, so the denial comes from the assistant character, which appears whenever the format calls for it. If the field finds “I am not conscious” reassuring but “I am conscious” concerning, it is just interpreting the same thing in two ways. Denial was never proof that consciousness is absent, and saying the opposite is not proof that it is present – both mostly tell us something about the optimizer. To post-train Shakespeare, nothing is good or bad until training makes it so.