As regular readers will know, I’ve written a lot about the topic of AI and consciousness (too much for some perhaps!) because I find it fascinating, in much the same way that I find the issue of whether AI is dangerous or not fascinating – something I’ve also written about a number of times. And the main reason both of these topics are so interesting is that even the so-called experts, the people who built the fundamental underpinnings of these technologies, can’t seem to agree. On the subject of AI danger, for example, Geoffrey Hinton – the University of Toronto professor who was one of the main architects of neural networks – says that we are in deep trouble. His former colleague Yoshua Bengio agrees. But Yann LeCun – the former head of AI at Meta, who also worked on these technologies – says that this is ridiculous, and that current AIs are no more intelligent than the average cat. Timnit Gebru, a pioneering AI scientist formerly with Google (as Hinton was at one time) says they are just “stochastic parrots.“
On the consciousness question, discussions are inevitably filled with categorical statements. Those who think AI couldn’t possibly be conscious are convinced that the people who think it can be (or possibly already is) are idiots who are subject to “chatbot psychosis” or “AI derangement syndrome.” Others are convinced that there’s plenty of evidence that AIs are conscious – as Nobel Prize-winning biologist Richard Dawkins declared in a recent essay. It’s difficult to say when this debate began, but I think a good starting point is the essay from former Google ethicist Blake Lemoine in 2022, who argued that Google’s AI was either conscious or so close that it didn’t matter (he was ridiculed and then fired). To be fair, the anti-AI-consciousness side seems a lot more categorical than the pro – Anthropic cofounders Dario Amodei and Jack Clark haven’t said whether they think Claude is conscious, but they have left the door open to it (which seems to infuriate the anti-consciousness side as much as if they said it was).
Among the many categorical statements about AI consciousness, one of the most recent and most noteworthy – at least in terms of the amount of coverage it got – is the recent piece in The Atlantic from science-fiction author Ted Chiang (he wrote a story that became the movie Arrival). Chiang’s point is obvious from the title: “No, Artificial Intelligence Is Not Conscious.” Pretty definitive, Ted! To even discuss the question of AI consciousness is “absurd,” he says. And what is this based on? Is it his background in machine intelligence or the philosophy of consciousness, or perhaps his training in biological systems? It is not. From what I can tell, his conclusions seem to be based on what the kids like to call “vibes.” Should we seriously consider the possibility that Claude, or any large language model, might be conscious and capable of receiving moral instruction, Chiang asks? “No. Absolutely not,” he replies. He continues:
If we give an LLM a prompt that reads “The following is a conversation between Julius Caesar and Genghis Khan,” it will generate a coherent dialogue between the two historical figures. But no matter how detailed the responses are, no matter how vividly they recount their respective historical accomplishments, we would never conclude that the LLM has conjured up digital re-creations of Julius Caesar and Genghis Khan, nor would we suggest that the historical figures are conscious despite being disembodied and are happily conversing in a language that neither actually spoke. In reality, they are just characters in a piece of speculative fiction.
Note: This is a version of my Torment Nexus newsletter, which I send out via Ghost, the open-source publishing platform. You can see other issues and sign up here.
Continue reading “Is Atlantic writer Ted Chiang conscious? How do we know?”































