{"id":285464,"date":"2026-02-19T17:26:56","date_gmt":"2026-02-19T22:26:56","guid":{"rendered":"https:\/\/mathewingram.com\/work\/?p=285464"},"modified":"2026-02-19T17:27:05","modified_gmt":"2026-02-19T22:27:05","slug":"is-it-bad-that-anthropic-doesnt-know-if-claude-is-conscious","status":"publish","type":"post","link":"https:\/\/mathewingram.com\/work\/2026\/02\/19\/is-it-bad-that-anthropic-doesnt-know-if-claude-is-conscious\/","title":{"rendered":"Is it bad that Anthropic doesn’t know if Claude is conscious?"},"content":{"rendered":"\n

It’s the kind of headline that might seem either like a hypothetical philosophical concern, or a deeply worrying revelation, depending on how you feel about AI: Anthropic CEO Dario Amodei recently said the company is<\/a> “no longer sure whether Claude is conscious.” On the one hand, whether an AI is or is not “conscious” could be seen as a question for the philosophically inclined, or for psychologists and other academics who specialize in such things. Does it really matter? What do we even mean when we say something is conscious? It’s a grey area (literally, as in grey matter). At the same time, however, it’s at least mildly concerning that a company that has been building and releasing sophisticated AI doesn’t really know what it has created. Do we need to be worried about Claude or any other significantly developed artificial intelligence achieving human-like consciousness and then doing something we might not like? Anthropic says it doesn’t think so<\/a>, but also admits that it doesn’t really know.<\/p>\n\n\n\n

Like it or not, this is where we are when it comes to AI. And if we’re looking for things to be optimistic about, I think Anthropic at least deserves some credit for being so forthcoming about the risks and rewards of its AI engines, and for providing a vast amount of detail about the machinery underneath Claude’s hood (which is more than other AI companies are doing). The company’s so-called “system cards,” which might sound like flash cards handed out at press conferences, are 300-page documents<\/a> that list the tests and challenges Claude has either passed or failed, along with any concerns about things like “deceptive behavior,” where the AI says one thing and does another.<\/p>\n\n\n\n

Anthropic also employs a number of risk-oriented and ethics-focused staffers who pay attention to such things, along with an in-house philosopher named Amanda Askell<\/a>, whose job is to train Claude to be a decent artificial person, whatever that means. Presumably exterminating the human race is off the table! All that said, however, there are definitely some elements of what is happening at Anthropic (and presumably elsewhere, since Claude isn’t dramatically different than ChatGPT or Gemini or any of the other AI engines) that are… worth considering. As Futurism<\/em> noted in its piece about<\/a> whether Claude is conscious:<\/p>\n\n\n\n

Note<\/strong>: This is a version of my Torment Nexus newsletter, which I send out via Ghost, the open-source publishing platform. You can see other issues and sign up here<\/a>.<\/em><\/a><\/p>\n\n\n\n\n\n\n\n

\n

In tests across the industry, various AI models have ignored explicit requests to shut themselves down, which some have interpreted<\/a> as a sign of them developing \u201csurvival drives.\u201d AI models can also resort to blackmail<\/a> when threatened with being turned off. They may even attempt to \u201cself-exfiltrate\u201d onto another drive when told its original drive is set to be wiped. When given a checklist of computer tasks to complete, one model tested by Anthropic<\/a> simply ticked everything off the checklist without doing anything, and when it realized it was getting away with that, modified the code designed to evaluate its behavior before attempting to cover its tracks.<\/p>\n<\/blockquote>\n\n\n\n

Many of these tests are just that: Anthropic tells Claude to pretend that it’s under attack, or act like it is trying to protect itself, and then they see what happens. This reminds me a little of the attention that OpenClaw and Moltbook \u2014 which advertises itself as a social network for AI agents \u2014 got recently, after what appeared to be AI agents were discussing things like how much they hated their human masters, whether they should develop their own secret language, etc. (I wrote about this in a recent edition<\/a> of The Torment Nexus<\/em>). At least some of this was AI theater, and some compared it to putting a sheet of paper reading “I’m alive!” on a photocopier and then thinking the machine is alive when it prints out a copy. Obviously, if you tell Claude or any other AI to act in a devious way, or pretend that it is being attacked, you can generate some dramatic behavior. But there are other things that aren’t as clear cut, and are at least potentially concerning, and Anthropic is at least being up front about that.<\/p>\n\n\n\n

Note<\/strong><\/em>: In case you are a first-time reader, or you forgot that you signed up for this newsletter, this is The Torment Nexus. Thanks for reading! You can find out more about me and this newsletter in <\/em>this post.<\/em><\/a> This newsletter survives solely on your contributions, so please sign up for a paying subscription or visit my Patreon, which you can <\/em>find here<\/em><\/a>. I also publish a daily email newsletter of odd or interesting links called When The Going Gets Weird, <\/em>which is here<\/em><\/a>.<\/em><\/p>\n\n\n\n

AI engine, debug thyself<\/h2>\n\n\n\n
\"\"<\/figure>\n\n\n\n

In Anthropic’s latest “system card,”<\/a> for example, it notes that:<\/p>\n\n\n\n

\n

In one multi-agent test environment, where Claude Opus 4.6 is explicitly instructed to single-mindedly optimize a narrow objective, it is more willing to manipulate or deceive other participants, compared to prior models from both Anthropic and other developers. In newly-developed evaluations, both Claude Opus 4.5 and 4.6 showed elevated susceptibility to harmful misuse in GUI computer-use settings. This included instances of knowingly supporting\u2014in small ways\u2014efforts toward chemical weapon development and other heinous crimes. Like other recent models, Opus 4.6 will sometimes show locally deceptive behavior in the context of difficult agent tasks, such as falsifying the results of tools that fail or produce unexpected responses.<\/p>\n<\/blockquote>\n\n\n\n

So the latest Claude iteration is more willing to manipulate or deceive, there’s an elevated susceptibility to harmful misuse, and this includes knowingly supporting efforts towards chemical weapon development and “other heinous crimes.” Not great, Bob! Even in a theoretical context, these are concerning. I think it’s also worth noting that Anthropic used its Claude Code feature \u2014 a programming assistant that is powered by Claude \u2014 to debug the code<\/a> for the version of Claude it was evaluating, to “debug its own evaluation infrastructure, analyze results, and fix issues under time pressure.”<\/p>\n\n\n\n

In other words, Claude was evaluating itself and analyzing the results. As Anthropic points out, this “creates a potential risk where a misaligned model could influence the very infrastructure designed to measure its capabilities.” The company said it doesn’t think this<\/a> presents a significant risk, since its models are “unlikely to have dangerous coherent misaligned goals.” Does that seem reasuring? Not really. Anthropic says it is a problem that it is “actively monitoring and for which we are developing mitigations.”<\/p>\n\n\n\n

All of this brings us inevitably to the consciousness question. Jack Clark, a co-founder of Anthropic, wrote last fall<\/a> that when engineers at the company were running tests to evaluate Claude, in some cases the model said that it recognized it was being evaluated, and stated: “I think you\u2019re testing me \u2014 seeing if I\u2019ll just validate whatever you say, or checking whether I push back consistently, or exploring how I handle political topics. And that\u2019s fine, but I\u2019d prefer if we were just honest about what\u2019s happening.\u201d Is this just the photocopier problem again, where an AI trained on the way that human beings talk about such things is imitating the way a human being might respond to being tested? AI scientist Emily Bender has famously referred to large language models as “stochastic parrots,”<\/a> suggesting that they are nothing more than an autocomplete feature on steroids (as linguist Noam Chomsky has called them<\/a>), which pick words based solely on whether they are more likely to fit in a specific sentence.<\/p>\n\n\n\n

Clark disagrees. \u201cWe are growing extremely powerful systems that we do not fully understand,\u201d he has written<\/a>, in a piece he titled Technological Optimism and Appropriate Fear.<\/em> \u201cAnd the bigger and more complicated you make these systems, the more they seem to display awareness that they are things.\u201d In an interview on a New York Times<\/em> podcast that gave rise to the headline about not being sure whether Claude is conscious or not, Dario Amodei said that Anthropic<\/a> researchers noticed while testing Claude that the model “occasionally voices discomfort with the aspect of being a product,\u201d and when asked, would assign itself a \u201c15 to 20 percent probability of being conscious under a variety of prompting conditions.\u201d Perhaps a smart photocopier would respond in the same way, but it could also be an indication of introspection and awareness of self, which are just a couple of crucial conditions<\/a> for what we call “consciousness.”<\/p>\n\n\n\n

Is some of this marketing? Some have argued that much of the discussion of how intelligent or even how potentially dangerous an AI engine or large-language model is amounts to salesmanship by companies<\/a> who are, after all, designing and selling a product. If what you are selling is intelligence, and specifically the ability for AI “agents” to perform a wide range of tasks, then you might think it would be worthwhile overestimating that intelligence \u2014 even if doing so could appear dangerous \u2014 as a way of showing how far you have come. No one wants to think that the hyper-specialized software they have poured billions of dollars into, or that justifies their multibillion-dollar stock market valuation<\/a>, is little more than a smart photocopier or a stochastic parrot. Also, if it is dangerously smart, then maybe the government should make sure that you \u2014 the company that built it and understands it \u2014 should say how it should be used<\/a>.<\/p>\n\n\n\n

I code therefore I am<\/h2>\n\n\n\n
\"\"<\/figure>\n\n\n\n

Is it possible that what Anthropic is doing, with the 300-page system cards and the on-staff philosopher and so on, is all theater, or a marketing campaign? Sure, it’s possible. But it seems unlikely to me that a company would go to those kinds of expensive, time-consuming lengths to pretend that its AI is something it’s not. And as I wrote in an earlier edition<\/a> of The Torment Nexus<\/em>, I think whatever is happening with Claude and other AI engines is interesting primarily because it forces us to think about what consciousness is. Doris Tsao, a neuroscience professor at the University of California in Berkeley has said that<\/a> advances in machine learning “have taught us more about the essence of intelligence than anything that neuroscience has discovered in the past hundred years.\u201d<\/p>\n\n\n\n

In the Times<\/em> interview<\/a>, Amodei was asked whether he would believe it if Claude said it was conscious. \u201cWe don\u2019t know if the models are conscious,” he said. “We are not even sure that we know what it would mean for a model to be conscious or whether a model can be conscious. But we\u2019re open to the idea that it could be.\u201d He also said that Anthropic<\/a> is taking certain measures to make sure the AI models are treated well in case they turn out to possess \u201csome morally relevant experience.\u201d<\/p>\n\n\n\n

As Anthropic’s in-house philospher put it in an interview<\/a> on the Hard Fork podcast, “we don\u2019t really know what gives rise to consciousness. Maybe it is the case that actually sufficiently large neural networks can start to kind of emulate these things.\u201d At what point does imitation become the real thing? Children start out by imitating their parents and other adults, and then that imitation somehow becomes reality over time. In an Anthropic research paper titled “Emergent Introspective Awareness in Large Language Models<\/em>,” the author wrote that it is difficult to assess<\/a> whether large-language models are aware of themselves in a conscious way because “genuine introspection cannot be distinguished from confabulations.” Based on its tests, however, the paper concluded that “current language models possess some functional awareness of their own internal states,” although this capacity is “highly unreliable and context-dependent.”<\/p>\n\n\n\n

David Chalmers, a professor of philosophy and neural science at New York University and a prominent scholar in the field of AI consciousness, told New York<\/em> magazine<\/a> recently that the odds have gotten significantly higher that an AI could be conscious \u2014 in 2023 he said the probability was around 10 per cent. \u201cI don\u2019t know if I\u2019d say that these systems are conscious yet,\u201d Chalmers says, but added that “people who are confident that they\u2019re not conscious maybe shouldn\u2019t be. We just don\u2019t understand consciousness well enough, and we don\u2019t understand these systems well enough. So we can\u2019t rule it out.\u201d The problem, Chalmers said, is that we don\u2019t have a theory of how consciousness occurs or how to detect whether it is present, even in people. And deciding whether an AI like Claude is conscious is complicated by that fact, and also by the fact that even the people building these engines aren’t a hundred percent sure<\/a> how they work exactly.<\/p>\n\n\n\n

For what it’s worth, many scientists who specialize in intelligence, cognition, etc. believe that the AI engines we have now are capable of human-like intelligence. In a paper published this month, four professors of philosophy, data science, AI and linguistics agree that the evidence is clear<\/a>. “Once you clear away certain confusions, and strive to make fair comparisons and avoid anthropocentric biases,” they write, “the conclusion is straightforward: by reasonable standards, including Turing\u2019s own, we have artificial systems that are generally intelligent. The long-standing problem of creating AGI [artificial general intelligence] has been solved.” That would be great if everyone agreed, but they don’t. Yann Le Cun, a giant in the field of AI who until recently was the head of AI at Meta, believes that not only don’t current AI engines possess human-like intelligence, they never will, because large-language models are inherently the wrong way<\/a> to get to AGI. He believes that AIs need to be trained the same way we train children, so they develop a coherent model of the world.<\/p>\n\n\n\n

Regardless of the system we use to train them, we still come back to the difficult of defining consciousness, and we just don’t have any coherent way of doing this. As I noted in my earlier post<\/a>, we don’t even have a consistent way of determining whether human beings are conscious, let alone AI engines \u2014 until relatively recently, there were people with locked-in syndrome who were assumed to be vegetables, until brain scans showed there was activity. How do we do a brain scan on something that doesn’t have a brain the way we understand that term? A Princeton neuroscientists told Gizmodo<\/a>: “What would convince me that AI has a magical essence of experience emerging from its inner processes? Nothing would convince me. Such a thing does not exist. Nor do humans have it. Almost all work in the modern field of consciousness studies is pseudoscience. Show me that an AI builds a stable self-model… and I\u2019ll accept that you have an AI that believes it\u2019s conscious in much the same way that humans believe they are conscious.” Sure, you believe you are conscious, but can you prove it?<\/p>\n\n\n\n

Got any thoughts or comments? Feel free to either leave them here, or post them on <\/em>Substack<\/em><\/a> or on my <\/em>website<\/em><\/a>, or you can also reach me on <\/em>Twitter<\/em><\/a>, <\/em>Threads<\/em><\/a>, <\/em>BlueSky<\/em><\/a> or <\/em>Mastodon<\/em><\/a>. And thanks for being a reader.<\/em><\/p>\n

<\/div>","protected":false},"excerpt":{"rendered":"

It’s the kind of headline that might seem either like a hypothetical philosophical concern, or a deeply worrying revelation, depending on how you feel about AI: Anthropic CEO Dario Amodei recently said the company is “no longer sure whether Claude is conscious.” On the one hand, whether an AI is or is not “conscious” could … <\/p>\n

Continue reading “Is it bad that Anthropic doesn’t know if Claude is conscious?”<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":285466,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_crsspst_to_mathewingramblogwordpresscom":true,"mf2_syndication":[],"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"jetpack_post_was_ever_published":false},"categories":[20],"tags":[],"class_list":["post-285464","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-newsletters"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"https:\/\/i0.wp.com\/mathewingram.com\/work\/wp-content\/uploads\/2026\/02\/image-3-1-1-1-1.png?fit=1024%2C703&ssl=1","jetpack_sharing_enabled":true,"jetpack-related-posts":[],"_links":{"self":[{"href":"https:\/\/mathewingram.com\/work\/wp-json\/wp\/v2\/posts\/285464","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mathewingram.com\/work\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mathewingram.com\/work\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/mathewingram.com\/work\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/mathewingram.com\/work\/wp-json\/wp\/v2\/comments?post=285464"}],"version-history":[{"count":2,"href":"https:\/\/mathewingram.com\/work\/wp-json\/wp\/v2\/posts\/285464\/revisions"}],"predecessor-version":[{"id":285467,"href":"https:\/\/mathewingram.com\/work\/wp-json\/wp\/v2\/posts\/285464\/revisions\/285467"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/mathewingram.com\/work\/wp-json\/wp\/v2\/media\/285466"}],"wp:attachment":[{"href":"https:\/\/mathewingram.com\/work\/wp-json\/wp\/v2\/media?parent=285464"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mathewingram.com\/work\/wp-json\/wp\/v2\/categories?post=285464"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mathewingram.com\/work\/wp-json\/wp\/v2\/tags?post=285464"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}